2024-09-10T18:27:35.907 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T18:27:35.923 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T18:27:36.005 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900244 branch: wip-guits-quincy-2024-09-09-1810-quincy description: orch:cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} email: gabrioux@ibm.com first_in_suite: false job_id: '7900244' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-quincy-2024-09-09-1810-quincy ceph: conf: client.0: debug ms: 1 debug objecter: 20 debug rados: 20 global: ms bind msgr1: false ms bind msgr2: true ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 10 osd: debug ms: 1 debug osd: 20 osd debug reject backfill probability: 0.3 osd delete sleep: 1 osd max backfills: 3 osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd snap trim sleep: 2 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - but it is still running - objects unfound and apparently lost - \(POOL_APP_NOT_ENABLED\) sha1: 5be57395ea235535910512ae73ba8e7ff25fe588 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: root install: ceph: flavor: default sha1: 5be57395ea235535910512ae73ba8e7ff25fe588 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: quincy-orch-centos-9-teuth sha1: fad5356a94e6044ce1062d748daa5a0f2ed53a7d owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 4625 sha1: 5be57395ea235535910512ae73ba8e7ff25fe588 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: quincy-orch-centos-9-teuth suite_path: /home/teuthworker/src/github.com_adk3798_ceph_fad5356a94e6044ce1062d748daa5a0f2ed53a7d/qa suite_relpath: qa suite_repo: https://github.com/adk3798/ceph.git suite_sha1: fad5356a94e6044ce1062d748daa5a0f2ed53a7d targets: smithi018.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHlP3dGFAfuqmR3RQjpO83VH//yXPNJwceXvRxrJEns8HwXLzojGXsqv0nWUavGAOoyXExeIJg+3EAyF9xgdkh0= smithi103.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCgaj5+pSkWZzeXrIa1GlZN8t8UD66s3+PG9rMyXSjFE/wQvwcH5gf/3E8xiu7Q4/hAuzuw+hW3KRedjjaWrmY4= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install nvmetcli nvme-cli -y - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - thrashosds: chance_pgnum_grow: 1 chance_pgnum_shrink: 1 chance_pgpnum_fix: 1 timeout: 1200 - full_sequential: - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_17:53:37 tube: smithi user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T18:27:36.005 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_adk3798_ceph_fad5356a94e6044ce1062d748daa5a0f2ed53a7d/qa; will attempt to use it 2024-09-10T18:27:36.006 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_adk3798_ceph_fad5356a94e6044ce1062d748daa5a0f2ed53a7d/qa/tasks 2024-09-10T18:27:36.006 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T18:27:36.008 INFO:teuthology.task.internal:Checking packages... 2024-09-10T18:27:36.029 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '5be57395ea235535910512ae73ba8e7ff25fe588' 2024-09-10T18:27:36.029 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T18:27:36.029 INFO:teuthology.packaging:ref: None 2024-09-10T18:27:36.029 INFO:teuthology.packaging:tag: None 2024-09-10T18:27:36.029 INFO:teuthology.packaging:branch: wip-guits-quincy-2024-09-09-1810-quincy 2024-09-10T18:27:36.029 INFO:teuthology.packaging:sha1: 5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:27:36.029 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-quincy-2024-09-09-1810-quincy 2024-09-10T18:27:36.167 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1606.g5be57395 2024-09-10T18:27:36.169 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T18:27:36.175 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T18:27:36.175 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T18:27:36.261 INFO:teuthology.task.internal:Saving configuration 2024-09-10T18:27:36.274 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T18:27:36.297 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T18:27:36.322 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi018.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900244', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 18:22:23.361636', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHlP3dGFAfuqmR3RQjpO83VH//yXPNJwceXvRxrJEns8HwXLzojGXsqv0nWUavGAOoyXExeIJg+3EAyF9xgdkh0='} 2024-09-10T18:27:36.346 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi103.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900244', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 18:22:23.360366', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCgaj5+pSkWZzeXrIa1GlZN8t8UD66s3+PG9rMyXSjFE/wQvwcH5gf/3E8xiu7Q4/hAuzuw+hW3KRedjjaWrmY4='} 2024-09-10T18:27:36.346 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T18:27:36.430 INFO:teuthology.task.internal:roles: ubuntu@smithi018.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-09-10T18:27:36.430 INFO:teuthology.task.internal:roles: ubuntu@smithi103.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-09-10T18:27:36.430 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T18:27:36.553 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9c3fe93ac0>, signals=[15]) 2024-09-10T18:27:36.553 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T18:27:36.618 INFO:teuthology.task.internal:Opening connections... 2024-09-10T18:27:36.618 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi018.front.sepia.ceph.com 2024-09-10T18:27:36.620 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:27:36.696 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi103.front.sepia.ceph.com 2024-09-10T18:27:36.697 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:27:36.770 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T18:27:36.775 DEBUG:teuthology.orchestra.run.smithi018:> uname -m 2024-09-10T18:27:36.793 INFO:teuthology.orchestra.run.smithi018.stdout:x86_64 2024-09-10T18:27:36.793 DEBUG:teuthology.orchestra.run.smithi018:> cat /etc/os-release 2024-09-10T18:27:36.850 INFO:teuthology.orchestra.run.smithi018.stdout:NAME="CentOS Stream" 2024-09-10T18:27:36.850 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION="9" 2024-09-10T18:27:36.850 INFO:teuthology.orchestra.run.smithi018.stdout:ID="centos" 2024-09-10T18:27:36.850 INFO:teuthology.orchestra.run.smithi018.stdout:ID_LIKE="rhel fedora" 2024-09-10T18:27:36.850 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION_ID="9" 2024-09-10T18:27:36.850 INFO:teuthology.orchestra.run.smithi018.stdout:PLATFORM_ID="platform:el9" 2024-09-10T18:27:36.850 INFO:teuthology.orchestra.run.smithi018.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T18:27:36.850 INFO:teuthology.orchestra.run.smithi018.stdout:ANSI_COLOR="0;31" 2024-09-10T18:27:36.850 INFO:teuthology.orchestra.run.smithi018.stdout:LOGO="fedora-logo-icon" 2024-09-10T18:27:36.850 INFO:teuthology.orchestra.run.smithi018.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T18:27:36.851 INFO:teuthology.orchestra.run.smithi018.stdout:HOME_URL="https://centos.org/" 2024-09-10T18:27:36.851 INFO:teuthology.orchestra.run.smithi018.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T18:27:36.851 INFO:teuthology.orchestra.run.smithi018.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T18:27:36.851 INFO:teuthology.orchestra.run.smithi018.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T18:27:36.851 INFO:teuthology.lock.ops:Updating smithi018.front.sepia.ceph.com on lock server 2024-09-10T18:27:36.874 DEBUG:teuthology.orchestra.run.smithi103:> uname -m 2024-09-10T18:27:36.894 INFO:teuthology.orchestra.run.smithi103.stdout:x86_64 2024-09-10T18:27:36.894 DEBUG:teuthology.orchestra.run.smithi103:> cat /etc/os-release 2024-09-10T18:27:36.950 INFO:teuthology.orchestra.run.smithi103.stdout:NAME="CentOS Stream" 2024-09-10T18:27:36.950 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION="9" 2024-09-10T18:27:36.950 INFO:teuthology.orchestra.run.smithi103.stdout:ID="centos" 2024-09-10T18:27:36.950 INFO:teuthology.orchestra.run.smithi103.stdout:ID_LIKE="rhel fedora" 2024-09-10T18:27:36.951 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION_ID="9" 2024-09-10T18:27:36.951 INFO:teuthology.orchestra.run.smithi103.stdout:PLATFORM_ID="platform:el9" 2024-09-10T18:27:36.951 INFO:teuthology.orchestra.run.smithi103.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T18:27:36.951 INFO:teuthology.orchestra.run.smithi103.stdout:ANSI_COLOR="0;31" 2024-09-10T18:27:36.951 INFO:teuthology.orchestra.run.smithi103.stdout:LOGO="fedora-logo-icon" 2024-09-10T18:27:36.951 INFO:teuthology.orchestra.run.smithi103.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T18:27:36.951 INFO:teuthology.orchestra.run.smithi103.stdout:HOME_URL="https://centos.org/" 2024-09-10T18:27:36.951 INFO:teuthology.orchestra.run.smithi103.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T18:27:36.951 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T18:27:36.951 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T18:27:36.952 INFO:teuthology.lock.ops:Updating smithi103.front.sepia.ceph.com on lock server 2024-09-10T18:27:36.974 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T18:27:36.983 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T18:27:37.013 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T18:27:37.014 DEBUG:teuthology.orchestra.run.smithi018:> test '!' -e /home/ubuntu/cephtest 2024-09-10T18:27:37.016 DEBUG:teuthology.orchestra.run.smithi103:> test '!' -e /home/ubuntu/cephtest 2024-09-10T18:27:37.033 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T18:27:37.103 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T18:27:37.103 DEBUG:teuthology.orchestra.run.smithi018:> test -z $(ls -A /var/lib/ceph) 2024-09-10T18:27:37.107 DEBUG:teuthology.orchestra.run.smithi103:> test -z $(ls -A /var/lib/ceph) 2024-09-10T18:27:37.435 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T18:27:37.493 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T18:27:37.564 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-10T18:27:37.564 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-10T18:27:37.565 DEBUG:teuthology.orchestra.run.smithi018:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T18:27:37.565 DEBUG:teuthology.orchestra.run.smithi103:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T18:27:37.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:27:37.582 DEBUG:teuthology.orchestra.run.smithi018:> uname -r 2024-09-10T18:27:37.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:27:37.583 DEBUG:teuthology.orchestra.run.smithi103:> uname -r 2024-09-10T18:27:37.640 INFO:teuthology.orchestra.run.smithi018.stdout:5.14.0-503.el9.x86_64 2024-09-10T18:27:37.641 INFO:teuthology.task.kernel:Running kernel on smithi018: 5.14.0-503.el9.x86_64 2024-09-10T18:27:37.641 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum install -y kernel 2024-09-10T18:27:37.642 INFO:teuthology.orchestra.run.smithi103.stdout:5.14.0-503.el9.x86_64 2024-09-10T18:27:37.642 INFO:teuthology.task.kernel:Running kernel on smithi103: 5.14.0-503.el9.x86_64 2024-09-10T18:27:37.642 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum install -y kernel 2024-09-10T18:27:40.643 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - BaseOS 6.5 MB/s | 8.2 MB 00:01 2024-09-10T18:27:41.311 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.2 MB 00:01 2024-09-10T18:27:42.494 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - AppStream 30 MB/s | 20 MB 00:00 2024-09-10T18:27:44.971 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - AppStream 8.2 MB/s | 20 MB 00:02 2024-09-10T18:27:46.679 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.5 MB 00:00 2024-09-10T18:27:48.313 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 19 kB 00:00 2024-09-10T18:27:49.217 INFO:teuthology.orchestra.run.smithi103.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-10T18:27:49.863 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - CRB 6.7 MB/s | 6.5 MB 00:00 2024-09-10T18:27:51.688 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 19 kB 00:00 2024-09-10T18:27:52.584 INFO:teuthology.orchestra.run.smithi018.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-10T18:27:54.631 INFO:teuthology.orchestra.run.smithi103.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-10T18:27:56.675 INFO:teuthology.orchestra.run.smithi103.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T18:27:56.675 INFO:teuthology.orchestra.run.smithi103.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T18:27:56.762 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-10T18:27:56.774 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:27:56.774 INFO:teuthology.orchestra.run.smithi103.stdout: Package Architecture Version Repository Size 2024-09-10T18:27:56.775 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:27:56.775 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-10T18:27:56.775 INFO:teuthology.orchestra.run.smithi103.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T18:27:56.775 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2024-09-10T18:27:56.775 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T18:27:56.775 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T18:27:56.775 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T18:27:56.775 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:27:56.775 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-10T18:27:56.775 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:27:56.775 INFO:teuthology.orchestra.run.smithi103.stdout:Install 4 Packages 2024-09-10T18:27:56.775 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:27:56.776 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 79 M 2024-09-10T18:27:56.776 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 126 M 2024-09-10T18:27:56.776 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-10T18:27:56.883 INFO:teuthology.orchestra.run.smithi103.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 262 kB/s | 23 kB 00:00 2024-09-10T18:27:57.258 INFO:teuthology.orchestra.run.smithi103.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-09-10T18:27:57.675 INFO:teuthology.orchestra.run.smithi103.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 39 MB/s | 35 MB 00:00 2024-09-10T18:27:57.992 INFO:teuthology.orchestra.run.smithi103.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 26 MB/s | 29 MB 00:01 2024-09-10T18:27:57.992 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:27:57.992 INFO:teuthology.orchestra.run.smithi103.stdout:Total 65 MB/s | 79 MB 00:01 2024-09-10T18:27:58.298 INFO:teuthology.orchestra.run.smithi018.stdout:lab-extras 44 kB/s | 1.7 kB 00:00 2024-09-10T18:27:58.680 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-10T18:27:59.075 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-10T18:27:59.075 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-10T18:28:00.034 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-10T18:28:00.034 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-10T18:28:00.421 INFO:teuthology.orchestra.run.smithi018.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T18:28:00.422 INFO:teuthology.orchestra.run.smithi018.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T18:28:00.516 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-09-10T18:28:00.521 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:28:00.521 INFO:teuthology.orchestra.run.smithi018.stdout: Package Architecture Version Repository Size 2024-09-10T18:28:00.521 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:28:00.521 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-09-10T18:28:00.521 INFO:teuthology.orchestra.run.smithi018.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T18:28:00.521 INFO:teuthology.orchestra.run.smithi018.stdout:Installing dependencies: 2024-09-10T18:28:00.521 INFO:teuthology.orchestra.run.smithi018.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T18:28:00.521 INFO:teuthology.orchestra.run.smithi018.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T18:28:00.521 INFO:teuthology.orchestra.run.smithi018.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T18:28:00.521 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:28:00.521 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-09-10T18:28:00.522 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:28:00.522 INFO:teuthology.orchestra.run.smithi018.stdout:Install 4 Packages 2024-09-10T18:28:00.522 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:28:00.522 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 79 M 2024-09-10T18:28:00.522 INFO:teuthology.orchestra.run.smithi018.stdout:Installed size: 126 M 2024-09-10T18:28:00.522 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-09-10T18:28:00.624 INFO:teuthology.orchestra.run.smithi018.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 281 kB/s | 23 kB 00:00 2024-09-10T18:28:02.510 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-10T18:28:03.165 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T18:28:03.253 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:28:03.607 INFO:teuthology.orchestra.run.smithi018.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 5.1 MB/s | 16 MB 00:03 2024-09-10T18:28:06.072 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:28:06.165 INFO:teuthology.orchestra.run.smithi018.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 6.1 MB/s | 35 MB 00:05 2024-09-10T18:28:06.196 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:28:06.732 INFO:teuthology.orchestra.run.smithi018.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 4.7 MB/s | 29 MB 00:06 2024-09-10T18:28:06.732 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:28:06.733 INFO:teuthology.orchestra.run.smithi018.stdout:Total 13 MB/s | 79 MB 00:06 2024-09-10T18:28:07.438 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-09-10T18:28:07.807 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-09-10T18:28:07.807 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-09-10T18:28:08.810 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-09-10T18:28:08.811 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-09-10T18:28:11.071 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:28:11.172 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:28:11.383 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-09-10T18:28:12.077 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T18:28:12.166 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:28:14.969 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:28:15.095 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:28:16.045 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:28:20.231 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:28:20.279 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:28:25.422 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:28:54.290 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:28:54.334 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:28:54.728 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:28:54.728 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T18:28:54.728 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:28:54.728 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:28:55.622 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:28:55.623 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:28:55.623 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-10T18:28:55.623 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T18:28:55.623 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T18:28:55.623 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T18:28:55.623 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T18:28:55.623 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:28:55.623 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-10T18:28:55.947 DEBUG:teuthology.orchestra.run.smithi103:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T18:28:56.080 INFO:teuthology.orchestra.run.smithi103.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T18:28:56.080 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T18:28:56.080 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T18:28:56.081 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T18:28:56.081 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T18:28:56.081 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-10T18:28:56.081 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi103.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T18:28:56.081 DEBUG:teuthology.orchestra.run.smithi103:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T18:28:56.140 INFO:teuthology.orchestra.run.smithi103.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T18:28:56.140 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T18:28:56.140 DEBUG:teuthology.orchestra.run.smithi103:> sudo rpm -qi grub2-tools 2024-09-10T18:28:56.183 INFO:teuthology.orchestra.run.smithi103.stdout:Name : grub2-tools 2024-09-10T18:28:56.183 INFO:teuthology.orchestra.run.smithi103.stdout:Epoch : 1 2024-09-10T18:28:56.183 INFO:teuthology.orchestra.run.smithi103.stdout:Version : 2.06 2024-09-10T18:28:56.183 INFO:teuthology.orchestra.run.smithi103.stdout:Release : 82.el9 2024-09-10T18:28:56.183 INFO:teuthology.orchestra.run.smithi103.stdout:Architecture: x86_64 2024-09-10T18:28:56.184 INFO:teuthology.orchestra.run.smithi103.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T18:28:56.184 INFO:teuthology.orchestra.run.smithi103.stdout:Group : Unspecified 2024-09-10T18:28:56.184 INFO:teuthology.orchestra.run.smithi103.stdout:Size : 8274670 2024-09-10T18:28:56.184 INFO:teuthology.orchestra.run.smithi103.stdout:License : GPLv3+ 2024-09-10T18:28:56.184 INFO:teuthology.orchestra.run.smithi103.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T18:28:56.184 INFO:teuthology.orchestra.run.smithi103.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T18:28:56.184 INFO:teuthology.orchestra.run.smithi103.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T18:28:56.184 INFO:teuthology.orchestra.run.smithi103.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T18:28:56.184 INFO:teuthology.orchestra.run.smithi103.stdout:Packager : builder@centos.org 2024-09-10T18:28:56.184 INFO:teuthology.orchestra.run.smithi103.stdout:Vendor : CentOS 2024-09-10T18:28:56.184 INFO:teuthology.orchestra.run.smithi103.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T18:28:56.184 INFO:teuthology.orchestra.run.smithi103.stdout:Summary : Support tools for GRUB. 2024-09-10T18:28:56.184 INFO:teuthology.orchestra.run.smithi103.stdout:Description : 2024-09-10T18:28:56.185 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:28:56.185 INFO:teuthology.orchestra.run.smithi103.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T18:28:56.185 INFO:teuthology.orchestra.run.smithi103.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T18:28:56.185 INFO:teuthology.orchestra.run.smithi103.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T18:28:56.185 INFO:teuthology.orchestra.run.smithi103.stdout:hardware devices. 2024-09-10T18:28:56.185 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:28:56.185 INFO:teuthology.orchestra.run.smithi103.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T18:28:56.186 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T18:28:56.186 INFO:teuthology.task.kernel:Updating grub on smithi103 to boot 5.14.0-505.el9.x86_64 2024-09-10T18:28:56.186 DEBUG:teuthology.orchestra.run.smithi103:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T18:28:56.807 INFO:teuthology.orchestra.run.smithi103.stderr:Generating grub configuration file ... 2024-09-10T18:28:58.139 INFO:teuthology.orchestra.run.smithi103.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T18:28:58.165 INFO:teuthology.orchestra.run.smithi103.stderr:done 2024-09-10T18:28:58.172 DEBUG:teuthology.orchestra.run.smithi103:> mktemp 2024-09-10T18:28:58.188 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/tmp.aKbubOPo8i 2024-09-10T18:28:58.188 DEBUG:teuthology.orchestra.run.smithi103:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.aKbubOPo8i 2024-09-10T18:28:58.255 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 0666 /tmp/tmp.aKbubOPo8i 2024-09-10T18:28:58.387 DEBUG:teuthology.orchestra.remote:smithi103:/tmp/tmp.aKbubOPo8i is 6KB 2024-09-10T18:28:58.436 DEBUG:teuthology.orchestra.run.smithi103:> rm -fr /tmp/tmp.aKbubOPo8i 2024-09-10T18:28:58.451 DEBUG:teuthology.orchestra.run.smithi103:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T18:28:58.514 INFO:teuthology.orchestra.run.smithi103.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T18:28:58.514 INFO:teuthology.orchestra.run.smithi103.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T18:28:58.514 INFO:teuthology.orchestra.run.smithi103.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T18:28:58.514 INFO:teuthology.orchestra.run.smithi103.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T18:28:58.515 DEBUG:teuthology.orchestra.run.smithi103:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T18:28:58.665 DEBUG:teuthology.orchestra.run.smithi103:> sudo shutdown -r now 2024-09-10T18:29:04.878 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:29:04.919 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:29:05.298 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:29:05.298 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T18:29:05.298 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:29:05.298 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:29:06.161 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:29:06.162 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:29:06.162 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-09-10T18:29:06.162 INFO:teuthology.orchestra.run.smithi018.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T18:29:06.162 INFO:teuthology.orchestra.run.smithi018.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T18:29:06.162 INFO:teuthology.orchestra.run.smithi018.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T18:29:06.162 INFO:teuthology.orchestra.run.smithi018.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T18:29:06.162 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:29:06.162 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-09-10T18:29:06.518 DEBUG:teuthology.orchestra.run.smithi018:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T18:29:06.601 INFO:teuthology.orchestra.run.smithi018.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T18:29:06.601 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T18:29:06.601 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T18:29:06.601 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T18:29:06.602 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T18:29:06.602 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-10T18:29:06.602 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi018.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T18:29:06.602 DEBUG:teuthology.orchestra.run.smithi018:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T18:29:06.659 INFO:teuthology.orchestra.run.smithi018.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T18:29:06.660 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T18:29:06.661 DEBUG:teuthology.orchestra.run.smithi018:> sudo rpm -qi grub2-tools 2024-09-10T18:29:06.692 INFO:teuthology.orchestra.run.smithi018.stdout:Name : grub2-tools 2024-09-10T18:29:06.692 INFO:teuthology.orchestra.run.smithi018.stdout:Epoch : 1 2024-09-10T18:29:06.692 INFO:teuthology.orchestra.run.smithi018.stdout:Version : 2.06 2024-09-10T18:29:06.692 INFO:teuthology.orchestra.run.smithi018.stdout:Release : 82.el9 2024-09-10T18:29:06.692 INFO:teuthology.orchestra.run.smithi018.stdout:Architecture: x86_64 2024-09-10T18:29:06.692 INFO:teuthology.orchestra.run.smithi018.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T18:29:06.693 INFO:teuthology.orchestra.run.smithi018.stdout:Group : Unspecified 2024-09-10T18:29:06.693 INFO:teuthology.orchestra.run.smithi018.stdout:Size : 8274670 2024-09-10T18:29:06.693 INFO:teuthology.orchestra.run.smithi018.stdout:License : GPLv3+ 2024-09-10T18:29:06.693 INFO:teuthology.orchestra.run.smithi018.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T18:29:06.693 INFO:teuthology.orchestra.run.smithi018.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T18:29:06.693 INFO:teuthology.orchestra.run.smithi018.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T18:29:06.693 INFO:teuthology.orchestra.run.smithi018.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T18:29:06.693 INFO:teuthology.orchestra.run.smithi018.stdout:Packager : builder@centos.org 2024-09-10T18:29:06.693 INFO:teuthology.orchestra.run.smithi018.stdout:Vendor : CentOS 2024-09-10T18:29:06.693 INFO:teuthology.orchestra.run.smithi018.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T18:29:06.693 INFO:teuthology.orchestra.run.smithi018.stdout:Summary : Support tools for GRUB. 2024-09-10T18:29:06.693 INFO:teuthology.orchestra.run.smithi018.stdout:Description : 2024-09-10T18:29:06.694 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:29:06.694 INFO:teuthology.orchestra.run.smithi018.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T18:29:06.694 INFO:teuthology.orchestra.run.smithi018.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T18:29:06.694 INFO:teuthology.orchestra.run.smithi018.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T18:29:06.694 INFO:teuthology.orchestra.run.smithi018.stdout:hardware devices. 2024-09-10T18:29:06.694 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:29:06.694 INFO:teuthology.orchestra.run.smithi018.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T18:29:06.695 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T18:29:06.695 INFO:teuthology.task.kernel:Updating grub on smithi018 to boot 5.14.0-505.el9.x86_64 2024-09-10T18:29:06.695 DEBUG:teuthology.orchestra.run.smithi018:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T18:29:07.267 INFO:teuthology.orchestra.run.smithi018.stderr:Generating grub configuration file ... 2024-09-10T18:29:08.604 INFO:teuthology.orchestra.run.smithi018.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T18:29:08.632 INFO:teuthology.orchestra.run.smithi018.stderr:done 2024-09-10T18:29:08.634 DEBUG:teuthology.orchestra.run.smithi018:> mktemp 2024-09-10T18:29:08.649 INFO:teuthology.orchestra.run.smithi018.stdout:/tmp/tmp.ApZX5dtb52 2024-09-10T18:29:08.649 DEBUG:teuthology.orchestra.run.smithi018:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ApZX5dtb52 2024-09-10T18:29:08.723 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod 0666 /tmp/tmp.ApZX5dtb52 2024-09-10T18:29:08.871 DEBUG:teuthology.orchestra.remote:smithi018:/tmp/tmp.ApZX5dtb52 is 6KB 2024-09-10T18:29:08.919 DEBUG:teuthology.orchestra.run.smithi018:> rm -fr /tmp/tmp.ApZX5dtb52 2024-09-10T18:29:08.934 DEBUG:teuthology.orchestra.run.smithi018:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T18:29:08.999 INFO:teuthology.orchestra.run.smithi018.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T18:29:08.999 INFO:teuthology.orchestra.run.smithi018.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T18:29:09.000 INFO:teuthology.orchestra.run.smithi018.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T18:29:09.000 INFO:teuthology.orchestra.run.smithi018.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T18:29:09.001 DEBUG:teuthology.orchestra.run.smithi018:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T18:29:09.152 DEBUG:teuthology.orchestra.run.smithi018:> sudo shutdown -r now 2024-09-10T18:29:28.685 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-10T18:29:28.686 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-10T18:29:28.687 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:29:39.165 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-10T18:29:39.165 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi018.front.sepia.ceph.com' 2024-09-10T18:29:39.166 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:29:47.197 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.103 2024-09-10T18:29:56.207 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-10T18:29:56.207 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:29:57.697 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.18 2024-09-10T18:29:59.261 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.103 2024-09-10T18:30:06.705 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi018.front.sepia.ceph.com' 2024-09-10T18:30:06.706 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:30:09.761 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.18 2024-09-10T18:30:11.264 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-10T18:30:11.265 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:30:21.769 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi018.front.sepia.ceph.com' 2024-09-10T18:30:21.770 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:30:53.625 DEBUG:teuthology.orchestra.run.smithi018:> true 2024-09-10T18:30:53.994 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi018.front.sepia.ceph.com' 2024-09-10T18:30:53.995 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-10T18:30:53.995 DEBUG:teuthology.orchestra.run.smithi018:> uname -r 2024-09-10T18:30:54.010 INFO:teuthology.orchestra.run.smithi018.stdout:5.14.0-505.el9.x86_64 2024-09-10T18:30:54.010 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T18:30:54.010 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T18:30:54.010 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T18:30:55.011 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-10T18:30:55.011 DEBUG:teuthology.orchestra.run.smithi018:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T18:30:55.103 INFO:teuthology.orchestra.run.smithi018.stdout:ttyS1 2024-09-10T18:30:55.126 DEBUG:teuthology.parallel:result is None 2024-09-10T18:31:11.267 DEBUG:teuthology.orchestra.remote:timed out 2024-09-10T18:31:26.268 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-10T18:31:26.270 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:31:26.656 DEBUG:teuthology.orchestra.run.smithi103:> true 2024-09-10T18:31:26.997 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-10T18:31:26.997 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-10T18:31:26.998 DEBUG:teuthology.orchestra.run.smithi103:> uname -r 2024-09-10T18:31:27.017 INFO:teuthology.orchestra.run.smithi103.stdout:5.14.0-505.el9.x86_64 2024-09-10T18:31:27.017 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T18:31:27.017 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T18:31:27.018 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T18:31:28.019 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-10T18:31:28.019 DEBUG:teuthology.orchestra.run.smithi103:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T18:31:28.095 INFO:teuthology.orchestra.run.smithi103.stdout:ttyS1 2024-09-10T18:31:28.115 DEBUG:teuthology.parallel:result is None 2024-09-10T18:31:28.115 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T18:31:28.122 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T18:31:28.123 DEBUG:teuthology.orchestra.run.smithi018:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T18:31:28.125 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T18:31:28.143 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T18:31:28.233 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T18:31:28.342 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T18:31:28.342 DEBUG:teuthology.orchestra.run.smithi018:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T18:31:28.345 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T18:31:28.377 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T18:31:28.402 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T18:31:28.402 DEBUG:teuthology.orchestra.run.smithi018:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T18:31:28.410 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T18:31:28.442 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T18:31:28.443 INFO:teuthology.orchestra.run.smithi018.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T18:31:28.454 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T18:31:28.456 INFO:teuthology.orchestra.run.smithi018.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T18:31:28.458 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T18:31:28.580 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T18:31:28.580 DEBUG:teuthology.orchestra.run.smithi018:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T18:31:28.583 DEBUG:teuthology.orchestra.run.smithi103:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T18:31:28.612 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T18:31:28.630 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T18:31:28.630 DEBUG:teuthology.orchestra.run.smithi018:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T18:31:28.657 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T18:31:28.674 DEBUG:teuthology.orchestra.run.smithi018:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T18:31:28.751 DEBUG:teuthology.orchestra.run.smithi018:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T18:31:28.807 DEBUG:teuthology.orchestra.run.smithi018:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T18:31:28.891 DEBUG:teuthology.orchestra.run.smithi018:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T18:31:28.935 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-10T18:31:28.935 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T18:31:29.003 DEBUG:teuthology.orchestra.run.smithi103:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T18:31:29.048 DEBUG:teuthology.orchestra.run.smithi103:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T18:31:29.102 DEBUG:teuthology.orchestra.run.smithi103:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T18:31:29.188 DEBUG:teuthology.orchestra.run.smithi103:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T18:31:29.231 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-10T18:31:29.232 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T18:31:29.300 DEBUG:teuthology.orchestra.run.smithi018:> sudo service rsyslog restart 2024-09-10T18:31:29.303 DEBUG:teuthology.orchestra.run.smithi103:> sudo service rsyslog restart 2024-09-10T18:31:29.358 INFO:teuthology.orchestra.run.smithi018.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T18:31:29.392 INFO:teuthology.orchestra.run.smithi103.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T18:31:29.810 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T18:31:29.818 INFO:teuthology.task.internal:Starting timer... 2024-09-10T18:31:29.818 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T18:31:30.011 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T18:31:30.231 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-10T18:31:30.232 DEBUG:teuthology.orchestra.run.smithi018:> sudo service auditd rotate 2024-09-10T18:31:30.305 INFO:teuthology.orchestra.run.smithi018.stdout:Rotating logs: 2024-09-10T18:31:30.307 DEBUG:teuthology.orchestra.run.smithi103:> sudo service auditd rotate 2024-09-10T18:31:30.350 INFO:teuthology.orchestra.run.smithi103.stdout:Rotating logs: 2024-09-10T18:31:30.352 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T18:31:30.352 DEBUG:teuthology.orchestra.run.smithi018:> /usr/sbin/getenforce 2024-09-10T18:31:30.491 INFO:teuthology.orchestra.run.smithi018.stdout:Permissive 2024-09-10T18:31:30.491 DEBUG:teuthology.orchestra.run.smithi103:> /usr/sbin/getenforce 2024-09-10T18:31:30.517 INFO:teuthology.orchestra.run.smithi103.stdout:Permissive 2024-09-10T18:31:30.517 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi018.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi103.front.sepia.ceph.com': 'permissive'} 2024-09-10T18:31:30.518 DEBUG:teuthology.orchestra.run.smithi018:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T18:31:30.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:31:30.547 DEBUG:teuthology.orchestra.run.smithi103:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T18:31:30.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:31:30.588 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T18:31:30.588 DEBUG:teuthology.orchestra.run.smithi018:> sudo /usr/sbin/setenforce permissive 2024-09-10T18:31:30.623 DEBUG:teuthology.orchestra.run.smithi103:> sudo /usr/sbin/setenforce permissive 2024-09-10T18:31:30.659 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T18:31:30.669 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T18:31:30.682 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T18:31:30.683 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi018.front.sepia.ceph.com,smithi103.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T18:35:12.218 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi018.front.sepia.ceph.com'), Remote(name='ubuntu@smithi103.front.sepia.ceph.com')] 2024-09-10T18:35:12.219 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi018.front.sepia.ceph.com' 2024-09-10T18:35:12.220 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:35:12.300 DEBUG:teuthology.orchestra.run.smithi018:> true 2024-09-10T18:35:12.381 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi018.front.sepia.ceph.com' 2024-09-10T18:35:12.382 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-10T18:35:12.382 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:35:12.456 DEBUG:teuthology.orchestra.run.smithi103:> true 2024-09-10T18:35:12.532 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-10T18:35:12.532 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T18:35:12.543 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T18:35:12.543 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T18:35:12.544 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T18:35:12.546 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T18:35:12.547 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T18:35:12.576 INFO:teuthology.orchestra.run.smithi018.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T18:35:12.593 INFO:teuthology.orchestra.run.smithi018.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T18:35:12.602 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T18:35:12.617 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T18:35:12.619 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: ntpd: command not found 2024-09-10T18:35:12.631 INFO:teuthology.orchestra.run.smithi018.stdout:506 Cannot talk to daemon 2024-09-10T18:35:12.641 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: ntpd: command not found 2024-09-10T18:35:12.647 INFO:teuthology.orchestra.run.smithi018.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T18:35:12.652 INFO:teuthology.orchestra.run.smithi103.stdout:506 Cannot talk to daemon 2024-09-10T18:35:12.662 INFO:teuthology.orchestra.run.smithi018.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T18:35:12.666 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T18:35:12.680 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T18:35:12.724 INFO:teuthology.orchestra.run.smithi018.stderr:bash: line 1: ntpq: command not found 2024-09-10T18:35:12.729 INFO:teuthology.orchestra.run.smithi018.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T18:35:12.729 INFO:teuthology.orchestra.run.smithi018.stdout:=============================================================================== 2024-09-10T18:35:12.729 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:35:12.729 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:35:12.729 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:35:12.729 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:35:12.735 INFO:teuthology.orchestra.run.smithi103.stderr:bash: line 1: ntpq: command not found 2024-09-10T18:35:12.739 INFO:teuthology.orchestra.run.smithi103.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T18:35:12.739 INFO:teuthology.orchestra.run.smithi103.stdout:=============================================================================== 2024-09-10T18:35:12.739 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:35:12.739 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:35:12.739 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:35:12.739 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:35:12.740 INFO:teuthology.run_tasks:Running task pexec... 2024-09-10T18:35:12.749 INFO:teuthology.task.pexec:Executing custom commands... 2024-09-10T18:35:12.749 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi018.front.sepia.ceph.com 2024-09-10T18:35:12.749 DEBUG:teuthology.orchestra.run.smithi018:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-10T18:35:12.750 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi103.front.sepia.ceph.com 2024-09-10T18:35:12.750 DEBUG:teuthology.orchestra.run.smithi103:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-10T18:35:13.129 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-10T18:35:13.137 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-09-10T18:35:13.139 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:35:13.139 INFO:teuthology.orchestra.run.smithi018.stdout: Package Architecture Version Repository Size 2024-09-10T18:35:13.139 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:35:13.139 INFO:teuthology.orchestra.run.smithi018.stdout:Removing: 2024-09-10T18:35:13.139 INFO:teuthology.orchestra.run.smithi018.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-10T18:35:13.139 INFO:teuthology.orchestra.run.smithi018.stdout:Removing unused dependencies: 2024-09-10T18:35:13.139 INFO:teuthology.orchestra.run.smithi018.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-10T18:35:13.139 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:35:13.140 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-09-10T18:35:13.140 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:35:13.140 INFO:teuthology.orchestra.run.smithi018.stdout:Remove 2 Packages 2024-09-10T18:35:13.140 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:35:13.140 INFO:teuthology.orchestra.run.smithi018.stdout:Freed space: 5.5 M 2024-09-10T18:35:13.140 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-09-10T18:35:13.142 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:35:13.142 INFO:teuthology.orchestra.run.smithi103.stdout: Package Architecture Version Repository Size 2024-09-10T18:35:13.142 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:35:13.142 INFO:teuthology.orchestra.run.smithi103.stdout:Removing: 2024-09-10T18:35:13.142 INFO:teuthology.orchestra.run.smithi103.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-10T18:35:13.143 INFO:teuthology.orchestra.run.smithi103.stdout:Removing unused dependencies: 2024-09-10T18:35:13.143 INFO:teuthology.orchestra.run.smithi103.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-10T18:35:13.143 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:35:13.143 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-10T18:35:13.143 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:35:13.143 INFO:teuthology.orchestra.run.smithi103.stdout:Remove 2 Packages 2024-09-10T18:35:13.143 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:35:13.143 INFO:teuthology.orchestra.run.smithi103.stdout:Freed space: 5.5 M 2024-09-10T18:35:13.143 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-10T18:35:13.144 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-09-10T18:35:13.144 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-09-10T18:35:13.147 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-10T18:35:13.147 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-10T18:35:13.201 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-09-10T18:35:13.201 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-09-10T18:35:13.205 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-10T18:35:13.206 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-10T18:35:13.347 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-10T18:35:13.353 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-09-10T18:35:13.422 INFO:teuthology.orchestra.run.smithi018.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-10T18:35:13.464 INFO:teuthology.orchestra.run.smithi103.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-10T18:35:13.483 INFO:teuthology.orchestra.run.smithi018.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-10T18:35:13.521 INFO:teuthology.orchestra.run.smithi103.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-10T18:35:13.963 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-10T18:35:13.963 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-10T18:35:14.023 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-10T18:35:14.023 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-10T18:35:14.189 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-10T18:35:14.190 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:35:14.190 INFO:teuthology.orchestra.run.smithi103.stdout:Removed: 2024-09-10T18:35:14.190 INFO:teuthology.orchestra.run.smithi103.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-10T18:35:14.190 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:35:14.190 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-10T18:35:14.299 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-10T18:35:14.299 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:35:14.300 INFO:teuthology.orchestra.run.smithi018.stdout:Removed: 2024-09-10T18:35:14.300 INFO:teuthology.orchestra.run.smithi018.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-10T18:35:14.300 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:35:14.300 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-09-10T18:35:15.038 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:00:48 ago on Tue 10 Sep 2024 06:34:26 PM UTC. 2024-09-10T18:35:15.125 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:00:44 ago on Tue 10 Sep 2024 06:34:30 PM UTC. 2024-09-10T18:35:15.175 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-10T18:35:15.176 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:35:15.176 INFO:teuthology.orchestra.run.smithi103.stdout: Package Architecture Version Repository Size 2024-09-10T18:35:15.176 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:35:15.176 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-10T18:35:15.176 INFO:teuthology.orchestra.run.smithi103.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-10T18:35:15.176 INFO:teuthology.orchestra.run.smithi103.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-10T18:35:15.176 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2024-09-10T18:35:15.176 INFO:teuthology.orchestra.run.smithi103.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-10T18:35:15.176 INFO:teuthology.orchestra.run.smithi103.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-10T18:35:15.177 INFO:teuthology.orchestra.run.smithi103.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-10T18:35:15.177 INFO:teuthology.orchestra.run.smithi103.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-10T18:35:15.177 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:35:15.177 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-10T18:35:15.177 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:35:15.177 INFO:teuthology.orchestra.run.smithi103.stdout:Install 6 Packages 2024-09-10T18:35:15.177 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:35:15.183 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 2.0 M 2024-09-10T18:35:15.183 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 9.2 M 2024-09-10T18:35:15.183 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-10T18:35:15.261 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-09-10T18:35:15.262 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:35:15.262 INFO:teuthology.orchestra.run.smithi018.stdout: Package Architecture Version Repository Size 2024-09-10T18:35:15.262 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:35:15.262 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-09-10T18:35:15.262 INFO:teuthology.orchestra.run.smithi018.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-10T18:35:15.262 INFO:teuthology.orchestra.run.smithi018.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-10T18:35:15.263 INFO:teuthology.orchestra.run.smithi018.stdout:Installing dependencies: 2024-09-10T18:35:15.263 INFO:teuthology.orchestra.run.smithi018.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-10T18:35:15.263 INFO:teuthology.orchestra.run.smithi018.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-10T18:35:15.263 INFO:teuthology.orchestra.run.smithi018.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-10T18:35:15.263 INFO:teuthology.orchestra.run.smithi018.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-10T18:35:15.263 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:35:15.263 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-09-10T18:35:15.263 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:35:15.263 INFO:teuthology.orchestra.run.smithi018.stdout:Install 6 Packages 2024-09-10T18:35:15.263 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:35:15.268 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 2.0 M 2024-09-10T18:35:15.268 INFO:teuthology.orchestra.run.smithi018.stdout:Installed size: 9.2 M 2024-09-10T18:35:15.268 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-09-10T18:35:15.387 INFO:teuthology.orchestra.run.smithi103.stdout:(1/6): nvmetcli-0.7-3.el9.noarch.rpm 298 kB/s | 44 kB 00:00 2024-09-10T18:35:15.412 INFO:teuthology.orchestra.run.smithi103.stdout:(2/6): libnvme-1.9-3.el9.x86_64.rpm 608 kB/s | 106 kB 00:00 2024-09-10T18:35:15.437 INFO:teuthology.orchestra.run.smithi018.stdout:(1/6): nvmetcli-0.7-3.el9.noarch.rpm 372 kB/s | 44 kB 00:00 2024-09-10T18:35:15.463 INFO:teuthology.orchestra.run.smithi103.stdout:(3/6): python3-kmod-0.9-32.el9.x86_64.rpm 1.6 MB/s | 84 kB 00:00 2024-09-10T18:35:15.471 INFO:teuthology.orchestra.run.smithi018.stdout:(2/6): libnvme-1.9-3.el9.x86_64.rpm 693 kB/s | 106 kB 00:00 2024-09-10T18:35:15.488 INFO:teuthology.orchestra.run.smithi103.stdout:(4/6): python3-configshell-1.1.30-1.el9.noarch. 712 kB/s | 72 kB 00:00 2024-09-10T18:35:15.513 INFO:teuthology.orchestra.run.smithi018.stdout:(3/6): python3-configshell-1.1.30-1.el9.noarch. 952 kB/s | 72 kB 00:00 2024-09-10T18:35:15.522 INFO:teuthology.orchestra.run.smithi103.stdout:(5/6): nvme-cli-2.9.1-6.el9.x86_64.rpm 3.1 MB/s | 906 kB 00:00 2024-09-10T18:35:15.546 INFO:teuthology.orchestra.run.smithi018.stdout:(4/6): python3-kmod-0.9-32.el9.x86_64.rpm 1.1 MB/s | 84 kB 00:00 2024-09-10T18:35:15.580 INFO:teuthology.orchestra.run.smithi018.stdout:(5/6): nvme-cli-2.9.1-6.el9.x86_64.rpm 3.4 MB/s | 906 kB 00:00 2024-09-10T18:35:15.647 INFO:teuthology.orchestra.run.smithi103.stdout:(6/6): python3-urwid-2.1.2-4.el9.x86_64.rpm 4.5 MB/s | 837 kB 00:00 2024-09-10T18:35:15.647 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:35:15.647 INFO:teuthology.orchestra.run.smithi103.stdout:Total 4.3 MB/s | 2.0 MB 00:00 2024-09-10T18:35:15.655 INFO:teuthology.orchestra.run.smithi018.stdout:(6/6): python3-urwid-2.1.2-4.el9.x86_64.rpm 5.8 MB/s | 837 kB 00:00 2024-09-10T18:35:15.656 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:35:15.656 INFO:teuthology.orchestra.run.smithi018.stdout:Total 5.2 MB/s | 2.0 MB 00:00 2024-09-10T18:35:15.721 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-10T18:35:15.732 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-09-10T18:35:15.735 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-10T18:35:15.735 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-10T18:35:15.747 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-09-10T18:35:15.747 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-09-10T18:35:16.014 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-10T18:35:16.014 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-10T18:35:16.062 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-09-10T18:35:16.062 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-09-10T18:35:16.441 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-10T18:35:16.504 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-09-10T18:35:16.525 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/6 2024-09-10T18:35:16.568 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/6 2024-09-10T18:35:16.593 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/6 2024-09-10T18:35:16.597 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/6 2024-09-10T18:35:16.635 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/6 2024-09-10T18:35:16.663 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/6 2024-09-10T18:35:16.806 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/6 2024-09-10T18:35:16.836 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-10T18:35:16.895 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/6 2024-09-10T18:35:16.935 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-10T18:35:17.285 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-10T18:35:17.314 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/6 2024-09-10T18:35:17.435 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-10T18:35:17.463 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/6 2024-09-10T18:35:18.265 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/6 2024-09-10T18:35:18.265 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/6 2024-09-10T18:35:18.265 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/6 2024-09-10T18:35:18.265 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/6 2024-09-10T18:35:18.265 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/6 2024-09-10T18:35:18.265 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/6 2024-09-10T18:35:18.451 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/6 2024-09-10T18:35:18.451 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/6 2024-09-10T18:35:18.451 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/6 2024-09-10T18:35:18.451 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/6 2024-09-10T18:35:18.451 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/6 2024-09-10T18:35:18.451 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/6 2024-09-10T18:35:18.611 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/6 2024-09-10T18:35:18.611 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:35:18.611 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-10T18:35:18.611 INFO:teuthology.orchestra.run.smithi103.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-10T18:35:18.611 INFO:teuthology.orchestra.run.smithi103.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-10T18:35:18.611 INFO:teuthology.orchestra.run.smithi103.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-10T18:35:18.611 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:35:18.611 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-10T18:35:18.762 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/6 2024-09-10T18:35:18.762 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:35:18.762 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-09-10T18:35:18.762 INFO:teuthology.orchestra.run.smithi018.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-10T18:35:18.762 INFO:teuthology.orchestra.run.smithi018.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-10T18:35:18.763 INFO:teuthology.orchestra.run.smithi018.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-10T18:35:18.763 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:35:18.763 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-09-10T18:35:18.901 DEBUG:teuthology.parallel:result is None 2024-09-10T18:35:19.086 DEBUG:teuthology.parallel:result is None 2024-09-10T18:35:19.086 INFO:teuthology.run_tasks:Running task install... 2024-09-10T18:35:19.096 DEBUG:teuthology.task.install:project ceph 2024-09-10T18:35:19.096 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588'}} 2024-09-10T18:35:19.097 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588'} 2024-09-10T18:35:19.097 INFO:teuthology.task.install:Using flavor: default 2024-09-10T18:35:19.103 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T18:35:19.103 INFO:teuthology.task.install:extra packages: [] 2024-09-10T18:35:19.103 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588', 'tag': None, 'wait_for_package': False} 2024-09-10T18:35:19.104 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:35:19.106 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588', 'tag': None, 'wait_for_package': False} 2024-09-10T18:35:19.106 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:35:19.250 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/ 2024-09-10T18:35:19.250 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1606.g5be57395 2024-09-10T18:35:19.253 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/ 2024-09-10T18:35:19.253 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1606.g5be57395 2024-09-10T18:35:19.367 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T18:35:19.367 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-10T18:35:19.367 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T18:35:19.371 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T18:35:19.371 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-10T18:35:19.371 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T18:35:19.396 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-10T18:35:19.396 DEBUG:teuthology.orchestra.run.smithi103:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5be57395ea235535910512ae73ba8e7ff25fe588/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T18:35:19.402 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-10T18:35:19.402 DEBUG:teuthology.orchestra.run.smithi018:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5be57395ea235535910512ae73ba8e7ff25fe588/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T18:35:19.470 DEBUG:teuthology.orchestra.run.smithi018:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T18:35:19.472 DEBUG:teuthology.orchestra.run.smithi103:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T18:35:19.549 DEBUG:teuthology.orchestra.run.smithi103:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T18:35:19.569 DEBUG:teuthology.orchestra.run.smithi018:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T18:35:19.594 INFO:teuthology.orchestra.run.smithi018.stdout:check_obsoletes = 1 2024-09-10T18:35:19.595 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum clean all 2024-09-10T18:35:19.613 INFO:teuthology.orchestra.run.smithi103.stdout:check_obsoletes = 1 2024-09-10T18:35:19.615 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum clean all 2024-09-10T18:35:19.835 INFO:teuthology.orchestra.run.smithi018.stdout:45 files removed 2024-09-10T18:35:19.844 INFO:teuthology.orchestra.run.smithi103.stdout:45 files removed 2024-09-10T18:35:19.862 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install ceph-radosgw 2024-09-10T18:35:19.870 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph-radosgw 2024-09-10T18:35:20.422 INFO:teuthology.orchestra.run.smithi103.stdout:ceph packages for x86_64 290 kB/s | 77 kB 00:00 2024-09-10T18:35:20.437 INFO:teuthology.orchestra.run.smithi018.stdout:ceph packages for x86_64 282 kB/s | 77 kB 00:00 2024-09-10T18:35:20.706 INFO:teuthology.orchestra.run.smithi103.stdout:ceph noarch packages 45 kB/s | 11 kB 00:00 2024-09-10T18:35:20.729 INFO:teuthology.orchestra.run.smithi018.stdout:ceph noarch packages 44 kB/s | 11 kB 00:00 2024-09-10T18:35:20.982 INFO:teuthology.orchestra.run.smithi103.stdout:ceph source packages 7.5 kB/s | 1.8 kB 00:00 2024-09-10T18:35:21.005 INFO:teuthology.orchestra.run.smithi018.stdout:ceph source packages 7.5 kB/s | 1.8 kB 00:00 2024-09-10T18:35:21.432 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - BaseOS 20 MB/s | 8.2 MB 00:00 2024-09-10T18:35:23.200 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - AppStream 29 MB/s | 20 MB 00:00 2024-09-10T18:35:23.679 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - BaseOS 3.1 MB/s | 8.2 MB 00:02 2024-09-10T18:35:25.397 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - AppStream 34 MB/s | 20 MB 00:00 2024-09-10T18:35:28.710 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.5 MB 00:01 2024-09-10T18:35:29.756 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.5 MB 00:00 2024-09-10T18:35:30.235 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - Extras packages 103 kB/s | 19 kB 00:00 2024-09-10T18:35:31.215 INFO:teuthology.orchestra.run.smithi103.stdout:Extra Packages for Enterprise Linux 26 MB/s | 23 MB 00:00 2024-09-10T18:35:31.215 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - Extras packages 93 kB/s | 19 kB 00:00 2024-09-10T18:35:32.136 INFO:teuthology.orchestra.run.smithi018.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-10T18:35:36.662 INFO:teuthology.orchestra.run.smithi103.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-09-10T18:35:37.759 INFO:teuthology.orchestra.run.smithi018.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-10T18:35:39.038 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-10T18:35:39.040 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:35:39.040 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repository Size 2024-09-10T18:35:39.040 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:35:39.040 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-10T18:35:39.041 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-radosgw x86_64 2:17.2.7-1606.g5be57395.el9 ceph 14 M 2024-09-10T18:35:39.041 INFO:teuthology.orchestra.run.smithi103.stdout:Upgrading: 2024-09-10T18:35:39.041 INFO:teuthology.orchestra.run.smithi103.stdout: librados2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 3.1 M 2024-09-10T18:35:39.041 INFO:teuthology.orchestra.run.smithi103.stdout: librbd1 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.9 M 2024-09-10T18:35:39.041 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2024-09-10T18:35:39.041 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-base x86_64 2:17.2.7-1606.g5be57395.el9 ceph 5.0 M 2024-09-10T18:35:39.041 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-common x86_64 2:17.2.7-1606.g5be57395.el9 ceph 22 M 2024-09-10T18:35:39.041 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-selinux x86_64 2:17.2.7-1606.g5be57395.el9 ceph 25 k 2024-09-10T18:35:39.041 INFO:teuthology.orchestra.run.smithi103.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T18:35:39.042 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 637 k 2024-09-10T18:35:39.042 INFO:teuthology.orchestra.run.smithi103.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T18:35:39.042 INFO:teuthology.orchestra.run.smithi103.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T18:35:39.042 INFO:teuthology.orchestra.run.smithi103.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T18:35:39.042 INFO:teuthology.orchestra.run.smithi103.stdout: libradosstriper1 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 458 k 2024-09-10T18:35:39.042 INFO:teuthology.orchestra.run.smithi103.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T18:35:39.042 INFO:teuthology.orchestra.run.smithi103.stdout: librgw2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 4.9 M 2024-09-10T18:35:39.042 INFO:teuthology.orchestra.run.smithi103.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T18:35:39.042 INFO:teuthology.orchestra.run.smithi103.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T18:35:39.042 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-argparse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 45 k 2024-09-10T18:35:39.043 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-common x86_64 2:17.2.7-1606.g5be57395.el9 ceph 120 k 2024-09-10T18:35:39.043 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cephfs x86_64 2:17.2.7-1606.g5be57395.el9 ceph 156 k 2024-09-10T18:35:39.043 INFO:teuthology.orchestra.run.smithi103.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T18:35:39.043 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rados x86_64 2:17.2.7-1606.g5be57395.el9 ceph 320 k 2024-09-10T18:35:39.043 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rbd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 295 k 2024-09-10T18:35:39.043 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rgw x86_64 2:17.2.7-1606.g5be57395.el9 ceph 99 k 2024-09-10T18:35:39.043 INFO:teuthology.orchestra.run.smithi103.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T18:35:39.043 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:35:39.043 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-10T18:35:39.043 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:35:39.044 INFO:teuthology.orchestra.run.smithi103.stdout:Install 22 Packages 2024-09-10T18:35:39.044 INFO:teuthology.orchestra.run.smithi103.stdout:Upgrade 2 Packages 2024-09-10T18:35:39.044 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:35:39.044 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 58 M 2024-09-10T18:35:39.044 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-10T18:35:39.664 INFO:teuthology.orchestra.run.smithi103.stdout:(1/24): ceph-base-17.2.7-1606.g5be57395.el9.x86 10 MB/s | 5.0 MB 00:00 2024-09-10T18:35:39.714 INFO:teuthology.orchestra.run.smithi103.stdout:(2/24): ceph-selinux-17.2.7-1606.g5be57395.el9. 502 kB/s | 25 kB 00:00 2024-09-10T18:35:39.823 INFO:teuthology.orchestra.run.smithi103.stdout:(3/24): libcephfs2-17.2.7-1606.g5be57395.el9.x8 5.7 MB/s | 637 kB 00:00 2024-09-10T18:35:39.923 INFO:teuthology.orchestra.run.smithi103.stdout:(4/24): libradosstriper1-17.2.7-1606.g5be57395. 4.5 MB/s | 458 kB 00:00 2024-09-10T18:35:40.282 INFO:teuthology.orchestra.run.smithi103.stdout:(5/24): librgw2-17.2.7-1606.g5be57395.el9.x86_6 14 MB/s | 4.9 MB 00:00 2024-09-10T18:35:40.287 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-09-10T18:35:40.289 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:35:40.289 INFO:teuthology.orchestra.run.smithi018.stdout: Package Arch Version Repository Size 2024-09-10T18:35:40.289 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:35:40.290 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-09-10T18:35:40.290 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-radosgw x86_64 2:17.2.7-1606.g5be57395.el9 ceph 14 M 2024-09-10T18:35:40.290 INFO:teuthology.orchestra.run.smithi018.stdout:Upgrading: 2024-09-10T18:35:40.290 INFO:teuthology.orchestra.run.smithi018.stdout: librados2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 3.1 M 2024-09-10T18:35:40.290 INFO:teuthology.orchestra.run.smithi018.stdout: librbd1 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.9 M 2024-09-10T18:35:40.290 INFO:teuthology.orchestra.run.smithi018.stdout:Installing dependencies: 2024-09-10T18:35:40.290 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-base x86_64 2:17.2.7-1606.g5be57395.el9 ceph 5.0 M 2024-09-10T18:35:40.290 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-common x86_64 2:17.2.7-1606.g5be57395.el9 ceph 22 M 2024-09-10T18:35:40.290 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-selinux x86_64 2:17.2.7-1606.g5be57395.el9 ceph 25 k 2024-09-10T18:35:40.291 INFO:teuthology.orchestra.run.smithi018.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T18:35:40.291 INFO:teuthology.orchestra.run.smithi018.stdout: libcephfs2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 637 k 2024-09-10T18:35:40.291 INFO:teuthology.orchestra.run.smithi018.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T18:35:40.291 INFO:teuthology.orchestra.run.smithi018.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T18:35:40.291 INFO:teuthology.orchestra.run.smithi018.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T18:35:40.291 INFO:teuthology.orchestra.run.smithi018.stdout: libradosstriper1 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 458 k 2024-09-10T18:35:40.291 INFO:teuthology.orchestra.run.smithi018.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T18:35:40.291 INFO:teuthology.orchestra.run.smithi018.stdout: librgw2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 4.9 M 2024-09-10T18:35:40.291 INFO:teuthology.orchestra.run.smithi018.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T18:35:40.291 INFO:teuthology.orchestra.run.smithi018.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T18:35:40.291 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ceph-argparse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 45 k 2024-09-10T18:35:40.291 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ceph-common x86_64 2:17.2.7-1606.g5be57395.el9 ceph 120 k 2024-09-10T18:35:40.291 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cephfs x86_64 2:17.2.7-1606.g5be57395.el9 ceph 156 k 2024-09-10T18:35:40.291 INFO:teuthology.orchestra.run.smithi018.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T18:35:40.292 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rados x86_64 2:17.2.7-1606.g5be57395.el9 ceph 320 k 2024-09-10T18:35:40.292 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rbd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 295 k 2024-09-10T18:35:40.292 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rgw x86_64 2:17.2.7-1606.g5be57395.el9 ceph 99 k 2024-09-10T18:35:40.292 INFO:teuthology.orchestra.run.smithi018.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T18:35:40.292 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:35:40.292 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-09-10T18:35:40.292 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:35:40.292 INFO:teuthology.orchestra.run.smithi018.stdout:Install 22 Packages 2024-09-10T18:35:40.292 INFO:teuthology.orchestra.run.smithi018.stdout:Upgrade 2 Packages 2024-09-10T18:35:40.292 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:35:40.293 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 58 M 2024-09-10T18:35:40.293 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-09-10T18:35:40.432 INFO:teuthology.orchestra.run.smithi103.stdout:(6/24): ceph-radosgw-17.2.7-1606.g5be57395.el9. 11 MB/s | 14 MB 00:01 2024-09-10T18:35:40.457 INFO:teuthology.orchestra.run.smithi103.stdout:(7/24): python3-ceph-argparse-17.2.7-1606.g5be5 258 kB/s | 45 kB 00:00 2024-09-10T18:35:40.483 INFO:teuthology.orchestra.run.smithi103.stdout:(8/24): python3-ceph-common-17.2.7-1606.g5be573 2.3 MB/s | 120 kB 00:00 2024-09-10T18:35:40.508 INFO:teuthology.orchestra.run.smithi103.stdout:(9/24): python3-cephfs-17.2.7-1606.g5be57395.el 3.0 MB/s | 156 kB 00:00 2024-09-10T18:35:40.725 INFO:teuthology.orchestra.run.smithi103.stdout:(10/24): ceph-common-17.2.7-1606.g5be57395.el9. 14 MB/s | 22 MB 00:01 2024-09-10T18:35:40.750 INFO:teuthology.orchestra.run.smithi103.stdout:(11/24): python3-rados-17.2.7-1606.g5be57395.el 1.2 MB/s | 320 kB 00:00 2024-09-10T18:35:40.776 INFO:teuthology.orchestra.run.smithi103.stdout:(12/24): python3-rgw-17.2.7-1606.g5be57395.el9. 1.9 MB/s | 99 kB 00:00 2024-09-10T18:35:40.810 INFO:teuthology.orchestra.run.smithi103.stdout:(13/24): python3-rbd-17.2.7-1606.g5be57395.el9. 978 kB/s | 295 kB 00:00 2024-09-10T18:35:41.127 INFO:teuthology.orchestra.run.smithi018.stdout:(1/24): ceph-base-17.2.7-1606.g5be57395.el9.x86 7.2 MB/s | 5.0 MB 00:00 2024-09-10T18:35:41.202 INFO:teuthology.orchestra.run.smithi018.stdout:(2/24): ceph-selinux-17.2.7-1606.g5be57395.el9. 336 kB/s | 25 kB 00:00 2024-09-10T18:35:41.319 INFO:teuthology.orchestra.run.smithi018.stdout:(3/24): libcephfs2-17.2.7-1606.g5be57395.el9.x8 5.3 MB/s | 637 kB 00:00 2024-09-10T18:35:41.418 INFO:teuthology.orchestra.run.smithi103.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 70 kB/s | 45 kB 00:00 2024-09-10T18:35:41.436 INFO:teuthology.orchestra.run.smithi018.stdout:(4/24): libradosstriper1-17.2.7-1606.g5be57395. 3.8 MB/s | 458 kB 00:00 2024-09-10T18:35:41.518 INFO:teuthology.orchestra.run.smithi103.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 209 kB/s | 160 kB 00:00 2024-09-10T18:35:41.678 INFO:teuthology.orchestra.run.smithi018.stdout:(5/24): ceph-common-17.2.7-1606.g5be57395.el9.x 18 MB/s | 22 MB 00:01 2024-09-10T18:35:41.752 INFO:teuthology.orchestra.run.smithi103.stdout:(16/24): librdkafka-1.6.1-102.el9.x86_64.rpm 703 kB/s | 662 kB 00:00 2024-09-10T18:35:41.827 INFO:teuthology.orchestra.run.smithi103.stdout:(17/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 4.5 MB/s | 309 kB 00:00 2024-09-10T18:35:41.828 INFO:teuthology.orchestra.run.smithi018.stdout:(6/24): ceph-radosgw-17.2.7-1606.g5be57395.el9. 10 MB/s | 14 MB 00:01 2024-09-10T18:35:41.853 INFO:teuthology.orchestra.run.smithi103.stdout:(18/24): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-10T18:35:41.853 INFO:teuthology.orchestra.run.smithi018.stdout:(7/24): python3-ceph-argparse-17.2.7-1606.g5be5 258 kB/s | 45 kB 00:00 2024-09-10T18:35:41.878 INFO:teuthology.orchestra.run.smithi103.stdout:(19/24): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-09-10T18:35:41.879 INFO:teuthology.orchestra.run.smithi018.stdout:(8/24): python3-ceph-common-17.2.7-1606.g5be573 2.3 MB/s | 120 kB 00:00 2024-09-10T18:35:41.904 INFO:teuthology.orchestra.run.smithi018.stdout:(9/24): python3-cephfs-17.2.7-1606.g5be57395.el 3.0 MB/s | 156 kB 00:00 2024-09-10T18:35:41.929 INFO:teuthology.orchestra.run.smithi018.stdout:(10/24): python3-rados-17.2.7-1606.g5be57395.el 6.2 MB/s | 320 kB 00:00 2024-09-10T18:35:41.936 INFO:teuthology.orchestra.run.smithi103.stdout:(20/24): thrift-0.15.0-2.el9.x86_64.rpm 27 MB/s | 1.6 MB 00:00 2024-09-10T18:35:41.955 INFO:teuthology.orchestra.run.smithi018.stdout:(11/24): python3-rbd-17.2.7-1606.g5be57395.el9. 5.7 MB/s | 295 kB 00:00 2024-09-10T18:35:41.980 INFO:teuthology.orchestra.run.smithi018.stdout:(12/24): python3-rgw-17.2.7-1606.g5be57395.el9. 1.9 MB/s | 99 kB 00:00 2024-09-10T18:35:42.072 INFO:teuthology.orchestra.run.smithi018.stdout:(13/24): librgw2-17.2.7-1606.g5be57395.el9.x86_ 7.7 MB/s | 4.9 MB 00:00 2024-09-10T18:35:42.137 INFO:teuthology.orchestra.run.smithi103.stdout:(21/24): librados2-17.2.7-1606.g5be57395.el9.x8 15 MB/s | 3.1 MB 00:00 2024-09-10T18:35:42.156 INFO:teuthology.orchestra.run.smithi018.stdout:(14/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 799 kB/s | 160 kB 00:00 2024-09-10T18:35:42.162 INFO:teuthology.orchestra.run.smithi103.stdout:(22/24): python3-prettytable-0.7.2-27.el9.noarc 66 kB/s | 42 kB 00:00 2024-09-10T18:35:42.181 INFO:teuthology.orchestra.run.smithi018.stdout:(15/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 225 kB/s | 45 kB 00:00 2024-09-10T18:35:42.254 INFO:teuthology.orchestra.run.smithi103.stdout:(23/24): librbd1-17.2.7-1606.g5be57395.el9.x86_ 25 MB/s | 2.9 MB 00:00 2024-09-10T18:35:42.256 INFO:teuthology.orchestra.run.smithi018.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 564 kB/s | 42 kB 00:00 2024-09-10T18:35:42.279 INFO:teuthology.orchestra.run.smithi103.stdout:(24/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 340 kB/s | 292 kB 00:00 2024-09-10T18:35:42.284 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:35:42.284 INFO:teuthology.orchestra.run.smithi103.stdout:Total 18 MB/s | 58 MB 00:03 2024-09-10T18:35:42.298 INFO:teuthology.orchestra.run.smithi018.stdout:(17/24): librdkafka-1.6.1-102.el9.x86_64.rpm 2.9 MB/s | 662 kB 00:00 2024-09-10T18:35:42.345 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-10T18:35:42.348 INFO:teuthology.orchestra.run.smithi018.stdout:(18/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.5 MB/s | 292 kB 00:00 2024-09-10T18:35:42.380 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-10T18:35:42.380 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-10T18:35:42.444 INFO:teuthology.orchestra.run.smithi018.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 354 kB/s | 49 kB 00:00 2024-09-10T18:35:42.507 INFO:teuthology.orchestra.run.smithi018.stdout:(20/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.2 MB/s | 309 kB 00:00 2024-09-10T18:35:42.532 INFO:teuthology.orchestra.run.smithi018.stdout:(21/24): libunwind-1.6.2-1.el9.x86_64.rpm 370 kB/s | 67 kB 00:00 2024-09-10T18:35:42.583 INFO:teuthology.orchestra.run.smithi018.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-10T18:35:42.725 INFO:teuthology.orchestra.run.smithi018.stdout:(23/24): librados2-17.2.7-1606.g5be57395.el9.x8 14 MB/s | 3.1 MB 00:00 2024-09-10T18:35:42.775 INFO:teuthology.orchestra.run.smithi018.stdout:(24/24): librbd1-17.2.7-1606.g5be57395.el9.x86_ 12 MB/s | 2.9 MB 00:00 2024-09-10T18:35:42.781 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:35:42.781 INFO:teuthology.orchestra.run.smithi018.stdout:Total 23 MB/s | 58 MB 00:02 2024-09-10T18:35:42.848 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-09-10T18:35:42.860 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-10T18:35:42.860 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-10T18:35:42.885 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-09-10T18:35:42.886 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-09-10T18:35:43.389 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-09-10T18:35:43.390 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-09-10T18:35:44.077 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-10T18:35:44.469 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-10T18:35:44.549 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2/26 2024-09-10T18:35:44.607 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-09-10T18:35:44.627 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2/26 2024-09-10T18:35:44.703 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 3/26 2024-09-10T18:35:44.750 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 4/26 2024-09-10T18:35:44.839 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 4/26 2024-09-10T18:35:44.885 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-10T18:35:44.922 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-10T18:35:45.008 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-10T18:35:45.069 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2/26 2024-09-10T18:35:45.172 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2/26 2024-09-10T18:35:45.251 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 3/26 2024-09-10T18:35:45.288 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 4/26 2024-09-10T18:35:45.373 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-10T18:35:45.376 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 4/26 2024-09-10T18:35:45.404 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 8/26 2024-09-10T18:35:45.427 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-10T18:35:45.466 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 8/26 2024-09-10T18:35:45.467 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-10T18:35:45.503 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-10T18:35:45.547 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9. 10/26 2024-09-10T18:35:45.584 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 11/26 2024-09-10T18:35:45.645 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 12/26 2024-09-10T18:35:45.682 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_6 13/26 2024-09-10T18:35:45.747 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_6 13/26 2024-09-10T18:35:45.789 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-10T18:35:45.832 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-09-10T18:35:45.865 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-10T18:35:45.930 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-10T18:35:45.957 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 8/26 2024-09-10T18:35:46.045 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 8/26 2024-09-10T18:35:46.082 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-10T18:35:46.126 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9. 10/26 2024-09-10T18:35:46.162 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 11/26 2024-09-10T18:35:46.172 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-10T18:35:46.217 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 18/26 2024-09-10T18:35:46.226 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 12/26 2024-09-10T18:35:46.251 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_6 13/26 2024-09-10T18:35:46.290 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 18/26 2024-09-10T18:35:46.317 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_6 13/26 2024-09-10T18:35:46.364 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 19/26 2024-09-10T18:35:46.367 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-10T18:35:46.394 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x8 20/26 2024-09-10T18:35:46.419 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-09-10T18:35:46.460 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-10T18:35:46.787 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-10T18:35:46.822 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 18/26 2024-09-10T18:35:46.914 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 18/26 2024-09-10T18:35:46.989 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 19/26 2024-09-10T18:35:47.033 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x8 20/26 2024-09-10T18:35:48.873 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 21/26 2024-09-10T18:35:48.909 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 21/26 2024-09-10T18:35:49.429 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 21/26 2024-09-10T18:35:49.462 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 22/26 2024-09-10T18:35:49.593 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 21/26 2024-09-10T18:35:49.605 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 22/26 2024-09-10T18:35:49.605 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T18:35:49.605 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T18:35:49.605 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:35:49.623 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 21/26 2024-09-10T18:35:49.647 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 23/26 2024-09-10T18:35:50.187 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 21/26 2024-09-10T18:35:50.227 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 22/26 2024-09-10T18:35:50.325 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 22/26 2024-09-10T18:35:50.326 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T18:35:50.326 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T18:35:50.326 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:35:50.370 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 23/26 2024-09-10T18:35:59.526 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 23/26 2024-09-10T18:35:59.526 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /sys 2024-09-10T18:35:59.527 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /proc 2024-09-10T18:35:59.527 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /mnt 2024-09-10T18:35:59.527 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /var/tmp 2024-09-10T18:35:59.527 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /home 2024-09-10T18:35:59.527 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /root 2024-09-10T18:35:59.527 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /tmp 2024-09-10T18:35:59.527 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:00.621 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 23/26 2024-09-10T18:36:00.621 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /sys 2024-09-10T18:36:00.621 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /proc 2024-09-10T18:36:00.622 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /mnt 2024-09-10T18:36:00.622 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /var/tmp 2024-09-10T18:36:00.622 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /home 2024-09-10T18:36:00.622 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /root 2024-09-10T18:36:00.622 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /tmp 2024-09-10T18:36:00.622 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:00.823 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 24/26 2024-09-10T18:36:00.914 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 24/26 2024-09-10T18:36:00.914 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:36:00.914 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T18:36:00.914 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T18:36:00.914 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T18:36:00.915 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:00.916 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-10T18:36:01.009 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-10T18:36:01.009 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-10T18:36:01.979 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 24/26 2024-09-10T18:36:02.059 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 24/26 2024-09-10T18:36:02.059 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:36:02.059 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T18:36:02.059 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T18:36:02.059 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T18:36:02.059 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:02.061 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-10T18:36:02.157 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-10T18:36:02.157 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-10T18:36:02.287 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-10T18:36:02.287 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 1/26 2024-09-10T18:36:02.287 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2/26 2024-09-10T18:36:02.287 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 3/26 2024-09-10T18:36:02.287 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 4/26 2024-09-10T18:36:02.287 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 5/26 2024-09-10T18:36:02.287 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_6 6/26 2024-09-10T18:36:02.287 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 7/26 2024-09-10T18:36:02.287 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9. 8/26 2024-09-10T18:36:02.287 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x8 9/26 2024-09-10T18:36:02.287 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 10/26 2024-09-10T18:36:02.288 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 11/26 2024-09-10T18:36:02.288 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 12/26 2024-09-10T18:36:02.288 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 13/26 2024-09-10T18:36:02.288 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-10T18:36:02.288 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-10T18:36:02.288 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-10T18:36:02.288 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-10T18:36:02.288 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-10T18:36:02.288 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-09-10T18:36:02.288 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-10T18:36:02.288 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-10T18:36:02.288 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-10T18:36:02.289 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados2-2:17.2.7-1606.g5be57395.el9.x86_64 23/26 2024-09-10T18:36:02.289 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-10T18:36:02.289 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 25/26 2024-09-10T18:36:03.133 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-10T18:36:03.133 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:03.133 INFO:teuthology.orchestra.run.smithi103.stdout:Upgraded: 2024-09-10T18:36:03.133 INFO:teuthology.orchestra.run.smithi103.stdout: librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:03.133 INFO:teuthology.orchestra.run.smithi103.stdout: librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:03.133 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-10T18:36:03.133 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:03.133 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:03.134 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:03.134 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:03.134 INFO:teuthology.orchestra.run.smithi103.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T18:36:03.134 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:03.134 INFO:teuthology.orchestra.run.smithi103.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T18:36:03.134 INFO:teuthology.orchestra.run.smithi103.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T18:36:03.134 INFO:teuthology.orchestra.run.smithi103.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T18:36:03.134 INFO:teuthology.orchestra.run.smithi103.stdout: libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:03.134 INFO:teuthology.orchestra.run.smithi103.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T18:36:03.134 INFO:teuthology.orchestra.run.smithi103.stdout: librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:03.134 INFO:teuthology.orchestra.run.smithi103.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T18:36:03.134 INFO:teuthology.orchestra.run.smithi103.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T18:36:03.134 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:03.135 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:03.135 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:03.135 INFO:teuthology.orchestra.run.smithi103.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T18:36:03.135 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:03.135 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:03.135 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:03.135 INFO:teuthology.orchestra.run.smithi103.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T18:36:03.135 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:03.135 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-10T18:36:03.350 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-10T18:36:03.350 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 1/26 2024-09-10T18:36:03.350 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2/26 2024-09-10T18:36:03.350 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 3/26 2024-09-10T18:36:03.350 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 4/26 2024-09-10T18:36:03.350 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 5/26 2024-09-10T18:36:03.350 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_6 6/26 2024-09-10T18:36:03.351 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 7/26 2024-09-10T18:36:03.351 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9. 8/26 2024-09-10T18:36:03.351 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x8 9/26 2024-09-10T18:36:03.351 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 10/26 2024-09-10T18:36:03.351 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 11/26 2024-09-10T18:36:03.351 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 12/26 2024-09-10T18:36:03.351 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 13/26 2024-09-10T18:36:03.351 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-10T18:36:03.351 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-10T18:36:03.351 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-10T18:36:03.351 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-10T18:36:03.351 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-10T18:36:03.351 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-09-10T18:36:03.352 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-10T18:36:03.352 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-10T18:36:03.352 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-10T18:36:03.352 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librados2-2:17.2.7-1606.g5be57395.el9.x86_64 23/26 2024-09-10T18:36:03.352 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-10T18:36:03.352 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 25/26 2024-09-10T18:36:03.511 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph-test 2024-09-10T18:36:04.085 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:00:28 ago on Tue 10 Sep 2024 06:35:36 PM UTC. 2024-09-10T18:36:04.203 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-10T18:36:04.203 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:04.203 INFO:teuthology.orchestra.run.smithi018.stdout:Upgraded: 2024-09-10T18:36:04.203 INFO:teuthology.orchestra.run.smithi018.stdout: librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:04.203 INFO:teuthology.orchestra.run.smithi018.stdout: librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:04.203 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-09-10T18:36:04.203 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:04.204 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:04.204 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:04.204 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:04.204 INFO:teuthology.orchestra.run.smithi018.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T18:36:04.204 INFO:teuthology.orchestra.run.smithi018.stdout: libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:04.204 INFO:teuthology.orchestra.run.smithi018.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T18:36:04.204 INFO:teuthology.orchestra.run.smithi018.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T18:36:04.204 INFO:teuthology.orchestra.run.smithi018.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T18:36:04.204 INFO:teuthology.orchestra.run.smithi018.stdout: libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:04.204 INFO:teuthology.orchestra.run.smithi018.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T18:36:04.204 INFO:teuthology.orchestra.run.smithi018.stdout: librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:04.204 INFO:teuthology.orchestra.run.smithi018.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T18:36:04.205 INFO:teuthology.orchestra.run.smithi018.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T18:36:04.205 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:04.205 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:04.205 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:04.205 INFO:teuthology.orchestra.run.smithi018.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T18:36:04.205 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:04.205 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:04.205 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:04.205 INFO:teuthology.orchestra.run.smithi018.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T18:36:04.205 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:04.205 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-09-10T18:36:04.223 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-10T18:36:04.224 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:36:04.224 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repository Size 2024-09-10T18:36:04.224 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:36:04.224 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-10T18:36:04.224 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-test x86_64 2:17.2.7-1606.g5be57395.el9 ceph 48 M 2024-09-10T18:36:04.224 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2024-09-10T18:36:04.224 INFO:teuthology.orchestra.run.smithi103.stdout: libcephsqlite x86_64 2:17.2.7-1606.g5be57395.el9 ceph 163 k 2024-09-10T18:36:04.224 INFO:teuthology.orchestra.run.smithi103.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T18:36:04.224 INFO:teuthology.orchestra.run.smithi103.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T18:36:04.224 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:04.225 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-10T18:36:04.225 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:36:04.225 INFO:teuthology.orchestra.run.smithi103.stdout:Install 4 Packages 2024-09-10T18:36:04.225 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:04.225 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 48 M 2024-09-10T18:36:04.225 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 193 M 2024-09-10T18:36:04.225 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-10T18:36:04.575 INFO:teuthology.orchestra.run.smithi103.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.3 MB/s | 304 kB 00:00 2024-09-10T18:36:04.600 INFO:teuthology.orchestra.run.smithi103.stdout:(2/4): libcephsqlite-17.2.7-1606.g5be57395.el9. 657 kB/s | 163 kB 00:00 2024-09-10T18:36:04.605 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install ceph-test 2024-09-10T18:36:04.633 INFO:teuthology.orchestra.run.smithi103.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-10T18:36:05.200 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:00:28 ago on Tue 10 Sep 2024 06:35:37 PM UTC. 2024-09-10T18:36:05.345 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-09-10T18:36:05.346 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:36:05.346 INFO:teuthology.orchestra.run.smithi018.stdout: Package Arch Version Repository Size 2024-09-10T18:36:05.346 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:36:05.346 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-09-10T18:36:05.346 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-test x86_64 2:17.2.7-1606.g5be57395.el9 ceph 48 M 2024-09-10T18:36:05.346 INFO:teuthology.orchestra.run.smithi018.stdout:Installing dependencies: 2024-09-10T18:36:05.347 INFO:teuthology.orchestra.run.smithi018.stdout: libcephsqlite x86_64 2:17.2.7-1606.g5be57395.el9 ceph 163 k 2024-09-10T18:36:05.347 INFO:teuthology.orchestra.run.smithi018.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T18:36:05.347 INFO:teuthology.orchestra.run.smithi018.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T18:36:05.347 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:05.347 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-09-10T18:36:05.347 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:36:05.347 INFO:teuthology.orchestra.run.smithi018.stdout:Install 4 Packages 2024-09-10T18:36:05.347 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:05.348 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 48 M 2024-09-10T18:36:05.348 INFO:teuthology.orchestra.run.smithi018.stdout:Installed size: 193 M 2024-09-10T18:36:05.348 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-09-10T18:36:05.661 INFO:teuthology.orchestra.run.smithi018.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.2 MB/s | 304 kB 00:00 2024-09-10T18:36:05.686 INFO:teuthology.orchestra.run.smithi018.stdout:(2/4): libcephsqlite-17.2.7-1606.g5be57395.el9. 596 kB/s | 163 kB 00:00 2024-09-10T18:36:05.711 INFO:teuthology.orchestra.run.smithi018.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-10T18:36:05.884 INFO:teuthology.orchestra.run.smithi103.stdout:(4/4): ceph-test-17.2.7-1606.g5be57395.el9.x86_ 31 MB/s | 48 MB 00:01 2024-09-10T18:36:05.886 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:36:05.886 INFO:teuthology.orchestra.run.smithi103.stdout:Total 29 MB/s | 48 MB 00:01 2024-09-10T18:36:05.914 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-10T18:36:05.932 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-10T18:36:05.933 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-10T18:36:06.297 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-10T18:36:06.297 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-10T18:36:06.823 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-10T18:36:06.913 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-10T18:36:06.958 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-10T18:36:06.986 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 3/4 2024-09-10T18:36:07.712 INFO:teuthology.orchestra.run.smithi018.stdout:(4/4): ceph-test-17.2.7-1606.g5be57395.el9.x86_ 21 MB/s | 48 MB 00:02 2024-09-10T18:36:07.714 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:36:07.714 INFO:teuthology.orchestra.run.smithi018.stdout:Total 20 MB/s | 48 MB 00:02 2024-09-10T18:36:07.742 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-09-10T18:36:07.763 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-09-10T18:36:07.763 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-09-10T18:36:08.149 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-09-10T18:36:08.150 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-09-10T18:36:08.689 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-09-10T18:36:08.783 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-10T18:36:08.828 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-10T18:36:08.864 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 3/4 2024-09-10T18:36:11.382 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 3/4 2024-09-10T18:36:11.455 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 4/4 2024-09-10T18:36:13.528 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 3/4 2024-09-10T18:36:13.606 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 4/4 2024-09-10T18:36:13.747 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 4/4 2024-09-10T18:36:13.747 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 1/4 2024-09-10T18:36:13.747 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 2/4 2024-09-10T18:36:13.748 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-10T18:36:14.061 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-10T18:36:14.061 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:14.061 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-10T18:36:14.061 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:14.061 INFO:teuthology.orchestra.run.smithi103.stdout: libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:14.062 INFO:teuthology.orchestra.run.smithi103.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T18:36:14.062 INFO:teuthology.orchestra.run.smithi103.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T18:36:14.062 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:14.062 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-10T18:36:14.349 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph 2024-09-10T18:36:14.919 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:00:38 ago on Tue 10 Sep 2024 06:35:36 PM UTC. 2024-09-10T18:36:15.058 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-10T18:36:15.061 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:36:15.061 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repository Size 2024-09-10T18:36:15.061 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:36:15.061 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-10T18:36:15.061 INFO:teuthology.orchestra.run.smithi103.stdout: ceph x86_64 2:17.2.7-1606.g5be57395.el9 ceph 6.5 k 2024-09-10T18:36:15.061 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2024-09-10T18:36:15.061 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mds x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.1 M 2024-09-10T18:36:15.061 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr x86_64 2:17.2.7-1606.g5be57395.el9 ceph 1.4 M 2024-09-10T18:36:15.061 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 240 k 2024-09-10T18:36:15.062 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mon x86_64 2:17.2.7-1606.g5be57395.el9 ceph 3.9 M 2024-09-10T18:36:15.062 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-osd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 15 M 2024-09-10T18:36:15.062 INFO:teuthology.orchestra.run.smithi103.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T18:36:15.062 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T18:36:15.062 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T18:36:15.062 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T18:36:15.062 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T18:36:15.062 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T18:36:15.062 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T18:36:15.062 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-collections 2024-09-10T18:36:15.062 INFO:teuthology.orchestra.run.smithi103.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-10T18:36:15.062 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T18:36:15.062 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T18:36:15.063 INFO:teuthology.orchestra.run.smithi103.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T18:36:15.063 INFO:teuthology.orchestra.run.smithi103.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T18:36:15.063 INFO:teuthology.orchestra.run.smithi103.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T18:36:15.063 INFO:teuthology.orchestra.run.smithi103.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T18:36:15.063 INFO:teuthology.orchestra.run.smithi103.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T18:36:15.063 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T18:36:15.063 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T18:36:15.063 INFO:teuthology.orchestra.run.smithi103.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T18:36:15.063 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T18:36:15.063 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T18:36:15.063 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T18:36:15.063 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T18:36:15.063 INFO:teuthology.orchestra.run.smithi103.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T18:36:15.063 INFO:teuthology.orchestra.run.smithi103.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T18:36:15.064 INFO:teuthology.orchestra.run.smithi103.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T18:36:15.064 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T18:36:15.064 INFO:teuthology.orchestra.run.smithi103.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T18:36:15.064 INFO:teuthology.orchestra.run.smithi103.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T18:36:15.064 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:15.064 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-10T18:36:15.064 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:36:15.064 INFO:teuthology.orchestra.run.smithi103.stdout:Install 34 Packages 2024-09-10T18:36:15.064 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:15.065 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 27 M 2024-09-10T18:36:15.065 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 96 M 2024-09-10T18:36:15.065 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-10T18:36:15.815 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 4/4 2024-09-10T18:36:15.815 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 1/4 2024-09-10T18:36:15.815 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 2/4 2024-09-10T18:36:15.815 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-10T18:36:15.916 INFO:teuthology.orchestra.run.smithi103.stdout:(1/34): ceph-17.2.7-1606.g5be57395.el9.x86_64.r 42 kB/s | 6.5 kB 00:00 2024-09-10T18:36:16.116 INFO:teuthology.orchestra.run.smithi103.stdout:(2/34): ceph-mgr-17.2.7-1606.g5be57395.el9.x86_ 4.0 MB/s | 1.4 MB 00:00 2024-09-10T18:36:16.143 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-10T18:36:16.143 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:16.144 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-09-10T18:36:16.144 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:16.144 INFO:teuthology.orchestra.run.smithi018.stdout: libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:16.144 INFO:teuthology.orchestra.run.smithi018.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T18:36:16.144 INFO:teuthology.orchestra.run.smithi018.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T18:36:16.144 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:16.144 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-09-10T18:36:16.192 INFO:teuthology.orchestra.run.smithi103.stdout:(3/34): ceph-mds-17.2.7-1606.g5be57395.el9.x86_ 4.8 MB/s | 2.1 MB 00:00 2024-09-10T18:36:16.267 INFO:teuthology.orchestra.run.smithi103.stdout:(4/34): ceph-mon-17.2.7-1606.g5be57395.el9.x86_ 11 MB/s | 3.9 MB 00:00 2024-09-10T18:36:16.292 INFO:teuthology.orchestra.run.smithi103.stdout:(5/34): ceph-mgr-modules-core-17.2.7-1606.g5be5 2.3 MB/s | 240 kB 00:00 2024-09-10T18:36:16.418 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install ceph 2024-09-10T18:36:16.443 INFO:teuthology.orchestra.run.smithi103.stdout:(6/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.4 MB/s | 253 kB 00:00 2024-09-10T18:36:16.493 INFO:teuthology.orchestra.run.smithi103.stdout:(7/34): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-10T18:36:16.677 INFO:teuthology.orchestra.run.smithi103.stdout:(8/34): ceph-osd-17.2.7-1606.g5be57395.el9.x86_ 27 MB/s | 15 MB 00:00 2024-09-10T18:36:16.734 INFO:teuthology.orchestra.run.smithi103.stdout:(9/34): python3-cryptography-36.0.1-4.el9.x86_6 2.8 MB/s | 1.2 MB 00:00 2024-09-10T18:36:16.760 INFO:teuthology.orchestra.run.smithi103.stdout:(10/34): python3-pycparser-2.20-6.el9.noarch.rp 506 kB/s | 135 kB 00:00 2024-09-10T18:36:16.844 INFO:teuthology.orchestra.run.smithi103.stdout:(11/34): python3-urllib3-1.26.5-6.el9.noarch.rp 2.5 MB/s | 215 kB 00:00 2024-09-10T18:36:16.869 INFO:teuthology.orchestra.run.smithi103.stdout:(12/34): python3-requests-2.25.1-8.el9.noarch.r 649 kB/s | 125 kB 00:00 2024-09-10T18:36:16.903 INFO:teuthology.orchestra.run.smithi103.stdout:(13/34): python3-mako-1.1.4-6.el9.noarch.rpm 1.2 MB/s | 172 kB 00:00 2024-09-10T18:36:16.937 INFO:teuthology.orchestra.run.smithi103.stdout:(14/34): python3-markupsafe-1.1.1-12.el9.x86_64 376 kB/s | 35 kB 00:00 2024-09-10T18:36:16.962 INFO:teuthology.orchestra.run.smithi103.stdout:(15/34): python3-pytz-2021.1-5.el9.noarch.rpm 870 kB/s | 51 kB 00:00 2024-09-10T18:36:16.997 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:00:39 ago on Tue 10 Sep 2024 06:35:37 PM UTC. 2024-09-10T18:36:17.004 INFO:teuthology.orchestra.run.smithi103.stdout:(16/34): python3-toml-0.10.2-6.el9.noarch.rpm 624 kB/s | 42 kB 00:00 2024-09-10T18:36:17.029 INFO:teuthology.orchestra.run.smithi103.stdout:(17/34): python3-packaging-20.9-5.el9.noarch.rp 484 kB/s | 77 kB 00:00 2024-09-10T18:36:17.063 INFO:teuthology.orchestra.run.smithi103.stdout:(18/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 433 kB/s | 43 kB 00:00 2024-09-10T18:36:17.096 INFO:teuthology.orchestra.run.smithi103.stdout:(19/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 461 kB/s | 11 kB 00:00 2024-09-10T18:36:17.122 INFO:teuthology.orchestra.run.smithi103.stdout:(20/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.5 MB/s | 172 kB 00:00 2024-09-10T18:36:17.139 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-09-10T18:36:17.141 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:36:17.141 INFO:teuthology.orchestra.run.smithi018.stdout: Package Arch Version Repository Size 2024-09-10T18:36:17.141 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:36:17.141 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-09-10T18:36:17.141 INFO:teuthology.orchestra.run.smithi018.stdout: ceph x86_64 2:17.2.7-1606.g5be57395.el9 ceph 6.5 k 2024-09-10T18:36:17.141 INFO:teuthology.orchestra.run.smithi018.stdout:Installing dependencies: 2024-09-10T18:36:17.141 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mds x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.1 M 2024-09-10T18:36:17.141 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr x86_64 2:17.2.7-1606.g5be57395.el9 ceph 1.4 M 2024-09-10T18:36:17.142 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 240 k 2024-09-10T18:36:17.142 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mon x86_64 2:17.2.7-1606.g5be57395.el9 ceph 3.9 M 2024-09-10T18:36:17.142 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-osd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 15 M 2024-09-10T18:36:17.142 INFO:teuthology.orchestra.run.smithi018.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T18:36:17.142 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T18:36:17.142 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T18:36:17.142 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T18:36:17.142 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T18:36:17.142 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T18:36:17.142 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T18:36:17.142 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-collections 2024-09-10T18:36:17.142 INFO:teuthology.orchestra.run.smithi018.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-10T18:36:17.142 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T18:36:17.143 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T18:36:17.143 INFO:teuthology.orchestra.run.smithi018.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T18:36:17.143 INFO:teuthology.orchestra.run.smithi018.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T18:36:17.143 INFO:teuthology.orchestra.run.smithi018.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T18:36:17.143 INFO:teuthology.orchestra.run.smithi018.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T18:36:17.143 INFO:teuthology.orchestra.run.smithi018.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T18:36:17.143 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T18:36:17.143 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T18:36:17.143 INFO:teuthology.orchestra.run.smithi018.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T18:36:17.143 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T18:36:17.143 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T18:36:17.143 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T18:36:17.143 INFO:teuthology.orchestra.run.smithi018.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T18:36:17.144 INFO:teuthology.orchestra.run.smithi018.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T18:36:17.144 INFO:teuthology.orchestra.run.smithi018.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T18:36:17.144 INFO:teuthology.orchestra.run.smithi018.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T18:36:17.144 INFO:teuthology.orchestra.run.smithi018.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T18:36:17.144 INFO:teuthology.orchestra.run.smithi018.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T18:36:17.144 INFO:teuthology.orchestra.run.smithi018.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T18:36:17.144 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:17.144 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-09-10T18:36:17.144 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:36:17.144 INFO:teuthology.orchestra.run.smithi018.stdout:Install 34 Packages 2024-09-10T18:36:17.144 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:17.145 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 27 M 2024-09-10T18:36:17.145 INFO:teuthology.orchestra.run.smithi018.stdout:Installed size: 96 M 2024-09-10T18:36:17.145 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-09-10T18:36:17.146 INFO:teuthology.orchestra.run.smithi103.stdout:(21/34): python3-jaraco-classes-3.2.1-5.el9.noa 371 kB/s | 18 kB 00:00 2024-09-10T18:36:17.180 INFO:teuthology.orchestra.run.smithi103.stdout:(22/34): python3-jaraco-collections-3.0.0-8.el9 405 kB/s | 23 kB 00:00 2024-09-10T18:36:17.205 INFO:teuthology.orchestra.run.smithi103.stdout:(23/34): python3-jaraco-functools-3.5.0-2.el9.n 332 kB/s | 19 kB 00:00 2024-09-10T18:36:17.230 INFO:teuthology.orchestra.run.smithi103.stdout:(24/34): python3-jaraco-text-3.2.0-6.el9.noarch 391 kB/s | 20 kB 00:00 2024-09-10T18:36:17.256 INFO:teuthology.orchestra.run.smithi103.stdout:(25/34): python3-logutils-0.3.5-21.el9.noarch.r 923 kB/s | 46 kB 00:00 2024-09-10T18:36:17.281 INFO:teuthology.orchestra.run.smithi103.stdout:(26/34): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-10T18:36:17.306 INFO:teuthology.orchestra.run.smithi103.stdout:(27/34): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-09-10T18:36:17.332 INFO:teuthology.orchestra.run.smithi103.stdout:(28/34): python3-portend-3.1.0-2.el9.noarch.rpm 328 kB/s | 16 kB 00:00 2024-09-10T18:36:17.357 INFO:teuthology.orchestra.run.smithi103.stdout:(29/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-09-10T18:36:17.382 INFO:teuthology.orchestra.run.smithi103.stdout:(30/34): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-09-10T18:36:17.407 INFO:teuthology.orchestra.run.smithi103.stdout:(31/34): python3-tempora-5.0.0-2.el9.noarch.rpm 474 kB/s | 36 kB 00:00 2024-09-10T18:36:17.441 INFO:teuthology.orchestra.run.smithi103.stdout:(32/34): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-10T18:36:17.466 INFO:teuthology.orchestra.run.smithi103.stdout:(33/34): python3-zc-lockfile-2.0-10.el9.noarch. 342 kB/s | 20 kB 00:00 2024-09-10T18:36:17.491 INFO:teuthology.orchestra.run.smithi103.stdout:(34/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-09-10T18:36:17.496 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:36:17.496 INFO:teuthology.orchestra.run.smithi103.stdout:Total 11 MB/s | 27 MB 00:02 2024-09-10T18:36:17.629 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-10T18:36:17.666 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-10T18:36:17.667 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-10T18:36:17.692 INFO:teuthology.orchestra.run.smithi018.stdout:(1/34): ceph-17.2.7-1606.g5be57395.el9.x86_64.r 44 kB/s | 6.5 kB 00:00 2024-09-10T18:36:17.901 INFO:teuthology.orchestra.run.smithi018.stdout:(2/34): ceph-mgr-17.2.7-1606.g5be57395.el9.x86_ 3.9 MB/s | 1.4 MB 00:00 2024-09-10T18:36:17.951 INFO:teuthology.orchestra.run.smithi018.stdout:(3/34): ceph-mds-17.2.7-1606.g5be57395.el9.x86_ 5.1 MB/s | 2.1 MB 00:00 2024-09-10T18:36:18.030 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-10T18:36:18.030 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-10T18:36:18.043 INFO:teuthology.orchestra.run.smithi018.stdout:(4/34): ceph-mgr-modules-core-17.2.7-1606.g5be5 2.6 MB/s | 240 kB 00:00 2024-09-10T18:36:18.102 INFO:teuthology.orchestra.run.smithi018.stdout:(5/34): ceph-mon-17.2.7-1606.g5be57395.el9.x86_ 9.6 MB/s | 3.9 MB 00:00 2024-09-10T18:36:18.244 INFO:teuthology.orchestra.run.smithi018.stdout:(6/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-09-10T18:36:18.294 INFO:teuthology.orchestra.run.smithi018.stdout:(7/34): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-10T18:36:18.461 INFO:teuthology.orchestra.run.smithi018.stdout:(8/34): ceph-osd-17.2.7-1606.g5be57395.el9.x86_ 27 MB/s | 15 MB 00:00 2024-09-10T18:36:18.486 INFO:teuthology.orchestra.run.smithi018.stdout:(9/34): python3-pycparser-2.20-6.el9.noarch.rpm 704 kB/s | 135 kB 00:00 2024-09-10T18:36:18.520 INFO:teuthology.orchestra.run.smithi018.stdout:(10/34): python3-cryptography-36.0.1-4.el9.x86_ 2.9 MB/s | 1.2 MB 00:00 2024-09-10T18:36:18.545 INFO:teuthology.orchestra.run.smithi018.stdout:(11/34): python3-urllib3-1.26.5-6.el9.noarch.rp 3.6 MB/s | 215 kB 00:00 2024-09-10T18:36:18.612 INFO:teuthology.orchestra.run.smithi018.stdout:(12/34): python3-requests-2.25.1-8.el9.noarch.r 827 kB/s | 125 kB 00:00 2024-09-10T18:36:18.854 INFO:teuthology.orchestra.run.smithi018.stdout:(13/34): python3-markupsafe-1.1.1-12.el9.x86_64 112 kB/s | 35 kB 00:00 2024-09-10T18:36:19.021 INFO:teuthology.orchestra.run.smithi018.stdout:(14/34): python3-packaging-20.9-5.el9.noarch.rp 189 kB/s | 77 kB 00:00 2024-09-10T18:36:19.071 INFO:teuthology.orchestra.run.smithi018.stdout:(15/34): python3-pytz-2021.1-5.el9.noarch.rpm 235 kB/s | 51 kB 00:00 2024-09-10T18:36:19.138 INFO:teuthology.orchestra.run.smithi018.stdout:(16/34): python3-mako-1.1.4-6.el9.noarch.rpm 279 kB/s | 172 kB 00:00 2024-09-10T18:36:19.163 INFO:teuthology.orchestra.run.smithi018.stdout:(17/34): python3-toml-0.10.2-6.el9.noarch.rpm 293 kB/s | 42 kB 00:00 2024-09-10T18:36:19.197 INFO:teuthology.orchestra.run.smithi018.stdout:(18/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 346 kB/s | 43 kB 00:00 2024-09-10T18:36:19.239 INFO:teuthology.orchestra.run.smithi018.stdout:(19/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 325 kB/s | 11 kB 00:00 2024-09-10T18:36:19.259 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-10T18:36:19.289 INFO:teuthology.orchestra.run.smithi018.stdout:(20/34): python3-jaraco-classes-3.2.1-5.el9.noa 367 kB/s | 18 kB 00:00 2024-09-10T18:36:19.323 INFO:teuthology.orchestra.run.smithi018.stdout:(21/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 932 kB/s | 172 kB 00:00 2024-09-10T18:36:19.336 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-10T18:36:19.348 INFO:teuthology.orchestra.run.smithi018.stdout:(22/34): python3-jaraco-collections-3.0.0-8.el9 395 kB/s | 23 kB 00:00 2024-09-10T18:36:19.368 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-10T18:36:19.373 INFO:teuthology.orchestra.run.smithi018.stdout:(23/34): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-09-10T18:36:19.398 INFO:teuthology.orchestra.run.smithi018.stdout:(24/34): python3-jaraco-text-3.2.0-6.el9.noarch 391 kB/s | 20 kB 00:00 2024-09-10T18:36:19.401 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-10T18:36:19.424 INFO:teuthology.orchestra.run.smithi018.stdout:(25/34): python3-logutils-0.3.5-21.el9.noarch.r 919 kB/s | 46 kB 00:00 2024-09-10T18:36:19.457 INFO:teuthology.orchestra.run.smithi018.stdout:(26/34): python3-more-itertools-8.12.0-2.el9.no 1.3 MB/s | 79 kB 00:00 2024-09-10T18:36:19.483 INFO:teuthology.orchestra.run.smithi018.stdout:(27/34): python3-cherrypy-18.6.1-2.el9.noarch.r 1.1 MB/s | 358 kB 00:00 2024-09-10T18:36:19.492 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-10T18:36:19.508 INFO:teuthology.orchestra.run.smithi018.stdout:(28/34): python3-portend-3.1.0-2.el9.noarch.rpm 325 kB/s | 16 kB 00:00 2024-09-10T18:36:19.533 INFO:teuthology.orchestra.run.smithi018.stdout:(29/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-09-10T18:36:19.559 INFO:teuthology.orchestra.run.smithi018.stdout:(30/34): python3-tempora-5.0.0-2.el9.noarch.rpm 708 kB/s | 36 kB 00:00 2024-09-10T18:36:19.572 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-10T18:36:19.586 INFO:teuthology.orchestra.run.smithi018.stdout:(31/34): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-09-10T18:36:19.611 INFO:teuthology.orchestra.run.smithi018.stdout:(32/34): python3-webob-1.8.8-2.el9.noarch.rpm 2.9 MB/s | 230 kB 00:00 2024-09-10T18:36:19.612 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-10T18:36:19.637 INFO:teuthology.orchestra.run.smithi018.stdout:(33/34): python3-zc-lockfile-2.0-10.el9.noarch. 397 kB/s | 20 kB 00:00 2024-09-10T18:36:19.647 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-10T18:36:19.670 INFO:teuthology.orchestra.run.smithi018.stdout:(34/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-09-10T18:36:19.674 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:36:19.674 INFO:teuthology.orchestra.run.smithi018.stdout:Total 11 MB/s | 27 MB 00:02 2024-09-10T18:36:19.686 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-10T18:36:19.729 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-10T18:36:19.771 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-10T18:36:19.801 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-09-10T18:36:19.811 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-10T18:36:19.842 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-09-10T18:36:19.843 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-09-10T18:36:19.846 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-10T18:36:19.881 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-10T18:36:19.929 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-10T18:36:19.964 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-10T18:36:20.043 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-10T18:36:20.198 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-10T18:36:20.236 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-09-10T18:36:20.236 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-09-10T18:36:20.286 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-10T18:36:20.340 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-10T18:36:20.379 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-10T18:36:20.435 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-10T18:36:20.491 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-10T18:36:20.694 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-10T18:36:20.741 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-10T18:36:20.811 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-10T18:36:20.900 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-10T18:36:20.949 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-10T18:36:21.065 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-10T18:36:21.220 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9. 29/34 2024-09-10T18:36:21.252 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 30/34 2024-09-10T18:36:21.278 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 30/34 2024-09-10T18:36:21.278 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:36:21.278 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T18:36:21.278 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T18:36:21.278 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T18:36:21.278 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:21.536 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-09-10T18:36:21.607 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-10T18:36:21.640 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-10T18:36:21.673 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-10T18:36:21.769 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-10T18:36:21.857 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-10T18:36:21.909 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-10T18:36:21.952 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-10T18:36:21.981 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-10T18:36:22.016 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-10T18:36:22.050 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-10T18:36:22.091 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-10T18:36:22.134 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-10T18:36:22.168 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-10T18:36:22.217 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-10T18:36:22.281 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-10T18:36:22.359 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-10T18:36:22.512 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-10T18:36:22.593 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-10T18:36:22.637 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 31/34 2024-09-10T18:36:22.644 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-10T18:36:22.672 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 31/34 2024-09-10T18:36:22.672 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:36:22.672 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T18:36:22.672 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T18:36:22.672 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T18:36:22.672 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:22.682 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-10T18:36:22.749 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-10T18:36:22.830 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-10T18:36:23.027 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-10T18:36:23.063 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 32/34 2024-09-10T18:36:23.069 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-10T18:36:23.088 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 32/34 2024-09-10T18:36:23.088 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:36:23.088 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T18:36:23.088 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T18:36:23.088 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T18:36:23.088 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:23.151 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-10T18:36:23.248 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-10T18:36:23.288 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 33/34 2024-09-10T18:36:23.299 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-10T18:36:23.312 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 33/34 2024-09-10T18:36:23.312 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:36:23.313 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T18:36:23.313 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T18:36:23.313 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T18:36:23.313 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:23.376 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-2:17.2.7-1606.g5be57395.el9.x86_64 34/34 2024-09-10T18:36:23.423 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-10T18:36:23.589 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9. 29/34 2024-09-10T18:36:23.621 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 30/34 2024-09-10T18:36:23.649 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 30/34 2024-09-10T18:36:23.650 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:36:23.650 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T18:36:23.650 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T18:36:23.650 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T18:36:23.650 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:25.039 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 31/34 2024-09-10T18:36:25.068 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 31/34 2024-09-10T18:36:25.068 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:36:25.068 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T18:36:25.068 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T18:36:25.069 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T18:36:25.069 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:25.146 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-2:17.2.7-1606.g5be57395.el9.x86_64 34/34 2024-09-10T18:36:25.146 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-2:17.2.7-1606.g5be57395.el9.x86_64 1/34 2024-09-10T18:36:25.146 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 2/34 2024-09-10T18:36:25.147 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 3/34 2024-09-10T18:36:25.147 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 4/34 2024-09-10T18:36:25.147 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 5/34 2024-09-10T18:36:25.147 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9. 6/34 2024-09-10T18:36:25.147 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-10T18:36:25.147 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-10T18:36:25.147 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-10T18:36:25.147 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-10T18:36:25.147 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-10T18:36:25.147 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-10T18:36:25.147 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-10T18:36:25.147 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-10T18:36:25.147 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-10T18:36:25.148 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-10T18:36:25.148 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-10T18:36:25.148 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-10T18:36:25.148 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-10T18:36:25.148 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-10T18:36:25.148 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-10T18:36:25.148 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-10T18:36:25.148 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-10T18:36:25.148 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-10T18:36:25.148 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-10T18:36:25.148 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-10T18:36:25.148 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-10T18:36:25.149 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-10T18:36:25.149 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-10T18:36:25.149 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-10T18:36:25.149 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-10T18:36:25.150 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-10T18:36:25.150 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-10T18:36:25.465 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 32/34 2024-09-10T18:36:25.490 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 32/34 2024-09-10T18:36:25.490 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:36:25.490 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T18:36:25.490 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T18:36:25.490 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T18:36:25.490 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:25.699 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 33/34 2024-09-10T18:36:25.724 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 33/34 2024-09-10T18:36:25.724 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:36:25.724 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T18:36:25.724 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T18:36:25.724 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T18:36:25.724 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:25.790 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-2:17.2.7-1606.g5be57395.el9.x86_64 34/34 2024-09-10T18:36:26.125 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-10T18:36:26.125 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:26.125 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-10T18:36:26.125 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:26.125 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:26.125 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:26.125 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:36:26.125 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:26.125 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:26.125 INFO:teuthology.orchestra.run.smithi103.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T18:36:26.125 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T18:36:26.126 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T18:36:26.126 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T18:36:26.126 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T18:36:26.126 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T18:36:26.126 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T18:36:26.126 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T18:36:26.126 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T18:36:26.126 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T18:36:26.126 INFO:teuthology.orchestra.run.smithi103.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T18:36:26.126 INFO:teuthology.orchestra.run.smithi103.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T18:36:26.126 INFO:teuthology.orchestra.run.smithi103.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T18:36:26.126 INFO:teuthology.orchestra.run.smithi103.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T18:36:26.126 INFO:teuthology.orchestra.run.smithi103.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T18:36:26.127 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T18:36:26.127 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T18:36:26.127 INFO:teuthology.orchestra.run.smithi103.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T18:36:26.127 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T18:36:26.127 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T18:36:26.127 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T18:36:26.127 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T18:36:26.127 INFO:teuthology.orchestra.run.smithi103.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T18:36:26.127 INFO:teuthology.orchestra.run.smithi103.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T18:36:26.127 INFO:teuthology.orchestra.run.smithi103.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T18:36:26.127 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T18:36:26.127 INFO:teuthology.orchestra.run.smithi103.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T18:36:26.127 INFO:teuthology.orchestra.run.smithi103.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T18:36:26.128 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:26.128 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-10T18:36:26.495 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph-base 2024-09-10T18:36:27.062 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:00:51 ago on Tue 10 Sep 2024 06:35:36 PM UTC. 2024-09-10T18:36:27.151 INFO:teuthology.orchestra.run.smithi103.stdout:Package ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:36:27.193 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-10T18:36:27.193 INFO:teuthology.orchestra.run.smithi103.stdout:Nothing to do. 2024-09-10T18:36:27.194 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-10T18:36:27.255 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install cephadm 2024-09-10T18:36:27.499 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-2:17.2.7-1606.g5be57395.el9.x86_64 34/34 2024-09-10T18:36:27.499 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-2:17.2.7-1606.g5be57395.el9.x86_64 1/34 2024-09-10T18:36:27.499 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 2/34 2024-09-10T18:36:27.500 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 3/34 2024-09-10T18:36:27.500 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 4/34 2024-09-10T18:36:27.500 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 5/34 2024-09-10T18:36:27.500 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9. 6/34 2024-09-10T18:36:27.500 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-10T18:36:27.500 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-10T18:36:27.500 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-10T18:36:27.500 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-10T18:36:27.500 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-10T18:36:27.500 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-10T18:36:27.500 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-10T18:36:27.500 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-10T18:36:27.500 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-10T18:36:27.501 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-10T18:36:27.501 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-10T18:36:27.501 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-10T18:36:27.501 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-10T18:36:27.501 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-10T18:36:27.501 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-10T18:36:27.501 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-10T18:36:27.501 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-10T18:36:27.501 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-10T18:36:27.501 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-10T18:36:27.501 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-10T18:36:27.501 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-10T18:36:27.502 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-10T18:36:27.502 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-10T18:36:27.503 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-10T18:36:27.503 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-10T18:36:27.503 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-10T18:36:27.503 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-10T18:36:27.814 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:00:51 ago on Tue 10 Sep 2024 06:35:36 PM UTC. 2024-09-10T18:36:27.943 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-10T18:36:27.944 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:36:27.944 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repository Size 2024-09-10T18:36:27.944 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:36:27.944 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-10T18:36:27.944 INFO:teuthology.orchestra.run.smithi103.stdout: cephadm noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 83 k 2024-09-10T18:36:27.944 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:27.944 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-10T18:36:27.944 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:36:27.944 INFO:teuthology.orchestra.run.smithi103.stdout:Install 1 Package 2024-09-10T18:36:27.944 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:27.944 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 83 k 2024-09-10T18:36:27.944 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 375 k 2024-09-10T18:36:27.945 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-10T18:36:28.140 INFO:teuthology.orchestra.run.smithi103.stdout:cephadm-17.2.7-1606.g5be57395.el9.noarch.rpm 424 kB/s | 83 kB 00:00 2024-09-10T18:36:28.140 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:36:28.141 INFO:teuthology.orchestra.run.smithi103.stdout:Total 420 kB/s | 83 kB 00:00 2024-09-10T18:36:28.141 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-10T18:36:28.146 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-10T18:36:28.146 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-10T18:36:28.148 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-10T18:36:28.148 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-10T18:36:28.223 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-10T18:36:28.421 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-10T18:36:28.421 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:28.421 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-09-10T18:36:28.421 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:28.421 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:28.421 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:28.421 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:36:28.421 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:28.421 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:28.421 INFO:teuthology.orchestra.run.smithi018.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T18:36:28.422 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T18:36:28.422 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T18:36:28.422 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T18:36:28.422 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T18:36:28.422 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T18:36:28.422 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T18:36:28.422 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T18:36:28.422 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T18:36:28.422 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T18:36:28.422 INFO:teuthology.orchestra.run.smithi018.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T18:36:28.422 INFO:teuthology.orchestra.run.smithi018.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T18:36:28.422 INFO:teuthology.orchestra.run.smithi018.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T18:36:28.422 INFO:teuthology.orchestra.run.smithi018.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T18:36:28.423 INFO:teuthology.orchestra.run.smithi018.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T18:36:28.423 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T18:36:28.423 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T18:36:28.423 INFO:teuthology.orchestra.run.smithi018.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T18:36:28.423 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T18:36:28.423 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T18:36:28.423 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T18:36:28.423 INFO:teuthology.orchestra.run.smithi018.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T18:36:28.423 INFO:teuthology.orchestra.run.smithi018.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T18:36:28.423 INFO:teuthology.orchestra.run.smithi018.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T18:36:28.423 INFO:teuthology.orchestra.run.smithi018.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T18:36:28.423 INFO:teuthology.orchestra.run.smithi018.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T18:36:28.424 INFO:teuthology.orchestra.run.smithi018.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T18:36:28.424 INFO:teuthology.orchestra.run.smithi018.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T18:36:28.424 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:28.424 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-09-10T18:36:28.640 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:36:28.735 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:36:28.820 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install ceph-base 2024-09-10T18:36:29.076 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:36:29.309 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:36:29.309 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:29.309 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-10T18:36:29.309 INFO:teuthology.orchestra.run.smithi103.stdout: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:36:29.310 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:29.310 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-10T18:36:29.413 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:00:52 ago on Tue 10 Sep 2024 06:35:37 PM UTC. 2024-09-10T18:36:29.513 INFO:teuthology.orchestra.run.smithi018.stdout:Package ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:36:29.518 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph-immutable-object-cache 2024-09-10T18:36:29.556 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-09-10T18:36:29.557 INFO:teuthology.orchestra.run.smithi018.stdout:Nothing to do. 2024-09-10T18:36:29.557 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-09-10T18:36:29.619 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install cephadm 2024-09-10T18:36:30.097 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:00:54 ago on Tue 10 Sep 2024 06:35:36 PM UTC. 2024-09-10T18:36:30.199 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:00:53 ago on Tue 10 Sep 2024 06:35:37 PM UTC. 2024-09-10T18:36:30.232 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-10T18:36:30.233 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:36:30.233 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repo Size 2024-09-10T18:36:30.233 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:36:30.233 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-10T18:36:30.233 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1606.g5be57395.el9 ceph 137 k 2024-09-10T18:36:30.233 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:30.233 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-10T18:36:30.233 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:36:30.233 INFO:teuthology.orchestra.run.smithi103.stdout:Install 1 Package 2024-09-10T18:36:30.233 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:30.234 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 137 k 2024-09-10T18:36:30.234 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 379 k 2024-09-10T18:36:30.234 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-10T18:36:30.336 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-09-10T18:36:30.336 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:36:30.337 INFO:teuthology.orchestra.run.smithi018.stdout: Package Arch Version Repository Size 2024-09-10T18:36:30.337 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:36:30.337 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-09-10T18:36:30.337 INFO:teuthology.orchestra.run.smithi018.stdout: cephadm noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 83 k 2024-09-10T18:36:30.337 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:30.337 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-09-10T18:36:30.337 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:36:30.337 INFO:teuthology.orchestra.run.smithi018.stdout:Install 1 Package 2024-09-10T18:36:30.337 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:30.337 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 83 k 2024-09-10T18:36:30.337 INFO:teuthology.orchestra.run.smithi018.stdout:Installed size: 375 k 2024-09-10T18:36:30.337 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-09-10T18:36:30.424 INFO:teuthology.orchestra.run.smithi103.stdout:ceph-immutable-object-cache-17.2.7-1606.g5be573 725 kB/s | 137 kB 00:00 2024-09-10T18:36:30.424 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:36:30.425 INFO:teuthology.orchestra.run.smithi103.stdout:Total 719 kB/s | 137 kB 00:00 2024-09-10T18:36:30.425 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-10T18:36:30.432 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-10T18:36:30.432 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-10T18:36:30.494 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-10T18:36:30.494 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-10T18:36:30.524 INFO:teuthology.orchestra.run.smithi018.stdout:cephadm-17.2.7-1606.g5be57395.el9.noarch.rpm 443 kB/s | 83 kB 00:00 2024-09-10T18:36:30.525 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:36:30.525 INFO:teuthology.orchestra.run.smithi018.stdout:Total 439 kB/s | 83 kB 00:00 2024-09-10T18:36:30.525 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-09-10T18:36:30.531 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-09-10T18:36:30.531 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-09-10T18:36:30.533 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-09-10T18:36:30.533 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-09-10T18:36:30.615 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-09-10T18:36:30.650 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-10T18:36:30.716 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1606.g5be57395. 1/1 2024-09-10T18:36:30.741 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1606.g5be57395. 1/1 2024-09-10T18:36:30.741 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:36:30.741 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T18:36:30.741 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:31.058 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:36:31.173 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:36:31.534 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:36:31.693 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1606.g5be57395. 1/1 2024-09-10T18:36:31.693 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:31.693 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-10T18:36:31.693 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-immutable-object-cache-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:31.693 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:31.693 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-10T18:36:31.792 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:36:31.793 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:31.793 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-09-10T18:36:31.793 INFO:teuthology.orchestra.run.smithi018.stdout: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:36:31.793 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:31.793 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-09-10T18:36:31.902 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph-mgr 2024-09-10T18:36:32.009 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install ceph-immutable-object-cache 2024-09-10T18:36:32.471 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:00:56 ago on Tue 10 Sep 2024 06:35:36 PM UTC. 2024-09-10T18:36:32.563 INFO:teuthology.orchestra.run.smithi103.stdout:Package ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:36:32.600 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:00:55 ago on Tue 10 Sep 2024 06:35:37 PM UTC. 2024-09-10T18:36:32.604 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-10T18:36:32.605 INFO:teuthology.orchestra.run.smithi103.stdout:Nothing to do. 2024-09-10T18:36:32.606 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-10T18:36:32.675 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph-mgr-dashboard 2024-09-10T18:36:32.738 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-09-10T18:36:32.739 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:36:32.739 INFO:teuthology.orchestra.run.smithi018.stdout: Package Arch Version Repo Size 2024-09-10T18:36:32.739 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:36:32.739 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-09-10T18:36:32.739 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1606.g5be57395.el9 ceph 137 k 2024-09-10T18:36:32.739 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:32.739 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-09-10T18:36:32.739 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:36:32.739 INFO:teuthology.orchestra.run.smithi018.stdout:Install 1 Package 2024-09-10T18:36:32.739 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:32.739 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 137 k 2024-09-10T18:36:32.739 INFO:teuthology.orchestra.run.smithi018.stdout:Installed size: 379 k 2024-09-10T18:36:32.740 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-09-10T18:36:32.961 INFO:teuthology.orchestra.run.smithi018.stdout:ceph-immutable-object-cache-17.2.7-1606.g5be573 622 kB/s | 137 kB 00:00 2024-09-10T18:36:32.961 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:36:32.961 INFO:teuthology.orchestra.run.smithi018.stdout:Total 618 kB/s | 137 kB 00:00 2024-09-10T18:36:32.962 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-09-10T18:36:32.969 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-09-10T18:36:32.970 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-09-10T18:36:33.035 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-09-10T18:36:33.035 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-09-10T18:36:33.217 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-09-10T18:36:33.247 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:00:57 ago on Tue 10 Sep 2024 06:35:36 PM UTC. 2024-09-10T18:36:33.274 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1606.g5be57395. 1/1 2024-09-10T18:36:33.301 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1606.g5be57395. 1/1 2024-09-10T18:36:33.301 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:36:33.301 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T18:36:33.301 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:33.382 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-10T18:36:33.383 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:36:33.383 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repository Size 2024-09-10T18:36:33.383 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:36:33.383 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-10T18:36:33.383 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 3.4 M 2024-09-10T18:36:33.383 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2024-09-10T18:36:33.383 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 24 k 2024-09-10T18:36:33.384 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 15 k 2024-09-10T18:36:33.384 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-10T18:36:33.384 INFO:teuthology.orchestra.run.smithi103.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T18:36:33.384 INFO:teuthology.orchestra.run.smithi103.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T18:36:33.384 INFO:teuthology.orchestra.run.smithi103.stdout:Installing weak dependencies: 2024-09-10T18:36:33.384 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-10T18:36:33.384 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:33.384 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-10T18:36:33.384 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:36:33.384 INFO:teuthology.orchestra.run.smithi103.stdout:Install 7 Packages 2024-09-10T18:36:33.384 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:33.385 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 3.7 M 2024-09-10T18:36:33.385 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 80 M 2024-09-10T18:36:33.385 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-10T18:36:33.687 INFO:teuthology.orchestra.run.smithi103.stdout:(1/7): ceph-grafana-dashboards-17.2.7-1606.g5be 165 kB/s | 24 kB 00:00 2024-09-10T18:36:33.713 INFO:teuthology.orchestra.run.smithi103.stdout:(2/7): ceph-prometheus-alerts-17.2.7-1606.g5be5 84 kB/s | 15 kB 00:00 2024-09-10T18:36:33.754 INFO:teuthology.orchestra.run.smithi103.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 136 kB/s | 9.0 kB 00:00 2024-09-10T18:36:33.780 INFO:teuthology.orchestra.run.smithi103.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-10T18:36:33.813 INFO:teuthology.orchestra.run.smithi103.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 408 kB/s | 41 kB 00:00 2024-09-10T18:36:33.847 INFO:teuthology.orchestra.run.smithi103.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 2.8 MB/s | 188 kB 00:00 2024-09-10T18:36:33.939 INFO:teuthology.orchestra.run.smithi103.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1606.g5be57395 8.4 MB/s | 3.4 MB 00:00 2024-09-10T18:36:33.943 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:36:33.943 INFO:teuthology.orchestra.run.smithi103.stdout:Total 6.5 MB/s | 3.7 MB 00:00 2024-09-10T18:36:33.943 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-10T18:36:33.951 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-10T18:36:33.952 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-10T18:36:34.042 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-10T18:36:34.043 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-10T18:36:34.202 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1606.g5be57395. 1/1 2024-09-10T18:36:34.202 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:34.202 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-09-10T18:36:34.202 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-immutable-object-cache-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:36:34.202 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:34.202 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-09-10T18:36:34.370 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-10T18:36:34.434 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install ceph-mgr 2024-09-10T18:36:34.440 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-10T18:36:34.474 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-10T18:36:34.529 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-10T18:36:34.570 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-10T18:36:34.599 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.n 5/7 2024-09-10T18:36:35.022 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:00:58 ago on Tue 10 Sep 2024 06:35:37 PM UTC. 2024-09-10T18:36:35.116 INFO:teuthology.orchestra.run.smithi018.stdout:Package ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:36:35.161 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-09-10T18:36:35.162 INFO:teuthology.orchestra.run.smithi018.stdout:Nothing to do. 2024-09-10T18:36:35.162 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-09-10T18:36:35.228 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install ceph-mgr-dashboard 2024-09-10T18:36:35.601 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9. 6/7 2024-09-10T18:36:35.645 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarc 7/7 2024-09-10T18:36:35.813 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:00:58 ago on Tue 10 Sep 2024 06:35:37 PM UTC. 2024-09-10T18:36:35.951 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-09-10T18:36:35.952 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:36:35.952 INFO:teuthology.orchestra.run.smithi018.stdout: Package Arch Version Repository Size 2024-09-10T18:36:35.952 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:36:35.952 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-09-10T18:36:35.952 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 3.4 M 2024-09-10T18:36:35.952 INFO:teuthology.orchestra.run.smithi018.stdout:Installing dependencies: 2024-09-10T18:36:35.953 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 24 k 2024-09-10T18:36:35.953 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 15 k 2024-09-10T18:36:35.953 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-10T18:36:35.953 INFO:teuthology.orchestra.run.smithi018.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T18:36:35.953 INFO:teuthology.orchestra.run.smithi018.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T18:36:35.953 INFO:teuthology.orchestra.run.smithi018.stdout:Installing weak dependencies: 2024-09-10T18:36:35.953 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-10T18:36:35.953 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:35.953 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-09-10T18:36:35.953 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:36:35.953 INFO:teuthology.orchestra.run.smithi018.stdout:Install 7 Packages 2024-09-10T18:36:35.953 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:35.954 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 3.7 M 2024-09-10T18:36:35.954 INFO:teuthology.orchestra.run.smithi018.stdout:Installed size: 80 M 2024-09-10T18:36:35.954 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-09-10T18:36:36.136 INFO:teuthology.orchestra.run.smithi018.stdout:(1/7): ceph-prometheus-alerts-17.2.7-1606.g5be5 98 kB/s | 15 kB 00:00 2024-09-10T18:36:36.161 INFO:teuthology.orchestra.run.smithi018.stdout:(2/7): ceph-grafana-dashboards-17.2.7-1606.g5be 140 kB/s | 24 kB 00:00 2024-09-10T18:36:36.203 INFO:teuthology.orchestra.run.smithi018.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 136 kB/s | 9.0 kB 00:00 2024-09-10T18:36:36.228 INFO:teuthology.orchestra.run.smithi018.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-10T18:36:36.253 INFO:teuthology.orchestra.run.smithi018.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 442 kB/s | 41 kB 00:00 2024-09-10T18:36:36.279 INFO:teuthology.orchestra.run.smithi018.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-10T18:36:36.387 INFO:teuthology.orchestra.run.smithi018.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1606.g5be57395 8.4 MB/s | 3.4 MB 00:00 2024-09-10T18:36:36.391 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:36:36.391 INFO:teuthology.orchestra.run.smithi018.stdout:Total 8.3 MB/s | 3.7 MB 00:00 2024-09-10T18:36:36.391 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-09-10T18:36:36.400 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-09-10T18:36:36.400 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-09-10T18:36:36.492 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-09-10T18:36:36.493 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-09-10T18:36:36.733 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarc 7/7 2024-09-10T18:36:36.733 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9. 1/7 2024-09-10T18:36:36.733 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarc 2/7 2024-09-10T18:36:36.733 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.n 3/7 2024-09-10T18:36:36.733 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-10T18:36:36.733 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-10T18:36:36.733 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-10T18:36:36.824 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-09-10T18:36:36.880 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-10T18:36:36.924 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-10T18:36:36.979 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-10T18:36:37.009 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-10T18:36:37.009 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:37.009 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-10T18:36:37.009 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:36:37.009 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:36:37.009 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:36:37.009 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-10T18:36:37.010 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-10T18:36:37.010 INFO:teuthology.orchestra.run.smithi103.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T18:36:37.010 INFO:teuthology.orchestra.run.smithi103.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T18:36:37.010 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:37.010 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-10T18:36:37.018 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-10T18:36:37.047 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.n 5/7 2024-09-10T18:36:37.227 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-10T18:36:37.785 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:01:01 ago on Tue 10 Sep 2024 06:35:36 PM UTC. 2024-09-10T18:36:37.917 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-10T18:36:37.918 INFO:teuthology.orchestra.run.smithi103.stdout:======================================================================================= 2024-09-10T18:36:37.918 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repository Size 2024-09-10T18:36:37.919 INFO:teuthology.orchestra.run.smithi103.stdout:======================================================================================= 2024-09-10T18:36:37.919 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-10T18:36:37.919 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 7.4 M 2024-09-10T18:36:37.919 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2024-09-10T18:36:37.919 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T18:36:37.919 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T18:36:37.919 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T18:36:37.919 INFO:teuthology.orchestra.run.smithi103.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T18:36:37.919 INFO:teuthology.orchestra.run.smithi103.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T18:36:37.919 INFO:teuthology.orchestra.run.smithi103.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T18:36:37.919 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T18:36:37.920 INFO:teuthology.orchestra.run.smithi103.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T18:36:37.920 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T18:36:37.920 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T18:36:37.920 INFO:teuthology.orchestra.run.smithi103.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T18:36:37.920 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:37.920 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-10T18:36:37.920 INFO:teuthology.orchestra.run.smithi103.stdout:======================================================================================= 2024-09-10T18:36:37.920 INFO:teuthology.orchestra.run.smithi103.stdout:Install 12 Packages 2024-09-10T18:36:37.920 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:37.921 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 38 M 2024-09-10T18:36:37.921 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 212 M 2024-09-10T18:36:37.921 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-10T18:36:38.099 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9. 6/7 2024-09-10T18:36:38.134 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarc 7/7 2024-09-10T18:36:38.357 INFO:teuthology.orchestra.run.smithi103.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 893 kB/s | 191 kB 00:00 2024-09-10T18:36:38.432 INFO:teuthology.orchestra.run.smithi103.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.7 MB/s | 801 kB 00:00 2024-09-10T18:36:38.624 INFO:teuthology.orchestra.run.smithi103.stdout:(3/12): ceph-mgr-diskprediction-local-17.2.7-16 15 MB/s | 7.4 MB 00:00 2024-09-10T18:36:38.649 INFO:teuthology.orchestra.run.smithi103.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 112 kB/s | 33 kB 00:00 2024-09-10T18:36:38.883 INFO:teuthology.orchestra.run.smithi103.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 165 kB/s | 38 kB 00:00 2024-09-10T18:36:38.941 INFO:teuthology.orchestra.run.smithi103.stdout:(6/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 57 kB/s | 18 kB 00:00 2024-09-10T18:36:39.233 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarc 7/7 2024-09-10T18:36:39.233 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9. 1/7 2024-09-10T18:36:39.233 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarc 2/7 2024-09-10T18:36:39.233 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.n 3/7 2024-09-10T18:36:39.233 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-10T18:36:39.233 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-10T18:36:39.233 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-10T18:36:39.442 INFO:teuthology.orchestra.run.smithi103.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 489 kB/s | 245 kB 00:00 2024-09-10T18:36:39.533 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-10T18:36:39.533 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:39.533 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-09-10T18:36:39.533 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:36:39.533 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:36:39.534 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:36:39.534 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-10T18:36:39.534 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-10T18:36:39.534 INFO:teuthology.orchestra.run.smithi018.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T18:36:39.534 INFO:teuthology.orchestra.run.smithi018.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T18:36:39.534 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:39.534 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-09-10T18:36:39.634 INFO:teuthology.orchestra.run.smithi103.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.5 MB/s | 3.0 MB 00:01 2024-09-10T18:36:39.759 INFO:teuthology.orchestra.run.smithi103.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.0 MB/s | 250 kB 00:00 2024-09-10T18:36:39.764 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-10T18:36:40.226 INFO:teuthology.orchestra.run.smithi103.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.5 MB/s | 5.1 MB 00:00 2024-09-10T18:36:40.354 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:01:03 ago on Tue 10 Sep 2024 06:35:37 PM UTC. 2024-09-10T18:36:40.493 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-09-10T18:36:40.495 INFO:teuthology.orchestra.run.smithi018.stdout:======================================================================================= 2024-09-10T18:36:40.495 INFO:teuthology.orchestra.run.smithi018.stdout: Package Arch Version Repository Size 2024-09-10T18:36:40.495 INFO:teuthology.orchestra.run.smithi018.stdout:======================================================================================= 2024-09-10T18:36:40.495 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-09-10T18:36:40.495 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 7.4 M 2024-09-10T18:36:40.495 INFO:teuthology.orchestra.run.smithi018.stdout:Installing dependencies: 2024-09-10T18:36:40.495 INFO:teuthology.orchestra.run.smithi018.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T18:36:40.495 INFO:teuthology.orchestra.run.smithi018.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T18:36:40.495 INFO:teuthology.orchestra.run.smithi018.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T18:36:40.495 INFO:teuthology.orchestra.run.smithi018.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T18:36:40.496 INFO:teuthology.orchestra.run.smithi018.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T18:36:40.496 INFO:teuthology.orchestra.run.smithi018.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T18:36:40.496 INFO:teuthology.orchestra.run.smithi018.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T18:36:40.496 INFO:teuthology.orchestra.run.smithi018.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T18:36:40.496 INFO:teuthology.orchestra.run.smithi018.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T18:36:40.496 INFO:teuthology.orchestra.run.smithi018.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T18:36:40.496 INFO:teuthology.orchestra.run.smithi018.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T18:36:40.496 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:40.496 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-09-10T18:36:40.496 INFO:teuthology.orchestra.run.smithi018.stdout:======================================================================================= 2024-09-10T18:36:40.496 INFO:teuthology.orchestra.run.smithi018.stdout:Install 12 Packages 2024-09-10T18:36:40.496 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:40.497 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 38 M 2024-09-10T18:36:40.497 INFO:teuthology.orchestra.run.smithi018.stdout:Installed size: 212 M 2024-09-10T18:36:40.497 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-09-10T18:36:40.713 INFO:teuthology.orchestra.run.smithi018.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 191 kB 00:00 2024-09-10T18:36:40.746 INFO:teuthology.orchestra.run.smithi018.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 3.9 MB/s | 801 kB 00:00 2024-09-10T18:36:40.751 INFO:teuthology.orchestra.run.smithi103.stdout:(11/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 2.6 MB/s | 4.9 MB 00:01 2024-09-10T18:36:40.871 INFO:teuthology.orchestra.run.smithi018.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 206 kB/s | 33 kB 00:00 2024-09-10T18:36:40.988 INFO:teuthology.orchestra.run.smithi018.stdout:(4/12): ceph-mgr-diskprediction-local-17.2.7-16 17 MB/s | 7.4 MB 00:00 2024-09-10T18:36:41.013 INFO:teuthology.orchestra.run.smithi018.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 128 kB/s | 18 kB 00:00 2024-09-10T18:36:41.039 INFO:teuthology.orchestra.run.smithi018.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 767 kB/s | 38 kB 00:00 2024-09-10T18:36:41.131 INFO:teuthology.orchestra.run.smithi018.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 2.6 MB/s | 245 kB 00:00 2024-09-10T18:36:41.281 INFO:teuthology.orchestra.run.smithi018.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 5.6 MB/s | 3.0 MB 00:00 2024-09-10T18:36:41.348 INFO:teuthology.orchestra.run.smithi018.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 3.7 MB/s | 250 kB 00:00 2024-09-10T18:36:42.440 INFO:teuthology.orchestra.run.smithi018.stdout:(10/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 15 MB/s | 16 MB 00:01 2024-09-10T18:36:42.523 INFO:teuthology.orchestra.run.smithi018.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.7 MB/s | 5.1 MB 00:01 2024-09-10T18:36:42.590 INFO:teuthology.orchestra.run.smithi018.stdout:(12/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 3.1 MB/s | 4.9 MB 00:01 2024-09-10T18:36:42.591 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:36:42.592 INFO:teuthology.orchestra.run.smithi018.stdout:Total 18 MB/s | 38 MB 00:02 2024-09-10T18:36:42.958 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-09-10T18:36:42.985 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-09-10T18:36:42.985 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-09-10T18:36:43.110 INFO:teuthology.orchestra.run.smithi103.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 4.9 MB/s | 16 MB 00:03 2024-09-10T18:36:43.111 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:36:43.111 INFO:teuthology.orchestra.run.smithi103.stdout:Total 7.4 MB/s | 38 MB 00:05 2024-09-10T18:36:43.468 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-10T18:36:43.495 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-10T18:36:43.495 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-10T18:36:43.525 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-09-10T18:36:43.526 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-09-10T18:36:43.991 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-10T18:36:43.992 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-10T18:36:44.310 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-09-10T18:36:44.453 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-10T18:36:44.502 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-10T18:36:44.609 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-10T18:36:44.649 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-10T18:36:44.746 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-10T18:36:44.823 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-10T18:36:44.842 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-10T18:36:44.872 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-10T18:36:44.877 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-10T18:36:44.965 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-10T18:36:44.982 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-10T18:36:45.026 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-10T18:36:45.212 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-10T18:36:45.250 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-10T18:36:45.327 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-10T18:36:45.659 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-10T18:36:45.773 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-10T18:36:45.979 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-10T18:36:46.097 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-10T18:36:47.119 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T18:36:47.342 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T18:36:48.210 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-10T18:36:48.247 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57 12/12 2024-09-10T18:36:48.381 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-10T18:36:48.410 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57 12/12 2024-09-10T18:36:50.870 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57 12/12 2024-09-10T18:36:50.870 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57 1/12 2024-09-10T18:36:50.870 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-10T18:36:50.870 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-10T18:36:50.870 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-10T18:36:50.870 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-10T18:36:50.870 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-10T18:36:50.870 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-10T18:36:50.870 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-10T18:36:50.870 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-10T18:36:50.871 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T18:36:50.871 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-10T18:36:51.127 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57 12/12 2024-09-10T18:36:51.127 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57 1/12 2024-09-10T18:36:51.127 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-10T18:36:51.127 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-10T18:36:51.127 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-10T18:36:51.127 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-10T18:36:51.127 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-10T18:36:51.127 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-10T18:36:51.127 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-10T18:36:51.127 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-10T18:36:51.128 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T18:36:51.128 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-10T18:36:51.250 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-10T18:36:51.250 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:51.250 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-09-10T18:36:51.250 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:36:51.250 INFO:teuthology.orchestra.run.smithi018.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T18:36:51.250 INFO:teuthology.orchestra.run.smithi018.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T18:36:51.250 INFO:teuthology.orchestra.run.smithi018.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T18:36:51.250 INFO:teuthology.orchestra.run.smithi018.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T18:36:51.250 INFO:teuthology.orchestra.run.smithi018.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T18:36:51.251 INFO:teuthology.orchestra.run.smithi018.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T18:36:51.251 INFO:teuthology.orchestra.run.smithi018.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T18:36:51.251 INFO:teuthology.orchestra.run.smithi018.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T18:36:51.251 INFO:teuthology.orchestra.run.smithi018.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T18:36:51.251 INFO:teuthology.orchestra.run.smithi018.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T18:36:51.251 INFO:teuthology.orchestra.run.smithi018.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T18:36:51.251 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:51.251 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-09-10T18:36:51.533 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-10T18:36:51.533 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:51.534 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-10T18:36:51.534 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:36:51.534 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T18:36:51.534 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T18:36:51.534 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T18:36:51.534 INFO:teuthology.orchestra.run.smithi103.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T18:36:51.534 INFO:teuthology.orchestra.run.smithi103.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T18:36:51.534 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T18:36:51.534 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T18:36:51.534 INFO:teuthology.orchestra.run.smithi103.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T18:36:51.534 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T18:36:51.534 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T18:36:51.534 INFO:teuthology.orchestra.run.smithi103.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T18:36:51.534 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:51.535 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-10T18:36:51.645 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install ceph-mgr-rook 2024-09-10T18:36:51.905 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph-mgr-rook 2024-09-10T18:36:52.240 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:01:15 ago on Tue 10 Sep 2024 06:35:37 PM UTC. 2024-09-10T18:36:52.379 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-09-10T18:36:52.380 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:36:52.381 INFO:teuthology.orchestra.run.smithi018.stdout: Package Arch Version Repository Size 2024-09-10T18:36:52.381 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:36:52.381 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-09-10T18:36:52.381 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-rook noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 47 k 2024-09-10T18:36:52.381 INFO:teuthology.orchestra.run.smithi018.stdout:Installing dependencies: 2024-09-10T18:36:52.381 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T18:36:52.381 INFO:teuthology.orchestra.run.smithi018.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T18:36:52.381 INFO:teuthology.orchestra.run.smithi018.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T18:36:52.381 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T18:36:52.381 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T18:36:52.381 INFO:teuthology.orchestra.run.smithi018.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T18:36:52.381 INFO:teuthology.orchestra.run.smithi018.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T18:36:52.382 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T18:36:52.382 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T18:36:52.382 INFO:teuthology.orchestra.run.smithi018.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T18:36:52.382 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T18:36:52.382 INFO:teuthology.orchestra.run.smithi018.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T18:36:52.382 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:52.382 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-09-10T18:36:52.382 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:36:52.382 INFO:teuthology.orchestra.run.smithi018.stdout:Install 13 Packages 2024-09-10T18:36:52.382 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:52.383 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 2.2 M 2024-09-10T18:36:52.383 INFO:teuthology.orchestra.run.smithi018.stdout:Installed size: 26 M 2024-09-10T18:36:52.383 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-09-10T18:36:52.498 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:01:16 ago on Tue 10 Sep 2024 06:35:36 PM UTC. 2024-09-10T18:36:52.624 INFO:teuthology.orchestra.run.smithi018.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 389 kB/s | 26 kB 00:00 2024-09-10T18:36:52.639 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-10T18:36:52.640 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:36:52.640 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repository Size 2024-09-10T18:36:52.641 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:36:52.641 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-10T18:36:52.641 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-rook noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 47 k 2024-09-10T18:36:52.641 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2024-09-10T18:36:52.641 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T18:36:52.641 INFO:teuthology.orchestra.run.smithi103.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T18:36:52.641 INFO:teuthology.orchestra.run.smithi103.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T18:36:52.641 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T18:36:52.641 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T18:36:52.641 INFO:teuthology.orchestra.run.smithi103.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T18:36:52.641 INFO:teuthology.orchestra.run.smithi103.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T18:36:52.641 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T18:36:52.641 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T18:36:52.642 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T18:36:52.642 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T18:36:52.642 INFO:teuthology.orchestra.run.smithi103.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T18:36:52.642 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:52.642 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-10T18:36:52.642 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:36:52.642 INFO:teuthology.orchestra.run.smithi103.stdout:Install 13 Packages 2024-09-10T18:36:52.642 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:52.643 INFO:teuthology.orchestra.run.smithi018.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 231 kB/s | 19 kB 00:00 2024-09-10T18:36:52.643 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 2.2 M 2024-09-10T18:36:52.643 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 26 M 2024-09-10T18:36:52.643 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-10T18:36:52.716 INFO:teuthology.orchestra.run.smithi018.stdout:(3/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-09-10T18:36:52.741 INFO:teuthology.orchestra.run.smithi018.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.2 MB/s | 222 kB 00:00 2024-09-10T18:36:52.767 INFO:teuthology.orchestra.run.smithi018.stdout:(5/13): ceph-mgr-rook-17.2.7-1606.g5be57395.el9 227 kB/s | 47 kB 00:00 2024-09-10T18:36:52.792 INFO:teuthology.orchestra.run.smithi018.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-09-10T18:36:52.826 INFO:teuthology.orchestra.run.smithi018.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.5 MB/s | 279 kB 00:00 2024-09-10T18:36:52.876 INFO:teuthology.orchestra.run.smithi018.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 296 kB/s | 32 kB 00:00 2024-09-10T18:36:52.901 INFO:teuthology.orchestra.run.smithi018.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 130 kB/s | 14 kB 00:00 2024-09-10T18:36:52.926 INFO:teuthology.orchestra.run.smithi018.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-10T18:36:52.952 INFO:teuthology.orchestra.run.smithi018.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 1.9 MB/s | 220 kB 00:00 2024-09-10T18:36:52.977 INFO:teuthology.orchestra.run.smithi018.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.7 MB/s | 90 kB 00:00 2024-09-10T18:36:52.987 INFO:teuthology.orchestra.run.smithi103.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 146 kB/s | 19 kB 00:00 2024-09-10T18:36:53.013 INFO:teuthology.orchestra.run.smithi103.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 164 kB/s | 26 kB 00:00 2024-09-10T18:36:53.019 INFO:teuthology.orchestra.run.smithi018.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.3 MB/s | 1.0 MB 00:00 2024-09-10T18:36:53.022 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:36:53.023 INFO:teuthology.orchestra.run.smithi018.stdout:Total 3.5 MB/s | 2.2 MB 00:00 2024-09-10T18:36:53.038 INFO:teuthology.orchestra.run.smithi103.stdout:(3/13): ceph-mgr-rook-17.2.7-1606.g5be57395.el9 261 kB/s | 47 kB 00:00 2024-09-10T18:36:53.091 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-09-10T18:36:53.107 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-09-10T18:36:53.108 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-09-10T18:36:53.163 INFO:teuthology.orchestra.run.smithi103.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.2 MB/s | 222 kB 00:00 2024-09-10T18:36:53.189 INFO:teuthology.orchestra.run.smithi103.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 903 kB/s | 159 kB 00:00 2024-09-10T18:36:53.203 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-09-10T18:36:53.204 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-09-10T18:36:53.222 INFO:teuthology.orchestra.run.smithi103.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 912 kB/s | 54 kB 00:00 2024-09-10T18:36:53.248 INFO:teuthology.orchestra.run.smithi103.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.3 MB/s | 279 kB 00:00 2024-09-10T18:36:53.281 INFO:teuthology.orchestra.run.smithi103.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 348 kB/s | 32 kB 00:00 2024-09-10T18:36:53.335 INFO:teuthology.orchestra.run.smithi103.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 128 kB/s | 14 kB 00:00 2024-09-10T18:36:53.365 INFO:teuthology.orchestra.run.smithi103.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.0 MB/s | 59 kB 00:00 2024-09-10T18:36:53.391 INFO:teuthology.orchestra.run.smithi103.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.5 MB/s | 90 kB 00:00 2024-09-10T18:36:53.416 INFO:teuthology.orchestra.run.smithi103.stdout:(12/13): python3-google-auth-2.34.0-1.el9.noarc 1.3 MB/s | 220 kB 00:00 2024-09-10T18:36:53.466 INFO:teuthology.orchestra.run.smithi103.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 5.8 MB/s | 1.0 MB 00:00 2024-09-10T18:36:53.469 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:36:53.469 INFO:teuthology.orchestra.run.smithi103.stdout:Total 2.7 MB/s | 2.2 MB 00:00 2024-09-10T18:36:53.531 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-10T18:36:53.547 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-10T18:36:53.547 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-10T18:36:53.633 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-10T18:36:53.634 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-10T18:36:53.722 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-09-10T18:36:53.900 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-10T18:36:53.964 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-10T18:36:54.020 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-10T18:36:54.064 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-10T18:36:54.106 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-10T18:36:54.137 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-10T18:36:54.212 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-10T18:36:54.316 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-10T18:36:54.322 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-10T18:36:54.368 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-10T18:36:54.375 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-10T18:36:54.431 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-10T18:36:54.469 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-10T18:36:54.511 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-10T18:36:54.622 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-10T18:36:54.732 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-10T18:36:54.788 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-10T18:36:55.138 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-10T18:36:55.196 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-10T18:36:55.230 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-10T18:36:55.277 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-10T18:36:55.299 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 13/13 2024-09-10T18:36:55.505 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-10T18:36:55.552 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-10T18:36:55.594 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-10T18:36:55.649 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-10T18:36:55.672 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 13/13 2024-09-10T18:36:56.102 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 13/13 2024-09-10T18:36:56.102 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 1/13 2024-09-10T18:36:56.102 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-10T18:36:56.102 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-10T18:36:56.103 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-10T18:36:56.103 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-10T18:36:56.103 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-10T18:36:56.103 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-10T18:36:56.103 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-10T18:36:56.103 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-10T18:36:56.103 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-10T18:36:56.103 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-10T18:36:56.103 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-10T18:36:56.416 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 13/13 2024-09-10T18:36:56.416 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 1/13 2024-09-10T18:36:56.416 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-10T18:36:56.416 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-10T18:36:56.416 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-10T18:36:56.416 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-10T18:36:56.416 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-10T18:36:56.416 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-10T18:36:56.416 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-10T18:36:56.417 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-10T18:36:56.417 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-10T18:36:56.417 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-10T18:36:56.417 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-10T18:36:56.455 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-10T18:36:56.455 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:56.455 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-09-10T18:36:56.455 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:36:56.455 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T18:36:56.455 INFO:teuthology.orchestra.run.smithi018.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T18:36:56.455 INFO:teuthology.orchestra.run.smithi018.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T18:36:56.455 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T18:36:56.456 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T18:36:56.456 INFO:teuthology.orchestra.run.smithi018.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T18:36:56.456 INFO:teuthology.orchestra.run.smithi018.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T18:36:56.456 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T18:36:56.456 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T18:36:56.456 INFO:teuthology.orchestra.run.smithi018.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T18:36:56.456 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T18:36:56.456 INFO:teuthology.orchestra.run.smithi018.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T18:36:56.456 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:56.456 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-09-10T18:36:56.712 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install ceph-mgr-cephadm 2024-09-10T18:36:56.757 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-10T18:36:56.758 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:56.758 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-10T18:36:56.758 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:36:56.758 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T18:36:56.758 INFO:teuthology.orchestra.run.smithi103.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T18:36:56.758 INFO:teuthology.orchestra.run.smithi103.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T18:36:56.758 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T18:36:56.758 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T18:36:56.758 INFO:teuthology.orchestra.run.smithi103.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T18:36:56.758 INFO:teuthology.orchestra.run.smithi103.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T18:36:56.758 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T18:36:56.758 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T18:36:56.759 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T18:36:56.759 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T18:36:56.759 INFO:teuthology.orchestra.run.smithi103.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T18:36:56.759 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:56.759 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-10T18:36:57.028 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph-mgr-cephadm 2024-09-10T18:36:57.306 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:01:20 ago on Tue 10 Sep 2024 06:35:37 PM UTC. 2024-09-10T18:36:57.444 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-09-10T18:36:57.445 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:36:57.445 INFO:teuthology.orchestra.run.smithi018.stdout: Package Arch Version Repository Size 2024-09-10T18:36:57.445 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:36:57.445 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-09-10T18:36:57.445 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 124 k 2024-09-10T18:36:57.445 INFO:teuthology.orchestra.run.smithi018.stdout:Installing dependencies: 2024-09-10T18:36:57.445 INFO:teuthology.orchestra.run.smithi018.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T18:36:57.445 INFO:teuthology.orchestra.run.smithi018.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T18:36:57.445 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T18:36:57.446 INFO:teuthology.orchestra.run.smithi018.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T18:36:57.446 INFO:teuthology.orchestra.run.smithi018.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T18:36:57.446 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:57.446 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-09-10T18:36:57.446 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:36:57.446 INFO:teuthology.orchestra.run.smithi018.stdout:Install 6 Packages 2024-09-10T18:36:57.446 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:36:57.446 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 7.0 M 2024-09-10T18:36:57.447 INFO:teuthology.orchestra.run.smithi018.stdout:Installed size: 33 M 2024-09-10T18:36:57.447 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-09-10T18:36:57.591 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:01:21 ago on Tue 10 Sep 2024 06:35:36 PM UTC. 2024-09-10T18:36:57.724 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-10T18:36:57.724 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:36:57.724 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repository Size 2024-09-10T18:36:57.724 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:36:57.724 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-10T18:36:57.725 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 124 k 2024-09-10T18:36:57.725 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2024-09-10T18:36:57.725 INFO:teuthology.orchestra.run.smithi103.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T18:36:57.725 INFO:teuthology.orchestra.run.smithi103.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T18:36:57.725 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T18:36:57.725 INFO:teuthology.orchestra.run.smithi103.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T18:36:57.725 INFO:teuthology.orchestra.run.smithi103.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T18:36:57.725 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:57.725 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-10T18:36:57.725 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:36:57.725 INFO:teuthology.orchestra.run.smithi103.stdout:Install 6 Packages 2024-09-10T18:36:57.725 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:36:57.726 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 7.0 M 2024-09-10T18:36:57.726 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 33 M 2024-09-10T18:36:57.726 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-10T18:36:57.764 INFO:teuthology.orchestra.run.smithi018.stdout:(1/6): ceph-mgr-cephadm-17.2.7-1606.g5be57395.e 600 kB/s | 124 kB 00:00 2024-09-10T18:36:57.815 INFO:teuthology.orchestra.run.smithi018.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 11 MB/s | 548 kB 00:00 2024-09-10T18:36:57.840 INFO:teuthology.orchestra.run.smithi018.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.3 MB/s | 58 kB 00:00 2024-09-10T18:36:57.865 INFO:teuthology.orchestra.run.smithi018.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-10T18:36:58.190 INFO:teuthology.orchestra.run.smithi018.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 392 kB/s | 248 kB 00:00 2024-09-10T18:36:58.238 INFO:teuthology.orchestra.run.smithi103.stdout:(1/6): ceph-mgr-cephadm-17.2.7-1606.g5be57395.e 615 kB/s | 124 kB 00:00 2024-09-10T18:36:58.263 INFO:teuthology.orchestra.run.smithi103.stdout:(2/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.1 MB/s | 248 kB 00:00 2024-09-10T18:36:58.321 INFO:teuthology.orchestra.run.smithi103.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 991 kB/s | 58 kB 00:00 2024-09-10T18:36:58.346 INFO:teuthology.orchestra.run.smithi103.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-10T18:36:58.388 INFO:teuthology.orchestra.run.smithi103.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 3.6 MB/s | 548 kB 00:00 2024-09-10T18:36:58.572 INFO:teuthology.orchestra.run.smithi103.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-09-10T18:36:58.574 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:36:58.575 INFO:teuthology.orchestra.run.smithi103.stdout:Total 8.2 MB/s | 7.0 MB 00:00 2024-09-10T18:36:58.644 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-10T18:36:58.655 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-10T18:36:58.655 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-10T18:36:58.754 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-10T18:36:58.754 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-10T18:36:59.060 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-10T18:36:59.240 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-10T18:36:59.299 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-10T18:36:59.707 INFO:teuthology.orchestra.run.smithi018.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 2.8 MB/s | 6.0 MB 00:02 2024-09-10T18:36:59.709 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:36:59.709 INFO:teuthology.orchestra.run.smithi018.stdout:Total 3.1 MB/s | 7.0 MB 00:02 2024-09-10T18:36:59.785 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-09-10T18:36:59.797 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-09-10T18:36:59.797 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-09-10T18:36:59.838 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-10T18:36:59.903 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-09-10T18:36:59.903 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-09-10T18:36:59.936 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-10T18:37:00.000 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-10T18:37:00.029 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 6/6 2024-09-10T18:37:00.232 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-09-10T18:37:00.413 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-10T18:37:00.469 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-10T18:37:00.818 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 6/6 2024-09-10T18:37:00.818 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/6 2024-09-10T18:37:00.818 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-10T18:37:00.818 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-10T18:37:00.818 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-10T18:37:00.818 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-10T18:37:01.031 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-10T18:37:01.098 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-10T18:37:01.098 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:37:01.099 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-10T18:37:01.099 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:37:01.099 INFO:teuthology.orchestra.run.smithi103.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T18:37:01.099 INFO:teuthology.orchestra.run.smithi103.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T18:37:01.099 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T18:37:01.099 INFO:teuthology.orchestra.run.smithi103.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T18:37:01.099 INFO:teuthology.orchestra.run.smithi103.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T18:37:01.099 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:37:01.099 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-10T18:37:01.123 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-10T18:37:01.186 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-10T18:37:01.213 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 6/6 2024-09-10T18:37:01.361 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph-fuse 2024-09-10T18:37:01.929 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:01:25 ago on Tue 10 Sep 2024 06:35:36 PM UTC. 2024-09-10T18:37:01.987 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 6/6 2024-09-10T18:37:01.988 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/6 2024-09-10T18:37:01.988 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-10T18:37:01.988 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-10T18:37:01.988 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-10T18:37:01.988 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-10T18:37:02.060 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-10T18:37:02.061 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:37:02.061 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repository Size 2024-09-10T18:37:02.061 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:37:02.061 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-10T18:37:02.061 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-fuse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 813 k 2024-09-10T18:37:02.061 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2024-09-10T18:37:02.062 INFO:teuthology.orchestra.run.smithi103.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T18:37:02.062 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:37:02.062 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-10T18:37:02.062 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:37:02.062 INFO:teuthology.orchestra.run.smithi103.stdout:Install 2 Packages 2024-09-10T18:37:02.062 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:37:02.062 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 893 k 2024-09-10T18:37:02.062 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 2.6 M 2024-09-10T18:37:02.063 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-10T18:37:02.263 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-10T18:37:02.263 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:37:02.263 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-09-10T18:37:02.263 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:37:02.264 INFO:teuthology.orchestra.run.smithi018.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T18:37:02.264 INFO:teuthology.orchestra.run.smithi018.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T18:37:02.264 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T18:37:02.264 INFO:teuthology.orchestra.run.smithi018.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T18:37:02.264 INFO:teuthology.orchestra.run.smithi018.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T18:37:02.264 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:37:02.264 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-09-10T18:37:02.342 INFO:teuthology.orchestra.run.smithi103.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 441 kB/s | 80 kB 00:00 2024-09-10T18:37:02.451 INFO:teuthology.orchestra.run.smithi103.stdout:(2/2): ceph-fuse-17.2.7-1606.g5be57395.el9.x86_ 2.7 MB/s | 813 kB 00:00 2024-09-10T18:37:02.451 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:37:02.452 INFO:teuthology.orchestra.run.smithi103.stdout:Total 2.2 MB/s | 893 kB 00:00 2024-09-10T18:37:02.463 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-10T18:37:02.485 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-10T18:37:02.485 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-10T18:37:02.539 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install ceph-fuse 2024-09-10T18:37:02.567 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-10T18:37:02.567 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-10T18:37:02.753 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-10T18:37:02.888 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-10T18:37:02.961 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2/2 2024-09-10T18:37:03.132 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:01:26 ago on Tue 10 Sep 2024 06:35:37 PM UTC. 2024-09-10T18:37:03.271 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-09-10T18:37:03.272 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:37:03.272 INFO:teuthology.orchestra.run.smithi018.stdout: Package Arch Version Repository Size 2024-09-10T18:37:03.272 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:37:03.272 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-09-10T18:37:03.272 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-fuse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 813 k 2024-09-10T18:37:03.272 INFO:teuthology.orchestra.run.smithi018.stdout:Installing dependencies: 2024-09-10T18:37:03.272 INFO:teuthology.orchestra.run.smithi018.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T18:37:03.272 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:37:03.272 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-09-10T18:37:03.272 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:37:03.273 INFO:teuthology.orchestra.run.smithi018.stdout:Install 2 Packages 2024-09-10T18:37:03.273 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:37:03.273 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 893 k 2024-09-10T18:37:03.273 INFO:teuthology.orchestra.run.smithi018.stdout:Installed size: 2.6 M 2024-09-10T18:37:03.273 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-09-10T18:37:03.451 INFO:teuthology.orchestra.run.smithi018.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 511 kB/s | 80 kB 00:00 2024-09-10T18:37:03.592 INFO:teuthology.orchestra.run.smithi018.stdout:(2/2): ceph-fuse-17.2.7-1606.g5be57395.el9.x86_ 2.7 MB/s | 813 kB 00:00 2024-09-10T18:37:03.593 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:37:03.593 INFO:teuthology.orchestra.run.smithi018.stdout:Total 2.7 MB/s | 893 kB 00:00 2024-09-10T18:37:03.605 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-09-10T18:37:03.627 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-09-10T18:37:03.627 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-09-10T18:37:03.693 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2/2 2024-09-10T18:37:03.693 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/2 2024-09-10T18:37:03.712 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-09-10T18:37:03.712 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-09-10T18:37:03.904 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-09-10T18:37:03.980 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-10T18:37:03.981 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:37:03.981 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-10T18:37:03.981 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-10T18:37:03.981 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:37:03.981 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-10T18:37:04.043 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-10T18:37:04.120 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2/2 2024-09-10T18:37:04.239 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph-volume 2024-09-10T18:37:04.764 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2/2 2024-09-10T18:37:04.764 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/2 2024-09-10T18:37:04.821 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:01:28 ago on Tue 10 Sep 2024 06:35:36 PM UTC. 2024-09-10T18:37:04.958 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-10T18:37:04.959 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:37:04.959 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repository Size 2024-09-10T18:37:04.959 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:37:04.959 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-10T18:37:04.959 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-volume noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 270 k 2024-09-10T18:37:04.960 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:37:04.960 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-10T18:37:04.960 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:37:04.960 INFO:teuthology.orchestra.run.smithi103.stdout:Install 1 Package 2024-09-10T18:37:04.960 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:37:04.960 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 270 k 2024-09-10T18:37:04.960 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 1.3 M 2024-09-10T18:37:04.960 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-10T18:37:05.071 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-10T18:37:05.071 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:37:05.071 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-09-10T18:37:05.071 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-10T18:37:05.071 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:37:05.071 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-09-10T18:37:05.199 INFO:teuthology.orchestra.run.smithi103.stdout:ceph-volume-17.2.7-1606.g5be57395.el9.noarch.rp 1.1 MB/s | 270 kB 00:00 2024-09-10T18:37:05.199 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:37:05.200 INFO:teuthology.orchestra.run.smithi103.stdout:Total 1.1 MB/s | 270 kB 00:00 2024-09-10T18:37:05.200 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-10T18:37:05.208 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-10T18:37:05.208 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-10T18:37:05.235 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-10T18:37:05.235 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-10T18:37:05.422 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install ceph-volume 2024-09-10T18:37:05.444 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-10T18:37:05.509 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:37:05.526 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:37:05.526 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:37:05.526 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T18:37:05.526 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:37:06.010 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:01:29 ago on Tue 10 Sep 2024 06:35:37 PM UTC. 2024-09-10T18:37:06.148 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-09-10T18:37:06.149 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:37:06.149 INFO:teuthology.orchestra.run.smithi018.stdout: Package Arch Version Repository Size 2024-09-10T18:37:06.149 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:37:06.149 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-09-10T18:37:06.150 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-volume noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 270 k 2024-09-10T18:37:06.150 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:37:06.150 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-09-10T18:37:06.150 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:37:06.150 INFO:teuthology.orchestra.run.smithi018.stdout:Install 1 Package 2024-09-10T18:37:06.150 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:37:06.150 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 270 k 2024-09-10T18:37:06.150 INFO:teuthology.orchestra.run.smithi018.stdout:Installed size: 1.3 M 2024-09-10T18:37:06.150 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-09-10T18:37:06.372 INFO:teuthology.orchestra.run.smithi018.stdout:ceph-volume-17.2.7-1606.g5be57395.el9.noarch.rp 1.2 MB/s | 270 kB 00:00 2024-09-10T18:37:06.373 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:37:06.373 INFO:teuthology.orchestra.run.smithi018.stdout:Total 1.2 MB/s | 270 kB 00:00 2024-09-10T18:37:06.373 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-09-10T18:37:06.383 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-09-10T18:37:06.383 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-09-10T18:37:06.411 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-09-10T18:37:06.411 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-09-10T18:37:06.572 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:37:06.573 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:37:06.573 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-10T18:37:06.573 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:37:06.573 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:37:06.573 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-10T18:37:06.637 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-09-10T18:37:06.716 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:37:06.734 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:37:06.734 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:37:06.734 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T18:37:06.734 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:37:06.914 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install librados-devel 2024-09-10T18:37:07.494 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:01:31 ago on Tue 10 Sep 2024 06:35:36 PM UTC. 2024-09-10T18:37:07.628 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-10T18:37:07.628 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:37:07.628 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repo Size 2024-09-10T18:37:07.628 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:37:07.628 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-10T18:37:07.628 INFO:teuthology.orchestra.run.smithi103.stdout: librados-devel x86_64 2:17.2.7-1606.g5be57395.el9 ceph 126 k 2024-09-10T18:37:07.628 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:37:07.629 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-10T18:37:07.629 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:37:07.629 INFO:teuthology.orchestra.run.smithi103.stdout:Install 1 Package 2024-09-10T18:37:07.629 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:37:07.629 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 126 k 2024-09-10T18:37:07.629 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 448 k 2024-09-10T18:37:07.629 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-10T18:37:07.752 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:37:07.752 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:37:07.752 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-09-10T18:37:07.752 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:37:07.752 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:37:07.752 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-09-10T18:37:07.831 INFO:teuthology.orchestra.run.smithi103.stdout:librados-devel-17.2.7-1606.g5be57395.el9.x86_64 627 kB/s | 126 kB 00:00 2024-09-10T18:37:07.832 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:37:07.832 INFO:teuthology.orchestra.run.smithi103.stdout:Total 622 kB/s | 126 kB 00:00 2024-09-10T18:37:07.832 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-10T18:37:07.837 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-10T18:37:07.837 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-10T18:37:07.901 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-10T18:37:07.901 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-10T18:37:08.055 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-10T18:37:08.073 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install librados-devel 2024-09-10T18:37:08.153 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:37:08.608 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:37:08.667 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:01:31 ago on Tue 10 Sep 2024 06:35:37 PM UTC. 2024-09-10T18:37:08.806 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-09-10T18:37:08.806 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:37:08.806 INFO:teuthology.orchestra.run.smithi018.stdout: Package Arch Version Repo Size 2024-09-10T18:37:08.806 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:37:08.806 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-09-10T18:37:08.807 INFO:teuthology.orchestra.run.smithi018.stdout: librados-devel x86_64 2:17.2.7-1606.g5be57395.el9 ceph 126 k 2024-09-10T18:37:08.807 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:37:08.807 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-09-10T18:37:08.807 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:37:08.807 INFO:teuthology.orchestra.run.smithi018.stdout:Install 1 Package 2024-09-10T18:37:08.807 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:37:08.807 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 126 k 2024-09-10T18:37:08.808 INFO:teuthology.orchestra.run.smithi018.stdout:Installed size: 448 k 2024-09-10T18:37:08.808 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-09-10T18:37:08.872 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:37:08.872 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:37:08.872 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-10T18:37:08.872 INFO:teuthology.orchestra.run.smithi103.stdout: librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:37:08.872 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:37:08.872 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-10T18:37:08.995 INFO:teuthology.orchestra.run.smithi018.stdout:librados-devel-17.2.7-1606.g5be57395.el9.x86_64 673 kB/s | 126 kB 00:00 2024-09-10T18:37:08.995 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:37:08.996 INFO:teuthology.orchestra.run.smithi018.stdout:Total 668 kB/s | 126 kB 00:00 2024-09-10T18:37:08.996 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-09-10T18:37:09.002 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-09-10T18:37:09.002 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-09-10T18:37:09.071 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-09-10T18:37:09.071 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-09-10T18:37:09.144 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install libcephfs2 2024-09-10T18:37:09.240 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-09-10T18:37:09.346 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:37:09.718 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:01:33 ago on Tue 10 Sep 2024 06:35:36 PM UTC. 2024-09-10T18:37:09.761 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:37:09.809 INFO:teuthology.orchestra.run.smithi103.stdout:Package libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:37:09.853 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-10T18:37:09.854 INFO:teuthology.orchestra.run.smithi103.stdout:Nothing to do. 2024-09-10T18:37:09.854 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-10T18:37:09.930 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install libcephfs-devel 2024-09-10T18:37:10.036 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:37:10.036 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:37:10.036 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-09-10T18:37:10.036 INFO:teuthology.orchestra.run.smithi018.stdout: librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:37:10.036 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:37:10.036 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-09-10T18:37:10.340 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install libcephfs2 2024-09-10T18:37:10.500 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:01:34 ago on Tue 10 Sep 2024 06:35:36 PM UTC. 2024-09-10T18:37:10.631 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-10T18:37:10.632 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:37:10.632 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repo Size 2024-09-10T18:37:10.632 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:37:10.632 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-10T18:37:10.632 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs-devel x86_64 2:17.2.7-1606.g5be57395.el9 ceph 25 k 2024-09-10T18:37:10.633 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:37:10.633 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-10T18:37:10.633 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:37:10.633 INFO:teuthology.orchestra.run.smithi103.stdout:Install 1 Package 2024-09-10T18:37:10.633 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:37:10.633 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 25 k 2024-09-10T18:37:10.633 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 110 k 2024-09-10T18:37:10.633 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-10T18:37:10.786 INFO:teuthology.orchestra.run.smithi103.stdout:libcephfs-devel-17.2.7-1606.g5be57395.el9.x86_6 163 kB/s | 25 kB 00:00 2024-09-10T18:37:10.786 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:37:10.786 INFO:teuthology.orchestra.run.smithi103.stdout:Total 162 kB/s | 25 kB 00:00 2024-09-10T18:37:10.787 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-10T18:37:10.790 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-10T18:37:10.790 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-10T18:37:10.792 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-10T18:37:10.792 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-10T18:37:10.881 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-10T18:37:10.948 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:01:33 ago on Tue 10 Sep 2024 06:35:37 PM UTC. 2024-09-10T18:37:11.008 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:37:11.045 INFO:teuthology.orchestra.run.smithi018.stdout:Package libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:37:11.091 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-09-10T18:37:11.093 INFO:teuthology.orchestra.run.smithi018.stdout:Nothing to do. 2024-09-10T18:37:11.093 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-09-10T18:37:11.162 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install libcephfs-devel 2024-09-10T18:37:11.372 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:37:11.612 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:37:11.612 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:37:11.612 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-10T18:37:11.612 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:37:11.613 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:37:11.613 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-10T18:37:11.752 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:01:34 ago on Tue 10 Sep 2024 06:35:37 PM UTC. 2024-09-10T18:37:11.803 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install librados2 2024-09-10T18:37:11.889 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-09-10T18:37:11.892 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:37:11.892 INFO:teuthology.orchestra.run.smithi018.stdout: Package Arch Version Repo Size 2024-09-10T18:37:11.892 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:37:11.893 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-09-10T18:37:11.893 INFO:teuthology.orchestra.run.smithi018.stdout: libcephfs-devel x86_64 2:17.2.7-1606.g5be57395.el9 ceph 25 k 2024-09-10T18:37:11.893 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:37:11.893 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-09-10T18:37:11.893 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:37:11.893 INFO:teuthology.orchestra.run.smithi018.stdout:Install 1 Package 2024-09-10T18:37:11.893 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:37:11.893 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 25 k 2024-09-10T18:37:11.893 INFO:teuthology.orchestra.run.smithi018.stdout:Installed size: 110 k 2024-09-10T18:37:11.894 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-09-10T18:37:12.041 INFO:teuthology.orchestra.run.smithi018.stdout:libcephfs-devel-17.2.7-1606.g5be57395.el9.x86_6 166 kB/s | 25 kB 00:00 2024-09-10T18:37:12.042 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:37:12.042 INFO:teuthology.orchestra.run.smithi018.stdout:Total 165 kB/s | 25 kB 00:00 2024-09-10T18:37:12.042 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-09-10T18:37:12.045 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-09-10T18:37:12.045 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-09-10T18:37:12.046 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-09-10T18:37:12.047 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-09-10T18:37:12.136 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-09-10T18:37:12.257 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:37:12.368 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:01:36 ago on Tue 10 Sep 2024 06:35:36 PM UTC. 2024-09-10T18:37:12.460 INFO:teuthology.orchestra.run.smithi103.stdout:Package librados2-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:37:12.502 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-10T18:37:12.503 INFO:teuthology.orchestra.run.smithi103.stdout:Nothing to do. 2024-09-10T18:37:12.503 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-10T18:37:12.577 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install librbd1 2024-09-10T18:37:12.646 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:37:12.885 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:37:12.885 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:37:12.885 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-09-10T18:37:12.885 INFO:teuthology.orchestra.run.smithi018.stdout: libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:37:12.885 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:37:12.885 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-09-10T18:37:13.072 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install librados2 2024-09-10T18:37:13.143 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:01:37 ago on Tue 10 Sep 2024 06:35:36 PM UTC. 2024-09-10T18:37:13.232 INFO:teuthology.orchestra.run.smithi103.stdout:Package librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:37:13.274 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-10T18:37:13.275 INFO:teuthology.orchestra.run.smithi103.stdout:Nothing to do. 2024-09-10T18:37:13.275 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-10T18:37:13.341 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install python3-rados 2024-09-10T18:37:13.651 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:01:36 ago on Tue 10 Sep 2024 06:35:37 PM UTC. 2024-09-10T18:37:13.742 INFO:teuthology.orchestra.run.smithi018.stdout:Package librados2-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:37:13.787 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-09-10T18:37:13.788 INFO:teuthology.orchestra.run.smithi018.stdout:Nothing to do. 2024-09-10T18:37:13.789 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-09-10T18:37:13.861 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install librbd1 2024-09-10T18:37:13.902 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:01:37 ago on Tue 10 Sep 2024 06:35:36 PM UTC. 2024-09-10T18:37:13.992 INFO:teuthology.orchestra.run.smithi103.stdout:Package python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:37:14.035 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-10T18:37:14.036 INFO:teuthology.orchestra.run.smithi103.stdout:Nothing to do. 2024-09-10T18:37:14.036 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-10T18:37:14.109 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install python3-rgw 2024-09-10T18:37:14.449 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:01:37 ago on Tue 10 Sep 2024 06:35:37 PM UTC. 2024-09-10T18:37:14.543 INFO:teuthology.orchestra.run.smithi018.stdout:Package librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:37:14.589 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-09-10T18:37:14.590 INFO:teuthology.orchestra.run.smithi018.stdout:Nothing to do. 2024-09-10T18:37:14.590 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-09-10T18:37:14.664 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install python3-rados 2024-09-10T18:37:14.670 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:01:38 ago on Tue 10 Sep 2024 06:35:36 PM UTC. 2024-09-10T18:37:14.761 INFO:teuthology.orchestra.run.smithi103.stdout:Package python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:37:14.805 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-10T18:37:14.806 INFO:teuthology.orchestra.run.smithi103.stdout:Nothing to do. 2024-09-10T18:37:14.806 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-10T18:37:14.877 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install python3-cephfs 2024-09-10T18:37:15.255 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:01:38 ago on Tue 10 Sep 2024 06:35:37 PM UTC. 2024-09-10T18:37:15.348 INFO:teuthology.orchestra.run.smithi018.stdout:Package python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:37:15.394 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-09-10T18:37:15.395 INFO:teuthology.orchestra.run.smithi018.stdout:Nothing to do. 2024-09-10T18:37:15.395 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-09-10T18:37:15.445 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:01:39 ago on Tue 10 Sep 2024 06:35:36 PM UTC. 2024-09-10T18:37:15.461 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install python3-rgw 2024-09-10T18:37:15.535 INFO:teuthology.orchestra.run.smithi103.stdout:Package python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:37:15.577 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-10T18:37:15.578 INFO:teuthology.orchestra.run.smithi103.stdout:Nothing to do. 2024-09-10T18:37:15.578 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-10T18:37:15.652 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install python3-rbd 2024-09-10T18:37:16.052 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:01:39 ago on Tue 10 Sep 2024 06:35:37 PM UTC. 2024-09-10T18:37:16.146 INFO:teuthology.orchestra.run.smithi018.stdout:Package python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:37:16.192 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-09-10T18:37:16.193 INFO:teuthology.orchestra.run.smithi018.stdout:Nothing to do. 2024-09-10T18:37:16.193 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-09-10T18:37:16.216 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:01:40 ago on Tue 10 Sep 2024 06:35:36 PM UTC. 2024-09-10T18:37:16.262 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install python3-cephfs 2024-09-10T18:37:16.306 INFO:teuthology.orchestra.run.smithi103.stdout:Package python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:37:16.348 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-10T18:37:16.349 INFO:teuthology.orchestra.run.smithi103.stdout:Nothing to do. 2024-09-10T18:37:16.349 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-10T18:37:16.410 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install rbd-fuse 2024-09-10T18:37:16.855 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:01:39 ago on Tue 10 Sep 2024 06:35:37 PM UTC. 2024-09-10T18:37:16.949 INFO:teuthology.orchestra.run.smithi018.stdout:Package python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:37:16.973 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:01:40 ago on Tue 10 Sep 2024 06:35:36 PM UTC. 2024-09-10T18:37:16.994 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-09-10T18:37:16.996 INFO:teuthology.orchestra.run.smithi018.stdout:Nothing to do. 2024-09-10T18:37:16.996 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-09-10T18:37:17.070 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install python3-rbd 2024-09-10T18:37:17.104 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-10T18:37:17.105 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:37:17.105 INFO:teuthology.orchestra.run.smithi103.stdout: Package Architecture Version Repository Size 2024-09-10T18:37:17.105 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:37:17.105 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-10T18:37:17.105 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-fuse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 82 k 2024-09-10T18:37:17.105 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:37:17.105 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-10T18:37:17.105 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:37:17.105 INFO:teuthology.orchestra.run.smithi103.stdout:Install 1 Package 2024-09-10T18:37:17.105 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:37:17.106 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 82 k 2024-09-10T18:37:17.106 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 222 k 2024-09-10T18:37:17.106 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-10T18:37:17.273 INFO:teuthology.orchestra.run.smithi103.stdout:rbd-fuse-17.2.7-1606.g5be57395.el9.x86_64.rpm 492 kB/s | 82 kB 00:00 2024-09-10T18:37:17.273 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:37:17.273 INFO:teuthology.orchestra.run.smithi103.stdout:Total 488 kB/s | 82 kB 00:00 2024-09-10T18:37:17.273 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-10T18:37:17.280 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-10T18:37:17.280 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-10T18:37:17.344 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-10T18:37:17.344 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-10T18:37:17.489 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-10T18:37:17.595 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:37:17.661 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:01:40 ago on Tue 10 Sep 2024 06:35:37 PM UTC. 2024-09-10T18:37:17.756 INFO:teuthology.orchestra.run.smithi018.stdout:Package python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:37:17.802 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-09-10T18:37:17.840 INFO:teuthology.orchestra.run.smithi018.stdout:Nothing to do. 2024-09-10T18:37:17.840 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-09-10T18:37:17.867 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install rbd-fuse 2024-09-10T18:37:17.964 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:37:18.220 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:37:18.221 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:37:18.221 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-10T18:37:18.221 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:37:18.221 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:37:18.221 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-10T18:37:18.450 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install rbd-mirror 2024-09-10T18:37:18.464 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:01:41 ago on Tue 10 Sep 2024 06:35:37 PM UTC. 2024-09-10T18:37:18.603 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-09-10T18:37:18.604 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:37:18.604 INFO:teuthology.orchestra.run.smithi018.stdout: Package Architecture Version Repository Size 2024-09-10T18:37:18.604 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:37:18.604 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-09-10T18:37:18.604 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-fuse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 82 k 2024-09-10T18:37:18.605 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:37:18.605 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-09-10T18:37:18.605 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:37:18.605 INFO:teuthology.orchestra.run.smithi018.stdout:Install 1 Package 2024-09-10T18:37:18.605 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:37:18.605 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 82 k 2024-09-10T18:37:18.605 INFO:teuthology.orchestra.run.smithi018.stdout:Installed size: 222 k 2024-09-10T18:37:18.605 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-09-10T18:37:18.787 INFO:teuthology.orchestra.run.smithi018.stdout:rbd-fuse-17.2.7-1606.g5be57395.el9.x86_64.rpm 452 kB/s | 82 kB 00:00 2024-09-10T18:37:18.787 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:37:18.787 INFO:teuthology.orchestra.run.smithi018.stdout:Total 447 kB/s | 82 kB 00:00 2024-09-10T18:37:18.788 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-09-10T18:37:18.795 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-09-10T18:37:18.795 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-09-10T18:37:18.862 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-09-10T18:37:18.862 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-09-10T18:37:19.012 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-09-10T18:37:19.025 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:01:43 ago on Tue 10 Sep 2024 06:35:36 PM UTC. 2024-09-10T18:37:19.111 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:37:19.161 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-10T18:37:19.162 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:37:19.162 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repo Size 2024-09-10T18:37:19.162 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:37:19.162 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-10T18:37:19.162 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-mirror x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.9 M 2024-09-10T18:37:19.162 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:37:19.162 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-10T18:37:19.163 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:37:19.163 INFO:teuthology.orchestra.run.smithi103.stdout:Install 1 Package 2024-09-10T18:37:19.163 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:37:19.163 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 2.9 M 2024-09-10T18:37:19.163 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 12 M 2024-09-10T18:37:19.163 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-10T18:37:19.510 INFO:teuthology.orchestra.run.smithi103.stdout:rbd-mirror-17.2.7-1606.g5be57395.el9.x86_64.rpm 8.5 MB/s | 2.9 MB 00:00 2024-09-10T18:37:19.511 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:37:19.511 INFO:teuthology.orchestra.run.smithi103.stdout:Total 8.4 MB/s | 2.9 MB 00:00 2024-09-10T18:37:19.511 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-10T18:37:19.520 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-10T18:37:19.520 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-10T18:37:19.579 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:37:19.600 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-10T18:37:19.600 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-10T18:37:19.825 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:37:19.825 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:37:19.825 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-09-10T18:37:19.825 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:37:19.825 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:37:19.826 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-09-10T18:37:20.013 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-10T18:37:20.086 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:37:20.111 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:37:20.112 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:37:20.112 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T18:37:20.112 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T18:37:20.112 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T18:37:20.112 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:37:20.130 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install rbd-mirror 2024-09-10T18:37:20.728 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:01:43 ago on Tue 10 Sep 2024 06:35:37 PM UTC. 2024-09-10T18:37:20.867 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-09-10T18:37:20.868 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:37:20.868 INFO:teuthology.orchestra.run.smithi018.stdout: Package Arch Version Repo Size 2024-09-10T18:37:20.868 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:37:20.868 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-09-10T18:37:20.868 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-mirror x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.9 M 2024-09-10T18:37:20.869 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:37:20.869 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-09-10T18:37:20.869 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:37:20.869 INFO:teuthology.orchestra.run.smithi018.stdout:Install 1 Package 2024-09-10T18:37:20.869 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:37:20.869 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 2.9 M 2024-09-10T18:37:20.869 INFO:teuthology.orchestra.run.smithi018.stdout:Installed size: 12 M 2024-09-10T18:37:20.869 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-09-10T18:37:21.136 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:37:21.136 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:37:21.136 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-10T18:37:21.136 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:37:21.137 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:37:21.137 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-10T18:37:21.223 INFO:teuthology.orchestra.run.smithi018.stdout:rbd-mirror-17.2.7-1606.g5be57395.el9.x86_64.rpm 8.3 MB/s | 2.9 MB 00:00 2024-09-10T18:37:21.224 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:37:21.224 INFO:teuthology.orchestra.run.smithi018.stdout:Total 8.3 MB/s | 2.9 MB 00:00 2024-09-10T18:37:21.224 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-09-10T18:37:21.234 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-09-10T18:37:21.234 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-09-10T18:37:21.319 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-09-10T18:37:21.319 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-09-10T18:37:21.399 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install rbd-nbd 2024-09-10T18:37:21.757 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-09-10T18:37:21.815 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:37:21.843 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:37:21.843 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:37:21.843 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T18:37:21.844 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T18:37:21.844 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T18:37:21.844 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:37:21.984 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:01:45 ago on Tue 10 Sep 2024 06:35:36 PM UTC. 2024-09-10T18:37:22.114 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-10T18:37:22.116 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:37:22.116 INFO:teuthology.orchestra.run.smithi103.stdout: Package Architecture Version Repository Size 2024-09-10T18:37:22.116 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:37:22.116 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-10T18:37:22.116 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-nbd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 165 k 2024-09-10T18:37:22.117 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:37:22.117 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-10T18:37:22.117 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-10T18:37:22.117 INFO:teuthology.orchestra.run.smithi103.stdout:Install 1 Package 2024-09-10T18:37:22.117 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:37:22.118 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 165 k 2024-09-10T18:37:22.118 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 477 k 2024-09-10T18:37:22.118 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-10T18:37:22.324 INFO:teuthology.orchestra.run.smithi103.stdout:rbd-nbd-17.2.7-1606.g5be57395.el9.x86_64.rpm 794 kB/s | 165 kB 00:00 2024-09-10T18:37:22.324 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:37:22.325 INFO:teuthology.orchestra.run.smithi103.stdout:Total 789 kB/s | 165 kB 00:00 2024-09-10T18:37:22.325 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-10T18:37:22.333 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-10T18:37:22.333 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-10T18:37:22.397 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-10T18:37:22.397 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-10T18:37:22.555 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-10T18:37:22.663 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:37:22.975 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:37:22.975 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:37:22.975 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-09-10T18:37:22.975 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:37:22.975 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:37:22.975 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-09-10T18:37:23.081 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:37:23.215 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install rbd-nbd 2024-09-10T18:37:23.353 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:37:23.354 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:37:23.354 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-10T18:37:23.354 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:37:23.354 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:37:23.354 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-10T18:37:23.595 DEBUG:teuthology.parallel:result is None 2024-09-10T18:37:23.812 INFO:teuthology.orchestra.run.smithi018.stdout:Last metadata expiration check: 0:01:46 ago on Tue 10 Sep 2024 06:35:37 PM UTC. 2024-09-10T18:37:23.949 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-09-10T18:37:23.950 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:37:23.950 INFO:teuthology.orchestra.run.smithi018.stdout: Package Architecture Version Repository Size 2024-09-10T18:37:23.950 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:37:23.950 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-09-10T18:37:23.950 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-nbd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 165 k 2024-09-10T18:37:23.950 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:37:23.951 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-09-10T18:37:23.951 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2024-09-10T18:37:23.951 INFO:teuthology.orchestra.run.smithi018.stdout:Install 1 Package 2024-09-10T18:37:23.951 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:37:23.951 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 165 k 2024-09-10T18:37:23.951 INFO:teuthology.orchestra.run.smithi018.stdout:Installed size: 477 k 2024-09-10T18:37:23.951 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-09-10T18:37:24.152 INFO:teuthology.orchestra.run.smithi018.stdout:rbd-nbd-17.2.7-1606.g5be57395.el9.x86_64.rpm 822 kB/s | 165 kB 00:00 2024-09-10T18:37:24.153 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:37:24.153 INFO:teuthology.orchestra.run.smithi018.stdout:Total 816 kB/s | 165 kB 00:00 2024-09-10T18:37:24.153 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-09-10T18:37:24.161 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-09-10T18:37:24.161 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-09-10T18:37:24.228 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-09-10T18:37:24.228 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-09-10T18:37:24.497 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-09-10T18:37:24.602 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:37:25.029 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:37:25.311 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:37:25.311 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:37:25.311 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-09-10T18:37:25.311 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:37:25.311 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:37:25.311 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-09-10T18:37:25.556 DEBUG:teuthology.parallel:result is None 2024-09-10T18:37:25.557 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:37:25.700 DEBUG:teuthology.orchestra.run.smithi018:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T18:37:25.723 INFO:teuthology.orchestra.run.smithi018.stdout:17.2.7-1606.g5be57395.el9 2024-09-10T18:37:25.723 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1606.g5be57395.el9 2024-09-10T18:37:25.724 INFO:teuthology.task.install:The correct ceph version 17.2.7-1606.g5be57395 is installed. 2024-09-10T18:37:25.726 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:37:25.860 DEBUG:teuthology.orchestra.run.smithi103:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T18:37:25.885 INFO:teuthology.orchestra.run.smithi103.stdout:17.2.7-1606.g5be57395.el9 2024-09-10T18:37:25.886 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1606.g5be57395.el9 2024-09-10T18:37:25.886 INFO:teuthology.task.install:The correct ceph version 17.2.7-1606.g5be57395 is installed. 2024-09-10T18:37:25.888 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T18:37:25.888 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-10T18:37:25.888 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T18:37:25.915 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-10T18:37:25.915 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T18:37:25.956 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T18:37:25.957 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-10T18:37:25.957 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T18:37:25.983 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T18:37:26.050 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-10T18:37:26.050 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T18:37:26.079 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T18:37:26.146 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T18:37:26.146 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-10T18:37:26.147 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T18:37:26.174 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T18:37:26.240 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-10T18:37:26.240 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T18:37:26.267 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T18:37:26.333 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T18:37:26.333 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-10T18:37:26.333 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T18:37:26.359 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T18:37:26.425 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-10T18:37:26.425 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T18:37:26.451 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T18:37:26.516 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-10T18:37:26.599 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'ms bind msgr1': False, 'ms bind msgr2': True, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', '\\(POOL_APP_NOT_ENABLED\\)'], 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588', 'cephadm_mode': 'root'} 2024-09-10T18:37:26.599 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:37:26.599 INFO:tasks.cephadm:Cluster fsid is ba327f6c-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:37:26.600 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-10T18:37:26.600 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.18', 'mon.c': '[v2:172.21.15.18:3301,v1:172.21.15.18:6790]', 'mon.b': '172.21.15.103'} 2024-09-10T18:37:26.600 INFO:tasks.cephadm:First mon is mon.a on smithi018 2024-09-10T18:37:26.600 INFO:tasks.cephadm:First mgr is y 2024-09-10T18:37:26.600 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-10T18:37:26.600 DEBUG:teuthology.orchestra.run.smithi018:> sudo hostname $(hostname -s) 2024-09-10T18:37:26.627 DEBUG:teuthology.orchestra.run.smithi103:> sudo hostname $(hostname -s) 2024-09-10T18:37:26.652 INFO:tasks.cephadm:Downloading cephadm (repo https://git.ceph.com/ceph-ci.git ref 5be57395ea235535910512ae73ba8e7ff25fe588)... 2024-09-10T18:37:26.652 DEBUG:teuthology.orchestra.run.smithi018:> git clone https://git.ceph.com/ceph-ci.git testrepo && cd testrepo && git show 5be57395ea235535910512ae73ba8e7ff25fe588:src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T18:37:26.683 INFO:teuthology.orchestra.run.smithi018.stderr:Cloning into 'testrepo'... 2024-09-10T18:38:25.859 INFO:teuthology.orchestra.run.smithi018.stderr:Updating files: 91% (11652/12697) Updating files: 92% (11682/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-10T18:38:25.894 INFO:teuthology.orchestra.run.smithi018.stdout:-rw-r--r--. 1 ubuntu ubuntu 378585 Sep 10 18:38 /home/ubuntu/cephtest/cephadm 2024-09-10T18:38:25.895 DEBUG:teuthology.orchestra.run.smithi103:> git clone https://git.ceph.com/ceph-ci.git testrepo && cd testrepo && git show 5be57395ea235535910512ae73ba8e7ff25fe588:src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T18:38:25.914 INFO:teuthology.orchestra.run.smithi103.stderr:Cloning into 'testrepo'... 2024-09-10T18:39:24.513 INFO:teuthology.orchestra.run.smithi103.stderr:Updating files: 98% (12548/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-10T18:39:24.546 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-r--r--. 1 ubuntu ubuntu 378585 Sep 10 18:39 /home/ubuntu/cephtest/cephadm 2024-09-10T18:39:24.547 DEBUG:teuthology.orchestra.run.smithi018:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T18:39:24.569 DEBUG:teuthology.orchestra.run.smithi103:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T18:39:24.617 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 on all hosts... 2024-09-10T18:39:24.618 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 pull 2024-09-10T18:39:24.620 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 pull 2024-09-10T18:39:26.144 INFO:teuthology.orchestra.run.smithi018.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588... 2024-09-10T18:39:28.292 INFO:teuthology.orchestra.run.smithi103.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588... 2024-09-10T18:39:54.054 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T18:39:54.054 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_version": "ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable)", 2024-09-10T18:39:54.054 INFO:teuthology.orchestra.run.smithi103.stdout: "image_id": "4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9", 2024-09-10T18:39:54.055 INFO:teuthology.orchestra.run.smithi103.stdout: "repo_digests": [ 2024-09-10T18:39:54.055 INFO:teuthology.orchestra.run.smithi103.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16" 2024-09-10T18:39:54.055 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:39:54.055 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T18:39:58.547 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T18:39:58.548 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_version": "ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable)", 2024-09-10T18:39:58.548 INFO:teuthology.orchestra.run.smithi018.stdout: "image_id": "4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9", 2024-09-10T18:39:58.548 INFO:teuthology.orchestra.run.smithi018.stdout: "repo_digests": [ 2024-09-10T18:39:58.548 INFO:teuthology.orchestra.run.smithi018.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16" 2024-09-10T18:39:58.548 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:39:58.548 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T18:39:58.567 DEBUG:teuthology.orchestra.run.smithi018:> sudo mkdir -p /etc/ceph 2024-09-10T18:39:58.616 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /etc/ceph 2024-09-10T18:39:58.649 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod 777 /etc/ceph 2024-09-10T18:39:58.674 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 777 /etc/ceph 2024-09-10T18:39:58.718 INFO:tasks.cephadm:Writing seed config... 2024-09-10T18:39:58.721 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-10T18:39:58.721 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-10T18:39:58.722 INFO:tasks.cephadm: override: [client.0] debug ms = 1 2024-09-10T18:39:58.722 INFO:tasks.cephadm: override: [client.0] debug objecter = 20 2024-09-10T18:39:58.722 INFO:tasks.cephadm: override: [client.0] debug rados = 20 2024-09-10T18:39:58.722 INFO:tasks.cephadm: override: [global] ms bind msgr1 = False 2024-09-10T18:39:58.722 INFO:tasks.cephadm: override: [global] ms bind msgr2 = True 2024-09-10T18:39:58.722 INFO:tasks.cephadm: override: [global] ms type = async 2024-09-10T18:39:58.723 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-10T18:39:58.723 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-10T18:39:58.723 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-10T18:39:58.723 INFO:tasks.cephadm: override: [mon] mon min osdmap epochs = 50 2024-09-10T18:39:58.724 INFO:tasks.cephadm: override: [mon] mon osdmap full prune interval = 2 2024-09-10T18:39:58.724 INFO:tasks.cephadm: override: [mon] mon osdmap full prune min = 15 2024-09-10T18:39:58.724 INFO:tasks.cephadm: override: [mon] mon osdmap full prune txsize = 2 2024-09-10T18:39:58.724 INFO:tasks.cephadm: override: [mon] paxos service trim min = 10 2024-09-10T18:39:58.724 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-10T18:39:58.725 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-10T18:39:58.725 INFO:tasks.cephadm: override: [osd] osd debug reject backfill probability = 0.3 2024-09-10T18:39:58.725 INFO:tasks.cephadm: override: [osd] osd delete sleep = 1 2024-09-10T18:39:58.725 INFO:tasks.cephadm: override: [osd] osd max backfills = 3 2024-09-10T18:39:58.725 INFO:tasks.cephadm: override: [osd] osd scrub max interval = 120 2024-09-10T18:39:58.726 INFO:tasks.cephadm: override: [osd] osd scrub min interval = 60 2024-09-10T18:39:58.726 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-10T18:39:58.726 INFO:tasks.cephadm: override: [osd] osd snap trim sleep = 2 2024-09-10T18:39:58.727 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-10T18:39:58.727 DEBUG:teuthology.orchestra.run.smithi018:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-10T18:39:58.744 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = ba327f6c-6fa3-11ef-bcea-c7b262605968 ms bind msgr1 = False ms bind msgr2 = True ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 120 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 osd debug reject backfill probability = 0.3 osd delete sleep = 1 osd max backfills = 3 osd scrub min interval = 60 osd snap trim sleep = 2 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon min osdmap epochs = 50 mon osdmap full prune interval = 2 mon osdmap full prune min = 15 mon osdmap full prune txsize = 2 paxos service trim min = 10 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client.0] debug ms = 1 debug objecter = 20 debug rados = 20 2024-09-10T18:39:58.745 DEBUG:teuthology.orchestra.run.smithi018:mon.a> sudo journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@mon.a.service 2024-09-10T18:39:58.787 DEBUG:teuthology.orchestra.run.smithi018:mgr.y> sudo journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@mgr.y.service 2024-09-10T18:39:58.832 INFO:tasks.cephadm:Bootstrapping... 2024-09-10T18:39:58.832 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 -v bootstrap --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.18 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:39:59.050 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:39:59.050 INFO:teuthology.orchestra.run.smithi018.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588', '-v', 'bootstrap', '--fsid', 'ba327f6c-6fa3-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.18', '--skip-admin-label'] 2024-09-10T18:39:59.075 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stdout 5.2.2 2024-09-10T18:39:59.076 INFO:teuthology.orchestra.run.smithi018.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-10T18:39:59.076 INFO:teuthology.orchestra.run.smithi018.stdout:Verifying podman|docker is present... 2024-09-10T18:39:59.107 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stdout 5.2.2 2024-09-10T18:39:59.107 INFO:teuthology.orchestra.run.smithi018.stdout:Verifying lvm2 is present... 2024-09-10T18:39:59.107 INFO:teuthology.orchestra.run.smithi018.stdout:Verifying time synchronization is in place... 2024-09-10T18:39:59.115 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T18:39:59.115 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T18:39:59.122 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T18:39:59.122 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout inactive 2024-09-10T18:39:59.130 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout enabled 2024-09-10T18:39:59.136 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout active 2024-09-10T18:39:59.137 INFO:teuthology.orchestra.run.smithi018.stdout:Unit chronyd.service is enabled and running 2024-09-10T18:39:59.137 INFO:teuthology.orchestra.run.smithi018.stdout:Repeating the final host check... 2024-09-10T18:39:59.159 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stdout 5.2.2 2024-09-10T18:39:59.159 INFO:teuthology.orchestra.run.smithi018.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-10T18:39:59.160 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl is present 2024-09-10T18:39:59.160 INFO:teuthology.orchestra.run.smithi018.stdout:lvcreate is present 2024-09-10T18:39:59.166 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T18:39:59.166 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T18:39:59.173 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T18:39:59.173 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout inactive 2024-09-10T18:39:59.180 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout enabled 2024-09-10T18:39:59.189 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout active 2024-09-10T18:39:59.189 INFO:teuthology.orchestra.run.smithi018.stdout:Unit chronyd.service is enabled and running 2024-09-10T18:39:59.190 INFO:teuthology.orchestra.run.smithi018.stdout:Host looks OK 2024-09-10T18:39:59.190 INFO:teuthology.orchestra.run.smithi018.stdout:Cluster fsid: ba327f6c-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:39:59.190 INFO:teuthology.orchestra.run.smithi018.stdout:Acquiring lock 140499415724960 on /run/cephadm/ba327f6c-6fa3-11ef-bcea-c7b262605968.lock 2024-09-10T18:39:59.190 INFO:teuthology.orchestra.run.smithi018.stdout:Lock 140499415724960 acquired on /run/cephadm/ba327f6c-6fa3-11ef-bcea-c7b262605968.lock 2024-09-10T18:39:59.190 INFO:teuthology.orchestra.run.smithi018.stdout:Verifying IP 172.21.15.18 port 3300 ... 2024-09-10T18:39:59.190 INFO:teuthology.orchestra.run.smithi018.stdout:Verifying IP 172.21.15.18 port 6789 ... 2024-09-10T18:39:59.190 INFO:teuthology.orchestra.run.smithi018.stdout:Base mon IP(s) is [172.21.15.18:3300, 172.21.15.18:6789], mon addrv is [v2:172.21.15.18:3300,v1:172.21.15.18:6789] 2024-09-10T18:39:59.192 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.18 metric 100 2024-09-10T18:39:59.192 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.18 metric 100 2024-09-10T18:39:59.195 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-10T18:39:59.195 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-10T18:39:59.195 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1791sec hoplimit 64 pref medium 2024-09-10T18:39:59.198 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-10T18:39:59.198 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-10T18:39:59.198 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T18:39:59.198 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-10T18:39:59.198 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1362/64 scope link 2024-09-10T18:39:59.198 INFO:teuthology.orchestra.run.smithi018.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T18:39:59.199 INFO:teuthology.orchestra.run.smithi018.stdout:Mon IP `172.21.15.18` is in CIDR network `172.21.0.0/20` 2024-09-10T18:39:59.199 INFO:teuthology.orchestra.run.smithi018.stdout:Mon IP `172.21.15.18` is in CIDR network `172.21.0.0/20` 2024-09-10T18:39:59.199 INFO:teuthology.orchestra.run.smithi018.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-10T18:39:59.199 INFO:teuthology.orchestra.run.smithi018.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-10T18:39:59.200 INFO:teuthology.orchestra.run.smithi018.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588... 2024-09-10T18:39:59.847 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stdout 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 2024-09-10T18:39:59.848 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588... 2024-09-10T18:39:59.848 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stderr Getting image source signatures 2024-09-10T18:39:59.848 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stderr Copying blob sha256:5df13064c03763eb21a89b455ab82ba4ec435f51a3f340cae76514aaf2996d4c 2024-09-10T18:39:59.848 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-10T18:39:59.848 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stderr Copying config sha256:4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 2024-09-10T18:39:59.848 INFO:teuthology.orchestra.run.smithi018.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-10T18:40:00.751 INFO:teuthology.orchestra.run.smithi018.stdout:ceph: stdout ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable) 2024-09-10T18:40:00.751 INFO:teuthology.orchestra.run.smithi018.stdout:Ceph version: ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable) 2024-09-10T18:40:00.751 INFO:teuthology.orchestra.run.smithi018.stdout:Extracting ceph user uid/gid from container image... 2024-09-10T18:40:01.731 INFO:teuthology.orchestra.run.smithi018.stdout:stat: stdout 167 167 2024-09-10T18:40:01.731 INFO:teuthology.orchestra.run.smithi018.stdout:Creating initial keys... 2024-09-10T18:40:02.703 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph-authtool: stdout AQCCkuBm3o7IAhAAZdSqe5fgz+0PiCzEyy4uFg== 2024-09-10T18:40:03.617 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph-authtool: stdout AQCDkuBmlV8JAhAAiXyeUFsWgsEHTcMhaFQJOg== 2024-09-10T18:40:04.530 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph-authtool: stdout AQCDkuBmM+XvOBAA/gvf6/GkVk1MGzfLGIh3GQ== 2024-09-10T18:40:04.531 INFO:teuthology.orchestra.run.smithi018.stdout:Creating initial monmap... 2024-09-10T18:40:05.445 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T18:40:05.445 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-09-10T18:40:05.445 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to ba327f6c-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:40:05.445 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T18:40:05.445 INFO:teuthology.orchestra.run.smithi018.stdout:monmaptool for a [v2:172.21.15.18:3300,v1:172.21.15.18:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T18:40:05.445 INFO:teuthology.orchestra.run.smithi018.stdout:setting min_mon_release = octopus 2024-09-10T18:40:05.445 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/monmaptool: set fsid to ba327f6c-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:40:05.445 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T18:40:05.446 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:40:05.446 INFO:teuthology.orchestra.run.smithi018.stdout:Creating mon... 2024-09-10T18:40:06.383 INFO:teuthology.orchestra.run.smithi018.stdout:create mon.a on 2024-09-10T18:40:06.591 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-10T18:40:06.738 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-10T18:40:06.892 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968.target. 2024-09-10T18:40:06.892 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968.target. 2024-09-10T18:40:07.071 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@mon.a 2024-09-10T18:40:07.071 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Failed to reset failed state of unit ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-10T18:40:07.241 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968.target.wants/ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@.service. 2024-09-10T18:40:07.715 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T18:40:07.715 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout disabled 2024-09-10T18:40:07.722 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T18:40:07.722 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout inactive 2024-09-10T18:40:07.723 INFO:teuthology.orchestra.run.smithi018.stdout:firewalld.service is not enabled 2024-09-10T18:40:07.723 INFO:teuthology.orchestra.run.smithi018.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T18:40:07.723 INFO:teuthology.orchestra.run.smithi018.stdout:Waiting for mon to start... 2024-09-10T18:40:07.723 INFO:teuthology.orchestra.run.smithi018.stdout:Waiting for mon... 2024-09-10T18:40:07.826 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:07 smithi018 systemd[1]: Started Ceph mon.a for ba327f6c-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:40:08.718 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout cluster: 2024-09-10T18:40:08.718 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout id: ba327f6c-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:40:08.718 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-10T18:40:08.718 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-09-10T18:40:08.719 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout services: 2024-09-10T18:40:08.719 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.511711s) 2024-09-10T18:40:08.719 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-10T18:40:08.719 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-10T18:40:08.719 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-09-10T18:40:08.719 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout data: 2024-09-10T18:40:08.719 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-10T18:40:08.719 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-10T18:40:08.719 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-10T18:40:08.719 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout pgs: 2024-09-10T18:40:08.719 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-09-10T18:40:08.719 INFO:teuthology.orchestra.run.smithi018.stdout:mon is available 2024-09-10T18:40:08.719 INFO:teuthology.orchestra.run.smithi018.stdout:Assimilating anything we can from ceph.conf... 2024-09-10T18:40:08.839 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:08 smithi018 ceph-mon[28974]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T18:40:08.839 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:08 smithi018 ceph-mon[28974]: monmap e1: 1 mons at {a=[v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:40:08.840 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:08 smithi018 ceph-mon[28974]: fsmap 2024-09-10T18:40:08.840 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:08 smithi018 ceph-mon[28974]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T18:40:08.840 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:08 smithi018 ceph-mon[28974]: mgrmap e1: no daemons active 2024-09-10T18:40:08.840 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:08 smithi018 ceph-mon[28974]: from='client.? 172.21.15.18:0/930415251' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-10T18:40:09.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:09 smithi018 ceph-mon[28974]: from='client.? 172.21.15.18:0/3271353070' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T18:40:09.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:09 smithi018 ceph-mon[28974]: from='client.? 172.21.15.18:0/3271353070' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-10T18:40:09.709 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-09-10T18:40:09.710 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T18:40:09.710 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout fsid = ba327f6c-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:40:09.710 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-10T18:40:09.710 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.18:3300,v1:172.21.15.18:6789] 2024-09-10T18:40:09.710 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T18:40:09.710 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T18:40:09.710 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T18:40:09.710 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T18:40:09.710 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-09-10T18:40:09.710 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T18:40:09.710 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T18:40:09.710 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-09-10T18:40:09.711 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T18:40:09.711 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T18:40:09.711 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T18:40:09.711 INFO:teuthology.orchestra.run.smithi018.stdout:Generating new minimal ceph.conf... 2024-09-10T18:40:10.657 INFO:teuthology.orchestra.run.smithi018.stdout:Restarting the monitor... 2024-09-10T18:40:10.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:10 smithi018 ceph-mon[28974]: from='client.? 172.21.15.18:0/775078178' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:10.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:10 smithi018 systemd[1]: Stopping Ceph mon.a for ba327f6c-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:40:11.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:10 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mon-a[28970]: 2024-09-10T18:40:10.922+0000 7fdd72ef0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:40:11.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:10 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mon-a[28970]: 2024-09-10T18:40:10.922+0000 7fdd72ef0640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-10T18:40:11.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:11 smithi018 podman[29216]: 2024-09-10 18:40:11.106966836 +0000 UTC m=+0.287088922 container died ef02df493aaa50bff0a82219074fde9c8e8eb18667d7d460cbfc3632e1ed6f4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-10T18:40:11.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:11 smithi018 podman[29216]: 2024-09-10 18:40:11.45541286 +0000 UTC m=+0.635534946 container cleanup ef02df493aaa50bff0a82219074fde9c8e8eb18667d7d460cbfc3632e1ed6f4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:40:11.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:11 smithi018 bash[29216]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mon-a 2024-09-10T18:40:12.083 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:11 smithi018 podman[29229]: 2024-09-10 18:40:11.722956053 +0000 UTC m=+0.610998627 container remove ef02df493aaa50bff0a82219074fde9c8e8eb18667d7d460cbfc3632e1ed6f4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:40:12.083 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:11 smithi018 systemd[1]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-10T18:40:12.083 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:11 smithi018 systemd[1]: Stopped Ceph mon.a for ba327f6c-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:40:12.083 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:11 smithi018 systemd[1]: Starting Ceph mon.a for ba327f6c-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:40:12.360 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 podman[29317]: 2024-09-10 18:40:12.082079547 +0000 UTC m=+0.104554896 container create c1f97cd118ccbc2e0de9e6e30b48098f1b3fe3cefb330b4408ff67b690c64cb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mon-a, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-10T18:40:12.361 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 podman[29317]: 2024-09-10 18:40:11.993964978 +0000 UTC m=+0.016440338 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:40:12.361 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 podman[29317]: 2024-09-10 18:40:12.28427609 +0000 UTC m=+0.306751438 container init c1f97cd118ccbc2e0de9e6e30b48098f1b3fe3cefb330b4408ff67b690c64cb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mon-a, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-10T18:40:12.361 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 podman[29317]: 2024-09-10 18:40:12.287452481 +0000 UTC m=+0.309927834 container start c1f97cd118ccbc2e0de9e6e30b48098f1b3fe3cefb330b4408ff67b690c64cb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:40:12.361 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T18:40:12.361 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable), process ceph-mon, pid 2 2024-09-10T18:40:12.361 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: pidfile_write: ignore empty --pid-file 2024-09-10T18:40:12.361 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: load: jerasure load: lrc 2024-09-10T18:40:12.361 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: RocksDB version: 6.15.5 2024-09-10T18:40:12.361 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-10T18:40:12.361 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Compile date Sep 9 2024 2024-09-10T18:40:12.362 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: DB SUMMARY 2024-09-10T18:40:12.362 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: DB Session ID: W4UGDE0VXL9MV6Z7N1L5 2024-09-10T18:40:12.362 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: CURRENT file: CURRENT 2024-09-10T18:40:12.362 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T18:40:12.362 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-09-10T18:40:12.362 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-10T18:40:12.362 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000010.log size: 89926 ; 2024-09-10T18:40:12.363 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.error_if_exists: 0 2024-09-10T18:40:12.363 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.create_if_missing: 0 2024-09-10T18:40:12.363 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.paranoid_checks: 1 2024-09-10T18:40:12.363 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T18:40:12.364 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.env: 0x5571264d7440 2024-09-10T18:40:12.364 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.fs: Posix File System 2024-09-10T18:40:12.364 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.info_log: 0x557128b98ec0 2024-09-10T18:40:12.364 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T18:40:12.364 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.statistics: (nil) 2024-09-10T18:40:12.364 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.use_fsync: 0 2024-09-10T18:40:12.364 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.max_log_file_size: 0 2024-09-10T18:40:12.364 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T18:40:12.364 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T18:40:12.364 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T18:40:12.365 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T18:40:12.365 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.allow_fallocate: 1 2024-09-10T18:40:12.365 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T18:40:12.365 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T18:40:12.365 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.use_direct_reads: 0 2024-09-10T18:40:12.365 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T18:40:12.365 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T18:40:12.365 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.db_log_dir: 2024-09-10T18:40:12.365 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2024-09-10T18:40:12.365 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T18:40:12.366 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T18:40:12.366 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T18:40:12.366 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T18:40:12.366 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T18:40:12.366 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T18:40:12.366 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T18:40:12.366 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T18:40:12.366 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.write_buffer_manager: 0x557128c567e0 2024-09-10T18:40:12.366 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T18:40:12.366 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-10T18:40:12.367 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T18:40:12.367 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T18:40:12.367 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T18:40:12.367 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T18:40:12.367 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T18:40:12.367 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T18:40:12.367 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T18:40:12.367 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.unordered_write: 0 2024-09-10T18:40:12.367 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T18:40:12.368 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T18:40:12.368 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T18:40:12.368 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T18:40:12.368 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.row_cache: None 2024-09-10T18:40:12.368 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.wal_filter: None 2024-09-10T18:40:12.368 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T18:40:12.368 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T18:40:12.368 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.preserve_deletes: 0 2024-09-10T18:40:12.368 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.two_write_queues: 0 2024-09-10T18:40:12.368 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T18:40:12.369 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.atomic_flush: 0 2024-09-10T18:40:12.369 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T18:40:12.369 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T18:40:12.369 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T18:40:12.369 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.log_readahead_size: 0 2024-09-10T18:40:12.369 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T18:40:12.369 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T18:40:12.369 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T18:40:12.369 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T18:40:12.370 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T18:40:12.370 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T18:40:12.370 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.max_background_jobs: 2 2024-09-10T18:40:12.370 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.max_background_compactions: -1 2024-09-10T18:40:12.370 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.max_subcompactions: 1 2024-09-10T18:40:12.370 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T18:40:12.370 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T18:40:12.370 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T18:40:12.370 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T18:40:12.370 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T18:40:12.370 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T18:40:12.371 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T18:40:12.371 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T18:40:12.371 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.max_open_files: -1 2024-09-10T18:40:12.371 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T18:40:12.371 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T18:40:12.371 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T18:40:12.371 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T18:40:12.371 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.max_background_flushes: -1 2024-09-10T18:40:12.371 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Compression algorithms supported: 2024-09-10T18:40:12.371 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: kZSTD supported: 0 2024-09-10T18:40:12.372 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: kXpressCompression supported: 0 2024-09-10T18:40:12.372 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: kBZip2Compression supported: 0 2024-09-10T18:40:12.372 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T18:40:12.372 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: kLZ4Compression supported: 1 2024-09-10T18:40:12.372 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: kZlibCompression supported: 1 2024-09-10T18:40:12.372 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T18:40:12.372 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: kSnappyCompression supported: 1 2024-09-10T18:40:12.372 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T18:40:12.372 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 2024-09-10T18:40:12.372 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-10T18:40:12.373 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T18:40:12.373 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.merge_operator: 2024-09-10T18:40:12.373 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.compaction_filter: None 2024-09-10T18:40:12.373 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T18:40:12.373 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T18:40:12.373 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T18:40:12.373 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T18:40:12.373 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557128b3fa38) 2024-09-10T18:40:12.373 INFO:journalctl@ceph.mon.a.smithi018.stdout: cache_index_and_filter_blocks: 1 2024-09-10T18:40:12.373 INFO:journalctl@ceph.mon.a.smithi018.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T18:40:12.374 INFO:journalctl@ceph.mon.a.smithi018.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T18:40:12.374 INFO:journalctl@ceph.mon.a.smithi018.stdout: pin_top_level_index_and_filter: 1 2024-09-10T18:40:12.374 INFO:journalctl@ceph.mon.a.smithi018.stdout: index_type: 0 2024-09-10T18:40:12.374 INFO:journalctl@ceph.mon.a.smithi018.stdout: data_block_index_type: 0 2024-09-10T18:40:12.374 INFO:journalctl@ceph.mon.a.smithi018.stdout: index_shortening: 1 2024-09-10T18:40:12.374 INFO:journalctl@ceph.mon.a.smithi018.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T18:40:12.374 INFO:journalctl@ceph.mon.a.smithi018.stdout: hash_index_allow_collision: 1 2024-09-10T18:40:12.374 INFO:journalctl@ceph.mon.a.smithi018.stdout: checksum: 1 2024-09-10T18:40:12.374 INFO:journalctl@ceph.mon.a.smithi018.stdout: no_block_cache: 0 2024-09-10T18:40:12.374 INFO:journalctl@ceph.mon.a.smithi018.stdout: block_cache: 0x557128b751f0 2024-09-10T18:40:12.375 INFO:journalctl@ceph.mon.a.smithi018.stdout: block_cache_name: BinnedLRUCache 2024-09-10T18:40:12.375 INFO:journalctl@ceph.mon.a.smithi018.stdout: block_cache_options: 2024-09-10T18:40:12.375 INFO:journalctl@ceph.mon.a.smithi018.stdout: capacity : 536870912 2024-09-10T18:40:12.375 INFO:journalctl@ceph.mon.a.smithi018.stdout: num_shard_bits : 4 2024-09-10T18:40:12.375 INFO:journalctl@ceph.mon.a.smithi018.stdout: strict_capacity_limit : 0 2024-09-10T18:40:12.375 INFO:journalctl@ceph.mon.a.smithi018.stdout: high_pri_pool_ratio: 0.000 2024-09-10T18:40:12.375 INFO:journalctl@ceph.mon.a.smithi018.stdout: block_cache_compressed: (nil) 2024-09-10T18:40:12.375 INFO:journalctl@ceph.mon.a.smithi018.stdout: persistent_cache: (nil) 2024-09-10T18:40:12.375 INFO:journalctl@ceph.mon.a.smithi018.stdout: block_size: 4096 2024-09-10T18:40:12.375 INFO:journalctl@ceph.mon.a.smithi018.stdout: block_size_deviation: 10 2024-09-10T18:40:12.375 INFO:journalctl@ceph.mon.a.smithi018.stdout: block_restart_interval: 16 2024-09-10T18:40:12.376 INFO:journalctl@ceph.mon.a.smithi018.stdout: index_block_restart_interval: 1 2024-09-10T18:40:12.376 INFO:journalctl@ceph.mon.a.smithi018.stdout: metadata_block_size: 4096 2024-09-10T18:40:12.376 INFO:journalctl@ceph.mon.a.smithi018.stdout: partition_filters: 0 2024-09-10T18:40:12.376 INFO:journalctl@ceph.mon.a.smithi018.stdout: use_delta_encoding: 1 2024-09-10T18:40:12.376 INFO:journalctl@ceph.mon.a.smithi018.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-10T18:40:12.376 INFO:journalctl@ceph.mon.a.smithi018.stdout: whole_key_filtering: 1 2024-09-10T18:40:12.376 INFO:journalctl@ceph.mon.a.smithi018.stdout: verify_compression: 0 2024-09-10T18:40:12.376 INFO:journalctl@ceph.mon.a.smithi018.stdout: read_amp_bytes_per_bit: 0 2024-09-10T18:40:12.376 INFO:journalctl@ceph.mon.a.smithi018.stdout: format_version: 4 2024-09-10T18:40:12.376 INFO:journalctl@ceph.mon.a.smithi018.stdout: enable_index_compression: 1 2024-09-10T18:40:12.376 INFO:journalctl@ceph.mon.a.smithi018.stdout: block_align: 0 2024-09-10T18:40:12.377 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T18:40:12.377 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T18:40:12.377 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.compression: NoCompression 2024-09-10T18:40:12.377 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T18:40:12.377 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T18:40:12.377 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T18:40:12.377 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.num_levels: 7 2024-09-10T18:40:12.377 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T18:40:12.377 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T18:40:12.377 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T18:40:12.377 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T18:40:12.378 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T18:40:12.378 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T18:40:12.378 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T18:40:12.378 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:40:12.378 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T18:40:12.378 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T18:40:12.378 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T18:40:12.378 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T18:40:12.378 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T18:40:12.378 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T18:40:12.378 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:40:12.379 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T18:40:12.379 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T18:40:12.379 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T18:40:12.379 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T18:40:12.379 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T18:40:12.379 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T18:40:12.379 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T18:40:12.379 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T18:40:12.379 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T18:40:12.379 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T18:40:12.380 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T18:40:12.380 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T18:40:12.380 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T18:40:12.380 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T18:40:12.380 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T18:40:12.380 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T18:40:12.380 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T18:40:12.380 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T18:40:12.381 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T18:40:12.381 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.arena_block_size: 4194304 2024-09-10T18:40:12.381 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T18:40:12.381 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T18:40:12.381 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-10T18:40:12.381 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T18:40:12.381 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T18:40:12.381 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T18:40:12.381 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T18:40:12.381 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T18:40:12.381 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T18:40:12.382 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T18:40:12.382 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T18:40:12.382 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T18:40:12.382 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T18:40:12.382 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T18:40:12.382 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T18:40:12.382 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.inplace_update_support: 0 2024-09-10T18:40:12.382 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T18:40:12.382 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T18:40:12.382 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T18:40:12.382 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T18:40:12.382 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.bloom_locality: 0 2024-09-10T18:40:12.383 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.max_successive_merges: 0 2024-09-10T18:40:12.383 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T18:40:12.383 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T18:40:12.383 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T18:40:12.383 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T18:40:12.383 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.ttl: 2592000 2024-09-10T18:40:12.383 INFO:teuthology.orchestra.run.smithi018.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-10T18:40:12.384 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T18:40:12.384 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.enable_blob_files: false 2024-09-10T18:40:12.384 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.min_blob_size: 0 2024-09-10T18:40:12.384 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T18:40:12.384 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T18:40:12.385 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T18:40:12.385 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T18:40:12.385 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T18:40:12.385 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-09-10T18:40:12.385 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-09-10T18:40:12.385 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725993612311170, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-09-10T18:40:12.385 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-09-10T18:40:12.385 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-10T18:40:12.385 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725993612311921, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 85902, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 83994, "index_size": 242, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 11534, "raw_average_key_size": 48, "raw_value_size": 77670, "raw_average_value_size": 327, "num_data_blocks": 10, "num_entries": 237, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1725993612, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "f88c6d3f-daba-49c4-b1e2-de309ce1d7a8", "db_session_id": "W4UGDE0VXL9MV6Z7N1L5"}} 2024-09-10T18:40:12.386 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-09-10T18:40:12.386 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725993612312293, "job": 1, "event": "recovery_finished"} 2024-09-10T18:40:12.386 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T18:40:12.386 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x557128ba8700 2024-09-10T18:40:12.386 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: DB pointer 0x557128b78000 2024-09-10T18:40:12.386 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-10T18:40:12.386 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-10T18:40:12.386 INFO:journalctl@ceph.mon.a.smithi018.stdout: ** DB Stats ** 2024-09-10T18:40:12.386 INFO:journalctl@ceph.mon.a.smithi018.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:40:12.386 INFO:journalctl@ceph.mon.a.smithi018.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T18:40:12.386 INFO:journalctl@ceph.mon.a.smithi018.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T18:40:12.386 INFO:journalctl@ceph.mon.a.smithi018.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:40:12.387 INFO:journalctl@ceph.mon.a.smithi018.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T18:40:12.387 INFO:journalctl@ceph.mon.a.smithi018.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-10T18:40:12.387 INFO:journalctl@ceph.mon.a.smithi018.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:40:12.387 INFO:journalctl@ceph.mon.a.smithi018.stdout: 2024-09-10T18:40:12.387 INFO:journalctl@ceph.mon.a.smithi018.stdout: ** Compaction Stats [default] ** 2024-09-10T18:40:12.387 INFO:journalctl@ceph.mon.a.smithi018.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:40:12.387 INFO:journalctl@ceph.mon.a.smithi018.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:40:12.387 INFO:journalctl@ceph.mon.a.smithi018.stdout: L0 2/0 85.67 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 173.6 0.00 0.00 1 0.000 0 0 2024-09-10T18:40:12.387 INFO:journalctl@ceph.mon.a.smithi018.stdout: Sum 2/0 85.67 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 173.6 0.00 0.00 1 0.000 0 0 2024-09-10T18:40:12.387 INFO:journalctl@ceph.mon.a.smithi018.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 173.6 0.00 0.00 1 0.000 0 0 2024-09-10T18:40:12.388 INFO:journalctl@ceph.mon.a.smithi018.stdout: 2024-09-10T18:40:12.388 INFO:journalctl@ceph.mon.a.smithi018.stdout: ** Compaction Stats [default] ** 2024-09-10T18:40:12.388 INFO:journalctl@ceph.mon.a.smithi018.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:40:12.388 INFO:journalctl@ceph.mon.a.smithi018.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:40:12.388 INFO:journalctl@ceph.mon.a.smithi018.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 173.6 0.00 0.00 1 0.000 0 0 2024-09-10T18:40:12.388 INFO:journalctl@ceph.mon.a.smithi018.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:40:12.388 INFO:journalctl@ceph.mon.a.smithi018.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:40:12.388 INFO:journalctl@ceph.mon.a.smithi018.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:40:12.388 INFO:journalctl@ceph.mon.a.smithi018.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:40:12.388 INFO:journalctl@ceph.mon.a.smithi018.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:40:12.388 INFO:journalctl@ceph.mon.a.smithi018.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:40:12.388 INFO:journalctl@ceph.mon.a.smithi018.stdout: Cumulative compaction: 0.00 GB write, 25.83 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:40:12.389 INFO:journalctl@ceph.mon.a.smithi018.stdout: Interval compaction: 0.00 GB write, 25.83 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:40:12.389 INFO:journalctl@ceph.mon.a.smithi018.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:40:12.389 INFO:journalctl@ceph.mon.a.smithi018.stdout: 2024-09-10T18:40:12.389 INFO:journalctl@ceph.mon.a.smithi018.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:40:12.389 INFO:journalctl@ceph.mon.a.smithi018.stdout: 2024-09-10T18:40:12.389 INFO:journalctl@ceph.mon.a.smithi018.stdout: ** Compaction Stats [default] ** 2024-09-10T18:40:12.389 INFO:journalctl@ceph.mon.a.smithi018.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:40:12.389 INFO:journalctl@ceph.mon.a.smithi018.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:40:12.389 INFO:journalctl@ceph.mon.a.smithi018.stdout: L0 2/0 85.67 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 173.6 0.00 0.00 1 0.000 0 0 2024-09-10T18:40:12.389 INFO:journalctl@ceph.mon.a.smithi018.stdout: Sum 2/0 85.67 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 173.6 0.00 0.00 1 0.000 0 0 2024-09-10T18:40:12.390 INFO:journalctl@ceph.mon.a.smithi018.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-10T18:40:12.390 INFO:journalctl@ceph.mon.a.smithi018.stdout: 2024-09-10T18:40:12.390 INFO:journalctl@ceph.mon.a.smithi018.stdout: ** Compaction Stats [default] ** 2024-09-10T18:40:12.390 INFO:journalctl@ceph.mon.a.smithi018.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:40:12.390 INFO:journalctl@ceph.mon.a.smithi018.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:40:12.390 INFO:journalctl@ceph.mon.a.smithi018.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 173.6 0.00 0.00 1 0.000 0 0 2024-09-10T18:40:12.390 INFO:journalctl@ceph.mon.a.smithi018.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:40:12.390 INFO:journalctl@ceph.mon.a.smithi018.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:40:12.390 INFO:journalctl@ceph.mon.a.smithi018.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:40:12.390 INFO:journalctl@ceph.mon.a.smithi018.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:40:12.390 INFO:journalctl@ceph.mon.a.smithi018.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:40:12.391 INFO:journalctl@ceph.mon.a.smithi018.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:40:12.391 INFO:journalctl@ceph.mon.a.smithi018.stdout: Cumulative compaction: 0.00 GB write, 25.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:40:12.391 INFO:journalctl@ceph.mon.a.smithi018.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:40:12.391 INFO:journalctl@ceph.mon.a.smithi018.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:40:12.391 INFO:journalctl@ceph.mon.a.smithi018.stdout: 2024-09-10T18:40:12.391 INFO:journalctl@ceph.mon.a.smithi018.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:40:12.391 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: starting mon.a rank 0 at public addrs [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] at bind addrs [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:40:12.391 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: mon.a@-1(???) e1 preinit fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:40:12.391 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: mon.a@-1(???).mds e1 new map 2024-09-10T18:40:12.391 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: mon.a@-1(???).mds e1 print_map 2024-09-10T18:40:12.391 INFO:journalctl@ceph.mon.a.smithi018.stdout: e1 2024-09-10T18:40:12.392 INFO:journalctl@ceph.mon.a.smithi018.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T18:40:12.392 INFO:journalctl@ceph.mon.a.smithi018.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T18:40:12.392 INFO:journalctl@ceph.mon.a.smithi018.stdout: legacy client fscid: -1 2024-09-10T18:40:12.392 INFO:journalctl@ceph.mon.a.smithi018.stdout: 2024-09-10T18:40:12.392 INFO:journalctl@ceph.mon.a.smithi018.stdout: No filesystems configured 2024-09-10T18:40:12.392 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T18:40:12.392 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:40:12.392 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:40:12.392 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:40:12.392 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-10T18:40:12.392 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 bash[29317]: c1f97cd118ccbc2e0de9e6e30b48098f1b3fe3cefb330b4408ff67b690c64cb6 2024-09-10T18:40:12.393 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 systemd[1]: Started Ceph mon.a for ba327f6c-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:40:12.655 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T18:40:12.656 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: monmap e1: 1 mons at {a=[v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:40:12.656 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: fsmap 2024-09-10T18:40:12.656 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T18:40:12.656 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:12 smithi018 ceph-mon[29336]: mgrmap e1: no daemons active 2024-09-10T18:40:13.294 INFO:teuthology.orchestra.run.smithi018.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-10T18:40:13.295 INFO:teuthology.orchestra.run.smithi018.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:40:13.295 INFO:teuthology.orchestra.run.smithi018.stdout:Creating mgr... 2024-09-10T18:40:13.295 INFO:teuthology.orchestra.run.smithi018.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-10T18:40:13.295 INFO:teuthology.orchestra.run.smithi018.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-10T18:40:13.480 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@mgr.y 2024-09-10T18:40:13.480 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Failed to reset failed state of unit ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@mgr.y.service: Unit ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@mgr.y.service not loaded. 2024-09-10T18:40:13.634 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968.target.wants/ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@.service. 2024-09-10T18:40:13.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:13 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1675063615' entity='client.admin' 2024-09-10T18:40:13.945 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:13 smithi018 podman[29549]: 2024-09-10 18:40:13.766115307 +0000 UTC m=+0.016267976 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:40:14.134 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T18:40:14.135 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout disabled 2024-09-10T18:40:14.142 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T18:40:14.142 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout inactive 2024-09-10T18:40:14.142 INFO:teuthology.orchestra.run.smithi018.stdout:firewalld.service is not enabled 2024-09-10T18:40:14.143 INFO:teuthology.orchestra.run.smithi018.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T18:40:14.152 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T18:40:14.152 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout disabled 2024-09-10T18:40:14.160 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T18:40:14.160 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout inactive 2024-09-10T18:40:14.160 INFO:teuthology.orchestra.run.smithi018.stdout:firewalld.service is not enabled 2024-09-10T18:40:14.161 INFO:teuthology.orchestra.run.smithi018.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-10T18:40:14.161 INFO:teuthology.orchestra.run.smithi018.stdout:Waiting for mgr to start... 2024-09-10T18:40:14.161 INFO:teuthology.orchestra.run.smithi018.stdout:Waiting for mgr... 2024-09-10T18:40:14.244 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:14 smithi018 podman[29549]: 2024-09-10 18:40:14.060667622 +0000 UTC m=+0.310820296 container init 91e59d2b2d2d2712e6adec4fbc7c43897964e4e5cf20eb2b0706ecee8af784aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-10T18:40:14.244 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:14 smithi018 podman[29549]: 2024-09-10 18:40:14.06352588 +0000 UTC m=+0.313678539 container start 91e59d2b2d2d2712e6adec4fbc7c43897964e4e5cf20eb2b0706ecee8af784aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T18:40:14.244 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:14 smithi018 bash[29549]: 91e59d2b2d2d2712e6adec4fbc7c43897964e4e5cf20eb2b0706ecee8af784aa 2024-09-10T18:40:14.244 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:14 smithi018 systemd[1]: Started Ceph mgr.y for ba327f6c-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:40:14.244 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:14 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:14.184+0000 7fde966e0ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:40:14.245 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:14 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:14.242+0000 7fde966e0ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:40:14.548 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:14 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:14.309+0000 7fde966e0ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:40:14.548 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:14 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:14.443+0000 7fde966e0ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:40:14.801 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:14 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:14.587+0000 7fde966e0ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:40:15.102 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:14 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1889024002' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T18:40:15.103 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:14 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:14.867+0000 7fde966e0ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:40:15.103 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:14 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:14.929+0000 7fde966e0ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:40:15.103 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:15 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:15.100+0000 7fde966e0ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:40:15.295 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-09-10T18:40:15.298 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:40:15.298 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "fsid": "ba327f6c-6fa3-11ef-bcea-c7b262605968", 2024-09-10T18:40:15.298 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T18:40:15.298 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T18:40:15.298 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T18:40:15.299 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T18:40:15.299 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:40:15.299 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T18:40:15.299 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T18:40:15.299 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 0 2024-09-10T18:40:15.299 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:40:15.299 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T18:40:15.299 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T18:40:15.299 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:40:15.299 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-10T18:40:15.299 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T18:40:15.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:40:15.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-10T18:40:15.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T18:40:15.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:40:15.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T18:40:15.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:40:15.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T18:40:15.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T18:40:15.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T18:40:15.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T18:40:15.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T18:40:15.300 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T18:40:15.303 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:40:15.304 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T18:40:15.304 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T18:40:15.304 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T18:40:15.304 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T18:40:15.304 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T18:40:15.304 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T18:40:15.304 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T18:40:15.304 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T18:40:15.304 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T18:40:15.304 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:40:15.304 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T18:40:15.304 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:40:15.304 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T18:40:15.305 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T18:40:15.305 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:40:15.305 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T18:40:15.305 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T18:40:15.305 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T18:40:15.305 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T18:40:15.305 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T18:40:15.305 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T18:40:15.305 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T18:40:15.305 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:40:15.305 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:40:15.305 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:40:15.305 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T18:40:15.305 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:40:15.306 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T18:40:07.670141+0000", 2024-09-10T18:40:15.306 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:40:15.306 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:40:15.306 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T18:40:15.306 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:40:15.306 INFO:teuthology.orchestra.run.smithi018.stdout:mgr not available, waiting (1/15)... 2024-09-10T18:40:16.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:15 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:15.983+0000 7fde966e0ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:40:16.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:16 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:16.050+0000 7fde966e0ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:40:16.922 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:16 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:16.804+0000 7fde966e0ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:40:17.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:16 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:16.920+0000 7fde966e0ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:40:17.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:17 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:17.045+0000 7fde966e0ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:40:17.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:17 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:17.107+0000 7fde966e0ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:40:17.504 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:17 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:17.238+0000 7fde966e0ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:40:17.504 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:17 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:17.307+0000 7fde966e0ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:40:17.504 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:17 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:17.370+0000 7fde966e0ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:40:17.758 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:17 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:17.515+0000 7fde966e0ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:40:17.758 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:17 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:17.582+0000 7fde966e0ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:40:17.758 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:17 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:17.651+0000 7fde966e0ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:40:17.758 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:17 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:17.737+0000 7fde966e0ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:40:18.124 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:17 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3996525710' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T18:40:18.124 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:17 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:17.800+0000 7fde966e0ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:40:18.124 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:17 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:17.860+0000 7fde966e0ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:40:18.393 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-09-10T18:40:18.394 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:40:18.394 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "fsid": "ba327f6c-6fa3-11ef-bcea-c7b262605968", 2024-09-10T18:40:18.394 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T18:40:18.394 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T18:40:18.394 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T18:40:18.394 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T18:40:18.394 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:40:18.395 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T18:40:18.395 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T18:40:18.395 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 0 2024-09-10T18:40:18.395 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:40:18.395 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T18:40:18.395 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T18:40:18.395 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:40:18.396 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-10T18:40:18.396 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T18:40:18.396 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:40:18.396 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-10T18:40:18.396 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T18:40:18.396 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:40:18.396 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T18:40:18.397 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:40:18.397 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T18:40:18.397 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T18:40:18.397 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T18:40:18.397 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T18:40:18.397 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T18:40:18.397 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T18:40:18.397 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:40:18.397 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T18:40:18.398 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T18:40:18.398 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T18:40:18.398 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T18:40:18.398 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T18:40:18.398 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T18:40:18.398 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T18:40:18.398 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T18:40:18.398 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T18:40:18.399 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:40:18.399 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T18:40:18.399 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:40:18.399 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T18:40:18.399 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T18:40:18.399 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:40:18.399 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T18:40:18.399 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T18:40:18.400 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T18:40:18.400 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T18:40:18.400 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T18:40:18.400 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T18:40:18.400 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T18:40:18.400 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:40:18.400 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:40:18.400 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:40:18.401 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T18:40:18.401 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:40:18.401 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T18:40:07.670141+0000", 2024-09-10T18:40:18.401 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:40:18.401 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:40:18.401 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T18:40:18.401 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:40:18.401 INFO:teuthology.orchestra.run.smithi018.stdout:mgr not available, waiting (2/15)... 2024-09-10T18:40:18.944 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:18 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:18.648+0000 7fde966e0ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:40:18.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:18 smithi018 ceph-mon[29336]: Activating manager daemon y 2024-09-10T18:40:18.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:18 smithi018 ceph-mon[29336]: mgrmap e2: y(active, starting, since 0.00300936s) 2024-09-10T18:40:18.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:18 smithi018 ceph-mon[29336]: from='mgr.14100 172.21.15.18:0/3559954471' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:40:18.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:18 smithi018 ceph-mon[29336]: from='mgr.14100 172.21.15.18:0/3559954471' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:40:18.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:18 smithi018 ceph-mon[29336]: from='mgr.14100 172.21.15.18:0/3559954471' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:40:18.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:18 smithi018 ceph-mon[29336]: from='mgr.14100 172.21.15.18:0/3559954471' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:40:18.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:18 smithi018 ceph-mon[29336]: from='mgr.14100 172.21.15.18:0/3559954471' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:40:18.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:18 smithi018 ceph-mon[29336]: Manager daemon y is now available 2024-09-10T18:40:18.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:18 smithi018 ceph-mon[29336]: from='mgr.14100 172.21.15.18:0/3559954471' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:40:18.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:18 smithi018 ceph-mon[29336]: from='mgr.14100 172.21.15.18:0/3559954471' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:40:18.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:18 smithi018 ceph-mon[29336]: from='mgr.14100 172.21.15.18:0/3559954471' entity='mgr.y' 2024-09-10T18:40:18.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:18 smithi018 ceph-mon[29336]: from='mgr.14100 172.21.15.18:0/3559954471' entity='mgr.y' 2024-09-10T18:40:18.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:18 smithi018 ceph-mon[29336]: from='mgr.14100 172.21.15.18:0/3559954471' entity='mgr.y' 2024-09-10T18:40:20.801 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:20 smithi018 ceph-mon[29336]: mgrmap e3: y(active, since 1.00662s) 2024-09-10T18:40:21.577 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-09-10T18:40:21.577 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:40:21.577 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "fsid": "ba327f6c-6fa3-11ef-bcea-c7b262605968", 2024-09-10T18:40:21.577 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T18:40:21.577 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T18:40:21.577 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T18:40:21.577 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T18:40:21.577 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:40:21.577 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T18:40:21.577 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T18:40:21.577 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 0 2024-09-10T18:40:21.578 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:40:21.578 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T18:40:21.578 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T18:40:21.578 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:40:21.578 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-09-10T18:40:21.578 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T18:40:21.578 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:40:21.578 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-10T18:40:21.578 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T18:40:21.578 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:40:21.578 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T18:40:21.578 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:40:21.578 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T18:40:21.579 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T18:40:21.579 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T18:40:21.579 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T18:40:21.579 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T18:40:21.579 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T18:40:21.583 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:40:21.583 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T18:40:21.583 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T18:40:21.583 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T18:40:21.583 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T18:40:21.583 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T18:40:21.583 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T18:40:21.583 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T18:40:21.583 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T18:40:21.583 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T18:40:21.584 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:40:21.584 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T18:40:21.584 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:40:21.584 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T18:40:21.584 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T18:40:21.584 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:40:21.584 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T18:40:21.584 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T18:40:21.584 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T18:40:21.584 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T18:40:21.584 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T18:40:21.584 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T18:40:21.584 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T18:40:21.585 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:40:21.585 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:40:21.585 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:40:21.585 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T18:40:21.585 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:40:21.585 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T18:40:07.670141+0000", 2024-09-10T18:40:21.585 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:40:21.585 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:40:21.585 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T18:40:21.586 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:40:21.586 INFO:teuthology.orchestra.run.smithi018.stdout:mgr is available 2024-09-10T18:40:21.899 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:21 smithi018 ceph-mon[29336]: mgrmap e4: y(active, since 2s) 2024-09-10T18:40:21.899 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:21 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3791551782' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T18:40:22.715 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-09-10T18:40:22.715 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T18:40:22.715 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout fsid = ba327f6c-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:40:22.715 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-10T18:40:22.715 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T18:40:22.715 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T18:40:22.715 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T18:40:22.715 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T18:40:22.715 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-09-10T18:40:22.716 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T18:40:22.716 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T18:40:22.716 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 2024-09-10T18:40:22.716 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T18:40:22.716 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T18:40:22.716 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T18:40:22.716 INFO:teuthology.orchestra.run.smithi018.stdout:Enabling cephadm module... 2024-09-10T18:40:22.936 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:22 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2445688982' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T18:40:24.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:24 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/721531726' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-10T18:40:24.524 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:24 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ignoring --setuser ceph since I am not root 2024-09-10T18:40:24.524 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:24 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ignoring --setgroup ceph since I am not root 2024-09-10T18:40:24.525 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:24 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:24.396+0000 7f3c10519ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:40:24.525 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:24 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:24.454+0000 7f3c10519ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:40:24.525 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:24 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:24.522+0000 7f3c10519ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:40:24.797 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:24 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:24.651+0000 7f3c10519ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:40:24.797 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:24 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:24.795+0000 7f3c10519ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:40:25.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:25 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/721531726' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-10T18:40:25.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:25 smithi018 ceph-mon[29336]: mgrmap e5: y(active, since 5s) 2024-09-10T18:40:25.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:25 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:25.065+0000 7f3c10519ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:40:25.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:25 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:25.128+0000 7f3c10519ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:40:25.445 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:25 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:25.300+0000 7f3c10519ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:40:26.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:26 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:26.191+0000 7f3c10519ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:40:26.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:26 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:26.259+0000 7f3c10519ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:40:27.324 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:27.014+0000 7f3c10519ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:40:27.324 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:27.132+0000 7f3c10519ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:40:27.324 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:27.259+0000 7f3c10519ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:40:27.324 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:27.322+0000 7f3c10519ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:40:27.577 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:27.458+0000 7f3c10519ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:40:27.577 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:27.530+0000 7f3c10519ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:40:27.880 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:27.594+0000 7f3c10519ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:40:27.880 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:27.738+0000 7f3c10519ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:40:27.880 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:27.807+0000 7f3c10519ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:40:27.880 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:27.878+0000 7f3c10519ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:40:28.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:27.964+0000 7f3c10519ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:40:28.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:28 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:28.025+0000 7f3c10519ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:40:28.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:28 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:28.087+0000 7f3c10519ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:40:29.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:28 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:28.877+0000 7f3c10519ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:40:29.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:28 smithi018 ceph-mon[29336]: Active manager daemon y restarted 2024-09-10T18:40:29.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:28 smithi018 ceph-mon[29336]: Activating manager daemon y 2024-09-10T18:40:29.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:28 smithi018 ceph-mon[29336]: osdmap e2: 0 total, 0 up, 0 in 2024-09-10T18:40:29.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:28 smithi018 ceph-mon[29336]: mgrmap e6: y(active, starting, since 0.00315526s) 2024-09-10T18:40:29.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:28 smithi018 ceph-mon[29336]: from='mgr.14118 172.21.15.18:0/819892295' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:40:29.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:28 smithi018 ceph-mon[29336]: from='mgr.14118 172.21.15.18:0/819892295' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:40:29.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:28 smithi018 ceph-mon[29336]: from='mgr.14118 172.21.15.18:0/819892295' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:40:29.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:28 smithi018 ceph-mon[29336]: from='mgr.14118 172.21.15.18:0/819892295' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:40:29.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:28 smithi018 ceph-mon[29336]: from='mgr.14118 172.21.15.18:0/819892295' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:40:29.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:28 smithi018 ceph-mon[29336]: Manager daemon y is now available 2024-09-10T18:40:29.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:28 smithi018 ceph-mon[29336]: from='mgr.14118 172.21.15.18:0/819892295' entity='mgr.y' 2024-09-10T18:40:29.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:28 smithi018 ceph-mon[29336]: from='mgr.14118 172.21.15.18:0/819892295' entity='mgr.y' 2024-09-10T18:40:29.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:28 smithi018 ceph-mon[29336]: from='mgr.14118 172.21.15.18:0/819892295' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:29.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:28 smithi018 ceph-mon[29336]: from='mgr.14118 172.21.15.18:0/819892295' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:29.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:28 smithi018 ceph-mon[29336]: from='mgr.14118 172.21.15.18:0/819892295' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:40:29.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:28 smithi018 ceph-mon[29336]: from='mgr.14118 172.21.15.18:0/819892295' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:40:30.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:29 smithi018 ceph-mon[29336]: Found migration_current of "None". Setting to last migration. 2024-09-10T18:40:30.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:29 smithi018 ceph-mon[29336]: mgrmap e7: y(active, since 1.00485s) 2024-09-10T18:40:31.445 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:31 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:31.146+0000 7f3bc2907640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:18:40:31] ENGINE Error in HTTPServer.serve 2024-09-10T18:40:31.445 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:31 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: Traceback (most recent call last): 2024-09-10T18:40:31.445 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:31 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:40:31.445 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:31 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: self._connections.run(self.expiration_interval) 2024-09-10T18:40:31.445 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:31 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:40:31.445 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:31 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: self._run(expiration_interval) 2024-09-10T18:40:31.445 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:31 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:40:31.445 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:31 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:40:31.445 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:31 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:40:31.446 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:31 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:40:31.446 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:31 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:40:31.446 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:31 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: s = self.context.wrap_socket( 2024-09-10T18:40:31.446 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:31 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:40:31.446 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:31 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: return self.sslsocket_class._create( 2024-09-10T18:40:31.446 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:31 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:40:31.446 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:31 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: self.do_handshake() 2024-09-10T18:40:31.446 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:31 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:40:31.446 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:31 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: self._sslobj.do_handshake() 2024-09-10T18:40:31.446 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:31 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:40:31.446 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:31 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:32.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:31 smithi018 ceph-mon[29336]: mgrmap e8: y(active, since 2s) 2024-09-10T18:40:32.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:31 smithi018 ceph-mon[29336]: [10/Sep/2024:18:40:31] ENGINE Bus STARTING 2024-09-10T18:40:32.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:31 smithi018 ceph-mon[29336]: [10/Sep/2024:18:40:31] ENGINE Serving on https://172.21.15.18:7150 2024-09-10T18:40:32.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:31 smithi018 ceph-mon[29336]: [10/Sep/2024:18:40:31] ENGINE Bus STARTED 2024-09-10T18:40:32.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:31 smithi018 ceph-mon[29336]: [10/Sep/2024:18:40:31] ENGINE Error in HTTPServer.serve 2024-09-10T18:40:32.195 INFO:journalctl@ceph.mon.a.smithi018.stdout: Traceback (most recent call last): 2024-09-10T18:40:32.195 INFO:journalctl@ceph.mon.a.smithi018.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:40:32.195 INFO:journalctl@ceph.mon.a.smithi018.stdout: self._connections.run(self.expiration_interval) 2024-09-10T18:40:32.195 INFO:journalctl@ceph.mon.a.smithi018.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:40:32.195 INFO:journalctl@ceph.mon.a.smithi018.stdout: self._run(expiration_interval) 2024-09-10T18:40:32.195 INFO:journalctl@ceph.mon.a.smithi018.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:40:32.195 INFO:journalctl@ceph.mon.a.smithi018.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:40:32.195 INFO:journalctl@ceph.mon.a.smithi018.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:40:32.196 INFO:journalctl@ceph.mon.a.smithi018.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:40:32.196 INFO:journalctl@ceph.mon.a.smithi018.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:40:32.196 INFO:journalctl@ceph.mon.a.smithi018.stdout: s = self.context.wrap_socket( 2024-09-10T18:40:32.196 INFO:journalctl@ceph.mon.a.smithi018.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:40:32.196 INFO:journalctl@ceph.mon.a.smithi018.stdout: return self.sslsocket_class._create( 2024-09-10T18:40:32.196 INFO:journalctl@ceph.mon.a.smithi018.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:40:32.196 INFO:journalctl@ceph.mon.a.smithi018.stdout: self.do_handshake() 2024-09-10T18:40:32.196 INFO:journalctl@ceph.mon.a.smithi018.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:40:32.196 INFO:journalctl@ceph.mon.a.smithi018.stdout: self._sslobj.do_handshake() 2024-09-10T18:40:32.196 INFO:journalctl@ceph.mon.a.smithi018.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:40:32.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:31 smithi018 ceph-mon[29336]: from='mgr.14118 172.21.15.18:0/819892295' entity='mgr.y' 2024-09-10T18:40:32.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:31 smithi018 ceph-mon[29336]: from='mgr.14118 172.21.15.18:0/819892295' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:34.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:33 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1273393493' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T18:40:36.031 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:40:36.032 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 8, 2024-09-10T18:40:36.032 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T18:40:36.032 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-10T18:40:36.032 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T18:40:36.032 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:40:36.032 INFO:teuthology.orchestra.run.smithi018.stdout:Waiting for the mgr to restart... 2024-09-10T18:40:36.032 INFO:teuthology.orchestra.run.smithi018.stdout:Waiting for mgr epoch 8... 2024-09-10T18:40:37.053 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:40:37.053 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 8, 2024-09-10T18:40:37.054 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T18:40:37.054 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:40:37.054 INFO:teuthology.orchestra.run.smithi018.stdout:mgr epoch 8 is available 2024-09-10T18:40:37.054 INFO:teuthology.orchestra.run.smithi018.stdout:Setting orchestrator backend to cephadm... 2024-09-10T18:40:37.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:37 smithi018 ceph-mon[29336]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T18:40:37.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:37 smithi018 ceph-mon[29336]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T18:40:38.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:38 smithi018 ceph-mon[29336]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:40:38.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:38 smithi018 ceph-mon[29336]: from='mgr.14118 172.21.15.18:0/819892295' entity='mgr.y' 2024-09-10T18:40:38.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:38 smithi018 ceph-mon[29336]: from='mgr.14118 172.21.15.18:0/819892295' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:39.348 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-10T18:40:39.348 INFO:teuthology.orchestra.run.smithi018.stdout:Generating ssh key... 2024-09-10T18:40:39.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:39 smithi018 ceph-mon[29336]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:40:40.686 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: Generating public/private rsa key pair. 2024-09-10T18:40:40.686 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: Your identification has been saved in /tmp/tmprzzn6q6h/key 2024-09-10T18:40:40.686 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: Your public key has been saved in /tmp/tmprzzn6q6h/key.pub 2024-09-10T18:40:40.686 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: The key fingerprint is: 2024-09-10T18:40:40.686 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: SHA256:0722t1B/u4IkCaW0gfp79V63u+hb+wKyBOczZa3TbUM ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:40:40.686 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: The key's randomart image is: 2024-09-10T18:40:40.686 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: +---[RSA 3072]----+ 2024-09-10T18:40:40.686 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: | . | 2024-09-10T18:40:40.686 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: | . o . | 2024-09-10T18:40:40.687 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: | . . = . | 2024-09-10T18:40:40.687 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: | . +....o .E | 2024-09-10T18:40:40.687 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: | . S+oo.oo. | 2024-09-10T18:40:40.687 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: | . =*.+o.oo| 2024-09-10T18:40:40.687 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: | . ..+==+ ++| 2024-09-10T18:40:40.687 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: | . . .+.+* =| 2024-09-10T18:40:40.687 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: | . ..o=+X*| 2024-09-10T18:40:40.687 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: +----[SHA256]-----+ 2024-09-10T18:40:40.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:40 smithi018 ceph-mon[29336]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:40:40.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:40 smithi018 ceph-mon[29336]: Generating ssh key... 2024-09-10T18:40:40.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:40 smithi018 ceph-mon[29336]: from='mgr.14118 172.21.15.18:0/819892295' entity='mgr.y' 2024-09-10T18:40:40.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:40 smithi018 ceph-mon[29336]: from='mgr.14118 172.21.15.18:0/819892295' entity='mgr.y' 2024-09-10T18:40:42.582 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:42 smithi018 ceph-mon[29336]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:40:42.586 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0ZliPM9wQSqtfZvetRsT/CAG8D+e2ldNaLtTYOvCa3kzTYHaNc45nIajY7uPWNJ8k6eKQbUsfGu5/g8Rtkn8N3JxxvzhOFRa1/Yc8/L29bPoqAIqK5AxjpRh5zG6GGdxTvvwIaxsmpmvEuyrDLQnLKifMybR8rf8GkS0gAAppnHErTsG7rpEl4hk7qulsA7BduMIZQsrfPkmehJgT0uhRNgIvptwiu06qywbQzY5idQDukv+1Tc7ri3zljgaiPukNPhZcIiEc/rTbIzAaVs+qT6HAu5bNgqAKl7TIrTBT2PACU+2J1nwCg9I1ivhgvCKX+9cdVWSUVmY6sbql1A0q3ZQhSsYhOO1L4uesfW/6CJntkk5UCvpoM/XlGOfdiFi1Pf+sTSv59fDoaH5eDAPPZFdoQlAtCCEZbEULGQ4xcBM/zcXE3PdkpinykUEZobOHyckkZ2j1RX0P6BgN8TDKEIY6iajqIHu+tdvjDJOQENulCfpBU5YLNjby46x7sYs= ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:40:42.586 INFO:teuthology.orchestra.run.smithi018.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-10T18:40:42.586 INFO:teuthology.orchestra.run.smithi018.stdout:Adding key to root@localhost authorized_keys... 2024-09-10T18:40:42.599 INFO:teuthology.orchestra.run.smithi018.stdout:Adding host smithi018... 2024-09-10T18:40:43.607 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:43 smithi018 ceph-mon[29336]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi018", "addr": "172.21.15.18", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:40:45.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:45 smithi018 ceph-mon[29336]: Deploying cephadm binary to smithi018 2024-09-10T18:40:47.127 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout Added host 'smithi018' with addr '172.21.15.18' 2024-09-10T18:40:47.127 INFO:teuthology.orchestra.run.smithi018.stdout:Deploying unmanaged mon service... 2024-09-10T18:40:47.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:47 smithi018 ceph-mon[29336]: from='mgr.14118 172.21.15.18:0/819892295' entity='mgr.y' 2024-09-10T18:40:47.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:47 smithi018 ceph-mon[29336]: Added host smithi018 2024-09-10T18:40:47.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:47 smithi018 ceph-mon[29336]: from='mgr.14118 172.21.15.18:0/819892295' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:48.560 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-10T18:40:48.560 INFO:teuthology.orchestra.run.smithi018.stdout:Deploying unmanaged mgr service... 2024-09-10T18:40:48.885 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:48 smithi018 ceph-mon[29336]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:40:48.885 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:48 smithi018 ceph-mon[29336]: Saving service mon spec with placement count:5 2024-09-10T18:40:48.886 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:48 smithi018 ceph-mon[29336]: from='mgr.14118 172.21.15.18:0/819892295' entity='mgr.y' 2024-09-10T18:40:49.801 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-10T18:40:50.434 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:50 smithi018 ceph-mon[29336]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:40:50.434 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:50 smithi018 ceph-mon[29336]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:40:50.434 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:50 smithi018 ceph-mon[29336]: Saving service mgr spec with placement count:2 2024-09-10T18:40:50.434 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:50 smithi018 ceph-mon[29336]: from='mgr.14118 172.21.15.18:0/819892295' entity='mgr.y' 2024-09-10T18:40:52.053 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:51 smithi018 ceph-mon[29336]: from='mgr.14118 172.21.15.18:0/819892295' entity='mgr.y' 2024-09-10T18:40:52.053 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:51 smithi018 ceph-mon[29336]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:40:52.053 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:51 smithi018 ceph-mon[29336]: from='mgr.14118 172.21.15.18:0/819892295' entity='mgr.y' 2024-09-10T18:40:52.053 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:51 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1862066214' entity='client.admin' 2024-09-10T18:40:53.569 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:53 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1394715093' entity='client.admin' 2024-09-10T18:40:53.569 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:53 smithi018 ceph-mon[29336]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:40:53.569 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:53 smithi018 ceph-mon[29336]: from='mgr.14118 172.21.15.18:0/819892295' entity='mgr.y' 2024-09-10T18:40:53.571 INFO:teuthology.orchestra.run.smithi018.stdout:Enabling the dashboard module... 2024-09-10T18:40:55.333 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:55 smithi018 ceph-mon[29336]: from='mgr.14118 172.21.15.18:0/819892295' entity='mgr.y' 2024-09-10T18:40:55.333 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:55 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2921384041' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-10T18:40:55.334 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:55 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ignoring --setuser ceph since I am not root 2024-09-10T18:40:55.334 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:55 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ignoring --setgroup ceph since I am not root 2024-09-10T18:40:55.334 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:55 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:55.207+0000 7f117ba89ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:40:55.334 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:55 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:55.264+0000 7f117ba89ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:40:55.334 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:55 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:55.331+0000 7f117ba89ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:40:55.596 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:55 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:55.461+0000 7f117ba89ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:40:55.874 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:55 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:55.604+0000 7f117ba89ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:40:55.875 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:55 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:55.872+0000 7f117ba89ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:40:56.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:56 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2921384041' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-10T18:40:56.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:56 smithi018 ceph-mon[29336]: mgrmap e9: y(active, since 26s) 2024-09-10T18:40:56.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:55 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:55.935+0000 7f117ba89ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:40:56.195 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:56 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:56.109+0000 7f117ba89ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:40:57.234 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:40:57.234 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-10T18:40:57.234 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T18:40:57.234 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-10T18:40:57.234 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T18:40:57.234 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:40:57.234 INFO:teuthology.orchestra.run.smithi018.stdout:Waiting for the mgr to restart... 2024-09-10T18:40:57.235 INFO:teuthology.orchestra.run.smithi018.stdout:Waiting for mgr epoch 9... 2024-09-10T18:40:57.355 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:57 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/243736872' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T18:40:57.355 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:57 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:57.002+0000 7f117ba89ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:40:57.355 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:57 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:57.070+0000 7f117ba89ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:40:57.937 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:57 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:57.819+0000 7f117ba89ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:40:57.937 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:57 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:57.935+0000 7f117ba89ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:40:58.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:58 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:58.060+0000 7f117ba89ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:40:58.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:58 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:58.122+0000 7f117ba89ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:40:58.526 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:58 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:58.253+0000 7f117ba89ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:40:58.526 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:58 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:58.322+0000 7f117ba89ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:40:58.526 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:58 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:58.383+0000 7f117ba89ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:40:58.526 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:58 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:58.524+0000 7f117ba89ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:40:58.809 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:58 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:58.591+0000 7f117ba89ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:40:58.810 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:58 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:58.660+0000 7f117ba89ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:40:58.810 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:58 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:58.746+0000 7f117ba89ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:40:58.810 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:58 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:58.807+0000 7f117ba89ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:40:59.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:58 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:58.868+0000 7f117ba89ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:40:59.915 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:40:59 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:40:59.657+0000 7f117ba89ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:40:59.915 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:59 smithi018 ceph-mon[29336]: Active manager daemon y restarted 2024-09-10T18:40:59.915 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:59 smithi018 ceph-mon[29336]: Activating manager daemon y 2024-09-10T18:40:59.915 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:59 smithi018 ceph-mon[29336]: osdmap e3: 0 total, 0 up, 0 in 2024-09-10T18:40:59.915 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:59 smithi018 ceph-mon[29336]: mgrmap e10: y(active, starting, since 0.00318767s) 2024-09-10T18:40:59.915 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:59 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:40:59.915 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:59 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:40:59.916 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:59 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:40:59.916 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:59 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:40:59.916 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:59 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:40:59.916 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:59 smithi018 ceph-mon[29336]: Manager daemon y is now available 2024-09-10T18:40:59.916 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:59 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:59.916 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:59 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:40:59.916 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:40:59 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:41:01.425 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:41:01.425 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-10T18:41:01.425 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T18:41:01.425 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:41:01.425 INFO:teuthology.orchestra.run.smithi018.stdout:mgr epoch 9 is available 2024-09-10T18:41:01.425 INFO:teuthology.orchestra.run.smithi018.stdout:Generating a dashboard self-signed certificate... 2024-09-10T18:41:01.876 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:01 smithi018 ceph-mon[29336]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T18:41:01.876 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:01 smithi018 ceph-mon[29336]: mgrmap e11: y(active, since 1.00632s) 2024-09-10T18:41:01.876 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:01 smithi018 ceph-mon[29336]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T18:41:01.876 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:41:01 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:41:01.754+0000 7f112c5e6640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:18:41:01] ENGINE Error in HTTPServer.serve 2024-09-10T18:41:01.876 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:41:01 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: Traceback (most recent call last): 2024-09-10T18:41:01.876 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:41:01 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:41:01.877 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:41:01 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: self._connections.run(self.expiration_interval) 2024-09-10T18:41:01.877 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:41:01 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:41:01.877 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:41:01 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: self._run(expiration_interval) 2024-09-10T18:41:01.877 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:41:01 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:41:01.877 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:41:01 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:41:01.877 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:41:01 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:41:01.877 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:41:01 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:41:01.877 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:41:01 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:41:01.877 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:41:01 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: s = self.context.wrap_socket( 2024-09-10T18:41:01.877 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:41:01 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:41:01.877 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:41:01 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: return self.sslsocket_class._create( 2024-09-10T18:41:01.878 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:41:01 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:41:01.878 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:41:01 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: self.do_handshake() 2024-09-10T18:41:01.878 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:41:01 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:41:01.878 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:41:01 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: self._sslobj.do_handshake() 2024-09-10T18:41:01.878 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:41:01 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:41:01.878 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:41:01 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:41:02.656 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-10T18:41:02.657 INFO:teuthology.orchestra.run.smithi018.stdout:Creating initial admin user... 2024-09-10T18:41:02.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:02 smithi018 ceph-mon[29336]: [10/Sep/2024:18:41:01] ENGINE Bus STARTING 2024-09-10T18:41:02.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:02 smithi018 ceph-mon[29336]: mgrmap e12: y(active, since 2s) 2024-09-10T18:41:02.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:02 smithi018 ceph-mon[29336]: [10/Sep/2024:18:41:01] ENGINE Serving on https://172.21.15.18:7150 2024-09-10T18:41:02.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:02 smithi018 ceph-mon[29336]: [10/Sep/2024:18:41:01] ENGINE Bus STARTED 2024-09-10T18:41:02.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:02 smithi018 ceph-mon[29336]: [10/Sep/2024:18:41:01] ENGINE Error in HTTPServer.serve 2024-09-10T18:41:02.945 INFO:journalctl@ceph.mon.a.smithi018.stdout: Traceback (most recent call last): 2024-09-10T18:41:02.945 INFO:journalctl@ceph.mon.a.smithi018.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:41:02.945 INFO:journalctl@ceph.mon.a.smithi018.stdout: self._connections.run(self.expiration_interval) 2024-09-10T18:41:02.945 INFO:journalctl@ceph.mon.a.smithi018.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:41:02.945 INFO:journalctl@ceph.mon.a.smithi018.stdout: self._run(expiration_interval) 2024-09-10T18:41:02.945 INFO:journalctl@ceph.mon.a.smithi018.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:41:02.945 INFO:journalctl@ceph.mon.a.smithi018.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:41:02.945 INFO:journalctl@ceph.mon.a.smithi018.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:41:02.945 INFO:journalctl@ceph.mon.a.smithi018.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:41:02.946 INFO:journalctl@ceph.mon.a.smithi018.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:41:02.946 INFO:journalctl@ceph.mon.a.smithi018.stdout: s = self.context.wrap_socket( 2024-09-10T18:41:02.946 INFO:journalctl@ceph.mon.a.smithi018.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:41:02.946 INFO:journalctl@ceph.mon.a.smithi018.stdout: return self.sslsocket_class._create( 2024-09-10T18:41:02.946 INFO:journalctl@ceph.mon.a.smithi018.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:41:02.946 INFO:journalctl@ceph.mon.a.smithi018.stdout: self.do_handshake() 2024-09-10T18:41:02.946 INFO:journalctl@ceph.mon.a.smithi018.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:41:02.946 INFO:journalctl@ceph.mon.a.smithi018.stdout: self._sslobj.do_handshake() 2024-09-10T18:41:02.946 INFO:journalctl@ceph.mon.a.smithi018.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:41:02.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:02 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:02.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:02 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:02.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:02 smithi018 ceph-mon[29336]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:41:02.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:02 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:02.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:02 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:04.097 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$QTW5e9xWMACJPsA2CHXq1uSDqQiYJ7fsE7rn.P0V8taYUeDc9295u", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1725993663, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-10T18:41:04.097 INFO:teuthology.orchestra.run.smithi018.stdout:Fetching dashboard port number... 2024-09-10T18:41:04.422 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:04 smithi018 ceph-mon[29336]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:41:04.422 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:04 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:04.422 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:04 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:05.478 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:05 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1821454510' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-10T18:41:06.159 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stdout 8443 2024-09-10T18:41:06.171 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T18:41:06.171 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout disabled 2024-09-10T18:41:06.181 INFO:teuthology.orchestra.run.smithi018.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T18:41:06.181 INFO:teuthology.orchestra.run.smithi018.stdout:systemctl: stdout inactive 2024-09-10T18:41:06.181 INFO:teuthology.orchestra.run.smithi018.stdout:firewalld.service is not enabled 2024-09-10T18:41:06.181 INFO:teuthology.orchestra.run.smithi018.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-10T18:41:06.184 INFO:teuthology.orchestra.run.smithi018.stdout:Ceph Dashboard is now available at: 2024-09-10T18:41:06.184 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:41:06.184 INFO:teuthology.orchestra.run.smithi018.stdout: URL: https://smithi018.front.sepia.ceph.com:8443/ 2024-09-10T18:41:06.185 INFO:teuthology.orchestra.run.smithi018.stdout: User: admin 2024-09-10T18:41:06.185 INFO:teuthology.orchestra.run.smithi018.stdout: Password: kvxa1jrdru 2024-09-10T18:41:06.185 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:41:06.185 INFO:teuthology.orchestra.run.smithi018.stdout:Saving cluster configuration to /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/config directory 2024-09-10T18:41:06.185 INFO:teuthology.orchestra.run.smithi018.stdout:Enabling autotune for osd_memory_target 2024-09-10T18:41:07.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:07 smithi018 ceph-mon[29336]: mgrmap e13: y(active, since 6s) 2024-09-10T18:41:08.605 INFO:teuthology.orchestra.run.smithi018.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-10T18:41:08.605 INFO:teuthology.orchestra.run.smithi018.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-10T18:41:08.605 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:41:08.606 INFO:teuthology.orchestra.run.smithi018.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:41:08.606 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:41:08.606 INFO:teuthology.orchestra.run.smithi018.stdout:Or, if you are only running a single cluster on this host: 2024-09-10T18:41:08.606 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:41:08.606 INFO:teuthology.orchestra.run.smithi018.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-10T18:41:08.606 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:41:08.606 INFO:teuthology.orchestra.run.smithi018.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-10T18:41:08.606 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:41:08.606 INFO:teuthology.orchestra.run.smithi018.stdout: ceph telemetry on 2024-09-10T18:41:08.606 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:41:08.606 INFO:teuthology.orchestra.run.smithi018.stdout:For more information see: 2024-09-10T18:41:08.606 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:41:08.606 INFO:teuthology.orchestra.run.smithi018.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-09-10T18:41:08.606 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:41:08.607 INFO:teuthology.orchestra.run.smithi018.stdout:Bootstrap complete. 2024-09-10T18:41:08.644 INFO:tasks.cephadm:Fetching config... 2024-09-10T18:41:08.644 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-10T18:41:08.644 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-10T18:41:08.662 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-10T18:41:08.662 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-10T18:41:08.662 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-10T18:41:08.719 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-10T18:41:08.719 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-10T18:41:08.719 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-10T18:41:08.785 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-10T18:41:08.786 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-10T18:41:08.786 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-10T18:41:08.841 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-10T18:41:08.842 DEBUG:teuthology.orchestra.run.smithi018:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0ZliPM9wQSqtfZvetRsT/CAG8D+e2ldNaLtTYOvCa3kzTYHaNc45nIajY7uPWNJ8k6eKQbUsfGu5/g8Rtkn8N3JxxvzhOFRa1/Yc8/L29bPoqAIqK5AxjpRh5zG6GGdxTvvwIaxsmpmvEuyrDLQnLKifMybR8rf8GkS0gAAppnHErTsG7rpEl4hk7qulsA7BduMIZQsrfPkmehJgT0uhRNgIvptwiu06qywbQzY5idQDukv+1Tc7ri3zljgaiPukNPhZcIiEc/rTbIzAaVs+qT6HAu5bNgqAKl7TIrTBT2PACU+2J1nwCg9I1ivhgvCKX+9cdVWSUVmY6sbql1A0q3ZQhSsYhOO1L4uesfW/6CJntkk5UCvpoM/XlGOfdiFi1Pf+sTSv59fDoaH5eDAPPZFdoQlAtCCEZbEULGQ4xcBM/zcXE3PdkpinykUEZobOHyckkZ2j1RX0P6BgN8TDKEIY6iajqIHu+tdvjDJOQENulCfpBU5YLNjby46x7sYs= ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T18:41:08.917 INFO:teuthology.orchestra.run.smithi018.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0ZliPM9wQSqtfZvetRsT/CAG8D+e2ldNaLtTYOvCa3kzTYHaNc45nIajY7uPWNJ8k6eKQbUsfGu5/g8Rtkn8N3JxxvzhOFRa1/Yc8/L29bPoqAIqK5AxjpRh5zG6GGdxTvvwIaxsmpmvEuyrDLQnLKifMybR8rf8GkS0gAAppnHErTsG7rpEl4hk7qulsA7BduMIZQsrfPkmehJgT0uhRNgIvptwiu06qywbQzY5idQDukv+1Tc7ri3zljgaiPukNPhZcIiEc/rTbIzAaVs+qT6HAu5bNgqAKl7TIrTBT2PACU+2J1nwCg9I1ivhgvCKX+9cdVWSUVmY6sbql1A0q3ZQhSsYhOO1L4uesfW/6CJntkk5UCvpoM/XlGOfdiFi1Pf+sTSv59fDoaH5eDAPPZFdoQlAtCCEZbEULGQ4xcBM/zcXE3PdkpinykUEZobOHyckkZ2j1RX0P6BgN8TDKEIY6iajqIHu+tdvjDJOQENulCfpBU5YLNjby46x7sYs= ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:41:08.931 DEBUG:teuthology.orchestra.run.smithi103:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0ZliPM9wQSqtfZvetRsT/CAG8D+e2ldNaLtTYOvCa3kzTYHaNc45nIajY7uPWNJ8k6eKQbUsfGu5/g8Rtkn8N3JxxvzhOFRa1/Yc8/L29bPoqAIqK5AxjpRh5zG6GGdxTvvwIaxsmpmvEuyrDLQnLKifMybR8rf8GkS0gAAppnHErTsG7rpEl4hk7qulsA7BduMIZQsrfPkmehJgT0uhRNgIvptwiu06qywbQzY5idQDukv+1Tc7ri3zljgaiPukNPhZcIiEc/rTbIzAaVs+qT6HAu5bNgqAKl7TIrTBT2PACU+2J1nwCg9I1ivhgvCKX+9cdVWSUVmY6sbql1A0q3ZQhSsYhOO1L4uesfW/6CJntkk5UCvpoM/XlGOfdiFi1Pf+sTSv59fDoaH5eDAPPZFdoQlAtCCEZbEULGQ4xcBM/zcXE3PdkpinykUEZobOHyckkZ2j1RX0P6BgN8TDKEIY6iajqIHu+tdvjDJOQENulCfpBU5YLNjby46x7sYs= ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T18:41:09.031 INFO:teuthology.orchestra.run.smithi103.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0ZliPM9wQSqtfZvetRsT/CAG8D+e2ldNaLtTYOvCa3kzTYHaNc45nIajY7uPWNJ8k6eKQbUsfGu5/g8Rtkn8N3JxxvzhOFRa1/Yc8/L29bPoqAIqK5AxjpRh5zG6GGdxTvvwIaxsmpmvEuyrDLQnLKifMybR8rf8GkS0gAAppnHErTsG7rpEl4hk7qulsA7BduMIZQsrfPkmehJgT0uhRNgIvptwiu06qywbQzY5idQDukv+1Tc7ri3zljgaiPukNPhZcIiEc/rTbIzAaVs+qT6HAu5bNgqAKl7TIrTBT2PACU+2J1nwCg9I1ivhgvCKX+9cdVWSUVmY6sbql1A0q3ZQhSsYhOO1L4uesfW/6CJntkk5UCvpoM/XlGOfdiFi1Pf+sTSv59fDoaH5eDAPPZFdoQlAtCCEZbEULGQ4xcBM/zcXE3PdkpinykUEZobOHyckkZ2j1RX0P6BgN8TDKEIY6iajqIHu+tdvjDJOQENulCfpBU5YLNjby46x7sYs= ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:41:09.045 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-10T18:41:09.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:09 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/385646735' entity='client.admin' 2024-09-10T18:41:09.301 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:41:12.017 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:11 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:12.018 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:11 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:12.018 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:11 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi018", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:41:12.018 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:11 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:12.018 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:11 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:41:12.018 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:11 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:12.018 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:11 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1259890884' entity='client.admin' 2024-09-10T18:41:12.018 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:11 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:41:12.018 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:11 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:12.450 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-10T18:41:12.451 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-10T18:41:12.783 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:41:15.742 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi103 2024-09-10T18:41:15.743 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-10T18:41:15.743 DEBUG:teuthology.orchestra.run.smithi103:> dd of=/etc/ceph/ceph.conf 2024-09-10T18:41:15.763 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-10T18:41:15.763 DEBUG:teuthology.orchestra.run.smithi103:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:41:15.821 INFO:tasks.cephadm:Adding host smithi103 to orchestrator... 2024-09-10T18:41:15.822 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph orch host add smithi103 2024-09-10T18:41:16.085 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:41:16.337 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:16 smithi018 ceph-mon[29336]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:41:16.337 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:16 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:16.337 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:16 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:41:16.337 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:16 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:16.337 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:16 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:41:16.337 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:16 smithi018 ceph-mon[29336]: Updating smithi018:/etc/ceph/ceph.conf 2024-09-10T18:41:16.337 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:16 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:16.337 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:16 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:16.337 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:16 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:17.306 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:17 smithi018 ceph-mon[29336]: Updating smithi018:/var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:41:17.306 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:17 smithi018 ceph-mon[29336]: Updating smithi018:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:41:17.306 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:17 smithi018 ceph-mon[29336]: Updating smithi018:/var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:41:18.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:18 smithi018 ceph-mon[29336]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi103", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:41:19.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:19 smithi018 ceph-mon[29336]: Deploying cephadm binary to smithi103 2024-09-10T18:41:20.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:20 smithi018 ceph-mon[29336]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:21.240 INFO:teuthology.orchestra.run.smithi018.stdout:Added host 'smithi103' with addr '172.21.15.103' 2024-09-10T18:41:21.826 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T18:41:22.072 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:41:22.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:22 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:22.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:22 smithi018 ceph-mon[29336]: Added host smithi103 2024-09-10T18:41:22.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:22 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:41:22.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:22 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:22.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:22 smithi018 ceph-mon[29336]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:22.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:22 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:23.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:23 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:23.734 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:41:23.734 INFO:teuthology.orchestra.run.smithi018.stdout:[{"addr": "172.21.15.18", "hostname": "smithi018", "labels": [], "status": ""}, {"addr": "172.21.15.103", "hostname": "smithi103", "labels": [], "status": ""}] 2024-09-10T18:41:24.323 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-10T18:41:24.324 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-10T18:41:24.576 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:41:24.906 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:24 smithi018 ceph-mon[29336]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:24.907 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:24 smithi018 ceph-mon[29336]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:41:26.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:26 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:26.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:26 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:26.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:26 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:26.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:26 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:26.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:26 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi103", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:41:26.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:26 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:26.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:26 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:41:26.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:26 smithi018 ceph-mon[29336]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-10T18:41:26.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:26 smithi018 ceph-mon[29336]: Updating smithi103:/var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:41:26.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:26 smithi018 ceph-mon[29336]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:26.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:26 smithi018 ceph-mon[29336]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:41:26.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:26 smithi018 ceph-mon[29336]: Updating smithi103:/var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:41:26.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:26 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:26.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:26 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:26.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:26 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:26.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:26 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1376742874' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-10T18:41:27.231 INFO:teuthology.orchestra.run.smithi018.stderr:adjusted tunables profile to default 2024-09-10T18:41:27.873 INFO:tasks.cephadm:Adding mon.a on smithi018 2024-09-10T18:41:27.873 INFO:tasks.cephadm:Adding mon.c on smithi018 2024-09-10T18:41:27.873 INFO:tasks.cephadm:Adding mon.b on smithi103 2024-09-10T18:41:27.873 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi018:172.21.15.18=a;smithi018:[v2:172.21.15.18:3301,v1:172.21.15.18:6790]=c;smithi103:172.21.15.103=b' 2024-09-10T18:41:28.125 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:41:28.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:28 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1376742874' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T18:41:28.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:28 smithi018 ceph-mon[29336]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:41:28.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:28 smithi018 ceph-mon[29336]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:29.692 INFO:teuthology.orchestra.run.smithi103.stdout:Scheduled mon update... 2024-09-10T18:41:30.369 DEBUG:teuthology.orchestra.run.smithi018:mon.c> sudo journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@mon.c.service 2024-09-10T18:41:30.372 DEBUG:teuthology.orchestra.run.smithi103:mon.b> sudo journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@mon.b.service 2024-09-10T18:41:30.374 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T18:41:30.374 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T18:41:30.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:30 smithi018 ceph-mon[29336]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:30.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:30 smithi018 ceph-mon[29336]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi018:172.21.15.18=a;smithi018:[v2:172.21.15.18:3301,v1:172.21.15.18:6790]=c;smithi103:172.21.15.103=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:41:30.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:30 smithi018 ceph-mon[29336]: Saving service mon spec with placement smithi018:172.21.15.18=a;smithi018:[v2:172.21.15.18:3301,v1:172.21.15.18:6790]=c;smithi103:172.21.15.103=b;count:3 2024-09-10T18:41:30.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:30 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:30.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:30 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:41:30.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:30 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:30.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:30 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:41:30.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:30 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:30.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:30 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:41:30.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:30 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:30.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:30 smithi018 ceph-mon[29336]: Deploying daemon mon.b on smithi103 2024-09-10T18:41:31.776 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:41:32.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:32 smithi018 ceph-mon[29336]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:34.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 systemd[1]: Starting Ceph mon.b for ba327f6c-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:41:34.541 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:41:34.541 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":1,"fsid":"ba327f6c-6fa3-11ef-bcea-c7b262605968","modified":"2024-09-10T18:40:04.849809Z","created":"2024-09-10T18:40:04.849809Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:3300","nonce":0},{"type":"v1","addr":"172.21.15.18:6789","nonce":0}]},"addr":"172.21.15.18:6789/0","public_addr":"172.21.15.18:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T18:41:34.542 INFO:teuthology.orchestra.run.smithi103.stderr:dumped monmap epoch 1 2024-09-10T18:41:34.863 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 podman[33131]: 2024-09-10 18:41:34.444264164 +0000 UTC m=+0.015421893 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:41:34.863 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 podman[33131]: 2024-09-10 18:41:34.567673408 +0000 UTC m=+0.138831135 container create f40a52d205a9fc4a55dfa78ebc60f5e3173b3d4b5ed9105e06d82f397dcde16a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mon-b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:41:34.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:34 smithi018 ceph-mon[29336]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:34.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:34 smithi018 ceph-mon[29336]: from='client.? 172.21.15.103:0/4171381390' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:41:35.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 podman[33131]: 2024-09-10 18:41:34.969490195 +0000 UTC m=+0.540647911 container init f40a52d205a9fc4a55dfa78ebc60f5e3173b3d4b5ed9105e06d82f397dcde16a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mon-b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:41:35.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 podman[33131]: 2024-09-10 18:41:34.972564888 +0000 UTC m=+0.543722604 container start f40a52d205a9fc4a55dfa78ebc60f5e3173b3d4b5ed9105e06d82f397dcde16a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mon-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-10T18:41:35.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T18:41:35.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable), process ceph-mon, pid 2 2024-09-10T18:41:35.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: pidfile_write: ignore empty --pid-file 2024-09-10T18:41:35.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: load: jerasure load: lrc 2024-09-10T18:41:35.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: RocksDB version: 6.15.5 2024-09-10T18:41:35.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-10T18:41:35.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Compile date Sep 9 2024 2024-09-10T18:41:35.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: DB SUMMARY 2024-09-10T18:41:35.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: DB Session ID: UP98P9UKXIBSUF9FA5Y8 2024-09-10T18:41:35.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: CURRENT file: CURRENT 2024-09-10T18:41:35.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T18:41:35.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-09-10T18:41:35.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-10T18:41:35.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 511 ; 2024-09-10T18:41:35.275 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.error_if_exists: 0 2024-09-10T18:41:35.275 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.create_if_missing: 0 2024-09-10T18:41:35.275 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.paranoid_checks: 1 2024-09-10T18:41:35.275 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T18:41:35.275 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.env: 0x5591b1394440 2024-09-10T18:41:35.275 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.fs: Posix File System 2024-09-10T18:41:35.275 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.info_log: 0x5591b36c4ec0 2024-09-10T18:41:35.275 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T18:41:35.275 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.statistics: (nil) 2024-09-10T18:41:35.277 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.use_fsync: 0 2024-09-10T18:41:35.277 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.max_log_file_size: 0 2024-09-10T18:41:35.277 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T18:41:35.277 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T18:41:35.277 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T18:41:35.277 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T18:41:35.277 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.allow_fallocate: 1 2024-09-10T18:41:35.278 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T18:41:35.278 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T18:41:35.278 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.use_direct_reads: 0 2024-09-10T18:41:35.278 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T18:41:35.278 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T18:41:35.278 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.db_log_dir: 2024-09-10T18:41:35.278 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-b/store.db 2024-09-10T18:41:35.278 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T18:41:35.278 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T18:41:35.278 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T18:41:35.278 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T18:41:35.278 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T18:41:35.278 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T18:41:35.279 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T18:41:35.279 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T18:41:35.279 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.write_buffer_manager: 0x5591b37827e0 2024-09-10T18:41:35.279 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T18:41:35.279 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-10T18:41:35.279 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T18:41:35.279 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T18:41:35.279 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T18:41:35.279 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T18:41:35.279 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T18:41:35.279 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T18:41:35.279 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T18:41:35.279 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.unordered_write: 0 2024-09-10T18:41:35.280 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T18:41:35.280 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T18:41:35.280 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T18:41:35.280 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T18:41:35.280 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.row_cache: None 2024-09-10T18:41:35.280 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.wal_filter: None 2024-09-10T18:41:35.280 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T18:41:35.280 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T18:41:35.280 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.preserve_deletes: 0 2024-09-10T18:41:35.280 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.two_write_queues: 0 2024-09-10T18:41:35.280 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T18:41:35.280 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.atomic_flush: 0 2024-09-10T18:41:35.280 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T18:41:35.281 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T18:41:35.281 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T18:41:35.281 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.log_readahead_size: 0 2024-09-10T18:41:35.281 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T18:41:35.281 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T18:41:35.281 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T18:41:35.281 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T18:41:35.281 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T18:41:35.281 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T18:41:35.281 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.max_background_jobs: 2 2024-09-10T18:41:35.281 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.max_background_compactions: -1 2024-09-10T18:41:35.281 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.max_subcompactions: 1 2024-09-10T18:41:35.281 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T18:41:35.282 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T18:41:35.282 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T18:41:35.282 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T18:41:35.282 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T18:41:35.282 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T18:41:35.282 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T18:41:35.282 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T18:41:35.282 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.max_open_files: -1 2024-09-10T18:41:35.282 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T18:41:35.282 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T18:41:35.282 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T18:41:35.282 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T18:41:35.282 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.max_background_flushes: -1 2024-09-10T18:41:35.282 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Compression algorithms supported: 2024-09-10T18:41:35.282 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: kZSTD supported: 0 2024-09-10T18:41:35.283 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: kXpressCompression supported: 0 2024-09-10T18:41:35.283 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: kBZip2Compression supported: 0 2024-09-10T18:41:35.283 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T18:41:35.283 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: kLZ4Compression supported: 1 2024-09-10T18:41:35.283 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: kZlibCompression supported: 1 2024-09-10T18:41:35.283 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T18:41:35.283 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: kSnappyCompression supported: 1 2024-09-10T18:41:35.283 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T18:41:35.283 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000003 2024-09-10T18:41:35.283 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-10T18:41:35.283 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T18:41:35.283 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.merge_operator: 2024-09-10T18:41:35.283 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.compaction_filter: None 2024-09-10T18:41:35.284 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T18:41:35.284 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T18:41:35.284 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T18:41:35.284 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T18:41:35.284 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5591b366ba38) 2024-09-10T18:41:35.284 INFO:journalctl@ceph.mon.b.smithi103.stdout: cache_index_and_filter_blocks: 1 2024-09-10T18:41:35.284 INFO:journalctl@ceph.mon.b.smithi103.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T18:41:35.284 INFO:journalctl@ceph.mon.b.smithi103.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T18:41:35.284 INFO:journalctl@ceph.mon.b.smithi103.stdout: pin_top_level_index_and_filter: 1 2024-09-10T18:41:35.284 INFO:journalctl@ceph.mon.b.smithi103.stdout: index_type: 0 2024-09-10T18:41:35.284 INFO:journalctl@ceph.mon.b.smithi103.stdout: data_block_index_type: 0 2024-09-10T18:41:35.284 INFO:journalctl@ceph.mon.b.smithi103.stdout: index_shortening: 1 2024-09-10T18:41:35.284 INFO:journalctl@ceph.mon.b.smithi103.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T18:41:35.285 INFO:journalctl@ceph.mon.b.smithi103.stdout: hash_index_allow_collision: 1 2024-09-10T18:41:35.285 INFO:journalctl@ceph.mon.b.smithi103.stdout: checksum: 1 2024-09-10T18:41:35.285 INFO:journalctl@ceph.mon.b.smithi103.stdout: no_block_cache: 0 2024-09-10T18:41:35.285 INFO:journalctl@ceph.mon.b.smithi103.stdout: block_cache: 0x5591b36a11f0 2024-09-10T18:41:35.285 INFO:journalctl@ceph.mon.b.smithi103.stdout: block_cache_name: BinnedLRUCache 2024-09-10T18:41:35.285 INFO:journalctl@ceph.mon.b.smithi103.stdout: block_cache_options: 2024-09-10T18:41:35.285 INFO:journalctl@ceph.mon.b.smithi103.stdout: capacity : 536870912 2024-09-10T18:41:35.285 INFO:journalctl@ceph.mon.b.smithi103.stdout: num_shard_bits : 4 2024-09-10T18:41:35.285 INFO:journalctl@ceph.mon.b.smithi103.stdout: strict_capacity_limit : 0 2024-09-10T18:41:35.285 INFO:journalctl@ceph.mon.b.smithi103.stdout: high_pri_pool_ratio: 0.000 2024-09-10T18:41:35.285 INFO:journalctl@ceph.mon.b.smithi103.stdout: block_cache_compressed: (nil) 2024-09-10T18:41:35.285 INFO:journalctl@ceph.mon.b.smithi103.stdout: persistent_cache: (nil) 2024-09-10T18:41:35.285 INFO:journalctl@ceph.mon.b.smithi103.stdout: block_size: 4096 2024-09-10T18:41:35.285 INFO:journalctl@ceph.mon.b.smithi103.stdout: block_size_deviation: 10 2024-09-10T18:41:35.286 INFO:journalctl@ceph.mon.b.smithi103.stdout: block_restart_interval: 16 2024-09-10T18:41:35.286 INFO:journalctl@ceph.mon.b.smithi103.stdout: index_block_restart_interval: 1 2024-09-10T18:41:35.286 INFO:journalctl@ceph.mon.b.smithi103.stdout: metadata_block_size: 4096 2024-09-10T18:41:35.286 INFO:journalctl@ceph.mon.b.smithi103.stdout: partition_filters: 0 2024-09-10T18:41:35.286 INFO:journalctl@ceph.mon.b.smithi103.stdout: use_delta_encoding: 1 2024-09-10T18:41:35.286 INFO:journalctl@ceph.mon.b.smithi103.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-10T18:41:35.286 INFO:journalctl@ceph.mon.b.smithi103.stdout: whole_key_filtering: 1 2024-09-10T18:41:35.286 INFO:journalctl@ceph.mon.b.smithi103.stdout: verify_compression: 0 2024-09-10T18:41:35.286 INFO:journalctl@ceph.mon.b.smithi103.stdout: read_amp_bytes_per_bit: 0 2024-09-10T18:41:35.286 INFO:journalctl@ceph.mon.b.smithi103.stdout: format_version: 4 2024-09-10T18:41:35.286 INFO:journalctl@ceph.mon.b.smithi103.stdout: enable_index_compression: 1 2024-09-10T18:41:35.286 INFO:journalctl@ceph.mon.b.smithi103.stdout: block_align: 0 2024-09-10T18:41:35.286 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T18:41:35.287 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T18:41:35.287 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.compression: NoCompression 2024-09-10T18:41:35.287 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T18:41:35.287 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T18:41:35.287 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T18:41:35.287 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.num_levels: 7 2024-09-10T18:41:35.287 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T18:41:35.287 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T18:41:35.287 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T18:41:35.287 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T18:41:35.287 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T18:41:35.287 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T18:41:35.287 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T18:41:35.287 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:41:35.288 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T18:41:35.288 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T18:41:35.288 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T18:41:35.288 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T18:41:35.288 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T18:41:35.288 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T18:41:35.288 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:41:35.288 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T18:41:35.288 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T18:41:35.288 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T18:41:35.288 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T18:41:35.288 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T18:41:35.288 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T18:41:35.288 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T18:41:35.289 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T18:41:35.289 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T18:41:35.289 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T18:41:35.289 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T18:41:35.289 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T18:41:35.289 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T18:41:35.289 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T18:41:35.289 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T18:41:35.289 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T18:41:35.289 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T18:41:35.289 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T18:41:35.290 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T18:41:35.290 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.arena_block_size: 4194304 2024-09-10T18:41:35.290 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T18:41:35.290 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T18:41:35.290 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-10T18:41:35.290 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T18:41:35.290 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T18:41:35.290 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T18:41:35.290 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T18:41:35.290 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T18:41:35.290 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T18:41:35.290 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T18:41:35.290 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T18:41:35.290 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T18:41:35.291 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T18:41:35.291 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T18:41:35.291 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T18:41:35.291 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.inplace_update_support: 0 2024-09-10T18:41:35.291 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T18:41:35.291 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T18:41:35.291 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T18:41:35.291 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T18:41:35.291 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.bloom_locality: 0 2024-09-10T18:41:35.291 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.max_successive_merges: 0 2024-09-10T18:41:35.291 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T18:41:35.291 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T18:41:35.291 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T18:41:35.292 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T18:41:35.292 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.ttl: 2592000 2024-09-10T18:41:35.292 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T18:41:35.292 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.enable_blob_files: false 2024-09-10T18:41:35.292 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.min_blob_size: 0 2024-09-10T18:41:35.292 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T18:41:35.292 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T18:41:35.292 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T18:41:35.292 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T18:41:35.292 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T18:41:35.292 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-09-10T18:41:35.292 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-09-10T18:41:35.293 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725993694996406, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-10T18:41:35.293 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-09-10T18:41:35.293 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-10T18:41:35.293 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725993694996696, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1571, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1725993694, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "a0131044-62f8-4134-8ac0-9c0e24a8cf6c", "db_session_id": "UP98P9UKXIBSUF9FA5Y8"}} 2024-09-10T18:41:35.293 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-09-10T18:41:35.293 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725993694997026, "job": 1, "event": "recovery_finished"} 2024-09-10T18:41:35.293 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T18:41:35.293 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x5591b36d4700 2024-09-10T18:41:35.293 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: DB pointer 0x5591b36a4000 2024-09-10T18:41:35.293 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-10T18:41:35.293 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-10T18:41:35.293 INFO:journalctl@ceph.mon.b.smithi103.stdout: ** DB Stats ** 2024-09-10T18:41:35.293 INFO:journalctl@ceph.mon.b.smithi103.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:41:35.294 INFO:journalctl@ceph.mon.b.smithi103.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T18:41:35.294 INFO:journalctl@ceph.mon.b.smithi103.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T18:41:35.294 INFO:journalctl@ceph.mon.b.smithi103.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:41:35.294 INFO:journalctl@ceph.mon.b.smithi103.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T18:41:35.294 INFO:journalctl@ceph.mon.b.smithi103.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-10T18:41:35.294 INFO:journalctl@ceph.mon.b.smithi103.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:41:35.294 INFO:journalctl@ceph.mon.b.smithi103.stdout: 2024-09-10T18:41:35.294 INFO:journalctl@ceph.mon.b.smithi103.stdout: ** Compaction Stats [default] ** 2024-09-10T18:41:35.294 INFO:journalctl@ceph.mon.b.smithi103.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:41:35.296 INFO:journalctl@ceph.mon.b.smithi103.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:41:35.296 INFO:journalctl@ceph.mon.b.smithi103.stdout: L0 1/0 1.53 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.4 0.00 0.00 1 0.000 0 0 2024-09-10T18:41:35.296 INFO:journalctl@ceph.mon.b.smithi103.stdout: Sum 1/0 1.53 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.4 0.00 0.00 1 0.000 0 0 2024-09-10T18:41:35.296 INFO:journalctl@ceph.mon.b.smithi103.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.4 0.00 0.00 1 0.000 0 0 2024-09-10T18:41:35.296 INFO:journalctl@ceph.mon.b.smithi103.stdout: 2024-09-10T18:41:35.296 INFO:journalctl@ceph.mon.b.smithi103.stdout: ** Compaction Stats [default] ** 2024-09-10T18:41:35.296 INFO:journalctl@ceph.mon.b.smithi103.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:41:35.296 INFO:journalctl@ceph.mon.b.smithi103.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:41:35.296 INFO:journalctl@ceph.mon.b.smithi103.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.4 0.00 0.00 1 0.000 0 0 2024-09-10T18:41:35.296 INFO:journalctl@ceph.mon.b.smithi103.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:41:35.296 INFO:journalctl@ceph.mon.b.smithi103.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:41:35.296 INFO:journalctl@ceph.mon.b.smithi103.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:41:35.297 INFO:journalctl@ceph.mon.b.smithi103.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:41:35.297 INFO:journalctl@ceph.mon.b.smithi103.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:41:35.297 INFO:journalctl@ceph.mon.b.smithi103.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:41:35.297 INFO:journalctl@ceph.mon.b.smithi103.stdout: Cumulative compaction: 0.00 GB write, 0.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:41:35.297 INFO:journalctl@ceph.mon.b.smithi103.stdout: Interval compaction: 0.00 GB write, 0.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:41:35.297 INFO:journalctl@ceph.mon.b.smithi103.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:41:35.297 INFO:journalctl@ceph.mon.b.smithi103.stdout: 2024-09-10T18:41:35.297 INFO:journalctl@ceph.mon.b.smithi103.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:41:35.297 INFO:journalctl@ceph.mon.b.smithi103.stdout: 2024-09-10T18:41:35.297 INFO:journalctl@ceph.mon.b.smithi103.stdout: ** Compaction Stats [default] ** 2024-09-10T18:41:35.297 INFO:journalctl@ceph.mon.b.smithi103.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:41:35.297 INFO:journalctl@ceph.mon.b.smithi103.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:41:35.297 INFO:journalctl@ceph.mon.b.smithi103.stdout: L0 1/0 1.53 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.4 0.00 0.00 1 0.000 0 0 2024-09-10T18:41:35.298 INFO:journalctl@ceph.mon.b.smithi103.stdout: Sum 1/0 1.53 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.4 0.00 0.00 1 0.000 0 0 2024-09-10T18:41:35.298 INFO:journalctl@ceph.mon.b.smithi103.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-10T18:41:35.298 INFO:journalctl@ceph.mon.b.smithi103.stdout: 2024-09-10T18:41:35.298 INFO:journalctl@ceph.mon.b.smithi103.stdout: ** Compaction Stats [default] ** 2024-09-10T18:41:35.298 INFO:journalctl@ceph.mon.b.smithi103.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:41:35.298 INFO:journalctl@ceph.mon.b.smithi103.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:41:35.298 INFO:journalctl@ceph.mon.b.smithi103.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.4 0.00 0.00 1 0.000 0 0 2024-09-10T18:41:35.298 INFO:journalctl@ceph.mon.b.smithi103.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:41:35.298 INFO:journalctl@ceph.mon.b.smithi103.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:41:35.298 INFO:journalctl@ceph.mon.b.smithi103.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:41:35.298 INFO:journalctl@ceph.mon.b.smithi103.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:41:35.298 INFO:journalctl@ceph.mon.b.smithi103.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:41:35.298 INFO:journalctl@ceph.mon.b.smithi103.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:41:35.298 INFO:journalctl@ceph.mon.b.smithi103.stdout: Cumulative compaction: 0.00 GB write, 0.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:41:35.299 INFO:journalctl@ceph.mon.b.smithi103.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:41:35.299 INFO:journalctl@ceph.mon.b.smithi103.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:41:35.299 INFO:journalctl@ceph.mon.b.smithi103.stdout: 2024-09-10T18:41:35.299 INFO:journalctl@ceph.mon.b.smithi103.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:41:35.299 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-10T18:41:35.299 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: using public_addr v2:172.21.15.103:0/0 -> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] 2024-09-10T18:41:35.299 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: starting mon.b rank -1 at public addrs [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] at bind addrs [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:41:35.299 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:34 smithi103 ceph-mon[33163]: mon.b@-1(???) e0 preinit fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:41:35.299 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:35 smithi103 ceph-mon[33163]: mon.b@-1(synchronizing).mds e1 new map 2024-09-10T18:41:35.299 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:35 smithi103 ceph-mon[33163]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-10T18:41:35.299 INFO:journalctl@ceph.mon.b.smithi103.stdout: e1 2024-09-10T18:41:35.299 INFO:journalctl@ceph.mon.b.smithi103.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T18:41:35.299 INFO:journalctl@ceph.mon.b.smithi103.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T18:41:35.299 INFO:journalctl@ceph.mon.b.smithi103.stdout: legacy client fscid: -1 2024-09-10T18:41:35.300 INFO:journalctl@ceph.mon.b.smithi103.stdout: 2024-09-10T18:41:35.300 INFO:journalctl@ceph.mon.b.smithi103.stdout: No filesystems configured 2024-09-10T18:41:35.300 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:35 smithi103 ceph-mon[33163]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-10T18:41:35.300 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:35 smithi103 ceph-mon[33163]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-10T18:41:35.300 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:35 smithi103 ceph-mon[33163]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-10T18:41:35.300 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:35 smithi103 ceph-mon[33163]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-10T18:41:35.300 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:35 smithi103 ceph-mon[33163]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-10T18:41:35.300 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:35 smithi103 ceph-mon[33163]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-10T18:41:35.300 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:35 smithi103 ceph-mon[33163]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T18:41:35.300 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:35 smithi103 ceph-mon[33163]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:41:35.300 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:35 smithi103 ceph-mon[33163]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:41:35.300 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:35 smithi103 ceph-mon[33163]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:41:35.300 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:35 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1376742874' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T18:41:35.301 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:35 smithi103 ceph-mon[33163]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:41:35.301 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:35 smithi103 ceph-mon[33163]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:35.301 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:35 smithi103 ceph-mon[33163]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:35.301 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:35 smithi103 ceph-mon[33163]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi018:172.21.15.18=a;smithi018:[v2:172.21.15.18:3301,v1:172.21.15.18:6790]=c;smithi103:172.21.15.103=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:41:35.301 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:35 smithi103 ceph-mon[33163]: Saving service mon spec with placement smithi018:172.21.15.18=a;smithi018:[v2:172.21.15.18:3301,v1:172.21.15.18:6790]=c;smithi103:172.21.15.103=b;count:3 2024-09-10T18:41:35.301 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:35 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:35.301 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:35 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:41:35.301 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:35 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:35.301 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:35 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:41:35.301 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:35 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:35.301 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:35 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:41:35.301 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:35 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:35.301 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:35 smithi103 ceph-mon[33163]: Deploying daemon mon.b on smithi103 2024-09-10T18:41:35.302 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:35 smithi103 ceph-mon[33163]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:35.302 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:35 smithi103 ceph-mon[33163]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:35.302 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:35 smithi103 ceph-mon[33163]: from='client.? 172.21.15.103:0/4171381390' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:41:35.302 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:35 smithi103 ceph-mon[33163]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-10T18:41:35.302 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:35 smithi103 bash[33131]: f40a52d205a9fc4a55dfa78ebc60f5e3173b3d4b5ed9105e06d82f397dcde16a 2024-09-10T18:41:35.302 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:35 smithi103 systemd[1]: Started Ceph mon.b for ba327f6c-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:41:36.564 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T18:41:36.564 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T18:41:36.832 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:41:40.349 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:40 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:41:40.349 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:40 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:41:40.349 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:40 smithi103 ceph-mon[33163]: mon.a calling monitor election 2024-09-10T18:41:40.349 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:40 smithi103 ceph-mon[33163]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:40.349 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:40 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:41:40.349 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:40 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:41:40.349 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:40 smithi103 ceph-mon[33163]: mon.b calling monitor election 2024-09-10T18:41:40.349 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:40 smithi103 ceph-mon[33163]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:40.349 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:40 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:41:40.349 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:40 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:41:40.350 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:40 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:41:40.350 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:40 smithi103 ceph-mon[33163]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T18:41:40.350 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:40 smithi103 ceph-mon[33163]: monmap e2: 2 mons at {a=[v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0],b=[v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:41:40.350 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:40 smithi103 ceph-mon[33163]: fsmap 2024-09-10T18:41:40.350 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:40 smithi103 ceph-mon[33163]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:41:40.350 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:40 smithi103 ceph-mon[33163]: mgrmap e13: y(active, since 40s) 2024-09-10T18:41:40.350 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:40 smithi103 ceph-mon[33163]: overall HEALTH_OK 2024-09-10T18:41:40.350 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:40 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:40.350 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:40 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:40.350 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:40 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:40.350 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:40 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:41:40.350 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:40 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:40.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:40 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:41:40.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:40 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:41:40.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:40 smithi018 ceph-mon[29336]: mon.a calling monitor election 2024-09-10T18:41:40.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:40 smithi018 ceph-mon[29336]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:40.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:40 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:41:40.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:40 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:41:40.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:40 smithi018 ceph-mon[29336]: mon.b calling monitor election 2024-09-10T18:41:40.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:40 smithi018 ceph-mon[29336]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:40.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:40 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:41:40.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:40 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:41:40.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:40 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:41:40.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:40 smithi018 ceph-mon[29336]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T18:41:40.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:40 smithi018 ceph-mon[29336]: monmap e2: 2 mons at {a=[v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0],b=[v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:41:40.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:40 smithi018 ceph-mon[29336]: fsmap 2024-09-10T18:41:40.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:40 smithi018 ceph-mon[29336]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:41:40.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:40 smithi018 ceph-mon[29336]: mgrmap e13: y(active, since 40s) 2024-09-10T18:41:40.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:40 smithi018 ceph-mon[29336]: overall HEALTH_OK 2024-09-10T18:41:40.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:40 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:40.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:40 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:40.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:40 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:40.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:40 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:41:40.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:40 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:41.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:41 smithi018 ceph-mon[29336]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:41.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:41 smithi018 ceph-mon[29336]: Deploying daemon mon.c on smithi018 2024-09-10T18:41:41.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:41 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:41:41.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:41 smithi103 ceph-mon[33163]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:41.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:41 smithi103 ceph-mon[33163]: Deploying daemon mon.c on smithi018 2024-09-10T18:41:41.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:41 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:41:41.539 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:41:41.539 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":2,"fsid":"ba327f6c-6fa3-11ef-bcea-c7b262605968","modified":"2024-09-10T18:41:35.022939Z","created":"2024-09-10T18:40:04.849809Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:3300","nonce":0},{"type":"v1","addr":"172.21.15.18:6789","nonce":0}]},"addr":"172.21.15.18:6789/0","public_addr":"172.21.15.18:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:3300","nonce":0},{"type":"v1","addr":"172.21.15.103:6789","nonce":0}]},"addr":"172.21.15.103:6789/0","public_addr":"172.21.15.103:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T18:41:41.540 INFO:teuthology.orchestra.run.smithi103.stderr:dumped monmap epoch 2 2024-09-10T18:41:42.287 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:41:42 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:41:42.023+0000 7f116001b640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-10T18:41:42.287 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:42 smithi018 ceph-mon[29336]: from='client.? 172.21.15.103:0/1079543880' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:41:42.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:42 smithi103 ceph-mon[33163]: from='client.? 172.21.15.103:0/1079543880' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:41:42.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:42 smithi018 podman[40177]: 2024-09-10 18:41:42.895933511 +0000 UTC m=+0.129707537 container create 2d199c7c18be616fc886c24a687b07c35f5e304347e2d50517fbceddef6dac02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mon-c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-10T18:41:43.144 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T18:41:43.144 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T18:41:43.396 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:41:43.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[29336]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:43.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 podman[40177]: 2024-09-10 18:41:43.123814474 +0000 UTC m=+0.357588501 container init 2d199c7c18be616fc886c24a687b07c35f5e304347e2d50517fbceddef6dac02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mon-c, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2) 2024-09-10T18:41:43.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 podman[40177]: 2024-09-10 18:41:43.127627885 +0000 UTC m=+0.361401917 container start 2d199c7c18be616fc886c24a687b07c35f5e304347e2d50517fbceddef6dac02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mon-c, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:41:43.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T18:41:43.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable), process ceph-mon, pid 2 2024-09-10T18:41:43.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: pidfile_write: ignore empty --pid-file 2024-09-10T18:41:43.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: load: jerasure load: lrc 2024-09-10T18:41:43.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: RocksDB version: 6.15.5 2024-09-10T18:41:43.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-10T18:41:43.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Compile date Sep 9 2024 2024-09-10T18:41:43.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: DB SUMMARY 2024-09-10T18:41:43.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: DB Session ID: Y8Y9GT91ASSZWDZELT2S 2024-09-10T18:41:43.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: CURRENT file: CURRENT 2024-09-10T18:41:43.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T18:41:43.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-09-10T18:41:43.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-10T18:41:43.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 636 ; 2024-09-10T18:41:43.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.error_if_exists: 0 2024-09-10T18:41:43.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.create_if_missing: 0 2024-09-10T18:41:43.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.paranoid_checks: 1 2024-09-10T18:41:43.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T18:41:43.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.env: 0x562d79119440 2024-09-10T18:41:43.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.fs: Posix File System 2024-09-10T18:41:43.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.info_log: 0x562d7b8f0ec0 2024-09-10T18:41:43.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T18:41:43.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.statistics: (nil) 2024-09-10T18:41:43.447 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.use_fsync: 0 2024-09-10T18:41:43.447 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.max_log_file_size: 0 2024-09-10T18:41:43.447 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T18:41:43.448 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T18:41:43.448 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T18:41:43.448 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T18:41:43.448 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.allow_fallocate: 1 2024-09-10T18:41:43.448 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T18:41:43.448 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T18:41:43.448 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.use_direct_reads: 0 2024-09-10T18:41:43.448 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T18:41:43.448 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T18:41:43.448 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.db_log_dir: 2024-09-10T18:41:43.448 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-c/store.db 2024-09-10T18:41:43.448 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T18:41:43.448 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T18:41:43.449 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T18:41:43.449 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T18:41:43.449 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T18:41:43.449 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T18:41:43.449 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T18:41:43.449 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T18:41:43.449 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.write_buffer_manager: 0x562d7b9ae7b0 2024-09-10T18:41:43.449 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T18:41:43.449 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-10T18:41:43.449 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T18:41:43.449 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T18:41:43.449 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T18:41:43.450 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T18:41:43.450 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T18:41:43.450 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T18:41:43.450 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T18:41:43.450 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.unordered_write: 0 2024-09-10T18:41:43.450 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T18:41:43.450 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T18:41:43.450 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T18:41:43.450 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T18:41:43.450 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.row_cache: None 2024-09-10T18:41:43.450 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.wal_filter: None 2024-09-10T18:41:43.450 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T18:41:43.450 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T18:41:43.451 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.preserve_deletes: 0 2024-09-10T18:41:43.451 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.two_write_queues: 0 2024-09-10T18:41:43.451 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T18:41:43.451 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.atomic_flush: 0 2024-09-10T18:41:43.451 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T18:41:43.451 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T18:41:43.451 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T18:41:43.451 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.log_readahead_size: 0 2024-09-10T18:41:43.451 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T18:41:43.451 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T18:41:43.451 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T18:41:43.451 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T18:41:43.451 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T18:41:43.451 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T18:41:43.452 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.max_background_jobs: 2 2024-09-10T18:41:43.452 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.max_background_compactions: -1 2024-09-10T18:41:43.452 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.max_subcompactions: 1 2024-09-10T18:41:43.452 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T18:41:43.452 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T18:41:43.452 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T18:41:43.452 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T18:41:43.452 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T18:41:43.452 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T18:41:43.452 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T18:41:43.452 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T18:41:43.452 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.max_open_files: -1 2024-09-10T18:41:43.452 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T18:41:43.453 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T18:41:43.453 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T18:41:43.453 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T18:41:43.453 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.max_background_flushes: -1 2024-09-10T18:41:43.453 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Compression algorithms supported: 2024-09-10T18:41:43.453 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: kZSTD supported: 0 2024-09-10T18:41:43.453 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: kXpressCompression supported: 0 2024-09-10T18:41:43.453 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: kBZip2Compression supported: 0 2024-09-10T18:41:43.453 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T18:41:43.453 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: kLZ4Compression supported: 1 2024-09-10T18:41:43.453 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: kZlibCompression supported: 1 2024-09-10T18:41:43.453 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T18:41:43.453 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: kSnappyCompression supported: 1 2024-09-10T18:41:43.453 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T18:41:43.454 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 2024-09-10T18:41:43.454 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-10T18:41:43.454 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T18:41:43.454 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.merge_operator: 2024-09-10T18:41:43.454 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.compaction_filter: None 2024-09-10T18:41:43.454 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T18:41:43.454 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T18:41:43.454 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T18:41:43.454 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T18:41:43.454 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562d7b897a38) 2024-09-10T18:41:43.454 INFO:journalctl@ceph.mon.c.smithi018.stdout: cache_index_and_filter_blocks: 1 2024-09-10T18:41:43.454 INFO:journalctl@ceph.mon.c.smithi018.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T18:41:43.454 INFO:journalctl@ceph.mon.c.smithi018.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T18:41:43.454 INFO:journalctl@ceph.mon.c.smithi018.stdout: pin_top_level_index_and_filter: 1 2024-09-10T18:41:43.455 INFO:journalctl@ceph.mon.c.smithi018.stdout: index_type: 0 2024-09-10T18:41:43.455 INFO:journalctl@ceph.mon.c.smithi018.stdout: data_block_index_type: 0 2024-09-10T18:41:43.455 INFO:journalctl@ceph.mon.c.smithi018.stdout: index_shortening: 1 2024-09-10T18:41:43.455 INFO:journalctl@ceph.mon.c.smithi018.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T18:41:43.455 INFO:journalctl@ceph.mon.c.smithi018.stdout: hash_index_allow_collision: 1 2024-09-10T18:41:43.455 INFO:journalctl@ceph.mon.c.smithi018.stdout: checksum: 1 2024-09-10T18:41:43.455 INFO:journalctl@ceph.mon.c.smithi018.stdout: no_block_cache: 0 2024-09-10T18:41:43.455 INFO:journalctl@ceph.mon.c.smithi018.stdout: block_cache: 0x562d7b8cd1f0 2024-09-10T18:41:43.455 INFO:journalctl@ceph.mon.c.smithi018.stdout: block_cache_name: BinnedLRUCache 2024-09-10T18:41:43.455 INFO:journalctl@ceph.mon.c.smithi018.stdout: block_cache_options: 2024-09-10T18:41:43.455 INFO:journalctl@ceph.mon.c.smithi018.stdout: capacity : 536870912 2024-09-10T18:41:43.455 INFO:journalctl@ceph.mon.c.smithi018.stdout: num_shard_bits : 4 2024-09-10T18:41:43.455 INFO:journalctl@ceph.mon.c.smithi018.stdout: strict_capacity_limit : 0 2024-09-10T18:41:43.456 INFO:journalctl@ceph.mon.c.smithi018.stdout: high_pri_pool_ratio: 0.000 2024-09-10T18:41:43.456 INFO:journalctl@ceph.mon.c.smithi018.stdout: block_cache_compressed: (nil) 2024-09-10T18:41:43.456 INFO:journalctl@ceph.mon.c.smithi018.stdout: persistent_cache: (nil) 2024-09-10T18:41:43.456 INFO:journalctl@ceph.mon.c.smithi018.stdout: block_size: 4096 2024-09-10T18:41:43.456 INFO:journalctl@ceph.mon.c.smithi018.stdout: block_size_deviation: 10 2024-09-10T18:41:43.456 INFO:journalctl@ceph.mon.c.smithi018.stdout: block_restart_interval: 16 2024-09-10T18:41:43.456 INFO:journalctl@ceph.mon.c.smithi018.stdout: index_block_restart_interval: 1 2024-09-10T18:41:43.456 INFO:journalctl@ceph.mon.c.smithi018.stdout: metadata_block_size: 4096 2024-09-10T18:41:43.456 INFO:journalctl@ceph.mon.c.smithi018.stdout: partition_filters: 0 2024-09-10T18:41:43.456 INFO:journalctl@ceph.mon.c.smithi018.stdout: use_delta_encoding: 1 2024-09-10T18:41:43.456 INFO:journalctl@ceph.mon.c.smithi018.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-10T18:41:43.456 INFO:journalctl@ceph.mon.c.smithi018.stdout: whole_key_filtering: 1 2024-09-10T18:41:43.456 INFO:journalctl@ceph.mon.c.smithi018.stdout: verify_compression: 0 2024-09-10T18:41:43.456 INFO:journalctl@ceph.mon.c.smithi018.stdout: read_amp_bytes_per_bit: 0 2024-09-10T18:41:43.457 INFO:journalctl@ceph.mon.c.smithi018.stdout: format_version: 4 2024-09-10T18:41:43.457 INFO:journalctl@ceph.mon.c.smithi018.stdout: enable_index_compression: 1 2024-09-10T18:41:43.457 INFO:journalctl@ceph.mon.c.smithi018.stdout: block_align: 0 2024-09-10T18:41:43.457 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T18:41:43.457 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T18:41:43.457 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.compression: NoCompression 2024-09-10T18:41:43.457 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T18:41:43.457 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T18:41:43.457 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T18:41:43.457 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.num_levels: 7 2024-09-10T18:41:43.457 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T18:41:43.457 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T18:41:43.457 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T18:41:43.458 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T18:41:43.458 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T18:41:43.458 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T18:41:43.458 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T18:41:43.458 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:41:43.458 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T18:41:43.458 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T18:41:43.458 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T18:41:43.458 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T18:41:43.458 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T18:41:43.458 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T18:41:43.458 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:41:43.458 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T18:41:43.458 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T18:41:43.459 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T18:41:43.459 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T18:41:43.459 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T18:41:43.459 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T18:41:43.459 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T18:41:43.459 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T18:41:43.460 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T18:41:43.460 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T18:41:43.460 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T18:41:43.460 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T18:41:43.460 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T18:41:43.460 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T18:41:43.460 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T18:41:43.460 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T18:41:43.460 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T18:41:43.460 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T18:41:43.460 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T18:41:43.460 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.arena_block_size: 4194304 2024-09-10T18:41:43.460 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T18:41:43.461 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T18:41:43.461 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-10T18:41:43.461 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T18:41:43.461 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T18:41:43.461 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T18:41:43.461 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T18:41:43.461 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T18:41:43.461 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T18:41:43.461 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T18:41:43.461 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T18:41:43.461 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T18:41:43.461 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T18:41:43.461 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T18:41:43.462 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T18:41:43.462 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.inplace_update_support: 0 2024-09-10T18:41:43.462 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T18:41:43.462 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T18:41:43.462 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T18:41:43.462 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T18:41:43.462 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.bloom_locality: 0 2024-09-10T18:41:43.462 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.max_successive_merges: 0 2024-09-10T18:41:43.462 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T18:41:43.462 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T18:41:43.462 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T18:41:43.462 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T18:41:43.462 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.ttl: 2592000 2024-09-10T18:41:43.462 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T18:41:43.463 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.enable_blob_files: false 2024-09-10T18:41:43.463 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.min_blob_size: 0 2024-09-10T18:41:43.463 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T18:41:43.463 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T18:41:43.463 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T18:41:43.463 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T18:41:43.463 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T18:41:43.464 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-09-10T18:41:43.464 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-09-10T18:41:43.464 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725993703151703, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-10T18:41:43.464 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-09-10T18:41:43.464 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-10T18:41:43.464 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725993703152020, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1696, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1725993703, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "c4a36141-bb4a-41d2-8c17-6e5f6ecc0016", "db_session_id": "Y8Y9GT91ASSZWDZELT2S"}} 2024-09-10T18:41:43.465 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-09-10T18:41:43.465 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725993703152355, "job": 1, "event": "recovery_finished"} 2024-09-10T18:41:43.465 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T18:41:43.465 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x562d7b900700 2024-09-10T18:41:43.465 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: DB pointer 0x562d7b8d0000 2024-09-10T18:41:43.465 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-10T18:41:43.465 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-10T18:41:43.465 INFO:journalctl@ceph.mon.c.smithi018.stdout: ** DB Stats ** 2024-09-10T18:41:43.465 INFO:journalctl@ceph.mon.c.smithi018.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:41:43.465 INFO:journalctl@ceph.mon.c.smithi018.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T18:41:43.465 INFO:journalctl@ceph.mon.c.smithi018.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T18:41:43.465 INFO:journalctl@ceph.mon.c.smithi018.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:41:43.466 INFO:journalctl@ceph.mon.c.smithi018.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T18:41:43.466 INFO:journalctl@ceph.mon.c.smithi018.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-10T18:41:43.466 INFO:journalctl@ceph.mon.c.smithi018.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:41:43.466 INFO:journalctl@ceph.mon.c.smithi018.stdout: 2024-09-10T18:41:43.466 INFO:journalctl@ceph.mon.c.smithi018.stdout: ** Compaction Stats [default] ** 2024-09-10T18:41:43.466 INFO:journalctl@ceph.mon.c.smithi018.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:41:43.466 INFO:journalctl@ceph.mon.c.smithi018.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:41:43.466 INFO:journalctl@ceph.mon.c.smithi018.stdout: L0 1/0 1.66 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-09-10T18:41:43.466 INFO:journalctl@ceph.mon.c.smithi018.stdout: Sum 1/0 1.66 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-09-10T18:41:43.466 INFO:journalctl@ceph.mon.c.smithi018.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-09-10T18:41:43.466 INFO:journalctl@ceph.mon.c.smithi018.stdout: 2024-09-10T18:41:43.466 INFO:journalctl@ceph.mon.c.smithi018.stdout: ** Compaction Stats [default] ** 2024-09-10T18:41:43.466 INFO:journalctl@ceph.mon.c.smithi018.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:41:43.467 INFO:journalctl@ceph.mon.c.smithi018.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:41:43.467 INFO:journalctl@ceph.mon.c.smithi018.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-09-10T18:41:43.467 INFO:journalctl@ceph.mon.c.smithi018.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:41:43.467 INFO:journalctl@ceph.mon.c.smithi018.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:41:43.467 INFO:journalctl@ceph.mon.c.smithi018.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:41:43.467 INFO:journalctl@ceph.mon.c.smithi018.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:41:43.467 INFO:journalctl@ceph.mon.c.smithi018.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:41:43.467 INFO:journalctl@ceph.mon.c.smithi018.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:41:43.467 INFO:journalctl@ceph.mon.c.smithi018.stdout: Cumulative compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:41:43.467 INFO:journalctl@ceph.mon.c.smithi018.stdout: Interval compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:41:43.467 INFO:journalctl@ceph.mon.c.smithi018.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:41:43.467 INFO:journalctl@ceph.mon.c.smithi018.stdout: 2024-09-10T18:41:43.468 INFO:journalctl@ceph.mon.c.smithi018.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:41:43.468 INFO:journalctl@ceph.mon.c.smithi018.stdout: 2024-09-10T18:41:43.468 INFO:journalctl@ceph.mon.c.smithi018.stdout: ** Compaction Stats [default] ** 2024-09-10T18:41:43.468 INFO:journalctl@ceph.mon.c.smithi018.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:41:43.468 INFO:journalctl@ceph.mon.c.smithi018.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:41:43.468 INFO:journalctl@ceph.mon.c.smithi018.stdout: L0 1/0 1.66 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-09-10T18:41:43.468 INFO:journalctl@ceph.mon.c.smithi018.stdout: Sum 1/0 1.66 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-09-10T18:41:43.468 INFO:journalctl@ceph.mon.c.smithi018.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-10T18:41:43.468 INFO:journalctl@ceph.mon.c.smithi018.stdout: 2024-09-10T18:41:43.468 INFO:journalctl@ceph.mon.c.smithi018.stdout: ** Compaction Stats [default] ** 2024-09-10T18:41:43.468 INFO:journalctl@ceph.mon.c.smithi018.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:41:43.468 INFO:journalctl@ceph.mon.c.smithi018.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:41:43.469 INFO:journalctl@ceph.mon.c.smithi018.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-09-10T18:41:43.469 INFO:journalctl@ceph.mon.c.smithi018.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:41:43.469 INFO:journalctl@ceph.mon.c.smithi018.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:41:43.469 INFO:journalctl@ceph.mon.c.smithi018.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:41:43.469 INFO:journalctl@ceph.mon.c.smithi018.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:41:43.469 INFO:journalctl@ceph.mon.c.smithi018.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:41:43.469 INFO:journalctl@ceph.mon.c.smithi018.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:41:43.469 INFO:journalctl@ceph.mon.c.smithi018.stdout: Cumulative compaction: 0.00 GB write, 0.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:41:43.469 INFO:journalctl@ceph.mon.c.smithi018.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:41:43.470 INFO:journalctl@ceph.mon.c.smithi018.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:41:43.470 INFO:journalctl@ceph.mon.c.smithi018.stdout: 2024-09-10T18:41:43.470 INFO:journalctl@ceph.mon.c.smithi018.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:41:43.470 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-10T18:41:43.470 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: using public_addrv [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] 2024-09-10T18:41:43.470 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: starting mon.c rank -1 at public addrs [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] at bind addrs [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:41:43.470 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: mon.c@-1(???) e0 preinit fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:41:43.470 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: mon.c@-1(synchronizing).mds e1 new map 2024-09-10T18:41:43.470 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-10T18:41:43.470 INFO:journalctl@ceph.mon.c.smithi018.stdout: e1 2024-09-10T18:41:43.470 INFO:journalctl@ceph.mon.c.smithi018.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T18:41:43.470 INFO:journalctl@ceph.mon.c.smithi018.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T18:41:43.471 INFO:journalctl@ceph.mon.c.smithi018.stdout: legacy client fscid: -1 2024-09-10T18:41:43.471 INFO:journalctl@ceph.mon.c.smithi018.stdout: 2024-09-10T18:41:43.471 INFO:journalctl@ceph.mon.c.smithi018.stdout: No filesystems configured 2024-09-10T18:41:43.471 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-10T18:41:43.471 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-10T18:41:43.471 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-10T18:41:43.471 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-10T18:41:43.471 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-10T18:41:43.471 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-10T18:41:43.471 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T18:41:43.471 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:41:43.471 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:41:43.471 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:41:43.472 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1376742874' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T18:41:43.472 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:41:43.472 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:43.472 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:43.472 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi018:172.21.15.18=a;smithi018:[v2:172.21.15.18:3301,v1:172.21.15.18:6790]=c;smithi103:172.21.15.103=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:41:43.472 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: Saving service mon spec with placement smithi018:172.21.15.18=a;smithi018:[v2:172.21.15.18:3301,v1:172.21.15.18:6790]=c;smithi103:172.21.15.103=b;count:3 2024-09-10T18:41:43.472 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:43.472 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:41:43.472 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:43.472 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:41:43.472 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:43.472 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:41:43.473 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:43.473 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: Deploying daemon mon.b on smithi103 2024-09-10T18:41:43.473 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:43.473 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:43.473 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: from='client.? 172.21.15.103:0/4171381390' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:41:43.473 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:41:43.473 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:41:43.473 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: mon.a calling monitor election 2024-09-10T18:41:43.473 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:43.473 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:41:43.473 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:41:43.473 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: mon.b calling monitor election 2024-09-10T18:41:43.473 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:43.473 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:41:43.474 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:41:43.474 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:41:43.474 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T18:41:43.474 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: monmap e2: 2 mons at {a=[v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0],b=[v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:41:43.474 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: fsmap 2024-09-10T18:41:43.474 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:41:43.474 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: mgrmap e13: y(active, since 40s) 2024-09-10T18:41:43.474 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: overall HEALTH_OK 2024-09-10T18:41:43.474 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:43.474 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:43.474 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:43.474 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:41:43.474 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:43.475 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:43.475 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: Deploying daemon mon.c on smithi018 2024-09-10T18:41:43.475 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:41:43.475 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: from='client.? 172.21.15.103:0/1079543880' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:41:43.475 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:43.475 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 ceph-mon[40196]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-10T18:41:43.475 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 bash[40177]: 2d199c7c18be616fc886c24a687b07c35f5e304347e2d50517fbceddef6dac02 2024-09-10T18:41:43.475 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:43 smithi018 systemd[1]: Started Ceph mon.c for ba327f6c-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:41:43.516 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:43 smithi103 ceph-mon[33163]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:48.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:48 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:41:48.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:48 smithi103 ceph-mon[33163]: mon.a calling monitor election 2024-09-10T18:41:48.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:48 smithi103 ceph-mon[33163]: mon.b calling monitor election 2024-09-10T18:41:48.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:48 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:41:48.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:48 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:41:48.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:48 smithi103 ceph-mon[33163]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:48.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:48 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:41:48.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:48 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:41:48.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:48 smithi103 ceph-mon[33163]: mon.c calling monitor election 2024-09-10T18:41:48.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:48 smithi103 ceph-mon[33163]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:48.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:48 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:41:48.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:48 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:41:48.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:48 smithi103 ceph-mon[33163]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:48.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:48 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:41:48.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:48 smithi103 ceph-mon[33163]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-10T18:41:48.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:48 smithi103 ceph-mon[33163]: monmap e3: 3 mons at {a=[v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0],b=[v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0],c=[v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:41:48.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:48 smithi103 ceph-mon[33163]: fsmap 2024-09-10T18:41:48.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:48 smithi103 ceph-mon[33163]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:41:48.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:48 smithi103 ceph-mon[33163]: mgrmap e13: y(active, since 48s) 2024-09-10T18:41:48.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:48 smithi103 ceph-mon[33163]: overall HEALTH_OK 2024-09-10T18:41:48.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:48 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:48.525 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:48 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:48.525 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:48 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:48.525 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:48 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:48.591 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T18:41:48.591 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":3,"fsid":"ba327f6c-6fa3-11ef-bcea-c7b262605968","modified":"2024-09-10T18:41:43.174252Z","created":"2024-09-10T18:40:04.849809Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:3300","nonce":0},{"type":"v1","addr":"172.21.15.18:6789","nonce":0}]},"addr":"172.21.15.18:6789/0","public_addr":"172.21.15.18:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:3300","nonce":0},{"type":"v1","addr":"172.21.15.103:6789","nonce":0}]},"addr":"172.21.15.103:6789/0","public_addr":"172.21.15.103:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:3301","nonce":0},{"type":"v1","addr":"172.21.15.18:6790","nonce":0}]},"addr":"172.21.15.18:6790/0","public_addr":"172.21.15.18:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-10T18:41:48.591 INFO:teuthology.orchestra.run.smithi103.stderr:dumped monmap epoch 3 2024-09-10T18:41:48.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:41:48.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[29336]: mon.a calling monitor election 2024-09-10T18:41:48.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[29336]: mon.b calling monitor election 2024-09-10T18:41:48.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:41:48.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:41:48.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[29336]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:48.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:41:48.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:41:48.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[29336]: mon.c calling monitor election 2024-09-10T18:41:48.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[29336]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:48.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:41:48.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:41:48.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[29336]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:48.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:41:48.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[29336]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-10T18:41:48.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[29336]: monmap e3: 3 mons at {a=[v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0],b=[v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0],c=[v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:41:48.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[29336]: fsmap 2024-09-10T18:41:48.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[29336]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:41:48.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[29336]: mgrmap e13: y(active, since 48s) 2024-09-10T18:41:48.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[29336]: overall HEALTH_OK 2024-09-10T18:41:48.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:48.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:48.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:48.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:48.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:41:48.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[40196]: mon.a calling monitor election 2024-09-10T18:41:48.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[40196]: mon.b calling monitor election 2024-09-10T18:41:48.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:41:48.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:41:48.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[40196]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:48.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:41:48.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:41:48.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[40196]: mon.c calling monitor election 2024-09-10T18:41:48.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[40196]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:48.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:41:48.698 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:41:48.698 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[40196]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:48.698 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:41:48.698 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[40196]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-10T18:41:48.698 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[40196]: monmap e3: 3 mons at {a=[v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0],b=[v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0],c=[v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:41:48.698 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[40196]: fsmap 2024-09-10T18:41:48.698 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[40196]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:41:48.698 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[40196]: mgrmap e13: y(active, since 48s) 2024-09-10T18:41:48.698 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[40196]: overall HEALTH_OK 2024-09-10T18:41:48.698 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:48.698 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:48.698 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:48.698 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:48 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:49.180 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-10T18:41:49.181 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-10T18:41:49.438 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:41:49.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:49 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:41:49.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:49 smithi103 ceph-mon[33163]: from='client.? 172.21.15.103:0/1969144223' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:41:49.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:49 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:41:49.559 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:49 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:41:49.559 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:49 smithi018 ceph-mon[29336]: from='client.? 172.21.15.103:0/1969144223' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:41:49.560 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:49 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:41:49.560 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:49 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:41:49.560 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:49 smithi018 ceph-mon[40196]: from='client.? 172.21.15.103:0/1969144223' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:41:49.560 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:49 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:41:50.661 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:50 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:50.661 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:50 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:50.661 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:50 smithi018 ceph-mon[29336]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:50.661 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:50 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:50.662 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:50 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:50.662 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:50 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:50.662 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:50 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:50.662 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:50 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:41:50.662 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:50 smithi018 ceph-mon[29336]: Updating smithi018:/etc/ceph/ceph.conf 2024-09-10T18:41:50.662 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:50 smithi018 ceph-mon[29336]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-10T18:41:50.662 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:50 smithi018 ceph-mon[29336]: Updating smithi103:/var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:41:50.662 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:50 smithi018 ceph-mon[29336]: Updating smithi018:/var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:41:50.662 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:50 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:50.662 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:50 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:50.662 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:50 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:50.663 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:50 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:50.663 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:50 smithi018 ceph-mon[40196]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:50.663 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:50 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:50.663 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:50 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:50.663 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:50 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:50.663 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:50 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:50.663 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:50 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:41:50.663 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:50 smithi018 ceph-mon[40196]: Updating smithi018:/etc/ceph/ceph.conf 2024-09-10T18:41:50.663 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:50 smithi018 ceph-mon[40196]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-10T18:41:50.663 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:50 smithi018 ceph-mon[40196]: Updating smithi103:/var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:41:50.663 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:50 smithi018 ceph-mon[40196]: Updating smithi018:/var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:41:50.663 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:50 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:50.663 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:50 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:50.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:50 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:50.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:50 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:50.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:50 smithi103 ceph-mon[33163]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:50.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:50 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:50.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:50 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:50.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:50 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:50.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:50 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:50.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:50 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:41:50.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:50 smithi103 ceph-mon[33163]: Updating smithi018:/etc/ceph/ceph.conf 2024-09-10T18:41:50.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:50 smithi103 ceph-mon[33163]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-10T18:41:50.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:50 smithi103 ceph-mon[33163]: Updating smithi103:/var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:41:50.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:50 smithi103 ceph-mon[33163]: Updating smithi018:/var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:41:50.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:50 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:50.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:50 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:51.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:51 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:51.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:51 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:51.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:51 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:51.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:51 smithi018 ceph-mon[29336]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T18:41:51.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:51 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:41:51.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:51 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:41:51.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:51 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:51.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:51 smithi018 ceph-mon[29336]: Reconfiguring daemon mon.a on smithi018 2024-09-10T18:41:51.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:51 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:51.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:51 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:51.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:51 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:51.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:51 smithi018 ceph-mon[40196]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T18:41:51.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:51 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:41:51.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:51 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:41:51.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:51 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:51.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:51 smithi018 ceph-mon[40196]: Reconfiguring daemon mon.a on smithi018 2024-09-10T18:41:51.568 INFO:teuthology.orchestra.run.smithi018.stdout:# minimal ceph.conf for ba327f6c-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:41:51.568 INFO:teuthology.orchestra.run.smithi018.stdout:[global] 2024-09-10T18:41:51.568 INFO:teuthology.orchestra.run.smithi018.stdout: fsid = ba327f6c-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:41:51.569 INFO:teuthology.orchestra.run.smithi018.stdout: mon_host = [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] [v2:172.21.15.18:3301/0,v1:172.21.15.18:6790/0] 2024-09-10T18:41:51.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:51 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:51.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:51 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:51.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:51 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:51.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:51 smithi103 ceph-mon[33163]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T18:41:51.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:51 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:41:51.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:51 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:41:51.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:51 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:51.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:51 smithi103 ceph-mon[33163]: Reconfiguring daemon mon.a on smithi018 2024-09-10T18:41:52.675 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:52 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1673035406' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:52.675 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:52 smithi018 ceph-mon[40196]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:52.676 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:52 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:52.676 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:52 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:52.676 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:52 smithi018 ceph-mon[40196]: Reconfiguring mon.c (monmap changed)... 2024-09-10T18:41:52.676 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:52 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:41:52.676 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:52 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:41:52.676 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:52 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:52.676 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:52 smithi018 ceph-mon[40196]: Reconfiguring daemon mon.c on smithi018 2024-09-10T18:41:52.676 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:52 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1673035406' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:52.676 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:52 smithi018 ceph-mon[29336]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:52.677 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:52 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:52.677 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:52 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:52.677 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:52 smithi018 ceph-mon[29336]: Reconfiguring mon.c (monmap changed)... 2024-09-10T18:41:52.677 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:52 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:41:52.677 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:52 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:41:52.677 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:52 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:52.677 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:52 smithi018 ceph-mon[29336]: Reconfiguring daemon mon.c on smithi018 2024-09-10T18:41:52.700 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-10T18:41:52.701 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-10T18:41:52.701 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T18:41:52.730 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-10T18:41:52.730 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:41:52.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:52 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1673035406' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:52.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:52 smithi103 ceph-mon[33163]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:52.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:52 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:52.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:52 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:52.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:52 smithi103 ceph-mon[33163]: Reconfiguring mon.c (monmap changed)... 2024-09-10T18:41:52.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:52 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:41:52.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:52 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:41:52.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:52 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:52.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:52 smithi103 ceph-mon[33163]: Reconfiguring daemon mon.c on smithi018 2024-09-10T18:41:52.797 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-10T18:41:52.797 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T18:41:52.830 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-10T18:41:52.830 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:41:52.899 INFO:tasks.cephadm:Adding mgr.y on smithi018 2024-09-10T18:41:52.899 INFO:tasks.cephadm:Adding mgr.x on smithi103 2024-09-10T18:41:52.899 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph orch apply mgr '2;smithi018=y;smithi103=x' 2024-09-10T18:41:53.177 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:41:54.969 INFO:teuthology.orchestra.run.smithi103.stdout:Scheduled mgr update... 2024-09-10T18:41:55.100 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:54 smithi103 ceph-mon[33163]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:55.100 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:54 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:55.100 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:54 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:55.100 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:54 smithi103 ceph-mon[33163]: Reconfiguring mon.b (monmap changed)... 2024-09-10T18:41:55.100 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:54 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:41:55.100 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:54 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:41:55.100 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:54 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:55.100 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:54 smithi103 ceph-mon[33163]: Reconfiguring daemon mon.b on smithi103 2024-09-10T18:41:55.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:54 smithi018 ceph-mon[40196]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:55.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:54 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:55.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:54 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:55.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:54 smithi018 ceph-mon[40196]: Reconfiguring mon.b (monmap changed)... 2024-09-10T18:41:55.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:54 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:41:55.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:54 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:41:55.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:54 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:55.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:54 smithi018 ceph-mon[40196]: Reconfiguring daemon mon.b on smithi103 2024-09-10T18:41:55.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:54 smithi018 ceph-mon[29336]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:55.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:54 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:55.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:54 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:55.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:54 smithi018 ceph-mon[29336]: Reconfiguring mon.b (monmap changed)... 2024-09-10T18:41:55.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:54 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:41:55.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:54 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:41:55.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:54 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:55.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:54 smithi018 ceph-mon[29336]: Reconfiguring daemon mon.b on smithi103 2024-09-10T18:41:55.870 DEBUG:teuthology.orchestra.run.smithi103:mgr.x> sudo journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@mgr.x.service 2024-09-10T18:41:55.872 INFO:tasks.cephadm:Deploying OSDs... 2024-09-10T18:41:55.872 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-10T18:41:55.872 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T18:41:55.892 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T18:41:55.893 DEBUG:teuthology.orchestra.run.smithi018:> stat /dev/vg_nvme/lv_1 2024-09-10T18:41:55.952 INFO:teuthology.orchestra.run.smithi018.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T18:41:55.952 INFO:teuthology.orchestra.run.smithi018.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:41:55.952 INFO:teuthology.orchestra.run.smithi018.stdout:Device: 5h/5d Inode: 972 Links: 1 2024-09-10T18:41:55.953 INFO:teuthology.orchestra.run.smithi018.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:41:55.953 INFO:teuthology.orchestra.run.smithi018.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:41:55.953 INFO:teuthology.orchestra.run.smithi018.stdout:Access: 2024-09-10 18:41:42.601713643 +0000 2024-09-10T18:41:55.953 INFO:teuthology.orchestra.run.smithi018.stdout:Modify: 2024-09-10 18:35:17.542432578 +0000 2024-09-10T18:41:55.953 INFO:teuthology.orchestra.run.smithi018.stdout:Change: 2024-09-10 18:35:17.542432578 +0000 2024-09-10T18:41:55.953 INFO:teuthology.orchestra.run.smithi018.stdout: Birth: 2024-09-10 18:35:17.542432578 +0000 2024-09-10T18:41:55.953 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T18:41:56.023 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records in 2024-09-10T18:41:56.023 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records out 2024-09-10T18:41:56.024 INFO:teuthology.orchestra.run.smithi018.stderr:512 bytes copied, 0.00015021 s, 3.4 MB/s 2024-09-10T18:41:56.025 DEBUG:teuthology.orchestra.run.smithi018:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T18:41:56.085 DEBUG:teuthology.orchestra.run.smithi018:> stat /dev/vg_nvme/lv_2 2024-09-10T18:41:56.143 INFO:teuthology.orchestra.run.smithi018.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T18:41:56.143 INFO:teuthology.orchestra.run.smithi018.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:41:56.143 INFO:teuthology.orchestra.run.smithi018.stdout:Device: 5h/5d Inode: 958 Links: 1 2024-09-10T18:41:56.144 INFO:teuthology.orchestra.run.smithi018.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:41:56.144 INFO:teuthology.orchestra.run.smithi018.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:41:56.144 INFO:teuthology.orchestra.run.smithi018.stdout:Access: 2024-09-10 18:41:42.601713643 +0000 2024-09-10T18:41:56.144 INFO:teuthology.orchestra.run.smithi018.stdout:Modify: 2024-09-10 18:35:17.540432603 +0000 2024-09-10T18:41:56.144 INFO:teuthology.orchestra.run.smithi018.stdout:Change: 2024-09-10 18:35:17.540432603 +0000 2024-09-10T18:41:56.144 INFO:teuthology.orchestra.run.smithi018.stdout: Birth: 2024-09-10 18:35:17.540432603 +0000 2024-09-10T18:41:56.144 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T18:41:56.210 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records in 2024-09-10T18:41:56.210 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records out 2024-09-10T18:41:56.210 INFO:teuthology.orchestra.run.smithi018.stderr:512 bytes copied, 0.000110155 s, 4.6 MB/s 2024-09-10T18:41:56.212 DEBUG:teuthology.orchestra.run.smithi018:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T18:41:56.270 DEBUG:teuthology.orchestra.run.smithi018:> stat /dev/vg_nvme/lv_3 2024-09-10T18:41:56.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:55 smithi103 ceph-mon[33163]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi018=y;smithi103=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:41:56.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:55 smithi103 ceph-mon[33163]: Saving service mgr spec with placement smithi018=y;smithi103=x;count:2 2024-09-10T18:41:56.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:55 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:56.328 INFO:teuthology.orchestra.run.smithi018.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T18:41:56.328 INFO:teuthology.orchestra.run.smithi018.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:41:56.328 INFO:teuthology.orchestra.run.smithi018.stdout:Device: 5h/5d Inode: 982 Links: 1 2024-09-10T18:41:56.328 INFO:teuthology.orchestra.run.smithi018.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:41:56.328 INFO:teuthology.orchestra.run.smithi018.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:41:56.328 INFO:teuthology.orchestra.run.smithi018.stdout:Access: 2024-09-10 18:41:42.602713631 +0000 2024-09-10T18:41:56.329 INFO:teuthology.orchestra.run.smithi018.stdout:Modify: 2024-09-10 18:35:17.544432554 +0000 2024-09-10T18:41:56.329 INFO:teuthology.orchestra.run.smithi018.stdout:Change: 2024-09-10 18:35:17.544432554 +0000 2024-09-10T18:41:56.329 INFO:teuthology.orchestra.run.smithi018.stdout: Birth: 2024-09-10 18:35:17.544432554 +0000 2024-09-10T18:41:56.329 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T18:41:56.392 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:55 smithi018 ceph-mon[29336]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi018=y;smithi103=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:41:56.392 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:55 smithi018 ceph-mon[29336]: Saving service mgr spec with placement smithi018=y;smithi103=x;count:2 2024-09-10T18:41:56.392 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:55 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:56.393 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:55 smithi018 ceph-mon[40196]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi018=y;smithi103=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:41:56.393 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:55 smithi018 ceph-mon[40196]: Saving service mgr spec with placement smithi018=y;smithi103=x;count:2 2024-09-10T18:41:56.393 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:55 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:56.394 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records in 2024-09-10T18:41:56.394 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records out 2024-09-10T18:41:56.394 INFO:teuthology.orchestra.run.smithi018.stderr:512 bytes copied, 9.8225e-05 s, 5.2 MB/s 2024-09-10T18:41:56.396 DEBUG:teuthology.orchestra.run.smithi018:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T18:41:56.454 DEBUG:teuthology.orchestra.run.smithi018:> stat /dev/vg_nvme/lv_4 2024-09-10T18:41:56.511 INFO:teuthology.orchestra.run.smithi018.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T18:41:56.511 INFO:teuthology.orchestra.run.smithi018.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:41:56.511 INFO:teuthology.orchestra.run.smithi018.stdout:Device: 5h/5d Inode: 968 Links: 1 2024-09-10T18:41:56.511 INFO:teuthology.orchestra.run.smithi018.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:41:56.511 INFO:teuthology.orchestra.run.smithi018.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:41:56.512 INFO:teuthology.orchestra.run.smithi018.stdout:Access: 2024-09-10 18:41:42.602713631 +0000 2024-09-10T18:41:56.512 INFO:teuthology.orchestra.run.smithi018.stdout:Modify: 2024-09-10 18:35:17.541432590 +0000 2024-09-10T18:41:56.512 INFO:teuthology.orchestra.run.smithi018.stdout:Change: 2024-09-10 18:35:17.541432590 +0000 2024-09-10T18:41:56.512 INFO:teuthology.orchestra.run.smithi018.stdout: Birth: 2024-09-10 18:35:17.541432590 +0000 2024-09-10T18:41:56.512 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T18:41:56.577 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records in 2024-09-10T18:41:56.577 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records out 2024-09-10T18:41:56.577 INFO:teuthology.orchestra.run.smithi018.stderr:512 bytes copied, 0.000109792 s, 4.7 MB/s 2024-09-10T18:41:56.578 DEBUG:teuthology.orchestra.run.smithi018:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T18:41:56.636 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-10T18:41:56.637 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T18:41:56.654 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T18:41:56.654 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_1 2024-09-10T18:41:56.710 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T18:41:56.710 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:41:56.710 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 972 Links: 1 2024-09-10T18:41:56.710 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:41:56.710 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:41:56.710 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-10 18:41:34.260237329 +0000 2024-09-10T18:41:56.710 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-10 18:35:17.390046431 +0000 2024-09-10T18:41:56.710 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-10 18:35:17.390046431 +0000 2024-09-10T18:41:56.710 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-10 18:35:17.390046431 +0000 2024-09-10T18:41:56.711 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T18:41:56.774 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-09-10T18:41:56.774 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-09-10T18:41:56.774 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.00021336 s, 2.4 MB/s 2024-09-10T18:41:56.775 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T18:41:56.832 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_2 2024-09-10T18:41:56.888 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T18:41:56.888 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:41:56.888 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 969 Links: 1 2024-09-10T18:41:56.889 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:41:56.889 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:41:56.889 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-10 18:41:34.261237309 +0000 2024-09-10T18:41:56.889 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-10 18:35:17.390046431 +0000 2024-09-10T18:41:56.889 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-10 18:35:17.390046431 +0000 2024-09-10T18:41:56.889 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-10 18:35:17.390046431 +0000 2024-09-10T18:41:56.889 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T18:41:56.952 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-09-10T18:41:56.953 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-09-10T18:41:56.953 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000181764 s, 2.8 MB/s 2024-09-10T18:41:56.954 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T18:41:57.010 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_3 2024-09-10T18:41:57.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:56 smithi103 ceph-mon[33163]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:57.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:56 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:57.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:56 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:57.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:56 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:41:57.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:56 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:57.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:56 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:41:57.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:56 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:57.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:56 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:41:57.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:56 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T18:41:57.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:56 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:41:57.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:56 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:57.040 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T18:41:57.040 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:41:57.040 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 983 Links: 1 2024-09-10T18:41:57.040 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:41:57.040 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:41:57.040 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-10 18:41:34.261237309 +0000 2024-09-10T18:41:57.040 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-10 18:35:17.392046389 +0000 2024-09-10T18:41:57.040 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-10 18:35:17.392046389 +0000 2024-09-10T18:41:57.041 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-10 18:35:17.392046389 +0000 2024-09-10T18:41:57.041 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T18:41:57.102 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-09-10T18:41:57.102 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-09-10T18:41:57.102 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000276161 s, 1.9 MB/s 2024-09-10T18:41:57.103 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T18:41:57.159 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_4 2024-09-10T18:41:57.214 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T18:41:57.214 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:41:57.215 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 976 Links: 1 2024-09-10T18:41:57.215 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:41:57.215 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:41:57.215 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-10 18:41:34.262237288 +0000 2024-09-10T18:41:57.215 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-10 18:35:17.391046410 +0000 2024-09-10T18:41:57.215 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-10 18:35:17.391046410 +0000 2024-09-10T18:41:57.215 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-10 18:35:17.391046410 +0000 2024-09-10T18:41:57.215 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T18:41:57.278 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-09-10T18:41:57.278 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-09-10T18:41:57.278 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000269969 s, 1.9 MB/s 2024-09-10T18:41:57.279 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T18:41:57.336 INFO:tasks.cephadm:Deploying osd.0 on smithi018 with /dev/vg_nvme/lv_4... 2024-09-10T18:41:57.336 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-10T18:41:57.365 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:56 smithi018 ceph-mon[29336]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:57.365 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:56 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:57.365 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:56 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:57.365 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:56 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:41:57.365 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:56 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:57.366 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:56 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:41:57.366 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:56 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:57.366 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:56 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:41:57.366 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:56 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T18:41:57.366 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:56 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:41:57.366 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:56 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:57.366 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:56 smithi018 ceph-mon[40196]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:57.366 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:56 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:57.366 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:56 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:57.366 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:56 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:41:57.366 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:56 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:57.367 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:56 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:41:57.367 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:56 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:57.367 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:56 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:41:57.367 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:56 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T18:41:57.367 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:56 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:41:57.367 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:56 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:57.606 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:41:58.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:57 smithi018 ceph-mon[29336]: Deploying daemon mgr.x on smithi103 2024-09-10T18:41:58.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:57 smithi018 ceph-mon[40196]: Deploying daemon mgr.x on smithi103 2024-09-10T18:41:58.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:57 smithi103 ceph-mon[33163]: Deploying daemon mgr.x on smithi103 2024-09-10T18:41:58.273 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:41:58 smithi103 systemd[1]: Starting Ceph mgr.x for ba327f6c-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:41:58.601 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:41:58 smithi103 podman[34981]: 2024-09-10 18:41:58.177326984 +0000 UTC m=+0.015660411 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:41:58.602 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:41:58 smithi103 podman[34981]: 2024-09-10 18:41:58.28147697 +0000 UTC m=+0.119810392 container create 108a62debc0ba072c28a06c4bb3ab9201aac96b5b3a341dec8113bccf93055ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-10T18:41:58.602 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:41:58 smithi103 podman[34981]: 2024-09-10 18:41:58.500486521 +0000 UTC m=+0.338819952 container init 108a62debc0ba072c28a06c4bb3ab9201aac96b5b3a341dec8113bccf93055ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-10T18:41:58.602 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:41:58 smithi103 podman[34981]: 2024-09-10 18:41:58.50323923 +0000 UTC m=+0.341572652 container start 108a62debc0ba072c28a06c4bb3ab9201aac96b5b3a341dec8113bccf93055ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:41:58.602 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:41:58 smithi103 ceph-mgr[35001]: -- 172.21.15.103:0/1227326444 <== mon.1 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a095a48340 con 0x55a095a54800 2024-09-10T18:41:58.602 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:41:58 smithi103 bash[34981]: 108a62debc0ba072c28a06c4bb3ab9201aac96b5b3a341dec8113bccf93055ab 2024-09-10T18:41:58.894 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:41:58 smithi103 systemd[1]: Started Ceph mgr.x for ba327f6c-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:41:58.894 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:41:58 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:41:58.620+0000 7f06b6b69ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:41:58.894 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:41:58 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:41:58.695+0000 7f06b6b69ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:41:58.894 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:41:58 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:41:58.761+0000 7f06b6b69ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:41:58.894 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:41:58 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:41:58.892+0000 7f06b6b69ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:41:59.272 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:41:59 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:41:59.046+0000 7f06b6b69ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:41:59.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:58 smithi103 ceph-mon[33163]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:59.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:58 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:59.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:58 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:59.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:58 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:59.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:58 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:59.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:41:58 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:41:59.293 INFO:teuthology.orchestra.run.smithi018.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-10T18:41:59.293 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:58 smithi018 ceph-mon[29336]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:59.293 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:58 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:59.293 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:58 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:59.294 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:58 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:59.294 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:58 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:59.294 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:41:58 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:41:59.294 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:58 smithi018 ceph-mon[40196]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:41:59.294 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:58 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:59.294 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:58 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:59.294 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:58 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:59.294 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:58 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:41:59.294 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:41:58 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:41:59.376 INFO:teuthology.orchestra.run.smithi018.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-10T18:41:59.401 INFO:teuthology.orchestra.run.smithi018.stderr: stderr: 10+0 records in 2024-09-10T18:41:59.401 INFO:teuthology.orchestra.run.smithi018.stderr:10+0 records out 2024-09-10T18:41:59.402 INFO:teuthology.orchestra.run.smithi018.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0085945 s, 1.2 GB/s 2024-09-10T18:41:59.402 INFO:teuthology.orchestra.run.smithi018.stderr:--> Zapping successful for: 2024-09-10T18:41:59.587 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:41:59 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:41:59.323+0000 7f06b6b69ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:41:59.587 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:41:59 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:41:59.388+0000 7f06b6b69ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:41:59.587 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:41:59 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:41:59.557+0000 7f06b6b69ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:42:00.030 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi018:vg_nvme/lv_4 2024-09-10T18:42:00.704 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:42:00 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:42:00.412+0000 7f06b6b69ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:42:00.705 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:42:00 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:42:00.475+0000 7f06b6b69ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:42:00.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:00 smithi018 ceph-mon[29336]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:42:00.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:00 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:00.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:00 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:00.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:00 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:00.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:00 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:00.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:00 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:42:00.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:00 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:00.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:00 smithi018 ceph-mon[29336]: Reconfiguring mgr.y (unknown last config time)... 2024-09-10T18:42:00.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:00 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:42:00.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:00 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:42:00.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:00 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:00.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:00 smithi018 ceph-mon[29336]: Reconfiguring daemon mgr.y on smithi018 2024-09-10T18:42:00.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:00 smithi018 ceph-mon[40196]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:42:00.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:00 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:00.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:00 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:00.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:00 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:00.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:00 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:00.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:00 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:42:00.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:00 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:00.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:00 smithi018 ceph-mon[40196]: Reconfiguring mgr.y (unknown last config time)... 2024-09-10T18:42:00.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:00 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:42:00.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:00 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:42:00.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:00 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:00.947 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:00 smithi018 ceph-mon[40196]: Reconfiguring daemon mgr.y on smithi018 2024-09-10T18:42:01.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:00 smithi103 ceph-mon[33163]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:42:01.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:00 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:01.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:00 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:01.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:00 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:01.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:00 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:01.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:00 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:42:01.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:00 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:01.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:00 smithi103 ceph-mon[33163]: Reconfiguring mgr.y (unknown last config time)... 2024-09-10T18:42:01.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:00 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:42:01.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:00 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:42:01.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:00 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:01.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:00 smithi103 ceph-mon[33163]: Reconfiguring daemon mgr.y on smithi018 2024-09-10T18:42:01.384 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:42:01.479 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:42:01 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:42:01.185+0000 7f06b6b69ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:42:01.479 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:42:01 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:42:01.294+0000 7f06b6b69ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:42:01.479 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:42:01 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:42:01.417+0000 7f06b6b69ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:42:01.479 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:42:01 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:42:01.477+0000 7f06b6b69ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:42:01.772 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:42:01 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:42:01.601+0000 7f06b6b69ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:42:01.772 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:42:01 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:42:01.666+0000 7f06b6b69ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:42:01.772 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:42:01 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:42:01.723+0000 7f06b6b69ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:42:02.120 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:42:01 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:42:01.856+0000 7f06b6b69ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:42:02.121 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:42:01 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:42:01.918+0000 7f06b6b69ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:42:02.121 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:42:01 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:42:01.981+0000 7f06b6b69ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:42:02.121 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:42:02 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:42:02.062+0000 7f06b6b69ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:42:02.121 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:42:02 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:42:02.119+0000 7f06b6b69ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:42:02.460 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:02 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:02.460 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:02 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:02.460 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:02 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:42:02.461 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:02 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:02.461 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:02 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:42:02.461 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:02 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:02.461 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:02 smithi018 ceph-mon[40196]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:42:02.461 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:02 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:02.461 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:02 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:02.461 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:02 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:42:02.461 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:02 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:02.461 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:02 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:42:02.461 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:02 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:02.462 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:02 smithi018 ceph-mon[29336]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:42:02.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:42:02 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:42:02.175+0000 7f06b6b69ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:42:02.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:02 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:02.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:02 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:02.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:02 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:42:02.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:02 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:02.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:02 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:42:02.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:02 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:02.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:02 smithi103 ceph-mon[33163]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:42:03.272 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:42:02 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:42:02.926+0000 7f06b6b69ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:42:03.679 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:03 smithi018 ceph-mon[29336]: Standby manager daemon x started 2024-09-10T18:42:03.680 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:03 smithi018 ceph-mon[29336]: from='mgr.? 172.21.15.103:0/2810938472' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:42:03.680 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:03 smithi018 ceph-mon[29336]: from='mgr.? 172.21.15.103:0/2810938472' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:42:03.680 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:03 smithi018 ceph-mon[29336]: from='mgr.? 172.21.15.103:0/2810938472' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:42:03.680 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:03 smithi018 ceph-mon[29336]: from='mgr.? 172.21.15.103:0/2810938472' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:42:03.680 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:03 smithi018 ceph-mon[29336]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:42:03.680 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:03 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:42:03.680 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:03 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:42:03.680 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:03 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:03.680 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:03 smithi018 ceph-mon[40196]: Standby manager daemon x started 2024-09-10T18:42:03.680 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:03 smithi018 ceph-mon[40196]: from='mgr.? 172.21.15.103:0/2810938472' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:42:03.681 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:03 smithi018 ceph-mon[40196]: from='mgr.? 172.21.15.103:0/2810938472' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:42:03.681 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:03 smithi018 ceph-mon[40196]: from='mgr.? 172.21.15.103:0/2810938472' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:42:03.681 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:03 smithi018 ceph-mon[40196]: from='mgr.? 172.21.15.103:0/2810938472' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:42:03.681 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:03 smithi018 ceph-mon[40196]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:42:03.681 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:03 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:42:03.681 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:03 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:42:03.681 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:03 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:03.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:03 smithi103 ceph-mon[33163]: Standby manager daemon x started 2024-09-10T18:42:03.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:03 smithi103 ceph-mon[33163]: from='mgr.? 172.21.15.103:0/2810938472' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:42:03.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:03 smithi103 ceph-mon[33163]: from='mgr.? 172.21.15.103:0/2810938472' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:42:03.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:03 smithi103 ceph-mon[33163]: from='mgr.? 172.21.15.103:0/2810938472' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:42:03.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:03 smithi103 ceph-mon[33163]: from='mgr.? 172.21.15.103:0/2810938472' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:42:03.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:03 smithi103 ceph-mon[33163]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:42:03.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:03 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:42:03.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:03 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:42:03.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:03 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:04.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:04 smithi018 ceph-mon[29336]: mgrmap e14: y(active, since 63s), standbys: x 2024-09-10T18:42:04.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:04 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:42:04.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:04 smithi018 ceph-mon[29336]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:42:04.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:04 smithi018 ceph-mon[40196]: mgrmap e14: y(active, since 63s), standbys: x 2024-09-10T18:42:04.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:04 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:42:04.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:04 smithi018 ceph-mon[40196]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:42:04.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:04 smithi103 ceph-mon[33163]: mgrmap e14: y(active, since 63s), standbys: x 2024-09-10T18:42:04.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:04 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:42:04.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:04 smithi103 ceph-mon[33163]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:42:05.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:05 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/44467710' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb4c7e67-300c-4a1a-b9f5-bb028eeeabc2"}]: dispatch 2024-09-10T18:42:05.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:05 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/44467710' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb4c7e67-300c-4a1a-b9f5-bb028eeeabc2"}]': finished 2024-09-10T18:42:05.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:05 smithi018 ceph-mon[29336]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T18:42:05.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:05 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:42:05.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:05 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/44467710' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb4c7e67-300c-4a1a-b9f5-bb028eeeabc2"}]: dispatch 2024-09-10T18:42:05.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:05 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/44467710' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb4c7e67-300c-4a1a-b9f5-bb028eeeabc2"}]': finished 2024-09-10T18:42:05.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:05 smithi018 ceph-mon[40196]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T18:42:05.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:05 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:42:05.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:05 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/44467710' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb4c7e67-300c-4a1a-b9f5-bb028eeeabc2"}]: dispatch 2024-09-10T18:42:05.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:05 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/44467710' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb4c7e67-300c-4a1a-b9f5-bb028eeeabc2"}]': finished 2024-09-10T18:42:05.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:05 smithi103 ceph-mon[33163]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T18:42:05.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:05 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:42:06.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:06 smithi018 ceph-mon[29336]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:42:06.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:06 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/4204806423' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:42:06.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:06 smithi018 ceph-mon[40196]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:42:06.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:06 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/4204806423' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:42:06.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:06 smithi103 ceph-mon[33163]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:42:06.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:06 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/4204806423' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:42:08.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:08 smithi018 ceph-mon[29336]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:42:08.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:08 smithi018 ceph-mon[40196]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:42:09.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:08 smithi103 ceph-mon[33163]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:42:11.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:10 smithi103 ceph-mon[33163]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:42:11.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:10 smithi018 ceph-mon[29336]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:42:11.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:10 smithi018 ceph-mon[40196]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:42:11.998 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:11 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T18:42:11.999 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:11 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:11.999 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:11 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T18:42:11.999 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:11 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:12.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:11 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T18:42:12.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:11 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:12.941 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:12 smithi018 ceph-mon[29336]: Deploying daemon osd.0 on smithi018 2024-09-10T18:42:12.941 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:12 smithi018 ceph-mon[29336]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:42:12.941 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:12 smithi018 ceph-mon[40196]: Deploying daemon osd.0 on smithi018 2024-09-10T18:42:12.941 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:12 smithi018 ceph-mon[40196]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:42:13.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:12 smithi103 ceph-mon[33163]: Deploying daemon osd.0 on smithi018 2024-09-10T18:42:13.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:12 smithi103 ceph-mon[33163]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:42:14.990 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:14 smithi018 ceph-mon[29336]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:42:14.990 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:14 smithi018 ceph-mon[40196]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:42:15.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:14 smithi103 ceph-mon[33163]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:42:16.817 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:16 smithi018 ceph-mon[29336]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:42:16.817 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:16 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:42:16.817 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:16 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:16.817 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:16 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:16.817 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:16 smithi018 ceph-mon[40196]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:42:16.817 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:16 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:42:16.817 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:16 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:16.817 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:16 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:17.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:16 smithi103 ceph-mon[33163]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:42:17.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:16 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:42:17.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:16 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:17.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:16 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:19.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:18 smithi103 ceph-mon[33163]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:42:19.035 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:18 smithi018 ceph-mon[29336]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:42:19.035 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:18 smithi018 ceph-mon[40196]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:42:19.324 INFO:teuthology.orchestra.run.smithi018.stdout:Created osd(s) 0 on host 'smithi018' 2024-09-10T18:42:19.914 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:19 smithi018 ceph-mon[40196]: from='osd.0 v2:172.21.15.18:6801/2131279292' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T18:42:19.914 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:19 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:19.914 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:19 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:19.914 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:19 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:19.914 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:19 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:19.915 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:19 smithi018 ceph-mon[29336]: from='osd.0 v2:172.21.15.18:6801/2131279292' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T18:42:19.915 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:19 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:19.915 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:19 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:19.915 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:19 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:19.915 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:19 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:19.939 DEBUG:teuthology.orchestra.run.smithi018:osd.0> sudo journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@osd.0.service 2024-09-10T18:42:19.941 INFO:tasks.cephadm:Deploying osd.1 on smithi018 with /dev/vg_nvme/lv_3... 2024-09-10T18:42:19.941 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-10T18:42:20.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:19 smithi103 ceph-mon[33163]: from='osd.0 v2:172.21.15.18:6801/2131279292' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T18:42:20.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:19 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:20.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:19 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:20.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:19 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:20.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:19 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:20.258 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:42:20.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:20 smithi018 ceph-mon[40196]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:42:20.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:20 smithi018 ceph-mon[40196]: from='osd.0 v2:172.21.15.18:6801/2131279292' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T18:42:20.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:20 smithi018 ceph-mon[40196]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T18:42:20.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:20 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:42:20.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:20 smithi018 ceph-mon[40196]: from='osd.0 v2:172.21.15.18:6801/2131279292' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-09-10T18:42:20.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:20 smithi018 ceph-mon[29336]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:42:20.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:20 smithi018 ceph-mon[29336]: from='osd.0 v2:172.21.15.18:6801/2131279292' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T18:42:20.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:20 smithi018 ceph-mon[29336]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T18:42:20.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:20 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:42:20.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:20 smithi018 ceph-mon[29336]: from='osd.0 v2:172.21.15.18:6801/2131279292' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-09-10T18:42:20.945 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:42:20 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0[44072]: 2024-09-10T18:42:20.752+0000 7f4d35928640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T18:42:20.946 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:42:20 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0[44072]: 2024-09-10T18:42:20.755+0000 7f4d3075f640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:42:21.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:20 smithi103 ceph-mon[33163]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:42:21.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:20 smithi103 ceph-mon[33163]: from='osd.0 v2:172.21.15.18:6801/2131279292' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T18:42:21.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:20 smithi103 ceph-mon[33163]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T18:42:21.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:20 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:42:21.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:20 smithi103 ceph-mon[33163]: from='osd.0 v2:172.21.15.18:6801/2131279292' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-09-10T18:42:21.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:21 smithi018 ceph-mon[40196]: from='osd.0 v2:172.21.15.18:6801/2131279292' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-09-10T18:42:21.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:21 smithi018 ceph-mon[40196]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T18:42:21.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:21 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:42:21.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:21 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:42:21.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:21 smithi018 ceph-mon[29336]: from='osd.0 v2:172.21.15.18:6801/2131279292' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-09-10T18:42:21.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:21 smithi018 ceph-mon[29336]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T18:42:21.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:21 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:42:21.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:21 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:42:22.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:21 smithi103 ceph-mon[33163]: from='osd.0 v2:172.21.15.18:6801/2131279292' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-09-10T18:42:22.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:21 smithi103 ceph-mon[33163]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T18:42:22.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:21 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:42:22.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:21 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:42:22.876 INFO:teuthology.orchestra.run.smithi018.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-10T18:42:22.980 INFO:teuthology.orchestra.run.smithi018.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-10T18:42:22.991 INFO:teuthology.orchestra.run.smithi018.stderr: stderr: 10+0 records in 2024-09-10T18:42:22.991 INFO:teuthology.orchestra.run.smithi018.stderr:10+0 records out 2024-09-10T18:42:22.991 INFO:teuthology.orchestra.run.smithi018.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00949366 s, 1.1 GB/s 2024-09-10T18:42:22.991 INFO:teuthology.orchestra.run.smithi018.stderr:--> Zapping successful for: 2024-09-10T18:42:23.008 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:22 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:42:23.009 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:22 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:42:23.009 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:22 smithi018 ceph-mon[40196]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:42:23.009 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:22 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:42:23.009 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:22 smithi018 ceph-mon[40196]: osd.0 v2:172.21.15.18:6801/2131279292 boot 2024-09-10T18:42:23.009 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:22 smithi018 ceph-mon[40196]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T18:42:23.009 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:22 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:42:23.009 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:22 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:42:23.009 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:22 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:42:23.009 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:22 smithi018 ceph-mon[29336]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:42:23.010 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:22 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:42:23.010 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:22 smithi018 ceph-mon[29336]: osd.0 v2:172.21.15.18:6801/2131279292 boot 2024-09-10T18:42:23.010 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:22 smithi018 ceph-mon[29336]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T18:42:23.010 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:22 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:42:23.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:22 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:42:23.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:22 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:42:23.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:22 smithi103 ceph-mon[33163]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:42:23.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:22 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:42:23.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:22 smithi103 ceph-mon[33163]: osd.0 v2:172.21.15.18:6801/2131279292 boot 2024-09-10T18:42:23.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:22 smithi103 ceph-mon[33163]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T18:42:23.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:22 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:42:23.903 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:23 smithi018 ceph-mon[29336]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T18:42:23.904 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:23 smithi018 ceph-mon[40196]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T18:42:23.905 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi018:vg_nvme/lv_3 2024-09-10T18:42:24.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:23 smithi103 ceph-mon[33163]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T18:42:24.599 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:42:25.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:24 smithi103 ceph-mon[33163]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:42:25.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:24 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:25.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:24 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:25.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:24 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:42:25.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:24 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:25.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:24 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:25.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:24 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:42:25.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:24 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:25.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:24 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:42:25.045 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:24 smithi018 ceph-mon[29336]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:42:25.045 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:24 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:25.045 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:24 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:25.045 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:24 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:42:25.045 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:24 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:25.045 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:24 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:25.045 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:24 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:42:25.045 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:24 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:25.045 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:24 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:42:25.046 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:24 smithi018 ceph-mon[40196]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:42:25.046 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:24 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:25.046 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:24 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:25.046 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:24 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:42:25.046 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:24 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:25.046 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:24 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:25.046 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:24 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:42:25.046 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:24 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:25.046 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:24 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:42:25.902 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:25 smithi018 ceph-mon[29336]: Detected new or changed devices on smithi018 2024-09-10T18:42:25.902 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:25 smithi018 ceph-mon[29336]: Adjusting osd_memory_target on smithi018 to 16068M 2024-09-10T18:42:25.902 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:25 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:25.903 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:25 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:42:25.903 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:25 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:25.903 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:25 smithi018 ceph-mon[40196]: Detected new or changed devices on smithi018 2024-09-10T18:42:25.903 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:25 smithi018 ceph-mon[40196]: Adjusting osd_memory_target on smithi018 to 16068M 2024-09-10T18:42:25.903 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:25 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:25.903 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:25 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:42:25.903 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:25 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:26.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:25 smithi103 ceph-mon[33163]: Detected new or changed devices on smithi018 2024-09-10T18:42:26.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:25 smithi103 ceph-mon[33163]: Adjusting osd_memory_target on smithi018 to 16068M 2024-09-10T18:42:26.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:25 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:26.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:25 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:42:26.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:25 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:27.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:27 smithi018 ceph-mon[29336]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:42:27.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:27 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:42:27.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:27 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:42:27.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:27 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:27.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:27 smithi018 ceph-mon[40196]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:42:27.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:27 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:42:27.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:27 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:42:27.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:27 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:27.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:27 smithi103 ceph-mon[33163]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:42:27.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:27 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:42:27.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:27 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:42:27.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:27 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:28.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:28 smithi018 ceph-mon[29336]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:42:28.271 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:28 smithi018 ceph-mon[40196]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:42:28.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:28 smithi103 ceph-mon[33163]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:42:29.122 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:29 smithi018 ceph-mon[29336]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:42:29.123 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:29 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3306611763' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63bf6a98-e349-4cc0-9502-a31693070c5b"}]: dispatch 2024-09-10T18:42:29.123 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:29 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3306611763' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63bf6a98-e349-4cc0-9502-a31693070c5b"}]': finished 2024-09-10T18:42:29.123 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:29 smithi018 ceph-mon[29336]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T18:42:29.123 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:29 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:42:29.123 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:29 smithi018 ceph-mon[40196]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:42:29.124 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:29 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3306611763' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63bf6a98-e349-4cc0-9502-a31693070c5b"}]: dispatch 2024-09-10T18:42:29.124 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:29 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3306611763' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63bf6a98-e349-4cc0-9502-a31693070c5b"}]': finished 2024-09-10T18:42:29.124 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:29 smithi018 ceph-mon[40196]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T18:42:29.124 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:29 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:42:29.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:29 smithi103 ceph-mon[33163]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:42:29.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:29 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3306611763' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63bf6a98-e349-4cc0-9502-a31693070c5b"}]: dispatch 2024-09-10T18:42:29.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:29 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3306611763' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63bf6a98-e349-4cc0-9502-a31693070c5b"}]': finished 2024-09-10T18:42:29.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:29 smithi103 ceph-mon[33163]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T18:42:29.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:29 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:42:30.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:30 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2208247065' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:42:30.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:30 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2208247065' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:42:30.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:30 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2208247065' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:42:31.265 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:31 smithi018 ceph-mon[29336]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:42:31.266 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:31 smithi018 ceph-mon[40196]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:42:31.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:31 smithi103 ceph-mon[33163]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:42:33.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:33 smithi103 ceph-mon[33163]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:42:33.371 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:33 smithi018 ceph-mon[40196]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:42:33.371 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:33 smithi018 ceph-mon[29336]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:42:35.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:35 smithi103 ceph-mon[33163]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:42:35.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:35 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T18:42:35.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:35 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:35.347 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:35 smithi018 ceph-mon[29336]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:42:35.347 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:35 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T18:42:35.347 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:35 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:35.347 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:35 smithi018 ceph-mon[40196]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:42:35.347 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:35 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T18:42:35.347 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:35 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:36.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:36 smithi103 ceph-mon[33163]: Deploying daemon osd.1 on smithi018 2024-09-10T18:42:36.292 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:36 smithi018 ceph-mon[29336]: Deploying daemon osd.1 on smithi018 2024-09-10T18:42:36.292 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:36 smithi018 ceph-mon[40196]: Deploying daemon osd.1 on smithi018 2024-09-10T18:42:37.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:37 smithi103 ceph-mon[33163]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:42:37.288 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:37 smithi018 ceph-mon[29336]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:42:37.289 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:37 smithi018 ceph-mon[40196]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:42:39.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:39 smithi103 ceph-mon[33163]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:42:39.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:39 smithi018 ceph-mon[29336]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:42:39.274 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:39 smithi018 ceph-mon[40196]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:42:40.397 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:40 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:42:40.397 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:40 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:40.397 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:40 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:40.398 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:40 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:42:40.398 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:40 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:40.398 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:40 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:40.442 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:40 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:42:40.442 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:40 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:40.442 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:40 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:41.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:41 smithi018 ceph-mon[29336]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:42:41.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:41 smithi018 ceph-mon[40196]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:42:41.410 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:41 smithi103 ceph-mon[33163]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:42:42.558 INFO:teuthology.orchestra.run.smithi018.stdout:Created osd(s) 1 on host 'smithi018' 2024-09-10T18:42:43.124 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:43 smithi018 ceph-mon[29336]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:42:43.124 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:43 smithi018 ceph-mon[29336]: from='osd.1 v2:172.21.15.18:6805/3855640936' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T18:42:43.124 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:43 smithi018 ceph-mon[29336]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T18:42:43.124 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:43 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:43.124 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:43 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:43.125 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:43 smithi018 ceph-mon[40196]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:42:43.125 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:43 smithi018 ceph-mon[40196]: from='osd.1 v2:172.21.15.18:6805/3855640936' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T18:42:43.125 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:43 smithi018 ceph-mon[40196]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T18:42:43.125 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:43 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:43.125 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:43 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:43.126 DEBUG:teuthology.orchestra.run.smithi018:osd.1> sudo journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@osd.1.service 2024-09-10T18:42:43.128 INFO:tasks.cephadm:Deploying osd.2 on smithi018 with /dev/vg_nvme/lv_2... 2024-09-10T18:42:43.128 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-10T18:42:43.429 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:42:43.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:43 smithi103 ceph-mon[33163]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:42:43.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:43 smithi103 ceph-mon[33163]: from='osd.1 v2:172.21.15.18:6805/3855640936' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T18:42:43.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:43 smithi103 ceph-mon[33163]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T18:42:43.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:43 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:43.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:43 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:44.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:44 smithi018 ceph-mon[29336]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T18:42:44.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:44 smithi018 ceph-mon[29336]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T18:42:44.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:44 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:42:44.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:44 smithi018 ceph-mon[29336]: from='osd.1 v2:172.21.15.18:6805/3855640936' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-09-10T18:42:44.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:44 smithi018 ceph-mon[29336]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-09-10T18:42:44.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:44 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:44.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:44 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:44.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:44 smithi018 ceph-mon[40196]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T18:42:44.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:44 smithi018 ceph-mon[40196]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T18:42:44.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:44 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:42:44.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:44 smithi018 ceph-mon[40196]: from='osd.1 v2:172.21.15.18:6805/3855640936' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-09-10T18:42:44.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:44 smithi018 ceph-mon[40196]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-09-10T18:42:44.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:44 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:44.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:44 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:44.196 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 10 18:42:44 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-1[49824]: 2024-09-10T18:42:44.038+0000 7f97a6d44640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T18:42:44.196 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 10 18:42:44 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-1[49824]: 2024-09-10T18:42:44.041+0000 7f97a337e640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:42:44.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:44 smithi103 ceph-mon[33163]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T18:42:44.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:44 smithi103 ceph-mon[33163]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T18:42:44.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:44 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:42:44.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:44 smithi103 ceph-mon[33163]: from='osd.1 v2:172.21.15.18:6805/3855640936' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-09-10T18:42:44.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:44 smithi103 ceph-mon[33163]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-09-10T18:42:44.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:44 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:44.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:44 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:45.063 INFO:teuthology.orchestra.run.smithi018.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-10T18:42:45.063 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:45 smithi018 ceph-mon[29336]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:42:45.064 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:45 smithi018 ceph-mon[29336]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-09-10T18:42:45.064 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:45 smithi018 ceph-mon[29336]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T18:42:45.064 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:45 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:42:45.064 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:45 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:42:45.064 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:45 smithi018 ceph-mon[40196]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:42:45.064 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:45 smithi018 ceph-mon[40196]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-09-10T18:42:45.064 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:45 smithi018 ceph-mon[40196]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T18:42:45.064 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:45 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:42:45.064 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:45 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:42:45.158 INFO:teuthology.orchestra.run.smithi018.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-10T18:42:45.170 INFO:teuthology.orchestra.run.smithi018.stderr: stderr: 10+0 records in 2024-09-10T18:42:45.170 INFO:teuthology.orchestra.run.smithi018.stderr:10+0 records out 2024-09-10T18:42:45.170 INFO:teuthology.orchestra.run.smithi018.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0101663 s, 1.0 GB/s 2024-09-10T18:42:45.170 INFO:teuthology.orchestra.run.smithi018.stderr:--> Zapping successful for: 2024-09-10T18:42:45.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:45 smithi103 ceph-mon[33163]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:42:45.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:45 smithi103 ceph-mon[33163]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-09-10T18:42:45.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:45 smithi103 ceph-mon[33163]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T18:42:45.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:45 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:42:45.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:45 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:42:46.411 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:46 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:42:46.411 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:46 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:42:46.411 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:46 smithi018 ceph-mon[40196]: osd.1 v2:172.21.15.18:6805/3855640936 boot 2024-09-10T18:42:46.411 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:46 smithi018 ceph-mon[40196]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T18:42:46.411 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:46 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:42:46.411 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:46 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:42:46.411 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:46 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:42:46.412 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:46 smithi018 ceph-mon[29336]: osd.1 v2:172.21.15.18:6805/3855640936 boot 2024-09-10T18:42:46.412 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:46 smithi018 ceph-mon[29336]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T18:42:46.412 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:46 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:42:46.436 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi018:vg_nvme/lv_2 2024-09-10T18:42:46.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:46 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:42:46.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:46 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:42:46.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:46 smithi103 ceph-mon[33163]: osd.1 v2:172.21.15.18:6805/3855640936 boot 2024-09-10T18:42:46.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:46 smithi103 ceph-mon[33163]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T18:42:46.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:46 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:42:46.903 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:42:47.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:47 smithi018 ceph-mon[29336]: pgmap v56: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:42:47.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:47 smithi018 ceph-mon[29336]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T18:42:47.279 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:47 smithi018 ceph-mon[40196]: pgmap v56: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:42:47.279 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:47 smithi018 ceph-mon[40196]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T18:42:47.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:47 smithi103 ceph-mon[33163]: pgmap v56: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:42:47.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:47 smithi103 ceph-mon[33163]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T18:42:49.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:49 smithi018 ceph-mon[29336]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:42:49.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:49 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:49.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:49 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:49.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:49 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:42:49.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:49 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:42:49.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:49 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:49.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:49 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:49.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:49 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:42:49.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:49 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:49.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:49 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:42:49.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:49 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:49.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:49 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:42:49.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:49 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:49.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:49 smithi018 ceph-mon[40196]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:42:49.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:49 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:49.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:49 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:49.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:49 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:42:49.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:49 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:42:49.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:49 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:49.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:49 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:49.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:49 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:42:49.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:49 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:49.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:49 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:42:49.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:49 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:49.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:49 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:42:49.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:49 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:49.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:49 smithi103 ceph-mon[33163]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:42:49.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:49 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:49.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:49 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:49.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:49 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:42:49.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:49 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:42:49.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:49 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:49.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:49 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:49.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:49 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:42:49.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:49 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:49.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:49 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:42:49.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:49 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:49.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:49 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:42:49.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:49 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:42:50.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:50 smithi018 ceph-mon[29336]: Detected new or changed devices on smithi018 2024-09-10T18:42:50.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:50 smithi018 ceph-mon[29336]: Adjusting osd_memory_target on smithi018 to 8034M 2024-09-10T18:42:50.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:50 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:42:50.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:50 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:42:50.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:50 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:50.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:50 smithi018 ceph-mon[40196]: Detected new or changed devices on smithi018 2024-09-10T18:42:50.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:50 smithi018 ceph-mon[40196]: Adjusting osd_memory_target on smithi018 to 8034M 2024-09-10T18:42:50.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:50 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:42:50.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:50 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:42:50.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:50 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:50.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:50 smithi103 ceph-mon[33163]: Detected new or changed devices on smithi018 2024-09-10T18:42:50.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:50 smithi103 ceph-mon[33163]: Adjusting osd_memory_target on smithi018 to 8034M 2024-09-10T18:42:50.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:50 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:42:50.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:50 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:42:50.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:50 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:51.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:51 smithi103 ceph-mon[33163]: from='client.24170 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:42:51.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:51 smithi103 ceph-mon[33163]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:42:51.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:51 smithi018 ceph-mon[29336]: from='client.24170 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:42:51.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:51 smithi018 ceph-mon[29336]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:42:51.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:51 smithi018 ceph-mon[40196]: from='client.24170 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:42:51.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:51 smithi018 ceph-mon[40196]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:42:52.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:52 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1416249909' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79a01c29-4f4d-43d1-b1ea-a3fcf014f84b"}]: dispatch 2024-09-10T18:42:52.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:52 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1416249909' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79a01c29-4f4d-43d1-b1ea-a3fcf014f84b"}]': finished 2024-09-10T18:42:52.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:52 smithi018 ceph-mon[40196]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T18:42:52.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:52 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:42:52.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:52 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1534878859' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:42:52.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:52 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1416249909' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79a01c29-4f4d-43d1-b1ea-a3fcf014f84b"}]: dispatch 2024-09-10T18:42:52.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:52 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1416249909' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79a01c29-4f4d-43d1-b1ea-a3fcf014f84b"}]': finished 2024-09-10T18:42:52.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:52 smithi018 ceph-mon[29336]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T18:42:52.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:52 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:42:52.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:52 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1534878859' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:42:52.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:52 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1416249909' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79a01c29-4f4d-43d1-b1ea-a3fcf014f84b"}]: dispatch 2024-09-10T18:42:52.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:52 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1416249909' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79a01c29-4f4d-43d1-b1ea-a3fcf014f84b"}]': finished 2024-09-10T18:42:52.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:52 smithi103 ceph-mon[33163]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T18:42:52.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:52 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:42:52.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:52 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1534878859' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:42:53.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:53 smithi018 ceph-mon[40196]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:42:53.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:53 smithi018 ceph-mon[29336]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:42:53.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:53 smithi103 ceph-mon[33163]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:42:55.358 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:55 smithi018 ceph-mon[40196]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:42:55.359 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:55 smithi018 ceph-mon[29336]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:42:55.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:55 smithi103 ceph-mon[33163]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:42:57.185 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:57 smithi018 ceph-mon[29336]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:42:57.185 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:57 smithi018 ceph-mon[40196]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:42:57.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:57 smithi103 ceph-mon[33163]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:42:58.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:58 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T18:42:58.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:58 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:58.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:58 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T18:42:58.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:58 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:58.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:58 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T18:42:58.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:58 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:59.113 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:59 smithi018 ceph-mon[29336]: Deploying daemon osd.2 on smithi018 2024-09-10T18:42:59.113 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:42:59 smithi018 ceph-mon[29336]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:42:59.113 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:59 smithi018 ceph-mon[40196]: Deploying daemon osd.2 on smithi018 2024-09-10T18:42:59.113 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:42:59 smithi018 ceph-mon[40196]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:42:59.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:59 smithi103 ceph-mon[33163]: Deploying daemon osd.2 on smithi018 2024-09-10T18:42:59.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:42:59 smithi103 ceph-mon[33163]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:43:01.136 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:01 smithi018 ceph-mon[29336]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:43:01.137 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:01 smithi018 ceph-mon[40196]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:43:01.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:01 smithi103 ceph-mon[33163]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:43:02.327 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:02 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:43:02.328 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:02 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:02.328 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:02 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:02.328 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:02 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:43:02.328 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:02 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:02.328 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:02 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:02.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:02 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:43:02.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:02 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:02.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:02 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:03.403 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:03 smithi018 ceph-mon[29336]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:43:03.403 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:03 smithi018 ceph-mon[40196]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:43:03.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:03 smithi103 ceph-mon[33163]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:43:05.074 INFO:teuthology.orchestra.run.smithi018.stdout:Created osd(s) 2 on host 'smithi018' 2024-09-10T18:43:05.383 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:05 smithi018 ceph-mon[29336]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:43:05.383 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:05 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:05.383 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:05 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:05.383 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:05 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:05.383 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:05 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:43:05.383 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:05 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:05.383 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:05 smithi018 ceph-mon[29336]: from='osd.2 v2:172.21.15.18:6809/4107623855' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:43:05.384 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:05 smithi018 ceph-mon[29336]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:43:05.384 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:05 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:43:05.384 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:05 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:05.384 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:05 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:05.384 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:05 smithi018 ceph-mon[40196]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:43:05.384 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:05 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:05.384 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:05 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:05.384 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:05 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:05.384 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:05 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:43:05.384 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:05 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:05.384 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:05 smithi018 ceph-mon[40196]: from='osd.2 v2:172.21.15.18:6809/4107623855' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:43:05.385 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:05 smithi018 ceph-mon[40196]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:43:05.385 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:05 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:43:05.385 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:05 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:05.385 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:05 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:05.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:05 smithi103 ceph-mon[33163]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:43:05.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:05 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:05.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:05 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:05.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:05 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:05.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:05 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:43:05.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:05 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:05.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:05 smithi103 ceph-mon[33163]: from='osd.2 v2:172.21.15.18:6809/4107623855' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:43:05.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:05 smithi103 ceph-mon[33163]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:43:05.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:05 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:43:05.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:05 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:05.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:05 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:05.701 DEBUG:teuthology.orchestra.run.smithi018:osd.2> sudo journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@osd.2.service 2024-09-10T18:43:05.704 INFO:tasks.cephadm:Deploying osd.3 on smithi018 with /dev/vg_nvme/lv_1... 2024-09-10T18:43:05.705 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-10T18:43:06.416 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:43:06.689 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:06 smithi018 ceph-mon[29336]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T18:43:06.689 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:06 smithi018 ceph-mon[29336]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T18:43:06.689 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:06 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:43:06.689 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:06 smithi018 ceph-mon[29336]: from='osd.2 v2:172.21.15.18:6809/4107623855' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-09-10T18:43:06.689 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:06 smithi018 ceph-mon[29336]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-09-10T18:43:06.689 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:06 smithi018 ceph-mon[29336]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:43:06.690 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:06 smithi018 ceph-mon[40196]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T18:43:06.690 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:06 smithi018 ceph-mon[40196]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T18:43:06.690 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:06 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:43:06.690 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:06 smithi018 ceph-mon[40196]: from='osd.2 v2:172.21.15.18:6809/4107623855' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-09-10T18:43:06.690 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:06 smithi018 ceph-mon[40196]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-09-10T18:43:06.690 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:06 smithi018 ceph-mon[40196]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:43:06.690 INFO:journalctl@ceph.osd.2.smithi018.stdout:Sep 10 18:43:06 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-2[55543]: 2024-09-10T18:43:06.389+0000 7f2171ef8640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T18:43:06.691 INFO:journalctl@ceph.osd.2.smithi018.stdout:Sep 10 18:43:06 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-2[55543]: 2024-09-10T18:43:06.393+0000 7f216cd2f640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:43:06.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:06 smithi103 ceph-mon[33163]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T18:43:06.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:06 smithi103 ceph-mon[33163]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T18:43:06.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:06 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:43:06.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:06 smithi103 ceph-mon[33163]: from='osd.2 v2:172.21.15.18:6809/4107623855' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-09-10T18:43:06.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:06 smithi103 ceph-mon[33163]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-09-10T18:43:06.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:06 smithi103 ceph-mon[33163]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:43:07.578 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:07 smithi018 ceph-mon[40196]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-09-10T18:43:07.579 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:07 smithi018 ceph-mon[40196]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T18:43:07.579 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:07 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:43:07.579 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:07 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:43:07.579 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:07 smithi018 ceph-mon[29336]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-09-10T18:43:07.579 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:07 smithi018 ceph-mon[29336]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T18:43:07.579 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:07 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:43:07.579 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:07 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:43:07.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:07 smithi103 ceph-mon[33163]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-09-10T18:43:07.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:07 smithi103 ceph-mon[33163]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T18:43:07.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:07 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:43:07.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:07 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:43:08.409 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:08 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:43:08.409 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:08 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:43:08.409 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:08 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:43:08.409 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:08 smithi018 ceph-mon[29336]: osd.2 v2:172.21.15.18:6809/4107623855 boot 2024-09-10T18:43:08.409 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:08 smithi018 ceph-mon[29336]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T18:43:08.409 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:08 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:43:08.409 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:08 smithi018 ceph-mon[29336]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:43:08.409 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:08 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-10T18:43:08.410 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:08 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:43:08.410 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:08 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:43:08.410 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:08 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:43:08.410 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:08 smithi018 ceph-mon[40196]: osd.2 v2:172.21.15.18:6809/4107623855 boot 2024-09-10T18:43:08.410 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:08 smithi018 ceph-mon[40196]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T18:43:08.410 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:08 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:43:08.410 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:08 smithi018 ceph-mon[40196]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:43:08.410 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:08 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-10T18:43:08.685 INFO:teuthology.orchestra.run.smithi018.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-10T18:43:08.742 INFO:teuthology.orchestra.run.smithi018.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-10T18:43:08.754 INFO:teuthology.orchestra.run.smithi018.stderr: stderr: 10+0 records in 2024-09-10T18:43:08.754 INFO:teuthology.orchestra.run.smithi018.stderr:10+0 records out 2024-09-10T18:43:08.754 INFO:teuthology.orchestra.run.smithi018.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0100603 s, 1.0 GB/s 2024-09-10T18:43:08.754 INFO:teuthology.orchestra.run.smithi018.stderr:--> Zapping successful for: 2024-09-10T18:43:08.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:08 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:43:08.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:08 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:43:08.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:08 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:43:08.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:08 smithi103 ceph-mon[33163]: osd.2 v2:172.21.15.18:6809/4107623855 boot 2024-09-10T18:43:08.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:08 smithi103 ceph-mon[33163]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T18:43:08.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:08 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:43:08.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:08 smithi103 ceph-mon[33163]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:43:08.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:08 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-10T18:43:09.373 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi018:vg_nvme/lv_1 2024-09-10T18:43:09.612 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:09 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-10T18:43:09.612 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:09 smithi018 ceph-mon[29336]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T18:43:09.612 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:09 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:09.613 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:09 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-10T18:43:09.613 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:09 smithi018 ceph-mon[40196]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T18:43:09.613 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:09 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:09.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:09 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-10T18:43:09.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:09 smithi103 ceph-mon[33163]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T18:43:09.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:09 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:10.078 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:43:10.189 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:43:09 smithi018 sudo[59176]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:43:10.189 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:43:09 smithi018 sudo[59176]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:43:10.189 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:43:09 smithi018 sudo[59176]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:43:10.189 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:43:09 smithi018 sudo[59176]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:43:10.444 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:43:10 smithi018 sudo[59259]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:43:10.444 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:43:10 smithi018 sudo[59259]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:43:10.444 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:43:10 smithi018 sudo[59259]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:43:10.445 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:43:10 smithi018 sudo[59259]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:43:10.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:10 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:10.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:10 smithi018 ceph-mon[29336]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T18:43:10.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:10 smithi018 ceph-mon[29336]: pgmap v75: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:43:10.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:10 smithi018 ceph-mon[29336]: Detected new or changed devices on smithi018 2024-09-10T18:43:10.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:10 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:10.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:10 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:10.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:10 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:43:10.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:10 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:43:10.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:10 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:43:10.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:10 smithi018 ceph-mon[29336]: Adjusting osd_memory_target on smithi018 to 5356M 2024-09-10T18:43:10.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:10 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:10.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:10 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:10.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:10 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:43:10.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:10 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:10.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:10 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:10.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:10 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:10.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:10 smithi018 ceph-mon[40196]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T18:43:10.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:10 smithi018 ceph-mon[40196]: pgmap v75: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:43:10.447 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:10 smithi018 ceph-mon[40196]: Detected new or changed devices on smithi018 2024-09-10T18:43:10.447 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:10 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:10.447 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:10 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:10.447 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:10 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:43:10.447 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:10 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:43:10.447 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:10 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:43:10.447 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:10 smithi018 ceph-mon[40196]: Adjusting osd_memory_target on smithi018 to 5356M 2024-09-10T18:43:10.447 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:10 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:10.447 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:10 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:10.447 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:10 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:43:10.447 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:10 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:10.447 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:10 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:10.674 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:10 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:10.674 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:10 smithi103 ceph-mon[33163]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T18:43:10.674 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:10 smithi103 ceph-mon[33163]: pgmap v75: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:43:10.674 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:10 smithi103 ceph-mon[33163]: Detected new or changed devices on smithi018 2024-09-10T18:43:10.675 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:10 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:10.675 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:10 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:10.675 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:10 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:43:10.675 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:10 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:43:10.675 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:10 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:43:10.675 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:10 smithi103 ceph-mon[33163]: Adjusting osd_memory_target on smithi018 to 5356M 2024-09-10T18:43:10.675 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:10 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:10.675 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:10 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:10.675 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:10 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:43:10.675 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:10 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:10.675 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:10 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:11.194 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 10 18:43:10 smithi018 sudo[59275]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:43:11.194 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 10 18:43:10 smithi018 sudo[59275]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:43:11.194 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 10 18:43:10 smithi018 sudo[59275]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:43:11.194 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 10 18:43:10 smithi018 sudo[59275]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:43:11.493 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:11 smithi018 ceph-mon[29336]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-10T18:43:11.494 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:11 smithi018 ceph-mon[29336]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T18:43:11.494 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:11 smithi018 ceph-mon[40196]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-10T18:43:11.494 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:11 smithi018 ceph-mon[40196]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T18:43:11.494 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 10 18:43:11 smithi018 sudo[59285]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:43:11.494 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 10 18:43:11 smithi018 sudo[59285]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:43:11.495 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 10 18:43:11 smithi018 sudo[59285]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:43:11.495 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 10 18:43:11 smithi018 sudo[59285]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:43:11.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:11 smithi103 ceph-mon[33163]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-10T18:43:11.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:11 smithi103 ceph-mon[33163]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T18:43:11.944 INFO:journalctl@ceph.osd.2.smithi018.stdout:Sep 10 18:43:11 smithi018 sudo[59331]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:43:11.944 INFO:journalctl@ceph.osd.2.smithi018.stdout:Sep 10 18:43:11 smithi018 sudo[59331]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:43:11.944 INFO:journalctl@ceph.osd.2.smithi018.stdout:Sep 10 18:43:11 smithi018 sudo[59331]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:43:11.944 INFO:journalctl@ceph.osd.2.smithi018.stdout:Sep 10 18:43:11 smithi018 sudo[59331]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:43:12.444 INFO:journalctl@ceph.osd.2.smithi018.stdout:Sep 10 18:43:12 smithi018 sudo[59454]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:43:12.444 INFO:journalctl@ceph.osd.2.smithi018.stdout:Sep 10 18:43:12 smithi018 sudo[59454]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:43:12.444 INFO:journalctl@ceph.osd.2.smithi018.stdout:Sep 10 18:43:12 smithi018 sudo[59454]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:43:12.444 INFO:journalctl@ceph.osd.2.smithi018.stdout:Sep 10 18:43:12 smithi018 sudo[59454]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:43:12.849 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:12 smithi018 sudo[59474]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:43:12.849 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:12 smithi018 sudo[59474]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:43:12.849 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:12 smithi018 sudo[59474]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:43:12.849 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:12 smithi018 sudo[59474]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:43:13.107 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:13 smithi018 ceph-mon[40196]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:43:13.107 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:13 smithi018 ceph-mon[40196]: from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:43:13.107 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:13 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:43:13.107 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:13 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:43:13.107 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:13 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:13.108 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:13 smithi018 ceph-mon[40196]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:43:13.108 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:13 smithi018 sudo[59481]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:43:13.108 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:13 smithi018 sudo[59481]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:43:13.108 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:13 smithi018 sudo[59481]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:43:13.108 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:13 smithi018 sudo[59481]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:43:13.108 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:13 smithi018 ceph-mon[29336]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:43:13.108 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:13 smithi018 ceph-mon[29336]: from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:43:13.108 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:13 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:43:13.108 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:13 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:43:13.108 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:13 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:13.109 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:13 smithi018 ceph-mon[29336]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:43:13.577 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:13 smithi103 sudo[35654]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:43:13.577 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:13 smithi103 sudo[35654]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:43:13.577 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:13 smithi103 sudo[35654]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:43:13.577 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:13 smithi103 sudo[35654]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:43:13.578 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:13 smithi103 sudo[35657]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:43:14.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:13 smithi103 sudo[35657]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:43:14.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:13 smithi103 sudo[35657]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:43:14.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:13 smithi103 sudo[35657]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:43:14.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:13 smithi103 ceph-mon[33163]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:43:14.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:13 smithi103 ceph-mon[33163]: from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:43:14.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:13 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:43:14.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:13 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:43:14.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:13 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:14.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:13 smithi103 ceph-mon[33163]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:43:14.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:14 smithi018 sudo[59552]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:43:14.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:14 smithi018 sudo[59552]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:43:14.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:14 smithi018 sudo[59552]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:43:14.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:14 smithi018 sudo[59552]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:43:14.746 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:14 smithi018 ceph-mon[29336]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:43:14.747 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:14 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:43:14.747 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:14 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:43:14.747 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:14 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:43:14.747 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:14 smithi018 ceph-mon[29336]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:43:14.747 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:14 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:43:14.747 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:14 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:43:14.747 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:14 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:43:14.747 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:14 smithi018 ceph-mon[29336]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:43:14.747 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:14 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:14.747 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:14 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:43:14.747 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:14 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:43:14.747 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:14 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:43:14.748 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:14 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/893738908' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1809e677-b98c-4924-982e-9297e37563af"}]: dispatch 2024-09-10T18:43:14.748 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:14 smithi018 sudo[59556]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:43:14.748 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:14 smithi018 sudo[59556]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:43:14.748 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:14 smithi018 sudo[59556]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:43:14.748 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:14 smithi018 sudo[59556]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:43:14.748 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:14 smithi018 ceph-mon[40196]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:43:14.748 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:14 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:43:14.748 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:14 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:43:14.748 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:14 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:43:14.748 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:14 smithi018 ceph-mon[40196]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:43:14.748 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:14 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:43:14.748 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:14 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:43:14.748 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:14 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:43:14.749 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:14 smithi018 ceph-mon[40196]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:43:14.749 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:14 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:14.749 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:14 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:43:14.749 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:14 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:43:14.749 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:14 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:43:14.749 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:14 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/893738908' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1809e677-b98c-4924-982e-9297e37563af"}]: dispatch 2024-09-10T18:43:14.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:14 smithi103 ceph-mon[33163]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:43:14.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:14 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:43:14.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:14 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:43:14.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:14 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:43:14.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:14 smithi103 ceph-mon[33163]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:43:14.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:14 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:43:14.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:14 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:43:14.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:14 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:43:14.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:14 smithi103 ceph-mon[33163]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:43:14.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:14 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:14.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:14 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:43:14.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:14 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:43:14.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:14 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:43:14.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:14 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/893738908' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1809e677-b98c-4924-982e-9297e37563af"}]: dispatch 2024-09-10T18:43:15.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:15 smithi103 ceph-mon[33163]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:43:15.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:15 smithi103 ceph-mon[33163]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:43:15.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:15 smithi103 ceph-mon[33163]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:43:15.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:15 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/893738908' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1809e677-b98c-4924-982e-9297e37563af"}]': finished 2024-09-10T18:43:15.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:15 smithi103 ceph-mon[33163]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T18:43:15.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:15 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:43:15.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:15 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/890178912' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:43:15.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:15 smithi018 ceph-mon[29336]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:43:15.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:15 smithi018 ceph-mon[29336]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:43:15.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:15 smithi018 ceph-mon[29336]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:43:15.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:15 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/893738908' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1809e677-b98c-4924-982e-9297e37563af"}]': finished 2024-09-10T18:43:15.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:15 smithi018 ceph-mon[29336]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T18:43:15.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:15 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:43:15.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:15 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/890178912' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:43:15.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:15 smithi018 ceph-mon[40196]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:43:15.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:15 smithi018 ceph-mon[40196]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:43:15.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:15 smithi018 ceph-mon[40196]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:43:15.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:15 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/893738908' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1809e677-b98c-4924-982e-9297e37563af"}]': finished 2024-09-10T18:43:15.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:15 smithi018 ceph-mon[40196]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T18:43:15.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:15 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:43:15.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:15 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/890178912' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:43:16.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:16 smithi103 ceph-mon[33163]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:43:16.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:16 smithi018 ceph-mon[29336]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:43:16.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:16 smithi018 ceph-mon[40196]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:43:19.355 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:19 smithi018 ceph-mon[29336]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-10T18:43:19.356 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:19 smithi018 ceph-mon[40196]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-10T18:43:19.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:19 smithi103 ceph-mon[33163]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-10T18:43:21.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:21 smithi018 ceph-mon[29336]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-09-10T18:43:21.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:21 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T18:43:21.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:21 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:21.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:21 smithi018 ceph-mon[40196]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-09-10T18:43:21.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:21 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T18:43:21.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:21 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:21.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:21 smithi103 ceph-mon[33163]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-09-10T18:43:21.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:21 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T18:43:21.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:21 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:22.119 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:22 smithi018 ceph-mon[29336]: Deploying daemon osd.3 on smithi018 2024-09-10T18:43:22.120 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:22 smithi018 ceph-mon[40196]: Deploying daemon osd.3 on smithi018 2024-09-10T18:43:22.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:22 smithi103 ceph-mon[33163]: Deploying daemon osd.3 on smithi018 2024-09-10T18:43:23.400 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:23 smithi018 ceph-mon[29336]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-10T18:43:23.400 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:23 smithi018 ceph-mon[40196]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-10T18:43:23.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:23 smithi103 ceph-mon[33163]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-10T18:43:25.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:25 smithi018 ceph-mon[29336]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-10T18:43:25.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:25 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:43:25.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:25 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:25.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:25 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:25.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:25 smithi018 ceph-mon[40196]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-10T18:43:25.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:25 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:43:25.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:25 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:25.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:25 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:25.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:25 smithi103 ceph-mon[33163]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-10T18:43:25.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:25 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:43:25.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:25 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:25.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:25 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:27.332 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:27 smithi018 ceph-mon[29336]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.6 KiB/s rd, 70 KiB/s wr, 5 op/s 2024-09-10T18:43:27.332 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:27 smithi018 ceph-mon[40196]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.6 KiB/s rd, 70 KiB/s wr, 5 op/s 2024-09-10T18:43:27.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:27 smithi103 ceph-mon[33163]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.6 KiB/s rd, 70 KiB/s wr, 5 op/s 2024-09-10T18:43:28.223 INFO:teuthology.orchestra.run.smithi018.stdout:Created osd(s) 3 on host 'smithi018' 2024-09-10T18:43:28.898 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:28 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:28.898 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:28 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:28.898 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:28 smithi018 ceph-mon[29336]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-10T18:43:28.899 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:28 smithi018 ceph-mon[29336]: from='osd.3 v2:172.21.15.18:6813/2732204253' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T18:43:28.899 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:28 smithi018 ceph-mon[29336]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T18:43:28.899 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:28 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:28.899 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:28 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:28.899 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:28 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:28.899 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:28 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:28.899 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:28 smithi018 ceph-mon[40196]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-10T18:43:28.899 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:28 smithi018 ceph-mon[40196]: from='osd.3 v2:172.21.15.18:6813/2732204253' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T18:43:28.899 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:28 smithi018 ceph-mon[40196]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T18:43:28.899 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:28 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:28.900 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:28 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:28.929 DEBUG:teuthology.orchestra.run.smithi018:osd.3> sudo journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@osd.3.service 2024-09-10T18:43:28.931 INFO:tasks.cephadm:Deploying osd.4 on smithi103 with /dev/vg_nvme/lv_4... 2024-09-10T18:43:28.932 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-10T18:43:28.961 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:28 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:28.961 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:28 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:28.961 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:28 smithi103 ceph-mon[33163]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-10T18:43:28.961 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:28 smithi103 ceph-mon[33163]: from='osd.3 v2:172.21.15.18:6813/2732204253' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T18:43:28.961 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:28 smithi103 ceph-mon[33163]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T18:43:28.961 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:28 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:28.961 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:28 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:29.181 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:43:29.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:29 smithi103 ceph-mon[33163]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T18:43:29.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:29 smithi103 ceph-mon[33163]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T18:43:29.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:29 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:43:29.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:29 smithi103 ceph-mon[33163]: from='osd.3 v2:172.21.15.18:6813/2732204253' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-09-10T18:43:29.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:29 smithi103 ceph-mon[33163]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-09-10T18:43:29.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:29 smithi018 ceph-mon[29336]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T18:43:29.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:29 smithi018 ceph-mon[29336]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T18:43:29.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:29 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:43:29.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:29 smithi018 ceph-mon[29336]: from='osd.3 v2:172.21.15.18:6813/2732204253' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-09-10T18:43:29.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:29 smithi018 ceph-mon[29336]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-09-10T18:43:29.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:29 smithi018 ceph-mon[40196]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T18:43:29.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:29 smithi018 ceph-mon[40196]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T18:43:29.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:29 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:43:29.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:29 smithi018 ceph-mon[40196]: from='osd.3 v2:172.21.15.18:6813/2732204253' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-09-10T18:43:29.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:29 smithi018 ceph-mon[40196]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-09-10T18:43:29.946 INFO:journalctl@ceph.osd.3.smithi018.stdout:Sep 10 18:43:29 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-3[61244]: 2024-09-10T18:43:29.542+0000 7fc4c3082640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T18:43:29.946 INFO:journalctl@ceph.osd.3.smithi018.stdout:Sep 10 18:43:29 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-3[61244]: 2024-09-10T18:43:29.546+0000 7fc4bfebd640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:43:30.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:30 smithi103 ceph-mon[33163]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-09-10T18:43:30.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:30 smithi103 ceph-mon[33163]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T18:43:30.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:30 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:43:30.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:30 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:43:30.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:30 smithi103 ceph-mon[33163]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:43:30.890 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-10T18:43:30.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:30 smithi018 ceph-mon[40196]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-09-10T18:43:30.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:30 smithi018 ceph-mon[40196]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T18:43:30.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:30 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:43:30.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:30 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:43:30.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:30 smithi018 ceph-mon[40196]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:43:30.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:30 smithi018 ceph-mon[29336]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-09-10T18:43:30.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:30 smithi018 ceph-mon[29336]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T18:43:30.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:30 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:43:30.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:30 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:43:30.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:30 smithi018 ceph-mon[29336]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:43:30.997 INFO:teuthology.orchestra.run.smithi103.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-10T18:43:31.010 INFO:teuthology.orchestra.run.smithi103.stderr: stderr: 10+0 records in 2024-09-10T18:43:31.011 INFO:teuthology.orchestra.run.smithi103.stderr:10+0 records out 2024-09-10T18:43:31.011 INFO:teuthology.orchestra.run.smithi103.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114023 s, 920 MB/s 2024-09-10T18:43:31.011 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping successful for: 2024-09-10T18:43:31.639 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_4 2024-09-10T18:43:31.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:31 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:43:31.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:31 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:43:31.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:31 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:43:31.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:31 smithi018 ceph-mon[29336]: osd.3 v2:172.21.15.18:6813/2732204253 boot 2024-09-10T18:43:31.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:31 smithi018 ceph-mon[29336]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T18:43:31.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:31 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:43:31.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:31 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:43:31.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:31 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:43:31.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:31 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:43:31.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:31 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:43:31.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:31 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:43:31.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:31 smithi018 ceph-mon[40196]: osd.3 v2:172.21.15.18:6813/2732204253 boot 2024-09-10T18:43:31.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:31 smithi018 ceph-mon[40196]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T18:43:31.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:31 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:43:31.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:31 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:43:31.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:31 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:43:31.948 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:43:32.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:31 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:43:32.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:31 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:43:32.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:31 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:43:32.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:31 smithi103 ceph-mon[33163]: osd.3 v2:172.21.15.18:6813/2732204253 boot 2024-09-10T18:43:32.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:31 smithi103 ceph-mon[33163]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T18:43:32.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:31 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:43:32.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:31 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:43:32.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:31 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:43:32.728 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:32 smithi103 ceph-mon[33163]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T18:43:32.729 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:32 smithi103 ceph-mon[33163]: Detected new or changed devices on smithi018 2024-09-10T18:43:32.729 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:32 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:32.729 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:32 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:32.729 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:32 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:43:32.730 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:32 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:43:32.730 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:32 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:43:32.730 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:32 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:43:32.730 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:32 smithi103 ceph-mon[33163]: Adjusting osd_memory_target on smithi018 to 4017M 2024-09-10T18:43:32.730 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:32 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:32.730 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:32 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:32.731 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:32 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:43:32.731 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:32 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:32.731 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:32 smithi103 ceph-mon[33163]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:43:32.731 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:32 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:43:32.731 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:32 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:32.731 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:32 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:43:32.732 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:32 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:32.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:32 smithi018 ceph-mon[29336]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T18:43:32.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:32 smithi018 ceph-mon[29336]: Detected new or changed devices on smithi018 2024-09-10T18:43:32.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:32 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:32.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:32 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:32.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:32 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:43:32.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:32 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:43:32.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:32 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:43:32.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:32 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:43:32.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:32 smithi018 ceph-mon[29336]: Adjusting osd_memory_target on smithi018 to 4017M 2024-09-10T18:43:32.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:32 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:32.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:32 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:32.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:32 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:43:32.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:32 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:32.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:32 smithi018 ceph-mon[29336]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:43:32.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:32 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:43:32.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:32 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:32.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:32 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:43:32.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:32 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:32.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:32 smithi018 ceph-mon[40196]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T18:43:32.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:32 smithi018 ceph-mon[40196]: Detected new or changed devices on smithi018 2024-09-10T18:43:32.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:32 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:32.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:32 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:32.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:32 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:43:32.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:32 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:43:32.947 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:32 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:43:32.947 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:32 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:43:32.947 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:32 smithi018 ceph-mon[40196]: Adjusting osd_memory_target on smithi018 to 4017M 2024-09-10T18:43:32.947 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:32 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:32.947 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:32 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:32.947 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:32 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:43:32.947 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:32 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:32.947 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:32 smithi018 ceph-mon[40196]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:43:32.947 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:32 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:43:32.947 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:32 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:32.947 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:32 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:43:32.947 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:32 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:34.368 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:34 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:43:34.368 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:34 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:43:34.368 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:34 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:34.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:34 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:43:34.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:34 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:43:34.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:34 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:34.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:34 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:43:34.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:34 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:43:34.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:34 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:35.374 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:35 smithi103 ceph-mon[33163]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:43:35.374 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:35 smithi103 ceph-mon[33163]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:43:35.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:35 smithi018 ceph-mon[29336]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:43:35.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:35 smithi018 ceph-mon[29336]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:43:35.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:35 smithi018 ceph-mon[40196]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:43:35.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:35 smithi018 ceph-mon[40196]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:43:36.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:36 smithi018 ceph-mon[29336]: from='client.? 172.21.15.103:0/2077966019' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b682a300-bb53-4f36-8a9f-0753164ec4b7"}]: dispatch 2024-09-10T18:43:36.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:36 smithi018 ceph-mon[29336]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b682a300-bb53-4f36-8a9f-0753164ec4b7"}]: dispatch 2024-09-10T18:43:36.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:36 smithi018 ceph-mon[29336]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b682a300-bb53-4f36-8a9f-0753164ec4b7"}]': finished 2024-09-10T18:43:36.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:36 smithi018 ceph-mon[29336]: osdmap e27: 5 total, 4 up, 5 in 2024-09-10T18:43:36.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:36 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:43:36.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:36 smithi018 ceph-mon[40196]: from='client.? 172.21.15.103:0/2077966019' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b682a300-bb53-4f36-8a9f-0753164ec4b7"}]: dispatch 2024-09-10T18:43:36.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:36 smithi018 ceph-mon[40196]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b682a300-bb53-4f36-8a9f-0753164ec4b7"}]: dispatch 2024-09-10T18:43:36.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:36 smithi018 ceph-mon[40196]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b682a300-bb53-4f36-8a9f-0753164ec4b7"}]': finished 2024-09-10T18:43:36.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:36 smithi018 ceph-mon[40196]: osdmap e27: 5 total, 4 up, 5 in 2024-09-10T18:43:36.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:36 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:43:36.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:36 smithi103 ceph-mon[33163]: from='client.? 172.21.15.103:0/2077966019' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b682a300-bb53-4f36-8a9f-0753164ec4b7"}]: dispatch 2024-09-10T18:43:36.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:36 smithi103 ceph-mon[33163]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b682a300-bb53-4f36-8a9f-0753164ec4b7"}]: dispatch 2024-09-10T18:43:36.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:36 smithi103 ceph-mon[33163]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b682a300-bb53-4f36-8a9f-0753164ec4b7"}]': finished 2024-09-10T18:43:36.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:36 smithi103 ceph-mon[33163]: osdmap e27: 5 total, 4 up, 5 in 2024-09-10T18:43:36.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:36 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:43:37.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:37 smithi018 ceph-mon[29336]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:43:37.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:37 smithi018 ceph-mon[29336]: from='client.? 172.21.15.103:0/1482054858' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:43:37.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:37 smithi018 ceph-mon[40196]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:43:37.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:37 smithi018 ceph-mon[40196]: from='client.? 172.21.15.103:0/1482054858' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:43:37.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:37 smithi103 ceph-mon[33163]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:43:37.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:37 smithi103 ceph-mon[33163]: from='client.? 172.21.15.103:0/1482054858' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:43:39.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:39 smithi103 ceph-mon[33163]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:43:39.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:39 smithi018 ceph-mon[29336]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:43:39.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:39 smithi018 ceph-mon[40196]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:43:41.158 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:41 smithi103 ceph-mon[33163]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:43:41.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:41 smithi018 ceph-mon[29336]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:43:41.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:41 smithi018 ceph-mon[40196]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:43:42.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:42 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T18:43:42.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:42 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:42.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:42 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T18:43:42.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:42 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:42.490 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:42 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T18:43:42.490 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:42 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:43.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:43 smithi018 ceph-mon[29336]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:43:43.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:43 smithi018 ceph-mon[29336]: Deploying daemon osd.4 on smithi103 2024-09-10T18:43:43.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:43 smithi018 ceph-mon[40196]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:43:43.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:43 smithi018 ceph-mon[40196]: Deploying daemon osd.4 on smithi103 2024-09-10T18:43:43.460 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:43 smithi103 ceph-mon[33163]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:43:43.460 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:43 smithi103 ceph-mon[33163]: Deploying daemon osd.4 on smithi103 2024-09-10T18:43:45.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:45 smithi103 ceph-mon[33163]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:43:45.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:45 smithi018 ceph-mon[29336]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:43:45.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:45 smithi018 ceph-mon[40196]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:43:47.415 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:47 smithi103 ceph-mon[33163]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:43:47.415 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:47 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:43:47.415 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:47 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:47.415 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:47 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:47.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:47 smithi018 ceph-mon[29336]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:43:47.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:47 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:43:47.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:47 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:47.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:47 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:47.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:47 smithi018 ceph-mon[40196]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:43:47.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:47 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:43:47.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:47 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:47.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:47 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:49.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:49 smithi103 ceph-mon[33163]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:43:49.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:49 smithi018 ceph-mon[29336]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:43:49.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:49 smithi018 ceph-mon[40196]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:43:50.356 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 4 on host 'smithi103' 2024-09-10T18:43:50.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:50 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:50.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:50 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:50.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:50 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:50.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:50 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:43:50.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:50 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:50.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:50 smithi018 ceph-mon[40196]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:43:50.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:50 smithi018 ceph-mon[40196]: from='osd.4 v2:172.21.15.103:6800/465860081' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:43:50.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:50 smithi018 ceph-mon[40196]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:43:50.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:50 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:43:50.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:50 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:50.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:50 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:50.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:50 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:50.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:50 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:50.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:50 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:50.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:50 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:43:50.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:50 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:50.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:50 smithi018 ceph-mon[29336]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:43:50.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:50 smithi018 ceph-mon[29336]: from='osd.4 v2:172.21.15.103:6800/465860081' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:43:50.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:50 smithi018 ceph-mon[29336]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:43:50.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:50 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:43:50.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:50 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:50.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:50 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:50.958 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:50 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:50.958 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:50 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:50.958 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:50 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:50.958 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:50 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:43:50.958 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:50 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:50.958 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:50 smithi103 ceph-mon[33163]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:43:50.958 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:50 smithi103 ceph-mon[33163]: from='osd.4 v2:172.21.15.103:6800/465860081' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:43:50.959 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:50 smithi103 ceph-mon[33163]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:43:50.959 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:50 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:43:50.959 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:50 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:50.959 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:50 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:50.982 DEBUG:teuthology.orchestra.run.smithi103:osd.4> sudo journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@osd.4.service 2024-09-10T18:43:50.984 INFO:tasks.cephadm:Deploying osd.5 on smithi103 with /dev/vg_nvme/lv_3... 2024-09-10T18:43:50.985 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-10T18:43:51.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:51 smithi018 ceph-mon[29336]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T18:43:51.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:51 smithi018 ceph-mon[29336]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T18:43:51.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:51 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:43:51.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:51 smithi018 ceph-mon[29336]: from='osd.4 v2:172.21.15.103:6800/465860081' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T18:43:51.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:51 smithi018 ceph-mon[29336]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T18:43:51.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:51 smithi018 ceph-mon[40196]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T18:43:51.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:51 smithi018 ceph-mon[40196]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T18:43:51.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:51 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:43:51.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:51 smithi018 ceph-mon[40196]: from='osd.4 v2:172.21.15.103:6800/465860081' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T18:43:51.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:51 smithi018 ceph-mon[40196]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T18:43:52.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:51 smithi103 ceph-mon[33163]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T18:43:52.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:51 smithi103 ceph-mon[33163]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T18:43:52.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:51 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:43:52.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:51 smithi103 ceph-mon[33163]: from='osd.4 v2:172.21.15.103:6800/465860081' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T18:43:52.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:51 smithi103 ceph-mon[33163]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T18:43:52.023 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 18:43:51 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-4[38039]: 2024-09-10T18:43:51.613+0000 7fbb176d2640 -1 osd.4 0 waiting for initial osdmap 2024-09-10T18:43:52.023 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 18:43:51 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-4[38039]: 2024-09-10T18:43:51.617+0000 7fbb10ce4640 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:43:52.192 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:43:52.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:52 smithi103 ceph-mon[33163]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-10T18:43:52.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:52 smithi103 ceph-mon[33163]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T18:43:52.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:52 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:43:52.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:52 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:43:52.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:52 smithi103 ceph-mon[33163]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:43:52.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:52 smithi018 ceph-mon[29336]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-10T18:43:52.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:52 smithi018 ceph-mon[29336]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T18:43:52.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:52 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:43:52.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:52 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:43:52.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:52 smithi018 ceph-mon[29336]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:43:52.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:52 smithi018 ceph-mon[40196]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-10T18:43:52.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:52 smithi018 ceph-mon[40196]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T18:43:52.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:52 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:43:52.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:52 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:43:52.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:52 smithi018 ceph-mon[40196]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:43:53.726 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:53 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:43:53.726 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:53 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:43:53.726 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:53 smithi103 ceph-mon[33163]: osd.4 v2:172.21.15.103:6800/465860081 boot 2024-09-10T18:43:53.726 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:53 smithi103 ceph-mon[33163]: osdmap e30: 5 total, 5 up, 5 in 2024-09-10T18:43:53.726 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:53 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:43:53.726 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:53 smithi103 ceph-mon[33163]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T18:43:53.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:53 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:43:53.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:53 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:43:53.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:53 smithi018 ceph-mon[29336]: osd.4 v2:172.21.15.103:6800/465860081 boot 2024-09-10T18:43:53.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:53 smithi018 ceph-mon[29336]: osdmap e30: 5 total, 5 up, 5 in 2024-09-10T18:43:53.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:53 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:43:53.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:53 smithi018 ceph-mon[29336]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T18:43:53.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:53 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:43:53.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:53 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:43:53.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:53 smithi018 ceph-mon[40196]: osd.4 v2:172.21.15.103:6800/465860081 boot 2024-09-10T18:43:53.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:53 smithi018 ceph-mon[40196]: osdmap e30: 5 total, 5 up, 5 in 2024-09-10T18:43:53.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:53 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:43:53.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:53 smithi018 ceph-mon[40196]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T18:43:54.842 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-10T18:43:54.842 INFO:teuthology.orchestra.run.smithi103.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-10T18:43:54.842 INFO:teuthology.orchestra.run.smithi103.stderr: stderr: 10+0 records in 2024-09-10T18:43:54.842 INFO:teuthology.orchestra.run.smithi103.stderr:10+0 records out 2024-09-10T18:43:54.842 INFO:teuthology.orchestra.run.smithi103.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124431 s, 843 MB/s 2024-09-10T18:43:54.843 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping successful for: 2024-09-10T18:43:55.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:55 smithi103 ceph-mon[33163]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:43:55.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:55 smithi103 ceph-mon[33163]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T18:43:55.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:55 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:55.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:55 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:55.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:55 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:43:55.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:55 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:55.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:55 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:55.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:55 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:43:55.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:55 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:55.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:55 smithi018 ceph-mon[40196]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:43:55.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:55 smithi018 ceph-mon[40196]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T18:43:55.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:55 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:55.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:55 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:55.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:55 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:43:55.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:55 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:55.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:55 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:55.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:55 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:43:55.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:55 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:55.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:55 smithi018 ceph-mon[29336]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:43:55.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:55 smithi018 ceph-mon[29336]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T18:43:55.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:55 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:55.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:55 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:55.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:55 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:43:55.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:55 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:55.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:55 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:55.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:55 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:43:55.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:55 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:43:55.452 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_3 2024-09-10T18:43:55.691 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:43:56.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:56 smithi018 ceph-mon[29336]: Detected new or changed devices on smithi103 2024-09-10T18:43:56.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:56 smithi018 ceph-mon[29336]: Adjusting osd_memory_target on smithi103 to 17088M 2024-09-10T18:43:56.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:56 smithi018 ceph-mon[40196]: Detected new or changed devices on smithi103 2024-09-10T18:43:56.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:56 smithi018 ceph-mon[40196]: Adjusting osd_memory_target on smithi103 to 17088M 2024-09-10T18:43:56.460 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:56 smithi103 ceph-mon[33163]: Detected new or changed devices on smithi103 2024-09-10T18:43:56.460 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:56 smithi103 ceph-mon[33163]: Adjusting osd_memory_target on smithi103 to 17088M 2024-09-10T18:43:57.360 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:57 smithi018 ceph-mon[29336]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:43:57.361 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:57 smithi018 ceph-mon[40196]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:43:57.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:57 smithi103 ceph-mon[33163]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:43:58.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:58 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:43:58.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:58 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:43:58.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:58 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:58.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:58 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:43:58.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:58 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:43:58.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:58 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:58.479 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:58 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:43:58.480 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:58 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:43:58.480 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:58 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:59.398 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:59 smithi103 ceph-mon[33163]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:43:59.398 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:43:59 smithi103 ceph-mon[33163]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T18:43:59.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:59 smithi018 ceph-mon[29336]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:43:59.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:43:59 smithi018 ceph-mon[29336]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T18:43:59.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:59 smithi018 ceph-mon[40196]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:43:59.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:43:59 smithi018 ceph-mon[40196]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T18:44:00.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:00 smithi103 ceph-mon[33163]: from='client.? 172.21.15.103:0/2076116832' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2bbfdb6c-0709-4fe4-8a4a-006f576f18e6"}]: dispatch 2024-09-10T18:44:00.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:00 smithi103 ceph-mon[33163]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2bbfdb6c-0709-4fe4-8a4a-006f576f18e6"}]: dispatch 2024-09-10T18:44:00.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:00 smithi103 ceph-mon[33163]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2bbfdb6c-0709-4fe4-8a4a-006f576f18e6"}]': finished 2024-09-10T18:44:00.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:00 smithi103 ceph-mon[33163]: osdmap e33: 6 total, 5 up, 6 in 2024-09-10T18:44:00.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:00 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:44:00.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:00 smithi018 ceph-mon[29336]: from='client.? 172.21.15.103:0/2076116832' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2bbfdb6c-0709-4fe4-8a4a-006f576f18e6"}]: dispatch 2024-09-10T18:44:00.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:00 smithi018 ceph-mon[29336]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2bbfdb6c-0709-4fe4-8a4a-006f576f18e6"}]: dispatch 2024-09-10T18:44:00.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:00 smithi018 ceph-mon[29336]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2bbfdb6c-0709-4fe4-8a4a-006f576f18e6"}]': finished 2024-09-10T18:44:00.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:00 smithi018 ceph-mon[29336]: osdmap e33: 6 total, 5 up, 6 in 2024-09-10T18:44:00.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:00 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:44:00.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:00 smithi018 ceph-mon[40196]: from='client.? 172.21.15.103:0/2076116832' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2bbfdb6c-0709-4fe4-8a4a-006f576f18e6"}]: dispatch 2024-09-10T18:44:00.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:00 smithi018 ceph-mon[40196]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2bbfdb6c-0709-4fe4-8a4a-006f576f18e6"}]: dispatch 2024-09-10T18:44:00.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:00 smithi018 ceph-mon[40196]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2bbfdb6c-0709-4fe4-8a4a-006f576f18e6"}]': finished 2024-09-10T18:44:00.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:00 smithi018 ceph-mon[40196]: osdmap e33: 6 total, 5 up, 6 in 2024-09-10T18:44:00.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:00 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:44:01.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:01 smithi018 ceph-mon[29336]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-10T18:44:01.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:01 smithi018 ceph-mon[29336]: from='client.? 172.21.15.103:0/49423882' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:44:01.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:01 smithi018 ceph-mon[40196]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-10T18:44:01.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:01 smithi018 ceph-mon[40196]: from='client.? 172.21.15.103:0/49423882' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:44:01.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:01 smithi103 ceph-mon[33163]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-10T18:44:01.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:01 smithi103 ceph-mon[33163]: from='client.? 172.21.15.103:0/49423882' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:44:03.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:03 smithi018 ceph-mon[29336]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T18:44:03.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:03 smithi018 ceph-mon[40196]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T18:44:03.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:03 smithi103 ceph-mon[33163]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T18:44:05.378 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:05 smithi103 ceph-mon[33163]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-10T18:44:05.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:05 smithi018 ceph-mon[29336]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-10T18:44:05.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:05 smithi018 ceph-mon[40196]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-10T18:44:06.376 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:06 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T18:44:06.376 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:06 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:44:06.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:06 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T18:44:06.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:06 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:44:06.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:06 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T18:44:06.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:06 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:44:07.411 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:07 smithi103 ceph-mon[33163]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T18:44:07.411 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:07 smithi103 ceph-mon[33163]: Deploying daemon osd.5 on smithi103 2024-09-10T18:44:07.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:07 smithi018 ceph-mon[29336]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T18:44:07.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:07 smithi018 ceph-mon[29336]: Deploying daemon osd.5 on smithi103 2024-09-10T18:44:07.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:07 smithi018 ceph-mon[40196]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T18:44:07.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:07 smithi018 ceph-mon[40196]: Deploying daemon osd.5 on smithi103 2024-09-10T18:44:09.229 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:09 smithi103 ceph-mon[33163]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:44:09.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:09 smithi018 ceph-mon[29336]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:44:09.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:09 smithi018 ceph-mon[40196]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:44:11.433 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:11 smithi103 ceph-mon[33163]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:44:11.433 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:11 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:44:11.433 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:11 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:11.433 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:11 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:11.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:11 smithi018 ceph-mon[29336]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:44:11.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:11 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:44:11.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:11 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:11.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:11 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:11.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:11 smithi018 ceph-mon[40196]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:44:11.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:11 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:44:11.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:11 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:11.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:11 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:13.218 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:13 smithi103 ceph-mon[33163]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:44:13.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:13 smithi018 ceph-mon[29336]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:44:13.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:13 smithi018 ceph-mon[40196]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:44:14.000 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 5 on host 'smithi103' 2024-09-10T18:44:14.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:14 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:44:14.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:14 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:44:14.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:14 smithi103 ceph-mon[33163]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T18:44:14.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:14 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:44:14.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:14 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:14.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:14 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:14.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:14 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:44:14.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:14 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:44:14.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:14 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:14.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:14 smithi103 ceph-mon[33163]: from='osd.5 v2:172.21.15.103:6804/3498719258' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:44:14.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:14 smithi103 ceph-mon[33163]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:44:14.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:14 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:44:14.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:14 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:14.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:14 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:14.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:14 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:44:14.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:14 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:44:14.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:14 smithi018 ceph-mon[29336]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T18:44:14.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:14 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:44:14.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:14 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:14.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:14 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:14.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:14 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:44:14.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:14 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:44:14.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:14 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:14.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:14 smithi018 ceph-mon[29336]: from='osd.5 v2:172.21.15.103:6804/3498719258' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:44:14.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:14 smithi018 ceph-mon[29336]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:44:14.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:14 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:44:14.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:14 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:14.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:14 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:14.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:14 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:44:14.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:14 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:44:14.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:14 smithi018 ceph-mon[40196]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T18:44:14.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:14 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:44:14.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:14 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:14.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:14 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:14.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:14 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:44:14.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:14 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:44:14.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:14 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:14.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:14 smithi018 ceph-mon[40196]: from='osd.5 v2:172.21.15.103:6804/3498719258' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:44:14.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:14 smithi018 ceph-mon[40196]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:44:14.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:14 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:44:14.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:14 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:14.447 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:14 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:14.699 DEBUG:teuthology.orchestra.run.smithi103:osd.5> sudo journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@osd.5.service 2024-09-10T18:44:14.702 INFO:tasks.cephadm:Deploying osd.6 on smithi103 with /dev/vg_nvme/lv_2... 2024-09-10T18:44:14.703 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-10T18:44:15.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:15 smithi103 ceph-mon[33163]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:44:15.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:15 smithi103 ceph-mon[33163]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T18:44:15.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:15 smithi103 ceph-mon[33163]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T18:44:15.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:15 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:44:15.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:15 smithi103 ceph-mon[33163]: from='osd.5 v2:172.21.15.103:6804/3498719258' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T18:44:15.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:15 smithi103 ceph-mon[33163]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T18:44:15.412 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:44:15.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:15 smithi018 ceph-mon[29336]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:44:15.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:15 smithi018 ceph-mon[29336]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T18:44:15.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:15 smithi018 ceph-mon[29336]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T18:44:15.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:15 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:44:15.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:15 smithi018 ceph-mon[29336]: from='osd.5 v2:172.21.15.103:6804/3498719258' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T18:44:15.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:15 smithi018 ceph-mon[29336]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T18:44:15.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:15 smithi018 ceph-mon[40196]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:44:15.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:15 smithi018 ceph-mon[40196]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T18:44:15.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:15 smithi018 ceph-mon[40196]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T18:44:15.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:15 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:44:15.447 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:15 smithi018 ceph-mon[40196]: from='osd.5 v2:172.21.15.103:6804/3498719258' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T18:44:15.447 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:15 smithi018 ceph-mon[40196]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T18:44:15.773 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 18:44:15 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5[43919]: 2024-09-10T18:44:15.310+0000 7f5ffb149640 -1 osd.5 0 waiting for initial osdmap 2024-09-10T18:44:15.773 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 18:44:15 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5[43919]: 2024-09-10T18:44:15.316+0000 7f5ff6781640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:44:16.380 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:16 smithi103 ceph-mon[33163]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-10T18:44:16.381 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:16 smithi103 ceph-mon[33163]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T18:44:16.381 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:16 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:44:16.381 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:16 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:44:16.381 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:16 smithi103 ceph-mon[33163]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:44:16.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:16 smithi018 ceph-mon[29336]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-10T18:44:16.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:16 smithi018 ceph-mon[29336]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T18:44:16.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:16 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:44:16.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:16 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:44:16.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:16 smithi018 ceph-mon[29336]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:44:16.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:16 smithi018 ceph-mon[40196]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-10T18:44:16.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:16 smithi018 ceph-mon[40196]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T18:44:16.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:16 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:44:16.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:16 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:44:16.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:16 smithi018 ceph-mon[40196]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:44:17.521 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:17 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:44:17.521 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:17 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:44:17.521 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:17 smithi103 ceph-mon[33163]: osd.5 v2:172.21.15.103:6804/3498719258 boot 2024-09-10T18:44:17.521 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:17 smithi103 ceph-mon[33163]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T18:44:17.521 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:17 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:44:17.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:17 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:44:17.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:17 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:44:17.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:17 smithi018 ceph-mon[29336]: osd.5 v2:172.21.15.103:6804/3498719258 boot 2024-09-10T18:44:17.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:17 smithi018 ceph-mon[29336]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T18:44:17.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:17 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:44:17.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:17 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:44:17.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:17 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:44:17.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:17 smithi018 ceph-mon[40196]: osd.5 v2:172.21.15.103:6804/3498719258 boot 2024-09-10T18:44:17.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:17 smithi018 ceph-mon[40196]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T18:44:17.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:17 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:44:18.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:18 smithi103 ceph-mon[33163]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T18:44:18.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:18 smithi103 ceph-mon[33163]: Detected new or changed devices on smithi103 2024-09-10T18:44:18.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:18 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:18.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:18 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:18.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:18 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:44:18.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:18 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:44:18.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:18 smithi103 ceph-mon[33163]: Adjusting osd_memory_target on smithi103 to 8544M 2024-09-10T18:44:18.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:18 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:18.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:18 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:44:18.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:18 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:44:18.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:18 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:18.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:18 smithi103 ceph-mon[33163]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:44:18.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:18 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:44:18.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:18 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:44:18.577 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-10T18:44:18.661 INFO:teuthology.orchestra.run.smithi103.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-10T18:44:18.676 INFO:teuthology.orchestra.run.smithi103.stderr: stderr: 10+0 records in 2024-09-10T18:44:18.677 INFO:teuthology.orchestra.run.smithi103.stderr:10+0 records out 2024-09-10T18:44:18.677 INFO:teuthology.orchestra.run.smithi103.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0140488 s, 746 MB/s 2024-09-10T18:44:18.677 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping successful for: 2024-09-10T18:44:18.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:18 smithi018 ceph-mon[29336]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T18:44:18.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:18 smithi018 ceph-mon[29336]: Detected new or changed devices on smithi103 2024-09-10T18:44:18.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:18 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:18.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:18 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:18.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:18 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:44:18.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:18 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:44:18.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:18 smithi018 ceph-mon[29336]: Adjusting osd_memory_target on smithi103 to 8544M 2024-09-10T18:44:18.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:18 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:18.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:18 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:44:18.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:18 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:44:18.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:18 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:18.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:18 smithi018 ceph-mon[29336]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:44:18.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:18 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:44:18.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:18 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:44:18.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:18 smithi018 ceph-mon[40196]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T18:44:18.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:18 smithi018 ceph-mon[40196]: Detected new or changed devices on smithi103 2024-09-10T18:44:18.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:18 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:18.698 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:18 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:18.698 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:18 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:44:18.698 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:18 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:44:18.698 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:18 smithi018 ceph-mon[40196]: Adjusting osd_memory_target on smithi103 to 8544M 2024-09-10T18:44:18.698 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:18 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:18.698 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:18 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:44:18.699 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:18 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:44:18.699 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:18 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:18.699 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:18 smithi018 ceph-mon[40196]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:44:18.699 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:18 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:44:18.699 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:18 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:44:19.314 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_2 2024-09-10T18:44:19.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:19 smithi103 ceph-mon[33163]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T18:44:19.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:19 smithi103 ceph-mon[33163]: 1.0 deep-scrub starts 2024-09-10T18:44:19.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:19 smithi103 ceph-mon[33163]: 1.0 deep-scrub ok 2024-09-10T18:44:19.555 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:44:19.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:19 smithi018 ceph-mon[29336]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T18:44:19.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:19 smithi018 ceph-mon[29336]: 1.0 deep-scrub starts 2024-09-10T18:44:19.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:19 smithi018 ceph-mon[29336]: 1.0 deep-scrub ok 2024-09-10T18:44:19.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:19 smithi018 ceph-mon[40196]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T18:44:19.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:19 smithi018 ceph-mon[40196]: 1.0 deep-scrub starts 2024-09-10T18:44:19.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:19 smithi018 ceph-mon[40196]: 1.0 deep-scrub ok 2024-09-10T18:44:20.646 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:20 smithi103 ceph-mon[33163]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:44:20.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:20 smithi018 ceph-mon[29336]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:44:20.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:20 smithi018 ceph-mon[40196]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:44:21.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:21 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:44:21.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:21 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:44:21.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:21 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:44:21.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:21 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:44:21.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:21 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:44:21.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:21 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:44:21.722 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:21 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:44:21.722 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:21 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:44:21.722 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:21 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:44:22.622 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:22 smithi103 ceph-mon[33163]: from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:44:22.622 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:22 smithi103 ceph-mon[33163]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:44:22.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:22 smithi018 ceph-mon[29336]: from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:44:22.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:22 smithi018 ceph-mon[29336]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:44:22.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:22 smithi018 ceph-mon[40196]: from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:44:22.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:22 smithi018 ceph-mon[40196]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:44:24.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:24 smithi018 ceph-mon[40196]: from='client.? 172.21.15.103:0/1093064638' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31f9711e-0ba8-4aeb-b852-7d93e8fcce33"}]: dispatch 2024-09-10T18:44:24.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:24 smithi018 ceph-mon[40196]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31f9711e-0ba8-4aeb-b852-7d93e8fcce33"}]: dispatch 2024-09-10T18:44:24.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:24 smithi018 ceph-mon[40196]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "31f9711e-0ba8-4aeb-b852-7d93e8fcce33"}]': finished 2024-09-10T18:44:24.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:24 smithi018 ceph-mon[40196]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T18:44:24.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:24 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:44:24.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:24 smithi018 ceph-mon[29336]: from='client.? 172.21.15.103:0/1093064638' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31f9711e-0ba8-4aeb-b852-7d93e8fcce33"}]: dispatch 2024-09-10T18:44:24.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:24 smithi018 ceph-mon[29336]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31f9711e-0ba8-4aeb-b852-7d93e8fcce33"}]: dispatch 2024-09-10T18:44:24.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:24 smithi018 ceph-mon[29336]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "31f9711e-0ba8-4aeb-b852-7d93e8fcce33"}]': finished 2024-09-10T18:44:24.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:24 smithi018 ceph-mon[29336]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T18:44:24.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:24 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:44:24.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:24 smithi103 ceph-mon[33163]: from='client.? 172.21.15.103:0/1093064638' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31f9711e-0ba8-4aeb-b852-7d93e8fcce33"}]: dispatch 2024-09-10T18:44:24.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:24 smithi103 ceph-mon[33163]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31f9711e-0ba8-4aeb-b852-7d93e8fcce33"}]: dispatch 2024-09-10T18:44:24.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:24 smithi103 ceph-mon[33163]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "31f9711e-0ba8-4aeb-b852-7d93e8fcce33"}]': finished 2024-09-10T18:44:24.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:24 smithi103 ceph-mon[33163]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T18:44:24.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:24 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:44:25.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:25 smithi018 ceph-mon[40196]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:44:25.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:25 smithi018 ceph-mon[40196]: from='client.? 172.21.15.103:0/1319178873' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:44:25.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:25 smithi018 ceph-mon[29336]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:44:25.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:25 smithi018 ceph-mon[29336]: from='client.? 172.21.15.103:0/1319178873' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:44:25.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:25 smithi103 ceph-mon[33163]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:44:25.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:25 smithi103 ceph-mon[33163]: from='client.? 172.21.15.103:0/1319178873' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:44:27.355 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:27 smithi103 ceph-mon[33163]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:44:27.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:27 smithi018 ceph-mon[40196]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:44:27.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:27 smithi018 ceph-mon[29336]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:44:29.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:29 smithi103 ceph-mon[33163]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:44:29.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:29 smithi018 ceph-mon[40196]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:44:29.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:29 smithi018 ceph-mon[29336]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:44:30.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:30 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T18:44:30.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:30 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:44:30.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:30 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T18:44:30.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:30 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:44:30.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:30 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T18:44:30.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:30 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:44:31.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:31 smithi103 ceph-mon[33163]: Deploying daemon osd.6 on smithi103 2024-09-10T18:44:31.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:31 smithi103 ceph-mon[33163]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:44:31.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:31 smithi018 ceph-mon[29336]: Deploying daemon osd.6 on smithi103 2024-09-10T18:44:31.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:31 smithi018 ceph-mon[29336]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:44:31.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:31 smithi018 ceph-mon[40196]: Deploying daemon osd.6 on smithi103 2024-09-10T18:44:31.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:31 smithi018 ceph-mon[40196]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:44:33.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:33 smithi018 ceph-mon[29336]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:44:33.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:33 smithi018 ceph-mon[40196]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:44:33.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:33 smithi103 ceph-mon[33163]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:44:35.235 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:35 smithi103 ceph-mon[33163]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:44:35.235 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:35 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:44:35.235 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:35 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:35.235 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:35 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:35.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:35 smithi018 ceph-mon[29336]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:44:35.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:35 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:44:35.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:35 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:35.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:35 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:35.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:35 smithi018 ceph-mon[40196]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:44:35.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:35 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:44:35.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:35 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:35.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:35 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:37.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:37 smithi018 ceph-mon[40196]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:44:37.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:37 smithi018 ceph-mon[29336]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:44:37.485 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:37 smithi103 ceph-mon[33163]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:44:38.214 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 6 on host 'smithi103' 2024-09-10T18:44:38.756 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:38 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:38.757 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:38 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:38.757 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:38 smithi103 ceph-mon[33163]: from='osd.6 v2:172.21.15.103:6808/184353788' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:44:38.757 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:38 smithi103 ceph-mon[33163]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:44:38.757 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:38 smithi103 ceph-mon[33163]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:44:38.757 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:38 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:44:38.757 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:38 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:44:38.757 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:38 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:38.757 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:38 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:44:38.757 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:38 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:38.757 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:38 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:38.876 DEBUG:teuthology.orchestra.run.smithi103:osd.6> sudo journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@osd.6.service 2024-09-10T18:44:38.879 INFO:tasks.cephadm:Deploying osd.7 on smithi103 with /dev/vg_nvme/lv_1... 2024-09-10T18:44:38.879 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-10T18:44:38.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:38 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:38.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:38 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:38.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:38 smithi018 ceph-mon[29336]: from='osd.6 v2:172.21.15.103:6808/184353788' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:44:38.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:38 smithi018 ceph-mon[29336]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:44:38.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:38 smithi018 ceph-mon[29336]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:44:38.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:38 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:44:38.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:38 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:44:38.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:38 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:38.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:38 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:44:38.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:38 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:38.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:38 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:38.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:38 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:38.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:38 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:38.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:38 smithi018 ceph-mon[40196]: from='osd.6 v2:172.21.15.103:6808/184353788' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:44:38.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:38 smithi018 ceph-mon[40196]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:44:38.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:38 smithi018 ceph-mon[40196]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:44:38.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:38 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:44:38.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:38 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:44:38.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:38 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:38.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:38 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:44:38.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:38 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:38.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:38 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:39.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:39 smithi103 ceph-mon[33163]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T18:44:39.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:39 smithi103 ceph-mon[33163]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T18:44:39.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:39 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:44:39.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:39 smithi103 ceph-mon[33163]: from='osd.6 v2:172.21.15.103:6808/184353788' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T18:44:39.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:39 smithi103 ceph-mon[33163]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T18:44:39.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 18:44:39 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T18:44:39.499+0000 7f636ca59640 -1 osd.6 0 waiting for initial osdmap 2024-09-10T18:44:39.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 18:44:39 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T18:44:39.504+0000 7f636688e640 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:44:39.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:39 smithi018 ceph-mon[40196]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T18:44:39.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:39 smithi018 ceph-mon[40196]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T18:44:39.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:39 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:44:39.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:39 smithi018 ceph-mon[40196]: from='osd.6 v2:172.21.15.103:6808/184353788' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T18:44:39.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:39 smithi018 ceph-mon[40196]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T18:44:39.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:39 smithi018 ceph-mon[29336]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T18:44:39.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:39 smithi018 ceph-mon[29336]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T18:44:39.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:39 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:44:39.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:39 smithi018 ceph-mon[29336]: from='osd.6 v2:172.21.15.103:6808/184353788' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T18:44:39.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:39 smithi018 ceph-mon[29336]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T18:44:40.112 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:44:40.624 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:40 smithi103 ceph-mon[33163]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-10T18:44:40.625 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:40 smithi103 ceph-mon[33163]: osdmap e42: 7 total, 6 up, 7 in 2024-09-10T18:44:40.625 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:40 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:44:40.625 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:40 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:44:40.625 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:40 smithi103 ceph-mon[33163]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:44:40.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:40 smithi018 ceph-mon[40196]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-10T18:44:40.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:40 smithi018 ceph-mon[40196]: osdmap e42: 7 total, 6 up, 7 in 2024-09-10T18:44:40.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:40 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:44:40.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:40 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:44:40.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:40 smithi018 ceph-mon[40196]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:44:40.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:40 smithi018 ceph-mon[29336]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-10T18:44:40.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:40 smithi018 ceph-mon[29336]: osdmap e42: 7 total, 6 up, 7 in 2024-09-10T18:44:40.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:40 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:44:40.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:40 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:44:40.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:40 smithi018 ceph-mon[29336]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:44:41.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:41 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:44:41.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:41 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:44:41.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:41 smithi103 ceph-mon[33163]: osd.6 v2:172.21.15.103:6808/184353788 boot 2024-09-10T18:44:41.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:41 smithi103 ceph-mon[33163]: osdmap e43: 7 total, 7 up, 7 in 2024-09-10T18:44:41.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:41 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:44:41.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:41 smithi103 ceph-mon[33163]: osdmap e44: 7 total, 7 up, 7 in 2024-09-10T18:44:41.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:41 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:44:41.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:41 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:44:41.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:41 smithi018 ceph-mon[40196]: osd.6 v2:172.21.15.103:6808/184353788 boot 2024-09-10T18:44:41.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:41 smithi018 ceph-mon[40196]: osdmap e43: 7 total, 7 up, 7 in 2024-09-10T18:44:41.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:41 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:44:41.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:41 smithi018 ceph-mon[40196]: osdmap e44: 7 total, 7 up, 7 in 2024-09-10T18:44:41.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:41 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:44:41.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:41 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:44:41.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:41 smithi018 ceph-mon[29336]: osd.6 v2:172.21.15.103:6808/184353788 boot 2024-09-10T18:44:41.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:41 smithi018 ceph-mon[29336]: osdmap e43: 7 total, 7 up, 7 in 2024-09-10T18:44:41.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:41 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:44:41.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:41 smithi018 ceph-mon[29336]: osdmap e44: 7 total, 7 up, 7 in 2024-09-10T18:44:42.705 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:42 smithi103 ceph-mon[33163]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:44:42.705 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:42 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:42.705 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:42 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:42.705 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:42 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:44:42.705 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:42 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:44:42.705 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:42 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:44:42.705 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:42 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:42.705 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:42 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:44:42.705 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:42 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:44:42.705 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:42 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:42.706 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:42 smithi103 ceph-mon[33163]: osdmap e45: 7 total, 7 up, 7 in 2024-09-10T18:44:42.833 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-10T18:44:42.919 INFO:teuthology.orchestra.run.smithi103.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-10T18:44:42.933 INFO:teuthology.orchestra.run.smithi103.stderr: stderr: 10+0 records in 2024-09-10T18:44:42.933 INFO:teuthology.orchestra.run.smithi103.stderr:10+0 records out 2024-09-10T18:44:42.933 INFO:teuthology.orchestra.run.smithi103.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115941 s, 904 MB/s 2024-09-10T18:44:42.933 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping successful for: 2024-09-10T18:44:42.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:42 smithi018 ceph-mon[29336]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:44:42.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:42 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:42.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:42 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:42.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:42 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:44:42.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:42 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:44:42.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:42 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:44:42.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:42 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:42.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:42 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:44:42.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:42 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:44:42.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:42 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:42.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:42 smithi018 ceph-mon[29336]: osdmap e45: 7 total, 7 up, 7 in 2024-09-10T18:44:42.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:42 smithi018 ceph-mon[40196]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:44:42.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:42 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:42.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:42 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:42.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:42 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:44:42.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:42 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:44:42.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:42 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:44:42.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:42 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:42.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:42 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:44:42.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:42 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:44:42.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:42 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:44:42.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:42 smithi018 ceph-mon[40196]: osdmap e45: 7 total, 7 up, 7 in 2024-09-10T18:44:43.711 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_1 2024-09-10T18:44:43.955 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:44:44.329 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:44 smithi103 ceph-mon[33163]: Detected new or changed devices on smithi103 2024-09-10T18:44:44.329 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:44 smithi103 ceph-mon[33163]: Adjusting osd_memory_target on smithi103 to 5696M 2024-09-10T18:44:44.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:44 smithi018 ceph-mon[29336]: Detected new or changed devices on smithi103 2024-09-10T18:44:44.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:44 smithi018 ceph-mon[29336]: Adjusting osd_memory_target on smithi103 to 5696M 2024-09-10T18:44:44.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:44 smithi018 ceph-mon[40196]: Detected new or changed devices on smithi103 2024-09-10T18:44:44.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:44 smithi018 ceph-mon[40196]: Adjusting osd_memory_target on smithi103 to 5696M 2024-09-10T18:44:45.335 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:45 smithi103 ceph-mon[33163]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:44:45.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:45 smithi018 ceph-mon[29336]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:44:45.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:45 smithi018 ceph-mon[40196]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:44:46.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:46 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:44:46.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:46 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:44:46.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:46 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:44:46.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:46 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:44:46.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:46 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:44:46.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:46 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:44:46.447 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:46 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:44:46.447 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:46 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:44:46.447 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:46 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:44:47.321 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:47 smithi103 ceph-mon[33163]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:44:47.321 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:47 smithi103 ceph-mon[33163]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T18:44:47.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:47 smithi018 ceph-mon[29336]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:44:47.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:47 smithi018 ceph-mon[29336]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T18:44:47.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:47 smithi018 ceph-mon[40196]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:44:47.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:47 smithi018 ceph-mon[40196]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T18:44:48.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:48 smithi018 ceph-mon[29336]: from='client.? 172.21.15.103:0/1223351480' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f12282f8-4747-45ff-8336-95440560a88f"}]: dispatch 2024-09-10T18:44:48.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:48 smithi018 ceph-mon[29336]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f12282f8-4747-45ff-8336-95440560a88f"}]: dispatch 2024-09-10T18:44:48.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:48 smithi018 ceph-mon[29336]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f12282f8-4747-45ff-8336-95440560a88f"}]': finished 2024-09-10T18:44:48.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:48 smithi018 ceph-mon[29336]: osdmap e46: 8 total, 7 up, 8 in 2024-09-10T18:44:48.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:48 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:44:48.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:48 smithi018 ceph-mon[40196]: from='client.? 172.21.15.103:0/1223351480' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f12282f8-4747-45ff-8336-95440560a88f"}]: dispatch 2024-09-10T18:44:48.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:48 smithi018 ceph-mon[40196]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f12282f8-4747-45ff-8336-95440560a88f"}]: dispatch 2024-09-10T18:44:48.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:48 smithi018 ceph-mon[40196]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f12282f8-4747-45ff-8336-95440560a88f"}]': finished 2024-09-10T18:44:48.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:48 smithi018 ceph-mon[40196]: osdmap e46: 8 total, 7 up, 8 in 2024-09-10T18:44:48.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:48 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:44:48.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:48 smithi103 ceph-mon[33163]: from='client.? 172.21.15.103:0/1223351480' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f12282f8-4747-45ff-8336-95440560a88f"}]: dispatch 2024-09-10T18:44:48.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:48 smithi103 ceph-mon[33163]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f12282f8-4747-45ff-8336-95440560a88f"}]: dispatch 2024-09-10T18:44:48.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:48 smithi103 ceph-mon[33163]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f12282f8-4747-45ff-8336-95440560a88f"}]': finished 2024-09-10T18:44:48.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:48 smithi103 ceph-mon[33163]: osdmap e46: 8 total, 7 up, 8 in 2024-09-10T18:44:48.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:48 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:44:49.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:49 smithi018 ceph-mon[29336]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-10T18:44:49.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:49 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:44:49.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:49 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:44:49.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:49 smithi018 ceph-mon[29336]: from='client.? 172.21.15.103:0/3381021733' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:44:49.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:49 smithi018 ceph-mon[40196]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-10T18:44:49.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:49 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:44:49.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:49 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:44:49.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:49 smithi018 ceph-mon[40196]: from='client.? 172.21.15.103:0/3381021733' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:44:49.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:49 smithi103 ceph-mon[33163]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-10T18:44:49.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:49 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:44:49.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:49 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:44:49.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:49 smithi103 ceph-mon[33163]: from='client.? 172.21.15.103:0/3381021733' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:44:50.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:50 smithi018 ceph-mon[29336]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T18:44:50.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:50 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:44:50.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:50 smithi018 ceph-mon[40196]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T18:44:50.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:50 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:44:50.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:50 smithi103 ceph-mon[33163]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T18:44:50.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:50 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:44:51.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:51 smithi018 ceph-mon[29336]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-10T18:44:51.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:51 smithi018 ceph-mon[40196]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-10T18:44:51.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:51 smithi103 ceph-mon[33163]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-10T18:44:53.391 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:53 smithi103 ceph-mon[33163]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T18:44:53.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:53 smithi018 ceph-mon[29336]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T18:44:53.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:53 smithi018 ceph-mon[40196]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T18:44:55.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:55 smithi018 ceph-mon[40196]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:44:55.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:55 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T18:44:55.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:55 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:44:55.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:55 smithi018 ceph-mon[29336]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:44:55.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:55 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T18:44:55.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:55 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:44:55.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:55 smithi103 ceph-mon[33163]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:44:55.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:55 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T18:44:55.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:55 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:44:56.408 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:56 smithi103 ceph-mon[33163]: Deploying daemon osd.7 on smithi103 2024-09-10T18:44:56.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:56 smithi018 ceph-mon[40196]: Deploying daemon osd.7 on smithi103 2024-09-10T18:44:56.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:56 smithi018 ceph-mon[29336]: Deploying daemon osd.7 on smithi103 2024-09-10T18:44:57.336 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:57 smithi103 ceph-mon[33163]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:44:57.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:57 smithi018 ceph-mon[40196]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:44:57.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:57 smithi018 ceph-mon[29336]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:44:59.402 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:59 smithi103 ceph-mon[33163]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:44:59.403 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:59 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:44:59.403 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:44:59 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:44:59.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:59 smithi018 ceph-mon[29336]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:44:59.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:59 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:44:59.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:44:59 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:44:59.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:59 smithi018 ceph-mon[40196]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:44:59.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:59 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:44:59.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:44:59 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:45:00.255 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:00 smithi103 ceph-mon[33163]: osdmap e48: 8 total, 7 up, 8 in 2024-09-10T18:45:00.256 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:00 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:45:00.256 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:00 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:45:00.256 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:00 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:00.256 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:00 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:00.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:00 smithi018 ceph-mon[40196]: osdmap e48: 8 total, 7 up, 8 in 2024-09-10T18:45:00.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:00 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:45:00.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:00 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:45:00.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:00 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:00.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:00 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:00.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:00 smithi018 ceph-mon[29336]: osdmap e48: 8 total, 7 up, 8 in 2024-09-10T18:45:00.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:00 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:45:00.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:00 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:45:00.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:00 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:00.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:00 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:01.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:01 smithi018 ceph-mon[40196]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:45:01.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:01 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:45:01.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:01 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:45:01.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:01 smithi018 ceph-mon[40196]: osdmap e49: 8 total, 7 up, 8 in 2024-09-10T18:45:01.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:01 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:45:01.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:01 smithi018 ceph-mon[29336]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:45:01.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:01 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:45:01.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:01 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:45:01.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:01 smithi018 ceph-mon[29336]: osdmap e49: 8 total, 7 up, 8 in 2024-09-10T18:45:01.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:01 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:45:01.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:01 smithi103 ceph-mon[33163]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:45:01.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:01 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:45:01.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:01 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:45:01.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:01 smithi103 ceph-mon[33163]: osdmap e49: 8 total, 7 up, 8 in 2024-09-10T18:45:01.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:01 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:45:02.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:02 smithi103 ceph-mon[33163]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:45:02.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:02 smithi018 ceph-mon[29336]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:45:02.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:02 smithi018 ceph-mon[40196]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:45:03.120 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 7 on host 'smithi103' 2024-09-10T18:45:03.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:03 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:03.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:03 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:03.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:03 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:45:03.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:03 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:45:03.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:03 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:03.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:03 smithi018 ceph-mon[29336]: from='osd.7 v2:172.21.15.103:6812/941024403' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:45:03.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:03 smithi018 ceph-mon[29336]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:45:03.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:03 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:45:03.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:03 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:03.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:03 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:03.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:03 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:03.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:03 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:03.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:03 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:45:03.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:03 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:45:03.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:03 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:03.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:03 smithi018 ceph-mon[40196]: from='osd.7 v2:172.21.15.103:6812/941024403' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:45:03.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:03 smithi018 ceph-mon[40196]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:45:03.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:03 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:45:03.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:03 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:03.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:03 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:03.757 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:03 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:03.757 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:03 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:03.757 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:03 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:45:03.757 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:03 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:45:03.758 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:03 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:03.758 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:03 smithi103 ceph-mon[33163]: from='osd.7 v2:172.21.15.103:6812/941024403' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:45:03.758 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:03 smithi103 ceph-mon[33163]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:45:03.758 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:03 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:45:03.758 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:03 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:03.758 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:03 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:03.783 DEBUG:teuthology.orchestra.run.smithi103:osd.7> sudo journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@osd.7.service 2024-09-10T18:45:03.786 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-09-10T18:45:03.786 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T18:45:04.052 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:45:04.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:04 smithi018 ceph-mon[29336]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T18:45:04.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:04 smithi018 ceph-mon[29336]: osdmap e50: 8 total, 7 up, 8 in 2024-09-10T18:45:04.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:04 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:45:04.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:04 smithi018 ceph-mon[29336]: from='osd.7 v2:172.21.15.103:6812/941024403' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T18:45:04.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:04 smithi018 ceph-mon[29336]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T18:45:04.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:04 smithi018 ceph-mon[29336]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:45:04.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:04 smithi018 ceph-mon[40196]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T18:45:04.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:04 smithi018 ceph-mon[40196]: osdmap e50: 8 total, 7 up, 8 in 2024-09-10T18:45:04.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:04 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:45:04.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:04 smithi018 ceph-mon[40196]: from='osd.7 v2:172.21.15.103:6812/941024403' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T18:45:04.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:04 smithi018 ceph-mon[40196]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T18:45:04.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:04 smithi018 ceph-mon[40196]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:45:04.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:04 smithi103 ceph-mon[33163]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T18:45:04.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:04 smithi103 ceph-mon[33163]: osdmap e50: 8 total, 7 up, 8 in 2024-09-10T18:45:04.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:04 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:45:04.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:04 smithi103 ceph-mon[33163]: from='osd.7 v2:172.21.15.103:6812/941024403' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T18:45:04.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:04 smithi103 ceph-mon[33163]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T18:45:04.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:04 smithi103 ceph-mon[33163]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:45:04.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 18:45:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[55499]: 2024-09-10T18:45:04.430+0000 7f5cd9b52640 -1 osd.7 0 waiting for initial osdmap 2024-09-10T18:45:04.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 18:45:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[55499]: 2024-09-10T18:45:04.436+0000 7f5cd3987640 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:45:05.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:05 smithi018 ceph-mon[29336]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-10T18:45:05.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:05 smithi018 ceph-mon[29336]: osdmap e51: 8 total, 7 up, 8 in 2024-09-10T18:45:05.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:05 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:45:05.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:05 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:45:05.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:05 smithi018 ceph-mon[40196]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-10T18:45:05.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:05 smithi018 ceph-mon[40196]: osdmap e51: 8 total, 7 up, 8 in 2024-09-10T18:45:05.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:05 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:45:05.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:05 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:45:05.762 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:45:05.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:05 smithi103 ceph-mon[33163]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-10T18:45:05.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:05 smithi103 ceph-mon[33163]: osdmap e51: 8 total, 7 up, 8 in 2024-09-10T18:45:05.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:05 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:45:05.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:05 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:45:06.399 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":52,"num_osds":8,"num_up_osds":8,"osd_up_since":1725993905,"num_in_osds":8,"osd_in_since":1725993887,"num_remapped_pgs":1} 2024-09-10T18:45:06.400 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T18:45:06.681 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:45:06.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:06 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:45:06.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:06 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:45:06.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:06 smithi018 ceph-mon[29336]: osd.7 v2:172.21.15.103:6812/941024403 boot 2024-09-10T18:45:06.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:06 smithi018 ceph-mon[29336]: osdmap e52: 8 total, 8 up, 8 in 2024-09-10T18:45:06.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:06 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:45:06.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:06 smithi018 ceph-mon[29336]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:06.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:06 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/629551077' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T18:45:06.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:06 smithi018 ceph-mon[29336]: osdmap e53: 8 total, 8 up, 8 in 2024-09-10T18:45:06.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:06 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:45:06.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:06 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:45:06.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:06 smithi018 ceph-mon[40196]: osd.7 v2:172.21.15.103:6812/941024403 boot 2024-09-10T18:45:06.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:06 smithi018 ceph-mon[40196]: osdmap e52: 8 total, 8 up, 8 in 2024-09-10T18:45:06.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:06 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:45:06.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:06 smithi018 ceph-mon[40196]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:06.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:06 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/629551077' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T18:45:06.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:06 smithi018 ceph-mon[40196]: osdmap e53: 8 total, 8 up, 8 in 2024-09-10T18:45:06.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:06 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:45:06.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:06 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:45:06.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:06 smithi103 ceph-mon[33163]: osd.7 v2:172.21.15.103:6812/941024403 boot 2024-09-10T18:45:06.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:06 smithi103 ceph-mon[33163]: osdmap e52: 8 total, 8 up, 8 in 2024-09-10T18:45:06.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:06 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:45:06.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:06 smithi103 ceph-mon[33163]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:06.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:06 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/629551077' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T18:45:06.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:06 smithi103 ceph-mon[33163]: osdmap e53: 8 total, 8 up, 8 in 2024-09-10T18:45:07.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:07 smithi018 ceph-mon[29336]: Detected new or changed devices on smithi103 2024-09-10T18:45:07.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:07 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:07.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:07 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:07.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:07 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:07.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:07 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:07.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:07 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:07.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:07 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:07.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:07 smithi018 ceph-mon[29336]: Adjusting osd_memory_target on smithi103 to 4272M 2024-09-10T18:45:07.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:07 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:07.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:07 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:45:07.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:07 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:45:07.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:07 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:07.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:07 smithi018 ceph-mon[29336]: osdmap e54: 8 total, 8 up, 8 in 2024-09-10T18:45:07.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:07 smithi018 ceph-mon[40196]: Detected new or changed devices on smithi103 2024-09-10T18:45:07.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:07 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:07.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:07 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:07.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:07 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:07.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:07 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:07.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:07 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:07.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:07 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:07.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:07 smithi018 ceph-mon[40196]: Adjusting osd_memory_target on smithi103 to 4272M 2024-09-10T18:45:07.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:07 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:07.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:07 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:45:07.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:07 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:45:07.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:07 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:07.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:07 smithi018 ceph-mon[40196]: osdmap e54: 8 total, 8 up, 8 in 2024-09-10T18:45:08.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:07 smithi103 ceph-mon[33163]: Detected new or changed devices on smithi103 2024-09-10T18:45:08.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:07 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:08.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:07 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:08.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:07 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:08.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:07 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:08.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:07 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:08.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:07 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:08.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:07 smithi103 ceph-mon[33163]: Adjusting osd_memory_target on smithi103 to 4272M 2024-09-10T18:45:08.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:07 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:08.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:07 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:45:08.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:07 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:45:08.024 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:07 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:08.024 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:07 smithi103 ceph-mon[33163]: osdmap e54: 8 total, 8 up, 8 in 2024-09-10T18:45:08.393 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:45:08.393 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":54,"fsid":"ba327f6c-6fa3-11ef-bcea-c7b262605968","created":"2024-09-10T18:40:07.669687+0000","modified":"2024-09-10T18:45:07.428759+0000","last_up_change":"2024-09-10T18:45:05.425833+0000","last_in_change":"2024-09-10T18:44:47.993378+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T18:43:08.162659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"eb4c7e67-300c-4a1a-b9f5-bb028eeeabc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6801","nonce":2131279292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2131279292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2131279292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6803","nonce":2131279292}]},"public_addr":"172.21.15.18:6801/2131279292","cluster_addr":"172.21.15.18:6802/2131279292","heartbeat_back_addr":"172.21.15.18:6804/2131279292","heartbeat_front_addr":"172.21.15.18:6803/2131279292","state":["exists","up"]},{"osd":1,"uuid":"63bf6a98-e349-4cc0-9502-a31693070c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6805","nonce":3855640936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":3855640936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":3855640936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6807","nonce":3855640936}]},"public_addr":"172.21.15.18:6805/3855640936","cluster_addr":"172.21.15.18:6806/3855640936","heartbeat_back_addr":"172.21.15.18:6808/3855640936","heartbeat_front_addr":"172.21.15.18:6807/3855640936","state":["exists","up"]},{"osd":2,"uuid":"79a01c29-4f4d-43d1-b1ea-a3fcf014f84b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6809","nonce":4107623855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":4107623855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":4107623855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6811","nonce":4107623855}]},"public_addr":"172.21.15.18:6809/4107623855","cluster_addr":"172.21.15.18:6810/4107623855","heartbeat_back_addr":"172.21.15.18:6812/4107623855","heartbeat_front_addr":"172.21.15.18:6811/4107623855","state":["exists","up"]},{"osd":3,"uuid":"1809e677-b98c-4924-982e-9297e37563af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6813","nonce":2732204253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2732204253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":2732204253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6815","nonce":2732204253}]},"public_addr":"172.21.15.18:6813/2732204253","cluster_addr":"172.21.15.18:6814/2732204253","heartbeat_back_addr":"172.21.15.18:6816/2732204253","heartbeat_front_addr":"172.21.15.18:6815/2732204253","state":["exists","up"]},{"osd":4,"uuid":"b682a300-bb53-4f36-8a9f-0753164ec4b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":465860081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6801","nonce":465860081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6803","nonce":465860081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":465860081}]},"public_addr":"172.21.15.103:6800/465860081","cluster_addr":"172.21.15.103:6801/465860081","heartbeat_back_addr":"172.21.15.103:6803/465860081","heartbeat_front_addr":"172.21.15.103:6802/465860081","state":["exists","up"]},{"osd":5,"uuid":"2bbfdb6c-0709-4fe4-8a4a-006f576f18e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3498719258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6805","nonce":3498719258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6807","nonce":3498719258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3498719258}]},"public_addr":"172.21.15.103:6804/3498719258","cluster_addr":"172.21.15.103:6805/3498719258","heartbeat_back_addr":"172.21.15.103:6807/3498719258","heartbeat_front_addr":"172.21.15.103:6806/3498719258","state":["exists","up"]},{"osd":6,"uuid":"31f9711e-0ba8-4aeb-b852-7d93e8fcce33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":184353788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6809","nonce":184353788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6811","nonce":184353788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":184353788}]},"public_addr":"172.21.15.103:6808/184353788","cluster_addr":"172.21.15.103:6809/184353788","heartbeat_back_addr":"172.21.15.103:6811/184353788","heartbeat_front_addr":"172.21.15.103:6810/184353788","state":["exists","up"]},{"osd":7,"uuid":"f12282f8-4747-45ff-8336-95440560a88f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":941024403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6813","nonce":941024403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6815","nonce":941024403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":941024403}]},"public_addr":"172.21.15.103:6812/941024403","cluster_addr":"172.21.15.103:6813/941024403","heartbeat_back_addr":"172.21.15.103:6815/941024403","heartbeat_front_addr":"172.21.15.103:6814/941024403","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:44:59.941207+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:44:12.766590+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:44:17.761121+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:44:48.362914+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:44:59.098350+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:44:14.921272+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:44:38.629971+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:45:03.659584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/178638059":"2024-09-11T18:40:59.659411+0000","172.21.15.18:6800/2748756682":"2024-09-11T18:40:59.659411+0000","172.21.15.18:0/2875894312":"2024-09-11T18:40:59.659411+0000","172.21.15.18:0/3090642970":"2024-09-11T18:40:59.659411+0000","172.21.15.18:6800/3869801150":"2024-09-11T18:40:28.880042+0000","172.21.15.18:0/1859541419":"2024-09-11T18:40:28.880042+0000","172.21.15.18:0/57192865":"2024-09-11T18:40:28.880042+0000","172.21.15.18:0/458572898":"2024-09-11T18:40:28.880042+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T18:45:09.071 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T18:43:08.162659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-09-10T18:45:09.072 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-10T18:45:09.331 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:45:09.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:09 smithi018 ceph-mon[29336]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:09.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:09 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1441938324' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:45:09.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:09 smithi018 ceph-mon[40196]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:09.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:09 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1441938324' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:45:09.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:09 smithi103 ceph-mon[33163]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:09.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:09 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1441938324' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:45:11.040 INFO:teuthology.orchestra.run.smithi018.stdout:pg_num: 1 2024-09-10T18:45:11.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:11 smithi018 ceph-mon[29336]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:11.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:11 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3756804066' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T18:45:11.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:11 smithi018 ceph-mon[40196]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:11.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:11 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3756804066' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T18:45:11.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:11 smithi103 ceph-mon[33163]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:11.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:11 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3756804066' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T18:45:11.734 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi018 2024-09-10T18:45:11.734 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph orch apply rgw foo.a --placement '1;smithi018=foo.a' 2024-09-10T18:45:11.989 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:45:13.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:13 smithi103 ceph-mon[33163]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:13.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:13 smithi018 ceph-mon[29336]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:13.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:13 smithi018 ceph-mon[40196]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:13.817 INFO:teuthology.orchestra.run.smithi103.stdout:Scheduled rgw.foo.a update... 2024-09-10T18:45:14.467 DEBUG:teuthology.orchestra.run.smithi018:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@rgw.foo.a.service 2024-09-10T18:45:14.470 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi103 2024-09-10T18:45:14.470 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd pool create datapool 3 3 replicated 2024-09-10T18:45:14.707 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:45:14.920 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:14 smithi018 ceph-mon[40196]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:14.920 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:14 smithi018 ceph-mon[40196]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi018=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:45:14.920 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:14 smithi018 ceph-mon[40196]: Saving service rgw.foo.a spec with placement smithi018=foo.a;count:1 2024-09-10T18:45:14.920 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:14 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:14.920 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:14 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:45:14.920 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:14 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:45:14.920 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:14 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:45:14.920 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:14 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:14.921 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:14 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-10T18:45:14.921 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:14 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-10T18:45:14.921 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:14 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:14.921 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:14 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:45:14.921 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:14 smithi018 ceph-mon[40196]: Deploying daemon rgw.foo.a on smithi018 2024-09-10T18:45:14.921 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:14 smithi018 ceph-mon[29336]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:14.921 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:14 smithi018 ceph-mon[29336]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi018=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:45:14.921 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:14 smithi018 ceph-mon[29336]: Saving service rgw.foo.a spec with placement smithi018=foo.a;count:1 2024-09-10T18:45:14.921 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:14 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:14.921 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:14 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:45:14.921 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:14 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:45:14.922 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:14 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:45:14.922 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:14 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:14.922 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:14 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-10T18:45:14.922 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:14 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-10T18:45:14.922 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:14 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:14.922 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:14 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:45:14.922 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:14 smithi018 ceph-mon[29336]: Deploying daemon rgw.foo.a on smithi018 2024-09-10T18:45:15.138 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:14 smithi103 ceph-mon[33163]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:15.138 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:14 smithi103 ceph-mon[33163]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi018=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:45:15.138 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:14 smithi103 ceph-mon[33163]: Saving service rgw.foo.a spec with placement smithi018=foo.a;count:1 2024-09-10T18:45:15.138 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:14 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:15.138 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:14 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:45:15.138 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:14 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:45:15.138 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:14 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:45:15.138 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:14 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:15.138 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:14 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-10T18:45:15.139 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:14 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-10T18:45:15.139 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:14 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:15.139 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:14 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:45:15.139 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:14 smithi103 ceph-mon[33163]: Deploying daemon rgw.foo.a on smithi018 2024-09-10T18:45:15.944 INFO:journalctl@ceph.rgw.foo.a.smithi018.stdout:Sep 10 18:45:15 smithi018 podman[66142]: 2024-09-10 18:45:15.855027456 +0000 UTC m=+0.140270727 container create 6c15e50f4ae2df669c3ea3ae8422e469ad872cf663c36a3d79c7dff2c949bdf3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-rgw-foo-a, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-10T18:45:16.444 INFO:journalctl@ceph.rgw.foo.a.smithi018.stdout:Sep 10 18:45:16 smithi018 podman[66142]: 2024-09-10 18:45:16.040283502 +0000 UTC m=+0.325526765 container init 6c15e50f4ae2df669c3ea3ae8422e469ad872cf663c36a3d79c7dff2c949bdf3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-rgw-foo-a, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-10T18:45:16.444 INFO:journalctl@ceph.rgw.foo.a.smithi018.stdout:Sep 10 18:45:16 smithi018 podman[66142]: 2024-09-10 18:45:16.043272601 +0000 UTC m=+0.328515863 container start 6c15e50f4ae2df669c3ea3ae8422e469ad872cf663c36a3d79c7dff2c949bdf3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-rgw-foo-a, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-10T18:45:16.444 INFO:journalctl@ceph.rgw.foo.a.smithi018.stdout:Sep 10 18:45:16 smithi018 bash[66142]: 6c15e50f4ae2df669c3ea3ae8422e469ad872cf663c36a3d79c7dff2c949bdf3 2024-09-10T18:45:16.445 INFO:journalctl@ceph.rgw.foo.a.smithi018.stdout:Sep 10 18:45:16 smithi018 systemd[1]: Started Ceph rgw.foo.a for ba327f6c-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:45:16.957 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'datapool' created 2024-09-10T18:45:17.202 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:16 smithi018 ceph-mon[40196]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:17.203 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:16 smithi018 ceph-mon[40196]: from='client.? 172.21.15.103:0/1385826437' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:45:17.203 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:16 smithi018 ceph-mon[29336]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:17.203 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:16 smithi018 ceph-mon[29336]: from='client.? 172.21.15.103:0/1385826437' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:45:17.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:16 smithi103 ceph-mon[33163]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:17.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:16 smithi103 ceph-mon[33163]: from='client.? 172.21.15.103:0/1385826437' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:45:17.625 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- rbd pool init datapool 2024-09-10T18:45:17.869 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:45:18.235 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:17 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:45:18.235 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:17 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-10T18:45:18.235 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:17 smithi103 ceph-mon[33163]: osdmap e55: 8 total, 8 up, 8 in 2024-09-10T18:45:18.235 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:17 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:18.235 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:17 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2965248470' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T18:45:18.236 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:17 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:18.236 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:17 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:18.236 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:17 smithi103 ceph-mon[33163]: Saving service rgw.foo.a spec with placement smithi018=foo.a;count:1 2024-09-10T18:45:18.236 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:17 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:18.236 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:17 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:18.236 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:17 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:45:18.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:17 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:45:18.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:17 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-10T18:45:18.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:17 smithi018 ceph-mon[40196]: osdmap e55: 8 total, 8 up, 8 in 2024-09-10T18:45:18.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:17 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:18.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:17 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2965248470' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T18:45:18.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:17 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:18.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:17 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:18.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:17 smithi018 ceph-mon[40196]: Saving service rgw.foo.a spec with placement smithi018=foo.a;count:1 2024-09-10T18:45:18.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:17 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:18.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:17 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:18.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:17 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:45:18.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:17 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:45:18.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:17 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-10T18:45:18.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:17 smithi018 ceph-mon[29336]: osdmap e55: 8 total, 8 up, 8 in 2024-09-10T18:45:18.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:17 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:18.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:17 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2965248470' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T18:45:18.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:17 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:18.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:17 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:18.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:17 smithi018 ceph-mon[29336]: Saving service rgw.foo.a spec with placement smithi018=foo.a;count:1 2024-09-10T18:45:18.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:17 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:18.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:17 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:18.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:17 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:45:19.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:19 smithi103 ceph-mon[33163]: pgmap v174: 36 pgs: 6 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:19.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:19 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2965248470' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-10T18:45:19.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:19 smithi103 ceph-mon[33163]: osdmap e56: 8 total, 8 up, 8 in 2024-09-10T18:45:19.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:19 smithi103 ceph-mon[33163]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:45:19.275 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:19 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:19.275 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:19 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:19.275 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:19 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:45:19.275 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:19 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:45:19.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:19 smithi018 ceph-mon[29336]: pgmap v174: 36 pgs: 6 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:19.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:19 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2965248470' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-10T18:45:19.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:19 smithi018 ceph-mon[29336]: osdmap e56: 8 total, 8 up, 8 in 2024-09-10T18:45:19.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:19 smithi018 ceph-mon[29336]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:45:19.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:19 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:19.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:19 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:19.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:19 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:45:19.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:19 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:45:19.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:19 smithi018 ceph-mon[40196]: pgmap v174: 36 pgs: 6 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:19.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:19 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2965248470' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-10T18:45:19.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:19 smithi018 ceph-mon[40196]: osdmap e56: 8 total, 8 up, 8 in 2024-09-10T18:45:19.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:19 smithi018 ceph-mon[40196]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:45:19.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:19 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:19.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:19 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:19.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:19 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:45:19.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:19 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:45:20.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:20 smithi018 ceph-mon[29336]: Checking dashboard <-> RGW credentials 2024-09-10T18:45:20.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:20 smithi018 ceph-mon[29336]: osdmap e57: 8 total, 8 up, 8 in 2024-09-10T18:45:20.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:20 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2965248470' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:45:20.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:20 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3902229305' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:45:20.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:20 smithi018 ceph-mon[29336]: from='client.? 172.21.15.103:0/3301786441' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:45:20.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:20 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:45:20.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:20 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:20.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:20 smithi018 ceph-mon[40196]: Checking dashboard <-> RGW credentials 2024-09-10T18:45:20.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:20 smithi018 ceph-mon[40196]: osdmap e57: 8 total, 8 up, 8 in 2024-09-10T18:45:20.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:20 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2965248470' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:45:20.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:20 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3902229305' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:45:20.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:20 smithi018 ceph-mon[40196]: from='client.? 172.21.15.103:0/3301786441' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:45:20.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:20 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:45:20.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:20 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:20.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:20 smithi103 ceph-mon[33163]: Checking dashboard <-> RGW credentials 2024-09-10T18:45:20.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:20 smithi103 ceph-mon[33163]: osdmap e57: 8 total, 8 up, 8 in 2024-09-10T18:45:20.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:20 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2965248470' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:45:20.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:20 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3902229305' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:45:20.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:20 smithi103 ceph-mon[33163]: from='client.? 172.21.15.103:0/3301786441' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:45:20.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:20 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:45:20.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:20 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:21.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:21 smithi018 ceph-mon[29336]: pgmap v177: 68 pgs: 9 creating+peering, 54 unknown, 5 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:21.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:21 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2965248470' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:45:21.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:21 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3902229305' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:45:21.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:21 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-10T18:45:21.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:21 smithi018 ceph-mon[29336]: osdmap e58: 8 total, 8 up, 8 in 2024-09-10T18:45:21.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:21 smithi018 ceph-mon[40196]: pgmap v177: 68 pgs: 9 creating+peering, 54 unknown, 5 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:21.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:21 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2965248470' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:45:21.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:21 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3902229305' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:45:21.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:21 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-10T18:45:21.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:21 smithi018 ceph-mon[40196]: osdmap e58: 8 total, 8 up, 8 in 2024-09-10T18:45:21.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:21 smithi103 ceph-mon[33163]: pgmap v177: 68 pgs: 9 creating+peering, 54 unknown, 5 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:21.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:21 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2965248470' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:45:21.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:21 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3902229305' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:45:21.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:21 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-10T18:45:21.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:21 smithi103 ceph-mon[33163]: osdmap e58: 8 total, 8 up, 8 in 2024-09-10T18:45:22.394 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:22 smithi103 ceph-mon[33163]: osdmap e59: 8 total, 8 up, 8 in 2024-09-10T18:45:22.394 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:22 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2965248470' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:45:22.394 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:22 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3902229305' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:45:22.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:22 smithi018 ceph-mon[40196]: osdmap e59: 8 total, 8 up, 8 in 2024-09-10T18:45:22.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:22 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2965248470' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:45:22.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:22 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3902229305' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:45:22.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:22 smithi018 ceph-mon[29336]: osdmap e59: 8 total, 8 up, 8 in 2024-09-10T18:45:22.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:22 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2965248470' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:45:22.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:22 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3902229305' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:45:22.750 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.103 --placement '1;smithi103=iscsi.a' 2024-09-10T18:45:22.999 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:45:23.373 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:23 smithi103 ceph-mon[33163]: pgmap v180: 100 pgs: 13 creating+peering, 45 unknown, 42 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-10T18:45:23.373 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:23 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2965248470' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:45:23.373 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:23 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3902229305' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:45:23.373 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:23 smithi103 ceph-mon[33163]: osdmap e60: 8 total, 8 up, 8 in 2024-09-10T18:45:23.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:23 smithi018 ceph-mon[29336]: pgmap v180: 100 pgs: 13 creating+peering, 45 unknown, 42 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-10T18:45:23.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:23 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2965248470' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:45:23.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:23 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3902229305' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:45:23.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:23 smithi018 ceph-mon[29336]: osdmap e60: 8 total, 8 up, 8 in 2024-09-10T18:45:23.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:23 smithi018 ceph-mon[40196]: pgmap v180: 100 pgs: 13 creating+peering, 45 unknown, 42 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-10T18:45:23.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:23 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2965248470' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:45:23.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:23 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3902229305' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:45:23.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:23 smithi018 ceph-mon[40196]: osdmap e60: 8 total, 8 up, 8 in 2024-09-10T18:45:24.076 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:24 smithi103 ceph-mon[33163]: osdmap e61: 8 total, 8 up, 8 in 2024-09-10T18:45:24.077 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:24 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3895312980' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:45:24.077 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:24 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1315532943' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:45:24.403 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:24 smithi103 ceph-mon[33163]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:45:24.404 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:24 smithi103 ceph-mon[33163]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:45:24.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:24 smithi018 ceph-mon[40196]: osdmap e61: 8 total, 8 up, 8 in 2024-09-10T18:45:24.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:24 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3895312980' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:45:24.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:24 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1315532943' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:45:24.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:24 smithi018 ceph-mon[40196]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:45:24.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:24 smithi018 ceph-mon[40196]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:45:24.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:24 smithi018 ceph-mon[29336]: osdmap e61: 8 total, 8 up, 8 in 2024-09-10T18:45:24.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:24 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3895312980' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:45:24.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:24 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1315532943' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:45:24.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:24 smithi018 ceph-mon[29336]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:45:24.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:24 smithi018 ceph-mon[29336]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:45:24.819 INFO:teuthology.orchestra.run.smithi103.stdout:Scheduled iscsi.datapool update... 2024-09-10T18:45:25.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:25 smithi018 ceph-mon[29336]: pgmap v183: 132 pgs: 6 creating+peering, 41 unknown, 85 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-10T18:45:25.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:25 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3895312980' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:45:25.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:25 smithi018 ceph-mon[29336]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:45:25.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:25 smithi018 ceph-mon[29336]: osdmap e62: 8 total, 8 up, 8 in 2024-09-10T18:45:25.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:25 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3895312980' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:45:25.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:25 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1315532943' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:45:25.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:25 smithi018 ceph-mon[29336]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:45:25.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:25 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:25.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:25 smithi018 ceph-mon[40196]: pgmap v183: 132 pgs: 6 creating+peering, 41 unknown, 85 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-10T18:45:25.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:25 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3895312980' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:45:25.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:25 smithi018 ceph-mon[40196]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:45:25.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:25 smithi018 ceph-mon[40196]: osdmap e62: 8 total, 8 up, 8 in 2024-09-10T18:45:25.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:25 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3895312980' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:45:25.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:25 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1315532943' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:45:25.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:25 smithi018 ceph-mon[40196]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:45:25.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:25 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:25.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:25 smithi103 ceph-mon[33163]: pgmap v183: 132 pgs: 6 creating+peering, 41 unknown, 85 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-10T18:45:25.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:25 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3895312980' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:45:25.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:25 smithi103 ceph-mon[33163]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:45:25.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:25 smithi103 ceph-mon[33163]: osdmap e62: 8 total, 8 up, 8 in 2024-09-10T18:45:25.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:25 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3895312980' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:45:25.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:25 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1315532943' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:45:25.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:25 smithi103 ceph-mon[33163]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:45:25.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:25 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:25.494 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-09-10T18:45:25.494 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-10T18:45:25.494 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-10T18:45:25.523 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-10T18:45:25.523 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-10T18:45:25.550 DEBUG:teuthology.orchestra.run.smithi103:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@iscsi.iscsi.a.service 2024-09-10T18:45:25.593 INFO:tasks.cephadm:Adding prometheus.a on smithi103 2024-09-10T18:45:25.593 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph orch apply prometheus '1;smithi103=a' 2024-09-10T18:45:26.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:26 smithi018 ceph-mon[29336]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.103", "placement": "1;smithi103=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:45:26.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:26 smithi018 ceph-mon[29336]: Saving service iscsi.datapool spec with placement smithi103=iscsi.a;count:1 2024-09-10T18:45:26.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:26 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3895312980' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:45:26.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:26 smithi018 ceph-mon[29336]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:45:26.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:26 smithi018 ceph-mon[29336]: osdmap e63: 8 total, 8 up, 8 in 2024-09-10T18:45:26.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:26 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:26.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:26 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:26.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:26 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:26.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:26 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-10T18:45:26.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:26 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-10T18:45:26.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:26 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:45:26.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:26 smithi018 ceph-mon[40196]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.103", "placement": "1;smithi103=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:45:26.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:26 smithi018 ceph-mon[40196]: Saving service iscsi.datapool spec with placement smithi103=iscsi.a;count:1 2024-09-10T18:45:26.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:26 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3895312980' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:45:26.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:26 smithi018 ceph-mon[40196]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:45:26.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:26 smithi018 ceph-mon[40196]: osdmap e63: 8 total, 8 up, 8 in 2024-09-10T18:45:26.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:26 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:26.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:26 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:26.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:26 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:26.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:26 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-10T18:45:26.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:26 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-10T18:45:26.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:26 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:45:26.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:26 smithi103 ceph-mon[33163]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.103", "placement": "1;smithi103=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:45:26.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:26 smithi103 ceph-mon[33163]: Saving service iscsi.datapool spec with placement smithi103=iscsi.a;count:1 2024-09-10T18:45:26.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:26 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3895312980' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:45:26.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:26 smithi103 ceph-mon[33163]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:45:26.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:26 smithi103 ceph-mon[33163]: osdmap e63: 8 total, 8 up, 8 in 2024-09-10T18:45:26.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:26 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:26.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:26 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:26.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:26 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:26.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:26 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-10T18:45:26.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:26 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-10T18:45:26.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:26 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:45:26.773 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:45:27.347 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:27 smithi018 ceph-mon[29336]: Deploying daemon iscsi.iscsi.a on smithi103 2024-09-10T18:45:27.347 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:27 smithi018 ceph-mon[29336]: pgmap v186: 132 pgs: 6 creating+peering, 28 unknown, 98 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 1.5 KiB/s wr, 34 op/s 2024-09-10T18:45:27.347 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:27 smithi018 ceph-mon[29336]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T18:45:27.348 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:27 smithi018 ceph-mon[29336]: Cluster is now healthy 2024-09-10T18:45:27.348 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:27 smithi018 ceph-mon[40196]: Deploying daemon iscsi.iscsi.a on smithi103 2024-09-10T18:45:27.348 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:27 smithi018 ceph-mon[40196]: pgmap v186: 132 pgs: 6 creating+peering, 28 unknown, 98 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 1.5 KiB/s wr, 34 op/s 2024-09-10T18:45:27.348 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:27 smithi018 ceph-mon[40196]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T18:45:27.348 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:27 smithi018 ceph-mon[40196]: Cluster is now healthy 2024-09-10T18:45:27.371 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:27 smithi103 ceph-mon[33163]: Deploying daemon iscsi.iscsi.a on smithi103 2024-09-10T18:45:27.371 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:27 smithi103 ceph-mon[33163]: pgmap v186: 132 pgs: 6 creating+peering, 28 unknown, 98 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 1.5 KiB/s wr, 34 op/s 2024-09-10T18:45:27.371 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:27 smithi103 ceph-mon[33163]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T18:45:27.371 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:27 smithi103 ceph-mon[33163]: Cluster is now healthy 2024-09-10T18:45:28.680 INFO:teuthology.orchestra.run.smithi103.stdout:Scheduled prometheus update... 2024-09-10T18:45:29.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:29 smithi018 ceph-mon[29336]: pgmap v187: 132 pgs: 2 creating+peering, 5 unknown, 125 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 69 KiB/s rd, 5.7 KiB/s wr, 163 op/s 2024-09-10T18:45:29.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:29 smithi018 ceph-mon[29336]: osdmap e64: 8 total, 8 up, 8 in 2024-09-10T18:45:29.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:29 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:29.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:29 smithi018 ceph-mon[40196]: pgmap v187: 132 pgs: 2 creating+peering, 5 unknown, 125 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 69 KiB/s rd, 5.7 KiB/s wr, 163 op/s 2024-09-10T18:45:29.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:29 smithi018 ceph-mon[40196]: osdmap e64: 8 total, 8 up, 8 in 2024-09-10T18:45:29.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:29 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:29.509 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:29 smithi103 ceph-mon[33163]: pgmap v187: 132 pgs: 2 creating+peering, 5 unknown, 125 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 69 KiB/s rd, 5.7 KiB/s wr, 163 op/s 2024-09-10T18:45:29.510 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:29 smithi103 ceph-mon[33163]: osdmap e64: 8 total, 8 up, 8 in 2024-09-10T18:45:29.510 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:29 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:29.536 DEBUG:teuthology.orchestra.run.smithi103:prometheus.a> sudo journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@prometheus.a.service 2024-09-10T18:45:29.539 INFO:tasks.cephadm:Adding node-exporter.a on smithi018 2024-09-10T18:45:29.539 INFO:tasks.cephadm:Adding node-exporter.b on smithi103 2024-09-10T18:45:29.539 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph orch apply node-exporter '2;smithi018=a;smithi103=b' 2024-09-10T18:45:29.885 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:45:30.306 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:30 smithi103 ceph-mon[33163]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi103=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:45:30.306 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:30 smithi103 ceph-mon[33163]: Saving service prometheus spec with placement smithi103=a;count:1 2024-09-10T18:45:30.306 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:30 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:45:30.306 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:30 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:45:30.306 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:30 smithi103 ceph-mon[33163]: osdmap e65: 8 total, 8 up, 8 in 2024-09-10T18:45:30.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:30 smithi018 ceph-mon[29336]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi103=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:45:30.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:30 smithi018 ceph-mon[29336]: Saving service prometheus spec with placement smithi103=a;count:1 2024-09-10T18:45:30.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:30 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:45:30.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:30 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:45:30.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:30 smithi018 ceph-mon[29336]: osdmap e65: 8 total, 8 up, 8 in 2024-09-10T18:45:30.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:30 smithi018 ceph-mon[40196]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi103=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:45:30.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:30 smithi018 ceph-mon[40196]: Saving service prometheus spec with placement smithi103=a;count:1 2024-09-10T18:45:30.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:30 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:45:30.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:30 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:45:30.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:30 smithi018 ceph-mon[40196]: osdmap e65: 8 total, 8 up, 8 in 2024-09-10T18:45:31.023 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:45:30 smithi103 podman[60145]: 2024-09-10 18:45:30.945878008 +0000 UTC m=+0.760563707 container start 8f9cfa438ded371803362d153d614ecc06f7debbe50a6ea4594fb04f7d50d30e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T18:45:31.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:31 smithi018 ceph-mon[29336]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 82 KiB/s rd, 6.5 KiB/s wr, 199 op/s 2024-09-10T18:45:31.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:31 smithi018 ceph-mon[40196]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 82 KiB/s rd, 6.5 KiB/s wr, 199 op/s 2024-09-10T18:45:31.508 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:31 smithi103 ceph-mon[33163]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 82 KiB/s rd, 6.5 KiB/s wr, 199 op/s 2024-09-10T18:45:31.509 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:45:31 smithi103 bash[60145]: 8f9cfa438ded371803362d153d614ecc06f7debbe50a6ea4594fb04f7d50d30e 2024-09-10T18:45:31.509 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:45:31 smithi103 systemd[1]: Started Ceph iscsi.iscsi.a for ba327f6c-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:45:31.509 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:45:31 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug Started the configuration object watcher 2024-09-10T18:45:31.509 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:45:31 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug Checking for config object changes every 1s 2024-09-10T18:45:31.509 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:45:31 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug Processing osd blocklist entries for this node 2024-09-10T18:45:31.509 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:45:31 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug Reading the configuration object to update local LIO configuration 2024-09-10T18:45:31.509 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:45:31 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug Configuration does not have an entry for this host(smithi103.front.sepia.ceph.com) - nothing to define to LIO 2024-09-10T18:45:31.773 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:45:31 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: * Serving Flask app 'rbd-target-api' (lazy loading) 2024-09-10T18:45:31.773 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:45:31 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: * Environment: production 2024-09-10T18:45:31.773 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:45:31 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-10T18:45:31.773 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:45:31 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: Use a production WSGI server instead. 2024-09-10T18:45:31.773 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:45:31 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: * Debug mode: off 2024-09-10T18:45:31.773 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:45:31 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug * Running on all addresses. 2024-09-10T18:45:31.773 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:45:31 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-10T18:45:31.773 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:45:31 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: * Running on all addresses. 2024-09-10T18:45:31.773 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:45:31 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-10T18:45:31.774 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:45:31 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug * Running on http://[fe80::ec4:7aff:fe8f:cc0b]:5000/ (Press CTRL+C to quit) 2024-09-10T18:45:31.774 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:45:31 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: * Running on http://[fe80::ec4:7aff:fe8f:cc0b]:5000/ (Press CTRL+C to quit) 2024-09-10T18:45:32.135 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:32 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:32.135 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:32 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:32.234 INFO:teuthology.orchestra.run.smithi103.stdout:Scheduled node-exporter update... 2024-09-10T18:45:32.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:32 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:32.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:32 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:32.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:32 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:32.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:32 smithi018 ceph-mon[29336]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-10T18:45:32.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:32 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:32.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:32 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:45:32.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:32 smithi018 ceph-mon[29336]: from='client.? 172.21.15.103:0/3219964488' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T18:45:32.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:32 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:32.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:32 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:32.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:32 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:32.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:32 smithi018 ceph-mon[40196]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-10T18:45:32.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:32 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:32.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:32 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:45:32.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:32 smithi018 ceph-mon[40196]: from='client.? 172.21.15.103:0/3219964488' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T18:45:32.474 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:32 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:32.474 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:32 smithi103 ceph-mon[33163]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-10T18:45:32.474 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:32 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:32.474 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:32 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:45:32.474 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:32 smithi103 ceph-mon[33163]: from='client.? 172.21.15.103:0/3219964488' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T18:45:33.389 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:33 smithi103 ceph-mon[33163]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 70 KiB/s rd, 5.6 KiB/s wr, 169 op/s 2024-09-10T18:45:33.390 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:33 smithi103 ceph-mon[33163]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-10T18:45:33.390 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:33 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:33.413 DEBUG:teuthology.orchestra.run.smithi018:node-exporter.a> sudo journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@node-exporter.a.service 2024-09-10T18:45:33.416 DEBUG:teuthology.orchestra.run.smithi103:node-exporter.b> sudo journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@node-exporter.b.service 2024-09-10T18:45:33.418 INFO:tasks.cephadm:Adding alertmanager.a on smithi018 2024-09-10T18:45:33.419 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph orch apply alertmanager '1;smithi018=a' 2024-09-10T18:45:33.442 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:33 smithi018 ceph-mon[40196]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 70 KiB/s rd, 5.6 KiB/s wr, 169 op/s 2024-09-10T18:45:33.442 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:33 smithi018 ceph-mon[40196]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-10T18:45:33.442 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:33 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:33.442 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:33 smithi018 ceph-mon[29336]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 70 KiB/s rd, 5.6 KiB/s wr, 169 op/s 2024-09-10T18:45:33.443 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:33 smithi018 ceph-mon[29336]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-10T18:45:33.443 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:33 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:34.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:34 smithi018 ceph-mon[40196]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi018=a;smithi103=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:45:34.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:34 smithi018 ceph-mon[40196]: Saving service node-exporter spec with placement smithi018=a;smithi103=b;count:2 2024-09-10T18:45:34.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:34 smithi018 ceph-mon[29336]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi018=a;smithi103=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:45:34.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:34 smithi018 ceph-mon[29336]: Saving service node-exporter spec with placement smithi018=a;smithi103=b;count:2 2024-09-10T18:45:34.446 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:45:34.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:34 smithi103 ceph-mon[33163]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi018=a;smithi103=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:45:34.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:34 smithi103 ceph-mon[33163]: Saving service node-exporter spec with placement smithi018=a;smithi103=b;count:2 2024-09-10T18:45:35.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:35 smithi018 ceph-mon[40196]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 51 KiB/s rd, 4.1 KiB/s wr, 124 op/s 2024-09-10T18:45:35.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:35 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:35.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:35 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:35.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:35 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:45:35.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:35 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:45:35.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:35 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:35.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:35 smithi018 ceph-mon[40196]: Deploying daemon prometheus.a on smithi103 2024-09-10T18:45:35.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:35 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:35.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:35 smithi018 ceph-mon[29336]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 51 KiB/s rd, 4.1 KiB/s wr, 124 op/s 2024-09-10T18:45:35.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:35 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:35.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:35 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:35.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:35 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:45:35.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:35 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:45:35.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:35 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:35.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:35 smithi018 ceph-mon[29336]: Deploying daemon prometheus.a on smithi103 2024-09-10T18:45:35.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:35 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:35.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:35 smithi103 ceph-mon[33163]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 51 KiB/s rd, 4.1 KiB/s wr, 124 op/s 2024-09-10T18:45:35.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:35 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:35.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:35 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:35.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:35 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:45:35.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:35 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:45:35.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:35 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:35.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:35 smithi103 ceph-mon[33163]: Deploying daemon prometheus.a on smithi103 2024-09-10T18:45:35.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:35 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:36.288 INFO:teuthology.orchestra.run.smithi103.stdout:Scheduled alertmanager update... 2024-09-10T18:45:36.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:36 smithi018 ceph-mon[40196]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 11 KiB/s rd, 895 B/s wr, 26 op/s 2024-09-10T18:45:36.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:36 smithi018 ceph-mon[29336]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 11 KiB/s rd, 895 B/s wr, 26 op/s 2024-09-10T18:45:36.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:36 smithi103 ceph-mon[33163]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 11 KiB/s rd, 895 B/s wr, 26 op/s 2024-09-10T18:45:37.323 DEBUG:teuthology.orchestra.run.smithi018:alertmanager.a> sudo journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@alertmanager.a.service 2024-09-10T18:45:37.326 INFO:tasks.cephadm:Adding grafana.a on smithi103 2024-09-10T18:45:37.327 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph orch apply grafana '1;smithi103=a' 2024-09-10T18:45:37.574 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:45:37.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:37 smithi018 ceph-mon[40196]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi018=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:45:37.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:37 smithi018 ceph-mon[40196]: Saving service alertmanager spec with placement smithi018=a;count:1 2024-09-10T18:45:37.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:37 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:37.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:37 smithi018 ceph-mon[29336]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi018=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:45:37.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:37 smithi018 ceph-mon[29336]: Saving service alertmanager spec with placement smithi018=a;count:1 2024-09-10T18:45:37.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:37 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:37.695 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:37 smithi103 ceph-mon[33163]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi018=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:45:37.695 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:37 smithi103 ceph-mon[33163]: Saving service alertmanager spec with placement smithi018=a;count:1 2024-09-10T18:45:37.695 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:37 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:38.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:38 smithi018 ceph-mon[29336]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 552 B/s rd, 110 B/s wr, 1 op/s 2024-09-10T18:45:38.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:38 smithi018 ceph-mon[40196]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 552 B/s rd, 110 B/s wr, 1 op/s 2024-09-10T18:45:38.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:38 smithi103 ceph-mon[33163]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 552 B/s rd, 110 B/s wr, 1 op/s 2024-09-10T18:45:40.577 INFO:teuthology.orchestra.run.smithi103.stdout:Scheduled grafana update... 2024-09-10T18:45:41.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:40 smithi103 ceph-mon[33163]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 510 B/s rd, 102 B/s wr, 1 op/s 2024-09-10T18:45:41.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:40 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:41.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:40 smithi018 ceph-mon[29336]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 510 B/s rd, 102 B/s wr, 1 op/s 2024-09-10T18:45:41.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:40 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:41.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:40 smithi018 ceph-mon[40196]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 510 B/s rd, 102 B/s wr, 1 op/s 2024-09-10T18:45:41.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:40 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:41.459 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:45:41 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:45:41.485 DEBUG:teuthology.orchestra.run.smithi103:grafana.a> sudo journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@grafana.a.service 2024-09-10T18:45:41.487 INFO:tasks.cephadm:Setting up client nodes... 2024-09-10T18:45:41.487 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T18:45:41.756 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:45:42.106 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:41 smithi018 ceph-mon[29336]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi103=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:45:42.106 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:41 smithi018 ceph-mon[29336]: Saving service grafana spec with placement smithi103=a;count:1 2024-09-10T18:45:42.106 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:41 smithi018 ceph-mon[40196]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi103=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:45:42.107 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:41 smithi018 ceph-mon[40196]: Saving service grafana spec with placement smithi103=a;count:1 2024-09-10T18:45:42.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:41 smithi103 ceph-mon[33163]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi103=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:45:42.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:41 smithi103 ceph-mon[33163]: Saving service grafana spec with placement smithi103=a;count:1 2024-09-10T18:45:42.777 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:42 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:42.777 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:42 smithi103 ceph-mon[33163]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:45:43.096 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:42 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:43.096 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:42 smithi018 ceph-mon[40196]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:45:43.096 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:42 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:43.096 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:42 smithi018 ceph-mon[29336]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:45:43.552 INFO:teuthology.orchestra.run.smithi018.stdout:[client.0] 2024-09-10T18:45:43.552 INFO:teuthology.orchestra.run.smithi018.stdout: key = AQDXk+BmKF6gIBAAx4ymU3oFKJdYftR6m5MTiA== 2024-09-10T18:45:44.199 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-10T18:45:44.200 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-10T18:45:44.200 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T18:45:44.240 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T18:45:44.744 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:44 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:45:44.745 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:44 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:45:44.745 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:44 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/4138919178' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:45:44.745 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:44 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/4138919178' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:45:44.745 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:44 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:45:44.745 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:44 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:45:44.745 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:44 smithi103 ceph-mon[33163]: pgmap v197: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:45:44.745 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:44 smithi103 ceph-mon[33163]: osdmap e66: 8 total, 8 up, 8 in 2024-09-10T18:45:44.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:44 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:45:44.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:44 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:45:44.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:44 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/4138919178' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:45:44.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:44 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/4138919178' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:45:44.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:44 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:45:44.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:44 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:45:44.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:44 smithi018 ceph-mon[40196]: pgmap v197: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:45:44.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:44 smithi018 ceph-mon[40196]: osdmap e66: 8 total, 8 up, 8 in 2024-09-10T18:45:44.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:44 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:45:44.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:44 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:45:44.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:44 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/4138919178' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:45:44.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:44 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/4138919178' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:45:44.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:44 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:45:44.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:44 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:45:44.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:44 smithi018 ceph-mon[29336]: pgmap v197: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:45:44.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:44 smithi018 ceph-mon[29336]: osdmap e66: 8 total, 8 up, 8 in 2024-09-10T18:45:45.898 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:45 smithi103 ceph-mon[33163]: 1.0 scrub starts 2024-09-10T18:45:45.899 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:45 smithi103 ceph-mon[33163]: 1.0 scrub ok 2024-09-10T18:45:46.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:45 smithi018 ceph-mon[29336]: 1.0 scrub starts 2024-09-10T18:45:46.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:45 smithi018 ceph-mon[29336]: 1.0 scrub ok 2024-09-10T18:45:46.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:45 smithi018 ceph-mon[40196]: 1.0 scrub starts 2024-09-10T18:45:46.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:45 smithi018 ceph-mon[40196]: 1.0 scrub ok 2024-09-10T18:45:46.357 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:45:47.011 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:46 smithi103 ceph-mon[33163]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 102 B/s wr, 1 op/s 2024-09-10T18:45:47.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:46 smithi018 ceph-mon[29336]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 102 B/s wr, 1 op/s 2024-09-10T18:45:47.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:46 smithi018 ceph-mon[40196]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 102 B/s wr, 1 op/s 2024-09-10T18:45:47.974 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:47 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:45:47.975 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:47 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:45:47.975 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:47 smithi103 ceph-mon[33163]: osdmap e67: 8 total, 8 up, 8 in 2024-09-10T18:45:48.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:47 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:45:48.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:47 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:45:48.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:47 smithi018 ceph-mon[29336]: osdmap e67: 8 total, 8 up, 8 in 2024-09-10T18:45:48.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:47 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:45:48.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:47 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:45:48.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:47 smithi018 ceph-mon[40196]: osdmap e67: 8 total, 8 up, 8 in 2024-09-10T18:45:48.914 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:48 smithi103 ceph-mon[33163]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:45:48.914 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:48 smithi103 ceph-mon[33163]: osdmap e68: 8 total, 8 up, 8 in 2024-09-10T18:45:49.186 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:45:49 smithi103 podman[61651]: 2024-09-10 18:45:49.160634181 +0000 UTC m=+0.840104511 container start e6b5cd4d0de26275b2a4a5d90d1e3fb84a393f831779894e5eea67ddd3baa97c (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:45:49.186 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:45:49 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[61672]: ts=2024-09-10T18:45:49.184Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-10T18:45:49.186 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:45:49 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[61672]: ts=2024-09-10T18:45:49.184Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-10T18:45:49.186 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:45:49 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[61672]: ts=2024-09-10T18:45:49.184Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi103 (none))" 2024-09-10T18:45:49.186 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:45:49 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[61672]: ts=2024-09-10T18:45:49.184Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-10T18:45:49.187 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:45:49 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[61672]: ts=2024-09-10T18:45:49.184Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-10T18:45:49.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:48 smithi018 ceph-mon[29336]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:45:49.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:48 smithi018 ceph-mon[29336]: osdmap e68: 8 total, 8 up, 8 in 2024-09-10T18:45:49.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:48 smithi018 ceph-mon[40196]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:45:49.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:48 smithi018 ceph-mon[40196]: osdmap e68: 8 total, 8 up, 8 in 2024-09-10T18:45:49.241 INFO:teuthology.orchestra.run.smithi103.stdout:[client.1] 2024-09-10T18:45:49.241 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQDdk+BmRsoGDhAAcWWcDOPsWW+w9NzCR4/rKQ== 2024-09-10T18:45:49.523 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:45:49 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[61672]: ts=2024-09-10T18:45:49.185Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-10T18:45:49.523 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:45:49 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[61672]: ts=2024-09-10T18:45:49.186Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-10T18:45:49.523 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:45:49 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[61672]: ts=2024-09-10T18:45:49.187Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-10T18:45:49.523 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:45:49 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[61672]: ts=2024-09-10T18:45:49.187Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-10T18:45:49.523 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:45:49 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[61672]: ts=2024-09-10T18:45:49.189Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-10T18:45:49.523 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:45:49 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[61672]: ts=2024-09-10T18:45:49.189Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.612µs 2024-09-10T18:45:49.523 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:45:49 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[61672]: ts=2024-09-10T18:45:49.189Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-10T18:45:49.523 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:45:49 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[61672]: ts=2024-09-10T18:45:49.190Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-09-10T18:45:49.524 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:45:49 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[61672]: ts=2024-09-10T18:45:49.190Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=25.578µs wal_replay_duration=377.567µs wbl_replay_duration=104ns total_replay_duration=421.381µs 2024-09-10T18:45:49.524 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:45:49 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[61672]: ts=2024-09-10T18:45:49.190Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-10T18:45:49.524 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:45:49 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[61672]: ts=2024-09-10T18:45:49.190Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-10T18:45:49.524 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:45:49 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[61672]: ts=2024-09-10T18:45:49.190Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-10T18:45:49.524 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:45:49 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[61672]: ts=2024-09-10T18:45:49.206Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=15.600035ms db_storage=952ns remote_storage=1.111µs web_handler=237ns query_engine=449ns scrape=375.79µs scrape_sd=17.442µs notify=1.093µs notify_sd=1.462µs rules=15.02797ms tracing=5.05µs 2024-09-10T18:45:49.524 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:45:49 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[61672]: ts=2024-09-10T18:45:49.206Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-10T18:45:49.524 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:45:49 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[61672]: ts=2024-09-10T18:45:49.206Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-10T18:45:49.524 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:45:49 smithi103 bash[61651]: e6b5cd4d0de26275b2a4a5d90d1e3fb84a393f831779894e5eea67ddd3baa97c 2024-09-10T18:45:49.524 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:45:49 smithi103 systemd[1]: Started Ceph prometheus.a for ba327f6c-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:45:49.912 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:49 smithi103 ceph-mon[33163]: from='client.? 172.21.15.103:0/1518880562' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:45:49.912 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:49 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:45:49.912 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:49 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:45:49.912 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:49 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:49.912 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:49 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:49.913 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:49 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:49.913 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:49 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-10T18:45:49.938 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-10T18:45:49.938 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-10T18:45:49.938 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-10T18:45:49.978 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T18:45:49.978 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-10T18:45:49.979 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-10T18:45:50.006 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:49 smithi018 ceph-mon[40196]: from='client.? 172.21.15.103:0/1518880562' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:45:50.006 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:49 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:45:50.007 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:49 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:45:50.007 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:49 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:50.007 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:49 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:50.007 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:49 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:50.007 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:49 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-10T18:45:50.007 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:49 smithi018 ceph-mon[29336]: from='client.? 172.21.15.103:0/1518880562' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:45:50.007 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:49 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:45:50.007 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:49 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:45:50.007 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:49 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:50.007 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:49 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:50.007 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:49 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:50.008 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:49 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-10T18:45:50.243 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:45:50.629 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:50 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ignoring --setuser ceph since I am not root 2024-09-10T18:45:50.629 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:50 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ignoring --setgroup ceph since I am not root 2024-09-10T18:45:50.629 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:50 smithi018 ceph-mgr[29568]: -- 172.21.15.18:0/2730295197 <== mon.2 v2:172.21.15.18:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564a0a88c340 con 0x564a0a898c00 2024-09-10T18:45:50.629 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:50 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:45:50.462+0000 7f67102ddec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:45:50.629 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:50 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:45:50.523+0000 7f67102ddec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:45:50.629 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:50 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:45:50.592+0000 7f67102ddec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:45:50.704 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:45:50 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ignoring --setuser ceph since I am not root 2024-09-10T18:45:50.704 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:45:50 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ignoring --setgroup ceph since I am not root 2024-09-10T18:45:50.704 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:45:50 smithi103 ceph-mgr[35001]: -- 172.21.15.103:0/1819284328 <== mon.1 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d0b29de340 con 0x55d0b29ea800 2024-09-10T18:45:50.704 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:45:50 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:45:50.456+0000 7f9edf8fbec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:45:50.704 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:45:50 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:45:50.511+0000 7f9edf8fbec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:45:50.704 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:45:50 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:45:50.577+0000 7f9edf8fbec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:45:50.704 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:45:50 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:45:50.701+0000 7f9edf8fbec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:45:50.878 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:50 smithi018 ceph-mon[29336]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:45:50.879 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:50 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:50.879 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:50 smithi018 ceph-mon[29336]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-10T18:45:50.879 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:50 smithi018 ceph-mon[29336]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-10T18:45:50.879 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:50 smithi018 ceph-mon[40196]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:45:50.879 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:50 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:50.879 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:50 smithi018 ceph-mon[40196]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-10T18:45:50.879 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:50 smithi018 ceph-mon[40196]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-10T18:45:50.880 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:50 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:45:50.733+0000 7f67102ddec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:45:50.880 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:50 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:45:50.876+0000 7f67102ddec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:45:51.022 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:45:50 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:45:50.838+0000 7f9edf8fbec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:45:51.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:50 smithi103 ceph-mon[33163]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:45:51.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:50 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' 2024-09-10T18:45:51.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:50 smithi103 ceph-mon[33163]: from='mgr.14150 172.21.15.18:0/2584281931' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-10T18:45:51.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:50 smithi103 ceph-mon[33163]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-10T18:45:51.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:51 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:45:51.147+0000 7f67102ddec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:45:51.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:51 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:45:51.210+0000 7f67102ddec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:45:51.445 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:51 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:45:51.379+0000 7f67102ddec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:45:51.523 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:45:51 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:45:51.093+0000 7f9edf8fbec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:45:51.523 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:45:51 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:45:51.153+0000 7f9edf8fbec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:45:51.523 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:45:51 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:45:51.313+0000 7f9edf8fbec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:45:52.219 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:45:52.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:52 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1540772210' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T18:45:52.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:45:52 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:45:52.150+0000 7f9edf8fbec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:45:52.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:45:52 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:45:52.213+0000 7f9edf8fbec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:45:52.666 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:52 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1540772210' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T18:45:52.666 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:52 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:45:52.288+0000 7f67102ddec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:45:52.666 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:52 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:45:52.356+0000 7f67102ddec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:45:52.666 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:52 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1540772210' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T18:45:53.064 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":732375116}]},"active_addr":"172.21.15.18:6800/732375116","active_change":"2024-09-10T18:40:59.659512+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":24106,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.18:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.18:0","nonce":185700038}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.18:0","nonce":2408360238}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.18:0","nonce":3021537013}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.18:0","nonce":2982361259}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.18:0","nonce":692931402}]}]} 2024-09-10T18:45:53.069 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-10T18:45:53.069 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-10T18:45:53.069 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T18:45:53.207 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:45:52 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:45:52.916+0000 7f9edf8fbec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:45:53.207 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:45:53 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:45:53.027+0000 7f9edf8fbec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:45:53.207 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:45:53 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:45:53.146+0000 7f9edf8fbec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:45:53.367 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:53 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:45:53.103+0000 7f67102ddec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:45:53.367 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:53 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:45:53.224+0000 7f67102ddec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:45:53.367 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:53 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:45:53.365+0000 7f67102ddec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:45:53.368 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:45:53.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:45:53 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:45:53.204+0000 7f9edf8fbec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:45:53.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:45:53 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:45:53.328+0000 7f9edf8fbec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:45:53.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:45:53 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:45:53.395+0000 7f9edf8fbec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:45:53.523 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:45:53 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:45:53.452+0000 7f9edf8fbec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:45:53.648 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:53 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:45:53.430+0000 7f67102ddec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:45:53.648 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:53 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:45:53.574+0000 7f67102ddec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:45:53.853 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:45:53 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:45:53.588+0000 7f9edf8fbec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:45:53.854 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:45:53 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:45:53.651+0000 7f9edf8fbec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:45:53.854 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:45:53 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:45:53.715+0000 7f9edf8fbec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:45:53.854 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:45:53 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:45:53.794+0000 7f9edf8fbec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:45:53.854 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:45:53 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:45:53.851+0000 7f9edf8fbec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:45:53.922 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:53 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:45:53.646+0000 7f67102ddec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:45:53.922 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:53 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:45:53.712+0000 7f67102ddec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:45:53.922 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:53 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:45:53.853+0000 7f67102ddec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:45:53.922 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:53 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:45:53.919+0000 7f67102ddec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:45:54.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:53 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:45:53.987+0000 7f67102ddec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:45:54.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:54 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:45:54.074+0000 7f67102ddec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:45:54.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:54 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:45:54.135+0000 7f67102ddec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:45:54.272 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:45:53 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:45:53.908+0000 7f9edf8fbec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:45:54.461 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:54 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:45:54.195+0000 7f67102ddec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:45:54.762 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:54 smithi103 ceph-mon[33163]: osdmap e69: 8 total, 8 up, 8 in 2024-09-10T18:45:54.762 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:45:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:45:54.653+0000 7f9edf8fbec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:45:54.762 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:45:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: [10/Sep/2024:18:45:54] ENGINE Bus STARTING 2024-09-10T18:45:54.763 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:45:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: CherryPy Checker: 2024-09-10T18:45:54.763 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:45:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: The Application mounted at '' has an empty config. 2024-09-10T18:45:54.763 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:45:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: 2024-09-10T18:45:54.763 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:45:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: [10/Sep/2024:18:45:54] ENGINE Serving on http://:::9283 2024-09-10T18:45:54.763 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:45:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: [10/Sep/2024:18:45:54] ENGINE Bus STARTED 2024-09-10T18:45:54.834 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:54 smithi018 ceph-mon[29336]: osdmap e69: 8 total, 8 up, 8 in 2024-09-10T18:45:54.834 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:54 smithi018 ceph-mon[40196]: osdmap e69: 8 total, 8 up, 8 in 2024-09-10T18:45:55.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:54 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:45:54.996+0000 7f67102ddec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:45:55.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:18:45:55] ENGINE Bus STARTING 2024-09-10T18:45:55.195 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: CherryPy Checker: 2024-09-10T18:45:55.195 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: The Application mounted at '' has an empty config. 2024-09-10T18:45:55.195 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:45:55.276 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:45:55.276 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":70,"fsid":"ba327f6c-6fa3-11ef-bcea-c7b262605968","created":"2024-09-10T18:40:07.669687+0000","modified":"2024-09-10T18:45:54.998743+0000","last_up_change":"2024-09-10T18:45:05.425833+0000","last_in_change":"2024-09-10T18:44:47.993378+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T18:43:08.162659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-10T18:45:16.512219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-10T18:45:16.943626+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":60,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-10T18:45:18.450026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-10T18:45:20.494618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-10T18:45:22.566992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"eb4c7e67-300c-4a1a-b9f5-bb028eeeabc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6801","nonce":2131279292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2131279292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2131279292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6803","nonce":2131279292}]},"public_addr":"172.21.15.18:6801/2131279292","cluster_addr":"172.21.15.18:6802/2131279292","heartbeat_back_addr":"172.21.15.18:6804/2131279292","heartbeat_front_addr":"172.21.15.18:6803/2131279292","state":["exists","up"]},{"osd":1,"uuid":"63bf6a98-e349-4cc0-9502-a31693070c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6805","nonce":3855640936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":3855640936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":3855640936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6807","nonce":3855640936}]},"public_addr":"172.21.15.18:6805/3855640936","cluster_addr":"172.21.15.18:6806/3855640936","heartbeat_back_addr":"172.21.15.18:6808/3855640936","heartbeat_front_addr":"172.21.15.18:6807/3855640936","state":["exists","up"]},{"osd":2,"uuid":"79a01c29-4f4d-43d1-b1ea-a3fcf014f84b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6809","nonce":4107623855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":4107623855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":4107623855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6811","nonce":4107623855}]},"public_addr":"172.21.15.18:6809/4107623855","cluster_addr":"172.21.15.18:6810/4107623855","heartbeat_back_addr":"172.21.15.18:6812/4107623855","heartbeat_front_addr":"172.21.15.18:6811/4107623855","state":["exists","up"]},{"osd":3,"uuid":"1809e677-b98c-4924-982e-9297e37563af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6813","nonce":2732204253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2732204253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":2732204253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6815","nonce":2732204253}]},"public_addr":"172.21.15.18:6813/2732204253","cluster_addr":"172.21.15.18:6814/2732204253","heartbeat_back_addr":"172.21.15.18:6816/2732204253","heartbeat_front_addr":"172.21.15.18:6815/2732204253","state":["exists","up"]},{"osd":4,"uuid":"b682a300-bb53-4f36-8a9f-0753164ec4b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":465860081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6801","nonce":465860081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6803","nonce":465860081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":465860081}]},"public_addr":"172.21.15.103:6800/465860081","cluster_addr":"172.21.15.103:6801/465860081","heartbeat_back_addr":"172.21.15.103:6803/465860081","heartbeat_front_addr":"172.21.15.103:6802/465860081","state":["exists","up"]},{"osd":5,"uuid":"2bbfdb6c-0709-4fe4-8a4a-006f576f18e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3498719258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6805","nonce":3498719258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6807","nonce":3498719258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3498719258}]},"public_addr":"172.21.15.103:6804/3498719258","cluster_addr":"172.21.15.103:6805/3498719258","heartbeat_back_addr":"172.21.15.103:6807/3498719258","heartbeat_front_addr":"172.21.15.103:6806/3498719258","state":["exists","up"]},{"osd":6,"uuid":"31f9711e-0ba8-4aeb-b852-7d93e8fcce33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":184353788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6809","nonce":184353788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6811","nonce":184353788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":184353788}]},"public_addr":"172.21.15.103:6808/184353788","cluster_addr":"172.21.15.103:6809/184353788","heartbeat_back_addr":"172.21.15.103:6811/184353788","heartbeat_front_addr":"172.21.15.103:6810/184353788","state":["exists","up"]},{"osd":7,"uuid":"f12282f8-4747-45ff-8336-95440560a88f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":941024403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6813","nonce":941024403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6815","nonce":941024403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":941024403}]},"public_addr":"172.21.15.103:6812/941024403","cluster_addr":"172.21.15.103:6813/941024403","heartbeat_back_addr":"172.21.15.103:6815/941024403","heartbeat_front_addr":"172.21.15.103:6814/941024403","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:44:59.941207+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:45:28.836311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:45:46.446949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:44:48.362914+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:44:59.098350+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:45:43.685374+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:45:43.218426+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:45:03.659584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/3021537013":"2024-09-11T18:45:54.998616+0000","172.21.15.18:0/458572898":"2024-09-11T18:40:28.880042+0000","172.21.15.18:0/1859541419":"2024-09-11T18:40:28.880042+0000","172.21.15.18:0/3090642970":"2024-09-11T18:40:59.659411+0000","172.21.15.18:0/2875894312":"2024-09-11T18:40:59.659411+0000","172.21.15.18:0/178638059":"2024-09-11T18:40:59.659411+0000","172.21.15.18:0/692931402":"2024-09-11T18:45:54.998616+0000","172.21.15.18:0/2408360238":"2024-09-11T18:45:54.998616+0000","172.21.15.18:0/57192865":"2024-09-11T18:40:28.880042+0000","172.21.15.18:6800/2748756682":"2024-09-11T18:40:59.659411+0000","172.21.15.18:6800/732375116":"2024-09-11T18:45:54.998616+0000","172.21.15.18:6800/3869801150":"2024-09-11T18:40:28.880042+0000","172.21.15.18:0/2982361259":"2024-09-11T18:45:54.998616+0000","172.21.15.18:0/185700038":"2024-09-11T18:45:54.998616+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T18:45:55.552 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[40196]: Standby manager daemon x restarted 2024-09-10T18:45:55.552 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[40196]: Standby manager daemon x started 2024-09-10T18:45:55.552 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[40196]: from='mgr.? 172.21.15.103:0/3578732071' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:45:55.552 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[40196]: from='mgr.? 172.21.15.103:0/3578732071' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:45:55.552 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[40196]: from='mgr.? 172.21.15.103:0/3578732071' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:45:55.552 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[40196]: from='mgr.? 172.21.15.103:0/3578732071' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:45:55.552 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[40196]: Active manager daemon y restarted 2024-09-10T18:45:55.552 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[40196]: Activating manager daemon y 2024-09-10T18:45:55.552 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[40196]: osdmap e70: 8 total, 8 up, 8 in 2024-09-10T18:45:55.552 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[40196]: mgrmap e18: y(active, starting, since 0.00901436s), standbys: x 2024-09-10T18:45:55.553 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:45:55.553 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:45:55.553 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:45:55.553 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:45:55.553 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:45:55.553 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:45:55.553 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:45:55.553 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:45:55.553 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:45:55.553 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:45:55.553 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:45:55.553 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:45:55.553 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:45:55.553 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:45:55.554 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:45:55.554 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:45:55.554 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[40196]: Manager daemon y is now available 2024-09-10T18:45:55.554 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:45:55.554 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:45:55.554 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:45:55.554 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:45:55.554 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:45:55.554 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:45:55.554 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:45:55.554 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2171772756' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:45:55.555 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[29336]: Standby manager daemon x restarted 2024-09-10T18:45:55.555 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[29336]: Standby manager daemon x started 2024-09-10T18:45:55.555 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[29336]: from='mgr.? 172.21.15.103:0/3578732071' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:45:55.555 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[29336]: from='mgr.? 172.21.15.103:0/3578732071' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:45:55.555 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[29336]: from='mgr.? 172.21.15.103:0/3578732071' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:45:55.555 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[29336]: from='mgr.? 172.21.15.103:0/3578732071' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:45:55.555 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[29336]: Active manager daemon y restarted 2024-09-10T18:45:55.555 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[29336]: Activating manager daemon y 2024-09-10T18:45:55.556 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[29336]: osdmap e70: 8 total, 8 up, 8 in 2024-09-10T18:45:55.556 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[29336]: mgrmap e18: y(active, starting, since 0.00901436s), standbys: x 2024-09-10T18:45:55.556 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:45:55.556 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:45:55.556 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:45:55.556 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:45:55.556 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:45:55.556 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:45:55.556 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:45:55.556 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:45:55.556 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:45:55.556 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:45:55.556 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:45:55.557 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:45:55.557 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:45:55.557 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:45:55.557 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:45:55.557 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:45:55.557 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[29336]: Manager daemon y is now available 2024-09-10T18:45:55.557 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:45:55.557 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:45:55.557 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:45:55.557 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:45:55.557 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:45:55.557 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:45:55.557 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:45:55.557 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2171772756' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:45:55.558 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:18:45:55] ENGINE Serving on http://:::9283 2024-09-10T18:45:55.558 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:55 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:18:45:55] ENGINE Bus STARTED 2024-09-10T18:45:55.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:55 smithi103 ceph-mon[33163]: Standby manager daemon x restarted 2024-09-10T18:45:55.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:55 smithi103 ceph-mon[33163]: Standby manager daemon x started 2024-09-10T18:45:55.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:55 smithi103 ceph-mon[33163]: from='mgr.? 172.21.15.103:0/3578732071' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:45:55.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:55 smithi103 ceph-mon[33163]: from='mgr.? 172.21.15.103:0/3578732071' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:45:55.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:55 smithi103 ceph-mon[33163]: from='mgr.? 172.21.15.103:0/3578732071' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:45:55.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:55 smithi103 ceph-mon[33163]: from='mgr.? 172.21.15.103:0/3578732071' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:45:55.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:55 smithi103 ceph-mon[33163]: Active manager daemon y restarted 2024-09-10T18:45:55.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:55 smithi103 ceph-mon[33163]: Activating manager daemon y 2024-09-10T18:45:55.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:55 smithi103 ceph-mon[33163]: osdmap e70: 8 total, 8 up, 8 in 2024-09-10T18:45:55.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:55 smithi103 ceph-mon[33163]: mgrmap e18: y(active, starting, since 0.00901436s), standbys: x 2024-09-10T18:45:55.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:45:55.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:45:55.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:45:55.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:45:55.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:45:55.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:45:55.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:45:55.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:45:55.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:45:55.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:45:55.775 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:45:55.775 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:45:55.775 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:45:55.775 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:45:55.775 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:45:55.775 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:45:55.775 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:55 smithi103 ceph-mon[33163]: Manager daemon y is now available 2024-09-10T18:45:55.775 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:45:55.775 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:55 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:45:55.775 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:45:55.775 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:45:55.775 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:55 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:45:55.775 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:45:55.775 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:55 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:45:55.776 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:55 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2171772756' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:45:56.098 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-10T18:45:56.098 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T18:45:56.272 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:45:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:45:56.909 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:45:57.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:57 smithi103 ceph-mon[33163]: mgrmap e19: y(active, since 1.01101s), standbys: x 2024-09-10T18:45:57.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:57 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:57.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:57 smithi103 ceph-mon[33163]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:57.278 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:57 smithi018 ceph-mon[40196]: mgrmap e19: y(active, since 1.01101s), standbys: x 2024-09-10T18:45:57.278 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:57 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:57.278 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:57 smithi018 ceph-mon[40196]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:57.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:57 smithi018 ceph-mon[29336]: mgrmap e19: y(active, since 1.01101s), standbys: x 2024-09-10T18:45:57.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:57 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:57.279 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:57 smithi018 ceph-mon[29336]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:58.108 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:58 smithi018 ceph-mon[29336]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:58.108 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:58 smithi018 ceph-mon[29336]: mgrmap e20: y(active, since 2s), standbys: x 2024-09-10T18:45:58.108 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:58 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:45:58.108 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:58 smithi018 ceph-mon[40196]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:58.108 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:58 smithi018 ceph-mon[40196]: mgrmap e20: y(active, since 2s), standbys: x 2024-09-10T18:45:58.108 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:58 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:45:58.109 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:57 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:45:57.849+0000 7f66af057640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:18:45:57] ENGINE Error in HTTPServer.serve 2024-09-10T18:45:58.109 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:57 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: Traceback (most recent call last): 2024-09-10T18:45:58.109 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:57 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:45:58.109 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:57 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: self._connections.run(self.expiration_interval) 2024-09-10T18:45:58.109 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:57 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:45:58.109 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:57 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: self._run(expiration_interval) 2024-09-10T18:45:58.109 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:57 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:45:58.109 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:57 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:45:58.110 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:57 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:45:58.110 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:57 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:45:58.110 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:57 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:45:58.110 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:57 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: s = self.context.wrap_socket( 2024-09-10T18:45:58.110 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:57 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:45:58.110 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:57 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: return self.sslsocket_class._create( 2024-09-10T18:45:58.110 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:57 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:45:58.110 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:57 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: self.do_handshake() 2024-09-10T18:45:58.110 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:57 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:45:58.110 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:57 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: self._sslobj.do_handshake() 2024-09-10T18:45:58.110 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:57 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:45:58.110 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:45:57 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:45:58.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:58 smithi103 ceph-mon[33163]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:58.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:58 smithi103 ceph-mon[33163]: mgrmap e20: y(active, since 2s), standbys: x 2024-09-10T18:45:58.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:58 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:45:58.752 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:45:58.753 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":71,"fsid":"ba327f6c-6fa3-11ef-bcea-c7b262605968","created":"2024-09-10T18:40:07.669687+0000","modified":"2024-09-10T18:45:58.458890+0000","last_up_change":"2024-09-10T18:45:05.425833+0000","last_in_change":"2024-09-10T18:44:47.993378+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T18:43:08.162659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-10T18:45:16.512219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-10T18:45:16.943626+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":60,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-10T18:45:18.450026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-10T18:45:20.494618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-10T18:45:22.566992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"eb4c7e67-300c-4a1a-b9f5-bb028eeeabc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6801","nonce":2131279292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":2131279292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":2131279292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6803","nonce":2131279292}]},"public_addr":"172.21.15.18:6801/2131279292","cluster_addr":"172.21.15.18:6802/2131279292","heartbeat_back_addr":"172.21.15.18:6804/2131279292","heartbeat_front_addr":"172.21.15.18:6803/2131279292","state":["exists","up"]},{"osd":1,"uuid":"63bf6a98-e349-4cc0-9502-a31693070c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6805","nonce":3855640936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":3855640936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":3855640936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6807","nonce":3855640936}]},"public_addr":"172.21.15.18:6805/3855640936","cluster_addr":"172.21.15.18:6806/3855640936","heartbeat_back_addr":"172.21.15.18:6808/3855640936","heartbeat_front_addr":"172.21.15.18:6807/3855640936","state":["exists","up"]},{"osd":2,"uuid":"79a01c29-4f4d-43d1-b1ea-a3fcf014f84b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6809","nonce":4107623855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":4107623855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":4107623855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6811","nonce":4107623855}]},"public_addr":"172.21.15.18:6809/4107623855","cluster_addr":"172.21.15.18:6810/4107623855","heartbeat_back_addr":"172.21.15.18:6812/4107623855","heartbeat_front_addr":"172.21.15.18:6811/4107623855","state":["exists","up"]},{"osd":3,"uuid":"1809e677-b98c-4924-982e-9297e37563af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6813","nonce":2732204253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2732204253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":2732204253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6815","nonce":2732204253}]},"public_addr":"172.21.15.18:6813/2732204253","cluster_addr":"172.21.15.18:6814/2732204253","heartbeat_back_addr":"172.21.15.18:6816/2732204253","heartbeat_front_addr":"172.21.15.18:6815/2732204253","state":["exists","up"]},{"osd":4,"uuid":"b682a300-bb53-4f36-8a9f-0753164ec4b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":465860081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6801","nonce":465860081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6803","nonce":465860081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":465860081}]},"public_addr":"172.21.15.103:6800/465860081","cluster_addr":"172.21.15.103:6801/465860081","heartbeat_back_addr":"172.21.15.103:6803/465860081","heartbeat_front_addr":"172.21.15.103:6802/465860081","state":["exists","up"]},{"osd":5,"uuid":"2bbfdb6c-0709-4fe4-8a4a-006f576f18e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3498719258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6805","nonce":3498719258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6807","nonce":3498719258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3498719258}]},"public_addr":"172.21.15.103:6804/3498719258","cluster_addr":"172.21.15.103:6805/3498719258","heartbeat_back_addr":"172.21.15.103:6807/3498719258","heartbeat_front_addr":"172.21.15.103:6806/3498719258","state":["exists","up"]},{"osd":6,"uuid":"31f9711e-0ba8-4aeb-b852-7d93e8fcce33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":184353788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6809","nonce":184353788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6811","nonce":184353788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":184353788}]},"public_addr":"172.21.15.103:6808/184353788","cluster_addr":"172.21.15.103:6809/184353788","heartbeat_back_addr":"172.21.15.103:6811/184353788","heartbeat_front_addr":"172.21.15.103:6810/184353788","state":["exists","up"]},{"osd":7,"uuid":"f12282f8-4747-45ff-8336-95440560a88f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":941024403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6813","nonce":941024403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6815","nonce":941024403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":941024403}]},"public_addr":"172.21.15.103:6812/941024403","cluster_addr":"172.21.15.103:6813/941024403","heartbeat_back_addr":"172.21.15.103:6815/941024403","heartbeat_front_addr":"172.21.15.103:6814/941024403","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:44:59.941207+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:45:28.836311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:45:46.446949+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:44:48.362914+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:44:59.098350+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:45:43.685374+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:45:43.218426+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:45:03.659584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/3021537013":"2024-09-11T18:45:54.998616+0000","172.21.15.18:0/458572898":"2024-09-11T18:40:28.880042+0000","172.21.15.18:0/1859541419":"2024-09-11T18:40:28.880042+0000","172.21.15.18:0/3090642970":"2024-09-11T18:40:59.659411+0000","172.21.15.18:0/2875894312":"2024-09-11T18:40:59.659411+0000","172.21.15.18:0/178638059":"2024-09-11T18:40:59.659411+0000","172.21.15.18:0/692931402":"2024-09-11T18:45:54.998616+0000","172.21.15.18:0/2408360238":"2024-09-11T18:45:54.998616+0000","172.21.15.18:0/57192865":"2024-09-11T18:40:28.880042+0000","172.21.15.18:6800/2748756682":"2024-09-11T18:40:59.659411+0000","172.21.15.18:6800/732375116":"2024-09-11T18:45:54.998616+0000","172.21.15.18:6800/3869801150":"2024-09-11T18:40:28.880042+0000","172.21.15.18:0/2982361259":"2024-09-11T18:45:54.998616+0000","172.21.15.18:0/185700038":"2024-09-11T18:45:54.998616+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T18:45:59.365 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[40196]: [10/Sep/2024:18:45:57] ENGINE Bus STARTING 2024-09-10T18:45:59.365 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[40196]: [10/Sep/2024:18:45:57] ENGINE Serving on https://172.21.15.18:7150 2024-09-10T18:45:59.365 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[40196]: [10/Sep/2024:18:45:57] ENGINE Bus STARTED 2024-09-10T18:45:59.365 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[40196]: [10/Sep/2024:18:45:57] ENGINE Error in HTTPServer.serve 2024-09-10T18:45:59.365 INFO:journalctl@ceph.mon.c.smithi018.stdout: Traceback (most recent call last): 2024-09-10T18:45:59.366 INFO:journalctl@ceph.mon.c.smithi018.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:45:59.366 INFO:journalctl@ceph.mon.c.smithi018.stdout: self._connections.run(self.expiration_interval) 2024-09-10T18:45:59.366 INFO:journalctl@ceph.mon.c.smithi018.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:45:59.366 INFO:journalctl@ceph.mon.c.smithi018.stdout: self._run(expiration_interval) 2024-09-10T18:45:59.366 INFO:journalctl@ceph.mon.c.smithi018.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:45:59.366 INFO:journalctl@ceph.mon.c.smithi018.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:45:59.366 INFO:journalctl@ceph.mon.c.smithi018.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:45:59.366 INFO:journalctl@ceph.mon.c.smithi018.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:45:59.366 INFO:journalctl@ceph.mon.c.smithi018.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:45:59.366 INFO:journalctl@ceph.mon.c.smithi018.stdout: s = self.context.wrap_socket( 2024-09-10T18:45:59.366 INFO:journalctl@ceph.mon.c.smithi018.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:45:59.366 INFO:journalctl@ceph.mon.c.smithi018.stdout: return self.sslsocket_class._create( 2024-09-10T18:45:59.367 INFO:journalctl@ceph.mon.c.smithi018.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:45:59.367 INFO:journalctl@ceph.mon.c.smithi018.stdout: self.do_handshake() 2024-09-10T18:45:59.367 INFO:journalctl@ceph.mon.c.smithi018.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:45:59.367 INFO:journalctl@ceph.mon.c.smithi018.stdout: self._sslobj.do_handshake() 2024-09-10T18:45:59.367 INFO:journalctl@ceph.mon.c.smithi018.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:45:59.367 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:45:59.367 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:45:59.367 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[40196]: osdmap e71: 8 total, 8 up, 8 in 2024-09-10T18:45:59.367 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:45:59.367 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:45:59.367 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3144021212' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:45:59.367 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:45:59.367 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:45:59.368 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:59.368 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:59.368 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:59.368 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:59.368 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:59.368 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:59.368 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:59.368 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:59.368 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:45:59.368 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[29336]: [10/Sep/2024:18:45:57] ENGINE Bus STARTING 2024-09-10T18:45:59.369 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[29336]: [10/Sep/2024:18:45:57] ENGINE Serving on https://172.21.15.18:7150 2024-09-10T18:45:59.369 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[29336]: [10/Sep/2024:18:45:57] ENGINE Bus STARTED 2024-09-10T18:45:59.369 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[29336]: [10/Sep/2024:18:45:57] ENGINE Error in HTTPServer.serve 2024-09-10T18:45:59.369 INFO:journalctl@ceph.mon.a.smithi018.stdout: Traceback (most recent call last): 2024-09-10T18:45:59.369 INFO:journalctl@ceph.mon.a.smithi018.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:45:59.369 INFO:journalctl@ceph.mon.a.smithi018.stdout: self._connections.run(self.expiration_interval) 2024-09-10T18:45:59.369 INFO:journalctl@ceph.mon.a.smithi018.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:45:59.369 INFO:journalctl@ceph.mon.a.smithi018.stdout: self._run(expiration_interval) 2024-09-10T18:45:59.369 INFO:journalctl@ceph.mon.a.smithi018.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:45:59.369 INFO:journalctl@ceph.mon.a.smithi018.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:45:59.369 INFO:journalctl@ceph.mon.a.smithi018.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:45:59.369 INFO:journalctl@ceph.mon.a.smithi018.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:45:59.370 INFO:journalctl@ceph.mon.a.smithi018.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:45:59.370 INFO:journalctl@ceph.mon.a.smithi018.stdout: s = self.context.wrap_socket( 2024-09-10T18:45:59.370 INFO:journalctl@ceph.mon.a.smithi018.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:45:59.370 INFO:journalctl@ceph.mon.a.smithi018.stdout: return self.sslsocket_class._create( 2024-09-10T18:45:59.370 INFO:journalctl@ceph.mon.a.smithi018.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:45:59.370 INFO:journalctl@ceph.mon.a.smithi018.stdout: self.do_handshake() 2024-09-10T18:45:59.370 INFO:journalctl@ceph.mon.a.smithi018.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:45:59.370 INFO:journalctl@ceph.mon.a.smithi018.stdout: self._sslobj.do_handshake() 2024-09-10T18:45:59.370 INFO:journalctl@ceph.mon.a.smithi018.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:45:59.370 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:45:59.370 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:45:59.370 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[29336]: osdmap e71: 8 total, 8 up, 8 in 2024-09-10T18:45:59.370 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:45:59.371 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:45:59.371 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3144021212' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:45:59.371 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:45:59.371 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:45:59.371 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:59.371 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:59.371 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:59.371 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:59.371 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:59.371 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:59.371 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:59.371 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:59.371 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:45:59 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:45:59.391 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-10T18:45:59.392 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-10T18:45:59.392 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-10T18:45:59.393 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-10T18:45:59.393 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-10T18:45:59.393 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-10T18:45:59.394 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-10T18:45:59.394 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-10T18:45:59.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:59 smithi103 ceph-mon[33163]: [10/Sep/2024:18:45:57] ENGINE Bus STARTING 2024-09-10T18:45:59.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:59 smithi103 ceph-mon[33163]: [10/Sep/2024:18:45:57] ENGINE Serving on https://172.21.15.18:7150 2024-09-10T18:45:59.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:59 smithi103 ceph-mon[33163]: [10/Sep/2024:18:45:57] ENGINE Bus STARTED 2024-09-10T18:45:59.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:59 smithi103 ceph-mon[33163]: [10/Sep/2024:18:45:57] ENGINE Error in HTTPServer.serve 2024-09-10T18:45:59.523 INFO:journalctl@ceph.mon.b.smithi103.stdout: Traceback (most recent call last): 2024-09-10T18:45:59.523 INFO:journalctl@ceph.mon.b.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:45:59.523 INFO:journalctl@ceph.mon.b.smithi103.stdout: self._connections.run(self.expiration_interval) 2024-09-10T18:45:59.523 INFO:journalctl@ceph.mon.b.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:45:59.523 INFO:journalctl@ceph.mon.b.smithi103.stdout: self._run(expiration_interval) 2024-09-10T18:45:59.523 INFO:journalctl@ceph.mon.b.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:45:59.523 INFO:journalctl@ceph.mon.b.smithi103.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:45:59.524 INFO:journalctl@ceph.mon.b.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:45:59.524 INFO:journalctl@ceph.mon.b.smithi103.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:45:59.524 INFO:journalctl@ceph.mon.b.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:45:59.524 INFO:journalctl@ceph.mon.b.smithi103.stdout: s = self.context.wrap_socket( 2024-09-10T18:45:59.524 INFO:journalctl@ceph.mon.b.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:45:59.524 INFO:journalctl@ceph.mon.b.smithi103.stdout: return self.sslsocket_class._create( 2024-09-10T18:45:59.524 INFO:journalctl@ceph.mon.b.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:45:59.524 INFO:journalctl@ceph.mon.b.smithi103.stdout: self.do_handshake() 2024-09-10T18:45:59.524 INFO:journalctl@ceph.mon.b.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:45:59.524 INFO:journalctl@ceph.mon.b.smithi103.stdout: self._sslobj.do_handshake() 2024-09-10T18:45:59.524 INFO:journalctl@ceph.mon.b.smithi103.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:45:59.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:59 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:45:59.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:59 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:45:59.525 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:59 smithi103 ceph-mon[33163]: osdmap e71: 8 total, 8 up, 8 in 2024-09-10T18:45:59.525 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:59 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:45:59.525 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:59 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:45:59.525 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:59 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3144021212' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:45:59.525 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:59 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:45:59.525 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:59 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:45:59.525 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:59 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:59.525 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:59 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:59.525 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:59 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:59.525 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:59 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:59.525 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:59 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:59.525 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:59 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:59.525 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:59 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:59.525 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:59 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:59.526 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:45:59 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:45:59.864 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:45:59.891 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:45:59.920 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:46:00.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:00 smithi018 ceph-mon[29336]: Adjusting osd_memory_target on smithi103 to 3760M 2024-09-10T18:46:00.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:00 smithi018 ceph-mon[29336]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:00.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:00 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:00.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:00 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:00.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:00 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:46:00.277 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:00 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:46:00.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:00 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:46:00.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:00 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:46:00.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:00 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:46:00.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:00 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:46:00.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:00 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:46:00.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:00 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:46:00.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:00 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:00.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:00 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:46:00.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:00 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:46:00.278 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:00 smithi018 ceph-mon[29336]: mgrmap e21: y(active, since 4s), standbys: x 2024-09-10T18:46:00.279 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:00 smithi018 ceph-mon[40196]: Adjusting osd_memory_target on smithi103 to 3760M 2024-09-10T18:46:00.279 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:00 smithi018 ceph-mon[40196]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:00.279 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:00 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:00.279 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:00 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:00.279 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:00 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:46:00.279 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:00 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:46:00.279 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:00 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:46:00.279 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:00 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:46:00.279 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:00 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:46:00.280 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:00 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:46:00.280 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:00 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:46:00.280 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:00 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:46:00.280 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:00 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:00.280 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:00 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:46:00.280 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:00 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:46:00.280 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:00 smithi018 ceph-mon[40196]: mgrmap e21: y(active, since 4s), standbys: x 2024-09-10T18:46:00.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:46:00 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:46:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:46:00.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:00 smithi103 ceph-mon[33163]: Adjusting osd_memory_target on smithi103 to 3760M 2024-09-10T18:46:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:00 smithi103 ceph-mon[33163]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:00 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:00 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:00 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:46:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:00 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:46:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:00 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:46:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:00 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:46:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:00 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:46:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:00 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:46:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:00 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:46:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:00 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:46:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:00 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:00.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:00 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:46:00.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:00 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:46:00.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:00 smithi103 ceph-mon[33163]: mgrmap e21: y(active, since 4s), standbys: x 2024-09-10T18:46:01.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:01 smithi018 ceph-mon[29336]: Adjusting osd_memory_target on smithi018 to 3761M 2024-09-10T18:46:01.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:01 smithi018 ceph-mon[29336]: Updating smithi018:/etc/ceph/ceph.conf 2024-09-10T18:46:01.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:01 smithi018 ceph-mon[29336]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-10T18:46:01.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:01 smithi018 ceph-mon[29336]: Updating smithi103:/var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:46:01.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:01 smithi018 ceph-mon[29336]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:46:01.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:01 smithi018 ceph-mon[29336]: Updating smithi018:/var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:46:01.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:01 smithi018 ceph-mon[29336]: Updating smithi103:/var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:46:01.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:01 smithi018 ceph-mon[29336]: Updating smithi018:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:46:01.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:01 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:01.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:01 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:01.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:01 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:01.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:01 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:01.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:01 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:01.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:01 smithi018 ceph-mon[40196]: Adjusting osd_memory_target on smithi018 to 3761M 2024-09-10T18:46:01.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:01 smithi018 ceph-mon[40196]: Updating smithi018:/etc/ceph/ceph.conf 2024-09-10T18:46:01.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:01 smithi018 ceph-mon[40196]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-10T18:46:01.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:01 smithi018 ceph-mon[40196]: Updating smithi103:/var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:46:01.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:01 smithi018 ceph-mon[40196]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:46:01.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:01 smithi018 ceph-mon[40196]: Updating smithi018:/var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:46:01.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:01 smithi018 ceph-mon[40196]: Updating smithi103:/var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:46:01.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:01 smithi018 ceph-mon[40196]: Updating smithi018:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:46:01.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:01 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:01.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:01 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:01.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:01 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:01.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:01 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:01.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:01 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:01.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:01 smithi103 ceph-mon[33163]: Adjusting osd_memory_target on smithi018 to 3761M 2024-09-10T18:46:01.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:01 smithi103 ceph-mon[33163]: Updating smithi018:/etc/ceph/ceph.conf 2024-09-10T18:46:01.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:01 smithi103 ceph-mon[33163]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-10T18:46:01.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:01 smithi103 ceph-mon[33163]: Updating smithi103:/var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:46:01.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:01 smithi103 ceph-mon[33163]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:46:01.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:01 smithi103 ceph-mon[33163]: Updating smithi018:/var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:46:01.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:01 smithi103 ceph-mon[33163]: Updating smithi103:/var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:46:01.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:01 smithi103 ceph-mon[33163]: Updating smithi018:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:46:01.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:01 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:01.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:01 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:01.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:01 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:01.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:01 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:01.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:01 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:01.955 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:46:01.956 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:46:01.956 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:46:01.957 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:46:02.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:02 smithi018 ceph-mon[40196]: Updating smithi018:/var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:46:02.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:02 smithi018 ceph-mon[40196]: Deploying daemon alertmanager.a on smithi018 2024-09-10T18:46:02.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:02 smithi018 ceph-mon[40196]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 35 KiB/s rd, 0 B/s wr, 14 op/s 2024-09-10T18:46:02.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:02 smithi018 ceph-mon[29336]: Updating smithi018:/var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:46:02.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:02 smithi018 ceph-mon[29336]: Deploying daemon alertmanager.a on smithi018 2024-09-10T18:46:02.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:02 smithi018 ceph-mon[29336]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 35 KiB/s rd, 0 B/s wr, 14 op/s 2024-09-10T18:46:02.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:02 smithi103 ceph-mon[33163]: Updating smithi018:/var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:46:02.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:02 smithi103 ceph-mon[33163]: Deploying daemon alertmanager.a on smithi018 2024-09-10T18:46:02.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:02 smithi103 ceph-mon[33163]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 35 KiB/s rd, 0 B/s wr, 14 op/s 2024-09-10T18:46:04.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:04 smithi018 ceph-mon[29336]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T18:46:04.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:04 smithi018 ceph-mon[29336]: osdmap e72: 8 total, 8 up, 8 in 2024-09-10T18:46:04.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:04 smithi018 ceph-mon[40196]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T18:46:04.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:04 smithi018 ceph-mon[40196]: osdmap e72: 8 total, 8 up, 8 in 2024-09-10T18:46:04.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:04 smithi103 ceph-mon[33163]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T18:46:04.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:04 smithi103 ceph-mon[33163]: osdmap e72: 8 total, 8 up, 8 in 2024-09-10T18:46:05.984 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:46:06.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:06 smithi018 ceph-mon[29336]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-10T18:46:06.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:06 smithi018 ceph-mon[40196]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-10T18:46:06.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:06 smithi103 ceph-mon[33163]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-10T18:46:06.523 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:46:06 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:46:07.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:07 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:07.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:07 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:07.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:07 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:08.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:08 smithi018 ceph-mon[29336]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 22 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-10T18:46:08.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:08 smithi018 ceph-mon[40196]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 22 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-10T18:46:08.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:08 smithi103 ceph-mon[33163]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 22 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-10T18:46:09.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:09 smithi018 ceph-mon[29336]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 19 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T18:46:09.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:09 smithi018 ceph-mon[40196]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 19 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T18:46:09.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:09 smithi103 ceph-mon[33163]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 19 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T18:46:10.512 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:10 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:46:10.512 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:10 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:46:10.512 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:10 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:46:10.512 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:10 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:46:10.512 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:10 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:46:10.513 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:10 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:46:10.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:10 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:46:10.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:10 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:46:10.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:10 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:46:10.523 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:46:10 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:46:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:46:11.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:11 smithi103 ceph-mon[33163]: osdmap e73: 8 total, 8 up, 8 in 2024-09-10T18:46:11.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:11 smithi103 ceph-mon[33163]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:46:11.861 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:11 smithi018 ceph-mon[40196]: osdmap e73: 8 total, 8 up, 8 in 2024-09-10T18:46:11.861 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:11 smithi018 ceph-mon[40196]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:46:11.862 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:11 smithi018 ceph-mon[29336]: osdmap e73: 8 total, 8 up, 8 in 2024-09-10T18:46:11.862 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:11 smithi018 ceph-mon[29336]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:46:11.890 INFO:teuthology.orchestra.run.smithi018.stdout:158913789976 2024-09-10T18:46:11.891 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-10T18:46:13.071 INFO:teuthology.orchestra.run.smithi018.stdout:107374182433 2024-09-10T18:46:13.072 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-10T18:46:14.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:14 smithi018 ceph-mon[29336]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T18:46:14.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:14 smithi018 ceph-mon[29336]: osdmap e74: 8 total, 8 up, 8 in 2024-09-10T18:46:14.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:14 smithi018 ceph-mon[40196]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T18:46:14.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:14 smithi018 ceph-mon[40196]: osdmap e74: 8 total, 8 up, 8 in 2024-09-10T18:46:14.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:14 smithi103 ceph-mon[33163]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T18:46:14.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:14 smithi103 ceph-mon[33163]: osdmap e74: 8 total, 8 up, 8 in 2024-09-10T18:46:15.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:15 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:46:15.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:15 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:46:15.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:15 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:46:15.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:15 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:46:15.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:15 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:46:15.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:15 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:46:15.526 INFO:teuthology.orchestra.run.smithi018.stdout:184683593748 2024-09-10T18:46:15.526 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-10T18:46:15.907 INFO:teuthology.orchestra.run.smithi018.stdout:55834574890 2024-09-10T18:46:15.908 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-10T18:46:16.397 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:16 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:46:16.397 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:16 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:46:16.398 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:16 smithi018 ceph-mon[29336]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-10T18:46:16.398 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:16 smithi018 ceph-mon[29336]: osdmap e75: 8 total, 8 up, 8 in 2024-09-10T18:46:16.398 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:16 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:46:16.398 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:16 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:46:16.399 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:16 smithi018 ceph-mon[40196]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-10T18:46:16.399 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:16 smithi018 ceph-mon[40196]: osdmap e75: 8 total, 8 up, 8 in 2024-09-10T18:46:16.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:46:16 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:46:16.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:16 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:46:16.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:16 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:46:16.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:16 smithi103 ceph-mon[33163]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-10T18:46:16.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:16 smithi103 ceph-mon[33163]: osdmap e75: 8 total, 8 up, 8 in 2024-09-10T18:46:17.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:17 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:17.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:17 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:17.493 INFO:teuthology.orchestra.run.smithi018.stdout:223338299408 2024-09-10T18:46:17.493 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-10T18:46:17.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:17 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:18.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:18 smithi018 ceph-mon[29336]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-10T18:46:18.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:18 smithi018 ceph-mon[29336]: 2.e deep-scrub starts 2024-09-10T18:46:18.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:18 smithi018 ceph-mon[29336]: 2.e deep-scrub ok 2024-09-10T18:46:18.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:18 smithi018 ceph-mon[40196]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-10T18:46:18.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:18 smithi018 ceph-mon[40196]: 2.e deep-scrub starts 2024-09-10T18:46:18.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:18 smithi018 ceph-mon[40196]: 2.e deep-scrub ok 2024-09-10T18:46:18.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:18 smithi103 ceph-mon[33163]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-10T18:46:18.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:18 smithi103 ceph-mon[33163]: 2.e deep-scrub starts 2024-09-10T18:46:18.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:18 smithi103 ceph-mon[33163]: 2.e deep-scrub ok 2024-09-10T18:46:19.246 INFO:teuthology.orchestra.run.smithi018.stdout:128849018911 2024-09-10T18:46:19.246 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-10T18:46:19.484 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:46:19.485 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:46:19.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:19 smithi103 ceph-mon[33163]: osdmap e76: 8 total, 8 up, 8 in 2024-09-10T18:46:19.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:19 smithi103 ceph-mon[33163]: 2.1e scrub starts 2024-09-10T18:46:19.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:19 smithi103 ceph-mon[33163]: 2.1e scrub ok 2024-09-10T18:46:19.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:19 smithi103 ceph-mon[33163]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:46:19.819 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:19 smithi018 ceph-mon[29336]: osdmap e76: 8 total, 8 up, 8 in 2024-09-10T18:46:19.820 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:19 smithi018 ceph-mon[29336]: 2.1e scrub starts 2024-09-10T18:46:19.820 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:19 smithi018 ceph-mon[29336]: 2.1e scrub ok 2024-09-10T18:46:19.820 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:19 smithi018 ceph-mon[29336]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:46:19.820 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:19 smithi018 ceph-mon[40196]: osdmap e76: 8 total, 8 up, 8 in 2024-09-10T18:46:19.820 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:19 smithi018 ceph-mon[40196]: 2.1e scrub starts 2024-09-10T18:46:19.820 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:19 smithi018 ceph-mon[40196]: 2.1e scrub ok 2024-09-10T18:46:19.820 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:19 smithi018 ceph-mon[40196]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:46:19.857 INFO:teuthology.orchestra.run.smithi018.stdout:34359738418 2024-09-10T18:46:19.857 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-10T18:46:20.516 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:20 smithi018 ceph-mon[40196]: 4.4 scrub starts 2024-09-10T18:46:20.516 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:20 smithi018 ceph-mon[40196]: 4.4 scrub ok 2024-09-10T18:46:20.516 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:20 smithi018 ceph-mon[29336]: 4.4 scrub starts 2024-09-10T18:46:20.516 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:20 smithi018 ceph-mon[29336]: 4.4 scrub ok 2024-09-10T18:46:20.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:20 smithi103 ceph-mon[33163]: 4.4 scrub starts 2024-09-10T18:46:20.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:20 smithi103 ceph-mon[33163]: 4.4 scrub ok 2024-09-10T18:46:20.523 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:46:20 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:46:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:46:21.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:21 smithi103 ceph-mon[33163]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:46:21.922 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:21 smithi018 ceph-mon[29336]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:46:21.922 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:21 smithi018 ceph-mon[40196]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:46:22.694 INFO:teuthology.orchestra.run.smithi018.stdout:77309411368 2024-09-10T18:46:22.695 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-10T18:46:22.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:22 smithi018 ceph-mon[29336]: 4.5 scrub starts 2024-09-10T18:46:22.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:22 smithi018 ceph-mon[29336]: 4.5 scrub ok 2024-09-10T18:46:22.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:22 smithi018 ceph-mon[29336]: 5.1a scrub starts 2024-09-10T18:46:22.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:22 smithi018 ceph-mon[29336]: 5.1a scrub ok 2024-09-10T18:46:22.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:22 smithi018 ceph-mon[40196]: 4.5 scrub starts 2024-09-10T18:46:22.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:22 smithi018 ceph-mon[40196]: 4.5 scrub ok 2024-09-10T18:46:22.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:22 smithi018 ceph-mon[40196]: 5.1a scrub starts 2024-09-10T18:46:22.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:22 smithi018 ceph-mon[40196]: 5.1a scrub ok 2024-09-10T18:46:23.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:22 smithi103 ceph-mon[33163]: 4.5 scrub starts 2024-09-10T18:46:23.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:22 smithi103 ceph-mon[33163]: 4.5 scrub ok 2024-09-10T18:46:23.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:22 smithi103 ceph-mon[33163]: 5.1a scrub starts 2024-09-10T18:46:23.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:22 smithi103 ceph-mon[33163]: 5.1a scrub ok 2024-09-10T18:46:23.690 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:46:23.690 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:46:23.690 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:46:23.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:23 smithi103 ceph-mon[33163]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:46:23.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:23 smithi103 ceph-mon[33163]: 2.14 scrub starts 2024-09-10T18:46:23.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:23 smithi103 ceph-mon[33163]: 2.14 scrub ok 2024-09-10T18:46:23.916 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:23 smithi018 ceph-mon[40196]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:46:23.916 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:23 smithi018 ceph-mon[40196]: 2.14 scrub starts 2024-09-10T18:46:23.916 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:23 smithi018 ceph-mon[40196]: 2.14 scrub ok 2024-09-10T18:46:23.916 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:23 smithi018 ceph-mon[29336]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:46:23.917 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:23 smithi018 ceph-mon[29336]: 2.14 scrub starts 2024-09-10T18:46:23.917 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:23 smithi018 ceph-mon[29336]: 2.14 scrub ok 2024-09-10T18:46:24.190 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:46:24 smithi018 systemd[1]: Starting Ceph alertmanager.a for ba327f6c-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:46:24.569 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:46:24.569 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:46:24.576 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:46:24.785 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:24 smithi018 ceph-mon[40196]: 4.15 scrub starts 2024-09-10T18:46:24.785 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:24 smithi018 ceph-mon[40196]: 4.15 scrub ok 2024-09-10T18:46:24.785 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:24 smithi018 ceph-mon[40196]: 4.1c scrub starts 2024-09-10T18:46:24.785 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:24 smithi018 ceph-mon[40196]: 4.1c scrub ok 2024-09-10T18:46:24.785 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:24 smithi018 ceph-mon[29336]: 4.15 scrub starts 2024-09-10T18:46:24.785 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:24 smithi018 ceph-mon[29336]: 4.15 scrub ok 2024-09-10T18:46:24.786 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:24 smithi018 ceph-mon[29336]: 4.1c scrub starts 2024-09-10T18:46:24.786 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:24 smithi018 ceph-mon[29336]: 4.1c scrub ok 2024-09-10T18:46:25.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:24 smithi103 ceph-mon[33163]: 4.15 scrub starts 2024-09-10T18:46:25.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:24 smithi103 ceph-mon[33163]: 4.15 scrub ok 2024-09-10T18:46:25.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:24 smithi103 ceph-mon[33163]: 4.1c scrub starts 2024-09-10T18:46:25.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:24 smithi103 ceph-mon[33163]: 4.1c scrub ok 2024-09-10T18:46:26.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:25 smithi103 ceph-mon[33163]: 4.16 deep-scrub starts 2024-09-10T18:46:26.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:25 smithi103 ceph-mon[33163]: 4.16 deep-scrub ok 2024-09-10T18:46:26.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:25 smithi103 ceph-mon[33163]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T18:46:26.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:25 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:46:26.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:25 smithi103 ceph-mon[33163]: 2.7 deep-scrub starts 2024-09-10T18:46:26.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:25 smithi103 ceph-mon[33163]: 2.7 deep-scrub ok 2024-09-10T18:46:26.147 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:25 smithi018 ceph-mon[40196]: 4.16 deep-scrub starts 2024-09-10T18:46:26.147 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:25 smithi018 ceph-mon[40196]: 4.16 deep-scrub ok 2024-09-10T18:46:26.147 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:25 smithi018 ceph-mon[40196]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T18:46:26.147 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:25 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:46:26.147 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:25 smithi018 ceph-mon[40196]: 2.7 deep-scrub starts 2024-09-10T18:46:26.147 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:25 smithi018 ceph-mon[40196]: 2.7 deep-scrub ok 2024-09-10T18:46:26.147 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:25 smithi018 ceph-mon[29336]: 4.16 deep-scrub starts 2024-09-10T18:46:26.147 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:25 smithi018 ceph-mon[29336]: 4.16 deep-scrub ok 2024-09-10T18:46:26.148 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:25 smithi018 ceph-mon[29336]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T18:46:26.148 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:25 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:46:26.148 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:25 smithi018 ceph-mon[29336]: 2.7 deep-scrub starts 2024-09-10T18:46:26.148 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:25 smithi018 ceph-mon[29336]: 2.7 deep-scrub ok 2024-09-10T18:46:26.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:46:26 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:46:27.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:26 smithi103 ceph-mon[33163]: 5.d scrub starts 2024-09-10T18:46:27.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:26 smithi103 ceph-mon[33163]: 5.d scrub ok 2024-09-10T18:46:27.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:26 smithi103 ceph-mon[33163]: 3.2 scrub starts 2024-09-10T18:46:27.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:26 smithi103 ceph-mon[33163]: 3.2 scrub ok 2024-09-10T18:46:27.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:26 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:27.052 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:26 smithi018 ceph-mon[29336]: 5.d scrub starts 2024-09-10T18:46:27.052 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:26 smithi018 ceph-mon[29336]: 5.d scrub ok 2024-09-10T18:46:27.052 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:26 smithi018 ceph-mon[29336]: 3.2 scrub starts 2024-09-10T18:46:27.052 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:26 smithi018 ceph-mon[29336]: 3.2 scrub ok 2024-09-10T18:46:27.052 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:26 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:27.053 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:26 smithi018 ceph-mon[40196]: 5.d scrub starts 2024-09-10T18:46:27.053 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:26 smithi018 ceph-mon[40196]: 5.d scrub ok 2024-09-10T18:46:27.053 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:26 smithi018 ceph-mon[40196]: 3.2 scrub starts 2024-09-10T18:46:27.053 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:26 smithi018 ceph-mon[40196]: 3.2 scrub ok 2024-09-10T18:46:27.053 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:26 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:27.784 INFO:teuthology.orchestra.run.smithi018.stdout:158913789978 2024-09-10T18:46:27.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:27 smithi018 ceph-mon[29336]: 5.4 scrub starts 2024-09-10T18:46:27.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:27 smithi018 ceph-mon[29336]: 5.4 scrub ok 2024-09-10T18:46:27.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:27 smithi018 ceph-mon[29336]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:46:27.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:27 smithi018 ceph-mon[29336]: 5.10 scrub starts 2024-09-10T18:46:27.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:27 smithi018 ceph-mon[29336]: 5.10 scrub ok 2024-09-10T18:46:27.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:27 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/4089501258' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T18:46:27.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:27 smithi018 ceph-mon[40196]: 5.4 scrub starts 2024-09-10T18:46:27.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:27 smithi018 ceph-mon[40196]: 5.4 scrub ok 2024-09-10T18:46:27.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:27 smithi018 ceph-mon[40196]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:46:27.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:27 smithi018 ceph-mon[40196]: 5.10 scrub starts 2024-09-10T18:46:27.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:27 smithi018 ceph-mon[40196]: 5.10 scrub ok 2024-09-10T18:46:27.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:27 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/4089501258' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T18:46:28.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:27 smithi103 ceph-mon[33163]: 5.4 scrub starts 2024-09-10T18:46:28.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:27 smithi103 ceph-mon[33163]: 5.4 scrub ok 2024-09-10T18:46:28.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:27 smithi103 ceph-mon[33163]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:46:28.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:27 smithi103 ceph-mon[33163]: 5.10 scrub starts 2024-09-10T18:46:28.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:27 smithi103 ceph-mon[33163]: 5.10 scrub ok 2024-09-10T18:46:28.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:27 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/4089501258' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T18:46:28.785 INFO:teuthology.orchestra.run.smithi018.stdout:107374182438 2024-09-10T18:46:29.007 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:28 smithi018 ceph-mon[40196]: 4.0 scrub starts 2024-09-10T18:46:29.007 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:28 smithi018 ceph-mon[40196]: 4.0 scrub ok 2024-09-10T18:46:29.007 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:28 smithi018 ceph-mon[40196]: 4.a scrub starts 2024-09-10T18:46:29.007 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:28 smithi018 ceph-mon[40196]: 4.a scrub ok 2024-09-10T18:46:29.007 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:28 smithi018 ceph-mon[40196]: 2.9 deep-scrub starts 2024-09-10T18:46:29.007 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:28 smithi018 ceph-mon[40196]: 2.9 deep-scrub ok 2024-09-10T18:46:29.008 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:28 smithi018 ceph-mon[29336]: 4.0 scrub starts 2024-09-10T18:46:29.008 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:28 smithi018 ceph-mon[29336]: 4.0 scrub ok 2024-09-10T18:46:29.008 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:28 smithi018 ceph-mon[29336]: 4.a scrub starts 2024-09-10T18:46:29.008 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:28 smithi018 ceph-mon[29336]: 4.a scrub ok 2024-09-10T18:46:29.008 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:28 smithi018 ceph-mon[29336]: 2.9 deep-scrub starts 2024-09-10T18:46:29.008 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:28 smithi018 ceph-mon[29336]: 2.9 deep-scrub ok 2024-09-10T18:46:29.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:28 smithi103 ceph-mon[33163]: 4.0 scrub starts 2024-09-10T18:46:29.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:28 smithi103 ceph-mon[33163]: 4.0 scrub ok 2024-09-10T18:46:29.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:28 smithi103 ceph-mon[33163]: 4.a scrub starts 2024-09-10T18:46:29.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:28 smithi103 ceph-mon[33163]: 4.a scrub ok 2024-09-10T18:46:29.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:28 smithi103 ceph-mon[33163]: 2.9 deep-scrub starts 2024-09-10T18:46:29.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:28 smithi103 ceph-mon[33163]: 2.9 deep-scrub ok 2024-09-10T18:46:29.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:29 smithi018 ceph-mon[29336]: 2.18 scrub starts 2024-09-10T18:46:29.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:29 smithi018 ceph-mon[29336]: 2.18 scrub ok 2024-09-10T18:46:29.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:29 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1415865296' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T18:46:29.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:29 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1387970960' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T18:46:29.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:29 smithi018 ceph-mon[29336]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 970 B/s rd, 0 op/s 2024-09-10T18:46:29.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:29 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:46:29.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:29 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:46:29.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:29 smithi018 ceph-mon[40196]: 2.18 scrub starts 2024-09-10T18:46:29.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:29 smithi018 ceph-mon[40196]: 2.18 scrub ok 2024-09-10T18:46:29.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:29 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1415865296' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T18:46:29.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:29 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1387970960' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T18:46:29.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:29 smithi018 ceph-mon[40196]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 970 B/s rd, 0 op/s 2024-09-10T18:46:29.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:29 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:46:29.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:29 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:46:30.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:29 smithi103 ceph-mon[33163]: 2.18 scrub starts 2024-09-10T18:46:30.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:29 smithi103 ceph-mon[33163]: 2.18 scrub ok 2024-09-10T18:46:30.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:29 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1415865296' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T18:46:30.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:29 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1387970960' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T18:46:30.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:29 smithi103 ceph-mon[33163]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 970 B/s rd, 0 op/s 2024-09-10T18:46:30.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:29 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:46:30.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:29 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:46:30.432 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913789976 got 158913789978 for osd.5 2024-09-10T18:46:30.432 DEBUG:teuthology.parallel:result is None 2024-09-10T18:46:30.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:46:30 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:46:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:46:31.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:30 smithi103 ceph-mon[33163]: 2.3 scrub starts 2024-09-10T18:46:31.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:30 smithi103 ceph-mon[33163]: 2.3 scrub ok 2024-09-10T18:46:31.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:30 smithi103 ceph-mon[33163]: 4.1 scrub starts 2024-09-10T18:46:31.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:30 smithi103 ceph-mon[33163]: 4.1 scrub ok 2024-09-10T18:46:31.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:30 smithi103 ceph-mon[33163]: osdmap e77: 8 total, 8 up, 8 in 2024-09-10T18:46:31.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:30 smithi103 ceph-mon[33163]: 6.13 scrub starts 2024-09-10T18:46:31.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:30 smithi103 ceph-mon[33163]: 6.13 scrub ok 2024-09-10T18:46:31.110 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:30 smithi018 ceph-mon[40196]: 2.3 scrub starts 2024-09-10T18:46:31.110 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:30 smithi018 ceph-mon[40196]: 2.3 scrub ok 2024-09-10T18:46:31.110 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:30 smithi018 ceph-mon[40196]: 4.1 scrub starts 2024-09-10T18:46:31.110 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:30 smithi018 ceph-mon[40196]: 4.1 scrub ok 2024-09-10T18:46:31.110 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:30 smithi018 ceph-mon[40196]: osdmap e77: 8 total, 8 up, 8 in 2024-09-10T18:46:31.110 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:30 smithi018 ceph-mon[40196]: 6.13 scrub starts 2024-09-10T18:46:31.110 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:30 smithi018 ceph-mon[40196]: 6.13 scrub ok 2024-09-10T18:46:31.110 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:30 smithi018 ceph-mon[29336]: 2.3 scrub starts 2024-09-10T18:46:31.111 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:30 smithi018 ceph-mon[29336]: 2.3 scrub ok 2024-09-10T18:46:31.111 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:30 smithi018 ceph-mon[29336]: 4.1 scrub starts 2024-09-10T18:46:31.111 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:30 smithi018 ceph-mon[29336]: 4.1 scrub ok 2024-09-10T18:46:31.111 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:30 smithi018 ceph-mon[29336]: osdmap e77: 8 total, 8 up, 8 in 2024-09-10T18:46:31.111 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:30 smithi018 ceph-mon[29336]: 6.13 scrub starts 2024-09-10T18:46:31.111 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:30 smithi018 ceph-mon[29336]: 6.13 scrub ok 2024-09-10T18:46:31.141 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182433 got 107374182438 for osd.3 2024-09-10T18:46:31.141 DEBUG:teuthology.parallel:result is None 2024-09-10T18:46:31.874 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:31 smithi018 ceph-mon[29336]: 4.9 scrub starts 2024-09-10T18:46:31.875 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:31 smithi018 ceph-mon[29336]: 4.9 scrub ok 2024-09-10T18:46:31.875 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:31 smithi018 ceph-mon[29336]: 5.b scrub starts 2024-09-10T18:46:31.875 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:31 smithi018 ceph-mon[29336]: 5.b scrub ok 2024-09-10T18:46:31.875 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:31 smithi018 ceph-mon[29336]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:46:31.875 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:31 smithi018 ceph-mon[40196]: 4.9 scrub starts 2024-09-10T18:46:31.875 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:31 smithi018 ceph-mon[40196]: 4.9 scrub ok 2024-09-10T18:46:31.875 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:31 smithi018 ceph-mon[40196]: 5.b scrub starts 2024-09-10T18:46:31.875 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:31 smithi018 ceph-mon[40196]: 5.b scrub ok 2024-09-10T18:46:31.875 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:31 smithi018 ceph-mon[40196]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:46:32.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:31 smithi103 ceph-mon[33163]: 4.9 scrub starts 2024-09-10T18:46:32.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:31 smithi103 ceph-mon[33163]: 4.9 scrub ok 2024-09-10T18:46:32.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:31 smithi103 ceph-mon[33163]: 5.b scrub starts 2024-09-10T18:46:32.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:31 smithi103 ceph-mon[33163]: 5.b scrub ok 2024-09-10T18:46:32.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:31 smithi103 ceph-mon[33163]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:46:32.891 INFO:teuthology.orchestra.run.smithi018.stdout:55834574894 2024-09-10T18:46:33.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:32 smithi103 ceph-mon[33163]: 5.1b scrub starts 2024-09-10T18:46:33.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:32 smithi103 ceph-mon[33163]: 5.1b scrub ok 2024-09-10T18:46:33.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:32 smithi103 ceph-mon[33163]: 4.c scrub starts 2024-09-10T18:46:33.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:32 smithi103 ceph-mon[33163]: 4.c scrub ok 2024-09-10T18:46:33.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:32 smithi103 ceph-mon[33163]: 2.10 scrub starts 2024-09-10T18:46:33.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:32 smithi103 ceph-mon[33163]: 2.10 scrub ok 2024-09-10T18:46:33.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:32 smithi103 ceph-mon[33163]: 6.3 scrub starts 2024-09-10T18:46:33.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:32 smithi103 ceph-mon[33163]: 6.3 scrub ok 2024-09-10T18:46:33.170 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:32 smithi018 ceph-mon[40196]: 5.1b scrub starts 2024-09-10T18:46:33.171 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:32 smithi018 ceph-mon[40196]: 5.1b scrub ok 2024-09-10T18:46:33.171 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:32 smithi018 ceph-mon[40196]: 4.c scrub starts 2024-09-10T18:46:33.171 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:32 smithi018 ceph-mon[40196]: 4.c scrub ok 2024-09-10T18:46:33.171 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:32 smithi018 ceph-mon[40196]: 2.10 scrub starts 2024-09-10T18:46:33.171 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:32 smithi018 ceph-mon[40196]: 2.10 scrub ok 2024-09-10T18:46:33.171 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:32 smithi018 ceph-mon[40196]: 6.3 scrub starts 2024-09-10T18:46:33.171 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:32 smithi018 ceph-mon[40196]: 6.3 scrub ok 2024-09-10T18:46:33.171 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:32 smithi018 ceph-mon[29336]: 5.1b scrub starts 2024-09-10T18:46:33.171 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:32 smithi018 ceph-mon[29336]: 5.1b scrub ok 2024-09-10T18:46:33.171 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:32 smithi018 ceph-mon[29336]: 4.c scrub starts 2024-09-10T18:46:33.171 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:32 smithi018 ceph-mon[29336]: 4.c scrub ok 2024-09-10T18:46:33.171 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:32 smithi018 ceph-mon[29336]: 2.10 scrub starts 2024-09-10T18:46:33.172 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:32 smithi018 ceph-mon[29336]: 2.10 scrub ok 2024-09-10T18:46:33.172 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:32 smithi018 ceph-mon[29336]: 6.3 scrub starts 2024-09-10T18:46:33.172 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:32 smithi018 ceph-mon[29336]: 6.3 scrub ok 2024-09-10T18:46:33.476 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574890 got 55834574894 for osd.1 2024-09-10T18:46:33.477 DEBUG:teuthology.parallel:result is None 2024-09-10T18:46:33.755 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:33 smithi018 ceph-mon[29336]: 6.e scrub starts 2024-09-10T18:46:33.755 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:33 smithi018 ceph-mon[29336]: 6.e scrub ok 2024-09-10T18:46:33.755 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:33 smithi018 ceph-mon[29336]: 5.17 scrub starts 2024-09-10T18:46:33.755 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:33 smithi018 ceph-mon[29336]: 5.17 scrub ok 2024-09-10T18:46:33.756 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:33 smithi018 ceph-mon[29336]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:46:33.756 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:33 smithi018 ceph-mon[29336]: 5.1f scrub starts 2024-09-10T18:46:33.756 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:33 smithi018 ceph-mon[29336]: 5.1f scrub ok 2024-09-10T18:46:33.756 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:33 smithi018 ceph-mon[29336]: osdmap e78: 8 total, 8 up, 8 in 2024-09-10T18:46:34.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:33 smithi103 ceph-mon[33163]: 6.e scrub starts 2024-09-10T18:46:34.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:33 smithi103 ceph-mon[33163]: 6.e scrub ok 2024-09-10T18:46:34.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:33 smithi103 ceph-mon[33163]: 5.17 scrub starts 2024-09-10T18:46:34.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:33 smithi103 ceph-mon[33163]: 5.17 scrub ok 2024-09-10T18:46:34.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:33 smithi103 ceph-mon[33163]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:46:34.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:33 smithi103 ceph-mon[33163]: 5.1f scrub starts 2024-09-10T18:46:34.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:33 smithi103 ceph-mon[33163]: 5.1f scrub ok 2024-09-10T18:46:34.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:33 smithi103 ceph-mon[33163]: osdmap e78: 8 total, 8 up, 8 in 2024-09-10T18:46:34.179 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:33 smithi018 ceph-mon[40196]: 6.e scrub starts 2024-09-10T18:46:34.179 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:33 smithi018 ceph-mon[40196]: 6.e scrub ok 2024-09-10T18:46:34.179 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:33 smithi018 ceph-mon[40196]: 5.17 scrub starts 2024-09-10T18:46:34.179 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:33 smithi018 ceph-mon[40196]: 5.17 scrub ok 2024-09-10T18:46:34.180 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:33 smithi018 ceph-mon[40196]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:46:34.180 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:33 smithi018 ceph-mon[40196]: 5.1f scrub starts 2024-09-10T18:46:34.180 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:33 smithi018 ceph-mon[40196]: 5.1f scrub ok 2024-09-10T18:46:34.180 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:33 smithi018 ceph-mon[40196]: osdmap e78: 8 total, 8 up, 8 in 2024-09-10T18:46:34.694 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:46:34 smithi018 podman[72223]: 2024-09-10 18:46:34.33782188 +0000 UTC m=+0.105341950 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-09-10T18:46:34.956 INFO:teuthology.orchestra.run.smithi018.stdout:184683593752 2024-09-10T18:46:34.957 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:46:34 smithi018 podman[72223]: 2024-09-10 18:46:34.853757061 +0000 UTC m=+0.621277137 volume create d79f05032aa5bfdafa52a969a8cd94519391fcf35464f9f93bd693b7ca57395a 2024-09-10T18:46:34.957 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:34 smithi018 ceph-mon[40196]: 6.1b scrub starts 2024-09-10T18:46:34.957 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:34 smithi018 ceph-mon[40196]: 6.1b scrub ok 2024-09-10T18:46:34.957 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:34 smithi018 ceph-mon[29336]: 6.1b scrub starts 2024-09-10T18:46:34.957 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:34 smithi018 ceph-mon[29336]: 6.1b scrub ok 2024-09-10T18:46:35.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:34 smithi103 ceph-mon[33163]: 6.1b scrub starts 2024-09-10T18:46:35.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:34 smithi103 ceph-mon[33163]: 6.1b scrub ok 2024-09-10T18:46:35.209 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:46:34 smithi018 podman[72223]: 2024-09-10 18:46:34.970908011 +0000 UTC m=+0.738428124 container create 593b72795192240c8e5e3e0a60106d943380cfb92f8aff77cd5b5cb0633e29ff (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:46:35.789 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:35 smithi018 ceph-mon[40196]: 5.15 scrub starts 2024-09-10T18:46:35.789 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:35 smithi018 ceph-mon[40196]: 5.15 scrub ok 2024-09-10T18:46:35.789 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:35 smithi018 ceph-mon[40196]: 5.1c scrub starts 2024-09-10T18:46:35.789 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:35 smithi018 ceph-mon[40196]: 5.1c scrub ok 2024-09-10T18:46:35.789 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:35 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1350653446' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T18:46:35.790 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:35 smithi018 ceph-mon[40196]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:46:35.790 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:35 smithi018 ceph-mon[40196]: 5.11 deep-scrub starts 2024-09-10T18:46:35.790 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:35 smithi018 ceph-mon[40196]: 5.11 deep-scrub ok 2024-09-10T18:46:35.790 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:35 smithi018 ceph-mon[29336]: 5.15 scrub starts 2024-09-10T18:46:35.790 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:35 smithi018 ceph-mon[29336]: 5.15 scrub ok 2024-09-10T18:46:35.790 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:35 smithi018 ceph-mon[29336]: 5.1c scrub starts 2024-09-10T18:46:35.790 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:35 smithi018 ceph-mon[29336]: 5.1c scrub ok 2024-09-10T18:46:35.790 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:35 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1350653446' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T18:46:35.790 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:35 smithi018 ceph-mon[29336]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:46:35.791 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:35 smithi018 ceph-mon[29336]: 5.11 deep-scrub starts 2024-09-10T18:46:35.791 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:35 smithi018 ceph-mon[29336]: 5.11 deep-scrub ok 2024-09-10T18:46:35.791 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:46:35 smithi018 podman[72223]: 2024-09-10 18:46:35.744128883 +0000 UTC m=+1.511648953 container init 593b72795192240c8e5e3e0a60106d943380cfb92f8aff77cd5b5cb0633e29ff (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:46:35.791 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:46:35 smithi018 podman[72223]: 2024-09-10 18:46:35.747861263 +0000 UTC m=+1.515381333 container start 593b72795192240c8e5e3e0a60106d943380cfb92f8aff77cd5b5cb0633e29ff (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:46:35.791 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:46:35 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a[72294]: ts=2024-09-10T18:46:35.762Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-10T18:46:35.791 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:46:35 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a[72294]: ts=2024-09-10T18:46:35.762Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-10T18:46:35.791 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:46:35 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a[72294]: ts=2024-09-10T18:46:35.763Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.18 port=9094 2024-09-10T18:46:35.791 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:46:35 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a[72294]: ts=2024-09-10T18:46:35.764Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-10T18:46:35.791 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:46:35 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a[72294]: ts=2024-09-10T18:46:35.788Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-10T18:46:36.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:35 smithi103 ceph-mon[33163]: 5.15 scrub starts 2024-09-10T18:46:36.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:35 smithi103 ceph-mon[33163]: 5.15 scrub ok 2024-09-10T18:46:36.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:35 smithi103 ceph-mon[33163]: 5.1c scrub starts 2024-09-10T18:46:36.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:35 smithi103 ceph-mon[33163]: 5.1c scrub ok 2024-09-10T18:46:36.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:35 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1350653446' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T18:46:36.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:35 smithi103 ceph-mon[33163]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:46:36.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:35 smithi103 ceph-mon[33163]: 5.11 deep-scrub starts 2024-09-10T18:46:36.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:35 smithi103 ceph-mon[33163]: 5.11 deep-scrub ok 2024-09-10T18:46:36.108 INFO:teuthology.orchestra.run.smithi018.stdout:128849018914 2024-09-10T18:46:36.109 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:46:35 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a[72294]: ts=2024-09-10T18:46:35.788Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-10T18:46:36.109 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:46:35 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a[72294]: ts=2024-09-10T18:46:35.791Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-10T18:46:36.109 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:46:35 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a[72294]: ts=2024-09-10T18:46:35.791Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-10T18:46:36.284 INFO:teuthology.orchestra.run.smithi018.stdout:77309411371 2024-09-10T18:46:36.523 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:46:36 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:46:36.894 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:36 smithi018 ceph-mon[29336]: 4.e scrub starts 2024-09-10T18:46:36.894 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:36 smithi018 ceph-mon[29336]: 4.e scrub ok 2024-09-10T18:46:36.894 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:36 smithi018 ceph-mon[29336]: 6.f scrub starts 2024-09-10T18:46:36.894 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:36 smithi018 ceph-mon[29336]: 6.f scrub ok 2024-09-10T18:46:36.894 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:36 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:36.894 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:36 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/215020737' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T18:46:36.894 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:36 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/48206998' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T18:46:36.894 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:36 smithi018 ceph-mon[29336]: 2.15 scrub starts 2024-09-10T18:46:36.894 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:36 smithi018 ceph-mon[29336]: 2.15 scrub ok 2024-09-10T18:46:36.895 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:36 smithi018 ceph-mon[40196]: 4.e scrub starts 2024-09-10T18:46:36.895 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:36 smithi018 ceph-mon[40196]: 4.e scrub ok 2024-09-10T18:46:36.895 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:36 smithi018 ceph-mon[40196]: 6.f scrub starts 2024-09-10T18:46:36.895 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:36 smithi018 ceph-mon[40196]: 6.f scrub ok 2024-09-10T18:46:36.895 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:36 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:36.895 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:36 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/215020737' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T18:46:36.895 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:36 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/48206998' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T18:46:36.895 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:36 smithi018 ceph-mon[40196]: 2.15 scrub starts 2024-09-10T18:46:36.895 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:36 smithi018 ceph-mon[40196]: 2.15 scrub ok 2024-09-10T18:46:36.921 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593748 got 184683593752 for osd.6 2024-09-10T18:46:36.921 DEBUG:teuthology.parallel:result is None 2024-09-10T18:46:37.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:36 smithi103 ceph-mon[33163]: 4.e scrub starts 2024-09-10T18:46:37.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:36 smithi103 ceph-mon[33163]: 4.e scrub ok 2024-09-10T18:46:37.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:36 smithi103 ceph-mon[33163]: 6.f scrub starts 2024-09-10T18:46:37.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:36 smithi103 ceph-mon[33163]: 6.f scrub ok 2024-09-10T18:46:37.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:36 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:37.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:36 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/215020737' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T18:46:37.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:36 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/48206998' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T18:46:37.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:36 smithi103 ceph-mon[33163]: 2.15 scrub starts 2024-09-10T18:46:37.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:36 smithi103 ceph-mon[33163]: 2.15 scrub ok 2024-09-10T18:46:37.743 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411368 got 77309411371 for osd.2 2024-09-10T18:46:37.743 DEBUG:teuthology.parallel:result is None 2024-09-10T18:46:37.993 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:46:37 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a[72294]: ts=2024-09-10T18:46:37.765Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000561024s 2024-09-10T18:46:37.993 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:37 smithi018 ceph-mon[40196]: 6.19 scrub starts 2024-09-10T18:46:37.993 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:37 smithi018 ceph-mon[40196]: 6.19 scrub ok 2024-09-10T18:46:37.993 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:37 smithi018 ceph-mon[40196]: 6.17 deep-scrub starts 2024-09-10T18:46:37.994 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:37 smithi018 ceph-mon[40196]: 6.17 deep-scrub ok 2024-09-10T18:46:37.994 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:37 smithi018 ceph-mon[40196]: 4.19 scrub starts 2024-09-10T18:46:37.994 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:37 smithi018 ceph-mon[40196]: 4.19 scrub ok 2024-09-10T18:46:37.994 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:37 smithi018 ceph-mon[40196]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:46:37.994 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:37 smithi018 ceph-mon[40196]: 5.12 scrub starts 2024-09-10T18:46:37.994 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:37 smithi018 ceph-mon[40196]: 5.12 scrub ok 2024-09-10T18:46:37.994 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:37 smithi018 ceph-mon[40196]: 6.1c deep-scrub starts 2024-09-10T18:46:37.994 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:37 smithi018 ceph-mon[40196]: 6.1c deep-scrub ok 2024-09-10T18:46:37.994 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:37 smithi018 ceph-mon[29336]: 6.19 scrub starts 2024-09-10T18:46:37.994 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:37 smithi018 ceph-mon[29336]: 6.19 scrub ok 2024-09-10T18:46:37.994 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:37 smithi018 ceph-mon[29336]: 6.17 deep-scrub starts 2024-09-10T18:46:37.995 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:37 smithi018 ceph-mon[29336]: 6.17 deep-scrub ok 2024-09-10T18:46:37.995 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:37 smithi018 ceph-mon[29336]: 4.19 scrub starts 2024-09-10T18:46:37.995 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:37 smithi018 ceph-mon[29336]: 4.19 scrub ok 2024-09-10T18:46:37.995 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:37 smithi018 ceph-mon[29336]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:46:37.995 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:37 smithi018 ceph-mon[29336]: 5.12 scrub starts 2024-09-10T18:46:37.995 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:37 smithi018 ceph-mon[29336]: 5.12 scrub ok 2024-09-10T18:46:37.995 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:37 smithi018 ceph-mon[29336]: 6.1c deep-scrub starts 2024-09-10T18:46:37.995 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:37 smithi018 ceph-mon[29336]: 6.1c deep-scrub ok 2024-09-10T18:46:38.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:37 smithi103 ceph-mon[33163]: 6.19 scrub starts 2024-09-10T18:46:38.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:37 smithi103 ceph-mon[33163]: 6.19 scrub ok 2024-09-10T18:46:38.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:37 smithi103 ceph-mon[33163]: 6.17 deep-scrub starts 2024-09-10T18:46:38.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:37 smithi103 ceph-mon[33163]: 6.17 deep-scrub ok 2024-09-10T18:46:38.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:37 smithi103 ceph-mon[33163]: 4.19 scrub starts 2024-09-10T18:46:38.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:37 smithi103 ceph-mon[33163]: 4.19 scrub ok 2024-09-10T18:46:38.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:37 smithi103 ceph-mon[33163]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:46:38.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:37 smithi103 ceph-mon[33163]: 5.12 scrub starts 2024-09-10T18:46:38.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:37 smithi103 ceph-mon[33163]: 5.12 scrub ok 2024-09-10T18:46:38.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:37 smithi103 ceph-mon[33163]: 6.1c deep-scrub starts 2024-09-10T18:46:38.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:37 smithi103 ceph-mon[33163]: 6.1c deep-scrub ok 2024-09-10T18:46:38.248 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:46:38 smithi018 bash[72223]: 593b72795192240c8e5e3e0a60106d943380cfb92f8aff77cd5b5cb0633e29ff 2024-09-10T18:46:38.248 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:46:38 smithi018 systemd[1]: Started Ceph alertmanager.a for ba327f6c-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:46:38.570 INFO:teuthology.orchestra.run.smithi018.stdout:223338299412 2024-09-10T18:46:38.571 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:46:38 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:18:46:38] ENGINE Bus STOPPING 2024-09-10T18:46:38.571 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:46:38 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:18:46:38] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:46:38.571 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:46:38 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:18:46:38] ENGINE Bus STOPPED 2024-09-10T18:46:38.571 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:46:38 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:18:46:38] ENGINE Bus STARTING 2024-09-10T18:46:38.571 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:46:38 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:18:46:38] ENGINE Serving on http://:::9283 2024-09-10T18:46:38.571 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:46:38 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:18:46:38] ENGINE Bus STARTED 2024-09-10T18:46:39.192 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018911 got 128849018914 for osd.4 2024-09-10T18:46:39.193 DEBUG:teuthology.parallel:result is None 2024-09-10T18:46:39.432 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:39 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:39.432 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:39 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:39.432 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:39 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:39.433 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:39 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:39.433 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:39 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:39.433 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:39 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:39.433 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:39 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:46:39.433 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:39 smithi018 ceph-mon[29336]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:46:39.433 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:39 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:39.433 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:39 smithi018 ceph-mon[29336]: Deploying daemon grafana.a on smithi103 2024-09-10T18:46:39.433 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:39 smithi018 ceph-mon[29336]: 2.a scrub starts 2024-09-10T18:46:39.433 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:39 smithi018 ceph-mon[29336]: 2.a scrub ok 2024-09-10T18:46:39.433 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:39 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/4069906265' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T18:46:39.433 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:39 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:39.433 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:39 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:39.434 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:39 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:39.434 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:39 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:39.434 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:39 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:39.434 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:39 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:39.434 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:39 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:46:39.434 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:39 smithi018 ceph-mon[40196]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:46:39.434 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:39 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:39.434 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:39 smithi018 ceph-mon[40196]: Deploying daemon grafana.a on smithi103 2024-09-10T18:46:39.434 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:39 smithi018 ceph-mon[40196]: 2.a scrub starts 2024-09-10T18:46:39.434 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:39 smithi018 ceph-mon[40196]: 2.a scrub ok 2024-09-10T18:46:39.434 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:39 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/4069906265' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T18:46:39.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:39 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:39.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:39 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:39.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:39 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:39.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:39 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:39.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:39 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:39.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:39 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:39.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:39 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:46:39.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:39 smithi103 ceph-mon[33163]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:46:39.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:39 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:39.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:39 smithi103 ceph-mon[33163]: Deploying daemon grafana.a on smithi103 2024-09-10T18:46:39.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:39 smithi103 ceph-mon[33163]: 2.a scrub starts 2024-09-10T18:46:39.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:39 smithi103 ceph-mon[33163]: 2.a scrub ok 2024-09-10T18:46:39.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:39 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/4069906265' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T18:46:39.887 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299408 got 223338299412 for osd.7 2024-09-10T18:46:39.888 DEBUG:teuthology.parallel:result is None 2024-09-10T18:46:40.247 INFO:teuthology.orchestra.run.smithi018.stdout:34359738422 2024-09-10T18:46:40.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:40 smithi018 ceph-mon[29336]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:46:40.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:40 smithi018 ceph-mon[29336]: 5.16 scrub starts 2024-09-10T18:46:40.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:40 smithi018 ceph-mon[29336]: 5.16 scrub ok 2024-09-10T18:46:40.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:40 smithi018 ceph-mon[29336]: 5.e scrub starts 2024-09-10T18:46:40.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:40 smithi018 ceph-mon[29336]: 5.e scrub ok 2024-09-10T18:46:40.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:40 smithi018 ceph-mon[29336]: 2.8 scrub starts 2024-09-10T18:46:40.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:40 smithi018 ceph-mon[29336]: 2.8 scrub ok 2024-09-10T18:46:40.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:40 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:40.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:40 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:46:40.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:40 smithi018 ceph-mon[40196]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:46:40.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:40 smithi018 ceph-mon[40196]: 5.16 scrub starts 2024-09-10T18:46:40.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:40 smithi018 ceph-mon[40196]: 5.16 scrub ok 2024-09-10T18:46:40.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:40 smithi018 ceph-mon[40196]: 5.e scrub starts 2024-09-10T18:46:40.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:40 smithi018 ceph-mon[40196]: 5.e scrub ok 2024-09-10T18:46:40.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:40 smithi018 ceph-mon[40196]: 2.8 scrub starts 2024-09-10T18:46:40.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:40 smithi018 ceph-mon[40196]: 2.8 scrub ok 2024-09-10T18:46:40.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:40 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:40.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:40 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:46:40.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:46:40 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:46:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:46:40.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:40 smithi103 ceph-mon[33163]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:46:40.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:40 smithi103 ceph-mon[33163]: 5.16 scrub starts 2024-09-10T18:46:40.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:40 smithi103 ceph-mon[33163]: 5.16 scrub ok 2024-09-10T18:46:40.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:40 smithi103 ceph-mon[33163]: 5.e scrub starts 2024-09-10T18:46:40.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:40 smithi103 ceph-mon[33163]: 5.e scrub ok 2024-09-10T18:46:40.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:40 smithi103 ceph-mon[33163]: 2.8 scrub starts 2024-09-10T18:46:40.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:40 smithi103 ceph-mon[33163]: 2.8 scrub ok 2024-09-10T18:46:40.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:40 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:40.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:40 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:46:40.995 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738418 got 34359738422 for osd.0 2024-09-10T18:46:40.995 DEBUG:teuthology.parallel:result is None 2024-09-10T18:46:40.995 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T18:46:40.996 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T18:46:41.258 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:46:41.380 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:41 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3596591974' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T18:46:41.380 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:41 smithi018 ceph-mon[29336]: 5.18 scrub starts 2024-09-10T18:46:41.380 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:41 smithi018 ceph-mon[29336]: 5.18 scrub ok 2024-09-10T18:46:41.380 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:41 smithi018 ceph-mon[29336]: 5.0 scrub starts 2024-09-10T18:46:41.380 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:41 smithi018 ceph-mon[29336]: 5.0 scrub ok 2024-09-10T18:46:41.380 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:41 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3596591974' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T18:46:41.380 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:41 smithi018 ceph-mon[40196]: 5.18 scrub starts 2024-09-10T18:46:41.380 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:41 smithi018 ceph-mon[40196]: 5.18 scrub ok 2024-09-10T18:46:41.380 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:41 smithi018 ceph-mon[40196]: 5.0 scrub starts 2024-09-10T18:46:41.380 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:41 smithi018 ceph-mon[40196]: 5.0 scrub ok 2024-09-10T18:46:41.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:41 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3596591974' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T18:46:41.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:41 smithi103 ceph-mon[33163]: 5.18 scrub starts 2024-09-10T18:46:41.424 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:41 smithi103 ceph-mon[33163]: 5.18 scrub ok 2024-09-10T18:46:41.424 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:41 smithi103 ceph-mon[33163]: 5.0 scrub starts 2024-09-10T18:46:41.424 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:41 smithi103 ceph-mon[33163]: 5.0 scrub ok 2024-09-10T18:46:42.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:42 smithi018 ceph-mon[29336]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:46:42.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:42 smithi018 ceph-mon[29336]: 2.17 scrub starts 2024-09-10T18:46:42.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:42 smithi018 ceph-mon[29336]: 2.17 scrub ok 2024-09-10T18:46:42.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:42 smithi018 ceph-mon[29336]: 5.1 scrub starts 2024-09-10T18:46:42.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:42 smithi018 ceph-mon[29336]: 5.1 scrub ok 2024-09-10T18:46:42.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:42 smithi018 ceph-mon[29336]: 5.6 scrub starts 2024-09-10T18:46:42.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:42 smithi018 ceph-mon[29336]: 5.6 scrub ok 2024-09-10T18:46:42.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:42 smithi018 ceph-mon[40196]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:46:42.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:42 smithi018 ceph-mon[40196]: 2.17 scrub starts 2024-09-10T18:46:42.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:42 smithi018 ceph-mon[40196]: 2.17 scrub ok 2024-09-10T18:46:42.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:42 smithi018 ceph-mon[40196]: 5.1 scrub starts 2024-09-10T18:46:42.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:42 smithi018 ceph-mon[40196]: 5.1 scrub ok 2024-09-10T18:46:42.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:42 smithi018 ceph-mon[40196]: 5.6 scrub starts 2024-09-10T18:46:42.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:42 smithi018 ceph-mon[40196]: 5.6 scrub ok 2024-09-10T18:46:42.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:42 smithi103 ceph-mon[33163]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:46:42.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:42 smithi103 ceph-mon[33163]: 2.17 scrub starts 2024-09-10T18:46:42.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:42 smithi103 ceph-mon[33163]: 2.17 scrub ok 2024-09-10T18:46:42.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:42 smithi103 ceph-mon[33163]: 5.1 scrub starts 2024-09-10T18:46:42.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:42 smithi103 ceph-mon[33163]: 5.1 scrub ok 2024-09-10T18:46:42.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:42 smithi103 ceph-mon[33163]: 5.6 scrub starts 2024-09-10T18:46:42.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:42 smithi103 ceph-mon[33163]: 5.6 scrub ok 2024-09-10T18:46:43.212 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:46:43.214 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-09-10T18:46:43.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:43 smithi018 ceph-mon[29336]: 6.d scrub starts 2024-09-10T18:46:43.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:43 smithi018 ceph-mon[29336]: 6.d scrub ok 2024-09-10T18:46:43.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:43 smithi018 ceph-mon[29336]: 2.1a scrub starts 2024-09-10T18:46:43.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:43 smithi018 ceph-mon[29336]: 2.1a scrub ok 2024-09-10T18:46:43.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:43 smithi018 ceph-mon[29336]: 2.f scrub starts 2024-09-10T18:46:43.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:43 smithi018 ceph-mon[29336]: 2.f scrub ok 2024-09-10T18:46:43.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:43 smithi018 ceph-mon[40196]: 6.d scrub starts 2024-09-10T18:46:43.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:43 smithi018 ceph-mon[40196]: 6.d scrub ok 2024-09-10T18:46:43.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:43 smithi018 ceph-mon[40196]: 2.1a scrub starts 2024-09-10T18:46:43.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:43 smithi018 ceph-mon[40196]: 2.1a scrub ok 2024-09-10T18:46:43.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:43 smithi018 ceph-mon[40196]: 2.f scrub starts 2024-09-10T18:46:43.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:43 smithi018 ceph-mon[40196]: 2.f scrub ok 2024-09-10T18:46:43.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:43 smithi103 ceph-mon[33163]: 6.d scrub starts 2024-09-10T18:46:43.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:43 smithi103 ceph-mon[33163]: 6.d scrub ok 2024-09-10T18:46:43.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:43 smithi103 ceph-mon[33163]: 2.1a scrub starts 2024-09-10T18:46:43.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:43 smithi103 ceph-mon[33163]: 2.1a scrub ok 2024-09-10T18:46:43.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:43 smithi103 ceph-mon[33163]: 2.f scrub starts 2024-09-10T18:46:43.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:43 smithi103 ceph-mon[33163]: 2.f scrub ok 2024-09-10T18:46:43.902 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-09-10T18:46:43.025553+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":928,"num_read_kb":785,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":555,"ondisk_log_size":555,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2386864,"kb_used_data":8048,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747574352,"statfs":{"total":767960285184,"available":765516136448,"internally_reserved":0,"allocated":8241152,"data_stored":4414428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002355"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:33.824105+0000","last_change":"2024-09-10T18:46:33.824057+0000","last_active":"2024-09-10T18:46:33.824105+0000","last_peered":"2024-09-10T18:46:33.824105+0000","last_clean":"2024-09-10T18:46:33.824105+0000","last_became_active":"2024-09-10T18:45:24.265488+0000","last_became_peered":"2024-09-10T18:45:24.265488+0000","last_unstale":"2024-09-10T18:46:33.824105+0000","last_undegraded":"2024-09-10T18:46:33.824105+0000","last_fullsized":"2024-09-10T18:46:33.824105+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:33.823989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:33.823989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:44.131457+0000","scrub_duration":0.0034389770000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742368+0000","last_change":"2024-09-10T18:45:17.959740+0000","last_active":"2024-09-10T18:46:29.742368+0000","last_peered":"2024-09-10T18:46:29.742368+0000","last_clean":"2024-09-10T18:46:29.742368+0000","last_became_active":"2024-09-10T18:45:17.959475+0000","last_became_peered":"2024-09-10T18:45:17.959475+0000","last_unstale":"2024-09-10T18:46:29.742368+0000","last_undegraded":"2024-09-10T18:46:29.742368+0000","last_fullsized":"2024-09-10T18:46:29.742368+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:41.295534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"63'15","reported_seq":64,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:36.856851+0000","last_change":"2024-09-10T18:46:36.856804+0000","last_active":"2024-09-10T18:46:36.856851+0000","last_peered":"2024-09-10T18:46:36.856851+0000","last_clean":"2024-09-10T18:46:36.856851+0000","last_became_active":"2024-09-10T18:45:20.052531+0000","last_became_peered":"2024-09-10T18:45:20.052531+0000","last_unstale":"2024-09-10T18:46:36.856851+0000","last_undegraded":"2024-09-10T18:46:36.856851+0000","last_fullsized":"2024-09-10T18:46:36.856851+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:46:36.856740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:36.856740+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:03.997270+0000","scrub_duration":0.0018131040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:40.313341+0000","last_change":"2024-09-10T18:46:40.313306+0000","last_active":"2024-09-10T18:46:40.313341+0000","last_peered":"2024-09-10T18:46:40.313341+0000","last_clean":"2024-09-10T18:46:40.313341+0000","last_became_active":"2024-09-10T18:45:22.065619+0000","last_became_peered":"2024-09-10T18:45:22.065619+0000","last_unstale":"2024-09-10T18:46:40.313341+0000","last_undegraded":"2024-09-10T18:46:40.313341+0000","last_fullsized":"2024-09-10T18:46:40.313341+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:40.313218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:40.313218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:52.103586+0000","scrub_duration":0.0017922750000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.791464+0000","last_change":"2024-09-10T18:46:18.890844+0000","last_active":"2024-09-10T18:46:29.791464+0000","last_peered":"2024-09-10T18:46:29.791464+0000","last_clean":"2024-09-10T18:46:29.791464+0000","last_became_active":"2024-09-10T18:45:17.966153+0000","last_became_peered":"2024-09-10T18:45:17.966153+0000","last_unstale":"2024-09-10T18:46:29.791464+0000","last_undegraded":"2024-09-10T18:46:29.791464+0000","last_fullsized":"2024-09-10T18:46:29.791464+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:18.890702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:18.890702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:47.062002+0000","scrub_duration":0.0019382589999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"63'9","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.319204+0000","last_change":"2024-09-10T18:45:20.061552+0000","last_active":"2024-09-10T18:46:30.319204+0000","last_peered":"2024-09-10T18:46:30.319204+0000","last_clean":"2024-09-10T18:46:30.319204+0000","last_became_active":"2024-09-10T18:45:20.061266+0000","last_became_peered":"2024-09-10T18:45:20.061266+0000","last_unstale":"2024-09-10T18:46:30.319204+0000","last_undegraded":"2024-09-10T18:46:30.319204+0000","last_fullsized":"2024-09-10T18:46:30.319204+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:43.961487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742146+0000","last_change":"2024-09-10T18:45:22.073139+0000","last_active":"2024-09-10T18:46:29.742146+0000","last_peered":"2024-09-10T18:46:29.742146+0000","last_clean":"2024-09-10T18:46:29.742146+0000","last_became_active":"2024-09-10T18:45:22.072822+0000","last_became_peered":"2024-09-10T18:45:22.072822+0000","last_unstale":"2024-09-10T18:46:29.742146+0000","last_undegraded":"2024-09-10T18:46:29.742146+0000","last_fullsized":"2024-09-10T18:46:29.742146+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:45.740469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.319442+0000","last_change":"2024-09-10T18:45:24.081275+0000","last_active":"2024-09-10T18:46:30.319442+0000","last_peered":"2024-09-10T18:46:30.319442+0000","last_clean":"2024-09-10T18:46:30.319442+0000","last_became_active":"2024-09-10T18:45:24.081002+0000","last_became_peered":"2024-09-10T18:45:24.081002+0000","last_unstale":"2024-09-10T18:46:30.319442+0000","last_undegraded":"2024-09-10T18:46:30.319442+0000","last_fullsized":"2024-09-10T18:46:30.319442+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:46.639608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.566850+0000","last_change":"2024-09-10T18:45:17.961978+0000","last_active":"2024-09-10T18:46:30.566850+0000","last_peered":"2024-09-10T18:46:30.566850+0000","last_clean":"2024-09-10T18:46:30.566850+0000","last_became_active":"2024-09-10T18:45:17.961097+0000","last_became_peered":"2024-09-10T18:45:17.961097+0000","last_unstale":"2024-09-10T18:46:30.566850+0000","last_undegraded":"2024-09-10T18:46:30.566850+0000","last_fullsized":"2024-09-10T18:46:30.566850+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"63'5","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.319541+0000","last_change":"2024-09-10T18:45:20.058010+0000","last_active":"2024-09-10T18:46:30.319541+0000","last_peered":"2024-09-10T18:46:30.319541+0000","last_clean":"2024-09-10T18:46:30.319541+0000","last_became_active":"2024-09-10T18:45:20.057567+0000","last_became_peered":"2024-09-10T18:45:20.057567+0000","last_unstale":"2024-09-10T18:46:30.319541+0000","last_undegraded":"2024-09-10T18:46:30.319541+0000","last_fullsized":"2024-09-10T18:46:30.319541+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:47.144699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.567431+0000","last_change":"2024-09-10T18:46:22.547871+0000","last_active":"2024-09-10T18:46:30.567431+0000","last_peered":"2024-09-10T18:46:30.567431+0000","last_clean":"2024-09-10T18:46:30.567431+0000","last_became_active":"2024-09-10T18:45:22.076575+0000","last_became_peered":"2024-09-10T18:45:22.076575+0000","last_unstale":"2024-09-10T18:46:30.567431+0000","last_undegraded":"2024-09-10T18:46:30.567431+0000","last_fullsized":"2024-09-10T18:46:30.567431+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:22.547800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:22.547800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:32.775669+0000","scrub_duration":0.001880622,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:36.158189+0000","last_change":"2024-09-10T18:46:36.158152+0000","last_active":"2024-09-10T18:46:36.158189+0000","last_peered":"2024-09-10T18:46:36.158189+0000","last_clean":"2024-09-10T18:46:36.158189+0000","last_became_active":"2024-09-10T18:45:24.079534+0000","last_became_peered":"2024-09-10T18:45:24.079534+0000","last_unstale":"2024-09-10T18:46:36.158189+0000","last_undegraded":"2024-09-10T18:46:36.158189+0000","last_fullsized":"2024-09-10T18:46:36.158189+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:36.158087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:36.158087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:41.325875+0000","scrub_duration":0.0014477100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.567032+0000","last_change":"2024-09-10T18:45:17.963368+0000","last_active":"2024-09-10T18:46:30.567032+0000","last_peered":"2024-09-10T18:46:30.567032+0000","last_clean":"2024-09-10T18:46:30.567032+0000","last_became_active":"2024-09-10T18:45:17.963121+0000","last_became_peered":"2024-09-10T18:45:17.963121+0000","last_unstale":"2024-09-10T18:46:30.567032+0000","last_undegraded":"2024-09-10T18:46:30.567032+0000","last_fullsized":"2024-09-10T18:46:30.567032+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:34.269484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"63'9","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.319602+0000","last_change":"2024-09-10T18:45:20.053258+0000","last_active":"2024-09-10T18:46:30.319602+0000","last_peered":"2024-09-10T18:46:30.319602+0000","last_clean":"2024-09-10T18:46:30.319602+0000","last_became_active":"2024-09-10T18:45:20.052652+0000","last_became_peered":"2024-09-10T18:45:20.052652+0000","last_unstale":"2024-09-10T18:46:30.319602+0000","last_undegraded":"2024-09-10T18:46:30.319602+0000","last_fullsized":"2024-09-10T18:46:30.319602+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:42.837022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:31.024367+0000","last_change":"2024-09-10T18:46:31.024322+0000","last_active":"2024-09-10T18:46:31.024367+0000","last_peered":"2024-09-10T18:46:31.024367+0000","last_clean":"2024-09-10T18:46:31.024367+0000","last_became_active":"2024-09-10T18:45:22.068326+0000","last_became_peered":"2024-09-10T18:45:22.068326+0000","last_unstale":"2024-09-10T18:46:31.024367+0000","last_undegraded":"2024-09-10T18:46:31.024367+0000","last_fullsized":"2024-09-10T18:46:31.024367+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:31.024255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:31.024255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:50.407493+0000","scrub_duration":0.00152395,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.740928+0000","last_change":"2024-09-10T18:45:24.081968+0000","last_active":"2024-09-10T18:46:29.740928+0000","last_peered":"2024-09-10T18:46:29.740928+0000","last_clean":"2024-09-10T18:46:29.740928+0000","last_became_active":"2024-09-10T18:45:24.081488+0000","last_became_peered":"2024-09-10T18:45:24.081488+0000","last_unstale":"2024-09-10T18:46:29.740928+0000","last_undegraded":"2024-09-10T18:46:29.740928+0000","last_fullsized":"2024-09-10T18:46:29.740928+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:51.266075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"63'17","reported_seq":57,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.319995+0000","last_change":"2024-09-10T18:45:20.057352+0000","last_active":"2024-09-10T18:46:30.319995+0000","last_peered":"2024-09-10T18:46:30.319995+0000","last_clean":"2024-09-10T18:46:30.319995+0000","last_became_active":"2024-09-10T18:45:20.057019+0000","last_became_peered":"2024-09-10T18:45:20.057019+0000","last_unstale":"2024-09-10T18:46:30.319995+0000","last_undegraded":"2024-09-10T18:46:30.319995+0000","last_fullsized":"2024-09-10T18:46:30.319995+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:40.656805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.566501+0000","last_change":"2024-09-10T18:45:17.964001+0000","last_active":"2024-09-10T18:46:30.566501+0000","last_peered":"2024-09-10T18:46:30.566501+0000","last_clean":"2024-09-10T18:46:30.566501+0000","last_became_active":"2024-09-10T18:45:17.963732+0000","last_became_peered":"2024-09-10T18:45:17.963732+0000","last_unstale":"2024-09-10T18:46:30.566501+0000","last_undegraded":"2024-09-10T18:46:30.566501+0000","last_fullsized":"2024-09-10T18:46:30.566501+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742330+0000","last_change":"2024-09-10T18:45:22.071993+0000","last_active":"2024-09-10T18:46:29.742330+0000","last_peered":"2024-09-10T18:46:29.742330+0000","last_clean":"2024-09-10T18:46:29.742330+0000","last_became_active":"2024-09-10T18:45:22.071443+0000","last_became_peered":"2024-09-10T18:45:22.071443+0000","last_unstale":"2024-09-10T18:46:29.742330+0000","last_undegraded":"2024-09-10T18:46:29.742330+0000","last_fullsized":"2024-09-10T18:46:29.742330+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:42.692893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:35.738263+0000","last_change":"2024-09-10T18:46:35.738217+0000","last_active":"2024-09-10T18:46:35.738263+0000","last_peered":"2024-09-10T18:46:35.738263+0000","last_clean":"2024-09-10T18:46:35.738263+0000","last_became_active":"2024-09-10T18:45:24.077897+0000","last_became_peered":"2024-09-10T18:45:24.077897+0000","last_unstale":"2024-09-10T18:46:35.738263+0000","last_undegraded":"2024-09-10T18:46:35.738263+0000","last_fullsized":"2024-09-10T18:46:35.738263+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:35.738143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:35.738143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:48.741747+0000","scrub_duration":0.0015293520000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"63'10","reported_seq":54,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:31.281648+0000","last_change":"2024-09-10T18:46:31.281588+0000","last_active":"2024-09-10T18:46:31.281648+0000","last_peered":"2024-09-10T18:46:31.281648+0000","last_clean":"2024-09-10T18:46:31.281648+0000","last_became_active":"2024-09-10T18:45:20.053428+0000","last_became_peered":"2024-09-10T18:45:20.053428+0000","last_unstale":"2024-09-10T18:46:31.281648+0000","last_undegraded":"2024-09-10T18:46:31.281648+0000","last_fullsized":"2024-09-10T18:46:31.281648+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:46:31.281519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:31.281519+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:58.508120+0000","scrub_duration":0.0019095119999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:38.445445+0000","last_change":"2024-09-10T18:46:38.445417+0000","last_active":"2024-09-10T18:46:38.445445+0000","last_peered":"2024-09-10T18:46:38.445445+0000","last_clean":"2024-09-10T18:46:38.445445+0000","last_became_active":"2024-09-10T18:45:17.966833+0000","last_became_peered":"2024-09-10T18:45:17.966833+0000","last_unstale":"2024-09-10T18:46:38.445445+0000","last_undegraded":"2024-09-10T18:46:38.445445+0000","last_fullsized":"2024-09-10T18:46:38.445445+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:38.445362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:38.445362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:02.460484+0000","scrub_duration":0.0012292029999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"63'8","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:34.072313+0000","last_change":"2024-09-10T18:46:25.787936+0000","last_active":"2024-09-10T18:46:34.072313+0000","last_peered":"2024-09-10T18:46:34.072313+0000","last_clean":"2024-09-10T18:46:34.072313+0000","last_became_active":"2024-09-10T18:45:22.068453+0000","last_became_peered":"2024-09-10T18:45:22.068453+0000","last_unstale":"2024-09-10T18:46:34.072313+0000","last_undegraded":"2024-09-10T18:46:34.072313+0000","last_fullsized":"2024-09-10T18:46:34.072313+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:25.787877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:25.787877+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:54.201629+0000","scrub_duration":0.0018093530000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:32.265084+0000","last_change":"2024-09-10T18:46:32.265034+0000","last_active":"2024-09-10T18:46:32.265084+0000","last_peered":"2024-09-10T18:46:32.265084+0000","last_clean":"2024-09-10T18:46:32.265084+0000","last_became_active":"2024-09-10T18:45:24.081082+0000","last_became_peered":"2024-09-10T18:45:24.081082+0000","last_unstale":"2024-09-10T18:46:32.265084+0000","last_undegraded":"2024-09-10T18:46:32.265084+0000","last_fullsized":"2024-09-10T18:46:32.265084+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:32.264963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:32.264963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:50.207705+0000","scrub_duration":0.002378509,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"63'15","reported_seq":54,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742827+0000","last_change":"2024-09-10T18:45:20.059509+0000","last_active":"2024-09-10T18:46:29.742827+0000","last_peered":"2024-09-10T18:46:29.742827+0000","last_clean":"2024-09-10T18:46:29.742827+0000","last_became_active":"2024-09-10T18:45:20.059222+0000","last_became_peered":"2024-09-10T18:45:20.059222+0000","last_unstale":"2024-09-10T18:46:29.742827+0000","last_undegraded":"2024-09-10T18:46:29.742827+0000","last_fullsized":"2024-09-10T18:46:29.742827+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:41.772741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742326+0000","last_change":"2024-09-10T18:46:28.320761+0000","last_active":"2024-09-10T18:46:29.742326+0000","last_peered":"2024-09-10T18:46:29.742326+0000","last_clean":"2024-09-10T18:46:29.742326+0000","last_became_active":"2024-09-10T18:45:17.965409+0000","last_became_peered":"2024-09-10T18:45:17.965409+0000","last_unstale":"2024-09-10T18:46:29.742326+0000","last_undegraded":"2024-09-10T18:46:29.742326+0000","last_fullsized":"2024-09-10T18:46:29.742326+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_clean_scrub_stamp":"2024-09-10T18:46:28.320704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:44.092028+0000","scrub_duration":0.001393476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"63'8","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:39.300173+0000","last_change":"2024-09-10T18:46:39.300139+0000","last_active":"2024-09-10T18:46:39.300173+0000","last_peered":"2024-09-10T18:46:39.300173+0000","last_clean":"2024-09-10T18:46:39.300173+0000","last_became_active":"2024-09-10T18:45:22.062497+0000","last_became_peered":"2024-09-10T18:45:22.062497+0000","last_unstale":"2024-09-10T18:46:39.300173+0000","last_undegraded":"2024-09-10T18:46:39.300173+0000","last_fullsized":"2024-09-10T18:46:39.300173+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:39.300065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:39.300065+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:44.830898+0000","scrub_duration":0.001858934,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742058+0000","last_change":"2024-09-10T18:45:24.082637+0000","last_active":"2024-09-10T18:46:29.742058+0000","last_peered":"2024-09-10T18:46:29.742058+0000","last_clean":"2024-09-10T18:46:29.742058+0000","last_became_active":"2024-09-10T18:45:24.082362+0000","last_became_peered":"2024-09-10T18:45:24.082362+0000","last_unstale":"2024-09-10T18:46:29.742058+0000","last_undegraded":"2024-09-10T18:46:29.742058+0000","last_fullsized":"2024-09-10T18:46:29.742058+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"63'11","reported_seq":58,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:35.274763+0000","last_change":"2024-09-10T18:46:35.274729+0000","last_active":"2024-09-10T18:46:35.274763+0000","last_peered":"2024-09-10T18:46:35.274763+0000","last_clean":"2024-09-10T18:46:35.274763+0000","last_became_active":"2024-09-10T18:45:20.061187+0000","last_became_peered":"2024-09-10T18:45:20.061187+0000","last_unstale":"2024-09-10T18:46:35.274763+0000","last_undegraded":"2024-09-10T18:46:35.274763+0000","last_fullsized":"2024-09-10T18:46:35.274763+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:46:35.274660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:35.274660+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:51.240811+0000","scrub_duration":0.0019745819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:39.520524+0000","last_change":"2024-09-10T18:46:39.520497+0000","last_active":"2024-09-10T18:46:39.520524+0000","last_peered":"2024-09-10T18:46:39.520524+0000","last_clean":"2024-09-10T18:46:39.520524+0000","last_became_active":"2024-09-10T18:45:17.966133+0000","last_became_peered":"2024-09-10T18:45:17.966133+0000","last_unstale":"2024-09-10T18:46:39.520524+0000","last_undegraded":"2024-09-10T18:46:39.520524+0000","last_fullsized":"2024-09-10T18:46:39.520524+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:39.520432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:39.520432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:49.040271+0000","scrub_duration":0.0013982510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742394+0000","last_change":"2024-09-10T18:45:22.069224+0000","last_active":"2024-09-10T18:46:29.742394+0000","last_peered":"2024-09-10T18:46:29.742394+0000","last_clean":"2024-09-10T18:46:29.742394+0000","last_became_active":"2024-09-10T18:45:22.068918+0000","last_became_peered":"2024-09-10T18:45:22.068918+0000","last_unstale":"2024-09-10T18:46:29.742394+0000","last_undegraded":"2024-09-10T18:46:29.742394+0000","last_fullsized":"2024-09-10T18:46:29.742394+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:35.444750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.790998+0000","last_change":"2024-09-10T18:45:24.264914+0000","last_active":"2024-09-10T18:46:29.790998+0000","last_peered":"2024-09-10T18:46:29.790998+0000","last_clean":"2024-09-10T18:46:29.790998+0000","last_became_active":"2024-09-10T18:45:24.264538+0000","last_became_peered":"2024-09-10T18:45:24.264538+0000","last_unstale":"2024-09-10T18:46:29.790998+0000","last_undegraded":"2024-09-10T18:46:29.790998+0000","last_fullsized":"2024-09-10T18:46:29.790998+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:46.643079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"63'14","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.319391+0000","last_change":"2024-09-10T18:46:29.330572+0000","last_active":"2024-09-10T18:46:30.319391+0000","last_peered":"2024-09-10T18:46:30.319391+0000","last_clean":"2024-09-10T18:46:30.319391+0000","last_became_active":"2024-09-10T18:45:20.052243+0000","last_became_peered":"2024-09-10T18:45:20.052243+0000","last_unstale":"2024-09-10T18:46:30.319391+0000","last_undegraded":"2024-09-10T18:46:30.319391+0000","last_fullsized":"2024-09-10T18:46:30.319391+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-10T18:46:29.330503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:29.330503+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:55.456983+0000","scrub_duration":0.0020437459999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.792661+0000","last_change":"2024-09-10T18:46:25.206801+0000","last_active":"2024-09-10T18:46:29.792661+0000","last_peered":"2024-09-10T18:46:29.792661+0000","last_clean":"2024-09-10T18:46:29.792661+0000","last_became_active":"2024-09-10T18:45:17.966027+0000","last_became_peered":"2024-09-10T18:45:17.966027+0000","last_unstale":"2024-09-10T18:46:29.792661+0000","last_undegraded":"2024-09-10T18:46:29.792661+0000","last_fullsized":"2024-09-10T18:46:29.792661+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_clean_scrub_stamp":"2024-09-10T18:46:25.206731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:46.038154+0000","scrub_duration":0.0016589910000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:40.808551+0000","last_change":"2024-09-10T18:46:40.808518+0000","last_active":"2024-09-10T18:46:40.808551+0000","last_peered":"2024-09-10T18:46:40.808551+0000","last_clean":"2024-09-10T18:46:40.808551+0000","last_became_active":"2024-09-10T18:45:22.074540+0000","last_became_peered":"2024-09-10T18:45:22.074540+0000","last_unstale":"2024-09-10T18:46:40.808551+0000","last_undegraded":"2024-09-10T18:46:40.808551+0000","last_fullsized":"2024-09-10T18:46:40.808551+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:40.808453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:40.808453+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:43.962254+0000","scrub_duration":0.0016757829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:32.590593+0000","last_change":"2024-09-10T18:46:32.590511+0000","last_active":"2024-09-10T18:46:32.590593+0000","last_peered":"2024-09-10T18:46:32.590593+0000","last_clean":"2024-09-10T18:46:32.590593+0000","last_became_active":"2024-09-10T18:45:24.265303+0000","last_became_peered":"2024-09-10T18:45:24.265303+0000","last_unstale":"2024-09-10T18:46:32.590593+0000","last_undegraded":"2024-09-10T18:46:32.590593+0000","last_fullsized":"2024-09-10T18:46:32.590593+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:32.590438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:32.590438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:01.680676+0000","scrub_duration":0.0013292270000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"63'18","reported_seq":71,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.790754+0000","last_change":"2024-09-10T18:46:27.801177+0000","last_active":"2024-09-10T18:46:29.790754+0000","last_peered":"2024-09-10T18:46:29.790754+0000","last_clean":"2024-09-10T18:46:29.790754+0000","last_became_active":"2024-09-10T18:45:20.420937+0000","last_became_peered":"2024-09-10T18:45:20.420937+0000","last_unstale":"2024-09-10T18:46:29.790754+0000","last_undegraded":"2024-09-10T18:46:29.790754+0000","last_fullsized":"2024-09-10T18:46:29.790754+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'18","last_scrub_stamp":"2024-09-10T18:46:27.801110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:27.801110+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:42.247024+0000","scrub_duration":0.001870095,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742628+0000","last_change":"2024-09-10T18:45:17.962000+0000","last_active":"2024-09-10T18:46:29.742628+0000","last_peered":"2024-09-10T18:46:29.742628+0000","last_clean":"2024-09-10T18:46:29.742628+0000","last_became_active":"2024-09-10T18:45:17.961695+0000","last_became_peered":"2024-09-10T18:45:17.961695+0000","last_unstale":"2024-09-10T18:46:29.742628+0000","last_undegraded":"2024-09-10T18:46:29.742628+0000","last_fullsized":"2024-09-10T18:46:29.742628+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:43.273962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.319288+0000","last_change":"2024-09-10T18:45:22.074309+0000","last_active":"2024-09-10T18:46:30.319288+0000","last_peered":"2024-09-10T18:46:30.319288+0000","last_clean":"2024-09-10T18:46:30.319288+0000","last_became_active":"2024-09-10T18:45:22.074012+0000","last_became_peered":"2024-09-10T18:45:22.074012+0000","last_unstale":"2024-09-10T18:46:30.319288+0000","last_undegraded":"2024-09-10T18:46:30.319288+0000","last_fullsized":"2024-09-10T18:46:30.319288+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:35.318930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.319530+0000","last_change":"2024-09-10T18:45:24.077245+0000","last_active":"2024-09-10T18:46:30.319530+0000","last_peered":"2024-09-10T18:46:30.319530+0000","last_clean":"2024-09-10T18:46:30.319530+0000","last_became_active":"2024-09-10T18:45:24.076831+0000","last_became_peered":"2024-09-10T18:45:24.076831+0000","last_unstale":"2024-09-10T18:46:30.319530+0000","last_undegraded":"2024-09-10T18:46:30.319530+0000","last_fullsized":"2024-09-10T18:46:30.319530+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:41.841015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"63'19","reported_seq":65,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.741641+0000","last_change":"2024-09-10T18:45:20.421532+0000","last_active":"2024-09-10T18:46:29.741641+0000","last_peered":"2024-09-10T18:46:29.741641+0000","last_clean":"2024-09-10T18:46:29.741641+0000","last_became_active":"2024-09-10T18:45:20.421046+0000","last_became_peered":"2024-09-10T18:45:20.421046+0000","last_unstale":"2024-09-10T18:46:29.741641+0000","last_undegraded":"2024-09-10T18:46:29.741641+0000","last_fullsized":"2024-09-10T18:46:29.741641+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.566674+0000","last_change":"2024-09-10T18:45:17.960558+0000","last_active":"2024-09-10T18:46:30.566674+0000","last_peered":"2024-09-10T18:46:30.566674+0000","last_clean":"2024-09-10T18:46:30.566674+0000","last_became_active":"2024-09-10T18:45:17.960226+0000","last_became_peered":"2024-09-10T18:45:17.960226+0000","last_unstale":"2024-09-10T18:46:30.566674+0000","last_undegraded":"2024-09-10T18:46:30.566674+0000","last_fullsized":"2024-09-10T18:46:30.566674+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:43.999118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.791889+0000","last_change":"2024-09-10T18:45:22.066434+0000","last_active":"2024-09-10T18:46:29.791889+0000","last_peered":"2024-09-10T18:46:29.791889+0000","last_clean":"2024-09-10T18:46:29.791889+0000","last_became_active":"2024-09-10T18:45:22.066148+0000","last_became_peered":"2024-09-10T18:45:22.066148+0000","last_unstale":"2024-09-10T18:46:29.791889+0000","last_undegraded":"2024-09-10T18:46:29.791889+0000","last_fullsized":"2024-09-10T18:46:29.791889+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:46.655683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742078+0000","last_change":"2024-09-10T18:45:24.265025+0000","last_active":"2024-09-10T18:46:29.742078+0000","last_peered":"2024-09-10T18:46:29.742078+0000","last_clean":"2024-09-10T18:46:29.742078+0000","last_became_active":"2024-09-10T18:45:24.264680+0000","last_became_peered":"2024-09-10T18:45:24.264680+0000","last_unstale":"2024-09-10T18:46:29.742078+0000","last_undegraded":"2024-09-10T18:46:29.742078+0000","last_fullsized":"2024-09-10T18:46:29.742078+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:47.094144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"63'10","reported_seq":44,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.741454+0000","last_change":"2024-09-10T18:45:20.057590+0000","last_active":"2024-09-10T18:46:29.741454+0000","last_peered":"2024-09-10T18:46:29.741454+0000","last_clean":"2024-09-10T18:46:29.741454+0000","last_became_active":"2024-09-10T18:45:20.057204+0000","last_became_peered":"2024-09-10T18:45:20.057204+0000","last_unstale":"2024-09-10T18:46:29.741454+0000","last_undegraded":"2024-09-10T18:46:29.741454+0000","last_fullsized":"2024-09-10T18:46:29.741454+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:38.786019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742270+0000","last_change":"2024-09-10T18:45:17.968539+0000","last_active":"2024-09-10T18:46:29.742270+0000","last_peered":"2024-09-10T18:46:29.742270+0000","last_clean":"2024-09-10T18:46:29.742270+0000","last_became_active":"2024-09-10T18:45:17.966632+0000","last_became_peered":"2024-09-10T18:45:17.966632+0000","last_unstale":"2024-09-10T18:46:29.742270+0000","last_undegraded":"2024-09-10T18:46:29.742270+0000","last_fullsized":"2024-09-10T18:46:29.742270+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:37.082480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"63'8","reported_seq":51,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:33.471982+0000","last_change":"2024-09-10T18:45:22.063631+0000","last_active":"2024-09-10T18:46:33.471982+0000","last_peered":"2024-09-10T18:46:33.471982+0000","last_clean":"2024-09-10T18:46:33.471982+0000","last_became_active":"2024-09-10T18:45:22.063326+0000","last_became_peered":"2024-09-10T18:45:22.063326+0000","last_unstale":"2024-09-10T18:46:33.471982+0000","last_undegraded":"2024-09-10T18:46:33.471982+0000","last_fullsized":"2024-09-10T18:46:33.471982+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:41.497821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.740793+0000","last_change":"2024-09-10T18:45:24.077780+0000","last_active":"2024-09-10T18:46:29.740793+0000","last_peered":"2024-09-10T18:46:29.740793+0000","last_clean":"2024-09-10T18:46:29.740793+0000","last_became_active":"2024-09-10T18:45:24.077412+0000","last_became_peered":"2024-09-10T18:45:24.077412+0000","last_unstale":"2024-09-10T18:46:29.740793+0000","last_undegraded":"2024-09-10T18:46:29.740793+0000","last_fullsized":"2024-09-10T18:46:29.740793+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:45.489998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"63'28","reported_seq":103,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:33.472587+0000","last_change":"2024-09-10T18:46:19.223319+0000","last_active":"2024-09-10T18:46:33.472587+0000","last_peered":"2024-09-10T18:46:33.472587+0000","last_clean":"2024-09-10T18:46:33.472587+0000","last_became_active":"2024-09-10T18:45:20.061413+0000","last_became_peered":"2024-09-10T18:45:20.061413+0000","last_unstale":"2024-09-10T18:46:33.472587+0000","last_undegraded":"2024-09-10T18:46:33.472587+0000","last_fullsized":"2024-09-10T18:46:33.472587+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'28","last_scrub_stamp":"2024-09-10T18:46:19.223250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:19.223250+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:23.644429+0000","scrub_duration":0.0032067810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.741949+0000","last_change":"2024-09-10T18:45:17.965134+0000","last_active":"2024-09-10T18:46:29.741949+0000","last_peered":"2024-09-10T18:46:29.741949+0000","last_clean":"2024-09-10T18:46:29.741949+0000","last_became_active":"2024-09-10T18:45:17.964494+0000","last_became_peered":"2024-09-10T18:45:17.964494+0000","last_unstale":"2024-09-10T18:46:29.741949+0000","last_undegraded":"2024-09-10T18:46:29.741949+0000","last_fullsized":"2024-09-10T18:46:29.741949+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:36.348849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.741750+0000","last_change":"2024-09-10T18:45:22.061738+0000","last_active":"2024-09-10T18:46:29.741750+0000","last_peered":"2024-09-10T18:46:29.741750+0000","last_clean":"2024-09-10T18:46:29.741750+0000","last_became_active":"2024-09-10T18:45:22.061428+0000","last_became_peered":"2024-09-10T18:45:22.061428+0000","last_unstale":"2024-09-10T18:46:29.741750+0000","last_undegraded":"2024-09-10T18:46:29.741750+0000","last_fullsized":"2024-09-10T18:46:29.741750+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:41.151863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.791463+0000","last_change":"2024-09-10T18:45:24.073623+0000","last_active":"2024-09-10T18:46:29.791463+0000","last_peered":"2024-09-10T18:46:29.791463+0000","last_clean":"2024-09-10T18:46:29.791463+0000","last_became_active":"2024-09-10T18:45:24.073307+0000","last_became_peered":"2024-09-10T18:45:24.073307+0000","last_unstale":"2024-09-10T18:46:29.791463+0000","last_undegraded":"2024-09-10T18:46:29.791463+0000","last_fullsized":"2024-09-10T18:46:29.791463+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:51.119822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"63'13","reported_seq":56,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742830+0000","last_change":"2024-09-10T18:45:20.058778+0000","last_active":"2024-09-10T18:46:29.742830+0000","last_peered":"2024-09-10T18:46:29.742830+0000","last_clean":"2024-09-10T18:46:29.742830+0000","last_became_active":"2024-09-10T18:45:20.058435+0000","last_became_peered":"2024-09-10T18:45:20.058435+0000","last_unstale":"2024-09-10T18:46:29.742830+0000","last_undegraded":"2024-09-10T18:46:29.742830+0000","last_fullsized":"2024-09-10T18:46:29.742830+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:44.903372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"65'5","reported_seq":114,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:39.364272+0000","last_change":"2024-09-10T18:45:22.059997+0000","last_active":"2024-09-10T18:46:39.364272+0000","last_peered":"2024-09-10T18:46:39.364272+0000","last_clean":"2024-09-10T18:46:39.364272+0000","last_became_active":"2024-09-10T18:45:17.969868+0000","last_became_peered":"2024-09-10T18:45:17.969868+0000","last_unstale":"2024-09-10T18:46:39.364272+0000","last_undegraded":"2024-09-10T18:46:39.364272+0000","last_fullsized":"2024-09-10T18:46:39.364272+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:45.998919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021933159999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":75,"num_read_kb":70,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.566415+0000","last_change":"2024-09-10T18:45:17.961131+0000","last_active":"2024-09-10T18:46:30.566415+0000","last_peered":"2024-09-10T18:46:30.566415+0000","last_clean":"2024-09-10T18:46:30.566415+0000","last_became_active":"2024-09-10T18:45:17.960859+0000","last_became_peered":"2024-09-10T18:45:17.960859+0000","last_unstale":"2024-09-10T18:46:30.566415+0000","last_undegraded":"2024-09-10T18:46:30.566415+0000","last_fullsized":"2024-09-10T18:46:30.566415+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:42.810415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.567280+0000","last_change":"2024-09-10T18:45:22.068719+0000","last_active":"2024-09-10T18:46:30.567280+0000","last_peered":"2024-09-10T18:46:30.567280+0000","last_clean":"2024-09-10T18:46:30.567280+0000","last_became_active":"2024-09-10T18:45:22.068350+0000","last_became_peered":"2024-09-10T18:45:22.068350+0000","last_unstale":"2024-09-10T18:46:30.567280+0000","last_undegraded":"2024-09-10T18:46:30.567280+0000","last_fullsized":"2024-09-10T18:46:30.567280+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:34.173124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.567760+0000","last_change":"2024-09-10T18:45:24.264879+0000","last_active":"2024-09-10T18:46:30.567760+0000","last_peered":"2024-09-10T18:46:30.567760+0000","last_clean":"2024-09-10T18:46:30.567760+0000","last_became_active":"2024-09-10T18:45:24.264617+0000","last_became_peered":"2024-09-10T18:45:24.264617+0000","last_unstale":"2024-09-10T18:46:30.567760+0000","last_undegraded":"2024-09-10T18:46:30.567760+0000","last_fullsized":"2024-09-10T18:46:30.567760+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:40.675862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"63'12","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742416+0000","last_change":"2024-09-10T18:45:20.061430+0000","last_active":"2024-09-10T18:46:29.742416+0000","last_peered":"2024-09-10T18:46:29.742416+0000","last_clean":"2024-09-10T18:46:29.742416+0000","last_became_active":"2024-09-10T18:45:20.061091+0000","last_became_peered":"2024-09-10T18:45:20.061091+0000","last_unstale":"2024-09-10T18:46:29.742416+0000","last_undegraded":"2024-09-10T18:46:29.742416+0000","last_fullsized":"2024-09-10T18:46:29.742416+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:38.785499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"58'1","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.741342+0000","last_change":"2024-09-10T18:45:22.055928+0000","last_active":"2024-09-10T18:46:29.741342+0000","last_peered":"2024-09-10T18:46:29.741342+0000","last_clean":"2024-09-10T18:46:29.741342+0000","last_became_active":"2024-09-10T18:45:17.963932+0000","last_became_peered":"2024-09-10T18:45:17.963932+0000","last_unstale":"2024-09-10T18:46:29.741342+0000","last_undegraded":"2024-09-10T18:46:29.741342+0000","last_fullsized":"2024-09-10T18:46:29.741342+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:45.517533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037812099999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.566408+0000","last_change":"2024-09-10T18:45:17.966921+0000","last_active":"2024-09-10T18:46:30.566408+0000","last_peered":"2024-09-10T18:46:30.566408+0000","last_clean":"2024-09-10T18:46:30.566408+0000","last_became_active":"2024-09-10T18:45:17.966493+0000","last_became_peered":"2024-09-10T18:45:17.966493+0000","last_unstale":"2024-09-10T18:46:30.566408+0000","last_undegraded":"2024-09-10T18:46:30.566408+0000","last_fullsized":"2024-09-10T18:46:30.566408+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:37.560371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742429+0000","last_change":"2024-09-10T18:45:22.064783+0000","last_active":"2024-09-10T18:46:29.742429+0000","last_peered":"2024-09-10T18:46:29.742429+0000","last_clean":"2024-09-10T18:46:29.742429+0000","last_became_active":"2024-09-10T18:45:22.064285+0000","last_became_peered":"2024-09-10T18:45:22.064285+0000","last_unstale":"2024-09-10T18:46:29.742429+0000","last_undegraded":"2024-09-10T18:46:29.742429+0000","last_fullsized":"2024-09-10T18:46:29.742429+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:46.039937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.741935+0000","last_change":"2024-09-10T18:45:24.084038+0000","last_active":"2024-09-10T18:46:29.741935+0000","last_peered":"2024-09-10T18:46:29.741935+0000","last_clean":"2024-09-10T18:46:29.741935+0000","last_became_active":"2024-09-10T18:45:24.083689+0000","last_became_peered":"2024-09-10T18:45:24.083689+0000","last_unstale":"2024-09-10T18:46:29.741935+0000","last_undegraded":"2024-09-10T18:46:29.741935+0000","last_fullsized":"2024-09-10T18:46:29.741935+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:36.059986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"63'14","reported_seq":61,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.792381+0000","last_change":"2024-09-10T18:46:22.294141+0000","last_active":"2024-09-10T18:46:29.792381+0000","last_peered":"2024-09-10T18:46:29.792381+0000","last_clean":"2024-09-10T18:46:29.792381+0000","last_became_active":"2024-09-10T18:45:20.421119+0000","last_became_peered":"2024-09-10T18:45:20.421119+0000","last_unstale":"2024-09-10T18:46:29.792381+0000","last_undegraded":"2024-09-10T18:46:29.792381+0000","last_fullsized":"2024-09-10T18:46:29.792381+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-10T18:46:22.294072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:22.294072+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:49.887899+0000","scrub_duration":0.0021435069999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"60'2","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.790956+0000","last_change":"2024-09-10T18:46:25.845131+0000","last_active":"2024-09-10T18:46:29.790956+0000","last_peered":"2024-09-10T18:46:29.790956+0000","last_clean":"2024-09-10T18:46:29.790956+0000","last_became_active":"2024-09-10T18:45:17.961660+0000","last_became_peered":"2024-09-10T18:45:17.961660+0000","last_unstale":"2024-09-10T18:46:29.790956+0000","last_undegraded":"2024-09-10T18:46:29.790956+0000","last_fullsized":"2024-09-10T18:46:29.790956+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-10T18:46:25.845070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:25.845070+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:50.370600+0000","scrub_duration":0.0020129710000000001,"objects_trimmed":0,"snaptrim_duration":0.0013942640000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.741918+0000","last_change":"2024-09-10T18:46:29.081362+0000","last_active":"2024-09-10T18:46:29.741918+0000","last_peered":"2024-09-10T18:46:29.741918+0000","last_clean":"2024-09-10T18:46:29.741918+0000","last_became_active":"2024-09-10T18:45:17.965163+0000","last_became_peered":"2024-09-10T18:45:17.965163+0000","last_unstale":"2024-09-10T18:46:29.741918+0000","last_undegraded":"2024-09-10T18:46:29.741918+0000","last_fullsized":"2024-09-10T18:46:29.741918+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:29.081296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:29.081296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:50.659900+0000","scrub_duration":0.0015488590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'81","reported_seq":75,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.567438+0000","last_change":"2024-09-10T18:45:45.481030+0000","last_active":"2024-09-10T18:46:30.567438+0000","last_peered":"2024-09-10T18:46:30.567438+0000","last_clean":"2024-09-10T18:46:30.567438+0000","last_became_active":"2024-09-10T18:45:07.433592+0000","last_became_peered":"2024-09-10T18:45:07.433592+0000","last_unstale":"2024-09-10T18:46:30.567438+0000","last_undegraded":"2024-09-10T18:46:30.567438+0000","last_fullsized":"2024-09-10T18:46:30.567438+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'74","last_scrub_stamp":"2024-09-10T18:45:45.480958+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:45:45.480958+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:58.297247+0000","scrub_duration":0.0019876809999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.567749+0000","last_change":"2024-09-10T18:46:26.687029+0000","last_active":"2024-09-10T18:46:30.567749+0000","last_peered":"2024-09-10T18:46:30.567749+0000","last_clean":"2024-09-10T18:46:30.567749+0000","last_became_active":"2024-09-10T18:45:22.068226+0000","last_became_peered":"2024-09-10T18:45:22.068226+0000","last_unstale":"2024-09-10T18:46:30.567749+0000","last_undegraded":"2024-09-10T18:46:30.567749+0000","last_fullsized":"2024-09-10T18:46:30.567749+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:26.686961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:26.686961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:32.456741+0000","scrub_duration":0.0015985109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742521+0000","last_change":"2024-09-10T18:45:24.073576+0000","last_active":"2024-09-10T18:46:29.742521+0000","last_peered":"2024-09-10T18:46:29.742521+0000","last_clean":"2024-09-10T18:46:29.742521+0000","last_became_active":"2024-09-10T18:45:24.073327+0000","last_became_peered":"2024-09-10T18:45:24.073327+0000","last_unstale":"2024-09-10T18:46:29.742521+0000","last_undegraded":"2024-09-10T18:46:29.742521+0000","last_fullsized":"2024-09-10T18:46:29.742521+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:45.713352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"63'19","reported_seq":70,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.792098+0000","last_change":"2024-09-10T18:46:28.229804+0000","last_active":"2024-09-10T18:46:29.792098+0000","last_peered":"2024-09-10T18:46:29.792098+0000","last_clean":"2024-09-10T18:46:29.792098+0000","last_became_active":"2024-09-10T18:45:20.421097+0000","last_became_peered":"2024-09-10T18:45:20.421097+0000","last_unstale":"2024-09-10T18:46:29.792098+0000","last_undegraded":"2024-09-10T18:46:29.792098+0000","last_fullsized":"2024-09-10T18:46:29.792098+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:46:28.229737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:28.229737+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:49.859218+0000","scrub_duration":0.002308184,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.566531+0000","last_change":"2024-09-10T18:45:17.960491+0000","last_active":"2024-09-10T18:46:30.566531+0000","last_peered":"2024-09-10T18:46:30.566531+0000","last_clean":"2024-09-10T18:46:30.566531+0000","last_became_active":"2024-09-10T18:45:17.959924+0000","last_became_peered":"2024-09-10T18:45:17.959924+0000","last_unstale":"2024-09-10T18:46:30.566531+0000","last_undegraded":"2024-09-10T18:46:30.566531+0000","last_fullsized":"2024-09-10T18:46:30.566531+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:43.353373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.781831+0000","last_change":"2024-09-10T18:46:30.781786+0000","last_active":"2024-09-10T18:46:30.781831+0000","last_peered":"2024-09-10T18:46:30.781831+0000","last_clean":"2024-09-10T18:46:30.781831+0000","last_became_active":"2024-09-10T18:45:22.072581+0000","last_became_peered":"2024-09-10T18:45:22.072581+0000","last_unstale":"2024-09-10T18:46:30.781831+0000","last_undegraded":"2024-09-10T18:46:30.781831+0000","last_fullsized":"2024-09-10T18:46:30.781831+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:30.781730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:30.781730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:38.248319+0000","scrub_duration":0.0020255870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.567360+0000","last_change":"2024-09-10T18:45:24.076661+0000","last_active":"2024-09-10T18:46:30.567360+0000","last_peered":"2024-09-10T18:46:30.567360+0000","last_clean":"2024-09-10T18:46:30.567360+0000","last_became_active":"2024-09-10T18:45:24.076392+0000","last_became_peered":"2024-09-10T18:45:24.076392+0000","last_unstale":"2024-09-10T18:46:30.567360+0000","last_undegraded":"2024-09-10T18:46:30.567360+0000","last_fullsized":"2024-09-10T18:46:30.567360+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:47.306399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"63'9","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.741579+0000","last_change":"2024-09-10T18:45:20.061357+0000","last_active":"2024-09-10T18:46:29.741579+0000","last_peered":"2024-09-10T18:46:29.741579+0000","last_clean":"2024-09-10T18:46:29.741579+0000","last_became_active":"2024-09-10T18:45:20.061021+0000","last_became_peered":"2024-09-10T18:45:20.061021+0000","last_unstale":"2024-09-10T18:46:29.741579+0000","last_undegraded":"2024-09-10T18:46:29.741579+0000","last_fullsized":"2024-09-10T18:46:29.741579+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:35.559073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742729+0000","last_change":"2024-09-10T18:45:17.963854+0000","last_active":"2024-09-10T18:46:29.742729+0000","last_peered":"2024-09-10T18:46:29.742729+0000","last_clean":"2024-09-10T18:46:29.742729+0000","last_became_active":"2024-09-10T18:45:17.963057+0000","last_became_peered":"2024-09-10T18:45:17.963057+0000","last_unstale":"2024-09-10T18:46:29.742729+0000","last_undegraded":"2024-09-10T18:46:29.742729+0000","last_fullsized":"2024-09-10T18:46:29.742729+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:40.229149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.566905+0000","last_change":"2024-09-10T18:45:22.070613+0000","last_active":"2024-09-10T18:46:30.566905+0000","last_peered":"2024-09-10T18:46:30.566905+0000","last_clean":"2024-09-10T18:46:30.566905+0000","last_became_active":"2024-09-10T18:45:22.070343+0000","last_became_peered":"2024-09-10T18:45:22.070343+0000","last_unstale":"2024-09-10T18:46:30.566905+0000","last_undegraded":"2024-09-10T18:46:30.566905+0000","last_fullsized":"2024-09-10T18:46:30.566905+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:46.315675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.741416+0000","last_change":"2024-09-10T18:45:24.078848+0000","last_active":"2024-09-10T18:46:29.741416+0000","last_peered":"2024-09-10T18:46:29.741416+0000","last_clean":"2024-09-10T18:46:29.741416+0000","last_became_active":"2024-09-10T18:45:24.078529+0000","last_became_peered":"2024-09-10T18:45:24.078529+0000","last_unstale":"2024-09-10T18:46:29.741416+0000","last_undegraded":"2024-09-10T18:46:29.741416+0000","last_fullsized":"2024-09-10T18:46:29.741416+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"63'15","reported_seq":54,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.741514+0000","last_change":"2024-09-10T18:45:20.420130+0000","last_active":"2024-09-10T18:46:29.741514+0000","last_peered":"2024-09-10T18:46:29.741514+0000","last_clean":"2024-09-10T18:46:29.741514+0000","last_became_active":"2024-09-10T18:45:20.419804+0000","last_became_peered":"2024-09-10T18:45:20.419804+0000","last_unstale":"2024-09-10T18:46:29.741514+0000","last_undegraded":"2024-09-10T18:46:29.741514+0000","last_fullsized":"2024-09-10T18:46:29.741514+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:38.377852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.566473+0000","last_change":"2024-09-10T18:46:17.819966+0000","last_active":"2024-09-10T18:46:30.566473+0000","last_peered":"2024-09-10T18:46:30.566473+0000","last_clean":"2024-09-10T18:46:30.566473+0000","last_became_active":"2024-09-10T18:45:17.959653+0000","last_became_peered":"2024-09-10T18:45:17.959653+0000","last_unstale":"2024-09-10T18:46:30.566473+0000","last_undegraded":"2024-09-10T18:46:30.566473+0000","last_fullsized":"2024-09-10T18:46:30.566473+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:17.819909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:17.819909+0000","last_clean_scrub_stamp":"2024-09-10T18:46:17.819909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:18.454145+0000","scrub_duration":0.001612109,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"63'8","reported_seq":54,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:33.822272+0000","last_change":"2024-09-10T18:45:22.073687+0000","last_active":"2024-09-10T18:46:33.822272+0000","last_peered":"2024-09-10T18:46:33.822272+0000","last_clean":"2024-09-10T18:46:33.822272+0000","last_became_active":"2024-09-10T18:45:22.073383+0000","last_became_peered":"2024-09-10T18:45:22.073383+0000","last_unstale":"2024-09-10T18:46:33.822272+0000","last_undegraded":"2024-09-10T18:46:33.822272+0000","last_fullsized":"2024-09-10T18:46:33.822272+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:44.506736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.741681+0000","last_change":"2024-09-10T18:45:24.265594+0000","last_active":"2024-09-10T18:46:29.741681+0000","last_peered":"2024-09-10T18:46:29.741681+0000","last_clean":"2024-09-10T18:46:29.741681+0000","last_became_active":"2024-09-10T18:45:24.265256+0000","last_became_peered":"2024-09-10T18:45:24.265256+0000","last_unstale":"2024-09-10T18:46:29.741681+0000","last_undegraded":"2024-09-10T18:46:29.741681+0000","last_fullsized":"2024-09-10T18:46:29.741681+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:34.150976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"63'12","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.320580+0000","last_change":"2024-09-10T18:46:30.320527+0000","last_active":"2024-09-10T18:46:30.320580+0000","last_peered":"2024-09-10T18:46:30.320580+0000","last_clean":"2024-09-10T18:46:30.320580+0000","last_became_active":"2024-09-10T18:45:20.056869+0000","last_became_peered":"2024-09-10T18:45:20.056869+0000","last_unstale":"2024-09-10T18:46:30.320580+0000","last_undegraded":"2024-09-10T18:46:30.320580+0000","last_fullsized":"2024-09-10T18:46:30.320580+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:46:30.320462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:30.320462+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:35.180189+0000","scrub_duration":0.0028822850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"56'2","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.318913+0000","last_change":"2024-09-10T18:45:17.963577+0000","last_active":"2024-09-10T18:46:30.318913+0000","last_peered":"2024-09-10T18:46:30.318913+0000","last_clean":"2024-09-10T18:46:30.318913+0000","last_became_active":"2024-09-10T18:45:17.963286+0000","last_became_peered":"2024-09-10T18:45:17.963286+0000","last_unstale":"2024-09-10T18:46:30.318913+0000","last_undegraded":"2024-09-10T18:46:30.318913+0000","last_fullsized":"2024-09-10T18:46:30.318913+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:37.304000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.566744+0000","last_change":"2024-09-10T18:45:22.069278+0000","last_active":"2024-09-10T18:46:30.566744+0000","last_peered":"2024-09-10T18:46:30.566744+0000","last_clean":"2024-09-10T18:46:30.566744+0000","last_became_active":"2024-09-10T18:45:22.068957+0000","last_became_peered":"2024-09-10T18:45:22.068957+0000","last_unstale":"2024-09-10T18:46:30.566744+0000","last_undegraded":"2024-09-10T18:46:30.566744+0000","last_fullsized":"2024-09-10T18:46:30.566744+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:41.217006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.790914+0000","last_change":"2024-09-10T18:45:24.078134+0000","last_active":"2024-09-10T18:46:29.790914+0000","last_peered":"2024-09-10T18:46:29.790914+0000","last_clean":"2024-09-10T18:46:29.790914+0000","last_became_active":"2024-09-10T18:45:24.077815+0000","last_became_peered":"2024-09-10T18:45:24.077815+0000","last_unstale":"2024-09-10T18:46:29.790914+0000","last_undegraded":"2024-09-10T18:46:29.790914+0000","last_fullsized":"2024-09-10T18:46:29.790914+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:49.952536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"76'13","reported_seq":59,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742416+0000","last_change":"2024-09-10T18:46:24.626912+0000","last_active":"2024-09-10T18:46:29.742416+0000","last_peered":"2024-09-10T18:46:29.742416+0000","last_clean":"2024-09-10T18:46:29.742416+0000","last_became_active":"2024-09-10T18:45:20.420421+0000","last_became_peered":"2024-09-10T18:45:20.420421+0000","last_unstale":"2024-09-10T18:46:29.742416+0000","last_undegraded":"2024-09-10T18:46:29.742416+0000","last_fullsized":"2024-09-10T18:46:29.742416+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'13","last_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_deep_scrub":"76'13","last_deep_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_clean_scrub_stamp":"2024-09-10T18:46:24.626859+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:41.087073+0000","scrub_duration":0.0035642949999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:31.734562+0000","last_change":"2024-09-10T18:46:31.734522+0000","last_active":"2024-09-10T18:46:31.734562+0000","last_peered":"2024-09-10T18:46:31.734562+0000","last_clean":"2024-09-10T18:46:31.734562+0000","last_became_active":"2024-09-10T18:45:17.965112+0000","last_became_peered":"2024-09-10T18:45:17.965112+0000","last_unstale":"2024-09-10T18:46:31.734562+0000","last_undegraded":"2024-09-10T18:46:31.734562+0000","last_fullsized":"2024-09-10T18:46:31.734562+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:31.734459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:31.734459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:41.550267+0000","scrub_duration":0.0015571960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:32.829641+0000","last_change":"2024-09-10T18:46:32.829583+0000","last_active":"2024-09-10T18:46:32.829641+0000","last_peered":"2024-09-10T18:46:32.829641+0000","last_clean":"2024-09-10T18:46:32.829641+0000","last_became_active":"2024-09-10T18:45:22.069040+0000","last_became_peered":"2024-09-10T18:45:22.069040+0000","last_unstale":"2024-09-10T18:46:32.829641+0000","last_undegraded":"2024-09-10T18:46:32.829641+0000","last_fullsized":"2024-09-10T18:46:32.829641+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:32.829508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:32.829508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:45.869787+0000","scrub_duration":0.001897239,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"63'1","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.567035+0000","last_change":"2024-09-10T18:45:24.079638+0000","last_active":"2024-09-10T18:46:30.567035+0000","last_peered":"2024-09-10T18:46:30.567035+0000","last_clean":"2024-09-10T18:46:30.567035+0000","last_became_active":"2024-09-10T18:45:24.079258+0000","last_became_peered":"2024-09-10T18:45:24.079258+0000","last_unstale":"2024-09-10T18:46:30.567035+0000","last_undegraded":"2024-09-10T18:46:30.567035+0000","last_fullsized":"2024-09-10T18:46:30.567035+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:50.668523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.792234+0000","last_change":"2024-09-10T18:45:17.965220+0000","last_active":"2024-09-10T18:46:29.792234+0000","last_peered":"2024-09-10T18:46:29.792234+0000","last_clean":"2024-09-10T18:46:29.792234+0000","last_became_active":"2024-09-10T18:45:17.964705+0000","last_became_peered":"2024-09-10T18:45:17.964705+0000","last_unstale":"2024-09-10T18:46:29.792234+0000","last_undegraded":"2024-09-10T18:46:29.792234+0000","last_fullsized":"2024-09-10T18:46:29.792234+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:45.947390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'6","reported_seq":38,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.791411+0000","last_change":"2024-09-10T18:45:20.422485+0000","last_active":"2024-09-10T18:46:29.791411+0000","last_peered":"2024-09-10T18:46:29.791411+0000","last_clean":"2024-09-10T18:46:29.791411+0000","last_became_active":"2024-09-10T18:45:20.421746+0000","last_became_peered":"2024-09-10T18:45:20.421746+0000","last_unstale":"2024-09-10T18:46:29.791411+0000","last_undegraded":"2024-09-10T18:46:29.791411+0000","last_fullsized":"2024-09-10T18:46:29.791411+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:47.024991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:39.084522+0000","last_change":"2024-09-10T18:46:39.084489+0000","last_active":"2024-09-10T18:46:39.084522+0000","last_peered":"2024-09-10T18:46:39.084522+0000","last_clean":"2024-09-10T18:46:39.084522+0000","last_became_active":"2024-09-10T18:45:22.069653+0000","last_became_peered":"2024-09-10T18:45:22.069653+0000","last_unstale":"2024-09-10T18:46:39.084522+0000","last_undegraded":"2024-09-10T18:46:39.084522+0000","last_fullsized":"2024-09-10T18:46:39.084522+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:39.084433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:39.084433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:05.431757+0000","scrub_duration":0.0014845819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.567759+0000","last_change":"2024-09-10T18:45:24.264869+0000","last_active":"2024-09-10T18:46:30.567759+0000","last_peered":"2024-09-10T18:46:30.567759+0000","last_clean":"2024-09-10T18:46:30.567759+0000","last_became_active":"2024-09-10T18:45:24.264611+0000","last_became_peered":"2024-09-10T18:45:24.264611+0000","last_unstale":"2024-09-10T18:46:30.567759+0000","last_undegraded":"2024-09-10T18:46:30.567759+0000","last_fullsized":"2024-09-10T18:46:30.567759+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:35.281392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"63'10","reported_seq":44,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.791515+0000","last_change":"2024-09-10T18:45:20.420217+0000","last_active":"2024-09-10T18:46:29.791515+0000","last_peered":"2024-09-10T18:46:29.791515+0000","last_clean":"2024-09-10T18:46:29.791515+0000","last_became_active":"2024-09-10T18:45:20.419887+0000","last_became_peered":"2024-09-10T18:45:20.419887+0000","last_unstale":"2024-09-10T18:46:29.791515+0000","last_undegraded":"2024-09-10T18:46:29.791515+0000","last_fullsized":"2024-09-10T18:46:29.791515+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:39.004054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.741925+0000","last_change":"2024-09-10T18:45:17.965293+0000","last_active":"2024-09-10T18:46:29.741925+0000","last_peered":"2024-09-10T18:46:29.741925+0000","last_clean":"2024-09-10T18:46:29.741925+0000","last_became_active":"2024-09-10T18:45:17.964870+0000","last_became_peered":"2024-09-10T18:45:17.964870+0000","last_unstale":"2024-09-10T18:46:29.741925+0000","last_undegraded":"2024-09-10T18:46:29.741925+0000","last_fullsized":"2024-09-10T18:46:29.741925+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:37.495417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"63'8","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:34.085287+0000","last_change":"2024-09-10T18:46:34.085224+0000","last_active":"2024-09-10T18:46:34.085287+0000","last_peered":"2024-09-10T18:46:34.085287+0000","last_clean":"2024-09-10T18:46:34.085287+0000","last_became_active":"2024-09-10T18:45:22.062940+0000","last_became_peered":"2024-09-10T18:45:22.062940+0000","last_unstale":"2024-09-10T18:46:34.085287+0000","last_undegraded":"2024-09-10T18:46:34.085287+0000","last_fullsized":"2024-09-10T18:46:34.085287+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:34.085156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:34.085156+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:54.777061+0000","scrub_duration":0.0020843120000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.741832+0000","last_change":"2024-09-10T18:45:24.074039+0000","last_active":"2024-09-10T18:46:29.741832+0000","last_peered":"2024-09-10T18:46:29.741832+0000","last_clean":"2024-09-10T18:46:29.741832+0000","last_became_active":"2024-09-10T18:45:24.073736+0000","last_became_peered":"2024-09-10T18:45:24.073736+0000","last_unstale":"2024-09-10T18:46:29.741832+0000","last_undegraded":"2024-09-10T18:46:29.741832+0000","last_fullsized":"2024-09-10T18:46:29.741832+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:47.511899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"63'9","reported_seq":55,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742224+0000","last_change":"2024-09-10T18:46:23.086274+0000","last_active":"2024-09-10T18:46:29.742224+0000","last_peered":"2024-09-10T18:46:29.742224+0000","last_clean":"2024-09-10T18:46:29.742224+0000","last_became_active":"2024-09-10T18:45:20.421124+0000","last_became_peered":"2024-09-10T18:45:20.421124+0000","last_unstale":"2024-09-10T18:46:29.742224+0000","last_undegraded":"2024-09-10T18:46:29.742224+0000","last_fullsized":"2024-09-10T18:46:29.742224+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:46:23.086203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:23.086203+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:44.793257+0000","scrub_duration":0.002168068,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.741702+0000","last_change":"2024-09-10T18:45:17.960387+0000","last_active":"2024-09-10T18:46:29.741702+0000","last_peered":"2024-09-10T18:46:29.741702+0000","last_clean":"2024-09-10T18:46:29.741702+0000","last_became_active":"2024-09-10T18:45:17.960106+0000","last_became_peered":"2024-09-10T18:45:17.960106+0000","last_unstale":"2024-09-10T18:46:29.741702+0000","last_undegraded":"2024-09-10T18:46:29.741702+0000","last_fullsized":"2024-09-10T18:46:29.741702+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"63'8","reported_seq":51,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:33.593126+0000","last_change":"2024-09-10T18:45:22.075677+0000","last_active":"2024-09-10T18:46:33.593126+0000","last_peered":"2024-09-10T18:46:33.593126+0000","last_clean":"2024-09-10T18:46:33.593126+0000","last_became_active":"2024-09-10T18:45:22.075368+0000","last_became_peered":"2024-09-10T18:45:22.075368+0000","last_unstale":"2024-09-10T18:46:33.593126+0000","last_undegraded":"2024-09-10T18:46:33.593126+0000","last_fullsized":"2024-09-10T18:46:33.593126+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:39.560512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:36.305295+0000","last_change":"2024-09-10T18:46:36.305262+0000","last_active":"2024-09-10T18:46:36.305295+0000","last_peered":"2024-09-10T18:46:36.305295+0000","last_clean":"2024-09-10T18:46:36.305295+0000","last_became_active":"2024-09-10T18:45:24.080196+0000","last_became_peered":"2024-09-10T18:45:24.080196+0000","last_unstale":"2024-09-10T18:46:36.305295+0000","last_undegraded":"2024-09-10T18:46:36.305295+0000","last_fullsized":"2024-09-10T18:46:36.305295+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_clean_scrub_stamp":"2024-09-10T18:46:36.305199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:04.747507+0000","scrub_duration":0.0013794809999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"63'9","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.741371+0000","last_change":"2024-09-10T18:45:20.057767+0000","last_active":"2024-09-10T18:46:29.741371+0000","last_peered":"2024-09-10T18:46:29.741371+0000","last_clean":"2024-09-10T18:46:29.741371+0000","last_became_active":"2024-09-10T18:45:20.057479+0000","last_became_peered":"2024-09-10T18:45:20.057479+0000","last_unstale":"2024-09-10T18:46:29.741371+0000","last_undegraded":"2024-09-10T18:46:29.741371+0000","last_fullsized":"2024-09-10T18:46:29.741371+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:40.060026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.792260+0000","last_change":"2024-09-10T18:46:23.292276+0000","last_active":"2024-09-10T18:46:29.792260+0000","last_peered":"2024-09-10T18:46:29.792260+0000","last_clean":"2024-09-10T18:46:29.792260+0000","last_became_active":"2024-09-10T18:45:17.962009+0000","last_became_peered":"2024-09-10T18:45:17.962009+0000","last_unstale":"2024-09-10T18:46:29.792260+0000","last_undegraded":"2024-09-10T18:46:29.792260+0000","last_fullsized":"2024-09-10T18:46:29.792260+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:23.292207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:23.292207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:27.416585+0000","scrub_duration":0.001659953,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.791128+0000","last_change":"2024-09-10T18:45:22.073186+0000","last_active":"2024-09-10T18:46:29.791128+0000","last_peered":"2024-09-10T18:46:29.791128+0000","last_clean":"2024-09-10T18:46:29.791128+0000","last_became_active":"2024-09-10T18:45:22.072877+0000","last_became_peered":"2024-09-10T18:45:22.072877+0000","last_unstale":"2024-09-10T18:46:29.791128+0000","last_undegraded":"2024-09-10T18:46:29.791128+0000","last_fullsized":"2024-09-10T18:46:29.791128+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:47.886075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.740781+0000","last_change":"2024-09-10T18:45:24.081786+0000","last_active":"2024-09-10T18:46:29.740781+0000","last_peered":"2024-09-10T18:46:29.740781+0000","last_clean":"2024-09-10T18:46:29.740781+0000","last_became_active":"2024-09-10T18:45:24.081050+0000","last_became_peered":"2024-09-10T18:45:24.081050+0000","last_unstale":"2024-09-10T18:46:29.740781+0000","last_undegraded":"2024-09-10T18:46:29.740781+0000","last_fullsized":"2024-09-10T18:46:29.740781+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:35.498799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"63'11","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.319566+0000","last_change":"2024-09-10T18:45:20.057563+0000","last_active":"2024-09-10T18:46:30.319566+0000","last_peered":"2024-09-10T18:46:30.319566+0000","last_clean":"2024-09-10T18:46:30.319566+0000","last_became_active":"2024-09-10T18:45:20.056885+0000","last_became_peered":"2024-09-10T18:45:20.056885+0000","last_unstale":"2024-09-10T18:46:30.319566+0000","last_undegraded":"2024-09-10T18:46:30.319566+0000","last_fullsized":"2024-09-10T18:46:30.319566+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:47.219111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:36.402643+0000","last_change":"2024-09-10T18:46:36.402610+0000","last_active":"2024-09-10T18:46:36.402643+0000","last_peered":"2024-09-10T18:46:36.402643+0000","last_clean":"2024-09-10T18:46:36.402643+0000","last_became_active":"2024-09-10T18:45:17.963542+0000","last_became_peered":"2024-09-10T18:45:17.963542+0000","last_unstale":"2024-09-10T18:46:36.402643+0000","last_undegraded":"2024-09-10T18:46:36.402643+0000","last_fullsized":"2024-09-10T18:46:36.402643+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:36.402548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:36.402548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:04.495178+0000","scrub_duration":0.0014103919999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:37.414825+0000","last_change":"2024-09-10T18:46:37.414794+0000","last_active":"2024-09-10T18:46:37.414825+0000","last_peered":"2024-09-10T18:46:37.414825+0000","last_clean":"2024-09-10T18:46:37.414825+0000","last_became_active":"2024-09-10T18:45:22.070065+0000","last_became_peered":"2024-09-10T18:45:22.070065+0000","last_unstale":"2024-09-10T18:46:37.414825+0000","last_undegraded":"2024-09-10T18:46:37.414825+0000","last_fullsized":"2024-09-10T18:46:37.414825+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:37.414734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:37.414734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:44.027335+0000","scrub_duration":0.001607996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.790714+0000","last_change":"2024-09-10T18:45:24.072477+0000","last_active":"2024-09-10T18:46:29.790714+0000","last_peered":"2024-09-10T18:46:29.790714+0000","last_clean":"2024-09-10T18:46:29.790714+0000","last_became_active":"2024-09-10T18:45:24.072224+0000","last_became_peered":"2024-09-10T18:45:24.072224+0000","last_unstale":"2024-09-10T18:46:29.790714+0000","last_undegraded":"2024-09-10T18:46:29.790714+0000","last_fullsized":"2024-09-10T18:46:29.790714+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:47.793658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"63'4","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.790674+0000","last_change":"2024-09-10T18:45:20.059495+0000","last_active":"2024-09-10T18:46:29.790674+0000","last_peered":"2024-09-10T18:46:29.790674+0000","last_clean":"2024-09-10T18:46:29.790674+0000","last_became_active":"2024-09-10T18:45:20.059205+0000","last_became_peered":"2024-09-10T18:45:20.059205+0000","last_unstale":"2024-09-10T18:46:29.790674+0000","last_undegraded":"2024-09-10T18:46:29.790674+0000","last_fullsized":"2024-09-10T18:46:29.790674+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:33.725425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742392+0000","last_change":"2024-09-10T18:45:17.963898+0000","last_active":"2024-09-10T18:46:29.742392+0000","last_peered":"2024-09-10T18:46:29.742392+0000","last_clean":"2024-09-10T18:46:29.742392+0000","last_became_active":"2024-09-10T18:45:17.963630+0000","last_became_peered":"2024-09-10T18:45:17.963630+0000","last_unstale":"2024-09-10T18:46:29.742392+0000","last_undegraded":"2024-09-10T18:46:29.742392+0000","last_fullsized":"2024-09-10T18:46:29.742392+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:44.130712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:35.172968+0000","last_change":"2024-09-10T18:46:35.172936+0000","last_active":"2024-09-10T18:46:35.172968+0000","last_peered":"2024-09-10T18:46:35.172968+0000","last_clean":"2024-09-10T18:46:35.172968+0000","last_became_active":"2024-09-10T18:45:22.072377+0000","last_became_peered":"2024-09-10T18:45:22.072377+0000","last_unstale":"2024-09-10T18:46:35.172968+0000","last_undegraded":"2024-09-10T18:46:35.172968+0000","last_fullsized":"2024-09-10T18:46:35.172968+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_clean_scrub_stamp":"2024-09-10T18:46:35.172861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:57.638389+0000","scrub_duration":0.001552933,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.567431+0000","last_change":"2024-09-10T18:45:24.077706+0000","last_active":"2024-09-10T18:46:30.567431+0000","last_peered":"2024-09-10T18:46:30.567431+0000","last_clean":"2024-09-10T18:46:30.567431+0000","last_became_active":"2024-09-10T18:45:24.077398+0000","last_became_peered":"2024-09-10T18:45:24.077398+0000","last_unstale":"2024-09-10T18:46:30.567431+0000","last_undegraded":"2024-09-10T18:46:30.567431+0000","last_fullsized":"2024-09-10T18:46:30.567431+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:35.781159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"63'11","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.790845+0000","last_change":"2024-09-10T18:45:20.422212+0000","last_active":"2024-09-10T18:46:29.790845+0000","last_peered":"2024-09-10T18:46:29.790845+0000","last_clean":"2024-09-10T18:46:29.790845+0000","last_became_active":"2024-09-10T18:45:20.421479+0000","last_became_peered":"2024-09-10T18:45:20.421479+0000","last_unstale":"2024-09-10T18:46:29.790845+0000","last_undegraded":"2024-09-10T18:46:29.790845+0000","last_fullsized":"2024-09-10T18:46:29.790845+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:38.573185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"56'1","reported_seq":38,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.792614+0000","last_change":"2024-09-10T18:45:17.961527+0000","last_active":"2024-09-10T18:46:29.792614+0000","last_peered":"2024-09-10T18:46:29.792614+0000","last_clean":"2024-09-10T18:46:29.792614+0000","last_became_active":"2024-09-10T18:45:17.961222+0000","last_became_peered":"2024-09-10T18:45:17.961222+0000","last_unstale":"2024-09-10T18:46:29.792614+0000","last_undegraded":"2024-09-10T18:46:29.792614+0000","last_fullsized":"2024-09-10T18:46:29.792614+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:38.797176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.567191+0000","last_change":"2024-09-10T18:46:27.667088+0000","last_active":"2024-09-10T18:46:30.567191+0000","last_peered":"2024-09-10T18:46:30.567191+0000","last_clean":"2024-09-10T18:46:30.567191+0000","last_became_active":"2024-09-10T18:45:22.072312+0000","last_became_peered":"2024-09-10T18:45:22.072312+0000","last_unstale":"2024-09-10T18:46:30.567191+0000","last_undegraded":"2024-09-10T18:46:30.567191+0000","last_fullsized":"2024-09-10T18:46:30.567191+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:27.667018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:27.667018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:38.065664+0000","scrub_duration":0.001388886,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.791673+0000","last_change":"2024-09-10T18:46:29.791617+0000","last_active":"2024-09-10T18:46:29.791673+0000","last_peered":"2024-09-10T18:46:29.791673+0000","last_clean":"2024-09-10T18:46:29.791673+0000","last_became_active":"2024-09-10T18:45:24.264818+0000","last_became_peered":"2024-09-10T18:45:24.264818+0000","last_unstale":"2024-09-10T18:46:29.791673+0000","last_undegraded":"2024-09-10T18:46:29.791673+0000","last_fullsized":"2024-09-10T18:46:29.791673+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:29.791563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:29.791563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:37.941643+0000","scrub_duration":0.0021031819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"78'2","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:37.581259+0000","last_change":"2024-09-10T18:46:37.581201+0000","last_active":"2024-09-10T18:46:37.581259+0000","last_peered":"2024-09-10T18:46:37.581259+0000","last_clean":"2024-09-10T18:46:37.581259+0000","last_became_active":"2024-09-10T18:45:24.076380+0000","last_became_peered":"2024-09-10T18:45:24.076380+0000","last_unstale":"2024-09-10T18:46:37.581259+0000","last_undegraded":"2024-09-10T18:46:37.581259+0000","last_fullsized":"2024-09-10T18:46:37.581259+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'2","last_scrub_stamp":"2024-09-10T18:46:37.581130+0000","last_deep_scrub":"78'2","last_deep_scrub_stamp":"2024-09-10T18:46:37.581130+0000","last_clean_scrub_stamp":"2024-09-10T18:46:37.581130+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:55.887929+0000","scrub_duration":0.0024563609999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.741574+0000","last_change":"2024-09-10T18:46:28.088657+0000","last_active":"2024-09-10T18:46:29.741574+0000","last_peered":"2024-09-10T18:46:29.741574+0000","last_clean":"2024-09-10T18:46:29.741574+0000","last_became_active":"2024-09-10T18:45:17.962108+0000","last_became_peered":"2024-09-10T18:45:17.962108+0000","last_unstale":"2024-09-10T18:46:29.741574+0000","last_undegraded":"2024-09-10T18:46:29.741574+0000","last_fullsized":"2024-09-10T18:46:29.741574+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:28.088588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:28.088588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:56.550863+0000","scrub_duration":0.00140508,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"63'10","reported_seq":44,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742320+0000","last_change":"2024-09-10T18:45:20.421384+0000","last_active":"2024-09-10T18:46:29.742320+0000","last_peered":"2024-09-10T18:46:29.742320+0000","last_clean":"2024-09-10T18:46:29.742320+0000","last_became_active":"2024-09-10T18:45:20.421094+0000","last_became_peered":"2024-09-10T18:45:20.421094+0000","last_unstale":"2024-09-10T18:46:29.742320+0000","last_undegraded":"2024-09-10T18:46:29.742320+0000","last_fullsized":"2024-09-10T18:46:29.742320+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:34.663846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":64,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:33.471933+0000","last_change":"2024-09-10T18:46:33.150919+0000","last_active":"2024-09-10T18:46:33.471933+0000","last_peered":"2024-09-10T18:46:33.471933+0000","last_clean":"2024-09-10T18:46:33.471933+0000","last_became_active":"2024-09-10T18:45:22.073281+0000","last_became_peered":"2024-09-10T18:45:22.073281+0000","last_unstale":"2024-09-10T18:46:33.471933+0000","last_undegraded":"2024-09-10T18:46:33.471933+0000","last_fullsized":"2024-09-10T18:46:33.471933+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:33.150852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:33.150852+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:51.299257+0000","scrub_duration":0.0017025339999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.741108+0000","last_change":"2024-09-10T18:45:24.082638+0000","last_active":"2024-09-10T18:46:29.741108+0000","last_peered":"2024-09-10T18:46:29.741108+0000","last_clean":"2024-09-10T18:46:29.741108+0000","last_became_active":"2024-09-10T18:45:24.082318+0000","last_became_peered":"2024-09-10T18:45:24.082318+0000","last_unstale":"2024-09-10T18:46:29.741108+0000","last_undegraded":"2024-09-10T18:46:29.741108+0000","last_fullsized":"2024-09-10T18:46:29.741108+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:42.162847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.791133+0000","last_change":"2024-09-10T18:45:17.966027+0000","last_active":"2024-09-10T18:46:29.791133+0000","last_peered":"2024-09-10T18:46:29.791133+0000","last_clean":"2024-09-10T18:46:29.791133+0000","last_became_active":"2024-09-10T18:45:17.965707+0000","last_became_peered":"2024-09-10T18:45:17.965707+0000","last_unstale":"2024-09-10T18:46:29.791133+0000","last_undegraded":"2024-09-10T18:46:29.791133+0000","last_fullsized":"2024-09-10T18:46:29.791133+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:33.654064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"63'11","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.792605+0000","last_change":"2024-09-10T18:45:20.059017+0000","last_active":"2024-09-10T18:46:29.792605+0000","last_peered":"2024-09-10T18:46:29.792605+0000","last_clean":"2024-09-10T18:46:29.792605+0000","last_became_active":"2024-09-10T18:45:20.058717+0000","last_became_peered":"2024-09-10T18:45:20.058717+0000","last_unstale":"2024-09-10T18:46:29.792605+0000","last_undegraded":"2024-09-10T18:46:29.792605+0000","last_fullsized":"2024-09-10T18:46:29.792605+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:42.592911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742318+0000","last_change":"2024-09-10T18:45:22.068991+0000","last_active":"2024-09-10T18:46:29.742318+0000","last_peered":"2024-09-10T18:46:29.742318+0000","last_clean":"2024-09-10T18:46:29.742318+0000","last_became_active":"2024-09-10T18:45:22.068676+0000","last_became_peered":"2024-09-10T18:45:22.068676+0000","last_unstale":"2024-09-10T18:46:29.742318+0000","last_undegraded":"2024-09-10T18:46:29.742318+0000","last_fullsized":"2024-09-10T18:46:29.742318+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:46.800364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.319983+0000","last_change":"2024-09-10T18:45:24.267570+0000","last_active":"2024-09-10T18:46:30.319983+0000","last_peered":"2024-09-10T18:46:30.319983+0000","last_clean":"2024-09-10T18:46:30.319983+0000","last_became_active":"2024-09-10T18:45:24.267251+0000","last_became_peered":"2024-09-10T18:45:24.267251+0000","last_unstale":"2024-09-10T18:46:30.319983+0000","last_undegraded":"2024-09-10T18:46:30.319983+0000","last_fullsized":"2024-09-10T18:46:30.319983+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:41.357380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"56'1","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.792272+0000","last_change":"2024-09-10T18:45:17.966530+0000","last_active":"2024-09-10T18:46:29.792272+0000","last_peered":"2024-09-10T18:46:29.792272+0000","last_clean":"2024-09-10T18:46:29.792272+0000","last_became_active":"2024-09-10T18:45:17.966286+0000","last_became_peered":"2024-09-10T18:45:17.966286+0000","last_unstale":"2024-09-10T18:46:29.792272+0000","last_undegraded":"2024-09-10T18:46:29.792272+0000","last_fullsized":"2024-09-10T18:46:29.792272+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:41.758824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"63'15","reported_seq":64,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.567321+0000","last_change":"2024-09-10T18:46:23.864133+0000","last_active":"2024-09-10T18:46:30.567321+0000","last_peered":"2024-09-10T18:46:30.567321+0000","last_clean":"2024-09-10T18:46:30.567321+0000","last_became_active":"2024-09-10T18:45:20.062728+0000","last_became_peered":"2024-09-10T18:45:20.062728+0000","last_unstale":"2024-09-10T18:46:30.567321+0000","last_undegraded":"2024-09-10T18:46:30.567321+0000","last_fullsized":"2024-09-10T18:46:30.567321+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:46:23.864075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:23.864075+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:29.106766+0000","scrub_duration":0.0020859780000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742537+0000","last_change":"2024-09-10T18:45:22.060202+0000","last_active":"2024-09-10T18:46:29.742537+0000","last_peered":"2024-09-10T18:46:29.742537+0000","last_clean":"2024-09-10T18:46:29.742537+0000","last_became_active":"2024-09-10T18:45:22.059794+0000","last_became_peered":"2024-09-10T18:45:22.059794+0000","last_unstale":"2024-09-10T18:46:29.742537+0000","last_undegraded":"2024-09-10T18:46:29.742537+0000","last_fullsized":"2024-09-10T18:46:29.742537+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:44.574409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.791325+0000","last_change":"2024-09-10T18:45:24.264775+0000","last_active":"2024-09-10T18:46:29.791325+0000","last_peered":"2024-09-10T18:46:29.791325+0000","last_clean":"2024-09-10T18:46:29.791325+0000","last_became_active":"2024-09-10T18:45:24.264427+0000","last_became_peered":"2024-09-10T18:45:24.264427+0000","last_unstale":"2024-09-10T18:46:29.791325+0000","last_undegraded":"2024-09-10T18:46:29.791325+0000","last_fullsized":"2024-09-10T18:46:29.791325+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:44.649431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.791165+0000","last_change":"2024-09-10T18:45:17.963530+0000","last_active":"2024-09-10T18:46:29.791165+0000","last_peered":"2024-09-10T18:46:29.791165+0000","last_clean":"2024-09-10T18:46:29.791165+0000","last_became_active":"2024-09-10T18:45:17.963257+0000","last_became_peered":"2024-09-10T18:45:17.963257+0000","last_unstale":"2024-09-10T18:46:29.791165+0000","last_undegraded":"2024-09-10T18:46:29.791165+0000","last_fullsized":"2024-09-10T18:46:29.791165+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:33.981037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"63'12","reported_seq":52,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.791203+0000","last_change":"2024-09-10T18:45:20.058693+0000","last_active":"2024-09-10T18:46:29.791203+0000","last_peered":"2024-09-10T18:46:29.791203+0000","last_clean":"2024-09-10T18:46:29.791203+0000","last_became_active":"2024-09-10T18:45:20.058391+0000","last_became_peered":"2024-09-10T18:45:20.058391+0000","last_unstale":"2024-09-10T18:46:29.791203+0000","last_undegraded":"2024-09-10T18:46:29.791203+0000","last_fullsized":"2024-09-10T18:46:29.791203+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:39.298321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:34.284014+0000","last_change":"2024-09-10T18:46:34.283979+0000","last_active":"2024-09-10T18:46:34.284014+0000","last_peered":"2024-09-10T18:46:34.284014+0000","last_clean":"2024-09-10T18:46:34.284014+0000","last_became_active":"2024-09-10T18:45:22.073250+0000","last_became_peered":"2024-09-10T18:45:22.073250+0000","last_unstale":"2024-09-10T18:46:34.284014+0000","last_undegraded":"2024-09-10T18:46:34.284014+0000","last_fullsized":"2024-09-10T18:46:34.284014+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:34.283901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:34.283901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:34.686745+0000","scrub_duration":0.002360334,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":395,"ondisk_log_size":395,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":70,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299413,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298724,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446428,"statfs":{"total":95995035648,"available":95689142272,"internally_reserved":0,"allocated":1404928,"data_stored":921391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.66600000000000004}]},{"osd":1,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.68700000000000006}]},{"osd":2,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.64900000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.622},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.55100000000000005}]},{"osd":4,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.379},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.39600000000000002}]},{"osd":5,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.36299999999999999}]},{"osd":6,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.434},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.45600000000000002}]}]},{"osd":6,"up_from":43,"seq":184683593754,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298712,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446440,"statfs":{"total":95995035648,"available":95689154560,"internally_reserved":0,"allocated":1392640,"data_stored":921467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:45:41 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.53300000000000003}]},{"osd":1,"last update":"Tue Sep 10 18:45:41 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.503}]},{"osd":2,"last update":"Tue Sep 10 18:45:41 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.51700000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:45:41 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.60499999999999998}]},{"osd":4,"last update":"Tue Sep 10 18:45:41 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.436}]},{"osd":5,"last update":"Tue Sep 10 18:45:41 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.36299999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.45600000000000002}]}]},{"osd":5,"up_from":37,"seq":158913789982,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298120,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447032,"statfs":{"total":95995035648,"available":95689760768,"internally_reserved":0,"allocated":786432,"data_stored":329602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.52200000000000002,"15min":0.52600000000000002},"min":{"1min":0.30299999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70099999999999996,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.58299999999999996,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.38200000000000001}]},{"osd":1,"last update":"Tue Sep 10 18:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.55200000000000005,"15min":0.56000000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.69699999999999995,"5min":0.87,"15min":0.87},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.64200000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.36399999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.51800000000000002,"15min":0.52000000000000002},"min":{"1min":0.33300000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.71299999999999997,"5min":0.755,"15min":0.755},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53600000000000003,"15min":0.54100000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79600000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.39000000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52500000000000002,"15min":0.52700000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76100000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.375},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.54200000000000004,"15min":0.54600000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80100000000000005,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.41799999999999998}]},{"osd":4,"last update":"Tue Sep 10 18:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.64500000000000002,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.40500000000000003,"15min":0.40300000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.65700000000000003,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.32000000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:45:42 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.40300000000000002}]},{"osd":7,"last update":"Tue Sep 10 18:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.34000000000000002}]}]},{"osd":4,"up_from":30,"seq":128849018916,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298144,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447008,"statfs":{"total":95995035648,"available":95689736192,"internally_reserved":0,"allocated":811008,"data_stored":330559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.36399999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.433},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.54000000000000004,"15min":0.54200000000000004},"min":{"1min":0.34399999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.44800000000000001}]},{"osd":1,"last update":"Tue Sep 10 18:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.57999999999999996,"15min":0.57599999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.609,"5min":2.609,"15min":2.609},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.54700000000000004,"15min":0.53200000000000003},"min":{"1min":0.39700000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.589,"5min":2.589,"15min":2.589},"last":0.56000000000000005}]},{"osd":2,"last update":"Tue Sep 10 18:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.55700000000000005,"15min":0.54400000000000004},"min":{"1min":0.40500000000000003,"5min":0.379,"15min":0.379},"max":{"1min":2.6030000000000002,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.54800000000000004,"15min":0.55400000000000005},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64000000000000001,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.52700000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52800000000000002,"15min":0.53000000000000003},"min":{"1min":0.36699999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68300000000000005,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.57599999999999996}]},{"osd":5,"last update":"Tue Sep 10 18:46:22 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.35899999999999999,"15min":0.34999999999999998},"min":{"1min":0.26800000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.44,"5min":0.371,"15min":0.35899999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.58399999999999996}]},{"osd":6,"last update":"Tue Sep 10 18:45:42 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.48299999999999998}]},{"osd":7,"last update":"Tue Sep 10 18:46:08 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.54400000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182441,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298156,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446996,"statfs":{"total":95995035648,"available":95689723904,"internally_reserved":0,"allocated":823296,"data_stored":330338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:35 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.378,"15min":0.38400000000000001},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.40100000000000002,"15min":0.40300000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.28699999999999998}]},{"osd":1,"last update":"Tue Sep 10 18:46:35 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.40400000000000003,"15min":0.40899999999999997},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40899999999999997,"15min":0.41499999999999998},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.63800000000000001,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.41799999999999998}]},{"osd":2,"last update":"Tue Sep 10 18:46:35 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.43099999999999999,"15min":0.438},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.41999999999999998,"15min":0.433},"min":{"1min":0.27200000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.53600000000000003,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.44700000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54000000000000004,"15min":0.54300000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.67200000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.56599999999999995,"15min":0.57199999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.71699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.54000000000000004}]},{"osd":5,"last update":"Tue Sep 10 18:46:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39700000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.70299999999999996,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.435},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.41399999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80800000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.52700000000000002}]},{"osd":6,"last update":"Tue Sep 10 18:45:41 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.51200000000000001}]},{"osd":7,"last update":"Tue Sep 10 18:46:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.495}]}]},{"osd":2,"up_from":18,"seq":77309411373,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298124,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447028,"statfs":{"total":95995035648,"available":95689756672,"internally_reserved":0,"allocated":790528,"data_stored":330037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.376,"15min":0.35899999999999999},"min":{"1min":0.246,"5min":0.223,"15min":0.223},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.371},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.38300000000000001,"15min":0.371},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.26000000000000001}]},{"osd":1,"last update":"Tue Sep 10 18:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.375,"15min":0.36699999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.42499999999999999,"15min":0.41299999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.40000000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:46:36 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46800000000000003,"15min":0.46500000000000002},"min":{"1min":0.30099999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.42899999999999999,"15min":0.43099999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.32300000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:45:59 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54900000000000004,"15min":0.54800000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.55700000000000005}]},{"osd":5,"last update":"Tue Sep 10 18:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56100000000000005,"15min":0.55800000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.62},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.42899999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.53400000000000003}]},{"osd":6,"last update":"Tue Sep 10 18:45:45 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.59699999999999998}]},{"osd":7,"last update":"Tue Sep 10 18:46:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.58299999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298168,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446984,"statfs":{"total":95995035648,"available":95689711616,"internally_reserved":0,"allocated":835584,"data_stored":330833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.33600000000000002,"15min":0.318},"min":{"1min":0.24199999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.245},{"interface":"front","average":{"1min":0.434,"5min":0.35999999999999999,"15min":0.33400000000000002},"min":{"1min":0.27500000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.53700000000000003}]},{"osd":2,"last update":"Tue Sep 10 18:46:11 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.372,"15min":0.36099999999999999},"min":{"1min":0.27900000000000003,"5min":0.248,"15min":0.248},"max":{"1min":0.47199999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.39000000000000001,"15min":0.38100000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.49099999999999999,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.47399999999999998}]},{"osd":3,"last update":"Tue Sep 10 18:46:34 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.45400000000000001,"15min":0.45800000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.42299999999999999,"15min":0.42199999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.55400000000000005,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.32500000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:45:56 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.56299999999999994,"15min":0.55600000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.54100000000000004,"15min":0.53300000000000003},"min":{"1min":0.41099999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.59499999999999997}]},{"osd":5,"last update":"Tue Sep 10 18:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60299999999999998,"15min":0.60399999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63700000000000001,"15min":0.64100000000000001},"min":{"1min":0.44600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93300000000000005,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.64300000000000002}]},{"osd":6,"last update":"Tue Sep 10 18:45:43 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.78400000000000003}]},{"osd":7,"last update":"Tue Sep 10 18:46:11 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.81200000000000006}]}]},{"osd":0,"up_from":8,"seq":34359738423,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298716,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446436,"statfs":{"total":95995035648,"available":95689150464,"internally_reserved":0,"allocated":1396736,"data_stored":920201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 18:45:50 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.32100000000000001,"15min":0.30199999999999999},"min":{"1min":0.224,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.219},{"interface":"front","average":{"1min":0.371,"5min":0.33100000000000002,"15min":0.313},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.307}]},{"osd":2,"last update":"Tue Sep 10 18:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.40300000000000002,"15min":0.38900000000000001},"min":{"1min":0.30299999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.376,"15min":0.36199999999999999},"min":{"1min":0.26600000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.32300000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:46:31 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.39500000000000002,"15min":0.39800000000000002},"min":{"1min":0.26100000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.66100000000000003,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.41199999999999998,"15min":0.41299999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.36499999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.56999999999999995,"15min":0.53600000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":4.6189999999999998,"5min":4.6189999999999998,"15min":4.6189999999999998},"last":0.61099999999999999}]},{"osd":5,"last update":"Tue Sep 10 18:46:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.42999999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.495},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.58699999999999997,"15min":0.55800000000000005},"min":{"1min":0.42699999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":4.6429999999999998,"5min":4.6429999999999998,"15min":4.6429999999999998},"last":0.51900000000000002}]},{"osd":6,"last update":"Tue Sep 10 18:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":4.6630000000000003,"5min":4.6630000000000003,"15min":4.6630000000000003},"last":0.55500000000000005}]},{"osd":7,"last update":"Tue Sep 10 18:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.57799999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T18:46:43.915 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T18:46:44.175 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:46:44.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:44 smithi018 ceph-mon[29336]: 4.3 scrub starts 2024-09-10T18:46:44.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:44 smithi018 ceph-mon[29336]: 4.3 scrub ok 2024-09-10T18:46:44.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:44 smithi018 ceph-mon[29336]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:46:44.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:44 smithi018 ceph-mon[29336]: from='client.14679 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:46:44.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:44 smithi018 ceph-mon[40196]: 4.3 scrub starts 2024-09-10T18:46:44.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:44 smithi018 ceph-mon[40196]: 4.3 scrub ok 2024-09-10T18:46:44.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:44 smithi018 ceph-mon[40196]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:46:44.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:44 smithi018 ceph-mon[40196]: from='client.14679 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:46:44.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:44 smithi103 ceph-mon[33163]: 4.3 scrub starts 2024-09-10T18:46:44.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:44 smithi103 ceph-mon[33163]: 4.3 scrub ok 2024-09-10T18:46:44.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:44 smithi103 ceph-mon[33163]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:46:44.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:44 smithi103 ceph-mon[33163]: from='client.14679 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:46:45.274 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:45 smithi018 ceph-mon[40196]: 4.d deep-scrub starts 2024-09-10T18:46:45.274 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:45 smithi018 ceph-mon[40196]: 4.d deep-scrub ok 2024-09-10T18:46:45.274 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:45 smithi018 ceph-mon[40196]: 2.19 scrub starts 2024-09-10T18:46:45.274 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:45 smithi018 ceph-mon[40196]: 2.19 scrub ok 2024-09-10T18:46:45.274 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:45 smithi018 ceph-mon[40196]: 5.8 scrub starts 2024-09-10T18:46:45.274 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:45 smithi018 ceph-mon[40196]: 5.8 scrub ok 2024-09-10T18:46:45.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:45 smithi018 ceph-mon[29336]: 4.d deep-scrub starts 2024-09-10T18:46:45.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:45 smithi018 ceph-mon[29336]: 4.d deep-scrub ok 2024-09-10T18:46:45.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:45 smithi018 ceph-mon[29336]: 2.19 scrub starts 2024-09-10T18:46:45.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:45 smithi018 ceph-mon[29336]: 2.19 scrub ok 2024-09-10T18:46:45.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:45 smithi018 ceph-mon[29336]: 5.8 scrub starts 2024-09-10T18:46:45.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:45 smithi018 ceph-mon[29336]: 5.8 scrub ok 2024-09-10T18:46:45.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:45 smithi103 ceph-mon[33163]: 4.d deep-scrub starts 2024-09-10T18:46:45.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:45 smithi103 ceph-mon[33163]: 4.d deep-scrub ok 2024-09-10T18:46:45.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:45 smithi103 ceph-mon[33163]: 2.19 scrub starts 2024-09-10T18:46:45.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:45 smithi103 ceph-mon[33163]: 2.19 scrub ok 2024-09-10T18:46:45.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:45 smithi103 ceph-mon[33163]: 5.8 scrub starts 2024-09-10T18:46:45.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:45 smithi103 ceph-mon[33163]: 5.8 scrub ok 2024-09-10T18:46:45.944 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:46:45 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a[72294]: ts=2024-09-10T18:46:45.767Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002243163s 2024-09-10T18:46:45.957 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:46:45.959 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-09-10T18:46:46.415 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:46 smithi018 ceph-mon[40196]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 886 B/s rd, 0 op/s 2024-09-10T18:46:46.415 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:46 smithi018 ceph-mon[40196]: 6.a deep-scrub starts 2024-09-10T18:46:46.415 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:46 smithi018 ceph-mon[40196]: 6.a deep-scrub ok 2024-09-10T18:46:46.416 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:46 smithi018 ceph-mon[40196]: 6.1e scrub starts 2024-09-10T18:46:46.416 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:46 smithi018 ceph-mon[40196]: 6.1e scrub ok 2024-09-10T18:46:46.416 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:46 smithi018 ceph-mon[29336]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 886 B/s rd, 0 op/s 2024-09-10T18:46:46.416 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:46 smithi018 ceph-mon[29336]: 6.a deep-scrub starts 2024-09-10T18:46:46.416 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:46 smithi018 ceph-mon[29336]: 6.a deep-scrub ok 2024-09-10T18:46:46.416 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:46 smithi018 ceph-mon[29336]: 6.1e scrub starts 2024-09-10T18:46:46.416 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:46 smithi018 ceph-mon[29336]: 6.1e scrub ok 2024-09-10T18:46:46.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:46 smithi103 ceph-mon[33163]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 886 B/s rd, 0 op/s 2024-09-10T18:46:46.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:46 smithi103 ceph-mon[33163]: 6.a deep-scrub starts 2024-09-10T18:46:46.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:46 smithi103 ceph-mon[33163]: 6.a deep-scrub ok 2024-09-10T18:46:46.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:46 smithi103 ceph-mon[33163]: 6.1e scrub starts 2024-09-10T18:46:46.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:46 smithi103 ceph-mon[33163]: 6.1e scrub ok 2024-09-10T18:46:46.523 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:46:46 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:46:46.787 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-09-10T18:46:45.026006+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":928,"num_read_kb":785,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":555,"ondisk_log_size":555,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2386864,"kb_used_data":8048,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747574352,"statfs":{"total":767960285184,"available":765516136448,"internally_reserved":0,"allocated":8241152,"data_stored":4414428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.556319"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:33.824105+0000","last_change":"2024-09-10T18:46:33.824057+0000","last_active":"2024-09-10T18:46:33.824105+0000","last_peered":"2024-09-10T18:46:33.824105+0000","last_clean":"2024-09-10T18:46:33.824105+0000","last_became_active":"2024-09-10T18:45:24.265488+0000","last_became_peered":"2024-09-10T18:45:24.265488+0000","last_unstale":"2024-09-10T18:46:33.824105+0000","last_undegraded":"2024-09-10T18:46:33.824105+0000","last_fullsized":"2024-09-10T18:46:33.824105+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:33.823989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:33.823989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:44.131457+0000","scrub_duration":0.0034389770000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742368+0000","last_change":"2024-09-10T18:45:17.959740+0000","last_active":"2024-09-10T18:46:29.742368+0000","last_peered":"2024-09-10T18:46:29.742368+0000","last_clean":"2024-09-10T18:46:29.742368+0000","last_became_active":"2024-09-10T18:45:17.959475+0000","last_became_peered":"2024-09-10T18:45:17.959475+0000","last_unstale":"2024-09-10T18:46:29.742368+0000","last_undegraded":"2024-09-10T18:46:29.742368+0000","last_fullsized":"2024-09-10T18:46:29.742368+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:41.295534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"63'15","reported_seq":64,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:36.856851+0000","last_change":"2024-09-10T18:46:36.856804+0000","last_active":"2024-09-10T18:46:36.856851+0000","last_peered":"2024-09-10T18:46:36.856851+0000","last_clean":"2024-09-10T18:46:36.856851+0000","last_became_active":"2024-09-10T18:45:20.052531+0000","last_became_peered":"2024-09-10T18:45:20.052531+0000","last_unstale":"2024-09-10T18:46:36.856851+0000","last_undegraded":"2024-09-10T18:46:36.856851+0000","last_fullsized":"2024-09-10T18:46:36.856851+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:46:36.856740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:36.856740+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:03.997270+0000","scrub_duration":0.0018131040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:40.313341+0000","last_change":"2024-09-10T18:46:40.313306+0000","last_active":"2024-09-10T18:46:40.313341+0000","last_peered":"2024-09-10T18:46:40.313341+0000","last_clean":"2024-09-10T18:46:40.313341+0000","last_became_active":"2024-09-10T18:45:22.065619+0000","last_became_peered":"2024-09-10T18:45:22.065619+0000","last_unstale":"2024-09-10T18:46:40.313341+0000","last_undegraded":"2024-09-10T18:46:40.313341+0000","last_fullsized":"2024-09-10T18:46:40.313341+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:40.313218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:40.313218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:52.103586+0000","scrub_duration":0.0017922750000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.791464+0000","last_change":"2024-09-10T18:46:18.890844+0000","last_active":"2024-09-10T18:46:29.791464+0000","last_peered":"2024-09-10T18:46:29.791464+0000","last_clean":"2024-09-10T18:46:29.791464+0000","last_became_active":"2024-09-10T18:45:17.966153+0000","last_became_peered":"2024-09-10T18:45:17.966153+0000","last_unstale":"2024-09-10T18:46:29.791464+0000","last_undegraded":"2024-09-10T18:46:29.791464+0000","last_fullsized":"2024-09-10T18:46:29.791464+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:18.890702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:18.890702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:47.062002+0000","scrub_duration":0.0019382589999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"63'9","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.319204+0000","last_change":"2024-09-10T18:45:20.061552+0000","last_active":"2024-09-10T18:46:30.319204+0000","last_peered":"2024-09-10T18:46:30.319204+0000","last_clean":"2024-09-10T18:46:30.319204+0000","last_became_active":"2024-09-10T18:45:20.061266+0000","last_became_peered":"2024-09-10T18:45:20.061266+0000","last_unstale":"2024-09-10T18:46:30.319204+0000","last_undegraded":"2024-09-10T18:46:30.319204+0000","last_fullsized":"2024-09-10T18:46:30.319204+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:43.961487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742146+0000","last_change":"2024-09-10T18:45:22.073139+0000","last_active":"2024-09-10T18:46:29.742146+0000","last_peered":"2024-09-10T18:46:29.742146+0000","last_clean":"2024-09-10T18:46:29.742146+0000","last_became_active":"2024-09-10T18:45:22.072822+0000","last_became_peered":"2024-09-10T18:45:22.072822+0000","last_unstale":"2024-09-10T18:46:29.742146+0000","last_undegraded":"2024-09-10T18:46:29.742146+0000","last_fullsized":"2024-09-10T18:46:29.742146+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:45.740469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.319442+0000","last_change":"2024-09-10T18:45:24.081275+0000","last_active":"2024-09-10T18:46:30.319442+0000","last_peered":"2024-09-10T18:46:30.319442+0000","last_clean":"2024-09-10T18:46:30.319442+0000","last_became_active":"2024-09-10T18:45:24.081002+0000","last_became_peered":"2024-09-10T18:45:24.081002+0000","last_unstale":"2024-09-10T18:46:30.319442+0000","last_undegraded":"2024-09-10T18:46:30.319442+0000","last_fullsized":"2024-09-10T18:46:30.319442+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:46.639608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.566850+0000","last_change":"2024-09-10T18:45:17.961978+0000","last_active":"2024-09-10T18:46:30.566850+0000","last_peered":"2024-09-10T18:46:30.566850+0000","last_clean":"2024-09-10T18:46:30.566850+0000","last_became_active":"2024-09-10T18:45:17.961097+0000","last_became_peered":"2024-09-10T18:45:17.961097+0000","last_unstale":"2024-09-10T18:46:30.566850+0000","last_undegraded":"2024-09-10T18:46:30.566850+0000","last_fullsized":"2024-09-10T18:46:30.566850+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"63'5","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.319541+0000","last_change":"2024-09-10T18:45:20.058010+0000","last_active":"2024-09-10T18:46:30.319541+0000","last_peered":"2024-09-10T18:46:30.319541+0000","last_clean":"2024-09-10T18:46:30.319541+0000","last_became_active":"2024-09-10T18:45:20.057567+0000","last_became_peered":"2024-09-10T18:45:20.057567+0000","last_unstale":"2024-09-10T18:46:30.319541+0000","last_undegraded":"2024-09-10T18:46:30.319541+0000","last_fullsized":"2024-09-10T18:46:30.319541+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:47.144699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.567431+0000","last_change":"2024-09-10T18:46:22.547871+0000","last_active":"2024-09-10T18:46:30.567431+0000","last_peered":"2024-09-10T18:46:30.567431+0000","last_clean":"2024-09-10T18:46:30.567431+0000","last_became_active":"2024-09-10T18:45:22.076575+0000","last_became_peered":"2024-09-10T18:45:22.076575+0000","last_unstale":"2024-09-10T18:46:30.567431+0000","last_undegraded":"2024-09-10T18:46:30.567431+0000","last_fullsized":"2024-09-10T18:46:30.567431+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:22.547800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:22.547800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:32.775669+0000","scrub_duration":0.001880622,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:36.158189+0000","last_change":"2024-09-10T18:46:36.158152+0000","last_active":"2024-09-10T18:46:36.158189+0000","last_peered":"2024-09-10T18:46:36.158189+0000","last_clean":"2024-09-10T18:46:36.158189+0000","last_became_active":"2024-09-10T18:45:24.079534+0000","last_became_peered":"2024-09-10T18:45:24.079534+0000","last_unstale":"2024-09-10T18:46:36.158189+0000","last_undegraded":"2024-09-10T18:46:36.158189+0000","last_fullsized":"2024-09-10T18:46:36.158189+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:36.158087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:36.158087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:41.325875+0000","scrub_duration":0.0014477100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.567032+0000","last_change":"2024-09-10T18:45:17.963368+0000","last_active":"2024-09-10T18:46:30.567032+0000","last_peered":"2024-09-10T18:46:30.567032+0000","last_clean":"2024-09-10T18:46:30.567032+0000","last_became_active":"2024-09-10T18:45:17.963121+0000","last_became_peered":"2024-09-10T18:45:17.963121+0000","last_unstale":"2024-09-10T18:46:30.567032+0000","last_undegraded":"2024-09-10T18:46:30.567032+0000","last_fullsized":"2024-09-10T18:46:30.567032+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:34.269484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"63'9","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.319602+0000","last_change":"2024-09-10T18:45:20.053258+0000","last_active":"2024-09-10T18:46:30.319602+0000","last_peered":"2024-09-10T18:46:30.319602+0000","last_clean":"2024-09-10T18:46:30.319602+0000","last_became_active":"2024-09-10T18:45:20.052652+0000","last_became_peered":"2024-09-10T18:45:20.052652+0000","last_unstale":"2024-09-10T18:46:30.319602+0000","last_undegraded":"2024-09-10T18:46:30.319602+0000","last_fullsized":"2024-09-10T18:46:30.319602+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:42.837022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:31.024367+0000","last_change":"2024-09-10T18:46:31.024322+0000","last_active":"2024-09-10T18:46:31.024367+0000","last_peered":"2024-09-10T18:46:31.024367+0000","last_clean":"2024-09-10T18:46:31.024367+0000","last_became_active":"2024-09-10T18:45:22.068326+0000","last_became_peered":"2024-09-10T18:45:22.068326+0000","last_unstale":"2024-09-10T18:46:31.024367+0000","last_undegraded":"2024-09-10T18:46:31.024367+0000","last_fullsized":"2024-09-10T18:46:31.024367+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:31.024255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:31.024255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:50.407493+0000","scrub_duration":0.00152395,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.740928+0000","last_change":"2024-09-10T18:45:24.081968+0000","last_active":"2024-09-10T18:46:29.740928+0000","last_peered":"2024-09-10T18:46:29.740928+0000","last_clean":"2024-09-10T18:46:29.740928+0000","last_became_active":"2024-09-10T18:45:24.081488+0000","last_became_peered":"2024-09-10T18:45:24.081488+0000","last_unstale":"2024-09-10T18:46:29.740928+0000","last_undegraded":"2024-09-10T18:46:29.740928+0000","last_fullsized":"2024-09-10T18:46:29.740928+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:51.266075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"63'17","reported_seq":57,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.319995+0000","last_change":"2024-09-10T18:45:20.057352+0000","last_active":"2024-09-10T18:46:30.319995+0000","last_peered":"2024-09-10T18:46:30.319995+0000","last_clean":"2024-09-10T18:46:30.319995+0000","last_became_active":"2024-09-10T18:45:20.057019+0000","last_became_peered":"2024-09-10T18:45:20.057019+0000","last_unstale":"2024-09-10T18:46:30.319995+0000","last_undegraded":"2024-09-10T18:46:30.319995+0000","last_fullsized":"2024-09-10T18:46:30.319995+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:40.656805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.566501+0000","last_change":"2024-09-10T18:45:17.964001+0000","last_active":"2024-09-10T18:46:30.566501+0000","last_peered":"2024-09-10T18:46:30.566501+0000","last_clean":"2024-09-10T18:46:30.566501+0000","last_became_active":"2024-09-10T18:45:17.963732+0000","last_became_peered":"2024-09-10T18:45:17.963732+0000","last_unstale":"2024-09-10T18:46:30.566501+0000","last_undegraded":"2024-09-10T18:46:30.566501+0000","last_fullsized":"2024-09-10T18:46:30.566501+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742330+0000","last_change":"2024-09-10T18:45:22.071993+0000","last_active":"2024-09-10T18:46:29.742330+0000","last_peered":"2024-09-10T18:46:29.742330+0000","last_clean":"2024-09-10T18:46:29.742330+0000","last_became_active":"2024-09-10T18:45:22.071443+0000","last_became_peered":"2024-09-10T18:45:22.071443+0000","last_unstale":"2024-09-10T18:46:29.742330+0000","last_undegraded":"2024-09-10T18:46:29.742330+0000","last_fullsized":"2024-09-10T18:46:29.742330+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:42.692893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:35.738263+0000","last_change":"2024-09-10T18:46:35.738217+0000","last_active":"2024-09-10T18:46:35.738263+0000","last_peered":"2024-09-10T18:46:35.738263+0000","last_clean":"2024-09-10T18:46:35.738263+0000","last_became_active":"2024-09-10T18:45:24.077897+0000","last_became_peered":"2024-09-10T18:45:24.077897+0000","last_unstale":"2024-09-10T18:46:35.738263+0000","last_undegraded":"2024-09-10T18:46:35.738263+0000","last_fullsized":"2024-09-10T18:46:35.738263+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:35.738143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:35.738143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:48.741747+0000","scrub_duration":0.0015293520000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"63'10","reported_seq":54,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:31.281648+0000","last_change":"2024-09-10T18:46:31.281588+0000","last_active":"2024-09-10T18:46:31.281648+0000","last_peered":"2024-09-10T18:46:31.281648+0000","last_clean":"2024-09-10T18:46:31.281648+0000","last_became_active":"2024-09-10T18:45:20.053428+0000","last_became_peered":"2024-09-10T18:45:20.053428+0000","last_unstale":"2024-09-10T18:46:31.281648+0000","last_undegraded":"2024-09-10T18:46:31.281648+0000","last_fullsized":"2024-09-10T18:46:31.281648+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:46:31.281519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:31.281519+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:58.508120+0000","scrub_duration":0.0019095119999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:38.445445+0000","last_change":"2024-09-10T18:46:38.445417+0000","last_active":"2024-09-10T18:46:38.445445+0000","last_peered":"2024-09-10T18:46:38.445445+0000","last_clean":"2024-09-10T18:46:38.445445+0000","last_became_active":"2024-09-10T18:45:17.966833+0000","last_became_peered":"2024-09-10T18:45:17.966833+0000","last_unstale":"2024-09-10T18:46:38.445445+0000","last_undegraded":"2024-09-10T18:46:38.445445+0000","last_fullsized":"2024-09-10T18:46:38.445445+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:38.445362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:38.445362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:02.460484+0000","scrub_duration":0.0012292029999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"63'8","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:34.072313+0000","last_change":"2024-09-10T18:46:25.787936+0000","last_active":"2024-09-10T18:46:34.072313+0000","last_peered":"2024-09-10T18:46:34.072313+0000","last_clean":"2024-09-10T18:46:34.072313+0000","last_became_active":"2024-09-10T18:45:22.068453+0000","last_became_peered":"2024-09-10T18:45:22.068453+0000","last_unstale":"2024-09-10T18:46:34.072313+0000","last_undegraded":"2024-09-10T18:46:34.072313+0000","last_fullsized":"2024-09-10T18:46:34.072313+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:25.787877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:25.787877+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:54.201629+0000","scrub_duration":0.0018093530000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:32.265084+0000","last_change":"2024-09-10T18:46:32.265034+0000","last_active":"2024-09-10T18:46:32.265084+0000","last_peered":"2024-09-10T18:46:32.265084+0000","last_clean":"2024-09-10T18:46:32.265084+0000","last_became_active":"2024-09-10T18:45:24.081082+0000","last_became_peered":"2024-09-10T18:45:24.081082+0000","last_unstale":"2024-09-10T18:46:32.265084+0000","last_undegraded":"2024-09-10T18:46:32.265084+0000","last_fullsized":"2024-09-10T18:46:32.265084+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:32.264963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:32.264963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:50.207705+0000","scrub_duration":0.002378509,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"63'15","reported_seq":54,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742827+0000","last_change":"2024-09-10T18:45:20.059509+0000","last_active":"2024-09-10T18:46:29.742827+0000","last_peered":"2024-09-10T18:46:29.742827+0000","last_clean":"2024-09-10T18:46:29.742827+0000","last_became_active":"2024-09-10T18:45:20.059222+0000","last_became_peered":"2024-09-10T18:45:20.059222+0000","last_unstale":"2024-09-10T18:46:29.742827+0000","last_undegraded":"2024-09-10T18:46:29.742827+0000","last_fullsized":"2024-09-10T18:46:29.742827+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:41.772741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742326+0000","last_change":"2024-09-10T18:46:28.320761+0000","last_active":"2024-09-10T18:46:29.742326+0000","last_peered":"2024-09-10T18:46:29.742326+0000","last_clean":"2024-09-10T18:46:29.742326+0000","last_became_active":"2024-09-10T18:45:17.965409+0000","last_became_peered":"2024-09-10T18:45:17.965409+0000","last_unstale":"2024-09-10T18:46:29.742326+0000","last_undegraded":"2024-09-10T18:46:29.742326+0000","last_fullsized":"2024-09-10T18:46:29.742326+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_clean_scrub_stamp":"2024-09-10T18:46:28.320704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:44.092028+0000","scrub_duration":0.001393476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"63'8","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:39.300173+0000","last_change":"2024-09-10T18:46:39.300139+0000","last_active":"2024-09-10T18:46:39.300173+0000","last_peered":"2024-09-10T18:46:39.300173+0000","last_clean":"2024-09-10T18:46:39.300173+0000","last_became_active":"2024-09-10T18:45:22.062497+0000","last_became_peered":"2024-09-10T18:45:22.062497+0000","last_unstale":"2024-09-10T18:46:39.300173+0000","last_undegraded":"2024-09-10T18:46:39.300173+0000","last_fullsized":"2024-09-10T18:46:39.300173+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:39.300065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:39.300065+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:44.830898+0000","scrub_duration":0.001858934,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742058+0000","last_change":"2024-09-10T18:45:24.082637+0000","last_active":"2024-09-10T18:46:29.742058+0000","last_peered":"2024-09-10T18:46:29.742058+0000","last_clean":"2024-09-10T18:46:29.742058+0000","last_became_active":"2024-09-10T18:45:24.082362+0000","last_became_peered":"2024-09-10T18:45:24.082362+0000","last_unstale":"2024-09-10T18:46:29.742058+0000","last_undegraded":"2024-09-10T18:46:29.742058+0000","last_fullsized":"2024-09-10T18:46:29.742058+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"63'11","reported_seq":58,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:35.274763+0000","last_change":"2024-09-10T18:46:35.274729+0000","last_active":"2024-09-10T18:46:35.274763+0000","last_peered":"2024-09-10T18:46:35.274763+0000","last_clean":"2024-09-10T18:46:35.274763+0000","last_became_active":"2024-09-10T18:45:20.061187+0000","last_became_peered":"2024-09-10T18:45:20.061187+0000","last_unstale":"2024-09-10T18:46:35.274763+0000","last_undegraded":"2024-09-10T18:46:35.274763+0000","last_fullsized":"2024-09-10T18:46:35.274763+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:46:35.274660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:35.274660+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:51.240811+0000","scrub_duration":0.0019745819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:39.520524+0000","last_change":"2024-09-10T18:46:39.520497+0000","last_active":"2024-09-10T18:46:39.520524+0000","last_peered":"2024-09-10T18:46:39.520524+0000","last_clean":"2024-09-10T18:46:39.520524+0000","last_became_active":"2024-09-10T18:45:17.966133+0000","last_became_peered":"2024-09-10T18:45:17.966133+0000","last_unstale":"2024-09-10T18:46:39.520524+0000","last_undegraded":"2024-09-10T18:46:39.520524+0000","last_fullsized":"2024-09-10T18:46:39.520524+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:39.520432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:39.520432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:49.040271+0000","scrub_duration":0.0013982510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742394+0000","last_change":"2024-09-10T18:45:22.069224+0000","last_active":"2024-09-10T18:46:29.742394+0000","last_peered":"2024-09-10T18:46:29.742394+0000","last_clean":"2024-09-10T18:46:29.742394+0000","last_became_active":"2024-09-10T18:45:22.068918+0000","last_became_peered":"2024-09-10T18:45:22.068918+0000","last_unstale":"2024-09-10T18:46:29.742394+0000","last_undegraded":"2024-09-10T18:46:29.742394+0000","last_fullsized":"2024-09-10T18:46:29.742394+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:35.444750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.790998+0000","last_change":"2024-09-10T18:45:24.264914+0000","last_active":"2024-09-10T18:46:29.790998+0000","last_peered":"2024-09-10T18:46:29.790998+0000","last_clean":"2024-09-10T18:46:29.790998+0000","last_became_active":"2024-09-10T18:45:24.264538+0000","last_became_peered":"2024-09-10T18:45:24.264538+0000","last_unstale":"2024-09-10T18:46:29.790998+0000","last_undegraded":"2024-09-10T18:46:29.790998+0000","last_fullsized":"2024-09-10T18:46:29.790998+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:46.643079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"63'14","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.319391+0000","last_change":"2024-09-10T18:46:29.330572+0000","last_active":"2024-09-10T18:46:30.319391+0000","last_peered":"2024-09-10T18:46:30.319391+0000","last_clean":"2024-09-10T18:46:30.319391+0000","last_became_active":"2024-09-10T18:45:20.052243+0000","last_became_peered":"2024-09-10T18:45:20.052243+0000","last_unstale":"2024-09-10T18:46:30.319391+0000","last_undegraded":"2024-09-10T18:46:30.319391+0000","last_fullsized":"2024-09-10T18:46:30.319391+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-10T18:46:29.330503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:29.330503+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:55.456983+0000","scrub_duration":0.0020437459999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.792661+0000","last_change":"2024-09-10T18:46:25.206801+0000","last_active":"2024-09-10T18:46:29.792661+0000","last_peered":"2024-09-10T18:46:29.792661+0000","last_clean":"2024-09-10T18:46:29.792661+0000","last_became_active":"2024-09-10T18:45:17.966027+0000","last_became_peered":"2024-09-10T18:45:17.966027+0000","last_unstale":"2024-09-10T18:46:29.792661+0000","last_undegraded":"2024-09-10T18:46:29.792661+0000","last_fullsized":"2024-09-10T18:46:29.792661+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_clean_scrub_stamp":"2024-09-10T18:46:25.206731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:46.038154+0000","scrub_duration":0.0016589910000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:40.808551+0000","last_change":"2024-09-10T18:46:40.808518+0000","last_active":"2024-09-10T18:46:40.808551+0000","last_peered":"2024-09-10T18:46:40.808551+0000","last_clean":"2024-09-10T18:46:40.808551+0000","last_became_active":"2024-09-10T18:45:22.074540+0000","last_became_peered":"2024-09-10T18:45:22.074540+0000","last_unstale":"2024-09-10T18:46:40.808551+0000","last_undegraded":"2024-09-10T18:46:40.808551+0000","last_fullsized":"2024-09-10T18:46:40.808551+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:40.808453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:40.808453+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:43.962254+0000","scrub_duration":0.0016757829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:32.590593+0000","last_change":"2024-09-10T18:46:32.590511+0000","last_active":"2024-09-10T18:46:32.590593+0000","last_peered":"2024-09-10T18:46:32.590593+0000","last_clean":"2024-09-10T18:46:32.590593+0000","last_became_active":"2024-09-10T18:45:24.265303+0000","last_became_peered":"2024-09-10T18:45:24.265303+0000","last_unstale":"2024-09-10T18:46:32.590593+0000","last_undegraded":"2024-09-10T18:46:32.590593+0000","last_fullsized":"2024-09-10T18:46:32.590593+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:32.590438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:32.590438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:01.680676+0000","scrub_duration":0.0013292270000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"63'18","reported_seq":71,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.790754+0000","last_change":"2024-09-10T18:46:27.801177+0000","last_active":"2024-09-10T18:46:29.790754+0000","last_peered":"2024-09-10T18:46:29.790754+0000","last_clean":"2024-09-10T18:46:29.790754+0000","last_became_active":"2024-09-10T18:45:20.420937+0000","last_became_peered":"2024-09-10T18:45:20.420937+0000","last_unstale":"2024-09-10T18:46:29.790754+0000","last_undegraded":"2024-09-10T18:46:29.790754+0000","last_fullsized":"2024-09-10T18:46:29.790754+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'18","last_scrub_stamp":"2024-09-10T18:46:27.801110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:27.801110+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:42.247024+0000","scrub_duration":0.001870095,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742628+0000","last_change":"2024-09-10T18:45:17.962000+0000","last_active":"2024-09-10T18:46:29.742628+0000","last_peered":"2024-09-10T18:46:29.742628+0000","last_clean":"2024-09-10T18:46:29.742628+0000","last_became_active":"2024-09-10T18:45:17.961695+0000","last_became_peered":"2024-09-10T18:45:17.961695+0000","last_unstale":"2024-09-10T18:46:29.742628+0000","last_undegraded":"2024-09-10T18:46:29.742628+0000","last_fullsized":"2024-09-10T18:46:29.742628+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:43.273962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.319288+0000","last_change":"2024-09-10T18:45:22.074309+0000","last_active":"2024-09-10T18:46:30.319288+0000","last_peered":"2024-09-10T18:46:30.319288+0000","last_clean":"2024-09-10T18:46:30.319288+0000","last_became_active":"2024-09-10T18:45:22.074012+0000","last_became_peered":"2024-09-10T18:45:22.074012+0000","last_unstale":"2024-09-10T18:46:30.319288+0000","last_undegraded":"2024-09-10T18:46:30.319288+0000","last_fullsized":"2024-09-10T18:46:30.319288+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:35.318930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.319530+0000","last_change":"2024-09-10T18:45:24.077245+0000","last_active":"2024-09-10T18:46:30.319530+0000","last_peered":"2024-09-10T18:46:30.319530+0000","last_clean":"2024-09-10T18:46:30.319530+0000","last_became_active":"2024-09-10T18:45:24.076831+0000","last_became_peered":"2024-09-10T18:45:24.076831+0000","last_unstale":"2024-09-10T18:46:30.319530+0000","last_undegraded":"2024-09-10T18:46:30.319530+0000","last_fullsized":"2024-09-10T18:46:30.319530+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:41.841015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"63'19","reported_seq":65,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.741641+0000","last_change":"2024-09-10T18:45:20.421532+0000","last_active":"2024-09-10T18:46:29.741641+0000","last_peered":"2024-09-10T18:46:29.741641+0000","last_clean":"2024-09-10T18:46:29.741641+0000","last_became_active":"2024-09-10T18:45:20.421046+0000","last_became_peered":"2024-09-10T18:45:20.421046+0000","last_unstale":"2024-09-10T18:46:29.741641+0000","last_undegraded":"2024-09-10T18:46:29.741641+0000","last_fullsized":"2024-09-10T18:46:29.741641+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.566674+0000","last_change":"2024-09-10T18:45:17.960558+0000","last_active":"2024-09-10T18:46:30.566674+0000","last_peered":"2024-09-10T18:46:30.566674+0000","last_clean":"2024-09-10T18:46:30.566674+0000","last_became_active":"2024-09-10T18:45:17.960226+0000","last_became_peered":"2024-09-10T18:45:17.960226+0000","last_unstale":"2024-09-10T18:46:30.566674+0000","last_undegraded":"2024-09-10T18:46:30.566674+0000","last_fullsized":"2024-09-10T18:46:30.566674+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:43.999118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.791889+0000","last_change":"2024-09-10T18:45:22.066434+0000","last_active":"2024-09-10T18:46:29.791889+0000","last_peered":"2024-09-10T18:46:29.791889+0000","last_clean":"2024-09-10T18:46:29.791889+0000","last_became_active":"2024-09-10T18:45:22.066148+0000","last_became_peered":"2024-09-10T18:45:22.066148+0000","last_unstale":"2024-09-10T18:46:29.791889+0000","last_undegraded":"2024-09-10T18:46:29.791889+0000","last_fullsized":"2024-09-10T18:46:29.791889+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:46.655683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742078+0000","last_change":"2024-09-10T18:45:24.265025+0000","last_active":"2024-09-10T18:46:29.742078+0000","last_peered":"2024-09-10T18:46:29.742078+0000","last_clean":"2024-09-10T18:46:29.742078+0000","last_became_active":"2024-09-10T18:45:24.264680+0000","last_became_peered":"2024-09-10T18:45:24.264680+0000","last_unstale":"2024-09-10T18:46:29.742078+0000","last_undegraded":"2024-09-10T18:46:29.742078+0000","last_fullsized":"2024-09-10T18:46:29.742078+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:47.094144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"63'10","reported_seq":44,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.741454+0000","last_change":"2024-09-10T18:45:20.057590+0000","last_active":"2024-09-10T18:46:29.741454+0000","last_peered":"2024-09-10T18:46:29.741454+0000","last_clean":"2024-09-10T18:46:29.741454+0000","last_became_active":"2024-09-10T18:45:20.057204+0000","last_became_peered":"2024-09-10T18:45:20.057204+0000","last_unstale":"2024-09-10T18:46:29.741454+0000","last_undegraded":"2024-09-10T18:46:29.741454+0000","last_fullsized":"2024-09-10T18:46:29.741454+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:38.786019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742270+0000","last_change":"2024-09-10T18:45:17.968539+0000","last_active":"2024-09-10T18:46:29.742270+0000","last_peered":"2024-09-10T18:46:29.742270+0000","last_clean":"2024-09-10T18:46:29.742270+0000","last_became_active":"2024-09-10T18:45:17.966632+0000","last_became_peered":"2024-09-10T18:45:17.966632+0000","last_unstale":"2024-09-10T18:46:29.742270+0000","last_undegraded":"2024-09-10T18:46:29.742270+0000","last_fullsized":"2024-09-10T18:46:29.742270+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:37.082480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"63'8","reported_seq":51,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:33.471982+0000","last_change":"2024-09-10T18:45:22.063631+0000","last_active":"2024-09-10T18:46:33.471982+0000","last_peered":"2024-09-10T18:46:33.471982+0000","last_clean":"2024-09-10T18:46:33.471982+0000","last_became_active":"2024-09-10T18:45:22.063326+0000","last_became_peered":"2024-09-10T18:45:22.063326+0000","last_unstale":"2024-09-10T18:46:33.471982+0000","last_undegraded":"2024-09-10T18:46:33.471982+0000","last_fullsized":"2024-09-10T18:46:33.471982+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:41.497821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.740793+0000","last_change":"2024-09-10T18:45:24.077780+0000","last_active":"2024-09-10T18:46:29.740793+0000","last_peered":"2024-09-10T18:46:29.740793+0000","last_clean":"2024-09-10T18:46:29.740793+0000","last_became_active":"2024-09-10T18:45:24.077412+0000","last_became_peered":"2024-09-10T18:45:24.077412+0000","last_unstale":"2024-09-10T18:46:29.740793+0000","last_undegraded":"2024-09-10T18:46:29.740793+0000","last_fullsized":"2024-09-10T18:46:29.740793+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:45.489998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"63'28","reported_seq":103,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:33.472587+0000","last_change":"2024-09-10T18:46:19.223319+0000","last_active":"2024-09-10T18:46:33.472587+0000","last_peered":"2024-09-10T18:46:33.472587+0000","last_clean":"2024-09-10T18:46:33.472587+0000","last_became_active":"2024-09-10T18:45:20.061413+0000","last_became_peered":"2024-09-10T18:45:20.061413+0000","last_unstale":"2024-09-10T18:46:33.472587+0000","last_undegraded":"2024-09-10T18:46:33.472587+0000","last_fullsized":"2024-09-10T18:46:33.472587+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'28","last_scrub_stamp":"2024-09-10T18:46:19.223250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:19.223250+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:23.644429+0000","scrub_duration":0.0032067810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.741949+0000","last_change":"2024-09-10T18:45:17.965134+0000","last_active":"2024-09-10T18:46:29.741949+0000","last_peered":"2024-09-10T18:46:29.741949+0000","last_clean":"2024-09-10T18:46:29.741949+0000","last_became_active":"2024-09-10T18:45:17.964494+0000","last_became_peered":"2024-09-10T18:45:17.964494+0000","last_unstale":"2024-09-10T18:46:29.741949+0000","last_undegraded":"2024-09-10T18:46:29.741949+0000","last_fullsized":"2024-09-10T18:46:29.741949+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:36.348849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.741750+0000","last_change":"2024-09-10T18:45:22.061738+0000","last_active":"2024-09-10T18:46:29.741750+0000","last_peered":"2024-09-10T18:46:29.741750+0000","last_clean":"2024-09-10T18:46:29.741750+0000","last_became_active":"2024-09-10T18:45:22.061428+0000","last_became_peered":"2024-09-10T18:45:22.061428+0000","last_unstale":"2024-09-10T18:46:29.741750+0000","last_undegraded":"2024-09-10T18:46:29.741750+0000","last_fullsized":"2024-09-10T18:46:29.741750+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:41.151863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.791463+0000","last_change":"2024-09-10T18:45:24.073623+0000","last_active":"2024-09-10T18:46:29.791463+0000","last_peered":"2024-09-10T18:46:29.791463+0000","last_clean":"2024-09-10T18:46:29.791463+0000","last_became_active":"2024-09-10T18:45:24.073307+0000","last_became_peered":"2024-09-10T18:45:24.073307+0000","last_unstale":"2024-09-10T18:46:29.791463+0000","last_undegraded":"2024-09-10T18:46:29.791463+0000","last_fullsized":"2024-09-10T18:46:29.791463+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:51.119822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"63'13","reported_seq":56,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742830+0000","last_change":"2024-09-10T18:45:20.058778+0000","last_active":"2024-09-10T18:46:29.742830+0000","last_peered":"2024-09-10T18:46:29.742830+0000","last_clean":"2024-09-10T18:46:29.742830+0000","last_became_active":"2024-09-10T18:45:20.058435+0000","last_became_peered":"2024-09-10T18:45:20.058435+0000","last_unstale":"2024-09-10T18:46:29.742830+0000","last_undegraded":"2024-09-10T18:46:29.742830+0000","last_fullsized":"2024-09-10T18:46:29.742830+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:44.903372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"65'5","reported_seq":114,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:39.364272+0000","last_change":"2024-09-10T18:45:22.059997+0000","last_active":"2024-09-10T18:46:39.364272+0000","last_peered":"2024-09-10T18:46:39.364272+0000","last_clean":"2024-09-10T18:46:39.364272+0000","last_became_active":"2024-09-10T18:45:17.969868+0000","last_became_peered":"2024-09-10T18:45:17.969868+0000","last_unstale":"2024-09-10T18:46:39.364272+0000","last_undegraded":"2024-09-10T18:46:39.364272+0000","last_fullsized":"2024-09-10T18:46:39.364272+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:45.998919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021933159999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":75,"num_read_kb":70,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.566415+0000","last_change":"2024-09-10T18:45:17.961131+0000","last_active":"2024-09-10T18:46:30.566415+0000","last_peered":"2024-09-10T18:46:30.566415+0000","last_clean":"2024-09-10T18:46:30.566415+0000","last_became_active":"2024-09-10T18:45:17.960859+0000","last_became_peered":"2024-09-10T18:45:17.960859+0000","last_unstale":"2024-09-10T18:46:30.566415+0000","last_undegraded":"2024-09-10T18:46:30.566415+0000","last_fullsized":"2024-09-10T18:46:30.566415+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:42.810415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.567280+0000","last_change":"2024-09-10T18:45:22.068719+0000","last_active":"2024-09-10T18:46:30.567280+0000","last_peered":"2024-09-10T18:46:30.567280+0000","last_clean":"2024-09-10T18:46:30.567280+0000","last_became_active":"2024-09-10T18:45:22.068350+0000","last_became_peered":"2024-09-10T18:45:22.068350+0000","last_unstale":"2024-09-10T18:46:30.567280+0000","last_undegraded":"2024-09-10T18:46:30.567280+0000","last_fullsized":"2024-09-10T18:46:30.567280+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:34.173124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.567760+0000","last_change":"2024-09-10T18:45:24.264879+0000","last_active":"2024-09-10T18:46:30.567760+0000","last_peered":"2024-09-10T18:46:30.567760+0000","last_clean":"2024-09-10T18:46:30.567760+0000","last_became_active":"2024-09-10T18:45:24.264617+0000","last_became_peered":"2024-09-10T18:45:24.264617+0000","last_unstale":"2024-09-10T18:46:30.567760+0000","last_undegraded":"2024-09-10T18:46:30.567760+0000","last_fullsized":"2024-09-10T18:46:30.567760+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:40.675862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"63'12","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742416+0000","last_change":"2024-09-10T18:45:20.061430+0000","last_active":"2024-09-10T18:46:29.742416+0000","last_peered":"2024-09-10T18:46:29.742416+0000","last_clean":"2024-09-10T18:46:29.742416+0000","last_became_active":"2024-09-10T18:45:20.061091+0000","last_became_peered":"2024-09-10T18:45:20.061091+0000","last_unstale":"2024-09-10T18:46:29.742416+0000","last_undegraded":"2024-09-10T18:46:29.742416+0000","last_fullsized":"2024-09-10T18:46:29.742416+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:38.785499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"58'1","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.741342+0000","last_change":"2024-09-10T18:45:22.055928+0000","last_active":"2024-09-10T18:46:29.741342+0000","last_peered":"2024-09-10T18:46:29.741342+0000","last_clean":"2024-09-10T18:46:29.741342+0000","last_became_active":"2024-09-10T18:45:17.963932+0000","last_became_peered":"2024-09-10T18:45:17.963932+0000","last_unstale":"2024-09-10T18:46:29.741342+0000","last_undegraded":"2024-09-10T18:46:29.741342+0000","last_fullsized":"2024-09-10T18:46:29.741342+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:45.517533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037812099999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.566408+0000","last_change":"2024-09-10T18:45:17.966921+0000","last_active":"2024-09-10T18:46:30.566408+0000","last_peered":"2024-09-10T18:46:30.566408+0000","last_clean":"2024-09-10T18:46:30.566408+0000","last_became_active":"2024-09-10T18:45:17.966493+0000","last_became_peered":"2024-09-10T18:45:17.966493+0000","last_unstale":"2024-09-10T18:46:30.566408+0000","last_undegraded":"2024-09-10T18:46:30.566408+0000","last_fullsized":"2024-09-10T18:46:30.566408+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:37.560371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742429+0000","last_change":"2024-09-10T18:45:22.064783+0000","last_active":"2024-09-10T18:46:29.742429+0000","last_peered":"2024-09-10T18:46:29.742429+0000","last_clean":"2024-09-10T18:46:29.742429+0000","last_became_active":"2024-09-10T18:45:22.064285+0000","last_became_peered":"2024-09-10T18:45:22.064285+0000","last_unstale":"2024-09-10T18:46:29.742429+0000","last_undegraded":"2024-09-10T18:46:29.742429+0000","last_fullsized":"2024-09-10T18:46:29.742429+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:46.039937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.741935+0000","last_change":"2024-09-10T18:45:24.084038+0000","last_active":"2024-09-10T18:46:29.741935+0000","last_peered":"2024-09-10T18:46:29.741935+0000","last_clean":"2024-09-10T18:46:29.741935+0000","last_became_active":"2024-09-10T18:45:24.083689+0000","last_became_peered":"2024-09-10T18:45:24.083689+0000","last_unstale":"2024-09-10T18:46:29.741935+0000","last_undegraded":"2024-09-10T18:46:29.741935+0000","last_fullsized":"2024-09-10T18:46:29.741935+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:36.059986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"63'14","reported_seq":61,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.792381+0000","last_change":"2024-09-10T18:46:22.294141+0000","last_active":"2024-09-10T18:46:29.792381+0000","last_peered":"2024-09-10T18:46:29.792381+0000","last_clean":"2024-09-10T18:46:29.792381+0000","last_became_active":"2024-09-10T18:45:20.421119+0000","last_became_peered":"2024-09-10T18:45:20.421119+0000","last_unstale":"2024-09-10T18:46:29.792381+0000","last_undegraded":"2024-09-10T18:46:29.792381+0000","last_fullsized":"2024-09-10T18:46:29.792381+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-10T18:46:22.294072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:22.294072+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:49.887899+0000","scrub_duration":0.0021435069999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"60'2","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.790956+0000","last_change":"2024-09-10T18:46:25.845131+0000","last_active":"2024-09-10T18:46:29.790956+0000","last_peered":"2024-09-10T18:46:29.790956+0000","last_clean":"2024-09-10T18:46:29.790956+0000","last_became_active":"2024-09-10T18:45:17.961660+0000","last_became_peered":"2024-09-10T18:45:17.961660+0000","last_unstale":"2024-09-10T18:46:29.790956+0000","last_undegraded":"2024-09-10T18:46:29.790956+0000","last_fullsized":"2024-09-10T18:46:29.790956+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-10T18:46:25.845070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:25.845070+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:50.370600+0000","scrub_duration":0.0020129710000000001,"objects_trimmed":0,"snaptrim_duration":0.0013942640000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.741918+0000","last_change":"2024-09-10T18:46:29.081362+0000","last_active":"2024-09-10T18:46:29.741918+0000","last_peered":"2024-09-10T18:46:29.741918+0000","last_clean":"2024-09-10T18:46:29.741918+0000","last_became_active":"2024-09-10T18:45:17.965163+0000","last_became_peered":"2024-09-10T18:45:17.965163+0000","last_unstale":"2024-09-10T18:46:29.741918+0000","last_undegraded":"2024-09-10T18:46:29.741918+0000","last_fullsized":"2024-09-10T18:46:29.741918+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:29.081296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:29.081296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:50.659900+0000","scrub_duration":0.0015488590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'81","reported_seq":75,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.567438+0000","last_change":"2024-09-10T18:45:45.481030+0000","last_active":"2024-09-10T18:46:30.567438+0000","last_peered":"2024-09-10T18:46:30.567438+0000","last_clean":"2024-09-10T18:46:30.567438+0000","last_became_active":"2024-09-10T18:45:07.433592+0000","last_became_peered":"2024-09-10T18:45:07.433592+0000","last_unstale":"2024-09-10T18:46:30.567438+0000","last_undegraded":"2024-09-10T18:46:30.567438+0000","last_fullsized":"2024-09-10T18:46:30.567438+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'74","last_scrub_stamp":"2024-09-10T18:45:45.480958+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:45:45.480958+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:58.297247+0000","scrub_duration":0.0019876809999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.567749+0000","last_change":"2024-09-10T18:46:26.687029+0000","last_active":"2024-09-10T18:46:30.567749+0000","last_peered":"2024-09-10T18:46:30.567749+0000","last_clean":"2024-09-10T18:46:30.567749+0000","last_became_active":"2024-09-10T18:45:22.068226+0000","last_became_peered":"2024-09-10T18:45:22.068226+0000","last_unstale":"2024-09-10T18:46:30.567749+0000","last_undegraded":"2024-09-10T18:46:30.567749+0000","last_fullsized":"2024-09-10T18:46:30.567749+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:26.686961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:26.686961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:32.456741+0000","scrub_duration":0.0015985109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742521+0000","last_change":"2024-09-10T18:45:24.073576+0000","last_active":"2024-09-10T18:46:29.742521+0000","last_peered":"2024-09-10T18:46:29.742521+0000","last_clean":"2024-09-10T18:46:29.742521+0000","last_became_active":"2024-09-10T18:45:24.073327+0000","last_became_peered":"2024-09-10T18:45:24.073327+0000","last_unstale":"2024-09-10T18:46:29.742521+0000","last_undegraded":"2024-09-10T18:46:29.742521+0000","last_fullsized":"2024-09-10T18:46:29.742521+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:45.713352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"63'19","reported_seq":70,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.792098+0000","last_change":"2024-09-10T18:46:28.229804+0000","last_active":"2024-09-10T18:46:29.792098+0000","last_peered":"2024-09-10T18:46:29.792098+0000","last_clean":"2024-09-10T18:46:29.792098+0000","last_became_active":"2024-09-10T18:45:20.421097+0000","last_became_peered":"2024-09-10T18:45:20.421097+0000","last_unstale":"2024-09-10T18:46:29.792098+0000","last_undegraded":"2024-09-10T18:46:29.792098+0000","last_fullsized":"2024-09-10T18:46:29.792098+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:46:28.229737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:28.229737+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:49.859218+0000","scrub_duration":0.002308184,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.566531+0000","last_change":"2024-09-10T18:45:17.960491+0000","last_active":"2024-09-10T18:46:30.566531+0000","last_peered":"2024-09-10T18:46:30.566531+0000","last_clean":"2024-09-10T18:46:30.566531+0000","last_became_active":"2024-09-10T18:45:17.959924+0000","last_became_peered":"2024-09-10T18:45:17.959924+0000","last_unstale":"2024-09-10T18:46:30.566531+0000","last_undegraded":"2024-09-10T18:46:30.566531+0000","last_fullsized":"2024-09-10T18:46:30.566531+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:43.353373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.781831+0000","last_change":"2024-09-10T18:46:30.781786+0000","last_active":"2024-09-10T18:46:30.781831+0000","last_peered":"2024-09-10T18:46:30.781831+0000","last_clean":"2024-09-10T18:46:30.781831+0000","last_became_active":"2024-09-10T18:45:22.072581+0000","last_became_peered":"2024-09-10T18:45:22.072581+0000","last_unstale":"2024-09-10T18:46:30.781831+0000","last_undegraded":"2024-09-10T18:46:30.781831+0000","last_fullsized":"2024-09-10T18:46:30.781831+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:30.781730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:30.781730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:38.248319+0000","scrub_duration":0.0020255870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.567360+0000","last_change":"2024-09-10T18:45:24.076661+0000","last_active":"2024-09-10T18:46:30.567360+0000","last_peered":"2024-09-10T18:46:30.567360+0000","last_clean":"2024-09-10T18:46:30.567360+0000","last_became_active":"2024-09-10T18:45:24.076392+0000","last_became_peered":"2024-09-10T18:45:24.076392+0000","last_unstale":"2024-09-10T18:46:30.567360+0000","last_undegraded":"2024-09-10T18:46:30.567360+0000","last_fullsized":"2024-09-10T18:46:30.567360+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:47.306399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"63'9","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.741579+0000","last_change":"2024-09-10T18:45:20.061357+0000","last_active":"2024-09-10T18:46:29.741579+0000","last_peered":"2024-09-10T18:46:29.741579+0000","last_clean":"2024-09-10T18:46:29.741579+0000","last_became_active":"2024-09-10T18:45:20.061021+0000","last_became_peered":"2024-09-10T18:45:20.061021+0000","last_unstale":"2024-09-10T18:46:29.741579+0000","last_undegraded":"2024-09-10T18:46:29.741579+0000","last_fullsized":"2024-09-10T18:46:29.741579+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:35.559073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742729+0000","last_change":"2024-09-10T18:45:17.963854+0000","last_active":"2024-09-10T18:46:29.742729+0000","last_peered":"2024-09-10T18:46:29.742729+0000","last_clean":"2024-09-10T18:46:29.742729+0000","last_became_active":"2024-09-10T18:45:17.963057+0000","last_became_peered":"2024-09-10T18:45:17.963057+0000","last_unstale":"2024-09-10T18:46:29.742729+0000","last_undegraded":"2024-09-10T18:46:29.742729+0000","last_fullsized":"2024-09-10T18:46:29.742729+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:40.229149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.566905+0000","last_change":"2024-09-10T18:45:22.070613+0000","last_active":"2024-09-10T18:46:30.566905+0000","last_peered":"2024-09-10T18:46:30.566905+0000","last_clean":"2024-09-10T18:46:30.566905+0000","last_became_active":"2024-09-10T18:45:22.070343+0000","last_became_peered":"2024-09-10T18:45:22.070343+0000","last_unstale":"2024-09-10T18:46:30.566905+0000","last_undegraded":"2024-09-10T18:46:30.566905+0000","last_fullsized":"2024-09-10T18:46:30.566905+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:46.315675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.741416+0000","last_change":"2024-09-10T18:45:24.078848+0000","last_active":"2024-09-10T18:46:29.741416+0000","last_peered":"2024-09-10T18:46:29.741416+0000","last_clean":"2024-09-10T18:46:29.741416+0000","last_became_active":"2024-09-10T18:45:24.078529+0000","last_became_peered":"2024-09-10T18:45:24.078529+0000","last_unstale":"2024-09-10T18:46:29.741416+0000","last_undegraded":"2024-09-10T18:46:29.741416+0000","last_fullsized":"2024-09-10T18:46:29.741416+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"63'15","reported_seq":54,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.741514+0000","last_change":"2024-09-10T18:45:20.420130+0000","last_active":"2024-09-10T18:46:29.741514+0000","last_peered":"2024-09-10T18:46:29.741514+0000","last_clean":"2024-09-10T18:46:29.741514+0000","last_became_active":"2024-09-10T18:45:20.419804+0000","last_became_peered":"2024-09-10T18:45:20.419804+0000","last_unstale":"2024-09-10T18:46:29.741514+0000","last_undegraded":"2024-09-10T18:46:29.741514+0000","last_fullsized":"2024-09-10T18:46:29.741514+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:38.377852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.566473+0000","last_change":"2024-09-10T18:46:17.819966+0000","last_active":"2024-09-10T18:46:30.566473+0000","last_peered":"2024-09-10T18:46:30.566473+0000","last_clean":"2024-09-10T18:46:30.566473+0000","last_became_active":"2024-09-10T18:45:17.959653+0000","last_became_peered":"2024-09-10T18:45:17.959653+0000","last_unstale":"2024-09-10T18:46:30.566473+0000","last_undegraded":"2024-09-10T18:46:30.566473+0000","last_fullsized":"2024-09-10T18:46:30.566473+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:17.819909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:17.819909+0000","last_clean_scrub_stamp":"2024-09-10T18:46:17.819909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:18.454145+0000","scrub_duration":0.001612109,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"63'8","reported_seq":54,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:33.822272+0000","last_change":"2024-09-10T18:45:22.073687+0000","last_active":"2024-09-10T18:46:33.822272+0000","last_peered":"2024-09-10T18:46:33.822272+0000","last_clean":"2024-09-10T18:46:33.822272+0000","last_became_active":"2024-09-10T18:45:22.073383+0000","last_became_peered":"2024-09-10T18:45:22.073383+0000","last_unstale":"2024-09-10T18:46:33.822272+0000","last_undegraded":"2024-09-10T18:46:33.822272+0000","last_fullsized":"2024-09-10T18:46:33.822272+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:44.506736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.741681+0000","last_change":"2024-09-10T18:45:24.265594+0000","last_active":"2024-09-10T18:46:29.741681+0000","last_peered":"2024-09-10T18:46:29.741681+0000","last_clean":"2024-09-10T18:46:29.741681+0000","last_became_active":"2024-09-10T18:45:24.265256+0000","last_became_peered":"2024-09-10T18:45:24.265256+0000","last_unstale":"2024-09-10T18:46:29.741681+0000","last_undegraded":"2024-09-10T18:46:29.741681+0000","last_fullsized":"2024-09-10T18:46:29.741681+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:34.150976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"63'12","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.320580+0000","last_change":"2024-09-10T18:46:30.320527+0000","last_active":"2024-09-10T18:46:30.320580+0000","last_peered":"2024-09-10T18:46:30.320580+0000","last_clean":"2024-09-10T18:46:30.320580+0000","last_became_active":"2024-09-10T18:45:20.056869+0000","last_became_peered":"2024-09-10T18:45:20.056869+0000","last_unstale":"2024-09-10T18:46:30.320580+0000","last_undegraded":"2024-09-10T18:46:30.320580+0000","last_fullsized":"2024-09-10T18:46:30.320580+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:46:30.320462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:30.320462+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:35.180189+0000","scrub_duration":0.0028822850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"56'2","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.318913+0000","last_change":"2024-09-10T18:45:17.963577+0000","last_active":"2024-09-10T18:46:30.318913+0000","last_peered":"2024-09-10T18:46:30.318913+0000","last_clean":"2024-09-10T18:46:30.318913+0000","last_became_active":"2024-09-10T18:45:17.963286+0000","last_became_peered":"2024-09-10T18:45:17.963286+0000","last_unstale":"2024-09-10T18:46:30.318913+0000","last_undegraded":"2024-09-10T18:46:30.318913+0000","last_fullsized":"2024-09-10T18:46:30.318913+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:37.304000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.566744+0000","last_change":"2024-09-10T18:45:22.069278+0000","last_active":"2024-09-10T18:46:30.566744+0000","last_peered":"2024-09-10T18:46:30.566744+0000","last_clean":"2024-09-10T18:46:30.566744+0000","last_became_active":"2024-09-10T18:45:22.068957+0000","last_became_peered":"2024-09-10T18:45:22.068957+0000","last_unstale":"2024-09-10T18:46:30.566744+0000","last_undegraded":"2024-09-10T18:46:30.566744+0000","last_fullsized":"2024-09-10T18:46:30.566744+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:41.217006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.790914+0000","last_change":"2024-09-10T18:45:24.078134+0000","last_active":"2024-09-10T18:46:29.790914+0000","last_peered":"2024-09-10T18:46:29.790914+0000","last_clean":"2024-09-10T18:46:29.790914+0000","last_became_active":"2024-09-10T18:45:24.077815+0000","last_became_peered":"2024-09-10T18:45:24.077815+0000","last_unstale":"2024-09-10T18:46:29.790914+0000","last_undegraded":"2024-09-10T18:46:29.790914+0000","last_fullsized":"2024-09-10T18:46:29.790914+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:49.952536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"76'13","reported_seq":59,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742416+0000","last_change":"2024-09-10T18:46:24.626912+0000","last_active":"2024-09-10T18:46:29.742416+0000","last_peered":"2024-09-10T18:46:29.742416+0000","last_clean":"2024-09-10T18:46:29.742416+0000","last_became_active":"2024-09-10T18:45:20.420421+0000","last_became_peered":"2024-09-10T18:45:20.420421+0000","last_unstale":"2024-09-10T18:46:29.742416+0000","last_undegraded":"2024-09-10T18:46:29.742416+0000","last_fullsized":"2024-09-10T18:46:29.742416+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'13","last_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_deep_scrub":"76'13","last_deep_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_clean_scrub_stamp":"2024-09-10T18:46:24.626859+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:41.087073+0000","scrub_duration":0.0035642949999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:31.734562+0000","last_change":"2024-09-10T18:46:31.734522+0000","last_active":"2024-09-10T18:46:31.734562+0000","last_peered":"2024-09-10T18:46:31.734562+0000","last_clean":"2024-09-10T18:46:31.734562+0000","last_became_active":"2024-09-10T18:45:17.965112+0000","last_became_peered":"2024-09-10T18:45:17.965112+0000","last_unstale":"2024-09-10T18:46:31.734562+0000","last_undegraded":"2024-09-10T18:46:31.734562+0000","last_fullsized":"2024-09-10T18:46:31.734562+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:31.734459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:31.734459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:41.550267+0000","scrub_duration":0.0015571960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:32.829641+0000","last_change":"2024-09-10T18:46:32.829583+0000","last_active":"2024-09-10T18:46:32.829641+0000","last_peered":"2024-09-10T18:46:32.829641+0000","last_clean":"2024-09-10T18:46:32.829641+0000","last_became_active":"2024-09-10T18:45:22.069040+0000","last_became_peered":"2024-09-10T18:45:22.069040+0000","last_unstale":"2024-09-10T18:46:32.829641+0000","last_undegraded":"2024-09-10T18:46:32.829641+0000","last_fullsized":"2024-09-10T18:46:32.829641+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:32.829508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:32.829508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:45.869787+0000","scrub_duration":0.001897239,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"63'1","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.567035+0000","last_change":"2024-09-10T18:45:24.079638+0000","last_active":"2024-09-10T18:46:30.567035+0000","last_peered":"2024-09-10T18:46:30.567035+0000","last_clean":"2024-09-10T18:46:30.567035+0000","last_became_active":"2024-09-10T18:45:24.079258+0000","last_became_peered":"2024-09-10T18:45:24.079258+0000","last_unstale":"2024-09-10T18:46:30.567035+0000","last_undegraded":"2024-09-10T18:46:30.567035+0000","last_fullsized":"2024-09-10T18:46:30.567035+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:50.668523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.792234+0000","last_change":"2024-09-10T18:45:17.965220+0000","last_active":"2024-09-10T18:46:29.792234+0000","last_peered":"2024-09-10T18:46:29.792234+0000","last_clean":"2024-09-10T18:46:29.792234+0000","last_became_active":"2024-09-10T18:45:17.964705+0000","last_became_peered":"2024-09-10T18:45:17.964705+0000","last_unstale":"2024-09-10T18:46:29.792234+0000","last_undegraded":"2024-09-10T18:46:29.792234+0000","last_fullsized":"2024-09-10T18:46:29.792234+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:45.947390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'6","reported_seq":38,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.791411+0000","last_change":"2024-09-10T18:45:20.422485+0000","last_active":"2024-09-10T18:46:29.791411+0000","last_peered":"2024-09-10T18:46:29.791411+0000","last_clean":"2024-09-10T18:46:29.791411+0000","last_became_active":"2024-09-10T18:45:20.421746+0000","last_became_peered":"2024-09-10T18:45:20.421746+0000","last_unstale":"2024-09-10T18:46:29.791411+0000","last_undegraded":"2024-09-10T18:46:29.791411+0000","last_fullsized":"2024-09-10T18:46:29.791411+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:47.024991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:39.084522+0000","last_change":"2024-09-10T18:46:39.084489+0000","last_active":"2024-09-10T18:46:39.084522+0000","last_peered":"2024-09-10T18:46:39.084522+0000","last_clean":"2024-09-10T18:46:39.084522+0000","last_became_active":"2024-09-10T18:45:22.069653+0000","last_became_peered":"2024-09-10T18:45:22.069653+0000","last_unstale":"2024-09-10T18:46:39.084522+0000","last_undegraded":"2024-09-10T18:46:39.084522+0000","last_fullsized":"2024-09-10T18:46:39.084522+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:39.084433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:39.084433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:05.431757+0000","scrub_duration":0.0014845819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.567759+0000","last_change":"2024-09-10T18:45:24.264869+0000","last_active":"2024-09-10T18:46:30.567759+0000","last_peered":"2024-09-10T18:46:30.567759+0000","last_clean":"2024-09-10T18:46:30.567759+0000","last_became_active":"2024-09-10T18:45:24.264611+0000","last_became_peered":"2024-09-10T18:45:24.264611+0000","last_unstale":"2024-09-10T18:46:30.567759+0000","last_undegraded":"2024-09-10T18:46:30.567759+0000","last_fullsized":"2024-09-10T18:46:30.567759+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:35.281392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"63'10","reported_seq":44,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.791515+0000","last_change":"2024-09-10T18:45:20.420217+0000","last_active":"2024-09-10T18:46:29.791515+0000","last_peered":"2024-09-10T18:46:29.791515+0000","last_clean":"2024-09-10T18:46:29.791515+0000","last_became_active":"2024-09-10T18:45:20.419887+0000","last_became_peered":"2024-09-10T18:45:20.419887+0000","last_unstale":"2024-09-10T18:46:29.791515+0000","last_undegraded":"2024-09-10T18:46:29.791515+0000","last_fullsized":"2024-09-10T18:46:29.791515+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:39.004054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.741925+0000","last_change":"2024-09-10T18:45:17.965293+0000","last_active":"2024-09-10T18:46:29.741925+0000","last_peered":"2024-09-10T18:46:29.741925+0000","last_clean":"2024-09-10T18:46:29.741925+0000","last_became_active":"2024-09-10T18:45:17.964870+0000","last_became_peered":"2024-09-10T18:45:17.964870+0000","last_unstale":"2024-09-10T18:46:29.741925+0000","last_undegraded":"2024-09-10T18:46:29.741925+0000","last_fullsized":"2024-09-10T18:46:29.741925+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:37.495417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"63'8","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:34.085287+0000","last_change":"2024-09-10T18:46:34.085224+0000","last_active":"2024-09-10T18:46:34.085287+0000","last_peered":"2024-09-10T18:46:34.085287+0000","last_clean":"2024-09-10T18:46:34.085287+0000","last_became_active":"2024-09-10T18:45:22.062940+0000","last_became_peered":"2024-09-10T18:45:22.062940+0000","last_unstale":"2024-09-10T18:46:34.085287+0000","last_undegraded":"2024-09-10T18:46:34.085287+0000","last_fullsized":"2024-09-10T18:46:34.085287+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:34.085156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:34.085156+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:54.777061+0000","scrub_duration":0.0020843120000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.741832+0000","last_change":"2024-09-10T18:45:24.074039+0000","last_active":"2024-09-10T18:46:29.741832+0000","last_peered":"2024-09-10T18:46:29.741832+0000","last_clean":"2024-09-10T18:46:29.741832+0000","last_became_active":"2024-09-10T18:45:24.073736+0000","last_became_peered":"2024-09-10T18:45:24.073736+0000","last_unstale":"2024-09-10T18:46:29.741832+0000","last_undegraded":"2024-09-10T18:46:29.741832+0000","last_fullsized":"2024-09-10T18:46:29.741832+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:47.511899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"63'9","reported_seq":55,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742224+0000","last_change":"2024-09-10T18:46:23.086274+0000","last_active":"2024-09-10T18:46:29.742224+0000","last_peered":"2024-09-10T18:46:29.742224+0000","last_clean":"2024-09-10T18:46:29.742224+0000","last_became_active":"2024-09-10T18:45:20.421124+0000","last_became_peered":"2024-09-10T18:45:20.421124+0000","last_unstale":"2024-09-10T18:46:29.742224+0000","last_undegraded":"2024-09-10T18:46:29.742224+0000","last_fullsized":"2024-09-10T18:46:29.742224+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:46:23.086203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:23.086203+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:44.793257+0000","scrub_duration":0.002168068,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.741702+0000","last_change":"2024-09-10T18:45:17.960387+0000","last_active":"2024-09-10T18:46:29.741702+0000","last_peered":"2024-09-10T18:46:29.741702+0000","last_clean":"2024-09-10T18:46:29.741702+0000","last_became_active":"2024-09-10T18:45:17.960106+0000","last_became_peered":"2024-09-10T18:45:17.960106+0000","last_unstale":"2024-09-10T18:46:29.741702+0000","last_undegraded":"2024-09-10T18:46:29.741702+0000","last_fullsized":"2024-09-10T18:46:29.741702+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"63'8","reported_seq":51,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:33.593126+0000","last_change":"2024-09-10T18:45:22.075677+0000","last_active":"2024-09-10T18:46:33.593126+0000","last_peered":"2024-09-10T18:46:33.593126+0000","last_clean":"2024-09-10T18:46:33.593126+0000","last_became_active":"2024-09-10T18:45:22.075368+0000","last_became_peered":"2024-09-10T18:45:22.075368+0000","last_unstale":"2024-09-10T18:46:33.593126+0000","last_undegraded":"2024-09-10T18:46:33.593126+0000","last_fullsized":"2024-09-10T18:46:33.593126+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:39.560512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:36.305295+0000","last_change":"2024-09-10T18:46:36.305262+0000","last_active":"2024-09-10T18:46:36.305295+0000","last_peered":"2024-09-10T18:46:36.305295+0000","last_clean":"2024-09-10T18:46:36.305295+0000","last_became_active":"2024-09-10T18:45:24.080196+0000","last_became_peered":"2024-09-10T18:45:24.080196+0000","last_unstale":"2024-09-10T18:46:36.305295+0000","last_undegraded":"2024-09-10T18:46:36.305295+0000","last_fullsized":"2024-09-10T18:46:36.305295+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_clean_scrub_stamp":"2024-09-10T18:46:36.305199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:04.747507+0000","scrub_duration":0.0013794809999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"63'9","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.741371+0000","last_change":"2024-09-10T18:45:20.057767+0000","last_active":"2024-09-10T18:46:29.741371+0000","last_peered":"2024-09-10T18:46:29.741371+0000","last_clean":"2024-09-10T18:46:29.741371+0000","last_became_active":"2024-09-10T18:45:20.057479+0000","last_became_peered":"2024-09-10T18:45:20.057479+0000","last_unstale":"2024-09-10T18:46:29.741371+0000","last_undegraded":"2024-09-10T18:46:29.741371+0000","last_fullsized":"2024-09-10T18:46:29.741371+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:40.060026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.792260+0000","last_change":"2024-09-10T18:46:23.292276+0000","last_active":"2024-09-10T18:46:29.792260+0000","last_peered":"2024-09-10T18:46:29.792260+0000","last_clean":"2024-09-10T18:46:29.792260+0000","last_became_active":"2024-09-10T18:45:17.962009+0000","last_became_peered":"2024-09-10T18:45:17.962009+0000","last_unstale":"2024-09-10T18:46:29.792260+0000","last_undegraded":"2024-09-10T18:46:29.792260+0000","last_fullsized":"2024-09-10T18:46:29.792260+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:23.292207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:23.292207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:27.416585+0000","scrub_duration":0.001659953,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.791128+0000","last_change":"2024-09-10T18:45:22.073186+0000","last_active":"2024-09-10T18:46:29.791128+0000","last_peered":"2024-09-10T18:46:29.791128+0000","last_clean":"2024-09-10T18:46:29.791128+0000","last_became_active":"2024-09-10T18:45:22.072877+0000","last_became_peered":"2024-09-10T18:45:22.072877+0000","last_unstale":"2024-09-10T18:46:29.791128+0000","last_undegraded":"2024-09-10T18:46:29.791128+0000","last_fullsized":"2024-09-10T18:46:29.791128+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:47.886075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.740781+0000","last_change":"2024-09-10T18:45:24.081786+0000","last_active":"2024-09-10T18:46:29.740781+0000","last_peered":"2024-09-10T18:46:29.740781+0000","last_clean":"2024-09-10T18:46:29.740781+0000","last_became_active":"2024-09-10T18:45:24.081050+0000","last_became_peered":"2024-09-10T18:45:24.081050+0000","last_unstale":"2024-09-10T18:46:29.740781+0000","last_undegraded":"2024-09-10T18:46:29.740781+0000","last_fullsized":"2024-09-10T18:46:29.740781+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:35.498799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"63'11","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.319566+0000","last_change":"2024-09-10T18:45:20.057563+0000","last_active":"2024-09-10T18:46:30.319566+0000","last_peered":"2024-09-10T18:46:30.319566+0000","last_clean":"2024-09-10T18:46:30.319566+0000","last_became_active":"2024-09-10T18:45:20.056885+0000","last_became_peered":"2024-09-10T18:45:20.056885+0000","last_unstale":"2024-09-10T18:46:30.319566+0000","last_undegraded":"2024-09-10T18:46:30.319566+0000","last_fullsized":"2024-09-10T18:46:30.319566+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:47.219111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:36.402643+0000","last_change":"2024-09-10T18:46:36.402610+0000","last_active":"2024-09-10T18:46:36.402643+0000","last_peered":"2024-09-10T18:46:36.402643+0000","last_clean":"2024-09-10T18:46:36.402643+0000","last_became_active":"2024-09-10T18:45:17.963542+0000","last_became_peered":"2024-09-10T18:45:17.963542+0000","last_unstale":"2024-09-10T18:46:36.402643+0000","last_undegraded":"2024-09-10T18:46:36.402643+0000","last_fullsized":"2024-09-10T18:46:36.402643+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:36.402548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:36.402548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:04.495178+0000","scrub_duration":0.0014103919999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:37.414825+0000","last_change":"2024-09-10T18:46:37.414794+0000","last_active":"2024-09-10T18:46:37.414825+0000","last_peered":"2024-09-10T18:46:37.414825+0000","last_clean":"2024-09-10T18:46:37.414825+0000","last_became_active":"2024-09-10T18:45:22.070065+0000","last_became_peered":"2024-09-10T18:45:22.070065+0000","last_unstale":"2024-09-10T18:46:37.414825+0000","last_undegraded":"2024-09-10T18:46:37.414825+0000","last_fullsized":"2024-09-10T18:46:37.414825+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:37.414734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:37.414734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:44.027335+0000","scrub_duration":0.001607996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.790714+0000","last_change":"2024-09-10T18:45:24.072477+0000","last_active":"2024-09-10T18:46:29.790714+0000","last_peered":"2024-09-10T18:46:29.790714+0000","last_clean":"2024-09-10T18:46:29.790714+0000","last_became_active":"2024-09-10T18:45:24.072224+0000","last_became_peered":"2024-09-10T18:45:24.072224+0000","last_unstale":"2024-09-10T18:46:29.790714+0000","last_undegraded":"2024-09-10T18:46:29.790714+0000","last_fullsized":"2024-09-10T18:46:29.790714+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:47.793658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"63'4","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.790674+0000","last_change":"2024-09-10T18:45:20.059495+0000","last_active":"2024-09-10T18:46:29.790674+0000","last_peered":"2024-09-10T18:46:29.790674+0000","last_clean":"2024-09-10T18:46:29.790674+0000","last_became_active":"2024-09-10T18:45:20.059205+0000","last_became_peered":"2024-09-10T18:45:20.059205+0000","last_unstale":"2024-09-10T18:46:29.790674+0000","last_undegraded":"2024-09-10T18:46:29.790674+0000","last_fullsized":"2024-09-10T18:46:29.790674+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:33.725425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742392+0000","last_change":"2024-09-10T18:45:17.963898+0000","last_active":"2024-09-10T18:46:29.742392+0000","last_peered":"2024-09-10T18:46:29.742392+0000","last_clean":"2024-09-10T18:46:29.742392+0000","last_became_active":"2024-09-10T18:45:17.963630+0000","last_became_peered":"2024-09-10T18:45:17.963630+0000","last_unstale":"2024-09-10T18:46:29.742392+0000","last_undegraded":"2024-09-10T18:46:29.742392+0000","last_fullsized":"2024-09-10T18:46:29.742392+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:44.130712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:35.172968+0000","last_change":"2024-09-10T18:46:35.172936+0000","last_active":"2024-09-10T18:46:35.172968+0000","last_peered":"2024-09-10T18:46:35.172968+0000","last_clean":"2024-09-10T18:46:35.172968+0000","last_became_active":"2024-09-10T18:45:22.072377+0000","last_became_peered":"2024-09-10T18:45:22.072377+0000","last_unstale":"2024-09-10T18:46:35.172968+0000","last_undegraded":"2024-09-10T18:46:35.172968+0000","last_fullsized":"2024-09-10T18:46:35.172968+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_clean_scrub_stamp":"2024-09-10T18:46:35.172861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:57.638389+0000","scrub_duration":0.001552933,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.567431+0000","last_change":"2024-09-10T18:45:24.077706+0000","last_active":"2024-09-10T18:46:30.567431+0000","last_peered":"2024-09-10T18:46:30.567431+0000","last_clean":"2024-09-10T18:46:30.567431+0000","last_became_active":"2024-09-10T18:45:24.077398+0000","last_became_peered":"2024-09-10T18:45:24.077398+0000","last_unstale":"2024-09-10T18:46:30.567431+0000","last_undegraded":"2024-09-10T18:46:30.567431+0000","last_fullsized":"2024-09-10T18:46:30.567431+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:35.781159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"63'11","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.790845+0000","last_change":"2024-09-10T18:45:20.422212+0000","last_active":"2024-09-10T18:46:29.790845+0000","last_peered":"2024-09-10T18:46:29.790845+0000","last_clean":"2024-09-10T18:46:29.790845+0000","last_became_active":"2024-09-10T18:45:20.421479+0000","last_became_peered":"2024-09-10T18:45:20.421479+0000","last_unstale":"2024-09-10T18:46:29.790845+0000","last_undegraded":"2024-09-10T18:46:29.790845+0000","last_fullsized":"2024-09-10T18:46:29.790845+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:38.573185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"56'1","reported_seq":38,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.792614+0000","last_change":"2024-09-10T18:45:17.961527+0000","last_active":"2024-09-10T18:46:29.792614+0000","last_peered":"2024-09-10T18:46:29.792614+0000","last_clean":"2024-09-10T18:46:29.792614+0000","last_became_active":"2024-09-10T18:45:17.961222+0000","last_became_peered":"2024-09-10T18:45:17.961222+0000","last_unstale":"2024-09-10T18:46:29.792614+0000","last_undegraded":"2024-09-10T18:46:29.792614+0000","last_fullsized":"2024-09-10T18:46:29.792614+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:38.797176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.567191+0000","last_change":"2024-09-10T18:46:27.667088+0000","last_active":"2024-09-10T18:46:30.567191+0000","last_peered":"2024-09-10T18:46:30.567191+0000","last_clean":"2024-09-10T18:46:30.567191+0000","last_became_active":"2024-09-10T18:45:22.072312+0000","last_became_peered":"2024-09-10T18:45:22.072312+0000","last_unstale":"2024-09-10T18:46:30.567191+0000","last_undegraded":"2024-09-10T18:46:30.567191+0000","last_fullsized":"2024-09-10T18:46:30.567191+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:27.667018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:27.667018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:38.065664+0000","scrub_duration":0.001388886,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.791673+0000","last_change":"2024-09-10T18:46:29.791617+0000","last_active":"2024-09-10T18:46:29.791673+0000","last_peered":"2024-09-10T18:46:29.791673+0000","last_clean":"2024-09-10T18:46:29.791673+0000","last_became_active":"2024-09-10T18:45:24.264818+0000","last_became_peered":"2024-09-10T18:45:24.264818+0000","last_unstale":"2024-09-10T18:46:29.791673+0000","last_undegraded":"2024-09-10T18:46:29.791673+0000","last_fullsized":"2024-09-10T18:46:29.791673+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:29.791563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:29.791563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:37.941643+0000","scrub_duration":0.0021031819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"78'2","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:37.581259+0000","last_change":"2024-09-10T18:46:37.581201+0000","last_active":"2024-09-10T18:46:37.581259+0000","last_peered":"2024-09-10T18:46:37.581259+0000","last_clean":"2024-09-10T18:46:37.581259+0000","last_became_active":"2024-09-10T18:45:24.076380+0000","last_became_peered":"2024-09-10T18:45:24.076380+0000","last_unstale":"2024-09-10T18:46:37.581259+0000","last_undegraded":"2024-09-10T18:46:37.581259+0000","last_fullsized":"2024-09-10T18:46:37.581259+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'2","last_scrub_stamp":"2024-09-10T18:46:37.581130+0000","last_deep_scrub":"78'2","last_deep_scrub_stamp":"2024-09-10T18:46:37.581130+0000","last_clean_scrub_stamp":"2024-09-10T18:46:37.581130+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:55.887929+0000","scrub_duration":0.0024563609999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.741574+0000","last_change":"2024-09-10T18:46:28.088657+0000","last_active":"2024-09-10T18:46:29.741574+0000","last_peered":"2024-09-10T18:46:29.741574+0000","last_clean":"2024-09-10T18:46:29.741574+0000","last_became_active":"2024-09-10T18:45:17.962108+0000","last_became_peered":"2024-09-10T18:45:17.962108+0000","last_unstale":"2024-09-10T18:46:29.741574+0000","last_undegraded":"2024-09-10T18:46:29.741574+0000","last_fullsized":"2024-09-10T18:46:29.741574+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:28.088588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:28.088588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:56.550863+0000","scrub_duration":0.00140508,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"63'10","reported_seq":44,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742320+0000","last_change":"2024-09-10T18:45:20.421384+0000","last_active":"2024-09-10T18:46:29.742320+0000","last_peered":"2024-09-10T18:46:29.742320+0000","last_clean":"2024-09-10T18:46:29.742320+0000","last_became_active":"2024-09-10T18:45:20.421094+0000","last_became_peered":"2024-09-10T18:45:20.421094+0000","last_unstale":"2024-09-10T18:46:29.742320+0000","last_undegraded":"2024-09-10T18:46:29.742320+0000","last_fullsized":"2024-09-10T18:46:29.742320+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:34.663846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":64,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:33.471933+0000","last_change":"2024-09-10T18:46:33.150919+0000","last_active":"2024-09-10T18:46:33.471933+0000","last_peered":"2024-09-10T18:46:33.471933+0000","last_clean":"2024-09-10T18:46:33.471933+0000","last_became_active":"2024-09-10T18:45:22.073281+0000","last_became_peered":"2024-09-10T18:45:22.073281+0000","last_unstale":"2024-09-10T18:46:33.471933+0000","last_undegraded":"2024-09-10T18:46:33.471933+0000","last_fullsized":"2024-09-10T18:46:33.471933+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:33.150852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:33.150852+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:51.299257+0000","scrub_duration":0.0017025339999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.741108+0000","last_change":"2024-09-10T18:45:24.082638+0000","last_active":"2024-09-10T18:46:29.741108+0000","last_peered":"2024-09-10T18:46:29.741108+0000","last_clean":"2024-09-10T18:46:29.741108+0000","last_became_active":"2024-09-10T18:45:24.082318+0000","last_became_peered":"2024-09-10T18:45:24.082318+0000","last_unstale":"2024-09-10T18:46:29.741108+0000","last_undegraded":"2024-09-10T18:46:29.741108+0000","last_fullsized":"2024-09-10T18:46:29.741108+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:42.162847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.791133+0000","last_change":"2024-09-10T18:45:17.966027+0000","last_active":"2024-09-10T18:46:29.791133+0000","last_peered":"2024-09-10T18:46:29.791133+0000","last_clean":"2024-09-10T18:46:29.791133+0000","last_became_active":"2024-09-10T18:45:17.965707+0000","last_became_peered":"2024-09-10T18:45:17.965707+0000","last_unstale":"2024-09-10T18:46:29.791133+0000","last_undegraded":"2024-09-10T18:46:29.791133+0000","last_fullsized":"2024-09-10T18:46:29.791133+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:33.654064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"63'11","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.792605+0000","last_change":"2024-09-10T18:45:20.059017+0000","last_active":"2024-09-10T18:46:29.792605+0000","last_peered":"2024-09-10T18:46:29.792605+0000","last_clean":"2024-09-10T18:46:29.792605+0000","last_became_active":"2024-09-10T18:45:20.058717+0000","last_became_peered":"2024-09-10T18:45:20.058717+0000","last_unstale":"2024-09-10T18:46:29.792605+0000","last_undegraded":"2024-09-10T18:46:29.792605+0000","last_fullsized":"2024-09-10T18:46:29.792605+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:42.592911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742318+0000","last_change":"2024-09-10T18:45:22.068991+0000","last_active":"2024-09-10T18:46:29.742318+0000","last_peered":"2024-09-10T18:46:29.742318+0000","last_clean":"2024-09-10T18:46:29.742318+0000","last_became_active":"2024-09-10T18:45:22.068676+0000","last_became_peered":"2024-09-10T18:45:22.068676+0000","last_unstale":"2024-09-10T18:46:29.742318+0000","last_undegraded":"2024-09-10T18:46:29.742318+0000","last_fullsized":"2024-09-10T18:46:29.742318+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:46.800364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.319983+0000","last_change":"2024-09-10T18:45:24.267570+0000","last_active":"2024-09-10T18:46:30.319983+0000","last_peered":"2024-09-10T18:46:30.319983+0000","last_clean":"2024-09-10T18:46:30.319983+0000","last_became_active":"2024-09-10T18:45:24.267251+0000","last_became_peered":"2024-09-10T18:45:24.267251+0000","last_unstale":"2024-09-10T18:46:30.319983+0000","last_undegraded":"2024-09-10T18:46:30.319983+0000","last_fullsized":"2024-09-10T18:46:30.319983+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:41.357380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"56'1","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.792272+0000","last_change":"2024-09-10T18:45:17.966530+0000","last_active":"2024-09-10T18:46:29.792272+0000","last_peered":"2024-09-10T18:46:29.792272+0000","last_clean":"2024-09-10T18:46:29.792272+0000","last_became_active":"2024-09-10T18:45:17.966286+0000","last_became_peered":"2024-09-10T18:45:17.966286+0000","last_unstale":"2024-09-10T18:46:29.792272+0000","last_undegraded":"2024-09-10T18:46:29.792272+0000","last_fullsized":"2024-09-10T18:46:29.792272+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:41.758824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"63'15","reported_seq":64,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:30.567321+0000","last_change":"2024-09-10T18:46:23.864133+0000","last_active":"2024-09-10T18:46:30.567321+0000","last_peered":"2024-09-10T18:46:30.567321+0000","last_clean":"2024-09-10T18:46:30.567321+0000","last_became_active":"2024-09-10T18:45:20.062728+0000","last_became_peered":"2024-09-10T18:45:20.062728+0000","last_unstale":"2024-09-10T18:46:30.567321+0000","last_undegraded":"2024-09-10T18:46:30.567321+0000","last_fullsized":"2024-09-10T18:46:30.567321+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:46:23.864075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:23.864075+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:29.106766+0000","scrub_duration":0.0020859780000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.742537+0000","last_change":"2024-09-10T18:45:22.060202+0000","last_active":"2024-09-10T18:46:29.742537+0000","last_peered":"2024-09-10T18:46:29.742537+0000","last_clean":"2024-09-10T18:46:29.742537+0000","last_became_active":"2024-09-10T18:45:22.059794+0000","last_became_peered":"2024-09-10T18:45:22.059794+0000","last_unstale":"2024-09-10T18:46:29.742537+0000","last_undegraded":"2024-09-10T18:46:29.742537+0000","last_fullsized":"2024-09-10T18:46:29.742537+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:44.574409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.791325+0000","last_change":"2024-09-10T18:45:24.264775+0000","last_active":"2024-09-10T18:46:29.791325+0000","last_peered":"2024-09-10T18:46:29.791325+0000","last_clean":"2024-09-10T18:46:29.791325+0000","last_became_active":"2024-09-10T18:45:24.264427+0000","last_became_peered":"2024-09-10T18:45:24.264427+0000","last_unstale":"2024-09-10T18:46:29.791325+0000","last_undegraded":"2024-09-10T18:46:29.791325+0000","last_fullsized":"2024-09-10T18:46:29.791325+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:44.649431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.791165+0000","last_change":"2024-09-10T18:45:17.963530+0000","last_active":"2024-09-10T18:46:29.791165+0000","last_peered":"2024-09-10T18:46:29.791165+0000","last_clean":"2024-09-10T18:46:29.791165+0000","last_became_active":"2024-09-10T18:45:17.963257+0000","last_became_peered":"2024-09-10T18:45:17.963257+0000","last_unstale":"2024-09-10T18:46:29.791165+0000","last_undegraded":"2024-09-10T18:46:29.791165+0000","last_fullsized":"2024-09-10T18:46:29.791165+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:33.981037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"63'12","reported_seq":52,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T18:46:29.791203+0000","last_change":"2024-09-10T18:45:20.058693+0000","last_active":"2024-09-10T18:46:29.791203+0000","last_peered":"2024-09-10T18:46:29.791203+0000","last_clean":"2024-09-10T18:46:29.791203+0000","last_became_active":"2024-09-10T18:45:20.058391+0000","last_became_peered":"2024-09-10T18:45:20.058391+0000","last_unstale":"2024-09-10T18:46:29.791203+0000","last_undegraded":"2024-09-10T18:46:29.791203+0000","last_fullsized":"2024-09-10T18:46:29.791203+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:39.298321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-10T18:46:34.284014+0000","last_change":"2024-09-10T18:46:34.283979+0000","last_active":"2024-09-10T18:46:34.284014+0000","last_peered":"2024-09-10T18:46:34.284014+0000","last_clean":"2024-09-10T18:46:34.284014+0000","last_became_active":"2024-09-10T18:45:22.073250+0000","last_became_peered":"2024-09-10T18:45:22.073250+0000","last_unstale":"2024-09-10T18:46:34.284014+0000","last_undegraded":"2024-09-10T18:46:34.284014+0000","last_fullsized":"2024-09-10T18:46:34.284014+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:34.283901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:34.283901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:34.686745+0000","scrub_duration":0.002360334,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":395,"ondisk_log_size":395,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":70,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299413,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298724,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446428,"statfs":{"total":95995035648,"available":95689142272,"internally_reserved":0,"allocated":1404928,"data_stored":921391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.66600000000000004}]},{"osd":1,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.68700000000000006}]},{"osd":2,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.64900000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.622},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.55100000000000005}]},{"osd":4,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.379},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.39600000000000002}]},{"osd":5,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.36299999999999999}]},{"osd":6,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.434},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.45600000000000002}]}]},{"osd":6,"up_from":43,"seq":184683593754,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298712,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446440,"statfs":{"total":95995035648,"available":95689154560,"internally_reserved":0,"allocated":1392640,"data_stored":921467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:45:41 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.53300000000000003}]},{"osd":1,"last update":"Tue Sep 10 18:45:41 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.503}]},{"osd":2,"last update":"Tue Sep 10 18:45:41 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.51700000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:45:41 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.60499999999999998}]},{"osd":4,"last update":"Tue Sep 10 18:45:41 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.436}]},{"osd":5,"last update":"Tue Sep 10 18:45:41 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.36299999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.45600000000000002}]}]},{"osd":5,"up_from":37,"seq":158913789982,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298120,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447032,"statfs":{"total":95995035648,"available":95689760768,"internally_reserved":0,"allocated":786432,"data_stored":329602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.52200000000000002,"15min":0.52600000000000002},"min":{"1min":0.30299999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70099999999999996,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.58299999999999996,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.38200000000000001}]},{"osd":1,"last update":"Tue Sep 10 18:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.55200000000000005,"15min":0.56000000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.69699999999999995,"5min":0.87,"15min":0.87},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.64200000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.36399999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.51800000000000002,"15min":0.52000000000000002},"min":{"1min":0.33300000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.71299999999999997,"5min":0.755,"15min":0.755},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53600000000000003,"15min":0.54100000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79600000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.39000000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52500000000000002,"15min":0.52700000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76100000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.375},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.54200000000000004,"15min":0.54600000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80100000000000005,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.41799999999999998}]},{"osd":4,"last update":"Tue Sep 10 18:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.64500000000000002,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.40500000000000003,"15min":0.40300000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.65700000000000003,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.32000000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:45:42 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.40300000000000002}]},{"osd":7,"last update":"Tue Sep 10 18:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.34000000000000002}]}]},{"osd":4,"up_from":30,"seq":128849018916,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298144,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447008,"statfs":{"total":95995035648,"available":95689736192,"internally_reserved":0,"allocated":811008,"data_stored":330559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.36399999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.433},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.54000000000000004,"15min":0.54200000000000004},"min":{"1min":0.34399999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.44800000000000001}]},{"osd":1,"last update":"Tue Sep 10 18:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.57999999999999996,"15min":0.57599999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.609,"5min":2.609,"15min":2.609},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.54700000000000004,"15min":0.53200000000000003},"min":{"1min":0.39700000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.589,"5min":2.589,"15min":2.589},"last":0.56000000000000005}]},{"osd":2,"last update":"Tue Sep 10 18:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.55700000000000005,"15min":0.54400000000000004},"min":{"1min":0.40500000000000003,"5min":0.379,"15min":0.379},"max":{"1min":2.6030000000000002,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.54800000000000004,"15min":0.55400000000000005},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64000000000000001,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.52700000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52800000000000002,"15min":0.53000000000000003},"min":{"1min":0.36699999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68300000000000005,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.57599999999999996}]},{"osd":5,"last update":"Tue Sep 10 18:46:22 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.35899999999999999,"15min":0.34999999999999998},"min":{"1min":0.26800000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.44,"5min":0.371,"15min":0.35899999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.58399999999999996}]},{"osd":6,"last update":"Tue Sep 10 18:45:42 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.48299999999999998}]},{"osd":7,"last update":"Tue Sep 10 18:46:08 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.54400000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182441,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298156,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446996,"statfs":{"total":95995035648,"available":95689723904,"internally_reserved":0,"allocated":823296,"data_stored":330338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:35 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.378,"15min":0.38400000000000001},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.40100000000000002,"15min":0.40300000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.28699999999999998}]},{"osd":1,"last update":"Tue Sep 10 18:46:35 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.40400000000000003,"15min":0.40899999999999997},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40899999999999997,"15min":0.41499999999999998},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.63800000000000001,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.41799999999999998}]},{"osd":2,"last update":"Tue Sep 10 18:46:35 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.43099999999999999,"15min":0.438},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.41999999999999998,"15min":0.433},"min":{"1min":0.27200000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.53600000000000003,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.44700000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54000000000000004,"15min":0.54300000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.67200000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.56599999999999995,"15min":0.57199999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.71699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.54000000000000004}]},{"osd":5,"last update":"Tue Sep 10 18:46:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39700000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.70299999999999996,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.435},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.41399999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80800000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.52700000000000002}]},{"osd":6,"last update":"Tue Sep 10 18:45:41 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.51200000000000001}]},{"osd":7,"last update":"Tue Sep 10 18:46:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.495}]}]},{"osd":2,"up_from":18,"seq":77309411373,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298124,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447028,"statfs":{"total":95995035648,"available":95689756672,"internally_reserved":0,"allocated":790528,"data_stored":330037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.376,"15min":0.35899999999999999},"min":{"1min":0.246,"5min":0.223,"15min":0.223},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.371},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.38300000000000001,"15min":0.371},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.26000000000000001}]},{"osd":1,"last update":"Tue Sep 10 18:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.375,"15min":0.36699999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.42499999999999999,"15min":0.41299999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.40000000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:46:36 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46800000000000003,"15min":0.46500000000000002},"min":{"1min":0.30099999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.42899999999999999,"15min":0.43099999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.32300000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:45:59 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54900000000000004,"15min":0.54800000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.55700000000000005}]},{"osd":5,"last update":"Tue Sep 10 18:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56100000000000005,"15min":0.55800000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.62},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.42899999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.53400000000000003}]},{"osd":6,"last update":"Tue Sep 10 18:45:45 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.59699999999999998}]},{"osd":7,"last update":"Tue Sep 10 18:46:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.58299999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298168,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446984,"statfs":{"total":95995035648,"available":95689711616,"internally_reserved":0,"allocated":835584,"data_stored":330833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.33600000000000002,"15min":0.318},"min":{"1min":0.24199999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.245},{"interface":"front","average":{"1min":0.434,"5min":0.35999999999999999,"15min":0.33400000000000002},"min":{"1min":0.27500000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.53700000000000003}]},{"osd":2,"last update":"Tue Sep 10 18:46:11 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.372,"15min":0.36099999999999999},"min":{"1min":0.27900000000000003,"5min":0.248,"15min":0.248},"max":{"1min":0.47199999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.39000000000000001,"15min":0.38100000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.49099999999999999,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.47399999999999998}]},{"osd":3,"last update":"Tue Sep 10 18:46:34 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.45400000000000001,"15min":0.45800000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.42299999999999999,"15min":0.42199999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.55400000000000005,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.32500000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:45:56 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.56299999999999994,"15min":0.55600000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.54100000000000004,"15min":0.53300000000000003},"min":{"1min":0.41099999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.59499999999999997}]},{"osd":5,"last update":"Tue Sep 10 18:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60299999999999998,"15min":0.60399999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63700000000000001,"15min":0.64100000000000001},"min":{"1min":0.44600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93300000000000005,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.64300000000000002}]},{"osd":6,"last update":"Tue Sep 10 18:45:43 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.78400000000000003}]},{"osd":7,"last update":"Tue Sep 10 18:46:11 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.81200000000000006}]}]},{"osd":0,"up_from":8,"seq":34359738423,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298716,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446436,"statfs":{"total":95995035648,"available":95689150464,"internally_reserved":0,"allocated":1396736,"data_stored":920201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 18:45:50 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.32100000000000001,"15min":0.30199999999999999},"min":{"1min":0.224,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.219},{"interface":"front","average":{"1min":0.371,"5min":0.33100000000000002,"15min":0.313},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.307}]},{"osd":2,"last update":"Tue Sep 10 18:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.40300000000000002,"15min":0.38900000000000001},"min":{"1min":0.30299999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.376,"15min":0.36199999999999999},"min":{"1min":0.26600000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.32300000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:46:31 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.39500000000000002,"15min":0.39800000000000002},"min":{"1min":0.26100000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.66100000000000003,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.41199999999999998,"15min":0.41299999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.36499999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.56999999999999995,"15min":0.53600000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":4.6189999999999998,"5min":4.6189999999999998,"15min":4.6189999999999998},"last":0.61099999999999999}]},{"osd":5,"last update":"Tue Sep 10 18:46:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.42999999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.495},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.58699999999999997,"15min":0.55800000000000005},"min":{"1min":0.42699999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":4.6429999999999998,"5min":4.6429999999999998,"15min":4.6429999999999998},"last":0.51900000000000002}]},{"osd":6,"last update":"Tue Sep 10 18:45:40 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":4.6630000000000003,"5min":4.6630000000000003,"15min":4.6630000000000003},"last":0.55500000000000005}]},{"osd":7,"last update":"Tue Sep 10 18:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.57799999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T18:46:46.793 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T18:46:46.793 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T18:46:46.793 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-10T18:46:46.794 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-10T18:46:47.066 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:46:47.402 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:47 smithi018 ceph-mon[40196]: 2.13 scrub starts 2024-09-10T18:46:47.402 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:47 smithi018 ceph-mon[40196]: 2.13 scrub ok 2024-09-10T18:46:47.402 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:47 smithi018 ceph-mon[40196]: from='client.24578 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:46:47.402 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:47 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:47.402 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:47 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:46:47.402 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:47 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:46:47.402 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:47 smithi018 ceph-mon[40196]: osdmap e79: 8 total, 8 up, 8 in 2024-09-10T18:46:47.402 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:47 smithi018 ceph-mon[40196]: 4.1f scrub starts 2024-09-10T18:46:47.402 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:47 smithi018 ceph-mon[40196]: 4.1f scrub ok 2024-09-10T18:46:47.402 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:47 smithi018 ceph-mon[40196]: 2.1d scrub starts 2024-09-10T18:46:47.403 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:47 smithi018 ceph-mon[40196]: 2.1d scrub ok 2024-09-10T18:46:47.403 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:47 smithi018 ceph-mon[40196]: 2.1 scrub starts 2024-09-10T18:46:47.403 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:47 smithi018 ceph-mon[40196]: 2.1 scrub ok 2024-09-10T18:46:47.403 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:47 smithi018 ceph-mon[40196]: 4.10 scrub starts 2024-09-10T18:46:47.403 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:47 smithi018 ceph-mon[40196]: 4.10 scrub ok 2024-09-10T18:46:47.403 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:47 smithi018 ceph-mon[29336]: 2.13 scrub starts 2024-09-10T18:46:47.403 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:47 smithi018 ceph-mon[29336]: 2.13 scrub ok 2024-09-10T18:46:47.403 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:47 smithi018 ceph-mon[29336]: from='client.24578 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:46:47.403 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:47 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:47.403 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:47 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:46:47.403 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:47 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:46:47.403 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:47 smithi018 ceph-mon[29336]: osdmap e79: 8 total, 8 up, 8 in 2024-09-10T18:46:47.404 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:47 smithi018 ceph-mon[29336]: 4.1f scrub starts 2024-09-10T18:46:47.404 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:47 smithi018 ceph-mon[29336]: 4.1f scrub ok 2024-09-10T18:46:47.404 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:47 smithi018 ceph-mon[29336]: 2.1d scrub starts 2024-09-10T18:46:47.404 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:47 smithi018 ceph-mon[29336]: 2.1d scrub ok 2024-09-10T18:46:47.404 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:47 smithi018 ceph-mon[29336]: 2.1 scrub starts 2024-09-10T18:46:47.404 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:47 smithi018 ceph-mon[29336]: 2.1 scrub ok 2024-09-10T18:46:47.404 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:47 smithi018 ceph-mon[29336]: 4.10 scrub starts 2024-09-10T18:46:47.404 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:47 smithi018 ceph-mon[29336]: 4.10 scrub ok 2024-09-10T18:46:47.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:47 smithi103 ceph-mon[33163]: 2.13 scrub starts 2024-09-10T18:46:47.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:47 smithi103 ceph-mon[33163]: 2.13 scrub ok 2024-09-10T18:46:47.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:47 smithi103 ceph-mon[33163]: from='client.24578 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:46:47.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:47 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:47.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:47 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:46:47.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:47 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:46:47.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:47 smithi103 ceph-mon[33163]: osdmap e79: 8 total, 8 up, 8 in 2024-09-10T18:46:47.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:47 smithi103 ceph-mon[33163]: 4.1f scrub starts 2024-09-10T18:46:47.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:47 smithi103 ceph-mon[33163]: 4.1f scrub ok 2024-09-10T18:46:47.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:47 smithi103 ceph-mon[33163]: 2.1d scrub starts 2024-09-10T18:46:47.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:47 smithi103 ceph-mon[33163]: 2.1d scrub ok 2024-09-10T18:46:47.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:47 smithi103 ceph-mon[33163]: 2.1 scrub starts 2024-09-10T18:46:47.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:47 smithi103 ceph-mon[33163]: 2.1 scrub ok 2024-09-10T18:46:47.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:47 smithi103 ceph-mon[33163]: 4.10 scrub starts 2024-09-10T18:46:47.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:47 smithi103 ceph-mon[33163]: 4.10 scrub ok 2024-09-10T18:46:48.429 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:48 smithi018 ceph-mon[29336]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:46:48.430 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:48 smithi018 ceph-mon[29336]: 2.b scrub starts 2024-09-10T18:46:48.430 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:48 smithi018 ceph-mon[29336]: 2.b scrub ok 2024-09-10T18:46:48.430 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:48 smithi018 ceph-mon[29336]: 2.1b scrub starts 2024-09-10T18:46:48.430 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:48 smithi018 ceph-mon[29336]: 2.1b scrub ok 2024-09-10T18:46:48.430 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:48 smithi018 ceph-mon[40196]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:46:48.430 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:48 smithi018 ceph-mon[40196]: 2.b scrub starts 2024-09-10T18:46:48.430 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:48 smithi018 ceph-mon[40196]: 2.b scrub ok 2024-09-10T18:46:48.430 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:48 smithi018 ceph-mon[40196]: 2.1b scrub starts 2024-09-10T18:46:48.430 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:48 smithi018 ceph-mon[40196]: 2.1b scrub ok 2024-09-10T18:46:48.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:48 smithi103 ceph-mon[33163]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:46:48.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:48 smithi103 ceph-mon[33163]: 2.b scrub starts 2024-09-10T18:46:48.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:48 smithi103 ceph-mon[33163]: 2.b scrub ok 2024-09-10T18:46:48.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:48 smithi103 ceph-mon[33163]: 2.1b scrub starts 2024-09-10T18:46:48.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:48 smithi103 ceph-mon[33163]: 2.1b scrub ok 2024-09-10T18:46:48.871 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:46:48.871 INFO:teuthology.orchestra.run.smithi018.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T18:46:49.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:49 smithi018 ceph-mon[29336]: 6.9 deep-scrub starts 2024-09-10T18:46:49.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:49 smithi018 ceph-mon[29336]: 6.9 deep-scrub ok 2024-09-10T18:46:49.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:49 smithi018 ceph-mon[29336]: 5.f deep-scrub starts 2024-09-10T18:46:49.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:49 smithi018 ceph-mon[29336]: 5.f deep-scrub ok 2024-09-10T18:46:49.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:49 smithi018 ceph-mon[29336]: 2.11 scrub starts 2024-09-10T18:46:49.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:49 smithi018 ceph-mon[29336]: 2.11 scrub ok 2024-09-10T18:46:49.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:49 smithi018 ceph-mon[29336]: 4.11 deep-scrub starts 2024-09-10T18:46:49.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:49 smithi018 ceph-mon[29336]: 4.11 deep-scrub ok 2024-09-10T18:46:49.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:49 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2825191244' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T18:46:49.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:49 smithi018 ceph-mon[40196]: 6.9 deep-scrub starts 2024-09-10T18:46:49.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:49 smithi018 ceph-mon[40196]: 6.9 deep-scrub ok 2024-09-10T18:46:49.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:49 smithi018 ceph-mon[40196]: 5.f deep-scrub starts 2024-09-10T18:46:49.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:49 smithi018 ceph-mon[40196]: 5.f deep-scrub ok 2024-09-10T18:46:49.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:49 smithi018 ceph-mon[40196]: 2.11 scrub starts 2024-09-10T18:46:49.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:49 smithi018 ceph-mon[40196]: 2.11 scrub ok 2024-09-10T18:46:49.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:49 smithi018 ceph-mon[40196]: 4.11 deep-scrub starts 2024-09-10T18:46:49.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:49 smithi018 ceph-mon[40196]: 4.11 deep-scrub ok 2024-09-10T18:46:49.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:49 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2825191244' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T18:46:49.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:49 smithi103 ceph-mon[33163]: 6.9 deep-scrub starts 2024-09-10T18:46:49.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:49 smithi103 ceph-mon[33163]: 6.9 deep-scrub ok 2024-09-10T18:46:49.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:49 smithi103 ceph-mon[33163]: 5.f deep-scrub starts 2024-09-10T18:46:49.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:49 smithi103 ceph-mon[33163]: 5.f deep-scrub ok 2024-09-10T18:46:49.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:49 smithi103 ceph-mon[33163]: 2.11 scrub starts 2024-09-10T18:46:49.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:49 smithi103 ceph-mon[33163]: 2.11 scrub ok 2024-09-10T18:46:49.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:49 smithi103 ceph-mon[33163]: 4.11 deep-scrub starts 2024-09-10T18:46:49.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:49 smithi103 ceph-mon[33163]: 4.11 deep-scrub ok 2024-09-10T18:46:49.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:49 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2825191244' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T18:46:49.614 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-10T18:46:49.615 INFO:tasks.cephadm:Setup complete, yielding 2024-09-10T18:46:49.615 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-10T18:46:49.635 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-10T18:46:49.635 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-guits-quincy-2024-09-09-1810-quincy'}, 'ceph': {'conf': {'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'ms bind msgr1': False, 'ms bind msgr2': True, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', '\\(POOL_APP_NOT_ENABLED\\)'], 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'cephadm': {'cephadm_mode': 'root'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'workunit': {'branch': 'quincy-orch-centos-9-teuth', 'sha1': 'fad5356a94e6044ce1062d748daa5a0f2ed53a7d'}} 2024-09-10T18:46:49.635 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-10T18:46:49.635 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-10T18:46:49.892 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:46:50.275 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:50 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:46:50.275 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:50 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:46:50.275 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:50 smithi018 ceph-mon[40196]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:46:50.275 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:50 smithi018 ceph-mon[40196]: 2.2 scrub starts 2024-09-10T18:46:50.275 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:50 smithi018 ceph-mon[40196]: 2.2 scrub ok 2024-09-10T18:46:50.275 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:50 smithi018 ceph-mon[40196]: 6.4 scrub starts 2024-09-10T18:46:50.275 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:50 smithi018 ceph-mon[40196]: 6.4 scrub ok 2024-09-10T18:46:50.275 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:50 smithi018 ceph-mon[40196]: osdmap e80: 8 total, 8 up, 8 in 2024-09-10T18:46:50.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:50 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:46:50.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:50 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:46:50.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:50 smithi018 ceph-mon[29336]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:46:50.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:50 smithi018 ceph-mon[29336]: 2.2 scrub starts 2024-09-10T18:46:50.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:50 smithi018 ceph-mon[29336]: 2.2 scrub ok 2024-09-10T18:46:50.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:50 smithi018 ceph-mon[29336]: 6.4 scrub starts 2024-09-10T18:46:50.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:50 smithi018 ceph-mon[29336]: 6.4 scrub ok 2024-09-10T18:46:50.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:50 smithi018 ceph-mon[29336]: osdmap e80: 8 total, 8 up, 8 in 2024-09-10T18:46:50.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:50 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:46:50.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:50 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:46:50.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:50 smithi103 ceph-mon[33163]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:46:50.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:50 smithi103 ceph-mon[33163]: 2.2 scrub starts 2024-09-10T18:46:50.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:50 smithi103 ceph-mon[33163]: 2.2 scrub ok 2024-09-10T18:46:50.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:50 smithi103 ceph-mon[33163]: 6.4 scrub starts 2024-09-10T18:46:50.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:50 smithi103 ceph-mon[33163]: 6.4 scrub ok 2024-09-10T18:46:50.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:50 smithi103 ceph-mon[33163]: osdmap e80: 8 total, 8 up, 8 in 2024-09-10T18:46:50.523 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:46:50 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:46:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:46:51.407 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:51 smithi018 ceph-mon[40196]: 4.1e scrub starts 2024-09-10T18:46:51.407 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:51 smithi018 ceph-mon[40196]: 4.1e scrub ok 2024-09-10T18:46:51.407 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:51 smithi018 ceph-mon[40196]: 2.12 scrub starts 2024-09-10T18:46:51.407 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:51 smithi018 ceph-mon[40196]: 2.12 scrub ok 2024-09-10T18:46:51.407 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:51 smithi018 ceph-mon[40196]: 2.4 scrub starts 2024-09-10T18:46:51.407 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:51 smithi018 ceph-mon[40196]: 2.4 scrub ok 2024-09-10T18:46:51.407 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:51 smithi018 ceph-mon[40196]: 2.1c scrub starts 2024-09-10T18:46:51.407 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:51 smithi018 ceph-mon[40196]: 2.1c scrub ok 2024-09-10T18:46:51.407 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:51 smithi018 ceph-mon[40196]: 4.14 scrub starts 2024-09-10T18:46:51.408 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:51 smithi018 ceph-mon[40196]: 4.14 scrub ok 2024-09-10T18:46:51.408 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:51 smithi018 ceph-mon[29336]: 4.1e scrub starts 2024-09-10T18:46:51.408 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:51 smithi018 ceph-mon[29336]: 4.1e scrub ok 2024-09-10T18:46:51.408 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:51 smithi018 ceph-mon[29336]: 2.12 scrub starts 2024-09-10T18:46:51.408 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:51 smithi018 ceph-mon[29336]: 2.12 scrub ok 2024-09-10T18:46:51.408 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:51 smithi018 ceph-mon[29336]: 2.4 scrub starts 2024-09-10T18:46:51.408 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:51 smithi018 ceph-mon[29336]: 2.4 scrub ok 2024-09-10T18:46:51.408 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:51 smithi018 ceph-mon[29336]: 2.1c scrub starts 2024-09-10T18:46:51.408 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:51 smithi018 ceph-mon[29336]: 2.1c scrub ok 2024-09-10T18:46:51.408 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:51 smithi018 ceph-mon[29336]: 4.14 scrub starts 2024-09-10T18:46:51.409 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:51 smithi018 ceph-mon[29336]: 4.14 scrub ok 2024-09-10T18:46:51.445 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:51 smithi103 ceph-mon[33163]: 4.1e scrub starts 2024-09-10T18:46:51.445 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:51 smithi103 ceph-mon[33163]: 4.1e scrub ok 2024-09-10T18:46:51.445 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:51 smithi103 ceph-mon[33163]: 2.12 scrub starts 2024-09-10T18:46:51.445 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:51 smithi103 ceph-mon[33163]: 2.12 scrub ok 2024-09-10T18:46:51.445 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:51 smithi103 ceph-mon[33163]: 2.4 scrub starts 2024-09-10T18:46:51.445 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:51 smithi103 ceph-mon[33163]: 2.4 scrub ok 2024-09-10T18:46:51.445 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:51 smithi103 ceph-mon[33163]: 2.1c scrub starts 2024-09-10T18:46:51.445 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:51 smithi103 ceph-mon[33163]: 2.1c scrub ok 2024-09-10T18:46:51.445 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:51 smithi103 ceph-mon[33163]: 4.14 scrub starts 2024-09-10T18:46:51.445 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:51 smithi103 ceph-mon[33163]: 4.14 scrub ok 2024-09-10T18:46:51.694 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:46:51 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:18:46:51] ENGINE Bus STOPPING 2024-09-10T18:46:52.188 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:46:51 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:18:46:51] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:46:52.188 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:46:51 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:18:46:51] ENGINE Bus STOPPED 2024-09-10T18:46:52.188 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:46:51 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:18:46:51] ENGINE Bus STARTING 2024-09-10T18:46:52.188 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:46:52 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:18:46:52] ENGINE Serving on http://:::9283 2024-09-10T18:46:52.188 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:46:52 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:18:46:52] ENGINE Bus STARTED 2024-09-10T18:46:52.338 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-10T18:46:52.338 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T18:46:52.338 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T18:46:52.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:52 smithi018 ceph-mon[29336]: 6.10 scrub starts 2024-09-10T18:46:52.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:52 smithi018 ceph-mon[29336]: 6.10 scrub ok 2024-09-10T18:46:52.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:52 smithi018 ceph-mon[29336]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:46:52.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:52 smithi018 ceph-mon[29336]: 4.8 scrub starts 2024-09-10T18:46:52.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:52 smithi018 ceph-mon[29336]: 4.8 scrub ok 2024-09-10T18:46:52.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:52 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3278639088' entity='client.admin' 2024-09-10T18:46:52.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:52 smithi018 ceph-mon[40196]: 6.10 scrub starts 2024-09-10T18:46:52.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:52 smithi018 ceph-mon[40196]: 6.10 scrub ok 2024-09-10T18:46:52.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:52 smithi018 ceph-mon[40196]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:46:52.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:52 smithi018 ceph-mon[40196]: 4.8 scrub starts 2024-09-10T18:46:52.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:52 smithi018 ceph-mon[40196]: 4.8 scrub ok 2024-09-10T18:46:52.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:52 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3278639088' entity='client.admin' 2024-09-10T18:46:52.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:52 smithi103 ceph-mon[33163]: 6.10 scrub starts 2024-09-10T18:46:52.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:52 smithi103 ceph-mon[33163]: 6.10 scrub ok 2024-09-10T18:46:52.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:52 smithi103 ceph-mon[33163]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:46:52.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:52 smithi103 ceph-mon[33163]: 4.8 scrub starts 2024-09-10T18:46:52.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:52 smithi103 ceph-mon[33163]: 4.8 scrub ok 2024-09-10T18:46:52.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:52 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3278639088' entity='client.admin' 2024-09-10T18:46:52.596 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:46:53.327 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:53 smithi018 ceph-mon[29336]: 4.b scrub starts 2024-09-10T18:46:53.327 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:53 smithi018 ceph-mon[29336]: 4.b scrub ok 2024-09-10T18:46:53.327 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:53 smithi018 ceph-mon[29336]: 5.2 scrub starts 2024-09-10T18:46:53.327 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:53 smithi018 ceph-mon[29336]: 5.2 scrub ok 2024-09-10T18:46:53.327 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:53 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:46:53.327 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:53 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:46:53.327 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:53 smithi018 ceph-mon[29336]: 6.15 scrub starts 2024-09-10T18:46:53.328 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:53 smithi018 ceph-mon[29336]: 6.15 scrub ok 2024-09-10T18:46:53.328 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:53 smithi018 ceph-mon[29336]: 4.1d scrub starts 2024-09-10T18:46:53.328 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:53 smithi018 ceph-mon[29336]: 4.1d scrub ok 2024-09-10T18:46:53.328 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:53 smithi018 ceph-mon[29336]: 6.2 deep-scrub starts 2024-09-10T18:46:53.328 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:53 smithi018 ceph-mon[29336]: 6.2 deep-scrub ok 2024-09-10T18:46:53.328 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:53 smithi018 ceph-mon[40196]: 4.b scrub starts 2024-09-10T18:46:53.328 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:53 smithi018 ceph-mon[40196]: 4.b scrub ok 2024-09-10T18:46:53.328 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:53 smithi018 ceph-mon[40196]: 5.2 scrub starts 2024-09-10T18:46:53.328 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:53 smithi018 ceph-mon[40196]: 5.2 scrub ok 2024-09-10T18:46:53.328 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:53 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:46:53.328 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:53 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:46:53.329 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:53 smithi018 ceph-mon[40196]: 6.15 scrub starts 2024-09-10T18:46:53.329 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:53 smithi018 ceph-mon[40196]: 6.15 scrub ok 2024-09-10T18:46:53.329 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:53 smithi018 ceph-mon[40196]: 4.1d scrub starts 2024-09-10T18:46:53.329 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:53 smithi018 ceph-mon[40196]: 4.1d scrub ok 2024-09-10T18:46:53.329 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:53 smithi018 ceph-mon[40196]: 6.2 deep-scrub starts 2024-09-10T18:46:53.329 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:53 smithi018 ceph-mon[40196]: 6.2 deep-scrub ok 2024-09-10T18:46:53.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:53 smithi103 ceph-mon[33163]: 4.b scrub starts 2024-09-10T18:46:53.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:53 smithi103 ceph-mon[33163]: 4.b scrub ok 2024-09-10T18:46:53.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:53 smithi103 ceph-mon[33163]: 5.2 scrub starts 2024-09-10T18:46:53.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:53 smithi103 ceph-mon[33163]: 5.2 scrub ok 2024-09-10T18:46:53.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:53 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:46:53.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:53 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:46:53.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:53 smithi103 ceph-mon[33163]: 6.15 scrub starts 2024-09-10T18:46:53.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:53 smithi103 ceph-mon[33163]: 6.15 scrub ok 2024-09-10T18:46:53.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:53 smithi103 ceph-mon[33163]: 4.1d scrub starts 2024-09-10T18:46:53.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:53 smithi103 ceph-mon[33163]: 4.1d scrub ok 2024-09-10T18:46:53.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:53 smithi103 ceph-mon[33163]: 6.2 deep-scrub starts 2024-09-10T18:46:53.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:53 smithi103 ceph-mon[33163]: 6.2 deep-scrub ok 2024-09-10T18:46:54.410 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:46:54.411 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:54 smithi018 ceph-mon[29336]: 4.6 scrub starts 2024-09-10T18:46:54.411 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:54 smithi018 ceph-mon[29336]: 4.6 scrub ok 2024-09-10T18:46:54.411 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:54 smithi018 ceph-mon[29336]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:46:54.412 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:54 smithi018 ceph-mon[29336]: osdmap e81: 8 total, 8 up, 8 in 2024-09-10T18:46:54.412 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:54 smithi018 ceph-mon[29336]: 6.12 scrub starts 2024-09-10T18:46:54.412 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:54 smithi018 ceph-mon[29336]: 6.12 scrub ok 2024-09-10T18:46:54.412 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:54 smithi018 ceph-mon[29336]: 2.c scrub starts 2024-09-10T18:46:54.412 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:54 smithi018 ceph-mon[29336]: 2.c scrub ok 2024-09-10T18:46:54.413 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:54 smithi018 ceph-mon[40196]: 4.6 scrub starts 2024-09-10T18:46:54.413 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:54 smithi018 ceph-mon[40196]: 4.6 scrub ok 2024-09-10T18:46:54.413 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:54 smithi018 ceph-mon[40196]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:46:54.413 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:54 smithi018 ceph-mon[40196]: osdmap e81: 8 total, 8 up, 8 in 2024-09-10T18:46:54.413 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:54 smithi018 ceph-mon[40196]: 6.12 scrub starts 2024-09-10T18:46:54.413 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:54 smithi018 ceph-mon[40196]: 6.12 scrub ok 2024-09-10T18:46:54.413 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:54 smithi018 ceph-mon[40196]: 2.c scrub starts 2024-09-10T18:46:54.413 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:54 smithi018 ceph-mon[40196]: 2.c scrub ok 2024-09-10T18:46:54.418 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-09-10T18:46:54.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-mon[33163]: 4.6 scrub starts 2024-09-10T18:46:54.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-mon[33163]: 4.6 scrub ok 2024-09-10T18:46:54.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-mon[33163]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:46:54.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-mon[33163]: osdmap e81: 8 total, 8 up, 8 in 2024-09-10T18:46:54.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-mon[33163]: 6.12 scrub starts 2024-09-10T18:46:54.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-mon[33163]: 6.12 scrub ok 2024-09-10T18:46:54.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-mon[33163]: 2.c scrub starts 2024-09-10T18:46:54.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-mon[33163]: 2.c scrub ok 2024-09-10T18:46:54.725 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 systemd[1]: Started Ceph grafana.a for ba327f6c-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:46:54.977 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=settings t=2024-09-10T18:46:54.724516327Z level=info msg="Starting Grafana" version=9.4.7 commit=4add91f03d branch=HEAD compiled=2023-03-16T23:56:52Z 2024-09-10T18:46:54.977 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=settings t=2024-09-10T18:46:54.724685766Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-09-10T18:46:54.977 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=settings t=2024-09-10T18:46:54.724721125Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-09-10T18:46:54.977 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=settings t=2024-09-10T18:46:54.724728221Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-09-10T18:46:54.977 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=settings t=2024-09-10T18:46:54.724734062Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-09-10T18:46:54.977 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=settings t=2024-09-10T18:46:54.724739983Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-09-10T18:46:54.977 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=settings t=2024-09-10T18:46:54.724745664Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-09-10T18:46:54.977 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=settings t=2024-09-10T18:46:54.7247518Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-09-10T18:46:54.978 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=settings t=2024-09-10T18:46:54.72476021Z level=info msg="Path Home" path=/usr/share/grafana 2024-09-10T18:46:54.978 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=settings t=2024-09-10T18:46:54.724769911Z level=info msg="Path Data" path=/var/lib/grafana 2024-09-10T18:46:54.978 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=settings t=2024-09-10T18:46:54.724779411Z level=info msg="Path Logs" path=/var/log/grafana 2024-09-10T18:46:54.978 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=settings t=2024-09-10T18:46:54.724788969Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-09-10T18:46:54.978 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=settings t=2024-09-10T18:46:54.724799252Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-09-10T18:46:54.978 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=settings t=2024-09-10T18:46:54.724808823Z level=info msg="App mode production" 2024-09-10T18:46:54.978 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=sqlstore t=2024-09-10T18:46:54.724879058Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-09-10T18:46:54.978 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=sqlstore t=2024-09-10T18:46:54.724898321Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-09-10T18:46:54.978 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.725750826Z level=info msg="Starting DB migrations" 2024-09-10T18:46:54.978 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.725829554Z level=info msg="Executing migration" id="create migration_log table" 2024-09-10T18:46:54.978 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.726357471Z level=info msg="Executing migration" id="create user table" 2024-09-10T18:46:54.980 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.726799255Z level=info msg="Executing migration" id="add unique index user.login" 2024-09-10T18:46:54.980 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.72726992Z level=info msg="Executing migration" id="add unique index user.email" 2024-09-10T18:46:54.980 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.727701219Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-09-10T18:46:54.980 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.728360418Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-09-10T18:46:54.980 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.728783718Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-09-10T18:46:54.980 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.729511124Z level=info msg="Executing migration" id="create user table v2" 2024-09-10T18:46:54.980 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.729877638Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-09-10T18:46:54.980 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.730288065Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-09-10T18:46:54.981 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.730712985Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-09-10T18:46:54.981 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.731089149Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-09-10T18:46:54.981 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.73386762Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-09-10T18:46:54.981 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.736480154Z level=info msg="Executing migration" id="Update user table charset" 2024-09-10T18:46:54.981 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.739145581Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-09-10T18:46:54.981 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.742024542Z level=info msg="Executing migration" id="Add missing user data" 2024-09-10T18:46:54.981 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.744554692Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-09-10T18:46:54.981 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.74740037Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-09-10T18:46:54.981 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.750004148Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-09-10T18:46:54.981 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.752527909Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-09-10T18:46:54.981 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.755169492Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-09-10T18:46:54.981 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.757891276Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-09-10T18:46:54.982 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.760611994Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-09-10T18:46:54.982 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.763278926Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-09-10T18:46:54.982 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.765888279Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-09-10T18:46:54.982 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.766270219Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-09-10T18:46:54.982 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.769047893Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-09-10T18:46:54.982 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.76945683Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-09-10T18:46:54.982 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.771884822Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-09-10T18:46:54.982 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.772458917Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-09-10T18:46:54.982 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.772802124Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-09-10T18:46:54.982 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.773446628Z level=info msg="Executing migration" id="create temp_user v2" 2024-09-10T18:46:54.982 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.773832152Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-09-10T18:46:54.982 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.774226904Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-09-10T18:46:54.982 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.7746638Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-09-10T18:46:54.982 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.77505768Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-09-10T18:46:54.983 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.775506542Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-09-10T18:46:54.983 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.775803834Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-09-10T18:46:54.983 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.776152804Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-09-10T18:46:54.983 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.776657405Z level=info msg="Executing migration" id="create star table" 2024-09-10T18:46:54.983 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.777046748Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-09-10T18:46:54.983 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.777506423Z level=info msg="Executing migration" id="create org table v1" 2024-09-10T18:46:54.983 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.778355943Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-09-10T18:46:54.983 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.779210916Z level=info msg="Executing migration" id="create org_user table v1" 2024-09-10T18:46:54.983 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.780192371Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-09-10T18:46:54.983 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.780879403Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-09-10T18:46:54.983 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.781867731Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-09-10T18:46:54.983 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.782595232Z level=info msg="Executing migration" id="Update org table charset" 2024-09-10T18:46:54.983 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.783415103Z level=info msg="Executing migration" id="Update org_user table charset" 2024-09-10T18:46:54.983 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.783728817Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-09-10T18:46:54.984 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.78436259Z level=info msg="Executing migration" id="create dashboard table" 2024-09-10T18:46:54.984 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.785380404Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-09-10T18:46:54.984 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.786070654Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-09-10T18:46:54.984 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.787022661Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-09-10T18:46:54.984 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.787711022Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-09-10T18:46:54.984 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.788693711Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-09-10T18:46:54.984 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.789463723Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-09-10T18:46:54.984 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.790359523Z level=info msg="Executing migration" id="create dashboard v2" 2024-09-10T18:46:54.984 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.791221669Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-09-10T18:46:54.984 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.79218839Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-09-10T18:46:54.984 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.792936788Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-09-10T18:46:54.984 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.793849013Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-09-10T18:46:54.984 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.794732824Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-09-10T18:46:54.985 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.79762708Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-09-10T18:46:54.985 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.800390851Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-09-10T18:46:54.985 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.803227297Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-09-10T18:46:54.985 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.806075269Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-09-10T18:46:54.985 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.808881159Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-09-10T18:46:54.985 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.81160351Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-09-10T18:46:54.985 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.814308169Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-09-10T18:46:54.985 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.817654981Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-09-10T18:46:54.985 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.820354637Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-09-10T18:46:54.985 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.823188775Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-09-10T18:46:54.985 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.826204447Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-09-10T18:46:54.986 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.828777317Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-09-10T18:46:54.986 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.831445947Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-09-10T18:46:54.986 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.834368249Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-09-10T18:46:54.986 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.837039149Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-09-10T18:46:54.986 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.839972411Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-09-10T18:46:54.986 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.84280099Z level=info msg="Executing migration" id="Update dashboard title length" 2024-09-10T18:46:54.986 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.845443786Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-09-10T18:46:54.986 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.84806201Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-09-10T18:46:54.986 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.850837624Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-09-10T18:46:54.986 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.85342055Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-09-10T18:46:54.986 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.855915638Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-09-10T18:46:54.986 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.858560937Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-09-10T18:46:54.986 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.861047543Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-09-10T18:46:54.986 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.863781854Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-09-10T18:46:54.986 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.866231498Z level=info msg="Executing migration" id="Add check_sum column" 2024-09-10T18:46:54.987 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.868708264Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-09-10T18:46:54.987 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.871513331Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-09-10T18:46:54.987 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.874010586Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-09-10T18:46:54.987 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.876882791Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-09-10T18:46:54.987 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.879503582Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-09-10T18:46:54.987 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.882023483Z level=info msg="Executing migration" id="create data_source table" 2024-09-10T18:46:54.987 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.884626651Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-09-10T18:46:54.987 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.887414104Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-09-10T18:46:54.987 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.889879108Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-09-10T18:46:54.987 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.890691241Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-09-10T18:46:54.987 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.891336635Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-09-10T18:46:54.987 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.892420355Z level=info msg="Executing migration" id="create data_source table v2" 2024-09-10T18:46:54.987 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.893224171Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-09-10T18:46:54.987 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.894055241Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-09-10T18:46:54.987 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.895039075Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-09-10T18:46:54.988 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.89568888Z level=info msg="Executing migration" id="Add column with_credentials" 2024-09-10T18:46:54.988 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.896662919Z level=info msg="Executing migration" id="Add secure json data column" 2024-09-10T18:46:54.988 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.897360256Z level=info msg="Executing migration" id="Update data_source table charset" 2024-09-10T18:46:54.988 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.898244Z level=info msg="Executing migration" id="Update initial version to 1" 2024-09-10T18:46:54.988 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.898740036Z level=info msg="Executing migration" id="Add read_only data column" 2024-09-10T18:46:54.988 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.899311561Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-09-10T18:46:54.988 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.90017446Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-09-10T18:46:54.988 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.900996679Z level=info msg="Executing migration" id="Add uid column" 2024-09-10T18:46:54.988 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.90194512Z level=info msg="Executing migration" id="Update uid value" 2024-09-10T18:46:54.988 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.902685847Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-09-10T18:46:54.988 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.903596573Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-09-10T18:46:54.988 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.904433315Z level=info msg="Executing migration" id="create api_key table" 2024-09-10T18:46:54.988 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.905286882Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-09-10T18:46:54.988 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.906157399Z level=info msg="Executing migration" id="add index api_key.key" 2024-09-10T18:46:54.989 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.907122999Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-09-10T18:46:54.989 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.907829944Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-09-10T18:46:54.989 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.908809032Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-09-10T18:46:54.989 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.909537336Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-09-10T18:46:54.989 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.910429466Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-09-10T18:46:54.989 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.911717874Z level=info msg="Executing migration" id="create api_key table v2" 2024-09-10T18:46:54.989 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.912132905Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-09-10T18:46:54.989 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.91297912Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-09-10T18:46:54.989 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.913818582Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-09-10T18:46:54.989 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.914888586Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-09-10T18:46:54.989 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.915220067Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-09-10T18:46:54.989 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.916150806Z level=info msg="Executing migration" id="Update api_key table charset" 2024-09-10T18:46:54.989 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.916544056Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-09-10T18:46:54.990 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.917113527Z level=info msg="Executing migration" id="Add service account foreign key" 2024-09-10T18:46:54.990 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.91801564Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-09-10T18:46:54.990 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.91835319Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-09-10T18:46:54.990 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.921009687Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-09-10T18:46:54.990 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.921619666Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-09-10T18:46:54.990 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.923936577Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-09-10T18:46:54.990 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.926810176Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-09-10T18:46:54.991 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.929563776Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-09-10T18:46:54.991 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.932075135Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-09-10T18:46:54.991 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.934725762Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-09-10T18:46:54.991 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.937476109Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-09-10T18:46:54.991 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.937805288Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-09-10T18:46:54.991 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.940499061Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-09-10T18:46:54.991 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.943447564Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-09-10T18:46:54.991 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.946207405Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-09-10T18:46:54.991 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.94653192Z level=info msg="Executing migration" id="create quota table v1" 2024-09-10T18:46:54.991 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.946923394Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-09-10T18:46:54.991 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.947366436Z level=info msg="Executing migration" id="Update quota table charset" 2024-09-10T18:46:54.991 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.947626099Z level=info msg="Executing migration" id="create plugin_setting table" 2024-09-10T18:46:54.991 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.948042365Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-09-10T18:46:54.992 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.948469976Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-09-10T18:46:54.992 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.949026076Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-09-10T18:46:54.992 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.949310334Z level=info msg="Executing migration" id="create session table" 2024-09-10T18:46:54.992 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.94976945Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-09-10T18:46:54.992 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.952471614Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-09-10T18:46:54.992 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.952789171Z level=info msg="Executing migration" id="create playlist table v2" 2024-09-10T18:46:54.992 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.953194791Z level=info msg="Executing migration" id="create playlist item table v2" 2024-09-10T18:46:54.992 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.953561065Z level=info msg="Executing migration" id="Update playlist table charset" 2024-09-10T18:46:54.992 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.95381247Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-09-10T18:46:54.992 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.954105056Z level=info msg="Executing migration" id="drop preferences table v2" 2024-09-10T18:46:54.992 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.954365171Z level=info msg="Executing migration" id="drop preferences table v3" 2024-09-10T18:46:54.992 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.954623936Z level=info msg="Executing migration" id="create preferences table v3" 2024-09-10T18:46:54.992 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.955010257Z level=info msg="Executing migration" id="Update preferences table charset" 2024-09-10T18:46:54.992 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.955288624Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-09-10T18:46:54.993 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.955846591Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-09-10T18:46:54.993 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.956151626Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-09-10T18:46:54.993 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.956706564Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-09-10T18:46:54.993 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.957342044Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-09-10T18:46:54.993 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.957636283Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-09-10T18:46:54.993 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.958052031Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-09-10T18:46:54.993 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.958490758Z level=info msg="Executing migration" id="create alert table v1" 2024-09-10T18:46:54.993 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.958922954Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-09-10T18:46:54.993 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.961815681Z level=info msg="Executing migration" id="add index alert state" 2024-09-10T18:46:54.993 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.96439447Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-09-10T18:46:54.993 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.967049859Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-09-10T18:46:54.993 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.967432721Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-09-10T18:46:54.993 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.970217524Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-09-10T18:46:54.994 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.972931592Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-09-10T18:46:54.994 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.975648178Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-09-10T18:46:54.994 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.976097648Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-09-10T18:46:54.994 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.976811239Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-09-10T18:46:54.994 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.97756778Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-09-10T18:46:54.994 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.978303136Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-09-10T18:46:54.994 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.978710918Z level=info msg="Executing migration" id="Add column is_default" 2024-09-10T18:46:54.994 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.979360628Z level=info msg="Executing migration" id="Add column frequency" 2024-09-10T18:46:54.994 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.980095735Z level=info msg="Executing migration" id="Add column send_reminder" 2024-09-10T18:46:54.994 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.980845048Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-09-10T18:46:55.119 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":43,"stamp":"2024-09-10T18:46:53.192279+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":938,"num_read_kb":795,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":568,"ondisk_log_size":568,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385360,"kb_used_data":6512,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575856,"statfs":{"total":767960285184,"available":765517676544,"internally_reserved":0,"allocated":6668288,"data_stored":3995684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.018568"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":37,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.394576+0000","last_change":"2024-09-10T18:46:33.824057+0000","last_active":"2024-09-10T18:46:46.394576+0000","last_peered":"2024-09-10T18:46:46.394576+0000","last_clean":"2024-09-10T18:46:46.394576+0000","last_became_active":"2024-09-10T18:45:24.265488+0000","last_became_peered":"2024-09-10T18:45:24.265488+0000","last_unstale":"2024-09-10T18:46:46.394576+0000","last_undegraded":"2024-09-10T18:46:46.394576+0000","last_fullsized":"2024-09-10T18:46:46.394576+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:33.823989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:33.823989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:44.131457+0000","scrub_duration":0.0034389770000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.176216+0000","last_change":"2024-09-10T18:45:17.959740+0000","last_active":"2024-09-10T18:46:46.176216+0000","last_peered":"2024-09-10T18:46:46.176216+0000","last_clean":"2024-09-10T18:46:46.176216+0000","last_became_active":"2024-09-10T18:45:17.959475+0000","last_became_peered":"2024-09-10T18:45:17.959475+0000","last_unstale":"2024-09-10T18:46:46.176216+0000","last_undegraded":"2024-09-10T18:46:46.176216+0000","last_fullsized":"2024-09-10T18:46:46.176216+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"63'15","reported_seq":66,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.394802+0000","last_change":"2024-09-10T18:46:36.856804+0000","last_active":"2024-09-10T18:46:46.394802+0000","last_peered":"2024-09-10T18:46:46.394802+0000","last_clean":"2024-09-10T18:46:46.394802+0000","last_became_active":"2024-09-10T18:45:20.052531+0000","last_became_peered":"2024-09-10T18:45:20.052531+0000","last_unstale":"2024-09-10T18:46:46.394802+0000","last_undegraded":"2024-09-10T18:46:46.394802+0000","last_fullsized":"2024-09-10T18:46:46.394802+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:46:36.856740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:36.856740+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:03.997270+0000","scrub_duration":0.0018131040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":39,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.174877+0000","last_change":"2024-09-10T18:46:40.313306+0000","last_active":"2024-09-10T18:46:46.174877+0000","last_peered":"2024-09-10T18:46:46.174877+0000","last_clean":"2024-09-10T18:46:46.174877+0000","last_became_active":"2024-09-10T18:45:22.065619+0000","last_became_peered":"2024-09-10T18:45:22.065619+0000","last_unstale":"2024-09-10T18:46:46.174877+0000","last_undegraded":"2024-09-10T18:46:46.174877+0000","last_fullsized":"2024-09-10T18:46:46.174877+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:40.313218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:40.313218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:52.103586+0000","scrub_duration":0.0017922750000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":43,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.395237+0000","last_change":"2024-09-10T18:46:18.890844+0000","last_active":"2024-09-10T18:46:46.395237+0000","last_peered":"2024-09-10T18:46:46.395237+0000","last_clean":"2024-09-10T18:46:46.395237+0000","last_became_active":"2024-09-10T18:45:17.966153+0000","last_became_peered":"2024-09-10T18:45:17.966153+0000","last_unstale":"2024-09-10T18:46:46.395237+0000","last_undegraded":"2024-09-10T18:46:46.395237+0000","last_fullsized":"2024-09-10T18:46:46.395237+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:18.890702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:18.890702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:47.062002+0000","scrub_duration":0.0019382589999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"63'9","reported_seq":47,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.174881+0000","last_change":"2024-09-10T18:45:20.061552+0000","last_active":"2024-09-10T18:46:46.174881+0000","last_peered":"2024-09-10T18:46:46.174881+0000","last_clean":"2024-09-10T18:46:46.174881+0000","last_became_active":"2024-09-10T18:45:20.061266+0000","last_became_peered":"2024-09-10T18:45:20.061266+0000","last_unstale":"2024-09-10T18:46:46.174881+0000","last_undegraded":"2024-09-10T18:46:46.174881+0000","last_fullsized":"2024-09-10T18:46:46.174881+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.175263+0000","last_change":"2024-09-10T18:45:22.073139+0000","last_active":"2024-09-10T18:46:46.175263+0000","last_peered":"2024-09-10T18:46:46.175263+0000","last_clean":"2024-09-10T18:46:46.175263+0000","last_became_active":"2024-09-10T18:45:22.072822+0000","last_became_peered":"2024-09-10T18:45:22.072822+0000","last_unstale":"2024-09-10T18:46:46.175263+0000","last_undegraded":"2024-09-10T18:46:46.175263+0000","last_fullsized":"2024-09-10T18:46:46.175263+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.174996+0000","last_change":"2024-09-10T18:45:24.081275+0000","last_active":"2024-09-10T18:46:46.174996+0000","last_peered":"2024-09-10T18:46:46.174996+0000","last_clean":"2024-09-10T18:46:46.174996+0000","last_became_active":"2024-09-10T18:45:24.081002+0000","last_became_peered":"2024-09-10T18:45:24.081002+0000","last_unstale":"2024-09-10T18:46:46.174996+0000","last_undegraded":"2024-09-10T18:46:46.174996+0000","last_fullsized":"2024-09-10T18:46:46.174996+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":43,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.607040+0000","last_change":"2024-09-10T18:46:46.605919+0000","last_active":"2024-09-10T18:46:46.607040+0000","last_peered":"2024-09-10T18:46:46.607040+0000","last_clean":"2024-09-10T18:46:46.607040+0000","last_became_active":"2024-09-10T18:45:17.961097+0000","last_became_peered":"2024-09-10T18:45:17.961097+0000","last_unstale":"2024-09-10T18:46:46.607040+0000","last_undegraded":"2024-09-10T18:46:46.607040+0000","last_fullsized":"2024-09-10T18:46:46.607040+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:46.605814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:46.605814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:51.831114+0000","scrub_duration":0.001570625,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"63'5","reported_seq":41,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.175038+0000","last_change":"2024-09-10T18:45:20.058010+0000","last_active":"2024-09-10T18:46:46.175038+0000","last_peered":"2024-09-10T18:46:46.175038+0000","last_clean":"2024-09-10T18:46:46.175038+0000","last_became_active":"2024-09-10T18:45:20.057567+0000","last_became_peered":"2024-09-10T18:45:20.057567+0000","last_unstale":"2024-09-10T18:46:46.175038+0000","last_undegraded":"2024-09-10T18:46:46.175038+0000","last_fullsized":"2024-09-10T18:46:46.175038+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":39,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.606034+0000","last_change":"2024-09-10T18:46:22.547871+0000","last_active":"2024-09-10T18:46:46.606034+0000","last_peered":"2024-09-10T18:46:46.606034+0000","last_clean":"2024-09-10T18:46:46.606034+0000","last_became_active":"2024-09-10T18:45:22.076575+0000","last_became_peered":"2024-09-10T18:45:22.076575+0000","last_unstale":"2024-09-10T18:46:46.606034+0000","last_undegraded":"2024-09-10T18:46:46.606034+0000","last_fullsized":"2024-09-10T18:46:46.606034+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:22.547800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:22.547800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:32.775669+0000","scrub_duration":0.001880622,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":37,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.232490+0000","last_change":"2024-09-10T18:46:36.158152+0000","last_active":"2024-09-10T18:46:46.232490+0000","last_peered":"2024-09-10T18:46:46.232490+0000","last_clean":"2024-09-10T18:46:46.232490+0000","last_became_active":"2024-09-10T18:45:24.079534+0000","last_became_peered":"2024-09-10T18:45:24.079534+0000","last_unstale":"2024-09-10T18:46:46.232490+0000","last_undegraded":"2024-09-10T18:46:46.232490+0000","last_fullsized":"2024-09-10T18:46:46.232490+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:36.158087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:36.158087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:41.325875+0000","scrub_duration":0.0014477100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":43,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T18:46:50.666491+0000","last_change":"2024-09-10T18:46:50.666447+0000","last_active":"2024-09-10T18:46:50.666491+0000","last_peered":"2024-09-10T18:46:50.666491+0000","last_clean":"2024-09-10T18:46:50.666491+0000","last_became_active":"2024-09-10T18:45:17.963121+0000","last_became_peered":"2024-09-10T18:45:17.963121+0000","last_unstale":"2024-09-10T18:46:50.666491+0000","last_undegraded":"2024-09-10T18:46:50.666491+0000","last_fullsized":"2024-09-10T18:46:50.666491+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:50.666379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.666379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:06.539690+0000","scrub_duration":0.0016221689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"63'9","reported_seq":47,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.175253+0000","last_change":"2024-09-10T18:45:20.053258+0000","last_active":"2024-09-10T18:46:46.175253+0000","last_peered":"2024-09-10T18:46:46.175253+0000","last_clean":"2024-09-10T18:46:46.175253+0000","last_became_active":"2024-09-10T18:45:20.052652+0000","last_became_peered":"2024-09-10T18:45:20.052652+0000","last_unstale":"2024-09-10T18:46:46.175253+0000","last_undegraded":"2024-09-10T18:46:46.175253+0000","last_fullsized":"2024-09-10T18:46:46.175253+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":39,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.232717+0000","last_change":"2024-09-10T18:46:31.024322+0000","last_active":"2024-09-10T18:46:46.232717+0000","last_peered":"2024-09-10T18:46:46.232717+0000","last_clean":"2024-09-10T18:46:46.232717+0000","last_became_active":"2024-09-10T18:45:22.068326+0000","last_became_peered":"2024-09-10T18:45:22.068326+0000","last_unstale":"2024-09-10T18:46:46.232717+0000","last_undegraded":"2024-09-10T18:46:46.232717+0000","last_fullsized":"2024-09-10T18:46:46.232717+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:31.024255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:31.024255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:50.407493+0000","scrub_duration":0.00152395,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.174823+0000","last_change":"2024-09-10T18:45:24.081968+0000","last_active":"2024-09-10T18:46:46.174823+0000","last_peered":"2024-09-10T18:46:46.174823+0000","last_clean":"2024-09-10T18:46:46.174823+0000","last_became_active":"2024-09-10T18:45:24.081488+0000","last_became_peered":"2024-09-10T18:45:24.081488+0000","last_unstale":"2024-09-10T18:46:46.174823+0000","last_undegraded":"2024-09-10T18:46:46.174823+0000","last_fullsized":"2024-09-10T18:46:46.174823+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:51.266075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"78'25","reported_seq":77,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.175406+0000","last_change":"2024-09-10T18:46:44.316200+0000","last_active":"2024-09-10T18:46:46.175406+0000","last_peered":"2024-09-10T18:46:46.175406+0000","last_clean":"2024-09-10T18:46:46.175406+0000","last_became_active":"2024-09-10T18:45:20.057019+0000","last_became_peered":"2024-09-10T18:45:20.057019+0000","last_unstale":"2024-09-10T18:46:46.175406+0000","last_undegraded":"2024-09-10T18:46:46.175406+0000","last_fullsized":"2024-09-10T18:46:46.175406+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'25","last_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_deep_scrub":"78'25","last_deep_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_clean_scrub_stamp":"2024-09-10T18:46:44.316146+0000","objects_scrubbed":8,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:06.777730+0000","scrub_duration":0.0046415040000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":43,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:47.595229+0000","last_change":"2024-09-10T18:46:47.595182+0000","last_active":"2024-09-10T18:46:47.595229+0000","last_peered":"2024-09-10T18:46:47.595229+0000","last_clean":"2024-09-10T18:46:47.595229+0000","last_became_active":"2024-09-10T18:45:17.963732+0000","last_became_peered":"2024-09-10T18:45:17.963732+0000","last_unstale":"2024-09-10T18:46:47.595229+0000","last_undegraded":"2024-09-10T18:46:47.595229+0000","last_fullsized":"2024-09-10T18:46:47.595229+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:47.595114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:47.595114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:53.271455+0000","scrub_duration":0.0014913249999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.175686+0000","last_change":"2024-09-10T18:45:22.071993+0000","last_active":"2024-09-10T18:46:46.175686+0000","last_peered":"2024-09-10T18:46:46.175686+0000","last_clean":"2024-09-10T18:46:46.175686+0000","last_became_active":"2024-09-10T18:45:22.071443+0000","last_became_peered":"2024-09-10T18:45:22.071443+0000","last_unstale":"2024-09-10T18:46:46.175686+0000","last_undegraded":"2024-09-10T18:46:46.175686+0000","last_fullsized":"2024-09-10T18:46:46.175686+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":37,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.609950+0000","last_change":"2024-09-10T18:46:35.738217+0000","last_active":"2024-09-10T18:46:46.609950+0000","last_peered":"2024-09-10T18:46:46.609950+0000","last_clean":"2024-09-10T18:46:46.609950+0000","last_became_active":"2024-09-10T18:45:24.077897+0000","last_became_peered":"2024-09-10T18:45:24.077897+0000","last_unstale":"2024-09-10T18:46:46.609950+0000","last_undegraded":"2024-09-10T18:46:46.609950+0000","last_fullsized":"2024-09-10T18:46:46.609950+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:35.738143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:35.738143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:48.741747+0000","scrub_duration":0.0015293520000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"63'10","reported_seq":56,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.175218+0000","last_change":"2024-09-10T18:46:31.281588+0000","last_active":"2024-09-10T18:46:46.175218+0000","last_peered":"2024-09-10T18:46:46.175218+0000","last_clean":"2024-09-10T18:46:46.175218+0000","last_became_active":"2024-09-10T18:45:20.053428+0000","last_became_peered":"2024-09-10T18:45:20.053428+0000","last_unstale":"2024-09-10T18:46:46.175218+0000","last_undegraded":"2024-09-10T18:46:46.175218+0000","last_fullsized":"2024-09-10T18:46:46.175218+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:46:31.281519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:31.281519+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:58.508120+0000","scrub_duration":0.0019095119999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":43,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.175542+0000","last_change":"2024-09-10T18:46:38.445417+0000","last_active":"2024-09-10T18:46:46.175542+0000","last_peered":"2024-09-10T18:46:46.175542+0000","last_clean":"2024-09-10T18:46:46.175542+0000","last_became_active":"2024-09-10T18:45:17.966833+0000","last_became_peered":"2024-09-10T18:45:17.966833+0000","last_unstale":"2024-09-10T18:46:46.175542+0000","last_undegraded":"2024-09-10T18:46:46.175542+0000","last_fullsized":"2024-09-10T18:46:46.175542+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:38.445362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:38.445362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:02.460484+0000","scrub_duration":0.0012292029999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"63'8","reported_seq":65,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.609700+0000","last_change":"2024-09-10T18:46:25.787936+0000","last_active":"2024-09-10T18:46:46.609700+0000","last_peered":"2024-09-10T18:46:46.609700+0000","last_clean":"2024-09-10T18:46:46.609700+0000","last_became_active":"2024-09-10T18:45:22.068453+0000","last_became_peered":"2024-09-10T18:45:22.068453+0000","last_unstale":"2024-09-10T18:46:46.609700+0000","last_undegraded":"2024-09-10T18:46:46.609700+0000","last_fullsized":"2024-09-10T18:46:46.609700+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:25.787877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:25.787877+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:54.201629+0000","scrub_duration":0.0018093530000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":37,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.175331+0000","last_change":"2024-09-10T18:46:32.265034+0000","last_active":"2024-09-10T18:46:46.175331+0000","last_peered":"2024-09-10T18:46:46.175331+0000","last_clean":"2024-09-10T18:46:46.175331+0000","last_became_active":"2024-09-10T18:45:24.081082+0000","last_became_peered":"2024-09-10T18:45:24.081082+0000","last_unstale":"2024-09-10T18:46:46.175331+0000","last_undegraded":"2024-09-10T18:46:46.175331+0000","last_fullsized":"2024-09-10T18:46:46.175331+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:32.264963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:32.264963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:50.207705+0000","scrub_duration":0.002378509,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"63'15","reported_seq":56,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.176567+0000","last_change":"2024-09-10T18:45:20.059509+0000","last_active":"2024-09-10T18:46:46.176567+0000","last_peered":"2024-09-10T18:46:46.176567+0000","last_clean":"2024-09-10T18:46:46.176567+0000","last_became_active":"2024-09-10T18:45:20.059222+0000","last_became_peered":"2024-09-10T18:45:20.059222+0000","last_unstale":"2024-09-10T18:46:46.176567+0000","last_undegraded":"2024-09-10T18:46:46.176567+0000","last_fullsized":"2024-09-10T18:46:46.176567+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":43,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.175454+0000","last_change":"2024-09-10T18:46:28.320761+0000","last_active":"2024-09-10T18:46:46.175454+0000","last_peered":"2024-09-10T18:46:46.175454+0000","last_clean":"2024-09-10T18:46:46.175454+0000","last_became_active":"2024-09-10T18:45:17.965409+0000","last_became_peered":"2024-09-10T18:45:17.965409+0000","last_unstale":"2024-09-10T18:46:46.175454+0000","last_undegraded":"2024-09-10T18:46:46.175454+0000","last_fullsized":"2024-09-10T18:46:46.175454+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_clean_scrub_stamp":"2024-09-10T18:46:28.320704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:44.092028+0000","scrub_duration":0.001393476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"63'8","reported_seq":65,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.175426+0000","last_change":"2024-09-10T18:46:39.300139+0000","last_active":"2024-09-10T18:46:46.175426+0000","last_peered":"2024-09-10T18:46:46.175426+0000","last_clean":"2024-09-10T18:46:46.175426+0000","last_became_active":"2024-09-10T18:45:22.062497+0000","last_became_peered":"2024-09-10T18:45:22.062497+0000","last_unstale":"2024-09-10T18:46:46.175426+0000","last_undegraded":"2024-09-10T18:46:46.175426+0000","last_fullsized":"2024-09-10T18:46:46.175426+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:39.300065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:39.300065+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:44.830898+0000","scrub_duration":0.001858934,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":37,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.232838+0000","last_change":"2024-09-10T18:46:42.057055+0000","last_active":"2024-09-10T18:46:46.232838+0000","last_peered":"2024-09-10T18:46:46.232838+0000","last_clean":"2024-09-10T18:46:46.232838+0000","last_became_active":"2024-09-10T18:45:24.082362+0000","last_became_peered":"2024-09-10T18:45:24.082362+0000","last_unstale":"2024-09-10T18:46:46.232838+0000","last_undegraded":"2024-09-10T18:46:46.232838+0000","last_fullsized":"2024-09-10T18:46:46.232838+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:42.056970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:42.056970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:50.036040+0000","scrub_duration":0.0016528479999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"63'11","reported_seq":60,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.175472+0000","last_change":"2024-09-10T18:46:35.274729+0000","last_active":"2024-09-10T18:46:46.175472+0000","last_peered":"2024-09-10T18:46:46.175472+0000","last_clean":"2024-09-10T18:46:46.175472+0000","last_became_active":"2024-09-10T18:45:20.061187+0000","last_became_peered":"2024-09-10T18:45:20.061187+0000","last_unstale":"2024-09-10T18:46:46.175472+0000","last_undegraded":"2024-09-10T18:46:46.175472+0000","last_fullsized":"2024-09-10T18:46:46.175472+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:46:35.274660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:35.274660+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:51.240811+0000","scrub_duration":0.0019745819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":43,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.606058+0000","last_change":"2024-09-10T18:46:39.520497+0000","last_active":"2024-09-10T18:46:46.606058+0000","last_peered":"2024-09-10T18:46:46.606058+0000","last_clean":"2024-09-10T18:46:46.606058+0000","last_became_active":"2024-09-10T18:45:17.966133+0000","last_became_peered":"2024-09-10T18:45:17.966133+0000","last_unstale":"2024-09-10T18:46:46.606058+0000","last_undegraded":"2024-09-10T18:46:46.606058+0000","last_fullsized":"2024-09-10T18:46:46.606058+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:39.520432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:39.520432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:49.040271+0000","scrub_duration":0.0013982510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":39,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:48.116462+0000","last_change":"2024-09-10T18:46:48.116414+0000","last_active":"2024-09-10T18:46:48.116462+0000","last_peered":"2024-09-10T18:46:48.116462+0000","last_clean":"2024-09-10T18:46:48.116462+0000","last_became_active":"2024-09-10T18:45:22.068918+0000","last_became_peered":"2024-09-10T18:45:22.068918+0000","last_unstale":"2024-09-10T18:46:48.116462+0000","last_undegraded":"2024-09-10T18:46:48.116462+0000","last_fullsized":"2024-09-10T18:46:48.116462+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_clean_scrub_stamp":"2024-09-10T18:46:48.116358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:48.149931+0000","scrub_duration":0.001429331,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.394713+0000","last_change":"2024-09-10T18:45:24.264914+0000","last_active":"2024-09-10T18:46:46.394713+0000","last_peered":"2024-09-10T18:46:46.394713+0000","last_clean":"2024-09-10T18:46:46.394713+0000","last_became_active":"2024-09-10T18:45:24.264538+0000","last_became_peered":"2024-09-10T18:45:24.264538+0000","last_unstale":"2024-09-10T18:46:46.394713+0000","last_undegraded":"2024-09-10T18:46:46.394713+0000","last_fullsized":"2024-09-10T18:46:46.394713+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"63'14","reported_seq":62,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.175175+0000","last_change":"2024-09-10T18:46:29.330572+0000","last_active":"2024-09-10T18:46:46.175175+0000","last_peered":"2024-09-10T18:46:46.175175+0000","last_clean":"2024-09-10T18:46:46.175175+0000","last_became_active":"2024-09-10T18:45:20.052243+0000","last_became_peered":"2024-09-10T18:45:20.052243+0000","last_unstale":"2024-09-10T18:46:46.175175+0000","last_undegraded":"2024-09-10T18:46:46.175175+0000","last_fullsized":"2024-09-10T18:46:46.175175+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-10T18:46:29.330503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:29.330503+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:55.456983+0000","scrub_duration":0.0020437459999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":43,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.231219+0000","last_change":"2024-09-10T18:46:25.206801+0000","last_active":"2024-09-10T18:46:46.231219+0000","last_peered":"2024-09-10T18:46:46.231219+0000","last_clean":"2024-09-10T18:46:46.231219+0000","last_became_active":"2024-09-10T18:45:17.966027+0000","last_became_peered":"2024-09-10T18:45:17.966027+0000","last_unstale":"2024-09-10T18:46:46.231219+0000","last_undegraded":"2024-09-10T18:46:46.231219+0000","last_fullsized":"2024-09-10T18:46:46.231219+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_clean_scrub_stamp":"2024-09-10T18:46:25.206731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:46.038154+0000","scrub_duration":0.0016589910000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":65,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.394409+0000","last_change":"2024-09-10T18:46:40.808518+0000","last_active":"2024-09-10T18:46:46.394409+0000","last_peered":"2024-09-10T18:46:46.394409+0000","last_clean":"2024-09-10T18:46:46.394409+0000","last_became_active":"2024-09-10T18:45:22.074540+0000","last_became_peered":"2024-09-10T18:45:22.074540+0000","last_unstale":"2024-09-10T18:46:46.394409+0000","last_undegraded":"2024-09-10T18:46:46.394409+0000","last_fullsized":"2024-09-10T18:46:46.394409+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:40.808453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:40.808453+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:43.962254+0000","scrub_duration":0.0016757829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":37,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.606533+0000","last_change":"2024-09-10T18:46:32.590511+0000","last_active":"2024-09-10T18:46:46.606533+0000","last_peered":"2024-09-10T18:46:46.606533+0000","last_clean":"2024-09-10T18:46:46.606533+0000","last_became_active":"2024-09-10T18:45:24.265303+0000","last_became_peered":"2024-09-10T18:45:24.265303+0000","last_unstale":"2024-09-10T18:46:46.606533+0000","last_undegraded":"2024-09-10T18:46:46.606533+0000","last_fullsized":"2024-09-10T18:46:46.606533+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:32.590438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:32.590438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:01.680676+0000","scrub_duration":0.0013292270000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"63'18","reported_seq":73,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.394517+0000","last_change":"2024-09-10T18:46:27.801177+0000","last_active":"2024-09-10T18:46:46.394517+0000","last_peered":"2024-09-10T18:46:46.394517+0000","last_clean":"2024-09-10T18:46:46.394517+0000","last_became_active":"2024-09-10T18:45:20.420937+0000","last_became_peered":"2024-09-10T18:45:20.420937+0000","last_unstale":"2024-09-10T18:46:46.394517+0000","last_undegraded":"2024-09-10T18:46:46.394517+0000","last_fullsized":"2024-09-10T18:46:46.394517+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'18","last_scrub_stamp":"2024-09-10T18:46:27.801110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:27.801110+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:42.247024+0000","scrub_duration":0.001870095,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.176309+0000","last_change":"2024-09-10T18:45:17.962000+0000","last_active":"2024-09-10T18:46:46.176309+0000","last_peered":"2024-09-10T18:46:46.176309+0000","last_clean":"2024-09-10T18:46:46.176309+0000","last_became_active":"2024-09-10T18:45:17.961695+0000","last_became_peered":"2024-09-10T18:45:17.961695+0000","last_unstale":"2024-09-10T18:46:46.176309+0000","last_undegraded":"2024-09-10T18:46:46.176309+0000","last_fullsized":"2024-09-10T18:46:46.176309+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":39,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.174928+0000","last_change":"2024-09-10T18:46:41.312735+0000","last_active":"2024-09-10T18:46:46.174928+0000","last_peered":"2024-09-10T18:46:46.174928+0000","last_clean":"2024-09-10T18:46:46.174928+0000","last_became_active":"2024-09-10T18:45:22.074012+0000","last_became_peered":"2024-09-10T18:45:22.074012+0000","last_unstale":"2024-09-10T18:46:46.174928+0000","last_undegraded":"2024-09-10T18:46:46.174928+0000","last_fullsized":"2024-09-10T18:46:46.174928+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:41.312671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:41.312671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:47.779534+0000","scrub_duration":0.0013475010000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.175082+0000","last_change":"2024-09-10T18:45:24.077245+0000","last_active":"2024-09-10T18:46:46.175082+0000","last_peered":"2024-09-10T18:46:46.175082+0000","last_clean":"2024-09-10T18:46:46.175082+0000","last_became_active":"2024-09-10T18:45:24.076831+0000","last_became_peered":"2024-09-10T18:45:24.076831+0000","last_unstale":"2024-09-10T18:46:46.175082+0000","last_undegraded":"2024-09-10T18:46:46.175082+0000","last_fullsized":"2024-09-10T18:46:46.175082+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"63'19","reported_seq":77,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.176466+0000","last_change":"2024-09-10T18:46:42.524291+0000","last_active":"2024-09-10T18:46:46.176466+0000","last_peered":"2024-09-10T18:46:46.176466+0000","last_clean":"2024-09-10T18:46:46.176466+0000","last_became_active":"2024-09-10T18:45:20.421046+0000","last_became_peered":"2024-09-10T18:45:20.421046+0000","last_unstale":"2024-09-10T18:46:46.176466+0000","last_undegraded":"2024-09-10T18:46:46.176466+0000","last_fullsized":"2024-09-10T18:46:46.176466+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:46:42.524195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:42.524195+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:45.664487+0000","scrub_duration":0.0022840299999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.607071+0000","last_change":"2024-09-10T18:45:17.960558+0000","last_active":"2024-09-10T18:46:46.607071+0000","last_peered":"2024-09-10T18:46:46.607071+0000","last_clean":"2024-09-10T18:46:46.607071+0000","last_became_active":"2024-09-10T18:45:17.960226+0000","last_became_peered":"2024-09-10T18:45:17.960226+0000","last_unstale":"2024-09-10T18:46:46.607071+0000","last_undegraded":"2024-09-10T18:46:46.607071+0000","last_fullsized":"2024-09-10T18:46:46.607071+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.230448+0000","last_change":"2024-09-10T18:45:22.066434+0000","last_active":"2024-09-10T18:46:46.230448+0000","last_peered":"2024-09-10T18:46:46.230448+0000","last_clean":"2024-09-10T18:46:46.230448+0000","last_became_active":"2024-09-10T18:45:22.066148+0000","last_became_peered":"2024-09-10T18:45:22.066148+0000","last_unstale":"2024-09-10T18:46:46.230448+0000","last_undegraded":"2024-09-10T18:46:46.230448+0000","last_fullsized":"2024-09-10T18:46:46.230448+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.175187+0000","last_change":"2024-09-10T18:45:24.265025+0000","last_active":"2024-09-10T18:46:46.175187+0000","last_peered":"2024-09-10T18:46:46.175187+0000","last_clean":"2024-09-10T18:46:46.175187+0000","last_became_active":"2024-09-10T18:45:24.264680+0000","last_became_peered":"2024-09-10T18:45:24.264680+0000","last_unstale":"2024-09-10T18:46:46.175187+0000","last_undegraded":"2024-09-10T18:46:46.175187+0000","last_fullsized":"2024-09-10T18:46:46.175187+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"63'10","reported_seq":46,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.175416+0000","last_change":"2024-09-10T18:45:20.057590+0000","last_active":"2024-09-10T18:46:46.175416+0000","last_peered":"2024-09-10T18:46:46.175416+0000","last_clean":"2024-09-10T18:46:46.175416+0000","last_became_active":"2024-09-10T18:45:20.057204+0000","last_became_peered":"2024-09-10T18:45:20.057204+0000","last_unstale":"2024-09-10T18:46:46.175416+0000","last_undegraded":"2024-09-10T18:46:46.175416+0000","last_fullsized":"2024-09-10T18:46:46.175416+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T18:46:50.443130+0000","last_change":"2024-09-10T18:46:50.443101+0000","last_active":"2024-09-10T18:46:50.443130+0000","last_peered":"2024-09-10T18:46:50.443130+0000","last_clean":"2024-09-10T18:46:50.443130+0000","last_became_active":"2024-09-10T18:45:17.966632+0000","last_became_peered":"2024-09-10T18:45:17.966632+0000","last_unstale":"2024-09-10T18:46:50.443130+0000","last_undegraded":"2024-09-10T18:46:50.443130+0000","last_fullsized":"2024-09-10T18:46:50.443130+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:50.443042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.443042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:18.989766+0000","scrub_duration":0.0014088550000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"63'8","reported_seq":55,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.176375+0000","last_change":"2024-09-10T18:45:22.063631+0000","last_active":"2024-09-10T18:46:46.176375+0000","last_peered":"2024-09-10T18:46:46.176375+0000","last_clean":"2024-09-10T18:46:46.176375+0000","last_became_active":"2024-09-10T18:45:22.063326+0000","last_became_peered":"2024-09-10T18:45:22.063326+0000","last_unstale":"2024-09-10T18:46:46.176375+0000","last_undegraded":"2024-09-10T18:46:46.176375+0000","last_fullsized":"2024-09-10T18:46:46.176375+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.174713+0000","last_change":"2024-09-10T18:45:24.077780+0000","last_active":"2024-09-10T18:46:46.174713+0000","last_peered":"2024-09-10T18:46:46.174713+0000","last_clean":"2024-09-10T18:46:46.174713+0000","last_became_active":"2024-09-10T18:45:24.077412+0000","last_became_peered":"2024-09-10T18:45:24.077412+0000","last_unstale":"2024-09-10T18:46:46.174713+0000","last_undegraded":"2024-09-10T18:46:46.174713+0000","last_fullsized":"2024-09-10T18:46:46.174713+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"63'28","reported_seq":107,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.176323+0000","last_change":"2024-09-10T18:46:19.223319+0000","last_active":"2024-09-10T18:46:46.176323+0000","last_peered":"2024-09-10T18:46:46.176323+0000","last_clean":"2024-09-10T18:46:46.176323+0000","last_became_active":"2024-09-10T18:45:20.061413+0000","last_became_peered":"2024-09-10T18:45:20.061413+0000","last_unstale":"2024-09-10T18:46:46.176323+0000","last_undegraded":"2024-09-10T18:46:46.176323+0000","last_fullsized":"2024-09-10T18:46:46.176323+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'28","last_scrub_stamp":"2024-09-10T18:46:19.223250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:19.223250+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:23.644429+0000","scrub_duration":0.0032067810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":43,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:49.120791+0000","last_change":"2024-09-10T18:46:49.120758+0000","last_active":"2024-09-10T18:46:49.120791+0000","last_peered":"2024-09-10T18:46:49.120791+0000","last_clean":"2024-09-10T18:46:49.120791+0000","last_became_active":"2024-09-10T18:45:17.964494+0000","last_became_peered":"2024-09-10T18:45:17.964494+0000","last_unstale":"2024-09-10T18:46:49.120791+0000","last_undegraded":"2024-09-10T18:46:49.120791+0000","last_fullsized":"2024-09-10T18:46:49.120791+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:49.120704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:49.120704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:51.579079+0000","scrub_duration":0.0011863380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.175586+0000","last_change":"2024-09-10T18:45:22.061738+0000","last_active":"2024-09-10T18:46:46.175586+0000","last_peered":"2024-09-10T18:46:46.175586+0000","last_clean":"2024-09-10T18:46:46.175586+0000","last_became_active":"2024-09-10T18:45:22.061428+0000","last_became_peered":"2024-09-10T18:45:22.061428+0000","last_unstale":"2024-09-10T18:46:46.175586+0000","last_undegraded":"2024-09-10T18:46:46.175586+0000","last_fullsized":"2024-09-10T18:46:46.175586+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.395144+0000","last_change":"2024-09-10T18:45:24.073623+0000","last_active":"2024-09-10T18:46:46.395144+0000","last_peered":"2024-09-10T18:46:46.395144+0000","last_clean":"2024-09-10T18:46:46.395144+0000","last_became_active":"2024-09-10T18:45:24.073307+0000","last_became_peered":"2024-09-10T18:45:24.073307+0000","last_unstale":"2024-09-10T18:46:46.395144+0000","last_undegraded":"2024-09-10T18:46:46.395144+0000","last_fullsized":"2024-09-10T18:46:46.395144+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:51.119822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"63'13","reported_seq":58,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.176598+0000","last_change":"2024-09-10T18:45:20.058778+0000","last_active":"2024-09-10T18:46:46.176598+0000","last_peered":"2024-09-10T18:46:46.176598+0000","last_clean":"2024-09-10T18:46:46.176598+0000","last_became_active":"2024-09-10T18:45:20.058435+0000","last_became_peered":"2024-09-10T18:45:20.058435+0000","last_unstale":"2024-09-10T18:46:46.176598+0000","last_undegraded":"2024-09-10T18:46:46.176598+0000","last_fullsized":"2024-09-10T18:46:46.176598+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"65'5","reported_seq":126,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T18:46:49.479910+0000","last_change":"2024-09-10T18:45:22.059997+0000","last_active":"2024-09-10T18:46:49.479910+0000","last_peered":"2024-09-10T18:46:49.479910+0000","last_clean":"2024-09-10T18:46:49.479910+0000","last_became_active":"2024-09-10T18:45:17.969868+0000","last_became_peered":"2024-09-10T18:45:17.969868+0000","last_unstale":"2024-09-10T18:46:49.479910+0000","last_undegraded":"2024-09-10T18:46:49.479910+0000","last_fullsized":"2024-09-10T18:46:49.479910+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021933159999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":85,"num_read_kb":80,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":43,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.712377+0000","last_change":"2024-09-10T18:46:46.712335+0000","last_active":"2024-09-10T18:46:46.712377+0000","last_peered":"2024-09-10T18:46:46.712377+0000","last_clean":"2024-09-10T18:46:46.712377+0000","last_became_active":"2024-09-10T18:45:17.960859+0000","last_became_peered":"2024-09-10T18:45:17.960859+0000","last_unstale":"2024-09-10T18:46:46.712377+0000","last_undegraded":"2024-09-10T18:46:46.712377+0000","last_fullsized":"2024-09-10T18:46:46.712377+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:46.712269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:46.712269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:54.660825+0000","scrub_duration":0.001509383,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":39,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.609887+0000","last_change":"2024-09-10T18:46:41.818890+0000","last_active":"2024-09-10T18:46:46.609887+0000","last_peered":"2024-09-10T18:46:46.609887+0000","last_clean":"2024-09-10T18:46:46.609887+0000","last_became_active":"2024-09-10T18:45:22.068350+0000","last_became_peered":"2024-09-10T18:45:22.068350+0000","last_unstale":"2024-09-10T18:46:46.609887+0000","last_undegraded":"2024-09-10T18:46:46.609887+0000","last_fullsized":"2024-09-10T18:46:46.609887+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:41.818827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:41.818827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:07.148553+0000","scrub_duration":0.0017408219999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.606884+0000","last_change":"2024-09-10T18:45:24.264879+0000","last_active":"2024-09-10T18:46:46.606884+0000","last_peered":"2024-09-10T18:46:46.606884+0000","last_clean":"2024-09-10T18:46:46.606884+0000","last_became_active":"2024-09-10T18:45:24.264617+0000","last_became_peered":"2024-09-10T18:45:24.264617+0000","last_unstale":"2024-09-10T18:46:46.606884+0000","last_undegraded":"2024-09-10T18:46:46.606884+0000","last_fullsized":"2024-09-10T18:46:46.606884+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"63'12","reported_seq":49,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.176178+0000","last_change":"2024-09-10T18:45:20.061430+0000","last_active":"2024-09-10T18:46:46.176178+0000","last_peered":"2024-09-10T18:46:46.176178+0000","last_clean":"2024-09-10T18:46:46.176178+0000","last_became_active":"2024-09-10T18:45:20.061091+0000","last_became_peered":"2024-09-10T18:45:20.061091+0000","last_unstale":"2024-09-10T18:46:46.176178+0000","last_undegraded":"2024-09-10T18:46:46.176178+0000","last_fullsized":"2024-09-10T18:46:46.176178+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"58'1","reported_seq":39,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.175078+0000","last_change":"2024-09-10T18:45:22.055928+0000","last_active":"2024-09-10T18:46:46.175078+0000","last_peered":"2024-09-10T18:46:46.175078+0000","last_clean":"2024-09-10T18:46:46.175078+0000","last_became_active":"2024-09-10T18:45:17.963932+0000","last_became_peered":"2024-09-10T18:45:17.963932+0000","last_unstale":"2024-09-10T18:46:46.175078+0000","last_undegraded":"2024-09-10T18:46:46.175078+0000","last_fullsized":"2024-09-10T18:46:46.175078+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037812099999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.606143+0000","last_change":"2024-09-10T18:45:17.966921+0000","last_active":"2024-09-10T18:46:46.606143+0000","last_peered":"2024-09-10T18:46:46.606143+0000","last_clean":"2024-09-10T18:46:46.606143+0000","last_became_active":"2024-09-10T18:45:17.966493+0000","last_became_peered":"2024-09-10T18:45:17.966493+0000","last_unstale":"2024-09-10T18:46:46.606143+0000","last_undegraded":"2024-09-10T18:46:46.606143+0000","last_fullsized":"2024-09-10T18:46:46.606143+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.233047+0000","last_change":"2024-09-10T18:45:22.064783+0000","last_active":"2024-09-10T18:46:46.233047+0000","last_peered":"2024-09-10T18:46:46.233047+0000","last_clean":"2024-09-10T18:46:46.233047+0000","last_became_active":"2024-09-10T18:45:22.064285+0000","last_became_peered":"2024-09-10T18:45:22.064285+0000","last_unstale":"2024-09-10T18:46:46.233047+0000","last_undegraded":"2024-09-10T18:46:46.233047+0000","last_fullsized":"2024-09-10T18:46:46.233047+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":37,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:49.422951+0000","last_change":"2024-09-10T18:46:49.422912+0000","last_active":"2024-09-10T18:46:49.422951+0000","last_peered":"2024-09-10T18:46:49.422951+0000","last_clean":"2024-09-10T18:46:49.422951+0000","last_became_active":"2024-09-10T18:45:24.083689+0000","last_became_peered":"2024-09-10T18:45:24.083689+0000","last_unstale":"2024-09-10T18:46:49.422951+0000","last_undegraded":"2024-09-10T18:46:49.422951+0000","last_fullsized":"2024-09-10T18:46:49.422951+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:49.422843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:49.422843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:00.864466+0000","scrub_duration":0.0014985300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"63'14","reported_seq":63,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.230694+0000","last_change":"2024-09-10T18:46:22.294141+0000","last_active":"2024-09-10T18:46:46.230694+0000","last_peered":"2024-09-10T18:46:46.230694+0000","last_clean":"2024-09-10T18:46:46.230694+0000","last_became_active":"2024-09-10T18:45:20.421119+0000","last_became_peered":"2024-09-10T18:45:20.421119+0000","last_unstale":"2024-09-10T18:46:46.230694+0000","last_undegraded":"2024-09-10T18:46:46.230694+0000","last_fullsized":"2024-09-10T18:46:46.230694+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-10T18:46:22.294072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:22.294072+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:49.887899+0000","scrub_duration":0.0021435069999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"60'2","reported_seq":50,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.394687+0000","last_change":"2024-09-10T18:46:25.845131+0000","last_active":"2024-09-10T18:46:46.394687+0000","last_peered":"2024-09-10T18:46:46.394687+0000","last_clean":"2024-09-10T18:46:46.394687+0000","last_became_active":"2024-09-10T18:45:17.961660+0000","last_became_peered":"2024-09-10T18:45:17.961660+0000","last_unstale":"2024-09-10T18:46:46.394687+0000","last_undegraded":"2024-09-10T18:46:46.394687+0000","last_fullsized":"2024-09-10T18:46:46.394687+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-10T18:46:25.845070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:25.845070+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:50.370600+0000","scrub_duration":0.0020129710000000001,"objects_trimmed":0,"snaptrim_duration":0.0013942640000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":43,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.232962+0000","last_change":"2024-09-10T18:46:29.081362+0000","last_active":"2024-09-10T18:46:46.232962+0000","last_peered":"2024-09-10T18:46:46.232962+0000","last_clean":"2024-09-10T18:46:46.232962+0000","last_became_active":"2024-09-10T18:45:17.965163+0000","last_became_peered":"2024-09-10T18:45:17.965163+0000","last_unstale":"2024-09-10T18:46:46.232962+0000","last_undegraded":"2024-09-10T18:46:46.232962+0000","last_fullsized":"2024-09-10T18:46:46.232962+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:29.081296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:29.081296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:50.659900+0000","scrub_duration":0.0015488590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'81","reported_seq":77,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.606328+0000","last_change":"2024-09-10T18:45:45.481030+0000","last_active":"2024-09-10T18:46:46.606328+0000","last_peered":"2024-09-10T18:46:46.606328+0000","last_clean":"2024-09-10T18:46:46.606328+0000","last_became_active":"2024-09-10T18:45:07.433592+0000","last_became_peered":"2024-09-10T18:45:07.433592+0000","last_unstale":"2024-09-10T18:46:46.606328+0000","last_undegraded":"2024-09-10T18:46:46.606328+0000","last_fullsized":"2024-09-10T18:46:46.606328+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'74","last_scrub_stamp":"2024-09-10T18:45:45.480958+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:45:45.480958+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:58.297247+0000","scrub_duration":0.0019876809999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":39,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.606829+0000","last_change":"2024-09-10T18:46:26.687029+0000","last_active":"2024-09-10T18:46:46.606829+0000","last_peered":"2024-09-10T18:46:46.606829+0000","last_clean":"2024-09-10T18:46:46.606829+0000","last_became_active":"2024-09-10T18:45:22.068226+0000","last_became_peered":"2024-09-10T18:45:22.068226+0000","last_unstale":"2024-09-10T18:46:46.606829+0000","last_undegraded":"2024-09-10T18:46:46.606829+0000","last_fullsized":"2024-09-10T18:46:46.606829+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:26.686961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:26.686961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:32.456741+0000","scrub_duration":0.0015985109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.233182+0000","last_change":"2024-09-10T18:45:24.073576+0000","last_active":"2024-09-10T18:46:46.233182+0000","last_peered":"2024-09-10T18:46:46.233182+0000","last_clean":"2024-09-10T18:46:46.233182+0000","last_became_active":"2024-09-10T18:45:24.073327+0000","last_became_peered":"2024-09-10T18:45:24.073327+0000","last_unstale":"2024-09-10T18:46:46.233182+0000","last_undegraded":"2024-09-10T18:46:46.233182+0000","last_fullsized":"2024-09-10T18:46:46.233182+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"63'19","reported_seq":72,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.230931+0000","last_change":"2024-09-10T18:46:28.229804+0000","last_active":"2024-09-10T18:46:46.230931+0000","last_peered":"2024-09-10T18:46:46.230931+0000","last_clean":"2024-09-10T18:46:46.230931+0000","last_became_active":"2024-09-10T18:45:20.421097+0000","last_became_peered":"2024-09-10T18:45:20.421097+0000","last_unstale":"2024-09-10T18:46:46.230931+0000","last_undegraded":"2024-09-10T18:46:46.230931+0000","last_fullsized":"2024-09-10T18:46:46.230931+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:46:28.229737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:28.229737+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:49.859218+0000","scrub_duration":0.002308184,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.610019+0000","last_change":"2024-09-10T18:45:17.960491+0000","last_active":"2024-09-10T18:46:46.610019+0000","last_peered":"2024-09-10T18:46:46.610019+0000","last_clean":"2024-09-10T18:46:46.610019+0000","last_became_active":"2024-09-10T18:45:17.959924+0000","last_became_peered":"2024-09-10T18:45:17.959924+0000","last_unstale":"2024-09-10T18:46:46.610019+0000","last_undegraded":"2024-09-10T18:46:46.610019+0000","last_fullsized":"2024-09-10T18:46:46.610019+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":39,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.609681+0000","last_change":"2024-09-10T18:46:30.781786+0000","last_active":"2024-09-10T18:46:46.609681+0000","last_peered":"2024-09-10T18:46:46.609681+0000","last_clean":"2024-09-10T18:46:46.609681+0000","last_became_active":"2024-09-10T18:45:22.072581+0000","last_became_peered":"2024-09-10T18:45:22.072581+0000","last_unstale":"2024-09-10T18:46:46.609681+0000","last_undegraded":"2024-09-10T18:46:46.609681+0000","last_fullsized":"2024-09-10T18:46:46.609681+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:30.781730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:30.781730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:38.248319+0000","scrub_duration":0.0020255870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.606230+0000","last_change":"2024-09-10T18:45:24.076661+0000","last_active":"2024-09-10T18:46:46.606230+0000","last_peered":"2024-09-10T18:46:46.606230+0000","last_clean":"2024-09-10T18:46:46.606230+0000","last_became_active":"2024-09-10T18:45:24.076392+0000","last_became_peered":"2024-09-10T18:45:24.076392+0000","last_unstale":"2024-09-10T18:46:46.606230+0000","last_undegraded":"2024-09-10T18:46:46.606230+0000","last_fullsized":"2024-09-10T18:46:46.606230+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"63'9","reported_seq":47,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.176366+0000","last_change":"2024-09-10T18:45:20.061357+0000","last_active":"2024-09-10T18:46:46.176366+0000","last_peered":"2024-09-10T18:46:46.176366+0000","last_clean":"2024-09-10T18:46:46.176366+0000","last_became_active":"2024-09-10T18:45:20.061021+0000","last_became_peered":"2024-09-10T18:45:20.061021+0000","last_unstale":"2024-09-10T18:46:46.176366+0000","last_undegraded":"2024-09-10T18:46:46.176366+0000","last_fullsized":"2024-09-10T18:46:46.176366+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.176761+0000","last_change":"2024-09-10T18:45:17.963854+0000","last_active":"2024-09-10T18:46:46.176761+0000","last_peered":"2024-09-10T18:46:46.176761+0000","last_clean":"2024-09-10T18:46:46.176761+0000","last_became_active":"2024-09-10T18:45:17.963057+0000","last_became_peered":"2024-09-10T18:45:17.963057+0000","last_unstale":"2024-09-10T18:46:46.176761+0000","last_undegraded":"2024-09-10T18:46:46.176761+0000","last_fullsized":"2024-09-10T18:46:46.176761+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.609756+0000","last_change":"2024-09-10T18:45:22.070613+0000","last_active":"2024-09-10T18:46:46.609756+0000","last_peered":"2024-09-10T18:46:46.609756+0000","last_clean":"2024-09-10T18:46:46.609756+0000","last_became_active":"2024-09-10T18:45:22.070343+0000","last_became_peered":"2024-09-10T18:45:22.070343+0000","last_unstale":"2024-09-10T18:46:46.609756+0000","last_undegraded":"2024-09-10T18:46:46.609756+0000","last_fullsized":"2024-09-10T18:46:46.609756+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":37,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:47.545475+0000","last_change":"2024-09-10T18:46:47.545406+0000","last_active":"2024-09-10T18:46:47.545475+0000","last_peered":"2024-09-10T18:46:47.545475+0000","last_clean":"2024-09-10T18:46:47.545475+0000","last_became_active":"2024-09-10T18:45:24.078529+0000","last_became_peered":"2024-09-10T18:45:24.078529+0000","last_unstale":"2024-09-10T18:46:47.545475+0000","last_undegraded":"2024-09-10T18:46:47.545475+0000","last_fullsized":"2024-09-10T18:46:47.545475+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:47.545343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:47.545343+0000","last_clean_scrub_stamp":"2024-09-10T18:46:47.545343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:02.171974+0000","scrub_duration":0.0012977799999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"63'15","reported_seq":56,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.232372+0000","last_change":"2024-09-10T18:45:20.420130+0000","last_active":"2024-09-10T18:46:46.232372+0000","last_peered":"2024-09-10T18:46:46.232372+0000","last_clean":"2024-09-10T18:46:46.232372+0000","last_became_active":"2024-09-10T18:45:20.419804+0000","last_became_peered":"2024-09-10T18:45:20.419804+0000","last_unstale":"2024-09-10T18:46:46.232372+0000","last_undegraded":"2024-09-10T18:46:46.232372+0000","last_fullsized":"2024-09-10T18:46:46.232372+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":43,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.610016+0000","last_change":"2024-09-10T18:46:17.819966+0000","last_active":"2024-09-10T18:46:46.610016+0000","last_peered":"2024-09-10T18:46:46.610016+0000","last_clean":"2024-09-10T18:46:46.610016+0000","last_became_active":"2024-09-10T18:45:17.959653+0000","last_became_peered":"2024-09-10T18:45:17.959653+0000","last_unstale":"2024-09-10T18:46:46.610016+0000","last_undegraded":"2024-09-10T18:46:46.610016+0000","last_fullsized":"2024-09-10T18:46:46.610016+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:17.819909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:17.819909+0000","last_clean_scrub_stamp":"2024-09-10T18:46:17.819909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:18.454145+0000","scrub_duration":0.001612109,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"63'8","reported_seq":58,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.606148+0000","last_change":"2024-09-10T18:45:22.073687+0000","last_active":"2024-09-10T18:46:46.606148+0000","last_peered":"2024-09-10T18:46:46.606148+0000","last_clean":"2024-09-10T18:46:46.606148+0000","last_became_active":"2024-09-10T18:45:22.073383+0000","last_became_peered":"2024-09-10T18:45:22.073383+0000","last_unstale":"2024-09-10T18:46:46.606148+0000","last_undegraded":"2024-09-10T18:46:46.606148+0000","last_fullsized":"2024-09-10T18:46:46.606148+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":37,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.232491+0000","last_change":"2024-09-10T18:46:45.083522+0000","last_active":"2024-09-10T18:46:46.232491+0000","last_peered":"2024-09-10T18:46:46.232491+0000","last_clean":"2024-09-10T18:46:46.232491+0000","last_became_active":"2024-09-10T18:45:24.265256+0000","last_became_peered":"2024-09-10T18:45:24.265256+0000","last_unstale":"2024-09-10T18:46:46.232491+0000","last_undegraded":"2024-09-10T18:46:46.232491+0000","last_fullsized":"2024-09-10T18:46:46.232491+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_clean_scrub_stamp":"2024-09-10T18:46:45.083425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:08.177993+0000","scrub_duration":0.0017589890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"63'12","reported_seq":64,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.175183+0000","last_change":"2024-09-10T18:46:30.320527+0000","last_active":"2024-09-10T18:46:46.175183+0000","last_peered":"2024-09-10T18:46:46.175183+0000","last_clean":"2024-09-10T18:46:46.175183+0000","last_became_active":"2024-09-10T18:45:20.056869+0000","last_became_peered":"2024-09-10T18:45:20.056869+0000","last_unstale":"2024-09-10T18:46:46.175183+0000","last_undegraded":"2024-09-10T18:46:46.175183+0000","last_fullsized":"2024-09-10T18:46:46.175183+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:46:30.320462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:30.320462+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:35.180189+0000","scrub_duration":0.0028822850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"56'2","reported_seq":60,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.175563+0000","last_change":"2024-09-10T18:46:42.303648+0000","last_active":"2024-09-10T18:46:46.175563+0000","last_peered":"2024-09-10T18:46:46.175563+0000","last_clean":"2024-09-10T18:46:46.175563+0000","last_became_active":"2024-09-10T18:45:17.963286+0000","last_became_peered":"2024-09-10T18:45:17.963286+0000","last_unstale":"2024-09-10T18:46:46.175563+0000","last_undegraded":"2024-09-10T18:46:46.175563+0000","last_fullsized":"2024-09-10T18:46:46.175563+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-10T18:46:42.303567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:42.303567+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:05.463057+0000","scrub_duration":0.0020684750000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":39,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.609569+0000","last_change":"2024-09-10T18:46:44.754345+0000","last_active":"2024-09-10T18:46:46.609569+0000","last_peered":"2024-09-10T18:46:46.609569+0000","last_clean":"2024-09-10T18:46:46.609569+0000","last_became_active":"2024-09-10T18:45:22.068957+0000","last_became_peered":"2024-09-10T18:45:22.068957+0000","last_unstale":"2024-09-10T18:46:46.609569+0000","last_undegraded":"2024-09-10T18:46:46.609569+0000","last_fullsized":"2024-09-10T18:46:46.609569+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:44.754286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:44.754286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:47.893148+0000","scrub_duration":0.001258386,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.394691+0000","last_change":"2024-09-10T18:45:24.078134+0000","last_active":"2024-09-10T18:46:46.394691+0000","last_peered":"2024-09-10T18:46:46.394691+0000","last_clean":"2024-09-10T18:46:46.394691+0000","last_became_active":"2024-09-10T18:45:24.077815+0000","last_became_peered":"2024-09-10T18:45:24.077815+0000","last_unstale":"2024-09-10T18:46:46.394691+0000","last_undegraded":"2024-09-10T18:46:46.394691+0000","last_fullsized":"2024-09-10T18:46:46.394691+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:49.952536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"76'13","reported_seq":61,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.176102+0000","last_change":"2024-09-10T18:46:24.626912+0000","last_active":"2024-09-10T18:46:46.176102+0000","last_peered":"2024-09-10T18:46:46.176102+0000","last_clean":"2024-09-10T18:46:46.176102+0000","last_became_active":"2024-09-10T18:45:20.420421+0000","last_became_peered":"2024-09-10T18:45:20.420421+0000","last_unstale":"2024-09-10T18:46:46.176102+0000","last_undegraded":"2024-09-10T18:46:46.176102+0000","last_fullsized":"2024-09-10T18:46:46.176102+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'13","last_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_deep_scrub":"76'13","last_deep_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_clean_scrub_stamp":"2024-09-10T18:46:24.626859+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:41.087073+0000","scrub_duration":0.0035642949999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":43,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.609711+0000","last_change":"2024-09-10T18:46:31.734522+0000","last_active":"2024-09-10T18:46:46.609711+0000","last_peered":"2024-09-10T18:46:46.609711+0000","last_clean":"2024-09-10T18:46:46.609711+0000","last_became_active":"2024-09-10T18:45:17.965112+0000","last_became_peered":"2024-09-10T18:45:17.965112+0000","last_unstale":"2024-09-10T18:46:46.609711+0000","last_undegraded":"2024-09-10T18:46:46.609711+0000","last_fullsized":"2024-09-10T18:46:46.609711+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:31.734459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:31.734459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:41.550267+0000","scrub_duration":0.0015571960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":39,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.395265+0000","last_change":"2024-09-10T18:46:32.829583+0000","last_active":"2024-09-10T18:46:46.395265+0000","last_peered":"2024-09-10T18:46:46.395265+0000","last_clean":"2024-09-10T18:46:46.395265+0000","last_became_active":"2024-09-10T18:45:22.069040+0000","last_became_peered":"2024-09-10T18:45:22.069040+0000","last_unstale":"2024-09-10T18:46:46.395265+0000","last_undegraded":"2024-09-10T18:46:46.395265+0000","last_fullsized":"2024-09-10T18:46:46.395265+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:32.829508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:32.829508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:45.869787+0000","scrub_duration":0.001897239,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"63'1","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.609745+0000","last_change":"2024-09-10T18:45:24.079638+0000","last_active":"2024-09-10T18:46:46.609745+0000","last_peered":"2024-09-10T18:46:46.609745+0000","last_clean":"2024-09-10T18:46:46.609745+0000","last_became_active":"2024-09-10T18:45:24.079258+0000","last_became_peered":"2024-09-10T18:45:24.079258+0000","last_unstale":"2024-09-10T18:46:46.609745+0000","last_undegraded":"2024-09-10T18:46:46.609745+0000","last_fullsized":"2024-09-10T18:46:46.609745+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:50.668523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":43,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:48.227504+0000","last_change":"2024-09-10T18:46:48.227450+0000","last_active":"2024-09-10T18:46:48.227504+0000","last_peered":"2024-09-10T18:46:48.227504+0000","last_clean":"2024-09-10T18:46:48.227504+0000","last_became_active":"2024-09-10T18:45:17.964705+0000","last_became_peered":"2024-09-10T18:45:17.964705+0000","last_unstale":"2024-09-10T18:46:48.227504+0000","last_undegraded":"2024-09-10T18:46:48.227504+0000","last_fullsized":"2024-09-10T18:46:48.227504+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:48.227378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:48.227378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:54.333207+0000","scrub_duration":0.0015447519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'6","reported_seq":40,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.395301+0000","last_change":"2024-09-10T18:45:20.422485+0000","last_active":"2024-09-10T18:46:46.395301+0000","last_peered":"2024-09-10T18:46:46.395301+0000","last_clean":"2024-09-10T18:46:46.395301+0000","last_became_active":"2024-09-10T18:45:20.421746+0000","last_became_peered":"2024-09-10T18:45:20.421746+0000","last_unstale":"2024-09-10T18:46:46.395301+0000","last_undegraded":"2024-09-10T18:46:46.395301+0000","last_fullsized":"2024-09-10T18:46:46.395301+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":39,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.233024+0000","last_change":"2024-09-10T18:46:39.084489+0000","last_active":"2024-09-10T18:46:46.233024+0000","last_peered":"2024-09-10T18:46:46.233024+0000","last_clean":"2024-09-10T18:46:46.233024+0000","last_became_active":"2024-09-10T18:45:22.069653+0000","last_became_peered":"2024-09-10T18:45:22.069653+0000","last_unstale":"2024-09-10T18:46:46.233024+0000","last_undegraded":"2024-09-10T18:46:46.233024+0000","last_fullsized":"2024-09-10T18:46:46.233024+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:39.084433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:39.084433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:05.431757+0000","scrub_duration":0.0014845819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.606880+0000","last_change":"2024-09-10T18:45:24.264869+0000","last_active":"2024-09-10T18:46:46.606880+0000","last_peered":"2024-09-10T18:46:46.606880+0000","last_clean":"2024-09-10T18:46:46.606880+0000","last_became_active":"2024-09-10T18:45:24.264611+0000","last_became_peered":"2024-09-10T18:45:24.264611+0000","last_unstale":"2024-09-10T18:46:46.606880+0000","last_undegraded":"2024-09-10T18:46:46.606880+0000","last_fullsized":"2024-09-10T18:46:46.606880+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"63'10","reported_seq":56,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T18:46:50.722424+0000","last_change":"2024-09-10T18:46:50.722392+0000","last_active":"2024-09-10T18:46:50.722424+0000","last_peered":"2024-09-10T18:46:50.722424+0000","last_clean":"2024-09-10T18:46:50.722424+0000","last_became_active":"2024-09-10T18:45:20.419887+0000","last_became_peered":"2024-09-10T18:45:20.419887+0000","last_unstale":"2024-09-10T18:46:50.722424+0000","last_undegraded":"2024-09-10T18:46:50.722424+0000","last_fullsized":"2024-09-10T18:46:50.722424+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:46:50.722316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.722316+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:52.342944+0000","scrub_duration":0.0022194070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":43,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T18:46:50.093463+0000","last_change":"2024-09-10T18:46:50.093435+0000","last_active":"2024-09-10T18:46:50.093463+0000","last_peered":"2024-09-10T18:46:50.093463+0000","last_clean":"2024-09-10T18:46:50.093463+0000","last_became_active":"2024-09-10T18:45:17.964870+0000","last_became_peered":"2024-09-10T18:45:17.964870+0000","last_unstale":"2024-09-10T18:46:50.093463+0000","last_undegraded":"2024-09-10T18:46:50.093463+0000","last_fullsized":"2024-09-10T18:46:50.093463+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:50.093372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.093372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:55.991532+0000","scrub_duration":0.0015580539999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"63'8","reported_seq":65,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.232999+0000","last_change":"2024-09-10T18:46:34.085224+0000","last_active":"2024-09-10T18:46:46.232999+0000","last_peered":"2024-09-10T18:46:46.232999+0000","last_clean":"2024-09-10T18:46:46.232999+0000","last_became_active":"2024-09-10T18:45:22.062940+0000","last_became_peered":"2024-09-10T18:45:22.062940+0000","last_unstale":"2024-09-10T18:46:46.232999+0000","last_undegraded":"2024-09-10T18:46:46.232999+0000","last_fullsized":"2024-09-10T18:46:46.232999+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:34.085156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:34.085156+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:54.777061+0000","scrub_duration":0.0020843120000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.175791+0000","last_change":"2024-09-10T18:45:24.074039+0000","last_active":"2024-09-10T18:46:46.175791+0000","last_peered":"2024-09-10T18:46:46.175791+0000","last_clean":"2024-09-10T18:46:46.175791+0000","last_became_active":"2024-09-10T18:45:24.073736+0000","last_became_peered":"2024-09-10T18:45:24.073736+0000","last_unstale":"2024-09-10T18:46:46.175791+0000","last_undegraded":"2024-09-10T18:46:46.175791+0000","last_fullsized":"2024-09-10T18:46:46.175791+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"63'9","reported_seq":57,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.233044+0000","last_change":"2024-09-10T18:46:23.086274+0000","last_active":"2024-09-10T18:46:46.233044+0000","last_peered":"2024-09-10T18:46:46.233044+0000","last_clean":"2024-09-10T18:46:46.233044+0000","last_became_active":"2024-09-10T18:45:20.421124+0000","last_became_peered":"2024-09-10T18:45:20.421124+0000","last_unstale":"2024-09-10T18:46:46.233044+0000","last_undegraded":"2024-09-10T18:46:46.233044+0000","last_fullsized":"2024-09-10T18:46:46.233044+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:46:23.086203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:23.086203+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:44.793257+0000","scrub_duration":0.002168068,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":43,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.176513+0000","last_change":"2024-09-10T18:46:45.541882+0000","last_active":"2024-09-10T18:46:46.176513+0000","last_peered":"2024-09-10T18:46:46.176513+0000","last_clean":"2024-09-10T18:46:46.176513+0000","last_became_active":"2024-09-10T18:45:17.960106+0000","last_became_peered":"2024-09-10T18:45:17.960106+0000","last_unstale":"2024-09-10T18:46:46.176513+0000","last_undegraded":"2024-09-10T18:46:46.176513+0000","last_fullsized":"2024-09-10T18:46:46.176513+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:45.541819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:45.541819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:15.015402+0000","scrub_duration":0.0015180809999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"63'8","reported_seq":55,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.394845+0000","last_change":"2024-09-10T18:45:22.075677+0000","last_active":"2024-09-10T18:46:46.394845+0000","last_peered":"2024-09-10T18:46:46.394845+0000","last_clean":"2024-09-10T18:46:46.394845+0000","last_became_active":"2024-09-10T18:45:22.075368+0000","last_became_peered":"2024-09-10T18:45:22.075368+0000","last_unstale":"2024-09-10T18:46:46.394845+0000","last_undegraded":"2024-09-10T18:46:46.394845+0000","last_fullsized":"2024-09-10T18:46:46.394845+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":37,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.175404+0000","last_change":"2024-09-10T18:46:36.305262+0000","last_active":"2024-09-10T18:46:46.175404+0000","last_peered":"2024-09-10T18:46:46.175404+0000","last_clean":"2024-09-10T18:46:46.175404+0000","last_became_active":"2024-09-10T18:45:24.080196+0000","last_became_peered":"2024-09-10T18:45:24.080196+0000","last_unstale":"2024-09-10T18:46:46.175404+0000","last_undegraded":"2024-09-10T18:46:46.175404+0000","last_fullsized":"2024-09-10T18:46:46.175404+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_clean_scrub_stamp":"2024-09-10T18:46:36.305199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:04.747507+0000","scrub_duration":0.0013794809999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"63'9","reported_seq":47,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.175330+0000","last_change":"2024-09-10T18:45:20.057767+0000","last_active":"2024-09-10T18:46:46.175330+0000","last_peered":"2024-09-10T18:46:46.175330+0000","last_clean":"2024-09-10T18:46:46.175330+0000","last_became_active":"2024-09-10T18:45:20.057479+0000","last_became_peered":"2024-09-10T18:45:20.057479+0000","last_unstale":"2024-09-10T18:46:46.175330+0000","last_undegraded":"2024-09-10T18:46:46.175330+0000","last_fullsized":"2024-09-10T18:46:46.175330+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":43,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.230643+0000","last_change":"2024-09-10T18:46:23.292276+0000","last_active":"2024-09-10T18:46:46.230643+0000","last_peered":"2024-09-10T18:46:46.230643+0000","last_clean":"2024-09-10T18:46:46.230643+0000","last_became_active":"2024-09-10T18:45:17.962009+0000","last_became_peered":"2024-09-10T18:45:17.962009+0000","last_unstale":"2024-09-10T18:46:46.230643+0000","last_undegraded":"2024-09-10T18:46:46.230643+0000","last_fullsized":"2024-09-10T18:46:46.230643+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:23.292207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:23.292207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:27.416585+0000","scrub_duration":0.001659953,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.394680+0000","last_change":"2024-09-10T18:45:22.073186+0000","last_active":"2024-09-10T18:46:46.394680+0000","last_peered":"2024-09-10T18:46:46.394680+0000","last_clean":"2024-09-10T18:46:46.394680+0000","last_became_active":"2024-09-10T18:45:22.072877+0000","last_became_peered":"2024-09-10T18:45:22.072877+0000","last_unstale":"2024-09-10T18:46:46.394680+0000","last_undegraded":"2024-09-10T18:46:46.394680+0000","last_fullsized":"2024-09-10T18:46:46.394680+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T18:46:50.499906+0000","last_change":"2024-09-10T18:46:50.499866+0000","last_active":"2024-09-10T18:46:50.499906+0000","last_peered":"2024-09-10T18:46:50.499906+0000","last_clean":"2024-09-10T18:46:50.499906+0000","last_became_active":"2024-09-10T18:45:24.081050+0000","last_became_peered":"2024-09-10T18:45:24.081050+0000","last_unstale":"2024-09-10T18:46:50.499906+0000","last_undegraded":"2024-09-10T18:46:50.499906+0000","last_fullsized":"2024-09-10T18:46:50.499906+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:50.499815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.499815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:13.479025+0000","scrub_duration":0.001285432,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"63'11","reported_seq":50,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.175119+0000","last_change":"2024-09-10T18:45:20.057563+0000","last_active":"2024-09-10T18:46:46.175119+0000","last_peered":"2024-09-10T18:46:46.175119+0000","last_clean":"2024-09-10T18:46:46.175119+0000","last_became_active":"2024-09-10T18:45:20.056885+0000","last_became_peered":"2024-09-10T18:45:20.056885+0000","last_unstale":"2024-09-10T18:46:46.175119+0000","last_undegraded":"2024-09-10T18:46:46.175119+0000","last_fullsized":"2024-09-10T18:46:46.175119+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":43,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.176729+0000","last_change":"2024-09-10T18:46:36.402610+0000","last_active":"2024-09-10T18:46:46.176729+0000","last_peered":"2024-09-10T18:46:46.176729+0000","last_clean":"2024-09-10T18:46:46.176729+0000","last_became_active":"2024-09-10T18:45:17.963542+0000","last_became_peered":"2024-09-10T18:45:17.963542+0000","last_unstale":"2024-09-10T18:46:46.176729+0000","last_undegraded":"2024-09-10T18:46:46.176729+0000","last_fullsized":"2024-09-10T18:46:46.176729+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:36.402548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:36.402548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:04.495178+0000","scrub_duration":0.0014103919999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":39,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.175179+0000","last_change":"2024-09-10T18:46:37.414794+0000","last_active":"2024-09-10T18:46:46.175179+0000","last_peered":"2024-09-10T18:46:46.175179+0000","last_clean":"2024-09-10T18:46:46.175179+0000","last_became_active":"2024-09-10T18:45:22.070065+0000","last_became_peered":"2024-09-10T18:45:22.070065+0000","last_unstale":"2024-09-10T18:46:46.175179+0000","last_undegraded":"2024-09-10T18:46:46.175179+0000","last_fullsized":"2024-09-10T18:46:46.175179+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:37.414734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:37.414734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:44.027335+0000","scrub_duration":0.001607996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.394412+0000","last_change":"2024-09-10T18:45:24.072477+0000","last_active":"2024-09-10T18:46:46.394412+0000","last_peered":"2024-09-10T18:46:46.394412+0000","last_clean":"2024-09-10T18:46:46.394412+0000","last_became_active":"2024-09-10T18:45:24.072224+0000","last_became_peered":"2024-09-10T18:45:24.072224+0000","last_unstale":"2024-09-10T18:46:46.394412+0000","last_undegraded":"2024-09-10T18:46:46.394412+0000","last_fullsized":"2024-09-10T18:46:46.394412+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"63'4","reported_seq":47,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.767013+0000","last_change":"2024-09-10T18:46:46.766906+0000","last_active":"2024-09-10T18:46:46.767013+0000","last_peered":"2024-09-10T18:46:46.767013+0000","last_clean":"2024-09-10T18:46:46.767013+0000","last_became_active":"2024-09-10T18:45:20.059205+0000","last_became_peered":"2024-09-10T18:45:20.059205+0000","last_unstale":"2024-09-10T18:46:46.767013+0000","last_undegraded":"2024-09-10T18:46:46.767013+0000","last_fullsized":"2024-09-10T18:46:46.767013+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4","last_scrub_stamp":"2024-09-10T18:46:46.766766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:46.766766+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:49.051695+0000","scrub_duration":0.0021847250000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.233109+0000","last_change":"2024-09-10T18:45:17.963898+0000","last_active":"2024-09-10T18:46:46.233109+0000","last_peered":"2024-09-10T18:46:46.233109+0000","last_clean":"2024-09-10T18:46:46.233109+0000","last_became_active":"2024-09-10T18:45:17.963630+0000","last_became_peered":"2024-09-10T18:45:17.963630+0000","last_unstale":"2024-09-10T18:46:46.233109+0000","last_undegraded":"2024-09-10T18:46:46.233109+0000","last_fullsized":"2024-09-10T18:46:46.233109+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":39,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.230450+0000","last_change":"2024-09-10T18:46:35.172936+0000","last_active":"2024-09-10T18:46:46.230450+0000","last_peered":"2024-09-10T18:46:46.230450+0000","last_clean":"2024-09-10T18:46:46.230450+0000","last_became_active":"2024-09-10T18:45:22.072377+0000","last_became_peered":"2024-09-10T18:45:22.072377+0000","last_unstale":"2024-09-10T18:46:46.230450+0000","last_undegraded":"2024-09-10T18:46:46.230450+0000","last_fullsized":"2024-09-10T18:46:46.230450+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_clean_scrub_stamp":"2024-09-10T18:46:35.172861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:57.638389+0000","scrub_duration":0.001552933,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.605974+0000","last_change":"2024-09-10T18:45:24.077706+0000","last_active":"2024-09-10T18:46:46.605974+0000","last_peered":"2024-09-10T18:46:46.605974+0000","last_clean":"2024-09-10T18:46:46.605974+0000","last_became_active":"2024-09-10T18:45:24.077398+0000","last_became_peered":"2024-09-10T18:45:24.077398+0000","last_unstale":"2024-09-10T18:46:46.605974+0000","last_undegraded":"2024-09-10T18:46:46.605974+0000","last_fullsized":"2024-09-10T18:46:46.605974+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"79'16","reported_seq":65,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:48.744382+0000","last_change":"2024-09-10T18:46:48.744340+0000","last_active":"2024-09-10T18:46:48.744382+0000","last_peered":"2024-09-10T18:46:48.744382+0000","last_clean":"2024-09-10T18:46:48.744382+0000","last_became_active":"2024-09-10T18:45:20.421479+0000","last_became_peered":"2024-09-10T18:45:20.421479+0000","last_unstale":"2024-09-10T18:46:48.744382+0000","last_undegraded":"2024-09-10T18:46:48.744382+0000","last_fullsized":"2024-09-10T18:46:48.744382+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"79'16","last_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_deep_scrub":"79'16","last_deep_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_clean_scrub_stamp":"2024-09-10T18:46:48.744287+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:57.831206+0000","scrub_duration":0.0044879259999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"56'1","reported_seq":50,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.231158+0000","last_change":"2024-09-10T18:46:41.147533+0000","last_active":"2024-09-10T18:46:46.231158+0000","last_peered":"2024-09-10T18:46:46.231158+0000","last_clean":"2024-09-10T18:46:46.231158+0000","last_became_active":"2024-09-10T18:45:17.961222+0000","last_became_peered":"2024-09-10T18:45:17.961222+0000","last_unstale":"2024-09-10T18:46:46.231158+0000","last_undegraded":"2024-09-10T18:46:46.231158+0000","last_fullsized":"2024-09-10T18:46:46.231158+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:46:41.147463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:41.147463+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:58.016926+0000","scrub_duration":0.0016797889999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":39,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.605857+0000","last_change":"2024-09-10T18:46:27.667088+0000","last_active":"2024-09-10T18:46:46.605857+0000","last_peered":"2024-09-10T18:46:46.605857+0000","last_clean":"2024-09-10T18:46:46.605857+0000","last_became_active":"2024-09-10T18:45:22.072312+0000","last_became_peered":"2024-09-10T18:45:22.072312+0000","last_unstale":"2024-09-10T18:46:46.605857+0000","last_undegraded":"2024-09-10T18:46:46.605857+0000","last_fullsized":"2024-09-10T18:46:46.605857+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:27.667018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:27.667018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:38.065664+0000","scrub_duration":0.001388886,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":37,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.394594+0000","last_change":"2024-09-10T18:46:29.791617+0000","last_active":"2024-09-10T18:46:46.394594+0000","last_peered":"2024-09-10T18:46:46.394594+0000","last_clean":"2024-09-10T18:46:46.394594+0000","last_became_active":"2024-09-10T18:45:24.264818+0000","last_became_peered":"2024-09-10T18:45:24.264818+0000","last_unstale":"2024-09-10T18:46:46.394594+0000","last_undegraded":"2024-09-10T18:46:46.394594+0000","last_fullsized":"2024-09-10T18:46:46.394594+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:29.791563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:29.791563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:37.941643+0000","scrub_duration":0.0021031819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"78'2","reported_seq":39,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.606815+0000","last_change":"2024-09-10T18:46:37.581201+0000","last_active":"2024-09-10T18:46:46.606815+0000","last_peered":"2024-09-10T18:46:46.606815+0000","last_clean":"2024-09-10T18:46:46.606815+0000","last_became_active":"2024-09-10T18:45:24.076380+0000","last_became_peered":"2024-09-10T18:45:24.076380+0000","last_unstale":"2024-09-10T18:46:46.606815+0000","last_undegraded":"2024-09-10T18:46:46.606815+0000","last_fullsized":"2024-09-10T18:46:46.606815+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'2","last_scrub_stamp":"2024-09-10T18:46:37.581130+0000","last_deep_scrub":"78'2","last_deep_scrub_stamp":"2024-09-10T18:46:37.581130+0000","last_clean_scrub_stamp":"2024-09-10T18:46:37.581130+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:55.887929+0000","scrub_duration":0.0024563609999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":43,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.232409+0000","last_change":"2024-09-10T18:46:28.088657+0000","last_active":"2024-09-10T18:46:46.232409+0000","last_peered":"2024-09-10T18:46:46.232409+0000","last_clean":"2024-09-10T18:46:46.232409+0000","last_became_active":"2024-09-10T18:45:17.962108+0000","last_became_peered":"2024-09-10T18:45:17.962108+0000","last_unstale":"2024-09-10T18:46:46.232409+0000","last_undegraded":"2024-09-10T18:46:46.232409+0000","last_fullsized":"2024-09-10T18:46:46.232409+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:28.088588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:28.088588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:56.550863+0000","scrub_duration":0.00140508,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"63'10","reported_seq":56,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T18:46:49.494656+0000","last_change":"2024-09-10T18:46:49.494628+0000","last_active":"2024-09-10T18:46:49.494656+0000","last_peered":"2024-09-10T18:46:49.494656+0000","last_clean":"2024-09-10T18:46:49.494656+0000","last_became_active":"2024-09-10T18:45:20.421094+0000","last_became_peered":"2024-09-10T18:45:20.421094+0000","last_unstale":"2024-09-10T18:46:49.494656+0000","last_undegraded":"2024-09-10T18:46:49.494656+0000","last_fullsized":"2024-09-10T18:46:49.494656+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:46:49.494565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:49.494565+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:08.379972+0000","scrub_duration":0.0036379630000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":68,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.230798+0000","last_change":"2024-09-10T18:46:33.150919+0000","last_active":"2024-09-10T18:46:46.230798+0000","last_peered":"2024-09-10T18:46:46.230798+0000","last_clean":"2024-09-10T18:46:46.230798+0000","last_became_active":"2024-09-10T18:45:22.073281+0000","last_became_peered":"2024-09-10T18:45:22.073281+0000","last_unstale":"2024-09-10T18:46:46.230798+0000","last_undegraded":"2024-09-10T18:46:46.230798+0000","last_fullsized":"2024-09-10T18:46:46.230798+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:33.150852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:33.150852+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:51.299257+0000","scrub_duration":0.0017025339999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.176429+0000","last_change":"2024-09-10T18:45:24.082638+0000","last_active":"2024-09-10T18:46:46.176429+0000","last_peered":"2024-09-10T18:46:46.176429+0000","last_clean":"2024-09-10T18:46:46.176429+0000","last_became_active":"2024-09-10T18:45:24.082318+0000","last_became_peered":"2024-09-10T18:45:24.082318+0000","last_unstale":"2024-09-10T18:46:46.176429+0000","last_undegraded":"2024-09-10T18:46:46.176429+0000","last_fullsized":"2024-09-10T18:46:46.176429+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":43,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.394881+0000","last_change":"2024-09-10T18:46:44.694805+0000","last_active":"2024-09-10T18:46:46.394881+0000","last_peered":"2024-09-10T18:46:46.394881+0000","last_clean":"2024-09-10T18:46:46.394881+0000","last_became_active":"2024-09-10T18:45:17.965707+0000","last_became_peered":"2024-09-10T18:45:17.965707+0000","last_unstale":"2024-09-10T18:46:46.394881+0000","last_undegraded":"2024-09-10T18:46:46.394881+0000","last_fullsized":"2024-09-10T18:46:46.394881+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:44.694737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:44.694737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:03.417079+0000","scrub_duration":0.0016330400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"63'11","reported_seq":60,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.231162+0000","last_change":"2024-09-10T18:46:46.230915+0000","last_active":"2024-09-10T18:46:46.231162+0000","last_peered":"2024-09-10T18:46:46.231162+0000","last_clean":"2024-09-10T18:46:46.231162+0000","last_became_active":"2024-09-10T18:45:20.058717+0000","last_became_peered":"2024-09-10T18:45:20.058717+0000","last_unstale":"2024-09-10T18:46:46.231162+0000","last_undegraded":"2024-09-10T18:46:46.231162+0000","last_fullsized":"2024-09-10T18:46:46.231162+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:46:46.230819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:46.230819+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:54.186112+0000","scrub_duration":0.002545322,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.175931+0000","last_change":"2024-09-10T18:45:22.068991+0000","last_active":"2024-09-10T18:46:46.175931+0000","last_peered":"2024-09-10T18:46:46.175931+0000","last_clean":"2024-09-10T18:46:46.175931+0000","last_became_active":"2024-09-10T18:45:22.068676+0000","last_became_peered":"2024-09-10T18:45:22.068676+0000","last_unstale":"2024-09-10T18:46:46.175931+0000","last_undegraded":"2024-09-10T18:46:46.175931+0000","last_fullsized":"2024-09-10T18:46:46.175931+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":37,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.175380+0000","last_change":"2024-09-10T18:46:45.285074+0000","last_active":"2024-09-10T18:46:46.175380+0000","last_peered":"2024-09-10T18:46:46.175380+0000","last_clean":"2024-09-10T18:46:46.175380+0000","last_became_active":"2024-09-10T18:45:24.267251+0000","last_became_peered":"2024-09-10T18:45:24.267251+0000","last_unstale":"2024-09-10T18:46:46.175380+0000","last_undegraded":"2024-09-10T18:46:46.175380+0000","last_fullsized":"2024-09-10T18:46:46.175380+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:45.285015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:45.285015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:56.843562+0000","scrub_duration":0.001362142,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"56'1","reported_seq":52,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.230612+0000","last_change":"2024-09-10T18:46:42.139314+0000","last_active":"2024-09-10T18:46:46.230612+0000","last_peered":"2024-09-10T18:46:46.230612+0000","last_clean":"2024-09-10T18:46:46.230612+0000","last_became_active":"2024-09-10T18:45:17.966286+0000","last_became_peered":"2024-09-10T18:45:17.966286+0000","last_unstale":"2024-09-10T18:46:46.230612+0000","last_undegraded":"2024-09-10T18:46:46.230612+0000","last_fullsized":"2024-09-10T18:46:46.230612+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:46:42.139256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:42.139256+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:57.842540+0000","scrub_duration":0.0014138690000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"63'15","reported_seq":66,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.609853+0000","last_change":"2024-09-10T18:46:23.864133+0000","last_active":"2024-09-10T18:46:46.609853+0000","last_peered":"2024-09-10T18:46:46.609853+0000","last_clean":"2024-09-10T18:46:46.609853+0000","last_became_active":"2024-09-10T18:45:20.062728+0000","last_became_peered":"2024-09-10T18:45:20.062728+0000","last_unstale":"2024-09-10T18:46:46.609853+0000","last_undegraded":"2024-09-10T18:46:46.609853+0000","last_fullsized":"2024-09-10T18:46:46.609853+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:46:23.864075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:23.864075+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:29.106766+0000","scrub_duration":0.0020859780000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.176591+0000","last_change":"2024-09-10T18:45:22.060202+0000","last_active":"2024-09-10T18:46:46.176591+0000","last_peered":"2024-09-10T18:46:46.176591+0000","last_clean":"2024-09-10T18:46:46.176591+0000","last_became_active":"2024-09-10T18:45:22.059794+0000","last_became_peered":"2024-09-10T18:45:22.059794+0000","last_unstale":"2024-09-10T18:46:46.176591+0000","last_undegraded":"2024-09-10T18:46:46.176591+0000","last_fullsized":"2024-09-10T18:46:46.176591+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.395202+0000","last_change":"2024-09-10T18:45:24.264775+0000","last_active":"2024-09-10T18:46:46.395202+0000","last_peered":"2024-09-10T18:46:46.395202+0000","last_clean":"2024-09-10T18:46:46.395202+0000","last_became_active":"2024-09-10T18:45:24.264427+0000","last_became_peered":"2024-09-10T18:45:24.264427+0000","last_unstale":"2024-09-10T18:46:46.395202+0000","last_undegraded":"2024-09-10T18:46:46.395202+0000","last_fullsized":"2024-09-10T18:46:46.395202+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":43,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:47.734895+0000","last_change":"2024-09-10T18:46:47.734860+0000","last_active":"2024-09-10T18:46:47.734895+0000","last_peered":"2024-09-10T18:46:47.734895+0000","last_clean":"2024-09-10T18:46:47.734895+0000","last_became_active":"2024-09-10T18:45:17.963257+0000","last_became_peered":"2024-09-10T18:45:17.963257+0000","last_unstale":"2024-09-10T18:46:47.734895+0000","last_undegraded":"2024-09-10T18:46:47.734895+0000","last_fullsized":"2024-09-10T18:46:47.734895+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:47.734803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:47.734803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:05.847815+0000","scrub_duration":0.001381134,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"63'12","reported_seq":54,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.395028+0000","last_change":"2024-09-10T18:45:20.058693+0000","last_active":"2024-09-10T18:46:46.395028+0000","last_peered":"2024-09-10T18:46:46.395028+0000","last_clean":"2024-09-10T18:46:46.395028+0000","last_became_active":"2024-09-10T18:45:20.058391+0000","last_became_peered":"2024-09-10T18:45:20.058391+0000","last_unstale":"2024-09-10T18:46:46.395028+0000","last_undegraded":"2024-09-10T18:46:46.395028+0000","last_fullsized":"2024-09-10T18:46:46.395028+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":39,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-10T18:46:46.175129+0000","last_change":"2024-09-10T18:46:34.283979+0000","last_active":"2024-09-10T18:46:46.175129+0000","last_peered":"2024-09-10T18:46:46.175129+0000","last_clean":"2024-09-10T18:46:46.175129+0000","last_became_active":"2024-09-10T18:45:22.073250+0000","last_became_peered":"2024-09-10T18:45:22.073250+0000","last_unstale":"2024-09-10T18:46:46.175129+0000","last_undegraded":"2024-09-10T18:46:46.175129+0000","last_fullsized":"2024-09-10T18:46:46.175129+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:34.283901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:34.283901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:34.686745+0000","scrub_duration":0.002360334,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":408,"ondisk_log_size":408,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":85,"num_read_kb":80,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299415,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298536,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446616,"statfs":{"total":95995035648,"available":95689334784,"internally_reserved":0,"allocated":1208320,"data_stored":869048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.499},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.61599999999999999}]},{"osd":1,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.64200000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.55200000000000005}]},{"osd":3,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.629}]},{"osd":4,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.59899999999999998}]},{"osd":5,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.377}]},{"osd":6,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.40699999999999997}]}]},{"osd":6,"up_from":43,"seq":184683593756,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298524,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446628,"statfs":{"total":95995035648,"available":95689347072,"internally_reserved":0,"allocated":1196032,"data_stored":869124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.46300000000000002,"15min":0.45900000000000002},"min":{"1min":0.32900000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.496,"15min":0.49199999999999999},"min":{"1min":0.32800000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.443}]},{"osd":1,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.315,"5min":0.249,"15min":0.249},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.318},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.46700000000000003,"15min":0.46300000000000002},"min":{"1min":0.33700000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.65900000000000003,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.496}]},{"osd":2,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.48299999999999998,"15min":0.48099999999999998},"min":{"1min":0.34000000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.371},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.49299999999999999,"15min":0.49099999999999999},"min":{"1min":0.33600000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68700000000000006,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.40799999999999997}]},{"osd":3,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.48399999999999999,"15min":0.48299999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.51200000000000001,"15min":0.51000000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.41799999999999998}]},{"osd":4,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.39700000000000002,"15min":0.38800000000000001},"min":{"1min":0.26900000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.60099999999999998}]},{"osd":5,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.41899999999999998,"15min":0.41499999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.42499999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.38700000000000001}]},{"osd":7,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.43099999999999999}]}]},{"osd":5,"up_from":37,"seq":158913789984,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297932,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447220,"statfs":{"total":95995035648,"available":95689953280,"internally_reserved":0,"allocated":589824,"data_stored":277259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.52200000000000002,"15min":0.52600000000000002},"min":{"1min":0.30299999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70099999999999996,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.624},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.58299999999999996,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.40799999999999997}]},{"osd":1,"last update":"Tue Sep 10 18:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.55200000000000005,"15min":0.56000000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.69699999999999995,"5min":0.87,"15min":0.87},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.497,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.64200000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.501}]},{"osd":2,"last update":"Tue Sep 10 18:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.51800000000000002,"15min":0.52000000000000002},"min":{"1min":0.33300000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.71299999999999997,"5min":0.755,"15min":0.755},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53600000000000003,"15min":0.54100000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79600000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.46600000000000003}]},{"osd":3,"last update":"Tue Sep 10 18:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52500000000000002,"15min":0.52700000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76100000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.54200000000000004,"15min":0.54600000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80100000000000005,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.41799999999999998}]},{"osd":4,"last update":"Tue Sep 10 18:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.64500000000000002,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.40500000000000003,"15min":0.40300000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.65700000000000003,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.30399999999999999}]},{"osd":6,"last update":"Tue Sep 10 18:46:44 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.42199999999999999,"15min":0.41599999999999998},"min":{"1min":0.29299999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.67500000000000004,"5min":0.753,"15min":0.753},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.46500000000000002,"15min":0.46100000000000002},"min":{"1min":0.36199999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.71199999999999997,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.42999999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.36599999999999999}]}]},{"osd":4,"up_from":30,"seq":128849018918,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297956,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447196,"statfs":{"total":95995035648,"available":95689928704,"internally_reserved":0,"allocated":614400,"data_stored":278216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.36399999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.54000000000000004,"15min":0.54200000000000004},"min":{"1min":0.34399999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.46000000000000002}]},{"osd":1,"last update":"Tue Sep 10 18:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.57999999999999996,"15min":0.57599999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.609,"5min":2.609,"15min":2.609},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.54700000000000004,"15min":0.53200000000000003},"min":{"1min":0.39700000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.589,"5min":2.589,"15min":2.589},"last":0.56499999999999995}]},{"osd":2,"last update":"Tue Sep 10 18:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.55700000000000005,"15min":0.54400000000000004},"min":{"1min":0.40500000000000003,"5min":0.379,"15min":0.379},"max":{"1min":2.6030000000000002,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.54800000000000004,"15min":0.55400000000000005},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64000000000000001,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.48699999999999999}]},{"osd":3,"last update":"Tue Sep 10 18:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52800000000000002,"15min":0.53000000000000003},"min":{"1min":0.36699999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68300000000000005,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.497,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.55700000000000005}]},{"osd":5,"last update":"Tue Sep 10 18:46:22 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.35899999999999999,"15min":0.34999999999999998},"min":{"1min":0.26800000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.44,"5min":0.371,"15min":0.35899999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.32500000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.42999999999999999,"15min":0.42399999999999999},"min":{"1min":0.311,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.68799999999999994,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.40100000000000002,"15min":0.39100000000000001},"min":{"1min":0.34799999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.38500000000000001}]},{"osd":7,"last update":"Tue Sep 10 18:46:08 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.435},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.36599999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182443,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297968,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447184,"statfs":{"total":95995035648,"available":95689916416,"internally_reserved":0,"allocated":626688,"data_stored":277995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:35 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.378,"15min":0.38400000000000001},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.40100000000000002,"15min":0.40300000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.223}]},{"osd":1,"last update":"Tue Sep 10 18:46:35 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.40400000000000003,"15min":0.40899999999999997},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40899999999999997,"15min":0.41499999999999998},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.63800000000000001,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.249}]},{"osd":2,"last update":"Tue Sep 10 18:46:35 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.43099999999999999,"15min":0.438},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.377,"5min":0.41999999999999998,"15min":0.433},"min":{"1min":0.27200000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.53600000000000003,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.38}]},{"osd":4,"last update":"Tue Sep 10 18:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54000000000000004,"15min":0.54300000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.67200000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.56599999999999995,"15min":0.57199999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.71699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.47099999999999997}]},{"osd":5,"last update":"Tue Sep 10 18:46:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39700000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.70299999999999996,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.435},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.41399999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80800000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.373}]},{"osd":6,"last update":"Tue Sep 10 18:46:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.56000000000000005,"15min":0.56299999999999994},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79100000000000004,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55400000000000005,"15min":0.55600000000000005},"min":{"1min":0.40799999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.53700000000000003}]},{"osd":7,"last update":"Tue Sep 10 18:46:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.38900000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411375,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297936,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447216,"statfs":{"total":95995035648,"available":95689949184,"internally_reserved":0,"allocated":593920,"data_stored":277694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.376,"15min":0.35899999999999999},"min":{"1min":0.246,"5min":0.223,"15min":0.223},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.38300000000000001,"15min":0.371},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.38600000000000001}]},{"osd":1,"last update":"Tue Sep 10 18:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.375,"15min":0.36699999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.42499999999999999,"15min":0.41299999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.29799999999999999}]},{"osd":3,"last update":"Tue Sep 10 18:46:36 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46800000000000003,"15min":0.46500000000000002},"min":{"1min":0.30099999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.42899999999999999,"15min":0.43099999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.35799999999999998}]},{"osd":4,"last update":"Tue Sep 10 18:45:59 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54900000000000004,"15min":0.54800000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.60199999999999998}]},{"osd":5,"last update":"Tue Sep 10 18:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56100000000000005,"15min":0.55800000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.42899999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.625}]},{"osd":6,"last update":"Tue Sep 10 18:46:49 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58999999999999997,"15min":0.58599999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.46200000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.61299999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:46:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.55200000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297980,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447172,"statfs":{"total":95995035648,"available":95689904128,"internally_reserved":0,"allocated":638976,"data_stored":278490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:48 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34399999999999997,"15min":0.32000000000000001},"min":{"1min":0.245,"5min":0.224,"15min":0.224},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.254},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.36599999999999999,"15min":0.33600000000000002},"min":{"1min":0.252,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.53700000000000003,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.35799999999999998}]},{"osd":2,"last update":"Tue Sep 10 18:46:11 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.372,"15min":0.36099999999999999},"min":{"1min":0.27900000000000003,"5min":0.248,"15min":0.248},"max":{"1min":0.47199999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.39000000000000001,"15min":0.38100000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.49099999999999999,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.28699999999999998}]},{"osd":3,"last update":"Tue Sep 10 18:46:34 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.45400000000000001,"15min":0.45800000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.42299999999999999,"15min":0.42199999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.55400000000000005,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.30499999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:45:56 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.56299999999999994,"15min":0.55600000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.54100000000000004,"15min":0.53300000000000003},"min":{"1min":0.41099999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.33600000000000002}]},{"osd":5,"last update":"Tue Sep 10 18:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60299999999999998,"15min":0.60399999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.37},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63700000000000001,"15min":0.64100000000000001},"min":{"1min":0.44600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93300000000000005,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.55600000000000005}]},{"osd":6,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.46300000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.66400000000000003,"15min":0.67100000000000004},"min":{"1min":0.41999999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82299999999999995,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.53200000000000003}]},{"osd":7,"last update":"Tue Sep 10 18:46:11 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.57699999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738425,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298528,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446624,"statfs":{"total":95995035648,"available":95689342976,"internally_reserved":0,"allocated":1200128,"data_stored":867858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 18:45:50 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.32100000000000001,"15min":0.30199999999999999},"min":{"1min":0.224,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.33100000000000002,"15min":0.313},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.38700000000000001}]},{"osd":2,"last update":"Tue Sep 10 18:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.40300000000000002,"15min":0.38900000000000001},"min":{"1min":0.30299999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.376,"15min":0.36199999999999999},"min":{"1min":0.26600000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.26700000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:46:31 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.39500000000000002,"15min":0.39800000000000002},"min":{"1min":0.26100000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.66100000000000003,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.372},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.41199999999999998,"15min":0.41299999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.30599999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.56999999999999995,"15min":0.53600000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":4.6189999999999998,"5min":4.6189999999999998,"15min":4.6189999999999998},"last":0.42299999999999999}]},{"osd":5,"last update":"Tue Sep 10 18:46:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.42999999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.434},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.58699999999999997,"15min":0.55800000000000005},"min":{"1min":0.42699999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":4.6429999999999998,"5min":4.6429999999999998,"15min":4.6429999999999998},"last":0.47699999999999998}]},{"osd":6,"last update":"Tue Sep 10 18:46:42 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56100000000000005,"15min":0.55500000000000005},"min":{"1min":0.42099999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.746,"15min":0.77300000000000002},"min":{"1min":0.40899999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.878,"5min":4.6630000000000003,"15min":4.6630000000000003},"last":0.46400000000000002}]},{"osd":7,"last update":"Tue Sep 10 18:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.48699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T18:46:55.126 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T18:46:55.228 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-mon[33163]: 2.16 scrub starts 2024-09-10T18:46:55.228 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-mon[33163]: 2.16 scrub ok 2024-09-10T18:46:55.228 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-mon[33163]: 2.0 scrub starts 2024-09-10T18:46:55.228 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-mon[33163]: 2.0 scrub ok 2024-09-10T18:46:55.228 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:55.228 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:55.228 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:55.229 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:55.229 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-mon[33163]: 5.14 deep-scrub starts 2024-09-10T18:46:55.229 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-mon[33163]: 5.14 deep-scrub ok 2024-09-10T18:46:55.229 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2024-09-10T18:46:55.229 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-09-10T18:46:55.229 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2024-09-10T18:46:55.229 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2024-09-10T18:46:55.229 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-09-10T18:46:55.229 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-09-10T18:46:55.229 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2024-09-10T18:46:55.229 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-09-10T18:46:55.230 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:46:55.230 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:55.230 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-mon[33163]: 4.1a scrub starts 2024-09-10T18:46:55.230 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-mon[33163]: 4.1a scrub ok 2024-09-10T18:46:55.231 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.981546917Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-09-10T18:46:55.231 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.982021619Z level=info msg="Executing migration" id="Update alert table charset" 2024-09-10T18:46:55.231 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.982709898Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-09-10T18:46:55.232 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.983423622Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-09-10T18:46:55.232 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.984225075Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-09-10T18:46:55.232 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.984721676Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-09-10T18:46:55.232 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.985402075Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-09-10T18:46:55.232 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.986100174Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-09-10T18:46:55.232 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.986858988Z level=info msg="Executing migration" id="Add for to alert table" 2024-09-10T18:46:55.232 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.987550136Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-09-10T18:46:55.232 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.988280392Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-09-10T18:46:55.232 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.989021711Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-09-10T18:46:55.232 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.989807976Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-09-10T18:46:55.232 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.990433318Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-09-10T18:46:55.233 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.991141623Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-09-10T18:46:55.233 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.991879909Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-09-10T18:46:55.233 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.992708551Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-09-10T18:46:55.233 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.993302823Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-09-10T18:46:55.233 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.994017868Z level=info msg="Executing migration" id="create annotation table v5" 2024-09-10T18:46:55.233 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.994484042Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-09-10T18:46:55.233 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.994931486Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-09-10T18:46:55.233 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.995643657Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-09-10T18:46:55.233 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.99610724Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-09-10T18:46:55.233 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.996860508Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-09-10T18:46:55.233 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.997562367Z level=info msg="Executing migration" id="Update annotation table charset" 2024-09-10T18:46:55.234 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.998245931Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-09-10T18:46:55.234 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.999023441Z level=info msg="Executing migration" id="Drop category_id index" 2024-09-10T18:46:55.234 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:54.999457593Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-09-10T18:46:55.234 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.002412578Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-09-10T18:46:55.234 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.005203827Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-09-10T18:46:55.234 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.008100276Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-09-10T18:46:55.234 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.008557607Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-09-10T18:46:55.234 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.011524293Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-09-10T18:46:55.234 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.014218517Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-09-10T18:46:55.234 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.021818579Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-09-10T18:46:55.234 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.023944846Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-09-10T18:46:55.235 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.025791903Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-09-10T18:46:55.235 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.02785263Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-09-10T18:46:55.235 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.029752325Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-09-10T18:46:55.235 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.030597018Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-09-10T18:46:55.235 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.03260326Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-09-10T18:46:55.235 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.034479993Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-09-10T18:46:55.235 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.036306454Z level=info msg="Executing migration" id="Add epoch_end column" 2024-09-10T18:46:55.235 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.037082369Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-09-10T18:46:55.235 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.038986655Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-09-10T18:46:55.235 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.041007201Z level=info msg="Executing migration" id="Move region to single row" 2024-09-10T18:46:55.235 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.042919933Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-09-10T18:46:55.235 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.043345805Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-09-10T18:46:55.235 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.045327008Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-09-10T18:46:55.236 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.045818428Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-09-10T18:46:55.236 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.047165854Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-09-10T18:46:55.236 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.049190577Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-09-10T18:46:55.236 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.049701037Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-09-10T18:46:55.236 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.051729246Z level=info msg="Executing migration" id="create test_data table" 2024-09-10T18:46:55.236 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.053437288Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-09-10T18:46:55.236 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.053869694Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-09-10T18:46:55.236 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.054309989Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-09-10T18:46:55.236 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.056315307Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-09-10T18:46:55.236 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.058119838Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-09-10T18:46:55.236 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.059980691Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-09-10T18:46:55.236 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.060295306Z level=info msg="Executing migration" id="create team table" 2024-09-10T18:46:55.236 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.062221087Z level=info msg="Executing migration" id="add index team.org_id" 2024-09-10T18:46:55.236 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.06421861Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-09-10T18:46:55.237 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.065955369Z level=info msg="Executing migration" id="create team member table" 2024-09-10T18:46:55.237 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.067877108Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-09-10T18:46:55.237 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.069786448Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-09-10T18:46:55.237 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.07028286Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-09-10T18:46:55.237 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.071953746Z level=info msg="Executing migration" id="Add column email to team table" 2024-09-10T18:46:55.237 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.073869284Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-09-10T18:46:55.237 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.075737625Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-09-10T18:46:55.237 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.076493426Z level=info msg="Executing migration" id="create dashboard acl table" 2024-09-10T18:46:55.237 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.078349199Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-09-10T18:46:55.237 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.08015113Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-09-10T18:46:55.237 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.082104806Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-09-10T18:46:55.237 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.082601874Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-09-10T18:46:55.238 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.084604717Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-09-10T18:46:55.238 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.08505818Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-09-10T18:46:55.238 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.085531269Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-09-10T18:46:55.238 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.085973436Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-09-10T18:46:55.238 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.086423624Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-09-10T18:46:55.238 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.086707648Z level=info msg="Executing migration" id="create tag table" 2024-09-10T18:46:55.238 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.087111322Z level=info msg="Executing migration" id="add index tag.key_value" 2024-09-10T18:46:55.238 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.087548193Z level=info msg="Executing migration" id="create login attempt table" 2024-09-10T18:46:55.238 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.087933274Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-09-10T18:46:55.239 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.088378986Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-09-10T18:46:55.239 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.088772781Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-09-10T18:46:55.239 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.090717893Z level=info msg="Executing migration" id="create login_attempt v2" 2024-09-10T18:46:55.239 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.091129795Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-09-10T18:46:55.239 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.091566378Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-09-10T18:46:55.239 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.091876316Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-09-10T18:46:55.239 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.092249532Z level=info msg="Executing migration" id="create user auth table" 2024-09-10T18:46:55.240 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.092626644Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-09-10T18:46:55.240 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.093088413Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-09-10T18:46:55.240 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.093389269Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-09-10T18:46:55.240 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.094144168Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-09-10T18:46:55.240 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.094886232Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-09-10T18:46:55.240 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.095657305Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-09-10T18:46:55.241 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.096448639Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-09-10T18:46:55.241 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.099288713Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-09-10T18:46:55.241 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.102068351Z level=info msg="Executing migration" id="create server_lock table" 2024-09-10T18:46:55.241 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.105047284Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-09-10T18:46:55.241 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.107759385Z level=info msg="Executing migration" id="create user auth token table" 2024-09-10T18:46:55.242 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.11055637Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-09-10T18:46:55.242 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.113107709Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-09-10T18:46:55.242 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.115771305Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-09-10T18:46:55.242 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.118340945Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-09-10T18:46:55.242 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.121084941Z level=info msg="Executing migration" id="create cache_data table" 2024-09-10T18:46:55.242 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.123374628Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-09-10T18:46:55.243 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.125966676Z level=info msg="Executing migration" id="create short_url table v1" 2024-09-10T18:46:55.243 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.128731185Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-09-10T18:46:55.243 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.131079686Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-09-10T18:46:55.243 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.133706207Z level=info msg="Executing migration" id="delete alert_definition table" 2024-09-10T18:46:55.243 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.134256379Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-09-10T18:46:55.243 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.137080616Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-09-10T18:46:55.243 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.139681307Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-09-10T18:46:55.244 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.142489685Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-09-10T18:46:55.244 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.142907382Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-09-10T18:46:55.244 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.145656752Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-09-10T18:46:55.244 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.148315323Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-09-10T18:46:55.244 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.151143996Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-09-10T18:46:55.244 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.151612632Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-09-10T18:46:55.244 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.152463303Z level=info msg="Executing migration" id="drop alert_definition table" 2024-09-10T18:46:55.244 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.153272336Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-09-10T18:46:55.245 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.154194519Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-09-10T18:46:55.245 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.154737456Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-09-10T18:46:55.245 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.155603451Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-09-10T18:46:55.245 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.156396984Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-09-10T18:46:55.245 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.157245082Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-09-10T18:46:55.245 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.157722188Z level=info msg="Executing migration" id="create alert_instance table" 2024-09-10T18:46:55.245 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.158566085Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-09-10T18:46:55.245 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.159420624Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-09-10T18:46:55.245 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.160214319Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-09-10T18:46:55.245 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.161060683Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-09-10T18:46:55.245 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.161893509Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-09-10T18:46:55.246 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.162706379Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-09-10T18:46:55.246 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.166087119Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-09-10T18:46:55.246 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.169430172Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-09-10T18:46:55.246 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.170353846Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-09-10T18:46:55.246 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.171082049Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-09-10T18:46:55.246 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.17193444Z level=info msg="Executing migration" id="create alert_rule table" 2024-09-10T18:46:55.246 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.172382783Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-09-10T18:46:55.246 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.173207013Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-09-10T18:46:55.246 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.174057543Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-09-10T18:46:55.246 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.174960864Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-09-10T18:46:55.246 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.175308082Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-09-10T18:46:55.246 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.176160009Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-09-10T18:46:55.246 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.177025182Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-09-10T18:46:55.246 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.177883627Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-09-10T18:46:55.247 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.178342601Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-09-10T18:46:55.247 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.17918418Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-09-10T18:46:55.247 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.180070768Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-09-10T18:46:55.247 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.180918459Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-09-10T18:46:55.247 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.181784112Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-09-10T18:46:55.247 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.182654816Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-09-10T18:46:55.247 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.183538187Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-09-10T18:46:55.247 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.186494229Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-09-10T18:46:55.247 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.189262448Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-09-10T18:46:55.247 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.189757589Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-09-10T18:46:55.247 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.192488359Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-09-10T18:46:55.247 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.19522014Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-09-10T18:46:55.247 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.19802401Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-09-10T18:46:55.248 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.200602994Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-09-10T18:46:55.248 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.204623447Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-09-10T18:46:55.248 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.207327535Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-09-10T18:46:55.248 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.210129689Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-09-10T18:46:55.248 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.213064368Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-09-10T18:46:55.248 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.215746728Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-09-10T18:46:55.248 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.218671924Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-09-10T18:46:55.248 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.221466832Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-09-10T18:46:55.248 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.224260214Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-09-10T18:46:55.248 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.226974321Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-09-10T18:46:55.248 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.229817585Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-09-10T18:46:55.406 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:46:55.433 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[40196]: 2.16 scrub starts 2024-09-10T18:46:55.433 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[40196]: 2.16 scrub ok 2024-09-10T18:46:55.433 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[40196]: 2.0 scrub starts 2024-09-10T18:46:55.433 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[40196]: 2.0 scrub ok 2024-09-10T18:46:55.433 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:55.433 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:55.433 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:55.433 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:55.433 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[40196]: 5.14 deep-scrub starts 2024-09-10T18:46:55.433 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[40196]: 5.14 deep-scrub ok 2024-09-10T18:46:55.434 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2024-09-10T18:46:55.434 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-09-10T18:46:55.434 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2024-09-10T18:46:55.434 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2024-09-10T18:46:55.434 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-09-10T18:46:55.434 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-09-10T18:46:55.434 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2024-09-10T18:46:55.434 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-09-10T18:46:55.434 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:46:55.434 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:55.434 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[40196]: 4.1a scrub starts 2024-09-10T18:46:55.434 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[40196]: 4.1a scrub ok 2024-09-10T18:46:55.435 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[29336]: 2.16 scrub starts 2024-09-10T18:46:55.435 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[29336]: 2.16 scrub ok 2024-09-10T18:46:55.435 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[29336]: 2.0 scrub starts 2024-09-10T18:46:55.435 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[29336]: 2.0 scrub ok 2024-09-10T18:46:55.435 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:55.435 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:55.435 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:55.435 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:55.435 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[29336]: 5.14 deep-scrub starts 2024-09-10T18:46:55.435 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[29336]: 5.14 deep-scrub ok 2024-09-10T18:46:55.436 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2024-09-10T18:46:55.436 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-09-10T18:46:55.436 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2024-09-10T18:46:55.436 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2024-09-10T18:46:55.436 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-09-10T18:46:55.436 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-09-10T18:46:55.436 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2024-09-10T18:46:55.436 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-09-10T18:46:55.436 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:46:55.436 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:46:55.436 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[29336]: 4.1a scrub starts 2024-09-10T18:46:55.436 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:55 smithi018 ceph-mon[29336]: 4.1a scrub ok 2024-09-10T18:46:55.478 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.232596127Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-09-10T18:46:55.479 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.2351471Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-09-10T18:46:55.479 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.237801512Z level=info msg="Executing migration" id="create provenance_type table" 2024-09-10T18:46:55.479 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.240376115Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-09-10T18:46:55.479 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.243018726Z level=info msg="Executing migration" id="create alert_image table" 2024-09-10T18:46:55.479 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.245908279Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-09-10T18:46:55.479 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.248445055Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-09-10T18:46:55.479 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.251132797Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-09-10T18:46:55.479 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.253893954Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-09-10T18:46:55.479 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.256364171Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-09-10T18:46:55.479 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.256442262Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-09-10T18:46:55.479 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.259174329Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-09-10T18:46:55.479 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.261736065Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-09-10T18:46:55.479 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.264435266Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-09-10T18:46:55.480 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.264571521Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-09-10T18:46:55.480 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.264684252Z level=info msg="alerts found to migrate" alerts=0 2024-09-10T18:46:55.480 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.267166737Z level=info msg="Executing migration" id="create library_element table v1" 2024-09-10T18:46:55.480 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.269729702Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-09-10T18:46:55.480 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.272459318Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-09-10T18:46:55.480 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.274940226Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-09-10T18:46:55.480 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.277523752Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-09-10T18:46:55.480 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.278237957Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-09-10T18:46:55.480 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.279051321Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-09-10T18:46:55.480 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.279950818Z level=info msg="Executing migration" id="create data_keys table" 2024-09-10T18:46:55.480 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.280650073Z level=info msg="Executing migration" id="create secrets table" 2024-09-10T18:46:55.480 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.28148874Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-09-10T18:46:55.480 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.285733522Z level=info msg="Executing migration" id="add name column into data_keys" 2024-09-10T18:46:55.481 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.28665182Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-09-10T18:46:55.481 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.287615203Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-09-10T18:46:55.481 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.29188351Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-09-10T18:46:55.481 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.296067302Z level=info msg="Executing migration" id="create kv_store table v1" 2024-09-10T18:46:55.481 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.296932495Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-09-10T18:46:55.481 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.297700561Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-09-10T18:46:55.481 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.298701965Z level=info msg="Executing migration" id="create permission table" 2024-09-10T18:46:55.481 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.299405369Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-09-10T18:46:55.481 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.300400834Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-09-10T18:46:55.481 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.301085232Z level=info msg="Executing migration" id="create role table" 2024-09-10T18:46:55.482 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.301954179Z level=info msg="Executing migration" id="add column display_name" 2024-09-10T18:46:55.482 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.302875461Z level=info msg="Executing migration" id="add column group_name" 2024-09-10T18:46:55.482 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.303781576Z level=info msg="Executing migration" id="add index role.org_id" 2024-09-10T18:46:55.482 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.304722399Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-09-10T18:46:55.482 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.305535409Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-09-10T18:46:55.482 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.306406101Z level=info msg="Executing migration" id="create team role table" 2024-09-10T18:46:55.483 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.307399924Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-09-10T18:46:55.483 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.308229375Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-09-10T18:46:55.483 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.308973867Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-09-10T18:46:55.483 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.309909769Z level=info msg="Executing migration" id="create user role table" 2024-09-10T18:46:55.483 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.310437203Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-09-10T18:46:55.483 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.310901797Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-09-10T18:46:55.483 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.311378491Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-09-10T18:46:55.483 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.311854168Z level=info msg="Executing migration" id="create builtin role table" 2024-09-10T18:46:55.483 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.312311469Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-09-10T18:46:55.483 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.316219304Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-09-10T18:46:55.483 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.316722295Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-09-10T18:46:55.483 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.317693187Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-09-10T18:46:55.483 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.318224969Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-09-10T18:46:55.484 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.322216279Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-09-10T18:46:55.484 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.32607983Z level=info msg="Executing migration" id="add unique index role.uid" 2024-09-10T18:46:55.484 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.329962527Z level=info msg="Executing migration" id="create seed assignment table" 2024-09-10T18:46:55.484 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.334007147Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-09-10T18:46:55.484 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.338072217Z level=info msg="Executing migration" id="add column hidden to role table" 2024-09-10T18:46:55.484 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.339779422Z level=info msg="Executing migration" id="create query_history table v1" 2024-09-10T18:46:55.484 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.341804023Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-09-10T18:46:55.484 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.343681426Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-09-10T18:46:55.484 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.345429684Z level=info msg="Executing migration" id="teams permissions migration" 2024-09-10T18:46:55.484 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.34731452Z level=info msg="Executing migration" id="dashboard permissions" 2024-09-10T18:46:55.484 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.349201444Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-09-10T18:46:55.484 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.35106287Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-09-10T18:46:55.484 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.35292442Z level=info msg="Executing migration" id="alerting notification permissions" 2024-09-10T18:46:55.485 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.354730402Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-09-10T18:46:55.485 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.356753592Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-09-10T18:46:55.485 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.358519632Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-09-10T18:46:55.485 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.360451985Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-09-10T18:46:55.485 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.362235035Z level=info msg="Executing migration" id="create correlation table v1" 2024-09-10T18:46:55.485 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.364179901Z level=info msg="Executing migration" id="add index correlations.uid" 2024-09-10T18:46:55.485 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.365946281Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-09-10T18:46:55.485 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.36781441Z level=info msg="Executing migration" id="add correlation config column" 2024-09-10T18:46:55.485 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.369857558Z level=info msg="Executing migration" id="create entity_events table" 2024-09-10T18:46:55.485 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.37177482Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-09-10T18:46:55.485 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.373660565Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-10T18:46:55.485 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.373786939Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-10T18:46:55.485 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.375370929Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-10T18:46:55.486 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.37548401Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-10T18:46:55.486 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.375782389Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-09-10T18:46:55.486 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.377731891Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-09-10T18:46:55.486 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.37959708Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-09-10T18:46:55.486 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.381495113Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-10T18:46:55.486 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.38350824Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-09-10T18:46:55.486 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.385387286Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-10T18:46:55.486 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.387167821Z level=info msg="Executing migration" id="Drop public config table" 2024-09-10T18:46:55.486 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.389039389Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-09-10T18:46:55.486 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.391052373Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-09-10T18:46:55.486 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.392869398Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-10T18:46:55.486 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.394806245Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-09-10T18:46:55.486 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.396693391Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-09-10T18:46:55.487 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.399803904Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-09-10T18:46:55.487 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.401690346Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-09-10T18:46:55.487 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.403737587Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-09-10T18:46:55.487 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.40556079Z level=info msg="Executing migration" id="add share column" 2024-09-10T18:46:55.487 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.409469069Z level=info msg="Executing migration" id="create default alerting folders" 2024-09-10T18:46:55.487 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.41350234Z level=info msg="Executing migration" id="create file table" 2024-09-10T18:46:55.487 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.417618885Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-09-10T18:46:55.487 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.421794739Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-09-10T18:46:55.487 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.425621033Z level=info msg="Executing migration" id="create file_meta table" 2024-09-10T18:46:55.487 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.426236211Z level=info msg="Executing migration" id="file table idx: path key" 2024-09-10T18:46:55.487 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.430409849Z level=info msg="Executing migration" id="set path collation in file table" 2024-09-10T18:46:55.487 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.434386601Z level=info msg="Executing migration" id="managed permissions migration" 2024-09-10T18:46:55.487 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.438357682Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-09-10T18:46:55.487 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.442230512Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-09-10T18:46:55.488 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.446370445Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-09-10T18:46:55.488 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.449910975Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-09-10T18:46:55.488 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.454038337Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-09-10T18:46:55.488 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.457980067Z level=info msg="Executing migration" id="update group index for alert rules" 2024-09-10T18:46:55.488 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.461950467Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-09-10T18:46:55.488 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.465927689Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-09-10T18:46:55.488 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.469883441Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-09-10T18:46:55.488 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.473784422Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-09-10T18:46:55.488 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.475675273Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-09-10T18:46:55.488 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.477516078Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-09-10T18:46:55.773 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.487719459Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-09-10T18:46:55.773 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.489902267Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-09-10T18:46:55.773 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.491934614Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-09-10T18:46:55.773 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.495529694Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-09-10T18:46:55.773 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.497771698Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-09-10T18:46:55.773 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.499590672Z level=info msg="Executing migration" id="create folder table" 2024-09-10T18:46:55.773 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.501703061Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-09-10T18:46:55.774 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.503661898Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-09-10T18:46:55.774 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.505520504Z level=info msg="Executing migration" id="Update folder title length" 2024-09-10T18:46:55.774 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.507432535Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-09-10T18:46:55.774 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=migrator t=2024-09-10T18:46:55.509221809Z level=info msg="migrations completed" performed=485 skipped=0 duration=783.418981ms 2024-09-10T18:46:55.774 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=sqlstore t=2024-09-10T18:46:55.509702815Z level=info msg="Created default organization" 2024-09-10T18:46:55.774 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=plugin.loader t=2024-09-10T18:46:55.56698218Z level=info msg="Plugin registered" pluginID=input 2024-09-10T18:46:55.774 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=plugin.loader t=2024-09-10T18:46:55.594224223Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-09-10T18:46:55.774 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=plugin.loader t=2024-09-10T18:46:55.594245163Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-09-10T18:46:55.774 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=secrets t=2024-09-10T18:46:55.594291078Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-09-10T18:46:55.774 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=query_data t=2024-09-10T18:46:55.595933313Z level=info msg="Query Service initialization" 2024-09-10T18:46:55.774 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=live.push_http t=2024-09-10T18:46:55.598188505Z level=info msg="Live Push Gateway initialization" 2024-09-10T18:46:56.341 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:46:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:46:56.341 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:56 smithi103 ceph-mon[33163]: from='client.24596 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:46:56.342 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:56 smithi103 ceph-mon[33163]: Deploying daemon node-exporter.a on smithi018 2024-09-10T18:46:56.342 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:56 smithi103 ceph-mon[33163]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:46:56.342 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:56 smithi103 ceph-mon[33163]: 5.a deep-scrub starts 2024-09-10T18:46:56.342 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:56 smithi103 ceph-mon[33163]: 5.a deep-scrub ok 2024-09-10T18:46:56.342 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:56 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]': finished 2024-09-10T18:46:56.342 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:56 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]': finished 2024-09-10T18:46:56.342 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:56 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]': finished 2024-09-10T18:46:56.342 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:56 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]': finished 2024-09-10T18:46:56.342 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:56 smithi103 ceph-mon[33163]: osdmap e82: 8 total, 8 up, 8 in 2024-09-10T18:46:56.395 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:56 smithi018 ceph-mon[29336]: from='client.24596 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:46:56.395 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:56 smithi018 ceph-mon[29336]: Deploying daemon node-exporter.a on smithi018 2024-09-10T18:46:56.395 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:56 smithi018 ceph-mon[29336]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:46:56.395 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:56 smithi018 ceph-mon[29336]: 5.a deep-scrub starts 2024-09-10T18:46:56.395 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:56 smithi018 ceph-mon[29336]: 5.a deep-scrub ok 2024-09-10T18:46:56.395 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:56 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]': finished 2024-09-10T18:46:56.395 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:56 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]': finished 2024-09-10T18:46:56.395 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:56 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]': finished 2024-09-10T18:46:56.395 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:56 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]': finished 2024-09-10T18:46:56.395 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:56 smithi018 ceph-mon[29336]: osdmap e82: 8 total, 8 up, 8 in 2024-09-10T18:46:56.396 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:56 smithi018 ceph-mon[40196]: from='client.24596 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:46:56.396 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:56 smithi018 ceph-mon[40196]: Deploying daemon node-exporter.a on smithi018 2024-09-10T18:46:56.396 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:56 smithi018 ceph-mon[40196]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:46:56.396 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:56 smithi018 ceph-mon[40196]: 5.a deep-scrub starts 2024-09-10T18:46:56.396 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:56 smithi018 ceph-mon[40196]: 5.a deep-scrub ok 2024-09-10T18:46:56.396 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:56 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]': finished 2024-09-10T18:46:56.396 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:56 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]': finished 2024-09-10T18:46:56.396 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:56 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]': finished 2024-09-10T18:46:56.396 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:56 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]': finished 2024-09-10T18:46:56.396 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:56 smithi018 ceph-mon[40196]: osdmap e82: 8 total, 8 up, 8 in 2024-09-10T18:46:56.711 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=infra.usagestats.collector t=2024-09-10T18:46:56.340350226Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-09-10T18:46:56.711 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=provisioning.datasources t=2024-09-10T18:46:56.34072601Z level=warn msg="[Deprecated] the datasource provisioning config is outdated. please upgrade" filename=/etc/grafana/provisioning/datasources/ceph-dashboard.yml 2024-09-10T18:46:56.711 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=provisioning.datasources t=2024-09-10T18:46:56.372160494Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-09-10T18:46:56.711 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=provisioning.datasources t=2024-09-10T18:46:56.403764714Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-09-10T18:46:56.711 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=provisioning.alerting t=2024-09-10T18:46:56.425551326Z level=info msg="starting to provision alerting" 2024-09-10T18:46:56.711 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=provisioning.alerting t=2024-09-10T18:46:56.42557051Z level=info msg="finished to provision alerting" 2024-09-10T18:46:56.711 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=grafanaStorageLogger t=2024-09-10T18:46:56.425728027Z level=info msg="storage starting" 2024-09-10T18:46:56.712 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=ngalert.state.manager t=2024-09-10T18:46:56.425902941Z level=info msg="Warming state cache for startup" 2024-09-10T18:46:56.712 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=http.server t=2024-09-10T18:46:56.42786543Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-09-10T18:46:56.712 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=sqlstore.transactions t=2024-09-10T18:46:56.436090237Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-10T18:46:56.712 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=ngalert.state.manager t=2024-09-10T18:46:56.441461527Z level=info msg="State cache has been initialized" states=0 duration=15.557074ms 2024-09-10T18:46:56.712 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=ticker t=2024-09-10T18:46:56.441521263Z level=info msg=starting first_tick=2024-09-10T18:47:00Z 2024-09-10T18:46:56.712 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=ngalert.multiorg.alertmanager t=2024-09-10T18:46:56.441526282Z level=info msg="starting MultiOrg Alertmanager" 2024-09-10T18:46:56.712 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=sqlstore.transactions t=2024-09-10T18:46:56.483975133Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-10T18:46:56.712 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:46:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=sqlstore.transactions t=2024-09-10T18:46:56.495251902Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-10T18:46:57.098 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:46:57.101 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-09-10T18:46:57.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:57 smithi103 ceph-mon[33163]: 6.7 scrub starts 2024-09-10T18:46:57.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:57 smithi103 ceph-mon[33163]: 6.7 scrub ok 2024-09-10T18:46:57.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:57 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:57.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:57 smithi103 ceph-mon[33163]: 6.14 scrub starts 2024-09-10T18:46:57.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:57 smithi103 ceph-mon[33163]: 6.14 scrub ok 2024-09-10T18:46:57.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:57 smithi103 ceph-mon[33163]: osdmap e83: 8 total, 8 up, 8 in 2024-09-10T18:46:57.383 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:57 smithi018 ceph-mon[40196]: 6.7 scrub starts 2024-09-10T18:46:57.383 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:57 smithi018 ceph-mon[40196]: 6.7 scrub ok 2024-09-10T18:46:57.383 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:57 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:57.383 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:57 smithi018 ceph-mon[40196]: 6.14 scrub starts 2024-09-10T18:46:57.383 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:57 smithi018 ceph-mon[40196]: 6.14 scrub ok 2024-09-10T18:46:57.383 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:57 smithi018 ceph-mon[40196]: osdmap e83: 8 total, 8 up, 8 in 2024-09-10T18:46:57.384 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:57 smithi018 ceph-mon[29336]: 6.7 scrub starts 2024-09-10T18:46:57.384 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:57 smithi018 ceph-mon[29336]: 6.7 scrub ok 2024-09-10T18:46:57.384 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:57 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:57.384 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:57 smithi018 ceph-mon[29336]: 6.14 scrub starts 2024-09-10T18:46:57.384 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:57 smithi018 ceph-mon[29336]: 6.14 scrub ok 2024-09-10T18:46:57.384 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:57 smithi018 ceph-mon[29336]: osdmap e83: 8 total, 8 up, 8 in 2024-09-10T18:46:57.385 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:46:57 smithi018 bash[73688]: Getting image source signatures 2024-09-10T18:46:57.385 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:46:57 smithi018 bash[73688]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-10T18:46:57.385 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:46:57 smithi018 bash[73688]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-10T18:46:57.385 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:46:57 smithi018 bash[73688]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-10T18:46:57.962 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":47,"stamp":"2024-09-10T18:46:57.028351+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596078,"num_objects":189,"num_object_clones":0,"num_object_copies":567,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":189,"num_whiteouts":0,"num_read":917,"num_read_kb":783,"num_write":527,"num_write_kb":1670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":554,"ondisk_log_size":554,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":397,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385460,"kb_used_data":6620,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575756,"statfs":{"total":767960285184,"available":765517574144,"internally_reserved":0,"allocated":6778880,"data_stored":4053556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-110,"num_objects":-7,"num_object_clones":0,"num_object_copies":-21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7,"num_whiteouts":0,"num_read":-21,"num_read_kb":-12,"num_write":-16,"num_write_kb":-1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001066"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.194941+0000","last_change":"2024-09-10T18:46:33.824057+0000","last_active":"2024-09-10T18:46:53.194941+0000","last_peered":"2024-09-10T18:46:53.194941+0000","last_clean":"2024-09-10T18:46:53.194941+0000","last_became_active":"2024-09-10T18:45:24.265488+0000","last_became_peered":"2024-09-10T18:45:24.265488+0000","last_unstale":"2024-09-10T18:46:53.194941+0000","last_undegraded":"2024-09-10T18:46:53.194941+0000","last_fullsized":"2024-09-10T18:46:53.194941+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:33.823989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:33.823989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:44.131457+0000","scrub_duration":0.0034389770000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.194213+0000","last_change":"2024-09-10T18:45:17.959740+0000","last_active":"2024-09-10T18:46:53.194213+0000","last_peered":"2024-09-10T18:46:53.194213+0000","last_clean":"2024-09-10T18:46:53.194213+0000","last_became_active":"2024-09-10T18:45:17.959475+0000","last_became_peered":"2024-09-10T18:45:17.959475+0000","last_unstale":"2024-09-10T18:46:53.194213+0000","last_undegraded":"2024-09-10T18:46:53.194213+0000","last_fullsized":"2024-09-10T18:46:53.194213+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"63'15","reported_seq":68,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.194251+0000","last_change":"2024-09-10T18:46:36.856804+0000","last_active":"2024-09-10T18:46:53.194251+0000","last_peered":"2024-09-10T18:46:53.194251+0000","last_clean":"2024-09-10T18:46:53.194251+0000","last_became_active":"2024-09-10T18:45:20.052531+0000","last_became_peered":"2024-09-10T18:45:20.052531+0000","last_unstale":"2024-09-10T18:46:53.194251+0000","last_undegraded":"2024-09-10T18:46:53.194251+0000","last_fullsized":"2024-09-10T18:46:53.194251+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:46:36.856740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:36.856740+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:03.997270+0000","scrub_duration":0.0018131040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.198335+0000","last_change":"2024-09-10T18:46:40.313306+0000","last_active":"2024-09-10T18:46:53.198335+0000","last_peered":"2024-09-10T18:46:53.198335+0000","last_clean":"2024-09-10T18:46:53.198335+0000","last_became_active":"2024-09-10T18:45:22.065619+0000","last_became_peered":"2024-09-10T18:45:22.065619+0000","last_unstale":"2024-09-10T18:46:53.198335+0000","last_undegraded":"2024-09-10T18:46:53.198335+0000","last_fullsized":"2024-09-10T18:46:53.198335+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:40.313218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:40.313218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:52.103586+0000","scrub_duration":0.0017922750000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.194686+0000","last_change":"2024-09-10T18:46:18.890844+0000","last_active":"2024-09-10T18:46:53.194686+0000","last_peered":"2024-09-10T18:46:53.194686+0000","last_clean":"2024-09-10T18:46:53.194686+0000","last_became_active":"2024-09-10T18:45:17.966153+0000","last_became_peered":"2024-09-10T18:45:17.966153+0000","last_unstale":"2024-09-10T18:46:53.194686+0000","last_undegraded":"2024-09-10T18:46:53.194686+0000","last_fullsized":"2024-09-10T18:46:53.194686+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:18.890702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:18.890702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:47.062002+0000","scrub_duration":0.0019382589999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"63'9","reported_seq":49,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.198339+0000","last_change":"2024-09-10T18:45:20.061552+0000","last_active":"2024-09-10T18:46:53.198339+0000","last_peered":"2024-09-10T18:46:53.198339+0000","last_clean":"2024-09-10T18:46:53.198339+0000","last_became_active":"2024-09-10T18:45:20.061266+0000","last_became_peered":"2024-09-10T18:45:20.061266+0000","last_unstale":"2024-09-10T18:46:53.198339+0000","last_undegraded":"2024-09-10T18:46:53.198339+0000","last_fullsized":"2024-09-10T18:46:53.198339+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.395420+0000","last_change":"2024-09-10T18:45:22.073139+0000","last_active":"2024-09-10T18:46:53.395420+0000","last_peered":"2024-09-10T18:46:53.395420+0000","last_clean":"2024-09-10T18:46:53.395420+0000","last_became_active":"2024-09-10T18:45:22.072822+0000","last_became_peered":"2024-09-10T18:45:22.072822+0000","last_unstale":"2024-09-10T18:46:53.395420+0000","last_undegraded":"2024-09-10T18:46:53.395420+0000","last_fullsized":"2024-09-10T18:46:53.395420+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.198569+0000","last_change":"2024-09-10T18:45:24.081275+0000","last_active":"2024-09-10T18:46:53.198569+0000","last_peered":"2024-09-10T18:46:53.198569+0000","last_clean":"2024-09-10T18:46:53.198569+0000","last_became_active":"2024-09-10T18:45:24.081002+0000","last_became_peered":"2024-09-10T18:45:24.081002+0000","last_unstale":"2024-09-10T18:46:53.198569+0000","last_undegraded":"2024-09-10T18:46:53.198569+0000","last_fullsized":"2024-09-10T18:46:53.198569+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.397899+0000","last_change":"2024-09-10T18:46:46.605919+0000","last_active":"2024-09-10T18:46:53.397899+0000","last_peered":"2024-09-10T18:46:53.397899+0000","last_clean":"2024-09-10T18:46:53.397899+0000","last_became_active":"2024-09-10T18:45:17.961097+0000","last_became_peered":"2024-09-10T18:45:17.961097+0000","last_unstale":"2024-09-10T18:46:53.397899+0000","last_undegraded":"2024-09-10T18:46:53.397899+0000","last_fullsized":"2024-09-10T18:46:53.397899+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:46.605814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:46.605814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:51.831114+0000","scrub_duration":0.001570625,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"63'5","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.198569+0000","last_change":"2024-09-10T18:45:20.058010+0000","last_active":"2024-09-10T18:46:53.198569+0000","last_peered":"2024-09-10T18:46:53.198569+0000","last_clean":"2024-09-10T18:46:53.198569+0000","last_became_active":"2024-09-10T18:45:20.057567+0000","last_became_peered":"2024-09-10T18:45:20.057567+0000","last_unstale":"2024-09-10T18:46:53.198569+0000","last_undegraded":"2024-09-10T18:46:53.198569+0000","last_fullsized":"2024-09-10T18:46:53.198569+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.397093+0000","last_change":"2024-09-10T18:46:22.547871+0000","last_active":"2024-09-10T18:46:53.397093+0000","last_peered":"2024-09-10T18:46:53.397093+0000","last_clean":"2024-09-10T18:46:53.397093+0000","last_became_active":"2024-09-10T18:45:22.076575+0000","last_became_peered":"2024-09-10T18:45:22.076575+0000","last_unstale":"2024-09-10T18:46:53.397093+0000","last_undegraded":"2024-09-10T18:46:53.397093+0000","last_fullsized":"2024-09-10T18:46:53.397093+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:22.547800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:22.547800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:32.775669+0000","scrub_duration":0.001880622,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.396431+0000","last_change":"2024-09-10T18:46:36.158152+0000","last_active":"2024-09-10T18:46:53.396431+0000","last_peered":"2024-09-10T18:46:53.396431+0000","last_clean":"2024-09-10T18:46:53.396431+0000","last_became_active":"2024-09-10T18:45:24.079534+0000","last_became_peered":"2024-09-10T18:45:24.079534+0000","last_unstale":"2024-09-10T18:46:53.396431+0000","last_undegraded":"2024-09-10T18:46:53.396431+0000","last_fullsized":"2024-09-10T18:46:53.396431+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:36.158087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:36.158087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:41.325875+0000","scrub_duration":0.0014477100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.397724+0000","last_change":"2024-09-10T18:46:50.666447+0000","last_active":"2024-09-10T18:46:53.397724+0000","last_peered":"2024-09-10T18:46:53.397724+0000","last_clean":"2024-09-10T18:46:53.397724+0000","last_became_active":"2024-09-10T18:45:17.963121+0000","last_became_peered":"2024-09-10T18:45:17.963121+0000","last_unstale":"2024-09-10T18:46:53.397724+0000","last_undegraded":"2024-09-10T18:46:53.397724+0000","last_fullsized":"2024-09-10T18:46:53.397724+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:50.666379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.666379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:06.539690+0000","scrub_duration":0.0016221689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"63'9","reported_seq":59,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.069938+0000","last_change":"2024-09-10T18:46:55.069893+0000","last_active":"2024-09-10T18:46:55.069938+0000","last_peered":"2024-09-10T18:46:55.069938+0000","last_clean":"2024-09-10T18:46:55.069938+0000","last_became_active":"2024-09-10T18:45:20.052652+0000","last_became_peered":"2024-09-10T18:45:20.052652+0000","last_unstale":"2024-09-10T18:46:55.069938+0000","last_undegraded":"2024-09-10T18:46:55.069938+0000","last_fullsized":"2024-09-10T18:46:55.069938+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:46:55.069833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:55.069833+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:12.396664+0000","scrub_duration":0.0016685510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.396071+0000","last_change":"2024-09-10T18:46:31.024322+0000","last_active":"2024-09-10T18:46:53.396071+0000","last_peered":"2024-09-10T18:46:53.396071+0000","last_clean":"2024-09-10T18:46:53.396071+0000","last_became_active":"2024-09-10T18:45:22.068326+0000","last_became_peered":"2024-09-10T18:45:22.068326+0000","last_unstale":"2024-09-10T18:46:53.396071+0000","last_undegraded":"2024-09-10T18:46:53.396071+0000","last_fullsized":"2024-09-10T18:46:53.396071+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:31.024255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:31.024255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:50.407493+0000","scrub_duration":0.00152395,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.193213+0000","last_change":"2024-09-10T18:45:24.081968+0000","last_active":"2024-09-10T18:46:53.193213+0000","last_peered":"2024-09-10T18:46:53.193213+0000","last_clean":"2024-09-10T18:46:53.193213+0000","last_became_active":"2024-09-10T18:45:24.081488+0000","last_became_peered":"2024-09-10T18:45:24.081488+0000","last_unstale":"2024-09-10T18:46:53.193213+0000","last_undegraded":"2024-09-10T18:46:53.193213+0000","last_fullsized":"2024-09-10T18:46:53.193213+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"78'25","reported_seq":79,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.199285+0000","last_change":"2024-09-10T18:46:44.316200+0000","last_active":"2024-09-10T18:46:53.199285+0000","last_peered":"2024-09-10T18:46:53.199285+0000","last_clean":"2024-09-10T18:46:53.199285+0000","last_became_active":"2024-09-10T18:45:20.057019+0000","last_became_peered":"2024-09-10T18:45:20.057019+0000","last_unstale":"2024-09-10T18:46:53.199285+0000","last_undegraded":"2024-09-10T18:46:53.199285+0000","last_fullsized":"2024-09-10T18:46:53.199285+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'25","last_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_deep_scrub":"78'25","last_deep_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_clean_scrub_stamp":"2024-09-10T18:46:44.316146+0000","objects_scrubbed":8,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:06.777730+0000","scrub_duration":0.0046415040000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.397337+0000","last_change":"2024-09-10T18:46:47.595182+0000","last_active":"2024-09-10T18:46:53.397337+0000","last_peered":"2024-09-10T18:46:53.397337+0000","last_clean":"2024-09-10T18:46:53.397337+0000","last_became_active":"2024-09-10T18:45:17.963732+0000","last_became_peered":"2024-09-10T18:45:17.963732+0000","last_unstale":"2024-09-10T18:46:53.397337+0000","last_undegraded":"2024-09-10T18:46:53.397337+0000","last_fullsized":"2024-09-10T18:46:53.397337+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:47.595114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:47.595114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:53.271455+0000","scrub_duration":0.0014913249999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.395771+0000","last_change":"2024-09-10T18:45:22.071993+0000","last_active":"2024-09-10T18:46:53.395771+0000","last_peered":"2024-09-10T18:46:53.395771+0000","last_clean":"2024-09-10T18:46:53.395771+0000","last_became_active":"2024-09-10T18:45:22.071443+0000","last_became_peered":"2024-09-10T18:45:22.071443+0000","last_unstale":"2024-09-10T18:46:53.395771+0000","last_undegraded":"2024-09-10T18:46:53.395771+0000","last_fullsized":"2024-09-10T18:46:53.395771+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.396337+0000","last_change":"2024-09-10T18:46:35.738217+0000","last_active":"2024-09-10T18:46:53.396337+0000","last_peered":"2024-09-10T18:46:53.396337+0000","last_clean":"2024-09-10T18:46:53.396337+0000","last_became_active":"2024-09-10T18:45:24.077897+0000","last_became_peered":"2024-09-10T18:45:24.077897+0000","last_unstale":"2024-09-10T18:46:53.396337+0000","last_undegraded":"2024-09-10T18:46:53.396337+0000","last_fullsized":"2024-09-10T18:46:53.396337+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:35.738143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:35.738143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:48.741747+0000","scrub_duration":0.0015293520000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"63'10","reported_seq":58,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.199028+0000","last_change":"2024-09-10T18:46:31.281588+0000","last_active":"2024-09-10T18:46:53.199028+0000","last_peered":"2024-09-10T18:46:53.199028+0000","last_clean":"2024-09-10T18:46:53.199028+0000","last_became_active":"2024-09-10T18:45:20.053428+0000","last_became_peered":"2024-09-10T18:45:20.053428+0000","last_unstale":"2024-09-10T18:46:53.199028+0000","last_undegraded":"2024-09-10T18:46:53.199028+0000","last_fullsized":"2024-09-10T18:46:53.199028+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:46:31.281519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:31.281519+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:58.508120+0000","scrub_duration":0.0019095119999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.395958+0000","last_change":"2024-09-10T18:46:38.445417+0000","last_active":"2024-09-10T18:46:53.395958+0000","last_peered":"2024-09-10T18:46:53.395958+0000","last_clean":"2024-09-10T18:46:53.395958+0000","last_became_active":"2024-09-10T18:45:17.966833+0000","last_became_peered":"2024-09-10T18:45:17.966833+0000","last_unstale":"2024-09-10T18:46:53.395958+0000","last_undegraded":"2024-09-10T18:46:53.395958+0000","last_fullsized":"2024-09-10T18:46:53.395958+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:38.445362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:38.445362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:02.460484+0000","scrub_duration":0.0012292029999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"63'8","reported_seq":68,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.396156+0000","last_change":"2024-09-10T18:46:25.787936+0000","last_active":"2024-09-10T18:46:53.396156+0000","last_peered":"2024-09-10T18:46:53.396156+0000","last_clean":"2024-09-10T18:46:53.396156+0000","last_became_active":"2024-09-10T18:45:22.068453+0000","last_became_peered":"2024-09-10T18:45:22.068453+0000","last_unstale":"2024-09-10T18:46:53.396156+0000","last_undegraded":"2024-09-10T18:46:53.396156+0000","last_fullsized":"2024-09-10T18:46:53.396156+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:25.787877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:25.787877+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:54.201629+0000","scrub_duration":0.0018093530000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.198799+0000","last_change":"2024-09-10T18:46:32.265034+0000","last_active":"2024-09-10T18:46:53.198799+0000","last_peered":"2024-09-10T18:46:53.198799+0000","last_clean":"2024-09-10T18:46:53.198799+0000","last_became_active":"2024-09-10T18:45:24.081082+0000","last_became_peered":"2024-09-10T18:45:24.081082+0000","last_unstale":"2024-09-10T18:46:53.198799+0000","last_undegraded":"2024-09-10T18:46:53.198799+0000","last_fullsized":"2024-09-10T18:46:53.198799+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:32.264963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:32.264963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:50.207705+0000","scrub_duration":0.002378509,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"peering","last_fresh":"2024-09-10T18:46:55.721662+0000","last_change":"2024-09-10T18:46:55.721662+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T18:46:55.721662+0000","last_undegraded":"2024-09-10T18:46:55.721662+0000","last_fullsized":"2024-09-10T18:46:55.721662+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.395627+0000","last_change":"2024-09-10T18:46:28.320761+0000","last_active":"2024-09-10T18:46:53.395627+0000","last_peered":"2024-09-10T18:46:53.395627+0000","last_clean":"2024-09-10T18:46:53.395627+0000","last_became_active":"2024-09-10T18:45:17.965409+0000","last_became_peered":"2024-09-10T18:45:17.965409+0000","last_unstale":"2024-09-10T18:46:53.395627+0000","last_undegraded":"2024-09-10T18:46:53.395627+0000","last_fullsized":"2024-09-10T18:46:53.395627+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_clean_scrub_stamp":"2024-09-10T18:46:28.320704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:44.092028+0000","scrub_duration":0.001393476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"63'8","reported_seq":68,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.199171+0000","last_change":"2024-09-10T18:46:39.300139+0000","last_active":"2024-09-10T18:46:53.199171+0000","last_peered":"2024-09-10T18:46:53.199171+0000","last_clean":"2024-09-10T18:46:53.199171+0000","last_became_active":"2024-09-10T18:45:22.062497+0000","last_became_peered":"2024-09-10T18:45:22.062497+0000","last_unstale":"2024-09-10T18:46:53.199171+0000","last_undegraded":"2024-09-10T18:46:53.199171+0000","last_fullsized":"2024-09-10T18:46:53.199171+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:39.300065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:39.300065+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:44.830898+0000","scrub_duration":0.001858934,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.396381+0000","last_change":"2024-09-10T18:46:42.057055+0000","last_active":"2024-09-10T18:46:53.396381+0000","last_peered":"2024-09-10T18:46:53.396381+0000","last_clean":"2024-09-10T18:46:53.396381+0000","last_became_active":"2024-09-10T18:45:24.082362+0000","last_became_peered":"2024-09-10T18:45:24.082362+0000","last_unstale":"2024-09-10T18:46:53.396381+0000","last_undegraded":"2024-09-10T18:46:53.396381+0000","last_fullsized":"2024-09-10T18:46:53.396381+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:42.056970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:42.056970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:50.036040+0000","scrub_duration":0.0016528479999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"63'11","reported_seq":64,"reported_epoch":82,"state":"peering","last_fresh":"2024-09-10T18:46:55.720426+0000","last_change":"2024-09-10T18:46:55.720623+0000","last_active":"2024-09-10T18:46:55.720401+0000","last_peered":"2024-09-10T18:46:53.199112+0000","last_clean":"2024-09-10T18:46:53.199112+0000","last_became_active":"2024-09-10T18:45:20.061187+0000","last_became_peered":"2024-09-10T18:45:20.061187+0000","last_unstale":"2024-09-10T18:46:55.720426+0000","last_undegraded":"2024-09-10T18:46:55.720426+0000","last_fullsized":"2024-09-10T18:46:55.720426+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:46:35.274660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:35.274660+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:00.640186+0000","scrub_duration":0.0019745819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2,6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.396003+0000","last_change":"2024-09-10T18:46:39.520497+0000","last_active":"2024-09-10T18:46:53.396003+0000","last_peered":"2024-09-10T18:46:53.396003+0000","last_clean":"2024-09-10T18:46:53.396003+0000","last_became_active":"2024-09-10T18:45:17.966133+0000","last_became_peered":"2024-09-10T18:45:17.966133+0000","last_unstale":"2024-09-10T18:46:53.396003+0000","last_undegraded":"2024-09-10T18:46:53.396003+0000","last_fullsized":"2024-09-10T18:46:53.396003+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:39.520432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:39.520432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:49.040271+0000","scrub_duration":0.0013982510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.397114+0000","last_change":"2024-09-10T18:46:48.116414+0000","last_active":"2024-09-10T18:46:53.397114+0000","last_peered":"2024-09-10T18:46:53.397114+0000","last_clean":"2024-09-10T18:46:53.397114+0000","last_became_active":"2024-09-10T18:45:22.068918+0000","last_became_peered":"2024-09-10T18:45:22.068918+0000","last_unstale":"2024-09-10T18:46:53.397114+0000","last_undegraded":"2024-09-10T18:46:53.397114+0000","last_fullsized":"2024-09-10T18:46:53.397114+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_clean_scrub_stamp":"2024-09-10T18:46:48.116358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:48.149931+0000","scrub_duration":0.001429331,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.194138+0000","last_change":"2024-09-10T18:45:24.264914+0000","last_active":"2024-09-10T18:46:53.194138+0000","last_peered":"2024-09-10T18:46:53.194138+0000","last_clean":"2024-09-10T18:46:53.194138+0000","last_became_active":"2024-09-10T18:45:24.264538+0000","last_became_peered":"2024-09-10T18:45:24.264538+0000","last_unstale":"2024-09-10T18:46:53.194138+0000","last_undegraded":"2024-09-10T18:46:53.194138+0000","last_fullsized":"2024-09-10T18:46:53.194138+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"63'14","reported_seq":64,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.198854+0000","last_change":"2024-09-10T18:46:29.330572+0000","last_active":"2024-09-10T18:46:53.198854+0000","last_peered":"2024-09-10T18:46:53.198854+0000","last_clean":"2024-09-10T18:46:53.198854+0000","last_became_active":"2024-09-10T18:45:20.052243+0000","last_became_peered":"2024-09-10T18:45:20.052243+0000","last_unstale":"2024-09-10T18:46:53.198854+0000","last_undegraded":"2024-09-10T18:46:53.198854+0000","last_fullsized":"2024-09-10T18:46:53.198854+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-10T18:46:29.330503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:29.330503+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:55.456983+0000","scrub_duration":0.0020437459999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.397903+0000","last_change":"2024-09-10T18:46:25.206801+0000","last_active":"2024-09-10T18:46:53.397903+0000","last_peered":"2024-09-10T18:46:53.397903+0000","last_clean":"2024-09-10T18:46:53.397903+0000","last_became_active":"2024-09-10T18:45:17.966027+0000","last_became_peered":"2024-09-10T18:45:17.966027+0000","last_unstale":"2024-09-10T18:46:53.397903+0000","last_undegraded":"2024-09-10T18:46:53.397903+0000","last_fullsized":"2024-09-10T18:46:53.397903+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_clean_scrub_stamp":"2024-09-10T18:46:25.206731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:46.038154+0000","scrub_duration":0.0016589910000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":68,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.194775+0000","last_change":"2024-09-10T18:46:40.808518+0000","last_active":"2024-09-10T18:46:53.194775+0000","last_peered":"2024-09-10T18:46:53.194775+0000","last_clean":"2024-09-10T18:46:53.194775+0000","last_became_active":"2024-09-10T18:45:22.074540+0000","last_became_peered":"2024-09-10T18:45:22.074540+0000","last_unstale":"2024-09-10T18:46:53.194775+0000","last_undegraded":"2024-09-10T18:46:53.194775+0000","last_fullsized":"2024-09-10T18:46:53.194775+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:40.808453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:40.808453+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:43.962254+0000","scrub_duration":0.0016757829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.397041+0000","last_change":"2024-09-10T18:46:32.590511+0000","last_active":"2024-09-10T18:46:53.397041+0000","last_peered":"2024-09-10T18:46:53.397041+0000","last_clean":"2024-09-10T18:46:53.397041+0000","last_became_active":"2024-09-10T18:45:24.265303+0000","last_became_peered":"2024-09-10T18:45:24.265303+0000","last_unstale":"2024-09-10T18:46:53.397041+0000","last_undegraded":"2024-09-10T18:46:53.397041+0000","last_fullsized":"2024-09-10T18:46:53.397041+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:32.590438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:32.590438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:01.680676+0000","scrub_duration":0.0013292270000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"63'18","reported_seq":75,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.194853+0000","last_change":"2024-09-10T18:46:27.801177+0000","last_active":"2024-09-10T18:46:53.194853+0000","last_peered":"2024-09-10T18:46:53.194853+0000","last_clean":"2024-09-10T18:46:53.194853+0000","last_became_active":"2024-09-10T18:45:20.420937+0000","last_became_peered":"2024-09-10T18:45:20.420937+0000","last_unstale":"2024-09-10T18:46:53.194853+0000","last_undegraded":"2024-09-10T18:46:53.194853+0000","last_fullsized":"2024-09-10T18:46:53.194853+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'18","last_scrub_stamp":"2024-09-10T18:46:27.801110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:27.801110+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:42.247024+0000","scrub_duration":0.001870095,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.396436+0000","last_change":"2024-09-10T18:45:17.962000+0000","last_active":"2024-09-10T18:46:53.396436+0000","last_peered":"2024-09-10T18:46:53.396436+0000","last_clean":"2024-09-10T18:46:53.396436+0000","last_became_active":"2024-09-10T18:45:17.961695+0000","last_became_peered":"2024-09-10T18:45:17.961695+0000","last_unstale":"2024-09-10T18:46:53.396436+0000","last_undegraded":"2024-09-10T18:46:53.396436+0000","last_fullsized":"2024-09-10T18:46:53.396436+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.198368+0000","last_change":"2024-09-10T18:46:41.312735+0000","last_active":"2024-09-10T18:46:53.198368+0000","last_peered":"2024-09-10T18:46:53.198368+0000","last_clean":"2024-09-10T18:46:53.198368+0000","last_became_active":"2024-09-10T18:45:22.074012+0000","last_became_peered":"2024-09-10T18:45:22.074012+0000","last_unstale":"2024-09-10T18:46:53.198368+0000","last_undegraded":"2024-09-10T18:46:53.198368+0000","last_fullsized":"2024-09-10T18:46:53.198368+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:41.312671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:41.312671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:47.779534+0000","scrub_duration":0.0013475010000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.199170+0000","last_change":"2024-09-10T18:46:53.087609+0000","last_active":"2024-09-10T18:46:53.199170+0000","last_peered":"2024-09-10T18:46:53.199170+0000","last_clean":"2024-09-10T18:46:53.199170+0000","last_became_active":"2024-09-10T18:45:24.076831+0000","last_became_peered":"2024-09-10T18:45:24.076831+0000","last_unstale":"2024-09-10T18:46:53.199170+0000","last_undegraded":"2024-09-10T18:46:53.199170+0000","last_fullsized":"2024-09-10T18:46:53.199170+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_clean_scrub_stamp":"2024-09-10T18:46:53.087539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:56.714234+0000","scrub_duration":0.001678971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"63'19","reported_seq":79,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.193770+0000","last_change":"2024-09-10T18:46:42.524291+0000","last_active":"2024-09-10T18:46:53.193770+0000","last_peered":"2024-09-10T18:46:53.193770+0000","last_clean":"2024-09-10T18:46:53.193770+0000","last_became_active":"2024-09-10T18:45:20.421046+0000","last_became_peered":"2024-09-10T18:45:20.421046+0000","last_unstale":"2024-09-10T18:46:53.193770+0000","last_undegraded":"2024-09-10T18:46:53.193770+0000","last_fullsized":"2024-09-10T18:46:53.193770+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:46:42.524195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:42.524195+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:45.664487+0000","scrub_duration":0.0022840299999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.397938+0000","last_change":"2024-09-10T18:45:17.960558+0000","last_active":"2024-09-10T18:46:53.397938+0000","last_peered":"2024-09-10T18:46:53.397938+0000","last_clean":"2024-09-10T18:46:53.397938+0000","last_became_active":"2024-09-10T18:45:17.960226+0000","last_became_peered":"2024-09-10T18:45:17.960226+0000","last_unstale":"2024-09-10T18:46:53.397938+0000","last_undegraded":"2024-09-10T18:46:53.397938+0000","last_fullsized":"2024-09-10T18:46:53.397938+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.396055+0000","last_change":"2024-09-10T18:46:52.235456+0000","last_active":"2024-09-10T18:46:53.396055+0000","last_peered":"2024-09-10T18:46:53.396055+0000","last_clean":"2024-09-10T18:46:53.396055+0000","last_became_active":"2024-09-10T18:45:22.066148+0000","last_became_peered":"2024-09-10T18:45:22.066148+0000","last_unstale":"2024-09-10T18:46:53.396055+0000","last_undegraded":"2024-09-10T18:46:53.396055+0000","last_fullsized":"2024-09-10T18:46:53.396055+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:52.235389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:52.235389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:06.785777+0000","scrub_duration":0.001679675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.395286+0000","last_change":"2024-09-10T18:45:24.265025+0000","last_active":"2024-09-10T18:46:53.395286+0000","last_peered":"2024-09-10T18:46:53.395286+0000","last_clean":"2024-09-10T18:46:53.395286+0000","last_became_active":"2024-09-10T18:45:24.264680+0000","last_became_peered":"2024-09-10T18:45:24.264680+0000","last_unstale":"2024-09-10T18:46:53.395286+0000","last_undegraded":"2024-09-10T18:46:53.395286+0000","last_fullsized":"2024-09-10T18:46:53.395286+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"63'10","reported_seq":48,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.395854+0000","last_change":"2024-09-10T18:45:20.057590+0000","last_active":"2024-09-10T18:46:53.395854+0000","last_peered":"2024-09-10T18:46:53.395854+0000","last_clean":"2024-09-10T18:46:53.395854+0000","last_became_active":"2024-09-10T18:45:20.057204+0000","last_became_peered":"2024-09-10T18:45:20.057204+0000","last_unstale":"2024-09-10T18:46:53.395854+0000","last_undegraded":"2024-09-10T18:46:53.395854+0000","last_fullsized":"2024-09-10T18:46:53.395854+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.396063+0000","last_change":"2024-09-10T18:46:50.443101+0000","last_active":"2024-09-10T18:46:53.396063+0000","last_peered":"2024-09-10T18:46:53.396063+0000","last_clean":"2024-09-10T18:46:53.396063+0000","last_became_active":"2024-09-10T18:45:17.966632+0000","last_became_peered":"2024-09-10T18:45:17.966632+0000","last_unstale":"2024-09-10T18:46:53.396063+0000","last_undegraded":"2024-09-10T18:46:53.396063+0000","last_fullsized":"2024-09-10T18:46:53.396063+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:50.443042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.443042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:18.989766+0000","scrub_duration":0.0014088550000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"63'8","reported_seq":58,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.193604+0000","last_change":"2024-09-10T18:45:22.063631+0000","last_active":"2024-09-10T18:46:53.193604+0000","last_peered":"2024-09-10T18:46:53.193604+0000","last_clean":"2024-09-10T18:46:53.193604+0000","last_became_active":"2024-09-10T18:45:22.063326+0000","last_became_peered":"2024-09-10T18:45:22.063326+0000","last_unstale":"2024-09-10T18:46:53.193604+0000","last_undegraded":"2024-09-10T18:46:53.193604+0000","last_fullsized":"2024-09-10T18:46:53.193604+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.193118+0000","last_change":"2024-09-10T18:45:24.077780+0000","last_active":"2024-09-10T18:46:53.193118+0000","last_peered":"2024-09-10T18:46:53.193118+0000","last_clean":"2024-09-10T18:46:53.193118+0000","last_became_active":"2024-09-10T18:45:24.077412+0000","last_became_peered":"2024-09-10T18:45:24.077412+0000","last_unstale":"2024-09-10T18:46:53.193118+0000","last_undegraded":"2024-09-10T18:46:53.193118+0000","last_fullsized":"2024-09-10T18:46:53.193118+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"63'28","reported_seq":110,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.395983+0000","last_change":"2024-09-10T18:46:19.223319+0000","last_active":"2024-09-10T18:46:53.395983+0000","last_peered":"2024-09-10T18:46:53.395983+0000","last_clean":"2024-09-10T18:46:53.395983+0000","last_became_active":"2024-09-10T18:45:20.061413+0000","last_became_peered":"2024-09-10T18:45:20.061413+0000","last_unstale":"2024-09-10T18:46:53.395983+0000","last_undegraded":"2024-09-10T18:46:53.395983+0000","last_fullsized":"2024-09-10T18:46:53.395983+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'28","last_scrub_stamp":"2024-09-10T18:46:19.223250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:19.223250+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:23.644429+0000","scrub_duration":0.0032067810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.396925+0000","last_change":"2024-09-10T18:46:49.120758+0000","last_active":"2024-09-10T18:46:53.396925+0000","last_peered":"2024-09-10T18:46:53.396925+0000","last_clean":"2024-09-10T18:46:53.396925+0000","last_became_active":"2024-09-10T18:45:17.964494+0000","last_became_peered":"2024-09-10T18:45:17.964494+0000","last_unstale":"2024-09-10T18:46:53.396925+0000","last_undegraded":"2024-09-10T18:46:53.396925+0000","last_fullsized":"2024-09-10T18:46:53.396925+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:49.120704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:49.120704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:51.579079+0000","scrub_duration":0.0011863380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.194254+0000","last_change":"2024-09-10T18:45:22.061738+0000","last_active":"2024-09-10T18:46:53.194254+0000","last_peered":"2024-09-10T18:46:53.194254+0000","last_clean":"2024-09-10T18:46:53.194254+0000","last_became_active":"2024-09-10T18:45:22.061428+0000","last_became_peered":"2024-09-10T18:45:22.061428+0000","last_unstale":"2024-09-10T18:46:53.194254+0000","last_undegraded":"2024-09-10T18:46:53.194254+0000","last_fullsized":"2024-09-10T18:46:53.194254+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.194580+0000","last_change":"2024-09-10T18:45:24.073623+0000","last_active":"2024-09-10T18:46:53.194580+0000","last_peered":"2024-09-10T18:46:53.194580+0000","last_clean":"2024-09-10T18:46:53.194580+0000","last_became_active":"2024-09-10T18:45:24.073307+0000","last_became_peered":"2024-09-10T18:45:24.073307+0000","last_unstale":"2024-09-10T18:46:53.194580+0000","last_undegraded":"2024-09-10T18:46:53.194580+0000","last_fullsized":"2024-09-10T18:46:53.194580+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"63'13","reported_seq":60,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.396537+0000","last_change":"2024-09-10T18:45:20.058778+0000","last_active":"2024-09-10T18:46:53.396537+0000","last_peered":"2024-09-10T18:46:53.396537+0000","last_clean":"2024-09-10T18:46:53.396537+0000","last_became_active":"2024-09-10T18:45:20.058435+0000","last_became_peered":"2024-09-10T18:45:20.058435+0000","last_unstale":"2024-09-10T18:46:53.396537+0000","last_undegraded":"2024-09-10T18:46:53.396537+0000","last_fullsized":"2024-09-10T18:46:53.396537+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"65'5","reported_seq":133,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.387182+0000","last_change":"2024-09-10T18:45:22.059997+0000","last_active":"2024-09-10T18:46:55.387182+0000","last_peered":"2024-09-10T18:46:55.387182+0000","last_clean":"2024-09-10T18:46:55.387182+0000","last_became_active":"2024-09-10T18:45:17.969868+0000","last_became_peered":"2024-09-10T18:45:17.969868+0000","last_unstale":"2024-09-10T18:46:55.387182+0000","last_undegraded":"2024-09-10T18:46:55.387182+0000","last_fullsized":"2024-09-10T18:46:55.387182+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021933159999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":90,"num_read_kb":85,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.395591+0000","last_change":"2024-09-10T18:46:46.712335+0000","last_active":"2024-09-10T18:46:53.395591+0000","last_peered":"2024-09-10T18:46:53.395591+0000","last_clean":"2024-09-10T18:46:53.395591+0000","last_became_active":"2024-09-10T18:45:17.960859+0000","last_became_peered":"2024-09-10T18:45:17.960859+0000","last_unstale":"2024-09-10T18:46:53.395591+0000","last_undegraded":"2024-09-10T18:46:53.395591+0000","last_fullsized":"2024-09-10T18:46:53.395591+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:46.712269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:46.712269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:54.660825+0000","scrub_duration":0.001509383,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.396436+0000","last_change":"2024-09-10T18:46:41.818890+0000","last_active":"2024-09-10T18:46:53.396436+0000","last_peered":"2024-09-10T18:46:53.396436+0000","last_clean":"2024-09-10T18:46:53.396436+0000","last_became_active":"2024-09-10T18:45:22.068350+0000","last_became_peered":"2024-09-10T18:45:22.068350+0000","last_unstale":"2024-09-10T18:46:53.396436+0000","last_undegraded":"2024-09-10T18:46:53.396436+0000","last_fullsized":"2024-09-10T18:46:53.396436+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:41.818827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:41.818827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:07.148553+0000","scrub_duration":0.0017408219999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.397694+0000","last_change":"2024-09-10T18:45:24.264879+0000","last_active":"2024-09-10T18:46:53.397694+0000","last_peered":"2024-09-10T18:46:53.397694+0000","last_clean":"2024-09-10T18:46:53.397694+0000","last_became_active":"2024-09-10T18:45:24.264617+0000","last_became_peered":"2024-09-10T18:45:24.264617+0000","last_unstale":"2024-09-10T18:46:53.397694+0000","last_undegraded":"2024-09-10T18:46:53.397694+0000","last_fullsized":"2024-09-10T18:46:53.397694+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"63'12","reported_seq":61,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.194519+0000","last_change":"2024-09-10T18:46:52.472574+0000","last_active":"2024-09-10T18:46:53.194519+0000","last_peered":"2024-09-10T18:46:53.194519+0000","last_clean":"2024-09-10T18:46:53.194519+0000","last_became_active":"2024-09-10T18:45:20.061091+0000","last_became_peered":"2024-09-10T18:45:20.061091+0000","last_unstale":"2024-09-10T18:46:53.194519+0000","last_undegraded":"2024-09-10T18:46:53.194519+0000","last_fullsized":"2024-09-10T18:46:53.194519+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:46:52.472511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:52.472511+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:19.552466+0000","scrub_duration":0.001886935,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"58'1","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.193572+0000","last_change":"2024-09-10T18:45:22.055928+0000","last_active":"2024-09-10T18:46:53.193572+0000","last_peered":"2024-09-10T18:46:53.193572+0000","last_clean":"2024-09-10T18:46:53.193572+0000","last_became_active":"2024-09-10T18:45:17.963932+0000","last_became_peered":"2024-09-10T18:45:17.963932+0000","last_unstale":"2024-09-10T18:46:53.193572+0000","last_undegraded":"2024-09-10T18:46:53.193572+0000","last_fullsized":"2024-09-10T18:46:53.193572+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037812099999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":82,"state":"peering","last_fresh":"2024-09-10T18:46:55.720919+0000","last_change":"2024-09-10T18:46:55.720919+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T18:46:55.720919+0000","last_undegraded":"2024-09-10T18:46:55.720919+0000","last_fullsized":"2024-09-10T18:46:55.720919+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:54.554426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:54.554426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:14.884393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.397115+0000","last_change":"2024-09-10T18:45:22.064783+0000","last_active":"2024-09-10T18:46:53.397115+0000","last_peered":"2024-09-10T18:46:53.397115+0000","last_clean":"2024-09-10T18:46:53.397115+0000","last_became_active":"2024-09-10T18:45:22.064285+0000","last_became_peered":"2024-09-10T18:45:22.064285+0000","last_unstale":"2024-09-10T18:46:53.397115+0000","last_undegraded":"2024-09-10T18:46:53.397115+0000","last_fullsized":"2024-09-10T18:46:53.397115+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.395662+0000","last_change":"2024-09-10T18:46:49.422912+0000","last_active":"2024-09-10T18:46:53.395662+0000","last_peered":"2024-09-10T18:46:53.395662+0000","last_clean":"2024-09-10T18:46:53.395662+0000","last_became_active":"2024-09-10T18:45:24.083689+0000","last_became_peered":"2024-09-10T18:45:24.083689+0000","last_unstale":"2024-09-10T18:46:53.395662+0000","last_undegraded":"2024-09-10T18:46:53.395662+0000","last_fullsized":"2024-09-10T18:46:53.395662+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:49.422843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:49.422843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:00.864466+0000","scrub_duration":0.0014985300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"63'14","reported_seq":65,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.397856+0000","last_change":"2024-09-10T18:46:22.294141+0000","last_active":"2024-09-10T18:46:53.397856+0000","last_peered":"2024-09-10T18:46:53.397856+0000","last_clean":"2024-09-10T18:46:53.397856+0000","last_became_active":"2024-09-10T18:45:20.421119+0000","last_became_peered":"2024-09-10T18:45:20.421119+0000","last_unstale":"2024-09-10T18:46:53.397856+0000","last_undegraded":"2024-09-10T18:46:53.397856+0000","last_fullsized":"2024-09-10T18:46:53.397856+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-10T18:46:22.294072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:22.294072+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:49.887899+0000","scrub_duration":0.0021435069999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"60'2","reported_seq":52,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.193987+0000","last_change":"2024-09-10T18:46:25.845131+0000","last_active":"2024-09-10T18:46:53.193987+0000","last_peered":"2024-09-10T18:46:53.193987+0000","last_clean":"2024-09-10T18:46:53.193987+0000","last_became_active":"2024-09-10T18:45:17.961660+0000","last_became_peered":"2024-09-10T18:45:17.961660+0000","last_unstale":"2024-09-10T18:46:53.193987+0000","last_undegraded":"2024-09-10T18:46:53.193987+0000","last_fullsized":"2024-09-10T18:46:53.193987+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-10T18:46:25.845070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:25.845070+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:50.370600+0000","scrub_duration":0.0020129710000000001,"objects_trimmed":0,"snaptrim_duration":0.0013942640000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.396198+0000","last_change":"2024-09-10T18:46:29.081362+0000","last_active":"2024-09-10T18:46:53.396198+0000","last_peered":"2024-09-10T18:46:53.396198+0000","last_clean":"2024-09-10T18:46:53.396198+0000","last_became_active":"2024-09-10T18:45:17.965163+0000","last_became_peered":"2024-09-10T18:45:17.965163+0000","last_unstale":"2024-09-10T18:46:53.396198+0000","last_undegraded":"2024-09-10T18:46:53.396198+0000","last_fullsized":"2024-09-10T18:46:53.396198+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:29.081296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:29.081296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:50.659900+0000","scrub_duration":0.0015488590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'81","reported_seq":79,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.396168+0000","last_change":"2024-09-10T18:45:45.481030+0000","last_active":"2024-09-10T18:46:53.396168+0000","last_peered":"2024-09-10T18:46:53.396168+0000","last_clean":"2024-09-10T18:46:53.396168+0000","last_became_active":"2024-09-10T18:45:07.433592+0000","last_became_peered":"2024-09-10T18:45:07.433592+0000","last_unstale":"2024-09-10T18:46:53.396168+0000","last_undegraded":"2024-09-10T18:46:53.396168+0000","last_fullsized":"2024-09-10T18:46:53.396168+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'74","last_scrub_stamp":"2024-09-10T18:45:45.480958+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:45:45.480958+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:58.297247+0000","scrub_duration":0.0019876809999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.397526+0000","last_change":"2024-09-10T18:46:26.687029+0000","last_active":"2024-09-10T18:46:53.397526+0000","last_peered":"2024-09-10T18:46:53.397526+0000","last_clean":"2024-09-10T18:46:53.397526+0000","last_became_active":"2024-09-10T18:45:22.068226+0000","last_became_peered":"2024-09-10T18:45:22.068226+0000","last_unstale":"2024-09-10T18:46:53.397526+0000","last_undegraded":"2024-09-10T18:46:53.397526+0000","last_fullsized":"2024-09-10T18:46:53.397526+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:26.686961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:26.686961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:32.456741+0000","scrub_duration":0.0015985109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.397227+0000","last_change":"2024-09-10T18:45:24.073576+0000","last_active":"2024-09-10T18:46:53.397227+0000","last_peered":"2024-09-10T18:46:53.397227+0000","last_clean":"2024-09-10T18:46:53.397227+0000","last_became_active":"2024-09-10T18:45:24.073327+0000","last_became_peered":"2024-09-10T18:45:24.073327+0000","last_unstale":"2024-09-10T18:46:53.397227+0000","last_undegraded":"2024-09-10T18:46:53.397227+0000","last_fullsized":"2024-09-10T18:46:53.397227+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"63'19","reported_seq":74,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.396256+0000","last_change":"2024-09-10T18:46:28.229804+0000","last_active":"2024-09-10T18:46:53.396256+0000","last_peered":"2024-09-10T18:46:53.396256+0000","last_clean":"2024-09-10T18:46:53.396256+0000","last_became_active":"2024-09-10T18:45:20.421097+0000","last_became_peered":"2024-09-10T18:45:20.421097+0000","last_unstale":"2024-09-10T18:46:53.396256+0000","last_undegraded":"2024-09-10T18:46:53.396256+0000","last_fullsized":"2024-09-10T18:46:53.396256+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:46:28.229737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:28.229737+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:49.859218+0000","scrub_duration":0.002308184,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.707829+0000","last_change":"2024-09-10T18:46:53.707765+0000","last_active":"2024-09-10T18:46:53.707829+0000","last_peered":"2024-09-10T18:46:53.707829+0000","last_clean":"2024-09-10T18:46:53.707829+0000","last_became_active":"2024-09-10T18:45:17.959924+0000","last_became_peered":"2024-09-10T18:45:17.959924+0000","last_unstale":"2024-09-10T18:46:53.707829+0000","last_undegraded":"2024-09-10T18:46:53.707829+0000","last_fullsized":"2024-09-10T18:46:53.707829+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:53.707694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:53.707694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:22.214797+0000","scrub_duration":0.0020804069999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.396156+0000","last_change":"2024-09-10T18:46:30.781786+0000","last_active":"2024-09-10T18:46:53.396156+0000","last_peered":"2024-09-10T18:46:53.396156+0000","last_clean":"2024-09-10T18:46:53.396156+0000","last_became_active":"2024-09-10T18:45:22.072581+0000","last_became_peered":"2024-09-10T18:45:22.072581+0000","last_unstale":"2024-09-10T18:46:53.396156+0000","last_undegraded":"2024-09-10T18:46:53.396156+0000","last_fullsized":"2024-09-10T18:46:53.396156+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:30.781730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:30.781730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:38.248319+0000","scrub_duration":0.0020255870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.396124+0000","last_change":"2024-09-10T18:45:24.076661+0000","last_active":"2024-09-10T18:46:53.396124+0000","last_peered":"2024-09-10T18:46:53.396124+0000","last_clean":"2024-09-10T18:46:53.396124+0000","last_became_active":"2024-09-10T18:45:24.076392+0000","last_became_peered":"2024-09-10T18:45:24.076392+0000","last_unstale":"2024-09-10T18:46:53.396124+0000","last_undegraded":"2024-09-10T18:46:53.396124+0000","last_fullsized":"2024-09-10T18:46:53.396124+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"63'9","reported_seq":59,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.193690+0000","last_change":"2024-09-10T18:46:51.486491+0000","last_active":"2024-09-10T18:46:53.193690+0000","last_peered":"2024-09-10T18:46:53.193690+0000","last_clean":"2024-09-10T18:46:53.193690+0000","last_became_active":"2024-09-10T18:45:20.061021+0000","last_became_peered":"2024-09-10T18:45:20.061021+0000","last_unstale":"2024-09-10T18:46:53.193690+0000","last_undegraded":"2024-09-10T18:46:53.193690+0000","last_fullsized":"2024-09-10T18:46:53.193690+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:46:51.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:51.486428+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:05.602014+0000","scrub_duration":0.0019382830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.396489+0000","last_change":"2024-09-10T18:45:17.963854+0000","last_active":"2024-09-10T18:46:53.396489+0000","last_peered":"2024-09-10T18:46:53.396489+0000","last_clean":"2024-09-10T18:46:53.396489+0000","last_became_active":"2024-09-10T18:45:17.963057+0000","last_became_peered":"2024-09-10T18:45:17.963057+0000","last_unstale":"2024-09-10T18:46:53.396489+0000","last_undegraded":"2024-09-10T18:46:53.396489+0000","last_fullsized":"2024-09-10T18:46:53.396489+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.684355+0000","last_change":"2024-09-10T18:46:55.684302+0000","last_active":"2024-09-10T18:46:55.684355+0000","last_peered":"2024-09-10T18:46:55.684355+0000","last_clean":"2024-09-10T18:46:55.684355+0000","last_became_active":"2024-09-10T18:45:22.070343+0000","last_became_peered":"2024-09-10T18:45:22.070343+0000","last_unstale":"2024-09-10T18:46:55.684355+0000","last_undegraded":"2024-09-10T18:46:55.684355+0000","last_fullsized":"2024-09-10T18:46:55.684355+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_clean_scrub_stamp":"2024-09-10T18:46:55.684250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:59.597477+0000","scrub_duration":0.0014319199999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.193485+0000","last_change":"2024-09-10T18:46:47.545406+0000","last_active":"2024-09-10T18:46:53.193485+0000","last_peered":"2024-09-10T18:46:53.193485+0000","last_clean":"2024-09-10T18:46:53.193485+0000","last_became_active":"2024-09-10T18:45:24.078529+0000","last_became_peered":"2024-09-10T18:45:24.078529+0000","last_unstale":"2024-09-10T18:46:53.193485+0000","last_undegraded":"2024-09-10T18:46:53.193485+0000","last_fullsized":"2024-09-10T18:46:53.193485+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:47.545343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:47.545343+0000","last_clean_scrub_stamp":"2024-09-10T18:46:47.545343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:02.171974+0000","scrub_duration":0.0012977799999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"63'15","reported_seq":68,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.395760+0000","last_change":"2024-09-10T18:46:51.114543+0000","last_active":"2024-09-10T18:46:53.395760+0000","last_peered":"2024-09-10T18:46:53.395760+0000","last_clean":"2024-09-10T18:46:53.395760+0000","last_became_active":"2024-09-10T18:45:20.419804+0000","last_became_peered":"2024-09-10T18:45:20.419804+0000","last_unstale":"2024-09-10T18:46:53.395760+0000","last_undegraded":"2024-09-10T18:46:53.395760+0000","last_fullsized":"2024-09-10T18:46:53.395760+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:46:51.114455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:51.114455+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:18.982344+0000","scrub_duration":0.0022557979999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.396609+0000","last_change":"2024-09-10T18:46:17.819966+0000","last_active":"2024-09-10T18:46:53.396609+0000","last_peered":"2024-09-10T18:46:53.396609+0000","last_clean":"2024-09-10T18:46:53.396609+0000","last_became_active":"2024-09-10T18:45:17.959653+0000","last_became_peered":"2024-09-10T18:45:17.959653+0000","last_unstale":"2024-09-10T18:46:53.396609+0000","last_undegraded":"2024-09-10T18:46:53.396609+0000","last_fullsized":"2024-09-10T18:46:53.396609+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:17.819909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:17.819909+0000","last_clean_scrub_stamp":"2024-09-10T18:46:17.819909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:18.454145+0000","scrub_duration":0.001612109,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"63'8","reported_seq":61,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.396723+0000","last_change":"2024-09-10T18:45:22.073687+0000","last_active":"2024-09-10T18:46:53.396723+0000","last_peered":"2024-09-10T18:46:53.396723+0000","last_clean":"2024-09-10T18:46:53.396723+0000","last_became_active":"2024-09-10T18:45:22.073383+0000","last_became_peered":"2024-09-10T18:45:22.073383+0000","last_unstale":"2024-09-10T18:46:53.396723+0000","last_undegraded":"2024-09-10T18:46:53.396723+0000","last_fullsized":"2024-09-10T18:46:53.396723+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.396708+0000","last_change":"2024-09-10T18:46:45.083522+0000","last_active":"2024-09-10T18:46:53.396708+0000","last_peered":"2024-09-10T18:46:53.396708+0000","last_clean":"2024-09-10T18:46:53.396708+0000","last_became_active":"2024-09-10T18:45:24.265256+0000","last_became_peered":"2024-09-10T18:45:24.265256+0000","last_unstale":"2024-09-10T18:46:53.396708+0000","last_undegraded":"2024-09-10T18:46:53.396708+0000","last_fullsized":"2024-09-10T18:46:53.396708+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_clean_scrub_stamp":"2024-09-10T18:46:45.083425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:08.177993+0000","scrub_duration":0.0017589890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"63'12","reported_seq":66,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.198450+0000","last_change":"2024-09-10T18:46:30.320527+0000","last_active":"2024-09-10T18:46:53.198450+0000","last_peered":"2024-09-10T18:46:53.198450+0000","last_clean":"2024-09-10T18:46:53.198450+0000","last_became_active":"2024-09-10T18:45:20.056869+0000","last_became_peered":"2024-09-10T18:45:20.056869+0000","last_unstale":"2024-09-10T18:46:53.198450+0000","last_undegraded":"2024-09-10T18:46:53.198450+0000","last_fullsized":"2024-09-10T18:46:53.198450+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:46:30.320462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:30.320462+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:35.180189+0000","scrub_duration":0.0028822850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"56'2","reported_seq":62,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.199077+0000","last_change":"2024-09-10T18:46:42.303648+0000","last_active":"2024-09-10T18:46:53.199077+0000","last_peered":"2024-09-10T18:46:53.199077+0000","last_clean":"2024-09-10T18:46:53.199077+0000","last_became_active":"2024-09-10T18:45:17.963286+0000","last_became_peered":"2024-09-10T18:45:17.963286+0000","last_unstale":"2024-09-10T18:46:53.199077+0000","last_undegraded":"2024-09-10T18:46:53.199077+0000","last_fullsized":"2024-09-10T18:46:53.199077+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-10T18:46:42.303567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:42.303567+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:05.463057+0000","scrub_duration":0.0020684750000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.395307+0000","last_change":"2024-09-10T18:46:44.754345+0000","last_active":"2024-09-10T18:46:53.395307+0000","last_peered":"2024-09-10T18:46:53.395307+0000","last_clean":"2024-09-10T18:46:53.395307+0000","last_became_active":"2024-09-10T18:45:22.068957+0000","last_became_peered":"2024-09-10T18:45:22.068957+0000","last_unstale":"2024-09-10T18:46:53.395307+0000","last_undegraded":"2024-09-10T18:46:53.395307+0000","last_fullsized":"2024-09-10T18:46:53.395307+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:44.754286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:44.754286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:47.893148+0000","scrub_duration":0.001258386,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.195024+0000","last_change":"2024-09-10T18:45:24.078134+0000","last_active":"2024-09-10T18:46:53.195024+0000","last_peered":"2024-09-10T18:46:53.195024+0000","last_clean":"2024-09-10T18:46:53.195024+0000","last_became_active":"2024-09-10T18:45:24.077815+0000","last_became_peered":"2024-09-10T18:45:24.077815+0000","last_unstale":"2024-09-10T18:46:53.195024+0000","last_undegraded":"2024-09-10T18:46:53.195024+0000","last_fullsized":"2024-09-10T18:46:53.195024+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"76'13","reported_seq":63,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.194513+0000","last_change":"2024-09-10T18:46:24.626912+0000","last_active":"2024-09-10T18:46:53.194513+0000","last_peered":"2024-09-10T18:46:53.194513+0000","last_clean":"2024-09-10T18:46:53.194513+0000","last_became_active":"2024-09-10T18:45:20.420421+0000","last_became_peered":"2024-09-10T18:45:20.420421+0000","last_unstale":"2024-09-10T18:46:53.194513+0000","last_undegraded":"2024-09-10T18:46:53.194513+0000","last_fullsized":"2024-09-10T18:46:53.194513+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'13","last_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_deep_scrub":"76'13","last_deep_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_clean_scrub_stamp":"2024-09-10T18:46:24.626859+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:41.087073+0000","scrub_duration":0.0035642949999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.395369+0000","last_change":"2024-09-10T18:46:31.734522+0000","last_active":"2024-09-10T18:46:53.395369+0000","last_peered":"2024-09-10T18:46:53.395369+0000","last_clean":"2024-09-10T18:46:53.395369+0000","last_became_active":"2024-09-10T18:45:17.965112+0000","last_became_peered":"2024-09-10T18:45:17.965112+0000","last_unstale":"2024-09-10T18:46:53.395369+0000","last_undegraded":"2024-09-10T18:46:53.395369+0000","last_fullsized":"2024-09-10T18:46:53.395369+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:31.734459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:31.734459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:41.550267+0000","scrub_duration":0.0015571960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.194132+0000","last_change":"2024-09-10T18:46:32.829583+0000","last_active":"2024-09-10T18:46:53.194132+0000","last_peered":"2024-09-10T18:46:53.194132+0000","last_clean":"2024-09-10T18:46:53.194132+0000","last_became_active":"2024-09-10T18:45:22.069040+0000","last_became_peered":"2024-09-10T18:45:22.069040+0000","last_unstale":"2024-09-10T18:46:53.194132+0000","last_undegraded":"2024-09-10T18:46:53.194132+0000","last_fullsized":"2024-09-10T18:46:53.194132+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:32.829508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:32.829508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:45.869787+0000","scrub_duration":0.001897239,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"63'1","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.395922+0000","last_change":"2024-09-10T18:45:24.079638+0000","last_active":"2024-09-10T18:46:53.395922+0000","last_peered":"2024-09-10T18:46:53.395922+0000","last_clean":"2024-09-10T18:46:53.395922+0000","last_became_active":"2024-09-10T18:45:24.079258+0000","last_became_peered":"2024-09-10T18:45:24.079258+0000","last_unstale":"2024-09-10T18:46:53.395922+0000","last_undegraded":"2024-09-10T18:46:53.395922+0000","last_fullsized":"2024-09-10T18:46:53.395922+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.396201+0000","last_change":"2024-09-10T18:46:48.227450+0000","last_active":"2024-09-10T18:46:53.396201+0000","last_peered":"2024-09-10T18:46:53.396201+0000","last_clean":"2024-09-10T18:46:53.396201+0000","last_became_active":"2024-09-10T18:45:17.964705+0000","last_became_peered":"2024-09-10T18:45:17.964705+0000","last_unstale":"2024-09-10T18:46:53.396201+0000","last_undegraded":"2024-09-10T18:46:53.396201+0000","last_fullsized":"2024-09-10T18:46:53.396201+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:48.227378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:48.227378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:54.333207+0000","scrub_duration":0.0015447519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'6","reported_seq":42,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.194283+0000","last_change":"2024-09-10T18:45:20.422485+0000","last_active":"2024-09-10T18:46:53.194283+0000","last_peered":"2024-09-10T18:46:53.194283+0000","last_clean":"2024-09-10T18:46:53.194283+0000","last_became_active":"2024-09-10T18:45:20.421746+0000","last_became_peered":"2024-09-10T18:45:20.421746+0000","last_unstale":"2024-09-10T18:46:53.194283+0000","last_undegraded":"2024-09-10T18:46:53.194283+0000","last_fullsized":"2024-09-10T18:46:53.194283+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.396863+0000","last_change":"2024-09-10T18:46:39.084489+0000","last_active":"2024-09-10T18:46:53.396863+0000","last_peered":"2024-09-10T18:46:53.396863+0000","last_clean":"2024-09-10T18:46:53.396863+0000","last_became_active":"2024-09-10T18:45:22.069653+0000","last_became_peered":"2024-09-10T18:45:22.069653+0000","last_unstale":"2024-09-10T18:46:53.396863+0000","last_undegraded":"2024-09-10T18:46:53.396863+0000","last_fullsized":"2024-09-10T18:46:53.396863+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:39.084433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:39.084433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:05.431757+0000","scrub_duration":0.0014845819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.397621+0000","last_change":"2024-09-10T18:46:52.611074+0000","last_active":"2024-09-10T18:46:53.397621+0000","last_peered":"2024-09-10T18:46:53.397621+0000","last_clean":"2024-09-10T18:46:53.397621+0000","last_became_active":"2024-09-10T18:45:24.264611+0000","last_became_peered":"2024-09-10T18:45:24.264611+0000","last_unstale":"2024-09-10T18:46:53.397621+0000","last_undegraded":"2024-09-10T18:46:53.397621+0000","last_fullsized":"2024-09-10T18:46:53.397621+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:52.611003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:52.611003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:04.517442+0000","scrub_duration":0.0015558849999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"63'10","reported_seq":60,"reported_epoch":82,"state":"peering","last_fresh":"2024-09-10T18:46:55.724077+0000","last_change":"2024-09-10T18:46:55.724370+0000","last_active":"2024-09-10T18:46:55.724059+0000","last_peered":"2024-09-10T18:46:53.195070+0000","last_clean":"2024-09-10T18:46:53.195070+0000","last_became_active":"2024-09-10T18:45:20.419887+0000","last_became_peered":"2024-09-10T18:45:20.419887+0000","last_unstale":"2024-09-10T18:46:55.724077+0000","last_undegraded":"2024-09-10T18:46:55.724077+0000","last_fullsized":"2024-09-10T18:46:55.724077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:46:50.722316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.722316+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:04.342162+0000","scrub_duration":0.0022194070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2,7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.396897+0000","last_change":"2024-09-10T18:46:50.093435+0000","last_active":"2024-09-10T18:46:53.396897+0000","last_peered":"2024-09-10T18:46:53.396897+0000","last_clean":"2024-09-10T18:46:53.396897+0000","last_became_active":"2024-09-10T18:45:17.964870+0000","last_became_peered":"2024-09-10T18:45:17.964870+0000","last_unstale":"2024-09-10T18:46:53.396897+0000","last_undegraded":"2024-09-10T18:46:53.396897+0000","last_fullsized":"2024-09-10T18:46:53.396897+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:50.093372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.093372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:55.991532+0000","scrub_duration":0.0015580539999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"63'8","reported_seq":68,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.396527+0000","last_change":"2024-09-10T18:46:34.085224+0000","last_active":"2024-09-10T18:46:53.396527+0000","last_peered":"2024-09-10T18:46:53.396527+0000","last_clean":"2024-09-10T18:46:53.396527+0000","last_became_active":"2024-09-10T18:45:22.062940+0000","last_became_peered":"2024-09-10T18:45:22.062940+0000","last_unstale":"2024-09-10T18:46:53.396527+0000","last_undegraded":"2024-09-10T18:46:53.396527+0000","last_fullsized":"2024-09-10T18:46:53.396527+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:34.085156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:34.085156+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:54.777061+0000","scrub_duration":0.0020843120000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.194255+0000","last_change":"2024-09-10T18:45:24.074039+0000","last_active":"2024-09-10T18:46:53.194255+0000","last_peered":"2024-09-10T18:46:53.194255+0000","last_clean":"2024-09-10T18:46:53.194255+0000","last_became_active":"2024-09-10T18:45:24.073736+0000","last_became_peered":"2024-09-10T18:45:24.073736+0000","last_unstale":"2024-09-10T18:46:53.194255+0000","last_undegraded":"2024-09-10T18:46:53.194255+0000","last_fullsized":"2024-09-10T18:46:53.194255+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"63'9","reported_seq":59,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.396556+0000","last_change":"2024-09-10T18:46:23.086274+0000","last_active":"2024-09-10T18:46:53.396556+0000","last_peered":"2024-09-10T18:46:53.396556+0000","last_clean":"2024-09-10T18:46:53.396556+0000","last_became_active":"2024-09-10T18:45:20.421124+0000","last_became_peered":"2024-09-10T18:45:20.421124+0000","last_unstale":"2024-09-10T18:46:53.396556+0000","last_undegraded":"2024-09-10T18:46:53.396556+0000","last_fullsized":"2024-09-10T18:46:53.396556+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:46:23.086203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:23.086203+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:44.793257+0000","scrub_duration":0.002168068,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.193901+0000","last_change":"2024-09-10T18:46:45.541882+0000","last_active":"2024-09-10T18:46:53.193901+0000","last_peered":"2024-09-10T18:46:53.193901+0000","last_clean":"2024-09-10T18:46:53.193901+0000","last_became_active":"2024-09-10T18:45:17.960106+0000","last_became_peered":"2024-09-10T18:45:17.960106+0000","last_unstale":"2024-09-10T18:46:53.193901+0000","last_undegraded":"2024-09-10T18:46:53.193901+0000","last_fullsized":"2024-09-10T18:46:53.193901+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:45.541819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:45.541819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:15.015402+0000","scrub_duration":0.0015180809999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"81'9","reported_seq":69,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:54.800940+0000","last_change":"2024-09-10T18:46:54.800878+0000","last_active":"2024-09-10T18:46:54.800940+0000","last_peered":"2024-09-10T18:46:54.800940+0000","last_clean":"2024-09-10T18:46:54.800940+0000","last_became_active":"2024-09-10T18:45:22.075368+0000","last_became_peered":"2024-09-10T18:45:22.075368+0000","last_unstale":"2024-09-10T18:46:54.800940+0000","last_undegraded":"2024-09-10T18:46:54.800940+0000","last_fullsized":"2024-09-10T18:46:54.800940+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"81'9","last_scrub_stamp":"2024-09-10T18:46:54.800790+0000","last_deep_scrub":"81'9","last_deep_scrub_stamp":"2024-09-10T18:46:54.800790+0000","last_clean_scrub_stamp":"2024-09-10T18:46:54.800790+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:14.444052+0000","scrub_duration":0.0034208709999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.198812+0000","last_change":"2024-09-10T18:46:36.305262+0000","last_active":"2024-09-10T18:46:53.198812+0000","last_peered":"2024-09-10T18:46:53.198812+0000","last_clean":"2024-09-10T18:46:53.198812+0000","last_became_active":"2024-09-10T18:45:24.080196+0000","last_became_peered":"2024-09-10T18:45:24.080196+0000","last_unstale":"2024-09-10T18:46:53.198812+0000","last_undegraded":"2024-09-10T18:46:53.198812+0000","last_fullsized":"2024-09-10T18:46:53.198812+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_clean_scrub_stamp":"2024-09-10T18:46:36.305199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:04.747507+0000","scrub_duration":0.0013794809999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"63'9","reported_seq":49,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.395825+0000","last_change":"2024-09-10T18:45:20.057767+0000","last_active":"2024-09-10T18:46:53.395825+0000","last_peered":"2024-09-10T18:46:53.395825+0000","last_clean":"2024-09-10T18:46:53.395825+0000","last_became_active":"2024-09-10T18:45:20.057479+0000","last_became_peered":"2024-09-10T18:45:20.057479+0000","last_unstale":"2024-09-10T18:46:53.395825+0000","last_undegraded":"2024-09-10T18:46:53.395825+0000","last_fullsized":"2024-09-10T18:46:53.395825+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.397345+0000","last_change":"2024-09-10T18:46:23.292276+0000","last_active":"2024-09-10T18:46:53.397345+0000","last_peered":"2024-09-10T18:46:53.397345+0000","last_clean":"2024-09-10T18:46:53.397345+0000","last_became_active":"2024-09-10T18:45:17.962009+0000","last_became_peered":"2024-09-10T18:45:17.962009+0000","last_unstale":"2024-09-10T18:46:53.397345+0000","last_undegraded":"2024-09-10T18:46:53.397345+0000","last_fullsized":"2024-09-10T18:46:53.397345+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:23.292207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:23.292207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:27.416585+0000","scrub_duration":0.001659953,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.195045+0000","last_change":"2024-09-10T18:45:22.073186+0000","last_active":"2024-09-10T18:46:53.195045+0000","last_peered":"2024-09-10T18:46:53.195045+0000","last_clean":"2024-09-10T18:46:53.195045+0000","last_became_active":"2024-09-10T18:45:22.072877+0000","last_became_peered":"2024-09-10T18:45:22.072877+0000","last_unstale":"2024-09-10T18:46:53.195045+0000","last_undegraded":"2024-09-10T18:46:53.195045+0000","last_fullsized":"2024-09-10T18:46:53.195045+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.193103+0000","last_change":"2024-09-10T18:46:50.499866+0000","last_active":"2024-09-10T18:46:53.193103+0000","last_peered":"2024-09-10T18:46:53.193103+0000","last_clean":"2024-09-10T18:46:53.193103+0000","last_became_active":"2024-09-10T18:45:24.081050+0000","last_became_peered":"2024-09-10T18:45:24.081050+0000","last_unstale":"2024-09-10T18:46:53.193103+0000","last_undegraded":"2024-09-10T18:46:53.193103+0000","last_fullsized":"2024-09-10T18:46:53.193103+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:50.499815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.499815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:13.479025+0000","scrub_duration":0.001285432,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"63'11","reported_seq":52,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.198678+0000","last_change":"2024-09-10T18:45:20.057563+0000","last_active":"2024-09-10T18:46:53.198678+0000","last_peered":"2024-09-10T18:46:53.198678+0000","last_clean":"2024-09-10T18:46:53.198678+0000","last_became_active":"2024-09-10T18:45:20.056885+0000","last_became_peered":"2024-09-10T18:45:20.056885+0000","last_unstale":"2024-09-10T18:46:53.198678+0000","last_undegraded":"2024-09-10T18:46:53.198678+0000","last_fullsized":"2024-09-10T18:46:53.198678+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.396472+0000","last_change":"2024-09-10T18:46:36.402610+0000","last_active":"2024-09-10T18:46:53.396472+0000","last_peered":"2024-09-10T18:46:53.396472+0000","last_clean":"2024-09-10T18:46:53.396472+0000","last_became_active":"2024-09-10T18:45:17.963542+0000","last_became_peered":"2024-09-10T18:45:17.963542+0000","last_unstale":"2024-09-10T18:46:53.396472+0000","last_undegraded":"2024-09-10T18:46:53.396472+0000","last_fullsized":"2024-09-10T18:46:53.396472+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:36.402548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:36.402548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:04.495178+0000","scrub_duration":0.0014103919999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.395539+0000","last_change":"2024-09-10T18:46:37.414794+0000","last_active":"2024-09-10T18:46:53.395539+0000","last_peered":"2024-09-10T18:46:53.395539+0000","last_clean":"2024-09-10T18:46:53.395539+0000","last_became_active":"2024-09-10T18:45:22.070065+0000","last_became_peered":"2024-09-10T18:45:22.070065+0000","last_unstale":"2024-09-10T18:46:53.395539+0000","last_undegraded":"2024-09-10T18:46:53.395539+0000","last_fullsized":"2024-09-10T18:46:53.395539+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:37.414734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:37.414734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:44.027335+0000","scrub_duration":0.001607996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.193835+0000","last_change":"2024-09-10T18:45:24.072477+0000","last_active":"2024-09-10T18:46:53.193835+0000","last_peered":"2024-09-10T18:46:53.193835+0000","last_clean":"2024-09-10T18:46:53.193835+0000","last_became_active":"2024-09-10T18:45:24.072224+0000","last_became_peered":"2024-09-10T18:45:24.072224+0000","last_unstale":"2024-09-10T18:46:53.193835+0000","last_undegraded":"2024-09-10T18:46:53.193835+0000","last_fullsized":"2024-09-10T18:46:53.193835+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"63'4","reported_seq":49,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.194792+0000","last_change":"2024-09-10T18:46:46.766906+0000","last_active":"2024-09-10T18:46:53.194792+0000","last_peered":"2024-09-10T18:46:53.194792+0000","last_clean":"2024-09-10T18:46:53.194792+0000","last_became_active":"2024-09-10T18:45:20.059205+0000","last_became_peered":"2024-09-10T18:45:20.059205+0000","last_unstale":"2024-09-10T18:46:53.194792+0000","last_undegraded":"2024-09-10T18:46:53.194792+0000","last_fullsized":"2024-09-10T18:46:53.194792+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4","last_scrub_stamp":"2024-09-10T18:46:46.766766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:46.766766+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:49.051695+0000","scrub_duration":0.0021847250000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:54.082884+0000","last_change":"2024-09-10T18:46:54.082828+0000","last_active":"2024-09-10T18:46:54.082884+0000","last_peered":"2024-09-10T18:46:54.082884+0000","last_clean":"2024-09-10T18:46:54.082884+0000","last_became_active":"2024-09-10T18:45:17.963630+0000","last_became_peered":"2024-09-10T18:45:17.963630+0000","last_unstale":"2024-09-10T18:46:54.082884+0000","last_undegraded":"2024-09-10T18:46:54.082884+0000","last_fullsized":"2024-09-10T18:46:54.082884+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:54.082761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:54.082761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:05.974208+0000","scrub_duration":0.001210458,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.395946+0000","last_change":"2024-09-10T18:46:35.172936+0000","last_active":"2024-09-10T18:46:53.395946+0000","last_peered":"2024-09-10T18:46:53.395946+0000","last_clean":"2024-09-10T18:46:53.395946+0000","last_became_active":"2024-09-10T18:45:22.072377+0000","last_became_peered":"2024-09-10T18:45:22.072377+0000","last_unstale":"2024-09-10T18:46:53.395946+0000","last_undegraded":"2024-09-10T18:46:53.395946+0000","last_fullsized":"2024-09-10T18:46:53.395946+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_clean_scrub_stamp":"2024-09-10T18:46:35.172861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:57.638389+0000","scrub_duration":0.001552933,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.596501+0000","last_change":"2024-09-10T18:46:53.596450+0000","last_active":"2024-09-10T18:46:53.596501+0000","last_peered":"2024-09-10T18:46:53.596501+0000","last_clean":"2024-09-10T18:46:53.596501+0000","last_became_active":"2024-09-10T18:45:24.077398+0000","last_became_peered":"2024-09-10T18:45:24.077398+0000","last_unstale":"2024-09-10T18:46:53.596501+0000","last_undegraded":"2024-09-10T18:46:53.596501+0000","last_fullsized":"2024-09-10T18:46:53.596501+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:53.596380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:53.596380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:54.667444+0000","scrub_duration":0.0020285519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"79'16","reported_seq":67,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.194041+0000","last_change":"2024-09-10T18:46:48.744340+0000","last_active":"2024-09-10T18:46:53.194041+0000","last_peered":"2024-09-10T18:46:53.194041+0000","last_clean":"2024-09-10T18:46:53.194041+0000","last_became_active":"2024-09-10T18:45:20.421479+0000","last_became_peered":"2024-09-10T18:45:20.421479+0000","last_unstale":"2024-09-10T18:46:53.194041+0000","last_undegraded":"2024-09-10T18:46:53.194041+0000","last_fullsized":"2024-09-10T18:46:53.194041+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"79'16","last_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_deep_scrub":"79'16","last_deep_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_clean_scrub_stamp":"2024-09-10T18:46:48.744287+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:57.831206+0000","scrub_duration":0.0044879259999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"56'1","reported_seq":52,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.397589+0000","last_change":"2024-09-10T18:46:41.147533+0000","last_active":"2024-09-10T18:46:53.397589+0000","last_peered":"2024-09-10T18:46:53.397589+0000","last_clean":"2024-09-10T18:46:53.397589+0000","last_became_active":"2024-09-10T18:45:17.961222+0000","last_became_peered":"2024-09-10T18:45:17.961222+0000","last_unstale":"2024-09-10T18:46:53.397589+0000","last_undegraded":"2024-09-10T18:46:53.397589+0000","last_fullsized":"2024-09-10T18:46:53.397589+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:46:41.147463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:41.147463+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:58.016926+0000","scrub_duration":0.0016797889999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.395664+0000","last_change":"2024-09-10T18:46:27.667088+0000","last_active":"2024-09-10T18:46:53.395664+0000","last_peered":"2024-09-10T18:46:53.395664+0000","last_clean":"2024-09-10T18:46:53.395664+0000","last_became_active":"2024-09-10T18:45:22.072312+0000","last_became_peered":"2024-09-10T18:45:22.072312+0000","last_unstale":"2024-09-10T18:46:53.395664+0000","last_undegraded":"2024-09-10T18:46:53.395664+0000","last_fullsized":"2024-09-10T18:46:53.395664+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:27.667018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:27.667018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:38.065664+0000","scrub_duration":0.001388886,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.194936+0000","last_change":"2024-09-10T18:46:29.791617+0000","last_active":"2024-09-10T18:46:53.194936+0000","last_peered":"2024-09-10T18:46:53.194936+0000","last_clean":"2024-09-10T18:46:53.194936+0000","last_became_active":"2024-09-10T18:45:24.264818+0000","last_became_peered":"2024-09-10T18:45:24.264818+0000","last_unstale":"2024-09-10T18:46:53.194936+0000","last_undegraded":"2024-09-10T18:46:53.194936+0000","last_fullsized":"2024-09-10T18:46:53.194936+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:29.791563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:29.791563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:37.941643+0000","scrub_duration":0.0021031819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"78'2","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.397504+0000","last_change":"2024-09-10T18:46:37.581201+0000","last_active":"2024-09-10T18:46:53.397504+0000","last_peered":"2024-09-10T18:46:53.397504+0000","last_clean":"2024-09-10T18:46:53.397504+0000","last_became_active":"2024-09-10T18:45:24.076380+0000","last_became_peered":"2024-09-10T18:45:24.076380+0000","last_unstale":"2024-09-10T18:46:53.397504+0000","last_undegraded":"2024-09-10T18:46:53.397504+0000","last_fullsized":"2024-09-10T18:46:53.397504+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'2","last_scrub_stamp":"2024-09-10T18:46:37.581130+0000","last_deep_scrub":"78'2","last_deep_scrub_stamp":"2024-09-10T18:46:37.581130+0000","last_clean_scrub_stamp":"2024-09-10T18:46:37.581130+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:55.887929+0000","scrub_duration":0.0024563609999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.395868+0000","last_change":"2024-09-10T18:46:28.088657+0000","last_active":"2024-09-10T18:46:53.395868+0000","last_peered":"2024-09-10T18:46:53.395868+0000","last_clean":"2024-09-10T18:46:53.395868+0000","last_became_active":"2024-09-10T18:45:17.962108+0000","last_became_peered":"2024-09-10T18:45:17.962108+0000","last_unstale":"2024-09-10T18:46:53.395868+0000","last_undegraded":"2024-09-10T18:46:53.395868+0000","last_fullsized":"2024-09-10T18:46:53.395868+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:28.088588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:28.088588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:56.550863+0000","scrub_duration":0.00140508,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"63'10","reported_seq":58,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.194425+0000","last_change":"2024-09-10T18:46:49.494628+0000","last_active":"2024-09-10T18:46:53.194425+0000","last_peered":"2024-09-10T18:46:53.194425+0000","last_clean":"2024-09-10T18:46:53.194425+0000","last_became_active":"2024-09-10T18:45:20.421094+0000","last_became_peered":"2024-09-10T18:45:20.421094+0000","last_unstale":"2024-09-10T18:46:53.194425+0000","last_undegraded":"2024-09-10T18:46:53.194425+0000","last_fullsized":"2024-09-10T18:46:53.194425+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:46:49.494565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:49.494565+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:08.379972+0000","scrub_duration":0.0036379630000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":71,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.397522+0000","last_change":"2024-09-10T18:46:33.150919+0000","last_active":"2024-09-10T18:46:53.397522+0000","last_peered":"2024-09-10T18:46:53.397522+0000","last_clean":"2024-09-10T18:46:53.397522+0000","last_became_active":"2024-09-10T18:45:22.073281+0000","last_became_peered":"2024-09-10T18:45:22.073281+0000","last_unstale":"2024-09-10T18:46:53.397522+0000","last_undegraded":"2024-09-10T18:46:53.397522+0000","last_fullsized":"2024-09-10T18:46:53.397522+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:33.150852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:33.150852+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:51.299257+0000","scrub_duration":0.0017025339999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.395918+0000","last_change":"2024-09-10T18:45:24.082638+0000","last_active":"2024-09-10T18:46:53.395918+0000","last_peered":"2024-09-10T18:46:53.395918+0000","last_clean":"2024-09-10T18:46:53.395918+0000","last_became_active":"2024-09-10T18:45:24.082318+0000","last_became_peered":"2024-09-10T18:45:24.082318+0000","last_unstale":"2024-09-10T18:46:53.395918+0000","last_undegraded":"2024-09-10T18:46:53.395918+0000","last_fullsized":"2024-09-10T18:46:53.395918+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.194380+0000","last_change":"2024-09-10T18:46:44.694805+0000","last_active":"2024-09-10T18:46:53.194380+0000","last_peered":"2024-09-10T18:46:53.194380+0000","last_clean":"2024-09-10T18:46:53.194380+0000","last_became_active":"2024-09-10T18:45:17.965707+0000","last_became_peered":"2024-09-10T18:45:17.965707+0000","last_unstale":"2024-09-10T18:46:53.194380+0000","last_undegraded":"2024-09-10T18:46:53.194380+0000","last_fullsized":"2024-09-10T18:46:53.194380+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:44.694737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:44.694737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:03.417079+0000","scrub_duration":0.0016330400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"63'11","reported_seq":62,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.397519+0000","last_change":"2024-09-10T18:46:46.230915+0000","last_active":"2024-09-10T18:46:53.397519+0000","last_peered":"2024-09-10T18:46:53.397519+0000","last_clean":"2024-09-10T18:46:53.397519+0000","last_became_active":"2024-09-10T18:45:20.058717+0000","last_became_peered":"2024-09-10T18:45:20.058717+0000","last_unstale":"2024-09-10T18:46:53.397519+0000","last_undegraded":"2024-09-10T18:46:53.397519+0000","last_fullsized":"2024-09-10T18:46:53.397519+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:46:46.230819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:46.230819+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:54.186112+0000","scrub_duration":0.002545322,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.194419+0000","last_change":"2024-09-10T18:45:22.068991+0000","last_active":"2024-09-10T18:46:53.194419+0000","last_peered":"2024-09-10T18:46:53.194419+0000","last_clean":"2024-09-10T18:46:53.194419+0000","last_became_active":"2024-09-10T18:45:22.068676+0000","last_became_peered":"2024-09-10T18:45:22.068676+0000","last_unstale":"2024-09-10T18:46:53.194419+0000","last_undegraded":"2024-09-10T18:46:53.194419+0000","last_fullsized":"2024-09-10T18:46:53.194419+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.198870+0000","last_change":"2024-09-10T18:46:45.285074+0000","last_active":"2024-09-10T18:46:53.198870+0000","last_peered":"2024-09-10T18:46:53.198870+0000","last_clean":"2024-09-10T18:46:53.198870+0000","last_became_active":"2024-09-10T18:45:24.267251+0000","last_became_peered":"2024-09-10T18:45:24.267251+0000","last_unstale":"2024-09-10T18:46:53.198870+0000","last_undegraded":"2024-09-10T18:46:53.198870+0000","last_fullsized":"2024-09-10T18:46:53.198870+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:45.285015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:45.285015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:56.843562+0000","scrub_duration":0.001362142,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"56'1","reported_seq":54,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.396398+0000","last_change":"2024-09-10T18:46:42.139314+0000","last_active":"2024-09-10T18:46:53.396398+0000","last_peered":"2024-09-10T18:46:53.396398+0000","last_clean":"2024-09-10T18:46:53.396398+0000","last_became_active":"2024-09-10T18:45:17.966286+0000","last_became_peered":"2024-09-10T18:45:17.966286+0000","last_unstale":"2024-09-10T18:46:53.396398+0000","last_undegraded":"2024-09-10T18:46:53.396398+0000","last_fullsized":"2024-09-10T18:46:53.396398+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:46:42.139256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:42.139256+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:57.842540+0000","scrub_duration":0.0014138690000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"63'15","reported_seq":68,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.396137+0000","last_change":"2024-09-10T18:46:23.864133+0000","last_active":"2024-09-10T18:46:53.396137+0000","last_peered":"2024-09-10T18:46:53.396137+0000","last_clean":"2024-09-10T18:46:53.396137+0000","last_became_active":"2024-09-10T18:45:20.062728+0000","last_became_peered":"2024-09-10T18:45:20.062728+0000","last_unstale":"2024-09-10T18:46:53.396137+0000","last_undegraded":"2024-09-10T18:46:53.396137+0000","last_fullsized":"2024-09-10T18:46:53.396137+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:46:23.864075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:23.864075+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:29.106766+0000","scrub_duration":0.0020859780000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.396382+0000","last_change":"2024-09-10T18:45:22.060202+0000","last_active":"2024-09-10T18:46:53.396382+0000","last_peered":"2024-09-10T18:46:53.396382+0000","last_clean":"2024-09-10T18:46:53.396382+0000","last_became_active":"2024-09-10T18:45:22.059794+0000","last_became_peered":"2024-09-10T18:45:22.059794+0000","last_unstale":"2024-09-10T18:46:53.396382+0000","last_undegraded":"2024-09-10T18:46:53.396382+0000","last_fullsized":"2024-09-10T18:46:53.396382+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.194052+0000","last_change":"2024-09-10T18:45:24.264775+0000","last_active":"2024-09-10T18:46:53.194052+0000","last_peered":"2024-09-10T18:46:53.194052+0000","last_clean":"2024-09-10T18:46:53.194052+0000","last_became_active":"2024-09-10T18:45:24.264427+0000","last_became_peered":"2024-09-10T18:45:24.264427+0000","last_unstale":"2024-09-10T18:46:53.194052+0000","last_undegraded":"2024-09-10T18:46:53.194052+0000","last_fullsized":"2024-09-10T18:46:53.194052+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.195165+0000","last_change":"2024-09-10T18:46:47.734860+0000","last_active":"2024-09-10T18:46:53.195165+0000","last_peered":"2024-09-10T18:46:53.195165+0000","last_clean":"2024-09-10T18:46:53.195165+0000","last_became_active":"2024-09-10T18:45:17.963257+0000","last_became_peered":"2024-09-10T18:45:17.963257+0000","last_unstale":"2024-09-10T18:46:53.195165+0000","last_undegraded":"2024-09-10T18:46:53.195165+0000","last_fullsized":"2024-09-10T18:46:53.195165+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:47.734803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:47.734803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:05.847815+0000","scrub_duration":0.001381134,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"63'12","reported_seq":66,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.193824+0000","last_change":"2024-09-10T18:46:52.771640+0000","last_active":"2024-09-10T18:46:53.193824+0000","last_peered":"2024-09-10T18:46:53.193824+0000","last_clean":"2024-09-10T18:46:53.193824+0000","last_became_active":"2024-09-10T18:45:20.058391+0000","last_became_peered":"2024-09-10T18:45:20.058391+0000","last_unstale":"2024-09-10T18:46:53.193824+0000","last_undegraded":"2024-09-10T18:46:53.193824+0000","last_fullsized":"2024-09-10T18:46:53.193824+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:46:52.771570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:52.771570+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:04.251299+0000","scrub_duration":0.0021490089999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T18:46:53.199042+0000","last_change":"2024-09-10T18:46:34.283979+0000","last_active":"2024-09-10T18:46:53.199042+0000","last_peered":"2024-09-10T18:46:53.199042+0000","last_clean":"2024-09-10T18:46:53.199042+0000","last_became_active":"2024-09-10T18:45:22.073250+0000","last_became_peered":"2024-09-10T18:45:22.073250+0000","last_unstale":"2024-09-10T18:46:53.199042+0000","last_undegraded":"2024-09-10T18:46:53.199042+0000","last_fullsized":"2024-09-10T18:46:53.199042+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:34.283901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:34.283901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:34.686745+0000","scrub_duration":0.002360334,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65,"ondisk_log_size":65,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3592,"num_objects":170,"num_object_clones":0,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":672,"num_read_kb":438,"num_write":401,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":90,"num_read_kb":85,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299416,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298544,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446608,"statfs":{"total":95995035648,"available":95689326592,"internally_reserved":0,"allocated":1220608,"data_stored":875458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.61599999999999999}]},{"osd":1,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.58099999999999996}]},{"osd":2,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.625},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.60199999999999998}]},{"osd":3,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.64000000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.44}]},{"osd":5,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.50800000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.497},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.47999999999999998}]}]},{"osd":6,"up_from":43,"seq":184683593757,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298536,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446616,"statfs":{"total":95995035648,"available":95689334784,"internally_reserved":0,"allocated":1208320,"data_stored":875534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.46300000000000002,"15min":0.45900000000000002},"min":{"1min":0.32900000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.496,"15min":0.49199999999999999},"min":{"1min":0.32800000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.40600000000000003}]},{"osd":1,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.315,"5min":0.249,"15min":0.249},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.46700000000000003,"15min":0.46300000000000002},"min":{"1min":0.33700000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.65900000000000003,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.49199999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.48299999999999998,"15min":0.48099999999999998},"min":{"1min":0.34000000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.49299999999999999,"15min":0.49099999999999999},"min":{"1min":0.33600000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68700000000000006,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.47999999999999998}]},{"osd":3,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.48399999999999999,"15min":0.48299999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.51200000000000001,"15min":0.51000000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.51000000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.39700000000000002,"15min":0.38800000000000001},"min":{"1min":0.26900000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.36799999999999999}]},{"osd":5,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.41899999999999998,"15min":0.41499999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.42499999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.34799999999999998}]},{"osd":7,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.53300000000000003}]}]},{"osd":5,"up_from":37,"seq":158913789985,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297944,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447208,"statfs":{"total":95995035648,"available":95689940992,"internally_reserved":0,"allocated":602112,"data_stored":283669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.52200000000000002,"15min":0.52600000000000002},"min":{"1min":0.30299999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70099999999999996,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.58299999999999996,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.50600000000000001}]},{"osd":1,"last update":"Tue Sep 10 18:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.55200000000000005,"15min":0.56000000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.69699999999999995,"5min":0.87,"15min":0.87},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.497,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.64200000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.57199999999999995}]},{"osd":2,"last update":"Tue Sep 10 18:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.51800000000000002,"15min":0.52000000000000002},"min":{"1min":0.33300000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.71299999999999997,"5min":0.755,"15min":0.755},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53600000000000003,"15min":0.54100000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79600000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.44500000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52500000000000002,"15min":0.52700000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76100000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.54200000000000004,"15min":0.54600000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80100000000000005,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.495}]},{"osd":4,"last update":"Tue Sep 10 18:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.64500000000000002,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.40500000000000003,"15min":0.40300000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.65700000000000003,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.377}]},{"osd":6,"last update":"Tue Sep 10 18:46:44 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.42199999999999999,"15min":0.41599999999999998},"min":{"1min":0.29299999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.67500000000000004,"5min":0.753,"15min":0.753},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.46500000000000002,"15min":0.46100000000000002},"min":{"1min":0.36199999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.71199999999999997,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.34300000000000003}]},{"osd":7,"last update":"Tue Sep 10 18:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.309},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.28000000000000003}]}]},{"osd":4,"up_from":30,"seq":128849018919,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297968,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447184,"statfs":{"total":95995035648,"available":95689916416,"internally_reserved":0,"allocated":626688,"data_stored":284626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.36399999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.54000000000000004,"15min":0.54200000000000004},"min":{"1min":0.34399999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.49099999999999999}]},{"osd":1,"last update":"Tue Sep 10 18:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.57999999999999996,"15min":0.57599999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.609,"5min":2.609,"15min":2.609},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.54700000000000004,"15min":0.53200000000000003},"min":{"1min":0.39700000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.589,"5min":2.589,"15min":2.589},"last":0.45300000000000001}]},{"osd":2,"last update":"Tue Sep 10 18:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.55700000000000005,"15min":0.54400000000000004},"min":{"1min":0.40500000000000003,"5min":0.379,"15min":0.379},"max":{"1min":2.6030000000000002,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.54800000000000004,"15min":0.55400000000000005},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64000000000000001,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.42399999999999999}]},{"osd":3,"last update":"Tue Sep 10 18:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52800000000000002,"15min":0.53000000000000003},"min":{"1min":0.36699999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68300000000000005,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.497,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.434}]},{"osd":5,"last update":"Tue Sep 10 18:46:22 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.35899999999999999,"15min":0.34999999999999998},"min":{"1min":0.26800000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.44,"5min":0.371,"15min":0.35899999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.29799999999999999}]},{"osd":6,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.42999999999999999,"15min":0.42399999999999999},"min":{"1min":0.311,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.68799999999999994,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.40100000000000002,"15min":0.39100000000000001},"min":{"1min":0.34799999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.316}]},{"osd":7,"last update":"Tue Sep 10 18:46:08 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.36499999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182444,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297992,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447160,"statfs":{"total":95995035648,"available":95689891840,"internally_reserved":0,"allocated":651264,"data_stored":290997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:35 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.378,"15min":0.38400000000000001},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.40100000000000002,"15min":0.40300000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.38500000000000001}]},{"osd":1,"last update":"Tue Sep 10 18:46:35 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.40400000000000003,"15min":0.40899999999999997},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40899999999999997,"15min":0.41499999999999998},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.63800000000000001,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.45900000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:46:35 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.43099999999999999,"15min":0.438},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.377,"5min":0.41999999999999998,"15min":0.433},"min":{"1min":0.27200000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.53600000000000003,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.47699999999999998}]},{"osd":4,"last update":"Tue Sep 10 18:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54000000000000004,"15min":0.54300000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.67200000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.56599999999999995,"15min":0.57199999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.71699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.63200000000000001}]},{"osd":5,"last update":"Tue Sep 10 18:46:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39700000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.70299999999999996,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.41399999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80800000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.65400000000000003}]},{"osd":6,"last update":"Tue Sep 10 18:46:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.56000000000000005,"15min":0.56299999999999994},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79100000000000004,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55400000000000005,"15min":0.55600000000000005},"min":{"1min":0.40799999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.68899999999999995}]},{"osd":7,"last update":"Tue Sep 10 18:46:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.79700000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411376,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297944,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447208,"statfs":{"total":95995035648,"available":95689940992,"internally_reserved":0,"allocated":606208,"data_stored":284104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.376,"15min":0.35899999999999999},"min":{"1min":0.246,"5min":0.223,"15min":0.223},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.38300000000000001,"15min":0.371},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.29399999999999998}]},{"osd":1,"last update":"Tue Sep 10 18:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.375,"15min":0.36699999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.42499999999999999,"15min":0.41299999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.34399999999999997}]},{"osd":3,"last update":"Tue Sep 10 18:46:36 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46800000000000003,"15min":0.46500000000000002},"min":{"1min":0.30099999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.42899999999999999,"15min":0.43099999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.32400000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:45:59 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54900000000000004,"15min":0.54800000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.375}]},{"osd":5,"last update":"Tue Sep 10 18:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56100000000000005,"15min":0.55800000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.42899999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.39900000000000002}]},{"osd":6,"last update":"Tue Sep 10 18:46:49 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58999999999999997,"15min":0.58599999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.46200000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.35599999999999998}]},{"osd":7,"last update":"Tue Sep 10 18:46:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.435}]}]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297992,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447160,"statfs":{"total":95995035648,"available":95689891840,"internally_reserved":0,"allocated":651264,"data_stored":284900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:48 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34399999999999997,"15min":0.32000000000000001},"min":{"1min":0.245,"5min":0.224,"15min":0.224},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.36599999999999999,"15min":0.33600000000000002},"min":{"1min":0.252,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.53700000000000003,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.44700000000000001}]},{"osd":2,"last update":"Tue Sep 10 18:46:11 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.372,"15min":0.36099999999999999},"min":{"1min":0.27900000000000003,"5min":0.248,"15min":0.248},"max":{"1min":0.47199999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.39000000000000001,"15min":0.38100000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.49099999999999999,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.35099999999999998}]},{"osd":3,"last update":"Tue Sep 10 18:46:34 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.45400000000000001,"15min":0.45800000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.42299999999999999,"15min":0.42199999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.55400000000000005,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.40500000000000003}]},{"osd":4,"last update":"Tue Sep 10 18:45:56 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.56299999999999994,"15min":0.55600000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.495},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.54100000000000004,"15min":0.53300000000000003},"min":{"1min":0.41099999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.38900000000000001}]},{"osd":5,"last update":"Tue Sep 10 18:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60299999999999998,"15min":0.60399999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63700000000000001,"15min":0.64100000000000001},"min":{"1min":0.44600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93300000000000005,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.54500000000000004}]},{"osd":6,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.46300000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.66400000000000003,"15min":0.67100000000000004},"min":{"1min":0.41999999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82299999999999995,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.51500000000000001}]},{"osd":7,"last update":"Tue Sep 10 18:46:11 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.59999999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738426,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298540,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446612,"statfs":{"total":95995035648,"available":95689330688,"internally_reserved":0,"allocated":1212416,"data_stored":874268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 18:46:52 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.32500000000000001,"15min":0.30299999999999999},"min":{"1min":0.219,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.58799999999999997,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33700000000000002,"15min":0.315},"min":{"1min":0.221,"5min":0.187,"15min":0.187},"max":{"1min":0.59399999999999997,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.27400000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.40300000000000002,"15min":0.38900000000000001},"min":{"1min":0.30299999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.376,"15min":0.36199999999999999},"min":{"1min":0.26600000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.35399999999999998}]},{"osd":3,"last update":"Tue Sep 10 18:46:31 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.39500000000000002,"15min":0.39800000000000002},"min":{"1min":0.26100000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.66100000000000003,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.312},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.41199999999999998,"15min":0.41299999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.29599999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.56999999999999995,"15min":0.53600000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":4.6189999999999998,"5min":4.6189999999999998,"15min":4.6189999999999998},"last":0.371}]},{"osd":5,"last update":"Tue Sep 10 18:46:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.42999999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.58699999999999997,"15min":0.55800000000000005},"min":{"1min":0.42699999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":4.6429999999999998,"5min":4.6429999999999998,"15min":4.6429999999999998},"last":1.6679999999999999}]},{"osd":6,"last update":"Tue Sep 10 18:46:42 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56100000000000005,"15min":0.55500000000000005},"min":{"1min":0.42099999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.746,"15min":0.77300000000000002},"min":{"1min":0.40899999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.878,"5min":4.6630000000000003,"15min":4.6630000000000003},"last":1.6539999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.46800000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T18:46:57.968 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T18:46:58.232 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:46:58.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:58 smithi103 ceph-mon[33163]: pgmap v47: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:58.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:58 smithi103 ceph-mon[33163]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:46:58.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:58 smithi103 ceph-mon[33163]: 6.1f scrub starts 2024-09-10T18:46:58.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:58 smithi103 ceph-mon[33163]: 6.1f scrub ok 2024-09-10T18:46:58.567 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:58 smithi018 ceph-mon[29336]: pgmap v47: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:58.567 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:58 smithi018 ceph-mon[29336]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:46:58.567 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:58 smithi018 ceph-mon[29336]: 6.1f scrub starts 2024-09-10T18:46:58.567 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:58 smithi018 ceph-mon[29336]: 6.1f scrub ok 2024-09-10T18:46:58.567 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:58 smithi018 ceph-mon[40196]: pgmap v47: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:58.567 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:58 smithi018 ceph-mon[40196]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:46:58.568 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:58 smithi018 ceph-mon[40196]: 6.1f scrub starts 2024-09-10T18:46:58.568 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:58 smithi018 ceph-mon[40196]: 6.1f scrub ok 2024-09-10T18:46:59.694 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:46:59 smithi018 bash[73688]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-10T18:46:59.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:59 smithi018 ceph-mon[29336]: 4.f scrub starts 2024-09-10T18:46:59.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:59 smithi018 ceph-mon[29336]: 4.f scrub ok 2024-09-10T18:46:59.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:59 smithi018 ceph-mon[29336]: 4.2 deep-scrub starts 2024-09-10T18:46:59.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:59 smithi018 ceph-mon[29336]: 4.2 deep-scrub ok 2024-09-10T18:46:59.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:59 smithi018 ceph-mon[29336]: 5.5 scrub starts 2024-09-10T18:46:59.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:59 smithi018 ceph-mon[29336]: 5.5 scrub ok 2024-09-10T18:46:59.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:46:59 smithi018 ceph-mon[29336]: pgmap v48: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:59.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:59 smithi018 ceph-mon[40196]: 4.f scrub starts 2024-09-10T18:46:59.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:59 smithi018 ceph-mon[40196]: 4.f scrub ok 2024-09-10T18:46:59.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:59 smithi018 ceph-mon[40196]: 4.2 deep-scrub starts 2024-09-10T18:46:59.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:59 smithi018 ceph-mon[40196]: 4.2 deep-scrub ok 2024-09-10T18:46:59.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:59 smithi018 ceph-mon[40196]: 5.5 scrub starts 2024-09-10T18:46:59.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:59 smithi018 ceph-mon[40196]: 5.5 scrub ok 2024-09-10T18:46:59.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:46:59 smithi018 ceph-mon[40196]: pgmap v48: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:59.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:59 smithi103 ceph-mon[33163]: 4.f scrub starts 2024-09-10T18:46:59.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:59 smithi103 ceph-mon[33163]: 4.f scrub ok 2024-09-10T18:46:59.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:59 smithi103 ceph-mon[33163]: 4.2 deep-scrub starts 2024-09-10T18:46:59.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:59 smithi103 ceph-mon[33163]: 4.2 deep-scrub ok 2024-09-10T18:46:59.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:59 smithi103 ceph-mon[33163]: 5.5 scrub starts 2024-09-10T18:46:59.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:59 smithi103 ceph-mon[33163]: 5.5 scrub ok 2024-09-10T18:46:59.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:46:59 smithi103 ceph-mon[33163]: pgmap v48: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:47:00.025 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:46:59 smithi018 bash[73688]: Writing manifest to image destination 2024-09-10T18:47:00.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:47:00 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:47:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:47:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:00 smithi103 ceph-mon[33163]: 5.7 scrub starts 2024-09-10T18:47:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:00 smithi103 ceph-mon[33163]: 5.7 scrub ok 2024-09-10T18:47:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:00 smithi103 ceph-mon[33163]: 4.12 scrub starts 2024-09-10T18:47:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:00 smithi103 ceph-mon[33163]: 4.12 scrub ok 2024-09-10T18:47:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:00 smithi103 ceph-mon[33163]: 6.5 scrub starts 2024-09-10T18:47:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:00 smithi103 ceph-mon[33163]: 6.5 scrub ok 2024-09-10T18:47:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:00 smithi103 ceph-mon[33163]: 4.18 deep-scrub starts 2024-09-10T18:47:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:00 smithi103 ceph-mon[33163]: 4.18 deep-scrub ok 2024-09-10T18:47:00.548 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-mon[40196]: 5.7 scrub starts 2024-09-10T18:47:00.549 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-mon[40196]: 5.7 scrub ok 2024-09-10T18:47:00.549 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-mon[40196]: 4.12 scrub starts 2024-09-10T18:47:00.549 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-mon[40196]: 4.12 scrub ok 2024-09-10T18:47:00.549 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-mon[40196]: 6.5 scrub starts 2024-09-10T18:47:00.549 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-mon[40196]: 6.5 scrub ok 2024-09-10T18:47:00.549 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-mon[40196]: 4.18 deep-scrub starts 2024-09-10T18:47:00.550 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-mon[40196]: 4.18 deep-scrub ok 2024-09-10T18:47:00.550 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-mon[29336]: 5.7 scrub starts 2024-09-10T18:47:00.550 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-mon[29336]: 5.7 scrub ok 2024-09-10T18:47:00.551 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-mon[29336]: 4.12 scrub starts 2024-09-10T18:47:00.551 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-mon[29336]: 4.12 scrub ok 2024-09-10T18:47:00.551 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-mon[29336]: 6.5 scrub starts 2024-09-10T18:47:00.551 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-mon[29336]: 6.5 scrub ok 2024-09-10T18:47:00.552 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-mon[29336]: 4.18 deep-scrub starts 2024-09-10T18:47:00.552 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-mon[29336]: 4.18 deep-scrub ok 2024-09-10T18:47:00.552 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 podman[73688]: 2024-09-10 18:47:00.224197867 +0000 UTC m=+3.266416903 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-10T18:47:00.552 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 podman[73688]: 2024-09-10 18:47:00.431718382 +0000 UTC m=+3.473937417 container create 9ca2531486a95de2ebb44bdbbbcd0d1f8ffcde49b99d3c88aa11791d66e5cdef (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T18:47:00.938 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 podman[73688]: 2024-09-10 18:47:00.633992657 +0000 UTC m=+3.676211702 container init 9ca2531486a95de2ebb44bdbbbcd0d1f8ffcde49b99d3c88aa11791d66e5cdef (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T18:47:00.939 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 podman[73688]: 2024-09-10 18:47:00.637444539 +0000 UTC m=+3.679663575 container start 9ca2531486a95de2ebb44bdbbbcd0d1f8ffcde49b99d3c88aa11791d66e5cdef (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T18:47:00.939 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.641Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-10T18:47:00.939 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.641Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-10T18:47:00.939 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.642Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-10T18:47:00.939 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.642Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-10T18:47:00.939 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.642Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-10T18:47:00.939 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.642Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-10T18:47:00.939 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-10T18:47:00.939 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=arp 2024-09-10T18:47:00.939 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-10T18:47:00.939 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-10T18:47:00.940 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-10T18:47:00.940 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-10T18:47:00.940 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-10T18:47:00.940 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-10T18:47:00.940 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-10T18:47:00.940 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-10T18:47:00.941 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=edac 2024-09-10T18:47:00.941 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-10T18:47:00.941 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-10T18:47:00.941 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-10T18:47:00.941 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-10T18:47:00.941 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-10T18:47:00.941 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-10T18:47:00.941 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-10T18:47:00.941 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-10T18:47:00.941 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-10T18:47:00.942 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-10T18:47:00.942 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-10T18:47:00.942 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-10T18:47:00.942 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-10T18:47:00.942 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-10T18:47:00.942 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-10T18:47:00.942 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-10T18:47:00.942 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=os 2024-09-10T18:47:00.942 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-10T18:47:00.942 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-10T18:47:00.942 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-10T18:47:00.942 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-10T18:47:00.942 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-10T18:47:00.943 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-10T18:47:00.943 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-10T18:47:00.943 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=stat 2024-09-10T18:47:00.943 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-10T18:47:00.943 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-10T18:47:00.943 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-10T18:47:00.943 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=time 2024-09-10T18:47:00.943 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-10T18:47:00.943 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=uname 2024-09-10T18:47:00.944 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-10T18:47:00.944 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-10T18:47:00.944 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.643Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-10T18:47:00.944 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.644Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-10T18:47:00.944 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a[73902]: ts=2024-09-10T18:47:00.644Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-10T18:47:00.944 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 bash[73688]: 9ca2531486a95de2ebb44bdbbbcd0d1f8ffcde49b99d3c88aa11791d66e5cdef 2024-09-10T18:47:00.944 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 18:47:00 smithi018 systemd[1]: Started Ceph node-exporter.a for ba327f6c-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:47:01.327 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:47:01.334 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-09-10T18:47:01.686 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:01 smithi018 ceph-mon[29336]: 2.1f scrub starts 2024-09-10T18:47:01.686 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:01 smithi018 ceph-mon[29336]: 2.1f scrub ok 2024-09-10T18:47:01.686 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:01 smithi018 ceph-mon[29336]: 2.5 scrub starts 2024-09-10T18:47:01.687 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:01 smithi018 ceph-mon[29336]: 2.5 scrub ok 2024-09-10T18:47:01.687 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:01 smithi018 ceph-mon[29336]: 6.c scrub starts 2024-09-10T18:47:01.687 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:01 smithi018 ceph-mon[29336]: 6.c scrub ok 2024-09-10T18:47:01.687 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:01 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:01.687 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:01 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:01.687 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:01 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:01.687 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:01 smithi018 ceph-mon[29336]: Deploying daemon node-exporter.b on smithi103 2024-09-10T18:47:01.687 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:01 smithi018 ceph-mon[29336]: 6.1a scrub starts 2024-09-10T18:47:01.688 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:01 smithi018 ceph-mon[29336]: 6.1a scrub ok 2024-09-10T18:47:01.688 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:01 smithi018 ceph-mon[29336]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s; 28 B/s, 2 objects/s recovering 2024-09-10T18:47:01.688 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:01 smithi018 ceph-mon[29336]: from='client.24608 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:47:01.688 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:01 smithi018 ceph-mon[40196]: 2.1f scrub starts 2024-09-10T18:47:01.688 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:01 smithi018 ceph-mon[40196]: 2.1f scrub ok 2024-09-10T18:47:01.688 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:01 smithi018 ceph-mon[40196]: 2.5 scrub starts 2024-09-10T18:47:01.689 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:01 smithi018 ceph-mon[40196]: 2.5 scrub ok 2024-09-10T18:47:01.689 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:01 smithi018 ceph-mon[40196]: 6.c scrub starts 2024-09-10T18:47:01.689 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:01 smithi018 ceph-mon[40196]: 6.c scrub ok 2024-09-10T18:47:01.689 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:01 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:01.689 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:01 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:01.689 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:01 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:01.689 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:01 smithi018 ceph-mon[40196]: Deploying daemon node-exporter.b on smithi103 2024-09-10T18:47:01.690 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:01 smithi018 ceph-mon[40196]: 6.1a scrub starts 2024-09-10T18:47:01.690 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:01 smithi018 ceph-mon[40196]: 6.1a scrub ok 2024-09-10T18:47:01.690 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:01 smithi018 ceph-mon[40196]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s; 28 B/s, 2 objects/s recovering 2024-09-10T18:47:01.690 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:01 smithi018 ceph-mon[40196]: from='client.24608 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:47:01.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:01 smithi103 ceph-mon[33163]: 2.1f scrub starts 2024-09-10T18:47:01.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:01 smithi103 ceph-mon[33163]: 2.1f scrub ok 2024-09-10T18:47:01.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:01 smithi103 ceph-mon[33163]: 2.5 scrub starts 2024-09-10T18:47:01.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:01 smithi103 ceph-mon[33163]: 2.5 scrub ok 2024-09-10T18:47:01.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:01 smithi103 ceph-mon[33163]: 6.c scrub starts 2024-09-10T18:47:01.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:01 smithi103 ceph-mon[33163]: 6.c scrub ok 2024-09-10T18:47:01.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:01 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:01.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:01 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:01.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:01 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:01.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:01 smithi103 ceph-mon[33163]: Deploying daemon node-exporter.b on smithi103 2024-09-10T18:47:01.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:01 smithi103 ceph-mon[33163]: 6.1a scrub starts 2024-09-10T18:47:01.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:01 smithi103 ceph-mon[33163]: 6.1a scrub ok 2024-09-10T18:47:01.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:01 smithi103 ceph-mon[33163]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s; 28 B/s, 2 objects/s recovering 2024-09-10T18:47:01.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:01 smithi103 ceph-mon[33163]: from='client.24608 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:47:01.774 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:01 smithi103 bash[63812]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-10T18:47:02.074 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":49,"stamp":"2024-09-10T18:47:01.029063+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":948,"num_read_kb":805,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1180956,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":578,"ondisk_log_size":578,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":397,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385632,"kb_used_data":6800,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575584,"statfs":{"total":767960285184,"available":765517398016,"internally_reserved":0,"allocated":6963200,"data_stored":4151764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.836783"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.724426+0000","last_change":"2024-09-10T18:46:33.824057+0000","last_active":"2024-09-10T18:46:55.724426+0000","last_peered":"2024-09-10T18:46:55.724426+0000","last_clean":"2024-09-10T18:46:55.724426+0000","last_became_active":"2024-09-10T18:45:24.265488+0000","last_became_peered":"2024-09-10T18:45:24.265488+0000","last_unstale":"2024-09-10T18:46:55.724426+0000","last_undegraded":"2024-09-10T18:46:55.724426+0000","last_fullsized":"2024-09-10T18:46:55.724426+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:33.823989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:33.823989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:44.131457+0000","scrub_duration":0.0034389770000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:00.536103+0000","last_change":"2024-09-10T18:47:00.536064+0000","last_active":"2024-09-10T18:47:00.536103+0000","last_peered":"2024-09-10T18:47:00.536103+0000","last_clean":"2024-09-10T18:47:00.536103+0000","last_became_active":"2024-09-10T18:45:17.959475+0000","last_became_peered":"2024-09-10T18:45:17.959475+0000","last_unstale":"2024-09-10T18:47:00.536103+0000","last_undegraded":"2024-09-10T18:47:00.536103+0000","last_fullsized":"2024-09-10T18:47:00.536103+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:00.535992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:47:00.535992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:13.246010+0000","scrub_duration":0.001731841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"63'15","reported_seq":69,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.724433+0000","last_change":"2024-09-10T18:46:36.856804+0000","last_active":"2024-09-10T18:46:55.724433+0000","last_peered":"2024-09-10T18:46:55.724433+0000","last_clean":"2024-09-10T18:46:55.724433+0000","last_became_active":"2024-09-10T18:45:20.052531+0000","last_became_peered":"2024-09-10T18:45:20.052531+0000","last_unstale":"2024-09-10T18:46:55.724433+0000","last_undegraded":"2024-09-10T18:46:55.724433+0000","last_fullsized":"2024-09-10T18:46:55.724433+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:46:36.856740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:36.856740+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:03.997270+0000","scrub_duration":0.0018131040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.719487+0000","last_change":"2024-09-10T18:46:40.313306+0000","last_active":"2024-09-10T18:46:55.719487+0000","last_peered":"2024-09-10T18:46:55.719487+0000","last_clean":"2024-09-10T18:46:55.719487+0000","last_became_active":"2024-09-10T18:45:22.065619+0000","last_became_peered":"2024-09-10T18:45:22.065619+0000","last_unstale":"2024-09-10T18:46:55.719487+0000","last_undegraded":"2024-09-10T18:46:55.719487+0000","last_fullsized":"2024-09-10T18:46:55.719487+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:40.313218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:40.313218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:52.103586+0000","scrub_duration":0.0017922750000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.725424+0000","last_change":"2024-09-10T18:46:18.890844+0000","last_active":"2024-09-10T18:46:55.725424+0000","last_peered":"2024-09-10T18:46:55.725424+0000","last_clean":"2024-09-10T18:46:55.725424+0000","last_became_active":"2024-09-10T18:45:17.966153+0000","last_became_peered":"2024-09-10T18:45:17.966153+0000","last_unstale":"2024-09-10T18:46:55.725424+0000","last_undegraded":"2024-09-10T18:46:55.725424+0000","last_fullsized":"2024-09-10T18:46:55.725424+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:18.890702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:18.890702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:47.062002+0000","scrub_duration":0.0019382589999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"83'13","reported_seq":64,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:59.906764+0000","last_change":"2024-09-10T18:46:59.906707+0000","last_active":"2024-09-10T18:46:59.906764+0000","last_peered":"2024-09-10T18:46:59.906764+0000","last_clean":"2024-09-10T18:46:59.906764+0000","last_became_active":"2024-09-10T18:45:20.061266+0000","last_became_peered":"2024-09-10T18:45:20.061266+0000","last_unstale":"2024-09-10T18:46:59.906764+0000","last_undegraded":"2024-09-10T18:46:59.906764+0000","last_fullsized":"2024-09-10T18:46:59.906764+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'13","last_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_clean_scrub_stamp":"2024-09-10T18:46:59.906634+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:29.738998+0000","scrub_duration":0.003988892,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.722125+0000","last_change":"2024-09-10T18:45:22.073139+0000","last_active":"2024-09-10T18:46:55.722125+0000","last_peered":"2024-09-10T18:46:55.722125+0000","last_clean":"2024-09-10T18:46:55.722125+0000","last_became_active":"2024-09-10T18:45:22.072822+0000","last_became_peered":"2024-09-10T18:45:22.072822+0000","last_unstale":"2024-09-10T18:46:55.722125+0000","last_undegraded":"2024-09-10T18:46:55.722125+0000","last_fullsized":"2024-09-10T18:46:55.722125+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:00.946332+0000","last_change":"2024-09-10T18:47:00.946309+0000","last_active":"2024-09-10T18:47:00.946332+0000","last_peered":"2024-09-10T18:47:00.946332+0000","last_clean":"2024-09-10T18:47:00.946332+0000","last_became_active":"2024-09-10T18:45:24.081002+0000","last_became_peered":"2024-09-10T18:45:24.081002+0000","last_unstale":"2024-09-10T18:47:00.946332+0000","last_undegraded":"2024-09-10T18:47:00.946332+0000","last_fullsized":"2024-09-10T18:47:00.946332+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:00.946255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:00.946255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:14.872519+0000","scrub_duration":0.001207629,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.724266+0000","last_change":"2024-09-10T18:46:46.605919+0000","last_active":"2024-09-10T18:46:55.724266+0000","last_peered":"2024-09-10T18:46:55.724266+0000","last_clean":"2024-09-10T18:46:55.724266+0000","last_became_active":"2024-09-10T18:45:17.961097+0000","last_became_peered":"2024-09-10T18:45:17.961097+0000","last_unstale":"2024-09-10T18:46:55.724266+0000","last_undegraded":"2024-09-10T18:46:55.724266+0000","last_fullsized":"2024-09-10T18:46:55.724266+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:46.605814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:46.605814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:51.831114+0000","scrub_duration":0.001570625,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"63'5","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.719685+0000","last_change":"2024-09-10T18:45:20.058010+0000","last_active":"2024-09-10T18:46:55.719685+0000","last_peered":"2024-09-10T18:46:55.719685+0000","last_clean":"2024-09-10T18:46:55.719685+0000","last_became_active":"2024-09-10T18:45:20.057567+0000","last_became_peered":"2024-09-10T18:45:20.057567+0000","last_unstale":"2024-09-10T18:46:55.719685+0000","last_undegraded":"2024-09-10T18:46:55.719685+0000","last_fullsized":"2024-09-10T18:46:55.719685+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.724304+0000","last_change":"2024-09-10T18:46:22.547871+0000","last_active":"2024-09-10T18:46:55.724304+0000","last_peered":"2024-09-10T18:46:55.724304+0000","last_clean":"2024-09-10T18:46:55.724304+0000","last_became_active":"2024-09-10T18:45:22.076575+0000","last_became_peered":"2024-09-10T18:45:22.076575+0000","last_unstale":"2024-09-10T18:46:55.724304+0000","last_undegraded":"2024-09-10T18:46:55.724304+0000","last_fullsized":"2024-09-10T18:46:55.724304+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:22.547800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:22.547800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:32.775669+0000","scrub_duration":0.001880622,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.721776+0000","last_change":"2024-09-10T18:46:36.158152+0000","last_active":"2024-09-10T18:46:55.721776+0000","last_peered":"2024-09-10T18:46:55.721776+0000","last_clean":"2024-09-10T18:46:55.721776+0000","last_became_active":"2024-09-10T18:45:24.079534+0000","last_became_peered":"2024-09-10T18:45:24.079534+0000","last_unstale":"2024-09-10T18:46:55.721776+0000","last_undegraded":"2024-09-10T18:46:55.721776+0000","last_fullsized":"2024-09-10T18:46:55.721776+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:36.158087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:36.158087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:41.325875+0000","scrub_duration":0.0014477100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.724122+0000","last_change":"2024-09-10T18:46:50.666447+0000","last_active":"2024-09-10T18:46:55.724122+0000","last_peered":"2024-09-10T18:46:55.724122+0000","last_clean":"2024-09-10T18:46:55.724122+0000","last_became_active":"2024-09-10T18:45:17.963121+0000","last_became_peered":"2024-09-10T18:45:17.963121+0000","last_unstale":"2024-09-10T18:46:55.724122+0000","last_undegraded":"2024-09-10T18:46:55.724122+0000","last_fullsized":"2024-09-10T18:46:55.724122+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:50.666379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.666379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:06.539690+0000","scrub_duration":0.0016221689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"63'9","reported_seq":60,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720234+0000","last_change":"2024-09-10T18:46:55.069893+0000","last_active":"2024-09-10T18:46:55.720234+0000","last_peered":"2024-09-10T18:46:55.720234+0000","last_clean":"2024-09-10T18:46:55.720234+0000","last_became_active":"2024-09-10T18:45:20.052652+0000","last_became_peered":"2024-09-10T18:45:20.052652+0000","last_unstale":"2024-09-10T18:46:55.720234+0000","last_undegraded":"2024-09-10T18:46:55.720234+0000","last_fullsized":"2024-09-10T18:46:55.720234+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:46:55.069833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:55.069833+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:12.396664+0000","scrub_duration":0.0016685510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.722124+0000","last_change":"2024-09-10T18:46:31.024322+0000","last_active":"2024-09-10T18:46:55.722124+0000","last_peered":"2024-09-10T18:46:55.722124+0000","last_clean":"2024-09-10T18:46:55.722124+0000","last_became_active":"2024-09-10T18:45:22.068326+0000","last_became_peered":"2024-09-10T18:45:22.068326+0000","last_unstale":"2024-09-10T18:46:55.722124+0000","last_undegraded":"2024-09-10T18:46:55.722124+0000","last_fullsized":"2024-09-10T18:46:55.722124+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:31.024255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:31.024255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:50.407493+0000","scrub_duration":0.00152395,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.719285+0000","last_change":"2024-09-10T18:45:24.081968+0000","last_active":"2024-09-10T18:46:55.719285+0000","last_peered":"2024-09-10T18:46:55.719285+0000","last_clean":"2024-09-10T18:46:55.719285+0000","last_became_active":"2024-09-10T18:45:24.081488+0000","last_became_peered":"2024-09-10T18:45:24.081488+0000","last_unstale":"2024-09-10T18:46:55.719285+0000","last_undegraded":"2024-09-10T18:46:55.719285+0000","last_fullsized":"2024-09-10T18:46:55.719285+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"78'25","reported_seq":80,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720049+0000","last_change":"2024-09-10T18:46:44.316200+0000","last_active":"2024-09-10T18:46:55.720049+0000","last_peered":"2024-09-10T18:46:55.720049+0000","last_clean":"2024-09-10T18:46:55.720049+0000","last_became_active":"2024-09-10T18:45:20.057019+0000","last_became_peered":"2024-09-10T18:45:20.057019+0000","last_unstale":"2024-09-10T18:46:55.720049+0000","last_undegraded":"2024-09-10T18:46:55.720049+0000","last_fullsized":"2024-09-10T18:46:55.720049+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'25","last_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_deep_scrub":"78'25","last_deep_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_clean_scrub_stamp":"2024-09-10T18:46:44.316146+0000","objects_scrubbed":8,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:06.777730+0000","scrub_duration":0.0046415040000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723847+0000","last_change":"2024-09-10T18:46:47.595182+0000","last_active":"2024-09-10T18:46:55.723847+0000","last_peered":"2024-09-10T18:46:55.723847+0000","last_clean":"2024-09-10T18:46:55.723847+0000","last_became_active":"2024-09-10T18:45:17.963732+0000","last_became_peered":"2024-09-10T18:45:17.963732+0000","last_unstale":"2024-09-10T18:46:55.723847+0000","last_undegraded":"2024-09-10T18:46:55.723847+0000","last_fullsized":"2024-09-10T18:46:55.723847+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:47.595114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:47.595114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:53.271455+0000","scrub_duration":0.0014913249999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.722651+0000","last_change":"2024-09-10T18:45:22.071993+0000","last_active":"2024-09-10T18:46:55.722651+0000","last_peered":"2024-09-10T18:46:55.722651+0000","last_clean":"2024-09-10T18:46:55.722651+0000","last_became_active":"2024-09-10T18:45:22.071443+0000","last_became_peered":"2024-09-10T18:45:22.071443+0000","last_unstale":"2024-09-10T18:46:55.722651+0000","last_undegraded":"2024-09-10T18:46:55.722651+0000","last_fullsized":"2024-09-10T18:46:55.722651+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720517+0000","last_change":"2024-09-10T18:46:35.738217+0000","last_active":"2024-09-10T18:46:55.720517+0000","last_peered":"2024-09-10T18:46:55.720517+0000","last_clean":"2024-09-10T18:46:55.720517+0000","last_became_active":"2024-09-10T18:45:24.077897+0000","last_became_peered":"2024-09-10T18:45:24.077897+0000","last_unstale":"2024-09-10T18:46:55.720517+0000","last_undegraded":"2024-09-10T18:46:55.720517+0000","last_fullsized":"2024-09-10T18:46:55.720517+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:35.738143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:35.738143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:48.741747+0000","scrub_duration":0.0015293520000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"63'10","reported_seq":59,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720013+0000","last_change":"2024-09-10T18:46:31.281588+0000","last_active":"2024-09-10T18:46:55.720013+0000","last_peered":"2024-09-10T18:46:55.720013+0000","last_clean":"2024-09-10T18:46:55.720013+0000","last_became_active":"2024-09-10T18:45:20.053428+0000","last_became_peered":"2024-09-10T18:45:20.053428+0000","last_unstale":"2024-09-10T18:46:55.720013+0000","last_undegraded":"2024-09-10T18:46:55.720013+0000","last_fullsized":"2024-09-10T18:46:55.720013+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:46:31.281519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:31.281519+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:58.508120+0000","scrub_duration":0.0019095119999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.722685+0000","last_change":"2024-09-10T18:46:38.445417+0000","last_active":"2024-09-10T18:46:55.722685+0000","last_peered":"2024-09-10T18:46:55.722685+0000","last_clean":"2024-09-10T18:46:55.722685+0000","last_became_active":"2024-09-10T18:45:17.966833+0000","last_became_peered":"2024-09-10T18:45:17.966833+0000","last_unstale":"2024-09-10T18:46:55.722685+0000","last_undegraded":"2024-09-10T18:46:55.722685+0000","last_fullsized":"2024-09-10T18:46:55.722685+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:38.445362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:38.445362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:02.460484+0000","scrub_duration":0.0012292029999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"63'8","reported_seq":70,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.094380+0000","last_change":"2024-09-10T18:46:25.787936+0000","last_active":"2024-09-10T18:46:56.094380+0000","last_peered":"2024-09-10T18:46:56.094380+0000","last_clean":"2024-09-10T18:46:56.094380+0000","last_became_active":"2024-09-10T18:45:22.068453+0000","last_became_peered":"2024-09-10T18:45:22.068453+0000","last_unstale":"2024-09-10T18:46:56.094380+0000","last_undegraded":"2024-09-10T18:46:56.094380+0000","last_fullsized":"2024-09-10T18:46:56.094380+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:25.787877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:25.787877+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:54.201629+0000","scrub_duration":0.0018093530000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.719928+0000","last_change":"2024-09-10T18:46:32.265034+0000","last_active":"2024-09-10T18:46:55.719928+0000","last_peered":"2024-09-10T18:46:55.719928+0000","last_clean":"2024-09-10T18:46:55.719928+0000","last_became_active":"2024-09-10T18:45:24.081082+0000","last_became_peered":"2024-09-10T18:45:24.081082+0000","last_unstale":"2024-09-10T18:46:55.719928+0000","last_undegraded":"2024-09-10T18:46:55.719928+0000","last_fullsized":"2024-09-10T18:46:55.719928+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:32.264963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:32.264963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:50.207705+0000","scrub_duration":0.002378509,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"63'15","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:58.122323+0000","last_change":"2024-09-10T18:46:58.122279+0000","last_active":"2024-09-10T18:46:58.122323+0000","last_peered":"2024-09-10T18:46:58.122323+0000","last_clean":"2024-09-10T18:46:58.122323+0000","last_became_active":"2024-09-10T18:46:56.742946+0000","last_became_peered":"2024-09-10T18:46:56.742946+0000","last_unstale":"2024-09-10T18:46:58.122323+0000","last_undegraded":"2024-09-10T18:46:58.122323+0000","last_fullsized":"2024-09-10T18:46:58.122323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:46:58.122219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:58.122219+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:05.975000+0000","scrub_duration":0.001678094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.722329+0000","last_change":"2024-09-10T18:46:28.320761+0000","last_active":"2024-09-10T18:46:55.722329+0000","last_peered":"2024-09-10T18:46:55.722329+0000","last_clean":"2024-09-10T18:46:55.722329+0000","last_became_active":"2024-09-10T18:45:17.965409+0000","last_became_peered":"2024-09-10T18:45:17.965409+0000","last_unstale":"2024-09-10T18:46:55.722329+0000","last_undegraded":"2024-09-10T18:46:55.722329+0000","last_fullsized":"2024-09-10T18:46:55.722329+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_clean_scrub_stamp":"2024-09-10T18:46:28.320704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:44.092028+0000","scrub_duration":0.001393476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"63'8","reported_seq":70,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.094750+0000","last_change":"2024-09-10T18:46:39.300139+0000","last_active":"2024-09-10T18:46:56.094750+0000","last_peered":"2024-09-10T18:46:56.094750+0000","last_clean":"2024-09-10T18:46:56.094750+0000","last_became_active":"2024-09-10T18:45:22.062497+0000","last_became_peered":"2024-09-10T18:45:22.062497+0000","last_unstale":"2024-09-10T18:46:56.094750+0000","last_undegraded":"2024-09-10T18:46:56.094750+0000","last_fullsized":"2024-09-10T18:46:56.094750+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:39.300065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:39.300065+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:44.830898+0000","scrub_duration":0.001858934,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.722887+0000","last_change":"2024-09-10T18:46:42.057055+0000","last_active":"2024-09-10T18:46:55.722887+0000","last_peered":"2024-09-10T18:46:55.722887+0000","last_clean":"2024-09-10T18:46:55.722887+0000","last_became_active":"2024-09-10T18:45:24.082362+0000","last_became_peered":"2024-09-10T18:45:24.082362+0000","last_unstale":"2024-09-10T18:46:55.722887+0000","last_undegraded":"2024-09-10T18:46:55.722887+0000","last_fullsized":"2024-09-10T18:46:55.722887+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:42.056970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:42.056970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:50.036040+0000","scrub_duration":0.0016528479999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"63'11","reported_seq":85,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.747580+0000","last_change":"2024-09-10T18:46:56.747580+0000","last_active":"2024-09-10T18:46:56.747580+0000","last_peered":"2024-09-10T18:46:56.747580+0000","last_clean":"2024-09-10T18:46:56.747580+0000","last_became_active":"2024-09-10T18:46:56.741936+0000","last_became_peered":"2024-09-10T18:46:56.741936+0000","last_unstale":"2024-09-10T18:46:56.747580+0000","last_undegraded":"2024-09-10T18:46:56.747580+0000","last_fullsized":"2024-09-10T18:46:56.747580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:46:35.274660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:35.274660+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:42.198733+0000","scrub_duration":0.0019745819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723072+0000","last_change":"2024-09-10T18:46:39.520497+0000","last_active":"2024-09-10T18:46:55.723072+0000","last_peered":"2024-09-10T18:46:55.723072+0000","last_clean":"2024-09-10T18:46:55.723072+0000","last_became_active":"2024-09-10T18:45:17.966133+0000","last_became_peered":"2024-09-10T18:45:17.966133+0000","last_unstale":"2024-09-10T18:46:55.723072+0000","last_undegraded":"2024-09-10T18:46:55.723072+0000","last_fullsized":"2024-09-10T18:46:55.723072+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:39.520432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:39.520432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:49.040271+0000","scrub_duration":0.0013982510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723932+0000","last_change":"2024-09-10T18:46:48.116414+0000","last_active":"2024-09-10T18:46:55.723932+0000","last_peered":"2024-09-10T18:46:55.723932+0000","last_clean":"2024-09-10T18:46:55.723932+0000","last_became_active":"2024-09-10T18:45:22.068918+0000","last_became_peered":"2024-09-10T18:45:22.068918+0000","last_unstale":"2024-09-10T18:46:55.723932+0000","last_undegraded":"2024-09-10T18:46:55.723932+0000","last_fullsized":"2024-09-10T18:46:55.723932+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_clean_scrub_stamp":"2024-09-10T18:46:48.116358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:48.149931+0000","scrub_duration":0.001429331,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:00.743766+0000","last_change":"2024-09-10T18:47:00.743740+0000","last_active":"2024-09-10T18:47:00.743766+0000","last_peered":"2024-09-10T18:47:00.743766+0000","last_clean":"2024-09-10T18:47:00.743766+0000","last_became_active":"2024-09-10T18:45:24.264538+0000","last_became_peered":"2024-09-10T18:45:24.264538+0000","last_unstale":"2024-09-10T18:47:00.743766+0000","last_undegraded":"2024-09-10T18:47:00.743766+0000","last_fullsized":"2024-09-10T18:47:00.743766+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:00.743680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:00.743680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:02.404002+0000","scrub_duration":0.0022093759999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"63'14","reported_seq":65,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.719793+0000","last_change":"2024-09-10T18:46:29.330572+0000","last_active":"2024-09-10T18:46:55.719793+0000","last_peered":"2024-09-10T18:46:55.719793+0000","last_clean":"2024-09-10T18:46:55.719793+0000","last_became_active":"2024-09-10T18:45:20.052243+0000","last_became_peered":"2024-09-10T18:45:20.052243+0000","last_unstale":"2024-09-10T18:46:55.719793+0000","last_undegraded":"2024-09-10T18:46:55.719793+0000","last_fullsized":"2024-09-10T18:46:55.719793+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-10T18:46:29.330503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:29.330503+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:55.456983+0000","scrub_duration":0.0020437459999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.724143+0000","last_change":"2024-09-10T18:46:25.206801+0000","last_active":"2024-09-10T18:46:55.724143+0000","last_peered":"2024-09-10T18:46:55.724143+0000","last_clean":"2024-09-10T18:46:55.724143+0000","last_became_active":"2024-09-10T18:45:17.966027+0000","last_became_peered":"2024-09-10T18:45:17.966027+0000","last_unstale":"2024-09-10T18:46:55.724143+0000","last_undegraded":"2024-09-10T18:46:55.724143+0000","last_fullsized":"2024-09-10T18:46:55.724143+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_clean_scrub_stamp":"2024-09-10T18:46:25.206731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:46.038154+0000","scrub_duration":0.0016589910000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":70,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.094463+0000","last_change":"2024-09-10T18:46:40.808518+0000","last_active":"2024-09-10T18:46:56.094463+0000","last_peered":"2024-09-10T18:46:56.094463+0000","last_clean":"2024-09-10T18:46:56.094463+0000","last_became_active":"2024-09-10T18:45:22.074540+0000","last_became_peered":"2024-09-10T18:45:22.074540+0000","last_unstale":"2024-09-10T18:46:56.094463+0000","last_undegraded":"2024-09-10T18:46:56.094463+0000","last_fullsized":"2024-09-10T18:46:56.094463+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:40.808453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:40.808453+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:43.962254+0000","scrub_duration":0.0016757829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723560+0000","last_change":"2024-09-10T18:46:32.590511+0000","last_active":"2024-09-10T18:46:55.723560+0000","last_peered":"2024-09-10T18:46:55.723560+0000","last_clean":"2024-09-10T18:46:55.723560+0000","last_became_active":"2024-09-10T18:45:24.265303+0000","last_became_peered":"2024-09-10T18:45:24.265303+0000","last_unstale":"2024-09-10T18:46:55.723560+0000","last_undegraded":"2024-09-10T18:46:55.723560+0000","last_fullsized":"2024-09-10T18:46:55.723560+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:32.590438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:32.590438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:01.680676+0000","scrub_duration":0.0013292270000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"63'18","reported_seq":76,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723920+0000","last_change":"2024-09-10T18:46:27.801177+0000","last_active":"2024-09-10T18:46:55.723920+0000","last_peered":"2024-09-10T18:46:55.723920+0000","last_clean":"2024-09-10T18:46:55.723920+0000","last_became_active":"2024-09-10T18:45:20.420937+0000","last_became_peered":"2024-09-10T18:45:20.420937+0000","last_unstale":"2024-09-10T18:46:55.723920+0000","last_undegraded":"2024-09-10T18:46:55.723920+0000","last_fullsized":"2024-09-10T18:46:55.723920+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'18","last_scrub_stamp":"2024-09-10T18:46:27.801110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:27.801110+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:42.247024+0000","scrub_duration":0.001870095,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723162+0000","last_change":"2024-09-10T18:45:17.962000+0000","last_active":"2024-09-10T18:46:55.723162+0000","last_peered":"2024-09-10T18:46:55.723162+0000","last_clean":"2024-09-10T18:46:55.723162+0000","last_became_active":"2024-09-10T18:45:17.961695+0000","last_became_peered":"2024-09-10T18:45:17.961695+0000","last_unstale":"2024-09-10T18:46:55.723162+0000","last_undegraded":"2024-09-10T18:46:55.723162+0000","last_fullsized":"2024-09-10T18:46:55.723162+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.719551+0000","last_change":"2024-09-10T18:46:41.312735+0000","last_active":"2024-09-10T18:46:55.719551+0000","last_peered":"2024-09-10T18:46:55.719551+0000","last_clean":"2024-09-10T18:46:55.719551+0000","last_became_active":"2024-09-10T18:45:22.074012+0000","last_became_peered":"2024-09-10T18:45:22.074012+0000","last_unstale":"2024-09-10T18:46:55.719551+0000","last_undegraded":"2024-09-10T18:46:55.719551+0000","last_fullsized":"2024-09-10T18:46:55.719551+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:41.312671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:41.312671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:47.779534+0000","scrub_duration":0.0013475010000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720156+0000","last_change":"2024-09-10T18:46:53.087609+0000","last_active":"2024-09-10T18:46:55.720156+0000","last_peered":"2024-09-10T18:46:55.720156+0000","last_clean":"2024-09-10T18:46:55.720156+0000","last_became_active":"2024-09-10T18:45:24.076831+0000","last_became_peered":"2024-09-10T18:45:24.076831+0000","last_unstale":"2024-09-10T18:46:55.720156+0000","last_undegraded":"2024-09-10T18:46:55.720156+0000","last_fullsized":"2024-09-10T18:46:55.720156+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_clean_scrub_stamp":"2024-09-10T18:46:53.087539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:56.714234+0000","scrub_duration":0.001678971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"63'19","reported_seq":80,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.719953+0000","last_change":"2024-09-10T18:46:42.524291+0000","last_active":"2024-09-10T18:46:55.719953+0000","last_peered":"2024-09-10T18:46:55.719953+0000","last_clean":"2024-09-10T18:46:55.719953+0000","last_became_active":"2024-09-10T18:45:20.421046+0000","last_became_peered":"2024-09-10T18:45:20.421046+0000","last_unstale":"2024-09-10T18:46:55.719953+0000","last_undegraded":"2024-09-10T18:46:55.719953+0000","last_fullsized":"2024-09-10T18:46:55.719953+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:46:42.524195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:42.524195+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:45.664487+0000","scrub_duration":0.0022840299999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:00.592392+0000","last_change":"2024-09-10T18:47:00.592359+0000","last_active":"2024-09-10T18:47:00.592392+0000","last_peered":"2024-09-10T18:47:00.592392+0000","last_clean":"2024-09-10T18:47:00.592392+0000","last_became_active":"2024-09-10T18:45:17.960226+0000","last_became_peered":"2024-09-10T18:45:17.960226+0000","last_unstale":"2024-09-10T18:47:00.592392+0000","last_undegraded":"2024-09-10T18:47:00.592392+0000","last_fullsized":"2024-09-10T18:47:00.592392+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:00.592279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:47:00.592279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:03.737142+0000","scrub_duration":0.001418576,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.721954+0000","last_change":"2024-09-10T18:46:52.235456+0000","last_active":"2024-09-10T18:46:55.721954+0000","last_peered":"2024-09-10T18:46:55.721954+0000","last_clean":"2024-09-10T18:46:55.721954+0000","last_became_active":"2024-09-10T18:45:22.066148+0000","last_became_peered":"2024-09-10T18:45:22.066148+0000","last_unstale":"2024-09-10T18:46:55.721954+0000","last_undegraded":"2024-09-10T18:46:55.721954+0000","last_fullsized":"2024-09-10T18:46:55.721954+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:52.235389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:52.235389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:06.785777+0000","scrub_duration":0.001679675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.722014+0000","last_change":"2024-09-10T18:45:24.265025+0000","last_active":"2024-09-10T18:46:55.722014+0000","last_peered":"2024-09-10T18:46:55.722014+0000","last_clean":"2024-09-10T18:46:55.722014+0000","last_became_active":"2024-09-10T18:45:24.264680+0000","last_became_peered":"2024-09-10T18:45:24.264680+0000","last_unstale":"2024-09-10T18:46:55.722014+0000","last_undegraded":"2024-09-10T18:46:55.722014+0000","last_fullsized":"2024-09-10T18:46:55.722014+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"83'15","reported_seq":64,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:58.436606+0000","last_change":"2024-09-10T18:46:58.436565+0000","last_active":"2024-09-10T18:46:58.436606+0000","last_peered":"2024-09-10T18:46:58.436606+0000","last_clean":"2024-09-10T18:46:58.436606+0000","last_became_active":"2024-09-10T18:45:20.057204+0000","last_became_peered":"2024-09-10T18:45:20.057204+0000","last_unstale":"2024-09-10T18:46:58.436606+0000","last_undegraded":"2024-09-10T18:46:58.436606+0000","last_fullsized":"2024-09-10T18:46:58.436606+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'15","last_scrub_stamp":"2024-09-10T18:46:58.436473+0000","last_deep_scrub":"83'15","last_deep_scrub_stamp":"2024-09-10T18:46:58.436473+0000","last_clean_scrub_stamp":"2024-09-10T18:46:58.436473+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:59.402142+0000","scrub_duration":0.0046048119999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723038+0000","last_change":"2024-09-10T18:46:50.443101+0000","last_active":"2024-09-10T18:46:55.723038+0000","last_peered":"2024-09-10T18:46:55.723038+0000","last_clean":"2024-09-10T18:46:55.723038+0000","last_became_active":"2024-09-10T18:45:17.966632+0000","last_became_peered":"2024-09-10T18:45:17.966632+0000","last_unstale":"2024-09-10T18:46:55.723038+0000","last_undegraded":"2024-09-10T18:46:55.723038+0000","last_fullsized":"2024-09-10T18:46:55.723038+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:50.443042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.443042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:18.989766+0000","scrub_duration":0.0014088550000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"63'8","reported_seq":60,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.094381+0000","last_change":"2024-09-10T18:45:22.063631+0000","last_active":"2024-09-10T18:46:56.094381+0000","last_peered":"2024-09-10T18:46:56.094381+0000","last_clean":"2024-09-10T18:46:56.094381+0000","last_became_active":"2024-09-10T18:45:22.063326+0000","last_became_peered":"2024-09-10T18:45:22.063326+0000","last_unstale":"2024-09-10T18:46:56.094381+0000","last_undegraded":"2024-09-10T18:46:56.094381+0000","last_fullsized":"2024-09-10T18:46:56.094381+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.719177+0000","last_change":"2024-09-10T18:45:24.077780+0000","last_active":"2024-09-10T18:46:55.719177+0000","last_peered":"2024-09-10T18:46:55.719177+0000","last_clean":"2024-09-10T18:46:55.719177+0000","last_became_active":"2024-09-10T18:45:24.077412+0000","last_became_peered":"2024-09-10T18:45:24.077412+0000","last_unstale":"2024-09-10T18:46:55.719177+0000","last_undegraded":"2024-09-10T18:46:55.719177+0000","last_fullsized":"2024-09-10T18:46:55.719177+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"63'28","reported_seq":112,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.094424+0000","last_change":"2024-09-10T18:46:19.223319+0000","last_active":"2024-09-10T18:46:56.094424+0000","last_peered":"2024-09-10T18:46:56.094424+0000","last_clean":"2024-09-10T18:46:56.094424+0000","last_became_active":"2024-09-10T18:45:20.061413+0000","last_became_peered":"2024-09-10T18:45:20.061413+0000","last_unstale":"2024-09-10T18:46:56.094424+0000","last_undegraded":"2024-09-10T18:46:56.094424+0000","last_fullsized":"2024-09-10T18:46:56.094424+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'28","last_scrub_stamp":"2024-09-10T18:46:19.223250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:19.223250+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:23.644429+0000","scrub_duration":0.0032067810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723685+0000","last_change":"2024-09-10T18:46:49.120758+0000","last_active":"2024-09-10T18:46:55.723685+0000","last_peered":"2024-09-10T18:46:55.723685+0000","last_clean":"2024-09-10T18:46:55.723685+0000","last_became_active":"2024-09-10T18:45:17.964494+0000","last_became_peered":"2024-09-10T18:45:17.964494+0000","last_unstale":"2024-09-10T18:46:55.723685+0000","last_undegraded":"2024-09-10T18:46:55.723685+0000","last_fullsized":"2024-09-10T18:46:55.723685+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:49.120704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:49.120704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:51.579079+0000","scrub_duration":0.0011863380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:58.523323+0000","last_change":"2024-09-10T18:46:58.523278+0000","last_active":"2024-09-10T18:46:58.523323+0000","last_peered":"2024-09-10T18:46:58.523323+0000","last_clean":"2024-09-10T18:46:58.523323+0000","last_became_active":"2024-09-10T18:45:22.061428+0000","last_became_peered":"2024-09-10T18:45:22.061428+0000","last_unstale":"2024-09-10T18:46:58.523323+0000","last_undegraded":"2024-09-10T18:46:58.523323+0000","last_fullsized":"2024-09-10T18:46:58.523323+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:58.523217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:58.523217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:04.494370+0000","scrub_duration":0.001307856,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.724755+0000","last_change":"2024-09-10T18:45:24.073623+0000","last_active":"2024-09-10T18:46:55.724755+0000","last_peered":"2024-09-10T18:46:55.724755+0000","last_clean":"2024-09-10T18:46:55.724755+0000","last_became_active":"2024-09-10T18:45:24.073307+0000","last_became_peered":"2024-09-10T18:45:24.073307+0000","last_unstale":"2024-09-10T18:46:55.724755+0000","last_undegraded":"2024-09-10T18:46:55.724755+0000","last_fullsized":"2024-09-10T18:46:55.724755+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"63'13","reported_seq":61,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723473+0000","last_change":"2024-09-10T18:45:20.058778+0000","last_active":"2024-09-10T18:46:55.723473+0000","last_peered":"2024-09-10T18:46:55.723473+0000","last_clean":"2024-09-10T18:46:55.723473+0000","last_became_active":"2024-09-10T18:45:20.058435+0000","last_became_peered":"2024-09-10T18:45:20.058435+0000","last_unstale":"2024-09-10T18:46:55.723473+0000","last_undegraded":"2024-09-10T18:46:55.723473+0000","last_fullsized":"2024-09-10T18:46:55.723473+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"65'5","reported_seq":139,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:59.392607+0000","last_change":"2024-09-10T18:45:22.059997+0000","last_active":"2024-09-10T18:46:59.392607+0000","last_peered":"2024-09-10T18:46:59.392607+0000","last_clean":"2024-09-10T18:46:59.392607+0000","last_became_active":"2024-09-10T18:45:17.969868+0000","last_became_peered":"2024-09-10T18:45:17.969868+0000","last_unstale":"2024-09-10T18:46:59.392607+0000","last_undegraded":"2024-09-10T18:46:59.392607+0000","last_fullsized":"2024-09-10T18:46:59.392607+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021933159999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":95,"num_read_kb":90,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.719852+0000","last_change":"2024-09-10T18:46:46.712335+0000","last_active":"2024-09-10T18:46:55.719852+0000","last_peered":"2024-09-10T18:46:55.719852+0000","last_clean":"2024-09-10T18:46:55.719852+0000","last_became_active":"2024-09-10T18:45:17.960859+0000","last_became_peered":"2024-09-10T18:45:17.960859+0000","last_unstale":"2024-09-10T18:46:55.719852+0000","last_undegraded":"2024-09-10T18:46:55.719852+0000","last_fullsized":"2024-09-10T18:46:55.719852+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:46.712269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:46.712269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:54.660825+0000","scrub_duration":0.001509383,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720164+0000","last_change":"2024-09-10T18:46:41.818890+0000","last_active":"2024-09-10T18:46:55.720164+0000","last_peered":"2024-09-10T18:46:55.720164+0000","last_clean":"2024-09-10T18:46:55.720164+0000","last_became_active":"2024-09-10T18:45:22.068350+0000","last_became_peered":"2024-09-10T18:45:22.068350+0000","last_unstale":"2024-09-10T18:46:55.720164+0000","last_undegraded":"2024-09-10T18:46:55.720164+0000","last_fullsized":"2024-09-10T18:46:55.720164+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:41.818827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:41.818827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:07.148553+0000","scrub_duration":0.0017408219999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:59.553615+0000","last_change":"2024-09-10T18:46:59.553576+0000","last_active":"2024-09-10T18:46:59.553615+0000","last_peered":"2024-09-10T18:46:59.553615+0000","last_clean":"2024-09-10T18:46:59.553615+0000","last_became_active":"2024-09-10T18:45:24.264617+0000","last_became_peered":"2024-09-10T18:45:24.264617+0000","last_unstale":"2024-09-10T18:46:59.553615+0000","last_undegraded":"2024-09-10T18:46:59.553615+0000","last_fullsized":"2024-09-10T18:46:59.553615+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:59.553505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:59.553505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:29.436053+0000","scrub_duration":0.0018297210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"63'12","reported_seq":62,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720387+0000","last_change":"2024-09-10T18:46:52.472574+0000","last_active":"2024-09-10T18:46:55.720387+0000","last_peered":"2024-09-10T18:46:55.720387+0000","last_clean":"2024-09-10T18:46:55.720387+0000","last_became_active":"2024-09-10T18:45:20.061091+0000","last_became_peered":"2024-09-10T18:45:20.061091+0000","last_unstale":"2024-09-10T18:46:55.720387+0000","last_undegraded":"2024-09-10T18:46:55.720387+0000","last_fullsized":"2024-09-10T18:46:55.720387+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:46:52.472511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:52.472511+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:19.552466+0000","scrub_duration":0.001886935,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"58'1","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.719619+0000","last_change":"2024-09-10T18:45:22.055928+0000","last_active":"2024-09-10T18:46:55.719619+0000","last_peered":"2024-09-10T18:46:55.719619+0000","last_clean":"2024-09-10T18:46:55.719619+0000","last_became_active":"2024-09-10T18:45:17.963932+0000","last_became_peered":"2024-09-10T18:45:17.963932+0000","last_unstale":"2024-09-10T18:46:55.719619+0000","last_undegraded":"2024-09-10T18:46:55.719619+0000","last_fullsized":"2024-09-10T18:46:55.719619+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037812099999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.742192+0000","last_change":"2024-09-10T18:46:56.742192+0000","last_active":"2024-09-10T18:46:56.742192+0000","last_peered":"2024-09-10T18:46:56.742192+0000","last_clean":"2024-09-10T18:46:56.742192+0000","last_became_active":"2024-09-10T18:46:56.741862+0000","last_became_peered":"2024-09-10T18:46:56.741862+0000","last_unstale":"2024-09-10T18:46:56.742192+0000","last_undegraded":"2024-09-10T18:46:56.742192+0000","last_fullsized":"2024-09-10T18:46:56.742192+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:54.554426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:54.554426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:17.934705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:59.132334+0000","last_change":"2024-09-10T18:46:59.132301+0000","last_active":"2024-09-10T18:46:59.132334+0000","last_peered":"2024-09-10T18:46:59.132334+0000","last_clean":"2024-09-10T18:46:59.132334+0000","last_became_active":"2024-09-10T18:45:22.064285+0000","last_became_peered":"2024-09-10T18:45:22.064285+0000","last_unstale":"2024-09-10T18:46:59.132334+0000","last_undegraded":"2024-09-10T18:46:59.132334+0000","last_fullsized":"2024-09-10T18:46:59.132334+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:59.132234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:59.132234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:03.150923+0000","scrub_duration":0.001471034,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.722567+0000","last_change":"2024-09-10T18:46:49.422912+0000","last_active":"2024-09-10T18:46:55.722567+0000","last_peered":"2024-09-10T18:46:55.722567+0000","last_clean":"2024-09-10T18:46:55.722567+0000","last_became_active":"2024-09-10T18:45:24.083689+0000","last_became_peered":"2024-09-10T18:45:24.083689+0000","last_unstale":"2024-09-10T18:46:55.722567+0000","last_undegraded":"2024-09-10T18:46:55.722567+0000","last_fullsized":"2024-09-10T18:46:55.722567+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:49.422843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:49.422843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:00.864466+0000","scrub_duration":0.0014985300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"63'14","reported_seq":66,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723666+0000","last_change":"2024-09-10T18:46:22.294141+0000","last_active":"2024-09-10T18:46:55.723666+0000","last_peered":"2024-09-10T18:46:55.723666+0000","last_clean":"2024-09-10T18:46:55.723666+0000","last_became_active":"2024-09-10T18:45:20.421119+0000","last_became_peered":"2024-09-10T18:45:20.421119+0000","last_unstale":"2024-09-10T18:46:55.723666+0000","last_undegraded":"2024-09-10T18:46:55.723666+0000","last_fullsized":"2024-09-10T18:46:55.723666+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-10T18:46:22.294072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:22.294072+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:49.887899+0000","scrub_duration":0.0021435069999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"60'2","reported_seq":53,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.724331+0000","last_change":"2024-09-10T18:46:25.845131+0000","last_active":"2024-09-10T18:46:55.724331+0000","last_peered":"2024-09-10T18:46:55.724331+0000","last_clean":"2024-09-10T18:46:55.724331+0000","last_became_active":"2024-09-10T18:45:17.961660+0000","last_became_peered":"2024-09-10T18:45:17.961660+0000","last_unstale":"2024-09-10T18:46:55.724331+0000","last_undegraded":"2024-09-10T18:46:55.724331+0000","last_fullsized":"2024-09-10T18:46:55.724331+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-10T18:46:25.845070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:25.845070+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:50.370600+0000","scrub_duration":0.0020129710000000001,"objects_trimmed":0,"snaptrim_duration":0.0013942640000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.722360+0000","last_change":"2024-09-10T18:46:29.081362+0000","last_active":"2024-09-10T18:46:55.722360+0000","last_peered":"2024-09-10T18:46:55.722360+0000","last_clean":"2024-09-10T18:46:55.722360+0000","last_became_active":"2024-09-10T18:45:17.965163+0000","last_became_peered":"2024-09-10T18:45:17.965163+0000","last_unstale":"2024-09-10T18:46:55.722360+0000","last_undegraded":"2024-09-10T18:46:55.722360+0000","last_fullsized":"2024-09-10T18:46:55.722360+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:29.081296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:29.081296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:50.659900+0000","scrub_duration":0.0015488590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'81","reported_seq":80,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723353+0000","last_change":"2024-09-10T18:45:45.481030+0000","last_active":"2024-09-10T18:46:55.723353+0000","last_peered":"2024-09-10T18:46:55.723353+0000","last_clean":"2024-09-10T18:46:55.723353+0000","last_became_active":"2024-09-10T18:45:07.433592+0000","last_became_peered":"2024-09-10T18:45:07.433592+0000","last_unstale":"2024-09-10T18:46:55.723353+0000","last_undegraded":"2024-09-10T18:46:55.723353+0000","last_fullsized":"2024-09-10T18:46:55.723353+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'74","last_scrub_stamp":"2024-09-10T18:45:45.480958+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:45:45.480958+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:46:58.297247+0000","scrub_duration":0.0019876809999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723308+0000","last_change":"2024-09-10T18:46:26.687029+0000","last_active":"2024-09-10T18:46:55.723308+0000","last_peered":"2024-09-10T18:46:55.723308+0000","last_clean":"2024-09-10T18:46:55.723308+0000","last_became_active":"2024-09-10T18:45:22.068226+0000","last_became_peered":"2024-09-10T18:45:22.068226+0000","last_unstale":"2024-09-10T18:46:55.723308+0000","last_undegraded":"2024-09-10T18:46:55.723308+0000","last_fullsized":"2024-09-10T18:46:55.723308+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:26.686961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:26.686961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:32.456741+0000","scrub_duration":0.0015985109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.066593+0000","last_change":"2024-09-10T18:46:56.066558+0000","last_active":"2024-09-10T18:46:56.066593+0000","last_peered":"2024-09-10T18:46:56.066593+0000","last_clean":"2024-09-10T18:46:56.066593+0000","last_became_active":"2024-09-10T18:45:24.073327+0000","last_became_peered":"2024-09-10T18:45:24.073327+0000","last_unstale":"2024-09-10T18:46:56.066593+0000","last_undegraded":"2024-09-10T18:46:56.066593+0000","last_fullsized":"2024-09-10T18:46:56.066593+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:56.066502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:56.066502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:06.081016+0000","scrub_duration":0.0011960810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"63'19","reported_seq":75,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.722652+0000","last_change":"2024-09-10T18:46:28.229804+0000","last_active":"2024-09-10T18:46:55.722652+0000","last_peered":"2024-09-10T18:46:55.722652+0000","last_clean":"2024-09-10T18:46:55.722652+0000","last_became_active":"2024-09-10T18:45:20.421097+0000","last_became_peered":"2024-09-10T18:45:20.421097+0000","last_unstale":"2024-09-10T18:46:55.722652+0000","last_undegraded":"2024-09-10T18:46:55.722652+0000","last_fullsized":"2024-09-10T18:46:55.722652+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:46:28.229737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:28.229737+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:49.859218+0000","scrub_duration":0.002308184,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720850+0000","last_change":"2024-09-10T18:46:53.707765+0000","last_active":"2024-09-10T18:46:55.720850+0000","last_peered":"2024-09-10T18:46:55.720850+0000","last_clean":"2024-09-10T18:46:55.720850+0000","last_became_active":"2024-09-10T18:45:17.959924+0000","last_became_peered":"2024-09-10T18:45:17.959924+0000","last_unstale":"2024-09-10T18:46:55.720850+0000","last_undegraded":"2024-09-10T18:46:55.720850+0000","last_fullsized":"2024-09-10T18:46:55.720850+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:53.707694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:53.707694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:22.214797+0000","scrub_duration":0.0020804069999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720321+0000","last_change":"2024-09-10T18:46:30.781786+0000","last_active":"2024-09-10T18:46:55.720321+0000","last_peered":"2024-09-10T18:46:55.720321+0000","last_clean":"2024-09-10T18:46:55.720321+0000","last_became_active":"2024-09-10T18:45:22.072581+0000","last_became_peered":"2024-09-10T18:45:22.072581+0000","last_unstale":"2024-09-10T18:46:55.720321+0000","last_undegraded":"2024-09-10T18:46:55.720321+0000","last_fullsized":"2024-09-10T18:46:55.720321+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:30.781730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:30.781730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:38.248319+0000","scrub_duration":0.0020255870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723278+0000","last_change":"2024-09-10T18:45:24.076661+0000","last_active":"2024-09-10T18:46:55.723278+0000","last_peered":"2024-09-10T18:46:55.723278+0000","last_clean":"2024-09-10T18:46:55.723278+0000","last_became_active":"2024-09-10T18:45:24.076392+0000","last_became_peered":"2024-09-10T18:45:24.076392+0000","last_unstale":"2024-09-10T18:46:55.723278+0000","last_undegraded":"2024-09-10T18:46:55.723278+0000","last_fullsized":"2024-09-10T18:46:55.723278+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"63'9","reported_seq":60,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.719877+0000","last_change":"2024-09-10T18:46:51.486491+0000","last_active":"2024-09-10T18:46:55.719877+0000","last_peered":"2024-09-10T18:46:55.719877+0000","last_clean":"2024-09-10T18:46:55.719877+0000","last_became_active":"2024-09-10T18:45:20.061021+0000","last_became_peered":"2024-09-10T18:45:20.061021+0000","last_unstale":"2024-09-10T18:46:55.719877+0000","last_undegraded":"2024-09-10T18:46:55.719877+0000","last_fullsized":"2024-09-10T18:46:55.719877+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:46:51.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:51.486428+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:05.602014+0000","scrub_duration":0.0019382830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723313+0000","last_change":"2024-09-10T18:45:17.963854+0000","last_active":"2024-09-10T18:46:55.723313+0000","last_peered":"2024-09-10T18:46:55.723313+0000","last_clean":"2024-09-10T18:46:55.723313+0000","last_became_active":"2024-09-10T18:45:17.963057+0000","last_became_peered":"2024-09-10T18:45:17.963057+0000","last_unstale":"2024-09-10T18:46:55.723313+0000","last_undegraded":"2024-09-10T18:46:55.723313+0000","last_fullsized":"2024-09-10T18:46:55.723313+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720471+0000","last_change":"2024-09-10T18:46:55.684302+0000","last_active":"2024-09-10T18:46:55.720471+0000","last_peered":"2024-09-10T18:46:55.720471+0000","last_clean":"2024-09-10T18:46:55.720471+0000","last_became_active":"2024-09-10T18:45:22.070343+0000","last_became_peered":"2024-09-10T18:45:22.070343+0000","last_unstale":"2024-09-10T18:46:55.720471+0000","last_undegraded":"2024-09-10T18:46:55.720471+0000","last_fullsized":"2024-09-10T18:46:55.720471+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_clean_scrub_stamp":"2024-09-10T18:46:55.684250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:59.597477+0000","scrub_duration":0.0014319199999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720233+0000","last_change":"2024-09-10T18:46:47.545406+0000","last_active":"2024-09-10T18:46:55.720233+0000","last_peered":"2024-09-10T18:46:55.720233+0000","last_clean":"2024-09-10T18:46:55.720233+0000","last_became_active":"2024-09-10T18:45:24.078529+0000","last_became_peered":"2024-09-10T18:45:24.078529+0000","last_unstale":"2024-09-10T18:46:55.720233+0000","last_undegraded":"2024-09-10T18:46:55.720233+0000","last_fullsized":"2024-09-10T18:46:55.720233+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:47.545343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:47.545343+0000","last_clean_scrub_stamp":"2024-09-10T18:46:47.545343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:02.171974+0000","scrub_duration":0.0012977799999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"63'15","reported_seq":69,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.721541+0000","last_change":"2024-09-10T18:46:51.114543+0000","last_active":"2024-09-10T18:46:55.721541+0000","last_peered":"2024-09-10T18:46:55.721541+0000","last_clean":"2024-09-10T18:46:55.721541+0000","last_became_active":"2024-09-10T18:45:20.419804+0000","last_became_peered":"2024-09-10T18:45:20.419804+0000","last_unstale":"2024-09-10T18:46:55.721541+0000","last_undegraded":"2024-09-10T18:46:55.721541+0000","last_fullsized":"2024-09-10T18:46:55.721541+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:46:51.114455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:51.114455+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:18.982344+0000","scrub_duration":0.0022557979999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720362+0000","last_change":"2024-09-10T18:46:17.819966+0000","last_active":"2024-09-10T18:46:55.720362+0000","last_peered":"2024-09-10T18:46:55.720362+0000","last_clean":"2024-09-10T18:46:55.720362+0000","last_became_active":"2024-09-10T18:45:17.959653+0000","last_became_peered":"2024-09-10T18:45:17.959653+0000","last_unstale":"2024-09-10T18:46:55.720362+0000","last_undegraded":"2024-09-10T18:46:55.720362+0000","last_fullsized":"2024-09-10T18:46:55.720362+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:17.819909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:17.819909+0000","last_clean_scrub_stamp":"2024-09-10T18:46:17.819909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:18.454145+0000","scrub_duration":0.001612109,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"63'8","reported_seq":63,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.094508+0000","last_change":"2024-09-10T18:45:22.073687+0000","last_active":"2024-09-10T18:46:56.094508+0000","last_peered":"2024-09-10T18:46:56.094508+0000","last_clean":"2024-09-10T18:46:56.094508+0000","last_became_active":"2024-09-10T18:45:22.073383+0000","last_became_peered":"2024-09-10T18:45:22.073383+0000","last_unstale":"2024-09-10T18:46:56.094508+0000","last_undegraded":"2024-09-10T18:46:56.094508+0000","last_fullsized":"2024-09-10T18:46:56.094508+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723356+0000","last_change":"2024-09-10T18:46:45.083522+0000","last_active":"2024-09-10T18:46:55.723356+0000","last_peered":"2024-09-10T18:46:55.723356+0000","last_clean":"2024-09-10T18:46:55.723356+0000","last_became_active":"2024-09-10T18:45:24.265256+0000","last_became_peered":"2024-09-10T18:45:24.265256+0000","last_unstale":"2024-09-10T18:46:55.723356+0000","last_undegraded":"2024-09-10T18:46:55.723356+0000","last_fullsized":"2024-09-10T18:46:55.723356+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_clean_scrub_stamp":"2024-09-10T18:46:45.083425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:08.177993+0000","scrub_duration":0.0017589890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"63'12","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.719562+0000","last_change":"2024-09-10T18:46:30.320527+0000","last_active":"2024-09-10T18:46:55.719562+0000","last_peered":"2024-09-10T18:46:55.719562+0000","last_clean":"2024-09-10T18:46:55.719562+0000","last_became_active":"2024-09-10T18:45:20.056869+0000","last_became_peered":"2024-09-10T18:45:20.056869+0000","last_unstale":"2024-09-10T18:46:55.719562+0000","last_undegraded":"2024-09-10T18:46:55.719562+0000","last_fullsized":"2024-09-10T18:46:55.719562+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:46:30.320462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:30.320462+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:35.180189+0000","scrub_duration":0.0028822850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"56'2","reported_seq":63,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720187+0000","last_change":"2024-09-10T18:46:42.303648+0000","last_active":"2024-09-10T18:46:55.720187+0000","last_peered":"2024-09-10T18:46:55.720187+0000","last_clean":"2024-09-10T18:46:55.720187+0000","last_became_active":"2024-09-10T18:45:17.963286+0000","last_became_peered":"2024-09-10T18:45:17.963286+0000","last_unstale":"2024-09-10T18:46:55.720187+0000","last_undegraded":"2024-09-10T18:46:55.720187+0000","last_fullsized":"2024-09-10T18:46:55.720187+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-10T18:46:42.303567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:42.303567+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:05.463057+0000","scrub_duration":0.0020684750000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.719539+0000","last_change":"2024-09-10T18:46:44.754345+0000","last_active":"2024-09-10T18:46:55.719539+0000","last_peered":"2024-09-10T18:46:55.719539+0000","last_clean":"2024-09-10T18:46:55.719539+0000","last_became_active":"2024-09-10T18:45:22.068957+0000","last_became_peered":"2024-09-10T18:45:22.068957+0000","last_unstale":"2024-09-10T18:46:55.719539+0000","last_undegraded":"2024-09-10T18:46:55.719539+0000","last_fullsized":"2024-09-10T18:46:55.719539+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:44.754286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:44.754286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:47.893148+0000","scrub_duration":0.001258386,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.725035+0000","last_change":"2024-09-10T18:45:24.078134+0000","last_active":"2024-09-10T18:46:55.725035+0000","last_peered":"2024-09-10T18:46:55.725035+0000","last_clean":"2024-09-10T18:46:55.725035+0000","last_became_active":"2024-09-10T18:45:24.077815+0000","last_became_peered":"2024-09-10T18:45:24.077815+0000","last_unstale":"2024-09-10T18:46:55.725035+0000","last_undegraded":"2024-09-10T18:46:55.725035+0000","last_fullsized":"2024-09-10T18:46:55.725035+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"76'13","reported_seq":64,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720318+0000","last_change":"2024-09-10T18:46:24.626912+0000","last_active":"2024-09-10T18:46:55.720318+0000","last_peered":"2024-09-10T18:46:55.720318+0000","last_clean":"2024-09-10T18:46:55.720318+0000","last_became_active":"2024-09-10T18:45:20.420421+0000","last_became_peered":"2024-09-10T18:45:20.420421+0000","last_unstale":"2024-09-10T18:46:55.720318+0000","last_undegraded":"2024-09-10T18:46:55.720318+0000","last_fullsized":"2024-09-10T18:46:55.720318+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'13","last_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_deep_scrub":"76'13","last_deep_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_clean_scrub_stamp":"2024-09-10T18:46:24.626859+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:41.087073+0000","scrub_duration":0.0035642949999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.719648+0000","last_change":"2024-09-10T18:46:31.734522+0000","last_active":"2024-09-10T18:46:55.719648+0000","last_peered":"2024-09-10T18:46:55.719648+0000","last_clean":"2024-09-10T18:46:55.719648+0000","last_became_active":"2024-09-10T18:45:17.965112+0000","last_became_peered":"2024-09-10T18:45:17.965112+0000","last_unstale":"2024-09-10T18:46:55.719648+0000","last_undegraded":"2024-09-10T18:46:55.719648+0000","last_fullsized":"2024-09-10T18:46:55.719648+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:31.734459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:31.734459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:41.550267+0000","scrub_duration":0.0015571960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.724880+0000","last_change":"2024-09-10T18:46:32.829583+0000","last_active":"2024-09-10T18:46:55.724880+0000","last_peered":"2024-09-10T18:46:55.724880+0000","last_clean":"2024-09-10T18:46:55.724880+0000","last_became_active":"2024-09-10T18:45:22.069040+0000","last_became_peered":"2024-09-10T18:45:22.069040+0000","last_unstale":"2024-09-10T18:46:55.724880+0000","last_undegraded":"2024-09-10T18:46:55.724880+0000","last_fullsized":"2024-09-10T18:46:55.724880+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:32.829508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:32.829508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:45.869787+0000","scrub_duration":0.001897239,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"63'1","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.721828+0000","last_change":"2024-09-10T18:46:56.721773+0000","last_active":"2024-09-10T18:46:56.721828+0000","last_peered":"2024-09-10T18:46:56.721828+0000","last_clean":"2024-09-10T18:46:56.721828+0000","last_became_active":"2024-09-10T18:45:24.079258+0000","last_became_peered":"2024-09-10T18:45:24.079258+0000","last_unstale":"2024-09-10T18:46:56.721828+0000","last_undegraded":"2024-09-10T18:46:56.721828+0000","last_fullsized":"2024-09-10T18:46:56.721828+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-10T18:46:56.721716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:56.721716+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:04.127384+0000","scrub_duration":0.0014420559999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.722675+0000","last_change":"2024-09-10T18:46:48.227450+0000","last_active":"2024-09-10T18:46:55.722675+0000","last_peered":"2024-09-10T18:46:55.722675+0000","last_clean":"2024-09-10T18:46:55.722675+0000","last_became_active":"2024-09-10T18:45:17.964705+0000","last_became_peered":"2024-09-10T18:45:17.964705+0000","last_unstale":"2024-09-10T18:46:55.722675+0000","last_undegraded":"2024-09-10T18:46:55.722675+0000","last_fullsized":"2024-09-10T18:46:55.722675+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:48.227378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:48.227378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:54.333207+0000","scrub_duration":0.0015447519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'6","reported_seq":43,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.724999+0000","last_change":"2024-09-10T18:45:20.422485+0000","last_active":"2024-09-10T18:46:55.724999+0000","last_peered":"2024-09-10T18:46:55.724999+0000","last_clean":"2024-09-10T18:46:55.724999+0000","last_became_active":"2024-09-10T18:45:20.421746+0000","last_became_peered":"2024-09-10T18:45:20.421746+0000","last_unstale":"2024-09-10T18:46:55.724999+0000","last_undegraded":"2024-09-10T18:46:55.724999+0000","last_fullsized":"2024-09-10T18:46:55.724999+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.722238+0000","last_change":"2024-09-10T18:46:39.084489+0000","last_active":"2024-09-10T18:46:55.722238+0000","last_peered":"2024-09-10T18:46:55.722238+0000","last_clean":"2024-09-10T18:46:55.722238+0000","last_became_active":"2024-09-10T18:45:22.069653+0000","last_became_peered":"2024-09-10T18:45:22.069653+0000","last_unstale":"2024-09-10T18:46:55.722238+0000","last_undegraded":"2024-09-10T18:46:55.722238+0000","last_fullsized":"2024-09-10T18:46:55.722238+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:39.084433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:39.084433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:05.431757+0000","scrub_duration":0.0014845819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723941+0000","last_change":"2024-09-10T18:46:52.611074+0000","last_active":"2024-09-10T18:46:55.723941+0000","last_peered":"2024-09-10T18:46:55.723941+0000","last_clean":"2024-09-10T18:46:55.723941+0000","last_became_active":"2024-09-10T18:45:24.264611+0000","last_became_peered":"2024-09-10T18:45:24.264611+0000","last_unstale":"2024-09-10T18:46:55.723941+0000","last_undegraded":"2024-09-10T18:46:55.723941+0000","last_fullsized":"2024-09-10T18:46:55.723941+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:52.611003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:52.611003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:04.517442+0000","scrub_duration":0.0015558849999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"63'10","reported_seq":81,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.759636+0000","last_change":"2024-09-10T18:46:56.759636+0000","last_active":"2024-09-10T18:46:56.759636+0000","last_peered":"2024-09-10T18:46:56.759636+0000","last_clean":"2024-09-10T18:46:56.759636+0000","last_became_active":"2024-09-10T18:46:56.754800+0000","last_became_peered":"2024-09-10T18:46:56.754800+0000","last_unstale":"2024-09-10T18:46:56.759636+0000","last_undegraded":"2024-09-10T18:46:56.759636+0000","last_fullsized":"2024-09-10T18:46:56.759636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:46:50.722316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.722316+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:55.786523+0000","scrub_duration":0.0022194070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723620+0000","last_change":"2024-09-10T18:46:50.093435+0000","last_active":"2024-09-10T18:46:55.723620+0000","last_peered":"2024-09-10T18:46:55.723620+0000","last_clean":"2024-09-10T18:46:55.723620+0000","last_became_active":"2024-09-10T18:45:17.964870+0000","last_became_peered":"2024-09-10T18:45:17.964870+0000","last_unstale":"2024-09-10T18:46:55.723620+0000","last_undegraded":"2024-09-10T18:46:55.723620+0000","last_fullsized":"2024-09-10T18:46:55.723620+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:50.093372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.093372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:55.991532+0000","scrub_duration":0.0015580539999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"63'8","reported_seq":70,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.094516+0000","last_change":"2024-09-10T18:46:34.085224+0000","last_active":"2024-09-10T18:46:56.094516+0000","last_peered":"2024-09-10T18:46:56.094516+0000","last_clean":"2024-09-10T18:46:56.094516+0000","last_became_active":"2024-09-10T18:45:22.062940+0000","last_became_peered":"2024-09-10T18:45:22.062940+0000","last_unstale":"2024-09-10T18:46:56.094516+0000","last_undegraded":"2024-09-10T18:46:56.094516+0000","last_fullsized":"2024-09-10T18:46:56.094516+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:34.085156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:34.085156+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:54.777061+0000","scrub_duration":0.0020843120000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.719914+0000","last_change":"2024-09-10T18:45:24.074039+0000","last_active":"2024-09-10T18:46:55.719914+0000","last_peered":"2024-09-10T18:46:55.719914+0000","last_clean":"2024-09-10T18:46:55.719914+0000","last_became_active":"2024-09-10T18:45:24.073736+0000","last_became_peered":"2024-09-10T18:45:24.073736+0000","last_unstale":"2024-09-10T18:46:55.719914+0000","last_undegraded":"2024-09-10T18:46:55.719914+0000","last_fullsized":"2024-09-10T18:46:55.719914+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"63'9","reported_seq":60,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723082+0000","last_change":"2024-09-10T18:46:23.086274+0000","last_active":"2024-09-10T18:46:55.723082+0000","last_peered":"2024-09-10T18:46:55.723082+0000","last_clean":"2024-09-10T18:46:55.723082+0000","last_became_active":"2024-09-10T18:45:20.421124+0000","last_became_peered":"2024-09-10T18:45:20.421124+0000","last_unstale":"2024-09-10T18:46:55.723082+0000","last_undegraded":"2024-09-10T18:46:55.723082+0000","last_fullsized":"2024-09-10T18:46:55.723082+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:46:23.086203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:23.086203+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:44.793257+0000","scrub_duration":0.002168068,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720102+0000","last_change":"2024-09-10T18:46:45.541882+0000","last_active":"2024-09-10T18:46:55.720102+0000","last_peered":"2024-09-10T18:46:55.720102+0000","last_clean":"2024-09-10T18:46:55.720102+0000","last_became_active":"2024-09-10T18:45:17.960106+0000","last_became_peered":"2024-09-10T18:45:17.960106+0000","last_unstale":"2024-09-10T18:46:55.720102+0000","last_undegraded":"2024-09-10T18:46:55.720102+0000","last_fullsized":"2024-09-10T18:46:55.720102+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:45.541819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:45.541819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:15.015402+0000","scrub_duration":0.0015180809999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"81'9","reported_seq":71,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.094514+0000","last_change":"2024-09-10T18:46:54.800878+0000","last_active":"2024-09-10T18:46:56.094514+0000","last_peered":"2024-09-10T18:46:56.094514+0000","last_clean":"2024-09-10T18:46:56.094514+0000","last_became_active":"2024-09-10T18:45:22.075368+0000","last_became_peered":"2024-09-10T18:45:22.075368+0000","last_unstale":"2024-09-10T18:46:56.094514+0000","last_undegraded":"2024-09-10T18:46:56.094514+0000","last_fullsized":"2024-09-10T18:46:56.094514+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"81'9","last_scrub_stamp":"2024-09-10T18:46:54.800790+0000","last_deep_scrub":"81'9","last_deep_scrub_stamp":"2024-09-10T18:46:54.800790+0000","last_clean_scrub_stamp":"2024-09-10T18:46:54.800790+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:14.444052+0000","scrub_duration":0.0034208709999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.719967+0000","last_change":"2024-09-10T18:46:36.305262+0000","last_active":"2024-09-10T18:46:55.719967+0000","last_peered":"2024-09-10T18:46:55.719967+0000","last_clean":"2024-09-10T18:46:55.719967+0000","last_became_active":"2024-09-10T18:45:24.080196+0000","last_became_peered":"2024-09-10T18:45:24.080196+0000","last_unstale":"2024-09-10T18:46:55.719967+0000","last_undegraded":"2024-09-10T18:46:55.719967+0000","last_fullsized":"2024-09-10T18:46:55.719967+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_clean_scrub_stamp":"2024-09-10T18:46:36.305199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:04.747507+0000","scrub_duration":0.0013794809999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"63'9","reported_seq":60,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:59.469284+0000","last_change":"2024-09-10T18:46:59.469257+0000","last_active":"2024-09-10T18:46:59.469284+0000","last_peered":"2024-09-10T18:46:59.469284+0000","last_clean":"2024-09-10T18:46:59.469284+0000","last_became_active":"2024-09-10T18:45:20.057479+0000","last_became_peered":"2024-09-10T18:45:20.057479+0000","last_unstale":"2024-09-10T18:46:59.469284+0000","last_undegraded":"2024-09-10T18:46:59.469284+0000","last_fullsized":"2024-09-10T18:46:59.469284+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:46:59.469198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:59.469198+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:26.476613+0000","scrub_duration":0.0022009489999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723247+0000","last_change":"2024-09-10T18:46:23.292276+0000","last_active":"2024-09-10T18:46:55.723247+0000","last_peered":"2024-09-10T18:46:55.723247+0000","last_clean":"2024-09-10T18:46:55.723247+0000","last_became_active":"2024-09-10T18:45:17.962009+0000","last_became_peered":"2024-09-10T18:45:17.962009+0000","last_unstale":"2024-09-10T18:46:55.723247+0000","last_undegraded":"2024-09-10T18:46:55.723247+0000","last_fullsized":"2024-09-10T18:46:55.723247+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:23.292207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:23.292207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:27.416585+0000","scrub_duration":0.001659953,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.724596+0000","last_change":"2024-09-10T18:45:22.073186+0000","last_active":"2024-09-10T18:46:55.724596+0000","last_peered":"2024-09-10T18:46:55.724596+0000","last_clean":"2024-09-10T18:46:55.724596+0000","last_became_active":"2024-09-10T18:45:22.072877+0000","last_became_peered":"2024-09-10T18:45:22.072877+0000","last_unstale":"2024-09-10T18:46:55.724596+0000","last_undegraded":"2024-09-10T18:46:55.724596+0000","last_fullsized":"2024-09-10T18:46:55.724596+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.719175+0000","last_change":"2024-09-10T18:46:50.499866+0000","last_active":"2024-09-10T18:46:55.719175+0000","last_peered":"2024-09-10T18:46:55.719175+0000","last_clean":"2024-09-10T18:46:55.719175+0000","last_became_active":"2024-09-10T18:45:24.081050+0000","last_became_peered":"2024-09-10T18:45:24.081050+0000","last_unstale":"2024-09-10T18:46:55.719175+0000","last_undegraded":"2024-09-10T18:46:55.719175+0000","last_fullsized":"2024-09-10T18:46:55.719175+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:50.499815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.499815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:13.479025+0000","scrub_duration":0.001285432,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"63'11","reported_seq":53,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.719761+0000","last_change":"2024-09-10T18:45:20.057563+0000","last_active":"2024-09-10T18:46:55.719761+0000","last_peered":"2024-09-10T18:46:55.719761+0000","last_clean":"2024-09-10T18:46:55.719761+0000","last_became_active":"2024-09-10T18:45:20.056885+0000","last_became_peered":"2024-09-10T18:45:20.056885+0000","last_unstale":"2024-09-10T18:46:55.719761+0000","last_undegraded":"2024-09-10T18:46:55.719761+0000","last_fullsized":"2024-09-10T18:46:55.719761+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723267+0000","last_change":"2024-09-10T18:46:36.402610+0000","last_active":"2024-09-10T18:46:55.723267+0000","last_peered":"2024-09-10T18:46:55.723267+0000","last_clean":"2024-09-10T18:46:55.723267+0000","last_became_active":"2024-09-10T18:45:17.963542+0000","last_became_peered":"2024-09-10T18:45:17.963542+0000","last_unstale":"2024-09-10T18:46:55.723267+0000","last_undegraded":"2024-09-10T18:46:55.723267+0000","last_fullsized":"2024-09-10T18:46:55.723267+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:36.402548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:36.402548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:04.495178+0000","scrub_duration":0.0014103919999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.722331+0000","last_change":"2024-09-10T18:46:37.414794+0000","last_active":"2024-09-10T18:46:55.722331+0000","last_peered":"2024-09-10T18:46:55.722331+0000","last_clean":"2024-09-10T18:46:55.722331+0000","last_became_active":"2024-09-10T18:45:22.070065+0000","last_became_peered":"2024-09-10T18:45:22.070065+0000","last_unstale":"2024-09-10T18:46:55.722331+0000","last_undegraded":"2024-09-10T18:46:55.722331+0000","last_fullsized":"2024-09-10T18:46:55.722331+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:37.414734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:37.414734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:44.027335+0000","scrub_duration":0.001607996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723863+0000","last_change":"2024-09-10T18:45:24.072477+0000","last_active":"2024-09-10T18:46:55.723863+0000","last_peered":"2024-09-10T18:46:55.723863+0000","last_clean":"2024-09-10T18:46:55.723863+0000","last_became_active":"2024-09-10T18:45:24.072224+0000","last_became_peered":"2024-09-10T18:45:24.072224+0000","last_unstale":"2024-09-10T18:46:55.723863+0000","last_undegraded":"2024-09-10T18:46:55.723863+0000","last_fullsized":"2024-09-10T18:46:55.723863+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"63'4","reported_seq":50,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723860+0000","last_change":"2024-09-10T18:46:46.766906+0000","last_active":"2024-09-10T18:46:55.723860+0000","last_peered":"2024-09-10T18:46:55.723860+0000","last_clean":"2024-09-10T18:46:55.723860+0000","last_became_active":"2024-09-10T18:45:20.059205+0000","last_became_peered":"2024-09-10T18:45:20.059205+0000","last_unstale":"2024-09-10T18:46:55.723860+0000","last_undegraded":"2024-09-10T18:46:55.723860+0000","last_fullsized":"2024-09-10T18:46:55.723860+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4","last_scrub_stamp":"2024-09-10T18:46:46.766766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:46.766766+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:49.051695+0000","scrub_duration":0.0021847250000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.722362+0000","last_change":"2024-09-10T18:46:54.082828+0000","last_active":"2024-09-10T18:46:55.722362+0000","last_peered":"2024-09-10T18:46:55.722362+0000","last_clean":"2024-09-10T18:46:55.722362+0000","last_became_active":"2024-09-10T18:45:17.963630+0000","last_became_peered":"2024-09-10T18:45:17.963630+0000","last_unstale":"2024-09-10T18:46:55.722362+0000","last_undegraded":"2024-09-10T18:46:55.722362+0000","last_fullsized":"2024-09-10T18:46:55.722362+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:54.082761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:54.082761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:05.974208+0000","scrub_duration":0.001210458,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.722008+0000","last_change":"2024-09-10T18:46:35.172936+0000","last_active":"2024-09-10T18:46:55.722008+0000","last_peered":"2024-09-10T18:46:55.722008+0000","last_clean":"2024-09-10T18:46:55.722008+0000","last_became_active":"2024-09-10T18:45:22.072377+0000","last_became_peered":"2024-09-10T18:45:22.072377+0000","last_unstale":"2024-09-10T18:46:55.722008+0000","last_undegraded":"2024-09-10T18:46:55.722008+0000","last_fullsized":"2024-09-10T18:46:55.722008+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_clean_scrub_stamp":"2024-09-10T18:46:35.172861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:57.638389+0000","scrub_duration":0.001552933,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.724306+0000","last_change":"2024-09-10T18:46:53.596450+0000","last_active":"2024-09-10T18:46:55.724306+0000","last_peered":"2024-09-10T18:46:55.724306+0000","last_clean":"2024-09-10T18:46:55.724306+0000","last_became_active":"2024-09-10T18:45:24.077398+0000","last_became_peered":"2024-09-10T18:45:24.077398+0000","last_unstale":"2024-09-10T18:46:55.724306+0000","last_undegraded":"2024-09-10T18:46:55.724306+0000","last_fullsized":"2024-09-10T18:46:55.724306+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:53.596380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:53.596380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:54.667444+0000","scrub_duration":0.0020285519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"79'16","reported_seq":68,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.724329+0000","last_change":"2024-09-10T18:46:48.744340+0000","last_active":"2024-09-10T18:46:55.724329+0000","last_peered":"2024-09-10T18:46:55.724329+0000","last_clean":"2024-09-10T18:46:55.724329+0000","last_became_active":"2024-09-10T18:45:20.421479+0000","last_became_peered":"2024-09-10T18:45:20.421479+0000","last_unstale":"2024-09-10T18:46:55.724329+0000","last_undegraded":"2024-09-10T18:46:55.724329+0000","last_fullsized":"2024-09-10T18:46:55.724329+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"79'16","last_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_deep_scrub":"79'16","last_deep_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_clean_scrub_stamp":"2024-09-10T18:46:48.744287+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:57.831206+0000","scrub_duration":0.0044879259999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"56'1","reported_seq":53,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.724103+0000","last_change":"2024-09-10T18:46:41.147533+0000","last_active":"2024-09-10T18:46:55.724103+0000","last_peered":"2024-09-10T18:46:55.724103+0000","last_clean":"2024-09-10T18:46:55.724103+0000","last_became_active":"2024-09-10T18:45:17.961222+0000","last_became_peered":"2024-09-10T18:45:17.961222+0000","last_unstale":"2024-09-10T18:46:55.724103+0000","last_undegraded":"2024-09-10T18:46:55.724103+0000","last_fullsized":"2024-09-10T18:46:55.724103+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:46:41.147463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:41.147463+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:58.016926+0000","scrub_duration":0.0016797889999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.722733+0000","last_change":"2024-09-10T18:46:27.667088+0000","last_active":"2024-09-10T18:46:55.722733+0000","last_peered":"2024-09-10T18:46:55.722733+0000","last_clean":"2024-09-10T18:46:55.722733+0000","last_became_active":"2024-09-10T18:45:22.072312+0000","last_became_peered":"2024-09-10T18:45:22.072312+0000","last_unstale":"2024-09-10T18:46:55.722733+0000","last_undegraded":"2024-09-10T18:46:55.722733+0000","last_fullsized":"2024-09-10T18:46:55.722733+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:27.667018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:27.667018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:38.065664+0000","scrub_duration":0.001388886,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.724517+0000","last_change":"2024-09-10T18:46:29.791617+0000","last_active":"2024-09-10T18:46:55.724517+0000","last_peered":"2024-09-10T18:46:55.724517+0000","last_clean":"2024-09-10T18:46:55.724517+0000","last_became_active":"2024-09-10T18:45:24.264818+0000","last_became_peered":"2024-09-10T18:45:24.264818+0000","last_unstale":"2024-09-10T18:46:55.724517+0000","last_undegraded":"2024-09-10T18:46:55.724517+0000","last_fullsized":"2024-09-10T18:46:55.724517+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:29.791563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:29.791563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:37.941643+0000","scrub_duration":0.0021031819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"78'2","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723235+0000","last_change":"2024-09-10T18:46:37.581201+0000","last_active":"2024-09-10T18:46:55.723235+0000","last_peered":"2024-09-10T18:46:55.723235+0000","last_clean":"2024-09-10T18:46:55.723235+0000","last_became_active":"2024-09-10T18:45:24.076380+0000","last_became_peered":"2024-09-10T18:45:24.076380+0000","last_unstale":"2024-09-10T18:46:55.723235+0000","last_undegraded":"2024-09-10T18:46:55.723235+0000","last_fullsized":"2024-09-10T18:46:55.723235+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'2","last_scrub_stamp":"2024-09-10T18:46:37.581130+0000","last_deep_scrub":"78'2","last_deep_scrub_stamp":"2024-09-10T18:46:37.581130+0000","last_clean_scrub_stamp":"2024-09-10T18:46:37.581130+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:55.887929+0000","scrub_duration":0.0024563609999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.721532+0000","last_change":"2024-09-10T18:46:28.088657+0000","last_active":"2024-09-10T18:46:55.721532+0000","last_peered":"2024-09-10T18:46:55.721532+0000","last_clean":"2024-09-10T18:46:55.721532+0000","last_became_active":"2024-09-10T18:45:17.962108+0000","last_became_peered":"2024-09-10T18:45:17.962108+0000","last_unstale":"2024-09-10T18:46:55.721532+0000","last_undegraded":"2024-09-10T18:46:55.721532+0000","last_fullsized":"2024-09-10T18:46:55.721532+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:28.088588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:28.088588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:56.550863+0000","scrub_duration":0.00140508,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"63'10","reported_seq":59,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720243+0000","last_change":"2024-09-10T18:46:49.494628+0000","last_active":"2024-09-10T18:46:55.720243+0000","last_peered":"2024-09-10T18:46:55.720243+0000","last_clean":"2024-09-10T18:46:55.720243+0000","last_became_active":"2024-09-10T18:45:20.421094+0000","last_became_peered":"2024-09-10T18:45:20.421094+0000","last_unstale":"2024-09-10T18:46:55.720243+0000","last_undegraded":"2024-09-10T18:46:55.720243+0000","last_fullsized":"2024-09-10T18:46:55.720243+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:46:49.494565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:49.494565+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:08.379972+0000","scrub_duration":0.0036379630000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":73,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.094508+0000","last_change":"2024-09-10T18:46:33.150919+0000","last_active":"2024-09-10T18:46:56.094508+0000","last_peered":"2024-09-10T18:46:56.094508+0000","last_clean":"2024-09-10T18:46:56.094508+0000","last_became_active":"2024-09-10T18:45:22.073281+0000","last_became_peered":"2024-09-10T18:45:22.073281+0000","last_unstale":"2024-09-10T18:46:56.094508+0000","last_undegraded":"2024-09-10T18:46:56.094508+0000","last_fullsized":"2024-09-10T18:46:56.094508+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:33.150852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:33.150852+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:51.299257+0000","scrub_duration":0.0017025339999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.722870+0000","last_change":"2024-09-10T18:45:24.082638+0000","last_active":"2024-09-10T18:46:55.722870+0000","last_peered":"2024-09-10T18:46:55.722870+0000","last_clean":"2024-09-10T18:46:55.722870+0000","last_became_active":"2024-09-10T18:45:24.082318+0000","last_became_peered":"2024-09-10T18:45:24.082318+0000","last_unstale":"2024-09-10T18:46:55.722870+0000","last_undegraded":"2024-09-10T18:46:55.722870+0000","last_fullsized":"2024-09-10T18:46:55.722870+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.724496+0000","last_change":"2024-09-10T18:46:44.694805+0000","last_active":"2024-09-10T18:46:55.724496+0000","last_peered":"2024-09-10T18:46:55.724496+0000","last_clean":"2024-09-10T18:46:55.724496+0000","last_became_active":"2024-09-10T18:45:17.965707+0000","last_became_peered":"2024-09-10T18:45:17.965707+0000","last_unstale":"2024-09-10T18:46:55.724496+0000","last_undegraded":"2024-09-10T18:46:55.724496+0000","last_fullsized":"2024-09-10T18:46:55.724496+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:44.694737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:44.694737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:03.417079+0000","scrub_duration":0.0016330400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"63'11","reported_seq":63,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.724078+0000","last_change":"2024-09-10T18:46:46.230915+0000","last_active":"2024-09-10T18:46:55.724078+0000","last_peered":"2024-09-10T18:46:55.724078+0000","last_clean":"2024-09-10T18:46:55.724078+0000","last_became_active":"2024-09-10T18:45:20.058717+0000","last_became_peered":"2024-09-10T18:45:20.058717+0000","last_unstale":"2024-09-10T18:46:55.724078+0000","last_undegraded":"2024-09-10T18:46:55.724078+0000","last_fullsized":"2024-09-10T18:46:55.724078+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:46:46.230819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:46.230819+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:54.186112+0000","scrub_duration":0.002545322,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720058+0000","last_change":"2024-09-10T18:45:22.068991+0000","last_active":"2024-09-10T18:46:55.720058+0000","last_peered":"2024-09-10T18:46:55.720058+0000","last_clean":"2024-09-10T18:46:55.720058+0000","last_became_active":"2024-09-10T18:45:22.068676+0000","last_became_peered":"2024-09-10T18:45:22.068676+0000","last_unstale":"2024-09-10T18:46:55.720058+0000","last_undegraded":"2024-09-10T18:46:55.720058+0000","last_fullsized":"2024-09-10T18:46:55.720058+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720003+0000","last_change":"2024-09-10T18:46:45.285074+0000","last_active":"2024-09-10T18:46:55.720003+0000","last_peered":"2024-09-10T18:46:55.720003+0000","last_clean":"2024-09-10T18:46:55.720003+0000","last_became_active":"2024-09-10T18:45:24.267251+0000","last_became_peered":"2024-09-10T18:45:24.267251+0000","last_unstale":"2024-09-10T18:46:55.720003+0000","last_undegraded":"2024-09-10T18:46:55.720003+0000","last_fullsized":"2024-09-10T18:46:55.720003+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:45.285015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:45.285015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:56.843562+0000","scrub_duration":0.001362142,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"56'1","reported_seq":55,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.722778+0000","last_change":"2024-09-10T18:46:42.139314+0000","last_active":"2024-09-10T18:46:55.722778+0000","last_peered":"2024-09-10T18:46:55.722778+0000","last_clean":"2024-09-10T18:46:55.722778+0000","last_became_active":"2024-09-10T18:45:17.966286+0000","last_became_peered":"2024-09-10T18:45:17.966286+0000","last_unstale":"2024-09-10T18:46:55.722778+0000","last_undegraded":"2024-09-10T18:46:55.722778+0000","last_fullsized":"2024-09-10T18:46:55.722778+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:46:42.139256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:42.139256+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:57.842540+0000","scrub_duration":0.0014138690000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"63'15","reported_seq":69,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720672+0000","last_change":"2024-09-10T18:46:23.864133+0000","last_active":"2024-09-10T18:46:55.720672+0000","last_peered":"2024-09-10T18:46:55.720672+0000","last_clean":"2024-09-10T18:46:55.720672+0000","last_became_active":"2024-09-10T18:45:20.062728+0000","last_became_peered":"2024-09-10T18:45:20.062728+0000","last_unstale":"2024-09-10T18:46:55.720672+0000","last_undegraded":"2024-09-10T18:46:55.720672+0000","last_fullsized":"2024-09-10T18:46:55.720672+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:46:23.864075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:23.864075+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:29.106766+0000","scrub_duration":0.0020859780000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723231+0000","last_change":"2024-09-10T18:45:22.060202+0000","last_active":"2024-09-10T18:46:55.723231+0000","last_peered":"2024-09-10T18:46:55.723231+0000","last_clean":"2024-09-10T18:46:55.723231+0000","last_became_active":"2024-09-10T18:45:22.059794+0000","last_became_peered":"2024-09-10T18:45:22.059794+0000","last_unstale":"2024-09-10T18:46:55.723231+0000","last_undegraded":"2024-09-10T18:46:55.723231+0000","last_fullsized":"2024-09-10T18:46:55.723231+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:57.811180+0000","last_change":"2024-09-10T18:46:57.811148+0000","last_active":"2024-09-10T18:46:57.811180+0000","last_peered":"2024-09-10T18:46:57.811180+0000","last_clean":"2024-09-10T18:46:57.811180+0000","last_became_active":"2024-09-10T18:45:24.264427+0000","last_became_peered":"2024-09-10T18:45:24.264427+0000","last_unstale":"2024-09-10T18:46:57.811180+0000","last_undegraded":"2024-09-10T18:46:57.811180+0000","last_fullsized":"2024-09-10T18:46:57.811180+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:57.811074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:57.811074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:12.950473+0000","scrub_duration":0.0013689419999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.725157+0000","last_change":"2024-09-10T18:46:47.734860+0000","last_active":"2024-09-10T18:46:55.725157+0000","last_peered":"2024-09-10T18:46:55.725157+0000","last_clean":"2024-09-10T18:46:55.725157+0000","last_became_active":"2024-09-10T18:45:17.963257+0000","last_became_peered":"2024-09-10T18:45:17.963257+0000","last_unstale":"2024-09-10T18:46:55.725157+0000","last_undegraded":"2024-09-10T18:46:55.725157+0000","last_fullsized":"2024-09-10T18:46:55.725157+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:47.734803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:47.734803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:05.847815+0000","scrub_duration":0.001381134,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"63'12","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.724558+0000","last_change":"2024-09-10T18:46:52.771640+0000","last_active":"2024-09-10T18:46:55.724558+0000","last_peered":"2024-09-10T18:46:55.724558+0000","last_clean":"2024-09-10T18:46:55.724558+0000","last_became_active":"2024-09-10T18:45:20.058391+0000","last_became_peered":"2024-09-10T18:45:20.058391+0000","last_unstale":"2024-09-10T18:46:55.724558+0000","last_undegraded":"2024-09-10T18:46:55.724558+0000","last_fullsized":"2024-09-10T18:46:55.724558+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:46:52.771570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:52.771570+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:04.251299+0000","scrub_duration":0.0021490089999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.719940+0000","last_change":"2024-09-10T18:46:34.283979+0000","last_active":"2024-09-10T18:46:55.719940+0000","last_peered":"2024-09-10T18:46:55.719940+0000","last_clean":"2024-09-10T18:46:55.719940+0000","last_became_active":"2024-09-10T18:45:22.073250+0000","last_became_peered":"2024-09-10T18:45:22.073250+0000","last_unstale":"2024-09-10T18:46:55.719940+0000","last_undegraded":"2024-09-10T18:46:55.719940+0000","last_fullsized":"2024-09-10T18:46:55.719940+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:34.283901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:34.283901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:34.686745+0000","scrub_duration":0.002360334,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65,"ondisk_log_size":65,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":417,"ondisk_log_size":417,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":95,"num_read_kb":90,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299417,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298568,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446584,"statfs":{"total":95995035648,"available":95689302016,"internally_reserved":0,"allocated":1245184,"data_stored":888558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.29999999999999999}]},{"osd":1,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.38500000000000001}]},{"osd":2,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.32400000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.35399999999999998}]},{"osd":4,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.432},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.496}]},{"osd":5,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.47699999999999998}]},{"osd":6,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.44400000000000001}]}]},{"osd":6,"up_from":43,"seq":184683593758,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298556,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446596,"statfs":{"total":95995035648,"available":95689314304,"internally_reserved":0,"allocated":1232896,"data_stored":888634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.46300000000000002,"15min":0.45900000000000002},"min":{"1min":0.32900000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.496,"15min":0.49199999999999999},"min":{"1min":0.32800000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.438}]},{"osd":1,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.315,"5min":0.249,"15min":0.249},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.46700000000000003,"15min":0.46300000000000002},"min":{"1min":0.33700000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.65900000000000003,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.45900000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.48299999999999998,"15min":0.48099999999999998},"min":{"1min":0.34000000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.49299999999999999,"15min":0.49099999999999999},"min":{"1min":0.33600000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68700000000000006,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.34999999999999998}]},{"osd":3,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.48399999999999999,"15min":0.48299999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.51200000000000001,"15min":0.51000000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.49299999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.39700000000000002,"15min":0.38800000000000001},"min":{"1min":0.26900000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.307}]},{"osd":5,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.41899999999999998,"15min":0.41499999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.42499999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.38100000000000001}]},{"osd":7,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.48299999999999998}]}]},{"osd":5,"up_from":37,"seq":158913789986,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297972,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447180,"statfs":{"total":95995035648,"available":95689912320,"internally_reserved":0,"allocated":630784,"data_stored":296879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.52200000000000002,"15min":0.52600000000000002},"min":{"1min":0.30299999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70099999999999996,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.58299999999999996,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.50600000000000001}]},{"osd":1,"last update":"Tue Sep 10 18:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.55200000000000005,"15min":0.56000000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.69699999999999995,"5min":0.87,"15min":0.87},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.497,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.64200000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.57199999999999995}]},{"osd":2,"last update":"Tue Sep 10 18:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.51800000000000002,"15min":0.52000000000000002},"min":{"1min":0.33300000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.71299999999999997,"5min":0.755,"15min":0.755},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53600000000000003,"15min":0.54100000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79600000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.44500000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52500000000000002,"15min":0.52700000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76100000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.54200000000000004,"15min":0.54600000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80100000000000005,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.495}]},{"osd":4,"last update":"Tue Sep 10 18:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.64500000000000002,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.40500000000000003,"15min":0.40300000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.65700000000000003,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.377}]},{"osd":6,"last update":"Tue Sep 10 18:46:44 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.42199999999999999,"15min":0.41599999999999998},"min":{"1min":0.29299999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.67500000000000004,"5min":0.753,"15min":0.753},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.46500000000000002,"15min":0.46100000000000002},"min":{"1min":0.36199999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.71199999999999997,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.34300000000000003}]},{"osd":7,"last update":"Tue Sep 10 18:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.309},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.28000000000000003}]}]},{"osd":4,"up_from":30,"seq":128849018920,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297988,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447164,"statfs":{"total":95995035648,"available":95689895936,"internally_reserved":0,"allocated":651264,"data_stored":297726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:58 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.52200000000000002,"15min":0.52400000000000002},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.76800000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.52200000000000002,"15min":0.53600000000000003},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.64000000000000001,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.52100000000000002}]},{"osd":1,"last update":"Tue Sep 10 18:46:58 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.68400000000000005,"5min":2.609,"15min":2.609},"last":0.627},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.54900000000000004,"15min":0.53300000000000003},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.80500000000000005,"5min":2.589,"15min":2.589},"last":0.65800000000000003}]},{"osd":2,"last update":"Tue Sep 10 18:46:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.55400000000000005,"15min":0.54300000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.70399999999999996,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54300000000000004,"15min":0.55300000000000005},"min":{"1min":0.33200000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.73299999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.60099999999999998}]},{"osd":3,"last update":"Tue Sep 10 18:46:58 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.52500000000000002,"15min":0.52900000000000003},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.67500000000000004,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.52900000000000003,"15min":0.53900000000000003},"min":{"1min":0.35299999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.751,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.501}]},{"osd":5,"last update":"Tue Sep 10 18:46:22 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.35899999999999999,"15min":0.34999999999999998},"min":{"1min":0.26800000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.44,"5min":0.371,"15min":0.35899999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.57699999999999996}]},{"osd":6,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.42999999999999999,"15min":0.42399999999999999},"min":{"1min":0.311,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.68799999999999994,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.40100000000000002,"15min":0.39100000000000001},"min":{"1min":0.34799999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.55600000000000005}]},{"osd":7,"last update":"Tue Sep 10 18:46:08 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.64400000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182445,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298008,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447144,"statfs":{"total":95995035648,"available":95689875456,"internally_reserved":0,"allocated":663552,"data_stored":297505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:35 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.378,"15min":0.38400000000000001},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.23599999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.40100000000000002,"15min":0.40300000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.38600000000000001}]},{"osd":1,"last update":"Tue Sep 10 18:46:35 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.40400000000000003,"15min":0.40899999999999997},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40899999999999997,"15min":0.41499999999999998},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.63800000000000001,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.316}]},{"osd":2,"last update":"Tue Sep 10 18:46:35 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.43099999999999999,"15min":0.438},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.377,"5min":0.41999999999999998,"15min":0.433},"min":{"1min":0.27200000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.53600000000000003,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.33700000000000002}]},{"osd":4,"last update":"Tue Sep 10 18:46:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54300000000000004,"15min":0.54400000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.375},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56200000000000006,"15min":0.57099999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.82299999999999995,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.41399999999999998}]},{"osd":5,"last update":"Tue Sep 10 18:46:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39700000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.70299999999999996,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.41399999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80800000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.48799999999999999}]},{"osd":6,"last update":"Tue Sep 10 18:46:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.56000000000000005,"15min":0.56299999999999994},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79100000000000004,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55400000000000005,"15min":0.55600000000000005},"min":{"1min":0.40799999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.443}]},{"osd":7,"last update":"Tue Sep 10 18:46:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.54800000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411377,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297976,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447176,"statfs":{"total":95995035648,"available":95689908224,"internally_reserved":0,"allocated":634880,"data_stored":297314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.376,"15min":0.35899999999999999},"min":{"1min":0.246,"5min":0.223,"15min":0.223},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.38300000000000001,"15min":0.371},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.39500000000000002}]},{"osd":1,"last update":"Tue Sep 10 18:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.375,"15min":0.36699999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.42499999999999999,"15min":0.41299999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.32000000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:46:36 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46800000000000003,"15min":0.46500000000000002},"min":{"1min":0.30099999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.42899999999999999,"15min":0.43099999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.33900000000000002}]},{"osd":4,"last update":"Tue Sep 10 18:46:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.69899999999999995,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56899999999999995,"15min":0.57099999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.64800000000000002,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.44800000000000001}]},{"osd":5,"last update":"Tue Sep 10 18:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56100000000000005,"15min":0.55800000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.42899999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.432}]},{"osd":6,"last update":"Tue Sep 10 18:46:49 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58999999999999997,"15min":0.58599999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.46200000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.59799999999999998}]},{"osd":7,"last update":"Tue Sep 10 18:46:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.52500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298004,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447148,"statfs":{"total":95995035648,"available":95689879552,"internally_reserved":0,"allocated":667648,"data_stored":297780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:48 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34399999999999997,"15min":0.32000000000000001},"min":{"1min":0.245,"5min":0.224,"15min":0.224},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.36599999999999999,"15min":0.33600000000000002},"min":{"1min":0.252,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.53700000000000003,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.375}]},{"osd":2,"last update":"Tue Sep 10 18:46:11 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.372,"15min":0.36099999999999999},"min":{"1min":0.27900000000000003,"5min":0.248,"15min":0.248},"max":{"1min":0.47199999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.39000000000000001,"15min":0.38100000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.49099999999999999,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.44400000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:46:34 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.45400000000000001,"15min":0.45800000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.42299999999999999,"15min":0.42199999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.55400000000000005,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.41299999999999998}]},{"osd":4,"last update":"Tue Sep 10 18:47:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.57099999999999995,"15min":0.55900000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.81399999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70399999999999996,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.51700000000000002}]},{"osd":5,"last update":"Tue Sep 10 18:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60299999999999998,"15min":0.60399999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63700000000000001,"15min":0.64100000000000001},"min":{"1min":0.44600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93300000000000005,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.53400000000000003}]},{"osd":6,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.46300000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.66400000000000003,"15min":0.67100000000000004},"min":{"1min":0.41999999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82299999999999995,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.502}]},{"osd":7,"last update":"Tue Sep 10 18:46:11 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.55900000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738427,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298560,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446592,"statfs":{"total":95995035648,"available":95689310208,"internally_reserved":0,"allocated":1236992,"data_stored":887368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 18:46:52 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.32500000000000001,"15min":0.30299999999999999},"min":{"1min":0.219,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.58799999999999997,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33700000000000002,"15min":0.315},"min":{"1min":0.221,"5min":0.187,"15min":0.187},"max":{"1min":0.59399999999999997,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.32100000000000001}]},{"osd":2,"last update":"Tue Sep 10 18:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.40300000000000002,"15min":0.38900000000000001},"min":{"1min":0.30299999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.376,"15min":0.36199999999999999},"min":{"1min":0.26600000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.33300000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:46:31 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.39500000000000002,"15min":0.39800000000000002},"min":{"1min":0.26100000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.66100000000000003,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.41199999999999998,"15min":0.41299999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.30499999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:47:00 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54400000000000004,"15min":0.54100000000000004},"min":{"1min":0.41899999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.79200000000000004,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.56899999999999995,"15min":0.53500000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.69899999999999995,"5min":4.6189999999999998,"15min":4.6189999999999998},"last":0.46400000000000002}]},{"osd":5,"last update":"Tue Sep 10 18:46:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.42999999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.58699999999999997,"15min":0.55800000000000005},"min":{"1min":0.42699999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":4.6429999999999998,"5min":4.6429999999999998,"15min":4.6429999999999998},"last":0.44600000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:46:42 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56100000000000005,"15min":0.55500000000000005},"min":{"1min":0.42099999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.746,"15min":0.77300000000000002},"min":{"1min":0.40899999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.878,"5min":4.6630000000000003,"15min":4.6630000000000003},"last":0.40799999999999997}]},{"osd":7,"last update":"Tue Sep 10 18:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.60599999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T18:47:02.082 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph status --format=json 2024-09-10T18:47:02.347 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:47:02.522 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:02 smithi103 bash[63812]: Getting image source signatures 2024-09-10T18:47:02.522 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:02 smithi103 bash[63812]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-10T18:47:02.523 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:02 smithi103 bash[63812]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-10T18:47:02.523 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:02 smithi103 bash[63812]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-10T18:47:03.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:02 smithi018 ceph-mon[40196]: 2.d scrub starts 2024-09-10T18:47:03.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:02 smithi018 ceph-mon[40196]: 2.d scrub ok 2024-09-10T18:47:03.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:02 smithi018 ceph-mon[40196]: 5.9 deep-scrub starts 2024-09-10T18:47:03.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:02 smithi018 ceph-mon[40196]: 5.9 deep-scrub ok 2024-09-10T18:47:03.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:02 smithi018 ceph-mon[29336]: 2.d scrub starts 2024-09-10T18:47:03.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:02 smithi018 ceph-mon[29336]: 2.d scrub ok 2024-09-10T18:47:03.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:02 smithi018 ceph-mon[29336]: 5.9 deep-scrub starts 2024-09-10T18:47:03.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:02 smithi018 ceph-mon[29336]: 5.9 deep-scrub ok 2024-09-10T18:47:03.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:02 smithi103 ceph-mon[33163]: 2.d scrub starts 2024-09-10T18:47:03.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:02 smithi103 ceph-mon[33163]: 2.d scrub ok 2024-09-10T18:47:03.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:02 smithi103 ceph-mon[33163]: 5.9 deep-scrub starts 2024-09-10T18:47:03.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:02 smithi103 ceph-mon[33163]: 5.9 deep-scrub ok 2024-09-10T18:47:03.935 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:03 smithi018 ceph-mon[40196]: 4.17 scrub starts 2024-09-10T18:47:03.935 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:03 smithi018 ceph-mon[40196]: 4.17 scrub ok 2024-09-10T18:47:03.935 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:03 smithi018 ceph-mon[40196]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 2 objects/s recovering 2024-09-10T18:47:03.935 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:03 smithi018 ceph-mon[40196]: 6.1d deep-scrub starts 2024-09-10T18:47:03.935 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:03 smithi018 ceph-mon[40196]: 6.1d deep-scrub ok 2024-09-10T18:47:03.935 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:03 smithi018 ceph-mon[40196]: 6.8 scrub starts 2024-09-10T18:47:03.935 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:03 smithi018 ceph-mon[40196]: 6.8 scrub ok 2024-09-10T18:47:03.936 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:03 smithi018 ceph-mon[29336]: 4.17 scrub starts 2024-09-10T18:47:03.936 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:03 smithi018 ceph-mon[29336]: 4.17 scrub ok 2024-09-10T18:47:03.936 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:03 smithi018 ceph-mon[29336]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 2 objects/s recovering 2024-09-10T18:47:03.936 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:03 smithi018 ceph-mon[29336]: 6.1d deep-scrub starts 2024-09-10T18:47:03.936 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:03 smithi018 ceph-mon[29336]: 6.1d deep-scrub ok 2024-09-10T18:47:03.936 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:03 smithi018 ceph-mon[29336]: 6.8 scrub starts 2024-09-10T18:47:03.936 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:03 smithi018 ceph-mon[29336]: 6.8 scrub ok 2024-09-10T18:47:04.022 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:03 smithi103 bash[63812]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-10T18:47:04.023 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:03 smithi103 bash[63812]: Writing manifest to image destination 2024-09-10T18:47:04.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:03 smithi103 ceph-mon[33163]: 4.17 scrub starts 2024-09-10T18:47:04.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:03 smithi103 ceph-mon[33163]: 4.17 scrub ok 2024-09-10T18:47:04.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:03 smithi103 ceph-mon[33163]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 2 objects/s recovering 2024-09-10T18:47:04.024 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:03 smithi103 ceph-mon[33163]: 6.1d deep-scrub starts 2024-09-10T18:47:04.024 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:03 smithi103 ceph-mon[33163]: 6.1d deep-scrub ok 2024-09-10T18:47:04.024 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:03 smithi103 ceph-mon[33163]: 6.8 scrub starts 2024-09-10T18:47:04.024 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:03 smithi103 ceph-mon[33163]: 6.8 scrub ok 2024-09-10T18:47:04.380 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:47:04.380 INFO:teuthology.orchestra.run.smithi018.stdout:{"fsid":"ba327f6c-6fa3-11ef-bcea-c7b262605968","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":316,"monmap":{"epoch":3,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":83,"num_osds":8,"num_up_osds":8,"osd_up_since":1725993905,"num_in_osds":8,"osd_in_since":1725993887,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":132}],"num_pgs":132,"num_pools":6,"num_objects":196,"data_bytes":596188,"bytes_used":2442887168,"bytes_avail":765517398016,"bytes_total":767960285184,"recovering_objects_per_sec":2,"recovering_bytes_per_sec":27,"recovering_keys_per_sec":0,"num_objects_recovered":17,"num_bytes_recovered":220,"num_keys_recovered":0,"read_bytes_sec":1279,"read_op_per_sec":1,"write_op_per_sec":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"services":{"dashboard":"https://172.21.15.18:8443/","prometheus":"http://172.21.15.18:9283/"}},"servicemap":{"epoch":14,"modified":"2024-09-10T18:46:57.028399+0000","services":{"rgw":{"daemons":{"summary":"","14460":{"start_epoch":12,"start_stamp":"2024-09-10T18:45:25.199225+0000","gid":14460,"addr":"172.21.15.18:0/3895312980","metadata":{"arch":"x86_64","ceph_release":"quincy","ceph_version":"ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable)","ceph_version_short":"17.2.7-1606-g5be57395","container_hostname":"smithi018","container_image":"quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16","cpu":"Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz","distro":"centos","distro_description":"CentOS Stream 9","distro_version":"9","frontend_config#0":"beast port=80","frontend_type#0":"beast","hostname":"smithi018","id":"foo.a","kernel_description":"#1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024","kernel_version":"5.14.0-505.el9.x86_64","mem_swap_kb":"0","mem_total_kb":"32493496","num_handles":"1","os":"Linux","pid":"2","realm_id":"","realm_name":"","zone_id":"0a27a0b0-e8a8-41c2-bb1b-8a59ee973f43","zone_name":"default","zonegroup_id":"48d110f7-3055-40a1-811b-d1f5b65fef07","zonegroup_name":"default"},"task_status":{}}}}}},"progress_events":{"239095f7-2473-41a5-9a5d-bd1e1a5523a5":{"message":"Updating node-exporter deployment (+2 -> 2) (6s)\n [==============..............] (remaining: 6s)","progress":0.5,"add_to_ceph_s":true},"ce2f3395-290c-40e4-b4f5-894215ffcef6":{"message":"Global Recovery Event (0s)\n [............................] ","progress":0,"add_to_ceph_s":true}}} 2024-09-10T18:47:04.380 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 podman[63812]: 2024-09-10 18:47:04.095731223 +0000 UTC m=+2.515968904 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-10T18:47:04.381 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 podman[63812]: 2024-09-10 18:47:04.194640169 +0000 UTC m=+2.614877845 container create 5606b97e04a08016a1f4cebb321b8923acfb5638165a7e4402ba137f47d54c17 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T18:47:04.773 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 podman[63812]: 2024-09-10 18:47:04.380106305 +0000 UTC m=+2.800343978 container init 5606b97e04a08016a1f4cebb321b8923acfb5638165a7e4402ba137f47d54c17 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T18:47:04.773 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 podman[63812]: 2024-09-10 18:47:04.383087966 +0000 UTC m=+2.803325640 container start 5606b97e04a08016a1f4cebb321b8923acfb5638165a7e4402ba137f47d54c17 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T18:47:04.773 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.386Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-10T18:47:04.773 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.386Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-10T18:47:04.773 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.387Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-10T18:47:04.773 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.387Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-10T18:47:04.773 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-10T18:47:04.773 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-10T18:47:04.774 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-10T18:47:04.774 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=arp 2024-09-10T18:47:04.774 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-10T18:47:04.774 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-10T18:47:04.774 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-10T18:47:04.774 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-10T18:47:04.774 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-10T18:47:04.774 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-10T18:47:04.775 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-10T18:47:04.775 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-10T18:47:04.775 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=edac 2024-09-10T18:47:04.775 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-10T18:47:04.775 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-10T18:47:04.775 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-10T18:47:04.775 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-10T18:47:04.775 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-10T18:47:04.775 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-10T18:47:04.775 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-10T18:47:04.775 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-10T18:47:04.775 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-10T18:47:04.776 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-10T18:47:04.776 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-10T18:47:04.776 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-10T18:47:04.776 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-10T18:47:04.776 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-10T18:47:04.776 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-10T18:47:04.776 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-10T18:47:04.776 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=os 2024-09-10T18:47:04.776 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-10T18:47:04.776 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-10T18:47:04.776 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-10T18:47:04.776 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-10T18:47:04.776 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-10T18:47:04.777 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-10T18:47:04.777 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-10T18:47:04.777 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=stat 2024-09-10T18:47:04.777 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-10T18:47:04.777 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-10T18:47:04.777 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-10T18:47:04.777 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=time 2024-09-10T18:47:04.777 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-10T18:47:04.777 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=uname 2024-09-10T18:47:04.777 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-10T18:47:04.777 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-10T18:47:04.777 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.388Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-10T18:47:04.777 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.389Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-10T18:47:04.778 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b[63886]: ts=2024-09-10T18:47:04.389Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-10T18:47:04.778 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 bash[63812]: 5606b97e04a08016a1f4cebb321b8923acfb5638165a7e4402ba137f47d54c17 2024-09-10T18:47:04.778 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 18:47:04 smithi103 systemd[1]: Started Ceph node-exporter.b for ba327f6c-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:47:05.032 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:04 smithi018 ceph-mon[40196]: 5.3 scrub starts 2024-09-10T18:47:05.032 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:04 smithi018 ceph-mon[40196]: 5.3 scrub ok 2024-09-10T18:47:05.033 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:04 smithi018 ceph-mon[40196]: 4.1b scrub starts 2024-09-10T18:47:05.033 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:04 smithi018 ceph-mon[40196]: 4.1b scrub ok 2024-09-10T18:47:05.033 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:04 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3310507607' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-10T18:47:05.033 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:04 smithi018 ceph-mon[40196]: 5.c deep-scrub starts 2024-09-10T18:47:05.033 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:04 smithi018 ceph-mon[40196]: 5.c deep-scrub ok 2024-09-10T18:47:05.033 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:04 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:05.033 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:04 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:05.033 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:04 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:05.033 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:04 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:05.033 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:04 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:05.033 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:04 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:47:05.034 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:04 smithi018 ceph-mon[29336]: 5.3 scrub starts 2024-09-10T18:47:05.034 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:04 smithi018 ceph-mon[29336]: 5.3 scrub ok 2024-09-10T18:47:05.034 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:04 smithi018 ceph-mon[29336]: 4.1b scrub starts 2024-09-10T18:47:05.034 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:04 smithi018 ceph-mon[29336]: 4.1b scrub ok 2024-09-10T18:47:05.034 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:04 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3310507607' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-10T18:47:05.034 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:04 smithi018 ceph-mon[29336]: 5.c deep-scrub starts 2024-09-10T18:47:05.034 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:04 smithi018 ceph-mon[29336]: 5.c deep-scrub ok 2024-09-10T18:47:05.034 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:04 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:05.034 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:04 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:05.034 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:04 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:05.035 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:04 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:05.035 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:04 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:05.035 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:04 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:47:05.058 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2024-09-10T18:47:05.058 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T18:47:05.195 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-mon[33163]: 5.3 scrub starts 2024-09-10T18:47:05.196 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-mon[33163]: 5.3 scrub ok 2024-09-10T18:47:05.196 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-mon[33163]: 4.1b scrub starts 2024-09-10T18:47:05.196 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-mon[33163]: 4.1b scrub ok 2024-09-10T18:47:05.196 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3310507607' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-10T18:47:05.196 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-mon[33163]: 5.c deep-scrub starts 2024-09-10T18:47:05.196 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-mon[33163]: 5.c deep-scrub ok 2024-09-10T18:47:05.196 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:05.196 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:05.196 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:05.196 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:05.196 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:05.196 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:04 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:47:05.979 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:47:06.390 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:06 smithi018 ceph-mon[40196]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 23 B/s, 1 objects/s recovering 2024-09-10T18:47:06.390 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:06 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:06.390 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:06 smithi018 ceph-mon[40196]: 2.6 scrub starts 2024-09-10T18:47:06.390 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:06 smithi018 ceph-mon[40196]: 2.6 scrub ok 2024-09-10T18:47:06.390 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:06 smithi018 ceph-mon[40196]: 1.0 scrub starts 2024-09-10T18:47:06.390 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:06 smithi018 ceph-mon[40196]: 1.0 scrub ok 2024-09-10T18:47:06.390 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:06 smithi018 ceph-mon[29336]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 23 B/s, 1 objects/s recovering 2024-09-10T18:47:06.390 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:06 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:06.391 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:06 smithi018 ceph-mon[29336]: 2.6 scrub starts 2024-09-10T18:47:06.391 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:06 smithi018 ceph-mon[29336]: 2.6 scrub ok 2024-09-10T18:47:06.391 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:06 smithi018 ceph-mon[29336]: 1.0 scrub starts 2024-09-10T18:47:06.391 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:06 smithi018 ceph-mon[29336]: 1.0 scrub ok 2024-09-10T18:47:06.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:47:06 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:47:06.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:06 smithi103 ceph-mon[33163]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 23 B/s, 1 objects/s recovering 2024-09-10T18:47:06.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:06 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:06.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:06 smithi103 ceph-mon[33163]: 2.6 scrub starts 2024-09-10T18:47:06.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:06 smithi103 ceph-mon[33163]: 2.6 scrub ok 2024-09-10T18:47:06.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:06 smithi103 ceph-mon[33163]: 1.0 scrub starts 2024-09-10T18:47:06.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:06 smithi103 ceph-mon[33163]: 1.0 scrub ok 2024-09-10T18:47:07.119 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:07 smithi018 ceph-mon[29336]: 6.11 scrub starts 2024-09-10T18:47:07.119 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:07 smithi018 ceph-mon[29336]: 6.11 scrub ok 2024-09-10T18:47:07.119 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:07 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:07.119 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:07 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:47:07.119 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:07 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:47:07.119 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:07 smithi018 ceph-mon[40196]: 6.11 scrub starts 2024-09-10T18:47:07.119 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:07 smithi018 ceph-mon[40196]: 6.11 scrub ok 2024-09-10T18:47:07.120 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:07 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:07.120 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:07 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:47:07.120 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:07 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:47:07.441 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:07 smithi103 ceph-mon[33163]: 6.11 scrub starts 2024-09-10T18:47:07.442 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:07 smithi103 ceph-mon[33163]: 6.11 scrub ok 2024-09-10T18:47:07.442 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:07 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:07.442 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:07 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:47:07.442 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:07 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:47:07.821 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:47:07.823 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-09-10T18:47:08.359 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:08 smithi103 ceph-mon[33163]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.6 KiB/s rd, 99 B/s wr, 5 op/s; 21 B/s, 1 objects/s recovering 2024-09-10T18:47:08.360 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:08 smithi103 ceph-mon[33163]: osdmap e84: 8 total, 8 up, 8 in 2024-09-10T18:47:08.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:08 smithi018 ceph-mon[29336]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.6 KiB/s rd, 99 B/s wr, 5 op/s; 21 B/s, 1 objects/s recovering 2024-09-10T18:47:08.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:08 smithi018 ceph-mon[29336]: osdmap e84: 8 total, 8 up, 8 in 2024-09-10T18:47:08.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:08 smithi018 ceph-mon[40196]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.6 KiB/s rd, 99 B/s wr, 5 op/s; 21 B/s, 1 objects/s recovering 2024-09-10T18:47:08.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:08 smithi018 ceph-mon[40196]: osdmap e84: 8 total, 8 up, 8 in 2024-09-10T18:47:08.673 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":53,"stamp":"2024-09-10T18:47:07.085263+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":953,"num_read_kb":810,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1180956,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":579,"ondisk_log_size":579,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385620,"kb_used_data":6800,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575596,"statfs":{"total":767960285184,"available":765517410304,"internally_reserved":0,"allocated":6963200,"data_stored":4151764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":27,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.056912"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.724426+0000","last_change":"2024-09-10T18:46:33.824057+0000","last_active":"2024-09-10T18:46:55.724426+0000","last_peered":"2024-09-10T18:46:55.724426+0000","last_clean":"2024-09-10T18:46:55.724426+0000","last_became_active":"2024-09-10T18:45:24.265488+0000","last_became_peered":"2024-09-10T18:45:24.265488+0000","last_unstale":"2024-09-10T18:46:55.724426+0000","last_undegraded":"2024-09-10T18:46:55.724426+0000","last_fullsized":"2024-09-10T18:46:55.724426+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:33.823989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:33.823989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:44.131457+0000","scrub_duration":0.0034389770000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:00.536103+0000","last_change":"2024-09-10T18:47:00.536064+0000","last_active":"2024-09-10T18:47:00.536103+0000","last_peered":"2024-09-10T18:47:00.536103+0000","last_clean":"2024-09-10T18:47:00.536103+0000","last_became_active":"2024-09-10T18:45:17.959475+0000","last_became_peered":"2024-09-10T18:45:17.959475+0000","last_unstale":"2024-09-10T18:47:00.536103+0000","last_undegraded":"2024-09-10T18:47:00.536103+0000","last_fullsized":"2024-09-10T18:47:00.536103+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:00.535992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:47:00.535992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:13.246010+0000","scrub_duration":0.001731841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"63'15","reported_seq":69,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.724433+0000","last_change":"2024-09-10T18:46:36.856804+0000","last_active":"2024-09-10T18:46:55.724433+0000","last_peered":"2024-09-10T18:46:55.724433+0000","last_clean":"2024-09-10T18:46:55.724433+0000","last_became_active":"2024-09-10T18:45:20.052531+0000","last_became_peered":"2024-09-10T18:45:20.052531+0000","last_unstale":"2024-09-10T18:46:55.724433+0000","last_undegraded":"2024-09-10T18:46:55.724433+0000","last_fullsized":"2024-09-10T18:46:55.724433+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:46:36.856740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:36.856740+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:03.997270+0000","scrub_duration":0.0018131040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.719487+0000","last_change":"2024-09-10T18:46:40.313306+0000","last_active":"2024-09-10T18:46:55.719487+0000","last_peered":"2024-09-10T18:46:55.719487+0000","last_clean":"2024-09-10T18:46:55.719487+0000","last_became_active":"2024-09-10T18:45:22.065619+0000","last_became_peered":"2024-09-10T18:45:22.065619+0000","last_unstale":"2024-09-10T18:46:55.719487+0000","last_undegraded":"2024-09-10T18:46:55.719487+0000","last_fullsized":"2024-09-10T18:46:55.719487+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:40.313218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:40.313218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:52.103586+0000","scrub_duration":0.0017922750000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.725424+0000","last_change":"2024-09-10T18:46:18.890844+0000","last_active":"2024-09-10T18:46:55.725424+0000","last_peered":"2024-09-10T18:46:55.725424+0000","last_clean":"2024-09-10T18:46:55.725424+0000","last_became_active":"2024-09-10T18:45:17.966153+0000","last_became_peered":"2024-09-10T18:45:17.966153+0000","last_unstale":"2024-09-10T18:46:55.725424+0000","last_undegraded":"2024-09-10T18:46:55.725424+0000","last_fullsized":"2024-09-10T18:46:55.725424+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:18.890702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:18.890702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:47.062002+0000","scrub_duration":0.0019382589999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"83'13","reported_seq":64,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:59.906764+0000","last_change":"2024-09-10T18:46:59.906707+0000","last_active":"2024-09-10T18:46:59.906764+0000","last_peered":"2024-09-10T18:46:59.906764+0000","last_clean":"2024-09-10T18:46:59.906764+0000","last_became_active":"2024-09-10T18:45:20.061266+0000","last_became_peered":"2024-09-10T18:45:20.061266+0000","last_unstale":"2024-09-10T18:46:59.906764+0000","last_undegraded":"2024-09-10T18:46:59.906764+0000","last_fullsized":"2024-09-10T18:46:59.906764+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'13","last_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_clean_scrub_stamp":"2024-09-10T18:46:59.906634+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:29.738998+0000","scrub_duration":0.003988892,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.722125+0000","last_change":"2024-09-10T18:45:22.073139+0000","last_active":"2024-09-10T18:46:55.722125+0000","last_peered":"2024-09-10T18:46:55.722125+0000","last_clean":"2024-09-10T18:46:55.722125+0000","last_became_active":"2024-09-10T18:45:22.072822+0000","last_became_peered":"2024-09-10T18:45:22.072822+0000","last_unstale":"2024-09-10T18:46:55.722125+0000","last_undegraded":"2024-09-10T18:46:55.722125+0000","last_fullsized":"2024-09-10T18:46:55.722125+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:00.946332+0000","last_change":"2024-09-10T18:47:00.946309+0000","last_active":"2024-09-10T18:47:00.946332+0000","last_peered":"2024-09-10T18:47:00.946332+0000","last_clean":"2024-09-10T18:47:00.946332+0000","last_became_active":"2024-09-10T18:45:24.081002+0000","last_became_peered":"2024-09-10T18:45:24.081002+0000","last_unstale":"2024-09-10T18:47:00.946332+0000","last_undegraded":"2024-09-10T18:47:00.946332+0000","last_fullsized":"2024-09-10T18:47:00.946332+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:00.946255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:00.946255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:14.872519+0000","scrub_duration":0.001207629,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.724266+0000","last_change":"2024-09-10T18:46:46.605919+0000","last_active":"2024-09-10T18:46:55.724266+0000","last_peered":"2024-09-10T18:46:55.724266+0000","last_clean":"2024-09-10T18:46:55.724266+0000","last_became_active":"2024-09-10T18:45:17.961097+0000","last_became_peered":"2024-09-10T18:45:17.961097+0000","last_unstale":"2024-09-10T18:46:55.724266+0000","last_undegraded":"2024-09-10T18:46:55.724266+0000","last_fullsized":"2024-09-10T18:46:55.724266+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:46.605814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:46.605814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:51.831114+0000","scrub_duration":0.001570625,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"63'5","reported_seq":54,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:04.002255+0000","last_change":"2024-09-10T18:47:04.002228+0000","last_active":"2024-09-10T18:47:04.002255+0000","last_peered":"2024-09-10T18:47:04.002255+0000","last_clean":"2024-09-10T18:47:04.002255+0000","last_became_active":"2024-09-10T18:45:20.057567+0000","last_became_peered":"2024-09-10T18:45:20.057567+0000","last_unstale":"2024-09-10T18:47:04.002255+0000","last_undegraded":"2024-09-10T18:47:04.002255+0000","last_fullsized":"2024-09-10T18:47:04.002255+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-10T18:47:04.002164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:04.002164+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:20.148733+0000","scrub_duration":0.026374999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.724304+0000","last_change":"2024-09-10T18:46:22.547871+0000","last_active":"2024-09-10T18:46:55.724304+0000","last_peered":"2024-09-10T18:46:55.724304+0000","last_clean":"2024-09-10T18:46:55.724304+0000","last_became_active":"2024-09-10T18:45:22.076575+0000","last_became_peered":"2024-09-10T18:45:22.076575+0000","last_unstale":"2024-09-10T18:46:55.724304+0000","last_undegraded":"2024-09-10T18:46:55.724304+0000","last_fullsized":"2024-09-10T18:46:55.724304+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:22.547800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:22.547800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:32.775669+0000","scrub_duration":0.001880622,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.721776+0000","last_change":"2024-09-10T18:46:36.158152+0000","last_active":"2024-09-10T18:46:55.721776+0000","last_peered":"2024-09-10T18:46:55.721776+0000","last_clean":"2024-09-10T18:46:55.721776+0000","last_became_active":"2024-09-10T18:45:24.079534+0000","last_became_peered":"2024-09-10T18:45:24.079534+0000","last_unstale":"2024-09-10T18:46:55.721776+0000","last_undegraded":"2024-09-10T18:46:55.721776+0000","last_fullsized":"2024-09-10T18:46:55.721776+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:36.158087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:36.158087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:41.325875+0000","scrub_duration":0.0014477100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.724122+0000","last_change":"2024-09-10T18:46:50.666447+0000","last_active":"2024-09-10T18:46:55.724122+0000","last_peered":"2024-09-10T18:46:55.724122+0000","last_clean":"2024-09-10T18:46:55.724122+0000","last_became_active":"2024-09-10T18:45:17.963121+0000","last_became_peered":"2024-09-10T18:45:17.963121+0000","last_unstale":"2024-09-10T18:46:55.724122+0000","last_undegraded":"2024-09-10T18:46:55.724122+0000","last_fullsized":"2024-09-10T18:46:55.724122+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:50.666379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.666379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:06.539690+0000","scrub_duration":0.0016221689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"63'9","reported_seq":60,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720234+0000","last_change":"2024-09-10T18:46:55.069893+0000","last_active":"2024-09-10T18:46:55.720234+0000","last_peered":"2024-09-10T18:46:55.720234+0000","last_clean":"2024-09-10T18:46:55.720234+0000","last_became_active":"2024-09-10T18:45:20.052652+0000","last_became_peered":"2024-09-10T18:45:20.052652+0000","last_unstale":"2024-09-10T18:46:55.720234+0000","last_undegraded":"2024-09-10T18:46:55.720234+0000","last_fullsized":"2024-09-10T18:46:55.720234+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:46:55.069833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:55.069833+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:12.396664+0000","scrub_duration":0.0016685510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.722124+0000","last_change":"2024-09-10T18:46:31.024322+0000","last_active":"2024-09-10T18:46:55.722124+0000","last_peered":"2024-09-10T18:46:55.722124+0000","last_clean":"2024-09-10T18:46:55.722124+0000","last_became_active":"2024-09-10T18:45:22.068326+0000","last_became_peered":"2024-09-10T18:45:22.068326+0000","last_unstale":"2024-09-10T18:46:55.722124+0000","last_undegraded":"2024-09-10T18:46:55.722124+0000","last_fullsized":"2024-09-10T18:46:55.722124+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:31.024255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:31.024255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:50.407493+0000","scrub_duration":0.00152395,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.719285+0000","last_change":"2024-09-10T18:45:24.081968+0000","last_active":"2024-09-10T18:46:55.719285+0000","last_peered":"2024-09-10T18:46:55.719285+0000","last_clean":"2024-09-10T18:46:55.719285+0000","last_became_active":"2024-09-10T18:45:24.081488+0000","last_became_peered":"2024-09-10T18:45:24.081488+0000","last_unstale":"2024-09-10T18:46:55.719285+0000","last_undegraded":"2024-09-10T18:46:55.719285+0000","last_fullsized":"2024-09-10T18:46:55.719285+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"78'25","reported_seq":80,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720049+0000","last_change":"2024-09-10T18:46:44.316200+0000","last_active":"2024-09-10T18:46:55.720049+0000","last_peered":"2024-09-10T18:46:55.720049+0000","last_clean":"2024-09-10T18:46:55.720049+0000","last_became_active":"2024-09-10T18:45:20.057019+0000","last_became_peered":"2024-09-10T18:45:20.057019+0000","last_unstale":"2024-09-10T18:46:55.720049+0000","last_undegraded":"2024-09-10T18:46:55.720049+0000","last_fullsized":"2024-09-10T18:46:55.720049+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'25","last_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_deep_scrub":"78'25","last_deep_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_clean_scrub_stamp":"2024-09-10T18:46:44.316146+0000","objects_scrubbed":8,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:06.777730+0000","scrub_duration":0.0046415040000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723847+0000","last_change":"2024-09-10T18:46:47.595182+0000","last_active":"2024-09-10T18:46:55.723847+0000","last_peered":"2024-09-10T18:46:55.723847+0000","last_clean":"2024-09-10T18:46:55.723847+0000","last_became_active":"2024-09-10T18:45:17.963732+0000","last_became_peered":"2024-09-10T18:45:17.963732+0000","last_unstale":"2024-09-10T18:46:55.723847+0000","last_undegraded":"2024-09-10T18:46:55.723847+0000","last_fullsized":"2024-09-10T18:46:55.723847+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:47.595114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:47.595114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:53.271455+0000","scrub_duration":0.0014913249999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:04.512502+0000","last_change":"2024-09-10T18:47:04.512478+0000","last_active":"2024-09-10T18:47:04.512502+0000","last_peered":"2024-09-10T18:47:04.512502+0000","last_clean":"2024-09-10T18:47:04.512502+0000","last_became_active":"2024-09-10T18:45:22.071443+0000","last_became_peered":"2024-09-10T18:45:22.071443+0000","last_unstale":"2024-09-10T18:47:04.512502+0000","last_undegraded":"2024-09-10T18:47:04.512502+0000","last_fullsized":"2024-09-10T18:47:04.512502+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:04.512425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:04.512425+0000","last_clean_scrub_stamp":"2024-09-10T18:47:04.512425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:26.715434+0000","scrub_duration":0.001363324,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720517+0000","last_change":"2024-09-10T18:46:35.738217+0000","last_active":"2024-09-10T18:46:55.720517+0000","last_peered":"2024-09-10T18:46:55.720517+0000","last_clean":"2024-09-10T18:46:55.720517+0000","last_became_active":"2024-09-10T18:45:24.077897+0000","last_became_peered":"2024-09-10T18:45:24.077897+0000","last_unstale":"2024-09-10T18:46:55.720517+0000","last_undegraded":"2024-09-10T18:46:55.720517+0000","last_fullsized":"2024-09-10T18:46:55.720517+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:35.738143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:35.738143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:48.741747+0000","scrub_duration":0.0015293520000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"63'10","reported_seq":59,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720013+0000","last_change":"2024-09-10T18:46:31.281588+0000","last_active":"2024-09-10T18:46:55.720013+0000","last_peered":"2024-09-10T18:46:55.720013+0000","last_clean":"2024-09-10T18:46:55.720013+0000","last_became_active":"2024-09-10T18:45:20.053428+0000","last_became_peered":"2024-09-10T18:45:20.053428+0000","last_unstale":"2024-09-10T18:46:55.720013+0000","last_undegraded":"2024-09-10T18:46:55.720013+0000","last_fullsized":"2024-09-10T18:46:55.720013+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:46:31.281519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:31.281519+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:58.508120+0000","scrub_duration":0.0019095119999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.722685+0000","last_change":"2024-09-10T18:46:38.445417+0000","last_active":"2024-09-10T18:46:55.722685+0000","last_peered":"2024-09-10T18:46:55.722685+0000","last_clean":"2024-09-10T18:46:55.722685+0000","last_became_active":"2024-09-10T18:45:17.966833+0000","last_became_peered":"2024-09-10T18:45:17.966833+0000","last_unstale":"2024-09-10T18:46:55.722685+0000","last_undegraded":"2024-09-10T18:46:55.722685+0000","last_fullsized":"2024-09-10T18:46:55.722685+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:38.445362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:38.445362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:02.460484+0000","scrub_duration":0.0012292029999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"63'8","reported_seq":71,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.739581+0000","last_change":"2024-09-10T18:46:25.787936+0000","last_active":"2024-09-10T18:46:56.739581+0000","last_peered":"2024-09-10T18:46:56.739581+0000","last_clean":"2024-09-10T18:46:56.739581+0000","last_became_active":"2024-09-10T18:45:22.068453+0000","last_became_peered":"2024-09-10T18:45:22.068453+0000","last_unstale":"2024-09-10T18:46:56.739581+0000","last_undegraded":"2024-09-10T18:46:56.739581+0000","last_fullsized":"2024-09-10T18:46:56.739581+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:25.787877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:25.787877+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:54.201629+0000","scrub_duration":0.0018093530000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.719928+0000","last_change":"2024-09-10T18:46:32.265034+0000","last_active":"2024-09-10T18:46:55.719928+0000","last_peered":"2024-09-10T18:46:55.719928+0000","last_clean":"2024-09-10T18:46:55.719928+0000","last_became_active":"2024-09-10T18:45:24.081082+0000","last_became_peered":"2024-09-10T18:45:24.081082+0000","last_unstale":"2024-09-10T18:46:55.719928+0000","last_undegraded":"2024-09-10T18:46:55.719928+0000","last_fullsized":"2024-09-10T18:46:55.719928+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:32.264963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:32.264963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:50.207705+0000","scrub_duration":0.002378509,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"63'15","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:58.122323+0000","last_change":"2024-09-10T18:46:58.122279+0000","last_active":"2024-09-10T18:46:58.122323+0000","last_peered":"2024-09-10T18:46:58.122323+0000","last_clean":"2024-09-10T18:46:58.122323+0000","last_became_active":"2024-09-10T18:46:56.742946+0000","last_became_peered":"2024-09-10T18:46:56.742946+0000","last_unstale":"2024-09-10T18:46:58.122323+0000","last_undegraded":"2024-09-10T18:46:58.122323+0000","last_fullsized":"2024-09-10T18:46:58.122323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:46:58.122219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:58.122219+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:05.975000+0000","scrub_duration":0.001678094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.722329+0000","last_change":"2024-09-10T18:46:28.320761+0000","last_active":"2024-09-10T18:46:55.722329+0000","last_peered":"2024-09-10T18:46:55.722329+0000","last_clean":"2024-09-10T18:46:55.722329+0000","last_became_active":"2024-09-10T18:45:17.965409+0000","last_became_peered":"2024-09-10T18:45:17.965409+0000","last_unstale":"2024-09-10T18:46:55.722329+0000","last_undegraded":"2024-09-10T18:46:55.722329+0000","last_fullsized":"2024-09-10T18:46:55.722329+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_clean_scrub_stamp":"2024-09-10T18:46:28.320704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:44.092028+0000","scrub_duration":0.001393476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"63'8","reported_seq":71,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.740259+0000","last_change":"2024-09-10T18:46:39.300139+0000","last_active":"2024-09-10T18:46:56.740259+0000","last_peered":"2024-09-10T18:46:56.740259+0000","last_clean":"2024-09-10T18:46:56.740259+0000","last_became_active":"2024-09-10T18:45:22.062497+0000","last_became_peered":"2024-09-10T18:45:22.062497+0000","last_unstale":"2024-09-10T18:46:56.740259+0000","last_undegraded":"2024-09-10T18:46:56.740259+0000","last_fullsized":"2024-09-10T18:46:56.740259+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:39.300065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:39.300065+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:44.830898+0000","scrub_duration":0.001858934,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.722887+0000","last_change":"2024-09-10T18:46:42.057055+0000","last_active":"2024-09-10T18:46:55.722887+0000","last_peered":"2024-09-10T18:46:55.722887+0000","last_clean":"2024-09-10T18:46:55.722887+0000","last_became_active":"2024-09-10T18:45:24.082362+0000","last_became_peered":"2024-09-10T18:45:24.082362+0000","last_unstale":"2024-09-10T18:46:55.722887+0000","last_undegraded":"2024-09-10T18:46:55.722887+0000","last_fullsized":"2024-09-10T18:46:55.722887+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:42.056970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:42.056970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:50.036040+0000","scrub_duration":0.0016528479999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"63'11","reported_seq":85,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.747580+0000","last_change":"2024-09-10T18:46:56.747580+0000","last_active":"2024-09-10T18:46:56.747580+0000","last_peered":"2024-09-10T18:46:56.747580+0000","last_clean":"2024-09-10T18:46:56.747580+0000","last_became_active":"2024-09-10T18:46:56.741936+0000","last_became_peered":"2024-09-10T18:46:56.741936+0000","last_unstale":"2024-09-10T18:46:56.747580+0000","last_undegraded":"2024-09-10T18:46:56.747580+0000","last_fullsized":"2024-09-10T18:46:56.747580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:46:35.274660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:35.274660+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:42.198733+0000","scrub_duration":0.0019745819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723072+0000","last_change":"2024-09-10T18:46:39.520497+0000","last_active":"2024-09-10T18:46:55.723072+0000","last_peered":"2024-09-10T18:46:55.723072+0000","last_clean":"2024-09-10T18:46:55.723072+0000","last_became_active":"2024-09-10T18:45:17.966133+0000","last_became_peered":"2024-09-10T18:45:17.966133+0000","last_unstale":"2024-09-10T18:46:55.723072+0000","last_undegraded":"2024-09-10T18:46:55.723072+0000","last_fullsized":"2024-09-10T18:46:55.723072+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:39.520432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:39.520432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:49.040271+0000","scrub_duration":0.0013982510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723932+0000","last_change":"2024-09-10T18:46:48.116414+0000","last_active":"2024-09-10T18:46:55.723932+0000","last_peered":"2024-09-10T18:46:55.723932+0000","last_clean":"2024-09-10T18:46:55.723932+0000","last_became_active":"2024-09-10T18:45:22.068918+0000","last_became_peered":"2024-09-10T18:45:22.068918+0000","last_unstale":"2024-09-10T18:46:55.723932+0000","last_undegraded":"2024-09-10T18:46:55.723932+0000","last_fullsized":"2024-09-10T18:46:55.723932+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_clean_scrub_stamp":"2024-09-10T18:46:48.116358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:48.149931+0000","scrub_duration":0.001429331,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:00.743766+0000","last_change":"2024-09-10T18:47:00.743740+0000","last_active":"2024-09-10T18:47:00.743766+0000","last_peered":"2024-09-10T18:47:00.743766+0000","last_clean":"2024-09-10T18:47:00.743766+0000","last_became_active":"2024-09-10T18:45:24.264538+0000","last_became_peered":"2024-09-10T18:45:24.264538+0000","last_unstale":"2024-09-10T18:47:00.743766+0000","last_undegraded":"2024-09-10T18:47:00.743766+0000","last_fullsized":"2024-09-10T18:47:00.743766+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:00.743680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:00.743680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:02.404002+0000","scrub_duration":0.0022093759999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"63'14","reported_seq":65,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.719793+0000","last_change":"2024-09-10T18:46:29.330572+0000","last_active":"2024-09-10T18:46:55.719793+0000","last_peered":"2024-09-10T18:46:55.719793+0000","last_clean":"2024-09-10T18:46:55.719793+0000","last_became_active":"2024-09-10T18:45:20.052243+0000","last_became_peered":"2024-09-10T18:45:20.052243+0000","last_unstale":"2024-09-10T18:46:55.719793+0000","last_undegraded":"2024-09-10T18:46:55.719793+0000","last_fullsized":"2024-09-10T18:46:55.719793+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-10T18:46:29.330503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:29.330503+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:55.456983+0000","scrub_duration":0.0020437459999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.724143+0000","last_change":"2024-09-10T18:46:25.206801+0000","last_active":"2024-09-10T18:46:55.724143+0000","last_peered":"2024-09-10T18:46:55.724143+0000","last_clean":"2024-09-10T18:46:55.724143+0000","last_became_active":"2024-09-10T18:45:17.966027+0000","last_became_peered":"2024-09-10T18:45:17.966027+0000","last_unstale":"2024-09-10T18:46:55.724143+0000","last_undegraded":"2024-09-10T18:46:55.724143+0000","last_fullsized":"2024-09-10T18:46:55.724143+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_clean_scrub_stamp":"2024-09-10T18:46:25.206731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:46.038154+0000","scrub_duration":0.0016589910000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":71,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.741525+0000","last_change":"2024-09-10T18:46:40.808518+0000","last_active":"2024-09-10T18:46:56.741525+0000","last_peered":"2024-09-10T18:46:56.741525+0000","last_clean":"2024-09-10T18:46:56.741525+0000","last_became_active":"2024-09-10T18:45:22.074540+0000","last_became_peered":"2024-09-10T18:45:22.074540+0000","last_unstale":"2024-09-10T18:46:56.741525+0000","last_undegraded":"2024-09-10T18:46:56.741525+0000","last_fullsized":"2024-09-10T18:46:56.741525+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:40.808453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:40.808453+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:43.962254+0000","scrub_duration":0.0016757829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723560+0000","last_change":"2024-09-10T18:46:32.590511+0000","last_active":"2024-09-10T18:46:55.723560+0000","last_peered":"2024-09-10T18:46:55.723560+0000","last_clean":"2024-09-10T18:46:55.723560+0000","last_became_active":"2024-09-10T18:45:24.265303+0000","last_became_peered":"2024-09-10T18:45:24.265303+0000","last_unstale":"2024-09-10T18:46:55.723560+0000","last_undegraded":"2024-09-10T18:46:55.723560+0000","last_fullsized":"2024-09-10T18:46:55.723560+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:32.590438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:32.590438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:01.680676+0000","scrub_duration":0.0013292270000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"63'18","reported_seq":76,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723920+0000","last_change":"2024-09-10T18:46:27.801177+0000","last_active":"2024-09-10T18:46:55.723920+0000","last_peered":"2024-09-10T18:46:55.723920+0000","last_clean":"2024-09-10T18:46:55.723920+0000","last_became_active":"2024-09-10T18:45:20.420937+0000","last_became_peered":"2024-09-10T18:45:20.420937+0000","last_unstale":"2024-09-10T18:46:55.723920+0000","last_undegraded":"2024-09-10T18:46:55.723920+0000","last_fullsized":"2024-09-10T18:46:55.723920+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'18","last_scrub_stamp":"2024-09-10T18:46:27.801110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:27.801110+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:42.247024+0000","scrub_duration":0.001870095,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:05.505464+0000","last_change":"2024-09-10T18:47:05.505431+0000","last_active":"2024-09-10T18:47:05.505464+0000","last_peered":"2024-09-10T18:47:05.505464+0000","last_clean":"2024-09-10T18:47:05.505464+0000","last_became_active":"2024-09-10T18:45:17.961695+0000","last_became_peered":"2024-09-10T18:45:17.961695+0000","last_unstale":"2024-09-10T18:47:05.505464+0000","last_undegraded":"2024-09-10T18:47:05.505464+0000","last_fullsized":"2024-09-10T18:47:05.505464+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:05.505376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:47:05.505376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:32.771817+0000","scrub_duration":0.0013302310000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.719551+0000","last_change":"2024-09-10T18:46:41.312735+0000","last_active":"2024-09-10T18:46:55.719551+0000","last_peered":"2024-09-10T18:46:55.719551+0000","last_clean":"2024-09-10T18:46:55.719551+0000","last_became_active":"2024-09-10T18:45:22.074012+0000","last_became_peered":"2024-09-10T18:45:22.074012+0000","last_unstale":"2024-09-10T18:46:55.719551+0000","last_undegraded":"2024-09-10T18:46:55.719551+0000","last_fullsized":"2024-09-10T18:46:55.719551+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:41.312671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:41.312671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:47.779534+0000","scrub_duration":0.0013475010000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720156+0000","last_change":"2024-09-10T18:46:53.087609+0000","last_active":"2024-09-10T18:46:55.720156+0000","last_peered":"2024-09-10T18:46:55.720156+0000","last_clean":"2024-09-10T18:46:55.720156+0000","last_became_active":"2024-09-10T18:45:24.076831+0000","last_became_peered":"2024-09-10T18:45:24.076831+0000","last_unstale":"2024-09-10T18:46:55.720156+0000","last_undegraded":"2024-09-10T18:46:55.720156+0000","last_fullsized":"2024-09-10T18:46:55.720156+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_clean_scrub_stamp":"2024-09-10T18:46:53.087539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:56.714234+0000","scrub_duration":0.001678971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"63'19","reported_seq":80,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.719953+0000","last_change":"2024-09-10T18:46:42.524291+0000","last_active":"2024-09-10T18:46:55.719953+0000","last_peered":"2024-09-10T18:46:55.719953+0000","last_clean":"2024-09-10T18:46:55.719953+0000","last_became_active":"2024-09-10T18:45:20.421046+0000","last_became_peered":"2024-09-10T18:45:20.421046+0000","last_unstale":"2024-09-10T18:46:55.719953+0000","last_undegraded":"2024-09-10T18:46:55.719953+0000","last_fullsized":"2024-09-10T18:46:55.719953+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:46:42.524195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:42.524195+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:45.664487+0000","scrub_duration":0.0022840299999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:00.592392+0000","last_change":"2024-09-10T18:47:00.592359+0000","last_active":"2024-09-10T18:47:00.592392+0000","last_peered":"2024-09-10T18:47:00.592392+0000","last_clean":"2024-09-10T18:47:00.592392+0000","last_became_active":"2024-09-10T18:45:17.960226+0000","last_became_peered":"2024-09-10T18:45:17.960226+0000","last_unstale":"2024-09-10T18:47:00.592392+0000","last_undegraded":"2024-09-10T18:47:00.592392+0000","last_fullsized":"2024-09-10T18:47:00.592392+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:00.592279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:47:00.592279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:03.737142+0000","scrub_duration":0.001418576,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.721954+0000","last_change":"2024-09-10T18:46:52.235456+0000","last_active":"2024-09-10T18:46:55.721954+0000","last_peered":"2024-09-10T18:46:55.721954+0000","last_clean":"2024-09-10T18:46:55.721954+0000","last_became_active":"2024-09-10T18:45:22.066148+0000","last_became_peered":"2024-09-10T18:45:22.066148+0000","last_unstale":"2024-09-10T18:46:55.721954+0000","last_undegraded":"2024-09-10T18:46:55.721954+0000","last_fullsized":"2024-09-10T18:46:55.721954+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:52.235389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:52.235389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:06.785777+0000","scrub_duration":0.001679675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.722014+0000","last_change":"2024-09-10T18:45:24.265025+0000","last_active":"2024-09-10T18:46:55.722014+0000","last_peered":"2024-09-10T18:46:55.722014+0000","last_clean":"2024-09-10T18:46:55.722014+0000","last_became_active":"2024-09-10T18:45:24.264680+0000","last_became_peered":"2024-09-10T18:45:24.264680+0000","last_unstale":"2024-09-10T18:46:55.722014+0000","last_undegraded":"2024-09-10T18:46:55.722014+0000","last_fullsized":"2024-09-10T18:46:55.722014+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"83'15","reported_seq":64,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:58.436606+0000","last_change":"2024-09-10T18:46:58.436565+0000","last_active":"2024-09-10T18:46:58.436606+0000","last_peered":"2024-09-10T18:46:58.436606+0000","last_clean":"2024-09-10T18:46:58.436606+0000","last_became_active":"2024-09-10T18:45:20.057204+0000","last_became_peered":"2024-09-10T18:45:20.057204+0000","last_unstale":"2024-09-10T18:46:58.436606+0000","last_undegraded":"2024-09-10T18:46:58.436606+0000","last_fullsized":"2024-09-10T18:46:58.436606+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'15","last_scrub_stamp":"2024-09-10T18:46:58.436473+0000","last_deep_scrub":"83'15","last_deep_scrub_stamp":"2024-09-10T18:46:58.436473+0000","last_clean_scrub_stamp":"2024-09-10T18:46:58.436473+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:59.402142+0000","scrub_duration":0.0046048119999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723038+0000","last_change":"2024-09-10T18:46:50.443101+0000","last_active":"2024-09-10T18:46:55.723038+0000","last_peered":"2024-09-10T18:46:55.723038+0000","last_clean":"2024-09-10T18:46:55.723038+0000","last_became_active":"2024-09-10T18:45:17.966632+0000","last_became_peered":"2024-09-10T18:45:17.966632+0000","last_unstale":"2024-09-10T18:46:55.723038+0000","last_undegraded":"2024-09-10T18:46:55.723038+0000","last_fullsized":"2024-09-10T18:46:55.723038+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:50.443042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.443042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:18.989766+0000","scrub_duration":0.0014088550000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"63'8","reported_seq":71,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:03.579298+0000","last_change":"2024-09-10T18:47:03.579260+0000","last_active":"2024-09-10T18:47:03.579298+0000","last_peered":"2024-09-10T18:47:03.579298+0000","last_clean":"2024-09-10T18:47:03.579298+0000","last_became_active":"2024-09-10T18:45:22.063326+0000","last_became_peered":"2024-09-10T18:45:22.063326+0000","last_unstale":"2024-09-10T18:47:03.579298+0000","last_undegraded":"2024-09-10T18:47:03.579298+0000","last_fullsized":"2024-09-10T18:47:03.579298+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:47:03.579186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:47:03.579186+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:22.186155+0000","scrub_duration":0.0021415639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.719177+0000","last_change":"2024-09-10T18:45:24.077780+0000","last_active":"2024-09-10T18:46:55.719177+0000","last_peered":"2024-09-10T18:46:55.719177+0000","last_clean":"2024-09-10T18:46:55.719177+0000","last_became_active":"2024-09-10T18:45:24.077412+0000","last_became_peered":"2024-09-10T18:45:24.077412+0000","last_unstale":"2024-09-10T18:46:55.719177+0000","last_undegraded":"2024-09-10T18:46:55.719177+0000","last_fullsized":"2024-09-10T18:46:55.719177+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"63'28","reported_seq":113,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.741055+0000","last_change":"2024-09-10T18:46:19.223319+0000","last_active":"2024-09-10T18:46:56.741055+0000","last_peered":"2024-09-10T18:46:56.741055+0000","last_clean":"2024-09-10T18:46:56.741055+0000","last_became_active":"2024-09-10T18:45:20.061413+0000","last_became_peered":"2024-09-10T18:45:20.061413+0000","last_unstale":"2024-09-10T18:46:56.741055+0000","last_undegraded":"2024-09-10T18:46:56.741055+0000","last_fullsized":"2024-09-10T18:46:56.741055+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'28","last_scrub_stamp":"2024-09-10T18:46:19.223250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:19.223250+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:23.644429+0000","scrub_duration":0.0032067810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723685+0000","last_change":"2024-09-10T18:46:49.120758+0000","last_active":"2024-09-10T18:46:55.723685+0000","last_peered":"2024-09-10T18:46:55.723685+0000","last_clean":"2024-09-10T18:46:55.723685+0000","last_became_active":"2024-09-10T18:45:17.964494+0000","last_became_peered":"2024-09-10T18:45:17.964494+0000","last_unstale":"2024-09-10T18:46:55.723685+0000","last_undegraded":"2024-09-10T18:46:55.723685+0000","last_fullsized":"2024-09-10T18:46:55.723685+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:49.120704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:49.120704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:51.579079+0000","scrub_duration":0.0011863380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:58.523323+0000","last_change":"2024-09-10T18:46:58.523278+0000","last_active":"2024-09-10T18:46:58.523323+0000","last_peered":"2024-09-10T18:46:58.523323+0000","last_clean":"2024-09-10T18:46:58.523323+0000","last_became_active":"2024-09-10T18:45:22.061428+0000","last_became_peered":"2024-09-10T18:45:22.061428+0000","last_unstale":"2024-09-10T18:46:58.523323+0000","last_undegraded":"2024-09-10T18:46:58.523323+0000","last_fullsized":"2024-09-10T18:46:58.523323+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:58.523217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:58.523217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:04.494370+0000","scrub_duration":0.001307856,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.724755+0000","last_change":"2024-09-10T18:45:24.073623+0000","last_active":"2024-09-10T18:46:55.724755+0000","last_peered":"2024-09-10T18:46:55.724755+0000","last_clean":"2024-09-10T18:46:55.724755+0000","last_became_active":"2024-09-10T18:45:24.073307+0000","last_became_peered":"2024-09-10T18:45:24.073307+0000","last_unstale":"2024-09-10T18:46:55.724755+0000","last_undegraded":"2024-09-10T18:46:55.724755+0000","last_fullsized":"2024-09-10T18:46:55.724755+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"63'13","reported_seq":61,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723473+0000","last_change":"2024-09-10T18:45:20.058778+0000","last_active":"2024-09-10T18:46:55.723473+0000","last_peered":"2024-09-10T18:46:55.723473+0000","last_clean":"2024-09-10T18:46:55.723473+0000","last_became_active":"2024-09-10T18:45:20.058435+0000","last_became_peered":"2024-09-10T18:45:20.058435+0000","last_unstale":"2024-09-10T18:46:55.723473+0000","last_undegraded":"2024-09-10T18:46:55.723473+0000","last_fullsized":"2024-09-10T18:46:55.723473+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"65'5","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:04.400265+0000","last_change":"2024-09-10T18:45:22.059997+0000","last_active":"2024-09-10T18:47:04.400265+0000","last_peered":"2024-09-10T18:47:04.400265+0000","last_clean":"2024-09-10T18:47:04.400265+0000","last_became_active":"2024-09-10T18:45:17.969868+0000","last_became_peered":"2024-09-10T18:45:17.969868+0000","last_unstale":"2024-09-10T18:47:04.400265+0000","last_undegraded":"2024-09-10T18:47:04.400265+0000","last_fullsized":"2024-09-10T18:47:04.400265+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021933159999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":100,"num_read_kb":95,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.719852+0000","last_change":"2024-09-10T18:46:46.712335+0000","last_active":"2024-09-10T18:46:55.719852+0000","last_peered":"2024-09-10T18:46:55.719852+0000","last_clean":"2024-09-10T18:46:55.719852+0000","last_became_active":"2024-09-10T18:45:17.960859+0000","last_became_peered":"2024-09-10T18:45:17.960859+0000","last_unstale":"2024-09-10T18:46:55.719852+0000","last_undegraded":"2024-09-10T18:46:55.719852+0000","last_fullsized":"2024-09-10T18:46:55.719852+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:46.712269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:46.712269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:54.660825+0000","scrub_duration":0.001509383,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720164+0000","last_change":"2024-09-10T18:46:41.818890+0000","last_active":"2024-09-10T18:46:55.720164+0000","last_peered":"2024-09-10T18:46:55.720164+0000","last_clean":"2024-09-10T18:46:55.720164+0000","last_became_active":"2024-09-10T18:45:22.068350+0000","last_became_peered":"2024-09-10T18:45:22.068350+0000","last_unstale":"2024-09-10T18:46:55.720164+0000","last_undegraded":"2024-09-10T18:46:55.720164+0000","last_fullsized":"2024-09-10T18:46:55.720164+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:41.818827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:41.818827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:07.148553+0000","scrub_duration":0.0017408219999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:59.553615+0000","last_change":"2024-09-10T18:46:59.553576+0000","last_active":"2024-09-10T18:46:59.553615+0000","last_peered":"2024-09-10T18:46:59.553615+0000","last_clean":"2024-09-10T18:46:59.553615+0000","last_became_active":"2024-09-10T18:45:24.264617+0000","last_became_peered":"2024-09-10T18:45:24.264617+0000","last_unstale":"2024-09-10T18:46:59.553615+0000","last_undegraded":"2024-09-10T18:46:59.553615+0000","last_fullsized":"2024-09-10T18:46:59.553615+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:59.553505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:59.553505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:29.436053+0000","scrub_duration":0.0018297210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"63'12","reported_seq":62,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720387+0000","last_change":"2024-09-10T18:46:52.472574+0000","last_active":"2024-09-10T18:46:55.720387+0000","last_peered":"2024-09-10T18:46:55.720387+0000","last_clean":"2024-09-10T18:46:55.720387+0000","last_became_active":"2024-09-10T18:45:20.061091+0000","last_became_peered":"2024-09-10T18:45:20.061091+0000","last_unstale":"2024-09-10T18:46:55.720387+0000","last_undegraded":"2024-09-10T18:46:55.720387+0000","last_fullsized":"2024-09-10T18:46:55.720387+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:46:52.472511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:52.472511+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:19.552466+0000","scrub_duration":0.001886935,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"58'1","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.719619+0000","last_change":"2024-09-10T18:45:22.055928+0000","last_active":"2024-09-10T18:46:55.719619+0000","last_peered":"2024-09-10T18:46:55.719619+0000","last_clean":"2024-09-10T18:46:55.719619+0000","last_became_active":"2024-09-10T18:45:17.963932+0000","last_became_peered":"2024-09-10T18:45:17.963932+0000","last_unstale":"2024-09-10T18:46:55.719619+0000","last_undegraded":"2024-09-10T18:46:55.719619+0000","last_fullsized":"2024-09-10T18:46:55.719619+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037812099999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.742192+0000","last_change":"2024-09-10T18:46:56.742192+0000","last_active":"2024-09-10T18:46:56.742192+0000","last_peered":"2024-09-10T18:46:56.742192+0000","last_clean":"2024-09-10T18:46:56.742192+0000","last_became_active":"2024-09-10T18:46:56.741862+0000","last_became_peered":"2024-09-10T18:46:56.741862+0000","last_unstale":"2024-09-10T18:46:56.742192+0000","last_undegraded":"2024-09-10T18:46:56.742192+0000","last_fullsized":"2024-09-10T18:46:56.742192+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:54.554426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:54.554426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:17.934705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:59.132334+0000","last_change":"2024-09-10T18:46:59.132301+0000","last_active":"2024-09-10T18:46:59.132334+0000","last_peered":"2024-09-10T18:46:59.132334+0000","last_clean":"2024-09-10T18:46:59.132334+0000","last_became_active":"2024-09-10T18:45:22.064285+0000","last_became_peered":"2024-09-10T18:45:22.064285+0000","last_unstale":"2024-09-10T18:46:59.132334+0000","last_undegraded":"2024-09-10T18:46:59.132334+0000","last_fullsized":"2024-09-10T18:46:59.132334+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:59.132234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:59.132234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:03.150923+0000","scrub_duration":0.001471034,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.722567+0000","last_change":"2024-09-10T18:46:49.422912+0000","last_active":"2024-09-10T18:46:55.722567+0000","last_peered":"2024-09-10T18:46:55.722567+0000","last_clean":"2024-09-10T18:46:55.722567+0000","last_became_active":"2024-09-10T18:45:24.083689+0000","last_became_peered":"2024-09-10T18:45:24.083689+0000","last_unstale":"2024-09-10T18:46:55.722567+0000","last_undegraded":"2024-09-10T18:46:55.722567+0000","last_fullsized":"2024-09-10T18:46:55.722567+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:49.422843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:49.422843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:00.864466+0000","scrub_duration":0.0014985300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"63'14","reported_seq":66,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723666+0000","last_change":"2024-09-10T18:46:22.294141+0000","last_active":"2024-09-10T18:46:55.723666+0000","last_peered":"2024-09-10T18:46:55.723666+0000","last_clean":"2024-09-10T18:46:55.723666+0000","last_became_active":"2024-09-10T18:45:20.421119+0000","last_became_peered":"2024-09-10T18:45:20.421119+0000","last_unstale":"2024-09-10T18:46:55.723666+0000","last_undegraded":"2024-09-10T18:46:55.723666+0000","last_fullsized":"2024-09-10T18:46:55.723666+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-10T18:46:22.294072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:22.294072+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:49.887899+0000","scrub_duration":0.0021435069999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"60'2","reported_seq":53,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.724331+0000","last_change":"2024-09-10T18:46:25.845131+0000","last_active":"2024-09-10T18:46:55.724331+0000","last_peered":"2024-09-10T18:46:55.724331+0000","last_clean":"2024-09-10T18:46:55.724331+0000","last_became_active":"2024-09-10T18:45:17.961660+0000","last_became_peered":"2024-09-10T18:45:17.961660+0000","last_unstale":"2024-09-10T18:46:55.724331+0000","last_undegraded":"2024-09-10T18:46:55.724331+0000","last_fullsized":"2024-09-10T18:46:55.724331+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-10T18:46:25.845070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:25.845070+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:50.370600+0000","scrub_duration":0.0020129710000000001,"objects_trimmed":0,"snaptrim_duration":0.0013942640000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.722360+0000","last_change":"2024-09-10T18:46:29.081362+0000","last_active":"2024-09-10T18:46:55.722360+0000","last_peered":"2024-09-10T18:46:55.722360+0000","last_clean":"2024-09-10T18:46:55.722360+0000","last_became_active":"2024-09-10T18:45:17.965163+0000","last_became_peered":"2024-09-10T18:45:17.965163+0000","last_unstale":"2024-09-10T18:46:55.722360+0000","last_undegraded":"2024-09-10T18:46:55.722360+0000","last_fullsized":"2024-09-10T18:46:55.722360+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:29.081296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:29.081296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:50.659900+0000","scrub_duration":0.0015488590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'81","reported_seq":90,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:05.558076+0000","last_change":"2024-09-10T18:47:05.558031+0000","last_active":"2024-09-10T18:47:05.558076+0000","last_peered":"2024-09-10T18:47:05.558076+0000","last_clean":"2024-09-10T18:47:05.558076+0000","last_became_active":"2024-09-10T18:45:07.433592+0000","last_became_peered":"2024-09-10T18:45:07.433592+0000","last_unstale":"2024-09-10T18:47:05.558076+0000","last_undegraded":"2024-09-10T18:47:05.558076+0000","last_fullsized":"2024-09-10T18:47:05.558076+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:47:05.557968+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:47:05.557968+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:22.825675+0000","scrub_duration":0.0015062890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723308+0000","last_change":"2024-09-10T18:46:26.687029+0000","last_active":"2024-09-10T18:46:55.723308+0000","last_peered":"2024-09-10T18:46:55.723308+0000","last_clean":"2024-09-10T18:46:55.723308+0000","last_became_active":"2024-09-10T18:45:22.068226+0000","last_became_peered":"2024-09-10T18:45:22.068226+0000","last_unstale":"2024-09-10T18:46:55.723308+0000","last_undegraded":"2024-09-10T18:46:55.723308+0000","last_fullsized":"2024-09-10T18:46:55.723308+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:26.686961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:26.686961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:32.456741+0000","scrub_duration":0.0015985109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.066593+0000","last_change":"2024-09-10T18:46:56.066558+0000","last_active":"2024-09-10T18:46:56.066593+0000","last_peered":"2024-09-10T18:46:56.066593+0000","last_clean":"2024-09-10T18:46:56.066593+0000","last_became_active":"2024-09-10T18:45:24.073327+0000","last_became_peered":"2024-09-10T18:45:24.073327+0000","last_unstale":"2024-09-10T18:46:56.066593+0000","last_undegraded":"2024-09-10T18:46:56.066593+0000","last_fullsized":"2024-09-10T18:46:56.066593+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:56.066502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:56.066502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:06.081016+0000","scrub_duration":0.0011960810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"63'19","reported_seq":75,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.722652+0000","last_change":"2024-09-10T18:46:28.229804+0000","last_active":"2024-09-10T18:46:55.722652+0000","last_peered":"2024-09-10T18:46:55.722652+0000","last_clean":"2024-09-10T18:46:55.722652+0000","last_became_active":"2024-09-10T18:45:20.421097+0000","last_became_peered":"2024-09-10T18:45:20.421097+0000","last_unstale":"2024-09-10T18:46:55.722652+0000","last_undegraded":"2024-09-10T18:46:55.722652+0000","last_fullsized":"2024-09-10T18:46:55.722652+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:46:28.229737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:28.229737+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:49.859218+0000","scrub_duration":0.002308184,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720850+0000","last_change":"2024-09-10T18:46:53.707765+0000","last_active":"2024-09-10T18:46:55.720850+0000","last_peered":"2024-09-10T18:46:55.720850+0000","last_clean":"2024-09-10T18:46:55.720850+0000","last_became_active":"2024-09-10T18:45:17.959924+0000","last_became_peered":"2024-09-10T18:45:17.959924+0000","last_unstale":"2024-09-10T18:46:55.720850+0000","last_undegraded":"2024-09-10T18:46:55.720850+0000","last_fullsized":"2024-09-10T18:46:55.720850+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:53.707694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:53.707694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:22.214797+0000","scrub_duration":0.0020804069999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720321+0000","last_change":"2024-09-10T18:46:30.781786+0000","last_active":"2024-09-10T18:46:55.720321+0000","last_peered":"2024-09-10T18:46:55.720321+0000","last_clean":"2024-09-10T18:46:55.720321+0000","last_became_active":"2024-09-10T18:45:22.072581+0000","last_became_peered":"2024-09-10T18:45:22.072581+0000","last_unstale":"2024-09-10T18:46:55.720321+0000","last_undegraded":"2024-09-10T18:46:55.720321+0000","last_fullsized":"2024-09-10T18:46:55.720321+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:30.781730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:30.781730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:38.248319+0000","scrub_duration":0.0020255870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:03.587359+0000","last_change":"2024-09-10T18:47:03.587334+0000","last_active":"2024-09-10T18:47:03.587359+0000","last_peered":"2024-09-10T18:47:03.587359+0000","last_clean":"2024-09-10T18:47:03.587359+0000","last_became_active":"2024-09-10T18:45:24.076392+0000","last_became_peered":"2024-09-10T18:45:24.076392+0000","last_unstale":"2024-09-10T18:47:03.587359+0000","last_undegraded":"2024-09-10T18:47:03.587359+0000","last_fullsized":"2024-09-10T18:47:03.587359+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:03.587252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:03.587252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:29.976291+0000","scrub_duration":0.001684867,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"63'9","reported_seq":60,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.719877+0000","last_change":"2024-09-10T18:46:51.486491+0000","last_active":"2024-09-10T18:46:55.719877+0000","last_peered":"2024-09-10T18:46:55.719877+0000","last_clean":"2024-09-10T18:46:55.719877+0000","last_became_active":"2024-09-10T18:45:20.061021+0000","last_became_peered":"2024-09-10T18:45:20.061021+0000","last_unstale":"2024-09-10T18:46:55.719877+0000","last_undegraded":"2024-09-10T18:46:55.719877+0000","last_fullsized":"2024-09-10T18:46:55.719877+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:46:51.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:51.486428+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:05.602014+0000","scrub_duration":0.0019382830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:02.496334+0000","last_change":"2024-09-10T18:47:02.496271+0000","last_active":"2024-09-10T18:47:02.496334+0000","last_peered":"2024-09-10T18:47:02.496334+0000","last_clean":"2024-09-10T18:47:02.496334+0000","last_became_active":"2024-09-10T18:45:17.963057+0000","last_became_peered":"2024-09-10T18:45:17.963057+0000","last_unstale":"2024-09-10T18:47:02.496334+0000","last_undegraded":"2024-09-10T18:47:02.496334+0000","last_fullsized":"2024-09-10T18:47:02.496334+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:02.496142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:47:02.496142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:10.459026+0000","scrub_duration":0.001628399,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720471+0000","last_change":"2024-09-10T18:46:55.684302+0000","last_active":"2024-09-10T18:46:55.720471+0000","last_peered":"2024-09-10T18:46:55.720471+0000","last_clean":"2024-09-10T18:46:55.720471+0000","last_became_active":"2024-09-10T18:45:22.070343+0000","last_became_peered":"2024-09-10T18:45:22.070343+0000","last_unstale":"2024-09-10T18:46:55.720471+0000","last_undegraded":"2024-09-10T18:46:55.720471+0000","last_fullsized":"2024-09-10T18:46:55.720471+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_clean_scrub_stamp":"2024-09-10T18:46:55.684250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:59.597477+0000","scrub_duration":0.0014319199999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720233+0000","last_change":"2024-09-10T18:46:47.545406+0000","last_active":"2024-09-10T18:46:55.720233+0000","last_peered":"2024-09-10T18:46:55.720233+0000","last_clean":"2024-09-10T18:46:55.720233+0000","last_became_active":"2024-09-10T18:45:24.078529+0000","last_became_peered":"2024-09-10T18:45:24.078529+0000","last_unstale":"2024-09-10T18:46:55.720233+0000","last_undegraded":"2024-09-10T18:46:55.720233+0000","last_fullsized":"2024-09-10T18:46:55.720233+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:47.545343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:47.545343+0000","last_clean_scrub_stamp":"2024-09-10T18:46:47.545343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:02.171974+0000","scrub_duration":0.0012977799999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"63'15","reported_seq":69,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.721541+0000","last_change":"2024-09-10T18:46:51.114543+0000","last_active":"2024-09-10T18:46:55.721541+0000","last_peered":"2024-09-10T18:46:55.721541+0000","last_clean":"2024-09-10T18:46:55.721541+0000","last_became_active":"2024-09-10T18:45:20.419804+0000","last_became_peered":"2024-09-10T18:45:20.419804+0000","last_unstale":"2024-09-10T18:46:55.721541+0000","last_undegraded":"2024-09-10T18:46:55.721541+0000","last_fullsized":"2024-09-10T18:46:55.721541+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:46:51.114455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:51.114455+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:18.982344+0000","scrub_duration":0.0022557979999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720362+0000","last_change":"2024-09-10T18:46:17.819966+0000","last_active":"2024-09-10T18:46:55.720362+0000","last_peered":"2024-09-10T18:46:55.720362+0000","last_clean":"2024-09-10T18:46:55.720362+0000","last_became_active":"2024-09-10T18:45:17.959653+0000","last_became_peered":"2024-09-10T18:45:17.959653+0000","last_unstale":"2024-09-10T18:46:55.720362+0000","last_undegraded":"2024-09-10T18:46:55.720362+0000","last_fullsized":"2024-09-10T18:46:55.720362+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:17.819909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:17.819909+0000","last_clean_scrub_stamp":"2024-09-10T18:46:17.819909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:18.454145+0000","scrub_duration":0.001612109,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"83'9","reported_seq":75,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:02.582313+0000","last_change":"2024-09-10T18:47:02.582283+0000","last_active":"2024-09-10T18:47:02.582313+0000","last_peered":"2024-09-10T18:47:02.582313+0000","last_clean":"2024-09-10T18:47:02.582313+0000","last_became_active":"2024-09-10T18:45:22.073383+0000","last_became_peered":"2024-09-10T18:45:22.073383+0000","last_unstale":"2024-09-10T18:47:02.582313+0000","last_undegraded":"2024-09-10T18:47:02.582313+0000","last_fullsized":"2024-09-10T18:47:02.582313+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'9","last_scrub_stamp":"2024-09-10T18:47:02.582229+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-09-10T18:47:02.582229+0000","last_clean_scrub_stamp":"2024-09-10T18:47:02.582229+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:10.931107+0000","scrub_duration":0.0020623740000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723356+0000","last_change":"2024-09-10T18:46:45.083522+0000","last_active":"2024-09-10T18:46:55.723356+0000","last_peered":"2024-09-10T18:46:55.723356+0000","last_clean":"2024-09-10T18:46:55.723356+0000","last_became_active":"2024-09-10T18:45:24.265256+0000","last_became_peered":"2024-09-10T18:45:24.265256+0000","last_unstale":"2024-09-10T18:46:55.723356+0000","last_undegraded":"2024-09-10T18:46:55.723356+0000","last_fullsized":"2024-09-10T18:46:55.723356+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_clean_scrub_stamp":"2024-09-10T18:46:45.083425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:08.177993+0000","scrub_duration":0.0017589890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"63'12","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.719562+0000","last_change":"2024-09-10T18:46:30.320527+0000","last_active":"2024-09-10T18:46:55.719562+0000","last_peered":"2024-09-10T18:46:55.719562+0000","last_clean":"2024-09-10T18:46:55.719562+0000","last_became_active":"2024-09-10T18:45:20.056869+0000","last_became_peered":"2024-09-10T18:45:20.056869+0000","last_unstale":"2024-09-10T18:46:55.719562+0000","last_undegraded":"2024-09-10T18:46:55.719562+0000","last_fullsized":"2024-09-10T18:46:55.719562+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:46:30.320462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:30.320462+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:35.180189+0000","scrub_duration":0.0028822850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"56'2","reported_seq":63,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720187+0000","last_change":"2024-09-10T18:46:42.303648+0000","last_active":"2024-09-10T18:46:55.720187+0000","last_peered":"2024-09-10T18:46:55.720187+0000","last_clean":"2024-09-10T18:46:55.720187+0000","last_became_active":"2024-09-10T18:45:17.963286+0000","last_became_peered":"2024-09-10T18:45:17.963286+0000","last_unstale":"2024-09-10T18:46:55.720187+0000","last_undegraded":"2024-09-10T18:46:55.720187+0000","last_fullsized":"2024-09-10T18:46:55.720187+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-10T18:46:42.303567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:42.303567+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:05.463057+0000","scrub_duration":0.0020684750000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.719539+0000","last_change":"2024-09-10T18:46:44.754345+0000","last_active":"2024-09-10T18:46:55.719539+0000","last_peered":"2024-09-10T18:46:55.719539+0000","last_clean":"2024-09-10T18:46:55.719539+0000","last_became_active":"2024-09-10T18:45:22.068957+0000","last_became_peered":"2024-09-10T18:45:22.068957+0000","last_unstale":"2024-09-10T18:46:55.719539+0000","last_undegraded":"2024-09-10T18:46:55.719539+0000","last_fullsized":"2024-09-10T18:46:55.719539+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:44.754286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:44.754286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:47.893148+0000","scrub_duration":0.001258386,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.725035+0000","last_change":"2024-09-10T18:45:24.078134+0000","last_active":"2024-09-10T18:46:55.725035+0000","last_peered":"2024-09-10T18:46:55.725035+0000","last_clean":"2024-09-10T18:46:55.725035+0000","last_became_active":"2024-09-10T18:45:24.077815+0000","last_became_peered":"2024-09-10T18:45:24.077815+0000","last_unstale":"2024-09-10T18:46:55.725035+0000","last_undegraded":"2024-09-10T18:46:55.725035+0000","last_fullsized":"2024-09-10T18:46:55.725035+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"76'13","reported_seq":64,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720318+0000","last_change":"2024-09-10T18:46:24.626912+0000","last_active":"2024-09-10T18:46:55.720318+0000","last_peered":"2024-09-10T18:46:55.720318+0000","last_clean":"2024-09-10T18:46:55.720318+0000","last_became_active":"2024-09-10T18:45:20.420421+0000","last_became_peered":"2024-09-10T18:45:20.420421+0000","last_unstale":"2024-09-10T18:46:55.720318+0000","last_undegraded":"2024-09-10T18:46:55.720318+0000","last_fullsized":"2024-09-10T18:46:55.720318+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'13","last_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_deep_scrub":"76'13","last_deep_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_clean_scrub_stamp":"2024-09-10T18:46:24.626859+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:41.087073+0000","scrub_duration":0.0035642949999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.719648+0000","last_change":"2024-09-10T18:46:31.734522+0000","last_active":"2024-09-10T18:46:55.719648+0000","last_peered":"2024-09-10T18:46:55.719648+0000","last_clean":"2024-09-10T18:46:55.719648+0000","last_became_active":"2024-09-10T18:45:17.965112+0000","last_became_peered":"2024-09-10T18:45:17.965112+0000","last_unstale":"2024-09-10T18:46:55.719648+0000","last_undegraded":"2024-09-10T18:46:55.719648+0000","last_fullsized":"2024-09-10T18:46:55.719648+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:31.734459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:31.734459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:41.550267+0000","scrub_duration":0.0015571960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.724880+0000","last_change":"2024-09-10T18:46:32.829583+0000","last_active":"2024-09-10T18:46:55.724880+0000","last_peered":"2024-09-10T18:46:55.724880+0000","last_clean":"2024-09-10T18:46:55.724880+0000","last_became_active":"2024-09-10T18:45:22.069040+0000","last_became_peered":"2024-09-10T18:45:22.069040+0000","last_unstale":"2024-09-10T18:46:55.724880+0000","last_undegraded":"2024-09-10T18:46:55.724880+0000","last_fullsized":"2024-09-10T18:46:55.724880+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:32.829508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:32.829508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:45.869787+0000","scrub_duration":0.001897239,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"63'1","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.721828+0000","last_change":"2024-09-10T18:46:56.721773+0000","last_active":"2024-09-10T18:46:56.721828+0000","last_peered":"2024-09-10T18:46:56.721828+0000","last_clean":"2024-09-10T18:46:56.721828+0000","last_became_active":"2024-09-10T18:45:24.079258+0000","last_became_peered":"2024-09-10T18:45:24.079258+0000","last_unstale":"2024-09-10T18:46:56.721828+0000","last_undegraded":"2024-09-10T18:46:56.721828+0000","last_fullsized":"2024-09-10T18:46:56.721828+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-10T18:46:56.721716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:56.721716+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:04.127384+0000","scrub_duration":0.0014420559999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.722675+0000","last_change":"2024-09-10T18:46:48.227450+0000","last_active":"2024-09-10T18:46:55.722675+0000","last_peered":"2024-09-10T18:46:55.722675+0000","last_clean":"2024-09-10T18:46:55.722675+0000","last_became_active":"2024-09-10T18:45:17.964705+0000","last_became_peered":"2024-09-10T18:45:17.964705+0000","last_unstale":"2024-09-10T18:46:55.722675+0000","last_undegraded":"2024-09-10T18:46:55.722675+0000","last_fullsized":"2024-09-10T18:46:55.722675+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:48.227378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:48.227378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:54.333207+0000","scrub_duration":0.0015447519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'6","reported_seq":53,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:02.700895+0000","last_change":"2024-09-10T18:47:02.700853+0000","last_active":"2024-09-10T18:47:02.700895+0000","last_peered":"2024-09-10T18:47:02.700895+0000","last_clean":"2024-09-10T18:47:02.700895+0000","last_became_active":"2024-09-10T18:45:20.421746+0000","last_became_peered":"2024-09-10T18:45:20.421746+0000","last_unstale":"2024-09-10T18:47:02.700895+0000","last_undegraded":"2024-09-10T18:47:02.700895+0000","last_fullsized":"2024-09-10T18:47:02.700895+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'6","last_scrub_stamp":"2024-09-10T18:47:02.700787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:02.700787+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:10.576660+0000","scrub_duration":0.0017032519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.722238+0000","last_change":"2024-09-10T18:46:39.084489+0000","last_active":"2024-09-10T18:46:55.722238+0000","last_peered":"2024-09-10T18:46:55.722238+0000","last_clean":"2024-09-10T18:46:55.722238+0000","last_became_active":"2024-09-10T18:45:22.069653+0000","last_became_peered":"2024-09-10T18:45:22.069653+0000","last_unstale":"2024-09-10T18:46:55.722238+0000","last_undegraded":"2024-09-10T18:46:55.722238+0000","last_fullsized":"2024-09-10T18:46:55.722238+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:39.084433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:39.084433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:05.431757+0000","scrub_duration":0.0014845819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723941+0000","last_change":"2024-09-10T18:46:52.611074+0000","last_active":"2024-09-10T18:46:55.723941+0000","last_peered":"2024-09-10T18:46:55.723941+0000","last_clean":"2024-09-10T18:46:55.723941+0000","last_became_active":"2024-09-10T18:45:24.264611+0000","last_became_peered":"2024-09-10T18:45:24.264611+0000","last_unstale":"2024-09-10T18:46:55.723941+0000","last_undegraded":"2024-09-10T18:46:55.723941+0000","last_fullsized":"2024-09-10T18:46:55.723941+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:52.611003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:52.611003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:04.517442+0000","scrub_duration":0.0015558849999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"63'10","reported_seq":81,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.759636+0000","last_change":"2024-09-10T18:46:56.759636+0000","last_active":"2024-09-10T18:46:56.759636+0000","last_peered":"2024-09-10T18:46:56.759636+0000","last_clean":"2024-09-10T18:46:56.759636+0000","last_became_active":"2024-09-10T18:46:56.754800+0000","last_became_peered":"2024-09-10T18:46:56.754800+0000","last_unstale":"2024-09-10T18:46:56.759636+0000","last_undegraded":"2024-09-10T18:46:56.759636+0000","last_fullsized":"2024-09-10T18:46:56.759636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:46:50.722316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.722316+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:55.786523+0000","scrub_duration":0.0022194070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723620+0000","last_change":"2024-09-10T18:46:50.093435+0000","last_active":"2024-09-10T18:46:55.723620+0000","last_peered":"2024-09-10T18:46:55.723620+0000","last_clean":"2024-09-10T18:46:55.723620+0000","last_became_active":"2024-09-10T18:45:17.964870+0000","last_became_peered":"2024-09-10T18:45:17.964870+0000","last_unstale":"2024-09-10T18:46:55.723620+0000","last_undegraded":"2024-09-10T18:46:55.723620+0000","last_fullsized":"2024-09-10T18:46:55.723620+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:50.093372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.093372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:55.991532+0000","scrub_duration":0.0015580539999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"63'8","reported_seq":71,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.741144+0000","last_change":"2024-09-10T18:46:34.085224+0000","last_active":"2024-09-10T18:46:56.741144+0000","last_peered":"2024-09-10T18:46:56.741144+0000","last_clean":"2024-09-10T18:46:56.741144+0000","last_became_active":"2024-09-10T18:45:22.062940+0000","last_became_peered":"2024-09-10T18:45:22.062940+0000","last_unstale":"2024-09-10T18:46:56.741144+0000","last_undegraded":"2024-09-10T18:46:56.741144+0000","last_fullsized":"2024-09-10T18:46:56.741144+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:34.085156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:34.085156+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:54.777061+0000","scrub_duration":0.0020843120000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.719914+0000","last_change":"2024-09-10T18:45:24.074039+0000","last_active":"2024-09-10T18:46:55.719914+0000","last_peered":"2024-09-10T18:46:55.719914+0000","last_clean":"2024-09-10T18:46:55.719914+0000","last_became_active":"2024-09-10T18:45:24.073736+0000","last_became_peered":"2024-09-10T18:45:24.073736+0000","last_unstale":"2024-09-10T18:46:55.719914+0000","last_undegraded":"2024-09-10T18:46:55.719914+0000","last_fullsized":"2024-09-10T18:46:55.719914+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"63'9","reported_seq":60,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723082+0000","last_change":"2024-09-10T18:46:23.086274+0000","last_active":"2024-09-10T18:46:55.723082+0000","last_peered":"2024-09-10T18:46:55.723082+0000","last_clean":"2024-09-10T18:46:55.723082+0000","last_became_active":"2024-09-10T18:45:20.421124+0000","last_became_peered":"2024-09-10T18:45:20.421124+0000","last_unstale":"2024-09-10T18:46:55.723082+0000","last_undegraded":"2024-09-10T18:46:55.723082+0000","last_fullsized":"2024-09-10T18:46:55.723082+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:46:23.086203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:23.086203+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:44.793257+0000","scrub_duration":0.002168068,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720102+0000","last_change":"2024-09-10T18:46:45.541882+0000","last_active":"2024-09-10T18:46:55.720102+0000","last_peered":"2024-09-10T18:46:55.720102+0000","last_clean":"2024-09-10T18:46:55.720102+0000","last_became_active":"2024-09-10T18:45:17.960106+0000","last_became_peered":"2024-09-10T18:45:17.960106+0000","last_unstale":"2024-09-10T18:46:55.720102+0000","last_undegraded":"2024-09-10T18:46:55.720102+0000","last_fullsized":"2024-09-10T18:46:55.720102+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:45.541819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:45.541819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:15.015402+0000","scrub_duration":0.0015180809999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"81'9","reported_seq":72,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.741897+0000","last_change":"2024-09-10T18:46:54.800878+0000","last_active":"2024-09-10T18:46:56.741897+0000","last_peered":"2024-09-10T18:46:56.741897+0000","last_clean":"2024-09-10T18:46:56.741897+0000","last_became_active":"2024-09-10T18:45:22.075368+0000","last_became_peered":"2024-09-10T18:45:22.075368+0000","last_unstale":"2024-09-10T18:46:56.741897+0000","last_undegraded":"2024-09-10T18:46:56.741897+0000","last_fullsized":"2024-09-10T18:46:56.741897+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"81'9","last_scrub_stamp":"2024-09-10T18:46:54.800790+0000","last_deep_scrub":"81'9","last_deep_scrub_stamp":"2024-09-10T18:46:54.800790+0000","last_clean_scrub_stamp":"2024-09-10T18:46:54.800790+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:14.444052+0000","scrub_duration":0.0034208709999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.719967+0000","last_change":"2024-09-10T18:46:36.305262+0000","last_active":"2024-09-10T18:46:55.719967+0000","last_peered":"2024-09-10T18:46:55.719967+0000","last_clean":"2024-09-10T18:46:55.719967+0000","last_became_active":"2024-09-10T18:45:24.080196+0000","last_became_peered":"2024-09-10T18:45:24.080196+0000","last_unstale":"2024-09-10T18:46:55.719967+0000","last_undegraded":"2024-09-10T18:46:55.719967+0000","last_fullsized":"2024-09-10T18:46:55.719967+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_clean_scrub_stamp":"2024-09-10T18:46:36.305199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:04.747507+0000","scrub_duration":0.0013794809999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"63'9","reported_seq":60,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:59.469284+0000","last_change":"2024-09-10T18:46:59.469257+0000","last_active":"2024-09-10T18:46:59.469284+0000","last_peered":"2024-09-10T18:46:59.469284+0000","last_clean":"2024-09-10T18:46:59.469284+0000","last_became_active":"2024-09-10T18:45:20.057479+0000","last_became_peered":"2024-09-10T18:45:20.057479+0000","last_unstale":"2024-09-10T18:46:59.469284+0000","last_undegraded":"2024-09-10T18:46:59.469284+0000","last_fullsized":"2024-09-10T18:46:59.469284+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:46:59.469198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:59.469198+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:26.476613+0000","scrub_duration":0.0022009489999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723247+0000","last_change":"2024-09-10T18:46:23.292276+0000","last_active":"2024-09-10T18:46:55.723247+0000","last_peered":"2024-09-10T18:46:55.723247+0000","last_clean":"2024-09-10T18:46:55.723247+0000","last_became_active":"2024-09-10T18:45:17.962009+0000","last_became_peered":"2024-09-10T18:45:17.962009+0000","last_unstale":"2024-09-10T18:46:55.723247+0000","last_undegraded":"2024-09-10T18:46:55.723247+0000","last_fullsized":"2024-09-10T18:46:55.723247+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:23.292207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:23.292207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:27.416585+0000","scrub_duration":0.001659953,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.724596+0000","last_change":"2024-09-10T18:45:22.073186+0000","last_active":"2024-09-10T18:46:55.724596+0000","last_peered":"2024-09-10T18:46:55.724596+0000","last_clean":"2024-09-10T18:46:55.724596+0000","last_became_active":"2024-09-10T18:45:22.072877+0000","last_became_peered":"2024-09-10T18:45:22.072877+0000","last_unstale":"2024-09-10T18:46:55.724596+0000","last_undegraded":"2024-09-10T18:46:55.724596+0000","last_fullsized":"2024-09-10T18:46:55.724596+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.719175+0000","last_change":"2024-09-10T18:46:50.499866+0000","last_active":"2024-09-10T18:46:55.719175+0000","last_peered":"2024-09-10T18:46:55.719175+0000","last_clean":"2024-09-10T18:46:55.719175+0000","last_became_active":"2024-09-10T18:45:24.081050+0000","last_became_peered":"2024-09-10T18:45:24.081050+0000","last_unstale":"2024-09-10T18:46:55.719175+0000","last_undegraded":"2024-09-10T18:46:55.719175+0000","last_fullsized":"2024-09-10T18:46:55.719175+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:50.499815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.499815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:13.479025+0000","scrub_duration":0.001285432,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"63'11","reported_seq":53,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.719761+0000","last_change":"2024-09-10T18:45:20.057563+0000","last_active":"2024-09-10T18:46:55.719761+0000","last_peered":"2024-09-10T18:46:55.719761+0000","last_clean":"2024-09-10T18:46:55.719761+0000","last_became_active":"2024-09-10T18:45:20.056885+0000","last_became_peered":"2024-09-10T18:45:20.056885+0000","last_unstale":"2024-09-10T18:46:55.719761+0000","last_undegraded":"2024-09-10T18:46:55.719761+0000","last_fullsized":"2024-09-10T18:46:55.719761+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723267+0000","last_change":"2024-09-10T18:46:36.402610+0000","last_active":"2024-09-10T18:46:55.723267+0000","last_peered":"2024-09-10T18:46:55.723267+0000","last_clean":"2024-09-10T18:46:55.723267+0000","last_became_active":"2024-09-10T18:45:17.963542+0000","last_became_peered":"2024-09-10T18:45:17.963542+0000","last_unstale":"2024-09-10T18:46:55.723267+0000","last_undegraded":"2024-09-10T18:46:55.723267+0000","last_fullsized":"2024-09-10T18:46:55.723267+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:36.402548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:36.402548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:04.495178+0000","scrub_duration":0.0014103919999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.722331+0000","last_change":"2024-09-10T18:46:37.414794+0000","last_active":"2024-09-10T18:46:55.722331+0000","last_peered":"2024-09-10T18:46:55.722331+0000","last_clean":"2024-09-10T18:46:55.722331+0000","last_became_active":"2024-09-10T18:45:22.070065+0000","last_became_peered":"2024-09-10T18:45:22.070065+0000","last_unstale":"2024-09-10T18:46:55.722331+0000","last_undegraded":"2024-09-10T18:46:55.722331+0000","last_fullsized":"2024-09-10T18:46:55.722331+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:37.414734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:37.414734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:44.027335+0000","scrub_duration":0.001607996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:05.789919+0000","last_change":"2024-09-10T18:47:05.789893+0000","last_active":"2024-09-10T18:47:05.789919+0000","last_peered":"2024-09-10T18:47:05.789919+0000","last_clean":"2024-09-10T18:47:05.789919+0000","last_became_active":"2024-09-10T18:45:24.072224+0000","last_became_peered":"2024-09-10T18:45:24.072224+0000","last_unstale":"2024-09-10T18:47:05.789919+0000","last_undegraded":"2024-09-10T18:47:05.789919+0000","last_fullsized":"2024-09-10T18:47:05.789919+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:05.789830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:05.789830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:28.845106+0000","scrub_duration":0.001271938,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"63'4","reported_seq":50,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723860+0000","last_change":"2024-09-10T18:46:46.766906+0000","last_active":"2024-09-10T18:46:55.723860+0000","last_peered":"2024-09-10T18:46:55.723860+0000","last_clean":"2024-09-10T18:46:55.723860+0000","last_became_active":"2024-09-10T18:45:20.059205+0000","last_became_peered":"2024-09-10T18:45:20.059205+0000","last_unstale":"2024-09-10T18:46:55.723860+0000","last_undegraded":"2024-09-10T18:46:55.723860+0000","last_fullsized":"2024-09-10T18:46:55.723860+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4","last_scrub_stamp":"2024-09-10T18:46:46.766766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:46.766766+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:49.051695+0000","scrub_duration":0.0021847250000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.722362+0000","last_change":"2024-09-10T18:46:54.082828+0000","last_active":"2024-09-10T18:46:55.722362+0000","last_peered":"2024-09-10T18:46:55.722362+0000","last_clean":"2024-09-10T18:46:55.722362+0000","last_became_active":"2024-09-10T18:45:17.963630+0000","last_became_peered":"2024-09-10T18:45:17.963630+0000","last_unstale":"2024-09-10T18:46:55.722362+0000","last_undegraded":"2024-09-10T18:46:55.722362+0000","last_fullsized":"2024-09-10T18:46:55.722362+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:54.082761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:54.082761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:05.974208+0000","scrub_duration":0.001210458,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.722008+0000","last_change":"2024-09-10T18:46:35.172936+0000","last_active":"2024-09-10T18:46:55.722008+0000","last_peered":"2024-09-10T18:46:55.722008+0000","last_clean":"2024-09-10T18:46:55.722008+0000","last_became_active":"2024-09-10T18:45:22.072377+0000","last_became_peered":"2024-09-10T18:45:22.072377+0000","last_unstale":"2024-09-10T18:46:55.722008+0000","last_undegraded":"2024-09-10T18:46:55.722008+0000","last_fullsized":"2024-09-10T18:46:55.722008+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_clean_scrub_stamp":"2024-09-10T18:46:35.172861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:57.638389+0000","scrub_duration":0.001552933,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.724306+0000","last_change":"2024-09-10T18:46:53.596450+0000","last_active":"2024-09-10T18:46:55.724306+0000","last_peered":"2024-09-10T18:46:55.724306+0000","last_clean":"2024-09-10T18:46:55.724306+0000","last_became_active":"2024-09-10T18:45:24.077398+0000","last_became_peered":"2024-09-10T18:45:24.077398+0000","last_unstale":"2024-09-10T18:46:55.724306+0000","last_undegraded":"2024-09-10T18:46:55.724306+0000","last_fullsized":"2024-09-10T18:46:55.724306+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:53.596380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:53.596380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:54.667444+0000","scrub_duration":0.0020285519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"79'16","reported_seq":68,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.724329+0000","last_change":"2024-09-10T18:46:48.744340+0000","last_active":"2024-09-10T18:46:55.724329+0000","last_peered":"2024-09-10T18:46:55.724329+0000","last_clean":"2024-09-10T18:46:55.724329+0000","last_became_active":"2024-09-10T18:45:20.421479+0000","last_became_peered":"2024-09-10T18:45:20.421479+0000","last_unstale":"2024-09-10T18:46:55.724329+0000","last_undegraded":"2024-09-10T18:46:55.724329+0000","last_fullsized":"2024-09-10T18:46:55.724329+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"79'16","last_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_deep_scrub":"79'16","last_deep_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_clean_scrub_stamp":"2024-09-10T18:46:48.744287+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:57.831206+0000","scrub_duration":0.0044879259999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"56'1","reported_seq":53,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.724103+0000","last_change":"2024-09-10T18:46:41.147533+0000","last_active":"2024-09-10T18:46:55.724103+0000","last_peered":"2024-09-10T18:46:55.724103+0000","last_clean":"2024-09-10T18:46:55.724103+0000","last_became_active":"2024-09-10T18:45:17.961222+0000","last_became_peered":"2024-09-10T18:45:17.961222+0000","last_unstale":"2024-09-10T18:46:55.724103+0000","last_undegraded":"2024-09-10T18:46:55.724103+0000","last_fullsized":"2024-09-10T18:46:55.724103+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:46:41.147463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:41.147463+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:58.016926+0000","scrub_duration":0.0016797889999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.722733+0000","last_change":"2024-09-10T18:46:27.667088+0000","last_active":"2024-09-10T18:46:55.722733+0000","last_peered":"2024-09-10T18:46:55.722733+0000","last_clean":"2024-09-10T18:46:55.722733+0000","last_became_active":"2024-09-10T18:45:22.072312+0000","last_became_peered":"2024-09-10T18:45:22.072312+0000","last_unstale":"2024-09-10T18:46:55.722733+0000","last_undegraded":"2024-09-10T18:46:55.722733+0000","last_fullsized":"2024-09-10T18:46:55.722733+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:27.667018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:27.667018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:38.065664+0000","scrub_duration":0.001388886,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.724517+0000","last_change":"2024-09-10T18:46:29.791617+0000","last_active":"2024-09-10T18:46:55.724517+0000","last_peered":"2024-09-10T18:46:55.724517+0000","last_clean":"2024-09-10T18:46:55.724517+0000","last_became_active":"2024-09-10T18:45:24.264818+0000","last_became_peered":"2024-09-10T18:45:24.264818+0000","last_unstale":"2024-09-10T18:46:55.724517+0000","last_undegraded":"2024-09-10T18:46:55.724517+0000","last_fullsized":"2024-09-10T18:46:55.724517+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:29.791563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:29.791563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:37.941643+0000","scrub_duration":0.0021031819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"78'2","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723235+0000","last_change":"2024-09-10T18:46:37.581201+0000","last_active":"2024-09-10T18:46:55.723235+0000","last_peered":"2024-09-10T18:46:55.723235+0000","last_clean":"2024-09-10T18:46:55.723235+0000","last_became_active":"2024-09-10T18:45:24.076380+0000","last_became_peered":"2024-09-10T18:45:24.076380+0000","last_unstale":"2024-09-10T18:46:55.723235+0000","last_undegraded":"2024-09-10T18:46:55.723235+0000","last_fullsized":"2024-09-10T18:46:55.723235+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'2","last_scrub_stamp":"2024-09-10T18:46:37.581130+0000","last_deep_scrub":"78'2","last_deep_scrub_stamp":"2024-09-10T18:46:37.581130+0000","last_clean_scrub_stamp":"2024-09-10T18:46:37.581130+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:55.887929+0000","scrub_duration":0.0024563609999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.721532+0000","last_change":"2024-09-10T18:46:28.088657+0000","last_active":"2024-09-10T18:46:55.721532+0000","last_peered":"2024-09-10T18:46:55.721532+0000","last_clean":"2024-09-10T18:46:55.721532+0000","last_became_active":"2024-09-10T18:45:17.962108+0000","last_became_peered":"2024-09-10T18:45:17.962108+0000","last_unstale":"2024-09-10T18:46:55.721532+0000","last_undegraded":"2024-09-10T18:46:55.721532+0000","last_fullsized":"2024-09-10T18:46:55.721532+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:28.088588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:28.088588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:56.550863+0000","scrub_duration":0.00140508,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"63'10","reported_seq":59,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720243+0000","last_change":"2024-09-10T18:46:49.494628+0000","last_active":"2024-09-10T18:46:55.720243+0000","last_peered":"2024-09-10T18:46:55.720243+0000","last_clean":"2024-09-10T18:46:55.720243+0000","last_became_active":"2024-09-10T18:45:20.421094+0000","last_became_peered":"2024-09-10T18:45:20.421094+0000","last_unstale":"2024-09-10T18:46:55.720243+0000","last_undegraded":"2024-09-10T18:46:55.720243+0000","last_fullsized":"2024-09-10T18:46:55.720243+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:46:49.494565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:49.494565+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:08.379972+0000","scrub_duration":0.0036379630000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":74,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.737868+0000","last_change":"2024-09-10T18:46:33.150919+0000","last_active":"2024-09-10T18:46:56.737868+0000","last_peered":"2024-09-10T18:46:56.737868+0000","last_clean":"2024-09-10T18:46:56.737868+0000","last_became_active":"2024-09-10T18:45:22.073281+0000","last_became_peered":"2024-09-10T18:45:22.073281+0000","last_unstale":"2024-09-10T18:46:56.737868+0000","last_undegraded":"2024-09-10T18:46:56.737868+0000","last_fullsized":"2024-09-10T18:46:56.737868+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:33.150852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:33.150852+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:51.299257+0000","scrub_duration":0.0017025339999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:03.475400+0000","last_change":"2024-09-10T18:47:03.475367+0000","last_active":"2024-09-10T18:47:03.475400+0000","last_peered":"2024-09-10T18:47:03.475400+0000","last_clean":"2024-09-10T18:47:03.475400+0000","last_became_active":"2024-09-10T18:45:24.082318+0000","last_became_peered":"2024-09-10T18:45:24.082318+0000","last_unstale":"2024-09-10T18:47:03.475400+0000","last_undegraded":"2024-09-10T18:47:03.475400+0000","last_fullsized":"2024-09-10T18:47:03.475400+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:03.475299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:03.475299+0000","last_clean_scrub_stamp":"2024-09-10T18:47:03.475299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:22.439695+0000","scrub_duration":0.001661945,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.724496+0000","last_change":"2024-09-10T18:46:44.694805+0000","last_active":"2024-09-10T18:46:55.724496+0000","last_peered":"2024-09-10T18:46:55.724496+0000","last_clean":"2024-09-10T18:46:55.724496+0000","last_became_active":"2024-09-10T18:45:17.965707+0000","last_became_peered":"2024-09-10T18:45:17.965707+0000","last_unstale":"2024-09-10T18:46:55.724496+0000","last_undegraded":"2024-09-10T18:46:55.724496+0000","last_fullsized":"2024-09-10T18:46:55.724496+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:44.694737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:44.694737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:03.417079+0000","scrub_duration":0.0016330400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"63'11","reported_seq":63,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.724078+0000","last_change":"2024-09-10T18:46:46.230915+0000","last_active":"2024-09-10T18:46:55.724078+0000","last_peered":"2024-09-10T18:46:55.724078+0000","last_clean":"2024-09-10T18:46:55.724078+0000","last_became_active":"2024-09-10T18:45:20.058717+0000","last_became_peered":"2024-09-10T18:45:20.058717+0000","last_unstale":"2024-09-10T18:46:55.724078+0000","last_undegraded":"2024-09-10T18:46:55.724078+0000","last_fullsized":"2024-09-10T18:46:55.724078+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:46:46.230819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:46.230819+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:54.186112+0000","scrub_duration":0.002545322,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720058+0000","last_change":"2024-09-10T18:45:22.068991+0000","last_active":"2024-09-10T18:46:55.720058+0000","last_peered":"2024-09-10T18:46:55.720058+0000","last_clean":"2024-09-10T18:46:55.720058+0000","last_became_active":"2024-09-10T18:45:22.068676+0000","last_became_peered":"2024-09-10T18:45:22.068676+0000","last_unstale":"2024-09-10T18:46:55.720058+0000","last_undegraded":"2024-09-10T18:46:55.720058+0000","last_fullsized":"2024-09-10T18:46:55.720058+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720003+0000","last_change":"2024-09-10T18:46:45.285074+0000","last_active":"2024-09-10T18:46:55.720003+0000","last_peered":"2024-09-10T18:46:55.720003+0000","last_clean":"2024-09-10T18:46:55.720003+0000","last_became_active":"2024-09-10T18:45:24.267251+0000","last_became_peered":"2024-09-10T18:45:24.267251+0000","last_unstale":"2024-09-10T18:46:55.720003+0000","last_undegraded":"2024-09-10T18:46:55.720003+0000","last_fullsized":"2024-09-10T18:46:55.720003+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:45.285015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:45.285015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:56.843562+0000","scrub_duration":0.001362142,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"56'1","reported_seq":55,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.722778+0000","last_change":"2024-09-10T18:46:42.139314+0000","last_active":"2024-09-10T18:46:55.722778+0000","last_peered":"2024-09-10T18:46:55.722778+0000","last_clean":"2024-09-10T18:46:55.722778+0000","last_became_active":"2024-09-10T18:45:17.966286+0000","last_became_peered":"2024-09-10T18:45:17.966286+0000","last_unstale":"2024-09-10T18:46:55.722778+0000","last_undegraded":"2024-09-10T18:46:55.722778+0000","last_fullsized":"2024-09-10T18:46:55.722778+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:46:42.139256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:42.139256+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:57.842540+0000","scrub_duration":0.0014138690000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"63'15","reported_seq":69,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.720672+0000","last_change":"2024-09-10T18:46:23.864133+0000","last_active":"2024-09-10T18:46:55.720672+0000","last_peered":"2024-09-10T18:46:55.720672+0000","last_clean":"2024-09-10T18:46:55.720672+0000","last_became_active":"2024-09-10T18:45:20.062728+0000","last_became_peered":"2024-09-10T18:45:20.062728+0000","last_unstale":"2024-09-10T18:46:55.720672+0000","last_undegraded":"2024-09-10T18:46:55.720672+0000","last_fullsized":"2024-09-10T18:46:55.720672+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:46:23.864075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:23.864075+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:29.106766+0000","scrub_duration":0.0020859780000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.723231+0000","last_change":"2024-09-10T18:45:22.060202+0000","last_active":"2024-09-10T18:46:55.723231+0000","last_peered":"2024-09-10T18:46:55.723231+0000","last_clean":"2024-09-10T18:46:55.723231+0000","last_became_active":"2024-09-10T18:45:22.059794+0000","last_became_peered":"2024-09-10T18:45:22.059794+0000","last_unstale":"2024-09-10T18:46:55.723231+0000","last_undegraded":"2024-09-10T18:46:55.723231+0000","last_fullsized":"2024-09-10T18:46:55.723231+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:57.811180+0000","last_change":"2024-09-10T18:46:57.811148+0000","last_active":"2024-09-10T18:46:57.811180+0000","last_peered":"2024-09-10T18:46:57.811180+0000","last_clean":"2024-09-10T18:46:57.811180+0000","last_became_active":"2024-09-10T18:45:24.264427+0000","last_became_peered":"2024-09-10T18:45:24.264427+0000","last_unstale":"2024-09-10T18:46:57.811180+0000","last_undegraded":"2024-09-10T18:46:57.811180+0000","last_fullsized":"2024-09-10T18:46:57.811180+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:57.811074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:57.811074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:12.950473+0000","scrub_duration":0.0013689419999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":46,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.725157+0000","last_change":"2024-09-10T18:46:47.734860+0000","last_active":"2024-09-10T18:46:55.725157+0000","last_peered":"2024-09-10T18:46:55.725157+0000","last_clean":"2024-09-10T18:46:55.725157+0000","last_became_active":"2024-09-10T18:45:17.963257+0000","last_became_peered":"2024-09-10T18:45:17.963257+0000","last_unstale":"2024-09-10T18:46:55.725157+0000","last_undegraded":"2024-09-10T18:46:55.725157+0000","last_fullsized":"2024-09-10T18:46:55.725157+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:47.734803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:47.734803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:05.847815+0000","scrub_duration":0.001381134,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"63'12","reported_seq":67,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.724558+0000","last_change":"2024-09-10T18:46:52.771640+0000","last_active":"2024-09-10T18:46:55.724558+0000","last_peered":"2024-09-10T18:46:55.724558+0000","last_clean":"2024-09-10T18:46:55.724558+0000","last_became_active":"2024-09-10T18:45:20.058391+0000","last_became_peered":"2024-09-10T18:45:20.058391+0000","last_unstale":"2024-09-10T18:46:55.724558+0000","last_undegraded":"2024-09-10T18:46:55.724558+0000","last_fullsized":"2024-09-10T18:46:55.724558+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:46:52.771570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:52.771570+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:04.251299+0000","scrub_duration":0.0021490089999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-10T18:46:55.719940+0000","last_change":"2024-09-10T18:46:34.283979+0000","last_active":"2024-09-10T18:46:55.719940+0000","last_peered":"2024-09-10T18:46:55.719940+0000","last_clean":"2024-09-10T18:46:55.719940+0000","last_became_active":"2024-09-10T18:45:22.073250+0000","last_became_peered":"2024-09-10T18:45:22.073250+0000","last_unstale":"2024-09-10T18:46:55.719940+0000","last_undegraded":"2024-09-10T18:46:55.719940+0000","last_fullsized":"2024-09-10T18:46:55.719940+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:34.283901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:34.283901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:34.686745+0000","scrub_duration":0.002360334,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":417,"ondisk_log_size":417,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":95,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299418,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298568,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446584,"statfs":{"total":95995035648,"available":95689302016,"internally_reserved":0,"allocated":1245184,"data_stored":888558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.435}]},{"osd":1,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.497},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.46400000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.40999999999999998}]},{"osd":3,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.44600000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.36499999999999999}]},{"osd":5,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.32900000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.42099999999999999}]}]},{"osd":6,"up_from":43,"seq":184683593759,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298556,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446596,"statfs":{"total":95995035648,"available":95689314304,"internally_reserved":0,"allocated":1232896,"data_stored":888634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.46300000000000002,"15min":0.45900000000000002},"min":{"1min":0.32900000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.496,"15min":0.49199999999999999},"min":{"1min":0.32800000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.45100000000000001}]},{"osd":1,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.315,"5min":0.249,"15min":0.249},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.46700000000000003,"15min":0.46300000000000002},"min":{"1min":0.33700000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.65900000000000003,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.33900000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.48299999999999998,"15min":0.48099999999999998},"min":{"1min":0.34000000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.49299999999999999,"15min":0.49099999999999999},"min":{"1min":0.33600000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68700000000000006,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.40799999999999997}]},{"osd":3,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.48399999999999999,"15min":0.48299999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.51200000000000001,"15min":0.51000000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.42699999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.37},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.39700000000000002,"15min":0.38800000000000001},"min":{"1min":0.26900000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.27700000000000002}]},{"osd":5,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.41899999999999998,"15min":0.41499999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.42499999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.32000000000000001}]},{"osd":7,"last update":"Tue Sep 10 18:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.442}]}]},{"osd":5,"up_from":37,"seq":158913789987,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297968,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447184,"statfs":{"total":95995035648,"available":95689916416,"internally_reserved":0,"allocated":630784,"data_stored":296879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.52200000000000002,"15min":0.52600000000000002},"min":{"1min":0.30299999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70099999999999996,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.58299999999999996,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.40000000000000002}]},{"osd":1,"last update":"Tue Sep 10 18:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.55200000000000005,"15min":0.56000000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.69699999999999995,"5min":0.87,"15min":0.87},"last":0.439},{"interface":"front","average":{"1min":0.497,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.64200000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.42399999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.51800000000000002,"15min":0.52000000000000002},"min":{"1min":0.33300000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.71299999999999997,"5min":0.755,"15min":0.755},"last":0.375},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53600000000000003,"15min":0.54100000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79600000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.35099999999999998}]},{"osd":3,"last update":"Tue Sep 10 18:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52500000000000002,"15min":0.52700000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76100000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.54200000000000004,"15min":0.54600000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80100000000000005,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.41099999999999998}]},{"osd":4,"last update":"Tue Sep 10 18:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.64500000000000002,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.40500000000000003,"15min":0.40300000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.65700000000000003,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.27800000000000002}]},{"osd":6,"last update":"Tue Sep 10 18:46:44 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.42199999999999999,"15min":0.41599999999999998},"min":{"1min":0.29299999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.67500000000000004,"5min":0.753,"15min":0.753},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.46500000000000002,"15min":0.46100000000000002},"min":{"1min":0.36199999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.71199999999999997,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.33900000000000002}]},{"osd":7,"last update":"Tue Sep 10 18:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.312},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.46600000000000003}]}]},{"osd":4,"up_from":30,"seq":128849018921,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297988,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447164,"statfs":{"total":95995035648,"available":95689895936,"internally_reserved":0,"allocated":651264,"data_stored":297726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:58 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.52200000000000002,"15min":0.52400000000000002},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.76800000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.52200000000000002,"15min":0.53600000000000003},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.64000000000000001,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.36399999999999999}]},{"osd":1,"last update":"Tue Sep 10 18:46:58 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.68400000000000005,"5min":2.609,"15min":2.609},"last":0.37},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.54900000000000004,"15min":0.53300000000000003},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.80500000000000005,"5min":2.589,"15min":2.589},"last":0.35899999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:46:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.55400000000000005,"15min":0.54300000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.70399999999999996,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54300000000000004,"15min":0.55300000000000005},"min":{"1min":0.33200000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.73299999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.378}]},{"osd":3,"last update":"Tue Sep 10 18:46:58 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.52500000000000002,"15min":0.52900000000000003},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.67500000000000004,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.52900000000000003,"15min":0.53900000000000003},"min":{"1min":0.35299999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.751,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.34799999999999998}]},{"osd":5,"last update":"Tue Sep 10 18:46:22 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.35899999999999999,"15min":0.34999999999999998},"min":{"1min":0.26800000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.44,"5min":0.371,"15min":0.35899999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.23799999999999999}]},{"osd":6,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.42999999999999999,"15min":0.42399999999999999},"min":{"1min":0.311,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.68799999999999994,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.40100000000000002,"15min":0.39100000000000001},"min":{"1min":0.34799999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.29899999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:46:08 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.318},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.26200000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182446,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298000,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447152,"statfs":{"total":95995035648,"available":95689883648,"internally_reserved":0,"allocated":663552,"data_stored":297505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:35 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.378,"15min":0.38400000000000001},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.40100000000000002,"15min":0.40300000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.29199999999999998}]},{"osd":1,"last update":"Tue Sep 10 18:46:35 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.40400000000000003,"15min":0.40899999999999997},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40899999999999997,"15min":0.41499999999999998},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.63800000000000001,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.33900000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:46:35 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.43099999999999999,"15min":0.438},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.375},{"interface":"front","average":{"1min":0.377,"5min":0.41999999999999998,"15min":0.433},"min":{"1min":0.27200000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.53600000000000003,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.35899999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:46:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54300000000000004,"15min":0.54400000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56200000000000006,"15min":0.57099999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.82299999999999995,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.44700000000000001}]},{"osd":5,"last update":"Tue Sep 10 18:46:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39700000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.70299999999999996,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.41399999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80800000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.52100000000000002}]},{"osd":6,"last update":"Tue Sep 10 18:46:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.56000000000000005,"15min":0.56299999999999994},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79100000000000004,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55400000000000005,"15min":0.55600000000000005},"min":{"1min":0.40799999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.49199999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:46:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.56100000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411378,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297976,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447176,"statfs":{"total":95995035648,"available":95689908224,"internally_reserved":0,"allocated":634880,"data_stored":297314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.376,"15min":0.35899999999999999},"min":{"1min":0.246,"5min":0.223,"15min":0.223},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.38300000000000001,"15min":0.371},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.38100000000000001}]},{"osd":1,"last update":"Tue Sep 10 18:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.375,"15min":0.36699999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.309},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.42499999999999999,"15min":0.41299999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.32500000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:46:36 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46800000000000003,"15min":0.46500000000000002},"min":{"1min":0.30099999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.371},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.42899999999999999,"15min":0.43099999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.34699999999999998}]},{"osd":4,"last update":"Tue Sep 10 18:46:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.69899999999999995,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56899999999999995,"15min":0.57099999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.64800000000000002,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.441}]},{"osd":5,"last update":"Tue Sep 10 18:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56100000000000005,"15min":0.55800000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.42899999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.39700000000000002}]},{"osd":6,"last update":"Tue Sep 10 18:46:49 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58999999999999997,"15min":0.58599999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.46200000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.48699999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:46:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.46400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298004,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447148,"statfs":{"total":95995035648,"available":95689879552,"internally_reserved":0,"allocated":667648,"data_stored":297780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:48 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34399999999999997,"15min":0.32000000000000001},"min":{"1min":0.245,"5min":0.224,"15min":0.224},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.36599999999999999,"15min":0.33600000000000002},"min":{"1min":0.252,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.53700000000000003,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.32600000000000001}]},{"osd":2,"last update":"Tue Sep 10 18:46:11 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.372,"15min":0.36099999999999999},"min":{"1min":0.27900000000000003,"5min":0.248,"15min":0.248},"max":{"1min":0.47199999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.39000000000000001,"15min":0.38100000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.49099999999999999,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.374}]},{"osd":3,"last update":"Tue Sep 10 18:46:34 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.45400000000000001,"15min":0.45800000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.42299999999999999,"15min":0.42199999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.55400000000000005,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.36699999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:47:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.57099999999999995,"15min":0.55900000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.81399999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.439},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70399999999999996,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.40400000000000003}]},{"osd":5,"last update":"Tue Sep 10 18:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60299999999999998,"15min":0.60399999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63700000000000001,"15min":0.64100000000000001},"min":{"1min":0.44600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93300000000000005,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.41599999999999998}]},{"osd":6,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.46300000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.66400000000000003,"15min":0.67100000000000004},"min":{"1min":0.41999999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82299999999999995,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.46000000000000002}]},{"osd":7,"last update":"Tue Sep 10 18:46:11 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.38900000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738428,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298560,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446592,"statfs":{"total":95995035648,"available":95689310208,"internally_reserved":0,"allocated":1236992,"data_stored":887368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 18:46:52 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.32500000000000001,"15min":0.30299999999999999},"min":{"1min":0.219,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.58799999999999997,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33700000000000002,"15min":0.315},"min":{"1min":0.221,"5min":0.187,"15min":0.187},"max":{"1min":0.59399999999999997,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.30299999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.40300000000000002,"15min":0.38900000000000001},"min":{"1min":0.30299999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.373},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.376,"15min":0.36199999999999999},"min":{"1min":0.26600000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.39100000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:46:31 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.39500000000000002,"15min":0.39800000000000002},"min":{"1min":0.26100000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.66100000000000003,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.41199999999999998,"15min":0.41299999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.33400000000000002}]},{"osd":4,"last update":"Tue Sep 10 18:47:00 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54400000000000004,"15min":0.54100000000000004},"min":{"1min":0.41899999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.79200000000000004,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.56899999999999995,"15min":0.53500000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.69899999999999995,"5min":4.6189999999999998,"15min":4.6189999999999998},"last":0.53800000000000003}]},{"osd":5,"last update":"Tue Sep 10 18:46:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.42999999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.58699999999999997,"15min":0.55800000000000005},"min":{"1min":0.42699999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":4.6429999999999998,"5min":4.6429999999999998,"15min":4.6429999999999998},"last":0.435}]},{"osd":6,"last update":"Tue Sep 10 18:46:42 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56100000000000005,"15min":0.55500000000000005},"min":{"1min":0.42099999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.746,"15min":0.77300000000000002},"min":{"1min":0.40899999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.878,"5min":4.6630000000000003,"15min":4.6630000000000003},"last":0.48699999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:46:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.505}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T18:47:09.410 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:09 smithi018 ceph-mon[40196]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:47:09.410 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:09 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:09.410 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:09 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:09.410 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:09 smithi018 ceph-mon[29336]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:47:09.410 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:09 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:09.410 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:09 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:09.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:09 smithi103 ceph-mon[33163]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:47:09.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:09 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:09.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:09 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:10.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:10 smithi018 ceph-mon[29336]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 102 B/s wr, 5 op/s; 21 B/s, 1 objects/s recovering 2024-09-10T18:47:10.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:10 smithi018 ceph-mon[29336]: 5.1d scrub starts 2024-09-10T18:47:10.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:10 smithi018 ceph-mon[29336]: 5.1d scrub ok 2024-09-10T18:47:10.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:10 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:10.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:10 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:10.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:10 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:47:10.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:10 smithi018 ceph-mon[40196]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 102 B/s wr, 5 op/s; 21 B/s, 1 objects/s recovering 2024-09-10T18:47:10.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:10 smithi018 ceph-mon[40196]: 5.1d scrub starts 2024-09-10T18:47:10.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:10 smithi018 ceph-mon[40196]: 5.1d scrub ok 2024-09-10T18:47:10.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:10 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:10.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:10 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:10.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:10 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:47:10.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:47:10 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:47:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:47:10.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:10 smithi103 ceph-mon[33163]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 102 B/s wr, 5 op/s; 21 B/s, 1 objects/s recovering 2024-09-10T18:47:10.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:10 smithi103 ceph-mon[33163]: 5.1d scrub starts 2024-09-10T18:47:10.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:10 smithi103 ceph-mon[33163]: 5.1d scrub ok 2024-09-10T18:47:10.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:10 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:10.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:10 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:10.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:10 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:47:11.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:11 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:11.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:11 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:47:11.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:11 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:47:11.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:11 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:11.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:11 smithi018 ceph-mon[29336]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-10T18:47:11.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:11 smithi018 ceph-mon[29336]: Reconfiguring daemon alertmanager.a on smithi018 2024-09-10T18:47:11.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:11 smithi018 ceph-mon[29336]: 4.7 deep-scrub starts 2024-09-10T18:47:11.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:11 smithi018 ceph-mon[29336]: 4.7 deep-scrub ok 2024-09-10T18:47:11.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:11 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:11.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:11 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:47:11.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:11 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:47:11.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:11 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:11.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:11 smithi018 ceph-mon[40196]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-10T18:47:11.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:11 smithi018 ceph-mon[40196]: Reconfiguring daemon alertmanager.a on smithi018 2024-09-10T18:47:11.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:11 smithi018 ceph-mon[40196]: 4.7 deep-scrub starts 2024-09-10T18:47:11.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:11 smithi018 ceph-mon[40196]: 4.7 deep-scrub ok 2024-09-10T18:47:11.424 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:11 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:11.424 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:11 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:47:11.424 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:11 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:47:11.424 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:11 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:11.424 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:11 smithi103 ceph-mon[33163]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-10T18:47:11.424 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:11 smithi103 ceph-mon[33163]: Reconfiguring daemon alertmanager.a on smithi018 2024-09-10T18:47:11.424 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:11 smithi103 ceph-mon[33163]: 4.7 deep-scrub starts 2024-09-10T18:47:11.424 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:11 smithi103 ceph-mon[33163]: 4.7 deep-scrub ok 2024-09-10T18:47:11.680 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T18:47:12.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:12 smithi018 ceph-mon[29336]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:47:12.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:12 smithi018 ceph-mon[40196]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:47:12.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:12 smithi103 ceph-mon[33163]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:47:13.541 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:47:13.664 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:47:13 smithi018 systemd[1]: Stopping Ceph alertmanager.a for ba327f6c-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:47:14.298 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:14 smithi018 ceph-mon[40196]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:47:14.298 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:14 smithi018 ceph-mon[40196]: 4.13 scrub starts 2024-09-10T18:47:14.299 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:14 smithi018 ceph-mon[40196]: 4.13 scrub ok 2024-09-10T18:47:14.299 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:14 smithi018 ceph-mon[29336]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:47:14.299 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:14 smithi018 ceph-mon[29336]: 4.13 scrub starts 2024-09-10T18:47:14.299 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:14 smithi018 ceph-mon[29336]: 4.13 scrub ok 2024-09-10T18:47:14.299 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:47:13 smithi018 podman[75109]: 2024-09-10 18:47:13.992956346 +0000 UTC m=+0.420427626 container died 3341795a5b76c8090c62ad61d9a159bb7ae27f19430490a1fadc67a236b3af89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=stupefied_morse, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T18:47:14.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:14 smithi103 ceph-mon[33163]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:47:14.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:14 smithi103 ceph-mon[33163]: 4.13 scrub starts 2024-09-10T18:47:14.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:14 smithi103 ceph-mon[33163]: 4.13 scrub ok 2024-09-10T18:47:14.681 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:47:14 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a[72294]: ts=2024-09-10T18:47:14.297Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-10T18:47:14.681 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:47:14 smithi018 podman[75156]: 2024-09-10 18:47:14.328103065 +0000 UTC m=+0.113050891 container died 593b72795192240c8e5e3e0a60106d943380cfb92f8aff77cd5b5cb0633e29ff (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:47:14.944 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:47:14 smithi018 podman[75156]: 2024-09-10 18:47:14.680201227 +0000 UTC m=+0.465149052 container cleanup 593b72795192240c8e5e3e0a60106d943380cfb92f8aff77cd5b5cb0633e29ff (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:47:14.944 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:47:14 smithi018 bash[75156]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a 2024-09-10T18:47:15.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:15 smithi018 ceph-mon[40196]: 6.0 scrub starts 2024-09-10T18:47:15.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:15 smithi018 ceph-mon[40196]: 6.0 scrub ok 2024-09-10T18:47:15.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:15 smithi018 ceph-mon[40196]: 5.13 scrub starts 2024-09-10T18:47:15.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:15 smithi018 ceph-mon[40196]: 5.13 scrub ok 2024-09-10T18:47:15.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:15 smithi018 ceph-mon[29336]: 6.0 scrub starts 2024-09-10T18:47:15.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:15 smithi018 ceph-mon[29336]: 6.0 scrub ok 2024-09-10T18:47:15.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:15 smithi018 ceph-mon[29336]: 5.13 scrub starts 2024-09-10T18:47:15.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:15 smithi018 ceph-mon[29336]: 5.13 scrub ok 2024-09-10T18:47:15.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:15 smithi103 ceph-mon[33163]: 6.0 scrub starts 2024-09-10T18:47:15.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:15 smithi103 ceph-mon[33163]: 6.0 scrub ok 2024-09-10T18:47:15.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:15 smithi103 ceph-mon[33163]: 5.13 scrub starts 2024-09-10T18:47:15.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:15 smithi103 ceph-mon[33163]: 5.13 scrub ok 2024-09-10T18:47:15.888 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:47:15.889 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:47:15 smithi018 bash[75239]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a 2024-09-10T18:47:15.895 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-09-10T18:47:16.194 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:47:16 smithi018 podman[75168]: 2024-09-10 18:47:16.114568007 +0000 UTC m=+1.783427995 container remove 593b72795192240c8e5e3e0a60106d943380cfb92f8aff77cd5b5cb0633e29ff (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:47:16.473 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:16 smithi018 ceph-mon[40196]: 3.1 scrub starts 2024-09-10T18:47:16.473 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:16 smithi018 ceph-mon[40196]: 3.1 scrub ok 2024-09-10T18:47:16.473 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:16 smithi018 ceph-mon[40196]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:47:16.473 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:16 smithi018 ceph-mon[29336]: 3.1 scrub starts 2024-09-10T18:47:16.473 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:16 smithi018 ceph-mon[29336]: 3.1 scrub ok 2024-09-10T18:47:16.474 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:16 smithi018 ceph-mon[29336]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:47:16.474 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:47:16 smithi018 podman[75168]: 2024-09-10 18:47:16.198515733 +0000 UTC m=+1.867375720 volume remove d79f05032aa5bfdafa52a969a8cd94519391fcf35464f9f93bd693b7ca57395a 2024-09-10T18:47:16.474 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:47:16 smithi018 systemd[1]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-10T18:47:16.474 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:47:16 smithi018 systemd[1]: Stopped Ceph alertmanager.a for ba327f6c-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:47:16.474 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:47:16 smithi018 systemd[1]: Starting Ceph alertmanager.a for ba327f6c-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:47:16.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:16 smithi103 ceph-mon[33163]: 3.1 scrub starts 2024-09-10T18:47:16.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:16 smithi103 ceph-mon[33163]: 3.1 scrub ok 2024-09-10T18:47:16.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:16 smithi103 ceph-mon[33163]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:47:16.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:47:16 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:47:16.991 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2024-09-10T18:47:15.033144+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":958,"num_read_kb":815,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1180956,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":584,"ondisk_log_size":584,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385736,"kb_used_data":6896,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575480,"statfs":{"total":767960285184,"available":765517291520,"internally_reserved":0,"allocated":7061504,"data_stored":4203972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002915"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.740866+0000","last_change":"2024-09-10T18:46:33.824057+0000","last_active":"2024-09-10T18:46:56.740866+0000","last_peered":"2024-09-10T18:46:56.740866+0000","last_clean":"2024-09-10T18:46:56.740866+0000","last_became_active":"2024-09-10T18:45:24.265488+0000","last_became_peered":"2024-09-10T18:45:24.265488+0000","last_unstale":"2024-09-10T18:46:56.740866+0000","last_undegraded":"2024-09-10T18:46:56.740866+0000","last_fullsized":"2024-09-10T18:46:56.740866+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:33.823989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:33.823989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:44.131457+0000","scrub_duration":0.0034389770000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:00.536124+0000","last_change":"2024-09-10T18:47:00.536064+0000","last_active":"2024-09-10T18:47:00.536124+0000","last_peered":"2024-09-10T18:47:00.536124+0000","last_clean":"2024-09-10T18:47:00.536124+0000","last_became_active":"2024-09-10T18:45:17.959475+0000","last_became_peered":"2024-09-10T18:45:17.959475+0000","last_unstale":"2024-09-10T18:47:00.536124+0000","last_undegraded":"2024-09-10T18:47:00.536124+0000","last_fullsized":"2024-09-10T18:47:00.536124+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:00.535992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:47:00.535992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:13.246010+0000","scrub_duration":0.001731841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"63'15","reported_seq":70,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.741145+0000","last_change":"2024-09-10T18:46:36.856804+0000","last_active":"2024-09-10T18:46:56.741145+0000","last_peered":"2024-09-10T18:46:56.741145+0000","last_clean":"2024-09-10T18:46:56.741145+0000","last_became_active":"2024-09-10T18:45:20.052531+0000","last_became_peered":"2024-09-10T18:45:20.052531+0000","last_unstale":"2024-09-10T18:46:56.741145+0000","last_undegraded":"2024-09-10T18:46:56.741145+0000","last_fullsized":"2024-09-10T18:46:56.741145+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:46:36.856740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:36.856740+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:03.997270+0000","scrub_duration":0.0018131040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.739031+0000","last_change":"2024-09-10T18:46:40.313306+0000","last_active":"2024-09-10T18:46:56.739031+0000","last_peered":"2024-09-10T18:46:56.739031+0000","last_clean":"2024-09-10T18:46:56.739031+0000","last_became_active":"2024-09-10T18:45:22.065619+0000","last_became_peered":"2024-09-10T18:45:22.065619+0000","last_unstale":"2024-09-10T18:46:56.739031+0000","last_undegraded":"2024-09-10T18:46:56.739031+0000","last_fullsized":"2024-09-10T18:46:56.739031+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:40.313218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:40.313218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:52.103586+0000","scrub_duration":0.0017922750000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.741286+0000","last_change":"2024-09-10T18:46:18.890844+0000","last_active":"2024-09-10T18:46:56.741286+0000","last_peered":"2024-09-10T18:46:56.741286+0000","last_clean":"2024-09-10T18:46:56.741286+0000","last_became_active":"2024-09-10T18:45:17.966153+0000","last_became_peered":"2024-09-10T18:45:17.966153+0000","last_unstale":"2024-09-10T18:46:56.741286+0000","last_undegraded":"2024-09-10T18:46:56.741286+0000","last_fullsized":"2024-09-10T18:46:56.741286+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:18.890702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:18.890702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:47.062002+0000","scrub_duration":0.0019382589999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"83'13","reported_seq":65,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:59.906804+0000","last_change":"2024-09-10T18:46:59.906707+0000","last_active":"2024-09-10T18:46:59.906804+0000","last_peered":"2024-09-10T18:46:59.906804+0000","last_clean":"2024-09-10T18:46:59.906804+0000","last_became_active":"2024-09-10T18:45:20.061266+0000","last_became_peered":"2024-09-10T18:45:20.061266+0000","last_unstale":"2024-09-10T18:46:59.906804+0000","last_undegraded":"2024-09-10T18:46:59.906804+0000","last_fullsized":"2024-09-10T18:46:59.906804+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'13","last_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_clean_scrub_stamp":"2024-09-10T18:46:59.906634+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:29.738998+0000","scrub_duration":0.003988892,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.737469+0000","last_change":"2024-09-10T18:45:22.073139+0000","last_active":"2024-09-10T18:46:56.737469+0000","last_peered":"2024-09-10T18:46:56.737469+0000","last_clean":"2024-09-10T18:46:56.737469+0000","last_became_active":"2024-09-10T18:45:22.072822+0000","last_became_peered":"2024-09-10T18:45:22.072822+0000","last_unstale":"2024-09-10T18:46:56.737469+0000","last_undegraded":"2024-09-10T18:46:56.737469+0000","last_fullsized":"2024-09-10T18:46:56.737469+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:00.946342+0000","last_change":"2024-09-10T18:47:00.946309+0000","last_active":"2024-09-10T18:47:00.946342+0000","last_peered":"2024-09-10T18:47:00.946342+0000","last_clean":"2024-09-10T18:47:00.946342+0000","last_became_active":"2024-09-10T18:45:24.081002+0000","last_became_peered":"2024-09-10T18:45:24.081002+0000","last_unstale":"2024-09-10T18:47:00.946342+0000","last_undegraded":"2024-09-10T18:47:00.946342+0000","last_fullsized":"2024-09-10T18:47:00.946342+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:00.946255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:00.946255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:14.872519+0000","scrub_duration":0.001207629,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.744772+0000","last_change":"2024-09-10T18:46:46.605919+0000","last_active":"2024-09-10T18:46:56.744772+0000","last_peered":"2024-09-10T18:46:56.744772+0000","last_clean":"2024-09-10T18:46:56.744772+0000","last_became_active":"2024-09-10T18:45:17.961097+0000","last_became_peered":"2024-09-10T18:45:17.961097+0000","last_unstale":"2024-09-10T18:46:56.744772+0000","last_undegraded":"2024-09-10T18:46:56.744772+0000","last_fullsized":"2024-09-10T18:46:56.744772+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:46.605814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:46.605814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:51.831114+0000","scrub_duration":0.001570625,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"63'5","reported_seq":55,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:04.002271+0000","last_change":"2024-09-10T18:47:04.002228+0000","last_active":"2024-09-10T18:47:04.002271+0000","last_peered":"2024-09-10T18:47:04.002271+0000","last_clean":"2024-09-10T18:47:04.002271+0000","last_became_active":"2024-09-10T18:45:20.057567+0000","last_became_peered":"2024-09-10T18:45:20.057567+0000","last_unstale":"2024-09-10T18:47:04.002271+0000","last_undegraded":"2024-09-10T18:47:04.002271+0000","last_fullsized":"2024-09-10T18:47:04.002271+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-10T18:47:04.002164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:04.002164+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:20.148733+0000","scrub_duration":0.026374999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.744217+0000","last_change":"2024-09-10T18:46:22.547871+0000","last_active":"2024-09-10T18:46:56.744217+0000","last_peered":"2024-09-10T18:46:56.744217+0000","last_clean":"2024-09-10T18:46:56.744217+0000","last_became_active":"2024-09-10T18:45:22.076575+0000","last_became_peered":"2024-09-10T18:45:22.076575+0000","last_unstale":"2024-09-10T18:46:56.744217+0000","last_undegraded":"2024-09-10T18:46:56.744217+0000","last_fullsized":"2024-09-10T18:46:56.744217+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:22.547800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:22.547800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:32.775669+0000","scrub_duration":0.001880622,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.740758+0000","last_change":"2024-09-10T18:46:36.158152+0000","last_active":"2024-09-10T18:46:56.740758+0000","last_peered":"2024-09-10T18:46:56.740758+0000","last_clean":"2024-09-10T18:46:56.740758+0000","last_became_active":"2024-09-10T18:45:24.079534+0000","last_became_peered":"2024-09-10T18:45:24.079534+0000","last_unstale":"2024-09-10T18:46:56.740758+0000","last_undegraded":"2024-09-10T18:46:56.740758+0000","last_fullsized":"2024-09-10T18:46:56.740758+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:36.158087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:36.158087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:41.325875+0000","scrub_duration":0.0014477100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.744265+0000","last_change":"2024-09-10T18:46:50.666447+0000","last_active":"2024-09-10T18:46:56.744265+0000","last_peered":"2024-09-10T18:46:56.744265+0000","last_clean":"2024-09-10T18:46:56.744265+0000","last_became_active":"2024-09-10T18:45:17.963121+0000","last_became_peered":"2024-09-10T18:45:17.963121+0000","last_unstale":"2024-09-10T18:46:56.744265+0000","last_undegraded":"2024-09-10T18:46:56.744265+0000","last_fullsized":"2024-09-10T18:46:56.744265+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:50.666379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.666379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:06.539690+0000","scrub_duration":0.0016221689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"63'9","reported_seq":61,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.739772+0000","last_change":"2024-09-10T18:46:55.069893+0000","last_active":"2024-09-10T18:46:56.739772+0000","last_peered":"2024-09-10T18:46:56.739772+0000","last_clean":"2024-09-10T18:46:56.739772+0000","last_became_active":"2024-09-10T18:45:20.052652+0000","last_became_peered":"2024-09-10T18:45:20.052652+0000","last_unstale":"2024-09-10T18:46:56.739772+0000","last_undegraded":"2024-09-10T18:46:56.739772+0000","last_fullsized":"2024-09-10T18:46:56.739772+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:46:55.069833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:55.069833+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:12.396664+0000","scrub_duration":0.0016685510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.740905+0000","last_change":"2024-09-10T18:46:31.024322+0000","last_active":"2024-09-10T18:46:56.740905+0000","last_peered":"2024-09-10T18:46:56.740905+0000","last_clean":"2024-09-10T18:46:56.740905+0000","last_became_active":"2024-09-10T18:45:22.068326+0000","last_became_peered":"2024-09-10T18:45:22.068326+0000","last_unstale":"2024-09-10T18:46:56.740905+0000","last_undegraded":"2024-09-10T18:46:56.740905+0000","last_fullsized":"2024-09-10T18:46:56.740905+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:31.024255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:31.024255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:50.407493+0000","scrub_duration":0.00152395,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.736968+0000","last_change":"2024-09-10T18:45:24.081968+0000","last_active":"2024-09-10T18:46:56.736968+0000","last_peered":"2024-09-10T18:46:56.736968+0000","last_clean":"2024-09-10T18:46:56.736968+0000","last_became_active":"2024-09-10T18:45:24.081488+0000","last_became_peered":"2024-09-10T18:45:24.081488+0000","last_unstale":"2024-09-10T18:46:56.736968+0000","last_undegraded":"2024-09-10T18:46:56.736968+0000","last_fullsized":"2024-09-10T18:46:56.736968+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"78'25","reported_seq":81,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.740003+0000","last_change":"2024-09-10T18:46:44.316200+0000","last_active":"2024-09-10T18:46:56.740003+0000","last_peered":"2024-09-10T18:46:56.740003+0000","last_clean":"2024-09-10T18:46:56.740003+0000","last_became_active":"2024-09-10T18:45:20.057019+0000","last_became_peered":"2024-09-10T18:45:20.057019+0000","last_unstale":"2024-09-10T18:46:56.740003+0000","last_undegraded":"2024-09-10T18:46:56.740003+0000","last_fullsized":"2024-09-10T18:46:56.740003+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'25","last_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_deep_scrub":"78'25","last_deep_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_clean_scrub_stamp":"2024-09-10T18:46:44.316146+0000","objects_scrubbed":8,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:06.777730+0000","scrub_duration":0.0046415040000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.744528+0000","last_change":"2024-09-10T18:46:47.595182+0000","last_active":"2024-09-10T18:46:56.744528+0000","last_peered":"2024-09-10T18:46:56.744528+0000","last_clean":"2024-09-10T18:46:56.744528+0000","last_became_active":"2024-09-10T18:45:17.963732+0000","last_became_peered":"2024-09-10T18:45:17.963732+0000","last_unstale":"2024-09-10T18:46:56.744528+0000","last_undegraded":"2024-09-10T18:46:56.744528+0000","last_fullsized":"2024-09-10T18:46:56.744528+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:47.595114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:47.595114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:53.271455+0000","scrub_duration":0.0014913249999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:04.512511+0000","last_change":"2024-09-10T18:47:04.512478+0000","last_active":"2024-09-10T18:47:04.512511+0000","last_peered":"2024-09-10T18:47:04.512511+0000","last_clean":"2024-09-10T18:47:04.512511+0000","last_became_active":"2024-09-10T18:45:22.071443+0000","last_became_peered":"2024-09-10T18:45:22.071443+0000","last_unstale":"2024-09-10T18:47:04.512511+0000","last_undegraded":"2024-09-10T18:47:04.512511+0000","last_fullsized":"2024-09-10T18:47:04.512511+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:04.512425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:04.512425+0000","last_clean_scrub_stamp":"2024-09-10T18:47:04.512425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:26.715434+0000","scrub_duration":0.001363324,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.740438+0000","last_change":"2024-09-10T18:46:35.738217+0000","last_active":"2024-09-10T18:46:56.740438+0000","last_peered":"2024-09-10T18:46:56.740438+0000","last_clean":"2024-09-10T18:46:56.740438+0000","last_became_active":"2024-09-10T18:45:24.077897+0000","last_became_peered":"2024-09-10T18:45:24.077897+0000","last_unstale":"2024-09-10T18:46:56.740438+0000","last_undegraded":"2024-09-10T18:46:56.740438+0000","last_fullsized":"2024-09-10T18:46:56.740438+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:35.738143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:35.738143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:48.741747+0000","scrub_duration":0.0015293520000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"63'10","reported_seq":60,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.739889+0000","last_change":"2024-09-10T18:46:31.281588+0000","last_active":"2024-09-10T18:46:56.739889+0000","last_peered":"2024-09-10T18:46:56.739889+0000","last_clean":"2024-09-10T18:46:56.739889+0000","last_became_active":"2024-09-10T18:45:20.053428+0000","last_became_peered":"2024-09-10T18:45:20.053428+0000","last_unstale":"2024-09-10T18:46:56.739889+0000","last_undegraded":"2024-09-10T18:46:56.739889+0000","last_fullsized":"2024-09-10T18:46:56.739889+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:46:31.281519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:31.281519+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:58.508120+0000","scrub_duration":0.0019095119999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.738107+0000","last_change":"2024-09-10T18:46:38.445417+0000","last_active":"2024-09-10T18:46:56.738107+0000","last_peered":"2024-09-10T18:46:56.738107+0000","last_clean":"2024-09-10T18:46:56.738107+0000","last_became_active":"2024-09-10T18:45:17.966833+0000","last_became_peered":"2024-09-10T18:45:17.966833+0000","last_unstale":"2024-09-10T18:46:56.738107+0000","last_undegraded":"2024-09-10T18:46:56.738107+0000","last_fullsized":"2024-09-10T18:46:56.738107+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:38.445362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:38.445362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:02.460484+0000","scrub_duration":0.0012292029999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"63'8","reported_seq":73,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:06.096578+0000","last_change":"2024-09-10T18:46:25.787936+0000","last_active":"2024-09-10T18:47:06.096578+0000","last_peered":"2024-09-10T18:47:06.096578+0000","last_clean":"2024-09-10T18:47:06.096578+0000","last_became_active":"2024-09-10T18:45:22.068453+0000","last_became_peered":"2024-09-10T18:45:22.068453+0000","last_unstale":"2024-09-10T18:47:06.096578+0000","last_undegraded":"2024-09-10T18:47:06.096578+0000","last_fullsized":"2024-09-10T18:47:06.096578+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:25.787877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:25.787877+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:54.201629+0000","scrub_duration":0.0018093530000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.740091+0000","last_change":"2024-09-10T18:46:32.265034+0000","last_active":"2024-09-10T18:46:56.740091+0000","last_peered":"2024-09-10T18:46:56.740091+0000","last_clean":"2024-09-10T18:46:56.740091+0000","last_became_active":"2024-09-10T18:45:24.081082+0000","last_became_peered":"2024-09-10T18:45:24.081082+0000","last_unstale":"2024-09-10T18:46:56.740091+0000","last_undegraded":"2024-09-10T18:46:56.740091+0000","last_fullsized":"2024-09-10T18:46:56.740091+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:32.264963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:32.264963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:50.207705+0000","scrub_duration":0.002378509,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"63'15","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:58.122352+0000","last_change":"2024-09-10T18:46:58.122279+0000","last_active":"2024-09-10T18:46:58.122352+0000","last_peered":"2024-09-10T18:46:58.122352+0000","last_clean":"2024-09-10T18:46:58.122352+0000","last_became_active":"2024-09-10T18:46:56.742946+0000","last_became_peered":"2024-09-10T18:46:56.742946+0000","last_unstale":"2024-09-10T18:46:58.122352+0000","last_undegraded":"2024-09-10T18:46:58.122352+0000","last_fullsized":"2024-09-10T18:46:58.122352+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:46:58.122219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:58.122219+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:05.975000+0000","scrub_duration":0.001678094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.737646+0000","last_change":"2024-09-10T18:46:28.320761+0000","last_active":"2024-09-10T18:46:56.737646+0000","last_peered":"2024-09-10T18:46:56.737646+0000","last_clean":"2024-09-10T18:46:56.737646+0000","last_became_active":"2024-09-10T18:45:17.965409+0000","last_became_peered":"2024-09-10T18:45:17.965409+0000","last_unstale":"2024-09-10T18:46:56.737646+0000","last_undegraded":"2024-09-10T18:46:56.737646+0000","last_fullsized":"2024-09-10T18:46:56.737646+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_clean_scrub_stamp":"2024-09-10T18:46:28.320704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:44.092028+0000","scrub_duration":0.001393476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"63'8","reported_seq":73,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:06.096669+0000","last_change":"2024-09-10T18:46:39.300139+0000","last_active":"2024-09-10T18:47:06.096669+0000","last_peered":"2024-09-10T18:47:06.096669+0000","last_clean":"2024-09-10T18:47:06.096669+0000","last_became_active":"2024-09-10T18:45:22.062497+0000","last_became_peered":"2024-09-10T18:45:22.062497+0000","last_unstale":"2024-09-10T18:47:06.096669+0000","last_undegraded":"2024-09-10T18:47:06.096669+0000","last_fullsized":"2024-09-10T18:47:06.096669+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:39.300065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:39.300065+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:44.830898+0000","scrub_duration":0.001858934,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.741063+0000","last_change":"2024-09-10T18:46:42.057055+0000","last_active":"2024-09-10T18:46:56.741063+0000","last_peered":"2024-09-10T18:46:56.741063+0000","last_clean":"2024-09-10T18:46:56.741063+0000","last_became_active":"2024-09-10T18:45:24.082362+0000","last_became_peered":"2024-09-10T18:45:24.082362+0000","last_unstale":"2024-09-10T18:46:56.741063+0000","last_undegraded":"2024-09-10T18:46:56.741063+0000","last_fullsized":"2024-09-10T18:46:56.741063+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:42.056970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:42.056970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:50.036040+0000","scrub_duration":0.0016528479999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"63'11","reported_seq":86,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.747805+0000","last_change":"2024-09-10T18:46:56.747580+0000","last_active":"2024-09-10T18:46:56.747805+0000","last_peered":"2024-09-10T18:46:56.747805+0000","last_clean":"2024-09-10T18:46:56.747805+0000","last_became_active":"2024-09-10T18:46:56.741936+0000","last_became_peered":"2024-09-10T18:46:56.741936+0000","last_unstale":"2024-09-10T18:46:56.747805+0000","last_undegraded":"2024-09-10T18:46:56.747805+0000","last_fullsized":"2024-09-10T18:46:56.747805+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:46:35.274660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:35.274660+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:42.198733+0000","scrub_duration":0.0019745819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.744613+0000","last_change":"2024-09-10T18:46:39.520497+0000","last_active":"2024-09-10T18:46:56.744613+0000","last_peered":"2024-09-10T18:46:56.744613+0000","last_clean":"2024-09-10T18:46:56.744613+0000","last_became_active":"2024-09-10T18:45:17.966133+0000","last_became_peered":"2024-09-10T18:45:17.966133+0000","last_unstale":"2024-09-10T18:46:56.744613+0000","last_undegraded":"2024-09-10T18:46:56.744613+0000","last_fullsized":"2024-09-10T18:46:56.744613+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:39.520432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:39.520432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:49.040271+0000","scrub_duration":0.0013982510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.741520+0000","last_change":"2024-09-10T18:46:48.116414+0000","last_active":"2024-09-10T18:46:56.741520+0000","last_peered":"2024-09-10T18:46:56.741520+0000","last_clean":"2024-09-10T18:46:56.741520+0000","last_became_active":"2024-09-10T18:45:22.068918+0000","last_became_peered":"2024-09-10T18:45:22.068918+0000","last_unstale":"2024-09-10T18:46:56.741520+0000","last_undegraded":"2024-09-10T18:46:56.741520+0000","last_fullsized":"2024-09-10T18:46:56.741520+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_clean_scrub_stamp":"2024-09-10T18:46:48.116358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:48.149931+0000","scrub_duration":0.001429331,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:00.743777+0000","last_change":"2024-09-10T18:47:00.743740+0000","last_active":"2024-09-10T18:47:00.743777+0000","last_peered":"2024-09-10T18:47:00.743777+0000","last_clean":"2024-09-10T18:47:00.743777+0000","last_became_active":"2024-09-10T18:45:24.264538+0000","last_became_peered":"2024-09-10T18:45:24.264538+0000","last_unstale":"2024-09-10T18:47:00.743777+0000","last_undegraded":"2024-09-10T18:47:00.743777+0000","last_fullsized":"2024-09-10T18:47:00.743777+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:00.743680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:00.743680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:02.404002+0000","scrub_duration":0.0022093759999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"63'14","reported_seq":66,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.739247+0000","last_change":"2024-09-10T18:46:29.330572+0000","last_active":"2024-09-10T18:46:56.739247+0000","last_peered":"2024-09-10T18:46:56.739247+0000","last_clean":"2024-09-10T18:46:56.739247+0000","last_became_active":"2024-09-10T18:45:20.052243+0000","last_became_peered":"2024-09-10T18:45:20.052243+0000","last_unstale":"2024-09-10T18:46:56.739247+0000","last_undegraded":"2024-09-10T18:46:56.739247+0000","last_fullsized":"2024-09-10T18:46:56.739247+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-10T18:46:29.330503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:29.330503+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:55.456983+0000","scrub_duration":0.0020437459999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.737985+0000","last_change":"2024-09-10T18:46:25.206801+0000","last_active":"2024-09-10T18:46:56.737985+0000","last_peered":"2024-09-10T18:46:56.737985+0000","last_clean":"2024-09-10T18:46:56.737985+0000","last_became_active":"2024-09-10T18:45:17.966027+0000","last_became_peered":"2024-09-10T18:45:17.966027+0000","last_unstale":"2024-09-10T18:46:56.737985+0000","last_undegraded":"2024-09-10T18:46:56.737985+0000","last_fullsized":"2024-09-10T18:46:56.737985+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_clean_scrub_stamp":"2024-09-10T18:46:25.206731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:46.038154+0000","scrub_duration":0.0016589910000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":73,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:06.096585+0000","last_change":"2024-09-10T18:46:40.808518+0000","last_active":"2024-09-10T18:47:06.096585+0000","last_peered":"2024-09-10T18:47:06.096585+0000","last_clean":"2024-09-10T18:47:06.096585+0000","last_became_active":"2024-09-10T18:45:22.074540+0000","last_became_peered":"2024-09-10T18:45:22.074540+0000","last_unstale":"2024-09-10T18:47:06.096585+0000","last_undegraded":"2024-09-10T18:47:06.096585+0000","last_fullsized":"2024-09-10T18:47:06.096585+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:40.808453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:40.808453+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:43.962254+0000","scrub_duration":0.0016757829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.744260+0000","last_change":"2024-09-10T18:46:32.590511+0000","last_active":"2024-09-10T18:46:56.744260+0000","last_peered":"2024-09-10T18:46:56.744260+0000","last_clean":"2024-09-10T18:46:56.744260+0000","last_became_active":"2024-09-10T18:45:24.265303+0000","last_became_peered":"2024-09-10T18:45:24.265303+0000","last_unstale":"2024-09-10T18:46:56.744260+0000","last_undegraded":"2024-09-10T18:46:56.744260+0000","last_fullsized":"2024-09-10T18:46:56.744260+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:32.590438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:32.590438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:01.680676+0000","scrub_duration":0.0013292270000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"63'18","reported_seq":77,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.741740+0000","last_change":"2024-09-10T18:46:27.801177+0000","last_active":"2024-09-10T18:46:56.741740+0000","last_peered":"2024-09-10T18:46:56.741740+0000","last_clean":"2024-09-10T18:46:56.741740+0000","last_became_active":"2024-09-10T18:45:20.420937+0000","last_became_peered":"2024-09-10T18:45:20.420937+0000","last_unstale":"2024-09-10T18:46:56.741740+0000","last_undegraded":"2024-09-10T18:46:56.741740+0000","last_fullsized":"2024-09-10T18:46:56.741740+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'18","last_scrub_stamp":"2024-09-10T18:46:27.801110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:27.801110+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:42.247024+0000","scrub_duration":0.001870095,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:05.505483+0000","last_change":"2024-09-10T18:47:05.505431+0000","last_active":"2024-09-10T18:47:05.505483+0000","last_peered":"2024-09-10T18:47:05.505483+0000","last_clean":"2024-09-10T18:47:05.505483+0000","last_became_active":"2024-09-10T18:45:17.961695+0000","last_became_peered":"2024-09-10T18:45:17.961695+0000","last_unstale":"2024-09-10T18:47:05.505483+0000","last_undegraded":"2024-09-10T18:47:05.505483+0000","last_fullsized":"2024-09-10T18:47:05.505483+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:05.505376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:47:05.505376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:32.771817+0000","scrub_duration":0.0013302310000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.739142+0000","last_change":"2024-09-10T18:46:41.312735+0000","last_active":"2024-09-10T18:46:56.739142+0000","last_peered":"2024-09-10T18:46:56.739142+0000","last_clean":"2024-09-10T18:46:56.739142+0000","last_became_active":"2024-09-10T18:45:22.074012+0000","last_became_peered":"2024-09-10T18:45:22.074012+0000","last_unstale":"2024-09-10T18:46:56.739142+0000","last_undegraded":"2024-09-10T18:46:56.739142+0000","last_fullsized":"2024-09-10T18:46:56.739142+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:41.312671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:41.312671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:47.779534+0000","scrub_duration":0.0013475010000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.739695+0000","last_change":"2024-09-10T18:46:53.087609+0000","last_active":"2024-09-10T18:46:56.739695+0000","last_peered":"2024-09-10T18:46:56.739695+0000","last_clean":"2024-09-10T18:46:56.739695+0000","last_became_active":"2024-09-10T18:45:24.076831+0000","last_became_peered":"2024-09-10T18:45:24.076831+0000","last_unstale":"2024-09-10T18:46:56.739695+0000","last_undegraded":"2024-09-10T18:46:56.739695+0000","last_fullsized":"2024-09-10T18:46:56.739695+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_clean_scrub_stamp":"2024-09-10T18:46:53.087539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:56.714234+0000","scrub_duration":0.001678971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"63'19","reported_seq":81,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.738327+0000","last_change":"2024-09-10T18:46:42.524291+0000","last_active":"2024-09-10T18:46:56.738327+0000","last_peered":"2024-09-10T18:46:56.738327+0000","last_clean":"2024-09-10T18:46:56.738327+0000","last_became_active":"2024-09-10T18:45:20.421046+0000","last_became_peered":"2024-09-10T18:45:20.421046+0000","last_unstale":"2024-09-10T18:46:56.738327+0000","last_undegraded":"2024-09-10T18:46:56.738327+0000","last_fullsized":"2024-09-10T18:46:56.738327+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:46:42.524195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:42.524195+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:45.664487+0000","scrub_duration":0.0022840299999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:00.592403+0000","last_change":"2024-09-10T18:47:00.592359+0000","last_active":"2024-09-10T18:47:00.592403+0000","last_peered":"2024-09-10T18:47:00.592403+0000","last_clean":"2024-09-10T18:47:00.592403+0000","last_became_active":"2024-09-10T18:45:17.960226+0000","last_became_peered":"2024-09-10T18:45:17.960226+0000","last_unstale":"2024-09-10T18:47:00.592403+0000","last_undegraded":"2024-09-10T18:47:00.592403+0000","last_fullsized":"2024-09-10T18:47:00.592403+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:00.592279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:47:00.592279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:03.737142+0000","scrub_duration":0.001418576,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.737317+0000","last_change":"2024-09-10T18:46:52.235456+0000","last_active":"2024-09-10T18:46:56.737317+0000","last_peered":"2024-09-10T18:46:56.737317+0000","last_clean":"2024-09-10T18:46:56.737317+0000","last_became_active":"2024-09-10T18:45:22.066148+0000","last_became_peered":"2024-09-10T18:45:22.066148+0000","last_unstale":"2024-09-10T18:46:56.737317+0000","last_undegraded":"2024-09-10T18:46:56.737317+0000","last_fullsized":"2024-09-10T18:46:56.737317+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:52.235389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:52.235389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:06.785777+0000","scrub_duration":0.001679675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.737399+0000","last_change":"2024-09-10T18:45:24.265025+0000","last_active":"2024-09-10T18:46:56.737399+0000","last_peered":"2024-09-10T18:46:56.737399+0000","last_clean":"2024-09-10T18:46:56.737399+0000","last_became_active":"2024-09-10T18:45:24.264680+0000","last_became_peered":"2024-09-10T18:45:24.264680+0000","last_unstale":"2024-09-10T18:46:56.737399+0000","last_undegraded":"2024-09-10T18:46:56.737399+0000","last_fullsized":"2024-09-10T18:46:56.737399+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"83'15","reported_seq":65,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:58.436638+0000","last_change":"2024-09-10T18:46:58.436565+0000","last_active":"2024-09-10T18:46:58.436638+0000","last_peered":"2024-09-10T18:46:58.436638+0000","last_clean":"2024-09-10T18:46:58.436638+0000","last_became_active":"2024-09-10T18:45:20.057204+0000","last_became_peered":"2024-09-10T18:45:20.057204+0000","last_unstale":"2024-09-10T18:46:58.436638+0000","last_undegraded":"2024-09-10T18:46:58.436638+0000","last_fullsized":"2024-09-10T18:46:58.436638+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'15","last_scrub_stamp":"2024-09-10T18:46:58.436473+0000","last_deep_scrub":"83'15","last_deep_scrub_stamp":"2024-09-10T18:46:58.436473+0000","last_clean_scrub_stamp":"2024-09-10T18:46:58.436473+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:59.402142+0000","scrub_duration":0.0046048119999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.741130+0000","last_change":"2024-09-10T18:46:50.443101+0000","last_active":"2024-09-10T18:46:56.741130+0000","last_peered":"2024-09-10T18:46:56.741130+0000","last_clean":"2024-09-10T18:46:56.741130+0000","last_became_active":"2024-09-10T18:45:17.966632+0000","last_became_peered":"2024-09-10T18:45:17.966632+0000","last_unstale":"2024-09-10T18:46:56.741130+0000","last_undegraded":"2024-09-10T18:46:56.741130+0000","last_fullsized":"2024-09-10T18:46:56.741130+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:50.443042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.443042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:18.989766+0000","scrub_duration":0.0014088550000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"63'8","reported_seq":73,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:06.096573+0000","last_change":"2024-09-10T18:47:03.579260+0000","last_active":"2024-09-10T18:47:06.096573+0000","last_peered":"2024-09-10T18:47:06.096573+0000","last_clean":"2024-09-10T18:47:06.096573+0000","last_became_active":"2024-09-10T18:45:22.063326+0000","last_became_peered":"2024-09-10T18:45:22.063326+0000","last_unstale":"2024-09-10T18:47:06.096573+0000","last_undegraded":"2024-09-10T18:47:06.096573+0000","last_fullsized":"2024-09-10T18:47:06.096573+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:47:03.579186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:47:03.579186+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:22.186155+0000","scrub_duration":0.0021415639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.736875+0000","last_change":"2024-09-10T18:45:24.077780+0000","last_active":"2024-09-10T18:46:56.736875+0000","last_peered":"2024-09-10T18:46:56.736875+0000","last_clean":"2024-09-10T18:46:56.736875+0000","last_became_active":"2024-09-10T18:45:24.077412+0000","last_became_peered":"2024-09-10T18:45:24.077412+0000","last_unstale":"2024-09-10T18:46:56.736875+0000","last_undegraded":"2024-09-10T18:46:56.736875+0000","last_fullsized":"2024-09-10T18:46:56.736875+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"63'28","reported_seq":115,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:06.096599+0000","last_change":"2024-09-10T18:46:19.223319+0000","last_active":"2024-09-10T18:47:06.096599+0000","last_peered":"2024-09-10T18:47:06.096599+0000","last_clean":"2024-09-10T18:47:06.096599+0000","last_became_active":"2024-09-10T18:45:20.061413+0000","last_became_peered":"2024-09-10T18:45:20.061413+0000","last_unstale":"2024-09-10T18:47:06.096599+0000","last_undegraded":"2024-09-10T18:47:06.096599+0000","last_fullsized":"2024-09-10T18:47:06.096599+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'28","last_scrub_stamp":"2024-09-10T18:46:19.223250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:19.223250+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:23.644429+0000","scrub_duration":0.0032067810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.741040+0000","last_change":"2024-09-10T18:46:49.120758+0000","last_active":"2024-09-10T18:46:56.741040+0000","last_peered":"2024-09-10T18:46:56.741040+0000","last_clean":"2024-09-10T18:46:56.741040+0000","last_became_active":"2024-09-10T18:45:17.964494+0000","last_became_peered":"2024-09-10T18:45:17.964494+0000","last_unstale":"2024-09-10T18:46:56.741040+0000","last_undegraded":"2024-09-10T18:46:56.741040+0000","last_fullsized":"2024-09-10T18:46:56.741040+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:49.120704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:49.120704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:51.579079+0000","scrub_duration":0.0011863380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:58.523343+0000","last_change":"2024-09-10T18:46:58.523278+0000","last_active":"2024-09-10T18:46:58.523343+0000","last_peered":"2024-09-10T18:46:58.523343+0000","last_clean":"2024-09-10T18:46:58.523343+0000","last_became_active":"2024-09-10T18:45:22.061428+0000","last_became_peered":"2024-09-10T18:45:22.061428+0000","last_unstale":"2024-09-10T18:46:58.523343+0000","last_undegraded":"2024-09-10T18:46:58.523343+0000","last_fullsized":"2024-09-10T18:46:58.523343+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:58.523217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:58.523217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:04.494370+0000","scrub_duration":0.001307856,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.741093+0000","last_change":"2024-09-10T18:45:24.073623+0000","last_active":"2024-09-10T18:46:56.741093+0000","last_peered":"2024-09-10T18:46:56.741093+0000","last_clean":"2024-09-10T18:46:56.741093+0000","last_became_active":"2024-09-10T18:45:24.073307+0000","last_became_peered":"2024-09-10T18:45:24.073307+0000","last_unstale":"2024-09-10T18:46:56.741093+0000","last_undegraded":"2024-09-10T18:46:56.741093+0000","last_fullsized":"2024-09-10T18:46:56.741093+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"84'18","reported_seq":77,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T18:47:10.515111+0000","last_change":"2024-09-10T18:47:10.515060+0000","last_active":"2024-09-10T18:47:10.515111+0000","last_peered":"2024-09-10T18:47:10.515111+0000","last_clean":"2024-09-10T18:47:10.515111+0000","last_became_active":"2024-09-10T18:45:20.058435+0000","last_became_peered":"2024-09-10T18:45:20.058435+0000","last_unstale":"2024-09-10T18:47:10.515111+0000","last_undegraded":"2024-09-10T18:47:10.515111+0000","last_fullsized":"2024-09-10T18:47:10.515111+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'18","last_scrub_stamp":"2024-09-10T18:47:10.514951+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-10T18:47:10.514951+0000","last_clean_scrub_stamp":"2024-09-10T18:47:10.514951+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:24.824689+0000","scrub_duration":0.005567575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"65'5","reported_seq":150,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T18:47:09.407510+0000","last_change":"2024-09-10T18:45:22.059997+0000","last_active":"2024-09-10T18:47:09.407510+0000","last_peered":"2024-09-10T18:47:09.407510+0000","last_clean":"2024-09-10T18:47:09.407510+0000","last_became_active":"2024-09-10T18:45:17.969868+0000","last_became_peered":"2024-09-10T18:45:17.969868+0000","last_unstale":"2024-09-10T18:47:09.407510+0000","last_undegraded":"2024-09-10T18:47:09.407510+0000","last_fullsized":"2024-09-10T18:47:09.407510+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021933159999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":105,"num_read_kb":100,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.739522+0000","last_change":"2024-09-10T18:46:46.712335+0000","last_active":"2024-09-10T18:46:56.739522+0000","last_peered":"2024-09-10T18:46:56.739522+0000","last_clean":"2024-09-10T18:46:56.739522+0000","last_became_active":"2024-09-10T18:45:17.960859+0000","last_became_peered":"2024-09-10T18:45:17.960859+0000","last_unstale":"2024-09-10T18:46:56.739522+0000","last_undegraded":"2024-09-10T18:46:56.739522+0000","last_fullsized":"2024-09-10T18:46:56.739522+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:46.712269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:46.712269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:54.660825+0000","scrub_duration":0.001509383,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.739633+0000","last_change":"2024-09-10T18:46:41.818890+0000","last_active":"2024-09-10T18:46:56.739633+0000","last_peered":"2024-09-10T18:46:56.739633+0000","last_clean":"2024-09-10T18:46:56.739633+0000","last_became_active":"2024-09-10T18:45:22.068350+0000","last_became_peered":"2024-09-10T18:45:22.068350+0000","last_unstale":"2024-09-10T18:46:56.739633+0000","last_undegraded":"2024-09-10T18:46:56.739633+0000","last_fullsized":"2024-09-10T18:46:56.739633+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:41.818827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:41.818827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:07.148553+0000","scrub_duration":0.0017408219999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:59.553629+0000","last_change":"2024-09-10T18:46:59.553576+0000","last_active":"2024-09-10T18:46:59.553629+0000","last_peered":"2024-09-10T18:46:59.553629+0000","last_clean":"2024-09-10T18:46:59.553629+0000","last_became_active":"2024-09-10T18:45:24.264617+0000","last_became_peered":"2024-09-10T18:45:24.264617+0000","last_unstale":"2024-09-10T18:46:59.553629+0000","last_undegraded":"2024-09-10T18:46:59.553629+0000","last_fullsized":"2024-09-10T18:46:59.553629+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:59.553505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:59.553505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:29.436053+0000","scrub_duration":0.0018297210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"63'12","reported_seq":63,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.738997+0000","last_change":"2024-09-10T18:46:52.472574+0000","last_active":"2024-09-10T18:46:56.738997+0000","last_peered":"2024-09-10T18:46:56.738997+0000","last_clean":"2024-09-10T18:46:56.738997+0000","last_became_active":"2024-09-10T18:45:20.061091+0000","last_became_peered":"2024-09-10T18:45:20.061091+0000","last_unstale":"2024-09-10T18:46:56.738997+0000","last_undegraded":"2024-09-10T18:46:56.738997+0000","last_fullsized":"2024-09-10T18:46:56.738997+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:46:52.472511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:52.472511+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:19.552466+0000","scrub_duration":0.001886935,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"58'1","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.737403+0000","last_change":"2024-09-10T18:45:22.055928+0000","last_active":"2024-09-10T18:46:56.737403+0000","last_peered":"2024-09-10T18:46:56.737403+0000","last_clean":"2024-09-10T18:46:56.737403+0000","last_became_active":"2024-09-10T18:45:17.963932+0000","last_became_peered":"2024-09-10T18:45:17.963932+0000","last_unstale":"2024-09-10T18:46:56.737403+0000","last_undegraded":"2024-09-10T18:46:56.737403+0000","last_fullsized":"2024-09-10T18:46:56.737403+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:45:16.944036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037812099999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.742417+0000","last_change":"2024-09-10T18:46:56.742192+0000","last_active":"2024-09-10T18:46:56.742417+0000","last_peered":"2024-09-10T18:46:56.742417+0000","last_clean":"2024-09-10T18:46:56.742417+0000","last_became_active":"2024-09-10T18:46:56.741862+0000","last_became_peered":"2024-09-10T18:46:56.741862+0000","last_unstale":"2024-09-10T18:46:56.742417+0000","last_undegraded":"2024-09-10T18:46:56.742417+0000","last_fullsized":"2024-09-10T18:46:56.742417+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:54.554426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:54.554426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:17.934705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:59.132347+0000","last_change":"2024-09-10T18:46:59.132301+0000","last_active":"2024-09-10T18:46:59.132347+0000","last_peered":"2024-09-10T18:46:59.132347+0000","last_clean":"2024-09-10T18:46:59.132347+0000","last_became_active":"2024-09-10T18:45:22.064285+0000","last_became_peered":"2024-09-10T18:45:22.064285+0000","last_unstale":"2024-09-10T18:46:59.132347+0000","last_undegraded":"2024-09-10T18:46:59.132347+0000","last_fullsized":"2024-09-10T18:46:59.132347+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:59.132234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:59.132234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:03.150923+0000","scrub_duration":0.001471034,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.740720+0000","last_change":"2024-09-10T18:46:49.422912+0000","last_active":"2024-09-10T18:46:56.740720+0000","last_peered":"2024-09-10T18:46:56.740720+0000","last_clean":"2024-09-10T18:46:56.740720+0000","last_became_active":"2024-09-10T18:45:24.083689+0000","last_became_peered":"2024-09-10T18:45:24.083689+0000","last_unstale":"2024-09-10T18:46:56.740720+0000","last_undegraded":"2024-09-10T18:46:56.740720+0000","last_fullsized":"2024-09-10T18:46:56.740720+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:49.422843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:49.422843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:00.864466+0000","scrub_duration":0.0014985300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"63'14","reported_seq":67,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.737731+0000","last_change":"2024-09-10T18:46:22.294141+0000","last_active":"2024-09-10T18:46:56.737731+0000","last_peered":"2024-09-10T18:46:56.737731+0000","last_clean":"2024-09-10T18:46:56.737731+0000","last_became_active":"2024-09-10T18:45:20.421119+0000","last_became_peered":"2024-09-10T18:45:20.421119+0000","last_unstale":"2024-09-10T18:46:56.737731+0000","last_undegraded":"2024-09-10T18:46:56.737731+0000","last_fullsized":"2024-09-10T18:46:56.737731+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-10T18:46:22.294072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:22.294072+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:49.887899+0000","scrub_duration":0.0021435069999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"60'2","reported_seq":54,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.741745+0000","last_change":"2024-09-10T18:46:25.845131+0000","last_active":"2024-09-10T18:46:56.741745+0000","last_peered":"2024-09-10T18:46:56.741745+0000","last_clean":"2024-09-10T18:46:56.741745+0000","last_became_active":"2024-09-10T18:45:17.961660+0000","last_became_peered":"2024-09-10T18:45:17.961660+0000","last_unstale":"2024-09-10T18:46:56.741745+0000","last_undegraded":"2024-09-10T18:46:56.741745+0000","last_fullsized":"2024-09-10T18:46:56.741745+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-10T18:46:25.845070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:25.845070+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:50.370600+0000","scrub_duration":0.0020129710000000001,"objects_trimmed":0,"snaptrim_duration":0.0013942640000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.741545+0000","last_change":"2024-09-10T18:46:29.081362+0000","last_active":"2024-09-10T18:46:56.741545+0000","last_peered":"2024-09-10T18:46:56.741545+0000","last_clean":"2024-09-10T18:46:56.741545+0000","last_became_active":"2024-09-10T18:45:17.965163+0000","last_became_peered":"2024-09-10T18:45:17.965163+0000","last_unstale":"2024-09-10T18:46:56.741545+0000","last_undegraded":"2024-09-10T18:46:56.741545+0000","last_fullsized":"2024-09-10T18:46:56.741545+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:29.081296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:29.081296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:50.659900+0000","scrub_duration":0.0015488590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'81","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:05.558102+0000","last_change":"2024-09-10T18:47:05.558031+0000","last_active":"2024-09-10T18:47:05.558102+0000","last_peered":"2024-09-10T18:47:05.558102+0000","last_clean":"2024-09-10T18:47:05.558102+0000","last_became_active":"2024-09-10T18:45:07.433592+0000","last_became_peered":"2024-09-10T18:45:07.433592+0000","last_unstale":"2024-09-10T18:47:05.558102+0000","last_undegraded":"2024-09-10T18:47:05.558102+0000","last_fullsized":"2024-09-10T18:47:05.558102+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:47:05.557968+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:47:05.557968+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:22.825675+0000","scrub_duration":0.0015062890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.744069+0000","last_change":"2024-09-10T18:46:26.687029+0000","last_active":"2024-09-10T18:46:56.744069+0000","last_peered":"2024-09-10T18:46:56.744069+0000","last_clean":"2024-09-10T18:46:56.744069+0000","last_became_active":"2024-09-10T18:45:22.068226+0000","last_became_peered":"2024-09-10T18:45:22.068226+0000","last_unstale":"2024-09-10T18:46:56.744069+0000","last_undegraded":"2024-09-10T18:46:56.744069+0000","last_fullsized":"2024-09-10T18:46:56.744069+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:26.686961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:26.686961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:32.456741+0000","scrub_duration":0.0015985109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.741707+0000","last_change":"2024-09-10T18:46:56.066558+0000","last_active":"2024-09-10T18:46:56.741707+0000","last_peered":"2024-09-10T18:46:56.741707+0000","last_clean":"2024-09-10T18:46:56.741707+0000","last_became_active":"2024-09-10T18:45:24.073327+0000","last_became_peered":"2024-09-10T18:45:24.073327+0000","last_unstale":"2024-09-10T18:46:56.741707+0000","last_undegraded":"2024-09-10T18:46:56.741707+0000","last_fullsized":"2024-09-10T18:46:56.741707+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:56.066502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:56.066502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:06.081016+0000","scrub_duration":0.0011960810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"63'19","reported_seq":76,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.737490+0000","last_change":"2024-09-10T18:46:28.229804+0000","last_active":"2024-09-10T18:46:56.737490+0000","last_peered":"2024-09-10T18:46:56.737490+0000","last_clean":"2024-09-10T18:46:56.737490+0000","last_became_active":"2024-09-10T18:45:20.421097+0000","last_became_peered":"2024-09-10T18:45:20.421097+0000","last_unstale":"2024-09-10T18:46:56.737490+0000","last_undegraded":"2024-09-10T18:46:56.737490+0000","last_fullsized":"2024-09-10T18:46:56.737490+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:46:28.229737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:28.229737+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:49.859218+0000","scrub_duration":0.002308184,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.739894+0000","last_change":"2024-09-10T18:46:53.707765+0000","last_active":"2024-09-10T18:46:56.739894+0000","last_peered":"2024-09-10T18:46:56.739894+0000","last_clean":"2024-09-10T18:46:56.739894+0000","last_became_active":"2024-09-10T18:45:17.959924+0000","last_became_peered":"2024-09-10T18:45:17.959924+0000","last_unstale":"2024-09-10T18:46:56.739894+0000","last_undegraded":"2024-09-10T18:46:56.739894+0000","last_fullsized":"2024-09-10T18:46:56.739894+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:53.707694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:53.707694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:22.214797+0000","scrub_duration":0.0020804069999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.739489+0000","last_change":"2024-09-10T18:46:30.781786+0000","last_active":"2024-09-10T18:46:56.739489+0000","last_peered":"2024-09-10T18:46:56.739489+0000","last_clean":"2024-09-10T18:46:56.739489+0000","last_became_active":"2024-09-10T18:45:22.072581+0000","last_became_peered":"2024-09-10T18:45:22.072581+0000","last_unstale":"2024-09-10T18:46:56.739489+0000","last_undegraded":"2024-09-10T18:46:56.739489+0000","last_fullsized":"2024-09-10T18:46:56.739489+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:30.781730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:30.781730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:38.248319+0000","scrub_duration":0.0020255870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:03.587369+0000","last_change":"2024-09-10T18:47:03.587334+0000","last_active":"2024-09-10T18:47:03.587369+0000","last_peered":"2024-09-10T18:47:03.587369+0000","last_clean":"2024-09-10T18:47:03.587369+0000","last_became_active":"2024-09-10T18:45:24.076392+0000","last_became_peered":"2024-09-10T18:45:24.076392+0000","last_unstale":"2024-09-10T18:47:03.587369+0000","last_undegraded":"2024-09-10T18:47:03.587369+0000","last_fullsized":"2024-09-10T18:47:03.587369+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:03.587252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:03.587252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:29.976291+0000","scrub_duration":0.001684867,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"63'9","reported_seq":61,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.738269+0000","last_change":"2024-09-10T18:46:51.486491+0000","last_active":"2024-09-10T18:46:56.738269+0000","last_peered":"2024-09-10T18:46:56.738269+0000","last_clean":"2024-09-10T18:46:56.738269+0000","last_became_active":"2024-09-10T18:45:20.061021+0000","last_became_peered":"2024-09-10T18:45:20.061021+0000","last_unstale":"2024-09-10T18:46:56.738269+0000","last_undegraded":"2024-09-10T18:46:56.738269+0000","last_fullsized":"2024-09-10T18:46:56.738269+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:46:51.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:51.486428+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:05.602014+0000","scrub_duration":0.0019382830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:02.496360+0000","last_change":"2024-09-10T18:47:02.496271+0000","last_active":"2024-09-10T18:47:02.496360+0000","last_peered":"2024-09-10T18:47:02.496360+0000","last_clean":"2024-09-10T18:47:02.496360+0000","last_became_active":"2024-09-10T18:45:17.963057+0000","last_became_peered":"2024-09-10T18:45:17.963057+0000","last_unstale":"2024-09-10T18:47:02.496360+0000","last_undegraded":"2024-09-10T18:47:02.496360+0000","last_fullsized":"2024-09-10T18:47:02.496360+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:02.496142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:47:02.496142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:10.459026+0000","scrub_duration":0.001628399,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.739427+0000","last_change":"2024-09-10T18:46:55.684302+0000","last_active":"2024-09-10T18:46:56.739427+0000","last_peered":"2024-09-10T18:46:56.739427+0000","last_clean":"2024-09-10T18:46:56.739427+0000","last_became_active":"2024-09-10T18:45:22.070343+0000","last_became_peered":"2024-09-10T18:45:22.070343+0000","last_unstale":"2024-09-10T18:46:56.739427+0000","last_undegraded":"2024-09-10T18:46:56.739427+0000","last_fullsized":"2024-09-10T18:46:56.739427+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_clean_scrub_stamp":"2024-09-10T18:46:55.684250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:59.597477+0000","scrub_duration":0.0014319199999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.737329+0000","last_change":"2024-09-10T18:46:47.545406+0000","last_active":"2024-09-10T18:46:56.737329+0000","last_peered":"2024-09-10T18:46:56.737329+0000","last_clean":"2024-09-10T18:46:56.737329+0000","last_became_active":"2024-09-10T18:45:24.078529+0000","last_became_peered":"2024-09-10T18:45:24.078529+0000","last_unstale":"2024-09-10T18:46:56.737329+0000","last_undegraded":"2024-09-10T18:46:56.737329+0000","last_fullsized":"2024-09-10T18:46:56.737329+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:47.545343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:47.545343+0000","last_clean_scrub_stamp":"2024-09-10T18:46:47.545343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:02.171974+0000","scrub_duration":0.0012977799999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"63'15","reported_seq":70,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.740597+0000","last_change":"2024-09-10T18:46:51.114543+0000","last_active":"2024-09-10T18:46:56.740597+0000","last_peered":"2024-09-10T18:46:56.740597+0000","last_clean":"2024-09-10T18:46:56.740597+0000","last_became_active":"2024-09-10T18:45:20.419804+0000","last_became_peered":"2024-09-10T18:45:20.419804+0000","last_unstale":"2024-09-10T18:46:56.740597+0000","last_undegraded":"2024-09-10T18:46:56.740597+0000","last_fullsized":"2024-09-10T18:46:56.740597+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:46:51.114455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:51.114455+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:18.982344+0000","scrub_duration":0.0022557979999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.740581+0000","last_change":"2024-09-10T18:46:17.819966+0000","last_active":"2024-09-10T18:46:56.740581+0000","last_peered":"2024-09-10T18:46:56.740581+0000","last_clean":"2024-09-10T18:46:56.740581+0000","last_became_active":"2024-09-10T18:45:17.959653+0000","last_became_peered":"2024-09-10T18:45:17.959653+0000","last_unstale":"2024-09-10T18:46:56.740581+0000","last_undegraded":"2024-09-10T18:46:56.740581+0000","last_fullsized":"2024-09-10T18:46:56.740581+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:17.819909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:17.819909+0000","last_clean_scrub_stamp":"2024-09-10T18:46:17.819909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:18.454145+0000","scrub_duration":0.001612109,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"83'9","reported_seq":77,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:06.096666+0000","last_change":"2024-09-10T18:47:02.582283+0000","last_active":"2024-09-10T18:47:06.096666+0000","last_peered":"2024-09-10T18:47:06.096666+0000","last_clean":"2024-09-10T18:47:06.096666+0000","last_became_active":"2024-09-10T18:45:22.073383+0000","last_became_peered":"2024-09-10T18:45:22.073383+0000","last_unstale":"2024-09-10T18:47:06.096666+0000","last_undegraded":"2024-09-10T18:47:06.096666+0000","last_fullsized":"2024-09-10T18:47:06.096666+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'9","last_scrub_stamp":"2024-09-10T18:47:02.582229+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-09-10T18:47:02.582229+0000","last_clean_scrub_stamp":"2024-09-10T18:47:02.582229+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:10.931107+0000","scrub_duration":0.0020623740000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.740874+0000","last_change":"2024-09-10T18:46:45.083522+0000","last_active":"2024-09-10T18:46:56.740874+0000","last_peered":"2024-09-10T18:46:56.740874+0000","last_clean":"2024-09-10T18:46:56.740874+0000","last_became_active":"2024-09-10T18:45:24.265256+0000","last_became_peered":"2024-09-10T18:45:24.265256+0000","last_unstale":"2024-09-10T18:46:56.740874+0000","last_undegraded":"2024-09-10T18:46:56.740874+0000","last_fullsized":"2024-09-10T18:46:56.740874+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_clean_scrub_stamp":"2024-09-10T18:46:45.083425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:08.177993+0000","scrub_duration":0.0017589890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"63'12","reported_seq":68,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.739032+0000","last_change":"2024-09-10T18:46:30.320527+0000","last_active":"2024-09-10T18:46:56.739032+0000","last_peered":"2024-09-10T18:46:56.739032+0000","last_clean":"2024-09-10T18:46:56.739032+0000","last_became_active":"2024-09-10T18:45:20.056869+0000","last_became_peered":"2024-09-10T18:45:20.056869+0000","last_unstale":"2024-09-10T18:46:56.739032+0000","last_undegraded":"2024-09-10T18:46:56.739032+0000","last_fullsized":"2024-09-10T18:46:56.739032+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:46:30.320462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:30.320462+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:35.180189+0000","scrub_duration":0.0028822850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"56'2","reported_seq":64,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.740336+0000","last_change":"2024-09-10T18:46:42.303648+0000","last_active":"2024-09-10T18:46:56.740336+0000","last_peered":"2024-09-10T18:46:56.740336+0000","last_clean":"2024-09-10T18:46:56.740336+0000","last_became_active":"2024-09-10T18:45:17.963286+0000","last_became_peered":"2024-09-10T18:45:17.963286+0000","last_unstale":"2024-09-10T18:46:56.740336+0000","last_undegraded":"2024-09-10T18:46:56.740336+0000","last_fullsized":"2024-09-10T18:46:56.740336+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-10T18:46:42.303567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:42.303567+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:05.463057+0000","scrub_duration":0.0020684750000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.739774+0000","last_change":"2024-09-10T18:46:44.754345+0000","last_active":"2024-09-10T18:46:56.739774+0000","last_peered":"2024-09-10T18:46:56.739774+0000","last_clean":"2024-09-10T18:46:56.739774+0000","last_became_active":"2024-09-10T18:45:22.068957+0000","last_became_peered":"2024-09-10T18:45:22.068957+0000","last_unstale":"2024-09-10T18:46:56.739774+0000","last_undegraded":"2024-09-10T18:46:56.739774+0000","last_fullsized":"2024-09-10T18:46:56.739774+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:44.754286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:44.754286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:47.893148+0000","scrub_duration":0.001258386,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.741014+0000","last_change":"2024-09-10T18:45:24.078134+0000","last_active":"2024-09-10T18:46:56.741014+0000","last_peered":"2024-09-10T18:46:56.741014+0000","last_clean":"2024-09-10T18:46:56.741014+0000","last_became_active":"2024-09-10T18:45:24.077815+0000","last_became_peered":"2024-09-10T18:45:24.077815+0000","last_unstale":"2024-09-10T18:46:56.741014+0000","last_undegraded":"2024-09-10T18:46:56.741014+0000","last_fullsized":"2024-09-10T18:46:56.741014+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"76'13","reported_seq":65,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.738910+0000","last_change":"2024-09-10T18:46:24.626912+0000","last_active":"2024-09-10T18:46:56.738910+0000","last_peered":"2024-09-10T18:46:56.738910+0000","last_clean":"2024-09-10T18:46:56.738910+0000","last_became_active":"2024-09-10T18:45:20.420421+0000","last_became_peered":"2024-09-10T18:45:20.420421+0000","last_unstale":"2024-09-10T18:46:56.738910+0000","last_undegraded":"2024-09-10T18:46:56.738910+0000","last_fullsized":"2024-09-10T18:46:56.738910+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'13","last_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_deep_scrub":"76'13","last_deep_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_clean_scrub_stamp":"2024-09-10T18:46:24.626859+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:41.087073+0000","scrub_duration":0.0035642949999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.739906+0000","last_change":"2024-09-10T18:46:31.734522+0000","last_active":"2024-09-10T18:46:56.739906+0000","last_peered":"2024-09-10T18:46:56.739906+0000","last_clean":"2024-09-10T18:46:56.739906+0000","last_became_active":"2024-09-10T18:45:17.965112+0000","last_became_peered":"2024-09-10T18:45:17.965112+0000","last_unstale":"2024-09-10T18:46:56.739906+0000","last_undegraded":"2024-09-10T18:46:56.739906+0000","last_fullsized":"2024-09-10T18:46:56.739906+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:31.734459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:31.734459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:41.550267+0000","scrub_duration":0.0015571960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.741554+0000","last_change":"2024-09-10T18:46:32.829583+0000","last_active":"2024-09-10T18:46:56.741554+0000","last_peered":"2024-09-10T18:46:56.741554+0000","last_clean":"2024-09-10T18:46:56.741554+0000","last_became_active":"2024-09-10T18:45:22.069040+0000","last_became_peered":"2024-09-10T18:45:22.069040+0000","last_unstale":"2024-09-10T18:46:56.741554+0000","last_undegraded":"2024-09-10T18:46:56.741554+0000","last_fullsized":"2024-09-10T18:46:56.741554+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:32.829508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:32.829508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:45.869787+0000","scrub_duration":0.001897239,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"63'1","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.739420+0000","last_change":"2024-09-10T18:46:56.721773+0000","last_active":"2024-09-10T18:46:56.739420+0000","last_peered":"2024-09-10T18:46:56.739420+0000","last_clean":"2024-09-10T18:46:56.739420+0000","last_became_active":"2024-09-10T18:45:24.079258+0000","last_became_peered":"2024-09-10T18:45:24.079258+0000","last_unstale":"2024-09-10T18:46:56.739420+0000","last_undegraded":"2024-09-10T18:46:56.739420+0000","last_fullsized":"2024-09-10T18:46:56.739420+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-10T18:46:56.721716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:56.721716+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:04.127384+0000","scrub_duration":0.0014420559999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.737428+0000","last_change":"2024-09-10T18:46:48.227450+0000","last_active":"2024-09-10T18:46:56.737428+0000","last_peered":"2024-09-10T18:46:56.737428+0000","last_clean":"2024-09-10T18:46:56.737428+0000","last_became_active":"2024-09-10T18:45:17.964705+0000","last_became_peered":"2024-09-10T18:45:17.964705+0000","last_unstale":"2024-09-10T18:46:56.737428+0000","last_undegraded":"2024-09-10T18:46:56.737428+0000","last_fullsized":"2024-09-10T18:46:56.737428+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:48.227378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:48.227378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:54.333207+0000","scrub_duration":0.0015447519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'6","reported_seq":54,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:02.700920+0000","last_change":"2024-09-10T18:47:02.700853+0000","last_active":"2024-09-10T18:47:02.700920+0000","last_peered":"2024-09-10T18:47:02.700920+0000","last_clean":"2024-09-10T18:47:02.700920+0000","last_became_active":"2024-09-10T18:45:20.421746+0000","last_became_peered":"2024-09-10T18:45:20.421746+0000","last_unstale":"2024-09-10T18:47:02.700920+0000","last_undegraded":"2024-09-10T18:47:02.700920+0000","last_fullsized":"2024-09-10T18:47:02.700920+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'6","last_scrub_stamp":"2024-09-10T18:47:02.700787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:02.700787+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:10.576660+0000","scrub_duration":0.0017032519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.741330+0000","last_change":"2024-09-10T18:46:39.084489+0000","last_active":"2024-09-10T18:46:56.741330+0000","last_peered":"2024-09-10T18:46:56.741330+0000","last_clean":"2024-09-10T18:46:56.741330+0000","last_became_active":"2024-09-10T18:45:22.069653+0000","last_became_peered":"2024-09-10T18:45:22.069653+0000","last_unstale":"2024-09-10T18:46:56.741330+0000","last_undegraded":"2024-09-10T18:46:56.741330+0000","last_fullsized":"2024-09-10T18:46:56.741330+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:39.084433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:39.084433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:05.431757+0000","scrub_duration":0.0014845819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.743930+0000","last_change":"2024-09-10T18:46:52.611074+0000","last_active":"2024-09-10T18:46:56.743930+0000","last_peered":"2024-09-10T18:46:56.743930+0000","last_clean":"2024-09-10T18:46:56.743930+0000","last_became_active":"2024-09-10T18:45:24.264611+0000","last_became_peered":"2024-09-10T18:45:24.264611+0000","last_unstale":"2024-09-10T18:46:56.743930+0000","last_undegraded":"2024-09-10T18:46:56.743930+0000","last_fullsized":"2024-09-10T18:46:56.743930+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:52.611003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:52.611003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:04.517442+0000","scrub_duration":0.0015558849999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"63'10","reported_seq":82,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.759869+0000","last_change":"2024-09-10T18:46:56.759636+0000","last_active":"2024-09-10T18:46:56.759869+0000","last_peered":"2024-09-10T18:46:56.759869+0000","last_clean":"2024-09-10T18:46:56.759869+0000","last_became_active":"2024-09-10T18:46:56.754800+0000","last_became_peered":"2024-09-10T18:46:56.754800+0000","last_unstale":"2024-09-10T18:46:56.759869+0000","last_undegraded":"2024-09-10T18:46:56.759869+0000","last_fullsized":"2024-09-10T18:46:56.759869+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:46:50.722316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.722316+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:55.786523+0000","scrub_duration":0.0022194070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.741035+0000","last_change":"2024-09-10T18:46:50.093435+0000","last_active":"2024-09-10T18:46:56.741035+0000","last_peered":"2024-09-10T18:46:56.741035+0000","last_clean":"2024-09-10T18:46:56.741035+0000","last_became_active":"2024-09-10T18:45:17.964870+0000","last_became_peered":"2024-09-10T18:45:17.964870+0000","last_unstale":"2024-09-10T18:46:56.741035+0000","last_undegraded":"2024-09-10T18:46:56.741035+0000","last_fullsized":"2024-09-10T18:46:56.741035+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:50.093372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.093372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:55.991532+0000","scrub_duration":0.0015580539999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"63'8","reported_seq":73,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:06.096655+0000","last_change":"2024-09-10T18:46:34.085224+0000","last_active":"2024-09-10T18:47:06.096655+0000","last_peered":"2024-09-10T18:47:06.096655+0000","last_clean":"2024-09-10T18:47:06.096655+0000","last_became_active":"2024-09-10T18:45:22.062940+0000","last_became_peered":"2024-09-10T18:45:22.062940+0000","last_unstale":"2024-09-10T18:47:06.096655+0000","last_undegraded":"2024-09-10T18:47:06.096655+0000","last_fullsized":"2024-09-10T18:47:06.096655+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:34.085156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:34.085156+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:54.777061+0000","scrub_duration":0.0020843120000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.738627+0000","last_change":"2024-09-10T18:45:24.074039+0000","last_active":"2024-09-10T18:46:56.738627+0000","last_peered":"2024-09-10T18:46:56.738627+0000","last_clean":"2024-09-10T18:46:56.738627+0000","last_became_active":"2024-09-10T18:45:24.073736+0000","last_became_peered":"2024-09-10T18:45:24.073736+0000","last_unstale":"2024-09-10T18:46:56.738627+0000","last_undegraded":"2024-09-10T18:46:56.738627+0000","last_fullsized":"2024-09-10T18:46:56.738627+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:45:23.053377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"63'9","reported_seq":61,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.741201+0000","last_change":"2024-09-10T18:46:23.086274+0000","last_active":"2024-09-10T18:46:56.741201+0000","last_peered":"2024-09-10T18:46:56.741201+0000","last_clean":"2024-09-10T18:46:56.741201+0000","last_became_active":"2024-09-10T18:45:20.421124+0000","last_became_peered":"2024-09-10T18:45:20.421124+0000","last_unstale":"2024-09-10T18:46:56.741201+0000","last_undegraded":"2024-09-10T18:46:56.741201+0000","last_fullsized":"2024-09-10T18:46:56.741201+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:46:23.086203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:23.086203+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:44.793257+0000","scrub_duration":0.002168068,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.738426+0000","last_change":"2024-09-10T18:46:45.541882+0000","last_active":"2024-09-10T18:46:56.738426+0000","last_peered":"2024-09-10T18:46:56.738426+0000","last_clean":"2024-09-10T18:46:56.738426+0000","last_became_active":"2024-09-10T18:45:17.960106+0000","last_became_peered":"2024-09-10T18:45:17.960106+0000","last_unstale":"2024-09-10T18:46:56.738426+0000","last_undegraded":"2024-09-10T18:46:56.738426+0000","last_fullsized":"2024-09-10T18:46:56.738426+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:45.541819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:45.541819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:15.015402+0000","scrub_duration":0.0015180809999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"81'9","reported_seq":74,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:06.096798+0000","last_change":"2024-09-10T18:46:54.800878+0000","last_active":"2024-09-10T18:47:06.096798+0000","last_peered":"2024-09-10T18:47:06.096798+0000","last_clean":"2024-09-10T18:47:06.096798+0000","last_became_active":"2024-09-10T18:45:22.075368+0000","last_became_peered":"2024-09-10T18:45:22.075368+0000","last_unstale":"2024-09-10T18:47:06.096798+0000","last_undegraded":"2024-09-10T18:47:06.096798+0000","last_fullsized":"2024-09-10T18:47:06.096798+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"81'9","last_scrub_stamp":"2024-09-10T18:46:54.800790+0000","last_deep_scrub":"81'9","last_deep_scrub_stamp":"2024-09-10T18:46:54.800790+0000","last_clean_scrub_stamp":"2024-09-10T18:46:54.800790+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:14.444052+0000","scrub_duration":0.0034208709999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.740116+0000","last_change":"2024-09-10T18:46:36.305262+0000","last_active":"2024-09-10T18:46:56.740116+0000","last_peered":"2024-09-10T18:46:56.740116+0000","last_clean":"2024-09-10T18:46:56.740116+0000","last_became_active":"2024-09-10T18:45:24.080196+0000","last_became_peered":"2024-09-10T18:45:24.080196+0000","last_unstale":"2024-09-10T18:46:56.740116+0000","last_undegraded":"2024-09-10T18:46:56.740116+0000","last_fullsized":"2024-09-10T18:46:56.740116+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_clean_scrub_stamp":"2024-09-10T18:46:36.305199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:04.747507+0000","scrub_duration":0.0013794809999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"63'9","reported_seq":61,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:59.469301+0000","last_change":"2024-09-10T18:46:59.469257+0000","last_active":"2024-09-10T18:46:59.469301+0000","last_peered":"2024-09-10T18:46:59.469301+0000","last_clean":"2024-09-10T18:46:59.469301+0000","last_became_active":"2024-09-10T18:45:20.057479+0000","last_became_peered":"2024-09-10T18:45:20.057479+0000","last_unstale":"2024-09-10T18:46:59.469301+0000","last_undegraded":"2024-09-10T18:46:59.469301+0000","last_fullsized":"2024-09-10T18:46:59.469301+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:46:59.469198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:59.469198+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:26.476613+0000","scrub_duration":0.0022009489999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.737659+0000","last_change":"2024-09-10T18:46:23.292276+0000","last_active":"2024-09-10T18:46:56.737659+0000","last_peered":"2024-09-10T18:46:56.737659+0000","last_clean":"2024-09-10T18:46:56.737659+0000","last_became_active":"2024-09-10T18:45:17.962009+0000","last_became_peered":"2024-09-10T18:45:17.962009+0000","last_unstale":"2024-09-10T18:46:56.737659+0000","last_undegraded":"2024-09-10T18:46:56.737659+0000","last_fullsized":"2024-09-10T18:46:56.737659+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:23.292207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:23.292207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:27.416585+0000","scrub_duration":0.001659953,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.741071+0000","last_change":"2024-09-10T18:45:22.073186+0000","last_active":"2024-09-10T18:46:56.741071+0000","last_peered":"2024-09-10T18:46:56.741071+0000","last_clean":"2024-09-10T18:46:56.741071+0000","last_became_active":"2024-09-10T18:45:22.072877+0000","last_became_peered":"2024-09-10T18:45:22.072877+0000","last_unstale":"2024-09-10T18:46:56.741071+0000","last_undegraded":"2024-09-10T18:46:56.741071+0000","last_fullsized":"2024-09-10T18:46:56.741071+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.736857+0000","last_change":"2024-09-10T18:46:50.499866+0000","last_active":"2024-09-10T18:46:56.736857+0000","last_peered":"2024-09-10T18:46:56.736857+0000","last_clean":"2024-09-10T18:46:56.736857+0000","last_became_active":"2024-09-10T18:45:24.081050+0000","last_became_peered":"2024-09-10T18:45:24.081050+0000","last_unstale":"2024-09-10T18:46:56.736857+0000","last_undegraded":"2024-09-10T18:46:56.736857+0000","last_fullsized":"2024-09-10T18:46:56.736857+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:50.499815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.499815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:13.479025+0000","scrub_duration":0.001285432,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"63'11","reported_seq":54,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.739517+0000","last_change":"2024-09-10T18:45:20.057563+0000","last_active":"2024-09-10T18:46:56.739517+0000","last_peered":"2024-09-10T18:46:56.739517+0000","last_clean":"2024-09-10T18:46:56.739517+0000","last_became_active":"2024-09-10T18:45:20.056885+0000","last_became_peered":"2024-09-10T18:45:20.056885+0000","last_unstale":"2024-09-10T18:46:56.739517+0000","last_undegraded":"2024-09-10T18:46:56.739517+0000","last_fullsized":"2024-09-10T18:46:56.739517+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:45:19.033249+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.740520+0000","last_change":"2024-09-10T18:46:36.402610+0000","last_active":"2024-09-10T18:46:56.740520+0000","last_peered":"2024-09-10T18:46:56.740520+0000","last_clean":"2024-09-10T18:46:56.740520+0000","last_became_active":"2024-09-10T18:45:17.963542+0000","last_became_peered":"2024-09-10T18:45:17.963542+0000","last_unstale":"2024-09-10T18:46:56.740520+0000","last_undegraded":"2024-09-10T18:46:56.740520+0000","last_fullsized":"2024-09-10T18:46:56.740520+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:36.402548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:36.402548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:04.495178+0000","scrub_duration":0.0014103919999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.737829+0000","last_change":"2024-09-10T18:46:37.414794+0000","last_active":"2024-09-10T18:46:56.737829+0000","last_peered":"2024-09-10T18:46:56.737829+0000","last_clean":"2024-09-10T18:46:56.737829+0000","last_became_active":"2024-09-10T18:45:22.070065+0000","last_became_peered":"2024-09-10T18:45:22.070065+0000","last_unstale":"2024-09-10T18:46:56.737829+0000","last_undegraded":"2024-09-10T18:46:56.737829+0000","last_fullsized":"2024-09-10T18:46:56.737829+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:37.414734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:37.414734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:44.027335+0000","scrub_duration":0.001607996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:05.789930+0000","last_change":"2024-09-10T18:47:05.789893+0000","last_active":"2024-09-10T18:47:05.789930+0000","last_peered":"2024-09-10T18:47:05.789930+0000","last_clean":"2024-09-10T18:47:05.789930+0000","last_became_active":"2024-09-10T18:45:24.072224+0000","last_became_peered":"2024-09-10T18:45:24.072224+0000","last_unstale":"2024-09-10T18:47:05.789930+0000","last_undegraded":"2024-09-10T18:47:05.789930+0000","last_fullsized":"2024-09-10T18:47:05.789930+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:05.789830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:05.789830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:28.845106+0000","scrub_duration":0.001271938,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"63'4","reported_seq":51,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.741611+0000","last_change":"2024-09-10T18:46:46.766906+0000","last_active":"2024-09-10T18:46:56.741611+0000","last_peered":"2024-09-10T18:46:56.741611+0000","last_clean":"2024-09-10T18:46:56.741611+0000","last_became_active":"2024-09-10T18:45:20.059205+0000","last_became_peered":"2024-09-10T18:45:20.059205+0000","last_unstale":"2024-09-10T18:46:56.741611+0000","last_undegraded":"2024-09-10T18:46:56.741611+0000","last_fullsized":"2024-09-10T18:46:56.741611+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4","last_scrub_stamp":"2024-09-10T18:46:46.766766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:46.766766+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:49.051695+0000","scrub_duration":0.0021847250000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.741394+0000","last_change":"2024-09-10T18:46:54.082828+0000","last_active":"2024-09-10T18:46:56.741394+0000","last_peered":"2024-09-10T18:46:56.741394+0000","last_clean":"2024-09-10T18:46:56.741394+0000","last_became_active":"2024-09-10T18:45:17.963630+0000","last_became_peered":"2024-09-10T18:45:17.963630+0000","last_unstale":"2024-09-10T18:46:56.741394+0000","last_undegraded":"2024-09-10T18:46:56.741394+0000","last_fullsized":"2024-09-10T18:46:56.741394+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:54.082761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:54.082761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:05.974208+0000","scrub_duration":0.001210458,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.737294+0000","last_change":"2024-09-10T18:46:35.172936+0000","last_active":"2024-09-10T18:46:56.737294+0000","last_peered":"2024-09-10T18:46:56.737294+0000","last_clean":"2024-09-10T18:46:56.737294+0000","last_became_active":"2024-09-10T18:45:22.072377+0000","last_became_peered":"2024-09-10T18:45:22.072377+0000","last_unstale":"2024-09-10T18:46:56.737294+0000","last_undegraded":"2024-09-10T18:46:56.737294+0000","last_fullsized":"2024-09-10T18:46:56.737294+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_clean_scrub_stamp":"2024-09-10T18:46:35.172861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:57.638389+0000","scrub_duration":0.001552933,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.744151+0000","last_change":"2024-09-10T18:46:53.596450+0000","last_active":"2024-09-10T18:46:56.744151+0000","last_peered":"2024-09-10T18:46:56.744151+0000","last_clean":"2024-09-10T18:46:56.744151+0000","last_became_active":"2024-09-10T18:45:24.077398+0000","last_became_peered":"2024-09-10T18:45:24.077398+0000","last_unstale":"2024-09-10T18:46:56.744151+0000","last_undegraded":"2024-09-10T18:46:56.744151+0000","last_fullsized":"2024-09-10T18:46:56.744151+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:53.596380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:53.596380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:54.667444+0000","scrub_duration":0.0020285519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"79'16","reported_seq":69,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.740952+0000","last_change":"2024-09-10T18:46:48.744340+0000","last_active":"2024-09-10T18:46:56.740952+0000","last_peered":"2024-09-10T18:46:56.740952+0000","last_clean":"2024-09-10T18:46:56.740952+0000","last_became_active":"2024-09-10T18:45:20.421479+0000","last_became_peered":"2024-09-10T18:45:20.421479+0000","last_unstale":"2024-09-10T18:46:56.740952+0000","last_undegraded":"2024-09-10T18:46:56.740952+0000","last_fullsized":"2024-09-10T18:46:56.740952+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"79'16","last_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_deep_scrub":"79'16","last_deep_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_clean_scrub_stamp":"2024-09-10T18:46:48.744287+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:57.831206+0000","scrub_duration":0.0044879259999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"56'1","reported_seq":54,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.737939+0000","last_change":"2024-09-10T18:46:41.147533+0000","last_active":"2024-09-10T18:46:56.737939+0000","last_peered":"2024-09-10T18:46:56.737939+0000","last_clean":"2024-09-10T18:46:56.737939+0000","last_became_active":"2024-09-10T18:45:17.961222+0000","last_became_peered":"2024-09-10T18:45:17.961222+0000","last_unstale":"2024-09-10T18:46:56.737939+0000","last_undegraded":"2024-09-10T18:46:56.737939+0000","last_fullsized":"2024-09-10T18:46:56.737939+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:46:41.147463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:41.147463+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:58.016926+0000","scrub_duration":0.0016797889999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.743816+0000","last_change":"2024-09-10T18:46:27.667088+0000","last_active":"2024-09-10T18:46:56.743816+0000","last_peered":"2024-09-10T18:46:56.743816+0000","last_clean":"2024-09-10T18:46:56.743816+0000","last_became_active":"2024-09-10T18:45:22.072312+0000","last_became_peered":"2024-09-10T18:45:22.072312+0000","last_unstale":"2024-09-10T18:46:56.743816+0000","last_undegraded":"2024-09-10T18:46:56.743816+0000","last_fullsized":"2024-09-10T18:46:56.743816+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:27.667018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:27.667018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:38.065664+0000","scrub_duration":0.001388886,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.740937+0000","last_change":"2024-09-10T18:46:29.791617+0000","last_active":"2024-09-10T18:46:56.740937+0000","last_peered":"2024-09-10T18:46:56.740937+0000","last_clean":"2024-09-10T18:46:56.740937+0000","last_became_active":"2024-09-10T18:45:24.264818+0000","last_became_peered":"2024-09-10T18:45:24.264818+0000","last_unstale":"2024-09-10T18:46:56.740937+0000","last_undegraded":"2024-09-10T18:46:56.740937+0000","last_fullsized":"2024-09-10T18:46:56.740937+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:29.791563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:29.791563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:37.941643+0000","scrub_duration":0.0021031819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"78'2","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.743998+0000","last_change":"2024-09-10T18:46:37.581201+0000","last_active":"2024-09-10T18:46:56.743998+0000","last_peered":"2024-09-10T18:46:56.743998+0000","last_clean":"2024-09-10T18:46:56.743998+0000","last_became_active":"2024-09-10T18:45:24.076380+0000","last_became_peered":"2024-09-10T18:45:24.076380+0000","last_unstale":"2024-09-10T18:46:56.743998+0000","last_undegraded":"2024-09-10T18:46:56.743998+0000","last_fullsized":"2024-09-10T18:46:56.743998+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'2","last_scrub_stamp":"2024-09-10T18:46:37.581130+0000","last_deep_scrub":"78'2","last_deep_scrub_stamp":"2024-09-10T18:46:37.581130+0000","last_clean_scrub_stamp":"2024-09-10T18:46:37.581130+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:55.887929+0000","scrub_duration":0.0024563609999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.740646+0000","last_change":"2024-09-10T18:46:28.088657+0000","last_active":"2024-09-10T18:46:56.740646+0000","last_peered":"2024-09-10T18:46:56.740646+0000","last_clean":"2024-09-10T18:46:56.740646+0000","last_became_active":"2024-09-10T18:45:17.962108+0000","last_became_peered":"2024-09-10T18:45:17.962108+0000","last_unstale":"2024-09-10T18:46:56.740646+0000","last_undegraded":"2024-09-10T18:46:56.740646+0000","last_fullsized":"2024-09-10T18:46:56.740646+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:28.088588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:28.088588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:56.550863+0000","scrub_duration":0.00140508,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"63'10","reported_seq":60,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.738831+0000","last_change":"2024-09-10T18:46:49.494628+0000","last_active":"2024-09-10T18:46:56.738831+0000","last_peered":"2024-09-10T18:46:56.738831+0000","last_clean":"2024-09-10T18:46:56.738831+0000","last_became_active":"2024-09-10T18:45:20.421094+0000","last_became_peered":"2024-09-10T18:45:20.421094+0000","last_unstale":"2024-09-10T18:46:56.738831+0000","last_undegraded":"2024-09-10T18:46:56.738831+0000","last_fullsized":"2024-09-10T18:46:56.738831+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:46:49.494565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:49.494565+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:08.379972+0000","scrub_duration":0.0036379630000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":76,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:06.096632+0000","last_change":"2024-09-10T18:46:33.150919+0000","last_active":"2024-09-10T18:47:06.096632+0000","last_peered":"2024-09-10T18:47:06.096632+0000","last_clean":"2024-09-10T18:47:06.096632+0000","last_became_active":"2024-09-10T18:45:22.073281+0000","last_became_peered":"2024-09-10T18:45:22.073281+0000","last_unstale":"2024-09-10T18:47:06.096632+0000","last_undegraded":"2024-09-10T18:47:06.096632+0000","last_fullsized":"2024-09-10T18:47:06.096632+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:33.150852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:33.150852+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:51.299257+0000","scrub_duration":0.0017025339999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:47:03.475420+0000","last_change":"2024-09-10T18:47:03.475367+0000","last_active":"2024-09-10T18:47:03.475420+0000","last_peered":"2024-09-10T18:47:03.475420+0000","last_clean":"2024-09-10T18:47:03.475420+0000","last_became_active":"2024-09-10T18:45:24.082318+0000","last_became_peered":"2024-09-10T18:45:24.082318+0000","last_unstale":"2024-09-10T18:47:03.475420+0000","last_undegraded":"2024-09-10T18:47:03.475420+0000","last_fullsized":"2024-09-10T18:47:03.475420+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:03.475299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:03.475299+0000","last_clean_scrub_stamp":"2024-09-10T18:47:03.475299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:22.439695+0000","scrub_duration":0.001661945,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.741269+0000","last_change":"2024-09-10T18:46:44.694805+0000","last_active":"2024-09-10T18:46:56.741269+0000","last_peered":"2024-09-10T18:46:56.741269+0000","last_clean":"2024-09-10T18:46:56.741269+0000","last_became_active":"2024-09-10T18:45:17.965707+0000","last_became_peered":"2024-09-10T18:45:17.965707+0000","last_unstale":"2024-09-10T18:46:56.741269+0000","last_undegraded":"2024-09-10T18:46:56.741269+0000","last_fullsized":"2024-09-10T18:46:56.741269+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:44.694737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:44.694737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:03.417079+0000","scrub_duration":0.0016330400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"63'11","reported_seq":64,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.737922+0000","last_change":"2024-09-10T18:46:46.230915+0000","last_active":"2024-09-10T18:46:56.737922+0000","last_peered":"2024-09-10T18:46:56.737922+0000","last_clean":"2024-09-10T18:46:56.737922+0000","last_became_active":"2024-09-10T18:45:20.058717+0000","last_became_peered":"2024-09-10T18:45:20.058717+0000","last_unstale":"2024-09-10T18:46:56.737922+0000","last_undegraded":"2024-09-10T18:46:56.737922+0000","last_fullsized":"2024-09-10T18:46:56.737922+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:46:46.230819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:46.230819+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:54.186112+0000","scrub_duration":0.002545322,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.738753+0000","last_change":"2024-09-10T18:45:22.068991+0000","last_active":"2024-09-10T18:46:56.738753+0000","last_peered":"2024-09-10T18:46:56.738753+0000","last_clean":"2024-09-10T18:46:56.738753+0000","last_became_active":"2024-09-10T18:45:22.068676+0000","last_became_peered":"2024-09-10T18:45:22.068676+0000","last_unstale":"2024-09-10T18:46:56.738753+0000","last_undegraded":"2024-09-10T18:46:56.738753+0000","last_fullsized":"2024-09-10T18:46:56.738753+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:45:21.043052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.740097+0000","last_change":"2024-09-10T18:46:45.285074+0000","last_active":"2024-09-10T18:46:56.740097+0000","last_peered":"2024-09-10T18:46:56.740097+0000","last_clean":"2024-09-10T18:46:56.740097+0000","last_became_active":"2024-09-10T18:45:24.267251+0000","last_became_peered":"2024-09-10T18:45:24.267251+0000","last_unstale":"2024-09-10T18:46:56.740097+0000","last_undegraded":"2024-09-10T18:46:56.740097+0000","last_fullsized":"2024-09-10T18:46:56.740097+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:45.285015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:45.285015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:56.843562+0000","scrub_duration":0.001362142,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"56'1","reported_seq":56,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.737545+0000","last_change":"2024-09-10T18:46:42.139314+0000","last_active":"2024-09-10T18:46:56.737545+0000","last_peered":"2024-09-10T18:46:56.737545+0000","last_clean":"2024-09-10T18:46:56.737545+0000","last_became_active":"2024-09-10T18:45:17.966286+0000","last_became_peered":"2024-09-10T18:45:17.966286+0000","last_unstale":"2024-09-10T18:46:56.737545+0000","last_undegraded":"2024-09-10T18:46:56.737545+0000","last_fullsized":"2024-09-10T18:46:56.737545+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:46:42.139256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:42.139256+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:57.842540+0000","scrub_duration":0.0014138690000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"63'15","reported_seq":70,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.739756+0000","last_change":"2024-09-10T18:46:23.864133+0000","last_active":"2024-09-10T18:46:56.739756+0000","last_peered":"2024-09-10T18:46:56.739756+0000","last_clean":"2024-09-10T18:46:56.739756+0000","last_became_active":"2024-09-10T18:45:20.062728+0000","last_became_peered":"2024-09-10T18:45:20.062728+0000","last_unstale":"2024-09-10T18:46:56.739756+0000","last_undegraded":"2024-09-10T18:46:56.739756+0000","last_fullsized":"2024-09-10T18:46:56.739756+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:46:23.864075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:23.864075+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:29.106766+0000","scrub_duration":0.0020859780000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T18:47:09.520861+0000","last_change":"2024-09-10T18:47:09.520829+0000","last_active":"2024-09-10T18:47:09.520861+0000","last_peered":"2024-09-10T18:47:09.520861+0000","last_clean":"2024-09-10T18:47:09.520861+0000","last_became_active":"2024-09-10T18:45:22.059794+0000","last_became_peered":"2024-09-10T18:45:22.059794+0000","last_unstale":"2024-09-10T18:47:09.520861+0000","last_undegraded":"2024-09-10T18:47:09.520861+0000","last_fullsized":"2024-09-10T18:47:09.520861+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:09.520768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:47:09.520768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:33.788561+0000","scrub_duration":0.0016284120000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:57.811197+0000","last_change":"2024-09-10T18:46:57.811148+0000","last_active":"2024-09-10T18:46:57.811197+0000","last_peered":"2024-09-10T18:46:57.811197+0000","last_clean":"2024-09-10T18:46:57.811197+0000","last_became_active":"2024-09-10T18:45:24.264427+0000","last_became_peered":"2024-09-10T18:45:24.264427+0000","last_unstale":"2024-09-10T18:46:57.811197+0000","last_undegraded":"2024-09-10T18:46:57.811197+0000","last_fullsized":"2024-09-10T18:46:57.811197+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:57.811074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:57.811074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:12.950473+0000","scrub_duration":0.0013689419999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.741220+0000","last_change":"2024-09-10T18:46:47.734860+0000","last_active":"2024-09-10T18:46:56.741220+0000","last_peered":"2024-09-10T18:46:56.741220+0000","last_clean":"2024-09-10T18:46:56.741220+0000","last_became_active":"2024-09-10T18:45:17.963257+0000","last_became_peered":"2024-09-10T18:45:17.963257+0000","last_unstale":"2024-09-10T18:46:56.741220+0000","last_undegraded":"2024-09-10T18:46:56.741220+0000","last_fullsized":"2024-09-10T18:46:56.741220+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:47.734803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:47.734803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:05.847815+0000","scrub_duration":0.001381134,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"63'12","reported_seq":68,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.741702+0000","last_change":"2024-09-10T18:46:52.771640+0000","last_active":"2024-09-10T18:46:56.741702+0000","last_peered":"2024-09-10T18:46:56.741702+0000","last_clean":"2024-09-10T18:46:56.741702+0000","last_became_active":"2024-09-10T18:45:20.058391+0000","last_became_peered":"2024-09-10T18:45:20.058391+0000","last_unstale":"2024-09-10T18:46:56.741702+0000","last_undegraded":"2024-09-10T18:46:56.741702+0000","last_fullsized":"2024-09-10T18:46:56.741702+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:46:52.771570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:52.771570+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:04.251299+0000","scrub_duration":0.0021490089999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T18:46:56.739801+0000","last_change":"2024-09-10T18:46:34.283979+0000","last_active":"2024-09-10T18:46:56.739801+0000","last_peered":"2024-09-10T18:46:56.739801+0000","last_clean":"2024-09-10T18:46:56.739801+0000","last_became_active":"2024-09-10T18:45:22.073250+0000","last_became_peered":"2024-09-10T18:45:22.073250+0000","last_unstale":"2024-09-10T18:46:56.739801+0000","last_undegraded":"2024-09-10T18:46:56.739801+0000","last_fullsized":"2024-09-10T18:46:56.739801+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:34.283901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:34.283901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:47:34.686745+0000","scrub_duration":0.002360334,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":422,"ondisk_log_size":422,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":100,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299419,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298584,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446568,"statfs":{"total":95995035648,"available":95689285632,"internally_reserved":0,"allocated":1257472,"data_stored":895084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.50600000000000001,"15min":0.50700000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.63100000000000001,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.49299999999999999,"15min":0.48799999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.66600000000000004,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.63900000000000001}]},{"osd":1,"last update":"Tue Sep 10 18:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.48899999999999999,"15min":0.48099999999999998},"min":{"1min":0.316,"5min":0.314,"15min":0.314},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.51600000000000001,"15min":0.51800000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.51200000000000001}]},{"osd":2,"last update":"Tue Sep 10 18:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.48999999999999999,"15min":0.48699999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.65400000000000003,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.51700000000000002,"15min":0.51600000000000001},"min":{"1min":0.32400000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.64900000000000002,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.50700000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.505,"15min":0.50600000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.622,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.52100000000000002,"15min":0.51900000000000002},"min":{"1min":0.35399999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.61099999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.38800000000000001,"15min":0.38600000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.433,"5min":0.38700000000000001,"15min":0.379},"min":{"1min":0.32500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.42399999999999999}]},{"osd":5,"last update":"Tue Sep 10 18:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.41099999999999998,"15min":0.41199999999999998},"min":{"1min":0.29999999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.61899999999999999,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.42299999999999999,"15min":0.41799999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.372}]},{"osd":6,"last update":"Tue Sep 10 18:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.45300000000000001,"15min":0.44900000000000001},"min":{"1min":0.32600000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.44,"15min":0.435},"min":{"1min":0.35599999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.40500000000000003}]}]},{"osd":6,"up_from":43,"seq":184683593760,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298572,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446580,"statfs":{"total":95995035648,"available":95689297920,"internally_reserved":0,"allocated":1245184,"data_stored":895160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.46300000000000002,"15min":0.45900000000000002},"min":{"1min":0.32900000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.496,"15min":0.49199999999999999},"min":{"1min":0.32800000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.49199999999999999}]},{"osd":1,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.315,"5min":0.249,"15min":0.249},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.46700000000000003,"15min":0.46300000000000002},"min":{"1min":0.33700000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.65900000000000003,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.32200000000000001}]},{"osd":2,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.48299999999999998,"15min":0.48099999999999998},"min":{"1min":0.34000000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.49299999999999999,"15min":0.49099999999999999},"min":{"1min":0.33600000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68700000000000006,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.44600000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.48399999999999999,"15min":0.48299999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.376},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.51200000000000001,"15min":0.51000000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.45700000000000002}]},{"osd":4,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.39700000000000002,"15min":0.38800000000000001},"min":{"1min":0.26900000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.42699999999999999}]},{"osd":5,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.41899999999999998,"15min":0.41499999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.42499999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.40100000000000002}]},{"osd":7,"last update":"Tue Sep 10 18:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.47399999999999998,"15min":0.46999999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.5,"15min":0.496},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.50800000000000001}]}]},{"osd":5,"up_from":37,"seq":158913789988,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297984,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447168,"statfs":{"total":95995035648,"available":95689900032,"internally_reserved":0,"allocated":643072,"data_stored":303405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.52200000000000002,"15min":0.52600000000000002},"min":{"1min":0.30299999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70099999999999996,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.501},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.58299999999999996,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.45200000000000001}]},{"osd":1,"last update":"Tue Sep 10 18:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.55200000000000005,"15min":0.56000000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.69699999999999995,"5min":0.87,"15min":0.87},"last":0.437},{"interface":"front","average":{"1min":0.497,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.64200000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.42499999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.51800000000000002,"15min":0.52000000000000002},"min":{"1min":0.33300000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.71299999999999997,"5min":0.755,"15min":0.755},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53600000000000003,"15min":0.54100000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79600000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.47399999999999998}]},{"osd":3,"last update":"Tue Sep 10 18:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52500000000000002,"15min":0.52700000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76100000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.54200000000000004,"15min":0.54600000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80100000000000005,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.373}]},{"osd":4,"last update":"Tue Sep 10 18:46:21 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.64500000000000002,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.311},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.40500000000000003,"15min":0.40300000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.65700000000000003,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.46200000000000002}]},{"osd":6,"last update":"Tue Sep 10 18:46:44 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.42199999999999999,"15min":0.41599999999999998},"min":{"1min":0.29299999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.67500000000000004,"5min":0.753,"15min":0.753},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.46500000000000002,"15min":0.46100000000000002},"min":{"1min":0.36199999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.71199999999999997,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.58699999999999997}]},{"osd":7,"last update":"Tue Sep 10 18:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47399999999999998,"15min":0.47599999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.68799999999999994,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.47399999999999998,"15min":0.47599999999999998},"min":{"1min":0.28000000000000003,"5min":0.254,"15min":0.254},"max":{"1min":0.67600000000000005,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.53800000000000003}]}]},{"osd":4,"up_from":30,"seq":128849018922,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298004,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447148,"statfs":{"total":95995035648,"available":95689879552,"internally_reserved":0,"allocated":663552,"data_stored":304252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:58 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.52200000000000002,"15min":0.52400000000000002},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.76800000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.52200000000000002,"15min":0.53600000000000003},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.64000000000000001,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.504}]},{"osd":1,"last update":"Tue Sep 10 18:46:58 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.68400000000000005,"5min":2.609,"15min":2.609},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.54900000000000004,"15min":0.53300000000000003},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.80500000000000005,"5min":2.589,"15min":2.589},"last":0.52200000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:46:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.55400000000000005,"15min":0.54300000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.70399999999999996,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54300000000000004,"15min":0.55300000000000005},"min":{"1min":0.33200000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.73299999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.53500000000000003}]},{"osd":3,"last update":"Tue Sep 10 18:46:58 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.52500000000000002,"15min":0.52900000000000003},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.67500000000000004,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.433},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.52900000000000003,"15min":0.53900000000000003},"min":{"1min":0.35299999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.751,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.375}]},{"osd":5,"last update":"Tue Sep 10 18:46:22 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.35899999999999999,"15min":0.34999999999999998},"min":{"1min":0.26800000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.44,"5min":0.371,"15min":0.35899999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.39300000000000002}]},{"osd":6,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.42999999999999999,"15min":0.42399999999999999},"min":{"1min":0.311,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.68799999999999994,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.40100000000000002,"15min":0.39100000000000001},"min":{"1min":0.34799999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.46899999999999997}]},{"osd":7,"last update":"Tue Sep 10 18:47:10 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.5,"15min":0.5},"min":{"1min":0.318,"5min":0.312,"15min":0.312},"max":{"1min":0.64700000000000002,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.434,"5min":0.44600000000000001,"15min":0.44800000000000001},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.35499999999999998}]}]},{"osd":3,"up_from":25,"seq":107374182447,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298016,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447136,"statfs":{"total":95995035648,"available":95689867264,"internally_reserved":0,"allocated":675840,"data_stored":304031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:35 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.378,"15min":0.38400000000000001},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.498},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.40100000000000002,"15min":0.40300000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.39200000000000002}]},{"osd":1,"last update":"Tue Sep 10 18:46:35 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.40400000000000003,"15min":0.40899999999999997},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.44},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40899999999999997,"15min":0.41499999999999998},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.63800000000000001,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.36599999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:46:35 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.43099999999999999,"15min":0.438},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.377,"5min":0.41999999999999998,"15min":0.433},"min":{"1min":0.27200000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.53600000000000003,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.41999999999999998}]},{"osd":4,"last update":"Tue Sep 10 18:46:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54300000000000004,"15min":0.54400000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56200000000000006,"15min":0.57099999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.82299999999999995,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.61699999999999999}]},{"osd":5,"last update":"Tue Sep 10 18:46:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39700000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.70299999999999996,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.41399999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80800000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.66300000000000003}]},{"osd":6,"last update":"Tue Sep 10 18:46:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.56000000000000005,"15min":0.56299999999999994},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79100000000000004,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.628},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55400000000000005,"15min":0.55600000000000005},"min":{"1min":0.40799999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.67500000000000004}]},{"osd":7,"last update":"Tue Sep 10 18:47:08 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.54500000000000004,"15min":0.53500000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57999999999999996,"15min":0.57699999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84099999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.68600000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411379,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297988,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447164,"statfs":{"total":95995035648,"available":95689895936,"internally_reserved":0,"allocated":647168,"data_stored":303840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.376,"15min":0.35899999999999999},"min":{"1min":0.246,"5min":0.223,"15min":0.223},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.38300000000000001,"15min":0.371},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.44500000000000001}]},{"osd":1,"last update":"Tue Sep 10 18:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.375,"15min":0.36699999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.42499999999999999,"15min":0.41299999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.46899999999999997}]},{"osd":3,"last update":"Tue Sep 10 18:46:36 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46800000000000003,"15min":0.46500000000000002},"min":{"1min":0.30099999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.42899999999999999,"15min":0.43099999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.38500000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:46:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.69899999999999995,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56899999999999995,"15min":0.57099999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.64800000000000002,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.60999999999999999}]},{"osd":5,"last update":"Tue Sep 10 18:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56100000000000005,"15min":0.55800000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.42899999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.53700000000000003}]},{"osd":6,"last update":"Tue Sep 10 18:46:49 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58999999999999997,"15min":0.58599999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.46200000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.52500000000000002}]},{"osd":7,"last update":"Tue Sep 10 18:46:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.59999999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298016,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447136,"statfs":{"total":95995035648,"available":95689867264,"internally_reserved":0,"allocated":679936,"data_stored":304306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:46:48 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34399999999999997,"15min":0.32000000000000001},"min":{"1min":0.245,"5min":0.224,"15min":0.224},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.20399999999999999},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.36599999999999999,"15min":0.33600000000000002},"min":{"1min":0.252,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.53700000000000003,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.28799999999999998}]},{"osd":2,"last update":"Tue Sep 10 18:46:11 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.372,"15min":0.36099999999999999},"min":{"1min":0.27900000000000003,"5min":0.248,"15min":0.248},"max":{"1min":0.47199999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.246},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.39000000000000001,"15min":0.38100000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.49099999999999999,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.30499999999999999}]},{"osd":3,"last update":"Tue Sep 10 18:46:34 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.45400000000000001,"15min":0.45800000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.42299999999999999,"15min":0.42199999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.55400000000000005,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.32700000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:47:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.57099999999999995,"15min":0.55900000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.81399999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.434},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70399999999999996,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.46800000000000003}]},{"osd":5,"last update":"Tue Sep 10 18:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60299999999999998,"15min":0.60399999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63700000000000001,"15min":0.64100000000000001},"min":{"1min":0.44600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93300000000000005,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.45200000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:46:43 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.46300000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.66400000000000003,"15min":0.67100000000000004},"min":{"1min":0.41999999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82299999999999995,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.41399999999999998}]},{"osd":7,"last update":"Tue Sep 10 18:46:11 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.48099999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738429,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298572,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446580,"statfs":{"total":95995035648,"available":95689297920,"internally_reserved":0,"allocated":1249280,"data_stored":893894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 18:46:52 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.32500000000000001,"15min":0.30299999999999999},"min":{"1min":0.219,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.58799999999999997,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33700000000000002,"15min":0.315},"min":{"1min":0.221,"5min":0.187,"15min":0.187},"max":{"1min":0.59399999999999997,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.30199999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.40300000000000002,"15min":0.38900000000000001},"min":{"1min":0.30299999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.376,"15min":0.36199999999999999},"min":{"1min":0.26600000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.32500000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:46:31 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.39500000000000002,"15min":0.39800000000000002},"min":{"1min":0.26100000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.66100000000000003,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.41199999999999998,"15min":0.41299999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.36199999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:47:00 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54400000000000004,"15min":0.54100000000000004},"min":{"1min":0.41899999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.79200000000000004,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.436},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.56899999999999995,"15min":0.53500000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.69899999999999995,"5min":4.6189999999999998,"15min":4.6189999999999998},"last":0.55500000000000005}]},{"osd":5,"last update":"Tue Sep 10 18:46:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.42999999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.497},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.58699999999999997,"15min":0.55800000000000005},"min":{"1min":0.42699999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":4.6429999999999998,"5min":4.6429999999999998,"15min":4.6429999999999998},"last":0.51300000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:46:42 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56100000000000005,"15min":0.55500000000000005},"min":{"1min":0.42099999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.746,"15min":0.77300000000000002},"min":{"1min":0.40899999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.878,"5min":4.6630000000000003,"15min":4.6630000000000003},"last":0.47599999999999998}]},{"osd":7,"last update":"Tue Sep 10 18:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.55100000000000005,"15min":0.55500000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.67700000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.73799999999999999,"5min":0.746,"15min":0.746},"last":0.57599999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T18:47:16.997 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T18:47:16.997 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd dump 2024-09-10T18:47:17.374 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:17 smithi018 ceph-mon[40196]: 5.1e scrub starts 2024-09-10T18:47:17.374 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:17 smithi018 ceph-mon[40196]: 5.1e scrub ok 2024-09-10T18:47:17.374 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:17 smithi018 ceph-mon[40196]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:47:17.374 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:17 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:17.375 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:47:17 smithi018 podman[75368]: 2024-09-10 18:47:17.095238769 +0000 UTC m=+0.020811156 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-09-10T18:47:17.375 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:17 smithi018 ceph-mon[29336]: 5.1e scrub starts 2024-09-10T18:47:17.375 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:17 smithi018 ceph-mon[29336]: 5.1e scrub ok 2024-09-10T18:47:17.375 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:17 smithi018 ceph-mon[29336]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:47:17.375 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:17 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:17.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:17 smithi103 ceph-mon[33163]: 5.1e scrub starts 2024-09-10T18:47:17.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:17 smithi103 ceph-mon[33163]: 5.1e scrub ok 2024-09-10T18:47:17.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:17 smithi103 ceph-mon[33163]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:47:17.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:17 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:17.670 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:47:17 smithi018 podman[75368]: 2024-09-10 18:47:17.373213389 +0000 UTC m=+0.298785765 volume create 7e3928d98152420fa3c16aa48f414214724372270d58229199a72f9b36de8b75 2024-09-10T18:47:17.670 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:47:17 smithi018 podman[75368]: 2024-09-10 18:47:17.487945035 +0000 UTC m=+0.413517413 container create bfb4ed37f3aa8bae5954b592c776c8b562c237c1b6d4a76128308ca545da84ba (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:47:17.997 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:47:18.119 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:47:17 smithi018 podman[75368]: 2024-09-10 18:47:17.782464981 +0000 UTC m=+0.708037366 container init bfb4ed37f3aa8bae5954b592c776c8b562c237c1b6d4a76128308ca545da84ba (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:47:18.119 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:47:17 smithi018 podman[75368]: 2024-09-10 18:47:17.785407119 +0000 UTC m=+0.710979499 container start bfb4ed37f3aa8bae5954b592c776c8b562c237c1b6d4a76128308ca545da84ba (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:47:18.119 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:47:17 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a[75395]: ts=2024-09-10T18:47:17.798Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-10T18:47:18.119 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:47:17 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a[75395]: ts=2024-09-10T18:47:17.798Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-10T18:47:18.120 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:47:17 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a[75395]: ts=2024-09-10T18:47:17.799Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.18 port=9094 2024-09-10T18:47:18.120 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:47:17 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a[75395]: ts=2024-09-10T18:47:17.800Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-10T18:47:18.120 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:47:17 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a[75395]: ts=2024-09-10T18:47:17.823Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-10T18:47:18.120 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:47:17 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a[75395]: ts=2024-09-10T18:47:17.824Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-10T18:47:18.120 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:47:17 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a[75395]: ts=2024-09-10T18:47:17.827Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-10T18:47:18.120 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:47:17 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a[75395]: ts=2024-09-10T18:47:17.827Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-10T18:47:18.120 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:47:17 smithi018 bash[75368]: bfb4ed37f3aa8bae5954b592c776c8b562c237c1b6d4a76128308ca545da84ba 2024-09-10T18:47:18.120 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:47:17 smithi018 systemd[1]: Started Ceph alertmanager.a for ba327f6c-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:47:18.378 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:18 smithi018 ceph-mon[29336]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:47:18.378 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:18 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:18.378 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:18 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:18.379 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:18 smithi018 ceph-mon[40196]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:47:18.379 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:18 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:18.379 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:18 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:18.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:18 smithi103 ceph-mon[33163]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:47:18.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:18 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:18.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:18 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:19.471 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:19 smithi018 ceph-mon[40196]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-10T18:47:19.471 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:19 smithi018 ceph-mon[40196]: Reconfiguring daemon prometheus.a on smithi103 2024-09-10T18:47:19.471 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:19 smithi018 ceph-mon[40196]: 6.b scrub starts 2024-09-10T18:47:19.471 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:19 smithi018 ceph-mon[40196]: 6.b scrub ok 2024-09-10T18:47:19.471 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:19 smithi018 ceph-mon[29336]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-10T18:47:19.471 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:19 smithi018 ceph-mon[29336]: Reconfiguring daemon prometheus.a on smithi103 2024-09-10T18:47:19.471 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:19 smithi018 ceph-mon[29336]: 6.b scrub starts 2024-09-10T18:47:19.472 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:19 smithi018 ceph-mon[29336]: 6.b scrub ok 2024-09-10T18:47:19.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:19 smithi103 ceph-mon[33163]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-10T18:47:19.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:19 smithi103 ceph-mon[33163]: Reconfiguring daemon prometheus.a on smithi103 2024-09-10T18:47:19.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:19 smithi103 ceph-mon[33163]: 6.b scrub starts 2024-09-10T18:47:19.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:19 smithi103 ceph-mon[33163]: 6.b scrub ok 2024-09-10T18:47:19.831 INFO:teuthology.orchestra.run.smithi018.stdout:epoch 84 2024-09-10T18:47:19.831 INFO:teuthology.orchestra.run.smithi018.stdout:fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:47:19.831 INFO:teuthology.orchestra.run.smithi018.stdout:created 2024-09-10T18:40:07.669687+0000 2024-09-10T18:47:19.832 INFO:teuthology.orchestra.run.smithi018.stdout:modified 2024-09-10T18:47:07.079460+0000 2024-09-10T18:47:19.832 INFO:teuthology.orchestra.run.smithi018.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-10T18:47:19.832 INFO:teuthology.orchestra.run.smithi018.stdout:crush_version 18 2024-09-10T18:47:19.832 INFO:teuthology.orchestra.run.smithi018.stdout:full_ratio 0.95 2024-09-10T18:47:19.832 INFO:teuthology.orchestra.run.smithi018.stdout:backfillfull_ratio 0.9 2024-09-10T18:47:19.833 INFO:teuthology.orchestra.run.smithi018.stdout:nearfull_ratio 0.85 2024-09-10T18:47:19.833 INFO:teuthology.orchestra.run.smithi018.stdout:require_min_compat_client luminous 2024-09-10T18:47:19.833 INFO:teuthology.orchestra.run.smithi018.stdout:min_compat_client luminous 2024-09-10T18:47:19.833 INFO:teuthology.orchestra.run.smithi018.stdout:require_osd_release quincy 2024-09-10T18:47:19.833 INFO:teuthology.orchestra.run.smithi018.stdout:stretch_mode_enabled false 2024-09-10T18:47:19.834 INFO:teuthology.orchestra.run.smithi018.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr 2024-09-10T18:47:19.834 INFO:teuthology.orchestra.run.smithi018.stdout:pool 2 '.rgw.root' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 57 flags hashpspool stripe_width 0 application rgw 2024-09-10T18:47:19.834 INFO:teuthology.orchestra.run.smithi018.stdout:pool 3 'datapool' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 3 pgp_num 3 autoscale_mode off last_change 60 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd 2024-09-10T18:47:19.834 INFO:teuthology.orchestra.run.smithi018.stdout:pool 4 'default.rgw.log' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 59 flags hashpspool stripe_width 0 application rgw 2024-09-10T18:47:19.834 INFO:teuthology.orchestra.run.smithi018.stdout:pool 5 'default.rgw.control' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 61 flags hashpspool stripe_width 0 application rgw 2024-09-10T18:47:19.834 INFO:teuthology.orchestra.run.smithi018.stdout:pool 6 'default.rgw.meta' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 63 flags hashpspool stripe_width 0 pg_autoscale_bias 4 application rgw 2024-09-10T18:47:19.835 INFO:teuthology.orchestra.run.smithi018.stdout:max_osd 8 2024-09-10T18:47:19.835 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0 up in weight 1 up_from 8 up_thru 61 down_at 0 last_clean_interval [0,0) v2:172.21.15.18:6801/2131279292 v2:172.21.15.18:6802/2131279292 exists,up eb4c7e67-300c-4a1a-b9f5-bb028eeeabc2 2024-09-10T18:47:19.835 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1 up in weight 1 up_from 13 up_thru 61 down_at 0 last_clean_interval [0,0) v2:172.21.15.18:6805/3855640936 v2:172.21.15.18:6806/3855640936 exists,up 63bf6a98-e349-4cc0-9502-a31693070c5b 2024-09-10T18:47:19.835 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2 up in weight 1 up_from 18 up_thru 82 down_at 0 last_clean_interval [0,0) v2:172.21.15.18:6809/4107623855 v2:172.21.15.18:6810/4107623855 exists,up 79a01c29-4f4d-43d1-b1ea-a3fcf014f84b 2024-09-10T18:47:19.835 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3 up in weight 1 up_from 25 up_thru 82 down_at 0 last_clean_interval [0,0) v2:172.21.15.18:6813/2732204253 v2:172.21.15.18:6814/2732204253 exists,up 1809e677-b98c-4924-982e-9297e37563af 2024-09-10T18:47:19.835 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4 up in weight 1 up_from 30 up_thru 82 down_at 0 last_clean_interval [0,0) v2:172.21.15.103:6800/465860081 v2:172.21.15.103:6801/465860081 exists,up b682a300-bb53-4f36-8a9f-0753164ec4b7 2024-09-10T18:47:19.836 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 up in weight 1 up_from 37 up_thru 82 down_at 0 last_clean_interval [0,0) v2:172.21.15.103:6804/3498719258 v2:172.21.15.103:6805/3498719258 exists,up 2bbfdb6c-0709-4fe4-8a4a-006f576f18e6 2024-09-10T18:47:19.836 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6 up in weight 1 up_from 43 up_thru 59 down_at 0 last_clean_interval [0,0) v2:172.21.15.103:6808/184353788 v2:172.21.15.103:6809/184353788 exists,up 31f9711e-0ba8-4aeb-b852-7d93e8fcce33 2024-09-10T18:47:19.836 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7 up in weight 1 up_from 52 up_thru 61 down_at 0 last_clean_interval [0,0) v2:172.21.15.103:6812/941024403 v2:172.21.15.103:6813/941024403 exists,up f12282f8-4747-45ff-8336-95440560a88f 2024-09-10T18:47:19.836 INFO:teuthology.orchestra.run.smithi018.stdout:pg_upmap_items 2.0 [7,2] 2024-09-10T18:47:19.836 INFO:teuthology.orchestra.run.smithi018.stdout:pg_upmap_items 4.e [1,2] 2024-09-10T18:47:19.837 INFO:teuthology.orchestra.run.smithi018.stdout:pg_upmap_items 4.f [1,5] 2024-09-10T18:47:19.837 INFO:teuthology.orchestra.run.smithi018.stdout:pg_upmap_items 4.14 [1,2] 2024-09-10T18:47:19.837 INFO:teuthology.orchestra.run.smithi018.stdout:blocklist 172.21.15.18:0/3021537013 expires 2024-09-11T18:45:54.998616+0000 2024-09-10T18:47:19.837 INFO:teuthology.orchestra.run.smithi018.stdout:blocklist 172.21.15.18:0/458572898 expires 2024-09-11T18:40:28.880042+0000 2024-09-10T18:47:19.837 INFO:teuthology.orchestra.run.smithi018.stdout:blocklist 172.21.15.18:0/1859541419 expires 2024-09-11T18:40:28.880042+0000 2024-09-10T18:47:19.838 INFO:teuthology.orchestra.run.smithi018.stdout:blocklist 172.21.15.18:0/3090642970 expires 2024-09-11T18:40:59.659411+0000 2024-09-10T18:47:19.838 INFO:teuthology.orchestra.run.smithi018.stdout:blocklist 172.21.15.18:0/2875894312 expires 2024-09-11T18:40:59.659411+0000 2024-09-10T18:47:19.838 INFO:teuthology.orchestra.run.smithi018.stdout:blocklist 172.21.15.18:0/178638059 expires 2024-09-11T18:40:59.659411+0000 2024-09-10T18:47:19.838 INFO:teuthology.orchestra.run.smithi018.stdout:blocklist 172.21.15.18:0/692931402 expires 2024-09-11T18:45:54.998616+0000 2024-09-10T18:47:19.838 INFO:teuthology.orchestra.run.smithi018.stdout:blocklist 172.21.15.18:0/2408360238 expires 2024-09-11T18:45:54.998616+0000 2024-09-10T18:47:19.838 INFO:teuthology.orchestra.run.smithi018.stdout:blocklist 172.21.15.18:0/57192865 expires 2024-09-11T18:40:28.880042+0000 2024-09-10T18:47:19.839 INFO:teuthology.orchestra.run.smithi018.stdout:blocklist 172.21.15.18:6800/2748756682 expires 2024-09-11T18:40:59.659411+0000 2024-09-10T18:47:19.839 INFO:teuthology.orchestra.run.smithi018.stdout:blocklist 172.21.15.18:6800/732375116 expires 2024-09-11T18:45:54.998616+0000 2024-09-10T18:47:19.839 INFO:teuthology.orchestra.run.smithi018.stdout:blocklist 172.21.15.18:6800/3869801150 expires 2024-09-11T18:40:28.880042+0000 2024-09-10T18:47:19.839 INFO:teuthology.orchestra.run.smithi018.stdout:blocklist 172.21.15.18:0/2982361259 expires 2024-09-11T18:45:54.998616+0000 2024-09-10T18:47:19.839 INFO:teuthology.orchestra.run.smithi018.stdout:blocklist 172.21.15.18:0/185700038 expires 2024-09-11T18:45:54.998616+0000 2024-09-10T18:47:20.147 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:47:19 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a[75395]: ts=2024-09-10T18:47:19.801Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000380184s 2024-09-10T18:47:20.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:20 smithi018 ceph-mon[29336]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 856 B/s rd, 0 op/s 2024-09-10T18:47:20.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:20 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/246589903' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T18:47:20.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:20 smithi018 ceph-mon[40196]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 856 B/s rd, 0 op/s 2024-09-10T18:47:20.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:20 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/246589903' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T18:47:20.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:20 smithi103 ceph-mon[33163]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 856 B/s rd, 0 op/s 2024-09-10T18:47:20.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:20 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/246589903' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T18:47:20.523 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:47:20 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:47:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:47:20.583 INFO:tasks.cephadm.ceph_manager.ceph:['osd.0 up in weight 1 up_from 8 up_thru 61 down_at 0 last_clean_interval [0,0) v2:172.21.15.18:6801/2131279292 v2:172.21.15.18:6802/2131279292 exists,up eb4c7e67-300c-4a1a-b9f5-bb028eeeabc2', 'osd.1 up in weight 1 up_from 13 up_thru 61 down_at 0 last_clean_interval [0,0) v2:172.21.15.18:6805/3855640936 v2:172.21.15.18:6806/3855640936 exists,up 63bf6a98-e349-4cc0-9502-a31693070c5b', 'osd.2 up in weight 1 up_from 18 up_thru 82 down_at 0 last_clean_interval [0,0) v2:172.21.15.18:6809/4107623855 v2:172.21.15.18:6810/4107623855 exists,up 79a01c29-4f4d-43d1-b1ea-a3fcf014f84b', 'osd.3 up in weight 1 up_from 25 up_thru 82 down_at 0 last_clean_interval [0,0) v2:172.21.15.18:6813/2732204253 v2:172.21.15.18:6814/2732204253 exists,up 1809e677-b98c-4924-982e-9297e37563af', 'osd.4 up in weight 1 up_from 30 up_thru 82 down_at 0 last_clean_interval [0,0) v2:172.21.15.103:6800/465860081 v2:172.21.15.103:6801/465860081 exists,up b682a300-bb53-4f36-8a9f-0753164ec4b7', 'osd.5 up in weight 1 up_from 37 up_thru 82 down_at 0 last_clean_interval [0,0) v2:172.21.15.103:6804/3498719258 v2:172.21.15.103:6805/3498719258 exists,up 2bbfdb6c-0709-4fe4-8a4a-006f576f18e6', 'osd.6 up in weight 1 up_from 43 up_thru 59 down_at 0 last_clean_interval [0,0) v2:172.21.15.103:6808/184353788 v2:172.21.15.103:6809/184353788 exists,up 31f9711e-0ba8-4aeb-b852-7d93e8fcce33', 'osd.7 up in weight 1 up_from 52 up_thru 61 down_at 0 last_clean_interval [0,0) v2:172.21.15.103:6812/941024403 v2:172.21.15.103:6813/941024403 exists,up f12282f8-4747-45ff-8336-95440560a88f'] 2024-09-10T18:47:20.584 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon mon.a config show 2024-09-10T18:47:20.842 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:47:21.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:21 smithi103 ceph-mon[33163]: 2.e scrub starts 2024-09-10T18:47:21.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:21 smithi103 ceph-mon[33163]: 2.e scrub ok 2024-09-10T18:47:21.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:21 smithi018 ceph-mon[29336]: 2.e scrub starts 2024-09-10T18:47:21.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:21 smithi018 ceph-mon[29336]: 2.e scrub ok 2024-09-10T18:47:21.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:21 smithi018 ceph-mon[40196]: 2.e scrub starts 2024-09-10T18:47:21.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:21 smithi018 ceph-mon[40196]: 2.e scrub ok 2024-09-10T18:47:22.266 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:22 smithi018 ceph-mon[29336]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:47:22.266 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:22 smithi018 ceph-mon[40196]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:47:22.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:22 smithi103 ceph-mon[33163]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:47:22.605 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T18:47:22.605 INFO:teuthology.orchestra.run.smithi018.stdout: "name": "mon.a", 2024-09-10T18:47:22.605 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster": "ceph", 2024-09-10T18:47:22.605 INFO:teuthology.orchestra.run.smithi018.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-10T18:47:22.605 INFO:teuthology.orchestra.run.smithi018.stdout: "admin_socket_mode": "", 2024-09-10T18:47:22.606 INFO:teuthology.orchestra.run.smithi018.stdout: "auth_allow_insecure_global_id_reclaim": "false", 2024-09-10T18:47:22.606 INFO:teuthology.orchestra.run.smithi018.stdout: "auth_client_required": "cephx, none", 2024-09-10T18:47:22.606 INFO:teuthology.orchestra.run.smithi018.stdout: "auth_cluster_required": "cephx", 2024-09-10T18:47:22.606 INFO:teuthology.orchestra.run.smithi018.stdout: "auth_debug": "true", 2024-09-10T18:47:22.606 INFO:teuthology.orchestra.run.smithi018.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-10T18:47:22.606 INFO:teuthology.orchestra.run.smithi018.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-10T18:47:22.606 INFO:teuthology.orchestra.run.smithi018.stdout: "auth_service_required": "cephx", 2024-09-10T18:47:22.606 INFO:teuthology.orchestra.run.smithi018.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-10T18:47:22.606 INFO:teuthology.orchestra.run.smithi018.stdout: "auth_supported": "", 2024-09-10T18:47:22.606 INFO:teuthology.orchestra.run.smithi018.stdout: "bdev_aio": "true", 2024-09-10T18:47:22.606 INFO:teuthology.orchestra.run.smithi018.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-10T18:47:22.606 INFO:teuthology.orchestra.run.smithi018.stdout: "bdev_aio_poll_ms": "250", 2024-09-10T18:47:22.606 INFO:teuthology.orchestra.run.smithi018.stdout: "bdev_aio_reap_max": "16", 2024-09-10T18:47:22.607 INFO:teuthology.orchestra.run.smithi018.stdout: "bdev_async_discard": "false", 2024-09-10T18:47:22.607 INFO:teuthology.orchestra.run.smithi018.stdout: "bdev_block_size": "4096", 2024-09-10T18:47:22.607 INFO:teuthology.orchestra.run.smithi018.stdout: "bdev_debug_aio": "false", 2024-09-10T18:47:22.607 INFO:teuthology.orchestra.run.smithi018.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-10T18:47:22.607 INFO:teuthology.orchestra.run.smithi018.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-10T18:47:22.607 INFO:teuthology.orchestra.run.smithi018.stdout: "bdev_debug_inflight_ios": "false", 2024-09-10T18:47:22.607 INFO:teuthology.orchestra.run.smithi018.stdout: "bdev_enable_discard": "false", 2024-09-10T18:47:22.607 INFO:teuthology.orchestra.run.smithi018.stdout: "bdev_flock_retry": "3", 2024-09-10T18:47:22.607 INFO:teuthology.orchestra.run.smithi018.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-10T18:47:22.607 INFO:teuthology.orchestra.run.smithi018.stdout: "bdev_inject_crash": "0", 2024-09-10T18:47:22.607 INFO:teuthology.orchestra.run.smithi018.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-10T18:47:22.607 INFO:teuthology.orchestra.run.smithi018.stdout: "bdev_ioring": "false", 2024-09-10T18:47:22.607 INFO:teuthology.orchestra.run.smithi018.stdout: "bdev_ioring_hipri": "false", 2024-09-10T18:47:22.608 INFO:teuthology.orchestra.run.smithi018.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-10T18:47:22.608 INFO:teuthology.orchestra.run.smithi018.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-10T18:47:22.608 INFO:teuthology.orchestra.run.smithi018.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-10T18:47:22.608 INFO:teuthology.orchestra.run.smithi018.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-10T18:47:22.608 INFO:teuthology.orchestra.run.smithi018.stdout: "bdev_type": "", 2024-09-10T18:47:22.608 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_alloc_size": "1048576", 2024-09-10T18:47:22.608 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_allocator": "hybrid", 2024-09-10T18:47:22.608 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_buffered_io": "true", 2024-09-10T18:47:22.608 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_check_for_zeros": "false", 2024-09-10T18:47:22.608 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-10T18:47:22.608 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-10T18:47:22.608 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_compact_log_sync": "false", 2024-09-10T18:47:22.608 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-10T18:47:22.609 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-10T18:47:22.609 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-10T18:47:22.609 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-10T18:47:22.609 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_max_log_runway": "4194304", 2024-09-10T18:47:22.609 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_max_prefetch": "1048576", 2024-09-10T18:47:22.609 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_min_flush_size": "524288", 2024-09-10T18:47:22.609 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_min_log_runway": "1048576", 2024-09-10T18:47:22.609 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_replay_recovery": "false", 2024-09-10T18:47:22.609 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-10T18:47:22.609 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-10T18:47:22.609 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_sync_write": "false", 2024-09-10T18:47:22.609 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-10T18:47:22.610 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-10T18:47:22.610 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-10T18:47:22.610 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_allocation_from_file": "true", 2024-09-10T18:47:22.610 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_allocator": "hybrid", 2024-09-10T18:47:22.610 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_async_db_compaction": "true", 2024-09-10T18:47:22.610 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-10T18:47:22.610 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-10T18:47:22.610 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-10T18:47:22.610 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-10T18:47:22.610 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-10T18:47:22.610 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-10T18:47:22.610 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-10T18:47:22.610 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_block_create": "true", 2024-09-10T18:47:22.611 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_block_db_create": "false", 2024-09-10T18:47:22.611 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_block_db_path": "", 2024-09-10T18:47:22.611 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_block_db_size": "0", 2024-09-10T18:47:22.611 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_block_path": "", 2024-09-10T18:47:22.611 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_block_preallocate_file": "false", 2024-09-10T18:47:22.611 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_block_size": "107374182400", 2024-09-10T18:47:22.611 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_block_wal_create": "false", 2024-09-10T18:47:22.611 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_block_wal_path": "", 2024-09-10T18:47:22.611 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_block_wal_size": "100663296", 2024-09-10T18:47:22.611 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_bluefs": "true", 2024-09-10T18:47:22.611 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-10T18:47:22.611 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-10T18:47:22.612 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-10T18:47:22.612 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-10T18:47:22.612 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-10T18:47:22.612 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-10T18:47:22.612 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-10T18:47:22.612 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-10T18:47:22.612 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_autotune": "true", 2024-09-10T18:47:22.612 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-10T18:47:22.612 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-10T18:47:22.612 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-10T18:47:22.612 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-10T18:47:22.612 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_size": "0", 2024-09-10T18:47:22.613 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-10T18:47:22.613 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-10T18:47:22.613 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-10T18:47:22.613 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-10T18:47:22.613 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_type": "2q", 2024-09-10T18:47:22.613 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-10T18:47:22.613 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_clone_cow": "true", 2024-09-10T18:47:22.614 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-10T18:47:22.614 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-10T18:47:22.614 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-10T18:47:22.614 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-10T18:47:22.614 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-10T18:47:22.614 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-10T18:47:22.614 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-10T18:47:22.614 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_compression_mode": "none", 2024-09-10T18:47:22.614 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-10T18:47:22.614 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_csum_type": "crc32c", 2024-09-10T18:47:22.614 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-10T18:47:22.615 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_debug_freelist": "false", 2024-09-10T18:47:22.615 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-10T18:47:22.615 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-10T18:47:22.615 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_debug_inject_bug21040": "false", 2024-09-10T18:47:22.615 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-10T18:47:22.615 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-10T18:47:22.615 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-10T18:47:22.615 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-10T18:47:22.615 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-10T18:47:22.615 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-10T18:47:22.615 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-10T18:47:22.615 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_debug_prefill": "0.000000", 2024-09-10T18:47:22.615 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-10T18:47:22.616 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-10T18:47:22.616 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-10T18:47:22.616 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_debug_small_allocations": "0", 2024-09-10T18:47:22.616 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-10T18:47:22.616 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_default_buffered_read": "true", 2024-09-10T18:47:22.616 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_default_buffered_write": "false", 2024-09-10T18:47:22.616 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-10T18:47:22.616 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-10T18:47:22.616 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-10T18:47:22.616 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-10T18:47:22.616 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-10T18:47:22.616 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-10T18:47:22.617 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-10T18:47:22.617 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-10T18:47:22.617 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fail_eio": "false", 2024-09-10T18:47:22.617 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-10T18:47:22.617 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-10T18:47:22.617 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-10T18:47:22.617 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-10T18:47:22.617 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-10T18:47:22.617 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-10T18:47:22.617 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fsck_on_mount": "false", 2024-09-10T18:47:22.618 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-10T18:47:22.618 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fsck_on_umount": "false", 2024-09-10T18:47:22.618 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-10T18:47:22.618 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-10T18:47:22.618 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-10T18:47:22.618 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-10T18:47:22.618 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-10T18:47:22.618 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-10T18:47:22.618 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-10T18:47:22.618 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-10T18:47:22.618 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_ignore_data_csum": "false", 2024-09-10T18:47:22.619 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-10T18:47:22.619 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-10T18:47:22.619 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-10T18:47:22.619 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-10T18:47:22.619 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_log_op_age": "5.000000", 2024-09-10T18:47:22.619 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_max_alloc_size": "0", 2024-09-10T18:47:22.619 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_max_blob_size": "0", 2024-09-10T18:47:22.619 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-10T18:47:22.619 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-10T18:47:22.619 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-10T18:47:22.619 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_max_deferred_txc": "32", 2024-09-10T18:47:22.620 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_min_alloc_size": "0", 2024-09-10T18:47:22.620 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-10T18:47:22.620 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-10T18:47:22.620 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_nid_prealloc": "1024", 2024-09-10T18:47:22.620 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-10T18:47:22.620 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-10T18:47:22.620 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-10T18:47:22.620 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-10T18:47:22.620 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_retry_disk_reads": "3", 2024-09-10T18:47:22.620 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_rocksdb_cf": "true", 2024-09-10T18:47:22.620 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L P", 2024-09-10T18:47:22.620 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_rocksdb_options": "compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152,max_background_compactions=2,max_total_wal_size=1073741824", 2024-09-10T18:47:22.621 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-10T18:47:22.621 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-10T18:47:22.621 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-10T18:47:22.621 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-10T18:47:22.621 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_spdk_mem": "512", 2024-09-10T18:47:22.621 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-10T18:47:22.621 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-10T18:47:22.621 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-10T18:47:22.621 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-10T18:47:22.621 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-10T18:47:22.621 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-10T18:47:22.621 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-10T18:47:22.621 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_tracing": "false", 2024-09-10T18:47:22.622 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-10T18:47:22.622 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-10T18:47:22.622 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-10T18:47:22.622 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-10T18:47:22.622 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-10T18:47:22.622 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-10T18:47:22.622 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-10T18:47:22.622 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-10T18:47:22.622 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-10T18:47:22.622 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_zero_block_detection": "false", 2024-09-10T18:47:22.622 INFO:teuthology.orchestra.run.smithi018.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-10T18:47:22.622 INFO:teuthology.orchestra.run.smithi018.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-10T18:47:22.623 INFO:teuthology.orchestra.run.smithi018.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-10T18:47:22.623 INFO:teuthology.orchestra.run.smithi018.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-10T18:47:22.623 INFO:teuthology.orchestra.run.smithi018.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-10T18:47:22.623 INFO:teuthology.orchestra.run.smithi018.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-10T18:47:22.623 INFO:teuthology.orchestra.run.smithi018.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-10T18:47:22.623 INFO:teuthology.orchestra.run.smithi018.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-10T18:47:22.623 INFO:teuthology.orchestra.run.smithi018.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-10T18:47:22.623 INFO:teuthology.orchestra.run.smithi018.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-10T18:47:22.623 INFO:teuthology.orchestra.run.smithi018.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-10T18:47:22.623 INFO:teuthology.orchestra.run.smithi018.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-10T18:47:22.623 INFO:teuthology.orchestra.run.smithi018.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-10T18:47:22.623 INFO:teuthology.orchestra.run.smithi018.stdout: "cephx_cluster_require_signatures": "false", 2024-09-10T18:47:22.623 INFO:teuthology.orchestra.run.smithi018.stdout: "cephx_cluster_require_version": "2", 2024-09-10T18:47:22.623 INFO:teuthology.orchestra.run.smithi018.stdout: "cephx_require_signatures": "false", 2024-09-10T18:47:22.624 INFO:teuthology.orchestra.run.smithi018.stdout: "cephx_require_version": "2", 2024-09-10T18:47:22.624 INFO:teuthology.orchestra.run.smithi018.stdout: "cephx_service_require_signatures": "false", 2024-09-10T18:47:22.624 INFO:teuthology.orchestra.run.smithi018.stdout: "cephx_service_require_version": "2", 2024-09-10T18:47:22.624 INFO:teuthology.orchestra.run.smithi018.stdout: "cephx_sign_messages": "true", 2024-09-10T18:47:22.624 INFO:teuthology.orchestra.run.smithi018.stdout: "chdir": "", 2024-09-10T18:47:22.624 INFO:teuthology.orchestra.run.smithi018.stdout: "client_acl_type": "", 2024-09-10T18:47:22.624 INFO:teuthology.orchestra.run.smithi018.stdout: "client_asio_thread_count": "2", 2024-09-10T18:47:22.624 INFO:teuthology.orchestra.run.smithi018.stdout: "client_cache_mid": "0.750000", 2024-09-10T18:47:22.624 INFO:teuthology.orchestra.run.smithi018.stdout: "client_cache_size": "16384", 2024-09-10T18:47:22.625 INFO:teuthology.orchestra.run.smithi018.stdout: "client_caps_release_delay": "5", 2024-09-10T18:47:22.625 INFO:teuthology.orchestra.run.smithi018.stdout: "client_check_pool_perm": "true", 2024-09-10T18:47:22.625 INFO:teuthology.orchestra.run.smithi018.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-10T18:47:22.625 INFO:teuthology.orchestra.run.smithi018.stdout: "client_debug_force_sync_read": "false", 2024-09-10T18:47:22.625 INFO:teuthology.orchestra.run.smithi018.stdout: "client_debug_getattr_caps": "false", 2024-09-10T18:47:22.625 INFO:teuthology.orchestra.run.smithi018.stdout: "client_debug_inject_features": "", 2024-09-10T18:47:22.625 INFO:teuthology.orchestra.run.smithi018.stdout: "client_debug_inject_tick_delay": "0", 2024-09-10T18:47:22.625 INFO:teuthology.orchestra.run.smithi018.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-10T18:47:22.625 INFO:teuthology.orchestra.run.smithi018.stdout: "client_die_on_failed_remount": "false", 2024-09-10T18:47:22.625 INFO:teuthology.orchestra.run.smithi018.stdout: "client_dirsize_rbytes": "true", 2024-09-10T18:47:22.625 INFO:teuthology.orchestra.run.smithi018.stdout: "client_force_lazyio": "false", 2024-09-10T18:47:22.625 INFO:teuthology.orchestra.run.smithi018.stdout: "client_fs": "", 2024-09-10T18:47:22.626 INFO:teuthology.orchestra.run.smithi018.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-10T18:47:22.626 INFO:teuthology.orchestra.run.smithi018.stdout: "client_inject_release_failure": "false", 2024-09-10T18:47:22.626 INFO:teuthology.orchestra.run.smithi018.stdout: "client_max_inline_size": "4096", 2024-09-10T18:47:22.626 INFO:teuthology.orchestra.run.smithi018.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-10T18:47:22.626 INFO:teuthology.orchestra.run.smithi018.stdout: "client_mds_namespace": "", 2024-09-10T18:47:22.626 INFO:teuthology.orchestra.run.smithi018.stdout: "client_metadata": "", 2024-09-10T18:47:22.626 INFO:teuthology.orchestra.run.smithi018.stdout: "client_mount_gid": "-1", 2024-09-10T18:47:22.626 INFO:teuthology.orchestra.run.smithi018.stdout: "client_mount_timeout": "300", 2024-09-10T18:47:22.626 INFO:teuthology.orchestra.run.smithi018.stdout: "client_mount_uid": "-1", 2024-09-10T18:47:22.626 INFO:teuthology.orchestra.run.smithi018.stdout: "client_mountpoint": "/", 2024-09-10T18:47:22.626 INFO:teuthology.orchestra.run.smithi018.stdout: "client_notify_timeout": "10", 2024-09-10T18:47:22.626 INFO:teuthology.orchestra.run.smithi018.stdout: "client_oc": "true", 2024-09-10T18:47:22.626 INFO:teuthology.orchestra.run.smithi018.stdout: "client_oc_max_dirty": "104857600", 2024-09-10T18:47:22.627 INFO:teuthology.orchestra.run.smithi018.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-10T18:47:22.627 INFO:teuthology.orchestra.run.smithi018.stdout: "client_oc_max_objects": "1000", 2024-09-10T18:47:22.627 INFO:teuthology.orchestra.run.smithi018.stdout: "client_oc_size": "209715200", 2024-09-10T18:47:22.627 INFO:teuthology.orchestra.run.smithi018.stdout: "client_oc_target_dirty": "8388608", 2024-09-10T18:47:22.627 INFO:teuthology.orchestra.run.smithi018.stdout: "client_permissions": "true", 2024-09-10T18:47:22.627 INFO:teuthology.orchestra.run.smithi018.stdout: "client_quota": "true", 2024-09-10T18:47:22.627 INFO:teuthology.orchestra.run.smithi018.stdout: "client_quota_df": "true", 2024-09-10T18:47:22.627 INFO:teuthology.orchestra.run.smithi018.stdout: "client_readahead_max_bytes": "0", 2024-09-10T18:47:22.627 INFO:teuthology.orchestra.run.smithi018.stdout: "client_readahead_max_periods": "4", 2024-09-10T18:47:22.627 INFO:teuthology.orchestra.run.smithi018.stdout: "client_readahead_min": "131072", 2024-09-10T18:47:22.627 INFO:teuthology.orchestra.run.smithi018.stdout: "client_reconnect_stale": "false", 2024-09-10T18:47:22.627 INFO:teuthology.orchestra.run.smithi018.stdout: "client_shutdown_timeout": "30", 2024-09-10T18:47:22.627 INFO:teuthology.orchestra.run.smithi018.stdout: "client_snapdir": ".snap", 2024-09-10T18:47:22.627 INFO:teuthology.orchestra.run.smithi018.stdout: "client_tick_interval": "1", 2024-09-10T18:47:22.628 INFO:teuthology.orchestra.run.smithi018.stdout: "client_trace": "", 2024-09-10T18:47:22.628 INFO:teuthology.orchestra.run.smithi018.stdout: "client_try_dentry_invalidate": "false", 2024-09-10T18:47:22.628 INFO:teuthology.orchestra.run.smithi018.stdout: "client_use_faked_inos": "false", 2024-09-10T18:47:22.628 INFO:teuthology.orchestra.run.smithi018.stdout: "client_use_random_mds": "false", 2024-09-10T18:47:22.628 INFO:teuthology.orchestra.run.smithi018.stdout: "clog_to_graylog": "false", 2024-09-10T18:47:22.628 INFO:teuthology.orchestra.run.smithi018.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-10T18:47:22.628 INFO:teuthology.orchestra.run.smithi018.stdout: "clog_to_graylog_port": "12201", 2024-09-10T18:47:22.628 INFO:teuthology.orchestra.run.smithi018.stdout: "clog_to_monitors": "default=true", 2024-09-10T18:47:22.628 INFO:teuthology.orchestra.run.smithi018.stdout: "clog_to_syslog": "false", 2024-09-10T18:47:22.628 INFO:teuthology.orchestra.run.smithi018.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-10T18:47:22.628 INFO:teuthology.orchestra.run.smithi018.stdout: "clog_to_syslog_level": "info", 2024-09-10T18:47:22.628 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addr": "-", 2024-09-10T18:47:22.629 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_network": "", 2024-09-10T18:47:22.629 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_network_interface": "", 2024-09-10T18:47:22.629 INFO:teuthology.orchestra.run.smithi018.stdout: "compressor_zlib_isal": "false", 2024-09-10T18:47:22.629 INFO:teuthology.orchestra.run.smithi018.stdout: "compressor_zlib_level": "5", 2024-09-10T18:47:22.629 INFO:teuthology.orchestra.run.smithi018.stdout: "compressor_zlib_winsize": "-15", 2024-09-10T18:47:22.629 INFO:teuthology.orchestra.run.smithi018.stdout: "compressor_zstd_level": "1", 2024-09-10T18:47:22.629 INFO:teuthology.orchestra.run.smithi018.stdout: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16", 2024-09-10T18:47:22.629 INFO:teuthology.orchestra.run.smithi018.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-10T18:47:22.629 INFO:teuthology.orchestra.run.smithi018.stdout: "crimson_alien_op_num_threads": "6", 2024-09-10T18:47:22.629 INFO:teuthology.orchestra.run.smithi018.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-10T18:47:22.629 INFO:teuthology.orchestra.run.smithi018.stdout: "crimson_osd_obc_lru_size": "10", 2024-09-10T18:47:22.629 INFO:teuthology.orchestra.run.smithi018.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-10T18:47:22.630 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_location": "", 2024-09-10T18:47:22.630 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_location_hook": "", 2024-09-10T18:47:22.630 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_location_hook_timeout": "10", 2024-09-10T18:47:22.630 INFO:teuthology.orchestra.run.smithi018.stdout: "daemonize": "false", 2024-09-10T18:47:22.630 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_alienstore": "0/5", 2024-09-10T18:47:22.630 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_allow_any_pool_priority": "false", 2024-09-10T18:47:22.630 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_asok": "1/5", 2024-09-10T18:47:22.630 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_asok_assert_abort": "false", 2024-09-10T18:47:22.630 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_asserts_on_shutdown": "true", 2024-09-10T18:47:22.630 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_auth": "1/5", 2024-09-10T18:47:22.630 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_bdev": "1/3", 2024-09-10T18:47:22.630 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_bluefs": "1/5", 2024-09-10T18:47:22.630 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_bluestore": "1/5", 2024-09-10T18:47:22.631 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_buffer": "0/1", 2024-09-10T18:47:22.631 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_ceph_exporter": "1/5", 2024-09-10T18:47:22.631 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_cephfs_mirror": "0/5", 2024-09-10T18:47:22.631 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_cephsqlite": "0/5", 2024-09-10T18:47:22.631 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_civetweb": "1/10", 2024-09-10T18:47:22.631 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_client": "0/5", 2024-09-10T18:47:22.631 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_compressor": "1/5", 2024-09-10T18:47:22.631 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_context": "0/1", 2024-09-10T18:47:22.631 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_crush": "1/1", 2024-09-10T18:47:22.631 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_crypto": "1/5", 2024-09-10T18:47:22.631 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_deliberately_leak_memory": "false", 2024-09-10T18:47:22.631 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_disable_randomized_ping": "false", 2024-09-10T18:47:22.631 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_dpdk": "1/5", 2024-09-10T18:47:22.631 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_eventtrace": "1/5", 2024-09-10T18:47:22.632 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_filer": "0/1", 2024-09-10T18:47:22.632 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_filestore": "1/3", 2024-09-10T18:47:22.632 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_finisher": "1/1", 2024-09-10T18:47:22.632 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_fuse": "1/5", 2024-09-10T18:47:22.632 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_heartbeat_testing_span": "0", 2024-09-10T18:47:22.632 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_heartbeatmap": "1/5", 2024-09-10T18:47:22.632 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-10T18:47:22.632 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_javaclient": "1/5", 2024-09-10T18:47:22.632 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_journal": "1/3", 2024-09-10T18:47:22.632 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_journaler": "0/5", 2024-09-10T18:47:22.632 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_kstore": "1/5", 2024-09-10T18:47:22.632 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_leveldb": "4/5", 2024-09-10T18:47:22.632 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_lockdep": "0/1", 2024-09-10T18:47:22.633 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_mclock": "1/5", 2024-09-10T18:47:22.633 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_mds": "1/5", 2024-09-10T18:47:22.633 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_mds_balancer": "1/5", 2024-09-10T18:47:22.633 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_mds_locker": "1/5", 2024-09-10T18:47:22.633 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_mds_log": "1/5", 2024-09-10T18:47:22.633 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_mds_log_expire": "1/5", 2024-09-10T18:47:22.633 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_mds_migrator": "1/5", 2024-09-10T18:47:22.633 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_memdb": "4/5", 2024-09-10T18:47:22.633 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_mgr": "2/5", 2024-09-10T18:47:22.633 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_mgrc": "1/5", 2024-09-10T18:47:22.633 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_mon": "20/20", 2024-09-10T18:47:22.633 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_monc": "0/10", 2024-09-10T18:47:22.633 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_ms": "1/1", 2024-09-10T18:47:22.634 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_none": "0/5", 2024-09-10T18:47:22.634 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_objclass": "0/5", 2024-09-10T18:47:22.634 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_objectcacher": "0/5", 2024-09-10T18:47:22.634 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_objecter": "0/1", 2024-09-10T18:47:22.634 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_optracker": "0/5", 2024-09-10T18:47:22.634 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_osd": "1/5", 2024-09-10T18:47:22.634 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_paxos": "20/20", 2024-09-10T18:47:22.634 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_perfcounter": "1/5", 2024-09-10T18:47:22.634 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_prioritycache": "1/5", 2024-09-10T18:47:22.634 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_rados": "0/5", 2024-09-10T18:47:22.634 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_rbd": "0/5", 2024-09-10T18:47:22.634 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_rbd_mirror": "0/5", 2024-09-10T18:47:22.634 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_rbd_pwl": "0/5", 2024-09-10T18:47:22.635 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_rbd_replay": "0/5", 2024-09-10T18:47:22.635 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_refs": "0/0", 2024-09-10T18:47:22.635 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_reserver": "1/1", 2024-09-10T18:47:22.635 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_rgw": "1/5", 2024-09-10T18:47:22.635 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_rgw_datacache": "1/5", 2024-09-10T18:47:22.635 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_rgw_sync": "1/5", 2024-09-10T18:47:22.635 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_rocksdb": "4/5", 2024-09-10T18:47:22.635 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_seastore": "0/5", 2024-09-10T18:47:22.635 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_seastore_cache": "0/5", 2024-09-10T18:47:22.635 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_seastore_cleaner": "0/5", 2024-09-10T18:47:22.635 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_seastore_device": "0/5", 2024-09-10T18:47:22.635 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_seastore_journal": "0/5", 2024-09-10T18:47:22.635 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_seastore_lba": "0/5", 2024-09-10T18:47:22.635 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_seastore_odata": "0/5", 2024-09-10T18:47:22.635 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_seastore_omap": "0/5", 2024-09-10T18:47:22.636 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_seastore_onode": "0/5", 2024-09-10T18:47:22.636 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_seastore_tm": "0/5", 2024-09-10T18:47:22.636 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_striper": "0/1", 2024-09-10T18:47:22.636 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_test": "0/5", 2024-09-10T18:47:22.636 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_throttle": "1/1", 2024-09-10T18:47:22.636 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_timer": "0/1", 2024-09-10T18:47:22.636 INFO:teuthology.orchestra.run.smithi018.stdout: "debug_tp": "0/5", 2024-09-10T18:47:22.636 INFO:teuthology.orchestra.run.smithi018.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-10T18:47:22.636 INFO:teuthology.orchestra.run.smithi018.stdout: "device_failure_prediction_mode": "none", 2024-09-10T18:47:22.636 INFO:teuthology.orchestra.run.smithi018.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-10T18:47:22.636 INFO:teuthology.orchestra.run.smithi018.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-10T18:47:22.636 INFO:teuthology.orchestra.run.smithi018.stdout: "err_to_graylog": "false", 2024-09-10T18:47:22.637 INFO:teuthology.orchestra.run.smithi018.stdout: "err_to_journald": "false", 2024-09-10T18:47:22.637 INFO:teuthology.orchestra.run.smithi018.stdout: "err_to_stderr": "true", 2024-09-10T18:47:22.637 INFO:teuthology.orchestra.run.smithi018.stdout: "err_to_syslog": "false", 2024-09-10T18:47:22.637 INFO:teuthology.orchestra.run.smithi018.stdout: "event_tracing": "false", 2024-09-10T18:47:22.637 INFO:teuthology.orchestra.run.smithi018.stdout: "exporter_addr": "0.0.0.0", 2024-09-10T18:47:22.637 INFO:teuthology.orchestra.run.smithi018.stdout: "exporter_http_port": "9926", 2024-09-10T18:47:22.637 INFO:teuthology.orchestra.run.smithi018.stdout: "exporter_prio_limit": "5", 2024-09-10T18:47:22.637 INFO:teuthology.orchestra.run.smithi018.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-10T18:47:22.637 INFO:teuthology.orchestra.run.smithi018.stdout: "exporter_sort_metrics": "true", 2024-09-10T18:47:22.637 INFO:teuthology.orchestra.run.smithi018.stdout: "exporter_stats_period": "5", 2024-09-10T18:47:22.637 INFO:teuthology.orchestra.run.smithi018.stdout: "fake_statfs_for_testing": "0", 2024-09-10T18:47:22.639 INFO:teuthology.orchestra.run.smithi018.stdout: "fatal_signal_handlers": "true", 2024-09-10T18:47:22.639 INFO:teuthology.orchestra.run.smithi018.stdout: "filer_max_purge_ops": "10", 2024-09-10T18:47:22.639 INFO:teuthology.orchestra.run.smithi018.stdout: "filer_max_truncate_ops": "128", 2024-09-10T18:47:22.639 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_apply_finisher_threads": "1", 2024-09-10T18:47:22.639 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_blackhole": "false", 2024-09-10T18:47:22.639 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_btrfs_clone_range": "true", 2024-09-10T18:47:22.639 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_btrfs_snap": "true", 2024-09-10T18:47:22.639 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_caller_concurrency": "10", 2024-09-10T18:47:22.639 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_collect_device_partition_information": "true", 2024-09-10T18:47:22.639 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_commit_timeout": "600.000000", 2024-09-10T18:47:22.639 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_debug_inject_read_err": "false", 2024-09-10T18:47:22.639 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_debug_omap_check": "false", 2024-09-10T18:47:22.640 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-10T18:47:22.640 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_debug_verify_split": "false", 2024-09-10T18:47:22.640 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_dump_file": "", 2024-09-10T18:47:22.640 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-10T18:47:22.640 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-10T18:47:22.640 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_fadvise": "true", 2024-09-10T18:47:22.640 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_fail_eio": "true", 2024-09-10T18:47:22.640 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_fd_cache_shards": "16", 2024-09-10T18:47:22.640 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_fd_cache_size": "128", 2024-09-10T18:47:22.640 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_fiemap": "false", 2024-09-10T18:47:22.641 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_fiemap_threshold": "4096", 2024-09-10T18:47:22.641 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-10T18:47:22.641 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-10T18:47:22.641 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_inject_stall": "0", 2024-09-10T18:47:22.641 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_journal_parallel": "false", 2024-09-10T18:47:22.641 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_journal_trailing": "false", 2024-09-10T18:47:22.641 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_journal_writeahead": "false", 2024-09-10T18:47:22.641 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_kill_at": "0", 2024-09-10T18:47:22.641 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-10T18:47:22.641 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-10T18:47:22.641 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-10T18:47:22.641 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-10T18:47:22.641 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-10T18:47:22.642 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_max_inline_xattrs": "0", 2024-09-10T18:47:22.642 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-10T18:47:22.642 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-10T18:47:22.642 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-10T18:47:22.642 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-10T18:47:22.642 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_max_xattr_value_size": "0", 2024-09-10T18:47:22.642 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-10T18:47:22.642 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-10T18:47:22.642 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-10T18:47:22.642 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_merge_threshold": "-10", 2024-09-10T18:47:22.642 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-10T18:47:22.642 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_odsync_write": "false", 2024-09-10T18:47:22.642 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_omap_backend": "rocksdb", 2024-09-10T18:47:22.643 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_omap_backend_path": "", 2024-09-10T18:47:22.643 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-10T18:47:22.643 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-10T18:47:22.643 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-10T18:47:22.643 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_op_thread_timeout": "60", 2024-09-10T18:47:22.643 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_op_threads": "2", 2024-09-10T18:47:22.643 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_punch_hole": "false", 2024-09-10T18:47:22.643 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-10T18:47:22.643 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-10T18:47:22.643 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-10T18:47:22.643 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-10T18:47:22.643 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-10T18:47:22.644 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-10T18:47:22.644 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-10T18:47:22.644 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-10T18:47:22.644 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-10T18:47:22.644 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_queue_max_ops": "50", 2024-09-10T18:47:22.644 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-10T18:47:22.644 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_seek_data_hole": "false", 2024-09-10T18:47:22.644 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_sloppy_crc": "false", 2024-09-10T18:47:22.644 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-10T18:47:22.644 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_splice": "false", 2024-09-10T18:47:22.644 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_split_multiple": "2", 2024-09-10T18:47:22.645 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_split_rand_factor": "20", 2024-09-10T18:47:22.645 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_update_to": "1000", 2024-09-10T18:47:22.645 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-10T18:47:22.645 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-10T18:47:22.645 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-10T18:47:22.645 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-10T18:47:22.645 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-10T18:47:22.645 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-10T18:47:22.645 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_wbthrottle_enable": "true", 2024-09-10T18:47:22.646 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-10T18:47:22.646 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-10T18:47:22.646 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-10T18:47:22.646 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-10T18:47:22.646 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-10T18:47:22.646 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-10T18:47:22.646 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_xfs_extsize": "false", 2024-09-10T18:47:22.646 INFO:teuthology.orchestra.run.smithi018.stdout: "filestore_zfs_snap": "false", 2024-09-10T18:47:22.646 INFO:teuthology.orchestra.run.smithi018.stdout: "fio_dir": "/tmp/fio", 2024-09-10T18:47:22.646 INFO:teuthology.orchestra.run.smithi018.stdout: "fsid": "ba327f6c-6fa3-11ef-bcea-c7b262605968", 2024-09-10T18:47:22.646 INFO:teuthology.orchestra.run.smithi018.stdout: "fuse_allow_other": "true", 2024-09-10T18:47:22.647 INFO:teuthology.orchestra.run.smithi018.stdout: "fuse_atomic_o_trunc": "true", 2024-09-10T18:47:22.647 INFO:teuthology.orchestra.run.smithi018.stdout: "fuse_big_writes": "true", 2024-09-10T18:47:22.647 INFO:teuthology.orchestra.run.smithi018.stdout: "fuse_debug": "false", 2024-09-10T18:47:22.647 INFO:teuthology.orchestra.run.smithi018.stdout: "fuse_default_permissions": "false", 2024-09-10T18:47:22.647 INFO:teuthology.orchestra.run.smithi018.stdout: "fuse_disable_pagecache": "false", 2024-09-10T18:47:22.647 INFO:teuthology.orchestra.run.smithi018.stdout: "fuse_max_write": "0", 2024-09-10T18:47:22.647 INFO:teuthology.orchestra.run.smithi018.stdout: "fuse_multithreaded": "true", 2024-09-10T18:47:22.647 INFO:teuthology.orchestra.run.smithi018.stdout: "fuse_require_active_mds": "true", 2024-09-10T18:47:22.647 INFO:teuthology.orchestra.run.smithi018.stdout: "fuse_set_user_groups": "true", 2024-09-10T18:47:22.647 INFO:teuthology.orchestra.run.smithi018.stdout: "fuse_splice_move": "true", 2024-09-10T18:47:22.647 INFO:teuthology.orchestra.run.smithi018.stdout: "fuse_splice_read": "true", 2024-09-10T18:47:22.648 INFO:teuthology.orchestra.run.smithi018.stdout: "fuse_splice_write": "true", 2024-09-10T18:47:22.648 INFO:teuthology.orchestra.run.smithi018.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-10T18:47:22.648 INFO:teuthology.orchestra.run.smithi018.stdout: "fuse_use_invalidate_cb": "true", 2024-09-10T18:47:22.648 INFO:teuthology.orchestra.run.smithi018.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-10T18:47:22.648 INFO:teuthology.orchestra.run.smithi018.stdout: "gss_target_name": "ceph", 2024-09-10T18:47:22.648 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_file": "", 2024-09-10T18:47:22.648 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_inject_failure": "0", 2024-09-10T18:47:22.648 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_interval": "5", 2024-09-10T18:47:22.648 INFO:teuthology.orchestra.run.smithi018.stdout: "host": "smithi018", 2024-09-10T18:47:22.648 INFO:teuthology.orchestra.run.smithi018.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-10T18:47:22.648 INFO:teuthology.orchestra.run.smithi018.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-10T18:47:22.648 INFO:teuthology.orchestra.run.smithi018.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-10T18:47:22.648 INFO:teuthology.orchestra.run.smithi018.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-10T18:47:22.649 INFO:teuthology.orchestra.run.smithi018.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-10T18:47:22.649 INFO:teuthology.orchestra.run.smithi018.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-10T18:47:22.649 INFO:teuthology.orchestra.run.smithi018.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-10T18:47:22.649 INFO:teuthology.orchestra.run.smithi018.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-10T18:47:22.649 INFO:teuthology.orchestra.run.smithi018.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-10T18:47:22.649 INFO:teuthology.orchestra.run.smithi018.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-10T18:47:22.649 INFO:teuthology.orchestra.run.smithi018.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-10T18:47:22.649 INFO:teuthology.orchestra.run.smithi018.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-10T18:47:22.649 INFO:teuthology.orchestra.run.smithi018.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-10T18:47:22.649 INFO:teuthology.orchestra.run.smithi018.stdout: "inject_early_sigterm": "false", 2024-09-10T18:47:22.650 INFO:teuthology.orchestra.run.smithi018.stdout: "jaeger_tracing_enable": "false", 2024-09-10T18:47:22.650 INFO:teuthology.orchestra.run.smithi018.stdout: "journal_aio": "true", 2024-09-10T18:47:22.650 INFO:teuthology.orchestra.run.smithi018.stdout: "journal_align_min_size": "65536", 2024-09-10T18:47:22.650 INFO:teuthology.orchestra.run.smithi018.stdout: "journal_block_align": "true", 2024-09-10T18:47:22.650 INFO:teuthology.orchestra.run.smithi018.stdout: "journal_block_size": "4096", 2024-09-10T18:47:22.650 INFO:teuthology.orchestra.run.smithi018.stdout: "journal_dio": "true", 2024-09-10T18:47:22.650 INFO:teuthology.orchestra.run.smithi018.stdout: "journal_discard": "false", 2024-09-10T18:47:22.650 INFO:teuthology.orchestra.run.smithi018.stdout: "journal_force_aio": "false", 2024-09-10T18:47:22.650 INFO:teuthology.orchestra.run.smithi018.stdout: "journal_ignore_corruption": "false", 2024-09-10T18:47:22.650 INFO:teuthology.orchestra.run.smithi018.stdout: "journal_max_write_bytes": "10485760", 2024-09-10T18:47:22.650 INFO:teuthology.orchestra.run.smithi018.stdout: "journal_max_write_entries": "100", 2024-09-10T18:47:22.650 INFO:teuthology.orchestra.run.smithi018.stdout: "journal_replay_from": "0", 2024-09-10T18:47:22.650 INFO:teuthology.orchestra.run.smithi018.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-10T18:47:22.650 INFO:teuthology.orchestra.run.smithi018.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-10T18:47:22.650 INFO:teuthology.orchestra.run.smithi018.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-10T18:47:22.651 INFO:teuthology.orchestra.run.smithi018.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-10T18:47:22.651 INFO:teuthology.orchestra.run.smithi018.stdout: "journal_write_header_frequency": "0", 2024-09-10T18:47:22.651 INFO:teuthology.orchestra.run.smithi018.stdout: "journal_zero_on_create": "false", 2024-09-10T18:47:22.651 INFO:teuthology.orchestra.run.smithi018.stdout: "journaler_prefetch_periods": "10", 2024-09-10T18:47:22.651 INFO:teuthology.orchestra.run.smithi018.stdout: "journaler_prezero_periods": "5", 2024-09-10T18:47:22.651 INFO:teuthology.orchestra.run.smithi018.stdout: "journaler_write_head_interval": "15", 2024-09-10T18:47:22.651 INFO:teuthology.orchestra.run.smithi018.stdout: "key": "", 2024-09-10T18:47:22.651 INFO:teuthology.orchestra.run.smithi018.stdout: "keyfile": "", 2024-09-10T18:47:22.651 INFO:teuthology.orchestra.run.smithi018.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-10T18:47:22.651 INFO:teuthology.orchestra.run.smithi018.stdout: "kstore_backend": "rocksdb", 2024-09-10T18:47:22.651 INFO:teuthology.orchestra.run.smithi018.stdout: "kstore_default_stripe_size": "65536", 2024-09-10T18:47:22.651 INFO:teuthology.orchestra.run.smithi018.stdout: "kstore_fsck_on_mount": "false", 2024-09-10T18:47:22.651 INFO:teuthology.orchestra.run.smithi018.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-10T18:47:22.651 INFO:teuthology.orchestra.run.smithi018.stdout: "kstore_max_bytes": "67108864", 2024-09-10T18:47:22.652 INFO:teuthology.orchestra.run.smithi018.stdout: "kstore_max_ops": "512", 2024-09-10T18:47:22.652 INFO:teuthology.orchestra.run.smithi018.stdout: "kstore_nid_prealloc": "1024", 2024-09-10T18:47:22.652 INFO:teuthology.orchestra.run.smithi018.stdout: "kstore_onode_map_size": "1024", 2024-09-10T18:47:22.652 INFO:teuthology.orchestra.run.smithi018.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-10T18:47:22.652 INFO:teuthology.orchestra.run.smithi018.stdout: "kstore_sync_submit_transaction": "false", 2024-09-10T18:47:22.652 INFO:teuthology.orchestra.run.smithi018.stdout: "kstore_sync_transaction": "false", 2024-09-10T18:47:22.652 INFO:teuthology.orchestra.run.smithi018.stdout: "leveldb_block_size": "65536", 2024-09-10T18:47:22.652 INFO:teuthology.orchestra.run.smithi018.stdout: "leveldb_bloom_size": "0", 2024-09-10T18:47:22.652 INFO:teuthology.orchestra.run.smithi018.stdout: "leveldb_cache_size": "536870912", 2024-09-10T18:47:22.652 INFO:teuthology.orchestra.run.smithi018.stdout: "leveldb_compact_on_mount": "false", 2024-09-10T18:47:22.652 INFO:teuthology.orchestra.run.smithi018.stdout: "leveldb_compression": "false", 2024-09-10T18:47:22.652 INFO:teuthology.orchestra.run.smithi018.stdout: "leveldb_log": "", 2024-09-10T18:47:22.653 INFO:teuthology.orchestra.run.smithi018.stdout: "leveldb_log_to_ceph_log": "true", 2024-09-10T18:47:22.653 INFO:teuthology.orchestra.run.smithi018.stdout: "leveldb_max_open_files": "0", 2024-09-10T18:47:22.653 INFO:teuthology.orchestra.run.smithi018.stdout: "leveldb_paranoid": "false", 2024-09-10T18:47:22.653 INFO:teuthology.orchestra.run.smithi018.stdout: "leveldb_write_buffer_size": "33554432", 2024-09-10T18:47:22.653 INFO:teuthology.orchestra.run.smithi018.stdout: "librados_thread_count": "2", 2024-09-10T18:47:22.653 INFO:teuthology.orchestra.run.smithi018.stdout: "lockdep": "false", 2024-09-10T18:47:22.653 INFO:teuthology.orchestra.run.smithi018.stdout: "lockdep_force_backtrace": "false", 2024-09-10T18:47:22.653 INFO:teuthology.orchestra.run.smithi018.stdout: "log_coarse_timestamps": "true", 2024-09-10T18:47:22.653 INFO:teuthology.orchestra.run.smithi018.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-10T18:47:22.653 INFO:teuthology.orchestra.run.smithi018.stdout: "log_flush_on_exit": "false", 2024-09-10T18:47:22.653 INFO:teuthology.orchestra.run.smithi018.stdout: "log_graylog_host": "127.0.0.1", 2024-09-10T18:47:22.653 INFO:teuthology.orchestra.run.smithi018.stdout: "log_graylog_port": "12201", 2024-09-10T18:47:22.653 INFO:teuthology.orchestra.run.smithi018.stdout: "log_max_new": "1000", 2024-09-10T18:47:22.654 INFO:teuthology.orchestra.run.smithi018.stdout: "log_max_recent": "10000", 2024-09-10T18:47:22.654 INFO:teuthology.orchestra.run.smithi018.stdout: "log_stderr_prefix": "", 2024-09-10T18:47:22.654 INFO:teuthology.orchestra.run.smithi018.stdout: "log_stop_at_utilization": "0.970000", 2024-09-10T18:47:22.654 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_file": "true", 2024-09-10T18:47:22.654 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_graylog": "false", 2024-09-10T18:47:22.654 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_journald": "false", 2024-09-10T18:47:22.654 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_stderr": "false", 2024-09-10T18:47:22.654 INFO:teuthology.orchestra.run.smithi018.stdout: "log_to_syslog": "false", 2024-09-10T18:47:22.654 INFO:teuthology.orchestra.run.smithi018.stdout: "max_rotating_auth_attempts": "10", 2024-09-10T18:47:22.654 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-10T18:47:22.654 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_action_on_write_error": "1", 2024-09-10T18:47:22.654 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_alternate_name_max": "8192", 2024-09-10T18:47:22.654 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_asio_thread_count": "2", 2024-09-10T18:47:22.654 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_bal_export_pin": "true", 2024-09-10T18:47:22.655 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_bal_fragment_dirs": "true", 2024-09-10T18:47:22.655 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-10T18:47:22.655 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_bal_fragment_interval": "5", 2024-09-10T18:47:22.655 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-10T18:47:22.655 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-10T18:47:22.656 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_bal_interval": "10", 2024-09-10T18:47:22.656 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_bal_max": "-1", 2024-09-10T18:47:22.656 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_bal_max_until": "-1", 2024-09-10T18:47:22.656 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_bal_merge_size": "50", 2024-09-10T18:47:22.656 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_bal_midchunk": "0.300000", 2024-09-10T18:47:22.656 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-10T18:47:22.656 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_bal_min_start": "0.200000", 2024-09-10T18:47:22.656 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_bal_minchunk": "0.001000", 2024-09-10T18:47:22.657 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_bal_mode": "0", 2024-09-10T18:47:22.657 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_bal_need_max": "1.200000", 2024-09-10T18:47:22.657 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_bal_need_min": "0.800000", 2024-09-10T18:47:22.657 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-10T18:47:22.657 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-10T18:47:22.657 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_bal_split_bits": "3", 2024-09-10T18:47:22.657 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-10T18:47:22.657 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_bal_split_size": "10000", 2024-09-10T18:47:22.657 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-10T18:47:22.657 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_bal_target_decay": "10.000000", 2024-09-10T18:47:22.658 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-10T18:47:22.658 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_beacon_grace": "15.000000", 2024-09-10T18:47:22.658 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_beacon_interval": "4.000000", 2024-09-10T18:47:22.658 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-10T18:47:22.658 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-10T18:47:22.658 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_cache_mid": "0.700000", 2024-09-10T18:47:22.658 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_cache_release_free_interval": "10", 2024-09-10T18:47:22.658 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_cache_reservation": "0.050000", 2024-09-10T18:47:22.658 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-10T18:47:22.658 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_cache_trim_interval": "1", 2024-09-10T18:47:22.658 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_cache_trim_threshold": "262144", 2024-09-10T18:47:22.658 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-10T18:47:22.659 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-10T18:47:22.659 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-10T18:47:22.659 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_client_prealloc_inos": "1000", 2024-09-10T18:47:22.659 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-10T18:47:22.659 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_connect_bootstrapping": "false", 2024-09-10T18:47:22.659 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_damage_table_max_entries": "10000", 2024-09-10T18:47:22.659 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-10T18:47:22.659 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_debug_auth_pins": "false", 2024-09-10T18:47:22.659 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_debug_frag": "false", 2024-09-10T18:47:22.659 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_debug_scatterstat": "false", 2024-09-10T18:47:22.659 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_debug_subtrees": "false", 2024-09-10T18:47:22.659 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_decay_halflife": "5.000000", 2024-09-10T18:47:22.660 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_default_dir_hash": "2", 2024-09-10T18:47:22.660 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_defer_session_stale": "true", 2024-09-10T18:47:22.660 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_deny_all_reconnect": "false", 2024-09-10T18:47:22.660 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_dir_keys_per_op": "16384", 2024-09-10T18:47:22.660 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_dir_max_commit_size": "10", 2024-09-10T18:47:22.660 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_dir_max_entries": "0", 2024-09-10T18:47:22.660 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-10T18:47:22.660 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-10T18:47:22.660 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_dump_cache_on_map": "false", 2024-09-10T18:47:22.660 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-10T18:47:22.660 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-10T18:47:22.661 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_early_reply": "true", 2024-09-10T18:47:22.661 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_enable_op_tracker": "true", 2024-09-10T18:47:22.661 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_enforce_unique_name": "true", 2024-09-10T18:47:22.661 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-10T18:47:22.661 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-10T18:47:22.661 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_export_ephemeral_random": "true", 2024-09-10T18:47:22.661 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-10T18:47:22.661 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-10T18:47:22.661 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-10T18:47:22.661 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-10T18:47:22.661 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-10T18:47:22.662 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-10T18:47:22.662 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_health_summarize_threshold": "10", 2024-09-10T18:47:22.662 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-10T18:47:22.662 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-10T18:47:22.662 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_inject_health_dummy": "false", 2024-09-10T18:47:22.662 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-10T18:47:22.662 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_inject_migrator_session_race": "false", 2024-09-10T18:47:22.662 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-10T18:47:22.662 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-10T18:47:22.662 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-10T18:47:22.662 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_join_fs": "", 2024-09-10T18:47:22.663 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_journal_format": "1", 2024-09-10T18:47:22.663 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_kill_create_at": "0", 2024-09-10T18:47:22.663 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_kill_export_at": "0", 2024-09-10T18:47:22.663 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_kill_import_at": "0", 2024-09-10T18:47:22.663 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_kill_journal_at": "0", 2024-09-10T18:47:22.663 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_kill_journal_expire_at": "0", 2024-09-10T18:47:22.663 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_kill_journal_replay_at": "0", 2024-09-10T18:47:22.663 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_kill_link_at": "0", 2024-09-10T18:47:22.663 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_kill_mdstable_at": "0", 2024-09-10T18:47:22.663 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_kill_openc_at": "0", 2024-09-10T18:47:22.663 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_kill_rename_at": "0", 2024-09-10T18:47:22.664 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_kill_skip_replaying_inotable": "false", 2024-09-10T18:47:22.664 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_log_events_per_segment": "1024", 2024-09-10T18:47:22.664 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_log_max_events": "-1", 2024-09-10T18:47:22.664 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_log_max_segments": "128", 2024-09-10T18:47:22.664 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_log_pause": "false", 2024-09-10T18:47:22.664 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_log_segment_size": "0", 2024-09-10T18:47:22.664 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-10T18:47:22.664 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_log_warn_factor": "2.000000", 2024-09-10T18:47:22.664 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_max_caps_per_client": "1048576", 2024-09-10T18:47:22.664 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_max_completed_flushes": "100000", 2024-09-10T18:47:22.664 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_max_completed_requests": "100000", 2024-09-10T18:47:22.665 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_max_export_size": "20971520", 2024-09-10T18:47:22.665 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_max_file_recover": "32", 2024-09-10T18:47:22.665 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_max_purge_files": "64", 2024-09-10T18:47:22.665 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_max_purge_ops": "8192", 2024-09-10T18:47:22.665 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-10T18:47:22.665 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-10T18:47:22.665 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_max_snaps_per_dir": "100", 2024-09-10T18:47:22.665 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_max_xattr_pairs_size": "65536", 2024-09-10T18:47:22.665 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_metrics_update_interval": "2", 2024-09-10T18:47:22.665 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_min_caps_per_client": "100", 2024-09-10T18:47:22.665 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_min_caps_working_set": "10000", 2024-09-10T18:47:22.665 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-10T18:47:22.666 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_numa_node": "-1", 2024-09-10T18:47:22.666 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-10T18:47:22.666 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_op_complaint_time": "30.000000", 2024-09-10T18:47:22.666 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_op_history_duration": "600", 2024-09-10T18:47:22.666 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_op_history_size": "20", 2024-09-10T18:47:22.666 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_op_history_slow_op_size": "20", 2024-09-10T18:47:22.666 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-10T18:47:22.666 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_op_log_threshold": "5", 2024-09-10T18:47:22.666 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_ping_grace": "15", 2024-09-10T18:47:22.666 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_ping_interval": "5", 2024-09-10T18:47:22.666 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-10T18:47:22.667 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-10T18:47:22.667 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_recall_max_caps": "30000", 2024-09-10T18:47:22.667 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-10T18:47:22.667 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-10T18:47:22.667 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-10T18:47:22.667 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_recall_warning_threshold": "262144", 2024-09-10T18:47:22.667 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-10T18:47:22.667 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_replay_interval": "1.000000", 2024-09-10T18:47:22.667 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-10T18:47:22.667 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-10T18:47:22.667 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_root_ino_gid": "0", 2024-09-10T18:47:22.667 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_root_ino_uid": "0", 2024-09-10T18:47:22.668 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-10T18:47:22.668 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-10T18:47:22.668 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-10T18:47:22.668 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-10T18:47:22.668 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-10T18:47:22.668 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-10T18:47:22.668 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-10T18:47:22.668 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-10T18:47:22.668 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-10T18:47:22.668 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-10T18:47:22.668 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_shutdown_check": "0", 2024-09-10T18:47:22.668 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_skip_ino": "0", 2024-09-10T18:47:22.669 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-10T18:47:22.669 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_snap_max_uid": "4294967294", 2024-09-10T18:47:22.669 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_snap_min_uid": "0", 2024-09-10T18:47:22.669 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_snap_rstat": "false", 2024-09-10T18:47:22.669 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_standby_replay_damaged": "false", 2024-09-10T18:47:22.669 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_symlink_recovery": "true", 2024-09-10T18:47:22.669 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-10T18:47:22.669 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_thrash_exports": "0", 2024-09-10T18:47:22.669 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_thrash_fragments": "0", 2024-09-10T18:47:22.669 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_tick_interval": "5.000000", 2024-09-10T18:47:22.669 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_valgrind_exit": "false", 2024-09-10T18:47:22.670 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_verify_backtrace": "1", 2024-09-10T18:47:22.670 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_verify_scatter": "false", 2024-09-10T18:47:22.670 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-10T18:47:22.670 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_wipe_sessions": "false", 2024-09-10T18:47:22.670 INFO:teuthology.orchestra.run.smithi018.stdout: "mempool_debug": "false", 2024-09-10T18:47:22.670 INFO:teuthology.orchestra.run.smithi018.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-10T18:47:22.670 INFO:teuthology.orchestra.run.smithi018.stdout: "memstore_device_bytes": "1073741824", 2024-09-10T18:47:22.670 INFO:teuthology.orchestra.run.smithi018.stdout: "memstore_page_set": "false", 2024-09-10T18:47:22.670 INFO:teuthology.orchestra.run.smithi018.stdout: "memstore_page_size": "65536", 2024-09-10T18:47:22.670 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr_client_bytes": "134217728", 2024-09-10T18:47:22.670 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr_client_messages": "512", 2024-09-10T18:47:22.670 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-10T18:47:22.671 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-10T18:47:22.671 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-10T18:47:22.671 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-10T18:47:22.671 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr_disabled_modules": "diskprediction_local", 2024-09-10T18:47:22.671 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-10T18:47:22.671 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr_max_pg_num_change": "128", 2024-09-10T18:47:22.671 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr_mds_bytes": "134217728", 2024-09-10T18:47:22.671 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr_mds_messages": "128", 2024-09-10T18:47:22.671 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-10T18:47:22.671 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr_mon_bytes": "134217728", 2024-09-10T18:47:22.671 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr_mon_messages": "128", 2024-09-10T18:47:22.671 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr_osd_bytes": "536870912", 2024-09-10T18:47:22.672 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr_osd_messages": "8192", 2024-09-10T18:47:22.672 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr_pool": "true", 2024-09-10T18:47:22.672 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-10T18:47:22.672 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr_standby_modules": "true", 2024-09-10T18:47:22.672 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr_stats_period": "5", 2024-09-10T18:47:22.672 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr_stats_threshold": "5", 2024-09-10T18:47:22.672 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr_tick_period": "2", 2024-09-10T18:47:22.672 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-10T18:47:22.672 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-10T18:47:22.672 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_allow_pool_delete": "true", 2024-09-10T18:47:22.672 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_allow_pool_size_one": "false", 2024-09-10T18:47:22.673 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_auth_validate_all_caps": "true", 2024-09-10T18:47:22.673 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-10T18:47:22.673 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-10T18:47:22.673 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_client_bytes": "104857600", 2024-09-10T18:47:22.673 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_client_directed_command_retry": "2", 2024-09-10T18:47:22.673 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-10T18:47:22.673 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-10T18:47:22.673 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-10T18:47:22.673 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-10T18:47:22.673 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_client_hunt_parallel": "3", 2024-09-10T18:47:22.673 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_client_log_interval": "1.000000", 2024-09-10T18:47:22.674 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-10T18:47:22.674 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_client_ping_interval": "10.000000", 2024-09-10T18:47:22.674 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-10T18:47:22.674 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-10T18:47:22.674 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-10T18:47:22.674 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-10T18:47:22.674 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_cluster_log_level": "debug", 2024-09-10T18:47:22.674 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_cluster_log_to_file": "false", 2024-09-10T18:47:22.674 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-10T18:47:22.674 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-10T18:47:22.674 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-10T18:47:22.675 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_cluster_log_to_journald": "true", 2024-09-10T18:47:22.675 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-10T18:47:22.675 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-10T18:47:22.675 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-10T18:47:22.675 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_compact_on_bootstrap": "false", 2024-09-10T18:47:22.675 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_compact_on_start": "false", 2024-09-10T18:47:22.675 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_compact_on_trim": "true", 2024-09-10T18:47:22.675 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-10T18:47:22.675 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-10T18:47:22.675 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-10T18:47:22.675 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_cpu_threads": "4", 2024-09-10T18:47:22.676 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_crush_min_required_version": "hammer", 2024-09-10T18:47:22.676 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_daemon_bytes": "419430400", 2024-09-10T18:47:22.676 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-10T18:47:22.676 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_data_avail_crit": "5", 2024-09-10T18:47:22.676 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_data_avail_warn": "5", 2024-09-10T18:47:22.676 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_data_size_warn": "16106127360", 2024-09-10T18:47:22.676 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-10T18:47:22.676 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-10T18:47:22.676 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_debug_dump_json": "false", 2024-09-10T18:47:22.676 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-10T18:47:22.676 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_debug_dump_transactions": "false", 2024-09-10T18:47:22.677 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_debug_extra_checks": "false", 2024-09-10T18:47:22.677 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-10T18:47:22.677 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-10T18:47:22.677 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_debug_no_require_pacific": "false", 2024-09-10T18:47:22.677 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_debug_no_require_quincy": "false", 2024-09-10T18:47:22.677 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-10T18:47:22.677 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-10T18:47:22.677 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-10T18:47:22.677 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_down_mkfs_grace": "60", 2024-09-10T18:47:22.677 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_election_default_strategy": "1", 2024-09-10T18:47:22.677 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_election_timeout": "5.000000", 2024-09-10T18:47:22.678 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-10T18:47:22.678 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_elector_ping_divisor": "2", 2024-09-10T18:47:22.678 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-10T18:47:22.678 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_enable_op_tracker": "true", 2024-09-10T18:47:22.678 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_fake_pool_delete": "false", 2024-09-10T18:47:22.678 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_force_quorum_join": "false", 2024-09-10T18:47:22.678 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_globalid_prealloc": "10000", 2024-09-10T18:47:22.678 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_health_detail_to_clog": "true", 2024-09-10T18:47:22.678 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_health_log_update_period": "5", 2024-09-10T18:47:22.678 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_health_max_detail": "50", 2024-09-10T18:47:22.678 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_health_to_clog": "true", 2024-09-10T18:47:22.678 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_health_to_clog_interval": "600", 2024-09-10T18:47:22.679 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-10T18:47:22.679 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_host": "[v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0]", 2024-09-10T18:47:22.679 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_host_override": "", 2024-09-10T18:47:22.679 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_initial_members": "", 2024-09-10T18:47:22.679 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-10T18:47:22.679 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-10T18:47:22.679 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-10T18:47:22.679 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-10T18:47:22.679 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-10T18:47:22.679 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_lease": "5.000000", 2024-09-10T18:47:22.680 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-10T18:47:22.680 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-10T18:47:22.680 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_log_full_interval": "50", 2024-09-10T18:47:22.680 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_log_max": "10000", 2024-09-10T18:47:22.680 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_log_max_summary": "50", 2024-09-10T18:47:22.680 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-10T18:47:22.680 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_max_log_epochs": "500", 2024-09-10T18:47:22.680 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-10T18:47:22.680 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-10T18:47:22.680 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_max_osd": "10000", 2024-09-10T18:47:22.680 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_max_pg_per_osd": "10000", 2024-09-10T18:47:22.681 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_max_pool_pg_num": "65536", 2024-09-10T18:47:22.681 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-10T18:47:22.681 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-10T18:47:22.681 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_mds_force_trim_to": "0", 2024-09-10T18:47:22.681 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_mds_skip_sanity": "false", 2024-09-10T18:47:22.681 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_memory_autotune": "true", 2024-09-10T18:47:22.681 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_memory_target": "2147483648", 2024-09-10T18:47:22.681 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_mgr_beacon_grace": "30", 2024-09-10T18:47:22.681 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-10T18:47:22.681 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_mgr_digest_period": "5", 2024-09-10T18:47:22.681 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_mgr_inactive_grace": "60", 2024-09-10T18:47:22.681 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-10T18:47:22.682 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-10T18:47:22.682 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_min_osdmap_epochs": "50", 2024-09-10T18:47:22.682 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_op_complaint_time": "30", 2024-09-10T18:47:22.682 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_op_history_duration": "600", 2024-09-10T18:47:22.682 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_op_history_size": "20", 2024-09-10T18:47:22.682 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_op_history_slow_op_size": "20", 2024-09-10T18:47:22.682 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-10T18:47:22.682 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_op_log_threshold": "5", 2024-09-10T18:47:22.682 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-10T18:47:22.682 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-10T18:47:22.682 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-10T18:47:22.683 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osd_auto_mark_in": "false", 2024-09-10T18:47:22.683 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-10T18:47:22.683 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-10T18:47:22.683 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-10T18:47:22.683 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osd_cache_size": "500", 2024-09-10T18:47:22.683 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-10T18:47:22.683 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-10T18:47:22.683 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-10T18:47:22.683 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osd_down_out_interval": "600", 2024-09-10T18:47:22.683 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-10T18:47:22.683 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-10T18:47:22.683 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osd_force_trim_to": "0", 2024-09-10T18:47:22.684 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-10T18:47:22.684 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-10T18:47:22.684 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-10T18:47:22.684 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-10T18:47:22.684 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-10T18:47:22.684 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-10T18:47:22.684 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-10T18:47:22.684 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-10T18:47:22.684 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osd_min_down_reporters": "2", 2024-09-10T18:47:22.684 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-10T18:47:22.684 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-10T18:47:22.685 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-10T18:47:22.685 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-10T18:47:22.685 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-10T18:47:22.685 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-10T18:47:22.685 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osd_report_timeout": "900", 2024-09-10T18:47:22.685 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-10T18:47:22.685 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-10T18:47:22.685 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-10T18:47:22.685 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-10T18:47:22.685 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-10T18:47:22.685 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osdmap_full_prune_interval": "2", 2024-09-10T18:47:22.685 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osdmap_full_prune_min": "15", 2024-09-10T18:47:22.686 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_osdmap_full_prune_txsize": "2", 2024-09-10T18:47:22.686 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-10T18:47:22.686 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_pg_stuck_threshold": "60", 2024-09-10T18:47:22.686 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-10T18:47:22.686 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-10T18:47:22.686 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-10T18:47:22.686 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-10T18:47:22.686 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-10T18:47:22.686 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-10T18:47:22.686 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_probe_timeout": "2.000000", 2024-09-10T18:47:22.686 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_reweight_max_change": "0.050000", 2024-09-10T18:47:22.686 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_reweight_max_osds": "4", 2024-09-10T18:47:22.687 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-10T18:47:22.687 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-10T18:47:22.687 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-10T18:47:22.687 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-10T18:47:22.687 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-10T18:47:22.687 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_scrub_interval": "86400", 2024-09-10T18:47:22.687 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_scrub_max_keys": "100", 2024-09-10T18:47:22.687 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_scrub_timeout": "300", 2024-09-10T18:47:22.687 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_session_timeout": "300", 2024-09-10T18:47:22.687 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_smart_report_timeout": "5", 2024-09-10T18:47:22.687 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_stat_smooth_intervals": "6", 2024-09-10T18:47:22.688 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-10T18:47:22.688 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_stretch_pool_min_size": "2", 2024-09-10T18:47:22.688 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_stretch_pool_size": "4", 2024-09-10T18:47:22.688 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-10T18:47:22.688 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-10T18:47:22.688 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_sync_debug": "false", 2024-09-10T18:47:22.688 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-10T18:47:22.688 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-10T18:47:22.688 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_sync_provider_kill_at": "0", 2024-09-10T18:47:22.688 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_sync_requester_kill_at": "0", 2024-09-10T18:47:22.688 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_sync_timeout": "60.000000", 2024-09-10T18:47:22.689 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_target_pg_per_osd": "100", 2024-09-10T18:47:22.689 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_tick_interval": "5", 2024-09-10T18:47:22.689 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_timecheck_interval": "300.000000", 2024-09-10T18:47:22.689 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-10T18:47:22.689 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_warn_older_version_delay": "604800", 2024-09-10T18:47:22.689 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-10T18:47:22.689 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-10T18:47:22.689 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-10T18:47:22.689 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_warn_on_filestore_osds": "true", 2024-09-10T18:47:22.689 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-10T18:47:22.689 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-10T18:47:22.690 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-10T18:47:22.690 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_warn_on_misplaced": "false", 2024-09-10T18:47:22.690 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-10T18:47:22.690 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_warn_on_older_version": "true", 2024-09-10T18:47:22.690 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-10T18:47:22.690 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_warn_on_pool_no_app": "true", 2024-09-10T18:47:22.690 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_warn_on_pool_no_redundancy": "true", 2024-09-10T18:47:22.690 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-10T18:47:22.690 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-10T18:47:22.690 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-10T18:47:22.690 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-10T18:47:22.690 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-10T18:47:22.691 INFO:teuthology.orchestra.run.smithi018.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-10T18:47:22.691 INFO:teuthology.orchestra.run.smithi018.stdout: "monmap": "", 2024-09-10T18:47:22.691 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_async_op_threads": "3", 2024-09-10T18:47:22.691 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-10T18:47:22.691 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_async_rdma_cm": "false", 2024-09-10T18:47:22.691 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_async_rdma_device_name": "", 2024-09-10T18:47:22.691 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_async_rdma_dscp": "96", 2024-09-10T18:47:22.691 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-10T18:47:22.691 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-10T18:47:22.691 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_async_rdma_local_gid": "", 2024-09-10T18:47:22.691 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-10T18:47:22.692 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_async_rdma_port_num": "1", 2024-09-10T18:47:22.692 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-10T18:47:22.692 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-10T18:47:22.692 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-10T18:47:22.692 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-10T18:47:22.692 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_async_rdma_sl": "3", 2024-09-10T18:47:22.692 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_async_rdma_support_srq": "true", 2024-09-10T18:47:22.692 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_async_rdma_type": "ib", 2024-09-10T18:47:22.692 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_async_reap_threshold": "5", 2024-09-10T18:47:22.692 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_bind_before_connect": "false", 2024-09-10T18:47:22.692 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_bind_ipv4": "true", 2024-09-10T18:47:22.693 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_bind_ipv6": "false", 2024-09-10T18:47:22.693 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_bind_msgr1": "false", 2024-09-10T18:47:22.693 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_bind_msgr2": "true", 2024-09-10T18:47:22.693 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_bind_port_max": "7568", 2024-09-10T18:47:22.693 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_bind_port_min": "6800", 2024-09-10T18:47:22.693 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-10T18:47:22.693 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_bind_retry_count": "3", 2024-09-10T18:47:22.693 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_bind_retry_delay": "5", 2024-09-10T18:47:22.693 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_blackhole_client": "false", 2024-09-10T18:47:22.693 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_blackhole_mds": "false", 2024-09-10T18:47:22.693 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_blackhole_mgr": "false", 2024-09-10T18:47:22.694 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_blackhole_mon": "false", 2024-09-10T18:47:22.694 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_blackhole_osd": "false", 2024-09-10T18:47:22.694 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_client_mode": "crc secure", 2024-09-10T18:47:22.694 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_cluster_mode": "crc secure", 2024-09-10T18:47:22.694 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_cluster_type": "", 2024-09-10T18:47:22.694 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_compress_secure": "false", 2024-09-10T18:47:22.694 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_connection_idle_timeout": "900", 2024-09-10T18:47:22.694 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_connection_ready_timeout": "10", 2024-09-10T18:47:22.694 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_crc_data": "true", 2024-09-10T18:47:22.694 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_crc_header": "true", 2024-09-10T18:47:22.695 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_die_on_bad_msg": "false", 2024-09-10T18:47:22.695 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_die_on_bug": "true", 2024-09-10T18:47:22.695 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_die_on_old_message": "true", 2024-09-10T18:47:22.695 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_die_on_skipped_message": "false", 2024-09-10T18:47:22.695 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-10T18:47:22.695 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-10T18:47:22.695 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_dpdk_coremask": "0xF", 2024-09-10T18:47:22.695 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-10T18:47:22.695 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-10T18:47:22.695 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_dpdk_enable_tso": "true", 2024-09-10T18:47:22.695 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-10T18:47:22.695 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-10T18:47:22.696 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_dpdk_hugepages": "", 2024-09-10T18:47:22.696 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-10T18:47:22.696 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-10T18:47:22.696 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_dpdk_lro": "true", 2024-09-10T18:47:22.696 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_dpdk_memory_channel": "4", 2024-09-10T18:47:22.696 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-10T18:47:22.696 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_dpdk_pmd": "", 2024-09-10T18:47:22.697 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_dpdk_port_id": "0", 2024-09-10T18:47:22.697 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-10T18:47:22.697 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-10T18:47:22.697 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_dump_on_send": "false", 2024-09-10T18:47:22.697 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_initial_backoff": "0.200000", 2024-09-10T18:47:22.697 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_inject_delay_max": "1.000000", 2024-09-10T18:47:22.697 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-10T18:47:22.697 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_inject_delay_type": "", 2024-09-10T18:47:22.697 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-10T18:47:22.697 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_inject_socket_failures": "0", 2024-09-10T18:47:22.698 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_learn_addr_from_peer": "true", 2024-09-10T18:47:22.698 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_max_accept_failures": "4", 2024-09-10T18:47:22.698 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_max_backoff": "15.000000", 2024-09-10T18:47:22.698 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_mon_client_mode": "secure crc", 2024-09-10T18:47:22.698 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-10T18:47:22.698 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_mon_service_mode": "secure crc", 2024-09-10T18:47:22.698 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_osd_compress_min_size": "1024", 2024-09-10T18:47:22.698 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_osd_compress_mode": "none", 2024-09-10T18:47:22.699 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_osd_compression_algorithm": "snappy zlib zstd lz4", 2024-09-10T18:47:22.699 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-10T18:47:22.699 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_pq_min_cost": "65536", 2024-09-10T18:47:22.699 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_public_type": "", 2024-09-10T18:47:22.699 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_service_mode": "crc secure", 2024-09-10T18:47:22.699 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_tcp_listen_backlog": "512", 2024-09-10T18:47:22.699 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_tcp_nodelay": "true", 2024-09-10T18:47:22.699 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-10T18:47:22.699 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_tcp_rcvbuf": "0", 2024-09-10T18:47:22.699 INFO:teuthology.orchestra.run.smithi018.stdout: "ms_type": "async+posix", 2024-09-10T18:47:22.699 INFO:teuthology.orchestra.run.smithi018.stdout: "no_config_file": "false", 2024-09-10T18:47:22.700 INFO:teuthology.orchestra.run.smithi018.stdout: "objecter_completion_locks_per_session": "32", 2024-09-10T18:47:22.700 INFO:teuthology.orchestra.run.smithi018.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-10T18:47:22.700 INFO:teuthology.orchestra.run.smithi018.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-10T18:47:22.700 INFO:teuthology.orchestra.run.smithi018.stdout: "objecter_inflight_ops": "1024", 2024-09-10T18:47:22.700 INFO:teuthology.orchestra.run.smithi018.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-10T18:47:22.700 INFO:teuthology.orchestra.run.smithi018.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-10T18:47:22.700 INFO:teuthology.orchestra.run.smithi018.stdout: "objecter_tick_interval": "5.000000", 2024-09-10T18:47:22.700 INFO:teuthology.orchestra.run.smithi018.stdout: "objecter_timeout": "10.000000", 2024-09-10T18:47:22.700 INFO:teuthology.orchestra.run.smithi018.stdout: "objectstore_blackhole": "false", 2024-09-10T18:47:22.700 INFO:teuthology.orchestra.run.smithi018.stdout: "openssl_engine_opts": "", 2024-09-10T18:47:22.700 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_agent_delay_time": "5.000000", 2024-09-10T18:47:22.700 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_agent_hist_halflife": "1000", 2024-09-10T18:47:22.701 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_agent_max_low_ops": "2", 2024-09-10T18:47:22.701 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_agent_max_ops": "4", 2024-09-10T18:47:22.701 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-10T18:47:22.701 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-10T18:47:22.701 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_agent_slop": "0.020000", 2024-09-10T18:47:22.701 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-10T18:47:22.701 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-10T18:47:22.701 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_asio_thread_count": "2", 2024-09-10T18:47:22.701 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_async_recovery_min_cost": "100", 2024-09-10T18:47:22.701 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-10T18:47:22.701 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-10T18:47:22.702 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_backfill_scan_max": "512", 2024-09-10T18:47:22.702 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_backfill_scan_min": "64", 2024-09-10T18:47:22.702 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_backoff_on_degraded": "false", 2024-09-10T18:47:22.702 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_backoff_on_peering": "false", 2024-09-10T18:47:22.702 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_backoff_on_unfound": "true", 2024-09-10T18:47:22.702 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_beacon_report_interval": "300", 2024-09-10T18:47:22.702 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_bench_duration": "30", 2024-09-10T18:47:22.702 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-10T18:47:22.702 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_bench_max_block_size": "67108864", 2024-09-10T18:47:22.702 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-10T18:47:22.702 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_blkin_trace_all": "false", 2024-09-10T18:47:22.702 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-10T18:47:22.703 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-10T18:47:22.703 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_check_for_log_corruption": "false", 2024-09-10T18:47:22.703 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-10T18:47:22.703 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-10T18:47:22.703 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-10T18:47:22.703 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-10T18:47:22.703 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_class_update_on_start": "true", 2024-09-10T18:47:22.703 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_client_message_cap": "256", 2024-09-10T18:47:22.703 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_client_message_size_cap": "524288000", 2024-09-10T18:47:22.703 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_client_op_priority": "63", 2024-09-10T18:47:22.703 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_client_watch_timeout": "30", 2024-09-10T18:47:22.704 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_command_max_records": "256", 2024-09-10T18:47:22.704 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_compact_on_start": "false", 2024-09-10T18:47:22.704 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-10T18:47:22.704 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_crush_chooseleaf_type": "1", 2024-09-10T18:47:22.704 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-10T18:47:22.704 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_crush_update_on_start": "true", 2024-09-10T18:47:22.704 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_crush_update_weight_set": "true", 2024-09-10T18:47:22.704 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-10T18:47:22.704 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-10T18:47:22.704 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-10T18:47:22.704 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-10T18:47:22.704 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-10T18:47:22.705 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_debug_feed_pullee": "-1", 2024-09-10T18:47:22.705 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-10T18:47:22.705 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-10T18:47:22.705 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-10T18:47:22.705 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_debug_misdirected_ops": "false", 2024-09-10T18:47:22.705 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_debug_no_acting_change": "false", 2024-09-10T18:47:22.705 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_debug_no_purge_strays": "false", 2024-09-10T18:47:22.705 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_debug_op_order": "false", 2024-09-10T18:47:22.705 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-10T18:47:22.705 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-10T18:47:22.705 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-10T18:47:22.706 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-10T18:47:22.706 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_debug_shutdown": "false", 2024-09-10T18:47:22.706 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-10T18:47:22.706 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-10T18:47:22.706 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-10T18:47:22.706 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-10T18:47:22.706 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_debug_verify_snaps": "false", 2024-09-10T18:47:22.706 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-10T18:47:22.706 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-10T18:47:22.706 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_deep_scrub_keys": "1024", 2024-09-10T18:47:22.706 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-10T18:47:22.707 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-10T18:47:22.707 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-10T18:47:22.707 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_deep_scrub_stride": "524288", 2024-09-10T18:47:22.707 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-10T18:47:22.707 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_default_data_pool_replay_window": "45", 2024-09-10T18:47:22.707 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_default_notify_timeout": "30", 2024-09-10T18:47:22.707 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_delete_sleep": "0.000000", 2024-09-10T18:47:22.707 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-10T18:47:22.707 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-10T18:47:22.707 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-10T18:47:22.707 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_discard_disconnected_ops": "true", 2024-09-10T18:47:22.708 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_enable_op_tracker": "true", 2024-09-10T18:47:22.708 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-10T18:47:22.708 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-10T18:47:22.708 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-10T18:47:22.708 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_fast_info": "true", 2024-09-10T18:47:22.708 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_fast_shutdown": "true", 2024-09-10T18:47:22.708 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-10T18:47:22.708 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-10T18:47:22.708 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-10T18:47:22.708 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-10T18:47:22.708 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-10T18:47:22.709 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_function_tracing": "false", 2024-09-10T18:47:22.709 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_heartbeat_grace": "20", 2024-09-10T18:47:22.709 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_heartbeat_interval": "6", 2024-09-10T18:47:22.709 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-10T18:47:22.709 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_heartbeat_min_peers": "10", 2024-09-10T18:47:22.709 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_heartbeat_min_size": "2000", 2024-09-10T18:47:22.709 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_heartbeat_stale": "600", 2024-09-10T18:47:22.709 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-10T18:47:22.709 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_hit_set_max_size": "100000", 2024-09-10T18:47:22.709 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_hit_set_min_size": "1000", 2024-09-10T18:47:22.709 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-10T18:47:22.710 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-10T18:47:22.710 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-10T18:47:22.710 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-10T18:47:22.710 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-10T18:47:22.710 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-10T18:47:22.710 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_journal_size": "5120", 2024-09-10T18:47:22.710 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_kill_backfill_at": "0", 2024-09-10T18:47:22.710 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-10T18:47:22.710 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_map_cache_size": "50", 2024-09-10T18:47:22.710 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_map_dedup": "true", 2024-09-10T18:47:22.710 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_map_message_max": "40", 2024-09-10T18:47:22.711 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-10T18:47:22.711 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_map_share_max_epochs": "40", 2024-09-10T18:47:22.711 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_max_attr_name_len": "100", 2024-09-10T18:47:22.711 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_max_attr_size": "0", 2024-09-10T18:47:22.711 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_max_backfills": "1", 2024-09-10T18:47:22.711 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_max_markdown_count": "5", 2024-09-10T18:47:22.711 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_max_markdown_period": "600", 2024-09-10T18:47:22.711 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_max_object_name_len": "2048", 2024-09-10T18:47:22.711 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_max_object_namespace_len": "256", 2024-09-10T18:47:22.711 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_max_object_size": "134217728", 2024-09-10T18:47:22.711 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-10T18:47:22.712 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-10T18:47:22.712 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_max_pg_blocked_by": "16", 2024-09-10T18:47:22.712 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_max_pg_log_entries": "10000", 2024-09-10T18:47:22.712 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-10T18:47:22.712 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_max_pgls": "1024", 2024-09-10T18:47:22.712 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_max_push_cost": "8388608", 2024-09-10T18:47:22.712 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_max_push_objects": "10", 2024-09-10T18:47:22.712 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_max_scrubs": "3", 2024-09-10T18:47:22.712 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-10T18:47:22.712 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_max_trimming_pgs": "2", 2024-09-10T18:47:22.712 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_max_write_op_reply_len": "64", 2024-09-10T18:47:22.713 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_max_write_size": "90", 2024-09-10T18:47:22.713 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-10T18:47:22.713 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-10T18:47:22.713 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-10T18:47:22.713 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-10T18:47:22.713 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-10T18:47:22.713 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-10T18:47:22.713 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-10T18:47:22.713 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-10T18:47:22.713 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mclock_profile": "balanced", 2024-09-10T18:47:22.713 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-10T18:47:22.713 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-10T18:47:22.714 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-10T18:47:22.714 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-10T18:47:22.714 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-10T18:47:22.714 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-10T18:47:22.714 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-10T18:47:22.714 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-10T18:47:22.714 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-10T18:47:22.714 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-10T18:47:22.714 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-10T18:47:22.714 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_memory_base": "805306368", 2024-09-10T18:47:22.715 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_memory_cache_min": "134217728", 2024-09-10T18:47:22.715 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-10T18:47:22.715 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-10T18:47:22.715 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_memory_target": "4294967296", 2024-09-10T18:47:22.715 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_memory_target_autotune": "false", 2024-09-10T18:47:22.715 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-10T18:47:22.715 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_min_pg_log_entries": "250", 2024-09-10T18:47:22.715 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_min_recovery_priority": "0", 2024-09-10T18:47:22.715 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-10T18:47:22.715 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-10T18:47:22.715 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mon_report_interval": "5", 2024-09-10T18:47:22.716 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-10T18:47:22.716 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-10T18:47:22.716 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_num_cache_shards": "32", 2024-09-10T18:47:22.716 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_num_op_tracker_shard": "32", 2024-09-10T18:47:22.716 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_numa_auto_affinity": "true", 2024-09-10T18:47:22.716 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_numa_node": "-1", 2024-09-10T18:47:22.716 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_numa_prefer_iface": "true", 2024-09-10T18:47:22.716 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-10T18:47:22.716 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_objecter_finishers": "1", 2024-09-10T18:47:22.716 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_objectstore": "bluestore", 2024-09-10T18:47:22.717 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_objectstore_fuse": "false", 2024-09-10T18:47:22.717 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_objectstore_tracing": "false", 2024-09-10T18:47:22.717 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_op_complaint_time": "30.000000", 2024-09-10T18:47:22.717 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_op_history_duration": "600", 2024-09-10T18:47:22.717 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_op_history_size": "20", 2024-09-10T18:47:22.717 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_op_history_slow_op_size": "20", 2024-09-10T18:47:22.717 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-10T18:47:22.717 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_op_log_threshold": "5", 2024-09-10T18:47:22.717 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_op_num_shards": "0", 2024-09-10T18:47:22.717 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_op_num_shards_hdd": "5", 2024-09-10T18:47:22.718 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_op_num_shards_ssd": "8", 2024-09-10T18:47:22.718 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-10T18:47:22.718 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-10T18:47:22.718 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-10T18:47:22.718 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-10T18:47:22.718 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_op_pq_min_cost": "65536", 2024-09-10T18:47:22.718 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-10T18:47:22.718 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_op_queue_cut_off": "high", 2024-09-10T18:47:22.718 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-10T18:47:22.718 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_op_thread_timeout": "15", 2024-09-10T18:47:22.718 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_open_classes_on_start": "true", 2024-09-10T18:47:22.719 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_os_flags": "0", 2024-09-10T18:47:22.719 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_peering_op_priority": "255", 2024-09-10T18:47:22.719 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pg_delete_cost": "1048576", 2024-09-10T18:47:22.719 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pg_delete_priority": "5", 2024-09-10T18:47:22.719 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-10T18:47:22.719 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-10T18:47:22.719 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-10T18:47:22.719 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pg_log_trim_max": "10000", 2024-09-10T18:47:22.719 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pg_log_trim_min": "100", 2024-09-10T18:47:22.719 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-10T18:47:22.719 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-10T18:47:22.720 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pg_stat_report_interval_max": "500", 2024-09-10T18:47:22.720 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-10T18:47:22.720 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-10T18:47:22.720 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-10T18:47:22.720 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-10T18:47:22.720 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-10T18:47:22.720 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-10T18:47:22.720 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-10T18:47:22.720 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-10T18:47:22.720 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-10T18:47:22.721 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-10T18:47:22.721 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-10T18:47:22.721 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-10T18:47:22.721 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-10T18:47:22.721 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-10T18:47:22.721 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pool_default_flags": "0", 2024-09-10T18:47:22.721 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-10T18:47:22.721 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pool_default_min_size": "0", 2024-09-10T18:47:22.721 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-10T18:47:22.721 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pool_default_pg_num": "32", 2024-09-10T18:47:22.721 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pool_default_pgp_num": "0", 2024-09-10T18:47:22.722 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-10T18:47:22.722 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pool_default_size": "3", 2024-09-10T18:47:22.722 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pool_default_type": "replicated", 2024-09-10T18:47:22.722 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-10T18:47:22.722 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-10T18:47:22.722 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_push_per_object_cost": "1000", 2024-09-10T18:47:22.722 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-10T18:47:22.722 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_recover_clone_overlap": "true", 2024-09-10T18:47:22.722 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-10T18:47:22.722 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_recovery_cost": "20971520", 2024-09-10T18:47:22.722 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-10T18:47:22.723 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_recovery_max_active": "0", 2024-09-10T18:47:22.723 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-10T18:47:22.723 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-10T18:47:22.723 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-10T18:47:22.723 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-10T18:47:22.723 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_recovery_max_single_start": "1", 2024-09-10T18:47:22.723 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_recovery_op_priority": "3", 2024-09-10T18:47:22.723 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-10T18:47:22.723 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_recovery_priority": "5", 2024-09-10T18:47:22.723 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-10T18:47:22.723 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_recovery_sleep": "0.000000", 2024-09-10T18:47:22.724 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-10T18:47:22.724 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-10T18:47:22.724 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-10T18:47:22.724 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_repair_during_recovery": "false", 2024-09-10T18:47:22.724 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_requested_scrub_priority": "120", 2024-09-10T18:47:22.724 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-10T18:47:22.724 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-10T18:47:22.724 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_scrub_auto_repair": "false", 2024-09-10T18:47:22.724 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-10T18:47:22.724 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-10T18:47:22.725 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_scrub_begin_hour": "0", 2024-09-10T18:47:22.725 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_scrub_begin_week_day": "0", 2024-09-10T18:47:22.725 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_scrub_chunk_max": "25", 2024-09-10T18:47:22.725 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_scrub_chunk_min": "5", 2024-09-10T18:47:22.725 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_scrub_cost": "52428800", 2024-09-10T18:47:22.725 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_scrub_during_recovery": "false", 2024-09-10T18:47:22.725 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_scrub_end_hour": "0", 2024-09-10T18:47:22.725 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_scrub_end_week_day": "0", 2024-09-10T18:47:22.725 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_scrub_event_cost": "4096", 2024-09-10T18:47:22.725 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-10T18:47:22.725 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-10T18:47:22.725 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_scrub_invalid_stats": "true", 2024-09-10T18:47:22.726 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-10T18:47:22.726 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-10T18:47:22.726 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_scrub_max_preemptions": "5", 2024-09-10T18:47:22.726 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-10T18:47:22.726 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_scrub_priority": "5", 2024-09-10T18:47:22.726 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_scrub_sleep": "0.000000", 2024-09-10T18:47:22.726 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-10T18:47:22.726 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_skip_data_digest": "false", 2024-09-10T18:47:22.726 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_smart_report_timeout": "5", 2024-09-10T18:47:22.726 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_snap_trim_cost": "1048576", 2024-09-10T18:47:22.727 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_snap_trim_priority": "5", 2024-09-10T18:47:22.727 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-10T18:47:22.727 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-10T18:47:22.727 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-10T18:47:22.727 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-10T18:47:22.727 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-10T18:47:22.727 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_target_transaction_size": "30", 2024-09-10T18:47:22.727 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-10T18:47:22.727 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-10T18:47:22.727 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-10T18:47:22.727 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-10T18:47:22.728 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-10T18:47:22.728 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-10T18:47:22.728 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-10T18:47:22.728 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-10T18:47:22.728 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-10T18:47:22.728 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-10T18:47:22.728 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_tracing": "false", 2024-09-10T18:47:22.728 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_use_stale_snap": "false", 2024-09-10T18:47:22.728 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T18:47:22.728 INFO:teuthology.orchestra.run.smithi018.stdout: "osdc_blkin_trace_all": "false", 2024-09-10T18:47:22.728 INFO:teuthology.orchestra.run.smithi018.stdout: "paxos_kill_at": "0", 2024-09-10T18:47:22.729 INFO:teuthology.orchestra.run.smithi018.stdout: "paxos_max_join_drift": "10", 2024-09-10T18:47:22.729 INFO:teuthology.orchestra.run.smithi018.stdout: "paxos_min": "500", 2024-09-10T18:47:22.729 INFO:teuthology.orchestra.run.smithi018.stdout: "paxos_min_wait": "0.050000", 2024-09-10T18:47:22.729 INFO:teuthology.orchestra.run.smithi018.stdout: "paxos_propose_interval": "1.000000", 2024-09-10T18:47:22.729 INFO:teuthology.orchestra.run.smithi018.stdout: "paxos_service_trim_max": "500", 2024-09-10T18:47:22.729 INFO:teuthology.orchestra.run.smithi018.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-10T18:47:22.729 INFO:teuthology.orchestra.run.smithi018.stdout: "paxos_service_trim_min": "10", 2024-09-10T18:47:22.729 INFO:teuthology.orchestra.run.smithi018.stdout: "paxos_stash_full_interval": "25", 2024-09-10T18:47:22.729 INFO:teuthology.orchestra.run.smithi018.stdout: "paxos_trim_max": "500", 2024-09-10T18:47:22.729 INFO:teuthology.orchestra.run.smithi018.stdout: "paxos_trim_min": "250", 2024-09-10T18:47:22.729 INFO:teuthology.orchestra.run.smithi018.stdout: "perf": "true", 2024-09-10T18:47:22.730 INFO:teuthology.orchestra.run.smithi018.stdout: "pid_file": "", 2024-09-10T18:47:22.730 INFO:teuthology.orchestra.run.smithi018.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-10T18:47:22.730 INFO:teuthology.orchestra.run.smithi018.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-10T18:47:22.730 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addr": "-", 2024-09-10T18:47:22.730 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addrv": "", 2024-09-10T18:47:22.730 INFO:teuthology.orchestra.run.smithi018.stdout: "public_bind_addr": "-", 2024-09-10T18:47:22.730 INFO:teuthology.orchestra.run.smithi018.stdout: "public_network": "", 2024-09-10T18:47:22.730 INFO:teuthology.orchestra.run.smithi018.stdout: "public_network_interface": "", 2024-09-10T18:47:22.730 INFO:teuthology.orchestra.run.smithi018.stdout: "qat_compressor_enabled": "false", 2024-09-10T18:47:22.730 INFO:teuthology.orchestra.run.smithi018.stdout: "rados_mon_op_timeout": "0", 2024-09-10T18:47:22.730 INFO:teuthology.orchestra.run.smithi018.stdout: "rados_osd_op_timeout": "0", 2024-09-10T18:47:22.731 INFO:teuthology.orchestra.run.smithi018.stdout: "rados_tracing": "false", 2024-09-10T18:47:22.731 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_atime_update_interval": "60", 2024-09-10T18:47:22.731 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-10T18:47:22.731 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_balance_parent_reads": "false", 2024-09-10T18:47:22.731 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_balance_snap_reads": "false", 2024-09-10T18:47:22.731 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_blkin_trace_all": "false", 2024-09-10T18:47:22.731 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-10T18:47:22.731 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-10T18:47:22.731 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_cache": "true", 2024-09-10T18:47:22.731 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-10T18:47:22.731 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-10T18:47:22.731 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-10T18:47:22.732 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-10T18:47:22.732 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_cache_policy": "writearound", 2024-09-10T18:47:22.732 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_cache_size": "33554432", 2024-09-10T18:47:22.732 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-10T18:47:22.732 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-10T18:47:22.732 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_clone_copy_on_read": "false", 2024-09-10T18:47:22.732 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_compression_hint": "none", 2024-09-10T18:47:22.732 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_concurrent_management_ops": "10", 2024-09-10T18:47:22.732 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-10T18:47:22.732 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_default_clone_format": "auto", 2024-09-10T18:47:22.732 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_default_data_pool": "", 2024-09-10T18:47:22.733 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_default_features": "61", 2024-09-10T18:47:22.733 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_default_format": "2", 2024-09-10T18:47:22.733 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_default_map_options": "", 2024-09-10T18:47:22.733 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_default_order": "22", 2024-09-10T18:47:22.733 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_default_pool": "rbd", 2024-09-10T18:47:22.733 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-10T18:47:22.733 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_default_stripe_count": "0", 2024-09-10T18:47:22.733 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_default_stripe_unit": "0", 2024-09-10T18:47:22.733 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-10T18:47:22.733 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-10T18:47:22.734 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-10T18:47:22.734 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_enable_alloc_hint": "true", 2024-09-10T18:47:22.734 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-10T18:47:22.734 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_io_scheduler": "simple", 2024-09-10T18:47:22.734 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-10T18:47:22.734 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-10T18:47:22.734 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-10T18:47:22.734 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-10T18:47:22.734 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-10T18:47:22.734 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-10T18:47:22.734 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-10T18:47:22.735 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-10T18:47:22.735 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-10T18:47:22.735 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_journal_order": "24", 2024-09-10T18:47:22.735 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_journal_pool": "", 2024-09-10T18:47:22.735 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_journal_splay_width": "4", 2024-09-10T18:47:22.735 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_localize_parent_reads": "false", 2024-09-10T18:47:22.735 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_localize_snap_reads": "false", 2024-09-10T18:47:22.735 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-10T18:47:22.735 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-10T18:47:22.735 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-10T18:47:22.735 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-10T18:47:22.736 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-10T18:47:22.736 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-10T18:47:22.736 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-10T18:47:22.736 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-10T18:47:22.736 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-10T18:47:22.736 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-10T18:47:22.736 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-10T18:47:22.736 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-10T18:47:22.736 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-10T18:47:22.736 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-10T18:47:22.736 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-10T18:47:22.737 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-10T18:47:22.737 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-10T18:47:22.737 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-10T18:47:22.737 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-10T18:47:22.737 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-10T18:47:22.737 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-10T18:47:22.737 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-10T18:47:22.737 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-10T18:47:22.737 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-10T18:47:22.737 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-10T18:47:22.737 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-10T18:47:22.738 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-10T18:47:22.738 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-10T18:47:22.738 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-10T18:47:22.738 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-10T18:47:22.738 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-10T18:47:22.738 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_mtime_update_interval": "60", 2024-09-10T18:47:22.738 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_non_blocking_aio": "true", 2024-09-10T18:47:22.738 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_op_thread_timeout": "60", 2024-09-10T18:47:22.738 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_op_threads": "1", 2024-09-10T18:47:22.738 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_parent_cache_enabled": "false", 2024-09-10T18:47:22.738 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-10T18:47:22.738 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-10T18:47:22.739 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-10T18:47:22.739 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_plugins": "", 2024-09-10T18:47:22.739 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_qos_bps_burst": "0", 2024-09-10T18:47:22.739 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-10T18:47:22.739 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_qos_bps_limit": "0", 2024-09-10T18:47:22.739 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_qos_exclude_ops": "0", 2024-09-10T18:47:22.739 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_qos_iops_burst": "0", 2024-09-10T18:47:22.739 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-10T18:47:22.739 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_qos_iops_limit": "0", 2024-09-10T18:47:22.739 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-10T18:47:22.739 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-10T18:47:22.739 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-10T18:47:22.740 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-10T18:47:22.740 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-10T18:47:22.740 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-10T18:47:22.740 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-10T18:47:22.740 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-10T18:47:22.740 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-10T18:47:22.740 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-10T18:47:22.740 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-10T18:47:22.740 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-10T18:47:22.740 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-10T18:47:22.740 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-10T18:47:22.740 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_read_from_replica_policy": "default", 2024-09-10T18:47:22.741 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-10T18:47:22.741 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-10T18:47:22.741 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-10T18:47:22.741 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-10T18:47:22.741 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_skip_partial_discard": "true", 2024-09-10T18:47:22.741 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-10T18:47:22.741 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_tracing": "false", 2024-09-10T18:47:22.741 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_validate_names": "true", 2024-09-10T18:47:22.741 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd_validate_pool": "true", 2024-09-10T18:47:22.741 INFO:teuthology.orchestra.run.smithi018.stdout: "restapi_base_url": "", 2024-09-10T18:47:22.741 INFO:teuthology.orchestra.run.smithi018.stdout: "restapi_log_level": "", 2024-09-10T18:47:22.741 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_acl_grants_max_num": "100", 2024-09-10T18:47:22.742 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_admin_entry": "admin", 2024-09-10T18:47:22.742 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_backend_store": "rados", 2024-09-10T18:47:22.742 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_barbican_url": "", 2024-09-10T18:47:22.742 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_beast_enable_async": "true", 2024-09-10T18:47:22.742 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-10T18:47:22.742 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-10T18:47:22.742 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-10T18:47:22.742 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-10T18:47:22.742 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-10T18:47:22.742 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_cache_enabled": "true", 2024-09-10T18:47:22.742 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_cache_expiry_interval": "900", 2024-09-10T18:47:22.742 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_cache_lru_size": "10000", 2024-09-10T18:47:22.743 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_content_length_compat": "false", 2024-09-10T18:47:22.743 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_copy_obj_progress": "true", 2024-09-10T18:47:22.743 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-10T18:47:22.743 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_cors_rules_max_num": "100", 2024-09-10T18:47:22.743 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_cross_domain_policy": "", 2024-09-10T18:47:22.743 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-10T18:47:22.743 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_crypt_kmip_addr": "", 2024-09-10T18:47:22.743 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-10T18:47:22.743 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-10T18:47:22.744 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-10T18:47:22.744 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-10T18:47:22.744 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_crypt_kmip_password": "", 2024-09-10T18:47:22.744 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-10T18:47:22.745 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_crypt_kmip_username": "", 2024-09-10T18:47:22.745 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_crypt_require_ssl": "true", 2024-09-10T18:47:22.745 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-10T18:47:22.745 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-10T18:47:22.745 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-10T18:47:22.745 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-10T18:47:22.745 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-10T18:47:22.745 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-10T18:47:22.745 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-10T18:47:22.745 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-10T18:47:22.745 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-10T18:47:22.745 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-10T18:47:22.746 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-10T18:47:22.746 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-10T18:47:22.746 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-10T18:47:22.746 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-10T18:47:22.746 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-10T18:47:22.746 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_crypt_vault_addr": "", 2024-09-10T18:47:22.746 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_crypt_vault_auth": "token", 2024-09-10T18:47:22.746 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_crypt_vault_namespace": "", 2024-09-10T18:47:22.746 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_crypt_vault_prefix": "", 2024-09-10T18:47:22.746 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-10T18:47:22.747 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-10T18:47:22.747 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-10T18:47:22.747 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-10T18:47:22.747 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_crypt_vault_token_file": "", 2024-09-10T18:47:22.747 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-10T18:47:22.747 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_curl_buffersize": "524288", 2024-09-10T18:47:22.747 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-10T18:47:22.747 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_curl_low_speed_time": "300", 2024-09-10T18:47:22.747 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-10T18:47:22.747 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-10T18:47:22.747 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-10T18:47:22.747 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-10T18:47:22.748 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-10T18:47:22.748 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-10T18:47:22.748 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-10T18:47:22.748 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-10T18:47:22.748 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-10T18:47:22.748 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-10T18:47:22.748 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_data_log_changes_size": "1000", 2024-09-10T18:47:22.748 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_data_log_num_shards": "128", 2024-09-10T18:47:22.748 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-10T18:47:22.748 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_data_log_window": "30", 2024-09-10T18:47:22.748 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_data_notify_interval_msec": "200", 2024-09-10T18:47:22.749 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-10T18:47:22.749 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-10T18:47:22.749 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-10T18:47:22.749 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-10T18:47:22.749 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-10T18:47:22.749 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-10T18:47:22.749 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-10T18:47:22.749 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-10T18:47:22.749 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-10T18:47:22.749 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-10T18:47:22.749 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-10T18:47:22.750 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-10T18:47:22.750 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-10T18:47:22.750 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-10T18:47:22.750 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-10T18:47:22.750 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-10T18:47:22.750 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-10T18:47:22.750 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-10T18:47:22.750 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-10T18:47:22.750 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-10T18:47:22.750 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-10T18:47:22.750 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-10T18:47:22.751 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_dns_name": "", 2024-09-10T18:47:22.751 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_dns_s3website_name": "", 2024-09-10T18:47:22.751 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_dynamic_resharding": "true", 2024-09-10T18:47:22.751 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-10T18:47:22.751 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_enable_gc_threads": "true", 2024-09-10T18:47:22.751 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_enable_lc_threads": "true", 2024-09-10T18:47:22.751 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_enable_ops_log": "false", 2024-09-10T18:47:22.751 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_enable_quota_threads": "true", 2024-09-10T18:47:22.751 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_enable_static_website": "false", 2024-09-10T18:47:22.751 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_enable_usage_log": "false", 2024-09-10T18:47:22.751 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_enforce_swift_acls": "true", 2024-09-10T18:47:22.752 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_exit_timeout_secs": "120", 2024-09-10T18:47:22.752 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_expose_bucket": "false", 2024-09-10T18:47:22.752 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_extended_http_attrs": "", 2024-09-10T18:47:22.752 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-10T18:47:22.752 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_frontends": "beast port=7480", 2024-09-10T18:47:22.752 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-10T18:47:22.752 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_gc_max_deferred": "50", 2024-09-10T18:47:22.752 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-10T18:47:22.752 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_gc_max_objs": "32", 2024-09-10T18:47:22.752 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-10T18:47:22.752 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-10T18:47:22.753 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-10T18:47:22.753 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-10T18:47:22.753 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_gc_processor_period": "3600", 2024-09-10T18:47:22.753 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-10T18:47:22.753 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-10T18:47:22.753 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-10T18:47:22.753 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-10T18:47:22.753 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_init_timeout": "300", 2024-09-10T18:47:22.753 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-10T18:47:22.753 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_kafka_connection_idle": "300", 2024-09-10T18:47:22.754 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-10T18:47:22.754 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-10T18:47:22.754 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-10T18:47:22.754 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_keystone_admin_domain": "", 2024-09-10T18:47:22.754 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_keystone_admin_password": "", 2024-09-10T18:47:22.754 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_keystone_admin_password_path": "", 2024-09-10T18:47:22.754 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_keystone_admin_project": "", 2024-09-10T18:47:22.754 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_keystone_admin_tenant": "", 2024-09-10T18:47:22.754 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_keystone_admin_token": "", 2024-09-10T18:47:22.754 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_keystone_admin_token_path": "", 2024-09-10T18:47:22.754 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_keystone_admin_user": "", 2024-09-10T18:47:22.754 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_keystone_api_version": "2", 2024-09-10T18:47:22.755 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_keystone_barbican_domain": "", 2024-09-10T18:47:22.755 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_keystone_barbican_password": "", 2024-09-10T18:47:22.755 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_keystone_barbican_project": "", 2024-09-10T18:47:22.755 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-10T18:47:22.755 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_keystone_barbican_user": "", 2024-09-10T18:47:22.755 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-10T18:47:22.755 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-10T18:47:22.755 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-10T18:47:22.755 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-10T18:47:22.755 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-10T18:47:22.755 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_keystone_url": "", 2024-09-10T18:47:22.755 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-10T18:47:22.756 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_lc_debug_interval": "-1", 2024-09-10T18:47:22.756 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_lc_lock_max_time": "90", 2024-09-10T18:47:22.756 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_lc_max_objs": "32", 2024-09-10T18:47:22.756 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_lc_max_rules": "1000", 2024-09-10T18:47:22.756 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_lc_max_worker": "3", 2024-09-10T18:47:22.756 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-10T18:47:22.756 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_lc_thread_delay": "0", 2024-09-10T18:47:22.756 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-10T18:47:22.756 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_ldap_dnattr": "uid", 2024-09-10T18:47:22.756 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-10T18:47:22.756 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_ldap_searchfilter": "", 2024-09-10T18:47:22.757 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-10T18:47:22.757 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_ldap_uri": "", 2024-09-10T18:47:22.757 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-10T18:47:22.757 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-10T18:47:22.757 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-10T18:47:22.757 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_log_http_headers": "", 2024-09-10T18:47:22.757 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-10T18:47:22.757 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-10T18:47:22.757 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_log_object_name_utc": "false", 2024-09-10T18:47:22.757 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_luarocks_location": "/tmp/luarocks", 2024-09-10T18:47:22.757 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_max_attr_name_len": "0", 2024-09-10T18:47:22.758 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_max_attr_size": "0", 2024-09-10T18:47:22.758 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-10T18:47:22.758 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_max_chunk_size": "4194304", 2024-09-10T18:47:22.758 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-10T18:47:22.758 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-10T18:47:22.758 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_max_listing_results": "1000", 2024-09-10T18:47:22.758 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_max_notify_retries": "10", 2024-09-10T18:47:22.758 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-10T18:47:22.758 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_max_put_param_size": "1048576", 2024-09-10T18:47:22.758 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_max_put_size": "5368709120", 2024-09-10T18:47:22.758 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_max_slo_entries": "1000", 2024-09-10T18:47:22.759 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_md_log_max_shards": "64", 2024-09-10T18:47:22.759 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-10T18:47:22.759 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-10T18:47:22.759 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_mp_lock_max_time": "600", 2024-09-10T18:47:22.759 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-10T18:47:22.759 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-10T18:47:22.759 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-10T18:47:22.759 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-10T18:47:22.759 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-10T18:47:22.759 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-10T18:47:22.759 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-10T18:47:22.759 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_nfs_max_gc": "300", 2024-09-10T18:47:22.760 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-10T18:47:22.760 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-10T18:47:22.760 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-10T18:47:22.760 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-10T18:47:22.760 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-10T18:47:22.760 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-10T18:47:22.760 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-10T18:47:22.760 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_num_async_rados_threads": "32", 2024-09-10T18:47:22.760 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_num_control_oids": "8", 2024-09-10T18:47:22.760 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_numa_node": "-1", 2024-09-10T18:47:22.760 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-10T18:47:22.761 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-10T18:47:22.761 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_objexp_chunk_size": "100", 2024-09-10T18:47:22.761 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_objexp_gc_interval": "600", 2024-09-10T18:47:22.761 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-10T18:47:22.761 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-10T18:47:22.761 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-10T18:47:22.761 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_op_thread_timeout": "600", 2024-09-10T18:47:22.761 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_op_tracing": "false", 2024-09-10T18:47:22.761 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_opa_token": "", 2024-09-10T18:47:22.761 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_opa_url": "", 2024-09-10T18:47:22.761 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_opa_verify_ssl": "true", 2024-09-10T18:47:22.761 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-10T18:47:22.762 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_ops_log_file_path": "", 2024-09-10T18:47:22.762 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_ops_log_rados": "true", 2024-09-10T18:47:22.762 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_ops_log_socket_path": "", 2024-09-10T18:47:22.762 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-10T18:47:22.762 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-10T18:47:22.762 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_period_push_interval": "2.000000", 2024-09-10T18:47:22.762 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-10T18:47:22.762 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-10T18:47:22.762 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_print_continue": "true", 2024-09-10T18:47:22.762 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-10T18:47:22.762 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-10T18:47:22.763 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-10T18:47:22.763 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-10T18:47:22.763 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-10T18:47:22.763 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_rados_tracing": "false", 2024-09-10T18:47:22.763 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_realm": "", 2024-09-10T18:47:22.763 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_realm_id": "", 2024-09-10T18:47:22.763 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-10T18:47:22.763 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_region": "", 2024-09-10T18:47:22.763 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-10T18:47:22.763 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-10T18:47:22.763 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-10T18:47:22.763 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-10T18:47:22.764 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_request_uri": "", 2024-09-10T18:47:22.764 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_reshard_batch_size": "64", 2024-09-10T18:47:22.764 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-10T18:47:22.764 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_reshard_max_aio": "128", 2024-09-10T18:47:22.764 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_reshard_num_logs": "16", 2024-09-10T18:47:22.764 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_reshard_thread_interval": "600", 2024-09-10T18:47:22.764 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_resolve_cname": "false", 2024-09-10T18:47:22.764 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-10T18:47:22.764 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_run_sync_thread": "true", 2024-09-10T18:47:22.764 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-10T18:47:22.764 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-10T18:47:22.765 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-10T18:47:22.765 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-10T18:47:22.765 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-10T18:47:22.765 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-10T18:47:22.765 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-10T18:47:22.765 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-10T18:47:22.765 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-10T18:47:22.765 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_scheduler_type": "throttler", 2024-09-10T18:47:22.765 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_script_uri": "", 2024-09-10T18:47:22.765 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_service_provider_name": "", 2024-09-10T18:47:22.765 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-10T18:47:22.766 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_sts_client_id": "", 2024-09-10T18:47:22.766 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_sts_client_secret": "", 2024-09-10T18:47:22.766 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_sts_entry": "sts", 2024-09-10T18:47:22.766 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_sts_key": "sts", 2024-09-10T18:47:22.766 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-10T18:47:22.766 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_sts_min_session_duration": "900", 2024-09-10T18:47:22.766 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_sts_token_introspection_url": "", 2024-09-10T18:47:22.766 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_swift_account_in_url": "false", 2024-09-10T18:47:22.766 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_swift_auth_entry": "auth", 2024-09-10T18:47:22.766 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_swift_auth_url": "", 2024-09-10T18:47:22.766 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_swift_custom_header": "", 2024-09-10T18:47:22.767 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-10T18:47:22.767 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_swift_need_stats": "true", 2024-09-10T18:47:22.767 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_swift_tenant_name": "", 2024-09-10T18:47:22.767 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_swift_token_expiration": "86400", 2024-09-10T18:47:22.767 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_swift_url": "", 2024-09-10T18:47:22.767 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_swift_url_prefix": "swift", 2024-09-10T18:47:22.767 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-10T18:47:22.767 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-10T18:47:22.767 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_sync_lease_period": "120", 2024-09-10T18:47:22.767 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-10T18:47:22.767 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-10T18:47:22.768 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-10T18:47:22.768 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-10T18:47:22.768 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-10T18:47:22.768 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-10T18:47:22.768 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-10T18:47:22.768 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-10T18:47:22.768 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-10T18:47:22.768 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_thread_pool_size": "512", 2024-09-10T18:47:22.768 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_torrent_comment": "", 2024-09-10T18:47:22.768 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_torrent_createby": "", 2024-09-10T18:47:22.768 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_torrent_encoding": "", 2024-09-10T18:47:22.768 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_torrent_flag": "false", 2024-09-10T18:47:22.769 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_torrent_origin": "", 2024-09-10T18:47:22.769 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-10T18:47:22.769 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_torrent_tracker": "", 2024-09-10T18:47:22.769 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_trust_forwarded_https": "false", 2024-09-10T18:47:22.769 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-10T18:47:22.769 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-10T18:47:22.769 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_usage_max_shards": "32", 2024-09-10T18:47:22.769 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_usage_max_user_shards": "1", 2024-09-10T18:47:22.769 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_use_opa_authz": "false", 2024-09-10T18:47:22.769 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-10T18:47:22.769 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-10T18:47:22.770 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_user_max_buckets": "1000", 2024-09-10T18:47:22.770 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-10T18:47:22.770 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-10T18:47:22.770 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-10T18:47:22.770 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-10T18:47:22.770 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_user_unique_email": "true", 2024-09-10T18:47:22.770 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_verify_ssl": "true", 2024-09-10T18:47:22.770 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-10T18:47:22.770 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_zone": "", 2024-09-10T18:47:22.770 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_zone_id": "", 2024-09-10T18:47:22.770 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-10T18:47:22.770 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_zonegroup": "", 2024-09-10T18:47:22.771 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_zonegroup_id": "", 2024-09-10T18:47:22.771 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-10T18:47:22.771 INFO:teuthology.orchestra.run.smithi018.stdout: "rocksdb_block_size": "4096", 2024-09-10T18:47:22.771 INFO:teuthology.orchestra.run.smithi018.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-10T18:47:22.771 INFO:teuthology.orchestra.run.smithi018.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-10T18:47:22.771 INFO:teuthology.orchestra.run.smithi018.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-10T18:47:22.771 INFO:teuthology.orchestra.run.smithi018.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-10T18:47:22.771 INFO:teuthology.orchestra.run.smithi018.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-10T18:47:22.771 INFO:teuthology.orchestra.run.smithi018.stdout: "rocksdb_cache_size": "536870912", 2024-09-10T18:47:22.771 INFO:teuthology.orchestra.run.smithi018.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-10T18:47:22.771 INFO:teuthology.orchestra.run.smithi018.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-10T18:47:22.771 INFO:teuthology.orchestra.run.smithi018.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-10T18:47:22.772 INFO:teuthology.orchestra.run.smithi018.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-10T18:47:22.772 INFO:teuthology.orchestra.run.smithi018.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-10T18:47:22.772 INFO:teuthology.orchestra.run.smithi018.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-10T18:47:22.772 INFO:teuthology.orchestra.run.smithi018.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-10T18:47:22.772 INFO:teuthology.orchestra.run.smithi018.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-10T18:47:22.772 INFO:teuthology.orchestra.run.smithi018.stdout: "rocksdb_index_type": "binary_search", 2024-09-10T18:47:22.772 INFO:teuthology.orchestra.run.smithi018.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-10T18:47:22.772 INFO:teuthology.orchestra.run.smithi018.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-10T18:47:22.772 INFO:teuthology.orchestra.run.smithi018.stdout: "rocksdb_partition_filters": "false", 2024-09-10T18:47:22.772 INFO:teuthology.orchestra.run.smithi018.stdout: "rocksdb_perf": "false", 2024-09-10T18:47:22.772 INFO:teuthology.orchestra.run.smithi018.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-10T18:47:22.773 INFO:teuthology.orchestra.run.smithi018.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-10T18:47:22.773 INFO:teuthology.orchestra.run.smithi018.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-10T18:47:22.773 INFO:teuthology.orchestra.run.smithi018.stdout: "run_dir": "/var/run/ceph", 2024-09-10T18:47:22.773 INFO:teuthology.orchestra.run.smithi018.stdout: "seastore_block_create": "true", 2024-09-10T18:47:22.773 INFO:teuthology.orchestra.run.smithi018.stdout: "seastore_cache_lru_size": "67108864", 2024-09-10T18:47:22.773 INFO:teuthology.orchestra.run.smithi018.stdout: "seastore_default_max_object_size": "16777216", 2024-09-10T18:47:22.773 INFO:teuthology.orchestra.run.smithi018.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-10T18:47:22.773 INFO:teuthology.orchestra.run.smithi018.stdout: "seastore_device_size": "107374182400", 2024-09-10T18:47:22.773 INFO:teuthology.orchestra.run.smithi018.stdout: "seastore_init_rewrite_segments_num_per_device": "6", 2024-09-10T18:47:22.773 INFO:teuthology.orchestra.run.smithi018.stdout: "seastore_journal_batch_capacity": "16", 2024-09-10T18:47:22.773 INFO:teuthology.orchestra.run.smithi018.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-10T18:47:22.773 INFO:teuthology.orchestra.run.smithi018.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-10T18:47:22.774 INFO:teuthology.orchestra.run.smithi018.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-10T18:47:22.774 INFO:teuthology.orchestra.run.smithi018.stdout: "seastore_segment_size": "67108864", 2024-09-10T18:47:22.774 INFO:teuthology.orchestra.run.smithi018.stdout: "setgroup": "ceph", 2024-09-10T18:47:22.774 INFO:teuthology.orchestra.run.smithi018.stdout: "setuser": "ceph", 2024-09-10T18:47:22.774 INFO:teuthology.orchestra.run.smithi018.stdout: "setuser_match_path": "", 2024-09-10T18:47:22.774 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-10T18:47:22.774 INFO:teuthology.orchestra.run.smithi018.stdout: "thp": "false", 2024-09-10T18:47:22.774 INFO:teuthology.orchestra.run.smithi018.stdout: "threadpool_default_timeout": "60", 2024-09-10T18:47:22.774 INFO:teuthology.orchestra.run.smithi018.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-10T18:47:22.774 INFO:teuthology.orchestra.run.smithi018.stdout: "throttler_perf_counter": "true" 2024-09-10T18:47:22.774 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T18:47:23.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:23 smithi018 ceph-mon[40196]: 6.16 scrub starts 2024-09-10T18:47:23.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:23 smithi018 ceph-mon[40196]: 6.16 scrub ok 2024-09-10T18:47:23.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:23 smithi018 ceph-mon[40196]: 5.19 scrub starts 2024-09-10T18:47:23.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:23 smithi018 ceph-mon[40196]: 5.19 scrub ok 2024-09-10T18:47:23.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:23 smithi018 ceph-mon[40196]: 3.0 deep-scrub starts 2024-09-10T18:47:23.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:23 smithi018 ceph-mon[40196]: 3.0 deep-scrub ok 2024-09-10T18:47:23.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:23 smithi018 ceph-mon[40196]: 6.6 deep-scrub starts 2024-09-10T18:47:23.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:23 smithi018 ceph-mon[40196]: 6.6 deep-scrub ok 2024-09-10T18:47:23.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:23 smithi018 ceph-mon[29336]: 6.16 scrub starts 2024-09-10T18:47:23.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:23 smithi018 ceph-mon[29336]: 6.16 scrub ok 2024-09-10T18:47:23.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:23 smithi018 ceph-mon[29336]: 5.19 scrub starts 2024-09-10T18:47:23.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:23 smithi018 ceph-mon[29336]: 5.19 scrub ok 2024-09-10T18:47:23.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:23 smithi018 ceph-mon[29336]: 3.0 deep-scrub starts 2024-09-10T18:47:23.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:23 smithi018 ceph-mon[29336]: 3.0 deep-scrub ok 2024-09-10T18:47:23.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:23 smithi018 ceph-mon[29336]: 6.6 deep-scrub starts 2024-09-10T18:47:23.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:23 smithi018 ceph-mon[29336]: 6.6 deep-scrub ok 2024-09-10T18:47:23.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:23 smithi103 ceph-mon[33163]: 6.16 scrub starts 2024-09-10T18:47:23.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:23 smithi103 ceph-mon[33163]: 6.16 scrub ok 2024-09-10T18:47:23.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:23 smithi103 ceph-mon[33163]: 5.19 scrub starts 2024-09-10T18:47:23.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:23 smithi103 ceph-mon[33163]: 5.19 scrub ok 2024-09-10T18:47:23.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:23 smithi103 ceph-mon[33163]: 3.0 deep-scrub starts 2024-09-10T18:47:23.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:23 smithi103 ceph-mon[33163]: 3.0 deep-scrub ok 2024-09-10T18:47:23.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:23 smithi103 ceph-mon[33163]: 6.6 deep-scrub starts 2024-09-10T18:47:23.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:23 smithi103 ceph-mon[33163]: 6.6 deep-scrub ok 2024-09-10T18:47:23.804 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-10T18:47:24.056 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:47:24.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:24 smithi018 ceph-mon[29336]: 6.18 scrub starts 2024-09-10T18:47:24.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:24 smithi018 ceph-mon[29336]: 6.18 scrub ok 2024-09-10T18:47:24.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:24 smithi018 ceph-mon[29336]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:47:24.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:24 smithi018 ceph-mon[40196]: 6.18 scrub starts 2024-09-10T18:47:24.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:24 smithi018 ceph-mon[40196]: 6.18 scrub ok 2024-09-10T18:47:24.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:24 smithi018 ceph-mon[40196]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:47:24.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:24 smithi103 ceph-mon[33163]: 6.18 scrub starts 2024-09-10T18:47:24.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:24 smithi103 ceph-mon[33163]: 6.18 scrub ok 2024-09-10T18:47:24.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:24 smithi103 ceph-mon[33163]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:47:25.233 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:25 smithi103 systemd[1]: Stopping Ceph prometheus.a for ba327f6c-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:47:25.505 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:25 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:47:25.505 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:25 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:47:25.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:25 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:47:25.522 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:25 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[61672]: ts=2024-09-10T18:47:25.334Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-10T18:47:25.522 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:25 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[61672]: ts=2024-09-10T18:47:25.334Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-10T18:47:25.522 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:25 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[61672]: ts=2024-09-10T18:47:25.334Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-10T18:47:25.522 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:25 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[61672]: ts=2024-09-10T18:47:25.334Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-10T18:47:25.523 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:25 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[61672]: ts=2024-09-10T18:47:25.334Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-10T18:47:25.523 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:25 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[61672]: ts=2024-09-10T18:47:25.334Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-10T18:47:25.523 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:25 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[61672]: ts=2024-09-10T18:47:25.334Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-10T18:47:25.523 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:25 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[61672]: ts=2024-09-10T18:47:25.334Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-10T18:47:25.523 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:25 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[61672]: ts=2024-09-10T18:47:25.334Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-10T18:47:25.523 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:25 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[61672]: ts=2024-09-10T18:47:25.334Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-10T18:47:25.523 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:25 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[61672]: ts=2024-09-10T18:47:25.334Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-10T18:47:25.523 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:25 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[61672]: ts=2024-09-10T18:47:25.334Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-10T18:47:25.523 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:25 smithi103 podman[64790]: 2024-09-10 18:47:25.344291532 +0000 UTC m=+0.126145156 container died e6b5cd4d0de26275b2a4a5d90d1e3fb84a393f831779894e5eea67ddd3baa97c (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:47:25.880 INFO:teuthology.orchestra.run.smithi018.stdout:mon.a: {} 2024-09-10T18:47:25.880 INFO:teuthology.orchestra.run.smithi018.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-10T18:47:25.888 INFO:teuthology.orchestra.run.smithi018.stdout:mon.b: {} 2024-09-10T18:47:25.889 INFO:teuthology.orchestra.run.smithi018.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-10T18:47:25.897 INFO:teuthology.orchestra.run.smithi018.stdout:mon.c: {} 2024-09-10T18:47:25.897 INFO:teuthology.orchestra.run.smithi018.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-10T18:47:25.986 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:25 smithi103 podman[64790]: 2024-09-10 18:47:25.708841981 +0000 UTC m=+0.490695594 container cleanup e6b5cd4d0de26275b2a4a5d90d1e3fb84a393f831779894e5eea67ddd3baa97c (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:47:25.986 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:25 smithi103 bash[64790]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a 2024-09-10T18:47:26.272 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:47:26 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:47:26.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:26 smithi103 ceph-mon[33163]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:47:26.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:26 smithi103 ceph-mon[33163]: 6.1 deep-scrub starts 2024-09-10T18:47:26.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:26 smithi103 ceph-mon[33163]: 6.1 deep-scrub ok 2024-09-10T18:47:26.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:26 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1258874198' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:47:26.274 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:25 smithi103 podman[64802]: 2024-09-10 18:47:25.985052138 +0000 UTC m=+0.638586695 container remove e6b5cd4d0de26275b2a4a5d90d1e3fb84a393f831779894e5eea67ddd3baa97c (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:47:26.274 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:26 smithi103 systemd[1]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-09-10T18:47:26.274 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:26 smithi103 systemd[1]: Stopped Ceph prometheus.a for ba327f6c-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:47:26.274 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:26 smithi103 systemd[1]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@prometheus.a.service: Consumed 1.037s CPU time. 2024-09-10T18:47:26.274 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:26 smithi103 systemd[1]: Starting Ceph prometheus.a for ba327f6c-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:47:26.521 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:26 smithi018 ceph-mon[29336]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:47:26.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:26 smithi018 ceph-mon[29336]: 6.1 deep-scrub starts 2024-09-10T18:47:26.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:26 smithi018 ceph-mon[29336]: 6.1 deep-scrub ok 2024-09-10T18:47:26.522 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:26 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1258874198' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:47:26.522 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:26 smithi018 ceph-mon[40196]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:47:26.522 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:26 smithi018 ceph-mon[40196]: 6.1 deep-scrub starts 2024-09-10T18:47:26.522 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:26 smithi018 ceph-mon[40196]: 6.1 deep-scrub ok 2024-09-10T18:47:26.522 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:26 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1258874198' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:47:26.598 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:26 smithi103 podman[64898]: 2024-09-10 18:47:26.242720079 +0000 UTC m=+0.017032313 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-10T18:47:26.598 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:26 smithi103 podman[64898]: 2024-09-10 18:47:26.386575937 +0000 UTC m=+0.160888212 container create 1b039b6cb12d53ab37cd6276fa8b4f30b4f7d39ce7fda8ee25daccf4dcfc096b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:47:26.598 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:26 smithi103 podman[64898]: 2024-09-10 18:47:26.572063384 +0000 UTC m=+0.346375618 container init 1b039b6cb12d53ab37cd6276fa8b4f30b4f7d39ce7fda8ee25daccf4dcfc096b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:47:26.598 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:26 smithi103 podman[64898]: 2024-09-10 18:47:26.574650885 +0000 UTC m=+0.348963119 container start 1b039b6cb12d53ab37cd6276fa8b4f30b4f7d39ce7fda8ee25daccf4dcfc096b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:47:26.692 INFO:teuthology.run_tasks:Running task full_sequential... 2024-09-10T18:47:26.704 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-10T18:47:26.708 INFO:tasks.radosbench:Beginning radosbench... 2024-09-10T18:47:26.708 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_0 2024-09-10T18:47:26.708 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd pool create unique_pool_0 16 2024-09-10T18:47:26.709 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-10T18:47:26.709 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-10T18:47:26.709 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T18:47:26.709 INFO:tasks.thrashosds.thrasher:Removing osd 2, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-10T18:47:26.709 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd out 2 2024-09-10T18:47:26.710 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-10T18:47:26.710 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:47:26.710 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-10T18:47:26.711 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T18:47:26.711 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-10T18:47:26.711 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-10T18:47:26.712 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-10T18:47:26.712 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-10T18:47:26.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:26.752 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:26.752 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:47:26.853 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:26.875 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:26.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:26.876 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:26.945 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:47:26 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:18:47:26] ENGINE Bus STOPPING 2024-09-10T18:47:26.977 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:47:27.017 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:27.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:27.018 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:47:27.023 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:26 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[64913]: ts=2024-09-10T18:47:26.597Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-10T18:47:27.023 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:26 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[64913]: ts=2024-09-10T18:47:26.597Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-10T18:47:27.023 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:26 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[64913]: ts=2024-09-10T18:47:26.597Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi103 (none))" 2024-09-10T18:47:27.023 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:26 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[64913]: ts=2024-09-10T18:47:26.597Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-10T18:47:27.023 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:26 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[64913]: ts=2024-09-10T18:47:26.597Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-10T18:47:27.023 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:26 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[64913]: ts=2024-09-10T18:47:26.599Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-10T18:47:27.023 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:26 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[64913]: ts=2024-09-10T18:47:26.599Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-10T18:47:27.023 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:26 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[64913]: ts=2024-09-10T18:47:26.600Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-10T18:47:27.023 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:26 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[64913]: ts=2024-09-10T18:47:26.600Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-10T18:47:27.023 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:26 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[64913]: ts=2024-09-10T18:47:26.602Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-10T18:47:27.024 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:26 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[64913]: ts=2024-09-10T18:47:26.602Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.644µs 2024-09-10T18:47:27.024 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:26 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[64913]: ts=2024-09-10T18:47:26.602Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-10T18:47:27.024 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:26 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[64913]: ts=2024-09-10T18:47:26.603Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-09-10T18:47:27.024 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:26 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[64913]: ts=2024-09-10T18:47:26.603Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-09-10T18:47:27.024 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:26 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[64913]: ts=2024-09-10T18:47:26.603Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=18.805µs wal_replay_duration=500.879µs wbl_replay_duration=89ns total_replay_duration=537.969µs 2024-09-10T18:47:27.024 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:26 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[64913]: ts=2024-09-10T18:47:26.604Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-10T18:47:27.024 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:26 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[64913]: ts=2024-09-10T18:47:26.604Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-10T18:47:27.024 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:26 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[64913]: ts=2024-09-10T18:47:26.604Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-10T18:47:27.024 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:26 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[64913]: ts=2024-09-10T18:47:26.619Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=14.891583ms db_storage=754ns remote_storage=989ns web_handler=490ns query_engine=594ns scrape=377.906µs scrape_sd=26.526µs notify=24.584µs notify_sd=7.894µs rules=14.231312ms tracing=5.856µs 2024-09-10T18:47:27.024 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:26 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[64913]: ts=2024-09-10T18:47:26.619Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-10T18:47:27.024 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:26 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[64913]: ts=2024-09-10T18:47:26.619Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-10T18:47:27.024 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:26 smithi103 bash[64898]: 1b039b6cb12d53ab37cd6276fa8b4f30b4f7d39ce7fda8ee25daccf4dcfc096b 2024-09-10T18:47:27.025 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 18:47:26 smithi103 systemd[1]: Started Ceph prometheus.a for ba327f6c-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:47:27.119 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:27.143 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:27.143 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:47:27.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:27.144 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:27.148 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:47:27.149 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:47:27.200 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:47:27.200 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:47:27.245 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:47:27.267 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:27.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:27.268 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:47:27.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:27.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-mon[29336]: 4.4 scrub starts 2024-09-10T18:47:27.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-mon[29336]: 4.4 scrub ok 2024-09-10T18:47:27.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:27.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:27.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:47:27.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi018.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:47:27.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:27.274 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:47:27.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi103.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:47:27.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:27.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:47:27.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi103.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:47:27.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:27.275 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:47:27.276 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:27.276 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-mon[40196]: 4.4 scrub starts 2024-09-10T18:47:27.276 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-mon[40196]: 4.4 scrub ok 2024-09-10T18:47:27.276 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:27.276 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:27.276 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:47:27.276 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi018.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:47:27.276 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:27.276 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:47:27.276 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi103.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:47:27.276 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:27.277 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:47:27.277 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi103.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:47:27.277 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:27.277 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:47:27.277 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:18:47:27] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:47:27.277 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:18:47:27] ENGINE Bus STOPPED 2024-09-10T18:47:27.277 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:18:47:27] ENGINE Bus STARTING 2024-09-10T18:47:27.277 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:18:47:27] ENGINE Serving on http://:::9283 2024-09-10T18:47:27.277 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:18:47:27] ENGINE Bus STARTED 2024-09-10T18:47:27.277 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:18:47:27] ENGINE Bus STOPPING 2024-09-10T18:47:27.277 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:18:47:27] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:47:27.278 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:18:47:27] ENGINE Bus STOPPED 2024-09-10T18:47:27.278 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:18:47:27] ENGINE Bus STARTING 2024-09-10T18:47:27.278 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:18:47:27] ENGINE Serving on http://:::9283 2024-09-10T18:47:27.278 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:18:47:27] ENGINE Bus STARTED 2024-09-10T18:47:27.369 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:27.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:27.393 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:27.394 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:27.482 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:27 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:27.482 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:27 smithi103 ceph-mon[33163]: 4.4 scrub starts 2024-09-10T18:47:27.482 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:27 smithi103 ceph-mon[33163]: 4.4 scrub ok 2024-09-10T18:47:27.482 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:27 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:27.482 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:27 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:27.482 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:27 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:47:27.482 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:27 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi018.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:47:27.482 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:27 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:27.482 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:27 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:47:27.482 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:27 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi103.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:47:27.482 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:27 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:27.483 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:27 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:47:27.483 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:27 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi103.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:47:27.483 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:27 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:27.483 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:27 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:47:27.494 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:47:27.517 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:27.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:27.518 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:47:27.554 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:18:47:27] ENGINE Bus STOPPING 2024-09-10T18:47:27.619 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:47:27.640 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:27.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:27.641 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:47:27.742 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:47:27.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:27.766 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:27.766 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:47:27.804 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:18:47:27] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:47:27.805 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:18:47:27] ENGINE Bus STOPPED 2024-09-10T18:47:27.805 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:18:47:27] ENGINE Bus STARTING 2024-09-10T18:47:27.867 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:27.893 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:27.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:27.894 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:27.995 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:28.016 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:28.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:28.017 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:28.118 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:28.139 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:28.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:28.140 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:28.179 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a[75395]: ts=2024-09-10T18:47:27.803Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003209671s 2024-09-10T18:47:28.179 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:18:47:27] ENGINE Serving on http://:::9283 2024-09-10T18:47:28.180 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:47:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:18:47:27] ENGINE Bus STARTED 2024-09-10T18:47:28.240 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:47:28.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:28.263 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:28.263 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:47:28.364 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:47:28.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:28.389 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:28.389 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:47:28.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:28 smithi018 ceph-mon[40196]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:47:28.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:28 smithi018 ceph-mon[40196]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi018.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:47:28.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:28 smithi018 ceph-mon[40196]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:47:28.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:28 smithi018 ceph-mon[40196]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi103.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:47:28.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:28 smithi018 ceph-mon[40196]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:47:28.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:28 smithi018 ceph-mon[40196]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi103.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:47:28.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:28 smithi018 ceph-mon[40196]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:47:28.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:28 smithi018 ceph-mon[29336]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:47:28.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:28 smithi018 ceph-mon[29336]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi018.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:47:28.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:28 smithi018 ceph-mon[29336]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:47:28.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:28 smithi018 ceph-mon[29336]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi103.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:47:28.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:28 smithi018 ceph-mon[29336]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:47:28.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:28 smithi018 ceph-mon[29336]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi103.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:47:28.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:28 smithi018 ceph-mon[29336]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:47:28.490 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:47:28.513 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:28.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:28.513 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:47:28.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:28 smithi103 ceph-mon[33163]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:47:28.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:28 smithi103 ceph-mon[33163]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi018.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:47:28.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:28 smithi103 ceph-mon[33163]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:47:28.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:28 smithi103 ceph-mon[33163]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi103.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:47:28.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:28 smithi103 ceph-mon[33163]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:47:28.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:28 smithi103 ceph-mon[33163]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi103.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:47:28.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:28 smithi103 ceph-mon[33163]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:47:28.614 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:47:28.637 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:28.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:28.638 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:47:28.739 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:47:28.762 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:28.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:28.763 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:47:28.864 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:47:28.886 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:28.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:28.888 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:47:28.988 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:47:29.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:29.011 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:29.011 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:47:29.112 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:47:29.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:29.134 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:29.135 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:47:29.235 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:47:29.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:29.257 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:29.258 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:47:29.358 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:47:29.380 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:29.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:29.381 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:47:29.482 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:29.505 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:29.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:29.506 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:29.607 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:47:29.629 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:29.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:29.630 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:47:29.731 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:29.753 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:29.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:29.754 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:29.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:29 smithi103 ceph-mon[33163]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:47:29.854 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:47:29.876 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:29.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:29.877 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:47:29.921 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:29 smithi018 ceph-mon[29336]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:47:29.921 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:29 smithi018 ceph-mon[40196]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:47:29.978 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:47:30.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:30.001 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:30.001 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:47:30.102 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:30.123 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:30.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:30.124 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:30.225 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:30.248 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:30.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:30.249 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:30.350 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:30.372 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:30.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:30.373 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:30.474 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:47:30.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:30.497 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:30.497 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:47:30.598 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:30.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:30.621 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:30.622 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:30.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:30 smithi018 ceph-mon[29336]: 2.14 scrub starts 2024-09-10T18:47:30.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:30 smithi018 ceph-mon[29336]: 2.14 scrub ok 2024-09-10T18:47:30.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:30 smithi018 ceph-mon[40196]: 2.14 scrub starts 2024-09-10T18:47:30.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:30 smithi018 ceph-mon[40196]: 2.14 scrub ok 2024-09-10T18:47:30.723 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:47:30.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:30.745 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:30.745 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:47:30.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:30 smithi103 ceph-mon[33163]: 2.14 scrub starts 2024-09-10T18:47:30.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:30 smithi103 ceph-mon[33163]: 2.14 scrub ok 2024-09-10T18:47:30.846 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:47:30.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:30.870 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:30.870 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:47:30.971 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:47:30.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:30.993 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:30.994 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:47:31.094 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:47:31.117 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:31.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:31.118 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:47:31.219 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:47:31.241 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:31.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:31.242 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:47:31.343 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:47:31.365 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:31.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:31.366 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:47:31.468 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:31.490 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:31.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:31.491 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:31.592 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:31.615 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:31.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:31.616 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:31.717 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:47:31.740 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:31.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:31.741 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:47:31.842 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:31.865 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:31.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:31.866 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:31.967 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:31.993 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:31.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:31.994 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:31.999 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:31 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:31.999 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:31 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:31.999 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:31 smithi018 ceph-mon[40196]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:47:31.999 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:31 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:32.000 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:31 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:32.000 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:31 smithi018 ceph-mon[29336]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:47:32.095 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:47:32.122 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:32.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:32.123 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:47:32.224 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:32.249 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:32.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:32.250 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:32.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:31 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:32.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:31 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:32.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:31 smithi103 ceph-mon[33163]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:47:32.351 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:32.374 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:32.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:32.376 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:32.476 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:47:32.501 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:32.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:32.502 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:47:32.602 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:32.629 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:32.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:32.629 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:32.730 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:47:32.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:32.756 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:32.756 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:47:32.857 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:47:32.881 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:32.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:32.882 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:47:32.983 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:47:33.006 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:33.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:33.007 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:47:33.108 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:47:33.131 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:33.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:33.132 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:47:33.233 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:47:33.259 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:33.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:33.260 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:47:33.361 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:47:33.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:33.388 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:33.389 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:47:33.392 INFO:teuthology.orchestra.run.smithi018.stderr:pool 'unique_pool_0' created 2024-09-10T18:47:33.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:33 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:47:33.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:33 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:47:33.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:33 smithi018 ceph-mon[29336]: osdmap e85: 8 total, 8 up, 8 in 2024-09-10T18:47:33.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:33 smithi018 ceph-mon[29336]: 5.4 scrub starts 2024-09-10T18:47:33.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:33 smithi018 ceph-mon[29336]: 5.4 scrub ok 2024-09-10T18:47:33.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:33 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:47:33.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:33 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:47:33.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:33 smithi018 ceph-mon[40196]: osdmap e85: 8 total, 8 up, 8 in 2024-09-10T18:47:33.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:33 smithi018 ceph-mon[40196]: 5.4 scrub starts 2024-09-10T18:47:33.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:33 smithi018 ceph-mon[40196]: 5.4 scrub ok 2024-09-10T18:47:33.489 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:47:33.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:33.512 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:33.512 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:47:33.613 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:47:33.636 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:33.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:33.637 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:47:33.738 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:33.760 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:33.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:33.762 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:33.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:33 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:47:33.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:33 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:47:33.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:33 smithi103 ceph-mon[33163]: osdmap e85: 8 total, 8 up, 8 in 2024-09-10T18:47:33.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:33 smithi103 ceph-mon[33163]: 5.4 scrub starts 2024-09-10T18:47:33.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:33 smithi103 ceph-mon[33163]: 5.4 scrub ok 2024-09-10T18:47:33.862 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:47:33.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:33.885 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:33.886 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:47:33.987 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:47:34.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:34.014 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:34.014 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:47:34.116 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:47:34.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:34.140 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:34.141 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:47:34.242 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:47:34.269 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:34.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:34.270 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:47:34.276 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2024-09-10T18:47:34.371 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:47:34.401 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:34.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:34.402 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:47:34.503 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:47:34.528 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:34.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:34.529 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:47:34.580 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:34 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1991707083' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]: dispatch 2024-09-10T18:47:34.580 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:34 smithi018 ceph-mon[29336]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:47:34.580 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:34 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1991707083' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]': finished 2024-09-10T18:47:34.580 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:34 smithi018 ceph-mon[29336]: osdmap e86: 8 total, 8 up, 8 in 2024-09-10T18:47:34.581 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:34 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1991707083' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]: dispatch 2024-09-10T18:47:34.581 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:34 smithi018 ceph-mon[40196]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:47:34.581 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:34 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1991707083' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]': finished 2024-09-10T18:47:34.581 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:34 smithi018 ceph-mon[40196]: osdmap e86: 8 total, 8 up, 8 in 2024-09-10T18:47:34.630 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:47:34.655 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:34.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:34.656 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:47:34.757 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:47:34.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:34 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1991707083' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]: dispatch 2024-09-10T18:47:34.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:34 smithi103 ceph-mon[33163]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:47:34.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:34 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1991707083' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]': finished 2024-09-10T18:47:34.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:34 smithi103 ceph-mon[33163]: osdmap e86: 8 total, 8 up, 8 in 2024-09-10T18:47:34.781 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:34.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:34.782 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:47:34.883 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:47:34.909 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:34.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:34.910 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:47:34.978 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:47:34.978 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:47:34.994 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:47:34.994 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:47:35.009 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:47:35.009 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:47:35.011 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:47:35.024 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:47:35.025 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:47:35.033 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:35.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:35.034 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:47:35.042 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:47:35.042 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:47:35.058 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:47:35.059 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:47:35.075 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:47:35.075 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:47:35.092 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:47:35.092 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:47:35.135 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:47:35.157 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:35.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:35.158 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:47:35.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:34 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mon-a[29332]: 2024-09-10T18:47:34.946+0000 7f4eb0f2a640 -1 mon.a@0(leader).osd e87 definitely_dead 0 2024-09-10T18:47:35.258 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:47:35.285 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:35.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:35.286 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:47:35.387 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:35.400 INFO:teuthology.orchestra.run.smithi018.stderr:marked out osd.2. 2024-09-10T18:47:35.421 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:35.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:35.422 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:35.523 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:47:35.549 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:35.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:35.550 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:47:35.651 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:47:35.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:35.676 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:35.676 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:47:35.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:35 smithi018 ceph-mon[40196]: osdmap e87: 8 total, 8 up, 8 in 2024-09-10T18:47:35.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:35 smithi018 ceph-mon[40196]: 4.1c scrub starts 2024-09-10T18:47:35.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:35 smithi018 ceph-mon[40196]: 4.1c scrub ok 2024-09-10T18:47:35.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:35 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/982846751' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["2"]}]: dispatch 2024-09-10T18:47:35.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:35 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["2"]}]: dispatch 2024-09-10T18:47:35.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:35 smithi018 ceph-mon[40196]: Client client.admin marked osd.2 out, while it was still marked up 2024-09-10T18:47:35.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:35 smithi018 ceph-mon[40196]: 5.1c scrub starts 2024-09-10T18:47:35.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:35 smithi018 ceph-mon[40196]: 5.1c scrub ok 2024-09-10T18:47:35.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:35 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/4100341538' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:47:35.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:35 smithi018 ceph-mon[29336]: osdmap e87: 8 total, 8 up, 8 in 2024-09-10T18:47:35.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:35 smithi018 ceph-mon[29336]: 4.1c scrub starts 2024-09-10T18:47:35.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:35 smithi018 ceph-mon[29336]: 4.1c scrub ok 2024-09-10T18:47:35.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:35 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/982846751' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["2"]}]: dispatch 2024-09-10T18:47:35.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:35 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["2"]}]: dispatch 2024-09-10T18:47:35.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:35 smithi018 ceph-mon[29336]: Client client.admin marked osd.2 out, while it was still marked up 2024-09-10T18:47:35.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:35 smithi018 ceph-mon[29336]: 5.1c scrub starts 2024-09-10T18:47:35.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:35 smithi018 ceph-mon[29336]: 5.1c scrub ok 2024-09-10T18:47:35.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:35 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/4100341538' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:47:35.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:35 smithi103 ceph-mon[33163]: osdmap e87: 8 total, 8 up, 8 in 2024-09-10T18:47:35.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:35 smithi103 ceph-mon[33163]: 4.1c scrub starts 2024-09-10T18:47:35.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:35 smithi103 ceph-mon[33163]: 4.1c scrub ok 2024-09-10T18:47:35.776 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:35 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/982846751' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["2"]}]: dispatch 2024-09-10T18:47:35.776 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:35 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["2"]}]: dispatch 2024-09-10T18:47:35.776 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:35 smithi103 ceph-mon[33163]: Client client.admin marked osd.2 out, while it was still marked up 2024-09-10T18:47:35.776 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:35 smithi103 ceph-mon[33163]: 5.1c scrub starts 2024-09-10T18:47:35.776 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:35 smithi103 ceph-mon[33163]: 5.1c scrub ok 2024-09-10T18:47:35.776 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:35 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/4100341538' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:47:35.777 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:47:35.800 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:35.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:35.801 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:47:35.901 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:47:35.924 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:35.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:35.925 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:47:36.026 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:36.053 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:36.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:36.054 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:36.155 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:36.177 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:36.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:36.178 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:36.279 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:47:36.305 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:36.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:36.306 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:47:36.392 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:47:36 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:47:36.407 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:36.447 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:36.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:36.448 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:36.549 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:47:36.571 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:36.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:36.572 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:47:36.673 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:36.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:36 smithi018 ceph-mon[29336]: pgmap v70: 148 pgs: 16 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:47:36.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:36 smithi018 ceph-mon[29336]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:47:36.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:36 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["2"]}]': finished 2024-09-10T18:47:36.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:36 smithi018 ceph-mon[29336]: osdmap e88: 8 total, 8 up, 7 in 2024-09-10T18:47:36.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:36 smithi018 ceph-mon[29336]: 4.9 scrub starts 2024-09-10T18:47:36.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:36 smithi018 ceph-mon[29336]: 4.9 scrub ok 2024-09-10T18:47:36.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:36 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:36.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:36 smithi018 ceph-mon[40196]: pgmap v70: 148 pgs: 16 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:47:36.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:36 smithi018 ceph-mon[40196]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:47:36.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:36 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["2"]}]': finished 2024-09-10T18:47:36.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:36 smithi018 ceph-mon[40196]: osdmap e88: 8 total, 8 up, 7 in 2024-09-10T18:47:36.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:36 smithi018 ceph-mon[40196]: 4.9 scrub starts 2024-09-10T18:47:36.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:36 smithi018 ceph-mon[40196]: 4.9 scrub ok 2024-09-10T18:47:36.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:36 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:36.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:36.696 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:36.696 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:36.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:36 smithi103 ceph-mon[33163]: pgmap v70: 148 pgs: 16 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:47:36.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:36 smithi103 ceph-mon[33163]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:47:36.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:36 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["2"]}]': finished 2024-09-10T18:47:36.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:36 smithi103 ceph-mon[33163]: osdmap e88: 8 total, 8 up, 7 in 2024-09-10T18:47:36.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:36 smithi103 ceph-mon[33163]: 4.9 scrub starts 2024-09-10T18:47:36.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:36 smithi103 ceph-mon[33163]: 4.9 scrub ok 2024-09-10T18:47:36.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:36 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:36.801 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:47:36.823 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:36.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:36.824 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:47:36.924 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:47:36.947 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:36.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:36.948 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:47:37.049 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:37.071 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:37.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:37.072 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:37.173 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:47:37.196 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:37.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:37.197 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:47:37.298 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:47:37.320 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:37.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:37.321 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:47:37.422 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:37 smithi018 ceph-mon[40196]: osdmap e89: 8 total, 8 up, 7 in 2024-09-10T18:47:37.422 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:37 smithi018 ceph-mon[40196]: 5.1a deep-scrub starts 2024-09-10T18:47:37.422 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:37 smithi018 ceph-mon[40196]: 5.1a deep-scrub ok 2024-09-10T18:47:37.422 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:37 smithi018 ceph-mon[40196]: pgmap v73: 148 pgs: 46 peering, 102 active+clean; 582 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 511 B/s rd, 0 op/s 2024-09-10T18:47:37.422 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:37 smithi018 ceph-mon[29336]: osdmap e89: 8 total, 8 up, 7 in 2024-09-10T18:47:37.422 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:37 smithi018 ceph-mon[29336]: 5.1a deep-scrub starts 2024-09-10T18:47:37.422 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:37 smithi018 ceph-mon[29336]: 5.1a deep-scrub ok 2024-09-10T18:47:37.422 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:37 smithi018 ceph-mon[29336]: pgmap v73: 148 pgs: 46 peering, 102 active+clean; 582 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 511 B/s rd, 0 op/s 2024-09-10T18:47:37.423 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:47:37.447 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:37.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:37.448 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:47:37.549 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:47:37.572 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:37.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:37.573 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:47:37.674 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:47:37.700 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:37.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:37.701 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:47:37.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:37 smithi103 ceph-mon[33163]: osdmap e89: 8 total, 8 up, 7 in 2024-09-10T18:47:37.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:37 smithi103 ceph-mon[33163]: 5.1a deep-scrub starts 2024-09-10T18:47:37.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:37 smithi103 ceph-mon[33163]: 5.1a deep-scrub ok 2024-09-10T18:47:37.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:37 smithi103 ceph-mon[33163]: pgmap v73: 148 pgs: 46 peering, 102 active+clean; 582 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 511 B/s rd, 0 op/s 2024-09-10T18:47:37.802 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:47:37.828 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:37.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:37.829 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:47:37.882 INFO:teuthology.orchestra.run.smithi018.stdout:ERROR: (22) Invalid argument 2024-09-10T18:47:37.882 INFO:teuthology.orchestra.run.smithi018.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:47:37.929 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:47:37.952 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:37.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:37.953 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:47:38.053 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:47:38.077 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:38.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:38.078 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:47:38.136 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:47:38.179 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:47:38.203 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:38.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:38.204 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:47:38.305 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:47:38.328 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:38.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:38.329 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:47:38.411 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is set 2024-09-10T18:47:38.430 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:47:38.455 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:38.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:38.456 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:47:38.557 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:38.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:38.581 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:38.581 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:38.682 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:47:38.689 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:38 smithi018 ceph-mon[40196]: Health check failed: Reduced data availability: 9 pgs inactive, 12 pgs peering (PG_AVAILABILITY) 2024-09-10T18:47:38.689 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:38 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1459321442' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:47:38.690 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:38 smithi018 ceph-mon[40196]: 6.f scrub starts 2024-09-10T18:47:38.690 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:38 smithi018 ceph-mon[40196]: 6.f scrub ok 2024-09-10T18:47:38.690 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:38 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:47:38.690 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:38 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:47:38.690 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:38 smithi018 ceph-mon[29336]: Health check failed: Reduced data availability: 9 pgs inactive, 12 pgs peering (PG_AVAILABILITY) 2024-09-10T18:47:38.690 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:38 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1459321442' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:47:38.690 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:38 smithi018 ceph-mon[29336]: 6.f scrub starts 2024-09-10T18:47:38.690 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:38 smithi018 ceph-mon[29336]: 6.f scrub ok 2024-09-10T18:47:38.690 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:38 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:47:38.690 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:38 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:47:38.705 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:38.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:38.706 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:47:38.715 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-10T18:47:38.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:38 smithi103 ceph-mon[33163]: Health check failed: Reduced data availability: 9 pgs inactive, 12 pgs peering (PG_AVAILABILITY) 2024-09-10T18:47:38.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:38 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1459321442' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:47:38.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:38 smithi103 ceph-mon[33163]: 6.f scrub starts 2024-09-10T18:47:38.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:38 smithi103 ceph-mon[33163]: 6.f scrub ok 2024-09-10T18:47:38.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:38 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:47:38.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:38 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:47:38.807 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:47:38.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:38.831 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:38.832 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:47:38.932 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:47:38.954 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:38.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:38.955 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:47:39.056 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:47:39.080 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:39.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:39.081 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:47:39.180 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T18:47:39.181 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:47:39.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:39.208 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:39.208 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:47:39.309 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:39.334 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:39.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:39.335 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:39.436 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:47:39.460 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:39.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:39.461 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:47:39.562 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:47:39.585 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:39.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:39.586 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:47:39.637 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:39 smithi018 ceph-mon[40196]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:47:39.637 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:39 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1459321442' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:47:39.637 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:39 smithi018 ceph-mon[40196]: osdmap e90: 8 total, 8 up, 7 in 2024-09-10T18:47:39.637 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:39 smithi018 ceph-mon[40196]: pgmap v75: 148 pgs: 46 peering, 102 active+clean; 582 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 440 B/s rd, 0 op/s 2024-09-10T18:47:39.637 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:39 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:47:39.637 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:39 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:47:39.637 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:39 smithi018 ceph-mon[29336]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:47:39.638 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:39 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1459321442' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:47:39.638 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:39 smithi018 ceph-mon[29336]: osdmap e90: 8 total, 8 up, 7 in 2024-09-10T18:47:39.638 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:39 smithi018 ceph-mon[29336]: pgmap v75: 148 pgs: 46 peering, 102 active+clean; 582 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 440 B/s rd, 0 op/s 2024-09-10T18:47:39.638 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:39 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:47:39.638 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:39 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:47:39.686 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:47:39.708 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:39.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:39.709 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:47:39.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:39 smithi103 ceph-mon[33163]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:47:39.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:39 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1459321442' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:47:39.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:39 smithi103 ceph-mon[33163]: osdmap e90: 8 total, 8 up, 7 in 2024-09-10T18:47:39.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:39 smithi103 ceph-mon[33163]: pgmap v75: 148 pgs: 46 peering, 102 active+clean; 582 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 440 B/s rd, 0 op/s 2024-09-10T18:47:39.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:39 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:47:39.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:39 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:47:39.810 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:47:39.836 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:39.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:39.837 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:47:39.938 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:47:39.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:39.962 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:39.962 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:47:40.063 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:40.089 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:40.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:40.090 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:40.120 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:47:40.191 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:40.213 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:40.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:40.214 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:40.315 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:47:40.337 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:40.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:40.338 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:47:40.439 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:47:40.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:40.463 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:40.463 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:47:40.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:47:40 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:47:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:47:40.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:40 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:47:40.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:40 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:47:40.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:40 smithi103 ceph-mon[33163]: osdmap e91: 8 total, 8 up, 7 in 2024-09-10T18:47:40.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:40 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:40.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:40 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:47:40.563 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:47:40.585 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:40.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:40.586 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:47:40.618 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:40 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:47:40.618 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:40 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:47:40.618 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:40 smithi018 ceph-mon[29336]: osdmap e91: 8 total, 8 up, 7 in 2024-09-10T18:47:40.618 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:40 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:40.618 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:40 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:47:40.618 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:40 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:47:40.619 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:40 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:47:40.619 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:40 smithi018 ceph-mon[40196]: osdmap e91: 8 total, 8 up, 7 in 2024-09-10T18:47:40.619 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:40 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:40.619 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:40 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:47:40.687 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:47:40.692 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:47:40.713 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:40.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:40.714 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:47:40.815 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:40.840 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:40.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:40.841 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:40.942 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:40.967 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:40.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:40.969 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:41.069 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:41.093 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:41.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:41.095 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:41.132 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-10T18:47:41.132 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T18:47:41.133 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-10T18:47:41.133 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-10T18:47:41.133 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-10T18:47:41.133 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.4 config set filestore_inject_stall 3 2024-09-10T18:47:41.196 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:47:41.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:41.220 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:41.220 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:47:41.321 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:47:41.344 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:41.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:41.345 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:47:41.382 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:47:41.417 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:47:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:47:40] "GET /metrics HTTP/1.1" 200 222579 "" "Prometheus/2.43.0" 2024-09-10T18:47:41.426 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-10T18:47:41.446 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:47:41.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:41.468 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:41.468 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:47:41.569 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:47:41.590 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:41.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:41.591 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:47:41.673 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:41 smithi018 ceph-mon[40196]: pgmap v77: 148 pgs: 148 active+clean; 582 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s; 361 B/s, 10 objects/s recovering 2024-09-10T18:47:41.673 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:41 smithi018 ceph-mon[29336]: pgmap v77: 148 pgs: 148 active+clean; 582 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s; 361 B/s, 10 objects/s recovering 2024-09-10T18:47:41.682 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:41 smithi103 ceph-mon[33163]: pgmap v77: 148 pgs: 148 active+clean; 582 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s; 361 B/s, 10 objects/s recovering 2024-09-10T18:47:41.692 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:41.713 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:41.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:41.714 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:41.815 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:41.837 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:41.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:41.838 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:41.939 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:41.963 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:41.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:41.964 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:42.065 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:47:42.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:42.088 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:42.088 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:47:42.189 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:42.209 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:42.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:42.211 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:42.311 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:47:42.332 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:42.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:42.333 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:47:42.427 INFO:teuthology.orchestra.run.smithi018.stderr:enabled application 'rados' on pool 'unique_pool_0' 2024-09-10T18:47:42.434 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:47:42.460 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:42.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:42.461 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:47:42.562 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:47:42.585 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:42.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:42.587 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:47:42.671 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:42 smithi103 ceph-mon[33163]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs inactive, 12 pgs peering) 2024-09-10T18:47:42.671 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:42 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2053069188' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:42.688 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:47:42.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:42 smithi018 ceph-mon[40196]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs inactive, 12 pgs peering) 2024-09-10T18:47:42.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:42 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2053069188' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:42.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:42 smithi018 ceph-mon[29336]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs inactive, 12 pgs peering) 2024-09-10T18:47:42.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:42 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2053069188' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:42.709 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:42.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:42.711 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:47:42.811 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:47:42.834 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:42.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:42.835 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:47:42.936 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:47:42.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:42.962 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:42.962 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:47:42.998 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T18:47:42.999 INFO:teuthology.orchestra.run.smithi103.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-10T18:47:42.999 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T18:47:43.063 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:47:43.087 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:43.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:43.088 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:47:43.189 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:47:43.212 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:43.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:43.213 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:47:43.313 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:47:43.336 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:43.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:43.337 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:47:43.437 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:47:43.460 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:43.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:43.461 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:47:43.562 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:47:43.587 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:43.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:43.588 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:47:43.688 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:47:43.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:43 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2053069188' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:47:43.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:43 smithi018 ceph-mon[29336]: osdmap e92: 8 total, 8 up, 7 in 2024-09-10T18:47:43.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:43 smithi018 ceph-mon[29336]: pgmap v79: 148 pgs: 148 active+clean; 582 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s; 340 B/s, 9 objects/s recovering 2024-09-10T18:47:43.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:43 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2053069188' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:47:43.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:43 smithi018 ceph-mon[40196]: osdmap e92: 8 total, 8 up, 7 in 2024-09-10T18:47:43.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:43 smithi018 ceph-mon[40196]: pgmap v79: 148 pgs: 148 active+clean; 582 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s; 340 B/s, 9 objects/s recovering 2024-09-10T18:47:43.710 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:43.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:43.711 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:47:43.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:43 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2053069188' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:47:43.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:43 smithi103 ceph-mon[33163]: osdmap e92: 8 total, 8 up, 7 in 2024-09-10T18:47:43.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:43 smithi103 ceph-mon[33163]: pgmap v79: 148 pgs: 148 active+clean; 582 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s; 340 B/s, 9 objects/s recovering 2024-09-10T18:47:43.811 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:47:43.832 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:43.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:43.834 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:47:43.934 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:47:43.959 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:43.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:43.960 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:47:44.061 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:44.083 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:44.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:44.084 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:44.133 DEBUG:teuthology.orchestra.run.smithi018:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_0 bench 90 write' 2024-09-10T18:47:44.136 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-10T18:47:44.136 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-10T18:47:44.185 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:44.197 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:hints = 1 2024-09-10T18:47:44.197 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-10T18:47:44.197 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Object prefix: benchmark_data_smithi018_78628 2024-09-10T18:47:44.198 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T18:47:44.198 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 0 0 0 0 0 0 - 0 2024-09-10T18:47:44.207 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:44.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:44.208 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:44.309 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:44.339 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:44.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:44.340 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:44.441 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:47:44.482 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:44.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:44.484 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:47:44.585 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:47:44.630 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:44.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:44.631 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:47:44.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:44 smithi018 ceph-mon[40196]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T18:47:44.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:44 smithi018 ceph-mon[29336]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T18:47:44.732 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:47:44.764 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:44.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:44.766 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:47:44.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:44 smithi103 ceph-mon[33163]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T18:47:44.867 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:47:44.873 INFO:teuthology.orchestra.run.smithi018.stdout:ERROR: (22) Invalid argument 2024-09-10T18:47:44.873 INFO:teuthology.orchestra.run.smithi018.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:47:44.907 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:44.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:44.908 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:47:45.009 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:47:45.050 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:45.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:45.051 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:47:45.152 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:47:45.191 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:45.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:45.192 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:47:45.197 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 1 16 3678 3662 228.868 228.875 0.00793668 0.00435642 2024-09-10T18:47:45.293 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:47:45.325 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:45.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:45.326 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:47:45.427 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:47:45.460 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:45.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:45.461 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:47:45.562 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:45.593 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:45.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:45.594 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:45.678 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:45 smithi018 ceph-mon[40196]: pgmap v80: 148 pgs: 148 active+clean; 582 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 771 B/s rd, 0 op/s; 307 B/s, 8 objects/s recovering 2024-09-10T18:47:45.679 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:45 smithi018 ceph-mon[29336]: pgmap v80: 148 pgs: 148 active+clean; 582 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 771 B/s rd, 0 op/s; 307 B/s, 8 objects/s recovering 2024-09-10T18:47:45.694 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:47:45.724 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:45.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:45.725 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:47:45.726 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-10T18:47:45.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:45 smithi103 ceph-mon[33163]: pgmap v80: 148 pgs: 148 active+clean; 582 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 771 B/s rd, 0 op/s; 307 B/s, 8 objects/s recovering 2024-09-10T18:47:45.826 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:45.862 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:45.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:45.863 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:45.963 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:47:45.995 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:45.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:45.996 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:47:46.096 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:47:46.137 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:46.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:46.138 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:47:46.198 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 2 16 7268 7252 226.538 224.375 0.00420791 0.00440234 2024-09-10T18:47:46.239 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:47:46.271 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:46.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:46.272 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:47:46.372 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:47:46.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:46.395 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:46.396 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:47:46.439 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:47:46 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:47:46.496 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:47:46.534 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:46.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:46.535 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:47:46.599 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:46 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:46.600 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:46 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:46.636 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:47:46.669 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:46.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:46.670 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:47:46.730 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:47:46.771 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:46.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:46 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:46.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:46.818 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:46.818 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:46.920 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:47:46.959 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:46.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:46.960 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:47:47.061 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:47.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:47.116 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:47.116 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:47.198 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 3 16 10429 10413 216.874 197.562 0.00591196 0.00460334 2024-09-10T18:47:47.216 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:47:47.248 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:47.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:47.249 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:47:47.350 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:47:47.387 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:47.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:47.388 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:47:47.489 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:47:47.522 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:47.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:47.589 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:47:47.690 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:47:47.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:47.735 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:47.736 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:47:47.836 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:47:47.869 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:47.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:47.870 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:47:47.971 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:48.001 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:48.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:48.002 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:48.103 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:47:48.144 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:48.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:48.145 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:47:48.198 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 4 16 12936 12920 201.819 156.688 0.00457652 0.00494675 2024-09-10T18:47:48.246 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:48.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:48.294 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:48.294 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:48.369 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:48 smithi018 ceph-mon[40196]: pgmap v81: 148 pgs: 148 active+clean; 418 MiB data, 2.9 GiB used, 623 GiB / 626 GiB avail; 1.2 KiB/s rd, 52 MiB/s wr, 835 op/s; 255 B/s, 7 objects/s recovering 2024-09-10T18:47:48.369 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:48 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:48.369 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:48 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:48.369 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:48 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:47:48.369 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:48 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:47:48.369 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:48 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:48.369 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:48 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:47:48.369 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:48 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1400396381' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:47:48.370 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:48 smithi018 ceph-mon[29336]: pgmap v81: 148 pgs: 148 active+clean; 418 MiB data, 2.9 GiB used, 623 GiB / 626 GiB avail; 1.2 KiB/s rd, 52 MiB/s wr, 835 op/s; 255 B/s, 7 objects/s recovering 2024-09-10T18:47:48.370 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:48 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:48.370 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:48 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:48.370 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:48 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:47:48.370 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:48 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:47:48.370 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:48 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:48.370 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:48 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:47:48.370 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:48 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1400396381' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:47:48.395 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:47:48.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:48.438 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:48.439 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:47:48.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:48 smithi103 ceph-mon[33163]: pgmap v81: 148 pgs: 148 active+clean; 418 MiB data, 2.9 GiB used, 623 GiB / 626 GiB avail; 1.2 KiB/s rd, 52 MiB/s wr, 835 op/s; 255 B/s, 7 objects/s recovering 2024-09-10T18:47:48.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:48 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:48.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:48 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:48.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:48 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:47:48.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:48 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:47:48.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:48 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:48.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:48 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:47:48.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:48 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1400396381' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:47:48.539 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:48.579 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:48.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:48.581 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:48.590 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-10T18:47:48.590 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T18:47:48.590 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-10T18:47:48.590 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd dump -f json-pretty 2024-09-10T18:47:48.681 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:47:48.715 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:48.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:48.717 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:47:48.817 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:47:48.850 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:48.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:48.851 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:47:48.952 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:47:48.981 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:48.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:48.982 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:47:49.083 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:47:49.103 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:49.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:49.104 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:47:49.198 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 5 16 15124 15108 188.805 136.75 0.00330122 0.00515378 2024-09-10T18:47:49.205 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:47:49.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:49.246 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:49.246 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:47:49.347 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:47:49.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:49.383 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:49.383 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:47:49.484 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:47:49.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:49.527 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:49.528 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:47:49.628 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:49.629 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:47:49.630 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:47:49.630 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:47:49.630 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:47:49.630 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:47:49.630 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:47:49.630 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:47:49.630 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:47:49.630 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:47:49.630 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:47:49.630 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:47:49.630 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:47:49.631 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:47:49.631 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:47:49.631 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:47:49.631 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:47:49.669 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:49.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:49.670 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:49.770 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:49.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:49 smithi103 ceph-mon[33163]: pgmap v82: 148 pgs: 148 active+clean; 418 MiB data, 2.9 GiB used, 623 GiB / 626 GiB avail; 1.0 KiB/s rd, 43 MiB/s wr, 694 op/s; 211 B/s, 6 objects/s recovering 2024-09-10T18:47:49.812 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:49.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:49.813 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:49.914 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:49.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:49 smithi018 ceph-mon[29336]: pgmap v82: 148 pgs: 148 active+clean; 418 MiB data, 2.9 GiB used, 623 GiB / 626 GiB avail; 1.0 KiB/s rd, 43 MiB/s wr, 694 op/s; 211 B/s, 6 objects/s recovering 2024-09-10T18:47:49.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:49 smithi018 ceph-mon[40196]: pgmap v82: 148 pgs: 148 active+clean; 418 MiB data, 2.9 GiB used, 623 GiB / 626 GiB avail; 1.0 KiB/s rd, 43 MiB/s wr, 694 op/s; 211 B/s, 6 objects/s recovering 2024-09-10T18:47:49.945 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:49.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:49.946 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:50.047 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:50.085 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:50.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:50.086 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:50.187 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:47:50.200 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 6 15 17758 17743 184.735 164.688 0.0044329 0.00540676 2024-09-10T18:47:50.226 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:50.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:50.227 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:47:50.328 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:47:50.369 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:50.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:50.370 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:47:50.471 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:50.503 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:47:50 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:47:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:47:50.510 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:50.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:50.512 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:50.612 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:47:50.645 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:50.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:50.646 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:47:50.660 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:50 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:47:50.660 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:50 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:47:50.660 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:50 smithi018 ceph-mon[40196]: osdmap e93: 8 total, 8 up, 7 in 2024-09-10T18:47:50.661 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:50 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:47:50.661 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:50 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:47:50.661 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:50 smithi018 ceph-mon[29336]: osdmap e93: 8 total, 8 up, 7 in 2024-09-10T18:47:50.749 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:47:50.754 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:50 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:47:50.754 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:50 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:47:50.754 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:50 smithi103 ceph-mon[33163]: osdmap e93: 8 total, 8 up, 7 in 2024-09-10T18:47:50.783 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:50.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:50.784 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:47:50.884 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:47:50.915 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:50.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:50.916 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:47:50.954 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:47:51.017 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:51.070 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:51.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:51.071 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:51.172 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:47:51.200 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 7 16 20286 20270 180.906 157.938 0.00587035 0.00552094 2024-09-10T18:47:51.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:51.216 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:51.216 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:47:51.262 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:47:50 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:47:50] "GET /metrics HTTP/1.1" 200 222579 "" "Prometheus/2.43.0" 2024-09-10T18:47:51.284 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:47:51.317 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:47:51.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:51.355 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:51.355 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:47:51.456 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:51.496 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:51.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:51.498 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:51.519 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is set 2024-09-10T18:47:51.519 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:51 smithi018 ceph-mon[40196]: pgmap v84: 148 pgs: 148 active+clean; 1.2 GiB data, 5.5 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 144 MiB/s wr, 2.31k op/s 2024-09-10T18:47:51.519 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:51 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2815399786' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:47:51.520 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:51 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:47:51.520 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:51 smithi018 ceph-mon[29336]: pgmap v84: 148 pgs: 148 active+clean; 1.2 GiB data, 5.5 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 144 MiB/s wr, 2.31k op/s 2024-09-10T18:47:51.520 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:51 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2815399786' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:47:51.520 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:51 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:47:51.599 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:51.640 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:51.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:51.641 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:51.742 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:51.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:51 smithi103 ceph-mon[33163]: pgmap v84: 148 pgs: 148 active+clean; 1.2 GiB data, 5.5 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 144 MiB/s wr, 2.31k op/s 2024-09-10T18:47:51.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:51 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2815399786' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:47:51.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:51 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:47:51.781 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:51.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:51.782 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:51.883 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:47:51.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:51.916 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:51.916 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:47:52.017 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:52.055 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:52.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:52.057 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:52.157 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:52.190 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:52.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:52.191 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:52.200 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 8 16 22566 22550 176.105 142.5 0.00606659 0.00567255 2024-09-10T18:47:52.293 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:47:52.331 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:52.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:52.332 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:47:52.413 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T18:47:52.433 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:47:52.465 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:52.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:52.466 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:47:52.567 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:47:52.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:52.590 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:52.590 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:47:52.690 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:52.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:52.723 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:52.723 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:52.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:52 smithi103 ceph-mon[33163]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:47:52.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:52 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:47:52.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:52 smithi103 ceph-mon[33163]: osdmap e94: 8 total, 8 up, 7 in 2024-09-10T18:47:52.824 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:47:52.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:52.863 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:52.863 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:47:52.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:52 smithi018 ceph-mon[40196]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:47:52.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:52 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:47:52.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:52 smithi018 ceph-mon[40196]: osdmap e94: 8 total, 8 up, 7 in 2024-09-10T18:47:52.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:52 smithi018 ceph-mon[29336]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:47:52.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:52 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:47:52.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:52 smithi018 ceph-mon[29336]: osdmap e94: 8 total, 8 up, 7 in 2024-09-10T18:47:52.964 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:47:52.995 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:52.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:52.996 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:47:53.097 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:47:53.136 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:53.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:53.137 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:47:53.200 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 9 16 24972 24956 173.245 150.375 0.00249765 0.00576563 2024-09-10T18:47:53.239 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:47:53.267 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:53.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:53.269 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:47:53.370 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:53.415 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:53.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:53.416 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:53.517 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:47:53.551 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:53.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:53.552 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:47:53.653 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:47:53.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:53.686 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:53.686 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:47:53.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:53 smithi103 ceph-mon[33163]: pgmap v86: 148 pgs: 148 active+clean; 1.2 GiB data, 5.5 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 155 MiB/s wr, 2.48k op/s 2024-09-10T18:47:53.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:53 smithi103 ceph-mon[33163]: osdmap e95: 8 total, 8 up, 7 in 2024-09-10T18:47:53.783 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:53 smithi018 ceph-mon[40196]: pgmap v86: 148 pgs: 148 active+clean; 1.2 GiB data, 5.5 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 155 MiB/s wr, 2.48k op/s 2024-09-10T18:47:53.783 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:53 smithi018 ceph-mon[40196]: osdmap e95: 8 total, 8 up, 7 in 2024-09-10T18:47:53.783 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:53 smithi018 ceph-mon[29336]: pgmap v86: 148 pgs: 148 active+clean; 1.2 GiB data, 5.5 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 155 MiB/s wr, 2.48k op/s 2024-09-10T18:47:53.783 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:53 smithi018 ceph-mon[29336]: osdmap e95: 8 total, 8 up, 7 in 2024-09-10T18:47:53.787 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:47:53.817 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:53.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:53.818 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:47:53.919 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:47:53.956 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:53.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:53.957 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:47:54.058 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:54.092 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:54.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:54.093 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:54.194 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:47:54.200 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 10 16 27473 27457 171.551 156.312 0.00555986 0.00582426 2024-09-10T18:47:54.226 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:54.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:54.227 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:47:54.328 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:47:54.367 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:54.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:54.368 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:47:54.469 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:47:54.507 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:54.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:54.508 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:47:54.609 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:47:54.635 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:54.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:54.636 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:47:54.737 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:47:54.778 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:54.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:54.780 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:47:54.880 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:54.911 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:54.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:54.912 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:55.013 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:47:55.043 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:55.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:55.044 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:47:55.145 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:47:55.182 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:55.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:55.183 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:47:55.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:54 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:55.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:54 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:55.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:54 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:55.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:54 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:55.200 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 11 16 29471 29455 167.307 124.875 0.00441219 0.0059712 2024-09-10T18:47:55.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:54 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:55.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:54 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:55.284 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:47:55.315 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:55.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:55.316 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:47:55.417 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:47:55.454 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:55.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:55.455 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:47:55.556 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:55.580 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:55.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:55.581 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:55.682 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:47:55.714 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:55.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:55.715 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:47:55.816 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:55.831 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-10T18:47:55.850 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:55.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:55.851 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:55.952 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:47:55.993 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:55.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:55.994 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:47:56.095 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:56.125 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:56.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:56.126 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:56.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:56 smithi018 ceph-mon[40196]: pgmap v88: 148 pgs: 148 active+clean; 1.2 GiB data, 5.5 GiB used, 620 GiB / 626 GiB avail; 853 B/s rd, 137 MiB/s wr, 2.20k op/s 2024-09-10T18:47:56.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:56 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:56.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:56 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:47:56.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:56 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 5]}]: dispatch 2024-09-10T18:47:56.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:56 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.5", "id": [0, 6]}]: dispatch 2024-09-10T18:47:56.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:56 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 5]}]: dispatch 2024-09-10T18:47:56.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:56 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.10", "id": [7, 3]}]: dispatch 2024-09-10T18:47:56.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:56 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.f", "id": [3, 0]}]: dispatch 2024-09-10T18:47:56.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:56 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.5", "id": [0, 6]}]: dispatch 2024-09-10T18:47:56.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:56 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.10", "id": [7, 3]}]: dispatch 2024-09-10T18:47:56.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:56 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.f", "id": [3, 0]}]: dispatch 2024-09-10T18:47:56.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:56 smithi018 ceph-mon[29336]: pgmap v88: 148 pgs: 148 active+clean; 1.2 GiB data, 5.5 GiB used, 620 GiB / 626 GiB avail; 853 B/s rd, 137 MiB/s wr, 2.20k op/s 2024-09-10T18:47:56.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:56 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:56.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:56 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:47:56.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:56 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 5]}]: dispatch 2024-09-10T18:47:56.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:56 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.5", "id": [0, 6]}]: dispatch 2024-09-10T18:47:56.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:56 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 5]}]: dispatch 2024-09-10T18:47:56.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:56 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.10", "id": [7, 3]}]: dispatch 2024-09-10T18:47:56.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:56 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.f", "id": [3, 0]}]: dispatch 2024-09-10T18:47:56.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:56 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.5", "id": [0, 6]}]: dispatch 2024-09-10T18:47:56.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:56 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.10", "id": [7, 3]}]: dispatch 2024-09-10T18:47:56.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:56 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.f", "id": [3, 0]}]: dispatch 2024-09-10T18:47:56.201 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 12 16 31327 31311 163.028 116 0.00584755 0.00612807 2024-09-10T18:47:56.227 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:56.270 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:56.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:56.271 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:56.372 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:47:56.405 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:56.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:56.406 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:47:56.507 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:47:56.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:47:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:47:56.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:56 smithi103 ceph-mon[33163]: pgmap v88: 148 pgs: 148 active+clean; 1.2 GiB data, 5.5 GiB used, 620 GiB / 626 GiB avail; 853 B/s rd, 137 MiB/s wr, 2.20k op/s 2024-09-10T18:47:56.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:56 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:47:56.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:56 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:47:56.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:56 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 5]}]: dispatch 2024-09-10T18:47:56.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:56 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.5", "id": [0, 6]}]: dispatch 2024-09-10T18:47:56.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:56 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 5]}]: dispatch 2024-09-10T18:47:56.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:56 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.10", "id": [7, 3]}]: dispatch 2024-09-10T18:47:56.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:56 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.f", "id": [3, 0]}]: dispatch 2024-09-10T18:47:56.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:56 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.5", "id": [0, 6]}]: dispatch 2024-09-10T18:47:56.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:56 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.10", "id": [7, 3]}]: dispatch 2024-09-10T18:47:56.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:56 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.f", "id": [3, 0]}]: dispatch 2024-09-10T18:47:56.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:56.546 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:56.546 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:47:56.647 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:56.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:56.698 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:56.699 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:56.799 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:47:56.820 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:56.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:56.822 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:47:56.922 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:56.954 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:56.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:56.955 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:57.056 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:57.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:57.097 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:57.098 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:57.198 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:57.201 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 13 16 33325 33309 160.093 124.875 0.0042692 0.00624093 2024-09-10T18:47:57.230 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:57.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:57.231 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:57.332 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:47:57.370 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:57.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:57.372 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:47:57.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:57 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 5]}]': finished 2024-09-10T18:47:57.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:57 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.5", "id": [0, 6]}]': finished 2024-09-10T18:47:57.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:57 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.10", "id": [7, 3]}]': finished 2024-09-10T18:47:57.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:57 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.f", "id": [3, 0]}]': finished 2024-09-10T18:47:57.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:57 smithi018 ceph-mon[40196]: osdmap e96: 8 total, 8 up, 7 in 2024-09-10T18:47:57.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:57 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:57.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:57 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 5]}]': finished 2024-09-10T18:47:57.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:57 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.5", "id": [0, 6]}]': finished 2024-09-10T18:47:57.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:57 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.10", "id": [7, 3]}]': finished 2024-09-10T18:47:57.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:57 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.f", "id": [3, 0]}]': finished 2024-09-10T18:47:57.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:57 smithi018 ceph-mon[29336]: osdmap e96: 8 total, 8 up, 7 in 2024-09-10T18:47:57.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:57 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:57.473 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:47:57.502 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:57.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:57.504 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:47:57.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:57 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 5]}]': finished 2024-09-10T18:47:57.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:57 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.5", "id": [0, 6]}]': finished 2024-09-10T18:47:57.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:57 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.10", "id": [7, 3]}]': finished 2024-09-10T18:47:57.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:57 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.f", "id": [3, 0]}]': finished 2024-09-10T18:47:57.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:57 smithi103 ceph-mon[33163]: osdmap e96: 8 total, 8 up, 7 in 2024-09-10T18:47:57.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:57 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:57.605 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:47:57.637 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:57.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:57.638 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:47:57.739 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:57.793 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:57.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:57.794 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:57.895 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:47:57.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:57.929 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:57.929 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:47:58.030 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:47:58.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:58.079 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:58.079 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:47:58.180 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:58.201 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 14 15 35497 35482 158.356 135.812 0.00527066 0.00630985 2024-09-10T18:47:58.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:58.216 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:58.216 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:58.317 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:47:58.341 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:58 smithi018 ceph-mon[29336]: pgmap v90: 148 pgs: 148 active+clean; 1.9 GiB data, 7.6 GiB used, 618 GiB / 626 GiB avail; 853 B/s rd, 117 MiB/s wr, 1.86k op/s 2024-09-10T18:47:58.341 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:58 smithi018 ceph-mon[29336]: osdmap e97: 8 total, 8 up, 7 in 2024-09-10T18:47:58.341 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:58 smithi018 ceph-mon[40196]: pgmap v90: 148 pgs: 148 active+clean; 1.9 GiB data, 7.6 GiB used, 618 GiB / 626 GiB avail; 853 B/s rd, 117 MiB/s wr, 1.86k op/s 2024-09-10T18:47:58.341 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:58 smithi018 ceph-mon[40196]: osdmap e97: 8 total, 8 up, 7 in 2024-09-10T18:47:58.352 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:58.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:58.353 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:47:58.454 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:58.492 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:58.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:58.493 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:58.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:58 smithi103 ceph-mon[33163]: pgmap v90: 148 pgs: 148 active+clean; 1.9 GiB data, 7.6 GiB used, 618 GiB / 626 GiB avail; 853 B/s rd, 117 MiB/s wr, 1.86k op/s 2024-09-10T18:47:58.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:58 smithi103 ceph-mon[33163]: osdmap e97: 8 total, 8 up, 7 in 2024-09-10T18:47:58.594 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:47:58.624 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:58.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:58.625 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:47:58.726 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:47:58.762 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:58.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:58.763 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:47:58.864 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:47:58.911 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:58.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:58.912 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:47:59.013 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:59.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:59.044 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:59.045 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:59.146 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:47:59.173 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:59.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:59.174 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:47:59.177 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:47:59.201 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 15 16 37530 37514 156.266 127 0.0066128 0.00639365 2024-09-10T18:47:59.212 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T18:47:59.213 INFO:teuthology.orchestra.run.smithi018.stdout: "size": 20, 2024-09-10T18:47:59.213 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 600, 2024-09-10T18:47:59.213 INFO:teuthology.orchestra.run.smithi018.stdout: "ops": [ 2024-09-10T18:47:59.213 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.213 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.14790.0:15084 7.f 7:f2b10b86:::benchmark_data_smithi018_78628_object15083:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-10T18:47:59.213 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:47:49.058358+0000", 2024-09-10T18:47:59.214 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 10.148654799999999, 2024-09-10T18:47:59.214 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.14719653699999999, 2024-09-10T18:47:59.214 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:47:59.214 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:47:59.214 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:47:59.214 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.14790", 2024-09-10T18:47:59.214 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:47:59.214 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 15084 2024-09-10T18:47:59.215 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.215 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:47:59.215 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.215 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:47:59.215 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.058358+0000", 2024-09-10T18:47:59.215 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.215 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.216 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.216 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:47:59.216 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.058358+0000", 2024-09-10T18:47:59.216 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.216 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.216 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.216 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:47:59.217 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.058357+0000", 2024-09-10T18:47:59.217 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999981 2024-09-10T18:47:59.217 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.217 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.217 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:47:59.217 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.058378+0000", 2024-09-10T18:47:59.217 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.0971999999999998e-05 2024-09-10T18:47:59.217 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.218 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.218 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:47:59.218 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.058390+0000", 2024-09-10T18:47:59.218 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.273e-05 2024-09-10T18:47:59.218 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.218 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.218 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:47:59.219 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.058396+0000", 2024-09-10T18:47:59.219 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.964e-06 2024-09-10T18:47:59.219 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.219 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.219 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:47:59.219 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.064785+0000", 2024-09-10T18:47:59.219 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.006388861 2024-09-10T18:47:59.219 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.220 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.220 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:47:59.220 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.064914+0000", 2024-09-10T18:47:59.220 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000128634 2024-09-10T18:47:59.220 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.220 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.221 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,7", 2024-09-10T18:47:59.221 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.065060+0000", 2024-09-10T18:47:59.221 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000145673 2024-09-10T18:47:59.221 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.221 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.221 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:47:59.221 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.067738+0000", 2024-09-10T18:47:59.222 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0026781830000000002 2024-09-10T18:47:59.222 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.222 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.222 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:47:59.222 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.068105+0000", 2024-09-10T18:47:59.222 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00036769599999999998 2024-09-10T18:47:59.222 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.223 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.223 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:47:59.223 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.205229+0000", 2024-09-10T18:47:59.223 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.137123209 2024-09-10T18:47:59.223 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.223 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.223 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:47:59.224 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.205511+0000", 2024-09-10T18:47:59.224 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00028202300000000001 2024-09-10T18:47:59.224 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.224 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.224 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:47:59.224 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.205555+0000", 2024-09-10T18:47:59.224 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.4271999999999999e-05 2024-09-10T18:47:59.225 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.225 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:47:59.225 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.225 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.225 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.225 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.14790.0:15089 7.f 7:f2492c59:::benchmark_data_smithi018_78628_object15088:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-10T18:47:59.225 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:47:49.059410+0000", 2024-09-10T18:47:59.226 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 10.147603241000001, 2024-09-10T18:47:59.226 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.14654346400000001, 2024-09-10T18:47:59.226 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:47:59.226 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:47:59.226 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:47:59.226 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.14790", 2024-09-10T18:47:59.226 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:47:59.227 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 15089 2024-09-10T18:47:59.227 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.227 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:47:59.227 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.227 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:47:59.227 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.059410+0000", 2024-09-10T18:47:59.227 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.228 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.228 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.228 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:47:59.228 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.059410+0000", 2024-09-10T18:47:59.228 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.228 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.228 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.229 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:47:59.229 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.059409+0000", 2024-09-10T18:47:59.229 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T18:47:59.229 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.229 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.229 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:47:59.230 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.059433+0000", 2024-09-10T18:47:59.230 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.3879e-05 2024-09-10T18:47:59.230 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.230 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.230 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:47:59.230 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.059454+0000", 2024-09-10T18:47:59.230 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.1019999999999999e-05 2024-09-10T18:47:59.230 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.231 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.231 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:47:59.231 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.059460+0000", 2024-09-10T18:47:59.231 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.6729999999999997e-06 2024-09-10T18:47:59.231 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.231 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.231 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:47:59.232 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.065892+0000", 2024-09-10T18:47:59.232 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0064317209999999996 2024-09-10T18:47:59.232 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.232 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.232 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:47:59.232 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.066021+0000", 2024-09-10T18:47:59.232 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00012932199999999999 2024-09-10T18:47:59.233 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.233 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.233 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,7", 2024-09-10T18:47:59.233 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.066437+0000", 2024-09-10T18:47:59.233 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00041599699999999998 2024-09-10T18:47:59.234 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.234 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.234 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:47:59.234 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.068144+0000", 2024-09-10T18:47:59.234 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00170663 2024-09-10T18:47:59.234 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.234 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.234 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:47:59.235 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.204771+0000", 2024-09-10T18:47:59.235 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.1366271 2024-09-10T18:47:59.235 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.235 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.235 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:47:59.235 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.205597+0000", 2024-09-10T18:47:59.235 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00082606499999999996 2024-09-10T18:47:59.236 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.236 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.236 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:47:59.236 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.205912+0000", 2024-09-10T18:47:59.236 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00031569600000000002 2024-09-10T18:47:59.236 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.236 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.237 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:47:59.237 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.205953+0000", 2024-09-10T18:47:59.237 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.0991000000000001e-05 2024-09-10T18:47:59.237 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.237 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:47:59.237 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.237 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.237 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.238 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.14790.0:15102 7.f 7:f53ad186:::benchmark_data_smithi018_78628_object15101:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-10T18:47:59.238 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:47:49.063568+0000", 2024-09-10T18:47:59.238 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 10.143444727, 2024-09-10T18:47:59.238 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.143460796, 2024-09-10T18:47:59.238 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:47:59.238 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:47:59.238 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:47:59.239 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.14790", 2024-09-10T18:47:59.239 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:47:59.239 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 15102 2024-09-10T18:47:59.239 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.239 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:47:59.239 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.239 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:47:59.240 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.063568+0000", 2024-09-10T18:47:59.240 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.240 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.240 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.240 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:47:59.240 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.063568+0000", 2024-09-10T18:47:59.240 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.240 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.241 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.241 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:47:59.241 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.063568+0000", 2024-09-10T18:47:59.241 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:47:59.241 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.241 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.241 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:47:59.242 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.063589+0000", 2024-09-10T18:47:59.242 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.1931999999999999e-05 2024-09-10T18:47:59.242 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.242 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.242 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:47:59.242 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.063599+0000", 2024-09-10T18:47:59.242 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.3449999999999995e-06 2024-09-10T18:47:59.242 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.243 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.243 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:47:59.243 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.063604+0000", 2024-09-10T18:47:59.243 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.5330000000000002e-06 2024-09-10T18:47:59.243 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.243 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.243 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:47:59.243 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.068172+0000", 2024-09-10T18:47:59.244 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0045676079999999999 2024-09-10T18:47:59.244 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.244 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.244 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:47:59.244 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.068284+0000", 2024-09-10T18:47:59.244 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00011201699999999999 2024-09-10T18:47:59.244 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.245 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.245 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,7", 2024-09-10T18:47:59.245 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.068607+0000", 2024-09-10T18:47:59.245 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00032275000000000001 2024-09-10T18:47:59.245 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.245 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.245 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:47:59.245 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.204807+0000", 2024-09-10T18:47:59.246 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.13619999099999999 2024-09-10T18:47:59.246 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.246 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.246 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:47:59.246 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.205591+0000", 2024-09-10T18:47:59.246 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00078381999999999996 2024-09-10T18:47:59.246 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.247 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.247 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:47:59.247 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.205993+0000", 2024-09-10T18:47:59.247 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00040267999999999999 2024-09-10T18:47:59.247 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.247 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.247 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:47:59.247 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.206268+0000", 2024-09-10T18:47:59.248 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00027449499999999998 2024-09-10T18:47:59.248 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.248 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.248 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:47:59.248 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.207029+0000", 2024-09-10T18:47:59.248 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000761533 2024-09-10T18:47:59.248 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.249 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:47:59.249 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.249 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.249 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.249 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.14790.0:15105 7.7 e92/86 7:eab71933:::benchmark_data_smithi018_78628_object15104:head v 92'956, mlcod=92'952)", 2024-09-10T18:47:59.249 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:47:49.065765+0000", 2024-09-10T18:47:59.249 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 10.141248621999999, 2024-09-10T18:47:59.249 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.14022161899999999, 2024-09-10T18:47:59.250 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:47:59.250 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:47:59.250 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:47:59.250 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.250 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:47:59.250 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.065765+0000", 2024-09-10T18:47:59.250 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.250 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.251 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.251 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:47:59.251 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.065765+0000", 2024-09-10T18:47:59.251 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.251 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.251 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.251 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:47:59.251 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.065764+0000", 2024-09-10T18:47:59.252 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T18:47:59.252 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.252 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.252 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:47:59.252 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.065806+0000", 2024-09-10T18:47:59.252 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.1813000000000001e-05 2024-09-10T18:47:59.252 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.253 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.253 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:47:59.253 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.065822+0000", 2024-09-10T18:47:59.253 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.6350000000000001e-05 2024-09-10T18:47:59.253 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.253 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.253 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:47:59.253 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.065828+0000", 2024-09-10T18:47:59.254 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.6670000000000001e-06 2024-09-10T18:47:59.254 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.254 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.254 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:47:59.254 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.066567+0000", 2024-09-10T18:47:59.254 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000739367 2024-09-10T18:47:59.254 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.255 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.255 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:47:59.255 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.067370+0000", 2024-09-10T18:47:59.255 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00080255400000000003 2024-09-10T18:47:59.255 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.255 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.255 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:47:59.255 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.205956+0000", 2024-09-10T18:47:59.256 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.138586655 2024-09-10T18:47:59.256 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.256 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.256 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:47:59.256 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.205986+0000", 2024-09-10T18:47:59.256 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.9887999999999999e-05 2024-09-10T18:47:59.256 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.257 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:47:59.257 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.257 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.257 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.257 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop_reply(client.14790.0:15084 7.f e92/86 ondisk, result = 0)", 2024-09-10T18:47:59.257 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:47:49.066332+0000", 2024-09-10T18:47:59.257 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 10.140680954, 2024-09-10T18:47:59.258 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.13922705699999999, 2024-09-10T18:47:59.258 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:47:59.258 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "started", 2024-09-10T18:47:59.258 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:47:59.258 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.258 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:47:59.258 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.066332+0000", 2024-09-10T18:47:59.258 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.259 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.259 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.259 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:47:59.259 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.066332+0000", 2024-09-10T18:47:59.259 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.259 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.259 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.259 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:47:59.260 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.066332+0000", 2024-09-10T18:47:59.260 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T18:47:59.260 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.260 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.260 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:47:59.260 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.066336+0000", 2024-09-10T18:47:59.260 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.826e-06 2024-09-10T18:47:59.260 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.261 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.261 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:47:59.261 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.066356+0000", 2024-09-10T18:47:59.261 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.9568e-05 2024-09-10T18:47:59.261 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.261 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.261 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:47:59.262 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.066361+0000", 2024-09-10T18:47:59.262 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.8609999999999997e-06 2024-09-10T18:47:59.262 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.262 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.262 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:47:59.262 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.067724+0000", 2024-09-10T18:47:59.262 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.001362994 2024-09-10T18:47:59.263 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.263 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.263 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:47:59.263 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.067732+0000", 2024-09-10T18:47:59.263 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.354e-06 2024-09-10T18:47:59.263 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.263 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.264 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:47:59.264 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.205559+0000", 2024-09-10T18:47:59.264 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.137827163 2024-09-10T18:47:59.264 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.264 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:47:59.264 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.264 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.264 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.265 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop_reply(client.14790.0:15089 7.f e92/86 ondisk, result = 0)", 2024-09-10T18:47:59.265 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:47:49.068389+0000", 2024-09-10T18:47:59.265 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 10.138623905999999, 2024-09-10T18:47:59.265 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.13638294200000001, 2024-09-10T18:47:59.266 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:47:59.267 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "started", 2024-09-10T18:47:59.267 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:47:59.267 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.267 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:47:59.267 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.068389+0000", 2024-09-10T18:47:59.267 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.268 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.268 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.268 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:47:59.268 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.068389+0000", 2024-09-10T18:47:59.268 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.268 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.268 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.268 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:47:59.269 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.068388+0000", 2024-09-10T18:47:59.269 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:47:59.269 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.269 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.269 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:47:59.269 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.068394+0000", 2024-09-10T18:47:59.269 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.2390000000000002e-06 2024-09-10T18:47:59.270 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.270 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.270 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:47:59.270 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.068404+0000", 2024-09-10T18:47:59.270 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0798999999999999e-05 2024-09-10T18:47:59.270 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.270 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.270 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:47:59.271 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.068410+0000", 2024-09-10T18:47:59.271 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.5729999999999998e-06 2024-09-10T18:47:59.271 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.271 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.271 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:47:59.271 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.204758+0000", 2024-09-10T18:47:59.271 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.13634766700000001 2024-09-10T18:47:59.272 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.272 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.272 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:47:59.272 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.204765+0000", 2024-09-10T18:47:59.272 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.6159999999999996e-06 2024-09-10T18:47:59.272 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.272 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.273 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:47:59.273 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.204772+0000", 2024-09-10T18:47:59.273 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.0090000000000001e-06 2024-09-10T18:47:59.273 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.273 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:47:59.273 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.273 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.273 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.274 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.14790.0:15114 7.5 e92/88 7:a766245d:::benchmark_data_smithi018_78628_object15113:head v 92'938, mlcod=92'936)", 2024-09-10T18:47:59.274 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:47:49.068813+0000", 2024-09-10T18:47:59.274 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 10.138200661000001, 2024-09-10T18:47:59.274 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.13709186000000001, 2024-09-10T18:47:59.274 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:47:59.274 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:47:59.274 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:47:59.274 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.275 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:47:59.275 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.068813+0000", 2024-09-10T18:47:59.275 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.275 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.275 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.275 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:47:59.275 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.068813+0000", 2024-09-10T18:47:59.276 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.276 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.276 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.276 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:47:59.276 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.068812+0000", 2024-09-10T18:47:59.276 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T18:47:59.276 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.276 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.277 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:47:59.277 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.068848+0000", 2024-09-10T18:47:59.277 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.5839e-05 2024-09-10T18:47:59.277 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.277 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.277 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:47:59.277 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.202967+0000", 2024-09-10T18:47:59.278 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.134119133 2024-09-10T18:47:59.278 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.278 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.278 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:47:59.278 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.202989+0000", 2024-09-10T18:47:59.278 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.1642999999999999e-05 2024-09-10T18:47:59.278 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.278 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.279 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:47:59.279 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.204630+0000", 2024-09-10T18:47:59.279 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0016412670000000001 2024-09-10T18:47:59.279 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.279 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.279 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:47:59.279 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.204652+0000", 2024-09-10T18:47:59.280 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.1885000000000001e-05 2024-09-10T18:47:59.280 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.280 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.280 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:47:59.280 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.205864+0000", 2024-09-10T18:47:59.280 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.001212631 2024-09-10T18:47:59.280 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.280 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.281 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:47:59.281 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.205904+0000", 2024-09-10T18:47:59.281 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.0030000000000001e-05 2024-09-10T18:47:59.281 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.281 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:47:59.281 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.281 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.282 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.282 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.14790.0:15116 7.e e92/88 7:75fa8b39:::benchmark_data_smithi018_78628_object15115:head v 92'910, mlcod=92'908)", 2024-09-10T18:47:59.282 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:47:49.069468+0000", 2024-09-10T18:47:59.282 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 10.137545386999999, 2024-09-10T18:47:59.282 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.13572091999999999, 2024-09-10T18:47:59.282 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:47:59.282 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:47:59.283 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:47:59.283 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.283 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:47:59.283 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.069468+0000", 2024-09-10T18:47:59.283 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.283 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.283 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.283 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:47:59.284 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.069468+0000", 2024-09-10T18:47:59.284 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.284 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.284 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.284 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:47:59.284 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.069467+0000", 2024-09-10T18:47:59.284 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:47:59.285 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.285 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.285 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:47:59.285 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.069700+0000", 2024-09-10T18:47:59.285 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00023323699999999999 2024-09-10T18:47:59.285 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.285 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.285 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:47:59.286 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.202952+0000", 2024-09-10T18:47:59.286 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.133252174 2024-09-10T18:47:59.286 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.286 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.286 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:47:59.286 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.202964+0000", 2024-09-10T18:47:59.286 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1657e-05 2024-09-10T18:47:59.286 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.286 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.286 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:47:59.286 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.203026+0000", 2024-09-10T18:47:59.286 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.2201999999999998e-05 2024-09-10T18:47:59.287 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.287 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.287 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:47:59.287 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.203050+0000", 2024-09-10T18:47:59.287 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.3719000000000001e-05 2024-09-10T18:47:59.287 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.287 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.287 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:47:59.287 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.205061+0000", 2024-09-10T18:47:59.287 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0020112789999999999 2024-09-10T18:47:59.287 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.287 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.287 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:47:59.287 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.205189+0000", 2024-09-10T18:47:59.288 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00012742299999999999 2024-09-10T18:47:59.288 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.288 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:47:59.288 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.288 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.288 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.288 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.14790.0:28470 7.6 7:6d539808:::benchmark_data_smithi018_78628_object28469:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-10T18:47:59.288 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:47:54.614333+0000", 2024-09-10T18:47:59.288 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 4.5926799899999997, 2024-09-10T18:47:59.288 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.12513708500000001, 2024-09-10T18:47:59.288 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:47:59.288 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:47:59.289 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:47:59.289 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.14790", 2024-09-10T18:47:59.289 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:47:59.289 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 28470 2024-09-10T18:47:59.289 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.289 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:47:59.289 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.289 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:47:59.289 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.614333+0000", 2024-09-10T18:47:59.289 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.289 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.289 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.289 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:47:59.290 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.614333+0000", 2024-09-10T18:47:59.290 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.290 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.290 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.290 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:47:59.290 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.614332+0000", 2024-09-10T18:47:59.290 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:47:59.290 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.290 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.290 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:47:59.290 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.614376+0000", 2024-09-10T18:47:59.290 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.4027999999999999e-05 2024-09-10T18:47:59.290 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.291 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.291 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:47:59.291 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.614387+0000", 2024-09-10T18:47:59.291 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0849999999999999e-05 2024-09-10T18:47:59.291 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.291 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.291 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:47:59.291 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.614393+0000", 2024-09-10T18:47:59.291 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.558e-06 2024-09-10T18:47:59.291 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.291 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.291 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:47:59.291 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.615279+0000", 2024-09-10T18:47:59.292 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00088614200000000003 2024-09-10T18:47:59.292 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.292 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.292 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:47:59.292 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.615393+0000", 2024-09-10T18:47:59.292 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00011410599999999999 2024-09-10T18:47:59.292 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.292 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.292 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,7", 2024-09-10T18:47:59.292 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.615519+0000", 2024-09-10T18:47:59.292 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00012562200000000001 2024-09-10T18:47:59.292 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.292 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.293 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:47:59.293 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.617524+0000", 2024-09-10T18:47:59.293 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0020056729999999999 2024-09-10T18:47:59.293 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.293 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.293 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:47:59.293 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.739351+0000", 2024-09-10T18:47:59.293 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.12182685 2024-09-10T18:47:59.293 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.293 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.293 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:47:59.293 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.739372+0000", 2024-09-10T18:47:59.294 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.0432999999999999e-05 2024-09-10T18:47:59.294 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.294 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.294 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:47:59.294 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.739428+0000", 2024-09-10T18:47:59.294 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.6199000000000002e-05 2024-09-10T18:47:59.294 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.294 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.294 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:47:59.294 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.739470+0000", 2024-09-10T18:47:59.294 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.2561000000000001e-05 2024-09-10T18:47:59.294 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.294 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:47:59.295 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.295 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.295 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.295 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.14790.0:28477 7.e e95/88 7:797e068c:::benchmark_data_smithi018_78628_object28476:head v 95'1742, mlcod=95'1740)", 2024-09-10T18:47:59.295 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:47:54.617168+0000", 2024-09-10T18:47:59.295 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 4.5898451680000001, 2024-09-10T18:47:59.295 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.12234350200000001, 2024-09-10T18:47:59.295 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:47:59.295 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:47:59.295 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:47:59.295 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.295 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:47:59.295 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.617168+0000", 2024-09-10T18:47:59.296 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.296 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.296 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.296 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:47:59.296 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.617168+0000", 2024-09-10T18:47:59.296 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.296 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.296 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.296 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:47:59.296 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.617167+0000", 2024-09-10T18:47:59.296 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:47:59.296 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.296 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.297 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:47:59.297 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.617197+0000", 2024-09-10T18:47:59.297 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.0134e-05 2024-09-10T18:47:59.297 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.297 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.297 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:47:59.297 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.617207+0000", 2024-09-10T18:47:59.297 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.7880000000000001e-06 2024-09-10T18:47:59.297 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.297 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.297 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:47:59.297 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.617212+0000", 2024-09-10T18:47:59.297 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.0309999999999998e-06 2024-09-10T18:47:59.298 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.298 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.298 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:47:59.298 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.617344+0000", 2024-09-10T18:47:59.298 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000132185 2024-09-10T18:47:59.298 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.298 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.298 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:47:59.298 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.617361+0000", 2024-09-10T18:47:59.298 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.613e-05 2024-09-10T18:47:59.298 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.298 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.298 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:47:59.298 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.739482+0000", 2024-09-10T18:47:59.299 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.122121775 2024-09-10T18:47:59.299 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.299 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.299 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:47:59.299 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.739512+0000", 2024-09-10T18:47:59.299 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.9193e-05 2024-09-10T18:47:59.299 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.299 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:47:59.299 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.299 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.299 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.299 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.14790.0:28482 7.6 7:67a86dfe:::benchmark_data_smithi018_78628_object28481:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-10T18:47:59.299 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:47:54.618019+0000", 2024-09-10T18:47:59.300 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 4.5889937320000005, 2024-09-10T18:47:59.300 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.123980324, 2024-09-10T18:47:59.300 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:47:59.300 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:47:59.300 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:47:59.300 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.14790", 2024-09-10T18:47:59.300 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:47:59.300 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 28482 2024-09-10T18:47:59.300 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.300 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:47:59.300 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.300 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:47:59.300 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.618019+0000", 2024-09-10T18:47:59.301 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.301 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.301 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.301 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:47:59.301 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.618019+0000", 2024-09-10T18:47:59.301 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.301 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.301 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.301 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:47:59.301 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.618018+0000", 2024-09-10T18:47:59.301 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T18:47:59.301 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.301 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.302 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:47:59.302 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.618040+0000", 2024-09-10T18:47:59.302 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.2289999999999998e-05 2024-09-10T18:47:59.302 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.302 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.302 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:47:59.302 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.618051+0000", 2024-09-10T18:47:59.302 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0441999999999999e-05 2024-09-10T18:47:59.302 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.302 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.302 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:47:59.302 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.618056+0000", 2024-09-10T18:47:59.302 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.4489999999999998e-06 2024-09-10T18:47:59.302 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.303 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.303 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:47:59.303 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.618199+0000", 2024-09-10T18:47:59.303 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000142418 2024-09-10T18:47:59.303 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.303 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.303 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:47:59.303 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.618300+0000", 2024-09-10T18:47:59.303 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000101053 2024-09-10T18:47:59.303 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.303 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.303 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,7", 2024-09-10T18:47:59.303 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.618453+0000", 2024-09-10T18:47:59.304 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000153487 2024-09-10T18:47:59.304 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.304 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.304 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:47:59.304 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.739510+0000", 2024-09-10T18:47:59.304 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.12105653600000001 2024-09-10T18:47:59.304 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.304 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.304 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:47:59.304 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.739521+0000", 2024-09-10T18:47:59.304 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0787e-05 2024-09-10T18:47:59.304 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.304 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.305 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:47:59.305 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.740943+0000", 2024-09-10T18:47:59.305 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.001422584 2024-09-10T18:47:59.305 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.305 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.305 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:47:59.305 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.741946+0000", 2024-09-10T18:47:59.305 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0010029920000000001 2024-09-10T18:47:59.305 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.305 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.305 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:47:59.305 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.742000+0000", 2024-09-10T18:47:59.305 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.3674999999999997e-05 2024-09-10T18:47:59.306 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.306 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:47:59.306 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.306 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.306 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.306 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.14790.0:28484 7.4 7:2f0e073d:::benchmark_data_smithi018_78628_object28483:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-10T18:47:59.306 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:47:54.618070+0000", 2024-09-10T18:47:59.306 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 4.5889431140000001, 2024-09-10T18:47:59.306 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.123936936, 2024-09-10T18:47:59.306 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:47:59.306 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:47:59.306 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:47:59.306 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.14790", 2024-09-10T18:47:59.307 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:47:59.307 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 28484 2024-09-10T18:47:59.307 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.307 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:47:59.307 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.307 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:47:59.307 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.618070+0000", 2024-09-10T18:47:59.307 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.307 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.307 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.307 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:47:59.307 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.618070+0000", 2024-09-10T18:47:59.307 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.308 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.308 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.308 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:47:59.308 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.618070+0000", 2024-09-10T18:47:59.308 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T18:47:59.308 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.308 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.308 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:47:59.308 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.618090+0000", 2024-09-10T18:47:59.308 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.0208999999999999e-05 2024-09-10T18:47:59.308 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.308 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.308 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:47:59.309 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.618098+0000", 2024-09-10T18:47:59.309 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.9400000000000002e-06 2024-09-10T18:47:59.309 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.309 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.309 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:47:59.309 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.618102+0000", 2024-09-10T18:47:59.309 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.6360000000000002e-06 2024-09-10T18:47:59.309 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.309 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.309 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:47:59.309 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.618150+0000", 2024-09-10T18:47:59.309 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.7753999999999997e-05 2024-09-10T18:47:59.309 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.310 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.310 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:47:59.310 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.618456+0000", 2024-09-10T18:47:59.310 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00030635800000000001 2024-09-10T18:47:59.310 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.310 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.310 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,4", 2024-09-10T18:47:59.310 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.618586+0000", 2024-09-10T18:47:59.310 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000129519 2024-09-10T18:47:59.310 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.310 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.310 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:47:59.310 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.740379+0000", 2024-09-10T18:47:59.311 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.12179308799999999 2024-09-10T18:47:59.311 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.311 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.311 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:47:59.311 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.740386+0000", 2024-09-10T18:47:59.311 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.5719999999999999e-06 2024-09-10T18:47:59.311 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.311 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.311 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:47:59.311 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.741823+0000", 2024-09-10T18:47:59.311 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.001437266 2024-09-10T18:47:59.311 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.311 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.312 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:47:59.312 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.741964+0000", 2024-09-10T18:47:59.312 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00014138599999999999 2024-09-10T18:47:59.312 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.312 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.312 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:47:59.312 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.742007+0000", 2024-09-10T18:47:59.312 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.2725999999999997e-05 2024-09-10T18:47:59.312 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.312 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:47:59.312 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.312 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.312 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.313 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.14790.0:28486 7.f 7:f3c67647:::benchmark_data_smithi018_78628_object28485:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-10T18:47:59.313 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:47:54.618589+0000", 2024-09-10T18:47:59.313 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 4.5884246229999999, 2024-09-10T18:47:59.313 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.12478149299999999, 2024-09-10T18:47:59.313 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:47:59.313 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:47:59.313 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:47:59.313 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.14790", 2024-09-10T18:47:59.313 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:47:59.313 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 28486 2024-09-10T18:47:59.313 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.313 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:47:59.313 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.314 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:47:59.314 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.618589+0000", 2024-09-10T18:47:59.314 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.314 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.314 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.314 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:47:59.314 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.618589+0000", 2024-09-10T18:47:59.314 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.314 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.314 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.314 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:47:59.314 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.618588+0000", 2024-09-10T18:47:59.314 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:47:59.315 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.315 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.315 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:47:59.315 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.618618+0000", 2024-09-10T18:47:59.315 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.0389999999999999e-05 2024-09-10T18:47:59.315 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.315 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.315 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:47:59.315 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.618629+0000", 2024-09-10T18:47:59.315 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.102e-05 2024-09-10T18:47:59.315 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.315 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.315 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:47:59.316 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.618636+0000", 2024-09-10T18:47:59.316 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.8220000000000001e-06 2024-09-10T18:47:59.316 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.316 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.316 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:47:59.316 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.619793+0000", 2024-09-10T18:47:59.316 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0011574479999999999 2024-09-10T18:47:59.316 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.316 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.316 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:47:59.316 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.619906+0000", 2024-09-10T18:47:59.316 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0001125 2024-09-10T18:47:59.317 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.317 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.317 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,7", 2024-09-10T18:47:59.317 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.620050+0000", 2024-09-10T18:47:59.317 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00014406900000000001 2024-09-10T18:47:59.317 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.317 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.317 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:47:59.317 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.741044+0000", 2024-09-10T18:47:59.317 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.12099391900000001 2024-09-10T18:47:59.317 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.317 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.317 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:47:59.318 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.742641+0000", 2024-09-10T18:47:59.318 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0015967659999999999 2024-09-10T18:47:59.318 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.318 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.318 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:47:59.318 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.742856+0000", 2024-09-10T18:47:59.318 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00021582200000000001 2024-09-10T18:47:59.324 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.324 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.324 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:47:59.324 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.743316+0000", 2024-09-10T18:47:59.324 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00045911599999999997 2024-09-10T18:47:59.324 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.324 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.324 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:47:59.325 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.743370+0000", 2024-09-10T18:47:59.325 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.4437000000000001e-05 2024-09-10T18:47:59.325 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.325 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:47:59.325 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.325 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.325 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.325 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop_reply(client.14790.0:28470 7.6 e95/86 ondisk, result = 0)", 2024-09-10T18:47:59.325 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:47:54.619770+0000", 2024-09-10T18:47:59.325 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 4.5872427399999998, 2024-09-10T18:47:59.325 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.119585753, 2024-09-10T18:47:59.325 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:47:59.325 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "started", 2024-09-10T18:47:59.326 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:47:59.326 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.326 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:47:59.326 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.619770+0000", 2024-09-10T18:47:59.326 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.326 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.326 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.326 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:47:59.326 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.619770+0000", 2024-09-10T18:47:59.326 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.326 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.326 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.326 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:47:59.327 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.619770+0000", 2024-09-10T18:47:59.327 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T18:47:59.327 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.327 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.327 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:47:59.327 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.619775+0000", 2024-09-10T18:47:59.327 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.0170000000000004e-06 2024-09-10T18:47:59.327 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.327 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.327 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:47:59.327 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.619782+0000", 2024-09-10T18:47:59.327 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.7389999999999999e-06 2024-09-10T18:47:59.327 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.328 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.328 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:47:59.328 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.619787+0000", 2024-09-10T18:47:59.328 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.6700000000000002e-06 2024-09-10T18:47:59.328 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.328 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.328 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:47:59.328 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.739164+0000", 2024-09-10T18:47:59.328 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.119376777 2024-09-10T18:47:59.328 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.328 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.328 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:47:59.328 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.739344+0000", 2024-09-10T18:47:59.329 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000179629 2024-09-10T18:47:59.329 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.329 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.329 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:47:59.329 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.739356+0000", 2024-09-10T18:47:59.329 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.2629000000000001e-05 2024-09-10T18:47:59.329 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.329 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:47:59.329 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.329 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.329 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.329 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.14790.0:28485 7.5 e95/88 7:a66d1465:::benchmark_data_smithi018_78628_object28484:head v 95'1728, mlcod=95'1725)", 2024-09-10T18:47:59.329 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:47:54.619795+0000", 2024-09-10T18:47:59.330 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 4.587218118, 2024-09-10T18:47:59.330 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.121402288, 2024-09-10T18:47:59.330 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:47:59.330 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:47:59.330 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:47:59.330 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.330 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:47:59.330 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.619795+0000", 2024-09-10T18:47:59.330 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.330 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.330 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.330 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:47:59.330 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.619795+0000", 2024-09-10T18:47:59.330 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.331 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.331 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.331 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:47:59.331 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.619795+0000", 2024-09-10T18:47:59.331 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T18:47:59.331 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.331 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.331 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:47:59.331 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.619826+0000", 2024-09-10T18:47:59.331 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.1287e-05 2024-09-10T18:47:59.331 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.331 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.331 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:47:59.332 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.619832+0000", 2024-09-10T18:47:59.332 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.8599999999999998e-06 2024-09-10T18:47:59.332 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.332 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.332 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:47:59.332 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.619836+0000", 2024-09-10T18:47:59.332 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.138e-06 2024-09-10T18:47:59.332 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.332 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.332 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:47:59.332 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.620052+0000", 2024-09-10T18:47:59.332 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00021582799999999999 2024-09-10T18:47:59.332 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.333 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.333 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:47:59.333 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.620071+0000", 2024-09-10T18:47:59.333 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.8689e-05 2024-09-10T18:47:59.333 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.333 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.333 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:47:59.333 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.741059+0000", 2024-09-10T18:47:59.333 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.120988331 2024-09-10T18:47:59.333 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.333 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.333 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:47:59.333 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.741197+0000", 2024-09-10T18:47:59.334 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000138443 2024-09-10T18:47:59.334 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.334 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:47:59.334 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.334 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.334 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.334 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop_reply(client.14790.0:28482 7.6 e95/86 ondisk, result = 0)", 2024-09-10T18:47:59.334 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:47:54.620137+0000", 2024-09-10T18:47:59.334 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 4.586876041, 2024-09-10T18:47:59.334 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.119375252, 2024-09-10T18:47:59.334 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:47:59.334 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "started", 2024-09-10T18:47:59.334 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:47:59.335 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.335 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:47:59.335 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.620137+0000", 2024-09-10T18:47:59.335 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.335 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.335 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.335 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:47:59.335 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.620137+0000", 2024-09-10T18:47:59.335 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.335 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.335 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.335 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:47:59.335 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.620137+0000", 2024-09-10T18:47:59.335 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T18:47:59.336 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.336 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.336 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:47:59.336 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.620141+0000", 2024-09-10T18:47:59.336 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.997e-06 2024-09-10T18:47:59.336 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.336 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.336 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:47:59.336 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.739337+0000", 2024-09-10T18:47:59.336 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.11919655699999999 2024-09-10T18:47:59.336 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.336 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.336 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:47:59.337 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.739346+0000", 2024-09-10T18:47:59.337 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.7739999999999993e-06 2024-09-10T18:47:59.337 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.337 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.337 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:47:59.337 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.739498+0000", 2024-09-10T18:47:59.337 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00015240800000000001 2024-09-10T18:47:59.337 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.337 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.337 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:47:59.337 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.739504+0000", 2024-09-10T18:47:59.337 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.1290000000000001e-06 2024-09-10T18:47:59.337 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.337 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.338 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:47:59.338 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.739512+0000", 2024-09-10T18:47:59.338 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.9070000000000007e-06 2024-09-10T18:47:59.338 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.338 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:47:59.338 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.338 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.338 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.338 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.14790.0:28491 7.4 7:24f84507:::benchmark_data_smithi018_78628_object28490:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-10T18:47:59.338 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:47:54.621242+0000", 2024-09-10T18:47:59.338 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 4.5857711669999999, 2024-09-10T18:47:59.338 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.12696676700000001, 2024-09-10T18:47:59.338 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:47:59.339 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:47:59.339 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:47:59.339 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.14790", 2024-09-10T18:47:59.339 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:47:59.339 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 28491 2024-09-10T18:47:59.339 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.339 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:47:59.339 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.339 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:47:59.339 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.621242+0000", 2024-09-10T18:47:59.339 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.339 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.339 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.340 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:47:59.341 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:47:59.341 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.621242+0000", 2024-09-10T18:47:59.341 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.341 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.341 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.341 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:47:59.341 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.621240+0000", 2024-09-10T18:47:59.341 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T18:47:59.342 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.342 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.342 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:47:59.342 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.621265+0000", 2024-09-10T18:47:59.342 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.4698e-05 2024-09-10T18:47:59.342 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.342 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.342 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:47:59.342 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.739001+0000", 2024-09-10T18:47:59.342 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.117735611 2024-09-10T18:47:59.342 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.342 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.342 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:47:59.343 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.739543+0000", 2024-09-10T18:47:59.343 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00054257399999999999 2024-09-10T18:47:59.343 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.343 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.343 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:47:59.343 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.740410+0000", 2024-09-10T18:47:59.343 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00086701300000000003 2024-09-10T18:47:59.343 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.343 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.343 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:47:59.343 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.740539+0000", 2024-09-10T18:47:59.343 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00012899 2024-09-10T18:47:59.343 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.344 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.344 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,4", 2024-09-10T18:47:59.344 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.740684+0000", 2024-09-10T18:47:59.344 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00014425700000000001 2024-09-10T18:47:59.344 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.344 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.344 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:47:59.344 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.747496+0000", 2024-09-10T18:47:59.344 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.006812027 2024-09-10T18:47:59.344 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.344 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.344 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:47:59.344 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.747529+0000", 2024-09-10T18:47:59.345 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.3906999999999999e-05 2024-09-10T18:47:59.345 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.345 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.345 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:47:59.345 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.748127+0000", 2024-09-10T18:47:59.345 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00059702300000000002 2024-09-10T18:47:59.345 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.345 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.345 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:47:59.345 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.748175+0000", 2024-09-10T18:47:59.345 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.8112999999999998e-05 2024-09-10T18:47:59.345 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.345 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.346 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:47:59.346 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.748209+0000", 2024-09-10T18:47:59.346 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.4159000000000003e-05 2024-09-10T18:47:59.346 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.346 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:47:59.346 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.346 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.346 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.346 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.14790.0:28492 7.e e95/88 7:774dea7f:::benchmark_data_smithi018_78628_object28491:head v 95'1743, mlcod=95'1740)", 2024-09-10T18:47:59.346 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:47:54.622160+0000", 2024-09-10T18:47:59.346 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 4.5848533849999997, 2024-09-10T18:47:59.346 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.119881946, 2024-09-10T18:47:59.346 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:47:59.346 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:47:59.347 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:47:59.347 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.347 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:47:59.347 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.622160+0000", 2024-09-10T18:47:59.347 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.347 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.347 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.347 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:47:59.347 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.622160+0000", 2024-09-10T18:47:59.347 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.348 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.348 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.348 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:47:59.348 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.622159+0000", 2024-09-10T18:47:59.348 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:47:59.348 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.348 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.348 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:47:59.348 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.622360+0000", 2024-09-10T18:47:59.348 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000201511 2024-09-10T18:47:59.348 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.348 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.348 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:47:59.349 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.739539+0000", 2024-09-10T18:47:59.349 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.117179109 2024-09-10T18:47:59.349 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.349 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.349 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:47:59.349 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.739544+0000", 2024-09-10T18:47:59.349 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.8040000000000001e-06 2024-09-10T18:47:59.349 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.349 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.349 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:47:59.349 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.739577+0000", 2024-09-10T18:47:59.349 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.2512000000000003e-05 2024-09-10T18:47:59.349 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.350 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.350 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:47:59.350 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.739596+0000", 2024-09-10T18:47:59.350 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.9239e-05 2024-09-10T18:47:59.350 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.350 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.350 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:47:59.350 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.742008+0000", 2024-09-10T18:47:59.350 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.002412226 2024-09-10T18:47:59.350 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.350 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.350 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:47:59.350 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:54.742042+0000", 2024-09-10T18:47:59.351 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.3466000000000001e-05 2024-09-10T18:47:59.351 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.351 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:47:59.351 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.351 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.351 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.351 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.14790.0:31053 7.6 7:60d7fa27:::benchmark_data_smithi018_78628_object31052:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-10T18:47:59.351 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:47:55.988707+0000", 2024-09-10T18:47:59.351 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 3.2183066130000002, 2024-09-10T18:47:59.351 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.10061068300000001, 2024-09-10T18:47:59.351 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:47:59.351 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:47:59.351 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:47:59.351 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.14790", 2024-09-10T18:47:59.352 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:47:59.352 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 31053 2024-09-10T18:47:59.352 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.352 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:47:59.352 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.352 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:47:59.352 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:55.988707+0000", 2024-09-10T18:47:59.352 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.352 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.352 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.352 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:47:59.352 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:55.988707+0000", 2024-09-10T18:47:59.352 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.353 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.353 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.353 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:47:59.353 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:55.988705+0000", 2024-09-10T18:47:59.353 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T18:47:59.353 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.353 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.353 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:47:59.353 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:55.988725+0000", 2024-09-10T18:47:59.353 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.9235000000000001e-05 2024-09-10T18:47:59.353 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.353 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.353 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:47:59.354 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:55.988734+0000", 2024-09-10T18:47:59.354 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.9250000000000007e-06 2024-09-10T18:47:59.354 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.354 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.354 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:47:59.354 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:55.988739+0000", 2024-09-10T18:47:59.354 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.5070000000000002e-06 2024-09-10T18:47:59.354 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.354 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.354 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:47:59.354 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:55.988776+0000", 2024-09-10T18:47:59.354 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.6958000000000001e-05 2024-09-10T18:47:59.354 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.354 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.355 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:47:59.355 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:55.988857+0000", 2024-09-10T18:47:59.355 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.0718999999999997e-05 2024-09-10T18:47:59.355 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.355 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.355 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,7", 2024-09-10T18:47:59.355 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:55.988936+0000", 2024-09-10T18:47:59.355 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.9171000000000005e-05 2024-09-10T18:47:59.355 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.355 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.355 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:47:59.355 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:55.990103+0000", 2024-09-10T18:47:59.355 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0011666999999999999 2024-09-10T18:47:59.355 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.356 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.356 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:47:59.356 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:55.990374+0000", 2024-09-10T18:47:59.356 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00027136100000000002 2024-09-10T18:47:59.356 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.356 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.356 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:47:59.356 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:56.089184+0000", 2024-09-10T18:47:59.356 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.098810296000000006 2024-09-10T18:47:59.356 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.356 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.356 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:47:59.356 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:56.089275+0000", 2024-09-10T18:47:59.357 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.1170999999999998e-05 2024-09-10T18:47:59.357 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.357 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.357 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:47:59.357 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:56.089317+0000", 2024-09-10T18:47:59.357 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.1885999999999998e-05 2024-09-10T18:47:59.357 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.357 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:47:59.357 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.357 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.357 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.357 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.14790.0:31055 7.6 7:64ca532e:::benchmark_data_smithi018_78628_object31054:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-10T18:47:59.357 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:47:55.990989+0000", 2024-09-10T18:47:59.358 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 3.2160239430000002, 2024-09-10T18:47:59.358 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.099314083999999997, 2024-09-10T18:47:59.358 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:47:59.358 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:47:59.358 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:47:59.358 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.14790", 2024-09-10T18:47:59.358 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:47:59.358 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 31055 2024-09-10T18:47:59.358 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.358 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:47:59.358 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.358 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:47:59.358 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:55.990989+0000", 2024-09-10T18:47:59.358 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.359 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.359 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.359 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:47:59.359 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:55.990989+0000", 2024-09-10T18:47:59.359 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:47:59.359 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.359 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.359 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:47:59.359 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:55.990989+0000", 2024-09-10T18:47:59.359 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T18:47:59.359 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.359 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.360 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:47:59.360 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:55.991006+0000", 2024-09-10T18:47:59.360 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.7728999999999999e-05 2024-09-10T18:47:59.360 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.360 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.360 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:47:59.360 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:55.991012+0000", 2024-09-10T18:47:59.360 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.054e-06 2024-09-10T18:47:59.360 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.360 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.360 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:47:59.360 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:55.991016+0000", 2024-09-10T18:47:59.360 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.1760000000000002e-06 2024-09-10T18:47:59.361 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.361 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.361 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:47:59.361 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:55.991049+0000", 2024-09-10T18:47:59.361 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.3420999999999997e-05 2024-09-10T18:47:59.361 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.361 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.361 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:47:59.361 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:55.991152+0000", 2024-09-10T18:47:59.361 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000102602 2024-09-10T18:47:59.361 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.361 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.361 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,7", 2024-09-10T18:47:59.362 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:55.991264+0000", 2024-09-10T18:47:59.362 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000112574 2024-09-10T18:47:59.362 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.362 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.362 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:47:59.362 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:56.078388+0000", 2024-09-10T18:47:59.362 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.087123682999999993 2024-09-10T18:47:59.362 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.362 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.362 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:47:59.362 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:56.079651+0000", 2024-09-10T18:47:59.362 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0012631719999999999 2024-09-10T18:47:59.362 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.363 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.363 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:47:59.363 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:56.090178+0000", 2024-09-10T18:47:59.363 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.010526584 2024-09-10T18:47:59.363 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.363 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.363 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:47:59.363 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:56.090263+0000", 2024-09-10T18:47:59.363 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.5746000000000002e-05 2024-09-10T18:47:59.363 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:47:59.363 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:47:59.363 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:47:59.363 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:56.090303+0000", 2024-09-10T18:47:59.363 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.0021999999999997e-05 2024-09-10T18:47:59.364 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.364 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:47:59.364 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.364 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:47:59.364 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:47:59.364 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T18:47:59.370 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:59.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:59.372 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:47:59.472 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:47:59.513 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:59.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:59.516 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:47:59.617 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:47:59.649 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:47:59.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:59.650 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:47:59.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:59 smithi018 ceph-mon[40196]: osdmap e98: 8 total, 8 up, 7 in 2024-09-10T18:47:59.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:47:59 smithi018 ceph-mon[40196]: pgmap v93: 148 pgs: 148 active+clean; 1.9 GiB data, 7.6 GiB used, 618 GiB / 626 GiB avail; 921 B/s rd, 126 MiB/s wr, 2.01k op/s 2024-09-10T18:47:59.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:59 smithi018 ceph-mon[29336]: osdmap e98: 8 total, 8 up, 7 in 2024-09-10T18:47:59.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:47:59 smithi018 ceph-mon[29336]: pgmap v93: 148 pgs: 148 active+clean; 1.9 GiB data, 7.6 GiB used, 618 GiB / 626 GiB avail; 921 B/s rd, 126 MiB/s wr, 2.01k op/s 2024-09-10T18:47:59.751 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:47:59.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:59 smithi103 ceph-mon[33163]: osdmap e98: 8 total, 8 up, 7 in 2024-09-10T18:47:59.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:47:59 smithi103 ceph-mon[33163]: pgmap v93: 148 pgs: 148 active+clean; 1.9 GiB data, 7.6 GiB used, 618 GiB / 626 GiB avail; 921 B/s rd, 126 MiB/s wr, 2.01k op/s 2024-09-10T18:47:59.794 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:59.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:59.795 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:47:59.896 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:47:59.935 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:47:59.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:47:59.936 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:00.037 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:00.081 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:00.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:00.082 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:00.183 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:00.201 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 16 16 39492 39476 154.163 122.625 0.00868043 0.00648185 2024-09-10T18:48:00.213 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:00.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:00.214 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:00.315 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:00.347 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:00.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:00.349 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:00.450 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:00.477 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:00.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:00.478 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:00.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:48:00 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:48:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:48:00.580 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:00.611 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:00.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:00.613 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:00.713 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:00.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:00.747 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:00.748 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:00.849 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:00.895 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:00.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:00.896 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:00.996 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:01.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:01.029 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:01.029 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:01.130 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:01.160 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:01.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:01.162 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:01.201 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 17 16 41534 41518 152.601 127.625 0.0120053 0.00654798 2024-09-10T18:48:01.262 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:01.293 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:01.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:01.294 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:01.303 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:48:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:48:00] "GET /metrics HTTP/1.1" 200 223250 "" "Prometheus/2.43.0" 2024-09-10T18:48:01.357 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-10T18:48:01.394 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:01.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:01.449 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:01.449 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:01.550 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:01.568 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:48:01.575 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:01.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:01.577 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:01.677 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:01.704 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:01.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:01.705 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:01.806 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:01.853 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:01.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:01.855 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:01.940 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:48:01.942 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T18:48:01.942 INFO:teuthology.orchestra.run.smithi018.stdout: "epoch": 98, 2024-09-10T18:48:01.942 INFO:teuthology.orchestra.run.smithi018.stdout: "fsid": "ba327f6c-6fa3-11ef-bcea-c7b262605968", 2024-09-10T18:48:01.942 INFO:teuthology.orchestra.run.smithi018.stdout: "created": "2024-09-10T18:40:07.669687+0000", 2024-09-10T18:48:01.942 INFO:teuthology.orchestra.run.smithi018.stdout: "modified": "2024-09-10T18:47:58.485307+0000", 2024-09-10T18:48:01.942 INFO:teuthology.orchestra.run.smithi018.stdout: "last_up_change": "2024-09-10T18:45:05.425833+0000", 2024-09-10T18:48:01.942 INFO:teuthology.orchestra.run.smithi018.stdout: "last_in_change": "2024-09-10T18:47:35.385915+0000", 2024-09-10T18:48:01.943 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-10T18:48:01.943 INFO:teuthology.orchestra.run.smithi018.stdout: "flags_num": 5806080, 2024-09-10T18:48:01.943 INFO:teuthology.orchestra.run.smithi018.stdout: "flags_set": [ 2024-09-10T18:48:01.943 INFO:teuthology.orchestra.run.smithi018.stdout: "nodeep-scrub", 2024-09-10T18:48:01.943 INFO:teuthology.orchestra.run.smithi018.stdout: "noscrub", 2024-09-10T18:48:01.943 INFO:teuthology.orchestra.run.smithi018.stdout: "pglog_hardlimit", 2024-09-10T18:48:01.943 INFO:teuthology.orchestra.run.smithi018.stdout: "purged_snapdirs", 2024-09-10T18:48:01.943 INFO:teuthology.orchestra.run.smithi018.stdout: "recovery_deletes", 2024-09-10T18:48:01.943 INFO:teuthology.orchestra.run.smithi018.stdout: "sortbitwise" 2024-09-10T18:48:01.943 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-09-10T18:48:01.943 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_version": 18, 2024-09-10T18:48:01.943 INFO:teuthology.orchestra.run.smithi018.stdout: "full_ratio": 0.94999998807907104, 2024-09-10T18:48:01.943 INFO:teuthology.orchestra.run.smithi018.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-10T18:48:01.944 INFO:teuthology.orchestra.run.smithi018.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-10T18:48:01.944 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_snapshot": "", 2024-09-10T18:48:01.944 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_max": 7, 2024-09-10T18:48:01.944 INFO:teuthology.orchestra.run.smithi018.stdout: "max_osd": 8, 2024-09-10T18:48:01.944 INFO:teuthology.orchestra.run.smithi018.stdout: "require_min_compat_client": "luminous", 2024-09-10T18:48:01.944 INFO:teuthology.orchestra.run.smithi018.stdout: "min_compat_client": "luminous", 2024-09-10T18:48:01.944 INFO:teuthology.orchestra.run.smithi018.stdout: "require_osd_release": "quincy", 2024-09-10T18:48:01.944 INFO:teuthology.orchestra.run.smithi018.stdout: "pools": [ 2024-09-10T18:48:01.944 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:01.944 INFO:teuthology.orchestra.run.smithi018.stdout: "pool": 1, 2024-09-10T18:48:01.944 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_name": ".mgr", 2024-09-10T18:48:01.944 INFO:teuthology.orchestra.run.smithi018.stdout: "create_time": "2024-09-10T18:43:08.162659+0000", 2024-09-10T18:48:01.945 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-09-10T18:48:01.945 INFO:teuthology.orchestra.run.smithi018.stdout: "flags_names": "hashpspool", 2024-09-10T18:48:01.945 INFO:teuthology.orchestra.run.smithi018.stdout: "type": 1, 2024-09-10T18:48:01.945 INFO:teuthology.orchestra.run.smithi018.stdout: "size": 3, 2024-09-10T18:48:01.945 INFO:teuthology.orchestra.run.smithi018.stdout: "min_size": 2, 2024-09-10T18:48:01.945 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_rule": 0, 2024-09-10T18:48:01.945 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_count": 0, 2024-09-10T18:48:01.945 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_target": 0, 2024-09-10T18:48:01.945 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T18:48:01.945 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T18:48:01.945 INFO:teuthology.orchestra.run.smithi018.stdout: "object_hash": 2, 2024-09-10T18:48:01.945 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscale_mode": "off", 2024-09-10T18:48:01.946 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num": 1, 2024-09-10T18:48:01.946 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num": 1, 2024-09-10T18:48:01.946 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num_target": 1, 2024-09-10T18:48:01.946 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_target": 1, 2024-09-10T18:48:01.946 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_pending": 1, 2024-09-10T18:48:01.946 INFO:teuthology.orchestra.run.smithi018.stdout: "last_pg_merge_meta": { 2024-09-10T18:48:01.946 INFO:teuthology.orchestra.run.smithi018.stdout: "source_pgid": "0.0", 2024-09-10T18:48:01.946 INFO:teuthology.orchestra.run.smithi018.stdout: "ready_epoch": 0, 2024-09-10T18:48:01.946 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_started": 0, 2024-09-10T18:48:01.946 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_clean": 0, 2024-09-10T18:48:01.946 INFO:teuthology.orchestra.run.smithi018.stdout: "source_version": "0'0", 2024-09-10T18:48:01.946 INFO:teuthology.orchestra.run.smithi018.stdout: "target_version": "0'0" 2024-09-10T18:48:01.946 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:01.947 INFO:teuthology.orchestra.run.smithi018.stdout: "last_change": "21", 2024-09-10T18:48:01.947 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend": "0", 2024-09-10T18:48:01.947 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T18:48:01.947 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T18:48:01.947 INFO:teuthology.orchestra.run.smithi018.stdout: "auid": 0, 2024-09-10T18:48:01.947 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_mode": "selfmanaged", 2024-09-10T18:48:01.947 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_seq": 0, 2024-09-10T18:48:01.947 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_epoch": 0, 2024-09-10T18:48:01.947 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_snaps": [], 2024-09-10T18:48:01.947 INFO:teuthology.orchestra.run.smithi018.stdout: "removed_snaps": "[]", 2024-09-10T18:48:01.947 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_bytes": 0, 2024-09-10T18:48:01.947 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_objects": 0, 2024-09-10T18:48:01.947 INFO:teuthology.orchestra.run.smithi018.stdout: "tiers": [], 2024-09-10T18:48:01.947 INFO:teuthology.orchestra.run.smithi018.stdout: "tier_of": -1, 2024-09-10T18:48:01.948 INFO:teuthology.orchestra.run.smithi018.stdout: "read_tier": -1, 2024-09-10T18:48:01.948 INFO:teuthology.orchestra.run.smithi018.stdout: "write_tier": -1, 2024-09-10T18:48:01.948 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_mode": "none", 2024-09-10T18:48:01.948 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_bytes": 0, 2024-09-10T18:48:01.948 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_objects": 0, 2024-09-10T18:48:01.948 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T18:48:01.948 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T18:48:01.948 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T18:48:01.948 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_flush_age": 0, 2024-09-10T18:48:01.948 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_evict_age": 0, 2024-09-10T18:48:01.948 INFO:teuthology.orchestra.run.smithi018.stdout: "erasure_code_profile": "", 2024-09-10T18:48:01.948 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_params": { 2024-09-10T18:48:01.949 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "none" 2024-09-10T18:48:01.949 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:01.949 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_period": 0, 2024-09-10T18:48:01.949 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_count": 0, 2024-09-10T18:48:01.949 INFO:teuthology.orchestra.run.smithi018.stdout: "use_gmt_hitset": true, 2024-09-10T18:48:01.949 INFO:teuthology.orchestra.run.smithi018.stdout: "min_read_recency_for_promote": 0, 2024-09-10T18:48:01.949 INFO:teuthology.orchestra.run.smithi018.stdout: "min_write_recency_for_promote": 0, 2024-09-10T18:48:01.949 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T18:48:01.949 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_search_last_n": 0, 2024-09-10T18:48:01.949 INFO:teuthology.orchestra.run.smithi018.stdout: "grade_table": [], 2024-09-10T18:48:01.949 INFO:teuthology.orchestra.run.smithi018.stdout: "stripe_width": 0, 2024-09-10T18:48:01.949 INFO:teuthology.orchestra.run.smithi018.stdout: "expected_num_objects": 0, 2024-09-10T18:48:01.949 INFO:teuthology.orchestra.run.smithi018.stdout: "fast_read": false, 2024-09-10T18:48:01.950 INFO:teuthology.orchestra.run.smithi018.stdout: "options": { 2024-09-10T18:48:01.950 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_max": 32, 2024-09-10T18:48:01.950 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_min": 1 2024-09-10T18:48:01.950 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:01.950 INFO:teuthology.orchestra.run.smithi018.stdout: "application_metadata": { 2024-09-10T18:48:01.950 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": {} 2024-09-10T18:48:01.950 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:01.950 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:01.950 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:01.950 INFO:teuthology.orchestra.run.smithi018.stdout: "pool": 2, 2024-09-10T18:48:01.950 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_name": ".rgw.root", 2024-09-10T18:48:01.950 INFO:teuthology.orchestra.run.smithi018.stdout: "create_time": "2024-09-10T18:45:16.512219+0000", 2024-09-10T18:48:01.950 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-09-10T18:48:01.951 INFO:teuthology.orchestra.run.smithi018.stdout: "flags_names": "hashpspool", 2024-09-10T18:48:01.951 INFO:teuthology.orchestra.run.smithi018.stdout: "type": 1, 2024-09-10T18:48:01.951 INFO:teuthology.orchestra.run.smithi018.stdout: "size": 3, 2024-09-10T18:48:01.951 INFO:teuthology.orchestra.run.smithi018.stdout: "min_size": 2, 2024-09-10T18:48:01.951 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_rule": 0, 2024-09-10T18:48:01.951 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_count": 0, 2024-09-10T18:48:01.951 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_target": 0, 2024-09-10T18:48:01.951 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T18:48:01.951 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T18:48:01.951 INFO:teuthology.orchestra.run.smithi018.stdout: "object_hash": 2, 2024-09-10T18:48:01.951 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscale_mode": "off", 2024-09-10T18:48:01.951 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num": 32, 2024-09-10T18:48:01.951 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num": 32, 2024-09-10T18:48:01.952 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num_target": 32, 2024-09-10T18:48:01.952 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_target": 32, 2024-09-10T18:48:01.952 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_pending": 32, 2024-09-10T18:48:01.952 INFO:teuthology.orchestra.run.smithi018.stdout: "last_pg_merge_meta": { 2024-09-10T18:48:01.952 INFO:teuthology.orchestra.run.smithi018.stdout: "source_pgid": "0.0", 2024-09-10T18:48:01.952 INFO:teuthology.orchestra.run.smithi018.stdout: "ready_epoch": 0, 2024-09-10T18:48:01.952 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_started": 0, 2024-09-10T18:48:01.952 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_clean": 0, 2024-09-10T18:48:01.952 INFO:teuthology.orchestra.run.smithi018.stdout: "source_version": "0'0", 2024-09-10T18:48:01.952 INFO:teuthology.orchestra.run.smithi018.stdout: "target_version": "0'0" 2024-09-10T18:48:01.952 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:01.952 INFO:teuthology.orchestra.run.smithi018.stdout: "last_change": "57", 2024-09-10T18:48:01.953 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend": "0", 2024-09-10T18:48:01.953 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T18:48:01.953 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T18:48:01.953 INFO:teuthology.orchestra.run.smithi018.stdout: "auid": 0, 2024-09-10T18:48:01.953 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_mode": "selfmanaged", 2024-09-10T18:48:01.953 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_seq": 0, 2024-09-10T18:48:01.953 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_epoch": 0, 2024-09-10T18:48:01.953 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_snaps": [], 2024-09-10T18:48:01.953 INFO:teuthology.orchestra.run.smithi018.stdout: "removed_snaps": "[]", 2024-09-10T18:48:01.953 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_bytes": 0, 2024-09-10T18:48:01.953 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_objects": 0, 2024-09-10T18:48:01.953 INFO:teuthology.orchestra.run.smithi018.stdout: "tiers": [], 2024-09-10T18:48:01.953 INFO:teuthology.orchestra.run.smithi018.stdout: "tier_of": -1, 2024-09-10T18:48:01.954 INFO:teuthology.orchestra.run.smithi018.stdout: "read_tier": -1, 2024-09-10T18:48:01.954 INFO:teuthology.orchestra.run.smithi018.stdout: "write_tier": -1, 2024-09-10T18:48:01.954 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_mode": "none", 2024-09-10T18:48:01.954 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_bytes": 0, 2024-09-10T18:48:01.954 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_objects": 0, 2024-09-10T18:48:01.954 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T18:48:01.954 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T18:48:01.954 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T18:48:01.954 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_flush_age": 0, 2024-09-10T18:48:01.954 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_evict_age": 0, 2024-09-10T18:48:01.954 INFO:teuthology.orchestra.run.smithi018.stdout: "erasure_code_profile": "", 2024-09-10T18:48:01.955 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_params": { 2024-09-10T18:48:01.955 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "none" 2024-09-10T18:48:01.955 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:01.955 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_period": 0, 2024-09-10T18:48:01.955 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_count": 0, 2024-09-10T18:48:01.955 INFO:teuthology.orchestra.run.smithi018.stdout: "use_gmt_hitset": true, 2024-09-10T18:48:01.955 INFO:teuthology.orchestra.run.smithi018.stdout: "min_read_recency_for_promote": 0, 2024-09-10T18:48:01.955 INFO:teuthology.orchestra.run.smithi018.stdout: "min_write_recency_for_promote": 0, 2024-09-10T18:48:01.955 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T18:48:01.955 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_search_last_n": 0, 2024-09-10T18:48:01.955 INFO:teuthology.orchestra.run.smithi018.stdout: "grade_table": [], 2024-09-10T18:48:01.955 INFO:teuthology.orchestra.run.smithi018.stdout: "stripe_width": 0, 2024-09-10T18:48:01.956 INFO:teuthology.orchestra.run.smithi018.stdout: "expected_num_objects": 0, 2024-09-10T18:48:01.956 INFO:teuthology.orchestra.run.smithi018.stdout: "fast_read": false, 2024-09-10T18:48:01.956 INFO:teuthology.orchestra.run.smithi018.stdout: "options": {}, 2024-09-10T18:48:01.956 INFO:teuthology.orchestra.run.smithi018.stdout: "application_metadata": { 2024-09-10T18:48:01.956 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw": {} 2024-09-10T18:48:01.956 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:01.956 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:01.956 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:01.956 INFO:teuthology.orchestra.run.smithi018.stdout: "pool": 3, 2024-09-10T18:48:01.956 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_name": "datapool", 2024-09-10T18:48:01.956 INFO:teuthology.orchestra.run.smithi018.stdout: "create_time": "2024-09-10T18:45:16.943626+0000", 2024-09-10T18:48:01.957 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 8193, 2024-09-10T18:48:01.957 INFO:teuthology.orchestra.run.smithi018.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-10T18:48:01.957 INFO:teuthology.orchestra.run.smithi018.stdout: "type": 1, 2024-09-10T18:48:01.957 INFO:teuthology.orchestra.run.smithi018.stdout: "size": 3, 2024-09-10T18:48:01.957 INFO:teuthology.orchestra.run.smithi018.stdout: "min_size": 2, 2024-09-10T18:48:01.957 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_rule": 0, 2024-09-10T18:48:01.957 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_count": 0, 2024-09-10T18:48:01.957 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_target": 0, 2024-09-10T18:48:01.957 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T18:48:01.957 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T18:48:01.957 INFO:teuthology.orchestra.run.smithi018.stdout: "object_hash": 2, 2024-09-10T18:48:01.957 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscale_mode": "off", 2024-09-10T18:48:01.957 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num": 3, 2024-09-10T18:48:01.957 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num": 3, 2024-09-10T18:48:01.958 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num_target": 3, 2024-09-10T18:48:01.958 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_target": 3, 2024-09-10T18:48:01.958 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_pending": 3, 2024-09-10T18:48:01.958 INFO:teuthology.orchestra.run.smithi018.stdout: "last_pg_merge_meta": { 2024-09-10T18:48:01.958 INFO:teuthology.orchestra.run.smithi018.stdout: "source_pgid": "0.0", 2024-09-10T18:48:01.958 INFO:teuthology.orchestra.run.smithi018.stdout: "ready_epoch": 0, 2024-09-10T18:48:01.958 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_started": 0, 2024-09-10T18:48:01.958 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_clean": 0, 2024-09-10T18:48:01.958 INFO:teuthology.orchestra.run.smithi018.stdout: "source_version": "0'0", 2024-09-10T18:48:01.958 INFO:teuthology.orchestra.run.smithi018.stdout: "target_version": "0'0" 2024-09-10T18:48:01.958 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:01.958 INFO:teuthology.orchestra.run.smithi018.stdout: "last_change": "60", 2024-09-10T18:48:01.959 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend": "0", 2024-09-10T18:48:01.959 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T18:48:01.959 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T18:48:01.959 INFO:teuthology.orchestra.run.smithi018.stdout: "auid": 0, 2024-09-10T18:48:01.959 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_mode": "selfmanaged", 2024-09-10T18:48:01.959 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_seq": 3, 2024-09-10T18:48:01.959 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_epoch": 60, 2024-09-10T18:48:01.959 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_snaps": [], 2024-09-10T18:48:01.959 INFO:teuthology.orchestra.run.smithi018.stdout: "removed_snaps": "[]", 2024-09-10T18:48:01.959 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_bytes": 0, 2024-09-10T18:48:01.959 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_objects": 0, 2024-09-10T18:48:01.959 INFO:teuthology.orchestra.run.smithi018.stdout: "tiers": [], 2024-09-10T18:48:01.959 INFO:teuthology.orchestra.run.smithi018.stdout: "tier_of": -1, 2024-09-10T18:48:01.959 INFO:teuthology.orchestra.run.smithi018.stdout: "read_tier": -1, 2024-09-10T18:48:01.960 INFO:teuthology.orchestra.run.smithi018.stdout: "write_tier": -1, 2024-09-10T18:48:01.960 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_mode": "none", 2024-09-10T18:48:01.960 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_bytes": 0, 2024-09-10T18:48:01.960 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_objects": 0, 2024-09-10T18:48:01.960 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T18:48:01.960 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T18:48:01.960 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T18:48:01.960 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_flush_age": 0, 2024-09-10T18:48:01.960 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_evict_age": 0, 2024-09-10T18:48:01.960 INFO:teuthology.orchestra.run.smithi018.stdout: "erasure_code_profile": "", 2024-09-10T18:48:01.960 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_params": { 2024-09-10T18:48:01.960 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "none" 2024-09-10T18:48:01.960 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:01.961 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_period": 0, 2024-09-10T18:48:01.961 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_count": 0, 2024-09-10T18:48:01.961 INFO:teuthology.orchestra.run.smithi018.stdout: "use_gmt_hitset": true, 2024-09-10T18:48:01.961 INFO:teuthology.orchestra.run.smithi018.stdout: "min_read_recency_for_promote": 0, 2024-09-10T18:48:01.961 INFO:teuthology.orchestra.run.smithi018.stdout: "min_write_recency_for_promote": 0, 2024-09-10T18:48:01.961 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T18:48:01.961 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_search_last_n": 0, 2024-09-10T18:48:01.961 INFO:teuthology.orchestra.run.smithi018.stdout: "grade_table": [], 2024-09-10T18:48:01.961 INFO:teuthology.orchestra.run.smithi018.stdout: "stripe_width": 0, 2024-09-10T18:48:01.961 INFO:teuthology.orchestra.run.smithi018.stdout: "expected_num_objects": 0, 2024-09-10T18:48:01.961 INFO:teuthology.orchestra.run.smithi018.stdout: "fast_read": false, 2024-09-10T18:48:01.961 INFO:teuthology.orchestra.run.smithi018.stdout: "options": {}, 2024-09-10T18:48:01.961 INFO:teuthology.orchestra.run.smithi018.stdout: "application_metadata": { 2024-09-10T18:48:01.962 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd": {} 2024-09-10T18:48:01.962 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:01.962 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:01.962 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:01.962 INFO:teuthology.orchestra.run.smithi018.stdout: "pool": 4, 2024-09-10T18:48:01.962 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_name": "default.rgw.log", 2024-09-10T18:48:01.962 INFO:teuthology.orchestra.run.smithi018.stdout: "create_time": "2024-09-10T18:45:18.450026+0000", 2024-09-10T18:48:01.962 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-09-10T18:48:01.962 INFO:teuthology.orchestra.run.smithi018.stdout: "flags_names": "hashpspool", 2024-09-10T18:48:01.962 INFO:teuthology.orchestra.run.smithi018.stdout: "type": 1, 2024-09-10T18:48:01.962 INFO:teuthology.orchestra.run.smithi018.stdout: "size": 3, 2024-09-10T18:48:01.962 INFO:teuthology.orchestra.run.smithi018.stdout: "min_size": 2, 2024-09-10T18:48:01.962 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_rule": 0, 2024-09-10T18:48:01.963 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_count": 0, 2024-09-10T18:48:01.963 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_target": 0, 2024-09-10T18:48:01.963 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T18:48:01.963 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T18:48:01.963 INFO:teuthology.orchestra.run.smithi018.stdout: "object_hash": 2, 2024-09-10T18:48:01.963 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscale_mode": "off", 2024-09-10T18:48:01.963 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num": 32, 2024-09-10T18:48:01.963 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num": 32, 2024-09-10T18:48:01.963 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num_target": 32, 2024-09-10T18:48:01.963 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_target": 32, 2024-09-10T18:48:01.963 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_pending": 32, 2024-09-10T18:48:01.963 INFO:teuthology.orchestra.run.smithi018.stdout: "last_pg_merge_meta": { 2024-09-10T18:48:01.963 INFO:teuthology.orchestra.run.smithi018.stdout: "source_pgid": "0.0", 2024-09-10T18:48:01.963 INFO:teuthology.orchestra.run.smithi018.stdout: "ready_epoch": 0, 2024-09-10T18:48:01.964 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_started": 0, 2024-09-10T18:48:01.964 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_clean": 0, 2024-09-10T18:48:01.964 INFO:teuthology.orchestra.run.smithi018.stdout: "source_version": "0'0", 2024-09-10T18:48:01.964 INFO:teuthology.orchestra.run.smithi018.stdout: "target_version": "0'0" 2024-09-10T18:48:01.964 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:01.964 INFO:teuthology.orchestra.run.smithi018.stdout: "last_change": "59", 2024-09-10T18:48:01.964 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend": "0", 2024-09-10T18:48:01.964 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T18:48:01.964 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T18:48:01.964 INFO:teuthology.orchestra.run.smithi018.stdout: "auid": 0, 2024-09-10T18:48:01.964 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_mode": "selfmanaged", 2024-09-10T18:48:01.964 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_seq": 0, 2024-09-10T18:48:01.964 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_epoch": 0, 2024-09-10T18:48:01.965 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_snaps": [], 2024-09-10T18:48:01.965 INFO:teuthology.orchestra.run.smithi018.stdout: "removed_snaps": "[]", 2024-09-10T18:48:01.965 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_bytes": 0, 2024-09-10T18:48:01.965 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_objects": 0, 2024-09-10T18:48:01.965 INFO:teuthology.orchestra.run.smithi018.stdout: "tiers": [], 2024-09-10T18:48:01.965 INFO:teuthology.orchestra.run.smithi018.stdout: "tier_of": -1, 2024-09-10T18:48:01.965 INFO:teuthology.orchestra.run.smithi018.stdout: "read_tier": -1, 2024-09-10T18:48:01.965 INFO:teuthology.orchestra.run.smithi018.stdout: "write_tier": -1, 2024-09-10T18:48:01.965 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_mode": "none", 2024-09-10T18:48:01.965 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_bytes": 0, 2024-09-10T18:48:01.965 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_objects": 0, 2024-09-10T18:48:01.965 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T18:48:01.965 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T18:48:01.966 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T18:48:01.966 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_flush_age": 0, 2024-09-10T18:48:01.966 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_evict_age": 0, 2024-09-10T18:48:01.966 INFO:teuthology.orchestra.run.smithi018.stdout: "erasure_code_profile": "", 2024-09-10T18:48:01.966 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_params": { 2024-09-10T18:48:01.966 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "none" 2024-09-10T18:48:01.966 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:01.966 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_period": 0, 2024-09-10T18:48:01.966 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_count": 0, 2024-09-10T18:48:01.966 INFO:teuthology.orchestra.run.smithi018.stdout: "use_gmt_hitset": true, 2024-09-10T18:48:01.966 INFO:teuthology.orchestra.run.smithi018.stdout: "min_read_recency_for_promote": 0, 2024-09-10T18:48:01.966 INFO:teuthology.orchestra.run.smithi018.stdout: "min_write_recency_for_promote": 0, 2024-09-10T18:48:01.966 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T18:48:01.967 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_search_last_n": 0, 2024-09-10T18:48:01.967 INFO:teuthology.orchestra.run.smithi018.stdout: "grade_table": [], 2024-09-10T18:48:01.967 INFO:teuthology.orchestra.run.smithi018.stdout: "stripe_width": 0, 2024-09-10T18:48:01.967 INFO:teuthology.orchestra.run.smithi018.stdout: "expected_num_objects": 0, 2024-09-10T18:48:01.967 INFO:teuthology.orchestra.run.smithi018.stdout: "fast_read": false, 2024-09-10T18:48:01.967 INFO:teuthology.orchestra.run.smithi018.stdout: "options": {}, 2024-09-10T18:48:01.967 INFO:teuthology.orchestra.run.smithi018.stdout: "application_metadata": { 2024-09-10T18:48:01.967 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw": {} 2024-09-10T18:48:01.967 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:01.967 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:01.967 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:01.967 INFO:teuthology.orchestra.run.smithi018.stdout: "pool": 5, 2024-09-10T18:48:01.967 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_name": "default.rgw.control", 2024-09-10T18:48:01.968 INFO:teuthology.orchestra.run.smithi018.stdout: "create_time": "2024-09-10T18:45:20.494618+0000", 2024-09-10T18:48:01.968 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-09-10T18:48:01.968 INFO:teuthology.orchestra.run.smithi018.stdout: "flags_names": "hashpspool", 2024-09-10T18:48:01.968 INFO:teuthology.orchestra.run.smithi018.stdout: "type": 1, 2024-09-10T18:48:01.968 INFO:teuthology.orchestra.run.smithi018.stdout: "size": 3, 2024-09-10T18:48:01.968 INFO:teuthology.orchestra.run.smithi018.stdout: "min_size": 2, 2024-09-10T18:48:01.968 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_rule": 0, 2024-09-10T18:48:01.968 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_count": 0, 2024-09-10T18:48:01.968 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_target": 0, 2024-09-10T18:48:01.968 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T18:48:01.968 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T18:48:01.968 INFO:teuthology.orchestra.run.smithi018.stdout: "object_hash": 2, 2024-09-10T18:48:01.968 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscale_mode": "off", 2024-09-10T18:48:01.969 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num": 32, 2024-09-10T18:48:01.969 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num": 32, 2024-09-10T18:48:01.969 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num_target": 32, 2024-09-10T18:48:01.969 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_target": 32, 2024-09-10T18:48:01.969 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_pending": 32, 2024-09-10T18:48:01.969 INFO:teuthology.orchestra.run.smithi018.stdout: "last_pg_merge_meta": { 2024-09-10T18:48:01.969 INFO:teuthology.orchestra.run.smithi018.stdout: "source_pgid": "0.0", 2024-09-10T18:48:01.969 INFO:teuthology.orchestra.run.smithi018.stdout: "ready_epoch": 0, 2024-09-10T18:48:01.969 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_started": 0, 2024-09-10T18:48:01.969 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_clean": 0, 2024-09-10T18:48:01.969 INFO:teuthology.orchestra.run.smithi018.stdout: "source_version": "0'0", 2024-09-10T18:48:01.969 INFO:teuthology.orchestra.run.smithi018.stdout: "target_version": "0'0" 2024-09-10T18:48:01.969 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:01.970 INFO:teuthology.orchestra.run.smithi018.stdout: "last_change": "61", 2024-09-10T18:48:01.970 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend": "0", 2024-09-10T18:48:01.970 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T18:48:01.970 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T18:48:01.970 INFO:teuthology.orchestra.run.smithi018.stdout: "auid": 0, 2024-09-10T18:48:01.970 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_mode": "selfmanaged", 2024-09-10T18:48:01.970 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_seq": 0, 2024-09-10T18:48:01.970 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_epoch": 0, 2024-09-10T18:48:01.970 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_snaps": [], 2024-09-10T18:48:01.970 INFO:teuthology.orchestra.run.smithi018.stdout: "removed_snaps": "[]", 2024-09-10T18:48:01.970 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_bytes": 0, 2024-09-10T18:48:01.970 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_objects": 0, 2024-09-10T18:48:01.970 INFO:teuthology.orchestra.run.smithi018.stdout: "tiers": [], 2024-09-10T18:48:01.971 INFO:teuthology.orchestra.run.smithi018.stdout: "tier_of": -1, 2024-09-10T18:48:01.971 INFO:teuthology.orchestra.run.smithi018.stdout: "read_tier": -1, 2024-09-10T18:48:01.971 INFO:teuthology.orchestra.run.smithi018.stdout: "write_tier": -1, 2024-09-10T18:48:01.971 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_mode": "none", 2024-09-10T18:48:01.971 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_bytes": 0, 2024-09-10T18:48:01.971 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_objects": 0, 2024-09-10T18:48:01.971 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T18:48:01.971 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T18:48:01.971 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T18:48:01.971 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_flush_age": 0, 2024-09-10T18:48:01.971 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_evict_age": 0, 2024-09-10T18:48:01.971 INFO:teuthology.orchestra.run.smithi018.stdout: "erasure_code_profile": "", 2024-09-10T18:48:01.971 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_params": { 2024-09-10T18:48:01.971 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "none" 2024-09-10T18:48:01.972 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:01.972 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_period": 0, 2024-09-10T18:48:01.972 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_count": 0, 2024-09-10T18:48:01.972 INFO:teuthology.orchestra.run.smithi018.stdout: "use_gmt_hitset": true, 2024-09-10T18:48:01.972 INFO:teuthology.orchestra.run.smithi018.stdout: "min_read_recency_for_promote": 0, 2024-09-10T18:48:01.972 INFO:teuthology.orchestra.run.smithi018.stdout: "min_write_recency_for_promote": 0, 2024-09-10T18:48:01.972 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T18:48:01.972 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_search_last_n": 0, 2024-09-10T18:48:01.972 INFO:teuthology.orchestra.run.smithi018.stdout: "grade_table": [], 2024-09-10T18:48:01.972 INFO:teuthology.orchestra.run.smithi018.stdout: "stripe_width": 0, 2024-09-10T18:48:01.972 INFO:teuthology.orchestra.run.smithi018.stdout: "expected_num_objects": 0, 2024-09-10T18:48:01.972 INFO:teuthology.orchestra.run.smithi018.stdout: "fast_read": false, 2024-09-10T18:48:01.973 INFO:teuthology.orchestra.run.smithi018.stdout: "options": {}, 2024-09-10T18:48:01.973 INFO:teuthology.orchestra.run.smithi018.stdout: "application_metadata": { 2024-09-10T18:48:01.973 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw": {} 2024-09-10T18:48:01.973 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:01.973 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:01.973 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:01.973 INFO:teuthology.orchestra.run.smithi018.stdout: "pool": 6, 2024-09-10T18:48:01.973 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_name": "default.rgw.meta", 2024-09-10T18:48:01.973 INFO:teuthology.orchestra.run.smithi018.stdout: "create_time": "2024-09-10T18:45:22.566992+0000", 2024-09-10T18:48:01.973 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-09-10T18:48:01.973 INFO:teuthology.orchestra.run.smithi018.stdout: "flags_names": "hashpspool", 2024-09-10T18:48:01.973 INFO:teuthology.orchestra.run.smithi018.stdout: "type": 1, 2024-09-10T18:48:01.973 INFO:teuthology.orchestra.run.smithi018.stdout: "size": 3, 2024-09-10T18:48:01.974 INFO:teuthology.orchestra.run.smithi018.stdout: "min_size": 2, 2024-09-10T18:48:01.974 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_rule": 0, 2024-09-10T18:48:01.974 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_count": 0, 2024-09-10T18:48:01.974 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_target": 0, 2024-09-10T18:48:01.974 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T18:48:01.974 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T18:48:01.974 INFO:teuthology.orchestra.run.smithi018.stdout: "object_hash": 2, 2024-09-10T18:48:01.974 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscale_mode": "off", 2024-09-10T18:48:01.974 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num": 32, 2024-09-10T18:48:01.974 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num": 32, 2024-09-10T18:48:01.974 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num_target": 32, 2024-09-10T18:48:01.974 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_target": 32, 2024-09-10T18:48:01.974 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_pending": 32, 2024-09-10T18:48:01.975 INFO:teuthology.orchestra.run.smithi018.stdout: "last_pg_merge_meta": { 2024-09-10T18:48:01.975 INFO:teuthology.orchestra.run.smithi018.stdout: "source_pgid": "0.0", 2024-09-10T18:48:01.975 INFO:teuthology.orchestra.run.smithi018.stdout: "ready_epoch": 0, 2024-09-10T18:48:01.975 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_started": 0, 2024-09-10T18:48:01.975 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_clean": 0, 2024-09-10T18:48:01.975 INFO:teuthology.orchestra.run.smithi018.stdout: "source_version": "0'0", 2024-09-10T18:48:01.975 INFO:teuthology.orchestra.run.smithi018.stdout: "target_version": "0'0" 2024-09-10T18:48:01.975 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:01.975 INFO:teuthology.orchestra.run.smithi018.stdout: "last_change": "63", 2024-09-10T18:48:01.975 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend": "0", 2024-09-10T18:48:01.975 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T18:48:01.975 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T18:48:01.976 INFO:teuthology.orchestra.run.smithi018.stdout: "auid": 0, 2024-09-10T18:48:01.976 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_mode": "selfmanaged", 2024-09-10T18:48:01.976 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_seq": 0, 2024-09-10T18:48:01.976 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_epoch": 0, 2024-09-10T18:48:01.976 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_snaps": [], 2024-09-10T18:48:01.976 INFO:teuthology.orchestra.run.smithi018.stdout: "removed_snaps": "[]", 2024-09-10T18:48:01.976 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_bytes": 0, 2024-09-10T18:48:01.976 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_objects": 0, 2024-09-10T18:48:01.976 INFO:teuthology.orchestra.run.smithi018.stdout: "tiers": [], 2024-09-10T18:48:01.976 INFO:teuthology.orchestra.run.smithi018.stdout: "tier_of": -1, 2024-09-10T18:48:01.976 INFO:teuthology.orchestra.run.smithi018.stdout: "read_tier": -1, 2024-09-10T18:48:01.976 INFO:teuthology.orchestra.run.smithi018.stdout: "write_tier": -1, 2024-09-10T18:48:01.976 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_mode": "none", 2024-09-10T18:48:01.976 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_bytes": 0, 2024-09-10T18:48:01.977 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_objects": 0, 2024-09-10T18:48:01.977 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T18:48:01.977 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T18:48:01.977 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T18:48:01.977 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_flush_age": 0, 2024-09-10T18:48:01.977 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_evict_age": 0, 2024-09-10T18:48:01.977 INFO:teuthology.orchestra.run.smithi018.stdout: "erasure_code_profile": "", 2024-09-10T18:48:01.977 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_params": { 2024-09-10T18:48:01.977 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "none" 2024-09-10T18:48:01.977 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:01.977 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_period": 0, 2024-09-10T18:48:01.977 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_count": 0, 2024-09-10T18:48:01.977 INFO:teuthology.orchestra.run.smithi018.stdout: "use_gmt_hitset": true, 2024-09-10T18:48:01.978 INFO:teuthology.orchestra.run.smithi018.stdout: "min_read_recency_for_promote": 0, 2024-09-10T18:48:01.978 INFO:teuthology.orchestra.run.smithi018.stdout: "min_write_recency_for_promote": 0, 2024-09-10T18:48:01.978 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T18:48:01.978 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_search_last_n": 0, 2024-09-10T18:48:01.978 INFO:teuthology.orchestra.run.smithi018.stdout: "grade_table": [], 2024-09-10T18:48:01.978 INFO:teuthology.orchestra.run.smithi018.stdout: "stripe_width": 0, 2024-09-10T18:48:01.978 INFO:teuthology.orchestra.run.smithi018.stdout: "expected_num_objects": 0, 2024-09-10T18:48:01.978 INFO:teuthology.orchestra.run.smithi018.stdout: "fast_read": false, 2024-09-10T18:48:01.978 INFO:teuthology.orchestra.run.smithi018.stdout: "options": { 2024-09-10T18:48:01.978 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:01.979 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscale_bias": 4 2024-09-10T18:48:01.979 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:01.979 INFO:teuthology.orchestra.run.smithi018.stdout: "application_metadata": { 2024-09-10T18:48:01.979 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw": {} 2024-09-10T18:48:01.980 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:01.980 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:01.980 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:01.980 INFO:teuthology.orchestra.run.smithi018.stdout: "pool": 7, 2024-09-10T18:48:01.980 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_name": "unique_pool_0", 2024-09-10T18:48:01.980 INFO:teuthology.orchestra.run.smithi018.stdout: "create_time": "2024-09-10T18:47:33.332792+0000", 2024-09-10T18:48:01.980 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-09-10T18:48:01.980 INFO:teuthology.orchestra.run.smithi018.stdout: "flags_names": "hashpspool", 2024-09-10T18:48:01.980 INFO:teuthology.orchestra.run.smithi018.stdout: "type": 1, 2024-09-10T18:48:01.980 INFO:teuthology.orchestra.run.smithi018.stdout: "size": 3, 2024-09-10T18:48:01.980 INFO:teuthology.orchestra.run.smithi018.stdout: "min_size": 2, 2024-09-10T18:48:01.980 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_rule": 0, 2024-09-10T18:48:01.980 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_count": 0, 2024-09-10T18:48:01.981 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_target": 0, 2024-09-10T18:48:01.981 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T18:48:01.981 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T18:48:01.981 INFO:teuthology.orchestra.run.smithi018.stdout: "object_hash": 2, 2024-09-10T18:48:01.981 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscale_mode": "off", 2024-09-10T18:48:01.981 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num": 16, 2024-09-10T18:48:01.981 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num": 16, 2024-09-10T18:48:01.981 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num_target": 16, 2024-09-10T18:48:01.981 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_target": 16, 2024-09-10T18:48:01.981 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_pending": 16, 2024-09-10T18:48:01.981 INFO:teuthology.orchestra.run.smithi018.stdout: "last_pg_merge_meta": { 2024-09-10T18:48:01.981 INFO:teuthology.orchestra.run.smithi018.stdout: "source_pgid": "0.0", 2024-09-10T18:48:01.981 INFO:teuthology.orchestra.run.smithi018.stdout: "ready_epoch": 0, 2024-09-10T18:48:01.982 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_started": 0, 2024-09-10T18:48:01.982 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_clean": 0, 2024-09-10T18:48:01.982 INFO:teuthology.orchestra.run.smithi018.stdout: "source_version": "0'0", 2024-09-10T18:48:01.982 INFO:teuthology.orchestra.run.smithi018.stdout: "target_version": "0'0" 2024-09-10T18:48:01.982 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:01.982 INFO:teuthology.orchestra.run.smithi018.stdout: "last_change": "92", 2024-09-10T18:48:01.982 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend": "0", 2024-09-10T18:48:01.982 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T18:48:01.982 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T18:48:01.982 INFO:teuthology.orchestra.run.smithi018.stdout: "auid": 0, 2024-09-10T18:48:01.982 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_mode": "selfmanaged", 2024-09-10T18:48:01.982 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_seq": 0, 2024-09-10T18:48:01.982 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_epoch": 0, 2024-09-10T18:48:01.982 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_snaps": [], 2024-09-10T18:48:01.983 INFO:teuthology.orchestra.run.smithi018.stdout: "removed_snaps": "[]", 2024-09-10T18:48:01.983 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_bytes": 0, 2024-09-10T18:48:01.983 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_objects": 0, 2024-09-10T18:48:01.983 INFO:teuthology.orchestra.run.smithi018.stdout: "tiers": [], 2024-09-10T18:48:01.983 INFO:teuthology.orchestra.run.smithi018.stdout: "tier_of": -1, 2024-09-10T18:48:01.983 INFO:teuthology.orchestra.run.smithi018.stdout: "read_tier": -1, 2024-09-10T18:48:01.983 INFO:teuthology.orchestra.run.smithi018.stdout: "write_tier": -1, 2024-09-10T18:48:01.983 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_mode": "none", 2024-09-10T18:48:01.983 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_bytes": 0, 2024-09-10T18:48:01.983 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_objects": 0, 2024-09-10T18:48:01.983 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T18:48:01.983 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T18:48:01.983 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T18:48:01.984 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_flush_age": 0, 2024-09-10T18:48:01.984 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_evict_age": 0, 2024-09-10T18:48:01.984 INFO:teuthology.orchestra.run.smithi018.stdout: "erasure_code_profile": "", 2024-09-10T18:48:01.984 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_params": { 2024-09-10T18:48:01.984 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "none" 2024-09-10T18:48:01.984 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:01.984 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_period": 0, 2024-09-10T18:48:01.984 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_count": 0, 2024-09-10T18:48:01.984 INFO:teuthology.orchestra.run.smithi018.stdout: "use_gmt_hitset": true, 2024-09-10T18:48:01.984 INFO:teuthology.orchestra.run.smithi018.stdout: "min_read_recency_for_promote": 0, 2024-09-10T18:48:01.984 INFO:teuthology.orchestra.run.smithi018.stdout: "min_write_recency_for_promote": 0, 2024-09-10T18:48:01.984 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T18:48:01.984 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_search_last_n": 0, 2024-09-10T18:48:01.985 INFO:teuthology.orchestra.run.smithi018.stdout: "grade_table": [], 2024-09-10T18:48:01.985 INFO:teuthology.orchestra.run.smithi018.stdout: "stripe_width": 0, 2024-09-10T18:48:01.985 INFO:teuthology.orchestra.run.smithi018.stdout: "expected_num_objects": 0, 2024-09-10T18:48:01.985 INFO:teuthology.orchestra.run.smithi018.stdout: "fast_read": false, 2024-09-10T18:48:01.985 INFO:teuthology.orchestra.run.smithi018.stdout: "options": {}, 2024-09-10T18:48:01.985 INFO:teuthology.orchestra.run.smithi018.stdout: "application_metadata": { 2024-09-10T18:48:01.985 INFO:teuthology.orchestra.run.smithi018.stdout: "rados": {} 2024-09-10T18:48:01.985 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:01.985 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:01.985 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-09-10T18:48:01.985 INFO:teuthology.orchestra.run.smithi018.stdout: "osds": [ 2024-09-10T18:48:01.985 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:01.985 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 0, 2024-09-10T18:48:01.985 INFO:teuthology.orchestra.run.smithi018.stdout: "uuid": "eb4c7e67-300c-4a1a-b9f5-bb028eeeabc2", 2024-09-10T18:48:01.986 INFO:teuthology.orchestra.run.smithi018.stdout: "up": 1, 2024-09-10T18:48:01.986 INFO:teuthology.orchestra.run.smithi018.stdout: "in": 1, 2024-09-10T18:48:01.986 INFO:teuthology.orchestra.run.smithi018.stdout: "weight": 1, 2024-09-10T18:48:01.986 INFO:teuthology.orchestra.run.smithi018.stdout: "primary_affinity": 1, 2024-09-10T18:48:01.986 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_begin": 0, 2024-09-10T18:48:01.986 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_end": 0, 2024-09-10T18:48:01.986 INFO:teuthology.orchestra.run.smithi018.stdout: "up_from": 8, 2024-09-10T18:48:01.986 INFO:teuthology.orchestra.run.smithi018.stdout: "up_thru": 96, 2024-09-10T18:48:01.986 INFO:teuthology.orchestra.run.smithi018.stdout: "down_at": 0, 2024-09-10T18:48:01.986 INFO:teuthology.orchestra.run.smithi018.stdout: "lost_at": 0, 2024-09-10T18:48:01.986 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addrs": { 2024-09-10T18:48:01.986 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:48:01.986 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:01.987 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:48:01.987 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6801", 2024-09-10T18:48:01.987 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 2131279292 2024-09-10T18:48:01.987 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:01.987 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:01.987 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:01.987 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addrs": { 2024-09-10T18:48:01.987 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:48:01.987 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:01.987 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:48:01.987 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6802", 2024-09-10T18:48:01.987 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 2131279292 2024-09-10T18:48:01.987 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:01.988 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:01.988 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:01.988 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addrs": { 2024-09-10T18:48:01.988 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:48:01.988 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:01.988 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:48:01.988 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6804", 2024-09-10T18:48:01.988 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 2131279292 2024-09-10T18:48:01.988 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:01.988 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:01.988 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:01.988 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addrs": { 2024-09-10T18:48:01.988 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:48:01.989 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:01.989 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:48:01.989 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6803", 2024-09-10T18:48:01.989 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 2131279292 2024-09-10T18:48:01.989 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:01.989 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:01.989 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:01.989 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addr": "172.21.15.18:6801/2131279292", 2024-09-10T18:48:01.989 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addr": "172.21.15.18:6802/2131279292", 2024-09-10T18:48:01.989 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addr": "172.21.15.18:6804/2131279292", 2024-09-10T18:48:01.989 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addr": "172.21.15.18:6803/2131279292", 2024-09-10T18:48:01.989 INFO:teuthology.orchestra.run.smithi018.stdout: "state": [ 2024-09-10T18:48:01.989 INFO:teuthology.orchestra.run.smithi018.stdout: "exists", 2024-09-10T18:48:01.989 INFO:teuthology.orchestra.run.smithi018.stdout: "up" 2024-09-10T18:48:01.990 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:01.990 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:01.990 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:01.990 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 1, 2024-09-10T18:48:01.990 INFO:teuthology.orchestra.run.smithi018.stdout: "uuid": "63bf6a98-e349-4cc0-9502-a31693070c5b", 2024-09-10T18:48:01.990 INFO:teuthology.orchestra.run.smithi018.stdout: "up": 1, 2024-09-10T18:48:01.990 INFO:teuthology.orchestra.run.smithi018.stdout: "in": 1, 2024-09-10T18:48:01.990 INFO:teuthology.orchestra.run.smithi018.stdout: "weight": 1, 2024-09-10T18:48:01.990 INFO:teuthology.orchestra.run.smithi018.stdout: "primary_affinity": 1, 2024-09-10T18:48:01.990 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_begin": 0, 2024-09-10T18:48:01.990 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_end": 0, 2024-09-10T18:48:01.990 INFO:teuthology.orchestra.run.smithi018.stdout: "up_from": 13, 2024-09-10T18:48:01.990 INFO:teuthology.orchestra.run.smithi018.stdout: "up_thru": 88, 2024-09-10T18:48:01.990 INFO:teuthology.orchestra.run.smithi018.stdout: "down_at": 0, 2024-09-10T18:48:01.991 INFO:teuthology.orchestra.run.smithi018.stdout: "lost_at": 0, 2024-09-10T18:48:01.991 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addrs": { 2024-09-10T18:48:01.991 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:48:01.991 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:01.991 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:48:01.991 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6805", 2024-09-10T18:48:01.991 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3855640936 2024-09-10T18:48:01.991 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:01.991 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:01.991 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:01.991 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addrs": { 2024-09-10T18:48:01.991 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:48:01.991 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:01.992 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:48:01.992 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6806", 2024-09-10T18:48:01.992 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3855640936 2024-09-10T18:48:01.992 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:01.992 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:01.992 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:01.992 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addrs": { 2024-09-10T18:48:01.992 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:48:01.992 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:01.992 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:48:01.992 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6808", 2024-09-10T18:48:01.992 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3855640936 2024-09-10T18:48:01.992 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:01.993 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:01.993 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:01.993 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addrs": { 2024-09-10T18:48:01.993 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:48:01.993 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:01.993 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:48:01.993 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6807", 2024-09-10T18:48:01.993 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3855640936 2024-09-10T18:48:01.993 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:01.993 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:01.993 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:01.993 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addr": "172.21.15.18:6805/3855640936", 2024-09-10T18:48:01.993 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addr": "172.21.15.18:6806/3855640936", 2024-09-10T18:48:01.994 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addr": "172.21.15.18:6808/3855640936", 2024-09-10T18:48:01.994 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addr": "172.21.15.18:6807/3855640936", 2024-09-10T18:48:01.994 INFO:teuthology.orchestra.run.smithi018.stdout: "state": [ 2024-09-10T18:48:01.994 INFO:teuthology.orchestra.run.smithi018.stdout: "exists", 2024-09-10T18:48:01.994 INFO:teuthology.orchestra.run.smithi018.stdout: "up" 2024-09-10T18:48:01.994 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:01.994 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:01.994 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:01.994 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 2, 2024-09-10T18:48:01.994 INFO:teuthology.orchestra.run.smithi018.stdout: "uuid": "79a01c29-4f4d-43d1-b1ea-a3fcf014f84b", 2024-09-10T18:48:01.994 INFO:teuthology.orchestra.run.smithi018.stdout: "up": 1, 2024-09-10T18:48:01.994 INFO:teuthology.orchestra.run.smithi018.stdout: "in": 0, 2024-09-10T18:48:01.994 INFO:teuthology.orchestra.run.smithi018.stdout: "weight": 0, 2024-09-10T18:48:01.994 INFO:teuthology.orchestra.run.smithi018.stdout: "primary_affinity": 1, 2024-09-10T18:48:01.995 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_begin": 0, 2024-09-10T18:48:01.995 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_end": 0, 2024-09-10T18:48:01.995 INFO:teuthology.orchestra.run.smithi018.stdout: "up_from": 18, 2024-09-10T18:48:01.995 INFO:teuthology.orchestra.run.smithi018.stdout: "up_thru": 86, 2024-09-10T18:48:01.995 INFO:teuthology.orchestra.run.smithi018.stdout: "down_at": 0, 2024-09-10T18:48:01.995 INFO:teuthology.orchestra.run.smithi018.stdout: "lost_at": 0, 2024-09-10T18:48:01.995 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addrs": { 2024-09-10T18:48:01.995 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:48:01.995 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:01.995 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:48:01.995 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6809", 2024-09-10T18:48:01.995 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 4107623855 2024-09-10T18:48:01.995 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:01.996 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:01.996 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:01.996 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addrs": { 2024-09-10T18:48:01.996 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:48:01.996 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:01.996 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:48:01.996 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6810", 2024-09-10T18:48:01.996 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 4107623855 2024-09-10T18:48:01.996 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:01.996 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:01.996 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:01.996 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addrs": { 2024-09-10T18:48:01.997 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:48:01.997 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:01.997 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:48:01.997 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6812", 2024-09-10T18:48:01.997 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 4107623855 2024-09-10T18:48:01.997 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:01.997 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:01.997 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:01.997 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addrs": { 2024-09-10T18:48:01.997 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:48:01.997 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:01.997 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:48:01.997 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6811", 2024-09-10T18:48:01.997 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 4107623855 2024-09-10T18:48:01.998 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:01.998 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:01.998 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:01.998 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addr": "172.21.15.18:6809/4107623855", 2024-09-10T18:48:01.998 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addr": "172.21.15.18:6810/4107623855", 2024-09-10T18:48:01.998 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addr": "172.21.15.18:6812/4107623855", 2024-09-10T18:48:01.998 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addr": "172.21.15.18:6811/4107623855", 2024-09-10T18:48:01.998 INFO:teuthology.orchestra.run.smithi018.stdout: "state": [ 2024-09-10T18:48:01.998 INFO:teuthology.orchestra.run.smithi018.stdout: "exists", 2024-09-10T18:48:01.998 INFO:teuthology.orchestra.run.smithi018.stdout: "up" 2024-09-10T18:48:01.998 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:01.998 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:01.998 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:01.999 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 3, 2024-09-10T18:48:01.999 INFO:teuthology.orchestra.run.smithi018.stdout: "uuid": "1809e677-b98c-4924-982e-9297e37563af", 2024-09-10T18:48:01.999 INFO:teuthology.orchestra.run.smithi018.stdout: "up": 1, 2024-09-10T18:48:01.999 INFO:teuthology.orchestra.run.smithi018.stdout: "in": 1, 2024-09-10T18:48:01.999 INFO:teuthology.orchestra.run.smithi018.stdout: "weight": 1, 2024-09-10T18:48:01.999 INFO:teuthology.orchestra.run.smithi018.stdout: "primary_affinity": 1, 2024-09-10T18:48:01.999 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_begin": 0, 2024-09-10T18:48:01.999 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_end": 0, 2024-09-10T18:48:01.999 INFO:teuthology.orchestra.run.smithi018.stdout: "up_from": 25, 2024-09-10T18:48:01.999 INFO:teuthology.orchestra.run.smithi018.stdout: "up_thru": 96, 2024-09-10T18:48:01.999 INFO:teuthology.orchestra.run.smithi018.stdout: "down_at": 0, 2024-09-10T18:48:01.999 INFO:teuthology.orchestra.run.smithi018.stdout: "lost_at": 0, 2024-09-10T18:48:01.999 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addrs": { 2024-09-10T18:48:01.999 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:48:02.000 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.000 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:48:02.000 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6813", 2024-09-10T18:48:02.000 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 2732204253 2024-09-10T18:48:02.000 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:02.000 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:02.000 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:02.000 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addrs": { 2024-09-10T18:48:02.000 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:48:02.000 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.000 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:48:02.000 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6814", 2024-09-10T18:48:02.000 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 2732204253 2024-09-10T18:48:02.001 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:02.001 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:02.001 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:02.001 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addrs": { 2024-09-10T18:48:02.001 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:48:02.001 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.001 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:48:02.001 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6816", 2024-09-10T18:48:02.001 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 2732204253 2024-09-10T18:48:02.001 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:02.001 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:02.001 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:02.001 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addrs": { 2024-09-10T18:48:02.002 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:48:02.002 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.002 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:48:02.002 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6815", 2024-09-10T18:48:02.002 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 2732204253 2024-09-10T18:48:02.002 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:02.002 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:02.002 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:02.002 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addr": "172.21.15.18:6813/2732204253", 2024-09-10T18:48:02.002 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addr": "172.21.15.18:6814/2732204253", 2024-09-10T18:48:02.002 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addr": "172.21.15.18:6816/2732204253", 2024-09-10T18:48:02.002 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addr": "172.21.15.18:6815/2732204253", 2024-09-10T18:48:02.003 INFO:teuthology.orchestra.run.smithi018.stdout: "state": [ 2024-09-10T18:48:02.003 INFO:teuthology.orchestra.run.smithi018.stdout: "exists", 2024-09-10T18:48:02.003 INFO:teuthology.orchestra.run.smithi018.stdout: "up" 2024-09-10T18:48:02.003 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:02.003 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:02.003 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.003 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 4, 2024-09-10T18:48:02.003 INFO:teuthology.orchestra.run.smithi018.stdout: "uuid": "b682a300-bb53-4f36-8a9f-0753164ec4b7", 2024-09-10T18:48:02.003 INFO:teuthology.orchestra.run.smithi018.stdout: "up": 1, 2024-09-10T18:48:02.003 INFO:teuthology.orchestra.run.smithi018.stdout: "in": 1, 2024-09-10T18:48:02.004 INFO:teuthology.orchestra.run.smithi018.stdout: "weight": 1, 2024-09-10T18:48:02.004 INFO:teuthology.orchestra.run.smithi018.stdout: "primary_affinity": 1, 2024-09-10T18:48:02.004 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_begin": 0, 2024-09-10T18:48:02.004 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_end": 0, 2024-09-10T18:48:02.004 INFO:teuthology.orchestra.run.smithi018.stdout: "up_from": 30, 2024-09-10T18:48:02.004 INFO:teuthology.orchestra.run.smithi018.stdout: "up_thru": 96, 2024-09-10T18:48:02.004 INFO:teuthology.orchestra.run.smithi018.stdout: "down_at": 0, 2024-09-10T18:48:02.004 INFO:teuthology.orchestra.run.smithi018.stdout: "lost_at": 0, 2024-09-10T18:48:02.004 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addrs": { 2024-09-10T18:48:02.004 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:48:02.004 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.005 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:48:02.005 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6800", 2024-09-10T18:48:02.005 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 465860081 2024-09-10T18:48:02.005 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:02.005 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:02.005 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:02.005 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addrs": { 2024-09-10T18:48:02.005 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:48:02.005 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.005 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:48:02.005 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6801", 2024-09-10T18:48:02.005 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 465860081 2024-09-10T18:48:02.005 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:02.005 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:02.006 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:02.006 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addrs": { 2024-09-10T18:48:02.006 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:48:02.006 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.006 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:48:02.006 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6803", 2024-09-10T18:48:02.006 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 465860081 2024-09-10T18:48:02.006 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:02.006 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:02.006 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:02.006 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addrs": { 2024-09-10T18:48:02.006 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:48:02.006 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.007 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:48:02.007 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6802", 2024-09-10T18:48:02.007 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 465860081 2024-09-10T18:48:02.007 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:02.007 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:02.007 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:02.007 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addr": "172.21.15.103:6800/465860081", 2024-09-10T18:48:02.007 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addr": "172.21.15.103:6801/465860081", 2024-09-10T18:48:02.007 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addr": "172.21.15.103:6803/465860081", 2024-09-10T18:48:02.007 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addr": "172.21.15.103:6802/465860081", 2024-09-10T18:48:02.007 INFO:teuthology.orchestra.run.smithi018.stdout: "state": [ 2024-09-10T18:48:02.007 INFO:teuthology.orchestra.run.smithi018.stdout: "exists", 2024-09-10T18:48:02.007 INFO:teuthology.orchestra.run.smithi018.stdout: "up" 2024-09-10T18:48:02.008 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:02.008 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:02.008 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.008 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 5, 2024-09-10T18:48:02.008 INFO:teuthology.orchestra.run.smithi018.stdout: "uuid": "2bbfdb6c-0709-4fe4-8a4a-006f576f18e6", 2024-09-10T18:48:02.008 INFO:teuthology.orchestra.run.smithi018.stdout: "up": 1, 2024-09-10T18:48:02.008 INFO:teuthology.orchestra.run.smithi018.stdout: "in": 1, 2024-09-10T18:48:02.008 INFO:teuthology.orchestra.run.smithi018.stdout: "weight": 1, 2024-09-10T18:48:02.008 INFO:teuthology.orchestra.run.smithi018.stdout: "primary_affinity": 1, 2024-09-10T18:48:02.008 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_begin": 0, 2024-09-10T18:48:02.008 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_end": 0, 2024-09-10T18:48:02.008 INFO:teuthology.orchestra.run.smithi018.stdout: "up_from": 37, 2024-09-10T18:48:02.008 INFO:teuthology.orchestra.run.smithi018.stdout: "up_thru": 88, 2024-09-10T18:48:02.009 INFO:teuthology.orchestra.run.smithi018.stdout: "down_at": 0, 2024-09-10T18:48:02.009 INFO:teuthology.orchestra.run.smithi018.stdout: "lost_at": 0, 2024-09-10T18:48:02.009 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addrs": { 2024-09-10T18:48:02.009 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:48:02.009 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.009 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:48:02.009 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6804", 2024-09-10T18:48:02.009 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3498719258 2024-09-10T18:48:02.009 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:02.009 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:02.009 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:02.009 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addrs": { 2024-09-10T18:48:02.009 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:48:02.009 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.010 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:48:02.010 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6805", 2024-09-10T18:48:02.010 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3498719258 2024-09-10T18:48:02.010 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:02.010 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:02.010 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:02.010 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addrs": { 2024-09-10T18:48:02.010 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:48:02.010 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.010 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:48:02.010 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6807", 2024-09-10T18:48:02.010 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3498719258 2024-09-10T18:48:02.010 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:02.011 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:02.011 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:02.011 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addrs": { 2024-09-10T18:48:02.011 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:48:02.011 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.011 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:48:02.011 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6806", 2024-09-10T18:48:02.011 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3498719258 2024-09-10T18:48:02.011 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:02.011 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:02.011 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:02.011 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addr": "172.21.15.103:6804/3498719258", 2024-09-10T18:48:02.011 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addr": "172.21.15.103:6805/3498719258", 2024-09-10T18:48:02.011 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addr": "172.21.15.103:6807/3498719258", 2024-09-10T18:48:02.012 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addr": "172.21.15.103:6806/3498719258", 2024-09-10T18:48:02.012 INFO:teuthology.orchestra.run.smithi018.stdout: "state": [ 2024-09-10T18:48:02.012 INFO:teuthology.orchestra.run.smithi018.stdout: "exists", 2024-09-10T18:48:02.012 INFO:teuthology.orchestra.run.smithi018.stdout: "up" 2024-09-10T18:48:02.012 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:02.012 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:02.012 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.012 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 6, 2024-09-10T18:48:02.012 INFO:teuthology.orchestra.run.smithi018.stdout: "uuid": "31f9711e-0ba8-4aeb-b852-7d93e8fcce33", 2024-09-10T18:48:02.012 INFO:teuthology.orchestra.run.smithi018.stdout: "up": 1, 2024-09-10T18:48:02.012 INFO:teuthology.orchestra.run.smithi018.stdout: "in": 1, 2024-09-10T18:48:02.012 INFO:teuthology.orchestra.run.smithi018.stdout: "weight": 1, 2024-09-10T18:48:02.012 INFO:teuthology.orchestra.run.smithi018.stdout: "primary_affinity": 1, 2024-09-10T18:48:02.013 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_begin": 0, 2024-09-10T18:48:02.013 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_end": 0, 2024-09-10T18:48:02.013 INFO:teuthology.orchestra.run.smithi018.stdout: "up_from": 43, 2024-09-10T18:48:02.013 INFO:teuthology.orchestra.run.smithi018.stdout: "up_thru": 96, 2024-09-10T18:48:02.013 INFO:teuthology.orchestra.run.smithi018.stdout: "down_at": 0, 2024-09-10T18:48:02.013 INFO:teuthology.orchestra.run.smithi018.stdout: "lost_at": 0, 2024-09-10T18:48:02.013 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addrs": { 2024-09-10T18:48:02.013 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:48:02.013 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.013 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:48:02.013 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6808", 2024-09-10T18:48:02.013 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 184353788 2024-09-10T18:48:02.013 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:02.013 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:02.014 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:02.014 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addrs": { 2024-09-10T18:48:02.014 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:48:02.014 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.014 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:48:02.014 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6809", 2024-09-10T18:48:02.014 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 184353788 2024-09-10T18:48:02.014 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:02.014 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:02.014 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:02.014 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addrs": { 2024-09-10T18:48:02.014 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:48:02.014 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.014 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:48:02.015 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6811", 2024-09-10T18:48:02.015 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 184353788 2024-09-10T18:48:02.015 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:02.015 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:02.015 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:02.015 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addrs": { 2024-09-10T18:48:02.015 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:48:02.015 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.015 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:48:02.015 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6810", 2024-09-10T18:48:02.015 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 184353788 2024-09-10T18:48:02.015 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:02.015 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:02.015 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:02.016 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addr": "172.21.15.103:6808/184353788", 2024-09-10T18:48:02.016 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addr": "172.21.15.103:6809/184353788", 2024-09-10T18:48:02.016 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addr": "172.21.15.103:6811/184353788", 2024-09-10T18:48:02.016 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addr": "172.21.15.103:6810/184353788", 2024-09-10T18:48:02.016 INFO:teuthology.orchestra.run.smithi018.stdout: "state": [ 2024-09-10T18:48:02.016 INFO:teuthology.orchestra.run.smithi018.stdout: "exists", 2024-09-10T18:48:02.016 INFO:teuthology.orchestra.run.smithi018.stdout: "up" 2024-09-10T18:48:02.016 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:02.016 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:02.016 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.016 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 7, 2024-09-10T18:48:02.016 INFO:teuthology.orchestra.run.smithi018.stdout: "uuid": "f12282f8-4747-45ff-8336-95440560a88f", 2024-09-10T18:48:02.016 INFO:teuthology.orchestra.run.smithi018.stdout: "up": 1, 2024-09-10T18:48:02.017 INFO:teuthology.orchestra.run.smithi018.stdout: "in": 1, 2024-09-10T18:48:02.017 INFO:teuthology.orchestra.run.smithi018.stdout: "weight": 1, 2024-09-10T18:48:02.017 INFO:teuthology.orchestra.run.smithi018.stdout: "primary_affinity": 1, 2024-09-10T18:48:02.017 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_begin": 0, 2024-09-10T18:48:02.017 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_end": 0, 2024-09-10T18:48:02.017 INFO:teuthology.orchestra.run.smithi018.stdout: "up_from": 52, 2024-09-10T18:48:02.017 INFO:teuthology.orchestra.run.smithi018.stdout: "up_thru": 88, 2024-09-10T18:48:02.017 INFO:teuthology.orchestra.run.smithi018.stdout: "down_at": 0, 2024-09-10T18:48:02.017 INFO:teuthology.orchestra.run.smithi018.stdout: "lost_at": 0, 2024-09-10T18:48:02.017 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addrs": { 2024-09-10T18:48:02.017 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:48:02.017 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.017 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:48:02.017 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6812", 2024-09-10T18:48:02.018 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 941024403 2024-09-10T18:48:02.018 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:02.018 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:02.018 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:02.018 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addrs": { 2024-09-10T18:48:02.018 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:48:02.018 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.018 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:48:02.018 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6813", 2024-09-10T18:48:02.018 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 941024403 2024-09-10T18:48:02.018 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:02.018 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:02.018 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:02.019 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addrs": { 2024-09-10T18:48:02.019 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:48:02.019 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.020 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:48:02.020 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6815", 2024-09-10T18:48:02.020 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 941024403 2024-09-10T18:48:02.020 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:02.020 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:02.020 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:02.020 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addrs": { 2024-09-10T18:48:02.020 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:48:02.020 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.020 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:48:02.020 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6814", 2024-09-10T18:48:02.020 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 941024403 2024-09-10T18:48:02.020 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:02.021 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:02.021 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:02.021 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addr": "172.21.15.103:6812/941024403", 2024-09-10T18:48:02.021 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addr": "172.21.15.103:6813/941024403", 2024-09-10T18:48:02.021 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addr": "172.21.15.103:6815/941024403", 2024-09-10T18:48:02.021 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addr": "172.21.15.103:6814/941024403", 2024-09-10T18:48:02.021 INFO:teuthology.orchestra.run.smithi018.stdout: "state": [ 2024-09-10T18:48:02.021 INFO:teuthology.orchestra.run.smithi018.stdout: "exists", 2024-09-10T18:48:02.021 INFO:teuthology.orchestra.run.smithi018.stdout: "up" 2024-09-10T18:48:02.021 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:02.021 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:02.021 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-09-10T18:48:02.021 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_xinfo": [ 2024-09-10T18:48:02.022 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.022 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 0, 2024-09-10T18:48:02.022 INFO:teuthology.orchestra.run.smithi018.stdout: "down_stamp": "0.000000", 2024-09-10T18:48:02.022 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_probability": 0, 2024-09-10T18:48:02.022 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_interval": 0, 2024-09-10T18:48:02.022 INFO:teuthology.orchestra.run.smithi018.stdout: "features": 4540138320759226367, 2024-09-10T18:48:02.022 INFO:teuthology.orchestra.run.smithi018.stdout: "old_weight": 0, 2024-09-10T18:48:02.022 INFO:teuthology.orchestra.run.smithi018.stdout: "last_purged_snaps_scrub": "2024-09-10T18:47:38.275413+0000", 2024-09-10T18:48:02.022 INFO:teuthology.orchestra.run.smithi018.stdout: "dead_epoch": 0 2024-09-10T18:48:02.022 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:02.022 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.022 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 1, 2024-09-10T18:48:02.022 INFO:teuthology.orchestra.run.smithi018.stdout: "down_stamp": "0.000000", 2024-09-10T18:48:02.022 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_probability": 0, 2024-09-10T18:48:02.023 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_interval": 0, 2024-09-10T18:48:02.023 INFO:teuthology.orchestra.run.smithi018.stdout: "features": 4540138320759226367, 2024-09-10T18:48:02.023 INFO:teuthology.orchestra.run.smithi018.stdout: "old_weight": 0, 2024-09-10T18:48:02.023 INFO:teuthology.orchestra.run.smithi018.stdout: "last_purged_snaps_scrub": "2024-09-10T18:47:48.859898+0000", 2024-09-10T18:48:02.023 INFO:teuthology.orchestra.run.smithi018.stdout: "dead_epoch": 0 2024-09-10T18:48:02.023 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:02.023 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.023 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 2, 2024-09-10T18:48:02.023 INFO:teuthology.orchestra.run.smithi018.stdout: "down_stamp": "0.000000", 2024-09-10T18:48:02.023 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_probability": 0, 2024-09-10T18:48:02.023 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_interval": 0, 2024-09-10T18:48:02.023 INFO:teuthology.orchestra.run.smithi018.stdout: "features": 4540138320759226367, 2024-09-10T18:48:02.023 INFO:teuthology.orchestra.run.smithi018.stdout: "old_weight": 65536, 2024-09-10T18:48:02.024 INFO:teuthology.orchestra.run.smithi018.stdout: "last_purged_snaps_scrub": "2024-09-10T18:46:52.525500+0000", 2024-09-10T18:48:02.024 INFO:teuthology.orchestra.run.smithi018.stdout: "dead_epoch": 0 2024-09-10T18:48:02.024 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:02.024 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.024 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 3, 2024-09-10T18:48:02.024 INFO:teuthology.orchestra.run.smithi018.stdout: "down_stamp": "0.000000", 2024-09-10T18:48:02.024 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_probability": 0, 2024-09-10T18:48:02.024 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_interval": 0, 2024-09-10T18:48:02.024 INFO:teuthology.orchestra.run.smithi018.stdout: "features": 4540138320759226367, 2024-09-10T18:48:02.024 INFO:teuthology.orchestra.run.smithi018.stdout: "old_weight": 0, 2024-09-10T18:48:02.024 INFO:teuthology.orchestra.run.smithi018.stdout: "last_purged_snaps_scrub": "2024-09-10T18:47:39.084179+0000", 2024-09-10T18:48:02.024 INFO:teuthology.orchestra.run.smithi018.stdout: "dead_epoch": 0 2024-09-10T18:48:02.024 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:02.025 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.025 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 4, 2024-09-10T18:48:02.025 INFO:teuthology.orchestra.run.smithi018.stdout: "down_stamp": "0.000000", 2024-09-10T18:48:02.025 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_probability": 0, 2024-09-10T18:48:02.025 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_interval": 0, 2024-09-10T18:48:02.025 INFO:teuthology.orchestra.run.smithi018.stdout: "features": 4540138320759226367, 2024-09-10T18:48:02.025 INFO:teuthology.orchestra.run.smithi018.stdout: "old_weight": 0, 2024-09-10T18:48:02.025 INFO:teuthology.orchestra.run.smithi018.stdout: "last_purged_snaps_scrub": "2024-09-10T18:47:31.475999+0000", 2024-09-10T18:48:02.025 INFO:teuthology.orchestra.run.smithi018.stdout: "dead_epoch": 0 2024-09-10T18:48:02.025 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:02.025 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.025 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 5, 2024-09-10T18:48:02.025 INFO:teuthology.orchestra.run.smithi018.stdout: "down_stamp": "0.000000", 2024-09-10T18:48:02.025 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_probability": 0, 2024-09-10T18:48:02.026 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_interval": 0, 2024-09-10T18:48:02.026 INFO:teuthology.orchestra.run.smithi018.stdout: "features": 4540138320759226367, 2024-09-10T18:48:02.026 INFO:teuthology.orchestra.run.smithi018.stdout: "old_weight": 0, 2024-09-10T18:48:02.026 INFO:teuthology.orchestra.run.smithi018.stdout: "last_purged_snaps_scrub": "2024-09-10T18:46:48.968665+0000", 2024-09-10T18:48:02.026 INFO:teuthology.orchestra.run.smithi018.stdout: "dead_epoch": 0 2024-09-10T18:48:02.026 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:02.026 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.026 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 6, 2024-09-10T18:48:02.026 INFO:teuthology.orchestra.run.smithi018.stdout: "down_stamp": "0.000000", 2024-09-10T18:48:02.026 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_probability": 0, 2024-09-10T18:48:02.026 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_interval": 0, 2024-09-10T18:48:02.026 INFO:teuthology.orchestra.run.smithi018.stdout: "features": 4540138320759226367, 2024-09-10T18:48:02.026 INFO:teuthology.orchestra.run.smithi018.stdout: "old_weight": 0, 2024-09-10T18:48:02.027 INFO:teuthology.orchestra.run.smithi018.stdout: "last_purged_snaps_scrub": "2024-09-10T18:47:06.337009+0000", 2024-09-10T18:48:02.027 INFO:teuthology.orchestra.run.smithi018.stdout: "dead_epoch": 0 2024-09-10T18:48:02.027 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:02.027 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.027 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 7, 2024-09-10T18:48:02.027 INFO:teuthology.orchestra.run.smithi018.stdout: "down_stamp": "0.000000", 2024-09-10T18:48:02.027 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_probability": 0, 2024-09-10T18:48:02.027 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_interval": 0, 2024-09-10T18:48:02.027 INFO:teuthology.orchestra.run.smithi018.stdout: "features": 4540138320759226367, 2024-09-10T18:48:02.027 INFO:teuthology.orchestra.run.smithi018.stdout: "old_weight": 0, 2024-09-10T18:48:02.027 INFO:teuthology.orchestra.run.smithi018.stdout: "last_purged_snaps_scrub": "2024-09-10T18:47:38.740068+0000", 2024-09-10T18:48:02.027 INFO:teuthology.orchestra.run.smithi018.stdout: "dead_epoch": 0 2024-09-10T18:48:02.027 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:02.027 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-09-10T18:48:02.028 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_upmap": [], 2024-09-10T18:48:02.028 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_upmap_items": [ 2024-09-10T18:48:02.028 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.028 INFO:teuthology.orchestra.run.smithi018.stdout: "pgid": "4.e", 2024-09-10T18:48:02.028 INFO:teuthology.orchestra.run.smithi018.stdout: "mappings": [ 2024-09-10T18:48:02.028 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.028 INFO:teuthology.orchestra.run.smithi018.stdout: "from": 1, 2024-09-10T18:48:02.028 INFO:teuthology.orchestra.run.smithi018.stdout: "to": 5 2024-09-10T18:48:02.028 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:02.028 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:02.028 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:02.028 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.028 INFO:teuthology.orchestra.run.smithi018.stdout: "pgid": "4.f", 2024-09-10T18:48:02.029 INFO:teuthology.orchestra.run.smithi018.stdout: "mappings": [ 2024-09-10T18:48:02.029 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.029 INFO:teuthology.orchestra.run.smithi018.stdout: "from": 1, 2024-09-10T18:48:02.029 INFO:teuthology.orchestra.run.smithi018.stdout: "to": 5 2024-09-10T18:48:02.029 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:02.029 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:02.029 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:02.029 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.029 INFO:teuthology.orchestra.run.smithi018.stdout: "pgid": "5.5", 2024-09-10T18:48:02.029 INFO:teuthology.orchestra.run.smithi018.stdout: "mappings": [ 2024-09-10T18:48:02.029 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.029 INFO:teuthology.orchestra.run.smithi018.stdout: "from": 0, 2024-09-10T18:48:02.029 INFO:teuthology.orchestra.run.smithi018.stdout: "to": 6 2024-09-10T18:48:02.030 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:02.030 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:02.030 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:02.030 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.030 INFO:teuthology.orchestra.run.smithi018.stdout: "pgid": "5.10", 2024-09-10T18:48:02.030 INFO:teuthology.orchestra.run.smithi018.stdout: "mappings": [ 2024-09-10T18:48:02.030 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.030 INFO:teuthology.orchestra.run.smithi018.stdout: "from": 7, 2024-09-10T18:48:02.030 INFO:teuthology.orchestra.run.smithi018.stdout: "to": 3 2024-09-10T18:48:02.030 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:02.030 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:02.030 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:02.030 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.030 INFO:teuthology.orchestra.run.smithi018.stdout: "pgid": "6.f", 2024-09-10T18:48:02.031 INFO:teuthology.orchestra.run.smithi018.stdout: "mappings": [ 2024-09-10T18:48:02.031 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:48:02.031 INFO:teuthology.orchestra.run.smithi018.stdout: "from": 3, 2024-09-10T18:48:02.031 INFO:teuthology.orchestra.run.smithi018.stdout: "to": 0 2024-09-10T18:48:02.031 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:02.031 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:48:02.031 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:02.031 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-09-10T18:48:02.031 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_temp": [], 2024-09-10T18:48:02.031 INFO:teuthology.orchestra.run.smithi018.stdout: "primary_temp": [], 2024-09-10T18:48:02.031 INFO:teuthology.orchestra.run.smithi018.stdout: "blocklist": { 2024-09-10T18:48:02.031 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/3021537013": "2024-09-11T18:45:54.998616+0000", 2024-09-10T18:48:02.031 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/458572898": "2024-09-11T18:40:28.880042+0000", 2024-09-10T18:48:02.031 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/1859541419": "2024-09-11T18:40:28.880042+0000", 2024-09-10T18:48:02.032 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/3090642970": "2024-09-11T18:40:59.659411+0000", 2024-09-10T18:48:02.032 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/2875894312": "2024-09-11T18:40:59.659411+0000", 2024-09-10T18:48:02.032 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/178638059": "2024-09-11T18:40:59.659411+0000", 2024-09-10T18:48:02.032 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/692931402": "2024-09-11T18:45:54.998616+0000", 2024-09-10T18:48:02.032 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/2408360238": "2024-09-11T18:45:54.998616+0000", 2024-09-10T18:48:02.032 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/57192865": "2024-09-11T18:40:28.880042+0000", 2024-09-10T18:48:02.032 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:6800/2748756682": "2024-09-11T18:40:59.659411+0000", 2024-09-10T18:48:02.032 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:6800/732375116": "2024-09-11T18:45:54.998616+0000", 2024-09-10T18:48:02.032 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:6800/3869801150": "2024-09-11T18:40:28.880042+0000", 2024-09-10T18:48:02.032 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/2982361259": "2024-09-11T18:45:54.998616+0000", 2024-09-10T18:48:02.032 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/185700038": "2024-09-11T18:45:54.998616+0000" 2024-09-10T18:48:02.032 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:02.032 INFO:teuthology.orchestra.run.smithi018.stdout: "range_blocklist": {}, 2024-09-10T18:48:02.032 INFO:teuthology.orchestra.run.smithi018.stdout: "erasure_code_profiles": { 2024-09-10T18:48:02.033 INFO:teuthology.orchestra.run.smithi018.stdout: "default": { 2024-09-10T18:48:02.033 INFO:teuthology.orchestra.run.smithi018.stdout: "crush-failure-domain": "osd", 2024-09-10T18:48:02.033 INFO:teuthology.orchestra.run.smithi018.stdout: "k": "2", 2024-09-10T18:48:02.033 INFO:teuthology.orchestra.run.smithi018.stdout: "m": "1", 2024-09-10T18:48:02.033 INFO:teuthology.orchestra.run.smithi018.stdout: "plugin": "jerasure", 2024-09-10T18:48:02.033 INFO:teuthology.orchestra.run.smithi018.stdout: "technique": "reed_sol_van" 2024-09-10T18:48:02.033 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:02.033 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:48:02.033 INFO:teuthology.orchestra.run.smithi018.stdout: "removed_snaps_queue": [], 2024-09-10T18:48:02.033 INFO:teuthology.orchestra.run.smithi018.stdout: "new_removed_snaps": [], 2024-09-10T18:48:02.033 INFO:teuthology.orchestra.run.smithi018.stdout: "new_purged_snaps": [], 2024-09-10T18:48:02.033 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_node_flags": {}, 2024-09-10T18:48:02.033 INFO:teuthology.orchestra.run.smithi018.stdout: "device_class_flags": {}, 2024-09-10T18:48:02.033 INFO:teuthology.orchestra.run.smithi018.stdout: "stretch_mode": { 2024-09-10T18:48:02.034 INFO:teuthology.orchestra.run.smithi018.stdout: "stretch_mode_enabled": false, 2024-09-10T18:48:02.034 INFO:teuthology.orchestra.run.smithi018.stdout: "stretch_bucket_count": 0, 2024-09-10T18:48:02.034 INFO:teuthology.orchestra.run.smithi018.stdout: "degraded_stretch_mode": 0, 2024-09-10T18:48:02.034 INFO:teuthology.orchestra.run.smithi018.stdout: "recovering_stretch_mode": 0, 2024-09-10T18:48:02.034 INFO:teuthology.orchestra.run.smithi018.stdout: "stretch_mode_bucket": 0 2024-09-10T18:48:02.034 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:48:02.034 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T18:48:02.048 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:02.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:02.049 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:02.150 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:02.181 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:02.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:02.182 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:02.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:02 smithi018 ceph-mon[40196]: pgmap v94: 148 pgs: 148 active+clean; 2.2 GiB data, 8.4 GiB used, 617 GiB / 626 GiB avail; 853 B/s rd, 169 MiB/s wr, 2.71k op/s; 18 B/s, 0 objects/s recovering 2024-09-10T18:48:02.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:02 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1812595581' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-10T18:48:02.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:02 smithi018 ceph-mon[29336]: pgmap v94: 148 pgs: 148 active+clean; 2.2 GiB data, 8.4 GiB used, 617 GiB / 626 GiB avail; 853 B/s rd, 169 MiB/s wr, 2.71k op/s; 18 B/s, 0 objects/s recovering 2024-09-10T18:48:02.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:02 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1812595581' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-10T18:48:02.201 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 18 16 43255 43239 150.099 107.562 0.00700965 0.00665697 2024-09-10T18:48:02.283 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:02.324 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:02.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:02.325 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:02.426 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:02.459 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:02.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:02.460 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:02.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:02 smithi103 ceph-mon[33163]: pgmap v94: 148 pgs: 148 active+clean; 2.2 GiB data, 8.4 GiB used, 617 GiB / 626 GiB avail; 853 B/s rd, 169 MiB/s wr, 2.71k op/s; 18 B/s, 0 objects/s recovering 2024-09-10T18:48:02.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:02 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1812595581' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-10T18:48:02.560 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:02.604 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:02.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:02.605 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:02.706 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:02.737 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:02.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:02.738 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:02.838 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:02.869 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:02.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:02.870 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:02.971 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:03.005 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:03.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:03.006 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:03.062 INFO:tasks.thrashosds.thrasher:j is {'epoch': 98, 'fsid': 'ba327f6c-6fa3-11ef-bcea-c7b262605968', 'created': '2024-09-10T18:40:07.669687+0000', 'modified': '2024-09-10T18:47:58.485307+0000', 'last_up_change': '2024-09-10T18:45:05.425833+0000', 'last_in_change': '2024-09-10T18:47:35.385915+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 7, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T18:43:08.162659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': '.rgw.root', 'create_time': '2024-09-10T18:45:16.512219+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 3, 'pool_name': 'datapool', 'create_time': '2024-09-10T18:45:16.943626+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 60, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-10T18:45:18.450026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-10T18:45:20.494618+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-10T18:45:22.566992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}}, {'pool': 7, 'pool_name': 'unique_pool_0', 'create_time': '2024-09-10T18:47:33.332792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': 'eb4c7e67-300c-4a1a-b9f5-bb028eeeabc2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 96, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6801', 'nonce': 2131279292}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6802', 'nonce': 2131279292}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6804', 'nonce': 2131279292}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6803', 'nonce': 2131279292}]}, 'public_addr': '172.21.15.18:6801/2131279292', 'cluster_addr': '172.21.15.18:6802/2131279292', 'heartbeat_back_addr': '172.21.15.18:6804/2131279292', 'heartbeat_front_addr': '172.21.15.18:6803/2131279292', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '63bf6a98-e349-4cc0-9502-a31693070c5b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 88, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6805', 'nonce': 3855640936}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6806', 'nonce': 3855640936}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6808', 'nonce': 3855640936}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6807', 'nonce': 3855640936}]}, 'public_addr': '172.21.15.18:6805/3855640936', 'cluster_addr': '172.21.15.18:6806/3855640936', 'heartbeat_back_addr': '172.21.15.18:6808/3855640936', 'heartbeat_front_addr': '172.21.15.18:6807/3855640936', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '79a01c29-4f4d-43d1-b1ea-a3fcf014f84b', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 86, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6809', 'nonce': 4107623855}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6810', 'nonce': 4107623855}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6812', 'nonce': 4107623855}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6811', 'nonce': 4107623855}]}, 'public_addr': '172.21.15.18:6809/4107623855', 'cluster_addr': '172.21.15.18:6810/4107623855', 'heartbeat_back_addr': '172.21.15.18:6812/4107623855', 'heartbeat_front_addr': '172.21.15.18:6811/4107623855', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '1809e677-b98c-4924-982e-9297e37563af', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 25, 'up_thru': 96, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6813', 'nonce': 2732204253}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6814', 'nonce': 2732204253}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6816', 'nonce': 2732204253}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6815', 'nonce': 2732204253}]}, 'public_addr': '172.21.15.18:6813/2732204253', 'cluster_addr': '172.21.15.18:6814/2732204253', 'heartbeat_back_addr': '172.21.15.18:6816/2732204253', 'heartbeat_front_addr': '172.21.15.18:6815/2732204253', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'b682a300-bb53-4f36-8a9f-0753164ec4b7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 30, 'up_thru': 96, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6800', 'nonce': 465860081}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6801', 'nonce': 465860081}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6803', 'nonce': 465860081}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6802', 'nonce': 465860081}]}, 'public_addr': '172.21.15.103:6800/465860081', 'cluster_addr': '172.21.15.103:6801/465860081', 'heartbeat_back_addr': '172.21.15.103:6803/465860081', 'heartbeat_front_addr': '172.21.15.103:6802/465860081', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '2bbfdb6c-0709-4fe4-8a4a-006f576f18e6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 37, 'up_thru': 88, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6804', 'nonce': 3498719258}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6805', 'nonce': 3498719258}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6807', 'nonce': 3498719258}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6806', 'nonce': 3498719258}]}, 'public_addr': '172.21.15.103:6804/3498719258', 'cluster_addr': '172.21.15.103:6805/3498719258', 'heartbeat_back_addr': '172.21.15.103:6807/3498719258', 'heartbeat_front_addr': '172.21.15.103:6806/3498719258', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '31f9711e-0ba8-4aeb-b852-7d93e8fcce33', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 43, 'up_thru': 96, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6808', 'nonce': 184353788}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6809', 'nonce': 184353788}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6811', 'nonce': 184353788}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6810', 'nonce': 184353788}]}, 'public_addr': '172.21.15.103:6808/184353788', 'cluster_addr': '172.21.15.103:6809/184353788', 'heartbeat_back_addr': '172.21.15.103:6811/184353788', 'heartbeat_front_addr': '172.21.15.103:6810/184353788', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'f12282f8-4747-45ff-8336-95440560a88f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 52, 'up_thru': 88, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6812', 'nonce': 941024403}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6813', 'nonce': 941024403}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6815', 'nonce': 941024403}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6814', 'nonce': 941024403}]}, 'public_addr': '172.21.15.103:6812/941024403', 'cluster_addr': '172.21.15.103:6813/941024403', 'heartbeat_back_addr': '172.21.15.103:6815/941024403', 'heartbeat_front_addr': '172.21.15.103:6814/941024403', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T18:47:38.275413+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T18:47:48.859898+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-10T18:46:52.525500+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T18:47:39.084179+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T18:47:31.475999+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T18:46:48.968665+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T18:47:06.337009+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T18:47:38.740068+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '4.e', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.f', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '5.5', 'mappings': [{'from': 0, 'to': 6}]}, {'pgid': '5.10', 'mappings': [{'from': 7, 'to': 3}]}, {'pgid': '6.f', 'mappings': [{'from': 3, 'to': 0}]}], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.18:0/3021537013': '2024-09-11T18:45:54.998616+0000', '172.21.15.18:0/458572898': '2024-09-11T18:40:28.880042+0000', '172.21.15.18:0/1859541419': '2024-09-11T18:40:28.880042+0000', '172.21.15.18:0/3090642970': '2024-09-11T18:40:59.659411+0000', '172.21.15.18:0/2875894312': '2024-09-11T18:40:59.659411+0000', '172.21.15.18:0/178638059': '2024-09-11T18:40:59.659411+0000', '172.21.15.18:0/692931402': '2024-09-11T18:45:54.998616+0000', '172.21.15.18:0/2408360238': '2024-09-11T18:45:54.998616+0000', '172.21.15.18:0/57192865': '2024-09-11T18:40:28.880042+0000', '172.21.15.18:6800/2748756682': '2024-09-11T18:40:59.659411+0000', '172.21.15.18:6800/732375116': '2024-09-11T18:45:54.998616+0000', '172.21.15.18:6800/3869801150': '2024-09-11T18:40:28.880042+0000', '172.21.15.18:0/2982361259': '2024-09-11T18:45:54.998616+0000', '172.21.15.18:0/185700038': '2024-09-11T18:45:54.998616+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-10T18:48:03.062 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T18:48:03.106 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:03.137 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:03.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:03.138 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:03.202 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 19 16 45170 45154 148.498 119.688 0.00401672 0.00672894 2024-09-10T18:48:03.239 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:03.269 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:03.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:03.270 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:03.371 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:03.401 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:03.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:03.402 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:03.503 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:03.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:03.551 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:03.551 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:03.651 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:03.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:03.696 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:03.696 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:03.796 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:03.829 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:03.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:03.830 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:03.931 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:03.973 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:03.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:03.975 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:04.056 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:48:04.056 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:48:04.076 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:04.093 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:48:04.094 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:48:04.106 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:04.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:04.108 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:04.117 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:04 smithi018 ceph-mon[29336]: pgmap v95: 148 pgs: 148 active+clean; 2.5 GiB data, 9.4 GiB used, 616 GiB / 626 GiB avail; 1.4 KiB/s rd, 142 MiB/s wr, 2.27k op/s; 15 B/s, 0 objects/s recovering 2024-09-10T18:48:04.118 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:04 smithi018 ceph-mon[29336]: osdmap e99: 8 total, 8 up, 7 in 2024-09-10T18:48:04.118 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:04 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3948150862' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:48:04.118 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:04 smithi018 ceph-mon[40196]: pgmap v95: 148 pgs: 148 active+clean; 2.5 GiB data, 9.4 GiB used, 616 GiB / 626 GiB avail; 1.4 KiB/s rd, 142 MiB/s wr, 2.27k op/s; 15 B/s, 0 objects/s recovering 2024-09-10T18:48:04.118 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:04 smithi018 ceph-mon[40196]: osdmap e99: 8 total, 8 up, 7 in 2024-09-10T18:48:04.118 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:04 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3948150862' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:48:04.127 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:48:04.127 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:48:04.169 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:48:04.169 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:48:04.202 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:2024-09-10T18:48:04.202228+0000 min lat: 0.00126001 max lat: 0.14754 avg lat: 0.00683391 2024-09-10T18:48:04.203 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T18:48:04.203 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 20 15 46817 46802 146.218 103 0.00645846 0.00683391 2024-09-10T18:48:04.204 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:48:04.204 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:48:04.209 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:04.238 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:48:04.239 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:48:04.252 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:04.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:04.253 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:04.277 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:48:04.278 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:48:04.310 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:48:04.310 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:48:04.355 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:04.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:04.384 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:04.384 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:04.485 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:04.515 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:04.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:04.516 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:04.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:04 smithi103 ceph-mon[33163]: pgmap v95: 148 pgs: 148 active+clean; 2.5 GiB data, 9.4 GiB used, 616 GiB / 626 GiB avail; 1.4 KiB/s rd, 142 MiB/s wr, 2.27k op/s; 15 B/s, 0 objects/s recovering 2024-09-10T18:48:04.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:04 smithi103 ceph-mon[33163]: osdmap e99: 8 total, 8 up, 7 in 2024-09-10T18:48:04.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:04 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3948150862' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:48:04.617 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:04.636 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:48:04.645 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:04.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:04.646 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:04.708 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:48:04.746 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:04.791 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:04.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:04.792 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:04.893 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:04.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:04.938 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:04.939 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:05.040 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:05.078 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:05.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:05.079 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:05.180 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:05.202 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 21 16 48591 48575 144.531 110.812 0.00819332 0.00691389 2024-09-10T18:48:05.219 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:05.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:05.220 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:05.321 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:05.352 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:05.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:05.353 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:05.454 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:05.492 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:05.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:05.493 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:05.594 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:05.631 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:05.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:05.633 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:05.734 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:05.783 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:05.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:05.784 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:05.886 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:05.914 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:05.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:05.916 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:06.017 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:06.070 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:06.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:06.071 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:06.172 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:06.202 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:06.203 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 22 16 50373 50357 143.024 111.375 0.00816861 0.00698676 2024-09-10T18:48:06.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:06.204 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:06.305 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:06.329 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:06 smithi018 ceph-mon[40196]: pgmap v97: 148 pgs: 148 active+clean; 2.5 GiB data, 9.4 GiB used, 616 GiB / 626 GiB avail; 642 B/s rd, 80 MiB/s wr, 1.28k op/s; 13 B/s, 0 objects/s recovering 2024-09-10T18:48:06.329 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:06 smithi018 ceph-mon[29336]: pgmap v97: 148 pgs: 148 active+clean; 2.5 GiB data, 9.4 GiB used, 616 GiB / 626 GiB avail; 642 B/s rd, 80 MiB/s wr, 1.28k op/s; 13 B/s, 0 objects/s recovering 2024-09-10T18:48:06.345 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:06.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:06.346 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:06.447 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:06.477 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:06.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:06.478 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:06.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:06 smithi103 ceph-mon[33163]: pgmap v97: 148 pgs: 148 active+clean; 2.5 GiB data, 9.4 GiB used, 616 GiB / 626 GiB avail; 642 B/s rd, 80 MiB/s wr, 1.28k op/s; 13 B/s, 0 objects/s recovering 2024-09-10T18:48:06.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:48:06 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:48:06.580 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:06.620 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:06.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:06.621 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:06.722 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:06.748 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:06.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:06.749 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:06.850 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:06.881 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:06.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:06.882 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:06.983 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:07.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:07.014 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:07.014 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:07.114 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:07.144 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:07.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:07.145 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:07.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:07 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:07.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:07 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:07.202 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 23 16 52086 52070 141.46 107.062 0.00601841 0.00706343 2024-09-10T18:48:07.246 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:07.274 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:07.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:07.275 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:07.376 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:07.415 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:07.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:07.416 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:07.517 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:07.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:07 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:07.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:07.553 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:07.554 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:07.655 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:07.702 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:07.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:07.703 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:07.804 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:07.831 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T18:48:07.842 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:07.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:07.844 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:07.945 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:07.976 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:07.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:07.978 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:08.078 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:08.109 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:08.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:08.110 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:08.151 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is unset 2024-09-10T18:48:08.203 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 24 16 53655 53639 139.648 98.0625 0.00595228 0.0071561 2024-09-10T18:48:08.211 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:08.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:08.254 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:08.254 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:08.355 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:08.389 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:08.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:08.391 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:08.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:08 smithi018 ceph-mon[29336]: pgmap v98: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1.2 KiB/s rd, 139 MiB/s wr, 2.23k op/s; 12 B/s, 0 objects/s recovering 2024-09-10T18:48:08.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:08 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3009354716' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:48:08.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:08 smithi018 ceph-mon[40196]: pgmap v98: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1.2 KiB/s rd, 139 MiB/s wr, 2.23k op/s; 12 B/s, 0 objects/s recovering 2024-09-10T18:48:08.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:08 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3009354716' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:48:08.491 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:08.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:08 smithi103 ceph-mon[33163]: pgmap v98: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1.2 KiB/s rd, 139 MiB/s wr, 2.23k op/s; 12 B/s, 0 objects/s recovering 2024-09-10T18:48:08.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:08 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3009354716' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:48:08.535 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:08.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:08.537 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:08.637 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:08.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:08.680 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:08.681 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:08.782 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:08.827 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:08.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:08.828 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:08.929 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:08.975 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:08.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:08.976 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:09.076 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:09.115 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:09.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:09.117 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:09.204 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 25 16 55302 55286 138.18 102.938 0.00572334 0.00723199 2024-09-10T18:48:09.218 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:09.248 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:09.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:09.249 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:09.350 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:09.384 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:09.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:09.386 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:09.427 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:09 smithi018 ceph-mon[40196]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:48:09.427 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:09 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3009354716' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:48:09.428 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:09 smithi018 ceph-mon[40196]: osdmap e100: 8 total, 8 up, 7 in 2024-09-10T18:48:09.428 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:09 smithi018 ceph-mon[40196]: osdmap e101: 8 total, 8 up, 7 in 2024-09-10T18:48:09.428 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:09 smithi018 ceph-mon[29336]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:48:09.428 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:09 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3009354716' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:48:09.428 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:09 smithi018 ceph-mon[29336]: osdmap e100: 8 total, 8 up, 7 in 2024-09-10T18:48:09.428 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:09 smithi018 ceph-mon[29336]: osdmap e101: 8 total, 8 up, 7 in 2024-09-10T18:48:09.487 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:09.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:09 smithi103 ceph-mon[33163]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:48:09.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:09 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3009354716' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:48:09.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:09 smithi103 ceph-mon[33163]: osdmap e100: 8 total, 8 up, 7 in 2024-09-10T18:48:09.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:09 smithi103 ceph-mon[33163]: osdmap e101: 8 total, 8 up, 7 in 2024-09-10T18:48:09.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:09.533 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:09.533 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:09.634 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:09.665 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:09.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:09.666 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:09.767 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:09.798 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:09.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:09.799 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:09.900 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:09.956 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:09.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:09.957 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:10.058 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:10.088 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:10.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:10.090 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:10.139 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:10 smithi018 ceph-mon[29336]: 6.3 scrub starts 2024-09-10T18:48:10.139 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:10 smithi018 ceph-mon[29336]: 6.3 scrub ok 2024-09-10T18:48:10.139 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:10 smithi018 ceph-mon[29336]: pgmap v101: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 853 B/s rd, 93 MiB/s wr, 1.48k op/s 2024-09-10T18:48:10.139 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:10 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:48:10.139 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:10 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:48:10.140 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:10 smithi018 ceph-mon[29336]: 5.d scrub starts 2024-09-10T18:48:10.140 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:10 smithi018 ceph-mon[29336]: 5.d scrub ok 2024-09-10T18:48:10.140 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:10 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:48:10.190 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:10.203 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 26 16 56966 56950 136.865 104 0.0149707 0.00730107 2024-09-10T18:48:10.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:10.231 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:10.231 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:10.332 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:10.375 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:10.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:10.377 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:10.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:10 smithi018 ceph-mon[40196]: 6.3 scrub starts 2024-09-10T18:48:10.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:10 smithi018 ceph-mon[40196]: 6.3 scrub ok 2024-09-10T18:48:10.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:10 smithi018 ceph-mon[40196]: pgmap v101: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 853 B/s rd, 93 MiB/s wr, 1.48k op/s 2024-09-10T18:48:10.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:10 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:48:10.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:10 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:48:10.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:10 smithi018 ceph-mon[40196]: 5.d scrub starts 2024-09-10T18:48:10.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:10 smithi018 ceph-mon[40196]: 5.d scrub ok 2024-09-10T18:48:10.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:10 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:48:10.477 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:10.507 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:10.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:10.508 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:10.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:48:10 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:48:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:48:10.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:10 smithi103 ceph-mon[33163]: 6.3 scrub starts 2024-09-10T18:48:10.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:10 smithi103 ceph-mon[33163]: 6.3 scrub ok 2024-09-10T18:48:10.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:10 smithi103 ceph-mon[33163]: pgmap v101: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 853 B/s rd, 93 MiB/s wr, 1.48k op/s 2024-09-10T18:48:10.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:10 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:48:10.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:10 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:48:10.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:10 smithi103 ceph-mon[33163]: 5.d scrub starts 2024-09-10T18:48:10.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:10 smithi103 ceph-mon[33163]: 5.d scrub ok 2024-09-10T18:48:10.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:10 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:48:10.609 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:10.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:10.652 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:10.652 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:10.753 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:10.782 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:10.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:10.783 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:10.884 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:10.916 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:10.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:10.917 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:11.017 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:11.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:11.046 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:11.046 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:11.147 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:11.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:11.179 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:11.179 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:11.205 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 27 16 58473 58457 135.276 94.1875 0.0116711 0.00738536 2024-09-10T18:48:11.241 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:48:10 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:48:10] "GET /metrics HTTP/1.1" 200 223453 "" "Prometheus/2.43.0" 2024-09-10T18:48:11.241 INFO:teuthology.orchestra.run.smithi018.stdout:ERROR: (22) Invalid argument 2024-09-10T18:48:11.241 INFO:teuthology.orchestra.run.smithi018.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:48:11.242 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:11 smithi018 ceph-mon[40196]: 5.10 scrub starts 2024-09-10T18:48:11.242 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:11 smithi018 ceph-mon[40196]: 5.10 scrub ok 2024-09-10T18:48:11.242 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:11 smithi018 ceph-mon[40196]: 5.12 scrub starts 2024-09-10T18:48:11.242 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:11 smithi018 ceph-mon[40196]: 5.12 scrub ok 2024-09-10T18:48:11.242 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:11 smithi018 ceph-mon[40196]: 6.19 scrub starts 2024-09-10T18:48:11.242 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:11 smithi018 ceph-mon[40196]: 6.19 scrub ok 2024-09-10T18:48:11.242 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:11 smithi018 ceph-mon[40196]: osdmap e102: 8 total, 8 up, 7 in 2024-09-10T18:48:11.242 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:11 smithi018 ceph-mon[40196]: 6.1c scrub starts 2024-09-10T18:48:11.242 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:11 smithi018 ceph-mon[40196]: 6.1c scrub ok 2024-09-10T18:48:11.242 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:11 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:48:11.242 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:11 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:48:11.243 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:11 smithi018 ceph-mon[29336]: 5.10 scrub starts 2024-09-10T18:48:11.243 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:11 smithi018 ceph-mon[29336]: 5.10 scrub ok 2024-09-10T18:48:11.243 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:11 smithi018 ceph-mon[29336]: 5.12 scrub starts 2024-09-10T18:48:11.243 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:11 smithi018 ceph-mon[29336]: 5.12 scrub ok 2024-09-10T18:48:11.243 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:11 smithi018 ceph-mon[29336]: 6.19 scrub starts 2024-09-10T18:48:11.243 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:11 smithi018 ceph-mon[29336]: 6.19 scrub ok 2024-09-10T18:48:11.243 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:11 smithi018 ceph-mon[29336]: osdmap e102: 8 total, 8 up, 7 in 2024-09-10T18:48:11.243 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:11 smithi018 ceph-mon[29336]: 6.1c scrub starts 2024-09-10T18:48:11.243 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:11 smithi018 ceph-mon[29336]: 6.1c scrub ok 2024-09-10T18:48:11.243 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:11 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:48:11.243 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:11 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:48:11.280 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:11.311 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:11.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:11.312 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:11.413 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:11.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:11 smithi103 ceph-mon[33163]: 5.10 scrub starts 2024-09-10T18:48:11.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:11 smithi103 ceph-mon[33163]: 5.10 scrub ok 2024-09-10T18:48:11.424 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:11 smithi103 ceph-mon[33163]: 5.12 scrub starts 2024-09-10T18:48:11.424 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:11 smithi103 ceph-mon[33163]: 5.12 scrub ok 2024-09-10T18:48:11.424 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:11 smithi103 ceph-mon[33163]: 6.19 scrub starts 2024-09-10T18:48:11.424 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:11 smithi103 ceph-mon[33163]: 6.19 scrub ok 2024-09-10T18:48:11.424 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:11 smithi103 ceph-mon[33163]: osdmap e102: 8 total, 8 up, 7 in 2024-09-10T18:48:11.424 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:11 smithi103 ceph-mon[33163]: 6.1c scrub starts 2024-09-10T18:48:11.424 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:11 smithi103 ceph-mon[33163]: 6.1c scrub ok 2024-09-10T18:48:11.424 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:11 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:48:11.424 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:11 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:48:11.450 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:48:11.453 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-09-10T18:48:11.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:11.456 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:11.456 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:11.557 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:11.602 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:11.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:11.603 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:11.704 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:11.748 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:11.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:11.749 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:11.849 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:11.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:11.890 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:11.890 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:11.931 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-10T18:48:11.991 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:12.025 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:12.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:12.026 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:12.127 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:12.181 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:12.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:12.182 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:12.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:12 smithi018 ceph-mon[29336]: 4.16 scrub starts 2024-09-10T18:48:12.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:12 smithi018 ceph-mon[29336]: 4.16 scrub ok 2024-09-10T18:48:12.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:12 smithi018 ceph-mon[29336]: pgmap v103: 148 pgs: 148 active+clean; 3.3 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 853 B/s rd, 136 MiB/s wr, 2.17k op/s 2024-09-10T18:48:12.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:12 smithi018 ceph-mon[29336]: osdmap e103: 8 total, 8 up, 7 in 2024-09-10T18:48:12.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:12 smithi018 ceph-mon[40196]: 4.16 scrub starts 2024-09-10T18:48:12.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:12 smithi018 ceph-mon[40196]: 4.16 scrub ok 2024-09-10T18:48:12.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:12 smithi018 ceph-mon[40196]: pgmap v103: 148 pgs: 148 active+clean; 3.3 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 853 B/s rd, 136 MiB/s wr, 2.17k op/s 2024-09-10T18:48:12.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:12 smithi018 ceph-mon[40196]: osdmap e103: 8 total, 8 up, 7 in 2024-09-10T18:48:12.205 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 28 16 59934 59918 133.705 91.3125 0.00628922 0.00747392 2024-09-10T18:48:12.283 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:12.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:12.319 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:12.319 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:12.420 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:12.455 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:12.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:12.456 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:12.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:12 smithi103 ceph-mon[33163]: 4.16 scrub starts 2024-09-10T18:48:12.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:12 smithi103 ceph-mon[33163]: 4.16 scrub ok 2024-09-10T18:48:12.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:12 smithi103 ceph-mon[33163]: pgmap v103: 148 pgs: 148 active+clean; 3.3 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 853 B/s rd, 136 MiB/s wr, 2.17k op/s 2024-09-10T18:48:12.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:12 smithi103 ceph-mon[33163]: osdmap e103: 8 total, 8 up, 7 in 2024-09-10T18:48:12.557 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:12.597 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:12.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:12.598 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:12.699 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:12.745 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:12.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:12.746 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:12.848 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:12.883 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:12.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:12.884 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:12.984 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:13.031 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:13.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:13.032 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:13.133 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:13.183 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:13.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:13.185 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:13.205 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 29 16 61531 61515 132.536 99.8125 0.00531934 0.00753977 2024-09-10T18:48:13.286 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:13.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:13.309 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:13.309 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:13.410 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:13.440 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:13.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:13.441 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:13.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:13 smithi018 ceph-mon[40196]: from='client.24710 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:48:13.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:13 smithi018 ceph-mon[40196]: 4.3 scrub starts 2024-09-10T18:48:13.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:13 smithi018 ceph-mon[40196]: 4.3 scrub ok 2024-09-10T18:48:13.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:13 smithi018 ceph-mon[40196]: 4.e scrub starts 2024-09-10T18:48:13.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:13 smithi018 ceph-mon[40196]: 4.e scrub ok 2024-09-10T18:48:13.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:13 smithi018 ceph-mon[40196]: 2.8 scrub starts 2024-09-10T18:48:13.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:13 smithi018 ceph-mon[40196]: 2.8 scrub ok 2024-09-10T18:48:13.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:13 smithi018 ceph-mon[29336]: from='client.24710 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:48:13.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:13 smithi018 ceph-mon[29336]: 4.3 scrub starts 2024-09-10T18:48:13.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:13 smithi018 ceph-mon[29336]: 4.3 scrub ok 2024-09-10T18:48:13.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:13 smithi018 ceph-mon[29336]: 4.e scrub starts 2024-09-10T18:48:13.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:13 smithi018 ceph-mon[29336]: 4.e scrub ok 2024-09-10T18:48:13.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:13 smithi018 ceph-mon[29336]: 2.8 scrub starts 2024-09-10T18:48:13.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:13 smithi018 ceph-mon[29336]: 2.8 scrub ok 2024-09-10T18:48:13.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:13 smithi103 ceph-mon[33163]: from='client.24710 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:48:13.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:13 smithi103 ceph-mon[33163]: 4.3 scrub starts 2024-09-10T18:48:13.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:13 smithi103 ceph-mon[33163]: 4.3 scrub ok 2024-09-10T18:48:13.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:13 smithi103 ceph-mon[33163]: 4.e scrub starts 2024-09-10T18:48:13.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:13 smithi103 ceph-mon[33163]: 4.e scrub ok 2024-09-10T18:48:13.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:13 smithi103 ceph-mon[33163]: 2.8 scrub starts 2024-09-10T18:48:13.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:13 smithi103 ceph-mon[33163]: 2.8 scrub ok 2024-09-10T18:48:13.542 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:13.574 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:13.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:13.576 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:13.676 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:13.712 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:13.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:13.713 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:13.814 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:13.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:13.856 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:13.857 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:13.931 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:48:13.958 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:13.967 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2024-09-10T18:48:11.049473+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3555137756,"num_objects":54434,"num_object_clones":0,"num_object_copies":163302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54434,"num_whiteouts":0,"num_read":1010,"num_read_kb":867,"num_write":54781,"num_write_kb":3472903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":1183107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54826,"ondisk_log_size":54826,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":12290988,"kb_used_data":10167900,"kb_used_omap":11,"kb_used_meta":2122996,"kb_avail":643925076,"statfs":{"total":671965249536,"available":659379277824,"internally_reserved":0,"allocated":10411929600,"data_stored":10408743187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11928,"internal_metadata":2173948264},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":853213184,"num_objects":13019,"num_object_clones":0,"num_object_copies":39057,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13019,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":13019,"num_write_kb":833216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001672"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":56,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.496906+0000","last_change":"2024-09-10T18:46:33.824057+0000","last_active":"2024-09-10T18:47:58.496906+0000","last_peered":"2024-09-10T18:47:58.496906+0000","last_clean":"2024-09-10T18:47:58.496906+0000","last_became_active":"2024-09-10T18:45:24.265488+0000","last_became_peered":"2024-09-10T18:45:24.265488+0000","last_unstale":"2024-09-10T18:47:58.496906+0000","last_undegraded":"2024-09-10T18:47:58.496906+0000","last_fullsized":"2024-09-10T18:47:58.496906+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:33.823989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:33.823989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034389770000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":62,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.502144+0000","last_change":"2024-09-10T18:47:00.536064+0000","last_active":"2024-09-10T18:47:58.502144+0000","last_peered":"2024-09-10T18:47:58.502144+0000","last_clean":"2024-09-10T18:47:58.502144+0000","last_became_active":"2024-09-10T18:45:17.959475+0000","last_became_peered":"2024-09-10T18:45:17.959475+0000","last_unstale":"2024-09-10T18:47:58.502144+0000","last_undegraded":"2024-09-10T18:47:58.502144+0000","last_fullsized":"2024-09-10T18:47:58.502144+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:00.535992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:47:00.535992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:13.246010+0000","scrub_duration":0.001731841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"63'15","reported_seq":111,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.495829+0000","last_change":"2024-09-10T18:47:36.436893+0000","last_active":"2024-09-10T18:47:58.495829+0000","last_peered":"2024-09-10T18:47:58.495829+0000","last_clean":"2024-09-10T18:47:58.495829+0000","last_became_active":"2024-09-10T18:47:36.413635+0000","last_became_peered":"2024-09-10T18:47:36.413635+0000","last_unstale":"2024-09-10T18:47:58.495829+0000","last_undegraded":"2024-09-10T18:47:58.495829+0000","last_fullsized":"2024-09-10T18:47:58.495829+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:46:36.856740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:36.856740+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018131040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":60,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.506348+0000","last_change":"2024-09-10T18:46:40.313306+0000","last_active":"2024-09-10T18:48:08.506348+0000","last_peered":"2024-09-10T18:48:08.506348+0000","last_clean":"2024-09-10T18:48:08.506348+0000","last_became_active":"2024-09-10T18:45:22.065619+0000","last_became_peered":"2024-09-10T18:45:22.065619+0000","last_unstale":"2024-09-10T18:48:08.506348+0000","last_undegraded":"2024-09-10T18:48:08.506348+0000","last_fullsized":"2024-09-10T18:48:08.506348+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:40.313218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:40.313218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017922750000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":62,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.497891+0000","last_change":"2024-09-10T18:46:18.890844+0000","last_active":"2024-09-10T18:47:58.497891+0000","last_peered":"2024-09-10T18:47:58.497891+0000","last_clean":"2024-09-10T18:47:58.497891+0000","last_became_active":"2024-09-10T18:45:17.966153+0000","last_became_peered":"2024-09-10T18:45:17.966153+0000","last_unstale":"2024-09-10T18:47:58.497891+0000","last_undegraded":"2024-09-10T18:47:58.497891+0000","last_fullsized":"2024-09-10T18:47:58.497891+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:18.890702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:18.890702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019382589999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"83'13","reported_seq":82,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.506502+0000","last_change":"2024-09-10T18:46:59.906707+0000","last_active":"2024-09-10T18:48:08.506502+0000","last_peered":"2024-09-10T18:48:08.506502+0000","last_clean":"2024-09-10T18:48:08.506502+0000","last_became_active":"2024-09-10T18:45:20.061266+0000","last_became_peered":"2024-09-10T18:45:20.061266+0000","last_unstale":"2024-09-10T18:48:08.506502+0000","last_undegraded":"2024-09-10T18:48:08.506502+0000","last_fullsized":"2024-09-10T18:48:08.506502+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'13","last_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_clean_scrub_stamp":"2024-09-10T18:46:59.906634+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:29.738998+0000","scrub_duration":0.003988892,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":58,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.495560+0000","last_change":"2024-09-10T18:47:21.589428+0000","last_active":"2024-09-10T18:47:58.495560+0000","last_peered":"2024-09-10T18:47:58.495560+0000","last_clean":"2024-09-10T18:47:58.495560+0000","last_became_active":"2024-09-10T18:45:22.072822+0000","last_became_peered":"2024-09-10T18:45:22.072822+0000","last_unstale":"2024-09-10T18:47:58.495560+0000","last_undegraded":"2024-09-10T18:47:58.495560+0000","last_fullsized":"2024-09-10T18:47:58.495560+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:21.589365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:47:21.589365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:34.839266+0000","scrub_duration":0.001676893,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":58,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.506236+0000","last_change":"2024-09-10T18:47:00.946309+0000","last_active":"2024-09-10T18:48:08.506236+0000","last_peered":"2024-09-10T18:48:08.506236+0000","last_clean":"2024-09-10T18:48:08.506236+0000","last_became_active":"2024-09-10T18:45:24.081002+0000","last_became_peered":"2024-09-10T18:45:24.081002+0000","last_unstale":"2024-09-10T18:48:08.506236+0000","last_undegraded":"2024-09-10T18:48:08.506236+0000","last_fullsized":"2024-09-10T18:48:08.506236+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:00.946255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:00.946255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:14.872519+0000","scrub_duration":0.001207629,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":65,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.499672+0000","last_change":"2024-09-10T18:46:46.605919+0000","last_active":"2024-09-10T18:48:08.499672+0000","last_peered":"2024-09-10T18:48:08.499672+0000","last_clean":"2024-09-10T18:48:08.499672+0000","last_became_active":"2024-09-10T18:45:17.961097+0000","last_became_peered":"2024-09-10T18:45:17.961097+0000","last_unstale":"2024-09-10T18:48:08.499672+0000","last_undegraded":"2024-09-10T18:48:08.499672+0000","last_fullsized":"2024-09-10T18:48:08.499672+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:46.605814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:46.605814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001570625,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"63'5","reported_seq":72,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.506677+0000","last_change":"2024-09-10T18:47:04.002228+0000","last_active":"2024-09-10T18:48:08.506677+0000","last_peered":"2024-09-10T18:48:08.506677+0000","last_clean":"2024-09-10T18:48:08.506677+0000","last_became_active":"2024-09-10T18:45:20.057567+0000","last_became_peered":"2024-09-10T18:45:20.057567+0000","last_unstale":"2024-09-10T18:48:08.506677+0000","last_undegraded":"2024-09-10T18:48:08.506677+0000","last_fullsized":"2024-09-10T18:48:08.506677+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-10T18:47:04.002164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:04.002164+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:20.148733+0000","scrub_duration":0.026374999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":71,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.498665+0000","last_change":"2024-09-10T18:47:36.510027+0000","last_active":"2024-09-10T18:48:08.498665+0000","last_peered":"2024-09-10T18:48:08.498665+0000","last_clean":"2024-09-10T18:48:08.498665+0000","last_became_active":"2024-09-10T18:45:22.076575+0000","last_became_peered":"2024-09-10T18:45:22.076575+0000","last_unstale":"2024-09-10T18:48:08.498665+0000","last_undegraded":"2024-09-10T18:48:08.498665+0000","last_fullsized":"2024-09-10T18:48:08.498665+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:36.509971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:36.509971+0000","last_clean_scrub_stamp":"2024-09-10T18:47:36.509971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:47.471372+0000","scrub_duration":0.0013665439999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":68,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:10.103050+0000","last_change":"2024-09-10T18:48:10.102991+0000","last_active":"2024-09-10T18:48:10.103050+0000","last_peered":"2024-09-10T18:48:10.103050+0000","last_clean":"2024-09-10T18:48:10.103050+0000","last_became_active":"2024-09-10T18:45:24.079534+0000","last_became_peered":"2024-09-10T18:45:24.079534+0000","last_unstale":"2024-09-10T18:48:10.103050+0000","last_undegraded":"2024-09-10T18:48:10.103050+0000","last_fullsized":"2024-09-10T18:48:10.103050+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:10.102865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:10.102865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:36.015071+0000","scrub_duration":0.0042821509999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":75,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.499391+0000","last_change":"2024-09-10T18:47:36.417065+0000","last_active":"2024-09-10T18:48:08.499391+0000","last_peered":"2024-09-10T18:48:08.499391+0000","last_clean":"2024-09-10T18:48:08.499391+0000","last_became_active":"2024-09-10T18:47:36.414393+0000","last_became_peered":"2024-09-10T18:47:36.414393+0000","last_unstale":"2024-09-10T18:48:08.499391+0000","last_undegraded":"2024-09-10T18:48:08.499391+0000","last_fullsized":"2024-09-10T18:48:08.499391+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:50.666379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.666379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:17.172723+0000","scrub_duration":0.0016221689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"63'9","reported_seq":78,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.506432+0000","last_change":"2024-09-10T18:46:55.069893+0000","last_active":"2024-09-10T18:48:08.506432+0000","last_peered":"2024-09-10T18:48:08.506432+0000","last_clean":"2024-09-10T18:48:08.506432+0000","last_became_active":"2024-09-10T18:45:20.052652+0000","last_became_peered":"2024-09-10T18:45:20.052652+0000","last_unstale":"2024-09-10T18:48:08.506432+0000","last_undegraded":"2024-09-10T18:48:08.506432+0000","last_fullsized":"2024-09-10T18:48:08.506432+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:46:55.069833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:55.069833+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:12.396664+0000","scrub_duration":0.0016685510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":60,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.500648+0000","last_change":"2024-09-10T18:46:31.024322+0000","last_active":"2024-09-10T18:48:08.500648+0000","last_peered":"2024-09-10T18:48:08.500648+0000","last_clean":"2024-09-10T18:48:08.500648+0000","last_became_active":"2024-09-10T18:45:22.068326+0000","last_became_peered":"2024-09-10T18:45:22.068326+0000","last_unstale":"2024-09-10T18:48:08.500648+0000","last_undegraded":"2024-09-10T18:48:08.500648+0000","last_fullsized":"2024-09-10T18:48:08.500648+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:31.024255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:31.024255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00152395,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":56,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.502103+0000","last_change":"2024-09-10T18:47:22.557898+0000","last_active":"2024-09-10T18:47:58.502103+0000","last_peered":"2024-09-10T18:47:58.502103+0000","last_clean":"2024-09-10T18:47:58.502103+0000","last_became_active":"2024-09-10T18:45:24.081488+0000","last_became_peered":"2024-09-10T18:45:24.081488+0000","last_unstale":"2024-09-10T18:47:58.502103+0000","last_undegraded":"2024-09-10T18:47:58.502103+0000","last_fullsized":"2024-09-10T18:47:58.502103+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:22.557837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:22.557837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:37.691769+0000","scrub_duration":0.001691374,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"102'3580","reported_seq":3605,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T18:48:10.995472+0000","last_change":"2024-09-10T18:47:36.423039+0000","last_active":"2024-09-10T18:48:10.995472+0000","last_peered":"2024-09-10T18:48:10.995472+0000","last_clean":"2024-09-10T18:48:10.995472+0000","last_became_active":"2024-09-10T18:47:36.422694+0000","last_became_peered":"2024-09-10T18:47:36.422694+0000","last_unstale":"2024-09-10T18:48:10.995472+0000","last_undegraded":"2024-09-10T18:48:10.995472+0000","last_fullsized":"2024-09-10T18:48:10.995472+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":3580,"log_dups_size":0,"ondisk_log_size":3580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:49.116440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234618880,"num_objects":3580,"num_object_clones":0,"num_object_copies":10740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3580,"num_write_kb":229120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"78'25","reported_seq":126,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.507562+0000","last_change":"2024-09-10T18:47:36.429540+0000","last_active":"2024-09-10T18:48:08.507562+0000","last_peered":"2024-09-10T18:48:08.507562+0000","last_clean":"2024-09-10T18:48:08.507562+0000","last_became_active":"2024-09-10T18:47:36.409300+0000","last_became_peered":"2024-09-10T18:47:36.409300+0000","last_unstale":"2024-09-10T18:48:08.507562+0000","last_undegraded":"2024-09-10T18:48:08.507562+0000","last_fullsized":"2024-09-10T18:48:08.507562+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'25","last_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_deep_scrub":"78'25","last_deep_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_clean_scrub_stamp":"2024-09-10T18:46:44.316146+0000","objects_scrubbed":8,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046415040000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":65,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.499061+0000","last_change":"2024-09-10T18:46:47.595182+0000","last_active":"2024-09-10T18:48:08.499061+0000","last_peered":"2024-09-10T18:48:08.499061+0000","last_clean":"2024-09-10T18:48:08.499061+0000","last_became_active":"2024-09-10T18:45:17.963732+0000","last_became_peered":"2024-09-10T18:45:17.963732+0000","last_unstale":"2024-09-10T18:48:08.499061+0000","last_undegraded":"2024-09-10T18:48:08.499061+0000","last_fullsized":"2024-09-10T18:48:08.499061+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:47.595114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:47.595114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014913249999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":58,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.496422+0000","last_change":"2024-09-10T18:47:04.512478+0000","last_active":"2024-09-10T18:47:58.496422+0000","last_peered":"2024-09-10T18:47:58.496422+0000","last_clean":"2024-09-10T18:47:58.496422+0000","last_became_active":"2024-09-10T18:45:22.071443+0000","last_became_peered":"2024-09-10T18:45:22.071443+0000","last_unstale":"2024-09-10T18:47:58.496422+0000","last_undegraded":"2024-09-10T18:47:58.496422+0000","last_fullsized":"2024-09-10T18:47:58.496422+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:04.512425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:04.512425+0000","last_clean_scrub_stamp":"2024-09-10T18:47:04.512425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:26.715434+0000","scrub_duration":0.001363324,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.501652+0000","last_change":"2024-09-10T18:47:57.104885+0000","last_active":"2024-09-10T18:47:58.501652+0000","last_peered":"2024-09-10T18:47:58.501652+0000","last_clean":"2024-09-10T18:47:58.501652+0000","last_became_active":"2024-09-10T18:47:57.104433+0000","last_became_peered":"2024-09-10T18:47:57.104433+0000","last_unstale":"2024-09-10T18:47:58.501652+0000","last_undegraded":"2024-09-10T18:47:58.501652+0000","last_fullsized":"2024-09-10T18:47:58.501652+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:38.016595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:38.016595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:45.235558+0000","scrub_duration":0.0015035510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.f","version":"99'3154","reported_seq":3173,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T18:48:06.071037+0000","last_change":"2024-09-10T18:47:34.396342+0000","last_active":"2024-09-10T18:48:06.071037+0000","last_peered":"2024-09-10T18:48:06.071037+0000","last_clean":"2024-09-10T18:48:06.071037+0000","last_became_active":"2024-09-10T18:47:34.396087+0000","last_became_peered":"2024-09-10T18:47:34.396087+0000","last_unstale":"2024-09-10T18:48:06.071037+0000","last_undegraded":"2024-09-10T18:48:06.071037+0000","last_fullsized":"2024-09-10T18:48:06.071037+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":3154,"log_dups_size":0,"ondisk_log_size":3154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:46.307943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":206700544,"num_objects":3154,"num_object_clones":0,"num_object_copies":9462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3154,"num_write_kb":201856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"63'10","reported_seq":77,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.507157+0000","last_change":"2024-09-10T18:46:31.281588+0000","last_active":"2024-09-10T18:48:08.507157+0000","last_peered":"2024-09-10T18:48:08.507157+0000","last_clean":"2024-09-10T18:48:08.507157+0000","last_became_active":"2024-09-10T18:45:20.053428+0000","last_became_peered":"2024-09-10T18:45:20.053428+0000","last_unstale":"2024-09-10T18:48:08.507157+0000","last_undegraded":"2024-09-10T18:48:08.507157+0000","last_fullsized":"2024-09-10T18:48:08.507157+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:46:31.281519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:31.281519+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019095119999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":62,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.497036+0000","last_change":"2024-09-10T18:46:38.445417+0000","last_active":"2024-09-10T18:47:58.497036+0000","last_peered":"2024-09-10T18:47:58.497036+0000","last_clean":"2024-09-10T18:47:58.497036+0000","last_became_active":"2024-09-10T18:45:17.966833+0000","last_became_peered":"2024-09-10T18:45:17.966833+0000","last_unstale":"2024-09-10T18:47:58.497036+0000","last_undegraded":"2024-09-10T18:47:58.497036+0000","last_fullsized":"2024-09-10T18:47:58.497036+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:38.445362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:38.445362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012292029999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"89'9","reported_seq":66,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:09.648642+0000","last_change":"2024-09-10T18:48:09.648420+0000","last_active":"2024-09-10T18:48:09.648642+0000","last_peered":"2024-09-10T18:48:09.648642+0000","last_clean":"2024-09-10T18:48:09.648642+0000","last_became_active":"2024-09-10T18:47:36.421188+0000","last_became_peered":"2024-09-10T18:47:36.421188+0000","last_unstale":"2024-09-10T18:48:09.648642+0000","last_undegraded":"2024-09-10T18:48:09.648642+0000","last_fullsized":"2024-09-10T18:48:09.648642+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'9","last_scrub_stamp":"2024-09-10T18:48:09.648322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:09.648322+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:30.082986+0000","scrub_duration":0.0066489619999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":68,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.506461+0000","last_change":"2024-09-10T18:47:36.414985+0000","last_active":"2024-09-10T18:48:08.506461+0000","last_peered":"2024-09-10T18:48:08.506461+0000","last_clean":"2024-09-10T18:48:08.506461+0000","last_became_active":"2024-09-10T18:47:36.414625+0000","last_became_peered":"2024-09-10T18:47:36.414625+0000","last_unstale":"2024-09-10T18:48:08.506461+0000","last_undegraded":"2024-09-10T18:48:08.506461+0000","last_fullsized":"2024-09-10T18:48:08.506461+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:32.264963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:32.264963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002378509,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"102'3672","reported_seq":3695,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T18:48:10.993564+0000","last_change":"2024-09-10T18:47:34.399541+0000","last_active":"2024-09-10T18:48:10.993564+0000","last_peered":"2024-09-10T18:48:10.993564+0000","last_clean":"2024-09-10T18:48:10.993564+0000","last_became_active":"2024-09-10T18:47:34.399261+0000","last_became_peered":"2024-09-10T18:47:34.399261+0000","last_unstale":"2024-09-10T18:48:10.993564+0000","last_undegraded":"2024-09-10T18:48:10.993564+0000","last_fullsized":"2024-09-10T18:48:10.993564+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":3672,"log_dups_size":0,"ondisk_log_size":3672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:45.482983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":240648192,"num_objects":3672,"num_object_clones":0,"num_object_copies":11016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3672,"num_write_kb":235008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"63'15","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.501364+0000","last_change":"2024-09-10T18:46:58.122279+0000","last_active":"2024-09-10T18:48:08.501364+0000","last_peered":"2024-09-10T18:48:08.501364+0000","last_clean":"2024-09-10T18:48:08.501364+0000","last_became_active":"2024-09-10T18:46:56.742946+0000","last_became_peered":"2024-09-10T18:46:56.742946+0000","last_unstale":"2024-09-10T18:48:08.501364+0000","last_undegraded":"2024-09-10T18:48:08.501364+0000","last_fullsized":"2024-09-10T18:48:08.501364+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:46:58.122219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:58.122219+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001678094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":62,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.495732+0000","last_change":"2024-09-10T18:46:28.320761+0000","last_active":"2024-09-10T18:47:58.495732+0000","last_peered":"2024-09-10T18:47:58.495732+0000","last_clean":"2024-09-10T18:47:58.495732+0000","last_became_active":"2024-09-10T18:45:17.965409+0000","last_became_peered":"2024-09-10T18:45:17.965409+0000","last_unstale":"2024-09-10T18:47:58.495732+0000","last_undegraded":"2024-09-10T18:47:58.495732+0000","last_fullsized":"2024-09-10T18:47:58.495732+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_clean_scrub_stamp":"2024-09-10T18:46:28.320704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001393476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"63'8","reported_seq":102,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.506613+0000","last_change":"2024-09-10T18:46:39.300139+0000","last_active":"2024-09-10T18:48:08.506613+0000","last_peered":"2024-09-10T18:48:08.506613+0000","last_clean":"2024-09-10T18:48:08.506613+0000","last_became_active":"2024-09-10T18:45:22.062497+0000","last_became_peered":"2024-09-10T18:45:22.062497+0000","last_unstale":"2024-09-10T18:48:08.506613+0000","last_undegraded":"2024-09-10T18:48:08.506613+0000","last_fullsized":"2024-09-10T18:48:08.506613+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:39.300065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:39.300065+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001858934,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":58,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.501083+0000","last_change":"2024-09-10T18:46:42.057055+0000","last_active":"2024-09-10T18:48:08.501083+0000","last_peered":"2024-09-10T18:48:08.501083+0000","last_clean":"2024-09-10T18:48:08.501083+0000","last_became_active":"2024-09-10T18:45:24.082362+0000","last_became_peered":"2024-09-10T18:45:24.082362+0000","last_unstale":"2024-09-10T18:48:08.501083+0000","last_undegraded":"2024-09-10T18:48:08.501083+0000","last_fullsized":"2024-09-10T18:48:08.501083+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:42.056970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:42.056970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016528479999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"102'3570","reported_seq":3593,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T18:48:10.993763+0000","last_change":"2024-09-10T18:47:34.400040+0000","last_active":"2024-09-10T18:48:10.993763+0000","last_peered":"2024-09-10T18:48:10.993763+0000","last_clean":"2024-09-10T18:48:10.993763+0000","last_became_active":"2024-09-10T18:47:34.399711+0000","last_became_peered":"2024-09-10T18:47:34.399711+0000","last_unstale":"2024-09-10T18:48:10.993763+0000","last_undegraded":"2024-09-10T18:48:10.993763+0000","last_fullsized":"2024-09-10T18:48:10.993763+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":3570,"log_dups_size":0,"ondisk_log_size":3570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:38.820891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233963520,"num_objects":3570,"num_object_clones":0,"num_object_copies":10710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3570,"num_write_kb":228480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"63'11","reported_seq":147,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.506697+0000","last_change":"2024-09-10T18:47:57.120557+0000","last_active":"2024-09-10T18:48:08.506697+0000","last_peered":"2024-09-10T18:48:08.506697+0000","last_clean":"2024-09-10T18:48:08.506697+0000","last_became_active":"2024-09-10T18:47:57.106935+0000","last_became_peered":"2024-09-10T18:47:57.106935+0000","last_unstale":"2024-09-10T18:48:08.506697+0000","last_undegraded":"2024-09-10T18:48:08.506697+0000","last_fullsized":"2024-09-10T18:48:08.506697+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:46:35.274660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:35.274660+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019745819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":65,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.498528+0000","last_change":"2024-09-10T18:46:39.520497+0000","last_active":"2024-09-10T18:48:08.498528+0000","last_peered":"2024-09-10T18:48:08.498528+0000","last_clean":"2024-09-10T18:48:08.498528+0000","last_became_active":"2024-09-10T18:45:17.966133+0000","last_became_peered":"2024-09-10T18:45:17.966133+0000","last_unstale":"2024-09-10T18:48:08.498528+0000","last_undegraded":"2024-09-10T18:48:08.498528+0000","last_fullsized":"2024-09-10T18:48:08.498528+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:39.520432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:39.520432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013982510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":60,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.501449+0000","last_change":"2024-09-10T18:46:48.116414+0000","last_active":"2024-09-10T18:48:08.501449+0000","last_peered":"2024-09-10T18:48:08.501449+0000","last_clean":"2024-09-10T18:48:08.501449+0000","last_became_active":"2024-09-10T18:45:22.068918+0000","last_became_peered":"2024-09-10T18:45:22.068918+0000","last_unstale":"2024-09-10T18:48:08.501449+0000","last_undegraded":"2024-09-10T18:48:08.501449+0000","last_fullsized":"2024-09-10T18:48:08.501449+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_clean_scrub_stamp":"2024-09-10T18:46:48.116358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001429331,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":56,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.495689+0000","last_change":"2024-09-10T18:47:00.743740+0000","last_active":"2024-09-10T18:47:58.495689+0000","last_peered":"2024-09-10T18:47:58.495689+0000","last_clean":"2024-09-10T18:47:58.495689+0000","last_became_active":"2024-09-10T18:45:24.264538+0000","last_became_peered":"2024-09-10T18:45:24.264538+0000","last_unstale":"2024-09-10T18:47:58.495689+0000","last_undegraded":"2024-09-10T18:47:58.495689+0000","last_fullsized":"2024-09-10T18:47:58.495689+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:00.743680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:00.743680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022093759999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2","version":"99'3118","reported_seq":3148,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T18:48:06.083352+0000","last_change":"2024-09-10T18:47:36.423507+0000","last_active":"2024-09-10T18:48:06.083352+0000","last_peered":"2024-09-10T18:48:06.083352+0000","last_clean":"2024-09-10T18:48:06.083352+0000","last_became_active":"2024-09-10T18:47:36.423234+0000","last_became_peered":"2024-09-10T18:47:36.423234+0000","last_unstale":"2024-09-10T18:48:06.083352+0000","last_undegraded":"2024-09-10T18:48:06.083352+0000","last_fullsized":"2024-09-10T18:48:06.083352+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":3118,"log_dups_size":0,"ondisk_log_size":3118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:58.836003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":204341248,"num_objects":3118,"num_object_clones":0,"num_object_copies":9354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3118,"num_write_kb":199552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"63'14","reported_seq":83,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.508170+0000","last_change":"2024-09-10T18:46:29.330572+0000","last_active":"2024-09-10T18:48:08.508170+0000","last_peered":"2024-09-10T18:48:08.508170+0000","last_clean":"2024-09-10T18:48:08.508170+0000","last_became_active":"2024-09-10T18:45:20.052243+0000","last_became_peered":"2024-09-10T18:45:20.052243+0000","last_unstale":"2024-09-10T18:48:08.508170+0000","last_undegraded":"2024-09-10T18:48:08.508170+0000","last_fullsized":"2024-09-10T18:48:08.508170+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-10T18:46:29.330503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:29.330503+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020437459999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":74,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.497911+0000","last_change":"2024-09-10T18:47:36.424885+0000","last_active":"2024-09-10T18:48:08.497911+0000","last_peered":"2024-09-10T18:48:08.497911+0000","last_clean":"2024-09-10T18:48:08.497911+0000","last_became_active":"2024-09-10T18:47:36.424466+0000","last_became_peered":"2024-09-10T18:47:36.424466+0000","last_unstale":"2024-09-10T18:48:08.497911+0000","last_undegraded":"2024-09-10T18:48:08.497911+0000","last_fullsized":"2024-09-10T18:48:08.497911+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_clean_scrub_stamp":"2024-09-10T18:46:25.206731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016589910000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":99,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:48:01.109116+0000","last_change":"2024-09-10T18:46:40.808518+0000","last_active":"2024-09-10T18:48:01.109116+0000","last_peered":"2024-09-10T18:48:01.109116+0000","last_clean":"2024-09-10T18:48:01.109116+0000","last_became_active":"2024-09-10T18:45:22.074540+0000","last_became_peered":"2024-09-10T18:45:22.074540+0000","last_unstale":"2024-09-10T18:48:01.109116+0000","last_undegraded":"2024-09-10T18:48:01.109116+0000","last_fullsized":"2024-09-10T18:48:01.109116+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:40.808453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:40.808453+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016757829999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":79,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.689535+0000","last_change":"2024-09-10T18:48:08.689486+0000","last_active":"2024-09-10T18:48:08.689535+0000","last_peered":"2024-09-10T18:48:08.689535+0000","last_clean":"2024-09-10T18:48:08.689535+0000","last_became_active":"2024-09-10T18:47:36.414665+0000","last_became_peered":"2024-09-10T18:47:36.414665+0000","last_unstale":"2024-09-10T18:48:08.689535+0000","last_undegraded":"2024-09-10T18:48:08.689535+0000","last_fullsized":"2024-09-10T18:48:08.689535+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:08.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:08.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:11.141061+0000","scrub_duration":0.0033094489999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"99'3152","reported_seq":3183,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T18:48:06.050498+0000","last_change":"2024-09-10T18:47:36.410975+0000","last_active":"2024-09-10T18:48:06.050498+0000","last_peered":"2024-09-10T18:48:06.050498+0000","last_clean":"2024-09-10T18:48:06.050498+0000","last_became_active":"2024-09-10T18:47:36.410639+0000","last_became_peered":"2024-09-10T18:47:36.410639+0000","last_unstale":"2024-09-10T18:48:06.050498+0000","last_undegraded":"2024-09-10T18:48:06.050498+0000","last_fullsized":"2024-09-10T18:48:06.050498+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":3152,"log_dups_size":0,"ondisk_log_size":3152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:59.070249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":206569472,"num_objects":3152,"num_object_clones":0,"num_object_copies":9456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3152,"num_write_kb":201728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"63'18","reported_seq":92,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.497312+0000","last_change":"2024-09-10T18:46:27.801177+0000","last_active":"2024-09-10T18:47:58.497312+0000","last_peered":"2024-09-10T18:47:58.497312+0000","last_clean":"2024-09-10T18:47:58.497312+0000","last_became_active":"2024-09-10T18:45:20.420937+0000","last_became_peered":"2024-09-10T18:45:20.420937+0000","last_unstale":"2024-09-10T18:47:58.497312+0000","last_undegraded":"2024-09-10T18:47:58.497312+0000","last_fullsized":"2024-09-10T18:47:58.497312+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'18","last_scrub_stamp":"2024-09-10T18:46:27.801110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:27.801110+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001870095,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":62,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.496406+0000","last_change":"2024-09-10T18:47:05.505431+0000","last_active":"2024-09-10T18:47:58.496406+0000","last_peered":"2024-09-10T18:47:58.496406+0000","last_clean":"2024-09-10T18:47:58.496406+0000","last_became_active":"2024-09-10T18:45:17.961695+0000","last_became_peered":"2024-09-10T18:45:17.961695+0000","last_unstale":"2024-09-10T18:47:58.496406+0000","last_undegraded":"2024-09-10T18:47:58.496406+0000","last_fullsized":"2024-09-10T18:47:58.496406+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:05.505376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:47:05.505376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:32.771817+0000","scrub_duration":0.0013302310000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":60,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.508146+0000","last_change":"2024-09-10T18:46:41.312735+0000","last_active":"2024-09-10T18:48:08.508146+0000","last_peered":"2024-09-10T18:48:08.508146+0000","last_clean":"2024-09-10T18:48:08.508146+0000","last_became_active":"2024-09-10T18:45:22.074012+0000","last_became_peered":"2024-09-10T18:45:22.074012+0000","last_unstale":"2024-09-10T18:48:08.508146+0000","last_undegraded":"2024-09-10T18:48:08.508146+0000","last_fullsized":"2024-09-10T18:48:08.508146+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:41.312671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:41.312671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013475010000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":68,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.506366+0000","last_change":"2024-09-10T18:47:36.411129+0000","last_active":"2024-09-10T18:48:08.506366+0000","last_peered":"2024-09-10T18:48:08.506366+0000","last_clean":"2024-09-10T18:48:08.506366+0000","last_became_active":"2024-09-10T18:47:36.410836+0000","last_became_peered":"2024-09-10T18:47:36.410836+0000","last_unstale":"2024-09-10T18:48:08.506366+0000","last_undegraded":"2024-09-10T18:48:08.506366+0000","last_fullsized":"2024-09-10T18:48:08.506366+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_clean_scrub_stamp":"2024-09-10T18:46:53.087539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:21.019651+0000","scrub_duration":0.001678971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"102'3595","reported_seq":3618,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T18:48:11.008816+0000","last_change":"2024-09-10T18:47:34.399316+0000","last_active":"2024-09-10T18:48:11.008816+0000","last_peered":"2024-09-10T18:48:11.008816+0000","last_clean":"2024-09-10T18:48:11.008816+0000","last_became_active":"2024-09-10T18:47:34.399002+0000","last_became_peered":"2024-09-10T18:47:34.399002+0000","last_unstale":"2024-09-10T18:48:11.008816+0000","last_undegraded":"2024-09-10T18:48:11.008816+0000","last_fullsized":"2024-09-10T18:48:11.008816+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":3595,"log_dups_size":0,"ondisk_log_size":3595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:01.449088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":235601920,"num_objects":3595,"num_object_clones":0,"num_object_copies":10785,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3595,"num_write_kb":230080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3","version":"63'19","reported_seq":96,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.501623+0000","last_change":"2024-09-10T18:46:42.524291+0000","last_active":"2024-09-10T18:47:58.501623+0000","last_peered":"2024-09-10T18:47:58.501623+0000","last_clean":"2024-09-10T18:47:58.501623+0000","last_became_active":"2024-09-10T18:45:20.421046+0000","last_became_peered":"2024-09-10T18:45:20.421046+0000","last_unstale":"2024-09-10T18:47:58.501623+0000","last_undegraded":"2024-09-10T18:47:58.501623+0000","last_fullsized":"2024-09-10T18:47:58.501623+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:46:42.524195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:42.524195+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022840299999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":65,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.499681+0000","last_change":"2024-09-10T18:47:00.592359+0000","last_active":"2024-09-10T18:48:08.499681+0000","last_peered":"2024-09-10T18:48:08.499681+0000","last_clean":"2024-09-10T18:48:08.499681+0000","last_became_active":"2024-09-10T18:45:17.960226+0000","last_became_peered":"2024-09-10T18:45:17.960226+0000","last_unstale":"2024-09-10T18:48:08.499681+0000","last_undegraded":"2024-09-10T18:48:08.499681+0000","last_fullsized":"2024-09-10T18:48:08.499681+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:00.592279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:47:00.592279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001418576,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":60,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.496146+0000","last_change":"2024-09-10T18:46:52.235456+0000","last_active":"2024-09-10T18:48:08.496146+0000","last_peered":"2024-09-10T18:48:08.496146+0000","last_clean":"2024-09-10T18:48:08.496146+0000","last_became_active":"2024-09-10T18:45:22.066148+0000","last_became_peered":"2024-09-10T18:45:22.066148+0000","last_unstale":"2024-09-10T18:48:08.496146+0000","last_undegraded":"2024-09-10T18:48:08.496146+0000","last_fullsized":"2024-09-10T18:48:08.496146+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:52.235389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:52.235389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001679675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":66,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.495492+0000","last_change":"2024-09-10T18:47:36.422607+0000","last_active":"2024-09-10T18:47:58.495492+0000","last_peered":"2024-09-10T18:47:58.495492+0000","last_clean":"2024-09-10T18:47:58.495492+0000","last_became_active":"2024-09-10T18:47:36.422295+0000","last_became_peered":"2024-09-10T18:47:36.422295+0000","last_unstale":"2024-09-10T18:47:58.495492+0000","last_undegraded":"2024-09-10T18:47:58.495492+0000","last_fullsized":"2024-09-10T18:47:58.495492+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:25.525085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:25.525085+0000","last_clean_scrub_stamp":"2024-09-10T18:47:25.525085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:27.418679+0000","scrub_duration":0.0012481079999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"102'3617","reported_seq":3640,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T18:48:11.018506+0000","last_change":"2024-09-10T18:47:34.397576+0000","last_active":"2024-09-10T18:48:11.018506+0000","last_peered":"2024-09-10T18:48:11.018506+0000","last_clean":"2024-09-10T18:48:11.018506+0000","last_became_active":"2024-09-10T18:47:34.396631+0000","last_became_peered":"2024-09-10T18:47:34.396631+0000","last_unstale":"2024-09-10T18:48:11.018506+0000","last_undegraded":"2024-09-10T18:48:11.018506+0000","last_fullsized":"2024-09-10T18:48:11.018506+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":3617,"log_dups_size":0,"ondisk_log_size":3617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:34.954853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":237043712,"num_objects":3617,"num_object_clones":0,"num_object_copies":10851,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3617,"num_write_kb":231488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"83'15","reported_seq":80,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.496746+0000","last_change":"2024-09-10T18:46:58.436565+0000","last_active":"2024-09-10T18:47:58.496746+0000","last_peered":"2024-09-10T18:47:58.496746+0000","last_clean":"2024-09-10T18:47:58.496746+0000","last_became_active":"2024-09-10T18:45:20.057204+0000","last_became_peered":"2024-09-10T18:45:20.057204+0000","last_unstale":"2024-09-10T18:47:58.496746+0000","last_undegraded":"2024-09-10T18:47:58.496746+0000","last_fullsized":"2024-09-10T18:47:58.496746+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'15","last_scrub_stamp":"2024-09-10T18:46:58.436473+0000","last_deep_scrub":"83'15","last_deep_scrub_stamp":"2024-09-10T18:46:58.436473+0000","last_clean_scrub_stamp":"2024-09-10T18:46:58.436473+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046048119999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":62,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.496646+0000","last_change":"2024-09-10T18:46:50.443101+0000","last_active":"2024-09-10T18:47:58.496646+0000","last_peered":"2024-09-10T18:47:58.496646+0000","last_clean":"2024-09-10T18:47:58.496646+0000","last_became_active":"2024-09-10T18:45:17.966632+0000","last_became_peered":"2024-09-10T18:45:17.966632+0000","last_unstale":"2024-09-10T18:47:58.496646+0000","last_undegraded":"2024-09-10T18:47:58.496646+0000","last_fullsized":"2024-09-10T18:47:58.496646+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:50.443042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.443042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:18.989766+0000","scrub_duration":0.0014088550000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"63'8","reported_seq":99,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:48:01.109961+0000","last_change":"2024-09-10T18:47:03.579260+0000","last_active":"2024-09-10T18:48:01.109961+0000","last_peered":"2024-09-10T18:48:01.109961+0000","last_clean":"2024-09-10T18:48:01.109961+0000","last_became_active":"2024-09-10T18:45:22.063326+0000","last_became_peered":"2024-09-10T18:45:22.063326+0000","last_unstale":"2024-09-10T18:48:01.109961+0000","last_undegraded":"2024-09-10T18:48:01.109961+0000","last_fullsized":"2024-09-10T18:48:01.109961+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:47:03.579186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:47:03.579186+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:22.186155+0000","scrub_duration":0.0021415639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":66,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.501937+0000","last_change":"2024-09-10T18:47:36.412408+0000","last_active":"2024-09-10T18:47:58.501937+0000","last_peered":"2024-09-10T18:47:58.501937+0000","last_clean":"2024-09-10T18:47:58.501937+0000","last_became_active":"2024-09-10T18:47:36.412111+0000","last_became_peered":"2024-09-10T18:47:36.412111+0000","last_unstale":"2024-09-10T18:47:58.501937+0000","last_undegraded":"2024-09-10T18:47:58.501937+0000","last_fullsized":"2024-09-10T18:47:58.501937+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:13.488525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:13.488525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:32.804683+0000","scrub_duration":0.001590344,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"102'3680","reported_seq":3702,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T18:48:11.016927+0000","last_change":"2024-09-10T18:47:34.400344+0000","last_active":"2024-09-10T18:48:11.016927+0000","last_peered":"2024-09-10T18:48:11.016927+0000","last_clean":"2024-09-10T18:48:11.016927+0000","last_became_active":"2024-09-10T18:47:34.400044+0000","last_became_peered":"2024-09-10T18:47:34.400044+0000","last_unstale":"2024-09-10T18:48:11.016927+0000","last_undegraded":"2024-09-10T18:48:11.016927+0000","last_fullsized":"2024-09-10T18:48:11.016927+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":3680,"log_dups_size":0,"ondisk_log_size":3680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:34.518578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":241172480,"num_objects":3680,"num_object_clones":0,"num_object_copies":11040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3680,"num_write_kb":235520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"89'29","reported_seq":183,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:48:01.109592+0000","last_change":"2024-09-10T18:47:36.443205+0000","last_active":"2024-09-10T18:48:01.109592+0000","last_peered":"2024-09-10T18:48:01.109592+0000","last_clean":"2024-09-10T18:48:01.109592+0000","last_became_active":"2024-09-10T18:47:36.420185+0000","last_became_peered":"2024-09-10T18:47:36.420185+0000","last_unstale":"2024-09-10T18:48:01.109592+0000","last_undegraded":"2024-09-10T18:48:01.109592+0000","last_fullsized":"2024-09-10T18:48:01.109592+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'28","last_scrub_stamp":"2024-09-10T18:47:26.525861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:26.525861+0000","objects_scrubbed":10,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:49.381758+0000","scrub_duration":0.002126719,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":64,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.500626+0000","last_change":"2024-09-10T18:46:49.120758+0000","last_active":"2024-09-10T18:48:08.500626+0000","last_peered":"2024-09-10T18:48:08.500626+0000","last_clean":"2024-09-10T18:48:08.500626+0000","last_became_active":"2024-09-10T18:45:17.964494+0000","last_became_peered":"2024-09-10T18:45:17.964494+0000","last_unstale":"2024-09-10T18:48:08.500626+0000","last_undegraded":"2024-09-10T18:48:08.500626+0000","last_fullsized":"2024-09-10T18:48:08.500626+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:49.120704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:49.120704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0011863380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.496556+0000","last_change":"2024-09-10T18:47:57.108424+0000","last_active":"2024-09-10T18:48:08.496556+0000","last_peered":"2024-09-10T18:48:08.496556+0000","last_clean":"2024-09-10T18:48:08.496556+0000","last_became_active":"2024-09-10T18:47:57.107478+0000","last_became_peered":"2024-09-10T18:47:57.107478+0000","last_unstale":"2024-09-10T18:48:08.496556+0000","last_undegraded":"2024-09-10T18:48:08.496556+0000","last_fullsized":"2024-09-10T18:48:08.496556+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:58.523217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:58.523217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:13.604149+0000","scrub_duration":0.001307856,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":56,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.495626+0000","last_change":"2024-09-10T18:47:22.891142+0000","last_active":"2024-09-10T18:47:58.495626+0000","last_peered":"2024-09-10T18:47:58.495626+0000","last_clean":"2024-09-10T18:47:58.495626+0000","last_became_active":"2024-09-10T18:45:24.073307+0000","last_became_peered":"2024-09-10T18:45:24.073307+0000","last_unstale":"2024-09-10T18:47:58.495626+0000","last_undegraded":"2024-09-10T18:47:58.495626+0000","last_fullsized":"2024-09-10T18:47:58.495626+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:22.891084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:22.891084+0000","last_clean_scrub_stamp":"2024-09-10T18:47:22.891084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:24.267347+0000","scrub_duration":0.0017788579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"99'3107","reported_seq":3128,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T18:48:06.075720+0000","last_change":"2024-09-10T18:47:34.395543+0000","last_active":"2024-09-10T18:48:06.075720+0000","last_peered":"2024-09-10T18:48:06.075720+0000","last_clean":"2024-09-10T18:48:06.075720+0000","last_became_active":"2024-09-10T18:47:34.395271+0000","last_became_peered":"2024-09-10T18:47:34.395271+0000","last_unstale":"2024-09-10T18:48:06.075720+0000","last_undegraded":"2024-09-10T18:48:06.075720+0000","last_fullsized":"2024-09-10T18:48:06.075720+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":3107,"log_dups_size":0,"ondisk_log_size":3107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:39.605507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":203620352,"num_objects":3107,"num_object_clones":0,"num_object_copies":9321,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3107,"num_write_kb":198848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"84'18","reported_seq":92,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.495219+0000","last_change":"2024-09-10T18:47:10.515060+0000","last_active":"2024-09-10T18:47:58.495219+0000","last_peered":"2024-09-10T18:47:58.495219+0000","last_clean":"2024-09-10T18:47:58.495219+0000","last_became_active":"2024-09-10T18:45:20.058435+0000","last_became_peered":"2024-09-10T18:45:20.058435+0000","last_unstale":"2024-09-10T18:47:58.495219+0000","last_undegraded":"2024-09-10T18:47:58.495219+0000","last_fullsized":"2024-09-10T18:47:58.495219+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'18","last_scrub_stamp":"2024-09-10T18:47:10.514951+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-10T18:47:10.514951+0000","last_clean_scrub_stamp":"2024-09-10T18:47:10.514951+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:24.824689+0000","scrub_duration":0.005567575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"84'6","reported_seq":244,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T18:48:04.460955+0000","last_change":"2024-09-10T18:47:36.439529+0000","last_active":"2024-09-10T18:48:04.460955+0000","last_peered":"2024-09-10T18:48:04.460955+0000","last_clean":"2024-09-10T18:48:04.460955+0000","last_became_active":"2024-09-10T18:47:36.418211+0000","last_became_peered":"2024-09-10T18:47:36.418211+0000","last_unstale":"2024-09-10T18:48:04.460955+0000","last_undegraded":"2024-09-10T18:48:04.460955+0000","last_fullsized":"2024-09-10T18:48:04.460955+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'6","last_scrub_stamp":"2024-09-10T18:47:22.545450+0000","last_deep_scrub":"84'6","last_deep_scrub_stamp":"2024-09-10T18:47:22.545450+0000","last_clean_scrub_stamp":"2024-09-10T18:47:22.545450+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:40.352240+0000","scrub_duration":0.0022780040000000001,"objects_trimmed":0,"snaptrim_duration":0.0021933159999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":158,"num_read_kb":153,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.495906+0000","last_change":"2024-09-10T18:47:36.422920+0000","last_active":"2024-09-10T18:47:58.495906+0000","last_peered":"2024-09-10T18:47:58.495906+0000","last_clean":"2024-09-10T18:47:58.495906+0000","last_became_active":"2024-09-10T18:47:36.422641+0000","last_became_peered":"2024-09-10T18:47:36.422641+0000","last_unstale":"2024-09-10T18:47:58.495906+0000","last_undegraded":"2024-09-10T18:47:58.495906+0000","last_fullsized":"2024-09-10T18:47:58.495906+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:46.712269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:46.712269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:06.875886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":25,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.501550+0000","last_change":"2024-09-10T18:47:36.419434+0000","last_active":"2024-09-10T18:48:08.501550+0000","last_peered":"2024-09-10T18:48:08.501550+0000","last_clean":"2024-09-10T18:48:08.501550+0000","last_became_active":"2024-09-10T18:47:36.419081+0000","last_became_peered":"2024-09-10T18:47:36.419081+0000","last_unstale":"2024-09-10T18:48:08.501550+0000","last_undegraded":"2024-09-10T18:48:08.501550+0000","last_fullsized":"2024-09-10T18:48:08.501550+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:41.818827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:41.818827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":59,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.499238+0000","last_change":"2024-09-10T18:46:59.553576+0000","last_active":"2024-09-10T18:48:08.499238+0000","last_peered":"2024-09-10T18:48:08.499238+0000","last_clean":"2024-09-10T18:48:08.499238+0000","last_became_active":"2024-09-10T18:45:24.264617+0000","last_became_peered":"2024-09-10T18:45:24.264617+0000","last_unstale":"2024-09-10T18:48:08.499238+0000","last_undegraded":"2024-09-10T18:48:08.499238+0000","last_fullsized":"2024-09-10T18:48:08.499238+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:59.553505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:59.553505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:29.436053+0000","scrub_duration":0.0018297210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"102'3591","reported_seq":3625,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T18:48:11.006031+0000","last_change":"2024-09-10T18:47:36.417383+0000","last_active":"2024-09-10T18:48:11.006031+0000","last_peered":"2024-09-10T18:48:11.006031+0000","last_clean":"2024-09-10T18:48:11.006031+0000","last_became_active":"2024-09-10T18:47:36.417075+0000","last_became_peered":"2024-09-10T18:47:36.417075+0000","last_unstale":"2024-09-10T18:48:11.006031+0000","last_undegraded":"2024-09-10T18:48:11.006031+0000","last_fullsized":"2024-09-10T18:48:11.006031+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":3591,"log_dups_size":0,"ondisk_log_size":3591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:42.573805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":235339776,"num_objects":3591,"num_object_clones":0,"num_object_copies":10773,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3591,"num_write_kb":229824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"63'12","reported_seq":102,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.502643+0000","last_change":"2024-09-10T18:47:36.439634+0000","last_active":"2024-09-10T18:47:58.502643+0000","last_peered":"2024-09-10T18:47:58.502643+0000","last_clean":"2024-09-10T18:47:58.502643+0000","last_became_active":"2024-09-10T18:47:36.417965+0000","last_became_peered":"2024-09-10T18:47:36.417965+0000","last_unstale":"2024-09-10T18:47:58.502643+0000","last_undegraded":"2024-09-10T18:47:58.502643+0000","last_fullsized":"2024-09-10T18:47:58.502643+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:46:52.472511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:52.472511+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:10.560863+0000","scrub_duration":0.001886935,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"58'1","reported_seq":68,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.501622+0000","last_change":"2024-09-10T18:47:14.490159+0000","last_active":"2024-09-10T18:47:58.501622+0000","last_peered":"2024-09-10T18:47:58.501622+0000","last_clean":"2024-09-10T18:47:58.501622+0000","last_became_active":"2024-09-10T18:45:17.963932+0000","last_became_peered":"2024-09-10T18:45:17.963932+0000","last_unstale":"2024-09-10T18:47:58.501622+0000","last_undegraded":"2024-09-10T18:47:58.501622+0000","last_fullsized":"2024-09-10T18:47:58.501622+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'1","last_scrub_stamp":"2024-09-10T18:47:14.490091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:47:14.490091+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:40.832261+0000","scrub_duration":0.0019528939999999999,"objects_trimmed":0,"snaptrim_duration":0.00037812099999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.498324+0000","last_change":"2024-09-10T18:47:36.417008+0000","last_active":"2024-09-10T18:48:08.498324+0000","last_peered":"2024-09-10T18:48:08.498324+0000","last_clean":"2024-09-10T18:48:08.498324+0000","last_became_active":"2024-09-10T18:47:36.416641+0000","last_became_peered":"2024-09-10T18:47:36.416641+0000","last_unstale":"2024-09-10T18:48:08.498324+0000","last_undegraded":"2024-09-10T18:48:08.498324+0000","last_fullsized":"2024-09-10T18:48:08.498324+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:54.554426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:54.554426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:19.518986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":60,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.501500+0000","last_change":"2024-09-10T18:46:59.132301+0000","last_active":"2024-09-10T18:48:08.501500+0000","last_peered":"2024-09-10T18:48:08.501500+0000","last_clean":"2024-09-10T18:48:08.501500+0000","last_became_active":"2024-09-10T18:45:22.064285+0000","last_became_peered":"2024-09-10T18:45:22.064285+0000","last_unstale":"2024-09-10T18:48:08.501500+0000","last_undegraded":"2024-09-10T18:48:08.501500+0000","last_fullsized":"2024-09-10T18:48:08.501500+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:59.132234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:59.132234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001471034,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":56,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.496052+0000","last_change":"2024-09-10T18:46:49.422912+0000","last_active":"2024-09-10T18:47:58.496052+0000","last_peered":"2024-09-10T18:47:58.496052+0000","last_clean":"2024-09-10T18:47:58.496052+0000","last_became_active":"2024-09-10T18:45:24.083689+0000","last_became_peered":"2024-09-10T18:45:24.083689+0000","last_unstale":"2024-09-10T18:47:58.496052+0000","last_undegraded":"2024-09-10T18:47:58.496052+0000","last_fullsized":"2024-09-10T18:47:58.496052+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:49.422843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:49.422843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014985300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"99'3084","reported_seq":3105,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T18:48:06.044448+0000","last_change":"2024-09-10T18:47:34.398452+0000","last_active":"2024-09-10T18:48:06.044448+0000","last_peered":"2024-09-10T18:48:06.044448+0000","last_clean":"2024-09-10T18:48:06.044448+0000","last_became_active":"2024-09-10T18:47:34.396853+0000","last_became_peered":"2024-09-10T18:47:34.396853+0000","last_unstale":"2024-09-10T18:48:06.044448+0000","last_undegraded":"2024-09-10T18:48:06.044448+0000","last_fullsized":"2024-09-10T18:48:06.044448+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":3084,"log_dups_size":0,"ondisk_log_size":3084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:56.235056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":202113024,"num_objects":3084,"num_object_clones":0,"num_object_copies":9252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3084,"num_write_kb":197376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"63'14","reported_seq":84,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.497017+0000","last_change":"2024-09-10T18:46:22.294141+0000","last_active":"2024-09-10T18:48:08.497017+0000","last_peered":"2024-09-10T18:48:08.497017+0000","last_clean":"2024-09-10T18:48:08.497017+0000","last_became_active":"2024-09-10T18:45:20.421119+0000","last_became_peered":"2024-09-10T18:45:20.421119+0000","last_unstale":"2024-09-10T18:48:08.497017+0000","last_undegraded":"2024-09-10T18:48:08.497017+0000","last_fullsized":"2024-09-10T18:48:08.497017+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-10T18:46:22.294072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:22.294072+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021435069999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"60'2","reported_seq":69,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.498414+0000","last_change":"2024-09-10T18:46:25.845131+0000","last_active":"2024-09-10T18:47:58.498414+0000","last_peered":"2024-09-10T18:47:58.498414+0000","last_clean":"2024-09-10T18:47:58.498414+0000","last_became_active":"2024-09-10T18:45:17.961660+0000","last_became_peered":"2024-09-10T18:45:17.961660+0000","last_unstale":"2024-09-10T18:47:58.498414+0000","last_undegraded":"2024-09-10T18:47:58.498414+0000","last_fullsized":"2024-09-10T18:47:58.498414+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-10T18:46:25.845070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:25.845070+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020129710000000001,"objects_trimmed":0,"snaptrim_duration":0.0013942640000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":74,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.500840+0000","last_change":"2024-09-10T18:47:36.415006+0000","last_active":"2024-09-10T18:48:08.500840+0000","last_peered":"2024-09-10T18:48:08.500840+0000","last_clean":"2024-09-10T18:48:08.500840+0000","last_became_active":"2024-09-10T18:47:36.414636+0000","last_became_peered":"2024-09-10T18:47:36.414636+0000","last_unstale":"2024-09-10T18:48:08.500840+0000","last_undegraded":"2024-09-10T18:48:08.500840+0000","last_fullsized":"2024-09-10T18:48:08.500840+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:29.081296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:29.081296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015488590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'81","reported_seq":109,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.498615+0000","last_change":"2024-09-10T18:47:05.558031+0000","last_active":"2024-09-10T18:48:08.498615+0000","last_peered":"2024-09-10T18:48:08.498615+0000","last_clean":"2024-09-10T18:48:08.498615+0000","last_became_active":"2024-09-10T18:45:07.433592+0000","last_became_peered":"2024-09-10T18:45:07.433592+0000","last_unstale":"2024-09-10T18:48:08.498615+0000","last_undegraded":"2024-09-10T18:48:08.498615+0000","last_fullsized":"2024-09-10T18:48:08.498615+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:47:05.557968+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:47:05.557968+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:22.825675+0000","scrub_duration":0.0015062890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":81,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.499196+0000","last_change":"2024-09-10T18:47:36.418948+0000","last_active":"2024-09-10T18:48:08.499196+0000","last_peered":"2024-09-10T18:48:08.499196+0000","last_clean":"2024-09-10T18:48:08.499196+0000","last_became_active":"2024-09-10T18:47:36.418320+0000","last_became_peered":"2024-09-10T18:47:36.418320+0000","last_unstale":"2024-09-10T18:48:08.499196+0000","last_undegraded":"2024-09-10T18:48:08.499196+0000","last_fullsized":"2024-09-10T18:48:08.499196+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:32.528904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:47:32.528904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:32.986798+0000","scrub_duration":0.0037979899999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":58,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.501625+0000","last_change":"2024-09-10T18:46:56.066558+0000","last_active":"2024-09-10T18:48:08.501625+0000","last_peered":"2024-09-10T18:48:08.501625+0000","last_clean":"2024-09-10T18:48:08.501625+0000","last_became_active":"2024-09-10T18:45:24.073327+0000","last_became_peered":"2024-09-10T18:45:24.073327+0000","last_unstale":"2024-09-10T18:48:08.501625+0000","last_undegraded":"2024-09-10T18:48:08.501625+0000","last_fullsized":"2024-09-10T18:48:08.501625+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:56.066502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:56.066502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0011960810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.9","version":"99'3224","reported_seq":3245,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T18:48:06.033133+0000","last_change":"2024-09-10T18:47:34.393877+0000","last_active":"2024-09-10T18:48:06.033133+0000","last_peered":"2024-09-10T18:48:06.033133+0000","last_clean":"2024-09-10T18:48:06.033133+0000","last_became_active":"2024-09-10T18:47:34.393620+0000","last_became_peered":"2024-09-10T18:47:34.393620+0000","last_unstale":"2024-09-10T18:48:06.033133+0000","last_undegraded":"2024-09-10T18:48:06.033133+0000","last_fullsized":"2024-09-10T18:48:06.033133+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":3224,"log_dups_size":0,"ondisk_log_size":3224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:46.971656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":211288064,"num_objects":3224,"num_object_clones":0,"num_object_copies":9672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3224,"num_write_kb":206336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"63'19","reported_seq":93,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.496329+0000","last_change":"2024-09-10T18:46:28.229804+0000","last_active":"2024-09-10T18:48:08.496329+0000","last_peered":"2024-09-10T18:48:08.496329+0000","last_clean":"2024-09-10T18:48:08.496329+0000","last_became_active":"2024-09-10T18:45:20.421097+0000","last_became_peered":"2024-09-10T18:45:20.421097+0000","last_unstale":"2024-09-10T18:48:08.496329+0000","last_undegraded":"2024-09-10T18:48:08.496329+0000","last_fullsized":"2024-09-10T18:48:08.496329+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:46:28.229737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:28.229737+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002308184,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":25,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.501026+0000","last_change":"2024-09-10T18:47:36.418124+0000","last_active":"2024-09-10T18:48:08.501026+0000","last_peered":"2024-09-10T18:48:08.501026+0000","last_clean":"2024-09-10T18:48:08.501026+0000","last_became_active":"2024-09-10T18:47:36.417800+0000","last_became_peered":"2024-09-10T18:47:36.417800+0000","last_unstale":"2024-09-10T18:48:08.501026+0000","last_undegraded":"2024-09-10T18:48:08.501026+0000","last_fullsized":"2024-09-10T18:48:08.501026+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:53.707694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:53.707694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":23,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.501356+0000","last_change":"2024-09-10T18:47:36.414024+0000","last_active":"2024-09-10T18:47:58.501356+0000","last_peered":"2024-09-10T18:47:58.501356+0000","last_clean":"2024-09-10T18:47:58.501356+0000","last_became_active":"2024-09-10T18:47:36.413618+0000","last_became_peered":"2024-09-10T18:47:36.413618+0000","last_unstale":"2024-09-10T18:47:58.501356+0000","last_undegraded":"2024-09-10T18:47:58.501356+0000","last_fullsized":"2024-09-10T18:47:58.501356+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:30.781730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:30.781730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":69,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.498579+0000","last_change":"2024-09-10T18:47:36.413013+0000","last_active":"2024-09-10T18:48:08.498579+0000","last_peered":"2024-09-10T18:48:08.498579+0000","last_clean":"2024-09-10T18:48:08.498579+0000","last_became_active":"2024-09-10T18:47:36.412610+0000","last_became_peered":"2024-09-10T18:47:36.412610+0000","last_unstale":"2024-09-10T18:48:08.498579+0000","last_undegraded":"2024-09-10T18:48:08.498579+0000","last_fullsized":"2024-09-10T18:48:08.498579+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:03.587252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:03.587252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:25.266841+0000","scrub_duration":0.001684867,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"102'3709","reported_seq":3735,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T18:48:11.013890+0000","last_change":"2024-09-10T18:47:36.413347+0000","last_active":"2024-09-10T18:48:11.013890+0000","last_peered":"2024-09-10T18:48:11.013890+0000","last_clean":"2024-09-10T18:48:11.013890+0000","last_became_active":"2024-09-10T18:47:36.412142+0000","last_became_peered":"2024-09-10T18:47:36.412142+0000","last_unstale":"2024-09-10T18:48:11.013890+0000","last_undegraded":"2024-09-10T18:48:11.013890+0000","last_fullsized":"2024-09-10T18:48:11.013890+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":3709,"log_dups_size":0,"ondisk_log_size":3709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:59.312878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243073024,"num_objects":3709,"num_object_clones":0,"num_object_copies":11127,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3709,"num_write_kb":237376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"63'9","reported_seq":76,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.501547+0000","last_change":"2024-09-10T18:46:51.486491+0000","last_active":"2024-09-10T18:47:58.501547+0000","last_peered":"2024-09-10T18:47:58.501547+0000","last_clean":"2024-09-10T18:47:58.501547+0000","last_became_active":"2024-09-10T18:45:20.061021+0000","last_became_peered":"2024-09-10T18:45:20.061021+0000","last_unstale":"2024-09-10T18:47:58.501547+0000","last_undegraded":"2024-09-10T18:47:58.501547+0000","last_fullsized":"2024-09-10T18:47:58.501547+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:46:51.486428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:51.486428+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:05.602014+0000","scrub_duration":0.0019382830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":62,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.497956+0000","last_change":"2024-09-10T18:47:02.496271+0000","last_active":"2024-09-10T18:47:58.497956+0000","last_peered":"2024-09-10T18:47:58.497956+0000","last_clean":"2024-09-10T18:47:58.497956+0000","last_became_active":"2024-09-10T18:45:17.963057+0000","last_became_peered":"2024-09-10T18:45:17.963057+0000","last_unstale":"2024-09-10T18:47:58.497956+0000","last_undegraded":"2024-09-10T18:47:58.497956+0000","last_fullsized":"2024-09-10T18:47:58.497956+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:02.496142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:47:02.496142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:10.459026+0000","scrub_duration":0.001628399,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":25,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.506540+0000","last_change":"2024-09-10T18:47:36.411830+0000","last_active":"2024-09-10T18:48:08.506540+0000","last_peered":"2024-09-10T18:48:08.506540+0000","last_clean":"2024-09-10T18:48:08.506540+0000","last_became_active":"2024-09-10T18:47:36.411517+0000","last_became_peered":"2024-09-10T18:47:36.411517+0000","last_unstale":"2024-09-10T18:48:08.506540+0000","last_undegraded":"2024-09-10T18:48:08.506540+0000","last_fullsized":"2024-09-10T18:48:08.506540+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_clean_scrub_stamp":"2024-09-10T18:46:55.684250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":66,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.501546+0000","last_change":"2024-09-10T18:47:36.413221+0000","last_active":"2024-09-10T18:47:58.501546+0000","last_peered":"2024-09-10T18:47:58.501546+0000","last_clean":"2024-09-10T18:47:58.501546+0000","last_became_active":"2024-09-10T18:47:36.412933+0000","last_became_peered":"2024-09-10T18:47:36.412933+0000","last_unstale":"2024-09-10T18:47:58.501546+0000","last_undegraded":"2024-09-10T18:47:58.501546+0000","last_fullsized":"2024-09-10T18:47:58.501546+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:47.545343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:47.545343+0000","last_clean_scrub_stamp":"2024-09-10T18:46:47.545343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:09.231915+0000","scrub_duration":0.0012977799999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.b","version":"99'3174","reported_seq":3194,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T18:48:06.077475+0000","last_change":"2024-09-10T18:47:34.398004+0000","last_active":"2024-09-10T18:48:06.077475+0000","last_peered":"2024-09-10T18:48:06.077475+0000","last_clean":"2024-09-10T18:48:06.077475+0000","last_became_active":"2024-09-10T18:47:34.397708+0000","last_became_peered":"2024-09-10T18:47:34.397708+0000","last_unstale":"2024-09-10T18:48:06.077475+0000","last_undegraded":"2024-09-10T18:48:06.077475+0000","last_fullsized":"2024-09-10T18:48:06.077475+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":3174,"log_dups_size":0,"ondisk_log_size":3174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:47.174258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":208011264,"num_objects":3174,"num_object_clones":0,"num_object_copies":9522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3174,"num_write_kb":203136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"63'15","reported_seq":87,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.500247+0000","last_change":"2024-09-10T18:46:51.114543+0000","last_active":"2024-09-10T18:48:08.500247+0000","last_peered":"2024-09-10T18:48:08.500247+0000","last_clean":"2024-09-10T18:48:08.500247+0000","last_became_active":"2024-09-10T18:45:20.419804+0000","last_became_peered":"2024-09-10T18:45:20.419804+0000","last_unstale":"2024-09-10T18:48:08.500247+0000","last_undegraded":"2024-09-10T18:48:08.500247+0000","last_fullsized":"2024-09-10T18:48:08.500247+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:46:51.114455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:51.114455+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:18.982344+0000","scrub_duration":0.0022557979999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":23,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.495764+0000","last_change":"2024-09-10T18:47:36.420713+0000","last_active":"2024-09-10T18:47:58.495764+0000","last_peered":"2024-09-10T18:47:58.495764+0000","last_clean":"2024-09-10T18:47:58.495764+0000","last_became_active":"2024-09-10T18:47:36.420475+0000","last_became_peered":"2024-09-10T18:47:36.420475+0000","last_unstale":"2024-09-10T18:47:58.495764+0000","last_undegraded":"2024-09-10T18:47:58.495764+0000","last_fullsized":"2024-09-10T18:47:58.495764+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:20.774289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:17.819909+0000","last_clean_scrub_stamp":"2024-09-10T18:47:20.774289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:45.742247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"83'9","reported_seq":107,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.498673+0000","last_change":"2024-09-10T18:47:02.582283+0000","last_active":"2024-09-10T18:48:08.498673+0000","last_peered":"2024-09-10T18:48:08.498673+0000","last_clean":"2024-09-10T18:48:08.498673+0000","last_became_active":"2024-09-10T18:45:22.073383+0000","last_became_peered":"2024-09-10T18:45:22.073383+0000","last_unstale":"2024-09-10T18:48:08.498673+0000","last_undegraded":"2024-09-10T18:48:08.498673+0000","last_fullsized":"2024-09-10T18:48:08.498673+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'9","last_scrub_stamp":"2024-09-10T18:47:02.582229+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-09-10T18:47:02.582229+0000","last_clean_scrub_stamp":"2024-09-10T18:47:02.582229+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:10.931107+0000","scrub_duration":0.0020623740000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":58,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.500389+0000","last_change":"2024-09-10T18:46:45.083522+0000","last_active":"2024-09-10T18:48:08.500389+0000","last_peered":"2024-09-10T18:48:08.500389+0000","last_clean":"2024-09-10T18:48:08.500389+0000","last_became_active":"2024-09-10T18:45:24.265256+0000","last_became_peered":"2024-09-10T18:45:24.265256+0000","last_unstale":"2024-09-10T18:48:08.500389+0000","last_undegraded":"2024-09-10T18:48:08.500389+0000","last_fullsized":"2024-09-10T18:48:08.500389+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_clean_scrub_stamp":"2024-09-10T18:46:45.083425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017589890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.a","version":"99'3211","reported_seq":3242,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T18:48:06.069329+0000","last_change":"2024-09-10T18:47:36.423957+0000","last_active":"2024-09-10T18:48:06.069329+0000","last_peered":"2024-09-10T18:48:06.069329+0000","last_clean":"2024-09-10T18:48:06.069329+0000","last_became_active":"2024-09-10T18:47:36.422868+0000","last_became_peered":"2024-09-10T18:47:36.422868+0000","last_unstale":"2024-09-10T18:48:06.069329+0000","last_undegraded":"2024-09-10T18:48:06.069329+0000","last_fullsized":"2024-09-10T18:48:06.069329+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":3211,"log_dups_size":0,"ondisk_log_size":3211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:44.562678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":210436096,"num_objects":3211,"num_object_clones":0,"num_object_copies":9633,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3211,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3211,"num_write_kb":205504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"63'12","reported_seq":95,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.508128+0000","last_change":"2024-09-10T18:47:35.928110+0000","last_active":"2024-09-10T18:48:08.508128+0000","last_peered":"2024-09-10T18:48:08.508128+0000","last_clean":"2024-09-10T18:48:08.508128+0000","last_became_active":"2024-09-10T18:45:20.056869+0000","last_became_peered":"2024-09-10T18:45:20.056869+0000","last_unstale":"2024-09-10T18:48:08.508128+0000","last_undegraded":"2024-09-10T18:48:08.508128+0000","last_fullsized":"2024-09-10T18:48:08.508128+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:47:35.928040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:35.928040+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:50.601492+0000","scrub_duration":0.0021332539999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"56'2","reported_seq":81,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.507225+0000","last_change":"2024-09-10T18:46:42.303648+0000","last_active":"2024-09-10T18:48:08.507225+0000","last_peered":"2024-09-10T18:48:08.507225+0000","last_clean":"2024-09-10T18:48:08.507225+0000","last_became_active":"2024-09-10T18:45:17.963286+0000","last_became_peered":"2024-09-10T18:45:17.963286+0000","last_unstale":"2024-09-10T18:48:08.507225+0000","last_undegraded":"2024-09-10T18:48:08.507225+0000","last_fullsized":"2024-09-10T18:48:08.507225+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-10T18:46:42.303567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:42.303567+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020684750000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":23,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.502022+0000","last_change":"2024-09-10T18:47:36.421711+0000","last_active":"2024-09-10T18:47:58.502022+0000","last_peered":"2024-09-10T18:47:58.502022+0000","last_clean":"2024-09-10T18:47:58.502022+0000","last_became_active":"2024-09-10T18:47:36.421151+0000","last_became_peered":"2024-09-10T18:47:36.421151+0000","last_unstale":"2024-09-10T18:47:58.502022+0000","last_undegraded":"2024-09-10T18:47:58.502022+0000","last_fullsized":"2024-09-10T18:47:58.502022+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:44.754286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:44.754286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":56,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.497062+0000","last_change":"2024-09-10T18:47:18.797303+0000","last_active":"2024-09-10T18:47:58.497062+0000","last_peered":"2024-09-10T18:47:58.497062+0000","last_clean":"2024-09-10T18:47:58.497062+0000","last_became_active":"2024-09-10T18:45:24.077815+0000","last_became_peered":"2024-09-10T18:45:24.077815+0000","last_unstale":"2024-09-10T18:47:58.497062+0000","last_undegraded":"2024-09-10T18:47:58.497062+0000","last_fullsized":"2024-09-10T18:47:58.497062+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:18.797245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:18.797245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:19.012885+0000","scrub_duration":0.0015800090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"76'13","reported_seq":80,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.502595+0000","last_change":"2024-09-10T18:46:24.626912+0000","last_active":"2024-09-10T18:47:58.502595+0000","last_peered":"2024-09-10T18:47:58.502595+0000","last_clean":"2024-09-10T18:47:58.502595+0000","last_became_active":"2024-09-10T18:45:20.420421+0000","last_became_peered":"2024-09-10T18:45:20.420421+0000","last_unstale":"2024-09-10T18:47:58.502595+0000","last_undegraded":"2024-09-10T18:47:58.502595+0000","last_fullsized":"2024-09-10T18:47:58.502595+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'13","last_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_deep_scrub":"76'13","last_deep_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_clean_scrub_stamp":"2024-09-10T18:46:24.626859+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0035642949999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":23,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.497337+0000","last_change":"2024-09-10T18:47:36.409670+0000","last_active":"2024-09-10T18:47:58.497337+0000","last_peered":"2024-09-10T18:47:58.497337+0000","last_clean":"2024-09-10T18:47:58.497337+0000","last_became_active":"2024-09-10T18:47:36.409299+0000","last_became_peered":"2024-09-10T18:47:36.409299+0000","last_unstale":"2024-09-10T18:47:58.497337+0000","last_undegraded":"2024-09-10T18:47:58.497337+0000","last_fullsized":"2024-09-10T18:47:58.497337+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:31.734459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:31.734459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":58,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.499449+0000","last_change":"2024-09-10T18:46:32.829583+0000","last_active":"2024-09-10T18:47:58.499449+0000","last_peered":"2024-09-10T18:47:58.499449+0000","last_clean":"2024-09-10T18:47:58.499449+0000","last_became_active":"2024-09-10T18:45:22.069040+0000","last_became_peered":"2024-09-10T18:45:22.069040+0000","last_unstale":"2024-09-10T18:47:58.499449+0000","last_undegraded":"2024-09-10T18:47:58.499449+0000","last_fullsized":"2024-09-10T18:47:58.499449+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:32.829508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:32.829508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001897239,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"63'1","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.506987+0000","last_change":"2024-09-10T18:47:36.423902+0000","last_active":"2024-09-10T18:48:08.506987+0000","last_peered":"2024-09-10T18:48:08.506987+0000","last_clean":"2024-09-10T18:48:08.506987+0000","last_became_active":"2024-09-10T18:47:36.406936+0000","last_became_peered":"2024-09-10T18:47:36.406936+0000","last_unstale":"2024-09-10T18:48:08.506987+0000","last_undegraded":"2024-09-10T18:48:08.506987+0000","last_fullsized":"2024-09-10T18:48:08.506987+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-10T18:46:56.721716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:56.721716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":64,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.496275+0000","last_change":"2024-09-10T18:46:48.227450+0000","last_active":"2024-09-10T18:48:08.496275+0000","last_peered":"2024-09-10T18:48:08.496275+0000","last_clean":"2024-09-10T18:48:08.496275+0000","last_became_active":"2024-09-10T18:45:17.964705+0000","last_became_peered":"2024-09-10T18:45:17.964705+0000","last_unstale":"2024-09-10T18:48:08.496275+0000","last_undegraded":"2024-09-10T18:48:08.496275+0000","last_fullsized":"2024-09-10T18:48:08.496275+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:48.227378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:48.227378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015447519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'6","reported_seq":69,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.499704+0000","last_change":"2024-09-10T18:47:02.700853+0000","last_active":"2024-09-10T18:47:58.499704+0000","last_peered":"2024-09-10T18:47:58.499704+0000","last_clean":"2024-09-10T18:47:58.499704+0000","last_became_active":"2024-09-10T18:45:20.421746+0000","last_became_peered":"2024-09-10T18:45:20.421746+0000","last_unstale":"2024-09-10T18:47:58.499704+0000","last_undegraded":"2024-09-10T18:47:58.499704+0000","last_fullsized":"2024-09-10T18:47:58.499704+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'6","last_scrub_stamp":"2024-09-10T18:47:02.700787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:02.700787+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:10.576660+0000","scrub_duration":0.0017032519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":60,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.501435+0000","last_change":"2024-09-10T18:46:39.084489+0000","last_active":"2024-09-10T18:48:08.501435+0000","last_peered":"2024-09-10T18:48:08.501435+0000","last_clean":"2024-09-10T18:48:08.501435+0000","last_became_active":"2024-09-10T18:45:22.069653+0000","last_became_peered":"2024-09-10T18:45:22.069653+0000","last_unstale":"2024-09-10T18:48:08.501435+0000","last_undegraded":"2024-09-10T18:48:08.501435+0000","last_fullsized":"2024-09-10T18:48:08.501435+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:39.084433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:39.084433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014845819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":59,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.499164+0000","last_change":"2024-09-10T18:46:52.611074+0000","last_active":"2024-09-10T18:48:08.499164+0000","last_peered":"2024-09-10T18:48:08.499164+0000","last_clean":"2024-09-10T18:48:08.499164+0000","last_became_active":"2024-09-10T18:45:24.264611+0000","last_became_peered":"2024-09-10T18:45:24.264611+0000","last_unstale":"2024-09-10T18:48:08.499164+0000","last_undegraded":"2024-09-10T18:48:08.499164+0000","last_fullsized":"2024-09-10T18:48:08.499164+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:52.611003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:52.611003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015558849999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"63'10","reported_seq":119,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.496469+0000","last_change":"2024-09-10T18:47:36.442817+0000","last_active":"2024-09-10T18:47:58.496469+0000","last_peered":"2024-09-10T18:47:58.496469+0000","last_clean":"2024-09-10T18:47:58.496469+0000","last_became_active":"2024-09-10T18:47:36.426260+0000","last_became_peered":"2024-09-10T18:47:36.426260+0000","last_unstale":"2024-09-10T18:47:58.496469+0000","last_undegraded":"2024-09-10T18:47:58.496469+0000","last_fullsized":"2024-09-10T18:47:58.496469+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:46:50.722316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.722316+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022194070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":64,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.500529+0000","last_change":"2024-09-10T18:46:50.093435+0000","last_active":"2024-09-10T18:48:08.500529+0000","last_peered":"2024-09-10T18:48:08.500529+0000","last_clean":"2024-09-10T18:48:08.500529+0000","last_became_active":"2024-09-10T18:45:17.964870+0000","last_became_peered":"2024-09-10T18:45:17.964870+0000","last_unstale":"2024-09-10T18:48:08.500529+0000","last_undegraded":"2024-09-10T18:48:08.500529+0000","last_fullsized":"2024-09-10T18:48:08.500529+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:50.093372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.093372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015580539999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"63'8","reported_seq":102,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.501155+0000","last_change":"2024-09-10T18:46:34.085224+0000","last_active":"2024-09-10T18:48:08.501155+0000","last_peered":"2024-09-10T18:48:08.501155+0000","last_clean":"2024-09-10T18:48:08.501155+0000","last_became_active":"2024-09-10T18:45:22.062940+0000","last_became_peered":"2024-09-10T18:45:22.062940+0000","last_unstale":"2024-09-10T18:48:08.501155+0000","last_undegraded":"2024-09-10T18:48:08.501155+0000","last_fullsized":"2024-09-10T18:48:08.501155+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:34.085156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:34.085156+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020843120000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":56,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.502318+0000","last_change":"2024-09-10T18:47:21.538033+0000","last_active":"2024-09-10T18:47:58.502318+0000","last_peered":"2024-09-10T18:47:58.502318+0000","last_clean":"2024-09-10T18:47:58.502318+0000","last_became_active":"2024-09-10T18:45:24.073736+0000","last_became_peered":"2024-09-10T18:45:24.073736+0000","last_unstale":"2024-09-10T18:47:58.502318+0000","last_undegraded":"2024-09-10T18:47:58.502318+0000","last_fullsized":"2024-09-10T18:47:58.502318+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:21.537964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:21.537964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:38.167513+0000","scrub_duration":0.001500305,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"63'9","reported_seq":78,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.501172+0000","last_change":"2024-09-10T18:46:23.086274+0000","last_active":"2024-09-10T18:48:08.501172+0000","last_peered":"2024-09-10T18:48:08.501172+0000","last_clean":"2024-09-10T18:48:08.501172+0000","last_became_active":"2024-09-10T18:45:20.421124+0000","last_became_peered":"2024-09-10T18:45:20.421124+0000","last_unstale":"2024-09-10T18:48:08.501172+0000","last_undegraded":"2024-09-10T18:48:08.501172+0000","last_fullsized":"2024-09-10T18:48:08.501172+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:46:23.086203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:23.086203+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002168068,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":62,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.501743+0000","last_change":"2024-09-10T18:46:45.541882+0000","last_active":"2024-09-10T18:47:58.501743+0000","last_peered":"2024-09-10T18:47:58.501743+0000","last_clean":"2024-09-10T18:47:58.501743+0000","last_became_active":"2024-09-10T18:45:17.960106+0000","last_became_peered":"2024-09-10T18:45:17.960106+0000","last_unstale":"2024-09-10T18:47:58.501743+0000","last_undegraded":"2024-09-10T18:47:58.501743+0000","last_fullsized":"2024-09-10T18:47:58.501743+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:45.541819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:45.541819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:15.015402+0000","scrub_duration":0.0015180809999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"89'10","reported_seq":114,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:48:01.109103+0000","last_change":"2024-09-10T18:47:36.434102+0000","last_active":"2024-09-10T18:48:01.109103+0000","last_peered":"2024-09-10T18:48:01.109103+0000","last_clean":"2024-09-10T18:48:01.109103+0000","last_became_active":"2024-09-10T18:47:36.424575+0000","last_became_peered":"2024-09-10T18:47:36.424575+0000","last_unstale":"2024-09-10T18:48:01.109103+0000","last_undegraded":"2024-09-10T18:48:01.109103+0000","last_fullsized":"2024-09-10T18:48:01.109103+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"81'9","last_scrub_stamp":"2024-09-10T18:46:54.800790+0000","last_deep_scrub":"81'9","last_deep_scrub_stamp":"2024-09-10T18:46:54.800790+0000","last_clean_scrub_stamp":"2024-09-10T18:46:54.800790+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:23.966607+0000","scrub_duration":0.0034208709999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":68,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.506944+0000","last_change":"2024-09-10T18:47:36.414372+0000","last_active":"2024-09-10T18:48:08.506944+0000","last_peered":"2024-09-10T18:48:08.506944+0000","last_clean":"2024-09-10T18:48:08.506944+0000","last_became_active":"2024-09-10T18:47:36.414047+0000","last_became_peered":"2024-09-10T18:47:36.414047+0000","last_unstale":"2024-09-10T18:48:08.506944+0000","last_undegraded":"2024-09-10T18:48:08.506944+0000","last_fullsized":"2024-09-10T18:48:08.506944+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_clean_scrub_stamp":"2024-09-10T18:46:36.305199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013794809999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"63'9","reported_seq":96,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.496663+0000","last_change":"2024-09-10T18:47:36.437948+0000","last_active":"2024-09-10T18:47:58.496663+0000","last_peered":"2024-09-10T18:47:58.496663+0000","last_clean":"2024-09-10T18:47:58.496663+0000","last_became_active":"2024-09-10T18:47:36.416484+0000","last_became_peered":"2024-09-10T18:47:36.416484+0000","last_unstale":"2024-09-10T18:47:58.496663+0000","last_undegraded":"2024-09-10T18:47:58.496663+0000","last_fullsized":"2024-09-10T18:47:58.496663+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:46:59.469198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:59.469198+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:10.069758+0000","scrub_duration":0.0022009489999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":74,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.497287+0000","last_change":"2024-09-10T18:47:29.023501+0000","last_active":"2024-09-10T18:48:08.497287+0000","last_peered":"2024-09-10T18:48:08.497287+0000","last_clean":"2024-09-10T18:48:08.497287+0000","last_became_active":"2024-09-10T18:45:17.962009+0000","last_became_peered":"2024-09-10T18:45:17.962009+0000","last_unstale":"2024-09-10T18:48:08.497287+0000","last_undegraded":"2024-09-10T18:48:08.497287+0000","last_fullsized":"2024-09-10T18:48:08.497287+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:29.023450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:47:29.023450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:37.485149+0000","scrub_duration":0.001196805,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":58,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.497130+0000","last_change":"2024-09-10T18:47:14.869132+0000","last_active":"2024-09-10T18:47:58.497130+0000","last_peered":"2024-09-10T18:47:58.497130+0000","last_clean":"2024-09-10T18:47:58.497130+0000","last_became_active":"2024-09-10T18:45:22.072877+0000","last_became_peered":"2024-09-10T18:45:22.072877+0000","last_unstale":"2024-09-10T18:47:58.497130+0000","last_undegraded":"2024-09-10T18:47:58.497130+0000","last_fullsized":"2024-09-10T18:47:58.497130+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:14.869076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:47:14.869076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:38.655935+0000","scrub_duration":0.00131624,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":56,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.501843+0000","last_change":"2024-09-10T18:46:50.499866+0000","last_active":"2024-09-10T18:47:58.501843+0000","last_peered":"2024-09-10T18:47:58.501843+0000","last_clean":"2024-09-10T18:47:58.501843+0000","last_became_active":"2024-09-10T18:45:24.081050+0000","last_became_peered":"2024-09-10T18:45:24.081050+0000","last_unstale":"2024-09-10T18:47:58.501843+0000","last_undegraded":"2024-09-10T18:47:58.501843+0000","last_fullsized":"2024-09-10T18:47:58.501843+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:50.499815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.499815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:13.479025+0000","scrub_duration":0.001285432,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"63'11","reported_seq":81,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.506766+0000","last_change":"2024-09-10T18:47:14.034427+0000","last_active":"2024-09-10T18:48:08.506766+0000","last_peered":"2024-09-10T18:48:08.506766+0000","last_clean":"2024-09-10T18:48:08.506766+0000","last_became_active":"2024-09-10T18:45:20.056885+0000","last_became_peered":"2024-09-10T18:45:20.056885+0000","last_unstale":"2024-09-10T18:48:08.506766+0000","last_undegraded":"2024-09-10T18:48:08.506766+0000","last_fullsized":"2024-09-10T18:48:08.506766+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:47:14.034361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:14.034361+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:16.811158+0000","scrub_duration":0.0018564,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":62,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.497884+0000","last_change":"2024-09-10T18:46:36.402610+0000","last_active":"2024-09-10T18:47:58.497884+0000","last_peered":"2024-09-10T18:47:58.497884+0000","last_clean":"2024-09-10T18:47:58.497884+0000","last_became_active":"2024-09-10T18:45:17.963542+0000","last_became_peered":"2024-09-10T18:45:17.963542+0000","last_unstale":"2024-09-10T18:47:58.497884+0000","last_undegraded":"2024-09-10T18:47:58.497884+0000","last_fullsized":"2024-09-10T18:47:58.497884+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:36.402548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:36.402548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:04.495178+0000","scrub_duration":0.0014103919999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":58,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.496486+0000","last_change":"2024-09-10T18:46:37.414794+0000","last_active":"2024-09-10T18:47:58.496486+0000","last_peered":"2024-09-10T18:47:58.496486+0000","last_clean":"2024-09-10T18:47:58.496486+0000","last_became_active":"2024-09-10T18:45:22.070065+0000","last_became_peered":"2024-09-10T18:45:22.070065+0000","last_unstale":"2024-09-10T18:47:58.496486+0000","last_undegraded":"2024-09-10T18:47:58.496486+0000","last_fullsized":"2024-09-10T18:47:58.496486+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:37.414734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:37.414734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001607996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":56,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.496046+0000","last_change":"2024-09-10T18:47:05.789893+0000","last_active":"2024-09-10T18:47:58.496046+0000","last_peered":"2024-09-10T18:47:58.496046+0000","last_clean":"2024-09-10T18:47:58.496046+0000","last_became_active":"2024-09-10T18:45:24.072224+0000","last_became_peered":"2024-09-10T18:45:24.072224+0000","last_unstale":"2024-09-10T18:47:58.496046+0000","last_undegraded":"2024-09-10T18:47:58.496046+0000","last_fullsized":"2024-09-10T18:47:58.496046+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:05.789830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:05.789830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:28.845106+0000","scrub_duration":0.001271938,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"63'4","reported_seq":66,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.497233+0000","last_change":"2024-09-10T18:46:46.766906+0000","last_active":"2024-09-10T18:47:58.497233+0000","last_peered":"2024-09-10T18:47:58.497233+0000","last_clean":"2024-09-10T18:47:58.497233+0000","last_became_active":"2024-09-10T18:45:20.059205+0000","last_became_peered":"2024-09-10T18:45:20.059205+0000","last_unstale":"2024-09-10T18:47:58.497233+0000","last_undegraded":"2024-09-10T18:47:58.497233+0000","last_fullsized":"2024-09-10T18:47:58.497233+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4","last_scrub_stamp":"2024-09-10T18:46:46.766766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:46.766766+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021847250000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":74,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.501545+0000","last_change":"2024-09-10T18:47:36.416170+0000","last_active":"2024-09-10T18:48:08.501545+0000","last_peered":"2024-09-10T18:48:08.501545+0000","last_clean":"2024-09-10T18:48:08.501545+0000","last_became_active":"2024-09-10T18:47:36.415866+0000","last_became_peered":"2024-09-10T18:47:36.415866+0000","last_unstale":"2024-09-10T18:48:08.501545+0000","last_undegraded":"2024-09-10T18:48:08.501545+0000","last_fullsized":"2024-09-10T18:48:08.501545+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:54.082761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:54.082761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:14.158996+0000","scrub_duration":0.001210458,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":60,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.496010+0000","last_change":"2024-09-10T18:46:35.172936+0000","last_active":"2024-09-10T18:48:08.496010+0000","last_peered":"2024-09-10T18:48:08.496010+0000","last_clean":"2024-09-10T18:48:08.496010+0000","last_became_active":"2024-09-10T18:45:22.072377+0000","last_became_peered":"2024-09-10T18:45:22.072377+0000","last_unstale":"2024-09-10T18:48:08.496010+0000","last_undegraded":"2024-09-10T18:48:08.496010+0000","last_fullsized":"2024-09-10T18:48:08.496010+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_clean_scrub_stamp":"2024-09-10T18:46:35.172861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001552933,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":69,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.498595+0000","last_change":"2024-09-10T18:47:36.416483+0000","last_active":"2024-09-10T18:48:08.498595+0000","last_peered":"2024-09-10T18:48:08.498595+0000","last_clean":"2024-09-10T18:48:08.498595+0000","last_became_active":"2024-09-10T18:47:36.416187+0000","last_became_peered":"2024-09-10T18:47:36.416187+0000","last_unstale":"2024-09-10T18:48:08.498595+0000","last_undegraded":"2024-09-10T18:48:08.498595+0000","last_fullsized":"2024-09-10T18:48:08.498595+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:53.596380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:53.596380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020285519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"79'16","reported_seq":84,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.495600+0000","last_change":"2024-09-10T18:46:48.744340+0000","last_active":"2024-09-10T18:47:58.495600+0000","last_peered":"2024-09-10T18:47:58.495600+0000","last_clean":"2024-09-10T18:47:58.495600+0000","last_became_active":"2024-09-10T18:45:20.421479+0000","last_became_peered":"2024-09-10T18:45:20.421479+0000","last_unstale":"2024-09-10T18:47:58.495600+0000","last_undegraded":"2024-09-10T18:47:58.495600+0000","last_fullsized":"2024-09-10T18:47:58.495600+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"79'16","last_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_deep_scrub":"79'16","last_deep_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_clean_scrub_stamp":"2024-09-10T18:46:48.744287+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044879259999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"56'1","reported_seq":85,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.497869+0000","last_change":"2024-09-10T18:47:36.430544+0000","last_active":"2024-09-10T18:48:08.497869+0000","last_peered":"2024-09-10T18:48:08.497869+0000","last_clean":"2024-09-10T18:48:08.497869+0000","last_became_active":"2024-09-10T18:47:36.421985+0000","last_became_peered":"2024-09-10T18:47:36.421985+0000","last_unstale":"2024-09-10T18:48:08.497869+0000","last_undegraded":"2024-09-10T18:48:08.497869+0000","last_fullsized":"2024-09-10T18:48:08.497869+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:46:41.147463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:41.147463+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016797889999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.496846+0000","last_change":"2024-09-10T18:47:57.107638+0000","last_active":"2024-09-10T18:47:58.496846+0000","last_peered":"2024-09-10T18:47:58.496846+0000","last_clean":"2024-09-10T18:47:58.496846+0000","last_became_active":"2024-09-10T18:47:57.107149+0000","last_became_peered":"2024-09-10T18:47:57.107149+0000","last_unstale":"2024-09-10T18:47:58.496846+0000","last_undegraded":"2024-09-10T18:47:58.496846+0000","last_fullsized":"2024-09-10T18:47:58.496846+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:27.667018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:27.667018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":56,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.496991+0000","last_change":"2024-09-10T18:46:29.791617+0000","last_active":"2024-09-10T18:47:58.496991+0000","last_peered":"2024-09-10T18:47:58.496991+0000","last_clean":"2024-09-10T18:47:58.496991+0000","last_became_active":"2024-09-10T18:45:24.264818+0000","last_became_peered":"2024-09-10T18:45:24.264818+0000","last_unstale":"2024-09-10T18:47:58.496991+0000","last_undegraded":"2024-09-10T18:47:58.496991+0000","last_fullsized":"2024-09-10T18:47:58.496991+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:29.791563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:29.791563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021031819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"78'2","reported_seq":85,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-10T18:48:10.610753+0000","last_change":"2024-09-10T18:48:10.610714+0000","last_active":"2024-09-10T18:48:10.610753+0000","last_peered":"2024-09-10T18:48:10.610753+0000","last_clean":"2024-09-10T18:48:10.610753+0000","last_became_active":"2024-09-10T18:47:36.418702+0000","last_became_peered":"2024-09-10T18:47:36.418702+0000","last_unstale":"2024-09-10T18:48:10.610753+0000","last_undegraded":"2024-09-10T18:48:10.610753+0000","last_fullsized":"2024-09-10T18:48:10.610753+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'2","last_scrub_stamp":"2024-09-10T18:48:10.610652+0000","last_deep_scrub":"78'2","last_deep_scrub_stamp":"2024-09-10T18:46:37.581130+0000","last_clean_scrub_stamp":"2024-09-10T18:48:10.610652+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:24.088784+0000","scrub_duration":0.002045275,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":64,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.500323+0000","last_change":"2024-09-10T18:46:28.088657+0000","last_active":"2024-09-10T18:48:08.500323+0000","last_peered":"2024-09-10T18:48:08.500323+0000","last_clean":"2024-09-10T18:48:08.500323+0000","last_became_active":"2024-09-10T18:45:17.962108+0000","last_became_peered":"2024-09-10T18:45:17.962108+0000","last_unstale":"2024-09-10T18:48:08.500323+0000","last_undegraded":"2024-09-10T18:48:08.500323+0000","last_fullsized":"2024-09-10T18:48:08.500323+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:28.088588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:28.088588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00140508,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"63'10","reported_seq":75,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.502545+0000","last_change":"2024-09-10T18:46:49.494628+0000","last_active":"2024-09-10T18:47:58.502545+0000","last_peered":"2024-09-10T18:47:58.502545+0000","last_clean":"2024-09-10T18:47:58.502545+0000","last_became_active":"2024-09-10T18:45:20.421094+0000","last_became_peered":"2024-09-10T18:45:20.421094+0000","last_unstale":"2024-09-10T18:47:58.502545+0000","last_undegraded":"2024-09-10T18:47:58.502545+0000","last_fullsized":"2024-09-10T18:47:58.502545+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:46:49.494565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:49.494565+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:08.379972+0000","scrub_duration":0.0036379630000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":105,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.497779+0000","last_change":"2024-09-10T18:46:33.150919+0000","last_active":"2024-09-10T18:48:08.497779+0000","last_peered":"2024-09-10T18:48:08.497779+0000","last_clean":"2024-09-10T18:48:08.497779+0000","last_became_active":"2024-09-10T18:45:22.073281+0000","last_became_peered":"2024-09-10T18:45:22.073281+0000","last_unstale":"2024-09-10T18:48:08.497779+0000","last_undegraded":"2024-09-10T18:48:08.497779+0000","last_fullsized":"2024-09-10T18:48:08.497779+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:33.150852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:33.150852+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017025339999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":56,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.497491+0000","last_change":"2024-09-10T18:47:03.475367+0000","last_active":"2024-09-10T18:47:58.497491+0000","last_peered":"2024-09-10T18:47:58.497491+0000","last_clean":"2024-09-10T18:47:58.497491+0000","last_became_active":"2024-09-10T18:45:24.082318+0000","last_became_peered":"2024-09-10T18:45:24.082318+0000","last_unstale":"2024-09-10T18:47:58.497491+0000","last_undegraded":"2024-09-10T18:47:58.497491+0000","last_fullsized":"2024-09-10T18:47:58.497491+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:03.475299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:03.475299+0000","last_clean_scrub_stamp":"2024-09-10T18:47:03.475299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:22.439695+0000","scrub_duration":0.001661945,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":62,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.495984+0000","last_change":"2024-09-10T18:46:44.694805+0000","last_active":"2024-09-10T18:47:58.495984+0000","last_peered":"2024-09-10T18:47:58.495984+0000","last_clean":"2024-09-10T18:47:58.495984+0000","last_became_active":"2024-09-10T18:45:17.965707+0000","last_became_peered":"2024-09-10T18:45:17.965707+0000","last_unstale":"2024-09-10T18:47:58.495984+0000","last_undegraded":"2024-09-10T18:47:58.495984+0000","last_fullsized":"2024-09-10T18:47:58.495984+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:44.694737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:44.694737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016330400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"63'11","reported_seq":81,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.497856+0000","last_change":"2024-09-10T18:46:46.230915+0000","last_active":"2024-09-10T18:48:08.497856+0000","last_peered":"2024-09-10T18:48:08.497856+0000","last_clean":"2024-09-10T18:48:08.497856+0000","last_became_active":"2024-09-10T18:45:20.058717+0000","last_became_peered":"2024-09-10T18:45:20.058717+0000","last_unstale":"2024-09-10T18:48:08.497856+0000","last_undegraded":"2024-09-10T18:48:08.497856+0000","last_fullsized":"2024-09-10T18:48:08.497856+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:46:46.230819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:46.230819+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002545322,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":68,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.502450+0000","last_change":"2024-09-10T18:47:36.419088+0000","last_active":"2024-09-10T18:47:58.502450+0000","last_peered":"2024-09-10T18:47:58.502450+0000","last_clean":"2024-09-10T18:47:58.502450+0000","last_became_active":"2024-09-10T18:47:36.418767+0000","last_became_peered":"2024-09-10T18:47:36.418767+0000","last_unstale":"2024-09-10T18:47:58.502450+0000","last_undegraded":"2024-09-10T18:47:58.502450+0000","last_fullsized":"2024-09-10T18:47:58.502450+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:15.536620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:47:15.536620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:21.452511+0000","scrub_duration":0.0015375409999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":58,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.506535+0000","last_change":"2024-09-10T18:46:45.285074+0000","last_active":"2024-09-10T18:48:08.506535+0000","last_peered":"2024-09-10T18:48:08.506535+0000","last_clean":"2024-09-10T18:48:08.506535+0000","last_became_active":"2024-09-10T18:45:24.267251+0000","last_became_peered":"2024-09-10T18:45:24.267251+0000","last_unstale":"2024-09-10T18:48:08.506535+0000","last_undegraded":"2024-09-10T18:48:08.506535+0000","last_fullsized":"2024-09-10T18:48:08.506535+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:45.285015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:45.285015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001362142,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"56'1","reported_seq":73,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.496464+0000","last_change":"2024-09-10T18:46:42.139314+0000","last_active":"2024-09-10T18:48:08.496464+0000","last_peered":"2024-09-10T18:48:08.496464+0000","last_clean":"2024-09-10T18:48:08.496464+0000","last_became_active":"2024-09-10T18:45:17.966286+0000","last_became_peered":"2024-09-10T18:45:17.966286+0000","last_unstale":"2024-09-10T18:48:08.496464+0000","last_undegraded":"2024-09-10T18:48:08.496464+0000","last_fullsized":"2024-09-10T18:48:08.496464+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:46:42.139256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:42.139256+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014138690000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"63'15","reported_seq":76,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.496504+0000","last_change":"2024-09-10T18:47:36.446154+0000","last_active":"2024-09-10T18:47:58.496504+0000","last_peered":"2024-09-10T18:47:58.496504+0000","last_clean":"2024-09-10T18:47:58.496504+0000","last_became_active":"2024-09-10T18:47:36.424373+0000","last_became_peered":"2024-09-10T18:47:36.424373+0000","last_unstale":"2024-09-10T18:47:58.496504+0000","last_undegraded":"2024-09-10T18:47:58.496504+0000","last_fullsized":"2024-09-10T18:47:58.496504+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:47:34.610915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:34.610915+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:42.501880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":58,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.497780+0000","last_change":"2024-09-10T18:47:09.520829+0000","last_active":"2024-09-10T18:47:58.497780+0000","last_peered":"2024-09-10T18:47:58.497780+0000","last_clean":"2024-09-10T18:47:58.497780+0000","last_became_active":"2024-09-10T18:45:22.059794+0000","last_became_peered":"2024-09-10T18:45:22.059794+0000","last_unstale":"2024-09-10T18:47:58.497780+0000","last_undegraded":"2024-09-10T18:47:58.497780+0000","last_fullsized":"2024-09-10T18:47:58.497780+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:09.520768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:47:09.520768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:33.788561+0000","scrub_duration":0.0016284120000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":56,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.499321+0000","last_change":"2024-09-10T18:46:57.811148+0000","last_active":"2024-09-10T18:47:58.499321+0000","last_peered":"2024-09-10T18:47:58.499321+0000","last_clean":"2024-09-10T18:47:58.499321+0000","last_became_active":"2024-09-10T18:45:24.264427+0000","last_became_peered":"2024-09-10T18:45:24.264427+0000","last_unstale":"2024-09-10T18:47:58.499321+0000","last_undegraded":"2024-09-10T18:47:58.499321+0000","last_fullsized":"2024-09-10T18:47:58.499321+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:57.811074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:57.811074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:12.950473+0000","scrub_duration":0.0013689419999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":62,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.498630+0000","last_change":"2024-09-10T18:46:47.734860+0000","last_active":"2024-09-10T18:47:58.498630+0000","last_peered":"2024-09-10T18:47:58.498630+0000","last_clean":"2024-09-10T18:47:58.498630+0000","last_became_active":"2024-09-10T18:45:17.963257+0000","last_became_peered":"2024-09-10T18:45:17.963257+0000","last_unstale":"2024-09-10T18:47:58.498630+0000","last_undegraded":"2024-09-10T18:47:58.498630+0000","last_fullsized":"2024-09-10T18:47:58.498630+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:47.734803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:47.734803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:05.847815+0000","scrub_duration":0.001381134,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"63'12","reported_seq":83,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T18:47:58.495471+0000","last_change":"2024-09-10T18:46:52.771640+0000","last_active":"2024-09-10T18:47:58.495471+0000","last_peered":"2024-09-10T18:47:58.495471+0000","last_clean":"2024-09-10T18:47:58.495471+0000","last_became_active":"2024-09-10T18:45:20.058391+0000","last_became_peered":"2024-09-10T18:45:20.058391+0000","last_unstale":"2024-09-10T18:47:58.495471+0000","last_undegraded":"2024-09-10T18:47:58.495471+0000","last_fullsized":"2024-09-10T18:47:58.495471+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:46:52.771570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:52.771570+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:04.251299+0000","scrub_duration":0.0021490089999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":80,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T18:48:08.507126+0000","last_change":"2024-09-10T18:47:36.416505+0000","last_active":"2024-09-10T18:48:08.507126+0000","last_peered":"2024-09-10T18:48:08.507126+0000","last_clean":"2024-09-10T18:48:08.507126+0000","last_became_active":"2024-09-10T18:47:36.416131+0000","last_became_peered":"2024-09-10T18:47:36.416131+0000","last_unstale":"2024-09-10T18:48:08.507126+0000","last_undegraded":"2024-09-10T18:48:08.507126+0000","last_fullsized":"2024-09-10T18:48:08.507126+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:34.958889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:47:34.958889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:57.335633+0000","scrub_duration":0.001401296,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":3554541568,"num_objects":54238,"num_object_clones":0,"num_object_copies":162714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":54238,"num_write_kb":3471232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":10563026944,"data_stored":10563026944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54238,"ondisk_log_size":54238,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":697,"num_read_kb":454,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":1238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":423,"ondisk_log_size":423,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":158,"num_read_kb":153,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299431,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1676592,"kb_used_data":1379236,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":92068560,"statfs":{"total":95995035648,"available":94278205440,"internally_reserved":0,"allocated":1412337664,"data_stored":1411878850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:48:08 2024","interfaces":[{"interface":"back","average":{"1min":2.1850000000000001,"5min":0.84199999999999997,"15min":0.61899999999999999},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":31.805,"5min":31.805,"15min":31.805},"last":0.54800000000000004},{"interface":"front","average":{"1min":2.0510000000000002,"5min":0.80600000000000005,"15min":0.59199999999999997},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":31.212,"5min":31.212,"15min":31.212},"last":1.974}]},{"osd":1,"last update":"Tue Sep 10 18:48:08 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48899999999999999,"15min":0.48099999999999998},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.73799999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":1.5600000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51500000000000001,"15min":0.51700000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.58499999999999996}]},{"osd":2,"last update":"Tue Sep 10 18:48:08 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":1.871},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52600000000000002,"15min":0.51900000000000002},"min":{"1min":0.36699999999999999,"5min":0.31,"15min":0.31},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":3.0209999999999999}]},{"osd":3,"last update":"Tue Sep 10 18:48:08 2024","interfaces":[{"interface":"back","average":{"1min":2.1899999999999999,"5min":0.84199999999999997,"15min":0.61799999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":31.478000000000002,"5min":31.478000000000002,"15min":31.478000000000002},"last":1.486},{"interface":"front","average":{"1min":2.6629999999999998,"5min":0.94999999999999996,"15min":0.66200000000000003},"min":{"1min":0.35199999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":33.731999999999999,"5min":33.731999999999999,"15min":33.731999999999999},"last":2.2559999999999998}]},{"osd":4,"last update":"Tue Sep 10 18:48:08 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.38300000000000001,"15min":0.38400000000000001},"min":{"1min":0.27800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.55200000000000005,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.629},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39000000000000001,"15min":0.38},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.55800000000000005,"5min":0.623,"15min":0.623},"last":0.29599999999999999}]},{"osd":5,"last update":"Tue Sep 10 18:48:08 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40899999999999997,"15min":0.41099999999999998},"min":{"1min":0.29399999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.42099999999999999,"15min":0.41699999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.61699999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.57699999999999996}]},{"osd":6,"last update":"Tue Sep 10 18:48:08 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.54200000000000004,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.432,"5min":0.44,"15min":0.435},"min":{"1min":0.32100000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.61199999999999999,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.67800000000000005}]}]},{"osd":6,"up_from":43,"seq":184683593772,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1684668,"kb_used_data":1387308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":92060484,"statfs":{"total":95995035648,"available":94269935616,"internally_reserved":0,"allocated":1420603392,"data_stored":1420136930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:47:44 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.48199999999999998,"15min":0.46600000000000003},"min":{"1min":0.26100000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.48899999999999999,"15min":0.48999999999999999},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.69699999999999995,"5min":0.755,"15min":0.755},"last":0.70899999999999996}]},{"osd":1,"last update":"Tue Sep 10 18:47:44 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.29999999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.46600000000000003,"15min":0.46200000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.52600000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:47:44 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48299999999999998,"15min":0.48099999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.49099999999999999,"15min":0.48999999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.44}]},{"osd":3,"last update":"Tue Sep 10 18:47:44 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.48799999999999999,"15min":0.48399999999999999},"min":{"1min":0.33700000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.51600000000000001,"15min":0.51200000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.45500000000000002}]},{"osd":4,"last update":"Tue Sep 10 18:47:44 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.40799999999999997,"15min":0.40600000000000003},"min":{"1min":0.23599999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.40100000000000002,"15min":0.38900000000000001},"min":{"1min":0.26700000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.66100000000000003,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.69099999999999995}]},{"osd":5,"last update":"Tue Sep 10 18:47:44 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.41599999999999998},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.433,"15min":0.42499999999999999},"min":{"1min":0.309,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.58499999999999996,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.67400000000000004}]},{"osd":7,"last update":"Tue Sep 10 18:48:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.495,"15min":0.47699999999999998},"min":{"1min":0.32000000000000001,"5min":0.309,"15min":0.309},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.51600000000000001,"15min":0.501},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.883,"5min":1.883,"15min":1.883},"last":0.41399999999999998}]}]},{"osd":5,"up_from":37,"seq":158913790000,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1215764,"kb_used_data":910088,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":92529388,"statfs":{"total":95995035648,"available":94750093312,"internally_reserved":0,"allocated":931930112,"data_stored":931491415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:47:21 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51600000000000001,"15min":0.52400000000000002},"min":{"1min":0.379,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.65700000000000003,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":1.49},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.48399999999999999,"15min":0.49399999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.59199999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":2.5169999999999999}]},{"osd":1,"last update":"Tue Sep 10 18:47:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54300000000000004,"15min":0.55700000000000005},"min":{"1min":0.32300000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.64100000000000001,"5min":0.87,"15min":0.87},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52300000000000002,"15min":0.53500000000000003},"min":{"1min":0.36399999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.58699999999999997,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.33800000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:47:21 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.51200000000000001,"15min":0.51800000000000002},"min":{"1min":0.375,"5min":0.311,"15min":0.311},"max":{"1min":0.56799999999999995,"5min":0.755,"15min":0.755},"last":0.434},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.52300000000000002,"15min":0.53700000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.65900000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":4.4720000000000004}]},{"osd":3,"last update":"Tue Sep 10 18:47:21 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52000000000000002,"15min":0.52500000000000002},"min":{"1min":0.375,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.61599999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":3.6760000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.52700000000000002,"15min":0.54100000000000004},"min":{"1min":0.37,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.60999999999999999,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":3.3359999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:47:21 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.375,"15min":0.38},"min":{"1min":0.255,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.39900000000000002,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.39800000000000002,"15min":0.40100000000000002},"min":{"1min":0.27800000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.56699999999999995,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.375}]},{"osd":6,"last update":"Tue Sep 10 18:47:47 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.42299999999999999,"15min":0.41699999999999998},"min":{"1min":0.29799999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":1.5489999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46700000000000003,"15min":0.46100000000000002},"min":{"1min":0.33900000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.64100000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":1.5660000000000001}]},{"osd":7,"last update":"Tue Sep 10 18:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47399999999999998,"15min":0.47599999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.68799999999999994,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.47399999999999998,"15min":0.47599999999999998},"min":{"1min":0.28000000000000003,"5min":0.254,"15min":0.254},"max":{"1min":0.67600000000000005,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.95399999999999996}]}]},{"osd":4,"up_from":30,"seq":128849018934,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1478572,"kb_used_data":1172896,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":92266580,"statfs":{"total":95995035648,"available":94480977920,"internally_reserved":0,"allocated":1201045504,"data_stored":1200589778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52800000000000002,"15min":0.52600000000000002},"min":{"1min":0.23200000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51600000000000001,"15min":0.53400000000000003},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":1.232}]},{"osd":1,"last update":"Tue Sep 10 18:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.54900000000000004,"15min":0.56499999999999995},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.82499999999999996,"5min":2.609,"15min":2.609},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.55100000000000005,"15min":0.53300000000000003},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.98799999999999999,"5min":2.589,"15min":2.589},"last":1.579}]},{"osd":2,"last update":"Tue Sep 10 18:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.80000000000000004,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53500000000000003,"15min":0.55000000000000004},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.62}]},{"osd":3,"last update":"Tue Sep 10 18:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.53400000000000003,"15min":0.53200000000000003},"min":{"1min":0.33800000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":1.732},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.53300000000000003,"15min":0.54000000000000004},"min":{"1min":0.34799999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.45400000000000001}]},{"osd":5,"last update":"Tue Sep 10 18:47:25 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.374,"15min":0.35499999999999998},"min":{"1min":0.29099999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.55100000000000005,"5min":0.623,"15min":0.623},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.379,"15min":0.36199999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.58399999999999996,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.32500000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:47:43 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.436,"15min":0.42599999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.41199999999999998,"15min":0.39500000000000002},"min":{"1min":0.29899999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.375}]},{"osd":7,"last update":"Tue Sep 10 18:47:10 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.5,"15min":0.5},"min":{"1min":0.318,"5min":0.312,"15min":0.312},"max":{"1min":0.64700000000000002,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.434,"5min":0.44600000000000001,"15min":0.44800000000000001},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.34999999999999998}]}]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2459220,"kb_used_data":2149388,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":91285932,"statfs":{"total":95995035648,"available":93476794368,"internally_reserved":0,"allocated":2200973312,"data_stored":2200517554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:47:37 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.371,"15min":0.38100000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":1.1479999999999999},{"interface":"front","average":{"1min":0.375,"5min":0.39500000000000002,"15min":0.40100000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.77500000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":2.3220000000000001}]},{"osd":1,"last update":"Tue Sep 10 18:47:37 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.39100000000000001,"15min":0.40500000000000003},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.52600000000000002,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":2.2970000000000002},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.40300000000000002,"15min":0.41299999999999998},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":2.206}]},{"osd":2,"last update":"Tue Sep 10 18:47:37 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.42399999999999999,"15min":0.436},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":2.1709999999999998},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41499999999999998,"15min":0.43099999999999999},"min":{"1min":0.27500000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":1.042}]},{"osd":4,"last update":"Tue Sep 10 18:47:59 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.58499999999999996,"15min":0.55800000000000005},"min":{"1min":0.375,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":3.0699999999999998,"5min":3.0699999999999998,"15min":3.0699999999999998},"last":1.097},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.60299999999999998,"15min":0.58499999999999996},"min":{"1min":0.35199999999999998,"5min":0.311,"15min":0.311},"max":{"1min":3.0990000000000002,"5min":3.0990000000000002,"15min":3.0990000000000002},"last":1.1220000000000001}]},{"osd":5,"last update":"Tue Sep 10 18:47:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54500000000000004,"15min":0.54200000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":2.718},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":2.5910000000000002}]},{"osd":6,"last update":"Tue Sep 10 18:47:45 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55600000000000005,"15min":0.56200000000000006},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":2.637},{"interface":"front","average":{"1min":0.63,"5min":0.56899999999999995,"15min":0.56100000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.948,"5min":1.948,"15min":1.948},"last":1.0720000000000001}]},{"osd":7,"last update":"Tue Sep 10 18:47:08 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.54500000000000004,"15min":0.53500000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":2.694},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57999999999999996,"15min":0.57699999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84099999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":2.665}]}]},{"osd":2,"up_from":0,"seq":77309411391,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1883996,"kb_used_data":1586640,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":91861156,"statfs":{"total":95995035648,"available":94065823744,"internally_reserved":0,"allocated":1624719360,"data_stored":1624259774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:47:50 2024","interfaces":[{"interface":"back","average":{"1min":0.28799999999999998,"5min":0.34000000000000002,"15min":0.31900000000000001},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.47199999999999998,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.374,"15min":0.33900000000000002},"min":{"1min":0.26200000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.64600000000000002,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":1.7490000000000001}]},{"osd":2,"last update":"Tue Sep 10 18:47:15 2024","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.35999999999999999,"15min":0.35699999999999998},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.41899999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.311},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.38600000000000001,"15min":0.38},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.52600000000000002,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.48099999999999998}]},{"osd":3,"last update":"Tue Sep 10 18:47:39 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.438,"15min":0.45200000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.51200000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":2.258},{"interface":"front","average":{"1min":0.373,"5min":0.41299999999999998,"15min":0.41899999999999998},"min":{"1min":0.30499999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.503,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":1.778}]},{"osd":4,"last update":"Tue Sep 10 18:48:03 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.63300000000000001,"15min":0.57899999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":4.1520000000000001,"5min":4.1520000000000001,"15min":4.1520000000000001},"last":1.821},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.55300000000000005,"15min":0.53700000000000003},"min":{"1min":0.33700000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":1.2989999999999999}]},{"osd":5,"last update":"Tue Sep 10 18:47:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.58799999999999997,"15min":0.59899999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.79500000000000004,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.61899999999999999,"15min":0.63500000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.70799999999999996,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":1.843}]},{"osd":6,"last update":"Tue Sep 10 18:47:43 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.60899999999999999,"15min":0.624},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.70599999999999996,"5min":1.149,"15min":1.149},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.64100000000000001,"15min":0.66300000000000003},"min":{"1min":0.41399999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81699999999999995,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":1.7989999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:47:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.63200000000000001,"15min":0.64200000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.79300000000000004,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.63800000000000001,"15min":0.64400000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.81200000000000006,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":1.863}]}]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1892176,"kb_used_data":1582344,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":91852976,"statfs":{"total":95995035648,"available":94057447424,"internally_reserved":0,"allocated":1620320256,"data_stored":1619868886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 18:47:54 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.38300000000000001,"15min":0.32300000000000001},"min":{"1min":0.24199999999999999,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.379,"15min":0.32900000000000001},"min":{"1min":0.23499999999999999,"5min":0.187,"15min":0.187},"max":{"1min":2.0990000000000002,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":0.90700000000000003}]},{"osd":2,"last update":"Tue Sep 10 18:47:21 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.40500000000000003,"15min":0.38900000000000001},"min":{"1min":0.33900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.60599999999999998,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.372,"5min":0.379,"15min":0.36299999999999999},"min":{"1min":0.26200000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.54100000000000004,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.56399999999999995}]},{"osd":3,"last update":"Tue Sep 10 18:47:35 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.38700000000000001,"15min":0.39500000000000002},"min":{"1min":0.27100000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.53800000000000003,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.40200000000000002,"15min":0.40999999999999998},"min":{"1min":0.29599999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.48599999999999999,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.76200000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.56799999999999995,"15min":0.54900000000000004},"min":{"1min":0.436,"5min":0.375,"15min":0.375},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.61699999999999999,"15min":0.55100000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":2.052,"5min":4.6189999999999998,"15min":4.6189999999999998},"last":1.29}]},{"osd":5,"last update":"Tue Sep 10 18:47:25 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.55900000000000005,"15min":0.55800000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.755,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.59299999999999997,"15min":0.56000000000000005},"min":{"1min":0.39200000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.6679999999999999,"5min":4.6429999999999998,"15min":4.6429999999999998},"last":2.9580000000000002}]},{"osd":6,"last update":"Tue Sep 10 18:47:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56699999999999995,"15min":0.55700000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":2.2050000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.70599999999999996,"15min":0.76000000000000001},"min":{"1min":0.39900000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.6539999999999999,"5min":4.6630000000000003,"15min":4.6630000000000003},"last":0.80400000000000005}]},{"osd":7,"last update":"Tue Sep 10 18:47:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.55100000000000005,"15min":0.55500000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.67700000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.73799999999999999,"5min":0.746,"15min":0.746},"last":1.319}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1637220352,"data_stored":1637220352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1645805568,"data_stored":1645805568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2260140032,"data_stored":2260140032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1203765248,"data_stored":1203765248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":955121664,"data_stored":955121664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1433403392,"data_stored":1433403392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1427570688,"data_stored":1427570688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T18:48:13.975 INFO:tasks.thrashosds.thrasher:Setting 6.18 to [6, 7, 5, 1, 3, 0] 2024-09-10T18:48:13.975 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '6.18', '6', '7', '5', '1', '3', '0'] 2024-09-10T18:48:13.975 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd pg-upmap-items 6.18 6 7 5 1 3 0 2024-09-10T18:48:14.011 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:14.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:14.012 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:14.113 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:14.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:14.160 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:14.161 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:14.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:14 smithi018 ceph-mon[40196]: 5.b scrub starts 2024-09-10T18:48:14.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:14 smithi018 ceph-mon[40196]: 5.b scrub ok 2024-09-10T18:48:14.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:14 smithi018 ceph-mon[40196]: pgmap v105: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.0 KiB/s rd, 104 MiB/s wr, 1.67k op/s 2024-09-10T18:48:14.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:14 smithi018 ceph-mon[40196]: 2.7 scrub starts 2024-09-10T18:48:14.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:14 smithi018 ceph-mon[40196]: 2.7 scrub ok 2024-09-10T18:48:14.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:14 smithi018 ceph-mon[29336]: 5.b scrub starts 2024-09-10T18:48:14.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:14 smithi018 ceph-mon[29336]: 5.b scrub ok 2024-09-10T18:48:14.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:14 smithi018 ceph-mon[29336]: pgmap v105: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.0 KiB/s rd, 104 MiB/s wr, 1.67k op/s 2024-09-10T18:48:14.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:14 smithi018 ceph-mon[29336]: 2.7 scrub starts 2024-09-10T18:48:14.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:14 smithi018 ceph-mon[29336]: 2.7 scrub ok 2024-09-10T18:48:14.206 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 30 16 62885 62869 130.939 84.625 0.015654 0.00763173 2024-09-10T18:48:14.262 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:14.290 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:14.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:14.291 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:14.392 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:14.426 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:14.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:14.427 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:14.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:14 smithi103 ceph-mon[33163]: 5.b scrub starts 2024-09-10T18:48:14.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:14 smithi103 ceph-mon[33163]: 5.b scrub ok 2024-09-10T18:48:14.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:14 smithi103 ceph-mon[33163]: pgmap v105: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.0 KiB/s rd, 104 MiB/s wr, 1.67k op/s 2024-09-10T18:48:14.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:14 smithi103 ceph-mon[33163]: 2.7 scrub starts 2024-09-10T18:48:14.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:14 smithi103 ceph-mon[33163]: 2.7 scrub ok 2024-09-10T18:48:14.528 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:14.564 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:14.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:14.565 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:14.666 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:14.697 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:14.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:14.698 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:14.799 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:14.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:14.839 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:14.839 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:14.940 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:14.967 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:14.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:14.969 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:15.069 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:15.110 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:15.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:15.112 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:15.206 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 31 16 64269 64253 129.506 86.5 0.00506207 0.00771637 2024-09-10T18:48:15.213 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:15.237 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:15.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:15.238 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:15.339 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:15.365 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:15.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:15.366 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:15.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:15 smithi018 ceph-mon[40196]: 2.1d scrub starts 2024-09-10T18:48:15.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:15 smithi018 ceph-mon[40196]: 2.1d scrub ok 2024-09-10T18:48:15.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:15 smithi018 ceph-mon[40196]: 2.9 scrub starts 2024-09-10T18:48:15.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:15 smithi018 ceph-mon[40196]: 2.9 scrub ok 2024-09-10T18:48:15.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:15 smithi018 ceph-mon[40196]: 4.a scrub starts 2024-09-10T18:48:15.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:15 smithi018 ceph-mon[40196]: 4.a scrub ok 2024-09-10T18:48:15.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:15 smithi018 ceph-mon[29336]: 2.1d scrub starts 2024-09-10T18:48:15.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:15 smithi018 ceph-mon[29336]: 2.1d scrub ok 2024-09-10T18:48:15.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:15 smithi018 ceph-mon[29336]: 2.9 scrub starts 2024-09-10T18:48:15.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:15 smithi018 ceph-mon[29336]: 2.9 scrub ok 2024-09-10T18:48:15.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:15 smithi018 ceph-mon[29336]: 4.a scrub starts 2024-09-10T18:48:15.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:15 smithi018 ceph-mon[29336]: 4.a scrub ok 2024-09-10T18:48:15.467 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:15.505 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:15.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:15.506 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:15.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:15 smithi103 ceph-mon[33163]: 2.1d scrub starts 2024-09-10T18:48:15.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:15 smithi103 ceph-mon[33163]: 2.1d scrub ok 2024-09-10T18:48:15.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:15 smithi103 ceph-mon[33163]: 2.9 scrub starts 2024-09-10T18:48:15.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:15 smithi103 ceph-mon[33163]: 2.9 scrub ok 2024-09-10T18:48:15.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:15 smithi103 ceph-mon[33163]: 4.a scrub starts 2024-09-10T18:48:15.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:15 smithi103 ceph-mon[33163]: 4.a scrub ok 2024-09-10T18:48:15.607 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:15.646 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:15.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:15.647 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:15.748 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:15.777 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:15.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:15.778 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:15.879 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:15.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:15.911 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:15.911 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:16.012 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:16.050 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:16.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:16.051 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:16.152 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:16.183 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:16.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:16.184 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:16.206 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 32 16 65568 65552 127.996 81.1875 0.00707711 0.00780733 2024-09-10T18:48:16.285 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:16.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:16.328 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:16.328 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:16.429 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:16.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:16 smithi018 ceph-mon[29336]: 2.10 scrub starts 2024-09-10T18:48:16.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:16 smithi018 ceph-mon[29336]: 2.10 scrub ok 2024-09-10T18:48:16.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:16 smithi018 ceph-mon[29336]: pgmap v106: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 780 B/s rd, 78 MiB/s wr, 1.25k op/s 2024-09-10T18:48:16.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:16 smithi018 ceph-mon[40196]: 2.10 scrub starts 2024-09-10T18:48:16.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:16 smithi018 ceph-mon[40196]: 2.10 scrub ok 2024-09-10T18:48:16.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:16 smithi018 ceph-mon[40196]: pgmap v106: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 780 B/s rd, 78 MiB/s wr, 1.25k op/s 2024-09-10T18:48:16.464 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:16.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:16.465 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:16.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:48:16 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:48:16.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:16 smithi103 ceph-mon[33163]: 2.10 scrub starts 2024-09-10T18:48:16.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:16 smithi103 ceph-mon[33163]: 2.10 scrub ok 2024-09-10T18:48:16.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:16 smithi103 ceph-mon[33163]: pgmap v106: 148 pgs: 148 active+clean; 3.6 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 780 B/s rd, 78 MiB/s wr, 1.25k op/s 2024-09-10T18:48:16.566 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:16.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:16.602 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:16.602 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:16.703 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:16.739 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:16.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:16.740 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:16.841 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:16.875 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:16.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:16.876 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:16.977 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:17.006 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:17.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:17.007 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:17.047 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-10T18:48:17.108 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:17.152 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:17.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:17.153 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:17.206 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 33 16 66920 66904 126.678 84.5 0.00800092 0.00788865 2024-09-10T18:48:17.253 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:17.280 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:17.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:17.281 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:17.348 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:48:17.373 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:17 smithi018 ceph-mon[40196]: 5.8 scrub starts 2024-09-10T18:48:17.373 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:17 smithi018 ceph-mon[40196]: 5.8 scrub ok 2024-09-10T18:48:17.374 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:17 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:17.378 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:17 smithi018 ceph-mon[29336]: 5.8 scrub starts 2024-09-10T18:48:17.378 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:17 smithi018 ceph-mon[29336]: 5.8 scrub ok 2024-09-10T18:48:17.378 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:17 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:17.382 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:17.427 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:17.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:17.429 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:17.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:17 smithi103 ceph-mon[33163]: 5.8 scrub starts 2024-09-10T18:48:17.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:17 smithi103 ceph-mon[33163]: 5.8 scrub ok 2024-09-10T18:48:17.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:17 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:17.529 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:17.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:17.575 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:17.575 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:17.676 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:17.725 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:17.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:17.726 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:17.827 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:17.869 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:17.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:17.870 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:17.970 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:17.999 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:17.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:18.000 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:18.101 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:18.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:18.143 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:18.143 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:18.206 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 34 16 68322 68306 125.529 87.625 0.00871724 0.00796125 2024-09-10T18:48:18.244 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:18.303 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:18.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:18.304 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:18.405 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:18.434 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:18.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:18.435 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:18.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:18 smithi018 ceph-mon[29336]: 6.13 scrub starts 2024-09-10T18:48:18.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:18 smithi018 ceph-mon[29336]: 6.13 scrub ok 2024-09-10T18:48:18.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:18 smithi018 ceph-mon[29336]: pgmap v107: 148 pgs: 148 active+clean; 4.0 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.2 KiB/s rd, 119 MiB/s wr, 1.90k op/s 2024-09-10T18:48:18.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:18 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:48:18.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:18 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:48:18.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:18 smithi018 ceph-mon[29336]: 4.5 scrub starts 2024-09-10T18:48:18.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:18 smithi018 ceph-mon[29336]: 4.5 scrub ok 2024-09-10T18:48:18.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:18 smithi018 ceph-mon[40196]: 6.13 scrub starts 2024-09-10T18:48:18.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:18 smithi018 ceph-mon[40196]: 6.13 scrub ok 2024-09-10T18:48:18.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:18 smithi018 ceph-mon[40196]: pgmap v107: 148 pgs: 148 active+clean; 4.0 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.2 KiB/s rd, 119 MiB/s wr, 1.90k op/s 2024-09-10T18:48:18.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:18 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:48:18.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:18 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:48:18.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:18 smithi018 ceph-mon[40196]: 4.5 scrub starts 2024-09-10T18:48:18.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:18 smithi018 ceph-mon[40196]: 4.5 scrub ok 2024-09-10T18:48:18.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:18 smithi103 ceph-mon[33163]: 6.13 scrub starts 2024-09-10T18:48:18.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:18 smithi103 ceph-mon[33163]: 6.13 scrub ok 2024-09-10T18:48:18.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:18 smithi103 ceph-mon[33163]: pgmap v107: 148 pgs: 148 active+clean; 4.0 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.2 KiB/s rd, 119 MiB/s wr, 1.90k op/s 2024-09-10T18:48:18.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:18 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:48:18.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:18 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:48:18.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:18 smithi103 ceph-mon[33163]: 4.5 scrub starts 2024-09-10T18:48:18.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:18 smithi103 ceph-mon[33163]: 4.5 scrub ok 2024-09-10T18:48:18.536 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:18.557 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:18.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:18.558 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:18.659 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:18.694 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:18.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:18.695 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:18.796 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:18.830 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:18.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:18.831 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:18.931 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:18.959 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:18.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:18.960 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:19.061 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:19.089 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:19.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:19.090 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:19.190 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:19.206 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 35 16 69635 69619 124.287 82.0625 0.0151138 0.00804055 2024-09-10T18:48:19.217 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:19.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:19.218 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:19.319 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:19.363 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:19.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:19.365 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:19.465 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:19.493 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:19.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:19.495 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:19.517 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:19 smithi018 ceph-mon[29336]: osdmap e104: 8 total, 8 up, 7 in 2024-09-10T18:48:19.518 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:19 smithi018 ceph-mon[29336]: osdmap e105: 8 total, 8 up, 7 in 2024-09-10T18:48:19.518 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:19 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:48:19.518 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:19 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:48:19.518 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:19 smithi018 ceph-mon[29336]: 5.e scrub starts 2024-09-10T18:48:19.518 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:19 smithi018 ceph-mon[29336]: 5.e scrub ok 2024-09-10T18:48:19.519 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:19 smithi018 ceph-mon[40196]: osdmap e104: 8 total, 8 up, 7 in 2024-09-10T18:48:19.519 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:19 smithi018 ceph-mon[40196]: osdmap e105: 8 total, 8 up, 7 in 2024-09-10T18:48:19.519 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:19 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:48:19.519 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:19 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:48:19.520 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:19 smithi018 ceph-mon[40196]: 5.e scrub starts 2024-09-10T18:48:19.520 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:19 smithi018 ceph-mon[40196]: 5.e scrub ok 2024-09-10T18:48:19.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:19 smithi103 ceph-mon[33163]: osdmap e104: 8 total, 8 up, 7 in 2024-09-10T18:48:19.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:19 smithi103 ceph-mon[33163]: osdmap e105: 8 total, 8 up, 7 in 2024-09-10T18:48:19.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:19 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:48:19.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:19 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:48:19.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:19 smithi103 ceph-mon[33163]: 5.e scrub starts 2024-09-10T18:48:19.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:19 smithi103 ceph-mon[33163]: 5.e scrub ok 2024-09-10T18:48:19.595 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:19.624 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:19.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:19.625 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:19.726 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:19.768 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:19.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:19.769 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:19.870 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:19.896 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:19.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:19.897 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:19.998 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:20.036 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:20.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:20.037 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:20.138 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:20.177 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:20.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:20.178 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:20.206 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 36 16 70868 70852 122.976 77.0625 0.00907497 0.00812718 2024-09-10T18:48:20.280 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:20.313 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:20.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:20.314 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:20.415 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:20.458 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:20.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:20.459 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:20.515 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:20 smithi018 ceph-mon[40196]: pgmap v110: 148 pgs: 148 active+clean; 4.0 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 680 B/s rd, 59 MiB/s wr, 938 op/s 2024-09-10T18:48:20.515 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:20 smithi018 ceph-mon[40196]: 4.15 scrub starts 2024-09-10T18:48:20.515 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:20 smithi018 ceph-mon[40196]: 4.15 scrub ok 2024-09-10T18:48:20.515 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:20 smithi018 ceph-mon[40196]: 5.1f scrub starts 2024-09-10T18:48:20.516 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:20 smithi018 ceph-mon[40196]: 5.1f scrub ok 2024-09-10T18:48:20.516 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:20 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:48:20.516 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:20 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:48:20.516 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:20 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:48:20.516 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:20 smithi018 ceph-mon[29336]: pgmap v110: 148 pgs: 148 active+clean; 4.0 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 680 B/s rd, 59 MiB/s wr, 938 op/s 2024-09-10T18:48:20.516 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:20 smithi018 ceph-mon[29336]: 4.15 scrub starts 2024-09-10T18:48:20.516 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:20 smithi018 ceph-mon[29336]: 4.15 scrub ok 2024-09-10T18:48:20.516 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:20 smithi018 ceph-mon[29336]: 5.1f scrub starts 2024-09-10T18:48:20.516 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:20 smithi018 ceph-mon[29336]: 5.1f scrub ok 2024-09-10T18:48:20.516 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:20 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:48:20.516 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:20 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:48:20.517 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:20 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:48:20.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:20 smithi103 ceph-mon[33163]: pgmap v110: 148 pgs: 148 active+clean; 4.0 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 680 B/s rd, 59 MiB/s wr, 938 op/s 2024-09-10T18:48:20.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:20 smithi103 ceph-mon[33163]: 4.15 scrub starts 2024-09-10T18:48:20.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:20 smithi103 ceph-mon[33163]: 4.15 scrub ok 2024-09-10T18:48:20.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:20 smithi103 ceph-mon[33163]: 5.1f scrub starts 2024-09-10T18:48:20.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:20 smithi103 ceph-mon[33163]: 5.1f scrub ok 2024-09-10T18:48:20.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:20 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:48:20.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:20 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:48:20.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:20 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:48:20.523 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:48:20 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:48:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:48:20.560 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:20.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:20.589 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:20.590 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:20.691 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:20.722 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:20.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:20.724 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:20.824 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:20.851 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:20.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:20.852 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:20.953 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:20.991 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:20.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:20.992 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:21.093 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:21.123 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:21.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:21.124 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:21.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:48:20 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:48:20] "GET /metrics HTTP/1.1" 200 223453 "" "Prometheus/2.43.0" 2024-09-10T18:48:21.206 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 37 16 72327 72311 122.116 91.1875 0.0132518 0.00818411 2024-09-10T18:48:21.224 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:21.269 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:21.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:21.270 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:21.370 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:21.409 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:21.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:21.410 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:21.510 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:21.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:21 smithi103 ceph-mon[33163]: 2.17 scrub starts 2024-09-10T18:48:21.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:21 smithi103 ceph-mon[33163]: 2.17 scrub ok 2024-09-10T18:48:21.538 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:21.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:21.539 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:21.613 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:21 smithi018 ceph-mon[29336]: 2.17 scrub starts 2024-09-10T18:48:21.613 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:21 smithi018 ceph-mon[29336]: 2.17 scrub ok 2024-09-10T18:48:21.614 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:21 smithi018 ceph-mon[40196]: 2.17 scrub starts 2024-09-10T18:48:21.614 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:21 smithi018 ceph-mon[40196]: 2.17 scrub ok 2024-09-10T18:48:21.640 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:21.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:21.697 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:21.697 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:21.798 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:21.822 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:21.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:21.822 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:21.923 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:21.974 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:21.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:21.975 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:22.076 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:22.114 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:22.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:22.116 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:22.207 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 38 16 73657 73641 121.09 83.125 0.0167112 0.00825299 2024-09-10T18:48:22.216 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:22.245 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:22.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:22.246 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:22.347 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:22.397 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:22.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:22.398 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:22.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:22 smithi018 ceph-mon[40196]: 4.0 scrub starts 2024-09-10T18:48:22.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:22 smithi018 ceph-mon[40196]: 4.0 scrub ok 2024-09-10T18:48:22.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:22 smithi018 ceph-mon[40196]: 4.2 scrub starts 2024-09-10T18:48:22.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:22 smithi018 ceph-mon[40196]: 4.2 scrub ok 2024-09-10T18:48:22.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:22 smithi018 ceph-mon[40196]: pgmap v111: 148 pgs: 148 active+clean; 4.2 GiB data, 14 GiB used, 611 GiB / 626 GiB avail; 639 B/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-10T18:48:22.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:22 smithi018 ceph-mon[29336]: 4.0 scrub starts 2024-09-10T18:48:22.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:22 smithi018 ceph-mon[29336]: 4.0 scrub ok 2024-09-10T18:48:22.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:22 smithi018 ceph-mon[29336]: 4.2 scrub starts 2024-09-10T18:48:22.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:22 smithi018 ceph-mon[29336]: 4.2 scrub ok 2024-09-10T18:48:22.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:22 smithi018 ceph-mon[29336]: pgmap v111: 148 pgs: 148 active+clean; 4.2 GiB data, 14 GiB used, 611 GiB / 626 GiB avail; 639 B/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-10T18:48:22.499 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:22.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:22 smithi103 ceph-mon[33163]: 4.0 scrub starts 2024-09-10T18:48:22.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:22 smithi103 ceph-mon[33163]: 4.0 scrub ok 2024-09-10T18:48:22.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:22 smithi103 ceph-mon[33163]: 4.2 scrub starts 2024-09-10T18:48:22.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:22 smithi103 ceph-mon[33163]: 4.2 scrub ok 2024-09-10T18:48:22.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:22 smithi103 ceph-mon[33163]: pgmap v111: 148 pgs: 148 active+clean; 4.2 GiB data, 14 GiB used, 611 GiB / 626 GiB avail; 639 B/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-10T18:48:22.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:22.532 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:22.532 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:22.633 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:22.655 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:22.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:22.656 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:22.757 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:22.784 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:22.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:22.785 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:22.886 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:22.912 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:22.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:22.913 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:23.014 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:23.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:23.066 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:23.066 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:23.166 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:23.190 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:23.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:23.191 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:23.206 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 39 16 74809 74793 119.832 72 0.0126387 0.00833972 2024-09-10T18:48:23.263 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:48:23.263 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:48:23.292 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:23.300 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:23 smithi018 ceph-mon[40196]: 6.4 scrub starts 2024-09-10T18:48:23.300 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:23 smithi018 ceph-mon[40196]: 6.4 scrub ok 2024-09-10T18:48:23.300 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:23 smithi018 ceph-mon[40196]: 5.6 scrub starts 2024-09-10T18:48:23.300 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:23 smithi018 ceph-mon[40196]: 5.6 scrub ok 2024-09-10T18:48:23.300 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:23 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1253994905' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:48:23.302 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:48:23.302 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:48:23.302 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:23 smithi018 ceph-mon[29336]: 6.4 scrub starts 2024-09-10T18:48:23.302 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:23 smithi018 ceph-mon[29336]: 6.4 scrub ok 2024-09-10T18:48:23.303 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:23 smithi018 ceph-mon[29336]: 5.6 scrub starts 2024-09-10T18:48:23.303 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:23 smithi018 ceph-mon[29336]: 5.6 scrub ok 2024-09-10T18:48:23.303 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:23 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1253994905' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:48:23.307 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:48:23.312 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:48:23.316 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:23.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:23.317 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:23.354 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:48:23.354 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:48:23.397 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:48:23.398 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:48:23.417 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:23.432 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:48:23.432 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:48:23.464 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:23.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:23.465 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:23.467 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:48:23.467 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:48:23.497 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:48:23.498 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:48:23.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:23 smithi103 ceph-mon[33163]: 6.4 scrub starts 2024-09-10T18:48:23.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:23 smithi103 ceph-mon[33163]: 6.4 scrub ok 2024-09-10T18:48:23.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:23 smithi103 ceph-mon[33163]: 5.6 scrub starts 2024-09-10T18:48:23.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:23 smithi103 ceph-mon[33163]: 5.6 scrub ok 2024-09-10T18:48:23.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:23 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1253994905' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:48:23.530 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:48:23.530 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:48:23.567 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:23.609 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:23.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:23.610 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:23.711 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:23.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:23.755 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:23.756 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:23.856 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:23.883 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:23.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:23.885 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:23.986 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:24.015 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:24.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:24.016 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:24.117 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:24.144 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:24.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:24.145 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:24.207 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:2024-09-10T18:48:24.206460+0000 min lat: 0.00126001 max lat: 0.14754 avg lat: 0.00843309 2024-09-10T18:48:24.207 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T18:48:24.207 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 40 16 75880 75864 118.509 66.9375 0.0129916 0.00843309 2024-09-10T18:48:24.245 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:24.265 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is unset 2024-09-10T18:48:24.285 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:24.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:24.286 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:24.387 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:24.428 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:24.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:24.429 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:24.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:24 smithi018 ceph-mon[40196]: 4.b scrub starts 2024-09-10T18:48:24.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:24 smithi018 ceph-mon[40196]: 4.b scrub ok 2024-09-10T18:48:24.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:24 smithi018 ceph-mon[40196]: pgmap v112: 148 pgs: 148 active+clean; 4.4 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 108 MiB/s wr, 1.73k op/s 2024-09-10T18:48:24.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:24 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2810563909' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:48:24.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:24 smithi018 ceph-mon[40196]: 4.1f scrub starts 2024-09-10T18:48:24.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:24 smithi018 ceph-mon[40196]: 4.1f scrub ok 2024-09-10T18:48:24.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:24 smithi018 ceph-mon[29336]: 4.b scrub starts 2024-09-10T18:48:24.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:24 smithi018 ceph-mon[29336]: 4.b scrub ok 2024-09-10T18:48:24.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:24 smithi018 ceph-mon[29336]: pgmap v112: 148 pgs: 148 active+clean; 4.4 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 108 MiB/s wr, 1.73k op/s 2024-09-10T18:48:24.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:24 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2810563909' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:48:24.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:24 smithi018 ceph-mon[29336]: 4.1f scrub starts 2024-09-10T18:48:24.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:24 smithi018 ceph-mon[29336]: 4.1f scrub ok 2024-09-10T18:48:24.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:24 smithi103 ceph-mon[33163]: 4.b scrub starts 2024-09-10T18:48:24.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:24 smithi103 ceph-mon[33163]: 4.b scrub ok 2024-09-10T18:48:24.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:24 smithi103 ceph-mon[33163]: pgmap v112: 148 pgs: 148 active+clean; 4.4 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 108 MiB/s wr, 1.73k op/s 2024-09-10T18:48:24.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:24 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2810563909' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:48:24.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:24 smithi103 ceph-mon[33163]: 4.1f scrub starts 2024-09-10T18:48:24.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:24 smithi103 ceph-mon[33163]: 4.1f scrub ok 2024-09-10T18:48:24.530 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:24.578 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:24.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:24.579 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:24.680 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:24.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:24.708 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:24.709 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:24.809 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:24.839 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:24.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:24.840 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:24.941 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:24.979 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:24.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:24.980 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:25.081 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:25.110 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:25.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:25.113 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:25.207 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 41 16 77184 77168 117.605 81.5 0.00955001 0.00849882 2024-09-10T18:48:25.214 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:25.239 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:25.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:25.240 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:25.341 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:25.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:25.385 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:25.386 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:25.487 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:25.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:25 smithi103 ceph-mon[33163]: 5.0 scrub starts 2024-09-10T18:48:25.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:25 smithi103 ceph-mon[33163]: 5.0 scrub ok 2024-09-10T18:48:25.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:25 smithi103 ceph-mon[33163]: 4.1e scrub starts 2024-09-10T18:48:25.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:25 smithi103 ceph-mon[33163]: 4.1e scrub ok 2024-09-10T18:48:25.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:25 smithi103 ceph-mon[33163]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:48:25.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:25 smithi103 ceph-mon[33163]: Cluster is now healthy 2024-09-10T18:48:25.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:25 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2810563909' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:48:25.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:25 smithi103 ceph-mon[33163]: osdmap e106: 8 total, 8 up, 7 in 2024-09-10T18:48:25.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:25 smithi103 ceph-mon[33163]: 2.11 scrub starts 2024-09-10T18:48:25.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:25 smithi103 ceph-mon[33163]: 2.11 scrub ok 2024-09-10T18:48:25.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:25 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:48:25.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:25 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:48:25.525 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:25.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:25.527 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:25.627 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:25.650 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:25 smithi018 ceph-mon[40196]: 5.0 scrub starts 2024-09-10T18:48:25.650 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:25 smithi018 ceph-mon[40196]: 5.0 scrub ok 2024-09-10T18:48:25.650 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:25 smithi018 ceph-mon[40196]: 4.1e scrub starts 2024-09-10T18:48:25.651 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:25 smithi018 ceph-mon[40196]: 4.1e scrub ok 2024-09-10T18:48:25.651 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:25 smithi018 ceph-mon[40196]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:48:25.651 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:25 smithi018 ceph-mon[40196]: Cluster is now healthy 2024-09-10T18:48:25.651 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:25 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2810563909' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:48:25.651 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:25 smithi018 ceph-mon[40196]: osdmap e106: 8 total, 8 up, 7 in 2024-09-10T18:48:25.651 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:25 smithi018 ceph-mon[40196]: 2.11 scrub starts 2024-09-10T18:48:25.652 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:25 smithi018 ceph-mon[40196]: 2.11 scrub ok 2024-09-10T18:48:25.652 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:25 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:48:25.652 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:25 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:48:25.653 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:25 smithi018 ceph-mon[29336]: 5.0 scrub starts 2024-09-10T18:48:25.653 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:25 smithi018 ceph-mon[29336]: 5.0 scrub ok 2024-09-10T18:48:25.653 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:25 smithi018 ceph-mon[29336]: 4.1e scrub starts 2024-09-10T18:48:25.653 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:25 smithi018 ceph-mon[29336]: 4.1e scrub ok 2024-09-10T18:48:25.653 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:25 smithi018 ceph-mon[29336]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:48:25.653 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:25 smithi018 ceph-mon[29336]: Cluster is now healthy 2024-09-10T18:48:25.654 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:25 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2810563909' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:48:25.654 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:25 smithi018 ceph-mon[29336]: osdmap e106: 8 total, 8 up, 7 in 2024-09-10T18:48:25.654 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:25 smithi018 ceph-mon[29336]: 2.11 scrub starts 2024-09-10T18:48:25.654 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:25 smithi018 ceph-mon[29336]: 2.11 scrub ok 2024-09-10T18:48:25.654 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:25 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:48:25.654 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:25 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:48:25.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:25.668 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:25.668 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:25.769 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:25.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:25.813 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:25.814 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:25.915 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:25.958 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:25.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:25.960 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:26.061 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:26.111 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:26.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:26.112 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:26.207 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 42 16 78548 78532 116.835 85.25 0.00848048 0.00855445 2024-09-10T18:48:26.213 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:26.253 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:26.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:26.255 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:26.356 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:26.387 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:26.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:26.388 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:26.489 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:26.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:48:26 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:48:26.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:26 smithi103 ceph-mon[33163]: 6.1b scrub starts 2024-09-10T18:48:26.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:26 smithi103 ceph-mon[33163]: 6.1b scrub ok 2024-09-10T18:48:26.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:26 smithi103 ceph-mon[33163]: pgmap v114: 148 pgs: 148 active+clean; 4.4 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 746 B/s rd, 62 MiB/s wr, 985 op/s 2024-09-10T18:48:26.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:26 smithi103 ceph-mon[33163]: 5.f scrub starts 2024-09-10T18:48:26.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:26 smithi103 ceph-mon[33163]: 5.f scrub ok 2024-09-10T18:48:26.529 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:26.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:26.531 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:26.548 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:26 smithi018 ceph-mon[29336]: 6.1b scrub starts 2024-09-10T18:48:26.549 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:26 smithi018 ceph-mon[29336]: 6.1b scrub ok 2024-09-10T18:48:26.549 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:26 smithi018 ceph-mon[29336]: pgmap v114: 148 pgs: 148 active+clean; 4.4 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 746 B/s rd, 62 MiB/s wr, 985 op/s 2024-09-10T18:48:26.549 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:26 smithi018 ceph-mon[29336]: 5.f scrub starts 2024-09-10T18:48:26.549 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:26 smithi018 ceph-mon[29336]: 5.f scrub ok 2024-09-10T18:48:26.549 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:26 smithi018 ceph-mon[40196]: 6.1b scrub starts 2024-09-10T18:48:26.549 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:26 smithi018 ceph-mon[40196]: 6.1b scrub ok 2024-09-10T18:48:26.549 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:26 smithi018 ceph-mon[40196]: pgmap v114: 148 pgs: 148 active+clean; 4.4 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 746 B/s rd, 62 MiB/s wr, 985 op/s 2024-09-10T18:48:26.549 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:26 smithi018 ceph-mon[40196]: 5.f scrub starts 2024-09-10T18:48:26.549 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:26 smithi018 ceph-mon[40196]: 5.f scrub ok 2024-09-10T18:48:26.631 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:26.677 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:26.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:26.678 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:26.779 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:26.828 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:26.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:26.829 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:26.930 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:26.959 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:26.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:26.960 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:27.061 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:27.099 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:27.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:27.101 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:27.201 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:27.207 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 43 16 79848 79832 116.007 81.25 0.0151966 0.00861427 2024-09-10T18:48:27.244 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:27.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:27.245 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:27.346 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:27.386 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:27.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:27.387 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:27.488 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:27.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:27 smithi103 ceph-mon[33163]: 6.9 scrub starts 2024-09-10T18:48:27.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:27 smithi103 ceph-mon[33163]: 6.9 scrub ok 2024-09-10T18:48:27.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:27 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:27.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:27 smithi103 ceph-mon[33163]: 2.b scrub starts 2024-09-10T18:48:27.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:27 smithi103 ceph-mon[33163]: 2.b scrub ok 2024-09-10T18:48:27.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:27 smithi103 ceph-mon[33163]: 5.1 scrub starts 2024-09-10T18:48:27.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:27 smithi103 ceph-mon[33163]: 5.1 scrub ok 2024-09-10T18:48:27.523 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:27 smithi018 ceph-mon[40196]: 6.9 scrub starts 2024-09-10T18:48:27.523 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:27 smithi018 ceph-mon[40196]: 6.9 scrub ok 2024-09-10T18:48:27.524 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:27 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:27.524 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:27 smithi018 ceph-mon[40196]: 2.b scrub starts 2024-09-10T18:48:27.524 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:27 smithi018 ceph-mon[40196]: 2.b scrub ok 2024-09-10T18:48:27.524 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:27 smithi018 ceph-mon[40196]: 5.1 scrub starts 2024-09-10T18:48:27.524 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:27 smithi018 ceph-mon[40196]: 5.1 scrub ok 2024-09-10T18:48:27.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:27 smithi018 ceph-mon[29336]: 6.9 scrub starts 2024-09-10T18:48:27.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:27 smithi018 ceph-mon[29336]: 6.9 scrub ok 2024-09-10T18:48:27.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:27 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:27.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:27 smithi018 ceph-mon[29336]: 2.b scrub starts 2024-09-10T18:48:27.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:27 smithi018 ceph-mon[29336]: 2.b scrub ok 2024-09-10T18:48:27.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:27 smithi018 ceph-mon[29336]: 5.1 scrub starts 2024-09-10T18:48:27.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:27 smithi018 ceph-mon[29336]: 5.1 scrub ok 2024-09-10T18:48:27.528 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:27.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:27.529 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:27.630 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:27.662 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:27.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:27.663 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:27.694 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T18:48:27.764 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:27.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:27.819 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:27.820 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:27.840 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T18:48:27.840 INFO:teuthology.orchestra.run.smithi018.stdout: "ops": [], 2024-09-10T18:48:27.841 INFO:teuthology.orchestra.run.smithi018.stdout: "complaint_time": 30, 2024-09-10T18:48:27.841 INFO:teuthology.orchestra.run.smithi018.stdout: "num_blocked_ops": 0 2024-09-10T18:48:27.841 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T18:48:27.920 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:27.971 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:27.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:27.973 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:28.074 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:28.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:28.119 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:28.119 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:28.207 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 44 15 81056 81041 115.088 75.5625 0.0067404 0.00868408 2024-09-10T18:48:28.220 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:28.260 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:28.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:28.261 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:28.342 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-10T18:48:28.362 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:28.386 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:28 smithi018 ceph-mon[40196]: pgmap v115: 148 pgs: 148 active+clean; 4.8 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.2 KiB/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-10T18:48:28.386 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:28 smithi018 ceph-mon[40196]: 5.11 scrub starts 2024-09-10T18:48:28.386 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:28 smithi018 ceph-mon[40196]: 5.11 scrub ok 2024-09-10T18:48:28.386 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:28 smithi018 ceph-mon[29336]: pgmap v115: 148 pgs: 148 active+clean; 4.8 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.2 KiB/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-10T18:48:28.386 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:28 smithi018 ceph-mon[29336]: 5.11 scrub starts 2024-09-10T18:48:28.386 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:28 smithi018 ceph-mon[29336]: 5.11 scrub ok 2024-09-10T18:48:28.388 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:28.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:28.389 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:28.490 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:28.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:28 smithi103 ceph-mon[33163]: pgmap v115: 148 pgs: 148 active+clean; 4.8 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.2 KiB/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-10T18:48:28.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:28 smithi103 ceph-mon[33163]: 5.11 scrub starts 2024-09-10T18:48:28.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:28 smithi103 ceph-mon[33163]: 5.11 scrub ok 2024-09-10T18:48:28.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:28.544 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:28.544 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:28.645 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:28.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:28.709 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:28.710 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:28.811 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:28.837 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:28.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:28.838 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:28.939 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:28.977 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:28.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:28.978 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:29.079 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:29.108 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:29.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:29.109 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:29.207 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 45 16 82277 82261 114.225 76.25 0.0101447 0.00875009 2024-09-10T18:48:29.210 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:29.239 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:29.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:29.240 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:29.341 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:29.385 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:29.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:29.386 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:29.487 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:29.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:29.520 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:29.520 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:29.621 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:29.660 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:29.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:29.661 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:29.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:29 smithi018 ceph-mon[29336]: 5.17 deep-scrub starts 2024-09-10T18:48:29.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:29 smithi018 ceph-mon[29336]: 5.17 deep-scrub ok 2024-09-10T18:48:29.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:29 smithi018 ceph-mon[29336]: osdmap e107: 8 total, 8 up, 7 in 2024-09-10T18:48:29.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:29 smithi018 ceph-mon[29336]: 4.6 scrub starts 2024-09-10T18:48:29.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:29 smithi018 ceph-mon[29336]: 4.6 scrub ok 2024-09-10T18:48:29.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:29 smithi018 ceph-mon[29336]: 5.18 deep-scrub starts 2024-09-10T18:48:29.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:29 smithi018 ceph-mon[29336]: 5.18 deep-scrub ok 2024-09-10T18:48:29.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:29 smithi018 ceph-mon[29336]: pgmap v117: 148 pgs: 148 active+clean; 4.8 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.2 KiB/s rd, 75 MiB/s wr, 1.21k op/s 2024-09-10T18:48:29.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:29 smithi018 ceph-mon[40196]: 5.17 deep-scrub starts 2024-09-10T18:48:29.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:29 smithi018 ceph-mon[40196]: 5.17 deep-scrub ok 2024-09-10T18:48:29.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:29 smithi018 ceph-mon[40196]: osdmap e107: 8 total, 8 up, 7 in 2024-09-10T18:48:29.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:29 smithi018 ceph-mon[40196]: 4.6 scrub starts 2024-09-10T18:48:29.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:29 smithi018 ceph-mon[40196]: 4.6 scrub ok 2024-09-10T18:48:29.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:29 smithi018 ceph-mon[40196]: 5.18 deep-scrub starts 2024-09-10T18:48:29.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:29 smithi018 ceph-mon[40196]: 5.18 deep-scrub ok 2024-09-10T18:48:29.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:29 smithi018 ceph-mon[40196]: pgmap v117: 148 pgs: 148 active+clean; 4.8 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.2 KiB/s rd, 75 MiB/s wr, 1.21k op/s 2024-09-10T18:48:29.762 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:29.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:29 smithi103 ceph-mon[33163]: 5.17 deep-scrub starts 2024-09-10T18:48:29.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:29 smithi103 ceph-mon[33163]: 5.17 deep-scrub ok 2024-09-10T18:48:29.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:29 smithi103 ceph-mon[33163]: osdmap e107: 8 total, 8 up, 7 in 2024-09-10T18:48:29.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:29 smithi103 ceph-mon[33163]: 4.6 scrub starts 2024-09-10T18:48:29.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:29 smithi103 ceph-mon[33163]: 4.6 scrub ok 2024-09-10T18:48:29.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:29 smithi103 ceph-mon[33163]: 5.18 deep-scrub starts 2024-09-10T18:48:29.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:29 smithi103 ceph-mon[33163]: 5.18 deep-scrub ok 2024-09-10T18:48:29.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:29 smithi103 ceph-mon[33163]: pgmap v117: 148 pgs: 148 active+clean; 4.8 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.2 KiB/s rd, 75 MiB/s wr, 1.21k op/s 2024-09-10T18:48:29.793 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:29.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:29.794 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:29.830 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-10T18:48:29.896 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:29.924 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:29.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:29.925 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:30.026 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:30.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:30.065 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:30.065 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:30.160 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:48:30.167 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:30.178 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:48:30.192 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:30.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:30.193 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:30.208 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 46 16 83504 83488 113.407 76.6875 0.0317961 0.00881168 2024-09-10T18:48:30.294 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:30.322 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:30.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:30.324 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:30.424 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:30.452 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:30.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:30.453 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:30.514 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:48:30 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:48:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:48:30.531 INFO:teuthology.orchestra.run.smithi018.stderr:set 6.18 pg_upmap_items mapping to [6->7,5->1,3->0] 2024-09-10T18:48:30.554 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:30.608 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:30.608 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:30 smithi018 ceph-mon[29336]: 6.12 scrub starts 2024-09-10T18:48:30.608 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:30 smithi018 ceph-mon[29336]: 6.12 scrub ok 2024-09-10T18:48:30.609 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:30 smithi018 ceph-mon[29336]: 2.1e scrub starts 2024-09-10T18:48:30.609 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:30 smithi018 ceph-mon[29336]: 2.1e scrub ok 2024-09-10T18:48:30.609 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:30 smithi018 ceph-mon[29336]: 2.a scrub starts 2024-09-10T18:48:30.609 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:30 smithi018 ceph-mon[29336]: 2.a scrub ok 2024-09-10T18:48:30.609 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:30 smithi018 ceph-mon[29336]: 2.1f deep-scrub starts 2024-09-10T18:48:30.609 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:30 smithi018 ceph-mon[29336]: 2.1f deep-scrub ok 2024-09-10T18:48:30.609 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:30 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/412404302' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "6.18", "id": [6, 7, 5, 1, 3, 0]}]: dispatch 2024-09-10T18:48:30.609 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:30 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "6.18", "id": [6, 7, 5, 1, 3, 0]}]: dispatch 2024-09-10T18:48:30.609 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:30 smithi018 ceph-mon[40196]: 6.12 scrub starts 2024-09-10T18:48:30.609 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:30 smithi018 ceph-mon[40196]: 6.12 scrub ok 2024-09-10T18:48:30.609 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:30 smithi018 ceph-mon[40196]: 2.1e scrub starts 2024-09-10T18:48:30.610 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:30 smithi018 ceph-mon[40196]: 2.1e scrub ok 2024-09-10T18:48:30.610 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:30 smithi018 ceph-mon[40196]: 2.a scrub starts 2024-09-10T18:48:30.610 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:30 smithi018 ceph-mon[40196]: 2.a scrub ok 2024-09-10T18:48:30.610 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:30 smithi018 ceph-mon[40196]: 2.1f deep-scrub starts 2024-09-10T18:48:30.610 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:30 smithi018 ceph-mon[40196]: 2.1f deep-scrub ok 2024-09-10T18:48:30.610 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:30 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/412404302' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "6.18", "id": [6, 7, 5, 1, 3, 0]}]: dispatch 2024-09-10T18:48:30.610 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:30 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "6.18", "id": [6, 7, 5, 1, 3, 0]}]: dispatch 2024-09-10T18:48:30.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:30.611 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:30.712 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:30.754 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:30.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:30.755 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:30.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:30 smithi103 ceph-mon[33163]: 6.12 scrub starts 2024-09-10T18:48:30.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:30 smithi103 ceph-mon[33163]: 6.12 scrub ok 2024-09-10T18:48:30.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:30 smithi103 ceph-mon[33163]: 2.1e scrub starts 2024-09-10T18:48:30.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:30 smithi103 ceph-mon[33163]: 2.1e scrub ok 2024-09-10T18:48:30.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:30 smithi103 ceph-mon[33163]: 2.a scrub starts 2024-09-10T18:48:30.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:30 smithi103 ceph-mon[33163]: 2.a scrub ok 2024-09-10T18:48:30.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:30 smithi103 ceph-mon[33163]: 2.1f deep-scrub starts 2024-09-10T18:48:30.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:30 smithi103 ceph-mon[33163]: 2.1f deep-scrub ok 2024-09-10T18:48:30.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:30 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/412404302' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "6.18", "id": [6, 7, 5, 1, 3, 0]}]: dispatch 2024-09-10T18:48:30.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:30 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "6.18", "id": [6, 7, 5, 1, 3, 0]}]: dispatch 2024-09-10T18:48:30.855 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:30.892 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:30.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:30.893 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:30.994 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:31.034 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:31.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:31.035 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:31.136 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:31.181 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:31.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:31.182 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:31.209 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 47 16 84746 84730 112.645 77.625 0.00897334 0.00887259 2024-09-10T18:48:31.283 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:31.322 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:48:30 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:48:30] "GET /metrics HTTP/1.1" 200 223515 "" "Prometheus/2.43.0" 2024-09-10T18:48:31.330 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:31.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:31.332 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:31.432 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:31.463 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:31.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:31.465 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:31.565 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:31.594 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:31.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:31.596 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:31.641 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:31 smithi018 ceph-mon[40196]: 4.10 scrub starts 2024-09-10T18:48:31.642 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:31 smithi018 ceph-mon[40196]: 4.10 scrub ok 2024-09-10T18:48:31.642 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:31 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "6.18", "id": [6, 7, 5, 1, 3, 0]}]': finished 2024-09-10T18:48:31.642 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:31 smithi018 ceph-mon[40196]: osdmap e108: 8 total, 8 up, 7 in 2024-09-10T18:48:31.642 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:31 smithi018 ceph-mon[40196]: 2.5 scrub starts 2024-09-10T18:48:31.642 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:31 smithi018 ceph-mon[40196]: pgmap v119: 148 pgs: 148 active+clean; 5.0 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 751 B/s rd, 86 MiB/s wr, 1.37k op/s 2024-09-10T18:48:31.642 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:31 smithi018 ceph-mon[29336]: 4.10 scrub starts 2024-09-10T18:48:31.642 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:31 smithi018 ceph-mon[29336]: 4.10 scrub ok 2024-09-10T18:48:31.642 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:31 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "6.18", "id": [6, 7, 5, 1, 3, 0]}]': finished 2024-09-10T18:48:31.643 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:31 smithi018 ceph-mon[29336]: osdmap e108: 8 total, 8 up, 7 in 2024-09-10T18:48:31.643 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:31 smithi018 ceph-mon[29336]: 2.5 scrub starts 2024-09-10T18:48:31.643 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:31 smithi018 ceph-mon[29336]: pgmap v119: 148 pgs: 148 active+clean; 5.0 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 751 B/s rd, 86 MiB/s wr, 1.37k op/s 2024-09-10T18:48:31.696 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:31.726 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:31.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:31.727 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:31.828 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:31.859 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:31.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:31.860 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:31.960 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:31.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:31.994 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:31.995 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:32.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:31 smithi103 ceph-mon[33163]: 4.10 scrub starts 2024-09-10T18:48:32.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:31 smithi103 ceph-mon[33163]: 4.10 scrub ok 2024-09-10T18:48:32.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:31 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "6.18", "id": [6, 7, 5, 1, 3, 0]}]': finished 2024-09-10T18:48:32.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:31 smithi103 ceph-mon[33163]: osdmap e108: 8 total, 8 up, 7 in 2024-09-10T18:48:32.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:31 smithi103 ceph-mon[33163]: 2.5 scrub starts 2024-09-10T18:48:32.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:31 smithi103 ceph-mon[33163]: pgmap v119: 148 pgs: 148 active+clean; 5.0 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 751 B/s rd, 86 MiB/s wr, 1.37k op/s 2024-09-10T18:48:32.095 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:32.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:32.124 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:32.125 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:32.209 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 48 16 86090 86074 112.049 84 0.00860792 0.00891957 2024-09-10T18:48:32.225 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:32.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:32.256 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:32.257 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:32.357 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:32.389 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:32.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:32.390 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:32.491 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:32.525 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:32.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:32.526 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:32.627 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:32.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:32.657 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:32.657 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:32.681 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:32 smithi018 ceph-mon[40196]: 2.5 scrub ok 2024-09-10T18:48:32.681 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:32 smithi018 ceph-mon[40196]: 6.d scrub starts 2024-09-10T18:48:32.681 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:32 smithi018 ceph-mon[40196]: 6.d scrub ok 2024-09-10T18:48:32.682 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:32 smithi018 ceph-mon[40196]: 2.1a scrub starts 2024-09-10T18:48:32.682 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:32 smithi018 ceph-mon[40196]: 2.1a scrub ok 2024-09-10T18:48:32.682 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:32 smithi018 ceph-mon[40196]: 6.e deep-scrub starts 2024-09-10T18:48:32.682 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:32 smithi018 ceph-mon[40196]: 6.e deep-scrub ok 2024-09-10T18:48:32.682 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:32 smithi018 ceph-mon[29336]: 2.5 scrub ok 2024-09-10T18:48:32.682 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:32 smithi018 ceph-mon[29336]: 6.d scrub starts 2024-09-10T18:48:32.682 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:32 smithi018 ceph-mon[29336]: 6.d scrub ok 2024-09-10T18:48:32.682 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:32 smithi018 ceph-mon[29336]: 2.1a scrub starts 2024-09-10T18:48:32.682 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:32 smithi018 ceph-mon[29336]: 2.1a scrub ok 2024-09-10T18:48:32.683 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:32 smithi018 ceph-mon[29336]: 6.e deep-scrub starts 2024-09-10T18:48:32.683 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:32 smithi018 ceph-mon[29336]: 6.e deep-scrub ok 2024-09-10T18:48:32.758 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:32.790 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:32.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:32.791 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:32.892 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:32.930 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:32.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:32.931 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:33.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:32 smithi103 ceph-mon[33163]: 2.5 scrub ok 2024-09-10T18:48:33.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:32 smithi103 ceph-mon[33163]: 6.d scrub starts 2024-09-10T18:48:33.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:32 smithi103 ceph-mon[33163]: 6.d scrub ok 2024-09-10T18:48:33.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:32 smithi103 ceph-mon[33163]: 2.1a scrub starts 2024-09-10T18:48:33.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:32 smithi103 ceph-mon[33163]: 2.1a scrub ok 2024-09-10T18:48:33.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:32 smithi103 ceph-mon[33163]: 6.e deep-scrub starts 2024-09-10T18:48:33.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:32 smithi103 ceph-mon[33163]: 6.e deep-scrub ok 2024-09-10T18:48:33.032 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:33.064 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:33.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:33.065 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:33.166 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:33.198 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:33.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:33.199 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:33.209 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 49 16 87443 87427 111.488 84.5625 0.00987524 0.00896475 2024-09-10T18:48:33.300 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:33.337 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:33.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:33.338 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:33.439 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:33.481 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:33.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:33.482 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:33.583 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:33.628 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:33.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:33.630 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:33.686 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:33 smithi018 ceph-mon[29336]: 5.1b scrub starts 2024-09-10T18:48:33.686 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:33 smithi018 ceph-mon[29336]: 5.1b scrub ok 2024-09-10T18:48:33.686 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:33 smithi018 ceph-mon[29336]: 2.15 scrub starts 2024-09-10T18:48:33.686 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:33 smithi018 ceph-mon[29336]: 2.15 scrub ok 2024-09-10T18:48:33.686 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:33 smithi018 ceph-mon[29336]: 6.10 deep-scrub starts 2024-09-10T18:48:33.686 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:33 smithi018 ceph-mon[29336]: 6.10 deep-scrub ok 2024-09-10T18:48:33.686 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:33 smithi018 ceph-mon[29336]: 6.15 scrub starts 2024-09-10T18:48:33.687 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:33 smithi018 ceph-mon[29336]: 6.15 scrub ok 2024-09-10T18:48:33.687 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:33 smithi018 ceph-mon[29336]: pgmap v120: 148 pgs: 148 active+clean; 5.2 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 1.2 KiB/s rd, 97 MiB/s wr, 1.56k op/s 2024-09-10T18:48:33.687 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:33 smithi018 ceph-mon[29336]: osdmap e109: 8 total, 8 up, 7 in 2024-09-10T18:48:33.687 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:33 smithi018 ceph-mon[40196]: 5.1b scrub starts 2024-09-10T18:48:33.687 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:33 smithi018 ceph-mon[40196]: 5.1b scrub ok 2024-09-10T18:48:33.687 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:33 smithi018 ceph-mon[40196]: 2.15 scrub starts 2024-09-10T18:48:33.687 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:33 smithi018 ceph-mon[40196]: 2.15 scrub ok 2024-09-10T18:48:33.687 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:33 smithi018 ceph-mon[40196]: 6.10 deep-scrub starts 2024-09-10T18:48:33.687 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:33 smithi018 ceph-mon[40196]: 6.10 deep-scrub ok 2024-09-10T18:48:33.687 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:33 smithi018 ceph-mon[40196]: 6.15 scrub starts 2024-09-10T18:48:33.687 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:33 smithi018 ceph-mon[40196]: 6.15 scrub ok 2024-09-10T18:48:33.687 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:33 smithi018 ceph-mon[40196]: pgmap v120: 148 pgs: 148 active+clean; 5.2 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 1.2 KiB/s rd, 97 MiB/s wr, 1.56k op/s 2024-09-10T18:48:33.688 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:33 smithi018 ceph-mon[40196]: osdmap e109: 8 total, 8 up, 7 in 2024-09-10T18:48:33.716 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:48:33.730 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:33.757 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:33.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:33.759 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:33.859 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:33.897 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:33.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:33.898 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:33.999 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:34.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:33 smithi103 ceph-mon[33163]: 5.1b scrub starts 2024-09-10T18:48:34.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:33 smithi103 ceph-mon[33163]: 5.1b scrub ok 2024-09-10T18:48:34.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:33 smithi103 ceph-mon[33163]: 2.15 scrub starts 2024-09-10T18:48:34.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:33 smithi103 ceph-mon[33163]: 2.15 scrub ok 2024-09-10T18:48:34.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:33 smithi103 ceph-mon[33163]: 6.10 deep-scrub starts 2024-09-10T18:48:34.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:33 smithi103 ceph-mon[33163]: 6.10 deep-scrub ok 2024-09-10T18:48:34.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:33 smithi103 ceph-mon[33163]: 6.15 scrub starts 2024-09-10T18:48:34.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:33 smithi103 ceph-mon[33163]: 6.15 scrub ok 2024-09-10T18:48:34.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:33 smithi103 ceph-mon[33163]: pgmap v120: 148 pgs: 148 active+clean; 5.2 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 1.2 KiB/s rd, 97 MiB/s wr, 1.56k op/s 2024-09-10T18:48:34.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:33 smithi103 ceph-mon[33163]: osdmap e109: 8 total, 8 up, 7 in 2024-09-10T18:48:34.024 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:34.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:34.026 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:34.126 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:34.152 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:34.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:34.153 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:34.209 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 50 15 88660 88645 110.78 76.125 0.0127471 0.00902215 2024-09-10T18:48:34.253 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:34.282 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:34.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:34.283 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:34.383 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:34.434 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:34.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:34.435 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:34.454 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:48:34.454 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:48:34.488 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:48:34.489 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:48:34.528 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:48:34.529 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:48:34.536 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:34.557 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is set 2024-09-10T18:48:34.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:34.567 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:34.567 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:34.570 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:48:34.571 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:48:34.604 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:48:34.604 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:48:34.640 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:48:34.640 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:48:34.668 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:34.672 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:48:34.672 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:48:34.689 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:48:34.690 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:48:34.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:34.691 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:34.692 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:34.793 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:34.798 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:34 smithi018 ceph-mon[40196]: 2.2 scrub starts 2024-09-10T18:48:34.798 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:34 smithi018 ceph-mon[40196]: 2.2 scrub ok 2024-09-10T18:48:34.798 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:34 smithi018 ceph-mon[40196]: 5.2 scrub starts 2024-09-10T18:48:34.798 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:34 smithi018 ceph-mon[40196]: 5.2 scrub ok 2024-09-10T18:48:34.798 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:34 smithi018 ceph-mon[40196]: 2.13 scrub starts 2024-09-10T18:48:34.798 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:34 smithi018 ceph-mon[40196]: 2.13 scrub ok 2024-09-10T18:48:34.798 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:34 smithi018 ceph-mon[40196]: 5.9 scrub starts 2024-09-10T18:48:34.798 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:34 smithi018 ceph-mon[40196]: 5.9 scrub ok 2024-09-10T18:48:34.798 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:34 smithi018 ceph-mon[40196]: 6.17 scrub starts 2024-09-10T18:48:34.798 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:34 smithi018 ceph-mon[40196]: 6.17 scrub ok 2024-09-10T18:48:34.798 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:34 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/4237665605' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:48:34.798 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:34 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:48:34.799 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:34 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3792752101' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:48:34.799 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:34 smithi018 ceph-mon[29336]: 2.2 scrub starts 2024-09-10T18:48:34.799 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:34 smithi018 ceph-mon[29336]: 2.2 scrub ok 2024-09-10T18:48:34.799 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:34 smithi018 ceph-mon[29336]: 5.2 scrub starts 2024-09-10T18:48:34.799 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:34 smithi018 ceph-mon[29336]: 5.2 scrub ok 2024-09-10T18:48:34.799 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:34 smithi018 ceph-mon[29336]: 2.13 scrub starts 2024-09-10T18:48:34.799 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:34 smithi018 ceph-mon[29336]: 2.13 scrub ok 2024-09-10T18:48:34.799 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:34 smithi018 ceph-mon[29336]: 5.9 scrub starts 2024-09-10T18:48:34.799 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:34 smithi018 ceph-mon[29336]: 5.9 scrub ok 2024-09-10T18:48:34.799 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:34 smithi018 ceph-mon[29336]: 6.17 scrub starts 2024-09-10T18:48:34.800 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:34 smithi018 ceph-mon[29336]: 6.17 scrub ok 2024-09-10T18:48:34.800 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:34 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/4237665605' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:48:34.800 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:34 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:48:34.800 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:34 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3792752101' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:48:34.824 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:34.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:34.825 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:34.926 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:34.964 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:34.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:34.965 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:35.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:34 smithi103 ceph-mon[33163]: 2.2 scrub starts 2024-09-10T18:48:35.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:34 smithi103 ceph-mon[33163]: 2.2 scrub ok 2024-09-10T18:48:35.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:34 smithi103 ceph-mon[33163]: 5.2 scrub starts 2024-09-10T18:48:35.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:34 smithi103 ceph-mon[33163]: 5.2 scrub ok 2024-09-10T18:48:35.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:34 smithi103 ceph-mon[33163]: 2.13 scrub starts 2024-09-10T18:48:35.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:34 smithi103 ceph-mon[33163]: 2.13 scrub ok 2024-09-10T18:48:35.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:34 smithi103 ceph-mon[33163]: 5.9 scrub starts 2024-09-10T18:48:35.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:34 smithi103 ceph-mon[33163]: 5.9 scrub ok 2024-09-10T18:48:35.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:34 smithi103 ceph-mon[33163]: 6.17 scrub starts 2024-09-10T18:48:35.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:34 smithi103 ceph-mon[33163]: 6.17 scrub ok 2024-09-10T18:48:35.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:34 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/4237665605' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:48:35.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:34 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:48:35.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:34 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3792752101' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:48:35.066 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:35.109 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:35.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:35.111 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:35.212 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 51 16 89876 89860 110.097 75.9375 0.0092213 0.00907813 2024-09-10T18:48:35.213 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:35.247 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:35.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:35.248 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:35.349 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:35.385 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:35.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:35.386 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:35.487 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:35.518 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:35.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:35.519 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:35.619 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:35.657 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:35.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:35.658 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:35.759 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:35.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:35.800 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:35.800 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:35.901 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:35.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:35 smithi018 ceph-mon[29336]: 2.3 scrub starts 2024-09-10T18:48:35.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:35 smithi018 ceph-mon[29336]: 2.3 scrub ok 2024-09-10T18:48:35.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:35 smithi018 ceph-mon[29336]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:48:35.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:35 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:48:35.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:35 smithi018 ceph-mon[29336]: osdmap e110: 8 total, 8 up, 7 in 2024-09-10T18:48:35.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:35 smithi018 ceph-mon[29336]: pgmap v123: 148 pgs: 148 active+clean; 5.2 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 853 B/s rd, 65 MiB/s wr, 1.03k op/s 2024-09-10T18:48:35.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:35 smithi018 ceph-mon[40196]: 2.3 scrub starts 2024-09-10T18:48:35.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:35 smithi018 ceph-mon[40196]: 2.3 scrub ok 2024-09-10T18:48:35.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:35 smithi018 ceph-mon[40196]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:48:35.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:35 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:48:35.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:35 smithi018 ceph-mon[40196]: osdmap e110: 8 total, 8 up, 7 in 2024-09-10T18:48:35.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:35 smithi018 ceph-mon[40196]: pgmap v123: 148 pgs: 148 active+clean; 5.2 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 853 B/s rd, 65 MiB/s wr, 1.03k op/s 2024-09-10T18:48:35.946 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:35.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:35.947 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:36.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:35 smithi103 ceph-mon[33163]: 2.3 scrub starts 2024-09-10T18:48:36.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:35 smithi103 ceph-mon[33163]: 2.3 scrub ok 2024-09-10T18:48:36.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:35 smithi103 ceph-mon[33163]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:48:36.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:35 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:48:36.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:35 smithi103 ceph-mon[33163]: osdmap e110: 8 total, 8 up, 7 in 2024-09-10T18:48:36.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:35 smithi103 ceph-mon[33163]: pgmap v123: 148 pgs: 148 active+clean; 5.2 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 853 B/s rd, 65 MiB/s wr, 1.03k op/s 2024-09-10T18:48:36.048 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:36.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:36.079 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:36.079 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:36.180 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:36.209 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 52 16 91185 91169 109.553 81.8125 0.0047784 0.00912304 2024-09-10T18:48:36.211 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:36.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:36.213 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:36.314 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:36.365 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-10T18:48:36.365 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T18:48:36.366 INFO:tasks.thrashosds.thrasher:Shrinking pool unique_pool_0 2024-09-10T18:48:36.366 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool unique_pool_0 by 10 min 8 2024-09-10T18:48:36.366 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T18:48:36.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:36.368 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:36.368 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:36.469 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:36.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:48:36 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:48:36.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:36.533 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:36.533 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:36.634 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:36.683 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:36.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:36.685 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:36.785 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:36.832 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:36.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:36.833 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:36.934 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:36.943 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:36 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:36.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:36 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:36.982 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:36.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:36.984 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:37.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:36 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:37.084 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:37.125 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:37.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:37.126 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:37.209 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 53 16 92449 92433 108.977 79 0.0104427 0.00917165 2024-09-10T18:48:37.227 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:37.260 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:37.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:37.261 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:37.362 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:37.409 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:37.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:37.410 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:37.511 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:37.539 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:37.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:37.540 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:37.641 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:37.659 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:37 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:48:37.659 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:37 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:48:37.659 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:37 smithi018 ceph-mon[40196]: osdmap e111: 8 total, 8 up, 7 in 2024-09-10T18:48:37.659 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:37 smithi018 ceph-mon[40196]: pgmap v125: 148 pgs: 148 active+clean; 5.6 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.7 KiB/s rd, 99 MiB/s wr, 1.59k op/s 2024-09-10T18:48:37.659 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:37 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:48:37.659 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:37 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:48:37.659 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:37 smithi018 ceph-mon[29336]: osdmap e111: 8 total, 8 up, 7 in 2024-09-10T18:48:37.659 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:37 smithi018 ceph-mon[29336]: pgmap v125: 148 pgs: 148 active+clean; 5.6 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.7 KiB/s rd, 99 MiB/s wr, 1.59k op/s 2024-09-10T18:48:37.665 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:37.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:37.666 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:37.767 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:37.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:37.806 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:37.807 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:37.907 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:37.954 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:37.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:37.956 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:37.983 INFO:teuthology.orchestra.run.smithi018.stdout:ERROR: (22) Invalid argument 2024-09-10T18:48:37.983 INFO:teuthology.orchestra.run.smithi018.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:48:38.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:37 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:48:38.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:37 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:48:38.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:37 smithi103 ceph-mon[33163]: osdmap e111: 8 total, 8 up, 7 in 2024-09-10T18:48:38.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:37 smithi103 ceph-mon[33163]: pgmap v125: 148 pgs: 148 active+clean; 5.6 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.7 KiB/s rd, 99 MiB/s wr, 1.59k op/s 2024-09-10T18:48:38.057 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:38.095 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:38.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:38.097 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:38.197 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:38.209 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 54 16 93674 93658 108.376 76.5625 0.0150069 0.00922144 2024-09-10T18:48:38.225 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:38.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:38.226 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:38.248 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:48:38.328 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:38.352 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-10T18:48:38.353 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:38.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:38.354 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:38.456 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:38.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:38.501 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:38.502 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:38.602 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:38.630 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:38.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:38.631 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:38.732 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:38.735 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-10T18:48:38.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:38.797 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:38.797 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:38.898 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:38.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:38.949 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:38.949 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:39.050 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:39.081 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:39.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:39.082 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:39.183 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:39.210 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 55 16 94790 94774 107.672 69.75 0.00573983 0.00928252 2024-09-10T18:48:39.229 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:39.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:39.230 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:39.331 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:39.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:39.366 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:39.366 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:39.437 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T18:48:39.466 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:39.491 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:39.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:39.492 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:39.593 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:39.638 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:39.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:39.640 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:39.740 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:39.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:39 smithi103 ceph-mon[33163]: pgmap v126: 148 pgs: 148 active+clean; 5.6 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 853 B/s rd, 67 MiB/s wr, 1.07k op/s 2024-09-10T18:48:39.791 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:39.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:39.792 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:39.893 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:39.937 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:39 smithi018 ceph-mon[29336]: pgmap v126: 148 pgs: 148 active+clean; 5.6 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 853 B/s rd, 67 MiB/s wr, 1.07k op/s 2024-09-10T18:48:39.937 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:39 smithi018 ceph-mon[40196]: pgmap v126: 148 pgs: 148 active+clean; 5.6 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 853 B/s rd, 67 MiB/s wr, 1.07k op/s 2024-09-10T18:48:39.940 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:39.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:39.942 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:39.984 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:48:40.043 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:40.048 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:48:40.067 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:40.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:40.068 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:40.169 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:40.200 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:40.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:40.201 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:40.211 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 56 16 95981 95965 107.077 74.4375 0.0039362 0.00933383 2024-09-10T18:48:40.301 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:40.331 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:40.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:40.332 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:40.433 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:40.461 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:40.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:40.462 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:40.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:48:40 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:48:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:48:40.564 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:40.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:40.609 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:40.609 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:40.710 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:40.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:40.757 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:40.758 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:40.858 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:40.898 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:40.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:40.899 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:41.000 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:41.040 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:41.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:41.042 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:41.071 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:48:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:48:40] "GET /metrics HTTP/1.1" 200 223564 "" "Prometheus/2.43.0" 2024-09-10T18:48:41.111 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:48:41.143 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:41.170 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:41.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:41.172 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:41.212 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 57 16 97267 97251 106.608 80.375 0.0127222 0.00937514 2024-09-10T18:48:41.272 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:41.298 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:41.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:41.299 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:41.400 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:41.422 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:41 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:48:41.422 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:41 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:48:41.422 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:41 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:48:41.422 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:41 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:48:41.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:41 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:48:41.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:41 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:48:41.436 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:41.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:41.437 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:41.538 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:41.583 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:41.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:41.584 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:41.685 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:41.726 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:41.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:41.727 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:41.828 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:41.861 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:41.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:41.862 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:41.963 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:42.002 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:42.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:42.003 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:42.104 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:42.139 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:42.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:42.140 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:42.212 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 58 16 98430 98414 106.023 72.6875 0.0170898 0.00942705 2024-09-10T18:48:42.241 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:42.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:42.267 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:42.267 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:42.368 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:42.395 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:42.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:42.396 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:42.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:42 smithi018 ceph-mon[40196]: pgmap v127: 148 pgs: 148 active+clean; 5.7 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 676 B/s rd, 78 MiB/s wr, 1.25k op/s 2024-09-10T18:48:42.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:42 smithi018 ceph-mon[29336]: pgmap v127: 148 pgs: 148 active+clean; 5.7 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 676 B/s rd, 78 MiB/s wr, 1.25k op/s 2024-09-10T18:48:42.475 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:42 smithi103 ceph-mon[33163]: pgmap v127: 148 pgs: 148 active+clean; 5.7 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 676 B/s rd, 78 MiB/s wr, 1.25k op/s 2024-09-10T18:48:42.498 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:42.537 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:42.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:42.538 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:42.639 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:42.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:42.675 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:42.676 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:42.776 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:42.811 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:42.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:42.812 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:42.835 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:48:42.840 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-09-10T18:48:42.913 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:42.948 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:42.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:42.949 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:43.050 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:43.082 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:43.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:43.083 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:43.184 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:43.212 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 59 16 99657 99641 105.526 76.6875 0.0122458 0.00947192 2024-09-10T18:48:43.227 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:43.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:43.228 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:43.329 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:43.366 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:43.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:43.367 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:43.468 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:43.501 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:43.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:43.502 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:43.602 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:43.639 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:43.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:43.640 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:43.741 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:43.782 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:43.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:43.783 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:43.884 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:43.925 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:43.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:43.926 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:44.027 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:44.056 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:44.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:44.057 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:44.158 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:44.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:44.190 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:44.190 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:44.212 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:2024-09-10T18:48:44.211704+0000 min lat: 0.00126001 max lat: 0.14754 avg lat: 0.00950521 2024-09-10T18:48:44.212 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T18:48:44.212 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 60 15 100988 100973 105.154 83.25 0.0118785 0.00950521 2024-09-10T18:48:44.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:44 smithi103 ceph-mon[33163]: from='client.24734 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:48:44.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:44 smithi103 ceph-mon[33163]: pgmap v128: 148 pgs: 148 active+clean; 5.9 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.2 KiB/s rd, 92 MiB/s wr, 1.47k op/s 2024-09-10T18:48:44.291 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:44.324 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:44.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:44.325 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:44.426 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:44.433 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:44 smithi018 ceph-mon[29336]: from='client.24734 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:48:44.433 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:44 smithi018 ceph-mon[29336]: pgmap v128: 148 pgs: 148 active+clean; 5.9 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.2 KiB/s rd, 92 MiB/s wr, 1.47k op/s 2024-09-10T18:48:44.434 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:44 smithi018 ceph-mon[40196]: from='client.24734 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:48:44.434 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:44 smithi018 ceph-mon[40196]: pgmap v128: 148 pgs: 148 active+clean; 5.9 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.2 KiB/s rd, 92 MiB/s wr, 1.47k op/s 2024-09-10T18:48:44.456 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:44.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:44.457 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:44.558 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:44.604 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:44.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:44.605 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:44.706 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:44.735 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:44.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:44.736 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:44.837 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:44.868 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:44.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:44.869 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:44.970 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:44.974 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2024-09-10T18:48:41.069675+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6164388060,"num_objects":94248,"num_object_clones":0,"num_object_copies":282744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94248,"num_whiteouts":0,"num_read":1040,"num_read_kb":897,"num_write":94595,"num_write_kb":6020999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":1183107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77340,"ondisk_log_size":77340,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":19975776,"kb_used_data":17844388,"kb_used_omap":11,"kb_used_meta":2131316,"kb_avail":636240288,"statfs":{"total":671965249536,"available":651510054912,"internally_reserved":0,"allocated":18272653312,"data_stored":18269088858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11928,"internal_metadata":2182467944},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,6,11],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":622133248,"num_objects":9493,"num_object_clones":0,"num_object_copies":28479,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9493,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":9493,"num_write_kb":607552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.565920"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":77,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.506555+0000","last_change":"2024-09-10T18:48:24.515065+0000","last_active":"2024-09-10T18:48:33.506555+0000","last_peered":"2024-09-10T18:48:33.506555+0000","last_clean":"2024-09-10T18:48:33.506555+0000","last_became_active":"2024-09-10T18:45:24.265488+0000","last_became_peered":"2024-09-10T18:45:24.265488+0000","last_unstale":"2024-09-10T18:48:33.506555+0000","last_undegraded":"2024-09-10T18:48:33.506555+0000","last_fullsized":"2024-09-10T18:48:33.506555+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:24.511946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:24.511946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:49.630089+0000","scrub_duration":0.014161415,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":83,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.509298+0000","last_change":"2024-09-10T18:48:29.718356+0000","last_active":"2024-09-10T18:48:33.509298+0000","last_peered":"2024-09-10T18:48:33.509298+0000","last_clean":"2024-09-10T18:48:33.509298+0000","last_became_active":"2024-09-10T18:45:17.959475+0000","last_became_peered":"2024-09-10T18:45:17.959475+0000","last_unstale":"2024-09-10T18:48:33.509298+0000","last_undegraded":"2024-09-10T18:48:33.509298+0000","last_fullsized":"2024-09-10T18:48:33.509298+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:29.718270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.718270+0000","last_clean_scrub_stamp":"2024-09-10T18:48:29.718270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:59.632977+0000","scrub_duration":0.010834984000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"63'15","reported_seq":122,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.507930+0000","last_change":"2024-09-10T18:47:36.436893+0000","last_active":"2024-09-10T18:48:33.507930+0000","last_peered":"2024-09-10T18:48:33.507930+0000","last_clean":"2024-09-10T18:48:33.507930+0000","last_became_active":"2024-09-10T18:47:36.413635+0000","last_became_peered":"2024-09-10T18:47:36.413635+0000","last_unstale":"2024-09-10T18:48:33.507930+0000","last_undegraded":"2024-09-10T18:48:33.507930+0000","last_fullsized":"2024-09-10T18:48:33.507930+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:46:36.856740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:36.856740+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018131040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":79,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.548883+0000","last_change":"2024-09-10T18:48:29.058001+0000","last_active":"2024-09-10T18:48:34.548883+0000","last_peered":"2024-09-10T18:48:34.548883+0000","last_clean":"2024-09-10T18:48:34.548883+0000","last_became_active":"2024-09-10T18:45:22.065619+0000","last_became_peered":"2024-09-10T18:45:22.065619+0000","last_unstale":"2024-09-10T18:48:34.548883+0000","last_undegraded":"2024-09-10T18:48:34.548883+0000","last_fullsized":"2024-09-10T18:48:34.548883+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:29.057932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.057932+0000","last_clean_scrub_stamp":"2024-09-10T18:48:29.057932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:39.110015+0000","scrub_duration":0.001460184,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":83,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.507371+0000","last_change":"2024-09-10T18:48:29.458396+0000","last_active":"2024-09-10T18:48:33.507371+0000","last_peered":"2024-09-10T18:48:33.507371+0000","last_clean":"2024-09-10T18:48:33.507371+0000","last_became_active":"2024-09-10T18:45:17.966153+0000","last_became_peered":"2024-09-10T18:45:17.966153+0000","last_unstale":"2024-09-10T18:48:33.507371+0000","last_undegraded":"2024-09-10T18:48:33.507371+0000","last_fullsized":"2024-09-10T18:48:33.507371+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:29.457892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:29.457892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:50.349265+0000","scrub_duration":0.012091342999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"83'13","reported_seq":91,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.549144+0000","last_change":"2024-09-10T18:46:59.906707+0000","last_active":"2024-09-10T18:48:34.549144+0000","last_peered":"2024-09-10T18:48:34.549144+0000","last_clean":"2024-09-10T18:48:34.549144+0000","last_became_active":"2024-09-10T18:45:20.061266+0000","last_became_peered":"2024-09-10T18:45:20.061266+0000","last_unstale":"2024-09-10T18:48:34.549144+0000","last_undegraded":"2024-09-10T18:48:34.549144+0000","last_fullsized":"2024-09-10T18:48:34.549144+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'13","last_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_clean_scrub_stamp":"2024-09-10T18:46:59.906634+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003988892,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":69,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.514933+0000","last_change":"2024-09-10T18:47:21.589428+0000","last_active":"2024-09-10T18:48:33.514933+0000","last_peered":"2024-09-10T18:48:33.514933+0000","last_clean":"2024-09-10T18:48:33.514933+0000","last_became_active":"2024-09-10T18:45:22.072822+0000","last_became_peered":"2024-09-10T18:45:22.072822+0000","last_unstale":"2024-09-10T18:48:33.514933+0000","last_undegraded":"2024-09-10T18:48:33.514933+0000","last_fullsized":"2024-09-10T18:48:33.514933+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:21.589365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:47:21.589365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:34.839266+0000","scrub_duration":0.001676893,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":67,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.550521+0000","last_change":"2024-09-10T18:47:00.946309+0000","last_active":"2024-09-10T18:48:34.550521+0000","last_peered":"2024-09-10T18:48:34.550521+0000","last_clean":"2024-09-10T18:48:34.550521+0000","last_became_active":"2024-09-10T18:45:24.081002+0000","last_became_peered":"2024-09-10T18:45:24.081002+0000","last_unstale":"2024-09-10T18:48:34.550521+0000","last_undegraded":"2024-09-10T18:48:34.550521+0000","last_fullsized":"2024-09-10T18:48:34.550521+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:00.946255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:00.946255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001207629,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":84,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.551431+0000","last_change":"2024-09-10T18:48:13.694567+0000","last_active":"2024-09-10T18:48:34.551431+0000","last_peered":"2024-09-10T18:48:34.551431+0000","last_clean":"2024-09-10T18:48:34.551431+0000","last_became_active":"2024-09-10T18:45:17.961097+0000","last_became_peered":"2024-09-10T18:45:17.961097+0000","last_unstale":"2024-09-10T18:48:34.551431+0000","last_undegraded":"2024-09-10T18:48:34.551431+0000","last_fullsized":"2024-09-10T18:48:34.551431+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:13.694295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:13.694295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:28.048348+0000","scrub_duration":0.0051419839999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"63'5","reported_seq":81,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.550518+0000","last_change":"2024-09-10T18:47:04.002228+0000","last_active":"2024-09-10T18:48:34.550518+0000","last_peered":"2024-09-10T18:48:34.550518+0000","last_clean":"2024-09-10T18:48:34.550518+0000","last_became_active":"2024-09-10T18:45:20.057567+0000","last_became_peered":"2024-09-10T18:45:20.057567+0000","last_unstale":"2024-09-10T18:48:34.550518+0000","last_undegraded":"2024-09-10T18:48:34.550518+0000","last_fullsized":"2024-09-10T18:48:34.550518+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-10T18:47:04.002164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:04.002164+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.026374999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":80,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.551124+0000","last_change":"2024-09-10T18:47:36.510027+0000","last_active":"2024-09-10T18:48:34.551124+0000","last_peered":"2024-09-10T18:48:34.551124+0000","last_clean":"2024-09-10T18:48:34.551124+0000","last_became_active":"2024-09-10T18:45:22.076575+0000","last_became_peered":"2024-09-10T18:45:22.076575+0000","last_unstale":"2024-09-10T18:48:34.551124+0000","last_undegraded":"2024-09-10T18:48:34.551124+0000","last_fullsized":"2024-09-10T18:48:34.551124+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:36.509971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:36.509971+0000","last_clean_scrub_stamp":"2024-09-10T18:47:36.509971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:47.471372+0000","scrub_duration":0.0013665439999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":77,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.561363+0000","last_change":"2024-09-10T18:48:10.102991+0000","last_active":"2024-09-10T18:48:34.561363+0000","last_peered":"2024-09-10T18:48:34.561363+0000","last_clean":"2024-09-10T18:48:34.561363+0000","last_became_active":"2024-09-10T18:45:24.079534+0000","last_became_peered":"2024-09-10T18:45:24.079534+0000","last_unstale":"2024-09-10T18:48:34.561363+0000","last_undegraded":"2024-09-10T18:48:34.561363+0000","last_fullsized":"2024-09-10T18:48:34.561363+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:10.102865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:10.102865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:36.015071+0000","scrub_duration":0.0042821509999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":84,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.551410+0000","last_change":"2024-09-10T18:47:36.417065+0000","last_active":"2024-09-10T18:48:34.551410+0000","last_peered":"2024-09-10T18:48:34.551410+0000","last_clean":"2024-09-10T18:48:34.551410+0000","last_became_active":"2024-09-10T18:47:36.414393+0000","last_became_peered":"2024-09-10T18:47:36.414393+0000","last_unstale":"2024-09-10T18:48:34.551410+0000","last_undegraded":"2024-09-10T18:48:34.551410+0000","last_fullsized":"2024-09-10T18:48:34.551410+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:50.666379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.666379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016221689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"63'9","reported_seq":87,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.550786+0000","last_change":"2024-09-10T18:46:55.069893+0000","last_active":"2024-09-10T18:48:34.550786+0000","last_peered":"2024-09-10T18:48:34.550786+0000","last_clean":"2024-09-10T18:48:34.550786+0000","last_became_active":"2024-09-10T18:45:20.052652+0000","last_became_peered":"2024-09-10T18:45:20.052652+0000","last_unstale":"2024-09-10T18:48:34.550786+0000","last_undegraded":"2024-09-10T18:48:34.550786+0000","last_fullsized":"2024-09-10T18:48:34.550786+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:46:55.069833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:55.069833+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016685510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":79,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.561517+0000","last_change":"2024-09-10T18:48:32.141849+0000","last_active":"2024-09-10T18:48:34.561517+0000","last_peered":"2024-09-10T18:48:34.561517+0000","last_clean":"2024-09-10T18:48:34.561517+0000","last_became_active":"2024-09-10T18:45:22.068326+0000","last_became_peered":"2024-09-10T18:45:22.068326+0000","last_unstale":"2024-09-10T18:48:34.561517+0000","last_undegraded":"2024-09-10T18:48:34.561517+0000","last_fullsized":"2024-09-10T18:48:34.561517+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:32.141755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:32.141755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:42.742732+0000","scrub_duration":0.0040319509999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":67,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.507825+0000","last_change":"2024-09-10T18:47:22.557898+0000","last_active":"2024-09-10T18:48:33.507825+0000","last_peered":"2024-09-10T18:48:33.507825+0000","last_clean":"2024-09-10T18:48:33.507825+0000","last_became_active":"2024-09-10T18:45:24.081488+0000","last_became_peered":"2024-09-10T18:45:24.081488+0000","last_unstale":"2024-09-10T18:48:33.507825+0000","last_undegraded":"2024-09-10T18:48:33.507825+0000","last_fullsized":"2024-09-10T18:48:33.507825+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:22.557837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:22.557837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:37.691769+0000","scrub_duration":0.001691374,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"111'6003","reported_seq":6037,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:41.016224+0000","last_change":"2024-09-10T18:47:36.423039+0000","last_active":"2024-09-10T18:48:41.016224+0000","last_peered":"2024-09-10T18:48:41.016224+0000","last_clean":"2024-09-10T18:48:41.016224+0000","last_became_active":"2024-09-10T18:47:36.422694+0000","last_became_peered":"2024-09-10T18:47:36.422694+0000","last_unstale":"2024-09-10T18:48:41.016224+0000","last_undegraded":"2024-09-10T18:48:41.016224+0000","last_fullsized":"2024-09-10T18:48:41.016224+0000","mapping_epoch":88,"log_start":"92'800","ondisk_log_start":"92'800","created":86,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":5203,"log_dups_size":0,"ondisk_log_size":5203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:49.116440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393412608,"num_objects":6003,"num_object_clones":0,"num_object_copies":18009,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6003,"num_write_kb":384192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"78'25","reported_seq":135,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.552006+0000","last_change":"2024-09-10T18:47:36.429540+0000","last_active":"2024-09-10T18:48:34.552006+0000","last_peered":"2024-09-10T18:48:34.552006+0000","last_clean":"2024-09-10T18:48:34.552006+0000","last_became_active":"2024-09-10T18:47:36.409300+0000","last_became_peered":"2024-09-10T18:47:36.409300+0000","last_unstale":"2024-09-10T18:48:34.552006+0000","last_undegraded":"2024-09-10T18:48:34.552006+0000","last_fullsized":"2024-09-10T18:48:34.552006+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'25","last_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_deep_scrub":"78'25","last_deep_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_clean_scrub_stamp":"2024-09-10T18:46:44.316146+0000","objects_scrubbed":8,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046415040000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":84,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.554484+0000","last_change":"2024-09-10T18:48:26.625704+0000","last_active":"2024-09-10T18:48:34.554484+0000","last_peered":"2024-09-10T18:48:34.554484+0000","last_clean":"2024-09-10T18:48:34.554484+0000","last_became_active":"2024-09-10T18:45:17.963732+0000","last_became_peered":"2024-09-10T18:45:17.963732+0000","last_unstale":"2024-09-10T18:48:34.554484+0000","last_undegraded":"2024-09-10T18:48:34.554484+0000","last_fullsized":"2024-09-10T18:48:34.554484+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:26.625635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:26.625635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:44.826250+0000","scrub_duration":0.0034872639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":69,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.517212+0000","last_change":"2024-09-10T18:47:04.512478+0000","last_active":"2024-09-10T18:48:33.517212+0000","last_peered":"2024-09-10T18:48:33.517212+0000","last_clean":"2024-09-10T18:48:33.517212+0000","last_became_active":"2024-09-10T18:45:22.071443+0000","last_became_peered":"2024-09-10T18:45:22.071443+0000","last_unstale":"2024-09-10T18:48:33.517212+0000","last_undegraded":"2024-09-10T18:48:33.517212+0000","last_fullsized":"2024-09-10T18:48:33.517212+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:04.512425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:04.512425+0000","last_clean_scrub_stamp":"2024-09-10T18:47:04.512425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001363324,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":23,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.508865+0000","last_change":"2024-09-10T18:47:57.104885+0000","last_active":"2024-09-10T18:48:33.508865+0000","last_peered":"2024-09-10T18:48:33.508865+0000","last_clean":"2024-09-10T18:48:33.508865+0000","last_became_active":"2024-09-10T18:47:57.104433+0000","last_became_peered":"2024-09-10T18:47:57.104433+0000","last_unstale":"2024-09-10T18:48:33.508865+0000","last_undegraded":"2024-09-10T18:48:33.508865+0000","last_fullsized":"2024-09-10T18:48:33.508865+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:38.016595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:38.016595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:45.235558+0000","scrub_duration":0.0015035510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.f","version":"110'5696","reported_seq":5728,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.030609+0000","last_change":"2024-09-10T18:47:34.396342+0000","last_active":"2024-09-10T18:48:36.030609+0000","last_peered":"2024-09-10T18:48:36.030609+0000","last_clean":"2024-09-10T18:48:36.030609+0000","last_became_active":"2024-09-10T18:47:34.396087+0000","last_became_peered":"2024-09-10T18:47:34.396087+0000","last_unstale":"2024-09-10T18:48:36.030609+0000","last_undegraded":"2024-09-10T18:48:36.030609+0000","last_fullsized":"2024-09-10T18:48:36.030609+0000","mapping_epoch":86,"log_start":"92'900","ondisk_log_start":"92'900","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":4796,"log_dups_size":0,"ondisk_log_size":4796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:46.307943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":373293056,"num_objects":5696,"num_object_clones":0,"num_object_copies":17088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5696,"num_write_kb":364544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"63'10","reported_seq":86,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.550998+0000","last_change":"2024-09-10T18:46:31.281588+0000","last_active":"2024-09-10T18:48:34.550998+0000","last_peered":"2024-09-10T18:48:34.550998+0000","last_clean":"2024-09-10T18:48:34.550998+0000","last_became_active":"2024-09-10T18:45:20.053428+0000","last_became_peered":"2024-09-10T18:45:20.053428+0000","last_unstale":"2024-09-10T18:48:34.550998+0000","last_undegraded":"2024-09-10T18:48:34.550998+0000","last_fullsized":"2024-09-10T18:48:34.550998+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:46:31.281519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:31.281519+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019095119999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":83,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.517059+0000","last_change":"2024-09-10T18:48:29.551161+0000","last_active":"2024-09-10T18:48:33.517059+0000","last_peered":"2024-09-10T18:48:33.517059+0000","last_clean":"2024-09-10T18:48:33.517059+0000","last_became_active":"2024-09-10T18:45:17.966833+0000","last_became_peered":"2024-09-10T18:45:17.966833+0000","last_unstale":"2024-09-10T18:48:33.517059+0000","last_undegraded":"2024-09-10T18:48:33.517059+0000","last_fullsized":"2024-09-10T18:48:33.517059+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:29.550946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:29.550946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:50.968295+0000","scrub_duration":0.016562341000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"89'9","reported_seq":81,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.114429+0000","last_change":"2024-09-10T18:48:09.648420+0000","last_active":"2024-09-10T18:48:36.114429+0000","last_peered":"2024-09-10T18:48:36.114429+0000","last_clean":"2024-09-10T18:48:36.114429+0000","last_became_active":"2024-09-10T18:47:36.421188+0000","last_became_peered":"2024-09-10T18:47:36.421188+0000","last_unstale":"2024-09-10T18:48:36.114429+0000","last_undegraded":"2024-09-10T18:48:36.114429+0000","last_fullsized":"2024-09-10T18:48:36.114429+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'9","last_scrub_stamp":"2024-09-10T18:48:09.648322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:09.648322+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:30.082986+0000","scrub_duration":0.0066489619999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":87,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.552536+0000","last_change":"2024-09-10T18:48:32.043742+0000","last_active":"2024-09-10T18:48:34.552536+0000","last_peered":"2024-09-10T18:48:34.552536+0000","last_clean":"2024-09-10T18:48:34.552536+0000","last_became_active":"2024-09-10T18:47:36.414625+0000","last_became_peered":"2024-09-10T18:47:36.414625+0000","last_unstale":"2024-09-10T18:48:34.552536+0000","last_undegraded":"2024-09-10T18:48:34.552536+0000","last_fullsized":"2024-09-10T18:48:34.552536+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:32.043666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:32.043666+0000","last_clean_scrub_stamp":"2024-09-10T18:48:32.043666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:49.740069+0000","scrub_duration":0.001572417,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"111'6116","reported_seq":6147,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:41.036536+0000","last_change":"2024-09-10T18:47:34.399541+0000","last_active":"2024-09-10T18:48:41.036536+0000","last_peered":"2024-09-10T18:48:41.036536+0000","last_clean":"2024-09-10T18:48:41.036536+0000","last_became_active":"2024-09-10T18:47:34.399261+0000","last_became_peered":"2024-09-10T18:47:34.399261+0000","last_unstale":"2024-09-10T18:48:41.036536+0000","last_undegraded":"2024-09-10T18:48:41.036536+0000","last_fullsized":"2024-09-10T18:48:41.036536+0000","mapping_epoch":86,"log_start":"92'900","ondisk_log_start":"92'900","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":5216,"log_dups_size":0,"ondisk_log_size":5216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:45.482983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":400818176,"num_objects":6116,"num_object_clones":0,"num_object_copies":18348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6116,"num_write_kb":391424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"63'15","reported_seq":63,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.561901+0000","last_change":"2024-09-10T18:46:58.122279+0000","last_active":"2024-09-10T18:48:34.561901+0000","last_peered":"2024-09-10T18:48:34.561901+0000","last_clean":"2024-09-10T18:48:34.561901+0000","last_became_active":"2024-09-10T18:46:56.742946+0000","last_became_peered":"2024-09-10T18:46:56.742946+0000","last_unstale":"2024-09-10T18:48:34.561901+0000","last_undegraded":"2024-09-10T18:48:34.561901+0000","last_fullsized":"2024-09-10T18:48:34.561901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:46:58.122219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:58.122219+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001678094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":83,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.515117+0000","last_change":"2024-09-10T18:48:13.707838+0000","last_active":"2024-09-10T18:48:33.515117+0000","last_peered":"2024-09-10T18:48:33.515117+0000","last_clean":"2024-09-10T18:48:33.515117+0000","last_became_active":"2024-09-10T18:45:17.965409+0000","last_became_peered":"2024-09-10T18:45:17.965409+0000","last_unstale":"2024-09-10T18:48:33.515117+0000","last_undegraded":"2024-09-10T18:48:33.515117+0000","last_fullsized":"2024-09-10T18:48:33.515117+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:13.707725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_clean_scrub_stamp":"2024-09-10T18:48:13.707725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:14.271680+0000","scrub_duration":0.0075610709999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"63'8","reported_seq":127,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.114629+0000","last_change":"2024-09-10T18:48:18.950238+0000","last_active":"2024-09-10T18:48:36.114629+0000","last_peered":"2024-09-10T18:48:36.114629+0000","last_clean":"2024-09-10T18:48:36.114629+0000","last_became_active":"2024-09-10T18:45:22.062497+0000","last_became_peered":"2024-09-10T18:45:22.062497+0000","last_unstale":"2024-09-10T18:48:36.114629+0000","last_undegraded":"2024-09-10T18:48:36.114629+0000","last_fullsized":"2024-09-10T18:48:36.114629+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:48:18.950140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:18.950140+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:34.771680+0000","scrub_duration":0.0032883439999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":77,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.561667+0000","last_change":"2024-09-10T18:48:31.170536+0000","last_active":"2024-09-10T18:48:34.561667+0000","last_peered":"2024-09-10T18:48:34.561667+0000","last_clean":"2024-09-10T18:48:34.561667+0000","last_became_active":"2024-09-10T18:45:24.082362+0000","last_became_peered":"2024-09-10T18:45:24.082362+0000","last_unstale":"2024-09-10T18:48:34.561667+0000","last_undegraded":"2024-09-10T18:48:34.561667+0000","last_fullsized":"2024-09-10T18:48:34.561667+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:31.170457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:31.170457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:50.433946+0000","scrub_duration":0.0084072260000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"111'6017","reported_seq":6049,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:41.012066+0000","last_change":"2024-09-10T18:47:34.400040+0000","last_active":"2024-09-10T18:48:41.012066+0000","last_peered":"2024-09-10T18:48:41.012066+0000","last_clean":"2024-09-10T18:48:41.012066+0000","last_became_active":"2024-09-10T18:47:34.399711+0000","last_became_peered":"2024-09-10T18:47:34.399711+0000","last_unstale":"2024-09-10T18:48:41.012066+0000","last_undegraded":"2024-09-10T18:48:41.012066+0000","last_fullsized":"2024-09-10T18:48:41.012066+0000","mapping_epoch":86,"log_start":"92'800","ondisk_log_start":"92'800","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":5217,"log_dups_size":0,"ondisk_log_size":5217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":394330112,"num_objects":6017,"num_object_clones":0,"num_object_copies":18051,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6017,"num_write_kb":385088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"63'11","reported_seq":166,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.552742+0000","last_change":"2024-09-10T18:48:11.900583+0000","last_active":"2024-09-10T18:48:34.552742+0000","last_peered":"2024-09-10T18:48:34.552742+0000","last_clean":"2024-09-10T18:48:34.552742+0000","last_became_active":"2024-09-10T18:47:57.106935+0000","last_became_peered":"2024-09-10T18:47:57.106935+0000","last_unstale":"2024-09-10T18:48:34.552742+0000","last_undegraded":"2024-09-10T18:48:34.552742+0000","last_fullsized":"2024-09-10T18:48:34.552742+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:48:11.900489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:11.900489+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:16.653578+0000","scrub_duration":0.0020768789999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":84,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.552073+0000","last_change":"2024-09-10T18:48:12.642771+0000","last_active":"2024-09-10T18:48:34.552073+0000","last_peered":"2024-09-10T18:48:34.552073+0000","last_clean":"2024-09-10T18:48:34.552073+0000","last_became_active":"2024-09-10T18:45:17.966133+0000","last_became_peered":"2024-09-10T18:45:17.966133+0000","last_unstale":"2024-09-10T18:48:34.552073+0000","last_undegraded":"2024-09-10T18:48:34.552073+0000","last_fullsized":"2024-09-10T18:48:34.552073+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:12.642693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:12.642693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:15.199200+0000","scrub_duration":0.0035654889999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":79,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.561955+0000","last_change":"2024-09-10T18:48:25.093064+0000","last_active":"2024-09-10T18:48:34.561955+0000","last_peered":"2024-09-10T18:48:34.561955+0000","last_clean":"2024-09-10T18:48:34.561955+0000","last_became_active":"2024-09-10T18:45:22.068918+0000","last_became_peered":"2024-09-10T18:45:22.068918+0000","last_unstale":"2024-09-10T18:48:34.561955+0000","last_undegraded":"2024-09-10T18:48:34.561955+0000","last_fullsized":"2024-09-10T18:48:34.561955+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:25.092984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_clean_scrub_stamp":"2024-09-10T18:48:25.092984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:45.861853+0000","scrub_duration":0.0038537620000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":67,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.506656+0000","last_change":"2024-09-10T18:47:00.743740+0000","last_active":"2024-09-10T18:48:33.506656+0000","last_peered":"2024-09-10T18:48:33.506656+0000","last_clean":"2024-09-10T18:48:33.506656+0000","last_became_active":"2024-09-10T18:45:24.264538+0000","last_became_peered":"2024-09-10T18:45:24.264538+0000","last_unstale":"2024-09-10T18:48:33.506656+0000","last_undegraded":"2024-09-10T18:48:33.506656+0000","last_fullsized":"2024-09-10T18:48:33.506656+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:00.743680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:00.743680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022093759999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2","version":"110'5693","reported_seq":5735,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.131459+0000","last_change":"2024-09-10T18:47:36.423507+0000","last_active":"2024-09-10T18:48:36.131459+0000","last_peered":"2024-09-10T18:48:36.131459+0000","last_clean":"2024-09-10T18:48:36.131459+0000","last_became_active":"2024-09-10T18:47:36.423234+0000","last_became_peered":"2024-09-10T18:47:36.423234+0000","last_unstale":"2024-09-10T18:48:36.131459+0000","last_undegraded":"2024-09-10T18:48:36.131459+0000","last_fullsized":"2024-09-10T18:48:36.131459+0000","mapping_epoch":88,"log_start":"94'1400","ondisk_log_start":"94'1400","created":86,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":4293,"log_dups_size":0,"ondisk_log_size":4293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:58.836003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":373096448,"num_objects":5693,"num_object_clones":0,"num_object_copies":17079,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5693,"num_write_kb":364352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"63'14","reported_seq":92,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.549066+0000","last_change":"2024-09-10T18:46:29.330572+0000","last_active":"2024-09-10T18:48:34.549066+0000","last_peered":"2024-09-10T18:48:34.549066+0000","last_clean":"2024-09-10T18:48:34.549066+0000","last_became_active":"2024-09-10T18:45:20.052243+0000","last_became_peered":"2024-09-10T18:45:20.052243+0000","last_unstale":"2024-09-10T18:48:34.549066+0000","last_undegraded":"2024-09-10T18:48:34.549066+0000","last_fullsized":"2024-09-10T18:48:34.549066+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-10T18:46:29.330503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:29.330503+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020437459999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":93,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.557317+0000","last_change":"2024-09-10T18:48:13.631307+0000","last_active":"2024-09-10T18:48:34.557317+0000","last_peered":"2024-09-10T18:48:34.557317+0000","last_clean":"2024-09-10T18:48:34.557317+0000","last_became_active":"2024-09-10T18:47:36.424466+0000","last_became_peered":"2024-09-10T18:47:36.424466+0000","last_unstale":"2024-09-10T18:48:34.557317+0000","last_undegraded":"2024-09-10T18:48:34.557317+0000","last_fullsized":"2024-09-10T18:48:34.557317+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:13.631078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_clean_scrub_stamp":"2024-09-10T18:48:13.631078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:29.790261+0000","scrub_duration":0.0043023660000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":127,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.565508+0000","last_change":"2024-09-10T18:48:23.479877+0000","last_active":"2024-09-10T18:48:34.565508+0000","last_peered":"2024-09-10T18:48:34.565508+0000","last_clean":"2024-09-10T18:48:34.565508+0000","last_became_active":"2024-09-10T18:45:22.074540+0000","last_became_peered":"2024-09-10T18:45:22.074540+0000","last_unstale":"2024-09-10T18:48:34.565508+0000","last_undegraded":"2024-09-10T18:48:34.565508+0000","last_fullsized":"2024-09-10T18:48:34.565508+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:48:23.479762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:23.479762+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:32.897955+0000","scrub_duration":0.01210871,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":88,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.551490+0000","last_change":"2024-09-10T18:48:08.689486+0000","last_active":"2024-09-10T18:48:34.551490+0000","last_peered":"2024-09-10T18:48:34.551490+0000","last_clean":"2024-09-10T18:48:34.551490+0000","last_became_active":"2024-09-10T18:47:36.414665+0000","last_became_peered":"2024-09-10T18:47:36.414665+0000","last_unstale":"2024-09-10T18:48:34.551490+0000","last_undegraded":"2024-09-10T18:48:34.551490+0000","last_fullsized":"2024-09-10T18:48:34.551490+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:08.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:08.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:11.141061+0000","scrub_duration":0.0033094489999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"110'5804","reported_seq":5846,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.124867+0000","last_change":"2024-09-10T18:47:36.410975+0000","last_active":"2024-09-10T18:48:36.124867+0000","last_peered":"2024-09-10T18:48:36.124867+0000","last_clean":"2024-09-10T18:48:36.124867+0000","last_became_active":"2024-09-10T18:47:36.410639+0000","last_became_peered":"2024-09-10T18:47:36.410639+0000","last_unstale":"2024-09-10T18:48:36.124867+0000","last_undegraded":"2024-09-10T18:48:36.124867+0000","last_fullsized":"2024-09-10T18:48:36.124867+0000","mapping_epoch":88,"log_start":"94'1500","ondisk_log_start":"94'1500","created":86,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":4304,"log_dups_size":0,"ondisk_log_size":4304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:59.070249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":380370944,"num_objects":5804,"num_object_clones":0,"num_object_copies":17412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5804,"num_write_kb":371456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"63'18","reported_seq":113,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.510409+0000","last_change":"2024-09-10T18:48:20.514584+0000","last_active":"2024-09-10T18:48:33.510409+0000","last_peered":"2024-09-10T18:48:33.510409+0000","last_clean":"2024-09-10T18:48:33.510409+0000","last_became_active":"2024-09-10T18:45:20.420937+0000","last_became_peered":"2024-09-10T18:45:20.420937+0000","last_unstale":"2024-09-10T18:48:33.510409+0000","last_undegraded":"2024-09-10T18:48:33.510409+0000","last_fullsized":"2024-09-10T18:48:33.510409+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'18","last_scrub_stamp":"2024-09-10T18:48:20.514501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:20.514501+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:32.154497+0000","scrub_duration":0.028109085999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":73,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.514834+0000","last_change":"2024-09-10T18:47:05.505431+0000","last_active":"2024-09-10T18:48:33.514834+0000","last_peered":"2024-09-10T18:48:33.514834+0000","last_clean":"2024-09-10T18:48:33.514834+0000","last_became_active":"2024-09-10T18:45:17.961695+0000","last_became_peered":"2024-09-10T18:45:17.961695+0000","last_unstale":"2024-09-10T18:48:33.514834+0000","last_undegraded":"2024-09-10T18:48:33.514834+0000","last_fullsized":"2024-09-10T18:48:33.514834+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:05.505376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:47:05.505376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013302310000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":79,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.548916+0000","last_change":"2024-09-10T18:48:27.016984+0000","last_active":"2024-09-10T18:48:34.548916+0000","last_peered":"2024-09-10T18:48:34.548916+0000","last_clean":"2024-09-10T18:48:34.548916+0000","last_became_active":"2024-09-10T18:45:22.074012+0000","last_became_peered":"2024-09-10T18:45:22.074012+0000","last_unstale":"2024-09-10T18:48:34.548916+0000","last_undegraded":"2024-09-10T18:48:34.548916+0000","last_fullsized":"2024-09-10T18:48:34.548916+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:27.016933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:27.016933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:28.747433+0000","scrub_duration":0.0059183930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":77,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.550712+0000","last_change":"2024-09-10T18:47:36.411129+0000","last_active":"2024-09-10T18:48:34.550712+0000","last_peered":"2024-09-10T18:48:34.550712+0000","last_clean":"2024-09-10T18:48:34.550712+0000","last_became_active":"2024-09-10T18:47:36.410836+0000","last_became_peered":"2024-09-10T18:47:36.410836+0000","last_unstale":"2024-09-10T18:48:34.550712+0000","last_undegraded":"2024-09-10T18:48:34.550712+0000","last_fullsized":"2024-09-10T18:48:34.550712+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_clean_scrub_stamp":"2024-09-10T18:46:53.087539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001678971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"111'5985","reported_seq":6017,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:41.023217+0000","last_change":"2024-09-10T18:47:34.399316+0000","last_active":"2024-09-10T18:48:41.023217+0000","last_peered":"2024-09-10T18:48:41.023217+0000","last_clean":"2024-09-10T18:48:41.023217+0000","last_became_active":"2024-09-10T18:47:34.399002+0000","last_became_peered":"2024-09-10T18:47:34.399002+0000","last_unstale":"2024-09-10T18:48:41.023217+0000","last_undegraded":"2024-09-10T18:48:41.023217+0000","last_fullsized":"2024-09-10T18:48:41.023217+0000","mapping_epoch":86,"log_start":"92'800","ondisk_log_start":"92'800","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":5185,"log_dups_size":0,"ondisk_log_size":5185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:01.449088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":392232960,"num_objects":5985,"num_object_clones":0,"num_object_copies":17955,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5985,"num_write_kb":383040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3","version":"63'19","reported_seq":117,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.508675+0000","last_change":"2024-09-10T18:48:11.781505+0000","last_active":"2024-09-10T18:48:33.508675+0000","last_peered":"2024-09-10T18:48:33.508675+0000","last_clean":"2024-09-10T18:48:33.508675+0000","last_became_active":"2024-09-10T18:45:20.421046+0000","last_became_peered":"2024-09-10T18:45:20.421046+0000","last_unstale":"2024-09-10T18:48:33.508675+0000","last_undegraded":"2024-09-10T18:48:33.508675+0000","last_fullsized":"2024-09-10T18:48:33.508675+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:48:11.781417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:11.781417+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:32.063852+0000","scrub_duration":0.0030109379999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":84,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.551498+0000","last_change":"2024-09-10T18:48:30.694038+0000","last_active":"2024-09-10T18:48:34.551498+0000","last_peered":"2024-09-10T18:48:34.551498+0000","last_clean":"2024-09-10T18:48:34.551498+0000","last_became_active":"2024-09-10T18:45:17.960226+0000","last_became_peered":"2024-09-10T18:45:17.960226+0000","last_unstale":"2024-09-10T18:48:34.551498+0000","last_undegraded":"2024-09-10T18:48:34.551498+0000","last_fullsized":"2024-09-10T18:48:34.551498+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:30.693950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:30.693950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:49.091630+0000","scrub_duration":0.0079194309999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":79,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.556097+0000","last_change":"2024-09-10T18:48:33.513604+0000","last_active":"2024-09-10T18:48:34.556097+0000","last_peered":"2024-09-10T18:48:34.556097+0000","last_clean":"2024-09-10T18:48:34.556097+0000","last_became_active":"2024-09-10T18:45:22.066148+0000","last_became_peered":"2024-09-10T18:45:22.066148+0000","last_unstale":"2024-09-10T18:48:34.556097+0000","last_undegraded":"2024-09-10T18:48:34.556097+0000","last_fullsized":"2024-09-10T18:48:34.556097+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:33.513500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:33.513500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:47.158850+0000","scrub_duration":0.0080948319999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":77,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.514856+0000","last_change":"2024-09-10T18:47:36.422607+0000","last_active":"2024-09-10T18:48:33.514856+0000","last_peered":"2024-09-10T18:48:33.514856+0000","last_clean":"2024-09-10T18:48:33.514856+0000","last_became_active":"2024-09-10T18:47:36.422295+0000","last_became_peered":"2024-09-10T18:47:36.422295+0000","last_unstale":"2024-09-10T18:48:33.514856+0000","last_undegraded":"2024-09-10T18:48:33.514856+0000","last_fullsized":"2024-09-10T18:48:33.514856+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:25.525085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:25.525085+0000","last_clean_scrub_stamp":"2024-09-10T18:47:25.525085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012481079999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"111'6071","reported_seq":6103,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:41.032733+0000","last_change":"2024-09-10T18:47:34.397576+0000","last_active":"2024-09-10T18:48:41.032733+0000","last_peered":"2024-09-10T18:48:41.032733+0000","last_clean":"2024-09-10T18:48:41.032733+0000","last_became_active":"2024-09-10T18:47:34.396631+0000","last_became_peered":"2024-09-10T18:47:34.396631+0000","last_unstale":"2024-09-10T18:48:41.032733+0000","last_undegraded":"2024-09-10T18:48:41.032733+0000","last_fullsized":"2024-09-10T18:48:41.032733+0000","mapping_epoch":86,"log_start":"92'900","ondisk_log_start":"92'900","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":5171,"log_dups_size":0,"ondisk_log_size":5171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":397869056,"num_objects":6071,"num_object_clones":0,"num_object_copies":18213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6071,"num_write_kb":388544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"83'15","reported_seq":101,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.517116+0000","last_change":"2024-09-10T18:48:20.633119+0000","last_active":"2024-09-10T18:48:33.517116+0000","last_peered":"2024-09-10T18:48:33.517116+0000","last_clean":"2024-09-10T18:48:33.517116+0000","last_became_active":"2024-09-10T18:45:20.057204+0000","last_became_peered":"2024-09-10T18:45:20.057204+0000","last_unstale":"2024-09-10T18:48:33.517116+0000","last_undegraded":"2024-09-10T18:48:33.517116+0000","last_fullsized":"2024-09-10T18:48:33.517116+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'15","last_scrub_stamp":"2024-09-10T18:48:20.633054+0000","last_deep_scrub":"83'15","last_deep_scrub_stamp":"2024-09-10T18:46:58.436473+0000","last_clean_scrub_stamp":"2024-09-10T18:48:20.633054+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:37.027685+0000","scrub_duration":0.008267412,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":73,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.517414+0000","last_change":"2024-09-10T18:46:50.443101+0000","last_active":"2024-09-10T18:48:33.517414+0000","last_peered":"2024-09-10T18:48:33.517414+0000","last_clean":"2024-09-10T18:48:33.517414+0000","last_became_active":"2024-09-10T18:45:17.966632+0000","last_became_peered":"2024-09-10T18:45:17.966632+0000","last_unstale":"2024-09-10T18:48:33.517414+0000","last_undegraded":"2024-09-10T18:48:33.517414+0000","last_fullsized":"2024-09-10T18:48:33.517414+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:50.443042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.443042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014088550000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"63'8","reported_seq":116,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.508595+0000","last_change":"2024-09-10T18:47:03.579260+0000","last_active":"2024-09-10T18:48:33.508595+0000","last_peered":"2024-09-10T18:48:33.508595+0000","last_clean":"2024-09-10T18:48:33.508595+0000","last_became_active":"2024-09-10T18:45:22.063326+0000","last_became_peered":"2024-09-10T18:45:22.063326+0000","last_unstale":"2024-09-10T18:48:33.508595+0000","last_undegraded":"2024-09-10T18:48:33.508595+0000","last_fullsized":"2024-09-10T18:48:33.508595+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:47:03.579186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:47:03.579186+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021415639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":77,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.507671+0000","last_change":"2024-09-10T18:47:36.412408+0000","last_active":"2024-09-10T18:48:33.507671+0000","last_peered":"2024-09-10T18:48:33.507671+0000","last_clean":"2024-09-10T18:48:33.507671+0000","last_became_active":"2024-09-10T18:47:36.412111+0000","last_became_peered":"2024-09-10T18:47:36.412111+0000","last_unstale":"2024-09-10T18:48:33.507671+0000","last_undegraded":"2024-09-10T18:48:33.507671+0000","last_fullsized":"2024-09-10T18:48:33.507671+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:13.488525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:13.488525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001590344,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"111'6110","reported_seq":6142,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:40.983883+0000","last_change":"2024-09-10T18:47:34.400344+0000","last_active":"2024-09-10T18:48:40.983883+0000","last_peered":"2024-09-10T18:48:40.983883+0000","last_clean":"2024-09-10T18:48:40.983883+0000","last_became_active":"2024-09-10T18:47:34.400044+0000","last_became_peered":"2024-09-10T18:47:34.400044+0000","last_unstale":"2024-09-10T18:48:40.983883+0000","last_undegraded":"2024-09-10T18:48:40.983883+0000","last_fullsized":"2024-09-10T18:48:40.983883+0000","mapping_epoch":86,"log_start":"92'900","ondisk_log_start":"92'900","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":5210,"log_dups_size":0,"ondisk_log_size":5210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":400424960,"num_objects":6110,"num_object_clones":0,"num_object_copies":18330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6110,"num_write_kb":391040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"89'29","reported_seq":200,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.517352+0000","last_change":"2024-09-10T18:47:36.443205+0000","last_active":"2024-09-10T18:48:33.517352+0000","last_peered":"2024-09-10T18:48:33.517352+0000","last_clean":"2024-09-10T18:48:33.517352+0000","last_became_active":"2024-09-10T18:47:36.420185+0000","last_became_peered":"2024-09-10T18:47:36.420185+0000","last_unstale":"2024-09-10T18:48:33.517352+0000","last_undegraded":"2024-09-10T18:48:33.517352+0000","last_fullsized":"2024-09-10T18:48:33.517352+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'28","last_scrub_stamp":"2024-09-10T18:47:26.525861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:26.525861+0000","objects_scrubbed":10,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:49.381758+0000","scrub_duration":0.002126719,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":83,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.561674+0000","last_change":"2024-09-10T18:48:33.186208+0000","last_active":"2024-09-10T18:48:34.561674+0000","last_peered":"2024-09-10T18:48:34.561674+0000","last_clean":"2024-09-10T18:48:34.561674+0000","last_became_active":"2024-09-10T18:45:17.964494+0000","last_became_peered":"2024-09-10T18:45:17.964494+0000","last_unstale":"2024-09-10T18:48:34.561674+0000","last_undegraded":"2024-09-10T18:48:34.561674+0000","last_fullsized":"2024-09-10T18:48:34.561674+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:33.186070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:33.186070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:00.114296+0000","scrub_duration":0.0068142150000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":24,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.556834+0000","last_change":"2024-09-10T18:47:57.108424+0000","last_active":"2024-09-10T18:48:34.556834+0000","last_peered":"2024-09-10T18:48:34.556834+0000","last_clean":"2024-09-10T18:48:34.556834+0000","last_became_active":"2024-09-10T18:47:57.107478+0000","last_became_peered":"2024-09-10T18:47:57.107478+0000","last_unstale":"2024-09-10T18:48:34.556834+0000","last_undegraded":"2024-09-10T18:48:34.556834+0000","last_fullsized":"2024-09-10T18:48:34.556834+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:58.523217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:58.523217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001307856,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":67,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.507156+0000","last_change":"2024-09-10T18:47:22.891142+0000","last_active":"2024-09-10T18:48:33.507156+0000","last_peered":"2024-09-10T18:48:33.507156+0000","last_clean":"2024-09-10T18:48:33.507156+0000","last_became_active":"2024-09-10T18:45:24.073307+0000","last_became_peered":"2024-09-10T18:45:24.073307+0000","last_unstale":"2024-09-10T18:48:33.507156+0000","last_undegraded":"2024-09-10T18:48:33.507156+0000","last_fullsized":"2024-09-10T18:48:33.507156+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:22.891084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:22.891084+0000","last_clean_scrub_stamp":"2024-09-10T18:47:22.891084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017788579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"110'5665","reported_seq":5697,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.069485+0000","last_change":"2024-09-10T18:47:34.395543+0000","last_active":"2024-09-10T18:48:36.069485+0000","last_peered":"2024-09-10T18:48:36.069485+0000","last_clean":"2024-09-10T18:48:36.069485+0000","last_became_active":"2024-09-10T18:47:34.395271+0000","last_became_peered":"2024-09-10T18:47:34.395271+0000","last_unstale":"2024-09-10T18:48:36.069485+0000","last_undegraded":"2024-09-10T18:48:36.069485+0000","last_fullsized":"2024-09-10T18:48:36.069485+0000","mapping_epoch":86,"log_start":"92'900","ondisk_log_start":"92'900","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":4765,"log_dups_size":0,"ondisk_log_size":4765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:39.605507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":371261440,"num_objects":5665,"num_object_clones":0,"num_object_copies":16995,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5665,"num_write_kb":362560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"84'18","reported_seq":103,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.515212+0000","last_change":"2024-09-10T18:47:10.515060+0000","last_active":"2024-09-10T18:48:33.515212+0000","last_peered":"2024-09-10T18:48:33.515212+0000","last_clean":"2024-09-10T18:48:33.515212+0000","last_became_active":"2024-09-10T18:45:20.058435+0000","last_became_peered":"2024-09-10T18:45:20.058435+0000","last_unstale":"2024-09-10T18:48:33.515212+0000","last_undegraded":"2024-09-10T18:48:33.515212+0000","last_fullsized":"2024-09-10T18:48:33.515212+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'18","last_scrub_stamp":"2024-09-10T18:47:10.514951+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-10T18:47:10.514951+0000","last_clean_scrub_stamp":"2024-09-10T18:47:10.514951+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005567575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"84'6","reported_seq":285,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.570237+0000","last_change":"2024-09-10T18:47:36.439529+0000","last_active":"2024-09-10T18:48:34.570237+0000","last_peered":"2024-09-10T18:48:34.570237+0000","last_clean":"2024-09-10T18:48:34.570237+0000","last_became_active":"2024-09-10T18:47:36.418211+0000","last_became_peered":"2024-09-10T18:47:36.418211+0000","last_unstale":"2024-09-10T18:48:34.570237+0000","last_undegraded":"2024-09-10T18:48:34.570237+0000","last_fullsized":"2024-09-10T18:48:34.570237+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'6","last_scrub_stamp":"2024-09-10T18:47:22.545450+0000","last_deep_scrub":"84'6","last_deep_scrub_stamp":"2024-09-10T18:47:22.545450+0000","last_clean_scrub_stamp":"2024-09-10T18:47:22.545450+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:40.352240+0000","scrub_duration":0.0022780040000000001,"objects_trimmed":0,"snaptrim_duration":0.0021933159999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":188,"num_read_kb":183,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":34,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.510811+0000","last_change":"2024-09-10T18:47:36.422920+0000","last_active":"2024-09-10T18:48:33.510811+0000","last_peered":"2024-09-10T18:48:33.510811+0000","last_clean":"2024-09-10T18:48:33.510811+0000","last_became_active":"2024-09-10T18:47:36.422641+0000","last_became_peered":"2024-09-10T18:47:36.422641+0000","last_unstale":"2024-09-10T18:48:33.510811+0000","last_undegraded":"2024-09-10T18:48:33.510811+0000","last_fullsized":"2024-09-10T18:48:33.510811+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:46.712269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:46.712269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":44,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.561913+0000","last_change":"2024-09-10T18:48:22.011954+0000","last_active":"2024-09-10T18:48:34.561913+0000","last_peered":"2024-09-10T18:48:34.561913+0000","last_clean":"2024-09-10T18:48:34.561913+0000","last_became_active":"2024-09-10T18:47:36.419081+0000","last_became_peered":"2024-09-10T18:47:36.419081+0000","last_unstale":"2024-09-10T18:48:34.561913+0000","last_undegraded":"2024-09-10T18:48:34.561913+0000","last_fullsized":"2024-09-10T18:48:34.561913+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:22.011870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:22.011870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:41.762812+0000","scrub_duration":0.0018770919999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":68,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.551219+0000","last_change":"2024-09-10T18:46:59.553576+0000","last_active":"2024-09-10T18:48:34.551219+0000","last_peered":"2024-09-10T18:48:34.551219+0000","last_clean":"2024-09-10T18:48:34.551219+0000","last_became_active":"2024-09-10T18:45:24.264617+0000","last_became_peered":"2024-09-10T18:45:24.264617+0000","last_unstale":"2024-09-10T18:48:34.551219+0000","last_undegraded":"2024-09-10T18:48:34.551219+0000","last_fullsized":"2024-09-10T18:48:34.551219+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:59.553505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:59.553505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018297210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"111'5983","reported_seq":6026,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:41.022593+0000","last_change":"2024-09-10T18:47:36.417383+0000","last_active":"2024-09-10T18:48:41.022593+0000","last_peered":"2024-09-10T18:48:41.022593+0000","last_clean":"2024-09-10T18:48:41.022593+0000","last_became_active":"2024-09-10T18:47:36.417075+0000","last_became_peered":"2024-09-10T18:47:36.417075+0000","last_unstale":"2024-09-10T18:48:41.022593+0000","last_undegraded":"2024-09-10T18:48:41.022593+0000","last_fullsized":"2024-09-10T18:48:41.022593+0000","mapping_epoch":88,"log_start":"94'1400","ondisk_log_start":"94'1400","created":86,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":4583,"log_dups_size":0,"ondisk_log_size":4583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:42.573805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":392101888,"num_objects":5983,"num_object_clones":0,"num_object_copies":17949,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5983,"num_write_kb":382912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"63'12","reported_seq":123,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.509410+0000","last_change":"2024-09-10T18:48:28.710224+0000","last_active":"2024-09-10T18:48:33.509410+0000","last_peered":"2024-09-10T18:48:33.509410+0000","last_clean":"2024-09-10T18:48:33.509410+0000","last_became_active":"2024-09-10T18:47:36.417965+0000","last_became_peered":"2024-09-10T18:47:36.417965+0000","last_unstale":"2024-09-10T18:48:33.509410+0000","last_undegraded":"2024-09-10T18:48:33.509410+0000","last_fullsized":"2024-09-10T18:48:33.509410+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:48:28.710113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:28.710113+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:58.705191+0000","scrub_duration":0.0069768260000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"58'1","reported_seq":79,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.508484+0000","last_change":"2024-09-10T18:47:14.490159+0000","last_active":"2024-09-10T18:48:33.508484+0000","last_peered":"2024-09-10T18:48:33.508484+0000","last_clean":"2024-09-10T18:48:33.508484+0000","last_became_active":"2024-09-10T18:45:17.963932+0000","last_became_peered":"2024-09-10T18:45:17.963932+0000","last_unstale":"2024-09-10T18:48:33.508484+0000","last_undegraded":"2024-09-10T18:48:33.508484+0000","last_fullsized":"2024-09-10T18:48:33.508484+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'1","last_scrub_stamp":"2024-09-10T18:47:14.490091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:47:14.490091+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:40.832261+0000","scrub_duration":0.0019528939999999999,"objects_trimmed":0,"snaptrim_duration":0.00037812099999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":32,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.551453+0000","last_change":"2024-09-10T18:47:36.417008+0000","last_active":"2024-09-10T18:48:34.551453+0000","last_peered":"2024-09-10T18:48:34.551453+0000","last_clean":"2024-09-10T18:48:34.551453+0000","last_became_active":"2024-09-10T18:47:36.416641+0000","last_became_peered":"2024-09-10T18:47:36.416641+0000","last_unstale":"2024-09-10T18:48:34.551453+0000","last_undegraded":"2024-09-10T18:48:34.551453+0000","last_fullsized":"2024-09-10T18:48:34.551453+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:54.554426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:54.554426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":69,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.561981+0000","last_change":"2024-09-10T18:46:59.132301+0000","last_active":"2024-09-10T18:48:34.561981+0000","last_peered":"2024-09-10T18:48:34.561981+0000","last_clean":"2024-09-10T18:48:34.561981+0000","last_became_active":"2024-09-10T18:45:22.064285+0000","last_became_peered":"2024-09-10T18:45:22.064285+0000","last_unstale":"2024-09-10T18:48:34.561981+0000","last_undegraded":"2024-09-10T18:48:34.561981+0000","last_fullsized":"2024-09-10T18:48:34.561981+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:59.132234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:59.132234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001471034,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":77,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.517146+0000","last_change":"2024-09-10T18:48:21.654220+0000","last_active":"2024-09-10T18:48:33.517146+0000","last_peered":"2024-09-10T18:48:33.517146+0000","last_clean":"2024-09-10T18:48:33.517146+0000","last_became_active":"2024-09-10T18:45:24.083689+0000","last_became_peered":"2024-09-10T18:45:24.083689+0000","last_unstale":"2024-09-10T18:48:33.517146+0000","last_undegraded":"2024-09-10T18:48:33.517146+0000","last_fullsized":"2024-09-10T18:48:33.517146+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:21.654007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:21.654007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:39.425642+0000","scrub_duration":0.0058335929999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"110'5612","reported_seq":5644,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.099171+0000","last_change":"2024-09-10T18:47:34.398452+0000","last_active":"2024-09-10T18:48:36.099171+0000","last_peered":"2024-09-10T18:48:36.099171+0000","last_clean":"2024-09-10T18:48:36.099171+0000","last_became_active":"2024-09-10T18:47:34.396853+0000","last_became_peered":"2024-09-10T18:47:34.396853+0000","last_unstale":"2024-09-10T18:48:36.099171+0000","last_undegraded":"2024-09-10T18:48:36.099171+0000","last_fullsized":"2024-09-10T18:48:36.099171+0000","mapping_epoch":86,"log_start":"92'800","ondisk_log_start":"92'800","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":4812,"log_dups_size":0,"ondisk_log_size":4812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:56.235056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":367788032,"num_objects":5612,"num_object_clones":0,"num_object_copies":16836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5612,"num_write_kb":359168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"63'14","reported_seq":103,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.557153+0000","last_change":"2024-09-10T18:48:17.669939+0000","last_active":"2024-09-10T18:48:34.557153+0000","last_peered":"2024-09-10T18:48:34.557153+0000","last_clean":"2024-09-10T18:48:34.557153+0000","last_became_active":"2024-09-10T18:45:20.421119+0000","last_became_peered":"2024-09-10T18:45:20.421119+0000","last_unstale":"2024-09-10T18:48:34.557153+0000","last_undegraded":"2024-09-10T18:48:34.557153+0000","last_fullsized":"2024-09-10T18:48:34.557153+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-10T18:48:17.669787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:17.669787+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:33.721948+0000","scrub_duration":0.0024048300000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"60'2","reported_seq":80,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.510754+0000","last_change":"2024-09-10T18:46:25.845131+0000","last_active":"2024-09-10T18:48:33.510754+0000","last_peered":"2024-09-10T18:48:33.510754+0000","last_clean":"2024-09-10T18:48:33.510754+0000","last_became_active":"2024-09-10T18:45:17.961660+0000","last_became_peered":"2024-09-10T18:45:17.961660+0000","last_unstale":"2024-09-10T18:48:33.510754+0000","last_undegraded":"2024-09-10T18:48:33.510754+0000","last_fullsized":"2024-09-10T18:48:33.510754+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-10T18:46:25.845070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:25.845070+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020129710000000001,"objects_trimmed":0,"snaptrim_duration":0.0013942640000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":93,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.561673+0000","last_change":"2024-09-10T18:48:34.150227+0000","last_active":"2024-09-10T18:48:34.561673+0000","last_peered":"2024-09-10T18:48:34.561673+0000","last_clean":"2024-09-10T18:48:34.561673+0000","last_became_active":"2024-09-10T18:47:36.414636+0000","last_became_peered":"2024-09-10T18:47:36.414636+0000","last_unstale":"2024-09-10T18:48:34.561673+0000","last_undegraded":"2024-09-10T18:48:34.561673+0000","last_fullsized":"2024-09-10T18:48:34.561673+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:34.150146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:34.150146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:02.371194+0000","scrub_duration":0.0018375380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'81","reported_seq":118,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.554737+0000","last_change":"2024-09-10T18:47:05.558031+0000","last_active":"2024-09-10T18:48:34.554737+0000","last_peered":"2024-09-10T18:48:34.554737+0000","last_clean":"2024-09-10T18:48:34.554737+0000","last_became_active":"2024-09-10T18:45:07.433592+0000","last_became_peered":"2024-09-10T18:45:07.433592+0000","last_unstale":"2024-09-10T18:48:34.554737+0000","last_undegraded":"2024-09-10T18:48:34.554737+0000","last_fullsized":"2024-09-10T18:48:34.554737+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:47:05.557968+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:47:05.557968+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015062890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":90,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.551249+0000","last_change":"2024-09-10T18:47:36.418948+0000","last_active":"2024-09-10T18:48:34.551249+0000","last_peered":"2024-09-10T18:48:34.551249+0000","last_clean":"2024-09-10T18:48:34.551249+0000","last_became_active":"2024-09-10T18:47:36.418320+0000","last_became_peered":"2024-09-10T18:47:36.418320+0000","last_unstale":"2024-09-10T18:48:34.551249+0000","last_undegraded":"2024-09-10T18:48:34.551249+0000","last_fullsized":"2024-09-10T18:48:34.551249+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:32.528904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:47:32.528904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037979899999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":67,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.562039+0000","last_change":"2024-09-10T18:46:56.066558+0000","last_active":"2024-09-10T18:48:34.562039+0000","last_peered":"2024-09-10T18:48:34.562039+0000","last_clean":"2024-09-10T18:48:34.562039+0000","last_became_active":"2024-09-10T18:45:24.073327+0000","last_became_peered":"2024-09-10T18:45:24.073327+0000","last_unstale":"2024-09-10T18:48:34.562039+0000","last_undegraded":"2024-09-10T18:48:34.562039+0000","last_fullsized":"2024-09-10T18:48:34.562039+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:56.066502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:56.066502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0011960810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.9","version":"110'5764","reported_seq":5796,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.093273+0000","last_change":"2024-09-10T18:47:34.393877+0000","last_active":"2024-09-10T18:48:36.093273+0000","last_peered":"2024-09-10T18:48:36.093273+0000","last_clean":"2024-09-10T18:48:36.093273+0000","last_became_active":"2024-09-10T18:47:34.393620+0000","last_became_peered":"2024-09-10T18:47:34.393620+0000","last_unstale":"2024-09-10T18:48:36.093273+0000","last_undegraded":"2024-09-10T18:48:36.093273+0000","last_fullsized":"2024-09-10T18:48:36.093273+0000","mapping_epoch":86,"log_start":"93'1000","ondisk_log_start":"93'1000","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":4764,"log_dups_size":0,"ondisk_log_size":4764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:46.971656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":377749504,"num_objects":5764,"num_object_clones":0,"num_object_copies":17292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5764,"num_write_kb":368896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"63'19","reported_seq":112,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.556282+0000","last_change":"2024-09-10T18:48:14.655578+0000","last_active":"2024-09-10T18:48:34.556282+0000","last_peered":"2024-09-10T18:48:34.556282+0000","last_clean":"2024-09-10T18:48:34.556282+0000","last_became_active":"2024-09-10T18:45:20.421097+0000","last_became_peered":"2024-09-10T18:45:20.421097+0000","last_unstale":"2024-09-10T18:48:34.556282+0000","last_undegraded":"2024-09-10T18:48:34.556282+0000","last_fullsized":"2024-09-10T18:48:34.556282+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:48:14.655527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:14.655527+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:31.443092+0000","scrub_duration":0.00174087,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":34,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.561774+0000","last_change":"2024-09-10T18:47:36.418124+0000","last_active":"2024-09-10T18:48:34.561774+0000","last_peered":"2024-09-10T18:48:34.561774+0000","last_clean":"2024-09-10T18:48:34.561774+0000","last_became_active":"2024-09-10T18:47:36.417800+0000","last_became_peered":"2024-09-10T18:47:36.417800+0000","last_unstale":"2024-09-10T18:48:34.561774+0000","last_undegraded":"2024-09-10T18:48:34.561774+0000","last_fullsized":"2024-09-10T18:48:34.561774+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:53.707694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:53.707694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":44,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.508514+0000","last_change":"2024-09-10T18:48:12.827398+0000","last_active":"2024-09-10T18:48:33.508514+0000","last_peered":"2024-09-10T18:48:33.508514+0000","last_clean":"2024-09-10T18:48:33.508514+0000","last_became_active":"2024-09-10T18:47:36.413618+0000","last_became_peered":"2024-09-10T18:47:36.413618+0000","last_unstale":"2024-09-10T18:48:33.508514+0000","last_undegraded":"2024-09-10T18:48:33.508514+0000","last_fullsized":"2024-09-10T18:48:33.508514+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:12.827317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:12.827317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:17.404178+0000","scrub_duration":0.0036790109999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":78,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.554654+0000","last_change":"2024-09-10T18:47:36.413013+0000","last_active":"2024-09-10T18:48:34.554654+0000","last_peered":"2024-09-10T18:48:34.554654+0000","last_clean":"2024-09-10T18:48:34.554654+0000","last_became_active":"2024-09-10T18:47:36.412610+0000","last_became_peered":"2024-09-10T18:47:36.412610+0000","last_unstale":"2024-09-10T18:48:34.554654+0000","last_undegraded":"2024-09-10T18:48:34.554654+0000","last_fullsized":"2024-09-10T18:48:34.554654+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:03.587252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:03.587252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001684867,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"111'6022","reported_seq":6057,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:41.022362+0000","last_change":"2024-09-10T18:47:36.413347+0000","last_active":"2024-09-10T18:48:41.022362+0000","last_peered":"2024-09-10T18:48:41.022362+0000","last_clean":"2024-09-10T18:48:41.022362+0000","last_became_active":"2024-09-10T18:47:36.412142+0000","last_became_peered":"2024-09-10T18:47:36.412142+0000","last_unstale":"2024-09-10T18:48:41.022362+0000","last_undegraded":"2024-09-10T18:48:41.022362+0000","last_fullsized":"2024-09-10T18:48:41.022362+0000","mapping_epoch":88,"log_start":"94'1400","ondisk_log_start":"94'1400","created":86,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":4622,"log_dups_size":0,"ondisk_log_size":4622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:59.312878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":394657792,"num_objects":6022,"num_object_clones":0,"num_object_copies":18066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6022,"num_write_kb":385408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"63'9","reported_seq":97,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.508663+0000","last_change":"2024-09-10T18:48:22.878265+0000","last_active":"2024-09-10T18:48:33.508663+0000","last_peered":"2024-09-10T18:48:33.508663+0000","last_clean":"2024-09-10T18:48:33.508663+0000","last_became_active":"2024-09-10T18:45:20.061021+0000","last_became_peered":"2024-09-10T18:45:20.061021+0000","last_unstale":"2024-09-10T18:48:33.508663+0000","last_undegraded":"2024-09-10T18:48:33.508663+0000","last_fullsized":"2024-09-10T18:48:33.508663+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:48:22.878179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:22.878179+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:49.491061+0000","scrub_duration":0.021357979999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":73,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.516241+0000","last_change":"2024-09-10T18:47:02.496271+0000","last_active":"2024-09-10T18:48:33.516241+0000","last_peered":"2024-09-10T18:48:33.516241+0000","last_clean":"2024-09-10T18:48:33.516241+0000","last_became_active":"2024-09-10T18:45:17.963057+0000","last_became_peered":"2024-09-10T18:45:17.963057+0000","last_unstale":"2024-09-10T18:48:33.516241+0000","last_undegraded":"2024-09-10T18:48:33.516241+0000","last_fullsized":"2024-09-10T18:48:33.516241+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:02.496142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:47:02.496142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001628399,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":34,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.550594+0000","last_change":"2024-09-10T18:47:36.411830+0000","last_active":"2024-09-10T18:48:34.550594+0000","last_peered":"2024-09-10T18:48:34.550594+0000","last_clean":"2024-09-10T18:48:34.550594+0000","last_became_active":"2024-09-10T18:47:36.411517+0000","last_became_peered":"2024-09-10T18:47:36.411517+0000","last_unstale":"2024-09-10T18:48:34.550594+0000","last_undegraded":"2024-09-10T18:48:34.550594+0000","last_fullsized":"2024-09-10T18:48:34.550594+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_clean_scrub_stamp":"2024-09-10T18:46:55.684250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":87,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.508365+0000","last_change":"2024-09-10T18:48:25.804464+0000","last_active":"2024-09-10T18:48:33.508365+0000","last_peered":"2024-09-10T18:48:33.508365+0000","last_clean":"2024-09-10T18:48:33.508365+0000","last_became_active":"2024-09-10T18:47:36.412933+0000","last_became_peered":"2024-09-10T18:47:36.412933+0000","last_unstale":"2024-09-10T18:48:33.508365+0000","last_undegraded":"2024-09-10T18:48:33.508365+0000","last_fullsized":"2024-09-10T18:48:33.508365+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:25.804376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:47.545343+0000","last_clean_scrub_stamp":"2024-09-10T18:48:25.804376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:26.796128+0000","scrub_duration":0.0098886110000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.b","version":"110'5672","reported_seq":5702,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.088014+0000","last_change":"2024-09-10T18:47:34.398004+0000","last_active":"2024-09-10T18:48:36.088014+0000","last_peered":"2024-09-10T18:48:36.088014+0000","last_clean":"2024-09-10T18:48:36.088014+0000","last_became_active":"2024-09-10T18:47:34.397708+0000","last_became_peered":"2024-09-10T18:47:34.397708+0000","last_unstale":"2024-09-10T18:48:36.088014+0000","last_undegraded":"2024-09-10T18:48:36.088014+0000","last_fullsized":"2024-09-10T18:48:36.088014+0000","mapping_epoch":86,"log_start":"93'1300","ondisk_log_start":"93'1300","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":4372,"log_dups_size":0,"ondisk_log_size":4372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:47.174258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":371720192,"num_objects":5672,"num_object_clones":0,"num_object_copies":17016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5672,"num_write_kb":363008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"63'15","reported_seq":96,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.561304+0000","last_change":"2024-09-10T18:46:51.114543+0000","last_active":"2024-09-10T18:48:34.561304+0000","last_peered":"2024-09-10T18:48:34.561304+0000","last_clean":"2024-09-10T18:48:34.561304+0000","last_became_active":"2024-09-10T18:45:20.419804+0000","last_became_peered":"2024-09-10T18:45:20.419804+0000","last_unstale":"2024-09-10T18:48:34.561304+0000","last_undegraded":"2024-09-10T18:48:34.561304+0000","last_fullsized":"2024-09-10T18:48:34.561304+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:46:51.114455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:51.114455+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022557979999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":34,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.507293+0000","last_change":"2024-09-10T18:47:36.420713+0000","last_active":"2024-09-10T18:48:33.507293+0000","last_peered":"2024-09-10T18:48:33.507293+0000","last_clean":"2024-09-10T18:48:33.507293+0000","last_became_active":"2024-09-10T18:47:36.420475+0000","last_became_peered":"2024-09-10T18:47:36.420475+0000","last_unstale":"2024-09-10T18:48:33.507293+0000","last_undegraded":"2024-09-10T18:48:33.507293+0000","last_fullsized":"2024-09-10T18:48:33.507293+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:20.774289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:17.819909+0000","last_clean_scrub_stamp":"2024-09-10T18:47:20.774289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:45.742247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"83'9","reported_seq":132,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.114438+0000","last_change":"2024-09-10T18:48:33.683570+0000","last_active":"2024-09-10T18:48:36.114438+0000","last_peered":"2024-09-10T18:48:36.114438+0000","last_clean":"2024-09-10T18:48:36.114438+0000","last_became_active":"2024-09-10T18:45:22.073383+0000","last_became_peered":"2024-09-10T18:45:22.073383+0000","last_unstale":"2024-09-10T18:48:36.114438+0000","last_undegraded":"2024-09-10T18:48:36.114438+0000","last_fullsized":"2024-09-10T18:48:36.114438+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'9","last_scrub_stamp":"2024-09-10T18:48:33.683477+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-09-10T18:47:02.582229+0000","last_clean_scrub_stamp":"2024-09-10T18:48:33.683477+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:57.226708+0000","scrub_duration":0.0026989119999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":67,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.561431+0000","last_change":"2024-09-10T18:46:45.083522+0000","last_active":"2024-09-10T18:48:34.561431+0000","last_peered":"2024-09-10T18:48:34.561431+0000","last_clean":"2024-09-10T18:48:34.561431+0000","last_became_active":"2024-09-10T18:45:24.265256+0000","last_became_peered":"2024-09-10T18:45:24.265256+0000","last_unstale":"2024-09-10T18:48:34.561431+0000","last_undegraded":"2024-09-10T18:48:34.561431+0000","last_fullsized":"2024-09-10T18:48:34.561431+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_clean_scrub_stamp":"2024-09-10T18:46:45.083425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017589890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.a","version":"110'5839","reported_seq":5881,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.145122+0000","last_change":"2024-09-10T18:47:36.423957+0000","last_active":"2024-09-10T18:48:36.145122+0000","last_peered":"2024-09-10T18:48:36.145122+0000","last_clean":"2024-09-10T18:48:36.145122+0000","last_became_active":"2024-09-10T18:47:36.422868+0000","last_became_peered":"2024-09-10T18:47:36.422868+0000","last_unstale":"2024-09-10T18:48:36.145122+0000","last_undegraded":"2024-09-10T18:48:36.145122+0000","last_fullsized":"2024-09-10T18:48:36.145122+0000","mapping_epoch":88,"log_start":"95'1600","ondisk_log_start":"95'1600","created":86,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":4239,"log_dups_size":0,"ondisk_log_size":4239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:44.562678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":382664704,"num_objects":5839,"num_object_clones":0,"num_object_copies":17517,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5839,"num_write_kb":373696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"63'12","reported_seq":104,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.548987+0000","last_change":"2024-09-10T18:47:35.928110+0000","last_active":"2024-09-10T18:48:34.548987+0000","last_peered":"2024-09-10T18:48:34.548987+0000","last_clean":"2024-09-10T18:48:34.548987+0000","last_became_active":"2024-09-10T18:45:20.056869+0000","last_became_peered":"2024-09-10T18:45:20.056869+0000","last_unstale":"2024-09-10T18:48:34.548987+0000","last_undegraded":"2024-09-10T18:48:34.548987+0000","last_fullsized":"2024-09-10T18:48:34.548987+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:47:35.928040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:35.928040+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:50.601492+0000","scrub_duration":0.0021332539999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"56'2","reported_seq":90,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.552689+0000","last_change":"2024-09-10T18:46:42.303648+0000","last_active":"2024-09-10T18:48:34.552689+0000","last_peered":"2024-09-10T18:48:34.552689+0000","last_clean":"2024-09-10T18:48:34.552689+0000","last_became_active":"2024-09-10T18:45:17.963286+0000","last_became_peered":"2024-09-10T18:45:17.963286+0000","last_unstale":"2024-09-10T18:48:34.552689+0000","last_undegraded":"2024-09-10T18:48:34.552689+0000","last_fullsized":"2024-09-10T18:48:34.552689+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-10T18:46:42.303567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:42.303567+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020684750000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":44,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.507750+0000","last_change":"2024-09-10T18:48:15.763428+0000","last_active":"2024-09-10T18:48:33.507750+0000","last_peered":"2024-09-10T18:48:33.507750+0000","last_clean":"2024-09-10T18:48:33.507750+0000","last_became_active":"2024-09-10T18:47:36.421151+0000","last_became_peered":"2024-09-10T18:47:36.421151+0000","last_unstale":"2024-09-10T18:48:33.507750+0000","last_undegraded":"2024-09-10T18:48:33.507750+0000","last_fullsized":"2024-09-10T18:48:33.507750+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:15.763348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:15.763348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:38.665471+0000","scrub_duration":0.0046035929999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":67,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.506720+0000","last_change":"2024-09-10T18:47:18.797303+0000","last_active":"2024-09-10T18:48:33.506720+0000","last_peered":"2024-09-10T18:48:33.506720+0000","last_clean":"2024-09-10T18:48:33.506720+0000","last_became_active":"2024-09-10T18:45:24.077815+0000","last_became_peered":"2024-09-10T18:45:24.077815+0000","last_unstale":"2024-09-10T18:48:33.506720+0000","last_undegraded":"2024-09-10T18:48:33.506720+0000","last_fullsized":"2024-09-10T18:48:33.506720+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:18.797245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:18.797245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015800090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"76'13","reported_seq":101,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.509401+0000","last_change":"2024-09-10T18:48:10.773290+0000","last_active":"2024-09-10T18:48:33.509401+0000","last_peered":"2024-09-10T18:48:33.509401+0000","last_clean":"2024-09-10T18:48:33.509401+0000","last_became_active":"2024-09-10T18:45:20.420421+0000","last_became_peered":"2024-09-10T18:45:20.420421+0000","last_unstale":"2024-09-10T18:48:33.509401+0000","last_undegraded":"2024-09-10T18:48:33.509401+0000","last_fullsized":"2024-09-10T18:48:33.509401+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'13","last_scrub_stamp":"2024-09-10T18:48:10.773207+0000","last_deep_scrub":"76'13","last_deep_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_clean_scrub_stamp":"2024-09-10T18:48:10.773207+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:35.455517+0000","scrub_duration":0.008348678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":44,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.514887+0000","last_change":"2024-09-10T18:48:14.699173+0000","last_active":"2024-09-10T18:48:33.514887+0000","last_peered":"2024-09-10T18:48:33.514887+0000","last_clean":"2024-09-10T18:48:33.514887+0000","last_became_active":"2024-09-10T18:47:36.409299+0000","last_became_peered":"2024-09-10T18:47:36.409299+0000","last_unstale":"2024-09-10T18:48:33.514887+0000","last_undegraded":"2024-09-10T18:48:33.514887+0000","last_fullsized":"2024-09-10T18:48:33.514887+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:14.695946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:14.695946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:15.444972+0000","scrub_duration":0.007154171,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":79,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.509104+0000","last_change":"2024-09-10T18:48:28.442445+0000","last_active":"2024-09-10T18:48:33.509104+0000","last_peered":"2024-09-10T18:48:33.509104+0000","last_clean":"2024-09-10T18:48:33.509104+0000","last_became_active":"2024-09-10T18:45:22.069040+0000","last_became_peered":"2024-09-10T18:45:22.069040+0000","last_unstale":"2024-09-10T18:48:33.509104+0000","last_undegraded":"2024-09-10T18:48:33.509104+0000","last_fullsized":"2024-09-10T18:48:33.509104+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:28.442370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:28.442370+0000","last_clean_scrub_stamp":"2024-09-10T18:48:28.442370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:50.109389+0000","scrub_duration":0.0098412350000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"63'1","reported_seq":47,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.550861+0000","last_change":"2024-09-10T18:47:36.423902+0000","last_active":"2024-09-10T18:48:34.550861+0000","last_peered":"2024-09-10T18:48:34.550861+0000","last_clean":"2024-09-10T18:48:34.550861+0000","last_became_active":"2024-09-10T18:47:36.406936+0000","last_became_peered":"2024-09-10T18:47:36.406936+0000","last_unstale":"2024-09-10T18:48:34.550861+0000","last_undegraded":"2024-09-10T18:48:34.550861+0000","last_fullsized":"2024-09-10T18:48:34.550861+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-10T18:46:56.721716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:56.721716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":83,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.556180+0000","last_change":"2024-09-10T18:48:24.571499+0000","last_active":"2024-09-10T18:48:34.556180+0000","last_peered":"2024-09-10T18:48:34.556180+0000","last_clean":"2024-09-10T18:48:34.556180+0000","last_became_active":"2024-09-10T18:45:17.964705+0000","last_became_peered":"2024-09-10T18:45:17.964705+0000","last_unstale":"2024-09-10T18:48:34.556180+0000","last_undegraded":"2024-09-10T18:48:34.556180+0000","last_fullsized":"2024-09-10T18:48:34.556180+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:24.571419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:24.571419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:51.133227+0000","scrub_duration":0.005895399,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'6","reported_seq":80,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.509234+0000","last_change":"2024-09-10T18:47:02.700853+0000","last_active":"2024-09-10T18:48:33.509234+0000","last_peered":"2024-09-10T18:48:33.509234+0000","last_clean":"2024-09-10T18:48:33.509234+0000","last_became_active":"2024-09-10T18:45:20.421746+0000","last_became_peered":"2024-09-10T18:45:20.421746+0000","last_unstale":"2024-09-10T18:48:33.509234+0000","last_undegraded":"2024-09-10T18:48:33.509234+0000","last_fullsized":"2024-09-10T18:48:33.509234+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'6","last_scrub_stamp":"2024-09-10T18:47:02.700787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:02.700787+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017032519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":69,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.562060+0000","last_change":"2024-09-10T18:46:39.084489+0000","last_active":"2024-09-10T18:48:34.562060+0000","last_peered":"2024-09-10T18:48:34.562060+0000","last_clean":"2024-09-10T18:48:34.562060+0000","last_became_active":"2024-09-10T18:45:22.069653+0000","last_became_peered":"2024-09-10T18:45:22.069653+0000","last_unstale":"2024-09-10T18:48:34.562060+0000","last_undegraded":"2024-09-10T18:48:34.562060+0000","last_fullsized":"2024-09-10T18:48:34.562060+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:39.084433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:39.084433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014845819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":78,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.551134+0000","last_change":"2024-09-10T18:48:32.654862+0000","last_active":"2024-09-10T18:48:34.551134+0000","last_peered":"2024-09-10T18:48:34.551134+0000","last_clean":"2024-09-10T18:48:34.551134+0000","last_became_active":"2024-09-10T18:45:24.264611+0000","last_became_peered":"2024-09-10T18:45:24.264611+0000","last_unstale":"2024-09-10T18:48:34.551134+0000","last_undegraded":"2024-09-10T18:48:34.551134+0000","last_fullsized":"2024-09-10T18:48:34.551134+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:32.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:32.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:01.743394+0000","scrub_duration":0.0049870009999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"63'10","reported_seq":130,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.507210+0000","last_change":"2024-09-10T18:47:36.442817+0000","last_active":"2024-09-10T18:48:33.507210+0000","last_peered":"2024-09-10T18:48:33.507210+0000","last_clean":"2024-09-10T18:48:33.507210+0000","last_became_active":"2024-09-10T18:47:36.426260+0000","last_became_peered":"2024-09-10T18:47:36.426260+0000","last_unstale":"2024-09-10T18:48:33.507210+0000","last_undegraded":"2024-09-10T18:48:33.507210+0000","last_fullsized":"2024-09-10T18:48:33.507210+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:46:50.722316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.722316+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022194070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":73,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.561720+0000","last_change":"2024-09-10T18:46:50.093435+0000","last_active":"2024-09-10T18:48:34.561720+0000","last_peered":"2024-09-10T18:48:34.561720+0000","last_clean":"2024-09-10T18:48:34.561720+0000","last_became_active":"2024-09-10T18:45:17.964870+0000","last_became_peered":"2024-09-10T18:45:17.964870+0000","last_unstale":"2024-09-10T18:48:34.561720+0000","last_undegraded":"2024-09-10T18:48:34.561720+0000","last_fullsized":"2024-09-10T18:48:34.561720+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:50.093372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.093372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015580539999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"63'8","reported_seq":117,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.114415+0000","last_change":"2024-09-10T18:46:34.085224+0000","last_active":"2024-09-10T18:48:36.114415+0000","last_peered":"2024-09-10T18:48:36.114415+0000","last_clean":"2024-09-10T18:48:36.114415+0000","last_became_active":"2024-09-10T18:45:22.062940+0000","last_became_peered":"2024-09-10T18:45:22.062940+0000","last_unstale":"2024-09-10T18:48:36.114415+0000","last_undegraded":"2024-09-10T18:48:36.114415+0000","last_fullsized":"2024-09-10T18:48:36.114415+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:34.085156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:34.085156+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020843120000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":67,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.509049+0000","last_change":"2024-09-10T18:47:21.538033+0000","last_active":"2024-09-10T18:48:33.509049+0000","last_peered":"2024-09-10T18:48:33.509049+0000","last_clean":"2024-09-10T18:48:33.509049+0000","last_became_active":"2024-09-10T18:45:24.073736+0000","last_became_peered":"2024-09-10T18:45:24.073736+0000","last_unstale":"2024-09-10T18:48:33.509049+0000","last_undegraded":"2024-09-10T18:48:33.509049+0000","last_fullsized":"2024-09-10T18:48:33.509049+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:21.537964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:21.537964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:38.167513+0000","scrub_duration":0.001500305,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"63'9","reported_seq":97,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.561775+0000","last_change":"2024-09-10T18:48:19.076178+0000","last_active":"2024-09-10T18:48:34.561775+0000","last_peered":"2024-09-10T18:48:34.561775+0000","last_clean":"2024-09-10T18:48:34.561775+0000","last_became_active":"2024-09-10T18:45:20.421124+0000","last_became_peered":"2024-09-10T18:45:20.421124+0000","last_unstale":"2024-09-10T18:48:34.561775+0000","last_undegraded":"2024-09-10T18:48:34.561775+0000","last_fullsized":"2024-09-10T18:48:34.561775+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:48:19.076099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:19.076099+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:32.177242+0000","scrub_duration":0.002691981,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":83,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.663572+0000","last_change":"2024-09-10T18:48:33.663516+0000","last_active":"2024-09-10T18:48:33.663572+0000","last_peered":"2024-09-10T18:48:33.663572+0000","last_clean":"2024-09-10T18:48:33.663572+0000","last_became_active":"2024-09-10T18:45:17.960106+0000","last_became_peered":"2024-09-10T18:45:17.960106+0000","last_unstale":"2024-09-10T18:48:33.663572+0000","last_undegraded":"2024-09-10T18:48:33.663572+0000","last_fullsized":"2024-09-10T18:48:33.663572+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:33.663415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:33.663415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:33.950247+0000","scrub_duration":0.002167352,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"89'10","reported_seq":132,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.559405+0000","last_change":"2024-09-10T18:47:36.434102+0000","last_active":"2024-09-10T18:48:34.559405+0000","last_peered":"2024-09-10T18:48:34.559405+0000","last_clean":"2024-09-10T18:48:34.559405+0000","last_became_active":"2024-09-10T18:47:36.424575+0000","last_became_peered":"2024-09-10T18:47:36.424575+0000","last_unstale":"2024-09-10T18:48:34.559405+0000","last_undegraded":"2024-09-10T18:48:34.559405+0000","last_fullsized":"2024-09-10T18:48:34.559405+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"81'9","last_scrub_stamp":"2024-09-10T18:46:54.800790+0000","last_deep_scrub":"81'9","last_deep_scrub_stamp":"2024-09-10T18:46:54.800790+0000","last_clean_scrub_stamp":"2024-09-10T18:46:54.800790+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034208709999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":87,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.552427+0000","last_change":"2024-09-10T18:48:34.045421+0000","last_active":"2024-09-10T18:48:34.552427+0000","last_peered":"2024-09-10T18:48:34.552427+0000","last_clean":"2024-09-10T18:48:34.552427+0000","last_became_active":"2024-09-10T18:47:36.414047+0000","last_became_peered":"2024-09-10T18:47:36.414047+0000","last_unstale":"2024-09-10T18:48:34.552427+0000","last_undegraded":"2024-09-10T18:48:34.552427+0000","last_fullsized":"2024-09-10T18:48:34.552427+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:34.045311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_clean_scrub_stamp":"2024-09-10T18:48:34.045311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:49.035924+0000","scrub_duration":0.0048840200000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"63'9","reported_seq":107,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.516641+0000","last_change":"2024-09-10T18:47:36.437948+0000","last_active":"2024-09-10T18:48:33.516641+0000","last_peered":"2024-09-10T18:48:33.516641+0000","last_clean":"2024-09-10T18:48:33.516641+0000","last_became_active":"2024-09-10T18:47:36.416484+0000","last_became_peered":"2024-09-10T18:47:36.416484+0000","last_unstale":"2024-09-10T18:48:33.516641+0000","last_undegraded":"2024-09-10T18:48:33.516641+0000","last_fullsized":"2024-09-10T18:48:33.516641+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:46:59.469198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:59.469198+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022009489999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":83,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.556836+0000","last_change":"2024-09-10T18:47:29.023501+0000","last_active":"2024-09-10T18:48:34.556836+0000","last_peered":"2024-09-10T18:48:34.556836+0000","last_clean":"2024-09-10T18:48:34.556836+0000","last_became_active":"2024-09-10T18:45:17.962009+0000","last_became_peered":"2024-09-10T18:45:17.962009+0000","last_unstale":"2024-09-10T18:48:34.556836+0000","last_undegraded":"2024-09-10T18:48:34.556836+0000","last_fullsized":"2024-09-10T18:48:34.556836+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:29.023450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:47:29.023450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:37.485149+0000","scrub_duration":0.001196805,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":69,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.510954+0000","last_change":"2024-09-10T18:47:14.869132+0000","last_active":"2024-09-10T18:48:33.510954+0000","last_peered":"2024-09-10T18:48:33.510954+0000","last_clean":"2024-09-10T18:48:33.510954+0000","last_became_active":"2024-09-10T18:45:22.072877+0000","last_became_peered":"2024-09-10T18:45:22.072877+0000","last_unstale":"2024-09-10T18:48:33.510954+0000","last_undegraded":"2024-09-10T18:48:33.510954+0000","last_fullsized":"2024-09-10T18:48:33.510954+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:14.869076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:47:14.869076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:38.655935+0000","scrub_duration":0.00131624,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":77,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.507517+0000","last_change":"2024-09-10T18:48:32.647084+0000","last_active":"2024-09-10T18:48:33.507517+0000","last_peered":"2024-09-10T18:48:33.507517+0000","last_clean":"2024-09-10T18:48:33.507517+0000","last_became_active":"2024-09-10T18:45:24.081050+0000","last_became_peered":"2024-09-10T18:45:24.081050+0000","last_unstale":"2024-09-10T18:48:33.507517+0000","last_undegraded":"2024-09-10T18:48:33.507517+0000","last_fullsized":"2024-09-10T18:48:33.507517+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:32.646988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:32.646988+0000","last_clean_scrub_stamp":"2024-09-10T18:48:32.646988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:39.006371+0000","scrub_duration":0.0038989210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"63'11","reported_seq":90,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.550592+0000","last_change":"2024-09-10T18:47:14.034427+0000","last_active":"2024-09-10T18:48:34.550592+0000","last_peered":"2024-09-10T18:48:34.550592+0000","last_clean":"2024-09-10T18:48:34.550592+0000","last_became_active":"2024-09-10T18:45:20.056885+0000","last_became_peered":"2024-09-10T18:45:20.056885+0000","last_unstale":"2024-09-10T18:48:34.550592+0000","last_undegraded":"2024-09-10T18:48:34.550592+0000","last_fullsized":"2024-09-10T18:48:34.550592+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:47:14.034361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:14.034361+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018564,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":83,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.516357+0000","last_change":"2024-09-10T18:48:32.517573+0000","last_active":"2024-09-10T18:48:33.516357+0000","last_peered":"2024-09-10T18:48:33.516357+0000","last_clean":"2024-09-10T18:48:33.516357+0000","last_became_active":"2024-09-10T18:45:17.963542+0000","last_became_peered":"2024-09-10T18:45:17.963542+0000","last_unstale":"2024-09-10T18:48:33.516357+0000","last_undegraded":"2024-09-10T18:48:33.516357+0000","last_fullsized":"2024-09-10T18:48:33.516357+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:32.517513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:32.517513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:48.139582+0000","scrub_duration":0.013400413999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":79,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.516509+0000","last_change":"2024-09-10T18:48:09.725726+0000","last_active":"2024-09-10T18:48:33.516509+0000","last_peered":"2024-09-10T18:48:33.516509+0000","last_clean":"2024-09-10T18:48:33.516509+0000","last_became_active":"2024-09-10T18:45:22.070065+0000","last_became_peered":"2024-09-10T18:45:22.070065+0000","last_unstale":"2024-09-10T18:48:33.516509+0000","last_undegraded":"2024-09-10T18:48:33.516509+0000","last_fullsized":"2024-09-10T18:48:33.516509+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:09.725488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:09.725488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:18.246207+0000","scrub_duration":0.0027892020000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":67,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.507437+0000","last_change":"2024-09-10T18:47:05.789893+0000","last_active":"2024-09-10T18:48:33.507437+0000","last_peered":"2024-09-10T18:48:33.507437+0000","last_clean":"2024-09-10T18:48:33.507437+0000","last_became_active":"2024-09-10T18:45:24.072224+0000","last_became_peered":"2024-09-10T18:45:24.072224+0000","last_unstale":"2024-09-10T18:48:33.507437+0000","last_undegraded":"2024-09-10T18:48:33.507437+0000","last_fullsized":"2024-09-10T18:48:33.507437+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:05.789830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:05.789830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001271938,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"63'4","reported_seq":87,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.510306+0000","last_change":"2024-09-10T18:48:30.455636+0000","last_active":"2024-09-10T18:48:33.510306+0000","last_peered":"2024-09-10T18:48:33.510306+0000","last_clean":"2024-09-10T18:48:33.510306+0000","last_became_active":"2024-09-10T18:45:20.059205+0000","last_became_peered":"2024-09-10T18:45:20.059205+0000","last_unstale":"2024-09-10T18:48:33.510306+0000","last_undegraded":"2024-09-10T18:48:33.510306+0000","last_fullsized":"2024-09-10T18:48:33.510306+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4","last_scrub_stamp":"2024-09-10T18:48:30.455433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:30.455433+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:32.585527+0000","scrub_duration":0.0078514339999999991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":83,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.562116+0000","last_change":"2024-09-10T18:47:36.416170+0000","last_active":"2024-09-10T18:48:34.562116+0000","last_peered":"2024-09-10T18:48:34.562116+0000","last_clean":"2024-09-10T18:48:34.562116+0000","last_became_active":"2024-09-10T18:47:36.415866+0000","last_became_peered":"2024-09-10T18:47:36.415866+0000","last_unstale":"2024-09-10T18:48:34.562116+0000","last_undegraded":"2024-09-10T18:48:34.562116+0000","last_fullsized":"2024-09-10T18:48:34.562116+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:54.082761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:54.082761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001210458,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":79,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.555907+0000","last_change":"2024-09-10T18:48:27.535769+0000","last_active":"2024-09-10T18:48:34.555907+0000","last_peered":"2024-09-10T18:48:34.555907+0000","last_clean":"2024-09-10T18:48:34.555907+0000","last_became_active":"2024-09-10T18:45:22.072377+0000","last_became_peered":"2024-09-10T18:45:22.072377+0000","last_unstale":"2024-09-10T18:48:34.555907+0000","last_undegraded":"2024-09-10T18:48:34.555907+0000","last_fullsized":"2024-09-10T18:48:34.555907+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:27.535676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_clean_scrub_stamp":"2024-09-10T18:48:27.535676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:36.593031+0000","scrub_duration":0.002291132,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":88,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.550996+0000","last_change":"2024-09-10T18:48:28.711019+0000","last_active":"2024-09-10T18:48:34.550996+0000","last_peered":"2024-09-10T18:48:34.550996+0000","last_clean":"2024-09-10T18:48:34.550996+0000","last_became_active":"2024-09-10T18:47:36.416187+0000","last_became_peered":"2024-09-10T18:47:36.416187+0000","last_unstale":"2024-09-10T18:48:34.550996+0000","last_undegraded":"2024-09-10T18:48:34.550996+0000","last_fullsized":"2024-09-10T18:48:34.550996+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:28.710918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:28.710918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:37.560639+0000","scrub_duration":0.0068877160000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"79'16","reported_seq":95,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.507602+0000","last_change":"2024-09-10T18:46:48.744340+0000","last_active":"2024-09-10T18:48:33.507602+0000","last_peered":"2024-09-10T18:48:33.507602+0000","last_clean":"2024-09-10T18:48:33.507602+0000","last_became_active":"2024-09-10T18:45:20.421479+0000","last_became_peered":"2024-09-10T18:45:20.421479+0000","last_unstale":"2024-09-10T18:48:33.507602+0000","last_undegraded":"2024-09-10T18:48:33.507602+0000","last_fullsized":"2024-09-10T18:48:33.507602+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"79'16","last_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_deep_scrub":"79'16","last_deep_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_clean_scrub_stamp":"2024-09-10T18:46:48.744287+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044879259999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"56'1","reported_seq":104,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.557318+0000","last_change":"2024-09-10T18:48:20.603918+0000","last_active":"2024-09-10T18:48:34.557318+0000","last_peered":"2024-09-10T18:48:34.557318+0000","last_clean":"2024-09-10T18:48:34.557318+0000","last_became_active":"2024-09-10T18:47:36.421985+0000","last_became_peered":"2024-09-10T18:47:36.421985+0000","last_unstale":"2024-09-10T18:48:34.557318+0000","last_undegraded":"2024-09-10T18:48:34.557318+0000","last_fullsized":"2024-09-10T18:48:34.557318+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:48:20.603805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:20.603805+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:49.866219+0000","scrub_duration":0.0039707509999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":33,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.507759+0000","last_change":"2024-09-10T18:48:09.323873+0000","last_active":"2024-09-10T18:48:33.507759+0000","last_peered":"2024-09-10T18:48:33.507759+0000","last_clean":"2024-09-10T18:48:33.507759+0000","last_became_active":"2024-09-10T18:47:57.107149+0000","last_became_peered":"2024-09-10T18:47:57.107149+0000","last_unstale":"2024-09-10T18:48:33.507759+0000","last_undegraded":"2024-09-10T18:48:33.507759+0000","last_fullsized":"2024-09-10T18:48:33.507759+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:09.323794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:09.323794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:28.508379+0000","scrub_duration":0.0056521669999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":77,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.506631+0000","last_change":"2024-09-10T18:48:16.410239+0000","last_active":"2024-09-10T18:48:33.506631+0000","last_peered":"2024-09-10T18:48:33.506631+0000","last_clean":"2024-09-10T18:48:33.506631+0000","last_became_active":"2024-09-10T18:45:24.264818+0000","last_became_peered":"2024-09-10T18:45:24.264818+0000","last_unstale":"2024-09-10T18:48:33.506631+0000","last_undegraded":"2024-09-10T18:48:33.506631+0000","last_fullsized":"2024-09-10T18:48:33.506631+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:16.410071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:16.410071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:22.248050+0000","scrub_duration":0.0061840250000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"78'2","reported_seq":94,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.551222+0000","last_change":"2024-09-10T18:48:10.610714+0000","last_active":"2024-09-10T18:48:34.551222+0000","last_peered":"2024-09-10T18:48:34.551222+0000","last_clean":"2024-09-10T18:48:34.551222+0000","last_became_active":"2024-09-10T18:47:36.418702+0000","last_became_peered":"2024-09-10T18:47:36.418702+0000","last_unstale":"2024-09-10T18:48:34.551222+0000","last_undegraded":"2024-09-10T18:48:34.551222+0000","last_fullsized":"2024-09-10T18:48:34.551222+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'2","last_scrub_stamp":"2024-09-10T18:48:10.610652+0000","last_deep_scrub":"78'2","last_deep_scrub_stamp":"2024-09-10T18:46:37.581130+0000","last_clean_scrub_stamp":"2024-09-10T18:48:10.610652+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:24.088784+0000","scrub_duration":0.002045275,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":73,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.561398+0000","last_change":"2024-09-10T18:46:28.088657+0000","last_active":"2024-09-10T18:48:34.561398+0000","last_peered":"2024-09-10T18:48:34.561398+0000","last_clean":"2024-09-10T18:48:34.561398+0000","last_became_active":"2024-09-10T18:45:17.962108+0000","last_became_peered":"2024-09-10T18:45:17.962108+0000","last_unstale":"2024-09-10T18:48:34.561398+0000","last_undegraded":"2024-09-10T18:48:34.561398+0000","last_fullsized":"2024-09-10T18:48:34.561398+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:28.088588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:28.088588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00140508,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"63'10","reported_seq":96,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.509334+0000","last_change":"2024-09-10T18:48:23.844841+0000","last_active":"2024-09-10T18:48:33.509334+0000","last_peered":"2024-09-10T18:48:33.509334+0000","last_clean":"2024-09-10T18:48:33.509334+0000","last_became_active":"2024-09-10T18:45:20.421094+0000","last_became_peered":"2024-09-10T18:45:20.421094+0000","last_unstale":"2024-09-10T18:48:33.509334+0000","last_undegraded":"2024-09-10T18:48:33.509334+0000","last_fullsized":"2024-09-10T18:48:33.509334+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:48:23.844741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:23.844741+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:49.178164+0000","scrub_duration":0.0025648530000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":130,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.115147+0000","last_change":"2024-09-10T18:48:19.605745+0000","last_active":"2024-09-10T18:48:36.115147+0000","last_peered":"2024-09-10T18:48:36.115147+0000","last_clean":"2024-09-10T18:48:36.115147+0000","last_became_active":"2024-09-10T18:45:22.073281+0000","last_became_peered":"2024-09-10T18:45:22.073281+0000","last_unstale":"2024-09-10T18:48:36.115147+0000","last_undegraded":"2024-09-10T18:48:36.115147+0000","last_fullsized":"2024-09-10T18:48:36.115147+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:48:19.605646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:19.605646+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:22.732603+0000","scrub_duration":0.001752665,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":67,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.515204+0000","last_change":"2024-09-10T18:47:03.475367+0000","last_active":"2024-09-10T18:48:33.515204+0000","last_peered":"2024-09-10T18:48:33.515204+0000","last_clean":"2024-09-10T18:48:33.515204+0000","last_became_active":"2024-09-10T18:45:24.082318+0000","last_became_peered":"2024-09-10T18:45:24.082318+0000","last_unstale":"2024-09-10T18:48:33.515204+0000","last_undegraded":"2024-09-10T18:48:33.515204+0000","last_fullsized":"2024-09-10T18:48:33.515204+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:03.475299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:03.475299+0000","last_clean_scrub_stamp":"2024-09-10T18:47:03.475299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001661945,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":73,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.510880+0000","last_change":"2024-09-10T18:46:44.694805+0000","last_active":"2024-09-10T18:48:33.510880+0000","last_peered":"2024-09-10T18:48:33.510880+0000","last_clean":"2024-09-10T18:48:33.510880+0000","last_became_active":"2024-09-10T18:45:17.965707+0000","last_became_peered":"2024-09-10T18:45:17.965707+0000","last_unstale":"2024-09-10T18:48:33.510880+0000","last_undegraded":"2024-09-10T18:48:33.510880+0000","last_fullsized":"2024-09-10T18:48:33.510880+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:44.694737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:44.694737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016330400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"63'11","reported_seq":100,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.557235+0000","last_change":"2024-09-10T18:48:23.546688+0000","last_active":"2024-09-10T18:48:34.557235+0000","last_peered":"2024-09-10T18:48:34.557235+0000","last_clean":"2024-09-10T18:48:34.557235+0000","last_became_active":"2024-09-10T18:45:20.058717+0000","last_became_peered":"2024-09-10T18:45:20.058717+0000","last_unstale":"2024-09-10T18:48:34.557235+0000","last_undegraded":"2024-09-10T18:48:34.557235+0000","last_fullsized":"2024-09-10T18:48:34.557235+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:48:23.546598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:23.546598+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:28.876387+0000","scrub_duration":0.0057630149999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":79,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.509261+0000","last_change":"2024-09-10T18:47:36.419088+0000","last_active":"2024-09-10T18:48:33.509261+0000","last_peered":"2024-09-10T18:48:33.509261+0000","last_clean":"2024-09-10T18:48:33.509261+0000","last_became_active":"2024-09-10T18:47:36.418767+0000","last_became_peered":"2024-09-10T18:47:36.418767+0000","last_unstale":"2024-09-10T18:48:33.509261+0000","last_undegraded":"2024-09-10T18:48:33.509261+0000","last_fullsized":"2024-09-10T18:48:33.509261+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:15.536620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:47:15.536620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015375409999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":67,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.552602+0000","last_change":"2024-09-10T18:46:45.285074+0000","last_active":"2024-09-10T18:48:34.552602+0000","last_peered":"2024-09-10T18:48:34.552602+0000","last_clean":"2024-09-10T18:48:34.552602+0000","last_became_active":"2024-09-10T18:45:24.267251+0000","last_became_peered":"2024-09-10T18:45:24.267251+0000","last_unstale":"2024-09-10T18:48:34.552602+0000","last_undegraded":"2024-09-10T18:48:34.552602+0000","last_fullsized":"2024-09-10T18:48:34.552602+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:45.285015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:45.285015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001362142,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"56'1","reported_seq":92,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.556419+0000","last_change":"2024-09-10T18:48:31.488364+0000","last_active":"2024-09-10T18:48:34.556419+0000","last_peered":"2024-09-10T18:48:34.556419+0000","last_clean":"2024-09-10T18:48:34.556419+0000","last_became_active":"2024-09-10T18:45:17.966286+0000","last_became_peered":"2024-09-10T18:45:17.966286+0000","last_unstale":"2024-09-10T18:48:34.556419+0000","last_undegraded":"2024-09-10T18:48:34.556419+0000","last_fullsized":"2024-09-10T18:48:34.556419+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:48:31.488272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:31.488272+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:37.912107+0000","scrub_duration":0.0028000949999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"63'15","reported_seq":87,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.517283+0000","last_change":"2024-09-10T18:47:36.446154+0000","last_active":"2024-09-10T18:48:33.517283+0000","last_peered":"2024-09-10T18:48:33.517283+0000","last_clean":"2024-09-10T18:48:33.517283+0000","last_became_active":"2024-09-10T18:47:36.424373+0000","last_became_peered":"2024-09-10T18:47:36.424373+0000","last_unstale":"2024-09-10T18:48:33.517283+0000","last_undegraded":"2024-09-10T18:48:33.517283+0000","last_fullsized":"2024-09-10T18:48:33.517283+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:47:34.610915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:34.610915+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:42.501880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":69,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.515442+0000","last_change":"2024-09-10T18:47:09.520829+0000","last_active":"2024-09-10T18:48:33.515442+0000","last_peered":"2024-09-10T18:48:33.515442+0000","last_clean":"2024-09-10T18:48:33.515442+0000","last_became_active":"2024-09-10T18:45:22.059794+0000","last_became_peered":"2024-09-10T18:45:22.059794+0000","last_unstale":"2024-09-10T18:48:33.515442+0000","last_undegraded":"2024-09-10T18:48:33.515442+0000","last_fullsized":"2024-09-10T18:48:33.515442+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:09.520768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:47:09.520768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016284120000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":67,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.508990+0000","last_change":"2024-09-10T18:46:57.811148+0000","last_active":"2024-09-10T18:48:33.508990+0000","last_peered":"2024-09-10T18:48:33.508990+0000","last_clean":"2024-09-10T18:48:33.508990+0000","last_became_active":"2024-09-10T18:45:24.264427+0000","last_became_peered":"2024-09-10T18:45:24.264427+0000","last_unstale":"2024-09-10T18:48:33.508990+0000","last_undegraded":"2024-09-10T18:48:33.508990+0000","last_fullsized":"2024-09-10T18:48:33.508990+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:57.811074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:57.811074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013689419999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":73,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.511266+0000","last_change":"2024-09-10T18:46:47.734860+0000","last_active":"2024-09-10T18:48:33.511266+0000","last_peered":"2024-09-10T18:48:33.511266+0000","last_clean":"2024-09-10T18:48:33.511266+0000","last_became_active":"2024-09-10T18:45:17.963257+0000","last_became_peered":"2024-09-10T18:45:17.963257+0000","last_unstale":"2024-09-10T18:48:33.511266+0000","last_undegraded":"2024-09-10T18:48:33.511266+0000","last_fullsized":"2024-09-10T18:48:33.511266+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:47.734803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:47.734803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001381134,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"63'12","reported_seq":94,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-10T18:48:33.506938+0000","last_change":"2024-09-10T18:46:52.771640+0000","last_active":"2024-09-10T18:48:33.506938+0000","last_peered":"2024-09-10T18:48:33.506938+0000","last_clean":"2024-09-10T18:48:33.506938+0000","last_became_active":"2024-09-10T18:45:20.058391+0000","last_became_peered":"2024-09-10T18:45:20.058391+0000","last_unstale":"2024-09-10T18:48:33.506938+0000","last_undegraded":"2024-09-10T18:48:33.506938+0000","last_fullsized":"2024-09-10T18:48:33.506938+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:46:52.771570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:52.771570+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021490089999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":89,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-10T18:48:34.551764+0000","last_change":"2024-09-10T18:47:36.416505+0000","last_active":"2024-09-10T18:48:34.551764+0000","last_peered":"2024-09-10T18:48:34.551764+0000","last_clean":"2024-09-10T18:48:34.551764+0000","last_became_active":"2024-09-10T18:47:36.416131+0000","last_became_peered":"2024-09-10T18:47:36.416131+0000","last_unstale":"2024-09-10T18:48:34.551764+0000","last_undegraded":"2024-09-10T18:48:34.551764+0000","last_fullsized":"2024-09-10T18:48:34.551764+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:34.958889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:47:34.958889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:57.335633+0000","scrub_duration":0.001401296,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":6163791872,"num_objects":94052,"num_object_clones":0,"num_object_copies":282156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94052,"num_write_kb":6019328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18418892800,"data_stored":18418892800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76752,"ondisk_log_size":76752,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":697,"num_read_kb":454,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":1238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":423,"ondisk_log_size":423,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":188,"num_read_kb":183,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299437,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2612888,"kb_used_data":2307216,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":91132264,"statfs":{"total":95995035648,"available":93319438336,"internally_reserved":0,"allocated":2362589184,"data_stored":2362081775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:48:08 2024","interfaces":[{"interface":"back","average":{"1min":2.1850000000000001,"5min":0.84199999999999997,"15min":0.61899999999999999},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":31.805,"5min":31.805,"15min":31.805},"last":0.34399999999999997},{"interface":"front","average":{"1min":2.0510000000000002,"5min":0.80600000000000005,"15min":0.59199999999999997},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":31.212,"5min":31.212,"15min":31.212},"last":0.84699999999999998}]},{"osd":1,"last update":"Tue Sep 10 18:48:08 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48899999999999999,"15min":0.48099999999999998},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.73799999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.314},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51500000000000001,"15min":0.51700000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.81799999999999995}]},{"osd":2,"last update":"Tue Sep 10 18:48:08 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":1.099},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52600000000000002,"15min":0.51900000000000002},"min":{"1min":0.36699999999999999,"5min":0.31,"15min":0.31},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.0880000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:48:08 2024","interfaces":[{"interface":"back","average":{"1min":2.1899999999999999,"5min":0.84199999999999997,"15min":0.61799999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":31.478000000000002,"5min":31.478000000000002,"15min":31.478000000000002},"last":3.2120000000000002},{"interface":"front","average":{"1min":2.6629999999999998,"5min":0.94999999999999996,"15min":0.66200000000000003},"min":{"1min":0.35199999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":33.731999999999999,"5min":33.731999999999999,"15min":33.731999999999999},"last":2.0150000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:48:08 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.38300000000000001,"15min":0.38400000000000001},"min":{"1min":0.27800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.55200000000000005,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39000000000000001,"15min":0.38},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.55800000000000005,"5min":0.623,"15min":0.623},"last":0.36499999999999999}]},{"osd":5,"last update":"Tue Sep 10 18:48:08 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40899999999999997,"15min":0.41099999999999998},"min":{"1min":0.29399999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.42099999999999999,"15min":0.41699999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.61699999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.51000000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:48:08 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.54200000000000004,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.432,"5min":0.44,"15min":0.435},"min":{"1min":0.32100000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.61199999999999999,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":1.016}]}]},{"osd":6,"up_from":43,"seq":184683593778,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2626656,"kb_used_data":2320984,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":91118496,"statfs":{"total":95995035648,"available":93305339904,"internally_reserved":0,"allocated":2376687616,"data_stored":2376172559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:47:44 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.48199999999999998,"15min":0.46600000000000003},"min":{"1min":0.26100000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":2.0600000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.48899999999999999,"15min":0.48999999999999999},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.69699999999999995,"5min":0.755,"15min":0.755},"last":3.1709999999999998}]},{"osd":1,"last update":"Tue Sep 10 18:47:44 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.29999999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.46600000000000003,"15min":0.46200000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":2.1909999999999998}]},{"osd":2,"last update":"Tue Sep 10 18:47:44 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48299999999999998,"15min":0.48099999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.49099999999999999,"15min":0.48999999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.39700000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:47:44 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.48799999999999999,"15min":0.48399999999999999},"min":{"1min":0.33700000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.51600000000000001,"15min":0.51200000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.46100000000000002}]},{"osd":4,"last update":"Tue Sep 10 18:47:44 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.40799999999999997,"15min":0.40600000000000003},"min":{"1min":0.23599999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.40100000000000002,"15min":0.38900000000000001},"min":{"1min":0.26700000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.66100000000000003,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.374}]},{"osd":5,"last update":"Tue Sep 10 18:47:44 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.41599999999999998},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.433,"15min":0.42499999999999999},"min":{"1min":0.309,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.58499999999999996,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.35699999999999998}]},{"osd":7,"last update":"Tue Sep 10 18:48:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.495,"15min":0.47699999999999998},"min":{"1min":0.32000000000000001,"5min":0.309,"15min":0.309},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.51600000000000001,"15min":0.501},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.883,"5min":1.883,"15min":1.883},"last":0.42499999999999999}]}]},{"osd":5,"up_from":37,"seq":158913790006,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1846664,"kb_used_data":1536832,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":91898488,"statfs":{"total":95995035648,"available":94104051712,"internally_reserved":0,"allocated":1573715968,"data_stored":1573223308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:48:22 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.53900000000000003,"15min":0.53100000000000003},"min":{"1min":0.376,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.999},{"interface":"front","average":{"1min":1.2789999999999999,"5min":0.64000000000000001,"15min":0.54600000000000004},"min":{"1min":0.40400000000000003,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":6.742,"5min":6.742,"15min":6.742},"last":0.93500000000000005}]},{"osd":1,"last update":"Tue Sep 10 18:48:22 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.746,"5min":1.746,"15min":1.746},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.224,"5min":0.65900000000000003,"15min":0.58099999999999996},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":6.6989999999999998,"5min":6.6989999999999998,"15min":6.6989999999999998},"last":1.7689999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:48:22 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.50800000000000001,"15min":0.51600000000000001},"min":{"1min":0.34799999999999998,"5min":0.311,"15min":0.311},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":4.0730000000000004},{"interface":"front","average":{"1min":1.3700000000000001,"5min":0.68799999999999994,"15min":0.59199999999999997},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":6.7210000000000001,"5min":6.7210000000000001,"15min":6.7210000000000001},"last":1.038}]},{"osd":3,"last update":"Tue Sep 10 18:48:22 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":0.624,"15min":0.56000000000000005},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":3.6760000000000002,"5min":3.6760000000000002,"15min":3.6760000000000002},"last":2.8570000000000002},{"interface":"front","average":{"1min":1.3999999999999999,"5min":0.69699999999999995,"15min":0.59699999999999998},"min":{"1min":0.372,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":5.8099999999999996,"5min":5.8099999999999996,"15min":5.8099999999999996},"last":0.45600000000000002}]},{"osd":4,"last update":"Tue Sep 10 18:48:22 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.39000000000000001,"15min":0.38500000000000001},"min":{"1min":0.248,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.88},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.38800000000000001,"15min":0.39800000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.502,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":1.0209999999999999}]},{"osd":6,"last update":"Tue Sep 10 18:47:47 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.42299999999999999,"15min":0.41699999999999998},"min":{"1min":0.29799999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46700000000000003,"15min":0.46100000000000002},"min":{"1min":0.33900000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.64100000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":1.1819999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:48:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.47099999999999997,"15min":0.47499999999999998},"min":{"1min":0.32100000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.62,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.46999999999999997,"15min":0.47499999999999998},"min":{"1min":0.30099999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.32900000000000001}]}]},{"osd":4,"up_from":30,"seq":128849018940,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2230100,"kb_used_data":1928588,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":91515052,"statfs":{"total":95995035648,"available":93711413248,"internally_reserved":0,"allocated":1974874112,"data_stored":1974369791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52800000000000002,"15min":0.52600000000000002},"min":{"1min":0.23200000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51600000000000001,"15min":0.53400000000000003},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.49099999999999999}]},{"osd":1,"last update":"Tue Sep 10 18:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.54900000000000004,"15min":0.56499999999999995},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.82499999999999996,"5min":2.609,"15min":2.609},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.55100000000000005,"15min":0.53300000000000003},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.98799999999999999,"5min":2.589,"15min":2.589},"last":0.438}]},{"osd":2,"last update":"Tue Sep 10 18:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.80000000000000004,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53500000000000003,"15min":0.55000000000000004},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.45000000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.53400000000000003,"15min":0.53200000000000003},"min":{"1min":0.33800000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.53300000000000003,"15min":0.54000000000000004},"min":{"1min":0.34799999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.997}]},{"osd":5,"last update":"Tue Sep 10 18:48:27 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.40699999999999997,"15min":0.36599999999999999},"min":{"1min":0.28899999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":2.8479999999999999,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.39900000000000002,"15min":0.36899999999999999},"min":{"1min":0.26000000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.47299999999999998}]},{"osd":6,"last update":"Tue Sep 10 18:47:43 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.436,"15min":0.42599999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.41199999999999998,"15min":0.39500000000000002},"min":{"1min":0.29899999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.39200000000000002}]},{"osd":7,"last update":"Tue Sep 10 18:48:12 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.49199999999999999,"15min":0.498},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.442,"15min":0.44700000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.70599999999999996}]}]},{"osd":3,"up_from":25,"seq":107374182464,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4279896,"kb_used_data":3982544,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":89465256,"statfs":{"total":95995035648,"available":91612422144,"internally_reserved":0,"allocated":4078125056,"data_stored":4077609867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,3,4],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:47:37 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.371,"15min":0.38100000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":2.6440000000000001},{"interface":"front","average":{"1min":0.375,"5min":0.39500000000000002,"15min":0.40100000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.77500000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.55800000000000005}]},{"osd":1,"last update":"Tue Sep 10 18:47:37 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.39100000000000001,"15min":0.40500000000000003},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.52600000000000002,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.40300000000000002,"15min":0.41299999999999998},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":2.2999999999999998}]},{"osd":2,"last update":"Tue Sep 10 18:47:37 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.42399999999999999,"15min":0.436},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":2.4100000000000001},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41499999999999998,"15min":0.43099999999999999},"min":{"1min":0.27500000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.31900000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:47:59 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.58499999999999996,"15min":0.55800000000000005},"min":{"1min":0.375,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":3.0699999999999998,"5min":3.0699999999999998,"15min":3.0699999999999998},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.60299999999999998,"15min":0.58499999999999996},"min":{"1min":0.35199999999999998,"5min":0.311,"15min":0.311},"max":{"1min":3.0990000000000002,"5min":3.0990000000000002,"15min":3.0990000000000002},"last":0.38800000000000001}]},{"osd":5,"last update":"Tue Sep 10 18:48:27 2024","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":0.67500000000000004,"15min":0.58499999999999996},"min":{"1min":0.433,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":2.9780000000000002,"5min":2.9780000000000002,"15min":2.9780000000000002},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.65000000000000002,"15min":0.59199999999999997},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":3.5819999999999999,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":2.5070000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:47:45 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55600000000000005,"15min":0.56200000000000006},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":2.4460000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.56899999999999995,"15min":0.56100000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.948,"5min":1.948,"15min":1.948},"last":0.753}]},{"osd":7,"last update":"Tue Sep 10 18:48:11 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.623,"15min":0.56100000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":2.694,"5min":2.694,"15min":2.694},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.60799999999999998,"15min":0.58599999999999997},"min":{"1min":0.434,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.665,"5min":2.665,"15min":2.665},"last":2.4780000000000002}]}]},{"osd":2,"up_from":0,"seq":77309411397,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3179616,"kb_used_data":2882260,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":90565536,"statfs":{"total":95995035648,"available":92739108864,"internally_reserved":0,"allocated":2951434240,"data_stored":2950915223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:47:50 2024","interfaces":[{"interface":"back","average":{"1min":0.28799999999999998,"5min":0.34000000000000002,"15min":0.31900000000000001},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.47199999999999998,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.374,"15min":0.33900000000000002},"min":{"1min":0.26200000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.64600000000000002,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":1.2929999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.42999999999999999,"15min":0.38},"min":{"1min":0.22900000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":3.8479999999999999,"5min":3.8479999999999999,"15min":3.8479999999999999},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.41199999999999998,"15min":0.38800000000000001},"min":{"1min":0.30299999999999999,"5min":0.248,"15min":0.248},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.42699999999999999}]},{"osd":3,"last update":"Tue Sep 10 18:47:39 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.438,"15min":0.45200000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.51200000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.373,"5min":0.41299999999999998,"15min":0.41899999999999998},"min":{"1min":0.30499999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.503,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":1.4099999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:48:03 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.63300000000000001,"15min":0.57899999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":4.1520000000000001,"5min":4.1520000000000001,"15min":4.1520000000000001},"last":1.3899999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.55300000000000005,"15min":0.53700000000000003},"min":{"1min":0.33700000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.47899999999999998}]},{"osd":5,"last update":"Tue Sep 10 18:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.63200000000000001,"15min":0.61399999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.68100000000000005,"15min":0.65600000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":4.1130000000000004,"5min":4.1130000000000004,"15min":4.1130000000000004},"last":1.3260000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:47:43 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.60899999999999999,"15min":0.624},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.70599999999999996,"5min":1.149,"15min":1.149},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.64100000000000001,"15min":0.66300000000000003},"min":{"1min":0.41399999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.81699999999999995,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":1.349}]},{"osd":7,"last update":"Tue Sep 10 18:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.64900000000000002,"15min":0.64800000000000002},"min":{"1min":0.41999999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.1800000000000002,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.70199999999999996,"15min":0.66600000000000004},"min":{"1min":0.42899999999999999,"5min":0.372,"15min":0.372},"max":{"1min":4.1989999999999998,"5min":4.1989999999999998,"15min":4.1989999999999998},"last":1.3700000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738446,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3199956,"kb_used_data":2885964,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":90545196,"statfs":{"total":95995035648,"available":92718280704,"internally_reserved":0,"allocated":2955227136,"data_stored":2954716335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,4],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 18:47:54 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.38300000000000001,"15min":0.32300000000000001},"min":{"1min":0.24199999999999999,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":1.4630000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.379,"15min":0.32900000000000001},"min":{"1min":0.23499999999999999,"5min":0.187,"15min":0.187},"max":{"1min":2.0990000000000002,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":0.878}]},{"osd":2,"last update":"Tue Sep 10 18:48:25 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.48399999999999999,"15min":0.41599999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":4.6509999999999998,"5min":4.6509999999999998,"15min":4.6509999999999998},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.47699999999999998,"15min":0.39500000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":4.6230000000000002,"5min":4.6230000000000002,"15min":4.6230000000000002},"last":0.80800000000000005}]},{"osd":3,"last update":"Tue Sep 10 18:47:35 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.38700000000000001,"15min":0.39500000000000002},"min":{"1min":0.27100000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.53800000000000003,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":1.181},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.40200000000000002,"15min":0.40999999999999998},"min":{"1min":0.29599999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.48599999999999999,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":1.421}]},{"osd":4,"last update":"Tue Sep 10 18:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.56799999999999995,"15min":0.54900000000000004},"min":{"1min":0.436,"5min":0.375,"15min":0.375},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":1.069},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.61699999999999999,"15min":0.55100000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":2.052,"5min":4.6189999999999998,"15min":4.6189999999999998},"last":1.51}]},{"osd":5,"last update":"Tue Sep 10 18:48:25 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.60899999999999999,"15min":0.57499999999999996},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":1.1279999999999999},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.68000000000000005,"15min":0.58899999999999997},"min":{"1min":0.40500000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.9580000000000002,"5min":4.6429999999999998,"15min":4.6429999999999998},"last":1.75}]},{"osd":6,"last update":"Tue Sep 10 18:47:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56699999999999995,"15min":0.55700000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":1.6200000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.70599999999999996,"15min":0.76000000000000001},"min":{"1min":0.39900000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.6539999999999999,"5min":4.6630000000000003,"15min":4.6630000000000003},"last":1.1020000000000001}]},{"osd":7,"last update":"Tue Sep 10 18:48:10 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.59799999999999998,"15min":0.56999999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":1.7749999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.64800000000000002,"15min":0.61099999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.5489999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2973368320,"data_stored":2973368320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2979856384,"data_stored":2979856384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4110811136,"data_stored":4110811136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1996750848,"data_stored":1996750848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1588854784,"data_stored":1588854784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2394882048,"data_stored":2394882048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2374369280,"data_stored":2374369280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T18:48:44.981 INFO:tasks.cephadm.ceph_manager.ceph:would drop below min_pgs, proj 6, currently 16 2024-09-10T18:48:45.011 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:45.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:45.012 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:45.113 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:45.152 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:45.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:45.153 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:45.212 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 61 16 102327 102311 104.801 83.625 0.00776254 0.00953744 2024-09-10T18:48:45.255 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:45.288 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:45.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:45.289 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:45.390 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:45.427 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:45.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:45.428 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:45.529 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:45.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:45.558 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:45.558 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:45.659 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:45.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:45.695 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:45.695 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:45.796 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:45.825 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:45.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:45.826 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:45.927 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:45.957 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:45.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:45.958 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:46.059 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:46.101 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:46.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:46.102 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:46.122 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:46 smithi018 ceph-mon[29336]: pgmap v129: 148 pgs: 148 active+clean; 5.9 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1023 B/s rd, 78 MiB/s wr, 1.25k op/s 2024-09-10T18:48:46.122 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:46 smithi018 ceph-mon[40196]: pgmap v129: 148 pgs: 148 active+clean; 5.9 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1023 B/s rd, 78 MiB/s wr, 1.25k op/s 2024-09-10T18:48:46.204 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:46.212 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 62 16 103634 103618 104.428 81.6875 0.00948502 0.00957149 2024-09-10T18:48:46.230 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:46.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:46.231 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:46.332 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:46.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:46.377 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:46.377 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:46.477 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:46.519 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:46.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:46.520 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:46.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:46 smithi103 ceph-mon[33163]: pgmap v129: 148 pgs: 148 active+clean; 5.9 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1023 B/s rd, 78 MiB/s wr, 1.25k op/s 2024-09-10T18:48:46.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:48:46 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:48:46.621 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:46.649 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:46.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:46.650 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:46.751 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:46.779 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:46.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:46.781 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:46.881 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:46.919 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:46.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:46.921 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:47.022 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:47.061 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:47.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:47.062 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:47.117 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is set 2024-09-10T18:48:47.163 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:47.208 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:47.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:47.209 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:47.212 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 63 16 104870 104854 103.997 77.25 0.0197242 0.00961093 2024-09-10T18:48:47.311 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:47.348 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:47.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:47.349 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:47.424 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:47 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:47.424 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:47 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1754661159' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:48:47.424 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:47 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:48:47.424 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:47 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:47.425 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:47 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1754661159' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:48:47.425 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:47 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:48:47.450 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:47.488 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:47.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:47.489 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:47.525 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:47 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:47.525 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:47 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1754661159' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:48:47.525 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:47 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:48:47.590 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:47.632 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:47.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:47.633 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:47.733 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:47.761 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:47.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:47.763 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:47.863 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:47.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:47.893 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:47.893 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:47.994 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:48.022 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:48.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:48.023 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:48.123 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:48.155 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:48.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:48.156 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:48.212 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 64 16 106197 106181 103.668 82.9375 0.0161961 0.00964138 2024-09-10T18:48:48.257 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:48.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:48.300 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:48.301 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:48.368 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:48 smithi018 ceph-mon[40196]: pgmap v130: 148 pgs: 148 active+clean; 6.3 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 976 B/s rd, 74 MiB/s wr, 1.19k op/s 2024-09-10T18:48:48.368 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:48 smithi018 ceph-mon[40196]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:48:48.368 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:48 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:48:48.368 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:48 smithi018 ceph-mon[40196]: osdmap e112: 8 total, 8 up, 7 in 2024-09-10T18:48:48.368 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:48 smithi018 ceph-mon[29336]: pgmap v130: 148 pgs: 148 active+clean; 6.3 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 976 B/s rd, 74 MiB/s wr, 1.19k op/s 2024-09-10T18:48:48.369 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:48 smithi018 ceph-mon[29336]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:48:48.369 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:48 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:48:48.369 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:48 smithi018 ceph-mon[29336]: osdmap e112: 8 total, 8 up, 7 in 2024-09-10T18:48:48.401 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:48.432 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:48.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:48.434 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:48.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:48 smithi103 ceph-mon[33163]: pgmap v130: 148 pgs: 148 active+clean; 6.3 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 976 B/s rd, 74 MiB/s wr, 1.19k op/s 2024-09-10T18:48:48.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:48 smithi103 ceph-mon[33163]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:48:48.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:48 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:48:48.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:48 smithi103 ceph-mon[33163]: osdmap e112: 8 total, 8 up, 7 in 2024-09-10T18:48:48.535 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:48.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:48.578 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:48.578 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:48.679 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:48.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:48.712 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:48.712 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:48.813 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:48.854 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:48.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:48.855 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:48.878 INFO:teuthology.orchestra.run.smithi018.stdout:ERROR: (22) Invalid argument 2024-09-10T18:48:48.878 INFO:teuthology.orchestra.run.smithi018.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:48:48.956 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:48.984 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:49.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:49.002 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:49.102 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:49.134 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:49.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:49.135 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:49.212 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 65 16 107430 107414 103.259 77.0625 0.00937312 0.00967979 2024-09-10T18:48:49.236 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:49.246 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:48:49.246 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:48:49.246 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:48:49.246 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:48:49.247 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:48:49.247 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:48:49.247 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:48:49.248 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:48:49.277 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:49.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:49.278 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:49.280 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:48:49.280 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:48:49.315 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:48:49.315 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:48:49.344 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:48:49.345 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:48:49.360 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:48:49.360 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:48:49.379 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:49.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:49.401 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:49.402 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:49.502 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:49.530 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:49.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:49.532 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:49.633 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:49.664 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:49.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:49.665 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:49.765 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:49.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:49 smithi103 ceph-mon[33163]: pgmap v132: 148 pgs: 148 active+clean; 6.3 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1023 B/s rd, 78 MiB/s wr, 1.25k op/s 2024-09-10T18:48:49.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:49 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/429217185' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:48:49.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:49.805 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:49.805 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:49.874 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-10T18:48:49.906 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:49.913 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:49 smithi018 ceph-mon[40196]: pgmap v132: 148 pgs: 148 active+clean; 6.3 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1023 B/s rd, 78 MiB/s wr, 1.25k op/s 2024-09-10T18:48:49.913 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:49 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/429217185' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:48:49.914 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:49 smithi018 ceph-mon[29336]: pgmap v132: 148 pgs: 148 active+clean; 6.3 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1023 B/s rd, 78 MiB/s wr, 1.25k op/s 2024-09-10T18:48:49.914 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:49 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/429217185' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:48:49.955 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:49.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:49.957 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:49.982 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-10T18:48:49.982 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T18:48:49.983 INFO:tasks.thrashosds.thrasher:Shrinking pool .mgr 2024-09-10T18:48:49.983 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool .mgr by 10 min 8 2024-09-10T18:48:49.983 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T18:48:50.058 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:50.087 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:50.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:50.088 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:50.189 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:50.213 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 66 16 108545 108529 102.749 69.6875 0.0146324 0.00972734 2024-09-10T18:48:50.215 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:50.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:50.216 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:50.317 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:50.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:50.377 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:50.377 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:50.478 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:50.514 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:50.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:50.515 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:50.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:48:50 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:48:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:48:50.615 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:50.647 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:50.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:50.648 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:50.749 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:50.781 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:50.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:50.782 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:50.882 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:50.912 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:50.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:50.913 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:51.014 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:51.057 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:51.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:51.058 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:51.159 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:51.182 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:51.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:51.183 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:51.213 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 67 16 109602 109586 102.201 66.0625 0.00785443 0.00978003 2024-09-10T18:48:51.284 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:51.312 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:51.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:51.313 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:51.414 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:51.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:48:50 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:48:50] "GET /metrics HTTP/1.1" 200 223564 "" "Prometheus/2.43.0" 2024-09-10T18:48:51.453 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:51.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:51.454 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:51.554 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:51.593 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:51.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:51.594 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:51.677 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-10T18:48:51.695 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:51.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:51.720 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:51.721 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:51.821 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:51.875 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:51.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:51.876 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:51.977 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:52.002 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:52.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:52.003 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:52.104 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:52.148 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:52.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:52.149 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:52.213 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 68 16 110841 110825 101.837 77.4375 0.0144557 0.00981487 2024-09-10T18:48:52.250 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:52.291 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:52.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:52.292 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:52.381 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:52 smithi018 ceph-mon[40196]: pgmap v133: 148 pgs: 148 active+clean; 6.5 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 1023 B/s rd, 78 MiB/s wr, 1.25k op/s 2024-09-10T18:48:52.381 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:52 smithi018 ceph-mon[29336]: pgmap v133: 148 pgs: 148 active+clean; 6.5 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 1023 B/s rd, 78 MiB/s wr, 1.25k op/s 2024-09-10T18:48:52.393 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:52.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:52.439 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:52.439 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:52.486 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:48:52.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:52 smithi103 ceph-mon[33163]: pgmap v133: 148 pgs: 148 active+clean; 6.5 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 1023 B/s rd, 78 MiB/s wr, 1.25k op/s 2024-09-10T18:48:52.541 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:52.561 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:48:52.570 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:52.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:52.571 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:52.672 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:52.699 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:52.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:52.700 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:52.801 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:52.847 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:52.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:52.848 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:52.948 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:52.979 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:52.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:52.980 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:53.081 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:53.114 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:53.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:53.115 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:53.214 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 69 16 112064 112048 101.469 76.4375 0.0299608 0.00984995 2024-09-10T18:48:53.215 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:53.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:53.248 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:53.248 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:53.349 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:53.391 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:53.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:53.392 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:53.493 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:53.528 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:53.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:53.529 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:53.630 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:53.668 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:53.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:53.669 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:53.770 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:53.809 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:53.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:53.810 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:53.911 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:53.950 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:53.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:53.951 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:54.052 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:54.096 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:54.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:54.097 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:54.198 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:54.213 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 70 16 113371 113355 101.186 81.6875 0.0127779 0.0098775 2024-09-10T18:48:54.236 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:54.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:54.237 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:54.338 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:54.371 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:54.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:54.372 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:54.402 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:54 smithi018 ceph-mon[29336]: pgmap v134: 148 pgs: 148 active+clean; 6.7 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 1023 B/s rd, 77 MiB/s wr, 1.24k op/s 2024-09-10T18:48:54.403 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:54 smithi018 ceph-mon[40196]: pgmap v134: 148 pgs: 148 active+clean; 6.7 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 1023 B/s rd, 77 MiB/s wr, 1.24k op/s 2024-09-10T18:48:54.439 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T18:48:54.473 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:54.493 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:48:54.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:54 smithi103 ceph-mon[33163]: pgmap v134: 148 pgs: 148 active+clean; 6.7 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 1023 B/s rd, 77 MiB/s wr, 1.24k op/s 2024-09-10T18:48:54.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:54.564 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:54.565 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:54.665 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:54.691 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:54.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:54.692 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:54.793 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:54.849 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:54.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:54.850 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:54.951 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:55.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:55.015 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:55.016 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:55.117 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:55.168 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:55.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:55.170 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:55.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:48:55.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:48:55.213 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 71 16 114545 114529 100.794 73.375 0.0185093 0.00991666 2024-09-10T18:48:55.271 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:55.300 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:55.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:55.301 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:55.402 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:55.448 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:55.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:55.449 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:55.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:48:55.550 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:55.591 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:55.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:55.592 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:55.693 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:55.731 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:55.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:55.732 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:55.832 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:55.861 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:55.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:55.862 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:55.963 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:56.004 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:56.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:56.005 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:56.106 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:56.149 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:56 smithi018 ceph-mon[29336]: pgmap v135: 148 pgs: 148 active+clean; 6.7 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 1023 B/s rd, 77 MiB/s wr, 1.24k op/s 2024-09-10T18:48:56.156 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:56.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:56.157 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:56.214 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 72 16 115824 115808 100.505 79.9375 0.00606956 0.00994507 2024-09-10T18:48:56.258 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:56.289 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:56.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:56.290 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:56.391 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:56.412 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:56 smithi018 ceph-mon[40196]: pgmap v135: 148 pgs: 148 active+clean; 6.7 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 1023 B/s rd, 77 MiB/s wr, 1.24k op/s 2024-09-10T18:48:56.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:56.441 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:56.441 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:56.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:56 smithi103 ceph-mon[33163]: pgmap v135: 148 pgs: 148 active+clean; 6.7 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 1023 B/s rd, 77 MiB/s wr, 1.24k op/s 2024-09-10T18:48:56.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:48:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:48:56.542 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:56.581 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:56.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:56.582 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:56.683 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:56.710 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:56.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:56.711 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:56.812 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:56.838 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:56.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:56.839 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:56.841 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:48:56.850 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-09-10T18:48:56.940 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:56.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:56.963 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:56.963 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:57.064 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:57.109 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:57.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:57.111 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:57.211 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:57.214 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 73 15 116848 116833 100.005 64.0625 0.010419 0.009995 2024-09-10T18:48:57.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:57.239 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:57.239 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:57.340 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:57.366 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:57.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:57.367 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:57.424 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:57 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:57.424 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:57 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:57.468 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:57.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:57.520 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:57.521 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:57.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:57 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:57.621 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:57.649 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:57.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:57.650 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:57.751 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:57.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:57.779 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:57.779 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:57.880 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:57.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:57.941 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:57.941 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:58.042 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:48:58.067 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:58.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:58.069 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:48:58.167 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is unset 2024-09-10T18:48:58.169 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:58.192 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:58.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:58.193 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:58.214 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 74 16 117842 117826 99.4927 62.0625 0.018898 0.010039 2024-09-10T18:48:58.294 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:48:58.340 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:58.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:58.341 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:48:58.442 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:58.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:58 smithi018 ceph-mon[40196]: from='client.24706 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:48:58.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:58 smithi018 ceph-mon[40196]: pgmap v136: 148 pgs: 148 active+clean; 7.1 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 1023 B/s rd, 76 MiB/s wr, 1.22k op/s 2024-09-10T18:48:58.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:58 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1137121019' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:48:58.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:58 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:48:58.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:58 smithi018 ceph-mon[29336]: from='client.24706 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:48:58.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:58 smithi018 ceph-mon[29336]: pgmap v136: 148 pgs: 148 active+clean; 7.1 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 1023 B/s rd, 76 MiB/s wr, 1.22k op/s 2024-09-10T18:48:58.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:58 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1137121019' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:48:58.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:58 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:48:58.474 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:58.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:58.475 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:58.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:58 smithi103 ceph-mon[33163]: from='client.24706 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:48:58.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:58 smithi103 ceph-mon[33163]: pgmap v136: 148 pgs: 148 active+clean; 7.1 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 1023 B/s rd, 76 MiB/s wr, 1.22k op/s 2024-09-10T18:48:58.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:58 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1137121019' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:48:58.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:58 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:48:58.576 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:58.619 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:58.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:58.621 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:58.722 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:48:58.748 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:58.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:58.749 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:48:58.850 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:58.890 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:58.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:58.891 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:58.970 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":135,"stamp":"2024-09-10T18:48:55.074034+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":7175215324,"num_objects":109672,"num_object_clones":0,"num_object_copies":329016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109672,"num_whiteouts":0,"num_read":1055,"num_read_kb":912,"num_write":110019,"num_write_kb":7008135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":1183107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77364,"ondisk_log_size":77364,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":23045032,"kb_used_data":20909488,"kb_used_omap":11,"kb_used_meta":2135476,"kb_avail":633171032,"statfs":{"total":671965249536,"available":648367136768,"internally_reserved":0,"allocated":21411315712,"data_stored":21408496798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11928,"internal_metadata":2186727784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":811204608,"num_objects":12378,"num_object_clones":0,"num_object_copies":37134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12378,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":12378,"num_write_kb":792192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003092"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":79,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.596727+0000","last_change":"2024-09-10T18:48:24.515065+0000","last_active":"2024-09-10T18:48:36.596727+0000","last_peered":"2024-09-10T18:48:36.596727+0000","last_clean":"2024-09-10T18:48:36.596727+0000","last_became_active":"2024-09-10T18:45:24.265488+0000","last_became_peered":"2024-09-10T18:45:24.265488+0000","last_unstale":"2024-09-10T18:48:36.596727+0000","last_undegraded":"2024-09-10T18:48:36.596727+0000","last_fullsized":"2024-09-10T18:48:36.596727+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:24.511946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:24.511946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:49.630089+0000","scrub_duration":0.014161415,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":85,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.597523+0000","last_change":"2024-09-10T18:48:29.718356+0000","last_active":"2024-09-10T18:48:36.597523+0000","last_peered":"2024-09-10T18:48:36.597523+0000","last_clean":"2024-09-10T18:48:36.597523+0000","last_became_active":"2024-09-10T18:45:17.959475+0000","last_became_peered":"2024-09-10T18:45:17.959475+0000","last_unstale":"2024-09-10T18:48:36.597523+0000","last_undegraded":"2024-09-10T18:48:36.597523+0000","last_fullsized":"2024-09-10T18:48:36.597523+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:29.718270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.718270+0000","last_clean_scrub_stamp":"2024-09-10T18:48:29.718270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:59.632977+0000","scrub_duration":0.010834984000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"63'15","reported_seq":124,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.597800+0000","last_change":"2024-09-10T18:47:36.436893+0000","last_active":"2024-09-10T18:48:36.597800+0000","last_peered":"2024-09-10T18:48:36.597800+0000","last_clean":"2024-09-10T18:48:36.597800+0000","last_became_active":"2024-09-10T18:47:36.413635+0000","last_became_peered":"2024-09-10T18:47:36.413635+0000","last_unstale":"2024-09-10T18:48:36.597800+0000","last_undegraded":"2024-09-10T18:48:36.597800+0000","last_fullsized":"2024-09-10T18:48:36.597800+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:46:36.856740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:36.856740+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018131040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":80,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.585899+0000","last_change":"2024-09-10T18:48:29.058001+0000","last_active":"2024-09-10T18:48:36.585899+0000","last_peered":"2024-09-10T18:48:36.585899+0000","last_clean":"2024-09-10T18:48:36.585899+0000","last_became_active":"2024-09-10T18:45:22.065619+0000","last_became_peered":"2024-09-10T18:45:22.065619+0000","last_unstale":"2024-09-10T18:48:36.585899+0000","last_undegraded":"2024-09-10T18:48:36.585899+0000","last_fullsized":"2024-09-10T18:48:36.585899+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:29.057932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.057932+0000","last_clean_scrub_stamp":"2024-09-10T18:48:29.057932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:39.110015+0000","scrub_duration":0.001460184,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":85,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.598304+0000","last_change":"2024-09-10T18:48:29.458396+0000","last_active":"2024-09-10T18:48:36.598304+0000","last_peered":"2024-09-10T18:48:36.598304+0000","last_clean":"2024-09-10T18:48:36.598304+0000","last_became_active":"2024-09-10T18:45:17.966153+0000","last_became_peered":"2024-09-10T18:45:17.966153+0000","last_unstale":"2024-09-10T18:48:36.598304+0000","last_undegraded":"2024-09-10T18:48:36.598304+0000","last_fullsized":"2024-09-10T18:48:36.598304+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:29.457892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:29.457892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:50.349265+0000","scrub_duration":0.012091342999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"83'13","reported_seq":92,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.586028+0000","last_change":"2024-09-10T18:46:59.906707+0000","last_active":"2024-09-10T18:48:36.586028+0000","last_peered":"2024-09-10T18:48:36.586028+0000","last_clean":"2024-09-10T18:48:36.586028+0000","last_became_active":"2024-09-10T18:45:20.061266+0000","last_became_peered":"2024-09-10T18:45:20.061266+0000","last_unstale":"2024-09-10T18:48:36.586028+0000","last_undegraded":"2024-09-10T18:48:36.586028+0000","last_fullsized":"2024-09-10T18:48:36.586028+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'13","last_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_clean_scrub_stamp":"2024-09-10T18:46:59.906634+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003988892,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":71,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.602713+0000","last_change":"2024-09-10T18:47:21.589428+0000","last_active":"2024-09-10T18:48:36.602713+0000","last_peered":"2024-09-10T18:48:36.602713+0000","last_clean":"2024-09-10T18:48:36.602713+0000","last_became_active":"2024-09-10T18:45:22.072822+0000","last_became_peered":"2024-09-10T18:45:22.072822+0000","last_unstale":"2024-09-10T18:48:36.602713+0000","last_undegraded":"2024-09-10T18:48:36.602713+0000","last_fullsized":"2024-09-10T18:48:36.602713+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:21.589365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:47:21.589365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001676893,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":68,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.586063+0000","last_change":"2024-09-10T18:47:00.946309+0000","last_active":"2024-09-10T18:48:36.586063+0000","last_peered":"2024-09-10T18:48:36.586063+0000","last_clean":"2024-09-10T18:48:36.586063+0000","last_became_active":"2024-09-10T18:45:24.081002+0000","last_became_peered":"2024-09-10T18:45:24.081002+0000","last_unstale":"2024-09-10T18:48:36.586063+0000","last_undegraded":"2024-09-10T18:48:36.586063+0000","last_fullsized":"2024-09-10T18:48:36.586063+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:00.946255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:00.946255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001207629,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":85,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.599961+0000","last_change":"2024-09-10T18:48:13.694567+0000","last_active":"2024-09-10T18:48:36.599961+0000","last_peered":"2024-09-10T18:48:36.599961+0000","last_clean":"2024-09-10T18:48:36.599961+0000","last_became_active":"2024-09-10T18:45:17.961097+0000","last_became_peered":"2024-09-10T18:45:17.961097+0000","last_unstale":"2024-09-10T18:48:36.599961+0000","last_undegraded":"2024-09-10T18:48:36.599961+0000","last_fullsized":"2024-09-10T18:48:36.599961+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:13.694295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:13.694295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:28.048348+0000","scrub_duration":0.0051419839999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"63'5","reported_seq":82,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.586218+0000","last_change":"2024-09-10T18:47:04.002228+0000","last_active":"2024-09-10T18:48:36.586218+0000","last_peered":"2024-09-10T18:48:36.586218+0000","last_clean":"2024-09-10T18:48:36.586218+0000","last_became_active":"2024-09-10T18:45:20.057567+0000","last_became_peered":"2024-09-10T18:45:20.057567+0000","last_unstale":"2024-09-10T18:48:36.586218+0000","last_undegraded":"2024-09-10T18:48:36.586218+0000","last_fullsized":"2024-09-10T18:48:36.586218+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-10T18:47:04.002164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:04.002164+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.026374999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":81,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.598051+0000","last_change":"2024-09-10T18:47:36.510027+0000","last_active":"2024-09-10T18:48:36.598051+0000","last_peered":"2024-09-10T18:48:36.598051+0000","last_clean":"2024-09-10T18:48:36.598051+0000","last_became_active":"2024-09-10T18:45:22.076575+0000","last_became_peered":"2024-09-10T18:45:22.076575+0000","last_unstale":"2024-09-10T18:48:36.598051+0000","last_undegraded":"2024-09-10T18:48:36.598051+0000","last_fullsized":"2024-09-10T18:48:36.598051+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:36.509971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:36.509971+0000","last_clean_scrub_stamp":"2024-09-10T18:47:36.509971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:47.471372+0000","scrub_duration":0.0013665439999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":78,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.597994+0000","last_change":"2024-09-10T18:48:10.102991+0000","last_active":"2024-09-10T18:48:36.597994+0000","last_peered":"2024-09-10T18:48:36.597994+0000","last_clean":"2024-09-10T18:48:36.597994+0000","last_became_active":"2024-09-10T18:45:24.079534+0000","last_became_peered":"2024-09-10T18:45:24.079534+0000","last_unstale":"2024-09-10T18:48:36.597994+0000","last_undegraded":"2024-09-10T18:48:36.597994+0000","last_fullsized":"2024-09-10T18:48:36.597994+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:10.102865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:10.102865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:36.015071+0000","scrub_duration":0.0042821509999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":85,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.599153+0000","last_change":"2024-09-10T18:47:36.417065+0000","last_active":"2024-09-10T18:48:36.599153+0000","last_peered":"2024-09-10T18:48:36.599153+0000","last_clean":"2024-09-10T18:48:36.599153+0000","last_became_active":"2024-09-10T18:47:36.414393+0000","last_became_peered":"2024-09-10T18:47:36.414393+0000","last_unstale":"2024-09-10T18:48:36.599153+0000","last_undegraded":"2024-09-10T18:48:36.599153+0000","last_fullsized":"2024-09-10T18:48:36.599153+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:50.666379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.666379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016221689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"63'9","reported_seq":88,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.586202+0000","last_change":"2024-09-10T18:46:55.069893+0000","last_active":"2024-09-10T18:48:36.586202+0000","last_peered":"2024-09-10T18:48:36.586202+0000","last_clean":"2024-09-10T18:48:36.586202+0000","last_became_active":"2024-09-10T18:45:20.052652+0000","last_became_peered":"2024-09-10T18:45:20.052652+0000","last_unstale":"2024-09-10T18:48:36.586202+0000","last_undegraded":"2024-09-10T18:48:36.586202+0000","last_fullsized":"2024-09-10T18:48:36.586202+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:46:55.069833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:55.069833+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016685510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":80,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.598392+0000","last_change":"2024-09-10T18:48:32.141849+0000","last_active":"2024-09-10T18:48:36.598392+0000","last_peered":"2024-09-10T18:48:36.598392+0000","last_clean":"2024-09-10T18:48:36.598392+0000","last_became_active":"2024-09-10T18:45:22.068326+0000","last_became_peered":"2024-09-10T18:45:22.068326+0000","last_unstale":"2024-09-10T18:48:36.598392+0000","last_undegraded":"2024-09-10T18:48:36.598392+0000","last_fullsized":"2024-09-10T18:48:36.598392+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:32.141755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:32.141755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:42.742732+0000","scrub_duration":0.0040319509999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":69,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.587544+0000","last_change":"2024-09-10T18:47:22.557898+0000","last_active":"2024-09-10T18:48:36.587544+0000","last_peered":"2024-09-10T18:48:36.587544+0000","last_clean":"2024-09-10T18:48:36.587544+0000","last_became_active":"2024-09-10T18:45:24.081488+0000","last_became_peered":"2024-09-10T18:45:24.081488+0000","last_unstale":"2024-09-10T18:48:36.587544+0000","last_undegraded":"2024-09-10T18:48:36.587544+0000","last_fullsized":"2024-09-10T18:48:36.587544+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:22.557837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:22.557837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001691374,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"112'6794","reported_seq":6829,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T18:48:51.007755+0000","last_change":"2024-09-10T18:47:36.423039+0000","last_active":"2024-09-10T18:48:51.007755+0000","last_peered":"2024-09-10T18:48:51.007755+0000","last_clean":"2024-09-10T18:48:51.007755+0000","last_became_active":"2024-09-10T18:47:36.422694+0000","last_became_peered":"2024-09-10T18:47:36.422694+0000","last_unstale":"2024-09-10T18:48:51.007755+0000","last_undegraded":"2024-09-10T18:48:51.007755+0000","last_fullsized":"2024-09-10T18:48:51.007755+0000","mapping_epoch":88,"log_start":"95'1600","ondisk_log_start":"95'1600","created":86,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":5194,"log_dups_size":0,"ondisk_log_size":5194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":445251584,"num_objects":6794,"num_object_clones":0,"num_object_copies":20382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6794,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6794,"num_write_kb":434816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"78'25","reported_seq":136,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.586514+0000","last_change":"2024-09-10T18:47:36.429540+0000","last_active":"2024-09-10T18:48:36.586514+0000","last_peered":"2024-09-10T18:48:36.586514+0000","last_clean":"2024-09-10T18:48:36.586514+0000","last_became_active":"2024-09-10T18:47:36.409300+0000","last_became_peered":"2024-09-10T18:47:36.409300+0000","last_unstale":"2024-09-10T18:48:36.586514+0000","last_undegraded":"2024-09-10T18:48:36.586514+0000","last_fullsized":"2024-09-10T18:48:36.586514+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'25","last_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_deep_scrub":"78'25","last_deep_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_clean_scrub_stamp":"2024-09-10T18:46:44.316146+0000","objects_scrubbed":8,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046415040000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":85,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.598838+0000","last_change":"2024-09-10T18:48:26.625704+0000","last_active":"2024-09-10T18:48:36.598838+0000","last_peered":"2024-09-10T18:48:36.598838+0000","last_clean":"2024-09-10T18:48:36.598838+0000","last_became_active":"2024-09-10T18:45:17.963732+0000","last_became_peered":"2024-09-10T18:45:17.963732+0000","last_unstale":"2024-09-10T18:48:36.598838+0000","last_undegraded":"2024-09-10T18:48:36.598838+0000","last_fullsized":"2024-09-10T18:48:36.598838+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:26.625635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:26.625635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:44.826250+0000","scrub_duration":0.0034872639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":71,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.604409+0000","last_change":"2024-09-10T18:47:04.512478+0000","last_active":"2024-09-10T18:48:36.604409+0000","last_peered":"2024-09-10T18:48:36.604409+0000","last_clean":"2024-09-10T18:48:36.604409+0000","last_became_active":"2024-09-10T18:45:22.071443+0000","last_became_peered":"2024-09-10T18:45:22.071443+0000","last_unstale":"2024-09-10T18:48:36.604409+0000","last_undegraded":"2024-09-10T18:48:36.604409+0000","last_fullsized":"2024-09-10T18:48:36.604409+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:04.512425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:04.512425+0000","last_clean_scrub_stamp":"2024-09-10T18:47:04.512425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001363324,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":25,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.591940+0000","last_change":"2024-09-10T18:47:57.104885+0000","last_active":"2024-09-10T18:48:36.591940+0000","last_peered":"2024-09-10T18:48:36.591940+0000","last_clean":"2024-09-10T18:48:36.591940+0000","last_became_active":"2024-09-10T18:47:57.104433+0000","last_became_peered":"2024-09-10T18:47:57.104433+0000","last_unstale":"2024-09-10T18:48:36.591940+0000","last_undegraded":"2024-09-10T18:48:36.591940+0000","last_fullsized":"2024-09-10T18:48:36.591940+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:38.016595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:38.016595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015035510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.f","version":"112'6821","reported_seq":6855,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T18:48:51.151640+0000","last_change":"2024-09-10T18:47:34.396342+0000","last_active":"2024-09-10T18:48:51.151640+0000","last_peered":"2024-09-10T18:48:51.151640+0000","last_clean":"2024-09-10T18:48:51.151640+0000","last_became_active":"2024-09-10T18:47:34.396087+0000","last_became_peered":"2024-09-10T18:47:34.396087+0000","last_unstale":"2024-09-10T18:48:51.151640+0000","last_undegraded":"2024-09-10T18:48:51.151640+0000","last_fullsized":"2024-09-10T18:48:51.151640+0000","mapping_epoch":86,"log_start":"96'2000","ondisk_log_start":"96'2000","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":4821,"log_dups_size":0,"ondisk_log_size":4821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":447021056,"num_objects":6821,"num_object_clones":0,"num_object_copies":20463,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6821,"num_write_kb":436544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"63'10","reported_seq":87,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.588611+0000","last_change":"2024-09-10T18:46:31.281588+0000","last_active":"2024-09-10T18:48:36.588611+0000","last_peered":"2024-09-10T18:48:36.588611+0000","last_clean":"2024-09-10T18:48:36.588611+0000","last_became_active":"2024-09-10T18:45:20.053428+0000","last_became_peered":"2024-09-10T18:45:20.053428+0000","last_unstale":"2024-09-10T18:48:36.588611+0000","last_undegraded":"2024-09-10T18:48:36.588611+0000","last_fullsized":"2024-09-10T18:48:36.588611+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:46:31.281519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:31.281519+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019095119999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":85,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.607497+0000","last_change":"2024-09-10T18:48:29.551161+0000","last_active":"2024-09-10T18:48:36.607497+0000","last_peered":"2024-09-10T18:48:36.607497+0000","last_clean":"2024-09-10T18:48:36.607497+0000","last_became_active":"2024-09-10T18:45:17.966833+0000","last_became_peered":"2024-09-10T18:45:17.966833+0000","last_unstale":"2024-09-10T18:48:36.607497+0000","last_undegraded":"2024-09-10T18:48:36.607497+0000","last_fullsized":"2024-09-10T18:48:36.607497+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:29.550946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:29.550946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:50.968295+0000","scrub_duration":0.016562341000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"89'9","reported_seq":84,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:46.116562+0000","last_change":"2024-09-10T18:48:09.648420+0000","last_active":"2024-09-10T18:48:46.116562+0000","last_peered":"2024-09-10T18:48:46.116562+0000","last_clean":"2024-09-10T18:48:46.116562+0000","last_became_active":"2024-09-10T18:47:36.421188+0000","last_became_peered":"2024-09-10T18:47:36.421188+0000","last_unstale":"2024-09-10T18:48:46.116562+0000","last_undegraded":"2024-09-10T18:48:46.116562+0000","last_fullsized":"2024-09-10T18:48:46.116562+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'9","last_scrub_stamp":"2024-09-10T18:48:09.648322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:09.648322+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:30.082986+0000","scrub_duration":0.0066489619999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":88,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.586658+0000","last_change":"2024-09-10T18:48:32.043742+0000","last_active":"2024-09-10T18:48:36.586658+0000","last_peered":"2024-09-10T18:48:36.586658+0000","last_clean":"2024-09-10T18:48:36.586658+0000","last_became_active":"2024-09-10T18:47:36.414625+0000","last_became_peered":"2024-09-10T18:47:36.414625+0000","last_unstale":"2024-09-10T18:48:36.586658+0000","last_undegraded":"2024-09-10T18:48:36.586658+0000","last_fullsized":"2024-09-10T18:48:36.586658+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:32.043666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:32.043666+0000","last_clean_scrub_stamp":"2024-09-10T18:48:32.043666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:49.740069+0000","scrub_duration":0.001572417,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"112'6933","reported_seq":6966,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T18:48:51.015962+0000","last_change":"2024-09-10T18:47:34.399541+0000","last_active":"2024-09-10T18:48:51.015962+0000","last_peered":"2024-09-10T18:48:51.015962+0000","last_clean":"2024-09-10T18:48:51.015962+0000","last_became_active":"2024-09-10T18:47:34.399261+0000","last_became_peered":"2024-09-10T18:47:34.399261+0000","last_unstale":"2024-09-10T18:48:51.015962+0000","last_undegraded":"2024-09-10T18:48:51.015962+0000","last_fullsized":"2024-09-10T18:48:51.015962+0000","mapping_epoch":86,"log_start":"95'1700","ondisk_log_start":"95'1700","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":5233,"log_dups_size":0,"ondisk_log_size":5233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":454361088,"num_objects":6933,"num_object_clones":0,"num_object_copies":20799,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6933,"num_write_kb":443712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"63'15","reported_seq":64,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.599058+0000","last_change":"2024-09-10T18:46:58.122279+0000","last_active":"2024-09-10T18:48:36.599058+0000","last_peered":"2024-09-10T18:48:36.599058+0000","last_clean":"2024-09-10T18:48:36.599058+0000","last_became_active":"2024-09-10T18:46:56.742946+0000","last_became_peered":"2024-09-10T18:46:56.742946+0000","last_unstale":"2024-09-10T18:48:36.599058+0000","last_undegraded":"2024-09-10T18:48:36.599058+0000","last_fullsized":"2024-09-10T18:48:36.599058+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:46:58.122219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:58.122219+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001678094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":85,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.602889+0000","last_change":"2024-09-10T18:48:13.707838+0000","last_active":"2024-09-10T18:48:36.602889+0000","last_peered":"2024-09-10T18:48:36.602889+0000","last_clean":"2024-09-10T18:48:36.602889+0000","last_became_active":"2024-09-10T18:45:17.965409+0000","last_became_peered":"2024-09-10T18:45:17.965409+0000","last_unstale":"2024-09-10T18:48:36.602889+0000","last_undegraded":"2024-09-10T18:48:36.602889+0000","last_fullsized":"2024-09-10T18:48:36.602889+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:13.707725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_clean_scrub_stamp":"2024-09-10T18:48:13.707725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:14.271680+0000","scrub_duration":0.0075610709999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"63'8","reported_seq":130,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:46.116796+0000","last_change":"2024-09-10T18:48:18.950238+0000","last_active":"2024-09-10T18:48:46.116796+0000","last_peered":"2024-09-10T18:48:46.116796+0000","last_clean":"2024-09-10T18:48:46.116796+0000","last_became_active":"2024-09-10T18:45:22.062497+0000","last_became_peered":"2024-09-10T18:45:22.062497+0000","last_unstale":"2024-09-10T18:48:46.116796+0000","last_undegraded":"2024-09-10T18:48:46.116796+0000","last_fullsized":"2024-09-10T18:48:46.116796+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:48:18.950140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:18.950140+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:34.771680+0000","scrub_duration":0.0032883439999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":78,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.598816+0000","last_change":"2024-09-10T18:48:31.170536+0000","last_active":"2024-09-10T18:48:36.598816+0000","last_peered":"2024-09-10T18:48:36.598816+0000","last_clean":"2024-09-10T18:48:36.598816+0000","last_became_active":"2024-09-10T18:45:24.082362+0000","last_became_peered":"2024-09-10T18:45:24.082362+0000","last_unstale":"2024-09-10T18:48:36.598816+0000","last_undegraded":"2024-09-10T18:48:36.598816+0000","last_fullsized":"2024-09-10T18:48:36.598816+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:31.170457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:31.170457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:50.433946+0000","scrub_duration":0.0084072260000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"112'6735","reported_seq":6767,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T18:48:51.024698+0000","last_change":"2024-09-10T18:47:34.400040+0000","last_active":"2024-09-10T18:48:51.024698+0000","last_peered":"2024-09-10T18:48:51.024698+0000","last_clean":"2024-09-10T18:48:51.024698+0000","last_became_active":"2024-09-10T18:47:34.399711+0000","last_became_peered":"2024-09-10T18:47:34.399711+0000","last_unstale":"2024-09-10T18:48:51.024698+0000","last_undegraded":"2024-09-10T18:48:51.024698+0000","last_fullsized":"2024-09-10T18:48:51.024698+0000","mapping_epoch":86,"log_start":"94'1500","ondisk_log_start":"94'1500","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":5235,"log_dups_size":0,"ondisk_log_size":5235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":441384960,"num_objects":6735,"num_object_clones":0,"num_object_copies":20205,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6735,"num_write_kb":431040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"63'11","reported_seq":167,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.586879+0000","last_change":"2024-09-10T18:48:11.900583+0000","last_active":"2024-09-10T18:48:36.586879+0000","last_peered":"2024-09-10T18:48:36.586879+0000","last_clean":"2024-09-10T18:48:36.586879+0000","last_became_active":"2024-09-10T18:47:57.106935+0000","last_became_peered":"2024-09-10T18:47:57.106935+0000","last_unstale":"2024-09-10T18:48:36.586879+0000","last_undegraded":"2024-09-10T18:48:36.586879+0000","last_fullsized":"2024-09-10T18:48:36.586879+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:48:11.900489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:11.900489+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:16.653578+0000","scrub_duration":0.0020768789999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":85,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.597764+0000","last_change":"2024-09-10T18:48:12.642771+0000","last_active":"2024-09-10T18:48:36.597764+0000","last_peered":"2024-09-10T18:48:36.597764+0000","last_clean":"2024-09-10T18:48:36.597764+0000","last_became_active":"2024-09-10T18:45:17.966133+0000","last_became_peered":"2024-09-10T18:45:17.966133+0000","last_unstale":"2024-09-10T18:48:36.597764+0000","last_undegraded":"2024-09-10T18:48:36.597764+0000","last_fullsized":"2024-09-10T18:48:36.597764+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:12.642693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:12.642693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:15.199200+0000","scrub_duration":0.0035654889999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":80,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.599236+0000","last_change":"2024-09-10T18:48:25.093064+0000","last_active":"2024-09-10T18:48:36.599236+0000","last_peered":"2024-09-10T18:48:36.599236+0000","last_clean":"2024-09-10T18:48:36.599236+0000","last_became_active":"2024-09-10T18:45:22.068918+0000","last_became_peered":"2024-09-10T18:45:22.068918+0000","last_unstale":"2024-09-10T18:48:36.599236+0000","last_undegraded":"2024-09-10T18:48:36.599236+0000","last_fullsized":"2024-09-10T18:48:36.599236+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:25.092984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_clean_scrub_stamp":"2024-09-10T18:48:25.092984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:45.861853+0000","scrub_duration":0.0038537620000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":69,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.598998+0000","last_change":"2024-09-10T18:47:00.743740+0000","last_active":"2024-09-10T18:48:36.598998+0000","last_peered":"2024-09-10T18:48:36.598998+0000","last_clean":"2024-09-10T18:48:36.598998+0000","last_became_active":"2024-09-10T18:45:24.264538+0000","last_became_peered":"2024-09-10T18:45:24.264538+0000","last_unstale":"2024-09-10T18:48:36.598998+0000","last_undegraded":"2024-09-10T18:48:36.598998+0000","last_fullsized":"2024-09-10T18:48:36.598998+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:00.743680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:00.743680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022093759999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2","version":"112'6860","reported_seq":6904,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T18:48:51.169885+0000","last_change":"2024-09-10T18:47:36.423507+0000","last_active":"2024-09-10T18:48:51.169885+0000","last_peered":"2024-09-10T18:48:51.169885+0000","last_clean":"2024-09-10T18:48:51.169885+0000","last_became_active":"2024-09-10T18:47:36.423234+0000","last_became_peered":"2024-09-10T18:47:36.423234+0000","last_unstale":"2024-09-10T18:48:51.169885+0000","last_undegraded":"2024-09-10T18:48:51.169885+0000","last_fullsized":"2024-09-10T18:48:51.169885+0000","mapping_epoch":88,"log_start":"98'2600","ondisk_log_start":"98'2600","created":86,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":4260,"log_dups_size":0,"ondisk_log_size":4260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:58.836003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":449576960,"num_objects":6860,"num_object_clones":0,"num_object_copies":20580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6860,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6860,"num_write_kb":439040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"63'14","reported_seq":93,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.586154+0000","last_change":"2024-09-10T18:46:29.330572+0000","last_active":"2024-09-10T18:48:36.586154+0000","last_peered":"2024-09-10T18:48:36.586154+0000","last_clean":"2024-09-10T18:48:36.586154+0000","last_became_active":"2024-09-10T18:45:20.052243+0000","last_became_peered":"2024-09-10T18:45:20.052243+0000","last_unstale":"2024-09-10T18:48:36.586154+0000","last_undegraded":"2024-09-10T18:48:36.586154+0000","last_fullsized":"2024-09-10T18:48:36.586154+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-10T18:46:29.330503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:29.330503+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020437459999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":94,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.596204+0000","last_change":"2024-09-10T18:48:13.631307+0000","last_active":"2024-09-10T18:48:36.596204+0000","last_peered":"2024-09-10T18:48:36.596204+0000","last_clean":"2024-09-10T18:48:36.596204+0000","last_became_active":"2024-09-10T18:47:36.424466+0000","last_became_peered":"2024-09-10T18:47:36.424466+0000","last_unstale":"2024-09-10T18:48:36.596204+0000","last_undegraded":"2024-09-10T18:48:36.596204+0000","last_fullsized":"2024-09-10T18:48:36.596204+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:13.631078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_clean_scrub_stamp":"2024-09-10T18:48:13.631078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:29.790261+0000","scrub_duration":0.0043023660000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":132,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T18:48:47.120112+0000","last_change":"2024-09-10T18:48:23.479877+0000","last_active":"2024-09-10T18:48:47.120112+0000","last_peered":"2024-09-10T18:48:47.120112+0000","last_clean":"2024-09-10T18:48:47.120112+0000","last_became_active":"2024-09-10T18:45:22.074540+0000","last_became_peered":"2024-09-10T18:45:22.074540+0000","last_unstale":"2024-09-10T18:48:47.120112+0000","last_undegraded":"2024-09-10T18:48:47.120112+0000","last_fullsized":"2024-09-10T18:48:47.120112+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:48:23.479762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:23.479762+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:32.897955+0000","scrub_duration":0.01210871,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":89,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.598432+0000","last_change":"2024-09-10T18:48:08.689486+0000","last_active":"2024-09-10T18:48:36.598432+0000","last_peered":"2024-09-10T18:48:36.598432+0000","last_clean":"2024-09-10T18:48:36.598432+0000","last_became_active":"2024-09-10T18:47:36.414665+0000","last_became_peered":"2024-09-10T18:47:36.414665+0000","last_unstale":"2024-09-10T18:48:36.598432+0000","last_undegraded":"2024-09-10T18:48:36.598432+0000","last_fullsized":"2024-09-10T18:48:36.598432+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:08.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:08.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:11.141061+0000","scrub_duration":0.0033094489999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"112'6944","reported_seq":6986,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T18:48:51.120500+0000","last_change":"2024-09-10T18:47:36.410975+0000","last_active":"2024-09-10T18:48:51.120500+0000","last_peered":"2024-09-10T18:48:51.120500+0000","last_clean":"2024-09-10T18:48:51.120500+0000","last_became_active":"2024-09-10T18:47:36.410639+0000","last_became_peered":"2024-09-10T18:47:36.410639+0000","last_unstale":"2024-09-10T18:48:51.120500+0000","last_undegraded":"2024-09-10T18:48:51.120500+0000","last_fullsized":"2024-09-10T18:48:51.120500+0000","mapping_epoch":88,"log_start":"98'2700","ondisk_log_start":"98'2700","created":86,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":4244,"log_dups_size":0,"ondisk_log_size":4244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:59.070249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":455081984,"num_objects":6944,"num_object_clones":0,"num_object_copies":20832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6944,"num_write_kb":444416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"63'18","reported_seq":115,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.595635+0000","last_change":"2024-09-10T18:48:20.514584+0000","last_active":"2024-09-10T18:48:36.595635+0000","last_peered":"2024-09-10T18:48:36.595635+0000","last_clean":"2024-09-10T18:48:36.595635+0000","last_became_active":"2024-09-10T18:45:20.420937+0000","last_became_peered":"2024-09-10T18:45:20.420937+0000","last_unstale":"2024-09-10T18:48:36.595635+0000","last_undegraded":"2024-09-10T18:48:36.595635+0000","last_fullsized":"2024-09-10T18:48:36.595635+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'18","last_scrub_stamp":"2024-09-10T18:48:20.514501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:20.514501+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:32.154497+0000","scrub_duration":0.028109085999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":75,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.602869+0000","last_change":"2024-09-10T18:47:05.505431+0000","last_active":"2024-09-10T18:48:36.602869+0000","last_peered":"2024-09-10T18:48:36.602869+0000","last_clean":"2024-09-10T18:48:36.602869+0000","last_became_active":"2024-09-10T18:45:17.961695+0000","last_became_peered":"2024-09-10T18:45:17.961695+0000","last_unstale":"2024-09-10T18:48:36.602869+0000","last_undegraded":"2024-09-10T18:48:36.602869+0000","last_fullsized":"2024-09-10T18:48:36.602869+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:05.505376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:47:05.505376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013302310000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":80,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.586076+0000","last_change":"2024-09-10T18:48:27.016984+0000","last_active":"2024-09-10T18:48:36.586076+0000","last_peered":"2024-09-10T18:48:36.586076+0000","last_clean":"2024-09-10T18:48:36.586076+0000","last_became_active":"2024-09-10T18:45:22.074012+0000","last_became_peered":"2024-09-10T18:45:22.074012+0000","last_unstale":"2024-09-10T18:48:36.586076+0000","last_undegraded":"2024-09-10T18:48:36.586076+0000","last_fullsized":"2024-09-10T18:48:36.586076+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:27.016933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:27.016933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:28.747433+0000","scrub_duration":0.0059183930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":78,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.586145+0000","last_change":"2024-09-10T18:47:36.411129+0000","last_active":"2024-09-10T18:48:36.586145+0000","last_peered":"2024-09-10T18:48:36.586145+0000","last_clean":"2024-09-10T18:48:36.586145+0000","last_became_active":"2024-09-10T18:47:36.410836+0000","last_became_peered":"2024-09-10T18:47:36.410836+0000","last_unstale":"2024-09-10T18:48:36.586145+0000","last_undegraded":"2024-09-10T18:48:36.586145+0000","last_fullsized":"2024-09-10T18:48:36.586145+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_clean_scrub_stamp":"2024-09-10T18:46:53.087539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001678971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"112'6768","reported_seq":6801,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T18:48:50.997772+0000","last_change":"2024-09-10T18:47:34.399316+0000","last_active":"2024-09-10T18:48:50.997772+0000","last_peered":"2024-09-10T18:48:50.997772+0000","last_clean":"2024-09-10T18:48:50.997772+0000","last_became_active":"2024-09-10T18:47:34.399002+0000","last_became_peered":"2024-09-10T18:47:34.399002+0000","last_unstale":"2024-09-10T18:48:50.997772+0000","last_undegraded":"2024-09-10T18:48:50.997772+0000","last_fullsized":"2024-09-10T18:48:50.997772+0000","mapping_epoch":86,"log_start":"94'1500","ondisk_log_start":"94'1500","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":5268,"log_dups_size":0,"ondisk_log_size":5268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:01.449088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":443547648,"num_objects":6768,"num_object_clones":0,"num_object_copies":20304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6768,"num_write_kb":433152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3","version":"63'19","reported_seq":119,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.588327+0000","last_change":"2024-09-10T18:48:11.781505+0000","last_active":"2024-09-10T18:48:36.588327+0000","last_peered":"2024-09-10T18:48:36.588327+0000","last_clean":"2024-09-10T18:48:36.588327+0000","last_became_active":"2024-09-10T18:45:20.421046+0000","last_became_peered":"2024-09-10T18:45:20.421046+0000","last_unstale":"2024-09-10T18:48:36.588327+0000","last_undegraded":"2024-09-10T18:48:36.588327+0000","last_fullsized":"2024-09-10T18:48:36.588327+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:48:11.781417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:11.781417+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:32.063852+0000","scrub_duration":0.0030109379999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":85,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.600027+0000","last_change":"2024-09-10T18:48:30.694038+0000","last_active":"2024-09-10T18:48:36.600027+0000","last_peered":"2024-09-10T18:48:36.600027+0000","last_clean":"2024-09-10T18:48:36.600027+0000","last_became_active":"2024-09-10T18:45:17.960226+0000","last_became_peered":"2024-09-10T18:45:17.960226+0000","last_unstale":"2024-09-10T18:48:36.600027+0000","last_undegraded":"2024-09-10T18:48:36.600027+0000","last_fullsized":"2024-09-10T18:48:36.600027+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:30.693950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:30.693950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:49.091630+0000","scrub_duration":0.0079194309999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":80,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.595434+0000","last_change":"2024-09-10T18:48:33.513604+0000","last_active":"2024-09-10T18:48:36.595434+0000","last_peered":"2024-09-10T18:48:36.595434+0000","last_clean":"2024-09-10T18:48:36.595434+0000","last_became_active":"2024-09-10T18:45:22.066148+0000","last_became_peered":"2024-09-10T18:45:22.066148+0000","last_unstale":"2024-09-10T18:48:36.595434+0000","last_undegraded":"2024-09-10T18:48:36.595434+0000","last_fullsized":"2024-09-10T18:48:36.595434+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:33.513500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:33.513500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:47.158850+0000","scrub_duration":0.0080948319999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":79,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.602644+0000","last_change":"2024-09-10T18:47:36.422607+0000","last_active":"2024-09-10T18:48:36.602644+0000","last_peered":"2024-09-10T18:48:36.602644+0000","last_clean":"2024-09-10T18:48:36.602644+0000","last_became_active":"2024-09-10T18:47:36.422295+0000","last_became_peered":"2024-09-10T18:47:36.422295+0000","last_unstale":"2024-09-10T18:48:36.602644+0000","last_undegraded":"2024-09-10T18:48:36.602644+0000","last_fullsized":"2024-09-10T18:48:36.602644+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:25.525085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:25.525085+0000","last_clean_scrub_stamp":"2024-09-10T18:47:25.525085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012481079999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"112'6828","reported_seq":6861,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T18:48:51.007300+0000","last_change":"2024-09-10T18:47:34.397576+0000","last_active":"2024-09-10T18:48:51.007300+0000","last_peered":"2024-09-10T18:48:51.007300+0000","last_clean":"2024-09-10T18:48:51.007300+0000","last_became_active":"2024-09-10T18:47:34.396631+0000","last_became_peered":"2024-09-10T18:47:34.396631+0000","last_unstale":"2024-09-10T18:48:51.007300+0000","last_undegraded":"2024-09-10T18:48:51.007300+0000","last_fullsized":"2024-09-10T18:48:51.007300+0000","mapping_epoch":86,"log_start":"95'1700","ondisk_log_start":"95'1700","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":5128,"log_dups_size":0,"ondisk_log_size":5128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":447479808,"num_objects":6828,"num_object_clones":0,"num_object_copies":20484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6828,"num_write_kb":436992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"83'15","reported_seq":103,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.607332+0000","last_change":"2024-09-10T18:48:20.633119+0000","last_active":"2024-09-10T18:48:36.607332+0000","last_peered":"2024-09-10T18:48:36.607332+0000","last_clean":"2024-09-10T18:48:36.607332+0000","last_became_active":"2024-09-10T18:45:20.057204+0000","last_became_peered":"2024-09-10T18:45:20.057204+0000","last_unstale":"2024-09-10T18:48:36.607332+0000","last_undegraded":"2024-09-10T18:48:36.607332+0000","last_fullsized":"2024-09-10T18:48:36.607332+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'15","last_scrub_stamp":"2024-09-10T18:48:20.633054+0000","last_deep_scrub":"83'15","last_deep_scrub_stamp":"2024-09-10T18:46:58.436473+0000","last_clean_scrub_stamp":"2024-09-10T18:48:20.633054+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:37.027685+0000","scrub_duration":0.008267412,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":75,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.604779+0000","last_change":"2024-09-10T18:46:50.443101+0000","last_active":"2024-09-10T18:48:36.604779+0000","last_peered":"2024-09-10T18:48:36.604779+0000","last_clean":"2024-09-10T18:48:36.604779+0000","last_became_active":"2024-09-10T18:45:17.966632+0000","last_became_peered":"2024-09-10T18:45:17.966632+0000","last_unstale":"2024-09-10T18:48:36.604779+0000","last_undegraded":"2024-09-10T18:48:36.604779+0000","last_fullsized":"2024-09-10T18:48:36.604779+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:50.443042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.443042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014088550000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"63'8","reported_seq":122,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T18:48:47.111769+0000","last_change":"2024-09-10T18:47:03.579260+0000","last_active":"2024-09-10T18:48:47.111769+0000","last_peered":"2024-09-10T18:48:47.111769+0000","last_clean":"2024-09-10T18:48:47.111769+0000","last_became_active":"2024-09-10T18:45:22.063326+0000","last_became_peered":"2024-09-10T18:45:22.063326+0000","last_unstale":"2024-09-10T18:48:47.111769+0000","last_undegraded":"2024-09-10T18:48:47.111769+0000","last_fullsized":"2024-09-10T18:48:47.111769+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:47:03.579186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:47:03.579186+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021415639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":79,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.587434+0000","last_change":"2024-09-10T18:47:36.412408+0000","last_active":"2024-09-10T18:48:36.587434+0000","last_peered":"2024-09-10T18:48:36.587434+0000","last_clean":"2024-09-10T18:48:36.587434+0000","last_became_active":"2024-09-10T18:47:36.412111+0000","last_became_peered":"2024-09-10T18:47:36.412111+0000","last_unstale":"2024-09-10T18:48:36.587434+0000","last_undegraded":"2024-09-10T18:48:36.587434+0000","last_fullsized":"2024-09-10T18:48:36.587434+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:13.488525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:13.488525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001590344,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"112'6843","reported_seq":6876,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T18:48:51.034060+0000","last_change":"2024-09-10T18:47:34.400344+0000","last_active":"2024-09-10T18:48:51.034060+0000","last_peered":"2024-09-10T18:48:51.034060+0000","last_clean":"2024-09-10T18:48:51.034060+0000","last_became_active":"2024-09-10T18:47:34.400044+0000","last_became_peered":"2024-09-10T18:47:34.400044+0000","last_unstale":"2024-09-10T18:48:51.034060+0000","last_undegraded":"2024-09-10T18:48:51.034060+0000","last_fullsized":"2024-09-10T18:48:51.034060+0000","mapping_epoch":86,"log_start":"94'1600","ondisk_log_start":"94'1600","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":5243,"log_dups_size":0,"ondisk_log_size":5243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":448462848,"num_objects":6843,"num_object_clones":0,"num_object_copies":20529,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6843,"num_write_kb":437952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"89'29","reported_seq":206,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T18:48:47.133007+0000","last_change":"2024-09-10T18:47:36.443205+0000","last_active":"2024-09-10T18:48:47.133007+0000","last_peered":"2024-09-10T18:48:47.133007+0000","last_clean":"2024-09-10T18:48:47.133007+0000","last_became_active":"2024-09-10T18:47:36.420185+0000","last_became_peered":"2024-09-10T18:47:36.420185+0000","last_unstale":"2024-09-10T18:48:47.133007+0000","last_undegraded":"2024-09-10T18:48:47.133007+0000","last_fullsized":"2024-09-10T18:48:47.133007+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'28","last_scrub_stamp":"2024-09-10T18:47:26.525861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:26.525861+0000","objects_scrubbed":10,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002126719,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":84,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.598717+0000","last_change":"2024-09-10T18:48:33.186208+0000","last_active":"2024-09-10T18:48:36.598717+0000","last_peered":"2024-09-10T18:48:36.598717+0000","last_clean":"2024-09-10T18:48:36.598717+0000","last_became_active":"2024-09-10T18:45:17.964494+0000","last_became_peered":"2024-09-10T18:45:17.964494+0000","last_unstale":"2024-09-10T18:48:36.598717+0000","last_undegraded":"2024-09-10T18:48:36.598717+0000","last_fullsized":"2024-09-10T18:48:36.598717+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:33.186070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:33.186070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:00.114296+0000","scrub_duration":0.0068142150000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":25,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.596020+0000","last_change":"2024-09-10T18:47:57.108424+0000","last_active":"2024-09-10T18:48:36.596020+0000","last_peered":"2024-09-10T18:48:36.596020+0000","last_clean":"2024-09-10T18:48:36.596020+0000","last_became_active":"2024-09-10T18:47:57.107478+0000","last_became_peered":"2024-09-10T18:47:57.107478+0000","last_unstale":"2024-09-10T18:48:36.596020+0000","last_undegraded":"2024-09-10T18:48:36.596020+0000","last_fullsized":"2024-09-10T18:48:36.596020+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:58.523217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:58.523217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001307856,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":69,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.598109+0000","last_change":"2024-09-10T18:47:22.891142+0000","last_active":"2024-09-10T18:48:36.598109+0000","last_peered":"2024-09-10T18:48:36.598109+0000","last_clean":"2024-09-10T18:48:36.598109+0000","last_became_active":"2024-09-10T18:45:24.073307+0000","last_became_peered":"2024-09-10T18:45:24.073307+0000","last_unstale":"2024-09-10T18:48:36.598109+0000","last_undegraded":"2024-09-10T18:48:36.598109+0000","last_fullsized":"2024-09-10T18:48:36.598109+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:22.891084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:22.891084+0000","last_clean_scrub_stamp":"2024-09-10T18:47:22.891084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017788579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"112'6774","reported_seq":6808,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T18:48:51.090832+0000","last_change":"2024-09-10T18:47:34.395543+0000","last_active":"2024-09-10T18:48:51.090832+0000","last_peered":"2024-09-10T18:48:51.090832+0000","last_clean":"2024-09-10T18:48:51.090832+0000","last_became_active":"2024-09-10T18:47:34.395271+0000","last_became_peered":"2024-09-10T18:47:34.395271+0000","last_unstale":"2024-09-10T18:48:51.090832+0000","last_undegraded":"2024-09-10T18:48:51.090832+0000","last_fullsized":"2024-09-10T18:48:51.090832+0000","mapping_epoch":86,"log_start":"96'2000","ondisk_log_start":"96'2000","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":4774,"log_dups_size":0,"ondisk_log_size":4774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":443940864,"num_objects":6774,"num_object_clones":0,"num_object_copies":20322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6774,"num_write_kb":433536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"84'18","reported_seq":105,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.605499+0000","last_change":"2024-09-10T18:47:10.515060+0000","last_active":"2024-09-10T18:48:36.605499+0000","last_peered":"2024-09-10T18:48:36.605499+0000","last_clean":"2024-09-10T18:48:36.605499+0000","last_became_active":"2024-09-10T18:45:20.058435+0000","last_became_peered":"2024-09-10T18:45:20.058435+0000","last_unstale":"2024-09-10T18:48:36.605499+0000","last_undegraded":"2024-09-10T18:48:36.605499+0000","last_fullsized":"2024-09-10T18:48:36.605499+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'18","last_scrub_stamp":"2024-09-10T18:47:10.514951+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-10T18:47:10.514951+0000","last_clean_scrub_stamp":"2024-09-10T18:47:10.514951+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005567575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"84'6","reported_seq":302,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T18:48:49.576211+0000","last_change":"2024-09-10T18:47:36.439529+0000","last_active":"2024-09-10T18:48:49.576211+0000","last_peered":"2024-09-10T18:48:49.576211+0000","last_clean":"2024-09-10T18:48:49.576211+0000","last_became_active":"2024-09-10T18:47:36.418211+0000","last_became_peered":"2024-09-10T18:47:36.418211+0000","last_unstale":"2024-09-10T18:48:49.576211+0000","last_undegraded":"2024-09-10T18:48:49.576211+0000","last_fullsized":"2024-09-10T18:48:49.576211+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'6","last_scrub_stamp":"2024-09-10T18:47:22.545450+0000","last_deep_scrub":"84'6","last_deep_scrub_stamp":"2024-09-10T18:47:22.545450+0000","last_clean_scrub_stamp":"2024-09-10T18:47:22.545450+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022780040000000001,"objects_trimmed":0,"snaptrim_duration":0.0021933159999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":203,"num_read_kb":198,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":36,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.597875+0000","last_change":"2024-09-10T18:47:36.422920+0000","last_active":"2024-09-10T18:48:36.597875+0000","last_peered":"2024-09-10T18:48:36.597875+0000","last_clean":"2024-09-10T18:48:36.597875+0000","last_became_active":"2024-09-10T18:47:36.422641+0000","last_became_peered":"2024-09-10T18:47:36.422641+0000","last_unstale":"2024-09-10T18:48:36.597875+0000","last_undegraded":"2024-09-10T18:48:36.597875+0000","last_fullsized":"2024-09-10T18:48:36.597875+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:46.712269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:46.712269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":45,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.599244+0000","last_change":"2024-09-10T18:48:22.011954+0000","last_active":"2024-09-10T18:48:36.599244+0000","last_peered":"2024-09-10T18:48:36.599244+0000","last_clean":"2024-09-10T18:48:36.599244+0000","last_became_active":"2024-09-10T18:47:36.419081+0000","last_became_peered":"2024-09-10T18:47:36.419081+0000","last_unstale":"2024-09-10T18:48:36.599244+0000","last_undegraded":"2024-09-10T18:48:36.599244+0000","last_fullsized":"2024-09-10T18:48:36.599244+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:22.011870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:22.011870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:41.762812+0000","scrub_duration":0.0018770919999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":69,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.599707+0000","last_change":"2024-09-10T18:46:59.553576+0000","last_active":"2024-09-10T18:48:36.599707+0000","last_peered":"2024-09-10T18:48:36.599707+0000","last_clean":"2024-09-10T18:48:36.599707+0000","last_became_active":"2024-09-10T18:45:24.264617+0000","last_became_peered":"2024-09-10T18:45:24.264617+0000","last_unstale":"2024-09-10T18:48:36.599707+0000","last_undegraded":"2024-09-10T18:48:36.599707+0000","last_fullsized":"2024-09-10T18:48:36.599707+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:59.553505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:59.553505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018297210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"112'6766","reported_seq":6810,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T18:48:51.006817+0000","last_change":"2024-09-10T18:47:36.417383+0000","last_active":"2024-09-10T18:48:51.006817+0000","last_peered":"2024-09-10T18:48:51.006817+0000","last_clean":"2024-09-10T18:48:51.006817+0000","last_became_active":"2024-09-10T18:47:36.417075+0000","last_became_peered":"2024-09-10T18:47:36.417075+0000","last_unstale":"2024-09-10T18:48:51.006817+0000","last_undegraded":"2024-09-10T18:48:51.006817+0000","last_fullsized":"2024-09-10T18:48:51.006817+0000","mapping_epoch":88,"log_start":"97'2200","ondisk_log_start":"97'2200","created":86,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":4566,"log_dups_size":0,"ondisk_log_size":4566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":443416576,"num_objects":6766,"num_object_clones":0,"num_object_copies":20298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6766,"num_write_kb":433024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"63'12","reported_seq":125,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.594675+0000","last_change":"2024-09-10T18:48:28.710224+0000","last_active":"2024-09-10T18:48:36.594675+0000","last_peered":"2024-09-10T18:48:36.594675+0000","last_clean":"2024-09-10T18:48:36.594675+0000","last_became_active":"2024-09-10T18:47:36.417965+0000","last_became_peered":"2024-09-10T18:47:36.417965+0000","last_unstale":"2024-09-10T18:48:36.594675+0000","last_undegraded":"2024-09-10T18:48:36.594675+0000","last_fullsized":"2024-09-10T18:48:36.594675+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:48:28.710113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:28.710113+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:58.705191+0000","scrub_duration":0.0069768260000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"58'1","reported_seq":81,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.589068+0000","last_change":"2024-09-10T18:47:14.490159+0000","last_active":"2024-09-10T18:48:36.589068+0000","last_peered":"2024-09-10T18:48:36.589068+0000","last_clean":"2024-09-10T18:48:36.589068+0000","last_became_active":"2024-09-10T18:45:17.963932+0000","last_became_peered":"2024-09-10T18:45:17.963932+0000","last_unstale":"2024-09-10T18:48:36.589068+0000","last_undegraded":"2024-09-10T18:48:36.589068+0000","last_fullsized":"2024-09-10T18:48:36.589068+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'1","last_scrub_stamp":"2024-09-10T18:47:14.490091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:47:14.490091+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019528939999999999,"objects_trimmed":0,"snaptrim_duration":0.00037812099999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":33,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.597502+0000","last_change":"2024-09-10T18:47:36.417008+0000","last_active":"2024-09-10T18:48:36.597502+0000","last_peered":"2024-09-10T18:48:36.597502+0000","last_clean":"2024-09-10T18:48:36.597502+0000","last_became_active":"2024-09-10T18:47:36.416641+0000","last_became_peered":"2024-09-10T18:47:36.416641+0000","last_unstale":"2024-09-10T18:48:36.597502+0000","last_undegraded":"2024-09-10T18:48:36.597502+0000","last_fullsized":"2024-09-10T18:48:36.597502+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:54.554426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:54.554426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":70,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.599237+0000","last_change":"2024-09-10T18:46:59.132301+0000","last_active":"2024-09-10T18:48:36.599237+0000","last_peered":"2024-09-10T18:48:36.599237+0000","last_clean":"2024-09-10T18:48:36.599237+0000","last_became_active":"2024-09-10T18:45:22.064285+0000","last_became_peered":"2024-09-10T18:45:22.064285+0000","last_unstale":"2024-09-10T18:48:36.599237+0000","last_undegraded":"2024-09-10T18:48:36.599237+0000","last_fullsized":"2024-09-10T18:48:36.599237+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:59.132234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:59.132234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001471034,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":79,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.604304+0000","last_change":"2024-09-10T18:48:21.654220+0000","last_active":"2024-09-10T18:48:36.604304+0000","last_peered":"2024-09-10T18:48:36.604304+0000","last_clean":"2024-09-10T18:48:36.604304+0000","last_became_active":"2024-09-10T18:45:24.083689+0000","last_became_peered":"2024-09-10T18:45:24.083689+0000","last_unstale":"2024-09-10T18:48:36.604304+0000","last_undegraded":"2024-09-10T18:48:36.604304+0000","last_fullsized":"2024-09-10T18:48:36.604304+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:21.654007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:21.654007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:39.425642+0000","scrub_duration":0.0058335929999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"112'6778","reported_seq":6812,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T18:48:51.104269+0000","last_change":"2024-09-10T18:47:34.398452+0000","last_active":"2024-09-10T18:48:51.104269+0000","last_peered":"2024-09-10T18:48:51.104269+0000","last_clean":"2024-09-10T18:48:51.104269+0000","last_became_active":"2024-09-10T18:47:34.396853+0000","last_became_peered":"2024-09-10T18:47:34.396853+0000","last_unstale":"2024-09-10T18:48:51.104269+0000","last_undegraded":"2024-09-10T18:48:51.104269+0000","last_fullsized":"2024-09-10T18:48:51.104269+0000","mapping_epoch":86,"log_start":"96'2000","ondisk_log_start":"96'2000","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":4778,"log_dups_size":0,"ondisk_log_size":4778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:56.235056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":444203008,"num_objects":6778,"num_object_clones":0,"num_object_copies":20334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6778,"num_write_kb":433792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"63'14","reported_seq":104,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.595628+0000","last_change":"2024-09-10T18:48:17.669939+0000","last_active":"2024-09-10T18:48:36.595628+0000","last_peered":"2024-09-10T18:48:36.595628+0000","last_clean":"2024-09-10T18:48:36.595628+0000","last_became_active":"2024-09-10T18:45:20.421119+0000","last_became_peered":"2024-09-10T18:45:20.421119+0000","last_unstale":"2024-09-10T18:48:36.595628+0000","last_undegraded":"2024-09-10T18:48:36.595628+0000","last_fullsized":"2024-09-10T18:48:36.595628+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-10T18:48:17.669787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:17.669787+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:33.721948+0000","scrub_duration":0.0024048300000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"60'2","reported_seq":82,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.596187+0000","last_change":"2024-09-10T18:46:25.845131+0000","last_active":"2024-09-10T18:48:36.596187+0000","last_peered":"2024-09-10T18:48:36.596187+0000","last_clean":"2024-09-10T18:48:36.596187+0000","last_became_active":"2024-09-10T18:45:17.961660+0000","last_became_peered":"2024-09-10T18:45:17.961660+0000","last_unstale":"2024-09-10T18:48:36.596187+0000","last_undegraded":"2024-09-10T18:48:36.596187+0000","last_fullsized":"2024-09-10T18:48:36.596187+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-10T18:46:25.845070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:25.845070+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020129710000000001,"objects_trimmed":0,"snaptrim_duration":0.0013942640000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":94,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.598618+0000","last_change":"2024-09-10T18:48:34.150227+0000","last_active":"2024-09-10T18:48:36.598618+0000","last_peered":"2024-09-10T18:48:36.598618+0000","last_clean":"2024-09-10T18:48:36.598618+0000","last_became_active":"2024-09-10T18:47:36.414636+0000","last_became_peered":"2024-09-10T18:47:36.414636+0000","last_unstale":"2024-09-10T18:48:36.598618+0000","last_undegraded":"2024-09-10T18:48:36.598618+0000","last_fullsized":"2024-09-10T18:48:36.598618+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:34.150146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:34.150146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:02.371194+0000","scrub_duration":0.0018375380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'81","reported_seq":119,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.597920+0000","last_change":"2024-09-10T18:47:05.558031+0000","last_active":"2024-09-10T18:48:36.597920+0000","last_peered":"2024-09-10T18:48:36.597920+0000","last_clean":"2024-09-10T18:48:36.597920+0000","last_became_active":"2024-09-10T18:45:07.433592+0000","last_became_peered":"2024-09-10T18:45:07.433592+0000","last_unstale":"2024-09-10T18:48:36.597920+0000","last_undegraded":"2024-09-10T18:48:36.597920+0000","last_fullsized":"2024-09-10T18:48:36.597920+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:47:05.557968+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:47:05.557968+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015062890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":91,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.599076+0000","last_change":"2024-09-10T18:47:36.418948+0000","last_active":"2024-09-10T18:48:36.599076+0000","last_peered":"2024-09-10T18:48:36.599076+0000","last_clean":"2024-09-10T18:48:36.599076+0000","last_became_active":"2024-09-10T18:47:36.418320+0000","last_became_peered":"2024-09-10T18:47:36.418320+0000","last_unstale":"2024-09-10T18:48:36.599076+0000","last_undegraded":"2024-09-10T18:48:36.599076+0000","last_fullsized":"2024-09-10T18:48:36.599076+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:32.528904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:47:32.528904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037979899999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":68,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.599391+0000","last_change":"2024-09-10T18:46:56.066558+0000","last_active":"2024-09-10T18:48:36.599391+0000","last_peered":"2024-09-10T18:48:36.599391+0000","last_clean":"2024-09-10T18:48:36.599391+0000","last_became_active":"2024-09-10T18:45:24.073327+0000","last_became_peered":"2024-09-10T18:45:24.073327+0000","last_unstale":"2024-09-10T18:48:36.599391+0000","last_undegraded":"2024-09-10T18:48:36.599391+0000","last_fullsized":"2024-09-10T18:48:36.599391+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:56.066502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:56.066502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0011960810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.9","version":"112'6967","reported_seq":7001,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T18:48:51.108325+0000","last_change":"2024-09-10T18:47:34.393877+0000","last_active":"2024-09-10T18:48:51.108325+0000","last_peered":"2024-09-10T18:48:51.108325+0000","last_clean":"2024-09-10T18:48:51.108325+0000","last_became_active":"2024-09-10T18:47:34.393620+0000","last_became_peered":"2024-09-10T18:47:34.393620+0000","last_unstale":"2024-09-10T18:48:51.108325+0000","last_undegraded":"2024-09-10T18:48:51.108325+0000","last_fullsized":"2024-09-10T18:48:51.108325+0000","mapping_epoch":86,"log_start":"97'2200","ondisk_log_start":"97'2200","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":4767,"log_dups_size":0,"ondisk_log_size":4767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":456589312,"num_objects":6967,"num_object_clones":0,"num_object_copies":20901,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6967,"num_write_kb":445888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"63'19","reported_seq":113,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.595811+0000","last_change":"2024-09-10T18:48:14.655578+0000","last_active":"2024-09-10T18:48:36.595811+0000","last_peered":"2024-09-10T18:48:36.595811+0000","last_clean":"2024-09-10T18:48:36.595811+0000","last_became_active":"2024-09-10T18:45:20.421097+0000","last_became_peered":"2024-09-10T18:45:20.421097+0000","last_unstale":"2024-09-10T18:48:36.595811+0000","last_undegraded":"2024-09-10T18:48:36.595811+0000","last_fullsized":"2024-09-10T18:48:36.595811+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:48:14.655527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:14.655527+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:31.443092+0000","scrub_duration":0.00174087,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":35,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.598645+0000","last_change":"2024-09-10T18:47:36.418124+0000","last_active":"2024-09-10T18:48:36.598645+0000","last_peered":"2024-09-10T18:48:36.598645+0000","last_clean":"2024-09-10T18:48:36.598645+0000","last_became_active":"2024-09-10T18:47:36.417800+0000","last_became_peered":"2024-09-10T18:47:36.417800+0000","last_unstale":"2024-09-10T18:48:36.598645+0000","last_undegraded":"2024-09-10T18:48:36.598645+0000","last_fullsized":"2024-09-10T18:48:36.598645+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:53.707694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:53.707694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":46,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.588125+0000","last_change":"2024-09-10T18:48:12.827398+0000","last_active":"2024-09-10T18:48:36.588125+0000","last_peered":"2024-09-10T18:48:36.588125+0000","last_clean":"2024-09-10T18:48:36.588125+0000","last_became_active":"2024-09-10T18:47:36.413618+0000","last_became_peered":"2024-09-10T18:47:36.413618+0000","last_unstale":"2024-09-10T18:48:36.588125+0000","last_undegraded":"2024-09-10T18:48:36.588125+0000","last_fullsized":"2024-09-10T18:48:36.588125+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:12.827317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:12.827317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:17.404178+0000","scrub_duration":0.0036790109999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":79,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.597843+0000","last_change":"2024-09-10T18:47:36.413013+0000","last_active":"2024-09-10T18:48:36.597843+0000","last_peered":"2024-09-10T18:48:36.597843+0000","last_clean":"2024-09-10T18:48:36.597843+0000","last_became_active":"2024-09-10T18:47:36.412610+0000","last_became_peered":"2024-09-10T18:47:36.412610+0000","last_unstale":"2024-09-10T18:48:36.597843+0000","last_undegraded":"2024-09-10T18:48:36.597843+0000","last_fullsized":"2024-09-10T18:48:36.597843+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:03.587252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:03.587252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001684867,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"112'6817","reported_seq":6853,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T18:48:50.992929+0000","last_change":"2024-09-10T18:47:36.413347+0000","last_active":"2024-09-10T18:48:50.992929+0000","last_peered":"2024-09-10T18:48:50.992929+0000","last_clean":"2024-09-10T18:48:50.992929+0000","last_became_active":"2024-09-10T18:47:36.412142+0000","last_became_peered":"2024-09-10T18:47:36.412142+0000","last_unstale":"2024-09-10T18:48:50.992929+0000","last_undegraded":"2024-09-10T18:48:50.992929+0000","last_fullsized":"2024-09-10T18:48:50.992929+0000","mapping_epoch":88,"log_start":"97'2200","ondisk_log_start":"97'2200","created":86,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":4617,"log_dups_size":0,"ondisk_log_size":4617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:59.312878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":446758912,"num_objects":6817,"num_object_clones":0,"num_object_copies":20451,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6817,"num_write_kb":436288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"63'9","reported_seq":99,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.588260+0000","last_change":"2024-09-10T18:48:22.878265+0000","last_active":"2024-09-10T18:48:36.588260+0000","last_peered":"2024-09-10T18:48:36.588260+0000","last_clean":"2024-09-10T18:48:36.588260+0000","last_became_active":"2024-09-10T18:45:20.061021+0000","last_became_peered":"2024-09-10T18:45:20.061021+0000","last_unstale":"2024-09-10T18:48:36.588260+0000","last_undegraded":"2024-09-10T18:48:36.588260+0000","last_fullsized":"2024-09-10T18:48:36.588260+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:48:22.878179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:22.878179+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:49.491061+0000","scrub_duration":0.021357979999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":75,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.603648+0000","last_change":"2024-09-10T18:47:02.496271+0000","last_active":"2024-09-10T18:48:36.603648+0000","last_peered":"2024-09-10T18:48:36.603648+0000","last_clean":"2024-09-10T18:48:36.603648+0000","last_became_active":"2024-09-10T18:45:17.963057+0000","last_became_peered":"2024-09-10T18:45:17.963057+0000","last_unstale":"2024-09-10T18:48:36.603648+0000","last_undegraded":"2024-09-10T18:48:36.603648+0000","last_fullsized":"2024-09-10T18:48:36.603648+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:02.496142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:47:02.496142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001628399,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":35,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.586073+0000","last_change":"2024-09-10T18:47:36.411830+0000","last_active":"2024-09-10T18:48:36.586073+0000","last_peered":"2024-09-10T18:48:36.586073+0000","last_clean":"2024-09-10T18:48:36.586073+0000","last_became_active":"2024-09-10T18:47:36.411517+0000","last_became_peered":"2024-09-10T18:47:36.411517+0000","last_unstale":"2024-09-10T18:48:36.586073+0000","last_undegraded":"2024-09-10T18:48:36.586073+0000","last_fullsized":"2024-09-10T18:48:36.586073+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_clean_scrub_stamp":"2024-09-10T18:46:55.684250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":89,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.588992+0000","last_change":"2024-09-10T18:48:25.804464+0000","last_active":"2024-09-10T18:48:36.588992+0000","last_peered":"2024-09-10T18:48:36.588992+0000","last_clean":"2024-09-10T18:48:36.588992+0000","last_became_active":"2024-09-10T18:47:36.412933+0000","last_became_peered":"2024-09-10T18:47:36.412933+0000","last_unstale":"2024-09-10T18:48:36.588992+0000","last_undegraded":"2024-09-10T18:48:36.588992+0000","last_fullsized":"2024-09-10T18:48:36.588992+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:25.804376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:47.545343+0000","last_clean_scrub_stamp":"2024-09-10T18:48:25.804376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:26.796128+0000","scrub_duration":0.0098886110000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.b","version":"112'6872","reported_seq":6906,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T18:48:51.109881+0000","last_change":"2024-09-10T18:47:34.398004+0000","last_active":"2024-09-10T18:48:51.109881+0000","last_peered":"2024-09-10T18:48:51.109881+0000","last_clean":"2024-09-10T18:48:51.109881+0000","last_became_active":"2024-09-10T18:47:34.397708+0000","last_became_peered":"2024-09-10T18:47:34.397708+0000","last_unstale":"2024-09-10T18:48:51.109881+0000","last_undegraded":"2024-09-10T18:48:51.109881+0000","last_fullsized":"2024-09-10T18:48:51.109881+0000","mapping_epoch":86,"log_start":"98'2500","ondisk_log_start":"98'2500","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":4372,"log_dups_size":0,"ondisk_log_size":4372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":450363392,"num_objects":6872,"num_object_clones":0,"num_object_copies":20616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6872,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6872,"num_write_kb":439808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"63'15","reported_seq":97,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.598011+0000","last_change":"2024-09-10T18:46:51.114543+0000","last_active":"2024-09-10T18:48:36.598011+0000","last_peered":"2024-09-10T18:48:36.598011+0000","last_clean":"2024-09-10T18:48:36.598011+0000","last_became_active":"2024-09-10T18:45:20.419804+0000","last_became_peered":"2024-09-10T18:45:20.419804+0000","last_unstale":"2024-09-10T18:48:36.598011+0000","last_undegraded":"2024-09-10T18:48:36.598011+0000","last_fullsized":"2024-09-10T18:48:36.598011+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:46:51.114455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:51.114455+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022557979999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":36,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.598238+0000","last_change":"2024-09-10T18:47:36.420713+0000","last_active":"2024-09-10T18:48:36.598238+0000","last_peered":"2024-09-10T18:48:36.598238+0000","last_clean":"2024-09-10T18:48:36.598238+0000","last_became_active":"2024-09-10T18:47:36.420475+0000","last_became_peered":"2024-09-10T18:47:36.420475+0000","last_unstale":"2024-09-10T18:48:36.598238+0000","last_undegraded":"2024-09-10T18:48:36.598238+0000","last_fullsized":"2024-09-10T18:48:36.598238+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:20.774289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:17.819909+0000","last_clean_scrub_stamp":"2024-09-10T18:47:20.774289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"83'9","reported_seq":135,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:46.116574+0000","last_change":"2024-09-10T18:48:33.683570+0000","last_active":"2024-09-10T18:48:46.116574+0000","last_peered":"2024-09-10T18:48:46.116574+0000","last_clean":"2024-09-10T18:48:46.116574+0000","last_became_active":"2024-09-10T18:45:22.073383+0000","last_became_peered":"2024-09-10T18:45:22.073383+0000","last_unstale":"2024-09-10T18:48:46.116574+0000","last_undegraded":"2024-09-10T18:48:46.116574+0000","last_fullsized":"2024-09-10T18:48:46.116574+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'9","last_scrub_stamp":"2024-09-10T18:48:33.683477+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-09-10T18:47:02.582229+0000","last_clean_scrub_stamp":"2024-09-10T18:48:33.683477+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:57.226708+0000","scrub_duration":0.0026989119999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":68,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.598222+0000","last_change":"2024-09-10T18:46:45.083522+0000","last_active":"2024-09-10T18:48:36.598222+0000","last_peered":"2024-09-10T18:48:36.598222+0000","last_clean":"2024-09-10T18:48:36.598222+0000","last_became_active":"2024-09-10T18:45:24.265256+0000","last_became_peered":"2024-09-10T18:45:24.265256+0000","last_unstale":"2024-09-10T18:48:36.598222+0000","last_undegraded":"2024-09-10T18:48:36.598222+0000","last_fullsized":"2024-09-10T18:48:36.598222+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_clean_scrub_stamp":"2024-09-10T18:46:45.083425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017589890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.a","version":"112'6976","reported_seq":7020,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T18:48:51.109129+0000","last_change":"2024-09-10T18:47:36.423957+0000","last_active":"2024-09-10T18:48:51.109129+0000","last_peered":"2024-09-10T18:48:51.109129+0000","last_clean":"2024-09-10T18:48:51.109129+0000","last_became_active":"2024-09-10T18:47:36.422868+0000","last_became_peered":"2024-09-10T18:47:36.422868+0000","last_unstale":"2024-09-10T18:48:51.109129+0000","last_undegraded":"2024-09-10T18:48:51.109129+0000","last_fullsized":"2024-09-10T18:48:51.109129+0000","mapping_epoch":88,"log_start":"98'2700","ondisk_log_start":"98'2700","created":86,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":4276,"log_dups_size":0,"ondisk_log_size":4276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":457179136,"num_objects":6976,"num_object_clones":0,"num_object_copies":20928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6976,"num_write_kb":446464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"63'12","reported_seq":105,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.586149+0000","last_change":"2024-09-10T18:47:35.928110+0000","last_active":"2024-09-10T18:48:36.586149+0000","last_peered":"2024-09-10T18:48:36.586149+0000","last_clean":"2024-09-10T18:48:36.586149+0000","last_became_active":"2024-09-10T18:45:20.056869+0000","last_became_peered":"2024-09-10T18:45:20.056869+0000","last_unstale":"2024-09-10T18:48:36.586149+0000","last_undegraded":"2024-09-10T18:48:36.586149+0000","last_fullsized":"2024-09-10T18:48:36.586149+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:47:35.928040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:35.928040+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:50.601492+0000","scrub_duration":0.0021332539999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"56'2","reported_seq":91,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.587014+0000","last_change":"2024-09-10T18:46:42.303648+0000","last_active":"2024-09-10T18:48:36.587014+0000","last_peered":"2024-09-10T18:48:36.587014+0000","last_clean":"2024-09-10T18:48:36.587014+0000","last_became_active":"2024-09-10T18:45:17.963286+0000","last_became_peered":"2024-09-10T18:45:17.963286+0000","last_unstale":"2024-09-10T18:48:36.587014+0000","last_undegraded":"2024-09-10T18:48:36.587014+0000","last_fullsized":"2024-09-10T18:48:36.587014+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-10T18:46:42.303567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:42.303567+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020684750000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":46,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.587871+0000","last_change":"2024-09-10T18:48:15.763428+0000","last_active":"2024-09-10T18:48:36.587871+0000","last_peered":"2024-09-10T18:48:36.587871+0000","last_clean":"2024-09-10T18:48:36.587871+0000","last_became_active":"2024-09-10T18:47:36.421151+0000","last_became_peered":"2024-09-10T18:47:36.421151+0000","last_unstale":"2024-09-10T18:48:36.587871+0000","last_undegraded":"2024-09-10T18:48:36.587871+0000","last_fullsized":"2024-09-10T18:48:36.587871+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:15.763348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:15.763348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:38.665471+0000","scrub_duration":0.0046035929999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":69,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.599499+0000","last_change":"2024-09-10T18:47:18.797303+0000","last_active":"2024-09-10T18:48:36.599499+0000","last_peered":"2024-09-10T18:48:36.599499+0000","last_clean":"2024-09-10T18:48:36.599499+0000","last_became_active":"2024-09-10T18:45:24.077815+0000","last_became_peered":"2024-09-10T18:45:24.077815+0000","last_unstale":"2024-09-10T18:48:36.599499+0000","last_undegraded":"2024-09-10T18:48:36.599499+0000","last_fullsized":"2024-09-10T18:48:36.599499+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:18.797245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:18.797245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015800090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"76'13","reported_seq":103,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.594597+0000","last_change":"2024-09-10T18:48:10.773290+0000","last_active":"2024-09-10T18:48:36.594597+0000","last_peered":"2024-09-10T18:48:36.594597+0000","last_clean":"2024-09-10T18:48:36.594597+0000","last_became_active":"2024-09-10T18:45:20.420421+0000","last_became_peered":"2024-09-10T18:45:20.420421+0000","last_unstale":"2024-09-10T18:48:36.594597+0000","last_undegraded":"2024-09-10T18:48:36.594597+0000","last_fullsized":"2024-09-10T18:48:36.594597+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'13","last_scrub_stamp":"2024-09-10T18:48:10.773207+0000","last_deep_scrub":"76'13","last_deep_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_clean_scrub_stamp":"2024-09-10T18:48:10.773207+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:35.455517+0000","scrub_duration":0.008348678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":46,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.603862+0000","last_change":"2024-09-10T18:48:14.699173+0000","last_active":"2024-09-10T18:48:36.603862+0000","last_peered":"2024-09-10T18:48:36.603862+0000","last_clean":"2024-09-10T18:48:36.603862+0000","last_became_active":"2024-09-10T18:47:36.409299+0000","last_became_peered":"2024-09-10T18:47:36.409299+0000","last_unstale":"2024-09-10T18:48:36.603862+0000","last_undegraded":"2024-09-10T18:48:36.603862+0000","last_fullsized":"2024-09-10T18:48:36.603862+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:14.695946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:14.695946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:15.444972+0000","scrub_duration":0.007154171,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":81,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.605344+0000","last_change":"2024-09-10T18:48:28.442445+0000","last_active":"2024-09-10T18:48:36.605344+0000","last_peered":"2024-09-10T18:48:36.605344+0000","last_clean":"2024-09-10T18:48:36.605344+0000","last_became_active":"2024-09-10T18:45:22.069040+0000","last_became_peered":"2024-09-10T18:45:22.069040+0000","last_unstale":"2024-09-10T18:48:36.605344+0000","last_undegraded":"2024-09-10T18:48:36.605344+0000","last_fullsized":"2024-09-10T18:48:36.605344+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:28.442370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:28.442370+0000","last_clean_scrub_stamp":"2024-09-10T18:48:28.442370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:50.109389+0000","scrub_duration":0.0098412350000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"63'1","reported_seq":48,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.588430+0000","last_change":"2024-09-10T18:47:36.423902+0000","last_active":"2024-09-10T18:48:36.588430+0000","last_peered":"2024-09-10T18:48:36.588430+0000","last_clean":"2024-09-10T18:48:36.588430+0000","last_became_active":"2024-09-10T18:47:36.406936+0000","last_became_peered":"2024-09-10T18:47:36.406936+0000","last_unstale":"2024-09-10T18:48:36.588430+0000","last_undegraded":"2024-09-10T18:48:36.588430+0000","last_fullsized":"2024-09-10T18:48:36.588430+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-10T18:46:56.721716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:56.721716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":84,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.596262+0000","last_change":"2024-09-10T18:48:24.571499+0000","last_active":"2024-09-10T18:48:36.596262+0000","last_peered":"2024-09-10T18:48:36.596262+0000","last_clean":"2024-09-10T18:48:36.596262+0000","last_became_active":"2024-09-10T18:45:17.964705+0000","last_became_peered":"2024-09-10T18:45:17.964705+0000","last_unstale":"2024-09-10T18:48:36.596262+0000","last_undegraded":"2024-09-10T18:48:36.596262+0000","last_fullsized":"2024-09-10T18:48:36.596262+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:24.571419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:24.571419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:51.133227+0000","scrub_duration":0.005895399,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'6","reported_seq":82,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.605409+0000","last_change":"2024-09-10T18:47:02.700853+0000","last_active":"2024-09-10T18:48:36.605409+0000","last_peered":"2024-09-10T18:48:36.605409+0000","last_clean":"2024-09-10T18:48:36.605409+0000","last_became_active":"2024-09-10T18:45:20.421746+0000","last_became_peered":"2024-09-10T18:45:20.421746+0000","last_unstale":"2024-09-10T18:48:36.605409+0000","last_undegraded":"2024-09-10T18:48:36.605409+0000","last_fullsized":"2024-09-10T18:48:36.605409+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'6","last_scrub_stamp":"2024-09-10T18:47:02.700787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:02.700787+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017032519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":70,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.599337+0000","last_change":"2024-09-10T18:46:39.084489+0000","last_active":"2024-09-10T18:48:36.599337+0000","last_peered":"2024-09-10T18:48:36.599337+0000","last_clean":"2024-09-10T18:48:36.599337+0000","last_became_active":"2024-09-10T18:45:22.069653+0000","last_became_peered":"2024-09-10T18:45:22.069653+0000","last_unstale":"2024-09-10T18:48:36.599337+0000","last_undegraded":"2024-09-10T18:48:36.599337+0000","last_fullsized":"2024-09-10T18:48:36.599337+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:39.084433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:39.084433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014845819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":79,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.599643+0000","last_change":"2024-09-10T18:48:32.654862+0000","last_active":"2024-09-10T18:48:36.599643+0000","last_peered":"2024-09-10T18:48:36.599643+0000","last_clean":"2024-09-10T18:48:36.599643+0000","last_became_active":"2024-09-10T18:45:24.264611+0000","last_became_peered":"2024-09-10T18:45:24.264611+0000","last_unstale":"2024-09-10T18:48:36.599643+0000","last_undegraded":"2024-09-10T18:48:36.599643+0000","last_fullsized":"2024-09-10T18:48:36.599643+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:32.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:32.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:01.743394+0000","scrub_duration":0.0049870009999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"63'10","reported_seq":132,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.599505+0000","last_change":"2024-09-10T18:47:36.442817+0000","last_active":"2024-09-10T18:48:36.599505+0000","last_peered":"2024-09-10T18:48:36.599505+0000","last_clean":"2024-09-10T18:48:36.599505+0000","last_became_active":"2024-09-10T18:47:36.426260+0000","last_became_peered":"2024-09-10T18:47:36.426260+0000","last_unstale":"2024-09-10T18:48:36.599505+0000","last_undegraded":"2024-09-10T18:48:36.599505+0000","last_fullsized":"2024-09-10T18:48:36.599505+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:46:50.722316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.722316+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022194070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":74,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.598581+0000","last_change":"2024-09-10T18:46:50.093435+0000","last_active":"2024-09-10T18:48:36.598581+0000","last_peered":"2024-09-10T18:48:36.598581+0000","last_clean":"2024-09-10T18:48:36.598581+0000","last_became_active":"2024-09-10T18:45:17.964870+0000","last_became_peered":"2024-09-10T18:45:17.964870+0000","last_unstale":"2024-09-10T18:48:36.598581+0000","last_undegraded":"2024-09-10T18:48:36.598581+0000","last_fullsized":"2024-09-10T18:48:36.598581+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:50.093372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:50.093372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015580539999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"63'8","reported_seq":120,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:46.116535+0000","last_change":"2024-09-10T18:46:34.085224+0000","last_active":"2024-09-10T18:48:46.116535+0000","last_peered":"2024-09-10T18:48:46.116535+0000","last_clean":"2024-09-10T18:48:46.116535+0000","last_became_active":"2024-09-10T18:45:22.062940+0000","last_became_peered":"2024-09-10T18:45:22.062940+0000","last_unstale":"2024-09-10T18:48:46.116535+0000","last_undegraded":"2024-09-10T18:48:46.116535+0000","last_fullsized":"2024-09-10T18:48:46.116535+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:46:34.085156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:34.085156+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020843120000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":69,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.591701+0000","last_change":"2024-09-10T18:47:21.538033+0000","last_active":"2024-09-10T18:48:36.591701+0000","last_peered":"2024-09-10T18:48:36.591701+0000","last_clean":"2024-09-10T18:48:36.591701+0000","last_became_active":"2024-09-10T18:45:24.073736+0000","last_became_peered":"2024-09-10T18:45:24.073736+0000","last_unstale":"2024-09-10T18:48:36.591701+0000","last_undegraded":"2024-09-10T18:48:36.591701+0000","last_fullsized":"2024-09-10T18:48:36.591701+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:21.537964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:21.537964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001500305,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"63'9","reported_seq":98,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.599044+0000","last_change":"2024-09-10T18:48:19.076178+0000","last_active":"2024-09-10T18:48:36.599044+0000","last_peered":"2024-09-10T18:48:36.599044+0000","last_clean":"2024-09-10T18:48:36.599044+0000","last_became_active":"2024-09-10T18:45:20.421124+0000","last_became_peered":"2024-09-10T18:45:20.421124+0000","last_unstale":"2024-09-10T18:48:36.599044+0000","last_undegraded":"2024-09-10T18:48:36.599044+0000","last_fullsized":"2024-09-10T18:48:36.599044+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:48:19.076099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:19.076099+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:32.177242+0000","scrub_duration":0.002691981,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":85,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.588429+0000","last_change":"2024-09-10T18:48:33.663516+0000","last_active":"2024-09-10T18:48:36.588429+0000","last_peered":"2024-09-10T18:48:36.588429+0000","last_clean":"2024-09-10T18:48:36.588429+0000","last_became_active":"2024-09-10T18:45:17.960106+0000","last_became_peered":"2024-09-10T18:45:17.960106+0000","last_unstale":"2024-09-10T18:48:36.588429+0000","last_undegraded":"2024-09-10T18:48:36.588429+0000","last_fullsized":"2024-09-10T18:48:36.588429+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:33.663415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:33.663415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:33.950247+0000","scrub_duration":0.002167352,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"89'10","reported_seq":137,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T18:48:47.122843+0000","last_change":"2024-09-10T18:47:36.434102+0000","last_active":"2024-09-10T18:48:47.122843+0000","last_peered":"2024-09-10T18:48:47.122843+0000","last_clean":"2024-09-10T18:48:47.122843+0000","last_became_active":"2024-09-10T18:47:36.424575+0000","last_became_peered":"2024-09-10T18:47:36.424575+0000","last_unstale":"2024-09-10T18:48:47.122843+0000","last_undegraded":"2024-09-10T18:48:47.122843+0000","last_fullsized":"2024-09-10T18:48:47.122843+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"81'9","last_scrub_stamp":"2024-09-10T18:46:54.800790+0000","last_deep_scrub":"81'9","last_deep_scrub_stamp":"2024-09-10T18:46:54.800790+0000","last_clean_scrub_stamp":"2024-09-10T18:46:54.800790+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034208709999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":88,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.586747+0000","last_change":"2024-09-10T18:48:34.045421+0000","last_active":"2024-09-10T18:48:36.586747+0000","last_peered":"2024-09-10T18:48:36.586747+0000","last_clean":"2024-09-10T18:48:36.586747+0000","last_became_active":"2024-09-10T18:47:36.414047+0000","last_became_peered":"2024-09-10T18:47:36.414047+0000","last_unstale":"2024-09-10T18:48:36.586747+0000","last_undegraded":"2024-09-10T18:48:36.586747+0000","last_fullsized":"2024-09-10T18:48:36.586747+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:34.045311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_clean_scrub_stamp":"2024-09-10T18:48:34.045311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:49.035924+0000","scrub_duration":0.0048840200000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"63'9","reported_seq":109,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.607260+0000","last_change":"2024-09-10T18:47:36.437948+0000","last_active":"2024-09-10T18:48:36.607260+0000","last_peered":"2024-09-10T18:48:36.607260+0000","last_clean":"2024-09-10T18:48:36.607260+0000","last_became_active":"2024-09-10T18:47:36.416484+0000","last_became_peered":"2024-09-10T18:47:36.416484+0000","last_unstale":"2024-09-10T18:48:36.607260+0000","last_undegraded":"2024-09-10T18:48:36.607260+0000","last_fullsized":"2024-09-10T18:48:36.607260+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:46:59.469198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:59.469198+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022009489999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":84,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.595463+0000","last_change":"2024-09-10T18:47:29.023501+0000","last_active":"2024-09-10T18:48:36.595463+0000","last_peered":"2024-09-10T18:48:36.595463+0000","last_clean":"2024-09-10T18:48:36.595463+0000","last_became_active":"2024-09-10T18:45:17.962009+0000","last_became_peered":"2024-09-10T18:45:17.962009+0000","last_unstale":"2024-09-10T18:48:36.595463+0000","last_undegraded":"2024-09-10T18:48:36.595463+0000","last_fullsized":"2024-09-10T18:48:36.595463+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:29.023450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:47:29.023450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001196805,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":71,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.599577+0000","last_change":"2024-09-10T18:47:14.869132+0000","last_active":"2024-09-10T18:48:36.599577+0000","last_peered":"2024-09-10T18:48:36.599577+0000","last_clean":"2024-09-10T18:48:36.599577+0000","last_became_active":"2024-09-10T18:45:22.072877+0000","last_became_peered":"2024-09-10T18:45:22.072877+0000","last_unstale":"2024-09-10T18:48:36.599577+0000","last_undegraded":"2024-09-10T18:48:36.599577+0000","last_fullsized":"2024-09-10T18:48:36.599577+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:14.869076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:47:14.869076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00131624,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":79,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.587346+0000","last_change":"2024-09-10T18:48:32.647084+0000","last_active":"2024-09-10T18:48:36.587346+0000","last_peered":"2024-09-10T18:48:36.587346+0000","last_clean":"2024-09-10T18:48:36.587346+0000","last_became_active":"2024-09-10T18:45:24.081050+0000","last_became_peered":"2024-09-10T18:45:24.081050+0000","last_unstale":"2024-09-10T18:48:36.587346+0000","last_undegraded":"2024-09-10T18:48:36.587346+0000","last_fullsized":"2024-09-10T18:48:36.587346+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:32.646988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:32.646988+0000","last_clean_scrub_stamp":"2024-09-10T18:48:32.646988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:39.006371+0000","scrub_duration":0.0038989210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"63'11","reported_seq":91,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.586262+0000","last_change":"2024-09-10T18:47:14.034427+0000","last_active":"2024-09-10T18:48:36.586262+0000","last_peered":"2024-09-10T18:48:36.586262+0000","last_clean":"2024-09-10T18:48:36.586262+0000","last_became_active":"2024-09-10T18:45:20.056885+0000","last_became_peered":"2024-09-10T18:45:20.056885+0000","last_unstale":"2024-09-10T18:48:36.586262+0000","last_undegraded":"2024-09-10T18:48:36.586262+0000","last_fullsized":"2024-09-10T18:48:36.586262+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:47:14.034361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:14.034361+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018564,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":85,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.603524+0000","last_change":"2024-09-10T18:48:32.517573+0000","last_active":"2024-09-10T18:48:36.603524+0000","last_peered":"2024-09-10T18:48:36.603524+0000","last_clean":"2024-09-10T18:48:36.603524+0000","last_became_active":"2024-09-10T18:45:17.963542+0000","last_became_peered":"2024-09-10T18:45:17.963542+0000","last_unstale":"2024-09-10T18:48:36.603524+0000","last_undegraded":"2024-09-10T18:48:36.603524+0000","last_fullsized":"2024-09-10T18:48:36.603524+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:32.517513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:32.517513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:48.139582+0000","scrub_duration":0.013400413999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":81,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.607022+0000","last_change":"2024-09-10T18:48:09.725726+0000","last_active":"2024-09-10T18:48:36.607022+0000","last_peered":"2024-09-10T18:48:36.607022+0000","last_clean":"2024-09-10T18:48:36.607022+0000","last_became_active":"2024-09-10T18:45:22.070065+0000","last_became_peered":"2024-09-10T18:45:22.070065+0000","last_unstale":"2024-09-10T18:48:36.607022+0000","last_undegraded":"2024-09-10T18:48:36.607022+0000","last_fullsized":"2024-09-10T18:48:36.607022+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:09.725488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:09.725488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:18.246207+0000","scrub_duration":0.0027892020000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":69,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.597211+0000","last_change":"2024-09-10T18:47:05.789893+0000","last_active":"2024-09-10T18:48:36.597211+0000","last_peered":"2024-09-10T18:48:36.597211+0000","last_clean":"2024-09-10T18:48:36.597211+0000","last_became_active":"2024-09-10T18:45:24.072224+0000","last_became_peered":"2024-09-10T18:45:24.072224+0000","last_unstale":"2024-09-10T18:48:36.597211+0000","last_undegraded":"2024-09-10T18:48:36.597211+0000","last_fullsized":"2024-09-10T18:48:36.597211+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:05.789830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:05.789830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001271938,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"63'4","reported_seq":89,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.595559+0000","last_change":"2024-09-10T18:48:30.455636+0000","last_active":"2024-09-10T18:48:36.595559+0000","last_peered":"2024-09-10T18:48:36.595559+0000","last_clean":"2024-09-10T18:48:36.595559+0000","last_became_active":"2024-09-10T18:45:20.059205+0000","last_became_peered":"2024-09-10T18:45:20.059205+0000","last_unstale":"2024-09-10T18:48:36.595559+0000","last_undegraded":"2024-09-10T18:48:36.595559+0000","last_fullsized":"2024-09-10T18:48:36.595559+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4","last_scrub_stamp":"2024-09-10T18:48:30.455433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:30.455433+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:32.585527+0000","scrub_duration":0.0078514339999999991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":84,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.599410+0000","last_change":"2024-09-10T18:47:36.416170+0000","last_active":"2024-09-10T18:48:36.599410+0000","last_peered":"2024-09-10T18:48:36.599410+0000","last_clean":"2024-09-10T18:48:36.599410+0000","last_became_active":"2024-09-10T18:47:36.415866+0000","last_became_peered":"2024-09-10T18:47:36.415866+0000","last_unstale":"2024-09-10T18:48:36.599410+0000","last_undegraded":"2024-09-10T18:48:36.599410+0000","last_fullsized":"2024-09-10T18:48:36.599410+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:54.082761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:54.082761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001210458,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":80,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.595163+0000","last_change":"2024-09-10T18:48:27.535769+0000","last_active":"2024-09-10T18:48:36.595163+0000","last_peered":"2024-09-10T18:48:36.595163+0000","last_clean":"2024-09-10T18:48:36.595163+0000","last_became_active":"2024-09-10T18:45:22.072377+0000","last_became_peered":"2024-09-10T18:45:22.072377+0000","last_unstale":"2024-09-10T18:48:36.595163+0000","last_undegraded":"2024-09-10T18:48:36.595163+0000","last_fullsized":"2024-09-10T18:48:36.595163+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:27.535676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_clean_scrub_stamp":"2024-09-10T18:48:27.535676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:36.593031+0000","scrub_duration":0.002291132,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":89,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.597631+0000","last_change":"2024-09-10T18:48:28.711019+0000","last_active":"2024-09-10T18:48:36.597631+0000","last_peered":"2024-09-10T18:48:36.597631+0000","last_clean":"2024-09-10T18:48:36.597631+0000","last_became_active":"2024-09-10T18:47:36.416187+0000","last_became_peered":"2024-09-10T18:47:36.416187+0000","last_unstale":"2024-09-10T18:48:36.597631+0000","last_undegraded":"2024-09-10T18:48:36.597631+0000","last_fullsized":"2024-09-10T18:48:36.597631+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:28.710918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:28.710918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:37.560639+0000","scrub_duration":0.0068877160000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"79'16","reported_seq":97,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.597545+0000","last_change":"2024-09-10T18:46:48.744340+0000","last_active":"2024-09-10T18:48:36.597545+0000","last_peered":"2024-09-10T18:48:36.597545+0000","last_clean":"2024-09-10T18:48:36.597545+0000","last_became_active":"2024-09-10T18:45:20.421479+0000","last_became_peered":"2024-09-10T18:45:20.421479+0000","last_unstale":"2024-09-10T18:48:36.597545+0000","last_undegraded":"2024-09-10T18:48:36.597545+0000","last_fullsized":"2024-09-10T18:48:36.597545+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"79'16","last_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_deep_scrub":"79'16","last_deep_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_clean_scrub_stamp":"2024-09-10T18:46:48.744287+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044879259999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"56'1","reported_seq":105,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.596161+0000","last_change":"2024-09-10T18:48:20.603918+0000","last_active":"2024-09-10T18:48:36.596161+0000","last_peered":"2024-09-10T18:48:36.596161+0000","last_clean":"2024-09-10T18:48:36.596161+0000","last_became_active":"2024-09-10T18:47:36.421985+0000","last_became_peered":"2024-09-10T18:47:36.421985+0000","last_unstale":"2024-09-10T18:48:36.596161+0000","last_undegraded":"2024-09-10T18:48:36.596161+0000","last_fullsized":"2024-09-10T18:48:36.596161+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:48:20.603805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:20.603805+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:49.866219+0000","scrub_duration":0.0039707509999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":35,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.595280+0000","last_change":"2024-09-10T18:48:09.323873+0000","last_active":"2024-09-10T18:48:36.595280+0000","last_peered":"2024-09-10T18:48:36.595280+0000","last_clean":"2024-09-10T18:48:36.595280+0000","last_became_active":"2024-09-10T18:47:57.107149+0000","last_became_peered":"2024-09-10T18:47:57.107149+0000","last_unstale":"2024-09-10T18:48:36.595280+0000","last_undegraded":"2024-09-10T18:48:36.595280+0000","last_fullsized":"2024-09-10T18:48:36.595280+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:09.323794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:09.323794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:28.508379+0000","scrub_duration":0.0056521669999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":79,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.599413+0000","last_change":"2024-09-10T18:48:16.410239+0000","last_active":"2024-09-10T18:48:36.599413+0000","last_peered":"2024-09-10T18:48:36.599413+0000","last_clean":"2024-09-10T18:48:36.599413+0000","last_became_active":"2024-09-10T18:45:24.264818+0000","last_became_peered":"2024-09-10T18:45:24.264818+0000","last_unstale":"2024-09-10T18:48:36.599413+0000","last_undegraded":"2024-09-10T18:48:36.599413+0000","last_fullsized":"2024-09-10T18:48:36.599413+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:16.410071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:16.410071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:22.248050+0000","scrub_duration":0.0061840250000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"78'2","reported_seq":95,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.598962+0000","last_change":"2024-09-10T18:48:10.610714+0000","last_active":"2024-09-10T18:48:36.598962+0000","last_peered":"2024-09-10T18:48:36.598962+0000","last_clean":"2024-09-10T18:48:36.598962+0000","last_became_active":"2024-09-10T18:47:36.418702+0000","last_became_peered":"2024-09-10T18:47:36.418702+0000","last_unstale":"2024-09-10T18:48:36.598962+0000","last_undegraded":"2024-09-10T18:48:36.598962+0000","last_fullsized":"2024-09-10T18:48:36.598962+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'2","last_scrub_stamp":"2024-09-10T18:48:10.610652+0000","last_deep_scrub":"78'2","last_deep_scrub_stamp":"2024-09-10T18:46:37.581130+0000","last_clean_scrub_stamp":"2024-09-10T18:48:10.610652+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:24.088784+0000","scrub_duration":0.002045275,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":74,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.598121+0000","last_change":"2024-09-10T18:46:28.088657+0000","last_active":"2024-09-10T18:48:36.598121+0000","last_peered":"2024-09-10T18:48:36.598121+0000","last_clean":"2024-09-10T18:48:36.598121+0000","last_became_active":"2024-09-10T18:45:17.962108+0000","last_became_peered":"2024-09-10T18:45:17.962108+0000","last_unstale":"2024-09-10T18:48:36.598121+0000","last_undegraded":"2024-09-10T18:48:36.598121+0000","last_fullsized":"2024-09-10T18:48:36.598121+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:28.088588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:28.088588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00140508,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"63'10","reported_seq":98,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.594520+0000","last_change":"2024-09-10T18:48:23.844841+0000","last_active":"2024-09-10T18:48:36.594520+0000","last_peered":"2024-09-10T18:48:36.594520+0000","last_clean":"2024-09-10T18:48:36.594520+0000","last_became_active":"2024-09-10T18:45:20.421094+0000","last_became_peered":"2024-09-10T18:45:20.421094+0000","last_unstale":"2024-09-10T18:48:36.594520+0000","last_undegraded":"2024-09-10T18:48:36.594520+0000","last_fullsized":"2024-09-10T18:48:36.594520+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:48:23.844741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:23.844741+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:49.178164+0000","scrub_duration":0.0025648530000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":133,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:46.116517+0000","last_change":"2024-09-10T18:48:19.605745+0000","last_active":"2024-09-10T18:48:46.116517+0000","last_peered":"2024-09-10T18:48:46.116517+0000","last_clean":"2024-09-10T18:48:46.116517+0000","last_became_active":"2024-09-10T18:45:22.073281+0000","last_became_peered":"2024-09-10T18:45:22.073281+0000","last_unstale":"2024-09-10T18:48:46.116517+0000","last_undegraded":"2024-09-10T18:48:46.116517+0000","last_fullsized":"2024-09-10T18:48:46.116517+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:48:19.605646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:19.605646+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:22.732603+0000","scrub_duration":0.001752665,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":69,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.603089+0000","last_change":"2024-09-10T18:47:03.475367+0000","last_active":"2024-09-10T18:48:36.603089+0000","last_peered":"2024-09-10T18:48:36.603089+0000","last_clean":"2024-09-10T18:48:36.603089+0000","last_became_active":"2024-09-10T18:45:24.082318+0000","last_became_peered":"2024-09-10T18:45:24.082318+0000","last_unstale":"2024-09-10T18:48:36.603089+0000","last_undegraded":"2024-09-10T18:48:36.603089+0000","last_fullsized":"2024-09-10T18:48:36.603089+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:03.475299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:03.475299+0000","last_clean_scrub_stamp":"2024-09-10T18:47:03.475299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001661945,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":75,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.597942+0000","last_change":"2024-09-10T18:46:44.694805+0000","last_active":"2024-09-10T18:48:36.597942+0000","last_peered":"2024-09-10T18:48:36.597942+0000","last_clean":"2024-09-10T18:48:36.597942+0000","last_became_active":"2024-09-10T18:45:17.965707+0000","last_became_peered":"2024-09-10T18:45:17.965707+0000","last_unstale":"2024-09-10T18:48:36.597942+0000","last_undegraded":"2024-09-10T18:48:36.597942+0000","last_fullsized":"2024-09-10T18:48:36.597942+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:44.694737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:44.694737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016330400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"63'11","reported_seq":101,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.596131+0000","last_change":"2024-09-10T18:48:23.546688+0000","last_active":"2024-09-10T18:48:36.596131+0000","last_peered":"2024-09-10T18:48:36.596131+0000","last_clean":"2024-09-10T18:48:36.596131+0000","last_became_active":"2024-09-10T18:45:20.058717+0000","last_became_peered":"2024-09-10T18:45:20.058717+0000","last_unstale":"2024-09-10T18:48:36.596131+0000","last_undegraded":"2024-09-10T18:48:36.596131+0000","last_fullsized":"2024-09-10T18:48:36.596131+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:48:23.546598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:23.546598+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:28.876387+0000","scrub_duration":0.0057630149999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":81,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.594446+0000","last_change":"2024-09-10T18:47:36.419088+0000","last_active":"2024-09-10T18:48:36.594446+0000","last_peered":"2024-09-10T18:48:36.594446+0000","last_clean":"2024-09-10T18:48:36.594446+0000","last_became_active":"2024-09-10T18:47:36.418767+0000","last_became_peered":"2024-09-10T18:47:36.418767+0000","last_unstale":"2024-09-10T18:48:36.594446+0000","last_undegraded":"2024-09-10T18:48:36.594446+0000","last_fullsized":"2024-09-10T18:48:36.594446+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:15.536620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:47:15.536620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015375409999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":68,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.586725+0000","last_change":"2024-09-10T18:46:45.285074+0000","last_active":"2024-09-10T18:48:36.586725+0000","last_peered":"2024-09-10T18:48:36.586725+0000","last_clean":"2024-09-10T18:48:36.586725+0000","last_became_active":"2024-09-10T18:45:24.267251+0000","last_became_peered":"2024-09-10T18:45:24.267251+0000","last_unstale":"2024-09-10T18:48:36.586725+0000","last_undegraded":"2024-09-10T18:48:36.586725+0000","last_fullsized":"2024-09-10T18:48:36.586725+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:45.285015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:45.285015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001362142,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"56'1","reported_seq":93,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.595715+0000","last_change":"2024-09-10T18:48:31.488364+0000","last_active":"2024-09-10T18:48:36.595715+0000","last_peered":"2024-09-10T18:48:36.595715+0000","last_clean":"2024-09-10T18:48:36.595715+0000","last_became_active":"2024-09-10T18:45:17.966286+0000","last_became_peered":"2024-09-10T18:45:17.966286+0000","last_unstale":"2024-09-10T18:48:36.595715+0000","last_undegraded":"2024-09-10T18:48:36.595715+0000","last_fullsized":"2024-09-10T18:48:36.595715+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:48:31.488272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:31.488272+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:37.912107+0000","scrub_duration":0.0028000949999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"63'15","reported_seq":89,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.604500+0000","last_change":"2024-09-10T18:47:36.446154+0000","last_active":"2024-09-10T18:48:36.604500+0000","last_peered":"2024-09-10T18:48:36.604500+0000","last_clean":"2024-09-10T18:48:36.604500+0000","last_became_active":"2024-09-10T18:47:36.424373+0000","last_became_peered":"2024-09-10T18:47:36.424373+0000","last_unstale":"2024-09-10T18:48:36.604500+0000","last_undegraded":"2024-09-10T18:48:36.604500+0000","last_fullsized":"2024-09-10T18:48:36.604500+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:47:34.610915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:34.610915+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":71,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.605212+0000","last_change":"2024-09-10T18:47:09.520829+0000","last_active":"2024-09-10T18:48:36.605212+0000","last_peered":"2024-09-10T18:48:36.605212+0000","last_clean":"2024-09-10T18:48:36.605212+0000","last_became_active":"2024-09-10T18:45:22.059794+0000","last_became_peered":"2024-09-10T18:45:22.059794+0000","last_unstale":"2024-09-10T18:48:36.605212+0000","last_undegraded":"2024-09-10T18:48:36.605212+0000","last_fullsized":"2024-09-10T18:48:36.605212+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:09.520768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:47:09.520768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016284120000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":69,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.605268+0000","last_change":"2024-09-10T18:46:57.811148+0000","last_active":"2024-09-10T18:48:36.605268+0000","last_peered":"2024-09-10T18:48:36.605268+0000","last_clean":"2024-09-10T18:48:36.605268+0000","last_became_active":"2024-09-10T18:45:24.264427+0000","last_became_peered":"2024-09-10T18:45:24.264427+0000","last_unstale":"2024-09-10T18:48:36.605268+0000","last_undegraded":"2024-09-10T18:48:36.605268+0000","last_fullsized":"2024-09-10T18:48:36.605268+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:57.811074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:57.811074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013689419999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":75,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.599801+0000","last_change":"2024-09-10T18:46:47.734860+0000","last_active":"2024-09-10T18:48:36.599801+0000","last_peered":"2024-09-10T18:48:36.599801+0000","last_clean":"2024-09-10T18:48:36.599801+0000","last_became_active":"2024-09-10T18:45:17.963257+0000","last_became_peered":"2024-09-10T18:45:17.963257+0000","last_unstale":"2024-09-10T18:48:36.599801+0000","last_undegraded":"2024-09-10T18:48:36.599801+0000","last_fullsized":"2024-09-10T18:48:36.599801+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:47.734803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:47.734803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001381134,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"63'12","reported_seq":96,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.601029+0000","last_change":"2024-09-10T18:46:52.771640+0000","last_active":"2024-09-10T18:48:36.601029+0000","last_peered":"2024-09-10T18:48:36.601029+0000","last_clean":"2024-09-10T18:48:36.601029+0000","last_became_active":"2024-09-10T18:45:20.058391+0000","last_became_peered":"2024-09-10T18:45:20.058391+0000","last_unstale":"2024-09-10T18:48:36.601029+0000","last_undegraded":"2024-09-10T18:48:36.601029+0000","last_fullsized":"2024-09-10T18:48:36.601029+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:46:52.771570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:52.771570+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021490089999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":90,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-10T18:48:36.588546+0000","last_change":"2024-09-10T18:47:36.416505+0000","last_active":"2024-09-10T18:48:36.588546+0000","last_peered":"2024-09-10T18:48:36.588546+0000","last_clean":"2024-09-10T18:48:36.588546+0000","last_became_active":"2024-09-10T18:47:36.416131+0000","last_became_peered":"2024-09-10T18:47:36.416131+0000","last_unstale":"2024-09-10T18:48:36.588546+0000","last_undegraded":"2024-09-10T18:48:36.588546+0000","last_fullsized":"2024-09-10T18:48:36.588546+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:34.958889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:47:34.958889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:48:57.335633+0000","scrub_duration":0.001401296,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":7174619136,"num_objects":109476,"num_object_clones":0,"num_object_copies":328428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109476,"num_write_kb":7006464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":21526609920,"data_stored":21526609920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76776,"ondisk_log_size":76776,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":697,"num_read_kb":454,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":1238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":423,"ondisk_log_size":423,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":203,"num_read_kb":198,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299439,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2924992,"kb_used_data":2615160,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":90820160,"statfs":{"total":95995035648,"available":92999843840,"internally_reserved":0,"allocated":2677923840,"data_stored":2677527408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:48:08 2024","interfaces":[{"interface":"back","average":{"1min":2.1850000000000001,"5min":0.84199999999999997,"15min":0.61899999999999999},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":31.805,"5min":31.805,"15min":31.805},"last":3.9929999999999999},{"interface":"front","average":{"1min":2.0510000000000002,"5min":0.80600000000000005,"15min":0.59199999999999997},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":31.212,"5min":31.212,"15min":31.212},"last":4.0730000000000004}]},{"osd":1,"last update":"Tue Sep 10 18:48:08 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48899999999999999,"15min":0.48099999999999998},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.73799999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":2.169},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51500000000000001,"15min":0.51700000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":2.7349999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:48:08 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":2.073},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52600000000000002,"15min":0.51900000000000002},"min":{"1min":0.36699999999999999,"5min":0.31,"15min":0.31},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.58999999999999997}]},{"osd":3,"last update":"Tue Sep 10 18:48:08 2024","interfaces":[{"interface":"back","average":{"1min":2.1899999999999999,"5min":0.84199999999999997,"15min":0.61799999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":31.478000000000002,"5min":31.478000000000002,"15min":31.478000000000002},"last":2.7189999999999999},{"interface":"front","average":{"1min":2.6629999999999998,"5min":0.94999999999999996,"15min":0.66200000000000003},"min":{"1min":0.35199999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":33.731999999999999,"5min":33.731999999999999,"15min":33.731999999999999},"last":2.2010000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:48:08 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.38300000000000001,"15min":0.38400000000000001},"min":{"1min":0.27800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.55200000000000005,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39000000000000001,"15min":0.38},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.55800000000000005,"5min":0.623,"15min":0.623},"last":0.86799999999999999}]},{"osd":5,"last update":"Tue Sep 10 18:48:08 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40899999999999997,"15min":0.41099999999999998},"min":{"1min":0.29399999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.42099999999999999,"15min":0.41699999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.61699999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.83799999999999997}]},{"osd":6,"last update":"Tue Sep 10 18:48:08 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.54200000000000004,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.432,"5min":0.44,"15min":0.435},"min":{"1min":0.32100000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.61199999999999999,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":1.323}]}]},{"osd":6,"up_from":43,"seq":184683593780,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2922696,"kb_used_data":2617024,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":90822456,"statfs":{"total":95995035648,"available":93002194944,"internally_reserved":0,"allocated":2679832576,"data_stored":2679428496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:48:46 2024","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":0.64900000000000002,"15min":0.52100000000000002},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":4.681,"5min":4.681,"15min":4.681},"last":1.115},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.58799999999999997,"15min":0.52300000000000002},"min":{"1min":0.315,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":3.1709999999999998,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":1.028}]},{"osd":1,"last update":"Tue Sep 10 18:48:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.55100000000000005,"15min":0.50900000000000001},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.56899999999999995,"15min":0.497},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":3.6280000000000001,"5min":3.6280000000000001,"15min":3.6280000000000001},"last":1.7529999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:48:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":0.58699999999999997,"15min":0.51600000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":3.359,"5min":3.359,"15min":3.359},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.52000000000000002,"15min":0.5},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.40699999999999997}]},{"osd":3,"last update":"Tue Sep 10 18:48:46 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.51900000000000002,"15min":0.495},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":0.55100000000000005},{"interface":"front","average":{"1min":1.069,"5min":0.628,"15min":0.54900000000000004},"min":{"1min":0.33600000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":4.5309999999999997,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":0.753}]},{"osd":4,"last update":"Tue Sep 10 18:48:46 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.41799999999999998,"15min":0.40899999999999997},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.41899999999999998,"15min":0.39500000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.309}]},{"osd":5,"last update":"Tue Sep 10 18:48:46 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.439,"15min":0.42199999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.46300000000000002,"15min":0.435},"min":{"1min":0.29799999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":0.50900000000000001}]},{"osd":7,"last update":"Tue Sep 10 18:48:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.495,"15min":0.47699999999999998},"min":{"1min":0.32000000000000001,"5min":0.309,"15min":0.309},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.51600000000000001,"15min":0.501},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.883,"5min":1.883,"15min":1.883},"last":0.53500000000000003}]}]},{"osd":5,"up_from":37,"seq":158913790008,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2043128,"kb_used_data":1737456,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":91702024,"statfs":{"total":95995035648,"available":93902872576,"internally_reserved":0,"allocated":1779154944,"data_stored":1778765628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:48:22 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.53900000000000003,"15min":0.53100000000000003},"min":{"1min":0.376,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.49299999999999999},{"interface":"front","average":{"1min":1.2789999999999999,"5min":0.64000000000000001,"15min":0.54600000000000004},"min":{"1min":0.40400000000000003,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":6.742,"5min":6.742,"15min":6.742},"last":0.65000000000000002}]},{"osd":1,"last update":"Tue Sep 10 18:48:22 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.746,"5min":1.746,"15min":1.746},"last":0.33100000000000002},{"interface":"front","average":{"1min":1.224,"5min":0.65900000000000003,"15min":0.58099999999999996},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":6.6989999999999998,"5min":6.6989999999999998,"15min":6.6989999999999998},"last":0.311}]},{"osd":2,"last update":"Tue Sep 10 18:48:22 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.50800000000000001,"15min":0.51600000000000001},"min":{"1min":0.34799999999999998,"5min":0.311,"15min":0.311},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.41999999999999998},{"interface":"front","average":{"1min":1.3700000000000001,"5min":0.68799999999999994,"15min":0.59199999999999997},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":6.7210000000000001,"5min":6.7210000000000001,"15min":6.7210000000000001},"last":0.40600000000000003}]},{"osd":3,"last update":"Tue Sep 10 18:48:22 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":0.624,"15min":0.56000000000000005},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":3.6760000000000002,"5min":3.6760000000000002,"15min":3.6760000000000002},"last":0.55100000000000005},{"interface":"front","average":{"1min":1.3999999999999999,"5min":0.69699999999999995,"15min":0.59699999999999998},"min":{"1min":0.372,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":5.8099999999999996,"5min":5.8099999999999996,"15min":5.8099999999999996},"last":2.625}]},{"osd":4,"last update":"Tue Sep 10 18:48:22 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.39000000000000001,"15min":0.38500000000000001},"min":{"1min":0.248,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.38800000000000001,"15min":0.39800000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.502,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.51400000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:48:47 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.49099999999999999,"15min":0.439},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.53600000000000003,"15min":0.48399999999999999},"min":{"1min":0.33000000000000002,"5min":0.31,"15min":0.31},"max":{"1min":2.2599999999999998,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":0.57399999999999995}]},{"osd":7,"last update":"Tue Sep 10 18:48:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.47099999999999997,"15min":0.47499999999999998},"min":{"1min":0.32100000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.62,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.46999999999999997,"15min":0.47499999999999998},"min":{"1min":0.30099999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.52900000000000003}]}]},{"osd":4,"up_from":30,"seq":128849018942,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2481280,"kb_used_data":2179768,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":91263872,"statfs":{"total":95995035648,"available":93454204928,"internally_reserved":0,"allocated":2232082432,"data_stored":2231685307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52800000000000002,"15min":0.52600000000000002},"min":{"1min":0.23200000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51600000000000001,"15min":0.53400000000000003},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":3.3279999999999998}]},{"osd":1,"last update":"Tue Sep 10 18:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.54900000000000004,"15min":0.56499999999999995},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.82499999999999996,"5min":2.609,"15min":2.609},"last":3.1909999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.55100000000000005,"15min":0.53300000000000003},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.98799999999999999,"5min":2.589,"15min":2.589},"last":0.76400000000000001}]},{"osd":2,"last update":"Tue Sep 10 18:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.80000000000000004,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53500000000000003,"15min":0.55000000000000004},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.40799999999999997}]},{"osd":3,"last update":"Tue Sep 10 18:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.53400000000000003,"15min":0.53200000000000003},"min":{"1min":0.33800000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.53300000000000003,"15min":0.54000000000000004},"min":{"1min":0.34799999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.59599999999999997}]},{"osd":5,"last update":"Tue Sep 10 18:48:27 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.40699999999999997,"15min":0.36599999999999999},"min":{"1min":0.28899999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":2.8479999999999999,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.39900000000000002,"15min":0.36899999999999999},"min":{"1min":0.26000000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.36099999999999999}]},{"osd":6,"last update":"Tue Sep 10 18:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.48499999999999999,"15min":0.442},"min":{"1min":0.30599999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":3.5529999999999999,"5min":3.5529999999999999,"15min":3.5529999999999999},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.48099999999999998,"15min":0.41799999999999998},"min":{"1min":0.33100000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":0.52800000000000002}]},{"osd":7,"last update":"Tue Sep 10 18:48:12 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.49199999999999999,"15min":0.498},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.442,"15min":0.44700000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.443}]}]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5111500,"kb_used_data":4805828,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":88633652,"statfs":{"total":95995035648,"available":90760859648,"internally_reserved":0,"allocated":4921167872,"data_stored":4920758292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:48:37 2024","interfaces":[{"interface":"back","average":{"1min":1.7589999999999999,"5min":0.64600000000000002,"15min":0.47299999999999998},"min":{"1min":0.223,"5min":0.215,"15min":0.215},"max":{"1min":7.3949999999999996,"5min":7.3949999999999996,"15min":7.3949999999999996},"last":47.601999999999997},{"interface":"front","average":{"1min":1.776,"5min":0.66900000000000004,"15min":0.49299999999999999},"min":{"1min":0.28100000000000003,"5min":0.217,"15min":0.217},"max":{"1min":4.6100000000000003,"5min":4.6100000000000003,"15min":4.6100000000000003},"last":46.787999999999997}]},{"osd":1,"last update":"Tue Sep 10 18:48:37 2024","interfaces":[{"interface":"back","average":{"1min":1.7949999999999999,"5min":0.66800000000000004,"15min":0.497},"min":{"1min":0.29899999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":4.3940000000000001,"5min":4.3940000000000001,"15min":4.3940000000000001},"last":0.25900000000000001},{"interface":"front","average":{"1min":1.548,"5min":0.628,"15min":0.48799999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":3.9430000000000001,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":0.27100000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:48:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":0.59299999999999997,"15min":0.49199999999999999},"min":{"1min":0.26600000000000001,"5min":0.246,"15min":0.246},"max":{"1min":3.907,"5min":3.907,"15min":3.907},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.46200000000000002,"15min":0.44700000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":0.248}]},{"osd":4,"last update":"Tue Sep 10 18:47:59 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.58499999999999996,"15min":0.55800000000000005},"min":{"1min":0.375,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":3.0699999999999998,"5min":3.0699999999999998,"15min":3.0699999999999998},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.60299999999999998,"15min":0.58499999999999996},"min":{"1min":0.35199999999999998,"5min":0.311,"15min":0.311},"max":{"1min":3.0990000000000002,"5min":3.0990000000000002,"15min":3.0990000000000002},"last":0.42799999999999999}]},{"osd":5,"last update":"Tue Sep 10 18:48:27 2024","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":0.67500000000000004,"15min":0.58499999999999996},"min":{"1min":0.433,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":2.9780000000000002,"5min":2.9780000000000002,"15min":2.9780000000000002},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.65000000000000002,"15min":0.59199999999999997},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":3.5819999999999999,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":0.40799999999999997}]},{"osd":6,"last update":"Tue Sep 10 18:48:49 2024","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":0.72799999999999998,"15min":0.61899999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":3.859,"5min":3.859,"15min":3.859},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.64300000000000002,"15min":0.58499999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":2.5310000000000001,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":0.375}]},{"osd":7,"last update":"Tue Sep 10 18:48:11 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.623,"15min":0.56100000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":2.694,"5min":2.694,"15min":2.694},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.60799999999999998,"15min":0.58599999999999997},"min":{"1min":0.434,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.665,"5min":2.665,"15min":2.665},"last":0.443}]}]},{"osd":2,"up_from":0,"seq":77309411399,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3779860,"kb_used_data":3482508,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":89965292,"statfs":{"total":95995035648,"available":92124459008,"internally_reserved":0,"allocated":3566088192,"data_stored":3565671003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:47:50 2024","interfaces":[{"interface":"back","average":{"1min":0.28799999999999998,"5min":0.34000000000000002,"15min":0.31900000000000001},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.47199999999999998,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":1.089},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.374,"15min":0.33900000000000002},"min":{"1min":0.26200000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.64600000000000002,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":1.127}]},{"osd":2,"last update":"Tue Sep 10 18:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.42999999999999999,"15min":0.38},"min":{"1min":0.22900000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":3.8479999999999999,"5min":3.8479999999999999,"15min":3.8479999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.41199999999999998,"15min":0.38800000000000001},"min":{"1min":0.30299999999999999,"5min":0.248,"15min":0.248},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":1.056}]},{"osd":3,"last update":"Tue Sep 10 18:48:44 2024","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":0.65000000000000002,"15min":0.52300000000000002},"min":{"1min":0.29199999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.6160000000000001,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":1.4259999999999999},{"interface":"front","average":{"1min":2.2349999999999999,"5min":0.77600000000000002,"15min":0.54000000000000004},"min":{"1min":0.35999999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":5.3140000000000001,"5min":5.3140000000000001,"15min":5.3140000000000001},"last":1.224}]},{"osd":4,"last update":"Tue Sep 10 18:48:03 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.63300000000000001,"15min":0.57899999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":4.1520000000000001,"5min":4.1520000000000001,"15min":4.1520000000000001},"last":1.2649999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.55300000000000005,"15min":0.53700000000000003},"min":{"1min":0.33700000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.67400000000000004}]},{"osd":5,"last update":"Tue Sep 10 18:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.63200000000000001,"15min":0.61399999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.3979999999999999},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.68100000000000005,"15min":0.65600000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":4.1130000000000004,"5min":4.1130000000000004,"15min":4.1130000000000004},"last":1.2969999999999999}]},{"osd":6,"last update":"Tue Sep 10 18:48:44 2024","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":0.68999999999999995,"15min":0.65100000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":3.157,"5min":3.157,"15min":3.157},"last":1.375},{"interface":"front","average":{"1min":1.2729999999999999,"5min":0.76100000000000001,"15min":0.70299999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":4.1769999999999996,"5min":4.1769999999999996,"15min":4.1769999999999996},"last":1.2450000000000001}]},{"osd":7,"last update":"Tue Sep 10 18:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.64900000000000002,"15min":0.64800000000000002},"min":{"1min":0.41999999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.1800000000000002,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":1.452},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.70199999999999996,"15min":0.66600000000000004},"min":{"1min":0.42899999999999999,"5min":0.372,"15min":0.372},"max":{"1min":4.1989999999999998,"5min":4.1989999999999998,"15min":4.1989999999999998},"last":1.343}]}]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3781576,"kb_used_data":3471744,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":89963576,"statfs":{"total":95995035648,"available":92122701824,"internally_reserved":0,"allocated":3555065856,"data_stored":3554660664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 18:47:54 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.38300000000000001,"15min":0.32300000000000001},"min":{"1min":0.24199999999999999,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":4.7599999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.379,"15min":0.32900000000000001},"min":{"1min":0.23499999999999999,"5min":0.187,"15min":0.187},"max":{"1min":2.0990000000000002,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":1.577}]},{"osd":2,"last update":"Tue Sep 10 18:48:25 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.48399999999999999,"15min":0.41599999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":4.6509999999999998,"5min":4.6509999999999998,"15min":4.6509999999999998},"last":1.2529999999999999},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.47699999999999998,"15min":0.39500000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":4.6230000000000002,"5min":4.6230000000000002,"15min":4.6230000000000002},"last":1.2709999999999999}]},{"osd":3,"last update":"Tue Sep 10 18:48:38 2024","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":0.57699999999999996,"15min":0.45900000000000002},"min":{"1min":0.26000000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":3.8119999999999998,"5min":3.8119999999999998,"15min":3.8119999999999998},"last":4.0309999999999997},{"interface":"front","average":{"1min":1.403,"5min":0.59999999999999998,"15min":0.47599999999999998},"min":{"1min":0.29499999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":3.6920000000000002,"5min":3.6920000000000002,"15min":3.6920000000000002},"last":1.056}]},{"osd":4,"last update":"Tue Sep 10 18:48:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.56799999999999995,"15min":0.54900000000000004},"min":{"1min":0.436,"5min":0.375,"15min":0.375},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":1.121},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.61699999999999999,"15min":0.55100000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":2.052,"5min":4.6189999999999998,"15min":4.6189999999999998},"last":1.198}]},{"osd":5,"last update":"Tue Sep 10 18:48:25 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.60899999999999999,"15min":0.57499999999999996},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":1.0900000000000001},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.68000000000000005,"15min":0.58899999999999997},"min":{"1min":0.40500000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.9580000000000002,"5min":4.6429999999999998,"15min":4.6429999999999998},"last":0.67300000000000004}]},{"osd":6,"last update":"Tue Sep 10 18:48:49 2024","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":0.72999999999999998,"15min":0.61199999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":3.855,"5min":3.855,"15min":3.855},"last":1.238},{"interface":"front","average":{"1min":1.0780000000000001,"5min":0.76400000000000001,"15min":0.77900000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":2.5830000000000002,"5min":4.6630000000000003,"15min":4.6630000000000003},"last":1.1499999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:48:10 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.59799999999999998,"15min":0.56999999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.64800000000000002,"15min":0.61099999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.264}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3572629504,"data_stored":3572629504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3587375104,"data_stored":3587375104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4942135296,"data_stored":4942135296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2248278016,"data_stored":2248278016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1795555328,"data_stored":1795555328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2696085504,"data_stored":2696085504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2684551168,"data_stored":2684551168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T18:48:58.977 INFO:tasks.cephadm.ceph_manager.ceph:would drop below min_pgs, proj -9, currently 1 2024-09-10T18:48:58.992 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:48:59.023 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:59.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:59.024 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:48:59.125 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:48:59.170 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:59.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:59.171 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:48:59.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:59 smithi018 ceph-mon[40196]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:48:59.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:59 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:48:59.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:48:59 smithi018 ceph-mon[40196]: osdmap e113: 8 total, 8 up, 7 in 2024-09-10T18:48:59.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:59 smithi018 ceph-mon[29336]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:48:59.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:59 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:48:59.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:48:59 smithi018 ceph-mon[29336]: osdmap e113: 8 total, 8 up, 7 in 2024-09-10T18:48:59.214 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 75 16 118953 118937 99.0915 69.4375 0.0138619 0.0100868 2024-09-10T18:48:59.253 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:48:59.272 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:48:59.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:59.325 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:48:59.325 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:48:59.426 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:59.458 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:59.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:59.459 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:59.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:59 smithi103 ceph-mon[33163]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:48:59.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:59 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:48:59.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:48:59 smithi103 ceph-mon[33163]: osdmap e113: 8 total, 8 up, 7 in 2024-09-10T18:48:59.560 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:48:59.594 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:59.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:59.595 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:48:59.696 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:59.722 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:59.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:59.723 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:59.825 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:48:59.857 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:48:59.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:48:59.859 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:48:59.959 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:00.012 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:00.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:00.013 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:00.114 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:00.158 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:00.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:00.159 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:00.214 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 76 16 120268 120252 98.869 82.1875 0.0140299 0.01011 2024-09-10T18:49:00.220 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:00 smithi018 ceph-mon[29336]: 5.1e scrub starts 2024-09-10T18:49:00.220 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:00 smithi018 ceph-mon[29336]: 5.1e scrub ok 2024-09-10T18:49:00.220 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:00 smithi018 ceph-mon[29336]: pgmap v138: 148 pgs: 148 active+clean; 7.1 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 1023 B/s rd, 76 MiB/s wr, 1.22k op/s 2024-09-10T18:49:00.220 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:00 smithi018 ceph-mon[29336]: 2.1c scrub starts 2024-09-10T18:49:00.220 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:00 smithi018 ceph-mon[29336]: 2.1c scrub ok 2024-09-10T18:49:00.220 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:00 smithi018 ceph-mon[40196]: 5.1e scrub starts 2024-09-10T18:49:00.220 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:00 smithi018 ceph-mon[40196]: 5.1e scrub ok 2024-09-10T18:49:00.220 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:00 smithi018 ceph-mon[40196]: pgmap v138: 148 pgs: 148 active+clean; 7.1 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 1023 B/s rd, 76 MiB/s wr, 1.22k op/s 2024-09-10T18:49:00.220 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:00 smithi018 ceph-mon[40196]: 2.1c scrub starts 2024-09-10T18:49:00.220 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:00 smithi018 ceph-mon[40196]: 2.1c scrub ok 2024-09-10T18:49:00.260 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:00.286 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:00.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:00.287 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:00.388 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:00.430 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:00.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:00.431 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:00.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:49:00 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:49:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:49:00.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:00 smithi103 ceph-mon[33163]: 5.1e scrub starts 2024-09-10T18:49:00.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:00 smithi103 ceph-mon[33163]: 5.1e scrub ok 2024-09-10T18:49:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:00 smithi103 ceph-mon[33163]: pgmap v138: 148 pgs: 148 active+clean; 7.1 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 1023 B/s rd, 76 MiB/s wr, 1.22k op/s 2024-09-10T18:49:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:00 smithi103 ceph-mon[33163]: 2.1c scrub starts 2024-09-10T18:49:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:00 smithi103 ceph-mon[33163]: 2.1c scrub ok 2024-09-10T18:49:00.531 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:00.562 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:00.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:00.563 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:00.589 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T18:49:00.589 INFO:teuthology.orchestra.run.smithi018.stdout: "ops": [], 2024-09-10T18:49:00.589 INFO:teuthology.orchestra.run.smithi018.stdout: "complaint_time": 30, 2024-09-10T18:49:00.589 INFO:teuthology.orchestra.run.smithi018.stdout: "num_blocked_ops": 0 2024-09-10T18:49:00.590 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T18:49:00.663 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:00.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:00.696 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:00.696 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:00.797 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:00.836 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:00.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:00.837 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:00.938 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:00.969 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:00.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:00.970 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:01.070 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:01.113 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:01.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:01.114 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:01.214 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 77 16 121511 121495 98.5939 77.6875 0.0125475 0.0101381 2024-09-10T18:49:01.215 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:01.244 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:01.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:01.245 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:01.346 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:01.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:01.378 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:01.378 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:01.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:01 smithi018 ceph-mon[40196]: 3.2 scrub starts 2024-09-10T18:49:01.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:01 smithi018 ceph-mon[40196]: 3.2 scrub ok 2024-09-10T18:49:01.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:01 smithi018 ceph-mon[40196]: 5.5 scrub starts 2024-09-10T18:49:01.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:01 smithi018 ceph-mon[40196]: 5.5 scrub ok 2024-09-10T18:49:01.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:01 smithi018 ceph-mon[29336]: 3.2 scrub starts 2024-09-10T18:49:01.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:01 smithi018 ceph-mon[29336]: 3.2 scrub ok 2024-09-10T18:49:01.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:01 smithi018 ceph-mon[29336]: 5.5 scrub starts 2024-09-10T18:49:01.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:01 smithi018 ceph-mon[29336]: 5.5 scrub ok 2024-09-10T18:49:01.445 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:49:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:49:00] "GET /metrics HTTP/1.1" 200 223602 "" "Prometheus/2.43.0" 2024-09-10T18:49:01.478 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:01.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:01 smithi103 ceph-mon[33163]: 3.2 scrub starts 2024-09-10T18:49:01.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:01 smithi103 ceph-mon[33163]: 3.2 scrub ok 2024-09-10T18:49:01.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:01 smithi103 ceph-mon[33163]: 5.5 scrub starts 2024-09-10T18:49:01.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:01 smithi103 ceph-mon[33163]: 5.5 scrub ok 2024-09-10T18:49:01.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:01.523 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:01.523 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:01.624 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:01.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:01.654 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:01.654 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:01.741 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-10T18:49:01.755 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:01.843 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:01.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:01.845 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:01.946 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:01.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:01.993 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:01.993 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:02.094 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:02.121 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:02.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:02.122 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:02.218 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 78 16 122734 122718 98.3098 76.4375 0.0197634 0.0101669 2024-09-10T18:49:02.222 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:02.229 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-10T18:49:02.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:02.273 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:02.273 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:02.374 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:02.406 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:02.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:02.407 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:02.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:02 smithi018 ceph-mon[29336]: 5.3 scrub starts 2024-09-10T18:49:02.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:02 smithi018 ceph-mon[29336]: 5.3 scrub ok 2024-09-10T18:49:02.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:02 smithi018 ceph-mon[29336]: pgmap v139: 148 pgs: 148 active+clean; 7.2 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1023 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-10T18:49:02.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:02 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:49:02.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:02 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:49:02.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:02 smithi018 ceph-mon[29336]: 7.7 scrub starts 2024-09-10T18:49:02.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:02 smithi018 ceph-mon[40196]: 5.3 scrub starts 2024-09-10T18:49:02.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:02 smithi018 ceph-mon[40196]: 5.3 scrub ok 2024-09-10T18:49:02.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:02 smithi018 ceph-mon[40196]: pgmap v139: 148 pgs: 148 active+clean; 7.2 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1023 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-10T18:49:02.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:02 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:49:02.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:02 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:49:02.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:02 smithi018 ceph-mon[40196]: 7.7 scrub starts 2024-09-10T18:49:02.508 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:02.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:02 smithi103 ceph-mon[33163]: 5.3 scrub starts 2024-09-10T18:49:02.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:02 smithi103 ceph-mon[33163]: 5.3 scrub ok 2024-09-10T18:49:02.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:02 smithi103 ceph-mon[33163]: pgmap v139: 148 pgs: 148 active+clean; 7.2 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1023 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-10T18:49:02.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:02 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:49:02.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:02 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:49:02.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:02 smithi103 ceph-mon[33163]: 7.7 scrub starts 2024-09-10T18:49:02.534 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:02.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:02.536 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:02.636 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:02.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:02.669 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:02.669 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:02.770 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:02.803 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:49:02.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:02.830 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:02.830 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:02.868 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:49:02.869 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:49:02.911 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:49:02.911 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:49:02.932 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:02.942 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:49:02.943 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:49:02.981 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:49:02.982 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:49:02.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:02.983 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:02.984 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:03.015 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:49:03.015 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:49:03.046 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:49:03.047 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:49:03.081 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:49:03.081 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:49:03.084 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:03.112 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:03.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:03.113 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:03.114 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:49:03.114 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:49:03.214 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:03.215 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 79 16 123803 123787 97.9111 66.8125 0.017907 0.0102085 2024-09-10T18:49:03.261 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:03.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:03.263 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:03.364 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:03.405 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:03.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:03.407 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:03.427 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:03 smithi018 ceph-mon[40196]: 6.0 scrub starts 2024-09-10T18:49:03.427 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:03 smithi018 ceph-mon[40196]: 6.0 scrub ok 2024-09-10T18:49:03.427 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:03 smithi018 ceph-mon[40196]: 4.14 scrub starts 2024-09-10T18:49:03.427 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:03 smithi018 ceph-mon[40196]: 4.14 scrub ok 2024-09-10T18:49:03.428 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:03 smithi018 ceph-mon[40196]: 5.15 scrub starts 2024-09-10T18:49:03.428 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:03 smithi018 ceph-mon[40196]: 5.15 scrub ok 2024-09-10T18:49:03.428 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:03 smithi018 ceph-mon[40196]: osdmap e114: 8 total, 8 up, 7 in 2024-09-10T18:49:03.428 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:03 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:49:03.428 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:03 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:49:03.428 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:03 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1151800992' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:49:03.429 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:03 smithi018 ceph-mon[29336]: 6.0 scrub starts 2024-09-10T18:49:03.429 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:03 smithi018 ceph-mon[29336]: 6.0 scrub ok 2024-09-10T18:49:03.429 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:03 smithi018 ceph-mon[29336]: 4.14 scrub starts 2024-09-10T18:49:03.429 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:03 smithi018 ceph-mon[29336]: 4.14 scrub ok 2024-09-10T18:49:03.429 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:03 smithi018 ceph-mon[29336]: 5.15 scrub starts 2024-09-10T18:49:03.429 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:03 smithi018 ceph-mon[29336]: 5.15 scrub ok 2024-09-10T18:49:03.429 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:03 smithi018 ceph-mon[29336]: osdmap e114: 8 total, 8 up, 7 in 2024-09-10T18:49:03.430 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:03 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:49:03.430 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:03 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:49:03.430 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:03 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1151800992' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:49:03.508 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:03.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:03 smithi103 ceph-mon[33163]: 6.0 scrub starts 2024-09-10T18:49:03.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:03 smithi103 ceph-mon[33163]: 6.0 scrub ok 2024-09-10T18:49:03.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:03 smithi103 ceph-mon[33163]: 4.14 scrub starts 2024-09-10T18:49:03.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:03 smithi103 ceph-mon[33163]: 4.14 scrub ok 2024-09-10T18:49:03.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:03 smithi103 ceph-mon[33163]: 5.15 scrub starts 2024-09-10T18:49:03.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:03 smithi103 ceph-mon[33163]: 5.15 scrub ok 2024-09-10T18:49:03.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:03 smithi103 ceph-mon[33163]: osdmap e114: 8 total, 8 up, 7 in 2024-09-10T18:49:03.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:03 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:49:03.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:03 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:49:03.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:03 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1151800992' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:49:03.544 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:03.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:03.545 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:03.646 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:03.690 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:03.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:03.691 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:03.792 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:03.826 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:03.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:03.827 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:03.928 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:03.949 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:03.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:03.950 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:03.978 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-10T18:49:03.978 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T18:49:03.978 INFO:tasks.thrashosds.thrasher:Shrinking pool .mgr 2024-09-10T18:49:03.979 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool .mgr by 10 min 8 2024-09-10T18:49:03.979 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T18:49:04.050 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:04.080 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:04.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:04.081 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:04.182 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:04.215 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:2024-09-10T18:49:04.214550+0000 min lat: 0.00126001 max lat: 0.14754 avg lat: 0.0102461 2024-09-10T18:49:04.215 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T18:49:04.215 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 80 16 124910 124894 97.552 69.1875 0.0152274 0.0102461 2024-09-10T18:49:04.228 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:04.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:04.229 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:04.329 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:04.376 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:04.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:04.377 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:04.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:04 smithi018 ceph-mon[40196]: 6.18 scrub starts 2024-09-10T18:49:04.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:04 smithi018 ceph-mon[40196]: 6.18 scrub ok 2024-09-10T18:49:04.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:04 smithi018 ceph-mon[40196]: 4.19 scrub starts 2024-09-10T18:49:04.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:04 smithi018 ceph-mon[40196]: 4.19 scrub ok 2024-09-10T18:49:04.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:04 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:49:04.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:04 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:49:04.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:04 smithi018 ceph-mon[40196]: pgmap v141: 148 pgs: 148 active+clean; 7.4 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1.2 KiB/s rd, 93 MiB/s wr, 1.49k op/s 2024-09-10T18:49:04.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:04 smithi018 ceph-mon[40196]: osdmap e115: 8 total, 8 up, 7 in 2024-09-10T18:49:04.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:04 smithi018 ceph-mon[40196]: 4.d scrub starts 2024-09-10T18:49:04.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:04 smithi018 ceph-mon[40196]: 4.d scrub ok 2024-09-10T18:49:04.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:04 smithi018 ceph-mon[29336]: 6.18 scrub starts 2024-09-10T18:49:04.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:04 smithi018 ceph-mon[29336]: 6.18 scrub ok 2024-09-10T18:49:04.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:04 smithi018 ceph-mon[29336]: 4.19 scrub starts 2024-09-10T18:49:04.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:04 smithi018 ceph-mon[29336]: 4.19 scrub ok 2024-09-10T18:49:04.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:04 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:49:04.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:04 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:49:04.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:04 smithi018 ceph-mon[29336]: pgmap v141: 148 pgs: 148 active+clean; 7.4 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1.2 KiB/s rd, 93 MiB/s wr, 1.49k op/s 2024-09-10T18:49:04.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:04 smithi018 ceph-mon[29336]: osdmap e115: 8 total, 8 up, 7 in 2024-09-10T18:49:04.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:04 smithi018 ceph-mon[29336]: 4.d scrub starts 2024-09-10T18:49:04.448 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:04 smithi018 ceph-mon[29336]: 4.d scrub ok 2024-09-10T18:49:04.478 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:04.521 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:04.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:04.522 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:04.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:04 smithi103 ceph-mon[33163]: 6.18 scrub starts 2024-09-10T18:49:04.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:04 smithi103 ceph-mon[33163]: 6.18 scrub ok 2024-09-10T18:49:04.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:04 smithi103 ceph-mon[33163]: 4.19 scrub starts 2024-09-10T18:49:04.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:04 smithi103 ceph-mon[33163]: 4.19 scrub ok 2024-09-10T18:49:04.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:04 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:49:04.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:04 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:49:04.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:04 smithi103 ceph-mon[33163]: pgmap v141: 148 pgs: 148 active+clean; 7.4 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1.2 KiB/s rd, 93 MiB/s wr, 1.49k op/s 2024-09-10T18:49:04.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:04 smithi103 ceph-mon[33163]: osdmap e115: 8 total, 8 up, 7 in 2024-09-10T18:49:04.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:04 smithi103 ceph-mon[33163]: 4.d scrub starts 2024-09-10T18:49:04.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:04 smithi103 ceph-mon[33163]: 4.d scrub ok 2024-09-10T18:49:04.618 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:49:04.624 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:04.651 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:04.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:04.652 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:04.753 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:04.777 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:04.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:04.778 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:04.879 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:04.910 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:04.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:04.911 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:05.012 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:05.041 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:05.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:05.042 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:05.143 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:05.173 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:05.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:05.174 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:05.215 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 81 16 125827 125811 97.0552 57.3125 0.0108863 0.0102985 2024-09-10T18:49:05.274 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:05.315 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:05.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:05.316 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:05.417 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:05.457 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:05.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:05.458 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:05.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:05 smithi103 ceph-mon[33163]: 6.16 scrub starts 2024-09-10T18:49:05.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:05 smithi103 ceph-mon[33163]: 6.16 scrub ok 2024-09-10T18:49:05.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:05 smithi103 ceph-mon[33163]: 4.11 scrub starts 2024-09-10T18:49:05.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:05 smithi103 ceph-mon[33163]: 4.11 scrub ok 2024-09-10T18:49:05.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:05 smithi103 ceph-mon[33163]: 4.1 scrub starts 2024-09-10T18:49:05.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:05 smithi103 ceph-mon[33163]: 4.1 scrub ok 2024-09-10T18:49:05.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:05 smithi103 ceph-mon[33163]: 2.14 scrub starts 2024-09-10T18:49:05.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:05 smithi103 ceph-mon[33163]: 2.14 scrub ok 2024-09-10T18:49:05.559 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:05.574 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:05 smithi018 ceph-mon[29336]: 6.16 scrub starts 2024-09-10T18:49:05.574 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:05 smithi018 ceph-mon[29336]: 6.16 scrub ok 2024-09-10T18:49:05.574 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:05 smithi018 ceph-mon[29336]: 4.11 scrub starts 2024-09-10T18:49:05.574 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:05 smithi018 ceph-mon[29336]: 4.11 scrub ok 2024-09-10T18:49:05.574 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:05 smithi018 ceph-mon[29336]: 4.1 scrub starts 2024-09-10T18:49:05.574 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:05 smithi018 ceph-mon[29336]: 4.1 scrub ok 2024-09-10T18:49:05.574 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:05 smithi018 ceph-mon[29336]: 2.14 scrub starts 2024-09-10T18:49:05.575 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:05 smithi018 ceph-mon[29336]: 2.14 scrub ok 2024-09-10T18:49:05.575 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:05 smithi018 ceph-mon[40196]: 6.16 scrub starts 2024-09-10T18:49:05.575 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:05 smithi018 ceph-mon[40196]: 6.16 scrub ok 2024-09-10T18:49:05.575 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:05 smithi018 ceph-mon[40196]: 4.11 scrub starts 2024-09-10T18:49:05.575 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:05 smithi018 ceph-mon[40196]: 4.11 scrub ok 2024-09-10T18:49:05.575 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:05 smithi018 ceph-mon[40196]: 4.1 scrub starts 2024-09-10T18:49:05.575 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:05 smithi018 ceph-mon[40196]: 4.1 scrub ok 2024-09-10T18:49:05.575 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:05 smithi018 ceph-mon[40196]: 2.14 scrub starts 2024-09-10T18:49:05.575 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:05 smithi018 ceph-mon[40196]: 2.14 scrub ok 2024-09-10T18:49:05.589 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:05.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:05.590 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:05.691 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:05.721 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:05.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:05.722 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:05.822 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:05.851 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:05.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:05.852 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:05.953 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:05.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:05.982 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:05.982 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:06.083 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:06.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:06.131 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:06.131 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:06.215 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 82 16 127053 127037 96.806 76.625 0.0116299 0.0103256 2024-09-10T18:49:06.232 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:06.263 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:06.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:06.264 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:06.365 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:06.394 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:06.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:06.395 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:06.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:06 smithi018 ceph-mon[40196]: 7.4 scrub starts 2024-09-10T18:49:06.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:06 smithi018 ceph-mon[40196]: 6.c scrub starts 2024-09-10T18:49:06.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:06 smithi018 ceph-mon[40196]: 6.c scrub ok 2024-09-10T18:49:06.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:06 smithi018 ceph-mon[40196]: pgmap v143: 148 pgs: 148 active+clean; 7.4 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 740 B/s rd, 51 MiB/s wr, 821 op/s 2024-09-10T18:49:06.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:06 smithi018 ceph-mon[40196]: 7.7 scrub ok 2024-09-10T18:49:06.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:06 smithi018 ceph-mon[40196]: 4.12 scrub starts 2024-09-10T18:49:06.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:06 smithi018 ceph-mon[40196]: 4.12 scrub ok 2024-09-10T18:49:06.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:06 smithi018 ceph-mon[29336]: 7.4 scrub starts 2024-09-10T18:49:06.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:06 smithi018 ceph-mon[29336]: 6.c scrub starts 2024-09-10T18:49:06.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:06 smithi018 ceph-mon[29336]: 6.c scrub ok 2024-09-10T18:49:06.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:06 smithi018 ceph-mon[29336]: pgmap v143: 148 pgs: 148 active+clean; 7.4 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 740 B/s rd, 51 MiB/s wr, 821 op/s 2024-09-10T18:49:06.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:06 smithi018 ceph-mon[29336]: 7.7 scrub ok 2024-09-10T18:49:06.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:06 smithi018 ceph-mon[29336]: 4.12 scrub starts 2024-09-10T18:49:06.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:06 smithi018 ceph-mon[29336]: 4.12 scrub ok 2024-09-10T18:49:06.495 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:06.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:49:06 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:49:06.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:06 smithi103 ceph-mon[33163]: 7.4 scrub starts 2024-09-10T18:49:06.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:06 smithi103 ceph-mon[33163]: 6.c scrub starts 2024-09-10T18:49:06.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:06 smithi103 ceph-mon[33163]: 6.c scrub ok 2024-09-10T18:49:06.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:06 smithi103 ceph-mon[33163]: pgmap v143: 148 pgs: 148 active+clean; 7.4 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 740 B/s rd, 51 MiB/s wr, 821 op/s 2024-09-10T18:49:06.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:06 smithi103 ceph-mon[33163]: 7.7 scrub ok 2024-09-10T18:49:06.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:06 smithi103 ceph-mon[33163]: 4.12 scrub starts 2024-09-10T18:49:06.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:06 smithi103 ceph-mon[33163]: 4.12 scrub ok 2024-09-10T18:49:06.534 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:06.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:06.535 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:06.636 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:06.675 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:06.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:06.676 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:06.777 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:06.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:06.816 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:06.817 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:06.917 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:06.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:06.959 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:06.959 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:07.060 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:07.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:07.101 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:07.101 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:07.202 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:07.215 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 83 15 128182 128167 96.4905 70.625 0.00932546 0.0103593 2024-09-10T18:49:07.236 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:07.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:07.237 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:07.338 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:07.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:07.382 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:07.382 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:07.483 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:07.486 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:07 smithi018 ceph-mon[40196]: 2.12 scrub starts 2024-09-10T18:49:07.487 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:07 smithi018 ceph-mon[40196]: 2.12 scrub ok 2024-09-10T18:49:07.487 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:07 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:07.487 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:07 smithi018 ceph-mon[40196]: 6.1e scrub starts 2024-09-10T18:49:07.487 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:07 smithi018 ceph-mon[40196]: 6.1e scrub ok 2024-09-10T18:49:07.487 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:07 smithi018 ceph-mon[29336]: 2.12 scrub starts 2024-09-10T18:49:07.487 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:07 smithi018 ceph-mon[29336]: 2.12 scrub ok 2024-09-10T18:49:07.487 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:07 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:07.487 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:07 smithi018 ceph-mon[29336]: 6.1e scrub starts 2024-09-10T18:49:07.487 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:07 smithi018 ceph-mon[29336]: 6.1e scrub ok 2024-09-10T18:49:07.515 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:07.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:07.516 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:07.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:07 smithi103 ceph-mon[33163]: 2.12 scrub starts 2024-09-10T18:49:07.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:07 smithi103 ceph-mon[33163]: 2.12 scrub ok 2024-09-10T18:49:07.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:07 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:07.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:07 smithi103 ceph-mon[33163]: 6.1e scrub starts 2024-09-10T18:49:07.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:07 smithi103 ceph-mon[33163]: 6.1e scrub ok 2024-09-10T18:49:07.617 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:07.644 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:07.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:07.645 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:07.746 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:07.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:07.804 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:07.804 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:07.807 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:49:07.905 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:07.952 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:07.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:07.953 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:08.054 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:08.083 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:08.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:08.085 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:08.185 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:08.215 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 84 16 129352 129336 96.2116 73.0625 0.0106306 0.0103892 2024-09-10T18:49:08.218 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:08.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:08.219 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:08.255 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is unset 2024-09-10T18:49:08.319 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:08.364 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:08.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:08.365 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:08.466 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:08.518 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:08.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:08.519 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:08.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:08 smithi103 ceph-mon[33163]: 7.4 scrub ok 2024-09-10T18:49:08.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:08 smithi103 ceph-mon[33163]: pgmap v144: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.7 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 1.2 KiB/s rd, 88 MiB/s wr, 1.41k op/s 2024-09-10T18:49:08.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:08 smithi103 ceph-mon[33163]: 2.18 scrub starts 2024-09-10T18:49:08.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:08 smithi103 ceph-mon[33163]: 2.18 scrub ok 2024-09-10T18:49:08.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:08 smithi103 ceph-mon[33163]: 2.d scrub starts 2024-09-10T18:49:08.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:08 smithi103 ceph-mon[33163]: 2.d scrub ok 2024-09-10T18:49:08.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:08 smithi103 ceph-mon[33163]: 2.0 scrub starts 2024-09-10T18:49:08.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:08 smithi103 ceph-mon[33163]: 2.0 scrub ok 2024-09-10T18:49:08.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:08 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/477341000' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:49:08.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:08 smithi018 ceph-mon[29336]: 7.4 scrub ok 2024-09-10T18:49:08.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:08 smithi018 ceph-mon[29336]: pgmap v144: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.7 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 1.2 KiB/s rd, 88 MiB/s wr, 1.41k op/s 2024-09-10T18:49:08.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:08 smithi018 ceph-mon[29336]: 2.18 scrub starts 2024-09-10T18:49:08.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:08 smithi018 ceph-mon[29336]: 2.18 scrub ok 2024-09-10T18:49:08.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:08 smithi018 ceph-mon[29336]: 2.d scrub starts 2024-09-10T18:49:08.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:08 smithi018 ceph-mon[29336]: 2.d scrub ok 2024-09-10T18:49:08.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:08 smithi018 ceph-mon[29336]: 2.0 scrub starts 2024-09-10T18:49:08.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:08 smithi018 ceph-mon[29336]: 2.0 scrub ok 2024-09-10T18:49:08.526 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:08 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/477341000' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:49:08.527 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:08 smithi018 ceph-mon[40196]: 7.4 scrub ok 2024-09-10T18:49:08.527 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:08 smithi018 ceph-mon[40196]: pgmap v144: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.7 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 1.2 KiB/s rd, 88 MiB/s wr, 1.41k op/s 2024-09-10T18:49:08.527 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:08 smithi018 ceph-mon[40196]: 2.18 scrub starts 2024-09-10T18:49:08.527 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:08 smithi018 ceph-mon[40196]: 2.18 scrub ok 2024-09-10T18:49:08.527 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:08 smithi018 ceph-mon[40196]: 2.d scrub starts 2024-09-10T18:49:08.527 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:08 smithi018 ceph-mon[40196]: 2.d scrub ok 2024-09-10T18:49:08.527 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:08 smithi018 ceph-mon[40196]: 2.0 scrub starts 2024-09-10T18:49:08.527 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:08 smithi018 ceph-mon[40196]: 2.0 scrub ok 2024-09-10T18:49:08.527 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:08 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/477341000' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:49:08.620 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:08.659 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:08.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:08.660 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:08.761 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:08.802 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:08.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:08.803 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:08.904 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:08.935 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:08.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:08.936 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:09.037 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:09.067 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:09.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:09.068 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:09.169 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:09.215 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 85 15 130517 130502 95.937 72.875 0.0201429 0.0104185 2024-09-10T18:49:09.224 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:09.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:09.225 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:09.284 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T18:49:09.326 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:09.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:09.355 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:09.355 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:09.419 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:09 smithi018 ceph-mon[29336]: 3.1 scrub starts 2024-09-10T18:49:09.419 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:09 smithi018 ceph-mon[29336]: 2.19 scrub starts 2024-09-10T18:49:09.419 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:09 smithi018 ceph-mon[29336]: 3.1 scrub ok 2024-09-10T18:49:09.419 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:09 smithi018 ceph-mon[29336]: 2.19 scrub ok 2024-09-10T18:49:09.419 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:09 smithi018 ceph-mon[29336]: 2.c scrub starts 2024-09-10T18:49:09.419 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:09 smithi018 ceph-mon[29336]: 2.c scrub ok 2024-09-10T18:49:09.420 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:09 smithi018 ceph-mon[29336]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:49:09.420 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:09 smithi018 ceph-mon[29336]: Cluster is now healthy 2024-09-10T18:49:09.420 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:09 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/477341000' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:49:09.420 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:09 smithi018 ceph-mon[29336]: osdmap e116: 8 total, 8 up, 7 in 2024-09-10T18:49:09.421 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:09 smithi018 ceph-mon[40196]: 3.1 scrub starts 2024-09-10T18:49:09.421 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:09 smithi018 ceph-mon[40196]: 2.19 scrub starts 2024-09-10T18:49:09.421 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:09 smithi018 ceph-mon[40196]: 3.1 scrub ok 2024-09-10T18:49:09.421 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:09 smithi018 ceph-mon[40196]: 2.19 scrub ok 2024-09-10T18:49:09.421 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:09 smithi018 ceph-mon[40196]: 2.c scrub starts 2024-09-10T18:49:09.421 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:09 smithi018 ceph-mon[40196]: 2.c scrub ok 2024-09-10T18:49:09.422 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:09 smithi018 ceph-mon[40196]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:49:09.422 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:09 smithi018 ceph-mon[40196]: Cluster is now healthy 2024-09-10T18:49:09.422 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:09 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/477341000' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:49:09.422 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:09 smithi018 ceph-mon[40196]: osdmap e116: 8 total, 8 up, 7 in 2024-09-10T18:49:09.456 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:09.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:09.498 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:09.498 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:09.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:09 smithi103 ceph-mon[33163]: 3.1 scrub starts 2024-09-10T18:49:09.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:09 smithi103 ceph-mon[33163]: 2.19 scrub starts 2024-09-10T18:49:09.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:09 smithi103 ceph-mon[33163]: 3.1 scrub ok 2024-09-10T18:49:09.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:09 smithi103 ceph-mon[33163]: 2.19 scrub ok 2024-09-10T18:49:09.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:09 smithi103 ceph-mon[33163]: 2.c scrub starts 2024-09-10T18:49:09.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:09 smithi103 ceph-mon[33163]: 2.c scrub ok 2024-09-10T18:49:09.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:09 smithi103 ceph-mon[33163]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:49:09.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:09 smithi103 ceph-mon[33163]: Cluster is now healthy 2024-09-10T18:49:09.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:09 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/477341000' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:49:09.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:09 smithi103 ceph-mon[33163]: osdmap e116: 8 total, 8 up, 7 in 2024-09-10T18:49:09.598 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:09.668 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:09.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:09.669 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:09.733 INFO:teuthology.orchestra.run.smithi018.stdout:ERROR: (22) Invalid argument 2024-09-10T18:49:09.733 INFO:teuthology.orchestra.run.smithi018.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:49:09.770 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:09.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:09.812 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:09.813 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:09.913 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:09.947 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:09.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:09.948 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:10.049 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:10.090 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:10.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:10.091 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:10.192 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:10.216 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 86 16 131661 131645 95.651 71.4375 0.0144067 0.0104502 2024-09-10T18:49:10.237 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:10.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:10.238 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:10.339 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:10.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:10.379 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:10.379 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:10.433 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-10T18:49:10.480 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:10.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:49:10 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:49:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:49:10.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:10 smithi103 ceph-mon[33163]: pgmap v146: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.7 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 745 B/s rd, 51 MiB/s wr, 809 op/s 2024-09-10T18:49:10.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:10 smithi103 ceph-mon[33163]: 2.4 scrub starts 2024-09-10T18:49:10.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:10 smithi103 ceph-mon[33163]: 2.4 scrub ok 2024-09-10T18:49:10.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:10 smithi103 ceph-mon[33163]: 7.d scrub starts 2024-09-10T18:49:10.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:10 smithi103 ceph-mon[33163]: 1.0 scrub starts 2024-09-10T18:49:10.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:10 smithi103 ceph-mon[33163]: 1.0 scrub ok 2024-09-10T18:49:10.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:10 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:49:10.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:10 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:49:10.528 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:10.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:10.530 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:10.598 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:10 smithi018 ceph-mon[29336]: pgmap v146: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.7 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 745 B/s rd, 51 MiB/s wr, 809 op/s 2024-09-10T18:49:10.598 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:10 smithi018 ceph-mon[29336]: 2.4 scrub starts 2024-09-10T18:49:10.598 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:10 smithi018 ceph-mon[29336]: 2.4 scrub ok 2024-09-10T18:49:10.598 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:10 smithi018 ceph-mon[29336]: 7.d scrub starts 2024-09-10T18:49:10.598 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:10 smithi018 ceph-mon[29336]: 1.0 scrub starts 2024-09-10T18:49:10.598 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:10 smithi018 ceph-mon[29336]: 1.0 scrub ok 2024-09-10T18:49:10.599 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:10 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:49:10.599 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:10 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:49:10.599 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:10 smithi018 ceph-mon[40196]: pgmap v146: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.7 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 745 B/s rd, 51 MiB/s wr, 809 op/s 2024-09-10T18:49:10.599 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:10 smithi018 ceph-mon[40196]: 2.4 scrub starts 2024-09-10T18:49:10.599 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:10 smithi018 ceph-mon[40196]: 2.4 scrub ok 2024-09-10T18:49:10.599 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:10 smithi018 ceph-mon[40196]: 7.d scrub starts 2024-09-10T18:49:10.599 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:10 smithi018 ceph-mon[40196]: 1.0 scrub starts 2024-09-10T18:49:10.599 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:10 smithi018 ceph-mon[40196]: 1.0 scrub ok 2024-09-10T18:49:10.599 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:10 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:49:10.599 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:10 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:49:10.630 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:10.658 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:10.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:10.659 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:10.760 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:10.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:10.806 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:10.807 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:10.907 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:10.944 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:10.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:10.945 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:11.046 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:11.079 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:11.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:11.080 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:11.181 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:11.218 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 87 16 132882 132866 95.4266 76.3125 0.0206112 0.010475 2024-09-10T18:49:11.227 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:11.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:11.228 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:11.329 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:11.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:11.358 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:11.358 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:11.422 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-10T18:49:11.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:49:10 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:49:10] "GET /metrics HTTP/1.1" 200 223623 "" "Prometheus/2.43.0" 2024-09-10T18:49:11.460 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:11.494 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:49:11.533 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:11.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:11.534 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:11.576 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:49:11.635 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:11.667 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:11.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:11.668 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:11.769 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:11.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:11.799 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:11.799 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:11.900 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:11.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:11.934 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:11.934 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:12.035 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:12.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:12.068 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:12.068 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:12.169 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:12.206 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:12.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:12.207 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:12.218 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 88 16 133967 133951 95.1128 67.8125 0.0102548 0.0105096 2024-09-10T18:49:12.308 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:12.345 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:12.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:12.346 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:12.447 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:12.490 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:12.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:12.491 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:12.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:12 smithi103 ceph-mon[33163]: 5.7 scrub starts 2024-09-10T18:49:12.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:12 smithi103 ceph-mon[33163]: 5.7 scrub ok 2024-09-10T18:49:12.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:12 smithi103 ceph-mon[33163]: pgmap v147: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 7.9 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 639 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-10T18:49:12.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:12 smithi103 ceph-mon[33163]: 6.1d scrub starts 2024-09-10T18:49:12.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:12 smithi103 ceph-mon[33163]: 6.1d scrub ok 2024-09-10T18:49:12.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:12 smithi103 ceph-mon[33163]: 7.c scrub starts 2024-09-10T18:49:12.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:12 smithi103 ceph-mon[33163]: 6.8 scrub starts 2024-09-10T18:49:12.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:12 smithi103 ceph-mon[33163]: 6.8 scrub ok 2024-09-10T18:49:12.592 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:12.635 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:12.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:12.637 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:12.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:12 smithi018 ceph-mon[40196]: 5.7 scrub starts 2024-09-10T18:49:12.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:12 smithi018 ceph-mon[40196]: 5.7 scrub ok 2024-09-10T18:49:12.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:12 smithi018 ceph-mon[40196]: pgmap v147: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 7.9 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 639 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-10T18:49:12.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:12 smithi018 ceph-mon[40196]: 6.1d scrub starts 2024-09-10T18:49:12.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:12 smithi018 ceph-mon[40196]: 6.1d scrub ok 2024-09-10T18:49:12.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:12 smithi018 ceph-mon[40196]: 7.c scrub starts 2024-09-10T18:49:12.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:12 smithi018 ceph-mon[40196]: 6.8 scrub starts 2024-09-10T18:49:12.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:12 smithi018 ceph-mon[40196]: 6.8 scrub ok 2024-09-10T18:49:12.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:12 smithi018 ceph-mon[29336]: 5.7 scrub starts 2024-09-10T18:49:12.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:12 smithi018 ceph-mon[29336]: 5.7 scrub ok 2024-09-10T18:49:12.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:12 smithi018 ceph-mon[29336]: pgmap v147: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 7.9 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 639 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-10T18:49:12.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:12 smithi018 ceph-mon[29336]: 6.1d scrub starts 2024-09-10T18:49:12.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:12 smithi018 ceph-mon[29336]: 6.1d scrub ok 2024-09-10T18:49:12.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:12 smithi018 ceph-mon[29336]: 7.c scrub starts 2024-09-10T18:49:12.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:12 smithi018 ceph-mon[29336]: 6.8 scrub starts 2024-09-10T18:49:12.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:12 smithi018 ceph-mon[29336]: 6.8 scrub ok 2024-09-10T18:49:12.738 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:12.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:12.773 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:12.773 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:12.874 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:12.914 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:12.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:12.915 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:13.016 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:13.047 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:13.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:13.048 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:13.149 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:13.181 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:13.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:13.182 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:13.218 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 89 16 135195 135179 94.9064 76.75 0.0052383 0.0105318 2024-09-10T18:49:13.283 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:13.305 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:13.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:13.306 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:13.407 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:13.451 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:13.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:13.452 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:13.499 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:13 smithi018 ceph-mon[40196]: 6.f scrub starts 2024-09-10T18:49:13.499 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:13 smithi018 ceph-mon[40196]: 6.f scrub ok 2024-09-10T18:49:13.499 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:13 smithi018 ceph-mon[40196]: 4.c deep-scrub starts 2024-09-10T18:49:13.499 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:13 smithi018 ceph-mon[40196]: 4.c deep-scrub ok 2024-09-10T18:49:13.499 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:13 smithi018 ceph-mon[40196]: 6.5 scrub starts 2024-09-10T18:49:13.499 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:13 smithi018 ceph-mon[40196]: 6.5 scrub ok 2024-09-10T18:49:13.499 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:13 smithi018 ceph-mon[40196]: 7.d scrub ok 2024-09-10T18:49:13.499 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:13 smithi018 ceph-mon[29336]: 6.f scrub starts 2024-09-10T18:49:13.499 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:13 smithi018 ceph-mon[29336]: 6.f scrub ok 2024-09-10T18:49:13.500 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:13 smithi018 ceph-mon[29336]: 4.c deep-scrub starts 2024-09-10T18:49:13.500 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:13 smithi018 ceph-mon[29336]: 4.c deep-scrub ok 2024-09-10T18:49:13.500 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:13 smithi018 ceph-mon[29336]: 6.5 scrub starts 2024-09-10T18:49:13.500 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:13 smithi018 ceph-mon[29336]: 6.5 scrub ok 2024-09-10T18:49:13.500 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:13 smithi018 ceph-mon[29336]: 7.d scrub ok 2024-09-10T18:49:13.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:13 smithi103 ceph-mon[33163]: 6.f scrub starts 2024-09-10T18:49:13.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:13 smithi103 ceph-mon[33163]: 6.f scrub ok 2024-09-10T18:49:13.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:13 smithi103 ceph-mon[33163]: 4.c deep-scrub starts 2024-09-10T18:49:13.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:13 smithi103 ceph-mon[33163]: 4.c deep-scrub ok 2024-09-10T18:49:13.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:13 smithi103 ceph-mon[33163]: 6.5 scrub starts 2024-09-10T18:49:13.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:13 smithi103 ceph-mon[33163]: 6.5 scrub ok 2024-09-10T18:49:13.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:13 smithi103 ceph-mon[33163]: 7.d scrub ok 2024-09-10T18:49:13.553 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:13.593 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:13.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:13.594 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:13.695 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:13.717 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:13.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:13.718 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:13.819 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:13.856 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:13.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:13.857 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:13.958 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:13.999 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:14.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:14.001 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:14.102 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:14.132 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:14.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:14.133 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:14.218 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Total time run: 90.0082 2024-09-10T18:49:14.219 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Total writes made: 136217 2024-09-10T18:49:14.219 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Write size: 65536 2024-09-10T18:49:14.219 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Object size: 65536 2024-09-10T18:49:14.219 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Bandwidth (MB/sec): 94.5865 2024-09-10T18:49:14.219 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Stddev Bandwidth: 33.6604 2024-09-10T18:49:14.219 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Max bandwidth (MB/sec): 228.875 2024-09-10T18:49:14.219 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Min bandwidth (MB/sec): 57.3125 2024-09-10T18:49:14.219 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Average IOPS: 1513 2024-09-10T18:49:14.219 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Stddev IOPS: 538.567 2024-09-10T18:49:14.219 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Max IOPS: 3662 2024-09-10T18:49:14.219 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Min IOPS: 917 2024-09-10T18:49:14.219 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Average Latency(s): 0.0105682 2024-09-10T18:49:14.219 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Stddev Latency(s): 0.00746017 2024-09-10T18:49:14.219 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Max latency(s): 0.14754 2024-09-10T18:49:14.220 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Min latency(s): 0.00126001 2024-09-10T18:49:14.224 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Cleaning up (deleting benchmark objects) 2024-09-10T18:49:14.234 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:14.264 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:14.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:14.265 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:14.366 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:14.394 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:14.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:14.395 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:14.496 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:14.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:14.523 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:14.523 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:14.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:14 smithi103 ceph-mon[33163]: 5.16 scrub starts 2024-09-10T18:49:14.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:14 smithi103 ceph-mon[33163]: 5.16 scrub ok 2024-09-10T18:49:14.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:14 smithi103 ceph-mon[33163]: pgmap v148: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 8.1 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 1.0 KiB/s rd, 72 MiB/s wr, 1.15k op/s 2024-09-10T18:49:14.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:14 smithi103 ceph-mon[33163]: 4.7 scrub starts 2024-09-10T18:49:14.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:14 smithi103 ceph-mon[33163]: 4.7 scrub ok 2024-09-10T18:49:14.624 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:14.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:14.664 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:14.664 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:14.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:14 smithi018 ceph-mon[40196]: 5.16 scrub starts 2024-09-10T18:49:14.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:14 smithi018 ceph-mon[40196]: 5.16 scrub ok 2024-09-10T18:49:14.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:14 smithi018 ceph-mon[40196]: pgmap v148: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 8.1 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 1.0 KiB/s rd, 72 MiB/s wr, 1.15k op/s 2024-09-10T18:49:14.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:14 smithi018 ceph-mon[40196]: 4.7 scrub starts 2024-09-10T18:49:14.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:14 smithi018 ceph-mon[40196]: 4.7 scrub ok 2024-09-10T18:49:14.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:14 smithi018 ceph-mon[29336]: 5.16 scrub starts 2024-09-10T18:49:14.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:14 smithi018 ceph-mon[29336]: 5.16 scrub ok 2024-09-10T18:49:14.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:14 smithi018 ceph-mon[29336]: pgmap v148: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 8.1 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 1.0 KiB/s rd, 72 MiB/s wr, 1.15k op/s 2024-09-10T18:49:14.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:14 smithi018 ceph-mon[29336]: 4.7 scrub starts 2024-09-10T18:49:14.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:14 smithi018 ceph-mon[29336]: 4.7 scrub ok 2024-09-10T18:49:14.764 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:14.816 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:14.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:14.817 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:14.918 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:14.947 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:14.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:14.948 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:15.049 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:15.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:15.092 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:15.093 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:15.193 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:15.232 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:15.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:15.234 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:15.334 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:15.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:15.364 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:15.365 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:15.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:15 smithi018 ceph-mon[40196]: 4.1d scrub starts 2024-09-10T18:49:15.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:15 smithi018 ceph-mon[40196]: 4.1d scrub ok 2024-09-10T18:49:15.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:15 smithi018 ceph-mon[40196]: 7.f scrub starts 2024-09-10T18:49:15.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:15 smithi018 ceph-mon[40196]: 7.c scrub ok 2024-09-10T18:49:15.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:15 smithi018 ceph-mon[40196]: 5.4 scrub starts 2024-09-10T18:49:15.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:15 smithi018 ceph-mon[40196]: 5.4 scrub ok 2024-09-10T18:49:15.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:15 smithi018 ceph-mon[29336]: 4.1d scrub starts 2024-09-10T18:49:15.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:15 smithi018 ceph-mon[29336]: 4.1d scrub ok 2024-09-10T18:49:15.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:15 smithi018 ceph-mon[29336]: 7.f scrub starts 2024-09-10T18:49:15.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:15 smithi018 ceph-mon[29336]: 7.c scrub ok 2024-09-10T18:49:15.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:15 smithi018 ceph-mon[29336]: 5.4 scrub starts 2024-09-10T18:49:15.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:15 smithi018 ceph-mon[29336]: 5.4 scrub ok 2024-09-10T18:49:15.466 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:15.504 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:15.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:15.505 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:15.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:15 smithi103 ceph-mon[33163]: 4.1d scrub starts 2024-09-10T18:49:15.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:15 smithi103 ceph-mon[33163]: 4.1d scrub ok 2024-09-10T18:49:15.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:15 smithi103 ceph-mon[33163]: 7.f scrub starts 2024-09-10T18:49:15.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:15 smithi103 ceph-mon[33163]: 7.c scrub ok 2024-09-10T18:49:15.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:15 smithi103 ceph-mon[33163]: 5.4 scrub starts 2024-09-10T18:49:15.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:15 smithi103 ceph-mon[33163]: 5.4 scrub ok 2024-09-10T18:49:15.606 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:15.645 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:15.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:15.646 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:15.747 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:15.773 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:15.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:15.774 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:15.875 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:15.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:15.897 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:15.898 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:15.998 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:16.026 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:16.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:16.027 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:16.050 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:49:16.054 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-09-10T18:49:16.128 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:16.170 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:16.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:16.171 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:16.272 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:16.300 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:16.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:16.301 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:16.402 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:16.445 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:16.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:16.447 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:16.451 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:16 smithi018 ceph-mon[40196]: 2.1b scrub starts 2024-09-10T18:49:16.451 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:16 smithi018 ceph-mon[40196]: 2.1b scrub ok 2024-09-10T18:49:16.451 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:16 smithi018 ceph-mon[40196]: pgmap v149: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 8.1 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 1023 B/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-10T18:49:16.451 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:16 smithi018 ceph-mon[29336]: 2.1b scrub starts 2024-09-10T18:49:16.451 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:16 smithi018 ceph-mon[29336]: 2.1b scrub ok 2024-09-10T18:49:16.451 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:16 smithi018 ceph-mon[29336]: pgmap v149: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 8.1 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 1023 B/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-10T18:49:16.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:49:16 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:49:16.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:16 smithi103 ceph-mon[33163]: 2.1b scrub starts 2024-09-10T18:49:16.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:16 smithi103 ceph-mon[33163]: 2.1b scrub ok 2024-09-10T18:49:16.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:16 smithi103 ceph-mon[33163]: pgmap v149: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 8.1 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 1023 B/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-10T18:49:16.548 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:16.576 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:16.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:16.577 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:16.678 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:16.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:16.710 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:16.710 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:16.811 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:16.852 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:16.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:16.853 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:16.954 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:16.994 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:16.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:16.995 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:17.096 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:17.128 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:17.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:17.130 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:17.231 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:17.267 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:17.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:17.268 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:17.368 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:17.409 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:17.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:17.411 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:17.411 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:17 smithi018 ceph-mon[40196]: from='client.24718 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:49:17.411 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:17 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:17.411 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:17 smithi018 ceph-mon[40196]: 7.5 scrub starts 2024-09-10T18:49:17.411 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:17 smithi018 ceph-mon[29336]: from='client.24718 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:49:17.411 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:17 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:17.411 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:17 smithi018 ceph-mon[29336]: 7.5 scrub starts 2024-09-10T18:49:17.511 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:17.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:17.549 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:17.549 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:17.650 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:17.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:17.694 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:17.695 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:17.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:17 smithi103 ceph-mon[33163]: from='client.24718 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:49:17.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:17 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:17.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:17 smithi103 ceph-mon[33163]: 7.5 scrub starts 2024-09-10T18:49:17.796 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:17.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:17.850 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:17.850 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:17.951 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:17.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:17.996 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:17.996 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:18.097 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:18.127 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:18.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:18.128 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:18.228 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:18.260 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:49:18.273 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:18.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:18.274 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:18.291 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":149,"stamp":"2024-09-10T18:49:15.080672+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8698796252,"num_objects":132920,"num_object_clones":0,"num_object_copies":398760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":132920,"num_whiteouts":0,"num_read":1075,"num_read_kb":932,"num_write":133267,"num_write_kb":8496007,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":1183107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77512,"ondisk_log_size":77512,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":27490748,"kb_used_data":25355200,"kb_used_omap":11,"kb_used_meta":2135476,"kb_avail":628725316,"statfs":{"total":671965249536,"available":643814723584,"internally_reserved":0,"allocated":25963724800,"data_stored":25960755232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11928,"internal_metadata":2186727784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":745144320,"num_objects":11370,"num_object_clones":0,"num_object_copies":34110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11370,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":11370,"num_write_kb":727680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003175"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":83,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.219636+0000","last_change":"2024-09-10T18:48:24.515065+0000","last_active":"2024-09-10T18:49:03.219636+0000","last_peered":"2024-09-10T18:49:03.219636+0000","last_clean":"2024-09-10T18:49:03.219636+0000","last_became_active":"2024-09-10T18:45:24.265488+0000","last_became_peered":"2024-09-10T18:45:24.265488+0000","last_unstale":"2024-09-10T18:49:03.219636+0000","last_undegraded":"2024-09-10T18:49:03.219636+0000","last_fullsized":"2024-09-10T18:49:03.219636+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:24.511946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:24.511946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:49.630089+0000","scrub_duration":0.014161415,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":89,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.218569+0000","last_change":"2024-09-10T18:48:29.718356+0000","last_active":"2024-09-10T18:49:03.218569+0000","last_peered":"2024-09-10T18:49:03.218569+0000","last_clean":"2024-09-10T18:49:03.218569+0000","last_became_active":"2024-09-10T18:45:17.959475+0000","last_became_peered":"2024-09-10T18:45:17.959475+0000","last_unstale":"2024-09-10T18:49:03.218569+0000","last_undegraded":"2024-09-10T18:49:03.218569+0000","last_fullsized":"2024-09-10T18:49:03.218569+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:29.718270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.718270+0000","last_clean_scrub_stamp":"2024-09-10T18:48:29.718270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:59.632977+0000","scrub_duration":0.010834984000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"63'15","reported_seq":138,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.219344+0000","last_change":"2024-09-10T18:49:02.544635+0000","last_active":"2024-09-10T18:49:03.219344+0000","last_peered":"2024-09-10T18:49:03.219344+0000","last_clean":"2024-09-10T18:49:03.219344+0000","last_became_active":"2024-09-10T18:47:36.413635+0000","last_became_peered":"2024-09-10T18:47:36.413635+0000","last_unstale":"2024-09-10T18:49:03.219344+0000","last_undegraded":"2024-09-10T18:49:03.219344+0000","last_fullsized":"2024-09-10T18:49:03.219344+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:49:02.544557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:49:02.544557+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:20.745798+0000","scrub_duration":0.0033455720000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":84,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.210656+0000","last_change":"2024-09-10T18:48:29.058001+0000","last_active":"2024-09-10T18:49:03.210656+0000","last_peered":"2024-09-10T18:49:03.210656+0000","last_clean":"2024-09-10T18:49:03.210656+0000","last_became_active":"2024-09-10T18:45:22.065619+0000","last_became_peered":"2024-09-10T18:45:22.065619+0000","last_unstale":"2024-09-10T18:49:03.210656+0000","last_undegraded":"2024-09-10T18:49:03.210656+0000","last_fullsized":"2024-09-10T18:49:03.210656+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:29.057932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.057932+0000","last_clean_scrub_stamp":"2024-09-10T18:48:29.057932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:39.110015+0000","scrub_duration":0.001460184,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":89,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.221372+0000","last_change":"2024-09-10T18:48:29.458396+0000","last_active":"2024-09-10T18:49:03.221372+0000","last_peered":"2024-09-10T18:49:03.221372+0000","last_clean":"2024-09-10T18:49:03.221372+0000","last_became_active":"2024-09-10T18:45:17.966153+0000","last_became_peered":"2024-09-10T18:45:17.966153+0000","last_unstale":"2024-09-10T18:49:03.221372+0000","last_undegraded":"2024-09-10T18:49:03.221372+0000","last_fullsized":"2024-09-10T18:49:03.221372+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:29.457892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:29.457892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:50.349265+0000","scrub_duration":0.012091342999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"83'13","reported_seq":96,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.210809+0000","last_change":"2024-09-10T18:46:59.906707+0000","last_active":"2024-09-10T18:49:03.210809+0000","last_peered":"2024-09-10T18:49:03.210809+0000","last_clean":"2024-09-10T18:49:03.210809+0000","last_became_active":"2024-09-10T18:45:20.061266+0000","last_became_peered":"2024-09-10T18:45:20.061266+0000","last_unstale":"2024-09-10T18:49:03.210809+0000","last_undegraded":"2024-09-10T18:49:03.210809+0000","last_fullsized":"2024-09-10T18:49:03.210809+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'13","last_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_clean_scrub_stamp":"2024-09-10T18:46:59.906634+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003988892,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":75,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.221607+0000","last_change":"2024-09-10T18:47:21.589428+0000","last_active":"2024-09-10T18:49:03.221607+0000","last_peered":"2024-09-10T18:49:03.221607+0000","last_clean":"2024-09-10T18:49:03.221607+0000","last_became_active":"2024-09-10T18:45:22.072822+0000","last_became_peered":"2024-09-10T18:45:22.072822+0000","last_unstale":"2024-09-10T18:49:03.221607+0000","last_undegraded":"2024-09-10T18:49:03.221607+0000","last_fullsized":"2024-09-10T18:49:03.221607+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:21.589365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:47:21.589365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001676893,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":72,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.210744+0000","last_change":"2024-09-10T18:47:00.946309+0000","last_active":"2024-09-10T18:49:03.210744+0000","last_peered":"2024-09-10T18:49:03.210744+0000","last_clean":"2024-09-10T18:49:03.210744+0000","last_became_active":"2024-09-10T18:45:24.081002+0000","last_became_peered":"2024-09-10T18:45:24.081002+0000","last_unstale":"2024-09-10T18:49:03.210744+0000","last_undegraded":"2024-09-10T18:49:03.210744+0000","last_fullsized":"2024-09-10T18:49:03.210744+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:00.946255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:00.946255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001207629,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":89,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.214572+0000","last_change":"2024-09-10T18:48:13.694567+0000","last_active":"2024-09-10T18:49:03.214572+0000","last_peered":"2024-09-10T18:49:03.214572+0000","last_clean":"2024-09-10T18:49:03.214572+0000","last_became_active":"2024-09-10T18:45:17.961097+0000","last_became_peered":"2024-09-10T18:45:17.961097+0000","last_unstale":"2024-09-10T18:49:03.214572+0000","last_undegraded":"2024-09-10T18:49:03.214572+0000","last_fullsized":"2024-09-10T18:49:03.214572+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:13.694295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:13.694295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:28.048348+0000","scrub_duration":0.0051419839999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"63'5","reported_seq":86,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.211024+0000","last_change":"2024-09-10T18:47:04.002228+0000","last_active":"2024-09-10T18:49:03.211024+0000","last_peered":"2024-09-10T18:49:03.211024+0000","last_clean":"2024-09-10T18:49:03.211024+0000","last_became_active":"2024-09-10T18:45:20.057567+0000","last_became_peered":"2024-09-10T18:45:20.057567+0000","last_unstale":"2024-09-10T18:49:03.211024+0000","last_undegraded":"2024-09-10T18:49:03.211024+0000","last_fullsized":"2024-09-10T18:49:03.211024+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-10T18:47:04.002164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:04.002164+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.026374999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":85,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.214001+0000","last_change":"2024-09-10T18:47:36.510027+0000","last_active":"2024-09-10T18:49:03.214001+0000","last_peered":"2024-09-10T18:49:03.214001+0000","last_clean":"2024-09-10T18:49:03.214001+0000","last_became_active":"2024-09-10T18:45:22.076575+0000","last_became_peered":"2024-09-10T18:45:22.076575+0000","last_unstale":"2024-09-10T18:49:03.214001+0000","last_undegraded":"2024-09-10T18:49:03.214001+0000","last_fullsized":"2024-09-10T18:49:03.214001+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:36.509971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:36.509971+0000","last_clean_scrub_stamp":"2024-09-10T18:47:36.509971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013665439999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":82,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.220727+0000","last_change":"2024-09-10T18:48:10.102991+0000","last_active":"2024-09-10T18:49:03.220727+0000","last_peered":"2024-09-10T18:49:03.220727+0000","last_clean":"2024-09-10T18:49:03.220727+0000","last_became_active":"2024-09-10T18:45:24.079534+0000","last_became_peered":"2024-09-10T18:45:24.079534+0000","last_unstale":"2024-09-10T18:49:03.220727+0000","last_undegraded":"2024-09-10T18:49:03.220727+0000","last_fullsized":"2024-09-10T18:49:03.220727+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:10.102865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:10.102865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:36.015071+0000","scrub_duration":0.0042821509999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":99,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.214800+0000","last_change":"2024-09-10T18:48:59.577187+0000","last_active":"2024-09-10T18:49:03.214800+0000","last_peered":"2024-09-10T18:49:03.214800+0000","last_clean":"2024-09-10T18:49:03.214800+0000","last_became_active":"2024-09-10T18:47:36.414393+0000","last_became_peered":"2024-09-10T18:47:36.414393+0000","last_unstale":"2024-09-10T18:49:03.214800+0000","last_undegraded":"2024-09-10T18:49:03.214800+0000","last_fullsized":"2024-09-10T18:49:03.214800+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:59.576980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:59.576980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:16.200779+0000","scrub_duration":0.0056458419999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"63'9","reported_seq":92,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.210937+0000","last_change":"2024-09-10T18:46:55.069893+0000","last_active":"2024-09-10T18:49:03.210937+0000","last_peered":"2024-09-10T18:49:03.210937+0000","last_clean":"2024-09-10T18:49:03.210937+0000","last_became_active":"2024-09-10T18:45:20.052652+0000","last_became_peered":"2024-09-10T18:45:20.052652+0000","last_unstale":"2024-09-10T18:49:03.210937+0000","last_undegraded":"2024-09-10T18:49:03.210937+0000","last_fullsized":"2024-09-10T18:49:03.210937+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:46:55.069833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:55.069833+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016685510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":84,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.220720+0000","last_change":"2024-09-10T18:48:32.141849+0000","last_active":"2024-09-10T18:49:03.220720+0000","last_peered":"2024-09-10T18:49:03.220720+0000","last_clean":"2024-09-10T18:49:03.220720+0000","last_became_active":"2024-09-10T18:45:22.068326+0000","last_became_peered":"2024-09-10T18:45:22.068326+0000","last_unstale":"2024-09-10T18:49:03.220720+0000","last_undegraded":"2024-09-10T18:49:03.220720+0000","last_fullsized":"2024-09-10T18:49:03.220720+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:32.141755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:32.141755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:42.742732+0000","scrub_duration":0.0040319509999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":83,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.213637+0000","last_change":"2024-09-10T18:49:02.511550+0000","last_active":"2024-09-10T18:49:03.213637+0000","last_peered":"2024-09-10T18:49:03.213637+0000","last_clean":"2024-09-10T18:49:03.213637+0000","last_became_active":"2024-09-10T18:45:24.081488+0000","last_became_peered":"2024-09-10T18:45:24.081488+0000","last_unstale":"2024-09-10T18:49:03.213637+0000","last_undegraded":"2024-09-10T18:49:03.213637+0000","last_fullsized":"2024-09-10T18:49:03.213637+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:02.511457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:02.511457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:18.180849+0000","scrub_duration":0.018959785999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"116'8261","reported_seq":8300,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T18:49:10.985005+0000","last_change":"2024-09-10T18:47:36.423039+0000","last_active":"2024-09-10T18:49:10.985005+0000","last_peered":"2024-09-10T18:49:10.985005+0000","last_clean":"2024-09-10T18:49:10.985005+0000","last_became_active":"2024-09-10T18:47:36.422694+0000","last_became_peered":"2024-09-10T18:47:36.422694+0000","last_unstale":"2024-09-10T18:49:10.985005+0000","last_undegraded":"2024-09-10T18:49:10.985005+0000","last_fullsized":"2024-09-10T18:49:10.985005+0000","mapping_epoch":88,"log_start":"99'3000","ondisk_log_start":"99'3000","created":86,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":5261,"log_dups_size":0,"ondisk_log_size":5261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":541392896,"num_objects":8261,"num_object_clones":0,"num_object_copies":24783,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8261,"num_write_kb":528704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"78'25","reported_seq":150,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.371657+0000","last_change":"2024-09-10T18:49:03.371577+0000","last_active":"2024-09-10T18:49:03.371657+0000","last_peered":"2024-09-10T18:49:03.371657+0000","last_clean":"2024-09-10T18:49:03.371657+0000","last_became_active":"2024-09-10T18:47:36.409300+0000","last_became_peered":"2024-09-10T18:47:36.409300+0000","last_unstale":"2024-09-10T18:49:03.371657+0000","last_undegraded":"2024-09-10T18:49:03.371657+0000","last_fullsized":"2024-09-10T18:49:03.371657+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'25","last_scrub_stamp":"2024-09-10T18:49:03.371486+0000","last_deep_scrub":"78'25","last_deep_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_clean_scrub_stamp":"2024-09-10T18:49:03.371486+0000","objects_scrubbed":8,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:04.078541+0000","scrub_duration":0.003278258,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":89,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.214511+0000","last_change":"2024-09-10T18:48:26.625704+0000","last_active":"2024-09-10T18:49:03.214511+0000","last_peered":"2024-09-10T18:49:03.214511+0000","last_clean":"2024-09-10T18:49:03.214511+0000","last_became_active":"2024-09-10T18:45:17.963732+0000","last_became_peered":"2024-09-10T18:45:17.963732+0000","last_unstale":"2024-09-10T18:49:03.214511+0000","last_undegraded":"2024-09-10T18:49:03.214511+0000","last_fullsized":"2024-09-10T18:49:03.214511+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:26.625635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:26.625635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:44.826250+0000","scrub_duration":0.0034872639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":75,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.221118+0000","last_change":"2024-09-10T18:47:04.512478+0000","last_active":"2024-09-10T18:49:03.221118+0000","last_peered":"2024-09-10T18:49:03.221118+0000","last_clean":"2024-09-10T18:49:03.221118+0000","last_became_active":"2024-09-10T18:45:22.071443+0000","last_became_peered":"2024-09-10T18:45:22.071443+0000","last_unstale":"2024-09-10T18:49:03.221118+0000","last_undegraded":"2024-09-10T18:49:03.221118+0000","last_fullsized":"2024-09-10T18:49:03.221118+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:04.512425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:04.512425+0000","last_clean_scrub_stamp":"2024-09-10T18:47:04.512425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001363324,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":29,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.218178+0000","last_change":"2024-09-10T18:47:57.104885+0000","last_active":"2024-09-10T18:49:03.218178+0000","last_peered":"2024-09-10T18:49:03.218178+0000","last_clean":"2024-09-10T18:49:03.218178+0000","last_became_active":"2024-09-10T18:47:57.104433+0000","last_became_peered":"2024-09-10T18:47:57.104433+0000","last_unstale":"2024-09-10T18:49:03.218178+0000","last_undegraded":"2024-09-10T18:49:03.218178+0000","last_fullsized":"2024-09-10T18:49:03.218178+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:38.016595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:38.016595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015035510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.f","version":"116'8237","reported_seq":8275,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T18:49:11.079003+0000","last_change":"2024-09-10T18:47:34.396342+0000","last_active":"2024-09-10T18:49:11.079003+0000","last_peered":"2024-09-10T18:49:11.079003+0000","last_clean":"2024-09-10T18:49:11.079003+0000","last_became_active":"2024-09-10T18:47:34.396087+0000","last_became_peered":"2024-09-10T18:47:34.396087+0000","last_unstale":"2024-09-10T18:49:11.079003+0000","last_undegraded":"2024-09-10T18:49:11.079003+0000","last_fullsized":"2024-09-10T18:49:11.079003+0000","mapping_epoch":86,"log_start":"101'3400","ondisk_log_start":"101'3400","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":4837,"log_dups_size":0,"ondisk_log_size":4837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":539820032,"num_objects":8237,"num_object_clones":0,"num_object_copies":24711,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8237,"num_write_kb":527168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"63'10","reported_seq":91,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.211330+0000","last_change":"2024-09-10T18:46:31.281588+0000","last_active":"2024-09-10T18:49:03.211330+0000","last_peered":"2024-09-10T18:49:03.211330+0000","last_clean":"2024-09-10T18:49:03.211330+0000","last_became_active":"2024-09-10T18:45:20.053428+0000","last_became_peered":"2024-09-10T18:45:20.053428+0000","last_unstale":"2024-09-10T18:49:03.211330+0000","last_undegraded":"2024-09-10T18:49:03.211330+0000","last_fullsized":"2024-09-10T18:49:03.211330+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:46:31.281519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:31.281519+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019095119999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":89,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.220979+0000","last_change":"2024-09-10T18:48:29.551161+0000","last_active":"2024-09-10T18:49:03.220979+0000","last_peered":"2024-09-10T18:49:03.220979+0000","last_clean":"2024-09-10T18:49:03.220979+0000","last_became_active":"2024-09-10T18:45:17.966833+0000","last_became_peered":"2024-09-10T18:45:17.966833+0000","last_unstale":"2024-09-10T18:49:03.220979+0000","last_undegraded":"2024-09-10T18:49:03.220979+0000","last_fullsized":"2024-09-10T18:49:03.220979+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:29.550946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:29.550946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:50.968295+0000","scrub_duration":0.016562341000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"89'9","reported_seq":92,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:06.122114+0000","last_change":"2024-09-10T18:48:09.648420+0000","last_active":"2024-09-10T18:49:06.122114+0000","last_peered":"2024-09-10T18:49:06.122114+0000","last_clean":"2024-09-10T18:49:06.122114+0000","last_became_active":"2024-09-10T18:47:36.421188+0000","last_became_peered":"2024-09-10T18:47:36.421188+0000","last_unstale":"2024-09-10T18:49:06.122114+0000","last_undegraded":"2024-09-10T18:49:06.122114+0000","last_fullsized":"2024-09-10T18:49:06.122114+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'9","last_scrub_stamp":"2024-09-10T18:48:09.648322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:09.648322+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:30.082986+0000","scrub_duration":0.0066489619999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":92,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.211472+0000","last_change":"2024-09-10T18:48:32.043742+0000","last_active":"2024-09-10T18:49:03.211472+0000","last_peered":"2024-09-10T18:49:03.211472+0000","last_clean":"2024-09-10T18:49:03.211472+0000","last_became_active":"2024-09-10T18:47:36.414625+0000","last_became_peered":"2024-09-10T18:47:36.414625+0000","last_unstale":"2024-09-10T18:49:03.211472+0000","last_undegraded":"2024-09-10T18:49:03.211472+0000","last_fullsized":"2024-09-10T18:49:03.211472+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:32.043666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:32.043666+0000","last_clean_scrub_stamp":"2024-09-10T18:48:32.043666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:49.740069+0000","scrub_duration":0.001572417,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"116'8403","reported_seq":8438,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T18:49:11.049032+0000","last_change":"2024-09-10T18:47:34.399541+0000","last_active":"2024-09-10T18:49:11.049032+0000","last_peered":"2024-09-10T18:49:11.049032+0000","last_clean":"2024-09-10T18:49:11.049032+0000","last_became_active":"2024-09-10T18:47:34.399261+0000","last_became_peered":"2024-09-10T18:47:34.399261+0000","last_unstale":"2024-09-10T18:49:11.049032+0000","last_undegraded":"2024-09-10T18:49:11.049032+0000","last_fullsized":"2024-09-10T18:49:11.049032+0000","mapping_epoch":86,"log_start":"99'3200","ondisk_log_start":"99'3200","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":5203,"log_dups_size":0,"ondisk_log_size":5203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":550699008,"num_objects":8403,"num_object_clones":0,"num_object_copies":25209,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8403,"num_write_kb":537792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"63'15","reported_seq":68,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.221367+0000","last_change":"2024-09-10T18:46:58.122279+0000","last_active":"2024-09-10T18:49:03.221367+0000","last_peered":"2024-09-10T18:49:03.221367+0000","last_clean":"2024-09-10T18:49:03.221367+0000","last_became_active":"2024-09-10T18:46:56.742946+0000","last_became_peered":"2024-09-10T18:46:56.742946+0000","last_unstale":"2024-09-10T18:49:03.221367+0000","last_undegraded":"2024-09-10T18:49:03.221367+0000","last_fullsized":"2024-09-10T18:49:03.221367+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:46:58.122219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:58.122219+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001678094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":89,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.221792+0000","last_change":"2024-09-10T18:48:13.707838+0000","last_active":"2024-09-10T18:49:03.221792+0000","last_peered":"2024-09-10T18:49:03.221792+0000","last_clean":"2024-09-10T18:49:03.221792+0000","last_became_active":"2024-09-10T18:45:17.965409+0000","last_became_peered":"2024-09-10T18:45:17.965409+0000","last_unstale":"2024-09-10T18:49:03.221792+0000","last_undegraded":"2024-09-10T18:49:03.221792+0000","last_fullsized":"2024-09-10T18:49:03.221792+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:13.707725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_clean_scrub_stamp":"2024-09-10T18:48:13.707725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:14.271680+0000","scrub_duration":0.0075610709999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"63'8","reported_seq":138,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:06.120618+0000","last_change":"2024-09-10T18:48:18.950238+0000","last_active":"2024-09-10T18:49:06.120618+0000","last_peered":"2024-09-10T18:49:06.120618+0000","last_clean":"2024-09-10T18:49:06.120618+0000","last_became_active":"2024-09-10T18:45:22.062497+0000","last_became_peered":"2024-09-10T18:45:22.062497+0000","last_unstale":"2024-09-10T18:49:06.120618+0000","last_undegraded":"2024-09-10T18:49:06.120618+0000","last_fullsized":"2024-09-10T18:49:06.120618+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:48:18.950140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:18.950140+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:34.771680+0000","scrub_duration":0.0032883439999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":82,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.220883+0000","last_change":"2024-09-10T18:48:31.170536+0000","last_active":"2024-09-10T18:49:03.220883+0000","last_peered":"2024-09-10T18:49:03.220883+0000","last_clean":"2024-09-10T18:49:03.220883+0000","last_became_active":"2024-09-10T18:45:24.082362+0000","last_became_peered":"2024-09-10T18:45:24.082362+0000","last_unstale":"2024-09-10T18:49:03.220883+0000","last_undegraded":"2024-09-10T18:49:03.220883+0000","last_fullsized":"2024-09-10T18:49:03.220883+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:31.170457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:31.170457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:50.433946+0000","scrub_duration":0.0084072260000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"116'8147","reported_seq":8351,"reported_epoch":116,"state":"active+clean+scrubbing","last_fresh":"2024-09-10T18:49:11.042424+0000","last_change":"2024-09-10T18:49:09.455022+0000","last_active":"2024-09-10T18:49:11.042424+0000","last_peered":"2024-09-10T18:49:11.042424+0000","last_clean":"2024-09-10T18:49:11.042424+0000","last_became_active":"2024-09-10T18:47:34.399711+0000","last_became_peered":"2024-09-10T18:47:34.399711+0000","last_unstale":"2024-09-10T18:49:11.042424+0000","last_undegraded":"2024-09-10T18:49:11.042424+0000","last_fullsized":"2024-09-10T18:49:11.042424+0000","mapping_epoch":86,"log_start":"99'2900","ondisk_log_start":"99'2900","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":4099,"log_size":5247,"log_dups_size":0,"ondisk_log_size":5247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 1s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":533921792,"num_objects":8147,"num_object_clones":0,"num_object_copies":24441,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8147,"num_write_kb":521408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"63'11","reported_seq":171,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.211584+0000","last_change":"2024-09-10T18:48:11.900583+0000","last_active":"2024-09-10T18:49:03.211584+0000","last_peered":"2024-09-10T18:49:03.211584+0000","last_clean":"2024-09-10T18:49:03.211584+0000","last_became_active":"2024-09-10T18:47:57.106935+0000","last_became_peered":"2024-09-10T18:47:57.106935+0000","last_unstale":"2024-09-10T18:49:03.211584+0000","last_undegraded":"2024-09-10T18:49:03.211584+0000","last_fullsized":"2024-09-10T18:49:03.211584+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:48:11.900489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:11.900489+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:16.653578+0000","scrub_duration":0.0020768789999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":89,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.215117+0000","last_change":"2024-09-10T18:48:12.642771+0000","last_active":"2024-09-10T18:49:03.215117+0000","last_peered":"2024-09-10T18:49:03.215117+0000","last_clean":"2024-09-10T18:49:03.215117+0000","last_became_active":"2024-09-10T18:45:17.966133+0000","last_became_peered":"2024-09-10T18:45:17.966133+0000","last_unstale":"2024-09-10T18:49:03.215117+0000","last_undegraded":"2024-09-10T18:49:03.215117+0000","last_fullsized":"2024-09-10T18:49:03.215117+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:12.642693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:12.642693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:15.199200+0000","scrub_duration":0.0035654889999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":84,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.221448+0000","last_change":"2024-09-10T18:48:25.093064+0000","last_active":"2024-09-10T18:49:03.221448+0000","last_peered":"2024-09-10T18:49:03.221448+0000","last_clean":"2024-09-10T18:49:03.221448+0000","last_became_active":"2024-09-10T18:45:22.068918+0000","last_became_peered":"2024-09-10T18:45:22.068918+0000","last_unstale":"2024-09-10T18:49:03.221448+0000","last_undegraded":"2024-09-10T18:49:03.221448+0000","last_fullsized":"2024-09-10T18:49:03.221448+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:25.092984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_clean_scrub_stamp":"2024-09-10T18:48:25.092984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:45.861853+0000","scrub_duration":0.0038537620000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":83,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:04.647846+0000","last_change":"2024-09-10T18:49:04.647711+0000","last_active":"2024-09-10T18:49:04.647846+0000","last_peered":"2024-09-10T18:49:04.647846+0000","last_clean":"2024-09-10T18:49:04.647846+0000","last_became_active":"2024-09-10T18:45:24.264538+0000","last_became_peered":"2024-09-10T18:45:24.264538+0000","last_unstale":"2024-09-10T18:49:04.647846+0000","last_undegraded":"2024-09-10T18:49:04.647846+0000","last_fullsized":"2024-09-10T18:49:04.647846+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:04.647638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:04.647638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:31.480091+0000","scrub_duration":0.115211389,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2","version":"116'8289","reported_seq":8336,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T18:49:11.184991+0000","last_change":"2024-09-10T18:47:36.423507+0000","last_active":"2024-09-10T18:49:11.184991+0000","last_peered":"2024-09-10T18:49:11.184991+0000","last_clean":"2024-09-10T18:49:11.184991+0000","last_became_active":"2024-09-10T18:47:36.423234+0000","last_became_peered":"2024-09-10T18:47:36.423234+0000","last_unstale":"2024-09-10T18:49:11.184991+0000","last_undegraded":"2024-09-10T18:49:11.184991+0000","last_fullsized":"2024-09-10T18:49:11.184991+0000","mapping_epoch":88,"log_start":"103'4000","ondisk_log_start":"103'4000","created":86,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":4289,"log_dups_size":0,"ondisk_log_size":4289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":543227904,"num_objects":8289,"num_object_clones":0,"num_object_copies":24867,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8289,"num_write_kb":530496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"63'14","reported_seq":107,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:04.392894+0000","last_change":"2024-09-10T18:49:04.392814+0000","last_active":"2024-09-10T18:49:04.392894+0000","last_peered":"2024-09-10T18:49:04.392894+0000","last_clean":"2024-09-10T18:49:04.392894+0000","last_became_active":"2024-09-10T18:45:20.052243+0000","last_became_peered":"2024-09-10T18:45:20.052243+0000","last_unstale":"2024-09-10T18:49:04.392894+0000","last_undegraded":"2024-09-10T18:49:04.392894+0000","last_fullsized":"2024-09-10T18:49:04.392894+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-10T18:49:04.392667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:49:04.392667+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:23.598321+0000","scrub_duration":0.0046253650000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":98,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.219527+0000","last_change":"2024-09-10T18:48:13.631307+0000","last_active":"2024-09-10T18:49:03.219527+0000","last_peered":"2024-09-10T18:49:03.219527+0000","last_clean":"2024-09-10T18:49:03.219527+0000","last_became_active":"2024-09-10T18:47:36.424466+0000","last_became_peered":"2024-09-10T18:47:36.424466+0000","last_unstale":"2024-09-10T18:49:03.219527+0000","last_undegraded":"2024-09-10T18:49:03.219527+0000","last_fullsized":"2024-09-10T18:49:03.219527+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:13.631078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_clean_scrub_stamp":"2024-09-10T18:48:13.631078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:29.790261+0000","scrub_duration":0.0043023660000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":140,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T18:49:08.264529+0000","last_change":"2024-09-10T18:48:23.479877+0000","last_active":"2024-09-10T18:49:08.264529+0000","last_peered":"2024-09-10T18:49:08.264529+0000","last_clean":"2024-09-10T18:49:08.264529+0000","last_became_active":"2024-09-10T18:45:22.074540+0000","last_became_peered":"2024-09-10T18:45:22.074540+0000","last_unstale":"2024-09-10T18:49:08.264529+0000","last_undegraded":"2024-09-10T18:49:08.264529+0000","last_fullsized":"2024-09-10T18:49:08.264529+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:48:23.479762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:23.479762+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:32.897955+0000","scrub_duration":0.01210871,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":93,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.214119+0000","last_change":"2024-09-10T18:48:08.689486+0000","last_active":"2024-09-10T18:49:03.214119+0000","last_peered":"2024-09-10T18:49:03.214119+0000","last_clean":"2024-09-10T18:49:03.214119+0000","last_became_active":"2024-09-10T18:47:36.414665+0000","last_became_peered":"2024-09-10T18:47:36.414665+0000","last_unstale":"2024-09-10T18:49:03.214119+0000","last_undegraded":"2024-09-10T18:49:03.214119+0000","last_fullsized":"2024-09-10T18:49:03.214119+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:08.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:08.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:11.141061+0000","scrub_duration":0.0033094489999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"116'8454","reported_seq":8502,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T18:49:11.195816+0000","last_change":"2024-09-10T18:47:36.410975+0000","last_active":"2024-09-10T18:49:11.195816+0000","last_peered":"2024-09-10T18:49:11.195816+0000","last_clean":"2024-09-10T18:49:11.195816+0000","last_became_active":"2024-09-10T18:47:36.410639+0000","last_became_peered":"2024-09-10T18:47:36.410639+0000","last_unstale":"2024-09-10T18:49:11.195816+0000","last_undegraded":"2024-09-10T18:49:11.195816+0000","last_fullsized":"2024-09-10T18:49:11.195816+0000","mapping_epoch":88,"log_start":"103'4200","ondisk_log_start":"103'4200","created":86,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":4254,"log_dups_size":0,"ondisk_log_size":4254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554041344,"num_objects":8454,"num_object_clones":0,"num_object_copies":25362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8454,"num_write_kb":541056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"63'18","reported_seq":119,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.218876+0000","last_change":"2024-09-10T18:48:20.514584+0000","last_active":"2024-09-10T18:49:03.218876+0000","last_peered":"2024-09-10T18:49:03.218876+0000","last_clean":"2024-09-10T18:49:03.218876+0000","last_became_active":"2024-09-10T18:45:20.420937+0000","last_became_peered":"2024-09-10T18:45:20.420937+0000","last_unstale":"2024-09-10T18:49:03.218876+0000","last_undegraded":"2024-09-10T18:49:03.218876+0000","last_fullsized":"2024-09-10T18:49:03.218876+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'18","last_scrub_stamp":"2024-09-10T18:48:20.514501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:20.514501+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:32.154497+0000","scrub_duration":0.028109085999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":79,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.222147+0000","last_change":"2024-09-10T18:47:05.505431+0000","last_active":"2024-09-10T18:49:03.222147+0000","last_peered":"2024-09-10T18:49:03.222147+0000","last_clean":"2024-09-10T18:49:03.222147+0000","last_became_active":"2024-09-10T18:45:17.961695+0000","last_became_peered":"2024-09-10T18:45:17.961695+0000","last_unstale":"2024-09-10T18:49:03.222147+0000","last_undegraded":"2024-09-10T18:49:03.222147+0000","last_fullsized":"2024-09-10T18:49:03.222147+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:05.505376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:47:05.505376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013302310000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":84,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.210714+0000","last_change":"2024-09-10T18:48:27.016984+0000","last_active":"2024-09-10T18:49:03.210714+0000","last_peered":"2024-09-10T18:49:03.210714+0000","last_clean":"2024-09-10T18:49:03.210714+0000","last_became_active":"2024-09-10T18:45:22.074012+0000","last_became_peered":"2024-09-10T18:45:22.074012+0000","last_unstale":"2024-09-10T18:49:03.210714+0000","last_undegraded":"2024-09-10T18:49:03.210714+0000","last_fullsized":"2024-09-10T18:49:03.210714+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:27.016933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:27.016933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:28.747433+0000","scrub_duration":0.0059183930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":82,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.211064+0000","last_change":"2024-09-10T18:47:36.411129+0000","last_active":"2024-09-10T18:49:03.211064+0000","last_peered":"2024-09-10T18:49:03.211064+0000","last_clean":"2024-09-10T18:49:03.211064+0000","last_became_active":"2024-09-10T18:47:36.410836+0000","last_became_peered":"2024-09-10T18:47:36.410836+0000","last_unstale":"2024-09-10T18:49:03.211064+0000","last_undegraded":"2024-09-10T18:49:03.211064+0000","last_fullsized":"2024-09-10T18:49:03.211064+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_clean_scrub_stamp":"2024-09-10T18:46:53.087539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001678971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"116'8233","reported_seq":8270,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T18:49:11.019279+0000","last_change":"2024-09-10T18:47:34.399316+0000","last_active":"2024-09-10T18:49:11.019279+0000","last_peered":"2024-09-10T18:49:11.019279+0000","last_clean":"2024-09-10T18:49:11.019279+0000","last_became_active":"2024-09-10T18:47:34.399002+0000","last_became_peered":"2024-09-10T18:47:34.399002+0000","last_unstale":"2024-09-10T18:49:11.019279+0000","last_undegraded":"2024-09-10T18:49:11.019279+0000","last_fullsized":"2024-09-10T18:49:11.019279+0000","mapping_epoch":86,"log_start":"99'3000","ondisk_log_start":"99'3000","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":5233,"log_dups_size":0,"ondisk_log_size":5233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":539557888,"num_objects":8233,"num_object_clones":0,"num_object_copies":24699,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8233,"num_write_kb":526912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3","version":"63'19","reported_seq":123,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.214645+0000","last_change":"2024-09-10T18:48:11.781505+0000","last_active":"2024-09-10T18:49:03.214645+0000","last_peered":"2024-09-10T18:49:03.214645+0000","last_clean":"2024-09-10T18:49:03.214645+0000","last_became_active":"2024-09-10T18:45:20.421046+0000","last_became_peered":"2024-09-10T18:45:20.421046+0000","last_unstale":"2024-09-10T18:49:03.214645+0000","last_undegraded":"2024-09-10T18:49:03.214645+0000","last_fullsized":"2024-09-10T18:49:03.214645+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:48:11.781417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:11.781417+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:32.063852+0000","scrub_duration":0.0030109379999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":89,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.214593+0000","last_change":"2024-09-10T18:48:30.694038+0000","last_active":"2024-09-10T18:49:03.214593+0000","last_peered":"2024-09-10T18:49:03.214593+0000","last_clean":"2024-09-10T18:49:03.214593+0000","last_became_active":"2024-09-10T18:45:17.960226+0000","last_became_peered":"2024-09-10T18:45:17.960226+0000","last_unstale":"2024-09-10T18:49:03.214593+0000","last_undegraded":"2024-09-10T18:49:03.214593+0000","last_fullsized":"2024-09-10T18:49:03.214593+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:30.693950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:30.693950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:49.091630+0000","scrub_duration":0.0079194309999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":84,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.218432+0000","last_change":"2024-09-10T18:48:33.513604+0000","last_active":"2024-09-10T18:49:03.218432+0000","last_peered":"2024-09-10T18:49:03.218432+0000","last_clean":"2024-09-10T18:49:03.218432+0000","last_became_active":"2024-09-10T18:45:22.066148+0000","last_became_peered":"2024-09-10T18:45:22.066148+0000","last_unstale":"2024-09-10T18:49:03.218432+0000","last_undegraded":"2024-09-10T18:49:03.218432+0000","last_fullsized":"2024-09-10T18:49:03.218432+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:33.513500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:33.513500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:47.158850+0000","scrub_duration":0.0080948319999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":83,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.221531+0000","last_change":"2024-09-10T18:47:36.422607+0000","last_active":"2024-09-10T18:49:03.221531+0000","last_peered":"2024-09-10T18:49:03.221531+0000","last_clean":"2024-09-10T18:49:03.221531+0000","last_became_active":"2024-09-10T18:47:36.422295+0000","last_became_peered":"2024-09-10T18:47:36.422295+0000","last_unstale":"2024-09-10T18:49:03.221531+0000","last_undegraded":"2024-09-10T18:49:03.221531+0000","last_fullsized":"2024-09-10T18:49:03.221531+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:25.525085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:25.525085+0000","last_clean_scrub_stamp":"2024-09-10T18:47:25.525085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012481079999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"116'8267","reported_seq":8303,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T18:49:11.049709+0000","last_change":"2024-09-10T18:47:34.397576+0000","last_active":"2024-09-10T18:49:11.049709+0000","last_peered":"2024-09-10T18:49:11.049709+0000","last_clean":"2024-09-10T18:49:11.049709+0000","last_became_active":"2024-09-10T18:47:34.396631+0000","last_became_peered":"2024-09-10T18:47:34.396631+0000","last_unstale":"2024-09-10T18:49:11.049709+0000","last_undegraded":"2024-09-10T18:49:11.049709+0000","last_fullsized":"2024-09-10T18:49:11.049709+0000","mapping_epoch":86,"log_start":"99'3100","ondisk_log_start":"99'3100","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":5167,"log_dups_size":0,"ondisk_log_size":5167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":541786112,"num_objects":8267,"num_object_clones":0,"num_object_copies":24801,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8267,"num_write_kb":529088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"83'15","reported_seq":107,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.220792+0000","last_change":"2024-09-10T18:48:20.633119+0000","last_active":"2024-09-10T18:49:03.220792+0000","last_peered":"2024-09-10T18:49:03.220792+0000","last_clean":"2024-09-10T18:49:03.220792+0000","last_became_active":"2024-09-10T18:45:20.057204+0000","last_became_peered":"2024-09-10T18:45:20.057204+0000","last_unstale":"2024-09-10T18:49:03.220792+0000","last_undegraded":"2024-09-10T18:49:03.220792+0000","last_fullsized":"2024-09-10T18:49:03.220792+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'15","last_scrub_stamp":"2024-09-10T18:48:20.633054+0000","last_deep_scrub":"83'15","last_deep_scrub_stamp":"2024-09-10T18:46:58.436473+0000","last_clean_scrub_stamp":"2024-09-10T18:48:20.633054+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:37.027685+0000","scrub_duration":0.008267412,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":89,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T18:49:09.449177+0000","last_change":"2024-09-10T18:49:09.449135+0000","last_active":"2024-09-10T18:49:09.449177+0000","last_peered":"2024-09-10T18:49:09.449177+0000","last_clean":"2024-09-10T18:49:09.449177+0000","last_became_active":"2024-09-10T18:45:17.966632+0000","last_became_peered":"2024-09-10T18:45:17.966632+0000","last_unstale":"2024-09-10T18:49:09.449177+0000","last_undegraded":"2024-09-10T18:49:09.449177+0000","last_fullsized":"2024-09-10T18:49:09.449177+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:09.449037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:09.449037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:18.822485+0000","scrub_duration":0.013065797000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"63'8","reported_seq":140,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T18:49:08.266742+0000","last_change":"2024-09-10T18:49:00.552860+0000","last_active":"2024-09-10T18:49:08.266742+0000","last_peered":"2024-09-10T18:49:08.266742+0000","last_clean":"2024-09-10T18:49:08.266742+0000","last_became_active":"2024-09-10T18:45:22.063326+0000","last_became_peered":"2024-09-10T18:45:22.063326+0000","last_unstale":"2024-09-10T18:49:08.266742+0000","last_undegraded":"2024-09-10T18:49:08.266742+0000","last_fullsized":"2024-09-10T18:49:08.266742+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:49:00.552765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:49:00.552765+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:28.435607+0000","scrub_duration":0.0038235579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":93,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.213498+0000","last_change":"2024-09-10T18:49:01.532633+0000","last_active":"2024-09-10T18:49:03.213498+0000","last_peered":"2024-09-10T18:49:03.213498+0000","last_clean":"2024-09-10T18:49:03.213498+0000","last_became_active":"2024-09-10T18:47:36.412111+0000","last_became_peered":"2024-09-10T18:47:36.412111+0000","last_unstale":"2024-09-10T18:49:03.213498+0000","last_undegraded":"2024-09-10T18:49:03.213498+0000","last_fullsized":"2024-09-10T18:49:03.213498+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:01.532550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:01.532550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:17.367791+0000","scrub_duration":0.0037383170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"116'8232","reported_seq":8589,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T18:49:10.947549+0000","last_change":"2024-09-10T18:49:05.169424+0000","last_active":"2024-09-10T18:49:10.947549+0000","last_peered":"2024-09-10T18:49:10.947549+0000","last_clean":"2024-09-10T18:49:10.947549+0000","last_became_active":"2024-09-10T18:47:34.400044+0000","last_became_peered":"2024-09-10T18:47:34.400044+0000","last_unstale":"2024-09-10T18:49:10.947549+0000","last_undegraded":"2024-09-10T18:49:10.947549+0000","last_fullsized":"2024-09-10T18:49:10.947549+0000","mapping_epoch":86,"log_start":"99'3000","ondisk_log_start":"99'3000","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"115'7848","last_scrub_stamp":"2024-09-10T18:49:05.169330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:49:05.169330+0000","objects_scrubbed":7724,"log_size":5232,"log_dups_size":0,"ondisk_log_size":5232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:16.559490+0000","scrub_duration":3.7256509850000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":539492352,"num_objects":8232,"num_object_clones":0,"num_object_copies":24696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8232,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8232,"num_write_kb":526848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"89'29","reported_seq":214,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T18:49:08.274514+0000","last_change":"2024-09-10T18:47:36.443205+0000","last_active":"2024-09-10T18:49:08.274514+0000","last_peered":"2024-09-10T18:49:08.274514+0000","last_clean":"2024-09-10T18:49:08.274514+0000","last_became_active":"2024-09-10T18:47:36.420185+0000","last_became_peered":"2024-09-10T18:47:36.420185+0000","last_unstale":"2024-09-10T18:49:08.274514+0000","last_undegraded":"2024-09-10T18:49:08.274514+0000","last_fullsized":"2024-09-10T18:49:08.274514+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'28","last_scrub_stamp":"2024-09-10T18:47:26.525861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:26.525861+0000","objects_scrubbed":10,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002126719,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":88,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.220863+0000","last_change":"2024-09-10T18:48:33.186208+0000","last_active":"2024-09-10T18:49:03.220863+0000","last_peered":"2024-09-10T18:49:03.220863+0000","last_clean":"2024-09-10T18:49:03.220863+0000","last_became_active":"2024-09-10T18:45:17.964494+0000","last_became_peered":"2024-09-10T18:45:17.964494+0000","last_unstale":"2024-09-10T18:49:03.220863+0000","last_undegraded":"2024-09-10T18:49:03.220863+0000","last_fullsized":"2024-09-10T18:49:03.220863+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:33.186070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:33.186070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:00.114296+0000","scrub_duration":0.0068142150000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":39,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.219075+0000","last_change":"2024-09-10T18:49:00.438027+0000","last_active":"2024-09-10T18:49:03.219075+0000","last_peered":"2024-09-10T18:49:03.219075+0000","last_clean":"2024-09-10T18:49:03.219075+0000","last_became_active":"2024-09-10T18:47:57.107478+0000","last_became_peered":"2024-09-10T18:47:57.107478+0000","last_unstale":"2024-09-10T18:49:03.219075+0000","last_undegraded":"2024-09-10T18:49:03.219075+0000","last_fullsized":"2024-09-10T18:49:03.219075+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:00.437877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:49:00.437877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:04.529124+0000","scrub_duration":0.012117157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":73,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.221170+0000","last_change":"2024-09-10T18:47:22.891142+0000","last_active":"2024-09-10T18:49:03.221170+0000","last_peered":"2024-09-10T18:49:03.221170+0000","last_clean":"2024-09-10T18:49:03.221170+0000","last_became_active":"2024-09-10T18:45:24.073307+0000","last_became_peered":"2024-09-10T18:45:24.073307+0000","last_unstale":"2024-09-10T18:49:03.221170+0000","last_undegraded":"2024-09-10T18:49:03.221170+0000","last_fullsized":"2024-09-10T18:49:03.221170+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:22.891084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:22.891084+0000","last_clean_scrub_stamp":"2024-09-10T18:47:22.891084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017788579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"116'8206","reported_seq":8567,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T18:49:11.107312+0000","last_change":"2024-09-10T18:49:07.011995+0000","last_active":"2024-09-10T18:49:11.107312+0000","last_peered":"2024-09-10T18:49:11.107312+0000","last_clean":"2024-09-10T18:49:11.107312+0000","last_became_active":"2024-09-10T18:47:34.395271+0000","last_became_peered":"2024-09-10T18:47:34.395271+0000","last_unstale":"2024-09-10T18:49:11.107312+0000","last_undegraded":"2024-09-10T18:49:11.107312+0000","last_fullsized":"2024-09-10T18:49:11.107312+0000","mapping_epoch":86,"log_start":"101'3400","ondisk_log_start":"101'3400","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"115'7913","last_scrub_stamp":"2024-09-10T18:49:07.011758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:49:07.011758+0000","objects_scrubbed":7815,"log_size":4806,"log_dups_size":0,"ondisk_log_size":4806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:20.945730+0000","scrub_duration":2.4953754379999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":537788416,"num_objects":8206,"num_object_clones":0,"num_object_copies":24618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8206,"num_write_kb":525184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"84'18","reported_seq":109,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.222028+0000","last_change":"2024-09-10T18:47:10.515060+0000","last_active":"2024-09-10T18:49:03.222028+0000","last_peered":"2024-09-10T18:49:03.222028+0000","last_clean":"2024-09-10T18:49:03.222028+0000","last_became_active":"2024-09-10T18:45:20.058435+0000","last_became_peered":"2024-09-10T18:45:20.058435+0000","last_unstale":"2024-09-10T18:49:03.222028+0000","last_undegraded":"2024-09-10T18:49:03.222028+0000","last_fullsized":"2024-09-10T18:49:03.222028+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'18","last_scrub_stamp":"2024-09-10T18:47:10.514951+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-10T18:47:10.514951+0000","last_clean_scrub_stamp":"2024-09-10T18:47:10.514951+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005567575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"84'6","reported_seq":326,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T18:49:09.636383+0000","last_change":"2024-09-10T18:47:36.439529+0000","last_active":"2024-09-10T18:49:09.636383+0000","last_peered":"2024-09-10T18:49:09.636383+0000","last_clean":"2024-09-10T18:49:09.636383+0000","last_became_active":"2024-09-10T18:47:36.418211+0000","last_became_peered":"2024-09-10T18:47:36.418211+0000","last_unstale":"2024-09-10T18:49:09.636383+0000","last_undegraded":"2024-09-10T18:49:09.636383+0000","last_fullsized":"2024-09-10T18:49:09.636383+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'6","last_scrub_stamp":"2024-09-10T18:47:22.545450+0000","last_deep_scrub":"84'6","last_deep_scrub_stamp":"2024-09-10T18:47:22.545450+0000","last_clean_scrub_stamp":"2024-09-10T18:47:22.545450+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022780040000000001,"objects_trimmed":0,"snaptrim_duration":0.0021933159999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":223,"num_read_kb":218,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":40,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.219367+0000","last_change":"2024-09-10T18:47:36.422920+0000","last_active":"2024-09-10T18:49:03.219367+0000","last_peered":"2024-09-10T18:49:03.219367+0000","last_clean":"2024-09-10T18:49:03.219367+0000","last_became_active":"2024-09-10T18:47:36.422641+0000","last_became_peered":"2024-09-10T18:47:36.422641+0000","last_unstale":"2024-09-10T18:49:03.219367+0000","last_undegraded":"2024-09-10T18:49:03.219367+0000","last_fullsized":"2024-09-10T18:49:03.219367+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:46.712269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:46.712269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":49,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.221328+0000","last_change":"2024-09-10T18:48:22.011954+0000","last_active":"2024-09-10T18:49:03.221328+0000","last_peered":"2024-09-10T18:49:03.221328+0000","last_clean":"2024-09-10T18:49:03.221328+0000","last_became_active":"2024-09-10T18:47:36.419081+0000","last_became_peered":"2024-09-10T18:47:36.419081+0000","last_unstale":"2024-09-10T18:49:03.221328+0000","last_undegraded":"2024-09-10T18:49:03.221328+0000","last_fullsized":"2024-09-10T18:49:03.221328+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:22.011870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:22.011870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:41.762812+0000","scrub_duration":0.0018770919999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":73,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.214372+0000","last_change":"2024-09-10T18:46:59.553576+0000","last_active":"2024-09-10T18:49:03.214372+0000","last_peered":"2024-09-10T18:49:03.214372+0000","last_clean":"2024-09-10T18:49:03.214372+0000","last_became_active":"2024-09-10T18:45:24.264617+0000","last_became_peered":"2024-09-10T18:45:24.264617+0000","last_unstale":"2024-09-10T18:49:03.214372+0000","last_undegraded":"2024-09-10T18:49:03.214372+0000","last_fullsized":"2024-09-10T18:49:03.214372+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:59.553505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:59.553505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018297210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"116'8268","reported_seq":8315,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T18:49:11.061164+0000","last_change":"2024-09-10T18:47:36.417383+0000","last_active":"2024-09-10T18:49:11.061164+0000","last_peered":"2024-09-10T18:49:11.061164+0000","last_clean":"2024-09-10T18:49:11.061164+0000","last_became_active":"2024-09-10T18:47:36.417075+0000","last_became_peered":"2024-09-10T18:47:36.417075+0000","last_unstale":"2024-09-10T18:49:11.061164+0000","last_undegraded":"2024-09-10T18:49:11.061164+0000","last_fullsized":"2024-09-10T18:49:11.061164+0000","mapping_epoch":88,"log_start":"103'3700","ondisk_log_start":"103'3700","created":86,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":4568,"log_dups_size":0,"ondisk_log_size":4568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":541851648,"num_objects":8268,"num_object_clones":0,"num_object_copies":24804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8268,"num_write_kb":529152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"63'12","reported_seq":129,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.215003+0000","last_change":"2024-09-10T18:48:28.710224+0000","last_active":"2024-09-10T18:49:03.215003+0000","last_peered":"2024-09-10T18:49:03.215003+0000","last_clean":"2024-09-10T18:49:03.215003+0000","last_became_active":"2024-09-10T18:47:36.417965+0000","last_became_peered":"2024-09-10T18:47:36.417965+0000","last_unstale":"2024-09-10T18:49:03.215003+0000","last_undegraded":"2024-09-10T18:49:03.215003+0000","last_fullsized":"2024-09-10T18:49:03.215003+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:48:28.710113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:28.710113+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:58.705191+0000","scrub_duration":0.0069768260000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"58'1","reported_seq":95,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:07.525698+0000","last_change":"2024-09-10T18:49:07.525629+0000","last_active":"2024-09-10T18:49:07.525698+0000","last_peered":"2024-09-10T18:49:07.525698+0000","last_clean":"2024-09-10T18:49:07.525698+0000","last_became_active":"2024-09-10T18:45:17.963932+0000","last_became_peered":"2024-09-10T18:45:17.963932+0000","last_unstale":"2024-09-10T18:49:07.525698+0000","last_undegraded":"2024-09-10T18:49:07.525698+0000","last_fullsized":"2024-09-10T18:49:07.525698+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'1","last_scrub_stamp":"2024-09-10T18:49:07.525525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:07.525525+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:30.019519+0000","scrub_duration":0.01484512,"objects_trimmed":0,"snaptrim_duration":0.00037812099999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":47,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:07.603618+0000","last_change":"2024-09-10T18:49:07.603571+0000","last_active":"2024-09-10T18:49:07.603618+0000","last_peered":"2024-09-10T18:49:07.603618+0000","last_clean":"2024-09-10T18:49:07.603618+0000","last_became_active":"2024-09-10T18:47:36.416641+0000","last_became_peered":"2024-09-10T18:47:36.416641+0000","last_unstale":"2024-09-10T18:49:07.603618+0000","last_undegraded":"2024-09-10T18:49:07.603618+0000","last_fullsized":"2024-09-10T18:49:07.603618+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:07.603516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:07.603516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:29.660107+0000","scrub_duration":0.0079160310000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":84,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T18:49:11.029184+0000","last_change":"2024-09-10T18:49:11.029146+0000","last_active":"2024-09-10T18:49:11.029184+0000","last_peered":"2024-09-10T18:49:11.029184+0000","last_clean":"2024-09-10T18:49:11.029184+0000","last_became_active":"2024-09-10T18:45:22.064285+0000","last_became_peered":"2024-09-10T18:45:22.064285+0000","last_unstale":"2024-09-10T18:49:11.029184+0000","last_undegraded":"2024-09-10T18:49:11.029184+0000","last_fullsized":"2024-09-10T18:49:11.029184+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:11.029069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:49:11.029069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:18.876406+0000","scrub_duration":0.0021808299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":83,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.221046+0000","last_change":"2024-09-10T18:48:21.654220+0000","last_active":"2024-09-10T18:49:03.221046+0000","last_peered":"2024-09-10T18:49:03.221046+0000","last_clean":"2024-09-10T18:49:03.221046+0000","last_became_active":"2024-09-10T18:45:24.083689+0000","last_became_peered":"2024-09-10T18:45:24.083689+0000","last_unstale":"2024-09-10T18:49:03.221046+0000","last_undegraded":"2024-09-10T18:49:03.221046+0000","last_fullsized":"2024-09-10T18:49:03.221046+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:21.654007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:21.654007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:39.425642+0000","scrub_duration":0.0058335929999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"116'8230","reported_seq":8268,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T18:49:11.121020+0000","last_change":"2024-09-10T18:47:34.398452+0000","last_active":"2024-09-10T18:49:11.121020+0000","last_peered":"2024-09-10T18:49:11.121020+0000","last_clean":"2024-09-10T18:49:11.121020+0000","last_became_active":"2024-09-10T18:47:34.396853+0000","last_became_peered":"2024-09-10T18:47:34.396853+0000","last_unstale":"2024-09-10T18:49:11.121020+0000","last_undegraded":"2024-09-10T18:49:11.121020+0000","last_fullsized":"2024-09-10T18:49:11.121020+0000","mapping_epoch":86,"log_start":"101'3400","ondisk_log_start":"101'3400","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":4830,"log_dups_size":0,"ondisk_log_size":4830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":539361280,"num_objects":8230,"num_object_clones":0,"num_object_copies":24690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8230,"num_write_kb":526720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"63'14","reported_seq":108,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.219387+0000","last_change":"2024-09-10T18:48:17.669939+0000","last_active":"2024-09-10T18:49:03.219387+0000","last_peered":"2024-09-10T18:49:03.219387+0000","last_clean":"2024-09-10T18:49:03.219387+0000","last_became_active":"2024-09-10T18:45:20.421119+0000","last_became_peered":"2024-09-10T18:45:20.421119+0000","last_unstale":"2024-09-10T18:49:03.219387+0000","last_undegraded":"2024-09-10T18:49:03.219387+0000","last_fullsized":"2024-09-10T18:49:03.219387+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-10T18:48:17.669787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:17.669787+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:33.721948+0000","scrub_duration":0.0024048300000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"60'2","reported_seq":96,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.219654+0000","last_change":"2024-09-10T18:48:59.558866+0000","last_active":"2024-09-10T18:49:03.219654+0000","last_peered":"2024-09-10T18:49:03.219654+0000","last_clean":"2024-09-10T18:49:03.219654+0000","last_became_active":"2024-09-10T18:45:17.961660+0000","last_became_peered":"2024-09-10T18:45:17.961660+0000","last_unstale":"2024-09-10T18:49:03.219654+0000","last_undegraded":"2024-09-10T18:49:03.219654+0000","last_fullsized":"2024-09-10T18:49:03.219654+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-10T18:48:59.558773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:59.558773+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:15.175215+0000","scrub_duration":0.0043139010000000002,"objects_trimmed":0,"snaptrim_duration":0.0013942640000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":98,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.220935+0000","last_change":"2024-09-10T18:48:34.150227+0000","last_active":"2024-09-10T18:49:03.220935+0000","last_peered":"2024-09-10T18:49:03.220935+0000","last_clean":"2024-09-10T18:49:03.220935+0000","last_became_active":"2024-09-10T18:47:36.414636+0000","last_became_peered":"2024-09-10T18:47:36.414636+0000","last_unstale":"2024-09-10T18:49:03.220935+0000","last_undegraded":"2024-09-10T18:49:03.220935+0000","last_fullsized":"2024-09-10T18:49:03.220935+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:34.150146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:34.150146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:02.371194+0000","scrub_duration":0.0018375380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'81","reported_seq":133,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T18:49:09.660816+0000","last_change":"2024-09-10T18:49:09.660774+0000","last_active":"2024-09-10T18:49:09.660816+0000","last_peered":"2024-09-10T18:49:09.660816+0000","last_clean":"2024-09-10T18:49:09.660816+0000","last_became_active":"2024-09-10T18:45:07.433592+0000","last_became_peered":"2024-09-10T18:45:07.433592+0000","last_unstale":"2024-09-10T18:49:09.660816+0000","last_undegraded":"2024-09-10T18:49:09.660816+0000","last_fullsized":"2024-09-10T18:49:09.660816+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:49:09.660690+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:49:09.660690+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:31.143136+0000","scrub_duration":0.016709736999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":95,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.214902+0000","last_change":"2024-09-10T18:47:36.418948+0000","last_active":"2024-09-10T18:49:03.214902+0000","last_peered":"2024-09-10T18:49:03.214902+0000","last_clean":"2024-09-10T18:49:03.214902+0000","last_became_active":"2024-09-10T18:47:36.418320+0000","last_became_peered":"2024-09-10T18:47:36.418320+0000","last_unstale":"2024-09-10T18:49:03.214902+0000","last_undegraded":"2024-09-10T18:49:03.214902+0000","last_fullsized":"2024-09-10T18:49:03.214902+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:32.528904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:47:32.528904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037979899999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":72,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.221664+0000","last_change":"2024-09-10T18:46:56.066558+0000","last_active":"2024-09-10T18:49:03.221664+0000","last_peered":"2024-09-10T18:49:03.221664+0000","last_clean":"2024-09-10T18:49:03.221664+0000","last_became_active":"2024-09-10T18:45:24.073327+0000","last_became_peered":"2024-09-10T18:45:24.073327+0000","last_unstale":"2024-09-10T18:49:03.221664+0000","last_undegraded":"2024-09-10T18:49:03.221664+0000","last_fullsized":"2024-09-10T18:49:03.221664+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:56.066502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:56.066502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0011960810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.9","version":"116'8464","reported_seq":8502,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T18:49:11.166149+0000","last_change":"2024-09-10T18:47:34.393877+0000","last_active":"2024-09-10T18:49:11.166149+0000","last_peered":"2024-09-10T18:49:11.166149+0000","last_clean":"2024-09-10T18:49:11.166149+0000","last_became_active":"2024-09-10T18:47:34.393620+0000","last_became_peered":"2024-09-10T18:47:34.393620+0000","last_unstale":"2024-09-10T18:49:11.166149+0000","last_undegraded":"2024-09-10T18:49:11.166149+0000","last_fullsized":"2024-09-10T18:49:11.166149+0000","mapping_epoch":86,"log_start":"102'3700","ondisk_log_start":"102'3700","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":4764,"log_dups_size":0,"ondisk_log_size":4764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554696704,"num_objects":8464,"num_object_clones":0,"num_object_copies":25392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8464,"num_write_kb":541696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"63'19","reported_seq":117,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.218606+0000","last_change":"2024-09-10T18:48:14.655578+0000","last_active":"2024-09-10T18:49:03.218606+0000","last_peered":"2024-09-10T18:49:03.218606+0000","last_clean":"2024-09-10T18:49:03.218606+0000","last_became_active":"2024-09-10T18:45:20.421097+0000","last_became_peered":"2024-09-10T18:45:20.421097+0000","last_unstale":"2024-09-10T18:49:03.218606+0000","last_undegraded":"2024-09-10T18:49:03.218606+0000","last_fullsized":"2024-09-10T18:49:03.218606+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:48:14.655527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:14.655527+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:31.443092+0000","scrub_duration":0.00174087,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":49,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:08.070238+0000","last_change":"2024-09-10T18:49:08.070170+0000","last_active":"2024-09-10T18:49:08.070238+0000","last_peered":"2024-09-10T18:49:08.070238+0000","last_clean":"2024-09-10T18:49:08.070238+0000","last_became_active":"2024-09-10T18:47:36.417800+0000","last_became_peered":"2024-09-10T18:47:36.417800+0000","last_unstale":"2024-09-10T18:49:08.070238+0000","last_undegraded":"2024-09-10T18:49:08.070238+0000","last_fullsized":"2024-09-10T18:49:08.070238+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:08.070063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:08.070063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:15.698762+0000","scrub_duration":0.0033662290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":50,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.214323+0000","last_change":"2024-09-10T18:48:12.827398+0000","last_active":"2024-09-10T18:49:03.214323+0000","last_peered":"2024-09-10T18:49:03.214323+0000","last_clean":"2024-09-10T18:49:03.214323+0000","last_became_active":"2024-09-10T18:47:36.413618+0000","last_became_peered":"2024-09-10T18:47:36.413618+0000","last_unstale":"2024-09-10T18:49:03.214323+0000","last_undegraded":"2024-09-10T18:49:03.214323+0000","last_fullsized":"2024-09-10T18:49:03.214323+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:12.827317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:12.827317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:17.404178+0000","scrub_duration":0.0036790109999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":83,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.215172+0000","last_change":"2024-09-10T18:47:36.413013+0000","last_active":"2024-09-10T18:49:03.215172+0000","last_peered":"2024-09-10T18:49:03.215172+0000","last_clean":"2024-09-10T18:49:03.215172+0000","last_became_active":"2024-09-10T18:47:36.412610+0000","last_became_peered":"2024-09-10T18:47:36.412610+0000","last_unstale":"2024-09-10T18:49:03.215172+0000","last_undegraded":"2024-09-10T18:49:03.215172+0000","last_fullsized":"2024-09-10T18:49:03.215172+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:03.587252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:03.587252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001684867,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"116'8266","reported_seq":8306,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T18:49:11.068004+0000","last_change":"2024-09-10T18:47:36.413347+0000","last_active":"2024-09-10T18:49:11.068004+0000","last_peered":"2024-09-10T18:49:11.068004+0000","last_clean":"2024-09-10T18:49:11.068004+0000","last_became_active":"2024-09-10T18:47:36.412142+0000","last_became_peered":"2024-09-10T18:47:36.412142+0000","last_unstale":"2024-09-10T18:49:11.068004+0000","last_undegraded":"2024-09-10T18:49:11.068004+0000","last_fullsized":"2024-09-10T18:49:11.068004+0000","mapping_epoch":88,"log_start":"102'3700","ondisk_log_start":"102'3700","created":86,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":4566,"log_dups_size":0,"ondisk_log_size":4566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":541720576,"num_objects":8266,"num_object_clones":0,"num_object_copies":24798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8266,"num_write_kb":529024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"63'9","reported_seq":103,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.214833+0000","last_change":"2024-09-10T18:48:22.878265+0000","last_active":"2024-09-10T18:49:03.214833+0000","last_peered":"2024-09-10T18:49:03.214833+0000","last_clean":"2024-09-10T18:49:03.214833+0000","last_became_active":"2024-09-10T18:45:20.061021+0000","last_became_peered":"2024-09-10T18:45:20.061021+0000","last_unstale":"2024-09-10T18:49:03.214833+0000","last_undegraded":"2024-09-10T18:49:03.214833+0000","last_fullsized":"2024-09-10T18:49:03.214833+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:48:22.878179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:22.878179+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:49.491061+0000","scrub_duration":0.021357979999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":89,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:07.384771+0000","last_change":"2024-09-10T18:49:07.384707+0000","last_active":"2024-09-10T18:49:07.384771+0000","last_peered":"2024-09-10T18:49:07.384771+0000","last_clean":"2024-09-10T18:49:07.384771+0000","last_became_active":"2024-09-10T18:45:17.963057+0000","last_became_peered":"2024-09-10T18:45:17.963057+0000","last_unstale":"2024-09-10T18:49:07.384771+0000","last_undegraded":"2024-09-10T18:49:07.384771+0000","last_fullsized":"2024-09-10T18:49:07.384771+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:07.384620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:07.384620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:25.314632+0000","scrub_duration":0.011420041000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":39,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.210835+0000","last_change":"2024-09-10T18:47:36.411830+0000","last_active":"2024-09-10T18:49:03.210835+0000","last_peered":"2024-09-10T18:49:03.210835+0000","last_clean":"2024-09-10T18:49:03.210835+0000","last_became_active":"2024-09-10T18:47:36.411517+0000","last_became_peered":"2024-09-10T18:47:36.411517+0000","last_unstale":"2024-09-10T18:49:03.210835+0000","last_undegraded":"2024-09-10T18:49:03.210835+0000","last_fullsized":"2024-09-10T18:49:03.210835+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_clean_scrub_stamp":"2024-09-10T18:46:55.684250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":93,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.221755+0000","last_change":"2024-09-10T18:48:25.804464+0000","last_active":"2024-09-10T18:49:03.221755+0000","last_peered":"2024-09-10T18:49:03.221755+0000","last_clean":"2024-09-10T18:49:03.221755+0000","last_became_active":"2024-09-10T18:47:36.412933+0000","last_became_peered":"2024-09-10T18:47:36.412933+0000","last_unstale":"2024-09-10T18:49:03.221755+0000","last_undegraded":"2024-09-10T18:49:03.221755+0000","last_fullsized":"2024-09-10T18:49:03.221755+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:25.804376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:47.545343+0000","last_clean_scrub_stamp":"2024-09-10T18:48:25.804376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:26.796128+0000","scrub_duration":0.0098886110000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.b","version":"116'8316","reported_seq":8354,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T18:49:11.082157+0000","last_change":"2024-09-10T18:47:34.398004+0000","last_active":"2024-09-10T18:49:11.082157+0000","last_peered":"2024-09-10T18:49:11.082157+0000","last_clean":"2024-09-10T18:49:11.082157+0000","last_became_active":"2024-09-10T18:47:34.397708+0000","last_became_peered":"2024-09-10T18:47:34.397708+0000","last_unstale":"2024-09-10T18:49:11.082157+0000","last_undegraded":"2024-09-10T18:49:11.082157+0000","last_fullsized":"2024-09-10T18:49:11.082157+0000","mapping_epoch":86,"log_start":"103'3900","ondisk_log_start":"103'3900","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":4416,"log_dups_size":0,"ondisk_log_size":4416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":544997376,"num_objects":8316,"num_object_clones":0,"num_object_copies":24948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8316,"num_write_kb":532224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"63'15","reported_seq":101,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.220433+0000","last_change":"2024-09-10T18:46:51.114543+0000","last_active":"2024-09-10T18:49:03.220433+0000","last_peered":"2024-09-10T18:49:03.220433+0000","last_clean":"2024-09-10T18:49:03.220433+0000","last_became_active":"2024-09-10T18:45:20.419804+0000","last_became_peered":"2024-09-10T18:45:20.419804+0000","last_unstale":"2024-09-10T18:49:03.220433+0000","last_undegraded":"2024-09-10T18:49:03.220433+0000","last_fullsized":"2024-09-10T18:49:03.220433+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:46:51.114455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:51.114455+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022557979999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":40,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.221304+0000","last_change":"2024-09-10T18:47:36.420713+0000","last_active":"2024-09-10T18:49:03.221304+0000","last_peered":"2024-09-10T18:49:03.221304+0000","last_clean":"2024-09-10T18:49:03.221304+0000","last_became_active":"2024-09-10T18:47:36.420475+0000","last_became_peered":"2024-09-10T18:47:36.420475+0000","last_unstale":"2024-09-10T18:49:03.221304+0000","last_undegraded":"2024-09-10T18:49:03.221304+0000","last_fullsized":"2024-09-10T18:49:03.221304+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:20.774289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:17.819909+0000","last_clean_scrub_stamp":"2024-09-10T18:47:20.774289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"83'9","reported_seq":143,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:06.121005+0000","last_change":"2024-09-10T18:48:33.683570+0000","last_active":"2024-09-10T18:49:06.121005+0000","last_peered":"2024-09-10T18:49:06.121005+0000","last_clean":"2024-09-10T18:49:06.121005+0000","last_became_active":"2024-09-10T18:45:22.073383+0000","last_became_peered":"2024-09-10T18:45:22.073383+0000","last_unstale":"2024-09-10T18:49:06.121005+0000","last_undegraded":"2024-09-10T18:49:06.121005+0000","last_fullsized":"2024-09-10T18:49:06.121005+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'9","last_scrub_stamp":"2024-09-10T18:48:33.683477+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-09-10T18:47:02.582229+0000","last_clean_scrub_stamp":"2024-09-10T18:48:33.683477+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:57.226708+0000","scrub_duration":0.0026989119999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":72,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.220487+0000","last_change":"2024-09-10T18:46:45.083522+0000","last_active":"2024-09-10T18:49:03.220487+0000","last_peered":"2024-09-10T18:49:03.220487+0000","last_clean":"2024-09-10T18:49:03.220487+0000","last_became_active":"2024-09-10T18:45:24.265256+0000","last_became_peered":"2024-09-10T18:45:24.265256+0000","last_unstale":"2024-09-10T18:49:03.220487+0000","last_undegraded":"2024-09-10T18:49:03.220487+0000","last_fullsized":"2024-09-10T18:49:03.220487+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_clean_scrub_stamp":"2024-09-10T18:46:45.083425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017589890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.a","version":"116'8451","reported_seq":8499,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T18:49:11.172022+0000","last_change":"2024-09-10T18:47:36.423957+0000","last_active":"2024-09-10T18:49:11.172022+0000","last_peered":"2024-09-10T18:49:11.172022+0000","last_clean":"2024-09-10T18:49:11.172022+0000","last_became_active":"2024-09-10T18:47:36.422868+0000","last_became_peered":"2024-09-10T18:47:36.422868+0000","last_unstale":"2024-09-10T18:49:11.172022+0000","last_undegraded":"2024-09-10T18:49:11.172022+0000","last_fullsized":"2024-09-10T18:49:11.172022+0000","mapping_epoch":88,"log_start":"103'4200","ondisk_log_start":"103'4200","created":86,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":4251,"log_dups_size":0,"ondisk_log_size":4251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":553844736,"num_objects":8451,"num_object_clones":0,"num_object_copies":25353,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8451,"num_write_kb":540864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"63'12","reported_seq":109,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.210775+0000","last_change":"2024-09-10T18:47:35.928110+0000","last_active":"2024-09-10T18:49:03.210775+0000","last_peered":"2024-09-10T18:49:03.210775+0000","last_clean":"2024-09-10T18:49:03.210775+0000","last_became_active":"2024-09-10T18:45:20.056869+0000","last_became_peered":"2024-09-10T18:45:20.056869+0000","last_unstale":"2024-09-10T18:49:03.210775+0000","last_undegraded":"2024-09-10T18:49:03.210775+0000","last_fullsized":"2024-09-10T18:49:03.210775+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:47:35.928040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:35.928040+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021332539999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"56'2","reported_seq":95,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.211431+0000","last_change":"2024-09-10T18:46:42.303648+0000","last_active":"2024-09-10T18:49:03.211431+0000","last_peered":"2024-09-10T18:49:03.211431+0000","last_clean":"2024-09-10T18:49:03.211431+0000","last_became_active":"2024-09-10T18:45:17.963286+0000","last_became_peered":"2024-09-10T18:45:17.963286+0000","last_unstale":"2024-09-10T18:49:03.211431+0000","last_undegraded":"2024-09-10T18:49:03.211431+0000","last_fullsized":"2024-09-10T18:49:03.211431+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-10T18:46:42.303567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:42.303567+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020684750000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":50,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.213585+0000","last_change":"2024-09-10T18:48:15.763428+0000","last_active":"2024-09-10T18:49:03.213585+0000","last_peered":"2024-09-10T18:49:03.213585+0000","last_clean":"2024-09-10T18:49:03.213585+0000","last_became_active":"2024-09-10T18:47:36.421151+0000","last_became_peered":"2024-09-10T18:47:36.421151+0000","last_unstale":"2024-09-10T18:49:03.213585+0000","last_undegraded":"2024-09-10T18:49:03.213585+0000","last_fullsized":"2024-09-10T18:49:03.213585+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:15.763348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:15.763348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:38.665471+0000","scrub_duration":0.0046035929999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":73,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.219770+0000","last_change":"2024-09-10T18:47:18.797303+0000","last_active":"2024-09-10T18:49:03.219770+0000","last_peered":"2024-09-10T18:49:03.219770+0000","last_clean":"2024-09-10T18:49:03.219770+0000","last_became_active":"2024-09-10T18:45:24.077815+0000","last_became_peered":"2024-09-10T18:45:24.077815+0000","last_unstale":"2024-09-10T18:49:03.219770+0000","last_undegraded":"2024-09-10T18:49:03.219770+0000","last_fullsized":"2024-09-10T18:49:03.219770+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:18.797245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:18.797245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015800090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"76'13","reported_seq":107,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.214919+0000","last_change":"2024-09-10T18:48:10.773290+0000","last_active":"2024-09-10T18:49:03.214919+0000","last_peered":"2024-09-10T18:49:03.214919+0000","last_clean":"2024-09-10T18:49:03.214919+0000","last_became_active":"2024-09-10T18:45:20.420421+0000","last_became_peered":"2024-09-10T18:45:20.420421+0000","last_unstale":"2024-09-10T18:49:03.214919+0000","last_undegraded":"2024-09-10T18:49:03.214919+0000","last_fullsized":"2024-09-10T18:49:03.214919+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'13","last_scrub_stamp":"2024-09-10T18:48:10.773207+0000","last_deep_scrub":"76'13","last_deep_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_clean_scrub_stamp":"2024-09-10T18:48:10.773207+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:35.455517+0000","scrub_duration":0.008348678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":50,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.220230+0000","last_change":"2024-09-10T18:48:14.699173+0000","last_active":"2024-09-10T18:49:03.220230+0000","last_peered":"2024-09-10T18:49:03.220230+0000","last_clean":"2024-09-10T18:49:03.220230+0000","last_became_active":"2024-09-10T18:47:36.409299+0000","last_became_peered":"2024-09-10T18:47:36.409299+0000","last_unstale":"2024-09-10T18:49:03.220230+0000","last_undegraded":"2024-09-10T18:49:03.220230+0000","last_fullsized":"2024-09-10T18:49:03.220230+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:14.695946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:14.695946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:15.444972+0000","scrub_duration":0.007154171,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":85,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.224094+0000","last_change":"2024-09-10T18:48:28.442445+0000","last_active":"2024-09-10T18:49:03.224094+0000","last_peered":"2024-09-10T18:49:03.224094+0000","last_clean":"2024-09-10T18:49:03.224094+0000","last_became_active":"2024-09-10T18:45:22.069040+0000","last_became_peered":"2024-09-10T18:45:22.069040+0000","last_unstale":"2024-09-10T18:49:03.224094+0000","last_undegraded":"2024-09-10T18:49:03.224094+0000","last_fullsized":"2024-09-10T18:49:03.224094+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:28.442370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:28.442370+0000","last_clean_scrub_stamp":"2024-09-10T18:48:28.442370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:50.109389+0000","scrub_duration":0.0098412350000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"63'1","reported_seq":52,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.211115+0000","last_change":"2024-09-10T18:47:36.423902+0000","last_active":"2024-09-10T18:49:03.211115+0000","last_peered":"2024-09-10T18:49:03.211115+0000","last_clean":"2024-09-10T18:49:03.211115+0000","last_became_active":"2024-09-10T18:47:36.406936+0000","last_became_peered":"2024-09-10T18:47:36.406936+0000","last_unstale":"2024-09-10T18:49:03.211115+0000","last_undegraded":"2024-09-10T18:49:03.211115+0000","last_fullsized":"2024-09-10T18:49:03.211115+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-10T18:46:56.721716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:56.721716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":88,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.220107+0000","last_change":"2024-09-10T18:48:24.571499+0000","last_active":"2024-09-10T18:49:03.220107+0000","last_peered":"2024-09-10T18:49:03.220107+0000","last_clean":"2024-09-10T18:49:03.220107+0000","last_became_active":"2024-09-10T18:45:17.964705+0000","last_became_peered":"2024-09-10T18:45:17.964705+0000","last_unstale":"2024-09-10T18:49:03.220107+0000","last_undegraded":"2024-09-10T18:49:03.220107+0000","last_fullsized":"2024-09-10T18:49:03.220107+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:24.571419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:24.571419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:51.133227+0000","scrub_duration":0.005895399,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'6","reported_seq":86,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.224208+0000","last_change":"2024-09-10T18:47:02.700853+0000","last_active":"2024-09-10T18:49:03.224208+0000","last_peered":"2024-09-10T18:49:03.224208+0000","last_clean":"2024-09-10T18:49:03.224208+0000","last_became_active":"2024-09-10T18:45:20.421746+0000","last_became_peered":"2024-09-10T18:45:20.421746+0000","last_unstale":"2024-09-10T18:49:03.224208+0000","last_undegraded":"2024-09-10T18:49:03.224208+0000","last_fullsized":"2024-09-10T18:49:03.224208+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'6","last_scrub_stamp":"2024-09-10T18:47:02.700787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:02.700787+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017032519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":74,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.221485+0000","last_change":"2024-09-10T18:46:39.084489+0000","last_active":"2024-09-10T18:49:03.221485+0000","last_peered":"2024-09-10T18:49:03.221485+0000","last_clean":"2024-09-10T18:49:03.221485+0000","last_became_active":"2024-09-10T18:45:22.069653+0000","last_became_peered":"2024-09-10T18:45:22.069653+0000","last_unstale":"2024-09-10T18:49:03.221485+0000","last_undegraded":"2024-09-10T18:49:03.221485+0000","last_fullsized":"2024-09-10T18:49:03.221485+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:39.084433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:46:39.084433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014845819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":83,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.214349+0000","last_change":"2024-09-10T18:48:32.654862+0000","last_active":"2024-09-10T18:49:03.214349+0000","last_peered":"2024-09-10T18:49:03.214349+0000","last_clean":"2024-09-10T18:49:03.214349+0000","last_became_active":"2024-09-10T18:45:24.264611+0000","last_became_peered":"2024-09-10T18:45:24.264611+0000","last_unstale":"2024-09-10T18:49:03.214349+0000","last_undegraded":"2024-09-10T18:49:03.214349+0000","last_fullsized":"2024-09-10T18:49:03.214349+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:32.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:32.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:01.743394+0000","scrub_duration":0.0049870009999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"63'10","reported_seq":146,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.220144+0000","last_change":"2024-09-10T18:49:01.556782+0000","last_active":"2024-09-10T18:49:03.220144+0000","last_peered":"2024-09-10T18:49:03.220144+0000","last_clean":"2024-09-10T18:49:03.220144+0000","last_became_active":"2024-09-10T18:47:36.426260+0000","last_became_peered":"2024-09-10T18:47:36.426260+0000","last_unstale":"2024-09-10T18:49:03.220144+0000","last_undegraded":"2024-09-10T18:49:03.220144+0000","last_fullsized":"2024-09-10T18:49:03.220144+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:49:01.556694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:49:01.556694+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:23.957988+0000","scrub_duration":0.0053223699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":88,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:06.079748+0000","last_change":"2024-09-10T18:49:06.079691+0000","last_active":"2024-09-10T18:49:06.079748+0000","last_peered":"2024-09-10T18:49:06.079748+0000","last_clean":"2024-09-10T18:49:06.079748+0000","last_became_active":"2024-09-10T18:45:17.964870+0000","last_became_peered":"2024-09-10T18:45:17.964870+0000","last_unstale":"2024-09-10T18:49:06.079748+0000","last_undegraded":"2024-09-10T18:49:06.079748+0000","last_fullsized":"2024-09-10T18:49:06.079748+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:06.079456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:06.079456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:12.151525+0000","scrub_duration":0.012719522,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"63'8","reported_seq":138,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:06.120692+0000","last_change":"2024-09-10T18:49:02.038795+0000","last_active":"2024-09-10T18:49:06.120692+0000","last_peered":"2024-09-10T18:49:06.120692+0000","last_clean":"2024-09-10T18:49:06.120692+0000","last_became_active":"2024-09-10T18:45:22.062940+0000","last_became_peered":"2024-09-10T18:45:22.062940+0000","last_unstale":"2024-09-10T18:49:06.120692+0000","last_undegraded":"2024-09-10T18:49:06.120692+0000","last_fullsized":"2024-09-10T18:49:06.120692+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:49:02.038709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:49:02.038709+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:15.977699+0000","scrub_duration":0.007468054,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":83,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.552126+0000","last_change":"2024-09-10T18:49:03.552056+0000","last_active":"2024-09-10T18:49:03.552126+0000","last_peered":"2024-09-10T18:49:03.552126+0000","last_clean":"2024-09-10T18:49:03.552126+0000","last_became_active":"2024-09-10T18:45:24.073736+0000","last_became_peered":"2024-09-10T18:45:24.073736+0000","last_unstale":"2024-09-10T18:49:03.552126+0000","last_undegraded":"2024-09-10T18:49:03.552126+0000","last_fullsized":"2024-09-10T18:49:03.552126+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:03.551966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:03.551966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:11.246649+0000","scrub_duration":0.015480603000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"63'9","reported_seq":102,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.221109+0000","last_change":"2024-09-10T18:48:19.076178+0000","last_active":"2024-09-10T18:49:03.221109+0000","last_peered":"2024-09-10T18:49:03.221109+0000","last_clean":"2024-09-10T18:49:03.221109+0000","last_became_active":"2024-09-10T18:45:20.421124+0000","last_became_peered":"2024-09-10T18:45:20.421124+0000","last_unstale":"2024-09-10T18:49:03.221109+0000","last_undegraded":"2024-09-10T18:49:03.221109+0000","last_fullsized":"2024-09-10T18:49:03.221109+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:48:19.076099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:19.076099+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:32.177242+0000","scrub_duration":0.002691981,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":89,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.214768+0000","last_change":"2024-09-10T18:48:33.663516+0000","last_active":"2024-09-10T18:49:03.214768+0000","last_peered":"2024-09-10T18:49:03.214768+0000","last_clean":"2024-09-10T18:49:03.214768+0000","last_became_active":"2024-09-10T18:45:17.960106+0000","last_became_peered":"2024-09-10T18:45:17.960106+0000","last_unstale":"2024-09-10T18:49:03.214768+0000","last_undegraded":"2024-09-10T18:49:03.214768+0000","last_fullsized":"2024-09-10T18:49:03.214768+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:33.663415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:33.663415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:33.950247+0000","scrub_duration":0.002167352,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"89'10","reported_seq":145,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-10T18:49:08.264855+0000","last_change":"2024-09-10T18:47:36.434102+0000","last_active":"2024-09-10T18:49:08.264855+0000","last_peered":"2024-09-10T18:49:08.264855+0000","last_clean":"2024-09-10T18:49:08.264855+0000","last_became_active":"2024-09-10T18:47:36.424575+0000","last_became_peered":"2024-09-10T18:47:36.424575+0000","last_unstale":"2024-09-10T18:49:08.264855+0000","last_undegraded":"2024-09-10T18:49:08.264855+0000","last_fullsized":"2024-09-10T18:49:08.264855+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"81'9","last_scrub_stamp":"2024-09-10T18:46:54.800790+0000","last_deep_scrub":"81'9","last_deep_scrub_stamp":"2024-09-10T18:46:54.800790+0000","last_clean_scrub_stamp":"2024-09-10T18:46:54.800790+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034208709999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":92,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.211408+0000","last_change":"2024-09-10T18:48:34.045421+0000","last_active":"2024-09-10T18:49:03.211408+0000","last_peered":"2024-09-10T18:49:03.211408+0000","last_clean":"2024-09-10T18:49:03.211408+0000","last_became_active":"2024-09-10T18:47:36.414047+0000","last_became_peered":"2024-09-10T18:47:36.414047+0000","last_unstale":"2024-09-10T18:49:03.211408+0000","last_undegraded":"2024-09-10T18:49:03.211408+0000","last_fullsized":"2024-09-10T18:49:03.211408+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:34.045311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_clean_scrub_stamp":"2024-09-10T18:48:34.045311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:49.035924+0000","scrub_duration":0.0048840200000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"63'9","reported_seq":123,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:05.359189+0000","last_change":"2024-09-10T18:49:05.359047+0000","last_active":"2024-09-10T18:49:05.359189+0000","last_peered":"2024-09-10T18:49:05.359189+0000","last_clean":"2024-09-10T18:49:05.359189+0000","last_became_active":"2024-09-10T18:47:36.416484+0000","last_became_peered":"2024-09-10T18:47:36.416484+0000","last_unstale":"2024-09-10T18:49:05.359189+0000","last_undegraded":"2024-09-10T18:49:05.359189+0000","last_fullsized":"2024-09-10T18:49:05.359189+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:49:05.358887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:49:05.358887+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:26.279687+0000","scrub_duration":0.013783619,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":98,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:04.430508+0000","last_change":"2024-09-10T18:49:04.430421+0000","last_active":"2024-09-10T18:49:04.430508+0000","last_peered":"2024-09-10T18:49:04.430508+0000","last_clean":"2024-09-10T18:49:04.430508+0000","last_became_active":"2024-09-10T18:45:17.962009+0000","last_became_peered":"2024-09-10T18:45:17.962009+0000","last_unstale":"2024-09-10T18:49:04.430508+0000","last_undegraded":"2024-09-10T18:49:04.430508+0000","last_fullsized":"2024-09-10T18:49:04.430508+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:04.430344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:04.430344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:13.393531+0000","scrub_duration":0.0025461400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":75,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.219832+0000","last_change":"2024-09-10T18:47:14.869132+0000","last_active":"2024-09-10T18:49:03.219832+0000","last_peered":"2024-09-10T18:49:03.219832+0000","last_clean":"2024-09-10T18:49:03.219832+0000","last_became_active":"2024-09-10T18:45:22.072877+0000","last_became_peered":"2024-09-10T18:45:22.072877+0000","last_unstale":"2024-09-10T18:49:03.219832+0000","last_undegraded":"2024-09-10T18:49:03.219832+0000","last_fullsized":"2024-09-10T18:49:03.219832+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:14.869076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:47:14.869076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00131624,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":83,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.213407+0000","last_change":"2024-09-10T18:48:32.647084+0000","last_active":"2024-09-10T18:49:03.213407+0000","last_peered":"2024-09-10T18:49:03.213407+0000","last_clean":"2024-09-10T18:49:03.213407+0000","last_became_active":"2024-09-10T18:45:24.081050+0000","last_became_peered":"2024-09-10T18:45:24.081050+0000","last_unstale":"2024-09-10T18:49:03.213407+0000","last_undegraded":"2024-09-10T18:49:03.213407+0000","last_fullsized":"2024-09-10T18:49:03.213407+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:32.646988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:32.646988+0000","last_clean_scrub_stamp":"2024-09-10T18:48:32.646988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:39.006371+0000","scrub_duration":0.0038989210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"63'11","reported_seq":95,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.211108+0000","last_change":"2024-09-10T18:47:14.034427+0000","last_active":"2024-09-10T18:49:03.211108+0000","last_peered":"2024-09-10T18:49:03.211108+0000","last_clean":"2024-09-10T18:49:03.211108+0000","last_became_active":"2024-09-10T18:45:20.056885+0000","last_became_peered":"2024-09-10T18:45:20.056885+0000","last_unstale":"2024-09-10T18:49:03.211108+0000","last_undegraded":"2024-09-10T18:49:03.211108+0000","last_fullsized":"2024-09-10T18:49:03.211108+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:47:14.034361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:14.034361+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018564,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":89,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.221976+0000","last_change":"2024-09-10T18:48:32.517573+0000","last_active":"2024-09-10T18:49:03.221976+0000","last_peered":"2024-09-10T18:49:03.221976+0000","last_clean":"2024-09-10T18:49:03.221976+0000","last_became_active":"2024-09-10T18:45:17.963542+0000","last_became_peered":"2024-09-10T18:45:17.963542+0000","last_unstale":"2024-09-10T18:49:03.221976+0000","last_undegraded":"2024-09-10T18:49:03.221976+0000","last_fullsized":"2024-09-10T18:49:03.221976+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:32.517513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:32.517513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:48.139582+0000","scrub_duration":0.013400413999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":85,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.220578+0000","last_change":"2024-09-10T18:48:09.725726+0000","last_active":"2024-09-10T18:49:03.220578+0000","last_peered":"2024-09-10T18:49:03.220578+0000","last_clean":"2024-09-10T18:49:03.220578+0000","last_became_active":"2024-09-10T18:45:22.070065+0000","last_became_peered":"2024-09-10T18:45:22.070065+0000","last_unstale":"2024-09-10T18:49:03.220578+0000","last_undegraded":"2024-09-10T18:49:03.220578+0000","last_fullsized":"2024-09-10T18:49:03.220578+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:09.725488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:09.725488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:18.246207+0000","scrub_duration":0.0027892020000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":73,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.218878+0000","last_change":"2024-09-10T18:47:05.789893+0000","last_active":"2024-09-10T18:49:03.218878+0000","last_peered":"2024-09-10T18:49:03.218878+0000","last_clean":"2024-09-10T18:49:03.218878+0000","last_became_active":"2024-09-10T18:45:24.072224+0000","last_became_peered":"2024-09-10T18:45:24.072224+0000","last_unstale":"2024-09-10T18:49:03.218878+0000","last_undegraded":"2024-09-10T18:49:03.218878+0000","last_fullsized":"2024-09-10T18:49:03.218878+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:05.789830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:05.789830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001271938,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"63'4","reported_seq":93,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.218786+0000","last_change":"2024-09-10T18:48:30.455636+0000","last_active":"2024-09-10T18:49:03.218786+0000","last_peered":"2024-09-10T18:49:03.218786+0000","last_clean":"2024-09-10T18:49:03.218786+0000","last_became_active":"2024-09-10T18:45:20.059205+0000","last_became_peered":"2024-09-10T18:45:20.059205+0000","last_unstale":"2024-09-10T18:49:03.218786+0000","last_undegraded":"2024-09-10T18:49:03.218786+0000","last_fullsized":"2024-09-10T18:49:03.218786+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4","last_scrub_stamp":"2024-09-10T18:48:30.455433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:30.455433+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:32.585527+0000","scrub_duration":0.0078514339999999991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":88,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.221533+0000","last_change":"2024-09-10T18:47:36.416170+0000","last_active":"2024-09-10T18:49:03.221533+0000","last_peered":"2024-09-10T18:49:03.221533+0000","last_clean":"2024-09-10T18:49:03.221533+0000","last_became_active":"2024-09-10T18:47:36.415866+0000","last_became_peered":"2024-09-10T18:47:36.415866+0000","last_unstale":"2024-09-10T18:49:03.221533+0000","last_undegraded":"2024-09-10T18:49:03.221533+0000","last_fullsized":"2024-09-10T18:49:03.221533+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:54.082761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:54.082761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001210458,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":84,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.219839+0000","last_change":"2024-09-10T18:48:27.535769+0000","last_active":"2024-09-10T18:49:03.219839+0000","last_peered":"2024-09-10T18:49:03.219839+0000","last_clean":"2024-09-10T18:49:03.219839+0000","last_became_active":"2024-09-10T18:45:22.072377+0000","last_became_peered":"2024-09-10T18:45:22.072377+0000","last_unstale":"2024-09-10T18:49:03.219839+0000","last_undegraded":"2024-09-10T18:49:03.219839+0000","last_fullsized":"2024-09-10T18:49:03.219839+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:27.535676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_clean_scrub_stamp":"2024-09-10T18:48:27.535676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:36.593031+0000","scrub_duration":0.002291132,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":93,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.213922+0000","last_change":"2024-09-10T18:48:28.711019+0000","last_active":"2024-09-10T18:49:03.213922+0000","last_peered":"2024-09-10T18:49:03.213922+0000","last_clean":"2024-09-10T18:49:03.213922+0000","last_became_active":"2024-09-10T18:47:36.416187+0000","last_became_peered":"2024-09-10T18:47:36.416187+0000","last_unstale":"2024-09-10T18:49:03.213922+0000","last_undegraded":"2024-09-10T18:49:03.213922+0000","last_fullsized":"2024-09-10T18:49:03.213922+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:28.710918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:28.710918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:37.560639+0000","scrub_duration":0.0068877160000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"79'16","reported_seq":111,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.566295+0000","last_change":"2024-09-10T18:49:03.566236+0000","last_active":"2024-09-10T18:49:03.566295+0000","last_peered":"2024-09-10T18:49:03.566295+0000","last_clean":"2024-09-10T18:49:03.566295+0000","last_became_active":"2024-09-10T18:45:20.421479+0000","last_became_peered":"2024-09-10T18:45:20.421479+0000","last_unstale":"2024-09-10T18:49:03.566295+0000","last_undegraded":"2024-09-10T18:49:03.566295+0000","last_fullsized":"2024-09-10T18:49:03.566295+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"79'16","last_scrub_stamp":"2024-09-10T18:49:03.566152+0000","last_deep_scrub":"79'16","last_deep_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_clean_scrub_stamp":"2024-09-10T18:49:03.566152+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:21.649879+0000","scrub_duration":0.0032760770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"56'1","reported_seq":109,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.219647+0000","last_change":"2024-09-10T18:48:20.603918+0000","last_active":"2024-09-10T18:49:03.219647+0000","last_peered":"2024-09-10T18:49:03.219647+0000","last_clean":"2024-09-10T18:49:03.219647+0000","last_became_active":"2024-09-10T18:47:36.421985+0000","last_became_peered":"2024-09-10T18:47:36.421985+0000","last_unstale":"2024-09-10T18:49:03.219647+0000","last_undegraded":"2024-09-10T18:49:03.219647+0000","last_fullsized":"2024-09-10T18:49:03.219647+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:48:20.603805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:20.603805+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:49.866219+0000","scrub_duration":0.0039707509999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":39,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.218353+0000","last_change":"2024-09-10T18:48:09.323873+0000","last_active":"2024-09-10T18:49:03.218353+0000","last_peered":"2024-09-10T18:49:03.218353+0000","last_clean":"2024-09-10T18:49:03.218353+0000","last_became_active":"2024-09-10T18:47:57.107149+0000","last_became_peered":"2024-09-10T18:47:57.107149+0000","last_unstale":"2024-09-10T18:49:03.218353+0000","last_undegraded":"2024-09-10T18:49:03.218353+0000","last_fullsized":"2024-09-10T18:49:03.218353+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:09.323794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:09.323794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:28.508379+0000","scrub_duration":0.0056521669999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":83,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.219706+0000","last_change":"2024-09-10T18:48:16.410239+0000","last_active":"2024-09-10T18:49:03.219706+0000","last_peered":"2024-09-10T18:49:03.219706+0000","last_clean":"2024-09-10T18:49:03.219706+0000","last_became_active":"2024-09-10T18:45:24.264818+0000","last_became_peered":"2024-09-10T18:45:24.264818+0000","last_unstale":"2024-09-10T18:49:03.219706+0000","last_undegraded":"2024-09-10T18:49:03.219706+0000","last_fullsized":"2024-09-10T18:49:03.219706+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:16.410071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:16.410071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:22.248050+0000","scrub_duration":0.0061840250000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"78'2","reported_seq":99,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.214522+0000","last_change":"2024-09-10T18:48:10.610714+0000","last_active":"2024-09-10T18:49:03.214522+0000","last_peered":"2024-09-10T18:49:03.214522+0000","last_clean":"2024-09-10T18:49:03.214522+0000","last_became_active":"2024-09-10T18:47:36.418702+0000","last_became_peered":"2024-09-10T18:47:36.418702+0000","last_unstale":"2024-09-10T18:49:03.214522+0000","last_undegraded":"2024-09-10T18:49:03.214522+0000","last_fullsized":"2024-09-10T18:49:03.214522+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'2","last_scrub_stamp":"2024-09-10T18:48:10.610652+0000","last_deep_scrub":"78'2","last_deep_scrub_stamp":"2024-09-10T18:46:37.581130+0000","last_clean_scrub_stamp":"2024-09-10T18:48:10.610652+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:24.088784+0000","scrub_duration":0.002045275,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":88,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:07.088498+0000","last_change":"2024-09-10T18:49:07.088445+0000","last_active":"2024-09-10T18:49:07.088498+0000","last_peered":"2024-09-10T18:49:07.088498+0000","last_clean":"2024-09-10T18:49:07.088498+0000","last_became_active":"2024-09-10T18:45:17.962108+0000","last_became_peered":"2024-09-10T18:45:17.962108+0000","last_unstale":"2024-09-10T18:49:07.088498+0000","last_undegraded":"2024-09-10T18:49:07.088498+0000","last_fullsized":"2024-09-10T18:49:07.088498+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:07.088371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:07.088371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:26.877956+0000","scrub_duration":0.0013360100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"63'10","reported_seq":102,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.214842+0000","last_change":"2024-09-10T18:48:23.844841+0000","last_active":"2024-09-10T18:49:03.214842+0000","last_peered":"2024-09-10T18:49:03.214842+0000","last_clean":"2024-09-10T18:49:03.214842+0000","last_became_active":"2024-09-10T18:45:20.421094+0000","last_became_peered":"2024-09-10T18:45:20.421094+0000","last_unstale":"2024-09-10T18:49:03.214842+0000","last_undegraded":"2024-09-10T18:49:03.214842+0000","last_fullsized":"2024-09-10T18:49:03.214842+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:48:23.844741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:23.844741+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:49.178164+0000","scrub_duration":0.0025648530000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":141,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:06.120657+0000","last_change":"2024-09-10T18:48:19.605745+0000","last_active":"2024-09-10T18:49:06.120657+0000","last_peered":"2024-09-10T18:49:06.120657+0000","last_clean":"2024-09-10T18:49:06.120657+0000","last_became_active":"2024-09-10T18:45:22.073281+0000","last_became_peered":"2024-09-10T18:45:22.073281+0000","last_unstale":"2024-09-10T18:49:06.120657+0000","last_undegraded":"2024-09-10T18:49:06.120657+0000","last_fullsized":"2024-09-10T18:49:06.120657+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:48:19.605646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:19.605646+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:22.732603+0000","scrub_duration":0.001752665,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":73,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.221468+0000","last_change":"2024-09-10T18:47:03.475367+0000","last_active":"2024-09-10T18:49:03.221468+0000","last_peered":"2024-09-10T18:49:03.221468+0000","last_clean":"2024-09-10T18:49:03.221468+0000","last_became_active":"2024-09-10T18:45:24.082318+0000","last_became_peered":"2024-09-10T18:45:24.082318+0000","last_unstale":"2024-09-10T18:49:03.221468+0000","last_undegraded":"2024-09-10T18:49:03.221468+0000","last_fullsized":"2024-09-10T18:49:03.221468+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:03.475299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:03.475299+0000","last_clean_scrub_stamp":"2024-09-10T18:47:03.475299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001661945,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":89,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:07.526592+0000","last_change":"2024-09-10T18:49:07.526539+0000","last_active":"2024-09-10T18:49:07.526592+0000","last_peered":"2024-09-10T18:49:07.526592+0000","last_clean":"2024-09-10T18:49:07.526592+0000","last_became_active":"2024-09-10T18:45:17.965707+0000","last_became_peered":"2024-09-10T18:45:17.965707+0000","last_unstale":"2024-09-10T18:49:07.526592+0000","last_undegraded":"2024-09-10T18:49:07.526592+0000","last_fullsized":"2024-09-10T18:49:07.526592+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:07.526460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:07.526460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:10.144795+0000","scrub_duration":0.010955793,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"63'11","reported_seq":105,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.219459+0000","last_change":"2024-09-10T18:48:23.546688+0000","last_active":"2024-09-10T18:49:03.219459+0000","last_peered":"2024-09-10T18:49:03.219459+0000","last_clean":"2024-09-10T18:49:03.219459+0000","last_became_active":"2024-09-10T18:45:20.058717+0000","last_became_peered":"2024-09-10T18:45:20.058717+0000","last_unstale":"2024-09-10T18:49:03.219459+0000","last_undegraded":"2024-09-10T18:49:03.219459+0000","last_fullsized":"2024-09-10T18:49:03.219459+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:48:23.546598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:23.546598+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:28.876387+0000","scrub_duration":0.0057630149999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":95,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.214761+0000","last_change":"2024-09-10T18:48:58.543128+0000","last_active":"2024-09-10T18:49:03.214761+0000","last_peered":"2024-09-10T18:49:03.214761+0000","last_clean":"2024-09-10T18:49:03.214761+0000","last_became_active":"2024-09-10T18:47:36.418767+0000","last_became_peered":"2024-09-10T18:47:36.418767+0000","last_unstale":"2024-09-10T18:49:03.214761+0000","last_undegraded":"2024-09-10T18:49:03.214761+0000","last_fullsized":"2024-09-10T18:49:03.214761+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:58.542812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:58.542812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:01.428535+0000","scrub_duration":0.010435696,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":82,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:06.423084+0000","last_change":"2024-09-10T18:49:06.423032+0000","last_active":"2024-09-10T18:49:06.423084+0000","last_peered":"2024-09-10T18:49:06.423084+0000","last_clean":"2024-09-10T18:49:06.423084+0000","last_became_active":"2024-09-10T18:45:24.267251+0000","last_became_peered":"2024-09-10T18:45:24.267251+0000","last_unstale":"2024-09-10T18:49:06.423084+0000","last_undegraded":"2024-09-10T18:49:06.423084+0000","last_fullsized":"2024-09-10T18:49:06.423084+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:06.422952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:06.422952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:33.673505+0000","scrub_duration":0.0017661879999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"56'1","reported_seq":97,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.218720+0000","last_change":"2024-09-10T18:48:31.488364+0000","last_active":"2024-09-10T18:49:03.218720+0000","last_peered":"2024-09-10T18:49:03.218720+0000","last_clean":"2024-09-10T18:49:03.218720+0000","last_became_active":"2024-09-10T18:45:17.966286+0000","last_became_peered":"2024-09-10T18:45:17.966286+0000","last_unstale":"2024-09-10T18:49:03.218720+0000","last_undegraded":"2024-09-10T18:49:03.218720+0000","last_fullsized":"2024-09-10T18:49:03.218720+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:48:31.488272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:31.488272+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:37.912107+0000","scrub_duration":0.0028000949999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"63'15","reported_seq":93,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.221193+0000","last_change":"2024-09-10T18:47:36.446154+0000","last_active":"2024-09-10T18:49:03.221193+0000","last_peered":"2024-09-10T18:49:03.221193+0000","last_clean":"2024-09-10T18:49:03.221193+0000","last_became_active":"2024-09-10T18:47:36.424373+0000","last_became_peered":"2024-09-10T18:47:36.424373+0000","last_unstale":"2024-09-10T18:49:03.221193+0000","last_undegraded":"2024-09-10T18:49:03.221193+0000","last_fullsized":"2024-09-10T18:49:03.221193+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:47:34.610915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:34.610915+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":75,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.221711+0000","last_change":"2024-09-10T18:47:09.520829+0000","last_active":"2024-09-10T18:49:03.221711+0000","last_peered":"2024-09-10T18:49:03.221711+0000","last_clean":"2024-09-10T18:49:03.221711+0000","last_became_active":"2024-09-10T18:45:22.059794+0000","last_became_peered":"2024-09-10T18:45:22.059794+0000","last_unstale":"2024-09-10T18:49:03.221711+0000","last_undegraded":"2024-09-10T18:49:03.221711+0000","last_fullsized":"2024-09-10T18:49:03.221711+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:09.520768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:47:09.520768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016284120000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":73,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.223988+0000","last_change":"2024-09-10T18:46:57.811148+0000","last_active":"2024-09-10T18:49:03.223988+0000","last_peered":"2024-09-10T18:49:03.223988+0000","last_clean":"2024-09-10T18:49:03.223988+0000","last_became_active":"2024-09-10T18:45:24.264427+0000","last_became_peered":"2024-09-10T18:45:24.264427+0000","last_unstale":"2024-09-10T18:49:03.223988+0000","last_undegraded":"2024-09-10T18:49:03.223988+0000","last_fullsized":"2024-09-10T18:49:03.223988+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:57.811074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:57.811074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013689419999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":79,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.223062+0000","last_change":"2024-09-10T18:46:47.734860+0000","last_active":"2024-09-10T18:49:03.223062+0000","last_peered":"2024-09-10T18:49:03.223062+0000","last_clean":"2024-09-10T18:49:03.223062+0000","last_became_active":"2024-09-10T18:45:17.963257+0000","last_became_peered":"2024-09-10T18:45:17.963257+0000","last_unstale":"2024-09-10T18:49:03.223062+0000","last_undegraded":"2024-09-10T18:49:03.223062+0000","last_fullsized":"2024-09-10T18:49:03.223062+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:47.734803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:47.734803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001381134,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"63'12","reported_seq":100,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.218190+0000","last_change":"2024-09-10T18:46:52.771640+0000","last_active":"2024-09-10T18:49:03.218190+0000","last_peered":"2024-09-10T18:49:03.218190+0000","last_clean":"2024-09-10T18:49:03.218190+0000","last_became_active":"2024-09-10T18:45:20.058391+0000","last_became_peered":"2024-09-10T18:45:20.058391+0000","last_unstale":"2024-09-10T18:49:03.218190+0000","last_undegraded":"2024-09-10T18:49:03.218190+0000","last_fullsized":"2024-09-10T18:49:03.218190+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:46:52.771570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:52.771570+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021490089999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":94,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-10T18:49:03.211253+0000","last_change":"2024-09-10T18:47:36.416505+0000","last_active":"2024-09-10T18:49:03.211253+0000","last_peered":"2024-09-10T18:49:03.211253+0000","last_clean":"2024-09-10T18:49:03.211253+0000","last_became_active":"2024-09-10T18:47:36.416131+0000","last_became_peered":"2024-09-10T18:47:36.416131+0000","last_unstale":"2024-09-10T18:49:03.211253+0000","last_undegraded":"2024-09-10T18:49:03.211253+0000","last_fullsized":"2024-09-10T18:49:03.211253+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:34.958889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:47:34.958889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001401296,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":8698200064,"num_objects":132724,"num_object_clones":0,"num_object_copies":398172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":132724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132724,"num_write_kb":8494336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":26099974144,"data_stored":26099974144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76924,"ondisk_log_size":76924,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":697,"num_read_kb":454,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":1238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":423,"ondisk_log_size":423,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":223,"num_read_kb":218,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299443,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3477680,"kb_used_data":3167848,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":90267472,"statfs":{"total":95995035648,"available":92433891328,"internally_reserved":0,"allocated":3243876352,"data_stored":3243458398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:49:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":0.94499999999999995,"15min":0.65300000000000002},"min":{"1min":0.219,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":3.9929999999999999,"5min":31.805,"15min":31.805},"last":2.0350000000000001},{"interface":"front","average":{"1min":1.8640000000000001,"5min":1.081,"15min":0.68400000000000005},"min":{"1min":0.29399999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":5.4119999999999999,"5min":31.212,"15min":31.212},"last":3.7309999999999999}]},{"osd":1,"last update":"Tue Sep 10 18:49:10 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.55200000000000005,"15min":0.502},"min":{"1min":0.30199999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":2.169,"5min":2.169,"15min":2.169},"last":1.8009999999999999},{"interface":"front","average":{"1min":1.0640000000000001,"5min":0.624,"15min":0.55400000000000005},"min":{"1min":0.255,"5min":0.25,"15min":0.25},"max":{"1min":3.036,"5min":3.036,"15min":3.036},"last":2.02}]},{"osd":2,"last update":"Tue Sep 10 18:49:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":0.60599999999999998,"15min":0.52600000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":5.1399999999999997,"5min":5.1399999999999997,"15min":5.1399999999999997},"last":1.9570000000000001},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.60699999999999998,"15min":0.54600000000000004},"min":{"1min":0.38200000000000001,"5min":0.31,"15min":0.31},"max":{"1min":3.0209999999999999,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":1.774}]},{"osd":3,"last update":"Tue Sep 10 18:49:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":0.95699999999999996,"15min":0.65700000000000003},"min":{"1min":0.26900000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":3.2120000000000002,"5min":31.478000000000002,"15min":31.478000000000002},"last":2.0489999999999999},{"interface":"front","average":{"1min":1.581,"5min":1.1619999999999999,"15min":0.73299999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":5.1020000000000003,"5min":33.731999999999999,"15min":33.731999999999999},"last":1.9039999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:49:10 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.45100000000000001,"15min":0.40699999999999997},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":2.7559999999999998,"5min":2.7559999999999998,"15min":2.7559999999999998},"last":1.996},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.438,"15min":0.39600000000000002},"min":{"1min":0.29599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":1.7150000000000001}]},{"osd":5,"last update":"Tue Sep 10 18:49:10 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.46899999999999997,"15min":0.43099999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":2.7240000000000002,"5min":2.7240000000000002,"15min":2.7240000000000002},"last":2.0760000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.45000000000000001,"15min":0.42699999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.7490000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:49:10 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.52900000000000003,"15min":0.47499999999999998},"min":{"1min":0.34499999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":2.8199999999999998,"5min":2.8199999999999998,"15min":2.8199999999999998},"last":2.1299999999999999},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.52400000000000002,"15min":0.46300000000000002},"min":{"1min":0.33700000000000002,"5min":0.309,"15min":0.309},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.9319999999999999}]}]},{"osd":6,"up_from":43,"seq":184683593784,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3476408,"kb_used_data":3170736,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":90268744,"statfs":{"total":95995035648,"available":92435193856,"internally_reserved":0,"allocated":3246833664,"data_stored":3246408062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:48:46 2024","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":0.64900000000000002,"15min":0.52100000000000002},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":4.681,"5min":4.681,"15min":4.681},"last":2.0179999999999998},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.58799999999999997,"15min":0.52300000000000002},"min":{"1min":0.315,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":3.1709999999999998,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":0.41199999999999998}]},{"osd":1,"last update":"Tue Sep 10 18:48:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.55100000000000005,"15min":0.50900000000000001},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.24299999999999999},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.56899999999999995,"15min":0.497},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":3.6280000000000001,"5min":3.6280000000000001,"15min":3.6280000000000001},"last":0.63700000000000001}]},{"osd":2,"last update":"Tue Sep 10 18:48:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":0.58699999999999997,"15min":0.51600000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":3.359,"5min":3.359,"15min":3.359},"last":0.435},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.52000000000000002,"15min":0.5},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.76600000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:48:46 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.51900000000000002,"15min":0.495},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.069,"5min":0.628,"15min":0.54900000000000004},"min":{"1min":0.33600000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":4.5309999999999997,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":0.75800000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:48:46 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.41799999999999998,"15min":0.40899999999999997},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.41899999999999998,"15min":0.39500000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.32700000000000001}]},{"osd":5,"last update":"Tue Sep 10 18:48:46 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.439,"15min":0.42199999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.46300000000000002,"15min":0.435},"min":{"1min":0.29799999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":0.313}]},{"osd":7,"last update":"Tue Sep 10 18:49:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.52400000000000002,"15min":0.48699999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.367,"5min":1.556,"15min":1.556},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.53700000000000003,"15min":0.50800000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.3320000000000001,"5min":1.883,"15min":1.883},"last":0.39900000000000002}]}]},{"osd":5,"up_from":37,"seq":158913790012,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2425576,"kb_used_data":2128224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":91319576,"statfs":{"total":95995035648,"available":93511245824,"internally_reserved":0,"allocated":2179301376,"data_stored":2178890538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:48:22 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.53900000000000003,"15min":0.53100000000000003},"min":{"1min":0.376,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.67100000000000004},{"interface":"front","average":{"1min":1.2789999999999999,"5min":0.64000000000000001,"15min":0.54600000000000004},"min":{"1min":0.40400000000000003,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":6.742,"5min":6.742,"15min":6.742},"last":0.46400000000000002}]},{"osd":1,"last update":"Tue Sep 10 18:48:22 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.746,"5min":1.746,"15min":1.746},"last":0.33300000000000002},{"interface":"front","average":{"1min":1.224,"5min":0.65900000000000003,"15min":0.58099999999999996},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":6.6989999999999998,"5min":6.6989999999999998,"15min":6.6989999999999998},"last":0.307}]},{"osd":2,"last update":"Tue Sep 10 18:48:22 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.50800000000000001,"15min":0.51600000000000001},"min":{"1min":0.34799999999999998,"5min":0.311,"15min":0.311},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":1.3700000000000001,"5min":0.68799999999999994,"15min":0.59199999999999997},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":6.7210000000000001,"5min":6.7210000000000001,"15min":6.7210000000000001},"last":0.40200000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:48:22 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":0.624,"15min":0.56000000000000005},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":3.6760000000000002,"5min":3.6760000000000002,"15min":3.6760000000000002},"last":0.64000000000000001},{"interface":"front","average":{"1min":1.3999999999999999,"5min":0.69699999999999995,"15min":0.59699999999999998},"min":{"1min":0.372,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":5.8099999999999996,"5min":5.8099999999999996,"15min":5.8099999999999996},"last":0.77900000000000003}]},{"osd":4,"last update":"Tue Sep 10 18:48:22 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.39000000000000001,"15min":0.38500000000000001},"min":{"1min":0.248,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.38800000000000001,"15min":0.39800000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.502,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.56100000000000005}]},{"osd":6,"last update":"Tue Sep 10 18:48:47 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.49099999999999999,"15min":0.439},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.53600000000000003,"15min":0.48399999999999999},"min":{"1min":0.33000000000000002,"5min":0.31,"15min":0.31},"max":{"1min":2.2599999999999998,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":0.66200000000000003}]},{"osd":7,"last update":"Tue Sep 10 18:48:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.47099999999999997,"15min":0.47499999999999998},"min":{"1min":0.32100000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.62,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.46999999999999997,"15min":0.47499999999999998},"min":{"1min":0.30099999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.36599999999999999}]}]},{"osd":4,"up_from":30,"seq":128849018946,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2963760,"kb_used_data":2649768,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":90781392,"statfs":{"total":95995035648,"available":92960145408,"internally_reserved":0,"allocated":2713362432,"data_stored":2712943785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:49:05 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.624,"15min":0.55800000000000005},"min":{"1min":0.215,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":7.8730000000000002,"5min":7.8730000000000002,"15min":7.8730000000000002},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.58799999999999997,"15min":0.55800000000000005},"min":{"1min":0.23999999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":3.3279999999999998,"5min":3.3279999999999998,"15min":3.3279999999999998},"last":0.56799999999999995}]},{"osd":1,"last update":"Tue Sep 10 18:49:05 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.624,"15min":0.58999999999999997},"min":{"1min":0.27900000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":3.1909999999999998,"5min":3.1909999999999998,"15min":3.1909999999999998},"last":8.1600000000000001},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.63200000000000001,"15min":0.56000000000000005},"min":{"1min":0.28399999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":2.0760000000000001,"5min":2.589,"15min":2.589},"last":0.42799999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:49:05 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.56200000000000006,"15min":0.54600000000000004},"min":{"1min":0.29099999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":2.0800000000000001,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.54900000000000004,"15min":0.55500000000000005},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":2.8109999999999999,"5min":2.8109999999999999,"15min":2.8109999999999999},"last":0.36599999999999999}]},{"osd":3,"last update":"Tue Sep 10 18:49:05 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":0.67100000000000004,"15min":0.57699999999999996},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":3.512,"5min":3.512,"15min":3.512},"last":0.315},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.61499999999999999,"15min":0.56699999999999995},"min":{"1min":0.32400000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":3.109,"5min":3.109,"15min":3.109},"last":0.29599999999999999}]},{"osd":5,"last update":"Tue Sep 10 18:48:27 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.40699999999999997,"15min":0.36599999999999999},"min":{"1min":0.28899999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":2.8479999999999999,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.39900000000000002,"15min":0.36899999999999999},"min":{"1min":0.26000000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.378}]},{"osd":6,"last update":"Tue Sep 10 18:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.48499999999999999,"15min":0.442},"min":{"1min":0.30599999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":3.5529999999999999,"5min":3.5529999999999999,"15min":3.5529999999999999},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.48099999999999998,"15min":0.41799999999999998},"min":{"1min":0.33100000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":0.33600000000000002}]},{"osd":7,"last update":"Tue Sep 10 18:48:12 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.49199999999999999,"15min":0.498},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.442,"15min":0.44700000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.39000000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182471,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6115772,"kb_used_data":5810100,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":87629380,"statfs":{"total":95995035648,"available":89732485120,"internally_reserved":0,"allocated":5949542400,"data_stored":5949111298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:48:37 2024","interfaces":[{"interface":"back","average":{"1min":1.7589999999999999,"5min":0.64600000000000002,"15min":0.47299999999999998},"min":{"1min":0.223,"5min":0.215,"15min":0.215},"max":{"1min":7.3949999999999996,"5min":7.3949999999999996,"15min":7.3949999999999996},"last":7.5490000000000004},{"interface":"front","average":{"1min":1.776,"5min":0.66900000000000004,"15min":0.49299999999999999},"min":{"1min":0.28100000000000003,"5min":0.217,"15min":0.217},"max":{"1min":4.6100000000000003,"5min":4.6100000000000003,"15min":4.6100000000000003},"last":1.367}]},{"osd":1,"last update":"Tue Sep 10 18:48:37 2024","interfaces":[{"interface":"back","average":{"1min":1.7949999999999999,"5min":0.66800000000000004,"15min":0.497},"min":{"1min":0.29899999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":4.3940000000000001,"5min":4.3940000000000001,"15min":4.3940000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.548,"5min":0.628,"15min":0.48799999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":3.9430000000000001,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":1.7050000000000001}]},{"osd":2,"last update":"Tue Sep 10 18:48:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":0.59299999999999997,"15min":0.49199999999999999},"min":{"1min":0.26600000000000001,"5min":0.246,"15min":0.246},"max":{"1min":3.907,"5min":3.907,"15min":3.907},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.46200000000000002,"15min":0.44700000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":0.52300000000000002}]},{"osd":4,"last update":"Tue Sep 10 18:49:02 2024","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":0.73599999999999999,"15min":0.60799999999999998},"min":{"1min":0.36399999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":6.3440000000000003,"5min":6.3440000000000003,"15min":6.3440000000000003},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.2789999999999999,"5min":0.74299999999999999,"15min":0.63100000000000001},"min":{"1min":0.313,"5min":0.311,"15min":0.311},"max":{"1min":6.29,"5min":6.29,"15min":6.29},"last":1.155}]},{"osd":5,"last update":"Tue Sep 10 18:48:27 2024","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":0.67500000000000004,"15min":0.58499999999999996},"min":{"1min":0.433,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":2.9780000000000002,"5min":2.9780000000000002,"15min":2.9780000000000002},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.65000000000000002,"15min":0.59199999999999997},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":3.5819999999999999,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":0.29299999999999998}]},{"osd":6,"last update":"Tue Sep 10 18:48:49 2024","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":0.72799999999999998,"15min":0.61899999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":3.859,"5min":3.859,"15min":3.859},"last":0.315},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.64300000000000002,"15min":0.58499999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":2.5310000000000001,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.133}]},{"osd":7,"last update":"Tue Sep 10 18:48:11 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.623,"15min":0.56100000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":2.694,"5min":2.694,"15min":2.694},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.60799999999999998,"15min":0.58599999999999997},"min":{"1min":0.434,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.665,"5min":2.665,"15min":2.665},"last":0.33700000000000002}]}]},{"osd":2,"up_from":0,"seq":77309411403,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4526088,"kb_used_data":4220412,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":89219064,"statfs":{"total":95995035648,"available":91360321536,"internally_reserved":0,"allocated":4321701888,"data_stored":4321263177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:48:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":0.54200000000000004,"15min":0.38600000000000001},"min":{"1min":0.38300000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":3.1520000000000001,"5min":3.1520000000000001,"15min":3.1520000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.502,"5min":0.60999999999999999,"15min":0.41799999999999998},"min":{"1min":0.442,"5min":0.252,"15min":0.21199999999999999},"max":{"1min":4.2450000000000001,"5min":4.2450000000000001,"15min":4.2450000000000001},"last":3.5169999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.42999999999999999,"15min":0.38},"min":{"1min":0.22900000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":3.8479999999999999,"5min":3.8479999999999999,"15min":3.8479999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.41199999999999998,"15min":0.38800000000000001},"min":{"1min":0.30299999999999999,"5min":0.248,"15min":0.248},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.78700000000000003}]},{"osd":3,"last update":"Tue Sep 10 18:48:44 2024","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":0.65000000000000002,"15min":0.52300000000000002},"min":{"1min":0.29199999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.6160000000000001,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":2.2349999999999999,"5min":0.77600000000000002,"15min":0.54000000000000004},"min":{"1min":0.35999999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":5.3140000000000001,"5min":5.3140000000000001,"15min":5.3140000000000001},"last":3.4860000000000002}]},{"osd":4,"last update":"Tue Sep 10 18:49:04 2024","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":0.74099999999999999,"15min":0.61499999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":3.1520000000000001,"5min":4.1520000000000001,"15min":4.1520000000000001},"last":3.3570000000000002},{"interface":"front","average":{"1min":1.1839999999999999,"5min":0.68400000000000005,"15min":0.57999999999999996},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":3.9540000000000002,"5min":3.9540000000000002,"15min":3.9540000000000002},"last":0.52500000000000002}]},{"osd":5,"last update":"Tue Sep 10 18:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.63200000000000001,"15min":0.61399999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.68100000000000005,"15min":0.65600000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":4.1130000000000004,"5min":4.1130000000000004,"15min":4.1130000000000004},"last":3.4020000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:48:44 2024","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":0.68999999999999995,"15min":0.65100000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":3.157,"5min":3.157,"15min":3.157},"last":0.71599999999999997},{"interface":"front","average":{"1min":1.2729999999999999,"5min":0.76100000000000001,"15min":0.70299999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":4.1769999999999996,"5min":4.1769999999999996,"15min":4.1769999999999996},"last":3.4580000000000002}]},{"osd":7,"last update":"Tue Sep 10 18:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.64900000000000002,"15min":0.64800000000000002},"min":{"1min":0.41999999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.1800000000000002,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.70199999999999996,"15min":0.66600000000000004},"min":{"1min":0.42899999999999999,"5min":0.372,"15min":0.372},"max":{"1min":4.1989999999999998,"5min":4.1989999999999998,"15min":4.1989999999999998},"last":3.4279999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4505464,"kb_used_data":4208112,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":89239688,"statfs":{"total":95995035648,"available":91381440512,"internally_reserved":0,"allocated":4309106688,"data_stored":4308679974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 18:48:55 2024","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":0.57799999999999996,"15min":0.38800000000000001},"min":{"1min":0.186,"5min":0.186,"15min":0.17699999999999999},"max":{"1min":4.7599999999999998,"5min":4.7599999999999998,"15min":4.7599999999999998},"last":0.51000000000000001},{"interface":"front","average":{"1min":1.575,"5min":0.63300000000000001,"15min":0.41399999999999998},"min":{"1min":0.38,"5min":0.187,"15min":0.187},"max":{"1min":4.5110000000000001,"5min":4.5110000000000001,"15min":4.5110000000000001},"last":0.39200000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:48:25 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.48399999999999999,"15min":0.41599999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":4.6509999999999998,"5min":4.6509999999999998,"15min":4.6509999999999998},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.47699999999999998,"15min":0.39500000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":4.6230000000000002,"5min":4.6230000000000002,"15min":4.6230000000000002},"last":2.0619999999999998}]},{"osd":3,"last update":"Tue Sep 10 18:48:38 2024","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":0.57699999999999996,"15min":0.45900000000000002},"min":{"1min":0.26000000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":3.8119999999999998,"5min":3.8119999999999998,"15min":3.8119999999999998},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.403,"5min":0.59999999999999998,"15min":0.47599999999999998},"min":{"1min":0.29499999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":3.6920000000000002,"5min":3.6920000000000002,"15min":3.6920000000000002},"last":6.7220000000000004}]},{"osd":4,"last update":"Tue Sep 10 18:49:03 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":0.69099999999999995,"15min":0.58999999999999997},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":4.0519999999999996,"5min":4.0519999999999996,"15min":4.0519999999999996},"last":2.097},{"interface":"front","average":{"1min":1.3700000000000001,"5min":0.78800000000000003,"15min":0.60799999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":3.8330000000000002,"5min":4.6189999999999998,"15min":4.6189999999999998},"last":0.53800000000000003}]},{"osd":5,"last update":"Tue Sep 10 18:48:25 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.60899999999999999,"15min":0.57499999999999996},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":2.1469999999999998},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.68000000000000005,"15min":0.58899999999999997},"min":{"1min":0.40500000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.9580000000000002,"5min":4.6429999999999998,"15min":4.6429999999999998},"last":0.61099999999999999}]},{"osd":6,"last update":"Tue Sep 10 18:48:49 2024","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":0.72999999999999998,"15min":0.61199999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":3.855,"5min":3.855,"15min":3.855},"last":0.55700000000000005},{"interface":"front","average":{"1min":1.0780000000000001,"5min":0.76400000000000001,"15min":0.77900000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":2.5830000000000002,"5min":4.6630000000000003,"15min":4.6630000000000003},"last":2.121}]},{"osd":7,"last update":"Tue Sep 10 18:48:10 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.59799999999999998,"15min":0.56999999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":0.63},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.64800000000000002,"15min":0.61099999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":0.58099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4331732992,"data_stored":4331732992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4350476288,"data_stored":4350476288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5991038976,"data_stored":5991038976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2728198144,"data_stored":2728198144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2178809856,"data_stored":2178809856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3264151552,"data_stored":3264151552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3255566336,"data_stored":3255566336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T18:49:18.302 INFO:tasks.cephadm.ceph_manager.ceph:would drop below min_pgs, proj -9, currently 1 2024-09-10T18:49:18.375 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:18.407 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:18.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:18.408 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:18.509 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:18.550 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:18.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:18.551 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:18.652 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:18.689 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:18.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:18.690 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:18.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:18 smithi018 ceph-mon[40196]: 2.1 scrub starts 2024-09-10T18:49:18.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:18 smithi018 ceph-mon[40196]: 2.1 scrub ok 2024-09-10T18:49:18.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:18 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:49:18.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:18 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:49:18.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:18 smithi018 ceph-mon[40196]: pgmap v150: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 8.2 GiB data, 27 GiB used, 599 GiB / 626 GiB avail; 1.1 KiB/s rd, 58 MiB/s wr, 1.16k op/s 2024-09-10T18:49:18.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:18 smithi018 ceph-mon[40196]: osdmap e117: 8 total, 8 up, 7 in 2024-09-10T18:49:18.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:18 smithi018 ceph-mon[40196]: 5.c scrub starts 2024-09-10T18:49:18.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:18 smithi018 ceph-mon[40196]: 5.c scrub ok 2024-09-10T18:49:18.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:18 smithi018 ceph-mon[40196]: 7.9 deep-scrub starts 2024-09-10T18:49:18.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:18 smithi018 ceph-mon[29336]: 2.1 scrub starts 2024-09-10T18:49:18.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:18 smithi018 ceph-mon[29336]: 2.1 scrub ok 2024-09-10T18:49:18.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:18 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:49:18.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:18 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:49:18.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:18 smithi018 ceph-mon[29336]: pgmap v150: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 8.2 GiB data, 27 GiB used, 599 GiB / 626 GiB avail; 1.1 KiB/s rd, 58 MiB/s wr, 1.16k op/s 2024-09-10T18:49:18.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:18 smithi018 ceph-mon[29336]: osdmap e117: 8 total, 8 up, 7 in 2024-09-10T18:49:18.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:18 smithi018 ceph-mon[29336]: 5.c scrub starts 2024-09-10T18:49:18.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:18 smithi018 ceph-mon[29336]: 5.c scrub ok 2024-09-10T18:49:18.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:18 smithi018 ceph-mon[29336]: 7.9 deep-scrub starts 2024-09-10T18:49:18.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:18 smithi103 ceph-mon[33163]: 2.1 scrub starts 2024-09-10T18:49:18.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:18 smithi103 ceph-mon[33163]: 2.1 scrub ok 2024-09-10T18:49:18.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:18 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:49:18.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:18 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:49:18.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:18 smithi103 ceph-mon[33163]: pgmap v150: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 8.2 GiB data, 27 GiB used, 599 GiB / 626 GiB avail; 1.1 KiB/s rd, 58 MiB/s wr, 1.16k op/s 2024-09-10T18:49:18.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:18 smithi103 ceph-mon[33163]: osdmap e117: 8 total, 8 up, 7 in 2024-09-10T18:49:18.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:18 smithi103 ceph-mon[33163]: 5.c scrub starts 2024-09-10T18:49:18.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:18 smithi103 ceph-mon[33163]: 5.c scrub ok 2024-09-10T18:49:18.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:18 smithi103 ceph-mon[33163]: 7.9 deep-scrub starts 2024-09-10T18:49:18.790 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:18.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:18.821 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:18.821 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:18.922 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:18.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:18.951 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:18.952 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:19.053 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:19.097 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:19.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:19.099 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:19.200 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:19.222 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:19.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:19.223 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:19.324 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:19.361 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:19.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:19.362 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:19.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:19 smithi018 ceph-mon[29336]: 7.f scrub ok 2024-09-10T18:49:19.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:19 smithi018 ceph-mon[29336]: 4.f scrub starts 2024-09-10T18:49:19.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:19 smithi018 ceph-mon[29336]: 4.f scrub ok 2024-09-10T18:49:19.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:19 smithi018 ceph-mon[29336]: 6.1 deep-scrub starts 2024-09-10T18:49:19.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:19 smithi018 ceph-mon[29336]: 6.1 deep-scrub ok 2024-09-10T18:49:19.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:19 smithi018 ceph-mon[29336]: 7.6 scrub starts 2024-09-10T18:49:19.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:19 smithi018 ceph-mon[40196]: 7.f scrub ok 2024-09-10T18:49:19.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:19 smithi018 ceph-mon[40196]: 4.f scrub starts 2024-09-10T18:49:19.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:19 smithi018 ceph-mon[40196]: 4.f scrub ok 2024-09-10T18:49:19.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:19 smithi018 ceph-mon[40196]: 6.1 deep-scrub starts 2024-09-10T18:49:19.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:19 smithi018 ceph-mon[40196]: 6.1 deep-scrub ok 2024-09-10T18:49:19.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:19 smithi018 ceph-mon[40196]: 7.6 scrub starts 2024-09-10T18:49:19.464 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:19.509 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:19.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:19.510 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:19.611 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:19.639 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:19.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:19.641 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:19.742 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:19.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:19 smithi103 ceph-mon[33163]: 7.f scrub ok 2024-09-10T18:49:19.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:19 smithi103 ceph-mon[33163]: 4.f scrub starts 2024-09-10T18:49:19.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:19 smithi103 ceph-mon[33163]: 4.f scrub ok 2024-09-10T18:49:19.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:19 smithi103 ceph-mon[33163]: 6.1 deep-scrub starts 2024-09-10T18:49:19.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:19 smithi103 ceph-mon[33163]: 6.1 deep-scrub ok 2024-09-10T18:49:19.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:19 smithi103 ceph-mon[33163]: 7.6 scrub starts 2024-09-10T18:49:19.790 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:19.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:19.791 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:19.892 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:19.953 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:19.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:19.954 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:20.055 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:20.111 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:20.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:20.112 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:20.213 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:20.237 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:20.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:20.238 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:20.339 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:20.362 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:20.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:20.363 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:20.463 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:20.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:20.489 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:20.489 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:20.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:20 smithi103 ceph-mon[33163]: 4.17 scrub starts 2024-09-10T18:49:20.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:20 smithi103 ceph-mon[33163]: 4.17 scrub ok 2024-09-10T18:49:20.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:20 smithi103 ceph-mon[33163]: 6.7 scrub starts 2024-09-10T18:49:20.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:20 smithi103 ceph-mon[33163]: 6.7 scrub ok 2024-09-10T18:49:20.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:20 smithi103 ceph-mon[33163]: pgmap v152: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 8.2 GiB data, 27 GiB used, 599 GiB / 626 GiB avail; 1.1 KiB/s rd, 58 MiB/s wr, 1.16k op/s 2024-09-10T18:49:20.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:20 smithi103 ceph-mon[33163]: 6.14 scrub starts 2024-09-10T18:49:20.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:20 smithi103 ceph-mon[33163]: 7.5 scrub ok 2024-09-10T18:49:20.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:20 smithi103 ceph-mon[33163]: 5.1a scrub starts 2024-09-10T18:49:20.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:20 smithi103 ceph-mon[33163]: 5.1a scrub ok 2024-09-10T18:49:20.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:20 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:49:20.524 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:49:20 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:49:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:49:20.590 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:20.618 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:20.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:20.619 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:20.634 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:20 smithi018 ceph-mon[40196]: 4.17 scrub starts 2024-09-10T18:49:20.634 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:20 smithi018 ceph-mon[40196]: 4.17 scrub ok 2024-09-10T18:49:20.634 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:20 smithi018 ceph-mon[40196]: 6.7 scrub starts 2024-09-10T18:49:20.634 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:20 smithi018 ceph-mon[40196]: 6.7 scrub ok 2024-09-10T18:49:20.634 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:20 smithi018 ceph-mon[40196]: pgmap v152: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 8.2 GiB data, 27 GiB used, 599 GiB / 626 GiB avail; 1.1 KiB/s rd, 58 MiB/s wr, 1.16k op/s 2024-09-10T18:49:20.634 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:20 smithi018 ceph-mon[40196]: 6.14 scrub starts 2024-09-10T18:49:20.634 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:20 smithi018 ceph-mon[40196]: 7.5 scrub ok 2024-09-10T18:49:20.634 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:20 smithi018 ceph-mon[40196]: 5.1a scrub starts 2024-09-10T18:49:20.634 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:20 smithi018 ceph-mon[40196]: 5.1a scrub ok 2024-09-10T18:49:20.634 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:20 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:49:20.635 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:20 smithi018 ceph-mon[29336]: 4.17 scrub starts 2024-09-10T18:49:20.635 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:20 smithi018 ceph-mon[29336]: 4.17 scrub ok 2024-09-10T18:49:20.635 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:20 smithi018 ceph-mon[29336]: 6.7 scrub starts 2024-09-10T18:49:20.635 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:20 smithi018 ceph-mon[29336]: 6.7 scrub ok 2024-09-10T18:49:20.635 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:20 smithi018 ceph-mon[29336]: pgmap v152: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 8.2 GiB data, 27 GiB used, 599 GiB / 626 GiB avail; 1.1 KiB/s rd, 58 MiB/s wr, 1.16k op/s 2024-09-10T18:49:20.635 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:20 smithi018 ceph-mon[29336]: 6.14 scrub starts 2024-09-10T18:49:20.635 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:20 smithi018 ceph-mon[29336]: 7.5 scrub ok 2024-09-10T18:49:20.635 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:20 smithi018 ceph-mon[29336]: 5.1a scrub starts 2024-09-10T18:49:20.635 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:20 smithi018 ceph-mon[29336]: 5.1a scrub ok 2024-09-10T18:49:20.635 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:20 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:49:20.720 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:20.762 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:20.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:20.763 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:20.864 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:20.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:20.911 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:20.911 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:21.012 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:21.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:21.069 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:21.069 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:21.117 INFO:teuthology.orchestra.run.smithi018.stdout:ERROR: (22) Invalid argument 2024-09-10T18:49:21.117 INFO:teuthology.orchestra.run.smithi018.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:49:21.170 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:21.178 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:49:21.178 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:49:21.178 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:49:21.178 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:49:21.179 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:49:21.179 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:49:21.179 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:49:21.179 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:49:21.195 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:49:20 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:49:20] "GET /metrics HTTP/1.1" 200 223623 "" "Prometheus/2.43.0" 2024-09-10T18:49:21.206 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:49:21.206 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:49:21.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:21.215 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:21.215 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:21.237 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:49:21.237 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:49:21.270 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:49:21.271 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:49:21.315 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:49:21.315 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:49:21.316 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:21.360 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:21.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:21.362 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:21.463 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:21.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:21.493 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:21.494 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:21.556 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:21 smithi018 ceph-mon[40196]: 2.f scrub starts 2024-09-10T18:49:21.557 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:21 smithi018 ceph-mon[40196]: 2.f scrub ok 2024-09-10T18:49:21.557 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:21 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:49:21.557 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:21 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:49:21.557 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:21 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:49:21.557 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:21 smithi018 ceph-mon[40196]: 7.8 scrub starts 2024-09-10T18:49:21.557 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:21 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2111032295' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:49:21.557 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:21 smithi018 ceph-mon[29336]: 2.f scrub starts 2024-09-10T18:49:21.557 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:21 smithi018 ceph-mon[29336]: 2.f scrub ok 2024-09-10T18:49:21.557 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:21 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:49:21.557 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:21 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:49:21.558 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:21 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:49:21.558 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:21 smithi018 ceph-mon[29336]: 7.8 scrub starts 2024-09-10T18:49:21.558 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:21 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2111032295' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:49:21.594 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:21.637 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:21.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:21.638 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:21.739 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:21.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:21 smithi103 ceph-mon[33163]: 2.f scrub starts 2024-09-10T18:49:21.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:21 smithi103 ceph-mon[33163]: 2.f scrub ok 2024-09-10T18:49:21.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:21 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:49:21.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:21 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:49:21.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:21 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:49:21.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:21 smithi103 ceph-mon[33163]: 7.8 scrub starts 2024-09-10T18:49:21.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:21 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2111032295' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:49:21.776 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:21.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:21.778 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:21.879 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:21.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:21.926 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:21.927 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:22.027 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:22.057 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:22.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:22.059 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:22.161 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:22.189 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:22.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:22.190 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:22.291 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:22.319 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:22.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:22.320 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:22.374 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-10T18:49:22.421 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:22.449 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:22.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:22.450 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:22.551 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:22.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:22.579 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:22.579 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:22.680 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:22.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:22 smithi018 ceph-mon[29336]: 5.b scrub starts 2024-09-10T18:49:22.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:22 smithi018 ceph-mon[29336]: 5.b scrub ok 2024-09-10T18:49:22.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:22 smithi018 ceph-mon[29336]: pgmap v153: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 8.0 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 1.1 KiB/s rd, 40 MiB/s wr, 1.13k op/s 2024-09-10T18:49:22.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:22 smithi018 ceph-mon[29336]: 5.a scrub starts 2024-09-10T18:49:22.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:22 smithi018 ceph-mon[29336]: 5.a scrub ok 2024-09-10T18:49:22.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:22 smithi018 ceph-mon[40196]: 5.b scrub starts 2024-09-10T18:49:22.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:22 smithi018 ceph-mon[40196]: 5.b scrub ok 2024-09-10T18:49:22.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:22 smithi018 ceph-mon[40196]: pgmap v153: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 8.0 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 1.1 KiB/s rd, 40 MiB/s wr, 1.13k op/s 2024-09-10T18:49:22.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:22 smithi018 ceph-mon[40196]: 5.a scrub starts 2024-09-10T18:49:22.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:22 smithi018 ceph-mon[40196]: 5.a scrub ok 2024-09-10T18:49:22.711 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:22.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:22.712 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:22.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:22 smithi103 ceph-mon[33163]: 5.b scrub starts 2024-09-10T18:49:22.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:22 smithi103 ceph-mon[33163]: 5.b scrub ok 2024-09-10T18:49:22.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:22 smithi103 ceph-mon[33163]: pgmap v153: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 8.0 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 1.1 KiB/s rd, 40 MiB/s wr, 1.13k op/s 2024-09-10T18:49:22.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:22 smithi103 ceph-mon[33163]: 5.a scrub starts 2024-09-10T18:49:22.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:22 smithi103 ceph-mon[33163]: 5.a scrub ok 2024-09-10T18:49:22.813 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:22.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:22.846 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:22.846 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:22.947 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:22.974 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:22.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:22.975 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:23.076 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:23.103 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:23.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:23.104 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:23.205 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:23.249 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:23.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:23.250 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:23.302 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-10T18:49:23.302 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T18:49:23.303 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-10T18:49:23.303 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-10T18:49:23.303 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-10T18:49:23.303 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.1 config set filestore_inject_stall 3 2024-09-10T18:49:23.351 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:23.405 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:23.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:23.408 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:23.509 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:23.569 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:23.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:23.570 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:23.614 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:23 smithi018 ceph-mon[40196]: 7.6 scrub ok 2024-09-10T18:49:23.614 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:23 smithi018 ceph-mon[29336]: 7.6 scrub ok 2024-09-10T18:49:23.671 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:23.734 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:23.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:23.736 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:23.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:23 smithi103 ceph-mon[33163]: 7.6 scrub ok 2024-09-10T18:49:23.781 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:49:23.836 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:23.837 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:49:23.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:23.896 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:23.897 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:23.997 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:24.037 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:24.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:24.038 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:24.139 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:24.167 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:24.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:24.169 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:24.269 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:24.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:24.298 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:24.298 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:24.318 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is set 2024-09-10T18:49:24.399 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:24.438 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:24.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:24.439 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:24.540 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:24.581 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:24.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:24.583 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:24.594 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:24 smithi018 ceph-mon[40196]: 7.8 scrub ok 2024-09-10T18:49:24.594 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:24 smithi018 ceph-mon[40196]: pgmap v154: 148 pgs: 1 active+clean+scrubbing+deep, 2 active+clean+scrubbing, 145 active+clean; 7.9 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 1.1 KiB/s rd, 22 MiB/s wr, 1.10k op/s 2024-09-10T18:49:24.594 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:24 smithi018 ceph-mon[40196]: 2.6 scrub starts 2024-09-10T18:49:24.594 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:24 smithi018 ceph-mon[40196]: 2.6 scrub ok 2024-09-10T18:49:24.594 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:24 smithi018 ceph-mon[40196]: 4.1a scrub starts 2024-09-10T18:49:24.594 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:24 smithi018 ceph-mon[40196]: 6.3 scrub starts 2024-09-10T18:49:24.594 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:24 smithi018 ceph-mon[40196]: 6.3 scrub ok 2024-09-10T18:49:24.594 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:24 smithi018 ceph-mon[40196]: 4.1a scrub ok 2024-09-10T18:49:24.595 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:24 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3525345472' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:49:24.595 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:24 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:49:24.595 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:24 smithi018 ceph-mon[29336]: 7.8 scrub ok 2024-09-10T18:49:24.595 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:24 smithi018 ceph-mon[29336]: pgmap v154: 148 pgs: 1 active+clean+scrubbing+deep, 2 active+clean+scrubbing, 145 active+clean; 7.9 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 1.1 KiB/s rd, 22 MiB/s wr, 1.10k op/s 2024-09-10T18:49:24.595 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:24 smithi018 ceph-mon[29336]: 2.6 scrub starts 2024-09-10T18:49:24.595 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:24 smithi018 ceph-mon[29336]: 2.6 scrub ok 2024-09-10T18:49:24.595 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:24 smithi018 ceph-mon[29336]: 4.1a scrub starts 2024-09-10T18:49:24.595 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:24 smithi018 ceph-mon[29336]: 6.3 scrub starts 2024-09-10T18:49:24.595 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:24 smithi018 ceph-mon[29336]: 6.3 scrub ok 2024-09-10T18:49:24.595 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:24 smithi018 ceph-mon[29336]: 4.1a scrub ok 2024-09-10T18:49:24.596 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:24 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3525345472' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:49:24.596 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:24 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:49:24.683 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:24.709 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:24.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:24.711 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:24.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:24 smithi103 ceph-mon[33163]: 7.8 scrub ok 2024-09-10T18:49:24.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:24 smithi103 ceph-mon[33163]: pgmap v154: 148 pgs: 1 active+clean+scrubbing+deep, 2 active+clean+scrubbing, 145 active+clean; 7.9 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 1.1 KiB/s rd, 22 MiB/s wr, 1.10k op/s 2024-09-10T18:49:24.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:24 smithi103 ceph-mon[33163]: 2.6 scrub starts 2024-09-10T18:49:24.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:24 smithi103 ceph-mon[33163]: 2.6 scrub ok 2024-09-10T18:49:24.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:24 smithi103 ceph-mon[33163]: 4.1a scrub starts 2024-09-10T18:49:24.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:24 smithi103 ceph-mon[33163]: 6.3 scrub starts 2024-09-10T18:49:24.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:24 smithi103 ceph-mon[33163]: 6.3 scrub ok 2024-09-10T18:49:24.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:24 smithi103 ceph-mon[33163]: 4.1a scrub ok 2024-09-10T18:49:24.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:24 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3525345472' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:49:24.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:24 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:49:24.812 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:24.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:24.849 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:24.849 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:24.950 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:24.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:24.993 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:24.994 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:25.092 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T18:49:25.095 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:25.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:25.134 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:25.134 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:25.235 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:25.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:25.295 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:25.295 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:25.396 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:25.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:25.425 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:25.425 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:25.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:25 smithi018 ceph-mon[29336]: 6.a scrub starts 2024-09-10T18:49:25.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:25 smithi018 ceph-mon[29336]: 6.a scrub ok 2024-09-10T18:49:25.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:25 smithi018 ceph-mon[29336]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:49:25.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:25 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:49:25.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:25 smithi018 ceph-mon[29336]: osdmap e118: 8 total, 8 up, 7 in 2024-09-10T18:49:25.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:25 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:49:25.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:25 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:49:25.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:25 smithi018 ceph-mon[40196]: 6.a scrub starts 2024-09-10T18:49:25.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:25 smithi018 ceph-mon[40196]: 6.a scrub ok 2024-09-10T18:49:25.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:25 smithi018 ceph-mon[40196]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:49:25.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:25 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:49:25.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:25 smithi018 ceph-mon[40196]: osdmap e118: 8 total, 8 up, 7 in 2024-09-10T18:49:25.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:25 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:49:25.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:25 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:49:25.526 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:25.554 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:25.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:25.555 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:25.656 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:25.684 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:25.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:25.685 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:25.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:25 smithi103 ceph-mon[33163]: 6.a scrub starts 2024-09-10T18:49:25.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:25 smithi103 ceph-mon[33163]: 6.a scrub ok 2024-09-10T18:49:25.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:25 smithi103 ceph-mon[33163]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:49:25.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:25 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:49:25.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:25 smithi103 ceph-mon[33163]: osdmap e118: 8 total, 8 up, 7 in 2024-09-10T18:49:25.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:25 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:49:25.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:25 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:49:25.786 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:25.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:25.820 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:25.820 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:25.921 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:25.948 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:25.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:25.949 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:26.052 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:26.092 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:26.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:26.093 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:26.194 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:26.224 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:26.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:26.226 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:26.326 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:26.369 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:26.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:26.370 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:26.471 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:26.496 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:26.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:26.497 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:26.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:26 smithi103 ceph-mon[33163]: pgmap v156: 148 pgs: 1 active+clean+scrubbing+deep, 2 active+clean+scrubbing, 145 active+clean; 7.9 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 656 op/s 2024-09-10T18:49:26.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:49:26 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:49:26.598 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:26.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:26.644 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:26.645 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:26.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:26 smithi018 ceph-mon[29336]: pgmap v156: 148 pgs: 1 active+clean+scrubbing+deep, 2 active+clean+scrubbing, 145 active+clean; 7.9 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 656 op/s 2024-09-10T18:49:26.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:26 smithi018 ceph-mon[40196]: pgmap v156: 148 pgs: 1 active+clean+scrubbing+deep, 2 active+clean+scrubbing, 145 active+clean; 7.9 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 656 op/s 2024-09-10T18:49:26.746 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:26.784 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:26.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:26.785 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:26.886 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:26.922 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:49:26.937 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:26.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:26.938 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:27.039 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:27.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:27.068 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:27.068 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:27.169 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:27.236 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:27.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:27.237 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:27.337 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:27.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:27.382 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:27.383 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:27.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:27 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:27.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:27 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:27.484 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:27.510 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:27.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:27.512 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:27.612 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:27.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:27.640 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:27.641 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:27.741 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:27.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:27 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:27.779 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:27.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:27.781 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:27.882 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:27.924 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-10T18:49:27.927 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:27.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:27.931 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:28.031 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:28.056 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:28.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:28.057 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:28.124 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T18:49:28.125 INFO:teuthology.orchestra.run.smithi018.stdout: "ops": [], 2024-09-10T18:49:28.125 INFO:teuthology.orchestra.run.smithi018.stdout: "complaint_time": 30, 2024-09-10T18:49:28.125 INFO:teuthology.orchestra.run.smithi018.stdout: "num_blocked_ops": 0 2024-09-10T18:49:28.125 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T18:49:28.157 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:28.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:28.199 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:28.200 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:28.300 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:28.326 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:28.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:28.327 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:28.428 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:28.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:28.476 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:28.476 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:28.577 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:28.618 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:28.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:28.619 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:28.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:28 smithi018 ceph-mon[29336]: pgmap v157: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.5 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.08k op/s 2024-09-10T18:49:28.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:28 smithi018 ceph-mon[40196]: pgmap v157: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.5 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.08k op/s 2024-09-10T18:49:28.720 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:28.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:28 smithi103 ceph-mon[33163]: pgmap v157: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.5 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.08k op/s 2024-09-10T18:49:28.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:28.776 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:28.776 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:28.812 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-10T18:49:28.877 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:28.992 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:28.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:28.993 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:29.094 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:29.116 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:29.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:29.117 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:29.121 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T18:49:29.122 INFO:teuthology.orchestra.run.smithi018.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-10T18:49:29.122 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T18:49:29.218 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:29.261 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:29.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:29.262 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:29.362 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:29.406 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:29.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:29.407 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:29.508 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:29.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:29.550 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:29.551 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:29.652 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:29.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:29 smithi018 ceph-mon[40196]: pgmap v158: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.5 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.06k op/s 2024-09-10T18:49:29.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:29.695 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:29.695 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:29.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:29 smithi018 ceph-mon[29336]: pgmap v158: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.5 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.06k op/s 2024-09-10T18:49:29.796 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:29.835 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:29.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:29.836 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:29.936 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:29.979 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:29.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:29.980 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:30.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:29 smithi103 ceph-mon[33163]: pgmap v158: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.5 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.06k op/s 2024-09-10T18:49:30.081 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:30.108 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:30.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:30.109 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:30.190 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:49:30.210 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:30.237 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:30.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:30.238 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:30.339 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:30.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:30.367 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:30.367 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:30.468 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:30.504 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:30.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:30.505 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:30.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:49:30 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:49:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:49:30.599 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:30 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:49:30.599 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:30 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:49:30.599 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:30 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:49:30.599 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:30 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:49:30.606 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:30.630 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:30.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:30.631 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:30.732 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:30.766 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:30.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:30.767 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:30.868 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:30.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:30.898 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:30.898 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:30.991 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:49:30.999 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:31.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:30 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:49:31.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:30 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:49:31.027 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:31.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:31.028 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:31.129 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:31.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:31.161 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:31.162 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:31.241 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:49:30 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:49:30] "GET /metrics HTTP/1.1" 200 223672 "" "Prometheus/2.43.0" 2024-09-10T18:49:31.262 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:31.299 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:31.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:31.300 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:31.401 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:31.429 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:31.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:31.430 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:31.531 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:31.558 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:31.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:31.560 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:31.661 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:31.717 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:31.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:31.718 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:31.788 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:31 smithi018 ceph-mon[29336]: osdmap e119: 8 total, 8 up, 7 in 2024-09-10T18:49:31.788 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:31 smithi018 ceph-mon[29336]: pgmap v160: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.4 GiB data, 24 GiB used, 601 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 986 op/s 2024-09-10T18:49:31.788 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:31 smithi018 ceph-mon[40196]: osdmap e119: 8 total, 8 up, 7 in 2024-09-10T18:49:31.789 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:31 smithi018 ceph-mon[40196]: pgmap v160: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.4 GiB data, 24 GiB used, 601 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 986 op/s 2024-09-10T18:49:31.819 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:31.847 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:31.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:31.848 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:31.949 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:31.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:31.977 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:31.977 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:32.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:31 smithi103 ceph-mon[33163]: osdmap e119: 8 total, 8 up, 7 in 2024-09-10T18:49:32.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:31 smithi103 ceph-mon[33163]: pgmap v160: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.4 GiB data, 24 GiB used, 601 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 986 op/s 2024-09-10T18:49:32.077 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:32.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:32.121 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:32.121 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:32.222 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:32.249 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:32.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:32.250 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:32.351 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:32.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:32.397 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:32.398 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:32.499 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:32.528 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:32.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:32.529 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:32.630 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:32.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:32.663 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:32.663 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:32.764 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:32.795 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:32.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:32.796 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:32.897 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:32.935 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:32.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:32.936 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:33.037 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:33.076 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:33.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:33.077 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:33.178 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:33.210 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:33.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:33.211 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:33.312 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:33.340 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:33.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:33.341 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:33.442 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:33.473 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:33.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:33.474 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:33.575 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:33.602 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:33.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:33.603 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:33.642 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:49:33.642 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:49:33.674 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:49:33.674 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:49:33.703 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:33.706 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:49:33.706 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:49:33.731 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:33.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:33.732 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:33.739 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:49:33.739 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:49:33.771 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:49:33.771 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:49:33.800 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:49:33.801 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:49:33.828 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:49:33.828 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:49:33.833 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:33.862 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:33.863 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:49:33.863 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:49:33.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:33.864 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:33.965 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:34.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:34.008 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:34.008 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:34.109 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:34.150 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:34.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:34.151 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:34.252 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:34.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:34.280 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:34.280 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:34.338 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:34 smithi018 ceph-mon[29336]: pgmap v161: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.2 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.19k op/s 2024-09-10T18:49:34.338 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:34 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3090629451' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:49:34.338 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:34 smithi018 ceph-mon[40196]: pgmap v161: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.2 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.19k op/s 2024-09-10T18:49:34.338 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:34 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3090629451' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:49:34.381 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:34.418 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:34.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:34.419 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:34.520 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:34.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:34 smithi103 ceph-mon[33163]: pgmap v161: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.2 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.19k op/s 2024-09-10T18:49:34.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:34 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3090629451' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:49:34.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:34.558 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:34.558 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:34.659 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:34.703 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:34.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:34.705 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:34.805 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:34.831 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:34.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:34.832 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:34.933 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:34.956 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:34.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:34.957 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:35.058 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:35.085 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:35.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:35.086 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:35.175 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-10T18:49:35.175 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T18:49:35.175 INFO:tasks.thrashosds.thrasher:Shrinking pool unique_pool_0 2024-09-10T18:49:35.176 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool unique_pool_0 by 10 min 8 2024-09-10T18:49:35.176 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T18:49:35.187 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:35.214 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:35.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:35.216 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:35.316 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:35.342 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:35.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:35.343 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:35.444 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:35.470 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:35.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:35.471 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:35.572 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:35.597 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:35.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:35.598 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:35.698 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:35.747 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:35.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:35.748 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:35.849 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:35.877 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:35.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:35.878 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:35.979 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:36.002 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:36.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:36.003 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:36.104 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:36.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:36.143 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:36.143 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:36.244 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:36.270 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:36.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:36.271 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:36.372 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:36.400 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:36.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:36.401 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:36.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:36 smithi018 ceph-mon[29336]: pgmap v162: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.2 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.05k op/s 2024-09-10T18:49:36.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:36 smithi018 ceph-mon[40196]: pgmap v162: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.2 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.05k op/s 2024-09-10T18:49:36.503 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:36.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:36 smithi103 ceph-mon[33163]: pgmap v162: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 7.2 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.05k op/s 2024-09-10T18:49:36.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:49:36 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:49:36.539 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:36.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:36.540 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:36.641 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:36.668 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:36.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:36.669 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:36.695 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T18:49:36.770 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:36.798 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:36.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:36.799 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:36.900 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:36.935 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:36.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:36.936 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:37.037 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:37.071 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:37.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:37.072 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:37.173 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:37.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:37.219 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:37.220 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:37.320 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:37.355 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:37.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:37.356 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:37.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:37 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:37.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:37 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:49:37.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:37 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:49:37.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:37 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:37.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:37 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:49:37.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:37 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:49:37.457 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:37.485 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:37.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:37.486 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:37.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:37 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:37.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:37 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:49:37.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:37 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:49:37.588 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:37.615 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:37.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:37.617 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:37.718 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:37.740 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:37.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:37.741 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:37.842 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:37.867 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:37.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:37.869 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:37.970 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:38.019 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:38.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:38.021 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:38.122 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:38.149 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:38.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:38.150 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:38.251 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:38.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:38.288 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:38.288 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:38.389 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:38.415 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:38 smithi018 ceph-mon[29336]: pgmap v163: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.9 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.07k op/s 2024-09-10T18:49:38.415 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:38 smithi018 ceph-mon[29336]: osdmap e120: 8 total, 8 up, 7 in 2024-09-10T18:49:38.415 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:38 smithi018 ceph-mon[40196]: pgmap v163: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.9 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.07k op/s 2024-09-10T18:49:38.415 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:38 smithi018 ceph-mon[40196]: osdmap e120: 8 total, 8 up, 7 in 2024-09-10T18:49:38.416 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:38.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:38.417 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:38.520 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:38.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:38 smithi103 ceph-mon[33163]: pgmap v163: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.9 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.07k op/s 2024-09-10T18:49:38.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:38 smithi103 ceph-mon[33163]: osdmap e120: 8 total, 8 up, 7 in 2024-09-10T18:49:38.561 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:38.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:38.561 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:38.662 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:38.685 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:38.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:38.686 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:38.787 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:38.814 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:38.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:38.815 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:38.916 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:38.925 INFO:teuthology.orchestra.run.smithi018.stdout:ERROR: (22) Invalid argument 2024-09-10T18:49:38.925 INFO:teuthology.orchestra.run.smithi018.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:49:38.941 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:38.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:38.942 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:39.042 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:39.079 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:39.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:39.080 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:39.181 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:39.216 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:39.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:39.218 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:39.318 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:39.356 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:39.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:39.357 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:39.458 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:39.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:39.508 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:39.508 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:39.609 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:39.635 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:39.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:39.637 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:39.737 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:39.779 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:39.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:39.780 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:39.881 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:39.897 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-10T18:49:39.928 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:39.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:39.929 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:39.971 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:49:39.981 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:49:40.029 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:40.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:40.060 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:40.060 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:40.161 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:40.182 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is set 2024-09-10T18:49:40.203 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:40.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:40.204 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:40.237 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:49:40.238 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:40 smithi103 ceph-mon[33163]: pgmap v165: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.9 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.25k op/s 2024-09-10T18:49:40.238 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:40 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1988243093' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:49:40.238 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:40 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:49:40.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:40 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:49:40.239 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:49:40 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:49:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:49:40.305 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:40.336 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:40.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:40.337 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:40.437 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:40.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:40 smithi018 ceph-mon[40196]: pgmap v165: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.9 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.25k op/s 2024-09-10T18:49:40.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:40 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1988243093' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:49:40.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:40 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:49:40.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:40 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:49:40.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:40 smithi018 ceph-mon[29336]: pgmap v165: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.9 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.25k op/s 2024-09-10T18:49:40.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:40 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1988243093' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:49:40.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:40 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:49:40.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:40 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:49:40.468 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:40.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:40.469 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:40.572 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:40.603 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:40.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:40.604 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:40.705 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:40.740 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:40.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:40.741 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:40.841 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:40.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:40.892 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:40.892 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:40.993 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:41.029 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:41.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:41.030 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:41.131 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:41.168 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:41.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:41.169 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:41.169 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:49:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:49:40] "GET /metrics HTTP/1.1" 200 223705 "" "Prometheus/2.43.0" 2024-09-10T18:49:41.270 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:41.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:41.298 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:41.298 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:41.399 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:41.428 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:41.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:41.429 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:41.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:41 smithi018 ceph-mon[40196]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:49:41.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:41 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:49:41.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:41 smithi018 ceph-mon[40196]: osdmap e121: 8 total, 8 up, 7 in 2024-09-10T18:49:41.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:41 smithi018 ceph-mon[29336]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:49:41.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:41 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:49:41.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:41 smithi018 ceph-mon[29336]: osdmap e121: 8 total, 8 up, 7 in 2024-09-10T18:49:41.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:41 smithi103 ceph-mon[33163]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:49:41.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:41 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:49:41.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:41 smithi103 ceph-mon[33163]: osdmap e121: 8 total, 8 up, 7 in 2024-09-10T18:49:41.530 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:41.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:41.564 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:41.564 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:41.665 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:41.693 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:41.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:41.694 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:41.795 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:41.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:41.824 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:41.824 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:41.925 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:41.962 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:41.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:41.963 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:42.064 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:42.102 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:42.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:42.103 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:42.204 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:42.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:42.245 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:42.246 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:42.293 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T18:49:42.293 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:49:42.347 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:42.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:42.382 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:42.382 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:42.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:42 smithi018 ceph-mon[29336]: pgmap v167: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.7 GiB data, 22 GiB used, 603 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 1.03k op/s 2024-09-10T18:49:42.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:42 smithi018 ceph-mon[40196]: pgmap v167: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.7 GiB data, 22 GiB used, 603 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 1.03k op/s 2024-09-10T18:49:42.483 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:42.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:42.523 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:42.523 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:42.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:42 smithi103 ceph-mon[33163]: pgmap v167: 148 pgs: 1 active+clean+scrubbing+deep, 147 active+clean; 6.7 GiB data, 22 GiB used, 603 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 1.03k op/s 2024-09-10T18:49:42.624 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:42.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:42.662 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:42.662 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:42.763 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:42.801 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:42.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:42.802 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:42.903 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:42.937 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:42.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:42.938 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:43.039 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:43.075 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:43.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:43.076 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:43.177 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:43.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:43.208 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:43.208 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:43.309 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:43.331 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:43.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:43.333 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:43.433 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:43.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:43.477 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:43.477 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:43.578 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:43.617 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:43.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:43.618 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:43.719 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:43.758 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:43.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:43.759 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:43.860 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:43.900 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:43.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:43.901 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:44.002 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:44.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:44.039 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:44.039 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:44.140 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:44.169 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:44.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:44.170 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:44.271 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:44.305 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:44.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:44.306 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:44.407 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:44.430 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:44.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:44.431 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:44.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:44 smithi018 ceph-mon[40196]: pgmap v168: 148 pgs: 148 active+clean; 6.5 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.38k op/s 2024-09-10T18:49:44.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:44 smithi018 ceph-mon[29336]: pgmap v168: 148 pgs: 148 active+clean; 6.5 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.38k op/s 2024-09-10T18:49:44.475 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:49:44.478 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-09-10T18:49:44.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:44 smithi103 ceph-mon[33163]: pgmap v168: 148 pgs: 148 active+clean; 6.5 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.38k op/s 2024-09-10T18:49:44.532 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:44.562 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:44.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:44.563 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:44.664 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:44.706 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:44.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:44.707 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:44.760 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-10T18:49:44.808 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:44.843 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:44.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:44.844 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:44.945 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:45.012 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:45.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:45.013 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:45.114 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:45.155 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:45.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:45.156 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:45.257 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:45.301 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:45.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:45.302 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:45.385 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-10T18:49:45.403 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:45.441 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:45.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:45.442 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:45.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:45 smithi018 ceph-mon[29336]: from='client.24736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:49:45.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:45 smithi018 ceph-mon[40196]: from='client.24736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:49:45.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:45 smithi103 ceph-mon[33163]: from='client.24736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:49:45.530 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2024-09-10T18:49:43.090667+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":7027235060,"num_objects":107415,"num_object_clones":0,"num_object_copies":322245,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107415,"num_whiteouts":0,"num_read":1106,"num_read_kb":963,"num_write":165760,"num_write_kb":8719560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":1183107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77567,"ondisk_log_size":77567,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":22909932,"kb_used_data":20799068,"kb_used_omap":11,"kb_used_meta":2110516,"kb_avail":633306132,"statfs":{"total":671965249536,"available":648505479168,"internally_reserved":0,"allocated":21298245632,"data_stored":21295498495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11928,"internal_metadata":2161168744},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-722272256,"num_objects":-11021,"num_object_clones":0,"num_object_copies":-33063,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-11021,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":11021,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002838"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":88,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.171201+0000","last_change":"2024-09-10T18:48:24.515065+0000","last_active":"2024-09-10T18:49:37.171201+0000","last_peered":"2024-09-10T18:49:37.171201+0000","last_clean":"2024-09-10T18:49:37.171201+0000","last_became_active":"2024-09-10T18:45:24.265488+0000","last_became_peered":"2024-09-10T18:45:24.265488+0000","last_unstale":"2024-09-10T18:49:37.171201+0000","last_undegraded":"2024-09-10T18:49:37.171201+0000","last_fullsized":"2024-09-10T18:49:37.171201+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:24.511946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:24.511946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:49.630089+0000","scrub_duration":0.014161415,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":94,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.167220+0000","last_change":"2024-09-10T18:48:29.718356+0000","last_active":"2024-09-10T18:49:37.167220+0000","last_peered":"2024-09-10T18:49:37.167220+0000","last_clean":"2024-09-10T18:49:37.167220+0000","last_became_active":"2024-09-10T18:45:17.959475+0000","last_became_peered":"2024-09-10T18:45:17.959475+0000","last_unstale":"2024-09-10T18:49:37.167220+0000","last_undegraded":"2024-09-10T18:49:37.167220+0000","last_fullsized":"2024-09-10T18:49:37.167220+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:29.718270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.718270+0000","last_clean_scrub_stamp":"2024-09-10T18:48:29.718270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:59.632977+0000","scrub_duration":0.010834984000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"63'15","reported_seq":143,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.171250+0000","last_change":"2024-09-10T18:49:02.544635+0000","last_active":"2024-09-10T18:49:37.171250+0000","last_peered":"2024-09-10T18:49:37.171250+0000","last_clean":"2024-09-10T18:49:37.171250+0000","last_became_active":"2024-09-10T18:47:36.413635+0000","last_became_peered":"2024-09-10T18:47:36.413635+0000","last_unstale":"2024-09-10T18:49:37.171250+0000","last_undegraded":"2024-09-10T18:49:37.171250+0000","last_fullsized":"2024-09-10T18:49:37.171250+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:49:02.544557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:49:02.544557+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:20.745798+0000","scrub_duration":0.0033455720000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":89,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.162668+0000","last_change":"2024-09-10T18:48:29.058001+0000","last_active":"2024-09-10T18:49:37.162668+0000","last_peered":"2024-09-10T18:49:37.162668+0000","last_clean":"2024-09-10T18:49:37.162668+0000","last_became_active":"2024-09-10T18:45:22.065619+0000","last_became_peered":"2024-09-10T18:45:22.065619+0000","last_unstale":"2024-09-10T18:49:37.162668+0000","last_undegraded":"2024-09-10T18:49:37.162668+0000","last_fullsized":"2024-09-10T18:49:37.162668+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:29.057932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.057932+0000","last_clean_scrub_stamp":"2024-09-10T18:48:29.057932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001460184,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":94,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.175317+0000","last_change":"2024-09-10T18:48:29.458396+0000","last_active":"2024-09-10T18:49:37.175317+0000","last_peered":"2024-09-10T18:49:37.175317+0000","last_clean":"2024-09-10T18:49:37.175317+0000","last_became_active":"2024-09-10T18:45:17.966153+0000","last_became_peered":"2024-09-10T18:45:17.966153+0000","last_unstale":"2024-09-10T18:49:37.175317+0000","last_undegraded":"2024-09-10T18:49:37.175317+0000","last_fullsized":"2024-09-10T18:49:37.175317+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:29.457892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:29.457892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:50.349265+0000","scrub_duration":0.012091342999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"83'13","reported_seq":101,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.162775+0000","last_change":"2024-09-10T18:46:59.906707+0000","last_active":"2024-09-10T18:49:37.162775+0000","last_peered":"2024-09-10T18:49:37.162775+0000","last_clean":"2024-09-10T18:49:37.162775+0000","last_became_active":"2024-09-10T18:45:20.061266+0000","last_became_peered":"2024-09-10T18:45:20.061266+0000","last_unstale":"2024-09-10T18:49:37.162775+0000","last_undegraded":"2024-09-10T18:49:37.162775+0000","last_fullsized":"2024-09-10T18:49:37.162775+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'13","last_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_clean_scrub_stamp":"2024-09-10T18:46:59.906634+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003988892,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":80,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.162144+0000","last_change":"2024-09-10T18:47:21.589428+0000","last_active":"2024-09-10T18:49:37.162144+0000","last_peered":"2024-09-10T18:49:37.162144+0000","last_clean":"2024-09-10T18:49:37.162144+0000","last_became_active":"2024-09-10T18:45:22.072822+0000","last_became_peered":"2024-09-10T18:45:22.072822+0000","last_unstale":"2024-09-10T18:49:37.162144+0000","last_undegraded":"2024-09-10T18:49:37.162144+0000","last_fullsized":"2024-09-10T18:49:37.162144+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:21.589365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:47:21.589365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001676893,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":77,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.162773+0000","last_change":"2024-09-10T18:47:00.946309+0000","last_active":"2024-09-10T18:49:37.162773+0000","last_peered":"2024-09-10T18:49:37.162773+0000","last_clean":"2024-09-10T18:49:37.162773+0000","last_became_active":"2024-09-10T18:45:24.081002+0000","last_became_peered":"2024-09-10T18:45:24.081002+0000","last_unstale":"2024-09-10T18:49:37.162773+0000","last_undegraded":"2024-09-10T18:49:37.162773+0000","last_fullsized":"2024-09-10T18:49:37.162773+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:00.946255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:00.946255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001207629,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":94,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.171235+0000","last_change":"2024-09-10T18:48:13.694567+0000","last_active":"2024-09-10T18:49:37.171235+0000","last_peered":"2024-09-10T18:49:37.171235+0000","last_clean":"2024-09-10T18:49:37.171235+0000","last_became_active":"2024-09-10T18:45:17.961097+0000","last_became_peered":"2024-09-10T18:45:17.961097+0000","last_unstale":"2024-09-10T18:49:37.171235+0000","last_undegraded":"2024-09-10T18:49:37.171235+0000","last_fullsized":"2024-09-10T18:49:37.171235+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:13.694295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:13.694295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051419839999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"63'5","reported_seq":91,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.162934+0000","last_change":"2024-09-10T18:47:04.002228+0000","last_active":"2024-09-10T18:49:37.162934+0000","last_peered":"2024-09-10T18:49:37.162934+0000","last_clean":"2024-09-10T18:49:37.162934+0000","last_became_active":"2024-09-10T18:45:20.057567+0000","last_became_peered":"2024-09-10T18:45:20.057567+0000","last_unstale":"2024-09-10T18:49:37.162934+0000","last_undegraded":"2024-09-10T18:49:37.162934+0000","last_fullsized":"2024-09-10T18:49:37.162934+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-10T18:47:04.002164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:04.002164+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.026374999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":100,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.168139+0000","last_change":"2024-09-10T18:49:19.568881+0000","last_active":"2024-09-10T18:49:37.168139+0000","last_peered":"2024-09-10T18:49:37.168139+0000","last_clean":"2024-09-10T18:49:37.168139+0000","last_became_active":"2024-09-10T18:45:22.076575+0000","last_became_peered":"2024-09-10T18:45:22.076575+0000","last_unstale":"2024-09-10T18:49:37.168139+0000","last_undegraded":"2024-09-10T18:49:37.168139+0000","last_fullsized":"2024-09-10T18:49:37.168139+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:19.568791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:36.509971+0000","last_clean_scrub_stamp":"2024-09-10T18:49:19.568791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:31.609022+0000","scrub_duration":0.0013364760000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":87,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.164747+0000","last_change":"2024-09-10T18:48:10.102991+0000","last_active":"2024-09-10T18:49:37.164747+0000","last_peered":"2024-09-10T18:49:37.164747+0000","last_clean":"2024-09-10T18:49:37.164747+0000","last_became_active":"2024-09-10T18:45:24.079534+0000","last_became_peered":"2024-09-10T18:45:24.079534+0000","last_unstale":"2024-09-10T18:49:37.164747+0000","last_undegraded":"2024-09-10T18:49:37.164747+0000","last_fullsized":"2024-09-10T18:49:37.164747+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:10.102865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:10.102865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0042821509999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":104,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.171445+0000","last_change":"2024-09-10T18:48:59.577187+0000","last_active":"2024-09-10T18:49:37.171445+0000","last_peered":"2024-09-10T18:49:37.171445+0000","last_clean":"2024-09-10T18:49:37.171445+0000","last_became_active":"2024-09-10T18:47:36.414393+0000","last_became_peered":"2024-09-10T18:47:36.414393+0000","last_unstale":"2024-09-10T18:49:37.171445+0000","last_undegraded":"2024-09-10T18:49:37.171445+0000","last_fullsized":"2024-09-10T18:49:37.171445+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:59.576980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:59.576980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:16.200779+0000","scrub_duration":0.0056458419999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"63'9","reported_seq":107,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.163046+0000","last_change":"2024-09-10T18:49:23.482901+0000","last_active":"2024-09-10T18:49:37.163046+0000","last_peered":"2024-09-10T18:49:37.163046+0000","last_clean":"2024-09-10T18:49:37.163046+0000","last_became_active":"2024-09-10T18:45:20.052652+0000","last_became_peered":"2024-09-10T18:45:20.052652+0000","last_unstale":"2024-09-10T18:49:37.163046+0000","last_undegraded":"2024-09-10T18:49:37.163046+0000","last_fullsized":"2024-09-10T18:49:37.163046+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:49:23.482807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:49:23.482807+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:23.624215+0000","scrub_duration":0.014419628,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":89,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.164972+0000","last_change":"2024-09-10T18:48:32.141849+0000","last_active":"2024-09-10T18:49:37.164972+0000","last_peered":"2024-09-10T18:49:37.164972+0000","last_clean":"2024-09-10T18:49:37.164972+0000","last_became_active":"2024-09-10T18:45:22.068326+0000","last_became_peered":"2024-09-10T18:45:22.068326+0000","last_unstale":"2024-09-10T18:49:37.164972+0000","last_undegraded":"2024-09-10T18:49:37.164972+0000","last_fullsized":"2024-09-10T18:49:37.164972+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:32.141755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:32.141755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:42.742732+0000","scrub_duration":0.0040319509999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":88,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.167081+0000","last_change":"2024-09-10T18:49:02.511550+0000","last_active":"2024-09-10T18:49:37.167081+0000","last_peered":"2024-09-10T18:49:37.167081+0000","last_clean":"2024-09-10T18:49:37.167081+0000","last_became_active":"2024-09-10T18:45:24.081488+0000","last_became_peered":"2024-09-10T18:45:24.081488+0000","last_unstale":"2024-09-10T18:49:37.167081+0000","last_undegraded":"2024-09-10T18:49:37.167081+0000","last_fullsized":"2024-09-10T18:49:37.167081+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:02.511457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:02.511457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:18.180849+0000","scrub_duration":0.018959785999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"121'10253","reported_seq":10297,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T18:49:41.022873+0000","last_change":"2024-09-10T18:47:36.423039+0000","last_active":"2024-09-10T18:49:41.022873+0000","last_peered":"2024-09-10T18:49:41.022873+0000","last_clean":"2024-09-10T18:49:41.022873+0000","last_became_active":"2024-09-10T18:47:36.422694+0000","last_became_peered":"2024-09-10T18:47:36.422694+0000","last_unstale":"2024-09-10T18:49:41.022873+0000","last_undegraded":"2024-09-10T18:49:41.022873+0000","last_fullsized":"2024-09-10T18:49:41.022873+0000","mapping_epoch":88,"log_start":"106'5000","ondisk_log_start":"106'5000","created":86,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":5253,"log_dups_size":0,"ondisk_log_size":5253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":439812096,"num_objects":6711,"num_object_clones":0,"num_object_copies":20133,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10253,"num_write_kb":542848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"78'25","reported_seq":155,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.163351+0000","last_change":"2024-09-10T18:49:03.371577+0000","last_active":"2024-09-10T18:49:37.163351+0000","last_peered":"2024-09-10T18:49:37.163351+0000","last_clean":"2024-09-10T18:49:37.163351+0000","last_became_active":"2024-09-10T18:47:36.409300+0000","last_became_peered":"2024-09-10T18:47:36.409300+0000","last_unstale":"2024-09-10T18:49:37.163351+0000","last_undegraded":"2024-09-10T18:49:37.163351+0000","last_fullsized":"2024-09-10T18:49:37.163351+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'25","last_scrub_stamp":"2024-09-10T18:49:03.371486+0000","last_deep_scrub":"78'25","last_deep_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_clean_scrub_stamp":"2024-09-10T18:49:03.371486+0000","objects_scrubbed":8,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:04.078541+0000","scrub_duration":0.003278258,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":94,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.170963+0000","last_change":"2024-09-10T18:48:26.625704+0000","last_active":"2024-09-10T18:49:37.170963+0000","last_peered":"2024-09-10T18:49:37.170963+0000","last_clean":"2024-09-10T18:49:37.170963+0000","last_became_active":"2024-09-10T18:45:17.963732+0000","last_became_peered":"2024-09-10T18:45:17.963732+0000","last_unstale":"2024-09-10T18:49:37.170963+0000","last_undegraded":"2024-09-10T18:49:37.170963+0000","last_fullsized":"2024-09-10T18:49:37.170963+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:26.625635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:26.625635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:44.826250+0000","scrub_duration":0.0034872639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":90,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.161776+0000","last_change":"2024-09-10T18:49:17.310050+0000","last_active":"2024-09-10T18:49:37.161776+0000","last_peered":"2024-09-10T18:49:37.161776+0000","last_clean":"2024-09-10T18:49:37.161776+0000","last_became_active":"2024-09-10T18:45:22.071443+0000","last_became_peered":"2024-09-10T18:45:22.071443+0000","last_unstale":"2024-09-10T18:49:37.161776+0000","last_undegraded":"2024-09-10T18:49:37.161776+0000","last_fullsized":"2024-09-10T18:49:37.161776+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:17.309861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:04.512425+0000","last_clean_scrub_stamp":"2024-09-10T18:49:17.309861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:38.825418+0000","scrub_duration":0.0051958760000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":44,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.166271+0000","last_change":"2024-09-10T18:49:11.545445+0000","last_active":"2024-09-10T18:49:37.166271+0000","last_peered":"2024-09-10T18:49:37.166271+0000","last_clean":"2024-09-10T18:49:37.166271+0000","last_became_active":"2024-09-10T18:47:57.104433+0000","last_became_peered":"2024-09-10T18:47:57.104433+0000","last_unstale":"2024-09-10T18:49:37.166271+0000","last_undegraded":"2024-09-10T18:49:37.166271+0000","last_fullsized":"2024-09-10T18:49:37.166271+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:11.545188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:11.545188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:19.456782+0000","scrub_duration":0.0019872409999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.f","version":"121'10312","reported_seq":10699,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T18:49:41.364906+0000","last_change":"2024-09-10T18:49:17.808384+0000","last_active":"2024-09-10T18:49:41.364906+0000","last_peered":"2024-09-10T18:49:41.364906+0000","last_clean":"2024-09-10T18:49:41.364906+0000","last_became_active":"2024-09-10T18:47:34.396087+0000","last_became_peered":"2024-09-10T18:47:34.396087+0000","last_unstale":"2024-09-10T18:49:41.364906+0000","last_undegraded":"2024-09-10T18:49:41.364906+0000","last_fullsized":"2024-09-10T18:49:41.364906+0000","mapping_epoch":86,"log_start":"108'5500","ondisk_log_start":"108'5500","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'8705","last_scrub_stamp":"2024-09-10T18:49:17.808165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:49:17.808165+0000","objects_scrubbed":8330,"log_size":4812,"log_dups_size":0,"ondisk_log_size":4812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:32.296732+0000","scrub_duration":4.3205227400000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":430702592,"num_objects":6572,"num_object_clones":0,"num_object_copies":19716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10312,"num_write_kb":540288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"116'15","reported_seq":111,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.163155+0000","last_change":"2024-09-10T18:49:12.374625+0000","last_active":"2024-09-10T18:49:37.163155+0000","last_peered":"2024-09-10T18:49:37.163155+0000","last_clean":"2024-09-10T18:49:37.163155+0000","last_became_active":"2024-09-10T18:45:20.053428+0000","last_became_peered":"2024-09-10T18:45:20.053428+0000","last_unstale":"2024-09-10T18:49:37.163155+0000","last_undegraded":"2024-09-10T18:49:37.163155+0000","last_fullsized":"2024-09-10T18:49:37.163155+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"116'15","last_scrub_stamp":"2024-09-10T18:49:12.374527+0000","last_deep_scrub":"116'15","last_deep_scrub_stamp":"2024-09-10T18:49:12.374527+0000","last_clean_scrub_stamp":"2024-09-10T18:49:12.374527+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:35.011875+0000","scrub_duration":0.0081453819999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":94,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.163782+0000","last_change":"2024-09-10T18:48:29.551161+0000","last_active":"2024-09-10T18:49:37.163782+0000","last_peered":"2024-09-10T18:49:37.163782+0000","last_clean":"2024-09-10T18:49:37.163782+0000","last_became_active":"2024-09-10T18:45:17.966833+0000","last_became_peered":"2024-09-10T18:45:17.966833+0000","last_unstale":"2024-09-10T18:49:37.163782+0000","last_undegraded":"2024-09-10T18:49:37.163782+0000","last_fullsized":"2024-09-10T18:49:37.163782+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:29.550946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:29.550946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:50.968295+0000","scrub_duration":0.016562341000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"89'9","reported_seq":103,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.171152+0000","last_change":"2024-09-10T18:48:09.648420+0000","last_active":"2024-09-10T18:49:37.171152+0000","last_peered":"2024-09-10T18:49:37.171152+0000","last_clean":"2024-09-10T18:49:37.171152+0000","last_became_active":"2024-09-10T18:47:36.421188+0000","last_became_peered":"2024-09-10T18:47:36.421188+0000","last_unstale":"2024-09-10T18:49:37.171152+0000","last_undegraded":"2024-09-10T18:49:37.171152+0000","last_fullsized":"2024-09-10T18:49:37.171152+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'9","last_scrub_stamp":"2024-09-10T18:48:09.648322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:09.648322+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0066489619999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":97,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.163181+0000","last_change":"2024-09-10T18:48:32.043742+0000","last_active":"2024-09-10T18:49:37.163181+0000","last_peered":"2024-09-10T18:49:37.163181+0000","last_clean":"2024-09-10T18:49:37.163181+0000","last_became_active":"2024-09-10T18:47:36.414625+0000","last_became_peered":"2024-09-10T18:47:36.414625+0000","last_unstale":"2024-09-10T18:49:37.163181+0000","last_undegraded":"2024-09-10T18:49:37.163181+0000","last_fullsized":"2024-09-10T18:49:37.163181+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:32.043666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:32.043666+0000","last_clean_scrub_stamp":"2024-09-10T18:48:32.043666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:49.740069+0000","scrub_duration":0.001572417,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"121'10429","reported_seq":10822,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T18:49:41.020719+0000","last_change":"2024-09-10T18:49:14.077746+0000","last_active":"2024-09-10T18:49:41.020719+0000","last_peered":"2024-09-10T18:49:41.020719+0000","last_clean":"2024-09-10T18:49:41.020719+0000","last_became_active":"2024-09-10T18:47:34.399261+0000","last_became_peered":"2024-09-10T18:47:34.399261+0000","last_unstale":"2024-09-10T18:49:41.020719+0000","last_undegraded":"2024-09-10T18:49:41.020719+0000","last_fullsized":"2024-09-10T18:49:41.020719+0000","mapping_epoch":86,"log_start":"107'5200","ondisk_log_start":"107'5200","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"116'8616","last_scrub_stamp":"2024-09-10T18:49:14.077662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:49:14.077662+0000","objects_scrubbed":8528,"log_size":5229,"log_dups_size":0,"ondisk_log_size":5229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:23.558482+0000","scrub_duration":2.6235428359999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":448069632,"num_objects":6837,"num_object_clones":0,"num_object_copies":20511,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10429,"num_write_kb":552512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"63'15","reported_seq":83,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.165323+0000","last_change":"2024-09-10T18:49:18.042505+0000","last_active":"2024-09-10T18:49:37.165323+0000","last_peered":"2024-09-10T18:49:37.165323+0000","last_clean":"2024-09-10T18:49:37.165323+0000","last_became_active":"2024-09-10T18:46:56.742946+0000","last_became_peered":"2024-09-10T18:46:56.742946+0000","last_unstale":"2024-09-10T18:49:37.165323+0000","last_undegraded":"2024-09-10T18:49:37.165323+0000","last_fullsized":"2024-09-10T18:49:37.165323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:49:18.042442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:49:18.042442+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:20.073484+0000","scrub_duration":0.0044167360000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":94,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.162310+0000","last_change":"2024-09-10T18:48:13.707838+0000","last_active":"2024-09-10T18:49:37.162310+0000","last_peered":"2024-09-10T18:49:37.162310+0000","last_clean":"2024-09-10T18:49:37.162310+0000","last_became_active":"2024-09-10T18:45:17.965409+0000","last_became_peered":"2024-09-10T18:45:17.965409+0000","last_unstale":"2024-09-10T18:49:37.162310+0000","last_undegraded":"2024-09-10T18:49:37.162310+0000","last_fullsized":"2024-09-10T18:49:37.162310+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:13.707725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_clean_scrub_stamp":"2024-09-10T18:48:13.707725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0075610709999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"63'8","reported_seq":149,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.163249+0000","last_change":"2024-09-10T18:48:18.950238+0000","last_active":"2024-09-10T18:49:37.163249+0000","last_peered":"2024-09-10T18:49:37.163249+0000","last_clean":"2024-09-10T18:49:37.163249+0000","last_became_active":"2024-09-10T18:45:22.062497+0000","last_became_peered":"2024-09-10T18:45:22.062497+0000","last_unstale":"2024-09-10T18:49:37.163249+0000","last_undegraded":"2024-09-10T18:49:37.163249+0000","last_fullsized":"2024-09-10T18:49:37.163249+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:48:18.950140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:18.950140+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0032883439999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":87,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.165068+0000","last_change":"2024-09-10T18:48:31.170536+0000","last_active":"2024-09-10T18:49:37.165068+0000","last_peered":"2024-09-10T18:49:37.165068+0000","last_clean":"2024-09-10T18:49:37.165068+0000","last_became_active":"2024-09-10T18:45:24.082362+0000","last_became_peered":"2024-09-10T18:45:24.082362+0000","last_unstale":"2024-09-10T18:49:37.165068+0000","last_undegraded":"2024-09-10T18:49:37.165068+0000","last_fullsized":"2024-09-10T18:49:37.165068+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:31.170457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:31.170457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:50.433946+0000","scrub_duration":0.0084072260000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"121'10133","reported_seq":10512,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T18:49:41.052503+0000","last_change":"2024-09-10T18:49:12.677884+0000","last_active":"2024-09-10T18:49:41.052503+0000","last_peered":"2024-09-10T18:49:41.052503+0000","last_clean":"2024-09-10T18:49:41.052503+0000","last_became_active":"2024-09-10T18:47:34.399711+0000","last_became_peered":"2024-09-10T18:47:34.399711+0000","last_unstale":"2024-09-10T18:49:41.052503+0000","last_undegraded":"2024-09-10T18:49:41.052503+0000","last_fullsized":"2024-09-10T18:49:41.052503+0000","mapping_epoch":86,"log_start":"106'4900","ondisk_log_start":"106'4900","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"116'8278","last_scrub_stamp":"2024-09-10T18:49:12.677787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:49:12.677787+0000","objects_scrubbed":8161,"log_size":5233,"log_dups_size":0,"ondisk_log_size":5233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:23.222138+0000","scrub_duration":3.2225789539999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":434634776,"num_objects":6633,"num_object_clones":0,"num_object_copies":19899,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6633,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10133,"num_write_kb":536449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"63'11","reported_seq":176,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.163260+0000","last_change":"2024-09-10T18:48:11.900583+0000","last_active":"2024-09-10T18:49:37.163260+0000","last_peered":"2024-09-10T18:49:37.163260+0000","last_clean":"2024-09-10T18:49:37.163260+0000","last_became_active":"2024-09-10T18:47:57.106935+0000","last_became_peered":"2024-09-10T18:47:57.106935+0000","last_unstale":"2024-09-10T18:49:37.163260+0000","last_undegraded":"2024-09-10T18:49:37.163260+0000","last_fullsized":"2024-09-10T18:49:37.163260+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:48:11.900489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:11.900489+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020768789999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":94,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.167862+0000","last_change":"2024-09-10T18:48:12.642771+0000","last_active":"2024-09-10T18:49:37.167862+0000","last_peered":"2024-09-10T18:49:37.167862+0000","last_clean":"2024-09-10T18:49:37.167862+0000","last_became_active":"2024-09-10T18:45:17.966133+0000","last_became_peered":"2024-09-10T18:45:17.966133+0000","last_unstale":"2024-09-10T18:49:37.167862+0000","last_undegraded":"2024-09-10T18:49:37.167862+0000","last_fullsized":"2024-09-10T18:49:37.167862+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:12.642693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:12.642693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0035654889999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":89,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.165402+0000","last_change":"2024-09-10T18:48:25.093064+0000","last_active":"2024-09-10T18:49:37.165402+0000","last_peered":"2024-09-10T18:49:37.165402+0000","last_clean":"2024-09-10T18:49:37.165402+0000","last_became_active":"2024-09-10T18:45:22.068918+0000","last_became_peered":"2024-09-10T18:45:22.068918+0000","last_unstale":"2024-09-10T18:49:37.165402+0000","last_undegraded":"2024-09-10T18:49:37.165402+0000","last_fullsized":"2024-09-10T18:49:37.165402+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:25.092984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_clean_scrub_stamp":"2024-09-10T18:48:25.092984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:45.861853+0000","scrub_duration":0.0038537620000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":88,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.174543+0000","last_change":"2024-09-10T18:49:04.647711+0000","last_active":"2024-09-10T18:49:37.174543+0000","last_peered":"2024-09-10T18:49:37.174543+0000","last_clean":"2024-09-10T18:49:37.174543+0000","last_became_active":"2024-09-10T18:45:24.264538+0000","last_became_peered":"2024-09-10T18:45:24.264538+0000","last_unstale":"2024-09-10T18:49:37.174543+0000","last_undegraded":"2024-09-10T18:49:37.174543+0000","last_fullsized":"2024-09-10T18:49:37.174543+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:04.647638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:04.647638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:31.480091+0000","scrub_duration":0.115211389,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2","version":"121'10310","reported_seq":10363,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T18:49:41.207331+0000","last_change":"2024-09-10T18:47:36.423507+0000","last_active":"2024-09-10T18:49:41.207331+0000","last_peered":"2024-09-10T18:49:41.207331+0000","last_clean":"2024-09-10T18:49:41.207331+0000","last_became_active":"2024-09-10T18:47:36.423234+0000","last_became_peered":"2024-09-10T18:47:36.423234+0000","last_unstale":"2024-09-10T18:49:41.207331+0000","last_undegraded":"2024-09-10T18:49:41.207331+0000","last_fullsized":"2024-09-10T18:49:41.207331+0000","mapping_epoch":88,"log_start":"111'6000","ondisk_log_start":"111'6000","created":86,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":4310,"log_dups_size":0,"ondisk_log_size":4310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436469760,"num_objects":6660,"num_object_clones":0,"num_object_copies":19980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10310,"num_write_kb":543040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"63'14","reported_seq":112,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.162801+0000","last_change":"2024-09-10T18:49:04.392814+0000","last_active":"2024-09-10T18:49:37.162801+0000","last_peered":"2024-09-10T18:49:37.162801+0000","last_clean":"2024-09-10T18:49:37.162801+0000","last_became_active":"2024-09-10T18:45:20.052243+0000","last_became_peered":"2024-09-10T18:45:20.052243+0000","last_unstale":"2024-09-10T18:49:37.162801+0000","last_undegraded":"2024-09-10T18:49:37.162801+0000","last_fullsized":"2024-09-10T18:49:37.162801+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-10T18:49:04.392667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:49:04.392667+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:23.598321+0000","scrub_duration":0.0046253650000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":103,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.169890+0000","last_change":"2024-09-10T18:48:13.631307+0000","last_active":"2024-09-10T18:49:37.169890+0000","last_peered":"2024-09-10T18:49:37.169890+0000","last_clean":"2024-09-10T18:49:37.169890+0000","last_became_active":"2024-09-10T18:47:36.424466+0000","last_became_peered":"2024-09-10T18:47:36.424466+0000","last_unstale":"2024-09-10T18:49:37.169890+0000","last_undegraded":"2024-09-10T18:49:37.169890+0000","last_fullsized":"2024-09-10T18:49:37.169890+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:13.631078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_clean_scrub_stamp":"2024-09-10T18:48:13.631078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043023660000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":151,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T18:49:40.191680+0000","last_change":"2024-09-10T18:48:23.479877+0000","last_active":"2024-09-10T18:49:40.191680+0000","last_peered":"2024-09-10T18:49:40.191680+0000","last_clean":"2024-09-10T18:49:40.191680+0000","last_became_active":"2024-09-10T18:45:22.074540+0000","last_became_peered":"2024-09-10T18:45:22.074540+0000","last_unstale":"2024-09-10T18:49:40.191680+0000","last_undegraded":"2024-09-10T18:49:40.191680+0000","last_fullsized":"2024-09-10T18:49:40.191680+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:48:23.479762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:23.479762+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.01210871,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":108,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.170630+0000","last_change":"2024-09-10T18:49:23.479699+0000","last_active":"2024-09-10T18:49:37.170630+0000","last_peered":"2024-09-10T18:49:37.170630+0000","last_clean":"2024-09-10T18:49:37.170630+0000","last_became_active":"2024-09-10T18:47:36.414665+0000","last_became_peered":"2024-09-10T18:47:36.414665+0000","last_unstale":"2024-09-10T18:49:37.170630+0000","last_undegraded":"2024-09-10T18:49:37.170630+0000","last_fullsized":"2024-09-10T18:49:37.170630+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:23.479647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:23.479647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:23.624042+0000","scrub_duration":0.0057738609999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"121'10500","reported_seq":10553,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T18:49:41.217311+0000","last_change":"2024-09-10T18:47:36.410975+0000","last_active":"2024-09-10T18:49:41.217311+0000","last_peered":"2024-09-10T18:49:41.217311+0000","last_clean":"2024-09-10T18:49:41.217311+0000","last_became_active":"2024-09-10T18:47:36.410639+0000","last_became_peered":"2024-09-10T18:47:36.410639+0000","last_unstale":"2024-09-10T18:49:41.217311+0000","last_undegraded":"2024-09-10T18:49:41.217311+0000","last_fullsized":"2024-09-10T18:49:41.217311+0000","mapping_epoch":88,"log_start":"111'6200","ondisk_log_start":"111'6200","created":86,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":4300,"log_dups_size":0,"ondisk_log_size":4300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":446300160,"num_objects":6810,"num_object_clones":0,"num_object_copies":20430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10500,"num_write_kb":553920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"63'18","reported_seq":124,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.174098+0000","last_change":"2024-09-10T18:48:20.514584+0000","last_active":"2024-09-10T18:49:37.174098+0000","last_peered":"2024-09-10T18:49:37.174098+0000","last_clean":"2024-09-10T18:49:37.174098+0000","last_became_active":"2024-09-10T18:45:20.420937+0000","last_became_peered":"2024-09-10T18:45:20.420937+0000","last_unstale":"2024-09-10T18:49:37.174098+0000","last_undegraded":"2024-09-10T18:49:37.174098+0000","last_fullsized":"2024-09-10T18:49:37.174098+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'18","last_scrub_stamp":"2024-09-10T18:48:20.514501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:20.514501+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.028109085999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":94,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.162773+0000","last_change":"2024-09-10T18:49:23.336052+0000","last_active":"2024-09-10T18:49:37.162773+0000","last_peered":"2024-09-10T18:49:37.162773+0000","last_clean":"2024-09-10T18:49:37.162773+0000","last_became_active":"2024-09-10T18:45:17.961695+0000","last_became_peered":"2024-09-10T18:45:17.961695+0000","last_unstale":"2024-09-10T18:49:37.162773+0000","last_undegraded":"2024-09-10T18:49:37.162773+0000","last_fullsized":"2024-09-10T18:49:37.162773+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:23.335968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:23.335968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:52.115506+0000","scrub_duration":0.0056344669999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":89,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.162711+0000","last_change":"2024-09-10T18:48:27.016984+0000","last_active":"2024-09-10T18:49:37.162711+0000","last_peered":"2024-09-10T18:49:37.162711+0000","last_clean":"2024-09-10T18:49:37.162711+0000","last_became_active":"2024-09-10T18:45:22.074012+0000","last_became_peered":"2024-09-10T18:45:22.074012+0000","last_unstale":"2024-09-10T18:49:37.162711+0000","last_undegraded":"2024-09-10T18:49:37.162711+0000","last_fullsized":"2024-09-10T18:49:37.162711+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:27.016933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:27.016933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0059183930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":87,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.162907+0000","last_change":"2024-09-10T18:47:36.411129+0000","last_active":"2024-09-10T18:49:37.162907+0000","last_peered":"2024-09-10T18:49:37.162907+0000","last_clean":"2024-09-10T18:49:37.162907+0000","last_became_active":"2024-09-10T18:47:36.410836+0000","last_became_peered":"2024-09-10T18:47:36.410836+0000","last_unstale":"2024-09-10T18:49:37.162907+0000","last_undegraded":"2024-09-10T18:49:37.162907+0000","last_fullsized":"2024-09-10T18:49:37.162907+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_clean_scrub_stamp":"2024-09-10T18:46:53.087539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001678971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"121'10281","reported_seq":10323,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T18:49:41.032045+0000","last_change":"2024-09-10T18:47:34.399316+0000","last_active":"2024-09-10T18:49:41.032045+0000","last_peered":"2024-09-10T18:49:41.032045+0000","last_clean":"2024-09-10T18:49:41.032045+0000","last_became_active":"2024-09-10T18:47:34.399002+0000","last_became_peered":"2024-09-10T18:47:34.399002+0000","last_unstale":"2024-09-10T18:49:41.032045+0000","last_undegraded":"2024-09-10T18:49:41.032045+0000","last_fullsized":"2024-09-10T18:49:41.032045+0000","mapping_epoch":86,"log_start":"107'5100","ondisk_log_start":"107'5100","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":5181,"log_dups_size":0,"ondisk_log_size":5181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":437059584,"num_objects":6669,"num_object_clones":0,"num_object_copies":20007,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10281,"num_write_kb":542400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3","version":"63'19","reported_seq":128,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.166250+0000","last_change":"2024-09-10T18:48:11.781505+0000","last_active":"2024-09-10T18:49:37.166250+0000","last_peered":"2024-09-10T18:49:37.166250+0000","last_clean":"2024-09-10T18:49:37.166250+0000","last_became_active":"2024-09-10T18:45:20.421046+0000","last_became_peered":"2024-09-10T18:45:20.421046+0000","last_unstale":"2024-09-10T18:49:37.166250+0000","last_undegraded":"2024-09-10T18:49:37.166250+0000","last_fullsized":"2024-09-10T18:49:37.166250+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:48:11.781417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:11.781417+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030109379999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":94,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.171302+0000","last_change":"2024-09-10T18:48:30.694038+0000","last_active":"2024-09-10T18:49:37.171302+0000","last_peered":"2024-09-10T18:49:37.171302+0000","last_clean":"2024-09-10T18:49:37.171302+0000","last_became_active":"2024-09-10T18:45:17.960226+0000","last_became_peered":"2024-09-10T18:45:17.960226+0000","last_unstale":"2024-09-10T18:49:37.171302+0000","last_undegraded":"2024-09-10T18:49:37.171302+0000","last_fullsized":"2024-09-10T18:49:37.171302+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:30.693950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:30.693950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:49.091630+0000","scrub_duration":0.0079194309999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":89,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.168887+0000","last_change":"2024-09-10T18:48:33.513604+0000","last_active":"2024-09-10T18:49:37.168887+0000","last_peered":"2024-09-10T18:49:37.168887+0000","last_clean":"2024-09-10T18:49:37.168887+0000","last_became_active":"2024-09-10T18:45:22.066148+0000","last_became_peered":"2024-09-10T18:45:22.066148+0000","last_unstale":"2024-09-10T18:49:37.168887+0000","last_undegraded":"2024-09-10T18:49:37.168887+0000","last_fullsized":"2024-09-10T18:49:37.168887+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:33.513500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:33.513500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:47.158850+0000","scrub_duration":0.0080948319999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":98,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.162075+0000","last_change":"2024-09-10T18:49:18.281313+0000","last_active":"2024-09-10T18:49:37.162075+0000","last_peered":"2024-09-10T18:49:37.162075+0000","last_clean":"2024-09-10T18:49:37.162075+0000","last_became_active":"2024-09-10T18:47:36.422295+0000","last_became_peered":"2024-09-10T18:47:36.422295+0000","last_unstale":"2024-09-10T18:49:37.162075+0000","last_undegraded":"2024-09-10T18:49:37.162075+0000","last_fullsized":"2024-09-10T18:49:37.162075+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:18.281230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:49:18.281230+0000","last_clean_scrub_stamp":"2024-09-10T18:49:18.281230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:42.524080+0000","scrub_duration":0.0066273570000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"121'10261","reported_seq":10301,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T18:49:41.049992+0000","last_change":"2024-09-10T18:47:34.397576+0000","last_active":"2024-09-10T18:49:41.049992+0000","last_peered":"2024-09-10T18:49:41.049992+0000","last_clean":"2024-09-10T18:49:41.049992+0000","last_became_active":"2024-09-10T18:47:34.396631+0000","last_became_peered":"2024-09-10T18:47:34.396631+0000","last_unstale":"2024-09-10T18:49:41.049992+0000","last_undegraded":"2024-09-10T18:49:41.049992+0000","last_fullsized":"2024-09-10T18:49:41.049992+0000","mapping_epoch":86,"log_start":"107'5100","ondisk_log_start":"107'5100","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":5161,"log_dups_size":0,"ondisk_log_size":5161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":438894592,"num_objects":6697,"num_object_clones":0,"num_object_copies":20091,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10261,"num_write_kb":542656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"83'15","reported_seq":112,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.163609+0000","last_change":"2024-09-10T18:48:20.633119+0000","last_active":"2024-09-10T18:49:37.163609+0000","last_peered":"2024-09-10T18:49:37.163609+0000","last_clean":"2024-09-10T18:49:37.163609+0000","last_became_active":"2024-09-10T18:45:20.057204+0000","last_became_peered":"2024-09-10T18:45:20.057204+0000","last_unstale":"2024-09-10T18:49:37.163609+0000","last_undegraded":"2024-09-10T18:49:37.163609+0000","last_fullsized":"2024-09-10T18:49:37.163609+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'15","last_scrub_stamp":"2024-09-10T18:48:20.633054+0000","last_deep_scrub":"83'15","last_deep_scrub_stamp":"2024-09-10T18:46:58.436473+0000","last_clean_scrub_stamp":"2024-09-10T18:48:20.633054+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.008267412,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":94,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.162002+0000","last_change":"2024-09-10T18:49:09.449135+0000","last_active":"2024-09-10T18:49:37.162002+0000","last_peered":"2024-09-10T18:49:37.162002+0000","last_clean":"2024-09-10T18:49:37.162002+0000","last_became_active":"2024-09-10T18:45:17.966632+0000","last_became_peered":"2024-09-10T18:45:17.966632+0000","last_unstale":"2024-09-10T18:49:37.162002+0000","last_undegraded":"2024-09-10T18:49:37.162002+0000","last_fullsized":"2024-09-10T18:49:37.162002+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:09.449037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:09.449037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:18.822485+0000","scrub_duration":0.013065797000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"63'8","reported_seq":151,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T18:49:40.195642+0000","last_change":"2024-09-10T18:49:00.552860+0000","last_active":"2024-09-10T18:49:40.195642+0000","last_peered":"2024-09-10T18:49:40.195642+0000","last_clean":"2024-09-10T18:49:40.195642+0000","last_became_active":"2024-09-10T18:45:22.063326+0000","last_became_peered":"2024-09-10T18:45:22.063326+0000","last_unstale":"2024-09-10T18:49:40.195642+0000","last_undegraded":"2024-09-10T18:49:40.195642+0000","last_fullsized":"2024-09-10T18:49:40.195642+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:49:00.552765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:49:00.552765+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:28.435607+0000","scrub_duration":0.0038235579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":98,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.166868+0000","last_change":"2024-09-10T18:49:01.532633+0000","last_active":"2024-09-10T18:49:37.166868+0000","last_peered":"2024-09-10T18:49:37.166868+0000","last_clean":"2024-09-10T18:49:37.166868+0000","last_became_active":"2024-09-10T18:47:36.412111+0000","last_became_peered":"2024-09-10T18:47:36.412111+0000","last_unstale":"2024-09-10T18:49:37.166868+0000","last_undegraded":"2024-09-10T18:49:37.166868+0000","last_fullsized":"2024-09-10T18:49:37.166868+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:01.532550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:01.532550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:17.367791+0000","scrub_duration":0.0037383170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"121'10304","reported_seq":10666,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T18:49:41.046573+0000","last_change":"2024-09-10T18:49:05.169424+0000","last_active":"2024-09-10T18:49:41.046573+0000","last_peered":"2024-09-10T18:49:41.046573+0000","last_clean":"2024-09-10T18:49:41.046573+0000","last_became_active":"2024-09-10T18:47:34.400044+0000","last_became_peered":"2024-09-10T18:47:34.400044+0000","last_unstale":"2024-09-10T18:49:41.046573+0000","last_undegraded":"2024-09-10T18:49:41.046573+0000","last_fullsized":"2024-09-10T18:49:41.046573+0000","mapping_epoch":86,"log_start":"106'5100","ondisk_log_start":"106'5100","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"115'7848","last_scrub_stamp":"2024-09-10T18:49:05.169330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:49:05.169330+0000","objects_scrubbed":7724,"log_size":5204,"log_dups_size":0,"ondisk_log_size":5204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:16.559490+0000","scrub_duration":3.7256509850000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":437387264,"num_objects":6674,"num_object_clones":0,"num_object_copies":20022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10304,"num_write_kb":543296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"89'29","reported_seq":225,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T18:49:40.184306+0000","last_change":"2024-09-10T18:47:36.443205+0000","last_active":"2024-09-10T18:49:40.184306+0000","last_peered":"2024-09-10T18:49:40.184306+0000","last_clean":"2024-09-10T18:49:40.184306+0000","last_became_active":"2024-09-10T18:47:36.420185+0000","last_became_peered":"2024-09-10T18:47:36.420185+0000","last_unstale":"2024-09-10T18:49:40.184306+0000","last_undegraded":"2024-09-10T18:49:40.184306+0000","last_fullsized":"2024-09-10T18:49:40.184306+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'28","last_scrub_stamp":"2024-09-10T18:47:26.525861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:26.525861+0000","objects_scrubbed":10,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002126719,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":93,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.165145+0000","last_change":"2024-09-10T18:48:33.186208+0000","last_active":"2024-09-10T18:49:37.165145+0000","last_peered":"2024-09-10T18:49:37.165145+0000","last_clean":"2024-09-10T18:49:37.165145+0000","last_became_active":"2024-09-10T18:45:17.964494+0000","last_became_peered":"2024-09-10T18:45:17.964494+0000","last_unstale":"2024-09-10T18:49:37.165145+0000","last_undegraded":"2024-09-10T18:49:37.165145+0000","last_fullsized":"2024-09-10T18:49:37.165145+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:33.186070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:33.186070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:00.114296+0000","scrub_duration":0.0068142150000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":44,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.169426+0000","last_change":"2024-09-10T18:49:00.438027+0000","last_active":"2024-09-10T18:49:37.169426+0000","last_peered":"2024-09-10T18:49:37.169426+0000","last_clean":"2024-09-10T18:49:37.169426+0000","last_became_active":"2024-09-10T18:47:57.107478+0000","last_became_peered":"2024-09-10T18:47:57.107478+0000","last_unstale":"2024-09-10T18:49:37.169426+0000","last_undegraded":"2024-09-10T18:49:37.169426+0000","last_fullsized":"2024-09-10T18:49:37.169426+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:00.437877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:49:00.437877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:04.529124+0000","scrub_duration":0.012117157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":78,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.175114+0000","last_change":"2024-09-10T18:47:22.891142+0000","last_active":"2024-09-10T18:49:37.175114+0000","last_peered":"2024-09-10T18:49:37.175114+0000","last_clean":"2024-09-10T18:49:37.175114+0000","last_became_active":"2024-09-10T18:45:24.073307+0000","last_became_peered":"2024-09-10T18:45:24.073307+0000","last_unstale":"2024-09-10T18:49:37.175114+0000","last_undegraded":"2024-09-10T18:49:37.175114+0000","last_fullsized":"2024-09-10T18:49:37.175114+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:22.891084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:22.891084+0000","last_clean_scrub_stamp":"2024-09-10T18:47:22.891084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017788579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"121'10227","reported_seq":10593,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T18:49:41.364621+0000","last_change":"2024-09-10T18:49:07.011995+0000","last_active":"2024-09-10T18:49:41.364621+0000","last_peered":"2024-09-10T18:49:41.364621+0000","last_clean":"2024-09-10T18:49:41.364621+0000","last_became_active":"2024-09-10T18:47:34.395271+0000","last_became_peered":"2024-09-10T18:47:34.395271+0000","last_unstale":"2024-09-10T18:49:41.364621+0000","last_undegraded":"2024-09-10T18:49:41.364621+0000","last_fullsized":"2024-09-10T18:49:41.364621+0000","mapping_epoch":86,"log_start":"108'5400","ondisk_log_start":"108'5400","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"115'7913","last_scrub_stamp":"2024-09-10T18:49:07.011758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:49:07.011758+0000","objects_scrubbed":7815,"log_size":4827,"log_dups_size":0,"ondisk_log_size":4827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:20.945730+0000","scrub_duration":2.4953754379999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":431947776,"num_objects":6591,"num_object_clones":0,"num_object_copies":19773,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10227,"num_write_kb":538176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"84'18","reported_seq":124,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.162218+0000","last_change":"2024-09-10T18:49:13.417595+0000","last_active":"2024-09-10T18:49:37.162218+0000","last_peered":"2024-09-10T18:49:37.162218+0000","last_clean":"2024-09-10T18:49:37.162218+0000","last_became_active":"2024-09-10T18:45:20.058435+0000","last_became_peered":"2024-09-10T18:45:20.058435+0000","last_unstale":"2024-09-10T18:49:37.162218+0000","last_undegraded":"2024-09-10T18:49:37.162218+0000","last_fullsized":"2024-09-10T18:49:37.162218+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'18","last_scrub_stamp":"2024-09-10T18:49:13.417302+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-10T18:47:10.514951+0000","last_clean_scrub_stamp":"2024-09-10T18:49:13.417302+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:34.815917+0000","scrub_duration":0.0096130710000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"84'6","reported_seq":361,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T18:49:40.184955+0000","last_change":"2024-09-10T18:47:36.439529+0000","last_active":"2024-09-10T18:49:40.184955+0000","last_peered":"2024-09-10T18:49:40.184955+0000","last_clean":"2024-09-10T18:49:40.184955+0000","last_became_active":"2024-09-10T18:47:36.418211+0000","last_became_peered":"2024-09-10T18:47:36.418211+0000","last_unstale":"2024-09-10T18:49:40.184955+0000","last_undegraded":"2024-09-10T18:49:40.184955+0000","last_fullsized":"2024-09-10T18:49:40.184955+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'6","last_scrub_stamp":"2024-09-10T18:47:22.545450+0000","last_deep_scrub":"84'6","last_deep_scrub_stamp":"2024-09-10T18:47:22.545450+0000","last_clean_scrub_stamp":"2024-09-10T18:47:22.545450+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022780040000000001,"objects_trimmed":0,"snaptrim_duration":0.0021933159999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":253,"num_read_kb":248,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":55,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.171245+0000","last_change":"2024-09-10T18:49:16.371789+0000","last_active":"2024-09-10T18:49:37.171245+0000","last_peered":"2024-09-10T18:49:37.171245+0000","last_clean":"2024-09-10T18:49:37.171245+0000","last_became_active":"2024-09-10T18:47:36.422641+0000","last_became_peered":"2024-09-10T18:47:36.422641+0000","last_unstale":"2024-09-10T18:49:37.171245+0000","last_undegraded":"2024-09-10T18:49:37.171245+0000","last_fullsized":"2024-09-10T18:49:37.171245+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:16.371702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:16.371702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:39.416796+0000","scrub_duration":0.0099806840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":54,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.165441+0000","last_change":"2024-09-10T18:48:22.011954+0000","last_active":"2024-09-10T18:49:37.165441+0000","last_peered":"2024-09-10T18:49:37.165441+0000","last_clean":"2024-09-10T18:49:37.165441+0000","last_became_active":"2024-09-10T18:47:36.419081+0000","last_became_peered":"2024-09-10T18:47:36.419081+0000","last_unstale":"2024-09-10T18:49:37.165441+0000","last_undegraded":"2024-09-10T18:49:37.165441+0000","last_fullsized":"2024-09-10T18:49:37.165441+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:22.011870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:22.011870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:41.762812+0000","scrub_duration":0.0018770919999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":88,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.171060+0000","last_change":"2024-09-10T18:49:12.603922+0000","last_active":"2024-09-10T18:49:37.171060+0000","last_peered":"2024-09-10T18:49:37.171060+0000","last_clean":"2024-09-10T18:49:37.171060+0000","last_became_active":"2024-09-10T18:45:24.264617+0000","last_became_peered":"2024-09-10T18:45:24.264617+0000","last_unstale":"2024-09-10T18:49:37.171060+0000","last_undegraded":"2024-09-10T18:49:37.171060+0000","last_fullsized":"2024-09-10T18:49:37.171060+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:12.603786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:12.603786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:36.865751+0000","scrub_duration":0.0019980200000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"121'10262","reported_seq":10654,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T18:49:41.052749+0000","last_change":"2024-09-10T18:49:19.513064+0000","last_active":"2024-09-10T18:49:41.052749+0000","last_peered":"2024-09-10T18:49:41.052749+0000","last_clean":"2024-09-10T18:49:41.052749+0000","last_became_active":"2024-09-10T18:47:36.417075+0000","last_became_peered":"2024-09-10T18:47:36.417075+0000","last_unstale":"2024-09-10T18:49:41.052749+0000","last_undegraded":"2024-09-10T18:49:41.052749+0000","last_fullsized":"2024-09-10T18:49:41.052749+0000","mapping_epoch":88,"log_start":"111'5700","ondisk_log_start":"111'5700","created":86,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'8906","last_scrub_stamp":"2024-09-10T18:49:19.512981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:49:19.512981+0000","objects_scrubbed":8201,"log_size":4562,"log_dups_size":0,"ondisk_log_size":4562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:23.685691+0000","scrub_duration":2.9222927400000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":441581568,"num_objects":6738,"num_object_clones":0,"num_object_copies":20214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10262,"num_write_kb":544000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"63'12","reported_seq":134,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.166214+0000","last_change":"2024-09-10T18:48:28.710224+0000","last_active":"2024-09-10T18:49:37.166214+0000","last_peered":"2024-09-10T18:49:37.166214+0000","last_clean":"2024-09-10T18:49:37.166214+0000","last_became_active":"2024-09-10T18:47:36.417965+0000","last_became_peered":"2024-09-10T18:47:36.417965+0000","last_unstale":"2024-09-10T18:49:37.166214+0000","last_undegraded":"2024-09-10T18:49:37.166214+0000","last_fullsized":"2024-09-10T18:49:37.166214+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:48:28.710113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:28.710113+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:58.705191+0000","scrub_duration":0.0069768260000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"58'1","reported_seq":100,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.185449+0000","last_change":"2024-09-10T18:49:07.525629+0000","last_active":"2024-09-10T18:49:37.185449+0000","last_peered":"2024-09-10T18:49:37.185449+0000","last_clean":"2024-09-10T18:49:37.185449+0000","last_became_active":"2024-09-10T18:45:17.963932+0000","last_became_peered":"2024-09-10T18:45:17.963932+0000","last_unstale":"2024-09-10T18:49:37.185449+0000","last_undegraded":"2024-09-10T18:49:37.185449+0000","last_fullsized":"2024-09-10T18:49:37.185449+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'1","last_scrub_stamp":"2024-09-10T18:49:07.525525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:07.525525+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:30.019519+0000","scrub_duration":0.01484512,"objects_trimmed":0,"snaptrim_duration":0.00037812099999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":52,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.167565+0000","last_change":"2024-09-10T18:49:07.603571+0000","last_active":"2024-09-10T18:49:37.167565+0000","last_peered":"2024-09-10T18:49:37.167565+0000","last_clean":"2024-09-10T18:49:37.167565+0000","last_became_active":"2024-09-10T18:47:36.416641+0000","last_became_peered":"2024-09-10T18:47:36.416641+0000","last_unstale":"2024-09-10T18:49:37.167565+0000","last_undegraded":"2024-09-10T18:49:37.167565+0000","last_fullsized":"2024-09-10T18:49:37.167565+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:07.603516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:07.603516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:29.660107+0000","scrub_duration":0.0079160310000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":89,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.165422+0000","last_change":"2024-09-10T18:49:11.029146+0000","last_active":"2024-09-10T18:49:37.165422+0000","last_peered":"2024-09-10T18:49:37.165422+0000","last_clean":"2024-09-10T18:49:37.165422+0000","last_became_active":"2024-09-10T18:45:22.064285+0000","last_became_peered":"2024-09-10T18:45:22.064285+0000","last_unstale":"2024-09-10T18:49:37.165422+0000","last_undegraded":"2024-09-10T18:49:37.165422+0000","last_fullsized":"2024-09-10T18:49:37.165422+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:11.029069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:49:11.029069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:18.876406+0000","scrub_duration":0.0021808299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":88,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.161534+0000","last_change":"2024-09-10T18:48:21.654220+0000","last_active":"2024-09-10T18:49:37.161534+0000","last_peered":"2024-09-10T18:49:37.161534+0000","last_clean":"2024-09-10T18:49:37.161534+0000","last_became_active":"2024-09-10T18:45:24.083689+0000","last_became_peered":"2024-09-10T18:45:24.083689+0000","last_unstale":"2024-09-10T18:49:37.161534+0000","last_undegraded":"2024-09-10T18:49:37.161534+0000","last_fullsized":"2024-09-10T18:49:37.161534+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:21.654007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:21.654007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058335929999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"121'10256","reported_seq":10630,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T18:49:41.375548+0000","last_change":"2024-09-10T18:49:22.290906+0000","last_active":"2024-09-10T18:49:41.375548+0000","last_peered":"2024-09-10T18:49:41.375548+0000","last_clean":"2024-09-10T18:49:41.375548+0000","last_became_active":"2024-09-10T18:47:34.396853+0000","last_became_peered":"2024-09-10T18:47:34.396853+0000","last_unstale":"2024-09-10T18:49:41.375548+0000","last_undegraded":"2024-09-10T18:49:41.375548+0000","last_fullsized":"2024-09-10T18:49:41.375548+0000","mapping_epoch":86,"log_start":"108'5400","ondisk_log_start":"108'5400","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'9002","last_scrub_stamp":"2024-09-10T18:49:22.290682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:49:22.290682+0000","objects_scrubbed":7999,"log_size":4856,"log_dups_size":0,"ondisk_log_size":4856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:24.221318+0000","scrub_duration":3.8586685599999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":432799744,"num_objects":6604,"num_object_clones":0,"num_object_copies":19812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10256,"num_write_kb":539520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"63'14","reported_seq":113,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.169600+0000","last_change":"2024-09-10T18:48:17.669939+0000","last_active":"2024-09-10T18:49:37.169600+0000","last_peered":"2024-09-10T18:49:37.169600+0000","last_clean":"2024-09-10T18:49:37.169600+0000","last_became_active":"2024-09-10T18:45:20.421119+0000","last_became_peered":"2024-09-10T18:45:20.421119+0000","last_unstale":"2024-09-10T18:49:37.169600+0000","last_undegraded":"2024-09-10T18:49:37.169600+0000","last_fullsized":"2024-09-10T18:49:37.169600+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-10T18:48:17.669787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:17.669787+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024048300000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"60'2","reported_seq":101,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.171357+0000","last_change":"2024-09-10T18:48:59.558866+0000","last_active":"2024-09-10T18:49:37.171357+0000","last_peered":"2024-09-10T18:49:37.171357+0000","last_clean":"2024-09-10T18:49:37.171357+0000","last_became_active":"2024-09-10T18:45:17.961660+0000","last_became_peered":"2024-09-10T18:45:17.961660+0000","last_unstale":"2024-09-10T18:49:37.171357+0000","last_undegraded":"2024-09-10T18:49:37.171357+0000","last_fullsized":"2024-09-10T18:49:37.171357+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-10T18:48:59.558773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:59.558773+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:15.175215+0000","scrub_duration":0.0043139010000000002,"objects_trimmed":0,"snaptrim_duration":0.0013942640000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":103,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.165207+0000","last_change":"2024-09-10T18:48:34.150227+0000","last_active":"2024-09-10T18:49:37.165207+0000","last_peered":"2024-09-10T18:49:37.165207+0000","last_clean":"2024-09-10T18:49:37.165207+0000","last_became_active":"2024-09-10T18:47:36.414636+0000","last_became_peered":"2024-09-10T18:47:36.414636+0000","last_unstale":"2024-09-10T18:49:37.165207+0000","last_undegraded":"2024-09-10T18:49:37.165207+0000","last_fullsized":"2024-09-10T18:49:37.165207+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:34.150146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:34.150146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:02.371194+0000","scrub_duration":0.0018375380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'81","reported_seq":138,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.167899+0000","last_change":"2024-09-10T18:49:09.660774+0000","last_active":"2024-09-10T18:49:37.167899+0000","last_peered":"2024-09-10T18:49:37.167899+0000","last_clean":"2024-09-10T18:49:37.167899+0000","last_became_active":"2024-09-10T18:45:07.433592+0000","last_became_peered":"2024-09-10T18:45:07.433592+0000","last_unstale":"2024-09-10T18:49:37.167899+0000","last_undegraded":"2024-09-10T18:49:37.167899+0000","last_fullsized":"2024-09-10T18:49:37.167899+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:49:09.660690+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:49:09.660690+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:31.143136+0000","scrub_duration":0.016709736999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":110,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.170877+0000","last_change":"2024-09-10T18:49:14.625034+0000","last_active":"2024-09-10T18:49:37.170877+0000","last_peered":"2024-09-10T18:49:37.170877+0000","last_clean":"2024-09-10T18:49:37.170877+0000","last_became_active":"2024-09-10T18:47:36.418320+0000","last_became_peered":"2024-09-10T18:47:36.418320+0000","last_unstale":"2024-09-10T18:49:37.170877+0000","last_undegraded":"2024-09-10T18:49:37.170877+0000","last_fullsized":"2024-09-10T18:49:37.170877+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:14.624957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:49:14.624957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:37.640823+0000","scrub_duration":0.0023025350000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":87,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.165500+0000","last_change":"2024-09-10T18:49:19.075811+0000","last_active":"2024-09-10T18:49:37.165500+0000","last_peered":"2024-09-10T18:49:37.165500+0000","last_clean":"2024-09-10T18:49:37.165500+0000","last_became_active":"2024-09-10T18:45:24.073327+0000","last_became_peered":"2024-09-10T18:45:24.073327+0000","last_unstale":"2024-09-10T18:49:37.165500+0000","last_undegraded":"2024-09-10T18:49:37.165500+0000","last_fullsized":"2024-09-10T18:49:37.165500+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:19.075748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:19.075748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:25.971565+0000","scrub_duration":0.0055106640000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.9","version":"121'14391","reported_seq":14619,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T18:49:41.382754+0000","last_change":"2024-09-10T18:49:40.255966+0000","last_active":"2024-09-10T18:49:41.382754+0000","last_peered":"2024-09-10T18:49:41.382754+0000","last_clean":"2024-09-10T18:49:41.382754+0000","last_became_active":"2024-09-10T18:47:34.393620+0000","last_became_peered":"2024-09-10T18:47:34.393620+0000","last_unstale":"2024-09-10T18:49:41.382754+0000","last_undegraded":"2024-09-10T18:49:41.382754+0000","last_fullsized":"2024-09-10T18:49:41.382754+0000","mapping_epoch":86,"log_start":"117'9602","ondisk_log_start":"117'9602","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":4474,"log_size":4789,"log_dups_size":0,"ondisk_log_size":4789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":446431232,"num_objects":6812,"num_object_clones":0,"num_object_copies":20436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6812,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10532,"num_write_kb":555008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"63'19","reported_seq":122,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.169053+0000","last_change":"2024-09-10T18:48:14.655578+0000","last_active":"2024-09-10T18:49:37.169053+0000","last_peered":"2024-09-10T18:49:37.169053+0000","last_clean":"2024-09-10T18:49:37.169053+0000","last_became_active":"2024-09-10T18:45:20.421097+0000","last_became_peered":"2024-09-10T18:45:20.421097+0000","last_unstale":"2024-09-10T18:49:37.169053+0000","last_undegraded":"2024-09-10T18:49:37.169053+0000","last_fullsized":"2024-09-10T18:49:37.169053+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:48:14.655527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:14.655527+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00174087,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":54,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.165309+0000","last_change":"2024-09-10T18:49:08.070170+0000","last_active":"2024-09-10T18:49:37.165309+0000","last_peered":"2024-09-10T18:49:37.165309+0000","last_clean":"2024-09-10T18:49:37.165309+0000","last_became_active":"2024-09-10T18:47:36.417800+0000","last_became_peered":"2024-09-10T18:47:36.417800+0000","last_unstale":"2024-09-10T18:49:37.165309+0000","last_undegraded":"2024-09-10T18:49:37.165309+0000","last_fullsized":"2024-09-10T18:49:37.165309+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:08.070063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:08.070063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:15.698762+0000","scrub_duration":0.0033662290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":65,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.166027+0000","last_change":"2024-09-10T18:49:20.477390+0000","last_active":"2024-09-10T18:49:37.166027+0000","last_peered":"2024-09-10T18:49:37.166027+0000","last_clean":"2024-09-10T18:49:37.166027+0000","last_became_active":"2024-09-10T18:47:36.413618+0000","last_became_peered":"2024-09-10T18:47:36.413618+0000","last_unstale":"2024-09-10T18:49:37.166027+0000","last_undegraded":"2024-09-10T18:49:37.166027+0000","last_fullsized":"2024-09-10T18:49:37.166027+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:20.477302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:49:20.477302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:43.626544+0000","scrub_duration":0.0047920660000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":98,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.167808+0000","last_change":"2024-09-10T18:49:11.634724+0000","last_active":"2024-09-10T18:49:37.167808+0000","last_peered":"2024-09-10T18:49:37.167808+0000","last_clean":"2024-09-10T18:49:37.167808+0000","last_became_active":"2024-09-10T18:47:36.412610+0000","last_became_peered":"2024-09-10T18:47:36.412610+0000","last_unstale":"2024-09-10T18:49:37.167808+0000","last_undegraded":"2024-09-10T18:49:37.167808+0000","last_fullsized":"2024-09-10T18:49:37.167808+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:11.634636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:11.634636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:37.083248+0000","scrub_duration":0.0058360749999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"121'10361","reported_seq":10736,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T18:49:41.025810+0000","last_change":"2024-09-10T18:49:22.931254+0000","last_active":"2024-09-10T18:49:41.025810+0000","last_peered":"2024-09-10T18:49:41.025810+0000","last_clean":"2024-09-10T18:49:41.025810+0000","last_became_active":"2024-09-10T18:47:36.412142+0000","last_became_peered":"2024-09-10T18:47:36.412142+0000","last_unstale":"2024-09-10T18:49:41.025810+0000","last_undegraded":"2024-09-10T18:49:41.025810+0000","last_fullsized":"2024-09-10T18:49:41.025810+0000","mapping_epoch":88,"log_start":"111'5800","ondisk_log_start":"111'5800","created":86,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'9054","last_scrub_stamp":"2024-09-10T18:49:22.931202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:49:22.931202+0000","objects_scrubbed":7988,"log_size":4561,"log_dups_size":0,"ondisk_log_size":4561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:28.321697+0000","scrub_duration":2.3919399270000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":432209920,"num_objects":6595,"num_object_clones":0,"num_object_copies":19785,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10361,"num_write_kb":542592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"63'9","reported_seq":108,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.166178+0000","last_change":"2024-09-10T18:48:22.878265+0000","last_active":"2024-09-10T18:49:37.166178+0000","last_peered":"2024-09-10T18:49:37.166178+0000","last_clean":"2024-09-10T18:49:37.166178+0000","last_became_active":"2024-09-10T18:45:20.061021+0000","last_became_peered":"2024-09-10T18:45:20.061021+0000","last_unstale":"2024-09-10T18:49:37.166178+0000","last_undegraded":"2024-09-10T18:49:37.166178+0000","last_fullsized":"2024-09-10T18:49:37.166178+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:48:22.878179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:22.878179+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:49.491061+0000","scrub_duration":0.021357979999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":94,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.162904+0000","last_change":"2024-09-10T18:49:07.384707+0000","last_active":"2024-09-10T18:49:37.162904+0000","last_peered":"2024-09-10T18:49:37.162904+0000","last_clean":"2024-09-10T18:49:37.162904+0000","last_became_active":"2024-09-10T18:45:17.963057+0000","last_became_peered":"2024-09-10T18:45:17.963057+0000","last_unstale":"2024-09-10T18:49:37.162904+0000","last_undegraded":"2024-09-10T18:49:37.162904+0000","last_fullsized":"2024-09-10T18:49:37.162904+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:07.384620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:07.384620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:25.314632+0000","scrub_duration":0.011420041000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":54,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.162818+0000","last_change":"2024-09-10T18:49:21.442258+0000","last_active":"2024-09-10T18:49:37.162818+0000","last_peered":"2024-09-10T18:49:37.162818+0000","last_clean":"2024-09-10T18:49:37.162818+0000","last_became_active":"2024-09-10T18:47:36.411517+0000","last_became_peered":"2024-09-10T18:47:36.411517+0000","last_unstale":"2024-09-10T18:49:37.162818+0000","last_undegraded":"2024-09-10T18:49:37.162818+0000","last_fullsized":"2024-09-10T18:49:37.162818+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:21.442183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_clean_scrub_stamp":"2024-09-10T18:49:21.442183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:24.591614+0000","scrub_duration":0.0039195419999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":98,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.185357+0000","last_change":"2024-09-10T18:48:25.804464+0000","last_active":"2024-09-10T18:49:37.185357+0000","last_peered":"2024-09-10T18:49:37.185357+0000","last_clean":"2024-09-10T18:49:37.185357+0000","last_became_active":"2024-09-10T18:47:36.412933+0000","last_became_peered":"2024-09-10T18:47:36.412933+0000","last_unstale":"2024-09-10T18:49:37.185357+0000","last_undegraded":"2024-09-10T18:49:37.185357+0000","last_fullsized":"2024-09-10T18:49:37.185357+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:25.804376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:47.545343+0000","last_clean_scrub_stamp":"2024-09-10T18:48:25.804376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0098886110000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.b","version":"121'10334","reported_seq":10377,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T18:49:41.171256+0000","last_change":"2024-09-10T18:47:34.398004+0000","last_active":"2024-09-10T18:49:41.171256+0000","last_peered":"2024-09-10T18:49:41.171256+0000","last_clean":"2024-09-10T18:49:41.171256+0000","last_became_active":"2024-09-10T18:47:34.397708+0000","last_became_peered":"2024-09-10T18:47:34.397708+0000","last_unstale":"2024-09-10T18:49:41.171256+0000","last_undegraded":"2024-09-10T18:49:41.171256+0000","last_fullsized":"2024-09-10T18:49:41.171256+0000","mapping_epoch":86,"log_start":"111'5900","ondisk_log_start":"111'5900","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":4434,"log_dups_size":0,"ondisk_log_size":4434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":441581568,"num_objects":6738,"num_object_clones":0,"num_object_copies":20214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10334,"num_write_kb":546304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"63'15","reported_seq":106,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.164646+0000","last_change":"2024-09-10T18:46:51.114543+0000","last_active":"2024-09-10T18:49:37.164646+0000","last_peered":"2024-09-10T18:49:37.164646+0000","last_clean":"2024-09-10T18:49:37.164646+0000","last_became_active":"2024-09-10T18:45:20.419804+0000","last_became_peered":"2024-09-10T18:45:20.419804+0000","last_unstale":"2024-09-10T18:49:37.164646+0000","last_undegraded":"2024-09-10T18:49:37.164646+0000","last_fullsized":"2024-09-10T18:49:37.164646+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:46:51.114455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:51.114455+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022557979999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":45,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.175251+0000","last_change":"2024-09-10T18:47:36.420713+0000","last_active":"2024-09-10T18:49:37.175251+0000","last_peered":"2024-09-10T18:49:37.175251+0000","last_clean":"2024-09-10T18:49:37.175251+0000","last_became_active":"2024-09-10T18:47:36.420475+0000","last_became_peered":"2024-09-10T18:47:36.420475+0000","last_unstale":"2024-09-10T18:49:37.175251+0000","last_undegraded":"2024-09-10T18:49:37.175251+0000","last_fullsized":"2024-09-10T18:49:37.175251+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:20.774289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:17.819909+0000","last_clean_scrub_stamp":"2024-09-10T18:47:20.774289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"83'9","reported_seq":154,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.167867+0000","last_change":"2024-09-10T18:48:33.683570+0000","last_active":"2024-09-10T18:49:37.167867+0000","last_peered":"2024-09-10T18:49:37.167867+0000","last_clean":"2024-09-10T18:49:37.167867+0000","last_became_active":"2024-09-10T18:45:22.073383+0000","last_became_peered":"2024-09-10T18:45:22.073383+0000","last_unstale":"2024-09-10T18:49:37.167867+0000","last_undegraded":"2024-09-10T18:49:37.167867+0000","last_fullsized":"2024-09-10T18:49:37.167867+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'9","last_scrub_stamp":"2024-09-10T18:48:33.683477+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-09-10T18:47:02.582229+0000","last_clean_scrub_stamp":"2024-09-10T18:48:33.683477+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:57.226708+0000","scrub_duration":0.0026989119999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":87,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.164860+0000","last_change":"2024-09-10T18:49:24.053252+0000","last_active":"2024-09-10T18:49:37.164860+0000","last_peered":"2024-09-10T18:49:37.164860+0000","last_clean":"2024-09-10T18:49:37.164860+0000","last_became_active":"2024-09-10T18:45:24.265256+0000","last_became_peered":"2024-09-10T18:45:24.265256+0000","last_unstale":"2024-09-10T18:49:37.164860+0000","last_undegraded":"2024-09-10T18:49:37.164860+0000","last_fullsized":"2024-09-10T18:49:37.164860+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:24.053158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_clean_scrub_stamp":"2024-09-10T18:49:24.053158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:27.258647+0000","scrub_duration":0.0012635400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.a","version":"121'10462","reported_seq":10515,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T18:49:41.194574+0000","last_change":"2024-09-10T18:47:36.423957+0000","last_active":"2024-09-10T18:49:41.194574+0000","last_peered":"2024-09-10T18:49:41.194574+0000","last_clean":"2024-09-10T18:49:41.194574+0000","last_became_active":"2024-09-10T18:47:36.422868+0000","last_became_peered":"2024-09-10T18:47:36.422868+0000","last_unstale":"2024-09-10T18:49:41.194574+0000","last_undegraded":"2024-09-10T18:49:41.194574+0000","last_fullsized":"2024-09-10T18:49:41.194574+0000","mapping_epoch":88,"log_start":"111'6200","ondisk_log_start":"111'6200","created":86,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":4262,"log_dups_size":0,"ondisk_log_size":4262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":450756608,"num_objects":6878,"num_object_clones":0,"num_object_copies":20634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10462,"num_write_kb":554880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"63'12","reported_seq":114,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.162718+0000","last_change":"2024-09-10T18:47:35.928110+0000","last_active":"2024-09-10T18:49:37.162718+0000","last_peered":"2024-09-10T18:49:37.162718+0000","last_clean":"2024-09-10T18:49:37.162718+0000","last_became_active":"2024-09-10T18:45:20.056869+0000","last_became_peered":"2024-09-10T18:45:20.056869+0000","last_unstale":"2024-09-10T18:49:37.162718+0000","last_undegraded":"2024-09-10T18:49:37.162718+0000","last_fullsized":"2024-09-10T18:49:37.162718+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:47:35.928040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:35.928040+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021332539999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"56'2","reported_seq":110,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.163515+0000","last_change":"2024-09-10T18:49:20.403122+0000","last_active":"2024-09-10T18:49:37.163515+0000","last_peered":"2024-09-10T18:49:37.163515+0000","last_clean":"2024-09-10T18:49:37.163515+0000","last_became_active":"2024-09-10T18:45:17.963286+0000","last_became_peered":"2024-09-10T18:45:17.963286+0000","last_unstale":"2024-09-10T18:49:37.163515+0000","last_undegraded":"2024-09-10T18:49:37.163515+0000","last_fullsized":"2024-09-10T18:49:37.163515+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-10T18:49:20.403067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:20.403067+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:40.186789+0000","scrub_duration":0.00336061,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":55,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.166942+0000","last_change":"2024-09-10T18:48:15.763428+0000","last_active":"2024-09-10T18:49:37.166942+0000","last_peered":"2024-09-10T18:49:37.166942+0000","last_clean":"2024-09-10T18:49:37.166942+0000","last_became_active":"2024-09-10T18:47:36.421151+0000","last_became_peered":"2024-09-10T18:47:36.421151+0000","last_unstale":"2024-09-10T18:49:37.166942+0000","last_undegraded":"2024-09-10T18:49:37.166942+0000","last_fullsized":"2024-09-10T18:49:37.166942+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:15.763348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:15.763348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046035929999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":78,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.174061+0000","last_change":"2024-09-10T18:47:18.797303+0000","last_active":"2024-09-10T18:49:37.174061+0000","last_peered":"2024-09-10T18:49:37.174061+0000","last_clean":"2024-09-10T18:49:37.174061+0000","last_became_active":"2024-09-10T18:45:24.077815+0000","last_became_peered":"2024-09-10T18:45:24.077815+0000","last_unstale":"2024-09-10T18:49:37.174061+0000","last_undegraded":"2024-09-10T18:49:37.174061+0000","last_fullsized":"2024-09-10T18:49:37.174061+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:18.797245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:18.797245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015800090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"76'13","reported_seq":112,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.166135+0000","last_change":"2024-09-10T18:48:10.773290+0000","last_active":"2024-09-10T18:49:37.166135+0000","last_peered":"2024-09-10T18:49:37.166135+0000","last_clean":"2024-09-10T18:49:37.166135+0000","last_became_active":"2024-09-10T18:45:20.420421+0000","last_became_peered":"2024-09-10T18:45:20.420421+0000","last_unstale":"2024-09-10T18:49:37.166135+0000","last_undegraded":"2024-09-10T18:49:37.166135+0000","last_fullsized":"2024-09-10T18:49:37.166135+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'13","last_scrub_stamp":"2024-09-10T18:48:10.773207+0000","last_deep_scrub":"76'13","last_deep_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_clean_scrub_stamp":"2024-09-10T18:48:10.773207+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.008348678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":55,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.163022+0000","last_change":"2024-09-10T18:48:14.699173+0000","last_active":"2024-09-10T18:49:37.163022+0000","last_peered":"2024-09-10T18:49:37.163022+0000","last_clean":"2024-09-10T18:49:37.163022+0000","last_became_active":"2024-09-10T18:47:36.409299+0000","last_became_peered":"2024-09-10T18:47:36.409299+0000","last_unstale":"2024-09-10T18:49:37.163022+0000","last_undegraded":"2024-09-10T18:49:37.163022+0000","last_fullsized":"2024-09-10T18:49:37.163022+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:14.695946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:14.695946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007154171,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":90,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.174559+0000","last_change":"2024-09-10T18:48:28.442445+0000","last_active":"2024-09-10T18:49:37.174559+0000","last_peered":"2024-09-10T18:49:37.174559+0000","last_clean":"2024-09-10T18:49:37.174559+0000","last_became_active":"2024-09-10T18:45:22.069040+0000","last_became_peered":"2024-09-10T18:45:22.069040+0000","last_unstale":"2024-09-10T18:49:37.174559+0000","last_undegraded":"2024-09-10T18:49:37.174559+0000","last_fullsized":"2024-09-10T18:49:37.174559+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:28.442370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:28.442370+0000","last_clean_scrub_stamp":"2024-09-10T18:48:28.442370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:50.109389+0000","scrub_duration":0.0098412350000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"63'1","reported_seq":60,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.162903+0000","last_change":"2024-09-10T18:49:19.409710+0000","last_active":"2024-09-10T18:49:37.162903+0000","last_peered":"2024-09-10T18:49:37.162903+0000","last_clean":"2024-09-10T18:49:37.162903+0000","last_became_active":"2024-09-10T18:47:36.406936+0000","last_became_peered":"2024-09-10T18:47:36.406936+0000","last_unstale":"2024-09-10T18:49:37.162903+0000","last_undegraded":"2024-09-10T18:49:37.162903+0000","last_fullsized":"2024-09-10T18:49:37.162903+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-10T18:46:56.721716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:56.721716+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":93,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.168831+0000","last_change":"2024-09-10T18:48:24.571499+0000","last_active":"2024-09-10T18:49:37.168831+0000","last_peered":"2024-09-10T18:49:37.168831+0000","last_clean":"2024-09-10T18:49:37.168831+0000","last_became_active":"2024-09-10T18:45:17.964705+0000","last_became_peered":"2024-09-10T18:45:17.964705+0000","last_unstale":"2024-09-10T18:49:37.168831+0000","last_undegraded":"2024-09-10T18:49:37.168831+0000","last_fullsized":"2024-09-10T18:49:37.168831+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:24.571419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:24.571419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:51.133227+0000","scrub_duration":0.005895399,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'6","reported_seq":101,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.174692+0000","last_change":"2024-09-10T18:49:18.363292+0000","last_active":"2024-09-10T18:49:37.174692+0000","last_peered":"2024-09-10T18:49:37.174692+0000","last_clean":"2024-09-10T18:49:37.174692+0000","last_became_active":"2024-09-10T18:45:20.421746+0000","last_became_peered":"2024-09-10T18:45:20.421746+0000","last_unstale":"2024-09-10T18:49:37.174692+0000","last_undegraded":"2024-09-10T18:49:37.174692+0000","last_fullsized":"2024-09-10T18:49:37.174692+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'6","last_scrub_stamp":"2024-09-10T18:49:18.363203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:49:18.363203+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:32.230123+0000","scrub_duration":0.0093609290000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":89,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.165522+0000","last_change":"2024-09-10T18:49:12.996132+0000","last_active":"2024-09-10T18:49:37.165522+0000","last_peered":"2024-09-10T18:49:37.165522+0000","last_clean":"2024-09-10T18:49:37.165522+0000","last_became_active":"2024-09-10T18:45:22.069653+0000","last_became_peered":"2024-09-10T18:45:22.069653+0000","last_unstale":"2024-09-10T18:49:37.165522+0000","last_undegraded":"2024-09-10T18:49:37.165522+0000","last_fullsized":"2024-09-10T18:49:37.165522+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:12.996035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:49:12.996035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:37.029555+0000","scrub_duration":0.0049239360000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":88,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.170597+0000","last_change":"2024-09-10T18:48:32.654862+0000","last_active":"2024-09-10T18:49:37.170597+0000","last_peered":"2024-09-10T18:49:37.170597+0000","last_clean":"2024-09-10T18:49:37.170597+0000","last_became_active":"2024-09-10T18:45:24.264611+0000","last_became_peered":"2024-09-10T18:45:24.264611+0000","last_unstale":"2024-09-10T18:49:37.170597+0000","last_undegraded":"2024-09-10T18:49:37.170597+0000","last_fullsized":"2024-09-10T18:49:37.170597+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:32.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:32.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:01.743394+0000","scrub_duration":0.0049870009999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"63'10","reported_seq":151,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.175050+0000","last_change":"2024-09-10T18:49:01.556782+0000","last_active":"2024-09-10T18:49:37.175050+0000","last_peered":"2024-09-10T18:49:37.175050+0000","last_clean":"2024-09-10T18:49:37.175050+0000","last_became_active":"2024-09-10T18:47:36.426260+0000","last_became_peered":"2024-09-10T18:47:36.426260+0000","last_unstale":"2024-09-10T18:49:37.175050+0000","last_undegraded":"2024-09-10T18:49:37.175050+0000","last_fullsized":"2024-09-10T18:49:37.175050+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:49:01.556694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:49:01.556694+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:23.957988+0000","scrub_duration":0.0053223699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":93,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.165009+0000","last_change":"2024-09-10T18:49:06.079691+0000","last_active":"2024-09-10T18:49:37.165009+0000","last_peered":"2024-09-10T18:49:37.165009+0000","last_clean":"2024-09-10T18:49:37.165009+0000","last_became_active":"2024-09-10T18:45:17.964870+0000","last_became_peered":"2024-09-10T18:45:17.964870+0000","last_unstale":"2024-09-10T18:49:37.165009+0000","last_undegraded":"2024-09-10T18:49:37.165009+0000","last_fullsized":"2024-09-10T18:49:37.165009+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:06.079456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:06.079456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:12.151525+0000","scrub_duration":0.012719522,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"63'8","reported_seq":149,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.165619+0000","last_change":"2024-09-10T18:49:02.038795+0000","last_active":"2024-09-10T18:49:37.165619+0000","last_peered":"2024-09-10T18:49:37.165619+0000","last_clean":"2024-09-10T18:49:37.165619+0000","last_became_active":"2024-09-10T18:45:22.062940+0000","last_became_peered":"2024-09-10T18:45:22.062940+0000","last_unstale":"2024-09-10T18:49:37.165619+0000","last_undegraded":"2024-09-10T18:49:37.165619+0000","last_fullsized":"2024-09-10T18:49:37.165619+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:49:02.038709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:49:02.038709+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:15.977699+0000","scrub_duration":0.007468054,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":88,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.165799+0000","last_change":"2024-09-10T18:49:03.552056+0000","last_active":"2024-09-10T18:49:37.165799+0000","last_peered":"2024-09-10T18:49:37.165799+0000","last_clean":"2024-09-10T18:49:37.165799+0000","last_became_active":"2024-09-10T18:45:24.073736+0000","last_became_peered":"2024-09-10T18:45:24.073736+0000","last_unstale":"2024-09-10T18:49:37.165799+0000","last_undegraded":"2024-09-10T18:49:37.165799+0000","last_fullsized":"2024-09-10T18:49:37.165799+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:03.551966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:03.551966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:11.246649+0000","scrub_duration":0.015480603000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"63'9","reported_seq":107,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.165690+0000","last_change":"2024-09-10T18:48:19.076178+0000","last_active":"2024-09-10T18:49:37.165690+0000","last_peered":"2024-09-10T18:49:37.165690+0000","last_clean":"2024-09-10T18:49:37.165690+0000","last_became_active":"2024-09-10T18:45:20.421124+0000","last_became_peered":"2024-09-10T18:45:20.421124+0000","last_unstale":"2024-09-10T18:49:37.165690+0000","last_undegraded":"2024-09-10T18:49:37.165690+0000","last_fullsized":"2024-09-10T18:49:37.165690+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:48:19.076099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:19.076099+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002691981,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":94,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.166372+0000","last_change":"2024-09-10T18:48:33.663516+0000","last_active":"2024-09-10T18:49:37.166372+0000","last_peered":"2024-09-10T18:49:37.166372+0000","last_clean":"2024-09-10T18:49:37.166372+0000","last_became_active":"2024-09-10T18:45:17.960106+0000","last_became_peered":"2024-09-10T18:45:17.960106+0000","last_unstale":"2024-09-10T18:49:37.166372+0000","last_undegraded":"2024-09-10T18:49:37.166372+0000","last_fullsized":"2024-09-10T18:49:37.166372+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:33.663415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:33.663415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002167352,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"89'10","reported_seq":156,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T18:49:40.187788+0000","last_change":"2024-09-10T18:47:36.434102+0000","last_active":"2024-09-10T18:49:40.187788+0000","last_peered":"2024-09-10T18:49:40.187788+0000","last_clean":"2024-09-10T18:49:40.187788+0000","last_became_active":"2024-09-10T18:47:36.424575+0000","last_became_peered":"2024-09-10T18:47:36.424575+0000","last_unstale":"2024-09-10T18:49:40.187788+0000","last_undegraded":"2024-09-10T18:49:40.187788+0000","last_fullsized":"2024-09-10T18:49:40.187788+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"81'9","last_scrub_stamp":"2024-09-10T18:46:54.800790+0000","last_deep_scrub":"81'9","last_deep_scrub_stamp":"2024-09-10T18:46:54.800790+0000","last_clean_scrub_stamp":"2024-09-10T18:46:54.800790+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034208709999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":97,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.163373+0000","last_change":"2024-09-10T18:48:34.045421+0000","last_active":"2024-09-10T18:49:37.163373+0000","last_peered":"2024-09-10T18:49:37.163373+0000","last_clean":"2024-09-10T18:49:37.163373+0000","last_became_active":"2024-09-10T18:47:36.414047+0000","last_became_peered":"2024-09-10T18:47:36.414047+0000","last_unstale":"2024-09-10T18:49:37.163373+0000","last_undegraded":"2024-09-10T18:49:37.163373+0000","last_fullsized":"2024-09-10T18:49:37.163373+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:34.045311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_clean_scrub_stamp":"2024-09-10T18:48:34.045311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:49.035924+0000","scrub_duration":0.0048840200000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"63'9","reported_seq":128,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.163534+0000","last_change":"2024-09-10T18:49:05.359047+0000","last_active":"2024-09-10T18:49:37.163534+0000","last_peered":"2024-09-10T18:49:37.163534+0000","last_clean":"2024-09-10T18:49:37.163534+0000","last_became_active":"2024-09-10T18:47:36.416484+0000","last_became_peered":"2024-09-10T18:47:36.416484+0000","last_unstale":"2024-09-10T18:49:37.163534+0000","last_undegraded":"2024-09-10T18:49:37.163534+0000","last_fullsized":"2024-09-10T18:49:37.163534+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:49:05.358887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:49:05.358887+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:26.279687+0000","scrub_duration":0.013783619,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":103,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.169591+0000","last_change":"2024-09-10T18:49:04.430421+0000","last_active":"2024-09-10T18:49:37.169591+0000","last_peered":"2024-09-10T18:49:37.169591+0000","last_clean":"2024-09-10T18:49:37.169591+0000","last_became_active":"2024-09-10T18:45:17.962009+0000","last_became_peered":"2024-09-10T18:45:17.962009+0000","last_unstale":"2024-09-10T18:49:37.169591+0000","last_undegraded":"2024-09-10T18:49:37.169591+0000","last_fullsized":"2024-09-10T18:49:37.169591+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:04.430344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:04.430344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:13.393531+0000","scrub_duration":0.0025461400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":80,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.174139+0000","last_change":"2024-09-10T18:47:14.869132+0000","last_active":"2024-09-10T18:49:37.174139+0000","last_peered":"2024-09-10T18:49:37.174139+0000","last_clean":"2024-09-10T18:49:37.174139+0000","last_became_active":"2024-09-10T18:45:22.072877+0000","last_became_peered":"2024-09-10T18:45:22.072877+0000","last_unstale":"2024-09-10T18:49:37.174139+0000","last_undegraded":"2024-09-10T18:49:37.174139+0000","last_fullsized":"2024-09-10T18:49:37.174139+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:14.869076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:47:14.869076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00131624,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":88,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.166524+0000","last_change":"2024-09-10T18:48:32.647084+0000","last_active":"2024-09-10T18:49:37.166524+0000","last_peered":"2024-09-10T18:49:37.166524+0000","last_clean":"2024-09-10T18:49:37.166524+0000","last_became_active":"2024-09-10T18:45:24.081050+0000","last_became_peered":"2024-09-10T18:45:24.081050+0000","last_unstale":"2024-09-10T18:49:37.166524+0000","last_undegraded":"2024-09-10T18:49:37.166524+0000","last_fullsized":"2024-09-10T18:49:37.166524+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:32.646988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:32.646988+0000","last_clean_scrub_stamp":"2024-09-10T18:48:32.646988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038989210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"63'11","reported_seq":100,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.162937+0000","last_change":"2024-09-10T18:47:14.034427+0000","last_active":"2024-09-10T18:49:37.162937+0000","last_peered":"2024-09-10T18:49:37.162937+0000","last_clean":"2024-09-10T18:49:37.162937+0000","last_became_active":"2024-09-10T18:45:20.056885+0000","last_became_peered":"2024-09-10T18:45:20.056885+0000","last_unstale":"2024-09-10T18:49:37.162937+0000","last_undegraded":"2024-09-10T18:49:37.162937+0000","last_fullsized":"2024-09-10T18:49:37.162937+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:47:14.034361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:14.034361+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018564,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":94,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.162780+0000","last_change":"2024-09-10T18:48:32.517573+0000","last_active":"2024-09-10T18:49:37.162780+0000","last_peered":"2024-09-10T18:49:37.162780+0000","last_clean":"2024-09-10T18:49:37.162780+0000","last_became_active":"2024-09-10T18:45:17.963542+0000","last_became_peered":"2024-09-10T18:45:17.963542+0000","last_unstale":"2024-09-10T18:49:37.162780+0000","last_undegraded":"2024-09-10T18:49:37.162780+0000","last_fullsized":"2024-09-10T18:49:37.162780+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:32.517513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:32.517513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:48.139582+0000","scrub_duration":0.013400413999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":90,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.163393+0000","last_change":"2024-09-10T18:48:09.725726+0000","last_active":"2024-09-10T18:49:37.163393+0000","last_peered":"2024-09-10T18:49:37.163393+0000","last_clean":"2024-09-10T18:49:37.163393+0000","last_became_active":"2024-09-10T18:45:22.070065+0000","last_became_peered":"2024-09-10T18:45:22.070065+0000","last_unstale":"2024-09-10T18:49:37.163393+0000","last_undegraded":"2024-09-10T18:49:37.163393+0000","last_fullsized":"2024-09-10T18:49:37.163393+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:09.725488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:09.725488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027892020000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":78,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.170795+0000","last_change":"2024-09-10T18:47:05.789893+0000","last_active":"2024-09-10T18:49:37.170795+0000","last_peered":"2024-09-10T18:49:37.170795+0000","last_clean":"2024-09-10T18:49:37.170795+0000","last_became_active":"2024-09-10T18:45:24.072224+0000","last_became_peered":"2024-09-10T18:45:24.072224+0000","last_unstale":"2024-09-10T18:49:37.170795+0000","last_undegraded":"2024-09-10T18:49:37.170795+0000","last_fullsized":"2024-09-10T18:49:37.170795+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:05.789830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:05.789830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001271938,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"63'4","reported_seq":98,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.174022+0000","last_change":"2024-09-10T18:48:30.455636+0000","last_active":"2024-09-10T18:49:37.174022+0000","last_peered":"2024-09-10T18:49:37.174022+0000","last_clean":"2024-09-10T18:49:37.174022+0000","last_became_active":"2024-09-10T18:45:20.059205+0000","last_became_peered":"2024-09-10T18:45:20.059205+0000","last_unstale":"2024-09-10T18:49:37.174022+0000","last_undegraded":"2024-09-10T18:49:37.174022+0000","last_fullsized":"2024-09-10T18:49:37.174022+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4","last_scrub_stamp":"2024-09-10T18:48:30.455433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:30.455433+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0078514339999999991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":93,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.165532+0000","last_change":"2024-09-10T18:47:36.416170+0000","last_active":"2024-09-10T18:49:37.165532+0000","last_peered":"2024-09-10T18:49:37.165532+0000","last_clean":"2024-09-10T18:49:37.165532+0000","last_became_active":"2024-09-10T18:47:36.415866+0000","last_became_peered":"2024-09-10T18:47:36.415866+0000","last_unstale":"2024-09-10T18:49:37.165532+0000","last_undegraded":"2024-09-10T18:49:37.165532+0000","last_fullsized":"2024-09-10T18:49:37.165532+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:54.082761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:46:54.082761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001210458,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":89,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.168671+0000","last_change":"2024-09-10T18:48:27.535769+0000","last_active":"2024-09-10T18:49:37.168671+0000","last_peered":"2024-09-10T18:49:37.168671+0000","last_clean":"2024-09-10T18:49:37.168671+0000","last_became_active":"2024-09-10T18:45:22.072377+0000","last_became_peered":"2024-09-10T18:45:22.072377+0000","last_unstale":"2024-09-10T18:49:37.168671+0000","last_undegraded":"2024-09-10T18:49:37.168671+0000","last_fullsized":"2024-09-10T18:49:37.168671+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:27.535676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_clean_scrub_stamp":"2024-09-10T18:48:27.535676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002291132,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":98,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.167690+0000","last_change":"2024-09-10T18:48:28.711019+0000","last_active":"2024-09-10T18:49:37.167690+0000","last_peered":"2024-09-10T18:49:37.167690+0000","last_clean":"2024-09-10T18:49:37.167690+0000","last_became_active":"2024-09-10T18:47:36.416187+0000","last_became_peered":"2024-09-10T18:47:36.416187+0000","last_unstale":"2024-09-10T18:49:37.167690+0000","last_undegraded":"2024-09-10T18:49:37.167690+0000","last_fullsized":"2024-09-10T18:49:37.167690+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:28.710918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:28.710918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0068877160000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"79'16","reported_seq":116,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.170989+0000","last_change":"2024-09-10T18:49:03.566236+0000","last_active":"2024-09-10T18:49:37.170989+0000","last_peered":"2024-09-10T18:49:37.170989+0000","last_clean":"2024-09-10T18:49:37.170989+0000","last_became_active":"2024-09-10T18:45:20.421479+0000","last_became_peered":"2024-09-10T18:45:20.421479+0000","last_unstale":"2024-09-10T18:49:37.170989+0000","last_undegraded":"2024-09-10T18:49:37.170989+0000","last_fullsized":"2024-09-10T18:49:37.170989+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"79'16","last_scrub_stamp":"2024-09-10T18:49:03.566152+0000","last_deep_scrub":"79'16","last_deep_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_clean_scrub_stamp":"2024-09-10T18:49:03.566152+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:21.649879+0000","scrub_duration":0.0032760770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"56'1","reported_seq":114,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.169870+0000","last_change":"2024-09-10T18:48:20.603918+0000","last_active":"2024-09-10T18:49:37.169870+0000","last_peered":"2024-09-10T18:49:37.169870+0000","last_clean":"2024-09-10T18:49:37.169870+0000","last_became_active":"2024-09-10T18:47:36.421985+0000","last_became_peered":"2024-09-10T18:47:36.421985+0000","last_unstale":"2024-09-10T18:49:37.169870+0000","last_undegraded":"2024-09-10T18:49:37.169870+0000","last_fullsized":"2024-09-10T18:49:37.169870+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:48:20.603805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:20.603805+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:49.866219+0000","scrub_duration":0.0039707509999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":44,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.171434+0000","last_change":"2024-09-10T18:48:09.323873+0000","last_active":"2024-09-10T18:49:37.171434+0000","last_peered":"2024-09-10T18:49:37.171434+0000","last_clean":"2024-09-10T18:49:37.171434+0000","last_became_active":"2024-09-10T18:47:57.107149+0000","last_became_peered":"2024-09-10T18:47:57.107149+0000","last_unstale":"2024-09-10T18:49:37.171434+0000","last_undegraded":"2024-09-10T18:49:37.171434+0000","last_fullsized":"2024-09-10T18:49:37.171434+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:09.323794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:09.323794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056521669999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":88,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.173860+0000","last_change":"2024-09-10T18:48:16.410239+0000","last_active":"2024-09-10T18:49:37.173860+0000","last_peered":"2024-09-10T18:49:37.173860+0000","last_clean":"2024-09-10T18:49:37.173860+0000","last_became_active":"2024-09-10T18:45:24.264818+0000","last_became_peered":"2024-09-10T18:45:24.264818+0000","last_unstale":"2024-09-10T18:49:37.173860+0000","last_undegraded":"2024-09-10T18:49:37.173860+0000","last_fullsized":"2024-09-10T18:49:37.173860+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:16.410071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:16.410071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0061840250000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"78'2","reported_seq":104,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.170790+0000","last_change":"2024-09-10T18:48:10.610714+0000","last_active":"2024-09-10T18:49:37.170790+0000","last_peered":"2024-09-10T18:49:37.170790+0000","last_clean":"2024-09-10T18:49:37.170790+0000","last_became_active":"2024-09-10T18:47:36.418702+0000","last_became_peered":"2024-09-10T18:47:36.418702+0000","last_unstale":"2024-09-10T18:49:37.170790+0000","last_undegraded":"2024-09-10T18:49:37.170790+0000","last_fullsized":"2024-09-10T18:49:37.170790+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'2","last_scrub_stamp":"2024-09-10T18:48:10.610652+0000","last_deep_scrub":"78'2","last_deep_scrub_stamp":"2024-09-10T18:46:37.581130+0000","last_clean_scrub_stamp":"2024-09-10T18:48:10.610652+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002045275,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":93,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.164711+0000","last_change":"2024-09-10T18:49:07.088445+0000","last_active":"2024-09-10T18:49:37.164711+0000","last_peered":"2024-09-10T18:49:37.164711+0000","last_clean":"2024-09-10T18:49:37.164711+0000","last_became_active":"2024-09-10T18:45:17.962108+0000","last_became_peered":"2024-09-10T18:45:17.962108+0000","last_unstale":"2024-09-10T18:49:37.164711+0000","last_undegraded":"2024-09-10T18:49:37.164711+0000","last_fullsized":"2024-09-10T18:49:37.164711+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:07.088371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:07.088371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:26.877956+0000","scrub_duration":0.0013360100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"63'10","reported_seq":107,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.166056+0000","last_change":"2024-09-10T18:48:23.844841+0000","last_active":"2024-09-10T18:49:37.166056+0000","last_peered":"2024-09-10T18:49:37.166056+0000","last_clean":"2024-09-10T18:49:37.166056+0000","last_became_active":"2024-09-10T18:45:20.421094+0000","last_became_peered":"2024-09-10T18:45:20.421094+0000","last_unstale":"2024-09-10T18:49:37.166056+0000","last_undegraded":"2024-09-10T18:49:37.166056+0000","last_fullsized":"2024-09-10T18:49:37.166056+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:48:23.844741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:23.844741+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:49.178164+0000","scrub_duration":0.0025648530000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":152,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.169787+0000","last_change":"2024-09-10T18:48:19.605745+0000","last_active":"2024-09-10T18:49:37.169787+0000","last_peered":"2024-09-10T18:49:37.169787+0000","last_clean":"2024-09-10T18:49:37.169787+0000","last_became_active":"2024-09-10T18:45:22.073281+0000","last_became_peered":"2024-09-10T18:45:22.073281+0000","last_unstale":"2024-09-10T18:49:37.169787+0000","last_undegraded":"2024-09-10T18:49:37.169787+0000","last_fullsized":"2024-09-10T18:49:37.169787+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:48:19.605646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:19.605646+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001752665,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":88,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.162318+0000","last_change":"2024-09-10T18:49:11.422481+0000","last_active":"2024-09-10T18:49:37.162318+0000","last_peered":"2024-09-10T18:49:37.162318+0000","last_clean":"2024-09-10T18:49:37.162318+0000","last_became_active":"2024-09-10T18:45:24.082318+0000","last_became_peered":"2024-09-10T18:45:24.082318+0000","last_unstale":"2024-09-10T18:49:37.162318+0000","last_undegraded":"2024-09-10T18:49:37.162318+0000","last_fullsized":"2024-09-10T18:49:37.162318+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:11.422279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:03.475299+0000","last_clean_scrub_stamp":"2024-09-10T18:49:11.422279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:39.824521+0000","scrub_duration":0.0030495840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":94,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.171334+0000","last_change":"2024-09-10T18:49:07.526539+0000","last_active":"2024-09-10T18:49:37.171334+0000","last_peered":"2024-09-10T18:49:37.171334+0000","last_clean":"2024-09-10T18:49:37.171334+0000","last_became_active":"2024-09-10T18:45:17.965707+0000","last_became_peered":"2024-09-10T18:45:17.965707+0000","last_unstale":"2024-09-10T18:49:37.171334+0000","last_undegraded":"2024-09-10T18:49:37.171334+0000","last_fullsized":"2024-09-10T18:49:37.171334+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:07.526460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:07.526460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:10.144795+0000","scrub_duration":0.010955793,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"63'11","reported_seq":110,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.169824+0000","last_change":"2024-09-10T18:48:23.546688+0000","last_active":"2024-09-10T18:49:37.169824+0000","last_peered":"2024-09-10T18:49:37.169824+0000","last_clean":"2024-09-10T18:49:37.169824+0000","last_became_active":"2024-09-10T18:45:20.058717+0000","last_became_peered":"2024-09-10T18:45:20.058717+0000","last_unstale":"2024-09-10T18:49:37.169824+0000","last_undegraded":"2024-09-10T18:49:37.169824+0000","last_fullsized":"2024-09-10T18:49:37.169824+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:48:23.546598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:23.546598+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0057630149999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":100,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.165980+0000","last_change":"2024-09-10T18:48:58.543128+0000","last_active":"2024-09-10T18:49:37.165980+0000","last_peered":"2024-09-10T18:49:37.165980+0000","last_clean":"2024-09-10T18:49:37.165980+0000","last_became_active":"2024-09-10T18:47:36.418767+0000","last_became_peered":"2024-09-10T18:47:36.418767+0000","last_unstale":"2024-09-10T18:49:37.165980+0000","last_undegraded":"2024-09-10T18:49:37.165980+0000","last_fullsized":"2024-09-10T18:49:37.165980+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:58.542812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:58.542812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:01.428535+0000","scrub_duration":0.010435696,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":87,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.163173+0000","last_change":"2024-09-10T18:49:06.423032+0000","last_active":"2024-09-10T18:49:37.163173+0000","last_peered":"2024-09-10T18:49:37.163173+0000","last_clean":"2024-09-10T18:49:37.163173+0000","last_became_active":"2024-09-10T18:45:24.267251+0000","last_became_peered":"2024-09-10T18:45:24.267251+0000","last_unstale":"2024-09-10T18:49:37.163173+0000","last_undegraded":"2024-09-10T18:49:37.163173+0000","last_fullsized":"2024-09-10T18:49:37.163173+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:06.422952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:06.422952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:33.673505+0000","scrub_duration":0.0017661879999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"56'1","reported_seq":102,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.169173+0000","last_change":"2024-09-10T18:48:31.488364+0000","last_active":"2024-09-10T18:49:37.169173+0000","last_peered":"2024-09-10T18:49:37.169173+0000","last_clean":"2024-09-10T18:49:37.169173+0000","last_became_active":"2024-09-10T18:45:17.966286+0000","last_became_peered":"2024-09-10T18:45:17.966286+0000","last_unstale":"2024-09-10T18:49:37.169173+0000","last_undegraded":"2024-09-10T18:49:37.169173+0000","last_fullsized":"2024-09-10T18:49:37.169173+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:48:31.488272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:31.488272+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0028000949999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"63'15","reported_seq":98,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.161854+0000","last_change":"2024-09-10T18:47:36.446154+0000","last_active":"2024-09-10T18:49:37.161854+0000","last_peered":"2024-09-10T18:49:37.161854+0000","last_clean":"2024-09-10T18:49:37.161854+0000","last_became_active":"2024-09-10T18:47:36.424373+0000","last_became_peered":"2024-09-10T18:47:36.424373+0000","last_unstale":"2024-09-10T18:49:37.161854+0000","last_undegraded":"2024-09-10T18:49:37.161854+0000","last_fullsized":"2024-09-10T18:49:37.161854+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:47:34.610915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:34.610915+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":80,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.162538+0000","last_change":"2024-09-10T18:47:09.520829+0000","last_active":"2024-09-10T18:49:37.162538+0000","last_peered":"2024-09-10T18:49:37.162538+0000","last_clean":"2024-09-10T18:49:37.162538+0000","last_became_active":"2024-09-10T18:45:22.059794+0000","last_became_peered":"2024-09-10T18:45:22.059794+0000","last_unstale":"2024-09-10T18:49:37.162538+0000","last_undegraded":"2024-09-10T18:49:37.162538+0000","last_fullsized":"2024-09-10T18:49:37.162538+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:09.520768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:47:09.520768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016284120000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":78,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.174439+0000","last_change":"2024-09-10T18:46:57.811148+0000","last_active":"2024-09-10T18:49:37.174439+0000","last_peered":"2024-09-10T18:49:37.174439+0000","last_clean":"2024-09-10T18:49:37.174439+0000","last_became_active":"2024-09-10T18:45:24.264427+0000","last_became_peered":"2024-09-10T18:45:24.264427+0000","last_unstale":"2024-09-10T18:49:37.174439+0000","last_undegraded":"2024-09-10T18:49:37.174439+0000","last_fullsized":"2024-09-10T18:49:37.174439+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:57.811074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:57.811074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013689419999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":94,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.174633+0000","last_change":"2024-09-10T18:49:14.365927+0000","last_active":"2024-09-10T18:49:37.174633+0000","last_peered":"2024-09-10T18:49:37.174633+0000","last_clean":"2024-09-10T18:49:37.174633+0000","last_became_active":"2024-09-10T18:45:17.963257+0000","last_became_peered":"2024-09-10T18:45:17.963257+0000","last_unstale":"2024-09-10T18:49:37.174633+0000","last_undegraded":"2024-09-10T18:49:37.174633+0000","last_fullsized":"2024-09-10T18:49:37.174633+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:14.365492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:14.365492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:29.522576+0000","scrub_duration":0.002395463,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"63'12","reported_seq":115,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.174392+0000","last_change":"2024-09-10T18:49:13.387562+0000","last_active":"2024-09-10T18:49:37.174392+0000","last_peered":"2024-09-10T18:49:37.174392+0000","last_clean":"2024-09-10T18:49:37.174392+0000","last_became_active":"2024-09-10T18:45:20.058391+0000","last_became_peered":"2024-09-10T18:45:20.058391+0000","last_unstale":"2024-09-10T18:49:37.174392+0000","last_undegraded":"2024-09-10T18:49:37.174392+0000","last_fullsized":"2024-09-10T18:49:37.174392+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:49:13.387425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:49:13.387425+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:19.344236+0000","scrub_duration":0.0091962799999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":99,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-10T18:49:37.163055+0000","last_change":"2024-09-10T18:47:36.416505+0000","last_active":"2024-09-10T18:49:37.163055+0000","last_peered":"2024-09-10T18:49:37.163055+0000","last_clean":"2024-09-10T18:49:37.163055+0000","last_became_active":"2024-09-10T18:47:36.416131+0000","last_became_peered":"2024-09-10T18:47:36.416131+0000","last_unstale":"2024-09-10T18:49:37.163055+0000","last_undegraded":"2024-09-10T18:49:37.163055+0000","last_fullsized":"2024-09-10T18:49:37.163055+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:34.958889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:47:34.958889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001401296,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":7026638872,"num_objects":107219,"num_object_clones":0,"num_object_copies":321657,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107219,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":165217,"num_write_kb":8717889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":21081960448,"data_stored":21081948232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76974,"ondisk_log_size":76974,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":697,"num_read_kb":454,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":1238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":428,"ondisk_log_size":428,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":253,"num_read_kb":248,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299449,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2893496,"kb_used_data":2587820,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":90851656,"statfs":{"total":95995035648,"available":93032095744,"internally_reserved":0,"allocated":2649927680,"data_stored":2649542473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:49:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":0.94499999999999995,"15min":0.65300000000000002},"min":{"1min":0.219,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":3.9929999999999999,"5min":31.805,"15min":31.805},"last":0.43099999999999999},{"interface":"front","average":{"1min":1.8640000000000001,"5min":1.081,"15min":0.68400000000000005},"min":{"1min":0.29399999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":5.4119999999999999,"5min":31.212,"15min":31.212},"last":0.29499999999999998}]},{"osd":1,"last update":"Tue Sep 10 18:49:10 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.55200000000000005,"15min":0.502},"min":{"1min":0.30199999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":2.169,"5min":2.169,"15min":2.169},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.0640000000000001,"5min":0.624,"15min":0.55400000000000005},"min":{"1min":0.255,"5min":0.25,"15min":0.25},"max":{"1min":3.036,"5min":3.036,"15min":3.036},"last":2.6760000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:49:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":0.60599999999999998,"15min":0.52600000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":5.1399999999999997,"5min":5.1399999999999997,"15min":5.1399999999999997},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.60699999999999998,"15min":0.54600000000000004},"min":{"1min":0.38200000000000001,"5min":0.31,"15min":0.31},"max":{"1min":3.0209999999999999,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":0.49199999999999999}]},{"osd":3,"last update":"Tue Sep 10 18:49:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":0.95699999999999996,"15min":0.65700000000000003},"min":{"1min":0.26900000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":3.2120000000000002,"5min":31.478000000000002,"15min":31.478000000000002},"last":0.54000000000000004},{"interface":"front","average":{"1min":1.581,"5min":1.1619999999999999,"15min":0.73299999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":5.1020000000000003,"5min":33.731999999999999,"15min":33.731999999999999},"last":0.46500000000000002}]},{"osd":4,"last update":"Tue Sep 10 18:49:10 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.45100000000000001,"15min":0.40699999999999997},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":2.7559999999999998,"5min":2.7559999999999998,"15min":2.7559999999999998},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.438,"15min":0.39600000000000002},"min":{"1min":0.29599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.53200000000000003}]},{"osd":5,"last update":"Tue Sep 10 18:49:10 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.46899999999999997,"15min":0.43099999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":2.7240000000000002,"5min":2.7240000000000002,"15min":2.7240000000000002},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.45000000000000001,"15min":0.42699999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.51300000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:49:10 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.52900000000000003,"15min":0.47499999999999998},"min":{"1min":0.34499999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":2.8199999999999998,"5min":2.8199999999999998,"15min":2.8199999999999998},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.52400000000000002,"15min":0.46300000000000002},"min":{"1min":0.33700000000000002,"5min":0.309,"15min":0.309},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.40500000000000003}]}]},{"osd":6,"up_from":43,"seq":184683593790,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2913796,"kb_used_data":2608120,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":90831356,"statfs":{"total":95995035648,"available":93011308544,"internally_reserved":0,"allocated":2670714880,"data_stored":2670317953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:48:46 2024","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":0.64900000000000002,"15min":0.52100000000000002},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":4.681,"5min":4.681,"15min":4.681},"last":19.029},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.58799999999999997,"15min":0.52300000000000002},"min":{"1min":0.315,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":3.1709999999999998,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":19.151}]},{"osd":1,"last update":"Tue Sep 10 18:48:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.55100000000000005,"15min":0.50900000000000001},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.316},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.56899999999999995,"15min":0.497},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":3.6280000000000001,"5min":3.6280000000000001,"15min":3.6280000000000001},"last":0.38100000000000001}]},{"osd":2,"last update":"Tue Sep 10 18:48:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":0.58699999999999997,"15min":0.51600000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":3.359,"5min":3.359,"15min":3.359},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.52000000000000002,"15min":0.5},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.39500000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:48:46 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.51900000000000002,"15min":0.495},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":0.33600000000000002},{"interface":"front","average":{"1min":1.069,"5min":0.628,"15min":0.54900000000000004},"min":{"1min":0.33600000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":4.5309999999999997,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":0.36299999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:48:46 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.41799999999999998,"15min":0.40899999999999997},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.41899999999999998,"15min":0.39500000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.41099999999999998}]},{"osd":5,"last update":"Tue Sep 10 18:48:46 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.439,"15min":0.42199999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.46300000000000002,"15min":0.435},"min":{"1min":0.29799999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":0.34999999999999998}]},{"osd":7,"last update":"Tue Sep 10 18:49:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.52400000000000002,"15min":0.48699999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.367,"5min":1.556,"15min":1.556},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.53700000000000003,"15min":0.50800000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.3320000000000001,"5min":1.883,"15min":1.883},"last":0.42099999999999999}]}]},{"osd":5,"up_from":37,"seq":158913790018,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2047904,"kb_used_data":1750552,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":91697248,"statfs":{"total":95995035648,"available":93897981952,"internally_reserved":0,"allocated":1792565248,"data_stored":1792192575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:49:23 2024","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":0.66900000000000004,"15min":0.57499999999999996},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":3.9169999999999998,"5min":3.9169999999999998,"15min":3.9169999999999998},"last":1.655},{"interface":"front","average":{"1min":1.163,"5min":0.77300000000000002,"15min":0.58999999999999997},"min":{"1min":0.33600000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":4.3369999999999997,"5min":6.742,"15min":6.742},"last":0.97399999999999998}]},{"osd":1,"last update":"Tue Sep 10 18:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.64600000000000002,"15min":0.59099999999999997},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":3.5150000000000001,"5min":3.5150000000000001,"15min":3.5150000000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.75600000000000001,"15min":0.61299999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":3.5030000000000001,"5min":6.6989999999999998,"15min":6.6989999999999998},"last":0.54200000000000004}]},{"osd":2,"last update":"Tue Sep 10 18:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.55500000000000005,"15min":0.53200000000000003},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":4.0730000000000004,"5min":4.0730000000000004,"15min":4.0730000000000004},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.70099999999999996,"15min":0.59599999999999997},"min":{"1min":0.27900000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":2.2970000000000002,"5min":6.7210000000000001,"15min":6.7210000000000001},"last":0.34200000000000003}]},{"osd":3,"last update":"Tue Sep 10 18:49:23 2024","interfaces":[{"interface":"back","average":{"1min":1.627,"5min":0.84399999999999997,"15min":0.63300000000000001},"min":{"1min":0.29899999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":6.5190000000000001,"5min":6.5190000000000001,"15min":6.5190000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":1.3460000000000001,"5min":0.85699999999999998,"15min":0.65100000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":3.4239999999999999,"5min":5.8099999999999996,"15min":5.8099999999999996},"last":0.59499999999999997}]},{"osd":4,"last update":"Tue Sep 10 18:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.41599999999999998,"15min":0.39400000000000002},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":2.133,"5min":2.133,"15min":2.133},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.40600000000000003,"15min":0.40400000000000003},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":2.153,"5min":2.153,"15min":2.153},"last":0.503}]},{"osd":6,"last update":"Tue Sep 10 18:48:47 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.49099999999999999,"15min":0.439},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.53600000000000003,"15min":0.48399999999999999},"min":{"1min":0.33000000000000002,"5min":0.31,"15min":0.31},"max":{"1min":2.2599999999999998,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":0.85499999999999998}]},{"osd":7,"last update":"Tue Sep 10 18:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.48599999999999999,"15min":0.47999999999999998},"min":{"1min":0.28399999999999997,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":2.1139999999999999,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.495,"5min":0.47299999999999998,"15min":0.47599999999999998},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.373}]}]},{"osd":4,"up_from":30,"seq":128849018952,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2465528,"kb_used_data":2164012,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":91279624,"statfs":{"total":95995035648,"available":93470334976,"internally_reserved":0,"allocated":2215948288,"data_stored":2215562388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:49:05 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.624,"15min":0.55800000000000005},"min":{"1min":0.215,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":7.8730000000000002,"5min":7.8730000000000002,"15min":7.8730000000000002},"last":1.272},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.58799999999999997,"15min":0.55800000000000005},"min":{"1min":0.23999999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":3.3279999999999998,"5min":3.3279999999999998,"15min":3.3279999999999998},"last":1.165}]},{"osd":1,"last update":"Tue Sep 10 18:49:05 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.624,"15min":0.58999999999999997},"min":{"1min":0.27900000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":3.1909999999999998,"5min":3.1909999999999998,"15min":3.1909999999999998},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.63200000000000001,"15min":0.56000000000000005},"min":{"1min":0.28399999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":2.0760000000000001,"5min":2.589,"15min":2.589},"last":0.23899999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:49:05 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.56200000000000006,"15min":0.54600000000000004},"min":{"1min":0.29099999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":2.0800000000000001,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.54900000000000004,"15min":0.55500000000000005},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":2.8109999999999999,"5min":2.8109999999999999,"15min":2.8109999999999999},"last":0.34100000000000003}]},{"osd":3,"last update":"Tue Sep 10 18:49:05 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":0.67100000000000004,"15min":0.57699999999999996},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":3.512,"5min":3.512,"15min":3.512},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.61499999999999999,"15min":0.56699999999999995},"min":{"1min":0.32400000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":3.109,"5min":3.109,"15min":3.109},"last":0.78300000000000003}]},{"osd":5,"last update":"Tue Sep 10 18:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.41999999999999998,"15min":0.37},"min":{"1min":0.25700000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.78700000000000003,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.42199999999999999,"15min":0.376},"min":{"1min":0.247,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.27200000000000002}]},{"osd":6,"last update":"Tue Sep 10 18:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.48499999999999999,"15min":0.442},"min":{"1min":0.30599999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":3.5529999999999999,"5min":3.5529999999999999,"15min":3.5529999999999999},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.48099999999999998,"15min":0.41799999999999998},"min":{"1min":0.33100000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":0.45800000000000002}]},{"osd":7,"last update":"Tue Sep 10 18:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.52400000000000002,"15min":0.50800000000000001},"min":{"1min":0.34899999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":2.8940000000000001,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.47899999999999998,"15min":0.45900000000000002},"min":{"1min":0.30299999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":2.0219999999999998,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":0.38200000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5064388,"kb_used_data":4766840,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":88680764,"statfs":{"total":95995035648,"available":90809102336,"internally_reserved":0,"allocated":4881244160,"data_stored":4880845805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:49:38 2024","interfaces":[{"interface":"back","average":{"1min":4.2969999999999997,"5min":1.4279999999999999,"15min":0.73399999999999999},"min":{"1min":0.35699999999999998,"5min":0.215,"15min":0.215},"max":{"1min":47.601999999999997,"5min":47.601999999999997,"15min":47.601999999999997},"last":0.67600000000000005},{"interface":"front","average":{"1min":4.2130000000000001,"5min":1.431,"15min":0.747},"min":{"1min":0.26700000000000002,"5min":0.217,"15min":0.217},"max":{"1min":46.787999999999997,"5min":46.787999999999997,"15min":46.787999999999997},"last":1.5629999999999999}]},{"osd":1,"last update":"Tue Sep 10 18:49:38 2024","interfaces":[{"interface":"back","average":{"1min":1.6679999999999999,"5min":0.91900000000000004,"15min":0.58099999999999996},"min":{"1min":0.25900000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":5.827,"5min":5.827,"15min":5.827},"last":3.488},{"interface":"front","average":{"1min":1.8140000000000001,"5min":0.90700000000000003,"15min":0.58099999999999996},"min":{"1min":0.27100000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":8.0030000000000001,"5min":8.0030000000000001,"15min":8.0030000000000001},"last":0.49399999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:49:38 2024","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.70099999999999996,"15min":0.52800000000000002},"min":{"1min":0.25900000000000001,"5min":0.246,"15min":0.246},"max":{"1min":4.5220000000000002,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":4.2629999999999999},{"interface":"front","average":{"1min":1.3939999999999999,"5min":0.65300000000000002,"15min":0.51100000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":6.3170000000000002,"5min":6.3170000000000002,"15min":6.3170000000000002},"last":0.92600000000000005}]},{"osd":4,"last update":"Tue Sep 10 18:49:02 2024","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":0.73599999999999999,"15min":0.60799999999999998},"min":{"1min":0.36399999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":6.3440000000000003,"5min":6.3440000000000003,"15min":6.3440000000000003},"last":0.44500000000000001},{"interface":"front","average":{"1min":1.2789999999999999,"5min":0.74299999999999999,"15min":0.63100000000000001},"min":{"1min":0.313,"5min":0.311,"15min":0.311},"max":{"1min":6.29,"5min":6.29,"15min":6.29},"last":0.41899999999999998}]},{"osd":5,"last update":"Tue Sep 10 18:49:33 2024","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":0.877,"15min":0.65300000000000002},"min":{"1min":0.36599999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":4.8179999999999996,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":1.776},{"interface":"front","average":{"1min":1.131,"5min":0.76400000000000001,"15min":0.63},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":4.5439999999999996,"5min":4.5439999999999996,"15min":4.5439999999999996},"last":2.7090000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:48:49 2024","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":0.72799999999999998,"15min":0.61899999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":3.859,"5min":3.859,"15min":3.859},"last":2.7490000000000001},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.64300000000000002,"15min":0.58499999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":2.5310000000000001,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":2.645}]},{"osd":7,"last update":"Tue Sep 10 18:49:15 2024","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":0.81000000000000005,"15min":0.624},"min":{"1min":0.38600000000000001,"5min":0.375,"15min":0.375},"max":{"1min":4.71,"5min":4.71,"15min":4.71},"last":1.595},{"interface":"front","average":{"1min":1.3859999999999999,"5min":0.77000000000000002,"15min":0.64000000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":3.8849999999999998,"5min":3.8849999999999998,"15min":3.8849999999999998},"last":0.51500000000000001}]}]},{"osd":2,"up_from":0,"seq":77309411409,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3781904,"kb_used_data":3476164,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":89963248,"statfs":{"total":95995035648,"available":92122365952,"internally_reserved":0,"allocated":3559591936,"data_stored":3559181900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:48:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":0.54200000000000004,"15min":0.38600000000000001},"min":{"1min":0.38300000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":3.1520000000000001,"5min":3.1520000000000001,"15min":3.1520000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":1.502,"5min":0.60999999999999999,"15min":0.41799999999999998},"min":{"1min":0.442,"5min":0.252,"15min":0.21199999999999999},"max":{"1min":4.2450000000000001,"5min":4.2450000000000001,"15min":4.2450000000000001},"last":3.484}]},{"osd":2,"last update":"Tue Sep 10 18:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.54300000000000004,"15min":0.41799999999999998},"min":{"1min":0.26700000000000002,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":3.1259999999999999,"5min":3.8479999999999999,"15min":3.8479999999999999},"last":0.627},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.502,"15min":0.41799999999999998},"min":{"1min":0.36799999999999999,"5min":0.248,"15min":0.248},"max":{"1min":2.1459999999999999,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":0.73199999999999998}]},{"osd":3,"last update":"Tue Sep 10 18:48:44 2024","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":0.65000000000000002,"15min":0.52300000000000002},"min":{"1min":0.29199999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.6160000000000001,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":3.2120000000000002},{"interface":"front","average":{"1min":2.2349999999999999,"5min":0.77600000000000002,"15min":0.54000000000000004},"min":{"1min":0.35999999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":5.3140000000000001,"5min":5.3140000000000001,"15min":5.3140000000000001},"last":3.0459999999999998}]},{"osd":4,"last update":"Tue Sep 10 18:49:04 2024","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":0.74099999999999999,"15min":0.61499999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":3.1520000000000001,"5min":4.1520000000000001,"15min":4.1520000000000001},"last":2.9510000000000001},{"interface":"front","average":{"1min":1.1839999999999999,"5min":0.68400000000000005,"15min":0.57999999999999996},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":3.9540000000000002,"5min":3.9540000000000002,"15min":3.9540000000000002},"last":0.65300000000000002}]},{"osd":5,"last update":"Tue Sep 10 18:49:25 2024","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":0.753,"15min":0.65400000000000003},"min":{"1min":0.33500000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":4.5869999999999997,"5min":4.5869999999999997,"15min":4.5869999999999997},"last":3.1190000000000002},{"interface":"front","average":{"1min":1.476,"5min":0.84799999999999998,"15min":0.71199999999999997},"min":{"1min":0.311,"5min":0.31,"15min":0.31},"max":{"1min":3.4020000000000001,"5min":4.1130000000000004,"15min":4.1130000000000004},"last":3.0059999999999998}]},{"osd":6,"last update":"Tue Sep 10 18:48:44 2024","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":0.68999999999999995,"15min":0.65100000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":3.157,"5min":3.157,"15min":3.157},"last":3.1539999999999999},{"interface":"front","average":{"1min":1.2729999999999999,"5min":0.76100000000000001,"15min":0.70299999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":4.1769999999999996,"5min":4.1769999999999996,"15min":4.1769999999999996},"last":2.9849999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:49:19 2024","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":0.73799999999999999,"15min":0.67800000000000005},"min":{"1min":0.40300000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":3.1930000000000001,"5min":3.1930000000000001,"15min":3.1930000000000001},"last":3.1840000000000002},{"interface":"front","average":{"1min":1.4390000000000001,"5min":0.85999999999999999,"15min":0.71799999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":3.4279999999999999,"5min":4.1989999999999998,"15min":4.1989999999999998},"last":3.0270000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3742916,"kb_used_data":3445560,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":90002236,"statfs":{"total":95995035648,"available":92162289664,"internally_reserved":0,"allocated":3528253440,"data_stored":3527855401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 18:48:55 2024","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":0.57799999999999996,"15min":0.38800000000000001},"min":{"1min":0.186,"5min":0.186,"15min":0.17699999999999999},"max":{"1min":4.7599999999999998,"5min":4.7599999999999998,"15min":4.7599999999999998},"last":0.38},{"interface":"front","average":{"1min":1.575,"5min":0.63300000000000001,"15min":0.41399999999999998},"min":{"1min":0.38,"5min":0.187,"15min":0.187},"max":{"1min":4.5110000000000001,"5min":4.5110000000000001,"15min":4.5110000000000001},"last":5.1719999999999997}]},{"osd":2,"last update":"Tue Sep 10 18:49:26 2024","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":0.67600000000000005,"15min":0.47999999999999998},"min":{"1min":0.26900000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":4.1719999999999997,"5min":4.6509999999999998,"15min":4.6509999999999998},"last":0.22700000000000001},{"interface":"front","average":{"1min":1.0669999999999999,"5min":0.62,"15min":0.443},"min":{"1min":0.251,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":3.266,"5min":4.6230000000000002,"15min":4.6230000000000002},"last":0.28199999999999997}]},{"osd":3,"last update":"Tue Sep 10 18:48:38 2024","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":0.57699999999999996,"15min":0.45900000000000002},"min":{"1min":0.26000000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":3.8119999999999998,"5min":3.8119999999999998,"15min":3.8119999999999998},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.403,"5min":0.59999999999999998,"15min":0.47599999999999998},"min":{"1min":0.29499999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":3.6920000000000002,"5min":3.6920000000000002,"15min":3.6920000000000002},"last":5.093}]},{"osd":4,"last update":"Tue Sep 10 18:49:03 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":0.69099999999999995,"15min":0.58999999999999997},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":4.0519999999999996,"5min":4.0519999999999996,"15min":4.0519999999999996},"last":0.372},{"interface":"front","average":{"1min":1.3700000000000001,"5min":0.78800000000000003,"15min":0.60799999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":3.8330000000000002,"5min":4.6189999999999998,"15min":4.6189999999999998},"last":0.47099999999999997}]},{"osd":5,"last update":"Tue Sep 10 18:49:26 2024","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":0.80400000000000005,"15min":0.64000000000000001},"min":{"1min":0.33800000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":4.1319999999999997,"5min":4.1319999999999997,"15min":4.1319999999999997},"last":0.32800000000000001},{"interface":"front","average":{"1min":1.3779999999999999,"5min":0.84699999999999998,"15min":0.64500000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":3.673,"5min":4.6429999999999998,"15min":4.6429999999999998},"last":0.39900000000000002}]},{"osd":6,"last update":"Tue Sep 10 18:48:49 2024","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":0.72999999999999998,"15min":0.61199999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":3.855,"5min":3.855,"15min":3.855},"last":0.41699999999999998},{"interface":"front","average":{"1min":1.0780000000000001,"5min":0.76400000000000001,"15min":0.77900000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":2.5830000000000002,"5min":4.6630000000000003,"15min":4.6630000000000003},"last":0.35099999999999998}]},{"osd":7,"last update":"Tue Sep 10 18:49:12 2024","interfaces":[{"interface":"back","average":{"1min":3.859,"5min":1.2589999999999999,"15min":0.79100000000000004},"min":{"1min":0.34100000000000003,"5min":0.311,"15min":0.311},"max":{"1min":49.313000000000002,"5min":49.313000000000002,"15min":49.313000000000002},"last":0.53600000000000003},{"interface":"front","average":{"1min":3.8650000000000002,"5min":1.302,"15min":0.82899999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":49.063000000000002,"5min":49.063000000000002,"15min":49.063000000000002},"last":0.45100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3490254848,"data_stored":3490250776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3521318912,"data_stored":3521314840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4827316224,"data_stored":4827316224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2199322624,"data_stored":2199322624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1770586112,"data_stored":1770586112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2648248320,"data_stored":2648244248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2624913408,"data_stored":2624913408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T18:49:45.538 INFO:tasks.cephadm.ceph_manager.ceph:would drop below min_pgs, proj 6, currently 16 2024-09-10T18:49:45.542 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:45.576 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:45.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:45.577 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:45.678 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:45.718 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:45.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:45.720 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:45.758 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:49:45.821 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:45.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:45.853 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:45.853 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:45.899 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:49:45.901 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:49:45.902 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:49:45.942 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:49:45.942 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:49:45.954 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:45.974 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:49:45.974 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:49:46.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:46.006 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:46.006 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:46.007 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:49:46.008 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:49:46.037 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:49:46.037 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:49:46.070 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:49:46.070 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:49:46.107 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:46.112 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:49:46.113 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:49:46.137 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:46.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:46.138 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:46.147 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:49:46.148 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:49:46.239 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:46.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:46.268 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:46.268 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:46.369 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:46.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:46.408 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:46.409 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:46.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:46 smithi018 ceph-mon[40196]: pgmap v169: 148 pgs: 148 active+clean; 6.5 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 685 op/s 2024-09-10T18:49:46.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:46 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1543453159' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:49:46.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:46 smithi018 ceph-mon[29336]: pgmap v169: 148 pgs: 148 active+clean; 6.5 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 685 op/s 2024-09-10T18:49:46.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:46 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1543453159' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:49:46.510 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:46.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:46 smithi103 ceph-mon[33163]: pgmap v169: 148 pgs: 148 active+clean; 6.5 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 685 op/s 2024-09-10T18:49:46.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:46 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1543453159' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:49:46.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:49:46 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:49:46.551 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:46.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:46.552 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:46.653 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:46.691 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:46.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:46.692 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:46.793 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:46.827 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:46.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:46.828 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:46.929 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:46.957 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:46.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:46.958 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:47.059 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:47.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:47.089 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:47.089 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:47.190 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:47.216 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:47 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:47.216 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:47 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:49:47.229 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:47.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:47.231 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:47.331 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:47.368 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:47.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:47.369 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:47.470 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:47.488 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:47 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:49:47.509 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:47.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:47.510 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:47.611 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:47.639 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:47.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:47.640 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:47.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:47 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:47.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:47 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:49:47.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:47 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:49:47.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:47 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:47.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:47 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:49:47.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:47 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:49:47.741 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:47.780 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T18:49:47.780 INFO:teuthology.orchestra.run.smithi103.stdout: "ops": [], 2024-09-10T18:49:47.780 INFO:teuthology.orchestra.run.smithi103.stdout: "complaint_time": 30, 2024-09-10T18:49:47.780 INFO:teuthology.orchestra.run.smithi103.stdout: "num_blocked_ops": 0 2024-09-10T18:49:47.780 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T18:49:47.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:47.782 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:47.782 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:47.883 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:47.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:47.922 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:47.923 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:48.024 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:48.055 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:48.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:48.056 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:48.157 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:48.199 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:48.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:48.200 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:48.300 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:48.328 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:48.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:48.329 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:48.363 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:48 smithi018 ceph-mon[29336]: pgmap v170: 148 pgs: 148 active+clean; 6.2 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.19k op/s 2024-09-10T18:49:48.363 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:48 smithi018 ceph-mon[29336]: osdmap e122: 8 total, 8 up, 7 in 2024-09-10T18:49:48.363 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:48 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:49:48.363 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:48 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:49:48.364 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:48 smithi018 ceph-mon[40196]: pgmap v170: 148 pgs: 148 active+clean; 6.2 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.19k op/s 2024-09-10T18:49:48.364 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:48 smithi018 ceph-mon[40196]: osdmap e122: 8 total, 8 up, 7 in 2024-09-10T18:49:48.364 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:48 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:49:48.364 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:48 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:49:48.430 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:48.443 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:48 smithi103 ceph-mon[33163]: pgmap v170: 148 pgs: 148 active+clean; 6.2 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.19k op/s 2024-09-10T18:49:48.443 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:48 smithi103 ceph-mon[33163]: osdmap e122: 8 total, 8 up, 7 in 2024-09-10T18:49:48.443 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:48 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:49:48.443 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:48 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:49:48.471 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:48.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:48.472 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:48.482 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-10T18:49:48.573 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:48.602 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:48.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:48.603 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:48.704 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:48.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:48.753 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:48.754 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:48.854 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:48.897 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:48.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:48.898 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:48.900 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:49:48.999 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:49.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:49.038 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:49.038 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:49.139 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:49.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:49.180 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:49.180 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:49.281 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:49.292 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:49 smithi103 ceph-mon[33163]: osdmap e123: 8 total, 8 up, 7 in 2024-09-10T18:49:49.292 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:49 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2579718686' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:49:49.314 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:49.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:49.315 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:49.416 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:49.444 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:49.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:49.445 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:49.533 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:49 smithi018 ceph-mon[40196]: osdmap e123: 8 total, 8 up, 7 in 2024-09-10T18:49:49.533 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:49 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2579718686' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:49:49.533 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:49 smithi018 ceph-mon[29336]: osdmap e123: 8 total, 8 up, 7 in 2024-09-10T18:49:49.534 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:49 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2579718686' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:49:49.535 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is unset 2024-09-10T18:49:49.546 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:49.574 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:49.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:49.575 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:49.675 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:49.706 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:49.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:49.707 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:49.807 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:49.829 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T18:49:49.850 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:49.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:49.851 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:49.952 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:50.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:50.000 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:50.000 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:50.101 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:50.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:50.153 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:50.154 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:50.254 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:50.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:50.305 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:50.305 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:50.406 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:50.515 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:49:50.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:50 smithi103 ceph-mon[33163]: pgmap v173: 148 pgs: 148 active+clean; 6.2 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.14k op/s 2024-09-10T18:49:50.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:50 smithi103 ceph-mon[33163]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:49:50.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:50 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2579718686' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:49:50.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:50 smithi103 ceph-mon[33163]: osdmap e124: 8 total, 8 up, 7 in 2024-09-10T18:49:50.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:50 smithi103 ceph-mon[33163]: 6.14 scrub starts 2024-09-10T18:49:50.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:50 smithi103 ceph-mon[33163]: 6.14 scrub ok 2024-09-10T18:49:50.523 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:49:50 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:49:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:49:50.539 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-10T18:49:50.539 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T18:49:50.539 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2024-09-10T18:49:50.540 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T18:49:50.544 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:50.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:50.545 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:50.573 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:50 smithi018 ceph-mon[29336]: pgmap v173: 148 pgs: 148 active+clean; 6.2 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.14k op/s 2024-09-10T18:49:50.574 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:50 smithi018 ceph-mon[29336]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:49:50.574 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:50 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2579718686' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:49:50.574 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:50 smithi018 ceph-mon[29336]: osdmap e124: 8 total, 8 up, 7 in 2024-09-10T18:49:50.574 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:50 smithi018 ceph-mon[29336]: 6.14 scrub starts 2024-09-10T18:49:50.574 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:50 smithi018 ceph-mon[29336]: 6.14 scrub ok 2024-09-10T18:49:50.574 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:50 smithi018 ceph-mon[40196]: pgmap v173: 148 pgs: 148 active+clean; 6.2 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.14k op/s 2024-09-10T18:49:50.574 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:50 smithi018 ceph-mon[40196]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:49:50.574 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:50 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2579718686' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:49:50.574 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:50 smithi018 ceph-mon[40196]: osdmap e124: 8 total, 8 up, 7 in 2024-09-10T18:49:50.575 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:50 smithi018 ceph-mon[40196]: 6.14 scrub starts 2024-09-10T18:49:50.575 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:50 smithi018 ceph-mon[40196]: 6.14 scrub ok 2024-09-10T18:49:50.646 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:50.668 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:50.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:50.669 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:50.770 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:50.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:50.821 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:50.822 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:50.922 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:50.950 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:50.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:50.951 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:51.052 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:51.078 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:51.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:51.079 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:51.180 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:51.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:49:50 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:49:50] "GET /metrics HTTP/1.1" 200 223705 "" "Prometheus/2.43.0" 2024-09-10T18:49:51.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:51.212 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:51.213 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:51.313 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:51.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:51.359 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:51.359 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:51.460 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:51.500 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:51.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:51.501 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:51.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:51 smithi103 ceph-mon[33163]: 2.16 scrub starts 2024-09-10T18:49:51.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:51 smithi103 ceph-mon[33163]: 2.16 scrub ok 2024-09-10T18:49:51.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:51 smithi103 ceph-mon[33163]: 5.1f scrub starts 2024-09-10T18:49:51.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:51 smithi103 ceph-mon[33163]: 5.1f scrub ok 2024-09-10T18:49:51.602 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:51.629 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:51.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:51.630 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:51.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:51 smithi018 ceph-mon[40196]: 2.16 scrub starts 2024-09-10T18:49:51.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:51 smithi018 ceph-mon[40196]: 2.16 scrub ok 2024-09-10T18:49:51.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:51 smithi018 ceph-mon[40196]: 5.1f scrub starts 2024-09-10T18:49:51.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:51 smithi018 ceph-mon[40196]: 5.1f scrub ok 2024-09-10T18:49:51.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:51 smithi018 ceph-mon[29336]: 2.16 scrub starts 2024-09-10T18:49:51.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:51 smithi018 ceph-mon[29336]: 2.16 scrub ok 2024-09-10T18:49:51.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:51 smithi018 ceph-mon[29336]: 5.1f scrub starts 2024-09-10T18:49:51.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:51 smithi018 ceph-mon[29336]: 5.1f scrub ok 2024-09-10T18:49:51.731 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:51.779 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:51.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:51.782 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:51.813 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:49:51.883 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:51.926 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:51.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:51.927 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:52.017 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T18:49:52.017 INFO:teuthology.orchestra.run.smithi103.stdout: "size": 20, 2024-09-10T18:49:52.017 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 600, 2024-09-10T18:49:52.017 INFO:teuthology.orchestra.run.smithi103.stdout: "ops": [ 2024-09-10T18:49:52.018 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.018 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:15123 7.1 7:8ab84aee:::benchmark_data_smithi018_78628_object15122:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-10T18:49:52.018 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:47:49.069832+0000", 2024-09-10T18:49:52.018 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 122.940170274, 2024-09-10T18:49:52.018 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.13743958000000001, 2024-09-10T18:49:52.018 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:49:52.018 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:49:52.018 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:49:52.018 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:49:52.018 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:49:52.018 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 15123 2024-09-10T18:49:52.018 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.018 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:49:52.019 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.019 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:49:52.019 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.069832+0000", 2024-09-10T18:49:52.019 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.019 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.019 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.019 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:49:52.019 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.069832+0000", 2024-09-10T18:49:52.019 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.019 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.019 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.019 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:49:52.019 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.069831+0000", 2024-09-10T18:49:52.019 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:49:52.020 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.020 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.020 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:49:52.020 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.069870+0000", 2024-09-10T18:49:52.020 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.9096e-05 2024-09-10T18:49:52.020 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.020 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.020 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:49:52.020 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.069891+0000", 2024-09-10T18:49:52.020 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.1656999999999999e-05 2024-09-10T18:49:52.020 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.020 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.020 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:49:52.021 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.069897+0000", 2024-09-10T18:49:52.021 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.1809999999999998e-06 2024-09-10T18:49:52.021 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.021 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.021 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:49:52.021 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.069941+0000", 2024-09-10T18:49:52.021 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.3631000000000003e-05 2024-09-10T18:49:52.021 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.021 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.021 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:49:52.021 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.070064+0000", 2024-09-10T18:49:52.022 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000122488 2024-09-10T18:49:52.022 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.022 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.022 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,6", 2024-09-10T18:49:52.022 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.070205+0000", 2024-09-10T18:49:52.022 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00014103600000000001 2024-09-10T18:49:52.022 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.022 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.022 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:49:52.022 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.071177+0000", 2024-09-10T18:49:52.022 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00097243299999999998 2024-09-10T18:49:52.022 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.022 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.022 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.023 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.071494+0000", 2024-09-10T18:49:52.023 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000317292 2024-09-10T18:49:52.023 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.023 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.023 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.023 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.207096+0000", 2024-09-10T18:49:52.023 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.13560148699999999 2024-09-10T18:49:52.023 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.023 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.023 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:49:52.024 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.207191+0000", 2024-09-10T18:49:52.024 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.5328000000000006e-05 2024-09-10T18:49:52.024 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.024 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.024 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:49:52.024 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.207271+0000", 2024-09-10T18:49:52.024 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.0190000000000003e-05 2024-09-10T18:49:52.024 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.025 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:49:52.025 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.026 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.026 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.026 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:21439 7.1 7:8450f9c1:::benchmark_data_smithi018_78628_object21438:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-10T18:49:52.026 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:47:51.670495+0000", 2024-09-10T18:49:52.026 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 120.33950681899999, 2024-09-10T18:49:52.026 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.043494719000000001, 2024-09-10T18:49:52.026 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:49:52.026 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:49:52.026 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:49:52.026 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:49:52.027 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:49:52.027 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 21439 2024-09-10T18:49:52.027 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.027 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:49:52.027 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.027 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:49:52.027 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.670495+0000", 2024-09-10T18:49:52.027 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.027 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.027 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.027 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:49:52.027 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.670495+0000", 2024-09-10T18:49:52.027 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.028 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.028 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.028 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:49:52.028 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.670494+0000", 2024-09-10T18:49:52.028 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:49:52.028 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.028 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.028 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:49:52.029 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.670547+0000", 2024-09-10T18:49:52.029 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.3179e-05 2024-09-10T18:49:52.029 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.029 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.029 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:49:52.029 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.670555+0000", 2024-09-10T18:49:52.029 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.011e-06 2024-09-10T18:49:52.029 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.029 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.029 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:49:52.029 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.670559+0000", 2024-09-10T18:49:52.029 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.7850000000000002e-06 2024-09-10T18:49:52.029 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.030 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.030 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:49:52.030 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.670582+0000", 2024-09-10T18:49:52.030 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.3124000000000001e-05 2024-09-10T18:49:52.030 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.030 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.030 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:49:52.030 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.670652+0000", 2024-09-10T18:49:52.030 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.0078000000000002e-05 2024-09-10T18:49:52.030 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.030 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.030 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,6", 2024-09-10T18:49:52.030 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.670736+0000", 2024-09-10T18:49:52.031 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.3774000000000001e-05 2024-09-10T18:49:52.031 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.031 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.031 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:49:52.031 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.671582+0000", 2024-09-10T18:49:52.031 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000846615 2024-09-10T18:49:52.031 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.031 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.031 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.031 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.673240+0000", 2024-09-10T18:49:52.031 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0016570560000000001 2024-09-10T18:49:52.031 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.031 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.032 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.032 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.713908+0000", 2024-09-10T18:49:52.032 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.040668514000000003 2024-09-10T18:49:52.032 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.032 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.032 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:49:52.032 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.713957+0000", 2024-09-10T18:49:52.032 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.8916999999999999e-05 2024-09-10T18:49:52.032 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.032 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.032 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:49:52.032 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.713990+0000", 2024-09-10T18:49:52.032 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.3016999999999999e-05 2024-09-10T18:49:52.033 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.033 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:49:52.033 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.033 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.033 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.033 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:21446 7.1 7:80f67a92:::benchmark_data_smithi018_78628_object21445:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-10T18:49:52.033 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:47:51.673275+0000", 2024-09-10T18:49:52.033 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 120.336727525, 2024-09-10T18:49:52.033 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.040831926999999997, 2024-09-10T18:49:52.033 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:49:52.033 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:49:52.033 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:49:52.033 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:49:52.034 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:49:52.034 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 21446 2024-09-10T18:49:52.034 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.034 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:49:52.034 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.034 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:49:52.034 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.673275+0000", 2024-09-10T18:49:52.034 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.034 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.034 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.034 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:49:52.034 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.673275+0000", 2024-09-10T18:49:52.035 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.035 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.035 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.035 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:49:52.035 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.673274+0000", 2024-09-10T18:49:52.035 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:49:52.035 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.035 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.035 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:49:52.035 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.673328+0000", 2024-09-10T18:49:52.035 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.4209000000000002e-05 2024-09-10T18:49:52.035 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.035 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.035 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:49:52.036 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.673339+0000", 2024-09-10T18:49:52.036 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.1433e-05 2024-09-10T18:49:52.036 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.036 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.036 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:49:52.036 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.673343+0000", 2024-09-10T18:49:52.036 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.76e-06 2024-09-10T18:49:52.036 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.036 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.036 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:49:52.036 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.673376+0000", 2024-09-10T18:49:52.036 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.2909e-05 2024-09-10T18:49:52.037 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.037 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.037 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:49:52.037 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.673523+0000", 2024-09-10T18:49:52.037 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00014723500000000001 2024-09-10T18:49:52.037 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.037 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.037 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,6", 2024-09-10T18:49:52.037 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.673668+0000", 2024-09-10T18:49:52.037 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000145279 2024-09-10T18:49:52.037 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.037 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.037 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:49:52.038 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.674479+0000", 2024-09-10T18:49:52.038 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00081081600000000001 2024-09-10T18:49:52.038 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.038 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.038 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.038 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.675834+0000", 2024-09-10T18:49:52.038 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.001354715 2024-09-10T18:49:52.038 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.038 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.038 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.038 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.714032+0000", 2024-09-10T18:49:52.038 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.038198013000000003 2024-09-10T18:49:52.038 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.039 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.039 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:49:52.039 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.714075+0000", 2024-09-10T18:49:52.039 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.3519000000000001e-05 2024-09-10T18:49:52.039 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.039 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.039 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:49:52.039 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.714106+0000", 2024-09-10T18:49:52.039 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.0964999999999999e-05 2024-09-10T18:49:52.039 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.039 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:49:52.039 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.039 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.040 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.040 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:21452 7.1 7:808adcdc:::benchmark_data_smithi018_78628_object21451:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-10T18:49:52.040 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:47:51.675096+0000", 2024-09-10T18:49:52.040 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 120.33490595000001, 2024-09-10T18:49:52.040 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.039757426999999998, 2024-09-10T18:49:52.040 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:49:52.040 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:49:52.040 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:49:52.040 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:49:52.040 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:49:52.040 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 21452 2024-09-10T18:49:52.040 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.041 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:49:52.041 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.041 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:49:52.041 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.675096+0000", 2024-09-10T18:49:52.041 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.041 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.041 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.041 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:49:52.041 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.675096+0000", 2024-09-10T18:49:52.041 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.041 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.041 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.041 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:49:52.042 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.675095+0000", 2024-09-10T18:49:52.042 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:49:52.042 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.042 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.042 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:49:52.042 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.675161+0000", 2024-09-10T18:49:52.042 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.5692999999999996e-05 2024-09-10T18:49:52.042 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.042 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.042 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:49:52.042 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.675170+0000", 2024-09-10T18:49:52.042 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.5059999999999995e-06 2024-09-10T18:49:52.042 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.043 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.043 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:49:52.043 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.675177+0000", 2024-09-10T18:49:52.043 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.0160000000000003e-06 2024-09-10T18:49:52.043 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.043 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.043 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:49:52.043 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.675219+0000", 2024-09-10T18:49:52.043 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.2608999999999998e-05 2024-09-10T18:49:52.043 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.043 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.043 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:49:52.043 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.675295+0000", 2024-09-10T18:49:52.043 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.5451999999999995e-05 2024-09-10T18:49:52.044 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.044 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.044 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,6", 2024-09-10T18:49:52.044 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.675380+0000", 2024-09-10T18:49:52.044 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.4895999999999995e-05 2024-09-10T18:49:52.044 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.044 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.044 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:49:52.044 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.675929+0000", 2024-09-10T18:49:52.044 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00054983799999999998 2024-09-10T18:49:52.044 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.044 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.044 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.045 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.676458+0000", 2024-09-10T18:49:52.045 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00052884099999999995 2024-09-10T18:49:52.045 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.045 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.045 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.045 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.714772+0000", 2024-09-10T18:49:52.045 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.038313426999999997 2024-09-10T18:49:52.045 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.045 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.045 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:49:52.045 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.714814+0000", 2024-09-10T18:49:52.045 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.2295000000000001e-05 2024-09-10T18:49:52.045 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.046 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.046 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:49:52.046 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:51.714854+0000", 2024-09-10T18:49:52.046 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.9623000000000003e-05 2024-09-10T18:49:52.046 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.046 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:49:52.046 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.046 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.046 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.046 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:22473 7.1 7:858c0b11:::benchmark_data_smithi018_78628_object22472:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-10T18:49:52.046 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:47:52.079538+0000", 2024-09-10T18:49:52.046 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 119.93046379499999, 2024-09-10T18:49:52.046 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.087482985999999999, 2024-09-10T18:49:52.047 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:49:52.047 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:49:52.047 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:49:52.047 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:49:52.047 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:49:52.047 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 22473 2024-09-10T18:49:52.047 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.047 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:49:52.047 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.047 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:49:52.047 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.079538+0000", 2024-09-10T18:49:52.047 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.047 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.047 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.048 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:49:52.048 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.079538+0000", 2024-09-10T18:49:52.048 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.048 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.048 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.048 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:49:52.048 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.079537+0000", 2024-09-10T18:49:52.048 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:49:52.048 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.048 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.048 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:49:52.048 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.079659+0000", 2024-09-10T18:49:52.048 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000121824 2024-09-10T18:49:52.049 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.049 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.049 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:49:52.049 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.079670+0000", 2024-09-10T18:49:52.049 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.1633e-05 2024-09-10T18:49:52.049 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.049 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.049 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:49:52.049 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.079676+0000", 2024-09-10T18:49:52.049 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.9279999999999998e-06 2024-09-10T18:49:52.049 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.049 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.049 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:49:52.049 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.079744+0000", 2024-09-10T18:49:52.050 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.792e-05 2024-09-10T18:49:52.050 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.050 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.050 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:49:52.050 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.079835+0000", 2024-09-10T18:49:52.050 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.1257999999999999e-05 2024-09-10T18:49:52.050 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.050 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.050 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,6", 2024-09-10T18:49:52.050 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.079910+0000", 2024-09-10T18:49:52.050 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.4733000000000004e-05 2024-09-10T18:49:52.050 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.050 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.051 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:49:52.051 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.080683+0000", 2024-09-10T18:49:52.051 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00077321599999999996 2024-09-10T18:49:52.051 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.051 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.051 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.051 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.081607+0000", 2024-09-10T18:49:52.051 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00092401699999999996 2024-09-10T18:49:52.051 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.051 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.051 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.051 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.166953+0000", 2024-09-10T18:49:52.051 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.085345859999999996 2024-09-10T18:49:52.052 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.052 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.052 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:49:52.052 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.166985+0000", 2024-09-10T18:49:52.052 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.2057999999999997e-05 2024-09-10T18:49:52.052 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.052 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.052 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:49:52.052 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.167021+0000", 2024-09-10T18:49:52.052 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.5911999999999998e-05 2024-09-10T18:49:52.052 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.052 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:49:52.053 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.053 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.053 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.053 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:29034 7.1 7:8f36cf78:::benchmark_data_smithi018_78628_object29033:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-10T18:49:52.053 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:47:54.955181+0000", 2024-09-10T18:49:52.054 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 117.05482060999999, 2024-09-10T18:49:52.054 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.056116859999999998, 2024-09-10T18:49:52.054 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:49:52.054 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:49:52.054 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:49:52.054 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:49:52.054 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:49:52.054 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 29034 2024-09-10T18:49:52.054 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.055 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:49:52.055 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.055 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:49:52.055 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:54.955181+0000", 2024-09-10T18:49:52.055 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.055 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.055 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.055 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:49:52.055 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:54.955181+0000", 2024-09-10T18:49:52.055 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.055 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.055 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.056 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:49:52.056 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:54.955180+0000", 2024-09-10T18:49:52.056 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:49:52.056 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.056 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.056 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:49:52.056 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:54.955284+0000", 2024-09-10T18:49:52.056 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000103849 2024-09-10T18:49:52.056 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.056 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.056 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:49:52.056 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:54.955297+0000", 2024-09-10T18:49:52.057 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.2819999999999999e-05 2024-09-10T18:49:52.057 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.057 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.057 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:49:52.057 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:54.955303+0000", 2024-09-10T18:49:52.057 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.9850000000000003e-06 2024-09-10T18:49:52.057 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.057 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.057 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:49:52.057 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:54.955340+0000", 2024-09-10T18:49:52.057 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.7116000000000002e-05 2024-09-10T18:49:52.057 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.058 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.058 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:49:52.058 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:54.955437+0000", 2024-09-10T18:49:52.058 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.6669000000000006e-05 2024-09-10T18:49:52.058 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.058 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.058 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,6", 2024-09-10T18:49:52.058 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:54.955567+0000", 2024-09-10T18:49:52.058 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00013025299999999999 2024-09-10T18:49:52.058 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.058 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.058 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:49:52.058 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:54.958146+0000", 2024-09-10T18:49:52.059 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0025796529999999999 2024-09-10T18:49:52.059 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.059 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.059 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.059 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:54.958487+0000", 2024-09-10T18:49:52.059 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00034067 2024-09-10T18:49:52.059 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.059 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.059 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.059 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.011122+0000", 2024-09-10T18:49:52.059 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.052634752999999999 2024-09-10T18:49:52.059 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.059 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.060 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:49:52.060 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.011229+0000", 2024-09-10T18:49:52.060 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000107162 2024-09-10T18:49:52.060 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.060 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.060 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:49:52.060 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.011298+0000", 2024-09-10T18:49:52.060 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.9245000000000005e-05 2024-09-10T18:49:52.060 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.060 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:49:52.060 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.060 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.061 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.061 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:29036 7.1 7:8b996176:::benchmark_data_smithi018_78628_object29035:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-10T18:49:52.061 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:47:54.955482+0000", 2024-09-10T18:49:52.061 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 117.05452053400001, 2024-09-10T18:49:52.061 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.058636228999999998, 2024-09-10T18:49:52.061 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:49:52.061 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:49:52.061 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:49:52.061 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:49:52.061 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:49:52.061 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 29036 2024-09-10T18:49:52.061 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.062 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:49:52.062 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.062 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:49:52.062 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:54.955482+0000", 2024-09-10T18:49:52.062 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.062 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.062 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.062 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:49:52.062 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:54.955482+0000", 2024-09-10T18:49:52.062 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.062 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.062 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.062 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:49:52.063 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:54.955481+0000", 2024-09-10T18:49:52.063 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999995 2024-09-10T18:49:52.063 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.063 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.063 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:49:52.063 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:54.955650+0000", 2024-09-10T18:49:52.063 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00016943199999999999 2024-09-10T18:49:52.063 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.063 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.063 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:49:52.063 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:54.955660+0000", 2024-09-10T18:49:52.063 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.9639999999999998e-06 2024-09-10T18:49:52.063 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.064 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.064 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:49:52.064 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:54.955666+0000", 2024-09-10T18:49:52.064 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.5670000000000002e-06 2024-09-10T18:49:52.064 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:52.066 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.066 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.066 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:49:52.066 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:54.956859+0000", 2024-09-10T18:49:52.066 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.001193349 2024-09-10T18:49:52.066 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.066 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.066 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:49:52.066 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:54.956967+0000", 2024-09-10T18:49:52.067 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00010814099999999999 2024-09-10T18:49:52.067 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.067 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.067 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,6", 2024-09-10T18:49:52.067 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:54.957118+0000", 2024-09-10T18:49:52.067 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00015047500000000001 2024-09-10T18:49:52.067 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.067 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.067 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:49:52.067 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:54.958252+0000", 2024-09-10T18:49:52.067 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.001134267 2024-09-10T18:49:52.067 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.067 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.067 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.067 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:54.958523+0000", 2024-09-10T18:49:52.068 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00027055299999999998 2024-09-10T18:49:52.068 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.068 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.068 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.068 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.014044+0000", 2024-09-10T18:49:52.068 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.055521595999999999 2024-09-10T18:49:52.068 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.068 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.068 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:49:52.068 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.014087+0000", 2024-09-10T18:49:52.068 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.2744999999999998e-05 2024-09-10T18:49:52.068 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.068 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.068 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:49:52.069 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.014118+0000", 2024-09-10T18:49:52.069 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.0830000000000001e-05 2024-09-10T18:49:52.069 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.069 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:49:52.069 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.069 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.069 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.069 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:30344 7.1 7:803ee176:::benchmark_data_smithi018_78628_object30343:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-10T18:49:52.069 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:47:55.587187+0000", 2024-09-10T18:49:52.069 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 116.42281502100001, 2024-09-10T18:49:52.069 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.040353365000000002, 2024-09-10T18:49:52.069 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:49:52.069 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:49:52.070 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:49:52.070 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:49:52.070 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:49:52.070 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 30344 2024-09-10T18:49:52.070 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.070 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:49:52.070 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.070 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:49:52.070 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.587187+0000", 2024-09-10T18:49:52.070 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.070 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.070 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.070 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:49:52.070 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.587187+0000", 2024-09-10T18:49:52.070 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.071 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.071 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.071 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:49:52.071 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.587185+0000", 2024-09-10T18:49:52.071 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999976 2024-09-10T18:49:52.071 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.071 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.071 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:49:52.071 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.587246+0000", 2024-09-10T18:49:52.071 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.1340000000000006e-05 2024-09-10T18:49:52.071 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.071 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.072 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:49:52.072 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.587264+0000", 2024-09-10T18:49:52.072 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.8335e-05 2024-09-10T18:49:52.072 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.072 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.072 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:49:52.072 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.587271+0000", 2024-09-10T18:49:52.072 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.2840000000000003e-06 2024-09-10T18:49:52.072 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.072 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.072 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:49:52.072 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.587369+0000", 2024-09-10T18:49:52.072 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.8208000000000006e-05 2024-09-10T18:49:52.073 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.073 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.073 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:49:52.073 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.587502+0000", 2024-09-10T18:49:52.073 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00013353799999999999 2024-09-10T18:49:52.073 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.073 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.073 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,6", 2024-09-10T18:49:52.073 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.587629+0000", 2024-09-10T18:49:52.073 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00012620700000000001 2024-09-10T18:49:52.073 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.073 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.073 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:49:52.073 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.589175+0000", 2024-09-10T18:49:52.074 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0015468109999999999 2024-09-10T18:49:52.074 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.074 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.074 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.074 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.589468+0000", 2024-09-10T18:49:52.074 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00029244200000000002 2024-09-10T18:49:52.074 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.074 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.074 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.074 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.627436+0000", 2024-09-10T18:49:52.074 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.037968090000000003 2024-09-10T18:49:52.074 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.074 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.074 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:49:52.075 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.627504+0000", 2024-09-10T18:49:52.075 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.7761000000000004e-05 2024-09-10T18:49:52.075 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.075 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.075 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:49:52.075 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.627540+0000", 2024-09-10T18:49:52.075 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.6677000000000001e-05 2024-09-10T18:49:52.075 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.075 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:49:52.075 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.076 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.076 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.076 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:30673 7.1 7:87ad79f2:::benchmark_data_smithi018_78628_object30672:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-10T18:49:52.076 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:47:55.754459+0000", 2024-09-10T18:49:52.076 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 116.25554351, 2024-09-10T18:49:52.076 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.061076683, 2024-09-10T18:49:52.076 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:49:52.076 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:49:52.076 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:49:52.076 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:49:52.076 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:49:52.076 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 30673 2024-09-10T18:49:52.076 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.077 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:49:52.077 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.077 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:49:52.077 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.754459+0000", 2024-09-10T18:49:52.077 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.077 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.077 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.077 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:49:52.077 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.754459+0000", 2024-09-10T18:49:52.077 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.077 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.077 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.077 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:49:52.077 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.754457+0000", 2024-09-10T18:49:52.078 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:49:52.078 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.078 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.078 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:49:52.078 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.754512+0000", 2024-09-10T18:49:52.078 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.5124999999999998e-05 2024-09-10T18:49:52.078 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.078 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.078 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:49:52.078 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.754524+0000", 2024-09-10T18:49:52.078 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.1418e-05 2024-09-10T18:49:52.078 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.078 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.078 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:49:52.079 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.754529+0000", 2024-09-10T18:49:52.079 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.7169999999999996e-06 2024-09-10T18:49:52.079 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.079 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.079 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:49:52.079 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.754583+0000", 2024-09-10T18:49:52.079 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.3497e-05 2024-09-10T18:49:52.079 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.079 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.079 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:49:52.079 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.754682+0000", 2024-09-10T18:49:52.079 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.9580999999999994e-05 2024-09-10T18:49:52.079 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.080 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.080 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,6", 2024-09-10T18:49:52.080 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.754786+0000", 2024-09-10T18:49:52.080 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000103471 2024-09-10T18:49:52.080 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.080 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.080 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:49:52.080 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.755989+0000", 2024-09-10T18:49:52.080 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0012030369999999999 2024-09-10T18:49:52.080 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.080 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.080 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.080 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.756129+0000", 2024-09-10T18:49:52.081 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00014043399999999999 2024-09-10T18:49:52.081 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.081 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.081 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.081 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.815454+0000", 2024-09-10T18:49:52.081 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.059324344000000001 2024-09-10T18:49:52.081 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.081 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.081 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:49:52.081 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.815503+0000", 2024-09-10T18:49:52.081 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.9531999999999999e-05 2024-09-10T18:49:52.081 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.081 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.081 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:49:52.082 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.815535+0000", 2024-09-10T18:49:52.082 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.2002000000000003e-05 2024-09-10T18:49:52.082 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.082 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:49:52.082 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.082 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.082 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.082 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:31048 7.1 7:824dcd4d:::benchmark_data_smithi018_78628_object31047:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-10T18:49:52.082 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:47:55.985783+0000", 2024-09-10T18:49:52.082 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 116.024219094, 2024-09-10T18:49:52.082 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.103195603, 2024-09-10T18:49:52.082 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:49:52.083 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:49:52.083 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:49:52.083 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:49:52.083 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:49:52.083 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 31048 2024-09-10T18:49:52.083 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.083 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:49:52.083 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.083 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:49:52.083 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.985783+0000", 2024-09-10T18:49:52.083 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.083 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.083 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.083 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:49:52.084 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.985783+0000", 2024-09-10T18:49:52.084 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.084 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.084 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.084 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:49:52.084 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.985782+0000", 2024-09-10T18:49:52.084 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:49:52.084 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.084 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.084 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:49:52.084 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.985811+0000", 2024-09-10T18:49:52.084 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.9643000000000001e-05 2024-09-10T18:49:52.084 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.084 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.085 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:49:52.085 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.985832+0000", 2024-09-10T18:49:52.085 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.0312000000000001e-05 2024-09-10T18:49:52.085 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.085 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.085 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:49:52.085 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.985837+0000", 2024-09-10T18:49:52.085 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.614e-06 2024-09-10T18:49:52.085 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.085 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.085 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:49:52.085 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.985884+0000", 2024-09-10T18:49:52.085 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.6588000000000002e-05 2024-09-10T18:49:52.085 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.086 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.086 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:49:52.086 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.986000+0000", 2024-09-10T18:49:52.086 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00011569399999999999 2024-09-10T18:49:52.086 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.086 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.086 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,6", 2024-09-10T18:49:52.086 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.986154+0000", 2024-09-10T18:49:52.086 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000154957 2024-09-10T18:49:52.086 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.086 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.086 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:49:52.086 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.987331+0000", 2024-09-10T18:49:52.086 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.001176292 2024-09-10T18:49:52.087 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.087 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.087 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.087 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.987443+0000", 2024-09-10T18:49:52.087 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000112465 2024-09-10T18:49:52.087 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.087 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.087 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.087 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:56.088894+0000", 2024-09-10T18:49:52.087 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.101450464 2024-09-10T18:49:52.087 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.087 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.087 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:49:52.087 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:56.088942+0000", 2024-09-10T18:49:52.088 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.8100999999999999e-05 2024-09-10T18:49:52.088 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.088 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.088 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:49:52.088 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:56.088979+0000", 2024-09-10T18:49:52.088 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.6746999999999999e-05 2024-09-10T18:49:52.088 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.088 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:49:52.088 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.088 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.088 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.088 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:31051 7.1 7:841c6334:::benchmark_data_smithi018_78628_object31050:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-10T18:49:52.088 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:47:55.988409+0000", 2024-09-10T18:49:52.088 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 116.021592732, 2024-09-10T18:49:52.089 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.10067891499999999, 2024-09-10T18:49:52.089 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:49:52.089 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:49:52.089 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:49:52.089 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:49:52.089 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:49:52.089 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 31051 2024-09-10T18:49:52.089 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.089 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:49:52.089 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.089 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:49:52.089 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.988409+0000", 2024-09-10T18:49:52.089 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.089 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.090 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.090 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:49:52.090 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.988409+0000", 2024-09-10T18:49:52.090 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.090 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.090 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.090 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:49:52.090 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.988408+0000", 2024-09-10T18:49:52.090 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:49:52.090 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.090 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.090 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:49:52.090 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.988475+0000", 2024-09-10T18:49:52.090 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.7288999999999999e-05 2024-09-10T18:49:52.090 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.091 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.091 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:49:52.091 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.988490+0000", 2024-09-10T18:49:52.091 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.4297999999999999e-05 2024-09-10T18:49:52.091 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.091 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.091 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:49:52.091 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.988495+0000", 2024-09-10T18:49:52.091 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.0010000000000001e-06 2024-09-10T18:49:52.091 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.091 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.091 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:49:52.091 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.988545+0000", 2024-09-10T18:49:52.091 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.0420000000000002e-05 2024-09-10T18:49:52.092 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.092 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.092 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:49:52.092 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.988639+0000", 2024-09-10T18:49:52.092 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.3896999999999999e-05 2024-09-10T18:49:52.092 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.092 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.092 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,6", 2024-09-10T18:49:52.092 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.988762+0000", 2024-09-10T18:49:52.092 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00012300599999999999 2024-09-10T18:49:52.092 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.092 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.092 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:49:52.093 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.989908+0000", 2024-09-10T18:49:52.093 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.001146001 2024-09-10T18:49:52.093 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.093 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.093 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.093 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.990133+0000", 2024-09-10T18:49:52.093 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00022450000000000001 2024-09-10T18:49:52.093 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.093 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.093 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.093 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:56.089036+0000", 2024-09-10T18:49:52.093 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.098903797000000002 2024-09-10T18:49:52.093 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.093 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.094 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:49:52.094 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:56.089067+0000", 2024-09-10T18:49:52.094 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.0215000000000001e-05 2024-09-10T18:49:52.094 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.094 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.094 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:49:52.094 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:56.089088+0000", 2024-09-10T18:49:52.094 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.1645e-05 2024-09-10T18:49:52.094 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.094 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:49:52.094 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.094 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.094 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.095 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:32508 7.1 7:8f601b4c:::benchmark_data_smithi018_78628_object32507:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-10T18:49:52.095 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:47:56.770553+0000", 2024-09-10T18:49:52.095 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 115.239449364, 2024-09-10T18:49:52.095 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.052042183999999998, 2024-09-10T18:49:52.095 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:49:52.095 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:49:52.095 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:49:52.095 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:49:52.095 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:49:52.095 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 32508 2024-09-10T18:49:52.095 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.095 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:49:52.095 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.095 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:49:52.096 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:56.770553+0000", 2024-09-10T18:49:52.096 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.096 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.096 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.096 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:49:52.096 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:56.770553+0000", 2024-09-10T18:49:52.096 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.096 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.096 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.096 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:49:52.096 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:56.770551+0000", 2024-09-10T18:49:52.096 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:49:52.096 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.097 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.097 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:49:52.097 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:56.770615+0000", 2024-09-10T18:49:52.097 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.3786000000000002e-05 2024-09-10T18:49:52.097 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.097 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.097 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:49:52.097 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:56.770626+0000", 2024-09-10T18:49:52.097 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.1408e-05 2024-09-10T18:49:52.097 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.097 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.097 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:49:52.097 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:56.770632+0000", 2024-09-10T18:49:52.098 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.5639999999999996e-06 2024-09-10T18:49:52.098 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.098 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.098 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:49:52.098 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:56.770684+0000", 2024-09-10T18:49:52.098 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.2397e-05 2024-09-10T18:49:52.098 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.098 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.098 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:49:52.098 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:56.770803+0000", 2024-09-10T18:49:52.098 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000119122 2024-09-10T18:49:52.098 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.098 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.099 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,6", 2024-09-10T18:49:52.099 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:56.770962+0000", 2024-09-10T18:49:52.099 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00015828399999999999 2024-09-10T18:49:52.099 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.099 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.099 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:49:52.099 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:56.772165+0000", 2024-09-10T18:49:52.099 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0012036760000000001 2024-09-10T18:49:52.099 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.099 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.099 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.099 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:56.772894+0000", 2024-09-10T18:49:52.099 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00072867899999999998 2024-09-10T18:49:52.100 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.100 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.100 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.100 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:56.822487+0000", 2024-09-10T18:49:52.100 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.049593409999999997 2024-09-10T18:49:52.100 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.100 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.100 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:49:52.100 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:56.822557+0000", 2024-09-10T18:49:52.100 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.9486999999999994e-05 2024-09-10T18:49:52.100 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.100 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.100 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:49:52.101 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:56.822595+0000", 2024-09-10T18:49:52.101 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.8003000000000002e-05 2024-09-10T18:49:52.101 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.101 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:49:52.101 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.101 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.101 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.101 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:109209 7.1 7:8662e22e:::benchmark_data_smithi018_78628_object109208:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-10T18:49:52.101 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:48:50.764480+0000", 2024-09-10T18:49:52.101 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 61.245522370000003, 2024-09-10T18:49:52.101 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.112708567, 2024-09-10T18:49:52.101 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:49:52.102 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:49:52.102 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:49:52.102 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:49:52.102 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:49:52.102 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 109209 2024-09-10T18:49:52.102 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.102 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:49:52.102 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.102 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:49:52.102 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.764480+0000", 2024-09-10T18:49:52.102 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.102 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.102 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.102 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:49:52.103 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.764480+0000", 2024-09-10T18:49:52.103 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.103 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.103 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.103 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:49:52.103 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.764478+0000", 2024-09-10T18:49:52.103 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:49:52.103 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.103 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.103 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:49:52.103 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.764530+0000", 2024-09-10T18:49:52.103 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.1928999999999997e-05 2024-09-10T18:49:52.103 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.103 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.104 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:49:52.104 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.764544+0000", 2024-09-10T18:49:52.104 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.4341999999999999e-05 2024-09-10T18:49:52.104 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.104 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.104 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:49:52.104 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.764550+0000", 2024-09-10T18:49:52.104 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.8860000000000001e-06 2024-09-10T18:49:52.104 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.104 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.104 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:49:52.104 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.764601+0000", 2024-09-10T18:49:52.105 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.0921000000000003e-05 2024-09-10T18:49:52.105 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.105 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.105 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:49:52.105 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.764724+0000", 2024-09-10T18:49:52.105 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00012290099999999999 2024-09-10T18:49:52.105 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.105 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.105 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,6", 2024-09-10T18:49:52.105 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.764872+0000", 2024-09-10T18:49:52.105 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00014779699999999999 2024-09-10T18:49:52.105 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.106 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.106 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.106 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.769126+0000", 2024-09-10T18:49:52.106 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0042536029999999999 2024-09-10T18:49:52.106 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.106 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.106 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:49:52.106 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.769133+0000", 2024-09-10T18:49:52.106 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.968e-06 2024-09-10T18:49:52.106 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.106 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.106 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.106 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.877097+0000", 2024-09-10T18:49:52.107 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.107964835 2024-09-10T18:49:52.107 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.107 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.107 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:49:52.107 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.877152+0000", 2024-09-10T18:49:52.107 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.4419999999999997e-05 2024-09-10T18:49:52.107 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.107 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.107 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:49:52.107 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.877188+0000", 2024-09-10T18:49:52.107 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.6483999999999998e-05 2024-09-10T18:49:52.107 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.107 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:49:52.108 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.108 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.108 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.108 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:109528 7.1 7:85156bee:::benchmark_data_smithi018_78628_object109527:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-10T18:49:52.108 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:48:51.119326+0000", 2024-09-10T18:49:52.108 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 60.890676339999999, 2024-09-10T18:49:52.108 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.052966654000000002, 2024-09-10T18:49:52.108 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:49:52.108 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:49:52.108 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:49:52.108 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:49:52.108 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:49:52.109 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 109528 2024-09-10T18:49:52.109 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.109 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:49:52.109 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.109 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:49:52.109 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:51.119326+0000", 2024-09-10T18:49:52.109 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.109 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.109 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.109 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:49:52.109 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:51.119326+0000", 2024-09-10T18:49:52.109 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.109 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.110 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.110 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:49:52.110 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:51.119324+0000", 2024-09-10T18:49:52.110 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:49:52.110 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.110 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.110 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:49:52.110 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:51.119455+0000", 2024-09-10T18:49:52.110 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000130572 2024-09-10T18:49:52.110 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.110 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.110 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:49:52.110 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:51.119466+0000", 2024-09-10T18:49:52.110 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0635999999999999e-05 2024-09-10T18:49:52.111 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.111 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.111 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:49:52.111 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:51.119471+0000", 2024-09-10T18:49:52.111 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.1059999999999999e-06 2024-09-10T18:49:52.111 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.111 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.111 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:49:52.111 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:51.119636+0000", 2024-09-10T18:49:52.111 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000165353 2024-09-10T18:49:52.111 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.111 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.111 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:49:52.112 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:51.119716+0000", 2024-09-10T18:49:52.112 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.9717999999999998e-05 2024-09-10T18:49:52.112 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.112 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.112 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,6", 2024-09-10T18:49:52.112 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:51.119810+0000", 2024-09-10T18:49:52.112 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.3931999999999994e-05 2024-09-10T18:49:52.112 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.112 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.112 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:49:52.112 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:51.122350+0000", 2024-09-10T18:49:52.113 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0025399709999999998 2024-09-10T18:49:52.113 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.113 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.113 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.113 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:51.122840+0000", 2024-09-10T18:49:52.113 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00049068400000000002 2024-09-10T18:49:52.113 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.113 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.113 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.113 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:51.172136+0000", 2024-09-10T18:49:52.113 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.049295881 2024-09-10T18:49:52.113 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.113 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.114 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:49:52.114 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:51.172241+0000", 2024-09-10T18:49:52.114 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00010500100000000001 2024-09-10T18:49:52.114 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.114 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.114 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:49:52.114 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:51.172292+0000", 2024-09-10T18:49:52.114 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.1112000000000001e-05 2024-09-10T18:49:52.114 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.114 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:49:52.114 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.114 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.114 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.114 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:111930 7.1 7:806f8396:::benchmark_data_smithi018_78628_object111929:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-10T18:49:52.115 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:48:53.078947+0000", 2024-09-10T18:49:52.115 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 58.931054996999997, 2024-09-10T18:49:52.115 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.041643014999999999, 2024-09-10T18:49:52.115 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:49:52.115 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:49:52.115 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:49:52.115 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:49:52.115 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:49:52.115 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 111930 2024-09-10T18:49:52.115 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.115 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:49:52.115 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.115 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:49:52.115 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:53.078947+0000", 2024-09-10T18:49:52.116 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.116 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.116 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.116 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:49:52.116 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:53.078947+0000", 2024-09-10T18:49:52.116 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.116 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.116 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.116 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:49:52.116 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:53.078946+0000", 2024-09-10T18:49:52.116 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:49:52.116 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.116 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.117 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:49:52.117 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:53.079012+0000", 2024-09-10T18:49:52.117 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.6155000000000006e-05 2024-09-10T18:49:52.117 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.117 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.117 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:49:52.117 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:53.079024+0000", 2024-09-10T18:49:52.117 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.1588e-05 2024-09-10T18:49:52.117 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.117 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.117 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:49:52.117 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:53.079028+0000", 2024-09-10T18:49:52.117 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.7199999999999997e-06 2024-09-10T18:49:52.118 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.118 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.118 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:49:52.118 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:53.079078+0000", 2024-09-10T18:49:52.118 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.9150000000000002e-05 2024-09-10T18:49:52.118 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.118 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.118 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:49:52.118 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:53.079210+0000", 2024-09-10T18:49:52.118 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000132068 2024-09-10T18:49:52.118 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.118 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.118 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,6", 2024-09-10T18:49:52.119 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:53.079300+0000", 2024-09-10T18:49:52.119 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.0643999999999995e-05 2024-09-10T18:49:52.119 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.119 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.119 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:49:52.119 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:53.081466+0000", 2024-09-10T18:49:52.119 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.002165695 2024-09-10T18:49:52.119 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.119 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.119 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.119 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:53.082866+0000", 2024-09-10T18:49:52.119 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0014003170000000001 2024-09-10T18:49:52.119 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.119 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.120 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.120 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:53.120492+0000", 2024-09-10T18:49:52.120 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.037625909999999999 2024-09-10T18:49:52.120 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.120 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.120 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:49:52.120 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:53.120545+0000", 2024-09-10T18:49:52.120 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.2485999999999999e-05 2024-09-10T18:49:52.120 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.120 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.120 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:49:52.120 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:53.120590+0000", 2024-09-10T18:49:52.121 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.5442999999999998e-05 2024-09-10T18:49:52.121 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.121 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:49:52.121 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.121 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.121 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.121 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:116262 7.1 7:8e191ede:::benchmark_data_smithi018_78628_object116261:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-10T18:49:52.121 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:48:56.575527+0000", 2024-09-10T18:49:52.121 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 55.434474772999998, 2024-09-10T18:49:52.121 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.083387096999999993, 2024-09-10T18:49:52.121 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:49:52.121 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:49:52.121 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:49:52.122 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:49:52.122 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:49:52.122 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 116262 2024-09-10T18:49:52.122 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.122 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:49:52.122 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.122 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:49:52.122 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:56.575527+0000", 2024-09-10T18:49:52.123 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.123 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.123 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.123 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:49:52.123 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:56.575527+0000", 2024-09-10T18:49:52.124 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.124 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.124 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.124 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:49:52.124 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:56.575526+0000", 2024-09-10T18:49:52.124 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:49:52.124 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.124 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.124 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:49:52.124 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:56.575582+0000", 2024-09-10T18:49:52.124 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.5711999999999998e-05 2024-09-10T18:49:52.124 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.124 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.125 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:49:52.125 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:56.575592+0000", 2024-09-10T18:49:52.125 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.6909999999999992e-06 2024-09-10T18:49:52.125 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.125 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.125 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:49:52.125 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:56.575596+0000", 2024-09-10T18:49:52.125 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.138e-06 2024-09-10T18:49:52.125 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.125 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.125 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:49:52.125 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:56.575635+0000", 2024-09-10T18:49:52.125 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.8787e-05 2024-09-10T18:49:52.126 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.126 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.126 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:49:52.126 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:56.575746+0000", 2024-09-10T18:49:52.126 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00011103300000000001 2024-09-10T18:49:52.126 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.126 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.126 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,6", 2024-09-10T18:49:52.126 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:56.575883+0000", 2024-09-10T18:49:52.126 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000137441 2024-09-10T18:49:52.126 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.126 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.126 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:49:52.127 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:56.578159+0000", 2024-09-10T18:49:52.127 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0022763589999999999 2024-09-10T18:49:52.127 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.127 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.127 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.127 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:56.578185+0000", 2024-09-10T18:49:52.127 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.5785000000000001e-05 2024-09-10T18:49:52.127 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.127 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.127 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.127 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:56.658806+0000", 2024-09-10T18:49:52.127 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.080620948999999997 2024-09-10T18:49:52.127 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.128 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.128 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:49:52.128 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:56.658874+0000", 2024-09-10T18:49:52.128 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.7411999999999998e-05 2024-09-10T18:49:52.128 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.128 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.128 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:49:52.128 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:56.658914+0000", 2024-09-10T18:49:52.128 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.0772999999999997e-05 2024-09-10T18:49:52.128 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.128 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:49:52.128 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.128 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.129 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.129 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:118717 7.1 7:8e012ec0:::benchmark_data_smithi018_78628_object118716:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-10T18:49:52.129 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:48:58.992713+0000", 2024-09-10T18:49:52.129 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 53.017289017000003, 2024-09-10T18:49:52.129 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.053185331000000002, 2024-09-10T18:49:52.129 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:49:52.129 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:49:52.129 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:49:52.129 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:49:52.129 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:49:52.129 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 118717 2024-09-10T18:49:52.129 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.129 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:49:52.130 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.130 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:49:52.130 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:58.992713+0000", 2024-09-10T18:49:52.130 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.130 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.130 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.130 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:49:52.130 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:58.992713+0000", 2024-09-10T18:49:52.130 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.130 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.130 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.130 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:49:52.130 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:58.992712+0000", 2024-09-10T18:49:52.131 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:49:52.131 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.131 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.131 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:49:52.131 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:58.992783+0000", 2024-09-10T18:49:52.131 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.1607999999999996e-05 2024-09-10T18:49:52.131 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.131 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.131 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:49:52.131 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:58.992796+0000", 2024-09-10T18:49:52.131 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.3142999999999999e-05 2024-09-10T18:49:52.131 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.131 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.132 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:49:52.132 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:58.992802+0000", 2024-09-10T18:49:52.132 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.9429999999999997e-06 2024-09-10T18:49:52.132 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.132 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.132 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:49:52.132 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:58.992854+0000", 2024-09-10T18:49:52.132 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.1653999999999997e-05 2024-09-10T18:49:52.132 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.132 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.132 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:49:52.132 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:58.992940+0000", 2024-09-10T18:49:52.133 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.6401000000000002e-05 2024-09-10T18:49:52.133 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.133 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.133 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,6", 2024-09-10T18:49:52.133 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:58.993026+0000", 2024-09-10T18:49:52.133 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.5327999999999993e-05 2024-09-10T18:49:52.133 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.133 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.133 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:49:52.133 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:58.995137+0000", 2024-09-10T18:49:52.133 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0021115909999999999 2024-09-10T18:49:52.133 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.133 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.134 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.134 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:59.002761+0000", 2024-09-10T18:49:52.134 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0076238620000000003 2024-09-10T18:49:52.134 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.134 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.134 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.134 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:59.045784+0000", 2024-09-10T18:49:52.134 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.043023036000000001 2024-09-10T18:49:52.134 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.134 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.134 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:49:52.134 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:59.045859+0000", 2024-09-10T18:49:52.134 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.4488999999999998e-05 2024-09-10T18:49:52.135 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.135 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.135 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:49:52.135 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:59.045898+0000", 2024-09-10T18:49:52.135 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.9677e-05 2024-09-10T18:49:52.135 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.135 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:49:52.135 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.135 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.135 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.135 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:118725 7.1 7:87c4908a:::benchmark_data_smithi018_78628_object118724:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-10T18:49:52.135 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:48:59.004548+0000", 2024-09-10T18:49:52.135 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 53.005453711000001, 2024-09-10T18:49:52.136 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.041458024000000003, 2024-09-10T18:49:52.136 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:49:52.136 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:49:52.136 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:49:52.136 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:49:52.136 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:49:52.136 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 118725 2024-09-10T18:49:52.136 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.136 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:49:52.136 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.136 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:49:52.136 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:59.004548+0000", 2024-09-10T18:49:52.136 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.136 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.137 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.137 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:49:52.137 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:59.004548+0000", 2024-09-10T18:49:52.137 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.137 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.137 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.137 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:49:52.137 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:59.004547+0000", 2024-09-10T18:49:52.137 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:49:52.137 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.137 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.137 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:49:52.137 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:59.004614+0000", 2024-09-10T18:49:52.137 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.7021999999999996e-05 2024-09-10T18:49:52.138 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.138 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.138 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:49:52.138 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:59.004627+0000", 2024-09-10T18:49:52.138 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.2777e-05 2024-09-10T18:49:52.138 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.138 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.138 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:49:52.138 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:59.004632+0000", 2024-09-10T18:49:52.138 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.5110000000000003e-06 2024-09-10T18:49:52.138 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.138 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.138 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:49:52.139 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:59.004672+0000", 2024-09-10T18:49:52.139 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.9480000000000001e-05 2024-09-10T18:49:52.139 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.139 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.139 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:49:52.139 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:59.004801+0000", 2024-09-10T18:49:52.139 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000129076 2024-09-10T18:49:52.139 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.139 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.139 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,6", 2024-09-10T18:49:52.139 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:59.004907+0000", 2024-09-10T18:49:52.139 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000106127 2024-09-10T18:49:52.139 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.140 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.140 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:49:52.140 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:59.008265+0000", 2024-09-10T18:49:52.140 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0033575060000000001 2024-09-10T18:49:52.140 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.140 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.140 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.140 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:59.012543+0000", 2024-09-10T18:49:52.140 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0042780989999999996 2024-09-10T18:49:52.140 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.140 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.140 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.140 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:59.045936+0000", 2024-09-10T18:49:52.140 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.033393052999999999 2024-09-10T18:49:52.141 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.141 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.141 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:49:52.141 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:59.045977+0000", 2024-09-10T18:49:52.141 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.1619000000000003e-05 2024-09-10T18:49:52.141 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.141 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.141 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:49:52.141 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:59.046006+0000", 2024-09-10T18:49:52.141 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.8929e-05 2024-09-10T18:49:52.141 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.141 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:49:52.141 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.142 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.142 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.142 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:118726 7.1 7:8b070f19:::benchmark_data_smithi018_78628_object118725:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-10T18:49:52.142 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:48:59.005004+0000", 2024-09-10T18:49:52.142 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 53.004997768999999, 2024-09-10T18:49:52.142 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.041101054999999997, 2024-09-10T18:49:52.142 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:49:52.142 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:49:52.142 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:49:52.142 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:49:52.142 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:49:52.142 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 118726 2024-09-10T18:49:52.142 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.143 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:49:52.143 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.143 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:49:52.143 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:59.005004+0000", 2024-09-10T18:49:52.143 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.143 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.143 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.143 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:49:52.143 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:59.005004+0000", 2024-09-10T18:49:52.143 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.143 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.143 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.143 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:49:52.144 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:59.005003+0000", 2024-09-10T18:49:52.144 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:49:52.144 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.144 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.144 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:49:52.144 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:59.005054+0000", 2024-09-10T18:49:52.144 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.0538999999999998e-05 2024-09-10T18:49:52.144 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.144 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.144 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:49:52.144 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:59.005065+0000", 2024-09-10T18:49:52.144 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.1469e-05 2024-09-10T18:49:52.144 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.145 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.145 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:49:52.145 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:59.005070+0000", 2024-09-10T18:49:52.145 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.189e-06 2024-09-10T18:49:52.145 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.145 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.145 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:49:52.145 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:59.006781+0000", 2024-09-10T18:49:52.145 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.001711569 2024-09-10T18:49:52.145 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.145 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.145 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:49:52.145 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:59.006863+0000", 2024-09-10T18:49:52.146 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.1346000000000003e-05 2024-09-10T18:49:52.146 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.146 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.146 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,6", 2024-09-10T18:49:52.146 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:59.006947+0000", 2024-09-10T18:49:52.146 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.4791999999999997e-05 2024-09-10T18:49:52.146 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.146 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.146 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:49:52.146 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:59.008379+0000", 2024-09-10T18:49:52.146 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0014312719999999999 2024-09-10T18:49:52.146 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.146 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.147 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.147 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:59.012565+0000", 2024-09-10T18:49:52.147 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0041866660000000003 2024-09-10T18:49:52.147 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.147 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.147 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.147 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:59.046037+0000", 2024-09-10T18:49:52.147 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.033471393000000002 2024-09-10T18:49:52.147 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.147 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.147 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:49:52.147 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:59.046078+0000", 2024-09-10T18:49:52.147 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.1028000000000001e-05 2024-09-10T18:49:52.148 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.148 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.148 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:49:52.148 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:59.046105+0000", 2024-09-10T18:49:52.148 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.7662e-05 2024-09-10T18:49:52.148 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.148 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:49:52.148 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.148 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.148 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.148 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:148312 7.1 7:8f7bbe01:::benchmark_data_smithi018_78628_object12092:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-10T18:49:52.148 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:49:25.514072+0000", 2024-09-10T18:49:52.148 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 26.495930029, 2024-09-10T18:49:52.149 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.038253064000000003, 2024-09-10T18:49:52.149 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:49:52.149 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:49:52.149 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:49:52.149 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:49:52.149 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:49:52.149 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 148312 2024-09-10T18:49:52.149 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.149 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:49:52.149 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.149 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:49:52.149 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:49:25.514072+0000", 2024-09-10T18:49:52.149 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.149 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.150 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.150 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:49:52.150 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:49:25.514072+0000", 2024-09-10T18:49:52.150 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:49:52.150 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.150 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.150 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:49:52.150 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:49:25.514071+0000", 2024-09-10T18:49:52.150 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:49:52.150 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.150 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.150 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:49:52.150 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:49:25.514076+0000", 2024-09-10T18:49:52.150 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.0080000000000002e-06 2024-09-10T18:49:52.151 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.151 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.151 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:49:52.151 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:49:25.514090+0000", 2024-09-10T18:49:52.151 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.4168000000000001e-05 2024-09-10T18:49:52.151 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.151 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.151 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:49:52.151 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:49:25.514096+0000", 2024-09-10T18:49:52.151 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.4539999999999998e-06 2024-09-10T18:49:52.151 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.151 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.151 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:49:52.152 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:49:25.546062+0000", 2024-09-10T18:49:52.152 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.031966429999999997 2024-09-10T18:49:52.152 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.152 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.152 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:49:52.152 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:49:25.546148+0000", 2024-09-10T18:49:52.152 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.5974000000000001e-05 2024-09-10T18:49:52.152 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.152 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.152 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,6", 2024-09-10T18:49:52.152 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:49:25.546267+0000", 2024-09-10T18:49:52.152 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000118909 2024-09-10T18:49:52.152 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.153 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.153 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.153 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:49:25.552184+0000", 2024-09-10T18:49:52.153 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0059169979999999997 2024-09-10T18:49:52.153 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.153 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.153 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:49:52.153 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:49:25.552191+0000", 2024-09-10T18:49:52.153 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.5579999999999997e-06 2024-09-10T18:49:52.153 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.153 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.153 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:49:52.153 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:49:25.552234+0000", 2024-09-10T18:49:52.153 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.3309000000000002e-05 2024-09-10T18:49:52.154 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.154 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.154 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:49:52.154 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:49:25.552280+0000", 2024-09-10T18:49:52.154 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.6338999999999998e-05 2024-09-10T18:49:52.154 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:49:52.154 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:49:52.154 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:49:52.154 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:49:25.552325+0000", 2024-09-10T18:49:52.154 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.4818999999999999e-05 2024-09-10T18:49:52.154 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.154 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:49:52.154 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.154 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:49:52.155 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:49:52.155 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T18:49:52.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:52.156 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:52.157 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:52.257 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:52.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:52 smithi103 ceph-mon[33163]: pgmap v175: 148 pgs: 148 active+clean; 6.0 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 1.59k op/s 2024-09-10T18:49:52.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:52 smithi103 ceph-mon[33163]: 2.8 scrub starts 2024-09-10T18:49:52.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:52 smithi103 ceph-mon[33163]: 2.8 scrub ok 2024-09-10T18:49:52.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:52 smithi103 ceph-mon[33163]: 5.1d scrub starts 2024-09-10T18:49:52.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:52 smithi103 ceph-mon[33163]: 5.1d scrub ok 2024-09-10T18:49:52.296 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:52.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:52.297 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:52.329 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-10T18:49:52.379 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:52 smithi018 ceph-mon[29336]: pgmap v175: 148 pgs: 148 active+clean; 6.0 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 1.59k op/s 2024-09-10T18:49:52.379 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:52 smithi018 ceph-mon[29336]: 2.8 scrub starts 2024-09-10T18:49:52.379 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:52 smithi018 ceph-mon[29336]: 2.8 scrub ok 2024-09-10T18:49:52.379 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:52 smithi018 ceph-mon[29336]: 5.1d scrub starts 2024-09-10T18:49:52.379 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:52 smithi018 ceph-mon[29336]: 5.1d scrub ok 2024-09-10T18:49:52.380 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:52 smithi018 ceph-mon[40196]: pgmap v175: 148 pgs: 148 active+clean; 6.0 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 1.59k op/s 2024-09-10T18:49:52.380 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:52 smithi018 ceph-mon[40196]: 2.8 scrub starts 2024-09-10T18:49:52.380 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:52 smithi018 ceph-mon[40196]: 2.8 scrub ok 2024-09-10T18:49:52.380 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:52 smithi018 ceph-mon[40196]: 5.1d scrub starts 2024-09-10T18:49:52.380 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:52 smithi018 ceph-mon[40196]: 5.1d scrub ok 2024-09-10T18:49:52.398 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:52.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:52.447 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:52.447 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:52.548 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:52.579 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:52.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:52.580 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:52.681 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:52.733 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:52.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:52.734 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:52.835 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:52.865 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:52.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:52.866 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:52.968 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:52.997 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:52.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:52.998 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:53.099 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:53.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:53.139 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:53.139 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:53.240 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:53.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:53.282 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:53.282 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:53.378 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-10T18:49:53.383 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:53.457 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:53.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:53.458 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:53.510 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:53 smithi018 ceph-mon[29336]: 6.1f scrub starts 2024-09-10T18:49:53.510 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:53 smithi018 ceph-mon[29336]: 6.1f scrub ok 2024-09-10T18:49:53.510 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:53 smithi018 ceph-mon[29336]: 7.9 scrub starts 2024-09-10T18:49:53.510 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:53 smithi018 ceph-mon[29336]: 4.8 scrub starts 2024-09-10T18:49:53.510 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:53 smithi018 ceph-mon[29336]: 4.8 scrub ok 2024-09-10T18:49:53.510 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:53 smithi018 ceph-mon[29336]: 6.1c scrub starts 2024-09-10T18:49:53.510 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:53 smithi018 ceph-mon[29336]: 6.1c scrub ok 2024-09-10T18:49:53.511 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:53 smithi018 ceph-mon[29336]: 5.19 scrub starts 2024-09-10T18:49:53.511 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:53 smithi018 ceph-mon[29336]: 5.19 scrub ok 2024-09-10T18:49:53.511 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:53 smithi018 ceph-mon[40196]: 6.1f scrub starts 2024-09-10T18:49:53.511 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:53 smithi018 ceph-mon[40196]: 6.1f scrub ok 2024-09-10T18:49:53.511 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:53 smithi018 ceph-mon[40196]: 7.9 scrub starts 2024-09-10T18:49:53.511 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:53 smithi018 ceph-mon[40196]: 4.8 scrub starts 2024-09-10T18:49:53.511 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:53 smithi018 ceph-mon[40196]: 4.8 scrub ok 2024-09-10T18:49:53.511 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:53 smithi018 ceph-mon[40196]: 6.1c scrub starts 2024-09-10T18:49:53.511 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:53 smithi018 ceph-mon[40196]: 6.1c scrub ok 2024-09-10T18:49:53.511 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:53 smithi018 ceph-mon[40196]: 5.19 scrub starts 2024-09-10T18:49:53.511 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:53 smithi018 ceph-mon[40196]: 5.19 scrub ok 2024-09-10T18:49:53.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:53 smithi103 ceph-mon[33163]: 6.1f scrub starts 2024-09-10T18:49:53.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:53 smithi103 ceph-mon[33163]: 6.1f scrub ok 2024-09-10T18:49:53.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:53 smithi103 ceph-mon[33163]: 7.9 scrub starts 2024-09-10T18:49:53.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:53 smithi103 ceph-mon[33163]: 4.8 scrub starts 2024-09-10T18:49:53.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:53 smithi103 ceph-mon[33163]: 4.8 scrub ok 2024-09-10T18:49:53.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:53 smithi103 ceph-mon[33163]: 6.1c scrub starts 2024-09-10T18:49:53.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:53 smithi103 ceph-mon[33163]: 6.1c scrub ok 2024-09-10T18:49:53.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:53 smithi103 ceph-mon[33163]: 5.19 scrub starts 2024-09-10T18:49:53.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:53 smithi103 ceph-mon[33163]: 5.19 scrub ok 2024-09-10T18:49:53.559 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:53.599 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:53.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:53.600 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:53.700 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:49:53.701 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:49:53.701 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:53.732 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:49:53.737 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:49:53.737 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:49:53.755 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:53.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:53.756 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:53.766 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:49:53.767 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:49:53.802 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:49:53.803 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:49:53.833 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:49:53.833 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:49:53.858 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:53.871 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:49:53.871 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:49:53.888 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:53.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:53.889 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:53.909 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:49:53.909 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:49:53.941 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:49:53.942 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:49:53.990 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:54.028 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:54.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:54.029 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:54.130 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:54.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:54.172 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:54.173 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:54.274 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:54.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:54.302 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:54.303 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:54.403 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:54.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:54.440 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:54.440 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:54.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:54 smithi018 ceph-mon[40196]: 6.b scrub starts 2024-09-10T18:49:54.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:54 smithi018 ceph-mon[40196]: 6.b scrub ok 2024-09-10T18:49:54.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:54 smithi018 ceph-mon[40196]: 7.e scrub starts 2024-09-10T18:49:54.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:54 smithi018 ceph-mon[40196]: pgmap v176: 148 pgs: 148 active+clean; 5.8 GiB data, 19 GiB used, 606 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-10T18:49:54.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:54 smithi018 ceph-mon[40196]: 2.1d scrub starts 2024-09-10T18:49:54.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:54 smithi018 ceph-mon[40196]: 2.1d scrub ok 2024-09-10T18:49:54.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:54 smithi018 ceph-mon[40196]: 6.1a scrub starts 2024-09-10T18:49:54.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:54 smithi018 ceph-mon[40196]: 6.1a scrub ok 2024-09-10T18:49:54.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:54 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1325918817' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:49:54.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:54 smithi018 ceph-mon[29336]: 6.b scrub starts 2024-09-10T18:49:54.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:54 smithi018 ceph-mon[29336]: 6.b scrub ok 2024-09-10T18:49:54.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:54 smithi018 ceph-mon[29336]: 7.e scrub starts 2024-09-10T18:49:54.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:54 smithi018 ceph-mon[29336]: pgmap v176: 148 pgs: 148 active+clean; 5.8 GiB data, 19 GiB used, 606 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-10T18:49:54.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:54 smithi018 ceph-mon[29336]: 2.1d scrub starts 2024-09-10T18:49:54.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:54 smithi018 ceph-mon[29336]: 2.1d scrub ok 2024-09-10T18:49:54.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:54 smithi018 ceph-mon[29336]: 6.1a scrub starts 2024-09-10T18:49:54.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:54 smithi018 ceph-mon[29336]: 6.1a scrub ok 2024-09-10T18:49:54.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:54 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1325918817' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:49:54.523 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:49:54.526 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-09-10T18:49:54.527 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:54 smithi103 ceph-mon[33163]: 6.b scrub starts 2024-09-10T18:49:54.527 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:54 smithi103 ceph-mon[33163]: 6.b scrub ok 2024-09-10T18:49:54.527 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:54 smithi103 ceph-mon[33163]: 7.e scrub starts 2024-09-10T18:49:54.527 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:54 smithi103 ceph-mon[33163]: pgmap v176: 148 pgs: 148 active+clean; 5.8 GiB data, 19 GiB used, 606 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-10T18:49:54.527 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:54 smithi103 ceph-mon[33163]: 2.1d scrub starts 2024-09-10T18:49:54.527 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:54 smithi103 ceph-mon[33163]: 2.1d scrub ok 2024-09-10T18:49:54.527 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:54 smithi103 ceph-mon[33163]: 6.1a scrub starts 2024-09-10T18:49:54.527 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:54 smithi103 ceph-mon[33163]: 6.1a scrub ok 2024-09-10T18:49:54.527 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:54 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1325918817' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:49:54.541 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:54.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:54.576 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:54.577 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:54.677 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:54.712 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:54.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:54.713 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:54.814 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:54.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:54.841 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:54.841 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:54.942 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:54.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:54.975 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:54.976 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:55.076 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:55.123 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:55.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:55.124 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:55.225 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:55.245 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:55.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:55.246 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:55.347 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:55.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:55.374 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:55.374 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:55.475 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:55.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:55.497 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:55.497 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:55.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:55 smithi103 ceph-mon[33163]: 7.0 scrub starts 2024-09-10T18:49:55.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:55 smithi103 ceph-mon[33163]: 7.9 scrub ok 2024-09-10T18:49:55.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:55 smithi103 ceph-mon[33163]: 5.d scrub starts 2024-09-10T18:49:55.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:55 smithi103 ceph-mon[33163]: 5.d scrub ok 2024-09-10T18:49:55.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:49:55.598 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:55.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:55.637 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:55.637 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:55.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:55 smithi018 ceph-mon[29336]: 7.0 scrub starts 2024-09-10T18:49:55.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:55 smithi018 ceph-mon[29336]: 7.9 scrub ok 2024-09-10T18:49:55.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:55 smithi018 ceph-mon[29336]: 5.d scrub starts 2024-09-10T18:49:55.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:55 smithi018 ceph-mon[29336]: 5.d scrub ok 2024-09-10T18:49:55.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:49:55.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:55 smithi018 ceph-mon[40196]: 7.0 scrub starts 2024-09-10T18:49:55.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:55 smithi018 ceph-mon[40196]: 7.9 scrub ok 2024-09-10T18:49:55.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:55 smithi018 ceph-mon[40196]: 5.d scrub starts 2024-09-10T18:49:55.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:55 smithi018 ceph-mon[40196]: 5.d scrub ok 2024-09-10T18:49:55.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:49:55.739 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:55.781 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:55.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:55.782 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:55.882 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:55.912 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:55.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:55.913 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:56.014 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:56.044 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:56.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:56.045 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:56.146 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:56.175 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:56.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:56.177 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:56.277 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:56.307 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:56.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:56.308 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:56.364 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:56 smithi018 ceph-mon[40196]: from='client.24748 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:49:56.364 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:56 smithi018 ceph-mon[40196]: 6.9 scrub starts 2024-09-10T18:49:56.364 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:56 smithi018 ceph-mon[40196]: 6.9 scrub ok 2024-09-10T18:49:56.364 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:56 smithi018 ceph-mon[40196]: pgmap v177: 148 pgs: 148 active+clean; 5.8 GiB data, 19 GiB used, 606 GiB / 626 GiB avail; 649 B/s rd, 0 B/s wr, 825 op/s 2024-09-10T18:49:56.364 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:56 smithi018 ceph-mon[40196]: 7.e scrub ok 2024-09-10T18:49:56.364 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:56 smithi018 ceph-mon[29336]: from='client.24748 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:49:56.364 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:56 smithi018 ceph-mon[29336]: 6.9 scrub starts 2024-09-10T18:49:56.364 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:56 smithi018 ceph-mon[29336]: 6.9 scrub ok 2024-09-10T18:49:56.365 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:56 smithi018 ceph-mon[29336]: pgmap v177: 148 pgs: 148 active+clean; 5.8 GiB data, 19 GiB used, 606 GiB / 626 GiB avail; 649 B/s rd, 0 B/s wr, 825 op/s 2024-09-10T18:49:56.365 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:56 smithi018 ceph-mon[29336]: 7.e scrub ok 2024-09-10T18:49:56.409 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:56.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:56.435 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:56.435 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:56.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:56 smithi103 ceph-mon[33163]: from='client.24748 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:49:56.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:56 smithi103 ceph-mon[33163]: 6.9 scrub starts 2024-09-10T18:49:56.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:56 smithi103 ceph-mon[33163]: 6.9 scrub ok 2024-09-10T18:49:56.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:56 smithi103 ceph-mon[33163]: pgmap v177: 148 pgs: 148 active+clean; 5.8 GiB data, 19 GiB used, 606 GiB / 626 GiB avail; 649 B/s rd, 0 B/s wr, 825 op/s 2024-09-10T18:49:56.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:56 smithi103 ceph-mon[33163]: 7.e scrub ok 2024-09-10T18:49:56.523 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:49:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:49:56.536 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:56.571 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:56.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:56.573 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:56.580 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:49:56.674 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:56.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:56.709 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:56.710 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:56.810 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:56.842 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:56.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:56.843 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:56.944 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:49:56.985 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:56.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:56.987 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:49:57.087 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:57.116 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:57.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:57.119 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:57.220 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:49:57.248 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:57.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:57.250 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:49:57.350 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:57.376 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":176,"stamp":"2024-09-10T18:49:53.095195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6186866932,"num_objects":94592,"num_object_clones":0,"num_object_copies":283776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94592,"num_whiteouts":0,"num_read":1116,"num_read_kb":973,"num_write":178583,"num_write_kb":8719560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":1183107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77590,"ondisk_log_size":77590,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":20432700,"kb_used_data":18284516,"kb_used_omap":11,"kb_used_meta":2147956,"kb_avail":635783364,"statfs":{"total":671965249536,"available":651042164736,"internally_reserved":0,"allocated":18723344384,"data_stored":18720478751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11928,"internal_metadata":2199507304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,6,6,11,7,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-426246144,"num_objects":-6504,"num_object_clones":0,"num_object_copies":-19512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6504,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":6504,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002672"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":91,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.236092+0000","last_change":"2024-09-10T18:48:24.515065+0000","last_active":"2024-09-10T18:49:48.236092+0000","last_peered":"2024-09-10T18:49:48.236092+0000","last_clean":"2024-09-10T18:49:48.236092+0000","last_became_active":"2024-09-10T18:45:24.265488+0000","last_became_peered":"2024-09-10T18:45:24.265488+0000","last_unstale":"2024-09-10T18:49:48.236092+0000","last_undegraded":"2024-09-10T18:49:48.236092+0000","last_fullsized":"2024-09-10T18:49:48.236092+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:24.511946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:24.511946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:49.630089+0000","scrub_duration":0.014161415,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":97,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.229677+0000","last_change":"2024-09-10T18:48:29.718356+0000","last_active":"2024-09-10T18:49:48.229677+0000","last_peered":"2024-09-10T18:49:48.229677+0000","last_clean":"2024-09-10T18:49:48.229677+0000","last_became_active":"2024-09-10T18:45:17.959475+0000","last_became_peered":"2024-09-10T18:45:17.959475+0000","last_unstale":"2024-09-10T18:49:48.229677+0000","last_undegraded":"2024-09-10T18:49:48.229677+0000","last_fullsized":"2024-09-10T18:49:48.229677+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:29.718270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.718270+0000","last_clean_scrub_stamp":"2024-09-10T18:48:29.718270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:59.632977+0000","scrub_duration":0.010834984000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"63'15","reported_seq":146,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.235856+0000","last_change":"2024-09-10T18:49:02.544635+0000","last_active":"2024-09-10T18:49:48.235856+0000","last_peered":"2024-09-10T18:49:48.235856+0000","last_clean":"2024-09-10T18:49:48.235856+0000","last_became_active":"2024-09-10T18:47:36.413635+0000","last_became_peered":"2024-09-10T18:47:36.413635+0000","last_unstale":"2024-09-10T18:49:48.235856+0000","last_undegraded":"2024-09-10T18:49:48.235856+0000","last_fullsized":"2024-09-10T18:49:48.235856+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:49:02.544557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:49:02.544557+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:20.745798+0000","scrub_duration":0.0033455720000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":92,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.227617+0000","last_change":"2024-09-10T18:48:29.058001+0000","last_active":"2024-09-10T18:49:48.227617+0000","last_peered":"2024-09-10T18:49:48.227617+0000","last_clean":"2024-09-10T18:49:48.227617+0000","last_became_active":"2024-09-10T18:45:22.065619+0000","last_became_peered":"2024-09-10T18:45:22.065619+0000","last_unstale":"2024-09-10T18:49:48.227617+0000","last_undegraded":"2024-09-10T18:49:48.227617+0000","last_fullsized":"2024-09-10T18:49:48.227617+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:29.057932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.057932+0000","last_clean_scrub_stamp":"2024-09-10T18:48:29.057932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001460184,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":97,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.237494+0000","last_change":"2024-09-10T18:48:29.458396+0000","last_active":"2024-09-10T18:49:48.237494+0000","last_peered":"2024-09-10T18:49:48.237494+0000","last_clean":"2024-09-10T18:49:48.237494+0000","last_became_active":"2024-09-10T18:45:17.966153+0000","last_became_peered":"2024-09-10T18:45:17.966153+0000","last_unstale":"2024-09-10T18:49:48.237494+0000","last_undegraded":"2024-09-10T18:49:48.237494+0000","last_fullsized":"2024-09-10T18:49:48.237494+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:29.457892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:29.457892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:50.349265+0000","scrub_duration":0.012091342999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"83'13","reported_seq":104,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.227747+0000","last_change":"2024-09-10T18:46:59.906707+0000","last_active":"2024-09-10T18:49:48.227747+0000","last_peered":"2024-09-10T18:49:48.227747+0000","last_clean":"2024-09-10T18:49:48.227747+0000","last_became_active":"2024-09-10T18:45:20.061266+0000","last_became_peered":"2024-09-10T18:45:20.061266+0000","last_unstale":"2024-09-10T18:49:48.227747+0000","last_undegraded":"2024-09-10T18:49:48.227747+0000","last_fullsized":"2024-09-10T18:49:48.227747+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'13","last_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_clean_scrub_stamp":"2024-09-10T18:46:59.906634+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003988892,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":83,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.233482+0000","last_change":"2024-09-10T18:47:21.589428+0000","last_active":"2024-09-10T18:49:48.233482+0000","last_peered":"2024-09-10T18:49:48.233482+0000","last_clean":"2024-09-10T18:49:48.233482+0000","last_became_active":"2024-09-10T18:45:22.072822+0000","last_became_peered":"2024-09-10T18:45:22.072822+0000","last_unstale":"2024-09-10T18:49:48.233482+0000","last_undegraded":"2024-09-10T18:49:48.233482+0000","last_fullsized":"2024-09-10T18:49:48.233482+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:21.589365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:47:21.589365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001676893,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":80,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.227736+0000","last_change":"2024-09-10T18:47:00.946309+0000","last_active":"2024-09-10T18:49:48.227736+0000","last_peered":"2024-09-10T18:49:48.227736+0000","last_clean":"2024-09-10T18:49:48.227736+0000","last_became_active":"2024-09-10T18:45:24.081002+0000","last_became_peered":"2024-09-10T18:45:24.081002+0000","last_unstale":"2024-09-10T18:49:48.227736+0000","last_undegraded":"2024-09-10T18:49:48.227736+0000","last_fullsized":"2024-09-10T18:49:48.227736+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:00.946255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:00.946255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001207629,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":97,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.233268+0000","last_change":"2024-09-10T18:48:13.694567+0000","last_active":"2024-09-10T18:49:48.233268+0000","last_peered":"2024-09-10T18:49:48.233268+0000","last_clean":"2024-09-10T18:49:48.233268+0000","last_became_active":"2024-09-10T18:45:17.961097+0000","last_became_peered":"2024-09-10T18:45:17.961097+0000","last_unstale":"2024-09-10T18:49:48.233268+0000","last_undegraded":"2024-09-10T18:49:48.233268+0000","last_fullsized":"2024-09-10T18:49:48.233268+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:13.694295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:13.694295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051419839999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"63'5","reported_seq":94,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.227679+0000","last_change":"2024-09-10T18:47:04.002228+0000","last_active":"2024-09-10T18:49:48.227679+0000","last_peered":"2024-09-10T18:49:48.227679+0000","last_clean":"2024-09-10T18:49:48.227679+0000","last_became_active":"2024-09-10T18:45:20.057567+0000","last_became_peered":"2024-09-10T18:45:20.057567+0000","last_unstale":"2024-09-10T18:49:48.227679+0000","last_undegraded":"2024-09-10T18:49:48.227679+0000","last_fullsized":"2024-09-10T18:49:48.227679+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-10T18:47:04.002164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:04.002164+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.026374999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":103,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.232200+0000","last_change":"2024-09-10T18:49:19.568881+0000","last_active":"2024-09-10T18:49:48.232200+0000","last_peered":"2024-09-10T18:49:48.232200+0000","last_clean":"2024-09-10T18:49:48.232200+0000","last_became_active":"2024-09-10T18:45:22.076575+0000","last_became_peered":"2024-09-10T18:45:22.076575+0000","last_unstale":"2024-09-10T18:49:48.232200+0000","last_undegraded":"2024-09-10T18:49:48.232200+0000","last_fullsized":"2024-09-10T18:49:48.232200+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:19.568791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:36.509971+0000","last_clean_scrub_stamp":"2024-09-10T18:49:19.568791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:31.609022+0000","scrub_duration":0.0013364760000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":90,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.232915+0000","last_change":"2024-09-10T18:48:10.102991+0000","last_active":"2024-09-10T18:49:48.232915+0000","last_peered":"2024-09-10T18:49:48.232915+0000","last_clean":"2024-09-10T18:49:48.232915+0000","last_became_active":"2024-09-10T18:45:24.079534+0000","last_became_peered":"2024-09-10T18:45:24.079534+0000","last_unstale":"2024-09-10T18:49:48.232915+0000","last_undegraded":"2024-09-10T18:49:48.232915+0000","last_fullsized":"2024-09-10T18:49:48.232915+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:10.102865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:10.102865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0042821509999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":107,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.234097+0000","last_change":"2024-09-10T18:48:59.577187+0000","last_active":"2024-09-10T18:49:48.234097+0000","last_peered":"2024-09-10T18:49:48.234097+0000","last_clean":"2024-09-10T18:49:48.234097+0000","last_became_active":"2024-09-10T18:47:36.414393+0000","last_became_peered":"2024-09-10T18:47:36.414393+0000","last_unstale":"2024-09-10T18:49:48.234097+0000","last_undegraded":"2024-09-10T18:49:48.234097+0000","last_fullsized":"2024-09-10T18:49:48.234097+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:59.576980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:59.576980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:16.200779+0000","scrub_duration":0.0056458419999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"63'9","reported_seq":110,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.227965+0000","last_change":"2024-09-10T18:49:23.482901+0000","last_active":"2024-09-10T18:49:48.227965+0000","last_peered":"2024-09-10T18:49:48.227965+0000","last_clean":"2024-09-10T18:49:48.227965+0000","last_became_active":"2024-09-10T18:45:20.052652+0000","last_became_peered":"2024-09-10T18:45:20.052652+0000","last_unstale":"2024-09-10T18:49:48.227965+0000","last_undegraded":"2024-09-10T18:49:48.227965+0000","last_fullsized":"2024-09-10T18:49:48.227965+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:49:23.482807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:49:23.482807+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:23.624215+0000","scrub_duration":0.014419628,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":92,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.233239+0000","last_change":"2024-09-10T18:48:32.141849+0000","last_active":"2024-09-10T18:49:48.233239+0000","last_peered":"2024-09-10T18:49:48.233239+0000","last_clean":"2024-09-10T18:49:48.233239+0000","last_became_active":"2024-09-10T18:45:22.068326+0000","last_became_peered":"2024-09-10T18:45:22.068326+0000","last_unstale":"2024-09-10T18:49:48.233239+0000","last_undegraded":"2024-09-10T18:49:48.233239+0000","last_fullsized":"2024-09-10T18:49:48.233239+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:32.141755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:32.141755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040319509999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":91,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.228608+0000","last_change":"2024-09-10T18:49:02.511550+0000","last_active":"2024-09-10T18:49:48.228608+0000","last_peered":"2024-09-10T18:49:48.228608+0000","last_clean":"2024-09-10T18:49:48.228608+0000","last_became_active":"2024-09-10T18:45:24.081488+0000","last_became_peered":"2024-09-10T18:45:24.081488+0000","last_unstale":"2024-09-10T18:49:48.228608+0000","last_undegraded":"2024-09-10T18:49:48.228608+0000","last_fullsized":"2024-09-10T18:49:48.228608+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:02.511457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:02.511457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:18.180849+0000","scrub_duration":0.018959785999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"124'11045","reported_seq":11092,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-10T18:49:51.028800+0000","last_change":"2024-09-10T18:47:36.423039+0000","last_active":"2024-09-10T18:49:51.028800+0000","last_peered":"2024-09-10T18:49:51.028800+0000","last_clean":"2024-09-10T18:49:51.028800+0000","last_became_active":"2024-09-10T18:47:36.422694+0000","last_became_peered":"2024-09-10T18:47:36.422694+0000","last_unstale":"2024-09-10T18:49:51.028800+0000","last_undegraded":"2024-09-10T18:49:51.028800+0000","last_fullsized":"2024-09-10T18:49:51.028800+0000","mapping_epoch":88,"log_start":"111'5800","ondisk_log_start":"111'5800","created":86,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":5245,"log_dups_size":0,"ondisk_log_size":5245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":387907584,"num_objects":5919,"num_object_clones":0,"num_object_copies":17757,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11045,"num_write_kb":542848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"78'25","reported_seq":158,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.228292+0000","last_change":"2024-09-10T18:49:03.371577+0000","last_active":"2024-09-10T18:49:48.228292+0000","last_peered":"2024-09-10T18:49:48.228292+0000","last_clean":"2024-09-10T18:49:48.228292+0000","last_became_active":"2024-09-10T18:47:36.409300+0000","last_became_peered":"2024-09-10T18:47:36.409300+0000","last_unstale":"2024-09-10T18:49:48.228292+0000","last_undegraded":"2024-09-10T18:49:48.228292+0000","last_fullsized":"2024-09-10T18:49:48.228292+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'25","last_scrub_stamp":"2024-09-10T18:49:03.371486+0000","last_deep_scrub":"78'25","last_deep_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_clean_scrub_stamp":"2024-09-10T18:49:03.371486+0000","objects_scrubbed":8,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:04.078541+0000","scrub_duration":0.003278258,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":97,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.232150+0000","last_change":"2024-09-10T18:48:26.625704+0000","last_active":"2024-09-10T18:49:48.232150+0000","last_peered":"2024-09-10T18:49:48.232150+0000","last_clean":"2024-09-10T18:49:48.232150+0000","last_became_active":"2024-09-10T18:45:17.963732+0000","last_became_peered":"2024-09-10T18:45:17.963732+0000","last_unstale":"2024-09-10T18:49:48.232150+0000","last_undegraded":"2024-09-10T18:49:48.232150+0000","last_fullsized":"2024-09-10T18:49:48.232150+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:26.625635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:26.625635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034872639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":93,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.233851+0000","last_change":"2024-09-10T18:49:17.310050+0000","last_active":"2024-09-10T18:49:48.233851+0000","last_peered":"2024-09-10T18:49:48.233851+0000","last_clean":"2024-09-10T18:49:48.233851+0000","last_became_active":"2024-09-10T18:45:22.071443+0000","last_became_peered":"2024-09-10T18:45:22.071443+0000","last_unstale":"2024-09-10T18:49:48.233851+0000","last_undegraded":"2024-09-10T18:49:48.233851+0000","last_fullsized":"2024-09-10T18:49:48.233851+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:17.309861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:04.512425+0000","last_clean_scrub_stamp":"2024-09-10T18:49:17.309861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:38.825418+0000","scrub_duration":0.0051958760000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":47,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.229117+0000","last_change":"2024-09-10T18:49:11.545445+0000","last_active":"2024-09-10T18:49:48.229117+0000","last_peered":"2024-09-10T18:49:48.229117+0000","last_clean":"2024-09-10T18:49:48.229117+0000","last_became_active":"2024-09-10T18:47:57.104433+0000","last_became_peered":"2024-09-10T18:47:57.104433+0000","last_unstale":"2024-09-10T18:49:48.229117+0000","last_undegraded":"2024-09-10T18:49:48.229117+0000","last_fullsized":"2024-09-10T18:49:48.229117+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:11.545188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:11.545188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:19.456782+0000","scrub_duration":0.0019872409999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.f","version":"124'11082","reported_seq":11471,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-10T18:49:51.395170+0000","last_change":"2024-09-10T18:49:17.808384+0000","last_active":"2024-09-10T18:49:51.395170+0000","last_peered":"2024-09-10T18:49:51.395170+0000","last_clean":"2024-09-10T18:49:51.395170+0000","last_became_active":"2024-09-10T18:47:34.396087+0000","last_became_peered":"2024-09-10T18:47:34.396087+0000","last_unstale":"2024-09-10T18:49:51.395170+0000","last_undegraded":"2024-09-10T18:49:51.395170+0000","last_fullsized":"2024-09-10T18:49:51.395170+0000","mapping_epoch":86,"log_start":"111'6300","ondisk_log_start":"111'6300","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'8705","last_scrub_stamp":"2024-09-10T18:49:17.808165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:49:17.808165+0000","objects_scrubbed":8330,"log_size":4782,"log_dups_size":0,"ondisk_log_size":4782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:32.296732+0000","scrub_duration":4.3205227400000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":380239872,"num_objects":5802,"num_object_clones":0,"num_object_copies":17406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11082,"num_write_kb":540288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"116'15","reported_seq":114,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.229308+0000","last_change":"2024-09-10T18:49:12.374625+0000","last_active":"2024-09-10T18:49:48.229308+0000","last_peered":"2024-09-10T18:49:48.229308+0000","last_clean":"2024-09-10T18:49:48.229308+0000","last_became_active":"2024-09-10T18:45:20.053428+0000","last_became_peered":"2024-09-10T18:45:20.053428+0000","last_unstale":"2024-09-10T18:49:48.229308+0000","last_undegraded":"2024-09-10T18:49:48.229308+0000","last_fullsized":"2024-09-10T18:49:48.229308+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"116'15","last_scrub_stamp":"2024-09-10T18:49:12.374527+0000","last_deep_scrub":"116'15","last_deep_scrub_stamp":"2024-09-10T18:49:12.374527+0000","last_clean_scrub_stamp":"2024-09-10T18:49:12.374527+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:35.011875+0000","scrub_duration":0.0081453819999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":97,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.237529+0000","last_change":"2024-09-10T18:48:29.551161+0000","last_active":"2024-09-10T18:49:48.237529+0000","last_peered":"2024-09-10T18:49:48.237529+0000","last_clean":"2024-09-10T18:49:48.237529+0000","last_became_active":"2024-09-10T18:45:17.966833+0000","last_became_peered":"2024-09-10T18:45:17.966833+0000","last_unstale":"2024-09-10T18:49:48.237529+0000","last_undegraded":"2024-09-10T18:49:48.237529+0000","last_fullsized":"2024-09-10T18:49:48.237529+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:29.550946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:29.550946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:50.968295+0000","scrub_duration":0.016562341000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"89'9","reported_seq":108,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.233091+0000","last_change":"2024-09-10T18:48:09.648420+0000","last_active":"2024-09-10T18:49:48.233091+0000","last_peered":"2024-09-10T18:49:48.233091+0000","last_clean":"2024-09-10T18:49:48.233091+0000","last_became_active":"2024-09-10T18:47:36.421188+0000","last_became_peered":"2024-09-10T18:47:36.421188+0000","last_unstale":"2024-09-10T18:49:48.233091+0000","last_undegraded":"2024-09-10T18:49:48.233091+0000","last_fullsized":"2024-09-10T18:49:48.233091+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'9","last_scrub_stamp":"2024-09-10T18:48:09.648322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:09.648322+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0066489619999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":100,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.228706+0000","last_change":"2024-09-10T18:48:32.043742+0000","last_active":"2024-09-10T18:49:48.228706+0000","last_peered":"2024-09-10T18:49:48.228706+0000","last_clean":"2024-09-10T18:49:48.228706+0000","last_became_active":"2024-09-10T18:47:36.414625+0000","last_became_peered":"2024-09-10T18:47:36.414625+0000","last_unstale":"2024-09-10T18:49:48.228706+0000","last_undegraded":"2024-09-10T18:49:48.228706+0000","last_fullsized":"2024-09-10T18:49:48.228706+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:32.043666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:32.043666+0000","last_clean_scrub_stamp":"2024-09-10T18:48:32.043666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:49.740069+0000","scrub_duration":0.001572417,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"124'11243","reported_seq":11639,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-10T18:49:51.053570+0000","last_change":"2024-09-10T18:49:14.077746+0000","last_active":"2024-09-10T18:49:51.053570+0000","last_peered":"2024-09-10T18:49:51.053570+0000","last_clean":"2024-09-10T18:49:51.053570+0000","last_became_active":"2024-09-10T18:47:34.399261+0000","last_became_peered":"2024-09-10T18:47:34.399261+0000","last_unstale":"2024-09-10T18:49:51.053570+0000","last_undegraded":"2024-09-10T18:49:51.053570+0000","last_fullsized":"2024-09-10T18:49:51.053570+0000","mapping_epoch":86,"log_start":"111'6000","ondisk_log_start":"111'6000","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"116'8616","last_scrub_stamp":"2024-09-10T18:49:14.077662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:49:14.077662+0000","objects_scrubbed":8528,"log_size":5243,"log_dups_size":0,"ondisk_log_size":5243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:23.558482+0000","scrub_duration":2.6235428359999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":394723328,"num_objects":6023,"num_object_clones":0,"num_object_copies":18069,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11243,"num_write_kb":552512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"63'15","reported_seq":86,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.233211+0000","last_change":"2024-09-10T18:49:18.042505+0000","last_active":"2024-09-10T18:49:48.233211+0000","last_peered":"2024-09-10T18:49:48.233211+0000","last_clean":"2024-09-10T18:49:48.233211+0000","last_became_active":"2024-09-10T18:46:56.742946+0000","last_became_peered":"2024-09-10T18:46:56.742946+0000","last_unstale":"2024-09-10T18:49:48.233211+0000","last_undegraded":"2024-09-10T18:49:48.233211+0000","last_fullsized":"2024-09-10T18:49:48.233211+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:49:18.042442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:49:18.042442+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:20.073484+0000","scrub_duration":0.0044167360000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":97,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.233687+0000","last_change":"2024-09-10T18:48:13.707838+0000","last_active":"2024-09-10T18:49:48.233687+0000","last_peered":"2024-09-10T18:49:48.233687+0000","last_clean":"2024-09-10T18:49:48.233687+0000","last_became_active":"2024-09-10T18:45:17.965409+0000","last_became_peered":"2024-09-10T18:45:17.965409+0000","last_unstale":"2024-09-10T18:49:48.233687+0000","last_undegraded":"2024-09-10T18:49:48.233687+0000","last_fullsized":"2024-09-10T18:49:48.233687+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:13.707725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_clean_scrub_stamp":"2024-09-10T18:48:13.707725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0075610709999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"63'8","reported_seq":154,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.228882+0000","last_change":"2024-09-10T18:48:18.950238+0000","last_active":"2024-09-10T18:49:48.228882+0000","last_peered":"2024-09-10T18:49:48.228882+0000","last_clean":"2024-09-10T18:49:48.228882+0000","last_became_active":"2024-09-10T18:45:22.062497+0000","last_became_peered":"2024-09-10T18:45:22.062497+0000","last_unstale":"2024-09-10T18:49:48.228882+0000","last_undegraded":"2024-09-10T18:49:48.228882+0000","last_fullsized":"2024-09-10T18:49:48.228882+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:48:18.950140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:18.950140+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0032883439999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":90,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.232802+0000","last_change":"2024-09-10T18:48:31.170536+0000","last_active":"2024-09-10T18:49:48.232802+0000","last_peered":"2024-09-10T18:49:48.232802+0000","last_clean":"2024-09-10T18:49:48.232802+0000","last_became_active":"2024-09-10T18:45:24.082362+0000","last_became_peered":"2024-09-10T18:45:24.082362+0000","last_unstale":"2024-09-10T18:49:48.232802+0000","last_undegraded":"2024-09-10T18:49:48.232802+0000","last_fullsized":"2024-09-10T18:49:48.232802+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:31.170457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:31.170457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:50.433946+0000","scrub_duration":0.0084072260000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"124'10920","reported_seq":11302,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-10T18:49:51.037518+0000","last_change":"2024-09-10T18:49:12.677884+0000","last_active":"2024-09-10T18:49:51.037518+0000","last_peered":"2024-09-10T18:49:51.037518+0000","last_clean":"2024-09-10T18:49:51.037518+0000","last_became_active":"2024-09-10T18:47:34.399711+0000","last_became_peered":"2024-09-10T18:47:34.399711+0000","last_unstale":"2024-09-10T18:49:51.037518+0000","last_undegraded":"2024-09-10T18:49:51.037518+0000","last_fullsized":"2024-09-10T18:49:51.037518+0000","mapping_epoch":86,"log_start":"111'5700","ondisk_log_start":"111'5700","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"116'8278","last_scrub_stamp":"2024-09-10T18:49:12.677787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:49:12.677787+0000","objects_scrubbed":8161,"log_size":5220,"log_dups_size":0,"ondisk_log_size":5220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:23.222138+0000","scrub_duration":3.2225789539999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":383057944,"num_objects":5846,"num_object_clones":0,"num_object_copies":17538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5846,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10920,"num_write_kb":536449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"63'11","reported_seq":179,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.229360+0000","last_change":"2024-09-10T18:48:11.900583+0000","last_active":"2024-09-10T18:49:48.229360+0000","last_peered":"2024-09-10T18:49:48.229360+0000","last_clean":"2024-09-10T18:49:48.229360+0000","last_became_active":"2024-09-10T18:47:57.106935+0000","last_became_peered":"2024-09-10T18:47:57.106935+0000","last_unstale":"2024-09-10T18:49:48.229360+0000","last_undegraded":"2024-09-10T18:49:48.229360+0000","last_fullsized":"2024-09-10T18:49:48.229360+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:48:11.900489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:11.900489+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020768789999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":97,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.232441+0000","last_change":"2024-09-10T18:48:12.642771+0000","last_active":"2024-09-10T18:49:48.232441+0000","last_peered":"2024-09-10T18:49:48.232441+0000","last_clean":"2024-09-10T18:49:48.232441+0000","last_became_active":"2024-09-10T18:45:17.966133+0000","last_became_peered":"2024-09-10T18:45:17.966133+0000","last_unstale":"2024-09-10T18:49:48.232441+0000","last_undegraded":"2024-09-10T18:49:48.232441+0000","last_fullsized":"2024-09-10T18:49:48.232441+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:12.642693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:12.642693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0035654889999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":92,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.233378+0000","last_change":"2024-09-10T18:48:25.093064+0000","last_active":"2024-09-10T18:49:48.233378+0000","last_peered":"2024-09-10T18:49:48.233378+0000","last_clean":"2024-09-10T18:49:48.233378+0000","last_became_active":"2024-09-10T18:45:22.068918+0000","last_became_peered":"2024-09-10T18:45:22.068918+0000","last_unstale":"2024-09-10T18:49:48.233378+0000","last_undegraded":"2024-09-10T18:49:48.233378+0000","last_fullsized":"2024-09-10T18:49:48.233378+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:25.092984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_clean_scrub_stamp":"2024-09-10T18:48:25.092984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038537620000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":91,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.235148+0000","last_change":"2024-09-10T18:49:04.647711+0000","last_active":"2024-09-10T18:49:48.235148+0000","last_peered":"2024-09-10T18:49:48.235148+0000","last_clean":"2024-09-10T18:49:48.235148+0000","last_became_active":"2024-09-10T18:45:24.264538+0000","last_became_peered":"2024-09-10T18:45:24.264538+0000","last_unstale":"2024-09-10T18:49:48.235148+0000","last_undegraded":"2024-09-10T18:49:48.235148+0000","last_fullsized":"2024-09-10T18:49:48.235148+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:04.647638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:04.647638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:31.480091+0000","scrub_duration":0.115211389,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2","version":"124'11084","reported_seq":11140,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-10T18:49:51.278237+0000","last_change":"2024-09-10T18:47:36.423507+0000","last_active":"2024-09-10T18:49:51.278237+0000","last_peered":"2024-09-10T18:49:51.278237+0000","last_clean":"2024-09-10T18:49:51.278237+0000","last_became_active":"2024-09-10T18:47:36.423234+0000","last_became_peered":"2024-09-10T18:47:36.423234+0000","last_unstale":"2024-09-10T18:49:51.278237+0000","last_undegraded":"2024-09-10T18:49:51.278237+0000","last_fullsized":"2024-09-10T18:49:51.278237+0000","mapping_epoch":88,"log_start":"112'6800","ondisk_log_start":"112'6800","created":86,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":4284,"log_dups_size":0,"ondisk_log_size":4284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":385744896,"num_objects":5886,"num_object_clones":0,"num_object_copies":17658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11084,"num_write_kb":543040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"63'14","reported_seq":115,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.228682+0000","last_change":"2024-09-10T18:49:04.392814+0000","last_active":"2024-09-10T18:49:48.228682+0000","last_peered":"2024-09-10T18:49:48.228682+0000","last_clean":"2024-09-10T18:49:48.228682+0000","last_became_active":"2024-09-10T18:45:20.052243+0000","last_became_peered":"2024-09-10T18:45:20.052243+0000","last_unstale":"2024-09-10T18:49:48.228682+0000","last_undegraded":"2024-09-10T18:49:48.228682+0000","last_fullsized":"2024-09-10T18:49:48.228682+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-10T18:49:04.392667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:49:04.392667+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:23.598321+0000","scrub_duration":0.0046253650000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":106,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.231062+0000","last_change":"2024-09-10T18:48:13.631307+0000","last_active":"2024-09-10T18:49:48.231062+0000","last_peered":"2024-09-10T18:49:48.231062+0000","last_clean":"2024-09-10T18:49:48.231062+0000","last_became_active":"2024-09-10T18:47:36.424466+0000","last_became_peered":"2024-09-10T18:47:36.424466+0000","last_unstale":"2024-09-10T18:49:48.231062+0000","last_undegraded":"2024-09-10T18:49:48.231062+0000","last_fullsized":"2024-09-10T18:49:48.231062+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:13.631078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_clean_scrub_stamp":"2024-09-10T18:48:13.631078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043023660000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":156,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-10T18:49:49.543795+0000","last_change":"2024-09-10T18:48:23.479877+0000","last_active":"2024-09-10T18:49:49.543795+0000","last_peered":"2024-09-10T18:49:49.543795+0000","last_clean":"2024-09-10T18:49:49.543795+0000","last_became_active":"2024-09-10T18:45:22.074540+0000","last_became_peered":"2024-09-10T18:45:22.074540+0000","last_unstale":"2024-09-10T18:49:49.543795+0000","last_undegraded":"2024-09-10T18:49:49.543795+0000","last_fullsized":"2024-09-10T18:49:49.543795+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:48:23.479762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:23.479762+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.01210871,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":111,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.231926+0000","last_change":"2024-09-10T18:49:23.479699+0000","last_active":"2024-09-10T18:49:48.231926+0000","last_peered":"2024-09-10T18:49:48.231926+0000","last_clean":"2024-09-10T18:49:48.231926+0000","last_became_active":"2024-09-10T18:47:36.414665+0000","last_became_peered":"2024-09-10T18:47:36.414665+0000","last_unstale":"2024-09-10T18:49:48.231926+0000","last_undegraded":"2024-09-10T18:49:48.231926+0000","last_fullsized":"2024-09-10T18:49:48.231926+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:23.479647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:23.479647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:23.624042+0000","scrub_duration":0.0057738609999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"124'11288","reported_seq":11344,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-10T18:49:51.277737+0000","last_change":"2024-09-10T18:47:36.410975+0000","last_active":"2024-09-10T18:49:51.277737+0000","last_peered":"2024-09-10T18:49:51.277737+0000","last_clean":"2024-09-10T18:49:51.277737+0000","last_became_active":"2024-09-10T18:47:36.410639+0000","last_became_peered":"2024-09-10T18:47:36.410639+0000","last_unstale":"2024-09-10T18:49:51.277737+0000","last_undegraded":"2024-09-10T18:49:51.277737+0000","last_fullsized":"2024-09-10T18:49:51.277737+0000","mapping_epoch":88,"log_start":"112'7000","ondisk_log_start":"112'7000","created":86,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":4288,"log_dups_size":0,"ondisk_log_size":4288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":394657792,"num_objects":6022,"num_object_clones":0,"num_object_copies":18066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11288,"num_write_kb":553920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"63'18","reported_seq":127,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.239429+0000","last_change":"2024-09-10T18:48:20.514584+0000","last_active":"2024-09-10T18:49:48.239429+0000","last_peered":"2024-09-10T18:49:48.239429+0000","last_clean":"2024-09-10T18:49:48.239429+0000","last_became_active":"2024-09-10T18:45:20.420937+0000","last_became_peered":"2024-09-10T18:45:20.420937+0000","last_unstale":"2024-09-10T18:49:48.239429+0000","last_undegraded":"2024-09-10T18:49:48.239429+0000","last_fullsized":"2024-09-10T18:49:48.239429+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'18","last_scrub_stamp":"2024-09-10T18:48:20.514501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:20.514501+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.028109085999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":97,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.233566+0000","last_change":"2024-09-10T18:49:23.336052+0000","last_active":"2024-09-10T18:49:48.233566+0000","last_peered":"2024-09-10T18:49:48.233566+0000","last_clean":"2024-09-10T18:49:48.233566+0000","last_became_active":"2024-09-10T18:45:17.961695+0000","last_became_peered":"2024-09-10T18:45:17.961695+0000","last_unstale":"2024-09-10T18:49:48.233566+0000","last_undegraded":"2024-09-10T18:49:48.233566+0000","last_fullsized":"2024-09-10T18:49:48.233566+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:23.335968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:23.335968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:52.115506+0000","scrub_duration":0.0056344669999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":92,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.228604+0000","last_change":"2024-09-10T18:48:27.016984+0000","last_active":"2024-09-10T18:49:48.228604+0000","last_peered":"2024-09-10T18:49:48.228604+0000","last_clean":"2024-09-10T18:49:48.228604+0000","last_became_active":"2024-09-10T18:45:22.074012+0000","last_became_peered":"2024-09-10T18:45:22.074012+0000","last_unstale":"2024-09-10T18:49:48.228604+0000","last_undegraded":"2024-09-10T18:49:48.228604+0000","last_fullsized":"2024-09-10T18:49:48.228604+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:27.016933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:27.016933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0059183930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":90,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.227959+0000","last_change":"2024-09-10T18:47:36.411129+0000","last_active":"2024-09-10T18:49:48.227959+0000","last_peered":"2024-09-10T18:49:48.227959+0000","last_clean":"2024-09-10T18:49:48.227959+0000","last_became_active":"2024-09-10T18:47:36.410836+0000","last_became_peered":"2024-09-10T18:47:36.410836+0000","last_unstale":"2024-09-10T18:49:48.227959+0000","last_undegraded":"2024-09-10T18:49:48.227959+0000","last_fullsized":"2024-09-10T18:49:48.227959+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_clean_scrub_stamp":"2024-09-10T18:46:53.087539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001678971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"124'11032","reported_seq":11077,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-10T18:49:51.008766+0000","last_change":"2024-09-10T18:47:34.399316+0000","last_active":"2024-09-10T18:49:51.008766+0000","last_peered":"2024-09-10T18:49:51.008766+0000","last_clean":"2024-09-10T18:49:51.008766+0000","last_became_active":"2024-09-10T18:47:34.399002+0000","last_became_peered":"2024-09-10T18:47:34.399002+0000","last_unstale":"2024-09-10T18:49:51.008766+0000","last_undegraded":"2024-09-10T18:49:51.008766+0000","last_fullsized":"2024-09-10T18:49:51.008766+0000","mapping_epoch":86,"log_start":"111'5800","ondisk_log_start":"111'5800","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":5232,"log_dups_size":0,"ondisk_log_size":5232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":387842048,"num_objects":5918,"num_object_clones":0,"num_object_copies":17754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11032,"num_write_kb":542400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3","version":"63'19","reported_seq":131,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.229444+0000","last_change":"2024-09-10T18:48:11.781505+0000","last_active":"2024-09-10T18:49:48.229444+0000","last_peered":"2024-09-10T18:49:48.229444+0000","last_clean":"2024-09-10T18:49:48.229444+0000","last_became_active":"2024-09-10T18:45:20.421046+0000","last_became_peered":"2024-09-10T18:45:20.421046+0000","last_unstale":"2024-09-10T18:49:48.229444+0000","last_undegraded":"2024-09-10T18:49:48.229444+0000","last_fullsized":"2024-09-10T18:49:48.229444+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:48:11.781417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:11.781417+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030109379999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":97,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.233333+0000","last_change":"2024-09-10T18:48:30.694038+0000","last_active":"2024-09-10T18:49:48.233333+0000","last_peered":"2024-09-10T18:49:48.233333+0000","last_clean":"2024-09-10T18:49:48.233333+0000","last_became_active":"2024-09-10T18:45:17.960226+0000","last_became_peered":"2024-09-10T18:45:17.960226+0000","last_unstale":"2024-09-10T18:49:48.233333+0000","last_undegraded":"2024-09-10T18:49:48.233333+0000","last_fullsized":"2024-09-10T18:49:48.233333+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:30.693950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:30.693950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0079194309999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":92,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.230648+0000","last_change":"2024-09-10T18:48:33.513604+0000","last_active":"2024-09-10T18:49:48.230648+0000","last_peered":"2024-09-10T18:49:48.230648+0000","last_clean":"2024-09-10T18:49:48.230648+0000","last_became_active":"2024-09-10T18:45:22.066148+0000","last_became_peered":"2024-09-10T18:45:22.066148+0000","last_unstale":"2024-09-10T18:49:48.230648+0000","last_undegraded":"2024-09-10T18:49:48.230648+0000","last_fullsized":"2024-09-10T18:49:48.230648+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:33.513500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:33.513500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0080948319999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":101,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.233404+0000","last_change":"2024-09-10T18:49:18.281313+0000","last_active":"2024-09-10T18:49:48.233404+0000","last_peered":"2024-09-10T18:49:48.233404+0000","last_clean":"2024-09-10T18:49:48.233404+0000","last_became_active":"2024-09-10T18:47:36.422295+0000","last_became_peered":"2024-09-10T18:47:36.422295+0000","last_unstale":"2024-09-10T18:49:48.233404+0000","last_undegraded":"2024-09-10T18:49:48.233404+0000","last_fullsized":"2024-09-10T18:49:48.233404+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:18.281230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:49:18.281230+0000","last_clean_scrub_stamp":"2024-09-10T18:49:18.281230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:42.524080+0000","scrub_duration":0.0066273570000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"124'11081","reported_seq":11126,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-10T18:49:50.991923+0000","last_change":"2024-09-10T18:47:34.397576+0000","last_active":"2024-09-10T18:49:50.991923+0000","last_peered":"2024-09-10T18:49:50.991923+0000","last_clean":"2024-09-10T18:49:50.991923+0000","last_became_active":"2024-09-10T18:47:34.396631+0000","last_became_peered":"2024-09-10T18:47:34.396631+0000","last_unstale":"2024-09-10T18:49:50.991923+0000","last_undegraded":"2024-09-10T18:49:50.991923+0000","last_fullsized":"2024-09-10T18:49:50.991923+0000","mapping_epoch":86,"log_start":"111'5900","ondisk_log_start":"111'5900","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":5181,"log_dups_size":0,"ondisk_log_size":5181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":385155072,"num_objects":5877,"num_object_clones":0,"num_object_copies":17631,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5877,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11081,"num_write_kb":542656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"83'15","reported_seq":115,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.237355+0000","last_change":"2024-09-10T18:48:20.633119+0000","last_active":"2024-09-10T18:49:48.237355+0000","last_peered":"2024-09-10T18:49:48.237355+0000","last_clean":"2024-09-10T18:49:48.237355+0000","last_became_active":"2024-09-10T18:45:20.057204+0000","last_became_peered":"2024-09-10T18:45:20.057204+0000","last_unstale":"2024-09-10T18:49:48.237355+0000","last_undegraded":"2024-09-10T18:49:48.237355+0000","last_fullsized":"2024-09-10T18:49:48.237355+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'15","last_scrub_stamp":"2024-09-10T18:48:20.633054+0000","last_deep_scrub":"83'15","last_deep_scrub_stamp":"2024-09-10T18:46:58.436473+0000","last_clean_scrub_stamp":"2024-09-10T18:48:20.633054+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.008267412,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":97,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.234203+0000","last_change":"2024-09-10T18:49:09.449135+0000","last_active":"2024-09-10T18:49:48.234203+0000","last_peered":"2024-09-10T18:49:48.234203+0000","last_clean":"2024-09-10T18:49:48.234203+0000","last_became_active":"2024-09-10T18:45:17.966632+0000","last_became_peered":"2024-09-10T18:45:17.966632+0000","last_unstale":"2024-09-10T18:49:48.234203+0000","last_undegraded":"2024-09-10T18:49:48.234203+0000","last_fullsized":"2024-09-10T18:49:48.234203+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:09.449037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:09.449037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:18.822485+0000","scrub_duration":0.013065797000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"63'8","reported_seq":156,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-10T18:49:49.533737+0000","last_change":"2024-09-10T18:49:00.552860+0000","last_active":"2024-09-10T18:49:49.533737+0000","last_peered":"2024-09-10T18:49:49.533737+0000","last_clean":"2024-09-10T18:49:49.533737+0000","last_became_active":"2024-09-10T18:45:22.063326+0000","last_became_peered":"2024-09-10T18:45:22.063326+0000","last_unstale":"2024-09-10T18:49:49.533737+0000","last_undegraded":"2024-09-10T18:49:49.533737+0000","last_fullsized":"2024-09-10T18:49:49.533737+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:49:00.552765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:49:00.552765+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:28.435607+0000","scrub_duration":0.0038235579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":101,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.228450+0000","last_change":"2024-09-10T18:49:01.532633+0000","last_active":"2024-09-10T18:49:48.228450+0000","last_peered":"2024-09-10T18:49:48.228450+0000","last_clean":"2024-09-10T18:49:48.228450+0000","last_became_active":"2024-09-10T18:47:36.412111+0000","last_became_peered":"2024-09-10T18:47:36.412111+0000","last_unstale":"2024-09-10T18:49:48.228450+0000","last_undegraded":"2024-09-10T18:49:48.228450+0000","last_fullsized":"2024-09-10T18:49:48.228450+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:01.532550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:01.532550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:17.367791+0000","scrub_duration":0.0037383170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"124'11112","reported_seq":11477,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-10T18:49:51.070377+0000","last_change":"2024-09-10T18:49:05.169424+0000","last_active":"2024-09-10T18:49:51.070377+0000","last_peered":"2024-09-10T18:49:51.070377+0000","last_clean":"2024-09-10T18:49:51.070377+0000","last_became_active":"2024-09-10T18:47:34.400044+0000","last_became_peered":"2024-09-10T18:47:34.400044+0000","last_unstale":"2024-09-10T18:49:51.070377+0000","last_undegraded":"2024-09-10T18:49:51.070377+0000","last_fullsized":"2024-09-10T18:49:51.070377+0000","mapping_epoch":86,"log_start":"111'5900","ondisk_log_start":"111'5900","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"115'7848","last_scrub_stamp":"2024-09-10T18:49:05.169330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:49:05.169330+0000","objects_scrubbed":7724,"log_size":5212,"log_dups_size":0,"ondisk_log_size":5212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:16.559490+0000","scrub_duration":3.7256509850000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":384434176,"num_objects":5866,"num_object_clones":0,"num_object_copies":17598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11112,"num_write_kb":543296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"89'29","reported_seq":230,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-10T18:49:49.540883+0000","last_change":"2024-09-10T18:47:36.443205+0000","last_active":"2024-09-10T18:49:49.540883+0000","last_peered":"2024-09-10T18:49:49.540883+0000","last_clean":"2024-09-10T18:49:49.540883+0000","last_became_active":"2024-09-10T18:47:36.420185+0000","last_became_peered":"2024-09-10T18:47:36.420185+0000","last_unstale":"2024-09-10T18:49:49.540883+0000","last_undegraded":"2024-09-10T18:49:49.540883+0000","last_fullsized":"2024-09-10T18:49:49.540883+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'28","last_scrub_stamp":"2024-09-10T18:47:26.525861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:26.525861+0000","objects_scrubbed":10,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002126719,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":96,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.234314+0000","last_change":"2024-09-10T18:48:33.186208+0000","last_active":"2024-09-10T18:49:48.234314+0000","last_peered":"2024-09-10T18:49:48.234314+0000","last_clean":"2024-09-10T18:49:48.234314+0000","last_became_active":"2024-09-10T18:45:17.964494+0000","last_became_peered":"2024-09-10T18:45:17.964494+0000","last_unstale":"2024-09-10T18:49:48.234314+0000","last_undegraded":"2024-09-10T18:49:48.234314+0000","last_fullsized":"2024-09-10T18:49:48.234314+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:33.186070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:33.186070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:00.114296+0000","scrub_duration":0.0068142150000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":47,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.230863+0000","last_change":"2024-09-10T18:49:00.438027+0000","last_active":"2024-09-10T18:49:48.230863+0000","last_peered":"2024-09-10T18:49:48.230863+0000","last_clean":"2024-09-10T18:49:48.230863+0000","last_became_active":"2024-09-10T18:47:57.107478+0000","last_became_peered":"2024-09-10T18:47:57.107478+0000","last_unstale":"2024-09-10T18:49:48.230863+0000","last_undegraded":"2024-09-10T18:49:48.230863+0000","last_fullsized":"2024-09-10T18:49:48.230863+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:00.437877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:49:00.437877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:04.529124+0000","scrub_duration":0.012117157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":81,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.237318+0000","last_change":"2024-09-10T18:47:22.891142+0000","last_active":"2024-09-10T18:49:48.237318+0000","last_peered":"2024-09-10T18:49:48.237318+0000","last_clean":"2024-09-10T18:49:48.237318+0000","last_became_active":"2024-09-10T18:45:24.073307+0000","last_became_peered":"2024-09-10T18:45:24.073307+0000","last_unstale":"2024-09-10T18:49:48.237318+0000","last_undegraded":"2024-09-10T18:49:48.237318+0000","last_fullsized":"2024-09-10T18:49:48.237318+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:22.891084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:22.891084+0000","last_clean_scrub_stamp":"2024-09-10T18:47:22.891084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017788579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"124'11025","reported_seq":11394,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-10T18:49:51.395555+0000","last_change":"2024-09-10T18:49:07.011995+0000","last_active":"2024-09-10T18:49:51.395555+0000","last_peered":"2024-09-10T18:49:51.395555+0000","last_clean":"2024-09-10T18:49:51.395555+0000","last_became_active":"2024-09-10T18:47:34.395271+0000","last_became_peered":"2024-09-10T18:47:34.395271+0000","last_unstale":"2024-09-10T18:49:51.395555+0000","last_undegraded":"2024-09-10T18:49:51.395555+0000","last_fullsized":"2024-09-10T18:49:51.395555+0000","mapping_epoch":86,"log_start":"111'6200","ondisk_log_start":"111'6200","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"115'7913","last_scrub_stamp":"2024-09-10T18:49:07.011758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:49:07.011758+0000","objects_scrubbed":7815,"log_size":4825,"log_dups_size":0,"ondisk_log_size":4825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:20.945730+0000","scrub_duration":2.4953754379999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":379650048,"num_objects":5793,"num_object_clones":0,"num_object_copies":17379,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11025,"num_write_kb":538176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"84'18","reported_seq":127,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.234363+0000","last_change":"2024-09-10T18:49:13.417595+0000","last_active":"2024-09-10T18:49:48.234363+0000","last_peered":"2024-09-10T18:49:48.234363+0000","last_clean":"2024-09-10T18:49:48.234363+0000","last_became_active":"2024-09-10T18:45:20.058435+0000","last_became_peered":"2024-09-10T18:45:20.058435+0000","last_unstale":"2024-09-10T18:49:48.234363+0000","last_undegraded":"2024-09-10T18:49:48.234363+0000","last_fullsized":"2024-09-10T18:49:48.234363+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'18","last_scrub_stamp":"2024-09-10T18:49:13.417302+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-10T18:47:10.514951+0000","last_clean_scrub_stamp":"2024-09-10T18:49:13.417302+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:34.815917+0000","scrub_duration":0.0096130710000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"84'6","reported_seq":374,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-10T18:49:49.724621+0000","last_change":"2024-09-10T18:47:36.439529+0000","last_active":"2024-09-10T18:49:49.724621+0000","last_peered":"2024-09-10T18:49:49.724621+0000","last_clean":"2024-09-10T18:49:49.724621+0000","last_became_active":"2024-09-10T18:47:36.418211+0000","last_became_peered":"2024-09-10T18:47:36.418211+0000","last_unstale":"2024-09-10T18:49:49.724621+0000","last_undegraded":"2024-09-10T18:49:49.724621+0000","last_fullsized":"2024-09-10T18:49:49.724621+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'6","last_scrub_stamp":"2024-09-10T18:47:22.545450+0000","last_deep_scrub":"84'6","last_deep_scrub_stamp":"2024-09-10T18:47:22.545450+0000","last_clean_scrub_stamp":"2024-09-10T18:47:22.545450+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022780040000000001,"objects_trimmed":0,"snaptrim_duration":0.0021933159999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":263,"num_read_kb":258,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":58,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.235920+0000","last_change":"2024-09-10T18:49:16.371789+0000","last_active":"2024-09-10T18:49:48.235920+0000","last_peered":"2024-09-10T18:49:48.235920+0000","last_clean":"2024-09-10T18:49:48.235920+0000","last_became_active":"2024-09-10T18:47:36.422641+0000","last_became_peered":"2024-09-10T18:47:36.422641+0000","last_unstale":"2024-09-10T18:49:48.235920+0000","last_undegraded":"2024-09-10T18:49:48.235920+0000","last_fullsized":"2024-09-10T18:49:48.235920+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:16.371702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:16.371702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:39.416796+0000","scrub_duration":0.0099806840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":57,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.234318+0000","last_change":"2024-09-10T18:48:22.011954+0000","last_active":"2024-09-10T18:49:48.234318+0000","last_peered":"2024-09-10T18:49:48.234318+0000","last_clean":"2024-09-10T18:49:48.234318+0000","last_became_active":"2024-09-10T18:47:36.419081+0000","last_became_peered":"2024-09-10T18:47:36.419081+0000","last_unstale":"2024-09-10T18:49:48.234318+0000","last_undegraded":"2024-09-10T18:49:48.234318+0000","last_fullsized":"2024-09-10T18:49:48.234318+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:22.011870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:22.011870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018770919999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":91,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.233012+0000","last_change":"2024-09-10T18:49:12.603922+0000","last_active":"2024-09-10T18:49:48.233012+0000","last_peered":"2024-09-10T18:49:48.233012+0000","last_clean":"2024-09-10T18:49:48.233012+0000","last_became_active":"2024-09-10T18:45:24.264617+0000","last_became_peered":"2024-09-10T18:45:24.264617+0000","last_unstale":"2024-09-10T18:49:48.233012+0000","last_undegraded":"2024-09-10T18:49:48.233012+0000","last_fullsized":"2024-09-10T18:49:48.233012+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:12.603786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:12.603786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:36.865751+0000","scrub_duration":0.0019980200000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"124'11061","reported_seq":11453,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-10T18:49:51.068953+0000","last_change":"2024-09-10T18:49:19.513064+0000","last_active":"2024-09-10T18:49:51.068953+0000","last_peered":"2024-09-10T18:49:51.068953+0000","last_clean":"2024-09-10T18:49:51.068953+0000","last_became_active":"2024-09-10T18:47:36.417075+0000","last_became_peered":"2024-09-10T18:47:36.417075+0000","last_unstale":"2024-09-10T18:49:51.068953+0000","last_undegraded":"2024-09-10T18:49:51.068953+0000","last_fullsized":"2024-09-10T18:49:51.068953+0000","mapping_epoch":88,"log_start":"112'6500","ondisk_log_start":"112'6500","created":86,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'8906","last_scrub_stamp":"2024-09-10T18:49:19.512981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:49:19.512981+0000","objects_scrubbed":8201,"log_size":4561,"log_dups_size":0,"ondisk_log_size":4561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:23.685691+0000","scrub_duration":2.9222927400000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":389218304,"num_objects":5939,"num_object_clones":0,"num_object_copies":17817,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11061,"num_write_kb":544000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"63'12","reported_seq":137,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.229403+0000","last_change":"2024-09-10T18:48:28.710224+0000","last_active":"2024-09-10T18:49:48.229403+0000","last_peered":"2024-09-10T18:49:48.229403+0000","last_clean":"2024-09-10T18:49:48.229403+0000","last_became_active":"2024-09-10T18:47:36.417965+0000","last_became_peered":"2024-09-10T18:47:36.417965+0000","last_unstale":"2024-09-10T18:49:48.229403+0000","last_undegraded":"2024-09-10T18:49:48.229403+0000","last_fullsized":"2024-09-10T18:49:48.229403+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:48:28.710113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:28.710113+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:58.705191+0000","scrub_duration":0.0069768260000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"58'1","reported_seq":103,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.229044+0000","last_change":"2024-09-10T18:49:07.525629+0000","last_active":"2024-09-10T18:49:48.229044+0000","last_peered":"2024-09-10T18:49:48.229044+0000","last_clean":"2024-09-10T18:49:48.229044+0000","last_became_active":"2024-09-10T18:45:17.963932+0000","last_became_peered":"2024-09-10T18:45:17.963932+0000","last_unstale":"2024-09-10T18:49:48.229044+0000","last_undegraded":"2024-09-10T18:49:48.229044+0000","last_fullsized":"2024-09-10T18:49:48.229044+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'1","last_scrub_stamp":"2024-09-10T18:49:07.525525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:07.525525+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:30.019519+0000","scrub_duration":0.01484512,"objects_trimmed":0,"snaptrim_duration":0.00037812099999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":55,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.231683+0000","last_change":"2024-09-10T18:49:07.603571+0000","last_active":"2024-09-10T18:49:48.231683+0000","last_peered":"2024-09-10T18:49:48.231683+0000","last_clean":"2024-09-10T18:49:48.231683+0000","last_became_active":"2024-09-10T18:47:36.416641+0000","last_became_peered":"2024-09-10T18:47:36.416641+0000","last_unstale":"2024-09-10T18:49:48.231683+0000","last_undegraded":"2024-09-10T18:49:48.231683+0000","last_fullsized":"2024-09-10T18:49:48.231683+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:07.603516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:07.603516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:29.660107+0000","scrub_duration":0.0079160310000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":92,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.233440+0000","last_change":"2024-09-10T18:49:11.029146+0000","last_active":"2024-09-10T18:49:48.233440+0000","last_peered":"2024-09-10T18:49:48.233440+0000","last_clean":"2024-09-10T18:49:48.233440+0000","last_became_active":"2024-09-10T18:45:22.064285+0000","last_became_peered":"2024-09-10T18:45:22.064285+0000","last_unstale":"2024-09-10T18:49:48.233440+0000","last_undegraded":"2024-09-10T18:49:48.233440+0000","last_fullsized":"2024-09-10T18:49:48.233440+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:11.029069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:49:11.029069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:18.876406+0000","scrub_duration":0.0021808299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":91,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.233765+0000","last_change":"2024-09-10T18:48:21.654220+0000","last_active":"2024-09-10T18:49:48.233765+0000","last_peered":"2024-09-10T18:49:48.233765+0000","last_clean":"2024-09-10T18:49:48.233765+0000","last_became_active":"2024-09-10T18:45:24.083689+0000","last_became_peered":"2024-09-10T18:45:24.083689+0000","last_unstale":"2024-09-10T18:49:48.233765+0000","last_undegraded":"2024-09-10T18:49:48.233765+0000","last_fullsized":"2024-09-10T18:49:48.233765+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:21.654007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:21.654007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058335929999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"124'11041","reported_seq":11418,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-10T18:49:51.398283+0000","last_change":"2024-09-10T18:49:22.290906+0000","last_active":"2024-09-10T18:49:51.398283+0000","last_peered":"2024-09-10T18:49:51.398283+0000","last_clean":"2024-09-10T18:49:51.398283+0000","last_became_active":"2024-09-10T18:47:34.396853+0000","last_became_peered":"2024-09-10T18:47:34.396853+0000","last_unstale":"2024-09-10T18:49:51.398283+0000","last_undegraded":"2024-09-10T18:49:51.398283+0000","last_fullsized":"2024-09-10T18:49:51.398283+0000","mapping_epoch":86,"log_start":"111'6200","ondisk_log_start":"111'6200","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'9002","last_scrub_stamp":"2024-09-10T18:49:22.290682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:49:22.290682+0000","objects_scrubbed":7999,"log_size":4841,"log_dups_size":0,"ondisk_log_size":4841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:24.221318+0000","scrub_duration":3.8586685599999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":381353984,"num_objects":5819,"num_object_clones":0,"num_object_copies":17457,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11041,"num_write_kb":539520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"63'14","reported_seq":116,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.231187+0000","last_change":"2024-09-10T18:48:17.669939+0000","last_active":"2024-09-10T18:49:48.231187+0000","last_peered":"2024-09-10T18:49:48.231187+0000","last_clean":"2024-09-10T18:49:48.231187+0000","last_became_active":"2024-09-10T18:45:20.421119+0000","last_became_peered":"2024-09-10T18:45:20.421119+0000","last_unstale":"2024-09-10T18:49:48.231187+0000","last_undegraded":"2024-09-10T18:49:48.231187+0000","last_fullsized":"2024-09-10T18:49:48.231187+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'14","last_scrub_stamp":"2024-09-10T18:48:17.669787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:17.669787+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024048300000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"60'2","reported_seq":104,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.241561+0000","last_change":"2024-09-10T18:48:59.558866+0000","last_active":"2024-09-10T18:49:48.241561+0000","last_peered":"2024-09-10T18:49:48.241561+0000","last_clean":"2024-09-10T18:49:48.241561+0000","last_became_active":"2024-09-10T18:45:17.961660+0000","last_became_peered":"2024-09-10T18:45:17.961660+0000","last_unstale":"2024-09-10T18:49:48.241561+0000","last_undegraded":"2024-09-10T18:49:48.241561+0000","last_fullsized":"2024-09-10T18:49:48.241561+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-10T18:48:59.558773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:59.558773+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:15.175215+0000","scrub_duration":0.0043139010000000002,"objects_trimmed":0,"snaptrim_duration":0.0013942640000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":106,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.233457+0000","last_change":"2024-09-10T18:48:34.150227+0000","last_active":"2024-09-10T18:49:48.233457+0000","last_peered":"2024-09-10T18:49:48.233457+0000","last_clean":"2024-09-10T18:49:48.233457+0000","last_became_active":"2024-09-10T18:47:36.414636+0000","last_became_peered":"2024-09-10T18:47:36.414636+0000","last_unstale":"2024-09-10T18:49:48.233457+0000","last_undegraded":"2024-09-10T18:49:48.233457+0000","last_fullsized":"2024-09-10T18:49:48.233457+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:34.150146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:34.150146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:02.371194+0000","scrub_duration":0.0018375380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'81","reported_seq":141,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.232051+0000","last_change":"2024-09-10T18:49:09.660774+0000","last_active":"2024-09-10T18:49:48.232051+0000","last_peered":"2024-09-10T18:49:48.232051+0000","last_clean":"2024-09-10T18:49:48.232051+0000","last_became_active":"2024-09-10T18:45:07.433592+0000","last_became_peered":"2024-09-10T18:45:07.433592+0000","last_unstale":"2024-09-10T18:49:48.232051+0000","last_undegraded":"2024-09-10T18:49:48.232051+0000","last_fullsized":"2024-09-10T18:49:48.232051+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:49:09.660690+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:49:09.660690+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:31.143136+0000","scrub_duration":0.016709736999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":113,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.233876+0000","last_change":"2024-09-10T18:49:14.625034+0000","last_active":"2024-09-10T18:49:48.233876+0000","last_peered":"2024-09-10T18:49:48.233876+0000","last_clean":"2024-09-10T18:49:48.233876+0000","last_became_active":"2024-09-10T18:47:36.418320+0000","last_became_peered":"2024-09-10T18:47:36.418320+0000","last_unstale":"2024-09-10T18:49:48.233876+0000","last_undegraded":"2024-09-10T18:49:48.233876+0000","last_fullsized":"2024-09-10T18:49:48.233876+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:14.624957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:49:14.624957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:37.640823+0000","scrub_duration":0.0023025350000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":90,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.233596+0000","last_change":"2024-09-10T18:49:19.075811+0000","last_active":"2024-09-10T18:49:48.233596+0000","last_peered":"2024-09-10T18:49:48.233596+0000","last_clean":"2024-09-10T18:49:48.233596+0000","last_became_active":"2024-09-10T18:45:24.073327+0000","last_became_peered":"2024-09-10T18:45:24.073327+0000","last_unstale":"2024-09-10T18:49:48.233596+0000","last_undegraded":"2024-09-10T18:49:48.233596+0000","last_fullsized":"2024-09-10T18:49:48.233596+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:19.075748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:19.075748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:25.971565+0000","scrub_duration":0.0055106640000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.9","version":"124'15237","reported_seq":15468,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-10T18:49:51.404354+0000","last_change":"2024-09-10T18:49:40.255966+0000","last_active":"2024-09-10T18:49:51.404354+0000","last_peered":"2024-09-10T18:49:51.404354+0000","last_clean":"2024-09-10T18:49:51.404354+0000","last_became_active":"2024-09-10T18:47:34.393620+0000","last_became_peered":"2024-09-10T18:47:34.393620+0000","last_unstale":"2024-09-10T18:49:51.404354+0000","last_undegraded":"2024-09-10T18:49:51.404354+0000","last_fullsized":"2024-09-10T18:49:51.404354+0000","mapping_epoch":86,"log_start":"117'10402","ondisk_log_start":"117'10402","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":4474,"log_size":4835,"log_dups_size":0,"ondisk_log_size":4835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":390987776,"num_objects":5966,"num_object_clones":0,"num_object_copies":17898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11378,"num_write_kb":555008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"63'19","reported_seq":125,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.230813+0000","last_change":"2024-09-10T18:48:14.655578+0000","last_active":"2024-09-10T18:49:48.230813+0000","last_peered":"2024-09-10T18:49:48.230813+0000","last_clean":"2024-09-10T18:49:48.230813+0000","last_became_active":"2024-09-10T18:45:20.421097+0000","last_became_peered":"2024-09-10T18:45:20.421097+0000","last_unstale":"2024-09-10T18:49:48.230813+0000","last_undegraded":"2024-09-10T18:49:48.230813+0000","last_fullsized":"2024-09-10T18:49:48.230813+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:48:14.655527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:14.655527+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00174087,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":57,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.233684+0000","last_change":"2024-09-10T18:49:08.070170+0000","last_active":"2024-09-10T18:49:48.233684+0000","last_peered":"2024-09-10T18:49:48.233684+0000","last_clean":"2024-09-10T18:49:48.233684+0000","last_became_active":"2024-09-10T18:47:36.417800+0000","last_became_peered":"2024-09-10T18:47:36.417800+0000","last_unstale":"2024-09-10T18:49:48.233684+0000","last_undegraded":"2024-09-10T18:49:48.233684+0000","last_fullsized":"2024-09-10T18:49:48.233684+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:08.070063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:08.070063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:15.698762+0000","scrub_duration":0.0033662290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":68,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.229061+0000","last_change":"2024-09-10T18:49:20.477390+0000","last_active":"2024-09-10T18:49:48.229061+0000","last_peered":"2024-09-10T18:49:48.229061+0000","last_clean":"2024-09-10T18:49:48.229061+0000","last_became_active":"2024-09-10T18:47:36.413618+0000","last_became_peered":"2024-09-10T18:47:36.413618+0000","last_unstale":"2024-09-10T18:49:48.229061+0000","last_undegraded":"2024-09-10T18:49:48.229061+0000","last_fullsized":"2024-09-10T18:49:48.229061+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:20.477302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:49:20.477302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:43.626544+0000","scrub_duration":0.0047920660000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":101,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.231969+0000","last_change":"2024-09-10T18:49:11.634724+0000","last_active":"2024-09-10T18:49:48.231969+0000","last_peered":"2024-09-10T18:49:48.231969+0000","last_clean":"2024-09-10T18:49:48.231969+0000","last_became_active":"2024-09-10T18:47:36.412610+0000","last_became_peered":"2024-09-10T18:47:36.412610+0000","last_unstale":"2024-09-10T18:49:48.231969+0000","last_undegraded":"2024-09-10T18:49:48.231969+0000","last_fullsized":"2024-09-10T18:49:48.231969+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:11.634636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:11.634636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:37.083248+0000","scrub_duration":0.0058360749999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"124'11145","reported_seq":11523,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-10T18:49:51.081837+0000","last_change":"2024-09-10T18:49:22.931254+0000","last_active":"2024-09-10T18:49:51.081837+0000","last_peered":"2024-09-10T18:49:51.081837+0000","last_clean":"2024-09-10T18:49:51.081837+0000","last_became_active":"2024-09-10T18:47:36.412142+0000","last_became_peered":"2024-09-10T18:47:36.412142+0000","last_unstale":"2024-09-10T18:49:51.081837+0000","last_undegraded":"2024-09-10T18:49:51.081837+0000","last_fullsized":"2024-09-10T18:49:51.081837+0000","mapping_epoch":88,"log_start":"111'6500","ondisk_log_start":"111'6500","created":86,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'9054","last_scrub_stamp":"2024-09-10T18:49:22.931202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:49:22.931202+0000","objects_scrubbed":7988,"log_size":4645,"log_dups_size":0,"ondisk_log_size":4645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:28.321697+0000","scrub_duration":2.3919399270000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":380829696,"num_objects":5811,"num_object_clones":0,"num_object_copies":17433,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11145,"num_write_kb":542592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"63'9","reported_seq":111,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.229349+0000","last_change":"2024-09-10T18:48:22.878265+0000","last_active":"2024-09-10T18:49:48.229349+0000","last_peered":"2024-09-10T18:49:48.229349+0000","last_clean":"2024-09-10T18:49:48.229349+0000","last_became_active":"2024-09-10T18:45:20.061021+0000","last_became_peered":"2024-09-10T18:45:20.061021+0000","last_unstale":"2024-09-10T18:49:48.229349+0000","last_undegraded":"2024-09-10T18:49:48.229349+0000","last_fullsized":"2024-09-10T18:49:48.229349+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:48:22.878179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:22.878179+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.021357979999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":97,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.234385+0000","last_change":"2024-09-10T18:49:07.384707+0000","last_active":"2024-09-10T18:49:48.234385+0000","last_peered":"2024-09-10T18:49:48.234385+0000","last_clean":"2024-09-10T18:49:48.234385+0000","last_became_active":"2024-09-10T18:45:17.963057+0000","last_became_peered":"2024-09-10T18:45:17.963057+0000","last_unstale":"2024-09-10T18:49:48.234385+0000","last_undegraded":"2024-09-10T18:49:48.234385+0000","last_fullsized":"2024-09-10T18:49:48.234385+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:07.384620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:07.384620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:25.314632+0000","scrub_duration":0.011420041000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":57,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.227807+0000","last_change":"2024-09-10T18:49:21.442258+0000","last_active":"2024-09-10T18:49:48.227807+0000","last_peered":"2024-09-10T18:49:48.227807+0000","last_clean":"2024-09-10T18:49:48.227807+0000","last_became_active":"2024-09-10T18:47:36.411517+0000","last_became_peered":"2024-09-10T18:47:36.411517+0000","last_unstale":"2024-09-10T18:49:48.227807+0000","last_undegraded":"2024-09-10T18:49:48.227807+0000","last_fullsized":"2024-09-10T18:49:48.227807+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:21.442183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_clean_scrub_stamp":"2024-09-10T18:49:21.442183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:24.591614+0000","scrub_duration":0.0039195419999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":101,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.229024+0000","last_change":"2024-09-10T18:48:25.804464+0000","last_active":"2024-09-10T18:49:48.229024+0000","last_peered":"2024-09-10T18:49:48.229024+0000","last_clean":"2024-09-10T18:49:48.229024+0000","last_became_active":"2024-09-10T18:47:36.412933+0000","last_became_peered":"2024-09-10T18:47:36.412933+0000","last_unstale":"2024-09-10T18:49:48.229024+0000","last_undegraded":"2024-09-10T18:49:48.229024+0000","last_fullsized":"2024-09-10T18:49:48.229024+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:25.804376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:47.545343+0000","last_clean_scrub_stamp":"2024-09-10T18:48:25.804376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0098886110000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.b","version":"124'11155","reported_seq":11201,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-10T18:49:51.190467+0000","last_change":"2024-09-10T18:47:34.398004+0000","last_active":"2024-09-10T18:49:51.190467+0000","last_peered":"2024-09-10T18:49:51.190467+0000","last_clean":"2024-09-10T18:49:51.190467+0000","last_became_active":"2024-09-10T18:47:34.397708+0000","last_became_peered":"2024-09-10T18:47:34.397708+0000","last_unstale":"2024-09-10T18:49:51.190467+0000","last_undegraded":"2024-09-10T18:49:51.190467+0000","last_fullsized":"2024-09-10T18:49:51.190467+0000","mapping_epoch":86,"log_start":"112'6800","ondisk_log_start":"112'6800","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":4355,"log_dups_size":0,"ondisk_log_size":4355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":387776512,"num_objects":5917,"num_object_clones":0,"num_object_copies":17751,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11155,"num_write_kb":546304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"63'15","reported_seq":109,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.233242+0000","last_change":"2024-09-10T18:46:51.114543+0000","last_active":"2024-09-10T18:49:48.233242+0000","last_peered":"2024-09-10T18:49:48.233242+0000","last_clean":"2024-09-10T18:49:48.233242+0000","last_became_active":"2024-09-10T18:45:20.419804+0000","last_became_peered":"2024-09-10T18:45:20.419804+0000","last_unstale":"2024-09-10T18:49:48.233242+0000","last_undegraded":"2024-09-10T18:49:48.233242+0000","last_fullsized":"2024-09-10T18:49:48.233242+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:46:51.114455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:46:51.114455+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022557979999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":48,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.237433+0000","last_change":"2024-09-10T18:47:36.420713+0000","last_active":"2024-09-10T18:49:48.237433+0000","last_peered":"2024-09-10T18:49:48.237433+0000","last_clean":"2024-09-10T18:49:48.237433+0000","last_became_active":"2024-09-10T18:47:36.420475+0000","last_became_peered":"2024-09-10T18:47:36.420475+0000","last_unstale":"2024-09-10T18:49:48.237433+0000","last_undegraded":"2024-09-10T18:49:48.237433+0000","last_fullsized":"2024-09-10T18:49:48.237433+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:20.774289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:17.819909+0000","last_clean_scrub_stamp":"2024-09-10T18:47:20.774289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"83'9","reported_seq":159,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.231862+0000","last_change":"2024-09-10T18:48:33.683570+0000","last_active":"2024-09-10T18:49:48.231862+0000","last_peered":"2024-09-10T18:49:48.231862+0000","last_clean":"2024-09-10T18:49:48.231862+0000","last_became_active":"2024-09-10T18:45:22.073383+0000","last_became_peered":"2024-09-10T18:45:22.073383+0000","last_unstale":"2024-09-10T18:49:48.231862+0000","last_undegraded":"2024-09-10T18:49:48.231862+0000","last_fullsized":"2024-09-10T18:49:48.231862+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'9","last_scrub_stamp":"2024-09-10T18:48:33.683477+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-09-10T18:47:02.582229+0000","last_clean_scrub_stamp":"2024-09-10T18:48:33.683477+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:57.226708+0000","scrub_duration":0.0026989119999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":90,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.233839+0000","last_change":"2024-09-10T18:49:24.053252+0000","last_active":"2024-09-10T18:49:48.233839+0000","last_peered":"2024-09-10T18:49:48.233839+0000","last_clean":"2024-09-10T18:49:48.233839+0000","last_became_active":"2024-09-10T18:45:24.265256+0000","last_became_peered":"2024-09-10T18:45:24.265256+0000","last_unstale":"2024-09-10T18:49:48.233839+0000","last_undegraded":"2024-09-10T18:49:48.233839+0000","last_fullsized":"2024-09-10T18:49:48.233839+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:24.053158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_clean_scrub_stamp":"2024-09-10T18:49:24.053158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:27.258647+0000","scrub_duration":0.0012635400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.a","version":"124'11348","reported_seq":11404,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-10T18:49:51.271207+0000","last_change":"2024-09-10T18:47:36.423957+0000","last_active":"2024-09-10T18:49:51.271207+0000","last_peered":"2024-09-10T18:49:51.271207+0000","last_clean":"2024-09-10T18:49:51.271207+0000","last_became_active":"2024-09-10T18:47:36.422868+0000","last_became_peered":"2024-09-10T18:47:36.422868+0000","last_unstale":"2024-09-10T18:49:51.271207+0000","last_undegraded":"2024-09-10T18:49:51.271207+0000","last_fullsized":"2024-09-10T18:49:51.271207+0000","mapping_epoch":88,"log_start":"112'7100","ondisk_log_start":"112'7100","created":86,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:33.382948+0000","last_clean_scrub_stamp":"2024-09-10T18:47:33.382948+0000","objects_scrubbed":0,"log_size":4248,"log_dups_size":0,"ondisk_log_size":4248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":392691712,"num_objects":5992,"num_object_clones":0,"num_object_copies":17976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11348,"num_write_kb":554880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"63'12","reported_seq":117,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.228638+0000","last_change":"2024-09-10T18:47:35.928110+0000","last_active":"2024-09-10T18:49:48.228638+0000","last_peered":"2024-09-10T18:49:48.228638+0000","last_clean":"2024-09-10T18:49:48.228638+0000","last_became_active":"2024-09-10T18:45:20.056869+0000","last_became_peered":"2024-09-10T18:45:20.056869+0000","last_unstale":"2024-09-10T18:49:48.228638+0000","last_undegraded":"2024-09-10T18:49:48.228638+0000","last_fullsized":"2024-09-10T18:49:48.228638+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:47:35.928040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:35.928040+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021332539999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"56'2","reported_seq":113,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.228563+0000","last_change":"2024-09-10T18:49:20.403122+0000","last_active":"2024-09-10T18:49:48.228563+0000","last_peered":"2024-09-10T18:49:48.228563+0000","last_clean":"2024-09-10T18:49:48.228563+0000","last_became_active":"2024-09-10T18:45:17.963286+0000","last_became_peered":"2024-09-10T18:45:17.963286+0000","last_unstale":"2024-09-10T18:49:48.228563+0000","last_undegraded":"2024-09-10T18:49:48.228563+0000","last_fullsized":"2024-09-10T18:49:48.228563+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-10T18:49:20.403067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:20.403067+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:40.186789+0000","scrub_duration":0.00336061,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":58,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.228529+0000","last_change":"2024-09-10T18:48:15.763428+0000","last_active":"2024-09-10T18:49:48.228529+0000","last_peered":"2024-09-10T18:49:48.228529+0000","last_clean":"2024-09-10T18:49:48.228529+0000","last_became_active":"2024-09-10T18:47:36.421151+0000","last_became_peered":"2024-09-10T18:47:36.421151+0000","last_unstale":"2024-09-10T18:49:48.228529+0000","last_undegraded":"2024-09-10T18:49:48.228529+0000","last_fullsized":"2024-09-10T18:49:48.228529+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:15.763348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:15.763348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046035929999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":81,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.236234+0000","last_change":"2024-09-10T18:47:18.797303+0000","last_active":"2024-09-10T18:49:48.236234+0000","last_peered":"2024-09-10T18:49:48.236234+0000","last_clean":"2024-09-10T18:49:48.236234+0000","last_became_active":"2024-09-10T18:45:24.077815+0000","last_became_peered":"2024-09-10T18:45:24.077815+0000","last_unstale":"2024-09-10T18:49:48.236234+0000","last_undegraded":"2024-09-10T18:49:48.236234+0000","last_fullsized":"2024-09-10T18:49:48.236234+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:18.797245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:18.797245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015800090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"76'13","reported_seq":115,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.229316+0000","last_change":"2024-09-10T18:48:10.773290+0000","last_active":"2024-09-10T18:49:48.229316+0000","last_peered":"2024-09-10T18:49:48.229316+0000","last_clean":"2024-09-10T18:49:48.229316+0000","last_became_active":"2024-09-10T18:45:20.420421+0000","last_became_peered":"2024-09-10T18:45:20.420421+0000","last_unstale":"2024-09-10T18:49:48.229316+0000","last_undegraded":"2024-09-10T18:49:48.229316+0000","last_fullsized":"2024-09-10T18:49:48.229316+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'13","last_scrub_stamp":"2024-09-10T18:48:10.773207+0000","last_deep_scrub":"76'13","last_deep_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_clean_scrub_stamp":"2024-09-10T18:48:10.773207+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.008348678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":58,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.237005+0000","last_change":"2024-09-10T18:48:14.699173+0000","last_active":"2024-09-10T18:49:48.237005+0000","last_peered":"2024-09-10T18:49:48.237005+0000","last_clean":"2024-09-10T18:49:48.237005+0000","last_became_active":"2024-09-10T18:47:36.409299+0000","last_became_peered":"2024-09-10T18:47:36.409299+0000","last_unstale":"2024-09-10T18:49:48.237005+0000","last_undegraded":"2024-09-10T18:49:48.237005+0000","last_fullsized":"2024-09-10T18:49:48.237005+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:14.695946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:14.695946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007154171,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":93,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.236335+0000","last_change":"2024-09-10T18:48:28.442445+0000","last_active":"2024-09-10T18:49:48.236335+0000","last_peered":"2024-09-10T18:49:48.236335+0000","last_clean":"2024-09-10T18:49:48.236335+0000","last_became_active":"2024-09-10T18:45:22.069040+0000","last_became_peered":"2024-09-10T18:45:22.069040+0000","last_unstale":"2024-09-10T18:49:48.236335+0000","last_undegraded":"2024-09-10T18:49:48.236335+0000","last_fullsized":"2024-09-10T18:49:48.236335+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:28.442370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:28.442370+0000","last_clean_scrub_stamp":"2024-09-10T18:48:28.442370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:50.109389+0000","scrub_duration":0.0098412350000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"63'1","reported_seq":73,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-10T18:49:49.544571+0000","last_change":"2024-09-10T18:49:49.544530+0000","last_active":"2024-09-10T18:49:49.544571+0000","last_peered":"2024-09-10T18:49:49.544571+0000","last_clean":"2024-09-10T18:49:49.544571+0000","last_became_active":"2024-09-10T18:47:36.406936+0000","last_became_peered":"2024-09-10T18:47:36.406936+0000","last_unstale":"2024-09-10T18:49:49.544571+0000","last_undegraded":"2024-09-10T18:49:49.544571+0000","last_fullsized":"2024-09-10T18:49:49.544571+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-10T18:49:49.544433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:49.544433+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:12.425144+0000","scrub_duration":0.0041109190000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":96,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.230861+0000","last_change":"2024-09-10T18:48:24.571499+0000","last_active":"2024-09-10T18:49:48.230861+0000","last_peered":"2024-09-10T18:49:48.230861+0000","last_clean":"2024-09-10T18:49:48.230861+0000","last_became_active":"2024-09-10T18:45:17.964705+0000","last_became_peered":"2024-09-10T18:45:17.964705+0000","last_unstale":"2024-09-10T18:49:48.230861+0000","last_undegraded":"2024-09-10T18:49:48.230861+0000","last_fullsized":"2024-09-10T18:49:48.230861+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:24.571419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:24.571419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:51.133227+0000","scrub_duration":0.005895399,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'6","reported_seq":104,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.236451+0000","last_change":"2024-09-10T18:49:18.363292+0000","last_active":"2024-09-10T18:49:48.236451+0000","last_peered":"2024-09-10T18:49:48.236451+0000","last_clean":"2024-09-10T18:49:48.236451+0000","last_became_active":"2024-09-10T18:45:20.421746+0000","last_became_peered":"2024-09-10T18:45:20.421746+0000","last_unstale":"2024-09-10T18:49:48.236451+0000","last_undegraded":"2024-09-10T18:49:48.236451+0000","last_fullsized":"2024-09-10T18:49:48.236451+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'6","last_scrub_stamp":"2024-09-10T18:49:18.363203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:49:18.363203+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:32.230123+0000","scrub_duration":0.0093609290000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":92,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.234499+0000","last_change":"2024-09-10T18:49:12.996132+0000","last_active":"2024-09-10T18:49:48.234499+0000","last_peered":"2024-09-10T18:49:48.234499+0000","last_clean":"2024-09-10T18:49:48.234499+0000","last_became_active":"2024-09-10T18:45:22.069653+0000","last_became_peered":"2024-09-10T18:45:22.069653+0000","last_unstale":"2024-09-10T18:49:48.234499+0000","last_undegraded":"2024-09-10T18:49:48.234499+0000","last_fullsized":"2024-09-10T18:49:48.234499+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:12.996035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:49:12.996035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:37.029555+0000","scrub_duration":0.0049239360000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":91,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.232920+0000","last_change":"2024-09-10T18:48:32.654862+0000","last_active":"2024-09-10T18:49:48.232920+0000","last_peered":"2024-09-10T18:49:48.232920+0000","last_clean":"2024-09-10T18:49:48.232920+0000","last_became_active":"2024-09-10T18:45:24.264611+0000","last_became_peered":"2024-09-10T18:45:24.264611+0000","last_unstale":"2024-09-10T18:49:48.232920+0000","last_undegraded":"2024-09-10T18:49:48.232920+0000","last_fullsized":"2024-09-10T18:49:48.232920+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:32.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:32.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:01.743394+0000","scrub_duration":0.0049870009999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"63'10","reported_seq":154,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.235534+0000","last_change":"2024-09-10T18:49:01.556782+0000","last_active":"2024-09-10T18:49:48.235534+0000","last_peered":"2024-09-10T18:49:48.235534+0000","last_clean":"2024-09-10T18:49:48.235534+0000","last_became_active":"2024-09-10T18:47:36.426260+0000","last_became_peered":"2024-09-10T18:47:36.426260+0000","last_unstale":"2024-09-10T18:49:48.235534+0000","last_undegraded":"2024-09-10T18:49:48.235534+0000","last_fullsized":"2024-09-10T18:49:48.235534+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:49:01.556694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:49:01.556694+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:23.957988+0000","scrub_duration":0.0053223699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":96,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.234074+0000","last_change":"2024-09-10T18:49:06.079691+0000","last_active":"2024-09-10T18:49:48.234074+0000","last_peered":"2024-09-10T18:49:48.234074+0000","last_clean":"2024-09-10T18:49:48.234074+0000","last_became_active":"2024-09-10T18:45:17.964870+0000","last_became_peered":"2024-09-10T18:45:17.964870+0000","last_unstale":"2024-09-10T18:49:48.234074+0000","last_undegraded":"2024-09-10T18:49:48.234074+0000","last_fullsized":"2024-09-10T18:49:48.234074+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:06.079456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:06.079456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:12.151525+0000","scrub_duration":0.012719522,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"63'8","reported_seq":154,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.232939+0000","last_change":"2024-09-10T18:49:02.038795+0000","last_active":"2024-09-10T18:49:48.232939+0000","last_peered":"2024-09-10T18:49:48.232939+0000","last_clean":"2024-09-10T18:49:48.232939+0000","last_became_active":"2024-09-10T18:45:22.062940+0000","last_became_peered":"2024-09-10T18:45:22.062940+0000","last_unstale":"2024-09-10T18:49:48.232939+0000","last_undegraded":"2024-09-10T18:49:48.232939+0000","last_fullsized":"2024-09-10T18:49:48.232939+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:49:02.038709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:49:02.038709+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:15.977699+0000","scrub_duration":0.007468054,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":91,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.228914+0000","last_change":"2024-09-10T18:49:03.552056+0000","last_active":"2024-09-10T18:49:48.228914+0000","last_peered":"2024-09-10T18:49:48.228914+0000","last_clean":"2024-09-10T18:49:48.228914+0000","last_became_active":"2024-09-10T18:45:24.073736+0000","last_became_peered":"2024-09-10T18:45:24.073736+0000","last_unstale":"2024-09-10T18:49:48.228914+0000","last_undegraded":"2024-09-10T18:49:48.228914+0000","last_fullsized":"2024-09-10T18:49:48.228914+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:03.551966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:03.551966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:11.246649+0000","scrub_duration":0.015480603000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"63'9","reported_seq":110,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.233010+0000","last_change":"2024-09-10T18:48:19.076178+0000","last_active":"2024-09-10T18:49:48.233010+0000","last_peered":"2024-09-10T18:49:48.233010+0000","last_clean":"2024-09-10T18:49:48.233010+0000","last_became_active":"2024-09-10T18:45:20.421124+0000","last_became_peered":"2024-09-10T18:45:20.421124+0000","last_unstale":"2024-09-10T18:49:48.233010+0000","last_undegraded":"2024-09-10T18:49:48.233010+0000","last_fullsized":"2024-09-10T18:49:48.233010+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:48:19.076099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:19.076099+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002691981,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":97,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.229596+0000","last_change":"2024-09-10T18:48:33.663516+0000","last_active":"2024-09-10T18:49:48.229596+0000","last_peered":"2024-09-10T18:49:48.229596+0000","last_clean":"2024-09-10T18:49:48.229596+0000","last_became_active":"2024-09-10T18:45:17.960106+0000","last_became_peered":"2024-09-10T18:45:17.960106+0000","last_unstale":"2024-09-10T18:49:48.229596+0000","last_undegraded":"2024-09-10T18:49:48.229596+0000","last_fullsized":"2024-09-10T18:49:48.229596+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:33.663415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:33.663415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002167352,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"89'10","reported_seq":161,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-10T18:49:49.536702+0000","last_change":"2024-09-10T18:47:36.434102+0000","last_active":"2024-09-10T18:49:49.536702+0000","last_peered":"2024-09-10T18:49:49.536702+0000","last_clean":"2024-09-10T18:49:49.536702+0000","last_became_active":"2024-09-10T18:47:36.424575+0000","last_became_peered":"2024-09-10T18:47:36.424575+0000","last_unstale":"2024-09-10T18:49:49.536702+0000","last_undegraded":"2024-09-10T18:49:49.536702+0000","last_fullsized":"2024-09-10T18:49:49.536702+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"81'9","last_scrub_stamp":"2024-09-10T18:46:54.800790+0000","last_deep_scrub":"81'9","last_deep_scrub_stamp":"2024-09-10T18:46:54.800790+0000","last_clean_scrub_stamp":"2024-09-10T18:46:54.800790+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034208709999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":100,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.228312+0000","last_change":"2024-09-10T18:48:34.045421+0000","last_active":"2024-09-10T18:49:48.228312+0000","last_peered":"2024-09-10T18:49:48.228312+0000","last_clean":"2024-09-10T18:49:48.228312+0000","last_became_active":"2024-09-10T18:47:36.414047+0000","last_became_peered":"2024-09-10T18:47:36.414047+0000","last_unstale":"2024-09-10T18:49:48.228312+0000","last_undegraded":"2024-09-10T18:49:48.228312+0000","last_fullsized":"2024-09-10T18:49:48.228312+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:34.045311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_clean_scrub_stamp":"2024-09-10T18:48:34.045311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048840200000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"63'9","reported_seq":131,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.237284+0000","last_change":"2024-09-10T18:49:05.359047+0000","last_active":"2024-09-10T18:49:48.237284+0000","last_peered":"2024-09-10T18:49:48.237284+0000","last_clean":"2024-09-10T18:49:48.237284+0000","last_became_active":"2024-09-10T18:47:36.416484+0000","last_became_peered":"2024-09-10T18:47:36.416484+0000","last_unstale":"2024-09-10T18:49:48.237284+0000","last_undegraded":"2024-09-10T18:49:48.237284+0000","last_fullsized":"2024-09-10T18:49:48.237284+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:49:05.358887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:49:05.358887+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:26.279687+0000","scrub_duration":0.013783619,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":106,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.231440+0000","last_change":"2024-09-10T18:49:04.430421+0000","last_active":"2024-09-10T18:49:48.231440+0000","last_peered":"2024-09-10T18:49:48.231440+0000","last_clean":"2024-09-10T18:49:48.231440+0000","last_became_active":"2024-09-10T18:45:17.962009+0000","last_became_peered":"2024-09-10T18:45:17.962009+0000","last_unstale":"2024-09-10T18:49:48.231440+0000","last_undegraded":"2024-09-10T18:49:48.231440+0000","last_fullsized":"2024-09-10T18:49:48.231440+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:04.430344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:04.430344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:13.393531+0000","scrub_duration":0.0025461400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":83,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.236298+0000","last_change":"2024-09-10T18:47:14.869132+0000","last_active":"2024-09-10T18:49:48.236298+0000","last_peered":"2024-09-10T18:49:48.236298+0000","last_clean":"2024-09-10T18:49:48.236298+0000","last_became_active":"2024-09-10T18:45:22.072877+0000","last_became_peered":"2024-09-10T18:45:22.072877+0000","last_unstale":"2024-09-10T18:49:48.236298+0000","last_undegraded":"2024-09-10T18:49:48.236298+0000","last_fullsized":"2024-09-10T18:49:48.236298+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:14.869076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:47:14.869076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00131624,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":91,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.228372+0000","last_change":"2024-09-10T18:48:32.647084+0000","last_active":"2024-09-10T18:49:48.228372+0000","last_peered":"2024-09-10T18:49:48.228372+0000","last_clean":"2024-09-10T18:49:48.228372+0000","last_became_active":"2024-09-10T18:45:24.081050+0000","last_became_peered":"2024-09-10T18:45:24.081050+0000","last_unstale":"2024-09-10T18:49:48.228372+0000","last_undegraded":"2024-09-10T18:49:48.228372+0000","last_fullsized":"2024-09-10T18:49:48.228372+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:32.646988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:32.646988+0000","last_clean_scrub_stamp":"2024-09-10T18:48:32.646988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038989210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"63'11","reported_seq":103,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.227972+0000","last_change":"2024-09-10T18:47:14.034427+0000","last_active":"2024-09-10T18:49:48.227972+0000","last_peered":"2024-09-10T18:49:48.227972+0000","last_clean":"2024-09-10T18:49:48.227972+0000","last_became_active":"2024-09-10T18:45:20.056885+0000","last_became_peered":"2024-09-10T18:45:20.056885+0000","last_unstale":"2024-09-10T18:49:48.227972+0000","last_undegraded":"2024-09-10T18:49:48.227972+0000","last_fullsized":"2024-09-10T18:49:48.227972+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:47:14.034361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:14.034361+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018564,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":97,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.234576+0000","last_change":"2024-09-10T18:48:32.517573+0000","last_active":"2024-09-10T18:49:48.234576+0000","last_peered":"2024-09-10T18:49:48.234576+0000","last_clean":"2024-09-10T18:49:48.234576+0000","last_became_active":"2024-09-10T18:45:17.963542+0000","last_became_peered":"2024-09-10T18:45:17.963542+0000","last_unstale":"2024-09-10T18:49:48.234576+0000","last_undegraded":"2024-09-10T18:49:48.234576+0000","last_fullsized":"2024-09-10T18:49:48.234576+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:32.517513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:32.517513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013400413999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":93,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.237123+0000","last_change":"2024-09-10T18:48:09.725726+0000","last_active":"2024-09-10T18:49:48.237123+0000","last_peered":"2024-09-10T18:49:48.237123+0000","last_clean":"2024-09-10T18:49:48.237123+0000","last_became_active":"2024-09-10T18:45:22.070065+0000","last_became_peered":"2024-09-10T18:45:22.070065+0000","last_unstale":"2024-09-10T18:49:48.237123+0000","last_undegraded":"2024-09-10T18:49:48.237123+0000","last_fullsized":"2024-09-10T18:49:48.237123+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:09.725488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:09.725488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027892020000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":81,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.235475+0000","last_change":"2024-09-10T18:47:05.789893+0000","last_active":"2024-09-10T18:49:48.235475+0000","last_peered":"2024-09-10T18:49:48.235475+0000","last_clean":"2024-09-10T18:49:48.235475+0000","last_became_active":"2024-09-10T18:45:24.072224+0000","last_became_peered":"2024-09-10T18:45:24.072224+0000","last_unstale":"2024-09-10T18:49:48.235475+0000","last_undegraded":"2024-09-10T18:49:48.235475+0000","last_fullsized":"2024-09-10T18:49:48.235475+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:05.789830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:47:05.789830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001271938,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"63'4","reported_seq":101,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.239358+0000","last_change":"2024-09-10T18:48:30.455636+0000","last_active":"2024-09-10T18:49:48.239358+0000","last_peered":"2024-09-10T18:49:48.239358+0000","last_clean":"2024-09-10T18:49:48.239358+0000","last_became_active":"2024-09-10T18:45:20.059205+0000","last_became_peered":"2024-09-10T18:45:20.059205+0000","last_unstale":"2024-09-10T18:49:48.239358+0000","last_undegraded":"2024-09-10T18:49:48.239358+0000","last_fullsized":"2024-09-10T18:49:48.239358+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4","last_scrub_stamp":"2024-09-10T18:48:30.455433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:30.455433+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0078514339999999991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":106,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-10T18:49:50.050334+0000","last_change":"2024-09-10T18:49:50.050306+0000","last_active":"2024-09-10T18:49:50.050334+0000","last_peered":"2024-09-10T18:49:50.050334+0000","last_clean":"2024-09-10T18:49:50.050334+0000","last_became_active":"2024-09-10T18:47:36.415866+0000","last_became_peered":"2024-09-10T18:47:36.415866+0000","last_unstale":"2024-09-10T18:49:50.050334+0000","last_undegraded":"2024-09-10T18:49:50.050334+0000","last_fullsized":"2024-09-10T18:49:50.050334+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:50.050248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:50.050248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:55.274973+0000","scrub_duration":0.0096805610000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":92,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.230546+0000","last_change":"2024-09-10T18:48:27.535769+0000","last_active":"2024-09-10T18:49:48.230546+0000","last_peered":"2024-09-10T18:49:48.230546+0000","last_clean":"2024-09-10T18:49:48.230546+0000","last_became_active":"2024-09-10T18:45:22.072377+0000","last_became_peered":"2024-09-10T18:45:22.072377+0000","last_unstale":"2024-09-10T18:49:48.230546+0000","last_undegraded":"2024-09-10T18:49:48.230546+0000","last_fullsized":"2024-09-10T18:49:48.230546+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:27.535676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_clean_scrub_stamp":"2024-09-10T18:48:27.535676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002291132,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":101,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.231975+0000","last_change":"2024-09-10T18:48:28.711019+0000","last_active":"2024-09-10T18:49:48.231975+0000","last_peered":"2024-09-10T18:49:48.231975+0000","last_clean":"2024-09-10T18:49:48.231975+0000","last_became_active":"2024-09-10T18:47:36.416187+0000","last_became_peered":"2024-09-10T18:47:36.416187+0000","last_unstale":"2024-09-10T18:49:48.231975+0000","last_undegraded":"2024-09-10T18:49:48.231975+0000","last_fullsized":"2024-09-10T18:49:48.231975+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:28.710918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:28.710918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0068877160000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"79'16","reported_seq":119,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.235642+0000","last_change":"2024-09-10T18:49:03.566236+0000","last_active":"2024-09-10T18:49:48.235642+0000","last_peered":"2024-09-10T18:49:48.235642+0000","last_clean":"2024-09-10T18:49:48.235642+0000","last_became_active":"2024-09-10T18:45:20.421479+0000","last_became_peered":"2024-09-10T18:45:20.421479+0000","last_unstale":"2024-09-10T18:49:48.235642+0000","last_undegraded":"2024-09-10T18:49:48.235642+0000","last_fullsized":"2024-09-10T18:49:48.235642+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"79'16","last_scrub_stamp":"2024-09-10T18:49:03.566152+0000","last_deep_scrub":"79'16","last_deep_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_clean_scrub_stamp":"2024-09-10T18:49:03.566152+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:21.649879+0000","scrub_duration":0.0032760770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"56'1","reported_seq":117,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.231026+0000","last_change":"2024-09-10T18:48:20.603918+0000","last_active":"2024-09-10T18:49:48.231026+0000","last_peered":"2024-09-10T18:49:48.231026+0000","last_clean":"2024-09-10T18:49:48.231026+0000","last_became_active":"2024-09-10T18:47:36.421985+0000","last_became_peered":"2024-09-10T18:47:36.421985+0000","last_unstale":"2024-09-10T18:49:48.231026+0000","last_undegraded":"2024-09-10T18:49:48.231026+0000","last_fullsized":"2024-09-10T18:49:48.231026+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:48:20.603805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:20.603805+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:49:49.866219+0000","scrub_duration":0.0039707509999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":47,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.239059+0000","last_change":"2024-09-10T18:48:09.323873+0000","last_active":"2024-09-10T18:49:48.239059+0000","last_peered":"2024-09-10T18:49:48.239059+0000","last_clean":"2024-09-10T18:49:48.239059+0000","last_became_active":"2024-09-10T18:47:57.107149+0000","last_became_peered":"2024-09-10T18:47:57.107149+0000","last_unstale":"2024-09-10T18:49:48.239059+0000","last_undegraded":"2024-09-10T18:49:48.239059+0000","last_fullsized":"2024-09-10T18:49:48.239059+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:09.323794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:09.323794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056521669999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":91,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.236163+0000","last_change":"2024-09-10T18:48:16.410239+0000","last_active":"2024-09-10T18:49:48.236163+0000","last_peered":"2024-09-10T18:49:48.236163+0000","last_clean":"2024-09-10T18:49:48.236163+0000","last_became_active":"2024-09-10T18:45:24.264818+0000","last_became_peered":"2024-09-10T18:45:24.264818+0000","last_unstale":"2024-09-10T18:49:48.236163+0000","last_undegraded":"2024-09-10T18:49:48.236163+0000","last_fullsized":"2024-09-10T18:49:48.236163+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:16.410071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:48:16.410071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0061840250000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"78'2","reported_seq":107,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.233789+0000","last_change":"2024-09-10T18:48:10.610714+0000","last_active":"2024-09-10T18:49:48.233789+0000","last_peered":"2024-09-10T18:49:48.233789+0000","last_clean":"2024-09-10T18:49:48.233789+0000","last_became_active":"2024-09-10T18:47:36.418702+0000","last_became_peered":"2024-09-10T18:47:36.418702+0000","last_unstale":"2024-09-10T18:49:48.233789+0000","last_undegraded":"2024-09-10T18:49:48.233789+0000","last_fullsized":"2024-09-10T18:49:48.233789+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'2","last_scrub_stamp":"2024-09-10T18:48:10.610652+0000","last_deep_scrub":"78'2","last_deep_scrub_stamp":"2024-09-10T18:46:37.581130+0000","last_clean_scrub_stamp":"2024-09-10T18:48:10.610652+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002045275,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":96,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.233365+0000","last_change":"2024-09-10T18:49:07.088445+0000","last_active":"2024-09-10T18:49:48.233365+0000","last_peered":"2024-09-10T18:49:48.233365+0000","last_clean":"2024-09-10T18:49:48.233365+0000","last_became_active":"2024-09-10T18:45:17.962108+0000","last_became_peered":"2024-09-10T18:45:17.962108+0000","last_unstale":"2024-09-10T18:49:48.233365+0000","last_undegraded":"2024-09-10T18:49:48.233365+0000","last_fullsized":"2024-09-10T18:49:48.233365+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:07.088371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:07.088371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:26.877956+0000","scrub_duration":0.0013360100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"63'10","reported_seq":110,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.229205+0000","last_change":"2024-09-10T18:48:23.844841+0000","last_active":"2024-09-10T18:49:48.229205+0000","last_peered":"2024-09-10T18:49:48.229205+0000","last_clean":"2024-09-10T18:49:48.229205+0000","last_became_active":"2024-09-10T18:45:20.421094+0000","last_became_peered":"2024-09-10T18:45:20.421094+0000","last_unstale":"2024-09-10T18:49:48.229205+0000","last_undegraded":"2024-09-10T18:49:48.229205+0000","last_fullsized":"2024-09-10T18:49:48.229205+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:48:23.844741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:23.844741+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025648530000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":167,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-10T18:49:50.465696+0000","last_change":"2024-09-10T18:49:50.465667+0000","last_active":"2024-09-10T18:49:50.465696+0000","last_peered":"2024-09-10T18:49:50.465696+0000","last_clean":"2024-09-10T18:49:50.465696+0000","last_became_active":"2024-09-10T18:45:22.073281+0000","last_became_peered":"2024-09-10T18:45:22.073281+0000","last_unstale":"2024-09-10T18:49:50.465696+0000","last_undegraded":"2024-09-10T18:49:50.465696+0000","last_fullsized":"2024-09-10T18:49:50.465696+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:49:50.465613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:49:50.465613+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:02.944119+0000","scrub_duration":0.0017767359999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":91,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.233389+0000","last_change":"2024-09-10T18:49:11.422481+0000","last_active":"2024-09-10T18:49:48.233389+0000","last_peered":"2024-09-10T18:49:48.233389+0000","last_clean":"2024-09-10T18:49:48.233389+0000","last_became_active":"2024-09-10T18:45:24.082318+0000","last_became_peered":"2024-09-10T18:45:24.082318+0000","last_unstale":"2024-09-10T18:49:48.233389+0000","last_undegraded":"2024-09-10T18:49:48.233389+0000","last_fullsized":"2024-09-10T18:49:48.233389+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:11.422279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:03.475299+0000","last_clean_scrub_stamp":"2024-09-10T18:49:11.422279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:39.824521+0000","scrub_duration":0.0030495840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":97,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.235994+0000","last_change":"2024-09-10T18:49:07.526539+0000","last_active":"2024-09-10T18:49:48.235994+0000","last_peered":"2024-09-10T18:49:48.235994+0000","last_clean":"2024-09-10T18:49:48.235994+0000","last_became_active":"2024-09-10T18:45:17.965707+0000","last_became_peered":"2024-09-10T18:45:17.965707+0000","last_unstale":"2024-09-10T18:49:48.235994+0000","last_undegraded":"2024-09-10T18:49:48.235994+0000","last_fullsized":"2024-09-10T18:49:48.235994+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:07.526460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:07.526460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:10.144795+0000","scrub_duration":0.010955793,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"63'11","reported_seq":113,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.230992+0000","last_change":"2024-09-10T18:48:23.546688+0000","last_active":"2024-09-10T18:49:48.230992+0000","last_peered":"2024-09-10T18:49:48.230992+0000","last_clean":"2024-09-10T18:49:48.230992+0000","last_became_active":"2024-09-10T18:45:20.058717+0000","last_became_peered":"2024-09-10T18:45:20.058717+0000","last_unstale":"2024-09-10T18:49:48.230992+0000","last_undegraded":"2024-09-10T18:49:48.230992+0000","last_fullsized":"2024-09-10T18:49:48.230992+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:48:23.546598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:48:23.546598+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0057630149999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":103,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.230128+0000","last_change":"2024-09-10T18:48:58.543128+0000","last_active":"2024-09-10T18:49:48.230128+0000","last_peered":"2024-09-10T18:49:48.230128+0000","last_clean":"2024-09-10T18:49:48.230128+0000","last_became_active":"2024-09-10T18:47:36.418767+0000","last_became_peered":"2024-09-10T18:47:36.418767+0000","last_unstale":"2024-09-10T18:49:48.230128+0000","last_undegraded":"2024-09-10T18:49:48.230128+0000","last_fullsized":"2024-09-10T18:49:48.230128+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:48:58.542812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:48:58.542812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:01.428535+0000","scrub_duration":0.010435696,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":90,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.228770+0000","last_change":"2024-09-10T18:49:06.423032+0000","last_active":"2024-09-10T18:49:48.228770+0000","last_peered":"2024-09-10T18:49:48.228770+0000","last_clean":"2024-09-10T18:49:48.228770+0000","last_became_active":"2024-09-10T18:45:24.267251+0000","last_became_peered":"2024-09-10T18:45:24.267251+0000","last_unstale":"2024-09-10T18:49:48.228770+0000","last_undegraded":"2024-09-10T18:49:48.228770+0000","last_fullsized":"2024-09-10T18:49:48.228770+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:06.422952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:06.422952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:33.673505+0000","scrub_duration":0.0017661879999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"56'1","reported_seq":105,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.230946+0000","last_change":"2024-09-10T18:48:31.488364+0000","last_active":"2024-09-10T18:49:48.230946+0000","last_peered":"2024-09-10T18:49:48.230946+0000","last_clean":"2024-09-10T18:49:48.230946+0000","last_became_active":"2024-09-10T18:45:17.966286+0000","last_became_peered":"2024-09-10T18:45:17.966286+0000","last_unstale":"2024-09-10T18:49:48.230946+0000","last_undegraded":"2024-09-10T18:49:48.230946+0000","last_fullsized":"2024-09-10T18:49:48.230946+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:48:31.488272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:48:31.488272+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0028000949999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"63'15","reported_seq":101,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.234141+0000","last_change":"2024-09-10T18:47:36.446154+0000","last_active":"2024-09-10T18:49:48.234141+0000","last_peered":"2024-09-10T18:49:48.234141+0000","last_clean":"2024-09-10T18:49:48.234141+0000","last_became_active":"2024-09-10T18:47:36.424373+0000","last_became_peered":"2024-09-10T18:47:36.424373+0000","last_unstale":"2024-09-10T18:49:48.234141+0000","last_undegraded":"2024-09-10T18:49:48.234141+0000","last_fullsized":"2024-09-10T18:49:48.234141+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:47:34.610915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:47:34.610915+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":83,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.233699+0000","last_change":"2024-09-10T18:47:09.520829+0000","last_active":"2024-09-10T18:49:48.233699+0000","last_peered":"2024-09-10T18:49:48.233699+0000","last_clean":"2024-09-10T18:49:48.233699+0000","last_became_active":"2024-09-10T18:45:22.059794+0000","last_became_peered":"2024-09-10T18:45:22.059794+0000","last_unstale":"2024-09-10T18:49:48.233699+0000","last_undegraded":"2024-09-10T18:49:48.233699+0000","last_fullsized":"2024-09-10T18:49:48.233699+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:09.520768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:47:09.520768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016284120000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":81,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.236230+0000","last_change":"2024-09-10T18:46:57.811148+0000","last_active":"2024-09-10T18:49:48.236230+0000","last_peered":"2024-09-10T18:49:48.236230+0000","last_clean":"2024-09-10T18:49:48.236230+0000","last_became_active":"2024-09-10T18:45:24.264427+0000","last_became_peered":"2024-09-10T18:45:24.264427+0000","last_unstale":"2024-09-10T18:49:48.236230+0000","last_undegraded":"2024-09-10T18:49:48.236230+0000","last_fullsized":"2024-09-10T18:49:48.236230+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:46:57.811074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:46:57.811074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013689419999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":97,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.236539+0000","last_change":"2024-09-10T18:49:14.365927+0000","last_active":"2024-09-10T18:49:48.236539+0000","last_peered":"2024-09-10T18:49:48.236539+0000","last_clean":"2024-09-10T18:49:48.236539+0000","last_became_active":"2024-09-10T18:45:17.963257+0000","last_became_peered":"2024-09-10T18:45:17.963257+0000","last_unstale":"2024-09-10T18:49:48.236539+0000","last_undegraded":"2024-09-10T18:49:48.236539+0000","last_fullsized":"2024-09-10T18:49:48.236539+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:14.365492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:14.365492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:29.522576+0000","scrub_duration":0.002395463,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"63'12","reported_seq":118,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.235026+0000","last_change":"2024-09-10T18:49:13.387562+0000","last_active":"2024-09-10T18:49:48.235026+0000","last_peered":"2024-09-10T18:49:48.235026+0000","last_clean":"2024-09-10T18:49:48.235026+0000","last_became_active":"2024-09-10T18:45:20.058391+0000","last_became_peered":"2024-09-10T18:45:20.058391+0000","last_unstale":"2024-09-10T18:49:48.235026+0000","last_undegraded":"2024-09-10T18:49:48.235026+0000","last_fullsized":"2024-09-10T18:49:48.235026+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:49:13.387425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:49:13.387425+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:50:19.344236+0000","scrub_duration":0.0091962799999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":102,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T18:49:48.229239+0000","last_change":"2024-09-10T18:47:36.416505+0000","last_active":"2024-09-10T18:49:48.229239+0000","last_peered":"2024-09-10T18:49:48.229239+0000","last_clean":"2024-09-10T18:49:48.229239+0000","last_became_active":"2024-09-10T18:47:36.416131+0000","last_became_peered":"2024-09-10T18:47:36.416131+0000","last_unstale":"2024-09-10T18:49:48.229239+0000","last_undegraded":"2024-09-10T18:49:48.229239+0000","last_fullsized":"2024-09-10T18:49:48.229239+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:47:34.958889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:47:34.958889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001401296,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":6186270744,"num_objects":94396,"num_object_clones":0,"num_object_copies":283188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94396,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":178040,"num_write_kb":8717889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18559479808,"data_stored":18559467592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76997,"ondisk_log_size":76997,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":697,"num_read_kb":454,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":1238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":428,"ondisk_log_size":428,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":263,"num_read_kb":258,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299451,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2594652,"kb_used_data":2284752,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":91150500,"statfs":{"total":95995035648,"available":93338112000,"internally_reserved":0,"allocated":2339586048,"data_stored":2339184687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:49:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":0.94499999999999995,"15min":0.65300000000000002},"min":{"1min":0.219,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":3.9929999999999999,"5min":31.805,"15min":31.805},"last":0.33200000000000002},{"interface":"front","average":{"1min":1.8640000000000001,"5min":1.081,"15min":0.68400000000000005},"min":{"1min":0.29399999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":5.4119999999999999,"5min":31.212,"15min":31.212},"last":0.23899999999999999}]},{"osd":1,"last update":"Tue Sep 10 18:49:10 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.55200000000000005,"15min":0.502},"min":{"1min":0.30199999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":2.169,"5min":2.169,"15min":2.169},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.0640000000000001,"5min":0.624,"15min":0.55400000000000005},"min":{"1min":0.255,"5min":0.25,"15min":0.25},"max":{"1min":3.036,"5min":3.036,"15min":3.036},"last":0.70199999999999996}]},{"osd":2,"last update":"Tue Sep 10 18:49:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":0.60599999999999998,"15min":0.52600000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":5.1399999999999997,"5min":5.1399999999999997,"15min":5.1399999999999997},"last":0.316},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.60699999999999998,"15min":0.54600000000000004},"min":{"1min":0.38200000000000001,"5min":0.31,"15min":0.31},"max":{"1min":3.0209999999999999,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":0.42099999999999999}]},{"osd":3,"last update":"Tue Sep 10 18:49:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":0.95699999999999996,"15min":0.65700000000000003},"min":{"1min":0.26900000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":3.2120000000000002,"5min":31.478000000000002,"15min":31.478000000000002},"last":0.40600000000000003},{"interface":"front","average":{"1min":1.581,"5min":1.1619999999999999,"15min":0.73299999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":5.1020000000000003,"5min":33.731999999999999,"15min":33.731999999999999},"last":0.30299999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:49:10 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.45100000000000001,"15min":0.40699999999999997},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":2.7559999999999998,"5min":2.7559999999999998,"15min":2.7559999999999998},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.438,"15min":0.39600000000000002},"min":{"1min":0.29599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.25800000000000001}]},{"osd":5,"last update":"Tue Sep 10 18:49:10 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.46899999999999997,"15min":0.43099999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":2.7240000000000002,"5min":2.7240000000000002,"15min":2.7240000000000002},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.45000000000000001,"15min":0.42699999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.38600000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:49:10 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.52900000000000003,"15min":0.47499999999999998},"min":{"1min":0.34499999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":2.8199999999999998,"5min":2.8199999999999998,"15min":2.8199999999999998},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.52400000000000002,"15min":0.46300000000000002},"min":{"1min":0.33700000000000002,"5min":0.309,"15min":0.309},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.46000000000000002}]}]},{"osd":6,"up_from":43,"seq":184683593792,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2603052,"kb_used_data":2293212,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":91142100,"statfs":{"total":95995035648,"available":93329510400,"internally_reserved":0,"allocated":2348249088,"data_stored":2347836007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:49:46 2024","interfaces":[{"interface":"back","average":{"1min":4.3280000000000003,"5min":1.423,"15min":0.77900000000000003},"min":{"1min":0.23499999999999999,"5min":0.222,"15min":0.222},"max":{"1min":46.384999999999998,"5min":46.384999999999998,"15min":46.384999999999998},"last":4.9420000000000002},{"interface":"front","average":{"1min":4.3879999999999999,"5min":1.3680000000000001,"15min":0.78300000000000003},"min":{"1min":0.317,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":48.372999999999998,"5min":48.372999999999998,"15min":48.372999999999998},"last":0.52800000000000002}]},{"osd":1,"last update":"Tue Sep 10 18:49:46 2024","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":0.73599999999999999,"15min":0.57099999999999995},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":6.1319999999999997,"5min":6.1319999999999997,"15min":6.1319999999999997},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.65300000000000002,"15min":0.52500000000000002},"min":{"1min":0.29699999999999999,"5min":0.214,"15min":0.214},"max":{"1min":3.407,"5min":3.6280000000000001,"15min":3.6280000000000001},"last":0.58699999999999997}]},{"osd":2,"last update":"Tue Sep 10 18:49:46 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.67700000000000005,"15min":0.54600000000000004},"min":{"1min":0.32600000000000001,"5min":0.312,"15min":0.312},"max":{"1min":4.4740000000000002,"5min":4.4740000000000002,"15min":4.4740000000000002},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.56599999999999995,"15min":0.51500000000000001},"min":{"1min":0.28499999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":2.9900000000000002,"5min":2.9900000000000002,"15min":2.9900000000000002},"last":1.0209999999999999}]},{"osd":3,"last update":"Tue Sep 10 18:49:46 2024","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":0.65600000000000003,"15min":0.54000000000000004},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":4.4889999999999999,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":1.631,"5min":0.85199999999999998,"15min":0.624},"min":{"1min":0.32600000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":5.3079999999999998,"5min":5.3079999999999998,"15min":5.3079999999999998},"last":0.57099999999999995}]},{"osd":4,"last update":"Tue Sep 10 18:49:46 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.47099999999999997,"15min":0.42699999999999999},"min":{"1min":0.22800000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":4.2969999999999997,"5min":4.2969999999999997,"15min":4.2969999999999997},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.47899999999999998,"15min":0.41499999999999998},"min":{"1min":0.26200000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":4.3209999999999997,"5min":4.3209999999999997,"15min":4.3209999999999997},"last":0.377}]},{"osd":5,"last update":"Tue Sep 10 18:49:46 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.499,"15min":0.442},"min":{"1min":0.28299999999999997,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":4.4210000000000003,"5min":4.4210000000000003,"15min":4.4210000000000003},"last":0.63},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.51500000000000001,"15min":0.45300000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":4.4500000000000002,"5min":4.4500000000000002,"15min":4.4500000000000002},"last":0.99399999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:49:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.52400000000000002,"15min":0.48699999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.367,"5min":1.556,"15min":1.556},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.53700000000000003,"15min":0.50800000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.3320000000000001,"5min":1.883,"15min":1.883},"last":2.0840000000000001}]}]},{"osd":5,"up_from":37,"seq":158913790020,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1833296,"kb_used_data":1523464,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":91911856,"statfs":{"total":95995035648,"available":94117740544,"internally_reserved":0,"allocated":1560027136,"data_stored":1559632891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:49:23 2024","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":0.66900000000000004,"15min":0.57499999999999996},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":3.9169999999999998,"5min":3.9169999999999998,"15min":3.9169999999999998},"last":1.575},{"interface":"front","average":{"1min":1.163,"5min":0.77300000000000002,"15min":0.58999999999999997},"min":{"1min":0.33600000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":4.3369999999999997,"5min":6.742,"15min":6.742},"last":0.36599999999999999}]},{"osd":1,"last update":"Tue Sep 10 18:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.64600000000000002,"15min":0.59099999999999997},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":3.5150000000000001,"5min":3.5150000000000001,"15min":3.5150000000000001},"last":1.335},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.75600000000000001,"15min":0.61299999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":3.5030000000000001,"5min":6.6989999999999998,"15min":6.6989999999999998},"last":1.294}]},{"osd":2,"last update":"Tue Sep 10 18:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.55500000000000005,"15min":0.53200000000000003},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":4.0730000000000004,"5min":4.0730000000000004,"15min":4.0730000000000004},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.70099999999999996,"15min":0.59599999999999997},"min":{"1min":0.27900000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":2.2970000000000002,"5min":6.7210000000000001,"15min":6.7210000000000001},"last":0.34000000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:49:23 2024","interfaces":[{"interface":"back","average":{"1min":1.627,"5min":0.84399999999999997,"15min":0.63300000000000001},"min":{"1min":0.29899999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":6.5190000000000001,"5min":6.5190000000000001,"15min":6.5190000000000001},"last":1.071},{"interface":"front","average":{"1min":1.3460000000000001,"5min":0.85699999999999998,"15min":0.65100000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":3.4239999999999999,"5min":5.8099999999999996,"15min":5.8099999999999996},"last":2.702}]},{"osd":4,"last update":"Tue Sep 10 18:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.41599999999999998,"15min":0.39400000000000002},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":2.133,"5min":2.133,"15min":2.133},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.40600000000000003,"15min":0.40400000000000003},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":2.153,"5min":2.153,"15min":2.153},"last":0.28899999999999998}]},{"osd":6,"last update":"Tue Sep 10 18:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.57499999999999996,"15min":0.46700000000000003},"min":{"1min":0.28799999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":5.0709999999999997,"5min":5.0709999999999997,"15min":5.0709999999999997},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.60699999999999998,"15min":0.50800000000000001},"min":{"1min":0.33000000000000002,"5min":0.31,"15min":0.31},"max":{"1min":5.0259999999999998,"5min":5.0259999999999998,"15min":5.0259999999999998},"last":0.35399999999999998}]},{"osd":7,"last update":"Tue Sep 10 18:49:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.48599999999999999,"15min":0.47999999999999998},"min":{"1min":0.28399999999999997,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":2.1139999999999999,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.47299999999999998,"15min":0.47599999999999998},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.30099999999999999}]}]},{"osd":4,"up_from":30,"seq":128849018954,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2203040,"kb_used_data":1901520,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":91542112,"statfs":{"total":95995035648,"available":93739122688,"internally_reserved":0,"allocated":1947156480,"data_stored":1946754426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:49:05 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.624,"15min":0.55800000000000005},"min":{"1min":0.215,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":7.8730000000000002,"5min":7.8730000000000002,"15min":7.8730000000000002},"last":0.21099999999999999},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.58799999999999997,"15min":0.55800000000000005},"min":{"1min":0.23999999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":3.3279999999999998,"5min":3.3279999999999998,"15min":3.3279999999999998},"last":0.32000000000000001}]},{"osd":1,"last update":"Tue Sep 10 18:49:05 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.624,"15min":0.58999999999999997},"min":{"1min":0.27900000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":3.1909999999999998,"5min":3.1909999999999998,"15min":3.1909999999999998},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.63200000000000001,"15min":0.56000000000000005},"min":{"1min":0.28399999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":2.0760000000000001,"5min":2.589,"15min":2.589},"last":0.27800000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:49:05 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.56200000000000006,"15min":0.54600000000000004},"min":{"1min":0.29099999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":2.0800000000000001,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.54900000000000004,"15min":0.55500000000000005},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":2.8109999999999999,"5min":2.8109999999999999,"15min":2.8109999999999999},"last":0.60299999999999998}]},{"osd":3,"last update":"Tue Sep 10 18:49:05 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":0.67100000000000004,"15min":0.57699999999999996},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":3.512,"5min":3.512,"15min":3.512},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.61499999999999999,"15min":0.56699999999999995},"min":{"1min":0.32400000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":3.109,"5min":3.109,"15min":3.109},"last":0.29499999999999998}]},{"osd":5,"last update":"Tue Sep 10 18:49:28 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.41999999999999998,"15min":0.37},"min":{"1min":0.25700000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.78700000000000003,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.42199999999999999,"15min":0.376},"min":{"1min":0.247,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.39300000000000002}]},{"osd":6,"last update":"Tue Sep 10 18:49:46 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.50900000000000001,"15min":0.45000000000000001},"min":{"1min":0.29699999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.335,"5min":3.5529999999999999,"15min":3.5529999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.52400000000000002,"15min":0.432},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":2.02,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":0.36699999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:49:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.52400000000000002,"15min":0.50800000000000001},"min":{"1min":0.34899999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":2.8940000000000001,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.47899999999999998,"15min":0.45900000000000002},"min":{"1min":0.30299999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":2.0219999999999998,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":0.57099999999999995}]}]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4508072,"kb_used_data":4194012,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":89237080,"statfs":{"total":95995035648,"available":91378769920,"internally_reserved":0,"allocated":4294668288,"data_stored":4294253779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,4,2,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:49:38 2024","interfaces":[{"interface":"back","average":{"1min":4.2969999999999997,"5min":1.4279999999999999,"15min":0.73399999999999999},"min":{"1min":0.35699999999999998,"5min":0.215,"15min":0.215},"max":{"1min":47.601999999999997,"5min":47.601999999999997,"15min":47.601999999999997},"last":2.2639999999999998},{"interface":"front","average":{"1min":4.2130000000000001,"5min":1.431,"15min":0.747},"min":{"1min":0.26700000000000002,"5min":0.217,"15min":0.217},"max":{"1min":46.787999999999997,"5min":46.787999999999997,"15min":46.787999999999997},"last":2.492}]},{"osd":1,"last update":"Tue Sep 10 18:49:38 2024","interfaces":[{"interface":"back","average":{"1min":1.6679999999999999,"5min":0.91900000000000004,"15min":0.58099999999999996},"min":{"1min":0.25900000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":5.827,"5min":5.827,"15min":5.827},"last":5.476},{"interface":"front","average":{"1min":1.8140000000000001,"5min":0.90700000000000003,"15min":0.58099999999999996},"min":{"1min":0.27100000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":8.0030000000000001,"5min":8.0030000000000001,"15min":8.0030000000000001},"last":3.222}]},{"osd":2,"last update":"Tue Sep 10 18:49:38 2024","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.70099999999999996,"15min":0.52800000000000002},"min":{"1min":0.25900000000000001,"5min":0.246,"15min":0.246},"max":{"1min":4.5220000000000002,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":2.1890000000000001},{"interface":"front","average":{"1min":1.3939999999999999,"5min":0.65300000000000002,"15min":0.51100000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":6.3170000000000002,"5min":6.3170000000000002,"15min":6.3170000000000002},"last":0.97399999999999998}]},{"osd":4,"last update":"Tue Sep 10 18:49:02 2024","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":0.73599999999999999,"15min":0.60799999999999998},"min":{"1min":0.36399999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":6.3440000000000003,"5min":6.3440000000000003,"15min":6.3440000000000003},"last":0.439},{"interface":"front","average":{"1min":1.2789999999999999,"5min":0.74299999999999999,"15min":0.63100000000000001},"min":{"1min":0.313,"5min":0.311,"15min":0.311},"max":{"1min":6.29,"5min":6.29,"15min":6.29},"last":0.38400000000000001}]},{"osd":5,"last update":"Tue Sep 10 18:49:33 2024","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":0.877,"15min":0.65300000000000002},"min":{"1min":0.36599999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":4.8179999999999996,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":2.0219999999999998},{"interface":"front","average":{"1min":1.131,"5min":0.76400000000000001,"15min":0.63},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":4.5439999999999996,"5min":4.5439999999999996,"15min":4.5439999999999996},"last":2.3090000000000002}]},{"osd":6,"last update":"Tue Sep 10 18:49:49 2024","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":0.90100000000000002,"15min":0.67700000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":4.6159999999999997,"5min":4.6159999999999997,"15min":4.6159999999999997},"last":2.456},{"interface":"front","average":{"1min":1.6319999999999999,"5min":0.85799999999999998,"15min":0.65700000000000003},"min":{"1min":0.35299999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":6.3719999999999999,"5min":6.3719999999999999,"15min":6.3719999999999999},"last":0.46800000000000003}]},{"osd":7,"last update":"Tue Sep 10 18:49:15 2024","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":0.81000000000000005,"15min":0.624},"min":{"1min":0.38600000000000001,"5min":0.375,"15min":0.375},"max":{"1min":4.71,"5min":4.71,"15min":4.71},"last":2.048},{"interface":"front","average":{"1min":1.3859999999999999,"5min":0.77000000000000002,"15min":0.64000000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":3.8849999999999998,"5min":3.8849999999999998,"15min":3.8849999999999998},"last":2.3340000000000001}]}]},{"osd":2,"up_from":0,"seq":77309411411,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3355760,"kb_used_data":3058408,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":90389392,"statfs":{"total":95995035648,"available":92558737408,"internally_reserved":0,"allocated":3131809792,"data_stored":3131383602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:48:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":0.54200000000000004,"15min":0.38600000000000001},"min":{"1min":0.38300000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":3.1520000000000001,"5min":3.1520000000000001,"15min":3.1520000000000001},"last":1.994},{"interface":"front","average":{"1min":1.502,"5min":0.60999999999999999,"15min":0.41799999999999998},"min":{"1min":0.442,"5min":0.252,"15min":0.21199999999999999},"max":{"1min":4.2450000000000001,"5min":4.2450000000000001,"15min":4.2450000000000001},"last":0.65300000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.54300000000000004,"15min":0.41799999999999998},"min":{"1min":0.26700000000000002,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":3.1259999999999999,"5min":3.8479999999999999,"15min":3.8479999999999999},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.502,"15min":0.41799999999999998},"min":{"1min":0.36799999999999999,"5min":0.248,"15min":0.248},"max":{"1min":2.1459999999999999,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":0.36499999999999999}]},{"osd":3,"last update":"Tue Sep 10 18:49:47 2024","interfaces":[{"interface":"back","average":{"1min":1.754,"5min":0.90900000000000003,"15min":0.60899999999999999},"min":{"1min":0.33700000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":5.0270000000000001,"5min":5.0270000000000001,"15min":5.0270000000000001},"last":1.444},{"interface":"front","average":{"1min":2.375,"5min":1.1659999999999999,"15min":0.67000000000000004},"min":{"1min":0.40100000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":7.2690000000000001,"5min":7.2690000000000001,"15min":7.2690000000000001},"last":0.61699999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:49:04 2024","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":0.74099999999999999,"15min":0.61499999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":3.1520000000000001,"5min":4.1520000000000001,"15min":4.1520000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":1.1839999999999999,"5min":0.68400000000000005,"15min":0.57999999999999996},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":3.9540000000000002,"5min":3.9540000000000002,"15min":3.9540000000000002},"last":1.381}]},{"osd":5,"last update":"Tue Sep 10 18:49:25 2024","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":0.753,"15min":0.65400000000000003},"min":{"1min":0.33500000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":4.5869999999999997,"5min":4.5869999999999997,"15min":4.5869999999999997},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.476,"5min":0.84799999999999998,"15min":0.71199999999999997},"min":{"1min":0.311,"5min":0.31,"15min":0.31},"max":{"1min":3.4020000000000001,"5min":4.1130000000000004,"15min":4.1130000000000004},"last":3.2280000000000002}]},{"osd":6,"last update":"Tue Sep 10 18:49:47 2024","interfaces":[{"interface":"back","average":{"1min":3.5529999999999999,"5min":1.274,"15min":0.84599999999999997},"min":{"1min":0.46400000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":38.039999999999999,"5min":38.039999999999999,"15min":38.039999999999999},"last":1.5209999999999999},{"interface":"front","average":{"1min":3.9390000000000001,"5min":1.4139999999999999,"15min":0.92100000000000004},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":38.100999999999999,"5min":38.100999999999999,"15min":38.100999999999999},"last":3.2709999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:49:19 2024","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":0.73799999999999999,"15min":0.67800000000000005},"min":{"1min":0.40300000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":3.1930000000000001,"5min":3.1930000000000001,"15min":3.1930000000000001},"last":1.47},{"interface":"front","average":{"1min":1.4390000000000001,"5min":0.85999999999999999,"15min":0.71799999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":3.4279999999999999,"5min":4.1989999999999998,"15min":4.1989999999999998},"last":0.68200000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3334828,"kb_used_data":3029148,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":90410324,"statfs":{"total":95995035648,"available":92580171776,"internally_reserved":0,"allocated":3101847552,"data_stored":3101433359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,0,4,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 18:48:55 2024","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":0.57799999999999996,"15min":0.38800000000000001},"min":{"1min":0.186,"5min":0.186,"15min":0.17699999999999999},"max":{"1min":4.7599999999999998,"5min":4.7599999999999998,"15min":4.7599999999999998},"last":3.0379999999999998},{"interface":"front","average":{"1min":1.575,"5min":0.63300000000000001,"15min":0.41399999999999998},"min":{"1min":0.38,"5min":0.187,"15min":0.187},"max":{"1min":4.5110000000000001,"5min":4.5110000000000001,"15min":4.5110000000000001},"last":3.4079999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:49:26 2024","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":0.67600000000000005,"15min":0.47999999999999998},"min":{"1min":0.26900000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":4.1719999999999997,"5min":4.6509999999999998,"15min":4.6509999999999998},"last":3.4399999999999999},{"interface":"front","average":{"1min":1.0669999999999999,"5min":0.62,"15min":0.443},"min":{"1min":0.251,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":3.266,"5min":4.6230000000000002,"15min":4.6230000000000002},"last":1.756}]},{"osd":3,"last update":"Tue Sep 10 18:49:41 2024","interfaces":[{"interface":"back","average":{"1min":3.0760000000000001,"5min":1.1120000000000001,"15min":0.63700000000000001},"min":{"1min":0.45500000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":8.0269999999999992,"5min":8.0269999999999992,"15min":8.0269999999999992},"last":3.069},{"interface":"front","average":{"1min":2.3809999999999998,"5min":0.99299999999999999,"15min":0.60699999999999998},"min":{"1min":0.32000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":7.0629999999999997,"5min":7.0629999999999997,"15min":7.0629999999999997},"last":1.7949999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:49:03 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":0.69099999999999995,"15min":0.58999999999999997},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":4.0519999999999996,"5min":4.0519999999999996,"15min":4.0519999999999996},"last":1.6970000000000001},{"interface":"front","average":{"1min":1.3700000000000001,"5min":0.78800000000000003,"15min":0.60799999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":3.8330000000000002,"5min":4.6189999999999998,"15min":4.6189999999999998},"last":2.9769999999999999}]},{"osd":5,"last update":"Tue Sep 10 18:49:26 2024","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":0.80400000000000005,"15min":0.64000000000000001},"min":{"1min":0.33800000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":4.1319999999999997,"5min":4.1319999999999997,"15min":4.1319999999999997},"last":1.6359999999999999},{"interface":"front","average":{"1min":1.3779999999999999,"5min":0.84699999999999998,"15min":0.64500000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":3.673,"5min":4.6429999999999998,"15min":4.6429999999999998},"last":3.3239999999999998}]},{"osd":6,"last update":"Tue Sep 10 18:48:49 2024","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":0.72999999999999998,"15min":0.61199999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":3.855,"5min":3.855,"15min":3.855},"last":2.9359999999999999},{"interface":"front","average":{"1min":1.0780000000000001,"5min":0.76400000000000001,"15min":0.77900000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":2.5830000000000002,"5min":4.6630000000000003,"15min":4.6630000000000003},"last":1.7290000000000001}]},{"osd":7,"last update":"Tue Sep 10 18:49:12 2024","interfaces":[{"interface":"back","average":{"1min":3.859,"5min":1.2589999999999999,"15min":0.79100000000000004},"min":{"1min":0.34100000000000003,"5min":0.311,"15min":0.311},"max":{"1min":49.313000000000002,"5min":49.313000000000002,"15min":49.313000000000002},"last":3.363},{"interface":"front","average":{"1min":3.8650000000000002,"5min":1.302,"15min":0.82899999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":49.063000000000002,"5min":49.063000000000002,"15min":49.063000000000002},"last":3.008}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3069382656,"data_stored":3069378584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3097759744,"data_stored":3097755672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4249092096,"data_stored":4249092096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1936588800,"data_stored":1936588800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1558446080,"data_stored":1558446080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2331774976,"data_stored":2331770904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2316435456,"data_stored":2316435456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T18:49:57.439 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2024-09-10T18:49:57.439 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd pool set .mgr pg_num 11 2024-09-10T18:49:57.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:57.440 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:57.440 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:57.478 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:57 smithi018 ceph-mon[40196]: 5.14 scrub starts 2024-09-10T18:49:57.478 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:57 smithi018 ceph-mon[40196]: 5.14 scrub ok 2024-09-10T18:49:57.478 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:57 smithi018 ceph-mon[40196]: 4.3 scrub starts 2024-09-10T18:49:57.479 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:57 smithi018 ceph-mon[40196]: 4.3 scrub ok 2024-09-10T18:49:57.479 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:57 smithi018 ceph-mon[40196]: 4.15 scrub starts 2024-09-10T18:49:57.479 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:57 smithi018 ceph-mon[40196]: 4.15 scrub ok 2024-09-10T18:49:57.479 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:57 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:57.479 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:57 smithi018 ceph-mon[40196]: 4.13 scrub starts 2024-09-10T18:49:57.479 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:57 smithi018 ceph-mon[40196]: 4.13 scrub ok 2024-09-10T18:49:57.479 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:57 smithi018 ceph-mon[40196]: 7.0 scrub ok 2024-09-10T18:49:57.479 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:57 smithi018 ceph-mon[29336]: 5.14 scrub starts 2024-09-10T18:49:57.479 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:57 smithi018 ceph-mon[29336]: 5.14 scrub ok 2024-09-10T18:49:57.480 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:57 smithi018 ceph-mon[29336]: 4.3 scrub starts 2024-09-10T18:49:57.480 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:57 smithi018 ceph-mon[29336]: 4.3 scrub ok 2024-09-10T18:49:57.480 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:57 smithi018 ceph-mon[29336]: 4.15 scrub starts 2024-09-10T18:49:57.480 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:57 smithi018 ceph-mon[29336]: 4.15 scrub ok 2024-09-10T18:49:57.480 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:57 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:57.480 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:57 smithi018 ceph-mon[29336]: 4.13 scrub starts 2024-09-10T18:49:57.480 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:57 smithi018 ceph-mon[29336]: 4.13 scrub ok 2024-09-10T18:49:57.480 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:57 smithi018 ceph-mon[29336]: 7.0 scrub ok 2024-09-10T18:49:57.520 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T18:49:57.520 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:49:57.520 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:57 smithi103 ceph-mon[33163]: 5.14 scrub starts 2024-09-10T18:49:57.520 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:57 smithi103 ceph-mon[33163]: 5.14 scrub ok 2024-09-10T18:49:57.520 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:57 smithi103 ceph-mon[33163]: 4.3 scrub starts 2024-09-10T18:49:57.520 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:57 smithi103 ceph-mon[33163]: 4.3 scrub ok 2024-09-10T18:49:57.520 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:57 smithi103 ceph-mon[33163]: 4.15 scrub starts 2024-09-10T18:49:57.521 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:57 smithi103 ceph-mon[33163]: 4.15 scrub ok 2024-09-10T18:49:57.521 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:57 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:57.521 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:57 smithi103 ceph-mon[33163]: 4.13 scrub starts 2024-09-10T18:49:57.521 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:57 smithi103 ceph-mon[33163]: 4.13 scrub ok 2024-09-10T18:49:57.521 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:57 smithi103 ceph-mon[33163]: 7.0 scrub ok 2024-09-10T18:49:57.541 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:57.569 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:57.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:57.570 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:57.671 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:49:57.696 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:57.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:57.697 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:49:57.798 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:57.825 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:57.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:57.827 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:57.928 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:49:57.954 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:57.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:57.956 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:49:58.057 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:58.107 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:58.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:58.108 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:58.209 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:58.233 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:58.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:58.235 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:58.335 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:58.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:58.381 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:58.381 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:58.387 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T18:49:58.482 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:58.485 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-10T18:49:58.533 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:58.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:58.534 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:49:58.588 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:58 smithi018 ceph-mon[40196]: 6.19 scrub starts 2024-09-10T18:49:58.588 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:58 smithi018 ceph-mon[40196]: 6.19 scrub ok 2024-09-10T18:49:58.588 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:58 smithi018 ceph-mon[40196]: pgmap v178: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.4 GiB data, 18 GiB used, 607 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.38k op/s 2024-09-10T18:49:58.588 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:58 smithi018 ceph-mon[40196]: 3.0 scrub starts 2024-09-10T18:49:58.588 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:58 smithi018 ceph-mon[40196]: 3.0 scrub ok 2024-09-10T18:49:58.591 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:58 smithi018 ceph-mon[29336]: 6.19 scrub starts 2024-09-10T18:49:58.591 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:58 smithi018 ceph-mon[29336]: 6.19 scrub ok 2024-09-10T18:49:58.592 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:58 smithi018 ceph-mon[29336]: pgmap v178: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.4 GiB data, 18 GiB used, 607 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.38k op/s 2024-09-10T18:49:58.592 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:58 smithi018 ceph-mon[29336]: 3.0 scrub starts 2024-09-10T18:49:58.592 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:58 smithi018 ceph-mon[29336]: 3.0 scrub ok 2024-09-10T18:49:58.635 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:58.661 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:58.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:58.662 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:58.680 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:49:58.763 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:58.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:58 smithi103 ceph-mon[33163]: 6.19 scrub starts 2024-09-10T18:49:58.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:58 smithi103 ceph-mon[33163]: 6.19 scrub ok 2024-09-10T18:49:58.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:58 smithi103 ceph-mon[33163]: pgmap v178: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.4 GiB data, 18 GiB used, 607 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.38k op/s 2024-09-10T18:49:58.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:58 smithi103 ceph-mon[33163]: 3.0 scrub starts 2024-09-10T18:49:58.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:58 smithi103 ceph-mon[33163]: 3.0 scrub ok 2024-09-10T18:49:58.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:58.802 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:58.802 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:58.818 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:49:58.904 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:58.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:58.954 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:58.955 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:59.055 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:59.096 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:59.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:59.097 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:59.197 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:59.231 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:59.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:59.233 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:59.334 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:49:59.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:59.375 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:59.375 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:49:59.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:59 smithi018 ceph-mon[29336]: 6.6 scrub starts 2024-09-10T18:49:59.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:59 smithi018 ceph-mon[29336]: 6.6 scrub ok 2024-09-10T18:49:59.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:59 smithi018 ceph-mon[29336]: 2.13 scrub starts 2024-09-10T18:49:59.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:59 smithi018 ceph-mon[29336]: 2.13 scrub ok 2024-09-10T18:49:59.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:59 smithi018 ceph-mon[29336]: 4.1f scrub starts 2024-09-10T18:49:59.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:49:59 smithi018 ceph-mon[29336]: 4.1f scrub ok 2024-09-10T18:49:59.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:59 smithi018 ceph-mon[40196]: 6.6 scrub starts 2024-09-10T18:49:59.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:59 smithi018 ceph-mon[40196]: 6.6 scrub ok 2024-09-10T18:49:59.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:59 smithi018 ceph-mon[40196]: 2.13 scrub starts 2024-09-10T18:49:59.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:59 smithi018 ceph-mon[40196]: 2.13 scrub ok 2024-09-10T18:49:59.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:59 smithi018 ceph-mon[40196]: 4.1f scrub starts 2024-09-10T18:49:59.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:49:59 smithi018 ceph-mon[40196]: 4.1f scrub ok 2024-09-10T18:49:59.476 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:59.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:59.518 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:59.518 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:59.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:59 smithi103 ceph-mon[33163]: 6.6 scrub starts 2024-09-10T18:49:59.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:59 smithi103 ceph-mon[33163]: 6.6 scrub ok 2024-09-10T18:49:59.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:59 smithi103 ceph-mon[33163]: 2.13 scrub starts 2024-09-10T18:49:59.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:59 smithi103 ceph-mon[33163]: 2.13 scrub ok 2024-09-10T18:49:59.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:59 smithi103 ceph-mon[33163]: 4.1f scrub starts 2024-09-10T18:49:59.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:49:59 smithi103 ceph-mon[33163]: 4.1f scrub ok 2024-09-10T18:49:59.619 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:49:59.646 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:49:59.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:59.647 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:49:59.748 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:49:59.788 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:59.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:59.789 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:49:59.890 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:49:59.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:49:59.931 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:49:59.932 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:00.032 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:00.059 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:00.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:00.059 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:00.160 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:00.190 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:00.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:00.191 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:00.292 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:00.336 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is unset 2024-09-10T18:50:00.347 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:00.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:00.350 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:00.451 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:00.492 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:00.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:00.493 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:00.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:50:00 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:50:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:50:00.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:00 smithi103 ceph-mon[33163]: 6.11 scrub starts 2024-09-10T18:50:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:00 smithi103 ceph-mon[33163]: 6.11 scrub ok 2024-09-10T18:50:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:00 smithi103 ceph-mon[33163]: 4.16 scrub starts 2024-09-10T18:50:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:00 smithi103 ceph-mon[33163]: 4.16 scrub ok 2024-09-10T18:50:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:00 smithi103 ceph-mon[33163]: pgmap v179: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.4 GiB data, 18 GiB used, 607 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.23k op/s 2024-09-10T18:50:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:00 smithi103 ceph-mon[33163]: 6.12 scrub starts 2024-09-10T18:50:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:00 smithi103 ceph-mon[33163]: 6.12 scrub ok 2024-09-10T18:50:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:00 smithi103 ceph-mon[33163]: 2.7 scrub starts 2024-09-10T18:50:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:00 smithi103 ceph-mon[33163]: 2.7 scrub ok 2024-09-10T18:50:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:00 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/276110970' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:50:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:00 smithi103 ceph-mon[33163]: Health detail: HEALTH_WARN nodeep-scrub flag(s) set 2024-09-10T18:50:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:00 smithi103 ceph-mon[33163]: [WRN] OSDMAP_FLAGS: nodeep-scrub flag(s) set 2024-09-10T18:50:00.594 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:00.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:00.625 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:00.626 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:00.636 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:00 smithi018 ceph-mon[40196]: 6.11 scrub starts 2024-09-10T18:50:00.636 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:00 smithi018 ceph-mon[40196]: 6.11 scrub ok 2024-09-10T18:50:00.636 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:00 smithi018 ceph-mon[40196]: 4.16 scrub starts 2024-09-10T18:50:00.637 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:00 smithi018 ceph-mon[40196]: 4.16 scrub ok 2024-09-10T18:50:00.637 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:00 smithi018 ceph-mon[40196]: pgmap v179: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.4 GiB data, 18 GiB used, 607 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.23k op/s 2024-09-10T18:50:00.637 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:00 smithi018 ceph-mon[40196]: 6.12 scrub starts 2024-09-10T18:50:00.637 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:00 smithi018 ceph-mon[40196]: 6.12 scrub ok 2024-09-10T18:50:00.637 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:00 smithi018 ceph-mon[40196]: 2.7 scrub starts 2024-09-10T18:50:00.637 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:00 smithi018 ceph-mon[40196]: 2.7 scrub ok 2024-09-10T18:50:00.637 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:00 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/276110970' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:50:00.637 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:00 smithi018 ceph-mon[40196]: Health detail: HEALTH_WARN nodeep-scrub flag(s) set 2024-09-10T18:50:00.637 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:00 smithi018 ceph-mon[40196]: [WRN] OSDMAP_FLAGS: nodeep-scrub flag(s) set 2024-09-10T18:50:00.637 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:00 smithi018 ceph-mon[29336]: 6.11 scrub starts 2024-09-10T18:50:00.638 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:00 smithi018 ceph-mon[29336]: 6.11 scrub ok 2024-09-10T18:50:00.638 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:00 smithi018 ceph-mon[29336]: 4.16 scrub starts 2024-09-10T18:50:00.638 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:00 smithi018 ceph-mon[29336]: 4.16 scrub ok 2024-09-10T18:50:00.638 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:00 smithi018 ceph-mon[29336]: pgmap v179: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.4 GiB data, 18 GiB used, 607 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.23k op/s 2024-09-10T18:50:00.638 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:00 smithi018 ceph-mon[29336]: 6.12 scrub starts 2024-09-10T18:50:00.638 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:00 smithi018 ceph-mon[29336]: 6.12 scrub ok 2024-09-10T18:50:00.638 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:00 smithi018 ceph-mon[29336]: 2.7 scrub starts 2024-09-10T18:50:00.638 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:00 smithi018 ceph-mon[29336]: 2.7 scrub ok 2024-09-10T18:50:00.638 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:00 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/276110970' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:50:00.638 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:00 smithi018 ceph-mon[29336]: Health detail: HEALTH_WARN nodeep-scrub flag(s) set 2024-09-10T18:50:00.638 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:00 smithi018 ceph-mon[29336]: [WRN] OSDMAP_FLAGS: nodeep-scrub flag(s) set 2024-09-10T18:50:00.727 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:00.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:00.764 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:00.764 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:00.865 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:00.875 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T18:50:00.875 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:50:00.912 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:00.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:00.913 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:01.013 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:01.044 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:01.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:01.045 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:01.146 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:01.172 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:01.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:01.173 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:01.274 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:01.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:01.306 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:01.306 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:01.311 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:50:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:50:00] "GET /metrics HTTP/1.1" 200 223754 "" "Prometheus/2.43.0" 2024-09-10T18:50:01.407 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:01.438 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:01.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:01.439 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:01.540 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:01.565 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:01.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:01.566 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:01.576 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:01 smithi018 ceph-mon[29336]: 5.6 scrub starts 2024-09-10T18:50:01.576 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:01 smithi018 ceph-mon[29336]: 5.6 scrub ok 2024-09-10T18:50:01.576 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:01 smithi018 ceph-mon[29336]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:50:01.576 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:01 smithi018 ceph-mon[29336]: Cluster is now healthy 2024-09-10T18:50:01.576 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:01 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/276110970' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:50:01.576 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:01 smithi018 ceph-mon[29336]: osdmap e125: 8 total, 8 up, 7 in 2024-09-10T18:50:01.576 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:01 smithi018 ceph-mon[29336]: 4.1c scrub starts 2024-09-10T18:50:01.576 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:01 smithi018 ceph-mon[29336]: 4.1c scrub ok 2024-09-10T18:50:01.576 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:01 smithi018 ceph-mon[40196]: 5.6 scrub starts 2024-09-10T18:50:01.577 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:01 smithi018 ceph-mon[40196]: 5.6 scrub ok 2024-09-10T18:50:01.577 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:01 smithi018 ceph-mon[40196]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:50:01.577 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:01 smithi018 ceph-mon[40196]: Cluster is now healthy 2024-09-10T18:50:01.577 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:01 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/276110970' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:50:01.577 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:01 smithi018 ceph-mon[40196]: osdmap e125: 8 total, 8 up, 7 in 2024-09-10T18:50:01.577 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:01 smithi018 ceph-mon[40196]: 4.1c scrub starts 2024-09-10T18:50:01.577 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:01 smithi018 ceph-mon[40196]: 4.1c scrub ok 2024-09-10T18:50:01.667 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:01.695 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:01.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:01.696 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:01.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:01 smithi103 ceph-mon[33163]: 5.6 scrub starts 2024-09-10T18:50:01.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:01 smithi103 ceph-mon[33163]: 5.6 scrub ok 2024-09-10T18:50:01.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:01 smithi103 ceph-mon[33163]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:50:01.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:01 smithi103 ceph-mon[33163]: Cluster is now healthy 2024-09-10T18:50:01.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:01 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/276110970' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:50:01.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:01 smithi103 ceph-mon[33163]: osdmap e125: 8 total, 8 up, 7 in 2024-09-10T18:50:01.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:01 smithi103 ceph-mon[33163]: 4.1c scrub starts 2024-09-10T18:50:01.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:01 smithi103 ceph-mon[33163]: 4.1c scrub ok 2024-09-10T18:50:01.797 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:01.822 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:01.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:01.823 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:01.898 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-10T18:50:01.924 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:01.969 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:01.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:01.970 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:02.071 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:02.083 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:50:02.101 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:02.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:02.102 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:02.203 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:02.248 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:50:02.249 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:02.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:02.250 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:02.351 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:02.393 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:02.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:02.395 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:02.495 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:02.517 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:02 smithi018 ceph-mon[40196]: pgmap v181: 148 pgs: 148 active+clean; 5.2 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.20k op/s 2024-09-10T18:50:02.517 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:02 smithi018 ceph-mon[40196]: 2.b scrub starts 2024-09-10T18:50:02.517 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:02 smithi018 ceph-mon[40196]: 2.b scrub ok 2024-09-10T18:50:02.517 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:02 smithi018 ceph-mon[40196]: 4.a scrub starts 2024-09-10T18:50:02.517 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:02 smithi018 ceph-mon[40196]: 4.a scrub ok 2024-09-10T18:50:02.517 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:02 smithi018 ceph-mon[40196]: 4.1b scrub starts 2024-09-10T18:50:02.517 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:02 smithi018 ceph-mon[40196]: 4.1b scrub ok 2024-09-10T18:50:02.519 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:02 smithi018 ceph-mon[29336]: pgmap v181: 148 pgs: 148 active+clean; 5.2 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.20k op/s 2024-09-10T18:50:02.520 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:02 smithi018 ceph-mon[29336]: 2.b scrub starts 2024-09-10T18:50:02.520 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:02 smithi018 ceph-mon[29336]: 2.b scrub ok 2024-09-10T18:50:02.520 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:02 smithi018 ceph-mon[29336]: 4.a scrub starts 2024-09-10T18:50:02.520 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:02 smithi018 ceph-mon[29336]: 4.a scrub ok 2024-09-10T18:50:02.520 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:02 smithi018 ceph-mon[29336]: 4.1b scrub starts 2024-09-10T18:50:02.520 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:02 smithi018 ceph-mon[29336]: 4.1b scrub ok 2024-09-10T18:50:02.531 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:02.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:02.532 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:02.633 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:02.634 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:02 smithi103 ceph-mon[33163]: pgmap v181: 148 pgs: 148 active+clean; 5.2 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.20k op/s 2024-09-10T18:50:02.634 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:02 smithi103 ceph-mon[33163]: 2.b scrub starts 2024-09-10T18:50:02.634 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:02 smithi103 ceph-mon[33163]: 2.b scrub ok 2024-09-10T18:50:02.634 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:02 smithi103 ceph-mon[33163]: 4.a scrub starts 2024-09-10T18:50:02.634 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:02 smithi103 ceph-mon[33163]: 4.a scrub ok 2024-09-10T18:50:02.634 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:02 smithi103 ceph-mon[33163]: 4.1b scrub starts 2024-09-10T18:50:02.635 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:02 smithi103 ceph-mon[33163]: 4.1b scrub ok 2024-09-10T18:50:02.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:02.659 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:02.659 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:02.760 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:02.788 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:02.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:02.789 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:02.890 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:02.929 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:02.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:02.930 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:03.031 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:03.067 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:03.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:03.068 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:03.169 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:03.211 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:03.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:03.212 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:03.313 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:03.345 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:03.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:03.346 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:03.347 INFO:teuthology.orchestra.run.smithi018.stderr:set pool 1 pg_num to 11 2024-09-10T18:50:03.447 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:03.477 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:03.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:03.478 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:03.578 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-10T18:50:03.579 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:03.594 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:03 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1898645664' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]: dispatch 2024-09-10T18:50:03.596 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:03 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1898645664' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]: dispatch 2024-09-10T18:50:03.630 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:03.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:03.632 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:03.666 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:03 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1898645664' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]: dispatch 2024-09-10T18:50:03.733 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:03.781 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:03.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:03.782 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:03.883 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:03.909 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:03.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:03.910 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:04.011 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:04.045 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:04.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:04.046 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:04.147 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:04.183 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:04.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:04.184 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:04.284 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T18:50:04.284 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:50:04.285 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:04.320 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:04.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:04.321 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:04.422 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:04.461 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:04.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:04.462 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:04.563 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:04.597 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:04.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:04.598 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:04.692 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:04 smithi103 ceph-mon[33163]: 5.8 deep-scrub starts 2024-09-10T18:50:04.692 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:04 smithi103 ceph-mon[33163]: 5.8 deep-scrub ok 2024-09-10T18:50:04.692 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:04 smithi103 ceph-mon[33163]: 5.1b scrub starts 2024-09-10T18:50:04.693 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:04 smithi103 ceph-mon[33163]: 5.1b scrub ok 2024-09-10T18:50:04.693 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:04 smithi103 ceph-mon[33163]: pgmap v182: 148 pgs: 148 active+clean; 5.0 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.17k op/s 2024-09-10T18:50:04.693 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:04 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1898645664' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]': finished 2024-09-10T18:50:04.693 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:04 smithi103 ceph-mon[33163]: osdmap e126: 8 total, 8 up, 7 in 2024-09-10T18:50:04.693 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:04 smithi103 ceph-mon[33163]: 4.5 deep-scrub starts 2024-09-10T18:50:04.693 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:04 smithi103 ceph-mon[33163]: 4.5 deep-scrub ok 2024-09-10T18:50:04.693 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:04 smithi103 ceph-mon[33163]: 7.b scrub starts 2024-09-10T18:50:04.693 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:04 smithi103 ceph-mon[33163]: 6.2 scrub starts 2024-09-10T18:50:04.693 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:04 smithi103 ceph-mon[33163]: 6.2 scrub ok 2024-09-10T18:50:04.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:04 smithi018 ceph-mon[40196]: 5.8 deep-scrub starts 2024-09-10T18:50:04.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:04 smithi018 ceph-mon[40196]: 5.8 deep-scrub ok 2024-09-10T18:50:04.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:04 smithi018 ceph-mon[40196]: 5.1b scrub starts 2024-09-10T18:50:04.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:04 smithi018 ceph-mon[40196]: 5.1b scrub ok 2024-09-10T18:50:04.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:04 smithi018 ceph-mon[40196]: pgmap v182: 148 pgs: 148 active+clean; 5.0 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.17k op/s 2024-09-10T18:50:04.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:04 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1898645664' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]': finished 2024-09-10T18:50:04.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:04 smithi018 ceph-mon[40196]: osdmap e126: 8 total, 8 up, 7 in 2024-09-10T18:50:04.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:04 smithi018 ceph-mon[40196]: 4.5 deep-scrub starts 2024-09-10T18:50:04.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:04 smithi018 ceph-mon[40196]: 4.5 deep-scrub ok 2024-09-10T18:50:04.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:04 smithi018 ceph-mon[40196]: 7.b scrub starts 2024-09-10T18:50:04.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:04 smithi018 ceph-mon[40196]: 6.2 scrub starts 2024-09-10T18:50:04.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:04 smithi018 ceph-mon[40196]: 6.2 scrub ok 2024-09-10T18:50:04.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:04 smithi018 ceph-mon[29336]: 5.8 deep-scrub starts 2024-09-10T18:50:04.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:04 smithi018 ceph-mon[29336]: 5.8 deep-scrub ok 2024-09-10T18:50:04.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:04 smithi018 ceph-mon[29336]: 5.1b scrub starts 2024-09-10T18:50:04.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:04 smithi018 ceph-mon[29336]: 5.1b scrub ok 2024-09-10T18:50:04.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:04 smithi018 ceph-mon[29336]: pgmap v182: 148 pgs: 148 active+clean; 5.0 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.17k op/s 2024-09-10T18:50:04.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:04 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1898645664' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]': finished 2024-09-10T18:50:04.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:04 smithi018 ceph-mon[29336]: osdmap e126: 8 total, 8 up, 7 in 2024-09-10T18:50:04.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:04 smithi018 ceph-mon[29336]: 4.5 deep-scrub starts 2024-09-10T18:50:04.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:04 smithi018 ceph-mon[29336]: 4.5 deep-scrub ok 2024-09-10T18:50:04.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:04 smithi018 ceph-mon[29336]: 7.b scrub starts 2024-09-10T18:50:04.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:04 smithi018 ceph-mon[29336]: 6.2 scrub starts 2024-09-10T18:50:04.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:04 smithi018 ceph-mon[29336]: 6.2 scrub ok 2024-09-10T18:50:04.698 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:04.727 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:04.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:04.728 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:04.829 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:04.855 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:04.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:04.856 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:04.956 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:04.984 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:04.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:04.985 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:05.086 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:05.138 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-10T18:50:05.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:05.139 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:05.139 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:05.239 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:05.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:05.281 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:05.281 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:05.352 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:05 smithi103 ceph-mon[33163]: 5.13 deep-scrub starts 2024-09-10T18:50:05.353 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:05 smithi103 ceph-mon[33163]: 5.13 deep-scrub ok 2024-09-10T18:50:05.381 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:05.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:05.422 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:05.422 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:05.493 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:50:05.523 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:05.560 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:05.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:05.561 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:05.624 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:05 smithi103 ceph-mon[33163]: 4.4 scrub starts 2024-09-10T18:50:05.624 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:05 smithi103 ceph-mon[33163]: 4.4 scrub ok 2024-09-10T18:50:05.625 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:05 smithi103 ceph-mon[33163]: 4.18 scrub starts 2024-09-10T18:50:05.625 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:05 smithi103 ceph-mon[33163]: 4.18 scrub ok 2024-09-10T18:50:05.625 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:05 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]: dispatch 2024-09-10T18:50:05.625 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:05 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]: dispatch 2024-09-10T18:50:05.661 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:05.683 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:05.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:05.684 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:05.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:05 smithi018 ceph-mon[40196]: 5.13 deep-scrub starts 2024-09-10T18:50:05.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:05 smithi018 ceph-mon[40196]: 5.13 deep-scrub ok 2024-09-10T18:50:05.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:05 smithi018 ceph-mon[40196]: 4.4 scrub starts 2024-09-10T18:50:05.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:05 smithi018 ceph-mon[40196]: 4.4 scrub ok 2024-09-10T18:50:05.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:05 smithi018 ceph-mon[40196]: 4.18 scrub starts 2024-09-10T18:50:05.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:05 smithi018 ceph-mon[40196]: 4.18 scrub ok 2024-09-10T18:50:05.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:05 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]: dispatch 2024-09-10T18:50:05.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:05 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]: dispatch 2024-09-10T18:50:05.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:05 smithi018 ceph-mon[29336]: 5.13 deep-scrub starts 2024-09-10T18:50:05.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:05 smithi018 ceph-mon[29336]: 5.13 deep-scrub ok 2024-09-10T18:50:05.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:05 smithi018 ceph-mon[29336]: 4.4 scrub starts 2024-09-10T18:50:05.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:05 smithi018 ceph-mon[29336]: 4.4 scrub ok 2024-09-10T18:50:05.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:05 smithi018 ceph-mon[29336]: 4.18 scrub starts 2024-09-10T18:50:05.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:05 smithi018 ceph-mon[29336]: 4.18 scrub ok 2024-09-10T18:50:05.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:05 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]: dispatch 2024-09-10T18:50:05.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:05 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]: dispatch 2024-09-10T18:50:05.785 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:05.819 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:05.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:05.820 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:05.921 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:05.958 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:05.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:05.959 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:06.060 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:06.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:06.099 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:06.099 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:06.200 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:06.226 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:06.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:06.227 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:06.328 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:06.359 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:06.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:06.360 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:06.461 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:06.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:06.499 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:06.499 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:06.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:06 smithi103 ceph-mon[33163]: 7.a scrub starts 2024-09-10T18:50:06.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:06 smithi103 ceph-mon[33163]: 6.10 deep-scrub starts 2024-09-10T18:50:06.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:06 smithi103 ceph-mon[33163]: 6.10 deep-scrub ok 2024-09-10T18:50:06.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:06 smithi103 ceph-mon[33163]: 5.f scrub starts 2024-09-10T18:50:06.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:06 smithi103 ceph-mon[33163]: 5.f scrub ok 2024-09-10T18:50:06.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:06 smithi103 ceph-mon[33163]: pgmap v184: 148 pgs: 148 active+clean; 5.0 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 744 op/s 2024-09-10T18:50:06.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:06 smithi103 ceph-mon[33163]: 7.b scrub ok 2024-09-10T18:50:06.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:06 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]': finished 2024-09-10T18:50:06.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:06 smithi103 ceph-mon[33163]: osdmap e127: 8 total, 8 up, 7 in 2024-09-10T18:50:06.523 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:50:06 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:50:06.599 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:06.621 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:06.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:06.622 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:06.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:06 smithi018 ceph-mon[29336]: 7.a scrub starts 2024-09-10T18:50:06.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:06 smithi018 ceph-mon[29336]: 6.10 deep-scrub starts 2024-09-10T18:50:06.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:06 smithi018 ceph-mon[29336]: 6.10 deep-scrub ok 2024-09-10T18:50:06.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:06 smithi018 ceph-mon[29336]: 5.f scrub starts 2024-09-10T18:50:06.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:06 smithi018 ceph-mon[29336]: 5.f scrub ok 2024-09-10T18:50:06.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:06 smithi018 ceph-mon[29336]: pgmap v184: 148 pgs: 148 active+clean; 5.0 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 744 op/s 2024-09-10T18:50:06.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:06 smithi018 ceph-mon[29336]: 7.b scrub ok 2024-09-10T18:50:06.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:06 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]': finished 2024-09-10T18:50:06.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:06 smithi018 ceph-mon[29336]: osdmap e127: 8 total, 8 up, 7 in 2024-09-10T18:50:06.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:06 smithi018 ceph-mon[40196]: 7.a scrub starts 2024-09-10T18:50:06.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:06 smithi018 ceph-mon[40196]: 6.10 deep-scrub starts 2024-09-10T18:50:06.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:06 smithi018 ceph-mon[40196]: 6.10 deep-scrub ok 2024-09-10T18:50:06.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:06 smithi018 ceph-mon[40196]: 5.f scrub starts 2024-09-10T18:50:06.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:06 smithi018 ceph-mon[40196]: 5.f scrub ok 2024-09-10T18:50:06.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:06 smithi018 ceph-mon[40196]: pgmap v184: 148 pgs: 148 active+clean; 5.0 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 744 op/s 2024-09-10T18:50:06.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:06 smithi018 ceph-mon[40196]: 7.b scrub ok 2024-09-10T18:50:06.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:06 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]': finished 2024-09-10T18:50:06.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:06 smithi018 ceph-mon[40196]: osdmap e127: 8 total, 8 up, 7 in 2024-09-10T18:50:06.723 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:06.758 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:06.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:06.759 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:06.860 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:06.898 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:06.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:06.899 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:07.000 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:07.030 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:07.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:07.032 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:07.133 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:07.159 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:07.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:07.160 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:07.261 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:07.300 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:07.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:07.301 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:07.402 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:07.424 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:07.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:07.425 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:07.526 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:07.555 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:07.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:07.556 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:07.657 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:07.678 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:07.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:07.679 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:07.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:07 smithi018 ceph-mon[40196]: 2.e deep-scrub starts 2024-09-10T18:50:07.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:07 smithi018 ceph-mon[40196]: 2.e deep-scrub ok 2024-09-10T18:50:07.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:07 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:50:07.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:07 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:50:07.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:07 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:07.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:07 smithi018 ceph-mon[40196]: 5.11 scrub starts 2024-09-10T18:50:07.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:07 smithi018 ceph-mon[40196]: 5.11 scrub ok 2024-09-10T18:50:07.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:07 smithi018 ceph-mon[40196]: osdmap e128: 8 total, 8 up, 7 in 2024-09-10T18:50:07.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:07 smithi018 ceph-mon[40196]: 2.9 scrub starts 2024-09-10T18:50:07.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:07 smithi018 ceph-mon[40196]: 2.5 scrub starts 2024-09-10T18:50:07.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:07 smithi018 ceph-mon[40196]: 2.5 scrub ok 2024-09-10T18:50:07.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:07 smithi018 ceph-mon[40196]: 2.9 scrub ok 2024-09-10T18:50:07.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:07 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]: dispatch 2024-09-10T18:50:07.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:07 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]: dispatch 2024-09-10T18:50:07.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:07 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2498678972' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:50:07.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:07 smithi018 ceph-mon[29336]: 2.e deep-scrub starts 2024-09-10T18:50:07.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:07 smithi018 ceph-mon[29336]: 2.e deep-scrub ok 2024-09-10T18:50:07.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:07 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:50:07.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:07 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:50:07.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:07 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:07.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:07 smithi018 ceph-mon[29336]: 5.11 scrub starts 2024-09-10T18:50:07.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:07 smithi018 ceph-mon[29336]: 5.11 scrub ok 2024-09-10T18:50:07.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:07 smithi018 ceph-mon[29336]: osdmap e128: 8 total, 8 up, 7 in 2024-09-10T18:50:07.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:07 smithi018 ceph-mon[29336]: 2.9 scrub starts 2024-09-10T18:50:07.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:07 smithi018 ceph-mon[29336]: 2.5 scrub starts 2024-09-10T18:50:07.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:07 smithi018 ceph-mon[29336]: 2.5 scrub ok 2024-09-10T18:50:07.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:07 smithi018 ceph-mon[29336]: 2.9 scrub ok 2024-09-10T18:50:07.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:07 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]: dispatch 2024-09-10T18:50:07.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:07 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]: dispatch 2024-09-10T18:50:07.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:07 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2498678972' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:50:07.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:07 smithi103 ceph-mon[33163]: 2.e deep-scrub starts 2024-09-10T18:50:07.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:07 smithi103 ceph-mon[33163]: 2.e deep-scrub ok 2024-09-10T18:50:07.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:07 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:50:07.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:07 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:50:07.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:07 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:07.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:07 smithi103 ceph-mon[33163]: 5.11 scrub starts 2024-09-10T18:50:07.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:07 smithi103 ceph-mon[33163]: 5.11 scrub ok 2024-09-10T18:50:07.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:07 smithi103 ceph-mon[33163]: osdmap e128: 8 total, 8 up, 7 in 2024-09-10T18:50:07.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:07 smithi103 ceph-mon[33163]: 2.9 scrub starts 2024-09-10T18:50:07.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:07 smithi103 ceph-mon[33163]: 2.5 scrub starts 2024-09-10T18:50:07.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:07 smithi103 ceph-mon[33163]: 2.5 scrub ok 2024-09-10T18:50:07.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:07 smithi103 ceph-mon[33163]: 2.9 scrub ok 2024-09-10T18:50:07.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:07 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]: dispatch 2024-09-10T18:50:07.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:07 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]: dispatch 2024-09-10T18:50:07.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:07 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2498678972' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:50:07.780 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:07.810 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:07.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:07.811 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:07.912 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:07.943 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:07.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:07.944 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:08.045 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:08.076 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:08.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:08.077 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:08.178 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:08.201 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:08.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:08.202 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:08.303 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:08.337 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:50:08.337 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:50:08.337 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:50:08.337 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:50:08.337 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:50:08.337 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:50:08.337 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:50:08.337 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:50:08.338 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:50:08.338 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:50:08.338 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:50:08.338 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:50:08.338 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:50:08.338 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:50:08.338 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:50:08.338 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:50:08.345 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:08.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:08.346 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:08.389 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:08 smithi018 ceph-mon[40196]: pgmap v187: 158 pgs: 1 active+clean+scrubbing, 1 peering, 10 unknown, 146 active+clean; 4.7 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 0 B/s wr, 1.51k op/s 2024-09-10T18:50:08.389 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:08 smithi018 ceph-mon[40196]: 2.1a scrub starts 2024-09-10T18:50:08.389 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:08 smithi018 ceph-mon[40196]: 2.1a scrub ok 2024-09-10T18:50:08.389 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:08 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]': finished 2024-09-10T18:50:08.390 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:08 smithi018 ceph-mon[29336]: pgmap v187: 158 pgs: 1 active+clean+scrubbing, 1 peering, 10 unknown, 146 active+clean; 4.7 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 0 B/s wr, 1.51k op/s 2024-09-10T18:50:08.390 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:08 smithi018 ceph-mon[29336]: 2.1a scrub starts 2024-09-10T18:50:08.390 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:08 smithi018 ceph-mon[29336]: 2.1a scrub ok 2024-09-10T18:50:08.390 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:08 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]': finished 2024-09-10T18:50:08.390 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:08 smithi018 ceph-mon[29336]: osdmap e129: 8 total, 8 up, 7 in 2024-09-10T18:50:08.390 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:08 smithi018 ceph-mon[29336]: 2.10 scrub starts 2024-09-10T18:50:08.390 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:08 smithi018 ceph-mon[29336]: 2.10 scrub ok 2024-09-10T18:50:08.447 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:08.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:08.469 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:08.469 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:08.570 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:08.603 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:08.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:08.605 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:08.636 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:08 smithi103 ceph-mon[33163]: pgmap v187: 158 pgs: 1 active+clean+scrubbing, 1 peering, 10 unknown, 146 active+clean; 4.7 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 0 B/s wr, 1.51k op/s 2024-09-10T18:50:08.637 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:08 smithi103 ceph-mon[33163]: 2.1a scrub starts 2024-09-10T18:50:08.637 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:08 smithi103 ceph-mon[33163]: 2.1a scrub ok 2024-09-10T18:50:08.637 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:08 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]': finished 2024-09-10T18:50:08.637 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:08 smithi103 ceph-mon[33163]: osdmap e129: 8 total, 8 up, 7 in 2024-09-10T18:50:08.637 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:08 smithi103 ceph-mon[33163]: 2.10 scrub starts 2024-09-10T18:50:08.637 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:08 smithi103 ceph-mon[33163]: 2.10 scrub ok 2024-09-10T18:50:08.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:08 smithi018 ceph-mon[40196]: osdmap e129: 8 total, 8 up, 7 in 2024-09-10T18:50:08.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:08 smithi018 ceph-mon[40196]: 2.10 scrub starts 2024-09-10T18:50:08.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:08 smithi018 ceph-mon[40196]: 2.10 scrub ok 2024-09-10T18:50:08.705 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:08.742 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:08.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:08.743 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:08.844 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:08.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:08.883 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:08.883 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:08.984 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:09.013 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:09.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:09.014 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:09.021 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T18:50:09.021 INFO:teuthology.orchestra.run.smithi103.stdout: "ops": [ 2024-09-10T18:50:09.021 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:09.021 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:198915 7.c 7:38b56880:::benchmark_data_smithi018_78628_object62695:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-10T18:50:09.021 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:50:09.000813+0000", 2024-09-10T18:50:09.021 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.018945109000000002, 2024-09-10T18:50:09.021 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.018966936, 2024-09-10T18:50:09.021 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:50:09.021 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "waiting for sub ops", 2024-09-10T18:50:09.021 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:50:09.021 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:50:09.022 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:50:09.022 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 198915 2024-09-10T18:50:09.022 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:09.022 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:50:09.022 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:09.022 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:50:09.022 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.000813+0000", 2024-09-10T18:50:09.022 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:09.022 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:09.022 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:09.022 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:50:09.022 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.000813+0000", 2024-09-10T18:50:09.022 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:09.023 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:09.023 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:09.023 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:50:09.023 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.000812+0000", 2024-09-10T18:50:09.023 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:50:09.023 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:09.023 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:09.023 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:50:09.023 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.000818+0000", 2024-09-10T18:50:09.023 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.426e-06 2024-09-10T18:50:09.023 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:09.023 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:09.023 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:50:09.024 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.000852+0000", 2024-09-10T18:50:09.024 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.3306999999999998e-05 2024-09-10T18:50:09.024 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:09.024 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:09.024 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:50:09.024 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.000858+0000", 2024-09-10T18:50:09.024 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.0510000000000002e-06 2024-09-10T18:50:09.024 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:09.024 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:09.024 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:50:09.024 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.000953+0000", 2024-09-10T18:50:09.024 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.4745000000000002e-05 2024-09-10T18:50:09.024 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:09.025 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:09.025 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:50:09.025 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.001087+0000", 2024-09-10T18:50:09.025 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00013440699999999999 2024-09-10T18:50:09.025 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:09.025 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:09.025 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,7", 2024-09-10T18:50:09.025 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.002049+0000", 2024-09-10T18:50:09.025 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00096162799999999996 2024-09-10T18:50:09.025 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:09.025 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:09.025 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:50:09.025 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.004659+0000", 2024-09-10T18:50:09.025 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0026108590000000001 2024-09-10T18:50:09.026 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:09.026 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:09.026 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:09.026 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.005784+0000", 2024-09-10T18:50:09.026 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0011245789999999999 2024-09-10T18:50:09.026 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:09.026 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:50:09.026 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:09.026 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:09.026 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:09.026 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:198918 7.7 7:e80c97fe:::benchmark_data_smithi018_78628_object62698:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-10T18:50:09.026 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:50:09.006104+0000", 2024-09-10T18:50:09.026 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.013654496, 2024-09-10T18:50:09.027 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.013749888, 2024-09-10T18:50:09.027 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:50:09.027 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "waiting for sub ops", 2024-09-10T18:50:09.027 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:50:09.027 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:50:09.027 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:50:09.027 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 198918 2024-09-10T18:50:09.027 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:09.027 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:50:09.027 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:09.027 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:50:09.027 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.006104+0000", 2024-09-10T18:50:09.027 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:09.027 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:09.028 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:09.028 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:50:09.028 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.006104+0000", 2024-09-10T18:50:09.028 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:09.028 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:09.028 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:09.028 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:50:09.028 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.006102+0000", 2024-09-10T18:50:09.028 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999981 2024-09-10T18:50:09.028 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:09.028 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:09.028 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:50:09.028 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.006110+0000", 2024-09-10T18:50:09.029 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.5089999999999996e-06 2024-09-10T18:50:09.029 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:09.029 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:09.029 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:50:09.029 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.006124+0000", 2024-09-10T18:50:09.029 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.4842e-05 2024-09-10T18:50:09.029 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:09.029 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:09.029 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:50:09.029 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.006131+0000", 2024-09-10T18:50:09.029 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.5740000000000001e-06 2024-09-10T18:50:09.029 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:09.030 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:09.030 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:50:09.030 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.006175+0000", 2024-09-10T18:50:09.030 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.3597999999999998e-05 2024-09-10T18:50:09.030 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:09.030 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:09.030 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:50:09.030 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.006301+0000", 2024-09-10T18:50:09.030 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000126153 2024-09-10T18:50:09.030 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:09.030 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:09.030 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,3", 2024-09-10T18:50:09.030 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.006411+0000", 2024-09-10T18:50:09.030 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00011027300000000001 2024-09-10T18:50:09.031 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:09.031 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:09.031 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:50:09.031 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.008386+0000", 2024-09-10T18:50:09.031 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0019754180000000001 2024-09-10T18:50:09.031 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:09.031 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:50:09.031 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:09.031 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:09.031 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-10T18:50:09.031 INFO:teuthology.orchestra.run.smithi103.stdout: "num_ops": 2 2024-09-10T18:50:09.031 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T18:50:09.115 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:09.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:09.138 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:09.138 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:09.239 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:09.277 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:09.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:09.278 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:09.379 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:09.395 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-10T18:50:09.395 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd reweight 0 1 2024-09-10T18:50:09.405 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:09.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:09.406 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:09.507 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:09.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:09.535 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:09.535 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:09.546 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:50:09.636 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:09.690 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:09 smithi103 ceph-mon[33163]: 4.1e scrub starts 2024-09-10T18:50:09.690 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:09 smithi103 ceph-mon[33163]: 7.2 scrub starts 2024-09-10T18:50:09.690 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:09 smithi103 ceph-mon[33163]: 4.1e scrub ok 2024-09-10T18:50:09.691 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:09 smithi103 ceph-mon[33163]: 7.a scrub ok 2024-09-10T18:50:09.691 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:09 smithi103 ceph-mon[33163]: 5.2 scrub starts 2024-09-10T18:50:09.691 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:09 smithi103 ceph-mon[33163]: 5.2 scrub ok 2024-09-10T18:50:09.691 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:09 smithi103 ceph-mon[33163]: osdmap e130: 8 total, 8 up, 7 in 2024-09-10T18:50:09.691 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:09 smithi103 ceph-mon[33163]: 5.12 deep-scrub starts 2024-09-10T18:50:09.694 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:09.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:09 smithi018 ceph-mon[29336]: 4.1e scrub starts 2024-09-10T18:50:09.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:09 smithi018 ceph-mon[29336]: 7.2 scrub starts 2024-09-10T18:50:09.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:09 smithi018 ceph-mon[29336]: 4.1e scrub ok 2024-09-10T18:50:09.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:09 smithi018 ceph-mon[29336]: 7.a scrub ok 2024-09-10T18:50:09.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:09 smithi018 ceph-mon[29336]: 5.2 scrub starts 2024-09-10T18:50:09.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:09 smithi018 ceph-mon[29336]: 5.2 scrub ok 2024-09-10T18:50:09.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:09 smithi018 ceph-mon[29336]: osdmap e130: 8 total, 8 up, 7 in 2024-09-10T18:50:09.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:09 smithi018 ceph-mon[29336]: 5.12 deep-scrub starts 2024-09-10T18:50:09.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:09 smithi018 ceph-mon[40196]: 4.1e scrub starts 2024-09-10T18:50:09.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:09 smithi018 ceph-mon[40196]: 7.2 scrub starts 2024-09-10T18:50:09.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:09 smithi018 ceph-mon[40196]: 4.1e scrub ok 2024-09-10T18:50:09.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:09 smithi018 ceph-mon[40196]: 7.a scrub ok 2024-09-10T18:50:09.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:09 smithi018 ceph-mon[40196]: 5.2 scrub starts 2024-09-10T18:50:09.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:09 smithi018 ceph-mon[40196]: 5.2 scrub ok 2024-09-10T18:50:09.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:09 smithi018 ceph-mon[40196]: osdmap e130: 8 total, 8 up, 7 in 2024-09-10T18:50:09.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:09 smithi018 ceph-mon[40196]: 5.12 deep-scrub starts 2024-09-10T18:50:09.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:09.698 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:09.725 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-10T18:50:09.799 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:09.846 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:09.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:09.847 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:09.948 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:09.991 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:09.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:09.993 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:10.093 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:10.115 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:50:10.124 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:10.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:10.125 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:10.226 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:10.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:10.274 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:10.274 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:10.374 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:10.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:10.417 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:10.417 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:10.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:10 smithi018 ceph-mon[29336]: 4.b deep-scrub starts 2024-09-10T18:50:10.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:10 smithi018 ceph-mon[29336]: 4.b deep-scrub ok 2024-09-10T18:50:10.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:10 smithi018 ceph-mon[29336]: 6.d scrub starts 2024-09-10T18:50:10.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:10 smithi018 ceph-mon[29336]: 6.d scrub ok 2024-09-10T18:50:10.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:10 smithi018 ceph-mon[29336]: pgmap v190: 158 pgs: 1 active+clean+scrubbing, 1 peering, 10 unknown, 146 active+clean; 4.7 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 0 B/s wr, 1.52k op/s 2024-09-10T18:50:10.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:10 smithi018 ceph-mon[29336]: 4.2 scrub starts 2024-09-10T18:50:10.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:10 smithi018 ceph-mon[29336]: 4.2 scrub ok 2024-09-10T18:50:10.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:10 smithi018 ceph-mon[29336]: 7.1 scrub starts 2024-09-10T18:50:10.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:10 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:50:10.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:10 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:50:10.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:10 smithi018 ceph-mon[40196]: 4.b deep-scrub starts 2024-09-10T18:50:10.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:10 smithi018 ceph-mon[40196]: 4.b deep-scrub ok 2024-09-10T18:50:10.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:10 smithi018 ceph-mon[40196]: 6.d scrub starts 2024-09-10T18:50:10.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:10 smithi018 ceph-mon[40196]: 6.d scrub ok 2024-09-10T18:50:10.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:10 smithi018 ceph-mon[40196]: pgmap v190: 158 pgs: 1 active+clean+scrubbing, 1 peering, 10 unknown, 146 active+clean; 4.7 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 0 B/s wr, 1.52k op/s 2024-09-10T18:50:10.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:10 smithi018 ceph-mon[40196]: 4.2 scrub starts 2024-09-10T18:50:10.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:10 smithi018 ceph-mon[40196]: 4.2 scrub ok 2024-09-10T18:50:10.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:10 smithi018 ceph-mon[40196]: 7.1 scrub starts 2024-09-10T18:50:10.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:10 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:50:10.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:10 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:50:10.518 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:10.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:50:10 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:50:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:50:10.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:10 smithi103 ceph-mon[33163]: 4.b deep-scrub starts 2024-09-10T18:50:10.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:10 smithi103 ceph-mon[33163]: 4.b deep-scrub ok 2024-09-10T18:50:10.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:10 smithi103 ceph-mon[33163]: 6.d scrub starts 2024-09-10T18:50:10.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:10 smithi103 ceph-mon[33163]: 6.d scrub ok 2024-09-10T18:50:10.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:10 smithi103 ceph-mon[33163]: pgmap v190: 158 pgs: 1 active+clean+scrubbing, 1 peering, 10 unknown, 146 active+clean; 4.7 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 0 B/s wr, 1.52k op/s 2024-09-10T18:50:10.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:10 smithi103 ceph-mon[33163]: 4.2 scrub starts 2024-09-10T18:50:10.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:10 smithi103 ceph-mon[33163]: 4.2 scrub ok 2024-09-10T18:50:10.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:10 smithi103 ceph-mon[33163]: 7.1 scrub starts 2024-09-10T18:50:10.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:10 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:50:10.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:10 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:50:10.565 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:10.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:10.566 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:10.667 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:10.699 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:10.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:10.700 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:10.801 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:10.839 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:10.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:10.840 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:10.940 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:10.978 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:50:10.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:10.979 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:10.979 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:11.080 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:11.129 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:11.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:11.130 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:11.231 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:11.260 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:11.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:11.261 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:11.362 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:11.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:11.398 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:11.398 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:11.399 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:50:10 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:50:10] "GET /metrics HTTP/1.1" 200 223806 "" "Prometheus/2.43.0" 2024-09-10T18:50:11.499 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:11.541 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:11.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:11.542 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:11.643 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:11.690 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:11.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:11.691 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:11.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:11 smithi018 ceph-mon[29336]: 5.9 scrub starts 2024-09-10T18:50:11.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:11 smithi018 ceph-mon[29336]: 5.9 scrub ok 2024-09-10T18:50:11.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:11 smithi018 ceph-mon[29336]: 7.2 scrub ok 2024-09-10T18:50:11.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:11 smithi018 ceph-mon[29336]: 7.1 scrub ok 2024-09-10T18:50:11.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:11 smithi018 ceph-mon[40196]: 5.9 scrub starts 2024-09-10T18:50:11.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:11 smithi018 ceph-mon[40196]: 5.9 scrub ok 2024-09-10T18:50:11.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:11 smithi018 ceph-mon[40196]: 7.2 scrub ok 2024-09-10T18:50:11.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:11 smithi018 ceph-mon[40196]: 7.1 scrub ok 2024-09-10T18:50:11.729 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:11 smithi103 ceph-mon[33163]: 5.9 scrub starts 2024-09-10T18:50:11.730 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:11 smithi103 ceph-mon[33163]: 5.9 scrub ok 2024-09-10T18:50:11.730 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:11 smithi103 ceph-mon[33163]: 7.2 scrub ok 2024-09-10T18:50:11.730 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:11 smithi103 ceph-mon[33163]: 7.1 scrub ok 2024-09-10T18:50:11.792 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:11.820 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:11.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:11.821 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:11.921 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:11.961 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:11.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:11.962 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:12.003 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T18:50:12.003 INFO:teuthology.orchestra.run.smithi103.stdout: "ops": [], 2024-09-10T18:50:12.003 INFO:teuthology.orchestra.run.smithi103.stdout: "complaint_time": 30, 2024-09-10T18:50:12.003 INFO:teuthology.orchestra.run.smithi103.stdout: "num_blocked_ops": 0 2024-09-10T18:50:12.003 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T18:50:12.063 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:12.108 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:12.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:12.109 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:12.202 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T18:50:12.210 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:12.291 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:12.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:12.292 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:12.393 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:12.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:12.446 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:12.446 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:12.547 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:12.593 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:12.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:12.595 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:12.690 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:12 smithi103 ceph-mon[33163]: pgmap v191: 158 pgs: 1 active+clean+scrubbing, 2 unknown, 155 active+clean; 4.5 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 0 B/s wr, 1.54k op/s; 8.1 KiB/s, 0 objects/s recovering 2024-09-10T18:50:12.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:12 smithi018 ceph-mon[40196]: pgmap v191: 158 pgs: 1 active+clean+scrubbing, 2 unknown, 155 active+clean; 4.5 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 0 B/s wr, 1.54k op/s; 8.1 KiB/s, 0 objects/s recovering 2024-09-10T18:50:12.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:12 smithi018 ceph-mon[29336]: pgmap v191: 158 pgs: 1 active+clean+scrubbing, 2 unknown, 155 active+clean; 4.5 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 0 B/s wr, 1.54k op/s; 8.1 KiB/s, 0 objects/s recovering 2024-09-10T18:50:12.696 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:12.735 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:12.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:12.736 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:12.837 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:12.877 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:12.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:12.878 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:12.979 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:13.009 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:13.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:13.009 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:13.102 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-10T18:50:13.110 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:13.182 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:13.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:13.184 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:13.284 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:13.321 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:13.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:13.322 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:13.422 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:13.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:13.470 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:13.470 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:13.479 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:13 smithi018 ceph-mon[40196]: 4.9 scrub starts 2024-09-10T18:50:13.479 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:13 smithi018 ceph-mon[40196]: 4.9 scrub ok 2024-09-10T18:50:13.479 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:13 smithi018 ceph-mon[29336]: 4.9 scrub starts 2024-09-10T18:50:13.479 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:13 smithi018 ceph-mon[29336]: 4.9 scrub ok 2024-09-10T18:50:13.497 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:50:13.571 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:13.609 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:13.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:13.610 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:13.711 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:13.744 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:13.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:13.746 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:13.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:13 smithi103 ceph-mon[33163]: 4.9 scrub starts 2024-09-10T18:50:13.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:13 smithi103 ceph-mon[33163]: 4.9 scrub ok 2024-09-10T18:50:13.847 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:13.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:13.871 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:13.871 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:13.972 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:14.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:14.002 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:14.002 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:14.103 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:14.143 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:14.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:14.144 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:14.245 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:14.277 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:14.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:14.278 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:14.380 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:14.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:14.408 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:14.408 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:14.431 INFO:teuthology.orchestra.run.smithi018.stderr:reweighted osd.0 to 1 (10000) 2024-09-10T18:50:14.508 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:14.545 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:14.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:14.546 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:14.591 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is set 2024-09-10T18:50:14.647 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:14.676 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:14.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:14.677 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:14.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:14 smithi018 ceph-mon[40196]: 7.3 scrub starts 2024-09-10T18:50:14.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:14 smithi018 ceph-mon[40196]: 2.2 deep-scrub starts 2024-09-10T18:50:14.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:14 smithi018 ceph-mon[40196]: 2.2 deep-scrub ok 2024-09-10T18:50:14.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:14 smithi018 ceph-mon[40196]: pgmap v192: 158 pgs: 158 active+clean; 4.3 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 31 KiB/s rd, 219 KiB/s wr, 1.10k op/s; 241 KiB/s, 0 objects/s recovering 2024-09-10T18:50:14.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:14 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2941586634' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:50:14.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:14 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:50:14.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:14 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2419769202' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-10T18:50:14.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:14 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-10T18:50:14.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:14 smithi018 ceph-mon[29336]: 7.3 scrub starts 2024-09-10T18:50:14.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:14 smithi018 ceph-mon[29336]: 2.2 deep-scrub starts 2024-09-10T18:50:14.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:14 smithi018 ceph-mon[29336]: 2.2 deep-scrub ok 2024-09-10T18:50:14.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:14 smithi018 ceph-mon[29336]: pgmap v192: 158 pgs: 158 active+clean; 4.3 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 31 KiB/s rd, 219 KiB/s wr, 1.10k op/s; 241 KiB/s, 0 objects/s recovering 2024-09-10T18:50:14.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:14 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2941586634' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:50:14.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:14 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:50:14.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:14 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2419769202' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-10T18:50:14.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:14 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-10T18:50:14.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:14 smithi103 ceph-mon[33163]: 7.3 scrub starts 2024-09-10T18:50:14.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:14 smithi103 ceph-mon[33163]: 2.2 deep-scrub starts 2024-09-10T18:50:14.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:14 smithi103 ceph-mon[33163]: 2.2 deep-scrub ok 2024-09-10T18:50:14.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:14 smithi103 ceph-mon[33163]: pgmap v192: 158 pgs: 158 active+clean; 4.3 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 31 KiB/s rd, 219 KiB/s wr, 1.10k op/s; 241 KiB/s, 0 objects/s recovering 2024-09-10T18:50:14.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:14 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2941586634' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:50:14.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:14 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:50:14.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:14 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2419769202' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-10T18:50:14.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:14 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-10T18:50:14.777 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:14.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:14.818 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:14.818 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:14.919 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:14.967 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:14.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:14.969 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:15.070 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:15.096 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:15.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:15.097 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:15.198 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:15.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:15.230 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:15.230 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:15.331 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:15.362 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:15.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:15.363 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:15.464 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:15.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:15.495 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:15.495 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:15.544 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:15 smithi103 ceph-mon[33163]: 6.13 scrub starts 2024-09-10T18:50:15.545 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:15 smithi103 ceph-mon[33163]: 6.13 scrub ok 2024-09-10T18:50:15.545 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:15 smithi103 ceph-mon[33163]: 2.3 scrub starts 2024-09-10T18:50:15.545 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:15 smithi103 ceph-mon[33163]: 2.3 scrub ok 2024-09-10T18:50:15.545 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:15 smithi103 ceph-mon[33163]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:50:15.545 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:15 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:50:15.545 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:15 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-10T18:50:15.545 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:15 smithi103 ceph-mon[33163]: osdmap e131: 8 total, 8 up, 7 in 2024-09-10T18:50:15.545 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:15 smithi103 ceph-mon[33163]: pgmap v194: 158 pgs: 158 active+clean; 4.3 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 27 KiB/s rd, 191 KiB/s wr, 754 op/s; 210 KiB/s, 0 objects/s recovering 2024-09-10T18:50:15.583 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T18:50:15.584 INFO:teuthology.orchestra.run.smithi103.stdout: "size": 20, 2024-09-10T18:50:15.584 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 600, 2024-09-10T18:50:15.584 INFO:teuthology.orchestra.run.smithi103.stdout: "ops": [ 2024-09-10T18:50:15.584 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.584 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:15105 7.7 7:eab71933:::benchmark_data_smithi018_78628_object15104:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-10T18:50:15.584 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:47:49.065245+0000", 2024-09-10T18:50:15.584 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 146.513463497, 2024-09-10T18:50:15.584 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.14106566300000001, 2024-09-10T18:50:15.584 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:50:15.584 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:50:15.585 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:50:15.585 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:50:15.585 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:50:15.585 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 15105 2024-09-10T18:50:15.585 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.585 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:50:15.585 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.585 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:50:15.585 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.065245+0000", 2024-09-10T18:50:15.586 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.586 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.586 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.586 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:50:15.586 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.065245+0000", 2024-09-10T18:50:15.586 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.586 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.586 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.586 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:50:15.586 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.065244+0000", 2024-09-10T18:50:15.586 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:50:15.586 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.586 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.586 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:50:15.587 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.065303+0000", 2024-09-10T18:50:15.587 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.9091000000000001e-05 2024-09-10T18:50:15.587 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.587 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.587 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:50:15.587 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.065315+0000", 2024-09-10T18:50:15.587 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.2245999999999999e-05 2024-09-10T18:50:15.587 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.587 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.587 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:50:15.587 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.065323+0000", 2024-09-10T18:50:15.587 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.2490000000000001e-06 2024-09-10T18:50:15.588 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.588 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.588 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:50:15.588 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.065366+0000", 2024-09-10T18:50:15.588 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.2777000000000001e-05 2024-09-10T18:50:15.588 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.588 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.588 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:50:15.588 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.065485+0000", 2024-09-10T18:50:15.588 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000118678 2024-09-10T18:50:15.588 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.588 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.589 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,3", 2024-09-10T18:50:15.589 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.065614+0000", 2024-09-10T18:50:15.589 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000129794 2024-09-10T18:50:15.589 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.589 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.589 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:50:15.589 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.066616+0000", 2024-09-10T18:50:15.589 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0010014819999999999 2024-09-10T18:50:15.589 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.589 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.589 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.590 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.070060+0000", 2024-09-10T18:50:15.590 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0034437949999999999 2024-09-10T18:50:15.590 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.590 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.590 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.590 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.206213+0000", 2024-09-10T18:50:15.591 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.136153679 2024-09-10T18:50:15.591 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.591 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.591 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:50:15.591 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.206268+0000", 2024-09-10T18:50:15.591 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.4923000000000003e-05 2024-09-10T18:50:15.591 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.591 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.591 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:50:15.591 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.206311+0000", 2024-09-10T18:50:15.592 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.2447999999999997e-05 2024-09-10T18:50:15.592 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.592 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:50:15.592 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.592 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.592 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.592 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:15112 7.d 7:b7509d44:::benchmark_data_smithi018_78628_object15111:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-10T18:50:15.592 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:47:49.068020+0000", 2024-09-10T18:50:15.592 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 146.51068897600001, 2024-09-10T18:50:15.592 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.13858927500000001, 2024-09-10T18:50:15.592 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:50:15.592 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:50:15.593 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:50:15.593 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:50:15.593 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:50:15.593 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 15112 2024-09-10T18:50:15.593 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.593 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:50:15.593 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.593 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:50:15.593 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.068020+0000", 2024-09-10T18:50:15.593 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.593 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.593 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.593 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:50:15.593 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.068020+0000", 2024-09-10T18:50:15.594 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.594 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.594 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.594 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:50:15.594 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.068018+0000", 2024-09-10T18:50:15.594 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:50:15.594 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.594 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.594 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:50:15.594 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.068098+0000", 2024-09-10T18:50:15.594 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.9399000000000003e-05 2024-09-10T18:50:15.594 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.594 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.595 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:50:15.595 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.068109+0000", 2024-09-10T18:50:15.595 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.1885999999999999e-05 2024-09-10T18:50:15.595 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.595 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.595 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:50:15.595 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.068115+0000", 2024-09-10T18:50:15.595 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.6849999999999998e-06 2024-09-10T18:50:15.595 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.595 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.595 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:50:15.595 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.068169+0000", 2024-09-10T18:50:15.595 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.3706999999999999e-05 2024-09-10T18:50:15.596 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.596 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.596 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:50:15.596 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.068288+0000", 2024-09-10T18:50:15.596 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000119145 2024-09-10T18:50:15.596 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.596 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.596 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1", 2024-09-10T18:50:15.596 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.068418+0000", 2024-09-10T18:50:15.596 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00013021899999999999 2024-09-10T18:50:15.596 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.596 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.596 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:50:15.597 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.069301+0000", 2024-09-10T18:50:15.597 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00088306099999999998 2024-09-10T18:50:15.597 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.597 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.597 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.597 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.071310+0000", 2024-09-10T18:50:15.597 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.002008747 2024-09-10T18:50:15.597 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.597 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.597 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.597 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.206544+0000", 2024-09-10T18:50:15.597 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.13523421799999999 2024-09-10T18:50:15.598 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.598 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.598 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:50:15.598 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.206590+0000", 2024-09-10T18:50:15.598 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.5500000000000001e-05 2024-09-10T18:50:15.598 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.598 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.598 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:50:15.598 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.206609+0000", 2024-09-10T18:50:15.598 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.9148e-05 2024-09-10T18:50:15.598 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.599 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:50:15.599 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.599 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.599 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.599 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:15115 7.7 7:e5c9908f:::benchmark_data_smithi018_78628_object15114:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-10T18:50:15.599 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:47:49.068386+0000", 2024-09-10T18:50:15.599 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 146.51032220799999, 2024-09-10T18:50:15.599 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.142342727, 2024-09-10T18:50:15.599 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:50:15.600 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:50:15.600 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:50:15.600 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:50:15.600 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:50:15.600 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 15115 2024-09-10T18:50:15.600 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.600 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:50:15.600 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.600 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:50:15.600 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.068386+0000", 2024-09-10T18:50:15.600 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.600 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.601 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.601 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:50:15.601 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.068386+0000", 2024-09-10T18:50:15.601 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.601 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.601 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.601 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:50:15.601 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.068386+0000", 2024-09-10T18:50:15.601 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:50:15.601 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.601 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.601 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:50:15.602 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.068472+0000", 2024-09-10T18:50:15.602 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.6475000000000001e-05 2024-09-10T18:50:15.602 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.602 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.602 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:50:15.602 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.068482+0000", 2024-09-10T18:50:15.602 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.6749999999999997e-06 2024-09-10T18:50:15.602 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.602 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.602 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:50:15.603 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.068487+0000", 2024-09-10T18:50:15.603 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.6130000000000003e-06 2024-09-10T18:50:15.603 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.603 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.603 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:50:15.603 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.068510+0000", 2024-09-10T18:50:15.603 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.2408999999999998e-05 2024-09-10T18:50:15.604 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.604 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.604 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:50:15.604 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.068577+0000", 2024-09-10T18:50:15.604 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.7699000000000004e-05 2024-09-10T18:50:15.604 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.605 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.605 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,3", 2024-09-10T18:50:15.605 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.068690+0000", 2024-09-10T18:50:15.605 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000112496 2024-09-10T18:50:15.605 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.605 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.606 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:50:15.606 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.071004+0000", 2024-09-10T18:50:15.606 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0023137990000000001 2024-09-10T18:50:15.606 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.606 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.606 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.606 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.204590+0000", 2024-09-10T18:50:15.607 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.13358632000000001 2024-09-10T18:50:15.607 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.607 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.607 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.607 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.210646+0000", 2024-09-10T18:50:15.607 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.006056367 2024-09-10T18:50:15.607 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.608 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.608 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:50:15.608 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.210692+0000", 2024-09-10T18:50:15.608 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.5713000000000001e-05 2024-09-10T18:50:15.608 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.608 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.608 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:50:15.609 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.210729+0000", 2024-09-10T18:50:15.609 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.6980000000000002e-05 2024-09-10T18:50:15.609 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.609 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:50:15.609 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.610 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.610 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.610 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:15119 7.d 7:be09ff45:::benchmark_data_smithi018_78628_object15118:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-10T18:50:15.610 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:47:49.069602+0000", 2024-09-10T18:50:15.610 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 146.50910695600001, 2024-09-10T18:50:15.611 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.14115032299999999, 2024-09-10T18:50:15.611 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:50:15.611 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:50:15.611 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:50:15.611 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:50:15.612 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:50:15.612 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 15119 2024-09-10T18:50:15.612 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.612 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:50:15.612 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.612 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:50:15.612 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.069602+0000", 2024-09-10T18:50:15.612 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.613 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.613 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.613 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:50:15.613 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.069602+0000", 2024-09-10T18:50:15.613 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.613 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.613 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.613 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:50:15.613 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.069549+0000", 2024-09-10T18:50:15.613 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999471 2024-09-10T18:50:15.613 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.613 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.614 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:50:15.614 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.069638+0000", 2024-09-10T18:50:15.614 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.9189000000000003e-05 2024-09-10T18:50:15.614 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.614 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.614 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:50:15.614 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.069649+0000", 2024-09-10T18:50:15.614 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.1233000000000001e-05 2024-09-10T18:50:15.614 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.614 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.614 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:50:15.614 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.069655+0000", 2024-09-10T18:50:15.614 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.7010000000000001e-06 2024-09-10T18:50:15.615 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.615 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.615 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:50:15.615 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.069714+0000", 2024-09-10T18:50:15.615 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.8666999999999999e-05 2024-09-10T18:50:15.615 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.615 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.615 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:50:15.615 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.069920+0000", 2024-09-10T18:50:15.615 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00020687400000000001 2024-09-10T18:50:15.616 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.616 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.616 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1", 2024-09-10T18:50:15.616 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.070727+0000", 2024-09-10T18:50:15.616 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00080686699999999996 2024-09-10T18:50:15.616 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.616 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.616 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:50:15.616 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.072018+0000", 2024-09-10T18:50:15.616 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.001290217 2024-09-10T18:50:15.616 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.616 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.616 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.617 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.072043+0000", 2024-09-10T18:50:15.617 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.5795999999999998e-05 2024-09-10T18:50:15.617 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.617 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.617 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.617 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.210677+0000", 2024-09-10T18:50:15.617 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.13863404300000001 2024-09-10T18:50:15.617 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.617 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.617 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:50:15.617 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.210721+0000", 2024-09-10T18:50:15.617 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.3427000000000003e-05 2024-09-10T18:50:15.618 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.618 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.618 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:50:15.618 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.210752+0000", 2024-09-10T18:50:15.618 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.1189999999999998e-05 2024-09-10T18:50:15.618 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.618 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:50:15.618 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.618 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.618 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.618 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:15122 7.7 7:ee9d697a:::benchmark_data_smithi018_78628_object15121:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-10T18:50:15.618 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:47:49.069766+0000", 2024-09-10T18:50:15.618 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 146.50894266899999, 2024-09-10T18:50:15.619 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.14106569199999999, 2024-09-10T18:50:15.619 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:50:15.619 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:50:15.619 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:50:15.619 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:50:15.619 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:50:15.619 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 15122 2024-09-10T18:50:15.619 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.619 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:50:15.619 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.619 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:50:15.619 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.069766+0000", 2024-09-10T18:50:15.619 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.619 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.620 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.620 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:50:15.620 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.069766+0000", 2024-09-10T18:50:15.620 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.620 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.620 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.620 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:50:15.620 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.069765+0000", 2024-09-10T18:50:15.620 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:15.622 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:50:15.622 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.622 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.622 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:50:15.622 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.069922+0000", 2024-09-10T18:50:15.622 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000157028 2024-09-10T18:50:15.622 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.622 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.622 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:50:15.622 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.069932+0000", 2024-09-10T18:50:15.622 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.9660000000000008e-06 2024-09-10T18:50:15.622 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.622 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.623 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:50:15.623 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.070005+0000", 2024-09-10T18:50:15.623 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.2491999999999996e-05 2024-09-10T18:50:15.623 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.623 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.623 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:50:15.623 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.070172+0000", 2024-09-10T18:50:15.623 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00016714 2024-09-10T18:50:15.623 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.623 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.623 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:50:15.623 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.070286+0000", 2024-09-10T18:50:15.623 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00011426599999999999 2024-09-10T18:50:15.624 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.624 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.624 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,3", 2024-09-10T18:50:15.624 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.070448+0000", 2024-09-10T18:50:15.624 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000162254 2024-09-10T18:50:15.624 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.624 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.624 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:50:15.624 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.071207+0000", 2024-09-10T18:50:15.624 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00075872099999999996 2024-09-10T18:50:15.624 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.624 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.624 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.625 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.208999+0000", 2024-09-10T18:50:15.625 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.13779218700000001 2024-09-10T18:50:15.625 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.625 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.625 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.625 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.210763+0000", 2024-09-10T18:50:15.625 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.001764231 2024-09-10T18:50:15.625 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.625 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.625 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:50:15.625 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.210793+0000", 2024-09-10T18:50:15.625 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.9889000000000001e-05 2024-09-10T18:50:15.626 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.626 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.626 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:50:15.626 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.210832+0000", 2024-09-10T18:50:15.626 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.8297999999999998e-05 2024-09-10T18:50:15.626 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.626 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:50:15.626 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.626 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.626 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.626 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:15124 7.d 7:b976dd72:::benchmark_data_smithi018_78628_object15123:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-10T18:50:15.626 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:47:49.071346+0000", 2024-09-10T18:50:15.626 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 146.50736228900001, 2024-09-10T18:50:15.627 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.139713948, 2024-09-10T18:50:15.627 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:50:15.627 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:50:15.627 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:50:15.627 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:50:15.627 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:50:15.627 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 15124 2024-09-10T18:50:15.627 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.627 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:50:15.627 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.627 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:50:15.627 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.071346+0000", 2024-09-10T18:50:15.627 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.627 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.628 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.628 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:50:15.628 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.071346+0000", 2024-09-10T18:50:15.628 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.628 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.628 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.628 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:50:15.628 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.071345+0000", 2024-09-10T18:50:15.628 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:50:15.628 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.628 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.628 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:50:15.629 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.071438+0000", 2024-09-10T18:50:15.629 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.2262999999999994e-05 2024-09-10T18:50:15.629 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.629 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.629 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:50:15.629 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.071449+0000", 2024-09-10T18:50:15.629 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.1552999999999999e-05 2024-09-10T18:50:15.629 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.629 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.629 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:50:15.629 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.071454+0000", 2024-09-10T18:50:15.629 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.0560000000000004e-06 2024-09-10T18:50:15.629 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.630 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.630 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:50:15.630 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.071493+0000", 2024-09-10T18:50:15.630 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.8414999999999997e-05 2024-09-10T18:50:15.630 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.630 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.630 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:50:15.630 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.071574+0000", 2024-09-10T18:50:15.630 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.1453999999999996e-05 2024-09-10T18:50:15.630 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.630 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.630 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1", 2024-09-10T18:50:15.630 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.071662+0000", 2024-09-10T18:50:15.631 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.7712000000000003e-05 2024-09-10T18:50:15.631 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.631 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.631 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:50:15.631 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.072157+0000", 2024-09-10T18:50:15.631 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00049492899999999996 2024-09-10T18:50:15.631 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.631 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.631 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.631 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.072393+0000", 2024-09-10T18:50:15.631 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000236135 2024-09-10T18:50:15.631 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.631 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.631 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.632 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.210977+0000", 2024-09-10T18:50:15.632 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.13858467699999999 2024-09-10T18:50:15.632 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.632 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.632 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:50:15.632 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.211024+0000", 2024-09-10T18:50:15.632 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.6901999999999999e-05 2024-09-10T18:50:15.632 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.632 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.632 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:50:15.632 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.211060+0000", 2024-09-10T18:50:15.632 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.5908999999999998e-05 2024-09-10T18:50:15.633 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.633 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:50:15.633 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.633 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.633 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.633 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:22451 7.7 7:e131e84d:::benchmark_data_smithi018_78628_object22450:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-10T18:50:15.633 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:47:52.070968+0000", 2024-09-10T18:50:15.633 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 143.50774028399999, 2024-09-10T18:50:15.633 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.094587983, 2024-09-10T18:50:15.633 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:50:15.633 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:50:15.633 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:50:15.633 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:50:15.634 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:50:15.634 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 22451 2024-09-10T18:50:15.634 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.634 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:50:15.634 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.634 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:50:15.634 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.070968+0000", 2024-09-10T18:50:15.634 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.634 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.634 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.634 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:50:15.634 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.070968+0000", 2024-09-10T18:50:15.634 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.634 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.635 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.635 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:50:15.635 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.070968+0000", 2024-09-10T18:50:15.635 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:50:15.635 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.635 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.635 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:50:15.635 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.071024+0000", 2024-09-10T18:50:15.635 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.6020999999999998e-05 2024-09-10T18:50:15.635 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.635 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.635 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:50:15.635 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.071033+0000", 2024-09-10T18:50:15.636 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.6080000000000002e-06 2024-09-10T18:50:15.636 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.636 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.636 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:50:15.636 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.071038+0000", 2024-09-10T18:50:15.636 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.1209999999999998e-06 2024-09-10T18:50:15.636 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.636 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.636 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:50:15.637 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.071078+0000", 2024-09-10T18:50:15.637 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.9993000000000001e-05 2024-09-10T18:50:15.637 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.637 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.637 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:50:15.637 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.071151+0000", 2024-09-10T18:50:15.637 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.2348000000000006e-05 2024-09-10T18:50:15.637 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.637 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.637 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,3", 2024-09-10T18:50:15.637 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.071234+0000", 2024-09-10T18:50:15.637 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.3697999999999997e-05 2024-09-10T18:50:15.637 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.638 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.638 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:50:15.638 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.072003+0000", 2024-09-10T18:50:15.638 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00076912400000000004 2024-09-10T18:50:15.638 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.638 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.638 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.638 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.164531+0000", 2024-09-10T18:50:15.638 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.092527988000000005 2024-09-10T18:50:15.638 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.639 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.639 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.639 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.165484+0000", 2024-09-10T18:50:15.639 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00095271099999999997 2024-09-10T18:50:15.639 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.639 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.639 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:50:15.639 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.165525+0000", 2024-09-10T18:50:15.639 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.1152000000000002e-05 2024-09-10T18:50:15.639 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.639 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.639 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:50:15.639 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.165556+0000", 2024-09-10T18:50:15.639 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.1034000000000001e-05 2024-09-10T18:50:15.640 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.640 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:50:15.640 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.640 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.640 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.640 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:22454 7.d 7:bc395976:::benchmark_data_smithi018_78628_object22453:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-10T18:50:15.640 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:47:52.072344+0000", 2024-09-10T18:50:15.640 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 143.50636442999999, 2024-09-10T18:50:15.640 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.093087982999999999, 2024-09-10T18:50:15.640 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:50:15.640 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:50:15.640 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:50:15.641 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:50:15.641 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:50:15.641 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 22454 2024-09-10T18:50:15.641 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.641 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:50:15.641 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.641 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:50:15.641 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.072344+0000", 2024-09-10T18:50:15.641 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.641 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.641 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.641 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:50:15.642 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.072344+0000", 2024-09-10T18:50:15.642 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.642 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.642 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.642 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:50:15.642 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.072343+0000", 2024-09-10T18:50:15.642 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:50:15.642 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.642 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.642 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:50:15.642 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.072395+0000", 2024-09-10T18:50:15.642 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.151e-05 2024-09-10T18:50:15.643 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.643 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.643 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:50:15.643 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.072405+0000", 2024-09-10T18:50:15.643 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0029e-05 2024-09-10T18:50:15.643 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.643 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.643 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:50:15.643 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.072412+0000", 2024-09-10T18:50:15.643 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.9739999999999996e-06 2024-09-10T18:50:15.644 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.644 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.644 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:50:15.644 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.072445+0000", 2024-09-10T18:50:15.644 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.2808999999999997e-05 2024-09-10T18:50:15.644 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.644 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.644 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:50:15.645 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.072523+0000", 2024-09-10T18:50:15.645 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.8051000000000002e-05 2024-09-10T18:50:15.645 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.645 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.645 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1", 2024-09-10T18:50:15.645 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.072599+0000", 2024-09-10T18:50:15.645 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.5847999999999996e-05 2024-09-10T18:50:15.645 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.645 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.645 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:50:15.645 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.073129+0000", 2024-09-10T18:50:15.645 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00053029100000000001 2024-09-10T18:50:15.646 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.646 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.646 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.646 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.075360+0000", 2024-09-10T18:50:15.646 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0022311599999999998 2024-09-10T18:50:15.646 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.646 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.646 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.646 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.165348+0000", 2024-09-10T18:50:15.646 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.089988316999999998 2024-09-10T18:50:15.646 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.647 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.647 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:50:15.647 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.165392+0000", 2024-09-10T18:50:15.647 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.4014999999999998e-05 2024-09-10T18:50:15.647 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.647 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.647 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:50:15.647 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.165432+0000", 2024-09-10T18:50:15.647 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.9844e-05 2024-09-10T18:50:15.648 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.648 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:50:15.648 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.648 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.648 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.648 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:22460 7.d 7:ba89638b:::benchmark_data_smithi018_78628_object22459:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-10T18:50:15.648 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:47:52.075162+0000", 2024-09-10T18:50:15.648 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 143.50354614400001, 2024-09-10T18:50:15.648 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.092846752000000005, 2024-09-10T18:50:15.648 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:50:15.648 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:50:15.648 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:50:15.649 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:50:15.649 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:50:15.649 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 22460 2024-09-10T18:50:15.649 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.649 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:50:15.649 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.649 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:50:15.649 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.075162+0000", 2024-09-10T18:50:15.649 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.649 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.649 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.649 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:50:15.650 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.075162+0000", 2024-09-10T18:50:15.650 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.650 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.650 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.650 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:50:15.650 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.075162+0000", 2024-09-10T18:50:15.650 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:50:15.650 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.650 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.650 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:50:15.650 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.075404+0000", 2024-09-10T18:50:15.651 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000242036 2024-09-10T18:50:15.651 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.651 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.651 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:50:15.651 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.075411+0000", 2024-09-10T18:50:15.651 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.5820000000000004e-06 2024-09-10T18:50:15.651 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.651 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.651 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:50:15.651 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.075416+0000", 2024-09-10T18:50:15.651 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.374e-06 2024-09-10T18:50:15.651 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.651 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.651 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:50:15.652 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.075454+0000", 2024-09-10T18:50:15.652 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.8849e-05 2024-09-10T18:50:15.652 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.652 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.652 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:50:15.652 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.164141+0000", 2024-09-10T18:50:15.652 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.088686451999999999 2024-09-10T18:50:15.652 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.652 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.652 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1", 2024-09-10T18:50:15.652 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.164331+0000", 2024-09-10T18:50:15.652 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00019046199999999999 2024-09-10T18:50:15.653 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.653 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.653 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:50:15.653 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.165831+0000", 2024-09-10T18:50:15.653 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0014998570000000001 2024-09-10T18:50:15.653 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.653 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.653 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.653 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.167119+0000", 2024-09-10T18:50:15.653 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0012874270000000001 2024-09-10T18:50:15.653 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.653 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.653 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.654 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.167961+0000", 2024-09-10T18:50:15.654 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00084205 2024-09-10T18:50:15.654 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.654 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.654 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:50:15.654 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.167989+0000", 2024-09-10T18:50:15.654 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.8518e-05 2024-09-10T18:50:15.654 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.654 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.654 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:50:15.654 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:52.168009+0000", 2024-09-10T18:50:15.654 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.0083000000000001e-05 2024-09-10T18:50:15.654 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.655 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:50:15.655 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.655 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.655 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.655 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:28501 7.7 7:ea9bf763:::benchmark_data_smithi018_78628_object28500:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-10T18:50:15.655 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:47:54.624655+0000", 2024-09-10T18:50:15.655 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 140.95405332499999, 2024-09-10T18:50:15.655 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.120695913, 2024-09-10T18:50:15.655 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:50:15.655 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:50:15.655 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:50:15.655 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:50:15.656 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:50:15.656 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 28501 2024-09-10T18:50:15.656 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.656 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:50:15.656 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.656 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:50:15.656 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:54.624655+0000", 2024-09-10T18:50:15.656 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.656 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.656 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.656 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:50:15.656 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:54.624655+0000", 2024-09-10T18:50:15.657 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.657 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.657 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.657 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:50:15.657 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:54.624654+0000", 2024-09-10T18:50:15.657 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999981 2024-09-10T18:50:15.657 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.657 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.657 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:50:15.657 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:54.624705+0000", 2024-09-10T18:50:15.657 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.1461000000000001e-05 2024-09-10T18:50:15.657 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.658 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.658 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:50:15.658 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:54.624720+0000", 2024-09-10T18:50:15.658 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.4439e-05 2024-09-10T18:50:15.658 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.658 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.658 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:50:15.658 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:54.624724+0000", 2024-09-10T18:50:15.658 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.921e-06 2024-09-10T18:50:15.658 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.658 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.658 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:50:15.658 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:54.624764+0000", 2024-09-10T18:50:15.658 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.9811000000000002e-05 2024-09-10T18:50:15.659 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.659 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.659 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:50:15.659 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:54.624895+0000", 2024-09-10T18:50:15.659 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000130457 2024-09-10T18:50:15.659 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.659 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.659 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,3", 2024-09-10T18:50:15.659 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:54.625036+0000", 2024-09-10T18:50:15.660 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00014170799999999999 2024-09-10T18:50:15.660 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.660 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.660 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:50:15.660 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:54.626071+0000", 2024-09-10T18:50:15.660 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.001034777 2024-09-10T18:50:15.660 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.660 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.660 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.660 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:54.741328+0000", 2024-09-10T18:50:15.660 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.11525671799999999 2024-09-10T18:50:15.660 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.660 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.660 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.661 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:54.745271+0000", 2024-09-10T18:50:15.661 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0039430990000000003 2024-09-10T18:50:15.661 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.661 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.661 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:50:15.661 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:54.745317+0000", 2024-09-10T18:50:15.661 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.5794000000000002e-05 2024-09-10T18:50:15.661 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.661 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.661 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:50:15.661 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:54.745351+0000", 2024-09-10T18:50:15.661 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.4409000000000002e-05 2024-09-10T18:50:15.661 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.661 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:50:15.662 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.662 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.662 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.662 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:31044 7.c 7:3c910e97:::benchmark_data_smithi018_78628_object31043:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-10T18:50:15.662 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:47:55.984502+0000", 2024-09-10T18:50:15.662 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 139.59420642500001, 2024-09-10T18:50:15.662 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.095102464999999997, 2024-09-10T18:50:15.662 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:50:15.662 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:50:15.662 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:50:15.662 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:50:15.662 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:50:15.663 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 31044 2024-09-10T18:50:15.663 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.663 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:50:15.663 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.663 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:50:15.663 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.984502+0000", 2024-09-10T18:50:15.663 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.663 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.663 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.663 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:50:15.663 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.984502+0000", 2024-09-10T18:50:15.664 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.664 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.664 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.664 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:50:15.664 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.984501+0000", 2024-09-10T18:50:15.664 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:50:15.664 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.664 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.664 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:50:15.664 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.984603+0000", 2024-09-10T18:50:15.664 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000101928 2024-09-10T18:50:15.664 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.664 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.664 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:50:15.664 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.984614+0000", 2024-09-10T18:50:15.665 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0839000000000001e-05 2024-09-10T18:50:15.665 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.665 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.665 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:50:15.665 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.984618+0000", 2024-09-10T18:50:15.665 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.2509999999999996e-06 2024-09-10T18:50:15.665 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.665 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.665 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:50:15.665 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.984673+0000", 2024-09-10T18:50:15.665 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.5120000000000001e-05 2024-09-10T18:50:15.665 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.666 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.666 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:50:15.666 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.984797+0000", 2024-09-10T18:50:15.666 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00012430699999999999 2024-09-10T18:50:15.666 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.666 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.666 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,7", 2024-09-10T18:50:15.666 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.984929+0000", 2024-09-10T18:50:15.666 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00013193699999999999 2024-09-10T18:50:15.666 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.666 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.666 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:50:15.667 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.986282+0000", 2024-09-10T18:50:15.667 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0013522600000000001 2024-09-10T18:50:15.667 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.667 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.667 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.667 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:55.986620+0000", 2024-09-10T18:50:15.667 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00033869899999999998 2024-09-10T18:50:15.667 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.667 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.667 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.667 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:56.079519+0000", 2024-09-10T18:50:15.667 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.092899027999999995 2024-09-10T18:50:15.667 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.668 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.668 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:50:15.668 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:56.079570+0000", 2024-09-10T18:50:15.668 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.0942000000000001e-05 2024-09-10T18:50:15.668 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.668 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.668 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:50:15.668 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:56.079605+0000", 2024-09-10T18:50:15.668 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.4477999999999998e-05 2024-09-10T18:50:15.668 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.668 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:50:15.668 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.669 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.669 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.669 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:109205 7.c 7:31293bd7:::benchmark_data_smithi018_78628_object109204:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-10T18:50:15.669 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:48:50.762657+0000", 2024-09-10T18:50:15.669 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 84.816051634999994, 2024-09-10T18:50:15.669 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.114499398, 2024-09-10T18:50:15.669 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:50:15.669 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:50:15.669 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:50:15.669 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:50:15.669 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:50:15.669 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 109205 2024-09-10T18:50:15.670 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.670 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:50:15.670 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.670 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:50:15.670 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.762657+0000", 2024-09-10T18:50:15.670 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.670 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.670 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.670 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:50:15.670 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.762657+0000", 2024-09-10T18:50:15.670 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.670 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.671 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.671 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:50:15.671 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.762656+0000", 2024-09-10T18:50:15.671 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:50:15.671 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.671 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.671 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:50:15.671 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.762708+0000", 2024-09-10T18:50:15.671 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.2298e-05 2024-09-10T18:50:15.671 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.671 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.671 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:50:15.671 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.762718+0000", 2024-09-10T18:50:15.672 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0023999999999999e-05 2024-09-10T18:50:15.672 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.672 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.672 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:50:15.672 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.762723+0000", 2024-09-10T18:50:15.672 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.6190000000000002e-06 2024-09-10T18:50:15.672 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.672 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.672 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:50:15.672 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.762775+0000", 2024-09-10T18:50:15.672 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.2178000000000001e-05 2024-09-10T18:50:15.672 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.673 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.673 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:50:15.673 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.762918+0000", 2024-09-10T18:50:15.673 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000143304 2024-09-10T18:50:15.673 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.673 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.673 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,7", 2024-09-10T18:50:15.673 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.763061+0000", 2024-09-10T18:50:15.673 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00014251199999999999 2024-09-10T18:50:15.673 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.673 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.673 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.673 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.766001+0000", 2024-09-10T18:50:15.674 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0029404729999999999 2024-09-10T18:50:15.674 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.674 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.674 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:50:15.674 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.766162+0000", 2024-09-10T18:50:15.674 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00016106200000000001 2024-09-10T18:50:15.674 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.674 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.674 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.674 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.877079+0000", 2024-09-10T18:50:15.674 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.110916627 2024-09-10T18:50:15.674 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.674 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.675 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:50:15.675 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.877119+0000", 2024-09-10T18:50:15.675 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.0071000000000003e-05 2024-09-10T18:50:15.675 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.675 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.675 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:50:15.675 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.877156+0000", 2024-09-10T18:50:15.675 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.7455e-05 2024-09-10T18:50:15.675 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.675 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:50:15.675 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.675 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.676 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.676 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:109213 7.7 7:e820724c:::benchmark_data_smithi018_78628_object109212:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-10T18:50:15.676 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:48:50.770196+0000", 2024-09-10T18:50:15.676 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 84.808512837999999, 2024-09-10T18:50:15.676 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.11007088700000001, 2024-09-10T18:50:15.676 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:50:15.676 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:50:15.676 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:50:15.676 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:50:15.676 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:50:15.676 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 109213 2024-09-10T18:50:15.677 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.677 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:50:15.677 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.677 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:50:15.677 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.770196+0000", 2024-09-10T18:50:15.677 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.677 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.677 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.677 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:50:15.677 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.770196+0000", 2024-09-10T18:50:15.677 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.677 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.677 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.677 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:50:15.678 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.770195+0000", 2024-09-10T18:50:15.678 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:50:15.678 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.678 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.678 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:50:15.678 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.770251+0000", 2024-09-10T18:50:15.678 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.6177000000000001e-05 2024-09-10T18:50:15.678 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.678 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.678 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:50:15.678 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.770261+0000", 2024-09-10T18:50:15.678 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0000000000000001e-05 2024-09-10T18:50:15.678 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.679 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.679 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:50:15.679 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.770265+0000", 2024-09-10T18:50:15.679 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.279e-06 2024-09-10T18:50:15.679 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.679 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.679 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:50:15.679 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.770307+0000", 2024-09-10T18:50:15.679 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.1934000000000002e-05 2024-09-10T18:50:15.679 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.679 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.679 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:50:15.679 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.770388+0000", 2024-09-10T18:50:15.680 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.0674999999999996e-05 2024-09-10T18:50:15.680 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.680 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.680 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,3", 2024-09-10T18:50:15.680 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.770472+0000", 2024-09-10T18:50:15.680 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.3961999999999993e-05 2024-09-10T18:50:15.680 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.680 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.680 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:50:15.680 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.772686+0000", 2024-09-10T18:50:15.680 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0022142479999999998 2024-09-10T18:50:15.680 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.681 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.681 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.681 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.878924+0000", 2024-09-10T18:50:15.681 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.106237763 2024-09-10T18:50:15.681 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.681 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.681 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.681 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.880182+0000", 2024-09-10T18:50:15.681 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0012578190000000001 2024-09-10T18:50:15.681 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.681 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.681 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:50:15.681 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.880230+0000", 2024-09-10T18:50:15.681 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.8198000000000002e-05 2024-09-10T18:50:15.682 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.682 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.682 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:50:15.682 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:48:50.880267+0000", 2024-09-10T18:50:15.682 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.6807999999999997e-05 2024-09-10T18:50:15.682 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.682 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:50:15.682 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.682 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.682 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.682 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:197438 7.c 7:336074c8:::benchmark_data_smithi018_78628_object61218:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-10T18:50:15.684 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:50:07.641559+0000", 2024-09-10T18:50:15.684 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 7.9371492129999996, 2024-09-10T18:50:15.684 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.092455809999999999, 2024-09-10T18:50:15.684 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:50:15.684 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:50:15.684 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:50:15.684 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:50:15.684 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:50:15.684 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 197438 2024-09-10T18:50:15.684 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.684 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:50:15.684 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.685 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:50:15.685 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.641559+0000", 2024-09-10T18:50:15.685 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.685 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.685 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.685 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:50:15.685 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.641559+0000", 2024-09-10T18:50:15.685 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.685 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.685 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.685 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:50:15.685 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.641559+0000", 2024-09-10T18:50:15.685 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:50:15.685 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.686 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.686 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:50:15.686 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.641563+0000", 2024-09-10T18:50:15.686 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.1359999999999999e-06 2024-09-10T18:50:15.686 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.686 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.686 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:50:15.686 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.641570+0000", 2024-09-10T18:50:15.686 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.5129999999999999e-06 2024-09-10T18:50:15.686 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.686 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.686 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:50:15.687 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.641575+0000", 2024-09-10T18:50:15.687 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.7090000000000001e-06 2024-09-10T18:50:15.687 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.687 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.687 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:50:15.687 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.641617+0000", 2024-09-10T18:50:15.687 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.2389999999999999e-05 2024-09-10T18:50:15.687 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.687 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.687 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:50:15.687 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.641717+0000", 2024-09-10T18:50:15.687 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.9939000000000007e-05 2024-09-10T18:50:15.687 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.687 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.688 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,7", 2024-09-10T18:50:15.688 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.641833+0000", 2024-09-10T18:50:15.688 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000116104 2024-09-10T18:50:15.688 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.688 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.688 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:50:15.688 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.644731+0000", 2024-09-10T18:50:15.688 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0028980770000000002 2024-09-10T18:50:15.688 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.688 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.688 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.688 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.644998+0000", 2024-09-10T18:50:15.688 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00026693700000000002 2024-09-10T18:50:15.689 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.689 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.689 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.689 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.733952+0000", 2024-09-10T18:50:15.689 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.088953303999999997 2024-09-10T18:50:15.689 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.689 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.689 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:50:15.689 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.733985+0000", 2024-09-10T18:50:15.689 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.3008e-05 2024-09-10T18:50:15.689 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.689 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.689 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:50:15.690 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.734015+0000", 2024-09-10T18:50:15.690 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.0460999999999999e-05 2024-09-10T18:50:15.690 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.690 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:50:15.690 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.690 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.690 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.690 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:197439 7.7 7:e841dde5:::benchmark_data_smithi018_78628_object61219:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-10T18:50:15.690 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:50:07.641620+0000", 2024-09-10T18:50:15.690 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 7.9370884650000004, 2024-09-10T18:50:15.690 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.091947688, 2024-09-10T18:50:15.690 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:50:15.690 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:50:15.690 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:50:15.691 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:50:15.691 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:50:15.691 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 197439 2024-09-10T18:50:15.691 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.691 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:50:15.691 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.691 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:50:15.691 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.641620+0000", 2024-09-10T18:50:15.691 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.691 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.691 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.691 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:50:15.691 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.641620+0000", 2024-09-10T18:50:15.691 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.691 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.692 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.692 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:50:15.692 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.641620+0000", 2024-09-10T18:50:15.692 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999995 2024-09-10T18:50:15.692 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.692 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.692 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:50:15.692 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.641623+0000", 2024-09-10T18:50:15.692 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.5319999999999998e-06 2024-09-10T18:50:15.692 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.692 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.692 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:50:15.692 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.641630+0000", 2024-09-10T18:50:15.692 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.7880000000000001e-06 2024-09-10T18:50:15.693 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.693 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.693 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:50:15.693 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.641635+0000", 2024-09-10T18:50:15.693 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.904e-06 2024-09-10T18:50:15.693 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.693 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.693 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:50:15.693 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.641667+0000", 2024-09-10T18:50:15.693 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.2282e-05 2024-09-10T18:50:15.693 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.693 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.693 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:50:15.693 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.641762+0000", 2024-09-10T18:50:15.694 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.4518999999999994e-05 2024-09-10T18:50:15.694 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.694 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.694 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,3", 2024-09-10T18:50:15.694 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.641868+0000", 2024-09-10T18:50:15.694 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000106524 2024-09-10T18:50:15.694 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.694 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.694 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:50:15.694 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.647055+0000", 2024-09-10T18:50:15.694 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0051865899999999996 2024-09-10T18:50:15.694 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.694 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.695 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.695 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.650736+0000", 2024-09-10T18:50:15.695 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0036812780000000001 2024-09-10T18:50:15.695 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.695 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.695 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.695 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.733509+0000", 2024-09-10T18:50:15.695 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.082773166999999995 2024-09-10T18:50:15.695 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.695 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.695 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:50:15.695 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.733542+0000", 2024-09-10T18:50:15.695 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.2416999999999998e-05 2024-09-10T18:50:15.695 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.695 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.696 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:50:15.696 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.733568+0000", 2024-09-10T18:50:15.696 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.6325999999999998e-05 2024-09-10T18:50:15.696 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.696 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:50:15.696 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.696 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.696 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.696 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:197441 7.7 7:e17ccabd:::benchmark_data_smithi018_78628_object61221:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-10T18:50:15.696 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:50:07.642125+0000", 2024-09-10T18:50:15.696 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 7.9365836229999998, 2024-09-10T18:50:15.696 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.091522216000000003, 2024-09-10T18:50:15.696 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:50:15.696 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:50:15.697 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:50:15.697 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:50:15.697 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:50:15.697 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 197441 2024-09-10T18:50:15.697 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.697 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:50:15.697 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.697 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:50:15.697 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.642125+0000", 2024-09-10T18:50:15.697 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.697 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.697 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.697 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:50:15.697 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.642125+0000", 2024-09-10T18:50:15.698 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.698 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.698 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.698 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:50:15.698 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.642124+0000", 2024-09-10T18:50:15.698 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:50:15.698 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.698 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.698 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:50:15.698 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.642129+0000", 2024-09-10T18:50:15.698 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.9840000000000001e-06 2024-09-10T18:50:15.698 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.698 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.699 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:50:15.699 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.642139+0000", 2024-09-10T18:50:15.699 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0282000000000001e-05 2024-09-10T18:50:15.699 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.699 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.699 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:50:15.699 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.642145+0000", 2024-09-10T18:50:15.699 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.0859999999999998e-06 2024-09-10T18:50:15.699 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.699 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.699 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:50:15.699 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.644682+0000", 2024-09-10T18:50:15.699 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0025362150000000001 2024-09-10T18:50:15.699 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.700 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.700 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:50:15.700 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.644801+0000", 2024-09-10T18:50:15.700 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000119584 2024-09-10T18:50:15.700 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.700 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.700 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,3", 2024-09-10T18:50:15.700 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.645008+0000", 2024-09-10T18:50:15.700 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00020688 2024-09-10T18:50:15.700 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.700 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.700 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:50:15.700 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.647254+0000", 2024-09-10T18:50:15.700 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0022461719999999998 2024-09-10T18:50:15.701 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.701 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.701 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.701 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.650762+0000", 2024-09-10T18:50:15.701 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0035076809999999999 2024-09-10T18:50:15.701 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.701 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.701 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.701 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.733597+0000", 2024-09-10T18:50:15.701 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.082834906 2024-09-10T18:50:15.701 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.701 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.701 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:50:15.701 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.733624+0000", 2024-09-10T18:50:15.702 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.7566999999999999e-05 2024-09-10T18:50:15.702 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.702 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.702 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:50:15.702 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:07.733647+0000", 2024-09-10T18:50:15.702 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.2711e-05 2024-09-10T18:50:15.702 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.702 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:50:15.702 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.702 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.702 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.702 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:198292 7.c 7:333a3e4a:::benchmark_data_smithi018_78628_object62072:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-10T18:50:15.702 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:50:08.409986+0000", 2024-09-10T18:50:15.703 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 7.1687222589999999, 2024-09-10T18:50:15.703 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.13510712999999999, 2024-09-10T18:50:15.703 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:50:15.703 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:50:15.703 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:50:15.703 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:50:15.703 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:50:15.703 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 198292 2024-09-10T18:50:15.703 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.703 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:50:15.703 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.703 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:50:15.703 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:08.409986+0000", 2024-09-10T18:50:15.703 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.704 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.704 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.704 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:50:15.704 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:08.409986+0000", 2024-09-10T18:50:15.704 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.704 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.704 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.704 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:50:15.704 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:08.409985+0000", 2024-09-10T18:50:15.704 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:50:15.704 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.704 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.704 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:50:15.704 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:08.409991+0000", 2024-09-10T18:50:15.705 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.1090000000000003e-06 2024-09-10T18:50:15.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.705 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.705 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:50:15.705 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:08.410008+0000", 2024-09-10T18:50:15.705 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.6441999999999999e-05 2024-09-10T18:50:15.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.705 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.705 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:50:15.705 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:08.410014+0000", 2024-09-10T18:50:15.705 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.3770000000000001e-06 2024-09-10T18:50:15.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.705 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.705 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:50:15.706 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:08.411583+0000", 2024-09-10T18:50:15.706 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0015692 2024-09-10T18:50:15.706 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.706 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.706 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:50:15.706 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:08.411682+0000", 2024-09-10T18:50:15.706 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.8467000000000004e-05 2024-09-10T18:50:15.706 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.706 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.706 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,7", 2024-09-10T18:50:15.706 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:08.411777+0000", 2024-09-10T18:50:15.706 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.4951999999999995e-05 2024-09-10T18:50:15.706 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.706 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.707 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:50:15.707 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:08.413945+0000", 2024-09-10T18:50:15.707 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0021688419999999998 2024-09-10T18:50:15.707 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.707 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.707 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.707 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:08.418309+0000", 2024-09-10T18:50:15.707 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.004363675 2024-09-10T18:50:15.707 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.707 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.707 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.707 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:08.545007+0000", 2024-09-10T18:50:15.707 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.12669828399999999 2024-09-10T18:50:15.708 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.708 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.708 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:50:15.708 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:08.545056+0000", 2024-09-10T18:50:15.708 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.8384000000000003e-05 2024-09-10T18:50:15.708 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.708 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.708 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:50:15.708 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:08.545093+0000", 2024-09-10T18:50:15.708 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.7675000000000001e-05 2024-09-10T18:50:15.708 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.708 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:50:15.708 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.709 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.709 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.709 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:198304 7.7 7:ee12c7de:::benchmark_data_smithi018_78628_object62084:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-10T18:50:15.709 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:50:08.417595+0000", 2024-09-10T18:50:15.709 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 7.1611136259999997, 2024-09-10T18:50:15.709 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.128516515, 2024-09-10T18:50:15.709 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:50:15.709 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:50:15.709 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:50:15.709 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:50:15.709 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:50:15.709 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 198304 2024-09-10T18:50:15.709 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.710 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:50:15.710 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.710 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:50:15.710 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:08.417595+0000", 2024-09-10T18:50:15.710 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.710 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.710 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.710 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:50:15.710 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:08.417595+0000", 2024-09-10T18:50:15.710 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.710 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.710 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.710 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:50:15.710 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:08.417594+0000", 2024-09-10T18:50:15.711 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:50:15.711 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.711 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.711 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:50:15.711 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:08.417600+0000", 2024-09-10T18:50:15.711 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.0979999999999999e-06 2024-09-10T18:50:15.711 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.711 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.711 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:50:15.712 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:08.417609+0000", 2024-09-10T18:50:15.712 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.764e-06 2024-09-10T18:50:15.712 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.712 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.712 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:50:15.712 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:08.417616+0000", 2024-09-10T18:50:15.712 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.3570000000000003e-06 2024-09-10T18:50:15.712 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.712 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.712 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:50:15.712 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:08.417694+0000", 2024-09-10T18:50:15.712 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.8670000000000004e-05 2024-09-10T18:50:15.713 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.713 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.713 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:50:15.713 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:08.417819+0000", 2024-09-10T18:50:15.713 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000124043 2024-09-10T18:50:15.713 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.713 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.713 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,3", 2024-09-10T18:50:15.713 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:08.417934+0000", 2024-09-10T18:50:15.713 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00011595999999999999 2024-09-10T18:50:15.713 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.713 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.713 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:50:15.714 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:08.419871+0000", 2024-09-10T18:50:15.714 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0019365529999999999 2024-09-10T18:50:15.714 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.714 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.714 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.714 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:08.420761+0000", 2024-09-10T18:50:15.714 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00088986499999999999 2024-09-10T18:50:15.714 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.714 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.714 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.714 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:08.546039+0000", 2024-09-10T18:50:15.714 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.12527845600000001 2024-09-10T18:50:15.714 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.714 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.715 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:50:15.715 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:08.546076+0000", 2024-09-10T18:50:15.715 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.6859000000000001e-05 2024-09-10T18:50:15.715 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.715 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.715 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:50:15.715 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:08.546111+0000", 2024-09-10T18:50:15.715 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.5299000000000002e-05 2024-09-10T18:50:15.715 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.715 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:50:15.715 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.715 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.715 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.715 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:198933 7.7 7:e7708c25:::benchmark_data_smithi018_78628_object62713:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-10T18:50:15.716 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:50:09.081029+0000", 2024-09-10T18:50:15.716 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 6.4976794770000001, 2024-09-10T18:50:15.716 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.097467554999999997, 2024-09-10T18:50:15.716 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:50:15.716 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:50:15.716 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:50:15.716 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:50:15.716 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:50:15.716 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 198933 2024-09-10T18:50:15.716 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.716 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:50:15.716 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.716 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:50:15.717 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.081029+0000", 2024-09-10T18:50:15.717 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.717 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.717 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.717 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:50:15.717 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.081029+0000", 2024-09-10T18:50:15.717 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.717 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.717 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.717 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:50:15.717 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.081028+0000", 2024-09-10T18:50:15.717 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:50:15.717 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.718 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.718 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:50:15.718 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.081034+0000", 2024-09-10T18:50:15.718 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.8170000000000004e-06 2024-09-10T18:50:15.718 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.718 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.718 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:50:15.718 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.081044+0000", 2024-09-10T18:50:15.718 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0242e-05 2024-09-10T18:50:15.718 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.718 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.718 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:50:15.718 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.081049+0000", 2024-09-10T18:50:15.718 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.6970000000000001e-06 2024-09-10T18:50:15.719 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.719 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.719 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:50:15.719 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.081100+0000", 2024-09-10T18:50:15.719 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.1456000000000003e-05 2024-09-10T18:50:15.719 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.719 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.719 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:50:15.719 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.081190+0000", 2024-09-10T18:50:15.719 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.9920999999999995e-05 2024-09-10T18:50:15.719 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.719 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.719 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,3", 2024-09-10T18:50:15.720 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.081274+0000", 2024-09-10T18:50:15.720 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.4078000000000004e-05 2024-09-10T18:50:15.720 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.720 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.720 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:50:15.720 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.085811+0000", 2024-09-10T18:50:15.720 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0045361669999999998 2024-09-10T18:50:15.720 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.720 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.720 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.720 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.086425+0000", 2024-09-10T18:50:15.720 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00061399800000000004 2024-09-10T18:50:15.721 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.721 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.721 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.721 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.178421+0000", 2024-09-10T18:50:15.721 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.091996300000000003 2024-09-10T18:50:15.721 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.721 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.721 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:50:15.721 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.178463+0000", 2024-09-10T18:50:15.721 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.2512999999999998e-05 2024-09-10T18:50:15.721 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.721 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.721 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:50:15.722 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.178497+0000", 2024-09-10T18:50:15.722 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.3359999999999999e-05 2024-09-10T18:50:15.722 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.722 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:50:15.722 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.722 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.722 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.722 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:198936 7.7 7:e8f1e011:::benchmark_data_smithi018_78628_object62716:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-10T18:50:15.722 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:50:09.081236+0000", 2024-09-10T18:50:15.722 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 6.497472846, 2024-09-10T18:50:15.722 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.105979642, 2024-09-10T18:50:15.722 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:50:15.722 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:50:15.722 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:50:15.723 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:50:15.723 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:50:15.723 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 198936 2024-09-10T18:50:15.723 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.723 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:50:15.723 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.723 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:50:15.723 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.081236+0000", 2024-09-10T18:50:15.723 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.723 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.723 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.723 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:50:15.723 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.081236+0000", 2024-09-10T18:50:15.724 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:50:15.724 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.724 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.724 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:50:15.724 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.081235+0000", 2024-09-10T18:50:15.724 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999995 2024-09-10T18:50:15.724 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.724 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.724 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:50:15.724 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.081239+0000", 2024-09-10T18:50:15.724 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.6200000000000001e-06 2024-09-10T18:50:15.724 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.724 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.725 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:50:15.725 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.081250+0000", 2024-09-10T18:50:15.725 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0852e-05 2024-09-10T18:50:15.725 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.725 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.725 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:50:15.725 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.081253+0000", 2024-09-10T18:50:15.725 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.7390000000000001e-06 2024-09-10T18:50:15.725 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.725 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.725 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:50:15.725 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.083745+0000", 2024-09-10T18:50:15.725 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0024915499999999999 2024-09-10T18:50:15.726 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.726 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.726 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:50:15.726 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.083880+0000", 2024-09-10T18:50:15.726 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000134716 2024-09-10T18:50:15.726 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.726 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.726 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,3", 2024-09-10T18:50:15.726 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.083978+0000", 2024-09-10T18:50:15.726 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.8451999999999999e-05 2024-09-10T18:50:15.726 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.726 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.726 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:50:15.726 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.085886+0000", 2024-09-10T18:50:15.727 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.001907588 2024-09-10T18:50:15.727 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.727 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.727 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.727 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.086500+0000", 2024-09-10T18:50:15.727 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000614746 2024-09-10T18:50:15.727 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.727 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.727 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:50:15.727 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.187159+0000", 2024-09-10T18:50:15.727 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.10065856300000001 2024-09-10T18:50:15.727 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.727 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.727 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:50:15.727 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.187185+0000", 2024-09-10T18:50:15.728 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.6220999999999998e-05 2024-09-10T18:50:15.728 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:50:15.728 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:50:15.728 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:50:15.728 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:50:09.187215+0000", 2024-09-10T18:50:15.728 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.0176e-05 2024-09-10T18:50:15.728 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.728 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:50:15.728 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.728 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:50:15.728 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:50:15.728 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T18:50:15.729 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:15 smithi018 ceph-mon[29336]: 6.13 scrub starts 2024-09-10T18:50:15.729 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:15 smithi018 ceph-mon[29336]: 6.13 scrub ok 2024-09-10T18:50:15.730 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:15 smithi018 ceph-mon[29336]: 2.3 scrub starts 2024-09-10T18:50:15.730 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:15 smithi018 ceph-mon[29336]: 2.3 scrub ok 2024-09-10T18:50:15.730 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:15 smithi018 ceph-mon[29336]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:50:15.730 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:15 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:50:15.730 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:15 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-10T18:50:15.730 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:15 smithi018 ceph-mon[29336]: osdmap e131: 8 total, 8 up, 7 in 2024-09-10T18:50:15.730 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:15 smithi018 ceph-mon[29336]: pgmap v194: 158 pgs: 158 active+clean; 4.3 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 27 KiB/s rd, 191 KiB/s wr, 754 op/s; 210 KiB/s, 0 objects/s recovering 2024-09-10T18:50:15.730 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:15 smithi018 ceph-mon[40196]: 6.13 scrub starts 2024-09-10T18:50:15.730 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:15 smithi018 ceph-mon[40196]: 6.13 scrub ok 2024-09-10T18:50:15.730 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:15 smithi018 ceph-mon[40196]: 2.3 scrub starts 2024-09-10T18:50:15.731 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:15 smithi018 ceph-mon[40196]: 2.3 scrub ok 2024-09-10T18:50:15.731 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:15 smithi018 ceph-mon[40196]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:50:15.731 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:15 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:50:15.731 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:15 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-10T18:50:15.731 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:15 smithi018 ceph-mon[40196]: osdmap e131: 8 total, 8 up, 7 in 2024-09-10T18:50:15.731 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:15 smithi018 ceph-mon[40196]: pgmap v194: 158 pgs: 158 active+clean; 4.3 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 27 KiB/s rd, 191 KiB/s wr, 754 op/s; 210 KiB/s, 0 objects/s recovering 2024-09-10T18:50:15.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:15.732 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:15.732 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:15.833 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:15.865 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:15.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:15.866 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:15.967 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:15.995 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:15.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:15.996 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:16.097 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:16.144 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:16.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:16.145 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:16.181 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:50:16.212 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd reweight 1 1 2024-09-10T18:50:16.246 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:16.301 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:16.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:16.302 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:16.403 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:16.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:16.449 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:16.449 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:16.550 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:16.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:16.596 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:16.596 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:16.646 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:16 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:16.646 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:50:16 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:50:16.682 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-10T18:50:16.697 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:16.724 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:16 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:16.724 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:16 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:16.774 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:16.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:16.775 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:16.876 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:16.908 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:16.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:16.909 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:17.010 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:17.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:17.052 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:17.052 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:17.067 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:50:17.153 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:17.183 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:17.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:17.184 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:17.284 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:17.327 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:17.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:17.328 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:17.429 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-10T18:50:17.429 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:17.452 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:17.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:17.453 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:17.542 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:50:17.554 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:17.578 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:17.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:17.579 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:17.680 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:17.735 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:17.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:17.736 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:17.739 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:17 smithi018 ceph-mon[29336]: pgmap v195: 158 pgs: 158 active+clean; 3.9 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 24 KiB/s rd, 169 KiB/s wr, 1.41k op/s; 186 KiB/s, 0 objects/s recovering 2024-09-10T18:50:17.739 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:17 smithi018 ceph-mon[40196]: pgmap v195: 158 pgs: 158 active+clean; 3.9 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 24 KiB/s rd, 169 KiB/s wr, 1.41k op/s; 186 KiB/s, 0 objects/s recovering 2024-09-10T18:50:17.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:17 smithi103 ceph-mon[33163]: pgmap v195: 158 pgs: 158 active+clean; 3.9 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 24 KiB/s rd, 169 KiB/s wr, 1.41k op/s; 186 KiB/s, 0 objects/s recovering 2024-09-10T18:50:17.837 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:17.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:17.893 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:17.893 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:17.994 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:18.020 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:18.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:18.021 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:18.122 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:18.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:18.164 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:18.164 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:18.265 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:18.293 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:18.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:18.294 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:18.395 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:18.436 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:18.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:18.437 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:18.538 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:18.570 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:18.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:18.571 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:18.672 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:18.700 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:18.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:18.701 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:18.802 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:18.852 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:50:18.853 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:50:18.861 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:18.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:18.862 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:18.893 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:50:18.893 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:50:18.923 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:18 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2341236644' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:50:18.924 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:50:18.924 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:50:18.924 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:18 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2341236644' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:50:18.963 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:18.967 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:50:18.967 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:50:18.998 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:50:18.998 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:50:19.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:19.010 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:19.010 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:19.030 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:50:19.030 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:50:19.065 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:50:19.065 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:50:19.108 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:50:19.109 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:50:19.111 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:19.139 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:19.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:19.140 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:19.241 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:19.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:18 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2341236644' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:50:19.277 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:19.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:19.278 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:19.379 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:19.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:19.413 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:19.414 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:19.514 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:19.552 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:19.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:19.553 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:19.654 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:19.684 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:19.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:19.685 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:19.786 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:19.817 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:19.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:19.819 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:19.919 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:19.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:19.964 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:19.964 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:20.065 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:20.091 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:20.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:20.092 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:20.140 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:19 smithi018 ceph-mon[40196]: pgmap v196: 158 pgs: 158 active+clean; 3.9 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 21 KiB/s rd, 148 KiB/s wr, 1.23k op/s; 162 KiB/s, 0 objects/s recovering 2024-09-10T18:50:20.140 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:19 smithi018 ceph-mon[29336]: pgmap v196: 158 pgs: 158 active+clean; 3.9 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 21 KiB/s rd, 148 KiB/s wr, 1.23k op/s; 162 KiB/s, 0 objects/s recovering 2024-09-10T18:50:20.140 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:50:20.193 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:20.234 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:20.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:20.236 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:20.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:19 smithi103 ceph-mon[33163]: pgmap v196: 158 pgs: 158 active+clean; 3.9 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 21 KiB/s rd, 148 KiB/s wr, 1.23k op/s; 162 KiB/s, 0 objects/s recovering 2024-09-10T18:50:20.337 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:20.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:20.370 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:20.371 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:20.472 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:20.497 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:50:20 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:50:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:50:20.508 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:20.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:20.509 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:20.609 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:20.650 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:20.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:20.651 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:20.752 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:20.759 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T18:50:20.759 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:50:20.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:20.794 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:20.794 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:20.895 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:20.934 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:20.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:20.935 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:20.964 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:20 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:50:20.966 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:20 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:50:21.036 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:21.079 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:21.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:21.081 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:21.181 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:21.226 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:21.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:21.227 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:21.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:20 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:50:21.328 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:21.369 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:50:20 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:50:20] "GET /metrics HTTP/1.1" 200 223806 "" "Prometheus/2.43.0" 2024-09-10T18:50:21.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:21.393 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:21.393 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:21.487 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-10T18:50:21.494 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:21.560 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:21.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:21.561 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:21.662 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:21.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:21.701 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:21.701 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:21.801 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:21.827 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:50:21.832 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:21.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:21.833 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:21.913 INFO:teuthology.orchestra.run.smithi018.stderr:reweighted osd.1 to 1 (10000) 2024-09-10T18:50:21.934 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:21.962 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:21.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:21.963 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:22.064 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:22.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:22.106 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:22.107 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:22.140 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T18:50:22.148 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:21 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:50:22.148 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:21 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:50:22.148 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:21 smithi103 ceph-mon[33163]: pgmap v197: 158 pgs: 158 active+clean; 3.7 GiB data, 13 GiB used, 612 GiB / 626 GiB avail; 5.0 KiB/s rd, 30 KiB/s wr, 1.26k op/s; 53 KiB/s, 0 objects/s recovering 2024-09-10T18:50:22.148 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:21 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:22.148 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:21 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:50:22.148 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:21 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:50:22.187 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:21 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:50:22.187 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:21 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:50:22.188 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:21 smithi018 ceph-mon[29336]: pgmap v197: 158 pgs: 158 active+clean; 3.7 GiB data, 13 GiB used, 612 GiB / 626 GiB avail; 5.0 KiB/s rd, 30 KiB/s wr, 1.26k op/s; 53 KiB/s, 0 objects/s recovering 2024-09-10T18:50:22.188 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:21 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:22.188 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:21 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:50:22.188 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:21 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:50:22.188 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:21 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:50:22.188 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:21 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:50:22.188 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:21 smithi018 ceph-mon[40196]: pgmap v197: 158 pgs: 158 active+clean; 3.7 GiB data, 13 GiB used, 612 GiB / 626 GiB avail; 5.0 KiB/s rd, 30 KiB/s wr, 1.26k op/s; 53 KiB/s, 0 objects/s recovering 2024-09-10T18:50:22.188 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:21 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:22.188 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:21 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:50:22.188 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:21 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:50:22.208 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:22.235 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:22.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:22.236 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:22.337 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:22.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:22.384 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:22.384 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:22.485 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:22.530 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:22.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:22.531 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:22.632 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:22.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:22.686 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:22.686 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:22.693 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd reweight 3 1 2024-09-10T18:50:22.787 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:22.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:22.826 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:22.827 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:22.928 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:22.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:22 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2391560027' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-10T18:50:22.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:22 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2391560027' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-10T18:50:22.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:22 smithi018 ceph-mon[29336]: osdmap e132: 8 total, 8 up, 7 in 2024-09-10T18:50:22.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:22 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2391560027' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-10T18:50:22.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:22 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2391560027' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-10T18:50:22.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:22 smithi018 ceph-mon[40196]: osdmap e132: 8 total, 8 up, 7 in 2024-09-10T18:50:23.006 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:23.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:23.008 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:23.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:22 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2391560027' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-10T18:50:23.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:22 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2391560027' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-10T18:50:23.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:22 smithi103 ceph-mon[33163]: osdmap e132: 8 total, 8 up, 7 in 2024-09-10T18:50:23.109 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:23.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:23.169 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:23.169 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:23.270 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:23.283 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:50:23.334 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:50:23.339 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:23.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:23.343 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:23.443 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:23.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:23.503 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:23.503 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:23.604 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:23.656 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:23.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:23.657 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:23.758 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:23.795 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:23.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:23.796 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:23.897 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:23.916 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is set 2024-09-10T18:50:23.937 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:23.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:23.938 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:24.039 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:24.068 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:24.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:24.069 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:24.170 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:24.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:23 smithi018 ceph-mon[29336]: pgmap v199: 158 pgs: 158 active+clean; 3.5 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.50k op/s 2024-09-10T18:50:24.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:23 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1506908892' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:50:24.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:23 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:50:24.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:23 smithi018 ceph-mon[40196]: pgmap v199: 158 pgs: 158 active+clean; 3.5 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.50k op/s 2024-09-10T18:50:24.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:23 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1506908892' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:50:24.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:23 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:50:24.211 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:24.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:24.212 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:24.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:23 smithi103 ceph-mon[33163]: pgmap v199: 158 pgs: 158 active+clean; 3.5 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.50k op/s 2024-09-10T18:50:24.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:23 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1506908892' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:50:24.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:23 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:50:24.314 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:24.347 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:24.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:24.348 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:24.449 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:24.483 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:24.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:24.484 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:24.516 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T18:50:24.516 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:50:24.585 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:24.623 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:24.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:24.624 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:24.725 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:24.762 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:24.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:24.763 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:24.864 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:24.901 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:24.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:24.902 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:25.002 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:25.043 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:25.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:25.045 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:25.146 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:25.177 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:25.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:25.179 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:25.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:24 smithi018 ceph-mon[40196]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:50:25.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:24 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:50:25.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:24 smithi018 ceph-mon[40196]: osdmap e133: 8 total, 8 up, 7 in 2024-09-10T18:50:25.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:24 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:50:25.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:24 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:50:25.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:24 smithi018 ceph-mon[29336]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:50:25.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:24 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:50:25.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:24 smithi018 ceph-mon[29336]: osdmap e133: 8 total, 8 up, 7 in 2024-09-10T18:50:25.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:24 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:50:25.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:24 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:50:25.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:24 smithi103 ceph-mon[33163]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:50:25.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:24 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:50:25.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:24 smithi103 ceph-mon[33163]: osdmap e133: 8 total, 8 up, 7 in 2024-09-10T18:50:25.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:24 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:50:25.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:24 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:50:25.280 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:25.310 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:25.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:25.311 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:25.411 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:25.452 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:25.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:25.453 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:25.477 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-10T18:50:25.554 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:25.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:25.596 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:25.596 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:25.697 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:25.735 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:25.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:25.737 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:25.835 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:50:25.838 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:25.879 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:25.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:25.880 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:25.981 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:26.008 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:26.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:26.009 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:26.110 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:26.123 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:25 smithi018 ceph-mon[40196]: osdmap e134: 8 total, 8 up, 7 in 2024-09-10T18:50:26.123 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:25 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:50:26.123 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:25 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:50:26.123 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:25 smithi018 ceph-mon[40196]: pgmap v202: 158 pgs: 158 active+clean; 3.5 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.09k op/s 2024-09-10T18:50:26.123 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:25 smithi018 ceph-mon[29336]: osdmap e134: 8 total, 8 up, 7 in 2024-09-10T18:50:26.123 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:25 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:50:26.123 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:25 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:50:26.123 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:25 smithi018 ceph-mon[29336]: pgmap v202: 158 pgs: 158 active+clean; 3.5 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.09k op/s 2024-09-10T18:50:26.140 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:26.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:26.141 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:26.242 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:26.272 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:26.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:25 smithi103 ceph-mon[33163]: osdmap e134: 8 total, 8 up, 7 in 2024-09-10T18:50:26.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:25 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:50:26.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:25 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:50:26.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:25 smithi103 ceph-mon[33163]: pgmap v202: 158 pgs: 158 active+clean; 3.5 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.09k op/s 2024-09-10T18:50:26.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:26.274 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:26.375 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:26.405 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:26.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:26.406 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:26.507 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:26.549 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:26.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:26.550 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:26.651 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:26.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:26.693 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:26.693 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:26.772 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:50:26 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:50:26.793 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:26.829 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:26.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:26.830 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:26.931 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:26.970 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:26.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:26.971 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:27.072 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:27.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:27.100 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:27.100 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:27.201 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:27.243 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:27.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:27.244 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:27.344 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:27.381 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:27 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:27.394 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:27.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:27.396 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:27.406 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:27 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:27.406 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:27 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:27.496 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:27.525 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:27.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:27.526 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:27.627 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:27.669 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:27.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:27.672 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:27.773 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:27.783 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T18:50:27.783 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:50:27.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:27.802 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:27.802 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:27.903 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:27.928 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:27.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:27.929 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:28.030 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:28.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:28.076 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:28.076 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:28.100 INFO:teuthology.orchestra.run.smithi018.stderr:reweighted osd.3 to 1 (10000) 2024-09-10T18:50:28.148 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:50:28.149 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:50:28.177 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:28.190 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:50:28.190 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:50:28.205 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:28.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:28.206 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:28.222 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:50:28.222 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:50:28.223 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:28 smithi018 ceph-mon[40196]: pgmap v203: 158 pgs: 158 active+clean; 3.1 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.64k op/s 2024-09-10T18:50:28.223 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:28 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1116014737' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-10T18:50:28.223 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:28 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-10T18:50:28.223 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:28 smithi018 ceph-mon[29336]: pgmap v203: 158 pgs: 158 active+clean; 3.1 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.64k op/s 2024-09-10T18:50:28.223 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:28 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1116014737' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-10T18:50:28.223 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:28 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-10T18:50:28.258 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:50:28.258 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:50:28.293 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:50:28.293 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:50:28.307 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:28.327 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:50:28.327 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:50:28.335 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:28.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:28.336 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:28.362 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:50:28.362 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:50:28.396 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:50:28.396 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:50:28.436 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:28.463 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:28.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:28.464 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:28.496 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-10T18:50:28.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:28 smithi103 ceph-mon[33163]: pgmap v203: 158 pgs: 158 active+clean; 3.1 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.64k op/s 2024-09-10T18:50:28.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:28 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1116014737' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-10T18:50:28.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:28 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-10T18:50:28.566 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:28.592 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:28.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:28.593 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:28.631 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-10T18:50:28.694 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:28.720 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:28.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:28.721 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:28.821 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:28.880 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd reweight 4 1 2024-09-10T18:50:28.908 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:28.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:28.909 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:29.010 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:29.054 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:29.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:29.055 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:29.155 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:29.206 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:29.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:29.208 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:29.308 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:29.347 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:29.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:29.348 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:29.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:29 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-10T18:50:29.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:29 smithi018 ceph-mon[40196]: osdmap e135: 8 total, 8 up, 7 in 2024-09-10T18:50:29.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:29 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/542674854' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:50:29.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:29 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-10T18:50:29.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:29 smithi018 ceph-mon[29336]: osdmap e135: 8 total, 8 up, 7 in 2024-09-10T18:50:29.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:29 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/542674854' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:50:29.449 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:29.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:29.476 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:29.477 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:29.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:29 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-10T18:50:29.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:29 smithi103 ceph-mon[33163]: osdmap e135: 8 total, 8 up, 7 in 2024-09-10T18:50:29.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:29 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/542674854' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:50:29.578 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:29.595 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:50:29.603 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:29.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:29.604 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:29.674 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:50:29.705 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:29.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:29.759 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:29.759 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:29.860 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:29.901 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:29.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:29.902 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:30.003 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:30.040 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:30.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:30.041 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:30.142 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:30.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:30.179 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:30.179 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:30.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:30 smithi018 ceph-mon[40196]: pgmap v205: 158 pgs: 158 active+clean; 3.1 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-10T18:50:30.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:30 smithi018 ceph-mon[29336]: pgmap v205: 158 pgs: 158 active+clean; 3.1 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-10T18:50:30.281 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:30.313 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:30.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:30.314 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:30.415 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:30.444 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:30.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:30.445 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:30.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:30 smithi103 ceph-mon[33163]: pgmap v205: 158 pgs: 158 active+clean; 3.1 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-10T18:50:30.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:50:30 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:50:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:50:30.546 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:30.576 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:30.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:30.577 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:30.678 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:30.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:30.707 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:30.708 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:30.808 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:30.836 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:30.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:30.837 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:30.940 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:30.977 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:30.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:30.979 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:31.079 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:31.119 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:31.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:31.120 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:31.220 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:31.244 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:31.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:31.245 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:31.346 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:31.373 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:31.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:31.374 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:31.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:50:30 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:50:30] "GET /metrics HTTP/1.1" 200 223795 "" "Prometheus/2.43.0" 2024-09-10T18:50:31.475 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:31.504 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:31.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:31.506 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:31.598 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T18:50:31.607 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:31.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:31.688 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:31.688 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:31.789 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:31.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:31.841 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:31.841 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:31.942 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:31.985 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:31.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:31.986 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:32.087 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:32.118 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:32.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:32.119 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:32.220 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:32.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:32.261 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:32.261 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:32.362 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:32.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:32.384 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:32.384 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:32.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:32 smithi018 ceph-mon[40196]: pgmap v206: 158 pgs: 158 active+clean; 2.9 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 711 B/s rd, 0 B/s wr, 1.33k op/s 2024-09-10T18:50:32.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:32 smithi018 ceph-mon[29336]: pgmap v206: 158 pgs: 158 active+clean; 2.9 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 711 B/s rd, 0 B/s wr, 1.33k op/s 2024-09-10T18:50:32.485 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:32.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:32 smithi103 ceph-mon[33163]: pgmap v206: 158 pgs: 158 active+clean; 2.9 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 711 B/s rd, 0 B/s wr, 1.33k op/s 2024-09-10T18:50:32.526 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:32.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:32.527 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:32.628 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:32.652 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:32.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:32.653 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:32.754 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:32.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:32.797 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:32.797 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:32.898 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:32.926 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:32.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:32.927 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:33.028 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:33.057 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:33.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:33.058 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:33.159 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:33.199 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:33.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:33.200 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:33.301 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:33.323 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:33.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:33.324 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:33.425 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:33.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:33 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:50:33.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:33 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:50:33.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:33 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:50:33.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:33 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:50:33.463 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:33.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:33.464 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:33.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:33 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:50:33.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:33 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:50:33.565 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:33.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:33.588 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:33.588 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:33.689 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:33.721 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:33.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:33.722 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:33.823 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:33.853 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:33.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:33.854 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:33.955 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:33.992 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:33.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:33.993 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:34.094 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:34.116 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:34.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:34.117 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:34.218 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:34.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:34.253 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:34.253 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:34.354 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:34.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:34.379 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:34.379 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:34.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:34 smithi018 ceph-mon[40196]: pgmap v207: 158 pgs: 158 active+clean; 2.8 GiB data, 10 GiB used, 615 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.53k op/s 2024-09-10T18:50:34.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:34 smithi018 ceph-mon[40196]: osdmap e136: 8 total, 8 up, 7 in 2024-09-10T18:50:34.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:34 smithi018 ceph-mon[29336]: pgmap v207: 158 pgs: 158 active+clean; 2.8 GiB data, 10 GiB used, 615 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.53k op/s 2024-09-10T18:50:34.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:34 smithi018 ceph-mon[29336]: osdmap e136: 8 total, 8 up, 7 in 2024-09-10T18:50:34.480 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:34.509 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:34.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:34.510 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:34.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:34 smithi103 ceph-mon[33163]: pgmap v207: 158 pgs: 158 active+clean; 2.8 GiB data, 10 GiB used, 615 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.53k op/s 2024-09-10T18:50:34.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:34 smithi103 ceph-mon[33163]: osdmap e136: 8 total, 8 up, 7 in 2024-09-10T18:50:34.611 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:34.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:34.643 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:34.643 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:34.744 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:34.784 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:34.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:34.785 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:34.886 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:34.924 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:34.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:34.925 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:35.026 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:35.064 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:35.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:35.065 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:35.166 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:35.196 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:35.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:35.197 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:35.298 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:35.333 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:35.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:35.334 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:35.435 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:35.462 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:35.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:35.463 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:35.564 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:35.591 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:35.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:35.593 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:35.693 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:35.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:35.734 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:35.734 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:35.835 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:35.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:35.869 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:35.869 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:35.940 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:50:35.970 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:36.023 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:36.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:36.024 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:36.125 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:36.168 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:36.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:36.169 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:36.270 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:36.298 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:36.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:36.300 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:36.400 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:36.430 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:36.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:36.431 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:36.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:36 smithi018 ceph-mon[29336]: pgmap v209: 158 pgs: 158 active+clean; 2.8 GiB data, 10 GiB used, 615 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 751 op/s 2024-09-10T18:50:36.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:36 smithi018 ceph-mon[40196]: pgmap v209: 158 pgs: 158 active+clean; 2.8 GiB data, 10 GiB used, 615 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 751 op/s 2024-09-10T18:50:36.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:36 smithi103 ceph-mon[33163]: pgmap v209: 158 pgs: 158 active+clean; 2.8 GiB data, 10 GiB used, 615 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 751 op/s 2024-09-10T18:50:36.523 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:50:36 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:50:36.532 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:36.576 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:36.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:36.578 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:36.678 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:36.706 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:36.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:36.707 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:36.808 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:36.838 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:36.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:36.839 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:36.940 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:36.972 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:36.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:36.973 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:37.074 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:37.111 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:37.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:37.112 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:37.212 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:37.240 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:37.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:37.241 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:37.341 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:37.377 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:37.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:37.378 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:37.430 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:37 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:37.431 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:37 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:37.478 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:37.514 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:37.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:37.515 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:37.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:37 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:37.616 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:37.647 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:37.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:37.648 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:37.749 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:37.781 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:37.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:37.782 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:37.882 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:37.908 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:37.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:37.909 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:38.010 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:38.064 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:38.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:38.065 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:38.166 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:38.210 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:38.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:38.211 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:38.312 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:38.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:38.355 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:38.355 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:38.456 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:38.484 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:38.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:38.485 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:38.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:38 smithi103 ceph-mon[33163]: pgmap v210: 158 pgs: 158 active+clean; 2.4 GiB data, 9.4 GiB used, 616 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.35k op/s 2024-09-10T18:50:38.571 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:38 smithi018 ceph-mon[40196]: pgmap v210: 158 pgs: 158 active+clean; 2.4 GiB data, 9.4 GiB used, 616 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.35k op/s 2024-09-10T18:50:38.571 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:38 smithi018 ceph-mon[29336]: pgmap v210: 158 pgs: 158 active+clean; 2.4 GiB data, 9.4 GiB used, 616 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.35k op/s 2024-09-10T18:50:38.587 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:38.612 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:38.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:38.613 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:38.714 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:38.741 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:38.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:38.742 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:38.842 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:38.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:38.880 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:38.881 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:38.981 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:39.017 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:39.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:39.018 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:39.075 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:50:39.119 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:39.144 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:39.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:39.145 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:39.217 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is unset 2024-09-10T18:50:39.246 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:39.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:39.289 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:39.289 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:39.390 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:39.432 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:39.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:39.433 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:39.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:39 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1574512636' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:50:39.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:39 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1574512636' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:50:39.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:39 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1574512636' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:50:39.534 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:39.577 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:39.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:39.578 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:39.679 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:39.710 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:39.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:39.711 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:39.812 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:39.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:39.852 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:39.852 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:39.953 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:39.995 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:39.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:39.996 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:40.096 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:40.123 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:40.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:40.124 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:40.225 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:40.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:40.265 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:40.265 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:40.366 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:40.388 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:40.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:40.389 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:40.489 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:40.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:40 smithi103 ceph-mon[33163]: pgmap v211: 158 pgs: 158 active+clean; 2.4 GiB data, 9.4 GiB used, 616 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.22k op/s 2024-09-10T18:50:40.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:40 smithi103 ceph-mon[33163]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:50:40.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:40 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1574512636' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:50:40.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:40 smithi103 ceph-mon[33163]: osdmap e137: 8 total, 8 up, 7 in 2024-09-10T18:50:40.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:40 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:50:40.524 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:50:40 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:50:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:50:40.533 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:40 smithi018 ceph-mon[40196]: pgmap v211: 158 pgs: 158 active+clean; 2.4 GiB data, 9.4 GiB used, 616 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.22k op/s 2024-09-10T18:50:40.533 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:40 smithi018 ceph-mon[40196]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:50:40.533 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:40 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1574512636' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:50:40.533 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:40 smithi018 ceph-mon[40196]: osdmap e137: 8 total, 8 up, 7 in 2024-09-10T18:50:40.533 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:40 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:50:40.533 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:40 smithi018 ceph-mon[29336]: pgmap v211: 158 pgs: 158 active+clean; 2.4 GiB data, 9.4 GiB used, 616 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.22k op/s 2024-09-10T18:50:40.533 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:40 smithi018 ceph-mon[29336]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:50:40.533 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:40 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1574512636' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:50:40.533 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:40 smithi018 ceph-mon[29336]: osdmap e137: 8 total, 8 up, 7 in 2024-09-10T18:50:40.533 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:40 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:50:40.540 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:40.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:40.541 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:40.634 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T18:50:40.634 INFO:teuthology.orchestra.run.smithi018.stdout: "ops": [ 2024-09-10T18:50:40.634 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.634 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.14790.0:238361 7.4 7:23ace3db:::benchmark_data_smithi018_78628_object102141:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137)", 2024-09-10T18:50:40.634 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:50:39.937197+0000", 2024-09-10T18:50:40.634 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 0.004798971, 2024-09-10T18:50:40.634 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0048296880000000004, 2024-09-10T18:50:40.635 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:50:40.635 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "waiting for sub ops", 2024-09-10T18:50:40.635 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:50:40.635 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.14790", 2024-09-10T18:50:40.635 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:50:40.635 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 238361 2024-09-10T18:50:40.635 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.635 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:50:40.635 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.635 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:50:40.635 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.937197+0000", 2024-09-10T18:50:40.635 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:50:40.635 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.636 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.636 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:50:40.636 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.937197+0000", 2024-09-10T18:50:40.636 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:50:40.636 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.636 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.636 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:50:40.636 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.937195+0000", 2024-09-10T18:50:40.636 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999981 2024-09-10T18:50:40.636 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.636 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.636 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:50:40.636 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.937203+0000", 2024-09-10T18:50:40.637 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.8720000000000002e-06 2024-09-10T18:50:40.637 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.637 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.637 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:50:40.637 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.937219+0000", 2024-09-10T18:50:40.637 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.5866999999999999e-05 2024-09-10T18:50:40.637 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.637 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.637 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:50:40.637 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.937226+0000", 2024-09-10T18:50:40.637 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.392e-06 2024-09-10T18:50:40.638 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.638 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.638 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:50:40.638 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.937332+0000", 2024-09-10T18:50:40.638 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000106026 2024-09-10T18:50:40.638 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.638 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.638 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:50:40.638 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.937442+0000", 2024-09-10T18:50:40.638 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000110801 2024-09-10T18:50:40.638 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.638 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.638 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,4", 2024-09-10T18:50:40.639 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.937559+0000", 2024-09-10T18:50:40.639 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000117 2024-09-10T18:50:40.639 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:50:40.639 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:50:40.639 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:50:40.639 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.639 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.639 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.14790.0:238362 7.9 7:95fd8ba1:::benchmark_data_smithi018_78628_object102142:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137)", 2024-09-10T18:50:40.639 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:50:39.937258+0000", 2024-09-10T18:50:40.639 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 0.0047383679999999997, 2024-09-10T18:50:40.639 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0048464049999999998, 2024-09-10T18:50:40.639 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:50:40.639 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "waiting for sub ops", 2024-09-10T18:50:40.639 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:50:40.640 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.14790", 2024-09-10T18:50:40.640 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:50:40.640 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 238362 2024-09-10T18:50:40.640 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.640 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:50:40.640 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.640 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:50:40.640 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.937258+0000", 2024-09-10T18:50:40.640 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:50:40.640 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.640 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.640 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:50:40.640 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.937258+0000", 2024-09-10T18:50:40.641 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:50:40.641 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.641 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.641 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:50:40.641 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.937257+0000", 2024-09-10T18:50:40.641 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T18:50:40.641 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.641 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.641 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:50:40.641 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.937261+0000", 2024-09-10T18:50:40.642 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.2119999999999997e-06 2024-09-10T18:50:40.642 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.642 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.642 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:50:40.642 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.937270+0000", 2024-09-10T18:50:40.642 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.1939999999999998e-06 2024-09-10T18:50:40.642 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.642 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.642 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:50:40.642 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.937274+0000", 2024-09-10T18:50:40.642 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.797e-06 2024-09-10T18:50:40.642 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.642 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.643 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:50:40.643 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.937325+0000", 2024-09-10T18:50:40.643 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.1100999999999997e-05 2024-09-10T18:50:40.643 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.643 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.643 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:50:40.643 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.937446+0000", 2024-09-10T18:50:40.643 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000120407 2024-09-10T18:50:40.643 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.643 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.643 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,4", 2024-09-10T18:50:40.643 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.937568+0000", 2024-09-10T18:50:40.643 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00012214999999999999 2024-09-10T18:50:40.644 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.644 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.644 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:50:40.644 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.941403+0000", 2024-09-10T18:50:40.644 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.003834629 2024-09-10T18:50:40.644 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:50:40.644 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:50:40.644 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:50:40.644 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.644 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.644 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.14790.0:238366 7.e e137/88 7:71a82c96:::benchmark_data_smithi018_78628_object102146:head v 137'14805, mlcod=137'14801)", 2024-09-10T18:50:40.644 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:50:39.938851+0000", 2024-09-10T18:50:40.644 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 0.003144867, 2024-09-10T18:50:40.645 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0033280979999999998, 2024-09-10T18:50:40.645 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:50:40.645 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "started", 2024-09-10T18:50:40.645 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:50:40.645 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.645 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:50:40.645 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.938851+0000", 2024-09-10T18:50:40.645 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:50:40.645 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.645 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.645 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:50:40.645 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.938851+0000", 2024-09-10T18:50:40.645 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:50:40.646 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.646 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.646 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:50:40.646 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.938850+0000", 2024-09-10T18:50:40.646 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:50:40.646 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.646 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.646 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:50:40.646 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.938856+0000", 2024-09-10T18:50:40.646 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.0959999999999997e-06 2024-09-10T18:50:40.646 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.646 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.646 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:50:40.646 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.938866+0000", 2024-09-10T18:50:40.647 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.7999999999999993e-06 2024-09-10T18:50:40.647 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.647 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.647 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:50:40.647 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.938872+0000", 2024-09-10T18:50:40.647 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.2650000000000004e-06 2024-09-10T18:50:40.647 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.647 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.647 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:50:40.647 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.938924+0000", 2024-09-10T18:50:40.648 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.2574000000000002e-05 2024-09-10T18:50:40.648 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.648 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.648 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:50:40.648 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.938941+0000", 2024-09-10T18:50:40.648 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.7146000000000002e-05 2024-09-10T18:50:40.648 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:50:40.648 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:50:40.648 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:50:40.648 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.648 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.648 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.14790.0:238360 7.7 e137/86)", 2024-09-10T18:50:40.648 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:50:39.940115+0000", 2024-09-10T18:50:40.649 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 0.0018816410000000001, 2024-09-10T18:50:40.649 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0021169409999999998, 2024-09-10T18:50:40.649 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:50:40.649 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "queued for pg", 2024-09-10T18:50:40.649 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:50:40.649 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.649 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:50:40.649 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.940115+0000", 2024-09-10T18:50:40.649 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:50:40.649 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.649 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.649 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:50:40.650 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.940115+0000", 2024-09-10T18:50:40.650 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:50:40.650 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.650 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.650 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:50:40.650 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.940114+0000", 2024-09-10T18:50:40.650 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:50:40.650 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.650 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.650 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:50:40.650 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.940120+0000", 2024-09-10T18:50:40.650 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.8900000000000004e-06 2024-09-10T18:50:40.650 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.650 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.650 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:50:40.651 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.940129+0000", 2024-09-10T18:50:40.651 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.3680000000000008e-06 2024-09-10T18:50:40.651 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.651 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.651 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:50:40.651 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.940135+0000", 2024-09-10T18:50:40.651 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.8830000000000003e-06 2024-09-10T18:50:40.651 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:50:40.651 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:50:40.651 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:50:40.651 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.651 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.651 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.14790.0:238369 7.6 7:658e7a7d:::benchmark_data_smithi018_78628_object102149:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137)", 2024-09-10T18:50:40.651 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:50:39.940964+0000", 2024-09-10T18:50:40.652 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 0.00103209, 2024-09-10T18:50:40.652 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0013160380000000001, 2024-09-10T18:50:40.652 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:50:40.652 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "waiting for sub ops", 2024-09-10T18:50:40.652 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:50:40.652 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.14790", 2024-09-10T18:50:40.652 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:50:40.652 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 238369 2024-09-10T18:50:40.652 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.653 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:50:40.653 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.653 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:50:40.653 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.940964+0000", 2024-09-10T18:50:40.653 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:50:40.653 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.653 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.653 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:50:40.653 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.940964+0000", 2024-09-10T18:50:40.653 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:50:40.653 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.653 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.653 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:50:40.654 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.940963+0000", 2024-09-10T18:50:40.654 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T18:50:40.654 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.654 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.654 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:50:40.654 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.940975+0000", 2024-09-10T18:50:40.654 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.2242000000000001e-05 2024-09-10T18:50:40.654 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.654 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.654 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:50:40.654 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.940987+0000", 2024-09-10T18:50:40.654 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.2261e-05 2024-09-10T18:50:40.654 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.654 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.655 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:50:40.655 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.940993+0000", 2024-09-10T18:50:40.655 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.0059999999999998e-06 2024-09-10T18:50:40.655 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.655 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.655 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:50:40.655 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.941143+0000", 2024-09-10T18:50:40.655 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00015040299999999999 2024-09-10T18:50:40.655 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.655 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.655 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:50:40.655 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.941304+0000", 2024-09-10T18:50:40.655 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00016038399999999999 2024-09-10T18:50:40.656 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.656 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.656 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,7", 2024-09-10T18:50:40.656 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.941412+0000", 2024-09-10T18:50:40.656 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000108607 2024-09-10T18:50:40.656 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:50:40.656 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:50:40.656 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:50:40.656 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.656 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.656 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.14790.0:238372 7.6 7.d9b5a0f6 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e137)", 2024-09-10T18:50:40.656 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:50:39.941565+0000", 2024-09-10T18:50:40.656 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 0.00043161700000000002, 2024-09-10T18:50:40.656 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00078094600000000003, 2024-09-10T18:50:40.657 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:50:40.657 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "queued for pg", 2024-09-10T18:50:40.657 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:50:40.657 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.14790", 2024-09-10T18:50:40.657 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:50:40.657 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 238372 2024-09-10T18:50:40.657 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.657 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:50:40.657 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.658 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:50:40.658 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.941565+0000", 2024-09-10T18:50:40.658 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:50:40.658 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.658 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.658 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:50:40.658 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.941565+0000", 2024-09-10T18:50:40.658 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:50:40.658 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.658 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.658 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:50:40.658 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.941564+0000", 2024-09-10T18:50:40.658 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:50:40.658 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.659 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.659 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:50:40.659 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.941569+0000", 2024-09-10T18:50:40.659 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.186e-06 2024-09-10T18:50:40.659 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.659 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.659 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:50:40.659 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.941582+0000", 2024-09-10T18:50:40.659 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.258e-05 2024-09-10T18:50:40.659 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.659 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.659 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:50:40.659 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.941587+0000", 2024-09-10T18:50:40.659 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.6810000000000003e-06 2024-09-10T18:50:40.660 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:50:40.660 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:50:40.660 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:50:40.660 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.660 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.660 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.14790.0:238367 7.e e137/88)", 2024-09-10T18:50:40.660 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:50:39.942013+0000", 2024-09-10T18:50:40.660 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 4294967295.9999833, 2024-09-10T18:50:40.660 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00037993599999999999, 2024-09-10T18:50:40.660 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:50:40.660 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "queued for pg", 2024-09-10T18:50:40.660 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:50:40.660 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.661 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:50:40.661 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.942013+0000", 2024-09-10T18:50:40.661 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:50:40.661 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.661 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.661 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:50:40.661 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.942013+0000", 2024-09-10T18:50:40.661 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:50:40.661 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.661 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.661 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:50:40.661 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.942012+0000", 2024-09-10T18:50:40.662 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T18:50:40.662 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.662 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.662 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:50:40.662 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.942019+0000", 2024-09-10T18:50:40.662 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.4980000000000003e-06 2024-09-10T18:50:40.662 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.662 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.662 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:50:40.662 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.942028+0000", 2024-09-10T18:50:40.662 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.5380000000000008e-06 2024-09-10T18:50:40.662 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:50:40.662 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:50:40.663 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:50:40.663 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:39.942033+0000", 2024-09-10T18:50:40.663 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.1279999999999999e-06 2024-09-10T18:50:40.663 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:50:40.663 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:50:40.663 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:50:40.663 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:50:40.663 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-09-10T18:50:40.663 INFO:teuthology.orchestra.run.smithi018.stdout: "num_ops": 7 2024-09-10T18:50:40.663 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T18:50:40.663 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:40.707 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:40.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:40.709 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:40.809 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:40.840 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:40.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:40.842 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:40.942 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:40.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:40.981 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:40.981 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:41.082 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:41.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:41.114 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:41.114 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:41.214 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:41.243 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:41.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:41.244 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:41.345 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:41.375 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:41.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:41.376 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:41.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:41 smithi018 ceph-mon[29336]: 7.3 scrub starts 2024-09-10T18:50:41.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:41 smithi018 ceph-mon[29336]: 1.a scrub starts 2024-09-10T18:50:41.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:41 smithi018 ceph-mon[29336]: 1.a scrub ok 2024-09-10T18:50:41.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:41 smithi018 ceph-mon[40196]: 7.3 scrub starts 2024-09-10T18:50:41.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:41 smithi018 ceph-mon[40196]: 1.a scrub starts 2024-09-10T18:50:41.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:41 smithi018 ceph-mon[40196]: 1.a scrub ok 2024-09-10T18:50:41.445 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:50:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:50:40] "GET /metrics HTTP/1.1" 200 223803 "" "Prometheus/2.43.0" 2024-09-10T18:50:41.476 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:41.521 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:41.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:41.522 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:41.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:41 smithi103 ceph-mon[33163]: 7.3 scrub starts 2024-09-10T18:50:41.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:41 smithi103 ceph-mon[33163]: 1.a scrub starts 2024-09-10T18:50:41.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:41 smithi103 ceph-mon[33163]: 1.a scrub ok 2024-09-10T18:50:41.623 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:41.652 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:41.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:41.653 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:41.754 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:41.781 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:41.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:41.782 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:41.883 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:41.916 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:41.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:41.917 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:42.018 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:42.054 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:42.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:42.055 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:42.156 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:42.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:42.186 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:42.186 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:42.286 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:42.316 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:42.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:42.317 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:42.418 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:42.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:42.445 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:42.445 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:42.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:42 smithi018 ceph-mon[40196]: 7.3 scrub ok 2024-09-10T18:50:42.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:42 smithi018 ceph-mon[40196]: 5.10 scrub starts 2024-09-10T18:50:42.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:42 smithi018 ceph-mon[40196]: 5.10 scrub ok 2024-09-10T18:50:42.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:42 smithi018 ceph-mon[40196]: pgmap v213: 158 pgs: 158 active+clean; 2.2 GiB data, 8.8 GiB used, 617 GiB / 626 GiB avail; 0 B/s wr, 1.19k op/s 2024-09-10T18:50:42.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:42 smithi018 ceph-mon[40196]: 5.1c scrub starts 2024-09-10T18:50:42.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:42 smithi018 ceph-mon[40196]: 5.1c scrub ok 2024-09-10T18:50:42.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:42 smithi018 ceph-mon[29336]: 7.3 scrub ok 2024-09-10T18:50:42.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:42 smithi018 ceph-mon[29336]: 5.10 scrub starts 2024-09-10T18:50:42.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:42 smithi018 ceph-mon[29336]: 5.10 scrub ok 2024-09-10T18:50:42.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:42 smithi018 ceph-mon[29336]: pgmap v213: 158 pgs: 158 active+clean; 2.2 GiB data, 8.8 GiB used, 617 GiB / 626 GiB avail; 0 B/s wr, 1.19k op/s 2024-09-10T18:50:42.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:42 smithi018 ceph-mon[29336]: 5.1c scrub starts 2024-09-10T18:50:42.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:42 smithi018 ceph-mon[29336]: 5.1c scrub ok 2024-09-10T18:50:42.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:42 smithi103 ceph-mon[33163]: 7.3 scrub ok 2024-09-10T18:50:42.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:42 smithi103 ceph-mon[33163]: 5.10 scrub starts 2024-09-10T18:50:42.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:42 smithi103 ceph-mon[33163]: 5.10 scrub ok 2024-09-10T18:50:42.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:42 smithi103 ceph-mon[33163]: pgmap v213: 158 pgs: 158 active+clean; 2.2 GiB data, 8.8 GiB used, 617 GiB / 626 GiB avail; 0 B/s wr, 1.19k op/s 2024-09-10T18:50:42.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:42 smithi103 ceph-mon[33163]: 5.1c scrub starts 2024-09-10T18:50:42.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:42 smithi103 ceph-mon[33163]: 5.1c scrub ok 2024-09-10T18:50:42.546 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:42.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:42.582 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:42.582 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:42.683 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:42.718 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:42.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:42.719 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:42.820 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:42.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:42.862 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:42.862 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:42.963 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:43.004 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:43.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:43.005 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:43.106 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:43.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:43.146 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:43.146 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:43.237 INFO:teuthology.orchestra.run.smithi018.stderr:reweighted osd.4 to 1 (10000) 2024-09-10T18:50:43.248 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:43.279 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:43.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:43.280 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:43.380 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:43.419 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:43.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:43.420 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:43.521 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:43.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:43 smithi103 ceph-mon[33163]: 4.e scrub starts 2024-09-10T18:50:43.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:43 smithi103 ceph-mon[33163]: 4.e scrub ok 2024-09-10T18:50:43.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:43 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3740464904' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-10T18:50:43.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:43.556 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:43.556 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:43.565 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:43 smithi018 ceph-mon[40196]: 4.e scrub starts 2024-09-10T18:50:43.566 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:43 smithi018 ceph-mon[40196]: 4.e scrub ok 2024-09-10T18:50:43.566 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:43 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3740464904' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-10T18:50:43.566 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:43 smithi018 ceph-mon[29336]: 4.e scrub starts 2024-09-10T18:50:43.566 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:43 smithi018 ceph-mon[29336]: 4.e scrub ok 2024-09-10T18:50:43.566 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:43 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3740464904' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-10T18:50:43.607 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-10T18:50:43.657 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:43.683 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:43.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:43.685 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:43.786 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:43.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:43.860 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:43.861 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:43.962 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:44.009 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:44.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:44.010 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:44.111 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:44.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:44.152 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:44.152 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:44.253 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:44.280 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:44.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:44.281 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:44.328 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:50:44.328 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:50:44.369 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:50:44.369 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:50:44.382 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:44.411 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:50:44.411 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:50:44.413 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:44 smithi018 ceph-mon[40196]: 1.1 scrub starts 2024-09-10T18:50:44.413 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:44 smithi018 ceph-mon[40196]: pgmap v214: 158 pgs: 158 active+clean; 2.0 GiB data, 8.1 GiB used, 618 GiB / 626 GiB avail; 0 B/s wr, 1.30k op/s 2024-09-10T18:50:44.413 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:44 smithi018 ceph-mon[40196]: 1.1 scrub ok 2024-09-10T18:50:44.413 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:44 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3740464904' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-10T18:50:44.413 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:44 smithi018 ceph-mon[40196]: osdmap e138: 8 total, 8 up, 7 in 2024-09-10T18:50:44.413 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:44 smithi018 ceph-mon[40196]: 2.17 scrub starts 2024-09-10T18:50:44.413 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:44 smithi018 ceph-mon[40196]: 2.17 scrub ok 2024-09-10T18:50:44.413 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:44 smithi018 ceph-mon[40196]: 6.15 scrub starts 2024-09-10T18:50:44.413 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:44 smithi018 ceph-mon[40196]: 6.15 scrub ok 2024-09-10T18:50:44.413 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:44 smithi018 ceph-mon[40196]: 5.1 scrub starts 2024-09-10T18:50:44.413 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:44 smithi018 ceph-mon[40196]: 5.1 scrub ok 2024-09-10T18:50:44.414 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:44 smithi018 ceph-mon[29336]: 1.1 scrub starts 2024-09-10T18:50:44.414 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:44 smithi018 ceph-mon[29336]: pgmap v214: 158 pgs: 158 active+clean; 2.0 GiB data, 8.1 GiB used, 618 GiB / 626 GiB avail; 0 B/s wr, 1.30k op/s 2024-09-10T18:50:44.414 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:44 smithi018 ceph-mon[29336]: 1.1 scrub ok 2024-09-10T18:50:44.414 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:44 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3740464904' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-10T18:50:44.414 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:44 smithi018 ceph-mon[29336]: osdmap e138: 8 total, 8 up, 7 in 2024-09-10T18:50:44.414 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:44 smithi018 ceph-mon[29336]: 2.17 scrub starts 2024-09-10T18:50:44.414 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:44 smithi018 ceph-mon[29336]: 2.17 scrub ok 2024-09-10T18:50:44.414 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:44 smithi018 ceph-mon[29336]: 6.15 scrub starts 2024-09-10T18:50:44.414 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:44 smithi018 ceph-mon[29336]: 6.15 scrub ok 2024-09-10T18:50:44.414 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:44 smithi018 ceph-mon[29336]: 5.1 scrub starts 2024-09-10T18:50:44.414 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:44 smithi018 ceph-mon[29336]: 5.1 scrub ok 2024-09-10T18:50:44.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:44.435 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:44.435 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:44.445 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:50:44.445 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:50:44.478 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:50:44.479 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:50:44.511 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:50:44.511 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:50:44.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:44 smithi103 ceph-mon[33163]: 1.1 scrub starts 2024-09-10T18:50:44.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:44 smithi103 ceph-mon[33163]: pgmap v214: 158 pgs: 158 active+clean; 2.0 GiB data, 8.1 GiB used, 618 GiB / 626 GiB avail; 0 B/s wr, 1.30k op/s 2024-09-10T18:50:44.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:44 smithi103 ceph-mon[33163]: 1.1 scrub ok 2024-09-10T18:50:44.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:44 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3740464904' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-10T18:50:44.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:44 smithi103 ceph-mon[33163]: osdmap e138: 8 total, 8 up, 7 in 2024-09-10T18:50:44.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:44 smithi103 ceph-mon[33163]: 2.17 scrub starts 2024-09-10T18:50:44.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:44 smithi103 ceph-mon[33163]: 2.17 scrub ok 2024-09-10T18:50:44.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:44 smithi103 ceph-mon[33163]: 6.15 scrub starts 2024-09-10T18:50:44.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:44 smithi103 ceph-mon[33163]: 6.15 scrub ok 2024-09-10T18:50:44.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:44 smithi103 ceph-mon[33163]: 5.1 scrub starts 2024-09-10T18:50:44.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:44 smithi103 ceph-mon[33163]: 5.1 scrub ok 2024-09-10T18:50:44.536 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:44.546 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:50:44.546 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:50:44.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:44.569 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:44.569 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:44.577 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:50:44.577 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:50:44.669 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:44.708 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:44.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:44.709 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:44.810 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:44.835 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:44.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:44.836 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:44.858 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd reweight 5 1 2024-09-10T18:50:44.936 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:44.969 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-10T18:50:44.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:44.979 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:44.979 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:45.080 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:45.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:45.108 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:45.108 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:45.209 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:45.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:45.262 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:45.263 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:45.363 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:45.409 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:45.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:45.410 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:45.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:45 smithi018 ceph-mon[29336]: 1.8 scrub starts 2024-09-10T18:50:45.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:45 smithi018 ceph-mon[29336]: 1.8 scrub ok 2024-09-10T18:50:45.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:45 smithi018 ceph-mon[29336]: 4.0 scrub starts 2024-09-10T18:50:45.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:45 smithi018 ceph-mon[29336]: 4.0 scrub ok 2024-09-10T18:50:45.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:45 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/302270107' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:50:45.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:45 smithi018 ceph-mon[40196]: 1.8 scrub starts 2024-09-10T18:50:45.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:45 smithi018 ceph-mon[40196]: 1.8 scrub ok 2024-09-10T18:50:45.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:45 smithi018 ceph-mon[40196]: 4.0 scrub starts 2024-09-10T18:50:45.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:45 smithi018 ceph-mon[40196]: 4.0 scrub ok 2024-09-10T18:50:45.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:45 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/302270107' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:50:45.511 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:45.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:45 smithi103 ceph-mon[33163]: 1.8 scrub starts 2024-09-10T18:50:45.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:45 smithi103 ceph-mon[33163]: 1.8 scrub ok 2024-09-10T18:50:45.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:45 smithi103 ceph-mon[33163]: 4.0 scrub starts 2024-09-10T18:50:45.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:45 smithi103 ceph-mon[33163]: 4.0 scrub ok 2024-09-10T18:50:45.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:45 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/302270107' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:50:45.538 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:45.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:45.539 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:45.568 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:50:45.639 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:45.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:45.666 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:45.666 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:45.767 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:45.781 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:50:45.793 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:45.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:45.795 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:45.895 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:45.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:45.927 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:45.927 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:46.028 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:46.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:46.060 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:46.060 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:46.161 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:46.201 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:46.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:46.202 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:46.303 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:46.338 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:46.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:46.339 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:46.354 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:46 smithi018 ceph-mon[29336]: 5.12 scrub starts 2024-09-10T18:50:46.354 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:46 smithi018 ceph-mon[29336]: 5.12 scrub ok 2024-09-10T18:50:46.354 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:46 smithi018 ceph-mon[29336]: 2.12 scrub starts 2024-09-10T18:50:46.354 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:46 smithi018 ceph-mon[29336]: pgmap v216: 158 pgs: 158 active+clean; 2.0 GiB data, 8.1 GiB used, 618 GiB / 626 GiB avail; 0 B/s wr, 841 op/s 2024-09-10T18:50:46.354 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:46 smithi018 ceph-mon[29336]: 2.12 scrub ok 2024-09-10T18:50:46.355 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:46 smithi018 ceph-mon[40196]: 5.12 scrub starts 2024-09-10T18:50:46.355 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:46 smithi018 ceph-mon[40196]: 5.12 scrub ok 2024-09-10T18:50:46.355 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:46 smithi018 ceph-mon[40196]: 2.12 scrub starts 2024-09-10T18:50:46.355 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:46 smithi018 ceph-mon[40196]: pgmap v216: 158 pgs: 158 active+clean; 2.0 GiB data, 8.1 GiB used, 618 GiB / 626 GiB avail; 0 B/s wr, 841 op/s 2024-09-10T18:50:46.355 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:46 smithi018 ceph-mon[40196]: 2.12 scrub ok 2024-09-10T18:50:46.440 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:46.470 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:46.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:46.471 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:46.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:46 smithi103 ceph-mon[33163]: 5.12 scrub starts 2024-09-10T18:50:46.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:46 smithi103 ceph-mon[33163]: 5.12 scrub ok 2024-09-10T18:50:46.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:46 smithi103 ceph-mon[33163]: 2.12 scrub starts 2024-09-10T18:50:46.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:46 smithi103 ceph-mon[33163]: pgmap v216: 158 pgs: 158 active+clean; 2.0 GiB data, 8.1 GiB used, 618 GiB / 626 GiB avail; 0 B/s wr, 841 op/s 2024-09-10T18:50:46.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:46 smithi103 ceph-mon[33163]: 2.12 scrub ok 2024-09-10T18:50:46.523 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:50:46 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:50:46.571 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:46.599 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:46.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:46.600 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:46.701 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:46.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:46.753 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:46.754 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:46.854 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:46.898 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:46.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:46.899 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:47.000 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:47.027 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:47.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:47.028 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:47.129 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:47.152 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:47.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:47.153 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:47.254 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:47.295 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:47.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:47.296 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:47.392 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:50:47.397 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:47.426 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:47.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:47.427 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:47.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:47 smithi018 ceph-mon[29336]: 4.10 scrub starts 2024-09-10T18:50:47.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:47 smithi018 ceph-mon[29336]: 4.10 scrub ok 2024-09-10T18:50:47.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:47 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:47.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:47 smithi018 ceph-mon[40196]: 4.10 scrub starts 2024-09-10T18:50:47.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:47 smithi018 ceph-mon[40196]: 4.10 scrub ok 2024-09-10T18:50:47.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:47 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:47.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:47 smithi103 ceph-mon[33163]: 4.10 scrub starts 2024-09-10T18:50:47.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:47 smithi103 ceph-mon[33163]: 4.10 scrub ok 2024-09-10T18:50:47.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:47 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:47.528 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:47.549 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T18:50:47.566 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:47.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:47.567 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:47.668 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:47.694 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:47.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:47.696 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:47.797 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:47.825 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:47.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:47.826 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:47.926 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:47.954 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:47.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:47.955 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:48.056 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:48.083 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:48.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:48.085 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:48.186 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:48.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:48.215 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:48.216 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:48.316 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:48.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:48.346 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:48.346 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:48.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:48 smithi018 ceph-mon[40196]: 2.c scrub starts 2024-09-10T18:50:48.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:48 smithi018 ceph-mon[40196]: 2.c scrub ok 2024-09-10T18:50:48.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:48 smithi018 ceph-mon[40196]: pgmap v217: 158 pgs: 158 active+clean; 1.6 GiB data, 6.9 GiB used, 619 GiB / 626 GiB avail; 0 B/s wr, 1.67k op/s 2024-09-10T18:50:48.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:48 smithi018 ceph-mon[29336]: 2.c scrub starts 2024-09-10T18:50:48.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:48 smithi018 ceph-mon[29336]: 2.c scrub ok 2024-09-10T18:50:48.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:48 smithi018 ceph-mon[29336]: pgmap v217: 158 pgs: 158 active+clean; 1.6 GiB data, 6.9 GiB used, 619 GiB / 626 GiB avail; 0 B/s wr, 1.67k op/s 2024-09-10T18:50:48.447 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:48.480 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:48.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:48.481 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:48.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:48 smithi103 ceph-mon[33163]: 2.c scrub starts 2024-09-10T18:50:48.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:48 smithi103 ceph-mon[33163]: 2.c scrub ok 2024-09-10T18:50:48.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:48 smithi103 ceph-mon[33163]: pgmap v217: 158 pgs: 158 active+clean; 1.6 GiB data, 6.9 GiB used, 619 GiB / 626 GiB avail; 0 B/s wr, 1.67k op/s 2024-09-10T18:50:48.582 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:48.610 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:48.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:48.611 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:48.712 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:48.747 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:48.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:48.748 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:48.849 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:48.880 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:48.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:48.881 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:48.982 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:49.010 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:49.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:49.011 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:49.112 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:49.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:49.163 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:49.164 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:49.264 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:49.314 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:49.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:49.315 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:49.416 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:49.461 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:49.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:49.463 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:49.564 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:49.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:49.605 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:49.605 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:49.692 INFO:teuthology.orchestra.run.smithi018.stdout:ERROR: (22) Invalid argument 2024-09-10T18:50:49.692 INFO:teuthology.orchestra.run.smithi018.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:50:49.706 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:49.733 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:49.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:49.734 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:49.817 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:49 smithi018 ceph-mon[40196]: 1.0 scrub starts 2024-09-10T18:50:49.817 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:49 smithi018 ceph-mon[40196]: 5.e scrub starts 2024-09-10T18:50:49.817 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:49 smithi018 ceph-mon[40196]: 1.0 scrub ok 2024-09-10T18:50:49.817 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:49 smithi018 ceph-mon[40196]: 5.e scrub ok 2024-09-10T18:50:49.817 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:49 smithi018 ceph-mon[40196]: pgmap v218: 158 pgs: 158 active+clean; 1.6 GiB data, 6.9 GiB used, 619 GiB / 626 GiB avail; 0 B/s wr, 1.35k op/s 2024-09-10T18:50:49.818 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:49 smithi018 ceph-mon[29336]: 1.0 scrub starts 2024-09-10T18:50:49.818 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:49 smithi018 ceph-mon[29336]: 5.e scrub starts 2024-09-10T18:50:49.818 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:49 smithi018 ceph-mon[29336]: 1.0 scrub ok 2024-09-10T18:50:49.818 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:49 smithi018 ceph-mon[29336]: 5.e scrub ok 2024-09-10T18:50:49.818 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:49 smithi018 ceph-mon[29336]: pgmap v218: 158 pgs: 158 active+clean; 1.6 GiB data, 6.9 GiB used, 619 GiB / 626 GiB avail; 0 B/s wr, 1.35k op/s 2024-09-10T18:50:49.835 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:49.879 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:49.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:49.880 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:49.980 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:50.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:49 smithi103 ceph-mon[33163]: 1.0 scrub starts 2024-09-10T18:50:50.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:49 smithi103 ceph-mon[33163]: 5.e scrub starts 2024-09-10T18:50:50.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:49 smithi103 ceph-mon[33163]: 1.0 scrub ok 2024-09-10T18:50:50.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:49 smithi103 ceph-mon[33163]: 5.e scrub ok 2024-09-10T18:50:50.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:49 smithi103 ceph-mon[33163]: pgmap v218: 158 pgs: 158 active+clean; 1.6 GiB data, 6.9 GiB used, 619 GiB / 626 GiB avail; 0 B/s wr, 1.35k op/s 2024-09-10T18:50:50.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:50.023 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:50.024 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:50.124 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:50.165 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:50.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:50.167 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:50.267 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:50.303 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:50.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:50.304 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:50.404 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:50.430 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:50.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:50.431 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:50.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:50:50 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:50:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:50:50.533 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:50.559 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:50.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:50.560 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:50.661 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:50.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:50.692 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:50.692 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:50.793 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:50.822 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:50.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:50.823 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:50.924 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:50.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:50 smithi018 ceph-mon[40196]: 4.6 scrub starts 2024-09-10T18:50:50.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:50 smithi018 ceph-mon[40196]: 4.6 scrub ok 2024-09-10T18:50:50.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:50 smithi018 ceph-mon[29336]: 4.6 scrub starts 2024-09-10T18:50:50.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:50 smithi018 ceph-mon[29336]: 4.6 scrub ok 2024-09-10T18:50:50.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:50.956 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:50.956 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:51.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:50 smithi103 ceph-mon[33163]: 4.6 scrub starts 2024-09-10T18:50:51.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:50 smithi103 ceph-mon[33163]: 4.6 scrub ok 2024-09-10T18:50:51.057 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:51.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:51.087 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:51.087 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:51.188 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:51.226 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:51.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:51.228 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:51.328 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:51.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:51.365 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:51.365 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:51.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:50:50 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:50:50] "GET /metrics HTTP/1.1" 200 223803 "" "Prometheus/2.43.0" 2024-09-10T18:50:51.466 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:51.492 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:51.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:51.493 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:51.563 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is unset 2024-09-10T18:50:51.594 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:51.627 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:51.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:51.628 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:51.729 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:51.756 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:51.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:51.757 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:51.858 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:51.887 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:51.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:51.889 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:51.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:51 smithi018 ceph-mon[40196]: 1.2 scrub starts 2024-09-10T18:50:51.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:51 smithi018 ceph-mon[40196]: 1.2 scrub ok 2024-09-10T18:50:51.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:51 smithi018 ceph-mon[40196]: 5.18 scrub starts 2024-09-10T18:50:51.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:51 smithi018 ceph-mon[40196]: 5.18 scrub ok 2024-09-10T18:50:51.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:51 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1364214975' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:50:51.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:51 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:50:51.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:51 smithi018 ceph-mon[40196]: pgmap v219: 158 pgs: 158 active+clean; 1.4 GiB data, 6.3 GiB used, 619 GiB / 626 GiB avail; 0 B/s wr, 1.34k op/s 2024-09-10T18:50:51.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:51 smithi018 ceph-mon[29336]: 1.2 scrub starts 2024-09-10T18:50:51.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:51 smithi018 ceph-mon[29336]: 1.2 scrub ok 2024-09-10T18:50:51.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:51 smithi018 ceph-mon[29336]: 5.18 scrub starts 2024-09-10T18:50:51.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:51 smithi018 ceph-mon[29336]: 5.18 scrub ok 2024-09-10T18:50:51.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:51 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1364214975' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:50:51.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:51 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:50:51.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:51 smithi018 ceph-mon[29336]: pgmap v219: 158 pgs: 158 active+clean; 1.4 GiB data, 6.3 GiB used, 619 GiB / 626 GiB avail; 0 B/s wr, 1.34k op/s 2024-09-10T18:50:51.990 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:52.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:51 smithi103 ceph-mon[33163]: 1.2 scrub starts 2024-09-10T18:50:52.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:51 smithi103 ceph-mon[33163]: 1.2 scrub ok 2024-09-10T18:50:52.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:51 smithi103 ceph-mon[33163]: 5.18 scrub starts 2024-09-10T18:50:52.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:51 smithi103 ceph-mon[33163]: 5.18 scrub ok 2024-09-10T18:50:52.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:51 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1364214975' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:50:52.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:51 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:50:52.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:51 smithi103 ceph-mon[33163]: pgmap v219: 158 pgs: 158 active+clean; 1.4 GiB data, 6.3 GiB used, 619 GiB / 626 GiB avail; 0 B/s wr, 1.34k op/s 2024-09-10T18:50:52.025 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:52.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:52.026 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:52.127 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:52.153 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:52.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:52.154 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:52.255 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:52.299 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:52.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:52.300 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:52.401 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:52.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:52.431 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:52.431 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:52.532 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:52.573 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:52.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:52.574 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:52.675 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:52.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:52 smithi018 ceph-mon[40196]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:50:52.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:52 smithi018 ceph-mon[40196]: Cluster is now healthy 2024-09-10T18:50:52.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:52 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:50:52.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:52 smithi018 ceph-mon[40196]: osdmap e139: 8 total, 8 up, 7 in 2024-09-10T18:50:52.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:52 smithi018 ceph-mon[29336]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:50:52.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:52 smithi018 ceph-mon[29336]: Cluster is now healthy 2024-09-10T18:50:52.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:52 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:50:52.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:52 smithi018 ceph-mon[29336]: osdmap e139: 8 total, 8 up, 7 in 2024-09-10T18:50:52.704 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:52.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:52.706 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:52.806 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:52.848 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:52.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:52.849 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:52.949 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:52.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:52.981 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:52.981 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:53.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:52 smithi103 ceph-mon[33163]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:50:53.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:52 smithi103 ceph-mon[33163]: Cluster is now healthy 2024-09-10T18:50:53.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:52 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:50:53.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:52 smithi103 ceph-mon[33163]: osdmap e139: 8 total, 8 up, 7 in 2024-09-10T18:50:53.082 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:53.112 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:53.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:53.113 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:53.214 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:53.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:53.250 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:53.251 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:53.352 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:53.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:53.395 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:53.395 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:53.496 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:53.531 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:53.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:53.532 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:53.633 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:53.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:53.675 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:53.675 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:53.776 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:53.805 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:53.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:53.806 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:53.811 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:53 smithi018 ceph-mon[40196]: 5.15 scrub starts 2024-09-10T18:50:53.811 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:53 smithi018 ceph-mon[40196]: 5.15 scrub ok 2024-09-10T18:50:53.811 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:53 smithi018 ceph-mon[40196]: 2.11 scrub starts 2024-09-10T18:50:53.811 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:53 smithi018 ceph-mon[40196]: 2.11 scrub ok 2024-09-10T18:50:53.811 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:53 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:50:53.811 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:53 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:50:53.812 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:53 smithi018 ceph-mon[40196]: osdmap e140: 8 total, 8 up, 7 in 2024-09-10T18:50:53.812 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:53 smithi018 ceph-mon[40196]: pgmap v222: 158 pgs: 158 active+clean; 1.2 GiB data, 5.6 GiB used, 620 GiB / 626 GiB avail; 0 B/s wr, 1.68k op/s 2024-09-10T18:50:53.812 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:53 smithi018 ceph-mon[29336]: 5.15 scrub starts 2024-09-10T18:50:53.812 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:53 smithi018 ceph-mon[29336]: 5.15 scrub ok 2024-09-10T18:50:53.812 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:53 smithi018 ceph-mon[29336]: 2.11 scrub starts 2024-09-10T18:50:53.812 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:53 smithi018 ceph-mon[29336]: 2.11 scrub ok 2024-09-10T18:50:53.812 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:53 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:50:53.812 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:53 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:50:53.812 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:53 smithi018 ceph-mon[29336]: osdmap e140: 8 total, 8 up, 7 in 2024-09-10T18:50:53.813 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:53 smithi018 ceph-mon[29336]: pgmap v222: 158 pgs: 158 active+clean; 1.2 GiB data, 5.6 GiB used, 620 GiB / 626 GiB avail; 0 B/s wr, 1.68k op/s 2024-09-10T18:50:53.822 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:50:53.861 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-10T18:50:53.906 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:53.961 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:53.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:53.962 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:54.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:53 smithi103 ceph-mon[33163]: 5.15 scrub starts 2024-09-10T18:50:54.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:53 smithi103 ceph-mon[33163]: 5.15 scrub ok 2024-09-10T18:50:54.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:53 smithi103 ceph-mon[33163]: 2.11 scrub starts 2024-09-10T18:50:54.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:53 smithi103 ceph-mon[33163]: 2.11 scrub ok 2024-09-10T18:50:54.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:53 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:50:54.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:53 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:50:54.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:53 smithi103 ceph-mon[33163]: osdmap e140: 8 total, 8 up, 7 in 2024-09-10T18:50:54.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:53 smithi103 ceph-mon[33163]: pgmap v222: 158 pgs: 158 active+clean; 1.2 GiB data, 5.6 GiB used, 620 GiB / 626 GiB avail; 0 B/s wr, 1.68k op/s 2024-09-10T18:50:54.062 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:54.086 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:54.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:54.087 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:54.188 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:54.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:54.241 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:54.241 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:54.341 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:54.372 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:54.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:54.373 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:54.474 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:54.501 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:54.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:54.502 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:54.595 INFO:teuthology.orchestra.run.smithi018.stderr:reweighted osd.5 to 1 (10000) 2024-09-10T18:50:54.602 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:54.650 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:54.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:54.651 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:54.752 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:54.796 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:54.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:54.798 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:54.898 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:54.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:54.942 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:54.942 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:54.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:54 smithi018 ceph-mon[40196]: 2.1f scrub starts 2024-09-10T18:50:54.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:54 smithi018 ceph-mon[40196]: 2.1f scrub ok 2024-09-10T18:50:54.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:54 smithi018 ceph-mon[40196]: 5.7 scrub starts 2024-09-10T18:50:54.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:54 smithi018 ceph-mon[40196]: 5.7 scrub ok 2024-09-10T18:50:54.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:54 smithi018 ceph-mon[40196]: 5.0 scrub starts 2024-09-10T18:50:54.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:54 smithi018 ceph-mon[40196]: 5.0 scrub ok 2024-09-10T18:50:54.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:54 smithi018 ceph-mon[40196]: 1.9 scrub starts 2024-09-10T18:50:54.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:54 smithi018 ceph-mon[40196]: 1.9 scrub ok 2024-09-10T18:50:54.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:54 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3911753795' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-10T18:50:54.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:54 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-10T18:50:54.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:54 smithi018 ceph-mon[29336]: 2.1f scrub starts 2024-09-10T18:50:54.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:54 smithi018 ceph-mon[29336]: 2.1f scrub ok 2024-09-10T18:50:54.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:54 smithi018 ceph-mon[29336]: 5.7 scrub starts 2024-09-10T18:50:54.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:54 smithi018 ceph-mon[29336]: 5.7 scrub ok 2024-09-10T18:50:54.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:54 smithi018 ceph-mon[29336]: 5.0 scrub starts 2024-09-10T18:50:54.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:54 smithi018 ceph-mon[29336]: 5.0 scrub ok 2024-09-10T18:50:54.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:54 smithi018 ceph-mon[29336]: 1.9 scrub starts 2024-09-10T18:50:54.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:54 smithi018 ceph-mon[29336]: 1.9 scrub ok 2024-09-10T18:50:54.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:54 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3911753795' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-10T18:50:54.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:54 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-10T18:50:55.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:54 smithi103 ceph-mon[33163]: 2.1f scrub starts 2024-09-10T18:50:55.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:54 smithi103 ceph-mon[33163]: 2.1f scrub ok 2024-09-10T18:50:55.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:54 smithi103 ceph-mon[33163]: 5.7 scrub starts 2024-09-10T18:50:55.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:54 smithi103 ceph-mon[33163]: 5.7 scrub ok 2024-09-10T18:50:55.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:54 smithi103 ceph-mon[33163]: 5.0 scrub starts 2024-09-10T18:50:55.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:54 smithi103 ceph-mon[33163]: 5.0 scrub ok 2024-09-10T18:50:55.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:54 smithi103 ceph-mon[33163]: 1.9 scrub starts 2024-09-10T18:50:55.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:54 smithi103 ceph-mon[33163]: 1.9 scrub ok 2024-09-10T18:50:55.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:54 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3911753795' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-10T18:50:55.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:54 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-10T18:50:55.043 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:55.070 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:55.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:55.071 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:55.172 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:55.206 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:55.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:55.207 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:55.308 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:55.337 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:55.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:55.338 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:55.438 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:55.464 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:55.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:55.465 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:55.491 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-10T18:50:55.566 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:55.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:55.623 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:55.624 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:55.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:55 smithi018 ceph-mon[29336]: 6.4 scrub starts 2024-09-10T18:50:55.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:55 smithi018 ceph-mon[29336]: 6.4 scrub ok 2024-09-10T18:50:55.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:55 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-10T18:50:55.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:55 smithi018 ceph-mon[29336]: osdmap e141: 8 total, 8 up, 7 in 2024-09-10T18:50:55.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:55 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:50:55.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:50:55.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:55 smithi018 ceph-mon[29336]: pgmap v224: 158 pgs: 158 active+clean; 1.2 GiB data, 5.6 GiB used, 620 GiB / 626 GiB avail; 0 B/s wr, 1.13k op/s 2024-09-10T18:50:55.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:55 smithi018 ceph-mon[40196]: 6.4 scrub starts 2024-09-10T18:50:55.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:55 smithi018 ceph-mon[40196]: 6.4 scrub ok 2024-09-10T18:50:55.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:55 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-10T18:50:55.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:55 smithi018 ceph-mon[40196]: osdmap e141: 8 total, 8 up, 7 in 2024-09-10T18:50:55.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:55 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:50:55.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:50:55.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:55 smithi018 ceph-mon[40196]: pgmap v224: 158 pgs: 158 active+clean; 1.2 GiB data, 5.6 GiB used, 620 GiB / 626 GiB avail; 0 B/s wr, 1.13k op/s 2024-09-10T18:50:55.725 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:55.773 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:55.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:55.774 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:55.875 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:55.903 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:55.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:55.904 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:56.005 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:56.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:55 smithi103 ceph-mon[33163]: 6.4 scrub starts 2024-09-10T18:50:56.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:55 smithi103 ceph-mon[33163]: 6.4 scrub ok 2024-09-10T18:50:56.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:55 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-10T18:50:56.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:55 smithi103 ceph-mon[33163]: osdmap e141: 8 total, 8 up, 7 in 2024-09-10T18:50:56.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:55 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:50:56.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:50:56.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:55 smithi103 ceph-mon[33163]: pgmap v224: 158 pgs: 158 active+clean; 1.2 GiB data, 5.6 GiB used, 620 GiB / 626 GiB avail; 0 B/s wr, 1.13k op/s 2024-09-10T18:50:56.054 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:56.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:56.055 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:56.156 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:56.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:56.195 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:56.195 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:56.296 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:56.330 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:56.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:56.331 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:56.432 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:56.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:56.471 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:56.471 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:56.572 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:50:56.597 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:50:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:50:56.598 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:56.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:56.599 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:50:56.700 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:56.729 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:56.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:56.730 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:56.830 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:56.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:56.861 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:56.861 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:56.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:56 smithi018 ceph-mon[40196]: 5.1e scrub starts 2024-09-10T18:50:56.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:56 smithi018 ceph-mon[40196]: 5.1e scrub ok 2024-09-10T18:50:56.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:56 smithi018 ceph-mon[40196]: 6.1b scrub starts 2024-09-10T18:50:56.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:56 smithi018 ceph-mon[40196]: 6.1b scrub ok 2024-09-10T18:50:56.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:56 smithi018 ceph-mon[40196]: 6.17 scrub starts 2024-09-10T18:50:56.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:56 smithi018 ceph-mon[40196]: 6.17 scrub ok 2024-09-10T18:50:56.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:56 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:56.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:56 smithi018 ceph-mon[29336]: 5.1e scrub starts 2024-09-10T18:50:56.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:56 smithi018 ceph-mon[29336]: 5.1e scrub ok 2024-09-10T18:50:56.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:56 smithi018 ceph-mon[29336]: 6.1b scrub starts 2024-09-10T18:50:56.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:56 smithi018 ceph-mon[29336]: 6.1b scrub ok 2024-09-10T18:50:56.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:56 smithi018 ceph-mon[29336]: 6.17 scrub starts 2024-09-10T18:50:56.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:56 smithi018 ceph-mon[29336]: 6.17 scrub ok 2024-09-10T18:50:56.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:56 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:56.962 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:56.990 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:56.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:56.991 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:57.003 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:50:57.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:56 smithi103 ceph-mon[33163]: 5.1e scrub starts 2024-09-10T18:50:57.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:56 smithi103 ceph-mon[33163]: 5.1e scrub ok 2024-09-10T18:50:57.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:56 smithi103 ceph-mon[33163]: 6.1b scrub starts 2024-09-10T18:50:57.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:56 smithi103 ceph-mon[33163]: 6.1b scrub ok 2024-09-10T18:50:57.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:56 smithi103 ceph-mon[33163]: 6.17 scrub starts 2024-09-10T18:50:57.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:56 smithi103 ceph-mon[33163]: 6.17 scrub ok 2024-09-10T18:50:57.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:56 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:57.045 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd reweight 6 1 2024-09-10T18:50:57.092 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:57.168 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:57.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:57.170 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:57.271 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:57.300 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:57.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:57.301 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:57.402 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:57.424 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:57.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:57.425 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:57.526 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:50:57.570 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:57.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:57.571 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:50:57.671 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:57.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:57.710 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:57.710 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:57.748 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:57 smithi018 ceph-mon[29336]: 6.e scrub starts 2024-09-10T18:50:57.748 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:57 smithi018 ceph-mon[29336]: 6.e scrub ok 2024-09-10T18:50:57.748 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:57 smithi018 ceph-mon[29336]: pgmap v225: 158 pgs: 158 active+clean; 767 MiB data, 4.4 GiB used, 621 GiB / 626 GiB avail; 0 B/s wr, 1.68k op/s 2024-09-10T18:50:57.748 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:57 smithi018 ceph-mon[40196]: 6.e scrub starts 2024-09-10T18:50:57.748 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:57 smithi018 ceph-mon[40196]: 6.e scrub ok 2024-09-10T18:50:57.748 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:57 smithi018 ceph-mon[40196]: pgmap v225: 158 pgs: 158 active+clean; 767 MiB data, 4.4 GiB used, 621 GiB / 626 GiB avail; 0 B/s wr, 1.68k op/s 2024-09-10T18:50:57.811 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:57.836 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:57.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:57.837 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:57.938 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:57.983 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:57.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:57.984 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:58.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:57 smithi103 ceph-mon[33163]: 6.e scrub starts 2024-09-10T18:50:58.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:57 smithi103 ceph-mon[33163]: 6.e scrub ok 2024-09-10T18:50:58.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:57 smithi103 ceph-mon[33163]: pgmap v225: 158 pgs: 158 active+clean; 767 MiB data, 4.4 GiB used, 621 GiB / 626 GiB avail; 0 B/s wr, 1.68k op/s 2024-09-10T18:50:58.085 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:50:58.108 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:58.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:58.109 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:50:58.210 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:58.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:58.253 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:58.254 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:58.263 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:50:58.264 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:50:58.296 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:50:58.297 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:50:58.329 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:50:58.330 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:50:58.354 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:58.364 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:50:58.364 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:50:58.405 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:50:58.405 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:50:58.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:58.415 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:58.415 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:58.436 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:50:58.436 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:50:58.465 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:50:58.465 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:50:58.492 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:50:58.492 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:50:58.515 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:50:58.543 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:58.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:58.544 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:50:58.644 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:58.671 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:58.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:58.672 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:58.773 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:58.816 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:58.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:58.817 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:58.918 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:58.943 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:58.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:58.944 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:58.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:58 smithi018 ceph-mon[40196]: 6.16 scrub starts 2024-09-10T18:50:58.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:58 smithi018 ceph-mon[40196]: 6.16 scrub ok 2024-09-10T18:50:58.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:58 smithi018 ceph-mon[40196]: 4.f deep-scrub starts 2024-09-10T18:50:58.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:58 smithi018 ceph-mon[40196]: 4.f deep-scrub ok 2024-09-10T18:50:58.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:58 smithi018 ceph-mon[40196]: 5.17 scrub starts 2024-09-10T18:50:58.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:58 smithi018 ceph-mon[40196]: 5.17 scrub ok 2024-09-10T18:50:58.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:58 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3825493062' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:50:58.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:58 smithi018 ceph-mon[40196]: osdmap e142: 8 total, 8 up, 7 in 2024-09-10T18:50:58.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:58 smithi018 ceph-mon[29336]: 6.16 scrub starts 2024-09-10T18:50:58.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:58 smithi018 ceph-mon[29336]: 6.16 scrub ok 2024-09-10T18:50:58.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:58 smithi018 ceph-mon[29336]: 4.f deep-scrub starts 2024-09-10T18:50:58.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:58 smithi018 ceph-mon[29336]: 4.f deep-scrub ok 2024-09-10T18:50:58.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:58 smithi018 ceph-mon[29336]: 5.17 scrub starts 2024-09-10T18:50:58.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:58 smithi018 ceph-mon[29336]: 5.17 scrub ok 2024-09-10T18:50:58.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:58 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3825493062' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:50:58.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:58 smithi018 ceph-mon[29336]: osdmap e142: 8 total, 8 up, 7 in 2024-09-10T18:50:59.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:58 smithi103 ceph-mon[33163]: 6.16 scrub starts 2024-09-10T18:50:59.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:58 smithi103 ceph-mon[33163]: 6.16 scrub ok 2024-09-10T18:50:59.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:58 smithi103 ceph-mon[33163]: 4.f deep-scrub starts 2024-09-10T18:50:59.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:58 smithi103 ceph-mon[33163]: 4.f deep-scrub ok 2024-09-10T18:50:59.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:58 smithi103 ceph-mon[33163]: 5.17 scrub starts 2024-09-10T18:50:59.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:58 smithi103 ceph-mon[33163]: 5.17 scrub ok 2024-09-10T18:50:59.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:58 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3825493062' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:50:59.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:58 smithi103 ceph-mon[33163]: osdmap e142: 8 total, 8 up, 7 in 2024-09-10T18:50:59.045 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:59.071 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:59.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:59.072 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:59.173 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:50:59.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:59.213 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:59.213 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:50:59.314 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:50:59.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:59.342 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:50:59.342 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:50:59.442 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:59.482 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:59.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:59.483 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:50:59.584 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:59.626 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:59.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:59.628 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:59.728 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:50:59.771 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:59.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:59.773 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:50:59.874 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:50:59.884 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:59 smithi018 ceph-mon[40196]: 2.1e scrub starts 2024-09-10T18:50:59.884 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:59 smithi018 ceph-mon[40196]: 2.1e scrub ok 2024-09-10T18:50:59.885 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:59 smithi018 ceph-mon[40196]: 4.d scrub starts 2024-09-10T18:50:59.885 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:59 smithi018 ceph-mon[40196]: 4.d scrub ok 2024-09-10T18:50:59.885 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:59 smithi018 ceph-mon[40196]: 2.1c scrub starts 2024-09-10T18:50:59.885 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:59 smithi018 ceph-mon[40196]: 2.1c scrub ok 2024-09-10T18:50:59.885 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:50:59 smithi018 ceph-mon[40196]: pgmap v227: 158 pgs: 158 active+clean; 767 MiB data, 4.4 GiB used, 621 GiB / 626 GiB avail; 0 B/s wr, 1.02k op/s 2024-09-10T18:50:59.885 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:59 smithi018 ceph-mon[29336]: 2.1e scrub starts 2024-09-10T18:50:59.885 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:59 smithi018 ceph-mon[29336]: 2.1e scrub ok 2024-09-10T18:50:59.885 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:59 smithi018 ceph-mon[29336]: 4.d scrub starts 2024-09-10T18:50:59.885 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:59 smithi018 ceph-mon[29336]: 4.d scrub ok 2024-09-10T18:50:59.885 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:59 smithi018 ceph-mon[29336]: 2.1c scrub starts 2024-09-10T18:50:59.885 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:59 smithi018 ceph-mon[29336]: 2.1c scrub ok 2024-09-10T18:50:59.886 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:50:59 smithi018 ceph-mon[29336]: pgmap v227: 158 pgs: 158 active+clean; 767 MiB data, 4.4 GiB used, 621 GiB / 626 GiB avail; 0 B/s wr, 1.02k op/s 2024-09-10T18:50:59.917 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:50:59.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:50:59.918 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:00.019 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:00.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:59 smithi103 ceph-mon[33163]: 2.1e scrub starts 2024-09-10T18:51:00.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:59 smithi103 ceph-mon[33163]: 2.1e scrub ok 2024-09-10T18:51:00.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:59 smithi103 ceph-mon[33163]: 4.d scrub starts 2024-09-10T18:51:00.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:59 smithi103 ceph-mon[33163]: 4.d scrub ok 2024-09-10T18:51:00.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:59 smithi103 ceph-mon[33163]: 2.1c scrub starts 2024-09-10T18:51:00.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:59 smithi103 ceph-mon[33163]: 2.1c scrub ok 2024-09-10T18:51:00.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:50:59 smithi103 ceph-mon[33163]: pgmap v227: 158 pgs: 158 active+clean; 767 MiB data, 4.4 GiB used, 621 GiB / 626 GiB avail; 0 B/s wr, 1.02k op/s 2024-09-10T18:51:00.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:00.054 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:00.054 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:00.155 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:00.183 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:00.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:00.185 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:00.285 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:00.315 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:00.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:00.316 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:00.416 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:00.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:00.445 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:00.445 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:00.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:51:00 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:51:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:51:00.547 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:00.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:00.585 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:00.586 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:00.687 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:00.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:00.735 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:00.735 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:00.836 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:00.871 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:00.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:00.872 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:00.940 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:00 smithi018 ceph-mon[40196]: 1.4 scrub starts 2024-09-10T18:51:00.940 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:00 smithi018 ceph-mon[40196]: 1.4 scrub ok 2024-09-10T18:51:00.940 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:00 smithi018 ceph-mon[29336]: 1.4 scrub starts 2024-09-10T18:51:00.940 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:00 smithi018 ceph-mon[29336]: 1.4 scrub ok 2024-09-10T18:51:00.973 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:01.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:00 smithi103 ceph-mon[33163]: 1.4 scrub starts 2024-09-10T18:51:01.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:00 smithi103 ceph-mon[33163]: 1.4 scrub ok 2024-09-10T18:51:01.024 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:51:01.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:01.031 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:01.032 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:01.132 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:01.160 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:01.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:01.161 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:01.195 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:51:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:51:00] "GET /metrics HTTP/1.1" 200 223787 "" "Prometheus/2.43.0" 2024-09-10T18:51:01.262 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:01.291 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:01.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:01.292 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:01.352 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:51:01.393 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:01.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:01.436 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:01.436 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:01.537 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:01.581 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:01.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:01.582 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:01.683 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:01.712 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:01.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:01.713 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:01.814 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:01.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:01.863 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:01.864 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:01.871 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:01 smithi018 ceph-mon[40196]: 2.15 deep-scrub starts 2024-09-10T18:51:01.871 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:01 smithi018 ceph-mon[40196]: 2.15 deep-scrub ok 2024-09-10T18:51:01.871 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:01 smithi018 ceph-mon[40196]: pgmap v228: 158 pgs: 158 active+clean; 642 MiB data, 4.1 GiB used, 622 GiB / 626 GiB avail; 0 B/s wr, 1.08k op/s 2024-09-10T18:51:01.871 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:01 smithi018 ceph-mon[40196]: 6.0 scrub starts 2024-09-10T18:51:01.872 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:01 smithi018 ceph-mon[29336]: 2.15 deep-scrub starts 2024-09-10T18:51:01.872 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:01 smithi018 ceph-mon[29336]: 2.15 deep-scrub ok 2024-09-10T18:51:01.872 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:01 smithi018 ceph-mon[29336]: pgmap v228: 158 pgs: 158 active+clean; 642 MiB data, 4.1 GiB used, 622 GiB / 626 GiB avail; 0 B/s wr, 1.08k op/s 2024-09-10T18:51:01.872 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:01 smithi018 ceph-mon[29336]: 6.0 scrub starts 2024-09-10T18:51:01.965 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:01.996 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:01.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:01.997 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:02.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:01 smithi103 ceph-mon[33163]: 2.15 deep-scrub starts 2024-09-10T18:51:02.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:01 smithi103 ceph-mon[33163]: 2.15 deep-scrub ok 2024-09-10T18:51:02.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:01 smithi103 ceph-mon[33163]: pgmap v228: 158 pgs: 158 active+clean; 642 MiB data, 4.1 GiB used, 622 GiB / 626 GiB avail; 0 B/s wr, 1.08k op/s 2024-09-10T18:51:02.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:01 smithi103 ceph-mon[33163]: 6.0 scrub starts 2024-09-10T18:51:02.098 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:02.137 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:02.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:02.138 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:02.239 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:02.276 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:02.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:02.277 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:02.359 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T18:51:02.378 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:02.407 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:02.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:02.408 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:02.508 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:02.537 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:02.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:02.539 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:02.639 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:02.667 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:02.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:02.668 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:02.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:02 smithi018 ceph-mon[29336]: 2.19 scrub starts 2024-09-10T18:51:02.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:02 smithi018 ceph-mon[29336]: 2.19 scrub ok 2024-09-10T18:51:02.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:02 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:51:02.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:02 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:51:02.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:02 smithi018 ceph-mon[29336]: 5.5 scrub starts 2024-09-10T18:51:02.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:02 smithi018 ceph-mon[29336]: 5.5 scrub ok 2024-09-10T18:51:02.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:02 smithi018 ceph-mon[29336]: 6.3 scrub starts 2024-09-10T18:51:02.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:02 smithi018 ceph-mon[29336]: 6.3 scrub ok 2024-09-10T18:51:02.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:02 smithi018 ceph-mon[29336]: 1.5 scrub starts 2024-09-10T18:51:02.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:02 smithi018 ceph-mon[29336]: 1.5 scrub ok 2024-09-10T18:51:02.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:02 smithi018 ceph-mon[29336]: 6.0 scrub ok 2024-09-10T18:51:02.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:02 smithi018 ceph-mon[29336]: osdmap e143: 8 total, 8 up, 7 in 2024-09-10T18:51:02.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:02 smithi018 ceph-mon[40196]: 2.19 scrub starts 2024-09-10T18:51:02.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:02 smithi018 ceph-mon[40196]: 2.19 scrub ok 2024-09-10T18:51:02.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:02 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:51:02.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:02 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:51:02.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:02 smithi018 ceph-mon[40196]: 5.5 scrub starts 2024-09-10T18:51:02.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:02 smithi018 ceph-mon[40196]: 5.5 scrub ok 2024-09-10T18:51:02.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:02 smithi018 ceph-mon[40196]: 6.3 scrub starts 2024-09-10T18:51:02.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:02 smithi018 ceph-mon[40196]: 6.3 scrub ok 2024-09-10T18:51:02.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:02 smithi018 ceph-mon[40196]: 1.5 scrub starts 2024-09-10T18:51:02.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:02 smithi018 ceph-mon[40196]: 1.5 scrub ok 2024-09-10T18:51:02.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:02 smithi018 ceph-mon[40196]: 6.0 scrub ok 2024-09-10T18:51:02.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:02 smithi018 ceph-mon[40196]: osdmap e143: 8 total, 8 up, 7 in 2024-09-10T18:51:02.769 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:02.822 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:02.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:02.824 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:02.924 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:02.951 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:02.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:02.952 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:03.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:02 smithi103 ceph-mon[33163]: 2.19 scrub starts 2024-09-10T18:51:03.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:02 smithi103 ceph-mon[33163]: 2.19 scrub ok 2024-09-10T18:51:03.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:02 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:51:03.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:02 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:51:03.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:02 smithi103 ceph-mon[33163]: 5.5 scrub starts 2024-09-10T18:51:03.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:02 smithi103 ceph-mon[33163]: 5.5 scrub ok 2024-09-10T18:51:03.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:02 smithi103 ceph-mon[33163]: 6.3 scrub starts 2024-09-10T18:51:03.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:02 smithi103 ceph-mon[33163]: 6.3 scrub ok 2024-09-10T18:51:03.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:02 smithi103 ceph-mon[33163]: 1.5 scrub starts 2024-09-10T18:51:03.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:02 smithi103 ceph-mon[33163]: 1.5 scrub ok 2024-09-10T18:51:03.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:02 smithi103 ceph-mon[33163]: 6.0 scrub ok 2024-09-10T18:51:03.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:02 smithi103 ceph-mon[33163]: osdmap e143: 8 total, 8 up, 7 in 2024-09-10T18:51:03.053 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:03.080 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:03.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:03.082 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:03.182 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:03.221 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:03.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:03.222 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:03.323 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:03.361 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:03.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:03.362 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:03.463 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:03.516 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:03.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:03.517 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:03.618 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:03.646 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:03.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:03.647 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:03.720 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T18:51:03.720 INFO:teuthology.orchestra.run.smithi018.stdout: "size": 20, 2024-09-10T18:51:03.720 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 600, 2024-09-10T18:51:03.720 INFO:teuthology.orchestra.run.smithi018.stdout: "ops": [ 2024-09-10T18:51:03.720 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.720 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.14790.0:15084 7.f 7:f2b10b86:::benchmark_data_smithi018_78628_object15083:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-10T18:51:03.720 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:47:49.058358+0000", 2024-09-10T18:51:03.720 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 194.65713962699999, 2024-09-10T18:51:03.721 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.14719653699999999, 2024-09-10T18:51:03.721 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:51:03.721 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:51:03.721 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:51:03.721 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.14790", 2024-09-10T18:51:03.721 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:51:03.721 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 15084 2024-09-10T18:51:03.721 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.721 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:51:03.721 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.721 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:51:03.721 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.058358+0000", 2024-09-10T18:51:03.722 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.722 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.722 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.722 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:51:03.722 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.058358+0000", 2024-09-10T18:51:03.722 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.722 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.722 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.722 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:51:03.722 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.058357+0000", 2024-09-10T18:51:03.722 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999981 2024-09-10T18:51:03.722 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.722 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.723 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:51:03.723 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.058378+0000", 2024-09-10T18:51:03.723 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.0971999999999998e-05 2024-09-10T18:51:03.723 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.723 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.723 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:51:03.723 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.058390+0000", 2024-09-10T18:51:03.723 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.273e-05 2024-09-10T18:51:03.723 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.723 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.723 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:51:03.723 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.058396+0000", 2024-09-10T18:51:03.723 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.964e-06 2024-09-10T18:51:03.723 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.724 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.724 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:51:03.724 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.064785+0000", 2024-09-10T18:51:03.724 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.006388861 2024-09-10T18:51:03.724 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.724 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.724 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:51:03.724 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.064914+0000", 2024-09-10T18:51:03.724 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000128634 2024-09-10T18:51:03.724 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.724 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.724 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,7", 2024-09-10T18:51:03.725 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.065060+0000", 2024-09-10T18:51:03.725 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000145673 2024-09-10T18:51:03.725 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.725 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.725 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:51:03.725 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.067738+0000", 2024-09-10T18:51:03.725 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0026781830000000002 2024-09-10T18:51:03.725 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.725 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.725 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:51:03.725 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.068105+0000", 2024-09-10T18:51:03.725 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00036769599999999998 2024-09-10T18:51:03.726 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.726 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.726 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:51:03.726 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.205229+0000", 2024-09-10T18:51:03.726 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.137123209 2024-09-10T18:51:03.726 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.726 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.726 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:51:03.726 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.205511+0000", 2024-09-10T18:51:03.726 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00028202300000000001 2024-09-10T18:51:03.726 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.726 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.727 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:51:03.727 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.205555+0000", 2024-09-10T18:51:03.727 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.4271999999999999e-05 2024-09-10T18:51:03.727 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.727 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:51:03.727 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.727 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.727 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.727 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.14790.0:15089 7.f 7:f2492c59:::benchmark_data_smithi018_78628_object15088:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-10T18:51:03.727 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:47:49.059410+0000", 2024-09-10T18:51:03.727 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 194.656088068, 2024-09-10T18:51:03.728 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.14654346400000001, 2024-09-10T18:51:03.728 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:51:03.728 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:51:03.728 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:51:03.728 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.14790", 2024-09-10T18:51:03.728 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:51:03.728 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 15089 2024-09-10T18:51:03.728 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.729 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:51:03.729 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.729 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:51:03.729 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.059410+0000", 2024-09-10T18:51:03.729 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.729 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.729 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.729 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:51:03.729 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.059410+0000", 2024-09-10T18:51:03.729 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.729 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.729 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.729 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:51:03.730 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.059409+0000", 2024-09-10T18:51:03.730 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T18:51:03.730 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.730 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.730 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:51:03.730 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.059433+0000", 2024-09-10T18:51:03.730 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.3879e-05 2024-09-10T18:51:03.730 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.730 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.730 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:51:03.730 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.059454+0000", 2024-09-10T18:51:03.730 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.1019999999999999e-05 2024-09-10T18:51:03.730 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.730 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.731 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:51:03.731 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.059460+0000", 2024-09-10T18:51:03.731 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.6729999999999997e-06 2024-09-10T18:51:03.731 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.731 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.731 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:51:03.731 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.065892+0000", 2024-09-10T18:51:03.731 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0064317209999999996 2024-09-10T18:51:03.731 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.731 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.731 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:51:03.731 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.066021+0000", 2024-09-10T18:51:03.731 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00012932199999999999 2024-09-10T18:51:03.731 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.732 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.732 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,7", 2024-09-10T18:51:03.732 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.066437+0000", 2024-09-10T18:51:03.732 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00041599699999999998 2024-09-10T18:51:03.732 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.732 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.732 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:51:03.732 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.068144+0000", 2024-09-10T18:51:03.732 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00170663 2024-09-10T18:51:03.732 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.732 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.732 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:51:03.732 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.204771+0000", 2024-09-10T18:51:03.733 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.1366271 2024-09-10T18:51:03.733 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.733 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.733 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:51:03.733 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.205597+0000", 2024-09-10T18:51:03.733 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00082606499999999996 2024-09-10T18:51:03.733 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.733 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.733 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:51:03.733 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.205912+0000", 2024-09-10T18:51:03.733 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00031569600000000002 2024-09-10T18:51:03.733 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.733 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.734 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:51:03.734 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.205953+0000", 2024-09-10T18:51:03.734 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.0991000000000001e-05 2024-09-10T18:51:03.734 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.734 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:51:03.734 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.734 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.734 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.734 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.14790.0:15102 7.f 7:f53ad186:::benchmark_data_smithi018_78628_object15101:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-10T18:51:03.735 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:47:49.063568+0000", 2024-09-10T18:51:03.735 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 194.65192955399999, 2024-09-10T18:51:03.735 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.143460796, 2024-09-10T18:51:03.735 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:51:03.735 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:51:03.735 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:51:03.735 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.14790", 2024-09-10T18:51:03.735 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:51:03.736 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 15102 2024-09-10T18:51:03.736 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.736 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:51:03.736 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.736 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:51:03.736 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.063568+0000", 2024-09-10T18:51:03.736 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.736 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.736 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.737 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:51:03.737 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.063568+0000", 2024-09-10T18:51:03.737 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.737 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.737 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.737 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:51:03.737 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.063568+0000", 2024-09-10T18:51:03.737 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:51:03.737 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.738 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.738 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:51:03.738 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.063589+0000", 2024-09-10T18:51:03.738 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.1931999999999999e-05 2024-09-10T18:51:03.738 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.738 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.738 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:51:03.738 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.063599+0000", 2024-09-10T18:51:03.738 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.3449999999999995e-06 2024-09-10T18:51:03.738 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.738 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.738 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:51:03.738 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.063604+0000", 2024-09-10T18:51:03.739 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.5330000000000002e-06 2024-09-10T18:51:03.739 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.739 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.739 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:51:03.739 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.068172+0000", 2024-09-10T18:51:03.739 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0045676079999999999 2024-09-10T18:51:03.739 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.739 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.739 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:51:03.739 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.068284+0000", 2024-09-10T18:51:03.739 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00011201699999999999 2024-09-10T18:51:03.739 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.739 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.740 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,7", 2024-09-10T18:51:03.740 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.068607+0000", 2024-09-10T18:51:03.740 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00032275000000000001 2024-09-10T18:51:03.740 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.740 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.740 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:51:03.740 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.204807+0000", 2024-09-10T18:51:03.740 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.13619999099999999 2024-09-10T18:51:03.740 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.740 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.740 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:51:03.740 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.205591+0000", 2024-09-10T18:51:03.740 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00078381999999999996 2024-09-10T18:51:03.741 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.741 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.741 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:51:03.741 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.205993+0000", 2024-09-10T18:51:03.741 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00040267999999999999 2024-09-10T18:51:03.741 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.741 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.741 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:51:03.741 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.206268+0000", 2024-09-10T18:51:03.741 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00027449499999999998 2024-09-10T18:51:03.741 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.741 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.742 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:51:03.742 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.207029+0000", 2024-09-10T18:51:03.742 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000761533 2024-09-10T18:51:03.742 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.742 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:51:03.742 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.742 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.742 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.742 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.14790.0:15105 7.7 e92/86 7:eab71933:::benchmark_data_smithi018_78628_object15104:head v 92'956, mlcod=92'952)", 2024-09-10T18:51:03.742 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:47:49.065765+0000", 2024-09-10T18:51:03.742 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 194.649733449, 2024-09-10T18:51:03.742 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.14022161899999999, 2024-09-10T18:51:03.742 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:51:03.743 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:51:03.743 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:51:03.743 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.743 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:51:03.743 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.065765+0000", 2024-09-10T18:51:03.743 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.743 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.743 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.743 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:51:03.743 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.065765+0000", 2024-09-10T18:51:03.743 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.743 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.743 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.744 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:51:03.744 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.065764+0000", 2024-09-10T18:51:03.744 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T18:51:03.744 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.744 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.744 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:51:03.744 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.065806+0000", 2024-09-10T18:51:03.744 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.1813000000000001e-05 2024-09-10T18:51:03.744 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.744 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.744 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:51:03.744 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.065822+0000", 2024-09-10T18:51:03.745 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.6350000000000001e-05 2024-09-10T18:51:03.745 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.745 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.745 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:51:03.745 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.065828+0000", 2024-09-10T18:51:03.745 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.6670000000000001e-06 2024-09-10T18:51:03.745 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.745 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.745 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:51:03.745 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.066567+0000", 2024-09-10T18:51:03.745 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000739367 2024-09-10T18:51:03.745 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.745 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.746 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:51:03.746 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.067370+0000", 2024-09-10T18:51:03.746 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00080255400000000003 2024-09-10T18:51:03.746 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.746 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.746 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:51:03.746 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.205956+0000", 2024-09-10T18:51:03.746 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.138586655 2024-09-10T18:51:03.746 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.746 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.746 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:51:03.746 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.205986+0000", 2024-09-10T18:51:03.746 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.9887999999999999e-05 2024-09-10T18:51:03.746 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.747 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:51:03.747 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.747 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.747 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.747 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop_reply(client.14790.0:15084 7.f e92/86 ondisk, result = 0)", 2024-09-10T18:51:03.747 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:47:49.066332+0000", 2024-09-10T18:51:03.747 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 194.64916578099999, 2024-09-10T18:51:03.747 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.13922705699999999, 2024-09-10T18:51:03.747 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:51:03.747 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "started", 2024-09-10T18:51:03.747 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:51:03.747 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.747 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:51:03.748 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.066332+0000", 2024-09-10T18:51:03.748 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.748 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.748 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.748 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:51:03.748 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.066332+0000", 2024-09-10T18:51:03.748 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.748 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.748 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.748 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:51:03.748 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.066332+0000", 2024-09-10T18:51:03.748 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T18:51:03.748 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.749 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.749 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:51:03.749 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.066336+0000", 2024-09-10T18:51:03.749 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.826e-06 2024-09-10T18:51:03.749 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.749 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.749 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:51:03.749 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.066356+0000", 2024-09-10T18:51:03.749 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.9568e-05 2024-09-10T18:51:03.749 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.749 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.749 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:51:03.749 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.066361+0000", 2024-09-10T18:51:03.749 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.8609999999999997e-06 2024-09-10T18:51:03.750 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.750 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.750 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:51:03.750 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.067724+0000", 2024-09-10T18:51:03.750 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.001362994 2024-09-10T18:51:03.750 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.750 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.750 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:51:03.750 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.067732+0000", 2024-09-10T18:51:03.750 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.354e-06 2024-09-10T18:51:03.750 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.750 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.750 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:51:03.751 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.205559+0000", 2024-09-10T18:51:03.751 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.137827163 2024-09-10T18:51:03.751 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.751 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:51:03.751 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.751 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.751 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.751 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop_reply(client.14790.0:15089 7.f e92/86 ondisk, result = 0)", 2024-09-10T18:51:03.751 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:47:49.068389+0000", 2024-09-10T18:51:03.751 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 194.64710873300001, 2024-09-10T18:51:03.751 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.13638294200000001, 2024-09-10T18:51:03.751 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:51:03.751 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "started", 2024-09-10T18:51:03.752 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:51:03.752 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.752 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:51:03.752 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.068389+0000", 2024-09-10T18:51:03.752 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.752 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.752 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.752 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:51:03.752 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.068389+0000", 2024-09-10T18:51:03.752 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.752 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.752 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.752 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:51:03.753 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.068388+0000", 2024-09-10T18:51:03.753 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:51:03.753 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.753 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.753 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:51:03.753 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.068394+0000", 2024-09-10T18:51:03.753 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.2390000000000002e-06 2024-09-10T18:51:03.753 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.753 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.753 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:51:03.753 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.068404+0000", 2024-09-10T18:51:03.753 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0798999999999999e-05 2024-09-10T18:51:03.753 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.754 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.754 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:51:03.754 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.068410+0000", 2024-09-10T18:51:03.754 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.5729999999999998e-06 2024-09-10T18:51:03.754 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.754 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.754 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:51:03.754 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.204758+0000", 2024-09-10T18:51:03.754 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.13634766700000001 2024-09-10T18:51:03.754 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.754 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.754 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:51:03.754 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.204765+0000", 2024-09-10T18:51:03.755 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.6159999999999996e-06 2024-09-10T18:51:03.755 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.755 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.755 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:51:03.755 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.204772+0000", 2024-09-10T18:51:03.755 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.0090000000000001e-06 2024-09-10T18:51:03.755 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.755 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:51:03.755 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.755 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.755 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.755 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.14790.0:15114 7.5 e92/88 7:a766245d:::benchmark_data_smithi018_78628_object15113:head v 92'938, mlcod=92'936)", 2024-09-10T18:51:03.755 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:47:49.068813+0000", 2024-09-10T18:51:03.756 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 194.646685488, 2024-09-10T18:51:03.756 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.13709186000000001, 2024-09-10T18:51:03.756 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:51:03.756 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:51:03.756 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:51:03.756 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.756 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:51:03.756 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.068813+0000", 2024-09-10T18:51:03.756 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.756 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.756 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.756 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:51:03.756 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.068813+0000", 2024-09-10T18:51:03.757 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.757 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.757 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.757 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:51:03.757 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.068812+0000", 2024-09-10T18:51:03.757 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T18:51:03.757 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.757 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.757 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:51:03.757 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.068848+0000", 2024-09-10T18:51:03.757 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.5839e-05 2024-09-10T18:51:03.757 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.758 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.758 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:51:03.758 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.202967+0000", 2024-09-10T18:51:03.758 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.134119133 2024-09-10T18:51:03.758 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.758 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.758 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:51:03.758 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.202989+0000", 2024-09-10T18:51:03.758 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.1642999999999999e-05 2024-09-10T18:51:03.759 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.759 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.759 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:51:03.759 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.204630+0000", 2024-09-10T18:51:03.759 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0016412670000000001 2024-09-10T18:51:03.759 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.759 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.759 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:51:03.759 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.204652+0000", 2024-09-10T18:51:03.759 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.1885000000000001e-05 2024-09-10T18:51:03.759 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.759 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.759 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:51:03.760 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.205864+0000", 2024-09-10T18:51:03.760 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.001212631 2024-09-10T18:51:03.760 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.760 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.760 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:51:03.760 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.205904+0000", 2024-09-10T18:51:03.760 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.0030000000000001e-05 2024-09-10T18:51:03.760 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.760 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:51:03.760 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.760 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.760 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.760 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.14790.0:15116 7.e e92/88 7:75fa8b39:::benchmark_data_smithi018_78628_object15115:head v 92'910, mlcod=92'908)", 2024-09-10T18:51:03.761 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:47:49.069468+0000", 2024-09-10T18:51:03.761 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 194.64603021400001, 2024-09-10T18:51:03.761 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.13572091999999999, 2024-09-10T18:51:03.761 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:51:03.761 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:51:03.761 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:51:03.761 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.761 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:51:03.761 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.069468+0000", 2024-09-10T18:51:03.761 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.761 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.761 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.762 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:51:03.762 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.069468+0000", 2024-09-10T18:51:03.762 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.762 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.762 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.762 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:51:03.762 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.069467+0000", 2024-09-10T18:51:03.762 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:51:03.762 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.762 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.762 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:51:03.762 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.069700+0000", 2024-09-10T18:51:03.762 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00023323699999999999 2024-09-10T18:51:03.762 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.763 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.763 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:51:03.763 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.202952+0000", 2024-09-10T18:51:03.763 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.133252174 2024-09-10T18:51:03.763 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.763 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.763 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:51:03.763 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.202964+0000", 2024-09-10T18:51:03.763 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1657e-05 2024-09-10T18:51:03.763 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.763 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.763 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:51:03.763 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.203026+0000", 2024-09-10T18:51:03.764 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.2201999999999998e-05 2024-09-10T18:51:03.764 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.764 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.764 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:51:03.764 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.203050+0000", 2024-09-10T18:51:03.764 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.3719000000000001e-05 2024-09-10T18:51:03.764 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.764 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.764 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:51:03.764 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.205061+0000", 2024-09-10T18:51:03.764 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0020112789999999999 2024-09-10T18:51:03.764 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.764 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.765 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:51:03.765 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.205189+0000", 2024-09-10T18:51:03.765 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00012742299999999999 2024-09-10T18:51:03.765 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.765 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:51:03.765 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.765 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.765 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.765 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.14790.0:146053 7.9 7:9cc19226:::benchmark_data_smithi018_78628_object9833:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-10T18:51:03.765 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:49:23.320888+0000", 2024-09-10T18:51:03.765 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 100.394609714, 2024-09-10T18:51:03.765 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.139333971, 2024-09-10T18:51:03.765 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:51:03.766 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:51:03.766 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:51:03.766 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.14790", 2024-09-10T18:51:03.766 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:51:03.766 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 146053 2024-09-10T18:51:03.766 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.766 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:51:03.766 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.766 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:51:03.766 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.320888+0000", 2024-09-10T18:51:03.766 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.766 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.766 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.766 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:51:03.767 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.320888+0000", 2024-09-10T18:51:03.767 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.767 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:03.768 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.769 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.769 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:51:03.769 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.320888+0000", 2024-09-10T18:51:03.769 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T18:51:03.769 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.769 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.769 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:51:03.769 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.320891+0000", 2024-09-10T18:51:03.769 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.202e-06 2024-09-10T18:51:03.769 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.769 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.769 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:51:03.769 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.320897+0000", 2024-09-10T18:51:03.770 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.0129999999999999e-06 2024-09-10T18:51:03.770 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.770 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.770 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:51:03.770 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.320901+0000", 2024-09-10T18:51:03.770 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.3680000000000004e-06 2024-09-10T18:51:03.770 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.770 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.770 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:51:03.770 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.329096+0000", 2024-09-10T18:51:03.770 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00819456 2024-09-10T18:51:03.770 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.770 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.771 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:51:03.771 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.329190+0000", 2024-09-10T18:51:03.771 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.3933999999999999e-05 2024-09-10T18:51:03.771 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.771 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.771 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,4", 2024-09-10T18:51:03.771 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.329289+0000", 2024-09-10T18:51:03.771 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.9083e-05 2024-09-10T18:51:03.771 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.771 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.771 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:51:03.771 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.336143+0000", 2024-09-10T18:51:03.771 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.006853568 2024-09-10T18:51:03.772 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.772 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.772 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:51:03.772 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.337129+0000", 2024-09-10T18:51:03.772 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0009866810000000001 2024-09-10T18:51:03.772 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.772 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.772 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:51:03.772 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.460133+0000", 2024-09-10T18:51:03.772 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.123003684 2024-09-10T18:51:03.772 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.772 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.772 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:51:03.773 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.460186+0000", 2024-09-10T18:51:03.773 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.2760000000000003e-05 2024-09-10T18:51:03.773 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.773 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.773 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:51:03.773 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.460222+0000", 2024-09-10T18:51:03.773 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.6562e-05 2024-09-10T18:51:03.773 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.773 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:51:03.773 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.773 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.773 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.773 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.14790.0:146359 7.9 7:95c11f99:::benchmark_data_smithi018_78628_object10139:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-10T18:51:03.774 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:49:23.667748+0000", 2024-09-10T18:51:03.774 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 100.047750472, 2024-09-10T18:51:03.774 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.219793406, 2024-09-10T18:51:03.774 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:51:03.774 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:51:03.774 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:51:03.774 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.14790", 2024-09-10T18:51:03.774 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:51:03.774 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 146359 2024-09-10T18:51:03.774 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.774 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:51:03.774 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.774 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:51:03.775 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.667748+0000", 2024-09-10T18:51:03.775 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.775 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.775 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.775 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:51:03.775 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.667748+0000", 2024-09-10T18:51:03.775 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.775 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.775 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.775 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:51:03.775 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.667746+0000", 2024-09-10T18:51:03.775 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999981 2024-09-10T18:51:03.775 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.776 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.776 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:51:03.776 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.667753+0000", 2024-09-10T18:51:03.776 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.3379999999999999e-06 2024-09-10T18:51:03.776 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.776 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.776 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:51:03.776 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.667770+0000", 2024-09-10T18:51:03.776 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.6586999999999999e-05 2024-09-10T18:51:03.776 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.776 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.776 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:51:03.776 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.667776+0000", 2024-09-10T18:51:03.776 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.9959999999999999e-06 2024-09-10T18:51:03.777 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.777 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.777 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:51:03.777 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.725036+0000", 2024-09-10T18:51:03.777 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.057260294000000003 2024-09-10T18:51:03.777 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.777 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.777 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:51:03.777 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.725156+0000", 2024-09-10T18:51:03.777 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00012037200000000001 2024-09-10T18:51:03.777 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.777 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.777 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,4", 2024-09-10T18:51:03.778 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.725281+0000", 2024-09-10T18:51:03.778 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00012493500000000001 2024-09-10T18:51:03.778 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.778 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.778 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:51:03.778 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.732309+0000", 2024-09-10T18:51:03.778 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0070278479999999997 2024-09-10T18:51:03.778 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.778 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.778 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:51:03.778 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.732341+0000", 2024-09-10T18:51:03.778 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.1523000000000003e-05 2024-09-10T18:51:03.778 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.778 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.779 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:51:03.779 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.887378+0000", 2024-09-10T18:51:03.779 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.1550378 2024-09-10T18:51:03.779 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.779 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.779 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:51:03.779 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.887469+0000", 2024-09-10T18:51:03.779 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.0984999999999997e-05 2024-09-10T18:51:03.779 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.779 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.779 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:51:03.779 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.887541+0000", 2024-09-10T18:51:03.779 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.1506000000000003e-05 2024-09-10T18:51:03.780 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.780 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:51:03.780 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.780 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.780 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.780 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.14790.0:146369 7.9 7:9de67bd4:::benchmark_data_smithi018_78628_object10149:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-10T18:51:03.780 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:49:23.694998+0000", 2024-09-10T18:51:03.780 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 100.02050016699999, 2024-09-10T18:51:03.780 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.19291802199999999, 2024-09-10T18:51:03.780 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:51:03.780 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:51:03.780 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:51:03.780 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.14790", 2024-09-10T18:51:03.781 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:51:03.781 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 146369 2024-09-10T18:51:03.781 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.781 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:51:03.781 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.781 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:51:03.781 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.694998+0000", 2024-09-10T18:51:03.781 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.781 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.781 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.781 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:51:03.781 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.694998+0000", 2024-09-10T18:51:03.781 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.782 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.782 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.782 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:51:03.782 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.694996+0000", 2024-09-10T18:51:03.782 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999981 2024-09-10T18:51:03.782 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.782 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.782 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:51:03.782 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.695006+0000", 2024-09-10T18:51:03.782 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.6979999999999993e-06 2024-09-10T18:51:03.782 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.782 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.782 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:51:03.782 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.695018+0000", 2024-09-10T18:51:03.782 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.2500999999999999e-05 2024-09-10T18:51:03.783 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.783 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.783 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:51:03.783 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.695025+0000", 2024-09-10T18:51:03.783 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.2609999999999999e-06 2024-09-10T18:51:03.783 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.783 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.783 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:51:03.783 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.728435+0000", 2024-09-10T18:51:03.783 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.033410178999999998 2024-09-10T18:51:03.783 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.783 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.784 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:51:03.784 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.728761+0000", 2024-09-10T18:51:03.784 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00032572900000000003 2024-09-10T18:51:03.784 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.784 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.784 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,4", 2024-09-10T18:51:03.784 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.729180+0000", 2024-09-10T18:51:03.784 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00041982899999999998 2024-09-10T18:51:03.785 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.785 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.785 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:51:03.785 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.735248+0000", 2024-09-10T18:51:03.785 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0060678750000000004 2024-09-10T18:51:03.785 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.785 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.785 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:51:03.785 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.735316+0000", 2024-09-10T18:51:03.785 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.7421000000000004e-05 2024-09-10T18:51:03.785 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.785 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.785 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:51:03.786 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.887830+0000", 2024-09-10T18:51:03.786 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.152514286 2024-09-10T18:51:03.786 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.786 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.786 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:51:03.786 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.887879+0000", 2024-09-10T18:51:03.786 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.9563e-05 2024-09-10T18:51:03.786 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.786 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.786 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:51:03.786 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.887916+0000", 2024-09-10T18:51:03.786 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.6366999999999999e-05 2024-09-10T18:51:03.786 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.787 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:51:03.787 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.787 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.787 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.787 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.14790.0:146386 7.9 7:99f25c3c:::benchmark_data_smithi018_78628_object10166:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-10T18:51:03.787 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:49:23.703036+0000", 2024-09-10T18:51:03.787 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 100.012462, 2024-09-10T18:51:03.787 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.185074444, 2024-09-10T18:51:03.787 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:51:03.787 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:51:03.787 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:51:03.787 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.14790", 2024-09-10T18:51:03.787 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:51:03.788 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 146386 2024-09-10T18:51:03.788 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.788 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:51:03.788 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.788 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:51:03.788 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.703036+0000", 2024-09-10T18:51:03.788 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.788 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.788 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.788 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:51:03.788 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.703036+0000", 2024-09-10T18:51:03.788 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.788 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.789 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.789 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:51:03.789 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.703033+0000", 2024-09-10T18:51:03.789 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999976 2024-09-10T18:51:03.789 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.789 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.789 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:51:03.789 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.703042+0000", 2024-09-10T18:51:03.789 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.8170000000000005e-06 2024-09-10T18:51:03.789 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.789 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.789 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:51:03.789 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.703057+0000", 2024-09-10T18:51:03.790 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.4837999999999999e-05 2024-09-10T18:51:03.790 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.790 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.790 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:51:03.790 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.703064+0000", 2024-09-10T18:51:03.790 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.5400000000000001e-06 2024-09-10T18:51:03.790 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.790 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.790 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:51:03.790 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.732360+0000", 2024-09-10T18:51:03.790 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.029296316999999999 2024-09-10T18:51:03.790 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.790 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.791 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:51:03.791 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.732479+0000", 2024-09-10T18:51:03.791 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00011935200000000001 2024-09-10T18:51:03.791 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.791 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.791 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,4", 2024-09-10T18:51:03.791 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.732613+0000", 2024-09-10T18:51:03.791 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000133387 2024-09-10T18:51:03.791 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.791 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.791 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:51:03.791 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.737489+0000", 2024-09-10T18:51:03.791 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0048760050000000001 2024-09-10T18:51:03.792 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.792 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.792 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:51:03.792 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.738046+0000", 2024-09-10T18:51:03.792 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00055733400000000004 2024-09-10T18:51:03.792 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.792 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.792 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:51:03.792 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.887991+0000", 2024-09-10T18:51:03.792 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.14994532299999999 2024-09-10T18:51:03.792 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.792 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.792 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:51:03.793 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.888067+0000", 2024-09-10T18:51:03.793 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.5295000000000003e-05 2024-09-10T18:51:03.793 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.793 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.793 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:51:03.793 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.888110+0000", 2024-09-10T18:51:03.793 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.3776000000000002e-05 2024-09-10T18:51:03.793 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.793 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:51:03.793 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.793 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.793 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.793 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.14790.0:146589 7.9 7:90c17eed:::benchmark_data_smithi018_78628_object10369:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-10T18:51:03.793 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:49:23.961045+0000", 2024-09-10T18:51:03.794 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 99.754452932999996, 2024-09-10T18:51:03.794 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.13920028000000001, 2024-09-10T18:51:03.794 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:51:03.794 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:51:03.794 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:51:03.794 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.14790", 2024-09-10T18:51:03.794 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:51:03.794 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 146589 2024-09-10T18:51:03.794 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.794 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:51:03.794 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.794 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:51:03.794 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.961045+0000", 2024-09-10T18:51:03.795 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.795 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.795 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.795 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:51:03.795 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.961045+0000", 2024-09-10T18:51:03.795 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.795 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.795 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.795 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:51:03.795 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.961044+0000", 2024-09-10T18:51:03.795 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T18:51:03.795 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.795 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.796 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:51:03.796 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.961051+0000", 2024-09-10T18:51:03.796 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.25e-06 2024-09-10T18:51:03.796 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.796 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.796 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:51:03.796 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.961063+0000", 2024-09-10T18:51:03.796 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.2252999999999999e-05 2024-09-10T18:51:03.796 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.796 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.796 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:51:03.796 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:23.961069+0000", 2024-09-10T18:51:03.796 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.173e-06 2024-09-10T18:51:03.797 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.797 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.797 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:51:03.797 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:24.000187+0000", 2024-09-10T18:51:03.797 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.039117353000000001 2024-09-10T18:51:03.797 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.797 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.797 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:51:03.797 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:24.000292+0000", 2024-09-10T18:51:03.797 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000105356 2024-09-10T18:51:03.797 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.797 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.797 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,4", 2024-09-10T18:51:03.798 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:24.000425+0000", 2024-09-10T18:51:03.798 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00013272100000000001 2024-09-10T18:51:03.798 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.798 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.798 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:51:03.798 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:24.006340+0000", 2024-09-10T18:51:03.798 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0059153010000000004 2024-09-10T18:51:03.798 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.798 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.798 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:51:03.798 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:24.006356+0000", 2024-09-10T18:51:03.798 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.6492999999999999e-05 2024-09-10T18:51:03.799 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.799 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.799 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:51:03.799 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:24.100171+0000", 2024-09-10T18:51:03.799 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.093814432000000003 2024-09-10T18:51:03.799 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.799 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.799 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:51:03.799 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:24.100211+0000", 2024-09-10T18:51:03.799 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.0575999999999999e-05 2024-09-10T18:51:03.799 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.799 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.799 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:51:03.800 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:24.100245+0000", 2024-09-10T18:51:03.800 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.3936999999999997e-05 2024-09-10T18:51:03.800 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.800 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:51:03.800 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.800 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.800 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.800 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.14790.0:149936 7.9 7:94aa4d09:::benchmark_data_smithi018_78628_object13716:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-10T18:51:03.800 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:49:26.873620+0000", 2024-09-10T18:51:03.800 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 96.841877921000005, 2024-09-10T18:51:03.800 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.161792673, 2024-09-10T18:51:03.800 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:51:03.800 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:51:03.801 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:51:03.801 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.14790", 2024-09-10T18:51:03.801 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:51:03.801 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 149936 2024-09-10T18:51:03.801 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.801 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:51:03.801 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.801 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:51:03.801 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:26.873620+0000", 2024-09-10T18:51:03.801 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.801 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.801 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.801 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:51:03.802 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:26.873620+0000", 2024-09-10T18:51:03.802 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.802 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.802 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.802 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:51:03.802 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:26.873619+0000", 2024-09-10T18:51:03.802 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T18:51:03.802 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.802 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.802 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:51:03.802 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:26.873626+0000", 2024-09-10T18:51:03.802 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.1289999999999998e-06 2024-09-10T18:51:03.802 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.803 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.803 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:51:03.803 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:26.873637+0000", 2024-09-10T18:51:03.803 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1072e-05 2024-09-10T18:51:03.803 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.803 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.803 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:51:03.803 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:26.873642+0000", 2024-09-10T18:51:03.803 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.3820000000000003e-06 2024-09-10T18:51:03.803 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.803 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.803 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:51:03.804 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:26.928063+0000", 2024-09-10T18:51:03.804 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.054421342999999997 2024-09-10T18:51:03.804 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.804 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.804 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:51:03.804 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:26.928199+0000", 2024-09-10T18:51:03.805 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00013511400000000001 2024-09-10T18:51:03.805 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.805 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.805 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,4", 2024-09-10T18:51:03.805 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:26.928324+0000", 2024-09-10T18:51:03.805 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000124931 2024-09-10T18:51:03.805 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.805 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.805 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:51:03.805 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:26.934054+0000", 2024-09-10T18:51:03.805 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0057301380000000001 2024-09-10T18:51:03.805 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.805 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.806 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:51:03.806 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:26.934241+0000", 2024-09-10T18:51:03.806 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000187388 2024-09-10T18:51:03.806 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.806 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.806 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:51:03.806 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:27.035346+0000", 2024-09-10T18:51:03.806 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.101104705 2024-09-10T18:51:03.806 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.806 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.806 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:51:03.806 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:27.035381+0000", 2024-09-10T18:51:03.806 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.5178000000000001e-05 2024-09-10T18:51:03.807 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.807 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.807 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:51:03.807 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:27.035413+0000", 2024-09-10T18:51:03.807 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.1838000000000002e-05 2024-09-10T18:51:03.807 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.807 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:51:03.807 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.807 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.807 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.807 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.14790.0:149945 7.9 7:9bf8e2c4:::benchmark_data_smithi018_78628_object13725:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-10T18:51:03.807 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:49:26.881346+0000", 2024-09-10T18:51:03.807 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 96.834151809000005, 2024-09-10T18:51:03.808 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.154192629, 2024-09-10T18:51:03.808 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:51:03.808 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:51:03.808 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:51:03.808 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.14790", 2024-09-10T18:51:03.808 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:51:03.808 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 149945 2024-09-10T18:51:03.808 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.808 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:51:03.808 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.808 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:51:03.808 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:26.881346+0000", 2024-09-10T18:51:03.809 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.809 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.809 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.809 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:51:03.809 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:26.881346+0000", 2024-09-10T18:51:03.809 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.809 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.809 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.809 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:51:03.809 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:26.881345+0000", 2024-09-10T18:51:03.809 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T18:51:03.809 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.809 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.810 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:51:03.810 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:26.881351+0000", 2024-09-10T18:51:03.810 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.2609999999999999e-06 2024-09-10T18:51:03.810 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.810 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.810 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:51:03.810 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:26.881392+0000", 2024-09-10T18:51:03.810 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.1229000000000001e-05 2024-09-10T18:51:03.810 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.810 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.810 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:51:03.810 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:26.881398+0000", 2024-09-10T18:51:03.810 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.9120000000000003e-06 2024-09-10T18:51:03.811 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.811 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.811 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:51:03.811 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:26.931008+0000", 2024-09-10T18:51:03.811 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.049610072999999998 2024-09-10T18:51:03.811 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.811 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.811 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:51:03.811 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:26.931115+0000", 2024-09-10T18:51:03.811 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00010668899999999999 2024-09-10T18:51:03.811 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.811 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.811 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,4", 2024-09-10T18:51:03.811 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:26.931223+0000", 2024-09-10T18:51:03.812 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0001087 2024-09-10T18:51:03.812 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.812 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.812 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:51:03.812 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:26.934085+0000", 2024-09-10T18:51:03.812 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0028619380000000001 2024-09-10T18:51:03.812 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.812 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.812 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:51:03.812 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:26.934725+0000", 2024-09-10T18:51:03.812 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00063966499999999998 2024-09-10T18:51:03.812 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.812 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.813 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:51:03.813 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:27.035472+0000", 2024-09-10T18:51:03.813 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.100746531 2024-09-10T18:51:03.813 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.813 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.813 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:51:03.813 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:27.035509+0000", 2024-09-10T18:51:03.815 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.6980000000000002e-05 2024-09-10T18:51:03.815 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.815 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.815 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:51:03.815 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:27.035539+0000", 2024-09-10T18:51:03.815 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.0298999999999999e-05 2024-09-10T18:51:03.815 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.815 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:51:03.815 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.815 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.815 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.816 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.14790.0:150797 7.9 7:9e1b9b8f:::benchmark_data_smithi018_78628_object14577:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-10T18:51:03.816 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:49:27.795344+0000", 2024-09-10T18:51:03.816 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 95.920153701000004, 2024-09-10T18:51:03.816 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.14791532199999999, 2024-09-10T18:51:03.816 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:51:03.816 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:51:03.816 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:51:03.816 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.14790", 2024-09-10T18:51:03.816 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:51:03.816 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 150797 2024-09-10T18:51:03.816 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.816 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:51:03.816 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.817 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:51:03.817 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:27.795344+0000", 2024-09-10T18:51:03.817 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.817 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.817 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.817 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:51:03.817 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:27.795344+0000", 2024-09-10T18:51:03.817 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.817 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.817 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.817 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:51:03.817 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:27.795343+0000", 2024-09-10T18:51:03.817 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999981 2024-09-10T18:51:03.818 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.818 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.818 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:51:03.818 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:27.795350+0000", 2024-09-10T18:51:03.818 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.5360000000000004e-06 2024-09-10T18:51:03.818 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.818 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.818 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:51:03.818 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:27.795365+0000", 2024-09-10T18:51:03.818 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.5415000000000001e-05 2024-09-10T18:51:03.818 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.818 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.818 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:51:03.819 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:27.795373+0000", 2024-09-10T18:51:03.819 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.4580000000000004e-06 2024-09-10T18:51:03.819 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.819 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.819 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:51:03.819 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:27.836825+0000", 2024-09-10T18:51:03.819 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.041451613999999998 2024-09-10T18:51:03.819 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.819 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.819 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:51:03.819 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:27.836922+0000", 2024-09-10T18:51:03.819 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.7317000000000004e-05 2024-09-10T18:51:03.819 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.819 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.820 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,4", 2024-09-10T18:51:03.820 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:27.837038+0000", 2024-09-10T18:51:03.820 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000115949 2024-09-10T18:51:03.820 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.820 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.820 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:51:03.820 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:27.843959+0000", 2024-09-10T18:51:03.820 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0069215689999999998 2024-09-10T18:51:03.820 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.820 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.820 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:51:03.820 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:27.843988+0000", 2024-09-10T18:51:03.820 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.8493e-05 2024-09-10T18:51:03.821 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.821 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.821 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:51:03.821 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:27.943165+0000", 2024-09-10T18:51:03.821 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.099177497000000003 2024-09-10T18:51:03.821 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.821 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.821 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:51:03.821 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:27.943218+0000", 2024-09-10T18:51:03.821 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.2818000000000002e-05 2024-09-10T18:51:03.821 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.821 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.822 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:51:03.822 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:27.943260+0000", 2024-09-10T18:51:03.822 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.1446999999999998e-05 2024-09-10T18:51:03.822 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.822 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:51:03.822 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.822 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.822 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.822 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.14790.0:155211 7.9 7:91bd671b:::benchmark_data_smithi018_78628_object18991:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-10T18:51:03.822 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:49:32.012064+0000", 2024-09-10T18:51:03.822 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 91.703434127999998, 2024-09-10T18:51:03.822 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.13539232100000001, 2024-09-10T18:51:03.822 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:51:03.822 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:51:03.823 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:51:03.823 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.14790", 2024-09-10T18:51:03.823 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:51:03.823 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 155211 2024-09-10T18:51:03.823 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.823 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:51:03.823 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.823 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:51:03.823 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:32.012064+0000", 2024-09-10T18:51:03.823 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.823 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.823 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.823 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:51:03.824 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:32.012064+0000", 2024-09-10T18:51:03.824 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.824 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.824 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.824 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:51:03.824 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:32.012063+0000", 2024-09-10T18:51:03.824 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:51:03.824 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.824 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.824 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:51:03.824 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:32.012069+0000", 2024-09-10T18:51:03.824 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.716e-06 2024-09-10T18:51:03.824 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.825 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.825 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:51:03.825 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:32.012080+0000", 2024-09-10T18:51:03.825 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1358e-05 2024-09-10T18:51:03.825 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.825 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.825 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:51:03.825 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:32.012086+0000", 2024-09-10T18:51:03.825 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.2859999999999996e-06 2024-09-10T18:51:03.825 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.825 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.825 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:51:03.825 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:32.040639+0000", 2024-09-10T18:51:03.826 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.02855241 2024-09-10T18:51:03.826 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.826 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.826 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:51:03.826 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:32.040755+0000", 2024-09-10T18:51:03.826 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00011664500000000001 2024-09-10T18:51:03.826 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.826 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.826 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,4", 2024-09-10T18:51:03.826 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:32.040879+0000", 2024-09-10T18:51:03.826 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00012321000000000001 2024-09-10T18:51:03.826 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.826 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.826 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:51:03.827 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:32.044152+0000", 2024-09-10T18:51:03.827 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0032732389999999998 2024-09-10T18:51:03.827 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.827 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.827 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:51:03.827 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:32.047777+0000", 2024-09-10T18:51:03.827 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.003625217 2024-09-10T18:51:03.827 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.827 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.827 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:51:03.827 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:32.147338+0000", 2024-09-10T18:51:03.827 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.099561329000000004 2024-09-10T18:51:03.827 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.828 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.828 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:51:03.828 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:32.147418+0000", 2024-09-10T18:51:03.828 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.9779999999999998e-05 2024-09-10T18:51:03.828 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.828 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.828 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:51:03.828 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:32.147456+0000", 2024-09-10T18:51:03.828 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.8126000000000001e-05 2024-09-10T18:51:03.828 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.828 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:51:03.828 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.829 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.829 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.829 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.14790.0:198282 7.6 7:6424f68e:::benchmark_data_smithi018_78628_object62062:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-10T18:51:03.829 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:50:08.398779+0000", 2024-09-10T18:51:03.829 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 55.316718760000001, 2024-09-10T18:51:03.829 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.15264042799999999, 2024-09-10T18:51:03.829 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:51:03.829 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:51:03.829 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:51:03.829 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.14790", 2024-09-10T18:51:03.829 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:51:03.829 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 198282 2024-09-10T18:51:03.829 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.830 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:51:03.830 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.830 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:51:03.830 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.398779+0000", 2024-09-10T18:51:03.830 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.830 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.830 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.830 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:51:03.830 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.398779+0000", 2024-09-10T18:51:03.830 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.830 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.830 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.830 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:51:03.831 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.398778+0000", 2024-09-10T18:51:03.831 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T18:51:03.831 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.831 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.831 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:51:03.831 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.398783+0000", 2024-09-10T18:51:03.831 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.0490000000000003e-06 2024-09-10T18:51:03.831 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.831 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.831 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:51:03.831 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.398795+0000", 2024-09-10T18:51:03.831 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1433e-05 2024-09-10T18:51:03.831 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.832 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.832 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:51:03.832 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.398801+0000", 2024-09-10T18:51:03.832 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.02e-06 2024-09-10T18:51:03.832 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.832 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.832 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:51:03.832 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.403943+0000", 2024-09-10T18:51:03.832 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0051421119999999999 2024-09-10T18:51:03.832 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.832 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.832 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:51:03.832 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.404074+0000", 2024-09-10T18:51:03.833 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00013184000000000001 2024-09-10T18:51:03.833 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.833 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.833 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,7", 2024-09-10T18:51:03.833 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.404239+0000", 2024-09-10T18:51:03.833 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00016497199999999999 2024-09-10T18:51:03.833 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.833 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.833 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:51:03.833 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.410482+0000", 2024-09-10T18:51:03.833 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0062423390000000004 2024-09-10T18:51:03.833 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.833 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.834 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:51:03.834 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.410490+0000", 2024-09-10T18:51:03.834 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.8180000000000004e-06 2024-09-10T18:51:03.834 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.834 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.834 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:51:03.834 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.551338+0000", 2024-09-10T18:51:03.834 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.14084830800000001 2024-09-10T18:51:03.834 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.834 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.834 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:51:03.834 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.551382+0000", 2024-09-10T18:51:03.834 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.4184999999999998e-05 2024-09-10T18:51:03.834 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.835 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.835 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:51:03.835 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.551420+0000", 2024-09-10T18:51:03.835 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.7552000000000002e-05 2024-09-10T18:51:03.835 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.835 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:51:03.835 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.835 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.835 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.835 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.14790.0:198289 7.6 7:6a14f75a:::benchmark_data_smithi018_78628_object62069:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-10T18:51:03.835 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:50:08.405587+0000", 2024-09-10T18:51:03.835 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 55.309910997999999, 2024-09-10T18:51:03.835 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.14612413499999999, 2024-09-10T18:51:03.836 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:51:03.836 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:51:03.836 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:51:03.836 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.14790", 2024-09-10T18:51:03.836 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:51:03.836 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 198289 2024-09-10T18:51:03.836 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.836 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:51:03.836 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.836 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:51:03.836 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.405587+0000", 2024-09-10T18:51:03.836 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.836 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.837 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.837 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:51:03.837 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.405587+0000", 2024-09-10T18:51:03.837 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.837 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.837 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.837 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:51:03.837 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.405586+0000", 2024-09-10T18:51:03.837 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:51:03.837 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.837 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.837 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:51:03.837 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.405591+0000", 2024-09-10T18:51:03.838 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.1919999999999996e-06 2024-09-10T18:51:03.838 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.838 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.838 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:51:03.838 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.405603+0000", 2024-09-10T18:51:03.838 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1481000000000001e-05 2024-09-10T18:51:03.838 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.838 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.838 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:51:03.838 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.405609+0000", 2024-09-10T18:51:03.838 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.7459999999999997e-06 2024-09-10T18:51:03.838 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.838 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.839 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:51:03.839 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.407515+0000", 2024-09-10T18:51:03.839 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0019066210000000001 2024-09-10T18:51:03.839 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.839 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.839 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:51:03.839 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.407614+0000", 2024-09-10T18:51:03.839 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.8622000000000005e-05 2024-09-10T18:51:03.839 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.839 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.839 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,7", 2024-09-10T18:51:03.839 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.407710+0000", 2024-09-10T18:51:03.839 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.5950000000000001e-05 2024-09-10T18:51:03.840 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.840 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.840 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:51:03.840 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.410721+0000", 2024-09-10T18:51:03.840 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0030114640000000002 2024-09-10T18:51:03.840 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.840 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.840 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:51:03.840 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.413555+0000", 2024-09-10T18:51:03.840 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.002833374 2024-09-10T18:51:03.840 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.840 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.840 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:51:03.841 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.551456+0000", 2024-09-10T18:51:03.841 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.13790112400000001 2024-09-10T18:51:03.841 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.841 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.841 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:51:03.841 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.551620+0000", 2024-09-10T18:51:03.841 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000164587 2024-09-10T18:51:03.841 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.841 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.841 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:51:03.841 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.551711+0000", 2024-09-10T18:51:03.841 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.0849999999999999e-05 2024-09-10T18:51:03.841 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.842 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:51:03.842 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.842 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.842 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.842 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.14790.0:198300 7.6 7:6bb91396:::benchmark_data_smithi018_78628_object62080:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-10T18:51:03.842 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:50:08.414075+0000", 2024-09-10T18:51:03.842 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 55.301422936000002, 2024-09-10T18:51:03.842 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.13817996499999999, 2024-09-10T18:51:03.842 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:51:03.842 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:51:03.842 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:51:03.842 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.14790", 2024-09-10T18:51:03.842 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:51:03.842 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 198300 2024-09-10T18:51:03.843 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.843 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:51:03.843 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.843 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:51:03.843 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.414075+0000", 2024-09-10T18:51:03.843 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.843 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.843 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.843 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:51:03.843 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.414075+0000", 2024-09-10T18:51:03.843 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:51:03.843 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.843 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.843 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:51:03.844 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.414075+0000", 2024-09-10T18:51:03.844 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T18:51:03.844 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.844 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.844 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:51:03.844 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.414077+0000", 2024-09-10T18:51:03.844 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.5790000000000002e-06 2024-09-10T18:51:03.844 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.844 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.844 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:51:03.844 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.414082+0000", 2024-09-10T18:51:03.844 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.8409999999999999e-06 2024-09-10T18:51:03.845 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.845 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.845 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:51:03.845 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.414086+0000", 2024-09-10T18:51:03.845 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.4929999999999998e-06 2024-09-10T18:51:03.845 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.845 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.845 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:51:03.845 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.414273+0000", 2024-09-10T18:51:03.845 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000187839 2024-09-10T18:51:03.845 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.845 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.845 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:51:03.845 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.414390+0000", 2024-09-10T18:51:03.846 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000116215 2024-09-10T18:51:03.846 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.846 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.846 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,7", 2024-09-10T18:51:03.846 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.414488+0000", 2024-09-10T18:51:03.846 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.8778999999999997e-05 2024-09-10T18:51:03.846 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.846 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.846 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:51:03.846 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.416920+0000", 2024-09-10T18:51:03.846 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0024319250000000001 2024-09-10T18:51:03.846 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.846 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.846 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:51:03.847 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.417992+0000", 2024-09-10T18:51:03.847 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.001071926 2024-09-10T18:51:03.847 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.847 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.847 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:51:03.847 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.552054+0000", 2024-09-10T18:51:03.847 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.13406162699999999 2024-09-10T18:51:03.847 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.847 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.847 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:51:03.847 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.552092+0000", 2024-09-10T18:51:03.847 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.8172e-05 2024-09-10T18:51:03.848 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:51:03.848 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:51:03.848 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:51:03.848 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.552255+0000", 2024-09-10T18:51:03.848 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000162998 2024-09-10T18:51:03.848 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.848 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:51:03.848 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.848 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:51:03.848 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:51:03.848 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T18:51:03.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:03.849 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:03.849 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:03.925 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:03 smithi018 ceph-mon[40196]: 6.7 scrub starts 2024-09-10T18:51:03.925 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:03 smithi018 ceph-mon[40196]: 6.7 scrub ok 2024-09-10T18:51:03.925 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:03 smithi018 ceph-mon[40196]: 7.5 scrub starts 2024-09-10T18:51:03.925 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:03 smithi018 ceph-mon[40196]: 7.5 scrub ok 2024-09-10T18:51:03.925 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:03 smithi018 ceph-mon[40196]: 4.1 deep-scrub starts 2024-09-10T18:51:03.926 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:03 smithi018 ceph-mon[40196]: 4.1 deep-scrub ok 2024-09-10T18:51:03.926 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:03 smithi018 ceph-mon[40196]: pgmap v230: 158 pgs: 158 active+clean; 379 MiB data, 3.3 GiB used, 623 GiB / 626 GiB avail; 0 B/s wr, 1.61k op/s 2024-09-10T18:51:03.926 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:03 smithi018 ceph-mon[40196]: 6.18 scrub starts 2024-09-10T18:51:03.926 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:03 smithi018 ceph-mon[40196]: 6.18 scrub ok 2024-09-10T18:51:03.926 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:03 smithi018 ceph-mon[40196]: osdmap e144: 8 total, 8 up, 7 in 2024-09-10T18:51:03.926 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:03 smithi018 ceph-mon[29336]: 6.7 scrub starts 2024-09-10T18:51:03.926 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:03 smithi018 ceph-mon[29336]: 6.7 scrub ok 2024-09-10T18:51:03.926 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:03 smithi018 ceph-mon[29336]: 7.5 scrub starts 2024-09-10T18:51:03.926 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:03 smithi018 ceph-mon[29336]: 7.5 scrub ok 2024-09-10T18:51:03.926 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:03 smithi018 ceph-mon[29336]: 4.1 deep-scrub starts 2024-09-10T18:51:03.926 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:03 smithi018 ceph-mon[29336]: 4.1 deep-scrub ok 2024-09-10T18:51:03.926 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:03 smithi018 ceph-mon[29336]: pgmap v230: 158 pgs: 158 active+clean; 379 MiB data, 3.3 GiB used, 623 GiB / 626 GiB avail; 0 B/s wr, 1.61k op/s 2024-09-10T18:51:03.926 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:03 smithi018 ceph-mon[29336]: 6.18 scrub starts 2024-09-10T18:51:03.927 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:03 smithi018 ceph-mon[29336]: 6.18 scrub ok 2024-09-10T18:51:03.927 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:03 smithi018 ceph-mon[29336]: osdmap e144: 8 total, 8 up, 7 in 2024-09-10T18:51:03.950 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:03.988 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:03.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:03.989 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:04.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:03 smithi103 ceph-mon[33163]: 6.7 scrub starts 2024-09-10T18:51:04.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:03 smithi103 ceph-mon[33163]: 6.7 scrub ok 2024-09-10T18:51:04.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:03 smithi103 ceph-mon[33163]: 7.5 scrub starts 2024-09-10T18:51:04.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:03 smithi103 ceph-mon[33163]: 7.5 scrub ok 2024-09-10T18:51:04.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:03 smithi103 ceph-mon[33163]: 4.1 deep-scrub starts 2024-09-10T18:51:04.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:03 smithi103 ceph-mon[33163]: 4.1 deep-scrub ok 2024-09-10T18:51:04.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:03 smithi103 ceph-mon[33163]: pgmap v230: 158 pgs: 158 active+clean; 379 MiB data, 3.3 GiB used, 623 GiB / 626 GiB avail; 0 B/s wr, 1.61k op/s 2024-09-10T18:51:04.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:03 smithi103 ceph-mon[33163]: 6.18 scrub starts 2024-09-10T18:51:04.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:03 smithi103 ceph-mon[33163]: 6.18 scrub ok 2024-09-10T18:51:04.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:03 smithi103 ceph-mon[33163]: osdmap e144: 8 total, 8 up, 7 in 2024-09-10T18:51:04.089 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:04.131 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:04.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:04.132 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:04.233 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:04.260 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:04.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:04.262 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:04.363 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:04.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:04.404 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:04.404 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:04.505 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:04.535 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:04.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:04.536 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:04.636 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:04.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:04.682 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:04.682 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:04.783 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:04.824 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:04.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:04.825 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:04.902 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:04 smithi018 ceph-mon[40196]: 2.a scrub starts 2024-09-10T18:51:04.902 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:04 smithi018 ceph-mon[40196]: 2.a scrub ok 2024-09-10T18:51:04.902 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:04 smithi018 ceph-mon[40196]: 3.2 scrub starts 2024-09-10T18:51:04.903 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:04 smithi018 ceph-mon[40196]: 3.2 scrub ok 2024-09-10T18:51:04.903 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:04 smithi018 ceph-mon[40196]: 2.18 scrub starts 2024-09-10T18:51:04.903 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:04 smithi018 ceph-mon[40196]: 2.18 scrub ok 2024-09-10T18:51:04.903 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:04 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:51:04.903 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:04 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:51:04.903 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:04 smithi018 ceph-mon[29336]: 2.a scrub starts 2024-09-10T18:51:04.903 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:04 smithi018 ceph-mon[29336]: 2.a scrub ok 2024-09-10T18:51:04.903 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:04 smithi018 ceph-mon[29336]: 3.2 scrub starts 2024-09-10T18:51:04.904 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:04 smithi018 ceph-mon[29336]: 3.2 scrub ok 2024-09-10T18:51:04.904 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:04 smithi018 ceph-mon[29336]: 2.18 scrub starts 2024-09-10T18:51:04.904 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:04 smithi018 ceph-mon[29336]: 2.18 scrub ok 2024-09-10T18:51:04.904 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:04 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:51:04.904 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:04 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:51:04.926 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:04.970 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:04.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:04.971 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:05.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:04 smithi103 ceph-mon[33163]: 2.a scrub starts 2024-09-10T18:51:05.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:04 smithi103 ceph-mon[33163]: 2.a scrub ok 2024-09-10T18:51:05.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:04 smithi103 ceph-mon[33163]: 3.2 scrub starts 2024-09-10T18:51:05.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:04 smithi103 ceph-mon[33163]: 3.2 scrub ok 2024-09-10T18:51:05.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:04 smithi103 ceph-mon[33163]: 2.18 scrub starts 2024-09-10T18:51:05.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:04 smithi103 ceph-mon[33163]: 2.18 scrub ok 2024-09-10T18:51:05.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:04 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:51:05.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:04 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:51:05.072 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:05.100 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:05.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:05.101 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:05.201 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:05.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:05.243 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:05.243 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:05.344 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:05.386 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:05.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:05.387 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:05.487 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:05.515 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:05.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:05.516 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:05.617 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:05.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:05.665 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:05.665 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:05.681 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is set 2024-09-10T18:51:05.766 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:05.806 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:05.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:05.807 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:05.849 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Removed 136217 objects 2024-09-10T18:51:05.849 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Clean up completed and total clean up time :111.624 2024-09-10T18:51:05.867 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-10T18:51:05.908 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:05.930 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:05.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:05.931 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:05.931 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:05 smithi018 ceph-mon[29336]: 1.3 scrub starts 2024-09-10T18:51:05.931 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:05 smithi018 ceph-mon[29336]: 1.3 scrub ok 2024-09-10T18:51:05.931 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:05 smithi018 ceph-mon[29336]: 7.8 deep-scrub starts 2024-09-10T18:51:05.932 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:05 smithi018 ceph-mon[29336]: osdmap e145: 8 total, 8 up, 7 in 2024-09-10T18:51:05.932 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:05 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2561226269' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:51:05.932 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:05 smithi018 ceph-mon[29336]: pgmap v233: 158 pgs: 158 active+clean; 379 MiB data, 3.3 GiB used, 623 GiB / 626 GiB avail; 0 B/s wr, 1.03k op/s 2024-09-10T18:51:05.932 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:05 smithi018 ceph-mon[40196]: 1.3 scrub starts 2024-09-10T18:51:05.932 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:05 smithi018 ceph-mon[40196]: 1.3 scrub ok 2024-09-10T18:51:05.932 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:05 smithi018 ceph-mon[40196]: 7.8 deep-scrub starts 2024-09-10T18:51:05.932 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:05 smithi018 ceph-mon[40196]: osdmap e145: 8 total, 8 up, 7 in 2024-09-10T18:51:05.932 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:05 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2561226269' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:51:05.932 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:05 smithi018 ceph-mon[40196]: pgmap v233: 158 pgs: 158 active+clean; 379 MiB data, 3.3 GiB used, 623 GiB / 626 GiB avail; 0 B/s wr, 1.03k op/s 2024-09-10T18:51:06.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:05 smithi103 ceph-mon[33163]: 1.3 scrub starts 2024-09-10T18:51:06.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:05 smithi103 ceph-mon[33163]: 1.3 scrub ok 2024-09-10T18:51:06.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:05 smithi103 ceph-mon[33163]: 7.8 deep-scrub starts 2024-09-10T18:51:06.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:05 smithi103 ceph-mon[33163]: osdmap e145: 8 total, 8 up, 7 in 2024-09-10T18:51:06.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:05 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2561226269' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:51:06.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:05 smithi103 ceph-mon[33163]: pgmap v233: 158 pgs: 158 active+clean; 379 MiB data, 3.3 GiB used, 623 GiB / 626 GiB avail; 0 B/s wr, 1.03k op/s 2024-09-10T18:51:06.032 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:06.054 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:06.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:06.055 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:06.156 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:06.178 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:06.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:06.178 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:06.279 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:06.303 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:06.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:06.304 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:06.405 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:06.427 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:06.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:06.428 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:06.529 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:06.550 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:06.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:06.551 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:06.577 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:51:06.652 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:06.677 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:06.677 INFO:teuthology.orchestra.run.smithi018.stderr:reweighted osd.6 to 1 (10000) 2024-09-10T18:51:06.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:06.678 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:06.679 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:51:06 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:51:06.779 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:06.801 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:06.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:06.802 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:06.903 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:06.926 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:06.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:06.927 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:06.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:06 smithi018 ceph-mon[40196]: 7.8 deep-scrub ok 2024-09-10T18:51:06.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:06 smithi018 ceph-mon[40196]: 2.4 deep-scrub starts 2024-09-10T18:51:06.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:06 smithi018 ceph-mon[40196]: 2.4 deep-scrub ok 2024-09-10T18:51:06.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:06 smithi018 ceph-mon[40196]: 6.a scrub starts 2024-09-10T18:51:06.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:06 smithi018 ceph-mon[40196]: 6.a scrub ok 2024-09-10T18:51:06.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:06 smithi018 ceph-mon[40196]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:51:06.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:06 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2561226269' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:51:06.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:06 smithi018 ceph-mon[40196]: osdmap e146: 8 total, 8 up, 7 in 2024-09-10T18:51:06.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:06 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:06.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:06 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1265724341' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-10T18:51:06.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:06 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-10T18:51:06.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:06 smithi018 ceph-mon[29336]: 7.8 deep-scrub ok 2024-09-10T18:51:06.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:06 smithi018 ceph-mon[29336]: 2.4 deep-scrub starts 2024-09-10T18:51:06.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:06 smithi018 ceph-mon[29336]: 2.4 deep-scrub ok 2024-09-10T18:51:06.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:06 smithi018 ceph-mon[29336]: 6.a scrub starts 2024-09-10T18:51:06.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:06 smithi018 ceph-mon[29336]: 6.a scrub ok 2024-09-10T18:51:06.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:06 smithi018 ceph-mon[29336]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:51:06.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:06 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2561226269' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:51:06.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:06 smithi018 ceph-mon[29336]: osdmap e146: 8 total, 8 up, 7 in 2024-09-10T18:51:06.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:06 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:06.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:06 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1265724341' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-10T18:51:06.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:06 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-10T18:51:07.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:06 smithi103 ceph-mon[33163]: 7.8 deep-scrub ok 2024-09-10T18:51:07.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:06 smithi103 ceph-mon[33163]: 2.4 deep-scrub starts 2024-09-10T18:51:07.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:06 smithi103 ceph-mon[33163]: 2.4 deep-scrub ok 2024-09-10T18:51:07.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:06 smithi103 ceph-mon[33163]: 6.a scrub starts 2024-09-10T18:51:07.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:06 smithi103 ceph-mon[33163]: 6.a scrub ok 2024-09-10T18:51:07.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:06 smithi103 ceph-mon[33163]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:51:07.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:06 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2561226269' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:51:07.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:06 smithi103 ceph-mon[33163]: osdmap e146: 8 total, 8 up, 7 in 2024-09-10T18:51:07.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:06 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:07.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:06 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1265724341' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-10T18:51:07.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:06 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-10T18:51:07.027 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:07.053 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:07.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:07.054 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:07.154 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:07.178 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:07.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:07.179 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:07.280 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:07.303 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:07.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:07.304 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:07.405 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:07.428 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:07.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:07.429 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:07.530 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:07.552 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:07.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:07.553 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:07.594 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:51:07.655 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:07.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:07.680 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:07.680 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:07.781 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:07.804 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:07.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:07.805 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:07.906 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:07.929 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:07.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:07.930 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:07.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:07 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-10T18:51:07.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:07 smithi018 ceph-mon[29336]: osdmap e147: 8 total, 8 up, 7 in 2024-09-10T18:51:07.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:07 smithi018 ceph-mon[29336]: pgmap v236: 158 pgs: 158 active+clean; 687 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 0 B/s wr, 1.51k op/s 2024-09-10T18:51:07.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:07 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-10T18:51:07.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:07 smithi018 ceph-mon[40196]: osdmap e147: 8 total, 8 up, 7 in 2024-09-10T18:51:07.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:07 smithi018 ceph-mon[40196]: pgmap v236: 158 pgs: 158 active+clean; 687 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 0 B/s wr, 1.51k op/s 2024-09-10T18:51:08.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:07 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-10T18:51:08.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:07 smithi103 ceph-mon[33163]: osdmap e147: 8 total, 8 up, 7 in 2024-09-10T18:51:08.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:07 smithi103 ceph-mon[33163]: pgmap v236: 158 pgs: 158 active+clean; 687 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 0 B/s wr, 1.51k op/s 2024-09-10T18:51:08.031 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:08.057 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:08.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:08.058 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:08.159 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:08.182 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:08.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:08.184 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:08.284 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:08.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:08.309 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:08.309 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:08.410 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:08.436 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:08.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:08.437 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:08.537 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:08.561 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:08.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:08.562 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:08.663 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:08.689 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:08.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:08.691 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:08.792 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:08.819 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:08.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:08.820 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:08.842 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd reweight 7 1 2024-09-10T18:51:08.921 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:08.944 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:08.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:08.945 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:09.046 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:09.070 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:09.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:09.071 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:09.171 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:09.187 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-10T18:51:09.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:09.198 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:09.199 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:09.299 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:09.324 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:09.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:09.325 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:09.426 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:09.452 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:09.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:09.453 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:09.553 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:09.577 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:09.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:09.578 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:09.679 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:09.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:09 smithi018 ceph-mon[29336]: pgmap v237: 158 pgs: 158 active+clean; 687 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 0 B/s wr, 1.08k op/s 2024-09-10T18:51:09.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:09 smithi018 ceph-mon[40196]: pgmap v237: 158 pgs: 158 active+clean; 687 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 0 B/s wr, 1.08k op/s 2024-09-10T18:51:09.703 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:09.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:09.704 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:09.805 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:09.828 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:09.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:09.829 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:09.930 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:09.953 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:09.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:09.954 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:10.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:09 smithi103 ceph-mon[33163]: pgmap v237: 158 pgs: 158 active+clean; 687 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 0 B/s wr, 1.08k op/s 2024-09-10T18:51:10.056 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:10.082 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:10.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:10.083 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:10.183 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:10.206 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:10.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:10.207 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:10.308 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:10.333 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:10.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:10.334 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:10.435 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:10.459 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:10.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:10.460 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:10.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:51:10 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:51:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:51:10.560 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:10.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:10.585 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:10.585 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:10.609 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:51:10.610 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:51:10.624 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:51:10.624 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:51:10.639 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:51:10.639 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:51:10.639 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:10 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:51:10.639 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:10 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:51:10.640 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:10 smithi018 ceph-mon[40196]: osdmap e148: 8 total, 8 up, 7 in 2024-09-10T18:51:10.640 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:10 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:51:10.640 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:10 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:51:10.640 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:10 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:51:10.640 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:10 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:51:10.640 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:10 smithi018 ceph-mon[29336]: osdmap e148: 8 total, 8 up, 7 in 2024-09-10T18:51:10.640 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:10 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:51:10.640 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:10 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:51:10.654 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:51:10.655 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:51:10.670 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:51:10.670 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:51:10.686 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:10.687 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:51:10.687 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:51:10.702 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:51:10.702 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:51:10.708 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:10.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:10.709 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:10.718 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:51:10.718 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:51:10.810 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:10.833 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:10.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:10.834 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:10.934 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:10.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:10.958 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:10.958 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:11.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:10 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:51:11.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:10 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:51:11.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:10 smithi103 ceph-mon[33163]: osdmap e148: 8 total, 8 up, 7 in 2024-09-10T18:51:11.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:10 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:51:11.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:10 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:51:11.059 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:11.083 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:11.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:11.084 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:11.184 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:11.210 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:11.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:11.211 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:11.235 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:51:10 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:51:10] "GET /metrics HTTP/1.1" 200 223777 "" "Prometheus/2.43.0" 2024-09-10T18:51:11.312 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:11.334 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:11.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:11.335 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:11.436 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:11.462 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:11.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:11.463 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:11.564 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:11.580 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_0 2024-09-10T18:51:11.580 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2024-09-10T18:51:11.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:11.587 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:11.587 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:11.608 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:51:11.609 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:51:11.688 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:11.711 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:11.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:11.712 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:11.813 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:11.835 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:11.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:11.836 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:11.937 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:11.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:11 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/4228159494' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:51:11.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:11 smithi018 ceph-mon[29336]: pgmap v239: 158 pgs: 158 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 0 B/s wr, 1.01k op/s 2024-09-10T18:51:11.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:11 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/4228159494' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:51:11.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:11 smithi018 ceph-mon[40196]: pgmap v239: 158 pgs: 158 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 0 B/s wr, 1.01k op/s 2024-09-10T18:51:11.962 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:11.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:11.963 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:12.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:11 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/4228159494' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:51:12.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:11 smithi103 ceph-mon[33163]: pgmap v239: 158 pgs: 158 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 0 B/s wr, 1.01k op/s 2024-09-10T18:51:12.064 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:12.091 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:12.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:12.092 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:12.193 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:12.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:12.220 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:12.220 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:12.321 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:12.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:12.348 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:12.349 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:12.420 INFO:teuthology.orchestra.run.smithi018.stdout:ERROR: (22) Invalid argument 2024-09-10T18:51:12.420 INFO:teuthology.orchestra.run.smithi018.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:51:12.450 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:12.473 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:12.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:12.474 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:12.575 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:12.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:12.601 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:12.601 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:12.702 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:12.727 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:12.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:12.728 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:12.829 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:12.854 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:12.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:12.855 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:12.956 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:12.980 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:12.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:12.981 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:13.082 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:13.109 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:13.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:13.112 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:13.213 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:13.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:13.240 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:13.240 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:13.341 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:13.368 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:13.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:13.369 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:13.470 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:13.495 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:13.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:13.496 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:13.597 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:13.624 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:13.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:13.625 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:13.725 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:13.752 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:13.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:13.753 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:13.854 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:13.880 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:13.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:13.881 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:13.982 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:14.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:14.008 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:14.008 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:14.106 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T18:51:14.109 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:14.136 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:14.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:14.137 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:14.237 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:14.260 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:14.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:14.261 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:14.362 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:14.383 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:14.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:14.384 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:14.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:14 smithi018 ceph-mon[29336]: pgmap v240: 158 pgs: 158 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 0 B/s wr, 809 op/s 2024-09-10T18:51:14.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:14 smithi018 ceph-mon[29336]: osdmap e149: 8 total, 8 up, 7 in 2024-09-10T18:51:14.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:14 smithi018 ceph-mon[40196]: pgmap v240: 158 pgs: 158 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 0 B/s wr, 809 op/s 2024-09-10T18:51:14.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:14 smithi018 ceph-mon[40196]: osdmap e149: 8 total, 8 up, 7 in 2024-09-10T18:51:14.485 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:14.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:14.512 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:14.512 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:14.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:14 smithi103 ceph-mon[33163]: pgmap v240: 158 pgs: 158 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 0 B/s wr, 809 op/s 2024-09-10T18:51:14.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:14 smithi103 ceph-mon[33163]: osdmap e149: 8 total, 8 up, 7 in 2024-09-10T18:51:14.613 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:14.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:14.636 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:14.636 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:14.737 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:14.760 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:14.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:14.761 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:14.862 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:14.884 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:14.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:14.885 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:14.986 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:15.011 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:15.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:15.012 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:15.113 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:15.135 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:15.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:15.136 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:15.237 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:15.259 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:15.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:15.260 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:15.361 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:15.386 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:15.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:15.387 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:15.488 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:15.514 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:15.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:15.515 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:15.525 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:51:15.615 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:15.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:15.641 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:15.641 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:15.744 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:15.767 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:15.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:15.768 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:15.869 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:15.894 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:15.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:15.896 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:15.913 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-10T18:51:15.996 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:16.021 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:16.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:16.022 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:16.123 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:16.146 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:16.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:16.147 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:16.248 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:16.273 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:16.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:16.274 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:16.374 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:16.398 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:16.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:16.399 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:16.413 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:16 smithi018 ceph-mon[29336]: pgmap v242: 158 pgs: 158 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:51:16.414 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:16 smithi018 ceph-mon[40196]: pgmap v242: 158 pgs: 158 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:51:16.500 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:16.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:16 smithi103 ceph-mon[33163]: pgmap v242: 158 pgs: 158 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:51:16.523 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:51:16 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:51:16.524 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:16.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:16.526 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:16.627 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:16.648 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:16.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:16.649 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:16.750 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:16.773 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:16.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:16.774 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:16.874 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:16.897 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:16.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:16.898 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:16.999 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:17.020 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:17.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:17.022 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:17.122 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:17.149 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:17.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:17.150 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:17.224 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is set 2024-09-10T18:51:17.251 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:17.275 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:17.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:17.276 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:17.377 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:17.401 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:17.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:17.402 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:17.408 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:17 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:17.408 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:17 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/4143664192' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:51:17.409 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:17 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:17.409 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:17 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/4143664192' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:51:17.502 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:17.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:17 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:17.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:17 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/4143664192' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:51:17.526 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:17.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:17.527 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:17.628 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:17.654 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:17.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:17.655 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:17.755 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:17.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:17.780 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:17.780 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:17.881 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:17.907 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:17.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:17.908 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:18.009 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:18.032 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:18.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:18.034 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:18.134 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:18.157 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:18.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:18.158 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:18.259 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:18.283 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:18.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:18.284 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:18.385 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:18.411 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:18.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:18.412 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:18.513 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:18.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:18 smithi103 ceph-mon[33163]: pgmap v243: 158 pgs: 158 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:18.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:18 smithi103 ceph-mon[33163]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:51:18.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:18 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/4143664192' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:51:18.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:18 smithi103 ceph-mon[33163]: osdmap e150: 8 total, 8 up, 7 in 2024-09-10T18:51:18.540 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:18.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:18.541 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:18.642 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:18.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:18.668 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:18.668 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:18.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:18 smithi018 ceph-mon[29336]: pgmap v243: 158 pgs: 158 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:18.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:18 smithi018 ceph-mon[29336]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:51:18.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:18 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/4143664192' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:51:18.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:18 smithi018 ceph-mon[29336]: osdmap e150: 8 total, 8 up, 7 in 2024-09-10T18:51:18.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:18 smithi018 ceph-mon[40196]: pgmap v243: 158 pgs: 158 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:18.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:18 smithi018 ceph-mon[40196]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:51:18.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:18 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/4143664192' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:51:18.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:18 smithi018 ceph-mon[40196]: osdmap e150: 8 total, 8 up, 7 in 2024-09-10T18:51:18.769 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:18.795 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:18.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:18.796 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:18.897 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:18.922 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:18.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:18.923 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:19.024 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:19.050 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:19.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:19.051 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:19.150 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:51:19.151 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:51:19.152 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:19.181 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:19.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:19.182 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:19.283 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:19.308 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:19.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:19.309 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:19.410 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:19.432 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:19.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:19.433 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:19.534 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:19.550 INFO:teuthology.orchestra.run.smithi018.stderr:reweighted osd.7 to 1 (10000) 2024-09-10T18:51:19.560 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:19.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:19.562 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:19.663 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:19.688 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:19.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:19.690 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:19.790 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:19.818 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:19.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:19.819 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:19.834 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:19 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:51:19.834 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:19 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:51:19.834 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:19 smithi018 ceph-mon[40196]: osdmap e151: 8 total, 8 up, 7 in 2024-09-10T18:51:19.834 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:19 smithi018 ceph-mon[40196]: pgmap v246: 158 pgs: 158 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:51:19.834 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:19 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/4166892337' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-10T18:51:19.834 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:19 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-10T18:51:19.834 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:19 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:51:19.834 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:19 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:51:19.834 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:19 smithi018 ceph-mon[29336]: osdmap e151: 8 total, 8 up, 7 in 2024-09-10T18:51:19.835 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:19 smithi018 ceph-mon[29336]: pgmap v246: 158 pgs: 158 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:51:19.835 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:19 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/4166892337' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-10T18:51:19.835 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:19 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-10T18:51:19.919 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:19.943 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:19.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:19.944 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:20.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:19 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:51:20.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:19 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:51:20.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:19 smithi103 ceph-mon[33163]: osdmap e151: 8 total, 8 up, 7 in 2024-09-10T18:51:20.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:19 smithi103 ceph-mon[33163]: pgmap v246: 158 pgs: 158 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:51:20.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:19 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/4166892337' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-10T18:51:20.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:19 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-10T18:51:20.045 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:20.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:20.071 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:20.071 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:20.172 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:20.197 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:20.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:20.198 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:20.299 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:20.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:20.325 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:20.325 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:20.426 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:20.452 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:20.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:20.453 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:20.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:51:20 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:51:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:51:20.554 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:20.578 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:20.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:20.579 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:20.680 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:20.703 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:20.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:20.704 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:20.805 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:20.828 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:20.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:20.829 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:20.930 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:20.938 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-10T18:51:20.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:20 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-10T18:51:20.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:20 smithi018 ceph-mon[29336]: osdmap e152: 8 total, 8 up, 7 in 2024-09-10T18:51:20.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:20 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-10T18:51:20.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:20 smithi018 ceph-mon[40196]: osdmap e152: 8 total, 8 up, 7 in 2024-09-10T18:51:20.952 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:20.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:20.953 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:21.025 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:20 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-10T18:51:21.025 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:20 smithi103 ceph-mon[33163]: osdmap e152: 8 total, 8 up, 7 in 2024-09-10T18:51:21.054 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:21.076 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:21.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:21.080 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:21.181 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:21.203 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:21.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:21.205 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:21.238 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:51:20 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:51:20] "GET /metrics HTTP/1.1" 200 223777 "" "Prometheus/2.43.0" 2024-09-10T18:51:21.305 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:21.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:21.329 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:21.329 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:21.430 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:21.453 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:21.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:21.454 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:21.555 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:21.578 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:21.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:21.579 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:21.679 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:21.702 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:21.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:21.703 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:21.804 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:21.827 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:21.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:21.828 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:21.854 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:21 smithi018 ceph-mon[29336]: pgmap v248: 158 pgs: 158 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:51:21.854 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:21 smithi018 ceph-mon[40196]: pgmap v248: 158 pgs: 158 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:51:21.929 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:21.950 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:21.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:21.951 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:22.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:21 smithi103 ceph-mon[33163]: pgmap v248: 158 pgs: 158 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:51:22.052 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:22.075 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:22.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:22.076 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:22.176 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:51:22.177 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:51:22.177 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:22.194 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:51:22.194 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:51:22.200 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:22.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:22.202 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:22.209 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:51:22.209 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:51:22.226 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:51:22.226 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:51:22.242 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:51:22.243 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:51:22.258 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:51:22.259 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:51:22.274 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:51:22.275 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:51:22.291 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:51:22.291 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:51:22.305 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:22.326 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:22.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:22.327 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:22.428 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:22.450 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:22.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:22.451 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:22.552 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:22.575 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:22.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:22.589 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:22.690 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:22.712 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:22.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:22.713 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:22.812 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:22 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:51:22.813 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:22 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:51:22.813 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:22 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:51:22.813 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:22 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:51:22.813 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:22 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:51:22.813 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:22 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1747463088' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:51:22.813 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:22 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:22.813 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:22 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:51:22.813 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:22 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:51:22.813 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:22 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:51:22.814 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:22 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:51:22.814 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:22 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:51:22.814 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:22 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:51:22.814 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:22 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:51:22.814 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:22 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1747463088' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:51:22.814 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:22 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:22.814 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:22 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:51:22.814 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:22 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:51:22.814 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:22.837 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:22.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:22.838 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:22.938 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:22.959 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:22.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:22.960 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:23.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:22 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:51:23.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:22 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:51:23.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:22 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:51:23.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:22 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:51:23.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:22 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:51:23.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:22 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1747463088' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:51:23.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:22 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:23.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:22 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:51:23.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:22 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:51:23.061 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:23.085 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:23.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:23.086 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:23.187 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:23.211 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:23.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:23.212 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:23.313 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:23.336 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:23.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:23.337 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:23.438 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:23.460 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:23.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:23.461 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:23.550 INFO:teuthology.orchestra.run.smithi018.stderr:pool 'unique_pool_0' removed 2024-09-10T18:51:23.562 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:23.614 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:23.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:23.616 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:23.716 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:23.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:23.740 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:23.740 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:23.841 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:23.864 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:23.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:23.865 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:23.869 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:23 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/291238726' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:51:23.869 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:23 smithi018 ceph-mon[40196]: pgmap v249: 158 pgs: 158 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:51:23.869 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:23 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/291238726' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T18:51:23.870 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:23 smithi018 ceph-mon[40196]: osdmap e153: 8 total, 8 up, 7 in 2024-09-10T18:51:23.870 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:23 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/291238726' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:51:23.870 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:23 smithi018 ceph-mon[29336]: pgmap v249: 158 pgs: 158 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:51:23.870 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:23 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/291238726' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T18:51:23.870 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:23 smithi018 ceph-mon[29336]: osdmap e153: 8 total, 8 up, 7 in 2024-09-10T18:51:23.966 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:23.996 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:23.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:23.997 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:24.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:23 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/291238726' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:51:24.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:23 smithi103 ceph-mon[33163]: pgmap v249: 158 pgs: 158 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:51:24.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:23 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/291238726' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T18:51:24.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:23 smithi103 ceph-mon[33163]: osdmap e153: 8 total, 8 up, 7 in 2024-09-10T18:51:24.098 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:24.120 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:24.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:24.121 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:24.222 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:24.243 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:24.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:24.244 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:24.345 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:24.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:24.367 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:24.368 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:24.468 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:24.492 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:24.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:24.493 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:24.594 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:24.615 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:24.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:24.617 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:24.717 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:24.739 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:24.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:24.740 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:24.841 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:24.865 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:24.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:24.866 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:24.967 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:24.991 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:24.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:24.992 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:25.092 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:25.094 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-10T18:51:25.095 INFO:tasks.radosbench:Beginning radosbench... 2024-09-10T18:51:25.095 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_1 2024-09-10T18:51:25.095 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd pool create unique_pool_1 16 2024-09-10T18:51:25.156 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:25.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:25.157 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:25.258 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:25.280 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:25.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:25.281 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:25.382 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:25.404 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:25.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:25.405 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:25.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:25 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:51:25.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:25 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:51:25.505 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:25.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:25 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:51:25.528 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:25.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:25.529 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:25.597 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T18:51:25.630 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:25.663 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:25.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:25.664 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:25.736 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-10T18:51:25.737 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T18:51:25.765 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:25.803 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:25.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:25.804 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:25.905 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:25.928 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:25.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:25.929 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:26.030 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:26.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:26.057 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:26.057 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:26.158 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:26.183 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:26.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:26.184 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:26.213 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:26 smithi018 ceph-mon[40196]: pgmap v251: 142 pgs: 142 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 621 B/s rd, 0 op/s 2024-09-10T18:51:26.213 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:26 smithi018 ceph-mon[29336]: pgmap v251: 142 pgs: 142 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 621 B/s rd, 0 op/s 2024-09-10T18:51:26.243 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T18:51:26.243 INFO:teuthology.orchestra.run.smithi018.stdout: "ops": [], 2024-09-10T18:51:26.243 INFO:teuthology.orchestra.run.smithi018.stdout: "complaint_time": 30, 2024-09-10T18:51:26.243 INFO:teuthology.orchestra.run.smithi018.stdout: "num_blocked_ops": 0 2024-09-10T18:51:26.243 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T18:51:26.285 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:26.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:26.309 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:26.310 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:26.411 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:26.434 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:26.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:26.435 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:26.440 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:51:26.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:26 smithi103 ceph-mon[33163]: pgmap v251: 142 pgs: 142 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 621 B/s rd, 0 op/s 2024-09-10T18:51:26.523 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:51:26 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:51:26.536 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:26.562 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:26.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:26.563 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:26.664 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:26.689 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:26.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:26.690 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:26.791 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:26.815 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:26.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:26.816 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:26.917 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:26.943 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:26.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:26.944 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:26.995 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-10T18:51:27.009 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:51:27.009 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:51:27.026 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:51:27.044 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:27.070 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:27.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:27.071 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:27.171 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:27.194 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:27.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:27.195 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:27.268 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:27 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:51:27.268 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:27 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:51:27.268 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:27 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:27.268 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:27 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:51:27.269 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:27 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:51:27.269 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:27 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:27.296 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:27.320 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:27.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:27.321 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:27.421 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:27.447 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:27.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:27.448 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:27.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:27 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:51:27.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:27 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:51:27.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:27 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:27.550 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:27.573 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:27.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:27.574 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:27.675 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:27.699 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:27.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:27.700 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:27.801 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:27.827 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:27.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:27.828 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:27.929 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:27.955 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:27.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:27.956 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:28.056 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:28.081 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:28.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:28.082 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:28.183 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:28.206 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:28.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:28.207 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:28.308 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:28.335 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:28.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:28.336 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:28.395 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:28 smithi018 ceph-mon[29336]: osdmap e154: 8 total, 8 up, 7 in 2024-09-10T18:51:28.395 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:28 smithi018 ceph-mon[29336]: pgmap v253: 142 pgs: 142 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:28.396 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:28 smithi018 ceph-mon[40196]: osdmap e154: 8 total, 8 up, 7 in 2024-09-10T18:51:28.396 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:28 smithi018 ceph-mon[40196]: pgmap v253: 142 pgs: 142 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:28.436 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:28.458 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:28.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:28.459 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:28.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:28 smithi103 ceph-mon[33163]: osdmap e154: 8 total, 8 up, 7 in 2024-09-10T18:51:28.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:28 smithi103 ceph-mon[33163]: pgmap v253: 142 pgs: 142 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:28.560 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:28.582 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:28.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:28.583 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:28.684 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:28.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:28.711 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:28.711 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:28.812 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:28.835 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:28.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:28.836 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:28.937 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:28.959 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:28.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:28.960 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:29.061 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:29.084 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:29.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:29.085 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:29.186 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:29.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:29.209 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:29.209 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:29.310 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:29.337 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:29.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:29.338 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:29.439 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:29.462 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:29.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:29.463 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:29.552 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is unset 2024-09-10T18:51:29.564 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:29.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:29.588 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:29.588 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:29.689 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:29.711 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:29.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:29.712 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:29.813 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:29.822 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:29 smithi018 ceph-mon[29336]: osdmap e155: 8 total, 8 up, 7 in 2024-09-10T18:51:29.822 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:29 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1429952906' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:51:29.822 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:29 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:51:29.822 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:29 smithi018 ceph-mon[29336]: pgmap v255: 142 pgs: 142 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 682 B/s rd, 0 op/s 2024-09-10T18:51:29.823 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:29 smithi018 ceph-mon[40196]: osdmap e155: 8 total, 8 up, 7 in 2024-09-10T18:51:29.823 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:29 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1429952906' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:51:29.823 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:29 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:51:29.823 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:29 smithi018 ceph-mon[40196]: pgmap v255: 142 pgs: 142 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 682 B/s rd, 0 op/s 2024-09-10T18:51:29.840 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:29.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:29.840 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:29.941 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:29.967 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:29.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:29.968 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:30.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:29 smithi103 ceph-mon[33163]: osdmap e155: 8 total, 8 up, 7 in 2024-09-10T18:51:30.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:29 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1429952906' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:51:30.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:29 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:51:30.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:29 smithi103 ceph-mon[33163]: pgmap v255: 142 pgs: 142 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 682 B/s rd, 0 op/s 2024-09-10T18:51:30.069 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:30.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:30.094 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:30.094 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:30.195 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:30.218 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:30.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:30.219 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:30.320 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:30.344 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:30.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:30.346 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:30.446 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:30.470 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:30.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:30.471 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:30.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:51:30 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:51:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:51:30.573 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:30.596 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:30.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:30.597 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:30.698 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:30.722 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:30.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:30.723 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:30.824 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:30.847 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:30.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:30.849 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:30.897 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:30 smithi018 ceph-mon[40196]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:51:30.898 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:30 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:51:30.898 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:30 smithi018 ceph-mon[40196]: osdmap e156: 8 total, 8 up, 7 in 2024-09-10T18:51:30.898 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:30 smithi018 ceph-mon[29336]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:51:30.898 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:30 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:51:30.898 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:30 smithi018 ceph-mon[29336]: osdmap e156: 8 total, 8 up, 7 in 2024-09-10T18:51:30.950 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:30.972 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:30.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:30.973 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:31.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:30 smithi103 ceph-mon[33163]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:51:31.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:30 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:51:31.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:30 smithi103 ceph-mon[33163]: osdmap e156: 8 total, 8 up, 7 in 2024-09-10T18:51:31.074 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:31.099 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:31.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:31.100 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:31.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:51:30 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:51:30] "GET /metrics HTTP/1.1" 200 221384 "" "Prometheus/2.43.0" 2024-09-10T18:51:31.200 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:31.223 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:31.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:31.224 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:31.325 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:31.350 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:31.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:31.351 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:31.452 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:31.476 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:31.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:31.477 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:31.578 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:31.601 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:31.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:31.602 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:31.703 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:31.727 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:31.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:31.728 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:31.829 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:31.857 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:31.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:31.858 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:31.861 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:31 smithi018 ceph-mon[40196]: 2.0 scrub starts 2024-09-10T18:51:31.861 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:31 smithi018 ceph-mon[40196]: 2.0 scrub ok 2024-09-10T18:51:31.861 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:31 smithi018 ceph-mon[40196]: pgmap v257: 142 pgs: 142 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:51:31.861 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:31 smithi018 ceph-mon[29336]: 2.0 scrub starts 2024-09-10T18:51:31.862 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:31 smithi018 ceph-mon[29336]: 2.0 scrub ok 2024-09-10T18:51:31.862 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:31 smithi018 ceph-mon[29336]: pgmap v257: 142 pgs: 142 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:51:31.959 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:31.986 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:31.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:31.987 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:32.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:31 smithi103 ceph-mon[33163]: 2.0 scrub starts 2024-09-10T18:51:32.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:31 smithi103 ceph-mon[33163]: 2.0 scrub ok 2024-09-10T18:51:32.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:31 smithi103 ceph-mon[33163]: pgmap v257: 142 pgs: 142 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:51:32.088 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:32.118 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:51:32.119 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:32.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:32.120 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:32.221 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:32.247 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:32.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:32.248 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:32.348 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:32.373 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:32.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:32.374 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:32.475 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:32.497 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:32.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:32.498 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:32.599 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:32.624 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:32.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:32.625 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:32.726 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:32.752 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:32.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:32.753 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:32.802 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:32 smithi018 ceph-mon[40196]: 2.d scrub starts 2024-09-10T18:51:32.802 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:32 smithi018 ceph-mon[40196]: 2.d scrub ok 2024-09-10T18:51:32.802 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:32 smithi018 ceph-mon[40196]: 4.1d scrub starts 2024-09-10T18:51:32.803 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:32 smithi018 ceph-mon[40196]: 4.1d scrub ok 2024-09-10T18:51:32.803 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:32 smithi018 ceph-mon[40196]: 4.1a scrub starts 2024-09-10T18:51:32.803 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:32 smithi018 ceph-mon[40196]: 4.1a scrub ok 2024-09-10T18:51:32.803 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:32 smithi018 ceph-mon[29336]: 2.d scrub starts 2024-09-10T18:51:32.803 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:32 smithi018 ceph-mon[29336]: 2.d scrub ok 2024-09-10T18:51:32.803 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:32 smithi018 ceph-mon[29336]: 4.1d scrub starts 2024-09-10T18:51:32.803 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:32 smithi018 ceph-mon[29336]: 4.1d scrub ok 2024-09-10T18:51:32.803 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:32 smithi018 ceph-mon[29336]: 4.1a scrub starts 2024-09-10T18:51:32.803 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:32 smithi018 ceph-mon[29336]: 4.1a scrub ok 2024-09-10T18:51:32.854 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:32.878 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:32.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:32.880 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:32.980 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:33.005 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:33.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:33.006 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:33.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:32 smithi103 ceph-mon[33163]: 2.d scrub starts 2024-09-10T18:51:33.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:32 smithi103 ceph-mon[33163]: 2.d scrub ok 2024-09-10T18:51:33.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:32 smithi103 ceph-mon[33163]: 4.1d scrub starts 2024-09-10T18:51:33.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:32 smithi103 ceph-mon[33163]: 4.1d scrub ok 2024-09-10T18:51:33.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:32 smithi103 ceph-mon[33163]: 4.1a scrub starts 2024-09-10T18:51:33.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:32 smithi103 ceph-mon[33163]: 4.1a scrub ok 2024-09-10T18:51:33.107 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:33.132 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:33.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:33.133 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:33.234 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:33.259 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:51:33.261 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-09-10T18:51:33.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:33.264 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:33.264 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:33.365 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:33.388 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:33.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:33.389 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:33.490 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:33.513 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:33.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:33.515 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:33.615 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:33.637 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:33.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:33.638 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:33.739 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:33.765 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:33.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:33.766 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:33.809 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:33 smithi018 ceph-mon[40196]: 4.12 scrub starts 2024-09-10T18:51:33.809 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:33 smithi018 ceph-mon[40196]: 4.12 scrub ok 2024-09-10T18:51:33.809 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:33 smithi018 ceph-mon[40196]: 5.16 scrub starts 2024-09-10T18:51:33.809 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:33 smithi018 ceph-mon[40196]: 5.16 scrub ok 2024-09-10T18:51:33.809 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:33 smithi018 ceph-mon[40196]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:51:33.809 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:33 smithi018 ceph-mon[40196]: pgmap v258: 142 pgs: 142 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:51:33.809 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:33 smithi018 ceph-mon[40196]: osdmap e157: 8 total, 8 up, 7 in 2024-09-10T18:51:33.810 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:33 smithi018 ceph-mon[29336]: 4.12 scrub starts 2024-09-10T18:51:33.810 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:33 smithi018 ceph-mon[29336]: 4.12 scrub ok 2024-09-10T18:51:33.810 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:33 smithi018 ceph-mon[29336]: 5.16 scrub starts 2024-09-10T18:51:33.810 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:33 smithi018 ceph-mon[29336]: 5.16 scrub ok 2024-09-10T18:51:33.810 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:33 smithi018 ceph-mon[29336]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:51:33.810 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:33 smithi018 ceph-mon[29336]: pgmap v258: 142 pgs: 142 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:51:33.810 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:33 smithi018 ceph-mon[29336]: osdmap e157: 8 total, 8 up, 7 in 2024-09-10T18:51:33.867 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:33.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:33.893 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:33.893 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:33.994 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:34.017 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:34.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:34.018 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:34.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:33 smithi103 ceph-mon[33163]: 4.12 scrub starts 2024-09-10T18:51:34.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:33 smithi103 ceph-mon[33163]: 4.12 scrub ok 2024-09-10T18:51:34.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:33 smithi103 ceph-mon[33163]: 5.16 scrub starts 2024-09-10T18:51:34.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:33 smithi103 ceph-mon[33163]: 5.16 scrub ok 2024-09-10T18:51:34.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:33 smithi103 ceph-mon[33163]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:51:34.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:33 smithi103 ceph-mon[33163]: pgmap v258: 142 pgs: 142 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:51:34.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:33 smithi103 ceph-mon[33163]: osdmap e157: 8 total, 8 up, 7 in 2024-09-10T18:51:34.119 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:34.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:34.145 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:34.146 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:34.247 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:34.269 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:34.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:34.271 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:34.371 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:34.394 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:34.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:34.395 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:34.496 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:34.520 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:34.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:34.521 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:34.622 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:34.645 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:34.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:34.646 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:34.747 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:34.770 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:34.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:34.771 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:34.872 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:34.875 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:34 smithi018 ceph-mon[40196]: 5.1a scrub starts 2024-09-10T18:51:34.876 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:34 smithi018 ceph-mon[40196]: 5.1a scrub ok 2024-09-10T18:51:34.876 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:34 smithi018 ceph-mon[29336]: 5.1a scrub starts 2024-09-10T18:51:34.876 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:34 smithi018 ceph-mon[29336]: 5.1a scrub ok 2024-09-10T18:51:34.898 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:34.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:34.899 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:35.001 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:35.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:34 smithi103 ceph-mon[33163]: 5.1a scrub starts 2024-09-10T18:51:35.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:34 smithi103 ceph-mon[33163]: 5.1a scrub ok 2024-09-10T18:51:35.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:35.028 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:35.028 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:35.129 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:35.155 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:35.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:35.156 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:35.257 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:35.280 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:35.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:35.281 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:35.382 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:35.406 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:35.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:35.407 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:35.508 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:35.531 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:35.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:35.533 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:35.630 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2024-09-10T18:51:31.133254+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":703528,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":1104,"num_read_kb":876,"num_write":450,"num_write_kb":339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":217051,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1428,"ondisk_log_size":1428,"up":426,"acting":426,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":437,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":2136252,"kb_used_data":8988,"kb_used_omap":11,"kb_used_meta":2127156,"kb_avail":654079812,"statfs":{"total":671965249536,"available":669777727488,"internally_reserved":0,"allocated":9203712,"data_stored":5600016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11928,"internal_metadata":2178208104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001016"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":130,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:20.126996+0000","last_change":"2024-09-10T18:50:55.243820+0000","last_active":"2024-09-10T18:51:20.126996+0000","last_peered":"2024-09-10T18:51:20.126996+0000","last_clean":"2024-09-10T18:51:20.126996+0000","last_became_active":"2024-09-10T18:45:24.265488+0000","last_became_peered":"2024-09-10T18:45:24.265488+0000","last_unstale":"2024-09-10T18:51:20.126996+0000","last_undegraded":"2024-09-10T18:51:20.126996+0000","last_fullsized":"2024-09-10T18:51:20.126996+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:55.243629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:50:55.243629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:57.394455+0000","scrub_duration":0.0090834450000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":136,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:19.549408+0000","last_change":"2024-09-10T18:50:52.829851+0000","last_active":"2024-09-10T18:51:19.549408+0000","last_peered":"2024-09-10T18:51:19.549408+0000","last_clean":"2024-09-10T18:51:19.549408+0000","last_became_active":"2024-09-10T18:45:17.959475+0000","last_became_peered":"2024-09-10T18:45:17.959475+0000","last_unstale":"2024-09-10T18:51:19.549408+0000","last_undegraded":"2024-09-10T18:51:19.549408+0000","last_fullsized":"2024-09-10T18:51:19.549408+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:52.829778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.718270+0000","last_clean_scrub_stamp":"2024-09-10T18:50:52.829778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:54.534366+0000","scrub_duration":0.0095184889999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"63'15","reported_seq":175,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:20.127187+0000","last_change":"2024-09-10T18:49:02.544635+0000","last_active":"2024-09-10T18:51:20.127187+0000","last_peered":"2024-09-10T18:51:20.127187+0000","last_clean":"2024-09-10T18:51:20.127187+0000","last_became_active":"2024-09-10T18:47:36.413635+0000","last_became_peered":"2024-09-10T18:47:36.413635+0000","last_unstale":"2024-09-10T18:51:20.127187+0000","last_undegraded":"2024-09-10T18:51:20.127187+0000","last_fullsized":"2024-09-10T18:51:20.127187+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:49:02.544557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:49:02.544557+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0033455720000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":134,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.975311+0000","last_change":"2024-09-10T18:50:50.546527+0000","last_active":"2024-09-10T18:51:28.975311+0000","last_peered":"2024-09-10T18:51:28.975311+0000","last_clean":"2024-09-10T18:51:28.975311+0000","last_became_active":"2024-09-10T18:45:22.065619+0000","last_became_peered":"2024-09-10T18:45:22.065619+0000","last_unstale":"2024-09-10T18:51:28.975311+0000","last_undegraded":"2024-09-10T18:51:28.975311+0000","last_fullsized":"2024-09-10T18:51:28.975311+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:50.546473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.057932+0000","last_clean_scrub_stamp":"2024-09-10T18:50:50.546473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:16.728084+0000","scrub_duration":0.0026598759999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":136,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:20.127813+0000","last_change":"2024-09-10T18:50:58.174634+0000","last_active":"2024-09-10T18:51:20.127813+0000","last_peered":"2024-09-10T18:51:20.127813+0000","last_clean":"2024-09-10T18:51:20.127813+0000","last_became_active":"2024-09-10T18:45:17.966153+0000","last_became_peered":"2024-09-10T18:45:17.966153+0000","last_unstale":"2024-09-10T18:51:20.127813+0000","last_undegraded":"2024-09-10T18:51:20.127813+0000","last_fullsized":"2024-09-10T18:51:20.127813+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:58.174399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:58.174399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:09.810946+0000","scrub_duration":0.012266312,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"83'13","reported_seq":146,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.975406+0000","last_change":"2024-09-10T18:50:04.577468+0000","last_active":"2024-09-10T18:51:28.975406+0000","last_peered":"2024-09-10T18:51:28.975406+0000","last_clean":"2024-09-10T18:51:28.975406+0000","last_became_active":"2024-09-10T18:45:20.061266+0000","last_became_peered":"2024-09-10T18:45:20.061266+0000","last_unstale":"2024-09-10T18:51:28.975406+0000","last_undegraded":"2024-09-10T18:51:28.975406+0000","last_fullsized":"2024-09-10T18:51:28.975406+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'13","last_scrub_stamp":"2024-09-10T18:50:04.577301+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_clean_scrub_stamp":"2024-09-10T18:50:04.577301+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004842224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":122,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:19.547883+0000","last_change":"2024-09-10T18:49:52.335982+0000","last_active":"2024-09-10T18:51:19.547883+0000","last_peered":"2024-09-10T18:51:19.547883+0000","last_clean":"2024-09-10T18:51:19.547883+0000","last_became_active":"2024-09-10T18:45:22.072822+0000","last_became_peered":"2024-09-10T18:45:22.072822+0000","last_unstale":"2024-09-10T18:51:19.547883+0000","last_undegraded":"2024-09-10T18:51:19.547883+0000","last_fullsized":"2024-09-10T18:51:19.547883+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:52.335893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:49:52.335893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.018709007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":122,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.976085+0000","last_change":"2024-09-10T18:49:53.592412+0000","last_active":"2024-09-10T18:51:28.976085+0000","last_peered":"2024-09-10T18:51:28.976085+0000","last_clean":"2024-09-10T18:51:28.976085+0000","last_became_active":"2024-09-10T18:45:24.081002+0000","last_became_peered":"2024-09-10T18:45:24.081002+0000","last_unstale":"2024-09-10T18:51:28.976085+0000","last_undegraded":"2024-09-10T18:51:28.976085+0000","last_fullsized":"2024-09-10T18:51:28.976085+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:53.592325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:53.592325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040599850000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":139,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.977026+0000","last_change":"2024-09-10T18:49:53.263097+0000","last_active":"2024-09-10T18:51:28.977026+0000","last_peered":"2024-09-10T18:51:28.977026+0000","last_clean":"2024-09-10T18:51:28.977026+0000","last_became_active":"2024-09-10T18:45:17.961097+0000","last_became_peered":"2024-09-10T18:45:17.961097+0000","last_unstale":"2024-09-10T18:51:28.977026+0000","last_undegraded":"2024-09-10T18:51:28.977026+0000","last_fullsized":"2024-09-10T18:51:28.977026+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:53.263009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:53.263009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0088114549999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"63'5","reported_seq":136,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.975500+0000","last_change":"2024-09-10T18:50:01.591311+0000","last_active":"2024-09-10T18:51:28.975500+0000","last_peered":"2024-09-10T18:51:28.975500+0000","last_clean":"2024-09-10T18:51:28.975500+0000","last_became_active":"2024-09-10T18:45:20.057567+0000","last_became_peered":"2024-09-10T18:45:20.057567+0000","last_unstale":"2024-09-10T18:51:28.975500+0000","last_undegraded":"2024-09-10T18:51:28.975500+0000","last_fullsized":"2024-09-10T18:51:28.975500+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-10T18:50:01.591224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:50:01.591224+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020612410000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":135,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.976145+0000","last_change":"2024-09-10T18:49:19.568881+0000","last_active":"2024-09-10T18:51:28.976145+0000","last_peered":"2024-09-10T18:51:28.976145+0000","last_clean":"2024-09-10T18:51:28.976145+0000","last_became_active":"2024-09-10T18:45:22.076575+0000","last_became_peered":"2024-09-10T18:45:22.076575+0000","last_unstale":"2024-09-10T18:51:28.976145+0000","last_undegraded":"2024-09-10T18:51:28.976145+0000","last_fullsized":"2024-09-10T18:51:28.976145+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:19.568791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:36.509971+0000","last_clean_scrub_stamp":"2024-09-10T18:49:19.568791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013364760000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":132,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.542811+0000","last_change":"2024-09-10T18:49:57.067679+0000","last_active":"2024-09-10T18:51:28.542811+0000","last_peered":"2024-09-10T18:51:28.542811+0000","last_clean":"2024-09-10T18:51:28.542811+0000","last_became_active":"2024-09-10T18:45:24.079534+0000","last_became_peered":"2024-09-10T18:45:24.079534+0000","last_unstale":"2024-09-10T18:51:28.542811+0000","last_undegraded":"2024-09-10T18:51:28.542811+0000","last_fullsized":"2024-09-10T18:51:28.542811+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:57.067604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:57.067604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016642440000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":149,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.977099+0000","last_change":"2024-09-10T18:50:58.526247+0000","last_active":"2024-09-10T18:51:28.977099+0000","last_peered":"2024-09-10T18:51:28.977099+0000","last_clean":"2024-09-10T18:51:28.977099+0000","last_became_active":"2024-09-10T18:47:36.414393+0000","last_became_peered":"2024-09-10T18:47:36.414393+0000","last_unstale":"2024-09-10T18:51:28.977099+0000","last_undegraded":"2024-09-10T18:51:28.977099+0000","last_fullsized":"2024-09-10T18:51:28.977099+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:58.526191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:58.526191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:06.694976+0000","scrub_duration":0.0021021659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"63'9","reported_seq":142,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.976403+0000","last_change":"2024-09-10T18:49:23.482901+0000","last_active":"2024-09-10T18:51:28.976403+0000","last_peered":"2024-09-10T18:51:28.976403+0000","last_clean":"2024-09-10T18:51:28.976403+0000","last_became_active":"2024-09-10T18:45:20.052652+0000","last_became_peered":"2024-09-10T18:45:20.052652+0000","last_unstale":"2024-09-10T18:51:28.976403+0000","last_undegraded":"2024-09-10T18:51:28.976403+0000","last_fullsized":"2024-09-10T18:51:28.976403+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:49:23.482807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:49:23.482807+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014419628,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":134,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.543037+0000","last_change":"2024-09-10T18:50:03.047387+0000","last_active":"2024-09-10T18:51:28.543037+0000","last_peered":"2024-09-10T18:51:28.543037+0000","last_clean":"2024-09-10T18:51:28.543037+0000","last_became_active":"2024-09-10T18:45:22.068326+0000","last_became_peered":"2024-09-10T18:45:22.068326+0000","last_unstale":"2024-09-10T18:51:28.543037+0000","last_undegraded":"2024-09-10T18:51:28.543037+0000","last_fullsized":"2024-09-10T18:51:28.543037+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:03.047305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:03.047305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023962509999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":130,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:19.548259+0000","last_change":"2024-09-10T18:51:03.523117+0000","last_active":"2024-09-10T18:51:19.548259+0000","last_peered":"2024-09-10T18:51:19.548259+0000","last_clean":"2024-09-10T18:51:19.548259+0000","last_became_active":"2024-09-10T18:45:24.081488+0000","last_became_peered":"2024-09-10T18:45:24.081488+0000","last_unstale":"2024-09-10T18:51:19.548259+0000","last_undegraded":"2024-09-10T18:51:19.548259+0000","last_fullsized":"2024-09-10T18:51:19.548259+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:03.523030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:03.523030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:33.287999+0000","scrub_duration":0.002492677,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.8","version":"70'81","reported_seq":51,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.542525+0000","last_change":"2024-09-10T18:50:44.076618+0000","last_active":"2024-09-10T18:51:28.542525+0000","last_peered":"2024-09-10T18:51:28.542525+0000","last_clean":"2024-09-10T18:51:28.542525+0000","last_became_active":"2024-09-10T18:50:08.397233+0000","last_became_peered":"2024-09-10T18:50:08.397233+0000","last_unstale":"2024-09-10T18:51:28.542525+0000","last_undegraded":"2024-09-10T18:51:28.542525+0000","last_fullsized":"2024-09-10T18:51:28.542525+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:50:44.076559+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:50:44.076559+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:50.654888+0000","scrub_duration":0.0035380699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"78'25","reported_seq":200,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.976226+0000","last_change":"2024-09-10T18:50:58.450309+0000","last_active":"2024-09-10T18:51:28.976226+0000","last_peered":"2024-09-10T18:51:28.976226+0000","last_clean":"2024-09-10T18:51:28.976226+0000","last_became_active":"2024-09-10T18:47:36.409300+0000","last_became_peered":"2024-09-10T18:47:36.409300+0000","last_unstale":"2024-09-10T18:51:28.976226+0000","last_undegraded":"2024-09-10T18:51:28.976226+0000","last_fullsized":"2024-09-10T18:51:28.976226+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'25","last_scrub_stamp":"2024-09-10T18:50:58.450203+0000","last_deep_scrub":"78'25","last_deep_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_clean_scrub_stamp":"2024-09-10T18:50:58.450203+0000","objects_scrubbed":8,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:14.332494+0000","scrub_duration":0.0058694890000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":139,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.976530+0000","last_change":"2024-09-10T18:50:01.419644+0000","last_active":"2024-09-10T18:51:28.976530+0000","last_peered":"2024-09-10T18:51:28.976530+0000","last_clean":"2024-09-10T18:51:28.976530+0000","last_became_active":"2024-09-10T18:45:17.963732+0000","last_became_peered":"2024-09-10T18:45:17.963732+0000","last_unstale":"2024-09-10T18:51:28.976530+0000","last_undegraded":"2024-09-10T18:51:28.976530+0000","last_fullsized":"2024-09-10T18:51:28.976530+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:01.419557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:01.419557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003472213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":122,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:19.548626+0000","last_change":"2024-09-10T18:49:17.310050+0000","last_active":"2024-09-10T18:51:19.548626+0000","last_peered":"2024-09-10T18:51:19.548626+0000","last_clean":"2024-09-10T18:51:19.548626+0000","last_became_active":"2024-09-10T18:45:22.071443+0000","last_became_peered":"2024-09-10T18:45:22.071443+0000","last_unstale":"2024-09-10T18:51:19.548626+0000","last_undegraded":"2024-09-10T18:51:19.548626+0000","last_fullsized":"2024-09-10T18:51:19.548626+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:17.309861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:04.512425+0000","last_clean_scrub_stamp":"2024-09-10T18:49:17.309861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051958760000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":76,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:19.548755+0000","last_change":"2024-09-10T18:49:11.545445+0000","last_active":"2024-09-10T18:51:19.548755+0000","last_peered":"2024-09-10T18:51:19.548755+0000","last_clean":"2024-09-10T18:51:19.548755+0000","last_became_active":"2024-09-10T18:47:57.104433+0000","last_became_peered":"2024-09-10T18:47:57.104433+0000","last_unstale":"2024-09-10T18:51:19.548755+0000","last_undegraded":"2024-09-10T18:51:19.548755+0000","last_fullsized":"2024-09-10T18:51:19.548755+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:11.545188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:11.545188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019872409999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.9","version":"70'81","reported_seq":203,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.976266+0000","last_change":"2024-09-10T18:50:53.567955+0000","last_active":"2024-09-10T18:51:28.976266+0000","last_peered":"2024-09-10T18:51:28.976266+0000","last_clean":"2024-09-10T18:51:28.976266+0000","last_became_active":"2024-09-10T18:50:08.399879+0000","last_became_peered":"2024-09-10T18:50:08.399879+0000","last_unstale":"2024-09-10T18:51:28.976266+0000","last_undegraded":"2024-09-10T18:51:28.976266+0000","last_fullsized":"2024-09-10T18:51:28.976266+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:50:53.567865+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:50:53.567865+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:54.014765+0000","scrub_duration":0.0066605650000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"116'15","reported_seq":146,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.975831+0000","last_change":"2024-09-10T18:49:12.374625+0000","last_active":"2024-09-10T18:51:28.975831+0000","last_peered":"2024-09-10T18:51:28.975831+0000","last_clean":"2024-09-10T18:51:28.975831+0000","last_became_active":"2024-09-10T18:45:20.053428+0000","last_became_peered":"2024-09-10T18:45:20.053428+0000","last_unstale":"2024-09-10T18:51:28.975831+0000","last_undegraded":"2024-09-10T18:51:28.975831+0000","last_fullsized":"2024-09-10T18:51:28.975831+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"116'15","last_scrub_stamp":"2024-09-10T18:49:12.374527+0000","last_deep_scrub":"116'15","last_deep_scrub_stamp":"2024-09-10T18:49:12.374527+0000","last_clean_scrub_stamp":"2024-09-10T18:49:12.374527+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0081453819999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":136,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:19.548206+0000","last_change":"2024-09-10T18:51:03.086045+0000","last_active":"2024-09-10T18:51:19.548206+0000","last_peered":"2024-09-10T18:51:19.548206+0000","last_clean":"2024-09-10T18:51:19.548206+0000","last_became_active":"2024-09-10T18:45:17.966833+0000","last_became_peered":"2024-09-10T18:45:17.966833+0000","last_unstale":"2024-09-10T18:51:19.548206+0000","last_undegraded":"2024-09-10T18:51:19.548206+0000","last_fullsized":"2024-09-10T18:51:19.548206+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:03.085839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:03.085839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:06.125680+0000","scrub_duration":0.0082351739999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"89'9","reported_seq":170,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.976866+0000","last_change":"2024-09-10T18:49:54.247659+0000","last_active":"2024-09-10T18:51:28.976866+0000","last_peered":"2024-09-10T18:51:28.976866+0000","last_clean":"2024-09-10T18:51:28.976866+0000","last_became_active":"2024-09-10T18:47:36.421188+0000","last_became_peered":"2024-09-10T18:47:36.421188+0000","last_unstale":"2024-09-10T18:51:28.976866+0000","last_undegraded":"2024-09-10T18:51:28.976866+0000","last_fullsized":"2024-09-10T18:51:28.976866+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'9","last_scrub_stamp":"2024-09-10T18:49:54.247585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:49:54.247585+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0054286500000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":142,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.975681+0000","last_change":"2024-09-10T18:50:56.384791+0000","last_active":"2024-09-10T18:51:28.975681+0000","last_peered":"2024-09-10T18:51:28.975681+0000","last_clean":"2024-09-10T18:51:28.975681+0000","last_became_active":"2024-09-10T18:47:36.414625+0000","last_became_peered":"2024-09-10T18:47:36.414625+0000","last_unstale":"2024-09-10T18:51:28.975681+0000","last_undegraded":"2024-09-10T18:51:28.975681+0000","last_fullsized":"2024-09-10T18:51:28.975681+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:56.384742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:32.043666+0000","last_clean_scrub_stamp":"2024-09-10T18:50:56.384742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:08.079927+0000","scrub_duration":0.001329467,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"70'81","reported_seq":46,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.542739+0000","last_change":"2024-09-10T18:50:40.100350+0000","last_active":"2024-09-10T18:51:28.542739+0000","last_peered":"2024-09-10T18:51:28.542739+0000","last_clean":"2024-09-10T18:51:28.542739+0000","last_became_active":"2024-09-10T18:50:08.398345+0000","last_became_peered":"2024-09-10T18:50:08.398345+0000","last_unstale":"2024-09-10T18:51:28.542739+0000","last_undegraded":"2024-09-10T18:51:28.542739+0000","last_fullsized":"2024-09-10T18:51:28.542739+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:50:40.100261+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:50:40.100261+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:47.324754+0000","scrub_duration":0.0020638319999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"141'22","reported_seq":135,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.543572+0000","last_change":"2024-09-10T18:50:57.119238+0000","last_active":"2024-09-10T18:51:28.543572+0000","last_peered":"2024-09-10T18:51:28.543572+0000","last_clean":"2024-09-10T18:51:28.543572+0000","last_became_active":"2024-09-10T18:46:56.742946+0000","last_became_peered":"2024-09-10T18:46:56.742946+0000","last_unstale":"2024-09-10T18:51:28.543572+0000","last_undegraded":"2024-09-10T18:51:28.543572+0000","last_fullsized":"2024-09-10T18:51:28.543572+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"141'22","last_scrub_stamp":"2024-09-10T18:50:57.119152+0000","last_deep_scrub":"141'22","last_deep_scrub_stamp":"2024-09-10T18:50:57.119152+0000","last_clean_scrub_stamp":"2024-09-10T18:50:57.119152+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:21.923801+0000","scrub_duration":0.020103994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":136,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:19.548057+0000","last_change":"2024-09-10T18:50:06.455339+0000","last_active":"2024-09-10T18:51:19.548057+0000","last_peered":"2024-09-10T18:51:19.548057+0000","last_clean":"2024-09-10T18:51:19.548057+0000","last_became_active":"2024-09-10T18:45:17.965409+0000","last_became_peered":"2024-09-10T18:45:17.965409+0000","last_unstale":"2024-09-10T18:51:19.548057+0000","last_undegraded":"2024-09-10T18:51:19.548057+0000","last_fullsized":"2024-09-10T18:51:19.548057+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:06.455254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_clean_scrub_stamp":"2024-09-10T18:50:06.455254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.018016590999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"63'8","reported_seq":216,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.976110+0000","last_change":"2024-09-10T18:50:48.530731+0000","last_active":"2024-09-10T18:51:28.976110+0000","last_peered":"2024-09-10T18:51:28.976110+0000","last_clean":"2024-09-10T18:51:28.976110+0000","last_became_active":"2024-09-10T18:45:22.062497+0000","last_became_peered":"2024-09-10T18:45:22.062497+0000","last_unstale":"2024-09-10T18:51:28.976110+0000","last_undegraded":"2024-09-10T18:51:28.976110+0000","last_fullsized":"2024-09-10T18:51:28.976110+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:50:48.530644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:48.530644+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:08.165585+0000","scrub_duration":0.022829841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":132,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.543239+0000","last_change":"2024-09-10T18:50:09.060853+0000","last_active":"2024-09-10T18:51:28.543239+0000","last_peered":"2024-09-10T18:51:28.543239+0000","last_clean":"2024-09-10T18:51:28.543239+0000","last_became_active":"2024-09-10T18:45:24.082362+0000","last_became_peered":"2024-09-10T18:45:24.082362+0000","last_unstale":"2024-09-10T18:51:28.543239+0000","last_undegraded":"2024-09-10T18:51:28.543239+0000","last_fullsized":"2024-09-10T18:51:28.543239+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:09.060766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:50:09.060766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.041565214000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"63'11","reported_seq":221,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.975971+0000","last_change":"2024-09-10T18:50:42.581936+0000","last_active":"2024-09-10T18:51:28.975971+0000","last_peered":"2024-09-10T18:51:28.975971+0000","last_clean":"2024-09-10T18:51:28.975971+0000","last_became_active":"2024-09-10T18:47:57.106935+0000","last_became_peered":"2024-09-10T18:47:57.106935+0000","last_unstale":"2024-09-10T18:51:28.975971+0000","last_undegraded":"2024-09-10T18:51:28.975971+0000","last_fullsized":"2024-09-10T18:51:28.975971+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:50:42.581837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:50:42.581837+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:08.935957+0000","scrub_duration":0.0093156460000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":139,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.976692+0000","last_change":"2024-09-10T18:49:51.223357+0000","last_active":"2024-09-10T18:51:28.976692+0000","last_peered":"2024-09-10T18:51:28.976692+0000","last_clean":"2024-09-10T18:51:28.976692+0000","last_became_active":"2024-09-10T18:45:17.966133+0000","last_became_peered":"2024-09-10T18:45:17.966133+0000","last_unstale":"2024-09-10T18:51:28.976692+0000","last_undegraded":"2024-09-10T18:51:28.976692+0000","last_fullsized":"2024-09-10T18:51:28.976692+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:51.223287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:51.223287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002675555,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":134,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.543684+0000","last_change":"2024-09-10T18:50:05.001733+0000","last_active":"2024-09-10T18:51:28.543684+0000","last_peered":"2024-09-10T18:51:28.543684+0000","last_clean":"2024-09-10T18:51:28.543684+0000","last_became_active":"2024-09-10T18:45:22.068918+0000","last_became_peered":"2024-09-10T18:45:22.068918+0000","last_unstale":"2024-09-10T18:51:28.543684+0000","last_undegraded":"2024-09-10T18:51:28.543684+0000","last_fullsized":"2024-09-10T18:51:28.543684+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:05.001646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_clean_scrub_stamp":"2024-09-10T18:50:05.001646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044088299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":120,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:20.127434+0000","last_change":"2024-09-10T18:49:04.647711+0000","last_active":"2024-09-10T18:51:20.127434+0000","last_peered":"2024-09-10T18:51:20.127434+0000","last_clean":"2024-09-10T18:51:20.127434+0000","last_became_active":"2024-09-10T18:45:24.264538+0000","last_became_peered":"2024-09-10T18:45:24.264538+0000","last_unstale":"2024-09-10T18:51:20.127434+0000","last_undegraded":"2024-09-10T18:51:20.127434+0000","last_fullsized":"2024-09-10T18:51:20.127434+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:04.647638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:04.647638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.115211389,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"70'81","reported_seq":48,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:20.127111+0000","last_change":"2024-09-10T18:50:59.164503+0000","last_active":"2024-09-10T18:51:20.127111+0000","last_peered":"2024-09-10T18:51:20.127111+0000","last_clean":"2024-09-10T18:51:20.127111+0000","last_became_active":"2024-09-10T18:50:08.399048+0000","last_became_peered":"2024-09-10T18:50:08.399048+0000","last_unstale":"2024-09-10T18:51:20.127111+0000","last_undegraded":"2024-09-10T18:51:20.127111+0000","last_fullsized":"2024-09-10T18:51:20.127111+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:50:59.164417+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:50:59.164417+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:19.822414+0000","scrub_duration":0.0063541700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"143'21","reported_seq":164,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.975631+0000","last_change":"2024-09-10T18:51:02.540313+0000","last_active":"2024-09-10T18:51:28.975631+0000","last_peered":"2024-09-10T18:51:28.975631+0000","last_clean":"2024-09-10T18:51:28.975631+0000","last_became_active":"2024-09-10T18:45:20.052243+0000","last_became_peered":"2024-09-10T18:45:20.052243+0000","last_unstale":"2024-09-10T18:51:28.975631+0000","last_undegraded":"2024-09-10T18:51:28.975631+0000","last_fullsized":"2024-09-10T18:51:28.975631+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"143'21","last_scrub_stamp":"2024-09-10T18:51:02.540216+0000","last_deep_scrub":"143'21","last_deep_scrub_stamp":"2024-09-10T18:51:02.540216+0000","last_clean_scrub_stamp":"2024-09-10T18:51:02.540216+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:26.124099+0000","scrub_duration":0.0060974410000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":148,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.701006+0000","last_change":"2024-09-10T18:49:59.397785+0000","last_active":"2024-09-10T18:51:28.701006+0000","last_peered":"2024-09-10T18:51:28.701006+0000","last_clean":"2024-09-10T18:51:28.701006+0000","last_became_active":"2024-09-10T18:47:36.424466+0000","last_became_peered":"2024-09-10T18:47:36.424466+0000","last_unstale":"2024-09-10T18:51:28.701006+0000","last_undegraded":"2024-09-10T18:51:28.701006+0000","last_fullsized":"2024-09-10T18:51:28.701006+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:59.397734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_clean_scrub_stamp":"2024-09-10T18:49:59.397734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00244567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":214,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T18:51:23.965244+0000","last_change":"2024-09-10T18:50:53.293010+0000","last_active":"2024-09-10T18:51:23.965244+0000","last_peered":"2024-09-10T18:51:23.965244+0000","last_clean":"2024-09-10T18:51:23.965244+0000","last_became_active":"2024-09-10T18:45:22.074540+0000","last_became_peered":"2024-09-10T18:45:22.074540+0000","last_unstale":"2024-09-10T18:51:23.965244+0000","last_undegraded":"2024-09-10T18:51:23.965244+0000","last_fullsized":"2024-09-10T18:51:23.965244+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:50:53.292838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:53.292838+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:07.773584+0000","scrub_duration":0.018341929999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":153,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.976197+0000","last_change":"2024-09-10T18:51:01.463186+0000","last_active":"2024-09-10T18:51:28.976197+0000","last_peered":"2024-09-10T18:51:28.976197+0000","last_clean":"2024-09-10T18:51:28.976197+0000","last_became_active":"2024-09-10T18:47:36.414665+0000","last_became_peered":"2024-09-10T18:47:36.414665+0000","last_unstale":"2024-09-10T18:51:28.976197+0000","last_undegraded":"2024-09-10T18:51:28.976197+0000","last_fullsized":"2024-09-10T18:51:28.976197+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:01.463123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:01.463123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:05.342543+0000","scrub_duration":0.0050178740000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"70'81","reported_seq":46,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.975324+0000","last_change":"2024-09-10T18:51:01.490009+0000","last_active":"2024-09-10T18:51:28.975324+0000","last_peered":"2024-09-10T18:51:28.975324+0000","last_clean":"2024-09-10T18:51:28.975324+0000","last_became_active":"2024-09-10T18:50:08.398081+0000","last_became_peered":"2024-09-10T18:50:08.398081+0000","last_unstale":"2024-09-10T18:51:28.975324+0000","last_undegraded":"2024-09-10T18:51:28.975324+0000","last_fullsized":"2024-09-10T18:51:28.975324+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:51:01.489919+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:51:01.489919+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:10.385199+0000","scrub_duration":0.0014613810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"63'18","reported_seq":166,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:20.126822+0000","last_change":"2024-09-10T18:50:44.111437+0000","last_active":"2024-09-10T18:51:20.126822+0000","last_peered":"2024-09-10T18:51:20.126822+0000","last_clean":"2024-09-10T18:51:20.126822+0000","last_became_active":"2024-09-10T18:45:20.420937+0000","last_became_peered":"2024-09-10T18:45:20.420937+0000","last_unstale":"2024-09-10T18:51:20.126822+0000","last_undegraded":"2024-09-10T18:51:20.126822+0000","last_fullsized":"2024-09-10T18:51:20.126822+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'18","last_scrub_stamp":"2024-09-10T18:50:44.111343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:50:44.111343+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:06.271060+0000","scrub_duration":0.013021517999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":126,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:19.548655+0000","last_change":"2024-09-10T18:49:23.336052+0000","last_active":"2024-09-10T18:51:19.548655+0000","last_peered":"2024-09-10T18:51:19.548655+0000","last_clean":"2024-09-10T18:51:19.548655+0000","last_became_active":"2024-09-10T18:45:17.961695+0000","last_became_peered":"2024-09-10T18:45:17.961695+0000","last_unstale":"2024-09-10T18:51:19.548655+0000","last_undegraded":"2024-09-10T18:51:19.548655+0000","last_fullsized":"2024-09-10T18:51:19.548655+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:23.335968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:23.335968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056344669999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":134,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.975305+0000","last_change":"2024-09-10T18:50:43.554137+0000","last_active":"2024-09-10T18:51:28.975305+0000","last_peered":"2024-09-10T18:51:28.975305+0000","last_clean":"2024-09-10T18:51:28.975305+0000","last_became_active":"2024-09-10T18:45:22.074012+0000","last_became_peered":"2024-09-10T18:45:22.074012+0000","last_unstale":"2024-09-10T18:51:28.975305+0000","last_undegraded":"2024-09-10T18:51:28.975305+0000","last_fullsized":"2024-09-10T18:51:28.975305+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:43.554063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:43.554063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:07.347038+0000","scrub_duration":0.0065283629999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":132,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.976320+0000","last_change":"2024-09-10T18:50:03.534693+0000","last_active":"2024-09-10T18:51:28.976320+0000","last_peered":"2024-09-10T18:51:28.976320+0000","last_clean":"2024-09-10T18:51:28.976320+0000","last_became_active":"2024-09-10T18:47:36.410836+0000","last_became_peered":"2024-09-10T18:47:36.410836+0000","last_unstale":"2024-09-10T18:51:28.976320+0000","last_undegraded":"2024-09-10T18:51:28.976320+0000","last_fullsized":"2024-09-10T18:51:28.976320+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:03.534613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_clean_scrub_stamp":"2024-09-10T18:50:03.534613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010201148,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"70'81","reported_seq":104,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.700724+0000","last_change":"2024-09-10T18:50:08.405831+0000","last_active":"2024-09-10T18:51:28.700724+0000","last_peered":"2024-09-10T18:51:28.700724+0000","last_clean":"2024-09-10T18:51:28.700724+0000","last_became_active":"2024-09-10T18:50:08.405424+0000","last_became_peered":"2024-09-10T18:50:08.405424+0000","last_unstale":"2024-09-10T18:51:28.700724+0000","last_undegraded":"2024-09-10T18:51:28.700724+0000","last_fullsized":"2024-09-10T18:51:28.700724+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:49:09.660690+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:49:09.660690+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019876809999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":20,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":107340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"63'19","reported_seq":170,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:19.548057+0000","last_change":"2024-09-10T18:49:55.690798+0000","last_active":"2024-09-10T18:51:19.548057+0000","last_peered":"2024-09-10T18:51:19.548057+0000","last_clean":"2024-09-10T18:51:19.548057+0000","last_became_active":"2024-09-10T18:45:20.421046+0000","last_became_peered":"2024-09-10T18:45:20.421046+0000","last_unstale":"2024-09-10T18:51:19.548057+0000","last_undegraded":"2024-09-10T18:51:19.548057+0000","last_fullsized":"2024-09-10T18:51:19.548057+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:49:55.690720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:49:55.690720+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005164465,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":139,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.977120+0000","last_change":"2024-09-10T18:50:06.454269+0000","last_active":"2024-09-10T18:51:28.977120+0000","last_peered":"2024-09-10T18:51:28.977120+0000","last_clean":"2024-09-10T18:51:28.977120+0000","last_became_active":"2024-09-10T18:45:17.960226+0000","last_became_peered":"2024-09-10T18:45:17.960226+0000","last_unstale":"2024-09-10T18:51:28.977120+0000","last_undegraded":"2024-09-10T18:51:28.977120+0000","last_fullsized":"2024-09-10T18:51:28.977120+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:06.454124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:06.454124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014135569000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":134,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.700169+0000","last_change":"2024-09-10T18:50:08.326814+0000","last_active":"2024-09-10T18:51:28.700169+0000","last_peered":"2024-09-10T18:51:28.700169+0000","last_clean":"2024-09-10T18:51:28.700169+0000","last_became_active":"2024-09-10T18:45:22.066148+0000","last_became_peered":"2024-09-10T18:45:22.066148+0000","last_unstale":"2024-09-10T18:51:28.700169+0000","last_undegraded":"2024-09-10T18:51:28.700169+0000","last_fullsized":"2024-09-10T18:51:28.700169+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:08.326738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:08.326738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024862109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":130,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:19.547882+0000","last_change":"2024-09-10T18:49:18.281313+0000","last_active":"2024-09-10T18:51:19.547882+0000","last_peered":"2024-09-10T18:51:19.547882+0000","last_clean":"2024-09-10T18:51:19.547882+0000","last_became_active":"2024-09-10T18:47:36.422295+0000","last_became_peered":"2024-09-10T18:47:36.422295+0000","last_unstale":"2024-09-10T18:51:19.547882+0000","last_undegraded":"2024-09-10T18:51:19.547882+0000","last_fullsized":"2024-09-10T18:51:19.547882+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:18.281230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:49:18.281230+0000","last_clean_scrub_stamp":"2024-09-10T18:49:18.281230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0066273570000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"70'81","reported_seq":104,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.700679+0000","last_change":"2024-09-10T18:50:08.397558+0000","last_active":"2024-09-10T18:51:28.700679+0000","last_peered":"2024-09-10T18:51:28.700679+0000","last_clean":"2024-09-10T18:51:28.700679+0000","last_became_active":"2024-09-10T18:50:08.397123+0000","last_became_peered":"2024-09-10T18:50:08.397123+0000","last_unstale":"2024-09-10T18:51:28.700679+0000","last_undegraded":"2024-09-10T18:51:28.700679+0000","last_fullsized":"2024-09-10T18:51:28.700679+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:49:09.660690+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:49:09.660690+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019876809999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":20,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":107340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"83'15","reported_seq":154,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:19.548128+0000","last_change":"2024-09-10T18:50:09.557013+0000","last_active":"2024-09-10T18:51:19.548128+0000","last_peered":"2024-09-10T18:51:19.548128+0000","last_clean":"2024-09-10T18:51:19.548128+0000","last_became_active":"2024-09-10T18:45:20.057204+0000","last_became_peered":"2024-09-10T18:45:20.057204+0000","last_unstale":"2024-09-10T18:51:19.548128+0000","last_undegraded":"2024-09-10T18:51:19.548128+0000","last_fullsized":"2024-09-10T18:51:19.548128+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'15","last_scrub_stamp":"2024-09-10T18:50:09.556916+0000","last_deep_scrub":"83'15","last_deep_scrub_stamp":"2024-09-10T18:46:58.436473+0000","last_clean_scrub_stamp":"2024-09-10T18:50:09.556916+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:24.162275+0000","scrub_duration":0.01222707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":136,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:19.548562+0000","last_change":"2024-09-10T18:51:05.140273+0000","last_active":"2024-09-10T18:51:19.548562+0000","last_peered":"2024-09-10T18:51:19.548562+0000","last_clean":"2024-09-10T18:51:19.548562+0000","last_became_active":"2024-09-10T18:45:17.966632+0000","last_became_peered":"2024-09-10T18:45:17.966632+0000","last_unstale":"2024-09-10T18:51:19.548562+0000","last_undegraded":"2024-09-10T18:51:19.548562+0000","last_fullsized":"2024-09-10T18:51:19.548562+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:05.140027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:05.140027+0000","last_clean_scrub_stamp":"2024-09-10T18:51:05.140027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:28.134498+0000","scrub_duration":0.0085472780000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"63'8","reported_seq":204,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T18:51:23.540114+0000","last_change":"2024-09-10T18:49:00.552860+0000","last_active":"2024-09-10T18:51:23.540114+0000","last_peered":"2024-09-10T18:51:23.540114+0000","last_clean":"2024-09-10T18:51:23.540114+0000","last_became_active":"2024-09-10T18:45:22.063326+0000","last_became_peered":"2024-09-10T18:45:22.063326+0000","last_unstale":"2024-09-10T18:51:23.540114+0000","last_undegraded":"2024-09-10T18:51:23.540114+0000","last_fullsized":"2024-09-10T18:51:23.540114+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:49:00.552765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:49:00.552765+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038235579999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":140,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:19.547131+0000","last_change":"2024-09-10T18:51:01.618543+0000","last_active":"2024-09-10T18:51:19.547131+0000","last_peered":"2024-09-10T18:51:19.547131+0000","last_clean":"2024-09-10T18:51:19.547131+0000","last_became_active":"2024-09-10T18:47:36.412111+0000","last_became_peered":"2024-09-10T18:47:36.412111+0000","last_unstale":"2024-09-10T18:51:19.547131+0000","last_undegraded":"2024-09-10T18:51:19.547131+0000","last_fullsized":"2024-09-10T18:51:19.547131+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:01.618458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:01.618458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:21.627693+0000","scrub_duration":0.0072066109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"70'81","reported_seq":46,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.542607+0000","last_change":"2024-09-10T18:50:43.117818+0000","last_active":"2024-09-10T18:51:28.542607+0000","last_peered":"2024-09-10T18:51:28.542607+0000","last_clean":"2024-09-10T18:51:28.542607+0000","last_became_active":"2024-09-10T18:50:08.399074+0000","last_became_peered":"2024-09-10T18:50:08.399074+0000","last_unstale":"2024-09-10T18:51:28.542607+0000","last_undegraded":"2024-09-10T18:51:28.542607+0000","last_fullsized":"2024-09-10T18:51:28.542607+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:50:43.117728+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:50:43.117728+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:52.214303+0000","scrub_duration":0.0039521169999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"89'29","reported_seq":288,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T18:51:23.542883+0000","last_change":"2024-09-10T18:50:04.462642+0000","last_active":"2024-09-10T18:51:23.542883+0000","last_peered":"2024-09-10T18:51:23.542883+0000","last_clean":"2024-09-10T18:51:23.542883+0000","last_became_active":"2024-09-10T18:47:36.420185+0000","last_became_peered":"2024-09-10T18:47:36.420185+0000","last_unstale":"2024-09-10T18:51:23.542883+0000","last_undegraded":"2024-09-10T18:51:23.542883+0000","last_fullsized":"2024-09-10T18:51:23.542883+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'29","last_scrub_stamp":"2024-09-10T18:50:04.459912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:50:04.459912+0000","objects_scrubbed":10,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0097323310000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":138,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.543128+0000","last_change":"2024-09-10T18:50:12.982700+0000","last_active":"2024-09-10T18:51:28.543128+0000","last_peered":"2024-09-10T18:51:28.543128+0000","last_clean":"2024-09-10T18:51:28.543128+0000","last_became_active":"2024-09-10T18:45:17.964494+0000","last_became_peered":"2024-09-10T18:45:17.964494+0000","last_unstale":"2024-09-10T18:51:28.543128+0000","last_undegraded":"2024-09-10T18:51:28.543128+0000","last_fullsized":"2024-09-10T18:51:28.543128+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:12.982599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:12.982599+0000","last_clean_scrub_stamp":"2024-09-10T18:50:12.982599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048150440000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":89,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.700464+0000","last_change":"2024-09-10T18:51:01.357880+0000","last_active":"2024-09-10T18:51:28.700464+0000","last_peered":"2024-09-10T18:51:28.700464+0000","last_clean":"2024-09-10T18:51:28.700464+0000","last_became_active":"2024-09-10T18:47:57.107478+0000","last_became_peered":"2024-09-10T18:47:57.107478+0000","last_unstale":"2024-09-10T18:51:28.700464+0000","last_undegraded":"2024-09-10T18:51:28.700464+0000","last_fullsized":"2024-09-10T18:51:28.700464+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:01.357830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:01.357830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:28.477016+0000","scrub_duration":0.002118796,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":120,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:20.127660+0000","last_change":"2024-09-10T18:49:57.442744+0000","last_active":"2024-09-10T18:51:20.127660+0000","last_peered":"2024-09-10T18:51:20.127660+0000","last_clean":"2024-09-10T18:51:20.127660+0000","last_became_active":"2024-09-10T18:45:24.073307+0000","last_became_peered":"2024-09-10T18:45:24.073307+0000","last_unstale":"2024-09-10T18:51:20.127660+0000","last_undegraded":"2024-09-10T18:51:20.127660+0000","last_fullsized":"2024-09-10T18:51:20.127660+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:57.442512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:22.891084+0000","last_clean_scrub_stamp":"2024-09-10T18:49:57.442512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:26.399541+0000","scrub_duration":0.0050025080000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"70'81","reported_seq":203,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.976312+0000","last_change":"2024-09-10T18:50:50.506911+0000","last_active":"2024-09-10T18:51:28.976312+0000","last_peered":"2024-09-10T18:51:28.976312+0000","last_clean":"2024-09-10T18:51:28.976312+0000","last_became_active":"2024-09-10T18:50:08.397665+0000","last_became_peered":"2024-09-10T18:50:08.397665+0000","last_unstale":"2024-09-10T18:51:28.976312+0000","last_undegraded":"2024-09-10T18:51:28.976312+0000","last_fullsized":"2024-09-10T18:51:28.976312+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:50:50.506769+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:50:50.506769+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:53.715025+0000","scrub_duration":0.0043821199999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"84'18","reported_seq":156,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:19.548738+0000","last_change":"2024-09-10T18:49:13.417595+0000","last_active":"2024-09-10T18:51:19.548738+0000","last_peered":"2024-09-10T18:51:19.548738+0000","last_clean":"2024-09-10T18:51:19.548738+0000","last_became_active":"2024-09-10T18:45:20.058435+0000","last_became_peered":"2024-09-10T18:45:20.058435+0000","last_unstale":"2024-09-10T18:51:19.548738+0000","last_undegraded":"2024-09-10T18:51:19.548738+0000","last_fullsized":"2024-09-10T18:51:19.548738+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'18","last_scrub_stamp":"2024-09-10T18:49:13.417302+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-10T18:47:10.514951+0000","last_clean_scrub_stamp":"2024-09-10T18:49:13.417302+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0096130710000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"84'6","reported_seq":508,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T18:51:24.948747+0000","last_change":"2024-09-10T18:49:57.418744+0000","last_active":"2024-09-10T18:51:24.948747+0000","last_peered":"2024-09-10T18:51:24.948747+0000","last_clean":"2024-09-10T18:51:24.948747+0000","last_became_active":"2024-09-10T18:47:36.418211+0000","last_became_peered":"2024-09-10T18:47:36.418211+0000","last_unstale":"2024-09-10T18:51:24.948747+0000","last_undegraded":"2024-09-10T18:51:24.948747+0000","last_fullsized":"2024-09-10T18:51:24.948747+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'6","last_scrub_stamp":"2024-09-10T18:49:57.416319+0000","last_deep_scrub":"84'6","last_deep_scrub_stamp":"2024-09-10T18:47:22.545450+0000","last_clean_scrub_stamp":"2024-09-10T18:49:57.416319+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044734659999999997,"objects_trimmed":0,"snaptrim_duration":0.0021933159999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":358,"num_read_kb":353,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":87,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:20.127190+0000","last_change":"2024-09-10T18:49:16.371789+0000","last_active":"2024-09-10T18:51:20.127190+0000","last_peered":"2024-09-10T18:51:20.127190+0000","last_clean":"2024-09-10T18:51:20.127190+0000","last_became_active":"2024-09-10T18:47:36.422641+0000","last_became_peered":"2024-09-10T18:47:36.422641+0000","last_unstale":"2024-09-10T18:51:20.127190+0000","last_undegraded":"2024-09-10T18:51:20.127190+0000","last_fullsized":"2024-09-10T18:51:20.127190+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:16.371702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:16.371702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0099806840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":99,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.543503+0000","last_change":"2024-09-10T18:50:00.088364+0000","last_active":"2024-09-10T18:51:28.543503+0000","last_peered":"2024-09-10T18:51:28.543503+0000","last_clean":"2024-09-10T18:51:28.543503+0000","last_became_active":"2024-09-10T18:47:36.419081+0000","last_became_peered":"2024-09-10T18:47:36.419081+0000","last_unstale":"2024-09-10T18:51:28.543503+0000","last_undegraded":"2024-09-10T18:51:28.543503+0000","last_fullsized":"2024-09-10T18:51:28.543503+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:00.088179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:00.088179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.034684810000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":123,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.976661+0000","last_change":"2024-09-10T18:49:12.603922+0000","last_active":"2024-09-10T18:51:28.976661+0000","last_peered":"2024-09-10T18:51:28.976661+0000","last_clean":"2024-09-10T18:51:28.976661+0000","last_became_active":"2024-09-10T18:45:24.264617+0000","last_became_peered":"2024-09-10T18:45:24.264617+0000","last_unstale":"2024-09-10T18:51:28.976661+0000","last_undegraded":"2024-09-10T18:51:28.976661+0000","last_fullsized":"2024-09-10T18:51:28.976661+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:12.603786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:12.603786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019980200000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"70'81","reported_seq":43,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:19.547380+0000","last_change":"2024-09-10T18:51:04.099758+0000","last_active":"2024-09-10T18:51:19.547380+0000","last_peered":"2024-09-10T18:51:19.547380+0000","last_clean":"2024-09-10T18:51:19.547380+0000","last_became_active":"2024-09-10T18:50:08.398205+0000","last_became_peered":"2024-09-10T18:50:08.398205+0000","last_unstale":"2024-09-10T18:51:19.547380+0000","last_undegraded":"2024-09-10T18:51:19.547380+0000","last_fullsized":"2024-09-10T18:51:19.547380+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:51:04.099703+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:51:04.099703+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:26.646801+0000","scrub_duration":0.007621203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"63'12","reported_seq":176,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:19.549436+0000","last_change":"2024-09-10T18:50:48.742956+0000","last_active":"2024-09-10T18:51:19.549436+0000","last_peered":"2024-09-10T18:51:19.549436+0000","last_clean":"2024-09-10T18:51:19.549436+0000","last_became_active":"2024-09-10T18:47:36.417965+0000","last_became_peered":"2024-09-10T18:47:36.417965+0000","last_unstale":"2024-09-10T18:51:19.549436+0000","last_undegraded":"2024-09-10T18:51:19.549436+0000","last_fullsized":"2024-09-10T18:51:19.549436+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:50:48.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:50:48.742875+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:50.790274+0000","scrub_duration":0.011146239,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"58'1","reported_seq":132,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:19.547465+0000","last_change":"2024-09-10T18:49:07.525629+0000","last_active":"2024-09-10T18:51:19.547465+0000","last_peered":"2024-09-10T18:51:19.547465+0000","last_clean":"2024-09-10T18:51:19.547465+0000","last_became_active":"2024-09-10T18:45:17.963932+0000","last_became_peered":"2024-09-10T18:45:17.963932+0000","last_unstale":"2024-09-10T18:51:19.547465+0000","last_undegraded":"2024-09-10T18:51:19.547465+0000","last_fullsized":"2024-09-10T18:51:19.547465+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'1","last_scrub_stamp":"2024-09-10T18:49:07.525525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:07.525525+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.01484512,"objects_trimmed":0,"snaptrim_duration":0.00037812099999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":97,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-10T18:51:30.467082+0000","last_change":"2024-09-10T18:51:30.467050+0000","last_active":"2024-09-10T18:51:30.467082+0000","last_peered":"2024-09-10T18:51:30.467082+0000","last_clean":"2024-09-10T18:51:30.467082+0000","last_became_active":"2024-09-10T18:47:36.416641+0000","last_became_peered":"2024-09-10T18:47:36.416641+0000","last_unstale":"2024-09-10T18:51:30.467082+0000","last_undegraded":"2024-09-10T18:51:30.467082+0000","last_fullsized":"2024-09-10T18:51:30.467082+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:30.466988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:30.466988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:44.004240+0000","scrub_duration":0.0019674089999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":134,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.543726+0000","last_change":"2024-09-10T18:50:52.999936+0000","last_active":"2024-09-10T18:51:28.543726+0000","last_peered":"2024-09-10T18:51:28.543726+0000","last_clean":"2024-09-10T18:51:28.543726+0000","last_became_active":"2024-09-10T18:45:22.064285+0000","last_became_peered":"2024-09-10T18:45:22.064285+0000","last_unstale":"2024-09-10T18:51:28.543726+0000","last_undegraded":"2024-09-10T18:51:28.543726+0000","last_fullsized":"2024-09-10T18:51:28.543726+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:52.999842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:52.999842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:02.826146+0000","scrub_duration":0.0065214100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":130,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:19.548213+0000","last_change":"2024-09-10T18:50:53.940855+0000","last_active":"2024-09-10T18:51:19.548213+0000","last_peered":"2024-09-10T18:51:19.548213+0000","last_clean":"2024-09-10T18:51:19.548213+0000","last_became_active":"2024-09-10T18:45:24.083689+0000","last_became_peered":"2024-09-10T18:45:24.083689+0000","last_unstale":"2024-09-10T18:51:19.548213+0000","last_undegraded":"2024-09-10T18:51:19.548213+0000","last_fullsized":"2024-09-10T18:51:19.548213+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:53.940489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:50:53.940489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:21.653605+0000","scrub_duration":0.007171691,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"126'21","reported_seq":165,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.700756+0000","last_change":"2024-09-10T18:50:03.380989+0000","last_active":"2024-09-10T18:51:28.700756+0000","last_peered":"2024-09-10T18:51:28.700756+0000","last_clean":"2024-09-10T18:51:28.700756+0000","last_became_active":"2024-09-10T18:45:20.421119+0000","last_became_peered":"2024-09-10T18:45:20.421119+0000","last_unstale":"2024-09-10T18:51:28.700756+0000","last_undegraded":"2024-09-10T18:51:28.700756+0000","last_fullsized":"2024-09-10T18:51:28.700756+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"126'21","last_scrub_stamp":"2024-09-10T18:50:03.380902+0000","last_deep_scrub":"126'21","last_deep_scrub_stamp":"2024-09-10T18:50:03.380902+0000","last_clean_scrub_stamp":"2024-09-10T18:50:03.380902+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0084173819999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"60'2","reported_seq":143,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:20.127366+0000","last_change":"2024-09-10T18:51:03.104641+0000","last_active":"2024-09-10T18:51:20.127366+0000","last_peered":"2024-09-10T18:51:20.127366+0000","last_clean":"2024-09-10T18:51:20.127366+0000","last_became_active":"2024-09-10T18:45:17.961660+0000","last_became_peered":"2024-09-10T18:45:17.961660+0000","last_unstale":"2024-09-10T18:51:20.127366+0000","last_undegraded":"2024-09-10T18:51:20.127366+0000","last_fullsized":"2024-09-10T18:51:20.127366+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-10T18:51:03.104545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:03.104545+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:16.497102+0000","scrub_duration":0.0070342290000000003,"objects_trimmed":0,"snaptrim_duration":0.0013942640000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":148,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.543291+0000","last_change":"2024-09-10T18:50:14.032609+0000","last_active":"2024-09-10T18:51:28.543291+0000","last_peered":"2024-09-10T18:51:28.543291+0000","last_clean":"2024-09-10T18:51:28.543291+0000","last_became_active":"2024-09-10T18:47:36.414636+0000","last_became_peered":"2024-09-10T18:47:36.414636+0000","last_unstale":"2024-09-10T18:51:28.543291+0000","last_undegraded":"2024-09-10T18:51:28.543291+0000","last_fullsized":"2024-09-10T18:51:28.543291+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:14.032522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:14.032522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:38.247378+0000","scrub_duration":0.022590255,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'81","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.976863+0000","last_change":"2024-09-10T18:50:48.508352+0000","last_active":"2024-09-10T18:51:28.976863+0000","last_peered":"2024-09-10T18:51:28.976863+0000","last_clean":"2024-09-10T18:51:28.976863+0000","last_became_active":"2024-09-10T18:50:06.386717+0000","last_became_peered":"2024-09-10T18:50:06.386717+0000","last_unstale":"2024-09-10T18:51:28.976863+0000","last_undegraded":"2024-09-10T18:51:28.976863+0000","last_fullsized":"2024-09-10T18:51:28.976863+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:50:48.508244+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:50:48.508244+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:53.803446+0000","scrub_duration":0.0035856120000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":145,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.976896+0000","last_change":"2024-09-10T18:49:14.625034+0000","last_active":"2024-09-10T18:51:28.976896+0000","last_peered":"2024-09-10T18:51:28.976896+0000","last_clean":"2024-09-10T18:51:28.976896+0000","last_became_active":"2024-09-10T18:47:36.418320+0000","last_became_peered":"2024-09-10T18:47:36.418320+0000","last_unstale":"2024-09-10T18:51:28.976896+0000","last_undegraded":"2024-09-10T18:51:28.976896+0000","last_fullsized":"2024-09-10T18:51:28.976896+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:14.624957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:49:14.624957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023025350000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":132,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.543773+0000","last_change":"2024-09-10T18:51:02.170258+0000","last_active":"2024-09-10T18:51:28.543773+0000","last_peered":"2024-09-10T18:51:28.543773+0000","last_clean":"2024-09-10T18:51:28.543773+0000","last_became_active":"2024-09-10T18:45:24.073327+0000","last_became_peered":"2024-09-10T18:45:24.073327+0000","last_unstale":"2024-09-10T18:51:28.543773+0000","last_undegraded":"2024-09-10T18:51:28.543773+0000","last_fullsized":"2024-09-10T18:51:28.543773+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:02.170065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:02.170065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:14.436899+0000","scrub_duration":0.0061532089999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"63'19","reported_seq":167,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.700297+0000","last_change":"2024-09-10T18:50:01.460246+0000","last_active":"2024-09-10T18:51:28.700297+0000","last_peered":"2024-09-10T18:51:28.700297+0000","last_clean":"2024-09-10T18:51:28.700297+0000","last_became_active":"2024-09-10T18:45:20.421097+0000","last_became_peered":"2024-09-10T18:45:20.421097+0000","last_unstale":"2024-09-10T18:51:28.700297+0000","last_undegraded":"2024-09-10T18:51:28.700297+0000","last_fullsized":"2024-09-10T18:51:28.700297+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:50:01.460159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:50:01.460159+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.019465403999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":99,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.543387+0000","last_change":"2024-09-10T18:50:47.083728+0000","last_active":"2024-09-10T18:51:28.543387+0000","last_peered":"2024-09-10T18:51:28.543387+0000","last_clean":"2024-09-10T18:51:28.543387+0000","last_became_active":"2024-09-10T18:47:36.417800+0000","last_became_peered":"2024-09-10T18:47:36.417800+0000","last_unstale":"2024-09-10T18:51:28.543387+0000","last_undegraded":"2024-09-10T18:51:28.543387+0000","last_fullsized":"2024-09-10T18:51:28.543387+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:47.083643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:47.083643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:16.602999+0000","scrub_duration":0.0071207509999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":97,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:19.547814+0000","last_change":"2024-09-10T18:49:20.477390+0000","last_active":"2024-09-10T18:51:19.547814+0000","last_peered":"2024-09-10T18:51:19.547814+0000","last_clean":"2024-09-10T18:51:19.547814+0000","last_became_active":"2024-09-10T18:47:36.413618+0000","last_became_peered":"2024-09-10T18:47:36.413618+0000","last_unstale":"2024-09-10T18:51:19.547814+0000","last_undegraded":"2024-09-10T18:51:19.547814+0000","last_fullsized":"2024-09-10T18:51:19.547814+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:20.477302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:49:20.477302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047920660000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":133,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.976693+0000","last_change":"2024-09-10T18:49:11.634724+0000","last_active":"2024-09-10T18:51:28.976693+0000","last_peered":"2024-09-10T18:51:28.976693+0000","last_clean":"2024-09-10T18:51:28.976693+0000","last_became_active":"2024-09-10T18:47:36.412610+0000","last_became_peered":"2024-09-10T18:47:36.412610+0000","last_unstale":"2024-09-10T18:51:28.976693+0000","last_undegraded":"2024-09-10T18:51:28.976693+0000","last_fullsized":"2024-09-10T18:51:28.976693+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:11.634636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:11.634636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058360749999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"130'13","reported_seq":154,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:19.547966+0000","last_change":"2024-09-10T18:50:08.785524+0000","last_active":"2024-09-10T18:51:19.547966+0000","last_peered":"2024-09-10T18:51:19.547966+0000","last_clean":"2024-09-10T18:51:19.547966+0000","last_became_active":"2024-09-10T18:45:20.061021+0000","last_became_peered":"2024-09-10T18:45:20.061021+0000","last_unstale":"2024-09-10T18:51:19.547966+0000","last_undegraded":"2024-09-10T18:51:19.547966+0000","last_fullsized":"2024-09-10T18:51:19.547966+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"130'13","last_scrub_stamp":"2024-09-10T18:50:08.785434+0000","last_deep_scrub":"130'13","last_deep_scrub_stamp":"2024-09-10T18:50:08.785434+0000","last_clean_scrub_stamp":"2024-09-10T18:50:08.785434+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:32.989910+0000","scrub_duration":0.016531225,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":126,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:19.548739+0000","last_change":"2024-09-10T18:49:07.384707+0000","last_active":"2024-09-10T18:51:19.548739+0000","last_peered":"2024-09-10T18:51:19.548739+0000","last_clean":"2024-09-10T18:51:19.548739+0000","last_became_active":"2024-09-10T18:45:17.963057+0000","last_became_peered":"2024-09-10T18:45:17.963057+0000","last_unstale":"2024-09-10T18:51:19.548739+0000","last_undegraded":"2024-09-10T18:51:19.548739+0000","last_fullsized":"2024-09-10T18:51:19.548739+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:07.384620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:07.384620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011420041000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":89,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.976169+0000","last_change":"2024-09-10T18:49:21.442258+0000","last_active":"2024-09-10T18:51:28.976169+0000","last_peered":"2024-09-10T18:51:28.976169+0000","last_clean":"2024-09-10T18:51:28.976169+0000","last_became_active":"2024-09-10T18:47:36.411517+0000","last_became_peered":"2024-09-10T18:47:36.411517+0000","last_unstale":"2024-09-10T18:51:28.976169+0000","last_undegraded":"2024-09-10T18:51:28.976169+0000","last_fullsized":"2024-09-10T18:51:28.976169+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:21.442183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_clean_scrub_stamp":"2024-09-10T18:49:21.442183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039195419999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":140,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:19.547396+0000","last_change":"2024-09-10T18:49:54.678748+0000","last_active":"2024-09-10T18:51:19.547396+0000","last_peered":"2024-09-10T18:51:19.547396+0000","last_clean":"2024-09-10T18:51:19.547396+0000","last_became_active":"2024-09-10T18:47:36.412933+0000","last_became_peered":"2024-09-10T18:47:36.412933+0000","last_unstale":"2024-09-10T18:51:19.547396+0000","last_undegraded":"2024-09-10T18:51:19.547396+0000","last_fullsized":"2024-09-10T18:51:19.547396+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:54.678678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:47.545343+0000","last_clean_scrub_stamp":"2024-09-10T18:49:54.678678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0091071069999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"63'15","reported_seq":151,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.542624+0000","last_change":"2024-09-10T18:49:52.031321+0000","last_active":"2024-09-10T18:51:28.542624+0000","last_peered":"2024-09-10T18:51:28.542624+0000","last_clean":"2024-09-10T18:51:28.542624+0000","last_became_active":"2024-09-10T18:45:20.419804+0000","last_became_peered":"2024-09-10T18:45:20.419804+0000","last_unstale":"2024-09-10T18:51:28.542624+0000","last_undegraded":"2024-09-10T18:51:28.542624+0000","last_fullsized":"2024-09-10T18:51:28.542624+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:49:52.031226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:49:52.031226+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0084966710000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":87,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:20.127756+0000","last_change":"2024-09-10T18:50:05.603208+0000","last_active":"2024-09-10T18:51:20.127756+0000","last_peered":"2024-09-10T18:51:20.127756+0000","last_clean":"2024-09-10T18:51:20.127756+0000","last_became_active":"2024-09-10T18:47:36.420475+0000","last_became_peered":"2024-09-10T18:47:36.420475+0000","last_unstale":"2024-09-10T18:51:20.127756+0000","last_undegraded":"2024-09-10T18:51:20.127756+0000","last_fullsized":"2024-09-10T18:51:20.127756+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:05.603125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:05.603125+0000","last_clean_scrub_stamp":"2024-09-10T18:50:05.603125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048485560000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"83'9","reported_seq":221,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.976181+0000","last_change":"2024-09-10T18:50:10.444294+0000","last_active":"2024-09-10T18:51:28.976181+0000","last_peered":"2024-09-10T18:51:28.976181+0000","last_clean":"2024-09-10T18:51:28.976181+0000","last_became_active":"2024-09-10T18:45:22.073383+0000","last_became_peered":"2024-09-10T18:45:22.073383+0000","last_unstale":"2024-09-10T18:51:28.976181+0000","last_undegraded":"2024-09-10T18:51:28.976181+0000","last_fullsized":"2024-09-10T18:51:28.976181+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'9","last_scrub_stamp":"2024-09-10T18:50:10.444201+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-09-10T18:47:02.582229+0000","last_clean_scrub_stamp":"2024-09-10T18:50:10.444201+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:35.937521+0000","scrub_duration":0.014241271,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":132,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.542846+0000","last_change":"2024-09-10T18:51:05.187154+0000","last_active":"2024-09-10T18:51:28.542846+0000","last_peered":"2024-09-10T18:51:28.542846+0000","last_clean":"2024-09-10T18:51:28.542846+0000","last_became_active":"2024-09-10T18:45:24.265256+0000","last_became_peered":"2024-09-10T18:45:24.265256+0000","last_unstale":"2024-09-10T18:51:28.542846+0000","last_undegraded":"2024-09-10T18:51:28.542846+0000","last_fullsized":"2024-09-10T18:51:28.542846+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:05.187073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_clean_scrub_stamp":"2024-09-10T18:51:05.187073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:11.730150+0000","scrub_duration":0.00225728,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"63'12","reported_seq":159,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.975524+0000","last_change":"2024-09-10T18:50:12.595966+0000","last_active":"2024-09-10T18:51:28.975524+0000","last_peered":"2024-09-10T18:51:28.975524+0000","last_clean":"2024-09-10T18:51:28.975524+0000","last_became_active":"2024-09-10T18:45:20.056869+0000","last_became_peered":"2024-09-10T18:45:20.056869+0000","last_unstale":"2024-09-10T18:51:28.975524+0000","last_undegraded":"2024-09-10T18:51:28.975524+0000","last_fullsized":"2024-09-10T18:51:28.975524+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:50:12.595881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:50:12.595881+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013834673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"56'2","reported_seq":145,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.976183+0000","last_change":"2024-09-10T18:49:20.403122+0000","last_active":"2024-09-10T18:51:28.976183+0000","last_peered":"2024-09-10T18:51:28.976183+0000","last_clean":"2024-09-10T18:51:28.976183+0000","last_became_active":"2024-09-10T18:45:17.963286+0000","last_became_peered":"2024-09-10T18:45:17.963286+0000","last_unstale":"2024-09-10T18:51:28.976183+0000","last_undegraded":"2024-09-10T18:51:28.976183+0000","last_fullsized":"2024-09-10T18:51:28.976183+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-10T18:49:20.403067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:20.403067+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00336061,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":97,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:19.547207+0000","last_change":"2024-09-10T18:50:02.711676+0000","last_active":"2024-09-10T18:51:19.547207+0000","last_peered":"2024-09-10T18:51:19.547207+0000","last_clean":"2024-09-10T18:51:19.547207+0000","last_became_active":"2024-09-10T18:47:36.421151+0000","last_became_peered":"2024-09-10T18:47:36.421151+0000","last_unstale":"2024-09-10T18:51:19.547207+0000","last_undegraded":"2024-09-10T18:51:19.547207+0000","last_fullsized":"2024-09-10T18:51:19.547207+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:02.711594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:02.711594+0000","last_clean_scrub_stamp":"2024-09-10T18:50:02.711594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:23.935286+0000","scrub_duration":0.001786826,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":120,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:20.127218+0000","last_change":"2024-09-10T18:49:52.490607+0000","last_active":"2024-09-10T18:51:20.127218+0000","last_peered":"2024-09-10T18:51:20.127218+0000","last_clean":"2024-09-10T18:51:20.127218+0000","last_became_active":"2024-09-10T18:45:24.077815+0000","last_became_peered":"2024-09-10T18:45:24.077815+0000","last_unstale":"2024-09-10T18:51:20.127218+0000","last_undegraded":"2024-09-10T18:51:20.127218+0000","last_fullsized":"2024-09-10T18:51:20.127218+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:52.490519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:52.490519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.04093488,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"76'13","reported_seq":154,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:19.549347+0000","last_change":"2024-09-10T18:49:58.699452+0000","last_active":"2024-09-10T18:51:19.549347+0000","last_peered":"2024-09-10T18:51:19.549347+0000","last_clean":"2024-09-10T18:51:19.549347+0000","last_became_active":"2024-09-10T18:45:20.420421+0000","last_became_peered":"2024-09-10T18:45:20.420421+0000","last_unstale":"2024-09-10T18:51:19.549347+0000","last_undegraded":"2024-09-10T18:51:19.549347+0000","last_fullsized":"2024-09-10T18:51:19.549347+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'13","last_scrub_stamp":"2024-09-10T18:49:58.699357+0000","last_deep_scrub":"76'13","last_deep_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_clean_scrub_stamp":"2024-09-10T18:49:58.699357+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:27.521366+0000","scrub_duration":0.013859232000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":97,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:19.547383+0000","last_change":"2024-09-10T18:50:07.454468+0000","last_active":"2024-09-10T18:51:19.547383+0000","last_peered":"2024-09-10T18:51:19.547383+0000","last_clean":"2024-09-10T18:51:19.547383+0000","last_became_active":"2024-09-10T18:47:36.409299+0000","last_became_peered":"2024-09-10T18:47:36.409299+0000","last_unstale":"2024-09-10T18:51:19.547383+0000","last_undegraded":"2024-09-10T18:51:19.547383+0000","last_fullsized":"2024-09-10T18:51:19.547383+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:07.454393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:07.454393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:29.900385+0000","scrub_duration":0.014100293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":132,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:20.127779+0000","last_change":"2024-09-10T18:50:57.190078+0000","last_active":"2024-09-10T18:51:20.127779+0000","last_peered":"2024-09-10T18:51:20.127779+0000","last_clean":"2024-09-10T18:51:20.127779+0000","last_became_active":"2024-09-10T18:45:22.069040+0000","last_became_peered":"2024-09-10T18:45:22.069040+0000","last_unstale":"2024-09-10T18:51:20.127779+0000","last_undegraded":"2024-09-10T18:51:20.127779+0000","last_fullsized":"2024-09-10T18:51:20.127779+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:57.189976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:28.442370+0000","last_clean_scrub_stamp":"2024-09-10T18:50:57.189976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:02.018261+0000","scrub_duration":0.016295923,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"63'1","reported_seq":105,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.975346+0000","last_change":"2024-09-10T18:49:49.544530+0000","last_active":"2024-09-10T18:51:28.975346+0000","last_peered":"2024-09-10T18:51:28.975346+0000","last_clean":"2024-09-10T18:51:28.975346+0000","last_became_active":"2024-09-10T18:47:36.406936+0000","last_became_peered":"2024-09-10T18:47:36.406936+0000","last_unstale":"2024-09-10T18:51:28.975346+0000","last_undegraded":"2024-09-10T18:51:28.975346+0000","last_fullsized":"2024-09-10T18:51:28.975346+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-10T18:49:49.544433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:49.544433+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041109190000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":138,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.700170+0000","last_change":"2024-09-10T18:50:52.260051+0000","last_active":"2024-09-10T18:51:28.700170+0000","last_peered":"2024-09-10T18:51:28.700170+0000","last_clean":"2024-09-10T18:51:28.700170+0000","last_became_active":"2024-09-10T18:45:17.964705+0000","last_became_peered":"2024-09-10T18:45:17.964705+0000","last_unstale":"2024-09-10T18:51:28.700170+0000","last_undegraded":"2024-09-10T18:51:28.700170+0000","last_fullsized":"2024-09-10T18:51:28.700170+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:52.259998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:52.259998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:58.126795+0000","scrub_duration":0.0032069059999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'6","reported_seq":133,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:20.127967+0000","last_change":"2024-09-10T18:49:18.363292+0000","last_active":"2024-09-10T18:51:20.127967+0000","last_peered":"2024-09-10T18:51:20.127967+0000","last_clean":"2024-09-10T18:51:20.127967+0000","last_became_active":"2024-09-10T18:45:20.421746+0000","last_became_peered":"2024-09-10T18:45:20.421746+0000","last_unstale":"2024-09-10T18:51:20.127967+0000","last_undegraded":"2024-09-10T18:51:20.127967+0000","last_fullsized":"2024-09-10T18:51:20.127967+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'6","last_scrub_stamp":"2024-09-10T18:49:18.363203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:49:18.363203+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0093609290000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":124,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.543601+0000","last_change":"2024-09-10T18:49:12.996132+0000","last_active":"2024-09-10T18:51:28.543601+0000","last_peered":"2024-09-10T18:51:28.543601+0000","last_clean":"2024-09-10T18:51:28.543601+0000","last_became_active":"2024-09-10T18:45:22.069653+0000","last_became_peered":"2024-09-10T18:45:22.069653+0000","last_unstale":"2024-09-10T18:51:28.543601+0000","last_undegraded":"2024-09-10T18:51:28.543601+0000","last_fullsized":"2024-09-10T18:51:28.543601+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:12.996035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:49:12.996035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0049239360000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":133,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.976574+0000","last_change":"2024-09-10T18:50:43.408152+0000","last_active":"2024-09-10T18:51:28.976574+0000","last_peered":"2024-09-10T18:51:28.976574+0000","last_clean":"2024-09-10T18:51:28.976574+0000","last_became_active":"2024-09-10T18:45:24.264611+0000","last_became_peered":"2024-09-10T18:45:24.264611+0000","last_unstale":"2024-09-10T18:51:28.976574+0000","last_undegraded":"2024-09-10T18:51:28.976574+0000","last_fullsized":"2024-09-10T18:51:28.976574+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:43.408011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:50:43.408011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:11.539468+0000","scrub_duration":0.0026253959999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"63'10","reported_seq":183,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:20.127767+0000","last_change":"2024-09-10T18:49:01.556782+0000","last_active":"2024-09-10T18:51:20.127767+0000","last_peered":"2024-09-10T18:51:20.127767+0000","last_clean":"2024-09-10T18:51:20.127767+0000","last_became_active":"2024-09-10T18:47:36.426260+0000","last_became_peered":"2024-09-10T18:47:36.426260+0000","last_unstale":"2024-09-10T18:51:20.127767+0000","last_undegraded":"2024-09-10T18:51:20.127767+0000","last_fullsized":"2024-09-10T18:51:20.127767+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:49:01.556694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:49:01.556694+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053223699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":138,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.543042+0000","last_change":"2024-09-10T18:50:45.120724+0000","last_active":"2024-09-10T18:51:28.543042+0000","last_peered":"2024-09-10T18:51:28.543042+0000","last_clean":"2024-09-10T18:51:28.543042+0000","last_became_active":"2024-09-10T18:45:17.964870+0000","last_became_peered":"2024-09-10T18:45:17.964870+0000","last_unstale":"2024-09-10T18:51:28.543042+0000","last_undegraded":"2024-09-10T18:51:28.543042+0000","last_fullsized":"2024-09-10T18:51:28.543042+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:45.120550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:45.120550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:00.695839+0000","scrub_duration":0.0055063930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"63'8","reported_seq":216,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.543374+0000","last_change":"2024-09-10T18:50:52.043047+0000","last_active":"2024-09-10T18:51:28.543374+0000","last_peered":"2024-09-10T18:51:28.543374+0000","last_clean":"2024-09-10T18:51:28.543374+0000","last_became_active":"2024-09-10T18:45:22.062940+0000","last_became_peered":"2024-09-10T18:45:22.062940+0000","last_unstale":"2024-09-10T18:51:28.543374+0000","last_undegraded":"2024-09-10T18:51:28.543374+0000","last_fullsized":"2024-09-10T18:51:28.543374+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:50:52.042960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:52.042960+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:11.293055+0000","scrub_duration":0.0090181189999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":130,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:19.549149+0000","last_change":"2024-09-10T18:50:56.767501+0000","last_active":"2024-09-10T18:51:19.549149+0000","last_peered":"2024-09-10T18:51:19.549149+0000","last_clean":"2024-09-10T18:51:19.549149+0000","last_became_active":"2024-09-10T18:45:24.073736+0000","last_became_peered":"2024-09-10T18:45:24.073736+0000","last_unstale":"2024-09-10T18:51:19.549149+0000","last_undegraded":"2024-09-10T18:51:19.549149+0000","last_fullsized":"2024-09-10T18:51:19.549149+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:56.767430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:50:56.767430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:18.035411+0000","scrub_duration":0.021971303000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"63'9","reported_seq":152,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.543474+0000","last_change":"2024-09-10T18:49:56.061777+0000","last_active":"2024-09-10T18:51:28.543474+0000","last_peered":"2024-09-10T18:51:28.543474+0000","last_clean":"2024-09-10T18:51:28.543474+0000","last_became_active":"2024-09-10T18:45:20.421124+0000","last_became_peered":"2024-09-10T18:45:20.421124+0000","last_unstale":"2024-09-10T18:51:28.543474+0000","last_undegraded":"2024-09-10T18:51:28.543474+0000","last_fullsized":"2024-09-10T18:51:28.543474+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:49:56.061694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:49:56.061694+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0042142459999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":136,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:19.548173+0000","last_change":"2024-09-10T18:49:57.653570+0000","last_active":"2024-09-10T18:51:19.548173+0000","last_peered":"2024-09-10T18:51:19.548173+0000","last_clean":"2024-09-10T18:51:19.548173+0000","last_became_active":"2024-09-10T18:45:17.960106+0000","last_became_peered":"2024-09-10T18:45:17.960106+0000","last_unstale":"2024-09-10T18:51:19.548173+0000","last_undegraded":"2024-09-10T18:51:19.548173+0000","last_fullsized":"2024-09-10T18:51:19.548173+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:57.653478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:57.653478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013420596999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"89'10","reported_seq":219,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-10T18:51:23.965571+0000","last_change":"2024-09-10T18:49:55.514077+0000","last_active":"2024-09-10T18:51:23.965571+0000","last_peered":"2024-09-10T18:51:23.965571+0000","last_clean":"2024-09-10T18:51:23.965571+0000","last_became_active":"2024-09-10T18:47:36.424575+0000","last_became_peered":"2024-09-10T18:47:36.424575+0000","last_unstale":"2024-09-10T18:51:23.965571+0000","last_undegraded":"2024-09-10T18:51:23.965571+0000","last_fullsized":"2024-09-10T18:51:23.965571+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'10","last_scrub_stamp":"2024-09-10T18:49:55.513847+0000","last_deep_scrub":"81'9","last_deep_scrub_stamp":"2024-09-10T18:46:54.800790+0000","last_clean_scrub_stamp":"2024-09-10T18:49:55.513847+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0099728100000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":142,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.975839+0000","last_change":"2024-09-10T18:50:55.384653+0000","last_active":"2024-09-10T18:51:28.975839+0000","last_peered":"2024-09-10T18:51:28.975839+0000","last_clean":"2024-09-10T18:51:28.975839+0000","last_became_active":"2024-09-10T18:47:36.414047+0000","last_became_peered":"2024-09-10T18:47:36.414047+0000","last_unstale":"2024-09-10T18:51:28.975839+0000","last_undegraded":"2024-09-10T18:51:28.975839+0000","last_fullsized":"2024-09-10T18:51:28.975839+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:55.384570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_clean_scrub_stamp":"2024-09-10T18:50:55.384570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:06.404917+0000","scrub_duration":0.0079403359999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"63'9","reported_seq":160,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:19.547776+0000","last_change":"2024-09-10T18:49:05.359047+0000","last_active":"2024-09-10T18:51:19.547776+0000","last_peered":"2024-09-10T18:51:19.547776+0000","last_clean":"2024-09-10T18:51:19.547776+0000","last_became_active":"2024-09-10T18:47:36.416484+0000","last_became_peered":"2024-09-10T18:47:36.416484+0000","last_unstale":"2024-09-10T18:51:19.547776+0000","last_undegraded":"2024-09-10T18:51:19.547776+0000","last_fullsized":"2024-09-10T18:51:19.547776+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:49:05.358887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:49:05.358887+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013783619,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":138,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.700537+0000","last_change":"2024-09-10T18:49:04.430421+0000","last_active":"2024-09-10T18:51:28.700537+0000","last_peered":"2024-09-10T18:51:28.700537+0000","last_clean":"2024-09-10T18:51:28.700537+0000","last_became_active":"2024-09-10T18:45:17.962009+0000","last_became_peered":"2024-09-10T18:45:17.962009+0000","last_unstale":"2024-09-10T18:51:28.700537+0000","last_undegraded":"2024-09-10T18:51:28.700537+0000","last_fullsized":"2024-09-10T18:51:28.700537+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:04.430344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:04.430344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025461400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":122,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:20.127214+0000","last_change":"2024-09-10T18:50:03.571896+0000","last_active":"2024-09-10T18:51:20.127214+0000","last_peered":"2024-09-10T18:51:20.127214+0000","last_clean":"2024-09-10T18:51:20.127214+0000","last_became_active":"2024-09-10T18:45:22.072877+0000","last_became_peered":"2024-09-10T18:45:22.072877+0000","last_unstale":"2024-09-10T18:51:20.127214+0000","last_undegraded":"2024-09-10T18:51:20.127214+0000","last_fullsized":"2024-09-10T18:51:20.127214+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:03.571786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:03.571786+0000","last_clean_scrub_stamp":"2024-09-10T18:50:03.571786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0096506809999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":130,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:19.547044+0000","last_change":"2024-09-10T18:50:04.631640+0000","last_active":"2024-09-10T18:51:19.547044+0000","last_peered":"2024-09-10T18:51:19.547044+0000","last_clean":"2024-09-10T18:51:19.547044+0000","last_became_active":"2024-09-10T18:45:24.081050+0000","last_became_peered":"2024-09-10T18:45:24.081050+0000","last_unstale":"2024-09-10T18:51:19.547044+0000","last_undegraded":"2024-09-10T18:51:19.547044+0000","last_fullsized":"2024-09-10T18:51:19.547044+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:04.631558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:04.631558+0000","last_clean_scrub_stamp":"2024-09-10T18:50:04.631558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0085779570000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"63'11","reported_seq":145,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.975543+0000","last_change":"2024-09-10T18:49:56.622986+0000","last_active":"2024-09-10T18:51:28.975543+0000","last_peered":"2024-09-10T18:51:28.975543+0000","last_clean":"2024-09-10T18:51:28.975543+0000","last_became_active":"2024-09-10T18:45:20.056885+0000","last_became_peered":"2024-09-10T18:45:20.056885+0000","last_unstale":"2024-09-10T18:51:28.975543+0000","last_undegraded":"2024-09-10T18:51:28.975543+0000","last_fullsized":"2024-09-10T18:51:28.975543+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:49:56.622879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:49:56.622879+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0077720799999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":136,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:19.548625+0000","last_change":"2024-09-10T18:51:00.029596+0000","last_active":"2024-09-10T18:51:19.548625+0000","last_peered":"2024-09-10T18:51:19.548625+0000","last_clean":"2024-09-10T18:51:19.548625+0000","last_became_active":"2024-09-10T18:45:17.963542+0000","last_became_peered":"2024-09-10T18:45:17.963542+0000","last_unstale":"2024-09-10T18:51:19.548625+0000","last_undegraded":"2024-09-10T18:51:19.548625+0000","last_fullsized":"2024-09-10T18:51:19.548625+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:00.029389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:00.029389+0000","last_clean_scrub_stamp":"2024-09-10T18:51:00.029389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:25.353957+0000","scrub_duration":0.0023207890000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":135,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:19.547693+0000","last_change":"2024-09-10T18:50:45.003161+0000","last_active":"2024-09-10T18:51:19.547693+0000","last_peered":"2024-09-10T18:51:19.547693+0000","last_clean":"2024-09-10T18:51:19.547693+0000","last_became_active":"2024-09-10T18:45:22.070065+0000","last_became_peered":"2024-09-10T18:45:22.070065+0000","last_unstale":"2024-09-10T18:51:19.547693+0000","last_undegraded":"2024-09-10T18:51:19.547693+0000","last_fullsized":"2024-09-10T18:51:19.547693+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:45.002838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:45.002838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:58.280429+0000","scrub_duration":0.017421118999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":120,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:20.126780+0000","last_change":"2024-09-10T18:49:58.479928+0000","last_active":"2024-09-10T18:51:20.126780+0000","last_peered":"2024-09-10T18:51:20.126780+0000","last_clean":"2024-09-10T18:51:20.126780+0000","last_became_active":"2024-09-10T18:45:24.072224+0000","last_became_peered":"2024-09-10T18:45:24.072224+0000","last_unstale":"2024-09-10T18:51:20.126780+0000","last_undegraded":"2024-09-10T18:51:20.126780+0000","last_fullsized":"2024-09-10T18:51:20.126780+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:58.479842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:58.479842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.025012936,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"63'4","reported_seq":140,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:20.126844+0000","last_change":"2024-09-10T18:50:46.171962+0000","last_active":"2024-09-10T18:51:20.126844+0000","last_peered":"2024-09-10T18:51:20.126844+0000","last_clean":"2024-09-10T18:51:20.126844+0000","last_became_active":"2024-09-10T18:45:20.059205+0000","last_became_peered":"2024-09-10T18:45:20.059205+0000","last_unstale":"2024-09-10T18:51:20.126844+0000","last_undegraded":"2024-09-10T18:51:20.126844+0000","last_fullsized":"2024-09-10T18:51:20.126844+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4","last_scrub_stamp":"2024-09-10T18:50:46.171568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:50:46.171568+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:59.994766+0000","scrub_duration":0.0054395399999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":138,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.543685+0000","last_change":"2024-09-10T18:49:50.050306+0000","last_active":"2024-09-10T18:51:28.543685+0000","last_peered":"2024-09-10T18:51:28.543685+0000","last_clean":"2024-09-10T18:51:28.543685+0000","last_became_active":"2024-09-10T18:47:36.415866+0000","last_became_peered":"2024-09-10T18:47:36.415866+0000","last_unstale":"2024-09-10T18:51:28.543685+0000","last_undegraded":"2024-09-10T18:51:28.543685+0000","last_fullsized":"2024-09-10T18:51:28.543685+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:50.050248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:50.050248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0096805610000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":134,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.700051+0000","last_change":"2024-09-10T18:50:06.348857+0000","last_active":"2024-09-10T18:51:28.700051+0000","last_peered":"2024-09-10T18:51:28.700051+0000","last_clean":"2024-09-10T18:51:28.700051+0000","last_became_active":"2024-09-10T18:45:22.072377+0000","last_became_peered":"2024-09-10T18:45:22.072377+0000","last_unstale":"2024-09-10T18:51:28.700051+0000","last_undegraded":"2024-09-10T18:51:28.700051+0000","last_fullsized":"2024-09-10T18:51:28.700051+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:06.348715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_clean_scrub_stamp":"2024-09-10T18:50:06.348715+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0042502529999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":143,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.976065+0000","last_change":"2024-09-10T18:49:59.333755+0000","last_active":"2024-09-10T18:51:28.976065+0000","last_peered":"2024-09-10T18:51:28.976065+0000","last_clean":"2024-09-10T18:51:28.976065+0000","last_became_active":"2024-09-10T18:47:36.416187+0000","last_became_peered":"2024-09-10T18:47:36.416187+0000","last_unstale":"2024-09-10T18:51:28.976065+0000","last_undegraded":"2024-09-10T18:51:28.976065+0000","last_fullsized":"2024-09-10T18:51:28.976065+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:59.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:59.333637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053932670000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"79'16","reported_seq":148,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:20.126928+0000","last_change":"2024-09-10T18:49:03.566236+0000","last_active":"2024-09-10T18:51:20.126928+0000","last_peered":"2024-09-10T18:51:20.126928+0000","last_clean":"2024-09-10T18:51:20.126928+0000","last_became_active":"2024-09-10T18:45:20.421479+0000","last_became_peered":"2024-09-10T18:45:20.421479+0000","last_unstale":"2024-09-10T18:51:20.126928+0000","last_undegraded":"2024-09-10T18:51:20.126928+0000","last_fullsized":"2024-09-10T18:51:20.126928+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"79'16","last_scrub_stamp":"2024-09-10T18:49:03.566152+0000","last_deep_scrub":"79'16","last_deep_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_clean_scrub_stamp":"2024-09-10T18:49:03.566152+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0032760770000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"56'1","reported_seq":159,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.700948+0000","last_change":"2024-09-10T18:50:43.279453+0000","last_active":"2024-09-10T18:51:28.700948+0000","last_peered":"2024-09-10T18:51:28.700948+0000","last_clean":"2024-09-10T18:51:28.700948+0000","last_became_active":"2024-09-10T18:47:36.421985+0000","last_became_peered":"2024-09-10T18:47:36.421985+0000","last_unstale":"2024-09-10T18:51:28.700948+0000","last_undegraded":"2024-09-10T18:51:28.700948+0000","last_fullsized":"2024-09-10T18:51:28.700948+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:50:43.279224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:43.279224+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:04.568803+0000","scrub_duration":0.0048255859999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":86,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:20.126566+0000","last_change":"2024-09-10T18:50:41.057828+0000","last_active":"2024-09-10T18:51:20.126566+0000","last_peered":"2024-09-10T18:51:20.126566+0000","last_clean":"2024-09-10T18:51:20.126566+0000","last_became_active":"2024-09-10T18:47:57.107149+0000","last_became_peered":"2024-09-10T18:47:57.107149+0000","last_unstale":"2024-09-10T18:51:20.126566+0000","last_undegraded":"2024-09-10T18:51:20.126566+0000","last_fullsized":"2024-09-10T18:51:20.126566+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:41.057750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:41.057750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:51.643566+0000","scrub_duration":0.013401967000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":130,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:20.127112+0000","last_change":"2024-09-10T18:50:13.793044+0000","last_active":"2024-09-10T18:51:20.127112+0000","last_peered":"2024-09-10T18:51:20.127112+0000","last_clean":"2024-09-10T18:51:20.127112+0000","last_became_active":"2024-09-10T18:45:24.264818+0000","last_became_peered":"2024-09-10T18:45:24.264818+0000","last_unstale":"2024-09-10T18:51:20.127112+0000","last_undegraded":"2024-09-10T18:51:20.127112+0000","last_fullsized":"2024-09-10T18:51:20.127112+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:13.792946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:50:13.792946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:42.869032+0000","scrub_duration":0.012475749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"78'2","reported_seq":149,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.976979+0000","last_change":"2024-09-10T18:49:52.265525+0000","last_active":"2024-09-10T18:51:28.976979+0000","last_peered":"2024-09-10T18:51:28.976979+0000","last_clean":"2024-09-10T18:51:28.976979+0000","last_became_active":"2024-09-10T18:47:36.418702+0000","last_became_peered":"2024-09-10T18:47:36.418702+0000","last_unstale":"2024-09-10T18:51:28.976979+0000","last_undegraded":"2024-09-10T18:51:28.976979+0000","last_fullsized":"2024-09-10T18:51:28.976979+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'2","last_scrub_stamp":"2024-09-10T18:49:52.265459+0000","last_deep_scrub":"78'2","last_deep_scrub_stamp":"2024-09-10T18:46:37.581130+0000","last_clean_scrub_stamp":"2024-09-10T18:49:52.265459+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012366897999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":138,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.542716+0000","last_change":"2024-09-10T18:51:03.197559+0000","last_active":"2024-09-10T18:51:28.542716+0000","last_peered":"2024-09-10T18:51:28.542716+0000","last_clean":"2024-09-10T18:51:28.542716+0000","last_became_active":"2024-09-10T18:45:17.962108+0000","last_became_peered":"2024-09-10T18:45:17.962108+0000","last_unstale":"2024-09-10T18:51:28.542716+0000","last_undegraded":"2024-09-10T18:51:28.542716+0000","last_fullsized":"2024-09-10T18:51:28.542716+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:03.197480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:03.197480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:15.493158+0000","scrub_duration":0.0024932259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"63'10","reported_seq":149,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:19.549375+0000","last_change":"2024-09-10T18:50:07.754452+0000","last_active":"2024-09-10T18:51:19.549375+0000","last_peered":"2024-09-10T18:51:19.549375+0000","last_clean":"2024-09-10T18:51:19.549375+0000","last_became_active":"2024-09-10T18:45:20.421094+0000","last_became_peered":"2024-09-10T18:45:20.421094+0000","last_unstale":"2024-09-10T18:51:19.549375+0000","last_undegraded":"2024-09-10T18:51:19.549375+0000","last_fullsized":"2024-09-10T18:51:19.549375+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:50:07.754001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:50:07.754001+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:37.540615+0000","scrub_duration":0.019947473,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":219,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.700868+0000","last_change":"2024-09-10T18:49:50.465667+0000","last_active":"2024-09-10T18:51:28.700868+0000","last_peered":"2024-09-10T18:51:28.700868+0000","last_clean":"2024-09-10T18:51:28.700868+0000","last_became_active":"2024-09-10T18:45:22.073281+0000","last_became_peered":"2024-09-10T18:45:22.073281+0000","last_unstale":"2024-09-10T18:51:28.700868+0000","last_undegraded":"2024-09-10T18:51:28.700868+0000","last_fullsized":"2024-09-10T18:51:28.700868+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:49:50.465613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:49:50.465613+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017767359999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":120,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:19.548063+0000","last_change":"2024-09-10T18:49:11.422481+0000","last_active":"2024-09-10T18:51:19.548063+0000","last_peered":"2024-09-10T18:51:19.548063+0000","last_clean":"2024-09-10T18:51:19.548063+0000","last_became_active":"2024-09-10T18:45:24.082318+0000","last_became_peered":"2024-09-10T18:45:24.082318+0000","last_unstale":"2024-09-10T18:51:19.548063+0000","last_undegraded":"2024-09-10T18:51:19.548063+0000","last_fullsized":"2024-09-10T18:51:19.548063+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:11.422279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:03.475299+0000","last_clean_scrub_stamp":"2024-09-10T18:49:11.422279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030495840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":136,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:20.127431+0000","last_change":"2024-09-10T18:51:01.165781+0000","last_active":"2024-09-10T18:51:20.127431+0000","last_peered":"2024-09-10T18:51:20.127431+0000","last_clean":"2024-09-10T18:51:20.127431+0000","last_became_active":"2024-09-10T18:45:17.965707+0000","last_became_peered":"2024-09-10T18:45:17.965707+0000","last_unstale":"2024-09-10T18:51:20.127431+0000","last_undegraded":"2024-09-10T18:51:20.127431+0000","last_fullsized":"2024-09-10T18:51:20.127431+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:01.165584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:01.165584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:28.929394+0000","scrub_duration":0.0071542790000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"63'11","reported_seq":155,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.700943+0000","last_change":"2024-09-10T18:49:58.405230+0000","last_active":"2024-09-10T18:51:28.700943+0000","last_peered":"2024-09-10T18:51:28.700943+0000","last_clean":"2024-09-10T18:51:28.700943+0000","last_became_active":"2024-09-10T18:45:20.058717+0000","last_became_peered":"2024-09-10T18:45:20.058717+0000","last_unstale":"2024-09-10T18:51:28.700943+0000","last_undegraded":"2024-09-10T18:51:28.700943+0000","last_fullsized":"2024-09-10T18:51:28.700943+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:49:58.405145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:49:58.405145+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040739369999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":142,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:19.549260+0000","last_change":"2024-09-10T18:50:54.765172+0000","last_active":"2024-09-10T18:51:19.549260+0000","last_peered":"2024-09-10T18:51:19.549260+0000","last_clean":"2024-09-10T18:51:19.549260+0000","last_became_active":"2024-09-10T18:47:36.418767+0000","last_became_peered":"2024-09-10T18:47:36.418767+0000","last_unstale":"2024-09-10T18:51:19.549260+0000","last_undegraded":"2024-09-10T18:51:19.549260+0000","last_fullsized":"2024-09-10T18:51:19.549260+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:54.764990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:54.764990+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:14.182768+0000","scrub_duration":0.0043824299999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":122,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.975788+0000","last_change":"2024-09-10T18:49:06.423032+0000","last_active":"2024-09-10T18:51:28.975788+0000","last_peered":"2024-09-10T18:51:28.975788+0000","last_clean":"2024-09-10T18:51:28.975788+0000","last_became_active":"2024-09-10T18:45:24.267251+0000","last_became_peered":"2024-09-10T18:45:24.267251+0000","last_unstale":"2024-09-10T18:51:28.975788+0000","last_undegraded":"2024-09-10T18:51:28.975788+0000","last_fullsized":"2024-09-10T18:51:28.975788+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:06.422952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:06.422952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017661879999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"56'1","reported_seq":147,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.700393+0000","last_change":"2024-09-10T18:50:07.341477+0000","last_active":"2024-09-10T18:51:28.700393+0000","last_peered":"2024-09-10T18:51:28.700393+0000","last_clean":"2024-09-10T18:51:28.700393+0000","last_became_active":"2024-09-10T18:45:17.966286+0000","last_became_peered":"2024-09-10T18:45:17.966286+0000","last_unstale":"2024-09-10T18:51:28.700393+0000","last_undegraded":"2024-09-10T18:51:28.700393+0000","last_fullsized":"2024-09-10T18:51:28.700393+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:50:07.341394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:07.341394+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019580940000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"63'15","reported_seq":140,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:19.548361+0000","last_change":"2024-09-10T18:50:00.455229+0000","last_active":"2024-09-10T18:51:19.548361+0000","last_peered":"2024-09-10T18:51:19.548361+0000","last_clean":"2024-09-10T18:51:19.548361+0000","last_became_active":"2024-09-10T18:47:36.424373+0000","last_became_peered":"2024-09-10T18:47:36.424373+0000","last_unstale":"2024-09-10T18:51:19.548361+0000","last_undegraded":"2024-09-10T18:51:19.548361+0000","last_fullsized":"2024-09-10T18:51:19.548361+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:50:00.455005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:50:00.455005+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.018138438999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":122,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:19.548356+0000","last_change":"2024-09-10T18:49:51.367158+0000","last_active":"2024-09-10T18:51:19.548356+0000","last_peered":"2024-09-10T18:51:19.548356+0000","last_clean":"2024-09-10T18:51:19.548356+0000","last_became_active":"2024-09-10T18:45:22.059794+0000","last_became_peered":"2024-09-10T18:45:22.059794+0000","last_unstale":"2024-09-10T18:51:19.548356+0000","last_undegraded":"2024-09-10T18:51:19.548356+0000","last_fullsized":"2024-09-10T18:51:19.548356+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:51.367083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:49:51.367083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051050030000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":120,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:20.127620+0000","last_change":"2024-09-10T18:49:51.464014+0000","last_active":"2024-09-10T18:51:20.127620+0000","last_peered":"2024-09-10T18:51:20.127620+0000","last_clean":"2024-09-10T18:51:20.127620+0000","last_became_active":"2024-09-10T18:45:24.264427+0000","last_became_peered":"2024-09-10T18:45:24.264427+0000","last_unstale":"2024-09-10T18:51:20.127620+0000","last_undegraded":"2024-09-10T18:51:20.127620+0000","last_fullsized":"2024-09-10T18:51:20.127620+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:51.463929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:51.463929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.028103624000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":126,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:20.127595+0000","last_change":"2024-09-10T18:49:14.365927+0000","last_active":"2024-09-10T18:51:20.127595+0000","last_peered":"2024-09-10T18:51:20.127595+0000","last_clean":"2024-09-10T18:51:20.127595+0000","last_became_active":"2024-09-10T18:45:17.963257+0000","last_became_peered":"2024-09-10T18:45:17.963257+0000","last_unstale":"2024-09-10T18:51:20.127595+0000","last_undegraded":"2024-09-10T18:51:20.127595+0000","last_fullsized":"2024-09-10T18:51:20.127595+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:14.365492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:14.365492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002395463,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"63'12","reported_seq":147,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T18:51:20.126458+0000","last_change":"2024-09-10T18:49:13.387562+0000","last_active":"2024-09-10T18:51:20.126458+0000","last_peered":"2024-09-10T18:51:20.126458+0000","last_clean":"2024-09-10T18:51:20.126458+0000","last_became_active":"2024-09-10T18:45:20.058391+0000","last_became_peered":"2024-09-10T18:45:20.058391+0000","last_unstale":"2024-09-10T18:51:20.126458+0000","last_undegraded":"2024-09-10T18:51:20.126458+0000","last_fullsized":"2024-09-10T18:51:20.126458+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:49:13.387425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:49:13.387425+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0091962799999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":144,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-10T18:51:28.975698+0000","last_change":"2024-09-10T18:50:41.560291+0000","last_active":"2024-09-10T18:51:28.975698+0000","last_peered":"2024-09-10T18:51:28.975698+0000","last_clean":"2024-09-10T18:51:28.975698+0000","last_became_active":"2024-09-10T18:47:36.416131+0000","last_became_peered":"2024-09-10T18:47:36.416131+0000","last_unstale":"2024-09-10T18:51:28.975698+0000","last_undegraded":"2024-09-10T18:51:28.975698+0000","last_fullsized":"2024-09-10T18:51:28.975698+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:41.560194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:41.560194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:08.620669+0000","scrub_duration":0.0067616830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":697,"num_read_kb":454,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":1238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":453,"ondisk_log_size":453,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":358,"num_read_kb":353,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":697708,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":40,"num_write":20,"num_write_kb":296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":214680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":891,"ondisk_log_size":891,"up":33,"acting":33,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299471,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":302888,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93442264,"statfs":{"total":95995035648,"available":95684878336,"internally_reserved":0,"allocated":1400832,"data_stored":885948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":1.256,"15min":0.75700000000000001},"min":{"1min":0.21099999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":3.073,"5min":31.805,"15min":31.805},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.93200000000000005,"5min":1.5169999999999999,"15min":0.82899999999999996},"min":{"1min":0.23999999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":6.5010000000000003,"5min":31.212,"15min":31.212},"last":0.68500000000000005}]},{"osd":1,"last update":"Tue Sep 10 18:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.98299999999999998,"15min":0.64600000000000002},"min":{"1min":0.23100000000000001,"5min":0.221,"15min":0.221},"max":{"1min":2.9460000000000002,"5min":21.34,"15min":21.34},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.94799999999999995,"15min":0.66200000000000003},"min":{"1min":0.23599999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.869,"5min":21.248000000000001,"15min":21.248000000000001},"last":0.58399999999999996}]},{"osd":2,"last update":"Tue Sep 10 18:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.625,"15min":0.53200000000000003},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.745,"5min":5.1399999999999997,"15min":5.1399999999999997},"last":0.498},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.72099999999999997,"15min":0.58399999999999996},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.1349999999999998,"5min":8.8859999999999992,"15min":8.8859999999999992},"last":0.55600000000000005}]},{"osd":3,"last update":"Tue Sep 10 18:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.73,"5min":1.4850000000000001,"15min":0.83299999999999996},"min":{"1min":0.251,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":9.3089999999999993,"5min":31.478000000000002,"15min":31.478000000000002},"last":0.70099999999999996},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.6699999999999999,"15min":0.90200000000000002},"min":{"1min":0.29499999999999998,"5min":0.254,"15min":0.254},"max":{"1min":9.1549999999999994,"5min":33.731999999999999,"15min":33.731999999999999},"last":0.76600000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.45700000000000002,"15min":0.40899999999999997},"min":{"1min":0.24099999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.6080000000000001,"5min":2.7559999999999998,"15min":2.7559999999999998},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.379,"5min":0.45300000000000001,"15min":0.40100000000000002},"min":{"1min":0.254,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.79800000000000004,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.45300000000000001}]},{"osd":5,"last update":"Tue Sep 10 18:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.46899999999999997,"15min":0.43099999999999999},"min":{"1min":0.26900000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.5489999999999999,"5min":2.7240000000000002,"15min":2.7240000000000002},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.44700000000000001,"15min":0.42599999999999999},"min":{"1min":0.27100000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.85899999999999999,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.72099999999999997}]},{"osd":6,"last update":"Tue Sep 10 18:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.54700000000000004,"15min":0.48099999999999998},"min":{"1min":0.27200000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.6299999999999999,"5min":2.8199999999999998,"15min":2.8199999999999998},"last":0.754},{"interface":"front","average":{"1min":0.44,"5min":0.53500000000000003,"15min":0.46700000000000003},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71999999999999997,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.629}]}]},{"osd":6,"up_from":43,"seq":184683593812,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":302824,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93442328,"statfs":{"total":95995035648,"available":95684943872,"internally_reserved":0,"allocated":1335296,"data_stored":814995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:50:47 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.518,"15min":0.81100000000000005},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":4.9420000000000002,"5min":46.384999999999998,"15min":46.384999999999998},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.877,"5min":1.4450000000000001,"15min":0.80900000000000005},"min":{"1min":0.245,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":2.5760000000000001,"5min":48.372999999999998,"15min":48.372999999999998},"last":0.48399999999999999}]},{"osd":1,"last update":"Tue Sep 10 18:50:47 2024","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":0.877,"15min":0.61799999999999999},"min":{"1min":0.22600000000000001,"5min":0.219,"15min":0.219},"max":{"1min":2.9049999999999998,"5min":6.1319999999999997,"15min":6.1319999999999997},"last":0.439},{"interface":"front","average":{"1min":0.875,"5min":0.73599999999999999,"15min":0.55200000000000005},"min":{"1min":0.28399999999999997,"5min":0.214,"15min":0.214},"max":{"1min":1.8979999999999999,"5min":3.6280000000000001,"15min":3.6280000000000001},"last":0.41699999999999998}]},{"osd":2,"last update":"Tue Sep 10 18:50:47 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.76100000000000001,"15min":0.57399999999999995},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":4.0119999999999996,"5min":4.4740000000000002,"15min":4.4740000000000002},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.60499999999999998,"15min":0.52800000000000002},"min":{"1min":0.35999999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.327,"5min":2.9900000000000002,"15min":2.9900000000000002},"last":0.53300000000000003}]},{"osd":3,"last update":"Tue Sep 10 18:50:47 2024","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":0.80400000000000005,"15min":0.58999999999999997},"min":{"1min":0.27600000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":4.2110000000000003,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":1.276,"5min":1.0049999999999999,"15min":0.67500000000000004},"min":{"1min":0.36899999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":4.1349999999999998,"5min":5.3079999999999998,"15min":5.3079999999999998},"last":0.63300000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:50:47 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.55900000000000005,"15min":0.45600000000000002},"min":{"1min":0.252,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":2.6920000000000002,"5min":4.2969999999999997,"15min":4.2969999999999997},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.51200000000000001,"15min":0.42599999999999999},"min":{"1min":0.28599999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.365,"5min":4.3209999999999997,"15min":4.3209999999999997},"last":0.64600000000000002}]},{"osd":5,"last update":"Tue Sep 10 18:50:47 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.56299999999999994,"15min":0.46300000000000002},"min":{"1min":0.28799999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":2.371,"5min":4.4210000000000003,"15min":4.4210000000000003},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.57899999999999996,"15min":0.47399999999999998},"min":{"1min":0.33400000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":2.0339999999999998,"5min":4.4500000000000002,"15min":4.4500000000000002},"last":0.66800000000000004}]},{"osd":7,"last update":"Tue Sep 10 18:51:12 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.66800000000000004,"15min":0.53500000000000003},"min":{"1min":0.35599999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":3.1509999999999998,"5min":4.5259999999999998,"15min":4.5259999999999998},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.67200000000000004,"15min":0.55300000000000005},"min":{"1min":0.36099999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.5920000000000001,"5min":4.1299999999999999,"15min":4.1299999999999999},"last":0.69399999999999995}]}]},{"osd":5,"up_from":37,"seq":158913790040,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":307292,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93437860,"statfs":{"total":95995035648,"available":95680368640,"internally_reserved":0,"allocated":1650688,"data_stored":1141599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:51:28 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":1.502,"15min":0.85099999999999998},"min":{"1min":0.23599999999999999,"5min":0.183,"15min":0.183},"max":{"1min":3.6859999999999999,"5min":65.697000000000003,"15min":65.697000000000003},"last":0.496},{"interface":"front","average":{"1min":0.753,"5min":1.5669999999999999,"15min":0.85299999999999998},"min":{"1min":0.29899999999999999,"5min":0.219,"15min":0.219},"max":{"1min":3.0419999999999998,"5min":65.683999999999997,"15min":65.683999999999997},"last":0.53400000000000003}]},{"osd":1,"last update":"Tue Sep 10 18:51:28 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.67800000000000005,"15min":0.59799999999999998},"min":{"1min":0.313,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.0700000000000001,"5min":3.5150000000000001,"15min":3.5150000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.80100000000000005,"15min":0.625},"min":{"1min":0.29299999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.1080000000000001,"5min":6.6989999999999998,"15min":6.6989999999999998},"last":0.57399999999999995}]},{"osd":2,"last update":"Tue Sep 10 18:51:28 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.65400000000000003,"15min":0.56399999999999995},"min":{"1min":0.34999999999999998,"5min":0.249,"15min":0.249},"max":{"1min":2.5449999999999999,"5min":7.8029999999999999,"15min":7.8029999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.72299999999999998,"15min":0.60099999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.052,"5min":6.7210000000000001,"15min":6.7210000000000001},"last":0.63400000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:51:28 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":1.6519999999999999,"15min":0.90100000000000002},"min":{"1min":0.28299999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":4.2450000000000001,"5min":66.593999999999994,"15min":66.593999999999994},"last":0.56200000000000006},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.7490000000000001,"15min":0.94599999999999995},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":6.6870000000000003,"5min":65.939999999999998,"15min":65.939999999999998},"last":0.58699999999999997}]},{"osd":4,"last update":"Tue Sep 10 18:51:28 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.47799999999999998,"15min":0.41399999999999998},"min":{"1min":0.253,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":1.1379999999999999,"5min":2.133,"15min":2.133},"last":0.434},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.498,"15min":0.435},"min":{"1min":0.24399999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":2.5099999999999998,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":0.41299999999999998}]},{"osd":6,"last update":"Tue Sep 10 18:50:51 2024","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":0.75900000000000001,"15min":0.52900000000000003},"min":{"1min":0.29799999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":11.17,"5min":11.17,"15min":11.17},"last":0.48399999999999999},{"interface":"front","average":{"1min":1.413,"5min":0.79800000000000004,"15min":0.57199999999999995},"min":{"1min":0.34999999999999998,"5min":0.33000000000000002,"15min":0.31},"max":{"1min":11.18,"5min":11.18,"15min":11.18},"last":0.51500000000000001}]},{"osd":7,"last update":"Tue Sep 10 18:51:14 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.69599999999999995,"15min":0.55000000000000004},"min":{"1min":0.33100000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.6880000000000002,"5min":10.782999999999999,"15min":10.782999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.67800000000000005,"15min":0.54400000000000004},"min":{"1min":0.31,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.8879999999999999,"5min":11.846,"15min":11.846},"last":0.39200000000000002}]}]},{"osd":4,"up_from":30,"seq":128849018974,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":306716,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93438436,"statfs":{"total":95995035648,"available":95680958464,"internally_reserved":0,"allocated":1064960,"data_stored":551614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:51:11 2024","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.456,"15min":0.83499999999999996},"min":{"1min":0.221,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":4.7050000000000001,"5min":55.776000000000003,"15min":55.776000000000003},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.998,"5min":1.421,"15min":0.83499999999999996},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":3.3260000000000001,"5min":55.737000000000002,"15min":55.737000000000002},"last":0.53300000000000003}]},{"osd":1,"last update":"Tue Sep 10 18:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.86399999999999999,"15min":0.67300000000000004},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":2.9409999999999998,"5min":8.1600000000000001,"15min":8.1600000000000001},"last":0.627},{"interface":"front","average":{"1min":1.01,"5min":0.72899999999999998,"15min":0.59999999999999998},"min":{"1min":0.28000000000000003,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":2.5990000000000002,"5min":2.5990000000000002,"15min":2.5990000000000002},"last":0.64500000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.55100000000000005,"15min":0.54900000000000004},"min":{"1min":0.307,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":2.153,"5min":2.153,"15min":2.6030000000000002},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.56299999999999994,"15min":0.55600000000000005},"min":{"1min":0.27900000000000003,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.4790000000000001,"5min":2.8109999999999999,"15min":2.8109999999999999},"last":0.56000000000000005}]},{"osd":3,"last update":"Tue Sep 10 18:51:11 2024","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.2010000000000001,"15min":0.753},"min":{"1min":0.26000000000000001,"5min":0.246,"15min":0.246},"max":{"1min":3.4009999999999998,"5min":24.957999999999998,"15min":24.957999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":1.093,"15min":0.72399999999999998},"min":{"1min":0.33300000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":2.1789999999999998,"5min":24.940000000000001,"15min":24.940000000000001},"last":0.66800000000000004}]},{"osd":5,"last update":"Tue Sep 10 18:50:33 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.5,"15min":0.39700000000000002},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":2.4449999999999998,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.46600000000000003,"15min":0.39100000000000001},"min":{"1min":0.26900000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":0.54400000000000004}]},{"osd":6,"last update":"Tue Sep 10 18:50:48 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.54900000000000004,"15min":0.46300000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.2149999999999999,"5min":3.5529999999999999,"15min":3.5529999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.60899999999999999,"15min":0.46100000000000002},"min":{"1min":0.309,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":2.3410000000000002,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":0.51500000000000001}]},{"osd":7,"last update":"Tue Sep 10 18:51:18 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.56699999999999995,"15min":0.52300000000000002},"min":{"1min":0.318,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.4359999999999999,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54300000000000004,"15min":0.48099999999999998},"min":{"1min":0.31,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.504,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":0.503}]}]},{"osd":3,"up_from":25,"seq":107374182498,"num_pgs":76,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":311444,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93433708,"statfs":{"total":95995035648,"available":95676116992,"internally_reserved":0,"allocated":1646592,"data_stored":1128755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:50:40 2024","interfaces":[{"interface":"back","average":{"1min":2.585,"5min":1.871,"15min":0.88},"min":{"1min":0.16700000000000001,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":9.1579999999999995,"5min":47.601999999999997,"15min":47.601999999999997},"last":0.40500000000000003},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.6539999999999999,"15min":0.82099999999999995},"min":{"1min":0.34300000000000003,"5min":0.217,"15min":0.217},"max":{"1min":3.7509999999999999,"5min":46.787999999999997,"15min":46.787999999999997},"last":0.224}]},{"osd":1,"last update":"Tue Sep 10 18:50:40 2024","interfaces":[{"interface":"back","average":{"1min":2.2210000000000001,"5min":1.2809999999999999,"15min":0.70099999999999996},"min":{"1min":0.48999999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":5.476,"5min":5.827,"15min":5.827},"last":0.28499999999999998},{"interface":"front","average":{"1min":2.5489999999999999,"5min":1.341,"15min":0.72299999999999998},"min":{"1min":0.245,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":9.4060000000000006,"5min":9.4060000000000006,"15min":9.4060000000000006},"last":0.39400000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:50:40 2024","interfaces":[{"interface":"back","average":{"1min":1.7430000000000001,"5min":0.96399999999999997,"15min":0.61499999999999999},"min":{"1min":0.26500000000000001,"5min":0.246,"15min":0.246},"max":{"1min":5.8220000000000001,"5min":5.8220000000000001,"15min":5.8220000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":1.536,"5min":0.879,"15min":0.58399999999999996},"min":{"1min":0.28000000000000003,"5min":0.248,"15min":0.248},"max":{"1min":7.0979999999999999,"5min":7.0979999999999999,"15min":7.0979999999999999},"last":0.32300000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:51:06 2024","interfaces":[{"interface":"back","average":{"1min":1.6339999999999999,"5min":1.0649999999999999,"15min":0.71599999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":7.1790000000000003,"5min":7.1790000000000003,"15min":7.1790000000000003},"last":0.45200000000000001},{"interface":"front","average":{"1min":1.7989999999999999,"5min":1.0820000000000001,"15min":0.74099999999999999},"min":{"1min":0.315,"5min":0.311,"15min":0.311},"max":{"1min":7.1349999999999998,"5min":7.1349999999999998,"15min":7.1349999999999998},"last":0.51100000000000001}]},{"osd":5,"last update":"Tue Sep 10 18:50:35 2024","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.018,"15min":0.69999999999999996},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":2.976,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":0.439},{"interface":"front","average":{"1min":1.857,"5min":1.022,"15min":0.71599999999999997},"min":{"1min":0.307,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":9.2509999999999994,"5min":9.2509999999999994,"15min":9.2509999999999994},"last":0.46700000000000003}]},{"osd":6,"last update":"Tue Sep 10 18:50:49 2024","interfaces":[{"interface":"back","average":{"1min":2.0129999999999999,"5min":1.1910000000000001,"15min":0.77300000000000002},"min":{"1min":0.32600000000000001,"5min":0.315,"15min":0.315},"max":{"1min":9.3239999999999998,"5min":9.3239999999999998,"15min":9.3239999999999998},"last":0.501},{"interface":"front","average":{"1min":2.077,"5min":1.1619999999999999,"15min":0.75800000000000001},"min":{"1min":0.42899999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":7.1559999999999997,"5min":7.1559999999999997,"15min":7.1559999999999997},"last":0.47599999999999998}]},{"osd":7,"last update":"Tue Sep 10 18:51:22 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":1.038,"15min":0.69999999999999996},"min":{"1min":0.377,"5min":0.375,"15min":0.375},"max":{"1min":1.726,"5min":4.71,"15min":4.71},"last":0.48499999999999999},{"interface":"front","average":{"1min":1.165,"5min":1.1879999999999999,"15min":0.78000000000000003},"min":{"1min":0.40500000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":3.4329999999999998,"5min":9.2949999999999999,"15min":9.2949999999999999},"last":0.434}]}]},{"osd":2,"up_from":0,"seq":77309411431,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":302552,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93442600,"statfs":{"total":95995035648,"available":95685222400,"internally_reserved":0,"allocated":1060864,"data_stored":538970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:50:52 2024","interfaces":[{"interface":"back","average":{"1min":2.1880000000000002,"5min":1.107,"15min":0.58399999999999996},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":13.531000000000001,"5min":13.531000000000001,"15min":13.531000000000001},"last":0.26800000000000002},{"interface":"front","average":{"1min":4.0449999999999999,"5min":1.6419999999999999,"15min":0.77500000000000002},"min":{"1min":0.76000000000000001,"5min":0.252,"15min":0.21199999999999999},"max":{"1min":16.206,"5min":16.206,"15min":16.206},"last":0.34300000000000003}]},{"osd":2,"last update":"Tue Sep 10 18:51:24 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.67600000000000005,"15min":0.46800000000000003},"min":{"1min":0.217,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":2.8159999999999998,"5min":3.8479999999999999,"15min":3.8479999999999999},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.84699999999999998,"15min":0.53800000000000003},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":3.754,"5min":7.093,"15min":7.093},"last":0.38900000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:50:48 2024","interfaces":[{"interface":"back","average":{"1min":2.8090000000000002,"5min":1.381,"15min":0.76600000000000001},"min":{"1min":0.56699999999999995,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":12.714,"5min":12.714,"15min":12.714},"last":0.48299999999999998},{"interface":"front","average":{"1min":2.585,"5min":1.5920000000000001,"15min":0.81399999999999995},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":12.696,"5min":12.696,"15min":12.696},"last":0.38100000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:51:07 2024","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.083,"15min":0.73299999999999998},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":3.9239999999999999,"5min":4.1520000000000001,"15min":4.1520000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.85399999999999998,"15min":0.64100000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":2.8119999999999998,"5min":3.9540000000000002,"15min":3.9540000000000002},"last":0.45600000000000002}]},{"osd":5,"last update":"Tue Sep 10 18:50:26 2024","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":0.88200000000000001,"15min":0.69699999999999995},"min":{"1min":0.33300000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.4100000000000001,"5min":4.5869999999999997,"15min":4.5869999999999997},"last":0.60599999999999998},{"interface":"front","average":{"1min":1.6830000000000001,"5min":1.056,"15min":0.78100000000000003},"min":{"1min":0.32900000000000001,"5min":0.31,"15min":0.31},"max":{"1min":3.6840000000000002,"5min":4.1130000000000004,"15min":4.1130000000000004},"last":0.5}]},{"osd":6,"last update":"Tue Sep 10 18:50:48 2024","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.393,"15min":0.88500000000000001},"min":{"1min":0.34899999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":3.4809999999999999,"5min":38.039999999999999,"15min":38.039999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":1.397,"5min":1.5580000000000001,"15min":0.96899999999999997},"min":{"1min":0.32800000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":3.5910000000000002,"5min":38.100999999999999,"15min":38.100999999999999},"last":0.52500000000000002}]},{"osd":7,"last update":"Tue Sep 10 18:51:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.02,"15min":0.77200000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":3.8279999999999998,"5min":4.6539999999999999,"15min":4.6539999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":0.81100000000000005},"min":{"1min":0.36199999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":3.9700000000000002,"5min":4.1989999999999998,"15min":4.1989999999999998},"last":0.44}]}]},{"osd":0,"up_from":8,"seq":34359738480,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":302536,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93442616,"statfs":{"total":95995035648,"available":95685238784,"internally_reserved":0,"allocated":1044480,"data_stored":538135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 18:51:00 2024","interfaces":[{"interface":"back","average":{"1min":2.3039999999999998,"5min":1.5620000000000001,"15min":0.72499999999999998},"min":{"1min":0.17599999999999999,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":8.4399999999999995,"5min":8.4399999999999995,"15min":8.4399999999999995},"last":0.27300000000000002},{"interface":"front","average":{"1min":2.8340000000000001,"5min":1.5740000000000001,"15min":0.73599999999999999},"min":{"1min":0.23200000000000001,"5min":0.221,"15min":0.187},"max":{"1min":7.9580000000000002,"5min":7.9580000000000002,"15min":7.9580000000000002},"last":0.372}]},{"osd":2,"last update":"Tue Sep 10 18:50:26 2024","interfaces":[{"interface":"back","average":{"1min":1.881,"5min":0.96299999999999997,"15min":0.57899999999999996},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":4.4829999999999997,"5min":4.6509999999999998,"15min":4.6509999999999998},"last":0.47899999999999998},{"interface":"front","average":{"1min":1.2050000000000001,"5min":0.77900000000000003,"15min":0.5},"min":{"1min":0.28000000000000003,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":3.774,"5min":4.6230000000000002,"15min":4.6230000000000002},"last":0.442}]},{"osd":3,"last update":"Tue Sep 10 18:50:41 2024","interfaces":[{"interface":"back","average":{"1min":2.6219999999999999,"5min":1.556,"15min":0.78500000000000003},"min":{"1min":0.48799999999999999,"5min":0.26000000000000001,"15min":0.25700000000000001},"max":{"1min":6.5750000000000002,"5min":8.0269999999999992,"15min":8.0269999999999992},"last":0.312},{"interface":"front","average":{"1min":2.4609999999999999,"5min":1.4039999999999999,"15min":0.74299999999999999},"min":{"1min":0.34100000000000003,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":9.8439999999999994,"5min":9.8439999999999994,"15min":9.8439999999999994},"last":0.34100000000000003}]},{"osd":4,"last update":"Tue Sep 10 18:51:05 2024","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":0.97499999999999998,"15min":0.68400000000000005},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":4.992,"5min":4.992,"15min":4.992},"last":0.53100000000000003},{"interface":"front","average":{"1min":1.6299999999999999,"5min":1.228,"15min":0.77200000000000002},"min":{"1min":0.40200000000000002,"5min":0.318,"15min":0.318},"max":{"1min":5.1379999999999999,"5min":5.1379999999999999,"15min":5.1379999999999999},"last":0.35599999999999998}]},{"osd":5,"last update":"Tue Sep 10 18:50:26 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":0.96499999999999997,"15min":0.69299999999999995},"min":{"1min":0.32800000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":3.6800000000000002,"5min":4.1319999999999997,"15min":4.1319999999999997},"last":0.46000000000000002},{"interface":"front","average":{"1min":1.762,"5min":1.091,"15min":0.72599999999999998},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":4.54,"5min":4.6429999999999998,"15min":4.6429999999999998},"last":0.44900000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:50:55 2024","interfaces":[{"interface":"back","average":{"1min":1.853,"5min":1.256,"15min":0.78700000000000003},"min":{"1min":0.38,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":3.9769999999999999,"5min":4.6660000000000004,"15min":4.6660000000000004},"last":0.43099999999999999},{"interface":"front","average":{"1min":1.4470000000000001,"5min":0.98299999999999998,"15min":0.85199999999999998},"min":{"1min":0.33600000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":3.6499999999999999,"5min":4.1890000000000001,"15min":4.6630000000000003},"last":0.41699999999999998}]},{"osd":7,"last update":"Tue Sep 10 18:51:18 2024","interfaces":[{"interface":"back","average":{"1min":1.8,"5min":1.72,"15min":0.94399999999999995},"min":{"1min":0.34000000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":5.3799999999999999,"5min":49.313000000000002,"15min":49.313000000000002},"last":0.50700000000000001},{"interface":"front","average":{"1min":1.401,"5min":1.784,"15min":0.98999999999999999},"min":{"1min":0.45100000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":3.887,"5min":49.063000000000002,"15min":49.063000000000002},"last":0.46800000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T18:51:35.638 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T18:51:35.638 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:35.661 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:35.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:35.662 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:35.763 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:35.789 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:35.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:35.790 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:35.848 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:35 smithi018 ceph-mon[40196]: 4.19 scrub starts 2024-09-10T18:51:35.848 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:35 smithi018 ceph-mon[40196]: 4.19 scrub ok 2024-09-10T18:51:35.848 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:35 smithi018 ceph-mon[40196]: 6.f scrub starts 2024-09-10T18:51:35.848 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:35 smithi018 ceph-mon[40196]: 6.f scrub ok 2024-09-10T18:51:35.848 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:35 smithi018 ceph-mon[40196]: pgmap v260: 142 pgs: 142 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 776 B/s rd, 0 op/s 2024-09-10T18:51:35.848 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:35 smithi018 ceph-mon[29336]: 4.19 scrub starts 2024-09-10T18:51:35.849 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:35 smithi018 ceph-mon[29336]: 4.19 scrub ok 2024-09-10T18:51:35.849 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:35 smithi018 ceph-mon[29336]: 6.f scrub starts 2024-09-10T18:51:35.849 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:35 smithi018 ceph-mon[29336]: 6.f scrub ok 2024-09-10T18:51:35.849 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:35 smithi018 ceph-mon[29336]: pgmap v260: 142 pgs: 142 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 776 B/s rd, 0 op/s 2024-09-10T18:51:35.890 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:35.915 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:35.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:35.916 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:36.016 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:36.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:35 smithi103 ceph-mon[33163]: 4.19 scrub starts 2024-09-10T18:51:36.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:35 smithi103 ceph-mon[33163]: 4.19 scrub ok 2024-09-10T18:51:36.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:35 smithi103 ceph-mon[33163]: 6.f scrub starts 2024-09-10T18:51:36.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:35 smithi103 ceph-mon[33163]: 6.f scrub ok 2024-09-10T18:51:36.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:35 smithi103 ceph-mon[33163]: pgmap v260: 142 pgs: 142 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 776 B/s rd, 0 op/s 2024-09-10T18:51:36.040 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:36.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:36.041 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:36.142 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:36.164 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:36.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:36.165 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:36.266 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:36.288 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:36.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:36.289 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:36.390 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:36.414 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:36.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:36.415 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:36.516 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:36.518 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-10T18:51:36.540 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:36.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:36.541 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:36.577 INFO:teuthology.orchestra.run.smithi018.stderr:pool 'unique_pool_1' created 2024-09-10T18:51:36.640 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:51:36.641 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:51:36.644 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:36.657 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:51:36.657 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:51:36.657 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:36 smithi018 ceph-mon[40196]: 5.a scrub starts 2024-09-10T18:51:36.657 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:36 smithi018 ceph-mon[40196]: 5.a scrub ok 2024-09-10T18:51:36.658 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:36 smithi018 ceph-mon[40196]: 6.5 scrub starts 2024-09-10T18:51:36.658 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:36 smithi018 ceph-mon[40196]: 6.5 scrub ok 2024-09-10T18:51:36.658 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:36 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/119378099' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]: dispatch 2024-09-10T18:51:36.658 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:36 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]: dispatch 2024-09-10T18:51:36.658 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:36 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:36.658 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:36 smithi018 ceph-mon[29336]: 5.a scrub starts 2024-09-10T18:51:36.658 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:36 smithi018 ceph-mon[29336]: 5.a scrub ok 2024-09-10T18:51:36.658 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:36 smithi018 ceph-mon[29336]: 6.5 scrub starts 2024-09-10T18:51:36.658 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:36 smithi018 ceph-mon[29336]: 6.5 scrub ok 2024-09-10T18:51:36.658 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:36 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/119378099' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]: dispatch 2024-09-10T18:51:36.658 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:36 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]: dispatch 2024-09-10T18:51:36.659 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:36 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:36.667 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:36.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:36.668 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:36.673 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:51:36.673 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:51:36.688 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:51:36.689 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:51:36.705 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:51:36.705 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:51:36.721 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:51:36.721 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:51:36.738 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:51:36.738 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:51:36.754 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:51:36.754 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:51:36.769 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:36.772 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:51:36 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:51:36.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:36 smithi103 ceph-mon[33163]: 5.a scrub starts 2024-09-10T18:51:36.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:36 smithi103 ceph-mon[33163]: 5.a scrub ok 2024-09-10T18:51:36.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:36 smithi103 ceph-mon[33163]: 6.5 scrub starts 2024-09-10T18:51:36.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:36 smithi103 ceph-mon[33163]: 6.5 scrub ok 2024-09-10T18:51:36.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:36 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/119378099' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]: dispatch 2024-09-10T18:51:36.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:36 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]: dispatch 2024-09-10T18:51:36.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:36 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:36.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:36.793 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:36.793 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:36.894 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:36.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:36.918 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:36.918 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:37.018 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:37.041 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:37.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:37.042 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:37.143 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:37.169 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:37.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:37.170 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:37.271 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:37.298 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:37.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:37.299 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:37.400 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:37.425 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:37.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:37.426 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:37.526 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:37.551 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:37.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:37.552 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:37.654 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:37.678 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:37.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:37.680 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:37.780 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:37.803 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:37.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:37.804 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:37.905 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:37.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:37.928 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:37.929 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:37.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:37 smithi018 ceph-mon[40196]: 4.8 scrub starts 2024-09-10T18:51:37.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:37 smithi018 ceph-mon[40196]: 4.8 scrub ok 2024-09-10T18:51:37.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:37 smithi018 ceph-mon[40196]: 6.1e scrub starts 2024-09-10T18:51:37.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:37 smithi018 ceph-mon[40196]: 6.1e scrub ok 2024-09-10T18:51:37.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:37 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]': finished 2024-09-10T18:51:37.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:37 smithi018 ceph-mon[40196]: osdmap e158: 8 total, 8 up, 7 in 2024-09-10T18:51:37.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:37 smithi018 ceph-mon[40196]: 1.7 scrub starts 2024-09-10T18:51:37.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:37 smithi018 ceph-mon[40196]: 1.7 scrub ok 2024-09-10T18:51:37.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:37 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2032495251' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:51:37.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:37 smithi018 ceph-mon[40196]: pgmap v262: 158 pgs: 16 unknown, 142 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:51:37.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:37 smithi018 ceph-mon[29336]: 4.8 scrub starts 2024-09-10T18:51:37.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:37 smithi018 ceph-mon[29336]: 4.8 scrub ok 2024-09-10T18:51:37.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:37 smithi018 ceph-mon[29336]: 6.1e scrub starts 2024-09-10T18:51:37.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:37 smithi018 ceph-mon[29336]: 6.1e scrub ok 2024-09-10T18:51:37.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:37 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]': finished 2024-09-10T18:51:37.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:37 smithi018 ceph-mon[29336]: osdmap e158: 8 total, 8 up, 7 in 2024-09-10T18:51:37.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:37 smithi018 ceph-mon[29336]: 1.7 scrub starts 2024-09-10T18:51:37.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:37 smithi018 ceph-mon[29336]: 1.7 scrub ok 2024-09-10T18:51:37.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:37 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2032495251' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:51:37.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:37 smithi018 ceph-mon[29336]: pgmap v262: 158 pgs: 16 unknown, 142 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:51:38.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:37 smithi103 ceph-mon[33163]: 4.8 scrub starts 2024-09-10T18:51:38.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:37 smithi103 ceph-mon[33163]: 4.8 scrub ok 2024-09-10T18:51:38.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:37 smithi103 ceph-mon[33163]: 6.1e scrub starts 2024-09-10T18:51:38.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:37 smithi103 ceph-mon[33163]: 6.1e scrub ok 2024-09-10T18:51:38.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:37 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]': finished 2024-09-10T18:51:38.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:37 smithi103 ceph-mon[33163]: osdmap e158: 8 total, 8 up, 7 in 2024-09-10T18:51:38.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:37 smithi103 ceph-mon[33163]: 1.7 scrub starts 2024-09-10T18:51:38.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:37 smithi103 ceph-mon[33163]: 1.7 scrub ok 2024-09-10T18:51:38.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:37 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2032495251' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:51:38.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:37 smithi103 ceph-mon[33163]: pgmap v262: 158 pgs: 16 unknown, 142 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:51:38.029 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:38.056 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:38.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:38.057 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:38.100 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd pool application enable unique_pool_1 rados --yes-i-really-mean-it || true 2024-09-10T18:51:38.157 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:38.181 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:38.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:38.182 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:38.283 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:38.307 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:38.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:38.308 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:38.391 INFO:teuthology.orchestra.run.smithi018.stdout:ERROR: (22) Invalid argument 2024-09-10T18:51:38.391 INFO:teuthology.orchestra.run.smithi018.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:51:38.409 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:38.431 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:38.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:38.432 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:38.533 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:38.559 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:38.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:38.560 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:38.661 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:38.685 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:38.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:38.686 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:38.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:38 smithi018 ceph-mon[40196]: 4.11 scrub starts 2024-09-10T18:51:38.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:38 smithi018 ceph-mon[40196]: 4.11 scrub ok 2024-09-10T18:51:38.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:38 smithi018 ceph-mon[40196]: 4.7 scrub starts 2024-09-10T18:51:38.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:38 smithi018 ceph-mon[40196]: 4.7 scrub ok 2024-09-10T18:51:38.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:38 smithi018 ceph-mon[40196]: 5.3 scrub starts 2024-09-10T18:51:38.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:38 smithi018 ceph-mon[40196]: 5.3 scrub ok 2024-09-10T18:51:38.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:38 smithi018 ceph-mon[40196]: 4.c scrub starts 2024-09-10T18:51:38.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:38 smithi018 ceph-mon[40196]: 4.c scrub ok 2024-09-10T18:51:38.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:38 smithi018 ceph-mon[40196]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:51:38.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:38 smithi018 ceph-mon[40196]: osdmap e159: 8 total, 8 up, 7 in 2024-09-10T18:51:38.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:38 smithi018 ceph-mon[29336]: 4.11 scrub starts 2024-09-10T18:51:38.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:38 smithi018 ceph-mon[29336]: 4.11 scrub ok 2024-09-10T18:51:38.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:38 smithi018 ceph-mon[29336]: 4.7 scrub starts 2024-09-10T18:51:38.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:38 smithi018 ceph-mon[29336]: 4.7 scrub ok 2024-09-10T18:51:38.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:38 smithi018 ceph-mon[29336]: 5.3 scrub starts 2024-09-10T18:51:38.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:38 smithi018 ceph-mon[29336]: 5.3 scrub ok 2024-09-10T18:51:38.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:38 smithi018 ceph-mon[29336]: 4.c scrub starts 2024-09-10T18:51:38.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:38 smithi018 ceph-mon[29336]: 4.c scrub ok 2024-09-10T18:51:38.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:38 smithi018 ceph-mon[29336]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:51:38.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:38 smithi018 ceph-mon[29336]: osdmap e159: 8 total, 8 up, 7 in 2024-09-10T18:51:38.728 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:51:38.787 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:38.809 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:38.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:38.810 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:38.910 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:38.933 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:38.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:38.934 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:39.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:38 smithi103 ceph-mon[33163]: 4.11 scrub starts 2024-09-10T18:51:39.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:38 smithi103 ceph-mon[33163]: 4.11 scrub ok 2024-09-10T18:51:39.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:38 smithi103 ceph-mon[33163]: 4.7 scrub starts 2024-09-10T18:51:39.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:38 smithi103 ceph-mon[33163]: 4.7 scrub ok 2024-09-10T18:51:39.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:38 smithi103 ceph-mon[33163]: 5.3 scrub starts 2024-09-10T18:51:39.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:38 smithi103 ceph-mon[33163]: 5.3 scrub ok 2024-09-10T18:51:39.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:38 smithi103 ceph-mon[33163]: 4.c scrub starts 2024-09-10T18:51:39.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:38 smithi103 ceph-mon[33163]: 4.c scrub ok 2024-09-10T18:51:39.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:38 smithi103 ceph-mon[33163]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:51:39.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:38 smithi103 ceph-mon[33163]: osdmap e159: 8 total, 8 up, 7 in 2024-09-10T18:51:39.034 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:39.057 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:39.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:39.058 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:39.158 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:39.182 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:39.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:39.183 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:39.284 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:39.310 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:39.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:39.311 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:39.315 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-10T18:51:39.412 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:39.438 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:39.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:39.439 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:39.539 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:39.564 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:39.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:39.565 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:39.629 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T18:51:39.666 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:39.696 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:39.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:39.697 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:39.798 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:39.824 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:39.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:39.825 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:39.926 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:39.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:39 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:51:39.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:39 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:51:39.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:39 smithi018 ceph-mon[29336]: 6.8 scrub starts 2024-09-10T18:51:39.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:39 smithi018 ceph-mon[29336]: 6.8 scrub ok 2024-09-10T18:51:39.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:39 smithi018 ceph-mon[29336]: osdmap e160: 8 total, 8 up, 7 in 2024-09-10T18:51:39.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:39 smithi018 ceph-mon[29336]: pgmap v265: 158 pgs: 16 unknown, 142 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 915 B/s rd, 0 op/s 2024-09-10T18:51:39.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:39 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:51:39.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:39 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:51:39.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:39 smithi018 ceph-mon[40196]: 6.8 scrub starts 2024-09-10T18:51:39.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:39 smithi018 ceph-mon[40196]: 6.8 scrub ok 2024-09-10T18:51:39.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:39 smithi018 ceph-mon[40196]: osdmap e160: 8 total, 8 up, 7 in 2024-09-10T18:51:39.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:39 smithi018 ceph-mon[40196]: pgmap v265: 158 pgs: 16 unknown, 142 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 915 B/s rd, 0 op/s 2024-09-10T18:51:39.952 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:39.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:39.953 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:40.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:39 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:51:40.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:39 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:51:40.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:39 smithi103 ceph-mon[33163]: 6.8 scrub starts 2024-09-10T18:51:40.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:39 smithi103 ceph-mon[33163]: 6.8 scrub ok 2024-09-10T18:51:40.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:39 smithi103 ceph-mon[33163]: osdmap e160: 8 total, 8 up, 7 in 2024-09-10T18:51:40.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:39 smithi103 ceph-mon[33163]: pgmap v265: 158 pgs: 16 unknown, 142 active+clean; 687 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 915 B/s rd, 0 op/s 2024-09-10T18:51:40.053 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:40.079 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:40.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:40.080 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:40.124 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:51:40.124 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:51:40.181 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:40.207 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:40.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:40.208 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:40.309 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:40.332 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:40.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:40.333 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:40.434 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:40.456 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:40.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:40.457 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:40.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:51:40 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:51:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:51:40.558 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:40.583 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:40.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:40.584 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:40.685 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:40.709 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:40.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:40.710 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:40.811 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:40.833 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:40.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:40.834 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:40.839 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:40 smithi018 ceph-mon[40196]: 5.c scrub starts 2024-09-10T18:51:40.839 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:40 smithi018 ceph-mon[40196]: 5.c scrub ok 2024-09-10T18:51:40.839 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:40 smithi018 ceph-mon[40196]: 4.14 scrub starts 2024-09-10T18:51:40.839 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:40 smithi018 ceph-mon[40196]: 4.14 scrub ok 2024-09-10T18:51:40.839 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:40 smithi018 ceph-mon[40196]: 3.1 scrub starts 2024-09-10T18:51:40.839 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:40 smithi018 ceph-mon[40196]: 3.1 scrub ok 2024-09-10T18:51:40.840 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:40 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:51:40.840 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:40 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:51:40.840 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:40 smithi018 ceph-mon[29336]: 5.c scrub starts 2024-09-10T18:51:40.840 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:40 smithi018 ceph-mon[29336]: 5.c scrub ok 2024-09-10T18:51:40.840 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:40 smithi018 ceph-mon[29336]: 4.14 scrub starts 2024-09-10T18:51:40.840 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:40 smithi018 ceph-mon[29336]: 4.14 scrub ok 2024-09-10T18:51:40.840 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:40 smithi018 ceph-mon[29336]: 3.1 scrub starts 2024-09-10T18:51:40.841 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:40 smithi018 ceph-mon[29336]: 3.1 scrub ok 2024-09-10T18:51:40.841 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:40 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:51:40.841 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:40 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:51:40.935 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:40.962 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:40.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:40.963 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:41.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:40 smithi103 ceph-mon[33163]: 5.c scrub starts 2024-09-10T18:51:41.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:40 smithi103 ceph-mon[33163]: 5.c scrub ok 2024-09-10T18:51:41.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:40 smithi103 ceph-mon[33163]: 4.14 scrub starts 2024-09-10T18:51:41.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:40 smithi103 ceph-mon[33163]: 4.14 scrub ok 2024-09-10T18:51:41.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:40 smithi103 ceph-mon[33163]: 3.1 scrub starts 2024-09-10T18:51:41.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:40 smithi103 ceph-mon[33163]: 3.1 scrub ok 2024-09-10T18:51:41.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:40 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:51:41.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:40 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:51:41.064 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:41.087 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:41.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:41.088 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:41.093 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:51:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:51:40] "GET /metrics HTTP/1.1" 200 223661 "" "Prometheus/2.43.0" 2024-09-10T18:51:41.189 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:41.215 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:41.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:41.216 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:41.316 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:41.340 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:41.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:41.342 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:41.442 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:41.469 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:41.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:41.470 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:41.570 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:41.597 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:41.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:41.598 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:41.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:41 smithi018 ceph-mon[29336]: 6.1d scrub starts 2024-09-10T18:51:41.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:41 smithi018 ceph-mon[29336]: 6.1d scrub ok 2024-09-10T18:51:41.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:41 smithi018 ceph-mon[29336]: 2.1b scrub starts 2024-09-10T18:51:41.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:41 smithi018 ceph-mon[29336]: 2.1b scrub ok 2024-09-10T18:51:41.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:41 smithi018 ceph-mon[29336]: 5.b scrub starts 2024-09-10T18:51:41.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:41 smithi018 ceph-mon[29336]: 5.b scrub ok 2024-09-10T18:51:41.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:41 smithi018 ceph-mon[29336]: 5.4 scrub starts 2024-09-10T18:51:41.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:41 smithi018 ceph-mon[29336]: 5.4 scrub ok 2024-09-10T18:51:41.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:41 smithi018 ceph-mon[29336]: 1.6 scrub starts 2024-09-10T18:51:41.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:41 smithi018 ceph-mon[29336]: 1.6 scrub ok 2024-09-10T18:51:41.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:41 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:51:41.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:41 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:51:41.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:41 smithi018 ceph-mon[29336]: pgmap v266: 158 pgs: 10 unknown, 148 active+clean; 582 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:51:41.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:41 smithi018 ceph-mon[40196]: 6.1d scrub starts 2024-09-10T18:51:41.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:41 smithi018 ceph-mon[40196]: 6.1d scrub ok 2024-09-10T18:51:41.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:41 smithi018 ceph-mon[40196]: 2.1b scrub starts 2024-09-10T18:51:41.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:41 smithi018 ceph-mon[40196]: 2.1b scrub ok 2024-09-10T18:51:41.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:41 smithi018 ceph-mon[40196]: 5.b scrub starts 2024-09-10T18:51:41.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:41 smithi018 ceph-mon[40196]: 5.b scrub ok 2024-09-10T18:51:41.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:41 smithi018 ceph-mon[40196]: 5.4 scrub starts 2024-09-10T18:51:41.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:41 smithi018 ceph-mon[40196]: 5.4 scrub ok 2024-09-10T18:51:41.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:41 smithi018 ceph-mon[40196]: 1.6 scrub starts 2024-09-10T18:51:41.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:41 smithi018 ceph-mon[40196]: 1.6 scrub ok 2024-09-10T18:51:41.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:41 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:51:41.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:41 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:51:41.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:41 smithi018 ceph-mon[40196]: pgmap v266: 158 pgs: 10 unknown, 148 active+clean; 582 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:51:41.698 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:41.721 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:41.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:41.722 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:41.823 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:41.847 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:41.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:41.848 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:41.949 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:41.975 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:41.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:41.976 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:42.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:41 smithi103 ceph-mon[33163]: 6.1d scrub starts 2024-09-10T18:51:42.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:41 smithi103 ceph-mon[33163]: 6.1d scrub ok 2024-09-10T18:51:42.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:41 smithi103 ceph-mon[33163]: 2.1b scrub starts 2024-09-10T18:51:42.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:41 smithi103 ceph-mon[33163]: 2.1b scrub ok 2024-09-10T18:51:42.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:41 smithi103 ceph-mon[33163]: 5.b scrub starts 2024-09-10T18:51:42.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:41 smithi103 ceph-mon[33163]: 5.b scrub ok 2024-09-10T18:51:42.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:41 smithi103 ceph-mon[33163]: 5.4 scrub starts 2024-09-10T18:51:42.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:41 smithi103 ceph-mon[33163]: 5.4 scrub ok 2024-09-10T18:51:42.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:41 smithi103 ceph-mon[33163]: 1.6 scrub starts 2024-09-10T18:51:42.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:41 smithi103 ceph-mon[33163]: 1.6 scrub ok 2024-09-10T18:51:42.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:41 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:51:42.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:41 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:51:42.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:41 smithi103 ceph-mon[33163]: pgmap v266: 158 pgs: 10 unknown, 148 active+clean; 582 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:51:42.077 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:42.101 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:42.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:42.102 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:42.203 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:42.226 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:42.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:42.227 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:42.328 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:42.351 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:42.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:42.352 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:42.453 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:42.476 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:42.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:42.477 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:42.486 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:51:42.578 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:42.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:42.602 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:42.602 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:42.648 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:51:42.648 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:42 smithi018 ceph-mon[40196]: 6.c scrub starts 2024-09-10T18:51:42.649 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:42 smithi018 ceph-mon[40196]: 6.c scrub ok 2024-09-10T18:51:42.649 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:42 smithi018 ceph-mon[40196]: 4.3 scrub starts 2024-09-10T18:51:42.649 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:42 smithi018 ceph-mon[40196]: 4.3 scrub ok 2024-09-10T18:51:42.649 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:42 smithi018 ceph-mon[40196]: osdmap e161: 8 total, 8 up, 7 in 2024-09-10T18:51:42.649 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:42 smithi018 ceph-mon[29336]: 6.c scrub starts 2024-09-10T18:51:42.650 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:42 smithi018 ceph-mon[29336]: 6.c scrub ok 2024-09-10T18:51:42.650 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:42 smithi018 ceph-mon[29336]: 4.3 scrub starts 2024-09-10T18:51:42.650 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:42 smithi018 ceph-mon[29336]: 4.3 scrub ok 2024-09-10T18:51:42.650 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:42 smithi018 ceph-mon[29336]: osdmap e161: 8 total, 8 up, 7 in 2024-09-10T18:51:42.655 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-09-10T18:51:42.704 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:42.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:42.726 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:42.727 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:42.827 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:42.849 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:42.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:42.850 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:42.951 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:42.973 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:42.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:42.974 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:43.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:42 smithi103 ceph-mon[33163]: 6.c scrub starts 2024-09-10T18:51:43.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:42 smithi103 ceph-mon[33163]: 6.c scrub ok 2024-09-10T18:51:43.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:42 smithi103 ceph-mon[33163]: 4.3 scrub starts 2024-09-10T18:51:43.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:42 smithi103 ceph-mon[33163]: 4.3 scrub ok 2024-09-10T18:51:43.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:42 smithi103 ceph-mon[33163]: osdmap e161: 8 total, 8 up, 7 in 2024-09-10T18:51:43.075 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:43.098 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:43.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:43.099 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:43.200 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:43.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:43.223 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:43.223 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:43.324 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:43.351 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:43.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:43.352 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:43.453 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:43.478 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:43.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:43.479 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:43.580 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:43.605 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:43.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:43.606 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:43.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:43 smithi018 ceph-mon[40196]: 2.f scrub starts 2024-09-10T18:51:43.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:43 smithi018 ceph-mon[40196]: 2.f scrub ok 2024-09-10T18:51:43.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:43 smithi018 ceph-mon[40196]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:51:43.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:43 smithi018 ceph-mon[40196]: pgmap v268: 158 pgs: 158 active+clean; 582 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:51:43.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:43 smithi018 ceph-mon[29336]: 2.f scrub starts 2024-09-10T18:51:43.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:43 smithi018 ceph-mon[29336]: 2.f scrub ok 2024-09-10T18:51:43.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:43 smithi018 ceph-mon[29336]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:51:43.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:43 smithi018 ceph-mon[29336]: pgmap v268: 158 pgs: 158 active+clean; 582 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:51:43.707 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:43.729 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:43.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:43.731 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:43.825 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":267,"stamp":"2024-09-10T18:51:41.590463+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":1099,"num_read_kb":851,"num_write":430,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":2371,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1428,"ondisk_log_size":1428,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":467,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":2136724,"kb_used_data":9468,"kb_used_omap":11,"kb_used_meta":2127156,"kb_avail":654079340,"statfs":{"total":671965249536,"available":669777244160,"internally_reserved":0,"allocated":9695232,"data_stored":5873337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11928,"internal_metadata":2178208104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-107340,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-15,"num_read_kb":-35,"num_write":-20,"num_write_kb":-296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":-214680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.021101"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":137,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.576876+0000","last_change":"2024-09-10T18:50:55.243820+0000","last_active":"2024-09-10T18:51:37.576876+0000","last_peered":"2024-09-10T18:51:37.576876+0000","last_clean":"2024-09-10T18:51:37.576876+0000","last_became_active":"2024-09-10T18:45:24.265488+0000","last_became_peered":"2024-09-10T18:45:24.265488+0000","last_unstale":"2024-09-10T18:51:37.576876+0000","last_undegraded":"2024-09-10T18:51:37.576876+0000","last_fullsized":"2024-09-10T18:51:37.576876+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:55.243629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:50:55.243629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:57.394455+0000","scrub_duration":0.0090834450000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":143,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.579823+0000","last_change":"2024-09-10T18:50:52.829851+0000","last_active":"2024-09-10T18:51:37.579823+0000","last_peered":"2024-09-10T18:51:37.579823+0000","last_clean":"2024-09-10T18:51:37.579823+0000","last_became_active":"2024-09-10T18:45:17.959475+0000","last_became_peered":"2024-09-10T18:45:17.959475+0000","last_unstale":"2024-09-10T18:51:37.579823+0000","last_undegraded":"2024-09-10T18:51:37.579823+0000","last_fullsized":"2024-09-10T18:51:37.579823+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:52.829778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.718270+0000","last_clean_scrub_stamp":"2024-09-10T18:50:52.829778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:54.534366+0000","scrub_duration":0.0095184889999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"63'15","reported_seq":192,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.579265+0000","last_change":"2024-09-10T18:51:34.023246+0000","last_active":"2024-09-10T18:51:37.579265+0000","last_peered":"2024-09-10T18:51:37.579265+0000","last_clean":"2024-09-10T18:51:37.579265+0000","last_became_active":"2024-09-10T18:47:36.413635+0000","last_became_peered":"2024-09-10T18:47:36.413635+0000","last_unstale":"2024-09-10T18:51:37.579265+0000","last_undegraded":"2024-09-10T18:51:37.579265+0000","last_fullsized":"2024-09-10T18:51:37.579265+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:51:34.023191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:34.023191+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:41.128415+0000","scrub_duration":0.0027158859999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":138,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.581473+0000","last_change":"2024-09-10T18:50:50.546527+0000","last_active":"2024-09-10T18:51:37.581473+0000","last_peered":"2024-09-10T18:51:37.581473+0000","last_clean":"2024-09-10T18:51:37.581473+0000","last_became_active":"2024-09-10T18:45:22.065619+0000","last_became_peered":"2024-09-10T18:45:22.065619+0000","last_unstale":"2024-09-10T18:51:37.581473+0000","last_undegraded":"2024-09-10T18:51:37.581473+0000","last_fullsized":"2024-09-10T18:51:37.581473+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:50.546473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.057932+0000","last_clean_scrub_stamp":"2024-09-10T18:50:50.546473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:16.728084+0000","scrub_duration":0.0026598759999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":143,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.580334+0000","last_change":"2024-09-10T18:50:58.174634+0000","last_active":"2024-09-10T18:51:37.580334+0000","last_peered":"2024-09-10T18:51:37.580334+0000","last_clean":"2024-09-10T18:51:37.580334+0000","last_became_active":"2024-09-10T18:45:17.966153+0000","last_became_peered":"2024-09-10T18:45:17.966153+0000","last_unstale":"2024-09-10T18:51:37.580334+0000","last_undegraded":"2024-09-10T18:51:37.580334+0000","last_fullsized":"2024-09-10T18:51:37.580334+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:58.174399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:58.174399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:09.810946+0000","scrub_duration":0.012266312,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"83'13","reported_seq":150,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.581600+0000","last_change":"2024-09-10T18:50:04.577468+0000","last_active":"2024-09-10T18:51:37.581600+0000","last_peered":"2024-09-10T18:51:37.581600+0000","last_clean":"2024-09-10T18:51:37.581600+0000","last_became_active":"2024-09-10T18:45:20.061266+0000","last_became_peered":"2024-09-10T18:45:20.061266+0000","last_unstale":"2024-09-10T18:51:37.581600+0000","last_undegraded":"2024-09-10T18:51:37.581600+0000","last_fullsized":"2024-09-10T18:51:37.581600+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'13","last_scrub_stamp":"2024-09-10T18:50:04.577301+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_clean_scrub_stamp":"2024-09-10T18:50:04.577301+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004842224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":129,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.582136+0000","last_change":"2024-09-10T18:49:52.335982+0000","last_active":"2024-09-10T18:51:37.582136+0000","last_peered":"2024-09-10T18:51:37.582136+0000","last_clean":"2024-09-10T18:51:37.582136+0000","last_became_active":"2024-09-10T18:45:22.072822+0000","last_became_peered":"2024-09-10T18:45:22.072822+0000","last_unstale":"2024-09-10T18:51:37.582136+0000","last_undegraded":"2024-09-10T18:51:37.582136+0000","last_fullsized":"2024-09-10T18:51:37.582136+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:52.335893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:49:52.335893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.018709007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":126,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.581711+0000","last_change":"2024-09-10T18:49:53.592412+0000","last_active":"2024-09-10T18:51:37.581711+0000","last_peered":"2024-09-10T18:51:37.581711+0000","last_clean":"2024-09-10T18:51:37.581711+0000","last_became_active":"2024-09-10T18:45:24.081002+0000","last_became_peered":"2024-09-10T18:45:24.081002+0000","last_unstale":"2024-09-10T18:51:37.581711+0000","last_undegraded":"2024-09-10T18:51:37.581711+0000","last_fullsized":"2024-09-10T18:51:37.581711+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:53.592325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:53.592325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040599850000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":143,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.911164+0000","last_change":"2024-09-10T18:49:53.263097+0000","last_active":"2024-09-10T18:51:37.911164+0000","last_peered":"2024-09-10T18:51:37.911164+0000","last_clean":"2024-09-10T18:51:37.911164+0000","last_became_active":"2024-09-10T18:45:17.961097+0000","last_became_peered":"2024-09-10T18:45:17.961097+0000","last_unstale":"2024-09-10T18:51:37.911164+0000","last_undegraded":"2024-09-10T18:51:37.911164+0000","last_fullsized":"2024-09-10T18:51:37.911164+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:53.263009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:53.263009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0088114549999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"63'5","reported_seq":140,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.581786+0000","last_change":"2024-09-10T18:50:01.591311+0000","last_active":"2024-09-10T18:51:37.581786+0000","last_peered":"2024-09-10T18:51:37.581786+0000","last_clean":"2024-09-10T18:51:37.581786+0000","last_became_active":"2024-09-10T18:45:20.057567+0000","last_became_peered":"2024-09-10T18:45:20.057567+0000","last_unstale":"2024-09-10T18:51:37.581786+0000","last_undegraded":"2024-09-10T18:51:37.581786+0000","last_fullsized":"2024-09-10T18:51:37.581786+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-10T18:50:01.591224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:50:01.591224+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020612410000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":149,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.908948+0000","last_change":"2024-09-10T18:51:33.438852+0000","last_active":"2024-09-10T18:51:37.908948+0000","last_peered":"2024-09-10T18:51:37.908948+0000","last_clean":"2024-09-10T18:51:37.908948+0000","last_became_active":"2024-09-10T18:45:22.076575+0000","last_became_peered":"2024-09-10T18:45:22.076575+0000","last_unstale":"2024-09-10T18:51:37.908948+0000","last_undegraded":"2024-09-10T18:51:37.908948+0000","last_fullsized":"2024-09-10T18:51:37.908948+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:33.438763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:36.509971+0000","last_clean_scrub_stamp":"2024-09-10T18:51:33.438763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:57.162571+0000","scrub_duration":0.0019905130000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":136,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.586832+0000","last_change":"2024-09-10T18:49:57.067679+0000","last_active":"2024-09-10T18:51:37.586832+0000","last_peered":"2024-09-10T18:51:37.586832+0000","last_clean":"2024-09-10T18:51:37.586832+0000","last_became_active":"2024-09-10T18:45:24.079534+0000","last_became_peered":"2024-09-10T18:45:24.079534+0000","last_unstale":"2024-09-10T18:51:37.586832+0000","last_undegraded":"2024-09-10T18:51:37.586832+0000","last_fullsized":"2024-09-10T18:51:37.586832+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:57.067604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:57.067604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016642440000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":153,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.909781+0000","last_change":"2024-09-10T18:50:58.526247+0000","last_active":"2024-09-10T18:51:37.909781+0000","last_peered":"2024-09-10T18:51:37.909781+0000","last_clean":"2024-09-10T18:51:37.909781+0000","last_became_active":"2024-09-10T18:47:36.414393+0000","last_became_peered":"2024-09-10T18:47:36.414393+0000","last_unstale":"2024-09-10T18:51:37.909781+0000","last_undegraded":"2024-09-10T18:51:37.909781+0000","last_fullsized":"2024-09-10T18:51:37.909781+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:58.526191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:58.526191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:06.694976+0000","scrub_duration":0.0021021659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"63'9","reported_seq":156,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.581890+0000","last_change":"2024-09-10T18:51:31.494087+0000","last_active":"2024-09-10T18:51:37.581890+0000","last_peered":"2024-09-10T18:51:37.581890+0000","last_clean":"2024-09-10T18:51:37.581890+0000","last_became_active":"2024-09-10T18:45:20.052652+0000","last_became_peered":"2024-09-10T18:45:20.052652+0000","last_unstale":"2024-09-10T18:51:37.581890+0000","last_undegraded":"2024-09-10T18:51:37.581890+0000","last_fullsized":"2024-09-10T18:51:37.581890+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:51:31.494021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:31.494021+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:39.434873+0000","scrub_duration":0.0024074779999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":138,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.587290+0000","last_change":"2024-09-10T18:50:03.047387+0000","last_active":"2024-09-10T18:51:37.587290+0000","last_peered":"2024-09-10T18:51:37.587290+0000","last_clean":"2024-09-10T18:51:37.587290+0000","last_became_active":"2024-09-10T18:45:22.068326+0000","last_became_peered":"2024-09-10T18:45:22.068326+0000","last_unstale":"2024-09-10T18:51:37.587290+0000","last_undegraded":"2024-09-10T18:51:37.587290+0000","last_fullsized":"2024-09-10T18:51:37.587290+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:03.047305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:03.047305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023962509999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":137,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.576761+0000","last_change":"2024-09-10T18:51:03.523117+0000","last_active":"2024-09-10T18:51:37.576761+0000","last_peered":"2024-09-10T18:51:37.576761+0000","last_clean":"2024-09-10T18:51:37.576761+0000","last_became_active":"2024-09-10T18:45:24.081488+0000","last_became_peered":"2024-09-10T18:45:24.081488+0000","last_unstale":"2024-09-10T18:51:37.576761+0000","last_undegraded":"2024-09-10T18:51:37.576761+0000","last_fullsized":"2024-09-10T18:51:37.576761+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:03.523030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:03.523030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:33.287999+0000","scrub_duration":0.002492677,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.588927+0000","last_change":"2024-09-10T18:51:37.588528+0000","last_active":"2024-09-10T18:51:37.588927+0000","last_peered":"2024-09-10T18:51:37.588927+0000","last_clean":"2024-09-10T18:51:37.588927+0000","last_became_active":"2024-09-10T18:51:37.588217+0000","last_became_peered":"2024-09-10T18:51:37.588217+0000","last_unstale":"2024-09-10T18:51:37.588927+0000","last_undegraded":"2024-09-10T18:51:37.588927+0000","last_fullsized":"2024-09-10T18:51:37.588927+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:42.284615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"70'81","reported_seq":55,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.587043+0000","last_change":"2024-09-10T18:50:44.076618+0000","last_active":"2024-09-10T18:51:37.587043+0000","last_peered":"2024-09-10T18:51:37.587043+0000","last_clean":"2024-09-10T18:51:37.587043+0000","last_became_active":"2024-09-10T18:50:08.397233+0000","last_became_peered":"2024-09-10T18:50:08.397233+0000","last_unstale":"2024-09-10T18:51:37.587043+0000","last_undegraded":"2024-09-10T18:51:37.587043+0000","last_fullsized":"2024-09-10T18:51:37.587043+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:50:44.076559+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:50:44.076559+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:50.654888+0000","scrub_duration":0.0035380699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"78'25","reported_seq":204,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.582464+0000","last_change":"2024-09-10T18:50:58.450309+0000","last_active":"2024-09-10T18:51:37.582464+0000","last_peered":"2024-09-10T18:51:37.582464+0000","last_clean":"2024-09-10T18:51:37.582464+0000","last_became_active":"2024-09-10T18:47:36.409300+0000","last_became_peered":"2024-09-10T18:47:36.409300+0000","last_unstale":"2024-09-10T18:51:37.582464+0000","last_undegraded":"2024-09-10T18:51:37.582464+0000","last_fullsized":"2024-09-10T18:51:37.582464+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'25","last_scrub_stamp":"2024-09-10T18:50:58.450203+0000","last_deep_scrub":"78'25","last_deep_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_clean_scrub_stamp":"2024-09-10T18:50:58.450203+0000","objects_scrubbed":8,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:14.332494+0000","scrub_duration":0.0058694890000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":143,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.909532+0000","last_change":"2024-09-10T18:50:01.419644+0000","last_active":"2024-09-10T18:51:37.909532+0000","last_peered":"2024-09-10T18:51:37.909532+0000","last_clean":"2024-09-10T18:51:37.909532+0000","last_became_active":"2024-09-10T18:45:17.963732+0000","last_became_peered":"2024-09-10T18:45:17.963732+0000","last_unstale":"2024-09-10T18:51:37.909532+0000","last_undegraded":"2024-09-10T18:51:37.909532+0000","last_fullsized":"2024-09-10T18:51:37.909532+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:01.419557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:01.419557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003472213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":139,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-10T18:51:39.055657+0000","last_change":"2024-09-10T18:51:39.055614+0000","last_active":"2024-09-10T18:51:39.055657+0000","last_peered":"2024-09-10T18:51:39.055657+0000","last_clean":"2024-09-10T18:51:39.055657+0000","last_became_active":"2024-09-10T18:45:22.071443+0000","last_became_peered":"2024-09-10T18:45:22.071443+0000","last_unstale":"2024-09-10T18:51:39.055657+0000","last_undegraded":"2024-09-10T18:51:39.055657+0000","last_fullsized":"2024-09-10T18:51:39.055657+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:39.055556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:04.512425+0000","last_clean_scrub_stamp":"2024-09-10T18:51:39.055556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:03.733831+0000","scrub_duration":0.0016498730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":93,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.579542+0000","last_change":"2024-09-10T18:51:34.251677+0000","last_active":"2024-09-10T18:51:37.579542+0000","last_peered":"2024-09-10T18:51:37.579542+0000","last_clean":"2024-09-10T18:51:37.579542+0000","last_became_active":"2024-09-10T18:47:57.104433+0000","last_became_peered":"2024-09-10T18:47:57.104433+0000","last_unstale":"2024-09-10T18:51:37.579542+0000","last_undegraded":"2024-09-10T18:51:37.579542+0000","last_fullsized":"2024-09-10T18:51:37.579542+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:34.251618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:34.251618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:39.525376+0000","scrub_duration":0.0041421139999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.588451+0000","last_change":"2024-09-10T18:51:37.588201+0000","last_active":"2024-09-10T18:51:37.588451+0000","last_peered":"2024-09-10T18:51:37.588451+0000","last_clean":"2024-09-10T18:51:37.588451+0000","last_became_active":"2024-09-10T18:51:37.587766+0000","last_became_peered":"2024-09-10T18:51:37.587766+0000","last_unstale":"2024-09-10T18:51:37.588451+0000","last_undegraded":"2024-09-10T18:51:37.588451+0000","last_fullsized":"2024-09-10T18:51:37.588451+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:00.739689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.9","version":"70'81","reported_seq":207,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.909092+0000","last_change":"2024-09-10T18:50:53.567955+0000","last_active":"2024-09-10T18:51:37.909092+0000","last_peered":"2024-09-10T18:51:37.909092+0000","last_clean":"2024-09-10T18:51:37.909092+0000","last_became_active":"2024-09-10T18:50:08.399879+0000","last_became_peered":"2024-09-10T18:50:08.399879+0000","last_unstale":"2024-09-10T18:51:37.909092+0000","last_undegraded":"2024-09-10T18:51:37.909092+0000","last_fullsized":"2024-09-10T18:51:37.909092+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:50:53.567865+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:50:53.567865+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:54.014765+0000","scrub_duration":0.0066605650000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"116'15","reported_seq":160,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.582114+0000","last_change":"2024-09-10T18:51:37.298645+0000","last_active":"2024-09-10T18:51:37.582114+0000","last_peered":"2024-09-10T18:51:37.582114+0000","last_clean":"2024-09-10T18:51:37.582114+0000","last_became_active":"2024-09-10T18:45:20.053428+0000","last_became_peered":"2024-09-10T18:45:20.053428+0000","last_unstale":"2024-09-10T18:51:37.582114+0000","last_undegraded":"2024-09-10T18:51:37.582114+0000","last_fullsized":"2024-09-10T18:51:37.582114+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"116'15","last_scrub_stamp":"2024-09-10T18:51:37.298554+0000","last_deep_scrub":"116'15","last_deep_scrub_stamp":"2024-09-10T18:49:12.374527+0000","last_clean_scrub_stamp":"2024-09-10T18:51:37.298554+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:49.476419+0000","scrub_duration":0.0020143299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":143,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.582503+0000","last_change":"2024-09-10T18:51:03.086045+0000","last_active":"2024-09-10T18:51:37.582503+0000","last_peered":"2024-09-10T18:51:37.582503+0000","last_clean":"2024-09-10T18:51:37.582503+0000","last_became_active":"2024-09-10T18:45:17.966833+0000","last_became_peered":"2024-09-10T18:45:17.966833+0000","last_unstale":"2024-09-10T18:51:37.582503+0000","last_undegraded":"2024-09-10T18:51:37.582503+0000","last_fullsized":"2024-09-10T18:51:37.582503+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:03.085839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:03.085839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:06.125680+0000","scrub_duration":0.0082351739999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"89'9","reported_seq":176,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.909674+0000","last_change":"2024-09-10T18:49:54.247659+0000","last_active":"2024-09-10T18:51:37.909674+0000","last_peered":"2024-09-10T18:51:37.909674+0000","last_clean":"2024-09-10T18:51:37.909674+0000","last_became_active":"2024-09-10T18:47:36.421188+0000","last_became_peered":"2024-09-10T18:47:36.421188+0000","last_unstale":"2024-09-10T18:51:37.909674+0000","last_undegraded":"2024-09-10T18:51:37.909674+0000","last_fullsized":"2024-09-10T18:51:37.909674+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'9","last_scrub_stamp":"2024-09-10T18:49:54.247585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:49:54.247585+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0054286500000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":146,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.582413+0000","last_change":"2024-09-10T18:50:56.384791+0000","last_active":"2024-09-10T18:51:37.582413+0000","last_peered":"2024-09-10T18:51:37.582413+0000","last_clean":"2024-09-10T18:51:37.582413+0000","last_became_active":"2024-09-10T18:47:36.414625+0000","last_became_peered":"2024-09-10T18:47:36.414625+0000","last_unstale":"2024-09-10T18:51:37.582413+0000","last_undegraded":"2024-09-10T18:51:37.582413+0000","last_fullsized":"2024-09-10T18:51:37.582413+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:56.384742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:32.043666+0000","last_clean_scrub_stamp":"2024-09-10T18:50:56.384742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:08.079927+0000","scrub_duration":0.001329467,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.590497+0000","last_change":"2024-09-10T18:51:37.590013+0000","last_active":"2024-09-10T18:51:37.590497+0000","last_peered":"2024-09-10T18:51:37.590497+0000","last_clean":"2024-09-10T18:51:37.590497+0000","last_became_active":"2024-09-10T18:51:37.589730+0000","last_became_peered":"2024-09-10T18:51:37.589730+0000","last_unstale":"2024-09-10T18:51:37.590497+0000","last_undegraded":"2024-09-10T18:51:37.590497+0000","last_fullsized":"2024-09-10T18:51:37.590497+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:45.681857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"70'81","reported_seq":50,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.587234+0000","last_change":"2024-09-10T18:50:40.100350+0000","last_active":"2024-09-10T18:51:37.587234+0000","last_peered":"2024-09-10T18:51:37.587234+0000","last_clean":"2024-09-10T18:51:37.587234+0000","last_became_active":"2024-09-10T18:50:08.398345+0000","last_became_peered":"2024-09-10T18:50:08.398345+0000","last_unstale":"2024-09-10T18:51:37.587234+0000","last_undegraded":"2024-09-10T18:51:37.587234+0000","last_fullsized":"2024-09-10T18:51:37.587234+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:50:40.100261+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:50:40.100261+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:47.324754+0000","scrub_duration":0.0020638319999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"141'22","reported_seq":139,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.589647+0000","last_change":"2024-09-10T18:50:57.119238+0000","last_active":"2024-09-10T18:51:37.589647+0000","last_peered":"2024-09-10T18:51:37.589647+0000","last_clean":"2024-09-10T18:51:37.589647+0000","last_became_active":"2024-09-10T18:46:56.742946+0000","last_became_peered":"2024-09-10T18:46:56.742946+0000","last_unstale":"2024-09-10T18:51:37.589647+0000","last_undegraded":"2024-09-10T18:51:37.589647+0000","last_fullsized":"2024-09-10T18:51:37.589647+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"141'22","last_scrub_stamp":"2024-09-10T18:50:57.119152+0000","last_deep_scrub":"141'22","last_deep_scrub_stamp":"2024-09-10T18:50:57.119152+0000","last_clean_scrub_stamp":"2024-09-10T18:50:57.119152+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:21.923801+0000","scrub_duration":0.020103994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":143,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.582276+0000","last_change":"2024-09-10T18:50:06.455339+0000","last_active":"2024-09-10T18:51:37.582276+0000","last_peered":"2024-09-10T18:51:37.582276+0000","last_clean":"2024-09-10T18:51:37.582276+0000","last_became_active":"2024-09-10T18:45:17.965409+0000","last_became_peered":"2024-09-10T18:45:17.965409+0000","last_unstale":"2024-09-10T18:51:37.582276+0000","last_undegraded":"2024-09-10T18:51:37.582276+0000","last_fullsized":"2024-09-10T18:51:37.582276+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:06.455254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_clean_scrub_stamp":"2024-09-10T18:50:06.455254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.018016590999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"63'8","reported_seq":222,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.582554+0000","last_change":"2024-09-10T18:50:48.530731+0000","last_active":"2024-09-10T18:51:37.582554+0000","last_peered":"2024-09-10T18:51:37.582554+0000","last_clean":"2024-09-10T18:51:37.582554+0000","last_became_active":"2024-09-10T18:45:22.062497+0000","last_became_peered":"2024-09-10T18:45:22.062497+0000","last_unstale":"2024-09-10T18:51:37.582554+0000","last_undegraded":"2024-09-10T18:51:37.582554+0000","last_fullsized":"2024-09-10T18:51:37.582554+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:50:48.530644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:48.530644+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:08.165585+0000","scrub_duration":0.022829841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":136,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.587873+0000","last_change":"2024-09-10T18:50:09.060853+0000","last_active":"2024-09-10T18:51:37.587873+0000","last_peered":"2024-09-10T18:51:37.587873+0000","last_clean":"2024-09-10T18:51:37.587873+0000","last_became_active":"2024-09-10T18:45:24.082362+0000","last_became_peered":"2024-09-10T18:45:24.082362+0000","last_unstale":"2024-09-10T18:51:37.587873+0000","last_undegraded":"2024-09-10T18:51:37.587873+0000","last_fullsized":"2024-09-10T18:51:37.587873+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:09.060766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:50:09.060766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.041565214000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.589568+0000","last_change":"2024-09-10T18:51:37.589268+0000","last_active":"2024-09-10T18:51:37.589568+0000","last_peered":"2024-09-10T18:51:37.589568+0000","last_clean":"2024-09-10T18:51:37.589568+0000","last_became_active":"2024-09-10T18:51:37.588723+0000","last_became_peered":"2024-09-10T18:51:37.588723+0000","last_unstale":"2024-09-10T18:51:37.589568+0000","last_undegraded":"2024-09-10T18:51:37.589568+0000","last_fullsized":"2024-09-10T18:51:37.589568+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:45.854850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"63'11","reported_seq":225,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.582603+0000","last_change":"2024-09-10T18:50:42.581936+0000","last_active":"2024-09-10T18:51:37.582603+0000","last_peered":"2024-09-10T18:51:37.582603+0000","last_clean":"2024-09-10T18:51:37.582603+0000","last_became_active":"2024-09-10T18:47:57.106935+0000","last_became_peered":"2024-09-10T18:47:57.106935+0000","last_unstale":"2024-09-10T18:51:37.582603+0000","last_undegraded":"2024-09-10T18:51:37.582603+0000","last_fullsized":"2024-09-10T18:51:37.582603+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:50:42.581837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:50:42.581837+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:08.935957+0000","scrub_duration":0.0093156460000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":143,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.908903+0000","last_change":"2024-09-10T18:49:51.223357+0000","last_active":"2024-09-10T18:51:37.908903+0000","last_peered":"2024-09-10T18:51:37.908903+0000","last_clean":"2024-09-10T18:51:37.908903+0000","last_became_active":"2024-09-10T18:45:17.966133+0000","last_became_peered":"2024-09-10T18:45:17.966133+0000","last_unstale":"2024-09-10T18:51:37.908903+0000","last_undegraded":"2024-09-10T18:51:37.908903+0000","last_fullsized":"2024-09-10T18:51:37.908903+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:51.223287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:51.223287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002675555,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":138,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.589088+0000","last_change":"2024-09-10T18:50:05.001733+0000","last_active":"2024-09-10T18:51:37.589088+0000","last_peered":"2024-09-10T18:51:37.589088+0000","last_clean":"2024-09-10T18:51:37.589088+0000","last_became_active":"2024-09-10T18:45:22.068918+0000","last_became_peered":"2024-09-10T18:45:22.068918+0000","last_unstale":"2024-09-10T18:51:37.589088+0000","last_undegraded":"2024-09-10T18:51:37.589088+0000","last_fullsized":"2024-09-10T18:51:37.589088+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:05.001646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_clean_scrub_stamp":"2024-09-10T18:50:05.001646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044088299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":137,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-10T18:51:41.073403+0000","last_change":"2024-09-10T18:51:41.073374+0000","last_active":"2024-09-10T18:51:41.073403+0000","last_peered":"2024-09-10T18:51:41.073403+0000","last_clean":"2024-09-10T18:51:41.073403+0000","last_became_active":"2024-09-10T18:45:24.264538+0000","last_became_peered":"2024-09-10T18:45:24.264538+0000","last_unstale":"2024-09-10T18:51:41.073403+0000","last_undegraded":"2024-09-10T18:51:41.073403+0000","last_fullsized":"2024-09-10T18:51:41.073403+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:41.073320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:41.073320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:00.507812+0000","scrub_duration":0.001777551,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.912035+0000","last_change":"2024-09-10T18:51:37.911703+0000","last_active":"2024-09-10T18:51:37.912035+0000","last_peered":"2024-09-10T18:51:37.912035+0000","last_clean":"2024-09-10T18:51:37.912035+0000","last_became_active":"2024-09-10T18:51:37.911394+0000","last_became_peered":"2024-09-10T18:51:37.911394+0000","last_unstale":"2024-09-10T18:51:37.912035+0000","last_undegraded":"2024-09-10T18:51:37.912035+0000","last_fullsized":"2024-09-10T18:51:37.912035+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:59.524921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"70'81","reported_seq":55,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.578427+0000","last_change":"2024-09-10T18:50:59.164503+0000","last_active":"2024-09-10T18:51:37.578427+0000","last_peered":"2024-09-10T18:51:37.578427+0000","last_clean":"2024-09-10T18:51:37.578427+0000","last_became_active":"2024-09-10T18:50:08.399048+0000","last_became_peered":"2024-09-10T18:50:08.399048+0000","last_unstale":"2024-09-10T18:51:37.578427+0000","last_undegraded":"2024-09-10T18:51:37.578427+0000","last_fullsized":"2024-09-10T18:51:37.578427+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:50:59.164417+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:50:59.164417+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:19.822414+0000","scrub_duration":0.0063541700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"143'21","reported_seq":168,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.581783+0000","last_change":"2024-09-10T18:51:02.540313+0000","last_active":"2024-09-10T18:51:37.581783+0000","last_peered":"2024-09-10T18:51:37.581783+0000","last_clean":"2024-09-10T18:51:37.581783+0000","last_became_active":"2024-09-10T18:45:20.052243+0000","last_became_peered":"2024-09-10T18:45:20.052243+0000","last_unstale":"2024-09-10T18:51:37.581783+0000","last_undegraded":"2024-09-10T18:51:37.581783+0000","last_fullsized":"2024-09-10T18:51:37.581783+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"143'21","last_scrub_stamp":"2024-09-10T18:51:02.540216+0000","last_deep_scrub":"143'21","last_deep_scrub_stamp":"2024-09-10T18:51:02.540216+0000","last_clean_scrub_stamp":"2024-09-10T18:51:02.540216+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:26.124099+0000","scrub_duration":0.0060974410000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":152,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.588305+0000","last_change":"2024-09-10T18:49:59.397785+0000","last_active":"2024-09-10T18:51:37.588305+0000","last_peered":"2024-09-10T18:51:37.588305+0000","last_clean":"2024-09-10T18:51:37.588305+0000","last_became_active":"2024-09-10T18:47:36.424466+0000","last_became_peered":"2024-09-10T18:47:36.424466+0000","last_unstale":"2024-09-10T18:51:37.588305+0000","last_undegraded":"2024-09-10T18:51:37.588305+0000","last_fullsized":"2024-09-10T18:51:37.588305+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:59.397734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_clean_scrub_stamp":"2024-09-10T18:49:59.397734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00244567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":224,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-10T18:51:38.594247+0000","last_change":"2024-09-10T18:50:53.293010+0000","last_active":"2024-09-10T18:51:38.594247+0000","last_peered":"2024-09-10T18:51:38.594247+0000","last_clean":"2024-09-10T18:51:38.594247+0000","last_became_active":"2024-09-10T18:45:22.074540+0000","last_became_peered":"2024-09-10T18:45:22.074540+0000","last_unstale":"2024-09-10T18:51:38.594247+0000","last_undegraded":"2024-09-10T18:51:38.594247+0000","last_fullsized":"2024-09-10T18:51:38.594247+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:50:53.292838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:53.292838+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:07.773584+0000","scrub_duration":0.018341929999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":157,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.909035+0000","last_change":"2024-09-10T18:51:01.463186+0000","last_active":"2024-09-10T18:51:37.909035+0000","last_peered":"2024-09-10T18:51:37.909035+0000","last_clean":"2024-09-10T18:51:37.909035+0000","last_became_active":"2024-09-10T18:47:36.414665+0000","last_became_peered":"2024-09-10T18:47:36.414665+0000","last_unstale":"2024-09-10T18:51:37.909035+0000","last_undegraded":"2024-09-10T18:51:37.909035+0000","last_fullsized":"2024-09-10T18:51:37.909035+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:01.463123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:01.463123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:05.342543+0000","scrub_duration":0.0050178740000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.909721+0000","last_change":"2024-09-10T18:51:37.909451+0000","last_active":"2024-09-10T18:51:37.909721+0000","last_peered":"2024-09-10T18:51:37.909721+0000","last_clean":"2024-09-10T18:51:37.909721+0000","last_became_active":"2024-09-10T18:51:37.909136+0000","last_became_peered":"2024-09-10T18:51:37.909136+0000","last_unstale":"2024-09-10T18:51:37.909721+0000","last_undegraded":"2024-09-10T18:51:37.909721+0000","last_fullsized":"2024-09-10T18:51:37.909721+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:03.096434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"70'81","reported_seq":50,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.582012+0000","last_change":"2024-09-10T18:51:01.490009+0000","last_active":"2024-09-10T18:51:37.582012+0000","last_peered":"2024-09-10T18:51:37.582012+0000","last_clean":"2024-09-10T18:51:37.582012+0000","last_became_active":"2024-09-10T18:50:08.398081+0000","last_became_peered":"2024-09-10T18:50:08.398081+0000","last_unstale":"2024-09-10T18:51:37.582012+0000","last_undegraded":"2024-09-10T18:51:37.582012+0000","last_fullsized":"2024-09-10T18:51:37.582012+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:51:01.489919+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:51:01.489919+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:10.385199+0000","scrub_duration":0.0014613810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"63'18","reported_seq":173,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.576908+0000","last_change":"2024-09-10T18:50:44.111437+0000","last_active":"2024-09-10T18:51:37.576908+0000","last_peered":"2024-09-10T18:51:37.576908+0000","last_clean":"2024-09-10T18:51:37.576908+0000","last_became_active":"2024-09-10T18:45:20.420937+0000","last_became_peered":"2024-09-10T18:45:20.420937+0000","last_unstale":"2024-09-10T18:51:37.576908+0000","last_undegraded":"2024-09-10T18:51:37.576908+0000","last_fullsized":"2024-09-10T18:51:37.576908+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'18","last_scrub_stamp":"2024-09-10T18:50:44.111343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:50:44.111343+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:06.271060+0000","scrub_duration":0.013021517999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":133,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.583204+0000","last_change":"2024-09-10T18:49:23.336052+0000","last_active":"2024-09-10T18:51:37.583204+0000","last_peered":"2024-09-10T18:51:37.583204+0000","last_clean":"2024-09-10T18:51:37.583204+0000","last_became_active":"2024-09-10T18:45:17.961695+0000","last_became_peered":"2024-09-10T18:45:17.961695+0000","last_unstale":"2024-09-10T18:51:37.583204+0000","last_undegraded":"2024-09-10T18:51:37.583204+0000","last_fullsized":"2024-09-10T18:51:37.583204+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:23.335968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:23.335968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056344669999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":138,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.581640+0000","last_change":"2024-09-10T18:50:43.554137+0000","last_active":"2024-09-10T18:51:37.581640+0000","last_peered":"2024-09-10T18:51:37.581640+0000","last_clean":"2024-09-10T18:51:37.581640+0000","last_became_active":"2024-09-10T18:45:22.074012+0000","last_became_peered":"2024-09-10T18:45:22.074012+0000","last_unstale":"2024-09-10T18:51:37.581640+0000","last_undegraded":"2024-09-10T18:51:37.581640+0000","last_fullsized":"2024-09-10T18:51:37.581640+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:43.554063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:43.554063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:07.347038+0000","scrub_duration":0.0065283629999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":136,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.581810+0000","last_change":"2024-09-10T18:50:03.534693+0000","last_active":"2024-09-10T18:51:37.581810+0000","last_peered":"2024-09-10T18:51:37.581810+0000","last_clean":"2024-09-10T18:51:37.581810+0000","last_became_active":"2024-09-10T18:47:36.410836+0000","last_became_peered":"2024-09-10T18:47:36.410836+0000","last_unstale":"2024-09-10T18:51:37.581810+0000","last_undegraded":"2024-09-10T18:51:37.581810+0000","last_fullsized":"2024-09-10T18:51:37.581810+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:03.534613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_clean_scrub_stamp":"2024-09-10T18:50:03.534613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010201148,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.584537+0000","last_change":"2024-09-10T18:51:37.584352+0000","last_active":"2024-09-10T18:51:37.584537+0000","last_peered":"2024-09-10T18:51:37.584537+0000","last_clean":"2024-09-10T18:51:37.584537+0000","last_became_active":"2024-09-10T18:51:37.583787+0000","last_became_peered":"2024-09-10T18:51:37.583787+0000","last_unstale":"2024-09-10T18:51:37.584537+0000","last_undegraded":"2024-09-10T18:51:37.584537+0000","last_fullsized":"2024-09-10T18:51:37.584537+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:38.451494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"70'81","reported_seq":118,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-10T18:51:40.561177+0000","last_change":"2024-09-10T18:51:40.561133+0000","last_active":"2024-09-10T18:51:40.561177+0000","last_peered":"2024-09-10T18:51:40.561177+0000","last_clean":"2024-09-10T18:51:40.561177+0000","last_became_active":"2024-09-10T18:50:08.405424+0000","last_became_peered":"2024-09-10T18:50:08.405424+0000","last_unstale":"2024-09-10T18:51:40.561177+0000","last_undegraded":"2024-09-10T18:51:40.561177+0000","last_fullsized":"2024-09-10T18:51:40.561177+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:51:40.561066+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:51:40.561066+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:47.503438+0000","scrub_duration":0.001694397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"63'19","reported_seq":187,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-10T18:51:41.161277+0000","last_change":"2024-09-10T18:51:41.161251+0000","last_active":"2024-09-10T18:51:41.161277+0000","last_peered":"2024-09-10T18:51:41.161277+0000","last_clean":"2024-09-10T18:51:41.161277+0000","last_became_active":"2024-09-10T18:45:20.421046+0000","last_became_peered":"2024-09-10T18:45:20.421046+0000","last_unstale":"2024-09-10T18:51:41.161277+0000","last_undegraded":"2024-09-10T18:51:41.161277+0000","last_fullsized":"2024-09-10T18:51:41.161277+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:51:41.161196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:41.161196+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:57.195830+0000","scrub_duration":0.0023699070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":143,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.911262+0000","last_change":"2024-09-10T18:50:06.454269+0000","last_active":"2024-09-10T18:51:37.911262+0000","last_peered":"2024-09-10T18:51:37.911262+0000","last_clean":"2024-09-10T18:51:37.911262+0000","last_became_active":"2024-09-10T18:45:17.960226+0000","last_became_peered":"2024-09-10T18:45:17.960226+0000","last_unstale":"2024-09-10T18:51:37.911262+0000","last_undegraded":"2024-09-10T18:51:37.911262+0000","last_fullsized":"2024-09-10T18:51:37.911262+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:06.454124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:06.454124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014135569000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":138,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.585879+0000","last_change":"2024-09-10T18:50:08.326814+0000","last_active":"2024-09-10T18:51:37.585879+0000","last_peered":"2024-09-10T18:51:37.585879+0000","last_clean":"2024-09-10T18:51:37.585879+0000","last_became_active":"2024-09-10T18:45:22.066148+0000","last_became_peered":"2024-09-10T18:45:22.066148+0000","last_unstale":"2024-09-10T18:51:37.585879+0000","last_undegraded":"2024-09-10T18:51:37.585879+0000","last_fullsized":"2024-09-10T18:51:37.585879+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:08.326738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:08.326738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024862109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":137,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.582030+0000","last_change":"2024-09-10T18:49:18.281313+0000","last_active":"2024-09-10T18:51:37.582030+0000","last_peered":"2024-09-10T18:51:37.582030+0000","last_clean":"2024-09-10T18:51:37.582030+0000","last_became_active":"2024-09-10T18:47:36.422295+0000","last_became_peered":"2024-09-10T18:47:36.422295+0000","last_unstale":"2024-09-10T18:51:37.582030+0000","last_undegraded":"2024-09-10T18:51:37.582030+0000","last_fullsized":"2024-09-10T18:51:37.582030+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:18.281230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:49:18.281230+0000","last_clean_scrub_stamp":"2024-09-10T18:49:18.281230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0066273570000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.910207+0000","last_change":"2024-09-10T18:51:37.909961+0000","last_active":"2024-09-10T18:51:37.910207+0000","last_peered":"2024-09-10T18:51:37.910207+0000","last_clean":"2024-09-10T18:51:37.910207+0000","last_became_active":"2024-09-10T18:51:37.909734+0000","last_became_peered":"2024-09-10T18:51:37.909734+0000","last_unstale":"2024-09-10T18:51:37.910207+0000","last_undegraded":"2024-09-10T18:51:37.910207+0000","last_fullsized":"2024-09-10T18:51:37.910207+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:50.174570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"70'81","reported_seq":118,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.587806+0000","last_change":"2024-09-10T18:51:36.604009+0000","last_active":"2024-09-10T18:51:37.587806+0000","last_peered":"2024-09-10T18:51:37.587806+0000","last_clean":"2024-09-10T18:51:37.587806+0000","last_became_active":"2024-09-10T18:50:08.397123+0000","last_became_peered":"2024-09-10T18:50:08.397123+0000","last_unstale":"2024-09-10T18:51:37.587806+0000","last_undegraded":"2024-09-10T18:51:37.587806+0000","last_fullsized":"2024-09-10T18:51:37.587806+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:51:36.603950+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.603950+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:41.125691+0000","scrub_duration":0.0013973970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"83'15","reported_seq":161,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.582430+0000","last_change":"2024-09-10T18:50:09.557013+0000","last_active":"2024-09-10T18:51:37.582430+0000","last_peered":"2024-09-10T18:51:37.582430+0000","last_clean":"2024-09-10T18:51:37.582430+0000","last_became_active":"2024-09-10T18:45:20.057204+0000","last_became_peered":"2024-09-10T18:45:20.057204+0000","last_unstale":"2024-09-10T18:51:37.582430+0000","last_undegraded":"2024-09-10T18:51:37.582430+0000","last_fullsized":"2024-09-10T18:51:37.582430+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'15","last_scrub_stamp":"2024-09-10T18:50:09.556916+0000","last_deep_scrub":"83'15","last_deep_scrub_stamp":"2024-09-10T18:46:58.436473+0000","last_clean_scrub_stamp":"2024-09-10T18:50:09.556916+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.01222707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":143,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.582887+0000","last_change":"2024-09-10T18:51:05.140273+0000","last_active":"2024-09-10T18:51:37.582887+0000","last_peered":"2024-09-10T18:51:37.582887+0000","last_clean":"2024-09-10T18:51:37.582887+0000","last_became_active":"2024-09-10T18:45:17.966632+0000","last_became_peered":"2024-09-10T18:45:17.966632+0000","last_unstale":"2024-09-10T18:51:37.582887+0000","last_undegraded":"2024-09-10T18:51:37.582887+0000","last_fullsized":"2024-09-10T18:51:37.582887+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:05.140027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:05.140027+0000","last_clean_scrub_stamp":"2024-09-10T18:51:05.140027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:28.134498+0000","scrub_duration":0.0085472780000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"63'8","reported_seq":224,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-10T18:51:38.582620+0000","last_change":"2024-09-10T18:51:37.206285+0000","last_active":"2024-09-10T18:51:38.582620+0000","last_peered":"2024-09-10T18:51:38.582620+0000","last_clean":"2024-09-10T18:51:38.582620+0000","last_became_active":"2024-09-10T18:45:22.063326+0000","last_became_peered":"2024-09-10T18:45:22.063326+0000","last_unstale":"2024-09-10T18:51:38.582620+0000","last_undegraded":"2024-09-10T18:51:38.582620+0000","last_fullsized":"2024-09-10T18:51:38.582620+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:51:37.206224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:37.206224+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:06.464879+0000","scrub_duration":0.001760707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":147,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.576595+0000","last_change":"2024-09-10T18:51:01.618543+0000","last_active":"2024-09-10T18:51:37.576595+0000","last_peered":"2024-09-10T18:51:37.576595+0000","last_clean":"2024-09-10T18:51:37.576595+0000","last_became_active":"2024-09-10T18:47:36.412111+0000","last_became_peered":"2024-09-10T18:47:36.412111+0000","last_unstale":"2024-09-10T18:51:37.576595+0000","last_undegraded":"2024-09-10T18:51:37.576595+0000","last_fullsized":"2024-09-10T18:51:37.576595+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:01.618458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:01.618458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:21.627693+0000","scrub_duration":0.0072066109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.912046+0000","last_change":"2024-09-10T18:51:37.910891+0000","last_active":"2024-09-10T18:51:37.912046+0000","last_peered":"2024-09-10T18:51:37.912046+0000","last_clean":"2024-09-10T18:51:37.912046+0000","last_became_active":"2024-09-10T18:51:37.910355+0000","last_became_peered":"2024-09-10T18:51:37.910355+0000","last_unstale":"2024-09-10T18:51:37.912046+0000","last_undegraded":"2024-09-10T18:51:37.912046+0000","last_fullsized":"2024-09-10T18:51:37.912046+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:41.408841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"70'81","reported_seq":50,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.586541+0000","last_change":"2024-09-10T18:50:43.117818+0000","last_active":"2024-09-10T18:51:37.586541+0000","last_peered":"2024-09-10T18:51:37.586541+0000","last_clean":"2024-09-10T18:51:37.586541+0000","last_became_active":"2024-09-10T18:50:08.399074+0000","last_became_peered":"2024-09-10T18:50:08.399074+0000","last_unstale":"2024-09-10T18:51:37.586541+0000","last_undegraded":"2024-09-10T18:51:37.586541+0000","last_fullsized":"2024-09-10T18:51:37.586541+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:50:43.117728+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:50:43.117728+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:52.214303+0000","scrub_duration":0.0039521169999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"89'29","reported_seq":298,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-10T18:51:38.581911+0000","last_change":"2024-09-10T18:50:04.462642+0000","last_active":"2024-09-10T18:51:38.581911+0000","last_peered":"2024-09-10T18:51:38.581911+0000","last_clean":"2024-09-10T18:51:38.581911+0000","last_became_active":"2024-09-10T18:47:36.420185+0000","last_became_peered":"2024-09-10T18:47:36.420185+0000","last_unstale":"2024-09-10T18:51:38.581911+0000","last_undegraded":"2024-09-10T18:51:38.581911+0000","last_fullsized":"2024-09-10T18:51:38.581911+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'29","last_scrub_stamp":"2024-09-10T18:50:04.459912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:50:04.459912+0000","objects_scrubbed":10,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0097323310000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":142,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.587694+0000","last_change":"2024-09-10T18:50:12.982700+0000","last_active":"2024-09-10T18:51:37.587694+0000","last_peered":"2024-09-10T18:51:37.587694+0000","last_clean":"2024-09-10T18:51:37.587694+0000","last_became_active":"2024-09-10T18:45:17.964494+0000","last_became_peered":"2024-09-10T18:45:17.964494+0000","last_unstale":"2024-09-10T18:51:37.587694+0000","last_undegraded":"2024-09-10T18:51:37.587694+0000","last_fullsized":"2024-09-10T18:51:37.587694+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:12.982599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:12.982599+0000","last_clean_scrub_stamp":"2024-09-10T18:50:12.982599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048150440000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":93,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.587782+0000","last_change":"2024-09-10T18:51:01.357880+0000","last_active":"2024-09-10T18:51:37.587782+0000","last_peered":"2024-09-10T18:51:37.587782+0000","last_clean":"2024-09-10T18:51:37.587782+0000","last_became_active":"2024-09-10T18:47:57.107478+0000","last_became_peered":"2024-09-10T18:47:57.107478+0000","last_unstale":"2024-09-10T18:51:37.587782+0000","last_undegraded":"2024-09-10T18:51:37.587782+0000","last_fullsized":"2024-09-10T18:51:37.587782+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:01.357830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:01.357830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:28.477016+0000","scrub_duration":0.002118796,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":127,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.580087+0000","last_change":"2024-09-10T18:49:57.442744+0000","last_active":"2024-09-10T18:51:37.580087+0000","last_peered":"2024-09-10T18:51:37.580087+0000","last_clean":"2024-09-10T18:51:37.580087+0000","last_became_active":"2024-09-10T18:45:24.073307+0000","last_became_peered":"2024-09-10T18:45:24.073307+0000","last_unstale":"2024-09-10T18:51:37.580087+0000","last_undegraded":"2024-09-10T18:51:37.580087+0000","last_fullsized":"2024-09-10T18:51:37.580087+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:57.442512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:22.891084+0000","last_clean_scrub_stamp":"2024-09-10T18:49:57.442512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0050025080000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.583785+0000","last_change":"2024-09-10T18:51:37.583541+0000","last_active":"2024-09-10T18:51:37.583785+0000","last_peered":"2024-09-10T18:51:37.583785+0000","last_clean":"2024-09-10T18:51:37.583785+0000","last_became_active":"2024-09-10T18:51:37.583248+0000","last_became_peered":"2024-09-10T18:51:37.583248+0000","last_unstale":"2024-09-10T18:51:37.583785+0000","last_undegraded":"2024-09-10T18:51:37.583785+0000","last_fullsized":"2024-09-10T18:51:37.583785+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:01.429540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"70'81","reported_seq":207,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.909388+0000","last_change":"2024-09-10T18:50:50.506911+0000","last_active":"2024-09-10T18:51:37.909388+0000","last_peered":"2024-09-10T18:51:37.909388+0000","last_clean":"2024-09-10T18:51:37.909388+0000","last_became_active":"2024-09-10T18:50:08.397665+0000","last_became_peered":"2024-09-10T18:50:08.397665+0000","last_unstale":"2024-09-10T18:51:37.909388+0000","last_undegraded":"2024-09-10T18:51:37.909388+0000","last_fullsized":"2024-09-10T18:51:37.909388+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:50:50.506769+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:50:50.506769+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:53.715025+0000","scrub_duration":0.0043821199999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"84'18","reported_seq":173,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.583450+0000","last_change":"2024-09-10T18:51:37.059537+0000","last_active":"2024-09-10T18:51:37.583450+0000","last_peered":"2024-09-10T18:51:37.583450+0000","last_clean":"2024-09-10T18:51:37.583450+0000","last_became_active":"2024-09-10T18:45:20.058435+0000","last_became_peered":"2024-09-10T18:45:20.058435+0000","last_unstale":"2024-09-10T18:51:37.583450+0000","last_undegraded":"2024-09-10T18:51:37.583450+0000","last_fullsized":"2024-09-10T18:51:37.583450+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'18","last_scrub_stamp":"2024-09-10T18:51:37.059477+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-10T18:47:10.514951+0000","last_clean_scrub_stamp":"2024-09-10T18:51:37.059477+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:06.269219+0000","scrub_duration":0.0020902970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"84'6","reported_seq":530,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-10T18:51:39.974652+0000","last_change":"2024-09-10T18:49:57.418744+0000","last_active":"2024-09-10T18:51:39.974652+0000","last_peered":"2024-09-10T18:51:39.974652+0000","last_clean":"2024-09-10T18:51:39.974652+0000","last_became_active":"2024-09-10T18:47:36.418211+0000","last_became_peered":"2024-09-10T18:47:36.418211+0000","last_unstale":"2024-09-10T18:51:39.974652+0000","last_undegraded":"2024-09-10T18:51:39.974652+0000","last_fullsized":"2024-09-10T18:51:39.974652+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'6","last_scrub_stamp":"2024-09-10T18:49:57.416319+0000","last_deep_scrub":"84'6","last_deep_scrub_stamp":"2024-09-10T18:47:22.545450+0000","last_clean_scrub_stamp":"2024-09-10T18:49:57.416319+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044734659999999997,"objects_trimmed":0,"snaptrim_duration":0.0021933159999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":373,"num_read_kb":368,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":94,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.579414+0000","last_change":"2024-09-10T18:49:16.371789+0000","last_active":"2024-09-10T18:51:37.579414+0000","last_peered":"2024-09-10T18:51:37.579414+0000","last_clean":"2024-09-10T18:51:37.579414+0000","last_became_active":"2024-09-10T18:47:36.422641+0000","last_became_peered":"2024-09-10T18:47:36.422641+0000","last_unstale":"2024-09-10T18:51:37.579414+0000","last_undegraded":"2024-09-10T18:51:37.579414+0000","last_fullsized":"2024-09-10T18:51:37.579414+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:16.371702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:16.371702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0099806840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":103,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.588680+0000","last_change":"2024-09-10T18:50:00.088364+0000","last_active":"2024-09-10T18:51:37.588680+0000","last_peered":"2024-09-10T18:51:37.588680+0000","last_clean":"2024-09-10T18:51:37.588680+0000","last_became_active":"2024-09-10T18:47:36.419081+0000","last_became_peered":"2024-09-10T18:47:36.419081+0000","last_unstale":"2024-09-10T18:51:37.588680+0000","last_undegraded":"2024-09-10T18:51:37.588680+0000","last_fullsized":"2024-09-10T18:51:37.588680+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:00.088179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:00.088179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.034684810000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":137,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.909570+0000","last_change":"2024-09-10T18:51:35.398902+0000","last_active":"2024-09-10T18:51:37.909570+0000","last_peered":"2024-09-10T18:51:37.909570+0000","last_clean":"2024-09-10T18:51:37.909570+0000","last_became_active":"2024-09-10T18:45:24.264617+0000","last_became_peered":"2024-09-10T18:45:24.264617+0000","last_unstale":"2024-09-10T18:51:37.909570+0000","last_undegraded":"2024-09-10T18:51:37.909570+0000","last_fullsized":"2024-09-10T18:51:37.909570+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:35.398819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:35.398819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:41.839709+0000","scrub_duration":0.0014954090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.590389+0000","last_change":"2024-09-10T18:51:37.589525+0000","last_active":"2024-09-10T18:51:37.590389+0000","last_peered":"2024-09-10T18:51:37.590389+0000","last_clean":"2024-09-10T18:51:37.590389+0000","last_became_active":"2024-09-10T18:51:37.588751+0000","last_became_peered":"2024-09-10T18:51:37.588751+0000","last_unstale":"2024-09-10T18:51:37.590389+0000","last_undegraded":"2024-09-10T18:51:37.590389+0000","last_fullsized":"2024-09-10T18:51:37.590389+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:57.311670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"70'81","reported_seq":50,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.582205+0000","last_change":"2024-09-10T18:51:04.099758+0000","last_active":"2024-09-10T18:51:37.582205+0000","last_peered":"2024-09-10T18:51:37.582205+0000","last_clean":"2024-09-10T18:51:37.582205+0000","last_became_active":"2024-09-10T18:50:08.398205+0000","last_became_peered":"2024-09-10T18:50:08.398205+0000","last_unstale":"2024-09-10T18:51:37.582205+0000","last_undegraded":"2024-09-10T18:51:37.582205+0000","last_fullsized":"2024-09-10T18:51:37.582205+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:51:04.099703+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:51:04.099703+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:26.646801+0000","scrub_duration":0.007621203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"63'12","reported_seq":183,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.578663+0000","last_change":"2024-09-10T18:50:48.742956+0000","last_active":"2024-09-10T18:51:37.578663+0000","last_peered":"2024-09-10T18:51:37.578663+0000","last_clean":"2024-09-10T18:51:37.578663+0000","last_became_active":"2024-09-10T18:47:36.417965+0000","last_became_peered":"2024-09-10T18:47:36.417965+0000","last_unstale":"2024-09-10T18:51:37.578663+0000","last_undegraded":"2024-09-10T18:51:37.578663+0000","last_fullsized":"2024-09-10T18:51:37.578663+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:50:48.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:50:48.742875+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:50.790274+0000","scrub_duration":0.011146239,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"58'1","reported_seq":149,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-10T18:51:39.175402+0000","last_change":"2024-09-10T18:51:39.175354+0000","last_active":"2024-09-10T18:51:39.175402+0000","last_peered":"2024-09-10T18:51:39.175402+0000","last_clean":"2024-09-10T18:51:39.175402+0000","last_became_active":"2024-09-10T18:45:17.963932+0000","last_became_peered":"2024-09-10T18:45:17.963932+0000","last_unstale":"2024-09-10T18:51:39.175402+0000","last_undegraded":"2024-09-10T18:51:39.175402+0000","last_fullsized":"2024-09-10T18:51:39.175402+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'1","last_scrub_stamp":"2024-09-10T18:51:39.175290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:39.175290+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:03.334161+0000","scrub_duration":0.0015267550000000001,"objects_trimmed":0,"snaptrim_duration":0.00037812099999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":101,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.908639+0000","last_change":"2024-09-10T18:51:30.467050+0000","last_active":"2024-09-10T18:51:37.908639+0000","last_peered":"2024-09-10T18:51:37.908639+0000","last_clean":"2024-09-10T18:51:37.908639+0000","last_became_active":"2024-09-10T18:47:36.416641+0000","last_became_peered":"2024-09-10T18:47:36.416641+0000","last_unstale":"2024-09-10T18:51:37.908639+0000","last_undegraded":"2024-09-10T18:51:37.908639+0000","last_fullsized":"2024-09-10T18:51:37.908639+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:30.466988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:30.466988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:44.004240+0000","scrub_duration":0.0019674089999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":138,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.589203+0000","last_change":"2024-09-10T18:50:52.999936+0000","last_active":"2024-09-10T18:51:37.589203+0000","last_peered":"2024-09-10T18:51:37.589203+0000","last_clean":"2024-09-10T18:51:37.589203+0000","last_became_active":"2024-09-10T18:45:22.064285+0000","last_became_peered":"2024-09-10T18:45:22.064285+0000","last_unstale":"2024-09-10T18:51:37.589203+0000","last_undegraded":"2024-09-10T18:51:37.589203+0000","last_fullsized":"2024-09-10T18:51:37.589203+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:52.999842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:52.999842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:02.826146+0000","scrub_duration":0.0065214100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":137,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.582701+0000","last_change":"2024-09-10T18:50:53.940855+0000","last_active":"2024-09-10T18:51:37.582701+0000","last_peered":"2024-09-10T18:51:37.582701+0000","last_clean":"2024-09-10T18:51:37.582701+0000","last_became_active":"2024-09-10T18:45:24.083689+0000","last_became_peered":"2024-09-10T18:45:24.083689+0000","last_unstale":"2024-09-10T18:51:37.582701+0000","last_undegraded":"2024-09-10T18:51:37.582701+0000","last_fullsized":"2024-09-10T18:51:37.582701+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:53.940489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:50:53.940489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:21.653605+0000","scrub_duration":0.007171691,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.591938+0000","last_change":"2024-09-10T18:51:37.591719+0000","last_active":"2024-09-10T18:51:37.591938+0000","last_peered":"2024-09-10T18:51:37.591938+0000","last_clean":"2024-09-10T18:51:37.591938+0000","last_became_active":"2024-09-10T18:51:37.591441+0000","last_became_peered":"2024-09-10T18:51:37.591441+0000","last_unstale":"2024-09-10T18:51:37.591938+0000","last_undegraded":"2024-09-10T18:51:37.591938+0000","last_fullsized":"2024-09-10T18:51:37.591938+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:43.944535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"126'21","reported_seq":169,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.588114+0000","last_change":"2024-09-10T18:50:03.380989+0000","last_active":"2024-09-10T18:51:37.588114+0000","last_peered":"2024-09-10T18:51:37.588114+0000","last_clean":"2024-09-10T18:51:37.588114+0000","last_became_active":"2024-09-10T18:45:20.421119+0000","last_became_peered":"2024-09-10T18:45:20.421119+0000","last_unstale":"2024-09-10T18:51:37.588114+0000","last_undegraded":"2024-09-10T18:51:37.588114+0000","last_fullsized":"2024-09-10T18:51:37.588114+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"126'21","last_scrub_stamp":"2024-09-10T18:50:03.380902+0000","last_deep_scrub":"126'21","last_deep_scrub_stamp":"2024-09-10T18:50:03.380902+0000","last_clean_scrub_stamp":"2024-09-10T18:50:03.380902+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0084173819999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"60'2","reported_seq":150,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.576866+0000","last_change":"2024-09-10T18:51:03.104641+0000","last_active":"2024-09-10T18:51:37.576866+0000","last_peered":"2024-09-10T18:51:37.576866+0000","last_clean":"2024-09-10T18:51:37.576866+0000","last_became_active":"2024-09-10T18:45:17.961660+0000","last_became_peered":"2024-09-10T18:45:17.961660+0000","last_unstale":"2024-09-10T18:51:37.576866+0000","last_undegraded":"2024-09-10T18:51:37.576866+0000","last_fullsized":"2024-09-10T18:51:37.576866+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-10T18:51:03.104545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:03.104545+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:16.497102+0000","scrub_duration":0.0070342290000000003,"objects_trimmed":0,"snaptrim_duration":0.0013942640000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":152,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.587565+0000","last_change":"2024-09-10T18:50:14.032609+0000","last_active":"2024-09-10T18:51:37.587565+0000","last_peered":"2024-09-10T18:51:37.587565+0000","last_clean":"2024-09-10T18:51:37.587565+0000","last_became_active":"2024-09-10T18:47:36.414636+0000","last_became_peered":"2024-09-10T18:47:36.414636+0000","last_unstale":"2024-09-10T18:51:37.587565+0000","last_undegraded":"2024-09-10T18:51:37.587565+0000","last_fullsized":"2024-09-10T18:51:37.587565+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:14.032522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:14.032522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.022590255,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'81","reported_seq":198,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.909952+0000","last_change":"2024-09-10T18:50:48.508352+0000","last_active":"2024-09-10T18:51:37.909952+0000","last_peered":"2024-09-10T18:51:37.909952+0000","last_clean":"2024-09-10T18:51:37.909952+0000","last_became_active":"2024-09-10T18:50:06.386717+0000","last_became_peered":"2024-09-10T18:50:06.386717+0000","last_unstale":"2024-09-10T18:51:37.909952+0000","last_undegraded":"2024-09-10T18:51:37.909952+0000","last_fullsized":"2024-09-10T18:51:37.909952+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:50:48.508244+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:50:48.508244+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:53.803446+0000","scrub_duration":0.0035856120000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":159,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-10T18:51:40.327767+0000","last_change":"2024-09-10T18:51:40.327714+0000","last_active":"2024-09-10T18:51:40.327767+0000","last_peered":"2024-09-10T18:51:40.327767+0000","last_clean":"2024-09-10T18:51:40.327767+0000","last_became_active":"2024-09-10T18:47:36.418320+0000","last_became_peered":"2024-09-10T18:47:36.418320+0000","last_unstale":"2024-09-10T18:51:40.327767+0000","last_undegraded":"2024-09-10T18:51:40.327767+0000","last_fullsized":"2024-09-10T18:51:40.327767+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:40.327645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:40.327645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:58.610158+0000","scrub_duration":0.001391827,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":136,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.589365+0000","last_change":"2024-09-10T18:51:02.170258+0000","last_active":"2024-09-10T18:51:37.589365+0000","last_peered":"2024-09-10T18:51:37.589365+0000","last_clean":"2024-09-10T18:51:37.589365+0000","last_became_active":"2024-09-10T18:45:24.073327+0000","last_became_peered":"2024-09-10T18:45:24.073327+0000","last_unstale":"2024-09-10T18:51:37.589365+0000","last_undegraded":"2024-09-10T18:51:37.589365+0000","last_fullsized":"2024-09-10T18:51:37.589365+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:02.170065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:02.170065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:14.436899+0000","scrub_duration":0.0061532089999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.910216+0000","last_change":"2024-09-10T18:51:37.909963+0000","last_active":"2024-09-10T18:51:37.910216+0000","last_peered":"2024-09-10T18:51:37.910216+0000","last_clean":"2024-09-10T18:51:37.910216+0000","last_became_active":"2024-09-10T18:51:37.909738+0000","last_became_peered":"2024-09-10T18:51:37.909738+0000","last_unstale":"2024-09-10T18:51:37.910216+0000","last_undegraded":"2024-09-10T18:51:37.910216+0000","last_fullsized":"2024-09-10T18:51:37.910216+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:03.481643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"63'19","reported_seq":171,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.586073+0000","last_change":"2024-09-10T18:50:01.460246+0000","last_active":"2024-09-10T18:51:37.586073+0000","last_peered":"2024-09-10T18:51:37.586073+0000","last_clean":"2024-09-10T18:51:37.586073+0000","last_became_active":"2024-09-10T18:45:20.421097+0000","last_became_peered":"2024-09-10T18:45:20.421097+0000","last_unstale":"2024-09-10T18:51:37.586073+0000","last_undegraded":"2024-09-10T18:51:37.586073+0000","last_fullsized":"2024-09-10T18:51:37.586073+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:50:01.460159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:50:01.460159+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.019465403999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":103,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.589861+0000","last_change":"2024-09-10T18:50:47.083728+0000","last_active":"2024-09-10T18:51:37.589861+0000","last_peered":"2024-09-10T18:51:37.589861+0000","last_clean":"2024-09-10T18:51:37.589861+0000","last_became_active":"2024-09-10T18:47:36.417800+0000","last_became_peered":"2024-09-10T18:47:36.417800+0000","last_unstale":"2024-09-10T18:51:37.589861+0000","last_undegraded":"2024-09-10T18:51:37.589861+0000","last_fullsized":"2024-09-10T18:51:37.589861+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:47.083643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:47.083643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:16.602999+0000","scrub_duration":0.0071207509999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":114,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-10T18:51:40.126777+0000","last_change":"2024-09-10T18:51:40.126736+0000","last_active":"2024-09-10T18:51:40.126777+0000","last_peered":"2024-09-10T18:51:40.126777+0000","last_clean":"2024-09-10T18:51:40.126777+0000","last_became_active":"2024-09-10T18:47:36.413618+0000","last_became_peered":"2024-09-10T18:47:36.413618+0000","last_unstale":"2024-09-10T18:51:40.126777+0000","last_undegraded":"2024-09-10T18:51:40.126777+0000","last_fullsized":"2024-09-10T18:51:40.126777+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:40.126652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:40.126652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:44.820089+0000","scrub_duration":0.0017754699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":147,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:38.318994+0000","last_change":"2024-09-10T18:51:38.318952+0000","last_active":"2024-09-10T18:51:38.318994+0000","last_peered":"2024-09-10T18:51:38.318994+0000","last_clean":"2024-09-10T18:51:38.318994+0000","last_became_active":"2024-09-10T18:47:36.412610+0000","last_became_peered":"2024-09-10T18:47:36.412610+0000","last_unstale":"2024-09-10T18:51:38.318994+0000","last_undegraded":"2024-09-10T18:51:38.318994+0000","last_fullsized":"2024-09-10T18:51:38.318994+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:38.318887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:38.318887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:47.048859+0000","scrub_duration":0.001335461,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.590497+0000","last_change":"2024-09-10T18:51:37.590054+0000","last_active":"2024-09-10T18:51:37.590497+0000","last_peered":"2024-09-10T18:51:37.590497+0000","last_clean":"2024-09-10T18:51:37.590497+0000","last_became_active":"2024-09-10T18:51:37.589351+0000","last_became_peered":"2024-09-10T18:51:37.589351+0000","last_unstale":"2024-09-10T18:51:37.590497+0000","last_undegraded":"2024-09-10T18:51:37.590497+0000","last_fullsized":"2024-09-10T18:51:37.590497+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:52.988787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"130'13","reported_seq":161,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.577496+0000","last_change":"2024-09-10T18:50:08.785524+0000","last_active":"2024-09-10T18:51:37.577496+0000","last_peered":"2024-09-10T18:51:37.577496+0000","last_clean":"2024-09-10T18:51:37.577496+0000","last_became_active":"2024-09-10T18:45:20.061021+0000","last_became_peered":"2024-09-10T18:45:20.061021+0000","last_unstale":"2024-09-10T18:51:37.577496+0000","last_undegraded":"2024-09-10T18:51:37.577496+0000","last_fullsized":"2024-09-10T18:51:37.577496+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"130'13","last_scrub_stamp":"2024-09-10T18:50:08.785434+0000","last_deep_scrub":"130'13","last_deep_scrub_stamp":"2024-09-10T18:50:08.785434+0000","last_clean_scrub_stamp":"2024-09-10T18:50:08.785434+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016531225,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":143,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.583037+0000","last_change":"2024-09-10T18:51:31.128571+0000","last_active":"2024-09-10T18:51:37.583037+0000","last_peered":"2024-09-10T18:51:37.583037+0000","last_clean":"2024-09-10T18:51:37.583037+0000","last_became_active":"2024-09-10T18:45:17.963057+0000","last_became_peered":"2024-09-10T18:45:17.963057+0000","last_unstale":"2024-09-10T18:51:37.583037+0000","last_undegraded":"2024-09-10T18:51:37.583037+0000","last_fullsized":"2024-09-10T18:51:37.583037+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:31.128514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:31.128514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:55.374112+0000","scrub_duration":0.0016209180000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":103,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.581700+0000","last_change":"2024-09-10T18:51:35.325521+0000","last_active":"2024-09-10T18:51:37.581700+0000","last_peered":"2024-09-10T18:51:37.581700+0000","last_clean":"2024-09-10T18:51:37.581700+0000","last_became_active":"2024-09-10T18:47:36.411517+0000","last_became_peered":"2024-09-10T18:47:36.411517+0000","last_unstale":"2024-09-10T18:51:37.581700+0000","last_undegraded":"2024-09-10T18:51:37.581700+0000","last_fullsized":"2024-09-10T18:51:37.581700+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:35.325451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_clean_scrub_stamp":"2024-09-10T18:51:35.325451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:56.285365+0000","scrub_duration":0.0018247820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":147,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.576810+0000","last_change":"2024-09-10T18:49:54.678748+0000","last_active":"2024-09-10T18:51:37.576810+0000","last_peered":"2024-09-10T18:51:37.576810+0000","last_clean":"2024-09-10T18:51:37.576810+0000","last_became_active":"2024-09-10T18:47:36.412933+0000","last_became_peered":"2024-09-10T18:47:36.412933+0000","last_unstale":"2024-09-10T18:51:37.576810+0000","last_undegraded":"2024-09-10T18:51:37.576810+0000","last_fullsized":"2024-09-10T18:51:37.576810+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:54.678678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:47.545343+0000","last_clean_scrub_stamp":"2024-09-10T18:49:54.678678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0091071069999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.591295+0000","last_change":"2024-09-10T18:51:37.590489+0000","last_active":"2024-09-10T18:51:37.591295+0000","last_peered":"2024-09-10T18:51:37.591295+0000","last_clean":"2024-09-10T18:51:37.591295+0000","last_became_active":"2024-09-10T18:51:37.590189+0000","last_became_peered":"2024-09-10T18:51:37.590189+0000","last_unstale":"2024-09-10T18:51:37.591295+0000","last_undegraded":"2024-09-10T18:51:37.591295+0000","last_fullsized":"2024-09-10T18:51:37.591295+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:47.653862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"63'15","reported_seq":165,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.587262+0000","last_change":"2024-09-10T18:51:36.277582+0000","last_active":"2024-09-10T18:51:37.587262+0000","last_peered":"2024-09-10T18:51:37.587262+0000","last_clean":"2024-09-10T18:51:37.587262+0000","last_became_active":"2024-09-10T18:45:20.419804+0000","last_became_peered":"2024-09-10T18:45:20.419804+0000","last_unstale":"2024-09-10T18:51:37.587262+0000","last_undegraded":"2024-09-10T18:51:37.587262+0000","last_fullsized":"2024-09-10T18:51:37.587262+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:51:36.277517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.277517+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:47.932042+0000","scrub_duration":0.0020736750000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":94,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.580243+0000","last_change":"2024-09-10T18:50:05.603208+0000","last_active":"2024-09-10T18:51:37.580243+0000","last_peered":"2024-09-10T18:51:37.580243+0000","last_clean":"2024-09-10T18:51:37.580243+0000","last_became_active":"2024-09-10T18:47:36.420475+0000","last_became_peered":"2024-09-10T18:47:36.420475+0000","last_unstale":"2024-09-10T18:51:37.580243+0000","last_undegraded":"2024-09-10T18:51:37.580243+0000","last_fullsized":"2024-09-10T18:51:37.580243+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:05.603125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:05.603125+0000","last_clean_scrub_stamp":"2024-09-10T18:50:05.603125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048485560000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"83'9","reported_seq":227,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.909091+0000","last_change":"2024-09-10T18:50:10.444294+0000","last_active":"2024-09-10T18:51:37.909091+0000","last_peered":"2024-09-10T18:51:37.909091+0000","last_clean":"2024-09-10T18:51:37.909091+0000","last_became_active":"2024-09-10T18:45:22.073383+0000","last_became_peered":"2024-09-10T18:45:22.073383+0000","last_unstale":"2024-09-10T18:51:37.909091+0000","last_undegraded":"2024-09-10T18:51:37.909091+0000","last_fullsized":"2024-09-10T18:51:37.909091+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'9","last_scrub_stamp":"2024-09-10T18:50:10.444201+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-09-10T18:47:02.582229+0000","last_clean_scrub_stamp":"2024-09-10T18:50:10.444201+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014241271,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":136,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.587928+0000","last_change":"2024-09-10T18:51:05.187154+0000","last_active":"2024-09-10T18:51:37.587928+0000","last_peered":"2024-09-10T18:51:37.587928+0000","last_clean":"2024-09-10T18:51:37.587928+0000","last_became_active":"2024-09-10T18:45:24.265256+0000","last_became_peered":"2024-09-10T18:45:24.265256+0000","last_unstale":"2024-09-10T18:51:37.587928+0000","last_undegraded":"2024-09-10T18:51:37.587928+0000","last_fullsized":"2024-09-10T18:51:37.587928+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:05.187073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_clean_scrub_stamp":"2024-09-10T18:51:05.187073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:11.730150+0000","scrub_duration":0.00225728,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.911592+0000","last_change":"2024-09-10T18:51:37.910484+0000","last_active":"2024-09-10T18:51:37.911592+0000","last_peered":"2024-09-10T18:51:37.911592+0000","last_clean":"2024-09-10T18:51:37.911592+0000","last_became_active":"2024-09-10T18:51:37.910181+0000","last_became_peered":"2024-09-10T18:51:37.910181+0000","last_unstale":"2024-09-10T18:51:37.911592+0000","last_undegraded":"2024-09-10T18:51:37.911592+0000","last_fullsized":"2024-09-10T18:51:37.911592+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:43.239258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"63'12","reported_seq":163,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.582165+0000","last_change":"2024-09-10T18:50:12.595966+0000","last_active":"2024-09-10T18:51:37.582165+0000","last_peered":"2024-09-10T18:51:37.582165+0000","last_clean":"2024-09-10T18:51:37.582165+0000","last_became_active":"2024-09-10T18:45:20.056869+0000","last_became_peered":"2024-09-10T18:45:20.056869+0000","last_unstale":"2024-09-10T18:51:37.582165+0000","last_undegraded":"2024-09-10T18:51:37.582165+0000","last_fullsized":"2024-09-10T18:51:37.582165+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:50:12.595881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:50:12.595881+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013834673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"56'2","reported_seq":149,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.582876+0000","last_change":"2024-09-10T18:49:20.403122+0000","last_active":"2024-09-10T18:51:37.582876+0000","last_peered":"2024-09-10T18:51:37.582876+0000","last_clean":"2024-09-10T18:51:37.582876+0000","last_became_active":"2024-09-10T18:45:17.963286+0000","last_became_peered":"2024-09-10T18:45:17.963286+0000","last_unstale":"2024-09-10T18:51:37.582876+0000","last_undegraded":"2024-09-10T18:51:37.582876+0000","last_fullsized":"2024-09-10T18:51:37.582876+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-10T18:49:20.403067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:20.403067+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00336061,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":104,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.576680+0000","last_change":"2024-09-10T18:50:02.711676+0000","last_active":"2024-09-10T18:51:37.576680+0000","last_peered":"2024-09-10T18:51:37.576680+0000","last_clean":"2024-09-10T18:51:37.576680+0000","last_became_active":"2024-09-10T18:47:36.421151+0000","last_became_peered":"2024-09-10T18:47:36.421151+0000","last_unstale":"2024-09-10T18:51:37.576680+0000","last_undegraded":"2024-09-10T18:51:37.576680+0000","last_fullsized":"2024-09-10T18:51:37.576680+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:02.711594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:02.711594+0000","last_clean_scrub_stamp":"2024-09-10T18:50:02.711594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001786826,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":127,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.577055+0000","last_change":"2024-09-10T18:49:52.490607+0000","last_active":"2024-09-10T18:51:37.577055+0000","last_peered":"2024-09-10T18:51:37.577055+0000","last_clean":"2024-09-10T18:51:37.577055+0000","last_became_active":"2024-09-10T18:45:24.077815+0000","last_became_peered":"2024-09-10T18:45:24.077815+0000","last_unstale":"2024-09-10T18:51:37.577055+0000","last_undegraded":"2024-09-10T18:51:37.577055+0000","last_fullsized":"2024-09-10T18:51:37.577055+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:52.490519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:52.490519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.04093488,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"76'13","reported_seq":161,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.578582+0000","last_change":"2024-09-10T18:49:58.699452+0000","last_active":"2024-09-10T18:51:37.578582+0000","last_peered":"2024-09-10T18:51:37.578582+0000","last_clean":"2024-09-10T18:51:37.578582+0000","last_became_active":"2024-09-10T18:45:20.420421+0000","last_became_peered":"2024-09-10T18:45:20.420421+0000","last_unstale":"2024-09-10T18:51:37.578582+0000","last_undegraded":"2024-09-10T18:51:37.578582+0000","last_fullsized":"2024-09-10T18:51:37.578582+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'13","last_scrub_stamp":"2024-09-10T18:49:58.699357+0000","last_deep_scrub":"76'13","last_deep_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_clean_scrub_stamp":"2024-09-10T18:49:58.699357+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013859232000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":104,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.583020+0000","last_change":"2024-09-10T18:50:07.454468+0000","last_active":"2024-09-10T18:51:37.583020+0000","last_peered":"2024-09-10T18:51:37.583020+0000","last_clean":"2024-09-10T18:51:37.583020+0000","last_became_active":"2024-09-10T18:47:36.409299+0000","last_became_peered":"2024-09-10T18:47:36.409299+0000","last_unstale":"2024-09-10T18:51:37.583020+0000","last_undegraded":"2024-09-10T18:51:37.583020+0000","last_fullsized":"2024-09-10T18:51:37.583020+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:07.454393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:07.454393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014100293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":139,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.579606+0000","last_change":"2024-09-10T18:50:57.190078+0000","last_active":"2024-09-10T18:51:37.579606+0000","last_peered":"2024-09-10T18:51:37.579606+0000","last_clean":"2024-09-10T18:51:37.579606+0000","last_became_active":"2024-09-10T18:45:22.069040+0000","last_became_peered":"2024-09-10T18:45:22.069040+0000","last_unstale":"2024-09-10T18:51:37.579606+0000","last_undegraded":"2024-09-10T18:51:37.579606+0000","last_fullsized":"2024-09-10T18:51:37.579606+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:57.189976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:28.442370+0000","last_clean_scrub_stamp":"2024-09-10T18:50:57.189976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:02.018261+0000","scrub_duration":0.016295923,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"63'1","reported_seq":109,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.581907+0000","last_change":"2024-09-10T18:49:49.544530+0000","last_active":"2024-09-10T18:51:37.581907+0000","last_peered":"2024-09-10T18:51:37.581907+0000","last_clean":"2024-09-10T18:51:37.581907+0000","last_became_active":"2024-09-10T18:47:36.406936+0000","last_became_peered":"2024-09-10T18:47:36.406936+0000","last_unstale":"2024-09-10T18:51:37.581907+0000","last_undegraded":"2024-09-10T18:51:37.581907+0000","last_fullsized":"2024-09-10T18:51:37.581907+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-10T18:49:49.544433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:49.544433+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041109190000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":142,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.587116+0000","last_change":"2024-09-10T18:50:52.260051+0000","last_active":"2024-09-10T18:51:37.587116+0000","last_peered":"2024-09-10T18:51:37.587116+0000","last_clean":"2024-09-10T18:51:37.587116+0000","last_became_active":"2024-09-10T18:45:17.964705+0000","last_became_peered":"2024-09-10T18:45:17.964705+0000","last_unstale":"2024-09-10T18:51:37.587116+0000","last_undegraded":"2024-09-10T18:51:37.587116+0000","last_fullsized":"2024-09-10T18:51:37.587116+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:52.259998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:52.259998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:58.126795+0000","scrub_duration":0.0032069059999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'6","reported_seq":140,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.579810+0000","last_change":"2024-09-10T18:49:18.363292+0000","last_active":"2024-09-10T18:51:37.579810+0000","last_peered":"2024-09-10T18:51:37.579810+0000","last_clean":"2024-09-10T18:51:37.579810+0000","last_became_active":"2024-09-10T18:45:20.421746+0000","last_became_peered":"2024-09-10T18:45:20.421746+0000","last_unstale":"2024-09-10T18:51:37.579810+0000","last_undegraded":"2024-09-10T18:51:37.579810+0000","last_fullsized":"2024-09-10T18:51:37.579810+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'6","last_scrub_stamp":"2024-09-10T18:49:18.363203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:49:18.363203+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0093609290000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":138,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.588808+0000","last_change":"2024-09-10T18:51:32.233743+0000","last_active":"2024-09-10T18:51:37.588808+0000","last_peered":"2024-09-10T18:51:37.588808+0000","last_clean":"2024-09-10T18:51:37.588808+0000","last_became_active":"2024-09-10T18:45:22.069653+0000","last_became_peered":"2024-09-10T18:45:22.069653+0000","last_unstale":"2024-09-10T18:51:37.588808+0000","last_undegraded":"2024-09-10T18:51:37.588808+0000","last_fullsized":"2024-09-10T18:51:37.588808+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:32.233676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:32.233676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:33.989510+0000","scrub_duration":0.0016772180000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":137,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.909701+0000","last_change":"2024-09-10T18:50:43.408152+0000","last_active":"2024-09-10T18:51:37.909701+0000","last_peered":"2024-09-10T18:51:37.909701+0000","last_clean":"2024-09-10T18:51:37.909701+0000","last_became_active":"2024-09-10T18:45:24.264611+0000","last_became_peered":"2024-09-10T18:45:24.264611+0000","last_unstale":"2024-09-10T18:51:37.909701+0000","last_undegraded":"2024-09-10T18:51:37.909701+0000","last_fullsized":"2024-09-10T18:51:37.909701+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:43.408011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:50:43.408011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:11.539468+0000","scrub_duration":0.0026253959999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"63'10","reported_seq":200,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-10T18:51:39.074781+0000","last_change":"2024-09-10T18:51:39.074731+0000","last_active":"2024-09-10T18:51:39.074781+0000","last_peered":"2024-09-10T18:51:39.074781+0000","last_clean":"2024-09-10T18:51:39.074781+0000","last_became_active":"2024-09-10T18:47:36.426260+0000","last_became_peered":"2024-09-10T18:47:36.426260+0000","last_unstale":"2024-09-10T18:51:39.074781+0000","last_undegraded":"2024-09-10T18:51:39.074781+0000","last_fullsized":"2024-09-10T18:51:39.074781+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:51:39.074664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:39.074664+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:05.841922+0000","scrub_duration":0.0044748410000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":142,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.587611+0000","last_change":"2024-09-10T18:50:45.120724+0000","last_active":"2024-09-10T18:51:37.587611+0000","last_peered":"2024-09-10T18:51:37.587611+0000","last_clean":"2024-09-10T18:51:37.587611+0000","last_became_active":"2024-09-10T18:45:17.964870+0000","last_became_peered":"2024-09-10T18:45:17.964870+0000","last_unstale":"2024-09-10T18:51:37.587611+0000","last_undegraded":"2024-09-10T18:51:37.587611+0000","last_fullsized":"2024-09-10T18:51:37.587611+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:45.120550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:45.120550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:00.695839+0000","scrub_duration":0.0055063930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"63'8","reported_seq":222,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.588038+0000","last_change":"2024-09-10T18:50:52.043047+0000","last_active":"2024-09-10T18:51:37.588038+0000","last_peered":"2024-09-10T18:51:37.588038+0000","last_clean":"2024-09-10T18:51:37.588038+0000","last_became_active":"2024-09-10T18:45:22.062940+0000","last_became_peered":"2024-09-10T18:45:22.062940+0000","last_unstale":"2024-09-10T18:51:37.588038+0000","last_undegraded":"2024-09-10T18:51:37.588038+0000","last_fullsized":"2024-09-10T18:51:37.588038+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:50:52.042960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:52.042960+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:11.293055+0000","scrub_duration":0.0090181189999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":137,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.577912+0000","last_change":"2024-09-10T18:50:56.767501+0000","last_active":"2024-09-10T18:51:37.577912+0000","last_peered":"2024-09-10T18:51:37.577912+0000","last_clean":"2024-09-10T18:51:37.577912+0000","last_became_active":"2024-09-10T18:45:24.073736+0000","last_became_peered":"2024-09-10T18:45:24.073736+0000","last_unstale":"2024-09-10T18:51:37.577912+0000","last_undegraded":"2024-09-10T18:51:37.577912+0000","last_fullsized":"2024-09-10T18:51:37.577912+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:56.767430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:50:56.767430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:18.035411+0000","scrub_duration":0.021971303000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"63'9","reported_seq":156,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.588591+0000","last_change":"2024-09-10T18:49:56.061777+0000","last_active":"2024-09-10T18:51:37.588591+0000","last_peered":"2024-09-10T18:51:37.588591+0000","last_clean":"2024-09-10T18:51:37.588591+0000","last_became_active":"2024-09-10T18:45:20.421124+0000","last_became_peered":"2024-09-10T18:45:20.421124+0000","last_unstale":"2024-09-10T18:51:37.588591+0000","last_undegraded":"2024-09-10T18:51:37.588591+0000","last_fullsized":"2024-09-10T18:51:37.588591+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:49:56.061694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:49:56.061694+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0042142459999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":143,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.577633+0000","last_change":"2024-09-10T18:49:57.653570+0000","last_active":"2024-09-10T18:51:37.577633+0000","last_peered":"2024-09-10T18:51:37.577633+0000","last_clean":"2024-09-10T18:51:37.577633+0000","last_became_active":"2024-09-10T18:45:17.960106+0000","last_became_peered":"2024-09-10T18:45:17.960106+0000","last_unstale":"2024-09-10T18:51:37.577633+0000","last_undegraded":"2024-09-10T18:51:37.577633+0000","last_fullsized":"2024-09-10T18:51:37.577633+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:57.653478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:57.653478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013420596999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"89'10","reported_seq":229,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-10T18:51:38.594664+0000","last_change":"2024-09-10T18:49:55.514077+0000","last_active":"2024-09-10T18:51:38.594664+0000","last_peered":"2024-09-10T18:51:38.594664+0000","last_clean":"2024-09-10T18:51:38.594664+0000","last_became_active":"2024-09-10T18:47:36.424575+0000","last_became_peered":"2024-09-10T18:47:36.424575+0000","last_unstale":"2024-09-10T18:51:38.594664+0000","last_undegraded":"2024-09-10T18:51:38.594664+0000","last_fullsized":"2024-09-10T18:51:38.594664+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'10","last_scrub_stamp":"2024-09-10T18:49:55.513847+0000","last_deep_scrub":"81'9","last_deep_scrub_stamp":"2024-09-10T18:46:54.800790+0000","last_clean_scrub_stamp":"2024-09-10T18:49:55.513847+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0099728100000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":146,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.582670+0000","last_change":"2024-09-10T18:50:55.384653+0000","last_active":"2024-09-10T18:51:37.582670+0000","last_peered":"2024-09-10T18:51:37.582670+0000","last_clean":"2024-09-10T18:51:37.582670+0000","last_became_active":"2024-09-10T18:47:36.414047+0000","last_became_peered":"2024-09-10T18:47:36.414047+0000","last_unstale":"2024-09-10T18:51:37.582670+0000","last_undegraded":"2024-09-10T18:51:37.582670+0000","last_fullsized":"2024-09-10T18:51:37.582670+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:55.384570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_clean_scrub_stamp":"2024-09-10T18:50:55.384570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:06.404917+0000","scrub_duration":0.0079403359999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"63'9","reported_seq":177,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.582357+0000","last_change":"2024-09-10T18:51:32.146681+0000","last_active":"2024-09-10T18:51:37.582357+0000","last_peered":"2024-09-10T18:51:37.582357+0000","last_clean":"2024-09-10T18:51:37.582357+0000","last_became_active":"2024-09-10T18:47:36.416484+0000","last_became_peered":"2024-09-10T18:47:36.416484+0000","last_unstale":"2024-09-10T18:51:37.582357+0000","last_undegraded":"2024-09-10T18:51:37.582357+0000","last_fullsized":"2024-09-10T18:51:37.582357+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:51:32.146622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:32.146622+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:53.233486+0000","scrub_duration":0.0023792729999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":142,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.587548+0000","last_change":"2024-09-10T18:49:04.430421+0000","last_active":"2024-09-10T18:51:37.587548+0000","last_peered":"2024-09-10T18:51:37.587548+0000","last_clean":"2024-09-10T18:51:37.587548+0000","last_became_active":"2024-09-10T18:45:17.962009+0000","last_became_peered":"2024-09-10T18:45:17.962009+0000","last_unstale":"2024-09-10T18:51:37.587548+0000","last_undegraded":"2024-09-10T18:51:37.587548+0000","last_fullsized":"2024-09-10T18:51:37.587548+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:04.430344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:04.430344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025461400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":129,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.577130+0000","last_change":"2024-09-10T18:50:03.571896+0000","last_active":"2024-09-10T18:51:37.577130+0000","last_peered":"2024-09-10T18:51:37.577130+0000","last_clean":"2024-09-10T18:51:37.577130+0000","last_became_active":"2024-09-10T18:45:22.072877+0000","last_became_peered":"2024-09-10T18:45:22.072877+0000","last_unstale":"2024-09-10T18:51:37.577130+0000","last_undegraded":"2024-09-10T18:51:37.577130+0000","last_fullsized":"2024-09-10T18:51:37.577130+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:03.571786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:03.571786+0000","last_clean_scrub_stamp":"2024-09-10T18:50:03.571786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0096506809999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":137,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.576514+0000","last_change":"2024-09-10T18:50:04.631640+0000","last_active":"2024-09-10T18:51:37.576514+0000","last_peered":"2024-09-10T18:51:37.576514+0000","last_clean":"2024-09-10T18:51:37.576514+0000","last_became_active":"2024-09-10T18:45:24.081050+0000","last_became_peered":"2024-09-10T18:45:24.081050+0000","last_unstale":"2024-09-10T18:51:37.576514+0000","last_undegraded":"2024-09-10T18:51:37.576514+0000","last_fullsized":"2024-09-10T18:51:37.576514+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:04.631558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:04.631558+0000","last_clean_scrub_stamp":"2024-09-10T18:50:04.631558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0085779570000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"63'11","reported_seq":149,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.582281+0000","last_change":"2024-09-10T18:49:56.622986+0000","last_active":"2024-09-10T18:51:37.582281+0000","last_peered":"2024-09-10T18:51:37.582281+0000","last_clean":"2024-09-10T18:51:37.582281+0000","last_became_active":"2024-09-10T18:45:20.056885+0000","last_became_peered":"2024-09-10T18:45:20.056885+0000","last_unstale":"2024-09-10T18:51:37.582281+0000","last_undegraded":"2024-09-10T18:51:37.582281+0000","last_fullsized":"2024-09-10T18:51:37.582281+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:49:56.622879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:49:56.622879+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0077720799999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":143,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.583376+0000","last_change":"2024-09-10T18:51:00.029596+0000","last_active":"2024-09-10T18:51:37.583376+0000","last_peered":"2024-09-10T18:51:37.583376+0000","last_clean":"2024-09-10T18:51:37.583376+0000","last_became_active":"2024-09-10T18:45:17.963542+0000","last_became_peered":"2024-09-10T18:45:17.963542+0000","last_unstale":"2024-09-10T18:51:37.583376+0000","last_undegraded":"2024-09-10T18:51:37.583376+0000","last_fullsized":"2024-09-10T18:51:37.583376+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:00.029389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:00.029389+0000","last_clean_scrub_stamp":"2024-09-10T18:51:00.029389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:25.353957+0000","scrub_duration":0.0023207890000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":142,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.582193+0000","last_change":"2024-09-10T18:50:45.003161+0000","last_active":"2024-09-10T18:51:37.582193+0000","last_peered":"2024-09-10T18:51:37.582193+0000","last_clean":"2024-09-10T18:51:37.582193+0000","last_became_active":"2024-09-10T18:45:22.070065+0000","last_became_peered":"2024-09-10T18:45:22.070065+0000","last_unstale":"2024-09-10T18:51:37.582193+0000","last_undegraded":"2024-09-10T18:51:37.582193+0000","last_fullsized":"2024-09-10T18:51:37.582193+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:45.002838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:45.002838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:58.280429+0000","scrub_duration":0.017421118999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":127,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.576252+0000","last_change":"2024-09-10T18:49:58.479928+0000","last_active":"2024-09-10T18:51:37.576252+0000","last_peered":"2024-09-10T18:51:37.576252+0000","last_clean":"2024-09-10T18:51:37.576252+0000","last_became_active":"2024-09-10T18:45:24.072224+0000","last_became_peered":"2024-09-10T18:45:24.072224+0000","last_unstale":"2024-09-10T18:51:37.576252+0000","last_undegraded":"2024-09-10T18:51:37.576252+0000","last_fullsized":"2024-09-10T18:51:37.576252+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:58.479842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:58.479842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.025012936,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"63'4","reported_seq":147,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.576826+0000","last_change":"2024-09-10T18:50:46.171962+0000","last_active":"2024-09-10T18:51:37.576826+0000","last_peered":"2024-09-10T18:51:37.576826+0000","last_clean":"2024-09-10T18:51:37.576826+0000","last_became_active":"2024-09-10T18:45:20.059205+0000","last_became_peered":"2024-09-10T18:45:20.059205+0000","last_unstale":"2024-09-10T18:51:37.576826+0000","last_undegraded":"2024-09-10T18:51:37.576826+0000","last_fullsized":"2024-09-10T18:51:37.576826+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4","last_scrub_stamp":"2024-09-10T18:50:46.171568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:50:46.171568+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:59.994766+0000","scrub_duration":0.0054395399999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":142,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.588909+0000","last_change":"2024-09-10T18:49:50.050306+0000","last_active":"2024-09-10T18:51:37.588909+0000","last_peered":"2024-09-10T18:51:37.588909+0000","last_clean":"2024-09-10T18:51:37.588909+0000","last_became_active":"2024-09-10T18:47:36.415866+0000","last_became_peered":"2024-09-10T18:47:36.415866+0000","last_unstale":"2024-09-10T18:51:37.588909+0000","last_undegraded":"2024-09-10T18:51:37.588909+0000","last_fullsized":"2024-09-10T18:51:37.588909+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:50.050248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:49:50.050248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0096805610000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":138,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.586509+0000","last_change":"2024-09-10T18:50:06.348857+0000","last_active":"2024-09-10T18:51:37.586509+0000","last_peered":"2024-09-10T18:51:37.586509+0000","last_clean":"2024-09-10T18:51:37.586509+0000","last_became_active":"2024-09-10T18:45:22.072377+0000","last_became_peered":"2024-09-10T18:45:22.072377+0000","last_unstale":"2024-09-10T18:51:37.586509+0000","last_undegraded":"2024-09-10T18:51:37.586509+0000","last_fullsized":"2024-09-10T18:51:37.586509+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:06.348715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_clean_scrub_stamp":"2024-09-10T18:50:06.348715+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0042502529999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":147,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.908986+0000","last_change":"2024-09-10T18:49:59.333755+0000","last_active":"2024-09-10T18:51:37.908986+0000","last_peered":"2024-09-10T18:51:37.908986+0000","last_clean":"2024-09-10T18:51:37.908986+0000","last_became_active":"2024-09-10T18:47:36.416187+0000","last_became_peered":"2024-09-10T18:47:36.416187+0000","last_unstale":"2024-09-10T18:51:37.908986+0000","last_undegraded":"2024-09-10T18:51:37.908986+0000","last_fullsized":"2024-09-10T18:51:37.908986+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:59.333637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:59.333637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053932670000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"79'16","reported_seq":165,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.579000+0000","last_change":"2024-09-10T18:51:36.997632+0000","last_active":"2024-09-10T18:51:37.579000+0000","last_peered":"2024-09-10T18:51:37.579000+0000","last_clean":"2024-09-10T18:51:37.579000+0000","last_became_active":"2024-09-10T18:45:20.421479+0000","last_became_peered":"2024-09-10T18:45:20.421479+0000","last_unstale":"2024-09-10T18:51:37.579000+0000","last_undegraded":"2024-09-10T18:51:37.579000+0000","last_fullsized":"2024-09-10T18:51:37.579000+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"79'16","last_scrub_stamp":"2024-09-10T18:51:36.997571+0000","last_deep_scrub":"79'16","last_deep_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.997571+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:45.812239+0000","scrub_duration":0.002345059,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"56'1","reported_seq":163,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.588259+0000","last_change":"2024-09-10T18:50:43.279453+0000","last_active":"2024-09-10T18:51:37.588259+0000","last_peered":"2024-09-10T18:51:37.588259+0000","last_clean":"2024-09-10T18:51:37.588259+0000","last_became_active":"2024-09-10T18:47:36.421985+0000","last_became_peered":"2024-09-10T18:47:36.421985+0000","last_unstale":"2024-09-10T18:51:37.588259+0000","last_undegraded":"2024-09-10T18:51:37.588259+0000","last_fullsized":"2024-09-10T18:51:37.588259+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:50:43.279224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:43.279224+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:04.568803+0000","scrub_duration":0.0048255859999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":93,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.576545+0000","last_change":"2024-09-10T18:50:41.057828+0000","last_active":"2024-09-10T18:51:37.576545+0000","last_peered":"2024-09-10T18:51:37.576545+0000","last_clean":"2024-09-10T18:51:37.576545+0000","last_became_active":"2024-09-10T18:47:57.107149+0000","last_became_peered":"2024-09-10T18:47:57.107149+0000","last_unstale":"2024-09-10T18:51:37.576545+0000","last_undegraded":"2024-09-10T18:51:37.576545+0000","last_fullsized":"2024-09-10T18:51:37.576545+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:41.057750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:41.057750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:51.643566+0000","scrub_duration":0.013401967000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":137,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.576963+0000","last_change":"2024-09-10T18:50:13.793044+0000","last_active":"2024-09-10T18:51:37.576963+0000","last_peered":"2024-09-10T18:51:37.576963+0000","last_clean":"2024-09-10T18:51:37.576963+0000","last_became_active":"2024-09-10T18:45:24.264818+0000","last_became_peered":"2024-09-10T18:45:24.264818+0000","last_unstale":"2024-09-10T18:51:37.576963+0000","last_undegraded":"2024-09-10T18:51:37.576963+0000","last_fullsized":"2024-09-10T18:51:37.576963+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:13.792946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:50:13.792946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:42.869032+0000","scrub_duration":0.012475749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"78'2","reported_seq":153,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.909297+0000","last_change":"2024-09-10T18:49:52.265525+0000","last_active":"2024-09-10T18:51:37.909297+0000","last_peered":"2024-09-10T18:51:37.909297+0000","last_clean":"2024-09-10T18:51:37.909297+0000","last_became_active":"2024-09-10T18:47:36.418702+0000","last_became_peered":"2024-09-10T18:47:36.418702+0000","last_unstale":"2024-09-10T18:51:37.909297+0000","last_undegraded":"2024-09-10T18:51:37.909297+0000","last_fullsized":"2024-09-10T18:51:37.909297+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'2","last_scrub_stamp":"2024-09-10T18:49:52.265459+0000","last_deep_scrub":"78'2","last_deep_scrub_stamp":"2024-09-10T18:46:37.581130+0000","last_clean_scrub_stamp":"2024-09-10T18:49:52.265459+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012366897999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":142,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.587367+0000","last_change":"2024-09-10T18:51:03.197559+0000","last_active":"2024-09-10T18:51:37.587367+0000","last_peered":"2024-09-10T18:51:37.587367+0000","last_clean":"2024-09-10T18:51:37.587367+0000","last_became_active":"2024-09-10T18:45:17.962108+0000","last_became_peered":"2024-09-10T18:45:17.962108+0000","last_unstale":"2024-09-10T18:51:37.587367+0000","last_undegraded":"2024-09-10T18:51:37.587367+0000","last_fullsized":"2024-09-10T18:51:37.587367+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:03.197480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:03.197480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:15.493158+0000","scrub_duration":0.0024932259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"63'10","reported_seq":156,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.578500+0000","last_change":"2024-09-10T18:50:07.754452+0000","last_active":"2024-09-10T18:51:37.578500+0000","last_peered":"2024-09-10T18:51:37.578500+0000","last_clean":"2024-09-10T18:51:37.578500+0000","last_became_active":"2024-09-10T18:45:20.421094+0000","last_became_peered":"2024-09-10T18:45:20.421094+0000","last_unstale":"2024-09-10T18:51:37.578500+0000","last_undegraded":"2024-09-10T18:51:37.578500+0000","last_fullsized":"2024-09-10T18:51:37.578500+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:50:07.754001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:50:07.754001+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.019947473,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":225,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.588062+0000","last_change":"2024-09-10T18:49:50.465667+0000","last_active":"2024-09-10T18:51:37.588062+0000","last_peered":"2024-09-10T18:51:37.588062+0000","last_clean":"2024-09-10T18:51:37.588062+0000","last_became_active":"2024-09-10T18:45:22.073281+0000","last_became_peered":"2024-09-10T18:45:22.073281+0000","last_unstale":"2024-09-10T18:51:37.588062+0000","last_undegraded":"2024-09-10T18:51:37.588062+0000","last_fullsized":"2024-09-10T18:51:37.588062+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:49:50.465613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:49:50.465613+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017767359999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":137,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-10T18:51:40.051341+0000","last_change":"2024-09-10T18:51:40.051314+0000","last_active":"2024-09-10T18:51:40.051341+0000","last_peered":"2024-09-10T18:51:40.051341+0000","last_clean":"2024-09-10T18:51:40.051341+0000","last_became_active":"2024-09-10T18:45:24.082318+0000","last_became_peered":"2024-09-10T18:45:24.082318+0000","last_unstale":"2024-09-10T18:51:40.051341+0000","last_undegraded":"2024-09-10T18:51:40.051341+0000","last_fullsized":"2024-09-10T18:51:40.051341+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:40.051241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:03.475299+0000","last_clean_scrub_stamp":"2024-09-10T18:51:40.051241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:48.927775+0000","scrub_duration":0.001564937,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":143,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.579492+0000","last_change":"2024-09-10T18:51:01.165781+0000","last_active":"2024-09-10T18:51:37.579492+0000","last_peered":"2024-09-10T18:51:37.579492+0000","last_clean":"2024-09-10T18:51:37.579492+0000","last_became_active":"2024-09-10T18:45:17.965707+0000","last_became_peered":"2024-09-10T18:45:17.965707+0000","last_unstale":"2024-09-10T18:51:37.579492+0000","last_undegraded":"2024-09-10T18:51:37.579492+0000","last_fullsized":"2024-09-10T18:51:37.579492+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:01.165584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:01.165584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:28.929394+0000","scrub_duration":0.0071542790000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"63'11","reported_seq":159,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.588159+0000","last_change":"2024-09-10T18:49:58.405230+0000","last_active":"2024-09-10T18:51:37.588159+0000","last_peered":"2024-09-10T18:51:37.588159+0000","last_clean":"2024-09-10T18:51:37.588159+0000","last_became_active":"2024-09-10T18:45:20.058717+0000","last_became_peered":"2024-09-10T18:45:20.058717+0000","last_unstale":"2024-09-10T18:51:37.588159+0000","last_undegraded":"2024-09-10T18:51:37.588159+0000","last_fullsized":"2024-09-10T18:51:37.588159+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:49:58.405145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:49:58.405145+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040739369999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":149,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.578421+0000","last_change":"2024-09-10T18:50:54.765172+0000","last_active":"2024-09-10T18:51:37.578421+0000","last_peered":"2024-09-10T18:51:37.578421+0000","last_clean":"2024-09-10T18:51:37.578421+0000","last_became_active":"2024-09-10T18:47:36.418767+0000","last_became_peered":"2024-09-10T18:47:36.418767+0000","last_unstale":"2024-09-10T18:51:37.578421+0000","last_undegraded":"2024-09-10T18:51:37.578421+0000","last_fullsized":"2024-09-10T18:51:37.578421+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:54.764990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:54.764990+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:14.182768+0000","scrub_duration":0.0043824299999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":136,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.582502+0000","last_change":"2024-09-10T18:51:36.324657+0000","last_active":"2024-09-10T18:51:37.582502+0000","last_peered":"2024-09-10T18:51:37.582502+0000","last_clean":"2024-09-10T18:51:37.582502+0000","last_became_active":"2024-09-10T18:45:24.267251+0000","last_became_peered":"2024-09-10T18:45:24.267251+0000","last_unstale":"2024-09-10T18:51:37.582502+0000","last_undegraded":"2024-09-10T18:51:37.582502+0000","last_fullsized":"2024-09-10T18:51:37.582502+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.324592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.324592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:56.184657+0000","scrub_duration":0.001354409,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"56'1","reported_seq":151,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.586281+0000","last_change":"2024-09-10T18:50:07.341477+0000","last_active":"2024-09-10T18:51:37.586281+0000","last_peered":"2024-09-10T18:51:37.586281+0000","last_clean":"2024-09-10T18:51:37.586281+0000","last_became_active":"2024-09-10T18:45:17.966286+0000","last_became_peered":"2024-09-10T18:45:17.966286+0000","last_unstale":"2024-09-10T18:51:37.586281+0000","last_undegraded":"2024-09-10T18:51:37.586281+0000","last_fullsized":"2024-09-10T18:51:37.586281+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:50:07.341394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:07.341394+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019580940000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"63'15","reported_seq":147,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.582797+0000","last_change":"2024-09-10T18:50:00.455229+0000","last_active":"2024-09-10T18:51:37.582797+0000","last_peered":"2024-09-10T18:51:37.582797+0000","last_clean":"2024-09-10T18:51:37.582797+0000","last_became_active":"2024-09-10T18:47:36.424373+0000","last_became_peered":"2024-09-10T18:47:36.424373+0000","last_unstale":"2024-09-10T18:51:37.582797+0000","last_undegraded":"2024-09-10T18:51:37.582797+0000","last_fullsized":"2024-09-10T18:51:37.582797+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:50:00.455005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:50:00.455005+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.018138438999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":129,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.582857+0000","last_change":"2024-09-10T18:49:51.367158+0000","last_active":"2024-09-10T18:51:37.582857+0000","last_peered":"2024-09-10T18:51:37.582857+0000","last_clean":"2024-09-10T18:51:37.582857+0000","last_became_active":"2024-09-10T18:45:22.059794+0000","last_became_peered":"2024-09-10T18:45:22.059794+0000","last_unstale":"2024-09-10T18:51:37.582857+0000","last_undegraded":"2024-09-10T18:51:37.582857+0000","last_fullsized":"2024-09-10T18:51:37.582857+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:51.367083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:49:51.367083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051050030000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":127,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.579310+0000","last_change":"2024-09-10T18:49:51.464014+0000","last_active":"2024-09-10T18:51:37.579310+0000","last_peered":"2024-09-10T18:51:37.579310+0000","last_clean":"2024-09-10T18:51:37.579310+0000","last_became_active":"2024-09-10T18:45:24.264427+0000","last_became_peered":"2024-09-10T18:45:24.264427+0000","last_unstale":"2024-09-10T18:51:37.579310+0000","last_undegraded":"2024-09-10T18:51:37.579310+0000","last_fullsized":"2024-09-10T18:51:37.579310+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:51.463929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:51.463929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.028103624000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":143,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-10T18:51:40.076347+0000","last_change":"2024-09-10T18:51:40.076322+0000","last_active":"2024-09-10T18:51:40.076347+0000","last_peered":"2024-09-10T18:51:40.076347+0000","last_clean":"2024-09-10T18:51:40.076347+0000","last_became_active":"2024-09-10T18:45:17.963257+0000","last_became_peered":"2024-09-10T18:45:17.963257+0000","last_unstale":"2024-09-10T18:51:40.076347+0000","last_undegraded":"2024-09-10T18:51:40.076347+0000","last_fullsized":"2024-09-10T18:51:40.076347+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:40.076262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:40.076262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:58.187702+0000","scrub_duration":0.0037079650000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"63'12","reported_seq":164,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.578258+0000","last_change":"2024-09-10T18:51:31.148694+0000","last_active":"2024-09-10T18:51:37.578258+0000","last_peered":"2024-09-10T18:51:37.578258+0000","last_clean":"2024-09-10T18:51:37.578258+0000","last_became_active":"2024-09-10T18:45:20.058391+0000","last_became_peered":"2024-09-10T18:45:20.058391+0000","last_unstale":"2024-09-10T18:51:37.578258+0000","last_undegraded":"2024-09-10T18:51:37.578258+0000","last_fullsized":"2024-09-10T18:51:37.578258+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:51:31.148630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:31.148630+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:33.947829+0000","scrub_duration":0.0021214699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":148,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-10T18:51:37.582025+0000","last_change":"2024-09-10T18:50:41.560291+0000","last_active":"2024-09-10T18:51:37.582025+0000","last_peered":"2024-09-10T18:51:37.582025+0000","last_clean":"2024-09-10T18:51:37.582025+0000","last_became_active":"2024-09-10T18:47:36.416131+0000","last_became_peered":"2024-09-10T18:47:36.416131+0000","last_unstale":"2024-09-10T18:51:37.582025+0000","last_undegraded":"2024-09-10T18:51:37.582025+0000","last_fullsized":"2024-09-10T18:51:37.582025+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:41.560194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:41.560194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:08.620669+0000","scrub_duration":0.0067616830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":48,"acting":48,"num_store_stats":7},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":697,"num_read_kb":454,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":1238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":453,"ondisk_log_size":453,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":373,"num_read_kb":368,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":891,"ondisk_log_size":891,"up":33,"acting":33,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299473,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":302928,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93442224,"statfs":{"total":95995035648,"available":95684837376,"internally_reserved":0,"allocated":1449984,"data_stored":913821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":1.256,"15min":0.75700000000000001},"min":{"1min":0.21099999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":3.073,"5min":31.805,"15min":31.805},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.93200000000000005,"5min":1.5169999999999999,"15min":0.82899999999999996},"min":{"1min":0.23999999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":6.5010000000000003,"5min":31.212,"15min":31.212},"last":0.61299999999999999}]},{"osd":1,"last update":"Tue Sep 10 18:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.98299999999999998,"15min":0.64600000000000002},"min":{"1min":0.23100000000000001,"5min":0.221,"15min":0.221},"max":{"1min":2.9460000000000002,"5min":21.34,"15min":21.34},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.94799999999999995,"15min":0.66200000000000003},"min":{"1min":0.23599999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.869,"5min":21.248000000000001,"15min":21.248000000000001},"last":0.72299999999999998}]},{"osd":2,"last update":"Tue Sep 10 18:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.625,"15min":0.53200000000000003},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.745,"5min":5.1399999999999997,"15min":5.1399999999999997},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.72099999999999997,"15min":0.58399999999999996},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.1349999999999998,"5min":8.8859999999999992,"15min":8.8859999999999992},"last":0.60199999999999998}]},{"osd":3,"last update":"Tue Sep 10 18:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.73,"5min":1.4850000000000001,"15min":0.83299999999999996},"min":{"1min":0.251,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":9.3089999999999993,"5min":31.478000000000002,"15min":31.478000000000002},"last":0.58099999999999996},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.6699999999999999,"15min":0.90200000000000002},"min":{"1min":0.29499999999999998,"5min":0.254,"15min":0.254},"max":{"1min":9.1549999999999994,"5min":33.731999999999999,"15min":33.731999999999999},"last":0.625}]},{"osd":4,"last update":"Tue Sep 10 18:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.45700000000000002,"15min":0.40899999999999997},"min":{"1min":0.24099999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.6080000000000001,"5min":2.7559999999999998,"15min":2.7559999999999998},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.379,"5min":0.45300000000000001,"15min":0.40100000000000002},"min":{"1min":0.254,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.79800000000000004,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.46400000000000002}]},{"osd":5,"last update":"Tue Sep 10 18:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.46899999999999997,"15min":0.43099999999999999},"min":{"1min":0.26900000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.5489999999999999,"5min":2.7240000000000002,"15min":2.7240000000000002},"last":0.443},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.44700000000000001,"15min":0.42599999999999999},"min":{"1min":0.27100000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.85899999999999999,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.42899999999999999}]},{"osd":6,"last update":"Tue Sep 10 18:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.54700000000000004,"15min":0.48099999999999998},"min":{"1min":0.27200000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.6299999999999999,"5min":2.8199999999999998,"15min":2.8199999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.44,"5min":0.53500000000000003,"15min":0.46700000000000003},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71999999999999997,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.40999999999999998}]}]},{"osd":6,"up_from":43,"seq":184683593814,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":302880,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93442272,"statfs":{"total":95995035648,"available":95684886528,"internally_reserved":0,"allocated":1396736,"data_stored":849373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:50:47 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.518,"15min":0.81100000000000005},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":4.9420000000000002,"5min":46.384999999999998,"15min":46.384999999999998},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.877,"5min":1.4450000000000001,"15min":0.80900000000000005},"min":{"1min":0.245,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":2.5760000000000001,"5min":48.372999999999998,"15min":48.372999999999998},"last":0.47299999999999998}]},{"osd":1,"last update":"Tue Sep 10 18:50:47 2024","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":0.877,"15min":0.61799999999999999},"min":{"1min":0.22600000000000001,"5min":0.219,"15min":0.219},"max":{"1min":2.9049999999999998,"5min":6.1319999999999997,"15min":6.1319999999999997},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.875,"5min":0.73599999999999999,"15min":0.55200000000000005},"min":{"1min":0.28399999999999997,"5min":0.214,"15min":0.214},"max":{"1min":1.8979999999999999,"5min":3.6280000000000001,"15min":3.6280000000000001},"last":0.49299999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:50:47 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.76100000000000001,"15min":0.57399999999999995},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":4.0119999999999996,"5min":4.4740000000000002,"15min":4.4740000000000002},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.60499999999999998,"15min":0.52800000000000002},"min":{"1min":0.35999999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.327,"5min":2.9900000000000002,"15min":2.9900000000000002},"last":0.58199999999999996}]},{"osd":3,"last update":"Tue Sep 10 18:50:47 2024","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":0.80400000000000005,"15min":0.58999999999999997},"min":{"1min":0.27600000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":4.2110000000000003,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":1.276,"5min":1.0049999999999999,"15min":0.67500000000000004},"min":{"1min":0.36899999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":4.1349999999999998,"5min":5.3079999999999998,"15min":5.3079999999999998},"last":0.55300000000000005}]},{"osd":4,"last update":"Tue Sep 10 18:50:47 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.55900000000000005,"15min":0.45600000000000002},"min":{"1min":0.252,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":2.6920000000000002,"5min":4.2969999999999997,"15min":4.2969999999999997},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.51200000000000001,"15min":0.42599999999999999},"min":{"1min":0.28599999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.365,"5min":4.3209999999999997,"15min":4.3209999999999997},"last":0.625}]},{"osd":5,"last update":"Tue Sep 10 18:50:47 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.56299999999999994,"15min":0.46300000000000002},"min":{"1min":0.28799999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":2.371,"5min":4.4210000000000003,"15min":4.4210000000000003},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.57899999999999996,"15min":0.47399999999999998},"min":{"1min":0.33400000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":2.0339999999999998,"5min":4.4500000000000002,"15min":4.4500000000000002},"last":0.70199999999999996}]},{"osd":7,"last update":"Tue Sep 10 18:51:12 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.66800000000000004,"15min":0.53500000000000003},"min":{"1min":0.35599999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":3.1509999999999998,"5min":4.5259999999999998,"15min":4.5259999999999998},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.67200000000000004,"15min":0.55300000000000005},"min":{"1min":0.36099999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.5920000000000001,"5min":4.1299999999999999,"15min":4.1299999999999999},"last":0.68100000000000005}]}]},{"osd":5,"up_from":37,"seq":158913790042,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":307348,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93437804,"statfs":{"total":95995035648,"available":95680311296,"internally_reserved":0,"allocated":1712128,"data_stored":1175977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:51:28 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":1.502,"15min":0.85099999999999998},"min":{"1min":0.23599999999999999,"5min":0.183,"15min":0.183},"max":{"1min":3.6859999999999999,"5min":65.697000000000003,"15min":65.697000000000003},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.753,"5min":1.5669999999999999,"15min":0.85299999999999998},"min":{"1min":0.29899999999999999,"5min":0.219,"15min":0.219},"max":{"1min":3.0419999999999998,"5min":65.683999999999997,"15min":65.683999999999997},"last":0.52800000000000002}]},{"osd":1,"last update":"Tue Sep 10 18:51:28 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.67800000000000005,"15min":0.59799999999999998},"min":{"1min":0.313,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.0700000000000001,"5min":3.5150000000000001,"15min":3.5150000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.80100000000000005,"15min":0.625},"min":{"1min":0.29299999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.1080000000000001,"5min":6.6989999999999998,"15min":6.6989999999999998},"last":0.48699999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:51:28 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.65400000000000003,"15min":0.56399999999999995},"min":{"1min":0.34999999999999998,"5min":0.249,"15min":0.249},"max":{"1min":2.5449999999999999,"5min":7.8029999999999999,"15min":7.8029999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.72299999999999998,"15min":0.60099999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.052,"5min":6.7210000000000001,"15min":6.7210000000000001},"last":0.503}]},{"osd":3,"last update":"Tue Sep 10 18:51:28 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":1.6519999999999999,"15min":0.90100000000000002},"min":{"1min":0.28299999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":4.2450000000000001,"5min":66.593999999999994,"15min":66.593999999999994},"last":0.56799999999999995},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.7490000000000001,"15min":0.94599999999999995},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":6.6870000000000003,"5min":65.939999999999998,"15min":65.939999999999998},"last":0.436}]},{"osd":4,"last update":"Tue Sep 10 18:51:28 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.47799999999999998,"15min":0.41399999999999998},"min":{"1min":0.253,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":1.1379999999999999,"5min":2.133,"15min":2.133},"last":0.373},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.498,"15min":0.435},"min":{"1min":0.24399999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":2.5099999999999998,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":0.623}]},{"osd":6,"last update":"Tue Sep 10 18:50:51 2024","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":0.75900000000000001,"15min":0.52900000000000003},"min":{"1min":0.29799999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":11.17,"5min":11.17,"15min":11.17},"last":0.45600000000000002},{"interface":"front","average":{"1min":1.413,"5min":0.79800000000000004,"15min":0.57199999999999995},"min":{"1min":0.34999999999999998,"5min":0.33000000000000002,"15min":0.31},"max":{"1min":11.18,"5min":11.18,"15min":11.18},"last":0.65800000000000003}]},{"osd":7,"last update":"Tue Sep 10 18:51:14 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.69599999999999995,"15min":0.55000000000000004},"min":{"1min":0.33100000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.6880000000000002,"5min":10.782999999999999,"15min":10.782999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.67800000000000005,"15min":0.54400000000000004},"min":{"1min":0.31,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.8879999999999999,"5min":11.846,"15min":11.846},"last":0.59999999999999998}]}]},{"osd":4,"up_from":30,"seq":128849018976,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":306784,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93438368,"statfs":{"total":95995035648,"available":95680888832,"internally_reserved":0,"allocated":1126400,"data_stored":585992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:51:11 2024","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.456,"15min":0.83499999999999996},"min":{"1min":0.221,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":4.7050000000000001,"5min":55.776000000000003,"15min":55.776000000000003},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.998,"5min":1.421,"15min":0.83499999999999996},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":3.3260000000000001,"5min":55.737000000000002,"15min":55.737000000000002},"last":0.47899999999999998}]},{"osd":1,"last update":"Tue Sep 10 18:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.86399999999999999,"15min":0.67300000000000004},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":2.9409999999999998,"5min":8.1600000000000001,"15min":8.1600000000000001},"last":0.39000000000000001},{"interface":"front","average":{"1min":1.01,"5min":0.72899999999999998,"15min":0.59999999999999998},"min":{"1min":0.28000000000000003,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":2.5990000000000002,"5min":2.5990000000000002,"15min":2.5990000000000002},"last":0.46200000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.55100000000000005,"15min":0.54900000000000004},"min":{"1min":0.307,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":2.153,"5min":2.153,"15min":2.6030000000000002},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.56299999999999994,"15min":0.55600000000000005},"min":{"1min":0.27900000000000003,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.4790000000000001,"5min":2.8109999999999999,"15min":2.8109999999999999},"last":0.37}]},{"osd":3,"last update":"Tue Sep 10 18:51:11 2024","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.2010000000000001,"15min":0.753},"min":{"1min":0.26000000000000001,"5min":0.246,"15min":0.246},"max":{"1min":3.4009999999999998,"5min":24.957999999999998,"15min":24.957999999999998},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.90300000000000002,"5min":1.093,"15min":0.72399999999999998},"min":{"1min":0.33300000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":2.1789999999999998,"5min":24.940000000000001,"15min":24.940000000000001},"last":0.35199999999999998}]},{"osd":5,"last update":"Tue Sep 10 18:51:36 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52200000000000002,"15min":0.40899999999999997},"min":{"1min":0.26000000000000001,"5min":0.222,"15min":0.222},"max":{"1min":1.151,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":0.443},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.48199999999999998,"15min":0.40200000000000002},"min":{"1min":0.28100000000000003,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.3859999999999999,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":0.502}]},{"osd":6,"last update":"Tue Sep 10 18:50:48 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.54900000000000004,"15min":0.46300000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.2149999999999999,"5min":3.5529999999999999,"15min":3.5529999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.60899999999999999,"15min":0.46100000000000002},"min":{"1min":0.309,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":2.3410000000000002,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":0.48899999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:51:18 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.56699999999999995,"15min":0.52300000000000002},"min":{"1min":0.318,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.4359999999999999,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54300000000000004,"15min":0.48099999999999998},"min":{"1min":0.31,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.504,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":0.51500000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182501,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":311528,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93433624,"statfs":{"total":95995035648,"available":95676030976,"internally_reserved":0,"allocated":1732608,"data_stored":1176193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:50:40 2024","interfaces":[{"interface":"back","average":{"1min":2.585,"5min":1.871,"15min":0.88},"min":{"1min":0.16700000000000001,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":9.1579999999999995,"5min":47.601999999999997,"15min":47.601999999999997},"last":0.42599999999999999},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.6539999999999999,"15min":0.82099999999999995},"min":{"1min":0.34300000000000003,"5min":0.217,"15min":0.217},"max":{"1min":3.7509999999999999,"5min":46.787999999999997,"15min":46.787999999999997},"last":0.443}]},{"osd":1,"last update":"Tue Sep 10 18:50:40 2024","interfaces":[{"interface":"back","average":{"1min":2.2210000000000001,"5min":1.2809999999999999,"15min":0.70099999999999996},"min":{"1min":0.48999999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":5.476,"5min":5.827,"15min":5.827},"last":0.48899999999999999},{"interface":"front","average":{"1min":2.5489999999999999,"5min":1.341,"15min":0.72299999999999998},"min":{"1min":0.245,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":9.4060000000000006,"5min":9.4060000000000006,"15min":9.4060000000000006},"last":0.40500000000000003}]},{"osd":2,"last update":"Tue Sep 10 18:50:40 2024","interfaces":[{"interface":"back","average":{"1min":1.7430000000000001,"5min":0.96399999999999997,"15min":0.61499999999999999},"min":{"1min":0.26500000000000001,"5min":0.246,"15min":0.246},"max":{"1min":5.8220000000000001,"5min":5.8220000000000001,"15min":5.8220000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":1.536,"5min":0.879,"15min":0.58399999999999996},"min":{"1min":0.28000000000000003,"5min":0.248,"15min":0.248},"max":{"1min":7.0979999999999999,"5min":7.0979999999999999,"15min":7.0979999999999999},"last":0.46200000000000002}]},{"osd":4,"last update":"Tue Sep 10 18:51:06 2024","interfaces":[{"interface":"back","average":{"1min":1.6339999999999999,"5min":1.0649999999999999,"15min":0.71599999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":7.1790000000000003,"5min":7.1790000000000003,"15min":7.1790000000000003},"last":0.68200000000000005},{"interface":"front","average":{"1min":1.7989999999999999,"5min":1.0820000000000001,"15min":0.74099999999999999},"min":{"1min":0.315,"5min":0.311,"15min":0.311},"max":{"1min":7.1349999999999998,"5min":7.1349999999999998,"15min":7.1349999999999998},"last":0.82499999999999996}]},{"osd":5,"last update":"Tue Sep 10 18:51:36 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":1.0620000000000001,"15min":0.71399999999999997},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":2.3180000000000001,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1279999999999999,"15min":0.751},"min":{"1min":0.40500000000000003,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":3.3799999999999999,"5min":9.2509999999999994,"15min":9.2509999999999994},"last":0.89300000000000002}]},{"osd":6,"last update":"Tue Sep 10 18:50:49 2024","interfaces":[{"interface":"back","average":{"1min":2.0129999999999999,"5min":1.1910000000000001,"15min":0.77300000000000002},"min":{"1min":0.32600000000000001,"5min":0.315,"15min":0.315},"max":{"1min":9.3239999999999998,"5min":9.3239999999999998,"15min":9.3239999999999998},"last":0.85599999999999998},{"interface":"front","average":{"1min":2.077,"5min":1.1619999999999999,"15min":0.75800000000000001},"min":{"1min":0.42899999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":7.1559999999999997,"5min":7.1559999999999997,"15min":7.1559999999999997},"last":0.878}]},{"osd":7,"last update":"Tue Sep 10 18:51:22 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":1.038,"15min":0.69999999999999996},"min":{"1min":0.377,"5min":0.375,"15min":0.375},"max":{"1min":1.726,"5min":4.71,"15min":4.71},"last":0.72699999999999998},{"interface":"front","average":{"1min":1.165,"5min":1.1879999999999999,"15min":0.78000000000000003},"min":{"1min":0.40500000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":3.4329999999999998,"5min":9.2949999999999999,"15min":9.2949999999999999},"last":0.91100000000000003}]}]},{"osd":2,"up_from":0,"seq":77309411433,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":302632,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93442520,"statfs":{"total":95995035648,"available":95685140480,"internally_reserved":0,"allocated":1146880,"data_stored":586408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:50:52 2024","interfaces":[{"interface":"back","average":{"1min":2.1880000000000002,"5min":1.107,"15min":0.58399999999999996},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":13.531000000000001,"5min":13.531000000000001,"15min":13.531000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":4.0449999999999999,"5min":1.6419999999999999,"15min":0.77500000000000002},"min":{"1min":0.76000000000000001,"5min":0.252,"15min":0.21199999999999999},"max":{"1min":16.206,"5min":16.206,"15min":16.206},"last":0.316}]},{"osd":2,"last update":"Tue Sep 10 18:51:24 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.67600000000000005,"15min":0.46800000000000003},"min":{"1min":0.217,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":2.8159999999999998,"5min":3.8479999999999999,"15min":3.8479999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.84699999999999998,"15min":0.53800000000000003},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":3.754,"5min":7.093,"15min":7.093},"last":0.33000000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:50:48 2024","interfaces":[{"interface":"back","average":{"1min":2.8090000000000002,"5min":1.381,"15min":0.76600000000000001},"min":{"1min":0.56699999999999995,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":12.714,"5min":12.714,"15min":12.714},"last":0.56399999999999995},{"interface":"front","average":{"1min":2.585,"5min":1.5920000000000001,"15min":0.81399999999999995},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":12.696,"5min":12.696,"15min":12.696},"last":0.39700000000000002}]},{"osd":4,"last update":"Tue Sep 10 18:51:07 2024","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.083,"15min":0.73299999999999998},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":3.9239999999999999,"5min":4.1520000000000001,"15min":4.1520000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.85399999999999998,"15min":0.64100000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":2.8119999999999998,"5min":3.9540000000000002,"15min":3.9540000000000002},"last":0.58999999999999997}]},{"osd":5,"last update":"Tue Sep 10 18:51:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":0.98899999999999999,"15min":0.73199999999999998},"min":{"1min":0.34100000000000003,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.871,"5min":4.5869999999999997,"15min":4.5869999999999997},"last":0.67900000000000005},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.1439999999999999,"15min":0.80800000000000005},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":4,"5min":4.1130000000000004,"15min":4.1130000000000004},"last":0.54400000000000004}]},{"osd":6,"last update":"Tue Sep 10 18:50:48 2024","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.393,"15min":0.88500000000000001},"min":{"1min":0.34899999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":3.4809999999999999,"5min":38.039999999999999,"15min":38.039999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":1.397,"5min":1.5580000000000001,"15min":0.96899999999999997},"min":{"1min":0.32800000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":3.5910000000000002,"5min":38.100999999999999,"15min":38.100999999999999},"last":0.621}]},{"osd":7,"last update":"Tue Sep 10 18:51:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.02,"15min":0.77200000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":3.8279999999999998,"5min":4.6539999999999999,"15min":4.6539999999999999},"last":0.749},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":0.81100000000000005},"min":{"1min":0.36199999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":3.9700000000000002,"5min":4.1989999999999998,"15min":4.1989999999999998},"last":0.57799999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738483,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":302624,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93442528,"statfs":{"total":95995035648,"available":95685148672,"internally_reserved":0,"allocated":1130496,"data_stored":585573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 18:51:00 2024","interfaces":[{"interface":"back","average":{"1min":2.3039999999999998,"5min":1.5620000000000001,"15min":0.72499999999999998},"min":{"1min":0.17599999999999999,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":8.4399999999999995,"5min":8.4399999999999995,"15min":8.4399999999999995},"last":0.23899999999999999},{"interface":"front","average":{"1min":2.8340000000000001,"5min":1.5740000000000001,"15min":0.73599999999999999},"min":{"1min":0.23200000000000001,"5min":0.221,"15min":0.187},"max":{"1min":7.9580000000000002,"5min":7.9580000000000002,"15min":7.9580000000000002},"last":0.255}]},{"osd":2,"last update":"Tue Sep 10 18:51:32 2024","interfaces":[{"interface":"back","average":{"1min":1.6200000000000001,"5min":1.202,"15min":0.66200000000000003},"min":{"1min":0.254,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":6.4429999999999996,"5min":6.4429999999999996,"15min":6.4429999999999996},"last":0.307},{"interface":"front","average":{"1min":1.377,"5min":0.97299999999999998,"15min":0.56799999999999995},"min":{"1min":0.28000000000000003,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":8.3230000000000004,"5min":8.3230000000000004,"15min":8.3230000000000004},"last":0.27400000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:50:41 2024","interfaces":[{"interface":"back","average":{"1min":2.6219999999999999,"5min":1.556,"15min":0.78500000000000003},"min":{"1min":0.48799999999999999,"5min":0.26000000000000001,"15min":0.25700000000000001},"max":{"1min":6.5750000000000002,"5min":8.0269999999999992,"15min":8.0269999999999992},"last":0.28899999999999998},{"interface":"front","average":{"1min":2.4609999999999999,"5min":1.4039999999999999,"15min":0.74299999999999999},"min":{"1min":0.34100000000000003,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":9.8439999999999994,"5min":9.8439999999999994,"15min":9.8439999999999994},"last":0.218}]},{"osd":4,"last update":"Tue Sep 10 18:51:05 2024","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":0.97499999999999998,"15min":0.68400000000000005},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":4.992,"5min":4.992,"15min":4.992},"last":0.34799999999999998},{"interface":"front","average":{"1min":1.6299999999999999,"5min":1.228,"15min":0.77200000000000002},"min":{"1min":0.40200000000000002,"5min":0.318,"15min":0.318},"max":{"1min":5.1379999999999999,"5min":5.1379999999999999,"15min":5.1379999999999999},"last":0.38900000000000001}]},{"osd":5,"last update":"Tue Sep 10 18:51:32 2024","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":1.0309999999999999,"15min":0.71799999999999997},"min":{"1min":0.36799999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":5.0129999999999999,"5min":5.0129999999999999,"15min":5.0129999999999999},"last":0.437},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.202,"15min":0.77700000000000002},"min":{"1min":0.33100000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":5.3479999999999999,"5min":5.3479999999999999,"15min":5.3479999999999999},"last":0.38}]},{"osd":6,"last update":"Tue Sep 10 18:50:55 2024","interfaces":[{"interface":"back","average":{"1min":1.853,"5min":1.256,"15min":0.78700000000000003},"min":{"1min":0.38,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":3.9769999999999999,"5min":4.6660000000000004,"15min":4.6660000000000004},"last":0.47699999999999998},{"interface":"front","average":{"1min":1.4470000000000001,"5min":0.98299999999999998,"15min":0.85199999999999998},"min":{"1min":0.33600000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":3.6499999999999999,"5min":4.1890000000000001,"15min":4.6630000000000003},"last":0.41399999999999998}]},{"osd":7,"last update":"Tue Sep 10 18:51:18 2024","interfaces":[{"interface":"back","average":{"1min":1.8,"5min":1.72,"15min":0.94399999999999995},"min":{"1min":0.34000000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":5.3799999999999999,"5min":49.313000000000002,"15min":49.313000000000002},"last":0.42599999999999999},{"interface":"front","average":{"1min":1.401,"5min":1.784,"15min":0.98999999999999999},"min":{"1min":0.45100000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":3.887,"5min":49.063000000000002,"15min":49.063000000000002},"last":0.497}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T18:51:43.833 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-10T18:51:43.833 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T18:51:43.834 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:43.859 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:43.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:43.860 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:43.960 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:43.986 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:43.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:43.987 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:44.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:43 smithi103 ceph-mon[33163]: 2.f scrub starts 2024-09-10T18:51:44.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:43 smithi103 ceph-mon[33163]: 2.f scrub ok 2024-09-10T18:51:44.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:43 smithi103 ceph-mon[33163]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:51:44.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:43 smithi103 ceph-mon[33163]: pgmap v268: 158 pgs: 158 active+clean; 582 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:51:44.088 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:44.112 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:44.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:44.113 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:44.214 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:44.237 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:44.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:44.238 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:44.339 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:44.362 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:44.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:44.364 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:44.464 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:44.488 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:44.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:44.489 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:44.589 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:44.613 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:44.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:44.614 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:44.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:44 smithi018 ceph-mon[29336]: 6.1 scrub starts 2024-09-10T18:51:44.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:44 smithi018 ceph-mon[29336]: 6.1 scrub ok 2024-09-10T18:51:44.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:44 smithi018 ceph-mon[29336]: 4.17 scrub starts 2024-09-10T18:51:44.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:44 smithi018 ceph-mon[29336]: 4.17 scrub ok 2024-09-10T18:51:44.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:44 smithi018 ceph-mon[29336]: 2.16 scrub starts 2024-09-10T18:51:44.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:44 smithi018 ceph-mon[29336]: 2.16 scrub ok 2024-09-10T18:51:44.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:44 smithi018 ceph-mon[40196]: 6.1 scrub starts 2024-09-10T18:51:44.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:44 smithi018 ceph-mon[40196]: 6.1 scrub ok 2024-09-10T18:51:44.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:44 smithi018 ceph-mon[40196]: 4.17 scrub starts 2024-09-10T18:51:44.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:44 smithi018 ceph-mon[40196]: 4.17 scrub ok 2024-09-10T18:51:44.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:44 smithi018 ceph-mon[40196]: 2.16 scrub starts 2024-09-10T18:51:44.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:44 smithi018 ceph-mon[40196]: 2.16 scrub ok 2024-09-10T18:51:44.715 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:44.738 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:44.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:44.739 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:44.840 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:44.863 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:44.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:44.864 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:44.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:44 smithi103 ceph-mon[33163]: 6.1 scrub starts 2024-09-10T18:51:44.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:44 smithi103 ceph-mon[33163]: 6.1 scrub ok 2024-09-10T18:51:44.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:44 smithi103 ceph-mon[33163]: 4.17 scrub starts 2024-09-10T18:51:44.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:44 smithi103 ceph-mon[33163]: 4.17 scrub ok 2024-09-10T18:51:44.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:44 smithi103 ceph-mon[33163]: 2.16 scrub starts 2024-09-10T18:51:44.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:44 smithi103 ceph-mon[33163]: 2.16 scrub ok 2024-09-10T18:51:44.965 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:44.988 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:44.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:44.989 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:45.089 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:45.112 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:45.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:45.113 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:45.214 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:45.235 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:45.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:45.236 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:45.337 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:45.359 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:45.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:45.361 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:45.461 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:45.486 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:45.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:45.487 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:45.588 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:45.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:45.617 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:45.617 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:45.718 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:45.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:45.740 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:45.740 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:45.841 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:45.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:45.865 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:45.865 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:45.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:45 smithi018 ceph-mon[40196]: 2.1 scrub starts 2024-09-10T18:51:45.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:45 smithi018 ceph-mon[40196]: 2.1 scrub ok 2024-09-10T18:51:45.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:45 smithi018 ceph-mon[40196]: 2.8 scrub starts 2024-09-10T18:51:45.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:45 smithi018 ceph-mon[40196]: 2.8 scrub ok 2024-09-10T18:51:45.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:45 smithi018 ceph-mon[40196]: 5.f scrub starts 2024-09-10T18:51:45.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:45 smithi018 ceph-mon[40196]: 5.f scrub ok 2024-09-10T18:51:45.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:45 smithi018 ceph-mon[40196]: 2.14 scrub starts 2024-09-10T18:51:45.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:45 smithi018 ceph-mon[40196]: 2.14 scrub ok 2024-09-10T18:51:45.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:45 smithi018 ceph-mon[40196]: pgmap v269: 158 pgs: 158 active+clean; 582 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:51:45.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:45 smithi018 ceph-mon[29336]: 2.1 scrub starts 2024-09-10T18:51:45.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:45 smithi018 ceph-mon[29336]: 2.1 scrub ok 2024-09-10T18:51:45.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:45 smithi018 ceph-mon[29336]: 2.8 scrub starts 2024-09-10T18:51:45.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:45 smithi018 ceph-mon[29336]: 2.8 scrub ok 2024-09-10T18:51:45.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:45 smithi018 ceph-mon[29336]: 5.f scrub starts 2024-09-10T18:51:45.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:45 smithi018 ceph-mon[29336]: 5.f scrub ok 2024-09-10T18:51:45.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:45 smithi018 ceph-mon[29336]: 2.14 scrub starts 2024-09-10T18:51:45.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:45 smithi018 ceph-mon[29336]: 2.14 scrub ok 2024-09-10T18:51:45.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:45 smithi018 ceph-mon[29336]: pgmap v269: 158 pgs: 158 active+clean; 582 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:51:45.966 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:45.988 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:45.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:45.989 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:46.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:45 smithi103 ceph-mon[33163]: 2.1 scrub starts 2024-09-10T18:51:46.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:45 smithi103 ceph-mon[33163]: 2.1 scrub ok 2024-09-10T18:51:46.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:45 smithi103 ceph-mon[33163]: 2.8 scrub starts 2024-09-10T18:51:46.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:45 smithi103 ceph-mon[33163]: 2.8 scrub ok 2024-09-10T18:51:46.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:45 smithi103 ceph-mon[33163]: 5.f scrub starts 2024-09-10T18:51:46.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:45 smithi103 ceph-mon[33163]: 5.f scrub ok 2024-09-10T18:51:46.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:45 smithi103 ceph-mon[33163]: 2.14 scrub starts 2024-09-10T18:51:46.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:45 smithi103 ceph-mon[33163]: 2.14 scrub ok 2024-09-10T18:51:46.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:45 smithi103 ceph-mon[33163]: pgmap v269: 158 pgs: 158 active+clean; 582 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:51:46.090 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:46.112 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:46.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:46.113 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:46.186 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:51:46.214 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:46.235 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:46.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:46.236 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:46.337 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:46.358 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:46.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:46.359 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:46.460 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:46.484 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:46.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:46.485 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:46.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:51:46 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:51:46.586 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:46.609 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:46.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:46.611 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:46.612 INFO:teuthology.orchestra.run.smithi018.stderr:enabled application 'rados' on pool 'unique_pool_1' 2024-09-10T18:51:46.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:46 smithi018 ceph-mon[29336]: 4.15 scrub starts 2024-09-10T18:51:46.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:46 smithi018 ceph-mon[29336]: 4.15 scrub ok 2024-09-10T18:51:46.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:46 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3561592650' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:51:46.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:46 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:46.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:46 smithi018 ceph-mon[40196]: 4.15 scrub starts 2024-09-10T18:51:46.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:46 smithi018 ceph-mon[40196]: 4.15 scrub ok 2024-09-10T18:51:46.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:46 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3561592650' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:51:46.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:46 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:46.711 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:46.733 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:46.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:46.734 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:46.835 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:46.856 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:46.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:46.857 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:46.958 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:46.980 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:46.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:46.981 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:47.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:46 smithi103 ceph-mon[33163]: 4.15 scrub starts 2024-09-10T18:51:47.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:46 smithi103 ceph-mon[33163]: 4.15 scrub ok 2024-09-10T18:51:47.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:46 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3561592650' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:51:47.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:46 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:47.082 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:47.104 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:47.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:47.105 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:47.206 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:47.228 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:47.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:47.229 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:47.330 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:47.352 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:47.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:47.353 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:47.454 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:47.476 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:47.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:47.477 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:47.578 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:47.601 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:47.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:47.602 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:47.617 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is unset 2024-09-10T18:51:47.703 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:47.725 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:47.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:47.726 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:47.827 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:47.851 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:47.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:47.852 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:47.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:47 smithi018 ceph-mon[40196]: 5.14 scrub starts 2024-09-10T18:51:47.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:47 smithi018 ceph-mon[40196]: 5.14 scrub ok 2024-09-10T18:51:47.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:47 smithi018 ceph-mon[40196]: 2.13 scrub starts 2024-09-10T18:51:47.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:47 smithi018 ceph-mon[40196]: 2.13 scrub ok 2024-09-10T18:51:47.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:47 smithi018 ceph-mon[40196]: 6.1c scrub starts 2024-09-10T18:51:47.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:47 smithi018 ceph-mon[40196]: 6.1c scrub ok 2024-09-10T18:51:47.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:47 smithi018 ceph-mon[40196]: 4.1f scrub starts 2024-09-10T18:51:47.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:47 smithi018 ceph-mon[40196]: 4.1f scrub ok 2024-09-10T18:51:47.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:47 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3561592650' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:51:47.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:47 smithi018 ceph-mon[40196]: osdmap e162: 8 total, 8 up, 7 in 2024-09-10T18:51:47.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:47 smithi018 ceph-mon[40196]: pgmap v271: 158 pgs: 158 active+clean; 582 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:51:47.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:47 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/628368613' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:51:47.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:47 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:51:47.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:47 smithi018 ceph-mon[29336]: 5.14 scrub starts 2024-09-10T18:51:47.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:47 smithi018 ceph-mon[29336]: 5.14 scrub ok 2024-09-10T18:51:47.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:47 smithi018 ceph-mon[29336]: 2.13 scrub starts 2024-09-10T18:51:47.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:47 smithi018 ceph-mon[29336]: 2.13 scrub ok 2024-09-10T18:51:47.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:47 smithi018 ceph-mon[29336]: 6.1c scrub starts 2024-09-10T18:51:47.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:47 smithi018 ceph-mon[29336]: 6.1c scrub ok 2024-09-10T18:51:47.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:47 smithi018 ceph-mon[29336]: 4.1f scrub starts 2024-09-10T18:51:47.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:47 smithi018 ceph-mon[29336]: 4.1f scrub ok 2024-09-10T18:51:47.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:47 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3561592650' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:51:47.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:47 smithi018 ceph-mon[29336]: osdmap e162: 8 total, 8 up, 7 in 2024-09-10T18:51:47.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:47 smithi018 ceph-mon[29336]: pgmap v271: 158 pgs: 158 active+clean; 582 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:51:47.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:47 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/628368613' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:51:47.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:47 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:51:47.953 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:47.976 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:47.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:47.977 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:48.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:47 smithi103 ceph-mon[33163]: 5.14 scrub starts 2024-09-10T18:51:48.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:47 smithi103 ceph-mon[33163]: 5.14 scrub ok 2024-09-10T18:51:48.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:47 smithi103 ceph-mon[33163]: 2.13 scrub starts 2024-09-10T18:51:48.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:47 smithi103 ceph-mon[33163]: 2.13 scrub ok 2024-09-10T18:51:48.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:47 smithi103 ceph-mon[33163]: 6.1c scrub starts 2024-09-10T18:51:48.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:47 smithi103 ceph-mon[33163]: 6.1c scrub ok 2024-09-10T18:51:48.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:47 smithi103 ceph-mon[33163]: 4.1f scrub starts 2024-09-10T18:51:48.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:47 smithi103 ceph-mon[33163]: 4.1f scrub ok 2024-09-10T18:51:48.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:47 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3561592650' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:51:48.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:47 smithi103 ceph-mon[33163]: osdmap e162: 8 total, 8 up, 7 in 2024-09-10T18:51:48.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:47 smithi103 ceph-mon[33163]: pgmap v271: 158 pgs: 158 active+clean; 582 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:51:48.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:47 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/628368613' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:51:48.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:47 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:51:48.078 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:48.101 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:48.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:48.102 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:48.203 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:48.225 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:48.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:48.226 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:48.278 INFO:teuthology.orchestra.run.smithi018.stdout:ERROR: (22) Invalid argument 2024-09-10T18:51:48.279 INFO:teuthology.orchestra.run.smithi018.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:51:48.327 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:48.349 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:48.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:48.350 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:48.451 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:48.473 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:48.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:48.474 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:48.574 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:48.596 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:48.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:48.597 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:48.698 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:48.721 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:48.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:48.722 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:48.823 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:48.847 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:48.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:48.848 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:48.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:48 smithi018 ceph-mon[40196]: 5.13 scrub starts 2024-09-10T18:51:48.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:48 smithi018 ceph-mon[40196]: 5.13 scrub ok 2024-09-10T18:51:48.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:48 smithi018 ceph-mon[40196]: 6.10 scrub starts 2024-09-10T18:51:48.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:48 smithi018 ceph-mon[40196]: 6.10 scrub ok 2024-09-10T18:51:48.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:48 smithi018 ceph-mon[40196]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T18:51:48.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:48 smithi018 ceph-mon[40196]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:51:48.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:48 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:51:48.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:48 smithi018 ceph-mon[40196]: osdmap e163: 8 total, 8 up, 7 in 2024-09-10T18:51:48.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:48 smithi018 ceph-mon[29336]: 5.13 scrub starts 2024-09-10T18:51:48.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:48 smithi018 ceph-mon[29336]: 5.13 scrub ok 2024-09-10T18:51:48.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:48 smithi018 ceph-mon[29336]: 6.10 scrub starts 2024-09-10T18:51:48.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:48 smithi018 ceph-mon[29336]: 6.10 scrub ok 2024-09-10T18:51:48.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:48 smithi018 ceph-mon[29336]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T18:51:48.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:48 smithi018 ceph-mon[29336]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:51:48.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:48 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:51:48.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:48 smithi018 ceph-mon[29336]: osdmap e163: 8 total, 8 up, 7 in 2024-09-10T18:51:48.949 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:48.972 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:48.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:48.973 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:49.017 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-10T18:51:49.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:48 smithi103 ceph-mon[33163]: 5.13 scrub starts 2024-09-10T18:51:49.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:48 smithi103 ceph-mon[33163]: 5.13 scrub ok 2024-09-10T18:51:49.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:48 smithi103 ceph-mon[33163]: 6.10 scrub starts 2024-09-10T18:51:49.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:48 smithi103 ceph-mon[33163]: 6.10 scrub ok 2024-09-10T18:51:49.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:48 smithi103 ceph-mon[33163]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T18:51:49.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:48 smithi103 ceph-mon[33163]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:51:49.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:48 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:51:49.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:48 smithi103 ceph-mon[33163]: osdmap e163: 8 total, 8 up, 7 in 2024-09-10T18:51:49.074 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:49.098 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:49.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:49.099 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:49.104 DEBUG:teuthology.orchestra.run.smithi018:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_1 bench 90 write' 2024-09-10T18:51:49.105 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2024-09-10T18:51:49.105 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T18:51:49.142 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-10T18:51:49.143 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-10T18:51:49.191 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:hints = 1 2024-09-10T18:51:49.191 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-10T18:51:49.191 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Object prefix: benchmark_data_smithi018_106531 2024-09-10T18:51:49.192 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T18:51:49.192 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 0 0 0 0 0 0 - 0 2024-09-10T18:51:49.199 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:49.247 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:49.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:49.248 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:49.349 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:49.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:49.382 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:49.382 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:49.483 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:49.521 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:49.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:49.522 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:49.623 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:49.656 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:49.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:49.657 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:49.758 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:49.797 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:49.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:49.798 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:49.899 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:49.933 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:49.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:49.934 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:49.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:49 smithi018 ceph-mon[40196]: 2.6 scrub starts 2024-09-10T18:51:49.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:49 smithi018 ceph-mon[40196]: 2.6 scrub ok 2024-09-10T18:51:49.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:49 smithi018 ceph-mon[40196]: 6.b scrub starts 2024-09-10T18:51:49.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:49 smithi018 ceph-mon[40196]: 6.b scrub ok 2024-09-10T18:51:49.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:49 smithi018 ceph-mon[40196]: 2.5 deep-scrub starts 2024-09-10T18:51:49.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:49 smithi018 ceph-mon[40196]: 2.5 deep-scrub ok 2024-09-10T18:51:49.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:49 smithi018 ceph-mon[40196]: 6.1a scrub starts 2024-09-10T18:51:49.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:49 smithi018 ceph-mon[40196]: 6.1a scrub ok 2024-09-10T18:51:49.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:49 smithi018 ceph-mon[40196]: pgmap v273: 158 pgs: 158 active+clean; 582 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 678 B/s rd, 0 op/s 2024-09-10T18:51:49.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:49 smithi018 ceph-mon[29336]: 2.6 scrub starts 2024-09-10T18:51:49.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:49 smithi018 ceph-mon[29336]: 2.6 scrub ok 2024-09-10T18:51:49.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:49 smithi018 ceph-mon[29336]: 6.b scrub starts 2024-09-10T18:51:49.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:49 smithi018 ceph-mon[29336]: 6.b scrub ok 2024-09-10T18:51:49.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:49 smithi018 ceph-mon[29336]: 2.5 deep-scrub starts 2024-09-10T18:51:49.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:49 smithi018 ceph-mon[29336]: 2.5 deep-scrub ok 2024-09-10T18:51:49.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:49 smithi018 ceph-mon[29336]: 6.1a scrub starts 2024-09-10T18:51:49.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:49 smithi018 ceph-mon[29336]: 6.1a scrub ok 2024-09-10T18:51:49.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:49 smithi018 ceph-mon[29336]: pgmap v273: 158 pgs: 158 active+clean; 582 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 678 B/s rd, 0 op/s 2024-09-10T18:51:50.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:49 smithi103 ceph-mon[33163]: 2.6 scrub starts 2024-09-10T18:51:50.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:49 smithi103 ceph-mon[33163]: 2.6 scrub ok 2024-09-10T18:51:50.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:49 smithi103 ceph-mon[33163]: 6.b scrub starts 2024-09-10T18:51:50.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:49 smithi103 ceph-mon[33163]: 6.b scrub ok 2024-09-10T18:51:50.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:49 smithi103 ceph-mon[33163]: 2.5 deep-scrub starts 2024-09-10T18:51:50.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:49 smithi103 ceph-mon[33163]: 2.5 deep-scrub ok 2024-09-10T18:51:50.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:49 smithi103 ceph-mon[33163]: 6.1a scrub starts 2024-09-10T18:51:50.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:49 smithi103 ceph-mon[33163]: 6.1a scrub ok 2024-09-10T18:51:50.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:49 smithi103 ceph-mon[33163]: pgmap v273: 158 pgs: 158 active+clean; 582 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 678 B/s rd, 0 op/s 2024-09-10T18:51:50.035 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:50.064 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:50.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:50.066 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:50.166 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:50.192 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 1 16 3921 3905 243.932 244.062 0.00256403 0.00408016 2024-09-10T18:51:50.205 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:50.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:50.206 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:50.307 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:50.342 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:50.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:50.343 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:50.444 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:50.478 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:50.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:50.479 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:50.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:51:50 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:51:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:51:50.580 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:50.614 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:50.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:50.615 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:50.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:50 smithi018 ceph-mon[40196]: 3.0 scrub starts 2024-09-10T18:51:50.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:50 smithi018 ceph-mon[40196]: 3.0 scrub ok 2024-09-10T18:51:50.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:50 smithi018 ceph-mon[29336]: 3.0 scrub starts 2024-09-10T18:51:50.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:50 smithi018 ceph-mon[29336]: 3.0 scrub ok 2024-09-10T18:51:50.716 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:50.756 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:50.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:50.757 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:50.858 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:50.897 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:50.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:50.898 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:50.999 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:51.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:50 smithi103 ceph-mon[33163]: 3.0 scrub starts 2024-09-10T18:51:51.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:50 smithi103 ceph-mon[33163]: 3.0 scrub ok 2024-09-10T18:51:51.043 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:51.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:51.044 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:51.145 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:51.177 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:51.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:51.178 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:51.192 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 2 15 7561 7546 235.739 227.562 0.0045655 0.0042324 2024-09-10T18:51:51.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:51:50 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:51:50] "GET /metrics HTTP/1.1" 200 223661 "" "Prometheus/2.43.0" 2024-09-10T18:51:51.279 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:51.315 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:51.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:51.317 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:51.417 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:51.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:51.451 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:51.451 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:51.514 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:51:51.552 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:51.574 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:51:51.575 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:51:51.578 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:51:51.598 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:51.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:51.599 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:51.616 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:51:51.616 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:51:51.645 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:51:51.645 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:51:51.678 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:51:51.678 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:51:51.700 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:51.711 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:51:51.711 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:51:51.738 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:51.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:51.739 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:51.744 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:51:51.744 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:51:51.771 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:51 smithi018 ceph-mon[40196]: 5.d scrub starts 2024-09-10T18:51:51.771 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:51 smithi018 ceph-mon[40196]: 5.d scrub ok 2024-09-10T18:51:51.771 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:51 smithi018 ceph-mon[40196]: 5.1f scrub starts 2024-09-10T18:51:51.771 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:51 smithi018 ceph-mon[40196]: 5.1f scrub ok 2024-09-10T18:51:51.771 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:51 smithi018 ceph-mon[40196]: pgmap v274: 158 pgs: 158 active+clean; 170 MiB data, 2.5 GiB used, 623 GiB / 626 GiB avail; 639 B/s rd, 21 MiB/s wr, 339 op/s 2024-09-10T18:51:51.771 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:51 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2481245995' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:51:51.772 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:51:51.772 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:51 smithi018 ceph-mon[29336]: 5.d scrub starts 2024-09-10T18:51:51.772 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:51 smithi018 ceph-mon[29336]: 5.d scrub ok 2024-09-10T18:51:51.772 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:51 smithi018 ceph-mon[29336]: 5.1f scrub starts 2024-09-10T18:51:51.772 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:51 smithi018 ceph-mon[29336]: 5.1f scrub ok 2024-09-10T18:51:51.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:51 smithi018 ceph-mon[29336]: pgmap v274: 158 pgs: 158 active+clean; 170 MiB data, 2.5 GiB used, 623 GiB / 626 GiB avail; 639 B/s rd, 21 MiB/s wr, 339 op/s 2024-09-10T18:51:51.773 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:51 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2481245995' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:51:51.773 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:51:51.790 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:51:51.790 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:51:51.840 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:51.876 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:51.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:51.877 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:51.978 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:52.015 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:52.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:52.016 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:52.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:51 smithi103 ceph-mon[33163]: 5.d scrub starts 2024-09-10T18:51:52.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:51 smithi103 ceph-mon[33163]: 5.d scrub ok 2024-09-10T18:51:52.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:51 smithi103 ceph-mon[33163]: 5.1f scrub starts 2024-09-10T18:51:52.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:51 smithi103 ceph-mon[33163]: 5.1f scrub ok 2024-09-10T18:51:52.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:51 smithi103 ceph-mon[33163]: pgmap v274: 158 pgs: 158 active+clean; 170 MiB data, 2.5 GiB used, 623 GiB / 626 GiB avail; 639 B/s rd, 21 MiB/s wr, 339 op/s 2024-09-10T18:51:52.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:51 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2481245995' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:51:52.117 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:52.152 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:52.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:52.153 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:52.192 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 3 16 10719 10703 222.926 197.312 0.00374721 0.0044771 2024-09-10T18:51:52.254 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:52.288 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:52.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:52.290 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:52.390 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:52.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:52.430 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:52.431 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:52.532 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:52.569 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:52.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:52.570 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:52.671 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:52.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:52 smithi018 ceph-mon[40196]: 2.9 scrub starts 2024-09-10T18:51:52.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:52 smithi018 ceph-mon[40196]: 2.9 scrub ok 2024-09-10T18:51:52.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:52 smithi018 ceph-mon[40196]: 2.1d scrub starts 2024-09-10T18:51:52.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:52 smithi018 ceph-mon[40196]: 2.1d scrub ok 2024-09-10T18:51:52.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:52 smithi018 ceph-mon[40196]: 6.19 scrub starts 2024-09-10T18:51:52.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:52 smithi018 ceph-mon[40196]: 6.19 scrub ok 2024-09-10T18:51:52.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:52 smithi018 ceph-mon[29336]: 2.9 scrub starts 2024-09-10T18:51:52.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:52 smithi018 ceph-mon[29336]: 2.9 scrub ok 2024-09-10T18:51:52.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:52 smithi018 ceph-mon[29336]: 2.1d scrub starts 2024-09-10T18:51:52.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:52 smithi018 ceph-mon[29336]: 2.1d scrub ok 2024-09-10T18:51:52.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:52 smithi018 ceph-mon[29336]: 6.19 scrub starts 2024-09-10T18:51:52.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:52 smithi018 ceph-mon[29336]: 6.19 scrub ok 2024-09-10T18:51:52.705 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:52.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:52.706 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:52.807 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:52.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:52.841 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:52.842 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:52.937 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-10T18:51:52.942 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:53.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:52 smithi103 ceph-mon[33163]: 2.9 scrub starts 2024-09-10T18:51:53.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:52 smithi103 ceph-mon[33163]: 2.9 scrub ok 2024-09-10T18:51:53.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:52 smithi103 ceph-mon[33163]: 2.1d scrub starts 2024-09-10T18:51:53.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:52 smithi103 ceph-mon[33163]: 2.1d scrub ok 2024-09-10T18:51:53.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:52 smithi103 ceph-mon[33163]: 6.19 scrub starts 2024-09-10T18:51:53.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:52 smithi103 ceph-mon[33163]: 6.19 scrub ok 2024-09-10T18:51:53.026 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:53.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:53.027 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:53.128 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:53.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:53.175 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:53.176 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:53.193 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 4 16 13993 13977 218.333 204.625 0.00599959 0.00457181 2024-09-10T18:51:53.277 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:53.333 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:53.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:53.355 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:53.456 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:53.494 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:53.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:53.496 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:53.596 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:53.631 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:53.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:53.632 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:53.733 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:53.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:53.780 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:53.781 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:53.881 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:53.916 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:53.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:53.917 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:53.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:53 smithi018 ceph-mon[29336]: 6.2 scrub starts 2024-09-10T18:51:53.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:53 smithi018 ceph-mon[29336]: 6.2 scrub ok 2024-09-10T18:51:53.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:53 smithi018 ceph-mon[29336]: 4.5 scrub starts 2024-09-10T18:51:53.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:53 smithi018 ceph-mon[29336]: 4.5 scrub ok 2024-09-10T18:51:53.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:53 smithi018 ceph-mon[29336]: pgmap v275: 158 pgs: 158 active+clean; 486 MiB data, 3.1 GiB used, 623 GiB / 626 GiB avail; 1.2 KiB/s rd, 61 MiB/s wr, 972 op/s 2024-09-10T18:51:53.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:53 smithi018 ceph-mon[40196]: 6.2 scrub starts 2024-09-10T18:51:53.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:53 smithi018 ceph-mon[40196]: 6.2 scrub ok 2024-09-10T18:51:53.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:53 smithi018 ceph-mon[40196]: 4.5 scrub starts 2024-09-10T18:51:53.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:53 smithi018 ceph-mon[40196]: 4.5 scrub ok 2024-09-10T18:51:53.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:53 smithi018 ceph-mon[40196]: pgmap v275: 158 pgs: 158 active+clean; 486 MiB data, 3.1 GiB used, 623 GiB / 626 GiB avail; 1.2 KiB/s rd, 61 MiB/s wr, 972 op/s 2024-09-10T18:51:54.018 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:54.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:53 smithi103 ceph-mon[33163]: 6.2 scrub starts 2024-09-10T18:51:54.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:53 smithi103 ceph-mon[33163]: 6.2 scrub ok 2024-09-10T18:51:54.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:53 smithi103 ceph-mon[33163]: 4.5 scrub starts 2024-09-10T18:51:54.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:53 smithi103 ceph-mon[33163]: 4.5 scrub ok 2024-09-10T18:51:54.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:53 smithi103 ceph-mon[33163]: pgmap v275: 158 pgs: 158 active+clean; 486 MiB data, 3.1 GiB used, 623 GiB / 626 GiB avail; 1.2 KiB/s rd, 61 MiB/s wr, 972 op/s 2024-09-10T18:51:54.053 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:54.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:54.054 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:54.155 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:54.193 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 5 16 17084 17068 213.291 193.188 0.0054689 0.00468116 2024-09-10T18:51:54.201 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:54.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:54.203 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:54.303 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:54.337 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:54.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:54.338 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:54.438 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:54.473 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:54.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:54.474 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:54.575 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:54.608 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:54.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:54.610 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:54.711 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:54.746 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:54.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:54.747 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:54.847 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:54.884 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:54.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:54.885 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:54.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:54 smithi018 ceph-mon[40196]: 6.9 deep-scrub starts 2024-09-10T18:51:54.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:54 smithi018 ceph-mon[40196]: 6.9 deep-scrub ok 2024-09-10T18:51:54.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:54 smithi018 ceph-mon[40196]: 5.2 scrub starts 2024-09-10T18:51:54.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:54 smithi018 ceph-mon[40196]: 5.2 scrub ok 2024-09-10T18:51:54.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:54 smithi018 ceph-mon[40196]: 5.8 scrub starts 2024-09-10T18:51:54.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:54 smithi018 ceph-mon[40196]: 5.8 scrub ok 2024-09-10T18:51:54.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:54 smithi018 ceph-mon[29336]: 6.9 deep-scrub starts 2024-09-10T18:51:54.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:54 smithi018 ceph-mon[29336]: 6.9 deep-scrub ok 2024-09-10T18:51:54.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:54 smithi018 ceph-mon[29336]: 5.2 scrub starts 2024-09-10T18:51:54.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:54 smithi018 ceph-mon[29336]: 5.2 scrub ok 2024-09-10T18:51:54.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:54 smithi018 ceph-mon[29336]: 5.8 scrub starts 2024-09-10T18:51:54.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:54 smithi018 ceph-mon[29336]: 5.8 scrub ok 2024-09-10T18:51:54.986 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:55.019 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:55.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:55.021 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:55.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:54 smithi103 ceph-mon[33163]: 6.9 deep-scrub starts 2024-09-10T18:51:55.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:54 smithi103 ceph-mon[33163]: 6.9 deep-scrub ok 2024-09-10T18:51:55.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:54 smithi103 ceph-mon[33163]: 5.2 scrub starts 2024-09-10T18:51:55.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:54 smithi103 ceph-mon[33163]: 5.2 scrub ok 2024-09-10T18:51:55.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:54 smithi103 ceph-mon[33163]: 5.8 scrub starts 2024-09-10T18:51:55.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:54 smithi103 ceph-mon[33163]: 5.8 scrub ok 2024-09-10T18:51:55.122 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:55.182 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:55.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:55.183 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:55.193 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 6 16 19987 19971 207.982 181.438 0.00798769 0.00479976 2024-09-10T18:51:55.204 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T18:51:55.284 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:55.314 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:55.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:55.315 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:55.416 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:55.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:55.459 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:55.460 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:55.560 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:55.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:55.593 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:55.593 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:55.694 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:55.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:55.731 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:55.731 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:55.832 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:55.870 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:55.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:55.871 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:55.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:55 smithi018 ceph-mon[40196]: 5.1d scrub starts 2024-09-10T18:51:55.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:55 smithi018 ceph-mon[40196]: 5.1d scrub ok 2024-09-10T18:51:55.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:55 smithi018 ceph-mon[40196]: 2.b scrub starts 2024-09-10T18:51:55.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:55 smithi018 ceph-mon[40196]: 2.b scrub ok 2024-09-10T18:51:55.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:55 smithi018 ceph-mon[40196]: 6.d deep-scrub starts 2024-09-10T18:51:55.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:55 smithi018 ceph-mon[40196]: 6.d deep-scrub ok 2024-09-10T18:51:55.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:55 smithi018 ceph-mon[40196]: 4.1b deep-scrub starts 2024-09-10T18:51:55.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:55 smithi018 ceph-mon[40196]: 4.1b deep-scrub ok 2024-09-10T18:51:55.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:55 smithi018 ceph-mon[40196]: 5.11 scrub starts 2024-09-10T18:51:55.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:55 smithi018 ceph-mon[40196]: 5.11 scrub ok 2024-09-10T18:51:55.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:55 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:51:55.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:51:55.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:55 smithi018 ceph-mon[40196]: pgmap v276: 158 pgs: 158 active+clean; 486 MiB data, 3.1 GiB used, 623 GiB / 626 GiB avail; 600 B/s rd, 57 MiB/s wr, 911 op/s 2024-09-10T18:51:55.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:55 smithi018 ceph-mon[29336]: 5.1d scrub starts 2024-09-10T18:51:55.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:55 smithi018 ceph-mon[29336]: 5.1d scrub ok 2024-09-10T18:51:55.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:55 smithi018 ceph-mon[29336]: 2.b scrub starts 2024-09-10T18:51:55.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:55 smithi018 ceph-mon[29336]: 2.b scrub ok 2024-09-10T18:51:55.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:55 smithi018 ceph-mon[29336]: 6.d deep-scrub starts 2024-09-10T18:51:55.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:55 smithi018 ceph-mon[29336]: 6.d deep-scrub ok 2024-09-10T18:51:55.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:55 smithi018 ceph-mon[29336]: 4.1b deep-scrub starts 2024-09-10T18:51:55.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:55 smithi018 ceph-mon[29336]: 4.1b deep-scrub ok 2024-09-10T18:51:55.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:55 smithi018 ceph-mon[29336]: 5.11 scrub starts 2024-09-10T18:51:55.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:55 smithi018 ceph-mon[29336]: 5.11 scrub ok 2024-09-10T18:51:55.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:55 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:51:55.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:51:55.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:55 smithi018 ceph-mon[29336]: pgmap v276: 158 pgs: 158 active+clean; 486 MiB data, 3.1 GiB used, 623 GiB / 626 GiB avail; 600 B/s rd, 57 MiB/s wr, 911 op/s 2024-09-10T18:51:55.971 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:56.008 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:56.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:56.009 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:56.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:55 smithi103 ceph-mon[33163]: 5.1d scrub starts 2024-09-10T18:51:56.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:55 smithi103 ceph-mon[33163]: 5.1d scrub ok 2024-09-10T18:51:56.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:55 smithi103 ceph-mon[33163]: 2.b scrub starts 2024-09-10T18:51:56.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:55 smithi103 ceph-mon[33163]: 2.b scrub ok 2024-09-10T18:51:56.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:55 smithi103 ceph-mon[33163]: 6.d deep-scrub starts 2024-09-10T18:51:56.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:55 smithi103 ceph-mon[33163]: 6.d deep-scrub ok 2024-09-10T18:51:56.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:55 smithi103 ceph-mon[33163]: 4.1b deep-scrub starts 2024-09-10T18:51:56.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:55 smithi103 ceph-mon[33163]: 4.1b deep-scrub ok 2024-09-10T18:51:56.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:55 smithi103 ceph-mon[33163]: 5.11 scrub starts 2024-09-10T18:51:56.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:55 smithi103 ceph-mon[33163]: 5.11 scrub ok 2024-09-10T18:51:56.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:55 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:51:56.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:51:56.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:55 smithi103 ceph-mon[33163]: pgmap v276: 158 pgs: 158 active+clean; 486 MiB data, 3.1 GiB used, 623 GiB / 626 GiB avail; 600 B/s rd, 57 MiB/s wr, 911 op/s 2024-09-10T18:51:56.110 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:56.147 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:56.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:56.148 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:56.193 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 7 15 22544 22529 201.108 159.875 0.00655132 0.00496685 2024-09-10T18:51:56.249 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:56.271 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:56.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:56.272 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:56.373 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:56.408 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:56.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:56.409 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:56.510 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:56.547 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:56.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:56.548 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:56.644 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:51:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:51:56.649 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:56.682 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:56.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:56.683 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:56.785 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:56.820 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:56.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:56.821 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:56.922 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:56.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:56 smithi018 ceph-mon[29336]: 6.12 deep-scrub starts 2024-09-10T18:51:56.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:56 smithi018 ceph-mon[29336]: 6.12 deep-scrub ok 2024-09-10T18:51:56.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:56 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:56.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:56 smithi018 ceph-mon[29336]: 4.16 scrub starts 2024-09-10T18:51:56.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:56 smithi018 ceph-mon[29336]: 4.16 scrub ok 2024-09-10T18:51:56.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:56 smithi018 ceph-mon[40196]: 6.12 deep-scrub starts 2024-09-10T18:51:56.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:56 smithi018 ceph-mon[40196]: 6.12 deep-scrub ok 2024-09-10T18:51:56.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:56 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:56.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:56 smithi018 ceph-mon[40196]: 4.16 scrub starts 2024-09-10T18:51:56.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:56 smithi018 ceph-mon[40196]: 4.16 scrub ok 2024-09-10T18:51:56.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:56.959 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:56.959 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:57.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:56 smithi103 ceph-mon[33163]: 6.12 deep-scrub starts 2024-09-10T18:51:57.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:56 smithi103 ceph-mon[33163]: 6.12 deep-scrub ok 2024-09-10T18:51:57.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:56 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:57.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:56 smithi103 ceph-mon[33163]: 4.16 scrub starts 2024-09-10T18:51:57.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:56 smithi103 ceph-mon[33163]: 4.16 scrub ok 2024-09-10T18:51:57.060 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:57.094 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:57.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:57.095 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:57.193 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 8 15 25077 25062 195.758 158.312 0.00447408 0.00510252 2024-09-10T18:51:57.196 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:57.235 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:51:57.235 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:57.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:57.237 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:57.338 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:57.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:57.378 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:57.378 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:57.478 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:57.514 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:57.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:57.515 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:57.616 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:51:57.656 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:57.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:57.657 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:51:57.757 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:57.790 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:57.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:57.791 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:57.892 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:57.913 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:57.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:57.915 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:57.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:57 smithi018 ceph-mon[29336]: 2.2 scrub starts 2024-09-10T18:51:57.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:57 smithi018 ceph-mon[29336]: 2.2 scrub ok 2024-09-10T18:51:57.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:57 smithi018 ceph-mon[29336]: pgmap v277: 158 pgs: 158 active+clean; 1.4 GiB data, 5.9 GiB used, 620 GiB / 626 GiB avail; 1023 B/s rd, 142 MiB/s wr, 2.27k op/s 2024-09-10T18:51:57.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:57 smithi018 ceph-mon[40196]: 2.2 scrub starts 2024-09-10T18:51:57.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:57 smithi018 ceph-mon[40196]: 2.2 scrub ok 2024-09-10T18:51:57.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:57 smithi018 ceph-mon[40196]: pgmap v277: 158 pgs: 158 active+clean; 1.4 GiB data, 5.9 GiB used, 620 GiB / 626 GiB avail; 1023 B/s rd, 142 MiB/s wr, 2.27k op/s 2024-09-10T18:51:58.015 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:58.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:57 smithi103 ceph-mon[33163]: 2.2 scrub starts 2024-09-10T18:51:58.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:57 smithi103 ceph-mon[33163]: 2.2 scrub ok 2024-09-10T18:51:58.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:57 smithi103 ceph-mon[33163]: pgmap v277: 158 pgs: 158 active+clean; 1.4 GiB data, 5.9 GiB used, 620 GiB / 626 GiB avail; 1023 B/s rd, 142 MiB/s wr, 2.27k op/s 2024-09-10T18:51:58.046 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:58.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:58.047 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:58.148 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:58.180 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:58.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:58.181 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:58.193 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 9 16 27398 27382 190.117 145 0.00796998 0.00525477 2024-09-10T18:51:58.282 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:58.322 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:58.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:58.323 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:58.424 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:58.452 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:58.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:58.454 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:58.555 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:58.591 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:58.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:58.592 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:58.693 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:58.732 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:58.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:58.733 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:58.834 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:51:58.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:58.866 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:58.866 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:51:58.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:58 smithi018 ceph-mon[29336]: 4.4 scrub starts 2024-09-10T18:51:58.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:58 smithi018 ceph-mon[29336]: 4.4 scrub ok 2024-09-10T18:51:58.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:58 smithi018 ceph-mon[29336]: 5.6 scrub starts 2024-09-10T18:51:58.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:58 smithi018 ceph-mon[29336]: 5.6 scrub ok 2024-09-10T18:51:58.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:58 smithi018 ceph-mon[29336]: 6.14 scrub starts 2024-09-10T18:51:58.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:58 smithi018 ceph-mon[29336]: 6.14 scrub ok 2024-09-10T18:51:58.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:58 smithi018 ceph-mon[29336]: 4.b deep-scrub starts 2024-09-10T18:51:58.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:58 smithi018 ceph-mon[29336]: 4.b deep-scrub ok 2024-09-10T18:51:58.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:58 smithi018 ceph-mon[40196]: 4.4 scrub starts 2024-09-10T18:51:58.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:58 smithi018 ceph-mon[40196]: 4.4 scrub ok 2024-09-10T18:51:58.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:58 smithi018 ceph-mon[40196]: 5.6 scrub starts 2024-09-10T18:51:58.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:58 smithi018 ceph-mon[40196]: 5.6 scrub ok 2024-09-10T18:51:58.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:58 smithi018 ceph-mon[40196]: 6.14 scrub starts 2024-09-10T18:51:58.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:58 smithi018 ceph-mon[40196]: 6.14 scrub ok 2024-09-10T18:51:58.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:58 smithi018 ceph-mon[40196]: 4.b deep-scrub starts 2024-09-10T18:51:58.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:58 smithi018 ceph-mon[40196]: 4.b deep-scrub ok 2024-09-10T18:51:58.967 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:51:59.007 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:59.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:59.009 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:51:59.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:58 smithi103 ceph-mon[33163]: 4.4 scrub starts 2024-09-10T18:51:59.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:58 smithi103 ceph-mon[33163]: 4.4 scrub ok 2024-09-10T18:51:59.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:58 smithi103 ceph-mon[33163]: 5.6 scrub starts 2024-09-10T18:51:59.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:58 smithi103 ceph-mon[33163]: 5.6 scrub ok 2024-09-10T18:51:59.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:58 smithi103 ceph-mon[33163]: 6.14 scrub starts 2024-09-10T18:51:59.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:58 smithi103 ceph-mon[33163]: 6.14 scrub ok 2024-09-10T18:51:59.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:58 smithi103 ceph-mon[33163]: 4.b deep-scrub starts 2024-09-10T18:51:59.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:58 smithi103 ceph-mon[33163]: 4.b deep-scrub ok 2024-09-10T18:51:59.109 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:51:59.151 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:59.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:59.152 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:51:59.194 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 10 16 29562 29546 184.623 135.25 0.00626798 0.00541004 2024-09-10T18:51:59.253 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:59.261 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:51:59.269 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-09-10T18:51:59.291 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:59.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:59.292 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:59.392 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:51:59.425 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:59.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:59.426 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:51:59.527 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:59.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:59.553 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:59.554 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:51:59.654 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:51:59.695 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:59.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:59.696 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:51:59.749 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:51:59.797 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:51:59.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:59.835 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:51:59.835 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:51:59.936 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:51:59.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:59 smithi018 ceph-mon[40196]: pgmap v278: 158 pgs: 158 active+clean; 1.4 GiB data, 5.9 GiB used, 620 GiB / 626 GiB avail; 888 B/s rd, 123 MiB/s wr, 1.97k op/s 2024-09-10T18:51:59.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:51:59 smithi018 ceph-mon[40196]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:51:59.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:59 smithi018 ceph-mon[29336]: pgmap v278: 158 pgs: 158 active+clean; 1.4 GiB data, 5.9 GiB used, 620 GiB / 626 GiB avail; 888 B/s rd, 123 MiB/s wr, 1.97k op/s 2024-09-10T18:51:59.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:51:59 smithi018 ceph-mon[29336]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:51:59.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:51:59.975 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:51:59.976 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:00.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:59 smithi103 ceph-mon[33163]: pgmap v278: 158 pgs: 158 active+clean; 1.4 GiB data, 5.9 GiB used, 620 GiB / 626 GiB avail; 888 B/s rd, 123 MiB/s wr, 1.97k op/s 2024-09-10T18:52:00.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:51:59 smithi103 ceph-mon[33163]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:52:00.077 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:00.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:00.115 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:00.115 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:00.194 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 11 16 31861 31845 180.901 143.688 0.0067646 0.00552263 2024-09-10T18:52:00.216 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:00.247 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:00.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:00.248 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:00.349 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:00.393 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:00.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:00.396 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:00.497 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:00.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:52:00 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:52:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:52:00.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:00.535 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:00.535 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:00.637 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:00.674 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:00.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:00.675 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:00.731 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T18:52:00.731 INFO:teuthology.orchestra.run.smithi018.stdout: "ops": [], 2024-09-10T18:52:00.731 INFO:teuthology.orchestra.run.smithi018.stdout: "complaint_time": 30, 2024-09-10T18:52:00.731 INFO:teuthology.orchestra.run.smithi018.stdout: "num_blocked_ops": 0 2024-09-10T18:52:00.731 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T18:52:00.776 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:00.807 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:00.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:00.809 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:00.909 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:00.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:00 smithi018 ceph-mon[29336]: 4.1c scrub starts 2024-09-10T18:52:00.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:00 smithi018 ceph-mon[29336]: 4.1c scrub ok 2024-09-10T18:52:00.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:00 smithi018 ceph-mon[29336]: 5.1b scrub starts 2024-09-10T18:52:00.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:00 smithi018 ceph-mon[29336]: 5.1b scrub ok 2024-09-10T18:52:00.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:00 smithi018 ceph-mon[29336]: 4.1e scrub starts 2024-09-10T18:52:00.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:00 smithi018 ceph-mon[29336]: 4.1e scrub ok 2024-09-10T18:52:00.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:00 smithi018 ceph-mon[40196]: 4.1c scrub starts 2024-09-10T18:52:00.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:00 smithi018 ceph-mon[40196]: 4.1c scrub ok 2024-09-10T18:52:00.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:00 smithi018 ceph-mon[40196]: 5.1b scrub starts 2024-09-10T18:52:00.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:00 smithi018 ceph-mon[40196]: 5.1b scrub ok 2024-09-10T18:52:00.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:00 smithi018 ceph-mon[40196]: 4.1e scrub starts 2024-09-10T18:52:00.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:00 smithi018 ceph-mon[40196]: 4.1e scrub ok 2024-09-10T18:52:00.947 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:00.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:00.949 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:01.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:00 smithi103 ceph-mon[33163]: 4.1c scrub starts 2024-09-10T18:52:01.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:00 smithi103 ceph-mon[33163]: 4.1c scrub ok 2024-09-10T18:52:01.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:00 smithi103 ceph-mon[33163]: 5.1b scrub starts 2024-09-10T18:52:01.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:00 smithi103 ceph-mon[33163]: 5.1b scrub ok 2024-09-10T18:52:01.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:00 smithi103 ceph-mon[33163]: 4.1e scrub starts 2024-09-10T18:52:01.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:00 smithi103 ceph-mon[33163]: 4.1e scrub ok 2024-09-10T18:52:01.049 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:01.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:01.090 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:01.090 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:01.191 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:01.195 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 12 16 34183 34167 177.919 145.125 0.00401935 0.00561537 2024-09-10T18:52:01.231 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:01.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:01.232 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:01.333 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:01.370 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:01.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:01.371 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:01.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:52:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:52:00] "GET /metrics HTTP/1.1" 200 223796 "" "Prometheus/2.43.0" 2024-09-10T18:52:01.473 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:01.506 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:01.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:01.507 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:01.608 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:01.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:01.643 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:01.643 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:01.722 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:01 smithi018 ceph-mon[40196]: pgmap v279: 158 pgs: 158 active+clean; 1.7 GiB data, 7.2 GiB used, 619 GiB / 626 GiB avail; 853 B/s rd, 141 MiB/s wr, 2.26k op/s 2024-09-10T18:52:01.722 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:01 smithi018 ceph-mon[40196]: 4.6 scrub starts 2024-09-10T18:52:01.722 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:01 smithi018 ceph-mon[40196]: 4.6 scrub ok 2024-09-10T18:52:01.722 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:01 smithi018 ceph-mon[29336]: pgmap v279: 158 pgs: 158 active+clean; 1.7 GiB data, 7.2 GiB used, 619 GiB / 626 GiB avail; 853 B/s rd, 141 MiB/s wr, 2.26k op/s 2024-09-10T18:52:01.722 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:01 smithi018 ceph-mon[29336]: 4.6 scrub starts 2024-09-10T18:52:01.722 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:01 smithi018 ceph-mon[29336]: 4.6 scrub ok 2024-09-10T18:52:01.723 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":278,"stamp":"2024-09-10T18:51:59.140452+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1486624988,"num_objects":22871,"num_object_clones":0,"num_object_copies":68613,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22871,"num_whiteouts":0,"num_read":1114,"num_read_kb":866,"num_write":23105,"num_write_kb":1451243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":2371,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24105,"ondisk_log_size":24105,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":474,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":6176048,"kb_used_data":4061240,"kb_used_omap":11,"kb_used_meta":2114676,"kb_avail":650040016,"statfs":{"total":671965249536,"available":665640976384,"internally_reserved":0,"allocated":4158709760,"data_stored":4154776726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11928,"internal_metadata":2165428584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[11,6,9],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1486028800,"num_objects":22675,"num_object_clones":0,"num_object_copies":68025,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22675,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":22675,"num_write_kb":1451200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.527703"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":140,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.078804+0000","last_change":"2024-09-10T18:50:55.243820+0000","last_active":"2024-09-10T18:51:47.078804+0000","last_peered":"2024-09-10T18:51:47.078804+0000","last_clean":"2024-09-10T18:51:47.078804+0000","last_became_active":"2024-09-10T18:45:24.265488+0000","last_became_peered":"2024-09-10T18:45:24.265488+0000","last_unstale":"2024-09-10T18:51:47.078804+0000","last_undegraded":"2024-09-10T18:51:47.078804+0000","last_fullsized":"2024-09-10T18:51:47.078804+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:55.243629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:50:55.243629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:57.394455+0000","scrub_duration":0.0090834450000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":146,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.611465+0000","last_change":"2024-09-10T18:50:52.829851+0000","last_active":"2024-09-10T18:51:46.611465+0000","last_peered":"2024-09-10T18:51:46.611465+0000","last_clean":"2024-09-10T18:51:46.611465+0000","last_became_active":"2024-09-10T18:45:17.959475+0000","last_became_peered":"2024-09-10T18:45:17.959475+0000","last_unstale":"2024-09-10T18:51:46.611465+0000","last_undegraded":"2024-09-10T18:51:46.611465+0000","last_fullsized":"2024-09-10T18:51:46.611465+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:52.829778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.718270+0000","last_clean_scrub_stamp":"2024-09-10T18:50:52.829778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:54.534366+0000","scrub_duration":0.0095184889999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"63'15","reported_seq":195,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.078708+0000","last_change":"2024-09-10T18:51:34.023246+0000","last_active":"2024-09-10T18:51:47.078708+0000","last_peered":"2024-09-10T18:51:47.078708+0000","last_clean":"2024-09-10T18:51:47.078708+0000","last_became_active":"2024-09-10T18:47:36.413635+0000","last_became_peered":"2024-09-10T18:47:36.413635+0000","last_unstale":"2024-09-10T18:51:47.078708+0000","last_undegraded":"2024-09-10T18:51:47.078708+0000","last_fullsized":"2024-09-10T18:51:47.078708+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:51:34.023191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:34.023191+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:41.128415+0000","scrub_duration":0.0027158859999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":140,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-10T18:51:41.591832+0000","last_change":"2024-09-10T18:50:50.546527+0000","last_active":"2024-09-10T18:51:41.591832+0000","last_peered":"2024-09-10T18:51:41.591832+0000","last_clean":"2024-09-10T18:51:41.591832+0000","last_became_active":"2024-09-10T18:45:22.065619+0000","last_became_peered":"2024-09-10T18:45:22.065619+0000","last_unstale":"2024-09-10T18:51:41.591832+0000","last_undegraded":"2024-09-10T18:51:41.591832+0000","last_fullsized":"2024-09-10T18:51:41.591832+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:50.546473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.057932+0000","last_clean_scrub_stamp":"2024-09-10T18:50:50.546473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:16.728084+0000","scrub_duration":0.0026598759999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":146,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.079432+0000","last_change":"2024-09-10T18:50:58.174634+0000","last_active":"2024-09-10T18:51:47.079432+0000","last_peered":"2024-09-10T18:51:47.079432+0000","last_clean":"2024-09-10T18:51:47.079432+0000","last_became_active":"2024-09-10T18:45:17.966153+0000","last_became_peered":"2024-09-10T18:45:17.966153+0000","last_unstale":"2024-09-10T18:51:47.079432+0000","last_undegraded":"2024-09-10T18:51:47.079432+0000","last_fullsized":"2024-09-10T18:51:47.079432+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:58.174399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:58.174399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:09.810946+0000","scrub_duration":0.012266312,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"83'13","reported_seq":152,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-10T18:51:41.591769+0000","last_change":"2024-09-10T18:50:04.577468+0000","last_active":"2024-09-10T18:51:41.591769+0000","last_peered":"2024-09-10T18:51:41.591769+0000","last_clean":"2024-09-10T18:51:41.591769+0000","last_became_active":"2024-09-10T18:45:20.061266+0000","last_became_peered":"2024-09-10T18:45:20.061266+0000","last_unstale":"2024-09-10T18:51:41.591769+0000","last_undegraded":"2024-09-10T18:51:41.591769+0000","last_fullsized":"2024-09-10T18:51:41.591769+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'13","last_scrub_stamp":"2024-09-10T18:50:04.577301+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_clean_scrub_stamp":"2024-09-10T18:50:04.577301+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004842224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":132,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.611587+0000","last_change":"2024-09-10T18:49:52.335982+0000","last_active":"2024-09-10T18:51:46.611587+0000","last_peered":"2024-09-10T18:51:46.611587+0000","last_clean":"2024-09-10T18:51:46.611587+0000","last_became_active":"2024-09-10T18:45:22.072822+0000","last_became_peered":"2024-09-10T18:45:22.072822+0000","last_unstale":"2024-09-10T18:51:46.611587+0000","last_undegraded":"2024-09-10T18:51:46.611587+0000","last_fullsized":"2024-09-10T18:51:46.611587+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:52.335893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:49:52.335893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.018709007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":138,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:48.432542+0000","last_change":"2024-09-10T18:51:48.432515+0000","last_active":"2024-09-10T18:51:48.432542+0000","last_peered":"2024-09-10T18:51:48.432542+0000","last_clean":"2024-09-10T18:51:48.432542+0000","last_became_active":"2024-09-10T18:45:24.081002+0000","last_became_peered":"2024-09-10T18:45:24.081002+0000","last_unstale":"2024-09-10T18:51:48.432542+0000","last_undegraded":"2024-09-10T18:51:48.432542+0000","last_fullsized":"2024-09-10T18:51:48.432542+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:48.432459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:48.432459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:56.682264+0000","scrub_duration":0.0013158130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":156,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:51.188573+0000","last_change":"2024-09-10T18:51:51.188534+0000","last_active":"2024-09-10T18:51:51.188573+0000","last_peered":"2024-09-10T18:51:51.188573+0000","last_clean":"2024-09-10T18:51:51.188573+0000","last_became_active":"2024-09-10T18:45:17.961097+0000","last_became_peered":"2024-09-10T18:45:17.961097+0000","last_unstale":"2024-09-10T18:51:51.188573+0000","last_undegraded":"2024-09-10T18:51:51.188573+0000","last_fullsized":"2024-09-10T18:51:51.188573+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:51.188458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:51.188458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:04.823887+0000","scrub_duration":0.0037631460000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"163'7","reported_seq":154,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:54.262003+0000","last_change":"2024-09-10T18:51:54.261967+0000","last_active":"2024-09-10T18:51:54.262003+0000","last_peered":"2024-09-10T18:51:54.262003+0000","last_clean":"2024-09-10T18:51:54.262003+0000","last_became_active":"2024-09-10T18:45:20.057567+0000","last_became_peered":"2024-09-10T18:45:20.057567+0000","last_unstale":"2024-09-10T18:51:54.262003+0000","last_undegraded":"2024-09-10T18:51:54.262003+0000","last_fullsized":"2024-09-10T18:51:54.262003+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"163'7","last_scrub_stamp":"2024-09-10T18:51:54.261879+0000","last_deep_scrub":"163'7","last_deep_scrub_stamp":"2024-09-10T18:51:54.261879+0000","last_clean_scrub_stamp":"2024-09-10T18:51:54.261879+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:12.396567+0000","scrub_duration":0.0087662790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":152,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.211897+0000","last_change":"2024-09-10T18:51:33.438852+0000","last_active":"2024-09-10T18:51:47.211897+0000","last_peered":"2024-09-10T18:51:47.211897+0000","last_clean":"2024-09-10T18:51:47.211897+0000","last_became_active":"2024-09-10T18:45:22.076575+0000","last_became_peered":"2024-09-10T18:45:22.076575+0000","last_unstale":"2024-09-10T18:51:47.211897+0000","last_undegraded":"2024-09-10T18:51:47.211897+0000","last_fullsized":"2024-09-10T18:51:47.211897+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:33.438763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:36.509971+0000","last_clean_scrub_stamp":"2024-09-10T18:51:33.438763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:57.162571+0000","scrub_duration":0.0019905130000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":149,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:51.287573+0000","last_change":"2024-09-10T18:51:51.287526+0000","last_active":"2024-09-10T18:51:51.287573+0000","last_peered":"2024-09-10T18:51:51.287573+0000","last_clean":"2024-09-10T18:51:51.287573+0000","last_became_active":"2024-09-10T18:45:24.079534+0000","last_became_peered":"2024-09-10T18:45:24.079534+0000","last_unstale":"2024-09-10T18:51:51.287573+0000","last_undegraded":"2024-09-10T18:51:51.287573+0000","last_fullsized":"2024-09-10T18:51:51.287573+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:51.287440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:51.287440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:10.109656+0000","scrub_duration":0.038379101999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":156,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.212595+0000","last_change":"2024-09-10T18:50:58.526247+0000","last_active":"2024-09-10T18:51:47.212595+0000","last_peered":"2024-09-10T18:51:47.212595+0000","last_clean":"2024-09-10T18:51:47.212595+0000","last_became_active":"2024-09-10T18:47:36.414393+0000","last_became_peered":"2024-09-10T18:47:36.414393+0000","last_unstale":"2024-09-10T18:51:47.212595+0000","last_undegraded":"2024-09-10T18:51:47.212595+0000","last_fullsized":"2024-09-10T18:51:47.212595+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:58.526191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:58.526191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:06.694976+0000","scrub_duration":0.0021021659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"63'9","reported_seq":158,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-10T18:51:41.592323+0000","last_change":"2024-09-10T18:51:31.494087+0000","last_active":"2024-09-10T18:51:41.592323+0000","last_peered":"2024-09-10T18:51:41.592323+0000","last_clean":"2024-09-10T18:51:41.592323+0000","last_became_active":"2024-09-10T18:45:20.052652+0000","last_became_peered":"2024-09-10T18:45:20.052652+0000","last_unstale":"2024-09-10T18:51:41.592323+0000","last_undegraded":"2024-09-10T18:51:41.592323+0000","last_fullsized":"2024-09-10T18:51:41.592323+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:51:31.494021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:31.494021+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:39.434873+0000","scrub_duration":0.0024074779999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":141,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.610867+0000","last_change":"2024-09-10T18:50:03.047387+0000","last_active":"2024-09-10T18:51:46.610867+0000","last_peered":"2024-09-10T18:51:46.610867+0000","last_clean":"2024-09-10T18:51:46.610867+0000","last_became_active":"2024-09-10T18:45:22.068326+0000","last_became_peered":"2024-09-10T18:45:22.068326+0000","last_unstale":"2024-09-10T18:51:46.610867+0000","last_undegraded":"2024-09-10T18:51:46.610867+0000","last_fullsized":"2024-09-10T18:51:46.610867+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:03.047305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:03.047305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023962509999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":140,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.609582+0000","last_change":"2024-09-10T18:51:03.523117+0000","last_active":"2024-09-10T18:51:46.609582+0000","last_peered":"2024-09-10T18:51:46.609582+0000","last_clean":"2024-09-10T18:51:46.609582+0000","last_became_active":"2024-09-10T18:45:24.081488+0000","last_became_peered":"2024-09-10T18:45:24.081488+0000","last_unstale":"2024-09-10T18:51:46.609582+0000","last_undegraded":"2024-09-10T18:51:46.609582+0000","last_fullsized":"2024-09-10T18:51:46.609582+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:03.523030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:03.523030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:33.287999+0000","scrub_duration":0.002492677,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"163'1480","reported_seq":1493,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:56.224440+0000","last_change":"2024-09-10T18:51:37.588528+0000","last_active":"2024-09-10T18:51:56.224440+0000","last_peered":"2024-09-10T18:51:56.224440+0000","last_clean":"2024-09-10T18:51:56.224440+0000","last_became_active":"2024-09-10T18:51:37.588217+0000","last_became_peered":"2024-09-10T18:51:37.588217+0000","last_unstale":"2024-09-10T18:51:56.224440+0000","last_undegraded":"2024-09-10T18:51:56.224440+0000","last_fullsized":"2024-09-10T18:51:56.224440+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":1480,"log_dups_size":0,"ondisk_log_size":1480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:42.284615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96993280,"num_objects":1480,"num_object_clones":0,"num_object_copies":4440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1480,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1480,"num_write_kb":94720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"70'81","reported_seq":58,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.610572+0000","last_change":"2024-09-10T18:50:44.076618+0000","last_active":"2024-09-10T18:51:46.610572+0000","last_peered":"2024-09-10T18:51:46.610572+0000","last_clean":"2024-09-10T18:51:46.610572+0000","last_became_active":"2024-09-10T18:50:08.397233+0000","last_became_peered":"2024-09-10T18:50:08.397233+0000","last_unstale":"2024-09-10T18:51:46.610572+0000","last_undegraded":"2024-09-10T18:51:46.610572+0000","last_fullsized":"2024-09-10T18:51:46.610572+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:50:44.076559+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:50:44.076559+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:50.654888+0000","scrub_duration":0.0035380699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"78'25","reported_seq":206,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-10T18:51:41.592877+0000","last_change":"2024-09-10T18:50:58.450309+0000","last_active":"2024-09-10T18:51:41.592877+0000","last_peered":"2024-09-10T18:51:41.592877+0000","last_clean":"2024-09-10T18:51:41.592877+0000","last_became_active":"2024-09-10T18:47:36.409300+0000","last_became_peered":"2024-09-10T18:47:36.409300+0000","last_unstale":"2024-09-10T18:51:41.592877+0000","last_undegraded":"2024-09-10T18:51:41.592877+0000","last_fullsized":"2024-09-10T18:51:41.592877+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'25","last_scrub_stamp":"2024-09-10T18:50:58.450203+0000","last_deep_scrub":"78'25","last_deep_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_clean_scrub_stamp":"2024-09-10T18:50:58.450203+0000","objects_scrubbed":8,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:14.332494+0000","scrub_duration":0.0058694890000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":156,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:54.175435+0000","last_change":"2024-09-10T18:51:54.175390+0000","last_active":"2024-09-10T18:51:54.175435+0000","last_peered":"2024-09-10T18:51:54.175435+0000","last_clean":"2024-09-10T18:51:54.175435+0000","last_became_active":"2024-09-10T18:45:17.963732+0000","last_became_peered":"2024-09-10T18:45:17.963732+0000","last_unstale":"2024-09-10T18:51:54.175435+0000","last_undegraded":"2024-09-10T18:51:54.175435+0000","last_fullsized":"2024-09-10T18:51:54.175435+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:54.175305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:54.175305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:20.881315+0000","scrub_duration":0.0029840190000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":142,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.611181+0000","last_change":"2024-09-10T18:51:39.055614+0000","last_active":"2024-09-10T18:51:46.611181+0000","last_peered":"2024-09-10T18:51:46.611181+0000","last_clean":"2024-09-10T18:51:46.611181+0000","last_became_active":"2024-09-10T18:45:22.071443+0000","last_became_peered":"2024-09-10T18:45:22.071443+0000","last_unstale":"2024-09-10T18:51:46.611181+0000","last_undegraded":"2024-09-10T18:51:46.611181+0000","last_fullsized":"2024-09-10T18:51:46.611181+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:39.055556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:04.512425+0000","last_clean_scrub_stamp":"2024-09-10T18:51:39.055556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:03.733831+0000","scrub_duration":0.0016498730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":96,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.610103+0000","last_change":"2024-09-10T18:51:34.251677+0000","last_active":"2024-09-10T18:51:46.610103+0000","last_peered":"2024-09-10T18:51:46.610103+0000","last_clean":"2024-09-10T18:51:46.610103+0000","last_became_active":"2024-09-10T18:47:57.104433+0000","last_became_peered":"2024-09-10T18:47:57.104433+0000","last_unstale":"2024-09-10T18:51:46.610103+0000","last_undegraded":"2024-09-10T18:51:46.610103+0000","last_fullsized":"2024-09-10T18:51:46.610103+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:34.251618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:34.251618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:39.525376+0000","scrub_duration":0.0041421139999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"163'1436","reported_seq":1449,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:56.411605+0000","last_change":"2024-09-10T18:51:37.588201+0000","last_active":"2024-09-10T18:51:56.411605+0000","last_peered":"2024-09-10T18:51:56.411605+0000","last_clean":"2024-09-10T18:51:56.411605+0000","last_became_active":"2024-09-10T18:51:37.587766+0000","last_became_peered":"2024-09-10T18:51:37.587766+0000","last_unstale":"2024-09-10T18:51:56.411605+0000","last_undegraded":"2024-09-10T18:51:56.411605+0000","last_fullsized":"2024-09-10T18:51:56.411605+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":1436,"log_dups_size":0,"ondisk_log_size":1436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:00.739689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":94109696,"num_objects":1436,"num_object_clones":0,"num_object_copies":4308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1436,"num_write_kb":91904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.9","version":"70'81","reported_seq":210,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.212153+0000","last_change":"2024-09-10T18:50:53.567955+0000","last_active":"2024-09-10T18:51:47.212153+0000","last_peered":"2024-09-10T18:51:47.212153+0000","last_clean":"2024-09-10T18:51:47.212153+0000","last_became_active":"2024-09-10T18:50:08.399879+0000","last_became_peered":"2024-09-10T18:50:08.399879+0000","last_unstale":"2024-09-10T18:51:47.212153+0000","last_undegraded":"2024-09-10T18:51:47.212153+0000","last_fullsized":"2024-09-10T18:51:47.212153+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:50:53.567865+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:50:53.567865+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:54.014765+0000","scrub_duration":0.0066605650000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"116'15","reported_seq":162,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-10T18:51:41.592448+0000","last_change":"2024-09-10T18:51:37.298645+0000","last_active":"2024-09-10T18:51:41.592448+0000","last_peered":"2024-09-10T18:51:41.592448+0000","last_clean":"2024-09-10T18:51:41.592448+0000","last_became_active":"2024-09-10T18:45:20.053428+0000","last_became_peered":"2024-09-10T18:45:20.053428+0000","last_unstale":"2024-09-10T18:51:41.592448+0000","last_undegraded":"2024-09-10T18:51:41.592448+0000","last_fullsized":"2024-09-10T18:51:41.592448+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"116'15","last_scrub_stamp":"2024-09-10T18:51:37.298554+0000","last_deep_scrub":"116'15","last_deep_scrub_stamp":"2024-09-10T18:49:12.374527+0000","last_clean_scrub_stamp":"2024-09-10T18:51:37.298554+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:49.476419+0000","scrub_duration":0.0020143299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":146,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.611228+0000","last_change":"2024-09-10T18:51:03.086045+0000","last_active":"2024-09-10T18:51:46.611228+0000","last_peered":"2024-09-10T18:51:46.611228+0000","last_clean":"2024-09-10T18:51:46.611228+0000","last_became_active":"2024-09-10T18:45:17.966833+0000","last_became_peered":"2024-09-10T18:45:17.966833+0000","last_unstale":"2024-09-10T18:51:46.611228+0000","last_undegraded":"2024-09-10T18:51:46.611228+0000","last_fullsized":"2024-09-10T18:51:46.611228+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:03.085839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:03.085839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:06.125680+0000","scrub_duration":0.0082351739999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"89'9","reported_seq":192,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:50.194402+0000","last_change":"2024-09-10T18:51:50.194342+0000","last_active":"2024-09-10T18:51:50.194402+0000","last_peered":"2024-09-10T18:51:50.194402+0000","last_clean":"2024-09-10T18:51:50.194402+0000","last_became_active":"2024-09-10T18:47:36.421188+0000","last_became_peered":"2024-09-10T18:47:36.421188+0000","last_unstale":"2024-09-10T18:51:50.194402+0000","last_undegraded":"2024-09-10T18:51:50.194402+0000","last_fullsized":"2024-09-10T18:51:50.194402+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'9","last_scrub_stamp":"2024-09-10T18:51:50.194254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:50.194254+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:06.035300+0000","scrub_duration":0.0025134200000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":148,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-10T18:51:41.592527+0000","last_change":"2024-09-10T18:50:56.384791+0000","last_active":"2024-09-10T18:51:41.592527+0000","last_peered":"2024-09-10T18:51:41.592527+0000","last_clean":"2024-09-10T18:51:41.592527+0000","last_became_active":"2024-09-10T18:47:36.414625+0000","last_became_peered":"2024-09-10T18:47:36.414625+0000","last_unstale":"2024-09-10T18:51:41.592527+0000","last_undegraded":"2024-09-10T18:51:41.592527+0000","last_fullsized":"2024-09-10T18:51:41.592527+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:56.384742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:32.043666+0000","last_clean_scrub_stamp":"2024-09-10T18:50:56.384742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:08.079927+0000","scrub_duration":0.001329467,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3","version":"163'1358","reported_seq":1371,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:56.233366+0000","last_change":"2024-09-10T18:51:37.590013+0000","last_active":"2024-09-10T18:51:56.233366+0000","last_peered":"2024-09-10T18:51:56.233366+0000","last_clean":"2024-09-10T18:51:56.233366+0000","last_became_active":"2024-09-10T18:51:37.589730+0000","last_became_peered":"2024-09-10T18:51:37.589730+0000","last_unstale":"2024-09-10T18:51:56.233366+0000","last_undegraded":"2024-09-10T18:51:56.233366+0000","last_fullsized":"2024-09-10T18:51:56.233366+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":1358,"log_dups_size":0,"ondisk_log_size":1358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:45.681857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88997888,"num_objects":1358,"num_object_clones":0,"num_object_copies":4074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1358,"num_write_kb":86912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"70'81","reported_seq":53,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.610647+0000","last_change":"2024-09-10T18:50:40.100350+0000","last_active":"2024-09-10T18:51:46.610647+0000","last_peered":"2024-09-10T18:51:46.610647+0000","last_clean":"2024-09-10T18:51:46.610647+0000","last_became_active":"2024-09-10T18:50:08.398345+0000","last_became_peered":"2024-09-10T18:50:08.398345+0000","last_unstale":"2024-09-10T18:51:46.610647+0000","last_undegraded":"2024-09-10T18:51:46.610647+0000","last_fullsized":"2024-09-10T18:51:46.610647+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:50:40.100261+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:50:40.100261+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020638319999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"141'22","reported_seq":142,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.611115+0000","last_change":"2024-09-10T18:50:57.119238+0000","last_active":"2024-09-10T18:51:46.611115+0000","last_peered":"2024-09-10T18:51:46.611115+0000","last_clean":"2024-09-10T18:51:46.611115+0000","last_became_active":"2024-09-10T18:46:56.742946+0000","last_became_peered":"2024-09-10T18:46:56.742946+0000","last_unstale":"2024-09-10T18:51:46.611115+0000","last_undegraded":"2024-09-10T18:51:46.611115+0000","last_fullsized":"2024-09-10T18:51:46.611115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"141'22","last_scrub_stamp":"2024-09-10T18:50:57.119152+0000","last_deep_scrub":"141'22","last_deep_scrub_stamp":"2024-09-10T18:50:57.119152+0000","last_clean_scrub_stamp":"2024-09-10T18:50:57.119152+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:21.923801+0000","scrub_duration":0.020103994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":156,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:51.055084+0000","last_change":"2024-09-10T18:51:51.055043+0000","last_active":"2024-09-10T18:51:51.055084+0000","last_peered":"2024-09-10T18:51:51.055084+0000","last_clean":"2024-09-10T18:51:51.055084+0000","last_became_active":"2024-09-10T18:45:17.965409+0000","last_became_peered":"2024-09-10T18:45:17.965409+0000","last_unstale":"2024-09-10T18:51:51.055084+0000","last_undegraded":"2024-09-10T18:51:51.055084+0000","last_fullsized":"2024-09-10T18:51:51.055084+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:51.054935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_clean_scrub_stamp":"2024-09-10T18:51:51.054935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:06.936657+0000","scrub_duration":0.004679758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"63'8","reported_seq":227,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.616183+0000","last_change":"2024-09-10T18:50:48.530731+0000","last_active":"2024-09-10T18:51:47.616183+0000","last_peered":"2024-09-10T18:51:47.616183+0000","last_clean":"2024-09-10T18:51:47.616183+0000","last_became_active":"2024-09-10T18:45:22.062497+0000","last_became_peered":"2024-09-10T18:45:22.062497+0000","last_unstale":"2024-09-10T18:51:47.616183+0000","last_undegraded":"2024-09-10T18:51:47.616183+0000","last_fullsized":"2024-09-10T18:51:47.616183+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:50:48.530644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:48.530644+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:08.165585+0000","scrub_duration":0.022829841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":149,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:54.218755+0000","last_change":"2024-09-10T18:51:54.218678+0000","last_active":"2024-09-10T18:51:54.218755+0000","last_peered":"2024-09-10T18:51:54.218755+0000","last_clean":"2024-09-10T18:51:54.218755+0000","last_became_active":"2024-09-10T18:45:24.082362+0000","last_became_peered":"2024-09-10T18:45:24.082362+0000","last_unstale":"2024-09-10T18:51:54.218755+0000","last_undegraded":"2024-09-10T18:51:54.218755+0000","last_fullsized":"2024-09-10T18:51:54.218755+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:54.218602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:54.218602+0000","last_clean_scrub_stamp":"2024-09-10T18:51:54.218602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:05.263122+0000","scrub_duration":0.0059638290000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"163'1389","reported_seq":1400,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:56.086734+0000","last_change":"2024-09-10T18:51:37.589268+0000","last_active":"2024-09-10T18:51:56.086734+0000","last_peered":"2024-09-10T18:51:56.086734+0000","last_clean":"2024-09-10T18:51:56.086734+0000","last_became_active":"2024-09-10T18:51:37.588723+0000","last_became_peered":"2024-09-10T18:51:37.588723+0000","last_unstale":"2024-09-10T18:51:56.086734+0000","last_undegraded":"2024-09-10T18:51:56.086734+0000","last_fullsized":"2024-09-10T18:51:56.086734+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":1389,"log_dups_size":0,"ondisk_log_size":1389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:45.854850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":91029504,"num_objects":1389,"num_object_clones":0,"num_object_copies":4167,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1389,"num_write_kb":88896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"63'11","reported_seq":227,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-10T18:51:41.592747+0000","last_change":"2024-09-10T18:50:42.581936+0000","last_active":"2024-09-10T18:51:41.592747+0000","last_peered":"2024-09-10T18:51:41.592747+0000","last_clean":"2024-09-10T18:51:41.592747+0000","last_became_active":"2024-09-10T18:47:57.106935+0000","last_became_peered":"2024-09-10T18:47:57.106935+0000","last_unstale":"2024-09-10T18:51:41.592747+0000","last_undegraded":"2024-09-10T18:51:41.592747+0000","last_fullsized":"2024-09-10T18:51:41.592747+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:50:42.581837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:50:42.581837+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:08.935957+0000","scrub_duration":0.0093156460000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":156,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.211946+0000","last_change":"2024-09-10T18:51:44.209450+0000","last_active":"2024-09-10T18:51:47.211946+0000","last_peered":"2024-09-10T18:51:47.211946+0000","last_clean":"2024-09-10T18:51:47.211946+0000","last_became_active":"2024-09-10T18:45:17.966133+0000","last_became_peered":"2024-09-10T18:45:17.966133+0000","last_unstale":"2024-09-10T18:51:47.211946+0000","last_undegraded":"2024-09-10T18:51:47.211946+0000","last_fullsized":"2024-09-10T18:51:47.211946+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:44.209392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:44.209392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:04.575890+0000","scrub_duration":0.001553123,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":151,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.611341+0000","last_change":"2024-09-10T18:51:44.388198+0000","last_active":"2024-09-10T18:51:46.611341+0000","last_peered":"2024-09-10T18:51:46.611341+0000","last_clean":"2024-09-10T18:51:46.611341+0000","last_became_active":"2024-09-10T18:45:22.068918+0000","last_became_peered":"2024-09-10T18:45:22.068918+0000","last_unstale":"2024-09-10T18:51:46.611341+0000","last_undegraded":"2024-09-10T18:51:46.611341+0000","last_fullsized":"2024-09-10T18:51:46.611341+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:44.388105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_clean_scrub_stamp":"2024-09-10T18:51:44.388105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:08.162498+0000","scrub_duration":0.0013288989999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":140,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.078846+0000","last_change":"2024-09-10T18:51:41.073374+0000","last_active":"2024-09-10T18:51:47.078846+0000","last_peered":"2024-09-10T18:51:47.078846+0000","last_clean":"2024-09-10T18:51:47.078846+0000","last_became_active":"2024-09-10T18:45:24.264538+0000","last_became_peered":"2024-09-10T18:45:24.264538+0000","last_unstale":"2024-09-10T18:51:47.078846+0000","last_undegraded":"2024-09-10T18:51:47.078846+0000","last_fullsized":"2024-09-10T18:51:47.078846+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:41.073320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:41.073320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:00.507812+0000","scrub_duration":0.001777551,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"163'1354","reported_seq":1367,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:56.063379+0000","last_change":"2024-09-10T18:51:37.911703+0000","last_active":"2024-09-10T18:51:56.063379+0000","last_peered":"2024-09-10T18:51:56.063379+0000","last_clean":"2024-09-10T18:51:56.063379+0000","last_became_active":"2024-09-10T18:51:37.911394+0000","last_became_peered":"2024-09-10T18:51:37.911394+0000","last_unstale":"2024-09-10T18:51:56.063379+0000","last_undegraded":"2024-09-10T18:51:56.063379+0000","last_fullsized":"2024-09-10T18:51:56.063379+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":1354,"log_dups_size":0,"ondisk_log_size":1354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:59.524921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88735744,"num_objects":1354,"num_object_clones":0,"num_object_copies":4062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1354,"num_write_kb":86656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"70'81","reported_seq":58,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.078692+0000","last_change":"2024-09-10T18:50:59.164503+0000","last_active":"2024-09-10T18:51:47.078692+0000","last_peered":"2024-09-10T18:51:47.078692+0000","last_clean":"2024-09-10T18:51:47.078692+0000","last_became_active":"2024-09-10T18:50:08.399048+0000","last_became_peered":"2024-09-10T18:50:08.399048+0000","last_unstale":"2024-09-10T18:51:47.078692+0000","last_undegraded":"2024-09-10T18:51:47.078692+0000","last_fullsized":"2024-09-10T18:51:47.078692+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:50:59.164417+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:50:59.164417+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:19.822414+0000","scrub_duration":0.0063541700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"143'21","reported_seq":170,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-10T18:51:41.591979+0000","last_change":"2024-09-10T18:51:02.540313+0000","last_active":"2024-09-10T18:51:41.591979+0000","last_peered":"2024-09-10T18:51:41.591979+0000","last_clean":"2024-09-10T18:51:41.591979+0000","last_became_active":"2024-09-10T18:45:20.052243+0000","last_became_peered":"2024-09-10T18:45:20.052243+0000","last_unstale":"2024-09-10T18:51:41.591979+0000","last_undegraded":"2024-09-10T18:51:41.591979+0000","last_fullsized":"2024-09-10T18:51:41.591979+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"143'21","last_scrub_stamp":"2024-09-10T18:51:02.540216+0000","last_deep_scrub":"143'21","last_deep_scrub_stamp":"2024-09-10T18:51:02.540216+0000","last_clean_scrub_stamp":"2024-09-10T18:51:02.540216+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:26.124099+0000","scrub_duration":0.0060974410000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":155,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.739630+0000","last_change":"2024-09-10T18:49:59.397785+0000","last_active":"2024-09-10T18:51:47.739630+0000","last_peered":"2024-09-10T18:51:47.739630+0000","last_clean":"2024-09-10T18:51:47.739630+0000","last_became_active":"2024-09-10T18:47:36.424466+0000","last_became_peered":"2024-09-10T18:47:36.424466+0000","last_unstale":"2024-09-10T18:51:47.739630+0000","last_undegraded":"2024-09-10T18:51:47.739630+0000","last_fullsized":"2024-09-10T18:51:47.739630+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:59.397734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_clean_scrub_stamp":"2024-09-10T18:49:59.397734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00244567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":230,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:51.153804+0000","last_change":"2024-09-10T18:50:53.293010+0000","last_active":"2024-09-10T18:51:51.153804+0000","last_peered":"2024-09-10T18:51:51.153804+0000","last_clean":"2024-09-10T18:51:51.153804+0000","last_became_active":"2024-09-10T18:45:22.074540+0000","last_became_peered":"2024-09-10T18:45:22.074540+0000","last_unstale":"2024-09-10T18:51:51.153804+0000","last_undegraded":"2024-09-10T18:51:51.153804+0000","last_fullsized":"2024-09-10T18:51:51.153804+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:50:53.292838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:53.292838+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:07.773584+0000","scrub_duration":0.018341929999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":160,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.212120+0000","last_change":"2024-09-10T18:51:01.463186+0000","last_active":"2024-09-10T18:51:47.212120+0000","last_peered":"2024-09-10T18:51:47.212120+0000","last_clean":"2024-09-10T18:51:47.212120+0000","last_became_active":"2024-09-10T18:47:36.414665+0000","last_became_peered":"2024-09-10T18:47:36.414665+0000","last_unstale":"2024-09-10T18:51:47.212120+0000","last_undegraded":"2024-09-10T18:51:47.212120+0000","last_fullsized":"2024-09-10T18:51:47.212120+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:01.463123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:01.463123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:05.342543+0000","scrub_duration":0.0050178740000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"163'1474","reported_seq":1487,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:56.516513+0000","last_change":"2024-09-10T18:51:37.909451+0000","last_active":"2024-09-10T18:51:56.516513+0000","last_peered":"2024-09-10T18:51:56.516513+0000","last_clean":"2024-09-10T18:51:56.516513+0000","last_became_active":"2024-09-10T18:51:37.909136+0000","last_became_peered":"2024-09-10T18:51:37.909136+0000","last_unstale":"2024-09-10T18:51:56.516513+0000","last_undegraded":"2024-09-10T18:51:56.516513+0000","last_fullsized":"2024-09-10T18:51:56.516513+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":1474,"log_dups_size":0,"ondisk_log_size":1474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:03.096434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96600064,"num_objects":1474,"num_object_clones":0,"num_object_copies":4422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1474,"num_write_kb":94336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"70'81","reported_seq":52,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-10T18:51:41.592131+0000","last_change":"2024-09-10T18:51:01.490009+0000","last_active":"2024-09-10T18:51:41.592131+0000","last_peered":"2024-09-10T18:51:41.592131+0000","last_clean":"2024-09-10T18:51:41.592131+0000","last_became_active":"2024-09-10T18:50:08.398081+0000","last_became_peered":"2024-09-10T18:50:08.398081+0000","last_unstale":"2024-09-10T18:51:41.592131+0000","last_undegraded":"2024-09-10T18:51:41.592131+0000","last_fullsized":"2024-09-10T18:51:41.592131+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:51:01.489919+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:51:01.489919+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:10.385199+0000","scrub_duration":0.0014613810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"63'18","reported_seq":176,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.078573+0000","last_change":"2024-09-10T18:50:44.111437+0000","last_active":"2024-09-10T18:51:47.078573+0000","last_peered":"2024-09-10T18:51:47.078573+0000","last_clean":"2024-09-10T18:51:47.078573+0000","last_became_active":"2024-09-10T18:45:20.420937+0000","last_became_peered":"2024-09-10T18:45:20.420937+0000","last_unstale":"2024-09-10T18:51:47.078573+0000","last_undegraded":"2024-09-10T18:51:47.078573+0000","last_fullsized":"2024-09-10T18:51:47.078573+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'18","last_scrub_stamp":"2024-09-10T18:50:44.111343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:50:44.111343+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:06.271060+0000","scrub_duration":0.013021517999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":146,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:48.048024+0000","last_change":"2024-09-10T18:51:48.047994+0000","last_active":"2024-09-10T18:51:48.048024+0000","last_peered":"2024-09-10T18:51:48.048024+0000","last_clean":"2024-09-10T18:51:48.048024+0000","last_became_active":"2024-09-10T18:45:17.961695+0000","last_became_peered":"2024-09-10T18:45:17.961695+0000","last_unstale":"2024-09-10T18:51:48.048024+0000","last_undegraded":"2024-09-10T18:51:48.048024+0000","last_fullsized":"2024-09-10T18:51:48.048024+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:48.047929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:48.047929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:49.914996+0000","scrub_duration":0.0018242130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":140,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-10T18:51:41.592038+0000","last_change":"2024-09-10T18:50:43.554137+0000","last_active":"2024-09-10T18:51:41.592038+0000","last_peered":"2024-09-10T18:51:41.592038+0000","last_clean":"2024-09-10T18:51:41.592038+0000","last_became_active":"2024-09-10T18:45:22.074012+0000","last_became_peered":"2024-09-10T18:45:22.074012+0000","last_unstale":"2024-09-10T18:51:41.592038+0000","last_undegraded":"2024-09-10T18:51:41.592038+0000","last_fullsized":"2024-09-10T18:51:41.592038+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:43.554063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:43.554063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:07.347038+0000","scrub_duration":0.0065283629999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":148,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:52.332367+0000","last_change":"2024-09-10T18:51:52.332328+0000","last_active":"2024-09-10T18:51:52.332367+0000","last_peered":"2024-09-10T18:51:52.332367+0000","last_clean":"2024-09-10T18:51:52.332367+0000","last_became_active":"2024-09-10T18:47:36.410836+0000","last_became_peered":"2024-09-10T18:47:36.410836+0000","last_unstale":"2024-09-10T18:51:52.332367+0000","last_undegraded":"2024-09-10T18:51:52.332367+0000","last_fullsized":"2024-09-10T18:51:52.332367+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:52.332258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_clean_scrub_stamp":"2024-09-10T18:51:52.332258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:02.920642+0000","scrub_duration":0.0030121940000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"163'1415","reported_seq":1428,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:56.414409+0000","last_change":"2024-09-10T18:51:37.584352+0000","last_active":"2024-09-10T18:51:56.414409+0000","last_peered":"2024-09-10T18:51:56.414409+0000","last_clean":"2024-09-10T18:51:56.414409+0000","last_became_active":"2024-09-10T18:51:37.583787+0000","last_became_peered":"2024-09-10T18:51:37.583787+0000","last_unstale":"2024-09-10T18:51:56.414409+0000","last_undegraded":"2024-09-10T18:51:56.414409+0000","last_fullsized":"2024-09-10T18:51:56.414409+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":1415,"log_dups_size":0,"ondisk_log_size":1415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:38.451494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92733440,"num_objects":1415,"num_object_clones":0,"num_object_copies":4245,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1415,"num_write_kb":90560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"70'81","reported_seq":121,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.739507+0000","last_change":"2024-09-10T18:51:40.561133+0000","last_active":"2024-09-10T18:51:47.739507+0000","last_peered":"2024-09-10T18:51:47.739507+0000","last_clean":"2024-09-10T18:51:47.739507+0000","last_became_active":"2024-09-10T18:50:08.405424+0000","last_became_peered":"2024-09-10T18:50:08.405424+0000","last_unstale":"2024-09-10T18:51:47.739507+0000","last_undegraded":"2024-09-10T18:51:47.739507+0000","last_fullsized":"2024-09-10T18:51:47.739507+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:51:40.561066+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:51:40.561066+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:47.503438+0000","scrub_duration":0.001694397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"63'19","reported_seq":190,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.610517+0000","last_change":"2024-09-10T18:51:41.161251+0000","last_active":"2024-09-10T18:51:46.610517+0000","last_peered":"2024-09-10T18:51:46.610517+0000","last_clean":"2024-09-10T18:51:46.610517+0000","last_became_active":"2024-09-10T18:45:20.421046+0000","last_became_peered":"2024-09-10T18:45:20.421046+0000","last_unstale":"2024-09-10T18:51:46.610517+0000","last_undegraded":"2024-09-10T18:51:46.610517+0000","last_fullsized":"2024-09-10T18:51:46.610517+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:51:41.161196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:41.161196+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:57.195830+0000","scrub_duration":0.0023699070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":156,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:48.196168+0000","last_change":"2024-09-10T18:51:48.196125+0000","last_active":"2024-09-10T18:51:48.196168+0000","last_peered":"2024-09-10T18:51:48.196168+0000","last_clean":"2024-09-10T18:51:48.196168+0000","last_became_active":"2024-09-10T18:45:17.960226+0000","last_became_peered":"2024-09-10T18:45:17.960226+0000","last_unstale":"2024-09-10T18:51:48.196168+0000","last_undegraded":"2024-09-10T18:51:48.196168+0000","last_fullsized":"2024-09-10T18:51:48.196168+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:48.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:48.196065+0000","last_clean_scrub_stamp":"2024-09-10T18:51:48.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:10.250986+0000","scrub_duration":0.0013037999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":151,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:53.548402+0000","last_change":"2024-09-10T18:51:53.548363+0000","last_active":"2024-09-10T18:51:53.548402+0000","last_peered":"2024-09-10T18:51:53.548402+0000","last_clean":"2024-09-10T18:51:53.548402+0000","last_became_active":"2024-09-10T18:45:22.066148+0000","last_became_peered":"2024-09-10T18:45:22.066148+0000","last_unstale":"2024-09-10T18:51:53.548402+0000","last_undegraded":"2024-09-10T18:51:53.548402+0000","last_fullsized":"2024-09-10T18:51:53.548402+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:53.548279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:53.548279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:03.372538+0000","scrub_duration":0.0048693970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":150,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.611495+0000","last_change":"2024-09-10T18:51:43.053775+0000","last_active":"2024-09-10T18:51:46.611495+0000","last_peered":"2024-09-10T18:51:46.611495+0000","last_clean":"2024-09-10T18:51:46.611495+0000","last_became_active":"2024-09-10T18:47:36.422295+0000","last_became_peered":"2024-09-10T18:47:36.422295+0000","last_unstale":"2024-09-10T18:51:46.611495+0000","last_undegraded":"2024-09-10T18:51:46.611495+0000","last_fullsized":"2024-09-10T18:51:46.611495+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:43.053721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:49:18.281230+0000","last_clean_scrub_stamp":"2024-09-10T18:51:43.053721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:10.077998+0000","scrub_duration":0.001882325,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.e","version":"163'1459","reported_seq":1472,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:56.511366+0000","last_change":"2024-09-10T18:51:37.909961+0000","last_active":"2024-09-10T18:51:56.511366+0000","last_peered":"2024-09-10T18:51:56.511366+0000","last_clean":"2024-09-10T18:51:56.511366+0000","last_became_active":"2024-09-10T18:51:37.909734+0000","last_became_peered":"2024-09-10T18:51:37.909734+0000","last_unstale":"2024-09-10T18:51:56.511366+0000","last_undegraded":"2024-09-10T18:51:56.511366+0000","last_fullsized":"2024-09-10T18:51:56.511366+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":1459,"log_dups_size":0,"ondisk_log_size":1459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:50.174570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95617024,"num_objects":1459,"num_object_clones":0,"num_object_copies":4377,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1459,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1459,"num_write_kb":93376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"70'81","reported_seq":121,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.739437+0000","last_change":"2024-09-10T18:51:36.604009+0000","last_active":"2024-09-10T18:51:47.739437+0000","last_peered":"2024-09-10T18:51:47.739437+0000","last_clean":"2024-09-10T18:51:47.739437+0000","last_became_active":"2024-09-10T18:50:08.397123+0000","last_became_peered":"2024-09-10T18:50:08.397123+0000","last_unstale":"2024-09-10T18:51:47.739437+0000","last_undegraded":"2024-09-10T18:51:47.739437+0000","last_fullsized":"2024-09-10T18:51:47.739437+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:51:36.603950+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.603950+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:41.125691+0000","scrub_duration":0.0013973970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"83'15","reported_seq":164,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.611109+0000","last_change":"2024-09-10T18:50:09.557013+0000","last_active":"2024-09-10T18:51:46.611109+0000","last_peered":"2024-09-10T18:51:46.611109+0000","last_clean":"2024-09-10T18:51:46.611109+0000","last_became_active":"2024-09-10T18:45:20.057204+0000","last_became_peered":"2024-09-10T18:45:20.057204+0000","last_unstale":"2024-09-10T18:51:46.611109+0000","last_undegraded":"2024-09-10T18:51:46.611109+0000","last_fullsized":"2024-09-10T18:51:46.611109+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'15","last_scrub_stamp":"2024-09-10T18:50:09.556916+0000","last_deep_scrub":"83'15","last_deep_scrub_stamp":"2024-09-10T18:46:58.436473+0000","last_clean_scrub_stamp":"2024-09-10T18:50:09.556916+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.01222707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":146,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.611418+0000","last_change":"2024-09-10T18:51:05.140273+0000","last_active":"2024-09-10T18:51:46.611418+0000","last_peered":"2024-09-10T18:51:46.611418+0000","last_clean":"2024-09-10T18:51:46.611418+0000","last_became_active":"2024-09-10T18:45:17.966632+0000","last_became_peered":"2024-09-10T18:45:17.966632+0000","last_unstale":"2024-09-10T18:51:46.611418+0000","last_undegraded":"2024-09-10T18:51:46.611418+0000","last_fullsized":"2024-09-10T18:51:46.611418+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:05.140027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:05.140027+0000","last_clean_scrub_stamp":"2024-09-10T18:51:05.140027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:28.134498+0000","scrub_duration":0.0085472780000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"63'8","reported_seq":230,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:51.153775+0000","last_change":"2024-09-10T18:51:37.206285+0000","last_active":"2024-09-10T18:51:51.153775+0000","last_peered":"2024-09-10T18:51:51.153775+0000","last_clean":"2024-09-10T18:51:51.153775+0000","last_became_active":"2024-09-10T18:45:22.063326+0000","last_became_peered":"2024-09-10T18:45:22.063326+0000","last_unstale":"2024-09-10T18:51:51.153775+0000","last_undegraded":"2024-09-10T18:51:51.153775+0000","last_fullsized":"2024-09-10T18:51:51.153775+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:51:37.206224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:37.206224+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:06.464879+0000","scrub_duration":0.001760707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":150,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.609510+0000","last_change":"2024-09-10T18:51:01.618543+0000","last_active":"2024-09-10T18:51:46.609510+0000","last_peered":"2024-09-10T18:51:46.609510+0000","last_clean":"2024-09-10T18:51:46.609510+0000","last_became_active":"2024-09-10T18:47:36.412111+0000","last_became_peered":"2024-09-10T18:47:36.412111+0000","last_unstale":"2024-09-10T18:51:46.609510+0000","last_undegraded":"2024-09-10T18:51:46.609510+0000","last_fullsized":"2024-09-10T18:51:46.609510+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:01.618458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:01.618458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:21.627693+0000","scrub_duration":0.0072066109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"163'1387","reported_seq":1400,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:56.066981+0000","last_change":"2024-09-10T18:51:37.910891+0000","last_active":"2024-09-10T18:51:56.066981+0000","last_peered":"2024-09-10T18:51:56.066981+0000","last_clean":"2024-09-10T18:51:56.066981+0000","last_became_active":"2024-09-10T18:51:37.910355+0000","last_became_peered":"2024-09-10T18:51:37.910355+0000","last_unstale":"2024-09-10T18:51:56.066981+0000","last_undegraded":"2024-09-10T18:51:56.066981+0000","last_fullsized":"2024-09-10T18:51:56.066981+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":1387,"log_dups_size":0,"ondisk_log_size":1387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:41.408841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90898432,"num_objects":1387,"num_object_clones":0,"num_object_copies":4161,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1387,"num_write_kb":88768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"70'81","reported_seq":53,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.610692+0000","last_change":"2024-09-10T18:50:43.117818+0000","last_active":"2024-09-10T18:51:46.610692+0000","last_peered":"2024-09-10T18:51:46.610692+0000","last_clean":"2024-09-10T18:51:46.610692+0000","last_became_active":"2024-09-10T18:50:08.399074+0000","last_became_peered":"2024-09-10T18:50:08.399074+0000","last_unstale":"2024-09-10T18:51:46.610692+0000","last_undegraded":"2024-09-10T18:51:46.610692+0000","last_fullsized":"2024-09-10T18:51:46.610692+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:50:43.117728+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:50:43.117728+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:52.214303+0000","scrub_duration":0.0039521169999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"89'29","reported_seq":304,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:51.153910+0000","last_change":"2024-09-10T18:50:04.462642+0000","last_active":"2024-09-10T18:51:51.153910+0000","last_peered":"2024-09-10T18:51:51.153910+0000","last_clean":"2024-09-10T18:51:51.153910+0000","last_became_active":"2024-09-10T18:47:36.420185+0000","last_became_peered":"2024-09-10T18:47:36.420185+0000","last_unstale":"2024-09-10T18:51:51.153910+0000","last_undegraded":"2024-09-10T18:51:51.153910+0000","last_fullsized":"2024-09-10T18:51:51.153910+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'29","last_scrub_stamp":"2024-09-10T18:50:04.459912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:50:04.459912+0000","objects_scrubbed":10,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0097323310000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":145,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.611277+0000","last_change":"2024-09-10T18:50:12.982700+0000","last_active":"2024-09-10T18:51:46.611277+0000","last_peered":"2024-09-10T18:51:46.611277+0000","last_clean":"2024-09-10T18:51:46.611277+0000","last_became_active":"2024-09-10T18:45:17.964494+0000","last_became_peered":"2024-09-10T18:45:17.964494+0000","last_unstale":"2024-09-10T18:51:46.611277+0000","last_undegraded":"2024-09-10T18:51:46.611277+0000","last_fullsized":"2024-09-10T18:51:46.611277+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:12.982599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:12.982599+0000","last_clean_scrub_stamp":"2024-09-10T18:50:12.982599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048150440000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":96,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.739297+0000","last_change":"2024-09-10T18:51:01.357880+0000","last_active":"2024-09-10T18:51:47.739297+0000","last_peered":"2024-09-10T18:51:47.739297+0000","last_clean":"2024-09-10T18:51:47.739297+0000","last_became_active":"2024-09-10T18:47:57.107478+0000","last_became_peered":"2024-09-10T18:47:57.107478+0000","last_unstale":"2024-09-10T18:51:47.739297+0000","last_undegraded":"2024-09-10T18:51:47.739297+0000","last_fullsized":"2024-09-10T18:51:47.739297+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:01.357830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:01.357830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:28.477016+0000","scrub_duration":0.002118796,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":130,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.079198+0000","last_change":"2024-09-10T18:49:57.442744+0000","last_active":"2024-09-10T18:51:47.079198+0000","last_peered":"2024-09-10T18:51:47.079198+0000","last_clean":"2024-09-10T18:51:47.079198+0000","last_became_active":"2024-09-10T18:45:24.073307+0000","last_became_peered":"2024-09-10T18:45:24.073307+0000","last_unstale":"2024-09-10T18:51:47.079198+0000","last_undegraded":"2024-09-10T18:51:47.079198+0000","last_fullsized":"2024-09-10T18:51:47.079198+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:57.442512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:22.891084+0000","last_clean_scrub_stamp":"2024-09-10T18:49:57.442512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0050025080000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"163'1446","reported_seq":1458,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:56.414168+0000","last_change":"2024-09-10T18:51:37.583541+0000","last_active":"2024-09-10T18:51:56.414168+0000","last_peered":"2024-09-10T18:51:56.414168+0000","last_clean":"2024-09-10T18:51:56.414168+0000","last_became_active":"2024-09-10T18:51:37.583248+0000","last_became_peered":"2024-09-10T18:51:37.583248+0000","last_unstale":"2024-09-10T18:51:56.414168+0000","last_undegraded":"2024-09-10T18:51:56.414168+0000","last_fullsized":"2024-09-10T18:51:56.414168+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":1446,"log_dups_size":0,"ondisk_log_size":1446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:01.429540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":94765056,"num_objects":1446,"num_object_clones":0,"num_object_copies":4338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1446,"num_write_kb":92544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"70'81","reported_seq":210,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.211985+0000","last_change":"2024-09-10T18:50:50.506911+0000","last_active":"2024-09-10T18:51:47.211985+0000","last_peered":"2024-09-10T18:51:47.211985+0000","last_clean":"2024-09-10T18:51:47.211985+0000","last_became_active":"2024-09-10T18:50:08.397665+0000","last_became_peered":"2024-09-10T18:50:08.397665+0000","last_unstale":"2024-09-10T18:51:47.211985+0000","last_undegraded":"2024-09-10T18:51:47.211985+0000","last_fullsized":"2024-09-10T18:51:47.211985+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:50:50.506769+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:50:50.506769+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:53.715025+0000","scrub_duration":0.0043821199999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"84'18","reported_seq":176,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.612357+0000","last_change":"2024-09-10T18:51:37.059537+0000","last_active":"2024-09-10T18:51:46.612357+0000","last_peered":"2024-09-10T18:51:46.612357+0000","last_clean":"2024-09-10T18:51:46.612357+0000","last_became_active":"2024-09-10T18:45:20.058435+0000","last_became_peered":"2024-09-10T18:45:20.058435+0000","last_unstale":"2024-09-10T18:51:46.612357+0000","last_undegraded":"2024-09-10T18:51:46.612357+0000","last_fullsized":"2024-09-10T18:51:46.612357+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'18","last_scrub_stamp":"2024-09-10T18:51:37.059477+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-10T18:47:10.514951+0000","last_clean_scrub_stamp":"2024-09-10T18:51:37.059477+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:06.269219+0000","scrub_duration":0.0020902970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"84'6","reported_seq":558,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:54.997219+0000","last_change":"2024-09-10T18:51:49.096654+0000","last_active":"2024-09-10T18:51:54.997219+0000","last_peered":"2024-09-10T18:51:54.997219+0000","last_clean":"2024-09-10T18:51:54.997219+0000","last_became_active":"2024-09-10T18:47:36.418211+0000","last_became_peered":"2024-09-10T18:47:36.418211+0000","last_unstale":"2024-09-10T18:51:54.997219+0000","last_undegraded":"2024-09-10T18:51:54.997219+0000","last_fullsized":"2024-09-10T18:51:54.997219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'6","last_scrub_stamp":"2024-09-10T18:51:49.096535+0000","last_deep_scrub":"84'6","last_deep_scrub_stamp":"2024-09-10T18:47:22.545450+0000","last_clean_scrub_stamp":"2024-09-10T18:51:49.096535+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:14.910896+0000","scrub_duration":0.0037496439999999999,"objects_trimmed":0,"snaptrim_duration":0.0021933159999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":388,"num_read_kb":383,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":107,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.078694+0000","last_change":"2024-09-10T18:51:44.101630+0000","last_active":"2024-09-10T18:51:47.078694+0000","last_peered":"2024-09-10T18:51:47.078694+0000","last_clean":"2024-09-10T18:51:47.078694+0000","last_became_active":"2024-09-10T18:47:36.422641+0000","last_became_peered":"2024-09-10T18:47:36.422641+0000","last_unstale":"2024-09-10T18:51:47.078694+0000","last_undegraded":"2024-09-10T18:51:47.078694+0000","last_fullsized":"2024-09-10T18:51:47.078694+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:44.101575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:44.101575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:48.501520+0000","scrub_duration":0.0012333190000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":106,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.611174+0000","last_change":"2024-09-10T18:50:00.088364+0000","last_active":"2024-09-10T18:51:46.611174+0000","last_peered":"2024-09-10T18:51:46.611174+0000","last_clean":"2024-09-10T18:51:46.611174+0000","last_became_active":"2024-09-10T18:47:36.419081+0000","last_became_peered":"2024-09-10T18:47:36.419081+0000","last_unstale":"2024-09-10T18:51:46.611174+0000","last_undegraded":"2024-09-10T18:51:46.611174+0000","last_fullsized":"2024-09-10T18:51:46.611174+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:00.088179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:00.088179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.034684810000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":140,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.212390+0000","last_change":"2024-09-10T18:51:35.398902+0000","last_active":"2024-09-10T18:51:47.212390+0000","last_peered":"2024-09-10T18:51:47.212390+0000","last_clean":"2024-09-10T18:51:47.212390+0000","last_became_active":"2024-09-10T18:45:24.264617+0000","last_became_peered":"2024-09-10T18:45:24.264617+0000","last_unstale":"2024-09-10T18:51:47.212390+0000","last_undegraded":"2024-09-10T18:51:47.212390+0000","last_fullsized":"2024-09-10T18:51:47.212390+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:35.398819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:35.398819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:41.839709+0000","scrub_duration":0.0014954090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"163'1470","reported_seq":1483,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:56.207811+0000","last_change":"2024-09-10T18:51:37.589525+0000","last_active":"2024-09-10T18:51:56.207811+0000","last_peered":"2024-09-10T18:51:56.207811+0000","last_clean":"2024-09-10T18:51:56.207811+0000","last_became_active":"2024-09-10T18:51:37.588751+0000","last_became_peered":"2024-09-10T18:51:37.588751+0000","last_unstale":"2024-09-10T18:51:56.207811+0000","last_undegraded":"2024-09-10T18:51:56.207811+0000","last_fullsized":"2024-09-10T18:51:56.207811+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:57.311670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96337920,"num_objects":1470,"num_object_clones":0,"num_object_copies":4410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1470,"num_write_kb":94080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"70'81","reported_seq":53,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.611626+0000","last_change":"2024-09-10T18:51:04.099758+0000","last_active":"2024-09-10T18:51:46.611626+0000","last_peered":"2024-09-10T18:51:46.611626+0000","last_clean":"2024-09-10T18:51:46.611626+0000","last_became_active":"2024-09-10T18:50:08.398205+0000","last_became_peered":"2024-09-10T18:50:08.398205+0000","last_unstale":"2024-09-10T18:51:46.611626+0000","last_undegraded":"2024-09-10T18:51:46.611626+0000","last_fullsized":"2024-09-10T18:51:46.611626+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:51:04.099703+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:51:04.099703+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:26.646801+0000","scrub_duration":0.007621203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"63'12","reported_seq":186,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.611386+0000","last_change":"2024-09-10T18:50:48.742956+0000","last_active":"2024-09-10T18:51:46.611386+0000","last_peered":"2024-09-10T18:51:46.611386+0000","last_clean":"2024-09-10T18:51:46.611386+0000","last_became_active":"2024-09-10T18:47:36.417965+0000","last_became_peered":"2024-09-10T18:47:36.417965+0000","last_unstale":"2024-09-10T18:51:46.611386+0000","last_undegraded":"2024-09-10T18:51:46.611386+0000","last_fullsized":"2024-09-10T18:51:46.611386+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:50:48.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:50:48.742875+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:50.790274+0000","scrub_duration":0.011146239,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"58'1","reported_seq":152,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.610028+0000","last_change":"2024-09-10T18:51:39.175354+0000","last_active":"2024-09-10T18:51:46.610028+0000","last_peered":"2024-09-10T18:51:46.610028+0000","last_clean":"2024-09-10T18:51:46.610028+0000","last_became_active":"2024-09-10T18:45:17.963932+0000","last_became_peered":"2024-09-10T18:45:17.963932+0000","last_unstale":"2024-09-10T18:51:46.610028+0000","last_undegraded":"2024-09-10T18:51:46.610028+0000","last_fullsized":"2024-09-10T18:51:46.610028+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'1","last_scrub_stamp":"2024-09-10T18:51:39.175290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:39.175290+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:03.334161+0000","scrub_duration":0.0015267550000000001,"objects_trimmed":0,"snaptrim_duration":0.00037812099999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":104,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.211699+0000","last_change":"2024-09-10T18:51:30.467050+0000","last_active":"2024-09-10T18:51:47.211699+0000","last_peered":"2024-09-10T18:51:47.211699+0000","last_clean":"2024-09-10T18:51:47.211699+0000","last_became_active":"2024-09-10T18:47:36.416641+0000","last_became_peered":"2024-09-10T18:47:36.416641+0000","last_unstale":"2024-09-10T18:51:47.211699+0000","last_undegraded":"2024-09-10T18:51:47.211699+0000","last_fullsized":"2024-09-10T18:51:47.211699+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:30.466988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:30.466988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:44.004240+0000","scrub_duration":0.0019674089999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":141,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.611287+0000","last_change":"2024-09-10T18:50:52.999936+0000","last_active":"2024-09-10T18:51:46.611287+0000","last_peered":"2024-09-10T18:51:46.611287+0000","last_clean":"2024-09-10T18:51:46.611287+0000","last_became_active":"2024-09-10T18:45:22.064285+0000","last_became_peered":"2024-09-10T18:45:22.064285+0000","last_unstale":"2024-09-10T18:51:46.611287+0000","last_undegraded":"2024-09-10T18:51:46.611287+0000","last_fullsized":"2024-09-10T18:51:46.611287+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:52.999842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:52.999842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:02.826146+0000","scrub_duration":0.0065214100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":140,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.611102+0000","last_change":"2024-09-10T18:50:53.940855+0000","last_active":"2024-09-10T18:51:46.611102+0000","last_peered":"2024-09-10T18:51:46.611102+0000","last_clean":"2024-09-10T18:51:46.611102+0000","last_became_active":"2024-09-10T18:45:24.083689+0000","last_became_peered":"2024-09-10T18:45:24.083689+0000","last_unstale":"2024-09-10T18:51:46.611102+0000","last_undegraded":"2024-09-10T18:51:46.611102+0000","last_fullsized":"2024-09-10T18:51:46.611102+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:53.940489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:50:53.940489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:21.653605+0000","scrub_duration":0.007171691,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"163'1361","reported_seq":1374,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:56.128198+0000","last_change":"2024-09-10T18:51:37.591719+0000","last_active":"2024-09-10T18:51:56.128198+0000","last_peered":"2024-09-10T18:51:56.128198+0000","last_clean":"2024-09-10T18:51:56.128198+0000","last_became_active":"2024-09-10T18:51:37.591441+0000","last_became_peered":"2024-09-10T18:51:37.591441+0000","last_unstale":"2024-09-10T18:51:56.128198+0000","last_undegraded":"2024-09-10T18:51:56.128198+0000","last_fullsized":"2024-09-10T18:51:56.128198+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":1361,"log_dups_size":0,"ondisk_log_size":1361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:43.944535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":89194496,"num_objects":1361,"num_object_clones":0,"num_object_copies":4083,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1361,"num_write_kb":87104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"126'21","reported_seq":182,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:52.576604+0000","last_change":"2024-09-10T18:51:52.576493+0000","last_active":"2024-09-10T18:51:52.576604+0000","last_peered":"2024-09-10T18:51:52.576604+0000","last_clean":"2024-09-10T18:51:52.576604+0000","last_became_active":"2024-09-10T18:45:20.421119+0000","last_became_peered":"2024-09-10T18:45:20.421119+0000","last_unstale":"2024-09-10T18:51:52.576604+0000","last_undegraded":"2024-09-10T18:51:52.576604+0000","last_fullsized":"2024-09-10T18:51:52.576604+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"126'21","last_scrub_stamp":"2024-09-10T18:51:52.576358+0000","last_deep_scrub":"126'21","last_deep_scrub_stamp":"2024-09-10T18:50:03.380902+0000","last_clean_scrub_stamp":"2024-09-10T18:51:52.576358+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:02.684753+0000","scrub_duration":0.0054386800000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"60'2","reported_seq":153,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.079076+0000","last_change":"2024-09-10T18:51:03.104641+0000","last_active":"2024-09-10T18:51:47.079076+0000","last_peered":"2024-09-10T18:51:47.079076+0000","last_clean":"2024-09-10T18:51:47.079076+0000","last_became_active":"2024-09-10T18:45:17.961660+0000","last_became_peered":"2024-09-10T18:45:17.961660+0000","last_unstale":"2024-09-10T18:51:47.079076+0000","last_undegraded":"2024-09-10T18:51:47.079076+0000","last_fullsized":"2024-09-10T18:51:47.079076+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-10T18:51:03.104545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:03.104545+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:16.497102+0000","scrub_duration":0.0070342290000000003,"objects_trimmed":0,"snaptrim_duration":0.0013942640000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":155,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.611052+0000","last_change":"2024-09-10T18:50:14.032609+0000","last_active":"2024-09-10T18:51:46.611052+0000","last_peered":"2024-09-10T18:51:46.611052+0000","last_clean":"2024-09-10T18:51:46.611052+0000","last_became_active":"2024-09-10T18:47:36.414636+0000","last_became_peered":"2024-09-10T18:47:36.414636+0000","last_unstale":"2024-09-10T18:51:46.611052+0000","last_undegraded":"2024-09-10T18:51:46.611052+0000","last_fullsized":"2024-09-10T18:51:46.611052+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:14.032522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:14.032522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.022590255,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'81","reported_seq":201,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.212083+0000","last_change":"2024-09-10T18:50:48.508352+0000","last_active":"2024-09-10T18:51:47.212083+0000","last_peered":"2024-09-10T18:51:47.212083+0000","last_clean":"2024-09-10T18:51:47.212083+0000","last_became_active":"2024-09-10T18:50:06.386717+0000","last_became_peered":"2024-09-10T18:50:06.386717+0000","last_unstale":"2024-09-10T18:51:47.212083+0000","last_undegraded":"2024-09-10T18:51:47.212083+0000","last_fullsized":"2024-09-10T18:51:47.212083+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:50:48.508244+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:50:48.508244+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:53.803446+0000","scrub_duration":0.0035856120000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":162,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.212300+0000","last_change":"2024-09-10T18:51:40.327714+0000","last_active":"2024-09-10T18:51:47.212300+0000","last_peered":"2024-09-10T18:51:47.212300+0000","last_clean":"2024-09-10T18:51:47.212300+0000","last_became_active":"2024-09-10T18:47:36.418320+0000","last_became_peered":"2024-09-10T18:47:36.418320+0000","last_unstale":"2024-09-10T18:51:47.212300+0000","last_undegraded":"2024-09-10T18:51:47.212300+0000","last_fullsized":"2024-09-10T18:51:47.212300+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:40.327645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:40.327645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:58.610158+0000","scrub_duration":0.001391827,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":139,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.611406+0000","last_change":"2024-09-10T18:51:02.170258+0000","last_active":"2024-09-10T18:51:46.611406+0000","last_peered":"2024-09-10T18:51:46.611406+0000","last_clean":"2024-09-10T18:51:46.611406+0000","last_became_active":"2024-09-10T18:45:24.073327+0000","last_became_peered":"2024-09-10T18:45:24.073327+0000","last_unstale":"2024-09-10T18:51:46.611406+0000","last_undegraded":"2024-09-10T18:51:46.611406+0000","last_fullsized":"2024-09-10T18:51:46.611406+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:02.170065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:02.170065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:14.436899+0000","scrub_duration":0.0061532089999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"163'1487","reported_seq":1500,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:56.482736+0000","last_change":"2024-09-10T18:51:37.909963+0000","last_active":"2024-09-10T18:51:56.482736+0000","last_peered":"2024-09-10T18:51:56.482736+0000","last_clean":"2024-09-10T18:51:56.482736+0000","last_became_active":"2024-09-10T18:51:37.909738+0000","last_became_peered":"2024-09-10T18:51:37.909738+0000","last_unstale":"2024-09-10T18:51:56.482736+0000","last_undegraded":"2024-09-10T18:51:56.482736+0000","last_fullsized":"2024-09-10T18:51:56.482736+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":1487,"log_dups_size":0,"ondisk_log_size":1487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:03.481643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":97452032,"num_objects":1487,"num_object_clones":0,"num_object_copies":4461,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1487,"num_write_kb":95168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"63'19","reported_seq":174,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.738939+0000","last_change":"2024-09-10T18:50:01.460246+0000","last_active":"2024-09-10T18:51:47.738939+0000","last_peered":"2024-09-10T18:51:47.738939+0000","last_clean":"2024-09-10T18:51:47.738939+0000","last_became_active":"2024-09-10T18:45:20.421097+0000","last_became_peered":"2024-09-10T18:45:20.421097+0000","last_unstale":"2024-09-10T18:51:47.738939+0000","last_undegraded":"2024-09-10T18:51:47.738939+0000","last_fullsized":"2024-09-10T18:51:47.738939+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:50:01.460159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:50:01.460159+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.019465403999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":106,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.611162+0000","last_change":"2024-09-10T18:50:47.083728+0000","last_active":"2024-09-10T18:51:46.611162+0000","last_peered":"2024-09-10T18:51:46.611162+0000","last_clean":"2024-09-10T18:51:46.611162+0000","last_became_active":"2024-09-10T18:47:36.417800+0000","last_became_peered":"2024-09-10T18:47:36.417800+0000","last_unstale":"2024-09-10T18:51:46.611162+0000","last_undegraded":"2024-09-10T18:51:46.611162+0000","last_fullsized":"2024-09-10T18:51:46.611162+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:47.083643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:47.083643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:16.602999+0000","scrub_duration":0.0071207509999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":117,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.610306+0000","last_change":"2024-09-10T18:51:40.126736+0000","last_active":"2024-09-10T18:51:46.610306+0000","last_peered":"2024-09-10T18:51:46.610306+0000","last_clean":"2024-09-10T18:51:46.610306+0000","last_became_active":"2024-09-10T18:47:36.413618+0000","last_became_peered":"2024-09-10T18:47:36.413618+0000","last_unstale":"2024-09-10T18:51:46.610306+0000","last_undegraded":"2024-09-10T18:51:46.610306+0000","last_fullsized":"2024-09-10T18:51:46.610306+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:40.126652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:40.126652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:44.820089+0000","scrub_duration":0.0017754699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":150,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.211922+0000","last_change":"2024-09-10T18:51:38.318952+0000","last_active":"2024-09-10T18:51:47.211922+0000","last_peered":"2024-09-10T18:51:47.211922+0000","last_clean":"2024-09-10T18:51:47.211922+0000","last_became_active":"2024-09-10T18:47:36.412610+0000","last_became_peered":"2024-09-10T18:47:36.412610+0000","last_unstale":"2024-09-10T18:51:47.211922+0000","last_undegraded":"2024-09-10T18:51:47.211922+0000","last_fullsized":"2024-09-10T18:51:47.211922+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:38.318887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:38.318887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:47.048859+0000","scrub_duration":0.001335461,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.7","version":"163'1362","reported_seq":1375,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:56.229220+0000","last_change":"2024-09-10T18:51:37.590054+0000","last_active":"2024-09-10T18:51:56.229220+0000","last_peered":"2024-09-10T18:51:56.229220+0000","last_clean":"2024-09-10T18:51:56.229220+0000","last_became_active":"2024-09-10T18:51:37.589351+0000","last_became_peered":"2024-09-10T18:51:37.589351+0000","last_unstale":"2024-09-10T18:51:56.229220+0000","last_undegraded":"2024-09-10T18:51:56.229220+0000","last_fullsized":"2024-09-10T18:51:56.229220+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:52.988787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":89260032,"num_objects":1362,"num_object_clones":0,"num_object_copies":4086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1362,"num_write_kb":87168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"130'13","reported_seq":164,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.610448+0000","last_change":"2024-09-10T18:50:08.785524+0000","last_active":"2024-09-10T18:51:46.610448+0000","last_peered":"2024-09-10T18:51:46.610448+0000","last_clean":"2024-09-10T18:51:46.610448+0000","last_became_active":"2024-09-10T18:45:20.061021+0000","last_became_peered":"2024-09-10T18:45:20.061021+0000","last_unstale":"2024-09-10T18:51:46.610448+0000","last_undegraded":"2024-09-10T18:51:46.610448+0000","last_fullsized":"2024-09-10T18:51:46.610448+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"130'13","last_scrub_stamp":"2024-09-10T18:50:08.785434+0000","last_deep_scrub":"130'13","last_deep_scrub_stamp":"2024-09-10T18:50:08.785434+0000","last_clean_scrub_stamp":"2024-09-10T18:50:08.785434+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016531225,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":146,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.611914+0000","last_change":"2024-09-10T18:51:31.128571+0000","last_active":"2024-09-10T18:51:46.611914+0000","last_peered":"2024-09-10T18:51:46.611914+0000","last_clean":"2024-09-10T18:51:46.611914+0000","last_became_active":"2024-09-10T18:45:17.963057+0000","last_became_peered":"2024-09-10T18:45:17.963057+0000","last_unstale":"2024-09-10T18:51:46.611914+0000","last_undegraded":"2024-09-10T18:51:46.611914+0000","last_fullsized":"2024-09-10T18:51:46.611914+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:31.128514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:31.128514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:55.374112+0000","scrub_duration":0.0016209180000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":105,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-10T18:51:41.592017+0000","last_change":"2024-09-10T18:51:35.325521+0000","last_active":"2024-09-10T18:51:41.592017+0000","last_peered":"2024-09-10T18:51:41.592017+0000","last_clean":"2024-09-10T18:51:41.592017+0000","last_became_active":"2024-09-10T18:47:36.411517+0000","last_became_peered":"2024-09-10T18:47:36.411517+0000","last_unstale":"2024-09-10T18:51:41.592017+0000","last_undegraded":"2024-09-10T18:51:41.592017+0000","last_fullsized":"2024-09-10T18:51:41.592017+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:35.325451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_clean_scrub_stamp":"2024-09-10T18:51:35.325451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:56.285365+0000","scrub_duration":0.0018247820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":160,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:53.345309+0000","last_change":"2024-09-10T18:51:53.345267+0000","last_active":"2024-09-10T18:51:53.345309+0000","last_peered":"2024-09-10T18:51:53.345309+0000","last_clean":"2024-09-10T18:51:53.345309+0000","last_became_active":"2024-09-10T18:47:36.412933+0000","last_became_peered":"2024-09-10T18:47:36.412933+0000","last_unstale":"2024-09-10T18:51:53.345309+0000","last_undegraded":"2024-09-10T18:51:53.345309+0000","last_fullsized":"2024-09-10T18:51:53.345309+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:53.345184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:53.345184+0000","last_clean_scrub_stamp":"2024-09-10T18:51:53.345184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:19.403333+0000","scrub_duration":0.003832687,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"163'1418","reported_seq":1431,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:56.074036+0000","last_change":"2024-09-10T18:51:37.590489+0000","last_active":"2024-09-10T18:51:56.074036+0000","last_peered":"2024-09-10T18:51:56.074036+0000","last_clean":"2024-09-10T18:51:56.074036+0000","last_became_active":"2024-09-10T18:51:37.590189+0000","last_became_peered":"2024-09-10T18:51:37.590189+0000","last_unstale":"2024-09-10T18:51:56.074036+0000","last_undegraded":"2024-09-10T18:51:56.074036+0000","last_fullsized":"2024-09-10T18:51:56.074036+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":1418,"log_dups_size":0,"ondisk_log_size":1418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:47.653862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92930048,"num_objects":1418,"num_object_clones":0,"num_object_copies":4254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1418,"num_write_kb":90752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"63'15","reported_seq":168,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.611204+0000","last_change":"2024-09-10T18:51:36.277582+0000","last_active":"2024-09-10T18:51:46.611204+0000","last_peered":"2024-09-10T18:51:46.611204+0000","last_clean":"2024-09-10T18:51:46.611204+0000","last_became_active":"2024-09-10T18:45:20.419804+0000","last_became_peered":"2024-09-10T18:45:20.419804+0000","last_unstale":"2024-09-10T18:51:46.611204+0000","last_undegraded":"2024-09-10T18:51:46.611204+0000","last_fullsized":"2024-09-10T18:51:46.611204+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:51:36.277517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.277517+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:47.932042+0000","scrub_duration":0.0020736750000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":97,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.079385+0000","last_change":"2024-09-10T18:50:05.603208+0000","last_active":"2024-09-10T18:51:47.079385+0000","last_peered":"2024-09-10T18:51:47.079385+0000","last_clean":"2024-09-10T18:51:47.079385+0000","last_became_active":"2024-09-10T18:47:36.420475+0000","last_became_peered":"2024-09-10T18:47:36.420475+0000","last_unstale":"2024-09-10T18:51:47.079385+0000","last_undegraded":"2024-09-10T18:51:47.079385+0000","last_fullsized":"2024-09-10T18:51:47.079385+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:05.603125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:05.603125+0000","last_clean_scrub_stamp":"2024-09-10T18:50:05.603125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048485560000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"83'9","reported_seq":233,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.896592+0000","last_change":"2024-09-10T18:50:10.444294+0000","last_active":"2024-09-10T18:51:47.896592+0000","last_peered":"2024-09-10T18:51:47.896592+0000","last_clean":"2024-09-10T18:51:47.896592+0000","last_became_active":"2024-09-10T18:45:22.073383+0000","last_became_peered":"2024-09-10T18:45:22.073383+0000","last_unstale":"2024-09-10T18:51:47.896592+0000","last_undegraded":"2024-09-10T18:51:47.896592+0000","last_fullsized":"2024-09-10T18:51:47.896592+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'9","last_scrub_stamp":"2024-09-10T18:50:10.444201+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-09-10T18:47:02.582229+0000","last_clean_scrub_stamp":"2024-09-10T18:50:10.444201+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014241271,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":139,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.611294+0000","last_change":"2024-09-10T18:51:05.187154+0000","last_active":"2024-09-10T18:51:46.611294+0000","last_peered":"2024-09-10T18:51:46.611294+0000","last_clean":"2024-09-10T18:51:46.611294+0000","last_became_active":"2024-09-10T18:45:24.265256+0000","last_became_peered":"2024-09-10T18:45:24.265256+0000","last_unstale":"2024-09-10T18:51:46.611294+0000","last_undegraded":"2024-09-10T18:51:46.611294+0000","last_fullsized":"2024-09-10T18:51:46.611294+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:05.187073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_clean_scrub_stamp":"2024-09-10T18:51:05.187073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:11.730150+0000","scrub_duration":0.00225728,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.5","version":"163'1379","reported_seq":1392,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:56.121036+0000","last_change":"2024-09-10T18:51:37.910484+0000","last_active":"2024-09-10T18:51:56.121036+0000","last_peered":"2024-09-10T18:51:56.121036+0000","last_clean":"2024-09-10T18:51:56.121036+0000","last_became_active":"2024-09-10T18:51:37.910181+0000","last_became_peered":"2024-09-10T18:51:37.910181+0000","last_unstale":"2024-09-10T18:51:56.121036+0000","last_undegraded":"2024-09-10T18:51:56.121036+0000","last_fullsized":"2024-09-10T18:51:56.121036+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":1379,"log_dups_size":0,"ondisk_log_size":1379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:43.239258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90374144,"num_objects":1379,"num_object_clones":0,"num_object_copies":4137,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1379,"num_write_kb":88256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"63'12","reported_seq":165,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-10T18:51:41.591887+0000","last_change":"2024-09-10T18:50:12.595966+0000","last_active":"2024-09-10T18:51:41.591887+0000","last_peered":"2024-09-10T18:51:41.591887+0000","last_clean":"2024-09-10T18:51:41.591887+0000","last_became_active":"2024-09-10T18:45:20.056869+0000","last_became_peered":"2024-09-10T18:45:20.056869+0000","last_unstale":"2024-09-10T18:51:41.591887+0000","last_undegraded":"2024-09-10T18:51:41.591887+0000","last_fullsized":"2024-09-10T18:51:41.591887+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:50:12.595881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:50:12.595881+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013834673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"56'2","reported_seq":161,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-10T18:51:42.425115+0000","last_change":"2024-09-10T18:51:42.425065+0000","last_active":"2024-09-10T18:51:42.425115+0000","last_peered":"2024-09-10T18:51:42.425115+0000","last_clean":"2024-09-10T18:51:42.425115+0000","last_became_active":"2024-09-10T18:45:17.963286+0000","last_became_peered":"2024-09-10T18:45:17.963286+0000","last_unstale":"2024-09-10T18:51:42.425115+0000","last_undegraded":"2024-09-10T18:51:42.425115+0000","last_fullsized":"2024-09-10T18:51:42.425115+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-10T18:51:42.424994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:42.424994+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:10.091197+0000","scrub_duration":0.001852226,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":117,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:54.343076+0000","last_change":"2024-09-10T18:51:54.343030+0000","last_active":"2024-09-10T18:51:54.343076+0000","last_peered":"2024-09-10T18:51:54.343076+0000","last_clean":"2024-09-10T18:51:54.343076+0000","last_became_active":"2024-09-10T18:47:36.421151+0000","last_became_peered":"2024-09-10T18:47:36.421151+0000","last_unstale":"2024-09-10T18:51:54.343076+0000","last_undegraded":"2024-09-10T18:51:54.343076+0000","last_fullsized":"2024-09-10T18:51:54.343076+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:54.342934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:02.711594+0000","last_clean_scrub_stamp":"2024-09-10T18:51:54.342934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:23.514769+0000","scrub_duration":0.005738529,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":140,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:48.065476+0000","last_change":"2024-09-10T18:51:48.065441+0000","last_active":"2024-09-10T18:51:48.065476+0000","last_peered":"2024-09-10T18:51:48.065476+0000","last_clean":"2024-09-10T18:51:48.065476+0000","last_became_active":"2024-09-10T18:45:24.077815+0000","last_became_peered":"2024-09-10T18:45:24.077815+0000","last_unstale":"2024-09-10T18:51:48.065476+0000","last_undegraded":"2024-09-10T18:51:48.065476+0000","last_fullsized":"2024-09-10T18:51:48.065476+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:48.065385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:48.065385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:04.487946+0000","scrub_duration":0.0014615870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"76'13","reported_seq":174,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:56.389612+0000","last_change":"2024-09-10T18:51:56.389512+0000","last_active":"2024-09-10T18:51:56.389612+0000","last_peered":"2024-09-10T18:51:56.389612+0000","last_clean":"2024-09-10T18:51:56.389612+0000","last_became_active":"2024-09-10T18:45:20.420421+0000","last_became_peered":"2024-09-10T18:45:20.420421+0000","last_unstale":"2024-09-10T18:51:56.389612+0000","last_undegraded":"2024-09-10T18:51:56.389612+0000","last_fullsized":"2024-09-10T18:51:56.389612+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'13","last_scrub_stamp":"2024-09-10T18:51:56.389281+0000","last_deep_scrub":"76'13","last_deep_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_clean_scrub_stamp":"2024-09-10T18:51:56.389281+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:59.529351+0000","scrub_duration":0.0031275529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":107,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.610855+0000","last_change":"2024-09-10T18:50:07.454468+0000","last_active":"2024-09-10T18:51:46.610855+0000","last_peered":"2024-09-10T18:51:46.610855+0000","last_clean":"2024-09-10T18:51:46.610855+0000","last_became_active":"2024-09-10T18:47:36.409299+0000","last_became_peered":"2024-09-10T18:47:36.409299+0000","last_unstale":"2024-09-10T18:51:46.610855+0000","last_undegraded":"2024-09-10T18:51:46.610855+0000","last_fullsized":"2024-09-10T18:51:46.610855+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:07.454393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:07.454393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014100293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":142,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.079431+0000","last_change":"2024-09-10T18:50:57.190078+0000","last_active":"2024-09-10T18:51:47.079431+0000","last_peered":"2024-09-10T18:51:47.079431+0000","last_clean":"2024-09-10T18:51:47.079431+0000","last_became_active":"2024-09-10T18:45:22.069040+0000","last_became_peered":"2024-09-10T18:45:22.069040+0000","last_unstale":"2024-09-10T18:51:47.079431+0000","last_undegraded":"2024-09-10T18:51:47.079431+0000","last_fullsized":"2024-09-10T18:51:47.079431+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:57.189976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:28.442370+0000","last_clean_scrub_stamp":"2024-09-10T18:50:57.189976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:02.018261+0000","scrub_duration":0.016295923,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"63'1","reported_seq":111,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-10T18:51:41.592129+0000","last_change":"2024-09-10T18:49:49.544530+0000","last_active":"2024-09-10T18:51:41.592129+0000","last_peered":"2024-09-10T18:51:41.592129+0000","last_clean":"2024-09-10T18:51:41.592129+0000","last_became_active":"2024-09-10T18:47:36.406936+0000","last_became_peered":"2024-09-10T18:47:36.406936+0000","last_unstale":"2024-09-10T18:51:41.592129+0000","last_undegraded":"2024-09-10T18:51:41.592129+0000","last_fullsized":"2024-09-10T18:51:41.592129+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-10T18:49:49.544433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:49.544433+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041109190000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":145,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.739040+0000","last_change":"2024-09-10T18:50:52.260051+0000","last_active":"2024-09-10T18:51:47.739040+0000","last_peered":"2024-09-10T18:51:47.739040+0000","last_clean":"2024-09-10T18:51:47.739040+0000","last_became_active":"2024-09-10T18:45:17.964705+0000","last_became_peered":"2024-09-10T18:45:17.964705+0000","last_unstale":"2024-09-10T18:51:47.739040+0000","last_undegraded":"2024-09-10T18:51:47.739040+0000","last_fullsized":"2024-09-10T18:51:47.739040+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:52.259998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:52.259998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:58.126795+0000","scrub_duration":0.0032069059999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'6","reported_seq":153,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.079529+0000","last_change":"2024-09-10T18:51:43.085607+0000","last_active":"2024-09-10T18:51:47.079529+0000","last_peered":"2024-09-10T18:51:47.079529+0000","last_clean":"2024-09-10T18:51:47.079529+0000","last_became_active":"2024-09-10T18:45:20.421746+0000","last_became_peered":"2024-09-10T18:45:20.421746+0000","last_unstale":"2024-09-10T18:51:47.079529+0000","last_undegraded":"2024-09-10T18:51:47.079529+0000","last_fullsized":"2024-09-10T18:51:47.079529+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'6","last_scrub_stamp":"2024-09-10T18:51:43.085549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:43.085549+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:44.636819+0000","scrub_duration":0.002071238,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":141,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.611288+0000","last_change":"2024-09-10T18:51:32.233743+0000","last_active":"2024-09-10T18:51:46.611288+0000","last_peered":"2024-09-10T18:51:46.611288+0000","last_clean":"2024-09-10T18:51:46.611288+0000","last_became_active":"2024-09-10T18:45:22.069653+0000","last_became_peered":"2024-09-10T18:45:22.069653+0000","last_unstale":"2024-09-10T18:51:46.611288+0000","last_undegraded":"2024-09-10T18:51:46.611288+0000","last_fullsized":"2024-09-10T18:51:46.611288+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:32.233676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:32.233676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:33.989510+0000","scrub_duration":0.0016772180000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":140,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.212267+0000","last_change":"2024-09-10T18:50:43.408152+0000","last_active":"2024-09-10T18:51:47.212267+0000","last_peered":"2024-09-10T18:51:47.212267+0000","last_clean":"2024-09-10T18:51:47.212267+0000","last_became_active":"2024-09-10T18:45:24.264611+0000","last_became_peered":"2024-09-10T18:45:24.264611+0000","last_unstale":"2024-09-10T18:51:47.212267+0000","last_undegraded":"2024-09-10T18:51:47.212267+0000","last_fullsized":"2024-09-10T18:51:47.212267+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:43.408011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:50:43.408011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:11.539468+0000","scrub_duration":0.0026253959999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"63'10","reported_seq":203,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.079397+0000","last_change":"2024-09-10T18:51:39.074731+0000","last_active":"2024-09-10T18:51:47.079397+0000","last_peered":"2024-09-10T18:51:47.079397+0000","last_clean":"2024-09-10T18:51:47.079397+0000","last_became_active":"2024-09-10T18:47:36.426260+0000","last_became_peered":"2024-09-10T18:47:36.426260+0000","last_unstale":"2024-09-10T18:51:47.079397+0000","last_undegraded":"2024-09-10T18:51:47.079397+0000","last_fullsized":"2024-09-10T18:51:47.079397+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:51:39.074664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:39.074664+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:05.841922+0000","scrub_duration":0.0044748410000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":145,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.611169+0000","last_change":"2024-09-10T18:50:45.120724+0000","last_active":"2024-09-10T18:51:46.611169+0000","last_peered":"2024-09-10T18:51:46.611169+0000","last_clean":"2024-09-10T18:51:46.611169+0000","last_became_active":"2024-09-10T18:45:17.964870+0000","last_became_peered":"2024-09-10T18:45:17.964870+0000","last_unstale":"2024-09-10T18:51:46.611169+0000","last_undegraded":"2024-09-10T18:51:46.611169+0000","last_fullsized":"2024-09-10T18:51:46.611169+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:45.120550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:45.120550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:00.695839+0000","scrub_duration":0.0055063930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"63'8","reported_seq":228,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.898323+0000","last_change":"2024-09-10T18:50:52.043047+0000","last_active":"2024-09-10T18:51:47.898323+0000","last_peered":"2024-09-10T18:51:47.898323+0000","last_clean":"2024-09-10T18:51:47.898323+0000","last_became_active":"2024-09-10T18:45:22.062940+0000","last_became_peered":"2024-09-10T18:45:22.062940+0000","last_unstale":"2024-09-10T18:51:47.898323+0000","last_undegraded":"2024-09-10T18:51:47.898323+0000","last_fullsized":"2024-09-10T18:51:47.898323+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:50:52.042960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:52.042960+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:11.293055+0000","scrub_duration":0.0090181189999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":140,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.611035+0000","last_change":"2024-09-10T18:50:56.767501+0000","last_active":"2024-09-10T18:51:46.611035+0000","last_peered":"2024-09-10T18:51:46.611035+0000","last_clean":"2024-09-10T18:51:46.611035+0000","last_became_active":"2024-09-10T18:45:24.073736+0000","last_became_peered":"2024-09-10T18:45:24.073736+0000","last_unstale":"2024-09-10T18:51:46.611035+0000","last_undegraded":"2024-09-10T18:51:46.611035+0000","last_fullsized":"2024-09-10T18:51:46.611035+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:56.767430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:50:56.767430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:18.035411+0000","scrub_duration":0.021971303000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"63'9","reported_seq":169,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.611522+0000","last_change":"2024-09-10T18:51:45.423930+0000","last_active":"2024-09-10T18:51:46.611522+0000","last_peered":"2024-09-10T18:51:46.611522+0000","last_clean":"2024-09-10T18:51:46.611522+0000","last_became_active":"2024-09-10T18:45:20.421124+0000","last_became_peered":"2024-09-10T18:45:20.421124+0000","last_unstale":"2024-09-10T18:51:46.611522+0000","last_undegraded":"2024-09-10T18:51:46.611522+0000","last_fullsized":"2024-09-10T18:51:46.611522+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:51:45.423873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:45.423873+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:55.984623+0000","scrub_duration":0.001903037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":156,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.610617+0000","last_change":"2024-09-10T18:51:46.194282+0000","last_active":"2024-09-10T18:51:46.610617+0000","last_peered":"2024-09-10T18:51:46.610617+0000","last_clean":"2024-09-10T18:51:46.610617+0000","last_became_active":"2024-09-10T18:45:17.960106+0000","last_became_peered":"2024-09-10T18:45:17.960106+0000","last_unstale":"2024-09-10T18:51:46.610617+0000","last_undegraded":"2024-09-10T18:51:46.610617+0000","last_fullsized":"2024-09-10T18:51:46.610617+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:46.194176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:46.194176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:04.442996+0000","scrub_duration":0.001545888,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"89'10","reported_seq":245,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:51.153590+0000","last_change":"2024-09-10T18:51:46.054778+0000","last_active":"2024-09-10T18:51:51.153590+0000","last_peered":"2024-09-10T18:51:51.153590+0000","last_clean":"2024-09-10T18:51:51.153590+0000","last_became_active":"2024-09-10T18:47:36.424575+0000","last_became_peered":"2024-09-10T18:47:36.424575+0000","last_unstale":"2024-09-10T18:51:51.153590+0000","last_undegraded":"2024-09-10T18:51:51.153590+0000","last_fullsized":"2024-09-10T18:51:51.153590+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'10","last_scrub_stamp":"2024-09-10T18:51:46.054720+0000","last_deep_scrub":"81'9","last_deep_scrub_stamp":"2024-09-10T18:46:54.800790+0000","last_clean_scrub_stamp":"2024-09-10T18:51:46.054720+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:07.645431+0000","scrub_duration":0.001410667,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":148,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-10T18:51:41.592554+0000","last_change":"2024-09-10T18:50:55.384653+0000","last_active":"2024-09-10T18:51:41.592554+0000","last_peered":"2024-09-10T18:51:41.592554+0000","last_clean":"2024-09-10T18:51:41.592554+0000","last_became_active":"2024-09-10T18:47:36.414047+0000","last_became_peered":"2024-09-10T18:47:36.414047+0000","last_unstale":"2024-09-10T18:51:41.592554+0000","last_undegraded":"2024-09-10T18:51:41.592554+0000","last_fullsized":"2024-09-10T18:51:41.592554+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:55.384570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_clean_scrub_stamp":"2024-09-10T18:50:55.384570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:06.404917+0000","scrub_duration":0.0079403359999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"63'9","reported_seq":180,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.611024+0000","last_change":"2024-09-10T18:51:32.146681+0000","last_active":"2024-09-10T18:51:46.611024+0000","last_peered":"2024-09-10T18:51:46.611024+0000","last_clean":"2024-09-10T18:51:46.611024+0000","last_became_active":"2024-09-10T18:47:36.416484+0000","last_became_peered":"2024-09-10T18:47:36.416484+0000","last_unstale":"2024-09-10T18:51:46.611024+0000","last_undegraded":"2024-09-10T18:51:46.611024+0000","last_fullsized":"2024-09-10T18:51:46.611024+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:51:32.146622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:32.146622+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:53.233486+0000","scrub_duration":0.0023792729999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":155,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.739276+0000","last_change":"2024-09-10T18:51:44.486612+0000","last_active":"2024-09-10T18:51:47.739276+0000","last_peered":"2024-09-10T18:51:47.739276+0000","last_clean":"2024-09-10T18:51:47.739276+0000","last_became_active":"2024-09-10T18:45:17.962009+0000","last_became_peered":"2024-09-10T18:45:17.962009+0000","last_unstale":"2024-09-10T18:51:47.739276+0000","last_undegraded":"2024-09-10T18:51:47.739276+0000","last_fullsized":"2024-09-10T18:51:47.739276+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:44.486550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:44.486550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:57.019047+0000","scrub_duration":0.0013873609999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":142,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.079289+0000","last_change":"2024-09-10T18:51:47.078689+0000","last_active":"2024-09-10T18:51:47.079289+0000","last_peered":"2024-09-10T18:51:47.079289+0000","last_clean":"2024-09-10T18:51:47.079289+0000","last_became_active":"2024-09-10T18:45:22.072877+0000","last_became_peered":"2024-09-10T18:45:22.072877+0000","last_unstale":"2024-09-10T18:51:47.079289+0000","last_undegraded":"2024-09-10T18:51:47.079289+0000","last_fullsized":"2024-09-10T18:51:47.079289+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:47.078574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:03.571786+0000","last_clean_scrub_stamp":"2024-09-10T18:51:47.078574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:06.636735+0000","scrub_duration":0.001570454,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":150,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.209137+0000","last_change":"2024-09-10T18:51:47.209085+0000","last_active":"2024-09-10T18:51:47.209137+0000","last_peered":"2024-09-10T18:51:47.209137+0000","last_clean":"2024-09-10T18:51:47.209137+0000","last_became_active":"2024-09-10T18:45:24.081050+0000","last_became_peered":"2024-09-10T18:45:24.081050+0000","last_unstale":"2024-09-10T18:51:47.209137+0000","last_undegraded":"2024-09-10T18:51:47.209137+0000","last_fullsized":"2024-09-10T18:51:47.209137+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:47.209023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:04.631558+0000","last_clean_scrub_stamp":"2024-09-10T18:51:47.209023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:13.589713+0000","scrub_duration":0.0019148069999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"63'11","reported_seq":151,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-10T18:51:41.592255+0000","last_change":"2024-09-10T18:49:56.622986+0000","last_active":"2024-09-10T18:51:41.592255+0000","last_peered":"2024-09-10T18:51:41.592255+0000","last_clean":"2024-09-10T18:51:41.592255+0000","last_became_active":"2024-09-10T18:45:20.056885+0000","last_became_peered":"2024-09-10T18:45:20.056885+0000","last_unstale":"2024-09-10T18:51:41.592255+0000","last_undegraded":"2024-09-10T18:51:41.592255+0000","last_fullsized":"2024-09-10T18:51:41.592255+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:49:56.622879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:49:56.622879+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0077720799999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":146,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.611880+0000","last_change":"2024-09-10T18:51:00.029596+0000","last_active":"2024-09-10T18:51:46.611880+0000","last_peered":"2024-09-10T18:51:46.611880+0000","last_clean":"2024-09-10T18:51:46.611880+0000","last_became_active":"2024-09-10T18:45:17.963542+0000","last_became_peered":"2024-09-10T18:45:17.963542+0000","last_unstale":"2024-09-10T18:51:46.611880+0000","last_undegraded":"2024-09-10T18:51:46.611880+0000","last_fullsized":"2024-09-10T18:51:46.611880+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:00.029389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:00.029389+0000","last_clean_scrub_stamp":"2024-09-10T18:51:00.029389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:25.353957+0000","scrub_duration":0.0023207890000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":145,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.610874+0000","last_change":"2024-09-10T18:50:45.003161+0000","last_active":"2024-09-10T18:51:46.610874+0000","last_peered":"2024-09-10T18:51:46.610874+0000","last_clean":"2024-09-10T18:51:46.610874+0000","last_became_active":"2024-09-10T18:45:22.070065+0000","last_became_peered":"2024-09-10T18:45:22.070065+0000","last_unstale":"2024-09-10T18:51:46.610874+0000","last_undegraded":"2024-09-10T18:51:46.610874+0000","last_fullsized":"2024-09-10T18:51:46.610874+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:45.002838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:45.002838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:58.280429+0000","scrub_duration":0.017421118999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":130,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.078319+0000","last_change":"2024-09-10T18:49:58.479928+0000","last_active":"2024-09-10T18:51:47.078319+0000","last_peered":"2024-09-10T18:51:47.078319+0000","last_clean":"2024-09-10T18:51:47.078319+0000","last_became_active":"2024-09-10T18:45:24.072224+0000","last_became_peered":"2024-09-10T18:45:24.072224+0000","last_unstale":"2024-09-10T18:51:47.078319+0000","last_undegraded":"2024-09-10T18:51:47.078319+0000","last_fullsized":"2024-09-10T18:51:47.078319+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:58.479842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:58.479842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.025012936,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"63'4","reported_seq":150,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.078523+0000","last_change":"2024-09-10T18:50:46.171962+0000","last_active":"2024-09-10T18:51:47.078523+0000","last_peered":"2024-09-10T18:51:47.078523+0000","last_clean":"2024-09-10T18:51:47.078523+0000","last_became_active":"2024-09-10T18:45:20.059205+0000","last_became_peered":"2024-09-10T18:45:20.059205+0000","last_unstale":"2024-09-10T18:51:47.078523+0000","last_undegraded":"2024-09-10T18:51:47.078523+0000","last_fullsized":"2024-09-10T18:51:47.078523+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4","last_scrub_stamp":"2024-09-10T18:50:46.171568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:50:46.171568+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:59.994766+0000","scrub_duration":0.0054395399999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":155,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.611391+0000","last_change":"2024-09-10T18:51:43.344854+0000","last_active":"2024-09-10T18:51:46.611391+0000","last_peered":"2024-09-10T18:51:46.611391+0000","last_clean":"2024-09-10T18:51:46.611391+0000","last_became_active":"2024-09-10T18:47:36.415866+0000","last_became_peered":"2024-09-10T18:47:36.415866+0000","last_unstale":"2024-09-10T18:51:46.611391+0000","last_undegraded":"2024-09-10T18:51:46.611391+0000","last_fullsized":"2024-09-10T18:51:46.611391+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:43.344751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:43.344751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:01.340612+0000","scrub_duration":0.0019741749999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":151,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:54.514859+0000","last_change":"2024-09-10T18:51:54.514805+0000","last_active":"2024-09-10T18:51:54.514859+0000","last_peered":"2024-09-10T18:51:54.514859+0000","last_clean":"2024-09-10T18:51:54.514859+0000","last_became_active":"2024-09-10T18:45:22.072377+0000","last_became_peered":"2024-09-10T18:45:22.072377+0000","last_unstale":"2024-09-10T18:51:54.514859+0000","last_undegraded":"2024-09-10T18:51:54.514859+0000","last_fullsized":"2024-09-10T18:51:54.514859+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:54.514747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_clean_scrub_stamp":"2024-09-10T18:51:54.514747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:24.200821+0000","scrub_duration":0.0015998679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":160,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:55.145770+0000","last_change":"2024-09-10T18:51:55.145730+0000","last_active":"2024-09-10T18:51:55.145770+0000","last_peered":"2024-09-10T18:51:55.145770+0000","last_clean":"2024-09-10T18:51:55.145770+0000","last_became_active":"2024-09-10T18:47:36.416187+0000","last_became_peered":"2024-09-10T18:47:36.416187+0000","last_unstale":"2024-09-10T18:51:55.145770+0000","last_undegraded":"2024-09-10T18:51:55.145770+0000","last_fullsized":"2024-09-10T18:51:55.145770+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:55.145638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:55.145638+0000","last_clean_scrub_stamp":"2024-09-10T18:51:55.145638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:06.238150+0000","scrub_duration":0.00318506,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"79'16","reported_seq":168,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.078492+0000","last_change":"2024-09-10T18:51:36.997632+0000","last_active":"2024-09-10T18:51:47.078492+0000","last_peered":"2024-09-10T18:51:47.078492+0000","last_clean":"2024-09-10T18:51:47.078492+0000","last_became_active":"2024-09-10T18:45:20.421479+0000","last_became_peered":"2024-09-10T18:45:20.421479+0000","last_unstale":"2024-09-10T18:51:47.078492+0000","last_undegraded":"2024-09-10T18:51:47.078492+0000","last_fullsized":"2024-09-10T18:51:47.078492+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"79'16","last_scrub_stamp":"2024-09-10T18:51:36.997571+0000","last_deep_scrub":"79'16","last_deep_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.997571+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:45.812239+0000","scrub_duration":0.002345059,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"56'1","reported_seq":166,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.739628+0000","last_change":"2024-09-10T18:50:43.279453+0000","last_active":"2024-09-10T18:51:47.739628+0000","last_peered":"2024-09-10T18:51:47.739628+0000","last_clean":"2024-09-10T18:51:47.739628+0000","last_became_active":"2024-09-10T18:47:36.421985+0000","last_became_peered":"2024-09-10T18:47:36.421985+0000","last_unstale":"2024-09-10T18:51:47.739628+0000","last_undegraded":"2024-09-10T18:51:47.739628+0000","last_fullsized":"2024-09-10T18:51:47.739628+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:50:43.279224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:43.279224+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:04.568803+0000","scrub_duration":0.0048255859999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":96,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.078332+0000","last_change":"2024-09-10T18:50:41.057828+0000","last_active":"2024-09-10T18:51:47.078332+0000","last_peered":"2024-09-10T18:51:47.078332+0000","last_clean":"2024-09-10T18:51:47.078332+0000","last_became_active":"2024-09-10T18:47:57.107149+0000","last_became_peered":"2024-09-10T18:47:57.107149+0000","last_unstale":"2024-09-10T18:51:47.078332+0000","last_undegraded":"2024-09-10T18:51:47.078332+0000","last_fullsized":"2024-09-10T18:51:47.078332+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:41.057750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:41.057750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:51:51.643566+0000","scrub_duration":0.013401967000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":140,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.078905+0000","last_change":"2024-09-10T18:50:13.793044+0000","last_active":"2024-09-10T18:51:47.078905+0000","last_peered":"2024-09-10T18:51:47.078905+0000","last_clean":"2024-09-10T18:51:47.078905+0000","last_became_active":"2024-09-10T18:45:24.264818+0000","last_became_peered":"2024-09-10T18:45:24.264818+0000","last_unstale":"2024-09-10T18:51:47.078905+0000","last_undegraded":"2024-09-10T18:51:47.078905+0000","last_fullsized":"2024-09-10T18:51:47.078905+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:13.792946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:50:13.792946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012475749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"78'2","reported_seq":166,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.212271+0000","last_change":"2024-09-10T18:51:46.249221+0000","last_active":"2024-09-10T18:51:47.212271+0000","last_peered":"2024-09-10T18:51:47.212271+0000","last_clean":"2024-09-10T18:51:47.212271+0000","last_became_active":"2024-09-10T18:47:36.418702+0000","last_became_peered":"2024-09-10T18:47:36.418702+0000","last_unstale":"2024-09-10T18:51:47.212271+0000","last_undegraded":"2024-09-10T18:51:47.212271+0000","last_fullsized":"2024-09-10T18:51:47.212271+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'2","last_scrub_stamp":"2024-09-10T18:51:46.249165+0000","last_deep_scrub":"78'2","last_deep_scrub_stamp":"2024-09-10T18:46:37.581130+0000","last_clean_scrub_stamp":"2024-09-10T18:51:46.249165+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:56.099034+0000","scrub_duration":0.001625867,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":145,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.610805+0000","last_change":"2024-09-10T18:51:03.197559+0000","last_active":"2024-09-10T18:51:46.610805+0000","last_peered":"2024-09-10T18:51:46.610805+0000","last_clean":"2024-09-10T18:51:46.610805+0000","last_became_active":"2024-09-10T18:45:17.962108+0000","last_became_peered":"2024-09-10T18:45:17.962108+0000","last_unstale":"2024-09-10T18:51:46.610805+0000","last_undegraded":"2024-09-10T18:51:46.610805+0000","last_fullsized":"2024-09-10T18:51:46.610805+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:03.197480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:03.197480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:15.493158+0000","scrub_duration":0.0024932259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"63'10","reported_seq":159,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.611401+0000","last_change":"2024-09-10T18:50:07.754452+0000","last_active":"2024-09-10T18:51:46.611401+0000","last_peered":"2024-09-10T18:51:46.611401+0000","last_clean":"2024-09-10T18:51:46.611401+0000","last_became_active":"2024-09-10T18:45:20.421094+0000","last_became_peered":"2024-09-10T18:45:20.421094+0000","last_unstale":"2024-09-10T18:51:46.611401+0000","last_undegraded":"2024-09-10T18:51:46.611401+0000","last_fullsized":"2024-09-10T18:51:46.611401+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:50:07.754001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:50:07.754001+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.019947473,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":241,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:50.581021+0000","last_change":"2024-09-10T18:51:50.580960+0000","last_active":"2024-09-10T18:51:50.581021+0000","last_peered":"2024-09-10T18:51:50.581021+0000","last_clean":"2024-09-10T18:51:50.581021+0000","last_became_active":"2024-09-10T18:45:22.073281+0000","last_became_peered":"2024-09-10T18:45:22.073281+0000","last_unstale":"2024-09-10T18:51:50.581021+0000","last_undegraded":"2024-09-10T18:51:50.581021+0000","last_fullsized":"2024-09-10T18:51:50.581021+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:51:50.580871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:50.580871+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:52.495673+0000","scrub_duration":0.002070484,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":140,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.610893+0000","last_change":"2024-09-10T18:51:40.051314+0000","last_active":"2024-09-10T18:51:46.610893+0000","last_peered":"2024-09-10T18:51:46.610893+0000","last_clean":"2024-09-10T18:51:46.610893+0000","last_became_active":"2024-09-10T18:45:24.082318+0000","last_became_peered":"2024-09-10T18:45:24.082318+0000","last_unstale":"2024-09-10T18:51:46.610893+0000","last_undegraded":"2024-09-10T18:51:46.610893+0000","last_fullsized":"2024-09-10T18:51:46.610893+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:40.051241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:03.475299+0000","last_clean_scrub_stamp":"2024-09-10T18:51:40.051241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:48.927775+0000","scrub_duration":0.001564937,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":146,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.078790+0000","last_change":"2024-09-10T18:51:01.165781+0000","last_active":"2024-09-10T18:51:47.078790+0000","last_peered":"2024-09-10T18:51:47.078790+0000","last_clean":"2024-09-10T18:51:47.078790+0000","last_became_active":"2024-09-10T18:45:17.965707+0000","last_became_peered":"2024-09-10T18:45:17.965707+0000","last_unstale":"2024-09-10T18:51:47.078790+0000","last_undegraded":"2024-09-10T18:51:47.078790+0000","last_fullsized":"2024-09-10T18:51:47.078790+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:01.165584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:01.165584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:28.929394+0000","scrub_duration":0.0071542790000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"63'11","reported_seq":172,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.739577+0000","last_change":"2024-09-10T18:51:46.517627+0000","last_active":"2024-09-10T18:51:47.739577+0000","last_peered":"2024-09-10T18:51:47.739577+0000","last_clean":"2024-09-10T18:51:47.739577+0000","last_became_active":"2024-09-10T18:45:20.058717+0000","last_became_peered":"2024-09-10T18:45:20.058717+0000","last_unstale":"2024-09-10T18:51:47.739577+0000","last_undegraded":"2024-09-10T18:51:47.739577+0000","last_fullsized":"2024-09-10T18:51:47.739577+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:51:46.517571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:46.517571+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:12.935571+0000","scrub_duration":0.001736118,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":152,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.611196+0000","last_change":"2024-09-10T18:50:54.765172+0000","last_active":"2024-09-10T18:51:46.611196+0000","last_peered":"2024-09-10T18:51:46.611196+0000","last_clean":"2024-09-10T18:51:46.611196+0000","last_became_active":"2024-09-10T18:47:36.418767+0000","last_became_peered":"2024-09-10T18:47:36.418767+0000","last_unstale":"2024-09-10T18:51:46.611196+0000","last_undegraded":"2024-09-10T18:51:46.611196+0000","last_fullsized":"2024-09-10T18:51:46.611196+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:54.764990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:54.764990+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:14.182768+0000","scrub_duration":0.0043824299999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":138,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-10T18:51:41.592616+0000","last_change":"2024-09-10T18:51:36.324657+0000","last_active":"2024-09-10T18:51:41.592616+0000","last_peered":"2024-09-10T18:51:41.592616+0000","last_clean":"2024-09-10T18:51:41.592616+0000","last_became_active":"2024-09-10T18:45:24.267251+0000","last_became_peered":"2024-09-10T18:45:24.267251+0000","last_unstale":"2024-09-10T18:51:41.592616+0000","last_undegraded":"2024-09-10T18:51:41.592616+0000","last_fullsized":"2024-09-10T18:51:41.592616+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.324592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.324592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:56.184657+0000","scrub_duration":0.001354409,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"56'1","reported_seq":154,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.739084+0000","last_change":"2024-09-10T18:50:07.341477+0000","last_active":"2024-09-10T18:51:47.739084+0000","last_peered":"2024-09-10T18:51:47.739084+0000","last_clean":"2024-09-10T18:51:47.739084+0000","last_became_active":"2024-09-10T18:45:17.966286+0000","last_became_peered":"2024-09-10T18:45:17.966286+0000","last_unstale":"2024-09-10T18:51:47.739084+0000","last_undegraded":"2024-09-10T18:51:47.739084+0000","last_fullsized":"2024-09-10T18:51:47.739084+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:50:07.341394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:07.341394+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019580940000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"63'15","reported_seq":150,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:46.611263+0000","last_change":"2024-09-10T18:50:00.455229+0000","last_active":"2024-09-10T18:51:46.611263+0000","last_peered":"2024-09-10T18:51:46.611263+0000","last_clean":"2024-09-10T18:51:46.611263+0000","last_became_active":"2024-09-10T18:47:36.424373+0000","last_became_peered":"2024-09-10T18:47:36.424373+0000","last_unstale":"2024-09-10T18:51:46.611263+0000","last_undegraded":"2024-09-10T18:51:46.611263+0000","last_fullsized":"2024-09-10T18:51:46.611263+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:50:00.455005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:50:00.455005+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.018138438999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":142,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-10T18:51:54.153047+0000","last_change":"2024-09-10T18:51:54.153006+0000","last_active":"2024-09-10T18:51:54.153047+0000","last_peered":"2024-09-10T18:51:54.153047+0000","last_clean":"2024-09-10T18:51:54.153047+0000","last_became_active":"2024-09-10T18:45:22.059794+0000","last_became_peered":"2024-09-10T18:45:22.059794+0000","last_unstale":"2024-09-10T18:51:54.153047+0000","last_undegraded":"2024-09-10T18:51:54.153047+0000","last_fullsized":"2024-09-10T18:51:54.153047+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:54.152908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:54.152908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:12.029577+0000","scrub_duration":0.0079641550000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":130,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.079345+0000","last_change":"2024-09-10T18:49:51.464014+0000","last_active":"2024-09-10T18:51:47.079345+0000","last_peered":"2024-09-10T18:51:47.079345+0000","last_clean":"2024-09-10T18:51:47.079345+0000","last_became_active":"2024-09-10T18:45:24.264427+0000","last_became_peered":"2024-09-10T18:45:24.264427+0000","last_unstale":"2024-09-10T18:51:47.079345+0000","last_undegraded":"2024-09-10T18:51:47.079345+0000","last_fullsized":"2024-09-10T18:51:47.079345+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:51.463929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:51.463929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.028103624000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":146,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.079456+0000","last_change":"2024-09-10T18:51:40.076322+0000","last_active":"2024-09-10T18:51:47.079456+0000","last_peered":"2024-09-10T18:51:47.079456+0000","last_clean":"2024-09-10T18:51:47.079456+0000","last_became_active":"2024-09-10T18:45:17.963257+0000","last_became_peered":"2024-09-10T18:45:17.963257+0000","last_unstale":"2024-09-10T18:51:47.079456+0000","last_undegraded":"2024-09-10T18:51:47.079456+0000","last_fullsized":"2024-09-10T18:51:47.079456+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:40.076262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:40.076262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:58.187702+0000","scrub_duration":0.0037079650000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"63'12","reported_seq":167,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T18:51:47.079089+0000","last_change":"2024-09-10T18:51:31.148694+0000","last_active":"2024-09-10T18:51:47.079089+0000","last_peered":"2024-09-10T18:51:47.079089+0000","last_clean":"2024-09-10T18:51:47.079089+0000","last_became_active":"2024-09-10T18:45:20.058391+0000","last_became_peered":"2024-09-10T18:45:20.058391+0000","last_unstale":"2024-09-10T18:51:47.079089+0000","last_undegraded":"2024-09-10T18:51:47.079089+0000","last_fullsized":"2024-09-10T18:51:47.079089+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:51:31.148630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:31.148630+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:33.947829+0000","scrub_duration":0.0021214699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":150,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-10T18:51:41.592404+0000","last_change":"2024-09-10T18:50:41.560291+0000","last_active":"2024-09-10T18:51:41.592404+0000","last_peered":"2024-09-10T18:51:41.592404+0000","last_clean":"2024-09-10T18:51:41.592404+0000","last_became_active":"2024-09-10T18:47:36.416131+0000","last_became_peered":"2024-09-10T18:47:36.416131+0000","last_unstale":"2024-09-10T18:51:41.592404+0000","last_undegraded":"2024-09-10T18:51:41.592404+0000","last_fullsized":"2024-09-10T18:51:41.592404+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:41.560194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:41.560194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:08.620669+0000","scrub_duration":0.0067616830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":1486028800,"num_objects":22675,"num_object_clones":0,"num_object_copies":68025,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22675,"num_write_kb":1451200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4434296832,"data_stored":4434296832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22675,"ondisk_log_size":22675,"up":48,"acting":48,"num_store_stats":7},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":697,"num_read_kb":454,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":1238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":455,"ondisk_log_size":455,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":388,"num_read_kb":383,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":891,"ondisk_log_size":891,"up":33,"acting":33,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299476,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":774324,"kb_used_data":476972,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":92970828,"statfs":{"total":95995035648,"available":95202127872,"internally_reserved":0,"allocated":488419328,"data_stored":487867288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":1.256,"15min":0.75700000000000001},"min":{"1min":0.21099999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":3.073,"5min":31.805,"15min":31.805},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.93200000000000005,"5min":1.5169999999999999,"15min":0.82899999999999996},"min":{"1min":0.23999999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":6.5010000000000003,"5min":31.212,"15min":31.212},"last":0.45800000000000002}]},{"osd":1,"last update":"Tue Sep 10 18:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.98299999999999998,"15min":0.64600000000000002},"min":{"1min":0.23100000000000001,"5min":0.221,"15min":0.221},"max":{"1min":2.9460000000000002,"5min":21.34,"15min":21.34},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.94799999999999995,"15min":0.66200000000000003},"min":{"1min":0.23599999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.869,"5min":21.248000000000001,"15min":21.248000000000001},"last":0.49099999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.625,"15min":0.53200000000000003},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.745,"5min":5.1399999999999997,"15min":5.1399999999999997},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.72099999999999997,"15min":0.58399999999999996},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.1349999999999998,"5min":8.8859999999999992,"15min":8.8859999999999992},"last":0.69699999999999995}]},{"osd":3,"last update":"Tue Sep 10 18:51:19 2024","interfaces":[{"interface":"back","average":{"1min":1.73,"5min":1.4850000000000001,"15min":0.83299999999999996},"min":{"1min":0.251,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":9.3089999999999993,"5min":31.478000000000002,"15min":31.478000000000002},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.6699999999999999,"15min":0.90200000000000002},"min":{"1min":0.29499999999999998,"5min":0.254,"15min":0.254},"max":{"1min":9.1549999999999994,"5min":33.731999999999999,"15min":33.731999999999999},"last":1.0409999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.45700000000000002,"15min":0.40899999999999997},"min":{"1min":0.24099999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.6080000000000001,"5min":2.7559999999999998,"15min":2.7559999999999998},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.379,"5min":0.45300000000000001,"15min":0.40100000000000002},"min":{"1min":0.254,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.79800000000000004,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.66600000000000004}]},{"osd":5,"last update":"Tue Sep 10 18:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.46899999999999997,"15min":0.43099999999999999},"min":{"1min":0.26900000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.5489999999999999,"5min":2.7240000000000002,"15min":2.7240000000000002},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.44700000000000001,"15min":0.42599999999999999},"min":{"1min":0.27100000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.85899999999999999,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.83299999999999996}]},{"osd":6,"last update":"Tue Sep 10 18:51:19 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.54700000000000004,"15min":0.48099999999999998},"min":{"1min":0.27200000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.6299999999999999,"5min":2.8199999999999998,"15min":2.8199999999999998},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.44,"5min":0.53500000000000003,"15min":0.46700000000000003},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71999999999999997,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.438}]}]},{"osd":6,"up_from":43,"seq":184683593817,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":874432,"kb_used_data":572920,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":92870720,"statfs":{"total":95995035648,"available":95099617280,"internally_reserved":0,"allocated":586670080,"data_stored":586106840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":1.591,"15min":0.83699999999999997},"min":{"1min":0.41399999999999998,"5min":0.215,"15min":0.215},"max":{"1min":2.9700000000000002,"5min":46.384999999999998,"15min":46.384999999999998},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.526,"15min":0.83699999999999997},"min":{"1min":0.43099999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":4.9500000000000002,"5min":48.372999999999998,"15min":48.372999999999998},"last":0.51600000000000001}]},{"osd":1,"last update":"Tue Sep 10 18:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.92700000000000005,"15min":0.63500000000000001},"min":{"1min":0.308,"5min":0.219,"15min":0.219},"max":{"1min":2.3039999999999998,"5min":6.1319999999999997,"15min":6.1319999999999997},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.82599999999999996,"15min":0.58399999999999996},"min":{"1min":0.34499999999999997,"5min":0.214,"15min":0.214},"max":{"1min":2.8929999999999998,"5min":3.6280000000000001,"15min":3.6280000000000001},"last":0.57899999999999996}]},{"osd":2,"last update":"Tue Sep 10 18:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.80800000000000005,"15min":0.58999999999999997},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.1240000000000001,"5min":4.4740000000000002,"15min":4.4740000000000002},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.60499999999999998,"15min":0.52900000000000003},"min":{"1min":0.34899999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.75800000000000001,"5min":2.9900000000000002,"15min":2.9900000000000002},"last":0.33900000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.87,"15min":0.61199999999999999},"min":{"1min":0.34300000000000003,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":2.992,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.101,"15min":0.70799999999999996},"min":{"1min":0.434,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":4.6109999999999998,"5min":5.3079999999999998,"15min":5.3079999999999998},"last":0.55800000000000005}]},{"osd":4,"last update":"Tue Sep 10 18:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.56899999999999995,"15min":0.45900000000000002},"min":{"1min":0.30199999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.69299999999999995,"5min":4.2969999999999997,"15min":4.2969999999999997},"last":0.247},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.54300000000000004,"15min":0.441},"min":{"1min":0.29199999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":2.9430000000000001,"5min":4.3209999999999997,"15min":4.3209999999999997},"last":0.375}]},{"osd":5,"last update":"Tue Sep 10 18:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59699999999999998,"15min":0.47699999999999998},"min":{"1min":0.26700000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":3.0950000000000002,"5min":4.4210000000000003,"15min":4.4210000000000003},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59199999999999997,"15min":0.48099999999999998},"min":{"1min":0.30399999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.89300000000000002,"5min":4.4500000000000002,"15min":4.4500000000000002},"last":0.27800000000000002}]},{"osd":7,"last update":"Tue Sep 10 18:51:12 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.66800000000000004,"15min":0.53500000000000003},"min":{"1min":0.35599999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":3.1509999999999998,"5min":4.5259999999999998,"15min":4.5259999999999998},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.67200000000000004,"15min":0.55300000000000005},"min":{"1min":0.36099999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.5920000000000001,"5min":4.1299999999999999,"15min":4.1299999999999999},"last":0.38600000000000001}]}]},{"osd":5,"up_from":37,"seq":158913790045,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1030260,"kb_used_data":720428,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":92714892,"statfs":{"total":95995035648,"available":94940049408,"internally_reserved":0,"allocated":737718272,"data_stored":737166244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,3,4],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:51:28 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":1.502,"15min":0.85099999999999998},"min":{"1min":0.23599999999999999,"5min":0.183,"15min":0.183},"max":{"1min":3.6859999999999999,"5min":65.697000000000003,"15min":65.697000000000003},"last":2.601},{"interface":"front","average":{"1min":0.753,"5min":1.5669999999999999,"15min":0.85299999999999998},"min":{"1min":0.29899999999999999,"5min":0.219,"15min":0.219},"max":{"1min":3.0419999999999998,"5min":65.683999999999997,"15min":65.683999999999997},"last":1.4319999999999999}]},{"osd":1,"last update":"Tue Sep 10 18:51:28 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.67800000000000005,"15min":0.59799999999999998},"min":{"1min":0.313,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.0700000000000001,"5min":3.5150000000000001,"15min":3.5150000000000001},"last":2.145},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.80100000000000005,"15min":0.625},"min":{"1min":0.29299999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.1080000000000001,"5min":6.6989999999999998,"15min":6.6989999999999998},"last":1.464}]},{"osd":2,"last update":"Tue Sep 10 18:51:28 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.65400000000000003,"15min":0.56399999999999995},"min":{"1min":0.34999999999999998,"5min":0.249,"15min":0.249},"max":{"1min":2.5449999999999999,"5min":7.8029999999999999,"15min":7.8029999999999999},"last":1.446},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.72299999999999998,"15min":0.60099999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.052,"5min":6.7210000000000001,"15min":6.7210000000000001},"last":2.1139999999999999}]},{"osd":3,"last update":"Tue Sep 10 18:51:28 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":1.6519999999999999,"15min":0.90100000000000002},"min":{"1min":0.28299999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":4.2450000000000001,"5min":66.593999999999994,"15min":66.593999999999994},"last":1.417},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.7490000000000001,"15min":0.94599999999999995},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":6.6870000000000003,"5min":65.939999999999998,"15min":65.939999999999998},"last":2.1659999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:51:28 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.47799999999999998,"15min":0.41399999999999998},"min":{"1min":0.253,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":1.1379999999999999,"5min":2.133,"15min":2.133},"last":1.498},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.498,"15min":0.435},"min":{"1min":0.24399999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":2.5099999999999998,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.3839999999999999}]},{"osd":6,"last update":"Tue Sep 10 18:51:52 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.76500000000000001,"15min":0.53400000000000003},"min":{"1min":0.26900000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":1.2629999999999999,"5min":11.17,"15min":11.17},"last":3.5720000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.82399999999999995,"15min":0.58199999999999996},"min":{"1min":0.36199999999999999,"5min":0.33000000000000002,"15min":0.31},"max":{"1min":2.5720000000000001,"5min":11.18,"15min":11.18},"last":1.7929999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:51:14 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.69599999999999995,"15min":0.55000000000000004},"min":{"1min":0.33100000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.6880000000000002,"5min":10.782999999999999,"15min":10.782999999999999},"last":1.484},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.67800000000000005,"15min":0.54400000000000004},"min":{"1min":0.31,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.8879999999999999,"5min":11.846,"15min":11.846},"last":2.569}]}]},{"osd":4,"up_from":30,"seq":128849018979,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":779448,"kb_used_data":482032,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":92965704,"statfs":{"total":95995035648,"available":95196880896,"internally_reserved":0,"allocated":493600768,"data_stored":493044483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:51:11 2024","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.456,"15min":0.83499999999999996},"min":{"1min":0.221,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":4.7050000000000001,"5min":55.776000000000003,"15min":55.776000000000003},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.998,"5min":1.421,"15min":0.83499999999999996},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":3.3260000000000001,"5min":55.737000000000002,"15min":55.737000000000002},"last":1.2250000000000001}]},{"osd":1,"last update":"Tue Sep 10 18:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.86399999999999999,"15min":0.67300000000000004},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":2.9409999999999998,"5min":8.1600000000000001,"15min":8.1600000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":1.01,"5min":0.72899999999999998,"15min":0.59999999999999998},"min":{"1min":0.28000000000000003,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":2.5990000000000002,"5min":2.5990000000000002,"15min":2.5990000000000002},"last":0.33900000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:51:11 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.55100000000000005,"15min":0.54900000000000004},"min":{"1min":0.307,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":2.153,"5min":2.153,"15min":2.6030000000000002},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.56299999999999994,"15min":0.55600000000000005},"min":{"1min":0.27900000000000003,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.4790000000000001,"5min":2.8109999999999999,"15min":2.8109999999999999},"last":1.49}]},{"osd":3,"last update":"Tue Sep 10 18:51:11 2024","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.2010000000000001,"15min":0.753},"min":{"1min":0.26000000000000001,"5min":0.246,"15min":0.246},"max":{"1min":3.4009999999999998,"5min":24.957999999999998,"15min":24.957999999999998},"last":1.4390000000000001},{"interface":"front","average":{"1min":0.90300000000000002,"5min":1.093,"15min":0.72399999999999998},"min":{"1min":0.33300000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":2.1789999999999998,"5min":24.940000000000001,"15min":24.940000000000001},"last":0.64000000000000001}]},{"osd":5,"last update":"Tue Sep 10 18:51:36 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52200000000000002,"15min":0.40899999999999997},"min":{"1min":0.26000000000000001,"5min":0.222,"15min":0.222},"max":{"1min":1.151,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.48199999999999998,"15min":0.40200000000000002},"min":{"1min":0.28100000000000003,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.3859999999999999,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":0.50700000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56399999999999995,"15min":0.47099999999999997},"min":{"1min":0.34999999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.4139999999999999,"5min":3.5529999999999999,"15min":3.5529999999999999},"last":2.4889999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.65000000000000002,"15min":0.47899999999999998},"min":{"1min":0.33400000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.7549999999999999,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":0.72799999999999998}]},{"osd":7,"last update":"Tue Sep 10 18:51:18 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.56699999999999995,"15min":0.52300000000000002},"min":{"1min":0.318,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.4359999999999999,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54300000000000004,"15min":0.48099999999999998},"min":{"1min":0.31,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.504,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.468}]}]},{"osd":3,"up_from":25,"seq":107374182504,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":935496,"kb_used_data":625664,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":92809656,"statfs":{"total":95995035648,"available":95037087744,"internally_reserved":0,"allocated":640679936,"data_stored":640107644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:51:43 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":1.911,"15min":0.89200000000000002},"min":{"1min":0.17499999999999999,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":2.2749999999999999,"5min":47.601999999999997,"15min":47.601999999999997},"last":0.246},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.7869999999999999,"15min":0.86399999999999999},"min":{"1min":0.224,"5min":0.217,"15min":0.217},"max":{"1min":4.8369999999999997,"5min":46.787999999999997,"15min":46.787999999999997},"last":0.40300000000000002}]},{"osd":1,"last update":"Tue Sep 10 18:51:43 2024","interfaces":[{"interface":"back","average":{"1min":3.8109999999999999,"5min":1.968,"15min":0.92800000000000005},"min":{"1min":0.23699999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":58.215000000000003,"5min":58.215000000000003,"15min":58.215000000000003},"last":0.38},{"interface":"front","average":{"1min":3.7799999999999998,"5min":2.0150000000000001,"15min":0.94799999999999995},"min":{"1min":0.26400000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":57.896999999999998,"5min":57.896999999999998,"15min":57.896999999999998},"last":0.67300000000000004}]},{"osd":2,"last update":"Tue Sep 10 18:51:43 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":1.024,"15min":0.63200000000000001},"min":{"1min":0.28699999999999998,"5min":0.246,"15min":0.246},"max":{"1min":3.0190000000000001,"5min":5.8220000000000001,"15min":5.8220000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.434,"5min":0.89000000000000001,"15min":0.58299999999999996},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":1.2490000000000001,"5min":7.0979999999999999,"15min":7.0979999999999999},"last":0.29799999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:51:06 2024","interfaces":[{"interface":"back","average":{"1min":1.6339999999999999,"5min":1.0649999999999999,"15min":0.71599999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":7.1790000000000003,"5min":7.1790000000000003,"15min":7.1790000000000003},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.7989999999999999,"5min":1.0820000000000001,"15min":0.74099999999999999},"min":{"1min":0.315,"5min":0.311,"15min":0.311},"max":{"1min":7.1349999999999998,"5min":7.1349999999999998,"15min":7.1349999999999998},"last":0.88800000000000001}]},{"osd":5,"last update":"Tue Sep 10 18:51:36 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":1.0620000000000001,"15min":0.71399999999999997},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":2.3180000000000001,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":0.35099999999999998},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1279999999999999,"15min":0.751},"min":{"1min":0.40500000000000003,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":3.3799999999999999,"5min":9.2509999999999994,"15min":9.2509999999999994},"last":0.94599999999999995}]},{"osd":6,"last update":"Tue Sep 10 18:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":1.264,"15min":0.79600000000000004},"min":{"1min":0.47999999999999998,"5min":0.315,"15min":0.315},"max":{"1min":3.411,"5min":9.3239999999999998,"15min":9.3239999999999998},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.68400000000000005,"5min":1.1899999999999999,"15min":0.76700000000000002},"min":{"1min":0.39300000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":2.2810000000000001,"5min":7.1559999999999997,"15min":7.1559999999999997},"last":1.0169999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:51:22 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":1.038,"15min":0.69999999999999996},"min":{"1min":0.377,"5min":0.375,"15min":0.375},"max":{"1min":1.726,"5min":4.71,"15min":4.71},"last":0.48799999999999999},{"interface":"front","average":{"1min":1.165,"5min":1.1879999999999999,"15min":0.78000000000000003},"min":{"1min":0.40500000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":3.4329999999999998,"5min":9.2949999999999999,"15min":9.2949999999999999},"last":0.70099999999999996}]}]},{"osd":2,"up_from":0,"seq":77309411436,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834574960,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":928588,"kb_used_data":627076,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":92816564,"statfs":{"total":95995035648,"available":95044161536,"internally_reserved":0,"allocated":642125824,"data_stored":641549475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:50:52 2024","interfaces":[{"interface":"back","average":{"1min":2.1880000000000002,"5min":1.107,"15min":0.58399999999999996},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":13.531000000000001,"5min":13.531000000000001,"15min":13.531000000000001},"last":1.1950000000000001},{"interface":"front","average":{"1min":4.0449999999999999,"5min":1.6419999999999999,"15min":0.77500000000000002},"min":{"1min":0.76000000000000001,"5min":0.252,"15min":0.21199999999999999},"max":{"1min":16.206,"5min":16.206,"15min":16.206},"last":0.36399999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:51:24 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.67600000000000005,"15min":0.46800000000000003},"min":{"1min":0.217,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":2.8159999999999998,"5min":3.8479999999999999,"15min":3.8479999999999999},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.84699999999999998,"15min":0.53800000000000003},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":3.754,"5min":7.093,"15min":7.093},"last":0.56399999999999995}]},{"osd":3,"last update":"Tue Sep 10 18:51:52 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.48,"15min":0.79800000000000004},"min":{"1min":0.36799999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.6509999999999998,"5min":12.714,"15min":12.714},"last":0.628},{"interface":"front","average":{"1min":0.90800000000000003,"5min":1.6950000000000001,"15min":0.84599999999999997},"min":{"1min":0.29899999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":4.4960000000000004,"5min":12.696,"15min":12.696},"last":0.29899999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:51:07 2024","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.083,"15min":0.73299999999999998},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":3.9239999999999999,"5min":4.1520000000000001,"15min":4.1520000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.85399999999999998,"15min":0.64100000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":2.8119999999999998,"5min":3.9540000000000002,"15min":3.9540000000000002},"last":0.91900000000000004}]},{"osd":5,"last update":"Tue Sep 10 18:51:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":0.98899999999999999,"15min":0.73199999999999998},"min":{"1min":0.34100000000000003,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.871,"5min":4.5869999999999997,"15min":4.5869999999999997},"last":2.1429999999999998},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.1439999999999999,"15min":0.80800000000000005},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":4,"5min":4.1130000000000004,"15min":4.1130000000000004},"last":0.42299999999999999}]},{"osd":6,"last update":"Tue Sep 10 18:51:52 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":1.466,"15min":0.91000000000000003},"min":{"1min":0.42999999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":3.851,"5min":38.039999999999999,"15min":38.039999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.6240000000000001,"15min":0.98799999999999999},"min":{"1min":0.33900000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":3.9500000000000002,"5min":38.100999999999999,"15min":38.100999999999999},"last":0.47999999999999998}]},{"osd":7,"last update":"Tue Sep 10 18:51:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.02,"15min":0.77200000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":3.8279999999999998,"5min":4.6539999999999999,"15min":4.6539999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":0.81100000000000005},"min":{"1min":0.36199999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":3.9700000000000002,"5min":4.1989999999999998,"15min":4.1989999999999998},"last":0.45100000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738486,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":853500,"kb_used_data":556148,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":92891652,"statfs":{"total":95995035648,"available":95121051648,"internally_reserved":0,"allocated":569495552,"data_stored":568934752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 18:51:00 2024","interfaces":[{"interface":"back","average":{"1min":2.3039999999999998,"5min":1.5620000000000001,"15min":0.72499999999999998},"min":{"1min":0.17599999999999999,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":8.4399999999999995,"5min":8.4399999999999995,"15min":8.4399999999999995},"last":0.441},{"interface":"front","average":{"1min":2.8340000000000001,"5min":1.5740000000000001,"15min":0.73599999999999999},"min":{"1min":0.23200000000000001,"5min":0.221,"15min":0.187},"max":{"1min":7.9580000000000002,"5min":7.9580000000000002,"15min":7.9580000000000002},"last":1.2829999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:51:32 2024","interfaces":[{"interface":"back","average":{"1min":1.6200000000000001,"5min":1.202,"15min":0.66200000000000003},"min":{"1min":0.254,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":6.4429999999999996,"5min":6.4429999999999996,"15min":6.4429999999999996},"last":0.29199999999999998},{"interface":"front","average":{"1min":1.377,"5min":0.97299999999999998,"15min":0.56799999999999995},"min":{"1min":0.28000000000000003,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":8.3230000000000004,"5min":8.3230000000000004,"15min":8.3230000000000004},"last":0.22600000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:51:44 2024","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.7350000000000001,"15min":0.84299999999999997},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":8.4740000000000002,"5min":8.4740000000000002,"15min":8.4740000000000002},"last":1.9550000000000001},{"interface":"front","average":{"1min":1.218,"5min":1.5649999999999999,"15min":0.79700000000000004},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":8.3670000000000009,"5min":9.8439999999999994,"15min":9.8439999999999994},"last":0.34300000000000003}]},{"osd":4,"last update":"Tue Sep 10 18:51:05 2024","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":0.97499999999999998,"15min":0.68400000000000005},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":4.992,"5min":4.992,"15min":4.992},"last":0.31},{"interface":"front","average":{"1min":1.6299999999999999,"5min":1.228,"15min":0.77200000000000002},"min":{"1min":0.40200000000000002,"5min":0.318,"15min":0.318},"max":{"1min":5.1379999999999999,"5min":5.1379999999999999,"15min":5.1379999999999999},"last":0.54200000000000004}]},{"osd":5,"last update":"Tue Sep 10 18:51:32 2024","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":1.0309999999999999,"15min":0.71799999999999997},"min":{"1min":0.36799999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":5.0129999999999999,"5min":5.0129999999999999,"15min":5.0129999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.202,"15min":0.77700000000000002},"min":{"1min":0.33100000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":5.3479999999999999,"5min":5.3479999999999999,"15min":5.3479999999999999},"last":1.3169999999999999}]},{"osd":6,"last update":"Tue Sep 10 18:50:55 2024","interfaces":[{"interface":"back","average":{"1min":1.853,"5min":1.256,"15min":0.78700000000000003},"min":{"1min":0.38,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":3.9769999999999999,"5min":4.6660000000000004,"15min":4.6660000000000004},"last":1.663},{"interface":"front","average":{"1min":1.4470000000000001,"5min":0.98299999999999998,"15min":0.85199999999999998},"min":{"1min":0.33600000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":3.6499999999999999,"5min":4.1890000000000001,"15min":4.6630000000000003},"last":0.75600000000000001}]},{"osd":7,"last update":"Tue Sep 10 18:51:18 2024","interfaces":[{"interface":"back","average":{"1min":1.8,"5min":1.72,"15min":0.94399999999999995},"min":{"1min":0.34000000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":5.3799999999999999,"5min":49.313000000000002,"15min":49.313000000000002},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.401,"5min":1.784,"15min":0.98999999999999999},"min":{"1min":0.45100000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":3.887,"5min":49.063000000000002,"15min":49.063000000000002},"last":1.6319999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":576651264,"data_stored":576651264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":649592832,"data_stored":649592832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":660144128,"data_stored":660144128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":546111488,"data_stored":546111488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":820314112,"data_stored":820314112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":632815616,"data_stored":632815616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":548667392,"data_stored":548667392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T18:52:01.731 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2024-09-10T18:52:01.732 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd pool set .mgr pg_num 21 2024-09-10T18:52:01.744 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:01.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:01.816 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:01.816 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:01.917 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:01.949 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:01.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:01.950 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:02.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:01 smithi103 ceph-mon[33163]: pgmap v279: 158 pgs: 158 active+clean; 1.7 GiB data, 7.2 GiB used, 619 GiB / 626 GiB avail; 853 B/s rd, 141 MiB/s wr, 2.26k op/s 2024-09-10T18:52:02.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:01 smithi103 ceph-mon[33163]: 4.6 scrub starts 2024-09-10T18:52:02.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:01 smithi103 ceph-mon[33163]: 4.6 scrub ok 2024-09-10T18:52:02.051 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:02.083 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:02.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:02.084 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:02.185 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:02.194 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 13 16 36445 36429 175.107 141.375 0.0084342 0.00570492 2024-09-10T18:52:02.263 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:02.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:02.265 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:02.365 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:02.401 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:02.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:02.402 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:02.503 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:02.538 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:02.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:02.539 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:02.640 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:02.675 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:02.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:02.676 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:02.777 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:02.813 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:02.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:02.814 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:02.915 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:02.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:02 smithi018 ceph-mon[40196]: 5.19 deep-scrub starts 2024-09-10T18:52:02.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:02 smithi018 ceph-mon[40196]: 5.19 deep-scrub ok 2024-09-10T18:52:02.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:02 smithi018 ceph-mon[40196]: 2.3 scrub starts 2024-09-10T18:52:02.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:02 smithi018 ceph-mon[40196]: 2.3 scrub ok 2024-09-10T18:52:02.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:02 smithi018 ceph-mon[40196]: 2.1f scrub starts 2024-09-10T18:52:02.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:02 smithi018 ceph-mon[40196]: 2.1f scrub ok 2024-09-10T18:52:02.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:02 smithi018 ceph-mon[29336]: 5.19 deep-scrub starts 2024-09-10T18:52:02.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:02 smithi018 ceph-mon[29336]: 5.19 deep-scrub ok 2024-09-10T18:52:02.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:02 smithi018 ceph-mon[29336]: 2.3 scrub starts 2024-09-10T18:52:02.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:02 smithi018 ceph-mon[29336]: 2.3 scrub ok 2024-09-10T18:52:02.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:02 smithi018 ceph-mon[29336]: 2.1f scrub starts 2024-09-10T18:52:02.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:02 smithi018 ceph-mon[29336]: 2.1f scrub ok 2024-09-10T18:52:02.956 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:02.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:02.957 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:03.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:02 smithi103 ceph-mon[33163]: 5.19 deep-scrub starts 2024-09-10T18:52:03.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:02 smithi103 ceph-mon[33163]: 5.19 deep-scrub ok 2024-09-10T18:52:03.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:02 smithi103 ceph-mon[33163]: 2.3 scrub starts 2024-09-10T18:52:03.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:02 smithi103 ceph-mon[33163]: 2.3 scrub ok 2024-09-10T18:52:03.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:02 smithi103 ceph-mon[33163]: 2.1f scrub starts 2024-09-10T18:52:03.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:02 smithi103 ceph-mon[33163]: 2.1f scrub ok 2024-09-10T18:52:03.060 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:03.092 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:03.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:03.093 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:03.194 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:03.195 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 14 16 38494 38478 171.745 128.062 0.00438514 0.00581602 2024-09-10T18:52:03.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:03.239 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:03.239 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:03.340 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:03.379 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:03.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:03.380 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:03.481 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:03.519 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:03.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:03.520 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:03.621 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:03.628 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-10T18:52:03.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:03.674 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:03.674 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:03.775 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:03.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:03.826 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:03.826 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:03.927 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:03.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:03 smithi018 ceph-mon[29336]: pgmap v280: 158 pgs: 158 active+clean; 2.1 GiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 1.2 KiB/s rd, 165 MiB/s wr, 2.64k op/s 2024-09-10T18:52:03.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:03 smithi018 ceph-mon[40196]: pgmap v280: 158 pgs: 158 active+clean; 2.1 GiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 1.2 KiB/s rd, 165 MiB/s wr, 2.64k op/s 2024-09-10T18:52:03.971 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:03.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:03.973 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:04.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:03 smithi103 ceph-mon[33163]: pgmap v280: 158 pgs: 158 active+clean; 2.1 GiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 1.2 KiB/s rd, 165 MiB/s wr, 2.64k op/s 2024-09-10T18:52:04.074 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:04.111 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:04.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:04.113 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:04.194 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 15 16 40339 40323 167.983 115.312 0.00975092 0.00594775 2024-09-10T18:52:04.214 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:04.246 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:04.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:04.247 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:04.348 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:04.385 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:04.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:04.386 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:04.487 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:04.521 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:04.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:04.522 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:04.623 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:04.655 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:04.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:04.656 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:04.757 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:04.793 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:04.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:04.794 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:04.895 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:04.922 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:52:04.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:04.939 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:04.939 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:04.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:04 smithi018 ceph-mon[29336]: 4.a scrub starts 2024-09-10T18:52:04.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:04 smithi018 ceph-mon[29336]: 4.a scrub ok 2024-09-10T18:52:04.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:04 smithi018 ceph-mon[40196]: 4.a scrub starts 2024-09-10T18:52:04.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:04 smithi018 ceph-mon[40196]: 4.a scrub ok 2024-09-10T18:52:04.988 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:52:04.988 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:52:05.016 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:52:05.016 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:52:05.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:04 smithi103 ceph-mon[33163]: 4.a scrub starts 2024-09-10T18:52:05.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:04 smithi103 ceph-mon[33163]: 4.a scrub ok 2024-09-10T18:52:05.039 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:05.047 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:52:05.048 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:52:05.080 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:52:05.081 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:52:05.081 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:05.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:05.083 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:05.117 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:52:05.118 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:52:05.151 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:52:05.151 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:52:05.184 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:05.185 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:52:05.185 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:52:05.194 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 16 16 42302 42286 165.151 122.688 0.00473562 0.00604898 2024-09-10T18:52:05.215 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:52:05.215 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:52:05.216 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:05.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:05.217 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:05.318 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:05.350 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:05.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:05.351 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:05.452 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:05.488 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:05.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:05.490 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:05.590 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:05.628 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:05.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:05.629 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:05.730 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:05.772 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:05.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:05.773 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:05.874 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:05.911 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:05.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:05.912 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:05.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:05 smithi018 ceph-mon[40196]: 1.a scrub starts 2024-09-10T18:52:05.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:05 smithi018 ceph-mon[40196]: 4.2 scrub starts 2024-09-10T18:52:05.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:05 smithi018 ceph-mon[40196]: 1.a scrub ok 2024-09-10T18:52:05.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:05 smithi018 ceph-mon[40196]: 4.2 scrub ok 2024-09-10T18:52:05.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:05 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2236615605' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:52:05.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:05 smithi018 ceph-mon[40196]: 6.11 scrub starts 2024-09-10T18:52:05.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:05 smithi018 ceph-mon[40196]: 6.11 scrub ok 2024-09-10T18:52:05.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:05 smithi018 ceph-mon[40196]: pgmap v281: 158 pgs: 158 active+clean; 2.1 GiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 852 B/s rd, 138 MiB/s wr, 2.22k op/s 2024-09-10T18:52:05.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:05 smithi018 ceph-mon[29336]: 1.a scrub starts 2024-09-10T18:52:05.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:05 smithi018 ceph-mon[29336]: 4.2 scrub starts 2024-09-10T18:52:05.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:05 smithi018 ceph-mon[29336]: 1.a scrub ok 2024-09-10T18:52:05.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:05 smithi018 ceph-mon[29336]: 4.2 scrub ok 2024-09-10T18:52:05.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:05 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2236615605' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:52:05.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:05 smithi018 ceph-mon[29336]: 6.11 scrub starts 2024-09-10T18:52:05.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:05 smithi018 ceph-mon[29336]: 6.11 scrub ok 2024-09-10T18:52:05.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:05 smithi018 ceph-mon[29336]: pgmap v281: 158 pgs: 158 active+clean; 2.1 GiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 852 B/s rd, 138 MiB/s wr, 2.22k op/s 2024-09-10T18:52:06.013 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:06.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:05 smithi103 ceph-mon[33163]: 1.a scrub starts 2024-09-10T18:52:06.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:05 smithi103 ceph-mon[33163]: 4.2 scrub starts 2024-09-10T18:52:06.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:05 smithi103 ceph-mon[33163]: 1.a scrub ok 2024-09-10T18:52:06.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:05 smithi103 ceph-mon[33163]: 4.2 scrub ok 2024-09-10T18:52:06.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:05 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2236615605' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:52:06.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:05 smithi103 ceph-mon[33163]: 6.11 scrub starts 2024-09-10T18:52:06.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:05 smithi103 ceph-mon[33163]: 6.11 scrub ok 2024-09-10T18:52:06.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:05 smithi103 ceph-mon[33163]: pgmap v281: 158 pgs: 158 active+clean; 2.1 GiB data, 8.0 GiB used, 618 GiB / 626 GiB avail; 852 B/s rd, 138 MiB/s wr, 2.22k op/s 2024-09-10T18:52:06.051 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:06.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:06.052 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:06.122 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:52:06.153 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:06.186 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:06.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:06.187 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:06.194 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 17 16 44322 44306 162.863 126.25 0.00731407 0.00613466 2024-09-10T18:52:06.288 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:06.329 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:06.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:06.330 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:06.431 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:06.467 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:06.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:06.468 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:06.569 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:06.602 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:06.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:06.603 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:06.704 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:06.710 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:52:06 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:52:06.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:06.746 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:06.746 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:06.847 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:06.890 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:06.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:06.891 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:06.939 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:06 smithi018 ceph-mon[40196]: 2.10 scrub starts 2024-09-10T18:52:06.939 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:06 smithi018 ceph-mon[40196]: 2.10 scrub ok 2024-09-10T18:52:06.939 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:06 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:06.939 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:06 smithi018 ceph-mon[29336]: 2.10 scrub starts 2024-09-10T18:52:06.939 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:06 smithi018 ceph-mon[29336]: 2.10 scrub ok 2024-09-10T18:52:06.939 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:06 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:06.992 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:07.024 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:06 smithi103 ceph-mon[33163]: 2.10 scrub starts 2024-09-10T18:52:07.024 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:06 smithi103 ceph-mon[33163]: 2.10 scrub ok 2024-09-10T18:52:07.024 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:06 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:07.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:07.031 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:07.031 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:07.132 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:07.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:07.169 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:07.169 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:07.195 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 18 16 46264 46248 160.557 121.375 0.0110934 0.00622338 2024-09-10T18:52:07.270 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:07.303 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:07.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:07.304 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:07.405 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:07.444 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:07.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:07.445 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:07.546 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:07.583 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:07.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:07.585 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:07.685 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:07.718 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:07.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:07.719 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:07.726 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is set 2024-09-10T18:52:07.797 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:07 smithi018 ceph-mon[40196]: 1.8 scrub starts 2024-09-10T18:52:07.797 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:07 smithi018 ceph-mon[40196]: 4.13 scrub starts 2024-09-10T18:52:07.797 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:07 smithi018 ceph-mon[40196]: 1.8 scrub ok 2024-09-10T18:52:07.797 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:07 smithi018 ceph-mon[40196]: 4.13 scrub ok 2024-09-10T18:52:07.797 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:07 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3224804158' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:52:07.797 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:07 smithi018 ceph-mon[40196]: pgmap v282: 158 pgs: 158 active+clean; 2.7 GiB data, 10 GiB used, 616 GiB / 626 GiB avail; 1.2 KiB/s rd, 191 MiB/s wr, 3.06k op/s 2024-09-10T18:52:07.797 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:07 smithi018 ceph-mon[29336]: 1.8 scrub starts 2024-09-10T18:52:07.797 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:07 smithi018 ceph-mon[29336]: 4.13 scrub starts 2024-09-10T18:52:07.797 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:07 smithi018 ceph-mon[29336]: 1.8 scrub ok 2024-09-10T18:52:07.797 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:07 smithi018 ceph-mon[29336]: 4.13 scrub ok 2024-09-10T18:52:07.798 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:07 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3224804158' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:52:07.798 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:07 smithi018 ceph-mon[29336]: pgmap v282: 158 pgs: 158 active+clean; 2.7 GiB data, 10 GiB used, 616 GiB / 626 GiB avail; 1.2 KiB/s rd, 191 MiB/s wr, 3.06k op/s 2024-09-10T18:52:07.820 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:07.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:07.859 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:07.859 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:07.960 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:07.994 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:07.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:07.995 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:08.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:07 smithi103 ceph-mon[33163]: 1.8 scrub starts 2024-09-10T18:52:08.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:07 smithi103 ceph-mon[33163]: 4.13 scrub starts 2024-09-10T18:52:08.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:07 smithi103 ceph-mon[33163]: 1.8 scrub ok 2024-09-10T18:52:08.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:07 smithi103 ceph-mon[33163]: 4.13 scrub ok 2024-09-10T18:52:08.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:07 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3224804158' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:52:08.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:07 smithi103 ceph-mon[33163]: pgmap v282: 158 pgs: 158 active+clean; 2.7 GiB data, 10 GiB used, 616 GiB / 626 GiB avail; 1.2 KiB/s rd, 191 MiB/s wr, 3.06k op/s 2024-09-10T18:52:08.096 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:08.130 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:08.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:08.131 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:08.141 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T18:52:08.195 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 19 16 48221 48205 158.54 122.312 0.00784667 0.00630243 2024-09-10T18:52:08.232 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:08.266 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:08.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:08.267 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:08.368 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:08.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:08.403 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:08.403 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:08.504 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:08.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:08.540 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:08.540 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:08.641 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:08.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:08.729 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:08.729 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:08.830 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:08.873 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:08.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:08.874 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:08.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:08 smithi018 ceph-mon[40196]: 4.18 scrub starts 2024-09-10T18:52:08.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:08 smithi018 ceph-mon[40196]: 4.18 scrub ok 2024-09-10T18:52:08.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:08 smithi018 ceph-mon[40196]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:52:08.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:08 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3224804158' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:52:08.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:08 smithi018 ceph-mon[40196]: osdmap e164: 8 total, 8 up, 7 in 2024-09-10T18:52:08.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:08 smithi018 ceph-mon[40196]: osdmap e165: 8 total, 8 up, 7 in 2024-09-10T18:52:08.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:08 smithi018 ceph-mon[29336]: 4.18 scrub starts 2024-09-10T18:52:08.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:08 smithi018 ceph-mon[29336]: 4.18 scrub ok 2024-09-10T18:52:08.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:08 smithi018 ceph-mon[29336]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:52:08.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:08 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3224804158' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:52:08.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:08 smithi018 ceph-mon[29336]: osdmap e164: 8 total, 8 up, 7 in 2024-09-10T18:52:08.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:08 smithi018 ceph-mon[29336]: osdmap e165: 8 total, 8 up, 7 in 2024-09-10T18:52:08.975 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:09.010 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:09.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:09.011 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:09.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:08 smithi103 ceph-mon[33163]: 4.18 scrub starts 2024-09-10T18:52:09.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:08 smithi103 ceph-mon[33163]: 4.18 scrub ok 2024-09-10T18:52:09.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:08 smithi103 ceph-mon[33163]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:52:09.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:08 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3224804158' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:52:09.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:08 smithi103 ceph-mon[33163]: osdmap e164: 8 total, 8 up, 7 in 2024-09-10T18:52:09.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:08 smithi103 ceph-mon[33163]: osdmap e165: 8 total, 8 up, 7 in 2024-09-10T18:52:09.112 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:09.144 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:09.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:09.145 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:09.195 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:2024-09-10T18:52:09.195157+0000 min lat: 0.00115328 max lat: 0.102006 avg lat: 0.00639032 2024-09-10T18:52:09.195 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T18:52:09.196 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 20 16 50063 50047 156.369 115.125 0.00696646 0.00639032 2024-09-10T18:52:09.246 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:09.279 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:09.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:09.280 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:09.381 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:09.415 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:09.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:09.417 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:09.517 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:09.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:09.561 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:09.561 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:09.662 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:09.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:09.696 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:09.696 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:09.797 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:09.829 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:09.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:09.830 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:09.866 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:09 smithi018 ceph-mon[40196]: pgmap v285: 158 pgs: 158 active+clean; 2.7 GiB data, 10 GiB used, 616 GiB / 626 GiB avail; 1.2 KiB/s rd, 136 MiB/s wr, 2.18k op/s 2024-09-10T18:52:09.867 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:09 smithi018 ceph-mon[29336]: pgmap v285: 158 pgs: 158 active+clean; 2.7 GiB data, 10 GiB used, 616 GiB / 626 GiB avail; 1.2 KiB/s rd, 136 MiB/s wr, 2.18k op/s 2024-09-10T18:52:09.931 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:09.965 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:09.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:09.967 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:10.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:09 smithi103 ceph-mon[33163]: pgmap v285: 158 pgs: 158 active+clean; 2.7 GiB data, 10 GiB used, 616 GiB / 626 GiB avail; 1.2 KiB/s rd, 136 MiB/s wr, 2.18k op/s 2024-09-10T18:52:10.067 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:10.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:10.120 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:10.120 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:10.196 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 21 16 51887 51871 154.349 114 0.0111164 0.00647395 2024-09-10T18:52:10.221 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:10.257 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:10.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:10.258 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:10.358 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:10.380 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:10.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:10.381 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:10.482 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:10.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:10.519 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:10.519 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:10.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:52:10 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:52:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:52:10.620 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:10.654 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:10.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:10.655 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:10.755 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:10.796 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:10.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:10.798 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:10.899 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:10.932 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:10.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:10.933 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:11.002 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:52:11.034 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:11.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:11.083 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:11.084 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:11.138 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:11 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:52:11.139 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:11 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:52:11.139 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:11 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:52:11.139 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:11 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:52:11.139 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:52:10 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:52:10] "GET /metrics HTTP/1.1" 200 223857 "" "Prometheus/2.43.0" 2024-09-10T18:52:11.185 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:11.196 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 22 16 53571 53555 152.117 105.25 0.00779355 0.00656869 2024-09-10T18:52:11.215 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:11.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:11.216 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:11.272 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-10T18:52:11.317 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:11.361 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:11.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:11.362 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:11.463 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:11.504 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:11.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:11.505 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:11.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:11 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:52:11.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:11 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:52:11.606 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:11.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:11.642 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:11.643 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:11.743 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:11.777 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:11.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:11.779 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:11.879 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:11.919 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:11.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:11.920 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:12.021 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:12.056 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:12.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:12.057 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:12.089 INFO:teuthology.orchestra.run.smithi018.stderr:set pool 1 pg_num to 21 2024-09-10T18:52:12.158 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:12.195 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:12.196 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 23 16 55258 55242 150.086 105.438 0.0095334 0.00665717 2024-09-10T18:52:12.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:12.197 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:12.298 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:12.329 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:12.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:12.330 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:12.431 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:12.433 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:12 smithi018 ceph-mon[40196]: pgmap v286: 158 pgs: 158 active+clean; 2.9 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 639 B/s rd, 106 MiB/s wr, 1.70k op/s 2024-09-10T18:52:12.433 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:12 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1216057803' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"}]: dispatch 2024-09-10T18:52:12.433 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:12 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"}]: dispatch 2024-09-10T18:52:12.433 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:12 smithi018 ceph-mon[29336]: pgmap v286: 158 pgs: 158 active+clean; 2.9 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 639 B/s rd, 106 MiB/s wr, 1.70k op/s 2024-09-10T18:52:12.433 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:12 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1216057803' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"}]: dispatch 2024-09-10T18:52:12.433 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:12 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"}]: dispatch 2024-09-10T18:52:12.466 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:12.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:12.467 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:12.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:12 smithi103 ceph-mon[33163]: pgmap v286: 158 pgs: 158 active+clean; 2.9 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 639 B/s rd, 106 MiB/s wr, 1.70k op/s 2024-09-10T18:52:12.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:12 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1216057803' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"}]: dispatch 2024-09-10T18:52:12.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:12 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"}]: dispatch 2024-09-10T18:52:12.568 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:12.602 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:12.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:12.603 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:12.703 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:12.738 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:12.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:12.740 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:12.840 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:12.887 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:12.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:12.888 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:12.989 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:13.025 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:13.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:13.026 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:13.127 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:13.164 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:13.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:13.166 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:13.196 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 24 16 57025 57009 148.434 110.438 0.0063995 0.00673232 2024-09-10T18:52:13.267 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:13.307 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:13.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:13.308 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:13.390 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:13 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"}]': finished 2024-09-10T18:52:13.391 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:13 smithi018 ceph-mon[40196]: osdmap e166: 8 total, 8 up, 7 in 2024-09-10T18:52:13.391 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:13 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"}]': finished 2024-09-10T18:52:13.391 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:13 smithi018 ceph-mon[29336]: osdmap e166: 8 total, 8 up, 7 in 2024-09-10T18:52:13.409 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:13.445 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:13.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:13.446 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:13.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:13 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"}]': finished 2024-09-10T18:52:13.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:13 smithi103 ceph-mon[33163]: osdmap e166: 8 total, 8 up, 7 in 2024-09-10T18:52:13.548 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:13.585 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:13.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:13.586 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:13.687 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:13.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:13.725 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:13.725 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:13.748 INFO:teuthology.orchestra.run.smithi018.stdout:ERROR: (22) Invalid argument 2024-09-10T18:52:13.749 INFO:teuthology.orchestra.run.smithi018.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:52:13.825 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:13.859 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:13.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:13.860 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:13.961 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:14.003 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:14.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:14.004 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:14.105 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:14.140 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:14.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:14.141 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:14.196 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 25 16 58678 58662 146.629 103.312 0.00915046 0.0068142 2024-09-10T18:52:14.242 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:14.280 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:14.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:14.281 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:14.382 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:14.414 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:14.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:14.415 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:14.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:14 smithi103 ceph-mon[33163]: pgmap v288: 158 pgs: 158 active+clean; 3.3 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 853 B/s rd, 96 MiB/s wr, 1.54k op/s 2024-09-10T18:52:14.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:14 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"}]: dispatch 2024-09-10T18:52:14.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:14 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"}]: dispatch 2024-09-10T18:52:14.424 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:14 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"}]': finished 2024-09-10T18:52:14.424 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:14 smithi103 ceph-mon[33163]: osdmap e167: 8 total, 8 up, 7 in 2024-09-10T18:52:14.433 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:52:14.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:14 smithi018 ceph-mon[40196]: pgmap v288: 158 pgs: 158 active+clean; 3.3 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 853 B/s rd, 96 MiB/s wr, 1.54k op/s 2024-09-10T18:52:14.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:14 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"}]: dispatch 2024-09-10T18:52:14.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:14 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"}]: dispatch 2024-09-10T18:52:14.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:14 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"}]': finished 2024-09-10T18:52:14.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:14 smithi018 ceph-mon[40196]: osdmap e167: 8 total, 8 up, 7 in 2024-09-10T18:52:14.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:14 smithi018 ceph-mon[29336]: pgmap v288: 158 pgs: 158 active+clean; 3.3 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 853 B/s rd, 96 MiB/s wr, 1.54k op/s 2024-09-10T18:52:14.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:14 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"}]: dispatch 2024-09-10T18:52:14.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:14 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"}]: dispatch 2024-09-10T18:52:14.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:14 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"}]': finished 2024-09-10T18:52:14.447 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:14 smithi018 ceph-mon[29336]: osdmap e167: 8 total, 8 up, 7 in 2024-09-10T18:52:14.516 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:14.558 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:14.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:14.559 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:14.660 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:14.699 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:14.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:14.700 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:14.801 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:14.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:14.837 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:14.837 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:14.938 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:14.974 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:14.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:14.975 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:15.028 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-10T18:52:15.076 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:15.112 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:15.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:15.113 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:15.196 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 26 16 60340 60324 144.984 103.875 0.00860521 0.00689185 2024-09-10T18:52:15.214 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:15.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:15.253 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:15.254 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:15.354 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:15.398 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:15.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:15.400 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:15.500 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:15.545 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:15.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:15.547 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:15.648 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:15.682 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:15.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:15.683 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:15.783 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:15.816 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:15.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:15.817 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:15.886 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:15 smithi018 ceph-mon[40196]: osdmap e168: 8 total, 8 up, 7 in 2024-09-10T18:52:15.887 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:15 smithi018 ceph-mon[40196]: pgmap v291: 168 pgs: 10 unknown, 158 active+clean; 3.3 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 853 B/s rd, 96 MiB/s wr, 1.54k op/s 2024-09-10T18:52:15.887 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:15 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"}]: dispatch 2024-09-10T18:52:15.887 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:15 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"}]: dispatch 2024-09-10T18:52:15.887 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:15 smithi018 ceph-mon[29336]: osdmap e168: 8 total, 8 up, 7 in 2024-09-10T18:52:15.887 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:15 smithi018 ceph-mon[29336]: pgmap v291: 168 pgs: 10 unknown, 158 active+clean; 3.3 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 853 B/s rd, 96 MiB/s wr, 1.54k op/s 2024-09-10T18:52:15.887 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:15 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"}]: dispatch 2024-09-10T18:52:15.887 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:15 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"}]: dispatch 2024-09-10T18:52:15.918 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:15.943 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:52:15.943 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:52:15.949 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:15.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:15.950 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:15.976 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:52:15.976 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:52:16.008 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:52:16.009 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:52:16.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:15 smithi103 ceph-mon[33163]: osdmap e168: 8 total, 8 up, 7 in 2024-09-10T18:52:16.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:15 smithi103 ceph-mon[33163]: pgmap v291: 168 pgs: 10 unknown, 158 active+clean; 3.3 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 853 B/s rd, 96 MiB/s wr, 1.54k op/s 2024-09-10T18:52:16.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:15 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"}]: dispatch 2024-09-10T18:52:16.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:15 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"}]: dispatch 2024-09-10T18:52:16.044 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:52:16.045 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:52:16.051 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:16.072 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:52:16.073 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:52:16.081 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:16.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:16.082 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:16.109 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:52:16.109 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:52:16.143 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:52:16.143 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:52:16.181 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:52:16.181 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:52:16.183 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:16.196 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 27 16 61940 61924 143.318 100 0.00745771 0.00697244 2024-09-10T18:52:16.215 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:16.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:16.216 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:16.317 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:16.352 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:16.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:16.353 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:16.454 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:16.485 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:16.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:16.486 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:16.586 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:16.625 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:16.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:16.626 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:16.727 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:16.756 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:16.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:16.757 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:16.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:16 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"}]': finished 2024-09-10T18:52:16.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:16 smithi103 ceph-mon[33163]: osdmap e169: 8 total, 8 up, 7 in 2024-09-10T18:52:16.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:16 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/207251861' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:52:16.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:16 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:16.773 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:52:16 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:52:16.858 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:16.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:16.896 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:16.896 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:16.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:16 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"}]': finished 2024-09-10T18:52:16.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:16 smithi018 ceph-mon[40196]: osdmap e169: 8 total, 8 up, 7 in 2024-09-10T18:52:16.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:16 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/207251861' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:52:16.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:16 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:16.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:16 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"}]': finished 2024-09-10T18:52:16.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:16 smithi018 ceph-mon[29336]: osdmap e169: 8 total, 8 up, 7 in 2024-09-10T18:52:16.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:16 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/207251861' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:52:16.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:16 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:16.998 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:17.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:17.031 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:17.031 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:17.132 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:17.170 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:17.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:17.171 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:17.196 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 28 16 63504 63488 141.691 97.75 0.00690668 0.00705239 2024-09-10T18:52:17.272 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:17.310 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:17.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:17.311 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:17.412 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:17.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:17.448 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:17.449 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:17.549 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:17.586 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:17.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:17.587 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:17.688 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:17.723 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:17.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:17.724 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:17.825 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:17.863 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:17.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:17.864 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:17.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:17 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:52:17.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:17 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:52:17.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:17 smithi018 ceph-mon[40196]: osdmap e170: 8 total, 8 up, 7 in 2024-09-10T18:52:17.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:17 smithi018 ceph-mon[40196]: pgmap v294: 168 pgs: 10 peering, 158 active+clean; 3.8 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 5.5 KiB/s rd, 130 MiB/s wr, 2.09k op/s; 22 KiB/s, 0 objects/s recovering 2024-09-10T18:52:17.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:17 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:52:17.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:17 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:52:17.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:17 smithi018 ceph-mon[29336]: osdmap e170: 8 total, 8 up, 7 in 2024-09-10T18:52:17.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:17 smithi018 ceph-mon[29336]: pgmap v294: 168 pgs: 10 peering, 158 active+clean; 3.8 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 5.5 KiB/s rd, 130 MiB/s wr, 2.09k op/s; 22 KiB/s, 0 objects/s recovering 2024-09-10T18:52:17.965 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:18.001 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:18.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:18.002 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:18.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:17 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:52:18.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:17 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:52:18.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:17 smithi103 ceph-mon[33163]: osdmap e170: 8 total, 8 up, 7 in 2024-09-10T18:52:18.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:17 smithi103 ceph-mon[33163]: pgmap v294: 168 pgs: 10 peering, 158 active+clean; 3.8 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 5.5 KiB/s rd, 130 MiB/s wr, 2.09k op/s; 22 KiB/s, 0 objects/s recovering 2024-09-10T18:52:18.103 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:18.143 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:18.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:18.144 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:18.196 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 29 16 64902 64886 139.817 87.375 0.0177104 0.00714621 2024-09-10T18:52:18.245 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:18.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:18.280 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:18.281 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:18.381 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:18.420 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:18.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:18.421 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:18.522 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:18.563 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:18.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:18.564 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:18.665 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:18.697 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:18.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:18.698 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:18.799 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:18.830 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:18.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:18.831 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:18.932 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:18.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:18 smithi018 ceph-mon[40196]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-09-10T18:52:18.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:18 smithi018 ceph-mon[40196]: osdmap e171: 8 total, 8 up, 7 in 2024-09-10T18:52:18.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:18 smithi018 ceph-mon[29336]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-09-10T18:52:18.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:18 smithi018 ceph-mon[29336]: osdmap e171: 8 total, 8 up, 7 in 2024-09-10T18:52:18.965 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:18.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:18.966 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:19.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:18 smithi103 ceph-mon[33163]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-09-10T18:52:19.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:18 smithi103 ceph-mon[33163]: osdmap e171: 8 total, 8 up, 7 in 2024-09-10T18:52:19.067 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:19.088 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:19.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:19.090 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:19.177 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-10T18:52:19.177 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T18:52:19.178 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T18:52:19.190 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:19.197 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 30 16 66181 66165 137.82 79.9375 0.00971528 0.00725059 2024-09-10T18:52:19.217 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:19.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:19.218 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:19.319 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:19.348 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:19.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:19.350 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:19.450 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:19.485 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:19.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:19.486 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:19.587 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:19.622 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:19.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:19.624 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:19.725 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:19.761 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:19.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:19.762 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:19.863 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:19.904 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:19.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:19.905 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:19.926 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:19 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:52:19.926 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:19 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:52:19.926 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:19 smithi018 ceph-mon[40196]: pgmap v296: 168 pgs: 10 peering, 158 active+clean; 3.8 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 4.8 KiB/s rd, 114 MiB/s wr, 1.83k op/s; 19 KiB/s, 0 objects/s recovering 2024-09-10T18:52:19.926 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:19 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:52:19.926 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:19 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:52:19.926 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:19 smithi018 ceph-mon[29336]: pgmap v296: 168 pgs: 10 peering, 158 active+clean; 3.8 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 4.8 KiB/s rd, 114 MiB/s wr, 1.83k op/s; 19 KiB/s, 0 objects/s recovering 2024-09-10T18:52:20.006 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:20.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:19 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:52:20.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:19 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:52:20.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:19 smithi103 ceph-mon[33163]: pgmap v296: 168 pgs: 10 peering, 158 active+clean; 3.8 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 4.8 KiB/s rd, 114 MiB/s wr, 1.83k op/s; 19 KiB/s, 0 objects/s recovering 2024-09-10T18:52:20.041 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:20.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:20.042 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:20.142 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:20.173 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:20.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:20.174 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:20.197 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 31 16 67353 67337 135.737 73.25 0.00956923 0.00736202 2024-09-10T18:52:20.275 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:20.312 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:20.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:20.313 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:20.414 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:20.449 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:20.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:20.450 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:20.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:52:20 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:52:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:52:20.551 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:20.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:20.589 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:20.589 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:20.690 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:20.724 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:20.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:20.725 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:20.826 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:20.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:20.865 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:20.865 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:20.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:20 smithi018 ceph-mon[40196]: osdmap e172: 8 total, 8 up, 7 in 2024-09-10T18:52:20.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:20 smithi018 ceph-mon[29336]: osdmap e172: 8 total, 8 up, 7 in 2024-09-10T18:52:20.966 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:21.003 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:21.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:21.003 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:21.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:20 smithi103 ceph-mon[33163]: osdmap e172: 8 total, 8 up, 7 in 2024-09-10T18:52:21.105 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:21.139 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:21.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:21.140 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:21.197 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 32 16 68834 68818 134.388 92.5625 0.0162016 0.00743621 2024-09-10T18:52:21.240 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:21.281 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:21.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:21.282 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:21.339 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:52:20 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:52:20] "GET /metrics HTTP/1.1" 200 223857 "" "Prometheus/2.43.0" 2024-09-10T18:52:21.383 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:21.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:21.422 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:21.422 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:21.523 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:21.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:21.564 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:21.564 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:21.665 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:21.697 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:21.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:21.698 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:21.798 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:21.839 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:21.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:21.840 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:21.940 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:21.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:21 smithi018 ceph-mon[40196]: pgmap v298: 168 pgs: 4 peering, 164 active+clean; 3.9 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 9.3 KiB/s rd, 122 MiB/s wr, 1.97k op/s; 44 KiB/s, 0 objects/s recovering 2024-09-10T18:52:21.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:21 smithi018 ceph-mon[29336]: pgmap v298: 168 pgs: 4 peering, 164 active+clean; 3.9 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 9.3 KiB/s rd, 122 MiB/s wr, 1.97k op/s; 44 KiB/s, 0 objects/s recovering 2024-09-10T18:52:21.978 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:21.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:21.979 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:22.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:21 smithi103 ceph-mon[33163]: pgmap v298: 168 pgs: 4 peering, 164 active+clean; 3.9 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 9.3 KiB/s rd, 122 MiB/s wr, 1.97k op/s; 44 KiB/s, 0 objects/s recovering 2024-09-10T18:52:22.080 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:22.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:22.116 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:22.116 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:22.197 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 33 16 70316 70300 133.122 92.625 0.00385385 0.00750194 2024-09-10T18:52:22.217 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:22.257 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:22.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:22.258 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:22.359 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:22.399 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:22.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:22.401 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:22.501 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:22.541 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:22.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:22.543 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:22.643 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:22.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:22.693 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:22.693 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:22.794 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:22.832 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:22.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:22.833 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:22.906 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:52:22.909 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:52:22.919 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:22 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:52:22.920 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:22 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:52:22.934 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:23.012 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:23.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:23.013 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:23.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:22 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:52:23.113 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:23.143 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:23.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:23.144 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:23.197 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 34 16 71603 71587 131.573 80.4375 0.0092823 0.00759501 2024-09-10T18:52:23.245 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:23.281 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:23.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:23.282 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:23.383 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:23.421 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:23.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:23.422 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:23.523 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:23.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:23.554 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:23.554 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:23.655 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:23.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:23.692 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:23.693 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:23.785 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:23 smithi018 ceph-mon[40196]: pgmap v299: 168 pgs: 168 active+clean; 4.2 GiB data, 14 GiB used, 611 GiB / 626 GiB avail; 5.3 KiB/s rd, 66 MiB/s wr, 1.05k op/s; 73 KiB/s, 0 objects/s recovering 2024-09-10T18:52:23.785 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:23 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:23.785 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:23 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:52:23.785 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:23 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:52:23.785 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:23 smithi018 ceph-mon[40196]: Health check update: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T18:52:23.785 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:23 smithi018 ceph-mon[40196]: osdmap e173: 8 total, 8 up, 7 in 2024-09-10T18:52:23.785 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:23 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/363330721' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:52:23.786 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:23 smithi018 ceph-mon[29336]: pgmap v299: 168 pgs: 168 active+clean; 4.2 GiB data, 14 GiB used, 611 GiB / 626 GiB avail; 5.3 KiB/s rd, 66 MiB/s wr, 1.05k op/s; 73 KiB/s, 0 objects/s recovering 2024-09-10T18:52:23.786 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:23 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:23.786 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:23 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:52:23.786 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:23 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:52:23.786 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:23 smithi018 ceph-mon[29336]: Health check update: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T18:52:23.786 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:23 smithi018 ceph-mon[29336]: osdmap e173: 8 total, 8 up, 7 in 2024-09-10T18:52:23.786 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:23 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/363330721' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:52:23.793 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:23.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:23.828 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:23.828 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:23.929 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:23.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:23.966 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:23.966 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:24.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:23 smithi103 ceph-mon[33163]: pgmap v299: 168 pgs: 168 active+clean; 4.2 GiB data, 14 GiB used, 611 GiB / 626 GiB avail; 5.3 KiB/s rd, 66 MiB/s wr, 1.05k op/s; 73 KiB/s, 0 objects/s recovering 2024-09-10T18:52:24.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:23 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:24.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:23 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:52:24.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:23 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:52:24.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:23 smithi103 ceph-mon[33163]: Health check update: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T18:52:24.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:23 smithi103 ceph-mon[33163]: osdmap e173: 8 total, 8 up, 7 in 2024-09-10T18:52:24.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:23 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/363330721' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:52:24.066 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:24.105 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:24.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:24.106 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:24.198 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 35 16 73037 73021 130.372 89.625 0.0105556 0.0076653 2024-09-10T18:52:24.206 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:24.240 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:24.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:24.241 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:24.342 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:24.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:24.382 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:24.382 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:24.482 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:24.517 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:24.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:24.518 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:24.571 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is set 2024-09-10T18:52:24.619 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:24.634 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T18:52:24.655 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:24.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:24.656 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:24.757 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:24.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:24.806 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:24.807 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:24.907 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:24.942 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:24.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:24.943 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:24.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:24 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:52:24.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:24 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:52:24.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:24 smithi018 ceph-mon[40196]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T18:52:24.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:24 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:52:24.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:24 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:52:24.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:24 smithi018 ceph-mon[40196]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:52:24.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:24 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/363330721' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:52:24.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:24 smithi018 ceph-mon[40196]: osdmap e174: 8 total, 8 up, 7 in 2024-09-10T18:52:24.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:24 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:52:24.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:24 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:52:24.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:24 smithi018 ceph-mon[29336]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T18:52:24.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:24 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:52:24.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:24 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:52:24.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:24 smithi018 ceph-mon[29336]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:52:24.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:24 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/363330721' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:52:24.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:24 smithi018 ceph-mon[29336]: osdmap e174: 8 total, 8 up, 7 in 2024-09-10T18:52:25.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:24 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:52:25.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:24 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:52:25.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:24 smithi103 ceph-mon[33163]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T18:52:25.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:24 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:52:25.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:24 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:52:25.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:24 smithi103 ceph-mon[33163]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:52:25.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:24 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/363330721' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:52:25.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:24 smithi103 ceph-mon[33163]: osdmap e174: 8 total, 8 up, 7 in 2024-09-10T18:52:25.044 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:25.089 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:25.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:25.091 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:25.191 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:25.198 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 36 16 74498 74482 129.287 91.3125 0.0113446 0.0077294 2024-09-10T18:52:25.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:25.224 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:25.225 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:25.325 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:25.370 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:25.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:25.371 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:25.472 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:25.505 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:25.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:25.506 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:25.607 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:25.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:25.653 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:25.654 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:25.755 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:25.789 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:25.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:25.790 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:25.847 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:25 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:52:25.847 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:25 smithi018 ceph-mon[29336]: pgmap v302: 168 pgs: 168 active+clean; 4.2 GiB data, 14 GiB used, 611 GiB / 626 GiB avail; 5.8 KiB/s rd, 72 MiB/s wr, 1.15k op/s; 80 KiB/s, 0 objects/s recovering 2024-09-10T18:52:25.847 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:25 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:52:25.847 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:25 smithi018 ceph-mon[40196]: pgmap v302: 168 pgs: 168 active+clean; 4.2 GiB data, 14 GiB used, 611 GiB / 626 GiB avail; 5.8 KiB/s rd, 72 MiB/s wr, 1.15k op/s; 80 KiB/s, 0 objects/s recovering 2024-09-10T18:52:25.891 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:25.914 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:25.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:25.915 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:26.016 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:26.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:25 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:52:26.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:25 smithi103 ceph-mon[33163]: pgmap v302: 168 pgs: 168 active+clean; 4.2 GiB data, 14 GiB used, 611 GiB / 626 GiB avail; 5.8 KiB/s rd, 72 MiB/s wr, 1.15k op/s; 80 KiB/s, 0 objects/s recovering 2024-09-10T18:52:26.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:26.049 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:26.049 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:26.150 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:26.188 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:26.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:26.189 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:26.198 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 37 16 75997 75981 128.325 93.6875 0.008898 0.00778775 2024-09-10T18:52:26.290 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:26.327 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:26.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:26.328 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:26.429 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:26.458 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:26.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:26.459 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:26.560 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:26.587 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:26.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:26.589 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:26.592 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T18:52:26.592 INFO:teuthology.orchestra.run.smithi018.stdout: "ops": [ 2024-09-10T18:52:26.592 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.592 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.24947.0:76573 8.f 8:f44525b1:::benchmark_data_smithi018_106531_object76572:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e174)", 2024-09-10T18:52:26.592 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:52:26.576883+0000", 2024-09-10T18:52:26.592 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 0.010042175, 2024-09-10T18:52:26.592 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.010080545, 2024-09-10T18:52:26.592 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:52:26.592 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "waiting for sub ops", 2024-09-10T18:52:26.593 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:52:26.593 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.24947", 2024-09-10T18:52:26.593 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:52:26.593 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 76573 2024-09-10T18:52:26.593 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.593 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:52:26.593 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.593 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:52:26.593 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.576883+0000", 2024-09-10T18:52:26.593 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:26.593 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.593 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.593 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:52:26.594 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.576883+0000", 2024-09-10T18:52:26.594 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:26.594 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.594 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.594 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:52:26.594 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.576882+0000", 2024-09-10T18:52:26.594 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:52:26.594 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.594 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.594 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:52:26.594 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.576922+0000", 2024-09-10T18:52:26.594 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.9638000000000002e-05 2024-09-10T18:52:26.595 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.595 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.595 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:52:26.595 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.576936+0000", 2024-09-10T18:52:26.595 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.3551e-05 2024-09-10T18:52:26.595 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.595 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.595 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:52:26.595 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.576943+0000", 2024-09-10T18:52:26.595 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.0119999999999999e-06 2024-09-10T18:52:26.595 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.595 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.596 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:52:26.596 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.580930+0000", 2024-09-10T18:52:26.596 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0039873110000000003 2024-09-10T18:52:26.596 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.596 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.596 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:52:26.596 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.581067+0000", 2024-09-10T18:52:26.596 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00013687700000000001 2024-09-10T18:52:26.596 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.596 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.596 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,4", 2024-09-10T18:52:26.597 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.581213+0000", 2024-09-10T18:52:26.597 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00014622800000000001 2024-09-10T18:52:26.597 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.597 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.597 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:52:26.597 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.585766+0000", 2024-09-10T18:52:26.597 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0045533680000000003 2024-09-10T18:52:26.597 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.597 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.597 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:52:26.597 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.585967+0000", 2024-09-10T18:52:26.597 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00020085200000000001 2024-09-10T18:52:26.598 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:26.598 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:52:26.598 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:26.598 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.598 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.598 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.24947.0:76576 8.f 8:f4017626:::benchmark_data_smithi018_106531_object76575:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e174)", 2024-09-10T18:52:26.598 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:52:26.576998+0000", 2024-09-10T18:52:26.598 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 0.0099273250000000007, 2024-09-10T18:52:26.598 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.010104228, 2024-09-10T18:52:26.598 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:52:26.598 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "waiting for sub ops", 2024-09-10T18:52:26.598 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:52:26.598 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.24947", 2024-09-10T18:52:26.599 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:52:26.599 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 76576 2024-09-10T18:52:26.599 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.599 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:52:26.599 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.599 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:52:26.599 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.576998+0000", 2024-09-10T18:52:26.599 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:26.599 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.599 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.599 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:52:26.599 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.576998+0000", 2024-09-10T18:52:26.599 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:26.599 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.600 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.600 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:52:26.600 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.576998+0000", 2024-09-10T18:52:26.600 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T18:52:26.600 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.600 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.600 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:52:26.600 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.577048+0000", 2024-09-10T18:52:26.600 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.028e-05 2024-09-10T18:52:26.600 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.600 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.600 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:52:26.600 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.577055+0000", 2024-09-10T18:52:26.601 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.3479999999999998e-06 2024-09-10T18:52:26.601 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.601 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.601 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:52:26.601 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.577060+0000", 2024-09-10T18:52:26.601 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.87e-06 2024-09-10T18:52:26.601 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.601 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.601 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:52:26.601 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.583425+0000", 2024-09-10T18:52:26.601 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0063645059999999998 2024-09-10T18:52:26.601 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.601 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.602 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:52:26.602 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.583541+0000", 2024-09-10T18:52:26.602 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000116395 2024-09-10T18:52:26.602 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.602 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.602 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,4", 2024-09-10T18:52:26.602 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.583674+0000", 2024-09-10T18:52:26.602 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00013297600000000001 2024-09-10T18:52:26.602 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.602 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.602 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:52:26.602 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.586703+0000", 2024-09-10T18:52:26.602 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0030287280000000001 2024-09-10T18:52:26.603 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:26.603 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:52:26.603 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:26.603 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.603 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.603 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.24947.0:76579 8.f 8:f2914ed2:::benchmark_data_smithi018_106531_object76578:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e174)", 2024-09-10T18:52:26.603 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:52:26.583232+0000", 2024-09-10T18:52:26.603 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 0.0036934720000000002, 2024-09-10T18:52:26.603 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0039614380000000003, 2024-09-10T18:52:26.603 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:52:26.603 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "waiting for sub ops", 2024-09-10T18:52:26.603 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:52:26.603 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.24947", 2024-09-10T18:52:26.603 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:52:26.604 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 76579 2024-09-10T18:52:26.604 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.604 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:52:26.604 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.604 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:52:26.604 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.583232+0000", 2024-09-10T18:52:26.604 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:26.604 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.604 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.604 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:52:26.604 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.583232+0000", 2024-09-10T18:52:26.604 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:26.604 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.605 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.605 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:52:26.605 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.583231+0000", 2024-09-10T18:52:26.605 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T18:52:26.605 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.605 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.605 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:52:26.605 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.583255+0000", 2024-09-10T18:52:26.605 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.4632999999999999e-05 2024-09-10T18:52:26.605 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.605 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.605 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:52:26.606 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.583267+0000", 2024-09-10T18:52:26.606 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.222e-05 2024-09-10T18:52:26.606 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.606 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.606 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:52:26.606 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.583273+0000", 2024-09-10T18:52:26.606 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.7790000000000001e-06 2024-09-10T18:52:26.606 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.606 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.606 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:52:26.606 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.586726+0000", 2024-09-10T18:52:26.606 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.003452353 2024-09-10T18:52:26.606 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.606 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.607 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:52:26.607 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.586836+0000", 2024-09-10T18:52:26.607 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00011010000000000001 2024-09-10T18:52:26.607 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.607 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.607 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,4", 2024-09-10T18:52:26.607 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.586964+0000", 2024-09-10T18:52:26.607 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00012854699999999999 2024-09-10T18:52:26.607 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:26.607 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:52:26.607 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:26.607 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.607 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.608 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.24947.0:76578 8.a e174/158 8:538cb718:::benchmark_data_smithi018_106531_object76577:head v 174'4750, mlcod=174'4746)", 2024-09-10T18:52:26.608 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:52:26.585771+0000", 2024-09-10T18:52:26.608 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 0.001154693, 2024-09-10T18:52:26.608 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0014992339999999999, 2024-09-10T18:52:26.608 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:52:26.608 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "started", 2024-09-10T18:52:26.608 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:52:26.608 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.608 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:52:26.608 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.585771+0000", 2024-09-10T18:52:26.608 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:26.609 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.609 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.609 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:52:26.609 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.585771+0000", 2024-09-10T18:52:26.609 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:26.609 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.609 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.609 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:52:26.609 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.585771+0000", 2024-09-10T18:52:26.609 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T18:52:26.609 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.609 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.610 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:52:26.610 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.585802+0000", 2024-09-10T18:52:26.610 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.1952000000000002e-05 2024-09-10T18:52:26.610 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.610 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.610 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:52:26.611 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.585810+0000", 2024-09-10T18:52:26.611 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.9780000000000005e-06 2024-09-10T18:52:26.611 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.611 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.611 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:52:26.611 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.585815+0000", 2024-09-10T18:52:26.612 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.4920000000000004e-06 2024-09-10T18:52:26.612 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.612 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.612 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:52:26.612 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.586017+0000", 2024-09-10T18:52:26.613 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000201762 2024-09-10T18:52:26.613 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:26.613 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:26.613 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:52:26.613 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:26.586035+0000", 2024-09-10T18:52:26.613 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.8323000000000001e-05 2024-09-10T18:52:26.614 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:26.614 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:52:26.614 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:26.614 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:26.614 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-09-10T18:52:26.614 INFO:teuthology.orchestra.run.smithi018.stdout: "num_ops": 4 2024-09-10T18:52:26.614 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T18:52:26.690 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:26.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:26 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:26.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:26 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:26.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:26.726 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:26.727 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:26.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:26 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:26.772 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:52:26 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:52:26.827 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:26.866 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:26.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:26.867 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:26.967 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:27.007 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:27.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:27.008 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:27.108 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:27.141 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:27.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:27.142 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:27.198 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 38 16 77527 77511 127.464 95.625 0.00755814 0.00784057 2024-09-10T18:52:27.243 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:27.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:27.282 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:27.282 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:27.382 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:27.420 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:27.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:27.421 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:27.522 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:27.554 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:27.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:27.556 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:27.626 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-10T18:52:27.657 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:27.693 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:27.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:27.694 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:27.795 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:27.834 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:27.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:27.835 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:27.935 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:27.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:27 smithi018 ceph-mon[40196]: pgmap v303: 168 pgs: 168 active+clean; 4.6 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 5.3 KiB/s rd, 117 MiB/s wr, 1.87k op/s; 63 KiB/s, 0 objects/s recovering 2024-09-10T18:52:27.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:27 smithi018 ceph-mon[29336]: pgmap v303: 168 pgs: 168 active+clean; 4.6 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 5.3 KiB/s rd, 117 MiB/s wr, 1.87k op/s; 63 KiB/s, 0 objects/s recovering 2024-09-10T18:52:27.972 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:27.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:27.973 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:28.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:27 smithi103 ceph-mon[33163]: pgmap v303: 168 pgs: 168 active+clean; 4.6 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 5.3 KiB/s rd, 117 MiB/s wr, 1.87k op/s; 63 KiB/s, 0 objects/s recovering 2024-09-10T18:52:28.074 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:28.102 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:28.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:28.103 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:28.198 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 39 15 78853 78838 126.322 82.9375 0.00823207 0.00791173 2024-09-10T18:52:28.204 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:28.251 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:28.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:28.252 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:28.353 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:28.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:28.399 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:28.399 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:28.500 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:28.535 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:28.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:28.536 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:28.637 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:28.674 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:28.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:28.675 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:28.776 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:28.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:28.813 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:28.813 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:28.914 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:28.955 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:28.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:28.956 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:29.037 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:52:29.042 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-09-10T18:52:29.057 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:29.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:29.092 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:29.092 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:29.194 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:29.198 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:2024-09-10T18:52:29.198233+0000 min lat: 0.00115328 max lat: 0.157801 avg lat: 0.0079672 2024-09-10T18:52:29.198 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T18:52:29.198 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 40 16 80308 80292 125.435 90.875 0.00978152 0.0079672 2024-09-10T18:52:29.230 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:29.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:29.231 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:29.332 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:29.334 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-10T18:52:29.367 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:29.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:29.369 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:29.469 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:29.507 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:29.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:29.508 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:29.609 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:29.610 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:52:29.675 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:29.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:29.676 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:29.777 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:29.815 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:29.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:29.817 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:29.917 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:29.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:29 smithi018 ceph-mon[29336]: osdmap e175: 8 total, 8 up, 7 in 2024-09-10T18:52:29.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:29 smithi018 ceph-mon[29336]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:52:29.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:29 smithi018 ceph-mon[29336]: pgmap v305: 168 pgs: 168 active+clean; 4.6 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 853 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-10T18:52:29.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:29 smithi018 ceph-mon[40196]: osdmap e175: 8 total, 8 up, 7 in 2024-09-10T18:52:29.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:29 smithi018 ceph-mon[40196]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:52:29.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:29 smithi018 ceph-mon[40196]: pgmap v305: 168 pgs: 168 active+clean; 4.6 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 853 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-10T18:52:29.951 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:29.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:29.952 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:30.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:29 smithi103 ceph-mon[33163]: osdmap e175: 8 total, 8 up, 7 in 2024-09-10T18:52:30.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:29 smithi103 ceph-mon[33163]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:52:30.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:29 smithi103 ceph-mon[33163]: pgmap v305: 168 pgs: 168 active+clean; 4.6 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 853 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-10T18:52:30.053 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:30.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:30.088 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:30.089 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:30.190 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:30.199 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 41 16 81779 81763 124.617 91.9375 0.021005 0.00801929 2024-09-10T18:52:30.216 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2024-09-10T18:52:28.564791+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4992554847,"num_objects":76369,"num_object_clones":0,"num_object_copies":229107,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76369,"num_whiteouts":0,"num_read":1170,"num_read_kb":943,"num_write":76626,"num_write_kb":4875332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":580511,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69711,"ondisk_log_size":69711,"up":504,"acting":504,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":504,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":16581084,"kb_used_data":14453844,"kb_used_omap":11,"kb_used_meta":2127156,"kb_avail":639634980,"statfs":{"total":671965249536,"available":654986219520,"internally_reserved":0,"allocated":14800736256,"data_stored":14796395990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11928,"internal_metadata":2178208104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,3,6,14],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":755236864,"num_objects":11524,"num_object_clones":0,"num_object_copies":34572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11524,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":11524,"num_write_kb":737536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.416980"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":151,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.574704+0000","last_change":"2024-09-10T18:50:55.243820+0000","last_active":"2024-09-10T18:52:23.574704+0000","last_peered":"2024-09-10T18:52:23.574704+0000","last_clean":"2024-09-10T18:52:23.574704+0000","last_became_active":"2024-09-10T18:45:24.265488+0000","last_became_peered":"2024-09-10T18:45:24.265488+0000","last_unstale":"2024-09-10T18:52:23.574704+0000","last_undegraded":"2024-09-10T18:52:23.574704+0000","last_fullsized":"2024-09-10T18:52:23.574704+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:55.243629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:50:55.243629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0090834450000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":167,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.570063+0000","last_change":"2024-09-10T18:52:02.360333+0000","last_active":"2024-09-10T18:52:23.570063+0000","last_peered":"2024-09-10T18:52:23.570063+0000","last_clean":"2024-09-10T18:52:23.570063+0000","last_became_active":"2024-09-10T18:45:17.959475+0000","last_became_peered":"2024-09-10T18:45:17.959475+0000","last_unstale":"2024-09-10T18:52:23.570063+0000","last_undegraded":"2024-09-10T18:52:23.570063+0000","last_fullsized":"2024-09-10T18:52:23.570063+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:02.360246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.718270+0000","last_clean_scrub_stamp":"2024-09-10T18:52:02.360246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:06.813925+0000","scrub_duration":0.0054286930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"63'15","reported_seq":206,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.574312+0000","last_change":"2024-09-10T18:51:34.023246+0000","last_active":"2024-09-10T18:52:23.574312+0000","last_peered":"2024-09-10T18:52:23.574312+0000","last_clean":"2024-09-10T18:52:23.574312+0000","last_became_active":"2024-09-10T18:47:36.413635+0000","last_became_peered":"2024-09-10T18:47:36.413635+0000","last_unstale":"2024-09-10T18:52:23.574312+0000","last_undegraded":"2024-09-10T18:52:23.574312+0000","last_fullsized":"2024-09-10T18:52:23.574312+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:51:34.023191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:34.023191+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:41.128415+0000","scrub_duration":0.0027158859999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":151,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.576442+0000","last_change":"2024-09-10T18:50:50.546527+0000","last_active":"2024-09-10T18:52:23.576442+0000","last_peered":"2024-09-10T18:52:23.576442+0000","last_clean":"2024-09-10T18:52:23.576442+0000","last_became_active":"2024-09-10T18:45:22.065619+0000","last_became_peered":"2024-09-10T18:45:22.065619+0000","last_unstale":"2024-09-10T18:52:23.576442+0000","last_undegraded":"2024-09-10T18:52:23.576442+0000","last_fullsized":"2024-09-10T18:52:23.576442+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:50.546473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.057932+0000","last_clean_scrub_stamp":"2024-09-10T18:50:50.546473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026598759999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":157,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.577196+0000","last_change":"2024-09-10T18:50:58.174634+0000","last_active":"2024-09-10T18:52:23.577196+0000","last_peered":"2024-09-10T18:52:23.577196+0000","last_clean":"2024-09-10T18:52:23.577196+0000","last_became_active":"2024-09-10T18:45:17.966153+0000","last_became_peered":"2024-09-10T18:45:17.966153+0000","last_unstale":"2024-09-10T18:52:23.577196+0000","last_undegraded":"2024-09-10T18:52:23.577196+0000","last_fullsized":"2024-09-10T18:52:23.577196+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:58.174399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:58.174399+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012266312,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"83'13","reported_seq":173,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.576484+0000","last_change":"2024-09-10T18:52:07.239403+0000","last_active":"2024-09-10T18:52:23.576484+0000","last_peered":"2024-09-10T18:52:23.576484+0000","last_clean":"2024-09-10T18:52:23.576484+0000","last_became_active":"2024-09-10T18:45:20.061266+0000","last_became_peered":"2024-09-10T18:45:20.061266+0000","last_unstale":"2024-09-10T18:52:23.576484+0000","last_undegraded":"2024-09-10T18:52:23.576484+0000","last_fullsized":"2024-09-10T18:52:23.576484+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'13","last_scrub_stamp":"2024-09-10T18:52:07.239318+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_clean_scrub_stamp":"2024-09-10T18:52:07.239318+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:24.825381+0000","scrub_duration":0.0044351590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":153,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.571462+0000","last_change":"2024-09-10T18:52:01.137291+0000","last_active":"2024-09-10T18:52:23.571462+0000","last_peered":"2024-09-10T18:52:23.571462+0000","last_clean":"2024-09-10T18:52:23.571462+0000","last_became_active":"2024-09-10T18:45:22.072822+0000","last_became_peered":"2024-09-10T18:45:22.072822+0000","last_unstale":"2024-09-10T18:52:23.571462+0000","last_undegraded":"2024-09-10T18:52:23.571462+0000","last_fullsized":"2024-09-10T18:52:23.571462+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:01.135988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:01.135988+0000","last_clean_scrub_stamp":"2024-09-10T18:52:01.135988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:05.351158+0000","scrub_duration":0.002631478,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":149,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.579951+0000","last_change":"2024-09-10T18:51:48.432515+0000","last_active":"2024-09-10T18:52:23.579951+0000","last_peered":"2024-09-10T18:52:23.579951+0000","last_clean":"2024-09-10T18:52:23.579951+0000","last_became_active":"2024-09-10T18:45:24.081002+0000","last_became_peered":"2024-09-10T18:45:24.081002+0000","last_unstale":"2024-09-10T18:52:23.579951+0000","last_undegraded":"2024-09-10T18:52:23.579951+0000","last_fullsized":"2024-09-10T18:52:23.579951+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:48.432459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:48.432459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:56.682264+0000","scrub_duration":0.0013158130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":167,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.568181+0000","last_change":"2024-09-10T18:51:51.188534+0000","last_active":"2024-09-10T18:52:23.568181+0000","last_peered":"2024-09-10T18:52:23.568181+0000","last_clean":"2024-09-10T18:52:23.568181+0000","last_became_active":"2024-09-10T18:45:17.961097+0000","last_became_peered":"2024-09-10T18:45:17.961097+0000","last_unstale":"2024-09-10T18:52:23.568181+0000","last_undegraded":"2024-09-10T18:52:23.568181+0000","last_fullsized":"2024-09-10T18:52:23.568181+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:51.188458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:51.188458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:04.823887+0000","scrub_duration":0.0037631460000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"163'7","reported_seq":165,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.576776+0000","last_change":"2024-09-10T18:51:54.261967+0000","last_active":"2024-09-10T18:52:23.576776+0000","last_peered":"2024-09-10T18:52:23.576776+0000","last_clean":"2024-09-10T18:52:23.576776+0000","last_became_active":"2024-09-10T18:45:20.057567+0000","last_became_peered":"2024-09-10T18:45:20.057567+0000","last_unstale":"2024-09-10T18:52:23.576776+0000","last_undegraded":"2024-09-10T18:52:23.576776+0000","last_fullsized":"2024-09-10T18:52:23.576776+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"163'7","last_scrub_stamp":"2024-09-10T18:51:54.261879+0000","last_deep_scrub":"163'7","last_deep_scrub_stamp":"2024-09-10T18:51:54.261879+0000","last_clean_scrub_stamp":"2024-09-10T18:51:54.261879+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:12.396567+0000","scrub_duration":0.0087662790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":163,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.567287+0000","last_change":"2024-09-10T18:51:33.438852+0000","last_active":"2024-09-10T18:52:23.567287+0000","last_peered":"2024-09-10T18:52:23.567287+0000","last_clean":"2024-09-10T18:52:23.567287+0000","last_became_active":"2024-09-10T18:45:22.076575+0000","last_became_peered":"2024-09-10T18:45:22.076575+0000","last_unstale":"2024-09-10T18:52:23.567287+0000","last_undegraded":"2024-09-10T18:52:23.567287+0000","last_fullsized":"2024-09-10T18:52:23.567287+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:33.438763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:36.509971+0000","last_clean_scrub_stamp":"2024-09-10T18:51:33.438763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:57.162571+0000","scrub_duration":0.0019905130000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":160,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.565899+0000","last_change":"2024-09-10T18:51:51.287526+0000","last_active":"2024-09-10T18:52:23.565899+0000","last_peered":"2024-09-10T18:52:23.565899+0000","last_clean":"2024-09-10T18:52:23.565899+0000","last_became_active":"2024-09-10T18:45:24.079534+0000","last_became_peered":"2024-09-10T18:45:24.079534+0000","last_unstale":"2024-09-10T18:52:23.565899+0000","last_undegraded":"2024-09-10T18:52:23.565899+0000","last_fullsized":"2024-09-10T18:52:23.565899+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:51.287440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:51.287440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:10.109656+0000","scrub_duration":0.038379101999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":167,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.568144+0000","last_change":"2024-09-10T18:50:58.526247+0000","last_active":"2024-09-10T18:52:23.568144+0000","last_peered":"2024-09-10T18:52:23.568144+0000","last_clean":"2024-09-10T18:52:23.568144+0000","last_became_active":"2024-09-10T18:47:36.414393+0000","last_became_peered":"2024-09-10T18:47:36.414393+0000","last_unstale":"2024-09-10T18:52:23.568144+0000","last_undegraded":"2024-09-10T18:52:23.568144+0000","last_fullsized":"2024-09-10T18:52:23.568144+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:58.526191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:58.526191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021021659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"63'9","reported_seq":169,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.580280+0000","last_change":"2024-09-10T18:51:31.494087+0000","last_active":"2024-09-10T18:52:23.580280+0000","last_peered":"2024-09-10T18:52:23.580280+0000","last_clean":"2024-09-10T18:52:23.580280+0000","last_became_active":"2024-09-10T18:45:20.052652+0000","last_became_peered":"2024-09-10T18:45:20.052652+0000","last_unstale":"2024-09-10T18:52:23.580280+0000","last_undegraded":"2024-09-10T18:52:23.580280+0000","last_fullsized":"2024-09-10T18:52:23.580280+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:51:31.494021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:31.494021+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:39.434873+0000","scrub_duration":0.0024074779999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":162,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.565813+0000","last_change":"2024-09-10T18:51:59.201412+0000","last_active":"2024-09-10T18:52:23.565813+0000","last_peered":"2024-09-10T18:52:23.565813+0000","last_clean":"2024-09-10T18:52:23.565813+0000","last_became_active":"2024-09-10T18:45:22.068326+0000","last_became_peered":"2024-09-10T18:45:22.068326+0000","last_unstale":"2024-09-10T18:52:23.565813+0000","last_undegraded":"2024-09-10T18:52:23.565813+0000","last_fullsized":"2024-09-10T18:52:23.565813+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:59.201336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:59.201336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:14.834777+0000","scrub_duration":0.0032166959999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":151,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.569349+0000","last_change":"2024-09-10T18:51:03.523117+0000","last_active":"2024-09-10T18:52:23.569349+0000","last_peered":"2024-09-10T18:52:23.569349+0000","last_clean":"2024-09-10T18:52:23.569349+0000","last_became_active":"2024-09-10T18:45:24.081488+0000","last_became_peered":"2024-09-10T18:45:24.081488+0000","last_unstale":"2024-09-10T18:52:23.569349+0000","last_undegraded":"2024-09-10T18:52:23.569349+0000","last_fullsized":"2024-09-10T18:52:23.569349+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:03.523030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:03.523030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:33.287999+0000","scrub_duration":0.002492677,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"174'4907","reported_seq":4931,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-10T18:52:26.320736+0000","last_change":"2024-09-10T18:51:37.588528+0000","last_active":"2024-09-10T18:52:26.320736+0000","last_peered":"2024-09-10T18:52:26.320736+0000","last_clean":"2024-09-10T18:52:26.320736+0000","last_became_active":"2024-09-10T18:51:37.588217+0000","last_became_peered":"2024-09-10T18:51:37.588217+0000","last_unstale":"2024-09-10T18:52:26.320736+0000","last_undegraded":"2024-09-10T18:52:26.320736+0000","last_fullsized":"2024-09-10T18:52:26.320736+0000","mapping_epoch":158,"log_start":"163'800","ondisk_log_start":"163'800","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":4107,"log_dups_size":0,"ondisk_log_size":4107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:42.284615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":321585152,"num_objects":4907,"num_object_clones":0,"num_object_copies":14721,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4907,"num_write_kb":314048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"70'81","reported_seq":79,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.565637+0000","last_change":"2024-09-10T18:52:06.204059+0000","last_active":"2024-09-10T18:52:23.565637+0000","last_peered":"2024-09-10T18:52:23.565637+0000","last_clean":"2024-09-10T18:52:23.565637+0000","last_became_active":"2024-09-10T18:50:08.397233+0000","last_became_peered":"2024-09-10T18:50:08.397233+0000","last_unstale":"2024-09-10T18:52:23.565637+0000","last_undegraded":"2024-09-10T18:52:23.565637+0000","last_fullsized":"2024-09-10T18:52:23.565637+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:52:06.203977+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:52:06.203977+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:23.193949+0000","scrub_duration":0.0074267229999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"78'25","reported_seq":217,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.577283+0000","last_change":"2024-09-10T18:50:58.450309+0000","last_active":"2024-09-10T18:52:23.577283+0000","last_peered":"2024-09-10T18:52:23.577283+0000","last_clean":"2024-09-10T18:52:23.577283+0000","last_became_active":"2024-09-10T18:47:36.409300+0000","last_became_peered":"2024-09-10T18:47:36.409300+0000","last_unstale":"2024-09-10T18:52:23.577283+0000","last_undegraded":"2024-09-10T18:52:23.577283+0000","last_fullsized":"2024-09-10T18:52:23.577283+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'25","last_scrub_stamp":"2024-09-10T18:50:58.450203+0000","last_deep_scrub":"78'25","last_deep_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_clean_scrub_stamp":"2024-09-10T18:50:58.450203+0000","objects_scrubbed":8,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058694890000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":167,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.567598+0000","last_change":"2024-09-10T18:51:54.175390+0000","last_active":"2024-09-10T18:52:23.567598+0000","last_peered":"2024-09-10T18:52:23.567598+0000","last_clean":"2024-09-10T18:52:23.567598+0000","last_became_active":"2024-09-10T18:45:17.963732+0000","last_became_peered":"2024-09-10T18:45:17.963732+0000","last_unstale":"2024-09-10T18:52:23.567598+0000","last_undegraded":"2024-09-10T18:52:23.567598+0000","last_fullsized":"2024-09-10T18:52:23.567598+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:54.175305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:54.175305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:20.881315+0000","scrub_duration":0.0029840190000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":153,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.571119+0000","last_change":"2024-09-10T18:51:39.055614+0000","last_active":"2024-09-10T18:52:23.571119+0000","last_peered":"2024-09-10T18:52:23.571119+0000","last_clean":"2024-09-10T18:52:23.571119+0000","last_became_active":"2024-09-10T18:45:22.071443+0000","last_became_peered":"2024-09-10T18:45:22.071443+0000","last_unstale":"2024-09-10T18:52:23.571119+0000","last_undegraded":"2024-09-10T18:52:23.571119+0000","last_fullsized":"2024-09-10T18:52:23.571119+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:39.055556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:04.512425+0000","last_clean_scrub_stamp":"2024-09-10T18:51:39.055556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:03.733831+0000","scrub_duration":0.0016498730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":107,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.569795+0000","last_change":"2024-09-10T18:51:34.251677+0000","last_active":"2024-09-10T18:52:23.569795+0000","last_peered":"2024-09-10T18:52:23.569795+0000","last_clean":"2024-09-10T18:52:23.569795+0000","last_became_active":"2024-09-10T18:47:57.104433+0000","last_became_peered":"2024-09-10T18:47:57.104433+0000","last_unstale":"2024-09-10T18:52:23.569795+0000","last_undegraded":"2024-09-10T18:52:23.569795+0000","last_fullsized":"2024-09-10T18:52:23.569795+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:34.251618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:34.251618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:39.525376+0000","scrub_duration":0.0041421139999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"174'4703","reported_seq":4727,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-10T18:52:26.430618+0000","last_change":"2024-09-10T18:51:37.588201+0000","last_active":"2024-09-10T18:52:26.430618+0000","last_peered":"2024-09-10T18:52:26.430618+0000","last_clean":"2024-09-10T18:52:26.430618+0000","last_became_active":"2024-09-10T18:51:37.587766+0000","last_became_peered":"2024-09-10T18:51:37.587766+0000","last_unstale":"2024-09-10T18:52:26.430618+0000","last_undegraded":"2024-09-10T18:52:26.430618+0000","last_fullsized":"2024-09-10T18:52:26.430618+0000","mapping_epoch":158,"log_start":"163'700","ondisk_log_start":"163'700","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":4003,"log_dups_size":0,"ondisk_log_size":4003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:00.739689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":308215808,"num_objects":4703,"num_object_clones":0,"num_object_copies":14109,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4703,"num_write_kb":300992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.9","version":"70'81","reported_seq":221,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.567354+0000","last_change":"2024-09-10T18:50:53.567955+0000","last_active":"2024-09-10T18:52:23.567354+0000","last_peered":"2024-09-10T18:52:23.567354+0000","last_clean":"2024-09-10T18:52:23.567354+0000","last_became_active":"2024-09-10T18:50:08.399879+0000","last_became_peered":"2024-09-10T18:50:08.399879+0000","last_unstale":"2024-09-10T18:52:23.567354+0000","last_undegraded":"2024-09-10T18:52:23.567354+0000","last_fullsized":"2024-09-10T18:52:23.567354+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:50:53.567865+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:50:53.567865+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0066605650000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"116'15","reported_seq":173,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.577160+0000","last_change":"2024-09-10T18:51:37.298645+0000","last_active":"2024-09-10T18:52:23.577160+0000","last_peered":"2024-09-10T18:52:23.577160+0000","last_clean":"2024-09-10T18:52:23.577160+0000","last_became_active":"2024-09-10T18:45:20.053428+0000","last_became_peered":"2024-09-10T18:45:20.053428+0000","last_unstale":"2024-09-10T18:52:23.577160+0000","last_undegraded":"2024-09-10T18:52:23.577160+0000","last_fullsized":"2024-09-10T18:52:23.577160+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"116'15","last_scrub_stamp":"2024-09-10T18:51:37.298554+0000","last_deep_scrub":"116'15","last_deep_scrub_stamp":"2024-09-10T18:49:12.374527+0000","last_clean_scrub_stamp":"2024-09-10T18:51:37.298554+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:49.476419+0000","scrub_duration":0.0020143299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":157,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.574814+0000","last_change":"2024-09-10T18:51:03.086045+0000","last_active":"2024-09-10T18:52:23.574814+0000","last_peered":"2024-09-10T18:52:23.574814+0000","last_clean":"2024-09-10T18:52:23.574814+0000","last_became_active":"2024-09-10T18:45:17.966833+0000","last_became_peered":"2024-09-10T18:45:17.966833+0000","last_unstale":"2024-09-10T18:52:23.574814+0000","last_undegraded":"2024-09-10T18:52:23.574814+0000","last_fullsized":"2024-09-10T18:52:23.574814+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:03.085839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:03.085839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0082351739999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"89'9","reported_seq":209,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.567991+0000","last_change":"2024-09-10T18:51:50.194342+0000","last_active":"2024-09-10T18:52:23.567991+0000","last_peered":"2024-09-10T18:52:23.567991+0000","last_clean":"2024-09-10T18:52:23.567991+0000","last_became_active":"2024-09-10T18:47:36.421188+0000","last_became_peered":"2024-09-10T18:47:36.421188+0000","last_unstale":"2024-09-10T18:52:23.567991+0000","last_undegraded":"2024-09-10T18:52:23.567991+0000","last_fullsized":"2024-09-10T18:52:23.567991+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'9","last_scrub_stamp":"2024-09-10T18:51:50.194254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:50.194254+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:06.035300+0000","scrub_duration":0.0025134200000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":159,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.577169+0000","last_change":"2024-09-10T18:50:56.384791+0000","last_active":"2024-09-10T18:52:23.577169+0000","last_peered":"2024-09-10T18:52:23.577169+0000","last_clean":"2024-09-10T18:52:23.577169+0000","last_became_active":"2024-09-10T18:47:36.414625+0000","last_became_peered":"2024-09-10T18:47:36.414625+0000","last_unstale":"2024-09-10T18:52:23.577169+0000","last_undegraded":"2024-09-10T18:52:23.577169+0000","last_fullsized":"2024-09-10T18:52:23.577169+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:56.384742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:32.043666+0000","last_clean_scrub_stamp":"2024-09-10T18:50:56.384742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001329467,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3","version":"174'4746","reported_seq":4770,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-10T18:52:26.326130+0000","last_change":"2024-09-10T18:51:37.590013+0000","last_active":"2024-09-10T18:52:26.326130+0000","last_peered":"2024-09-10T18:52:26.326130+0000","last_clean":"2024-09-10T18:52:26.326130+0000","last_became_active":"2024-09-10T18:51:37.589730+0000","last_became_peered":"2024-09-10T18:51:37.589730+0000","last_unstale":"2024-09-10T18:52:26.326130+0000","last_undegraded":"2024-09-10T18:52:26.326130+0000","last_fullsized":"2024-09-10T18:52:26.326130+0000","mapping_epoch":158,"log_start":"163'600","ondisk_log_start":"163'600","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":4146,"log_dups_size":0,"ondisk_log_size":4146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:45.681857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":311033856,"num_objects":4746,"num_object_clones":0,"num_object_copies":14238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4746,"num_write_kb":303744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"70'81","reported_seq":74,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.564193+0000","last_change":"2024-09-10T18:52:04.178564+0000","last_active":"2024-09-10T18:52:23.564193+0000","last_peered":"2024-09-10T18:52:23.564193+0000","last_clean":"2024-09-10T18:52:23.564193+0000","last_became_active":"2024-09-10T18:50:08.398345+0000","last_became_peered":"2024-09-10T18:50:08.398345+0000","last_unstale":"2024-09-10T18:52:23.564193+0000","last_undegraded":"2024-09-10T18:52:23.564193+0000","last_fullsized":"2024-09-10T18:52:23.564193+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:52:04.178483+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:52:04.178483+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:13.260845+0000","scrub_duration":0.0041829420000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"141'22","reported_seq":153,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.564576+0000","last_change":"2024-09-10T18:50:57.119238+0000","last_active":"2024-09-10T18:52:23.564576+0000","last_peered":"2024-09-10T18:52:23.564576+0000","last_clean":"2024-09-10T18:52:23.564576+0000","last_became_active":"2024-09-10T18:46:56.742946+0000","last_became_peered":"2024-09-10T18:46:56.742946+0000","last_unstale":"2024-09-10T18:52:23.564576+0000","last_undegraded":"2024-09-10T18:52:23.564576+0000","last_fullsized":"2024-09-10T18:52:23.564576+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"141'22","last_scrub_stamp":"2024-09-10T18:50:57.119152+0000","last_deep_scrub":"141'22","last_deep_scrub_stamp":"2024-09-10T18:50:57.119152+0000","last_clean_scrub_stamp":"2024-09-10T18:50:57.119152+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.020103994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":167,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.571648+0000","last_change":"2024-09-10T18:51:51.055043+0000","last_active":"2024-09-10T18:52:23.571648+0000","last_peered":"2024-09-10T18:52:23.571648+0000","last_clean":"2024-09-10T18:52:23.571648+0000","last_became_active":"2024-09-10T18:45:17.965409+0000","last_became_peered":"2024-09-10T18:45:17.965409+0000","last_unstale":"2024-09-10T18:52:23.571648+0000","last_undegraded":"2024-09-10T18:52:23.571648+0000","last_fullsized":"2024-09-10T18:52:23.571648+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:51.054935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_clean_scrub_stamp":"2024-09-10T18:51:51.054935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:06.936657+0000","scrub_duration":0.004679758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"63'8","reported_seq":244,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.577307+0000","last_change":"2024-09-10T18:50:48.530731+0000","last_active":"2024-09-10T18:52:23.577307+0000","last_peered":"2024-09-10T18:52:23.577307+0000","last_clean":"2024-09-10T18:52:23.577307+0000","last_became_active":"2024-09-10T18:45:22.062497+0000","last_became_peered":"2024-09-10T18:45:22.062497+0000","last_unstale":"2024-09-10T18:52:23.577307+0000","last_undegraded":"2024-09-10T18:52:23.577307+0000","last_fullsized":"2024-09-10T18:52:23.577307+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:50:48.530644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:48.530644+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.022829841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":160,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.564055+0000","last_change":"2024-09-10T18:51:54.218678+0000","last_active":"2024-09-10T18:52:23.564055+0000","last_peered":"2024-09-10T18:52:23.564055+0000","last_clean":"2024-09-10T18:52:23.564055+0000","last_became_active":"2024-09-10T18:45:24.082362+0000","last_became_peered":"2024-09-10T18:45:24.082362+0000","last_unstale":"2024-09-10T18:52:23.564055+0000","last_undegraded":"2024-09-10T18:52:23.564055+0000","last_fullsized":"2024-09-10T18:52:23.564055+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:54.218602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:54.218602+0000","last_clean_scrub_stamp":"2024-09-10T18:51:54.218602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:05.263122+0000","scrub_duration":0.0059638290000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"70'81","reported_seq":83,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.572718+0000","last_change":"2024-09-10T18:52:16.601079+0000","last_active":"2024-09-10T18:52:23.572718+0000","last_peered":"2024-09-10T18:52:23.572718+0000","last_clean":"2024-09-10T18:52:23.572718+0000","last_became_active":"2024-09-10T18:52:16.599938+0000","last_became_peered":"2024-09-10T18:52:16.599938+0000","last_unstale":"2024-09-10T18:52:23.572718+0000","last_undegraded":"2024-09-10T18:52:23.572718+0000","last_fullsized":"2024-09-10T18:52:23.572718+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:51:04.099703+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:51:04.099703+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007621203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"174'4739","reported_seq":4762,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-10T18:52:26.101939+0000","last_change":"2024-09-10T18:51:37.589268+0000","last_active":"2024-09-10T18:52:26.101939+0000","last_peered":"2024-09-10T18:52:26.101939+0000","last_clean":"2024-09-10T18:52:26.101939+0000","last_became_active":"2024-09-10T18:51:37.588723+0000","last_became_peered":"2024-09-10T18:51:37.588723+0000","last_unstale":"2024-09-10T18:52:26.101939+0000","last_undegraded":"2024-09-10T18:52:26.101939+0000","last_fullsized":"2024-09-10T18:52:26.101939+0000","mapping_epoch":158,"log_start":"163'400","ondisk_log_start":"163'400","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":4339,"log_dups_size":0,"ondisk_log_size":4339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:45.854850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":310575104,"num_objects":4739,"num_object_clones":0,"num_object_copies":14217,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4739,"num_write_kb":303296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"63'11","reported_seq":238,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.577380+0000","last_change":"2024-09-10T18:50:42.581936+0000","last_active":"2024-09-10T18:52:23.577380+0000","last_peered":"2024-09-10T18:52:23.577380+0000","last_clean":"2024-09-10T18:52:23.577380+0000","last_became_active":"2024-09-10T18:47:57.106935+0000","last_became_peered":"2024-09-10T18:47:57.106935+0000","last_unstale":"2024-09-10T18:52:23.577380+0000","last_undegraded":"2024-09-10T18:52:23.577380+0000","last_fullsized":"2024-09-10T18:52:23.577380+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:50:42.581837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:50:42.581837+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0093156460000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":167,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.567122+0000","last_change":"2024-09-10T18:51:44.209450+0000","last_active":"2024-09-10T18:52:23.567122+0000","last_peered":"2024-09-10T18:52:23.567122+0000","last_clean":"2024-09-10T18:52:23.567122+0000","last_became_active":"2024-09-10T18:45:17.966133+0000","last_became_peered":"2024-09-10T18:45:17.966133+0000","last_unstale":"2024-09-10T18:52:23.567122+0000","last_undegraded":"2024-09-10T18:52:23.567122+0000","last_fullsized":"2024-09-10T18:52:23.567122+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:44.209392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:44.209392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:04.575890+0000","scrub_duration":0.001553123,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":162,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.564764+0000","last_change":"2024-09-10T18:51:44.388198+0000","last_active":"2024-09-10T18:52:23.564764+0000","last_peered":"2024-09-10T18:52:23.564764+0000","last_clean":"2024-09-10T18:52:23.564764+0000","last_became_active":"2024-09-10T18:45:22.068918+0000","last_became_peered":"2024-09-10T18:45:22.068918+0000","last_unstale":"2024-09-10T18:52:23.564764+0000","last_undegraded":"2024-09-10T18:52:23.564764+0000","last_fullsized":"2024-09-10T18:52:23.564764+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:44.388105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_clean_scrub_stamp":"2024-09-10T18:51:44.388105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:08.162498+0000","scrub_duration":0.0013288989999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":151,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.576699+0000","last_change":"2024-09-10T18:51:41.073374+0000","last_active":"2024-09-10T18:52:23.576699+0000","last_peered":"2024-09-10T18:52:23.576699+0000","last_clean":"2024-09-10T18:52:23.576699+0000","last_became_active":"2024-09-10T18:45:24.264538+0000","last_became_peered":"2024-09-10T18:45:24.264538+0000","last_unstale":"2024-09-10T18:52:23.576699+0000","last_undegraded":"2024-09-10T18:52:23.576699+0000","last_fullsized":"2024-09-10T18:52:23.576699+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:41.073320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:41.073320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:00.507812+0000","scrub_duration":0.001777551,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"174'4701","reported_seq":4725,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-10T18:52:26.088018+0000","last_change":"2024-09-10T18:51:37.911703+0000","last_active":"2024-09-10T18:52:26.088018+0000","last_peered":"2024-09-10T18:52:26.088018+0000","last_clean":"2024-09-10T18:52:26.088018+0000","last_became_active":"2024-09-10T18:51:37.911394+0000","last_became_peered":"2024-09-10T18:51:37.911394+0000","last_unstale":"2024-09-10T18:52:26.088018+0000","last_undegraded":"2024-09-10T18:52:26.088018+0000","last_fullsized":"2024-09-10T18:52:26.088018+0000","mapping_epoch":158,"log_start":"163'600","ondisk_log_start":"163'600","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":4101,"log_dups_size":0,"ondisk_log_size":4101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:59.524921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":308084736,"num_objects":4701,"num_object_clones":0,"num_object_copies":14103,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4701,"num_write_kb":300864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"70'81","reported_seq":79,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.576819+0000","last_change":"2024-09-10T18:52:14.590084+0000","last_active":"2024-09-10T18:52:23.576819+0000","last_peered":"2024-09-10T18:52:23.576819+0000","last_clean":"2024-09-10T18:52:23.576819+0000","last_became_active":"2024-09-10T18:52:14.589638+0000","last_became_peered":"2024-09-10T18:52:14.589638+0000","last_unstale":"2024-09-10T18:52:23.576819+0000","last_undegraded":"2024-09-10T18:52:23.576819+0000","last_fullsized":"2024-09-10T18:52:23.576819+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:50:59.164417+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:50:59.164417+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0063541700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109226,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"143'21","reported_seq":181,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.576780+0000","last_change":"2024-09-10T18:51:02.540313+0000","last_active":"2024-09-10T18:52:23.576780+0000","last_peered":"2024-09-10T18:52:23.576780+0000","last_clean":"2024-09-10T18:52:23.576780+0000","last_became_active":"2024-09-10T18:45:20.052243+0000","last_became_peered":"2024-09-10T18:45:20.052243+0000","last_unstale":"2024-09-10T18:52:23.576780+0000","last_undegraded":"2024-09-10T18:52:23.576780+0000","last_fullsized":"2024-09-10T18:52:23.576780+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"143'21","last_scrub_stamp":"2024-09-10T18:51:02.540216+0000","last_deep_scrub":"143'21","last_deep_scrub_stamp":"2024-09-10T18:51:02.540216+0000","last_clean_scrub_stamp":"2024-09-10T18:51:02.540216+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:26.124099+0000","scrub_duration":0.0060974410000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":166,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.561803+0000","last_change":"2024-09-10T18:49:59.397785+0000","last_active":"2024-09-10T18:52:23.561803+0000","last_peered":"2024-09-10T18:52:23.561803+0000","last_clean":"2024-09-10T18:52:23.561803+0000","last_became_active":"2024-09-10T18:47:36.424466+0000","last_became_peered":"2024-09-10T18:47:36.424466+0000","last_unstale":"2024-09-10T18:52:23.561803+0000","last_undegraded":"2024-09-10T18:52:23.561803+0000","last_fullsized":"2024-09-10T18:52:23.561803+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:59.397734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_clean_scrub_stamp":"2024-09-10T18:49:59.397734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00244567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":247,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-10T18:52:24.574023+0000","last_change":"2024-09-10T18:50:53.293010+0000","last_active":"2024-09-10T18:52:24.574023+0000","last_peered":"2024-09-10T18:52:24.574023+0000","last_clean":"2024-09-10T18:52:24.574023+0000","last_became_active":"2024-09-10T18:45:22.074540+0000","last_became_peered":"2024-09-10T18:45:22.074540+0000","last_unstale":"2024-09-10T18:52:24.574023+0000","last_undegraded":"2024-09-10T18:52:24.574023+0000","last_fullsized":"2024-09-10T18:52:24.574023+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:50:53.292838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:53.292838+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.018341929999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":171,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.567260+0000","last_change":"2024-09-10T18:51:01.463186+0000","last_active":"2024-09-10T18:52:23.567260+0000","last_peered":"2024-09-10T18:52:23.567260+0000","last_clean":"2024-09-10T18:52:23.567260+0000","last_became_active":"2024-09-10T18:47:36.414665+0000","last_became_peered":"2024-09-10T18:47:36.414665+0000","last_unstale":"2024-09-10T18:52:23.567260+0000","last_undegraded":"2024-09-10T18:52:23.567260+0000","last_fullsized":"2024-09-10T18:52:23.567260+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:01.463123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:01.463123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0050178740000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"174'4791","reported_seq":4815,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-10T18:52:26.528631+0000","last_change":"2024-09-10T18:51:37.909451+0000","last_active":"2024-09-10T18:52:26.528631+0000","last_peered":"2024-09-10T18:52:26.528631+0000","last_clean":"2024-09-10T18:52:26.528631+0000","last_became_active":"2024-09-10T18:51:37.909136+0000","last_became_peered":"2024-09-10T18:51:37.909136+0000","last_unstale":"2024-09-10T18:52:26.528631+0000","last_undegraded":"2024-09-10T18:52:26.528631+0000","last_fullsized":"2024-09-10T18:52:26.528631+0000","mapping_epoch":158,"log_start":"163'300","ondisk_log_start":"163'300","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":4491,"log_dups_size":0,"ondisk_log_size":4491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:03.096434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":313982976,"num_objects":4791,"num_object_clones":0,"num_object_copies":14373,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4791,"num_write_kb":306624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"70'81","reported_seq":73,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.576724+0000","last_change":"2024-09-10T18:52:14.585276+0000","last_active":"2024-09-10T18:52:23.576724+0000","last_peered":"2024-09-10T18:52:23.576724+0000","last_clean":"2024-09-10T18:52:23.576724+0000","last_became_active":"2024-09-10T18:52:14.584798+0000","last_became_peered":"2024-09-10T18:52:14.584798+0000","last_unstale":"2024-09-10T18:52:23.576724+0000","last_undegraded":"2024-09-10T18:52:23.576724+0000","last_fullsized":"2024-09-10T18:52:23.576724+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:51:01.489919+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:51:01.489919+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014613810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"63'18","reported_seq":187,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.573991+0000","last_change":"2024-09-10T18:50:44.111437+0000","last_active":"2024-09-10T18:52:23.573991+0000","last_peered":"2024-09-10T18:52:23.573991+0000","last_clean":"2024-09-10T18:52:23.573991+0000","last_became_active":"2024-09-10T18:45:20.420937+0000","last_became_peered":"2024-09-10T18:45:20.420937+0000","last_unstale":"2024-09-10T18:52:23.573991+0000","last_undegraded":"2024-09-10T18:52:23.573991+0000","last_fullsized":"2024-09-10T18:52:23.573991+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'18","last_scrub_stamp":"2024-09-10T18:50:44.111343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:50:44.111343+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013021517999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":157,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.571122+0000","last_change":"2024-09-10T18:51:48.047994+0000","last_active":"2024-09-10T18:52:23.571122+0000","last_peered":"2024-09-10T18:52:23.571122+0000","last_clean":"2024-09-10T18:52:23.571122+0000","last_became_active":"2024-09-10T18:45:17.961695+0000","last_became_peered":"2024-09-10T18:45:17.961695+0000","last_unstale":"2024-09-10T18:52:23.571122+0000","last_undegraded":"2024-09-10T18:52:23.571122+0000","last_fullsized":"2024-09-10T18:52:23.571122+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:48.047929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:48.047929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:49.914996+0000","scrub_duration":0.0018242130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":151,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.576808+0000","last_change":"2024-09-10T18:50:43.554137+0000","last_active":"2024-09-10T18:52:23.576808+0000","last_peered":"2024-09-10T18:52:23.576808+0000","last_clean":"2024-09-10T18:52:23.576808+0000","last_became_active":"2024-09-10T18:45:22.074012+0000","last_became_peered":"2024-09-10T18:45:22.074012+0000","last_unstale":"2024-09-10T18:52:23.576808+0000","last_undegraded":"2024-09-10T18:52:23.576808+0000","last_fullsized":"2024-09-10T18:52:23.576808+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:43.554063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:43.554063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0065283629999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":159,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.580210+0000","last_change":"2024-09-10T18:51:52.332328+0000","last_active":"2024-09-10T18:52:23.580210+0000","last_peered":"2024-09-10T18:52:23.580210+0000","last_clean":"2024-09-10T18:52:23.580210+0000","last_became_active":"2024-09-10T18:47:36.410836+0000","last_became_peered":"2024-09-10T18:47:36.410836+0000","last_unstale":"2024-09-10T18:52:23.580210+0000","last_undegraded":"2024-09-10T18:52:23.580210+0000","last_fullsized":"2024-09-10T18:52:23.580210+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:52.332258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_clean_scrub_stamp":"2024-09-10T18:51:52.332258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:02.920642+0000","scrub_duration":0.0030121940000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"174'4797","reported_seq":4821,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-10T18:52:26.432231+0000","last_change":"2024-09-10T18:51:37.584352+0000","last_active":"2024-09-10T18:52:26.432231+0000","last_peered":"2024-09-10T18:52:26.432231+0000","last_clean":"2024-09-10T18:52:26.432231+0000","last_became_active":"2024-09-10T18:51:37.583787+0000","last_became_peered":"2024-09-10T18:51:37.583787+0000","last_unstale":"2024-09-10T18:52:26.432231+0000","last_undegraded":"2024-09-10T18:52:26.432231+0000","last_fullsized":"2024-09-10T18:52:26.432231+0000","mapping_epoch":158,"log_start":"163'800","ondisk_log_start":"163'800","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":3997,"log_dups_size":0,"ondisk_log_size":3997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:38.451494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":314376192,"num_objects":4797,"num_object_clones":0,"num_object_copies":14391,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4797,"num_write_kb":307008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"70'81","reported_seq":142,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.563181+0000","last_change":"2024-09-10T18:52:14.592382+0000","last_active":"2024-09-10T18:52:23.563181+0000","last_peered":"2024-09-10T18:52:23.563181+0000","last_clean":"2024-09-10T18:52:23.563181+0000","last_became_active":"2024-09-10T18:52:14.590633+0000","last_became_peered":"2024-09-10T18:52:14.590633+0000","last_unstale":"2024-09-10T18:52:23.563181+0000","last_undegraded":"2024-09-10T18:52:23.563181+0000","last_fullsized":"2024-09-10T18:52:23.563181+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:51:40.561066+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:51:40.561066+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001694397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"63'19","reported_seq":201,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.570448+0000","last_change":"2024-09-10T18:51:41.161251+0000","last_active":"2024-09-10T18:52:23.570448+0000","last_peered":"2024-09-10T18:52:23.570448+0000","last_clean":"2024-09-10T18:52:23.570448+0000","last_became_active":"2024-09-10T18:45:20.421046+0000","last_became_peered":"2024-09-10T18:45:20.421046+0000","last_unstale":"2024-09-10T18:52:23.570448+0000","last_undegraded":"2024-09-10T18:52:23.570448+0000","last_fullsized":"2024-09-10T18:52:23.570448+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:51:41.161196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:41.161196+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:57.195830+0000","scrub_duration":0.0023699070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":167,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.568221+0000","last_change":"2024-09-10T18:51:48.196125+0000","last_active":"2024-09-10T18:52:23.568221+0000","last_peered":"2024-09-10T18:52:23.568221+0000","last_clean":"2024-09-10T18:52:23.568221+0000","last_became_active":"2024-09-10T18:45:17.960226+0000","last_became_peered":"2024-09-10T18:45:17.960226+0000","last_unstale":"2024-09-10T18:52:23.568221+0000","last_undegraded":"2024-09-10T18:52:23.568221+0000","last_fullsized":"2024-09-10T18:52:23.568221+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:48.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:48.196065+0000","last_clean_scrub_stamp":"2024-09-10T18:51:48.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:10.250986+0000","scrub_duration":0.0013037999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":162,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.560766+0000","last_change":"2024-09-10T18:51:53.548363+0000","last_active":"2024-09-10T18:52:23.560766+0000","last_peered":"2024-09-10T18:52:23.560766+0000","last_clean":"2024-09-10T18:52:23.560766+0000","last_became_active":"2024-09-10T18:45:22.066148+0000","last_became_peered":"2024-09-10T18:45:22.066148+0000","last_unstale":"2024-09-10T18:52:23.560766+0000","last_undegraded":"2024-09-10T18:52:23.560766+0000","last_fullsized":"2024-09-10T18:52:23.560766+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:53.548279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:53.548279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:03.372538+0000","scrub_duration":0.0048693970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":161,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.570892+0000","last_change":"2024-09-10T18:51:43.053775+0000","last_active":"2024-09-10T18:52:23.570892+0000","last_peered":"2024-09-10T18:52:23.570892+0000","last_clean":"2024-09-10T18:52:23.570892+0000","last_became_active":"2024-09-10T18:47:36.422295+0000","last_became_peered":"2024-09-10T18:47:36.422295+0000","last_unstale":"2024-09-10T18:52:23.570892+0000","last_undegraded":"2024-09-10T18:52:23.570892+0000","last_fullsized":"2024-09-10T18:52:23.570892+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:43.053721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:49:18.281230+0000","last_clean_scrub_stamp":"2024-09-10T18:51:43.053721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:10.077998+0000","scrub_duration":0.001882325,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.e","version":"174'4731","reported_seq":4755,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-10T18:52:26.488466+0000","last_change":"2024-09-10T18:51:37.909961+0000","last_active":"2024-09-10T18:52:26.488466+0000","last_peered":"2024-09-10T18:52:26.488466+0000","last_clean":"2024-09-10T18:52:26.488466+0000","last_became_active":"2024-09-10T18:51:37.909734+0000","last_became_peered":"2024-09-10T18:51:37.909734+0000","last_unstale":"2024-09-10T18:52:26.488466+0000","last_undegraded":"2024-09-10T18:52:26.488466+0000","last_fullsized":"2024-09-10T18:52:26.488466+0000","mapping_epoch":158,"log_start":"163'200","ondisk_log_start":"163'200","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":4531,"log_dups_size":0,"ondisk_log_size":4531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:50.174570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":310050816,"num_objects":4731,"num_object_clones":0,"num_object_copies":14193,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4731,"num_write_kb":302784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"70'81","reported_seq":142,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.561300+0000","last_change":"2024-09-10T18:52:14.584467+0000","last_active":"2024-09-10T18:52:23.561300+0000","last_peered":"2024-09-10T18:52:23.561300+0000","last_clean":"2024-09-10T18:52:23.561300+0000","last_became_active":"2024-09-10T18:52:14.582796+0000","last_became_peered":"2024-09-10T18:52:14.582796+0000","last_unstale":"2024-09-10T18:52:23.561300+0000","last_undegraded":"2024-09-10T18:52:23.561300+0000","last_fullsized":"2024-09-10T18:52:23.561300+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:51:36.603950+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.603950+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013973970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"83'15","reported_seq":185,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.575080+0000","last_change":"2024-09-10T18:52:04.182859+0000","last_active":"2024-09-10T18:52:23.575080+0000","last_peered":"2024-09-10T18:52:23.575080+0000","last_clean":"2024-09-10T18:52:23.575080+0000","last_became_active":"2024-09-10T18:45:20.057204+0000","last_became_peered":"2024-09-10T18:45:20.057204+0000","last_unstale":"2024-09-10T18:52:23.575080+0000","last_undegraded":"2024-09-10T18:52:23.575080+0000","last_fullsized":"2024-09-10T18:52:23.575080+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'15","last_scrub_stamp":"2024-09-10T18:52:04.182782+0000","last_deep_scrub":"83'15","last_deep_scrub_stamp":"2024-09-10T18:46:58.436473+0000","last_clean_scrub_stamp":"2024-09-10T18:52:04.182782+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:24.316427+0000","scrub_duration":0.0049024990000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":157,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.571393+0000","last_change":"2024-09-10T18:51:05.140273+0000","last_active":"2024-09-10T18:52:23.571393+0000","last_peered":"2024-09-10T18:52:23.571393+0000","last_clean":"2024-09-10T18:52:23.571393+0000","last_became_active":"2024-09-10T18:45:17.966632+0000","last_became_peered":"2024-09-10T18:45:17.966632+0000","last_unstale":"2024-09-10T18:52:23.571393+0000","last_undegraded":"2024-09-10T18:52:23.571393+0000","last_fullsized":"2024-09-10T18:52:23.571393+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:05.140027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:05.140027+0000","last_clean_scrub_stamp":"2024-09-10T18:51:05.140027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:28.134498+0000","scrub_duration":0.0085472780000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"63'8","reported_seq":247,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-10T18:52:24.573439+0000","last_change":"2024-09-10T18:51:37.206285+0000","last_active":"2024-09-10T18:52:24.573439+0000","last_peered":"2024-09-10T18:52:24.573439+0000","last_clean":"2024-09-10T18:52:24.573439+0000","last_became_active":"2024-09-10T18:45:22.063326+0000","last_became_peered":"2024-09-10T18:45:22.063326+0000","last_unstale":"2024-09-10T18:52:24.573439+0000","last_undegraded":"2024-09-10T18:52:24.573439+0000","last_fullsized":"2024-09-10T18:52:24.573439+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:51:37.206224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:37.206224+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:06.464879+0000","scrub_duration":0.001760707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":161,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.569186+0000","last_change":"2024-09-10T18:51:01.618543+0000","last_active":"2024-09-10T18:52:23.569186+0000","last_peered":"2024-09-10T18:52:23.569186+0000","last_clean":"2024-09-10T18:52:23.569186+0000","last_became_active":"2024-09-10T18:47:36.412111+0000","last_became_peered":"2024-09-10T18:47:36.412111+0000","last_unstale":"2024-09-10T18:52:23.569186+0000","last_undegraded":"2024-09-10T18:52:23.569186+0000","last_fullsized":"2024-09-10T18:52:23.569186+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:01.618458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:01.618458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0072066109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"174'4775","reported_seq":4799,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-10T18:52:26.090570+0000","last_change":"2024-09-10T18:51:37.910891+0000","last_active":"2024-09-10T18:52:26.090570+0000","last_peered":"2024-09-10T18:52:26.090570+0000","last_clean":"2024-09-10T18:52:26.090570+0000","last_became_active":"2024-09-10T18:51:37.910355+0000","last_became_peered":"2024-09-10T18:51:37.910355+0000","last_unstale":"2024-09-10T18:52:26.090570+0000","last_undegraded":"2024-09-10T18:52:26.090570+0000","last_fullsized":"2024-09-10T18:52:26.090570+0000","mapping_epoch":158,"log_start":"163'700","ondisk_log_start":"163'700","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":4075,"log_dups_size":0,"ondisk_log_size":4075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:41.408841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":312934400,"num_objects":4775,"num_object_clones":0,"num_object_copies":14325,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4775,"num_write_kb":305600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"70'81","reported_seq":74,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.564877+0000","last_change":"2024-09-10T18:52:14.595038+0000","last_active":"2024-09-10T18:52:23.564877+0000","last_peered":"2024-09-10T18:52:23.564877+0000","last_clean":"2024-09-10T18:52:23.564877+0000","last_became_active":"2024-09-10T18:52:14.593853+0000","last_became_peered":"2024-09-10T18:52:14.593853+0000","last_unstale":"2024-09-10T18:52:23.564877+0000","last_undegraded":"2024-09-10T18:52:23.564877+0000","last_fullsized":"2024-09-10T18:52:23.564877+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:50:43.117728+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:50:43.117728+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039521169999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"89'29","reported_seq":331,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-10T18:52:24.567320+0000","last_change":"2024-09-10T18:51:57.140326+0000","last_active":"2024-09-10T18:52:24.567320+0000","last_peered":"2024-09-10T18:52:24.567320+0000","last_clean":"2024-09-10T18:52:24.567320+0000","last_became_active":"2024-09-10T18:47:36.420185+0000","last_became_peered":"2024-09-10T18:47:36.420185+0000","last_unstale":"2024-09-10T18:52:24.567320+0000","last_undegraded":"2024-09-10T18:52:24.567320+0000","last_fullsized":"2024-09-10T18:52:24.567320+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'29","last_scrub_stamp":"2024-09-10T18:51:57.140236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:57.140236+0000","objects_scrubbed":10,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:11.334267+0000","scrub_duration":0.012827955,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":166,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.565489+0000","last_change":"2024-09-10T18:51:56.185295+0000","last_active":"2024-09-10T18:52:23.565489+0000","last_peered":"2024-09-10T18:52:23.565489+0000","last_clean":"2024-09-10T18:52:23.565489+0000","last_became_active":"2024-09-10T18:45:17.964494+0000","last_became_peered":"2024-09-10T18:45:17.964494+0000","last_unstale":"2024-09-10T18:52:23.565489+0000","last_undegraded":"2024-09-10T18:52:23.565489+0000","last_fullsized":"2024-09-10T18:52:23.565489+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:56.185217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:12.982599+0000","last_clean_scrub_stamp":"2024-09-10T18:51:56.185217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:09.976307+0000","scrub_duration":0.0035214040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":107,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.560722+0000","last_change":"2024-09-10T18:51:01.357880+0000","last_active":"2024-09-10T18:52:23.560722+0000","last_peered":"2024-09-10T18:52:23.560722+0000","last_clean":"2024-09-10T18:52:23.560722+0000","last_became_active":"2024-09-10T18:47:57.107478+0000","last_became_peered":"2024-09-10T18:47:57.107478+0000","last_unstale":"2024-09-10T18:52:23.560722+0000","last_undegraded":"2024-09-10T18:52:23.560722+0000","last_fullsized":"2024-09-10T18:52:23.560722+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:01.357830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:01.357830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:28.477016+0000","scrub_duration":0.002118796,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":141,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.577061+0000","last_change":"2024-09-10T18:49:57.442744+0000","last_active":"2024-09-10T18:52:23.577061+0000","last_peered":"2024-09-10T18:52:23.577061+0000","last_clean":"2024-09-10T18:52:23.577061+0000","last_became_active":"2024-09-10T18:45:24.073307+0000","last_became_peered":"2024-09-10T18:45:24.073307+0000","last_unstale":"2024-09-10T18:52:23.577061+0000","last_undegraded":"2024-09-10T18:52:23.577061+0000","last_fullsized":"2024-09-10T18:52:23.577061+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:57.442512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:22.891084+0000","last_clean_scrub_stamp":"2024-09-10T18:49:57.442512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0050025080000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"174'4782","reported_seq":4806,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-10T18:52:26.394941+0000","last_change":"2024-09-10T18:51:37.583541+0000","last_active":"2024-09-10T18:52:26.394941+0000","last_peered":"2024-09-10T18:52:26.394941+0000","last_clean":"2024-09-10T18:52:26.394941+0000","last_became_active":"2024-09-10T18:51:37.583248+0000","last_became_peered":"2024-09-10T18:51:37.583248+0000","last_unstale":"2024-09-10T18:52:26.394941+0000","last_undegraded":"2024-09-10T18:52:26.394941+0000","last_fullsized":"2024-09-10T18:52:26.394941+0000","mapping_epoch":158,"log_start":"163'800","ondisk_log_start":"163'800","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":3982,"log_dups_size":0,"ondisk_log_size":3982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:01.429540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":313393152,"num_objects":4782,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4782,"num_write_kb":306048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"70'81","reported_seq":231,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.567276+0000","last_change":"2024-09-10T18:52:14.576467+0000","last_active":"2024-09-10T18:52:23.567276+0000","last_peered":"2024-09-10T18:52:23.567276+0000","last_clean":"2024-09-10T18:52:23.567276+0000","last_became_active":"2024-09-10T18:52:14.576152+0000","last_became_peered":"2024-09-10T18:52:14.576152+0000","last_unstale":"2024-09-10T18:52:23.567276+0000","last_undegraded":"2024-09-10T18:52:23.567276+0000","last_fullsized":"2024-09-10T18:52:23.567276+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:50:50.506769+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:50:50.506769+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043821199999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"84'18","reported_seq":187,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.575458+0000","last_change":"2024-09-10T18:51:37.059537+0000","last_active":"2024-09-10T18:52:23.575458+0000","last_peered":"2024-09-10T18:52:23.575458+0000","last_clean":"2024-09-10T18:52:23.575458+0000","last_became_active":"2024-09-10T18:45:20.058435+0000","last_became_peered":"2024-09-10T18:45:20.058435+0000","last_unstale":"2024-09-10T18:52:23.575458+0000","last_undegraded":"2024-09-10T18:52:23.575458+0000","last_fullsized":"2024-09-10T18:52:23.575458+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'18","last_scrub_stamp":"2024-09-10T18:51:37.059477+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-10T18:47:10.514951+0000","last_clean_scrub_stamp":"2024-09-10T18:51:37.059477+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:06.269219+0000","scrub_duration":0.0020902970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"84'6","reported_seq":599,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-10T18:52:25.105931+0000","last_change":"2024-09-10T18:51:49.096654+0000","last_active":"2024-09-10T18:52:25.105931+0000","last_peered":"2024-09-10T18:52:25.105931+0000","last_clean":"2024-09-10T18:52:25.105931+0000","last_became_active":"2024-09-10T18:47:36.418211+0000","last_became_peered":"2024-09-10T18:47:36.418211+0000","last_unstale":"2024-09-10T18:52:25.105931+0000","last_undegraded":"2024-09-10T18:52:25.105931+0000","last_fullsized":"2024-09-10T18:52:25.105931+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'6","last_scrub_stamp":"2024-09-10T18:51:49.096535+0000","last_deep_scrub":"84'6","last_deep_scrub_stamp":"2024-09-10T18:47:22.545450+0000","last_clean_scrub_stamp":"2024-09-10T18:51:49.096535+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:14.910896+0000","scrub_duration":0.0037496439999999999,"objects_trimmed":0,"snaptrim_duration":0.0021933159999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":418,"num_read_kb":413,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":118,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.574384+0000","last_change":"2024-09-10T18:51:44.101630+0000","last_active":"2024-09-10T18:52:23.574384+0000","last_peered":"2024-09-10T18:52:23.574384+0000","last_clean":"2024-09-10T18:52:23.574384+0000","last_became_active":"2024-09-10T18:47:36.422641+0000","last_became_peered":"2024-09-10T18:47:36.422641+0000","last_unstale":"2024-09-10T18:52:23.574384+0000","last_undegraded":"2024-09-10T18:52:23.574384+0000","last_fullsized":"2024-09-10T18:52:23.574384+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:44.101575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:44.101575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:48.501520+0000","scrub_duration":0.0012333190000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":127,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.564206+0000","last_change":"2024-09-10T18:51:57.206683+0000","last_active":"2024-09-10T18:52:23.564206+0000","last_peered":"2024-09-10T18:52:23.564206+0000","last_clean":"2024-09-10T18:52:23.564206+0000","last_became_active":"2024-09-10T18:47:36.419081+0000","last_became_peered":"2024-09-10T18:47:36.419081+0000","last_unstale":"2024-09-10T18:52:23.564206+0000","last_undegraded":"2024-09-10T18:52:23.564206+0000","last_fullsized":"2024-09-10T18:52:23.564206+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:57.206586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:57.206586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:21.908977+0000","scrub_duration":0.00597548,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":151,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.567902+0000","last_change":"2024-09-10T18:51:35.398902+0000","last_active":"2024-09-10T18:52:23.567902+0000","last_peered":"2024-09-10T18:52:23.567902+0000","last_clean":"2024-09-10T18:52:23.567902+0000","last_became_active":"2024-09-10T18:45:24.264617+0000","last_became_peered":"2024-09-10T18:45:24.264617+0000","last_unstale":"2024-09-10T18:52:23.567902+0000","last_undegraded":"2024-09-10T18:52:23.567902+0000","last_fullsized":"2024-09-10T18:52:23.567902+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:35.398819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:35.398819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:41.839709+0000","scrub_duration":0.0014954090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"174'4724","reported_seq":4748,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-10T18:52:26.308507+0000","last_change":"2024-09-10T18:51:37.589525+0000","last_active":"2024-09-10T18:52:26.308507+0000","last_peered":"2024-09-10T18:52:26.308507+0000","last_clean":"2024-09-10T18:52:26.308507+0000","last_became_active":"2024-09-10T18:51:37.588751+0000","last_became_peered":"2024-09-10T18:51:37.588751+0000","last_unstale":"2024-09-10T18:52:26.308507+0000","last_undegraded":"2024-09-10T18:52:26.308507+0000","last_fullsized":"2024-09-10T18:52:26.308507+0000","mapping_epoch":158,"log_start":"163'600","ondisk_log_start":"163'600","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":4124,"log_dups_size":0,"ondisk_log_size":4124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:57.311670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":309592064,"num_objects":4724,"num_object_clones":0,"num_object_copies":14172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4724,"num_write_kb":302336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"70'81","reported_seq":74,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.572365+0000","last_change":"2024-09-10T18:52:14.590061+0000","last_active":"2024-09-10T18:52:23.572365+0000","last_peered":"2024-09-10T18:52:23.572365+0000","last_clean":"2024-09-10T18:52:23.572365+0000","last_became_active":"2024-09-10T18:52:14.589009+0000","last_became_peered":"2024-09-10T18:52:14.589009+0000","last_unstale":"2024-09-10T18:52:23.572365+0000","last_undegraded":"2024-09-10T18:52:23.572365+0000","last_fullsized":"2024-09-10T18:52:23.572365+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:51:04.099703+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:51:04.099703+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007621203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"63'12","reported_seq":207,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.571777+0000","last_change":"2024-09-10T18:52:01.328924+0000","last_active":"2024-09-10T18:52:23.571777+0000","last_peered":"2024-09-10T18:52:23.571777+0000","last_clean":"2024-09-10T18:52:23.571777+0000","last_became_active":"2024-09-10T18:47:36.417965+0000","last_became_peered":"2024-09-10T18:47:36.417965+0000","last_unstale":"2024-09-10T18:52:23.571777+0000","last_undegraded":"2024-09-10T18:52:23.571777+0000","last_fullsized":"2024-09-10T18:52:23.571777+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:52:01.328828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:52:01.328828+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:27.642992+0000","scrub_duration":0.020091725000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"58'1","reported_seq":163,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.569610+0000","last_change":"2024-09-10T18:51:39.175354+0000","last_active":"2024-09-10T18:52:23.569610+0000","last_peered":"2024-09-10T18:52:23.569610+0000","last_clean":"2024-09-10T18:52:23.569610+0000","last_became_active":"2024-09-10T18:45:17.963932+0000","last_became_peered":"2024-09-10T18:45:17.963932+0000","last_unstale":"2024-09-10T18:52:23.569610+0000","last_undegraded":"2024-09-10T18:52:23.569610+0000","last_fullsized":"2024-09-10T18:52:23.569610+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'1","last_scrub_stamp":"2024-09-10T18:51:39.175290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:39.175290+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:03.334161+0000","scrub_duration":0.0015267550000000001,"objects_trimmed":0,"snaptrim_duration":0.00037812099999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":115,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.566890+0000","last_change":"2024-09-10T18:51:30.467050+0000","last_active":"2024-09-10T18:52:23.566890+0000","last_peered":"2024-09-10T18:52:23.566890+0000","last_clean":"2024-09-10T18:52:23.566890+0000","last_became_active":"2024-09-10T18:47:36.416641+0000","last_became_peered":"2024-09-10T18:47:36.416641+0000","last_unstale":"2024-09-10T18:52:23.566890+0000","last_undegraded":"2024-09-10T18:52:23.566890+0000","last_fullsized":"2024-09-10T18:52:23.566890+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:30.466988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:30.466988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:44.004240+0000","scrub_duration":0.0019674089999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":152,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.564845+0000","last_change":"2024-09-10T18:50:52.999936+0000","last_active":"2024-09-10T18:52:23.564845+0000","last_peered":"2024-09-10T18:52:23.564845+0000","last_clean":"2024-09-10T18:52:23.564845+0000","last_became_active":"2024-09-10T18:45:22.064285+0000","last_became_peered":"2024-09-10T18:45:22.064285+0000","last_unstale":"2024-09-10T18:52:23.564845+0000","last_undegraded":"2024-09-10T18:52:23.564845+0000","last_fullsized":"2024-09-10T18:52:23.564845+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:52.999842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:52.999842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0065214100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":151,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.571034+0000","last_change":"2024-09-10T18:50:53.940855+0000","last_active":"2024-09-10T18:52:23.571034+0000","last_peered":"2024-09-10T18:52:23.571034+0000","last_clean":"2024-09-10T18:52:23.571034+0000","last_became_active":"2024-09-10T18:45:24.083689+0000","last_became_peered":"2024-09-10T18:45:24.083689+0000","last_unstale":"2024-09-10T18:52:23.571034+0000","last_undegraded":"2024-09-10T18:52:23.571034+0000","last_fullsized":"2024-09-10T18:52:23.571034+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:53.940489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:50:53.940489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007171691,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"174'4684","reported_seq":4708,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-10T18:52:26.140536+0000","last_change":"2024-09-10T18:51:37.591719+0000","last_active":"2024-09-10T18:52:26.140536+0000","last_peered":"2024-09-10T18:52:26.140536+0000","last_clean":"2024-09-10T18:52:26.140536+0000","last_became_active":"2024-09-10T18:51:37.591441+0000","last_became_peered":"2024-09-10T18:51:37.591441+0000","last_unstale":"2024-09-10T18:52:26.140536+0000","last_undegraded":"2024-09-10T18:52:26.140536+0000","last_fullsized":"2024-09-10T18:52:26.140536+0000","mapping_epoch":158,"log_start":"163'300","ondisk_log_start":"163'300","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":4384,"log_dups_size":0,"ondisk_log_size":4384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:43.944535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306970624,"num_objects":4684,"num_object_clones":0,"num_object_copies":14052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4684,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4684,"num_write_kb":299776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"126'21","reported_seq":193,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.560946+0000","last_change":"2024-09-10T18:51:52.576493+0000","last_active":"2024-09-10T18:52:23.560946+0000","last_peered":"2024-09-10T18:52:23.560946+0000","last_clean":"2024-09-10T18:52:23.560946+0000","last_became_active":"2024-09-10T18:45:20.421119+0000","last_became_peered":"2024-09-10T18:45:20.421119+0000","last_unstale":"2024-09-10T18:52:23.560946+0000","last_undegraded":"2024-09-10T18:52:23.560946+0000","last_fullsized":"2024-09-10T18:52:23.560946+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"126'21","last_scrub_stamp":"2024-09-10T18:51:52.576358+0000","last_deep_scrub":"126'21","last_deep_scrub_stamp":"2024-09-10T18:50:03.380902+0000","last_clean_scrub_stamp":"2024-09-10T18:51:52.576358+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:02.684753+0000","scrub_duration":0.0054386800000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"60'2","reported_seq":164,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.574465+0000","last_change":"2024-09-10T18:51:03.104641+0000","last_active":"2024-09-10T18:52:23.574465+0000","last_peered":"2024-09-10T18:52:23.574465+0000","last_clean":"2024-09-10T18:52:23.574465+0000","last_became_active":"2024-09-10T18:45:17.961660+0000","last_became_peered":"2024-09-10T18:45:17.961660+0000","last_unstale":"2024-09-10T18:52:23.574465+0000","last_undegraded":"2024-09-10T18:52:23.574465+0000","last_fullsized":"2024-09-10T18:52:23.574465+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-10T18:51:03.104545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:03.104545+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0070342290000000003,"objects_trimmed":0,"snaptrim_duration":0.0013942640000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":176,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.566031+0000","last_change":"2024-09-10T18:52:01.209203+0000","last_active":"2024-09-10T18:52:23.566031+0000","last_peered":"2024-09-10T18:52:23.566031+0000","last_clean":"2024-09-10T18:52:23.566031+0000","last_became_active":"2024-09-10T18:47:36.414636+0000","last_became_peered":"2024-09-10T18:47:36.414636+0000","last_unstale":"2024-09-10T18:52:23.566031+0000","last_undegraded":"2024-09-10T18:52:23.566031+0000","last_fullsized":"2024-09-10T18:52:23.566031+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:01.209144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:52:01.209144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:27.978547+0000","scrub_duration":0.0033612889999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'81","reported_seq":222,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.567200+0000","last_change":"2024-09-10T18:52:14.581591+0000","last_active":"2024-09-10T18:52:23.567200+0000","last_peered":"2024-09-10T18:52:23.567200+0000","last_clean":"2024-09-10T18:52:23.567200+0000","last_became_active":"2024-09-10T18:52:14.579154+0000","last_became_peered":"2024-09-10T18:52:14.579154+0000","last_unstale":"2024-09-10T18:52:23.567200+0000","last_undegraded":"2024-09-10T18:52:23.567200+0000","last_fullsized":"2024-09-10T18:52:23.567200+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:50:48.508244+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:50:48.508244+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0035856120000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":173,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.567791+0000","last_change":"2024-09-10T18:51:40.327714+0000","last_active":"2024-09-10T18:52:23.567791+0000","last_peered":"2024-09-10T18:52:23.567791+0000","last_clean":"2024-09-10T18:52:23.567791+0000","last_became_active":"2024-09-10T18:47:36.418320+0000","last_became_peered":"2024-09-10T18:47:36.418320+0000","last_unstale":"2024-09-10T18:52:23.567791+0000","last_undegraded":"2024-09-10T18:52:23.567791+0000","last_fullsized":"2024-09-10T18:52:23.567791+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:40.327645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:40.327645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:58.610158+0000","scrub_duration":0.001391827,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":150,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.565183+0000","last_change":"2024-09-10T18:51:02.170258+0000","last_active":"2024-09-10T18:52:23.565183+0000","last_peered":"2024-09-10T18:52:23.565183+0000","last_clean":"2024-09-10T18:52:23.565183+0000","last_became_active":"2024-09-10T18:45:24.073327+0000","last_became_peered":"2024-09-10T18:45:24.073327+0000","last_unstale":"2024-09-10T18:52:23.565183+0000","last_undegraded":"2024-09-10T18:52:23.565183+0000","last_fullsized":"2024-09-10T18:52:23.565183+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:02.170065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:02.170065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0061532089999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.f","version":"70'81","reported_seq":151,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.561606+0000","last_change":"2024-09-10T18:52:16.598800+0000","last_active":"2024-09-10T18:52:23.561606+0000","last_peered":"2024-09-10T18:52:23.561606+0000","last_clean":"2024-09-10T18:52:23.561606+0000","last_became_active":"2024-09-10T18:52:16.598295+0000","last_became_peered":"2024-09-10T18:52:16.598295+0000","last_unstale":"2024-09-10T18:52:23.561606+0000","last_undegraded":"2024-09-10T18:52:23.561606+0000","last_fullsized":"2024-09-10T18:52:23.561606+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:51:36.603950+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.603950+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013973970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.6","version":"174'4859","reported_seq":4883,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-10T18:52:26.525079+0000","last_change":"2024-09-10T18:51:37.909963+0000","last_active":"2024-09-10T18:52:26.525079+0000","last_peered":"2024-09-10T18:52:26.525079+0000","last_clean":"2024-09-10T18:52:26.525079+0000","last_became_active":"2024-09-10T18:51:37.909738+0000","last_became_peered":"2024-09-10T18:51:37.909738+0000","last_unstale":"2024-09-10T18:52:26.525079+0000","last_undegraded":"2024-09-10T18:52:26.525079+0000","last_fullsized":"2024-09-10T18:52:26.525079+0000","mapping_epoch":158,"log_start":"163'300","ondisk_log_start":"163'300","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":4559,"log_dups_size":0,"ondisk_log_size":4559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:03.481643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":318439424,"num_objects":4859,"num_object_clones":0,"num_object_copies":14577,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4859,"num_write_kb":310976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"63'19","reported_seq":195,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.560968+0000","last_change":"2024-09-10T18:52:03.486438+0000","last_active":"2024-09-10T18:52:23.560968+0000","last_peered":"2024-09-10T18:52:23.560968+0000","last_clean":"2024-09-10T18:52:23.560968+0000","last_became_active":"2024-09-10T18:45:20.421097+0000","last_became_peered":"2024-09-10T18:45:20.421097+0000","last_unstale":"2024-09-10T18:52:23.560968+0000","last_undegraded":"2024-09-10T18:52:23.560968+0000","last_fullsized":"2024-09-10T18:52:23.560968+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:52:03.486356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:52:03.486356+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:30.272925+0000","scrub_duration":0.013086754000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":117,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.565296+0000","last_change":"2024-09-10T18:50:47.083728+0000","last_active":"2024-09-10T18:52:23.565296+0000","last_peered":"2024-09-10T18:52:23.565296+0000","last_clean":"2024-09-10T18:52:23.565296+0000","last_became_active":"2024-09-10T18:47:36.417800+0000","last_became_peered":"2024-09-10T18:47:36.417800+0000","last_unstale":"2024-09-10T18:52:23.565296+0000","last_undegraded":"2024-09-10T18:52:23.565296+0000","last_fullsized":"2024-09-10T18:52:23.565296+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:47.083643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:47.083643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0071207509999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":128,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.570238+0000","last_change":"2024-09-10T18:51:40.126736+0000","last_active":"2024-09-10T18:52:23.570238+0000","last_peered":"2024-09-10T18:52:23.570238+0000","last_clean":"2024-09-10T18:52:23.570238+0000","last_became_active":"2024-09-10T18:47:36.413618+0000","last_became_peered":"2024-09-10T18:47:36.413618+0000","last_unstale":"2024-09-10T18:52:23.570238+0000","last_undegraded":"2024-09-10T18:52:23.570238+0000","last_fullsized":"2024-09-10T18:52:23.570238+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:40.126652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:40.126652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:44.820089+0000","scrub_duration":0.0017754699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":161,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.567171+0000","last_change":"2024-09-10T18:51:38.318952+0000","last_active":"2024-09-10T18:52:23.567171+0000","last_peered":"2024-09-10T18:52:23.567171+0000","last_clean":"2024-09-10T18:52:23.567171+0000","last_became_active":"2024-09-10T18:47:36.412610+0000","last_became_peered":"2024-09-10T18:47:36.412610+0000","last_unstale":"2024-09-10T18:52:23.567171+0000","last_undegraded":"2024-09-10T18:52:23.567171+0000","last_fullsized":"2024-09-10T18:52:23.567171+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:38.318887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:38.318887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:47.048859+0000","scrub_duration":0.001335461,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.e","version":"70'81","reported_seq":14,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.576729+0000","last_change":"2024-09-10T18:52:16.604754+0000","last_active":"2024-09-10T18:52:23.576729+0000","last_peered":"2024-09-10T18:52:23.576729+0000","last_clean":"2024-09-10T18:52:23.576729+0000","last_became_active":"2024-09-10T18:52:16.604236+0000","last_became_peered":"2024-09-10T18:52:16.604236+0000","last_unstale":"2024-09-10T18:52:23.576729+0000","last_undegraded":"2024-09-10T18:52:23.576729+0000","last_fullsized":"2024-09-10T18:52:23.576729+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:51:40.561066+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:51:40.561066+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019876809999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":5,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":53670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"174'4633","reported_seq":4657,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-10T18:52:26.325910+0000","last_change":"2024-09-10T18:51:37.590054+0000","last_active":"2024-09-10T18:52:26.325910+0000","last_peered":"2024-09-10T18:52:26.325910+0000","last_clean":"2024-09-10T18:52:26.325910+0000","last_became_active":"2024-09-10T18:51:37.589351+0000","last_became_peered":"2024-09-10T18:51:37.589351+0000","last_unstale":"2024-09-10T18:52:26.325910+0000","last_undegraded":"2024-09-10T18:52:26.325910+0000","last_fullsized":"2024-09-10T18:52:26.325910+0000","mapping_epoch":158,"log_start":"163'500","ondisk_log_start":"163'500","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":4133,"log_dups_size":0,"ondisk_log_size":4133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:52.988787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":303628288,"num_objects":4633,"num_object_clones":0,"num_object_copies":13899,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4633,"num_write_kb":296512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"130'13","reported_seq":185,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.570377+0000","last_change":"2024-09-10T18:51:58.427843+0000","last_active":"2024-09-10T18:52:23.570377+0000","last_peered":"2024-09-10T18:52:23.570377+0000","last_clean":"2024-09-10T18:52:23.570377+0000","last_became_active":"2024-09-10T18:45:20.061021+0000","last_became_peered":"2024-09-10T18:45:20.061021+0000","last_unstale":"2024-09-10T18:52:23.570377+0000","last_undegraded":"2024-09-10T18:52:23.570377+0000","last_fullsized":"2024-09-10T18:52:23.570377+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"130'13","last_scrub_stamp":"2024-09-10T18:51:58.427534+0000","last_deep_scrub":"130'13","last_deep_scrub_stamp":"2024-09-10T18:51:58.427534+0000","last_clean_scrub_stamp":"2024-09-10T18:51:58.427534+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:28.183231+0000","scrub_duration":0.0051175980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":157,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.571363+0000","last_change":"2024-09-10T18:51:31.128571+0000","last_active":"2024-09-10T18:52:23.571363+0000","last_peered":"2024-09-10T18:52:23.571363+0000","last_clean":"2024-09-10T18:52:23.571363+0000","last_became_active":"2024-09-10T18:45:17.963057+0000","last_became_peered":"2024-09-10T18:45:17.963057+0000","last_unstale":"2024-09-10T18:52:23.571363+0000","last_undegraded":"2024-09-10T18:52:23.571363+0000","last_fullsized":"2024-09-10T18:52:23.571363+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:31.128514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:31.128514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:55.374112+0000","scrub_duration":0.0016209180000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":116,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.580069+0000","last_change":"2024-09-10T18:51:35.325521+0000","last_active":"2024-09-10T18:52:23.580069+0000","last_peered":"2024-09-10T18:52:23.580069+0000","last_clean":"2024-09-10T18:52:23.580069+0000","last_became_active":"2024-09-10T18:47:36.411517+0000","last_became_peered":"2024-09-10T18:47:36.411517+0000","last_unstale":"2024-09-10T18:52:23.580069+0000","last_undegraded":"2024-09-10T18:52:23.580069+0000","last_fullsized":"2024-09-10T18:52:23.580069+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:35.325451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_clean_scrub_stamp":"2024-09-10T18:51:35.325451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:56.285365+0000","scrub_duration":0.0018247820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":171,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.569531+0000","last_change":"2024-09-10T18:51:53.345267+0000","last_active":"2024-09-10T18:52:23.569531+0000","last_peered":"2024-09-10T18:52:23.569531+0000","last_clean":"2024-09-10T18:52:23.569531+0000","last_became_active":"2024-09-10T18:47:36.412933+0000","last_became_peered":"2024-09-10T18:47:36.412933+0000","last_unstale":"2024-09-10T18:52:23.569531+0000","last_undegraded":"2024-09-10T18:52:23.569531+0000","last_fullsized":"2024-09-10T18:52:23.569531+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:53.345184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:53.345184+0000","last_clean_scrub_stamp":"2024-09-10T18:51:53.345184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:19.403333+0000","scrub_duration":0.003832687,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.d","version":"70'81","reported_seq":14,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.567742+0000","last_change":"2024-09-10T18:52:16.609729+0000","last_active":"2024-09-10T18:52:23.567742+0000","last_peered":"2024-09-10T18:52:23.567742+0000","last_clean":"2024-09-10T18:52:23.567742+0000","last_became_active":"2024-09-10T18:52:16.605492+0000","last_became_peered":"2024-09-10T18:52:16.605492+0000","last_unstale":"2024-09-10T18:52:23.567742+0000","last_undegraded":"2024-09-10T18:52:23.567742+0000","last_fullsized":"2024-09-10T18:52:23.567742+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:51:01.489919+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:51:01.489919+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014613810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.4","version":"174'4780","reported_seq":4804,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-10T18:52:26.067239+0000","last_change":"2024-09-10T18:51:37.590489+0000","last_active":"2024-09-10T18:52:26.067239+0000","last_peered":"2024-09-10T18:52:26.067239+0000","last_clean":"2024-09-10T18:52:26.067239+0000","last_became_active":"2024-09-10T18:51:37.590189+0000","last_became_peered":"2024-09-10T18:51:37.590189+0000","last_unstale":"2024-09-10T18:52:26.067239+0000","last_undegraded":"2024-09-10T18:52:26.067239+0000","last_fullsized":"2024-09-10T18:52:26.067239+0000","mapping_epoch":158,"log_start":"163'700","ondisk_log_start":"163'700","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":4080,"log_dups_size":0,"ondisk_log_size":4080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:47.653862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":313262080,"num_objects":4780,"num_object_clones":0,"num_object_copies":14340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4780,"num_write_kb":305920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"63'15","reported_seq":179,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.565829+0000","last_change":"2024-09-10T18:51:36.277582+0000","last_active":"2024-09-10T18:52:23.565829+0000","last_peered":"2024-09-10T18:52:23.565829+0000","last_clean":"2024-09-10T18:52:23.565829+0000","last_became_active":"2024-09-10T18:45:20.419804+0000","last_became_peered":"2024-09-10T18:45:20.419804+0000","last_unstale":"2024-09-10T18:52:23.565829+0000","last_undegraded":"2024-09-10T18:52:23.565829+0000","last_fullsized":"2024-09-10T18:52:23.565829+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:51:36.277517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.277517+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:47.932042+0000","scrub_duration":0.0020736750000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":108,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.577131+0000","last_change":"2024-09-10T18:50:05.603208+0000","last_active":"2024-09-10T18:52:23.577131+0000","last_peered":"2024-09-10T18:52:23.577131+0000","last_clean":"2024-09-10T18:52:23.577131+0000","last_became_active":"2024-09-10T18:47:36.420475+0000","last_became_peered":"2024-09-10T18:47:36.420475+0000","last_unstale":"2024-09-10T18:52:23.577131+0000","last_undegraded":"2024-09-10T18:52:23.577131+0000","last_fullsized":"2024-09-10T18:52:23.577131+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:05.603125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:05.603125+0000","last_clean_scrub_stamp":"2024-09-10T18:50:05.603125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048485560000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"83'9","reported_seq":250,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.567270+0000","last_change":"2024-09-10T18:50:10.444294+0000","last_active":"2024-09-10T18:52:23.567270+0000","last_peered":"2024-09-10T18:52:23.567270+0000","last_clean":"2024-09-10T18:52:23.567270+0000","last_became_active":"2024-09-10T18:45:22.073383+0000","last_became_peered":"2024-09-10T18:45:22.073383+0000","last_unstale":"2024-09-10T18:52:23.567270+0000","last_undegraded":"2024-09-10T18:52:23.567270+0000","last_fullsized":"2024-09-10T18:52:23.567270+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'9","last_scrub_stamp":"2024-09-10T18:50:10.444201+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-09-10T18:47:02.582229+0000","last_clean_scrub_stamp":"2024-09-10T18:50:10.444201+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014241271,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":150,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.564370+0000","last_change":"2024-09-10T18:51:05.187154+0000","last_active":"2024-09-10T18:52:23.564370+0000","last_peered":"2024-09-10T18:52:23.564370+0000","last_clean":"2024-09-10T18:52:23.564370+0000","last_became_active":"2024-09-10T18:45:24.265256+0000","last_became_peered":"2024-09-10T18:45:24.265256+0000","last_unstale":"2024-09-10T18:52:23.564370+0000","last_undegraded":"2024-09-10T18:52:23.564370+0000","last_fullsized":"2024-09-10T18:52:23.564370+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:05.187073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_clean_scrub_stamp":"2024-09-10T18:51:05.187073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00225728,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.c","version":"70'81","reported_seq":94,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.575646+0000","last_change":"2024-09-10T18:52:16.610809+0000","last_active":"2024-09-10T18:52:23.575646+0000","last_peered":"2024-09-10T18:52:23.575646+0000","last_clean":"2024-09-10T18:52:23.575646+0000","last_became_active":"2024-09-10T18:52:16.596677+0000","last_became_peered":"2024-09-10T18:52:16.596677+0000","last_unstale":"2024-09-10T18:52:23.575646+0000","last_undegraded":"2024-09-10T18:52:23.575646+0000","last_fullsized":"2024-09-10T18:52:23.575646+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:50:59.164417+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:50:59.164417+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0063541700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109227,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"174'4819","reported_seq":4843,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-10T18:52:26.133629+0000","last_change":"2024-09-10T18:51:37.910484+0000","last_active":"2024-09-10T18:52:26.133629+0000","last_peered":"2024-09-10T18:52:26.133629+0000","last_clean":"2024-09-10T18:52:26.133629+0000","last_became_active":"2024-09-10T18:51:37.910181+0000","last_became_peered":"2024-09-10T18:51:37.910181+0000","last_unstale":"2024-09-10T18:52:26.133629+0000","last_undegraded":"2024-09-10T18:52:26.133629+0000","last_fullsized":"2024-09-10T18:52:26.133629+0000","mapping_epoch":158,"log_start":"163'400","ondisk_log_start":"163'400","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":4419,"log_dups_size":0,"ondisk_log_size":4419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:43.239258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":315817984,"num_objects":4819,"num_object_clones":0,"num_object_copies":14457,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4819,"num_write_kb":308416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"63'12","reported_seq":176,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.576473+0000","last_change":"2024-09-10T18:50:12.595966+0000","last_active":"2024-09-10T18:52:23.576473+0000","last_peered":"2024-09-10T18:52:23.576473+0000","last_clean":"2024-09-10T18:52:23.576473+0000","last_became_active":"2024-09-10T18:45:20.056869+0000","last_became_peered":"2024-09-10T18:45:20.056869+0000","last_unstale":"2024-09-10T18:52:23.576473+0000","last_undegraded":"2024-09-10T18:52:23.576473+0000","last_fullsized":"2024-09-10T18:52:23.576473+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:50:12.595881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:50:12.595881+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013834673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"56'2","reported_seq":172,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.577572+0000","last_change":"2024-09-10T18:51:42.425065+0000","last_active":"2024-09-10T18:52:23.577572+0000","last_peered":"2024-09-10T18:52:23.577572+0000","last_clean":"2024-09-10T18:52:23.577572+0000","last_became_active":"2024-09-10T18:45:17.963286+0000","last_became_peered":"2024-09-10T18:45:17.963286+0000","last_unstale":"2024-09-10T18:52:23.577572+0000","last_undegraded":"2024-09-10T18:52:23.577572+0000","last_fullsized":"2024-09-10T18:52:23.577572+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-10T18:51:42.424994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:42.424994+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:10.091197+0000","scrub_duration":0.001852226,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":128,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.569271+0000","last_change":"2024-09-10T18:51:54.343030+0000","last_active":"2024-09-10T18:52:23.569271+0000","last_peered":"2024-09-10T18:52:23.569271+0000","last_clean":"2024-09-10T18:52:23.569271+0000","last_became_active":"2024-09-10T18:47:36.421151+0000","last_became_peered":"2024-09-10T18:47:36.421151+0000","last_unstale":"2024-09-10T18:52:23.569271+0000","last_undegraded":"2024-09-10T18:52:23.569271+0000","last_fullsized":"2024-09-10T18:52:23.569271+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:54.342934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:02.711594+0000","last_clean_scrub_stamp":"2024-09-10T18:51:54.342934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:23.514769+0000","scrub_duration":0.005738529,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":151,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.574876+0000","last_change":"2024-09-10T18:51:48.065441+0000","last_active":"2024-09-10T18:52:23.574876+0000","last_peered":"2024-09-10T18:52:23.574876+0000","last_clean":"2024-09-10T18:52:23.574876+0000","last_became_active":"2024-09-10T18:45:24.077815+0000","last_became_peered":"2024-09-10T18:45:24.077815+0000","last_unstale":"2024-09-10T18:52:23.574876+0000","last_undegraded":"2024-09-10T18:52:23.574876+0000","last_fullsized":"2024-09-10T18:52:23.574876+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:48.065385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:48.065385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:04.487946+0000","scrub_duration":0.0014615870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.13","version":"70'81","reported_seq":83,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.572496+0000","last_change":"2024-09-10T18:52:16.603691+0000","last_active":"2024-09-10T18:52:23.572496+0000","last_peered":"2024-09-10T18:52:23.572496+0000","last_clean":"2024-09-10T18:52:23.572496+0000","last_became_active":"2024-09-10T18:52:16.602463+0000","last_became_peered":"2024-09-10T18:52:16.602463+0000","last_unstale":"2024-09-10T18:52:23.572496+0000","last_undegraded":"2024-09-10T18:52:23.572496+0000","last_fullsized":"2024-09-10T18:52:23.572496+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:51:04.099703+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:51:04.099703+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007621203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"76'13","reported_seq":185,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.571708+0000","last_change":"2024-09-10T18:51:56.389512+0000","last_active":"2024-09-10T18:52:23.571708+0000","last_peered":"2024-09-10T18:52:23.571708+0000","last_clean":"2024-09-10T18:52:23.571708+0000","last_became_active":"2024-09-10T18:45:20.420421+0000","last_became_peered":"2024-09-10T18:45:20.420421+0000","last_unstale":"2024-09-10T18:52:23.571708+0000","last_undegraded":"2024-09-10T18:52:23.571708+0000","last_fullsized":"2024-09-10T18:52:23.571708+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'13","last_scrub_stamp":"2024-09-10T18:51:56.389281+0000","last_deep_scrub":"76'13","last_deep_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_clean_scrub_stamp":"2024-09-10T18:51:56.389281+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:59.529351+0000","scrub_duration":0.0031275529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":128,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.571668+0000","last_change":"2024-09-10T18:52:05.160227+0000","last_active":"2024-09-10T18:52:23.571668+0000","last_peered":"2024-09-10T18:52:23.571668+0000","last_clean":"2024-09-10T18:52:23.571668+0000","last_became_active":"2024-09-10T18:47:36.409299+0000","last_became_peered":"2024-09-10T18:47:36.409299+0000","last_unstale":"2024-09-10T18:52:23.571668+0000","last_undegraded":"2024-09-10T18:52:23.571668+0000","last_fullsized":"2024-09-10T18:52:23.571668+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:05.160144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:52:05.160144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:05.185262+0000","scrub_duration":0.0041652920000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":153,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.578595+0000","last_change":"2024-09-10T18:50:57.190078+0000","last_active":"2024-09-10T18:52:23.578595+0000","last_peered":"2024-09-10T18:52:23.578595+0000","last_clean":"2024-09-10T18:52:23.578595+0000","last_became_active":"2024-09-10T18:45:22.069040+0000","last_became_peered":"2024-09-10T18:45:22.069040+0000","last_unstale":"2024-09-10T18:52:23.578595+0000","last_undegraded":"2024-09-10T18:52:23.578595+0000","last_fullsized":"2024-09-10T18:52:23.578595+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:57.189976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:28.442370+0000","last_clean_scrub_stamp":"2024-09-10T18:50:57.189976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016295923,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"63'1","reported_seq":132,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.577467+0000","last_change":"2024-09-10T18:51:57.216173+0000","last_active":"2024-09-10T18:52:23.577467+0000","last_peered":"2024-09-10T18:52:23.577467+0000","last_clean":"2024-09-10T18:52:23.577467+0000","last_became_active":"2024-09-10T18:47:36.406936+0000","last_became_peered":"2024-09-10T18:47:36.406936+0000","last_unstale":"2024-09-10T18:52:23.577467+0000","last_undegraded":"2024-09-10T18:52:23.577467+0000","last_fullsized":"2024-09-10T18:52:23.577467+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-10T18:51:57.216086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:57.216086+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:07.634282+0000","scrub_duration":0.0048352930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.12","version":"70'81","reported_seq":14,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.564577+0000","last_change":"2024-09-10T18:52:16.599661+0000","last_active":"2024-09-10T18:52:23.564577+0000","last_peered":"2024-09-10T18:52:23.564577+0000","last_clean":"2024-09-10T18:52:23.564577+0000","last_became_active":"2024-09-10T18:52:16.598921+0000","last_became_peered":"2024-09-10T18:52:16.598921+0000","last_unstale":"2024-09-10T18:52:23.564577+0000","last_undegraded":"2024-09-10T18:52:23.564577+0000","last_fullsized":"2024-09-10T18:52:23.564577+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:50:50.506769+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:50:50.506769+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019876809999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":10,"num_write":6,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":53670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":156,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.562815+0000","last_change":"2024-09-10T18:50:52.260051+0000","last_active":"2024-09-10T18:52:23.562815+0000","last_peered":"2024-09-10T18:52:23.562815+0000","last_clean":"2024-09-10T18:52:23.562815+0000","last_became_active":"2024-09-10T18:45:17.964705+0000","last_became_peered":"2024-09-10T18:45:17.964705+0000","last_unstale":"2024-09-10T18:52:23.562815+0000","last_undegraded":"2024-09-10T18:52:23.562815+0000","last_fullsized":"2024-09-10T18:52:23.562815+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:52.259998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:52.259998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0032069059999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'6","reported_seq":164,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.579351+0000","last_change":"2024-09-10T18:51:43.085607+0000","last_active":"2024-09-10T18:52:23.579351+0000","last_peered":"2024-09-10T18:52:23.579351+0000","last_clean":"2024-09-10T18:52:23.579351+0000","last_became_active":"2024-09-10T18:45:20.421746+0000","last_became_peered":"2024-09-10T18:45:20.421746+0000","last_unstale":"2024-09-10T18:52:23.579351+0000","last_undegraded":"2024-09-10T18:52:23.579351+0000","last_fullsized":"2024-09-10T18:52:23.579351+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'6","last_scrub_stamp":"2024-09-10T18:51:43.085549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:43.085549+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:44.636819+0000","scrub_duration":0.002071238,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":152,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.564382+0000","last_change":"2024-09-10T18:51:32.233743+0000","last_active":"2024-09-10T18:52:23.564382+0000","last_peered":"2024-09-10T18:52:23.564382+0000","last_clean":"2024-09-10T18:52:23.564382+0000","last_became_active":"2024-09-10T18:45:22.069653+0000","last_became_peered":"2024-09-10T18:45:22.069653+0000","last_unstale":"2024-09-10T18:52:23.564382+0000","last_undegraded":"2024-09-10T18:52:23.564382+0000","last_fullsized":"2024-09-10T18:52:23.564382+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:32.233676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:32.233676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:33.989510+0000","scrub_duration":0.0016772180000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":151,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.568169+0000","last_change":"2024-09-10T18:50:43.408152+0000","last_active":"2024-09-10T18:52:23.568169+0000","last_peered":"2024-09-10T18:52:23.568169+0000","last_clean":"2024-09-10T18:52:23.568169+0000","last_became_active":"2024-09-10T18:45:24.264611+0000","last_became_peered":"2024-09-10T18:45:24.264611+0000","last_unstale":"2024-09-10T18:52:23.568169+0000","last_undegraded":"2024-09-10T18:52:23.568169+0000","last_fullsized":"2024-09-10T18:52:23.568169+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:43.408011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:50:43.408011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026253959999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.11","version":"70'81","reported_seq":82,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.569342+0000","last_change":"2024-09-10T18:52:16.591442+0000","last_active":"2024-09-10T18:52:23.569342+0000","last_peered":"2024-09-10T18:52:23.569342+0000","last_clean":"2024-09-10T18:52:23.569342+0000","last_became_active":"2024-09-10T18:52:16.590431+0000","last_became_peered":"2024-09-10T18:52:16.590431+0000","last_unstale":"2024-09-10T18:52:23.569342+0000","last_undegraded":"2024-09-10T18:52:23.569342+0000","last_fullsized":"2024-09-10T18:52:23.569342+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:50:43.117728+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:50:43.117728+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019876809999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":10,"num_write":6,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":53670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"63'10","reported_seq":214,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.575339+0000","last_change":"2024-09-10T18:51:39.074731+0000","last_active":"2024-09-10T18:52:23.575339+0000","last_peered":"2024-09-10T18:52:23.575339+0000","last_clean":"2024-09-10T18:52:23.575339+0000","last_became_active":"2024-09-10T18:47:36.426260+0000","last_became_peered":"2024-09-10T18:47:36.426260+0000","last_unstale":"2024-09-10T18:52:23.575339+0000","last_undegraded":"2024-09-10T18:52:23.575339+0000","last_fullsized":"2024-09-10T18:52:23.575339+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:51:39.074664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:39.074664+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:05.841922+0000","scrub_duration":0.0044748410000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":156,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.565064+0000","last_change":"2024-09-10T18:50:45.120724+0000","last_active":"2024-09-10T18:52:23.565064+0000","last_peered":"2024-09-10T18:52:23.565064+0000","last_clean":"2024-09-10T18:52:23.565064+0000","last_became_active":"2024-09-10T18:45:17.964870+0000","last_became_peered":"2024-09-10T18:45:17.964870+0000","last_unstale":"2024-09-10T18:52:23.565064+0000","last_undegraded":"2024-09-10T18:52:23.565064+0000","last_fullsized":"2024-09-10T18:52:23.565064+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:45.120550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:45.120550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0055063930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"63'8","reported_seq":245,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.564191+0000","last_change":"2024-09-10T18:50:52.043047+0000","last_active":"2024-09-10T18:52:23.564191+0000","last_peered":"2024-09-10T18:52:23.564191+0000","last_clean":"2024-09-10T18:52:23.564191+0000","last_became_active":"2024-09-10T18:45:22.062940+0000","last_became_peered":"2024-09-10T18:45:22.062940+0000","last_unstale":"2024-09-10T18:52:23.564191+0000","last_undegraded":"2024-09-10T18:52:23.564191+0000","last_fullsized":"2024-09-10T18:52:23.564191+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:50:52.042960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:52.042960+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0090181189999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":151,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.571385+0000","last_change":"2024-09-10T18:50:56.767501+0000","last_active":"2024-09-10T18:52:23.571385+0000","last_peered":"2024-09-10T18:52:23.571385+0000","last_clean":"2024-09-10T18:52:23.571385+0000","last_became_active":"2024-09-10T18:45:24.073736+0000","last_became_peered":"2024-09-10T18:45:24.073736+0000","last_unstale":"2024-09-10T18:52:23.571385+0000","last_undegraded":"2024-09-10T18:52:23.571385+0000","last_fullsized":"2024-09-10T18:52:23.571385+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:56.767430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:50:56.767430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.021971303000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.10","version":"70'81","reported_seq":14,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.564066+0000","last_change":"2024-09-10T18:52:16.601909+0000","last_active":"2024-09-10T18:52:23.564066+0000","last_peered":"2024-09-10T18:52:23.564066+0000","last_clean":"2024-09-10T18:52:23.564066+0000","last_became_active":"2024-09-10T18:52:16.601616+0000","last_became_peered":"2024-09-10T18:52:16.601616+0000","last_unstale":"2024-09-10T18:52:23.564066+0000","last_undegraded":"2024-09-10T18:52:23.564066+0000","last_fullsized":"2024-09-10T18:52:23.564066+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:50:48.508244+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:50:48.508244+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019876809999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":5,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":53670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"63'9","reported_seq":180,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.564326+0000","last_change":"2024-09-10T18:51:45.423930+0000","last_active":"2024-09-10T18:52:23.564326+0000","last_peered":"2024-09-10T18:52:23.564326+0000","last_clean":"2024-09-10T18:52:23.564326+0000","last_became_active":"2024-09-10T18:45:20.421124+0000","last_became_peered":"2024-09-10T18:45:20.421124+0000","last_unstale":"2024-09-10T18:52:23.564326+0000","last_undegraded":"2024-09-10T18:52:23.564326+0000","last_fullsized":"2024-09-10T18:52:23.564326+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:51:45.423873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:45.423873+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:55.984623+0000","scrub_duration":0.001903037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":167,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.570572+0000","last_change":"2024-09-10T18:51:46.194282+0000","last_active":"2024-09-10T18:52:23.570572+0000","last_peered":"2024-09-10T18:52:23.570572+0000","last_clean":"2024-09-10T18:52:23.570572+0000","last_became_active":"2024-09-10T18:45:17.960106+0000","last_became_peered":"2024-09-10T18:45:17.960106+0000","last_unstale":"2024-09-10T18:52:23.570572+0000","last_undegraded":"2024-09-10T18:52:23.570572+0000","last_fullsized":"2024-09-10T18:52:23.570572+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:46.194176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:46.194176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:04.442996+0000","scrub_duration":0.001545888,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"89'10","reported_seq":262,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-10T18:52:24.570497+0000","last_change":"2024-09-10T18:51:46.054778+0000","last_active":"2024-09-10T18:52:24.570497+0000","last_peered":"2024-09-10T18:52:24.570497+0000","last_clean":"2024-09-10T18:52:24.570497+0000","last_became_active":"2024-09-10T18:47:36.424575+0000","last_became_peered":"2024-09-10T18:47:36.424575+0000","last_unstale":"2024-09-10T18:52:24.570497+0000","last_undegraded":"2024-09-10T18:52:24.570497+0000","last_fullsized":"2024-09-10T18:52:24.570497+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'10","last_scrub_stamp":"2024-09-10T18:51:46.054720+0000","last_deep_scrub":"81'9","last_deep_scrub_stamp":"2024-09-10T18:46:54.800790+0000","last_clean_scrub_stamp":"2024-09-10T18:51:46.054720+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:07.645431+0000","scrub_duration":0.001410667,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":159,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.577376+0000","last_change":"2024-09-10T18:50:55.384653+0000","last_active":"2024-09-10T18:52:23.577376+0000","last_peered":"2024-09-10T18:52:23.577376+0000","last_clean":"2024-09-10T18:52:23.577376+0000","last_became_active":"2024-09-10T18:47:36.414047+0000","last_became_peered":"2024-09-10T18:47:36.414047+0000","last_unstale":"2024-09-10T18:52:23.577376+0000","last_undegraded":"2024-09-10T18:52:23.577376+0000","last_fullsized":"2024-09-10T18:52:23.577376+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:55.384570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_clean_scrub_stamp":"2024-09-10T18:50:55.384570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0079403359999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"63'9","reported_seq":191,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.574070+0000","last_change":"2024-09-10T18:51:32.146681+0000","last_active":"2024-09-10T18:52:23.574070+0000","last_peered":"2024-09-10T18:52:23.574070+0000","last_clean":"2024-09-10T18:52:23.574070+0000","last_became_active":"2024-09-10T18:47:36.416484+0000","last_became_peered":"2024-09-10T18:47:36.416484+0000","last_unstale":"2024-09-10T18:52:23.574070+0000","last_undegraded":"2024-09-10T18:52:23.574070+0000","last_fullsized":"2024-09-10T18:52:23.574070+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:51:32.146622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:32.146622+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:53.233486+0000","scrub_duration":0.0023792729999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":166,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.560965+0000","last_change":"2024-09-10T18:51:44.486612+0000","last_active":"2024-09-10T18:52:23.560965+0000","last_peered":"2024-09-10T18:52:23.560965+0000","last_clean":"2024-09-10T18:52:23.560965+0000","last_became_active":"2024-09-10T18:45:17.962009+0000","last_became_peered":"2024-09-10T18:45:17.962009+0000","last_unstale":"2024-09-10T18:52:23.560965+0000","last_undegraded":"2024-09-10T18:52:23.560965+0000","last_fullsized":"2024-09-10T18:52:23.560965+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:44.486550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:44.486550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:57.019047+0000","scrub_duration":0.0013873609999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":153,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.574950+0000","last_change":"2024-09-10T18:51:47.078689+0000","last_active":"2024-09-10T18:52:23.574950+0000","last_peered":"2024-09-10T18:52:23.574950+0000","last_clean":"2024-09-10T18:52:23.574950+0000","last_became_active":"2024-09-10T18:45:22.072877+0000","last_became_peered":"2024-09-10T18:45:22.072877+0000","last_unstale":"2024-09-10T18:52:23.574950+0000","last_undegraded":"2024-09-10T18:52:23.574950+0000","last_fullsized":"2024-09-10T18:52:23.574950+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:47.078574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:03.571786+0000","last_clean_scrub_stamp":"2024-09-10T18:51:47.078574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:06.636735+0000","scrub_duration":0.001570454,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":161,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.569109+0000","last_change":"2024-09-10T18:51:47.209085+0000","last_active":"2024-09-10T18:52:23.569109+0000","last_peered":"2024-09-10T18:52:23.569109+0000","last_clean":"2024-09-10T18:52:23.569109+0000","last_became_active":"2024-09-10T18:45:24.081050+0000","last_became_peered":"2024-09-10T18:45:24.081050+0000","last_unstale":"2024-09-10T18:52:23.569109+0000","last_undegraded":"2024-09-10T18:52:23.569109+0000","last_fullsized":"2024-09-10T18:52:23.569109+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:47.209023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:04.631558+0000","last_clean_scrub_stamp":"2024-09-10T18:51:47.209023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:13.589713+0000","scrub_duration":0.0019148069999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"63'11","reported_seq":172,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.576867+0000","last_change":"2024-09-10T18:52:06.206180+0000","last_active":"2024-09-10T18:52:23.576867+0000","last_peered":"2024-09-10T18:52:23.576867+0000","last_clean":"2024-09-10T18:52:23.576867+0000","last_became_active":"2024-09-10T18:45:20.056885+0000","last_became_peered":"2024-09-10T18:45:20.056885+0000","last_unstale":"2024-09-10T18:52:23.576867+0000","last_undegraded":"2024-09-10T18:52:23.576867+0000","last_fullsized":"2024-09-10T18:52:23.576867+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:52:06.206060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:52:06.206060+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:16.631162+0000","scrub_duration":0.0078519460000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":157,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.571291+0000","last_change":"2024-09-10T18:51:00.029596+0000","last_active":"2024-09-10T18:52:23.571291+0000","last_peered":"2024-09-10T18:52:23.571291+0000","last_clean":"2024-09-10T18:52:23.571291+0000","last_became_active":"2024-09-10T18:45:17.963542+0000","last_became_peered":"2024-09-10T18:45:17.963542+0000","last_unstale":"2024-09-10T18:52:23.571291+0000","last_undegraded":"2024-09-10T18:52:23.571291+0000","last_fullsized":"2024-09-10T18:52:23.571291+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:00.029389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:00.029389+0000","last_clean_scrub_stamp":"2024-09-10T18:51:00.029389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:25.353957+0000","scrub_duration":0.0023207890000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":156,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.573920+0000","last_change":"2024-09-10T18:50:45.003161+0000","last_active":"2024-09-10T18:52:23.573920+0000","last_peered":"2024-09-10T18:52:23.573920+0000","last_clean":"2024-09-10T18:52:23.573920+0000","last_became_active":"2024-09-10T18:45:22.070065+0000","last_became_peered":"2024-09-10T18:45:22.070065+0000","last_unstale":"2024-09-10T18:52:23.573920+0000","last_undegraded":"2024-09-10T18:52:23.573920+0000","last_fullsized":"2024-09-10T18:52:23.573920+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:45.002838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:45.002838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.017421118999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":151,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.573343+0000","last_change":"2024-09-10T18:52:05.098106+0000","last_active":"2024-09-10T18:52:23.573343+0000","last_peered":"2024-09-10T18:52:23.573343+0000","last_clean":"2024-09-10T18:52:23.573343+0000","last_became_active":"2024-09-10T18:45:24.072224+0000","last_became_peered":"2024-09-10T18:45:24.072224+0000","last_unstale":"2024-09-10T18:52:23.573343+0000","last_undegraded":"2024-09-10T18:52:23.573343+0000","last_fullsized":"2024-09-10T18:52:23.573343+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:05.098031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:52:05.098031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:11.896763+0000","scrub_duration":0.0067119739999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"63'4","reported_seq":161,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.573882+0000","last_change":"2024-09-10T18:50:46.171962+0000","last_active":"2024-09-10T18:52:23.573882+0000","last_peered":"2024-09-10T18:52:23.573882+0000","last_clean":"2024-09-10T18:52:23.573882+0000","last_became_active":"2024-09-10T18:45:20.059205+0000","last_became_peered":"2024-09-10T18:45:20.059205+0000","last_unstale":"2024-09-10T18:52:23.573882+0000","last_undegraded":"2024-09-10T18:52:23.573882+0000","last_fullsized":"2024-09-10T18:52:23.573882+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4","last_scrub_stamp":"2024-09-10T18:50:46.171568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:50:46.171568+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0054395399999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":166,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.564489+0000","last_change":"2024-09-10T18:51:43.344854+0000","last_active":"2024-09-10T18:52:23.564489+0000","last_peered":"2024-09-10T18:52:23.564489+0000","last_clean":"2024-09-10T18:52:23.564489+0000","last_became_active":"2024-09-10T18:47:36.415866+0000","last_became_peered":"2024-09-10T18:47:36.415866+0000","last_unstale":"2024-09-10T18:52:23.564489+0000","last_undegraded":"2024-09-10T18:52:23.564489+0000","last_fullsized":"2024-09-10T18:52:23.564489+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:43.344751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:43.344751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:01.340612+0000","scrub_duration":0.0019741749999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":162,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.560410+0000","last_change":"2024-09-10T18:51:54.514805+0000","last_active":"2024-09-10T18:52:23.560410+0000","last_peered":"2024-09-10T18:52:23.560410+0000","last_clean":"2024-09-10T18:52:23.560410+0000","last_became_active":"2024-09-10T18:45:22.072377+0000","last_became_peered":"2024-09-10T18:45:22.072377+0000","last_unstale":"2024-09-10T18:52:23.560410+0000","last_undegraded":"2024-09-10T18:52:23.560410+0000","last_fullsized":"2024-09-10T18:52:23.560410+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:54.514747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_clean_scrub_stamp":"2024-09-10T18:51:54.514747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:24.200821+0000","scrub_duration":0.0015998679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":171,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.567105+0000","last_change":"2024-09-10T18:51:55.145730+0000","last_active":"2024-09-10T18:52:23.567105+0000","last_peered":"2024-09-10T18:52:23.567105+0000","last_clean":"2024-09-10T18:52:23.567105+0000","last_became_active":"2024-09-10T18:47:36.416187+0000","last_became_peered":"2024-09-10T18:47:36.416187+0000","last_unstale":"2024-09-10T18:52:23.567105+0000","last_undegraded":"2024-09-10T18:52:23.567105+0000","last_fullsized":"2024-09-10T18:52:23.567105+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:55.145638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:55.145638+0000","last_clean_scrub_stamp":"2024-09-10T18:51:55.145638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:06.238150+0000","scrub_duration":0.00318506,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.14","version":"70'81","reported_seq":171,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.567522+0000","last_change":"2024-09-10T18:52:16.600806+0000","last_active":"2024-09-10T18:52:23.567522+0000","last_peered":"2024-09-10T18:52:23.567522+0000","last_clean":"2024-09-10T18:52:23.567522+0000","last_became_active":"2024-09-10T18:52:16.600485+0000","last_became_peered":"2024-09-10T18:52:16.600485+0000","last_unstale":"2024-09-10T18:52:23.567522+0000","last_undegraded":"2024-09-10T18:52:23.567522+0000","last_fullsized":"2024-09-10T18:52:23.567522+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:50:59.164417+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:50:59.164417+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016709736999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17890,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":7,"num_write":3,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":35780,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"79'16","reported_seq":179,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.574013+0000","last_change":"2024-09-10T18:51:36.997632+0000","last_active":"2024-09-10T18:52:23.574013+0000","last_peered":"2024-09-10T18:52:23.574013+0000","last_clean":"2024-09-10T18:52:23.574013+0000","last_became_active":"2024-09-10T18:45:20.421479+0000","last_became_peered":"2024-09-10T18:45:20.421479+0000","last_unstale":"2024-09-10T18:52:23.574013+0000","last_undegraded":"2024-09-10T18:52:23.574013+0000","last_fullsized":"2024-09-10T18:52:23.574013+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"79'16","last_scrub_stamp":"2024-09-10T18:51:36.997571+0000","last_deep_scrub":"79'16","last_deep_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.997571+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:45.812239+0000","scrub_duration":0.002345059,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"56'1","reported_seq":177,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.561743+0000","last_change":"2024-09-10T18:50:43.279453+0000","last_active":"2024-09-10T18:52:23.561743+0000","last_peered":"2024-09-10T18:52:23.561743+0000","last_clean":"2024-09-10T18:52:23.561743+0000","last_became_active":"2024-09-10T18:47:36.421985+0000","last_became_peered":"2024-09-10T18:47:36.421985+0000","last_unstale":"2024-09-10T18:52:23.561743+0000","last_undegraded":"2024-09-10T18:52:23.561743+0000","last_fullsized":"2024-09-10T18:52:23.561743+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:50:43.279224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:43.279224+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048255859999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":107,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.573389+0000","last_change":"2024-09-10T18:50:41.057828+0000","last_active":"2024-09-10T18:52:23.573389+0000","last_peered":"2024-09-10T18:52:23.573389+0000","last_clean":"2024-09-10T18:52:23.573389+0000","last_became_active":"2024-09-10T18:47:57.107149+0000","last_became_peered":"2024-09-10T18:47:57.107149+0000","last_unstale":"2024-09-10T18:52:23.573389+0000","last_undegraded":"2024-09-10T18:52:23.573389+0000","last_fullsized":"2024-09-10T18:52:23.573389+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:41.057750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:41.057750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013401967000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":151,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.574788+0000","last_change":"2024-09-10T18:50:13.793044+0000","last_active":"2024-09-10T18:52:23.574788+0000","last_peered":"2024-09-10T18:52:23.574788+0000","last_clean":"2024-09-10T18:52:23.574788+0000","last_became_active":"2024-09-10T18:45:24.264818+0000","last_became_peered":"2024-09-10T18:45:24.264818+0000","last_unstale":"2024-09-10T18:52:23.574788+0000","last_undegraded":"2024-09-10T18:52:23.574788+0000","last_fullsized":"2024-09-10T18:52:23.574788+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:13.792946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:50:13.792946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012475749,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"78'2","reported_seq":177,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.568063+0000","last_change":"2024-09-10T18:51:46.249221+0000","last_active":"2024-09-10T18:52:23.568063+0000","last_peered":"2024-09-10T18:52:23.568063+0000","last_clean":"2024-09-10T18:52:23.568063+0000","last_became_active":"2024-09-10T18:47:36.418702+0000","last_became_peered":"2024-09-10T18:47:36.418702+0000","last_unstale":"2024-09-10T18:52:23.568063+0000","last_undegraded":"2024-09-10T18:52:23.568063+0000","last_fullsized":"2024-09-10T18:52:23.568063+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'2","last_scrub_stamp":"2024-09-10T18:51:46.249165+0000","last_deep_scrub":"78'2","last_deep_scrub_stamp":"2024-09-10T18:46:37.581130+0000","last_clean_scrub_stamp":"2024-09-10T18:51:46.249165+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:56.099034+0000","scrub_duration":0.001625867,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":156,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.566020+0000","last_change":"2024-09-10T18:51:03.197559+0000","last_active":"2024-09-10T18:52:23.566020+0000","last_peered":"2024-09-10T18:52:23.566020+0000","last_clean":"2024-09-10T18:52:23.566020+0000","last_became_active":"2024-09-10T18:45:17.962108+0000","last_became_peered":"2024-09-10T18:45:17.962108+0000","last_unstale":"2024-09-10T18:52:23.566020+0000","last_undegraded":"2024-09-10T18:52:23.566020+0000","last_fullsized":"2024-09-10T18:52:23.566020+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:03.197480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:03.197480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024932259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"63'10","reported_seq":180,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.571638+0000","last_change":"2024-09-10T18:52:00.358388+0000","last_active":"2024-09-10T18:52:23.571638+0000","last_peered":"2024-09-10T18:52:23.571638+0000","last_clean":"2024-09-10T18:52:23.571638+0000","last_became_active":"2024-09-10T18:45:20.421094+0000","last_became_peered":"2024-09-10T18:45:20.421094+0000","last_unstale":"2024-09-10T18:52:23.571638+0000","last_undegraded":"2024-09-10T18:52:23.571638+0000","last_fullsized":"2024-09-10T18:52:23.571638+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:52:00.358298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:52:00.358298+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:04.823464+0000","scrub_duration":0.01044082,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"63'8","reported_seq":258,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.561536+0000","last_change":"2024-09-10T18:51:50.580960+0000","last_active":"2024-09-10T18:52:23.561536+0000","last_peered":"2024-09-10T18:52:23.561536+0000","last_clean":"2024-09-10T18:52:23.561536+0000","last_became_active":"2024-09-10T18:45:22.073281+0000","last_became_peered":"2024-09-10T18:45:22.073281+0000","last_unstale":"2024-09-10T18:52:23.561536+0000","last_undegraded":"2024-09-10T18:52:23.561536+0000","last_fullsized":"2024-09-10T18:52:23.561536+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:51:50.580871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:50.580871+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:52.495673+0000","scrub_duration":0.002070484,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":151,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.570810+0000","last_change":"2024-09-10T18:51:40.051314+0000","last_active":"2024-09-10T18:52:23.570810+0000","last_peered":"2024-09-10T18:52:23.570810+0000","last_clean":"2024-09-10T18:52:23.570810+0000","last_became_active":"2024-09-10T18:45:24.082318+0000","last_became_peered":"2024-09-10T18:45:24.082318+0000","last_unstale":"2024-09-10T18:52:23.570810+0000","last_undegraded":"2024-09-10T18:52:23.570810+0000","last_fullsized":"2024-09-10T18:52:23.570810+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:40.051241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:03.475299+0000","last_clean_scrub_stamp":"2024-09-10T18:51:40.051241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:48.927775+0000","scrub_duration":0.001564937,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":157,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.574444+0000","last_change":"2024-09-10T18:51:01.165781+0000","last_active":"2024-09-10T18:52:23.574444+0000","last_peered":"2024-09-10T18:52:23.574444+0000","last_clean":"2024-09-10T18:52:23.574444+0000","last_became_active":"2024-09-10T18:45:17.965707+0000","last_became_peered":"2024-09-10T18:45:17.965707+0000","last_unstale":"2024-09-10T18:52:23.574444+0000","last_undegraded":"2024-09-10T18:52:23.574444+0000","last_fullsized":"2024-09-10T18:52:23.574444+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:01.165584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:01.165584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:28.929394+0000","scrub_duration":0.0071542790000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"63'11","reported_seq":183,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.561675+0000","last_change":"2024-09-10T18:51:46.517627+0000","last_active":"2024-09-10T18:52:23.561675+0000","last_peered":"2024-09-10T18:52:23.561675+0000","last_clean":"2024-09-10T18:52:23.561675+0000","last_became_active":"2024-09-10T18:45:20.058717+0000","last_became_peered":"2024-09-10T18:45:20.058717+0000","last_unstale":"2024-09-10T18:52:23.561675+0000","last_undegraded":"2024-09-10T18:52:23.561675+0000","last_fullsized":"2024-09-10T18:52:23.561675+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:51:46.517571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:46.517571+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:12.935571+0000","scrub_duration":0.001736118,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":163,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.571566+0000","last_change":"2024-09-10T18:50:54.765172+0000","last_active":"2024-09-10T18:52:23.571566+0000","last_peered":"2024-09-10T18:52:23.571566+0000","last_clean":"2024-09-10T18:52:23.571566+0000","last_became_active":"2024-09-10T18:47:36.418767+0000","last_became_peered":"2024-09-10T18:47:36.418767+0000","last_unstale":"2024-09-10T18:52:23.571566+0000","last_undegraded":"2024-09-10T18:52:23.571566+0000","last_fullsized":"2024-09-10T18:52:23.571566+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:54.764990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:54.764990+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043824299999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":149,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.577237+0000","last_change":"2024-09-10T18:51:36.324657+0000","last_active":"2024-09-10T18:52:23.577237+0000","last_peered":"2024-09-10T18:52:23.577237+0000","last_clean":"2024-09-10T18:52:23.577237+0000","last_became_active":"2024-09-10T18:45:24.267251+0000","last_became_peered":"2024-09-10T18:45:24.267251+0000","last_unstale":"2024-09-10T18:52:23.577237+0000","last_undegraded":"2024-09-10T18:52:23.577237+0000","last_fullsized":"2024-09-10T18:52:23.577237+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.324592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.324592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:56.184657+0000","scrub_duration":0.001354409,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"56'1","reported_seq":165,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.561155+0000","last_change":"2024-09-10T18:50:07.341477+0000","last_active":"2024-09-10T18:52:23.561155+0000","last_peered":"2024-09-10T18:52:23.561155+0000","last_clean":"2024-09-10T18:52:23.561155+0000","last_became_active":"2024-09-10T18:45:17.966286+0000","last_became_peered":"2024-09-10T18:45:17.966286+0000","last_unstale":"2024-09-10T18:52:23.561155+0000","last_undegraded":"2024-09-10T18:52:23.561155+0000","last_fullsized":"2024-09-10T18:52:23.561155+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:50:07.341394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:50:07.341394+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019580940000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"63'15","reported_seq":171,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.571155+0000","last_change":"2024-09-10T18:51:59.160864+0000","last_active":"2024-09-10T18:52:23.571155+0000","last_peered":"2024-09-10T18:52:23.571155+0000","last_clean":"2024-09-10T18:52:23.571155+0000","last_became_active":"2024-09-10T18:47:36.424373+0000","last_became_peered":"2024-09-10T18:47:36.424373+0000","last_unstale":"2024-09-10T18:52:23.571155+0000","last_undegraded":"2024-09-10T18:52:23.571155+0000","last_fullsized":"2024-09-10T18:52:23.571155+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:51:59.160774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:59.160774+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:06.186757+0000","scrub_duration":0.0030853120000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":153,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.571071+0000","last_change":"2024-09-10T18:51:54.153006+0000","last_active":"2024-09-10T18:52:23.571071+0000","last_peered":"2024-09-10T18:52:23.571071+0000","last_clean":"2024-09-10T18:52:23.571071+0000","last_became_active":"2024-09-10T18:45:22.059794+0000","last_became_peered":"2024-09-10T18:45:22.059794+0000","last_unstale":"2024-09-10T18:52:23.571071+0000","last_undegraded":"2024-09-10T18:52:23.571071+0000","last_fullsized":"2024-09-10T18:52:23.571071+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:54.152908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:54.152908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:12.029577+0000","scrub_duration":0.0079641550000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":141,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.578481+0000","last_change":"2024-09-10T18:49:51.464014+0000","last_active":"2024-09-10T18:52:23.578481+0000","last_peered":"2024-09-10T18:52:23.578481+0000","last_clean":"2024-09-10T18:52:23.578481+0000","last_became_active":"2024-09-10T18:45:24.264427+0000","last_became_peered":"2024-09-10T18:45:24.264427+0000","last_unstale":"2024-09-10T18:52:23.578481+0000","last_undegraded":"2024-09-10T18:52:23.578481+0000","last_fullsized":"2024-09-10T18:52:23.578481+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:49:51.463929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:49:51.463929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.028103624000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":157,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.575233+0000","last_change":"2024-09-10T18:51:40.076322+0000","last_active":"2024-09-10T18:52:23.575233+0000","last_peered":"2024-09-10T18:52:23.575233+0000","last_clean":"2024-09-10T18:52:23.575233+0000","last_became_active":"2024-09-10T18:45:17.963257+0000","last_became_peered":"2024-09-10T18:45:17.963257+0000","last_unstale":"2024-09-10T18:52:23.575233+0000","last_undegraded":"2024-09-10T18:52:23.575233+0000","last_fullsized":"2024-09-10T18:52:23.575233+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:40.076262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:40.076262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:58.187702+0000","scrub_duration":0.0037079650000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"63'12","reported_seq":178,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.576857+0000","last_change":"2024-09-10T18:51:31.148694+0000","last_active":"2024-09-10T18:52:23.576857+0000","last_peered":"2024-09-10T18:52:23.576857+0000","last_clean":"2024-09-10T18:52:23.576857+0000","last_became_active":"2024-09-10T18:45:20.058391+0000","last_became_peered":"2024-09-10T18:45:20.058391+0000","last_unstale":"2024-09-10T18:52:23.576857+0000","last_undegraded":"2024-09-10T18:52:23.576857+0000","last_fullsized":"2024-09-10T18:52:23.576857+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:51:31.148630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:31.148630+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:52:33.947829+0000","scrub_duration":0.0021214699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":161,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-10T18:52:23.576870+0000","last_change":"2024-09-10T18:50:41.560291+0000","last_active":"2024-09-10T18:52:23.576870+0000","last_peered":"2024-09-10T18:52:23.576870+0000","last_clean":"2024-09-10T18:52:23.576870+0000","last_became_active":"2024-09-10T18:47:36.416131+0000","last_became_peered":"2024-09-10T18:47:36.416131+0000","last_unstale":"2024-09-10T18:52:23.576870+0000","last_undegraded":"2024-09-10T18:52:23.576870+0000","last_fullsized":"2024-09-10T18:52:23.576870+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:41.560194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:50:41.560194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0067616830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":4991942656,"num_objects":76171,"num_object_clones":0,"num_object_copies":228513,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":76171,"num_write_kb":4874944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":14958002176,"data_stored":14958002176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":67471,"ondisk_log_size":67471,"up":48,"acting":48,"num_store_stats":7},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":697,"num_read_kb":454,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":1238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":455,"ondisk_log_size":455,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":418,"num_read_kb":413,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":606371,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":47,"num_write":25,"num_write_kb":345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":578140,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1701,"ondisk_log_size":1701,"up":63,"acting":63,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299482,"num_pgs":74,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2103352,"kb_used_data":1797680,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":91641800,"statfs":{"total":95995035648,"available":93841203200,"internally_reserved":0,"allocated":1840824320,"data_stored":1840214104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:52:19 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.3560000000000001,"15min":0.79000000000000004},"min":{"1min":0.315,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":3.508,"5min":31.805,"15min":31.805},"last":0.217},{"interface":"front","average":{"1min":0.77800000000000002,"5min":1.569,"15min":0.84899999999999998},"min":{"1min":0.377,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":2.1179999999999999,"5min":31.212,"15min":31.212},"last":1.3029999999999999}]},{"osd":1,"last update":"Tue Sep 10 18:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":1.0600000000000001,"15min":0.67600000000000005},"min":{"1min":0.34799999999999998,"5min":0.221,"15min":0.221},"max":{"1min":2.4660000000000002,"5min":21.34,"15min":21.34},"last":0.248},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.998,"15min":0.67800000000000005},"min":{"1min":0.255,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":2.2810000000000001,"5min":21.248000000000001,"15min":21.248000000000001},"last":1.427}]},{"osd":2,"last update":"Tue Sep 10 18:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.64300000000000002,"15min":0.53900000000000003},"min":{"1min":0.29599999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.6930000000000001,"5min":5.1399999999999997,"15min":5.1399999999999997},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.78600000000000003,"15min":0.60599999999999998},"min":{"1min":0.442,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.917,"5min":8.8859999999999992,"15min":8.8859999999999992},"last":0.32000000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":1.5349999999999999,"15min":0.84899999999999998},"min":{"1min":0.26900000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.9770000000000001,"5min":31.478000000000002,"15min":31.478000000000002},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.86499999999999999,"5min":1.7370000000000001,"15min":0.92600000000000005},"min":{"1min":0.39800000000000002,"5min":0.254,"15min":0.254},"max":{"1min":1.9470000000000001,"5min":33.731999999999999,"15min":33.731999999999999},"last":0.379}]},{"osd":4,"last update":"Tue Sep 10 18:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.53600000000000003,"15min":0.436},"min":{"1min":0.28199999999999997,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":2.302,"5min":2.7559999999999998,"15min":2.7559999999999998},"last":1.3420000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.52400000000000002,"15min":0.42899999999999999},"min":{"1min":0.255,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":0.91600000000000004}]},{"osd":5,"last update":"Tue Sep 10 18:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.56599999999999995,"15min":0.46300000000000002},"min":{"1min":0.316,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":2.0950000000000002,"5min":2.7240000000000002,"15min":2.7240000000000002},"last":1.407},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.53800000000000003,"15min":0.45900000000000002},"min":{"1min":0.35599999999999998,"5min":0.253,"15min":0.253},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.52300000000000002}]},{"osd":6,"last update":"Tue Sep 10 18:52:19 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.63600000000000001,"15min":0.51200000000000001},"min":{"1min":0.27100000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":2.2490000000000001,"5min":2.8199999999999998,"15min":2.8199999999999998},"last":1.4470000000000001},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.60099999999999998,"15min":0.49099999999999999},"min":{"1min":0.34000000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":2.004,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.498}]}]},{"osd":6,"up_from":43,"seq":184683593823,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2404232,"kb_used_data":2102716,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":91340920,"statfs":{"total":95995035648,"available":93533102080,"internally_reserved":0,"allocated":2153181184,"data_stored":2152559768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":1.591,"15min":0.83699999999999997},"min":{"1min":0.41399999999999998,"5min":0.215,"15min":0.215},"max":{"1min":2.9700000000000002,"5min":46.384999999999998,"15min":46.384999999999998},"last":1.3700000000000001},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.526,"15min":0.83699999999999997},"min":{"1min":0.43099999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":4.9500000000000002,"5min":48.372999999999998,"15min":48.372999999999998},"last":1.258}]},{"osd":1,"last update":"Tue Sep 10 18:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.92700000000000005,"15min":0.63500000000000001},"min":{"1min":0.308,"5min":0.219,"15min":0.219},"max":{"1min":2.3039999999999998,"5min":6.1319999999999997,"15min":6.1319999999999997},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.82599999999999996,"15min":0.58399999999999996},"min":{"1min":0.34499999999999997,"5min":0.214,"15min":0.214},"max":{"1min":2.8929999999999998,"5min":3.6280000000000001,"15min":3.6280000000000001},"last":0.66900000000000004}]},{"osd":2,"last update":"Tue Sep 10 18:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.80800000000000005,"15min":0.58999999999999997},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.1240000000000001,"5min":4.4740000000000002,"15min":4.4740000000000002},"last":1.167},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.60499999999999998,"15min":0.52900000000000003},"min":{"1min":0.34899999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.75800000000000001,"5min":2.9900000000000002,"15min":2.9900000000000002},"last":0.63400000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.87,"15min":0.61199999999999999},"min":{"1min":0.34300000000000003,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":2.992,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.101,"15min":0.70799999999999996},"min":{"1min":0.434,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":4.6109999999999998,"5min":5.3079999999999998,"15min":5.3079999999999998},"last":1.2230000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.56899999999999995,"15min":0.45900000000000002},"min":{"1min":0.30199999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.69299999999999995,"5min":4.2969999999999997,"15min":4.2969999999999997},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.54300000000000004,"15min":0.441},"min":{"1min":0.29199999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":2.9430000000000001,"5min":4.3209999999999997,"15min":4.3209999999999997},"last":0.505}]},{"osd":5,"last update":"Tue Sep 10 18:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59699999999999998,"15min":0.47699999999999998},"min":{"1min":0.26700000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":3.0950000000000002,"5min":4.4210000000000003,"15min":4.4210000000000003},"last":6.8440000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59199999999999997,"15min":0.48099999999999998},"min":{"1min":0.30399999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.89300000000000002,"5min":4.4500000000000002,"15min":4.4500000000000002},"last":0.68200000000000005}]},{"osd":7,"last update":"Tue Sep 10 18:52:12 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.73899999999999999,"15min":0.56100000000000005},"min":{"1min":0.38700000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":2.625,"5min":4.5259999999999998,"15min":4.5259999999999998},"last":1.3169999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.71999999999999997,"15min":0.57099999999999995},"min":{"1min":0.373,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":2.6819999999999999,"5min":4.1299999999999999,"15min":4.1299999999999999},"last":0.60599999999999998}]}]},{"osd":5,"up_from":37,"seq":158913790051,"num_pgs":74,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2979900,"kb_used_data":2682544,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":90765252,"statfs":{"total":95995035648,"available":92943618048,"internally_reserved":0,"allocated":2746925056,"data_stored":2746314852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:51:28 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":1.502,"15min":0.85099999999999998},"min":{"1min":0.23599999999999999,"5min":0.183,"15min":0.183},"max":{"1min":3.6859999999999999,"5min":65.697000000000003,"15min":65.697000000000003},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.753,"5min":1.5669999999999999,"15min":0.85299999999999998},"min":{"1min":0.29899999999999999,"5min":0.219,"15min":0.219},"max":{"1min":3.0419999999999998,"5min":65.683999999999997,"15min":65.683999999999997},"last":0.92200000000000004}]},{"osd":1,"last update":"Tue Sep 10 18:51:28 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.67800000000000005,"15min":0.59799999999999998},"min":{"1min":0.313,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.0700000000000001,"5min":3.5150000000000001,"15min":3.5150000000000001},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.80100000000000005,"15min":0.625},"min":{"1min":0.29299999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.1080000000000001,"5min":6.6989999999999998,"15min":6.6989999999999998},"last":0.95099999999999996}]},{"osd":2,"last update":"Tue Sep 10 18:51:28 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.65400000000000003,"15min":0.56399999999999995},"min":{"1min":0.34999999999999998,"5min":0.249,"15min":0.249},"max":{"1min":2.5449999999999999,"5min":7.8029999999999999,"15min":7.8029999999999999},"last":1.1679999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.72299999999999998,"15min":0.60099999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.052,"5min":6.7210000000000001,"15min":6.7210000000000001},"last":0.34599999999999997}]},{"osd":3,"last update":"Tue Sep 10 18:51:28 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":1.6519999999999999,"15min":0.90100000000000002},"min":{"1min":0.28299999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":4.2450000000000001,"5min":66.593999999999994,"15min":66.593999999999994},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.7490000000000001,"15min":0.94599999999999995},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":6.6870000000000003,"5min":65.939999999999998,"15min":65.939999999999998},"last":0.32000000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:51:28 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.47799999999999998,"15min":0.41399999999999998},"min":{"1min":0.253,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":1.1379999999999999,"5min":2.133,"15min":2.133},"last":3.516},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.498,"15min":0.435},"min":{"1min":0.24399999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":2.5099999999999998,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.228}]},{"osd":6,"last update":"Tue Sep 10 18:51:52 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.76500000000000001,"15min":0.53400000000000003},"min":{"1min":0.26900000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":1.2629999999999999,"5min":11.17,"15min":11.17},"last":3.8610000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.82399999999999995,"15min":0.58199999999999996},"min":{"1min":0.36199999999999999,"5min":0.33000000000000002,"15min":0.31},"max":{"1min":2.5720000000000001,"5min":11.18,"15min":11.18},"last":3.9020000000000001}]},{"osd":7,"last update":"Tue Sep 10 18:52:15 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.75800000000000001,"15min":0.56999999999999995},"min":{"1min":0.33900000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.8039999999999998,"5min":10.782999999999999,"15min":10.782999999999999},"last":4.415},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.751,"15min":0.56699999999999995},"min":{"1min":0.314,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":3.669,"5min":11.846,"15min":11.846},"last":0.374}]}]},{"osd":4,"up_from":30,"seq":128849018985,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2099196,"kb_used_data":1793524,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":91645956,"statfs":{"total":95995035648,"available":93845458944,"internally_reserved":0,"allocated":1836568576,"data_stored":1835954115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:52:11 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":1.4890000000000001,"15min":0.84499999999999997},"min":{"1min":0.23499999999999999,"5min":0.21099999999999999,"15min":0.20699999999999999},"max":{"1min":2.1739999999999999,"5min":55.776000000000003,"15min":55.776000000000003},"last":1.597},{"interface":"front","average":{"1min":0.71499999999999997,"5min":1.4730000000000001,"15min":0.84599999999999997},"min":{"1min":0.33000000000000002,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":2.0470000000000002,"5min":55.737000000000002,"15min":55.737000000000002},"last":0.76000000000000001}]},{"osd":1,"last update":"Tue Sep 10 18:52:11 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.90500000000000003,"15min":0.68100000000000005},"min":{"1min":0.253,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":2.4849999999999999,"5min":8.1600000000000001,"15min":8.1600000000000001},"last":3.3809999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.75,"15min":0.60799999999999998},"min":{"1min":0.33900000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.48,"5min":2.5990000000000002,"15min":2.5990000000000002},"last":1.5309999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:52:11 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56599999999999995,"15min":0.55200000000000005},"min":{"1min":0.33000000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.4419999999999999,"5min":2.153,"15min":2.6030000000000002},"last":1.5720000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.58699999999999997,"15min":0.56200000000000006},"min":{"1min":0.33800000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.585,"5min":2.8109999999999999,"15min":2.8109999999999999},"last":0.82099999999999995}]},{"osd":3,"last update":"Tue Sep 10 18:52:11 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":1.2529999999999999,"15min":0.76900000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":2.1459999999999999,"5min":24.957999999999998,"15min":24.957999999999998},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.874,"5min":1.1639999999999999,"15min":0.746},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":2.5139999999999998,"5min":24.940000000000001,"15min":24.940000000000001},"last":3.4180000000000001}]},{"osd":5,"last update":"Tue Sep 10 18:51:36 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52200000000000002,"15min":0.40899999999999997},"min":{"1min":0.26000000000000001,"5min":0.222,"15min":0.222},"max":{"1min":1.151,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":3.4420000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.48199999999999998,"15min":0.40200000000000002},"min":{"1min":0.28100000000000003,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.3859999999999999,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.506}]},{"osd":6,"last update":"Tue Sep 10 18:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56399999999999995,"15min":0.47099999999999997},"min":{"1min":0.34999999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.4139999999999999,"5min":3.5529999999999999,"15min":3.5529999999999999},"last":1.552},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.65000000000000002,"15min":0.47899999999999998},"min":{"1min":0.33400000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.7549999999999999,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":3.843}]},{"osd":7,"last update":"Tue Sep 10 18:52:20 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.62,"15min":0.54000000000000004},"min":{"1min":0.33900000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.7769999999999999,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":2.8650000000000002},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.61799999999999999,"15min":0.505},"min":{"1min":0.29999999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":3.1219999999999999,"5min":3.1219999999999999,"15min":3.1219999999999999},"last":3.1669999999999998}]}]},{"osd":3,"up_from":25,"seq":107374182510,"num_pgs":76,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2424532,"kb_used_data":2114692,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":91320620,"statfs":{"total":95995035648,"available":93512314880,"internally_reserved":0,"allocated":2165444608,"data_stored":2164814140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,5],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:51:43 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":1.911,"15min":0.89200000000000002},"min":{"1min":0.17499999999999999,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":2.2749999999999999,"5min":47.601999999999997,"15min":47.601999999999997},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.7869999999999999,"15min":0.86399999999999999},"min":{"1min":0.224,"5min":0.217,"15min":0.217},"max":{"1min":4.8369999999999997,"5min":46.787999999999997,"15min":46.787999999999997},"last":1.4370000000000001}]},{"osd":1,"last update":"Tue Sep 10 18:51:43 2024","interfaces":[{"interface":"back","average":{"1min":3.8109999999999999,"5min":1.968,"15min":0.92800000000000005},"min":{"1min":0.23699999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":58.215000000000003,"5min":58.215000000000003,"15min":58.215000000000003},"last":0.64400000000000002},{"interface":"front","average":{"1min":3.7799999999999998,"5min":2.0150000000000001,"15min":0.94799999999999995},"min":{"1min":0.26400000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":57.896999999999998,"5min":57.896999999999998,"15min":57.896999999999998},"last":0.78200000000000003}]},{"osd":2,"last update":"Tue Sep 10 18:51:43 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":1.024,"15min":0.63200000000000001},"min":{"1min":0.28699999999999998,"5min":0.246,"15min":0.246},"max":{"1min":3.0190000000000001,"5min":5.8220000000000001,"15min":5.8220000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.434,"5min":0.89000000000000001,"15min":0.58299999999999996},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":1.2490000000000001,"5min":7.0979999999999999,"15min":7.0979999999999999},"last":0.76500000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:52:12 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":1.1120000000000001,"15min":0.73299999999999998},"min":{"1min":0.39400000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":2.464,"5min":7.1790000000000003,"15min":7.1790000000000003},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.70699999999999996,"5min":1.1120000000000001,"15min":0.75},"min":{"1min":0.42399999999999999,"5min":0.313,"15min":0.311},"max":{"1min":1.2589999999999999,"5min":7.1349999999999998,"15min":7.1349999999999998},"last":0.70099999999999996}]},{"osd":5,"last update":"Tue Sep 10 18:51:36 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":1.0620000000000001,"15min":0.71399999999999997},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":2.3180000000000001,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1279999999999999,"15min":0.751},"min":{"1min":0.40500000000000003,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":3.3799999999999999,"5min":9.2509999999999994,"15min":9.2509999999999994},"last":0.61699999999999999}]},{"osd":6,"last update":"Tue Sep 10 18:51:50 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":1.264,"15min":0.79600000000000004},"min":{"1min":0.47999999999999998,"5min":0.315,"15min":0.315},"max":{"1min":3.411,"5min":9.3239999999999998,"15min":9.3239999999999998},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.68400000000000005,"5min":1.1899999999999999,"15min":0.76700000000000002},"min":{"1min":0.39300000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":2.2810000000000001,"5min":7.1559999999999997,"15min":7.1559999999999997},"last":0.73299999999999998}]},{"osd":7,"last update":"Tue Sep 10 18:52:25 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":1.0860000000000001,"15min":0.72099999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":3.1059999999999999,"5min":4.71,"15min":4.71},"last":1.665},{"interface":"front","average":{"1min":0.91300000000000003,"5min":1.2509999999999999,"15min":0.80200000000000005},"min":{"1min":0.39900000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":2.9969999999999999,"5min":9.2949999999999999,"15min":9.2949999999999999},"last":0.84199999999999997}]}]},{"osd":2,"up_from":0,"seq":77309411442,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834574966,"num_pgs":74,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2421008,"kb_used_data":2119496,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":91324144,"statfs":{"total":95995035648,"available":93515923456,"internally_reserved":0,"allocated":2170363904,"data_stored":2169729379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:51:56 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":1.2050000000000001,"15min":0.61899999999999999},"min":{"1min":0.26800000000000002,"5min":0.193,"15min":0.193},"max":{"1min":2.988,"5min":13.531000000000001,"15min":13.531000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":1.7110000000000001,"15min":0.80000000000000004},"min":{"1min":0.316,"5min":0.26200000000000001,"15min":0.21199999999999999},"max":{"1min":3.0529999999999999,"5min":16.206,"15min":16.206},"last":0.45900000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:51:24 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.67600000000000005,"15min":0.46800000000000003},"min":{"1min":0.217,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":2.8159999999999998,"5min":3.8479999999999999,"15min":3.8479999999999999},"last":1.4970000000000001},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.84699999999999998,"15min":0.53800000000000003},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":3.754,"5min":7.093,"15min":7.093},"last":0.59799999999999998}]},{"osd":3,"last update":"Tue Sep 10 18:51:52 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.48,"15min":0.79800000000000004},"min":{"1min":0.36799999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.6509999999999998,"5min":12.714,"15min":12.714},"last":4.8920000000000003},{"interface":"front","average":{"1min":0.90800000000000003,"5min":1.6950000000000001,"15min":0.84599999999999997},"min":{"1min":0.29899999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":4.4960000000000004,"5min":12.696,"15min":12.696},"last":0.42799999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:52:08 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":1.1100000000000001,"15min":0.74399999999999999},"min":{"1min":0.45000000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":2.0019999999999998,"5min":4.1520000000000001,"15min":4.1520000000000001},"last":15.965999999999999},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.90300000000000002,"15min":0.65600000000000003},"min":{"1min":0.42299999999999999,"5min":0.246,"15min":0.246},"max":{"1min":2.262,"5min":3.9540000000000002,"15min":3.9540000000000002},"last":0.80300000000000005}]},{"osd":5,"last update":"Tue Sep 10 18:51:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":0.98899999999999999,"15min":0.73199999999999998},"min":{"1min":0.34100000000000003,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.871,"5min":4.5869999999999997,"15min":4.5869999999999997},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.1439999999999999,"15min":0.80800000000000005},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":4,"5min":4.1130000000000004,"15min":4.1130000000000004},"last":16.039999999999999}]},{"osd":6,"last update":"Tue Sep 10 18:51:52 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":1.466,"15min":0.91000000000000003},"min":{"1min":0.42999999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":3.851,"5min":38.039999999999999,"15min":38.039999999999999},"last":0.626},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.6240000000000001,"15min":0.98799999999999999},"min":{"1min":0.33900000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":3.9500000000000002,"5min":38.100999999999999,"15min":38.100999999999999},"last":16.013000000000002}]},{"osd":7,"last update":"Tue Sep 10 18:51:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.02,"15min":0.77200000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":3.8279999999999998,"5min":4.6539999999999999,"15min":4.6539999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":0.81100000000000005},"min":{"1min":0.36199999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":3.9700000000000002,"5min":4.1989999999999998,"15min":4.1989999999999998},"last":0.48599999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738492,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2148864,"kb_used_data":1843192,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":91596288,"statfs":{"total":95995035648,"available":93794598912,"internally_reserved":0,"allocated":1887428608,"data_stored":1886809632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 18:52:02 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":1.641,"15min":0.753},"min":{"1min":0.217,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":3.093,"5min":8.4399999999999995,"15min":8.4399999999999995},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.94799999999999995,"5min":1.696,"15min":0.77900000000000003},"min":{"1min":0.252,"5min":0.23200000000000001,"15min":0.187},"max":{"1min":5.306,"5min":7.9580000000000002,"15min":7.9580000000000002},"last":0.52600000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:51:32 2024","interfaces":[{"interface":"back","average":{"1min":1.6200000000000001,"5min":1.202,"15min":0.66200000000000003},"min":{"1min":0.254,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":6.4429999999999996,"5min":6.4429999999999996,"15min":6.4429999999999996},"last":0.22900000000000001},{"interface":"front","average":{"1min":1.377,"5min":0.97299999999999998,"15min":0.56799999999999995},"min":{"1min":0.28000000000000003,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":8.3230000000000004,"5min":8.3230000000000004,"15min":8.3230000000000004},"last":0.35699999999999998}]},{"osd":3,"last update":"Tue Sep 10 18:51:44 2024","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.7350000000000001,"15min":0.84299999999999997},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":8.4740000000000002,"5min":8.4740000000000002,"15min":8.4740000000000002},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.218,"5min":1.5649999999999999,"15min":0.79700000000000004},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":8.3670000000000009,"5min":9.8439999999999994,"15min":9.8439999999999994},"last":0.40699999999999997}]},{"osd":4,"last update":"Tue Sep 10 18:52:07 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":1.008,"15min":0.69699999999999995},"min":{"1min":0.31,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":2.605,"5min":4.992,"15min":4.992},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":1.2609999999999999,"15min":0.78300000000000003},"min":{"1min":0.35599999999999998,"5min":0.318,"15min":0.318},"max":{"1min":1.389,"5min":5.1379999999999999,"15min":5.1379999999999999},"last":1.0660000000000001}]},{"osd":5,"last update":"Tue Sep 10 18:51:32 2024","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":1.0309999999999999,"15min":0.71799999999999997},"min":{"1min":0.36799999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":5.0129999999999999,"5min":5.0129999999999999,"15min":5.0129999999999999},"last":0.434},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.202,"15min":0.77700000000000002},"min":{"1min":0.33100000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":5.3479999999999999,"5min":5.3479999999999999,"15min":5.3479999999999999},"last":0.28499999999999998}]},{"osd":6,"last update":"Tue Sep 10 18:51:57 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":1.3240000000000001,"15min":0.81200000000000006},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":3.024,"5min":4.6660000000000004,"15min":4.6660000000000004},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.80600000000000005,"5min":1.0269999999999999,"15min":0.85299999999999998},"min":{"1min":0.33900000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":5.0350000000000001,"5min":5.0350000000000001,"15min":5.0350000000000001},"last":0.47999999999999998}]},{"osd":7,"last update":"Tue Sep 10 18:52:21 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.8029999999999999,"15min":0.96999999999999997},"min":{"1min":0.31900000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":3.028,"5min":49.313000000000002,"15min":49.313000000000002},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.8660000000000001,"15min":1.016},"min":{"1min":0.42999999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":2.4980000000000002,"5min":49.063000000000002,"15min":49.063000000000002},"last":0.91300000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1887830016,"data_stored":1887830016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2180186112,"data_stored":2180186112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2178809856,"data_stored":2178809856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1871314944,"data_stored":1871314944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2795307008,"data_stored":2795307008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2172846080,"data_stored":2172846080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1871708160,"data_stored":1871708160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T18:52:30.248 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:30.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:30.249 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:30.350 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:30.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:30.389 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:30.390 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:30.467 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:52:30.490 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:30.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:52:30 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:52:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:52:30.535 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:30.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:30.536 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:30.637 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:30.679 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:30.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:30.680 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:30.781 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:30.815 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:30.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:30.816 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:30.917 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:30.954 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:30.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:30.955 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:31.056 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:31.091 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:31.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:31.093 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:31.194 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:31.199 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 42 16 83280 83264 123.884 93.8125 0.00860541 0.00806752 2024-09-10T18:52:31.230 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:31.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:31.231 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:31.331 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:31.367 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:31.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:31.368 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:31.415 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:52:30 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:52:30] "GET /metrics HTTP/1.1" 200 223940 "" "Prometheus/2.43.0" 2024-09-10T18:52:31.469 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:31.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:31.511 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:31.511 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:31.612 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:31.650 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:31.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:31.651 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:31.752 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:31.790 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:31.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:31.791 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:31.892 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:31.927 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:31.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:31.928 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:32.029 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:32.067 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:32.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:32.069 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:32.169 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:32.199 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 43 16 84704 84688 123.072 89 0.0135947 0.00812045 2024-09-10T18:52:32.205 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:32.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:32.206 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:32.307 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:32.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:32.344 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:32.344 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:32.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:32 smithi018 ceph-mon[29336]: pgmap v306: 168 pgs: 168 active+clean; 4.8 GiB data, 16 GiB used, 609 GiB / 626 GiB avail; 674 B/s rd, 74 MiB/s wr, 1.19k op/s 2024-09-10T18:52:32.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:32 smithi018 ceph-mon[40196]: pgmap v306: 168 pgs: 168 active+clean; 4.8 GiB data, 16 GiB used, 609 GiB / 626 GiB avail; 674 B/s rd, 74 MiB/s wr, 1.19k op/s 2024-09-10T18:52:32.445 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:32.484 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:32.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:32.485 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:32.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:32 smithi103 ceph-mon[33163]: pgmap v306: 168 pgs: 168 active+clean; 4.8 GiB data, 16 GiB used, 609 GiB / 626 GiB avail; 674 B/s rd, 74 MiB/s wr, 1.19k op/s 2024-09-10T18:52:32.587 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:32.622 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:32.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:32.623 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:32.665 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:52:32.724 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:32.765 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:32.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:32.766 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:32.867 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:32.902 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:32.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:32.903 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:33.004 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:33.043 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:33.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:33.044 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:33.145 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:33.183 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:33.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:33.184 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:33.199 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 44 16 86062 86046 122.204 84.875 0.0145918 0.00817815 2024-09-10T18:52:33.286 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:33.321 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:33.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:33.323 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:33.423 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:33.457 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:33.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:33.458 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:33.559 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:33.594 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:33.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:33.595 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:33.696 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:33.733 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:33.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:33.734 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:33.835 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:33.871 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:33.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:33.872 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:33.973 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:34.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:34.012 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:34.013 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:34.113 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:34.153 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:34.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:34.154 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:34.199 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 45 16 87517 87501 121.509 90.9375 0.0092079 0.00822483 2024-09-10T18:52:34.255 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:34.290 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:34.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:34.292 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:34.393 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:34.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:34.429 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:34.429 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:34.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:34 smithi018 ceph-mon[40196]: pgmap v307: 168 pgs: 168 active+clean; 5.1 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 105 MiB/s wr, 1.69k op/s 2024-09-10T18:52:34.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:34 smithi018 ceph-mon[29336]: pgmap v307: 168 pgs: 168 active+clean; 5.1 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 105 MiB/s wr, 1.69k op/s 2024-09-10T18:52:34.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:34 smithi103 ceph-mon[33163]: pgmap v307: 168 pgs: 168 active+clean; 5.1 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 105 MiB/s wr, 1.69k op/s 2024-09-10T18:52:34.530 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:34.562 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:34.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:34.564 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:34.665 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:34.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:34.705 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:34.705 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:34.806 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:34.839 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:34.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:34.840 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:34.941 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:34.981 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:34.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:34.982 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:35.083 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:35.114 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:35.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:35.115 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:35.199 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 46 16 89037 89021 120.933 95 0.00557859 0.00826471 2024-09-10T18:52:35.216 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:35.226 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-10T18:52:35.226 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T18:52:35.226 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-10T18:52:35.226 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd test-reweight-by-utilization 110 0.05 2024-09-10T18:52:35.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:35.261 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:35.261 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:35.362 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:35.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:35.409 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:35.409 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:35.418 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:35 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:52:35.418 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:35 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:52:35.418 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:35 smithi018 ceph-mon[29336]: osdmap e176: 8 total, 8 up, 7 in 2024-09-10T18:52:35.418 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:35 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:52:35.418 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:35 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:52:35.418 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:35 smithi018 ceph-mon[40196]: osdmap e176: 8 total, 8 up, 7 in 2024-09-10T18:52:35.511 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:35.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:35 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:52:35.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:35 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:52:35.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:35 smithi103 ceph-mon[33163]: osdmap e176: 8 total, 8 up, 7 in 2024-09-10T18:52:35.543 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:35.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:35.544 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:35.645 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:35.673 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:52:35.673 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:52:35.673 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:52:35.673 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:52:35.673 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:52:35.673 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:52:35.673 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:52:35.673 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:52:35.673 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:52:35.673 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:52:35.674 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:52:35.674 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:52:35.674 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:52:35.674 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:52:35.674 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:52:35.674 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:52:35.689 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:35.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:35.690 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:35.790 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:35.827 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:35.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:35.828 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:35.929 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:35.966 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:35.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:35.967 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:36.068 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:36.108 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:36.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:36.109 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:36.199 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 47 16 90495 90479 120.298 91.125 0.00947575 0.00830809 2024-09-10T18:52:36.211 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:36.254 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:36.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:36.255 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:36.356 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:36.394 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:36.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:36.395 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:36.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:36 smithi018 ceph-mon[40196]: pgmap v309: 168 pgs: 168 active+clean; 5.1 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 639 B/s rd, 57 MiB/s wr, 909 op/s 2024-09-10T18:52:36.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:36 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1251706291' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:52:36.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:36 smithi018 ceph-mon[29336]: pgmap v309: 168 pgs: 168 active+clean; 5.1 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 639 B/s rd, 57 MiB/s wr, 909 op/s 2024-09-10T18:52:36.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:36 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1251706291' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:52:36.496 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:36.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:36 smithi103 ceph-mon[33163]: pgmap v309: 168 pgs: 168 active+clean; 5.1 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 639 B/s rd, 57 MiB/s wr, 909 op/s 2024-09-10T18:52:36.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:36 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1251706291' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:52:36.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:52:36 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:52:36.546 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:36.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:36.547 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:36.648 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:36.681 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:36.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:36.682 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:36.783 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:36.823 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:36.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:36.824 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:36.925 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:36.961 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:36.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:36.962 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:37.063 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:37.100 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:37.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:37.104 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:37.200 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 48 16 92002 91986 119.754 94.1875 0.00535943 0.00834565 2024-09-10T18:52:37.205 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:37.241 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:37.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:37.242 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:37.343 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:37.380 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:37.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:37.381 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:37.482 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:37.517 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:37.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:37.518 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:37.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:37 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:37.619 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:37.658 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:37.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:37.659 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:37.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:37 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:37.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:37 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:37.761 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:37.799 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:37.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:37.800 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:37.901 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:37.938 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:37.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:37.939 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:38.040 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:38.080 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:38.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:38.081 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:38.182 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:38.199 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 49 16 93544 93528 119.276 96.375 0.00759863 0.00837883 2024-09-10T18:52:38.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:38.219 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:38.219 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:38.320 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:38.357 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:38.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:38.359 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:38.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:38 smithi018 ceph-mon[29336]: pgmap v310: 168 pgs: 168 active+clean; 5.5 GiB data, 18 GiB used, 607 GiB / 626 GiB avail; 1.2 KiB/s rd, 105 MiB/s wr, 1.69k op/s 2024-09-10T18:52:38.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:38 smithi018 ceph-mon[40196]: pgmap v310: 168 pgs: 168 active+clean; 5.5 GiB data, 18 GiB used, 607 GiB / 626 GiB avail; 1.2 KiB/s rd, 105 MiB/s wr, 1.69k op/s 2024-09-10T18:52:38.459 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:38.493 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:38.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:38.494 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:38.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:38 smithi103 ceph-mon[33163]: pgmap v310: 168 pgs: 168 active+clean; 5.5 GiB data, 18 GiB used, 607 GiB / 626 GiB avail; 1.2 KiB/s rd, 105 MiB/s wr, 1.69k op/s 2024-09-10T18:52:38.595 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:38.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:38.633 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:38.633 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:38.736 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:38.774 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:38.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:38.775 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:38.794 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:52:38.827 INFO:teuthology.orchestra.run.smithi018.stdout:ERROR: (22) Invalid argument 2024-09-10T18:52:38.828 INFO:teuthology.orchestra.run.smithi018.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:52:38.876 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:38.915 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:38.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:38.916 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:39.017 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:39.052 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:39.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:39.053 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:39.154 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:39.192 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:39.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:39.193 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:39.200 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 50 16 94996 94980 118.706 90.75 0.0157021 0.00841985 2024-09-10T18:52:39.293 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:39.335 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:39.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:39.336 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:39.437 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:39.480 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:39.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:39.481 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:39.582 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:39.591 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is unset 2024-09-10T18:52:39.619 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:39.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:39.620 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:39.721 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:39.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:39.761 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:39.761 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:39.862 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:39.892 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:39.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:39.893 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:39.924 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:39 smithi018 ceph-mon[40196]: osdmap e177: 8 total, 8 up, 7 in 2024-09-10T18:52:39.924 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:39 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3166905935' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:52:39.924 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:39 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:52:39.924 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:39 smithi018 ceph-mon[40196]: pgmap v312: 168 pgs: 168 active+clean; 5.5 GiB data, 18 GiB used, 607 GiB / 626 GiB avail; 1.2 KiB/s rd, 99 MiB/s wr, 1.59k op/s 2024-09-10T18:52:39.925 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:39 smithi018 ceph-mon[29336]: osdmap e177: 8 total, 8 up, 7 in 2024-09-10T18:52:39.925 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:39 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3166905935' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:52:39.925 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:39 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:52:39.925 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:39 smithi018 ceph-mon[29336]: pgmap v312: 168 pgs: 168 active+clean; 5.5 GiB data, 18 GiB used, 607 GiB / 626 GiB avail; 1.2 KiB/s rd, 99 MiB/s wr, 1.59k op/s 2024-09-10T18:52:39.933 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T18:52:39.995 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:40.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:39 smithi103 ceph-mon[33163]: osdmap e177: 8 total, 8 up, 7 in 2024-09-10T18:52:40.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:39 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3166905935' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:52:40.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:39 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:52:40.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:39 smithi103 ceph-mon[33163]: pgmap v312: 168 pgs: 168 active+clean; 5.5 GiB data, 18 GiB used, 607 GiB / 626 GiB avail; 1.2 KiB/s rd, 99 MiB/s wr, 1.59k op/s 2024-09-10T18:52:40.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:40.039 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:40.040 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:40.140 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:40.174 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:40.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:40.175 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:40.200 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 51 15 96479 96464 118.197 92.75 0.00797781 0.00845593 2024-09-10T18:52:40.276 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:40.312 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:40.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:40.314 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:40.416 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:40.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:40.455 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:40.455 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:40.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:52:40 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:52:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:52:40.557 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:40.594 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:40.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:40.595 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:40.695 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:40.731 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:40.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:40.733 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:40.834 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:40.865 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:40.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:40.866 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:40.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:40 smithi018 ceph-mon[29336]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:52:40.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:40 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:52:40.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:40 smithi018 ceph-mon[29336]: osdmap e178: 8 total, 8 up, 7 in 2024-09-10T18:52:40.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:40 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:52:40.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:40 smithi018 ceph-mon[40196]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:52:40.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:40 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:52:40.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:40 smithi018 ceph-mon[40196]: osdmap e178: 8 total, 8 up, 7 in 2024-09-10T18:52:40.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:40 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:52:40.967 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:41.003 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:41.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:41.004 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:41.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:40 smithi103 ceph-mon[33163]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:52:41.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:40 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:52:41.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:40 smithi103 ceph-mon[33163]: osdmap e178: 8 total, 8 up, 7 in 2024-09-10T18:52:41.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:40 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:52:41.106 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:41.144 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:41.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:41.145 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:41.193 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-10T18:52:41.201 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 52 16 97994 97978 117.741 94.625 0.0102899 0.00848844 2024-09-10T18:52:41.232 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:52:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:52:40] "GET /metrics HTTP/1.1" 200 223957 "" "Prometheus/2.43.0" 2024-09-10T18:52:41.246 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:41.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:41.289 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:41.289 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:41.392 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:41.425 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:41.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:41.426 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:41.527 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:41.567 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:41.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:41.568 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:41.669 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:41.704 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:41.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:41.705 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:41.806 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:41.838 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:41.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:41.839 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:41.861 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:41 smithi018 ceph-mon[40196]: 2.12 scrub starts 2024-09-10T18:52:41.861 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:41 smithi018 ceph-mon[40196]: 2.12 scrub ok 2024-09-10T18:52:41.861 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:41 smithi018 ceph-mon[40196]: 4.9 scrub starts 2024-09-10T18:52:41.861 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:41 smithi018 ceph-mon[40196]: 4.9 scrub ok 2024-09-10T18:52:41.862 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:41 smithi018 ceph-mon[40196]: 5.9 scrub starts 2024-09-10T18:52:41.862 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:41 smithi018 ceph-mon[40196]: 5.9 scrub ok 2024-09-10T18:52:41.862 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:41 smithi018 ceph-mon[40196]: pgmap v314: 168 pgs: 168 active+clean; 5.7 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 738 B/s rd, 82 MiB/s wr, 1.31k op/s 2024-09-10T18:52:41.862 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:41 smithi018 ceph-mon[29336]: 2.12 scrub starts 2024-09-10T18:52:41.862 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:41 smithi018 ceph-mon[29336]: 2.12 scrub ok 2024-09-10T18:52:41.862 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:41 smithi018 ceph-mon[29336]: 4.9 scrub starts 2024-09-10T18:52:41.862 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:41 smithi018 ceph-mon[29336]: 4.9 scrub ok 2024-09-10T18:52:41.862 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:41 smithi018 ceph-mon[29336]: 5.9 scrub starts 2024-09-10T18:52:41.862 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:41 smithi018 ceph-mon[29336]: 5.9 scrub ok 2024-09-10T18:52:41.863 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:41 smithi018 ceph-mon[29336]: pgmap v314: 168 pgs: 168 active+clean; 5.7 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 738 B/s rd, 82 MiB/s wr, 1.31k op/s 2024-09-10T18:52:41.940 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:41.941 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:52:41.979 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:41.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:41.980 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:42.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:41 smithi103 ceph-mon[33163]: 2.12 scrub starts 2024-09-10T18:52:42.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:41 smithi103 ceph-mon[33163]: 2.12 scrub ok 2024-09-10T18:52:42.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:41 smithi103 ceph-mon[33163]: 4.9 scrub starts 2024-09-10T18:52:42.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:41 smithi103 ceph-mon[33163]: 4.9 scrub ok 2024-09-10T18:52:42.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:41 smithi103 ceph-mon[33163]: 5.9 scrub starts 2024-09-10T18:52:42.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:41 smithi103 ceph-mon[33163]: 5.9 scrub ok 2024-09-10T18:52:42.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:41 smithi103 ceph-mon[33163]: pgmap v314: 168 pgs: 168 active+clean; 5.7 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 738 B/s rd, 82 MiB/s wr, 1.31k op/s 2024-09-10T18:52:42.080 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:42.118 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:42.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:42.119 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:42.201 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 53 16 99490 99474 117.284 93.5 0.0094583 0.00852176 2024-09-10T18:52:42.220 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:42.253 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:42.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:42.254 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:42.354 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:42.390 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:42.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:42.391 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:42.491 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:42.522 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:42.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:42.523 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:42.624 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:42.664 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:42.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:42.665 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:42.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:42 smithi018 ceph-mon[40196]: 6.1f scrub starts 2024-09-10T18:52:42.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:42 smithi018 ceph-mon[40196]: 6.1f scrub ok 2024-09-10T18:52:42.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:42 smithi018 ceph-mon[40196]: 5.7 scrub starts 2024-09-10T18:52:42.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:42 smithi018 ceph-mon[40196]: 5.7 scrub ok 2024-09-10T18:52:42.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:42 smithi018 ceph-mon[40196]: 1.9 scrub starts 2024-09-10T18:52:42.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:42 smithi018 ceph-mon[40196]: 1.9 scrub ok 2024-09-10T18:52:42.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:42 smithi018 ceph-mon[40196]: 2.7 scrub starts 2024-09-10T18:52:42.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:42 smithi018 ceph-mon[40196]: 2.7 scrub ok 2024-09-10T18:52:42.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:42 smithi018 ceph-mon[29336]: 6.1f scrub starts 2024-09-10T18:52:42.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:42 smithi018 ceph-mon[29336]: 6.1f scrub ok 2024-09-10T18:52:42.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:42 smithi018 ceph-mon[29336]: 5.7 scrub starts 2024-09-10T18:52:42.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:42 smithi018 ceph-mon[29336]: 5.7 scrub ok 2024-09-10T18:52:42.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:42 smithi018 ceph-mon[29336]: 1.9 scrub starts 2024-09-10T18:52:42.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:42 smithi018 ceph-mon[29336]: 1.9 scrub ok 2024-09-10T18:52:42.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:42 smithi018 ceph-mon[29336]: 2.7 scrub starts 2024-09-10T18:52:42.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:42 smithi018 ceph-mon[29336]: 2.7 scrub ok 2024-09-10T18:52:42.766 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:42.799 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:42.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:42.800 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:42.901 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:42.909 INFO:teuthology.orchestra.run.smithi018.stdout:moved 3 / 504 (0.595238%) 2024-09-10T18:52:42.909 INFO:teuthology.orchestra.run.smithi018.stdout:avg 72 2024-09-10T18:52:42.909 INFO:teuthology.orchestra.run.smithi018.stdout:stddev 3.0706 -> 3.16228 (expected baseline 7.85584) 2024-09-10T18:52:42.909 INFO:teuthology.orchestra.run.smithi018.stdout:min osd.0 with 67 -> 67 pgs (0.930556 -> 0.930556 * mean) 2024-09-10T18:52:42.909 INFO:teuthology.orchestra.run.smithi018.stdout:max osd.3 with 76 -> 76 pgs (1.05556 -> 1.05556 * mean) 2024-09-10T18:52:42.909 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:52:42.910 INFO:teuthology.orchestra.run.smithi018.stdout:oload 110 2024-09-10T18:52:42.910 INFO:teuthology.orchestra.run.smithi018.stdout:max_change 0.05 2024-09-10T18:52:42.910 INFO:teuthology.orchestra.run.smithi018.stdout:max_change_osds 4 2024-09-10T18:52:42.910 INFO:teuthology.orchestra.run.smithi018.stdout:average_utilization 0.0305 2024-09-10T18:52:42.910 INFO:teuthology.orchestra.run.smithi018.stdout:overload_utilization 0.0335 2024-09-10T18:52:42.910 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 weight 1.0000 -> 0.9500 2024-09-10T18:52:42.910 INFO:teuthology.orchestra.run.smithi018.stderr:no change 2024-09-10T18:52:42.945 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:42.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:42.946 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:43.013 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-10T18:52:43.021 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:52:42 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:52:42.905+0000 7f66f486f640 -1 mgr.server reply reply (1) Operation not permitted no change 2024-09-10T18:52:43.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:42 smithi103 ceph-mon[33163]: 6.1f scrub starts 2024-09-10T18:52:43.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:42 smithi103 ceph-mon[33163]: 6.1f scrub ok 2024-09-10T18:52:43.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:42 smithi103 ceph-mon[33163]: 5.7 scrub starts 2024-09-10T18:52:43.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:42 smithi103 ceph-mon[33163]: 5.7 scrub ok 2024-09-10T18:52:43.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:42 smithi103 ceph-mon[33163]: 1.9 scrub starts 2024-09-10T18:52:43.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:42 smithi103 ceph-mon[33163]: 1.9 scrub ok 2024-09-10T18:52:43.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:42 smithi103 ceph-mon[33163]: 2.7 scrub starts 2024-09-10T18:52:43.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:42 smithi103 ceph-mon[33163]: 2.7 scrub ok 2024-09-10T18:52:43.047 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:43.090 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:43.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:43.091 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:43.192 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:43.202 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 54 16 100949 100933 116.798 91.1875 0.0187825 0.00855707 2024-09-10T18:52:43.222 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:43.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:43.224 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:43.324 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:43.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:43.365 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:43.366 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:43.466 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:43.505 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:43.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:43.506 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:43.607 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:43.644 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:43.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:43.645 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:43.746 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:43.780 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:43.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:43.782 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:43.882 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:43.913 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:43.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:43.915 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:43.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:43 smithi018 ceph-mon[29336]: 5.15 scrub starts 2024-09-10T18:52:43.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:43 smithi018 ceph-mon[29336]: 5.15 scrub ok 2024-09-10T18:52:43.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:43 smithi018 ceph-mon[29336]: 6.17 scrub starts 2024-09-10T18:52:43.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:43 smithi018 ceph-mon[29336]: 6.17 scrub ok 2024-09-10T18:52:43.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:43 smithi018 ceph-mon[29336]: 2.1a scrub starts 2024-09-10T18:52:43.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:43 smithi018 ceph-mon[29336]: 2.1a scrub ok 2024-09-10T18:52:43.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:43 smithi018 ceph-mon[29336]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:52:43.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:43 smithi018 ceph-mon[29336]: pgmap v315: 168 pgs: 168 active+clean; 6.0 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.2 KiB/s rd, 115 MiB/s wr, 1.84k op/s 2024-09-10T18:52:43.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:43 smithi018 ceph-mon[29336]: osdmap e179: 8 total, 8 up, 7 in 2024-09-10T18:52:43.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:43 smithi018 ceph-mon[40196]: 5.15 scrub starts 2024-09-10T18:52:43.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:43 smithi018 ceph-mon[40196]: 5.15 scrub ok 2024-09-10T18:52:43.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:43 smithi018 ceph-mon[40196]: 6.17 scrub starts 2024-09-10T18:52:43.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:43 smithi018 ceph-mon[40196]: 6.17 scrub ok 2024-09-10T18:52:43.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:43 smithi018 ceph-mon[40196]: 2.1a scrub starts 2024-09-10T18:52:43.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:43 smithi018 ceph-mon[40196]: 2.1a scrub ok 2024-09-10T18:52:43.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:43 smithi018 ceph-mon[40196]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:52:43.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:43 smithi018 ceph-mon[40196]: pgmap v315: 168 pgs: 168 active+clean; 6.0 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.2 KiB/s rd, 115 MiB/s wr, 1.84k op/s 2024-09-10T18:52:43.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:43 smithi018 ceph-mon[40196]: osdmap e179: 8 total, 8 up, 7 in 2024-09-10T18:52:44.015 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:44.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:43 smithi103 ceph-mon[33163]: 5.15 scrub starts 2024-09-10T18:52:44.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:43 smithi103 ceph-mon[33163]: 5.15 scrub ok 2024-09-10T18:52:44.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:43 smithi103 ceph-mon[33163]: 6.17 scrub starts 2024-09-10T18:52:44.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:43 smithi103 ceph-mon[33163]: 6.17 scrub ok 2024-09-10T18:52:44.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:43 smithi103 ceph-mon[33163]: 2.1a scrub starts 2024-09-10T18:52:44.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:43 smithi103 ceph-mon[33163]: 2.1a scrub ok 2024-09-10T18:52:44.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:43 smithi103 ceph-mon[33163]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:52:44.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:43 smithi103 ceph-mon[33163]: pgmap v315: 168 pgs: 168 active+clean; 6.0 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.2 KiB/s rd, 115 MiB/s wr, 1.84k op/s 2024-09-10T18:52:44.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:43 smithi103 ceph-mon[33163]: osdmap e179: 8 total, 8 up, 7 in 2024-09-10T18:52:44.053 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:44.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:44.054 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:44.155 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:44.186 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:44.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:44.187 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:44.202 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 55 16 102423 102407 116.349 92.125 0.0165533 0.00858951 2024-09-10T18:52:44.288 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:44.325 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:44.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:44.326 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:44.427 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:44.465 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:44.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:44.466 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:44.567 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:44.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:44.602 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:44.602 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:44.703 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:44.732 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:44 smithi018 ceph-mon[40196]: 5.12 scrub starts 2024-09-10T18:52:44.732 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:44 smithi018 ceph-mon[40196]: 5.12 scrub ok 2024-09-10T18:52:44.732 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:44 smithi018 ceph-mon[40196]: 6.3 scrub starts 2024-09-10T18:52:44.732 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:44 smithi018 ceph-mon[40196]: 6.3 scrub ok 2024-09-10T18:52:44.732 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:44 smithi018 ceph-mon[40196]: 5.1 scrub starts 2024-09-10T18:52:44.733 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:44 smithi018 ceph-mon[29336]: 5.12 scrub starts 2024-09-10T18:52:44.733 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:44 smithi018 ceph-mon[29336]: 5.12 scrub ok 2024-09-10T18:52:44.733 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:44 smithi018 ceph-mon[29336]: 6.3 scrub starts 2024-09-10T18:52:44.733 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:44 smithi018 ceph-mon[29336]: 6.3 scrub ok 2024-09-10T18:52:44.733 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:44 smithi018 ceph-mon[29336]: 5.1 scrub starts 2024-09-10T18:52:44.739 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:44.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:44.740 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:44.779 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-10T18:52:44.779 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd reweight-by-utilization 110 1.0 2024-09-10T18:52:44.841 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:44.908 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:44.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:44.909 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:44.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:44 smithi103 ceph-mon[33163]: 5.12 scrub starts 2024-09-10T18:52:44.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:44 smithi103 ceph-mon[33163]: 5.12 scrub ok 2024-09-10T18:52:44.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:44 smithi103 ceph-mon[33163]: 6.3 scrub starts 2024-09-10T18:52:44.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:44 smithi103 ceph-mon[33163]: 6.3 scrub ok 2024-09-10T18:52:44.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:44 smithi103 ceph-mon[33163]: 5.1 scrub starts 2024-09-10T18:52:45.010 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:45.053 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:45.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:45.054 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:45.155 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:45.195 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:45.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:45.196 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:45.202 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 56 16 103896 103880 115.915 92.0625 0.00755413 0.00862212 2024-09-10T18:52:45.298 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:45.321 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:52:45.355 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:45.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:45.356 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:45.391 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:52:45.392 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:52:45.402 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:52:45.426 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:52:45.426 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:52:45.457 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:45.458 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:52:45.458 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:52:45.487 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:52:45.487 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:52:45.494 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:45.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:45.496 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:45.521 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:52:45.521 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:52:45.551 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:52:45.552 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:52:45.585 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:52:45.586 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:52:45.596 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:45.619 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:52:45.619 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:52:45.628 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:45.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:45.629 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:45.730 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:45.773 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:45.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:45.774 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:45.854 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:52:45.874 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:45.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:45.918 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:45.918 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:45.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:45 smithi018 ceph-mon[29336]: 2.1c scrub starts 2024-09-10T18:52:45.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:45 smithi018 ceph-mon[29336]: 5.1 scrub ok 2024-09-10T18:52:45.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:45 smithi018 ceph-mon[29336]: 2.1c scrub ok 2024-09-10T18:52:45.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:45 smithi018 ceph-mon[29336]: pgmap v317: 168 pgs: 168 active+clean; 6.0 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 776 B/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-10T18:52:45.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:45 smithi018 ceph-mon[29336]: 6.e scrub starts 2024-09-10T18:52:45.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:45 smithi018 ceph-mon[29336]: 6.e scrub ok 2024-09-10T18:52:45.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:45 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/4164555085' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:52:45.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:45 smithi018 ceph-mon[40196]: 2.1c scrub starts 2024-09-10T18:52:45.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:45 smithi018 ceph-mon[40196]: 5.1 scrub ok 2024-09-10T18:52:45.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:45 smithi018 ceph-mon[40196]: 2.1c scrub ok 2024-09-10T18:52:45.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:45 smithi018 ceph-mon[40196]: pgmap v317: 168 pgs: 168 active+clean; 6.0 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 776 B/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-10T18:52:45.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:45 smithi018 ceph-mon[40196]: 6.e scrub starts 2024-09-10T18:52:45.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:45 smithi018 ceph-mon[40196]: 6.e scrub ok 2024-09-10T18:52:45.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:45 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/4164555085' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:52:46.018 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:46.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:45 smithi103 ceph-mon[33163]: 2.1c scrub starts 2024-09-10T18:52:46.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:45 smithi103 ceph-mon[33163]: 5.1 scrub ok 2024-09-10T18:52:46.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:45 smithi103 ceph-mon[33163]: 2.1c scrub ok 2024-09-10T18:52:46.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:45 smithi103 ceph-mon[33163]: pgmap v317: 168 pgs: 168 active+clean; 6.0 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 776 B/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-10T18:52:46.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:45 smithi103 ceph-mon[33163]: 6.e scrub starts 2024-09-10T18:52:46.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:45 smithi103 ceph-mon[33163]: 6.e scrub ok 2024-09-10T18:52:46.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:45 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/4164555085' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:52:46.059 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:46.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:46.060 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:46.160 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:46.197 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:46.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:46.198 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:46.202 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 57 16 105341 105325 115.466 90.3125 0.00976342 0.00865576 2024-09-10T18:52:46.300 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:46.333 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:46.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:46.334 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:46.435 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:46.471 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:46.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:46.472 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:46.573 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:46.611 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:46.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:46.612 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:46.712 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:46.744 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:46.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:46.745 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:46.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:46 smithi103 ceph-mon[33163]: 2.e scrub starts 2024-09-10T18:52:46.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:46 smithi103 ceph-mon[33163]: 2.e scrub ok 2024-09-10T18:52:46.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:46 smithi103 ceph-mon[33163]: 2.a scrub starts 2024-09-10T18:52:46.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:46 smithi103 ceph-mon[33163]: 2.a scrub ok 2024-09-10T18:52:46.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:46 smithi103 ceph-mon[33163]: 5.e scrub starts 2024-09-10T18:52:46.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:46 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:46.773 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:52:46 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:52:46.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:46 smithi018 ceph-mon[29336]: 2.e scrub starts 2024-09-10T18:52:46.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:46 smithi018 ceph-mon[29336]: 2.e scrub ok 2024-09-10T18:52:46.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:46 smithi018 ceph-mon[29336]: 2.a scrub starts 2024-09-10T18:52:46.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:46 smithi018 ceph-mon[29336]: 2.a scrub ok 2024-09-10T18:52:46.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:46 smithi018 ceph-mon[29336]: 5.e scrub starts 2024-09-10T18:52:46.775 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:46 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:46.775 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:46 smithi018 ceph-mon[40196]: 2.e scrub starts 2024-09-10T18:52:46.775 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:46 smithi018 ceph-mon[40196]: 2.e scrub ok 2024-09-10T18:52:46.775 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:46 smithi018 ceph-mon[40196]: 2.a scrub starts 2024-09-10T18:52:46.775 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:46 smithi018 ceph-mon[40196]: 2.a scrub ok 2024-09-10T18:52:46.775 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:46 smithi018 ceph-mon[40196]: 5.e scrub starts 2024-09-10T18:52:46.775 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:46 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:46.846 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:46.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:46.883 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:46.883 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:46.984 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:47.017 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:47.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:47.018 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:47.119 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:47.155 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:47.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:47.156 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:47.203 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 58 16 106881 106865 115.135 96.25 0.00727622 0.00868069 2024-09-10T18:52:47.257 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:47.290 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:47.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:47.291 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:47.392 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:47.427 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:47.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:47.428 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:47.529 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:47.564 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:47.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:47.565 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:47.665 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:47.697 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:47.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:47.698 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:47.799 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:47.817 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:47 smithi018 ceph-mon[40196]: 5.e scrub ok 2024-09-10T18:52:47.817 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:47 smithi018 ceph-mon[40196]: 2.11 scrub starts 2024-09-10T18:52:47.817 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:47 smithi018 ceph-mon[40196]: 2.11 scrub ok 2024-09-10T18:52:47.817 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:47 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:52:47.817 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:47 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:52:47.817 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:47 smithi018 ceph-mon[40196]: pgmap v318: 168 pgs: 168 active+clean; 6.4 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.2 KiB/s rd, 116 MiB/s wr, 1.86k op/s 2024-09-10T18:52:47.817 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:47 smithi018 ceph-mon[40196]: 5.1c scrub starts 2024-09-10T18:52:47.817 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:47 smithi018 ceph-mon[29336]: 5.e scrub ok 2024-09-10T18:52:47.817 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:47 smithi018 ceph-mon[29336]: 2.11 scrub starts 2024-09-10T18:52:47.818 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:47 smithi018 ceph-mon[29336]: 2.11 scrub ok 2024-09-10T18:52:47.818 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:47 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:52:47.818 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:47 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:52:47.818 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:47 smithi018 ceph-mon[29336]: pgmap v318: 168 pgs: 168 active+clean; 6.4 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.2 KiB/s rd, 116 MiB/s wr, 1.86k op/s 2024-09-10T18:52:47.818 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:47 smithi018 ceph-mon[29336]: 5.1c scrub starts 2024-09-10T18:52:47.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:47.837 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:47.837 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:47.940 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:47.981 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:47.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:47.982 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:48.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:47 smithi103 ceph-mon[33163]: 5.e scrub ok 2024-09-10T18:52:48.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:47 smithi103 ceph-mon[33163]: 2.11 scrub starts 2024-09-10T18:52:48.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:47 smithi103 ceph-mon[33163]: 2.11 scrub ok 2024-09-10T18:52:48.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:47 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:52:48.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:47 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:52:48.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:47 smithi103 ceph-mon[33163]: pgmap v318: 168 pgs: 168 active+clean; 6.4 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.2 KiB/s rd, 116 MiB/s wr, 1.86k op/s 2024-09-10T18:52:48.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:47 smithi103 ceph-mon[33163]: 5.1c scrub starts 2024-09-10T18:52:48.083 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:48.124 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:48.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:48.125 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:48.203 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 59 16 108382 108366 114.773 93.8125 0.00530477 0.00870761 2024-09-10T18:52:48.225 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:48.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:48.263 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:48.263 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:48.364 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:48.399 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:48.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:48.400 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:48.501 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:48.540 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:48.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:48.541 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:48.641 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:48.675 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:48.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:48.676 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:48.777 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:48.816 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:48.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:48.819 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:48.819 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T18:52:48.871 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:48 smithi018 ceph-mon[29336]: 5.1c scrub ok 2024-09-10T18:52:48.872 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:48 smithi018 ceph-mon[29336]: osdmap e180: 8 total, 8 up, 7 in 2024-09-10T18:52:48.872 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:48 smithi018 ceph-mon[29336]: 2.17 scrub starts 2024-09-10T18:52:48.872 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:48 smithi018 ceph-mon[29336]: 2.17 scrub ok 2024-09-10T18:52:48.872 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:48 smithi018 ceph-mon[29336]: osdmap e181: 8 total, 8 up, 7 in 2024-09-10T18:52:48.872 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:48 smithi018 ceph-mon[40196]: 5.1c scrub ok 2024-09-10T18:52:48.872 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:48 smithi018 ceph-mon[40196]: osdmap e180: 8 total, 8 up, 7 in 2024-09-10T18:52:48.872 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:48 smithi018 ceph-mon[40196]: 2.17 scrub starts 2024-09-10T18:52:48.872 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:48 smithi018 ceph-mon[40196]: 2.17 scrub ok 2024-09-10T18:52:48.872 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:48 smithi018 ceph-mon[40196]: osdmap e181: 8 total, 8 up, 7 in 2024-09-10T18:52:48.920 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:48.954 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:48.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:48.956 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:49.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:48 smithi103 ceph-mon[33163]: 5.1c scrub ok 2024-09-10T18:52:49.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:48 smithi103 ceph-mon[33163]: osdmap e180: 8 total, 8 up, 7 in 2024-09-10T18:52:49.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:48 smithi103 ceph-mon[33163]: 2.17 scrub starts 2024-09-10T18:52:49.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:48 smithi103 ceph-mon[33163]: 2.17 scrub ok 2024-09-10T18:52:49.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:48 smithi103 ceph-mon[33163]: osdmap e181: 8 total, 8 up, 7 in 2024-09-10T18:52:49.057 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:49.099 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:49.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:49.100 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:49.201 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:49.205 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:2024-09-10T18:52:49.202983+0000 min lat: 0.00115328 max lat: 0.157801 avg lat: 0.00874231 2024-09-10T18:52:49.205 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T18:52:49.205 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 60 16 109792 109776 114.328 88.125 0.0192944 0.00874231 2024-09-10T18:52:49.243 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:49.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:49.245 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:49.345 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:49.380 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:49.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:49.381 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:49.482 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:49.521 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:49.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:49.522 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:49.623 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:49.711 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:49.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:49.712 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:49.813 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:49.824 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:49 smithi018 ceph-mon[29336]: 6.6 scrub starts 2024-09-10T18:52:49.824 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:49 smithi018 ceph-mon[29336]: 6.6 scrub ok 2024-09-10T18:52:49.824 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:49 smithi018 ceph-mon[29336]: 6.a scrub starts 2024-09-10T18:52:49.824 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:49 smithi018 ceph-mon[29336]: 6.a scrub ok 2024-09-10T18:52:49.824 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:49 smithi018 ceph-mon[29336]: 6.15 scrub starts 2024-09-10T18:52:49.824 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:49 smithi018 ceph-mon[29336]: 6.15 scrub ok 2024-09-10T18:52:49.824 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:49 smithi018 ceph-mon[29336]: pgmap v321: 168 pgs: 168 active+clean; 6.4 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 852 B/s rd, 77 MiB/s wr, 1.23k op/s 2024-09-10T18:52:49.825 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:49 smithi018 ceph-mon[40196]: 6.6 scrub starts 2024-09-10T18:52:49.825 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:49 smithi018 ceph-mon[40196]: 6.6 scrub ok 2024-09-10T18:52:49.825 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:49 smithi018 ceph-mon[40196]: 6.a scrub starts 2024-09-10T18:52:49.825 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:49 smithi018 ceph-mon[40196]: 6.a scrub ok 2024-09-10T18:52:49.825 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:49 smithi018 ceph-mon[40196]: 6.15 scrub starts 2024-09-10T18:52:49.825 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:49 smithi018 ceph-mon[40196]: 6.15 scrub ok 2024-09-10T18:52:49.825 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:49 smithi018 ceph-mon[40196]: pgmap v321: 168 pgs: 168 active+clean; 6.4 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 852 B/s rd, 77 MiB/s wr, 1.23k op/s 2024-09-10T18:52:49.849 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:49.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:49.850 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:49.951 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:49.989 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:49.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:49.990 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:50.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:49 smithi103 ceph-mon[33163]: 6.6 scrub starts 2024-09-10T18:52:50.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:49 smithi103 ceph-mon[33163]: 6.6 scrub ok 2024-09-10T18:52:50.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:49 smithi103 ceph-mon[33163]: 6.a scrub starts 2024-09-10T18:52:50.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:49 smithi103 ceph-mon[33163]: 6.a scrub ok 2024-09-10T18:52:50.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:49 smithi103 ceph-mon[33163]: 6.15 scrub starts 2024-09-10T18:52:50.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:49 smithi103 ceph-mon[33163]: 6.15 scrub ok 2024-09-10T18:52:50.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:49 smithi103 ceph-mon[33163]: pgmap v321: 168 pgs: 168 active+clean; 6.4 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 852 B/s rd, 77 MiB/s wr, 1.23k op/s 2024-09-10T18:52:50.091 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:50.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:50.135 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:50.135 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:50.203 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 61 16 111269 111253 113.967 92.3125 0.00983621 0.00876974 2024-09-10T18:52:50.236 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:50.274 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:50.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:50.275 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:50.376 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:50.413 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:50.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:50.414 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:50.515 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:50.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:52:50 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:52:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:52:50.548 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:50.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:50.549 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:50.650 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:50.666 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is unset 2024-09-10T18:52:50.679 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:50.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:50.680 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:50.781 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:50.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:50.819 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:50.819 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:50.920 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:50.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:50 smithi018 ceph-mon[40196]: 6.13 scrub starts 2024-09-10T18:52:50.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:50 smithi018 ceph-mon[40196]: 6.13 scrub ok 2024-09-10T18:52:50.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:50 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/128268598' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:52:50.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:50 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:52:50.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:50 smithi018 ceph-mon[29336]: 6.13 scrub starts 2024-09-10T18:52:50.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:50 smithi018 ceph-mon[29336]: 6.13 scrub ok 2024-09-10T18:52:50.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:50 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/128268598' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:52:50.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:50 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:52:50.953 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:50.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:50.954 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:51.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:50 smithi103 ceph-mon[33163]: 6.13 scrub starts 2024-09-10T18:52:51.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:50 smithi103 ceph-mon[33163]: 6.13 scrub ok 2024-09-10T18:52:51.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:50 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/128268598' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:52:51.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:50 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:52:51.055 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:51.086 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:51.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:51.087 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:51.188 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:51.203 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 62 16 112766 112750 113.638 93.5625 0.00680744 0.0087955 2024-09-10T18:52:51.223 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:51.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:51.224 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:51.325 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:51.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:51.363 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:51.364 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:51.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:52:50 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:52:50] "GET /metrics HTTP/1.1" 200 223957 "" "Prometheus/2.43.0" 2024-09-10T18:52:51.464 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:51.505 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:51.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:51.506 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:51.607 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:51.646 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:51.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:51.646 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:51.747 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:51.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:51.782 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:51.782 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:51.810 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:51 smithi018 ceph-mon[29336]: 1.1 scrub starts 2024-09-10T18:52:51.811 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:51 smithi018 ceph-mon[29336]: 1.1 scrub ok 2024-09-10T18:52:51.811 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:51 smithi018 ceph-mon[29336]: 1.6 scrub starts 2024-09-10T18:52:51.811 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:51 smithi018 ceph-mon[29336]: 1.6 scrub ok 2024-09-10T18:52:51.811 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:51 smithi018 ceph-mon[29336]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:52:51.811 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:51 smithi018 ceph-mon[29336]: Cluster is now healthy 2024-09-10T18:52:51.811 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:51 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:52:51.811 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:51 smithi018 ceph-mon[29336]: osdmap e182: 8 total, 8 up, 7 in 2024-09-10T18:52:51.811 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:51 smithi018 ceph-mon[29336]: pgmap v323: 168 pgs: 168 active+clean; 6.6 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 853 B/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-10T18:52:51.811 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:51 smithi018 ceph-mon[29336]: 4.e scrub starts 2024-09-10T18:52:51.811 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:51 smithi018 ceph-mon[29336]: 4.e scrub ok 2024-09-10T18:52:51.811 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:51 smithi018 ceph-mon[40196]: 1.1 scrub starts 2024-09-10T18:52:51.812 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:51 smithi018 ceph-mon[40196]: 1.1 scrub ok 2024-09-10T18:52:51.812 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:51 smithi018 ceph-mon[40196]: 1.6 scrub starts 2024-09-10T18:52:51.812 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:51 smithi018 ceph-mon[40196]: 1.6 scrub ok 2024-09-10T18:52:51.812 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:51 smithi018 ceph-mon[40196]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:52:51.812 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:51 smithi018 ceph-mon[40196]: Cluster is now healthy 2024-09-10T18:52:51.812 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:51 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:52:51.812 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:51 smithi018 ceph-mon[40196]: osdmap e182: 8 total, 8 up, 7 in 2024-09-10T18:52:51.812 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:51 smithi018 ceph-mon[40196]: pgmap v323: 168 pgs: 168 active+clean; 6.6 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 853 B/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-10T18:52:51.812 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:51 smithi018 ceph-mon[40196]: 4.e scrub starts 2024-09-10T18:52:51.812 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:51 smithi018 ceph-mon[40196]: 4.e scrub ok 2024-09-10T18:52:51.883 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:51.922 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:51.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:51.924 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:52.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:51 smithi103 ceph-mon[33163]: 1.1 scrub starts 2024-09-10T18:52:52.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:51 smithi103 ceph-mon[33163]: 1.1 scrub ok 2024-09-10T18:52:52.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:51 smithi103 ceph-mon[33163]: 1.6 scrub starts 2024-09-10T18:52:52.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:51 smithi103 ceph-mon[33163]: 1.6 scrub ok 2024-09-10T18:52:52.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:51 smithi103 ceph-mon[33163]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:52:52.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:51 smithi103 ceph-mon[33163]: Cluster is now healthy 2024-09-10T18:52:52.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:51 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:52:52.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:51 smithi103 ceph-mon[33163]: osdmap e182: 8 total, 8 up, 7 in 2024-09-10T18:52:52.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:51 smithi103 ceph-mon[33163]: pgmap v323: 168 pgs: 168 active+clean; 6.6 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 853 B/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-10T18:52:52.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:51 smithi103 ceph-mon[33163]: 4.e scrub starts 2024-09-10T18:52:52.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:51 smithi103 ceph-mon[33163]: 4.e scrub ok 2024-09-10T18:52:52.024 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:52.061 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:52.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:52.062 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:52.163 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:52.201 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:52.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:52.202 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:52.203 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 63 16 114255 114239 113.311 93.0625 0.0122643 0.00882072 2024-09-10T18:52:52.303 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:52.324 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:52.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:52.325 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:52.425 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:52.452 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:52.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:52.453 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:52.553 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:52.589 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:52.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:52.590 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:52.690 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:52.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:52 smithi018 ceph-mon[40196]: 6.7 scrub starts 2024-09-10T18:52:52.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:52 smithi018 ceph-mon[40196]: 6.7 scrub ok 2024-09-10T18:52:52.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:52 smithi018 ceph-mon[40196]: 1.7 scrub starts 2024-09-10T18:52:52.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:52 smithi018 ceph-mon[40196]: 1.7 scrub ok 2024-09-10T18:52:52.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:52 smithi018 ceph-mon[29336]: 6.7 scrub starts 2024-09-10T18:52:52.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:52 smithi018 ceph-mon[29336]: 6.7 scrub ok 2024-09-10T18:52:52.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:52 smithi018 ceph-mon[29336]: 1.7 scrub starts 2024-09-10T18:52:52.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:52 smithi018 ceph-mon[29336]: 1.7 scrub ok 2024-09-10T18:52:52.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:52.729 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:52.729 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:52.830 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:52.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:52.863 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:52.864 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:52.964 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:53.003 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:53.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:53.004 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:53.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:52 smithi103 ceph-mon[33163]: 6.7 scrub starts 2024-09-10T18:52:53.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:52 smithi103 ceph-mon[33163]: 6.7 scrub ok 2024-09-10T18:52:53.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:52 smithi103 ceph-mon[33163]: 1.7 scrub starts 2024-09-10T18:52:53.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:52 smithi103 ceph-mon[33163]: 1.7 scrub ok 2024-09-10T18:52:53.105 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:53.140 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:53.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:53.141 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:53.203 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 64 15 115392 115377 112.652 71.125 0.00900712 0.00887251 2024-09-10T18:52:53.242 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:53.280 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:53.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:53.281 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:53.382 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:53.415 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:52:53.418 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:53.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:53.420 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:53.453 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T18:52:53.453 INFO:teuthology.orchestra.run.smithi018.stdout: "size": 20, 2024-09-10T18:52:53.453 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 600, 2024-09-10T18:52:53.453 INFO:teuthology.orchestra.run.smithi018.stdout: "ops": [ 2024-09-10T18:52:53.453 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.453 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.14790.0:15121 7.2 7:478eee66:::benchmark_data_smithi018_78628_object15120:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-10T18:52:53.453 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:47:49.069828+0000", 2024-09-10T18:52:53.453 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 304.37754690600002, 2024-09-10T18:52:53.453 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.13612658799999999, 2024-09-10T18:52:53.454 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:52:53.454 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:52:53.454 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:52:53.454 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.14790", 2024-09-10T18:52:53.454 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:52:53.454 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 15121 2024-09-10T18:52:53.454 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.454 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:52:53.454 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.454 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:52:53.454 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.069828+0000", 2024-09-10T18:52:53.454 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.454 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.455 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.455 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:52:53.455 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.069828+0000", 2024-09-10T18:52:53.455 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.455 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.455 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.455 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:52:53.455 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.069827+0000", 2024-09-10T18:52:53.455 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:52:53.455 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.455 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.455 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:52:53.455 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.069847+0000", 2024-09-10T18:52:53.456 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.9604999999999999e-05 2024-09-10T18:52:53.456 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.456 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.456 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:52:53.456 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.069854+0000", 2024-09-10T18:52:53.456 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.002e-06 2024-09-10T18:52:53.456 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.456 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.456 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:52:53.456 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.069857+0000", 2024-09-10T18:52:53.456 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.579e-06 2024-09-10T18:52:53.456 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.457 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.457 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:52:53.457 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.069883+0000", 2024-09-10T18:52:53.457 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.5367999999999998e-05 2024-09-10T18:52:53.457 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.457 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.460 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:52:53.460 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.069994+0000", 2024-09-10T18:52:53.460 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00011136 2024-09-10T18:52:53.460 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.460 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.460 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,7", 2024-09-10T18:52:53.460 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.203177+0000", 2024-09-10T18:52:53.460 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.13318323200000001 2024-09-10T18:52:53.461 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.461 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.461 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:52:53.461 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.204882+0000", 2024-09-10T18:52:53.461 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0017046240000000001 2024-09-10T18:52:53.461 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.461 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.461 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:52:53.461 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.205827+0000", 2024-09-10T18:52:53.461 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00094547800000000003 2024-09-10T18:52:53.461 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.461 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.462 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:52:53.462 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.205846+0000", 2024-09-10T18:52:53.462 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.857e-05 2024-09-10T18:52:53.462 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.462 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.462 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:52:53.462 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.205919+0000", 2024-09-10T18:52:53.462 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.3083000000000005e-05 2024-09-10T18:52:53.462 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.462 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.462 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:52:53.462 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.205954+0000", 2024-09-10T18:52:53.462 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.5620000000000001e-05 2024-09-10T18:52:53.463 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.463 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:52:53.463 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.463 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.463 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.463 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.14790.0:15116 7.e e92/88 7:75fa8b39:::benchmark_data_smithi018_78628_object15115:head v 92'910, mlcod=92'908)", 2024-09-10T18:52:53.463 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:47:49.069844+0000", 2024-09-10T18:52:53.463 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 304.37753079800001, 2024-09-10T18:52:53.463 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.13642999, 2024-09-10T18:52:53.463 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:52:53.463 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:52:53.463 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:52:53.463 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.464 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:52:53.464 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.069844+0000", 2024-09-10T18:52:53.464 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.464 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.464 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.464 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:52:53.464 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.069844+0000", 2024-09-10T18:52:53.464 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.464 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.464 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.464 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:52:53.464 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.069843+0000", 2024-09-10T18:52:53.465 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T18:52:53.465 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.465 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.465 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:52:53.465 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.069878+0000", 2024-09-10T18:52:53.465 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.5024000000000002e-05 2024-09-10T18:52:53.465 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.465 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.465 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:52:53.465 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.069888+0000", 2024-09-10T18:52:53.465 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0052e-05 2024-09-10T18:52:53.465 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.466 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.466 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:52:53.466 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.069894+0000", 2024-09-10T18:52:53.466 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.0509999999999996e-06 2024-09-10T18:52:53.466 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.466 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.466 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:52:53.466 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.069930+0000", 2024-09-10T18:52:53.466 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.6309000000000001e-05 2024-09-10T18:52:53.466 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.466 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.466 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:52:53.466 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.069947+0000", 2024-09-10T18:52:53.467 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.6750999999999999e-05 2024-09-10T18:52:53.467 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.467 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.467 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:52:53.467 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.206242+0000", 2024-09-10T18:52:53.467 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.13629587500000001 2024-09-10T18:52:53.467 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.467 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.467 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:52:53.467 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.206274+0000", 2024-09-10T18:52:53.467 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.1513999999999997e-05 2024-09-10T18:52:53.467 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.467 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:52:53.467 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.468 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.468 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.468 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.14790.0:15123 7.1 e92/86 7:8ab84aee:::benchmark_data_smithi018_78628_object15122:head v 92'901, mlcod=92'898)", 2024-09-10T18:52:53.468 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:47:49.070295+0000", 2024-09-10T18:52:53.468 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 304.37707988800003, 2024-09-10T18:52:53.468 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.13662692900000001, 2024-09-10T18:52:53.468 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:52:53.468 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:52:53.468 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:52:53.468 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.468 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:52:53.468 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.070295+0000", 2024-09-10T18:52:53.468 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.469 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.469 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.469 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:52:53.469 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.070295+0000", 2024-09-10T18:52:53.469 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.469 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.469 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.469 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:52:53.469 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.070294+0000", 2024-09-10T18:52:53.469 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T18:52:53.469 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.469 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.469 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:52:53.470 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.070359+0000", 2024-09-10T18:52:53.470 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.4734999999999996e-05 2024-09-10T18:52:53.470 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.470 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.470 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:52:53.470 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.203553+0000", 2024-09-10T18:52:53.470 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.133194387 2024-09-10T18:52:53.470 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.470 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.470 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:52:53.470 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.203559+0000", 2024-09-10T18:52:53.470 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.2839999999999997e-06 2024-09-10T18:52:53.470 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.470 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.470 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:52:53.471 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.203597+0000", 2024-09-10T18:52:53.471 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.7885000000000001e-05 2024-09-10T18:52:53.471 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.471 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.471 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:52:53.471 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.203614+0000", 2024-09-10T18:52:53.471 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.6912e-05 2024-09-10T18:52:53.471 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.471 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.471 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:52:53.471 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.206893+0000", 2024-09-10T18:52:53.471 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.003279779 2024-09-10T18:52:53.471 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.471 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.472 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:52:53.472 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.206922+0000", 2024-09-10T18:52:53.472 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.8498999999999999e-05 2024-09-10T18:52:53.472 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.472 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:52:53.472 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.472 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.472 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.472 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.14790.0:15122 7.7 e92/86 7:ee9d697a:::benchmark_data_smithi018_78628_object15121:head v 92'958, mlcod=92'954)", 2024-09-10T18:52:53.472 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:47:49.070588+0000", 2024-09-10T18:52:53.472 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 304.376786896, 2024-09-10T18:52:53.472 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.13717164700000001, 2024-09-10T18:52:53.472 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:52:53.472 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:52:53.473 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:52:53.473 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.473 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:52:53.473 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.070588+0000", 2024-09-10T18:52:53.473 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.473 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.473 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.473 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:52:53.473 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.070588+0000", 2024-09-10T18:52:53.473 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.473 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.473 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.473 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:52:53.473 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.070587+0000", 2024-09-10T18:52:53.474 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T18:52:53.474 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.474 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.474 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:52:53.474 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.070648+0000", 2024-09-10T18:52:53.474 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.0353000000000003e-05 2024-09-10T18:52:53.474 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.474 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.474 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:52:53.474 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.202941+0000", 2024-09-10T18:52:53.474 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.132293829 2024-09-10T18:52:53.474 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.474 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.474 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:52:53.475 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.202956+0000", 2024-09-10T18:52:53.475 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.4642e-05 2024-09-10T18:52:53.475 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.475 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.475 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:52:53.475 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.203002+0000", 2024-09-10T18:52:53.475 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.5967000000000002e-05 2024-09-10T18:52:53.475 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.475 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.475 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:52:53.475 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.203032+0000", 2024-09-10T18:52:53.475 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.9451999999999999e-05 2024-09-10T18:52:53.475 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.476 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.476 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:52:53.476 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.207617+0000", 2024-09-10T18:52:53.476 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0045850420000000001 2024-09-10T18:52:53.476 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.476 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.476 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:52:53.476 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:47:49.207760+0000", 2024-09-10T18:52:53.476 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000142967 2024-09-10T18:52:53.476 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.476 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:52:53.476 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.476 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.476 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.477 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.14790.0:109189 7.a 7:5fe4c471:::benchmark_data_smithi018_78628_object109188:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-10T18:52:53.477 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:48:50.751793+0000", 2024-09-10T18:52:53.477 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 242.69558173300001, 2024-09-10T18:52:53.477 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.13653480000000001, 2024-09-10T18:52:53.477 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:52:53.477 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:52:53.477 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:52:53.477 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.14790", 2024-09-10T18:52:53.477 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:52:53.477 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 109189 2024-09-10T18:52:53.477 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.477 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:52:53.477 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.477 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:52:53.477 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:48:50.751793+0000", 2024-09-10T18:52:53.478 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.478 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.478 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.478 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:52:53.478 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:48:50.751793+0000", 2024-09-10T18:52:53.478 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.478 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.478 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.478 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:52:53.478 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:48:50.751792+0000", 2024-09-10T18:52:53.478 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T18:52:53.478 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.478 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.478 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:52:53.479 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:48:50.751812+0000", 2024-09-10T18:52:53.479 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.9417999999999999e-05 2024-09-10T18:52:53.479 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.479 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.479 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:52:53.479 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:48:50.751818+0000", 2024-09-10T18:52:53.479 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.9370000000000001e-06 2024-09-10T18:52:53.479 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.479 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.479 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:52:53.479 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:48:50.751822+0000", 2024-09-10T18:52:53.479 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.0999999999999997e-06 2024-09-10T18:52:53.479 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.479 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.480 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:52:53.480 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:48:50.756693+0000", 2024-09-10T18:52:53.480 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0048715329999999999 2024-09-10T18:52:53.480 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.480 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.480 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:52:53.480 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:48:50.756811+0000", 2024-09-10T18:52:53.480 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000117204 2024-09-10T18:52:53.480 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.480 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.480 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,5", 2024-09-10T18:52:53.480 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:48:50.756948+0000", 2024-09-10T18:52:53.480 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000136959 2024-09-10T18:52:53.481 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.481 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.481 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:52:53.481 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:48:50.879139+0000", 2024-09-10T18:52:53.481 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.122190967 2024-09-10T18:52:53.481 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.481 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.481 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:52:53.481 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:48:50.879264+0000", 2024-09-10T18:52:53.481 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00012574299999999999 2024-09-10T18:52:53.481 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.481 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.481 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:52:53.481 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:48:50.888247+0000", 2024-09-10T18:52:53.482 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0089824180000000007 2024-09-10T18:52:53.482 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.482 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.482 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:52:53.482 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:48:50.888291+0000", 2024-09-10T18:52:53.482 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.4060000000000002e-05 2024-09-10T18:52:53.482 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.482 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.482 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:52:53.482 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:48:50.888328+0000", 2024-09-10T18:52:53.482 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.7070000000000003e-05 2024-09-10T18:52:53.482 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.482 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:52:53.483 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.483 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.483 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.483 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.14790.0:177356 7.a 7:56e7abbb:::benchmark_data_smithi018_78628_object41136:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124)", 2024-09-10T18:52:53.483 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:49:50.562174+0000", 2024-09-10T18:52:53.483 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 182.885200303, 2024-09-10T18:52:53.483 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.14744474599999999, 2024-09-10T18:52:53.483 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:52:53.483 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:52:53.483 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:52:53.483 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.14790", 2024-09-10T18:52:53.483 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:52:53.483 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 177356 2024-09-10T18:52:53.484 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.484 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:52:53.484 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.484 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:52:53.484 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:50.562174+0000", 2024-09-10T18:52:53.484 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.484 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.484 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.484 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:52:53.484 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:50.562174+0000", 2024-09-10T18:52:53.484 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.484 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.484 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.485 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:52:53.485 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:50.562163+0000", 2024-09-10T18:52:53.485 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999989 2024-09-10T18:52:53.486 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.486 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.486 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:52:53.486 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:50.562190+0000", 2024-09-10T18:52:53.486 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.6262e-05 2024-09-10T18:52:53.486 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.486 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.486 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:52:53.486 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:50.562277+0000", 2024-09-10T18:52:53.486 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.7004999999999998e-05 2024-09-10T18:52:53.486 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.486 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.486 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:52:53.486 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:50.562284+0000", 2024-09-10T18:52:53.487 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.6310000000000003e-06 2024-09-10T18:52:53.487 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.487 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.487 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:52:53.487 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:50.567372+0000", 2024-09-10T18:52:53.487 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0050881559999999999 2024-09-10T18:52:53.487 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.487 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.487 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:52:53.487 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:50.567490+0000", 2024-09-10T18:52:53.487 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000117538 2024-09-10T18:52:53.487 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.487 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.487 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,5", 2024-09-10T18:52:53.488 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:50.567604+0000", 2024-09-10T18:52:53.488 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000113563 2024-09-10T18:52:53.488 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.488 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.488 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:52:53.488 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:50.576290+0000", 2024-09-10T18:52:53.488 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0086859209999999992 2024-09-10T18:52:53.488 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.488 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.488 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:52:53.488 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:50.576305+0000", 2024-09-10T18:52:53.488 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.541e-05 2024-09-10T18:52:53.488 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.488 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.489 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:52:53.489 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:50.709498+0000", 2024-09-10T18:52:53.489 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.133192638 2024-09-10T18:52:53.489 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.489 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.489 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:52:53.489 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:50.709565+0000", 2024-09-10T18:52:53.489 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.7471999999999994e-05 2024-09-10T18:52:53.489 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.489 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.489 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:52:53.489 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:50.709619+0000", 2024-09-10T18:52:53.489 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.4218000000000002e-05 2024-09-10T18:52:53.489 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.490 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:52:53.490 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.490 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.490 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.490 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.14790.0:177366 7.a 7:52a5f767:::benchmark_data_smithi018_78628_object41146:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124)", 2024-09-10T18:52:53.490 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:49:50.576923+0000", 2024-09-10T18:52:53.490 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 182.87045148199999, 2024-09-10T18:52:53.490 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.13892997400000001, 2024-09-10T18:52:53.490 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:52:53.490 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:52:53.490 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:52:53.490 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.14790", 2024-09-10T18:52:53.490 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:52:53.490 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 177366 2024-09-10T18:52:53.491 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.491 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:52:53.491 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.491 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:52:53.491 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:50.576923+0000", 2024-09-10T18:52:53.491 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.491 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.491 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.491 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:52:53.491 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:50.576923+0000", 2024-09-10T18:52:53.491 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.491 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.491 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.491 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:52:53.491 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:50.576922+0000", 2024-09-10T18:52:53.492 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T18:52:53.492 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.492 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.492 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:52:53.492 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:50.576928+0000", 2024-09-10T18:52:53.492 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.9050000000000002e-06 2024-09-10T18:52:53.492 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.492 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.492 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:52:53.492 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:50.576939+0000", 2024-09-10T18:52:53.492 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1114999999999999e-05 2024-09-10T18:52:53.492 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.492 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.492 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:52:53.493 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:50.576945+0000", 2024-09-10T18:52:53.493 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.7749999999999998e-06 2024-09-10T18:52:53.493 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.493 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.493 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:52:53.493 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:50.577012+0000", 2024-09-10T18:52:53.493 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.7225999999999996e-05 2024-09-10T18:52:53.493 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.493 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.493 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:52:53.493 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:50.577177+0000", 2024-09-10T18:52:53.493 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000165057 2024-09-10T18:52:53.494 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.494 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.494 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,5", 2024-09-10T18:52:53.494 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:50.577304+0000", 2024-09-10T18:52:53.494 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00012744000000000001 2024-09-10T18:52:53.494 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.494 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.494 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:52:53.494 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:50.581829+0000", 2024-09-10T18:52:53.494 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0045251040000000003 2024-09-10T18:52:53.494 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.494 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.494 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:52:53.494 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:50.582936+0000", 2024-09-10T18:52:53.494 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.001106814 2024-09-10T18:52:53.495 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.495 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.495 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:52:53.495 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:50.715772+0000", 2024-09-10T18:52:53.495 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.13283552000000001 2024-09-10T18:52:53.495 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.495 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.495 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:52:53.495 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:50.715816+0000", 2024-09-10T18:52:53.495 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.4350000000000001e-05 2024-09-10T18:52:53.495 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.495 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.495 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:52:53.495 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:49:50.715853+0000", 2024-09-10T18:52:53.495 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.7128000000000001e-05 2024-09-10T18:52:53.496 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.496 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:52:53.496 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.496 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.496 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.496 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.14790.0:198282 7.6 e130/86 7:6424f68e:::benchmark_data_smithi018_78628_object62062:head v 130'12277, mlcod=129'12275)", 2024-09-10T18:52:53.496 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:50:08.404597+0000", 2024-09-10T18:52:53.496 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 165.04277814700001, 2024-09-10T18:52:53.496 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.14311860300000001, 2024-09-10T18:52:53.496 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:52:53.496 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:52:53.496 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:52:53.496 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.496 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:52:53.497 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.404597+0000", 2024-09-10T18:52:53.497 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.497 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.497 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.497 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:52:53.497 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.404597+0000", 2024-09-10T18:52:53.497 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.497 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.497 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.497 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:52:53.497 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.404596+0000", 2024-09-10T18:52:53.497 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T18:52:53.497 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.498 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.498 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:52:53.498 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.404602+0000", 2024-09-10T18:52:53.498 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.224e-06 2024-09-10T18:52:53.498 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.498 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.498 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:52:53.498 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.404612+0000", 2024-09-10T18:52:53.498 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.8199999999999992e-06 2024-09-10T18:52:53.498 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.498 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.498 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:52:53.498 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.404618+0000", 2024-09-10T18:52:53.498 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.558e-06 2024-09-10T18:52:53.499 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.499 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.499 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:52:53.499 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.404691+0000", 2024-09-10T18:52:53.499 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.3775000000000004e-05 2024-09-10T18:52:53.499 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.499 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.499 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:52:53.499 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.404712+0000", 2024-09-10T18:52:53.499 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.1109999999999999e-05 2024-09-10T18:52:53.499 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.499 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.499 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:52:53.499 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.547555+0000", 2024-09-10T18:52:53.500 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.14284206599999999 2024-09-10T18:52:53.500 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.500 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.500 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:52:53.500 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.547715+0000", 2024-09-10T18:52:53.500 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00016074899999999999 2024-09-10T18:52:53.500 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.500 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:52:53.500 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.500 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.500 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.500 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.14790.0:198287 7.e e130/88 7:7c4ef593:::benchmark_data_smithi018_78628_object62067:head v 130'12303, mlcod=129'12300)", 2024-09-10T18:52:53.500 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:50:08.408810+0000", 2024-09-10T18:52:53.500 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 165.038564374, 2024-09-10T18:52:53.501 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.139016426, 2024-09-10T18:52:53.501 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:52:53.501 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:52:53.501 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:52:53.501 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.501 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:52:53.501 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.408810+0000", 2024-09-10T18:52:53.501 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.501 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.501 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.501 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:52:53.501 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.408810+0000", 2024-09-10T18:52:53.501 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.502 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.502 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.502 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:52:53.502 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.408810+0000", 2024-09-10T18:52:53.502 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967296 2024-09-10T18:52:53.502 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.502 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.502 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:52:53.502 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.408814+0000", 2024-09-10T18:52:53.502 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.8099999999999999e-06 2024-09-10T18:52:53.502 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.502 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.502 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:52:53.502 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.408818+0000", 2024-09-10T18:52:53.503 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.3889999999999998e-06 2024-09-10T18:52:53.503 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.503 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.503 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:52:53.503 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.408822+0000", 2024-09-10T18:52:53.503 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.2169999999999999e-06 2024-09-10T18:52:53.503 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.503 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.503 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:52:53.503 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.408867+0000", 2024-09-10T18:52:53.503 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.5686000000000003e-05 2024-09-10T18:52:53.503 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.503 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.503 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:52:53.504 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.408886+0000", 2024-09-10T18:52:53.504 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.8377e-05 2024-09-10T18:52:53.504 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.504 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.504 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:52:53.504 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.547790+0000", 2024-09-10T18:52:53.504 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.13890459899999999 2024-09-10T18:52:53.504 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.504 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.504 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:52:53.504 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.547827+0000", 2024-09-10T18:52:53.504 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.6548000000000003e-05 2024-09-10T18:52:53.505 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.505 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:52:53.505 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.505 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.505 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.505 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.14790.0:198289 7.6 e130/86 7:6a14f75a:::benchmark_data_smithi018_78628_object62069:head v 130'12278, mlcod=129'12275)", 2024-09-10T18:52:53.505 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:50:08.408848+0000", 2024-09-10T18:52:53.505 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 165.038526305, 2024-09-10T18:52:53.505 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.14199599399999999, 2024-09-10T18:52:53.505 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:52:53.505 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:52:53.505 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:52:53.505 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.505 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:52:53.506 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.408848+0000", 2024-09-10T18:52:53.506 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.506 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.506 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.506 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:52:53.506 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.408848+0000", 2024-09-10T18:52:53.506 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.506 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.506 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.506 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:52:53.506 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.408848+0000", 2024-09-10T18:52:53.506 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T18:52:53.506 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.507 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.507 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:52:53.507 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.408852+0000", 2024-09-10T18:52:53.507 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.7699999999999999e-06 2024-09-10T18:52:53.507 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.507 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.507 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:52:53.507 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.408857+0000", 2024-09-10T18:52:53.507 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.4790000000000004e-06 2024-09-10T18:52:53.507 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.507 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.507 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:52:53.507 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.408861+0000", 2024-09-10T18:52:53.507 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.7309999999999999e-06 2024-09-10T18:52:53.508 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.508 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.508 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:52:53.508 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.409301+0000", 2024-09-10T18:52:53.508 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00043960499999999998 2024-09-10T18:52:53.508 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.508 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.508 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:52:53.508 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.409319+0000", 2024-09-10T18:52:53.508 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.8017999999999999e-05 2024-09-10T18:52:53.508 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.508 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.508 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:52:53.508 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.550822+0000", 2024-09-10T18:52:53.509 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.14150333800000001 2024-09-10T18:52:53.509 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.509 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.509 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:52:53.509 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.550844+0000", 2024-09-10T18:52:53.509 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.2348999999999999e-05 2024-09-10T18:52:53.509 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.509 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:52:53.509 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.509 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.509 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.509 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.14790.0:198295 7.a 7:5c08cab9:::benchmark_data_smithi018_78628_object62075:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-10T18:52:53.509 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:50:08.410651+0000", 2024-09-10T18:52:53.510 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 165.03672358, 2024-09-10T18:52:53.510 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.141414446, 2024-09-10T18:52:53.510 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:52:53.510 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:52:53.510 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:52:53.510 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.14790", 2024-09-10T18:52:53.510 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:52:53.510 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 198295 2024-09-10T18:52:53.510 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.510 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:52:53.510 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.510 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:52:53.510 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.410651+0000", 2024-09-10T18:52:53.510 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.511 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.511 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.511 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:52:53.511 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.410651+0000", 2024-09-10T18:52:53.511 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.511 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.511 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.511 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:52:53.511 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.410650+0000", 2024-09-10T18:52:53.511 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:52:53.511 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.511 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.511 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:52:53.511 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.410655+0000", 2024-09-10T18:52:53.512 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.374e-06 2024-09-10T18:52:53.512 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.512 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.512 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:52:53.512 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.410663+0000", 2024-09-10T18:52:53.512 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.6170000000000007e-06 2024-09-10T18:52:53.512 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.512 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.512 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:52:53.512 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.410668+0000", 2024-09-10T18:52:53.512 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.0690000000000001e-06 2024-09-10T18:52:53.512 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.512 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.512 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:52:53.513 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.542072+0000", 2024-09-10T18:52:53.513 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.13140318000000001 2024-09-10T18:52:53.513 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.513 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.513 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:52:53.513 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.542207+0000", 2024-09-10T18:52:53.513 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000135816 2024-09-10T18:52:53.513 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.513 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.513 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,5", 2024-09-10T18:52:53.513 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.542331+0000", 2024-09-10T18:52:53.513 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00012353700000000001 2024-09-10T18:52:53.513 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.514 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.514 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:52:53.514 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.550736+0000", 2024-09-10T18:52:53.514 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0084046180000000009 2024-09-10T18:52:53.514 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.514 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.514 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:52:53.514 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.551708+0000", 2024-09-10T18:52:53.514 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00097254000000000004 2024-09-10T18:52:53.514 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.514 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.514 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:52:53.514 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.551994+0000", 2024-09-10T18:52:53.514 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00028613199999999998 2024-09-10T18:52:53.515 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.515 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.515 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:52:53.515 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.552032+0000", 2024-09-10T18:52:53.515 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.7753999999999998e-05 2024-09-10T18:52:53.515 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.515 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.515 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:52:53.515 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.552066+0000", 2024-09-10T18:52:53.515 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.3612999999999998e-05 2024-09-10T18:52:53.515 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.515 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:52:53.515 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.515 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.516 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.516 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.14790.0:198908 7.2 7:46f200e0:::benchmark_data_smithi018_78628_object62688:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-10T18:52:53.516 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:50:08.998255+0000", 2024-09-10T18:52:53.516 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 164.449119661, 2024-09-10T18:52:53.516 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.17809398900000001, 2024-09-10T18:52:53.516 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:52:53.516 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:52:53.516 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:52:53.516 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.14790", 2024-09-10T18:52:53.516 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:52:53.516 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 198908 2024-09-10T18:52:53.516 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.516 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:52:53.517 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.517 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:52:53.517 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.998255+0000", 2024-09-10T18:52:53.517 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.517 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.517 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.517 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:52:53.517 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.998255+0000", 2024-09-10T18:52:53.517 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.517 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.517 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.517 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:52:53.517 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.998254+0000", 2024-09-10T18:52:53.518 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T18:52:53.518 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.518 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.518 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:52:53.518 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:08.998259+0000", 2024-09-10T18:52:53.518 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.8069999999999996e-06 2024-09-10T18:52:53.518 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.518 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.518 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:52:53.518 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:09.070652+0000", 2024-09-10T18:52:53.518 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.072392090000000006 2024-09-10T18:52:53.518 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.518 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.518 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:52:53.519 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:09.070664+0000", 2024-09-10T18:52:53.519 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.2342e-05 2024-09-10T18:52:53.519 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.519 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.519 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:52:53.519 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:09.072283+0000", 2024-09-10T18:52:53.519 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.001619487 2024-09-10T18:52:53.519 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.519 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.519 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:52:53.519 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:09.072381+0000", 2024-09-10T18:52:53.519 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.7181000000000003e-05 2024-09-10T18:52:53.519 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.519 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.520 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,7", 2024-09-10T18:52:53.520 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:09.072482+0000", 2024-09-10T18:52:53.520 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00010184 2024-09-10T18:52:53.520 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.520 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.520 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:52:53.520 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:09.077851+0000", 2024-09-10T18:52:53.520 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0053684919999999999 2024-09-10T18:52:53.520 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.520 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.520 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:52:53.520 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:09.081664+0000", 2024-09-10T18:52:53.520 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0038130550000000001 2024-09-10T18:52:53.521 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.521 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.521 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:52:53.521 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:09.176232+0000", 2024-09-10T18:52:53.521 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.094568236999999999 2024-09-10T18:52:53.521 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.521 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.521 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:52:53.521 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:09.176290+0000", 2024-09-10T18:52:53.521 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.8247e-05 2024-09-10T18:52:53.521 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.521 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.521 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:52:53.521 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:50:09.176349+0000", 2024-09-10T18:52:53.522 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.8717e-05 2024-09-10T18:52:53.522 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.522 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:52:53.522 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.522 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.522 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.522 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.24947.0:66011 8.f 8:f0264b5f:::benchmark_data_smithi018_106531_object66010:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e171)", 2024-09-10T18:52:53.522 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:52:18.961075+0000", 2024-09-10T18:52:53.522 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 34.486299795999997, 2024-09-10T18:52:53.522 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.152612634, 2024-09-10T18:52:53.522 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:52:53.522 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:52:53.522 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:52:53.523 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.24947", 2024-09-10T18:52:53.523 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:52:53.523 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 66011 2024-09-10T18:52:53.523 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.523 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:52:53.523 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.523 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:52:53.523 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.961075+0000", 2024-09-10T18:52:53.523 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.523 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.523 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.523 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:52:53.523 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.961075+0000", 2024-09-10T18:52:53.524 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.524 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.524 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.524 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:52:53.524 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.961073+0000", 2024-09-10T18:52:53.524 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999981 2024-09-10T18:52:53.524 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.524 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.524 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:52:53.524 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.961099+0000", 2024-09-10T18:52:53.524 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.5420999999999999e-05 2024-09-10T18:52:53.524 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.524 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.525 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:52:53.525 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.961111+0000", 2024-09-10T18:52:53.525 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.2221999999999999e-05 2024-09-10T18:52:53.525 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.525 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.525 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:52:53.525 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.961116+0000", 2024-09-10T18:52:53.525 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.5339999999999999e-06 2024-09-10T18:52:53.525 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.525 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.525 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:52:53.525 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.961169+0000", 2024-09-10T18:52:53.525 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.2778999999999997e-05 2024-09-10T18:52:53.526 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.526 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.526 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:52:53.526 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.961284+0000", 2024-09-10T18:52:53.526 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000114945 2024-09-10T18:52:53.526 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.526 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.526 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,4", 2024-09-10T18:52:53.526 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.961897+0000", 2024-09-10T18:52:53.526 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00061306700000000002 2024-09-10T18:52:53.526 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.526 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.526 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:52:53.527 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.966511+0000", 2024-09-10T18:52:53.527 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0046141350000000001 2024-09-10T18:52:53.527 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.527 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.527 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:52:53.527 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.967499+0000", 2024-09-10T18:52:53.527 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00098793699999999993 2024-09-10T18:52:53.527 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.527 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.527 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:52:53.527 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:53.529 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:19.113479+0000", 2024-09-10T18:52:53.529 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.14598009100000001 2024-09-10T18:52:53.529 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.529 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.529 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:52:53.529 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:19.113647+0000", 2024-09-10T18:52:53.529 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00016764199999999999 2024-09-10T18:52:53.529 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.529 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.529 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:52:53.529 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:19.113688+0000", 2024-09-10T18:52:53.529 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.0713999999999997e-05 2024-09-10T18:52:53.529 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.530 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:52:53.530 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.530 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.530 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.530 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.24947.0:66012 8.f 8:f2fb0e40:::benchmark_data_smithi018_106531_object66011:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e171)", 2024-09-10T18:52:53.530 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:52:18.961500+0000", 2024-09-10T18:52:53.530 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 34.485874344999999, 2024-09-10T18:52:53.530 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.15491201099999999, 2024-09-10T18:52:53.530 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:52:53.530 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:52:53.530 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:52:53.530 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.24947", 2024-09-10T18:52:53.530 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:52:53.531 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 66012 2024-09-10T18:52:53.531 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.531 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:52:53.531 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.531 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:52:53.531 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.961500+0000", 2024-09-10T18:52:53.531 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.531 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.531 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.531 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:52:53.531 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.961500+0000", 2024-09-10T18:52:53.531 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.531 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.531 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.532 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:52:53.532 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.961500+0000", 2024-09-10T18:52:53.532 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:52:53.532 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.532 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.532 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:52:53.532 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.961521+0000", 2024-09-10T18:52:53.532 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.1094000000000002e-05 2024-09-10T18:52:53.532 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.532 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.532 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:52:53.532 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.961530+0000", 2024-09-10T18:52:53.532 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.8149999999999995e-06 2024-09-10T18:52:53.533 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.533 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.533 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:52:53.533 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.961535+0000", 2024-09-10T18:52:53.533 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.0930000000000002e-06 2024-09-10T18:52:53.533 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.533 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.533 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:52:53.533 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.964492+0000", 2024-09-10T18:52:53.533 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0029572000000000001 2024-09-10T18:52:53.533 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.533 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.533 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:52:53.534 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.964601+0000", 2024-09-10T18:52:53.534 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000108719 2024-09-10T18:52:53.534 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.534 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.534 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,4", 2024-09-10T18:52:53.534 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.964786+0000", 2024-09-10T18:52:53.534 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000184928 2024-09-10T18:52:53.534 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.534 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.534 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:52:53.534 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.966624+0000", 2024-09-10T18:52:53.534 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.001838447 2024-09-10T18:52:53.534 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.534 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.535 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:52:53.535 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.967608+0000", 2024-09-10T18:52:53.535 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0009837540000000001 2024-09-10T18:52:53.535 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.535 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.535 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:52:53.535 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:19.116311+0000", 2024-09-10T18:52:53.535 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.14870287500000001 2024-09-10T18:52:53.535 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.535 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.535 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:52:53.535 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:19.116360+0000", 2024-09-10T18:52:53.535 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.9611000000000003e-05 2024-09-10T18:52:53.535 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.536 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.536 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:52:53.536 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:19.116412+0000", 2024-09-10T18:52:53.536 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.2255e-05 2024-09-10T18:52:53.536 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.536 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:52:53.536 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.536 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.536 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.536 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.24947.0:66018 8.b 8:d70f809e:::benchmark_data_smithi018_106531_object66017:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e171)", 2024-09-10T18:52:53.536 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:52:18.964419+0000", 2024-09-10T18:52:53.536 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 34.482956084000001, 2024-09-10T18:52:53.536 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.15031661700000001, 2024-09-10T18:52:53.536 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:52:53.537 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:52:53.537 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:52:53.537 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.24947", 2024-09-10T18:52:53.537 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:52:53.537 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 66018 2024-09-10T18:52:53.537 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.537 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:52:53.537 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.537 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:52:53.537 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.964419+0000", 2024-09-10T18:52:53.537 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.537 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.537 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.538 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:52:53.538 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.964419+0000", 2024-09-10T18:52:53.538 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.538 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.538 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.538 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:52:53.538 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.964417+0000", 2024-09-10T18:52:53.538 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T18:52:53.538 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.538 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.538 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:52:53.538 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.964440+0000", 2024-09-10T18:52:53.538 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.2563000000000001e-05 2024-09-10T18:52:53.539 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.539 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.539 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:52:53.539 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.964451+0000", 2024-09-10T18:52:53.539 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0511e-05 2024-09-10T18:52:53.539 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.539 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.539 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:52:53.539 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.964456+0000", 2024-09-10T18:52:53.539 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.66e-06 2024-09-10T18:52:53.539 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.539 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.539 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:52:53.539 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.964494+0000", 2024-09-10T18:52:53.539 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.8028999999999998e-05 2024-09-10T18:52:53.539 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.540 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.540 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:52:53.540 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.964584+0000", 2024-09-10T18:52:53.540 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.0103000000000001e-05 2024-09-10T18:52:53.540 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.540 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.540 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,4", 2024-09-10T18:52:53.540 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.964668+0000", 2024-09-10T18:52:53.540 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.3269000000000005e-05 2024-09-10T18:52:53.540 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.540 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.540 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:52:53.540 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.966437+0000", 2024-09-10T18:52:53.541 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0017695709999999999 2024-09-10T18:52:53.541 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.541 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.541 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:52:53.541 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.967475+0000", 2024-09-10T18:52:53.541 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.001038326 2024-09-10T18:52:53.541 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.541 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.541 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:52:53.541 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:19.114649+0000", 2024-09-10T18:52:53.541 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.147173995 2024-09-10T18:52:53.541 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.541 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.541 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:52:53.541 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:19.114699+0000", 2024-09-10T18:52:53.542 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.9870000000000002e-05 2024-09-10T18:52:53.542 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.542 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.542 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:52:53.542 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:19.114735+0000", 2024-09-10T18:52:53.542 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.5976999999999998e-05 2024-09-10T18:52:53.542 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.542 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:52:53.542 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.542 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.542 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.542 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.24947.0:66025 8.b 8:d14b3c0d:::benchmark_data_smithi018_106531_object66024:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e171)", 2024-09-10T18:52:53.542 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:52:18.970327+0000", 2024-09-10T18:52:53.542 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 34.477047599000002, 2024-09-10T18:52:53.543 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.14454103300000001, 2024-09-10T18:52:53.543 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:52:53.543 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:52:53.543 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:52:53.543 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.24947", 2024-09-10T18:52:53.543 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:52:53.543 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 66025 2024-09-10T18:52:53.543 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.543 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:52:53.543 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.543 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:52:53.543 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.970327+0000", 2024-09-10T18:52:53.544 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.544 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.544 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.544 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:52:53.544 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.970327+0000", 2024-09-10T18:52:53.544 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.544 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.544 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.544 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:52:53.544 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.970326+0000", 2024-09-10T18:52:53.544 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T18:52:53.544 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.544 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.545 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:52:53.545 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.970347+0000", 2024-09-10T18:52:53.545 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.1552e-05 2024-09-10T18:52:53.545 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.545 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.545 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:52:53.545 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.970358+0000", 2024-09-10T18:52:53.545 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0511e-05 2024-09-10T18:52:53.545 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.545 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.545 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:52:53.545 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.970362+0000", 2024-09-10T18:52:53.545 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.4050000000000002e-06 2024-09-10T18:52:53.545 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.546 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.546 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:52:53.546 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.970402+0000", 2024-09-10T18:52:53.546 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.9721000000000001e-05 2024-09-10T18:52:53.546 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.546 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.546 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:52:53.546 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.970510+0000", 2024-09-10T18:52:53.546 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000107587 2024-09-10T18:52:53.546 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.546 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.546 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,4", 2024-09-10T18:52:53.546 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.970607+0000", 2024-09-10T18:52:53.546 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.747e-05 2024-09-10T18:52:53.546 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.547 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.547 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:52:53.547 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.972399+0000", 2024-09-10T18:52:53.547 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0017920150000000001 2024-09-10T18:52:53.547 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.547 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.547 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:52:53.547 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:19.114772+0000", 2024-09-10T18:52:53.547 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.14237293400000001 2024-09-10T18:52:53.547 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.547 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.547 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:52:53.547 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:19.114798+0000", 2024-09-10T18:52:53.548 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.6529e-05 2024-09-10T18:52:53.548 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.548 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.548 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:52:53.548 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:19.114836+0000", 2024-09-10T18:52:53.548 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.8002e-05 2024-09-10T18:52:53.548 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.548 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.548 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:52:53.548 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:19.114868+0000", 2024-09-10T18:52:53.548 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.1776999999999997e-05 2024-09-10T18:52:53.548 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.548 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:52:53.549 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.549 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.549 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.549 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.24947.0:66028 8.b 8:d887f2c0:::benchmark_data_smithi018_106531_object66027:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e171)", 2024-09-10T18:52:53.549 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:52:18.973345+0000", 2024-09-10T18:52:53.549 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 34.474030296999999, 2024-09-10T18:52:53.549 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.14970175399999999, 2024-09-10T18:52:53.549 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:52:53.549 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:52:53.549 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:52:53.549 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.24947", 2024-09-10T18:52:53.549 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:52:53.549 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 66028 2024-09-10T18:52:53.550 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.550 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:52:53.550 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.550 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:52:53.550 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.973345+0000", 2024-09-10T18:52:53.550 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.550 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.550 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.550 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:52:53.550 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.973345+0000", 2024-09-10T18:52:53.550 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.550 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.550 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.550 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:52:53.551 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.973343+0000", 2024-09-10T18:52:53.551 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:52:53.551 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.551 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.551 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:52:53.551 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.973372+0000", 2024-09-10T18:52:53.551 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.8946999999999999e-05 2024-09-10T18:52:53.551 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.551 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.551 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:52:53.551 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.973394+0000", 2024-09-10T18:52:53.551 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.1518999999999998e-05 2024-09-10T18:52:53.551 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.552 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.552 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:52:53.552 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.973400+0000", 2024-09-10T18:52:53.552 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.0209999999999996e-06 2024-09-10T18:52:53.552 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.552 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.552 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:52:53.552 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.973438+0000", 2024-09-10T18:52:53.552 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.7679999999999998e-05 2024-09-10T18:52:53.552 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.552 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.552 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:52:53.552 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.973507+0000", 2024-09-10T18:52:53.553 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.9511000000000006e-05 2024-09-10T18:52:53.553 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.553 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.553 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,4", 2024-09-10T18:52:53.553 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:19.111106+0000", 2024-09-10T18:52:53.553 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.137598729 2024-09-10T18:52:53.553 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.553 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.553 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:52:53.553 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:19.114904+0000", 2024-09-10T18:52:53.553 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0037978640000000002 2024-09-10T18:52:53.553 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.553 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.554 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:52:53.554 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:19.121246+0000", 2024-09-10T18:52:53.554 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0063419119999999999 2024-09-10T18:52:53.554 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.554 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.554 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:52:53.554 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:19.122945+0000", 2024-09-10T18:52:53.554 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.001699386 2024-09-10T18:52:53.554 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.554 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.554 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:52:53.554 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:19.123007+0000", 2024-09-10T18:52:53.554 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.2329000000000006e-05 2024-09-10T18:52:53.555 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.555 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.555 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:52:53.555 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:19.123046+0000", 2024-09-10T18:52:53.555 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.8909000000000003e-05 2024-09-10T18:52:53.555 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.555 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:52:53.555 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.555 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.555 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.555 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.24947.0:66030 8.f 8:f78c112f:::benchmark_data_smithi018_106531_object66029:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e171)", 2024-09-10T18:52:53.555 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:52:18.973618+0000", 2024-09-10T18:52:53.555 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 34.473757149999997, 2024-09-10T18:52:53.555 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.14788314499999999, 2024-09-10T18:52:53.556 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:52:53.556 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:52:53.556 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:52:53.556 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.24947", 2024-09-10T18:52:53.556 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:52:53.556 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 66030 2024-09-10T18:52:53.556 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.556 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:52:53.556 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.556 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:52:53.556 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.973618+0000", 2024-09-10T18:52:53.556 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.556 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.557 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.557 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:52:53.557 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.973618+0000", 2024-09-10T18:52:53.557 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.557 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.557 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.557 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:52:53.557 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.973617+0000", 2024-09-10T18:52:53.557 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:52:53.557 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.557 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.557 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:52:53.557 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.973638+0000", 2024-09-10T18:52:53.558 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.1078999999999999e-05 2024-09-10T18:52:53.558 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.558 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.558 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:52:53.558 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:19.110171+0000", 2024-09-10T18:52:53.558 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.13653354700000001 2024-09-10T18:52:53.558 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.558 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.558 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:52:53.558 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:19.110190+0000", 2024-09-10T18:52:53.558 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.8113000000000001e-05 2024-09-10T18:52:53.558 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.558 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.558 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:52:53.559 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:19.110276+0000", 2024-09-10T18:52:53.559 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.6546999999999997e-05 2024-09-10T18:52:53.559 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.559 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.559 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:52:53.559 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:19.110416+0000", 2024-09-10T18:52:53.559 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000140319 2024-09-10T18:52:53.559 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.559 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.559 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,4", 2024-09-10T18:52:53.559 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:19.110678+0000", 2024-09-10T18:52:53.559 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00026151100000000002 2024-09-10T18:52:53.559 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.559 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.559 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:52:53.560 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:19.117849+0000", 2024-09-10T18:52:53.560 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0071714810000000004 2024-09-10T18:52:53.560 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.560 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.560 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:52:53.560 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:19.118762+0000", 2024-09-10T18:52:53.560 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00091277800000000005 2024-09-10T18:52:53.560 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.560 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.560 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:52:53.560 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:19.121413+0000", 2024-09-10T18:52:53.560 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0026504609999999998 2024-09-10T18:52:53.560 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.561 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.561 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:52:53.561 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:19.121465+0000", 2024-09-10T18:52:53.561 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.2673000000000002e-05 2024-09-10T18:52:53.561 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.561 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.561 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:52:53.561 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:19.121501+0000", 2024-09-10T18:52:53.561 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.5439999999999999e-05 2024-09-10T18:52:53.561 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.561 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:52:53.561 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.561 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.562 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.562 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop_reply(client.24947.0:66025 8.b e171/158 ondisk, result = 0)", 2024-09-10T18:52:53.562 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:52:18.974438+0000", 2024-09-10T18:52:53.562 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 34.472936492999999, 2024-09-10T18:52:53.562 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.14033485000000001, 2024-09-10T18:52:53.562 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:52:53.562 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "started", 2024-09-10T18:52:53.562 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:52:53.562 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.562 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:52:53.562 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.974438+0000", 2024-09-10T18:52:53.562 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.562 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.562 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.563 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:52:53.563 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.974438+0000", 2024-09-10T18:52:53.563 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.563 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.563 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.563 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:52:53.563 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.974438+0000", 2024-09-10T18:52:53.563 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T18:52:53.563 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.563 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.563 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:52:53.563 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:18.974441+0000", 2024-09-10T18:52:53.563 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.0029999999999999e-06 2024-09-10T18:52:53.563 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.564 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.564 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:52:53.564 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:19.110334+0000", 2024-09-10T18:52:53.564 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.13589306200000001 2024-09-10T18:52:53.564 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.564 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.564 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:52:53.564 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:19.110339+0000", 2024-09-10T18:52:53.564 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.4739999999999996e-06 2024-09-10T18:52:53.564 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.564 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.564 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:52:53.564 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:19.114762+0000", 2024-09-10T18:52:53.564 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.004422484 2024-09-10T18:52:53.565 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.565 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.565 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:52:53.565 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:19.114767+0000", 2024-09-10T18:52:53.565 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.2079999999999999e-06 2024-09-10T18:52:53.565 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.565 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.565 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:52:53.565 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:19.114773+0000", 2024-09-10T18:52:53.565 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.1940000000000003e-06 2024-09-10T18:52:53.565 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.565 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:52:53.565 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.566 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.566 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.566 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.24947.0:114359 8.0 8:063bec2d:::benchmark_data_smithi018_106531_object114358:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e182)", 2024-09-10T18:52:53.566 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:52:52.269356+0000", 2024-09-10T18:52:53.566 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 1.1780187719999999, 2024-09-10T18:52:53.566 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.186539963, 2024-09-10T18:52:53.566 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:52:53.566 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:52:53.566 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:52:53.566 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.24947", 2024-09-10T18:52:53.566 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:52:53.566 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 114359 2024-09-10T18:52:53.566 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.566 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:52:53.567 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.567 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:52:53.567 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:52.269356+0000", 2024-09-10T18:52:53.567 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.567 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.567 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.567 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:52:53.567 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:52.269356+0000", 2024-09-10T18:52:53.567 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:52:53.567 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.567 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.567 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:52:53.567 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:52.269355+0000", 2024-09-10T18:52:53.568 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:52:53.568 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.568 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.568 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:52:53.568 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:52.269373+0000", 2024-09-10T18:52:53.568 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.7713000000000001e-05 2024-09-10T18:52:53.568 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.568 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.568 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:52:53.568 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:52.269383+0000", 2024-09-10T18:52:53.568 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0346999999999999e-05 2024-09-10T18:52:53.568 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.568 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.569 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:52:53.569 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:52.269387+0000", 2024-09-10T18:52:53.569 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.9380000000000002e-06 2024-09-10T18:52:53.569 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.569 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.569 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:52:53.569 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:52.269426+0000", 2024-09-10T18:52:53.569 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.8924000000000002e-05 2024-09-10T18:52:53.569 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.569 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.569 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:52:53.569 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:52.269552+0000", 2024-09-10T18:52:53.569 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00012583599999999999 2024-09-10T18:52:53.569 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.570 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.570 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,5", 2024-09-10T18:52:53.570 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:52.269721+0000", 2024-09-10T18:52:53.570 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00016967100000000001 2024-09-10T18:52:53.570 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.570 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.570 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:52:53.570 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:52.272122+0000", 2024-09-10T18:52:53.570 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0024001019999999999 2024-09-10T18:52:53.570 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.570 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.570 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:52:53.570 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:52.272295+0000", 2024-09-10T18:52:53.570 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00017317999999999999 2024-09-10T18:52:53.570 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.571 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.571 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:52:53.571 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:52.455799+0000", 2024-09-10T18:52:53.571 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.183504418 2024-09-10T18:52:53.571 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.571 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.571 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:52:53.571 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:52.455853+0000", 2024-09-10T18:52:53.571 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.3906999999999997e-05 2024-09-10T18:52:53.571 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:52:53.571 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:52:53.571 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:52:53.571 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:52.455896+0000", 2024-09-10T18:52:53.571 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.2969000000000001e-05 2024-09-10T18:52:53.571 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.571 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:52:53.572 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.572 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:52:53.572 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:52:53.572 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T18:52:53.577 INFO:teuthology.orchestra.run.smithi018.stdout:moved 15 / 504 (2.97619%) 2024-09-10T18:52:53.577 INFO:teuthology.orchestra.run.smithi018.stdout:avg 72 2024-09-10T18:52:53.578 INFO:teuthology.orchestra.run.smithi018.stdout:stddev 3.0706 -> 6.09449 (expected baseline 7.85584) 2024-09-10T18:52:53.578 INFO:teuthology.orchestra.run.smithi018.stdout:min osd.0 with 67 -> 69 pgs (0.930556 -> 0.958333 * mean) 2024-09-10T18:52:53.578 INFO:teuthology.orchestra.run.smithi018.stdout:max osd.3 with 76 -> 77 pgs (1.05556 -> 1.06944 * mean) 2024-09-10T18:52:53.578 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:52:53.578 INFO:teuthology.orchestra.run.smithi018.stdout:oload 110 2024-09-10T18:52:53.578 INFO:teuthology.orchestra.run.smithi018.stdout:max_change 1 2024-09-10T18:52:53.578 INFO:teuthology.orchestra.run.smithi018.stdout:max_change_osds 4 2024-09-10T18:52:53.578 INFO:teuthology.orchestra.run.smithi018.stdout:average_utilization 0.0349 2024-09-10T18:52:53.578 INFO:teuthology.orchestra.run.smithi018.stdout:overload_utilization 0.0384 2024-09-10T18:52:53.578 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 weight 1.0000 -> 0.7495 2024-09-10T18:52:53.622 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:53.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:53.624 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:53.725 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:53.748 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:53.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:53.749 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:53.820 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:53 smithi018 ceph-mon[29336]: 5.10 scrub starts 2024-09-10T18:52:53.820 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:53 smithi018 ceph-mon[29336]: 5.10 scrub ok 2024-09-10T18:52:53.820 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:53 smithi018 ceph-mon[29336]: 2.18 scrub starts 2024-09-10T18:52:53.820 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:53 smithi018 ceph-mon[29336]: 2.18 scrub ok 2024-09-10T18:52:53.820 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:53 smithi018 ceph-mon[29336]: 5.1e scrub starts 2024-09-10T18:52:53.821 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:53 smithi018 ceph-mon[29336]: 5.1e scrub ok 2024-09-10T18:52:53.821 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:53 smithi018 ceph-mon[29336]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 1.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:52:53.821 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:53 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd reweightn", "weights": "{'5':'49119'}"}]: dispatch 2024-09-10T18:52:53.821 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:53 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd reweightn", "weights": "{'5':'49119'}"}]: dispatch 2024-09-10T18:52:53.821 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:53 smithi018 ceph-mon[29336]: pgmap v324: 168 pgs: 168 active+clean; 6.9 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 853 B/s rd, 77 MiB/s wr, 1.23k op/s 2024-09-10T18:52:53.821 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:53 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd reweightn", "weights": "{'5':'49119'}"}]': finished 2024-09-10T18:52:53.821 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:53 smithi018 ceph-mon[29336]: osdmap e183: 8 total, 8 up, 7 in 2024-09-10T18:52:53.821 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:53 smithi018 ceph-mon[40196]: 5.10 scrub starts 2024-09-10T18:52:53.821 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:53 smithi018 ceph-mon[40196]: 5.10 scrub ok 2024-09-10T18:52:53.821 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:53 smithi018 ceph-mon[40196]: 2.18 scrub starts 2024-09-10T18:52:53.821 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:53 smithi018 ceph-mon[40196]: 2.18 scrub ok 2024-09-10T18:52:53.822 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:53 smithi018 ceph-mon[40196]: 5.1e scrub starts 2024-09-10T18:52:53.822 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:53 smithi018 ceph-mon[40196]: 5.1e scrub ok 2024-09-10T18:52:53.822 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:53 smithi018 ceph-mon[40196]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 1.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:52:53.822 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:53 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd reweightn", "weights": "{'5':'49119'}"}]: dispatch 2024-09-10T18:52:53.822 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:53 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd reweightn", "weights": "{'5':'49119'}"}]: dispatch 2024-09-10T18:52:53.822 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:53 smithi018 ceph-mon[40196]: pgmap v324: 168 pgs: 168 active+clean; 6.9 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 853 B/s rd, 77 MiB/s wr, 1.23k op/s 2024-09-10T18:52:53.822 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:53 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd reweightn", "weights": "{'5':'49119'}"}]': finished 2024-09-10T18:52:53.822 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:53 smithi018 ceph-mon[40196]: osdmap e183: 8 total, 8 up, 7 in 2024-09-10T18:52:53.850 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:53.871 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:53.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:53.872 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:53.973 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:53.996 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:53.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:53.997 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:54.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:53 smithi103 ceph-mon[33163]: 5.10 scrub starts 2024-09-10T18:52:54.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:53 smithi103 ceph-mon[33163]: 5.10 scrub ok 2024-09-10T18:52:54.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:53 smithi103 ceph-mon[33163]: 2.18 scrub starts 2024-09-10T18:52:54.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:53 smithi103 ceph-mon[33163]: 2.18 scrub ok 2024-09-10T18:52:54.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:53 smithi103 ceph-mon[33163]: 5.1e scrub starts 2024-09-10T18:52:54.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:53 smithi103 ceph-mon[33163]: 5.1e scrub ok 2024-09-10T18:52:54.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:53 smithi103 ceph-mon[33163]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 1.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:52:54.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:53 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd reweightn", "weights": "{'5':'49119'}"}]: dispatch 2024-09-10T18:52:54.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:53 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd reweightn", "weights": "{'5':'49119'}"}]: dispatch 2024-09-10T18:52:54.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:53 smithi103 ceph-mon[33163]: pgmap v324: 168 pgs: 168 active+clean; 6.9 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 853 B/s rd, 77 MiB/s wr, 1.23k op/s 2024-09-10T18:52:54.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:53 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd reweightn", "weights": "{'5':'49119'}"}]': finished 2024-09-10T18:52:54.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:53 smithi103 ceph-mon[33163]: osdmap e183: 8 total, 8 up, 7 in 2024-09-10T18:52:54.098 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:54.121 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:54.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:54.122 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:54.204 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 65 16 116059 116043 111.559 41.625 0.00303047 0.00888142 2024-09-10T18:52:54.222 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:54.245 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:54.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:54.246 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:54.347 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:54.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:54.371 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:54.372 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:54.473 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:54.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:54.495 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:54.496 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:54.596 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:54.628 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:54.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:54.629 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:54.730 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:54.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:54.757 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:54.757 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:54.858 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:54.882 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:54.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:54.883 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:54.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:54 smithi018 ceph-mon[29336]: 1.11 scrub starts 2024-09-10T18:52:54.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:54 smithi018 ceph-mon[29336]: 1.11 scrub ok 2024-09-10T18:52:54.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:54 smithi018 ceph-mon[29336]: 1.f deep-scrub starts 2024-09-10T18:52:54.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:54 smithi018 ceph-mon[29336]: 1.f deep-scrub ok 2024-09-10T18:52:54.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:54 smithi018 ceph-mon[29336]: 1.5 scrub starts 2024-09-10T18:52:54.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:54 smithi018 ceph-mon[29336]: 1.5 scrub ok 2024-09-10T18:52:54.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:54 smithi018 ceph-mon[29336]: osdmap e184: 8 total, 8 up, 7 in 2024-09-10T18:52:54.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:54 smithi018 ceph-mon[40196]: 1.11 scrub starts 2024-09-10T18:52:54.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:54 smithi018 ceph-mon[40196]: 1.11 scrub ok 2024-09-10T18:52:54.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:54 smithi018 ceph-mon[40196]: 1.f deep-scrub starts 2024-09-10T18:52:54.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:54 smithi018 ceph-mon[40196]: 1.f deep-scrub ok 2024-09-10T18:52:54.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:54 smithi018 ceph-mon[40196]: 1.5 scrub starts 2024-09-10T18:52:54.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:54 smithi018 ceph-mon[40196]: 1.5 scrub ok 2024-09-10T18:52:54.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:54 smithi018 ceph-mon[40196]: osdmap e184: 8 total, 8 up, 7 in 2024-09-10T18:52:54.984 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:55.008 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:55.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:55.009 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:55.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:54 smithi103 ceph-mon[33163]: 1.11 scrub starts 2024-09-10T18:52:55.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:54 smithi103 ceph-mon[33163]: 1.11 scrub ok 2024-09-10T18:52:55.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:54 smithi103 ceph-mon[33163]: 1.f deep-scrub starts 2024-09-10T18:52:55.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:54 smithi103 ceph-mon[33163]: 1.f deep-scrub ok 2024-09-10T18:52:55.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:54 smithi103 ceph-mon[33163]: 1.5 scrub starts 2024-09-10T18:52:55.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:54 smithi103 ceph-mon[33163]: 1.5 scrub ok 2024-09-10T18:52:55.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:54 smithi103 ceph-mon[33163]: osdmap e184: 8 total, 8 up, 7 in 2024-09-10T18:52:55.110 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:55.134 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:55.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:55.135 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:55.204 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 66 16 116059 116043 109.869 0 - 0.00888142 2024-09-10T18:52:55.220 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-10T18:52:55.235 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:55.258 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:55.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:55.260 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:55.360 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:55.384 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:55.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:55.385 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:55.486 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:55.510 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:55.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:55.511 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:55.612 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:55.635 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:55.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:55.636 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:55.736 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:55.771 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:55.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:55.772 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:55.873 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:55.911 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:55.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:55.912 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:55.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:55 smithi018 ceph-mon[40196]: 1.2 scrub starts 2024-09-10T18:52:55.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:55 smithi018 ceph-mon[40196]: 1.2 scrub ok 2024-09-10T18:52:55.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:55 smithi018 ceph-mon[40196]: 1.12 deep-scrub starts 2024-09-10T18:52:55.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:55 smithi018 ceph-mon[40196]: 1.12 deep-scrub ok 2024-09-10T18:52:55.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:55 smithi018 ceph-mon[40196]: 5.5 scrub starts 2024-09-10T18:52:55.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:55 smithi018 ceph-mon[40196]: 4.d scrub starts 2024-09-10T18:52:55.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:55 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:52:55.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:52:55.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:55 smithi018 ceph-mon[40196]: 4.d scrub ok 2024-09-10T18:52:55.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:55 smithi018 ceph-mon[40196]: pgmap v327: 168 pgs: 168 active+clean; 6.9 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 853 B/s rd, 77 MiB/s wr, 1.23k op/s 2024-09-10T18:52:55.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:55 smithi018 ceph-mon[40196]: osdmap e185: 8 total, 8 up, 7 in 2024-09-10T18:52:55.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:55 smithi018 ceph-mon[29336]: 1.2 scrub starts 2024-09-10T18:52:55.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:55 smithi018 ceph-mon[29336]: 1.2 scrub ok 2024-09-10T18:52:55.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:55 smithi018 ceph-mon[29336]: 1.12 deep-scrub starts 2024-09-10T18:52:55.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:55 smithi018 ceph-mon[29336]: 1.12 deep-scrub ok 2024-09-10T18:52:55.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:55 smithi018 ceph-mon[29336]: 5.5 scrub starts 2024-09-10T18:52:55.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:55 smithi018 ceph-mon[29336]: 4.d scrub starts 2024-09-10T18:52:55.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:55 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:52:55.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:52:55.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:55 smithi018 ceph-mon[29336]: 4.d scrub ok 2024-09-10T18:52:55.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:55 smithi018 ceph-mon[29336]: pgmap v327: 168 pgs: 168 active+clean; 6.9 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 853 B/s rd, 77 MiB/s wr, 1.23k op/s 2024-09-10T18:52:55.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:55 smithi018 ceph-mon[29336]: osdmap e185: 8 total, 8 up, 7 in 2024-09-10T18:52:56.013 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:56.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:55 smithi103 ceph-mon[33163]: 1.2 scrub starts 2024-09-10T18:52:56.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:55 smithi103 ceph-mon[33163]: 1.2 scrub ok 2024-09-10T18:52:56.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:55 smithi103 ceph-mon[33163]: 1.12 deep-scrub starts 2024-09-10T18:52:56.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:55 smithi103 ceph-mon[33163]: 1.12 deep-scrub ok 2024-09-10T18:52:56.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:55 smithi103 ceph-mon[33163]: 5.5 scrub starts 2024-09-10T18:52:56.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:55 smithi103 ceph-mon[33163]: 4.d scrub starts 2024-09-10T18:52:56.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:55 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:52:56.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:52:56.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:55 smithi103 ceph-mon[33163]: 4.d scrub ok 2024-09-10T18:52:56.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:55 smithi103 ceph-mon[33163]: pgmap v327: 168 pgs: 168 active+clean; 6.9 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 853 B/s rd, 77 MiB/s wr, 1.23k op/s 2024-09-10T18:52:56.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:55 smithi103 ceph-mon[33163]: osdmap e185: 8 total, 8 up, 7 in 2024-09-10T18:52:56.049 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:56.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:56.050 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:56.151 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:56.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:56.186 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:56.187 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:56.204 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 67 16 116851 116835 108.968 24.75 0.00832881 0.00917253 2024-09-10T18:52:56.288 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:56.319 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:56.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:56.320 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:56.421 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:56.458 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:56.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:56.459 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:56.560 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:52:56.598 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:56.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:56.599 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:52:56.700 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:56.736 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:56.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:56.738 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:56.772 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:52:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:52:56.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:56 smithi103 ceph-mon[33163]: 6.0 scrub starts 2024-09-10T18:52:56.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:56 smithi103 ceph-mon[33163]: 6.0 scrub ok 2024-09-10T18:52:56.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:56 smithi103 ceph-mon[33163]: 8.9 starting backfill to osd.3 from (0'0,0'0] MAX to 182'7234 2024-09-10T18:52:56.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:56 smithi103 ceph-mon[33163]: 8.a starting backfill to osd.4 from (0'0,0'0] MAX to 182'7239 2024-09-10T18:52:56.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:56 smithi103 ceph-mon[33163]: 1.e scrub starts 2024-09-10T18:52:56.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:56 smithi103 ceph-mon[33163]: 1.e scrub ok 2024-09-10T18:52:56.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:56 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.11", "id": [4, 3]}]: dispatch 2024-09-10T18:52:56.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:56 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.11", "id": [4, 3]}]: dispatch 2024-09-10T18:52:56.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:56 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.19", "id": [1, 6]}]: dispatch 2024-09-10T18:52:56.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:56 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.19", "id": [1, 6]}]: dispatch 2024-09-10T18:52:56.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:56 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:56.839 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:56.875 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:56.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:56.876 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:56.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:56 smithi018 ceph-mon[40196]: 6.0 scrub starts 2024-09-10T18:52:56.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:56 smithi018 ceph-mon[40196]: 6.0 scrub ok 2024-09-10T18:52:56.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:56 smithi018 ceph-mon[40196]: 8.9 starting backfill to osd.3 from (0'0,0'0] MAX to 182'7234 2024-09-10T18:52:56.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:56 smithi018 ceph-mon[40196]: 8.a starting backfill to osd.4 from (0'0,0'0] MAX to 182'7239 2024-09-10T18:52:56.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:56 smithi018 ceph-mon[40196]: 1.e scrub starts 2024-09-10T18:52:56.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:56 smithi018 ceph-mon[40196]: 1.e scrub ok 2024-09-10T18:52:56.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:56 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.11", "id": [4, 3]}]: dispatch 2024-09-10T18:52:56.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:56 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.11", "id": [4, 3]}]: dispatch 2024-09-10T18:52:56.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:56 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.19", "id": [1, 6]}]: dispatch 2024-09-10T18:52:56.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:56 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.19", "id": [1, 6]}]: dispatch 2024-09-10T18:52:56.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:56 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:56.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:56 smithi018 ceph-mon[29336]: 6.0 scrub starts 2024-09-10T18:52:56.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:56 smithi018 ceph-mon[29336]: 6.0 scrub ok 2024-09-10T18:52:56.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:56 smithi018 ceph-mon[29336]: 8.9 starting backfill to osd.3 from (0'0,0'0] MAX to 182'7234 2024-09-10T18:52:56.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:56 smithi018 ceph-mon[29336]: 8.a starting backfill to osd.4 from (0'0,0'0] MAX to 182'7239 2024-09-10T18:52:56.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:56 smithi018 ceph-mon[29336]: 1.e scrub starts 2024-09-10T18:52:56.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:56 smithi018 ceph-mon[29336]: 1.e scrub ok 2024-09-10T18:52:56.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:56 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.11", "id": [4, 3]}]: dispatch 2024-09-10T18:52:56.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:56 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.11", "id": [4, 3]}]: dispatch 2024-09-10T18:52:56.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:56 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.19", "id": [1, 6]}]: dispatch 2024-09-10T18:52:56.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:56 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.19", "id": [1, 6]}]: dispatch 2024-09-10T18:52:56.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:56 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:56.977 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:57.011 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:57.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:57.012 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:57.113 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:57.135 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:57.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:57.136 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:57.204 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 68 16 118228 118212 108.631 86.0625 0.118463 0.00920026 2024-09-10T18:52:57.237 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:57.274 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:57.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:57.275 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:57.376 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:57.413 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:57.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:57.415 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:57.516 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:57.557 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:57.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:57.558 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:57.659 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:57.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:57.691 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:57.691 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:57.792 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:57.839 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:57.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:57.840 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:57.941 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:57.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:57 smithi018 ceph-mon[29336]: 6.1b scrub starts 2024-09-10T18:52:57.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:57 smithi018 ceph-mon[29336]: 6.1b scrub ok 2024-09-10T18:52:57.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:57 smithi018 ceph-mon[29336]: 1.0 deep-scrub starts 2024-09-10T18:52:57.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:57 smithi018 ceph-mon[29336]: 1.0 deep-scrub ok 2024-09-10T18:52:57.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:57 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.11", "id": [4, 3]}]': finished 2024-09-10T18:52:57.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:57 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.19", "id": [1, 6]}]': finished 2024-09-10T18:52:57.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:57 smithi018 ceph-mon[29336]: osdmap e186: 8 total, 8 up, 7 in 2024-09-10T18:52:57.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:57 smithi018 ceph-mon[29336]: pgmap v330: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 7.1 GiB data, 23 GiB used, 602 GiB / 626 GiB avail; 64 MiB/s wr, 1.03k op/s; 14588/351774 objects misplaced (4.147%); 51 KiB/s, 3 objects/s recovering 2024-09-10T18:52:57.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:57 smithi018 ceph-mon[40196]: 6.1b scrub starts 2024-09-10T18:52:57.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:57 smithi018 ceph-mon[40196]: 6.1b scrub ok 2024-09-10T18:52:57.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:57 smithi018 ceph-mon[40196]: 1.0 deep-scrub starts 2024-09-10T18:52:57.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:57 smithi018 ceph-mon[40196]: 1.0 deep-scrub ok 2024-09-10T18:52:57.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:57 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.11", "id": [4, 3]}]': finished 2024-09-10T18:52:57.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:57 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.19", "id": [1, 6]}]': finished 2024-09-10T18:52:57.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:57 smithi018 ceph-mon[40196]: osdmap e186: 8 total, 8 up, 7 in 2024-09-10T18:52:57.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:57 smithi018 ceph-mon[40196]: pgmap v330: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 7.1 GiB data, 23 GiB used, 602 GiB / 626 GiB avail; 64 MiB/s wr, 1.03k op/s; 14588/351774 objects misplaced (4.147%); 51 KiB/s, 3 objects/s recovering 2024-09-10T18:52:57.978 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:57.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:57.979 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:58.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:57 smithi103 ceph-mon[33163]: 6.1b scrub starts 2024-09-10T18:52:58.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:57 smithi103 ceph-mon[33163]: 6.1b scrub ok 2024-09-10T18:52:58.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:57 smithi103 ceph-mon[33163]: 1.0 deep-scrub starts 2024-09-10T18:52:58.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:57 smithi103 ceph-mon[33163]: 1.0 deep-scrub ok 2024-09-10T18:52:58.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:57 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.11", "id": [4, 3]}]': finished 2024-09-10T18:52:58.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:57 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.19", "id": [1, 6]}]': finished 2024-09-10T18:52:58.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:57 smithi103 ceph-mon[33163]: osdmap e186: 8 total, 8 up, 7 in 2024-09-10T18:52:58.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:57 smithi103 ceph-mon[33163]: pgmap v330: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 7.1 GiB data, 23 GiB used, 602 GiB / 626 GiB avail; 64 MiB/s wr, 1.03k op/s; 14588/351774 objects misplaced (4.147%); 51 KiB/s, 3 objects/s recovering 2024-09-10T18:52:58.080 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:52:58.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:58.126 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:58.126 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:52:58.204 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 69 16 119669 119653 108.362 90.0625 0.00906974 0.00922382 2024-09-10T18:52:58.228 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:58.263 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:58.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:58.264 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:58.365 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:58.401 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:58.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:58.402 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:58.502 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:58.547 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:58.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:58.548 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:58.649 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:58.677 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:58.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:58.678 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:58.779 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:58.812 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:58.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:58.813 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:58.914 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:52:58.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:58 smithi018 ceph-mon[40196]: 4.10 scrub starts 2024-09-10T18:52:58.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:58 smithi018 ceph-mon[40196]: 4.10 scrub ok 2024-09-10T18:52:58.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:58 smithi018 ceph-mon[40196]: 1.14 deep-scrub starts 2024-09-10T18:52:58.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:58 smithi018 ceph-mon[40196]: 1.10 scrub starts 2024-09-10T18:52:58.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:58 smithi018 ceph-mon[40196]: 1.10 scrub ok 2024-09-10T18:52:58.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:58 smithi018 ceph-mon[40196]: 1.14 deep-scrub ok 2024-09-10T18:52:58.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:58 smithi018 ceph-mon[40196]: 6.16 deep-scrub starts 2024-09-10T18:52:58.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:58 smithi018 ceph-mon[40196]: 6.16 deep-scrub ok 2024-09-10T18:52:58.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:58 smithi018 ceph-mon[40196]: 8.5 scrub starts 2024-09-10T18:52:58.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:58 smithi018 ceph-mon[40196]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-10T18:52:58.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:58 smithi018 ceph-mon[40196]: 5.18 scrub starts 2024-09-10T18:52:58.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:58 smithi018 ceph-mon[40196]: 5.18 scrub ok 2024-09-10T18:52:58.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:58 smithi018 ceph-mon[40196]: osdmap e187: 8 total, 8 up, 7 in 2024-09-10T18:52:58.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:58 smithi018 ceph-mon[29336]: 4.10 scrub starts 2024-09-10T18:52:58.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:58 smithi018 ceph-mon[29336]: 4.10 scrub ok 2024-09-10T18:52:58.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:58 smithi018 ceph-mon[29336]: 1.14 deep-scrub starts 2024-09-10T18:52:58.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:58 smithi018 ceph-mon[29336]: 1.10 scrub starts 2024-09-10T18:52:58.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:58 smithi018 ceph-mon[29336]: 1.10 scrub ok 2024-09-10T18:52:58.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:58 smithi018 ceph-mon[29336]: 1.14 deep-scrub ok 2024-09-10T18:52:58.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:58 smithi018 ceph-mon[29336]: 6.16 deep-scrub starts 2024-09-10T18:52:58.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:58 smithi018 ceph-mon[29336]: 6.16 deep-scrub ok 2024-09-10T18:52:58.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:58 smithi018 ceph-mon[29336]: 8.5 scrub starts 2024-09-10T18:52:58.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:58 smithi018 ceph-mon[29336]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-10T18:52:58.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:58 smithi018 ceph-mon[29336]: 5.18 scrub starts 2024-09-10T18:52:58.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:58 smithi018 ceph-mon[29336]: 5.18 scrub ok 2024-09-10T18:52:58.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:58 smithi018 ceph-mon[29336]: osdmap e187: 8 total, 8 up, 7 in 2024-09-10T18:52:58.949 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:58.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:58.950 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:52:59.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:58 smithi103 ceph-mon[33163]: 4.10 scrub starts 2024-09-10T18:52:59.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:58 smithi103 ceph-mon[33163]: 4.10 scrub ok 2024-09-10T18:52:59.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:58 smithi103 ceph-mon[33163]: 1.14 deep-scrub starts 2024-09-10T18:52:59.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:58 smithi103 ceph-mon[33163]: 1.10 scrub starts 2024-09-10T18:52:59.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:58 smithi103 ceph-mon[33163]: 1.10 scrub ok 2024-09-10T18:52:59.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:58 smithi103 ceph-mon[33163]: 1.14 deep-scrub ok 2024-09-10T18:52:59.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:58 smithi103 ceph-mon[33163]: 6.16 deep-scrub starts 2024-09-10T18:52:59.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:58 smithi103 ceph-mon[33163]: 6.16 deep-scrub ok 2024-09-10T18:52:59.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:58 smithi103 ceph-mon[33163]: 8.5 scrub starts 2024-09-10T18:52:59.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:58 smithi103 ceph-mon[33163]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-10T18:52:59.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:58 smithi103 ceph-mon[33163]: 5.18 scrub starts 2024-09-10T18:52:59.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:58 smithi103 ceph-mon[33163]: 5.18 scrub ok 2024-09-10T18:52:59.024 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:58 smithi103 ceph-mon[33163]: osdmap e187: 8 total, 8 up, 7 in 2024-09-10T18:52:59.051 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:52:59.090 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:59.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:59.091 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:52:59.192 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:59.204 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 70 16 121052 121036 108.049 86.4375 0.00809455 0.00925087 2024-09-10T18:52:59.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:59.233 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:59.233 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:59.333 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:52:59.373 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:59.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:59.374 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:52:59.475 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:52:59.515 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:52:59.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:59.516 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:52:59.620 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:59.660 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:59.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:59.661 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:52:59.762 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:52:59.798 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:59.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:59.799 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:52:59.900 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:52:59.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:52:59.937 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:52:59.937 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:00.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:59 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:53:00.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:59 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:53:00.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:59 smithi103 ceph-mon[33163]: 4.1 scrub starts 2024-09-10T18:53:00.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:59 smithi103 ceph-mon[33163]: 4.1 scrub ok 2024-09-10T18:53:00.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:59 smithi103 ceph-mon[33163]: osdmap e188: 8 total, 8 up, 7 in 2024-09-10T18:53:00.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:52:59 smithi103 ceph-mon[33163]: pgmap v333: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 7.1 GiB data, 23 GiB used, 602 GiB / 626 GiB avail; 64 MiB/s wr, 1.03k op/s; 14588/351774 objects misplaced (4.147%); 51 KiB/s, 3 objects/s recovering 2024-09-10T18:53:00.038 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:00.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:00.075 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:00.075 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:00.176 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:00.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:59 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:53:00.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:59 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:53:00.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:59 smithi018 ceph-mon[40196]: 4.1 scrub starts 2024-09-10T18:53:00.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:59 smithi018 ceph-mon[40196]: 4.1 scrub ok 2024-09-10T18:53:00.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:59 smithi018 ceph-mon[40196]: osdmap e188: 8 total, 8 up, 7 in 2024-09-10T18:53:00.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:52:59 smithi018 ceph-mon[40196]: pgmap v333: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 7.1 GiB data, 23 GiB used, 602 GiB / 626 GiB avail; 64 MiB/s wr, 1.03k op/s; 14588/351774 objects misplaced (4.147%); 51 KiB/s, 3 objects/s recovering 2024-09-10T18:53:00.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:59 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:53:00.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:59 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:53:00.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:59 smithi018 ceph-mon[29336]: 4.1 scrub starts 2024-09-10T18:53:00.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:59 smithi018 ceph-mon[29336]: 4.1 scrub ok 2024-09-10T18:53:00.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:59 smithi018 ceph-mon[29336]: osdmap e188: 8 total, 8 up, 7 in 2024-09-10T18:53:00.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:52:59 smithi018 ceph-mon[29336]: pgmap v333: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 7.1 GiB data, 23 GiB used, 602 GiB / 626 GiB avail; 64 MiB/s wr, 1.03k op/s; 14588/351774 objects misplaced (4.147%); 51 KiB/s, 3 objects/s recovering 2024-09-10T18:53:00.204 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 71 15 122414 122399 107.727 85.1875 0.00589197 0.00927795 2024-09-10T18:53:00.217 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:00.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:00.219 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:00.302 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-10T18:53:00.303 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd test-reweight-by-utilization 110 0.05 2024-09-10T18:53:00.320 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:00.391 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:00.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:00.392 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:00.494 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:00.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:53:00 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:53:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:53:00.535 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:00.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:00.537 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:00.638 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:00.670 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:00.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:00.671 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:00.772 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:00.819 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:00.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:00.820 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:00.921 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:00.954 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:00 smithi018 ceph-mon[40196]: 2.c scrub starts 2024-09-10T18:53:00.954 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:00 smithi018 ceph-mon[40196]: 2.c scrub ok 2024-09-10T18:53:00.954 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:00 smithi018 ceph-mon[40196]: 6.18 deep-scrub starts 2024-09-10T18:53:00.954 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:00 smithi018 ceph-mon[40196]: 6.18 deep-scrub ok 2024-09-10T18:53:00.954 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:00 smithi018 ceph-mon[29336]: 2.c scrub starts 2024-09-10T18:53:00.954 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:00 smithi018 ceph-mon[29336]: 2.c scrub ok 2024-09-10T18:53:00.954 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:00 smithi018 ceph-mon[29336]: 6.18 deep-scrub starts 2024-09-10T18:53:00.954 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:00 smithi018 ceph-mon[29336]: 6.18 deep-scrub ok 2024-09-10T18:53:00.955 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:00.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:00.956 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:01.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:00 smithi103 ceph-mon[33163]: 2.c scrub starts 2024-09-10T18:53:01.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:00 smithi103 ceph-mon[33163]: 2.c scrub ok 2024-09-10T18:53:01.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:00 smithi103 ceph-mon[33163]: 6.18 deep-scrub starts 2024-09-10T18:53:01.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:00 smithi103 ceph-mon[33163]: 6.18 deep-scrub ok 2024-09-10T18:53:01.057 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:01.089 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:01.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:01.090 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:01.191 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:01.205 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 72 16 123843 123827 107.47 89.25 0.00960585 0.00930049 2024-09-10T18:53:01.226 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:01.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:01.227 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:01.327 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:01.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:01.361 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:01.361 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:01.401 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:53:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:53:00] "GET /metrics HTTP/1.1" 200 223988 "" "Prometheus/2.43.0" 2024-09-10T18:53:01.447 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-10T18:53:01.462 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:01.478 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:53:01.528 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:01.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:01.529 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:01.630 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:01.671 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:01.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:01.673 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:01.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:01 smithi103 ceph-mon[33163]: 4.f scrub starts 2024-09-10T18:53:01.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:01 smithi103 ceph-mon[33163]: 4.f scrub ok 2024-09-10T18:53:01.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:01 smithi103 ceph-mon[33163]: 1.3 scrub starts 2024-09-10T18:53:01.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:01 smithi103 ceph-mon[33163]: 1.3 scrub ok 2024-09-10T18:53:01.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:01 smithi103 ceph-mon[33163]: 4.1a scrub starts 2024-09-10T18:53:01.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:01 smithi103 ceph-mon[33163]: 4.1a scrub ok 2024-09-10T18:53:01.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:01 smithi103 ceph-mon[33163]: pgmap v334: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 7.3 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 66 MiB/s wr, 1.05k op/s; 14588/357120 objects misplaced (4.085%); 21 KiB/s, 2 objects/s recovering 2024-09-10T18:53:01.773 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:01.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:01.817 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:01.817 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:01.855 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:53:01.919 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:01.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:01.964 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:01.964 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:02.045 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:01 smithi018 ceph-mon[40196]: 4.f scrub starts 2024-09-10T18:53:02.045 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:01 smithi018 ceph-mon[40196]: 4.f scrub ok 2024-09-10T18:53:02.045 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:01 smithi018 ceph-mon[40196]: 1.3 scrub starts 2024-09-10T18:53:02.045 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:01 smithi018 ceph-mon[40196]: 1.3 scrub ok 2024-09-10T18:53:02.045 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:01 smithi018 ceph-mon[40196]: 4.1a scrub starts 2024-09-10T18:53:02.045 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:01 smithi018 ceph-mon[40196]: 4.1a scrub ok 2024-09-10T18:53:02.045 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:01 smithi018 ceph-mon[40196]: pgmap v334: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 7.3 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 66 MiB/s wr, 1.05k op/s; 14588/357120 objects misplaced (4.085%); 21 KiB/s, 2 objects/s recovering 2024-09-10T18:53:02.046 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:01 smithi018 ceph-mon[29336]: 4.f scrub starts 2024-09-10T18:53:02.046 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:01 smithi018 ceph-mon[29336]: 4.f scrub ok 2024-09-10T18:53:02.046 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:01 smithi018 ceph-mon[29336]: 1.3 scrub starts 2024-09-10T18:53:02.046 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:01 smithi018 ceph-mon[29336]: 1.3 scrub ok 2024-09-10T18:53:02.046 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:01 smithi018 ceph-mon[29336]: 4.1a scrub starts 2024-09-10T18:53:02.046 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:01 smithi018 ceph-mon[29336]: 4.1a scrub ok 2024-09-10T18:53:02.046 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:01 smithi018 ceph-mon[29336]: pgmap v334: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 7.3 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 66 MiB/s wr, 1.05k op/s; 14588/357120 objects misplaced (4.085%); 21 KiB/s, 2 objects/s recovering 2024-09-10T18:53:02.065 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:02.103 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:02.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:02.104 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:02.204 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 73 16 125134 125118 107.103 80.6875 0.0108232 0.00933198 2024-09-10T18:53:02.206 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:02.239 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:02.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:02.240 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:02.341 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:02.377 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:53:02.377 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:53:02.387 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:53:02.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:02.389 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:02.389 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:02.393 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:53:02.393 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:53:02.433 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:53:02.434 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:53:02.475 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:53:02.475 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:53:02.490 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:02.510 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:53:02.511 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:53:02.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:02.533 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:02.534 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:02.544 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:53:02.544 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:53:02.577 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:53:02.578 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:53:02.610 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:53:02.610 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:53:02.635 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:02.676 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:02.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:02.677 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:02.778 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:02.809 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:02.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:02.810 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:02.911 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:02.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:02 smithi018 ceph-mon[29336]: 5.17 deep-scrub starts 2024-09-10T18:53:02.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:02 smithi018 ceph-mon[29336]: 5.17 deep-scrub ok 2024-09-10T18:53:02.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:02 smithi018 ceph-mon[29336]: 8.5 scrub ok 2024-09-10T18:53:02.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:02 smithi018 ceph-mon[29336]: 1.d scrub starts 2024-09-10T18:53:02.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:02 smithi018 ceph-mon[29336]: 1.d scrub ok 2024-09-10T18:53:02.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:02 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2286899769' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:53:02.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:02 smithi018 ceph-mon[40196]: 5.17 deep-scrub starts 2024-09-10T18:53:02.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:02 smithi018 ceph-mon[40196]: 5.17 deep-scrub ok 2024-09-10T18:53:02.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:02 smithi018 ceph-mon[40196]: 8.5 scrub ok 2024-09-10T18:53:02.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:02 smithi018 ceph-mon[40196]: 1.d scrub starts 2024-09-10T18:53:02.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:02 smithi018 ceph-mon[40196]: 1.d scrub ok 2024-09-10T18:53:02.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:02 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2286899769' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:53:02.956 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:02.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:02.958 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:03.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:02 smithi103 ceph-mon[33163]: 5.17 deep-scrub starts 2024-09-10T18:53:03.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:02 smithi103 ceph-mon[33163]: 5.17 deep-scrub ok 2024-09-10T18:53:03.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:02 smithi103 ceph-mon[33163]: 8.5 scrub ok 2024-09-10T18:53:03.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:02 smithi103 ceph-mon[33163]: 1.d scrub starts 2024-09-10T18:53:03.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:02 smithi103 ceph-mon[33163]: 1.d scrub ok 2024-09-10T18:53:03.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:02 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2286899769' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:53:03.058 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:03.093 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:03.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:03.094 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:03.195 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:03.205 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 74 16 126461 126445 106.775 82.9375 0.016131 0.00936093 2024-09-10T18:53:03.228 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:03.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:03.229 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:03.330 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:03.372 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:03.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:03.374 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:03.474 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:03.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:03.516 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:03.516 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:03.617 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:03.659 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:03.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:03.660 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:03.760 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:03.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:03 smithi018 ceph-mon[29336]: 4.0 scrub starts 2024-09-10T18:53:03.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:03 smithi018 ceph-mon[29336]: 4.0 scrub ok 2024-09-10T18:53:03.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:03 smithi018 ceph-mon[29336]: 8.2 scrub starts 2024-09-10T18:53:03.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:03 smithi018 ceph-mon[29336]: pgmap v335: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 7.6 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 72 MiB/s wr, 1.14k op/s; 15467/372678 objects misplaced (4.150%) 2024-09-10T18:53:03.774 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:03 smithi018 ceph-mon[29336]: osdmap e189: 8 total, 8 up, 7 in 2024-09-10T18:53:03.775 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:03 smithi018 ceph-mon[40196]: 4.0 scrub starts 2024-09-10T18:53:03.775 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:03 smithi018 ceph-mon[40196]: 4.0 scrub ok 2024-09-10T18:53:03.775 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:03 smithi018 ceph-mon[40196]: 8.2 scrub starts 2024-09-10T18:53:03.775 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:03 smithi018 ceph-mon[40196]: pgmap v335: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 7.6 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 72 MiB/s wr, 1.14k op/s; 15467/372678 objects misplaced (4.150%) 2024-09-10T18:53:03.775 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:03 smithi018 ceph-mon[40196]: osdmap e189: 8 total, 8 up, 7 in 2024-09-10T18:53:03.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:03.799 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:03.799 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:03.900 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:03.932 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:03.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:03.934 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:04.003 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:03 smithi103 ceph-mon[33163]: 4.0 scrub starts 2024-09-10T18:53:04.003 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:03 smithi103 ceph-mon[33163]: 4.0 scrub ok 2024-09-10T18:53:04.003 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:03 smithi103 ceph-mon[33163]: 8.2 scrub starts 2024-09-10T18:53:04.003 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:03 smithi103 ceph-mon[33163]: pgmap v335: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 7.6 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 72 MiB/s wr, 1.14k op/s; 15467/372678 objects misplaced (4.150%) 2024-09-10T18:53:04.003 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:03 smithi103 ceph-mon[33163]: osdmap e189: 8 total, 8 up, 7 in 2024-09-10T18:53:04.034 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:04.054 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T18:53:04.054 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:53:04.077 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:04.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:04.078 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:04.179 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:04.205 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 75 16 127717 127701 106.398 78.5 0.0202389 0.00939399 2024-09-10T18:53:04.210 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:04.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:04.212 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:04.312 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:04.348 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:04.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:04.349 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:04.450 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:04.478 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:04.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:04.479 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:04.580 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:04.623 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:04.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:04.624 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:04.725 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:04.745 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-10T18:53:04.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:04.778 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:04.778 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:04.880 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:04.936 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:04.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:04.937 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:05.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:04 smithi103 ceph-mon[33163]: 5.0 scrub starts 2024-09-10T18:53:05.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:04 smithi103 ceph-mon[33163]: 5.0 scrub ok 2024-09-10T18:53:05.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:04 smithi103 ceph-mon[33163]: 5.16 scrub starts 2024-09-10T18:53:05.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:04 smithi103 ceph-mon[33163]: 5.16 scrub ok 2024-09-10T18:53:05.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:04 smithi103 ceph-mon[33163]: 2.14 scrub starts 2024-09-10T18:53:05.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:04 smithi103 ceph-mon[33163]: 2.14 scrub ok 2024-09-10T18:53:05.038 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:05.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:05.069 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:05.069 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:05.157 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:04 smithi018 ceph-mon[40196]: 5.0 scrub starts 2024-09-10T18:53:05.157 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:04 smithi018 ceph-mon[40196]: 5.0 scrub ok 2024-09-10T18:53:05.157 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:04 smithi018 ceph-mon[40196]: 5.16 scrub starts 2024-09-10T18:53:05.157 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:04 smithi018 ceph-mon[40196]: 5.16 scrub ok 2024-09-10T18:53:05.157 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:04 smithi018 ceph-mon[40196]: 2.14 scrub starts 2024-09-10T18:53:05.157 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:04 smithi018 ceph-mon[40196]: 2.14 scrub ok 2024-09-10T18:53:05.158 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:04 smithi018 ceph-mon[29336]: 5.0 scrub starts 2024-09-10T18:53:05.158 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:04 smithi018 ceph-mon[29336]: 5.0 scrub ok 2024-09-10T18:53:05.158 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:04 smithi018 ceph-mon[29336]: 5.16 scrub starts 2024-09-10T18:53:05.158 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:04 smithi018 ceph-mon[29336]: 5.16 scrub ok 2024-09-10T18:53:05.158 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:04 smithi018 ceph-mon[29336]: 2.14 scrub starts 2024-09-10T18:53:05.158 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:04 smithi018 ceph-mon[29336]: 2.14 scrub ok 2024-09-10T18:53:05.171 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:05.202 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:05.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:05.203 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:05.205 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 76 16 128839 128823 105.921 70.125 0.00721996 0.00943583 2024-09-10T18:53:05.304 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:05.309 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:53:05.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:05.344 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:05.345 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:05.445 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:05.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:05.480 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:05.480 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:05.539 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T18:53:05.581 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:05.622 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:05.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:05.623 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:05.723 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:05.763 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:05.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:05.765 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:05.865 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:05.906 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:05.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:05.907 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:06.008 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:06.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:05 smithi103 ceph-mon[33163]: 2.1e scrub starts 2024-09-10T18:53:06.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:05 smithi103 ceph-mon[33163]: 2.1e scrub ok 2024-09-10T18:53:06.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:05 smithi103 ceph-mon[33163]: 1.b scrub starts 2024-09-10T18:53:06.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:05 smithi103 ceph-mon[33163]: 1.b scrub ok 2024-09-10T18:53:06.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:05 smithi103 ceph-mon[33163]: pgmap v337: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 7.6 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 62 MiB/s wr, 984 op/s; 15467/372678 objects misplaced (4.150%) 2024-09-10T18:53:06.038 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:06.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:06.039 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:06.140 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:06.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:06.163 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:06.163 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:06.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:05 smithi018 ceph-mon[40196]: 2.1e scrub starts 2024-09-10T18:53:06.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:05 smithi018 ceph-mon[40196]: 2.1e scrub ok 2024-09-10T18:53:06.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:05 smithi018 ceph-mon[40196]: 1.b scrub starts 2024-09-10T18:53:06.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:05 smithi018 ceph-mon[40196]: 1.b scrub ok 2024-09-10T18:53:06.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:05 smithi018 ceph-mon[40196]: pgmap v337: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 7.6 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 62 MiB/s wr, 984 op/s; 15467/372678 objects misplaced (4.150%) 2024-09-10T18:53:06.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:05 smithi018 ceph-mon[29336]: 2.1e scrub starts 2024-09-10T18:53:06.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:05 smithi018 ceph-mon[29336]: 2.1e scrub ok 2024-09-10T18:53:06.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:05 smithi018 ceph-mon[29336]: 1.b scrub starts 2024-09-10T18:53:06.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:05 smithi018 ceph-mon[29336]: 1.b scrub ok 2024-09-10T18:53:06.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:05 smithi018 ceph-mon[29336]: pgmap v337: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 7.6 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 62 MiB/s wr, 984 op/s; 15467/372678 objects misplaced (4.150%) 2024-09-10T18:53:06.206 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 77 16 129887 129871 105.396 65.5 0.0213109 0.00948325 2024-09-10T18:53:06.263 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:06.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:06.305 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:06.306 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:06.406 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:06.445 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:06.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:06.447 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:06.547 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:06.578 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:06.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:06.579 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:06.679 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:06.728 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:06.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:06.729 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:06.730 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:53:06 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:53:06.830 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:06.873 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:06.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:06.874 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:06.975 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:06.992 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:06 smithi103 ceph-mon[33163]: 6.f scrub starts 2024-09-10T18:53:06.992 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:06 smithi103 ceph-mon[33163]: 6.f scrub ok 2024-09-10T18:53:06.992 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:06 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:53:06.992 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:06 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:53:06.992 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:06 smithi103 ceph-mon[33163]: osdmap e190: 8 total, 8 up, 7 in 2024-09-10T18:53:06.992 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:06 smithi103 ceph-mon[33163]: 1.c scrub starts 2024-09-10T18:53:06.992 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:06 smithi103 ceph-mon[33163]: 1.c scrub ok 2024-09-10T18:53:06.992 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:06 smithi103 ceph-mon[33163]: 4.c scrub starts 2024-09-10T18:53:06.992 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:06 smithi103 ceph-mon[33163]: 4.c scrub ok 2024-09-10T18:53:06.992 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:06 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:07.017 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:06 smithi018 ceph-mon[40196]: 6.f scrub starts 2024-09-10T18:53:07.017 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:06 smithi018 ceph-mon[40196]: 6.f scrub ok 2024-09-10T18:53:07.017 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:06 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:53:07.017 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:06 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:53:07.018 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:06 smithi018 ceph-mon[40196]: osdmap e190: 8 total, 8 up, 7 in 2024-09-10T18:53:07.018 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:06 smithi018 ceph-mon[40196]: 1.c scrub starts 2024-09-10T18:53:07.018 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:06 smithi018 ceph-mon[40196]: 1.c scrub ok 2024-09-10T18:53:07.018 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:06 smithi018 ceph-mon[40196]: 4.c scrub starts 2024-09-10T18:53:07.018 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:06 smithi018 ceph-mon[40196]: 4.c scrub ok 2024-09-10T18:53:07.018 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:06 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:07.018 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:06 smithi018 ceph-mon[29336]: 6.f scrub starts 2024-09-10T18:53:07.018 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:06 smithi018 ceph-mon[29336]: 6.f scrub ok 2024-09-10T18:53:07.018 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:06 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:53:07.019 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:06 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:53:07.019 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:06 smithi018 ceph-mon[29336]: osdmap e190: 8 total, 8 up, 7 in 2024-09-10T18:53:07.019 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:06 smithi018 ceph-mon[29336]: 1.c scrub starts 2024-09-10T18:53:07.019 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:06 smithi018 ceph-mon[29336]: 1.c scrub ok 2024-09-10T18:53:07.019 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:06 smithi018 ceph-mon[29336]: 4.c scrub starts 2024-09-10T18:53:07.019 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:06 smithi018 ceph-mon[29336]: 4.c scrub ok 2024-09-10T18:53:07.019 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:06 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:07.022 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:07.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:07.023 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:07.124 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:07.163 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:07.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:07.164 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:07.168 INFO:teuthology.orchestra.run.smithi018.stdout:moved 6 / 504 (1.19048%) 2024-09-10T18:53:07.168 INFO:teuthology.orchestra.run.smithi018.stdout:avg 72 2024-09-10T18:53:07.168 INFO:teuthology.orchestra.run.smithi018.stdout:stddev 6.04743 -> 8.29802 (expected baseline 7.85584) 2024-09-10T18:53:07.168 INFO:teuthology.orchestra.run.smithi018.stdout:min osd.5 with 59 -> 53 pgs (0.819444 -> 0.736111 * mean) 2024-09-10T18:53:07.168 INFO:teuthology.orchestra.run.smithi018.stdout:max osd.3 with 78 -> 79 pgs (1.08333 -> 1.09722 * mean) 2024-09-10T18:53:07.168 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:53:07.168 INFO:teuthology.orchestra.run.smithi018.stdout:oload 110 2024-09-10T18:53:07.168 INFO:teuthology.orchestra.run.smithi018.stdout:max_change 0.05 2024-09-10T18:53:07.168 INFO:teuthology.orchestra.run.smithi018.stdout:max_change_osds 4 2024-09-10T18:53:07.169 INFO:teuthology.orchestra.run.smithi018.stdout:average_utilization 0.0412 2024-09-10T18:53:07.169 INFO:teuthology.orchestra.run.smithi018.stdout:overload_utilization 0.0453 2024-09-10T18:53:07.169 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 weight 0.7495 -> 0.6995 2024-09-10T18:53:07.169 INFO:teuthology.orchestra.run.smithi018.stderr:no change 2024-09-10T18:53:07.206 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 78 16 131099 131083 105.016 75.75 0.0244641 0.00951766 2024-09-10T18:53:07.243 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T18:53:07.243 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:53:07.265 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:07.307 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:07.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:07.308 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:07.409 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:07.443 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:07.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:07.444 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:07.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:53:07 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:53:07.163+0000 7f66f486f640 -1 mgr.server reply reply (1) Operation not permitted no change 2024-09-10T18:53:07.544 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:07.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:07.583 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:07.583 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:07.684 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:07.723 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:07.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:07.724 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:07.740 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is set 2024-09-10T18:53:07.825 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:07.860 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:07.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:07.861 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:07.962 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:07.983 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:07.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:07.984 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:08.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:07 smithi103 ceph-mon[33163]: 8.8 scrub starts 2024-09-10T18:53:08.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:07 smithi103 ceph-mon[33163]: 1.13 scrub starts 2024-09-10T18:53:08.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:07 smithi103 ceph-mon[33163]: 1.13 scrub ok 2024-09-10T18:53:08.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:07 smithi103 ceph-mon[33163]: 8.2 scrub ok 2024-09-10T18:53:08.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:07 smithi103 ceph-mon[33163]: pgmap v339: 168 pgs: 2 active+clean+scrubbing, 2 active+remapped+backfill_toofull, 164 active+clean; 7.9 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 0 B/s rd, 101 MiB/s wr, 1.62k op/s; 16198/390762 objects misplaced (4.145%) 2024-09-10T18:53:08.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:07 smithi103 ceph-mon[33163]: from='client.25031 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:53:08.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:07 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/84107241' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:53:08.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:07 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:53:08.085 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:08.123 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:08.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:08.124 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:08.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:07 smithi018 ceph-mon[40196]: 8.8 scrub starts 2024-09-10T18:53:08.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:07 smithi018 ceph-mon[40196]: 1.13 scrub starts 2024-09-10T18:53:08.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:07 smithi018 ceph-mon[40196]: 1.13 scrub ok 2024-09-10T18:53:08.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:07 smithi018 ceph-mon[40196]: 8.2 scrub ok 2024-09-10T18:53:08.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:07 smithi018 ceph-mon[40196]: pgmap v339: 168 pgs: 2 active+clean+scrubbing, 2 active+remapped+backfill_toofull, 164 active+clean; 7.9 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 0 B/s rd, 101 MiB/s wr, 1.62k op/s; 16198/390762 objects misplaced (4.145%) 2024-09-10T18:53:08.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:07 smithi018 ceph-mon[40196]: from='client.25031 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:53:08.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:07 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/84107241' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:53:08.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:07 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:53:08.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:07 smithi018 ceph-mon[29336]: 8.8 scrub starts 2024-09-10T18:53:08.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:07 smithi018 ceph-mon[29336]: 1.13 scrub starts 2024-09-10T18:53:08.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:07 smithi018 ceph-mon[29336]: 1.13 scrub ok 2024-09-10T18:53:08.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:07 smithi018 ceph-mon[29336]: 8.2 scrub ok 2024-09-10T18:53:08.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:07 smithi018 ceph-mon[29336]: pgmap v339: 168 pgs: 2 active+clean+scrubbing, 2 active+remapped+backfill_toofull, 164 active+clean; 7.9 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 0 B/s rd, 101 MiB/s wr, 1.62k op/s; 16198/390762 objects misplaced (4.145%) 2024-09-10T18:53:08.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:07 smithi018 ceph-mon[29336]: from='client.25031 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:53:08.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:07 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/84107241' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:53:08.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:07 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:53:08.206 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 79 15 132279 132264 104.62 73.8125 0.0129875 0.00955387 2024-09-10T18:53:08.225 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:08.258 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:08.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:08.259 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:08.332 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-10T18:53:08.359 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:08.392 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:08.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:08.393 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:08.494 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:08.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:08.538 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:08.538 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:08.639 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:08.681 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:08.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:08.682 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:08.772 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:53:08.783 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:08.815 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:08.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:08.816 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:08.917 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:08.954 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:08.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:08.955 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:09.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:08 smithi103 ceph-mon[33163]: 6.5 deep-scrub starts 2024-09-10T18:53:09.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:08 smithi103 ceph-mon[33163]: 6.5 deep-scrub ok 2024-09-10T18:53:09.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:08 smithi103 ceph-mon[33163]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:53:09.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:08 smithi103 ceph-mon[33163]: 5.1f deep-scrub starts 2024-09-10T18:53:09.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:08 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:53:09.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:08 smithi103 ceph-mon[33163]: osdmap e191: 8 total, 8 up, 7 in 2024-09-10T18:53:09.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:08 smithi103 ceph-mon[33163]: 5.1f deep-scrub ok 2024-09-10T18:53:09.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:08 smithi103 ceph-mon[33163]: 3.2 scrub starts 2024-09-10T18:53:09.056 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:09.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:09.095 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:09.095 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:09.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:08 smithi018 ceph-mon[40196]: 6.5 deep-scrub starts 2024-09-10T18:53:09.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:08 smithi018 ceph-mon[40196]: 6.5 deep-scrub ok 2024-09-10T18:53:09.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:08 smithi018 ceph-mon[40196]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:53:09.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:08 smithi018 ceph-mon[40196]: 5.1f deep-scrub starts 2024-09-10T18:53:09.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:08 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:53:09.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:08 smithi018 ceph-mon[40196]: osdmap e191: 8 total, 8 up, 7 in 2024-09-10T18:53:09.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:08 smithi018 ceph-mon[40196]: 5.1f deep-scrub ok 2024-09-10T18:53:09.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:08 smithi018 ceph-mon[40196]: 3.2 scrub starts 2024-09-10T18:53:09.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:08 smithi018 ceph-mon[29336]: 6.5 deep-scrub starts 2024-09-10T18:53:09.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:08 smithi018 ceph-mon[29336]: 6.5 deep-scrub ok 2024-09-10T18:53:09.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:08 smithi018 ceph-mon[29336]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:53:09.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:08 smithi018 ceph-mon[29336]: 5.1f deep-scrub starts 2024-09-10T18:53:09.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:08 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:53:09.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:08 smithi018 ceph-mon[29336]: osdmap e191: 8 total, 8 up, 7 in 2024-09-10T18:53:09.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:08 smithi018 ceph-mon[29336]: 5.1f deep-scrub ok 2024-09-10T18:53:09.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:08 smithi018 ceph-mon[29336]: 3.2 scrub starts 2024-09-10T18:53:09.196 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:09.206 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:2024-09-10T18:53:09.206286+0000 min lat: 0.00115328 max lat: 2.09392 avg lat: 0.00958008 2024-09-10T18:53:09.206 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T18:53:09.206 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 80 16 133587 133571 104.333 81.6875 0.00544924 0.00958008 2024-09-10T18:53:09.230 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:09.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:09.231 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:09.332 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:09.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:09.369 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:09.370 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:09.470 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:09.497 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:09.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:09.498 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:09.599 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:09.640 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:09.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:09.641 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:09.741 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:09.762 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:53:09.788 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-10T18:53:09.789 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd test-reweight-by-utilization 110 0.05 2024-09-10T18:53:09.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:09.800 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:09.800 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:09.901 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:09.938 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:09.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:09.939 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:10.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:09 smithi103 ceph-mon[33163]: pgmap v341: 168 pgs: 2 active+clean+scrubbing, 2 active+remapped+backfill_toofull, 164 active+clean; 7.9 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 853 B/s rd, 63 MiB/s wr, 1.00k op/s; 16198/390762 objects misplaced (4.145%) 2024-09-10T18:53:10.041 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:10.088 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:10.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:10.089 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:10.095 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:09 smithi018 ceph-mon[40196]: pgmap v341: 168 pgs: 2 active+clean+scrubbing, 2 active+remapped+backfill_toofull, 164 active+clean; 7.9 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 853 B/s rd, 63 MiB/s wr, 1.00k op/s; 16198/390762 objects misplaced (4.145%) 2024-09-10T18:53:10.099 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:09 smithi018 ceph-mon[29336]: pgmap v341: 168 pgs: 2 active+clean+scrubbing, 2 active+remapped+backfill_toofull, 164 active+clean; 7.9 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 853 B/s rd, 63 MiB/s wr, 1.00k op/s; 16198/390762 objects misplaced (4.145%) 2024-09-10T18:53:10.190 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:10.206 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 81 16 134873 134857 104.037 80.375 0.00444691 0.00960663 2024-09-10T18:53:10.226 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:10.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:10.227 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:10.328 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:10.372 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:10.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:10.373 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:10.474 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:10.515 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:10.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:10.516 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:10.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:53:10 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:53:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:53:10.617 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:10.647 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:10.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:10.648 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:10.749 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:10.780 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:10.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:10.783 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:10.884 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:10.923 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:10.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:10.924 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:11.025 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:11.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:11.061 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:11.062 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:11.135 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:53:11.162 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:11.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:11.196 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:11.196 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:11.210 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 82 16 136278 136262 103.837 87.8125 0.012158 0.0096263 2024-09-10T18:53:11.270 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-10T18:53:11.278 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:11 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:53:11.278 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:11 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:53:11.279 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:11 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:53:11.279 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:11 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:53:11.279 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:53:10 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:53:10] "GET /metrics HTTP/1.1" 200 224058 "" "Prometheus/2.43.0" 2024-09-10T18:53:11.298 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:11.331 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:11.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:11.332 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:11.358 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:11 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:53:11.358 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:11 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:53:11.434 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:11.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:11.484 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:11.484 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:11.585 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:11.624 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:11.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:11.625 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:11.726 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:11.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:11.785 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:11.785 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:11.886 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:11.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:11.922 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:11.922 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:12.022 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:12.024 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T18:53:12.024 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:53:12.066 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:12.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:12.067 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:12.168 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:12.203 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:12.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:12.204 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:12.208 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 83 16 137625 137609 103.601 84.1875 0.0115919 0.00964823 2024-09-10T18:53:12.306 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:12.342 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:12.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:12.343 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:12.351 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:12 smithi018 ceph-mon[40196]: pgmap v342: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 8.0 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 674 B/s rd, 63 MiB/s wr, 1.01k op/s; 16198/395751 objects misplaced (4.093%) 2024-09-10T18:53:12.351 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:12 smithi018 ceph-mon[29336]: pgmap v342: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 8.0 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 674 B/s rd, 63 MiB/s wr, 1.01k op/s; 16198/395751 objects misplaced (4.093%) 2024-09-10T18:53:12.444 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:12 smithi103 ceph-mon[33163]: pgmap v342: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 8.0 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 674 B/s rd, 63 MiB/s wr, 1.01k op/s; 16198/395751 objects misplaced (4.093%) 2024-09-10T18:53:12.444 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:12.478 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:12.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:12.480 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:12.580 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:12.625 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:12.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:12.626 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:12.711 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:53:12.711 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:53:12.726 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:12.744 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:53:12.745 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:53:12.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:12.770 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:12.770 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:12.772 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:53:12.772 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:53:12.796 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-10T18:53:12.798 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:53:12.798 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:53:12.836 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:53:12.837 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:53:12.871 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:12.872 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:53:12.873 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:53:12.913 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:12.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:12.914 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:12.918 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:53:12.919 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:53:12.948 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:53:12.948 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:53:13.015 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:13.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:13.063 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:13.063 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:13.164 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:13.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:13 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3636502958' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:53:13.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:13 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3636502958' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:53:13.204 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:13.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:13.205 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:13.210 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 84 16 138967 138951 103.363 83.875 0.00665969 0.00966993 2024-09-10T18:53:13.281 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:53:13.307 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:13.349 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:13.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:13.350 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:13.415 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:13 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3636502958' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:53:13.451 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:13.492 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:13.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:13.493 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:13.594 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:13.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:13.637 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:13.637 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:13.738 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:13.782 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:13.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:13.783 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:13.884 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:13.921 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:13.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:13.922 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:14.023 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:14.071 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:14.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:14.072 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:14.157 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:53:14.173 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:14.211 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 85 16 140220 140204 103.068 78.3125 0.0142814 0.00969756 2024-09-10T18:53:14.226 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:14.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:14.227 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:14.276 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:14 smithi018 ceph-mon[29336]: pgmap v343: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 8.3 GiB data, 27 GiB used, 599 GiB / 626 GiB avail; 1.2 KiB/s rd, 97 MiB/s wr, 1.56k op/s; 17031/410037 objects misplaced (4.154%) 2024-09-10T18:53:14.277 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:14 smithi018 ceph-mon[40196]: pgmap v343: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 8.3 GiB data, 27 GiB used, 599 GiB / 626 GiB avail; 1.2 KiB/s rd, 97 MiB/s wr, 1.56k op/s; 17031/410037 objects misplaced (4.154%) 2024-09-10T18:53:14.327 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:14.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:14.371 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:14.371 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:14.469 INFO:teuthology.orchestra.run.smithi018.stdout:moved 6 / 504 (1.19048%) 2024-09-10T18:53:14.470 INFO:teuthology.orchestra.run.smithi018.stdout:avg 72 2024-09-10T18:53:14.470 INFO:teuthology.orchestra.run.smithi018.stdout:stddev 6.04743 -> 8.29802 (expected baseline 7.85584) 2024-09-10T18:53:14.470 INFO:teuthology.orchestra.run.smithi018.stdout:min osd.5 with 59 -> 53 pgs (0.819444 -> 0.736111 * mean) 2024-09-10T18:53:14.470 INFO:teuthology.orchestra.run.smithi018.stdout:max osd.3 with 78 -> 79 pgs (1.08333 -> 1.09722 * mean) 2024-09-10T18:53:14.470 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:53:14.470 INFO:teuthology.orchestra.run.smithi018.stdout:oload 110 2024-09-10T18:53:14.470 INFO:teuthology.orchestra.run.smithi018.stdout:max_change 0.05 2024-09-10T18:53:14.470 INFO:teuthology.orchestra.run.smithi018.stdout:max_change_osds 4 2024-09-10T18:53:14.470 INFO:teuthology.orchestra.run.smithi018.stdout:average_utilization 0.0430 2024-09-10T18:53:14.470 INFO:teuthology.orchestra.run.smithi018.stdout:overload_utilization 0.0473 2024-09-10T18:53:14.470 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 weight 0.7495 -> 0.6995 2024-09-10T18:53:14.471 INFO:teuthology.orchestra.run.smithi018.stderr:no change 2024-09-10T18:53:14.472 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:14.497 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:14 smithi103 ceph-mon[33163]: pgmap v343: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 8.3 GiB data, 27 GiB used, 599 GiB / 626 GiB avail; 1.2 KiB/s rd, 97 MiB/s wr, 1.56k op/s; 17031/410037 objects misplaced (4.154%) 2024-09-10T18:53:14.517 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:14.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:14.518 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:14.619 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:14.628 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:53:14 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T18:53:14.463+0000 7f66f486f640 -1 mgr.server reply reply (1) Operation not permitted no change 2024-09-10T18:53:14.657 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:14.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:14.658 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:14.759 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:14.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:14.798 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:14.798 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:14.899 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:14.947 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:14.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:14.948 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:15.048 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:15.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:15.074 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:15.074 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:15.175 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:15.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:15 smithi018 ceph-mon[40196]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:53:15.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:15 smithi018 ceph-mon[29336]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:53:15.213 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 86 16 141525 141509 102.817 81.5625 0.0123109 0.00972171 2024-09-10T18:53:15.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:15.214 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:15.214 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:15.315 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:15.352 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:15.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:15.353 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:15.454 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:15.493 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:15.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:15.494 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:15.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:15 smithi103 ceph-mon[33163]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:53:15.594 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:15.624 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:15.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:15.626 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:15.650 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-10T18:53:15.650 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd reweight-by-utilization 110 0.05 2024-09-10T18:53:15.727 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:15.760 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:15.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:15.761 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:15.861 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:15.901 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:15.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:15.902 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:16.003 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:16.040 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:16.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:16.041 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:16.136 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:16 smithi018 ceph-mon[29336]: pgmap v344: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 8.3 GiB data, 27 GiB used, 599 GiB / 626 GiB avail; 1.1 KiB/s rd, 82 MiB/s wr, 1.32k op/s; 17031/410037 objects misplaced (4.154%) 2024-09-10T18:53:16.136 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:16 smithi018 ceph-mon[40196]: pgmap v344: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 8.3 GiB data, 27 GiB used, 599 GiB / 626 GiB avail; 1.1 KiB/s rd, 82 MiB/s wr, 1.32k op/s; 17031/410037 objects misplaced (4.154%) 2024-09-10T18:53:16.143 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:16.166 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:16.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:16.167 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:16.212 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 87 16 142674 142658 102.46 71.8125 0.00882424 0.00975509 2024-09-10T18:53:16.268 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:16.306 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:16.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:16.307 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:16.360 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:16 smithi103 ceph-mon[33163]: pgmap v344: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 8.3 GiB data, 27 GiB used, 599 GiB / 626 GiB avail; 1.1 KiB/s rd, 82 MiB/s wr, 1.32k op/s; 17031/410037 objects misplaced (4.154%) 2024-09-10T18:53:16.408 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:16.441 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:16.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:16.442 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:16.543 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:16.574 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:16.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:16.575 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:16.614 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T18:53:16.614 INFO:teuthology.orchestra.run.smithi103.stdout: "ops": [ 2024-09-10T18:53:16.614 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.614 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:143157 8.d 8:bc7205eb:::benchmark_data_smithi018_106531_object143156:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e191)", 2024-09-10T18:53:16.614 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:53:16.598122+0000", 2024-09-10T18:53:16.614 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.012769134999999999, 2024-09-10T18:53:16.614 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.012802087, 2024-09-10T18:53:16.614 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:53:16.615 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "waiting for sub ops", 2024-09-10T18:53:16.615 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:53:16.615 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:53:16.615 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:53:16.615 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 143157 2024-09-10T18:53:16.615 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.615 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:53:16.615 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.615 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:53:16.615 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.598122+0000", 2024-09-10T18:53:16.615 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:53:16.615 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.615 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.616 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:53:16.616 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.598122+0000", 2024-09-10T18:53:16.616 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:53:16.616 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.616 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.616 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:53:16.616 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.598120+0000", 2024-09-10T18:53:16.616 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:53:16.616 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.616 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.617 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:53:16.617 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.598149+0000", 2024-09-10T18:53:16.617 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.8787999999999999e-05 2024-09-10T18:53:16.617 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.617 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.617 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:53:16.617 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.598160+0000", 2024-09-10T18:53:16.617 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.1144999999999999e-05 2024-09-10T18:53:16.617 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.617 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.617 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:53:16.617 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.598166+0000", 2024-09-10T18:53:16.618 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.6999999999999996e-06 2024-09-10T18:53:16.618 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.618 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.618 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:53:16.618 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.601863+0000", 2024-09-10T18:53:16.618 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0036965779999999998 2024-09-10T18:53:16.618 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.618 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.618 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:53:16.618 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.601973+0000", 2024-09-10T18:53:16.618 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000110562 2024-09-10T18:53:16.618 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.618 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.619 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 6,7", 2024-09-10T18:53:16.619 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.602107+0000", 2024-09-10T18:53:16.619 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000133456 2024-09-10T18:53:16.619 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.619 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.619 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:53:16.619 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.609908+0000", 2024-09-10T18:53:16.619 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00780097 2024-09-10T18:53:16.619 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:53:16.619 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:53:16.619 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:53:16.619 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.619 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.620 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:143158 8.8 8:117be568:::benchmark_data_smithi018_106531_object143157:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e191)", 2024-09-10T18:53:16.620 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:53:16.600877+0000", 2024-09-10T18:53:16.620 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.010014051, 2024-09-10T18:53:16.620 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.010127311999999999, 2024-09-10T18:53:16.620 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:53:16.620 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "waiting for sub ops", 2024-09-10T18:53:16.620 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:53:16.620 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:53:16.620 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:53:16.620 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 143158 2024-09-10T18:53:16.620 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.620 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:53:16.621 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.621 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:53:16.621 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.600877+0000", 2024-09-10T18:53:16.621 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:53:16.621 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.621 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.621 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:53:16.621 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.600877+0000", 2024-09-10T18:53:16.621 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:53:16.621 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.621 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.621 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:53:16.621 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.600876+0000", 2024-09-10T18:53:16.622 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:53:16.622 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.622 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.622 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:53:16.622 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.600911+0000", 2024-09-10T18:53:16.622 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.5123000000000002e-05 2024-09-10T18:53:16.622 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.622 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.622 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:53:16.622 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.600927+0000", 2024-09-10T18:53:16.622 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.5866999999999999e-05 2024-09-10T18:53:16.622 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.622 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.623 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:53:16.623 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.600933+0000", 2024-09-10T18:53:16.623 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.2589999999999997e-06 2024-09-10T18:53:16.623 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.623 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.623 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:53:16.623 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.600997+0000", 2024-09-10T18:53:16.623 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.4005999999999997e-05 2024-09-10T18:53:16.623 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.623 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.623 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:53:16.623 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.601130+0000", 2024-09-10T18:53:16.623 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00013369900000000001 2024-09-10T18:53:16.624 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.624 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.624 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,7", 2024-09-10T18:53:16.624 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.601578+0000", 2024-09-10T18:53:16.624 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00044782699999999998 2024-09-10T18:53:16.624 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.624 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.624 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:53:16.624 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.609937+0000", 2024-09-10T18:53:16.624 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0083583370000000004 2024-09-10T18:53:16.624 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.624 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.624 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:53:16.624 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.609960+0000", 2024-09-10T18:53:16.625 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.3726999999999998e-05 2024-09-10T18:53:16.625 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:53:16.625 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:53:16.625 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:53:16.625 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.625 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.625 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:143162 8.4 8:23c0fb8a:::benchmark_data_smithi018_106531_object143161:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e191)", 2024-09-10T18:53:16.625 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:53:16.604191+0000", 2024-09-10T18:53:16.625 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0067003369999999998, 2024-09-10T18:53:16.625 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0069024799999999999, 2024-09-10T18:53:16.625 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:53:16.625 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "waiting for sub ops", 2024-09-10T18:53:16.625 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:53:16.626 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:53:16.626 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:53:16.626 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 143162 2024-09-10T18:53:16.626 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.626 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:53:16.626 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.626 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:53:16.626 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.604191+0000", 2024-09-10T18:53:16.626 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:53:16.626 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.626 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.626 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:53:16.626 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.604191+0000", 2024-09-10T18:53:16.626 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:53:16.627 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.627 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.627 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:53:16.627 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.604189+0000", 2024-09-10T18:53:16.627 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:53:16.627 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.627 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.627 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:53:16.627 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.604218+0000", 2024-09-10T18:53:16.627 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.8694e-05 2024-09-10T18:53:16.627 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.627 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.627 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:53:16.627 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.604229+0000", 2024-09-10T18:53:16.628 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.1136e-05 2024-09-10T18:53:16.628 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.628 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.628 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:53:16.628 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.604235+0000", 2024-09-10T18:53:16.628 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.7860000000000002e-06 2024-09-10T18:53:16.628 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.628 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.628 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:53:16.628 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.604284+0000", 2024-09-10T18:53:16.629 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.8758999999999999e-05 2024-09-10T18:53:16.629 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.629 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.629 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:53:16.629 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.604398+0000", 2024-09-10T18:53:16.629 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000114549 2024-09-10T18:53:16.629 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.629 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.630 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 4,6", 2024-09-10T18:53:16.630 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.604524+0000", 2024-09-10T18:53:16.630 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00012582000000000001 2024-09-10T18:53:16.630 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:53:16.630 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:53:16.630 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:53:16.630 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.630 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.630 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:143163 8.4 8:252d470e:::benchmark_data_smithi018_106531_object143162:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e191)", 2024-09-10T18:53:16.631 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:53:16.604258+0000", 2024-09-10T18:53:16.631 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0066330199999999999, 2024-09-10T18:53:16.631 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0069070859999999998, 2024-09-10T18:53:16.631 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:53:16.631 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "waiting for sub ops", 2024-09-10T18:53:16.631 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:53:16.631 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:53:16.631 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:53:16.631 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 143163 2024-09-10T18:53:16.631 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.631 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:53:16.631 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.631 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:53:16.632 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.604258+0000", 2024-09-10T18:53:16.632 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:53:16.632 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.632 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.632 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:53:16.632 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.604258+0000", 2024-09-10T18:53:16.632 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:53:16.632 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.632 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.632 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:53:16.632 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.604257+0000", 2024-09-10T18:53:16.632 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999995 2024-09-10T18:53:16.632 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.633 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.633 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:53:16.633 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.604282+0000", 2024-09-10T18:53:16.633 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.4867000000000001e-05 2024-09-10T18:53:16.633 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.633 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.633 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:53:16.633 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.604289+0000", 2024-09-10T18:53:16.633 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.5710000000000003e-06 2024-09-10T18:53:16.633 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.633 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.633 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:53:16.633 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.604293+0000", 2024-09-10T18:53:16.634 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.2010000000000001e-06 2024-09-10T18:53:16.634 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.634 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.634 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:53:16.634 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.607152+0000", 2024-09-10T18:53:16.634 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.002859084 2024-09-10T18:53:16.634 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.634 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.634 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:53:16.634 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.607321+0000", 2024-09-10T18:53:16.634 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00016860700000000001 2024-09-10T18:53:16.634 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.634 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.634 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 4,6", 2024-09-10T18:53:16.635 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.607452+0000", 2024-09-10T18:53:16.635 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00013119299999999999 2024-09-10T18:53:16.635 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:53:16.635 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:53:16.635 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:53:16.635 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.635 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.635 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_repop(client.24947.0:143166 8.1 e191/158 8:83167ab9:::benchmark_data_smithi018_106531_object143165:head v 191'9052, mlcod=191'9050)", 2024-09-10T18:53:16.635 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:53:16.606936+0000", 2024-09-10T18:53:16.635 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0039547769999999996, 2024-09-10T18:53:16.635 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.004293687, 2024-09-10T18:53:16.635 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:53:16.636 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "started", 2024-09-10T18:53:16.636 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:53:16.636 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.636 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:53:16.636 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.606936+0000", 2024-09-10T18:53:16.636 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:53:16.636 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.636 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.636 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:53:16.636 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.606936+0000", 2024-09-10T18:53:16.636 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:53:16.636 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.636 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.637 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:53:16.637 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.606935+0000", 2024-09-10T18:53:16.637 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:53:16.637 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.637 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.637 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:53:16.637 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.606976+0000", 2024-09-10T18:53:16.637 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.1353999999999997e-05 2024-09-10T18:53:16.637 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.637 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.637 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:53:16.637 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.606989+0000", 2024-09-10T18:53:16.637 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.2479e-05 2024-09-10T18:53:16.638 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.638 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.638 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:53:16.638 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.606995+0000", 2024-09-10T18:53:16.638 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.3400000000000003e-06 2024-09-10T18:53:16.638 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.638 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.638 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:53:16.638 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.607182+0000", 2024-09-10T18:53:16.638 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00018675300000000001 2024-09-10T18:53:16.638 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.638 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.638 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:53:16.639 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.607199+0000", 2024-09-10T18:53:16.639 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.7506000000000001e-05 2024-09-10T18:53:16.639 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:53:16.639 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:53:16.639 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:53:16.639 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.639 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.639 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_repop(client.24947.0:143165 8.2 e191/158 8:4180f578:::benchmark_data_smithi018_106531_object143164:head v 191'8854, mlcod=191'8852)", 2024-09-10T18:53:16.639 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:53:16.607083+0000", 2024-09-10T18:53:16.639 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0038081579999999999, 2024-09-10T18:53:16.639 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0041982419999999996, 2024-09-10T18:53:16.639 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:53:16.639 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "started", 2024-09-10T18:53:16.639 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:53:16.640 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.640 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:53:16.640 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.607083+0000", 2024-09-10T18:53:16.640 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:53:16.640 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.640 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.640 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:53:16.640 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.607083+0000", 2024-09-10T18:53:16.640 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:53:16.640 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.640 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.640 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:53:16.641 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.607082+0000", 2024-09-10T18:53:16.641 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999995 2024-09-10T18:53:16.641 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.641 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.641 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:53:16.641 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.607103+0000", 2024-09-10T18:53:16.641 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.0907000000000002e-05 2024-09-10T18:53:16.641 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.641 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.641 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:53:16.641 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.607112+0000", 2024-09-10T18:53:16.641 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.6549999999999993e-06 2024-09-10T18:53:16.641 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.642 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.642 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:53:16.642 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.607116+0000", 2024-09-10T18:53:16.642 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.6600000000000003e-06 2024-09-10T18:53:16.642 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.642 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.642 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:53:16.642 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.609619+0000", 2024-09-10T18:53:16.642 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0025027529999999999 2024-09-10T18:53:16.642 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.642 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.642 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:53:16.642 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.609640+0000", 2024-09-10T18:53:16.643 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.1027e-05 2024-09-10T18:53:16.643 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:53:16.643 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:53:16.643 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:53:16.643 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.643 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.643 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_repop(client.24947.0:143156 8.9 e191/184 8:904ad254:::benchmark_data_smithi018_106531_object143155:head v 191'8908, mlcod=191'8905)", 2024-09-10T18:53:16.643 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:53:16.607175+0000", 2024-09-10T18:53:16.643 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.003715711, 2024-09-10T18:53:16.643 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0041557859999999999, 2024-09-10T18:53:16.643 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:53:16.643 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "started", 2024-09-10T18:53:16.643 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:53:16.643 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.644 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:53:16.644 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.607175+0000", 2024-09-10T18:53:16.644 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:53:16.644 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.644 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.644 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:53:16.644 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.607175+0000", 2024-09-10T18:53:16.644 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:53:16.644 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.644 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.644 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:53:16.644 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.607174+0000", 2024-09-10T18:53:16.645 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:53:16.645 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.645 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.645 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:53:16.645 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.607200+0000", 2024-09-10T18:53:16.645 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.5687999999999999e-05 2024-09-10T18:53:16.645 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.645 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.645 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:53:16.645 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.607209+0000", 2024-09-10T18:53:16.645 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.6000000000000007e-06 2024-09-10T18:53:16.645 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.646 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.646 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:53:16.646 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.607214+0000", 2024-09-10T18:53:16.646 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.9479999999999999e-06 2024-09-10T18:53:16.646 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.646 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.646 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:53:16.646 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.609244+0000", 2024-09-10T18:53:16.646 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0020308029999999999 2024-09-10T18:53:16.646 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:16.646 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:16.646 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:53:16.646 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:16.609266+0000", 2024-09-10T18:53:16.647 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.1452999999999999e-05 2024-09-10T18:53:16.647 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:53:16.647 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:53:16.647 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:53:16.647 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:53:16.647 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-10T18:53:16.647 INFO:teuthology.orchestra.run.smithi103.stdout: "num_ops": 7 2024-09-10T18:53:16.647 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T18:53:16.676 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:16.714 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:16.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:16.715 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:16.772 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:53:16 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:53:16.817 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:16.860 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:16.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:16.861 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:16.901 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T18:53:16.957 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:53:16.962 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:17.020 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:17.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:17.021 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:17.122 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:17.165 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:17.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:17.168 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:17.213 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 88 16 143976 143960 102.219 81.375 0.0104597 0.00977866 2024-09-10T18:53:17.268 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:17.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:17.306 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:17.307 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:17.346 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-10T18:53:17.408 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:17.421 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:17 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:17.421 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:17 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:17.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:17 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:17.437 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:17.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:17.438 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:17.539 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:17.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:17.580 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:17.580 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:17.681 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:17.736 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:17.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:17.737 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:17.837 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:17.841 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:53:17.873 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:17.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:17.874 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:17.975 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:18.014 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:18.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:18.016 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:18.117 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:18.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:18.156 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:18.156 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:18.213 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 89 16 145272 145256 101.981 81 0.00403279 0.00980148 2024-09-10T18:53:18.257 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:18.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:18 smithi103 ceph-mon[33163]: pgmap v345: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 8.7 GiB data, 28 GiB used, 598 GiB / 626 GiB avail; 1023 B/s rd, 80 MiB/s wr, 1.28k op/s; 17833/429165 objects misplaced (4.155%) 2024-09-10T18:53:18.294 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:18.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:18.295 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:18.395 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:18.432 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:18.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:18.433 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:18.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:18 smithi018 ceph-mon[29336]: pgmap v345: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 8.7 GiB data, 28 GiB used, 598 GiB / 626 GiB avail; 1023 B/s rd, 80 MiB/s wr, 1.28k op/s; 17833/429165 objects misplaced (4.155%) 2024-09-10T18:53:18.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:18 smithi018 ceph-mon[40196]: pgmap v345: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 8.7 GiB data, 28 GiB used, 598 GiB / 626 GiB avail; 1023 B/s rd, 80 MiB/s wr, 1.28k op/s; 17833/429165 objects misplaced (4.155%) 2024-09-10T18:53:18.534 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:18.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:18.569 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:18.569 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:18.670 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:18.714 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:18.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:18.715 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:18.813 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:53:18.816 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:18.866 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:18.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:18.868 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:18.969 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:19.010 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:19.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:19.011 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:19.112 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:19.155 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is set 2024-09-10T18:53:19.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:19.165 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:19.165 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:19.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:19 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/592539348' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:53:19.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:19 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/592539348' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:53:19.214 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Total time run: 90.0051 2024-09-10T18:53:19.214 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Total writes made: 146627 2024-09-10T18:53:19.214 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Write size: 65536 2024-09-10T18:53:19.214 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Object size: 65536 2024-09-10T18:53:19.214 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Bandwidth (MB/sec): 101.819 2024-09-10T18:53:19.214 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Stddev Bandwidth: 37.6912 2024-09-10T18:53:19.214 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Max bandwidth (MB/sec): 244.062 2024-09-10T18:53:19.214 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Min bandwidth (MB/sec): 0 2024-09-10T18:53:19.214 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Average IOPS: 1629 2024-09-10T18:53:19.214 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Stddev IOPS: 603.06 2024-09-10T18:53:19.215 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Max IOPS: 3905 2024-09-10T18:53:19.215 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Min IOPS: 0 2024-09-10T18:53:19.215 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Average Latency(s): 0.00981753 2024-09-10T18:53:19.215 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Stddev Latency(s): 0.0222507 2024-09-10T18:53:19.215 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Max latency(s): 2.09392 2024-09-10T18:53:19.215 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Min latency(s): 0.00115328 2024-09-10T18:53:19.218 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Cleaning up (deleting benchmark objects) 2024-09-10T18:53:19.267 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:19.299 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:19.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:19.301 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:19.402 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:19.443 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:19.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:19.444 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:19.448 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:19 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/592539348' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:53:19.545 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:19.582 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:19.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:19.583 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:19.684 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:19.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:19.729 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:19.729 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:19.830 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:19.866 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:19.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:19.867 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:19.968 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:19.999 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:20.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:20.000 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:20.101 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:20.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:20.137 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:20.137 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:20.139 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:20 smithi103 ceph-mon[33163]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:53:20.139 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:20 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/592539348' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:53:20.139 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:20 smithi103 ceph-mon[33163]: osdmap e192: 8 total, 8 up, 7 in 2024-09-10T18:53:20.139 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:20 smithi103 ceph-mon[33163]: pgmap v347: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 8.7 GiB data, 28 GiB used, 598 GiB / 626 GiB avail; 1023 B/s rd, 80 MiB/s wr, 1.28k op/s; 17833/429165 objects misplaced (4.155%) 2024-09-10T18:53:20.139 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:20 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd reweightn", "weights": "{'5':'45843'}"}]: dispatch 2024-09-10T18:53:20.139 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:20 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd reweightn", "weights": "{'5':'45843'}"}]: dispatch 2024-09-10T18:53:20.163 INFO:teuthology.orchestra.run.smithi018.stdout:moved 6 / 504 (1.19048%) 2024-09-10T18:53:20.163 INFO:teuthology.orchestra.run.smithi018.stdout:avg 72 2024-09-10T18:53:20.164 INFO:teuthology.orchestra.run.smithi018.stdout:stddev 6.04743 -> 8.29802 (expected baseline 7.85584) 2024-09-10T18:53:20.164 INFO:teuthology.orchestra.run.smithi018.stdout:min osd.5 with 59 -> 53 pgs (0.819444 -> 0.736111 * mean) 2024-09-10T18:53:20.164 INFO:teuthology.orchestra.run.smithi018.stdout:max osd.3 with 78 -> 79 pgs (1.08333 -> 1.09722 * mean) 2024-09-10T18:53:20.164 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:53:20.164 INFO:teuthology.orchestra.run.smithi018.stdout:oload 110 2024-09-10T18:53:20.164 INFO:teuthology.orchestra.run.smithi018.stdout:max_change 0.05 2024-09-10T18:53:20.164 INFO:teuthology.orchestra.run.smithi018.stdout:max_change_osds 4 2024-09-10T18:53:20.164 INFO:teuthology.orchestra.run.smithi018.stdout:average_utilization 0.0449 2024-09-10T18:53:20.164 INFO:teuthology.orchestra.run.smithi018.stdout:overload_utilization 0.0494 2024-09-10T18:53:20.164 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5 weight 0.7495 -> 0.6995 2024-09-10T18:53:20.239 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:20.270 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:20.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:20.271 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:20.372 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:20.393 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:20.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:20.394 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:20.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:20 smithi018 ceph-mon[40196]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:53:20.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:20 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/592539348' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:53:20.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:20 smithi018 ceph-mon[40196]: osdmap e192: 8 total, 8 up, 7 in 2024-09-10T18:53:20.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:20 smithi018 ceph-mon[40196]: pgmap v347: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 8.7 GiB data, 28 GiB used, 598 GiB / 626 GiB avail; 1023 B/s rd, 80 MiB/s wr, 1.28k op/s; 17833/429165 objects misplaced (4.155%) 2024-09-10T18:53:20.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:20 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd reweightn", "weights": "{'5':'45843'}"}]: dispatch 2024-09-10T18:53:20.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:20 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd reweightn", "weights": "{'5':'45843'}"}]: dispatch 2024-09-10T18:53:20.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:20 smithi018 ceph-mon[29336]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:53:20.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:20 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/592539348' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:53:20.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:20 smithi018 ceph-mon[29336]: osdmap e192: 8 total, 8 up, 7 in 2024-09-10T18:53:20.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:20 smithi018 ceph-mon[29336]: pgmap v347: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 8.7 GiB data, 28 GiB used, 598 GiB / 626 GiB avail; 1023 B/s rd, 80 MiB/s wr, 1.28k op/s; 17833/429165 objects misplaced (4.155%) 2024-09-10T18:53:20.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:20 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd reweightn", "weights": "{'5':'45843'}"}]: dispatch 2024-09-10T18:53:20.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:20 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd reweightn", "weights": "{'5':'45843'}"}]: dispatch 2024-09-10T18:53:20.494 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:20.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:20.516 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:20.516 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:20.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:53:20 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:53:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:53:20.618 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:20.640 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:20.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:20.641 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:20.710 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T18:53:20.710 INFO:teuthology.orchestra.run.smithi103.stdout: "ops": [], 2024-09-10T18:53:20.710 INFO:teuthology.orchestra.run.smithi103.stdout: "complaint_time": 30, 2024-09-10T18:53:20.710 INFO:teuthology.orchestra.run.smithi103.stdout: "num_blocked_ops": 0 2024-09-10T18:53:20.710 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T18:53:20.742 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:20.764 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:20.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:20.765 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:20.866 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:20.889 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:20.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:20.890 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:20.990 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:21.014 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:21.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:21.015 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:21.116 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:21.140 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:21.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:21.141 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:21.242 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:21.267 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:21.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:21.268 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:21.369 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:21.394 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:21.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:21.395 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:21.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:21 smithi018 ceph-mon[29336]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:53:21.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:21 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd reweightn", "weights": "{'5':'45843'}"}]': finished 2024-09-10T18:53:21.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:21 smithi018 ceph-mon[29336]: osdmap e193: 8 total, 8 up, 7 in 2024-09-10T18:53:21.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:21 smithi018 ceph-mon[40196]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:53:21.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:21 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd reweightn", "weights": "{'5':'45843'}"}]': finished 2024-09-10T18:53:21.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:21 smithi018 ceph-mon[40196]: osdmap e193: 8 total, 8 up, 7 in 2024-09-10T18:53:21.445 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:53:20 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:53:20] "GET /metrics HTTP/1.1" 200 224058 "" "Prometheus/2.43.0" 2024-09-10T18:53:21.496 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:21.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:21 smithi103 ceph-mon[33163]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:53:21.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:21 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd reweightn", "weights": "{'5':'45843'}"}]': finished 2024-09-10T18:53:21.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:21 smithi103 ceph-mon[33163]: osdmap e193: 8 total, 8 up, 7 in 2024-09-10T18:53:21.524 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:21.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:21.525 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:21.625 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:21.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:21.652 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:21.652 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:21.753 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:21.780 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:21.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:21.781 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:21.881 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:21.908 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:21.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:21.910 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:22.010 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:22.036 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:22.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:22.037 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:22.138 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:22.162 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:22.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:22.163 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:22.264 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:22.301 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:22.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:22.302 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:22.402 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:22.436 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:22.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:22.437 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:22.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:22 smithi018 ceph-mon[40196]: osdmap e194: 8 total, 8 up, 7 in 2024-09-10T18:53:22.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:22 smithi018 ceph-mon[40196]: pgmap v350: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 8.7 GiB data, 28 GiB used, 598 GiB / 626 GiB avail; 853 B/s rd, 69 MiB/s wr, 1.12k op/s; 17833/429627 objects misplaced (4.151%) 2024-09-10T18:53:22.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:22 smithi018 ceph-mon[40196]: osdmap e195: 8 total, 8 up, 7 in 2024-09-10T18:53:22.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:22 smithi018 ceph-mon[29336]: osdmap e194: 8 total, 8 up, 7 in 2024-09-10T18:53:22.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:22 smithi018 ceph-mon[29336]: pgmap v350: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 8.7 GiB data, 28 GiB used, 598 GiB / 626 GiB avail; 853 B/s rd, 69 MiB/s wr, 1.12k op/s; 17833/429627 objects misplaced (4.151%) 2024-09-10T18:53:22.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:22 smithi018 ceph-mon[29336]: osdmap e195: 8 total, 8 up, 7 in 2024-09-10T18:53:22.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:22 smithi103 ceph-mon[33163]: osdmap e194: 8 total, 8 up, 7 in 2024-09-10T18:53:22.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:22 smithi103 ceph-mon[33163]: pgmap v350: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 8.7 GiB data, 28 GiB used, 598 GiB / 626 GiB avail; 853 B/s rd, 69 MiB/s wr, 1.12k op/s; 17833/429627 objects misplaced (4.151%) 2024-09-10T18:53:22.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:22 smithi103 ceph-mon[33163]: osdmap e195: 8 total, 8 up, 7 in 2024-09-10T18:53:22.538 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:22.573 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:22.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:22.574 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:22.612 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-10T18:53:22.675 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:22.709 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:22.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:22.710 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:22.811 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:22.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:22.856 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:22.856 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:22.957 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:22.996 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:22.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:22.997 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:23.068 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:53:23.098 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:23.155 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:23.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:23.156 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:23.257 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:23.290 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:23.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:23.291 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:23.392 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:23.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:23.431 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:23.432 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:23.436 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:23 smithi018 ceph-mon[40196]: 8.0 starting backfill to osd.4 from (0'0,0'0] MAX to 192'9159 2024-09-10T18:53:23.436 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:23 smithi018 ceph-mon[29336]: 8.0 starting backfill to osd.4 from (0'0,0'0] MAX to 192'9159 2024-09-10T18:53:23.471 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:23 smithi103 ceph-mon[33163]: 8.0 starting backfill to osd.4 from (0'0,0'0] MAX to 192'9159 2024-09-10T18:53:23.532 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:23.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:23.609 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:23.609 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:23.710 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:23.760 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:23.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:23.762 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:23.863 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:23.904 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:23.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:23.904 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:24.005 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:24.023 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-10T18:53:24.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:24.042 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:24.144 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:24.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:24.178 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:24.178 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:24.279 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:24.309 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:24 smithi018 ceph-mon[29336]: pgmap v352: 168 pgs: 1 remapped+peering, 2 active+remapped+backfill_toofull, 165 active+clean; 8.9 GiB data, 29 GiB used, 597 GiB / 626 GiB avail; 1.5 KiB/s rd, 59 MiB/s wr, 1.28k op/s; 18165/436392 objects misplaced (4.163%); 0 B/s, 0 objects/s recovering 2024-09-10T18:53:24.309 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:24 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:53:24.309 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:24 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:53:24.309 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:24 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:53:24.310 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:24 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:53:24.310 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:24 smithi018 ceph-mon[40196]: pgmap v352: 168 pgs: 1 remapped+peering, 2 active+remapped+backfill_toofull, 165 active+clean; 8.9 GiB data, 29 GiB used, 597 GiB / 626 GiB avail; 1.5 KiB/s rd, 59 MiB/s wr, 1.28k op/s; 18165/436392 objects misplaced (4.163%); 0 B/s, 0 objects/s recovering 2024-09-10T18:53:24.310 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:24 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:53:24.310 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:24 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:53:24.310 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:24 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:53:24.311 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:24 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:53:24.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:24.328 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:24.328 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:24.392 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:53:24.393 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:53:24.429 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:24.435 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:53:24.436 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:53:24.470 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:53:24.471 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:53:24.475 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:53:24.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:24.480 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:24.481 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:24.486 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:24 smithi103 ceph-mon[33163]: pgmap v352: 168 pgs: 1 remapped+peering, 2 active+remapped+backfill_toofull, 165 active+clean; 8.9 GiB data, 29 GiB used, 597 GiB / 626 GiB avail; 1.5 KiB/s rd, 59 MiB/s wr, 1.28k op/s; 18165/436392 objects misplaced (4.163%); 0 B/s, 0 objects/s recovering 2024-09-10T18:53:24.487 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:24 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:53:24.487 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:24 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:53:24.487 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:24 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:53:24.487 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:24 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:53:24.507 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:53:24.508 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:53:24.542 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:53:24.543 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:53:24.575 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:53:24.575 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:53:24.582 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:24.602 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:53:24.602 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:53:24.618 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:24.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:24.619 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:24.636 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:53:24.636 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:53:24.720 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:24.752 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:24.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:24.753 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:24.854 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:24.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:24.895 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:24.895 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:24.977 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T18:53:24.977 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:53:24.997 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:25.031 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:25.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:25.032 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:25.132 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:25.173 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:25.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:25.174 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:25.275 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:25.309 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:25.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:25.310 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:25.412 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:25.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:25 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1944156338' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:53:25.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:25 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:53:25.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:25 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1944156338' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:53:25.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:25 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:53:25.445 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:25.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:25.446 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:25.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:25 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1944156338' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:53:25.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:25 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:53:25.547 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:25.582 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:25.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:25.584 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:25.655 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-10T18:53:25.684 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:25.712 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:25.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:25.713 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:25.814 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:25.846 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:25.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:25.847 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:25.947 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:25.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:25.996 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:25.996 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:26.087 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:53:26.096 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:26.143 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:26.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:26.145 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:26.246 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:26.283 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:26.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:26.283 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:26.384 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:26.423 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:26.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:26.423 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:26.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:26 smithi018 ceph-mon[29336]: pgmap v353: 168 pgs: 1 remapped+peering, 2 active+remapped+backfill_toofull, 165 active+clean; 8.9 GiB data, 29 GiB used, 597 GiB / 626 GiB avail; 1023 B/s rd, 39 MiB/s wr, 855 op/s; 18165/436392 objects misplaced (4.163%); 0 B/s, 0 objects/s recovering 2024-09-10T18:53:26.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:26 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:53:26.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:26 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:53:26.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:26 smithi018 ceph-mon[40196]: pgmap v353: 168 pgs: 1 remapped+peering, 2 active+remapped+backfill_toofull, 165 active+clean; 8.9 GiB data, 29 GiB used, 597 GiB / 626 GiB avail; 1023 B/s rd, 39 MiB/s wr, 855 op/s; 18165/436392 objects misplaced (4.163%); 0 B/s, 0 objects/s recovering 2024-09-10T18:53:26.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:26 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:53:26.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:26 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:53:26.483 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:26 smithi103 ceph-mon[33163]: pgmap v353: 168 pgs: 1 remapped+peering, 2 active+remapped+backfill_toofull, 165 active+clean; 8.9 GiB data, 29 GiB used, 597 GiB / 626 GiB avail; 1023 B/s rd, 39 MiB/s wr, 855 op/s; 18165/436392 objects misplaced (4.163%); 0 B/s, 0 objects/s recovering 2024-09-10T18:53:26.483 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:26 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:53:26.483 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:26 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:53:26.525 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:26.559 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:26.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:26.560 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:26.661 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:26.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:26.697 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:26.697 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:26.772 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:53:26 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:53:26.798 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:26.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:26.842 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:26.842 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:26.943 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:26.980 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:26.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:26.981 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:27.082 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:27.118 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:27.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:27.119 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:27.220 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:27.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:27.253 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:27.253 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:27.354 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:27.387 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:27 smithi018 ceph-mon[40196]: osdmap e196: 8 total, 8 up, 7 in 2024-09-10T18:53:27.388 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:27 smithi018 ceph-mon[29336]: osdmap e196: 8 total, 8 up, 7 in 2024-09-10T18:53:27.390 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:27.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:27.391 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:27.492 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:27.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:27 smithi103 ceph-mon[33163]: osdmap e196: 8 total, 8 up, 7 in 2024-09-10T18:53:27.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:27.542 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:27.542 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:27.642 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:27.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:27.681 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:27.681 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:27.782 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:27.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:27.837 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:27.837 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:27.938 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:27.970 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:27.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:27.972 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:28.073 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:28.102 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T18:53:28.102 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:53:28.108 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T18:53:28.110 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:28.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:28.111 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:28.212 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:28.231 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is unset 2024-09-10T18:53:28.247 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:28.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:28.249 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:28.349 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:28.381 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:28.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:28.382 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:28.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:28 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:28.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:28 smithi018 ceph-mon[40196]: pgmap v355: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 8.5 GiB data, 28 GiB used, 598 GiB / 626 GiB avail; 1.8 KiB/s rd, 37 MiB/s wr, 1.71k op/s; 25532/420015 objects misplaced (6.079%); 6.9 MiB/s, 111 objects/s recovering 2024-09-10T18:53:28.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:28 smithi018 ceph-mon[40196]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-10T18:53:28.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:28 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1375331315' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:53:28.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:28 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:53:28.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:28 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:28.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:28 smithi018 ceph-mon[29336]: pgmap v355: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 8.5 GiB data, 28 GiB used, 598 GiB / 626 GiB avail; 1.8 KiB/s rd, 37 MiB/s wr, 1.71k op/s; 25532/420015 objects misplaced (6.079%); 6.9 MiB/s, 111 objects/s recovering 2024-09-10T18:53:28.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:28 smithi018 ceph-mon[29336]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-10T18:53:28.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:28 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1375331315' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:53:28.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:28 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:53:28.479 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-10T18:53:28.480 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T18:53:28.480 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_1 2024-09-10T18:53:28.481 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T18:53:28.482 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:28.502 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:28 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:28.502 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:28 smithi103 ceph-mon[33163]: pgmap v355: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 8.5 GiB data, 28 GiB used, 598 GiB / 626 GiB avail; 1.8 KiB/s rd, 37 MiB/s wr, 1.71k op/s; 25532/420015 objects misplaced (6.079%); 6.9 MiB/s, 111 objects/s recovering 2024-09-10T18:53:28.502 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:28 smithi103 ceph-mon[33163]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-10T18:53:28.502 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:28 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1375331315' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:53:28.503 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:28 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:53:28.510 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:28.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:28.511 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:28.612 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:28.649 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:28.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:28.650 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:28.751 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:28.777 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:28.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:28.778 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:28.826 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-10T18:53:28.879 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:28.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:28.938 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:28.938 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:29.039 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:29.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:29.081 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:29.081 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:29.182 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:29.216 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:29.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:29.217 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:29.226 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:53:29.318 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:29.350 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:29.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:29.351 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:29.452 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:29.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:29.488 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:29.488 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:29.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:29 smithi103 ceph-mon[33163]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:53:29.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:29 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:53:29.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:29 smithi103 ceph-mon[33163]: osdmap e197: 8 total, 8 up, 7 in 2024-09-10T18:53:29.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:29 smithi103 ceph-mon[33163]: 6.10 scrub starts 2024-09-10T18:53:29.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:29 smithi103 ceph-mon[33163]: 2.0 scrub starts 2024-09-10T18:53:29.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:29 smithi103 ceph-mon[33163]: 2.0 scrub ok 2024-09-10T18:53:29.589 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:29.623 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:29.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:29.624 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:29.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:29 smithi018 ceph-mon[40196]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:53:29.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:29 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:53:29.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:29 smithi018 ceph-mon[40196]: osdmap e197: 8 total, 8 up, 7 in 2024-09-10T18:53:29.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:29 smithi018 ceph-mon[40196]: 6.10 scrub starts 2024-09-10T18:53:29.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:29 smithi018 ceph-mon[40196]: 2.0 scrub starts 2024-09-10T18:53:29.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:29 smithi018 ceph-mon[40196]: 2.0 scrub ok 2024-09-10T18:53:29.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:29 smithi018 ceph-mon[29336]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:53:29.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:29 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:53:29.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:29 smithi018 ceph-mon[29336]: osdmap e197: 8 total, 8 up, 7 in 2024-09-10T18:53:29.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:29 smithi018 ceph-mon[29336]: 6.10 scrub starts 2024-09-10T18:53:29.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:29 smithi018 ceph-mon[29336]: 2.0 scrub starts 2024-09-10T18:53:29.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:29 smithi018 ceph-mon[29336]: 2.0 scrub ok 2024-09-10T18:53:29.724 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:29.753 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:29.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:29.754 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:29.855 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:29.892 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:29.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:29.893 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:29.993 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:30.026 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:30.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:30.027 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:30.128 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:30.150 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:30.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:30.151 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:30.252 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:30.291 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:30.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:30.292 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:30.392 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:30.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:30.429 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:30.429 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:30.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:30 smithi103 ceph-mon[33163]: pgmap v357: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 8.5 GiB data, 28 GiB used, 598 GiB / 626 GiB avail; 730 B/s rd, 0 B/s wr, 779 op/s; 25532/420015 objects misplaced (6.079%); 6.0 MiB/s, 95 objects/s recovering 2024-09-10T18:53:30.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:30 smithi103 ceph-mon[33163]: 6.4 scrub starts 2024-09-10T18:53:30.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:30 smithi103 ceph-mon[33163]: 6.8 scrub starts 2024-09-10T18:53:30.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:30 smithi103 ceph-mon[33163]: 5.3 scrub starts 2024-09-10T18:53:30.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:30 smithi103 ceph-mon[33163]: 8.4 scrub starts 2024-09-10T18:53:30.523 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:53:30 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:53:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:53:30.530 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:30.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:30.561 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:30.561 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:30.662 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:30.693 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:30.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:30.694 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:30.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:30 smithi018 ceph-mon[29336]: pgmap v357: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 8.5 GiB data, 28 GiB used, 598 GiB / 626 GiB avail; 730 B/s rd, 0 B/s wr, 779 op/s; 25532/420015 objects misplaced (6.079%); 6.0 MiB/s, 95 objects/s recovering 2024-09-10T18:53:30.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:30 smithi018 ceph-mon[29336]: 6.4 scrub starts 2024-09-10T18:53:30.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:30 smithi018 ceph-mon[29336]: 6.8 scrub starts 2024-09-10T18:53:30.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:30 smithi018 ceph-mon[29336]: 5.3 scrub starts 2024-09-10T18:53:30.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:30 smithi018 ceph-mon[29336]: 8.4 scrub starts 2024-09-10T18:53:30.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:30 smithi018 ceph-mon[40196]: pgmap v357: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 8.5 GiB data, 28 GiB used, 598 GiB / 626 GiB avail; 730 B/s rd, 0 B/s wr, 779 op/s; 25532/420015 objects misplaced (6.079%); 6.0 MiB/s, 95 objects/s recovering 2024-09-10T18:53:30.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:30 smithi018 ceph-mon[40196]: 6.4 scrub starts 2024-09-10T18:53:30.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:30 smithi018 ceph-mon[40196]: 6.8 scrub starts 2024-09-10T18:53:30.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:30 smithi018 ceph-mon[40196]: 5.3 scrub starts 2024-09-10T18:53:30.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:30 smithi018 ceph-mon[40196]: 8.4 scrub starts 2024-09-10T18:53:30.795 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:30.826 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:30.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:30.827 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:30.928 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:30.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:30.971 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:30.972 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:31.072 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:31.105 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:31.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:31.106 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:31.113 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T18:53:31.113 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:53:31.207 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:31.226 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:53:30 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:53:30] "GET /metrics HTTP/1.1" 200 224061 "" "Prometheus/2.43.0" 2024-09-10T18:53:31.248 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:31.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:31.249 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:31.350 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:31.390 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:31.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:31.391 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:31.491 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:31.517 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:31 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:53:31.517 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:31 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:53:31.529 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:31.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:31.530 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:31.624 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:31 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:53:31.624 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:31 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:53:31.624 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:31 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:53:31.625 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:31 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:53:31.632 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:31.701 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:31.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:31.702 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:31.735 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:53:31.768 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:53:31.804 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:31.838 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-10T18:53:31.857 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:31.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:31.858 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:31.960 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:31.996 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:31.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:31.997 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:32.098 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:32.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:32.145 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:32.145 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:32.228 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:32 smithi103 ceph-mon[33163]: pgmap v358: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 8.5 GiB data, 28 GiB used, 598 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 733 op/s; 25532/418809 objects misplaced (6.096%); 5.2 MiB/s, 83 objects/s recovering 2024-09-10T18:53:32.247 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:32.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:32.297 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:32.297 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:32.303 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:53:32.398 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:32.437 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:32 smithi018 ceph-mon[40196]: pgmap v358: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 8.5 GiB data, 28 GiB used, 598 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 733 op/s; 25532/418809 objects misplaced (6.096%); 5.2 MiB/s, 83 objects/s recovering 2024-09-10T18:53:32.437 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:32 smithi018 ceph-mon[40196]: osdmap e198: 8 total, 8 up, 7 in 2024-09-10T18:53:32.437 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:32 smithi018 ceph-mon[40196]: 8.8 scrub starts 2024-09-10T18:53:32.437 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:32 smithi018 ceph-mon[29336]: pgmap v358: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 8.5 GiB data, 28 GiB used, 598 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 733 op/s; 25532/418809 objects misplaced (6.096%); 5.2 MiB/s, 83 objects/s recovering 2024-09-10T18:53:32.437 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:32 smithi018 ceph-mon[29336]: osdmap e198: 8 total, 8 up, 7 in 2024-09-10T18:53:32.438 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:32 smithi018 ceph-mon[29336]: 8.8 scrub starts 2024-09-10T18:53:32.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:32.442 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:32.442 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:32.485 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:32 smithi103 ceph-mon[33163]: osdmap e198: 8 total, 8 up, 7 in 2024-09-10T18:53:32.485 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:32 smithi103 ceph-mon[33163]: 8.8 scrub starts 2024-09-10T18:53:32.544 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:32.586 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:32.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:32.587 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:32.688 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:32.727 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:32.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:32.728 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:32.829 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:32.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:32.872 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:32.872 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:32.973 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:33.016 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:33.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:33.018 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:33.118 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:33.154 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:33.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:33.155 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:33.256 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:33.294 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:33.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:33.296 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:33.397 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:33.433 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:33.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:33.434 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:33.493 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:33 smithi103 ceph-mon[33163]: 2.15 scrub starts 2024-09-10T18:53:33.493 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:33 smithi103 ceph-mon[33163]: 2.15 scrub ok 2024-09-10T18:53:33.493 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:33 smithi103 ceph-mon[33163]: 4.8 scrub starts 2024-09-10T18:53:33.493 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:33 smithi103 ceph-mon[33163]: 8.e scrub starts 2024-09-10T18:53:33.534 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:33.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:33.573 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:33.573 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:33.674 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:33.674 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-10T18:53:33.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:33 smithi018 ceph-mon[40196]: 2.15 scrub starts 2024-09-10T18:53:33.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:33 smithi018 ceph-mon[40196]: 2.15 scrub ok 2024-09-10T18:53:33.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:33 smithi018 ceph-mon[40196]: 4.8 scrub starts 2024-09-10T18:53:33.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:33 smithi018 ceph-mon[40196]: 8.e scrub starts 2024-09-10T18:53:33.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:33 smithi018 ceph-mon[29336]: 2.15 scrub starts 2024-09-10T18:53:33.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:33 smithi018 ceph-mon[29336]: 2.15 scrub ok 2024-09-10T18:53:33.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:33 smithi018 ceph-mon[29336]: 4.8 scrub starts 2024-09-10T18:53:33.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:33 smithi018 ceph-mon[29336]: 8.e scrub starts 2024-09-10T18:53:33.706 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:33.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:33.707 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:33.808 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:33.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:33.849 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:33.849 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:33.950 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:33.998 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:33.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:33.999 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:34.100 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:34.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:34.137 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:34.137 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:34.238 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:34.259 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T18:53:34.259 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:53:34.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:34.274 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:34.274 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:34.375 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:34.409 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:34.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:34.410 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:34.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:34 smithi018 ceph-mon[29336]: pgmap v360: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 8.1 GiB data, 27 GiB used, 599 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.65k op/s; 23041/401010 objects misplaced (5.746%); 18 MiB/s, 292 objects/s recovering 2024-09-10T18:53:34.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:34 smithi018 ceph-mon[29336]: 6.1c scrub starts 2024-09-10T18:53:34.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:34 smithi018 ceph-mon[40196]: pgmap v360: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 8.1 GiB data, 27 GiB used, 599 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.65k op/s; 23041/401010 objects misplaced (5.746%); 18 MiB/s, 292 objects/s recovering 2024-09-10T18:53:34.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:34 smithi018 ceph-mon[40196]: 6.1c scrub starts 2024-09-10T18:53:34.510 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:34.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:34 smithi103 ceph-mon[33163]: pgmap v360: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 8.1 GiB data, 27 GiB used, 599 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.65k op/s; 23041/401010 objects misplaced (5.746%); 18 MiB/s, 292 objects/s recovering 2024-09-10T18:53:34.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:34 smithi103 ceph-mon[33163]: 6.1c scrub starts 2024-09-10T18:53:34.538 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:34.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:34.539 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:34.640 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:34.681 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:34.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:34.682 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:34.783 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:34.823 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:34.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:34.824 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:34.925 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:34.957 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:34.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:34.958 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:35.059 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:35.097 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:35.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:35.098 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:35.198 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:53:35.198 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:53:35.199 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:35.223 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:53:35.224 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:53:35.234 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:35.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:35.235 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:35.253 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:53:35.253 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:53:35.267 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-10T18:53:35.279 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:53:35.279 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:53:35.308 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:53:35.308 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:53:35.336 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:35.339 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:53:35.339 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:53:35.365 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:53:35.365 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:53:35.366 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:35.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:35.366 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:35.399 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:53:35.399 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:53:35.409 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:35 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3255816817' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:53:35.409 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:35 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3255816817' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:53:35.468 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:35.499 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:35.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:35.500 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:35.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:35 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3255816817' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:53:35.602 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:35.656 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:35.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:35.657 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:35.758 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:35.790 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:53:35.805 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:35.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:35.806 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:35.890 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:53:35.907 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:35.947 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:35.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:35.948 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:36.049 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:36.086 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:36.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:36.087 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:36.188 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:36.227 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:53:36.231 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-09-10T18:53:36.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:36.233 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:36.233 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:36.334 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:36.365 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:36.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:36.366 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:36.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:36 smithi018 ceph-mon[40196]: pgmap v361: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 8.1 GiB data, 27 GiB used, 599 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 792 op/s; 23041/401010 objects misplaced (5.746%); 11 MiB/s, 171 objects/s recovering 2024-09-10T18:53:36.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:36 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:53:36.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:36 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:53:36.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:36 smithi018 ceph-mon[29336]: pgmap v361: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 8.1 GiB data, 27 GiB used, 599 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 792 op/s; 23041/401010 objects misplaced (5.746%); 11 MiB/s, 171 objects/s recovering 2024-09-10T18:53:36.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:36 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:53:36.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:36 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:53:36.467 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:36.497 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:36 smithi103 ceph-mon[33163]: pgmap v361: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 8.1 GiB data, 27 GiB used, 599 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 792 op/s; 23041/401010 objects misplaced (5.746%); 11 MiB/s, 171 objects/s recovering 2024-09-10T18:53:36.497 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:36 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:53:36.498 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:36 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:53:36.516 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:36.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:36.517 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:36.618 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:36.652 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:36.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:36.653 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:36.754 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:36.772 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:53:36 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:53:36.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:36.790 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:36.791 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:36.891 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:36.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:36.927 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:36.928 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:37.028 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:37.064 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:37.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:37.066 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:37.166 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:37.204 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:37.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:37.205 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:37.306 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:37.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:37.352 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:37.353 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:37.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:37 smithi018 ceph-mon[40196]: from='client.24961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:53:37.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:37 smithi018 ceph-mon[40196]: 2.4 scrub starts 2024-09-10T18:53:37.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:37 smithi018 ceph-mon[40196]: 2.4 scrub ok 2024-09-10T18:53:37.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:37 smithi018 ceph-mon[40196]: osdmap e199: 8 total, 8 up, 7 in 2024-09-10T18:53:37.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:37 smithi018 ceph-mon[40196]: 4.15 scrub starts 2024-09-10T18:53:37.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:37 smithi018 ceph-mon[29336]: from='client.24961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:53:37.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:37 smithi018 ceph-mon[29336]: 2.4 scrub starts 2024-09-10T18:53:37.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:37 smithi018 ceph-mon[29336]: 2.4 scrub ok 2024-09-10T18:53:37.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:37 smithi018 ceph-mon[29336]: osdmap e199: 8 total, 8 up, 7 in 2024-09-10T18:53:37.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:37 smithi018 ceph-mon[29336]: 4.15 scrub starts 2024-09-10T18:53:37.453 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:37.496 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:37.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:37.497 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:37.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:37 smithi103 ceph-mon[33163]: from='client.24961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:53:37.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:37 smithi103 ceph-mon[33163]: 2.4 scrub starts 2024-09-10T18:53:37.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:37 smithi103 ceph-mon[33163]: 2.4 scrub ok 2024-09-10T18:53:37.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:37 smithi103 ceph-mon[33163]: osdmap e199: 8 total, 8 up, 7 in 2024-09-10T18:53:37.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:37 smithi103 ceph-mon[33163]: 4.15 scrub starts 2024-09-10T18:53:37.598 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:37.630 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:37.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:37.632 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:37.732 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:37.768 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:37.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:37.769 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:37.870 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:37.909 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:37.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:37.910 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:37.945 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":361,"stamp":"2024-09-10T18:53:35.170933+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8747848484,"num_objects":133670,"num_object_clones":0,"num_object_copies":401010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":23041,"num_objects_unfound":0,"num_objects_dirty":133670,"num_whiteouts":0,"num_read":1218,"num_read_kb":980,"num_write":160221,"num_write_kb":9384320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2136,"num_bytes_recovered":133934986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71289,"ondisk_log_size":71289,"up":504,"acting":504,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":507,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":27999792,"kb_used_data":25847408,"kb_used_omap":11,"kb_used_meta":2152116,"kb_avail":628216272,"statfs":{"total":671965249536,"available":643293462528,"internally_reserved":0,"allocated":26467745792,"data_stored":26465000021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11928,"internal_metadata":2203767144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-415170560,"num_objects":-6335,"num_object_clones":0,"num_object_copies":-19005,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-2491,"num_objects_unfound":0,"num_objects_dirty":-6335,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":6335,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1370,"num_bytes_recovered":89784320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002396"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":185,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.237306+0000","last_change":"2024-09-10T18:52:55.776643+0000","last_active":"2024-09-10T18:53:28.237306+0000","last_peered":"2024-09-10T18:53:28.237306+0000","last_clean":"2024-09-10T18:53:28.237306+0000","last_became_active":"2024-09-10T18:45:24.265488+0000","last_became_peered":"2024-09-10T18:45:24.265488+0000","last_unstale":"2024-09-10T18:53:28.237306+0000","last_undegraded":"2024-09-10T18:53:28.237306+0000","last_fullsized":"2024-09-10T18:53:28.237306+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:55.776552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:52:55.776552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:06.631241+0000","scrub_duration":0.003664676,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":191,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.228639+0000","last_change":"2024-09-10T18:52:02.360333+0000","last_active":"2024-09-10T18:53:28.228639+0000","last_peered":"2024-09-10T18:53:28.228639+0000","last_clean":"2024-09-10T18:53:28.228639+0000","last_became_active":"2024-09-10T18:45:17.959475+0000","last_became_peered":"2024-09-10T18:45:17.959475+0000","last_unstale":"2024-09-10T18:53:28.228639+0000","last_undegraded":"2024-09-10T18:53:28.228639+0000","last_fullsized":"2024-09-10T18:53:28.228639+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:02.360246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.718270+0000","last_clean_scrub_stamp":"2024-09-10T18:52:02.360246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0054286930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"63'15","reported_seq":230,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.237206+0000","last_change":"2024-09-10T18:51:34.023246+0000","last_active":"2024-09-10T18:53:28.237206+0000","last_peered":"2024-09-10T18:53:28.237206+0000","last_clean":"2024-09-10T18:53:28.237206+0000","last_became_active":"2024-09-10T18:47:36.413635+0000","last_became_peered":"2024-09-10T18:47:36.413635+0000","last_unstale":"2024-09-10T18:53:28.237206+0000","last_undegraded":"2024-09-10T18:53:28.237206+0000","last_fullsized":"2024-09-10T18:53:28.237206+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:51:34.023191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:34.023191+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027158859999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":184,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.209313+0000","last_change":"2024-09-10T18:52:58.092268+0000","last_active":"2024-09-10T18:53:26.209313+0000","last_peered":"2024-09-10T18:53:26.209313+0000","last_clean":"2024-09-10T18:53:26.209313+0000","last_became_active":"2024-09-10T18:45:22.065619+0000","last_became_peered":"2024-09-10T18:45:22.065619+0000","last_unstale":"2024-09-10T18:53:26.209313+0000","last_undegraded":"2024-09-10T18:53:26.209313+0000","last_fullsized":"2024-09-10T18:53:26.209313+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:58.092186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.057932+0000","last_clean_scrub_stamp":"2024-09-10T18:52:58.092186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:01.637289+0000","scrub_duration":0.0036111530000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":201,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.236803+0000","last_change":"2024-09-10T18:53:21.274544+0000","last_active":"2024-09-10T18:53:28.236803+0000","last_peered":"2024-09-10T18:53:28.236803+0000","last_clean":"2024-09-10T18:53:28.236803+0000","last_became_active":"2024-09-10T18:53:21.274153+0000","last_became_peered":"2024-09-10T18:53:21.274153+0000","last_unstale":"2024-09-10T18:53:28.236803+0000","last_undegraded":"2024-09-10T18:53:28.236803+0000","last_fullsized":"2024-09-10T18:53:28.236803+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:03.790824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:53:03.790824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:22.354696+0000","scrub_duration":0.003204742,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"83'13","reported_seq":196,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.209389+0000","last_change":"2024-09-10T18:52:07.239403+0000","last_active":"2024-09-10T18:53:26.209389+0000","last_peered":"2024-09-10T18:53:26.209389+0000","last_clean":"2024-09-10T18:53:26.209389+0000","last_became_active":"2024-09-10T18:45:20.061266+0000","last_became_peered":"2024-09-10T18:45:20.061266+0000","last_unstale":"2024-09-10T18:53:26.209389+0000","last_undegraded":"2024-09-10T18:53:26.209389+0000","last_fullsized":"2024-09-10T18:53:26.209389+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'13","last_scrub_stamp":"2024-09-10T18:52:07.239318+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_clean_scrub_stamp":"2024-09-10T18:52:07.239318+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044351590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":20,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.228927+0000","last_change":"2024-09-10T18:52:58.104967+0000","last_active":"2024-09-10T18:53:26.228927+0000","last_peered":"2024-09-10T18:53:26.228927+0000","last_clean":"2024-09-10T18:53:26.228927+0000","last_became_active":"2024-09-10T18:52:58.104329+0000","last_became_peered":"2024-09-10T18:52:58.104329+0000","last_unstale":"2024-09-10T18:53:26.228927+0000","last_undegraded":"2024-09-10T18:53:26.228927+0000","last_fullsized":"2024-09-10T18:53:26.228927+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:01.135988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:01.135988+0000","last_clean_scrub_stamp":"2024-09-10T18:52:01.135988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002631478,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,7],"acting":[6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":172,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.206039+0000","last_change":"2024-09-10T18:51:48.432515+0000","last_active":"2024-09-10T18:53:26.206039+0000","last_peered":"2024-09-10T18:53:26.206039+0000","last_clean":"2024-09-10T18:53:26.206039+0000","last_became_active":"2024-09-10T18:45:24.081002+0000","last_became_peered":"2024-09-10T18:45:24.081002+0000","last_unstale":"2024-09-10T18:53:26.206039+0000","last_undegraded":"2024-09-10T18:53:26.206039+0000","last_fullsized":"2024-09-10T18:53:26.206039+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:48.432459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:48.432459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013158130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":190,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.212407+0000","last_change":"2024-09-10T18:51:51.188534+0000","last_active":"2024-09-10T18:53:26.212407+0000","last_peered":"2024-09-10T18:53:26.212407+0000","last_clean":"2024-09-10T18:53:26.212407+0000","last_became_active":"2024-09-10T18:45:17.961097+0000","last_became_peered":"2024-09-10T18:45:17.961097+0000","last_unstale":"2024-09-10T18:53:26.212407+0000","last_undegraded":"2024-09-10T18:53:26.212407+0000","last_fullsized":"2024-09-10T18:53:26.212407+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:51.188458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:51.188458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037631460000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"163'7","reported_seq":188,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.209319+0000","last_change":"2024-09-10T18:51:54.261967+0000","last_active":"2024-09-10T18:53:26.209319+0000","last_peered":"2024-09-10T18:53:26.209319+0000","last_clean":"2024-09-10T18:53:26.209319+0000","last_became_active":"2024-09-10T18:45:20.057567+0000","last_became_peered":"2024-09-10T18:45:20.057567+0000","last_unstale":"2024-09-10T18:53:26.209319+0000","last_undegraded":"2024-09-10T18:53:26.209319+0000","last_fullsized":"2024-09-10T18:53:26.209319+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"163'7","last_scrub_stamp":"2024-09-10T18:51:54.261879+0000","last_deep_scrub":"163'7","last_deep_scrub_stamp":"2024-09-10T18:51:54.261879+0000","last_clean_scrub_stamp":"2024-09-10T18:51:54.261879+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0087662790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":186,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.209542+0000","last_change":"2024-09-10T18:51:33.438852+0000","last_active":"2024-09-10T18:53:26.209542+0000","last_peered":"2024-09-10T18:53:26.209542+0000","last_clean":"2024-09-10T18:53:26.209542+0000","last_became_active":"2024-09-10T18:45:22.076575+0000","last_became_peered":"2024-09-10T18:45:22.076575+0000","last_unstale":"2024-09-10T18:53:26.209542+0000","last_undegraded":"2024-09-10T18:53:26.209542+0000","last_fullsized":"2024-09-10T18:53:26.209542+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:33.438763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:36.509971+0000","last_clean_scrub_stamp":"2024-09-10T18:51:33.438763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019905130000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":27,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.231478+0000","last_change":"2024-09-10T18:52:54.623058+0000","last_active":"2024-09-10T18:53:28.231478+0000","last_peered":"2024-09-10T18:53:28.231478+0000","last_clean":"2024-09-10T18:53:28.231478+0000","last_became_active":"2024-09-10T18:52:54.622536+0000","last_became_peered":"2024-09-10T18:52:54.622536+0000","last_unstale":"2024-09-10T18:53:28.231478+0000","last_undegraded":"2024-09-10T18:53:28.231478+0000","last_fullsized":"2024-09-10T18:53:28.231478+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:51.287440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:51.287440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":200,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.210914+0000","last_change":"2024-09-10T18:52:44.199130+0000","last_active":"2024-09-10T18:53:26.210914+0000","last_peered":"2024-09-10T18:53:26.210914+0000","last_clean":"2024-09-10T18:53:26.210914+0000","last_became_active":"2024-09-10T18:47:36.414393+0000","last_became_peered":"2024-09-10T18:47:36.414393+0000","last_unstale":"2024-09-10T18:53:26.210914+0000","last_undegraded":"2024-09-10T18:53:26.210914+0000","last_fullsized":"2024-09-10T18:53:26.210914+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:44.199050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:52:44.199050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:52.763826+0000","scrub_duration":0.010596744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"63'9","reported_seq":202,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.206198+0000","last_change":"2024-09-10T18:53:01.112930+0000","last_active":"2024-09-10T18:53:26.206198+0000","last_peered":"2024-09-10T18:53:26.206198+0000","last_clean":"2024-09-10T18:53:26.206198+0000","last_became_active":"2024-09-10T18:45:20.052652+0000","last_became_peered":"2024-09-10T18:45:20.052652+0000","last_unstale":"2024-09-10T18:53:26.206198+0000","last_undegraded":"2024-09-10T18:53:26.206198+0000","last_fullsized":"2024-09-10T18:53:26.206198+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:53:01.112771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:53:01.112771+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:10.914620+0000","scrub_duration":0.0055933110000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":185,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.215632+0000","last_change":"2024-09-10T18:51:59.201412+0000","last_active":"2024-09-10T18:53:26.215632+0000","last_peered":"2024-09-10T18:53:26.215632+0000","last_clean":"2024-09-10T18:53:26.215632+0000","last_became_active":"2024-09-10T18:45:22.068326+0000","last_became_peered":"2024-09-10T18:45:22.068326+0000","last_unstale":"2024-09-10T18:53:26.215632+0000","last_undegraded":"2024-09-10T18:53:26.215632+0000","last_fullsized":"2024-09-10T18:53:26.215632+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:59.201336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:59.201336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0032166959999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":185,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.224791+0000","last_change":"2024-09-10T18:52:59.425795+0000","last_active":"2024-09-10T18:53:28.224791+0000","last_peered":"2024-09-10T18:53:28.224791+0000","last_clean":"2024-09-10T18:53:28.224791+0000","last_became_active":"2024-09-10T18:45:24.081488+0000","last_became_peered":"2024-09-10T18:45:24.081488+0000","last_unstale":"2024-09-10T18:53:28.224791+0000","last_undegraded":"2024-09-10T18:53:28.224791+0000","last_fullsized":"2024-09-10T18:53:28.224791+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:59.425718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:59.425718+0000","last_clean_scrub_stamp":"2024-09-10T18:52:59.425718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:26.429945+0000","scrub_duration":0.0046586320000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"198'10127","reported_seq":10175,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-10T18:53:31.398915+0000","last_change":"2024-09-10T18:51:37.588528+0000","last_active":"2024-09-10T18:53:31.398915+0000","last_peered":"2024-09-10T18:53:31.398915+0000","last_clean":"2024-09-10T18:53:31.398915+0000","last_became_active":"2024-09-10T18:51:37.588217+0000","last_became_peered":"2024-09-10T18:51:37.588217+0000","last_unstale":"2024-09-10T18:53:31.398915+0000","last_undegraded":"2024-09-10T18:53:31.398915+0000","last_fullsized":"2024-09-10T18:53:31.398915+0000","mapping_epoch":158,"log_start":"178'6198","ondisk_log_start":"178'6198","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":3929,"log_dups_size":0,"ondisk_log_size":3929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":550699008,"num_objects":8403,"num_object_clones":0,"num_object_copies":25209,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10127,"num_write_kb":592960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"70'81","reported_seq":32,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.240375+0000","last_change":"2024-09-10T18:52:54.622783+0000","last_active":"2024-09-10T18:53:28.240375+0000","last_peered":"2024-09-10T18:53:28.240375+0000","last_clean":"2024-09-10T18:53:28.240375+0000","last_became_active":"2024-09-10T18:52:54.619039+0000","last_became_peered":"2024-09-10T18:52:54.619039+0000","last_unstale":"2024-09-10T18:53:28.240375+0000","last_undegraded":"2024-09-10T18:53:28.240375+0000","last_fullsized":"2024-09-10T18:53:28.240375+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:52:06.203977+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:52:06.203977+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019876809999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":20,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":370027,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"78'25","reported_seq":250,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.212648+0000","last_change":"2024-09-10T18:52:55.075174+0000","last_active":"2024-09-10T18:53:26.212648+0000","last_peered":"2024-09-10T18:53:26.212648+0000","last_clean":"2024-09-10T18:53:26.212648+0000","last_became_active":"2024-09-10T18:47:36.409300+0000","last_became_peered":"2024-09-10T18:47:36.409300+0000","last_unstale":"2024-09-10T18:53:26.212648+0000","last_undegraded":"2024-09-10T18:53:26.212648+0000","last_fullsized":"2024-09-10T18:53:26.212648+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'25","last_scrub_stamp":"2024-09-10T18:52:55.075103+0000","last_deep_scrub":"78'25","last_deep_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_clean_scrub_stamp":"2024-09-10T18:52:55.075103+0000","objects_scrubbed":8,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:56.555975+0000","scrub_duration":0.0020572170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":190,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.210017+0000","last_change":"2024-09-10T18:51:54.175390+0000","last_active":"2024-09-10T18:53:26.210017+0000","last_peered":"2024-09-10T18:53:26.210017+0000","last_clean":"2024-09-10T18:53:26.210017+0000","last_became_active":"2024-09-10T18:45:17.963732+0000","last_became_peered":"2024-09-10T18:45:17.963732+0000","last_unstale":"2024-09-10T18:53:26.210017+0000","last_undegraded":"2024-09-10T18:53:26.210017+0000","last_fullsized":"2024-09-10T18:53:26.210017+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:54.175305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:54.175305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029840190000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":177,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.230118+0000","last_change":"2024-09-10T18:51:39.055614+0000","last_active":"2024-09-10T18:53:28.230118+0000","last_peered":"2024-09-10T18:53:28.230118+0000","last_clean":"2024-09-10T18:53:28.230118+0000","last_became_active":"2024-09-10T18:45:22.071443+0000","last_became_peered":"2024-09-10T18:45:22.071443+0000","last_unstale":"2024-09-10T18:53:28.230118+0000","last_undegraded":"2024-09-10T18:53:28.230118+0000","last_fullsized":"2024-09-10T18:53:28.230118+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:39.055556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:04.512425+0000","last_clean_scrub_stamp":"2024-09-10T18:51:39.055556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016498730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":141,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.226392+0000","last_change":"2024-09-10T18:53:05.317961+0000","last_active":"2024-09-10T18:53:28.226392+0000","last_peered":"2024-09-10T18:53:28.226392+0000","last_clean":"2024-09-10T18:53:28.226392+0000","last_became_active":"2024-09-10T18:47:57.104433+0000","last_became_peered":"2024-09-10T18:47:57.104433+0000","last_unstale":"2024-09-10T18:53:28.226392+0000","last_undegraded":"2024-09-10T18:53:28.226392+0000","last_fullsized":"2024-09-10T18:53:28.226392+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:05.317828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:53:05.317828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:22.490945+0000","scrub_duration":0.011044185999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"198'9896","reported_seq":12558,"reported_epoch":198,"state":"active+remapped+backfilling","last_fresh":"2024-09-10T18:53:31.484562+0000","last_change":"2024-09-10T18:53:22.228011+0000","last_active":"2024-09-10T18:53:31.484562+0000","last_peered":"2024-09-10T18:53:31.484562+0000","last_clean":"2024-09-10T18:53:20.171514+0000","last_became_active":"2024-09-10T18:53:22.211080+0000","last_became_peered":"2024-09-10T18:53:22.211080+0000","last_unstale":"2024-09-10T18:53:31.484562+0000","last_undegraded":"2024-09-10T18:53:31.484562+0000","last_fullsized":"2024-09-10T18:53:31.484562+0000","mapping_epoch":194,"log_start":"178'6062","ondisk_log_start":"178'6062","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":3834,"log_dups_size":0,"ondisk_log_size":3834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":543031296,"num_objects":8286,"num_object_clones":0,"num_object_copies":24858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7043,"num_objects_unfound":0,"num_objects_dirty":8286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9896,"num_write_kb":581824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1301,"num_bytes_recovered":85262336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,5],"avail_no_missing":["3","1","5"],"object_location_counts":[{"shards":"1,3,5","objects":8286}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.9","version":"70'81","reported_seq":254,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.210270+0000","last_change":"2024-09-10T18:52:41.212451+0000","last_active":"2024-09-10T18:53:26.210270+0000","last_peered":"2024-09-10T18:53:26.210270+0000","last_clean":"2024-09-10T18:53:26.210270+0000","last_became_active":"2024-09-10T18:50:08.399879+0000","last_became_peered":"2024-09-10T18:50:08.399879+0000","last_unstale":"2024-09-10T18:53:26.210270+0000","last_undegraded":"2024-09-10T18:53:26.210270+0000","last_fullsized":"2024-09-10T18:53:26.210270+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:52:41.212357+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:52:41.212357+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:05.015225+0000","scrub_duration":0.0029345460000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"116'15","reported_seq":206,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.211658+0000","last_change":"2024-09-10T18:53:06.056470+0000","last_active":"2024-09-10T18:53:26.211658+0000","last_peered":"2024-09-10T18:53:26.211658+0000","last_clean":"2024-09-10T18:53:26.211658+0000","last_became_active":"2024-09-10T18:45:20.053428+0000","last_became_peered":"2024-09-10T18:45:20.053428+0000","last_unstale":"2024-09-10T18:53:26.211658+0000","last_undegraded":"2024-09-10T18:53:26.211658+0000","last_fullsized":"2024-09-10T18:53:26.211658+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"116'15","last_scrub_stamp":"2024-09-10T18:53:06.056392+0000","last_deep_scrub":"116'15","last_deep_scrub_stamp":"2024-09-10T18:49:12.374527+0000","last_clean_scrub_stamp":"2024-09-10T18:53:06.056392+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:32.014324+0000","scrub_duration":0.0078605010000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":191,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.231099+0000","last_change":"2024-09-10T18:52:45.139636+0000","last_active":"2024-09-10T18:53:28.231099+0000","last_peered":"2024-09-10T18:53:28.231099+0000","last_clean":"2024-09-10T18:53:28.231099+0000","last_became_active":"2024-09-10T18:45:17.966833+0000","last_became_peered":"2024-09-10T18:45:17.966833+0000","last_unstale":"2024-09-10T18:53:28.231099+0000","last_undegraded":"2024-09-10T18:53:28.231099+0000","last_fullsized":"2024-09-10T18:53:28.231099+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:45.139534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:52:45.139534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:47.025693+0000","scrub_duration":0.002710269,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"89'9","reported_seq":245,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.212055+0000","last_change":"2024-09-10T18:51:50.194342+0000","last_active":"2024-09-10T18:53:26.212055+0000","last_peered":"2024-09-10T18:53:26.212055+0000","last_clean":"2024-09-10T18:53:26.212055+0000","last_became_active":"2024-09-10T18:47:36.421188+0000","last_became_peered":"2024-09-10T18:47:36.421188+0000","last_unstale":"2024-09-10T18:53:26.212055+0000","last_undegraded":"2024-09-10T18:53:26.212055+0000","last_fullsized":"2024-09-10T18:53:26.212055+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'9","last_scrub_stamp":"2024-09-10T18:51:50.194254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:50.194254+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025134200000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":202,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.206182+0000","last_change":"2024-09-10T18:52:54.599606+0000","last_active":"2024-09-10T18:53:26.206182+0000","last_peered":"2024-09-10T18:53:26.206182+0000","last_clean":"2024-09-10T18:53:26.206182+0000","last_became_active":"2024-09-10T18:52:54.599288+0000","last_became_peered":"2024-09-10T18:52:54.599288+0000","last_unstale":"2024-09-10T18:53:26.206182+0000","last_undegraded":"2024-09-10T18:53:26.206182+0000","last_fullsized":"2024-09-10T18:53:26.206182+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:45.193171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:32.043666+0000","last_clean_scrub_stamp":"2024-09-10T18:52:45.193171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:02.607036+0000","scrub_duration":0.0078342310000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3","version":"198'9881","reported_seq":9929,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-10T18:53:31.377631+0000","last_change":"2024-09-10T18:51:37.590013+0000","last_active":"2024-09-10T18:53:31.377631+0000","last_peered":"2024-09-10T18:53:31.377631+0000","last_clean":"2024-09-10T18:53:31.377631+0000","last_became_active":"2024-09-10T18:51:37.589730+0000","last_became_peered":"2024-09-10T18:51:37.589730+0000","last_unstale":"2024-09-10T18:53:31.377631+0000","last_undegraded":"2024-09-10T18:53:31.377631+0000","last_fullsized":"2024-09-10T18:53:31.377631+0000","mapping_epoch":158,"log_start":"177'5900","ondisk_log_start":"177'5900","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":3981,"log_dups_size":0,"ondisk_log_size":3981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":544276480,"num_objects":8305,"num_object_clones":0,"num_object_copies":24915,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9881,"num_write_kb":581952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"70'81","reported_seq":97,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.216626+0000","last_change":"2024-09-10T18:52:04.178564+0000","last_active":"2024-09-10T18:53:26.216626+0000","last_peered":"2024-09-10T18:53:26.216626+0000","last_clean":"2024-09-10T18:53:26.216626+0000","last_became_active":"2024-09-10T18:50:08.398345+0000","last_became_peered":"2024-09-10T18:50:08.398345+0000","last_unstale":"2024-09-10T18:53:26.216626+0000","last_undegraded":"2024-09-10T18:53:26.216626+0000","last_fullsized":"2024-09-10T18:53:26.216626+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:52:04.178483+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:52:04.178483+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041829420000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"141'22","reported_seq":186,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.215133+0000","last_change":"2024-09-10T18:53:00.188252+0000","last_active":"2024-09-10T18:53:26.215133+0000","last_peered":"2024-09-10T18:53:26.215133+0000","last_clean":"2024-09-10T18:53:26.215133+0000","last_became_active":"2024-09-10T18:46:56.742946+0000","last_became_peered":"2024-09-10T18:46:56.742946+0000","last_unstale":"2024-09-10T18:53:26.215133+0000","last_undegraded":"2024-09-10T18:53:26.215133+0000","last_fullsized":"2024-09-10T18:53:26.215133+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"141'22","last_scrub_stamp":"2024-09-10T18:53:00.188166+0000","last_deep_scrub":"141'22","last_deep_scrub_stamp":"2024-09-10T18:50:57.119152+0000","last_clean_scrub_stamp":"2024-09-10T18:53:00.188166+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:16.440290+0000","scrub_duration":0.0091410329999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":191,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.231943+0000","last_change":"2024-09-10T18:51:51.055043+0000","last_active":"2024-09-10T18:53:28.231943+0000","last_peered":"2024-09-10T18:53:28.231943+0000","last_clean":"2024-09-10T18:53:28.231943+0000","last_became_active":"2024-09-10T18:45:17.965409+0000","last_became_peered":"2024-09-10T18:45:17.965409+0000","last_unstale":"2024-09-10T18:53:28.231943+0000","last_undegraded":"2024-09-10T18:53:28.231943+0000","last_fullsized":"2024-09-10T18:53:28.231943+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:51.054935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_clean_scrub_stamp":"2024-09-10T18:51:51.054935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004679758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"63'8","reported_seq":290,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.206263+0000","last_change":"2024-09-10T18:52:46.200337+0000","last_active":"2024-09-10T18:53:26.206263+0000","last_peered":"2024-09-10T18:53:26.206263+0000","last_clean":"2024-09-10T18:53:26.206263+0000","last_became_active":"2024-09-10T18:45:22.062497+0000","last_became_peered":"2024-09-10T18:45:22.062497+0000","last_unstale":"2024-09-10T18:53:26.206263+0000","last_undegraded":"2024-09-10T18:53:26.206263+0000","last_fullsized":"2024-09-10T18:53:26.206263+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:52:46.200266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:52:46.200266+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:51.032488+0000","scrub_duration":0.049044943000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":183,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.215155+0000","last_change":"2024-09-10T18:51:54.218678+0000","last_active":"2024-09-10T18:53:26.215155+0000","last_peered":"2024-09-10T18:53:26.215155+0000","last_clean":"2024-09-10T18:53:26.215155+0000","last_became_active":"2024-09-10T18:45:24.082362+0000","last_became_peered":"2024-09-10T18:45:24.082362+0000","last_unstale":"2024-09-10T18:53:26.215155+0000","last_undegraded":"2024-09-10T18:53:26.215155+0000","last_fullsized":"2024-09-10T18:53:26.215155+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:54.218602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:54.218602+0000","last_clean_scrub_stamp":"2024-09-10T18:51:54.218602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0059638290000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"70'81","reported_seq":127,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.233038+0000","last_change":"2024-09-10T18:53:21.170675+0000","last_active":"2024-09-10T18:53:28.233038+0000","last_peered":"2024-09-10T18:53:28.233038+0000","last_clean":"2024-09-10T18:53:28.233038+0000","last_became_active":"2024-09-10T18:53:21.170311+0000","last_became_peered":"2024-09-10T18:53:21.170311+0000","last_unstale":"2024-09-10T18:53:28.233038+0000","last_undegraded":"2024-09-10T18:53:28.233038+0000","last_fullsized":"2024-09-10T18:53:28.233038+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:53:04.265619+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:53:04.265619+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:12.933301+0000","scrub_duration":0.016066791,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"197'9856","reported_seq":10229,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:31.091684+0000","last_change":"2024-09-10T18:53:07.156722+0000","last_active":"2024-09-10T18:53:31.091684+0000","last_peered":"2024-09-10T18:53:31.091684+0000","last_clean":"2024-09-10T18:53:31.091684+0000","last_became_active":"2024-09-10T18:51:37.588723+0000","last_became_peered":"2024-09-10T18:51:37.588723+0000","last_unstale":"2024-09-10T18:53:31.091684+0000","last_undegraded":"2024-09-10T18:53:31.091684+0000","last_fullsized":"2024-09-10T18:53:31.091684+0000","mapping_epoch":158,"log_start":"176'5757","ondisk_log_start":"176'5757","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"190'8100","last_scrub_stamp":"2024-09-10T18:53:07.156542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:53:07.156542+0000","objects_scrubbed":7946,"log_size":4099,"log_dups_size":0,"ondisk_log_size":4099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:24.720270+0000","scrub_duration":4.0548768180000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":542900224,"num_objects":8284,"num_object_clones":0,"num_object_copies":24852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9856,"num_write_kb":580480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"63'11","reported_seq":271,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.208385+0000","last_change":"2024-09-10T18:52:51.192206+0000","last_active":"2024-09-10T18:53:26.208385+0000","last_peered":"2024-09-10T18:53:26.208385+0000","last_clean":"2024-09-10T18:53:26.208385+0000","last_became_active":"2024-09-10T18:47:57.106935+0000","last_became_peered":"2024-09-10T18:47:57.106935+0000","last_unstale":"2024-09-10T18:53:26.208385+0000","last_undegraded":"2024-09-10T18:53:26.208385+0000","last_fullsized":"2024-09-10T18:53:26.208385+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:52:51.192069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:52:51.192069+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:55.329737+0000","scrub_duration":0.004119012,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":190,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.215786+0000","last_change":"2024-09-10T18:51:44.209450+0000","last_active":"2024-09-10T18:53:26.215786+0000","last_peered":"2024-09-10T18:53:26.215786+0000","last_clean":"2024-09-10T18:53:26.215786+0000","last_became_active":"2024-09-10T18:45:17.966133+0000","last_became_peered":"2024-09-10T18:45:17.966133+0000","last_unstale":"2024-09-10T18:53:26.215786+0000","last_undegraded":"2024-09-10T18:53:26.215786+0000","last_fullsized":"2024-09-10T18:53:26.215786+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:44.209392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:44.209392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001553123,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":185,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.215278+0000","last_change":"2024-09-10T18:51:44.388198+0000","last_active":"2024-09-10T18:53:26.215278+0000","last_peered":"2024-09-10T18:53:26.215278+0000","last_clean":"2024-09-10T18:53:26.215278+0000","last_became_active":"2024-09-10T18:45:22.068918+0000","last_became_peered":"2024-09-10T18:45:22.068918+0000","last_unstale":"2024-09-10T18:53:26.215278+0000","last_undegraded":"2024-09-10T18:53:26.215278+0000","last_fullsized":"2024-09-10T18:53:26.215278+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:44.388105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_clean_scrub_stamp":"2024-09-10T18:51:44.388105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013288989999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":175,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.236618+0000","last_change":"2024-09-10T18:51:41.073374+0000","last_active":"2024-09-10T18:53:28.236618+0000","last_peered":"2024-09-10T18:53:28.236618+0000","last_clean":"2024-09-10T18:53:28.236618+0000","last_became_active":"2024-09-10T18:45:24.264538+0000","last_became_peered":"2024-09-10T18:45:24.264538+0000","last_unstale":"2024-09-10T18:53:28.236618+0000","last_undegraded":"2024-09-10T18:53:28.236618+0000","last_fullsized":"2024-09-10T18:53:28.236618+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:41.073320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:41.073320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001777551,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"197'9887","reported_seq":9935,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:31.177672+0000","last_change":"2024-09-10T18:51:37.911703+0000","last_active":"2024-09-10T18:53:31.177672+0000","last_peered":"2024-09-10T18:53:31.177672+0000","last_clean":"2024-09-10T18:53:31.177672+0000","last_became_active":"2024-09-10T18:51:37.911394+0000","last_became_peered":"2024-09-10T18:51:37.911394+0000","last_unstale":"2024-09-10T18:53:31.177672+0000","last_undegraded":"2024-09-10T18:53:31.177672+0000","last_fullsized":"2024-09-10T18:53:31.177672+0000","mapping_epoch":158,"log_start":"173'4500","ondisk_log_start":"173'4500","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":5387,"log_dups_size":0,"ondisk_log_size":5387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":541458456,"num_objects":8263,"num_object_clones":0,"num_object_copies":24789,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8263,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9887,"num_write_kb":580737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"70'81","reported_seq":113,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.236817+0000","last_change":"2024-09-10T18:52:54.623119+0000","last_active":"2024-09-10T18:53:28.236817+0000","last_peered":"2024-09-10T18:53:28.236817+0000","last_clean":"2024-09-10T18:53:28.236817+0000","last_became_active":"2024-09-10T18:52:54.622187+0000","last_became_peered":"2024-09-10T18:52:54.622187+0000","last_unstale":"2024-09-10T18:53:28.236817+0000","last_undegraded":"2024-09-10T18:53:28.236817+0000","last_fullsized":"2024-09-10T18:53:28.236817+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:50:59.164417+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:50:59.164417+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0063541700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109226,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"143'21","reported_seq":214,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.208670+0000","last_change":"2024-09-10T18:52:59.095130+0000","last_active":"2024-09-10T18:53:26.208670+0000","last_peered":"2024-09-10T18:53:26.208670+0000","last_clean":"2024-09-10T18:53:26.208670+0000","last_became_active":"2024-09-10T18:45:20.052243+0000","last_became_peered":"2024-09-10T18:45:20.052243+0000","last_unstale":"2024-09-10T18:53:26.208670+0000","last_undegraded":"2024-09-10T18:53:26.208670+0000","last_fullsized":"2024-09-10T18:53:26.208670+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"143'21","last_scrub_stamp":"2024-09-10T18:52:59.095048+0000","last_deep_scrub":"143'21","last_deep_scrub_stamp":"2024-09-10T18:51:02.540216+0000","last_clean_scrub_stamp":"2024-09-10T18:52:59.095048+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:16.731540+0000","scrub_duration":0.013944524,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":199,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.224428+0000","last_change":"2024-09-10T18:52:41.623788+0000","last_active":"2024-09-10T18:53:26.224428+0000","last_peered":"2024-09-10T18:53:26.224428+0000","last_clean":"2024-09-10T18:53:26.224428+0000","last_became_active":"2024-09-10T18:47:36.424466+0000","last_became_peered":"2024-09-10T18:47:36.424466+0000","last_unstale":"2024-09-10T18:53:26.224428+0000","last_undegraded":"2024-09-10T18:53:26.224428+0000","last_fullsized":"2024-09-10T18:53:26.224428+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:41.623564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_clean_scrub_stamp":"2024-09-10T18:52:41.623564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:07.985199+0000","scrub_duration":0.0056375949999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":294,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:31.176133+0000","last_change":"2024-09-10T18:53:02.761328+0000","last_active":"2024-09-10T18:53:31.176133+0000","last_peered":"2024-09-10T18:53:31.176133+0000","last_clean":"2024-09-10T18:53:31.176133+0000","last_became_active":"2024-09-10T18:45:22.074540+0000","last_became_peered":"2024-09-10T18:45:22.074540+0000","last_unstale":"2024-09-10T18:53:31.176133+0000","last_undegraded":"2024-09-10T18:53:31.176133+0000","last_fullsized":"2024-09-10T18:53:31.176133+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:53:02.761109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:53:02.761109+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:11.137248+0000","scrub_duration":0.012855039,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":204,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.209657+0000","last_change":"2024-09-10T18:52:43.174900+0000","last_active":"2024-09-10T18:53:26.209657+0000","last_peered":"2024-09-10T18:53:26.209657+0000","last_clean":"2024-09-10T18:53:26.209657+0000","last_became_active":"2024-09-10T18:47:36.414665+0000","last_became_peered":"2024-09-10T18:47:36.414665+0000","last_unstale":"2024-09-10T18:53:26.209657+0000","last_undegraded":"2024-09-10T18:53:26.209657+0000","last_fullsized":"2024-09-10T18:53:26.209657+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:43.174812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:52:43.174812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:07.876619+0000","scrub_duration":0.0023010819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"198'10020","reported_seq":10066,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-10T18:53:31.550204+0000","last_change":"2024-09-10T18:51:37.909451+0000","last_active":"2024-09-10T18:53:31.550204+0000","last_peered":"2024-09-10T18:53:31.550204+0000","last_clean":"2024-09-10T18:53:31.550204+0000","last_became_active":"2024-09-10T18:51:37.909136+0000","last_became_peered":"2024-09-10T18:51:37.909136+0000","last_unstale":"2024-09-10T18:53:31.550204+0000","last_undegraded":"2024-09-10T18:53:31.550204+0000","last_fullsized":"2024-09-10T18:53:31.550204+0000","mapping_epoch":158,"log_start":"176'5614","ondisk_log_start":"176'5614","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":4406,"log_dups_size":0,"ondisk_log_size":4406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":543686656,"num_objects":8296,"num_object_clones":0,"num_object_copies":24888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10020,"num_write_kb":586112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"70'81","reported_seq":106,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.211871+0000","last_change":"2024-09-10T18:52:54.120550+0000","last_active":"2024-09-10T18:53:26.211871+0000","last_peered":"2024-09-10T18:53:26.211871+0000","last_clean":"2024-09-10T18:53:26.211871+0000","last_became_active":"2024-09-10T18:52:14.584798+0000","last_became_peered":"2024-09-10T18:52:14.584798+0000","last_unstale":"2024-09-10T18:53:26.211871+0000","last_undegraded":"2024-09-10T18:53:26.211871+0000","last_fullsized":"2024-09-10T18:53:26.211871+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:52:54.120497+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:52:54.120497+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:17.290464+0000","scrub_duration":0.001089507,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"63'18","reported_seq":221,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.243816+0000","last_change":"2024-09-10T18:53:01.749831+0000","last_active":"2024-09-10T18:53:28.243816+0000","last_peered":"2024-09-10T18:53:28.243816+0000","last_clean":"2024-09-10T18:53:28.243816+0000","last_became_active":"2024-09-10T18:45:20.420937+0000","last_became_peered":"2024-09-10T18:45:20.420937+0000","last_unstale":"2024-09-10T18:53:28.243816+0000","last_undegraded":"2024-09-10T18:53:28.243816+0000","last_fullsized":"2024-09-10T18:53:28.243816+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'18","last_scrub_stamp":"2024-09-10T18:53:01.746777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:53:01.746777+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:25.564358+0000","scrub_duration":0.020214684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":181,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.230893+0000","last_change":"2024-09-10T18:51:48.047994+0000","last_active":"2024-09-10T18:53:28.230893+0000","last_peered":"2024-09-10T18:53:28.230893+0000","last_clean":"2024-09-10T18:53:28.230893+0000","last_became_active":"2024-09-10T18:45:17.961695+0000","last_became_peered":"2024-09-10T18:45:17.961695+0000","last_unstale":"2024-09-10T18:53:28.230893+0000","last_undegraded":"2024-09-10T18:53:28.230893+0000","last_fullsized":"2024-09-10T18:53:28.230893+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:48.047929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:48.047929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018242130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":184,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.208518+0000","last_change":"2024-09-10T18:52:44.198221+0000","last_active":"2024-09-10T18:53:26.208518+0000","last_peered":"2024-09-10T18:53:26.208518+0000","last_clean":"2024-09-10T18:53:26.208518+0000","last_became_active":"2024-09-10T18:45:22.074012+0000","last_became_peered":"2024-09-10T18:45:22.074012+0000","last_unstale":"2024-09-10T18:53:26.208518+0000","last_undegraded":"2024-09-10T18:53:26.208518+0000","last_fullsized":"2024-09-10T18:53:26.208518+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:44.198150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:52:44.198150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:58.821277+0000","scrub_duration":0.0087865340000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":182,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.206160+0000","last_change":"2024-09-10T18:51:52.332328+0000","last_active":"2024-09-10T18:53:26.206160+0000","last_peered":"2024-09-10T18:53:26.206160+0000","last_clean":"2024-09-10T18:53:26.206160+0000","last_became_active":"2024-09-10T18:47:36.410836+0000","last_became_peered":"2024-09-10T18:47:36.410836+0000","last_unstale":"2024-09-10T18:53:26.206160+0000","last_undegraded":"2024-09-10T18:53:26.206160+0000","last_fullsized":"2024-09-10T18:53:26.206160+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:52.332258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_clean_scrub_stamp":"2024-09-10T18:51:52.332258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030121940000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"198'10134","reported_seq":10182,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-10T18:53:31.458885+0000","last_change":"2024-09-10T18:51:37.584352+0000","last_active":"2024-09-10T18:53:31.458885+0000","last_peered":"2024-09-10T18:53:31.458885+0000","last_clean":"2024-09-10T18:53:31.458885+0000","last_became_active":"2024-09-10T18:51:37.583787+0000","last_became_peered":"2024-09-10T18:51:37.583787+0000","last_unstale":"2024-09-10T18:53:31.458885+0000","last_undegraded":"2024-09-10T18:53:31.458885+0000","last_fullsized":"2024-09-10T18:53:31.458885+0000","mapping_epoch":158,"log_start":"178'6323","ondisk_log_start":"178'6323","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":3811,"log_dups_size":0,"ondisk_log_size":3811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":558891008,"num_objects":8528,"num_object_clones":0,"num_object_copies":25584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10134,"num_write_kb":597184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"70'81","reported_seq":175,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.223430+0000","last_change":"2024-09-10T18:52:50.640372+0000","last_active":"2024-09-10T18:53:26.223430+0000","last_peered":"2024-09-10T18:53:26.223430+0000","last_clean":"2024-09-10T18:53:26.223430+0000","last_became_active":"2024-09-10T18:52:14.590633+0000","last_became_peered":"2024-09-10T18:52:14.590633+0000","last_unstale":"2024-09-10T18:53:26.223430+0000","last_undegraded":"2024-09-10T18:53:26.223430+0000","last_fullsized":"2024-09-10T18:53:26.223430+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:52:50.640292+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:52:50.640292+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:04.404186+0000","scrub_duration":0.0043487049999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"63'19","reported_seq":225,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.226397+0000","last_change":"2024-09-10T18:51:41.161251+0000","last_active":"2024-09-10T18:53:28.226397+0000","last_peered":"2024-09-10T18:53:28.226397+0000","last_clean":"2024-09-10T18:53:28.226397+0000","last_became_active":"2024-09-10T18:45:20.421046+0000","last_became_peered":"2024-09-10T18:45:20.421046+0000","last_unstale":"2024-09-10T18:53:28.226397+0000","last_undegraded":"2024-09-10T18:53:28.226397+0000","last_fullsized":"2024-09-10T18:53:28.226397+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:51:41.161196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:41.161196+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023699070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":190,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.212477+0000","last_change":"2024-09-10T18:51:48.196125+0000","last_active":"2024-09-10T18:53:26.212477+0000","last_peered":"2024-09-10T18:53:26.212477+0000","last_clean":"2024-09-10T18:53:26.212477+0000","last_became_active":"2024-09-10T18:45:17.960226+0000","last_became_peered":"2024-09-10T18:45:17.960226+0000","last_unstale":"2024-09-10T18:53:26.212477+0000","last_undegraded":"2024-09-10T18:53:26.212477+0000","last_fullsized":"2024-09-10T18:53:26.212477+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:48.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:48.196065+0000","last_clean_scrub_stamp":"2024-09-10T18:51:48.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013037999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":185,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.222264+0000","last_change":"2024-09-10T18:51:53.548363+0000","last_active":"2024-09-10T18:53:26.222264+0000","last_peered":"2024-09-10T18:53:26.222264+0000","last_clean":"2024-09-10T18:53:26.222264+0000","last_became_active":"2024-09-10T18:45:22.066148+0000","last_became_peered":"2024-09-10T18:45:22.066148+0000","last_unstale":"2024-09-10T18:53:26.222264+0000","last_undegraded":"2024-09-10T18:53:26.222264+0000","last_fullsized":"2024-09-10T18:53:26.222264+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:53.548279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:53.548279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048693970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":185,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.232568+0000","last_change":"2024-09-10T18:51:43.053775+0000","last_active":"2024-09-10T18:53:28.232568+0000","last_peered":"2024-09-10T18:53:28.232568+0000","last_clean":"2024-09-10T18:53:28.232568+0000","last_became_active":"2024-09-10T18:47:36.422295+0000","last_became_peered":"2024-09-10T18:47:36.422295+0000","last_unstale":"2024-09-10T18:53:28.232568+0000","last_undegraded":"2024-09-10T18:53:28.232568+0000","last_fullsized":"2024-09-10T18:53:28.232568+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:43.053721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:49:18.281230+0000","last_clean_scrub_stamp":"2024-09-10T18:51:43.053721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001882325,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.e","version":"198'9894","reported_seq":9942,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-10T18:53:31.556796+0000","last_change":"2024-09-10T18:51:37.909961+0000","last_active":"2024-09-10T18:53:31.556796+0000","last_peered":"2024-09-10T18:53:31.556796+0000","last_clean":"2024-09-10T18:53:31.556796+0000","last_became_active":"2024-09-10T18:51:37.909734+0000","last_became_peered":"2024-09-10T18:51:37.909734+0000","last_unstale":"2024-09-10T18:53:31.556796+0000","last_undegraded":"2024-09-10T18:53:31.556796+0000","last_fullsized":"2024-09-10T18:53:31.556796+0000","mapping_epoch":158,"log_start":"176'5522","ondisk_log_start":"176'5522","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":4372,"log_dups_size":0,"ondisk_log_size":4372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":537133056,"num_objects":8196,"num_object_clones":0,"num_object_copies":24588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9894,"num_write_kb":578880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"70'81","reported_seq":175,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.223959+0000","last_change":"2024-09-10T18:52:51.630205+0000","last_active":"2024-09-10T18:53:26.223959+0000","last_peered":"2024-09-10T18:53:26.223959+0000","last_clean":"2024-09-10T18:53:26.223959+0000","last_became_active":"2024-09-10T18:52:14.582796+0000","last_became_peered":"2024-09-10T18:52:14.582796+0000","last_unstale":"2024-09-10T18:53:26.223959+0000","last_undegraded":"2024-09-10T18:53:26.223959+0000","last_fullsized":"2024-09-10T18:53:26.223959+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:52:51.629760+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:52:51.629760+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:05.611792+0000","scrub_duration":0.013268933,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"83'15","reported_seq":231,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.229715+0000","last_change":"2024-09-10T18:52:54.629316+0000","last_active":"2024-09-10T18:53:28.229715+0000","last_peered":"2024-09-10T18:53:28.229715+0000","last_clean":"2024-09-10T18:53:28.229715+0000","last_became_active":"2024-09-10T18:52:54.617596+0000","last_became_peered":"2024-09-10T18:52:54.617596+0000","last_unstale":"2024-09-10T18:53:28.229715+0000","last_undegraded":"2024-09-10T18:53:28.229715+0000","last_fullsized":"2024-09-10T18:53:28.229715+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'15","last_scrub_stamp":"2024-09-10T18:52:04.182782+0000","last_deep_scrub":"83'15","last_deep_scrub_stamp":"2024-09-10T18:46:58.436473+0000","last_clean_scrub_stamp":"2024-09-10T18:52:04.182782+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0049024990000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":181,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.230333+0000","last_change":"2024-09-10T18:51:05.140273+0000","last_active":"2024-09-10T18:53:28.230333+0000","last_peered":"2024-09-10T18:53:28.230333+0000","last_clean":"2024-09-10T18:53:28.230333+0000","last_became_active":"2024-09-10T18:45:17.966632+0000","last_became_peered":"2024-09-10T18:45:17.966632+0000","last_unstale":"2024-09-10T18:53:28.230333+0000","last_undegraded":"2024-09-10T18:53:28.230333+0000","last_fullsized":"2024-09-10T18:53:28.230333+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:05.140027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:05.140027+0000","last_clean_scrub_stamp":"2024-09-10T18:51:05.140027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0085472780000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"194'9","reported_seq":301,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:31.173474+0000","last_change":"2024-09-10T18:53:29.454656+0000","last_active":"2024-09-10T18:53:31.173474+0000","last_peered":"2024-09-10T18:53:31.173474+0000","last_clean":"2024-09-10T18:53:31.173474+0000","last_became_active":"2024-09-10T18:53:21.169054+0000","last_became_peered":"2024-09-10T18:53:21.169054+0000","last_unstale":"2024-09-10T18:53:31.173474+0000","last_undegraded":"2024-09-10T18:53:31.173474+0000","last_fullsized":"2024-09-10T18:53:31.173474+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:51:37.206224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:37.206224+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001760707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":205,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.224657+0000","last_change":"2024-09-10T18:52:55.515651+0000","last_active":"2024-09-10T18:53:28.224657+0000","last_peered":"2024-09-10T18:53:28.224657+0000","last_clean":"2024-09-10T18:53:28.224657+0000","last_became_active":"2024-09-10T18:52:54.600324+0000","last_became_peered":"2024-09-10T18:52:54.600324+0000","last_unstale":"2024-09-10T18:53:28.224657+0000","last_undegraded":"2024-09-10T18:53:28.224657+0000","last_fullsized":"2024-09-10T18:53:28.224657+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:55.515593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:52:55.515593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:58.865161+0000","scrub_duration":0.0012616260000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"197'10163","reported_seq":10345,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:31.196196+0000","last_change":"2024-09-10T18:53:07.803312+0000","last_active":"2024-09-10T18:53:31.196196+0000","last_peered":"2024-09-10T18:53:31.196196+0000","last_clean":"2024-09-10T18:53:31.196196+0000","last_became_active":"2024-09-10T18:51:37.910355+0000","last_became_peered":"2024-09-10T18:51:37.910355+0000","last_unstale":"2024-09-10T18:53:31.196196+0000","last_undegraded":"2024-09-10T18:53:31.196196+0000","last_fullsized":"2024-09-10T18:53:31.196196+0000","mapping_epoch":158,"log_start":"174'4800","ondisk_log_start":"174'4800","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":3198,"log_size":5363,"log_dups_size":0,"ondisk_log_size":5363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":557645824,"num_objects":8509,"num_object_clones":0,"num_object_copies":25527,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10163,"num_write_kb":597504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"70'81","reported_seq":107,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.216468+0000","last_change":"2024-09-10T18:52:50.091946+0000","last_active":"2024-09-10T18:53:26.216468+0000","last_peered":"2024-09-10T18:53:26.216468+0000","last_clean":"2024-09-10T18:53:26.216468+0000","last_became_active":"2024-09-10T18:52:14.593853+0000","last_became_peered":"2024-09-10T18:52:14.593853+0000","last_unstale":"2024-09-10T18:53:26.216468+0000","last_undegraded":"2024-09-10T18:53:26.216468+0000","last_fullsized":"2024-09-10T18:53:26.216468+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:52:50.091862+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:52:50.091862+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:18.620278+0000","scrub_duration":0.013302506,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"89'29","reported_seq":368,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:31.172517+0000","last_change":"2024-09-10T18:51:57.140326+0000","last_active":"2024-09-10T18:53:31.172517+0000","last_peered":"2024-09-10T18:53:31.172517+0000","last_clean":"2024-09-10T18:53:31.172517+0000","last_became_active":"2024-09-10T18:47:36.420185+0000","last_became_peered":"2024-09-10T18:47:36.420185+0000","last_unstale":"2024-09-10T18:53:31.172517+0000","last_undegraded":"2024-09-10T18:53:31.172517+0000","last_fullsized":"2024-09-10T18:53:31.172517+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'29","last_scrub_stamp":"2024-09-10T18:51:57.140236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:57.140236+0000","objects_scrubbed":10,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012827955,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":189,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.217204+0000","last_change":"2024-09-10T18:51:56.185295+0000","last_active":"2024-09-10T18:53:26.217204+0000","last_peered":"2024-09-10T18:53:26.217204+0000","last_clean":"2024-09-10T18:53:26.217204+0000","last_became_active":"2024-09-10T18:45:17.964494+0000","last_became_peered":"2024-09-10T18:45:17.964494+0000","last_unstale":"2024-09-10T18:53:26.217204+0000","last_undegraded":"2024-09-10T18:53:26.217204+0000","last_fullsized":"2024-09-10T18:53:26.217204+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:56.185217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:12.982599+0000","last_clean_scrub_stamp":"2024-09-10T18:51:56.185217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0035214040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":133,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.223024+0000","last_change":"2024-09-10T18:52:54.620808+0000","last_active":"2024-09-10T18:53:26.223024+0000","last_peered":"2024-09-10T18:53:26.223024+0000","last_clean":"2024-09-10T18:53:26.223024+0000","last_became_active":"2024-09-10T18:47:57.107478+0000","last_became_peered":"2024-09-10T18:47:57.107478+0000","last_unstale":"2024-09-10T18:53:26.223024+0000","last_undegraded":"2024-09-10T18:53:26.223024+0000","last_fullsized":"2024-09-10T18:53:26.223024+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:01.357830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:01.357830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002118796,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":175,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.236352+0000","last_change":"2024-09-10T18:52:47.909320+0000","last_active":"2024-09-10T18:53:28.236352+0000","last_peered":"2024-09-10T18:53:28.236352+0000","last_clean":"2024-09-10T18:53:28.236352+0000","last_became_active":"2024-09-10T18:45:24.073307+0000","last_became_peered":"2024-09-10T18:45:24.073307+0000","last_unstale":"2024-09-10T18:53:28.236352+0000","last_undegraded":"2024-09-10T18:53:28.236352+0000","last_fullsized":"2024-09-10T18:53:28.236352+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:47.909248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:22.891084+0000","last_clean_scrub_stamp":"2024-09-10T18:52:47.909248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:00.665126+0000","scrub_duration":0.0077988500000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"198'9973","reported_seq":10021,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-10T18:53:31.445585+0000","last_change":"2024-09-10T18:51:37.583541+0000","last_active":"2024-09-10T18:53:31.445585+0000","last_peered":"2024-09-10T18:53:31.445585+0000","last_clean":"2024-09-10T18:53:31.445585+0000","last_became_active":"2024-09-10T18:51:37.583248+0000","last_became_peered":"2024-09-10T18:51:37.583248+0000","last_unstale":"2024-09-10T18:53:31.445585+0000","last_undegraded":"2024-09-10T18:53:31.445585+0000","last_fullsized":"2024-09-10T18:53:31.445585+0000","mapping_epoch":158,"log_start":"178'6124","ondisk_log_start":"178'6124","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":3849,"log_dups_size":0,"ondisk_log_size":3849,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":541392896,"num_objects":8261,"num_object_clones":0,"num_object_copies":24783,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9973,"num_write_kb":583488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"70'81","reported_seq":264,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.209684+0000","last_change":"2024-09-10T18:52:54.096172+0000","last_active":"2024-09-10T18:53:26.209684+0000","last_peered":"2024-09-10T18:53:26.209684+0000","last_clean":"2024-09-10T18:53:26.209684+0000","last_became_active":"2024-09-10T18:52:14.576152+0000","last_became_peered":"2024-09-10T18:52:14.576152+0000","last_unstale":"2024-09-10T18:53:26.209684+0000","last_undegraded":"2024-09-10T18:53:26.209684+0000","last_fullsized":"2024-09-10T18:53:26.209684+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:52:54.096066+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:52:54.096066+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:02.123770+0000","scrub_duration":0.0012293219999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"84'18","reported_seq":211,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.232779+0000","last_change":"2024-09-10T18:51:37.059537+0000","last_active":"2024-09-10T18:53:28.232779+0000","last_peered":"2024-09-10T18:53:28.232779+0000","last_clean":"2024-09-10T18:53:28.232779+0000","last_became_active":"2024-09-10T18:45:20.058435+0000","last_became_peered":"2024-09-10T18:45:20.058435+0000","last_unstale":"2024-09-10T18:53:28.232779+0000","last_undegraded":"2024-09-10T18:53:28.232779+0000","last_fullsized":"2024-09-10T18:53:28.232779+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'18","last_scrub_stamp":"2024-09-10T18:51:37.059477+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-10T18:47:10.514951+0000","last_clean_scrub_stamp":"2024-09-10T18:51:37.059477+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020902970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"84'6","reported_seq":688,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-10T18:53:31.239453+0000","last_change":"2024-09-10T18:51:49.096654+0000","last_active":"2024-09-10T18:53:31.239453+0000","last_peered":"2024-09-10T18:53:31.239453+0000","last_clean":"2024-09-10T18:53:31.239453+0000","last_became_active":"2024-09-10T18:47:36.418211+0000","last_became_peered":"2024-09-10T18:47:36.418211+0000","last_unstale":"2024-09-10T18:53:31.239453+0000","last_undegraded":"2024-09-10T18:53:31.239453+0000","last_fullsized":"2024-09-10T18:53:31.239453+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'6","last_scrub_stamp":"2024-09-10T18:51:49.096535+0000","last_deep_scrub":"84'6","last_deep_scrub_stamp":"2024-09-10T18:47:22.545450+0000","last_clean_scrub_stamp":"2024-09-10T18:51:49.096535+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037496439999999999,"objects_trimmed":0,"snaptrim_duration":0.0021933159999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":483,"num_read_kb":478,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":142,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.237118+0000","last_change":"2024-09-10T18:51:44.101630+0000","last_active":"2024-09-10T18:53:28.237118+0000","last_peered":"2024-09-10T18:53:28.237118+0000","last_clean":"2024-09-10T18:53:28.237118+0000","last_became_active":"2024-09-10T18:47:36.422641+0000","last_became_peered":"2024-09-10T18:47:36.422641+0000","last_unstale":"2024-09-10T18:53:28.237118+0000","last_undegraded":"2024-09-10T18:53:28.237118+0000","last_fullsized":"2024-09-10T18:53:28.237118+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:44.101575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:44.101575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012333190000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":150,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.216206+0000","last_change":"2024-09-10T18:51:57.206683+0000","last_active":"2024-09-10T18:53:26.216206+0000","last_peered":"2024-09-10T18:53:26.216206+0000","last_clean":"2024-09-10T18:53:26.216206+0000","last_became_active":"2024-09-10T18:47:36.419081+0000","last_became_peered":"2024-09-10T18:47:36.419081+0000","last_unstale":"2024-09-10T18:53:26.216206+0000","last_undegraded":"2024-09-10T18:53:26.216206+0000","last_fullsized":"2024-09-10T18:53:26.216206+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:57.206586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:57.206586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00597548,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":184,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.211977+0000","last_change":"2024-09-10T18:53:07.150581+0000","last_active":"2024-09-10T18:53:26.211977+0000","last_peered":"2024-09-10T18:53:26.211977+0000","last_clean":"2024-09-10T18:53:26.211977+0000","last_became_active":"2024-09-10T18:45:24.264617+0000","last_became_peered":"2024-09-10T18:45:24.264617+0000","last_unstale":"2024-09-10T18:53:26.211977+0000","last_undegraded":"2024-09-10T18:53:26.211977+0000","last_fullsized":"2024-09-10T18:53:26.211977+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:07.150480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:07.150480+0000","last_clean_scrub_stamp":"2024-09-10T18:53:07.150480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:26.750313+0000","scrub_duration":0.021644442,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"198'10032","reported_seq":10098,"reported_epoch":198,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-10T18:53:31.352217+0000","last_change":"2024-09-10T18:53:25.637984+0000","last_active":"2024-09-10T18:53:31.352217+0000","last_peered":"2024-09-10T18:53:31.352217+0000","last_clean":"2024-09-10T18:52:53.577415+0000","last_became_active":"2024-09-10T18:52:55.625611+0000","last_became_peered":"2024-09-10T18:52:55.625611+0000","last_unstale":"2024-09-10T18:53:31.352217+0000","last_undegraded":"2024-09-10T18:53:31.352217+0000","last_fullsized":"2024-09-10T18:53:31.352217+0000","mapping_epoch":184,"log_start":"178'6093","ondisk_log_start":"178'6093","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":3939,"log_dups_size":0,"ondisk_log_size":3939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":548012032,"num_objects":8362,"num_object_clones":0,"num_object_copies":25086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":8362,"num_objects_unfound":0,"num_objects_dirty":8362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10032,"num_write_kb":588608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,5],"avail_no_missing":["1","3","5"],"object_location_counts":[{"shards":"1,3,5","objects":8362}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"70'81","reported_seq":108,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.232689+0000","last_change":"2024-09-10T18:53:00.338829+0000","last_active":"2024-09-10T18:53:28.232689+0000","last_peered":"2024-09-10T18:53:28.232689+0000","last_clean":"2024-09-10T18:53:28.232689+0000","last_became_active":"2024-09-10T18:52:14.589009+0000","last_became_peered":"2024-09-10T18:52:14.589009+0000","last_unstale":"2024-09-10T18:53:28.232689+0000","last_undegraded":"2024-09-10T18:53:28.232689+0000","last_fullsized":"2024-09-10T18:53:28.232689+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:53:00.338611+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:53:00.338611+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:24.997016+0000","scrub_duration":0.0023458709999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"63'12","reported_seq":231,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.228439+0000","last_change":"2024-09-10T18:52:01.328924+0000","last_active":"2024-09-10T18:53:28.228439+0000","last_peered":"2024-09-10T18:53:28.228439+0000","last_clean":"2024-09-10T18:53:28.228439+0000","last_became_active":"2024-09-10T18:47:36.417965+0000","last_became_peered":"2024-09-10T18:47:36.417965+0000","last_unstale":"2024-09-10T18:53:28.228439+0000","last_undegraded":"2024-09-10T18:53:28.228439+0000","last_fullsized":"2024-09-10T18:53:28.228439+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:52:01.328828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:52:01.328828+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.020091725000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"58'1","reported_seq":187,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.225548+0000","last_change":"2024-09-10T18:51:39.175354+0000","last_active":"2024-09-10T18:53:28.225548+0000","last_peered":"2024-09-10T18:53:28.225548+0000","last_clean":"2024-09-10T18:53:28.225548+0000","last_became_active":"2024-09-10T18:45:17.963932+0000","last_became_peered":"2024-09-10T18:45:17.963932+0000","last_unstale":"2024-09-10T18:53:28.225548+0000","last_undegraded":"2024-09-10T18:53:28.225548+0000","last_fullsized":"2024-09-10T18:53:28.225548+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'1","last_scrub_stamp":"2024-09-10T18:51:39.175290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:39.175290+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015267550000000001,"objects_trimmed":0,"snaptrim_duration":0.00037812099999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":148,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.415910+0000","last_change":"2024-09-10T18:53:28.415720+0000","last_active":"2024-09-10T18:53:28.415910+0000","last_peered":"2024-09-10T18:53:28.415910+0000","last_clean":"2024-09-10T18:53:28.415910+0000","last_became_active":"2024-09-10T18:47:36.416641+0000","last_became_peered":"2024-09-10T18:47:36.416641+0000","last_unstale":"2024-09-10T18:53:28.415910+0000","last_undegraded":"2024-09-10T18:53:28.415910+0000","last_fullsized":"2024-09-10T18:53:28.415910+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:28.415315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:53:28.415315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:51.063863+0000","scrub_duration":0.001771613,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":27,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.231901+0000","last_change":"2024-09-10T18:52:54.622307+0000","last_active":"2024-09-10T18:53:28.231901+0000","last_peered":"2024-09-10T18:53:28.231901+0000","last_clean":"2024-09-10T18:53:28.231901+0000","last_became_active":"2024-09-10T18:52:54.621835+0000","last_became_peered":"2024-09-10T18:52:54.621835+0000","last_unstale":"2024-09-10T18:53:28.231901+0000","last_undegraded":"2024-09-10T18:53:28.231901+0000","last_fullsized":"2024-09-10T18:53:28.231901+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:41.150673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:52:41.150673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:55.571455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":178,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:29.342123+0000","last_change":"2024-09-10T18:53:29.342100+0000","last_active":"2024-09-10T18:53:29.342123+0000","last_peered":"2024-09-10T18:53:29.342123+0000","last_clean":"2024-09-10T18:53:29.342123+0000","last_became_active":"2024-09-10T18:45:24.083689+0000","last_became_peered":"2024-09-10T18:45:24.083689+0000","last_unstale":"2024-09-10T18:53:29.342123+0000","last_undegraded":"2024-09-10T18:53:29.342123+0000","last_fullsized":"2024-09-10T18:53:29.342123+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:50:53.940489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:50:53.940489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007171691,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"197'9900","reported_seq":11438,"reported_epoch":197,"state":"active+remapped+backfilling","last_fresh":"2024-09-10T18:53:31.225524+0000","last_change":"2024-09-10T18:53:25.697733+0000","last_active":"2024-09-10T18:53:31.225524+0000","last_peered":"2024-09-10T18:53:31.225524+0000","last_clean":"2024-09-10T18:52:53.571923+0000","last_became_active":"2024-09-10T18:52:55.662031+0000","last_became_peered":"2024-09-10T18:52:55.662031+0000","last_unstale":"2024-09-10T18:53:31.225524+0000","last_undegraded":"2024-09-10T18:53:31.225524+0000","last_fullsized":"2024-09-10T18:53:31.225524+0000","mapping_epoch":184,"log_start":"176'5753","ondisk_log_start":"176'5753","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":4147,"log_dups_size":0,"ondisk_log_size":4147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":547618816,"num_objects":8356,"num_object_clones":0,"num_object_copies":25068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7636,"num_objects_unfound":0,"num_objects_dirty":8356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9900,"num_write_kb":584192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":737,"num_bytes_recovered":48300032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,5],"avail_no_missing":["6","0","5"],"object_location_counts":[{"shards":"0,5,6","objects":8356}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"126'21","reported_seq":216,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.223255+0000","last_change":"2024-09-10T18:51:52.576493+0000","last_active":"2024-09-10T18:53:26.223255+0000","last_peered":"2024-09-10T18:53:26.223255+0000","last_clean":"2024-09-10T18:53:26.223255+0000","last_became_active":"2024-09-10T18:45:20.421119+0000","last_became_peered":"2024-09-10T18:45:20.421119+0000","last_unstale":"2024-09-10T18:53:26.223255+0000","last_undegraded":"2024-09-10T18:53:26.223255+0000","last_fullsized":"2024-09-10T18:53:26.223255+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"126'21","last_scrub_stamp":"2024-09-10T18:51:52.576358+0000","last_deep_scrub":"126'21","last_deep_scrub_stamp":"2024-09-10T18:50:03.380902+0000","last_clean_scrub_stamp":"2024-09-10T18:51:52.576358+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0054386800000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"60'2","reported_seq":194,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.241083+0000","last_change":"2024-09-10T18:53:07.806328+0000","last_active":"2024-09-10T18:53:28.241083+0000","last_peered":"2024-09-10T18:53:28.241083+0000","last_clean":"2024-09-10T18:53:28.241083+0000","last_became_active":"2024-09-10T18:45:17.961660+0000","last_became_peered":"2024-09-10T18:45:17.961660+0000","last_unstale":"2024-09-10T18:53:28.241083+0000","last_undegraded":"2024-09-10T18:53:28.241083+0000","last_fullsized":"2024-09-10T18:53:28.241083+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-10T18:51:03.104545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:03.104545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0070342290000000003,"objects_trimmed":0,"snaptrim_duration":0.0013942640000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":199,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.215799+0000","last_change":"2024-09-10T18:52:01.209203+0000","last_active":"2024-09-10T18:53:26.215799+0000","last_peered":"2024-09-10T18:53:26.215799+0000","last_clean":"2024-09-10T18:53:26.215799+0000","last_became_active":"2024-09-10T18:47:36.414636+0000","last_became_peered":"2024-09-10T18:47:36.414636+0000","last_unstale":"2024-09-10T18:53:26.215799+0000","last_undegraded":"2024-09-10T18:53:26.215799+0000","last_fullsized":"2024-09-10T18:53:26.215799+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:01.209144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:52:01.209144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0033612889999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'81","reported_seq":255,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.215934+0000","last_change":"2024-09-10T18:52:56.085734+0000","last_active":"2024-09-10T18:53:26.215934+0000","last_peered":"2024-09-10T18:53:26.215934+0000","last_clean":"2024-09-10T18:53:26.215934+0000","last_became_active":"2024-09-10T18:52:14.579154+0000","last_became_peered":"2024-09-10T18:52:14.579154+0000","last_unstale":"2024-09-10T18:53:26.215934+0000","last_undegraded":"2024-09-10T18:53:26.215934+0000","last_fullsized":"2024-09-10T18:53:26.215934+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:52:56.085580+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:56.085580+0000","last_clean_scrub_stamp":"2024-09-10T18:52:56.085580+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:04.343246+0000","scrub_duration":0.012963819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":196,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.210112+0000","last_change":"2024-09-10T18:51:40.327714+0000","last_active":"2024-09-10T18:53:26.210112+0000","last_peered":"2024-09-10T18:53:26.210112+0000","last_clean":"2024-09-10T18:53:26.210112+0000","last_became_active":"2024-09-10T18:47:36.418320+0000","last_became_peered":"2024-09-10T18:47:36.418320+0000","last_unstale":"2024-09-10T18:53:26.210112+0000","last_undegraded":"2024-09-10T18:53:26.210112+0000","last_fullsized":"2024-09-10T18:53:26.210112+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:40.327645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:40.327645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001391827,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":183,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.215461+0000","last_change":"2024-09-10T18:52:51.111361+0000","last_active":"2024-09-10T18:53:26.215461+0000","last_peered":"2024-09-10T18:53:26.215461+0000","last_clean":"2024-09-10T18:53:26.215461+0000","last_became_active":"2024-09-10T18:45:24.073327+0000","last_became_peered":"2024-09-10T18:45:24.073327+0000","last_unstale":"2024-09-10T18:53:26.215461+0000","last_undegraded":"2024-09-10T18:53:26.215461+0000","last_fullsized":"2024-09-10T18:53:26.215461+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:51.111278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:52:51.111278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:00.275369+0000","scrub_duration":0.004757987,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.f","version":"70'81","reported_seq":184,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.224240+0000","last_change":"2024-09-10T18:52:53.609959+0000","last_active":"2024-09-10T18:53:26.224240+0000","last_peered":"2024-09-10T18:53:26.224240+0000","last_clean":"2024-09-10T18:53:26.224240+0000","last_became_active":"2024-09-10T18:52:16.598295+0000","last_became_peered":"2024-09-10T18:52:16.598295+0000","last_unstale":"2024-09-10T18:53:26.224240+0000","last_undegraded":"2024-09-10T18:53:26.224240+0000","last_fullsized":"2024-09-10T18:53:26.224240+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:52:53.609885+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:53.609885+0000","last_clean_scrub_stamp":"2024-09-10T18:52:53.609885+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:54.425866+0000","scrub_duration":0.0031868339999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.6","version":"198'10041","reported_seq":10089,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-10T18:53:31.538221+0000","last_change":"2024-09-10T18:51:37.909963+0000","last_active":"2024-09-10T18:53:31.538221+0000","last_peered":"2024-09-10T18:53:31.538221+0000","last_clean":"2024-09-10T18:53:31.538221+0000","last_became_active":"2024-09-10T18:51:37.909738+0000","last_became_peered":"2024-09-10T18:51:37.909738+0000","last_unstale":"2024-09-10T18:53:31.538221+0000","last_undegraded":"2024-09-10T18:53:31.538221+0000","last_fullsized":"2024-09-10T18:53:31.538221+0000","mapping_epoch":158,"log_start":"176'5623","ondisk_log_start":"176'5623","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":4418,"log_dups_size":0,"ondisk_log_size":4418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":547291136,"num_objects":8351,"num_object_clones":0,"num_object_copies":25053,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10041,"num_write_kb":588544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"63'19","reported_seq":218,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.222431+0000","last_change":"2024-09-10T18:52:03.486438+0000","last_active":"2024-09-10T18:53:26.222431+0000","last_peered":"2024-09-10T18:53:26.222431+0000","last_clean":"2024-09-10T18:53:26.222431+0000","last_became_active":"2024-09-10T18:45:20.421097+0000","last_became_peered":"2024-09-10T18:45:20.421097+0000","last_unstale":"2024-09-10T18:53:26.222431+0000","last_undegraded":"2024-09-10T18:53:26.222431+0000","last_fullsized":"2024-09-10T18:53:26.222431+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:52:03.486356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:52:03.486356+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:30.272925+0000","scrub_duration":0.013086754000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":150,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.215290+0000","last_change":"2024-09-10T18:52:59.223015+0000","last_active":"2024-09-10T18:53:26.215290+0000","last_peered":"2024-09-10T18:53:26.215290+0000","last_clean":"2024-09-10T18:53:26.215290+0000","last_became_active":"2024-09-10T18:47:36.417800+0000","last_became_peered":"2024-09-10T18:47:36.417800+0000","last_unstale":"2024-09-10T18:53:26.215290+0000","last_undegraded":"2024-09-10T18:53:26.215290+0000","last_fullsized":"2024-09-10T18:53:26.215290+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:59.222933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:52:59.222933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:21.728076+0000","scrub_duration":0.012835052,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":162,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.226064+0000","last_change":"2024-09-10T18:52:54.621815+0000","last_active":"2024-09-10T18:53:28.226064+0000","last_peered":"2024-09-10T18:53:28.226064+0000","last_clean":"2024-09-10T18:53:28.226064+0000","last_became_active":"2024-09-10T18:52:54.621449+0000","last_became_peered":"2024-09-10T18:52:54.621449+0000","last_unstale":"2024-09-10T18:53:28.226064+0000","last_undegraded":"2024-09-10T18:53:28.226064+0000","last_fullsized":"2024-09-10T18:53:28.226064+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:40.126652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:40.126652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017754699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":187,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:29.370446+0000","last_change":"2024-09-10T18:53:29.370446+0000","last_active":"2024-09-10T18:53:29.370446+0000","last_peered":"2024-09-10T18:53:29.370446+0000","last_clean":"2024-09-10T18:53:29.370446+0000","last_became_active":"2024-09-10T18:47:36.412610+0000","last_became_peered":"2024-09-10T18:47:36.412610+0000","last_unstale":"2024-09-10T18:53:29.370446+0000","last_undegraded":"2024-09-10T18:53:29.370446+0000","last_fullsized":"2024-09-10T18:53:29.370446+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:38.318887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:38.318887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001335461,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.e","version":"70'81","reported_seq":47,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.205943+0000","last_change":"2024-09-10T18:52:56.070162+0000","last_active":"2024-09-10T18:53:26.205943+0000","last_peered":"2024-09-10T18:53:26.205943+0000","last_clean":"2024-09-10T18:53:26.205943+0000","last_became_active":"2024-09-10T18:52:16.604236+0000","last_became_peered":"2024-09-10T18:52:16.604236+0000","last_unstale":"2024-09-10T18:53:26.205943+0000","last_undegraded":"2024-09-10T18:53:26.205943+0000","last_fullsized":"2024-09-10T18:53:26.205943+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:52:56.069927+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:52:56.069927+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:01.069879+0000","scrub_duration":0.0061090470000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"198'9862","reported_seq":9910,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-10T18:53:31.381873+0000","last_change":"2024-09-10T18:51:37.590054+0000","last_active":"2024-09-10T18:53:31.381873+0000","last_peered":"2024-09-10T18:53:31.381873+0000","last_clean":"2024-09-10T18:53:31.381873+0000","last_became_active":"2024-09-10T18:51:37.589351+0000","last_became_peered":"2024-09-10T18:51:37.589351+0000","last_unstale":"2024-09-10T18:53:31.381873+0000","last_undegraded":"2024-09-10T18:53:31.381873+0000","last_fullsized":"2024-09-10T18:53:31.381873+0000","mapping_epoch":158,"log_start":"177'5880","ondisk_log_start":"177'5880","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":3982,"log_dups_size":0,"ondisk_log_size":3982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":543031296,"num_objects":8286,"num_object_clones":0,"num_object_copies":24858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9862,"num_write_kb":580736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"130'13","reported_seq":209,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.226320+0000","last_change":"2024-09-10T18:51:58.427843+0000","last_active":"2024-09-10T18:53:28.226320+0000","last_peered":"2024-09-10T18:53:28.226320+0000","last_clean":"2024-09-10T18:53:28.226320+0000","last_became_active":"2024-09-10T18:45:20.061021+0000","last_became_peered":"2024-09-10T18:45:20.061021+0000","last_unstale":"2024-09-10T18:53:28.226320+0000","last_undegraded":"2024-09-10T18:53:28.226320+0000","last_fullsized":"2024-09-10T18:53:28.226320+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"130'13","last_scrub_stamp":"2024-09-10T18:51:58.427534+0000","last_deep_scrub":"130'13","last_deep_scrub_stamp":"2024-09-10T18:51:58.427534+0000","last_clean_scrub_stamp":"2024-09-10T18:51:58.427534+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051175980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":181,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.230709+0000","last_change":"2024-09-10T18:51:31.128571+0000","last_active":"2024-09-10T18:53:28.230709+0000","last_peered":"2024-09-10T18:53:28.230709+0000","last_clean":"2024-09-10T18:53:28.230709+0000","last_became_active":"2024-09-10T18:45:17.963057+0000","last_became_peered":"2024-09-10T18:45:17.963057+0000","last_unstale":"2024-09-10T18:53:28.230709+0000","last_undegraded":"2024-09-10T18:53:28.230709+0000","last_fullsized":"2024-09-10T18:53:28.230709+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:31.128514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:31.128514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016209180000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":139,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.206110+0000","last_change":"2024-09-10T18:51:35.325521+0000","last_active":"2024-09-10T18:53:26.206110+0000","last_peered":"2024-09-10T18:53:26.206110+0000","last_clean":"2024-09-10T18:53:26.206110+0000","last_became_active":"2024-09-10T18:47:36.411517+0000","last_became_peered":"2024-09-10T18:47:36.411517+0000","last_unstale":"2024-09-10T18:53:26.206110+0000","last_undegraded":"2024-09-10T18:53:26.206110+0000","last_fullsized":"2024-09-10T18:53:26.206110+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:35.325451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_clean_scrub_stamp":"2024-09-10T18:51:35.325451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018247820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":195,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.225447+0000","last_change":"2024-09-10T18:51:53.345267+0000","last_active":"2024-09-10T18:53:28.225447+0000","last_peered":"2024-09-10T18:53:28.225447+0000","last_clean":"2024-09-10T18:53:28.225447+0000","last_became_active":"2024-09-10T18:47:36.412933+0000","last_became_peered":"2024-09-10T18:47:36.412933+0000","last_unstale":"2024-09-10T18:53:28.225447+0000","last_undegraded":"2024-09-10T18:53:28.225447+0000","last_fullsized":"2024-09-10T18:53:28.225447+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:53.345184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:53.345184+0000","last_clean_scrub_stamp":"2024-09-10T18:51:53.345184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003832687,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.d","version":"70'81","reported_seq":47,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.210993+0000","last_change":"2024-09-10T18:53:01.067942+0000","last_active":"2024-09-10T18:53:26.210993+0000","last_peered":"2024-09-10T18:53:26.210993+0000","last_clean":"2024-09-10T18:53:26.210993+0000","last_became_active":"2024-09-10T18:52:16.605492+0000","last_became_peered":"2024-09-10T18:52:16.605492+0000","last_unstale":"2024-09-10T18:53:26.210993+0000","last_undegraded":"2024-09-10T18:53:26.210993+0000","last_fullsized":"2024-09-10T18:53:26.210993+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:53:01.067794+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:53:01.067794+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:22.137189+0000","scrub_duration":0.0033709180000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.4","version":"197'10057","reported_seq":10107,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:31.177769+0000","last_change":"2024-09-10T18:53:29.974359+0000","last_active":"2024-09-10T18:53:31.177769+0000","last_peered":"2024-09-10T18:53:31.177769+0000","last_clean":"2024-09-10T18:53:31.177769+0000","last_became_active":"2024-09-10T18:51:37.590189+0000","last_became_peered":"2024-09-10T18:51:37.590189+0000","last_unstale":"2024-09-10T18:53:31.177769+0000","last_undegraded":"2024-09-10T18:53:31.177769+0000","last_fullsized":"2024-09-10T18:53:31.177769+0000","mapping_epoch":158,"log_start":"173'4600","ondisk_log_start":"173'4600","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.561398+0000","objects_scrubbed":0,"log_size":5457,"log_dups_size":0,"ondisk_log_size":5457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":547684352,"num_objects":8357,"num_object_clones":0,"num_object_copies":25071,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10057,"num_write_kb":589248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"63'15","reported_seq":79,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.213265+0000","last_change":"2024-09-10T18:52:54.587738+0000","last_active":"2024-09-10T18:53:26.213265+0000","last_peered":"2024-09-10T18:53:26.213265+0000","last_clean":"2024-09-10T18:53:26.213265+0000","last_became_active":"2024-09-10T18:52:54.579763+0000","last_became_peered":"2024-09-10T18:52:54.579763+0000","last_unstale":"2024-09-10T18:53:26.213265+0000","last_undegraded":"2024-09-10T18:53:26.213265+0000","last_fullsized":"2024-09-10T18:53:26.213265+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:51:36.277517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.277517+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":142,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.236667+0000","last_change":"2024-09-10T18:52:44.851169+0000","last_active":"2024-09-10T18:53:28.236667+0000","last_peered":"2024-09-10T18:53:28.236667+0000","last_clean":"2024-09-10T18:53:28.236667+0000","last_became_active":"2024-09-10T18:47:36.420475+0000","last_became_peered":"2024-09-10T18:47:36.420475+0000","last_unstale":"2024-09-10T18:53:28.236667+0000","last_undegraded":"2024-09-10T18:53:28.236667+0000","last_fullsized":"2024-09-10T18:53:28.236667+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:44.851096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:05.603125+0000","last_clean_scrub_stamp":"2024-09-10T18:52:44.851096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:13.934457+0000","scrub_duration":0.0054082180000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"83'9","reported_seq":296,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.210193+0000","last_change":"2024-09-10T18:52:40.259700+0000","last_active":"2024-09-10T18:53:26.210193+0000","last_peered":"2024-09-10T18:53:26.210193+0000","last_clean":"2024-09-10T18:53:26.210193+0000","last_became_active":"2024-09-10T18:45:22.073383+0000","last_became_peered":"2024-09-10T18:45:22.073383+0000","last_unstale":"2024-09-10T18:53:26.210193+0000","last_undegraded":"2024-09-10T18:53:26.210193+0000","last_fullsized":"2024-09-10T18:53:26.210193+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'9","last_scrub_stamp":"2024-09-10T18:52:40.259614+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-09-10T18:47:02.582229+0000","last_clean_scrub_stamp":"2024-09-10T18:52:40.259614+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:58.450162+0000","scrub_duration":0.010303679,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":183,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.216846+0000","last_change":"2024-09-10T18:52:48.093291+0000","last_active":"2024-09-10T18:53:26.216846+0000","last_peered":"2024-09-10T18:53:26.216846+0000","last_clean":"2024-09-10T18:53:26.216846+0000","last_became_active":"2024-09-10T18:45:24.265256+0000","last_became_peered":"2024-09-10T18:45:24.265256+0000","last_unstale":"2024-09-10T18:53:26.216846+0000","last_undegraded":"2024-09-10T18:53:26.216846+0000","last_fullsized":"2024-09-10T18:53:26.216846+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:48.093216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_clean_scrub_stamp":"2024-09-10T18:52:48.093216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:02.506826+0000","scrub_duration":0.0065930010000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.c","version":"70'81","reported_seq":128,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.236174+0000","last_change":"2024-09-10T18:53:05.814914+0000","last_active":"2024-09-10T18:53:28.236174+0000","last_peered":"2024-09-10T18:53:28.236174+0000","last_clean":"2024-09-10T18:53:28.236174+0000","last_became_active":"2024-09-10T18:52:16.596677+0000","last_became_peered":"2024-09-10T18:52:16.596677+0000","last_unstale":"2024-09-10T18:53:28.236174+0000","last_undegraded":"2024-09-10T18:53:28.236174+0000","last_fullsized":"2024-09-10T18:53:28.236174+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:53:05.814851+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:53:05.814851+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:28.213122+0000","scrub_duration":0.0025032510000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"197'10058","reported_seq":10420,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:31.205152+0000","last_change":"2024-09-10T18:53:00.842427+0000","last_active":"2024-09-10T18:53:31.205152+0000","last_peered":"2024-09-10T18:53:31.205152+0000","last_clean":"2024-09-10T18:53:31.205152+0000","last_became_active":"2024-09-10T18:51:37.910181+0000","last_became_peered":"2024-09-10T18:51:37.910181+0000","last_unstale":"2024-09-10T18:53:31.205152+0000","last_undegraded":"2024-09-10T18:53:31.205152+0000","last_fullsized":"2024-09-10T18:53:31.205152+0000","mapping_epoch":158,"log_start":"177'5985","ondisk_log_start":"177'5985","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'7800","last_scrub_stamp":"2024-09-10T18:53:00.842273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:36.561398+0000","last_clean_scrub_stamp":"2024-09-10T18:53:00.842273+0000","objects_scrubbed":7641,"log_size":4073,"log_dups_size":0,"ondisk_log_size":4073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:11.339750+0000","scrub_duration":3.2607367800000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":552599552,"num_objects":8432,"num_object_clones":0,"num_object_copies":25296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10058,"num_write_kb":591680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"63'12","reported_seq":209,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.208607+0000","last_change":"2024-09-10T18:52:40.240222+0000","last_active":"2024-09-10T18:53:26.208607+0000","last_peered":"2024-09-10T18:53:26.208607+0000","last_clean":"2024-09-10T18:53:26.208607+0000","last_became_active":"2024-09-10T18:45:20.056869+0000","last_became_peered":"2024-09-10T18:45:20.056869+0000","last_unstale":"2024-09-10T18:53:26.208607+0000","last_undegraded":"2024-09-10T18:53:26.208607+0000","last_fullsized":"2024-09-10T18:53:26.208607+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:52:40.240135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:52:40.240135+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:56.713004+0000","scrub_duration":0.007644624,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"56'2","reported_seq":195,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.210444+0000","last_change":"2024-09-10T18:51:42.425065+0000","last_active":"2024-09-10T18:53:26.210444+0000","last_peered":"2024-09-10T18:53:26.210444+0000","last_clean":"2024-09-10T18:53:26.210444+0000","last_became_active":"2024-09-10T18:45:17.963286+0000","last_became_peered":"2024-09-10T18:45:17.963286+0000","last_unstale":"2024-09-10T18:53:26.210444+0000","last_undegraded":"2024-09-10T18:53:26.210444+0000","last_fullsized":"2024-09-10T18:53:26.210444+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-10T18:51:42.424994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:42.424994+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001852226,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":162,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.224723+0000","last_change":"2024-09-10T18:53:21.169394+0000","last_active":"2024-09-10T18:53:28.224723+0000","last_peered":"2024-09-10T18:53:28.224723+0000","last_clean":"2024-09-10T18:53:28.224723+0000","last_became_active":"2024-09-10T18:53:21.168890+0000","last_became_peered":"2024-09-10T18:53:21.168890+0000","last_unstale":"2024-09-10T18:53:28.224723+0000","last_undegraded":"2024-09-10T18:53:28.224723+0000","last_fullsized":"2024-09-10T18:53:28.224723+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:54.342934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:02.711594+0000","last_clean_scrub_stamp":"2024-09-10T18:51:54.342934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005738529,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":175,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.237442+0000","last_change":"2024-09-10T18:51:48.065441+0000","last_active":"2024-09-10T18:53:28.237442+0000","last_peered":"2024-09-10T18:53:28.237442+0000","last_clean":"2024-09-10T18:53:28.237442+0000","last_became_active":"2024-09-10T18:45:24.077815+0000","last_became_peered":"2024-09-10T18:45:24.077815+0000","last_unstale":"2024-09-10T18:53:28.237442+0000","last_undegraded":"2024-09-10T18:53:28.237442+0000","last_fullsized":"2024-09-10T18:53:28.237442+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:48.065385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:48.065385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014615870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.13","version":"70'81","reported_seq":117,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.232838+0000","last_change":"2024-09-10T18:53:06.263065+0000","last_active":"2024-09-10T18:53:28.232838+0000","last_peered":"2024-09-10T18:53:28.232838+0000","last_clean":"2024-09-10T18:53:28.232838+0000","last_became_active":"2024-09-10T18:52:16.602463+0000","last_became_peered":"2024-09-10T18:52:16.602463+0000","last_unstale":"2024-09-10T18:53:28.232838+0000","last_undegraded":"2024-09-10T18:53:28.232838+0000","last_fullsized":"2024-09-10T18:53:28.232838+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:53:06.262959+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:53:06.262959+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:23.661349+0000","scrub_duration":0.0036690479999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"76'13","reported_seq":209,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.228366+0000","last_change":"2024-09-10T18:51:56.389512+0000","last_active":"2024-09-10T18:53:28.228366+0000","last_peered":"2024-09-10T18:53:28.228366+0000","last_clean":"2024-09-10T18:53:28.228366+0000","last_became_active":"2024-09-10T18:45:20.420421+0000","last_became_peered":"2024-09-10T18:45:20.420421+0000","last_unstale":"2024-09-10T18:53:28.228366+0000","last_undegraded":"2024-09-10T18:53:28.228366+0000","last_fullsized":"2024-09-10T18:53:28.228366+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'13","last_scrub_stamp":"2024-09-10T18:51:56.389281+0000","last_deep_scrub":"76'13","last_deep_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_clean_scrub_stamp":"2024-09-10T18:51:56.389281+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0031275529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":152,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.229577+0000","last_change":"2024-09-10T18:52:05.160227+0000","last_active":"2024-09-10T18:53:28.229577+0000","last_peered":"2024-09-10T18:53:28.229577+0000","last_clean":"2024-09-10T18:53:28.229577+0000","last_became_active":"2024-09-10T18:47:36.409299+0000","last_became_peered":"2024-09-10T18:47:36.409299+0000","last_unstale":"2024-09-10T18:53:28.229577+0000","last_undegraded":"2024-09-10T18:53:28.229577+0000","last_fullsized":"2024-09-10T18:53:28.229577+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:05.160144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:52:05.160144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041652920000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":187,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.239070+0000","last_change":"2024-09-10T18:53:00.755016+0000","last_active":"2024-09-10T18:53:28.239070+0000","last_peered":"2024-09-10T18:53:28.239070+0000","last_clean":"2024-09-10T18:53:28.239070+0000","last_became_active":"2024-09-10T18:45:22.069040+0000","last_became_peered":"2024-09-10T18:45:22.069040+0000","last_unstale":"2024-09-10T18:53:28.239070+0000","last_undegraded":"2024-09-10T18:53:28.239070+0000","last_fullsized":"2024-09-10T18:53:28.239070+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:00.754912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:00.754912+0000","last_clean_scrub_stamp":"2024-09-10T18:53:00.754912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:16.944355+0000","scrub_duration":0.0075269270000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"63'1","reported_seq":155,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.211452+0000","last_change":"2024-09-10T18:51:57.216173+0000","last_active":"2024-09-10T18:53:26.211452+0000","last_peered":"2024-09-10T18:53:26.211452+0000","last_clean":"2024-09-10T18:53:26.211452+0000","last_became_active":"2024-09-10T18:47:36.406936+0000","last_became_peered":"2024-09-10T18:47:36.406936+0000","last_unstale":"2024-09-10T18:53:26.211452+0000","last_undegraded":"2024-09-10T18:53:26.211452+0000","last_fullsized":"2024-09-10T18:53:26.211452+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-10T18:51:57.216086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:57.216086+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048352930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.12","version":"70'81","reported_seq":47,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.217079+0000","last_change":"2024-09-10T18:52:54.127527+0000","last_active":"2024-09-10T18:53:26.217079+0000","last_peered":"2024-09-10T18:53:26.217079+0000","last_clean":"2024-09-10T18:53:26.217079+0000","last_became_active":"2024-09-10T18:52:16.598921+0000","last_became_peered":"2024-09-10T18:52:16.598921+0000","last_unstale":"2024-09-10T18:53:26.217079+0000","last_undegraded":"2024-09-10T18:53:26.217079+0000","last_fullsized":"2024-09-10T18:53:26.217079+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:52:54.127476+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:54.127476+0000","last_clean_scrub_stamp":"2024-09-10T18:52:54.127476+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:08.088523+0000","scrub_duration":0.0011557379999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":189,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.229379+0000","last_change":"2024-09-10T18:52:46.673283+0000","last_active":"2024-09-10T18:53:26.229379+0000","last_peered":"2024-09-10T18:53:26.229379+0000","last_clean":"2024-09-10T18:53:26.229379+0000","last_became_active":"2024-09-10T18:45:17.964705+0000","last_became_peered":"2024-09-10T18:45:17.964705+0000","last_unstale":"2024-09-10T18:53:26.229379+0000","last_undegraded":"2024-09-10T18:53:26.229379+0000","last_fullsized":"2024-09-10T18:53:26.229379+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:46.673208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:52:46.673208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:56.093373+0000","scrub_duration":0.020993344000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'6","reported_seq":188,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.239279+0000","last_change":"2024-09-10T18:51:43.085607+0000","last_active":"2024-09-10T18:53:28.239279+0000","last_peered":"2024-09-10T18:53:28.239279+0000","last_clean":"2024-09-10T18:53:28.239279+0000","last_became_active":"2024-09-10T18:45:20.421746+0000","last_became_peered":"2024-09-10T18:45:20.421746+0000","last_unstale":"2024-09-10T18:53:28.239279+0000","last_undegraded":"2024-09-10T18:53:28.239279+0000","last_fullsized":"2024-09-10T18:53:28.239279+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'6","last_scrub_stamp":"2024-09-10T18:51:43.085549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:43.085549+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002071238,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":185,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.216363+0000","last_change":"2024-09-10T18:53:03.116972+0000","last_active":"2024-09-10T18:53:26.216363+0000","last_peered":"2024-09-10T18:53:26.216363+0000","last_clean":"2024-09-10T18:53:26.216363+0000","last_became_active":"2024-09-10T18:45:22.069653+0000","last_became_peered":"2024-09-10T18:45:22.069653+0000","last_unstale":"2024-09-10T18:53:26.216363+0000","last_undegraded":"2024-09-10T18:53:26.216363+0000","last_fullsized":"2024-09-10T18:53:26.216363+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:03.116899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:53:03.116899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:29.586498+0000","scrub_duration":0.010530331,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":184,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.211905+0000","last_change":"2024-09-10T18:52:48.182345+0000","last_active":"2024-09-10T18:53:26.211905+0000","last_peered":"2024-09-10T18:53:26.211905+0000","last_clean":"2024-09-10T18:53:26.211905+0000","last_became_active":"2024-09-10T18:45:24.264611+0000","last_became_peered":"2024-09-10T18:45:24.264611+0000","last_unstale":"2024-09-10T18:53:26.211905+0000","last_undegraded":"2024-09-10T18:53:26.211905+0000","last_fullsized":"2024-09-10T18:53:26.211905+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:48.182265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:52:48.182265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:05.295246+0000","scrub_duration":0.020719859,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.11","version":"70'81","reported_seq":126,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.225265+0000","last_change":"2024-09-10T18:52:54.623041+0000","last_active":"2024-09-10T18:53:28.225265+0000","last_peered":"2024-09-10T18:53:28.225265+0000","last_clean":"2024-09-10T18:53:28.225265+0000","last_became_active":"2024-09-10T18:52:54.622526+0000","last_became_peered":"2024-09-10T18:52:54.622526+0000","last_unstale":"2024-09-10T18:53:28.225265+0000","last_undegraded":"2024-09-10T18:53:28.225265+0000","last_fullsized":"2024-09-10T18:53:28.225265+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:52:53.556903+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:52:53.556903+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:02.882031+0000","scrub_duration":0.004894626,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"63'10","reported_seq":238,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.235848+0000","last_change":"2024-09-10T18:51:39.074731+0000","last_active":"2024-09-10T18:53:28.235848+0000","last_peered":"2024-09-10T18:53:28.235848+0000","last_clean":"2024-09-10T18:53:28.235848+0000","last_became_active":"2024-09-10T18:47:36.426260+0000","last_became_peered":"2024-09-10T18:47:36.426260+0000","last_unstale":"2024-09-10T18:53:28.235848+0000","last_undegraded":"2024-09-10T18:53:28.235848+0000","last_fullsized":"2024-09-10T18:53:28.235848+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:51:39.074664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:39.074664+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044748410000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":189,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.217172+0000","last_change":"2024-09-10T18:52:40.135242+0000","last_active":"2024-09-10T18:53:26.217172+0000","last_peered":"2024-09-10T18:53:26.217172+0000","last_clean":"2024-09-10T18:53:26.217172+0000","last_became_active":"2024-09-10T18:45:17.964870+0000","last_became_peered":"2024-09-10T18:45:17.964870+0000","last_unstale":"2024-09-10T18:53:26.217172+0000","last_undegraded":"2024-09-10T18:53:26.217172+0000","last_fullsized":"2024-09-10T18:53:26.217172+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:40.135091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:52:40.135091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:08.268094+0000","scrub_duration":0.002735824,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"63'8","reported_seq":292,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.229457+0000","last_change":"2024-09-10T18:52:42.137185+0000","last_active":"2024-09-10T18:53:28.229457+0000","last_peered":"2024-09-10T18:53:28.229457+0000","last_clean":"2024-09-10T18:53:28.229457+0000","last_became_active":"2024-09-10T18:45:22.062940+0000","last_became_peered":"2024-09-10T18:45:22.062940+0000","last_unstale":"2024-09-10T18:53:28.229457+0000","last_undegraded":"2024-09-10T18:53:28.229457+0000","last_fullsized":"2024-09-10T18:53:28.229457+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:52:42.136982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:52:42.136982+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:45.124618+0000","scrub_duration":0.0098948820000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":185,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.226740+0000","last_change":"2024-09-10T18:52:57.476780+0000","last_active":"2024-09-10T18:53:28.226740+0000","last_peered":"2024-09-10T18:53:28.226740+0000","last_clean":"2024-09-10T18:53:28.226740+0000","last_became_active":"2024-09-10T18:45:24.073736+0000","last_became_peered":"2024-09-10T18:45:24.073736+0000","last_unstale":"2024-09-10T18:53:28.226740+0000","last_undegraded":"2024-09-10T18:53:28.226740+0000","last_fullsized":"2024-09-10T18:53:28.226740+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:57.476724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:57.476724+0000","last_clean_scrub_stamp":"2024-09-10T18:52:57.476724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:16.234686+0000","scrub_duration":0.005644067,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.10","version":"70'81","reported_seq":47,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.216169+0000","last_change":"2024-09-10T18:52:57.175458+0000","last_active":"2024-09-10T18:53:26.216169+0000","last_peered":"2024-09-10T18:53:26.216169+0000","last_clean":"2024-09-10T18:53:26.216169+0000","last_became_active":"2024-09-10T18:52:16.601616+0000","last_became_peered":"2024-09-10T18:52:16.601616+0000","last_unstale":"2024-09-10T18:53:26.216169+0000","last_undegraded":"2024-09-10T18:53:26.216169+0000","last_fullsized":"2024-09-10T18:53:26.216169+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:52:57.175370+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:52:57.175370+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:13.659007+0000","scrub_duration":0.0012721060000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"63'9","reported_seq":64,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.212141+0000","last_change":"2024-09-10T18:52:54.624608+0000","last_active":"2024-09-10T18:53:26.212141+0000","last_peered":"2024-09-10T18:53:26.212141+0000","last_clean":"2024-09-10T18:53:26.212141+0000","last_became_active":"2024-09-10T18:52:54.618780+0000","last_became_peered":"2024-09-10T18:52:54.618780+0000","last_unstale":"2024-09-10T18:53:26.212141+0000","last_undegraded":"2024-09-10T18:53:26.212141+0000","last_fullsized":"2024-09-10T18:53:26.212141+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:51:45.423873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:45.423873+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":191,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.226530+0000","last_change":"2024-09-10T18:51:46.194282+0000","last_active":"2024-09-10T18:53:28.226530+0000","last_peered":"2024-09-10T18:53:28.226530+0000","last_clean":"2024-09-10T18:53:28.226530+0000","last_became_active":"2024-09-10T18:45:17.960106+0000","last_became_peered":"2024-09-10T18:45:17.960106+0000","last_unstale":"2024-09-10T18:53:28.226530+0000","last_undegraded":"2024-09-10T18:53:28.226530+0000","last_fullsized":"2024-09-10T18:53:28.226530+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:46.194176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:46.194176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001545888,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"89'10","reported_seq":299,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:31.173797+0000","last_change":"2024-09-10T18:51:46.054778+0000","last_active":"2024-09-10T18:53:31.173797+0000","last_peered":"2024-09-10T18:53:31.173797+0000","last_clean":"2024-09-10T18:53:31.173797+0000","last_became_active":"2024-09-10T18:47:36.424575+0000","last_became_peered":"2024-09-10T18:47:36.424575+0000","last_unstale":"2024-09-10T18:53:31.173797+0000","last_undegraded":"2024-09-10T18:53:31.173797+0000","last_fullsized":"2024-09-10T18:53:31.173797+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'10","last_scrub_stamp":"2024-09-10T18:51:46.054720+0000","last_deep_scrub":"81'9","last_deep_scrub_stamp":"2024-09-10T18:46:54.800790+0000","last_clean_scrub_stamp":"2024-09-10T18:51:46.054720+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001410667,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":192,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.208800+0000","last_change":"2024-09-10T18:52:42.196849+0000","last_active":"2024-09-10T18:53:26.208800+0000","last_peered":"2024-09-10T18:53:26.208800+0000","last_clean":"2024-09-10T18:53:26.208800+0000","last_became_active":"2024-09-10T18:47:36.414047+0000","last_became_peered":"2024-09-10T18:47:36.414047+0000","last_unstale":"2024-09-10T18:53:26.208800+0000","last_undegraded":"2024-09-10T18:53:26.208800+0000","last_fullsized":"2024-09-10T18:53:26.208800+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:42.196733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_clean_scrub_stamp":"2024-09-10T18:52:42.196733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:06.998609+0000","scrub_duration":0.0039644839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"63'9","reported_seq":215,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.229535+0000","last_change":"2024-09-10T18:51:32.146681+0000","last_active":"2024-09-10T18:53:28.229535+0000","last_peered":"2024-09-10T18:53:28.229535+0000","last_clean":"2024-09-10T18:53:28.229535+0000","last_became_active":"2024-09-10T18:47:36.416484+0000","last_became_peered":"2024-09-10T18:47:36.416484+0000","last_unstale":"2024-09-10T18:53:28.229535+0000","last_undegraded":"2024-09-10T18:53:28.229535+0000","last_fullsized":"2024-09-10T18:53:28.229535+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:51:32.146622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:32.146622+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023792729999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":209,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.223094+0000","last_change":"2024-09-10T18:53:03.697342+0000","last_active":"2024-09-10T18:53:26.223094+0000","last_peered":"2024-09-10T18:53:26.223094+0000","last_clean":"2024-09-10T18:53:26.223094+0000","last_became_active":"2024-09-10T18:52:54.620353+0000","last_became_peered":"2024-09-10T18:52:54.620353+0000","last_unstale":"2024-09-10T18:53:26.223094+0000","last_undegraded":"2024-09-10T18:53:26.223094+0000","last_fullsized":"2024-09-10T18:53:26.223094+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:03.697178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:53:03.697178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:11.615002+0000","scrub_duration":0.010029720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":177,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.237511+0000","last_change":"2024-09-10T18:51:47.078689+0000","last_active":"2024-09-10T18:53:28.237511+0000","last_peered":"2024-09-10T18:53:28.237511+0000","last_clean":"2024-09-10T18:53:28.237511+0000","last_became_active":"2024-09-10T18:45:22.072877+0000","last_became_peered":"2024-09-10T18:45:22.072877+0000","last_unstale":"2024-09-10T18:53:28.237511+0000","last_undegraded":"2024-09-10T18:53:28.237511+0000","last_fullsized":"2024-09-10T18:53:28.237511+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:47.078574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:03.571786+0000","last_clean_scrub_stamp":"2024-09-10T18:51:47.078574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001570454,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":198,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.404207+0000","last_change":"2024-09-10T18:53:28.404173+0000","last_active":"2024-09-10T18:53:28.404207+0000","last_peered":"2024-09-10T18:53:28.404207+0000","last_clean":"2024-09-10T18:53:28.404207+0000","last_became_active":"2024-09-10T18:53:21.169239+0000","last_became_peered":"2024-09-10T18:53:21.169239+0000","last_unstale":"2024-09-10T18:53:28.404207+0000","last_undegraded":"2024-09-10T18:53:28.404207+0000","last_fullsized":"2024-09-10T18:53:28.404207+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:47.209023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:04.631558+0000","last_clean_scrub_stamp":"2024-09-10T18:51:47.209023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019148069999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"63'11","reported_seq":195,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.209404+0000","last_change":"2024-09-10T18:52:06.206180+0000","last_active":"2024-09-10T18:53:26.209404+0000","last_peered":"2024-09-10T18:53:26.209404+0000","last_clean":"2024-09-10T18:53:26.209404+0000","last_became_active":"2024-09-10T18:45:20.056885+0000","last_became_peered":"2024-09-10T18:45:20.056885+0000","last_unstale":"2024-09-10T18:53:26.209404+0000","last_undegraded":"2024-09-10T18:53:26.209404+0000","last_fullsized":"2024-09-10T18:53:26.209404+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:52:06.206060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:52:06.206060+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0078519460000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":181,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.230640+0000","last_change":"2024-09-10T18:51:00.029596+0000","last_active":"2024-09-10T18:53:28.230640+0000","last_peered":"2024-09-10T18:53:28.230640+0000","last_clean":"2024-09-10T18:53:28.230640+0000","last_became_active":"2024-09-10T18:45:17.963542+0000","last_became_peered":"2024-09-10T18:45:17.963542+0000","last_unstale":"2024-09-10T18:53:28.230640+0000","last_undegraded":"2024-09-10T18:53:28.230640+0000","last_fullsized":"2024-09-10T18:53:28.230640+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:00.029389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:00.029389+0000","last_clean_scrub_stamp":"2024-09-10T18:51:00.029389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023207890000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":190,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.229397+0000","last_change":"2024-09-10T18:52:43.135942+0000","last_active":"2024-09-10T18:53:28.229397+0000","last_peered":"2024-09-10T18:53:28.229397+0000","last_clean":"2024-09-10T18:53:28.229397+0000","last_became_active":"2024-09-10T18:45:22.070065+0000","last_became_peered":"2024-09-10T18:45:22.070065+0000","last_unstale":"2024-09-10T18:53:28.229397+0000","last_undegraded":"2024-09-10T18:53:28.229397+0000","last_fullsized":"2024-09-10T18:53:28.229397+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:43.135170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:52:43.135170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:04.311899+0000","scrub_duration":0.011350617,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":175,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.237686+0000","last_change":"2024-09-10T18:52:05.098106+0000","last_active":"2024-09-10T18:53:28.237686+0000","last_peered":"2024-09-10T18:53:28.237686+0000","last_clean":"2024-09-10T18:53:28.237686+0000","last_became_active":"2024-09-10T18:45:24.072224+0000","last_became_peered":"2024-09-10T18:45:24.072224+0000","last_unstale":"2024-09-10T18:53:28.237686+0000","last_undegraded":"2024-09-10T18:53:28.237686+0000","last_fullsized":"2024-09-10T18:53:28.237686+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:05.098031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:52:05.098031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0067119739999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"63'4","reported_seq":195,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.243734+0000","last_change":"2024-09-10T18:52:56.832280+0000","last_active":"2024-09-10T18:53:28.243734+0000","last_peered":"2024-09-10T18:53:28.243734+0000","last_clean":"2024-09-10T18:53:28.243734+0000","last_became_active":"2024-09-10T18:45:20.059205+0000","last_became_peered":"2024-09-10T18:45:20.059205+0000","last_unstale":"2024-09-10T18:53:28.243734+0000","last_undegraded":"2024-09-10T18:53:28.243734+0000","last_fullsized":"2024-09-10T18:53:28.243734+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4","last_scrub_stamp":"2024-09-10T18:52:56.832176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:52:56.832176+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:23.717679+0000","scrub_duration":0.015287545,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":189,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.216423+0000","last_change":"2024-09-10T18:51:43.344854+0000","last_active":"2024-09-10T18:53:26.216423+0000","last_peered":"2024-09-10T18:53:26.216423+0000","last_clean":"2024-09-10T18:53:26.216423+0000","last_became_active":"2024-09-10T18:47:36.415866+0000","last_became_peered":"2024-09-10T18:47:36.415866+0000","last_unstale":"2024-09-10T18:53:26.216423+0000","last_undegraded":"2024-09-10T18:53:26.216423+0000","last_fullsized":"2024-09-10T18:53:26.216423+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:43.344751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:43.344751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019741749999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":195,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.229103+0000","last_change":"2024-09-10T18:52:58.103815+0000","last_active":"2024-09-10T18:53:26.229103+0000","last_peered":"2024-09-10T18:53:26.229103+0000","last_clean":"2024-09-10T18:53:26.229103+0000","last_became_active":"2024-09-10T18:52:58.103465+0000","last_became_peered":"2024-09-10T18:52:58.103465+0000","last_unstale":"2024-09-10T18:53:26.229103+0000","last_undegraded":"2024-09-10T18:53:26.229103+0000","last_fullsized":"2024-09-10T18:53:26.229103+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:54.514747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_clean_scrub_stamp":"2024-09-10T18:51:54.514747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015998679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":194,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.209818+0000","last_change":"2024-09-10T18:51:55.145730+0000","last_active":"2024-09-10T18:53:26.209818+0000","last_peered":"2024-09-10T18:53:26.209818+0000","last_clean":"2024-09-10T18:53:26.209818+0000","last_became_active":"2024-09-10T18:47:36.416187+0000","last_became_peered":"2024-09-10T18:47:36.416187+0000","last_unstale":"2024-09-10T18:53:26.209818+0000","last_undegraded":"2024-09-10T18:53:26.209818+0000","last_fullsized":"2024-09-10T18:53:26.209818+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:55.145638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:55.145638+0000","last_clean_scrub_stamp":"2024-09-10T18:51:55.145638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00318506,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.14","version":"70'81","reported_seq":204,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.210765+0000","last_change":"2024-09-10T18:52:57.184463+0000","last_active":"2024-09-10T18:53:26.210765+0000","last_peered":"2024-09-10T18:53:26.210765+0000","last_clean":"2024-09-10T18:53:26.210765+0000","last_became_active":"2024-09-10T18:52:16.600485+0000","last_became_peered":"2024-09-10T18:52:16.600485+0000","last_unstale":"2024-09-10T18:53:26.210765+0000","last_undegraded":"2024-09-10T18:53:26.210765+0000","last_fullsized":"2024-09-10T18:53:26.210765+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:52:57.184383+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:57.184383+0000","last_clean_scrub_stamp":"2024-09-10T18:52:57.184383+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:57.903584+0000","scrub_duration":0.101414052,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"79'16","reported_seq":203,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.237451+0000","last_change":"2024-09-10T18:51:36.997632+0000","last_active":"2024-09-10T18:53:28.237451+0000","last_peered":"2024-09-10T18:53:28.237451+0000","last_clean":"2024-09-10T18:53:28.237451+0000","last_became_active":"2024-09-10T18:45:20.421479+0000","last_became_peered":"2024-09-10T18:45:20.421479+0000","last_unstale":"2024-09-10T18:53:28.237451+0000","last_undegraded":"2024-09-10T18:53:28.237451+0000","last_fullsized":"2024-09-10T18:53:28.237451+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"79'16","last_scrub_stamp":"2024-09-10T18:51:36.997571+0000","last_deep_scrub":"79'16","last_deep_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.997571+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002345059,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"56'1","reported_seq":210,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.224370+0000","last_change":"2024-09-10T18:52:47.666751+0000","last_active":"2024-09-10T18:53:26.224370+0000","last_peered":"2024-09-10T18:53:26.224370+0000","last_clean":"2024-09-10T18:53:26.224370+0000","last_became_active":"2024-09-10T18:47:36.421985+0000","last_became_peered":"2024-09-10T18:47:36.421985+0000","last_unstale":"2024-09-10T18:53:26.224370+0000","last_undegraded":"2024-09-10T18:53:26.224370+0000","last_fullsized":"2024-09-10T18:53:26.224370+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:52:47.666649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:52:47.666649+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:06.869776+0000","scrub_duration":0.0065630849999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":141,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.243250+0000","last_change":"2024-09-10T18:52:51.817600+0000","last_active":"2024-09-10T18:53:28.243250+0000","last_peered":"2024-09-10T18:53:28.243250+0000","last_clean":"2024-09-10T18:53:28.243250+0000","last_became_active":"2024-09-10T18:47:57.107149+0000","last_became_peered":"2024-09-10T18:47:57.107149+0000","last_unstale":"2024-09-10T18:53:28.243250+0000","last_undegraded":"2024-09-10T18:53:28.243250+0000","last_fullsized":"2024-09-10T18:53:28.243250+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:51.817521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:52:51.817521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:04.521360+0000","scrub_duration":0.001468528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":185,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.237374+0000","last_change":"2024-09-10T18:52:48.901562+0000","last_active":"2024-09-10T18:53:28.237374+0000","last_peered":"2024-09-10T18:53:28.237374+0000","last_clean":"2024-09-10T18:53:28.237374+0000","last_became_active":"2024-09-10T18:45:24.264818+0000","last_became_peered":"2024-09-10T18:45:24.264818+0000","last_unstale":"2024-09-10T18:53:28.237374+0000","last_undegraded":"2024-09-10T18:53:28.237374+0000","last_fullsized":"2024-09-10T18:53:28.237374+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:48.901462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:52:48.901462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:52.259573+0000","scrub_duration":0.0039921779999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"78'2","reported_seq":200,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.210040+0000","last_change":"2024-09-10T18:51:46.249221+0000","last_active":"2024-09-10T18:53:26.210040+0000","last_peered":"2024-09-10T18:53:26.210040+0000","last_clean":"2024-09-10T18:53:26.210040+0000","last_became_active":"2024-09-10T18:47:36.418702+0000","last_became_peered":"2024-09-10T18:47:36.418702+0000","last_unstale":"2024-09-10T18:53:26.210040+0000","last_undegraded":"2024-09-10T18:53:26.210040+0000","last_fullsized":"2024-09-10T18:53:26.210040+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'2","last_scrub_stamp":"2024-09-10T18:51:46.249165+0000","last_deep_scrub":"78'2","last_deep_scrub_stamp":"2024-09-10T18:46:37.581130+0000","last_clean_scrub_stamp":"2024-09-10T18:51:46.249165+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001625867,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":189,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.216791+0000","last_change":"2024-09-10T18:52:52.152441+0000","last_active":"2024-09-10T18:53:26.216791+0000","last_peered":"2024-09-10T18:53:26.216791+0000","last_clean":"2024-09-10T18:53:26.216791+0000","last_became_active":"2024-09-10T18:45:17.962108+0000","last_became_peered":"2024-09-10T18:45:17.962108+0000","last_unstale":"2024-09-10T18:53:26.216791+0000","last_undegraded":"2024-09-10T18:53:26.216791+0000","last_fullsized":"2024-09-10T18:53:26.216791+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:52.152309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:52:52.152309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:13.604747+0000","scrub_duration":0.0044316169999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"63'10","reported_seq":204,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.228293+0000","last_change":"2024-09-10T18:52:00.358388+0000","last_active":"2024-09-10T18:53:28.228293+0000","last_peered":"2024-09-10T18:53:28.228293+0000","last_clean":"2024-09-10T18:53:28.228293+0000","last_became_active":"2024-09-10T18:45:20.421094+0000","last_became_peered":"2024-09-10T18:45:20.421094+0000","last_unstale":"2024-09-10T18:53:28.228293+0000","last_undegraded":"2024-09-10T18:53:28.228293+0000","last_fullsized":"2024-09-10T18:53:28.228293+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:52:00.358298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:52:00.358298+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.01044082,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"190'9","reported_seq":306,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.236390+0000","last_change":"2024-09-10T18:53:07.746395+0000","last_active":"2024-09-10T18:53:28.236390+0000","last_peered":"2024-09-10T18:53:28.236390+0000","last_clean":"2024-09-10T18:53:28.236390+0000","last_became_active":"2024-09-10T18:45:22.073281+0000","last_became_peered":"2024-09-10T18:45:22.073281+0000","last_unstale":"2024-09-10T18:53:28.236390+0000","last_undegraded":"2024-09-10T18:53:28.236390+0000","last_fullsized":"2024-09-10T18:53:28.236390+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"190'9","last_scrub_stamp":"2024-09-10T18:53:07.746309+0000","last_deep_scrub":"190'9","last_deep_scrub_stamp":"2024-09-10T18:53:07.746309+0000","last_clean_scrub_stamp":"2024-09-10T18:53:07.746309+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:21.881973+0000","scrub_duration":0.0082424100000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":175,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.230117+0000","last_change":"2024-09-10T18:51:40.051314+0000","last_active":"2024-09-10T18:53:28.230117+0000","last_peered":"2024-09-10T18:53:28.230117+0000","last_clean":"2024-09-10T18:53:28.230117+0000","last_became_active":"2024-09-10T18:45:24.082318+0000","last_became_peered":"2024-09-10T18:45:24.082318+0000","last_unstale":"2024-09-10T18:53:28.230117+0000","last_undegraded":"2024-09-10T18:53:28.230117+0000","last_fullsized":"2024-09-10T18:53:28.230117+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:40.051241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:03.475299+0000","last_clean_scrub_stamp":"2024-09-10T18:51:40.051241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001564937,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":181,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.237785+0000","last_change":"2024-09-10T18:51:01.165781+0000","last_active":"2024-09-10T18:53:28.237785+0000","last_peered":"2024-09-10T18:53:28.237785+0000","last_clean":"2024-09-10T18:53:28.237785+0000","last_became_active":"2024-09-10T18:45:17.965707+0000","last_became_peered":"2024-09-10T18:45:17.965707+0000","last_unstale":"2024-09-10T18:53:28.237785+0000","last_undegraded":"2024-09-10T18:53:28.237785+0000","last_fullsized":"2024-09-10T18:53:28.237785+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:01.165584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:01.165584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0071542790000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"63'11","reported_seq":206,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.224304+0000","last_change":"2024-09-10T18:51:46.517627+0000","last_active":"2024-09-10T18:53:26.224304+0000","last_peered":"2024-09-10T18:53:26.224304+0000","last_clean":"2024-09-10T18:53:26.224304+0000","last_became_active":"2024-09-10T18:45:20.058717+0000","last_became_peered":"2024-09-10T18:45:20.058717+0000","last_unstale":"2024-09-10T18:53:26.224304+0000","last_undegraded":"2024-09-10T18:53:26.224304+0000","last_fullsized":"2024-09-10T18:53:26.224304+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-10T18:51:46.517571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:46.517571+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001736118,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":197,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.228217+0000","last_change":"2024-09-10T18:52:52.592374+0000","last_active":"2024-09-10T18:53:28.228217+0000","last_peered":"2024-09-10T18:53:28.228217+0000","last_clean":"2024-09-10T18:53:28.228217+0000","last_became_active":"2024-09-10T18:47:36.418767+0000","last_became_peered":"2024-09-10T18:47:36.418767+0000","last_unstale":"2024-09-10T18:53:28.228217+0000","last_undegraded":"2024-09-10T18:53:28.228217+0000","last_fullsized":"2024-09-10T18:53:28.228217+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:52.592184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:52:52.592184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:53.446090+0000","scrub_duration":0.0039284209999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":172,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.206152+0000","last_change":"2024-09-10T18:51:36.324657+0000","last_active":"2024-09-10T18:53:26.206152+0000","last_peered":"2024-09-10T18:53:26.206152+0000","last_clean":"2024-09-10T18:53:26.206152+0000","last_became_active":"2024-09-10T18:45:24.267251+0000","last_became_peered":"2024-09-10T18:45:24.267251+0000","last_unstale":"2024-09-10T18:53:26.206152+0000","last_undegraded":"2024-09-10T18:53:26.206152+0000","last_fullsized":"2024-09-10T18:53:26.206152+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:36.324592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:36.324592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001354409,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"56'1","reported_seq":198,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.222605+0000","last_change":"2024-09-10T18:52:42.673383+0000","last_active":"2024-09-10T18:53:26.222605+0000","last_peered":"2024-09-10T18:53:26.222605+0000","last_clean":"2024-09-10T18:53:26.222605+0000","last_became_active":"2024-09-10T18:45:17.966286+0000","last_became_peered":"2024-09-10T18:45:17.966286+0000","last_unstale":"2024-09-10T18:53:26.222605+0000","last_undegraded":"2024-09-10T18:53:26.222605+0000","last_fullsized":"2024-09-10T18:53:26.222605+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:52:42.673300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:52:42.673300+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:54:02.373254+0000","scrub_duration":0.011239885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"63'15","reported_seq":195,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.230194+0000","last_change":"2024-09-10T18:51:59.160864+0000","last_active":"2024-09-10T18:53:28.230194+0000","last_peered":"2024-09-10T18:53:28.230194+0000","last_clean":"2024-09-10T18:53:28.230194+0000","last_became_active":"2024-09-10T18:47:36.424373+0000","last_became_peered":"2024-09-10T18:47:36.424373+0000","last_unstale":"2024-09-10T18:53:28.230194+0000","last_undegraded":"2024-09-10T18:53:28.230194+0000","last_fullsized":"2024-09-10T18:53:28.230194+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:51:59.160774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:59.160774+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030853120000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":177,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.230364+0000","last_change":"2024-09-10T18:51:54.153006+0000","last_active":"2024-09-10T18:53:28.230364+0000","last_peered":"2024-09-10T18:53:28.230364+0000","last_clean":"2024-09-10T18:53:28.230364+0000","last_became_active":"2024-09-10T18:45:22.059794+0000","last_became_peered":"2024-09-10T18:45:22.059794+0000","last_unstale":"2024-09-10T18:53:28.230364+0000","last_undegraded":"2024-09-10T18:53:28.230364+0000","last_fullsized":"2024-09-10T18:53:28.230364+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:54.152908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:54.152908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0079641550000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":185,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.238707+0000","last_change":"2024-09-10T18:52:54.624699+0000","last_active":"2024-09-10T18:53:28.238707+0000","last_peered":"2024-09-10T18:53:28.238707+0000","last_clean":"2024-09-10T18:53:28.238707+0000","last_became_active":"2024-09-10T18:52:54.624256+0000","last_became_peered":"2024-09-10T18:52:54.624256+0000","last_unstale":"2024-09-10T18:53:28.238707+0000","last_undegraded":"2024-09-10T18:53:28.238707+0000","last_fullsized":"2024-09-10T18:53:28.238707+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:40.829804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:52:40.829804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:56.322219+0000","scrub_duration":0.0049205200000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":181,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.237727+0000","last_change":"2024-09-10T18:51:40.076322+0000","last_active":"2024-09-10T18:53:28.237727+0000","last_peered":"2024-09-10T18:53:28.237727+0000","last_clean":"2024-09-10T18:53:28.237727+0000","last_became_active":"2024-09-10T18:45:17.963257+0000","last_became_peered":"2024-09-10T18:45:17.963257+0000","last_unstale":"2024-09-10T18:53:28.237727+0000","last_undegraded":"2024-09-10T18:53:28.237727+0000","last_fullsized":"2024-09-10T18:53:28.237727+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:40.076262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:40.076262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037079650000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"63'12","reported_seq":202,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-10T18:53:28.236851+0000","last_change":"2024-09-10T18:51:31.148694+0000","last_active":"2024-09-10T18:53:28.236851+0000","last_peered":"2024-09-10T18:53:28.236851+0000","last_clean":"2024-09-10T18:53:28.236851+0000","last_became_active":"2024-09-10T18:45:20.058391+0000","last_became_peered":"2024-09-10T18:45:20.058391+0000","last_unstale":"2024-09-10T18:53:28.236851+0000","last_undegraded":"2024-09-10T18:53:28.236851+0000","last_fullsized":"2024-09-10T18:53:28.236851+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:51:31.148630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:31.148630+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021214699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":194,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-10T18:53:26.211546+0000","last_change":"2024-09-10T18:52:47.197788+0000","last_active":"2024-09-10T18:53:26.211546+0000","last_peered":"2024-09-10T18:53:26.211546+0000","last_clean":"2024-09-10T18:53:26.211546+0000","last_became_active":"2024-09-10T18:47:36.416131+0000","last_became_peered":"2024-09-10T18:47:36.416131+0000","last_unstale":"2024-09-10T18:53:26.211546+0000","last_undegraded":"2024-09-10T18:53:26.211546+0000","last_fullsized":"2024-09-10T18:53:26.211546+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:47.197711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:52:47.197711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:53:56.193794+0000","scrub_duration":0.0093186050000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":8747352088,"num_objects":133475,"num_object_clones":0,"num_object_copies":400425,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":23041,"num_objects_unfound":0,"num_objects_dirty":133475,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":159781,"num_write_kb":9384129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2038,"num_bytes_recovered":133562368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":26381725696,"data_stored":26381713480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69047,"ondisk_log_size":69047,"up":48,"acting":48,"num_store_stats":7},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":695,"num_read_kb":452,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":1458,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":455,"ondisk_log_size":455,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":483,"num_read_kb":478,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":490576,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":20,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":370027,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1701,"ondisk_log_size":1701,"up":63,"acting":63,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299495,"num_pgs":77,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3543044,"kb_used_data":3237372,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":90202108,"statfs":{"total":95995035648,"available":92366958592,"internally_reserved":0,"allocated":3315068928,"data_stored":3314685659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:53:20 2024","interfaces":[{"interface":"back","average":{"1min":1.746,"5min":1.2689999999999999,"15min":0.873},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":11.85,"5min":21.231000000000002,"15min":31.805},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.595,"5min":1.4770000000000001,"15min":0.92300000000000004},"min":{"1min":0.22800000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":11.484999999999999,"5min":21.422999999999998,"15min":31.212},"last":2.0510000000000002}]},{"osd":1,"last update":"Tue Sep 10 18:53:20 2024","interfaces":[{"interface":"back","average":{"1min":1.643,"5min":1.2929999999999999,"15min":0.753},"min":{"1min":0.23100000000000001,"5min":0.221,"15min":0.221},"max":{"1min":8.6189999999999998,"5min":21.34,"15min":21.34},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.663,"5min":1.228,"15min":0.754},"min":{"1min":0.25700000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":8.5899999999999999,"5min":21.248000000000001,"15min":21.248000000000001},"last":2.0859999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:53:20 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.69399999999999995,"15min":0.55500000000000005},"min":{"1min":0.27300000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":3.0670000000000002,"5min":5.1399999999999997,"15min":5.1399999999999997},"last":0.376},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.86799999999999999,"15min":0.63700000000000001},"min":{"1min":0.32000000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.9830000000000001,"5min":8.8859999999999992,"15min":8.8859999999999992},"last":0.878}]},{"osd":3,"last update":"Tue Sep 10 18:53:20 2024","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.353,"15min":0.90100000000000002},"min":{"1min":0.33700000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":11.282,"5min":24.559999999999999,"15min":31.478000000000002},"last":0.34000000000000002},{"interface":"front","average":{"1min":1.7729999999999999,"5min":1.5589999999999999,"15min":1.0089999999999999},"min":{"1min":0.315,"5min":0.254,"15min":0.254},"max":{"1min":10.185,"5min":24.645,"15min":33.731999999999999},"last":1.3320000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:53:20 2024","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":0.748,"15min":0.505},"min":{"1min":0.24099999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":4.0789999999999997,"5min":4.0789999999999997,"15min":4.0789999999999997},"last":3.1240000000000001},{"interface":"front","average":{"1min":1.706,"5min":0.78700000000000003,"15min":0.51700000000000002},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":4.4630000000000001,"5min":4.4630000000000001,"15min":4.4630000000000001},"last":0.83799999999999997}]},{"osd":5,"last update":"Tue Sep 10 18:53:20 2024","interfaces":[{"interface":"back","average":{"1min":7.7729999999999997,"5min":2.04,"15min":0.95399999999999996},"min":{"1min":0.28499999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":113.255,"5min":113.255,"15min":113.255},"last":2.1110000000000002},{"interface":"front","average":{"1min":7.6020000000000003,"5min":1.9770000000000001,"15min":0.93799999999999994},"min":{"1min":0.25900000000000001,"5min":0.253,"15min":0.253},"max":{"1min":113.851,"5min":113.851,"15min":113.851},"last":0.81899999999999995}]},{"osd":6,"last update":"Tue Sep 10 18:53:20 2024","interfaces":[{"interface":"back","average":{"1min":6.3170000000000002,"5min":1.8149999999999999,"15min":0.90300000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":87.716999999999999,"5min":87.716999999999999,"15min":87.716999999999999},"last":2.1760000000000002},{"interface":"front","average":{"1min":5.9790000000000001,"5min":1.71,"15min":0.86099999999999999},"min":{"1min":0.308,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":87.738,"5min":87.738,"15min":87.738},"last":0.96099999999999997}]}]},{"osd":6,"up_from":43,"seq":184683593836,"num_pgs":74,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4061400,"kb_used_data":3751500,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":89683752,"statfs":{"total":95995035648,"available":91836162048,"internally_reserved":0,"allocated":3841536000,"data_stored":3841137545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:52:50 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":1.6259999999999999,"15min":0.85499999999999998},"min":{"1min":0.38,"5min":0.215,"15min":0.215},"max":{"1min":1.4119999999999999,"5min":46.384999999999998,"15min":46.384999999999998},"last":1.389},{"interface":"front","average":{"1min":1.534,"5min":1.7410000000000001,"15min":0.90700000000000003},"min":{"1min":0.27900000000000003,"5min":0.245,"15min":0.22600000000000001},"max":{"1min":4.2290000000000001,"5min":48.372999999999998,"15min":48.372999999999998},"last":1.1910000000000001}]},{"osd":1,"last update":"Tue Sep 10 18:52:50 2024","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.1319999999999999,"15min":0.70199999999999996},"min":{"1min":0.30199999999999999,"5min":0.219,"15min":0.219},"max":{"1min":6.3230000000000004,"5min":6.3230000000000004,"15min":6.3230000000000004},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.169,"5min":0.96799999999999997,"15min":0.63200000000000001},"min":{"1min":0.26200000000000001,"5min":0.214,"15min":0.214},"max":{"1min":6.7690000000000001,"5min":6.7690000000000001,"15min":6.7690000000000001},"last":1.329}]},{"osd":2,"last update":"Tue Sep 10 18:52:50 2024","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":0.94999999999999996,"15min":0.63800000000000001},"min":{"1min":0.33500000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.0430000000000001,"5min":4.4740000000000002,"15min":4.4740000000000002},"last":1.135},{"interface":"front","average":{"1min":1.077,"5min":0.72399999999999998,"15min":0.56799999999999995},"min":{"1min":0.33900000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":2.726,"5min":2.9900000000000002,"15min":2.9900000000000002},"last":1.012}]},{"osd":3,"last update":"Tue Sep 10 18:52:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":0.98299999999999998,"15min":0.65100000000000002},"min":{"1min":0.34300000000000003,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":5.7750000000000004,"5min":5.7750000000000004,"15min":5.7750000000000004},"last":1.361},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.2809999999999999,"15min":0.76900000000000002},"min":{"1min":0.41999999999999998,"5min":0.32600000000000001,"15min":0.29799999999999999},"max":{"1min":5.0940000000000003,"5min":5.3079999999999998,"15min":5.3079999999999998},"last":0.98699999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:52:50 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.67100000000000004,"15min":0.49399999999999999},"min":{"1min":0.247,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":2.6469999999999998,"5min":4.2969999999999997,"15min":4.2969999999999997},"last":1.1120000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.61099999999999999,"15min":0.46500000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":4.0110000000000001,"5min":4.3209999999999997,"15min":4.3209999999999997},"last":1.175}]},{"osd":5,"last update":"Tue Sep 10 18:52:50 2024","interfaces":[{"interface":"back","average":{"1min":1.7050000000000001,"5min":0.85399999999999998,"15min":0.56299999999999994},"min":{"1min":0.30399999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":6.8440000000000003,"5min":6.8440000000000003,"15min":6.8440000000000003},"last":1.097},{"interface":"front","average":{"1min":1.2569999999999999,"5min":0.75700000000000001,"15min":0.53700000000000003},"min":{"1min":0.27800000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":2.7130000000000001,"5min":4.4500000000000002,"15min":4.4500000000000002},"last":1.462}]},{"osd":7,"last update":"Tue Sep 10 18:53:15 2024","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":0.91200000000000003,"15min":0.625},"min":{"1min":0.29799999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":5.3680000000000003,"5min":5.3680000000000003,"15min":5.3680000000000003},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.349,"5min":0.875,"15min":0.628},"min":{"1min":0.34799999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":4.2450000000000001,"5min":4.2450000000000001,"15min":4.2450000000000001},"last":1.2949999999999999}]}]},{"osd":5,"up_from":37,"seq":158913790064,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5131832,"kb_used_data":4821936,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":88613320,"statfs":{"total":95995035648,"available":90740039680,"internally_reserved":0,"allocated":4937662464,"data_stored":4937290731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:52:30 2024","interfaces":[{"interface":"back","average":{"1min":1.657,"5min":1.734,"15min":0.92600000000000005},"min":{"1min":0.22800000000000001,"5min":0.183,"15min":0.183},"max":{"1min":13.157,"5min":65.697000000000003,"15min":65.697000000000003},"last":0.57099999999999995},{"interface":"front","average":{"1min":1.333,"5min":1.742,"15min":0.90800000000000003},"min":{"1min":0.255,"5min":0.219,"15min":0.219},"max":{"1min":13.333,"5min":65.683999999999997,"15min":65.683999999999997},"last":0.76200000000000001}]},{"osd":1,"last update":"Tue Sep 10 18:52:30 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71899999999999997,"15min":0.60799999999999998},"min":{"1min":0.28100000000000003,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":2.145,"5min":3.5150000000000001,"15min":3.5150000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.82999999999999996,"15min":0.63100000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":1.464,"5min":6.6989999999999998,"15min":6.6989999999999998},"last":19.474}]},{"osd":2,"last update":"Tue Sep 10 18:52:30 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.68200000000000005,"15min":0.57099999999999995},"min":{"1min":0.28000000000000003,"5min":0.249,"15min":0.249},"max":{"1min":1.5580000000000001,"5min":7.8029999999999999,"15min":7.8029999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.76400000000000001,"15min":0.60999999999999999},"min":{"1min":0.307,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":2.1139999999999999,"5min":6.7210000000000001,"15min":6.7210000000000001},"last":0.83799999999999997}]},{"osd":3,"last update":"Tue Sep 10 18:52:30 2024","interfaces":[{"interface":"back","average":{"1min":2.0859999999999999,"5min":1.9690000000000001,"15min":1.0049999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":29.573,"5min":66.593999999999994,"15min":66.593999999999994},"last":0.95699999999999996},{"interface":"front","average":{"1min":2.456,"5min":2.1459999999999999,"15min":1.073},"min":{"1min":0.32000000000000001,"5min":0.245,"15min":0.245},"max":{"1min":29.623999999999999,"5min":65.939999999999998,"15min":65.939999999999998},"last":20.062999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:52:30 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.58199999999999996,"15min":0.44600000000000001},"min":{"1min":0.27400000000000002,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":3.714,"5min":3.714,"15min":3.714},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.55200000000000005,"15min":0.45100000000000001},"min":{"1min":0.25,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.4830000000000001,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":0.70899999999999996}]},{"osd":6,"last update":"Tue Sep 10 18:52:54 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":0.92900000000000005,"15min":0.58799999999999997},"min":{"1min":0.26100000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":5.0069999999999997,"5min":11.17,"15min":11.17},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.22,"5min":0.97399999999999998,"15min":0.63300000000000001},"min":{"1min":0.34799999999999998,"5min":0.33000000000000002,"15min":0.31},"max":{"1min":4.8819999999999997,"5min":11.18,"15min":11.18},"last":0.68799999999999994}]},{"osd":7,"last update":"Tue Sep 10 18:53:17 2024","interfaces":[{"interface":"back","average":{"1min":1.869,"5min":1.0389999999999999,"15min":0.66200000000000003},"min":{"1min":0.29299999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":4.8479999999999999,"5min":10.782999999999999,"15min":10.782999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.323,"5min":0.92400000000000004,"15min":0.624},"min":{"1min":0.29199999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":4.8899999999999997,"5min":11.846,"15min":11.846},"last":0.745}]}]},{"osd":4,"up_from":30,"seq":128849018998,"num_pgs":75,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3594316,"kb_used_data":3288644,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":90150836,"statfs":{"total":95995035648,"available":92314456064,"internally_reserved":0,"allocated":3367571456,"data_stored":3367180476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:53:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.5900000000000001,"15min":0.88100000000000001},"min":{"1min":0.215,"5min":0.21099999999999999,"15min":0.20699999999999999},"max":{"1min":2.4249999999999998,"5min":55.776000000000003,"15min":55.776000000000003},"last":0.219},{"interface":"front","average":{"1min":1.181,"5min":1.607,"15min":0.88900000000000001},"min":{"1min":0.26200000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":4.1429999999999998,"5min":55.737000000000002,"15min":55.737000000000002},"last":0.76300000000000001}]},{"osd":1,"last update":"Tue Sep 10 18:53:12 2024","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.0509999999999999,"15min":0.72399999999999998},"min":{"1min":0.22800000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":3.5600000000000001,"5min":8.1600000000000001,"15min":8.1600000000000001},"last":0.30499999999999999},{"interface":"front","average":{"1min":1.1879999999999999,"5min":0.88100000000000001,"15min":0.65200000000000002},"min":{"1min":0.28499999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":2.6150000000000002,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":0.28000000000000003}]},{"osd":2,"last update":"Tue Sep 10 18:53:12 2024","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":0.68899999999999995,"15min":0.59099999999999997},"min":{"1min":0.32700000000000001,"5min":0.29099999999999998,"15min":0.28199999999999997},"max":{"1min":2.375,"5min":2.375,"15min":2.6030000000000002},"last":0.25900000000000001},{"interface":"front","average":{"1min":1.3580000000000001,"5min":0.755,"15min":0.61599999999999999},"min":{"1min":0.35799999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":4.1639999999999997,"5min":4.1639999999999997,"15min":4.1639999999999997},"last":0.73999999999999999}]},{"osd":3,"last update":"Tue Sep 10 18:53:12 2024","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":1.4510000000000001,"15min":0.83899999999999997},"min":{"1min":0.26600000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":4.1130000000000004,"5min":24.957999999999998,"15min":24.957999999999998},"last":0.71199999999999997},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.2949999999999999,"15min":0.79000000000000004},"min":{"1min":0.26000000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":3.6160000000000001,"5min":24.940000000000001,"15min":24.940000000000001},"last":0.33000000000000002}]},{"osd":5,"last update":"Tue Sep 10 18:52:37 2024","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":0.69799999999999995,"15min":0.47299999999999998},"min":{"1min":0.23400000000000001,"5min":0.222,"15min":0.222},"max":{"1min":3.4420000000000002,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":2.4430000000000001},{"interface":"front","average":{"1min":1.0760000000000001,"5min":0.61899999999999999,"15min":0.45000000000000001},"min":{"1min":0.28499999999999998,"5min":0.247,"15min":0.23799999999999999},"max":{"1min":4.1230000000000002,"5min":4.1230000000000002,"15min":4.1230000000000002},"last":1.0629999999999999}]},{"osd":6,"last update":"Tue Sep 10 18:52:54 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.64700000000000002,"15min":0.501},"min":{"1min":0.29799999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.4889999999999999,"5min":3.5529999999999999,"15min":3.5529999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":1.5229999999999999,"5min":0.86599999999999999,"15min":0.55500000000000005},"min":{"1min":0.28999999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":5.4470000000000001,"5min":5.4470000000000001,"15min":5.4470000000000001},"last":0.67600000000000005}]},{"osd":7,"last update":"Tue Sep 10 18:53:25 2024","interfaces":[{"interface":"back","average":{"1min":1.8220000000000001,"5min":0.89200000000000002,"15min":0.628},"min":{"1min":0.31,"5min":0.31,"15min":0.30599999999999999},"max":{"1min":5.9139999999999997,"5min":5.9139999999999997,"15min":5.9139999999999997},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.671,"5min":0.86699999999999999,"15min":0.58599999999999997},"min":{"1min":0.32900000000000001,"5min":0.29999999999999999,"15min":0.25900000000000001},"max":{"1min":5.2229999999999999,"5min":5.2229999999999999,"15min":5.2229999999999999},"last":1.1020000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182523,"num_pgs":79,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4119508,"kb_used_data":3809612,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":89625644,"statfs":{"total":95995035648,"available":91776659456,"internally_reserved":0,"allocated":3901042688,"data_stored":3900643239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:52:46 2024","interfaces":[{"interface":"back","average":{"1min":4.9260000000000002,"5min":2.8260000000000001,"15min":1.1950000000000001},"min":{"1min":0.17299999999999999,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":83.126000000000005,"5min":83.126000000000005,"15min":83.126000000000005},"last":0.40600000000000003},{"interface":"front","average":{"1min":5.1159999999999997,"5min":2.7349999999999999,"15min":1.1779999999999999},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":81.632999999999996,"5min":81.632999999999996,"15min":81.632999999999996},"last":0.58199999999999996}]},{"osd":1,"last update":"Tue Sep 10 18:52:46 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":2.0209999999999999,"15min":0.94099999999999995},"min":{"1min":0.22,"5min":0.22,"15min":0.20200000000000001},"max":{"1min":2.79,"5min":58.215000000000003,"15min":58.215000000000003},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.93700000000000006,"5min":2.125,"15min":0.98199999999999998},"min":{"1min":0.26700000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":2.4500000000000002,"5min":57.896999999999998,"15min":57.896999999999998},"last":0.51200000000000001}]},{"osd":2,"last update":"Tue Sep 10 18:52:46 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":1.079,"15min":0.64800000000000002},"min":{"1min":0.313,"5min":0.25900000000000001,"15min":0.246},"max":{"1min":2.161,"5min":5.8220000000000001,"15min":5.8220000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.91100000000000003,"15min":0.58799999999999997},"min":{"1min":0.24399999999999999,"5min":0.218,"15min":0.218},"max":{"1min":1.036,"5min":7.0979999999999999,"15min":7.0979999999999999},"last":0.53500000000000003}]},{"osd":4,"last update":"Tue Sep 10 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":1.153,"15min":0.76100000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":2.6749999999999998,"5min":7.1790000000000003,"15min":7.1790000000000003},"last":1.599},{"interface":"front","average":{"1min":0.66100000000000003,"5min":1.0880000000000001,"15min":0.755},"min":{"1min":0.32900000000000001,"5min":0.313,"15min":0.311},"max":{"1min":1.2809999999999999,"5min":7.1349999999999998,"15min":7.1349999999999998},"last":1.0389999999999999}]},{"osd":5,"last update":"Tue Sep 10 18:52:37 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":1.143,"15min":0.74299999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":2.5760000000000001,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":0.74199999999999999},{"interface":"front","average":{"1min":1.161,"5min":1.2430000000000001,"15min":0.79100000000000004},"min":{"1min":0.40899999999999997,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":3.2389999999999999,"5min":9.2509999999999994,"15min":9.2509999999999994},"last":0.79500000000000004}]},{"osd":6,"last update":"Tue Sep 10 18:52:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.371,"15min":0.83099999999999996},"min":{"1min":0.44500000000000001,"5min":0.315,"15min":0.315},"max":{"1min":2.1230000000000002,"5min":9.3239999999999998,"15min":9.3239999999999998},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.254,"5min":1.3149999999999999,"15min":0.81299999999999994},"min":{"1min":0.42799999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":6.4900000000000002,"5min":7.1559999999999997,"15min":7.1559999999999997},"last":1.0780000000000001}]},{"osd":7,"last update":"Tue Sep 10 18:53:26 2024","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.1759999999999999,"15min":0.77600000000000002},"min":{"1min":0.36199999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":4.6760000000000002,"5min":4.71,"15min":4.71},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.5529999999999999,"5min":1.4179999999999999,"15min":0.86699999999999999},"min":{"1min":0.39300000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":4.6630000000000003,"5min":9.2949999999999999,"15min":9.2949999999999999},"last":0.85199999999999998}]}]},{"osd":2,"up_from":0,"seq":77309411455,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834574979,"num_pgs":77,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4044836,"kb_used_data":3743324,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":89700316,"statfs":{"total":95995035648,"available":91853123584,"internally_reserved":0,"allocated":3833163776,"data_stored":3832755059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:52:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":1.278,"15min":0.64200000000000002},"min":{"1min":0.215,"5min":0.193,"15min":0.193},"max":{"1min":1.913,"5min":13.531000000000001,"15min":13.531000000000001},"last":1.744},{"interface":"front","average":{"1min":0.93600000000000005,"5min":1.8260000000000001,"15min":0.83999999999999997},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":4.3630000000000004,"5min":16.206,"15min":16.206},"last":1.4259999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:53:27 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.78400000000000003,"15min":0.52300000000000002},"min":{"1min":0.26100000000000001,"5min":0.215,"15min":0.20599999999999999},"max":{"1min":1.728,"5min":3.1259999999999999,"15min":3.8479999999999999},"last":2.383},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.90400000000000003,"15min":0.56399999999999995},"min":{"1min":0.25,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.972,"5min":7.093,"15min":7.093},"last":2.8639999999999999}]},{"osd":3,"last update":"Tue Sep 10 18:52:53 2024","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.6299999999999999,"15min":0.84199999999999997},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":4.8920000000000003,"5min":12.714,"15min":12.714},"last":1.782},{"interface":"front","average":{"1min":0.77500000000000002,"5min":1.7749999999999999,"15min":0.87},"min":{"1min":0.34399999999999997,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.331,"5min":12.696,"15min":12.696},"last":1.5489999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":2.2629999999999999,"5min":1.3899999999999999,"15min":0.85799999999999998},"min":{"1min":0.33100000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":15.965999999999999,"5min":15.965999999999999,"15min":15.965999999999999},"last":1.456},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.048,"15min":0.70999999999999996},"min":{"1min":0.39400000000000002,"5min":0.246,"15min":0.246},"max":{"1min":2.7959999999999998,"5min":3.9540000000000002,"15min":3.9540000000000002},"last":2.3100000000000001}]},{"osd":5,"last update":"Tue Sep 10 18:52:34 2024","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.139,"15min":0.77700000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.7290000000000001,"5min":4.5869999999999997,"15min":4.5869999999999997},"last":1.8140000000000001},{"interface":"front","average":{"1min":1.9830000000000001,"5min":1.4299999999999999,"15min":0.89800000000000002},"min":{"1min":0.42299999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":16.039999999999999,"5min":16.039999999999999,"15min":16.039999999999999},"last":2.4590000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:52:53 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.6060000000000001,"15min":0.94899999999999995},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":4.133,"5min":38.039999999999999,"15min":38.039999999999999},"last":2.835},{"interface":"front","average":{"1min":2.2999999999999998,"5min":1.972,"15min":1.0960000000000001},"min":{"1min":0.45900000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":16.013000000000002,"5min":38.100999999999999,"15min":38.100999999999999},"last":2.6629999999999998}]},{"osd":7,"last update":"Tue Sep 10 18:53:27 2024","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.1890000000000001,"15min":0.82799999999999996},"min":{"1min":0.38400000000000001,"5min":0.315,"15min":0.315},"max":{"1min":3.1150000000000002,"5min":4.6539999999999999,"15min":4.6539999999999999},"last":1.946},{"interface":"front","average":{"1min":1.23,"5min":1.2050000000000001,"15min":0.85199999999999998},"min":{"1min":0.375,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":3.0209999999999999,"5min":3.9700000000000002,"15min":4.1989999999999998},"last":2.431}]}]},{"osd":0,"up_from":8,"seq":34359738505,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3504856,"kb_used_data":3195020,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":90240296,"statfs":{"total":95995035648,"available":92406063104,"internally_reserved":0,"allocated":3271700480,"data_stored":3271307312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 18:53:08 2024","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.762,"15min":0.81200000000000006},"min":{"1min":0.16500000000000001,"5min":0.16500000000000001,"15min":0.16500000000000001},"max":{"1min":4.2169999999999996,"5min":8.4399999999999995,"15min":8.4399999999999995},"last":0.379},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.8660000000000001,"15min":0.84999999999999998},"min":{"1min":0.309,"5min":0.23200000000000001,"15min":0.187},"max":{"1min":3.649,"5min":7.9580000000000002,"15min":7.9580000000000002},"last":2.8650000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:52:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.3360000000000001,"15min":0.70699999999999996},"min":{"1min":0.22900000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":3.1349999999999998,"5min":6.4429999999999996,"15min":6.4429999999999996},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":1.028,"15min":0.58699999999999997},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":2.556,"5min":8.3230000000000004,"15min":8.3230000000000004},"last":0.45600000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:52:44 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.8740000000000001,"15min":0.88700000000000001},"min":{"1min":0.39600000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":2.8500000000000001,"5min":8.4740000000000002,"15min":8.4740000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.78300000000000003,"5min":1.649,"15min":0.82099999999999995},"min":{"1min":0.27900000000000003,"5min":0.218,"15min":0.218},"max":{"1min":2.8290000000000002,"5min":9.8439999999999994,"15min":9.8439999999999994},"last":1.655}]},{"osd":4,"last update":"Tue Sep 10 18:53:08 2024","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.159,"15min":0.755},"min":{"1min":0.38500000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":4.032,"5min":4.992,"15min":4.992},"last":1.627},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.355,"15min":0.82999999999999996},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.4910000000000001,"5min":5.1379999999999999,"15min":5.1379999999999999},"last":3.3799999999999999}]},{"osd":5,"last update":"Tue Sep 10 18:52:32 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":1.115,"15min":0.74399999999999999},"min":{"1min":0.33500000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":2.7850000000000001,"5min":5.0129999999999999,"15min":5.0129999999999999},"last":1.7030000000000001},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.3100000000000001,"15min":0.81499999999999995},"min":{"1min":0.28499999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":3.0579999999999998,"5min":5.3479999999999999,"15min":5.3479999999999999},"last":0.82799999999999996}]},{"osd":6,"last update":"Tue Sep 10 18:53:02 2024","interfaces":[{"interface":"back","average":{"1min":1.569,"5min":1.5209999999999999,"15min":0.88},"min":{"1min":0.47999999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":5.1890000000000001,"5min":5.1890000000000001,"15min":5.1890000000000001},"last":3.427},{"interface":"front","average":{"1min":1.21,"5min":1.1519999999999999,"15min":0.88100000000000001},"min":{"1min":0.36499999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":2.9689999999999999,"5min":5.0350000000000001,"15min":5.0350000000000001},"last":1.6799999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:53:22 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":1.8300000000000001,"15min":0.995},"min":{"1min":0.374,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":2.448,"5min":49.313000000000002,"15min":49.313000000000002},"last":1.268},{"interface":"front","average":{"1min":1.044,"5min":1.899,"15min":1.046},"min":{"1min":0.48499999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":2.7679999999999998,"5min":49.063000000000002,"15min":49.063000000000002},"last":0.64400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3263102976,"data_stored":3263102976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3829661696,"data_stored":3829661696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3882090496,"data_stored":3882090496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3364487168,"data_stored":3364487168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4926738432,"data_stored":4926734360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3828158464,"data_stored":3828154392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3287486464,"data_stored":3287482392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T18:53:37.952 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd pool set unique_pool_1 pgp_num 16 2024-09-10T18:53:38.011 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:38.043 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:38.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:38.044 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:38.145 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:38.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:38.185 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:38.185 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:38.286 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:38.308 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:38.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:38.309 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:38.410 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:38.445 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:38.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:38.446 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:38.519 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T18:53:38.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:38 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:38.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:38 smithi103 ceph-mon[33163]: pgmap v363: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 7.8 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.56k op/s; 20580/382542 objects misplaced (5.380%); 21 MiB/s, 340 objects/s recovering 2024-09-10T18:53:38.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:38 smithi103 ceph-mon[33163]: 5.1a scrub starts 2024-09-10T18:53:38.547 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:38.559 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:38 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:38.560 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:38 smithi018 ceph-mon[40196]: pgmap v363: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 7.8 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.56k op/s; 20580/382542 objects misplaced (5.380%); 21 MiB/s, 340 objects/s recovering 2024-09-10T18:53:38.560 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:38 smithi018 ceph-mon[40196]: 5.1a scrub starts 2024-09-10T18:53:38.560 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:38 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:38.560 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:38 smithi018 ceph-mon[29336]: pgmap v363: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 7.8 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.56k op/s; 20580/382542 objects misplaced (5.380%); 21 MiB/s, 340 objects/s recovering 2024-09-10T18:53:38.560 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:38 smithi018 ceph-mon[29336]: 5.1a scrub starts 2024-09-10T18:53:38.589 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:38.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:38.590 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:38.691 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:38.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:38.732 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:38.732 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:38.833 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:38.873 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:38.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:38.875 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:38.964 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:53:38.976 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:38.999 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:53:39.035 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:39.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:39.037 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:39.138 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:39.166 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:39.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:39.167 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:39.268 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:39.312 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:39.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:39.313 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:39.406 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:39 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:53:39.406 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:39 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:53:39.406 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:39 smithi018 ceph-mon[40196]: osdmap e200: 8 total, 8 up, 7 in 2024-09-10T18:53:39.406 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:39 smithi018 ceph-mon[40196]: 4.3 scrub starts 2024-09-10T18:53:39.406 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:39 smithi018 ceph-mon[40196]: 4.3 scrub ok 2024-09-10T18:53:39.406 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:39 smithi018 ceph-mon[40196]: 8.d scrub starts 2024-09-10T18:53:39.406 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:39 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:53:39.406 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:39 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:53:39.406 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:39 smithi018 ceph-mon[29336]: osdmap e200: 8 total, 8 up, 7 in 2024-09-10T18:53:39.406 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:39 smithi018 ceph-mon[29336]: 4.3 scrub starts 2024-09-10T18:53:39.406 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:39 smithi018 ceph-mon[29336]: 4.3 scrub ok 2024-09-10T18:53:39.407 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:39 smithi018 ceph-mon[29336]: 8.d scrub starts 2024-09-10T18:53:39.414 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:39.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:39.448 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:39.448 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:39.549 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:39.578 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:39.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:39.579 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:39.680 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:39.715 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:39 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:53:39.715 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:39 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:53:39.715 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:39 smithi103 ceph-mon[33163]: osdmap e200: 8 total, 8 up, 7 in 2024-09-10T18:53:39.715 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:39 smithi103 ceph-mon[33163]: 4.3 scrub starts 2024-09-10T18:53:39.715 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:39 smithi103 ceph-mon[33163]: 4.3 scrub ok 2024-09-10T18:53:39.715 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:39 smithi103 ceph-mon[33163]: 8.d scrub starts 2024-09-10T18:53:39.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:39.723 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:39.724 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:39.824 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:39.864 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:39.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:39.865 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:39.877 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T18:53:39.877 INFO:teuthology.orchestra.run.smithi103.stdout: "ops": [ 2024-09-10T18:53:39.877 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:39.877 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_repop(client.24947.0:170261 8.5 e200/158 8:aa8feab9:::benchmark_data_smithi018_106531_object23631:head v 200'10729, mlcod=200'10725)", 2024-09-10T18:53:39.877 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:53:39.870253+0000", 2024-09-10T18:53:39.877 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0052861380000000001, 2024-09-10T18:53:39.877 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0053125910000000002, 2024-09-10T18:53:39.878 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:53:39.878 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "started", 2024-09-10T18:53:39.878 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:53:39.878 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:39.878 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:53:39.878 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:39.870253+0000", 2024-09-10T18:53:39.878 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:53:39.878 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:39.878 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:39.878 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:53:39.878 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:39.870253+0000", 2024-09-10T18:53:39.878 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:53:39.878 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:39.878 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:39.878 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:53:39.879 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:39.870253+0000", 2024-09-10T18:53:39.879 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999995 2024-09-10T18:53:39.879 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:39.879 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:39.879 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:53:39.879 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:39.870256+0000", 2024-09-10T18:53:39.879 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.6540000000000001e-06 2024-09-10T18:53:39.879 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:39.879 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:39.879 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:53:39.879 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:39.870261+0000", 2024-09-10T18:53:39.879 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.9509999999999997e-06 2024-09-10T18:53:39.879 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:39.879 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:39.879 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:53:39.880 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:39.870265+0000", 2024-09-10T18:53:39.880 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.743e-06 2024-09-10T18:53:39.880 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:39.880 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:39.880 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:53:39.880 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:39.872877+0000", 2024-09-10T18:53:39.880 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0026121149999999999 2024-09-10T18:53:39.880 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:39.880 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:39.880 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:53:39.880 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:39.872894+0000", 2024-09-10T18:53:39.880 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.6725e-05 2024-09-10T18:53:39.880 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:53:39.881 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:53:39.881 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:53:39.881 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:39.881 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:39.881 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_repop(client.24947.0:170266 8.8 e200/158 8:1375d9ff:::benchmark_data_smithi018_106531_object23636:head v 200'10815, mlcod=200'10812)", 2024-09-10T18:53:39.881 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:53:39.871863+0000", 2024-09-10T18:53:39.881 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.003676323, 2024-09-10T18:53:39.881 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.003767121, 2024-09-10T18:53:39.881 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:53:39.881 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "started", 2024-09-10T18:53:39.881 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:53:39.881 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:39.881 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:53:39.882 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:39.871863+0000", 2024-09-10T18:53:39.882 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:53:39.882 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:39.882 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:39.882 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:53:39.882 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:39.871863+0000", 2024-09-10T18:53:39.882 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:53:39.882 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:39.882 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:39.882 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:53:39.882 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:39.871862+0000", 2024-09-10T18:53:39.882 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:53:39.882 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:39.882 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:39.883 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:53:39.883 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:39.871869+0000", 2024-09-10T18:53:39.883 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.8650000000000003e-06 2024-09-10T18:53:39.883 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:39.883 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:39.883 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:53:39.883 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:39.871878+0000", 2024-09-10T18:53:39.883 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.7450000000000007e-06 2024-09-10T18:53:39.883 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:39.883 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:39.883 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:53:39.883 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:39.871885+0000", 2024-09-10T18:53:39.883 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.3319999999999997e-06 2024-09-10T18:53:39.883 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:39.884 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:39.884 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:53:39.884 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:39.872943+0000", 2024-09-10T18:53:39.884 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.001058149 2024-09-10T18:53:39.884 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:39.884 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:39.884 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:53:39.884 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:39.872958+0000", 2024-09-10T18:53:39.884 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.5277999999999998e-05 2024-09-10T18:53:39.884 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:53:39.884 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:53:39.884 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:53:39.884 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:39.884 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:39.885 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_repop(client.24947.0:170270 8.8 e200/158)", 2024-09-10T18:53:39.885 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:53:39.875316+0000", 2024-09-10T18:53:39.885 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.00022283599999999999, 2024-09-10T18:53:39.885 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00036706599999999999, 2024-09-10T18:53:39.885 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:53:39.885 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "queued for pg", 2024-09-10T18:53:39.885 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:53:39.885 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:39.885 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:53:39.885 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:39.875316+0000", 2024-09-10T18:53:39.885 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:53:39.885 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:39.885 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:39.886 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:53:39.886 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:39.875316+0000", 2024-09-10T18:53:39.886 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:53:39.886 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:39.886 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:39.886 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:53:39.886 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:39.875315+0000", 2024-09-10T18:53:39.886 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:53:39.886 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:39.886 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:39.886 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:53:39.886 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:39.875322+0000", 2024-09-10T18:53:39.886 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.187e-06 2024-09-10T18:53:39.886 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:39.886 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:39.887 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:53:39.887 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:39.875333+0000", 2024-09-10T18:53:39.887 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0444e-05 2024-09-10T18:53:39.887 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:39.887 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:39.887 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:53:39.887 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:39.875338+0000", 2024-09-10T18:53:39.887 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.4099999999999999e-06 2024-09-10T18:53:39.887 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:53:39.887 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:53:39.887 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:53:39.887 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:39.887 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:39.887 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_repop(client.24947.0:170262 8.8 e200/158 8:1f85af1c:::benchmark_data_smithi018_106531_object23632:head v 200'10814, mlcod=200'10812)", 2024-09-10T18:53:39.888 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:53:39.870167+0000", 2024-09-10T18:53:39.888 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0053717529999999999, 2024-09-10T18:53:39.888 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0055614239999999997, 2024-09-10T18:53:39.888 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:53:39.888 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "started", 2024-09-10T18:53:39.888 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:53:39.888 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:39.888 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:53:39.888 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:39.870167+0000", 2024-09-10T18:53:39.888 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:53:39.888 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:39.888 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:39.888 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:53:39.888 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:39.870167+0000", 2024-09-10T18:53:39.889 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:53:39.889 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:39.889 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:39.889 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:53:39.889 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:39.870166+0000", 2024-09-10T18:53:39.889 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:53:39.889 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:39.889 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:39.889 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:53:39.889 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:39.870174+0000", 2024-09-10T18:53:39.889 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.0150000000000003e-06 2024-09-10T18:53:39.889 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:39.889 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:39.889 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:53:39.890 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:39.870190+0000", 2024-09-10T18:53:39.890 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.5563e-05 2024-09-10T18:53:39.890 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:39.890 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:39.890 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:53:39.890 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:39.870196+0000", 2024-09-10T18:53:39.890 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.3709999999999996e-06 2024-09-10T18:53:39.890 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:39.890 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:39.890 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:53:39.890 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:39.870263+0000", 2024-09-10T18:53:39.890 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.6285999999999995e-05 2024-09-10T18:53:39.890 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:53:39.890 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:53:39.890 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:53:39.891 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:39.870281+0000", 2024-09-10T18:53:39.891 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.8359000000000002e-05 2024-09-10T18:53:39.891 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:53:39.891 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:53:39.891 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:53:39.891 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:53:39.891 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-10T18:53:39.891 INFO:teuthology.orchestra.run.smithi103.stdout: "num_ops": 4 2024-09-10T18:53:39.891 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T18:53:39.966 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:40.005 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:40.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:40.006 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:40.107 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:40.146 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:40.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:40.147 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:40.248 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:40.281 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:40.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:40.282 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:40.300 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is unset 2024-09-10T18:53:40.383 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:40.421 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:40.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:40.422 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:40.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:53:40 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:53:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:53:40.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:40 smithi103 ceph-mon[33163]: pgmap v365: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 7.8 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.37k op/s; 20580/382542 objects misplaced (5.380%); 21 MiB/s, 342 objects/s recovering 2024-09-10T18:53:40.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:40 smithi103 ceph-mon[33163]: 4.16 scrub starts 2024-09-10T18:53:40.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:40 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:53:40.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:40 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:53:40.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:40 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:53:40.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:40 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2284725142' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:53:40.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:40 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:53:40.523 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:40.564 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:40.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:40.565 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:40.630 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-10T18:53:40.666 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:40.694 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:40.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:40 smithi018 ceph-mon[29336]: pgmap v365: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 7.8 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.37k op/s; 20580/382542 objects misplaced (5.380%); 21 MiB/s, 342 objects/s recovering 2024-09-10T18:53:40.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:40 smithi018 ceph-mon[29336]: 4.16 scrub starts 2024-09-10T18:53:40.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:40 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:53:40.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:40 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:53:40.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:40 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:53:40.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:40 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2284725142' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:53:40.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:40 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:53:40.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:40 smithi018 ceph-mon[40196]: pgmap v365: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 7.8 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.37k op/s; 20580/382542 objects misplaced (5.380%); 21 MiB/s, 342 objects/s recovering 2024-09-10T18:53:40.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:40 smithi018 ceph-mon[40196]: 4.16 scrub starts 2024-09-10T18:53:40.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:40 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:53:40.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:40 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:53:40.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:40 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:53:40.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:40 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2284725142' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:53:40.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:40 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:53:40.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:40.697 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:40.797 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:40.836 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:40.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:40.837 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:40.938 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:40.968 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:40.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:40.969 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:41.063 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:53:41.069 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:41.099 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:41.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:41.100 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:41.200 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:41.238 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:41.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:41.239 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:41.311 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:53:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:53:40] "GET /metrics HTTP/1.1" 200 224113 "" "Prometheus/2.43.0" 2024-09-10T18:53:41.339 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:41.376 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:41.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:41.377 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:41.467 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:41 smithi103 ceph-mon[33163]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:53:41.467 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:41 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:53:41.467 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:41 smithi103 ceph-mon[33163]: osdmap e201: 8 total, 8 up, 7 in 2024-09-10T18:53:41.467 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:41 smithi103 ceph-mon[33163]: 8.1 scrub starts 2024-09-10T18:53:41.478 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:41.513 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:41.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:41.514 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:41.615 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:41.648 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:41.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:41.649 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:41.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:41 smithi018 ceph-mon[40196]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:53:41.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:41 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:53:41.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:41 smithi018 ceph-mon[40196]: osdmap e201: 8 total, 8 up, 7 in 2024-09-10T18:53:41.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:41 smithi018 ceph-mon[40196]: 8.1 scrub starts 2024-09-10T18:53:41.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:41 smithi018 ceph-mon[29336]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:53:41.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:41 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:53:41.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:41 smithi018 ceph-mon[29336]: osdmap e201: 8 total, 8 up, 7 in 2024-09-10T18:53:41.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:41 smithi018 ceph-mon[29336]: 8.1 scrub starts 2024-09-10T18:53:41.750 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:41.791 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:41.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:41.792 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:41.893 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:41.929 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:41.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:41.930 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:42.030 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:42.068 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:42.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:42.069 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:42.170 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:42.205 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:42.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:42.206 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:42.307 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:42.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:42.355 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:42.355 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:42.456 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:42.493 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:42.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:42.496 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:42.508 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:42 smithi103 ceph-mon[33163]: pgmap v367: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 7.7 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.09k op/s; 20580/381303 objects misplaced (5.397%); 14 MiB/s, 225 objects/s recovering 2024-09-10T18:53:42.508 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:42 smithi103 ceph-mon[33163]: 8.3 deep-scrub starts 2024-09-10T18:53:42.597 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:42.633 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:42.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:42.634 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:42.674 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:42 smithi018 ceph-mon[40196]: pgmap v367: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 7.7 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.09k op/s; 20580/381303 objects misplaced (5.397%); 14 MiB/s, 225 objects/s recovering 2024-09-10T18:53:42.675 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:42 smithi018 ceph-mon[40196]: 8.3 deep-scrub starts 2024-09-10T18:53:42.675 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:42 smithi018 ceph-mon[29336]: pgmap v367: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 7.7 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.09k op/s; 20580/381303 objects misplaced (5.397%); 14 MiB/s, 225 objects/s recovering 2024-09-10T18:53:42.675 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:42 smithi018 ceph-mon[29336]: 8.3 deep-scrub starts 2024-09-10T18:53:42.734 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:42.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:42.777 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:42.777 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:42.878 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:42.911 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:42.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:42.913 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:43.013 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:43.046 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:43.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:43.047 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:43.148 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:43.186 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:43.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:43.187 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:43.287 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:43.313 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:43.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:43.314 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:43.356 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-10T18:53:43.415 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:43.453 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:43.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:43.455 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:43.555 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:43.583 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:43.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:43.584 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:43.685 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:43.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:43 smithi018 ceph-mon[29336]: 8.c deep-scrub starts 2024-09-10T18:53:43.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:43 smithi018 ceph-mon[29336]: 5.4 scrub starts 2024-09-10T18:53:43.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:43 smithi018 ceph-mon[29336]: 5.4 scrub ok 2024-09-10T18:53:43.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:43 smithi018 ceph-mon[40196]: 8.c deep-scrub starts 2024-09-10T18:53:43.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:43 smithi018 ceph-mon[40196]: 5.4 scrub starts 2024-09-10T18:53:43.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:43 smithi018 ceph-mon[40196]: 5.4 scrub ok 2024-09-10T18:53:43.718 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:43.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:43.719 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:43.747 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:43 smithi103 ceph-mon[33163]: 8.c deep-scrub starts 2024-09-10T18:53:43.747 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:43 smithi103 ceph-mon[33163]: 5.4 scrub starts 2024-09-10T18:53:43.747 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:43 smithi103 ceph-mon[33163]: 5.4 scrub ok 2024-09-10T18:53:43.820 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:43.854 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:43.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:43.855 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:43.955 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:43.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:43.989 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:43.989 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:44.090 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:44.115 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:44.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:44.115 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:44.216 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:44.252 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:44.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:44.253 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:44.353 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:44.362 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:53:44.380 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:44.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:44.381 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:44.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:44 smithi018 ceph-mon[29336]: 8.1 scrub ok 2024-09-10T18:53:44.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:44 smithi018 ceph-mon[29336]: pgmap v368: 168 pgs: 1 active+clean+scrubbing, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 164 active+clean; 7.4 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.48k op/s; 18213/363777 objects misplaced (5.007%); 19 MiB/s, 296 objects/s recovering 2024-09-10T18:53:44.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:44 smithi018 ceph-mon[29336]: 3.1 scrub starts 2024-09-10T18:53:44.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:44 smithi018 ceph-mon[29336]: 2.8 scrub starts 2024-09-10T18:53:44.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:44 smithi018 ceph-mon[29336]: 2.8 scrub ok 2024-09-10T18:53:44.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:44 smithi018 ceph-mon[40196]: 8.1 scrub ok 2024-09-10T18:53:44.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:44 smithi018 ceph-mon[40196]: pgmap v368: 168 pgs: 1 active+clean+scrubbing, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 164 active+clean; 7.4 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.48k op/s; 18213/363777 objects misplaced (5.007%); 19 MiB/s, 296 objects/s recovering 2024-09-10T18:53:44.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:44 smithi018 ceph-mon[40196]: 3.1 scrub starts 2024-09-10T18:53:44.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:44 smithi018 ceph-mon[40196]: 2.8 scrub starts 2024-09-10T18:53:44.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:44 smithi018 ceph-mon[40196]: 2.8 scrub ok 2024-09-10T18:53:44.482 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:44.508 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:44.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:44.509 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:44.610 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:44.642 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:44.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:44.643 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:44.744 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:44.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:44 smithi103 ceph-mon[33163]: 8.1 scrub ok 2024-09-10T18:53:44.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:44 smithi103 ceph-mon[33163]: pgmap v368: 168 pgs: 1 active+clean+scrubbing, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 164 active+clean; 7.4 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.48k op/s; 18213/363777 objects misplaced (5.007%); 19 MiB/s, 296 objects/s recovering 2024-09-10T18:53:44.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:44 smithi103 ceph-mon[33163]: 3.1 scrub starts 2024-09-10T18:53:44.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:44 smithi103 ceph-mon[33163]: 2.8 scrub starts 2024-09-10T18:53:44.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:44 smithi103 ceph-mon[33163]: 2.8 scrub ok 2024-09-10T18:53:44.774 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:44.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:44.775 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:44.876 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:44.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:44.911 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:44.912 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:44.946 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T18:53:44.946 INFO:teuthology.orchestra.run.smithi103.stdout: "ops": [], 2024-09-10T18:53:44.946 INFO:teuthology.orchestra.run.smithi103.stdout: "complaint_time": 30, 2024-09-10T18:53:44.946 INFO:teuthology.orchestra.run.smithi103.stdout: "num_blocked_ops": 0 2024-09-10T18:53:44.946 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T18:53:45.013 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:45.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:45.043 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:45.043 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:45.144 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:45.172 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:45.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:45.173 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:45.273 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:45.311 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:45.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:45.312 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:45.344 INFO:teuthology.orchestra.run.smithi018.stderr:set pool 8 pgp_num to 16 2024-09-10T18:53:45.414 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:45.435 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:45.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:45.436 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:45.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:45 smithi018 ceph-mon[40196]: 8.6 scrub starts 2024-09-10T18:53:45.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:45 smithi018 ceph-mon[40196]: 2.1d scrub starts 2024-09-10T18:53:45.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:45 smithi018 ceph-mon[40196]: 6.1d scrub starts 2024-09-10T18:53:45.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:45 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3733767250' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pgp_num", "val": "16"}]: dispatch 2024-09-10T18:53:45.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:45 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pgp_num", "val": "16"}]: dispatch 2024-09-10T18:53:45.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:45 smithi018 ceph-mon[40196]: 2.16 scrub starts 2024-09-10T18:53:45.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:45 smithi018 ceph-mon[29336]: 8.6 scrub starts 2024-09-10T18:53:45.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:45 smithi018 ceph-mon[29336]: 2.1d scrub starts 2024-09-10T18:53:45.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:45 smithi018 ceph-mon[29336]: 6.1d scrub starts 2024-09-10T18:53:45.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:45 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3733767250' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pgp_num", "val": "16"}]: dispatch 2024-09-10T18:53:45.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:45 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pgp_num", "val": "16"}]: dispatch 2024-09-10T18:53:45.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:45 smithi018 ceph-mon[29336]: 2.16 scrub starts 2024-09-10T18:53:45.536 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:45.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:45.573 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:45.574 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:45.674 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:45.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:45.711 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:45.711 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:45.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:45 smithi103 ceph-mon[33163]: 8.6 scrub starts 2024-09-10T18:53:45.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:45 smithi103 ceph-mon[33163]: 2.1d scrub starts 2024-09-10T18:53:45.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:45 smithi103 ceph-mon[33163]: 6.1d scrub starts 2024-09-10T18:53:45.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:45 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3733767250' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pgp_num", "val": "16"}]: dispatch 2024-09-10T18:53:45.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:45 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pgp_num", "val": "16"}]: dispatch 2024-09-10T18:53:45.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:45 smithi103 ceph-mon[33163]: 2.16 scrub starts 2024-09-10T18:53:45.812 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:45.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:45.852 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:45.852 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:45.953 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:45.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:45.997 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:45.997 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:46.098 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:46.133 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:46.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:46.134 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:46.234 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:46.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:46.270 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:46.270 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:46.371 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:46.406 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:46.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:46.407 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:46.439 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:46 smithi018 ceph-mon[40196]: pgmap v369: 168 pgs: 1 active+clean+scrubbing, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 164 active+clean; 7.4 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 782 op/s; 18213/363777 objects misplaced (5.007%); 11 MiB/s, 172 objects/s recovering 2024-09-10T18:53:46.439 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:46 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pgp_num", "val": "16"}]': finished 2024-09-10T18:53:46.439 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:46 smithi018 ceph-mon[40196]: osdmap e202: 8 total, 8 up, 7 in 2024-09-10T18:53:46.439 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:46 smithi018 ceph-mon[40196]: 5.d deep-scrub starts 2024-09-10T18:53:46.439 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:46 smithi018 ceph-mon[40196]: 2.6 scrub starts 2024-09-10T18:53:46.439 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:46 smithi018 ceph-mon[40196]: 5.d deep-scrub ok 2024-09-10T18:53:46.439 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:46 smithi018 ceph-mon[29336]: pgmap v369: 168 pgs: 1 active+clean+scrubbing, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 164 active+clean; 7.4 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 782 op/s; 18213/363777 objects misplaced (5.007%); 11 MiB/s, 172 objects/s recovering 2024-09-10T18:53:46.440 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:46 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pgp_num", "val": "16"}]': finished 2024-09-10T18:53:46.440 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:46 smithi018 ceph-mon[29336]: osdmap e202: 8 total, 8 up, 7 in 2024-09-10T18:53:46.440 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:46 smithi018 ceph-mon[29336]: 5.d deep-scrub starts 2024-09-10T18:53:46.440 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:46 smithi018 ceph-mon[29336]: 2.6 scrub starts 2024-09-10T18:53:46.440 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:46 smithi018 ceph-mon[29336]: 5.d deep-scrub ok 2024-09-10T18:53:46.482 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:53:46.482 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:53:46.508 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:46.508 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:46 smithi103 ceph-mon[33163]: pgmap v369: 168 pgs: 1 active+clean+scrubbing, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 164 active+clean; 7.4 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 782 op/s; 18213/363777 objects misplaced (5.007%); 11 MiB/s, 172 objects/s recovering 2024-09-10T18:53:46.508 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:46 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pgp_num", "val": "16"}]': finished 2024-09-10T18:53:46.509 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:46 smithi103 ceph-mon[33163]: osdmap e202: 8 total, 8 up, 7 in 2024-09-10T18:53:46.509 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:46 smithi103 ceph-mon[33163]: 5.d deep-scrub starts 2024-09-10T18:53:46.509 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:46 smithi103 ceph-mon[33163]: 2.6 scrub starts 2024-09-10T18:53:46.509 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:46 smithi103 ceph-mon[33163]: 5.d deep-scrub ok 2024-09-10T18:53:46.517 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:53:46.517 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:53:46.545 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:46.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:46.546 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:46.548 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:53:46.549 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:53:46.553 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-10T18:53:46.577 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:53:46.578 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:53:46.609 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:53:46.609 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:53:46.638 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:53:46.638 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:53:46.646 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:46.666 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:53:46.666 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:53:46.681 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:46.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:46.682 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:46.698 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:53:46.698 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:53:46.772 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:53:46 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:53:46.783 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:46.815 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:46.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:46.816 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:46.916 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:46.933 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:53:46.939 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:46.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:46.941 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:47.041 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:47.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:47.085 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:47.085 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:47.186 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:47.219 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:47.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:47.220 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:47.321 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:47.359 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:47.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:47.360 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:47.461 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:47.498 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:47.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:47.499 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:47.600 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:47.633 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:47.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:47.634 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:47.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:47 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3365142522' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:53:47.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:47 smithi018 ceph-mon[40196]: 8.7 deep-scrub starts 2024-09-10T18:53:47.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:47 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3365142522' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:53:47.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:47 smithi018 ceph-mon[29336]: 8.7 deep-scrub starts 2024-09-10T18:53:47.735 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:47.768 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:47.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:47.769 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:47.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:47 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3365142522' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:53:47.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:47 smithi103 ceph-mon[33163]: 8.7 deep-scrub starts 2024-09-10T18:53:47.871 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:47.913 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:47.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:47.914 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:48.015 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:48.046 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:48.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:48.047 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:48.147 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:48.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:48.189 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:48.189 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:48.290 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:48.331 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:48.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:48.333 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:48.433 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:48.466 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:48.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:48.467 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:48.568 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:48.605 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:48.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:48.606 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:48.692 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:48 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:48.692 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:48 smithi018 ceph-mon[40196]: pgmap v371: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 7.0 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.49k op/s; 16134/346848 objects misplaced (4.652%); 21 MiB/s, 333 objects/s recovering 2024-09-10T18:53:48.693 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:48 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:48.693 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:48 smithi018 ceph-mon[29336]: pgmap v371: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 7.0 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.49k op/s; 16134/346848 objects misplaced (4.652%); 21 MiB/s, 333 objects/s recovering 2024-09-10T18:53:48.695 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:48 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:48.695 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:48 smithi103 ceph-mon[33163]: pgmap v371: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 7.0 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.49k op/s; 16134/346848 objects misplaced (4.652%); 21 MiB/s, 333 objects/s recovering 2024-09-10T18:53:48.706 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:48.746 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:48.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:48.747 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:48.848 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T18:53:48.848 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:53:48.848 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:48.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:48.886 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:48.886 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:48.987 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:49.024 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:49.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:49.025 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:49.126 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:49.165 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:49.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:49.166 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:49.267 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:49.305 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:49.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:49.306 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:49.407 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:49.453 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:49.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:49.454 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:49.541 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-10T18:53:49.555 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:49.594 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:49.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:49.595 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:49.606 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is set 2024-09-10T18:53:49.696 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:49.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:49.736 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:49.737 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:49.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:49 smithi103 ceph-mon[33163]: 4.12 scrub starts 2024-09-10T18:53:49.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:49 smithi103 ceph-mon[33163]: pgmap v372: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 7.0 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.34k op/s; 16134/346848 objects misplaced (4.652%); 19 MiB/s, 299 objects/s recovering 2024-09-10T18:53:49.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:49 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3751715586' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:53:49.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:49 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:53:49.837 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:49.848 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T18:53:49.869 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:49.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:49.870 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:49.890 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:49 smithi018 ceph-mon[29336]: 4.12 scrub starts 2024-09-10T18:53:49.890 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:49 smithi018 ceph-mon[29336]: pgmap v372: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 7.0 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.34k op/s; 16134/346848 objects misplaced (4.652%); 19 MiB/s, 299 objects/s recovering 2024-09-10T18:53:49.890 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:49 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3751715586' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:53:49.890 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:49 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:53:49.891 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:49 smithi018 ceph-mon[40196]: 4.12 scrub starts 2024-09-10T18:53:49.891 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:49 smithi018 ceph-mon[40196]: pgmap v372: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 7.0 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.34k op/s; 16134/346848 objects misplaced (4.652%); 19 MiB/s, 299 objects/s recovering 2024-09-10T18:53:49.891 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:49 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3751715586' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:53:49.891 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:49 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:53:49.970 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:50.014 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:50.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:50.015 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:50.115 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:50.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:50.163 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:50.163 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:50.264 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:53:50.296 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:50.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:50.297 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:53:50.378 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:53:50.398 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:50.460 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:50.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:50.461 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:50.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:53:50 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:53:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:53:50.562 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:50.596 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:50.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:50.597 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:50.698 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:50.733 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:50.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:50.734 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:50.835 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:50.871 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:50.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:50.872 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:50.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:50 smithi018 ceph-mon[40196]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:53:50.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:50 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:53:50.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:50 smithi018 ceph-mon[40196]: osdmap e203: 8 total, 8 up, 7 in 2024-09-10T18:53:50.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:50 smithi018 ceph-mon[29336]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:53:50.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:50 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:53:50.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:50 smithi018 ceph-mon[29336]: osdmap e203: 8 total, 8 up, 7 in 2024-09-10T18:53:50.972 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:51.003 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:51.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:51.004 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:51.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:50 smithi103 ceph-mon[33163]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:53:51.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:50 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:53:51.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:50 smithi103 ceph-mon[33163]: osdmap e203: 8 total, 8 up, 7 in 2024-09-10T18:53:51.105 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:51.141 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:51.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:51.142 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:51.211 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-10T18:53:51.211 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T18:53:51.212 INFO:tasks.thrashosds.thrasher:Killing osd 0, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-10T18:53:51.212 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-10T18:53:51.212 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@osd.0 2024-09-10T18:53:51.243 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:51.245 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:53:50 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:53:50] "GET /metrics HTTP/1.1" 200 224113 "" "Prometheus/2.43.0" 2024-09-10T18:53:51.280 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:51.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:51.281 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:51.382 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:51.416 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:51.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:51.417 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:51.518 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:51.547 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:53:51 smithi018 systemd[1]: Stopping Ceph osd.0 for ba327f6c-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:53:51.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:51.556 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:51.556 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:51.657 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:51.691 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:51.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:51.692 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:51.793 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:51.828 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:51.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:51.829 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:51.930 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:51.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:51 smithi018 ceph-mon[40196]: pgmap v374: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 7.0 GiB data, 23 GiB used, 602 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 756 op/s; 16134/345615 objects misplaced (4.668%); 10 MiB/s, 160 objects/s recovering 2024-09-10T18:53:51.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:51 smithi018 ceph-mon[29336]: pgmap v374: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 7.0 GiB data, 23 GiB used, 602 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 756 op/s; 16134/345615 objects misplaced (4.668%); 10 MiB/s, 160 objects/s recovering 2024-09-10T18:53:51.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:51.969 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:51.970 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:52.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:51 smithi103 ceph-mon[33163]: pgmap v374: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 7.0 GiB data, 23 GiB used, 602 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 756 op/s; 16134/345615 objects misplaced (4.668%); 10 MiB/s, 160 objects/s recovering 2024-09-10T18:53:52.071 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:52.111 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:52.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:52.112 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:52.213 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:52.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:52.249 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:52.249 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:52.350 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:52.354 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-10T18:53:52.371 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:52.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:52.372 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:52.472 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:52.513 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:52.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:52.514 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:52.615 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:52.660 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:52.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:52.661 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:52.762 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:52.797 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:52.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:52.800 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:52.900 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:52.937 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:52.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:52.938 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:53.039 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:53.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:53.075 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:53.075 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:53.176 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:53.211 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:53.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:53.211 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:53.312 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:53.344 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:53.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:53.345 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:53.446 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:53.483 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:53.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:53.485 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:53.585 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:53.621 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:53.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:53.622 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:53.723 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:53.759 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:53.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:53.760 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:53.861 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:53.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:53.896 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:53.896 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:53.997 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:54.033 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:54.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:54.034 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:54.135 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:54.158 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:54.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:54.160 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:54.260 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:54.296 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:54.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:54.297 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:54.397 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:54.419 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:54.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:54.420 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:54.521 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:54.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:54 smithi103 ceph-mon[33163]: pgmap v375: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 6.6 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.52k op/s; 13864/327333 objects misplaced (4.235%); 21 MiB/s, 336 objects/s recovering 2024-09-10T18:53:54.550 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:54.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:54.551 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:54.652 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:54.686 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:54.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:54.687 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:54.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:54 smithi018 ceph-mon[29336]: pgmap v375: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 6.6 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.52k op/s; 13864/327333 objects misplaced (4.235%); 21 MiB/s, 336 objects/s recovering 2024-09-10T18:53:54.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:54 smithi018 ceph-mon[40196]: pgmap v375: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 6.6 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.52k op/s; 13864/327333 objects misplaced (4.235%); 21 MiB/s, 336 objects/s recovering 2024-09-10T18:53:54.789 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:54.809 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:54.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:54.810 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:54.911 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:54.953 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:54.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:54.954 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:55.055 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:55.092 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:55.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:55.093 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:55.194 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:55.216 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:55.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:55.217 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:55.318 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:55.355 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:55.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:55.356 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:55.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:53:55.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:53:55.456 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:55.495 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:55.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:55.496 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:55.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:53:55.597 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:55.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:55.630 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:55.630 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:55.731 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:55.767 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:55.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:55.768 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:55.869 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:55.891 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:55.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:55.892 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:55.993 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:56.032 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:56.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:56.033 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:56.133 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:56.170 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:56.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:56.171 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:56.272 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:56.293 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:56.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:56.294 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:56.395 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:56.418 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:56.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:56.419 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:56.517 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:56 smithi103 ceph-mon[33163]: pgmap v376: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 6.6 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.24k op/s; 13864/327333 objects misplaced (4.235%); 17 MiB/s, 273 objects/s recovering 2024-09-10T18:53:56.520 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:56.558 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:56.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:56.559 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:56.660 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:56.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:56.683 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:56.683 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:56.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:56 smithi018 ceph-mon[29336]: pgmap v376: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 6.6 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.24k op/s; 13864/327333 objects misplaced (4.235%); 17 MiB/s, 273 objects/s recovering 2024-09-10T18:53:56.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:56 smithi018 ceph-mon[40196]: pgmap v376: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 6.6 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.24k op/s; 13864/327333 objects misplaced (4.235%); 17 MiB/s, 273 objects/s recovering 2024-09-10T18:53:56.772 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:53:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:53:56.784 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:56.819 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:56.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:56.820 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:56.920 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:56.956 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:56.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:56.957 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:57.058 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:57.093 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:57.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:57.095 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:57.195 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:57.236 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:57.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:57.237 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:57.338 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:57.371 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:57.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:57.373 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:57.473 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:57.490 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:53:57.512 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:57.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:57.513 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:57.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:57 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 4]}]: dispatch 2024-09-10T18:53:57.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:57 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 4]}]: dispatch 2024-09-10T18:53:57.614 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:57.629 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:57 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 4]}]: dispatch 2024-09-10T18:53:57.629 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:57 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 4]}]: dispatch 2024-09-10T18:53:57.629 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:57 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 4]}]: dispatch 2024-09-10T18:53:57.629 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:57 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 4]}]: dispatch 2024-09-10T18:53:57.649 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:57.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:57.650 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:57.750 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:57.785 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:57.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:57.787 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:57.887 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:57.921 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:57.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:57.922 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:53:58.022 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:58.055 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:58.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:58.056 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:58.157 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:58.194 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:58.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:58.195 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:58.296 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:53:58.340 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:58.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:58.341 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:53:58.442 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:58.478 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:58.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:58.479 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:58.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:58 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:58.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:58 smithi103 ceph-mon[33163]: pgmap v377: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 6.3 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.21k op/s; 12012/310680 objects misplaced (3.866%); 16 MiB/s, 256 objects/s recovering 2024-09-10T18:53:58.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:58 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 4]}]': finished 2024-09-10T18:53:58.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:58 smithi103 ceph-mon[33163]: osdmap e204: 8 total, 8 up, 7 in 2024-09-10T18:53:58.557 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:58 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:58.557 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:58 smithi018 ceph-mon[40196]: pgmap v377: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 6.3 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.21k op/s; 12012/310680 objects misplaced (3.866%); 16 MiB/s, 256 objects/s recovering 2024-09-10T18:53:58.557 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:58 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 4]}]': finished 2024-09-10T18:53:58.557 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:58 smithi018 ceph-mon[40196]: osdmap e204: 8 total, 8 up, 7 in 2024-09-10T18:53:58.557 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:58 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:58.557 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:58 smithi018 ceph-mon[29336]: pgmap v377: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 6.3 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.21k op/s; 12012/310680 objects misplaced (3.866%); 16 MiB/s, 256 objects/s recovering 2024-09-10T18:53:58.557 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:58 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 4]}]': finished 2024-09-10T18:53:58.558 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:58 smithi018 ceph-mon[29336]: osdmap e204: 8 total, 8 up, 7 in 2024-09-10T18:53:58.580 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:58.614 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:58.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:58.615 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:58.716 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:58.749 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:58.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:58.750 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:58.851 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:58.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:58.887 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:58.887 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:58.935 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:53:58 smithi018 podman[119433]: 2024-09-10 18:53:58.557769611 +0000 UTC m=+7.275661340 container died e5549116159f398f840f923fb0930fcf96d4efb4e1a92af12b39d2aedf4ab5f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=kind_swanson, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-10T18:53:58.988 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:53:59.020 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:59.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:59.021 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:53:59.122 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:59.165 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:59.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:59.166 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:59.267 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:59.294 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:59.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:59.295 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:59.380 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:53:59.381 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:53:59.396 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:53:59.415 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:53:59.415 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:53:59.434 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:59.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:59.435 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:53:59.439 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:53:59.444 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:53:59.445 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:53:59.478 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:53:59.479 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:53:59.479 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:53:59 smithi018 ceph-mon[29336]: osdmap e205: 8 total, 8 up, 7 in 2024-09-10T18:53:59.479 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:53:59 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0[44072]: 2024-09-10T18:53:59.435+0000 7f4d3694c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:53:59.479 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:53:59 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0[44072]: 2024-09-10T18:53:59.435+0000 7f4d3694c640 -1 osd.0 205 *** Got signal Terminated *** 2024-09-10T18:53:59.480 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:53:59 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0[44072]: 2024-09-10T18:53:59.435+0000 7f4d3694c640 -1 osd.0 205 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:53:59.480 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:53:59 smithi018 ceph-mon[40196]: osdmap e205: 8 total, 8 up, 7 in 2024-09-10T18:53:59.518 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:53:59.518 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:53:59.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:53:59 smithi103 ceph-mon[33163]: osdmap e205: 8 total, 8 up, 7 in 2024-09-10T18:53:59.536 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:53:59.551 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:53:59.551 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:53:59.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:59.575 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:53:59.575 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:53:59.579 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:53:59.579 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:53:59.614 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:53:59.614 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:53:59.676 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:53:59.715 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:59.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:59.716 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:53:59.817 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:53:59.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:59.851 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:59.851 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:53:59.952 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:53:59.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:53:59.987 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:53:59.988 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:00.088 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:00.124 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:00.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:00.125 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:00.226 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:00.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:00.266 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:00.266 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:00.367 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:00.389 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:00.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:00.390 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:00.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:00 smithi018 ceph-mon[40196]: pgmap v380: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 6.3 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.46k op/s; 12012/310680 objects misplaced (3.866%); 20 MiB/s, 321 objects/s recovering 2024-09-10T18:54:00.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:00 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2893862300' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:54:00.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:00 smithi018 ceph-mon[40196]: osd.0 marked itself down and dead 2024-09-10T18:54:00.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:00 smithi018 ceph-mon[29336]: pgmap v380: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 6.3 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.46k op/s; 12012/310680 objects misplaced (3.866%); 20 MiB/s, 321 objects/s recovering 2024-09-10T18:54:00.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:00 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2893862300' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:54:00.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:00 smithi018 ceph-mon[29336]: osd.0 marked itself down and dead 2024-09-10T18:54:00.490 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:00.513 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:00.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:00.514 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:00.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:00 smithi103 ceph-mon[33163]: pgmap v380: 168 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 165 active+clean; 6.3 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.46k op/s; 12012/310680 objects misplaced (3.866%); 20 MiB/s, 321 objects/s recovering 2024-09-10T18:54:00.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:00 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2893862300' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:54:00.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:00 smithi103 ceph-mon[33163]: osd.0 marked itself down and dead 2024-09-10T18:54:00.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:54:00 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:54:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:54:00.615 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:00.638 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:00.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:00.639 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:00.740 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:00.763 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:00.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:00.764 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:00.865 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:00.887 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:00.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:00.889 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:00.989 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:01.014 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:01.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:01.015 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:01.116 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:01.139 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:01.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:01.140 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:01.241 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:01.264 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:01.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:01.265 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:01.288 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:54:01 smithi018 podman[119964]: 2024-09-10 18:54:01.018387313 +0000 UTC m=+1.934190226 container died 54bceced104fd484218e3a7fe764e2f4f0be2ffe76956dffaae101c6e0ec7274 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-10T18:54:01.288 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:54:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:54:00] "GET /metrics HTTP/1.1" 200 224118 "" "Prometheus/2.43.0" 2024-09-10T18:54:01.365 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:01.403 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:01.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:01.404 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:01.505 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:01.542 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:01.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:01.543 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:01.546 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:01 smithi018 ceph-mon[29336]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-10T18:54:01.546 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:01 smithi018 ceph-mon[29336]: osdmap e206: 8 total, 7 up, 7 in 2024-09-10T18:54:01.546 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:01 smithi018 ceph-mon[40196]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-10T18:54:01.547 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:01 smithi018 ceph-mon[40196]: osdmap e206: 8 total, 7 up, 7 in 2024-09-10T18:54:01.644 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:01.680 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:01.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:01.682 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:01.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:01 smithi103 ceph-mon[33163]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-10T18:54:01.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:01 smithi103 ceph-mon[33163]: osdmap e206: 8 total, 7 up, 7 in 2024-09-10T18:54:01.783 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:01.813 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:01.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:01.814 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:01.815 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:54:01 smithi018 podman[119964]: 2024-09-10 18:54:01.545868103 +0000 UTC m=+2.461671001 container cleanup 54bceced104fd484218e3a7fe764e2f4f0be2ffe76956dffaae101c6e0ec7274 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-10T18:54:01.815 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:54:01 smithi018 bash[119964]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0 2024-09-10T18:54:01.915 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:01.952 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:01.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:01.953 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:02.053 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:02.094 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:02.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:02.095 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:02.194 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:54:01 smithi018 podman[120160]: 2024-09-10 18:54:01.814535533 +0000 UTC m=+0.972345931 container remove 54bceced104fd484218e3a7fe764e2f4f0be2ffe76956dffaae101c6e0ec7274 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-10T18:54:02.196 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:02.225 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:02.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:02.226 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:02.327 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:02.363 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:02.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:02.364 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:02.464 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:02.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:02.498 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:02.498 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:02.599 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:02.610 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:02 smithi018 ceph-mon[40196]: pgmap v382: 168 pgs: 11 peering, 18 stale+active+clean, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 136 active+clean; 6.3 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 1.03k op/s; 12012/308790 objects misplaced (3.890%); 12 MiB/s, 194 objects/s recovering 2024-09-10T18:54:02.610 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:02 smithi018 ceph-mon[40196]: Health check failed: Reduced data availability: 2 pgs inactive, 3 pgs peering (PG_AVAILABILITY) 2024-09-10T18:54:02.610 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:02 smithi018 ceph-mon[40196]: osdmap e207: 8 total, 7 up, 7 in 2024-09-10T18:54:02.610 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:02 smithi018 ceph-mon[40196]: 8.9 continuing backfill to osd.3 from (190'8053,205'12128] 8:99f52f0a:::benchmark_data_smithi018_106531_object88730:head to 205'12128 2024-09-10T18:54:02.610 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:02 smithi018 ceph-mon[29336]: pgmap v382: 168 pgs: 11 peering, 18 stale+active+clean, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 136 active+clean; 6.3 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 1.03k op/s; 12012/308790 objects misplaced (3.890%); 12 MiB/s, 194 objects/s recovering 2024-09-10T18:54:02.610 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:02 smithi018 ceph-mon[29336]: Health check failed: Reduced data availability: 2 pgs inactive, 3 pgs peering (PG_AVAILABILITY) 2024-09-10T18:54:02.611 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:02 smithi018 ceph-mon[29336]: osdmap e207: 8 total, 7 up, 7 in 2024-09-10T18:54:02.611 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:02 smithi018 ceph-mon[29336]: 8.9 continuing backfill to osd.3 from (190'8053,205'12128] 8:99f52f0a:::benchmark_data_smithi018_106531_object88730:head to 205'12128 2024-09-10T18:54:02.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:02.644 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:02.644 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:02.745 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:02.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:02 smithi103 ceph-mon[33163]: pgmap v382: 168 pgs: 11 peering, 18 stale+active+clean, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 136 active+clean; 6.3 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 1.03k op/s; 12012/308790 objects misplaced (3.890%); 12 MiB/s, 194 objects/s recovering 2024-09-10T18:54:02.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:02 smithi103 ceph-mon[33163]: Health check failed: Reduced data availability: 2 pgs inactive, 3 pgs peering (PG_AVAILABILITY) 2024-09-10T18:54:02.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:02 smithi103 ceph-mon[33163]: osdmap e207: 8 total, 7 up, 7 in 2024-09-10T18:54:02.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:02 smithi103 ceph-mon[33163]: 8.9 continuing backfill to osd.3 from (190'8053,205'12128] 8:99f52f0a:::benchmark_data_smithi018_106531_object88730:head to 205'12128 2024-09-10T18:54:02.774 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:02.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:02.775 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:02.876 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:02.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:02.908 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:02.908 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:03.009 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:03.048 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:03.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:03.049 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:03.150 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:03.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:03.190 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:03.190 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:03.291 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:03.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:03.324 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:03.324 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:03.425 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:03.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:03.465 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:03.465 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:03.565 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:03.602 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:03.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:03.603 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:03.660 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:03 smithi018 ceph-mon[29336]: Health check failed: Degraded data redundancy: 12342/295197 objects degraded (4.181%), 14 pgs degraded (PG_DEGRADED) 2024-09-10T18:54:03.660 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:03 smithi018 ceph-mon[40196]: Health check failed: Degraded data redundancy: 12342/295197 objects degraded (4.181%), 14 pgs degraded (PG_DEGRADED) 2024-09-10T18:54:03.704 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:03.736 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:03.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:03.737 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:03.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:03 smithi103 ceph-mon[33163]: Health check failed: Degraded data redundancy: 12342/295197 objects degraded (4.181%), 14 pgs degraded (PG_DEGRADED) 2024-09-10T18:54:03.838 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:03.873 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:03.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:03.874 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:03.961 INFO:teuthology.orchestra.run.smithi018.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-10T18:54:03.975 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:04.006 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:04.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:04.007 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:04.108 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:04.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:04.142 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:04.143 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:04.243 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:04.277 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:04.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:04.278 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:04.378 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:04.420 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:04.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:04.421 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:04.522 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:04.561 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:04.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:04.562 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:04.592 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:04 smithi018 ceph-mon[29336]: pgmap v384: 168 pgs: 1 active+remapped+backfilling, 18 active+undersized, 1 remapped+peering, 36 peering, 1 active+remapped+backfill_toofull, 14 active+undersized+degraded, 97 active+clean; 6.0 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.5 KiB/s rd, 12 KiB/s wr, 871 op/s; 12342/295197 objects degraded (4.181%); 7201/295197 objects misplaced (2.439%); 20 MiB/s, 323 objects/s recovering 2024-09-10T18:54:04.592 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:04 smithi018 ceph-mon[40196]: pgmap v384: 168 pgs: 1 active+remapped+backfilling, 18 active+undersized, 1 remapped+peering, 36 peering, 1 active+remapped+backfill_toofull, 14 active+undersized+degraded, 97 active+clean; 6.0 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.5 KiB/s rd, 12 KiB/s wr, 871 op/s; 12342/295197 objects degraded (4.181%); 7201/295197 objects misplaced (2.439%); 20 MiB/s, 323 objects/s recovering 2024-09-10T18:54:04.663 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:04.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:04.702 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:04.702 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:04.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:04 smithi103 ceph-mon[33163]: pgmap v384: 168 pgs: 1 active+remapped+backfilling, 18 active+undersized, 1 remapped+peering, 36 peering, 1 active+remapped+backfill_toofull, 14 active+undersized+degraded, 97 active+clean; 6.0 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.5 KiB/s rd, 12 KiB/s wr, 871 op/s; 12342/295197 objects degraded (4.181%); 7201/295197 objects misplaced (2.439%); 20 MiB/s, 323 objects/s recovering 2024-09-10T18:54:04.802 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:04.840 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:04.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:04.841 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:04.942 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:04.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:04.980 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:04.981 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:05.081 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:05.116 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:05.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:05.117 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:05.204 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-10T18:54:05.205 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-10T18:54:05.217 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:05.280 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:05.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:05.281 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:05.365 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is set 2024-09-10T18:54:05.382 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:05.410 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:05.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:05.411 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:05.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:05 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2937178587' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:54:05.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:05 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2937178587' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:54:05.512 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:05.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:05.546 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:05.546 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:05.647 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:05.681 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:05.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:05.682 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:05.700 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T18:54:05.741 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:54:05 smithi018 podman[120670]: 2024-09-10 18:54:05.366619973 +0000 UTC m=+0.036799316 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:54:05.741 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:54:05 smithi018 podman[120670]: 2024-09-10 18:54:05.512824161 +0000 UTC m=+0.183003481 container create 269f5cbcf93b76c58ce52dec1ffbe4d5511019bf19a690c275c3335207e75f35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0-deactivate, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-10T18:54:05.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:05 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2937178587' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:54:05.782 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:05.819 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:05.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:05.820 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:05.921 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:05.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:05.957 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:05.958 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:06.058 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:06.077 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:54:05 smithi018 podman[120670]: 2024-09-10 18:54:05.967808841 +0000 UTC m=+0.637988158 container init 269f5cbcf93b76c58ce52dec1ffbe4d5511019bf19a690c275c3335207e75f35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-10T18:54:06.077 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:54:05 smithi018 podman[120670]: 2024-09-10 18:54:05.975326389 +0000 UTC m=+0.645505703 container start 269f5cbcf93b76c58ce52dec1ffbe4d5511019bf19a690c275c3335207e75f35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True) 2024-09-10T18:54:06.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:06.097 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:06.097 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:06.179 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:54:06.198 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:06.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:06.239 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:06.239 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:06.340 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:06.348 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:54:06 smithi018 podman[120670]: 2024-09-10 18:54:06.076049363 +0000 UTC m=+0.746228697 container attach 269f5cbcf93b76c58ce52dec1ffbe4d5511019bf19a690c275c3335207e75f35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-10T18:54:06.372 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:06.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:06.373 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:06.474 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:06.509 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:06.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:06.510 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:06.611 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:06.639 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:06.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:06.640 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:06.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:06 smithi018 ceph-mon[40196]: pgmap v385: 168 pgs: 1 active+remapped+backfilling, 18 active+undersized, 1 remapped+peering, 36 peering, 1 active+remapped+backfill_toofull, 14 active+undersized+degraded, 97 active+clean; 6.0 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.3 KiB/s rd, 11 KiB/s wr, 746 op/s; 12342/295197 objects degraded (4.181%); 7201/295197 objects misplaced (2.439%); 17 MiB/s, 277 objects/s recovering 2024-09-10T18:54:06.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:06 smithi018 ceph-mon[40196]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:54:06.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:06 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2937178587' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:54:06.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:06 smithi018 ceph-mon[40196]: osdmap e208: 8 total, 7 up, 7 in 2024-09-10T18:54:06.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:06 smithi018 ceph-mon[29336]: pgmap v385: 168 pgs: 1 active+remapped+backfilling, 18 active+undersized, 1 remapped+peering, 36 peering, 1 active+remapped+backfill_toofull, 14 active+undersized+degraded, 97 active+clean; 6.0 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.3 KiB/s rd, 11 KiB/s wr, 746 op/s; 12342/295197 objects degraded (4.181%); 7201/295197 objects misplaced (2.439%); 17 MiB/s, 277 objects/s recovering 2024-09-10T18:54:06.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:06 smithi018 ceph-mon[29336]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:54:06.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:06 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2937178587' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:54:06.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:06 smithi018 ceph-mon[29336]: osdmap e208: 8 total, 7 up, 7 in 2024-09-10T18:54:06.741 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:06.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:06 smithi103 ceph-mon[33163]: pgmap v385: 168 pgs: 1 active+remapped+backfilling, 18 active+undersized, 1 remapped+peering, 36 peering, 1 active+remapped+backfill_toofull, 14 active+undersized+degraded, 97 active+clean; 6.0 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.3 KiB/s rd, 11 KiB/s wr, 746 op/s; 12342/295197 objects degraded (4.181%); 7201/295197 objects misplaced (2.439%); 17 MiB/s, 277 objects/s recovering 2024-09-10T18:54:06.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:06 smithi103 ceph-mon[33163]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:54:06.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:06 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2937178587' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:54:06.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:06 smithi103 ceph-mon[33163]: osdmap e208: 8 total, 7 up, 7 in 2024-09-10T18:54:06.773 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:54:06 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:54:06.773 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:06.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:06.774 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:06.875 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:06.908 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:06.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:06.909 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:07.010 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:07.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:07.044 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:07.044 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:07.145 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:07.184 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:07.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:07.185 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:07.286 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:07.317 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:07.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:07.319 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:07.401 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:54:07 smithi018 podman[120670]: 2024-09-10 18:54:07.167822932 +0000 UTC m=+1.838002260 container died 269f5cbcf93b76c58ce52dec1ffbe4d5511019bf19a690c275c3335207e75f35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-10T18:54:07.419 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:07.454 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:07.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:07.455 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:07.556 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:07.591 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:07.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:07.592 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:07.692 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:07.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:07 smithi018 ceph-mon[40196]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-10T18:54:07.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:07 smithi018 ceph-mon[40196]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 7 pgs inactive, 11 pgs peering) 2024-09-10T18:54:07.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:07 smithi018 ceph-mon[29336]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-10T18:54:07.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:07 smithi018 ceph-mon[29336]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 7 pgs inactive, 11 pgs peering) 2024-09-10T18:54:07.724 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:07.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:07.726 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:07.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:07 smithi103 ceph-mon[33163]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-10T18:54:07.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:07 smithi103 ceph-mon[33163]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 7 pgs inactive, 11 pgs peering) 2024-09-10T18:54:07.827 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:07.866 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:07.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:07.868 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:07.968 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:08.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:08.010 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:08.010 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:08.111 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:08.149 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:08.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:08.150 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:08.251 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:08.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:08.288 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:08.288 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:08.389 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:08.422 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:08.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:08.423 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:08.524 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:08.555 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:08.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:08.557 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:08.657 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:08.680 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:08.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:08.681 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:08.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:08 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:08.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:08 smithi018 ceph-mon[29336]: pgmap v387: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 1 active+remapped+backfilling, 41 active+undersized, 1 active+remapped+backfill_toofull, 27 active+undersized+degraded, 97 active+clean; 5.6 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 2.0 KiB/s rd, 11 KiB/s wr, 1.79k op/s; 30454/273639 objects degraded (11.129%); 11679/273639 objects misplaced (4.268%); 24 MiB/s, 384 objects/s recovering 2024-09-10T18:54:08.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:08 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:08.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:08 smithi018 ceph-mon[40196]: pgmap v387: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 1 active+remapped+backfilling, 41 active+undersized, 1 active+remapped+backfill_toofull, 27 active+undersized+degraded, 97 active+clean; 5.6 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 2.0 KiB/s rd, 11 KiB/s wr, 1.79k op/s; 30454/273639 objects degraded (11.129%); 11679/273639 objects misplaced (4.268%); 24 MiB/s, 384 objects/s recovering 2024-09-10T18:54:08.769 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-10T18:54:08.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:08 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:08.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:08 smithi103 ceph-mon[33163]: pgmap v387: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 1 active+remapped+backfilling, 41 active+undersized, 1 active+remapped+backfill_toofull, 27 active+undersized+degraded, 97 active+clean; 5.6 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 2.0 KiB/s rd, 11 KiB/s wr, 1.79k op/s; 30454/273639 objects degraded (11.129%); 11679/273639 objects misplaced (4.268%); 24 MiB/s, 384 objects/s recovering 2024-09-10T18:54:08.781 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:08.834 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:08.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:08.835 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:08.870 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:54:08.887 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@osd.0.service' 2024-09-10T18:54:08.936 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:08.958 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:08.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:08.959 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:09.006 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:54:09.006 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-10T18:54:09.007 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.0 2024-09-10T18:54:09.007 INFO:tasks.thrashosds.thrasher:remote for osd 0 is ubuntu@smithi018.front.sepia.ceph.com 2024-09-10T18:54:09.007 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -n osd.0 --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op list-pgs 2024-09-10T18:54:09.059 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:09.082 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:09.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:09.083 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:09.184 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:09.206 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:09.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:09.207 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:09.308 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:09.329 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:09.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:09.330 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:09.416 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:09 smithi018 ceph-mon[40196]: osdmap e209: 8 total, 7 up, 7 in 2024-09-10T18:54:09.417 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:09 smithi018 ceph-mon[40196]: Health check update: Degraded data redundancy: 30454/273639 objects degraded (11.129%), 28 pgs degraded (PG_DEGRADED) 2024-09-10T18:54:09.417 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:09 smithi018 ceph-mon[29336]: osdmap e209: 8 total, 7 up, 7 in 2024-09-10T18:54:09.417 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:09 smithi018 ceph-mon[29336]: Health check update: Degraded data redundancy: 30454/273639 objects degraded (11.129%), 28 pgs degraded (PG_DEGRADED) 2024-09-10T18:54:09.430 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:09.436 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/osd.0/config 2024-09-10T18:54:09.453 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:09.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:09.455 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:09.555 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:09.593 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:09.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:09.594 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:09.694 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:09.727 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:09.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:09.728 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:09.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:09 smithi103 ceph-mon[33163]: osdmap e209: 8 total, 7 up, 7 in 2024-09-10T18:54:09.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:09 smithi103 ceph-mon[33163]: Health check update: Degraded data redundancy: 30454/273639 objects degraded (11.129%), 28 pgs degraded (PG_DEGRADED) 2024-09-10T18:54:09.829 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:09.857 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:09.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:09.858 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:09.958 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:09.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:09.995 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:09.995 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:10.096 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:10.125 INFO:teuthology.orchestra.run.smithi018.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-10T18:54:10.135 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:10.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:10.136 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:10.237 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:10.274 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:10.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:10.275 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:10.376 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:10.405 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:10.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:10.406 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:10.507 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:10.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:54:10 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:54:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:54:10.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:10 smithi103 ceph-mon[33163]: pgmap v389: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 1 active+remapped+backfilling, 41 active+undersized, 1 active+remapped+backfill_toofull, 27 active+undersized+degraded, 97 active+clean; 5.6 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 2.0 KiB/s rd, 9.4 KiB/s wr, 1.25k op/s; 30454/273639 objects degraded (11.129%); 11679/273639 objects misplaced (4.268%); 17 MiB/s, 263 objects/s recovering 2024-09-10T18:54:10.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:10 smithi103 ceph-mon[33163]: osdmap e210: 8 total, 7 up, 7 in 2024-09-10T18:54:10.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:10 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:54:10.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:10 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:10.543 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:10.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:10.544 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:10.645 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:10.675 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:10.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:10.676 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:10.685 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:10 smithi018 ceph-mon[29336]: pgmap v389: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 1 active+remapped+backfilling, 41 active+undersized, 1 active+remapped+backfill_toofull, 27 active+undersized+degraded, 97 active+clean; 5.6 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 2.0 KiB/s rd, 9.4 KiB/s wr, 1.25k op/s; 30454/273639 objects degraded (11.129%); 11679/273639 objects misplaced (4.268%); 17 MiB/s, 263 objects/s recovering 2024-09-10T18:54:10.685 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:10 smithi018 ceph-mon[29336]: osdmap e210: 8 total, 7 up, 7 in 2024-09-10T18:54:10.686 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:10 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:54:10.686 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:10 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:10.686 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:10 smithi018 ceph-mon[40196]: pgmap v389: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 1 active+remapped+backfilling, 41 active+undersized, 1 active+remapped+backfill_toofull, 27 active+undersized+degraded, 97 active+clean; 5.6 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 2.0 KiB/s rd, 9.4 KiB/s wr, 1.25k op/s; 30454/273639 objects degraded (11.129%); 11679/273639 objects misplaced (4.268%); 17 MiB/s, 263 objects/s recovering 2024-09-10T18:54:10.686 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:10 smithi018 ceph-mon[40196]: osdmap e210: 8 total, 7 up, 7 in 2024-09-10T18:54:10.686 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:10 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:54:10.686 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:10 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:10.777 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:10.813 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:10.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:10.815 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:10.916 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:10.959 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:10.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:10.960 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:11.061 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:11.090 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:11.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:11.092 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:11.192 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:11.241 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:11.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:11.243 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:11.343 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:11.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:11.384 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:11.384 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:11.441 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:54:10 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:54:10] "GET /metrics HTTP/1.1" 200 224269 "" "Prometheus/2.43.0" 2024-09-10T18:54:11.485 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:11.516 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:11.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:11.517 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:11.618 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:11.646 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:11.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:11.648 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:11.707 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-10T18:54:11.708 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-10T18:54:11.748 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:11.783 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:11.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:11.784 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:11.885 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:11.921 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:11.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:11.922 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:12.023 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:12.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:12.064 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:12.065 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:12.165 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:12.204 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:12.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:12.205 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:12.306 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:12.345 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:12.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:12.346 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:12.447 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:12.484 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:12.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:12.488 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:12.589 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:12.626 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:12.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:12.628 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:12.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:12 smithi018 ceph-mon[40196]: pgmap v391: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 1 active+remapped+backfilling, 41 active+undersized, 1 active+remapped+backfill_toofull, 27 active+undersized+degraded, 97 active+clean; 5.5 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 1.32k op/s; 29702/271383 objects degraded (10.945%); 11679/271383 objects misplaced (4.304%); 7.7 MiB/s, 122 objects/s recovering 2024-09-10T18:54:12.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:12 smithi018 ceph-mon[29336]: pgmap v391: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 1 active+remapped+backfilling, 41 active+undersized, 1 active+remapped+backfill_toofull, 27 active+undersized+degraded, 97 active+clean; 5.5 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 1.32k op/s; 29702/271383 objects degraded (10.945%); 11679/271383 objects misplaced (4.304%); 7.7 MiB/s, 122 objects/s recovering 2024-09-10T18:54:12.729 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:12.757 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:12.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:12.758 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:12.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:12 smithi103 ceph-mon[33163]: pgmap v391: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 1 active+remapped+backfilling, 41 active+undersized, 1 active+remapped+backfill_toofull, 27 active+undersized+degraded, 97 active+clean; 5.5 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 1.32k op/s; 29702/271383 objects degraded (10.945%); 11679/271383 objects misplaced (4.304%); 7.7 MiB/s, 122 objects/s recovering 2024-09-10T18:54:12.858 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:12.897 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:12.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:12.898 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:12.966 INFO:teuthology.orchestra.run.smithi018.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-10T18:54:12.967 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:54:12.967 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:54:12.975 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:54:12.998 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:54:12.998 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:54:12.999 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:13.024 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:54:13.024 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:54:13.031 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:13.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:13.032 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:13.055 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:54:13.055 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:54:13.090 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:54:13.090 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:54:13.123 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:54:13.123 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:54:13.133 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:13.154 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:54:13.155 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:54:13.166 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:13.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:13.167 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:13.268 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:13.305 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:13.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:13.306 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:13.406 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:13.442 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:13.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:13.443 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:13.544 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:13.575 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:13.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:13.576 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:13.677 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:13.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:13 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1297443308' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:54:13.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:13 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1297443308' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:54:13.706 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:13.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:13.707 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:13.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:13 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1297443308' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:54:13.808 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:13.842 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:13.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:13.843 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:13.944 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:13.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:13.983 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:13.983 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:14.084 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:14.111 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:14.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:14.112 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:14.213 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:14.246 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:14.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:14.247 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:14.348 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:14.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:14.383 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:14.383 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:14.484 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:14.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:14.515 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:14.515 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:14.616 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:14.658 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:14.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:14.659 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:14.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:14 smithi018 ceph-mon[40196]: pgmap v392: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 41 active+undersized, 1 active+remapped+backfill_toofull, 27 active+undersized+degraded, 98 active+clean; 5.2 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.69k op/s; 28397/255399 objects degraded (11.119%); 10695/255399 objects misplaced (4.188%); 7.9 MiB/s, 125 objects/s recovering 2024-09-10T18:54:14.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:14 smithi018 ceph-mon[40196]: Health check update: Degraded data redundancy: 28397/255399 objects degraded (11.119%), 28 pgs degraded (PG_DEGRADED) 2024-09-10T18:54:14.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:14 smithi018 ceph-mon[29336]: pgmap v392: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 41 active+undersized, 1 active+remapped+backfill_toofull, 27 active+undersized+degraded, 98 active+clean; 5.2 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.69k op/s; 28397/255399 objects degraded (11.119%); 10695/255399 objects misplaced (4.188%); 7.9 MiB/s, 125 objects/s recovering 2024-09-10T18:54:14.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:14 smithi018 ceph-mon[29336]: Health check update: Degraded data redundancy: 28397/255399 objects degraded (11.119%), 28 pgs degraded (PG_DEGRADED) 2024-09-10T18:54:14.760 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:14.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:14 smithi103 ceph-mon[33163]: pgmap v392: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 41 active+undersized, 1 active+remapped+backfill_toofull, 27 active+undersized+degraded, 98 active+clean; 5.2 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.69k op/s; 28397/255399 objects degraded (11.119%); 10695/255399 objects misplaced (4.188%); 7.9 MiB/s, 125 objects/s recovering 2024-09-10T18:54:14.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:14 smithi103 ceph-mon[33163]: Health check update: Degraded data redundancy: 28397/255399 objects degraded (11.119%), 28 pgs degraded (PG_DEGRADED) 2024-09-10T18:54:14.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:14.798 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:14.798 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:14.899 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:14.936 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:14.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:14.937 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:15.038 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:15.084 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:15.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:15.085 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:15.186 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:15.225 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:15.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:15.226 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:15.327 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:15.355 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:15.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:15.356 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:15.457 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:15.491 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:15.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:15.492 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:15.593 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:15.628 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:15.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:15.629 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:15.730 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:15.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:15.778 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:15.778 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:15.879 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:15.916 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:15.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:15.917 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:16.018 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:16.063 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:16.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:16.064 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:16.119 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-10T18:54:16.120 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-10T18:54:16.159 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:54:16.165 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:16.200 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:16.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:16.201 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:16.302 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:16.339 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:16.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:16.340 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:16.440 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:16.479 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:16.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:16.480 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:16.515 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:16 smithi018 ceph-mon[29336]: pgmap v393: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 41 active+undersized, 1 active+remapped+backfill_toofull, 27 active+undersized+degraded, 98 active+clean; 5.2 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 760 op/s; 28397/255399 objects degraded (11.119%); 10695/255399 objects misplaced (4.188%); 1.9 MiB/s, 31 objects/s recovering 2024-09-10T18:54:16.515 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:16 smithi018 ceph-mon[40196]: pgmap v393: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 41 active+undersized, 1 active+remapped+backfill_toofull, 27 active+undersized+degraded, 98 active+clean; 5.2 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 760 op/s; 28397/255399 objects degraded (11.119%); 10695/255399 objects misplaced (4.188%); 1.9 MiB/s, 31 objects/s recovering 2024-09-10T18:54:16.580 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:16.614 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:16.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:16.615 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:16.716 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:16.727 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:16 smithi103 ceph-mon[33163]: pgmap v393: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 41 active+undersized, 1 active+remapped+backfill_toofull, 27 active+undersized+degraded, 98 active+clean; 5.2 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 760 op/s; 28397/255399 objects degraded (11.119%); 10695/255399 objects misplaced (4.188%); 1.9 MiB/s, 31 objects/s recovering 2024-09-10T18:54:16.728 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:54:16 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:54:16.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:16.758 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:16.758 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:16.859 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:16.900 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:16.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:16.901 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:17.002 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:17.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:17.029 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:17.029 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:17.130 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:17.168 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:17.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:17.169 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:17.270 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:17.307 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:17.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:17.308 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:17.409 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:17.442 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is unset 2024-09-10T18:54:17.447 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:17.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:17.448 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:17.549 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:17.560 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:17 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1560429938' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:54:17.560 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:17 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1560429938' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:54:17.588 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:17.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:17.589 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:17.689 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:17.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:17.725 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:17.725 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:17.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:17 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1560429938' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:54:17.825 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:17.861 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:17.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:17.862 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:17.963 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:17.998 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:17.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:17.999 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:18.012 INFO:teuthology.orchestra.run.smithi018.stdout:8.e 2024-09-10T18:54:18.012 INFO:teuthology.orchestra.run.smithi018.stdout:8.c 2024-09-10T18:54:18.012 INFO:teuthology.orchestra.run.smithi018.stdout:8.9 2024-09-10T18:54:18.012 INFO:teuthology.orchestra.run.smithi018.stdout:8.6 2024-09-10T18:54:18.012 INFO:teuthology.orchestra.run.smithi018.stdout:6.f 2024-09-10T18:54:18.012 INFO:teuthology.orchestra.run.smithi018.stdout:6.d 2024-09-10T18:54:18.012 INFO:teuthology.orchestra.run.smithi018.stdout:6.a 2024-09-10T18:54:18.013 INFO:teuthology.orchestra.run.smithi018.stdout:6.9 2024-09-10T18:54:18.013 INFO:teuthology.orchestra.run.smithi018.stdout:4.12 2024-09-10T18:54:18.013 INFO:teuthology.orchestra.run.smithi018.stdout:1.0 2024-09-10T18:54:18.013 INFO:teuthology.orchestra.run.smithi018.stdout:4.0 2024-09-10T18:54:18.016 INFO:teuthology.orchestra.run.smithi018.stdout:4.19 2024-09-10T18:54:18.016 INFO:teuthology.orchestra.run.smithi018.stdout:2.10 2024-09-10T18:54:18.016 INFO:teuthology.orchestra.run.smithi018.stdout:2.f 2024-09-10T18:54:18.016 INFO:teuthology.orchestra.run.smithi018.stdout:2.0 2024-09-10T18:54:18.016 INFO:teuthology.orchestra.run.smithi018.stdout:5.13 2024-09-10T18:54:18.017 INFO:teuthology.orchestra.run.smithi018.stdout:2.8 2024-09-10T18:54:18.017 INFO:teuthology.orchestra.run.smithi018.stdout:4.6 2024-09-10T18:54:18.017 INFO:teuthology.orchestra.run.smithi018.stdout:5.4 2024-09-10T18:54:18.017 INFO:teuthology.orchestra.run.smithi018.stdout:2.5 2024-09-10T18:54:18.017 INFO:teuthology.orchestra.run.smithi018.stdout:4.b 2024-09-10T18:54:18.017 INFO:teuthology.orchestra.run.smithi018.stdout:5.3 2024-09-10T18:54:18.017 INFO:teuthology.orchestra.run.smithi018.stdout:2.1f 2024-09-10T18:54:18.017 INFO:teuthology.orchestra.run.smithi018.stdout:2.1e 2024-09-10T18:54:18.017 INFO:teuthology.orchestra.run.smithi018.stdout:2.1d 2024-09-10T18:54:18.017 INFO:teuthology.orchestra.run.smithi018.stdout:4.1c 2024-09-10T18:54:18.017 INFO:teuthology.orchestra.run.smithi018.stdout:4.3 2024-09-10T18:54:18.017 INFO:teuthology.orchestra.run.smithi018.stdout:6.13 2024-09-10T18:54:18.017 INFO:teuthology.orchestra.run.smithi018.stdout:2.15 2024-09-10T18:54:18.018 INFO:teuthology.orchestra.run.smithi018.stdout:8.2 2024-09-10T18:54:18.018 INFO:teuthology.orchestra.run.smithi018.stdout:4.1a 2024-09-10T18:54:18.018 INFO:teuthology.orchestra.run.smithi018.stdout:4.16 2024-09-10T18:54:18.018 INFO:teuthology.orchestra.run.smithi018.stdout:1.1 2024-09-10T18:54:18.018 INFO:teuthology.orchestra.run.smithi018.stdout:6.19 2024-09-10T18:54:18.018 INFO:teuthology.orchestra.run.smithi018.stdout:2.4 2024-09-10T18:54:18.018 INFO:teuthology.orchestra.run.smithi018.stdout:1.6 2024-09-10T18:54:18.018 INFO:teuthology.orchestra.run.smithi018.stdout:1.c 2024-09-10T18:54:18.018 INFO:teuthology.orchestra.run.smithi018.stdout:2.19 2024-09-10T18:54:18.018 INFO:teuthology.orchestra.run.smithi018.stdout:3.1 2024-09-10T18:54:18.018 INFO:teuthology.orchestra.run.smithi018.stdout:5.a 2024-09-10T18:54:18.018 INFO:teuthology.orchestra.run.smithi018.stdout:5.1b 2024-09-10T18:54:18.018 INFO:teuthology.orchestra.run.smithi018.stdout:3.0 2024-09-10T18:54:18.018 INFO:teuthology.orchestra.run.smithi018.stdout:2.c 2024-09-10T18:54:18.019 INFO:teuthology.orchestra.run.smithi018.stdout:1.b 2024-09-10T18:54:18.019 INFO:teuthology.orchestra.run.smithi018.stdout:1.2 2024-09-10T18:54:18.019 INFO:teuthology.orchestra.run.smithi018.stdout:2.1 2024-09-10T18:54:18.019 INFO:teuthology.orchestra.run.smithi018.stdout:4.1e 2024-09-10T18:54:18.019 INFO:teuthology.orchestra.run.smithi018.stdout:8.1 2024-09-10T18:54:18.019 INFO:teuthology.orchestra.run.smithi018.stdout:4.7 2024-09-10T18:54:18.019 INFO:teuthology.orchestra.run.smithi018.stdout:5.15 2024-09-10T18:54:18.019 INFO:teuthology.orchestra.run.smithi018.stdout:5.7 2024-09-10T18:54:18.019 INFO:teuthology.orchestra.run.smithi018.stdout:5.1d 2024-09-10T18:54:18.019 INFO:teuthology.orchestra.run.smithi018.stdout:5.b 2024-09-10T18:54:18.019 INFO:teuthology.orchestra.run.smithi018.stdout:5.1e 2024-09-10T18:54:18.019 INFO:teuthology.orchestra.run.smithi018.stdout:5.2 2024-09-10T18:54:18.019 INFO:teuthology.orchestra.run.smithi018.stdout:1.11 2024-09-10T18:54:18.019 INFO:teuthology.orchestra.run.smithi018.stdout:5.8 2024-09-10T18:54:18.019 INFO:teuthology.orchestra.run.smithi018.stdout:5.c 2024-09-10T18:54:18.020 INFO:teuthology.orchestra.run.smithi018.stdout:6.17 2024-09-10T18:54:18.020 INFO:teuthology.orchestra.run.smithi018.stdout:5.d 2024-09-10T18:54:18.020 INFO:teuthology.orchestra.run.smithi018.stdout:2.13 2024-09-10T18:54:18.020 INFO:teuthology.orchestra.run.smithi018.stdout:5.e 2024-09-10T18:54:18.020 INFO:teuthology.orchestra.run.smithi018.stdout:6.0 2024-09-10T18:54:18.020 INFO:teuthology.orchestra.run.smithi018.stdout:6.14 2024-09-10T18:54:18.020 INFO:teuthology.orchestra.run.smithi018.stdout:4.2 2024-09-10T18:54:18.020 INFO:teuthology.orchestra.run.smithi018.stdout:6.10 2024-09-10T18:54:18.020 INFO:teuthology.orchestra.run.smithi018.stdout:6.11 2024-09-10T18:54:18.020 INFO:teuthology.orchestra.run.smithi018.stdout:6.16 2024-09-10T18:54:18.020 INFO:teuthology.orchestra.run.smithi018.stdout:6.18 2024-09-10T18:54:18.100 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:18.120 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T18:54:18.128 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:18.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:18.129 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:18.230 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:18.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:18.262 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:18.263 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:18.363 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:18.399 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:18.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:18.400 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:18.501 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:18.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:18.531 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:18.531 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:18.632 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:18.663 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:18.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:18.664 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:18.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:18 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:18.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:18 smithi018 ceph-mon[29336]: pgmap v394: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 41 active+undersized, 1 active+remapped+backfill_toofull, 27 active+undersized+degraded, 98 active+clean; 4.7 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.52k op/s; 25963/233685 objects degraded (11.110%); 9797/233685 objects misplaced (4.192%); 1.8 MiB/s, 28 objects/s recovering 2024-09-10T18:54:18.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:18 smithi018 ceph-mon[29336]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:54:18.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:18 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1560429938' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:54:18.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:18 smithi018 ceph-mon[29336]: osdmap e211: 8 total, 7 up, 7 in 2024-09-10T18:54:18.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:18 smithi018 ceph-mon[29336]: 6.4 scrub starts 2024-09-10T18:54:18.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:18 smithi018 ceph-mon[29336]: 6.4 scrub ok 2024-09-10T18:54:18.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:18 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:18.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:18 smithi018 ceph-mon[40196]: pgmap v394: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 41 active+undersized, 1 active+remapped+backfill_toofull, 27 active+undersized+degraded, 98 active+clean; 4.7 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.52k op/s; 25963/233685 objects degraded (11.110%); 9797/233685 objects misplaced (4.192%); 1.8 MiB/s, 28 objects/s recovering 2024-09-10T18:54:18.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:18 smithi018 ceph-mon[40196]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:54:18.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:18 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1560429938' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:54:18.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:18 smithi018 ceph-mon[40196]: osdmap e211: 8 total, 7 up, 7 in 2024-09-10T18:54:18.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:18 smithi018 ceph-mon[40196]: 6.4 scrub starts 2024-09-10T18:54:18.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:18 smithi018 ceph-mon[40196]: 6.4 scrub ok 2024-09-10T18:54:18.765 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:18.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:18 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:18.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:18 smithi103 ceph-mon[33163]: pgmap v394: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 41 active+undersized, 1 active+remapped+backfill_toofull, 27 active+undersized+degraded, 98 active+clean; 4.7 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.52k op/s; 25963/233685 objects degraded (11.110%); 9797/233685 objects misplaced (4.192%); 1.8 MiB/s, 28 objects/s recovering 2024-09-10T18:54:18.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:18 smithi103 ceph-mon[33163]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:54:18.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:18 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1560429938' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:54:18.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:18 smithi103 ceph-mon[33163]: osdmap e211: 8 total, 7 up, 7 in 2024-09-10T18:54:18.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:18 smithi103 ceph-mon[33163]: 6.4 scrub starts 2024-09-10T18:54:18.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:18 smithi103 ceph-mon[33163]: 6.4 scrub ok 2024-09-10T18:54:18.788 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:18.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:18.789 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:18.890 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:18.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:18.930 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:18.931 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:19.031 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:19.065 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:19.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:19.066 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:19.167 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:19.197 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:19.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:19.199 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:19.299 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:19.329 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:19.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:19.330 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:19.431 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:19.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:19.471 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:19.471 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:19.572 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:19.610 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:19.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:19.611 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:19.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:19 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:54:19.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:19 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:54:19.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:19 smithi018 ceph-mon[40196]: Health check update: Degraded data redundancy: 25963/233685 objects degraded (11.110%), 28 pgs degraded (PG_DEGRADED) 2024-09-10T18:54:19.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:19 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:54:19.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:19 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:54:19.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:19 smithi018 ceph-mon[29336]: Health check update: Degraded data redundancy: 25963/233685 objects degraded (11.110%), 28 pgs degraded (PG_DEGRADED) 2024-09-10T18:54:19.711 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:19.743 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:19.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:19.745 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:19.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:19 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:54:19.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:19 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:54:19.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:19 smithi103 ceph-mon[33163]: Health check update: Degraded data redundancy: 25963/233685 objects degraded (11.110%), 28 pgs degraded (PG_DEGRADED) 2024-09-10T18:54:19.846 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:19.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:19.890 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:19.891 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:19.991 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:20.021 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:20.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:20.022 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:20.123 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:20.151 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:20.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:20.152 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:20.253 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:20.292 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:20.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:20.293 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:20.393 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:20.423 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:20.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:20.424 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:20.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:20 smithi103 ceph-mon[33163]: pgmap v396: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 41 active+undersized, 1 active+remapped+backfill_toofull, 27 active+undersized+degraded, 98 active+clean; 4.7 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.36k op/s; 25963/233685 objects degraded (11.110%); 9797/233685 objects misplaced (4.192%); 1.6 MiB/s, 25 objects/s recovering 2024-09-10T18:54:20.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:20 smithi103 ceph-mon[33163]: 3.2 scrub starts 2024-09-10T18:54:20.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:20 smithi103 ceph-mon[33163]: 3.2 scrub ok 2024-09-10T18:54:20.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:20 smithi103 ceph-mon[33163]: osdmap e212: 8 total, 7 up, 7 in 2024-09-10T18:54:20.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:20 smithi103 ceph-mon[33163]: 5.5 scrub starts 2024-09-10T18:54:20.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:20 smithi103 ceph-mon[33163]: 5.5 scrub ok 2024-09-10T18:54:20.523 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:54:20 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:54:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:54:20.525 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:20.567 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:20.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:20.569 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:20.669 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:20.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:20 smithi018 ceph-mon[40196]: pgmap v396: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 41 active+undersized, 1 active+remapped+backfill_toofull, 27 active+undersized+degraded, 98 active+clean; 4.7 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.36k op/s; 25963/233685 objects degraded (11.110%); 9797/233685 objects misplaced (4.192%); 1.6 MiB/s, 25 objects/s recovering 2024-09-10T18:54:20.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:20 smithi018 ceph-mon[40196]: 3.2 scrub starts 2024-09-10T18:54:20.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:20 smithi018 ceph-mon[40196]: 3.2 scrub ok 2024-09-10T18:54:20.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:20 smithi018 ceph-mon[40196]: osdmap e212: 8 total, 7 up, 7 in 2024-09-10T18:54:20.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:20 smithi018 ceph-mon[40196]: 5.5 scrub starts 2024-09-10T18:54:20.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:20 smithi018 ceph-mon[40196]: 5.5 scrub ok 2024-09-10T18:54:20.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:20 smithi018 ceph-mon[29336]: pgmap v396: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 41 active+undersized, 1 active+remapped+backfill_toofull, 27 active+undersized+degraded, 98 active+clean; 4.7 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.36k op/s; 25963/233685 objects degraded (11.110%); 9797/233685 objects misplaced (4.192%); 1.6 MiB/s, 25 objects/s recovering 2024-09-10T18:54:20.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:20 smithi018 ceph-mon[29336]: 3.2 scrub starts 2024-09-10T18:54:20.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:20 smithi018 ceph-mon[29336]: 3.2 scrub ok 2024-09-10T18:54:20.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:20 smithi018 ceph-mon[29336]: osdmap e212: 8 total, 7 up, 7 in 2024-09-10T18:54:20.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:20 smithi018 ceph-mon[29336]: 5.5 scrub starts 2024-09-10T18:54:20.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:20 smithi018 ceph-mon[29336]: 5.5 scrub ok 2024-09-10T18:54:20.700 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:20.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:20.701 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:20.802 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:20.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:20.841 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:20.841 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:20.942 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:20.976 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:20.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:20.977 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:21.078 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:21.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:21.121 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:21.122 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:21.222 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:21.250 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:21.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:21.251 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:21.352 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:21.378 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:21.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:21.379 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:21.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:54:20 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:54:20] "GET /metrics HTTP/1.1" 200 224269 "" "Prometheus/2.43.0" 2024-09-10T18:54:21.480 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:21.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:21.518 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:21.518 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:21.619 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:21.651 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:21.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:21.652 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:21.753 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:21.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:21 smithi103 ceph-mon[33163]: 6.8 scrub starts 2024-09-10T18:54:21.788 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:21 smithi103 ceph-mon[33163]: 6.8 scrub ok 2024-09-10T18:54:21.788 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:21 smithi103 ceph-mon[33163]: 8.4 scrub starts 2024-09-10T18:54:21.788 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:21 smithi103 ceph-mon[33163]: 6.1e scrub starts 2024-09-10T18:54:21.788 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:21 smithi103 ceph-mon[33163]: 6.1e scrub ok 2024-09-10T18:54:21.789 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:21.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:21.791 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:21.891 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:21.921 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:21.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:21.922 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:21.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:21 smithi018 ceph-mon[40196]: 6.8 scrub starts 2024-09-10T18:54:21.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:21 smithi018 ceph-mon[40196]: 6.8 scrub ok 2024-09-10T18:54:21.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:21 smithi018 ceph-mon[40196]: 8.4 scrub starts 2024-09-10T18:54:21.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:21 smithi018 ceph-mon[40196]: 6.1e scrub starts 2024-09-10T18:54:21.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:21 smithi018 ceph-mon[40196]: 6.1e scrub ok 2024-09-10T18:54:21.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:21 smithi018 ceph-mon[29336]: 6.8 scrub starts 2024-09-10T18:54:21.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:21 smithi018 ceph-mon[29336]: 6.8 scrub ok 2024-09-10T18:54:21.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:21 smithi018 ceph-mon[29336]: 8.4 scrub starts 2024-09-10T18:54:21.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:21 smithi018 ceph-mon[29336]: 6.1e scrub starts 2024-09-10T18:54:21.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:21 smithi018 ceph-mon[29336]: 6.1e scrub ok 2024-09-10T18:54:22.023 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:22.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:22.058 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:22.059 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:22.159 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:22.189 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:22.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:22.190 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:22.290 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:22.334 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:22.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:22.335 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:22.436 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:22.477 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:22.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:22.478 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:22.579 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:22.619 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:22.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:22.620 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:22.721 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:22.755 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:22.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:22.756 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:22.758 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:22 smithi018 ceph-mon[40196]: pgmap v398: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 41 active+undersized, 1 active+remapped+backfill_toofull, 27 active+undersized+degraded, 98 active+clean; 4.7 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 1.02k op/s; 25066/230994 objects degraded (10.851%); 9797/230994 objects misplaced (4.241%) 2024-09-10T18:54:22.758 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:22 smithi018 ceph-mon[40196]: 4.1d scrub starts 2024-09-10T18:54:22.758 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:22 smithi018 ceph-mon[40196]: 4.1d scrub ok 2024-09-10T18:54:22.758 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:22 smithi018 ceph-mon[40196]: 8.3 scrub starts 2024-09-10T18:54:22.758 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:22 smithi018 ceph-mon[40196]: 6.1a scrub starts 2024-09-10T18:54:22.758 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:22 smithi018 ceph-mon[40196]: 6.1a scrub ok 2024-09-10T18:54:22.758 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:22 smithi018 ceph-mon[29336]: pgmap v398: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 41 active+undersized, 1 active+remapped+backfill_toofull, 27 active+undersized+degraded, 98 active+clean; 4.7 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 1.02k op/s; 25066/230994 objects degraded (10.851%); 9797/230994 objects misplaced (4.241%) 2024-09-10T18:54:22.758 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:22 smithi018 ceph-mon[29336]: 4.1d scrub starts 2024-09-10T18:54:22.758 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:22 smithi018 ceph-mon[29336]: 4.1d scrub ok 2024-09-10T18:54:22.758 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:22 smithi018 ceph-mon[29336]: 8.3 scrub starts 2024-09-10T18:54:22.758 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:22 smithi018 ceph-mon[29336]: 6.1a scrub starts 2024-09-10T18:54:22.759 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:22 smithi018 ceph-mon[29336]: 6.1a scrub ok 2024-09-10T18:54:22.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:22 smithi103 ceph-mon[33163]: pgmap v398: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 41 active+undersized, 1 active+remapped+backfill_toofull, 27 active+undersized+degraded, 98 active+clean; 4.7 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 1.02k op/s; 25066/230994 objects degraded (10.851%); 9797/230994 objects misplaced (4.241%) 2024-09-10T18:54:22.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:22 smithi103 ceph-mon[33163]: 4.1d scrub starts 2024-09-10T18:54:22.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:22 smithi103 ceph-mon[33163]: 4.1d scrub ok 2024-09-10T18:54:22.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:22 smithi103 ceph-mon[33163]: 8.3 scrub starts 2024-09-10T18:54:22.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:22 smithi103 ceph-mon[33163]: 6.1a scrub starts 2024-09-10T18:54:22.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:22 smithi103 ceph-mon[33163]: 6.1a scrub ok 2024-09-10T18:54:22.857 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:22.898 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:22.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:22.899 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:23.000 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:23.030 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:23.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:23.032 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:23.132 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:23.135 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -n osd.0 --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op export --pgid 1.0 --file /var/log/ceph/exp.1.0.0 2024-09-10T18:54:23.171 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:23.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:23.172 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:23.273 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:23.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:23.310 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:23.310 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:23.411 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:23.443 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:23.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:23.444 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:23.545 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:23.581 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:23.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:23.582 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:23.617 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/osd.0/config 2024-09-10T18:54:23.683 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:23.718 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:23.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:23.719 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:23.737 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:23 smithi018 ceph-mon[29336]: 8.f scrub starts 2024-09-10T18:54:23.737 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:23 smithi018 ceph-mon[29336]: 4.8 scrub starts 2024-09-10T18:54:23.737 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:23 smithi018 ceph-mon[29336]: 4.8 scrub ok 2024-09-10T18:54:23.737 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:23 smithi018 ceph-mon[29336]: 8.4 scrub ok 2024-09-10T18:54:23.737 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:23 smithi018 ceph-mon[40196]: 8.f scrub starts 2024-09-10T18:54:23.737 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:23 smithi018 ceph-mon[40196]: 4.8 scrub starts 2024-09-10T18:54:23.737 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:23 smithi018 ceph-mon[40196]: 4.8 scrub ok 2024-09-10T18:54:23.738 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:23 smithi018 ceph-mon[40196]: 8.4 scrub ok 2024-09-10T18:54:23.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:23 smithi103 ceph-mon[33163]: 8.f scrub starts 2024-09-10T18:54:23.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:23 smithi103 ceph-mon[33163]: 4.8 scrub starts 2024-09-10T18:54:23.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:23 smithi103 ceph-mon[33163]: 4.8 scrub ok 2024-09-10T18:54:23.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:23 smithi103 ceph-mon[33163]: 8.4 scrub ok 2024-09-10T18:54:23.820 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:23.848 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:23.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:23.849 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:23.950 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:23.977 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:23.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:23.979 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:24.079 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:24.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:24.109 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:24.109 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:24.210 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:24.254 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:24.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:24.255 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:24.356 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:24.388 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:24.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:24.389 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:24.488 INFO:teuthology.orchestra.run.smithi018.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-10T18:54:24.490 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:24.518 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:24.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:24.519 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:24.620 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:24.647 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:24.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:24.649 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:24.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:24 smithi018 ceph-mon[40196]: 8.3 scrub ok 2024-09-10T18:54:24.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:24 smithi018 ceph-mon[40196]: pgmap v399: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 1 active+clean+scrubbing, 41 active+undersized, 1 active+remapped+backfill_toofull, 27 active+undersized+degraded, 97 active+clean; 4.3 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.81k op/s; 23548/211896 objects degraded (11.113%); 8905/211896 objects misplaced (4.203%) 2024-09-10T18:54:24.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:24 smithi018 ceph-mon[40196]: 8.f scrub ok 2024-09-10T18:54:24.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:24 smithi018 ceph-mon[40196]: Health check update: Degraded data redundancy: 23548/211896 objects degraded (11.113%), 28 pgs degraded (PG_DEGRADED) 2024-09-10T18:54:24.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:24 smithi018 ceph-mon[40196]: 4.5 scrub starts 2024-09-10T18:54:24.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:24 smithi018 ceph-mon[40196]: 4.5 scrub ok 2024-09-10T18:54:24.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:24 smithi018 ceph-mon[40196]: 8.8 scrub starts 2024-09-10T18:54:24.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:24 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:54:24.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:24 smithi018 ceph-mon[29336]: 8.3 scrub ok 2024-09-10T18:54:24.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:24 smithi018 ceph-mon[29336]: pgmap v399: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 1 active+clean+scrubbing, 41 active+undersized, 1 active+remapped+backfill_toofull, 27 active+undersized+degraded, 97 active+clean; 4.3 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.81k op/s; 23548/211896 objects degraded (11.113%); 8905/211896 objects misplaced (4.203%) 2024-09-10T18:54:24.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:24 smithi018 ceph-mon[29336]: 8.f scrub ok 2024-09-10T18:54:24.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:24 smithi018 ceph-mon[29336]: Health check update: Degraded data redundancy: 23548/211896 objects degraded (11.113%), 28 pgs degraded (PG_DEGRADED) 2024-09-10T18:54:24.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:24 smithi018 ceph-mon[29336]: 4.5 scrub starts 2024-09-10T18:54:24.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:24 smithi018 ceph-mon[29336]: 4.5 scrub ok 2024-09-10T18:54:24.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:24 smithi018 ceph-mon[29336]: 8.8 scrub starts 2024-09-10T18:54:24.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:24 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:54:24.750 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:24.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:24 smithi103 ceph-mon[33163]: 8.3 scrub ok 2024-09-10T18:54:24.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:24 smithi103 ceph-mon[33163]: pgmap v399: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 1 active+clean+scrubbing, 41 active+undersized, 1 active+remapped+backfill_toofull, 27 active+undersized+degraded, 97 active+clean; 4.3 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.81k op/s; 23548/211896 objects degraded (11.113%); 8905/211896 objects misplaced (4.203%) 2024-09-10T18:54:24.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:24 smithi103 ceph-mon[33163]: 8.f scrub ok 2024-09-10T18:54:24.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:24 smithi103 ceph-mon[33163]: Health check update: Degraded data redundancy: 23548/211896 objects degraded (11.113%), 28 pgs degraded (PG_DEGRADED) 2024-09-10T18:54:24.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:24 smithi103 ceph-mon[33163]: 4.5 scrub starts 2024-09-10T18:54:24.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:24 smithi103 ceph-mon[33163]: 4.5 scrub ok 2024-09-10T18:54:24.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:24 smithi103 ceph-mon[33163]: 8.8 scrub starts 2024-09-10T18:54:24.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:24 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:54:24.787 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:24.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:24.788 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:24.889 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:24.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:24.927 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:24.927 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:25.028 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:25.054 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:25.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:25.056 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:25.157 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:25.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:25.197 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:25.198 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:25.298 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:25.340 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:25.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:25.341 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:25.441 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:25.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:25.472 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:25.473 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:25.573 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:25.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:25.614 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:25.614 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:25.616 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-10T18:54:25.715 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:25.751 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:25.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:25.752 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:25.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:25 smithi103 ceph-mon[33163]: 6.1d scrub starts 2024-09-10T18:54:25.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:25 smithi103 ceph-mon[33163]: 6.1d scrub ok 2024-09-10T18:54:25.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:25 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:25.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:25 smithi103 ceph-mon[33163]: 6.2 scrub starts 2024-09-10T18:54:25.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:25 smithi103 ceph-mon[33163]: 6.2 scrub ok 2024-09-10T18:54:25.854 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:25.880 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:25.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:25.881 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:25.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:25 smithi018 ceph-mon[40196]: 6.1d scrub starts 2024-09-10T18:54:25.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:25 smithi018 ceph-mon[40196]: 6.1d scrub ok 2024-09-10T18:54:25.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:25 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:25.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:25 smithi018 ceph-mon[40196]: 6.2 scrub starts 2024-09-10T18:54:25.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:25 smithi018 ceph-mon[40196]: 6.2 scrub ok 2024-09-10T18:54:25.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:25 smithi018 ceph-mon[29336]: 6.1d scrub starts 2024-09-10T18:54:25.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:25 smithi018 ceph-mon[29336]: 6.1d scrub ok 2024-09-10T18:54:25.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:25 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:25.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:25 smithi018 ceph-mon[29336]: 6.2 scrub starts 2024-09-10T18:54:25.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:25 smithi018 ceph-mon[29336]: 6.2 scrub ok 2024-09-10T18:54:25.982 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:26.011 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:26.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:26.012 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:26.113 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:26.150 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:26.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:26.151 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:26.253 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:26.288 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:26.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:26.290 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:26.390 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:26.418 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:26.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:26.419 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:26.520 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:26.552 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:26.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:26.553 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:26.654 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:26.676 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:26.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:26.677 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:26.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:26 smithi103 ceph-mon[33163]: pgmap v400: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 1 active+clean+scrubbing, 41 active+undersized, 1 active+remapped+backfill_toofull, 27 active+undersized+degraded, 97 active+clean; 4.3 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 908 op/s; 23548/211896 objects degraded (11.113%); 8905/211896 objects misplaced (4.203%) 2024-09-10T18:54:26.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:26 smithi103 ceph-mon[33163]: 8.8 scrub ok 2024-09-10T18:54:26.773 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:54:26 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:54:26.777 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:26.821 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:26 smithi018 ceph-mon[29336]: pgmap v400: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 1 active+clean+scrubbing, 41 active+undersized, 1 active+remapped+backfill_toofull, 27 active+undersized+degraded, 97 active+clean; 4.3 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 908 op/s; 23548/211896 objects degraded (11.113%); 8905/211896 objects misplaced (4.203%) 2024-09-10T18:54:26.821 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:26 smithi018 ceph-mon[29336]: 8.8 scrub ok 2024-09-10T18:54:26.821 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:26 smithi018 ceph-mon[40196]: pgmap v400: 168 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 1 active+clean+scrubbing, 41 active+undersized, 1 active+remapped+backfill_toofull, 27 active+undersized+degraded, 97 active+clean; 4.3 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 908 op/s; 23548/211896 objects degraded (11.113%); 8905/211896 objects misplaced (4.203%) 2024-09-10T18:54:26.821 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:26 smithi018 ceph-mon[40196]: 8.8 scrub ok 2024-09-10T18:54:26.823 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:26.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:26.824 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:26.925 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:26.956 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:26.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:26.957 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:27.058 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:27.083 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:27.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:27.084 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:27.184 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:27.227 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:27.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:27.228 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:27.329 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:27.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:27.366 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:27.366 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:27.467 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:27.495 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:27.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:27.496 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:27.597 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:27.635 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:27.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:27.636 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:27.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:27 smithi018 ceph-mon[40196]: 8.d scrub starts 2024-09-10T18:54:27.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:27 smithi018 ceph-mon[29336]: 8.d scrub starts 2024-09-10T18:54:27.737 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:27.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:27 smithi103 ceph-mon[33163]: 8.d scrub starts 2024-09-10T18:54:27.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:27.783 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:27.783 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:27.884 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:27.911 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:27.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:27.912 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:28.013 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:28.047 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:28.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:28.048 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:28.149 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:28.177 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:28.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:28.178 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:28.279 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:28.316 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:28.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:28.317 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:28.418 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:28.446 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:28.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:28.447 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:28.548 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:28.581 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:28.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:28.583 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:28.605 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-10T18:54:28.605 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-10T18:54:28.612 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:54:28.684 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:28.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:28.724 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:28.724 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:28.734 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:28 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:28.734 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:28 smithi018 ceph-mon[40196]: pgmap v401: 168 pgs: 1 active+remapped+backfilling, 1 active+undersized+degraded+remapped+backfill_toofull, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 3.9 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.46k op/s; 21162/191058 objects degraded (11.076%); 7954/191058 objects misplaced (4.163%); 689 KiB/s, 10 objects/s recovering 2024-09-10T18:54:28.734 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:28 smithi018 ceph-mon[40196]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-10T18:54:28.734 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:28 smithi018 ceph-mon[40196]: 6.1c scrub starts 2024-09-10T18:54:28.734 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:28 smithi018 ceph-mon[40196]: 6.1c scrub ok 2024-09-10T18:54:28.734 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:28 smithi018 ceph-mon[40196]: 5.11 scrub starts 2024-09-10T18:54:28.734 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:28 smithi018 ceph-mon[40196]: 5.11 scrub ok 2024-09-10T18:54:28.735 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:28 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:28.735 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:28 smithi018 ceph-mon[29336]: pgmap v401: 168 pgs: 1 active+remapped+backfilling, 1 active+undersized+degraded+remapped+backfill_toofull, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 3.9 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.46k op/s; 21162/191058 objects degraded (11.076%); 7954/191058 objects misplaced (4.163%); 689 KiB/s, 10 objects/s recovering 2024-09-10T18:54:28.735 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:28 smithi018 ceph-mon[29336]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-10T18:54:28.735 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:28 smithi018 ceph-mon[29336]: 6.1c scrub starts 2024-09-10T18:54:28.735 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:28 smithi018 ceph-mon[29336]: 6.1c scrub ok 2024-09-10T18:54:28.735 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:28 smithi018 ceph-mon[29336]: 5.11 scrub starts 2024-09-10T18:54:28.735 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:28 smithi018 ceph-mon[29336]: 5.11 scrub ok 2024-09-10T18:54:28.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:28 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:28.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:28 smithi103 ceph-mon[33163]: pgmap v401: 168 pgs: 1 active+remapped+backfilling, 1 active+undersized+degraded+remapped+backfill_toofull, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 3.9 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.46k op/s; 21162/191058 objects degraded (11.076%); 7954/191058 objects misplaced (4.163%); 689 KiB/s, 10 objects/s recovering 2024-09-10T18:54:28.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:28 smithi103 ceph-mon[33163]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-10T18:54:28.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:28 smithi103 ceph-mon[33163]: 6.1c scrub starts 2024-09-10T18:54:28.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:28 smithi103 ceph-mon[33163]: 6.1c scrub ok 2024-09-10T18:54:28.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:28 smithi103 ceph-mon[33163]: 5.11 scrub starts 2024-09-10T18:54:28.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:28 smithi103 ceph-mon[33163]: 5.11 scrub ok 2024-09-10T18:54:28.825 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:28.864 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:28.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:28.865 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:28.965 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:29.003 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:29.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:29.004 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:29.105 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:29.141 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:29.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:29.142 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:29.243 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:29.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:29.282 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:29.282 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:29.382 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:29.419 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:29.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:29.420 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:29.520 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:29.556 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:29.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:29.557 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:29.620 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:29 smithi018 ceph-mon[40196]: 8.d scrub ok 2024-09-10T18:54:29.620 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:29 smithi018 ceph-mon[40196]: Health check update: Degraded data redundancy: 21162/191058 objects degraded (11.076%), 28 pgs degraded (PG_DEGRADED) 2024-09-10T18:54:29.621 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:29 smithi018 ceph-mon[40196]: 4.1f scrub starts 2024-09-10T18:54:29.621 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:29 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:54:29.621 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:29 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:54:29.621 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:29 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:54:29.621 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:29 smithi018 ceph-mon[29336]: 8.d scrub ok 2024-09-10T18:54:29.621 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:29 smithi018 ceph-mon[29336]: Health check update: Degraded data redundancy: 21162/191058 objects degraded (11.076%), 28 pgs degraded (PG_DEGRADED) 2024-09-10T18:54:29.621 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:29 smithi018 ceph-mon[29336]: 4.1f scrub starts 2024-09-10T18:54:29.621 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:29 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:54:29.621 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:29 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:54:29.621 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:29 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:54:29.658 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:29.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:29.701 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:29.702 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:29.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:29 smithi103 ceph-mon[33163]: 8.d scrub ok 2024-09-10T18:54:29.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:29 smithi103 ceph-mon[33163]: Health check update: Degraded data redundancy: 21162/191058 objects degraded (11.076%), 28 pgs degraded (PG_DEGRADED) 2024-09-10T18:54:29.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:29 smithi103 ceph-mon[33163]: 4.1f scrub starts 2024-09-10T18:54:29.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:29 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:54:29.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:29 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:54:29.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:29 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:54:29.802 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:29.831 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:29.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:29.832 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:29.933 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:29.968 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:29.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:29.969 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:30.070 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:30.103 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:30.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:30.104 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:30.205 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:30.245 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:30.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:30.246 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:30.347 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:30.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:30.387 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:30.387 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:30.488 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:30.519 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:30.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:30.520 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:30.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:30 smithi103 ceph-mon[33163]: pgmap v402: 168 pgs: 1 active+remapped+backfilling, 1 active+undersized+degraded+remapped+backfill_toofull, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 3.9 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.42k op/s; 21162/191058 objects degraded (11.076%); 7954/191058 objects misplaced (4.163%); 672 KiB/s, 10 objects/s recovering 2024-09-10T18:54:30.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:30 smithi103 ceph-mon[33163]: 5.19 scrub starts 2024-09-10T18:54:30.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:30 smithi103 ceph-mon[33163]: 5.19 scrub ok 2024-09-10T18:54:30.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:30 smithi103 ceph-mon[33163]: 2.16 scrub starts 2024-09-10T18:54:30.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:54:30 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:54:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:54:30.551 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:30 smithi018 ceph-mon[40196]: pgmap v402: 168 pgs: 1 active+remapped+backfilling, 1 active+undersized+degraded+remapped+backfill_toofull, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 3.9 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.42k op/s; 21162/191058 objects degraded (11.076%); 7954/191058 objects misplaced (4.163%); 672 KiB/s, 10 objects/s recovering 2024-09-10T18:54:30.552 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:30 smithi018 ceph-mon[40196]: 5.19 scrub starts 2024-09-10T18:54:30.552 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:30 smithi018 ceph-mon[40196]: 5.19 scrub ok 2024-09-10T18:54:30.552 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:30 smithi018 ceph-mon[40196]: 2.16 scrub starts 2024-09-10T18:54:30.552 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:30 smithi018 ceph-mon[29336]: pgmap v402: 168 pgs: 1 active+remapped+backfilling, 1 active+undersized+degraded+remapped+backfill_toofull, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 3.9 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.42k op/s; 21162/191058 objects degraded (11.076%); 7954/191058 objects misplaced (4.163%); 672 KiB/s, 10 objects/s recovering 2024-09-10T18:54:30.552 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:30 smithi018 ceph-mon[29336]: 5.19 scrub starts 2024-09-10T18:54:30.552 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:30 smithi018 ceph-mon[29336]: 5.19 scrub ok 2024-09-10T18:54:30.552 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:30 smithi018 ceph-mon[29336]: 2.16 scrub starts 2024-09-10T18:54:30.621 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:30.659 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:30.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:30.659 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:30.760 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:30.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:30.798 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:30.798 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:30.899 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:30.930 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:30.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:30.932 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:31.032 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:31.068 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:31.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:31.070 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:31.170 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:31.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:54:30 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:54:30] "GET /metrics HTTP/1.1" 200 224241 "" "Prometheus/2.43.0" 2024-09-10T18:54:31.201 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:31.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:31.202 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:31.303 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:31.334 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:31.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:31.335 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:31.350 INFO:teuthology.orchestra.run.smithi018.stderr:Exporting 1.0 info 1.0( v 70'81 (0'0,70'81] local-lis/les=167/168 n=0 ec=19/19 lis/c=167/167 les/c/f=168/168/0 sis=167) 2024-09-10T18:54:31.352 INFO:teuthology.orchestra.run.smithi018.stderr:Export successful 2024-09-10T18:54:31.357 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:54:31.375 INFO:teuthology.orchestra.run.smithi018.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-10T18:54:31.408 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:54:31.409 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:54:31.417 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:54:31.436 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:31.450 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:54:31.450 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:54:31.483 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:31.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:31.484 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:31.485 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:54:31.485 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:54:31.519 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:54:31.520 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:54:31.556 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:54:31.556 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:54:31.585 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:31.591 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:54:31.592 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:54:31.616 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:31.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:31.617 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:31.625 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:54:31.625 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:54:31.718 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:31.756 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:31.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:31.757 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:31.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:31 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:54:31.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:31 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:54:31.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:31 smithi103 ceph-mon[33163]: 4.17 scrub starts 2024-09-10T18:54:31.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:31 smithi103 ceph-mon[33163]: 4.a scrub starts 2024-09-10T18:54:31.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:31 smithi103 ceph-mon[33163]: osdmap e213: 8 total, 7 up, 7 in 2024-09-10T18:54:31.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:31 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/836774714' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:54:31.858 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:31.872 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:31 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:54:31.872 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:31 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:54:31.872 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:31 smithi018 ceph-mon[40196]: 4.17 scrub starts 2024-09-10T18:54:31.872 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:31 smithi018 ceph-mon[40196]: 4.a scrub starts 2024-09-10T18:54:31.873 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:31 smithi018 ceph-mon[40196]: osdmap e213: 8 total, 7 up, 7 in 2024-09-10T18:54:31.873 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:31 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/836774714' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:54:31.876 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:31 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:54:31.876 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:31 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:54:31.876 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:31 smithi018 ceph-mon[29336]: 4.17 scrub starts 2024-09-10T18:54:31.876 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:31 smithi018 ceph-mon[29336]: 4.a scrub starts 2024-09-10T18:54:31.876 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:31 smithi018 ceph-mon[29336]: osdmap e213: 8 total, 7 up, 7 in 2024-09-10T18:54:31.876 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:31 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/836774714' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:54:31.911 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:31.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:31.913 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:32.013 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:32.050 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:32.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:32.051 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:32.151 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:32.187 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:32.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:32.188 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:32.289 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:32.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:32.328 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:32.329 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:32.429 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:32.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:32.464 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:32.464 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:32.565 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:32.597 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:32.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:32.600 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:32.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:32 smithi018 ceph-mon[40196]: pgmap v404: 168 pgs: 1 active+remapped+backfilling, 1 active+undersized+degraded+remapped+backfill_toofull, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 3.8 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.41k op/s; 20367/188673 objects degraded (10.795%); 7954/188673 objects misplaced (4.216%); 672 KiB/s, 10 objects/s recovering 2024-09-10T18:54:32.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:32 smithi018 ceph-mon[40196]: 4.15 scrub starts 2024-09-10T18:54:32.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:32 smithi018 ceph-mon[29336]: pgmap v404: 168 pgs: 1 active+remapped+backfilling, 1 active+undersized+degraded+remapped+backfill_toofull, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 3.8 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.41k op/s; 20367/188673 objects degraded (10.795%); 7954/188673 objects misplaced (4.216%); 672 KiB/s, 10 objects/s recovering 2024-09-10T18:54:32.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:32 smithi018 ceph-mon[29336]: 4.15 scrub starts 2024-09-10T18:54:32.701 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:32.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:32.737 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:32.738 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:32.760 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-10T18:54:32.761 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-10T18:54:32.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:32 smithi103 ceph-mon[33163]: pgmap v404: 168 pgs: 1 active+remapped+backfilling, 1 active+undersized+degraded+remapped+backfill_toofull, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 3.8 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.41k op/s; 20367/188673 objects degraded (10.795%); 7954/188673 objects misplaced (4.216%); 672 KiB/s, 10 objects/s recovering 2024-09-10T18:54:32.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:32 smithi103 ceph-mon[33163]: 4.15 scrub starts 2024-09-10T18:54:32.838 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:32.874 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:32.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:32.875 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:32.976 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:33.012 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:33.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:33.014 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:33.114 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:33.153 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:33.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:33.154 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:33.254 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:33.296 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:33.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:33.297 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:33.398 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:33.429 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:33.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:33.430 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:33.531 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:33.563 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:33.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:33.564 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:33.665 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:33.694 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:33.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:33.695 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:33.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:33 smithi103 ceph-mon[33163]: 5.1a scrub starts 2024-09-10T18:54:33.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:33 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:54:33.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:33 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:54:33.796 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:33.832 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:33.833 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:33 smithi018 ceph-mon[40196]: 5.1a scrub starts 2024-09-10T18:54:33.833 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:33 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:54:33.833 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:33 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:54:33.834 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:33 smithi018 ceph-mon[29336]: 5.1a scrub starts 2024-09-10T18:54:33.834 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:33 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:54:33.834 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:33 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:54:33.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:33.834 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:33.935 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:33.967 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:33.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:33.969 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:34.070 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:34.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:34.110 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:34.110 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:34.211 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:34.240 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:34.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:34.241 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:34.342 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:34.373 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:34.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:34.375 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:34.475 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:34.505 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:34.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:34.506 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:34.607 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:34.644 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:34.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:34.645 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:34.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:34 smithi018 ceph-mon[29336]: pgmap v405: 168 pgs: 1 active+remapped+backfilling, 1 active+undersized+degraded+remapped+backfill_toofull, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 3.4 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.40k op/s; 18870/170028 objects degraded (11.098%); 6451/170028 objects misplaced (3.794%); 4.5 MiB/s, 72 objects/s recovering 2024-09-10T18:54:34.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:34 smithi018 ceph-mon[29336]: osdmap e214: 8 total, 7 up, 7 in 2024-09-10T18:54:34.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:34 smithi018 ceph-mon[29336]: Health check update: Degraded data redundancy: 18870/170028 objects degraded (11.098%), 28 pgs degraded (PG_DEGRADED) 2024-09-10T18:54:34.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:34 smithi018 ceph-mon[29336]: 6.12 scrub starts 2024-09-10T18:54:34.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:34 smithi018 ceph-mon[29336]: 5.f scrub starts 2024-09-10T18:54:34.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:34 smithi018 ceph-mon[40196]: pgmap v405: 168 pgs: 1 active+remapped+backfilling, 1 active+undersized+degraded+remapped+backfill_toofull, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 3.4 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.40k op/s; 18870/170028 objects degraded (11.098%); 6451/170028 objects misplaced (3.794%); 4.5 MiB/s, 72 objects/s recovering 2024-09-10T18:54:34.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:34 smithi018 ceph-mon[40196]: osdmap e214: 8 total, 7 up, 7 in 2024-09-10T18:54:34.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:34 smithi018 ceph-mon[40196]: Health check update: Degraded data redundancy: 18870/170028 objects degraded (11.098%), 28 pgs degraded (PG_DEGRADED) 2024-09-10T18:54:34.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:34 smithi018 ceph-mon[40196]: 6.12 scrub starts 2024-09-10T18:54:34.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:34 smithi018 ceph-mon[40196]: 5.f scrub starts 2024-09-10T18:54:34.745 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:34.761 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T18:54:34.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:34 smithi103 ceph-mon[33163]: pgmap v405: 168 pgs: 1 active+remapped+backfilling, 1 active+undersized+degraded+remapped+backfill_toofull, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 3.4 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.40k op/s; 18870/170028 objects degraded (11.098%); 6451/170028 objects misplaced (3.794%); 4.5 MiB/s, 72 objects/s recovering 2024-09-10T18:54:34.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:34 smithi103 ceph-mon[33163]: osdmap e214: 8 total, 7 up, 7 in 2024-09-10T18:54:34.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:34 smithi103 ceph-mon[33163]: Health check update: Degraded data redundancy: 18870/170028 objects degraded (11.098%), 28 pgs degraded (PG_DEGRADED) 2024-09-10T18:54:34.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:34 smithi103 ceph-mon[33163]: 6.12 scrub starts 2024-09-10T18:54:34.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:34 smithi103 ceph-mon[33163]: 5.f scrub starts 2024-09-10T18:54:34.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:34.782 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:34.783 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:34.883 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:34.922 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:34.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:34.923 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:35.024 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:35.063 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:35.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:35.064 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:35.165 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:35.205 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:35.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:35.207 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:35.297 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -n osd.0 --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --force --op remove --pgid 1.0 2024-09-10T18:54:35.308 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:35.346 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:35.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:35.347 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:35.448 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:35.478 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:35.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:35.479 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:35.540 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:35 smithi018 ceph-mon[40196]: 2.2 scrub starts 2024-09-10T18:54:35.540 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:35 smithi018 ceph-mon[40196]: pgmap v407: 168 pgs: 1 active+remapped+backfilling, 1 active+undersized+degraded+remapped+backfill_toofull, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 3.4 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 876 op/s; 18870/170028 objects degraded (11.098%); 6451/170028 objects misplaced (3.794%); 4.9 MiB/s, 77 objects/s recovering 2024-09-10T18:54:35.540 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:35 smithi018 ceph-mon[29336]: 2.2 scrub starts 2024-09-10T18:54:35.540 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:35 smithi018 ceph-mon[29336]: pgmap v407: 168 pgs: 1 active+remapped+backfilling, 1 active+undersized+degraded+remapped+backfill_toofull, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 3.4 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 876 op/s; 18870/170028 objects degraded (11.098%); 6451/170028 objects misplaced (3.794%); 4.9 MiB/s, 77 objects/s recovering 2024-09-10T18:54:35.580 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:35.616 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:35.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:35.618 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:35.718 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:35.750 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:35.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:35.751 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:35.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:35 smithi103 ceph-mon[33163]: 2.2 scrub starts 2024-09-10T18:54:35.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:35 smithi103 ceph-mon[33163]: pgmap v407: 168 pgs: 1 active+remapped+backfilling, 1 active+undersized+degraded+remapped+backfill_toofull, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 3.4 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 876 op/s; 18870/170028 objects degraded (11.098%); 6451/170028 objects misplaced (3.794%); 4.9 MiB/s, 77 objects/s recovering 2024-09-10T18:54:35.852 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:35.890 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:35.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:35.891 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:35.992 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:36.032 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:36.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:36.033 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:36.074 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T18:54:36.074 INFO:teuthology.orchestra.run.smithi018.stdout: "ops": [], 2024-09-10T18:54:36.075 INFO:teuthology.orchestra.run.smithi018.stdout: "num_ops": 0 2024-09-10T18:54:36.075 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T18:54:36.134 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:36.168 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:36.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:36.169 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:36.270 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:36.291 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:36.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:36.292 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:36.392 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:36.426 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:36.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:36.427 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:36.439 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/osd.0/config 2024-09-10T18:54:36.529 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:36.565 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:36.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:36.566 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:36.667 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:36.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:36 smithi018 ceph-mon[40196]: 2.b scrub starts 2024-09-10T18:54:36.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:36 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:54:36.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:36 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:54:36.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:36 smithi018 ceph-mon[29336]: 2.b scrub starts 2024-09-10T18:54:36.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:36 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:54:36.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:36 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:54:36.702 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:36.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:36.703 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:36.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:36 smithi103 ceph-mon[33163]: 2.b scrub starts 2024-09-10T18:54:36.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:36 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:54:36.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:36 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:54:36.773 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:54:36 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:54:36.804 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:36.846 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:36.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:36.847 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:36.948 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:36.984 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:36.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:36.986 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:36.996 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-10T18:54:37.028 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:54:37.086 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:37.118 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:37.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:37.119 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:37.220 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:37.269 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:37.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:37.270 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:37.370 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:37.399 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:37.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:37.400 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:37.501 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:37.521 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is unset 2024-09-10T18:54:37.535 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:37.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:37.536 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:37.637 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:37.674 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:37.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:37.675 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:37.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:37 smithi018 ceph-mon[40196]: osdmap e215: 8 total, 7 up, 7 in 2024-09-10T18:54:37.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:37 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:37.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:37 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2942335040' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:54:37.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:37 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:54:37.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:37 smithi018 ceph-mon[40196]: 1.a scrub starts 2024-09-10T18:54:37.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:37 smithi018 ceph-mon[40196]: pgmap v409: 168 pgs: 1 active+remapped+backfilling, 1 active+undersized+degraded+remapped+backfilling, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 3.0 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 2.22k op/s; 15861/148815 objects degraded (10.658%); 5052/148815 objects misplaced (3.395%); 20 MiB/s, 321 objects/s recovering 2024-09-10T18:54:37.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:37 smithi018 ceph-mon[29336]: osdmap e215: 8 total, 7 up, 7 in 2024-09-10T18:54:37.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:37 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:37.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:37 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2942335040' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:54:37.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:37 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:54:37.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:37 smithi018 ceph-mon[29336]: 1.a scrub starts 2024-09-10T18:54:37.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:37 smithi018 ceph-mon[29336]: pgmap v409: 168 pgs: 1 active+remapped+backfilling, 1 active+undersized+degraded+remapped+backfilling, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 3.0 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 2.22k op/s; 15861/148815 objects degraded (10.658%); 5052/148815 objects misplaced (3.395%); 20 MiB/s, 321 objects/s recovering 2024-09-10T18:54:37.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:37 smithi103 ceph-mon[33163]: osdmap e215: 8 total, 7 up, 7 in 2024-09-10T18:54:37.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:37 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:37.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:37 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2942335040' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:54:37.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:37 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:54:37.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:37 smithi103 ceph-mon[33163]: 1.a scrub starts 2024-09-10T18:54:37.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:37 smithi103 ceph-mon[33163]: pgmap v409: 168 pgs: 1 active+remapped+backfilling, 1 active+undersized+degraded+remapped+backfilling, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 3.0 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 2.22k op/s; 15861/148815 objects degraded (10.658%); 5052/148815 objects misplaced (3.395%); 20 MiB/s, 321 objects/s recovering 2024-09-10T18:54:37.776 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:37.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:37.818 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:37.818 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:37.919 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:37.957 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:37.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:37.959 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:38.059 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:38.098 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:38.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:38.099 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:38.200 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:38.229 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:38.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:38.230 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:38.331 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:38.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:38.369 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:38.369 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:38.470 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:38.505 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:38.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:38.506 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:38.607 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:38.634 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:38.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:38.635 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:38.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:38 smithi018 ceph-mon[40196]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-10T18:54:38.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:38 smithi018 ceph-mon[40196]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:54:38.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:38 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:54:38.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:38 smithi018 ceph-mon[40196]: osdmap e216: 8 total, 7 up, 7 in 2024-09-10T18:54:38.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:38 smithi018 ceph-mon[29336]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-10T18:54:38.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:38 smithi018 ceph-mon[29336]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:54:38.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:38 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:54:38.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:38 smithi018 ceph-mon[29336]: osdmap e216: 8 total, 7 up, 7 in 2024-09-10T18:54:38.736 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:38.768 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:38.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:38.769 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:38.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:38 smithi103 ceph-mon[33163]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-10T18:54:38.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:38 smithi103 ceph-mon[33163]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:54:38.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:38 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:54:38.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:38 smithi103 ceph-mon[33163]: osdmap e216: 8 total, 7 up, 7 in 2024-09-10T18:54:38.870 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:38.905 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:38.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:38.906 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:39.007 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:39.038 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:39.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:39.039 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:39.140 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:39.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:39.175 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:39.175 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:39.276 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:39.304 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:39.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:39.305 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:39.406 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:39.434 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:39.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:39.436 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:39.536 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:39.576 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:39.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:39.577 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:39.678 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:39.709 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:39.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:39.710 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:39.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:39 smithi103 ceph-mon[33163]: Health check update: Degraded data redundancy: 15861/148815 objects degraded (10.658%), 28 pgs degraded (PG_DEGRADED) 2024-09-10T18:54:39.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:39 smithi103 ceph-mon[33163]: osdmap e217: 8 total, 7 up, 7 in 2024-09-10T18:54:39.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:39 smithi103 ceph-mon[33163]: 5.6 deep-scrub starts 2024-09-10T18:54:39.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:39 smithi103 ceph-mon[33163]: pgmap v412: 168 pgs: 1 active+remapped+backfilling, 1 active+undersized+degraded+remapped+backfilling, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 3.0 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 896 B/s rd, 0 B/s wr, 1.24k op/s; 15861/148815 objects degraded (10.658%); 5052/148815 objects misplaced (3.395%); 15 MiB/s, 247 objects/s recovering 2024-09-10T18:54:39.811 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:39.824 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:39 smithi018 ceph-mon[40196]: Health check update: Degraded data redundancy: 15861/148815 objects degraded (10.658%), 28 pgs degraded (PG_DEGRADED) 2024-09-10T18:54:39.824 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:39 smithi018 ceph-mon[40196]: osdmap e217: 8 total, 7 up, 7 in 2024-09-10T18:54:39.824 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:39 smithi018 ceph-mon[40196]: 5.6 deep-scrub starts 2024-09-10T18:54:39.824 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:39 smithi018 ceph-mon[40196]: pgmap v412: 168 pgs: 1 active+remapped+backfilling, 1 active+undersized+degraded+remapped+backfilling, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 3.0 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 896 B/s rd, 0 B/s wr, 1.24k op/s; 15861/148815 objects degraded (10.658%); 5052/148815 objects misplaced (3.395%); 15 MiB/s, 247 objects/s recovering 2024-09-10T18:54:39.825 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:39 smithi018 ceph-mon[29336]: Health check update: Degraded data redundancy: 15861/148815 objects degraded (10.658%), 28 pgs degraded (PG_DEGRADED) 2024-09-10T18:54:39.825 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:39 smithi018 ceph-mon[29336]: osdmap e217: 8 total, 7 up, 7 in 2024-09-10T18:54:39.825 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:39 smithi018 ceph-mon[29336]: 5.6 deep-scrub starts 2024-09-10T18:54:39.825 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:39 smithi018 ceph-mon[29336]: pgmap v412: 168 pgs: 1 active+remapped+backfilling, 1 active+undersized+degraded+remapped+backfilling, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 3.0 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 896 B/s rd, 0 B/s wr, 1.24k op/s; 15861/148815 objects degraded (10.658%); 5052/148815 objects misplaced (3.395%); 15 MiB/s, 247 objects/s recovering 2024-09-10T18:54:39.838 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:39.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:39.839 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:39.939 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:39.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:39.962 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:39.963 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:40.064 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:40.085 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:40.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:40.086 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:40.187 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:40.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:40.210 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:40.210 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:40.310 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:40.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:40.333 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:40.334 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:40.434 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:40.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:40.461 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:40.462 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:40.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:54:40 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:54:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:54:40.563 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:40.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:40.588 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:40.588 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:40.689 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:40.717 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:40.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:40.718 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:40.818 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:40.845 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:40.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:40.846 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:40.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:40 smithi018 ceph-mon[40196]: 4.11 scrub starts 2024-09-10T18:54:40.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:40 smithi018 ceph-mon[40196]: 4.11 scrub ok 2024-09-10T18:54:40.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:40 smithi018 ceph-mon[40196]: osdmap e218: 8 total, 7 up, 7 in 2024-09-10T18:54:40.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:40 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:54:40.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:40 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:40.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:40 smithi018 ceph-mon[29336]: 4.11 scrub starts 2024-09-10T18:54:40.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:40 smithi018 ceph-mon[29336]: 4.11 scrub ok 2024-09-10T18:54:40.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:40 smithi018 ceph-mon[29336]: osdmap e218: 8 total, 7 up, 7 in 2024-09-10T18:54:40.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:40 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:54:40.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:40 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:40.947 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:40.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:40.983 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:40.983 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:41.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:40 smithi103 ceph-mon[33163]: 4.11 scrub starts 2024-09-10T18:54:41.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:40 smithi103 ceph-mon[33163]: 4.11 scrub ok 2024-09-10T18:54:41.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:40 smithi103 ceph-mon[33163]: osdmap e218: 8 total, 7 up, 7 in 2024-09-10T18:54:41.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:40 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:54:41.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:40 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:41.084 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:41.122 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:41.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:41.124 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:41.224 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:41.264 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:41.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:41.265 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:41.366 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:41.400 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:41.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:41.401 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:41.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:54:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:54:40] "GET /metrics HTTP/1.1" 200 224299 "" "Prometheus/2.43.0" 2024-09-10T18:54:41.502 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:41.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:41.532 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:41.532 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:41.634 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:41.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:41.672 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:41.673 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:41.774 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:41.802 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:41.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:41.804 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:41.904 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:41.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:41.941 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:41.942 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:41.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:41 smithi018 ceph-mon[40196]: 1.f scrub starts 2024-09-10T18:54:41.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:41 smithi018 ceph-mon[40196]: osdmap e219: 8 total, 7 up, 7 in 2024-09-10T18:54:41.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:41 smithi018 ceph-mon[40196]: 2.1c scrub starts 2024-09-10T18:54:41.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:41 smithi018 ceph-mon[40196]: 2.1c scrub ok 2024-09-10T18:54:41.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:41 smithi018 ceph-mon[40196]: pgmap v415: 168 pgs: 1 active+remapped+backfilling, 1 active+undersized+degraded+remapped+backfilling, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 3.0 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 0 B/s wr, 177 op/s; 15151/146685 objects degraded (10.329%); 5052/146685 objects misplaced (3.444%) 2024-09-10T18:54:41.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:41 smithi018 ceph-mon[29336]: 1.f scrub starts 2024-09-10T18:54:41.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:41 smithi018 ceph-mon[29336]: osdmap e219: 8 total, 7 up, 7 in 2024-09-10T18:54:41.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:41 smithi018 ceph-mon[29336]: 2.1c scrub starts 2024-09-10T18:54:41.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:41 smithi018 ceph-mon[29336]: 2.1c scrub ok 2024-09-10T18:54:41.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:41 smithi018 ceph-mon[29336]: pgmap v415: 168 pgs: 1 active+remapped+backfilling, 1 active+undersized+degraded+remapped+backfilling, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 3.0 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 0 B/s wr, 177 op/s; 15151/146685 objects degraded (10.329%); 5052/146685 objects misplaced (3.444%) 2024-09-10T18:54:42.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:41 smithi103 ceph-mon[33163]: 1.f scrub starts 2024-09-10T18:54:42.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:41 smithi103 ceph-mon[33163]: osdmap e219: 8 total, 7 up, 7 in 2024-09-10T18:54:42.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:41 smithi103 ceph-mon[33163]: 2.1c scrub starts 2024-09-10T18:54:42.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:41 smithi103 ceph-mon[33163]: 2.1c scrub ok 2024-09-10T18:54:42.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:41 smithi103 ceph-mon[33163]: pgmap v415: 168 pgs: 1 active+remapped+backfilling, 1 active+undersized+degraded+remapped+backfilling, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 3.0 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 0 B/s wr, 177 op/s; 15151/146685 objects degraded (10.329%); 5052/146685 objects misplaced (3.444%) 2024-09-10T18:54:42.037 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:54:42.043 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:42.082 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:42.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:42.083 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:42.183 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:42.227 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:42.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:42.228 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:42.328 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:42.364 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:42.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:42.365 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:42.466 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:42.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:42.504 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:42.505 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:42.605 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:42.645 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:42.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:42.646 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:42.747 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:42.775 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:42.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:42.776 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:42.845 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:42 smithi018 ceph-mon[40196]: 8.9 scrub starts 2024-09-10T18:54:42.845 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:42 smithi018 ceph-mon[40196]: 1.14 deep-scrub starts 2024-09-10T18:54:42.845 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:42 smithi018 ceph-mon[29336]: 8.9 scrub starts 2024-09-10T18:54:42.845 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:42 smithi018 ceph-mon[29336]: 1.14 deep-scrub starts 2024-09-10T18:54:42.878 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:42.905 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:42.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:42.906 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:43.006 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:43.018 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-10T18:54:43.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:42 smithi103 ceph-mon[33163]: 8.9 scrub starts 2024-09-10T18:54:43.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:42 smithi103 ceph-mon[33163]: 1.14 deep-scrub starts 2024-09-10T18:54:43.041 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:43.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:43.043 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:43.143 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:43.176 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:43.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:43.177 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:43.274 INFO:teuthology.orchestra.run.smithi018.stdout: marking collection for removal 2024-09-10T18:54:43.274 INFO:teuthology.orchestra.run.smithi018.stdout:setting '_remove' omap key 2024-09-10T18:54:43.274 INFO:teuthology.orchestra.run.smithi018.stdout:finish_remove_pgs 1.0_head removing 1.0 2024-09-10T18:54:43.275 INFO:teuthology.orchestra.run.smithi018.stdout:Remove successful 2024-09-10T18:54:43.277 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:43.314 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:43.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:43.316 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:43.417 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:43.452 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:43.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:43.453 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:43.554 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:43.589 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:43.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:43.590 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:43.690 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:43.718 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:43.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:43.719 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:43.820 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:43.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:43.853 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:43.853 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:43.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:43 smithi018 ceph-mon[40196]: 2.11 scrub starts 2024-09-10T18:54:43.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:43 smithi018 ceph-mon[40196]: 8.9 scrub ok 2024-09-10T18:54:43.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:43 smithi018 ceph-mon[40196]: 1.4 scrub starts 2024-09-10T18:54:43.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:43 smithi018 ceph-mon[40196]: 1.4 scrub ok 2024-09-10T18:54:43.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:43 smithi018 ceph-mon[40196]: pgmap v416: 168 pgs: 1 active+remapped+backfilling, 1 active+clean+remapped+scrubbing, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 2.7 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 900 B/s rd, 0 B/s wr, 1.02k op/s; 13633/131355 objects degraded (10.379%); 3195/131355 objects misplaced (2.432%); 19 MiB/s, 305 objects/s recovering 2024-09-10T18:54:43.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:43 smithi018 ceph-mon[40196]: Health check update: Degraded data redundancy: 15151/146685 objects degraded (10.329%), 28 pgs degraded (PG_DEGRADED) 2024-09-10T18:54:43.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:43 smithi018 ceph-mon[40196]: osdmap e220: 8 total, 7 up, 7 in 2024-09-10T18:54:43.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:43 smithi018 ceph-mon[29336]: 2.11 scrub starts 2024-09-10T18:54:43.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:43 smithi018 ceph-mon[29336]: 8.9 scrub ok 2024-09-10T18:54:43.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:43 smithi018 ceph-mon[29336]: 1.4 scrub starts 2024-09-10T18:54:43.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:43 smithi018 ceph-mon[29336]: 1.4 scrub ok 2024-09-10T18:54:43.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:43 smithi018 ceph-mon[29336]: pgmap v416: 168 pgs: 1 active+remapped+backfilling, 1 active+clean+remapped+scrubbing, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 2.7 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 900 B/s rd, 0 B/s wr, 1.02k op/s; 13633/131355 objects degraded (10.379%); 3195/131355 objects misplaced (2.432%); 19 MiB/s, 305 objects/s recovering 2024-09-10T18:54:43.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:43 smithi018 ceph-mon[29336]: Health check update: Degraded data redundancy: 15151/146685 objects degraded (10.329%), 28 pgs degraded (PG_DEGRADED) 2024-09-10T18:54:43.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:43 smithi018 ceph-mon[29336]: osdmap e220: 8 total, 7 up, 7 in 2024-09-10T18:54:43.954 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:43.985 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:43.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:43.986 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:44.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:43 smithi103 ceph-mon[33163]: 2.11 scrub starts 2024-09-10T18:54:44.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:43 smithi103 ceph-mon[33163]: 8.9 scrub ok 2024-09-10T18:54:44.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:43 smithi103 ceph-mon[33163]: 1.4 scrub starts 2024-09-10T18:54:44.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:43 smithi103 ceph-mon[33163]: 1.4 scrub ok 2024-09-10T18:54:44.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:43 smithi103 ceph-mon[33163]: pgmap v416: 168 pgs: 1 active+remapped+backfilling, 1 active+clean+remapped+scrubbing, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 2.7 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 900 B/s rd, 0 B/s wr, 1.02k op/s; 13633/131355 objects degraded (10.379%); 3195/131355 objects misplaced (2.432%); 19 MiB/s, 305 objects/s recovering 2024-09-10T18:54:44.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:43 smithi103 ceph-mon[33163]: Health check update: Degraded data redundancy: 15151/146685 objects degraded (10.329%), 28 pgs degraded (PG_DEGRADED) 2024-09-10T18:54:44.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:43 smithi103 ceph-mon[33163]: osdmap e220: 8 total, 7 up, 7 in 2024-09-10T18:54:44.086 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:44.120 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:44.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:44.121 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:44.222 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:44.250 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:44.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:44.251 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:44.352 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:44.387 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:44.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:44.388 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:44.489 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:44.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:44.533 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:44.533 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:44.634 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:44.666 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:44.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:44.667 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:44.753 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:44 smithi018 ceph-mon[40196]: 2.1a deep-scrub starts 2024-09-10T18:54:44.753 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:44 smithi018 ceph-mon[40196]: 1.8 scrub starts 2024-09-10T18:54:44.753 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:44 smithi018 ceph-mon[40196]: 5.9 scrub starts 2024-09-10T18:54:44.753 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:44 smithi018 ceph-mon[40196]: 2.3 scrub starts 2024-09-10T18:54:44.753 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:44 smithi018 ceph-mon[40196]: 2.3 scrub ok 2024-09-10T18:54:44.753 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:44 smithi018 ceph-mon[29336]: 2.1a deep-scrub starts 2024-09-10T18:54:44.753 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:44 smithi018 ceph-mon[29336]: 1.8 scrub starts 2024-09-10T18:54:44.753 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:44 smithi018 ceph-mon[29336]: 5.9 scrub starts 2024-09-10T18:54:44.753 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:44 smithi018 ceph-mon[29336]: 2.3 scrub starts 2024-09-10T18:54:44.754 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:44 smithi018 ceph-mon[29336]: 2.3 scrub ok 2024-09-10T18:54:44.768 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:44.802 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:44.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:44.803 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:44.904 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:44.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:44 smithi103 ceph-mon[33163]: 2.1a deep-scrub starts 2024-09-10T18:54:44.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:44 smithi103 ceph-mon[33163]: 1.8 scrub starts 2024-09-10T18:54:44.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:44 smithi103 ceph-mon[33163]: 5.9 scrub starts 2024-09-10T18:54:44.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:44 smithi103 ceph-mon[33163]: 2.3 scrub starts 2024-09-10T18:54:44.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:44 smithi103 ceph-mon[33163]: 2.3 scrub ok 2024-09-10T18:54:44.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:44.944 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:44.944 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:45.045 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:45.072 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:45.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:45.073 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:45.174 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:45.202 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:45.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:45.203 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:45.207 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:54:45.208 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:54:45.208 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:54:45.208 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:54:45.208 INFO:teuthology.orchestra.run.smithi018.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-10T18:54:45.208 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:54:45.208 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:54:45.208 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:54:45.208 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:54:45.208 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:54:45.209 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:54:45.209 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:54:45.209 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:54:45.209 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:54:45.209 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:54:45.304 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:45.335 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:45.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:45.336 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:45.437 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:45.477 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:45.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:45.478 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:45.578 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:45.614 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:45.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:45.615 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:45.716 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:45.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:45.739 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:45.739 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:45.840 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:45.862 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:45.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:45.863 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:45.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:45 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:54:45.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:45 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:54:45.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:45 smithi018 ceph-mon[29336]: osdmap e221: 8 total, 7 up, 7 in 2024-09-10T18:54:45.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:45 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/107949459' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:54:45.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:45 smithi018 ceph-mon[29336]: pgmap v419: 168 pgs: 1 active+remapped+backfilling, 1 active+clean+remapped+scrubbing, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 2.7 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 914 B/s rd, 0 B/s wr, 1.04k op/s; 13633/131355 objects degraded (10.379%); 3195/131355 objects misplaced (2.432%); 19 MiB/s, 309 objects/s recovering 2024-09-10T18:54:45.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:45 smithi018 ceph-mon[29336]: 4.1b scrub starts 2024-09-10T18:54:45.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:45 smithi018 ceph-mon[29336]: 4.1b scrub ok 2024-09-10T18:54:45.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:45 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:54:45.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:45 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:54:45.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:45 smithi018 ceph-mon[40196]: osdmap e221: 8 total, 7 up, 7 in 2024-09-10T18:54:45.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:45 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/107949459' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:54:45.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:45 smithi018 ceph-mon[40196]: pgmap v419: 168 pgs: 1 active+remapped+backfilling, 1 active+clean+remapped+scrubbing, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 2.7 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 914 B/s rd, 0 B/s wr, 1.04k op/s; 13633/131355 objects degraded (10.379%); 3195/131355 objects misplaced (2.432%); 19 MiB/s, 309 objects/s recovering 2024-09-10T18:54:45.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:45 smithi018 ceph-mon[40196]: 4.1b scrub starts 2024-09-10T18:54:45.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:45 smithi018 ceph-mon[40196]: 4.1b scrub ok 2024-09-10T18:54:45.964 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:45.987 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:45.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:45.988 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:46.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:45 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:54:46.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:45 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:54:46.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:45 smithi103 ceph-mon[33163]: osdmap e221: 8 total, 7 up, 7 in 2024-09-10T18:54:46.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:45 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/107949459' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:54:46.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:45 smithi103 ceph-mon[33163]: pgmap v419: 168 pgs: 1 active+remapped+backfilling, 1 active+clean+remapped+scrubbing, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 2.7 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 914 B/s rd, 0 B/s wr, 1.04k op/s; 13633/131355 objects degraded (10.379%); 3195/131355 objects misplaced (2.432%); 19 MiB/s, 309 objects/s recovering 2024-09-10T18:54:46.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:45 smithi103 ceph-mon[33163]: 4.1b scrub starts 2024-09-10T18:54:46.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:45 smithi103 ceph-mon[33163]: 4.1b scrub ok 2024-09-10T18:54:46.089 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:46.111 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:46.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:46.112 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:46.213 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:46.237 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:46.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:46.238 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:46.339 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:46.362 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:46.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:46.363 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:46.464 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:46.486 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:46.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:46.487 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:46.528 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-10T18:54:46.528 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-10T18:54:46.588 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:46.610 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:46.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:46.611 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:46.712 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:46.739 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:46.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:46.740 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:46.841 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:46.867 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:46 smithi018 ceph-mon[40196]: osdmap e222: 8 total, 7 up, 7 in 2024-09-10T18:54:46.867 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:46 smithi018 ceph-mon[40196]: 1.9 scrub starts 2024-09-10T18:54:46.867 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:46 smithi018 ceph-mon[40196]: 1.9 scrub ok 2024-09-10T18:54:46.867 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:46 smithi018 ceph-mon[29336]: osdmap e222: 8 total, 7 up, 7 in 2024-09-10T18:54:46.867 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:46 smithi018 ceph-mon[29336]: 1.9 scrub starts 2024-09-10T18:54:46.867 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:46 smithi018 ceph-mon[29336]: 1.9 scrub ok 2024-09-10T18:54:46.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:46.875 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:46.876 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:46.976 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:47.009 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:47.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:47.010 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:47.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:46 smithi103 ceph-mon[33163]: osdmap e222: 8 total, 7 up, 7 in 2024-09-10T18:54:47.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:46 smithi103 ceph-mon[33163]: 1.9 scrub starts 2024-09-10T18:54:47.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:46 smithi103 ceph-mon[33163]: 1.9 scrub ok 2024-09-10T18:54:47.022 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:54:46 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:54:47.111 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:47.146 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:47.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:47.147 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:47.235 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -n osd.0 --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.1.0.0 2024-09-10T18:54:47.241 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:54:47.248 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:47.288 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:47.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:47.289 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:47.390 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:47.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:47.428 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:47.428 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:47.529 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:47.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:47.562 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:47.562 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:47.663 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:47.699 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:47.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:47.700 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:47.801 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:47.815 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:47 smithi018 ceph-mon[40196]: 1.7 scrub starts 2024-09-10T18:54:47.816 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:47 smithi018 ceph-mon[40196]: 1.7 scrub ok 2024-09-10T18:54:47.816 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:47 smithi018 ceph-mon[40196]: 2.6 deep-scrub starts 2024-09-10T18:54:47.816 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:47 smithi018 ceph-mon[40196]: 2.6 deep-scrub ok 2024-09-10T18:54:47.816 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:47 smithi018 ceph-mon[40196]: 2.1b scrub starts 2024-09-10T18:54:47.816 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:47 smithi018 ceph-mon[40196]: 2.1b scrub ok 2024-09-10T18:54:47.816 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:47 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:47.816 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:47 smithi018 ceph-mon[40196]: osdmap e223: 8 total, 7 up, 7 in 2024-09-10T18:54:47.816 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:47 smithi018 ceph-mon[40196]: pgmap v422: 168 pgs: 1 peering, 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 2.3 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.62k op/s; 11586/111960 objects degraded (10.348%); 2313/111960 objects misplaced (2.066%); 6.6 MiB/s, 106 objects/s recovering 2024-09-10T18:54:47.816 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:47 smithi018 ceph-mon[29336]: 1.7 scrub starts 2024-09-10T18:54:47.816 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:47 smithi018 ceph-mon[29336]: 1.7 scrub ok 2024-09-10T18:54:47.817 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:47 smithi018 ceph-mon[29336]: 2.6 deep-scrub starts 2024-09-10T18:54:47.817 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:47 smithi018 ceph-mon[29336]: 2.6 deep-scrub ok 2024-09-10T18:54:47.817 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:47 smithi018 ceph-mon[29336]: 2.1b scrub starts 2024-09-10T18:54:47.817 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:47 smithi018 ceph-mon[29336]: 2.1b scrub ok 2024-09-10T18:54:47.817 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:47 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:47.817 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:47 smithi018 ceph-mon[29336]: osdmap e223: 8 total, 7 up, 7 in 2024-09-10T18:54:47.817 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:47 smithi018 ceph-mon[29336]: pgmap v422: 168 pgs: 1 peering, 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 2.3 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.62k op/s; 11586/111960 objects degraded (10.348%); 2313/111960 objects misplaced (2.066%); 6.6 MiB/s, 106 objects/s recovering 2024-09-10T18:54:47.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:47.834 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:47.834 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:47.851 INFO:teuthology.orchestra.run.smithi018.stdout:ERROR: (22) Invalid argument 2024-09-10T18:54:47.851 INFO:teuthology.orchestra.run.smithi018.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:54:47.935 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:47.976 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:47.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:47.977 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:48.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:47 smithi103 ceph-mon[33163]: 1.7 scrub starts 2024-09-10T18:54:48.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:47 smithi103 ceph-mon[33163]: 1.7 scrub ok 2024-09-10T18:54:48.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:47 smithi103 ceph-mon[33163]: 2.6 deep-scrub starts 2024-09-10T18:54:48.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:47 smithi103 ceph-mon[33163]: 2.6 deep-scrub ok 2024-09-10T18:54:48.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:47 smithi103 ceph-mon[33163]: 2.1b scrub starts 2024-09-10T18:54:48.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:47 smithi103 ceph-mon[33163]: 2.1b scrub ok 2024-09-10T18:54:48.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:47 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:48.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:47 smithi103 ceph-mon[33163]: osdmap e223: 8 total, 7 up, 7 in 2024-09-10T18:54:48.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:47 smithi103 ceph-mon[33163]: pgmap v422: 168 pgs: 1 peering, 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 2.3 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.62k op/s; 11586/111960 objects degraded (10.348%); 2313/111960 objects misplaced (2.066%); 6.6 MiB/s, 106 objects/s recovering 2024-09-10T18:54:48.078 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:48.114 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:48.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:48.115 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:48.216 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:48.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:48.247 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:48.248 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:48.348 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:48.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:48.384 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:48.384 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:48.485 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:48.514 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:48.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:48.515 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:48.529 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T18:54:48.616 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:48.643 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:48.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:48.644 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:48.651 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-10T18:54:48.745 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:48.781 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:48.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:48.782 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:48.841 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:48 smithi018 ceph-mon[29336]: 4.13 deep-scrub starts 2024-09-10T18:54:48.841 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:48 smithi018 ceph-mon[29336]: 4.13 deep-scrub ok 2024-09-10T18:54:48.841 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:48 smithi018 ceph-mon[29336]: 6.15 scrub starts 2024-09-10T18:54:48.841 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:48 smithi018 ceph-mon[29336]: 6.15 scrub ok 2024-09-10T18:54:48.842 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:48 smithi018 ceph-mon[29336]: Health check update: Degraded data redundancy: 11586/111960 objects degraded (10.348%), 27 pgs degraded (PG_DEGRADED) 2024-09-10T18:54:48.842 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:48 smithi018 ceph-mon[29336]: osdmap e224: 8 total, 7 up, 7 in 2024-09-10T18:54:48.842 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:48 smithi018 ceph-mon[40196]: 4.13 deep-scrub starts 2024-09-10T18:54:48.842 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:48 smithi018 ceph-mon[40196]: 4.13 deep-scrub ok 2024-09-10T18:54:48.842 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:48 smithi018 ceph-mon[40196]: 6.15 scrub starts 2024-09-10T18:54:48.842 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:48 smithi018 ceph-mon[40196]: 6.15 scrub ok 2024-09-10T18:54:48.842 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:48 smithi018 ceph-mon[40196]: Health check update: Degraded data redundancy: 11586/111960 objects degraded (10.348%), 27 pgs degraded (PG_DEGRADED) 2024-09-10T18:54:48.842 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:48 smithi018 ceph-mon[40196]: osdmap e224: 8 total, 7 up, 7 in 2024-09-10T18:54:48.884 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:48.924 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:48.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:48.925 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:49.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:48 smithi103 ceph-mon[33163]: 4.13 deep-scrub starts 2024-09-10T18:54:49.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:48 smithi103 ceph-mon[33163]: 4.13 deep-scrub ok 2024-09-10T18:54:49.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:48 smithi103 ceph-mon[33163]: 6.15 scrub starts 2024-09-10T18:54:49.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:48 smithi103 ceph-mon[33163]: 6.15 scrub ok 2024-09-10T18:54:49.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:48 smithi103 ceph-mon[33163]: Health check update: Degraded data redundancy: 11586/111960 objects degraded (10.348%), 27 pgs degraded (PG_DEGRADED) 2024-09-10T18:54:49.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:48 smithi103 ceph-mon[33163]: osdmap e224: 8 total, 7 up, 7 in 2024-09-10T18:54:49.026 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:49.057 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:49.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:49.058 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:49.088 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/osd.0/config 2024-09-10T18:54:49.159 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:49.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:49.193 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:49.194 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:49.294 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:49.332 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:49.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:49.333 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:49.434 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:49.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:49.470 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:49.470 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:49.571 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:49.604 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:49.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:49.605 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:49.706 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:49.734 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:49.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:49.735 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:49.836 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:49.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:49.880 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:49.880 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:49.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:49 smithi018 ceph-mon[29336]: 4.18 scrub starts 2024-09-10T18:54:49.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:49 smithi018 ceph-mon[29336]: 4.18 scrub ok 2024-09-10T18:54:49.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:49 smithi018 ceph-mon[29336]: 6.7 scrub starts 2024-09-10T18:54:49.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:49 smithi018 ceph-mon[29336]: 6.7 scrub ok 2024-09-10T18:54:49.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:49 smithi018 ceph-mon[29336]: pgmap v424: 168 pgs: 1 peering, 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 2.3 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.42k op/s; 11586/111960 objects degraded (10.348%); 2313/111960 objects misplaced (2.066%); 5.8 MiB/s, 93 objects/s recovering 2024-09-10T18:54:49.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:49 smithi018 ceph-mon[29336]: 4.e deep-scrub starts 2024-09-10T18:54:49.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:49 smithi018 ceph-mon[29336]: 4.e deep-scrub ok 2024-09-10T18:54:49.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:49 smithi018 ceph-mon[40196]: 4.18 scrub starts 2024-09-10T18:54:49.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:49 smithi018 ceph-mon[40196]: 4.18 scrub ok 2024-09-10T18:54:49.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:49 smithi018 ceph-mon[40196]: 6.7 scrub starts 2024-09-10T18:54:49.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:49 smithi018 ceph-mon[40196]: 6.7 scrub ok 2024-09-10T18:54:49.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:49 smithi018 ceph-mon[40196]: pgmap v424: 168 pgs: 1 peering, 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 2.3 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.42k op/s; 11586/111960 objects degraded (10.348%); 2313/111960 objects misplaced (2.066%); 5.8 MiB/s, 93 objects/s recovering 2024-09-10T18:54:49.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:49 smithi018 ceph-mon[40196]: 4.e deep-scrub starts 2024-09-10T18:54:49.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:49 smithi018 ceph-mon[40196]: 4.e deep-scrub ok 2024-09-10T18:54:49.981 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:50.013 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:50.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:50.014 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:50.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:49 smithi103 ceph-mon[33163]: 4.18 scrub starts 2024-09-10T18:54:50.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:49 smithi103 ceph-mon[33163]: 4.18 scrub ok 2024-09-10T18:54:50.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:49 smithi103 ceph-mon[33163]: 6.7 scrub starts 2024-09-10T18:54:50.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:49 smithi103 ceph-mon[33163]: 6.7 scrub ok 2024-09-10T18:54:50.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:49 smithi103 ceph-mon[33163]: pgmap v424: 168 pgs: 1 peering, 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 2.3 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.42k op/s; 11586/111960 objects degraded (10.348%); 2313/111960 objects misplaced (2.066%); 5.8 MiB/s, 93 objects/s recovering 2024-09-10T18:54:50.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:49 smithi103 ceph-mon[33163]: 4.e deep-scrub starts 2024-09-10T18:54:50.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:49 smithi103 ceph-mon[33163]: 4.e deep-scrub ok 2024-09-10T18:54:50.115 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:50.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:50.154 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:50.154 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:50.255 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:50.290 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:50.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:50.291 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:50.392 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:50.427 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:50.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:50.428 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:50.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:54:50 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:54:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:54:50.529 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:50.566 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:50.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:50.567 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:50.668 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:50.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:50.704 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:50.704 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:50.805 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:50.835 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:50.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:50.836 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:50.937 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:50.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:50 smithi018 ceph-mon[40196]: 6.3 scrub starts 2024-09-10T18:54:50.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:50 smithi018 ceph-mon[40196]: 6.3 scrub ok 2024-09-10T18:54:50.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:50 smithi018 ceph-mon[29336]: 6.3 scrub starts 2024-09-10T18:54:50.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:50 smithi018 ceph-mon[29336]: 6.3 scrub ok 2024-09-10T18:54:50.966 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:50.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:50.968 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:51.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:50 smithi103 ceph-mon[33163]: 6.3 scrub starts 2024-09-10T18:54:51.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:50 smithi103 ceph-mon[33163]: 6.3 scrub ok 2024-09-10T18:54:51.069 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:51.106 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:51.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:51.107 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:51.208 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:51.243 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:51.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:51.244 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:51.345 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:51.373 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:51.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:51.374 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:51.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:54:50 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:54:50] "GET /metrics HTTP/1.1" 200 224299 "" "Prometheus/2.43.0" 2024-09-10T18:54:51.475 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:51.516 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:51.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:51.517 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:51.617 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:51.647 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:51.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:51.648 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:51.749 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:51.782 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:51.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:51.783 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:51.884 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:51.918 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:51.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:51.919 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:51.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:51 smithi018 ceph-mon[40196]: 8.7 scrub starts 2024-09-10T18:54:51.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:51 smithi018 ceph-mon[40196]: 8.7 scrub ok 2024-09-10T18:54:51.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:51 smithi018 ceph-mon[40196]: 1.12 scrub starts 2024-09-10T18:54:51.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:51 smithi018 ceph-mon[40196]: 1.12 scrub ok 2024-09-10T18:54:51.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:51 smithi018 ceph-mon[40196]: pgmap v425: 168 pgs: 1 peering, 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 2.2 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.21k op/s; 10781/109545 objects degraded (9.842%); 2313/109545 objects misplaced (2.111%); 4.4 MiB/s, 70 objects/s recovering 2024-09-10T18:54:51.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:51 smithi018 ceph-mon[29336]: 8.7 scrub starts 2024-09-10T18:54:51.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:51 smithi018 ceph-mon[29336]: 8.7 scrub ok 2024-09-10T18:54:51.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:51 smithi018 ceph-mon[29336]: 1.12 scrub starts 2024-09-10T18:54:51.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:51 smithi018 ceph-mon[29336]: 1.12 scrub ok 2024-09-10T18:54:51.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:51 smithi018 ceph-mon[29336]: pgmap v425: 168 pgs: 1 peering, 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 2.2 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.21k op/s; 10781/109545 objects degraded (9.842%); 2313/109545 objects misplaced (2.111%); 4.4 MiB/s, 70 objects/s recovering 2024-09-10T18:54:52.021 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:52.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:51 smithi103 ceph-mon[33163]: 8.7 scrub starts 2024-09-10T18:54:52.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:51 smithi103 ceph-mon[33163]: 8.7 scrub ok 2024-09-10T18:54:52.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:51 smithi103 ceph-mon[33163]: 1.12 scrub starts 2024-09-10T18:54:52.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:51 smithi103 ceph-mon[33163]: 1.12 scrub ok 2024-09-10T18:54:52.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:51 smithi103 ceph-mon[33163]: pgmap v425: 168 pgs: 1 peering, 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 98 active+clean; 2.2 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.21k op/s; 10781/109545 objects degraded (9.842%); 2313/109545 objects misplaced (2.111%); 4.4 MiB/s, 70 objects/s recovering 2024-09-10T18:54:52.058 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:52.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:52.059 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:52.160 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:52.195 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:52.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:52.196 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:52.296 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:52.327 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:52.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:52.328 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:52.429 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:52.463 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:52.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:52.464 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:52.565 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:52.618 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:52.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:52.619 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:52.720 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:52.765 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:52.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:52.766 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:52.867 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:52.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:52.906 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:52.906 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:52.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:52 smithi018 ceph-mon[29336]: 6.c deep-scrub starts 2024-09-10T18:54:52.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:52 smithi018 ceph-mon[29336]: 6.c deep-scrub ok 2024-09-10T18:54:52.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:52 smithi018 ceph-mon[29336]: 2.12 scrub starts 2024-09-10T18:54:52.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:52 smithi018 ceph-mon[29336]: 2.12 scrub ok 2024-09-10T18:54:52.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:52 smithi018 ceph-mon[40196]: 6.c deep-scrub starts 2024-09-10T18:54:52.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:52 smithi018 ceph-mon[40196]: 6.c deep-scrub ok 2024-09-10T18:54:52.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:52 smithi018 ceph-mon[40196]: 2.12 scrub starts 2024-09-10T18:54:52.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:52 smithi018 ceph-mon[40196]: 2.12 scrub ok 2024-09-10T18:54:53.007 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:53.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:52 smithi103 ceph-mon[33163]: 6.c deep-scrub starts 2024-09-10T18:54:53.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:52 smithi103 ceph-mon[33163]: 6.c deep-scrub ok 2024-09-10T18:54:53.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:52 smithi103 ceph-mon[33163]: 2.12 scrub starts 2024-09-10T18:54:53.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:52 smithi103 ceph-mon[33163]: 2.12 scrub ok 2024-09-10T18:54:53.039 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:53.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:53.040 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:53.090 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:54:53.091 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:54:53.142 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:53.182 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:53.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:53.183 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:53.284 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:53.321 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:53.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:53.322 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:53.423 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:53.445 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:53.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:53.446 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:53.547 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:53.583 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:53.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:53.584 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:53.604 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is set 2024-09-10T18:54:53.685 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:53.723 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:53.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:53.724 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:53.824 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:53.849 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:53 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:54:53.850 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:53 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:54:53.850 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:53 smithi018 ceph-mon[29336]: 2.17 scrub starts 2024-09-10T18:54:53.850 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:53 smithi018 ceph-mon[29336]: 2.17 scrub ok 2024-09-10T18:54:53.850 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:53 smithi018 ceph-mon[29336]: 2.d scrub starts 2024-09-10T18:54:53.850 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:53 smithi018 ceph-mon[29336]: 2.d scrub ok 2024-09-10T18:54:53.850 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:53 smithi018 ceph-mon[29336]: 8.b scrub starts 2024-09-10T18:54:53.850 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:53 smithi018 ceph-mon[29336]: osdmap e225: 8 total, 7 up, 7 in 2024-09-10T18:54:53.850 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:53 smithi018 ceph-mon[29336]: pgmap v427: 168 pgs: 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 99 active+clean; 1.8 GiB data, 10 GiB used, 616 GiB / 626 GiB avail; 781 B/s rd, 0 B/s wr, 1.07k op/s; 9489/90825 objects degraded (10.448%); 1858/90825 objects misplaced (2.046%) 2024-09-10T18:54:53.850 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:53 smithi018 ceph-mon[29336]: 5.1c scrub starts 2024-09-10T18:54:53.850 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:53 smithi018 ceph-mon[29336]: 5.1c scrub ok 2024-09-10T18:54:53.850 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:53 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2479052350' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:54:53.850 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:53 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:54:53.850 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:53 smithi018 ceph-mon[29336]: Health check update: Degraded data redundancy: 10781/109545 objects degraded (9.842%), 27 pgs degraded (PG_DEGRADED) 2024-09-10T18:54:53.851 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:53 smithi018 ceph-mon[29336]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:54:53.851 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:53 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:54:53.851 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:53 smithi018 ceph-mon[29336]: osdmap e226: 8 total, 7 up, 7 in 2024-09-10T18:54:53.851 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:53 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:54:53.851 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:53 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:54:53.851 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:53 smithi018 ceph-mon[40196]: 2.17 scrub starts 2024-09-10T18:54:53.851 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:53 smithi018 ceph-mon[40196]: 2.17 scrub ok 2024-09-10T18:54:53.851 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:53 smithi018 ceph-mon[40196]: 2.d scrub starts 2024-09-10T18:54:53.851 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:53 smithi018 ceph-mon[40196]: 2.d scrub ok 2024-09-10T18:54:53.851 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:53 smithi018 ceph-mon[40196]: 8.b scrub starts 2024-09-10T18:54:53.851 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:53 smithi018 ceph-mon[40196]: osdmap e225: 8 total, 7 up, 7 in 2024-09-10T18:54:53.852 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:53 smithi018 ceph-mon[40196]: pgmap v427: 168 pgs: 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 99 active+clean; 1.8 GiB data, 10 GiB used, 616 GiB / 626 GiB avail; 781 B/s rd, 0 B/s wr, 1.07k op/s; 9489/90825 objects degraded (10.448%); 1858/90825 objects misplaced (2.046%) 2024-09-10T18:54:53.852 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:53 smithi018 ceph-mon[40196]: 5.1c scrub starts 2024-09-10T18:54:53.852 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:53 smithi018 ceph-mon[40196]: 5.1c scrub ok 2024-09-10T18:54:53.852 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:53 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2479052350' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:54:53.852 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:53 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:54:53.852 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:53 smithi018 ceph-mon[40196]: Health check update: Degraded data redundancy: 10781/109545 objects degraded (9.842%), 27 pgs degraded (PG_DEGRADED) 2024-09-10T18:54:53.852 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:53 smithi018 ceph-mon[40196]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:54:53.852 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:53 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:54:53.852 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:53 smithi018 ceph-mon[40196]: osdmap e226: 8 total, 7 up, 7 in 2024-09-10T18:54:53.856 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:53.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:53.857 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:53.958 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:53.997 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:53.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:53.998 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:54.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:53 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:54:54.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:53 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:54:54.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:53 smithi103 ceph-mon[33163]: 2.17 scrub starts 2024-09-10T18:54:54.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:53 smithi103 ceph-mon[33163]: 2.17 scrub ok 2024-09-10T18:54:54.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:53 smithi103 ceph-mon[33163]: 2.d scrub starts 2024-09-10T18:54:54.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:53 smithi103 ceph-mon[33163]: 2.d scrub ok 2024-09-10T18:54:54.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:53 smithi103 ceph-mon[33163]: 8.b scrub starts 2024-09-10T18:54:54.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:53 smithi103 ceph-mon[33163]: osdmap e225: 8 total, 7 up, 7 in 2024-09-10T18:54:54.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:53 smithi103 ceph-mon[33163]: pgmap v427: 168 pgs: 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 99 active+clean; 1.8 GiB data, 10 GiB used, 616 GiB / 626 GiB avail; 781 B/s rd, 0 B/s wr, 1.07k op/s; 9489/90825 objects degraded (10.448%); 1858/90825 objects misplaced (2.046%) 2024-09-10T18:54:54.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:53 smithi103 ceph-mon[33163]: 5.1c scrub starts 2024-09-10T18:54:54.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:53 smithi103 ceph-mon[33163]: 5.1c scrub ok 2024-09-10T18:54:54.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:53 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2479052350' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:54:54.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:53 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:54:54.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:53 smithi103 ceph-mon[33163]: Health check update: Degraded data redundancy: 10781/109545 objects degraded (9.842%), 27 pgs degraded (PG_DEGRADED) 2024-09-10T18:54:54.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:53 smithi103 ceph-mon[33163]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:54:54.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:53 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:54:54.024 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:53 smithi103 ceph-mon[33163]: osdmap e226: 8 total, 7 up, 7 in 2024-09-10T18:54:54.099 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:54.128 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:54.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:54.129 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:54.230 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:54.266 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:54.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:54.267 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:54.368 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:54.405 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:54.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:54.406 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:54.506 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:54.537 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:54.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:54.538 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:54.639 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:54.670 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:54.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:54.671 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:54.772 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:54.805 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:54.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:54.806 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:54.907 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:54.936 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:54.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:54.937 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:55.013 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:54 smithi018 ceph-mon[29336]: 8.b scrub ok 2024-09-10T18:54:55.013 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:54 smithi018 ceph-mon[29336]: 2.7 scrub starts 2024-09-10T18:54:55.013 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:54 smithi018 ceph-mon[29336]: 2.7 scrub ok 2024-09-10T18:54:55.014 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:54 smithi018 ceph-mon[29336]: 8.a scrub starts 2024-09-10T18:54:55.014 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:54 smithi018 ceph-mon[29336]: 6.b deep-scrub starts 2024-09-10T18:54:55.014 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:54 smithi018 ceph-mon[29336]: 6.b deep-scrub ok 2024-09-10T18:54:55.014 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:54 smithi018 ceph-mon[40196]: 8.b scrub ok 2024-09-10T18:54:55.014 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:54 smithi018 ceph-mon[40196]: 2.7 scrub starts 2024-09-10T18:54:55.014 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:54 smithi018 ceph-mon[40196]: 2.7 scrub ok 2024-09-10T18:54:55.014 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:54 smithi018 ceph-mon[40196]: 8.a scrub starts 2024-09-10T18:54:55.014 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:54 smithi018 ceph-mon[40196]: 6.b deep-scrub starts 2024-09-10T18:54:55.014 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:54 smithi018 ceph-mon[40196]: 6.b deep-scrub ok 2024-09-10T18:54:55.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:54 smithi103 ceph-mon[33163]: 8.b scrub ok 2024-09-10T18:54:55.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:54 smithi103 ceph-mon[33163]: 2.7 scrub starts 2024-09-10T18:54:55.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:54 smithi103 ceph-mon[33163]: 2.7 scrub ok 2024-09-10T18:54:55.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:54 smithi103 ceph-mon[33163]: 8.a scrub starts 2024-09-10T18:54:55.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:54 smithi103 ceph-mon[33163]: 6.b deep-scrub starts 2024-09-10T18:54:55.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:54 smithi103 ceph-mon[33163]: 6.b deep-scrub ok 2024-09-10T18:54:55.039 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:55.068 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:55.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:55.069 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:55.170 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:55.208 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:55.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:55.209 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:55.309 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:55.339 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:55.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:55.340 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:55.441 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:55.474 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:55.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:55.475 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:55.576 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:55.605 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:55.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:55.606 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:55.707 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:55.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:55.737 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:55.737 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:55.838 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:55.866 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:55.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:55.867 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:55.968 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:55.998 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:56.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:56.000 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:56.101 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:56.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:56.136 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:56.136 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:56.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:56 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:54:56.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:56 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:56.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:56 smithi018 ceph-mon[40196]: pgmap v429: 168 pgs: 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 99 active+clean; 1.8 GiB data, 10 GiB used, 616 GiB / 626 GiB avail; 774 B/s rd, 0 B/s wr, 1.07k op/s; 9489/90825 objects degraded (10.448%); 1858/90825 objects misplaced (2.046%) 2024-09-10T18:54:56.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:56 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:54:56.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:56 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:56.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:56 smithi018 ceph-mon[29336]: pgmap v429: 168 pgs: 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 99 active+clean; 1.8 GiB data, 10 GiB used, 616 GiB / 626 GiB avail; 774 B/s rd, 0 B/s wr, 1.07k op/s; 9489/90825 objects degraded (10.448%); 1858/90825 objects misplaced (2.046%) 2024-09-10T18:54:56.237 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:56.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:56.277 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:56.278 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:56.378 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:56.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:56.416 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:56.416 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:56.517 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:56.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:56 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:54:56.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:56 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:56.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:56 smithi103 ceph-mon[33163]: pgmap v429: 168 pgs: 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 99 active+clean; 1.8 GiB data, 10 GiB used, 616 GiB / 626 GiB avail; 774 B/s rd, 0 B/s wr, 1.07k op/s; 9489/90825 objects degraded (10.448%); 1858/90825 objects misplaced (2.046%) 2024-09-10T18:54:56.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:56.553 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:56.554 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:56.655 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:56.689 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:56.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:56.690 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:56.791 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:56.819 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:56.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:56.820 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:54:56.896 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-10T18:54:56.921 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:56.955 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:56.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:56.956 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:57.022 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:54:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:54:57.058 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:57.090 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:57.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:57.091 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:57.192 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:57.226 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:57.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:57.227 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:57.328 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:57.361 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:57.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:57.362 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:57.463 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:57.496 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:57.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:57.497 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:57.598 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:57.637 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:57.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:57.638 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:57.738 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:57.771 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:57.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:57.773 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:57.837 INFO:teuthology.orchestra.run.smithi018.stdout:get_pg_num_history pg_num_history pg_num_history(e205 pg_nums {1={127=11,167=21},2={55=32},3={55=3},4={57=32},5={59=32},6={61=32},7={86=16},8={158=16}} deleted_pools 153,7) 2024-09-10T18:54:57.837 INFO:teuthology.orchestra.run.smithi018.stdout:Importing pgid 1.0 2024-09-10T18:54:57.837 INFO:teuthology.orchestra.run.smithi018.stdout:write_pg epoch 185 info 1.0( v 70'81 (0'0,70'81] local-lis/les=167/168 n=0 ec=19/19 lis/c=167/167 les/c/f=168/168/0 sis=167) 2024-09-10T18:54:57.837 INFO:teuthology.orchestra.run.smithi018.stdout:Import successful 2024-09-10T18:54:57.874 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:57.910 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:57.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:57.913 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:58.013 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:58.064 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:58.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:58.066 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:58.166 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:58.200 INFO:teuthology.orchestra.run.smithi018.stdout:ERROR: (22) Invalid argument 2024-09-10T18:54:58.200 INFO:teuthology.orchestra.run.smithi018.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:54:58.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:58.205 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:58.205 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:58.306 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:58.338 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:58.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:58.339 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:58.407 INFO:teuthology.orchestra.run.smithi018.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-10T18:54:58.430 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:54:58.430 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:54:58.440 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:58.452 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:54:58.452 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:54:58.474 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:54:58.475 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:54:58.475 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:58 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:58.475 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:58 smithi018 ceph-mon[40196]: pgmap v430: 168 pgs: 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 99 active+clean; 1.4 GiB data, 8.9 GiB used, 617 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.83k op/s; 7118/68106 objects degraded (10.451%); 1393/68106 objects misplaced (2.045%) 2024-09-10T18:54:58.475 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:58 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:58.475 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:58 smithi018 ceph-mon[29336]: pgmap v430: 168 pgs: 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 99 active+clean; 1.4 GiB data, 8.9 GiB used, 617 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.83k op/s; 7118/68106 objects degraded (10.451%); 1393/68106 objects misplaced (2.045%) 2024-09-10T18:54:58.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:58.485 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:58.486 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:58.505 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:54:58.505 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:54:58.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:58 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:58.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:58 smithi103 ceph-mon[33163]: pgmap v430: 168 pgs: 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 99 active+clean; 1.4 GiB data, 8.9 GiB used, 617 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.83k op/s; 7118/68106 objects degraded (10.451%); 1393/68106 objects misplaced (2.045%) 2024-09-10T18:54:58.536 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:54:58.537 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:54:58.565 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:54:58.565 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:54:58.587 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:58.590 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:54:58.590 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:54:58.626 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:58.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:58.627 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:58.728 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:54:58.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:58.762 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:58.763 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:54:58.863 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:58.893 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:58.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:58.894 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:58.995 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:54:59.025 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:59.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:59.026 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:54:59.127 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:59.161 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:59.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:59.162 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:59.262 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:54:59.298 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:59.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:59.299 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:54:59.400 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:54:59.427 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:59.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:59.429 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:54:59.527 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:59 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1641043297' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:54:59.527 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:54:59 smithi103 ceph-mon[33163]: Health check update: Degraded data redundancy: 7118/68106 objects degraded (10.451%), 27 pgs degraded (PG_DEGRADED) 2024-09-10T18:54:59.529 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:59.566 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:59.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:59.568 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:59.668 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:54:59.689 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:59 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1641043297' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:54:59.689 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:54:59 smithi018 ceph-mon[29336]: Health check update: Degraded data redundancy: 7118/68106 objects degraded (10.451%), 27 pgs degraded (PG_DEGRADED) 2024-09-10T18:54:59.689 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:59 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1641043297' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:54:59.689 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:54:59 smithi018 ceph-mon[40196]: Health check update: Degraded data redundancy: 7118/68106 objects degraded (10.451%), 27 pgs degraded (PG_DEGRADED) 2024-09-10T18:54:59.705 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:59.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:59.706 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:54:59.726 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-10T18:54:59.726 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-10T18:54:59.807 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:54:59.841 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:54:59.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:59.842 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:54:59.942 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:54:59.971 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:54:59.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:59.972 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:00.073 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:00.116 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:00.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:00.117 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:00.195 DEBUG:teuthology.orchestra.run.smithi018:> sudo rm -f /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/exp.1.0.0 2024-09-10T18:55:00.217 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:00.298 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:00.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:00.299 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:00.400 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:00.428 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:00.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:00.429 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:00.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:00 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:55:00.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:00 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:55:00.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:00 smithi018 ceph-mon[40196]: pgmap v431: 168 pgs: 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 99 active+clean; 1.4 GiB data, 8.9 GiB used, 617 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.73k op/s; 7118/68106 objects degraded (10.451%); 1393/68106 objects misplaced (2.045%) 2024-09-10T18:55:00.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:00 smithi018 ceph-mon[40196]: osdmap e227: 8 total, 7 up, 7 in 2024-09-10T18:55:00.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:00 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:55:00.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:00 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:55:00.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:00 smithi018 ceph-mon[29336]: pgmap v431: 168 pgs: 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 99 active+clean; 1.4 GiB data, 8.9 GiB used, 617 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.73k op/s; 7118/68106 objects degraded (10.451%); 1393/68106 objects misplaced (2.045%) 2024-09-10T18:55:00.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:00 smithi018 ceph-mon[29336]: osdmap e227: 8 total, 7 up, 7 in 2024-09-10T18:55:00.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:00 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:55:00.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:00 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:55:00.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:00 smithi103 ceph-mon[33163]: pgmap v431: 168 pgs: 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 99 active+clean; 1.4 GiB data, 8.9 GiB used, 617 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.73k op/s; 7118/68106 objects degraded (10.451%); 1393/68106 objects misplaced (2.045%) 2024-09-10T18:55:00.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:00 smithi103 ceph-mon[33163]: osdmap e227: 8 total, 7 up, 7 in 2024-09-10T18:55:00.523 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:55:00 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:55:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:55:00.530 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:00.557 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:00.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:00.558 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:00.659 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:00.698 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:00.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:00.699 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:00.800 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:00.814 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-10T18:55:00.829 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:00.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:00.830 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:00.865 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:55:00.931 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:00.964 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:00.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:00.965 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:01.066 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:01.103 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:01.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:01.104 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:01.141 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:55:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:55:00] "GET /metrics HTTP/1.1" 200 224274 "" "Prometheus/2.43.0" 2024-09-10T18:55:01.204 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:01.241 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:01.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:01.242 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:01.343 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:01.376 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:01.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:01.377 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:01.478 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:01.511 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:01.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:01.512 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:01.613 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:01.641 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:01.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:01.642 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:01.728 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T18:55:01.743 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:01.777 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:01.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:01.778 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:01.879 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:01.912 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:01.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:01.913 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:02.014 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:02.052 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:02.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:02.053 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:02.154 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:02.188 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:02.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:02.189 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:02.290 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:02.314 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:55:02.324 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:02.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:02.326 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:02.426 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:02.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:02 smithi018 ceph-mon[29336]: pgmap v433: 168 pgs: 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 99 active+clean; 1.4 GiB data, 8.9 GiB used, 617 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 946 op/s; 7118/68106 objects degraded (10.451%); 1393/68106 objects misplaced (2.045%) 2024-09-10T18:55:02.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:02 smithi018 ceph-mon[40196]: pgmap v433: 168 pgs: 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 99 active+clean; 1.4 GiB data, 8.9 GiB used, 617 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 946 op/s; 7118/68106 objects degraded (10.451%); 1393/68106 objects misplaced (2.045%) 2024-09-10T18:55:02.463 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:02.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:02.464 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:02.565 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:02.597 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:02.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:02.598 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:02.699 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:02.728 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:02.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:02.729 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:02.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:02 smithi103 ceph-mon[33163]: pgmap v433: 168 pgs: 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 99 active+clean; 1.4 GiB data, 8.9 GiB used, 617 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 946 op/s; 7118/68106 objects degraded (10.451%); 1393/68106 objects misplaced (2.045%) 2024-09-10T18:55:02.830 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:02.863 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:02.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:02.864 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:02.965 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:02.995 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:02.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:02.996 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:03.097 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:03.135 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:03.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:03.136 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:03.237 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:03.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:03.277 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:03.277 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:03.378 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:03.410 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:03.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:03.411 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:03.512 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:03.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:03.546 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:03.546 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:03.647 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:03.677 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:03.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:03.678 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:03.779 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:03.817 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:03.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:03.818 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:03.871 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:55:03.919 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:03.952 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:03.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:03.953 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:04.054 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:04.091 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:04.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:04.092 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:04.193 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:04.226 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:04.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:04.227 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:04.328 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:04.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:04.369 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:04.369 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:04.470 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:04.502 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:04.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:04.503 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:04.604 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:04.619 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:04 smithi018 ceph-mon[29336]: pgmap v434: 168 pgs: 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 99 active+clean; 927 MiB data, 7.6 GiB used, 618 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.59k op/s; 4721/45054 objects degraded (10.479%); 947/45054 objects misplaced (2.102%) 2024-09-10T18:55:04.619 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:04 smithi018 ceph-mon[29336]: Health check update: Degraded data redundancy: 4721/45054 objects degraded (10.479%), 27 pgs degraded, 68 pgs undersized (PG_DEGRADED) 2024-09-10T18:55:04.619 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:04 smithi018 ceph-mon[29336]: osdmap e228: 8 total, 7 up, 7 in 2024-09-10T18:55:04.619 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:04 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1389836075' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:55:04.619 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:04 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:55:04.620 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:04 smithi018 ceph-mon[40196]: pgmap v434: 168 pgs: 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 99 active+clean; 927 MiB data, 7.6 GiB used, 618 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.59k op/s; 4721/45054 objects degraded (10.479%); 947/45054 objects misplaced (2.102%) 2024-09-10T18:55:04.620 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:04 smithi018 ceph-mon[40196]: Health check update: Degraded data redundancy: 4721/45054 objects degraded (10.479%), 27 pgs degraded, 68 pgs undersized (PG_DEGRADED) 2024-09-10T18:55:04.620 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:04 smithi018 ceph-mon[40196]: osdmap e228: 8 total, 7 up, 7 in 2024-09-10T18:55:04.620 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:04 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1389836075' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:55:04.620 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:04 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:55:04.620 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is set 2024-09-10T18:55:04.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:04.646 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:04.646 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:04.747 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:04.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:04 smithi103 ceph-mon[33163]: pgmap v434: 168 pgs: 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 99 active+clean; 927 MiB data, 7.6 GiB used, 618 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.59k op/s; 4721/45054 objects degraded (10.479%); 947/45054 objects misplaced (2.102%) 2024-09-10T18:55:04.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:04 smithi103 ceph-mon[33163]: Health check update: Degraded data redundancy: 4721/45054 objects degraded (10.479%), 27 pgs degraded, 68 pgs undersized (PG_DEGRADED) 2024-09-10T18:55:04.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:04 smithi103 ceph-mon[33163]: osdmap e228: 8 total, 7 up, 7 in 2024-09-10T18:55:04.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:04 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1389836075' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:55:04.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:04 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:55:04.779 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:04.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:04.780 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:04.881 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:04.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:04.917 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:04.918 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:05.018 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:05.051 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:05.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:05.052 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:05.153 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:05.181 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:05.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:05.182 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:05.230 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [0] live_osds: [1, 2, 3, 4, 5, 6, 7] 2024-09-10T18:55:05.230 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T18:55:05.230 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-10T18:55:05.230 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd dump -f json-pretty 2024-09-10T18:55:05.283 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:05.320 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:05.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:05.321 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:05.422 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:05.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:05 smithi018 ceph-mon[40196]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:55:05.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:05 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:55:05.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:05 smithi018 ceph-mon[40196]: osdmap e229: 8 total, 7 up, 7 in 2024-09-10T18:55:05.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:05 smithi018 ceph-mon[29336]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:55:05.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:05 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:55:05.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:05 smithi018 ceph-mon[29336]: osdmap e229: 8 total, 7 up, 7 in 2024-09-10T18:55:05.454 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:05.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:05.456 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:05.556 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:05.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:05.589 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:05.589 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:05.690 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:05.724 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:05.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:05.725 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:05.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:05 smithi103 ceph-mon[33163]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:55:05.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:05 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:55:05.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:05 smithi103 ceph-mon[33163]: osdmap e229: 8 total, 7 up, 7 in 2024-09-10T18:55:05.826 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:05.866 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:05.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:05.867 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:05.967 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:06.004 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:06.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:06.005 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:06.105 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:06.143 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:06.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:06.144 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:06.244 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:06.271 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:06.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:06.272 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:06.373 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:06.405 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:06.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:06.406 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:06.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:06 smithi018 ceph-mon[40196]: pgmap v437: 168 pgs: 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 99 active+clean; 927 MiB data, 7.6 GiB used, 618 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.28k op/s; 4721/45054 objects degraded (10.479%); 947/45054 objects misplaced (2.102%) 2024-09-10T18:55:06.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:06 smithi018 ceph-mon[29336]: pgmap v437: 168 pgs: 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 99 active+clean; 927 MiB data, 7.6 GiB used, 618 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.28k op/s; 4721/45054 objects degraded (10.479%); 947/45054 objects misplaced (2.102%) 2024-09-10T18:55:06.507 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:06.543 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:06.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:06.544 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:06.598 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:06 smithi103 ceph-mon[33163]: pgmap v437: 168 pgs: 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 99 active+clean; 927 MiB data, 7.6 GiB used, 618 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.28k op/s; 4721/45054 objects degraded (10.479%); 947/45054 objects misplaced (2.102%) 2024-09-10T18:55:06.645 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:06.686 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:06.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:06.687 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:06.788 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:06.821 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:06.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:06.822 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:06.923 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:06.958 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:06.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:06.959 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:07.022 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:55:06 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:55:07.060 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:07.094 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:07.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:07.095 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:07.195 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:07.228 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:07.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:07.230 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:07.331 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:07.351 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:07.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:07.352 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:07.453 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:07.489 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:07.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:07.490 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:07.532 INFO:teuthology.orchestra.run.smithi018.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-10T18:55:07.554 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:55:07.554 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:55:07.579 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:55:07.579 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:55:07.591 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:07.608 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:55:07.608 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:55:07.626 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:07.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:07.627 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:07.640 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:55:07.640 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:55:07.671 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:55:07.671 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:55:07.702 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:55:07.703 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:55:07.726 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:55:07.726 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:55:07.728 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:07.748 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:07.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:07.749 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:07.756 INFO:teuthology.orchestra.run.smithi018.stdout:ERROR: (22) Invalid argument 2024-09-10T18:55:07.756 INFO:teuthology.orchestra.run.smithi018.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:55:07.850 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:07.888 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:07.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:07.889 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:07.990 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:08.031 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:08.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:08.033 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:08.134 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:08.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:08.172 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:08.172 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:08.273 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:08.300 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:08.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:08.301 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:08.402 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:08.430 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:08.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:08.431 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:08.532 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:08.545 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:08 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:08.545 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:08 smithi018 ceph-mon[29336]: pgmap v438: 168 pgs: 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 99 active+clean; 452 MiB data, 6.3 GiB used, 620 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.93k op/s; 2319/22248 objects degraded (10.423%); 455/22248 objects misplaced (2.045%) 2024-09-10T18:55:08.545 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:08 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/79875956' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:55:08.545 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:08 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:08.546 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:08 smithi018 ceph-mon[40196]: pgmap v438: 168 pgs: 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 99 active+clean; 452 MiB data, 6.3 GiB used, 620 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.93k op/s; 2319/22248 objects degraded (10.423%); 455/22248 objects misplaced (2.045%) 2024-09-10T18:55:08.546 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:08 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/79875956' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:55:08.566 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:08.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:08.567 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:08.576 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-10T18:55:08.577 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-10T18:55:08.668 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:08.700 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:08.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:08.701 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:08.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:08 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:08.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:08 smithi103 ceph-mon[33163]: pgmap v438: 168 pgs: 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 99 active+clean; 452 MiB data, 6.3 GiB used, 620 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.93k op/s; 2319/22248 objects degraded (10.423%); 455/22248 objects misplaced (2.045%) 2024-09-10T18:55:08.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:08 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/79875956' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:55:08.802 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:08.828 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:08.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:08.830 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:08.931 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:08.959 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:08.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:08.961 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:09.061 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:09.106 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:09.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:09.107 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:09.208 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:09.244 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:09.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:09.245 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:09.346 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:09.379 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:09.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:09.380 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:09.481 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:09.512 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:09.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:09.513 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:09.566 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:09 smithi018 ceph-mon[29336]: Health check update: Degraded data redundancy: 2319/22248 objects degraded (10.423%), 27 pgs degraded, 68 pgs undersized (PG_DEGRADED) 2024-09-10T18:55:09.566 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:09 smithi018 ceph-mon[29336]: osdmap e230: 8 total, 7 up, 7 in 2024-09-10T18:55:09.566 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:09 smithi018 ceph-mon[40196]: Health check update: Degraded data redundancy: 2319/22248 objects degraded (10.423%), 27 pgs degraded, 68 pgs undersized (PG_DEGRADED) 2024-09-10T18:55:09.566 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:09 smithi018 ceph-mon[40196]: osdmap e230: 8 total, 7 up, 7 in 2024-09-10T18:55:09.615 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:09.646 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:09.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:09.647 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:09.748 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:09.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:09 smithi103 ceph-mon[33163]: Health check update: Degraded data redundancy: 2319/22248 objects degraded (10.423%), 27 pgs degraded, 68 pgs undersized (PG_DEGRADED) 2024-09-10T18:55:09.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:09 smithi103 ceph-mon[33163]: osdmap e230: 8 total, 7 up, 7 in 2024-09-10T18:55:09.781 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:09.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:09.782 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:09.883 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:09.921 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:09.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:09.922 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:10.023 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:10.052 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:10.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:10.053 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:10.154 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:10.193 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:10.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:10.194 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:10.295 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:10.330 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:10.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:10.331 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:10.341 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-10T18:55:10.355 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:55:10.431 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:10.465 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:10.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:10.466 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:10.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:10 smithi103 ceph-mon[33163]: pgmap v440: 168 pgs: 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 99 active+clean; 452 MiB data, 6.3 GiB used, 620 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 1.27k op/s; 2319/22248 objects degraded (10.423%); 455/22248 objects misplaced (2.045%) 2024-09-10T18:55:10.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:10 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:10.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:55:10 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:55:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:55:10.567 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:10.578 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T18:55:10.597 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:10.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:10.599 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:10.628 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:10 smithi018 ceph-mon[29336]: pgmap v440: 168 pgs: 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 99 active+clean; 452 MiB data, 6.3 GiB used, 620 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 1.27k op/s; 2319/22248 objects degraded (10.423%); 455/22248 objects misplaced (2.045%) 2024-09-10T18:55:10.628 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:10 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:10.628 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:10 smithi018 ceph-mon[40196]: pgmap v440: 168 pgs: 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 99 active+clean; 452 MiB data, 6.3 GiB used, 620 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 1.27k op/s; 2319/22248 objects degraded (10.423%); 455/22248 objects misplaced (2.045%) 2024-09-10T18:55:10.628 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:10 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:10.701 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:10.731 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:10.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:10.732 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:10.833 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:10.868 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:10.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:10.869 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:10.970 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:11.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:11.005 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:11.005 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:11.106 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:11.143 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:11.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:11.144 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:11.156 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:55:11.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:55:10 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:55:10] "GET /metrics HTTP/1.1" 200 224270 "" "Prometheus/2.43.0" 2024-09-10T18:55:11.226 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Removed 146627 objects 2024-09-10T18:55:11.226 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Clean up completed and total clean up time :112.008 2024-09-10T18:55:11.245 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:11.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:11.268 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:11.268 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:11.369 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:11.391 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:11.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:11.393 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:11.494 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:11.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:11.516 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:11.517 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:11.604 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-10T18:55:11.617 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:11.639 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:11.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:11.640 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:11.741 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:11.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:11.763 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:11.763 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:11.864 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:11.887 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:11.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:11.888 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:11.989 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:12.011 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:12.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:12.012 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:12.113 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:12.138 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:12.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:12.139 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:12.240 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:12.262 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:12.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:12.264 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:12.364 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:12.389 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:12.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:12.390 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:12.491 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:12.514 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:12.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:12.516 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:12.517 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:12 smithi018 ceph-mon[40196]: pgmap v441: 168 pgs: 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 99 active+clean; 452 MiB data, 6.3 GiB used, 620 GiB / 626 GiB avail; 672 B/s rd, 0 B/s wr, 999 op/s; 2319/22248 objects degraded (10.423%); 455/22248 objects misplaced (2.045%) 2024-09-10T18:55:12.518 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:12 smithi018 ceph-mon[29336]: pgmap v441: 168 pgs: 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 99 active+clean; 452 MiB data, 6.3 GiB used, 620 GiB / 626 GiB avail; 672 B/s rd, 0 B/s wr, 999 op/s; 2319/22248 objects degraded (10.423%); 455/22248 objects misplaced (2.045%) 2024-09-10T18:55:12.616 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:12.640 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:12.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:12.641 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:12.742 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:12.767 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:12.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:12.768 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:12.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:12 smithi103 ceph-mon[33163]: pgmap v441: 168 pgs: 1 active+clean+remapped, 41 active+undersized, 27 active+undersized+degraded, 99 active+clean; 452 MiB data, 6.3 GiB used, 620 GiB / 626 GiB avail; 672 B/s rd, 0 B/s wr, 999 op/s; 2319/22248 objects degraded (10.423%); 455/22248 objects misplaced (2.045%) 2024-09-10T18:55:12.869 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:12.895 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:12.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:12.896 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:12.997 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:13.024 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:13.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:13.025 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:13.126 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:13.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:13.151 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:13.152 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:13.252 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:13.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:13.280 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:13.280 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:13.381 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:13.408 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:13.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:13.409 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:13.510 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:13.536 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:13.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:13.537 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:13.638 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:13.665 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:13.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:13.666 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:13.767 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:13.794 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:13.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:13.795 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:13.896 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:13.922 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:13.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:13.923 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:14.024 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:14.050 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:14.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:14.051 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:14.152 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:14.176 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:14.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:14.177 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:14.277 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:14.302 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:14.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:14.303 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:14.404 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:14.428 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:14.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:14.430 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:14.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:14 smithi018 ceph-mon[29336]: pgmap v442: 168 pgs: 1 active+clean+remapped, 43 active+undersized, 25 active+undersized+degraded, 99 active+clean; 468 KiB data, 5.1 GiB used, 621 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.72k op/s; 79/591 objects degraded (13.367%) 2024-09-10T18:55:14.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:14 smithi018 ceph-mon[29336]: Health check update: Degraded data redundancy: 79/591 objects degraded (13.367%), 25 pgs degraded, 68 pgs undersized (PG_DEGRADED) 2024-09-10T18:55:14.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:14 smithi018 ceph-mon[40196]: pgmap v442: 168 pgs: 1 active+clean+remapped, 43 active+undersized, 25 active+undersized+degraded, 99 active+clean; 468 KiB data, 5.1 GiB used, 621 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.72k op/s; 79/591 objects degraded (13.367%) 2024-09-10T18:55:14.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:14 smithi018 ceph-mon[40196]: Health check update: Degraded data redundancy: 79/591 objects degraded (13.367%), 25 pgs degraded, 68 pgs undersized (PG_DEGRADED) 2024-09-10T18:55:14.530 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:14.554 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:14.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:14.555 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:14.574 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:14 smithi103 ceph-mon[33163]: pgmap v442: 168 pgs: 1 active+clean+remapped, 43 active+undersized, 25 active+undersized+degraded, 99 active+clean; 468 KiB data, 5.1 GiB used, 621 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.72k op/s; 79/591 objects degraded (13.367%) 2024-09-10T18:55:14.574 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:14 smithi103 ceph-mon[33163]: Health check update: Degraded data redundancy: 79/591 objects degraded (13.367%), 25 pgs degraded, 68 pgs undersized (PG_DEGRADED) 2024-09-10T18:55:14.656 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:14.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:14.682 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:14.682 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:14.783 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:14.807 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:14.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:14.808 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:14.909 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:14.933 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:14.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:14.934 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:15.035 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:15.056 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:15.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:15.058 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:15.161 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:15.182 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:15.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:15.183 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:15.284 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:15.305 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:15.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:15.306 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:15.407 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:15.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:15.433 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:15.433 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:15.534 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:15.558 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:15.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:15.559 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:15.660 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:15.683 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:15.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:15.684 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:15.785 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:15.806 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:15.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:15.807 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:15.908 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:15.930 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:15.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:15.931 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:15.941 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:55:15.941 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T18:55:15.941 INFO:teuthology.orchestra.run.smithi018.stdout: "epoch": 230, 2024-09-10T18:55:15.941 INFO:teuthology.orchestra.run.smithi018.stdout: "fsid": "ba327f6c-6fa3-11ef-bcea-c7b262605968", 2024-09-10T18:55:15.941 INFO:teuthology.orchestra.run.smithi018.stdout: "created": "2024-09-10T18:40:07.669687+0000", 2024-09-10T18:55:15.941 INFO:teuthology.orchestra.run.smithi018.stdout: "modified": "2024-09-10T18:55:08.601033+0000", 2024-09-10T18:55:15.941 INFO:teuthology.orchestra.run.smithi018.stdout: "last_up_change": "2024-09-10T18:54:00.270083+0000", 2024-09-10T18:55:15.942 INFO:teuthology.orchestra.run.smithi018.stdout: "last_in_change": "2024-09-10T18:47:35.385915+0000", 2024-09-10T18:55:15.942 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-10T18:55:15.942 INFO:teuthology.orchestra.run.smithi018.stdout: "flags_num": 5806080, 2024-09-10T18:55:15.942 INFO:teuthology.orchestra.run.smithi018.stdout: "flags_set": [ 2024-09-10T18:55:15.942 INFO:teuthology.orchestra.run.smithi018.stdout: "nodeep-scrub", 2024-09-10T18:55:15.943 INFO:teuthology.orchestra.run.smithi018.stdout: "noscrub", 2024-09-10T18:55:15.943 INFO:teuthology.orchestra.run.smithi018.stdout: "pglog_hardlimit", 2024-09-10T18:55:15.943 INFO:teuthology.orchestra.run.smithi018.stdout: "purged_snapdirs", 2024-09-10T18:55:15.943 INFO:teuthology.orchestra.run.smithi018.stdout: "recovery_deletes", 2024-09-10T18:55:15.943 INFO:teuthology.orchestra.run.smithi018.stdout: "sortbitwise" 2024-09-10T18:55:15.943 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-09-10T18:55:15.943 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_version": 18, 2024-09-10T18:55:15.943 INFO:teuthology.orchestra.run.smithi018.stdout: "full_ratio": 0.94999998807907104, 2024-09-10T18:55:15.944 INFO:teuthology.orchestra.run.smithi018.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-10T18:55:15.944 INFO:teuthology.orchestra.run.smithi018.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-10T18:55:15.944 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_snapshot": "", 2024-09-10T18:55:15.944 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_max": 8, 2024-09-10T18:55:15.944 INFO:teuthology.orchestra.run.smithi018.stdout: "max_osd": 8, 2024-09-10T18:55:15.944 INFO:teuthology.orchestra.run.smithi018.stdout: "require_min_compat_client": "luminous", 2024-09-10T18:55:15.944 INFO:teuthology.orchestra.run.smithi018.stdout: "min_compat_client": "luminous", 2024-09-10T18:55:15.945 INFO:teuthology.orchestra.run.smithi018.stdout: "require_osd_release": "quincy", 2024-09-10T18:55:15.945 INFO:teuthology.orchestra.run.smithi018.stdout: "pools": [ 2024-09-10T18:55:15.945 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:15.945 INFO:teuthology.orchestra.run.smithi018.stdout: "pool": 1, 2024-09-10T18:55:15.945 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_name": ".mgr", 2024-09-10T18:55:15.945 INFO:teuthology.orchestra.run.smithi018.stdout: "create_time": "2024-09-10T18:43:08.162659+0000", 2024-09-10T18:55:15.945 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-09-10T18:55:15.945 INFO:teuthology.orchestra.run.smithi018.stdout: "flags_names": "hashpspool", 2024-09-10T18:55:15.945 INFO:teuthology.orchestra.run.smithi018.stdout: "type": 1, 2024-09-10T18:55:15.945 INFO:teuthology.orchestra.run.smithi018.stdout: "size": 3, 2024-09-10T18:55:15.945 INFO:teuthology.orchestra.run.smithi018.stdout: "min_size": 2, 2024-09-10T18:55:15.945 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_rule": 0, 2024-09-10T18:55:15.945 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_count": 0, 2024-09-10T18:55:15.946 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_target": 0, 2024-09-10T18:55:15.946 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T18:55:15.946 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T18:55:15.946 INFO:teuthology.orchestra.run.smithi018.stdout: "object_hash": 2, 2024-09-10T18:55:15.946 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscale_mode": "off", 2024-09-10T18:55:15.946 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num": 21, 2024-09-10T18:55:15.946 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num": 21, 2024-09-10T18:55:15.946 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num_target": 21, 2024-09-10T18:55:15.946 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_target": 21, 2024-09-10T18:55:15.946 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_pending": 21, 2024-09-10T18:55:15.946 INFO:teuthology.orchestra.run.smithi018.stdout: "last_pg_merge_meta": { 2024-09-10T18:55:15.946 INFO:teuthology.orchestra.run.smithi018.stdout: "source_pgid": "0.0", 2024-09-10T18:55:15.947 INFO:teuthology.orchestra.run.smithi018.stdout: "ready_epoch": 0, 2024-09-10T18:55:15.947 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_started": 0, 2024-09-10T18:55:15.947 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_clean": 0, 2024-09-10T18:55:15.947 INFO:teuthology.orchestra.run.smithi018.stdout: "source_version": "0'0", 2024-09-10T18:55:15.947 INFO:teuthology.orchestra.run.smithi018.stdout: "target_version": "0'0" 2024-09-10T18:55:15.947 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:15.947 INFO:teuthology.orchestra.run.smithi018.stdout: "last_change": "169", 2024-09-10T18:55:15.947 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend": "0", 2024-09-10T18:55:15.947 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T18:55:15.947 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_preluminous": "167", 2024-09-10T18:55:15.948 INFO:teuthology.orchestra.run.smithi018.stdout: "auid": 0, 2024-09-10T18:55:15.948 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_mode": "selfmanaged", 2024-09-10T18:55:15.948 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_seq": 0, 2024-09-10T18:55:15.948 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_epoch": 0, 2024-09-10T18:55:15.948 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_snaps": [], 2024-09-10T18:55:15.948 INFO:teuthology.orchestra.run.smithi018.stdout: "removed_snaps": "[]", 2024-09-10T18:55:15.948 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_bytes": 0, 2024-09-10T18:55:15.948 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_objects": 0, 2024-09-10T18:55:15.948 INFO:teuthology.orchestra.run.smithi018.stdout: "tiers": [], 2024-09-10T18:55:15.948 INFO:teuthology.orchestra.run.smithi018.stdout: "tier_of": -1, 2024-09-10T18:55:15.948 INFO:teuthology.orchestra.run.smithi018.stdout: "read_tier": -1, 2024-09-10T18:55:15.949 INFO:teuthology.orchestra.run.smithi018.stdout: "write_tier": -1, 2024-09-10T18:55:15.949 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_mode": "none", 2024-09-10T18:55:15.949 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_bytes": 0, 2024-09-10T18:55:15.949 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_objects": 0, 2024-09-10T18:55:15.949 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T18:55:15.949 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T18:55:15.949 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T18:55:15.949 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_flush_age": 0, 2024-09-10T18:55:15.949 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_evict_age": 0, 2024-09-10T18:55:15.949 INFO:teuthology.orchestra.run.smithi018.stdout: "erasure_code_profile": "", 2024-09-10T18:55:15.950 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_params": { 2024-09-10T18:55:15.950 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "none" 2024-09-10T18:55:15.951 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:15.951 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_period": 0, 2024-09-10T18:55:15.951 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_count": 0, 2024-09-10T18:55:15.951 INFO:teuthology.orchestra.run.smithi018.stdout: "use_gmt_hitset": true, 2024-09-10T18:55:15.951 INFO:teuthology.orchestra.run.smithi018.stdout: "min_read_recency_for_promote": 0, 2024-09-10T18:55:15.951 INFO:teuthology.orchestra.run.smithi018.stdout: "min_write_recency_for_promote": 0, 2024-09-10T18:55:15.951 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T18:55:15.951 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_search_last_n": 0, 2024-09-10T18:55:15.951 INFO:teuthology.orchestra.run.smithi018.stdout: "grade_table": [], 2024-09-10T18:55:15.951 INFO:teuthology.orchestra.run.smithi018.stdout: "stripe_width": 0, 2024-09-10T18:55:15.951 INFO:teuthology.orchestra.run.smithi018.stdout: "expected_num_objects": 0, 2024-09-10T18:55:15.952 INFO:teuthology.orchestra.run.smithi018.stdout: "fast_read": false, 2024-09-10T18:55:15.952 INFO:teuthology.orchestra.run.smithi018.stdout: "options": { 2024-09-10T18:55:15.952 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_max": 32, 2024-09-10T18:55:15.952 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_min": 1 2024-09-10T18:55:15.952 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:15.952 INFO:teuthology.orchestra.run.smithi018.stdout: "application_metadata": { 2024-09-10T18:55:15.952 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": {} 2024-09-10T18:55:15.952 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:15.952 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:15.952 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:15.952 INFO:teuthology.orchestra.run.smithi018.stdout: "pool": 2, 2024-09-10T18:55:15.952 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_name": ".rgw.root", 2024-09-10T18:55:15.953 INFO:teuthology.orchestra.run.smithi018.stdout: "create_time": "2024-09-10T18:45:16.512219+0000", 2024-09-10T18:55:15.953 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-09-10T18:55:15.953 INFO:teuthology.orchestra.run.smithi018.stdout: "flags_names": "hashpspool", 2024-09-10T18:55:15.953 INFO:teuthology.orchestra.run.smithi018.stdout: "type": 1, 2024-09-10T18:55:15.953 INFO:teuthology.orchestra.run.smithi018.stdout: "size": 3, 2024-09-10T18:55:15.953 INFO:teuthology.orchestra.run.smithi018.stdout: "min_size": 2, 2024-09-10T18:55:15.953 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_rule": 0, 2024-09-10T18:55:15.953 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_count": 0, 2024-09-10T18:55:15.953 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_target": 0, 2024-09-10T18:55:15.953 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T18:55:15.953 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T18:55:15.953 INFO:teuthology.orchestra.run.smithi018.stdout: "object_hash": 2, 2024-09-10T18:55:15.954 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscale_mode": "off", 2024-09-10T18:55:15.954 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num": 32, 2024-09-10T18:55:15.954 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num": 32, 2024-09-10T18:55:15.954 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num_target": 32, 2024-09-10T18:55:15.954 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_target": 32, 2024-09-10T18:55:15.954 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_pending": 32, 2024-09-10T18:55:15.954 INFO:teuthology.orchestra.run.smithi018.stdout: "last_pg_merge_meta": { 2024-09-10T18:55:15.954 INFO:teuthology.orchestra.run.smithi018.stdout: "source_pgid": "0.0", 2024-09-10T18:55:15.954 INFO:teuthology.orchestra.run.smithi018.stdout: "ready_epoch": 0, 2024-09-10T18:55:15.954 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_started": 0, 2024-09-10T18:55:15.954 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_clean": 0, 2024-09-10T18:55:15.954 INFO:teuthology.orchestra.run.smithi018.stdout: "source_version": "0'0", 2024-09-10T18:55:15.954 INFO:teuthology.orchestra.run.smithi018.stdout: "target_version": "0'0" 2024-09-10T18:55:15.954 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:15.955 INFO:teuthology.orchestra.run.smithi018.stdout: "last_change": "57", 2024-09-10T18:55:15.955 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend": "0", 2024-09-10T18:55:15.955 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T18:55:15.955 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T18:55:15.955 INFO:teuthology.orchestra.run.smithi018.stdout: "auid": 0, 2024-09-10T18:55:15.955 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_mode": "selfmanaged", 2024-09-10T18:55:15.955 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_seq": 0, 2024-09-10T18:55:15.955 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_epoch": 0, 2024-09-10T18:55:15.955 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_snaps": [], 2024-09-10T18:55:15.955 INFO:teuthology.orchestra.run.smithi018.stdout: "removed_snaps": "[]", 2024-09-10T18:55:15.955 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_bytes": 0, 2024-09-10T18:55:15.955 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_objects": 0, 2024-09-10T18:55:15.955 INFO:teuthology.orchestra.run.smithi018.stdout: "tiers": [], 2024-09-10T18:55:15.956 INFO:teuthology.orchestra.run.smithi018.stdout: "tier_of": -1, 2024-09-10T18:55:15.956 INFO:teuthology.orchestra.run.smithi018.stdout: "read_tier": -1, 2024-09-10T18:55:15.956 INFO:teuthology.orchestra.run.smithi018.stdout: "write_tier": -1, 2024-09-10T18:55:15.956 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_mode": "none", 2024-09-10T18:55:15.956 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_bytes": 0, 2024-09-10T18:55:15.956 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_objects": 0, 2024-09-10T18:55:15.956 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T18:55:15.956 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T18:55:15.956 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T18:55:15.956 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_flush_age": 0, 2024-09-10T18:55:15.956 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_evict_age": 0, 2024-09-10T18:55:15.956 INFO:teuthology.orchestra.run.smithi018.stdout: "erasure_code_profile": "", 2024-09-10T18:55:15.956 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_params": { 2024-09-10T18:55:15.956 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "none" 2024-09-10T18:55:15.957 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:15.957 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_period": 0, 2024-09-10T18:55:15.957 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_count": 0, 2024-09-10T18:55:15.957 INFO:teuthology.orchestra.run.smithi018.stdout: "use_gmt_hitset": true, 2024-09-10T18:55:15.957 INFO:teuthology.orchestra.run.smithi018.stdout: "min_read_recency_for_promote": 0, 2024-09-10T18:55:15.957 INFO:teuthology.orchestra.run.smithi018.stdout: "min_write_recency_for_promote": 0, 2024-09-10T18:55:15.957 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T18:55:15.957 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_search_last_n": 0, 2024-09-10T18:55:15.957 INFO:teuthology.orchestra.run.smithi018.stdout: "grade_table": [], 2024-09-10T18:55:15.957 INFO:teuthology.orchestra.run.smithi018.stdout: "stripe_width": 0, 2024-09-10T18:55:15.957 INFO:teuthology.orchestra.run.smithi018.stdout: "expected_num_objects": 0, 2024-09-10T18:55:15.957 INFO:teuthology.orchestra.run.smithi018.stdout: "fast_read": false, 2024-09-10T18:55:15.957 INFO:teuthology.orchestra.run.smithi018.stdout: "options": {}, 2024-09-10T18:55:15.958 INFO:teuthology.orchestra.run.smithi018.stdout: "application_metadata": { 2024-09-10T18:55:15.958 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw": {} 2024-09-10T18:55:15.958 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:15.958 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:15.958 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:15.958 INFO:teuthology.orchestra.run.smithi018.stdout: "pool": 3, 2024-09-10T18:55:15.958 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_name": "datapool", 2024-09-10T18:55:15.958 INFO:teuthology.orchestra.run.smithi018.stdout: "create_time": "2024-09-10T18:45:16.943626+0000", 2024-09-10T18:55:15.958 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 8193, 2024-09-10T18:55:15.958 INFO:teuthology.orchestra.run.smithi018.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-10T18:55:15.958 INFO:teuthology.orchestra.run.smithi018.stdout: "type": 1, 2024-09-10T18:55:15.958 INFO:teuthology.orchestra.run.smithi018.stdout: "size": 3, 2024-09-10T18:55:15.958 INFO:teuthology.orchestra.run.smithi018.stdout: "min_size": 2, 2024-09-10T18:55:15.958 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_rule": 0, 2024-09-10T18:55:15.959 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_count": 0, 2024-09-10T18:55:15.959 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_target": 0, 2024-09-10T18:55:15.959 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T18:55:15.959 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T18:55:15.959 INFO:teuthology.orchestra.run.smithi018.stdout: "object_hash": 2, 2024-09-10T18:55:15.959 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscale_mode": "off", 2024-09-10T18:55:15.959 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num": 3, 2024-09-10T18:55:15.959 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num": 3, 2024-09-10T18:55:15.959 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num_target": 3, 2024-09-10T18:55:15.959 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_target": 3, 2024-09-10T18:55:15.959 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_pending": 3, 2024-09-10T18:55:15.959 INFO:teuthology.orchestra.run.smithi018.stdout: "last_pg_merge_meta": { 2024-09-10T18:55:15.959 INFO:teuthology.orchestra.run.smithi018.stdout: "source_pgid": "0.0", 2024-09-10T18:55:15.960 INFO:teuthology.orchestra.run.smithi018.stdout: "ready_epoch": 0, 2024-09-10T18:55:15.960 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_started": 0, 2024-09-10T18:55:15.960 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_clean": 0, 2024-09-10T18:55:15.960 INFO:teuthology.orchestra.run.smithi018.stdout: "source_version": "0'0", 2024-09-10T18:55:15.960 INFO:teuthology.orchestra.run.smithi018.stdout: "target_version": "0'0" 2024-09-10T18:55:15.960 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:15.960 INFO:teuthology.orchestra.run.smithi018.stdout: "last_change": "60", 2024-09-10T18:55:15.960 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend": "0", 2024-09-10T18:55:15.960 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T18:55:15.960 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T18:55:15.960 INFO:teuthology.orchestra.run.smithi018.stdout: "auid": 0, 2024-09-10T18:55:15.960 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_mode": "selfmanaged", 2024-09-10T18:55:15.960 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_seq": 3, 2024-09-10T18:55:15.961 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_epoch": 60, 2024-09-10T18:55:15.961 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_snaps": [], 2024-09-10T18:55:15.961 INFO:teuthology.orchestra.run.smithi018.stdout: "removed_snaps": "[]", 2024-09-10T18:55:15.961 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_bytes": 0, 2024-09-10T18:55:15.961 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_objects": 0, 2024-09-10T18:55:15.961 INFO:teuthology.orchestra.run.smithi018.stdout: "tiers": [], 2024-09-10T18:55:15.961 INFO:teuthology.orchestra.run.smithi018.stdout: "tier_of": -1, 2024-09-10T18:55:15.961 INFO:teuthology.orchestra.run.smithi018.stdout: "read_tier": -1, 2024-09-10T18:55:15.961 INFO:teuthology.orchestra.run.smithi018.stdout: "write_tier": -1, 2024-09-10T18:55:15.961 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_mode": "none", 2024-09-10T18:55:15.961 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_bytes": 0, 2024-09-10T18:55:15.961 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_objects": 0, 2024-09-10T18:55:15.961 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T18:55:15.961 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T18:55:15.962 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T18:55:15.962 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_flush_age": 0, 2024-09-10T18:55:15.962 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_evict_age": 0, 2024-09-10T18:55:15.962 INFO:teuthology.orchestra.run.smithi018.stdout: "erasure_code_profile": "", 2024-09-10T18:55:15.962 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_params": { 2024-09-10T18:55:15.962 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "none" 2024-09-10T18:55:15.963 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:15.963 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_period": 0, 2024-09-10T18:55:15.963 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_count": 0, 2024-09-10T18:55:15.963 INFO:teuthology.orchestra.run.smithi018.stdout: "use_gmt_hitset": true, 2024-09-10T18:55:15.963 INFO:teuthology.orchestra.run.smithi018.stdout: "min_read_recency_for_promote": 0, 2024-09-10T18:55:15.963 INFO:teuthology.orchestra.run.smithi018.stdout: "min_write_recency_for_promote": 0, 2024-09-10T18:55:15.964 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T18:55:15.964 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_search_last_n": 0, 2024-09-10T18:55:15.964 INFO:teuthology.orchestra.run.smithi018.stdout: "grade_table": [], 2024-09-10T18:55:15.964 INFO:teuthology.orchestra.run.smithi018.stdout: "stripe_width": 0, 2024-09-10T18:55:15.964 INFO:teuthology.orchestra.run.smithi018.stdout: "expected_num_objects": 0, 2024-09-10T18:55:15.964 INFO:teuthology.orchestra.run.smithi018.stdout: "fast_read": false, 2024-09-10T18:55:15.964 INFO:teuthology.orchestra.run.smithi018.stdout: "options": {}, 2024-09-10T18:55:15.964 INFO:teuthology.orchestra.run.smithi018.stdout: "application_metadata": { 2024-09-10T18:55:15.964 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd": {} 2024-09-10T18:55:15.964 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:15.964 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:15.964 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:15.964 INFO:teuthology.orchestra.run.smithi018.stdout: "pool": 4, 2024-09-10T18:55:15.965 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_name": "default.rgw.log", 2024-09-10T18:55:15.965 INFO:teuthology.orchestra.run.smithi018.stdout: "create_time": "2024-09-10T18:45:18.450026+0000", 2024-09-10T18:55:15.965 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-09-10T18:55:15.965 INFO:teuthology.orchestra.run.smithi018.stdout: "flags_names": "hashpspool", 2024-09-10T18:55:15.965 INFO:teuthology.orchestra.run.smithi018.stdout: "type": 1, 2024-09-10T18:55:15.965 INFO:teuthology.orchestra.run.smithi018.stdout: "size": 3, 2024-09-10T18:55:15.965 INFO:teuthology.orchestra.run.smithi018.stdout: "min_size": 2, 2024-09-10T18:55:15.965 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_rule": 0, 2024-09-10T18:55:15.965 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_count": 0, 2024-09-10T18:55:15.965 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_target": 0, 2024-09-10T18:55:15.965 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T18:55:15.965 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T18:55:15.965 INFO:teuthology.orchestra.run.smithi018.stdout: "object_hash": 2, 2024-09-10T18:55:15.965 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscale_mode": "off", 2024-09-10T18:55:15.966 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num": 32, 2024-09-10T18:55:15.966 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num": 32, 2024-09-10T18:55:15.966 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num_target": 32, 2024-09-10T18:55:15.966 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_target": 32, 2024-09-10T18:55:15.966 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_pending": 32, 2024-09-10T18:55:15.966 INFO:teuthology.orchestra.run.smithi018.stdout: "last_pg_merge_meta": { 2024-09-10T18:55:15.966 INFO:teuthology.orchestra.run.smithi018.stdout: "source_pgid": "0.0", 2024-09-10T18:55:15.966 INFO:teuthology.orchestra.run.smithi018.stdout: "ready_epoch": 0, 2024-09-10T18:55:15.966 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_started": 0, 2024-09-10T18:55:15.966 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_clean": 0, 2024-09-10T18:55:15.966 INFO:teuthology.orchestra.run.smithi018.stdout: "source_version": "0'0", 2024-09-10T18:55:15.966 INFO:teuthology.orchestra.run.smithi018.stdout: "target_version": "0'0" 2024-09-10T18:55:15.966 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:15.966 INFO:teuthology.orchestra.run.smithi018.stdout: "last_change": "59", 2024-09-10T18:55:15.967 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend": "0", 2024-09-10T18:55:15.967 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T18:55:15.967 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T18:55:15.967 INFO:teuthology.orchestra.run.smithi018.stdout: "auid": 0, 2024-09-10T18:55:15.967 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_mode": "selfmanaged", 2024-09-10T18:55:15.967 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_seq": 0, 2024-09-10T18:55:15.967 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_epoch": 0, 2024-09-10T18:55:15.967 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_snaps": [], 2024-09-10T18:55:15.967 INFO:teuthology.orchestra.run.smithi018.stdout: "removed_snaps": "[]", 2024-09-10T18:55:15.967 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_bytes": 0, 2024-09-10T18:55:15.967 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_objects": 0, 2024-09-10T18:55:15.967 INFO:teuthology.orchestra.run.smithi018.stdout: "tiers": [], 2024-09-10T18:55:15.967 INFO:teuthology.orchestra.run.smithi018.stdout: "tier_of": -1, 2024-09-10T18:55:15.968 INFO:teuthology.orchestra.run.smithi018.stdout: "read_tier": -1, 2024-09-10T18:55:15.968 INFO:teuthology.orchestra.run.smithi018.stdout: "write_tier": -1, 2024-09-10T18:55:15.968 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_mode": "none", 2024-09-10T18:55:15.968 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_bytes": 0, 2024-09-10T18:55:15.968 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_objects": 0, 2024-09-10T18:55:15.968 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T18:55:15.968 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T18:55:15.968 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T18:55:15.968 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_flush_age": 0, 2024-09-10T18:55:15.968 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_evict_age": 0, 2024-09-10T18:55:15.968 INFO:teuthology.orchestra.run.smithi018.stdout: "erasure_code_profile": "", 2024-09-10T18:55:15.968 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_params": { 2024-09-10T18:55:15.968 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "none" 2024-09-10T18:55:15.968 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:15.969 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_period": 0, 2024-09-10T18:55:15.969 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_count": 0, 2024-09-10T18:55:15.969 INFO:teuthology.orchestra.run.smithi018.stdout: "use_gmt_hitset": true, 2024-09-10T18:55:15.969 INFO:teuthology.orchestra.run.smithi018.stdout: "min_read_recency_for_promote": 0, 2024-09-10T18:55:15.969 INFO:teuthology.orchestra.run.smithi018.stdout: "min_write_recency_for_promote": 0, 2024-09-10T18:55:15.969 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T18:55:15.969 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_search_last_n": 0, 2024-09-10T18:55:15.969 INFO:teuthology.orchestra.run.smithi018.stdout: "grade_table": [], 2024-09-10T18:55:15.969 INFO:teuthology.orchestra.run.smithi018.stdout: "stripe_width": 0, 2024-09-10T18:55:15.969 INFO:teuthology.orchestra.run.smithi018.stdout: "expected_num_objects": 0, 2024-09-10T18:55:15.969 INFO:teuthology.orchestra.run.smithi018.stdout: "fast_read": false, 2024-09-10T18:55:15.969 INFO:teuthology.orchestra.run.smithi018.stdout: "options": {}, 2024-09-10T18:55:15.969 INFO:teuthology.orchestra.run.smithi018.stdout: "application_metadata": { 2024-09-10T18:55:15.970 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw": {} 2024-09-10T18:55:15.970 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:15.970 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:15.970 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:15.970 INFO:teuthology.orchestra.run.smithi018.stdout: "pool": 5, 2024-09-10T18:55:15.970 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_name": "default.rgw.control", 2024-09-10T18:55:15.970 INFO:teuthology.orchestra.run.smithi018.stdout: "create_time": "2024-09-10T18:45:20.494618+0000", 2024-09-10T18:55:15.970 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-09-10T18:55:15.970 INFO:teuthology.orchestra.run.smithi018.stdout: "flags_names": "hashpspool", 2024-09-10T18:55:15.970 INFO:teuthology.orchestra.run.smithi018.stdout: "type": 1, 2024-09-10T18:55:15.970 INFO:teuthology.orchestra.run.smithi018.stdout: "size": 3, 2024-09-10T18:55:15.970 INFO:teuthology.orchestra.run.smithi018.stdout: "min_size": 2, 2024-09-10T18:55:15.970 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_rule": 0, 2024-09-10T18:55:15.971 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_count": 0, 2024-09-10T18:55:15.971 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_target": 0, 2024-09-10T18:55:15.971 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T18:55:15.971 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T18:55:15.971 INFO:teuthology.orchestra.run.smithi018.stdout: "object_hash": 2, 2024-09-10T18:55:15.971 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscale_mode": "off", 2024-09-10T18:55:15.971 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num": 32, 2024-09-10T18:55:15.971 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num": 32, 2024-09-10T18:55:15.971 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num_target": 32, 2024-09-10T18:55:15.971 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_target": 32, 2024-09-10T18:55:15.971 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_pending": 32, 2024-09-10T18:55:15.971 INFO:teuthology.orchestra.run.smithi018.stdout: "last_pg_merge_meta": { 2024-09-10T18:55:15.971 INFO:teuthology.orchestra.run.smithi018.stdout: "source_pgid": "0.0", 2024-09-10T18:55:15.972 INFO:teuthology.orchestra.run.smithi018.stdout: "ready_epoch": 0, 2024-09-10T18:55:15.972 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_started": 0, 2024-09-10T18:55:15.972 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_clean": 0, 2024-09-10T18:55:15.972 INFO:teuthology.orchestra.run.smithi018.stdout: "source_version": "0'0", 2024-09-10T18:55:15.972 INFO:teuthology.orchestra.run.smithi018.stdout: "target_version": "0'0" 2024-09-10T18:55:15.972 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:15.972 INFO:teuthology.orchestra.run.smithi018.stdout: "last_change": "61", 2024-09-10T18:55:15.972 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend": "0", 2024-09-10T18:55:15.972 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T18:55:15.972 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T18:55:15.972 INFO:teuthology.orchestra.run.smithi018.stdout: "auid": 0, 2024-09-10T18:55:15.972 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_mode": "selfmanaged", 2024-09-10T18:55:15.973 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_seq": 0, 2024-09-10T18:55:15.973 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_epoch": 0, 2024-09-10T18:55:15.973 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_snaps": [], 2024-09-10T18:55:15.973 INFO:teuthology.orchestra.run.smithi018.stdout: "removed_snaps": "[]", 2024-09-10T18:55:15.973 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_bytes": 0, 2024-09-10T18:55:15.973 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_objects": 0, 2024-09-10T18:55:15.973 INFO:teuthology.orchestra.run.smithi018.stdout: "tiers": [], 2024-09-10T18:55:15.973 INFO:teuthology.orchestra.run.smithi018.stdout: "tier_of": -1, 2024-09-10T18:55:15.973 INFO:teuthology.orchestra.run.smithi018.stdout: "read_tier": -1, 2024-09-10T18:55:15.973 INFO:teuthology.orchestra.run.smithi018.stdout: "write_tier": -1, 2024-09-10T18:55:15.973 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_mode": "none", 2024-09-10T18:55:15.973 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_bytes": 0, 2024-09-10T18:55:15.974 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_objects": 0, 2024-09-10T18:55:15.974 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T18:55:15.974 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T18:55:15.974 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T18:55:15.974 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_flush_age": 0, 2024-09-10T18:55:15.974 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_evict_age": 0, 2024-09-10T18:55:15.974 INFO:teuthology.orchestra.run.smithi018.stdout: "erasure_code_profile": "", 2024-09-10T18:55:15.974 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_params": { 2024-09-10T18:55:15.974 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "none" 2024-09-10T18:55:15.974 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:15.974 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_period": 0, 2024-09-10T18:55:15.974 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_count": 0, 2024-09-10T18:55:15.974 INFO:teuthology.orchestra.run.smithi018.stdout: "use_gmt_hitset": true, 2024-09-10T18:55:15.974 INFO:teuthology.orchestra.run.smithi018.stdout: "min_read_recency_for_promote": 0, 2024-09-10T18:55:15.975 INFO:teuthology.orchestra.run.smithi018.stdout: "min_write_recency_for_promote": 0, 2024-09-10T18:55:15.975 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T18:55:15.975 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_search_last_n": 0, 2024-09-10T18:55:15.975 INFO:teuthology.orchestra.run.smithi018.stdout: "grade_table": [], 2024-09-10T18:55:15.975 INFO:teuthology.orchestra.run.smithi018.stdout: "stripe_width": 0, 2024-09-10T18:55:15.975 INFO:teuthology.orchestra.run.smithi018.stdout: "expected_num_objects": 0, 2024-09-10T18:55:15.975 INFO:teuthology.orchestra.run.smithi018.stdout: "fast_read": false, 2024-09-10T18:55:15.975 INFO:teuthology.orchestra.run.smithi018.stdout: "options": {}, 2024-09-10T18:55:15.975 INFO:teuthology.orchestra.run.smithi018.stdout: "application_metadata": { 2024-09-10T18:55:15.975 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw": {} 2024-09-10T18:55:15.975 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:15.975 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:15.975 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:15.975 INFO:teuthology.orchestra.run.smithi018.stdout: "pool": 6, 2024-09-10T18:55:15.975 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_name": "default.rgw.meta", 2024-09-10T18:55:15.976 INFO:teuthology.orchestra.run.smithi018.stdout: "create_time": "2024-09-10T18:45:22.566992+0000", 2024-09-10T18:55:15.976 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-09-10T18:55:15.976 INFO:teuthology.orchestra.run.smithi018.stdout: "flags_names": "hashpspool", 2024-09-10T18:55:15.976 INFO:teuthology.orchestra.run.smithi018.stdout: "type": 1, 2024-09-10T18:55:15.976 INFO:teuthology.orchestra.run.smithi018.stdout: "size": 3, 2024-09-10T18:55:15.976 INFO:teuthology.orchestra.run.smithi018.stdout: "min_size": 2, 2024-09-10T18:55:15.976 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_rule": 0, 2024-09-10T18:55:15.976 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_count": 0, 2024-09-10T18:55:15.976 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_target": 0, 2024-09-10T18:55:15.976 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T18:55:15.976 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T18:55:15.976 INFO:teuthology.orchestra.run.smithi018.stdout: "object_hash": 2, 2024-09-10T18:55:15.976 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscale_mode": "off", 2024-09-10T18:55:15.976 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num": 32, 2024-09-10T18:55:15.977 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num": 32, 2024-09-10T18:55:15.977 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num_target": 32, 2024-09-10T18:55:15.977 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_target": 32, 2024-09-10T18:55:15.977 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_pending": 32, 2024-09-10T18:55:15.977 INFO:teuthology.orchestra.run.smithi018.stdout: "last_pg_merge_meta": { 2024-09-10T18:55:15.977 INFO:teuthology.orchestra.run.smithi018.stdout: "source_pgid": "0.0", 2024-09-10T18:55:15.977 INFO:teuthology.orchestra.run.smithi018.stdout: "ready_epoch": 0, 2024-09-10T18:55:15.977 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_started": 0, 2024-09-10T18:55:15.980 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_clean": 0, 2024-09-10T18:55:15.980 INFO:teuthology.orchestra.run.smithi018.stdout: "source_version": "0'0", 2024-09-10T18:55:15.980 INFO:teuthology.orchestra.run.smithi018.stdout: "target_version": "0'0" 2024-09-10T18:55:15.981 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:15.981 INFO:teuthology.orchestra.run.smithi018.stdout: "last_change": "63", 2024-09-10T18:55:15.981 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend": "0", 2024-09-10T18:55:15.981 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T18:55:15.981 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T18:55:15.981 INFO:teuthology.orchestra.run.smithi018.stdout: "auid": 0, 2024-09-10T18:55:15.981 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_mode": "selfmanaged", 2024-09-10T18:55:15.981 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_seq": 0, 2024-09-10T18:55:15.981 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_epoch": 0, 2024-09-10T18:55:15.981 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_snaps": [], 2024-09-10T18:55:15.981 INFO:teuthology.orchestra.run.smithi018.stdout: "removed_snaps": "[]", 2024-09-10T18:55:15.981 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_bytes": 0, 2024-09-10T18:55:15.981 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_objects": 0, 2024-09-10T18:55:15.982 INFO:teuthology.orchestra.run.smithi018.stdout: "tiers": [], 2024-09-10T18:55:15.982 INFO:teuthology.orchestra.run.smithi018.stdout: "tier_of": -1, 2024-09-10T18:55:15.982 INFO:teuthology.orchestra.run.smithi018.stdout: "read_tier": -1, 2024-09-10T18:55:15.982 INFO:teuthology.orchestra.run.smithi018.stdout: "write_tier": -1, 2024-09-10T18:55:15.982 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_mode": "none", 2024-09-10T18:55:15.982 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_bytes": 0, 2024-09-10T18:55:15.982 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_objects": 0, 2024-09-10T18:55:15.982 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T18:55:15.982 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T18:55:15.982 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T18:55:15.982 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_flush_age": 0, 2024-09-10T18:55:15.982 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_evict_age": 0, 2024-09-10T18:55:15.983 INFO:teuthology.orchestra.run.smithi018.stdout: "erasure_code_profile": "", 2024-09-10T18:55:15.983 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_params": { 2024-09-10T18:55:15.983 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "none" 2024-09-10T18:55:15.983 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:15.983 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_period": 0, 2024-09-10T18:55:15.983 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_count": 0, 2024-09-10T18:55:15.983 INFO:teuthology.orchestra.run.smithi018.stdout: "use_gmt_hitset": true, 2024-09-10T18:55:15.983 INFO:teuthology.orchestra.run.smithi018.stdout: "min_read_recency_for_promote": 0, 2024-09-10T18:55:15.983 INFO:teuthology.orchestra.run.smithi018.stdout: "min_write_recency_for_promote": 0, 2024-09-10T18:55:15.983 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T18:55:15.983 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_search_last_n": 0, 2024-09-10T18:55:15.983 INFO:teuthology.orchestra.run.smithi018.stdout: "grade_table": [], 2024-09-10T18:55:15.983 INFO:teuthology.orchestra.run.smithi018.stdout: "stripe_width": 0, 2024-09-10T18:55:15.984 INFO:teuthology.orchestra.run.smithi018.stdout: "expected_num_objects": 0, 2024-09-10T18:55:15.984 INFO:teuthology.orchestra.run.smithi018.stdout: "fast_read": false, 2024-09-10T18:55:15.984 INFO:teuthology.orchestra.run.smithi018.stdout: "options": { 2024-09-10T18:55:15.984 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscale_bias": 4 2024-09-10T18:55:15.984 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:15.984 INFO:teuthology.orchestra.run.smithi018.stdout: "application_metadata": { 2024-09-10T18:55:15.984 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw": {} 2024-09-10T18:55:15.984 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:15.984 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:15.984 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:15.984 INFO:teuthology.orchestra.run.smithi018.stdout: "pool": 8, 2024-09-10T18:55:15.984 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_name": "unique_pool_1", 2024-09-10T18:55:15.984 INFO:teuthology.orchestra.run.smithi018.stdout: "create_time": "2024-09-10T18:51:36.493636+0000", 2024-09-10T18:55:15.985 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-09-10T18:55:15.985 INFO:teuthology.orchestra.run.smithi018.stdout: "flags_names": "hashpspool", 2024-09-10T18:55:15.985 INFO:teuthology.orchestra.run.smithi018.stdout: "type": 1, 2024-09-10T18:55:15.985 INFO:teuthology.orchestra.run.smithi018.stdout: "size": 3, 2024-09-10T18:55:15.985 INFO:teuthology.orchestra.run.smithi018.stdout: "min_size": 2, 2024-09-10T18:55:15.985 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_rule": 0, 2024-09-10T18:55:15.985 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_count": 0, 2024-09-10T18:55:15.985 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_target": 0, 2024-09-10T18:55:15.985 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T18:55:15.985 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T18:55:15.985 INFO:teuthology.orchestra.run.smithi018.stdout: "object_hash": 2, 2024-09-10T18:55:15.985 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscale_mode": "off", 2024-09-10T18:55:15.985 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num": 16, 2024-09-10T18:55:15.986 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num": 16, 2024-09-10T18:55:15.986 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num_target": 16, 2024-09-10T18:55:15.986 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_target": 16, 2024-09-10T18:55:15.986 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_pending": 16, 2024-09-10T18:55:15.986 INFO:teuthology.orchestra.run.smithi018.stdout: "last_pg_merge_meta": { 2024-09-10T18:55:15.986 INFO:teuthology.orchestra.run.smithi018.stdout: "source_pgid": "0.0", 2024-09-10T18:55:15.986 INFO:teuthology.orchestra.run.smithi018.stdout: "ready_epoch": 0, 2024-09-10T18:55:15.986 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_started": 0, 2024-09-10T18:55:15.986 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_clean": 0, 2024-09-10T18:55:15.986 INFO:teuthology.orchestra.run.smithi018.stdout: "source_version": "0'0", 2024-09-10T18:55:15.986 INFO:teuthology.orchestra.run.smithi018.stdout: "target_version": "0'0" 2024-09-10T18:55:15.986 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:15.986 INFO:teuthology.orchestra.run.smithi018.stdout: "last_change": "202", 2024-09-10T18:55:15.987 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend": "0", 2024-09-10T18:55:15.987 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T18:55:15.987 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T18:55:15.987 INFO:teuthology.orchestra.run.smithi018.stdout: "auid": 0, 2024-09-10T18:55:15.987 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_mode": "selfmanaged", 2024-09-10T18:55:15.987 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_seq": 0, 2024-09-10T18:55:15.987 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_epoch": 0, 2024-09-10T18:55:15.987 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_snaps": [], 2024-09-10T18:55:15.987 INFO:teuthology.orchestra.run.smithi018.stdout: "removed_snaps": "[]", 2024-09-10T18:55:15.987 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_bytes": 0, 2024-09-10T18:55:15.987 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_objects": 0, 2024-09-10T18:55:15.987 INFO:teuthology.orchestra.run.smithi018.stdout: "tiers": [], 2024-09-10T18:55:15.987 INFO:teuthology.orchestra.run.smithi018.stdout: "tier_of": -1, 2024-09-10T18:55:15.987 INFO:teuthology.orchestra.run.smithi018.stdout: "read_tier": -1, 2024-09-10T18:55:15.988 INFO:teuthology.orchestra.run.smithi018.stdout: "write_tier": -1, 2024-09-10T18:55:15.988 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_mode": "none", 2024-09-10T18:55:15.988 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_bytes": 0, 2024-09-10T18:55:15.988 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_objects": 0, 2024-09-10T18:55:15.988 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T18:55:15.988 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T18:55:15.988 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T18:55:15.988 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_flush_age": 0, 2024-09-10T18:55:15.988 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_evict_age": 0, 2024-09-10T18:55:15.988 INFO:teuthology.orchestra.run.smithi018.stdout: "erasure_code_profile": "", 2024-09-10T18:55:15.988 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_params": { 2024-09-10T18:55:15.988 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "none" 2024-09-10T18:55:15.989 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:15.989 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_period": 0, 2024-09-10T18:55:15.989 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_count": 0, 2024-09-10T18:55:15.989 INFO:teuthology.orchestra.run.smithi018.stdout: "use_gmt_hitset": true, 2024-09-10T18:55:15.989 INFO:teuthology.orchestra.run.smithi018.stdout: "min_read_recency_for_promote": 0, 2024-09-10T18:55:15.989 INFO:teuthology.orchestra.run.smithi018.stdout: "min_write_recency_for_promote": 0, 2024-09-10T18:55:15.989 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T18:55:15.989 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_search_last_n": 0, 2024-09-10T18:55:15.989 INFO:teuthology.orchestra.run.smithi018.stdout: "grade_table": [], 2024-09-10T18:55:15.989 INFO:teuthology.orchestra.run.smithi018.stdout: "stripe_width": 0, 2024-09-10T18:55:15.989 INFO:teuthology.orchestra.run.smithi018.stdout: "expected_num_objects": 0, 2024-09-10T18:55:15.989 INFO:teuthology.orchestra.run.smithi018.stdout: "fast_read": false, 2024-09-10T18:55:15.989 INFO:teuthology.orchestra.run.smithi018.stdout: "options": {}, 2024-09-10T18:55:15.990 INFO:teuthology.orchestra.run.smithi018.stdout: "application_metadata": { 2024-09-10T18:55:15.990 INFO:teuthology.orchestra.run.smithi018.stdout: "rados": {} 2024-09-10T18:55:15.990 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:15.990 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:15.990 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-09-10T18:55:15.990 INFO:teuthology.orchestra.run.smithi018.stdout: "osds": [ 2024-09-10T18:55:15.990 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:15.990 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 0, 2024-09-10T18:55:15.990 INFO:teuthology.orchestra.run.smithi018.stdout: "uuid": "eb4c7e67-300c-4a1a-b9f5-bb028eeeabc2", 2024-09-10T18:55:15.990 INFO:teuthology.orchestra.run.smithi018.stdout: "up": 0, 2024-09-10T18:55:15.990 INFO:teuthology.orchestra.run.smithi018.stdout: "in": 1, 2024-09-10T18:55:15.990 INFO:teuthology.orchestra.run.smithi018.stdout: "weight": 1, 2024-09-10T18:55:15.990 INFO:teuthology.orchestra.run.smithi018.stdout: "primary_affinity": 1, 2024-09-10T18:55:15.990 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_begin": 0, 2024-09-10T18:55:15.991 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_end": 0, 2024-09-10T18:55:15.991 INFO:teuthology.orchestra.run.smithi018.stdout: "up_from": 8, 2024-09-10T18:55:15.991 INFO:teuthology.orchestra.run.smithi018.stdout: "up_thru": 193, 2024-09-10T18:55:15.991 INFO:teuthology.orchestra.run.smithi018.stdout: "down_at": 206, 2024-09-10T18:55:15.991 INFO:teuthology.orchestra.run.smithi018.stdout: "lost_at": 0, 2024-09-10T18:55:15.991 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addrs": { 2024-09-10T18:55:15.991 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:55:15.991 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:15.991 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:55:15.991 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6801", 2024-09-10T18:55:15.991 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 2131279292 2024-09-10T18:55:15.991 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:15.991 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:15.992 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:15.992 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addrs": { 2024-09-10T18:55:15.992 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:55:15.992 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:15.992 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:55:15.992 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6802", 2024-09-10T18:55:15.992 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 2131279292 2024-09-10T18:55:15.992 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:15.992 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:15.992 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:15.992 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addrs": { 2024-09-10T18:55:15.992 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:55:15.992 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:15.993 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:55:15.993 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6804", 2024-09-10T18:55:15.993 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 2131279292 2024-09-10T18:55:15.993 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:15.993 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:15.993 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:15.993 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addrs": { 2024-09-10T18:55:15.993 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:55:15.993 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:15.993 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:55:15.993 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6803", 2024-09-10T18:55:15.993 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 2131279292 2024-09-10T18:55:15.993 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:15.993 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:15.994 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:15.994 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addr": "172.21.15.18:6801/2131279292", 2024-09-10T18:55:15.994 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addr": "172.21.15.18:6802/2131279292", 2024-09-10T18:55:15.994 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addr": "172.21.15.18:6804/2131279292", 2024-09-10T18:55:15.994 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addr": "172.21.15.18:6803/2131279292", 2024-09-10T18:55:15.994 INFO:teuthology.orchestra.run.smithi018.stdout: "state": [ 2024-09-10T18:55:15.994 INFO:teuthology.orchestra.run.smithi018.stdout: "exists" 2024-09-10T18:55:15.994 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:15.994 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:15.994 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:15.994 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 1, 2024-09-10T18:55:15.994 INFO:teuthology.orchestra.run.smithi018.stdout: "uuid": "63bf6a98-e349-4cc0-9502-a31693070c5b", 2024-09-10T18:55:15.994 INFO:teuthology.orchestra.run.smithi018.stdout: "up": 1, 2024-09-10T18:55:15.995 INFO:teuthology.orchestra.run.smithi018.stdout: "in": 1, 2024-09-10T18:55:15.995 INFO:teuthology.orchestra.run.smithi018.stdout: "weight": 1, 2024-09-10T18:55:15.995 INFO:teuthology.orchestra.run.smithi018.stdout: "primary_affinity": 1, 2024-09-10T18:55:15.995 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_begin": 0, 2024-09-10T18:55:15.995 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_end": 0, 2024-09-10T18:55:15.995 INFO:teuthology.orchestra.run.smithi018.stdout: "up_from": 13, 2024-09-10T18:55:15.995 INFO:teuthology.orchestra.run.smithi018.stdout: "up_thru": 222, 2024-09-10T18:55:15.995 INFO:teuthology.orchestra.run.smithi018.stdout: "down_at": 0, 2024-09-10T18:55:15.995 INFO:teuthology.orchestra.run.smithi018.stdout: "lost_at": 0, 2024-09-10T18:55:15.995 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addrs": { 2024-09-10T18:55:15.995 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:55:15.995 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:15.995 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:55:15.996 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6805", 2024-09-10T18:55:15.996 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3855640936 2024-09-10T18:55:15.996 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:15.996 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:15.996 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:15.996 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addrs": { 2024-09-10T18:55:15.996 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:55:15.996 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:15.996 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:55:15.996 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6806", 2024-09-10T18:55:15.997 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3855640936 2024-09-10T18:55:15.997 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:15.997 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:15.997 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:15.997 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addrs": { 2024-09-10T18:55:15.997 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:55:15.997 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:15.997 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:55:15.997 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6808", 2024-09-10T18:55:15.997 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3855640936 2024-09-10T18:55:15.997 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:15.997 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:15.997 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:15.998 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addrs": { 2024-09-10T18:55:15.998 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:55:15.998 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:15.998 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:55:15.998 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6807", 2024-09-10T18:55:15.998 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3855640936 2024-09-10T18:55:15.998 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:15.998 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:15.998 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:15.998 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addr": "172.21.15.18:6805/3855640936", 2024-09-10T18:55:15.998 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addr": "172.21.15.18:6806/3855640936", 2024-09-10T18:55:15.998 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addr": "172.21.15.18:6808/3855640936", 2024-09-10T18:55:15.998 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addr": "172.21.15.18:6807/3855640936", 2024-09-10T18:55:15.998 INFO:teuthology.orchestra.run.smithi018.stdout: "state": [ 2024-09-10T18:55:15.999 INFO:teuthology.orchestra.run.smithi018.stdout: "exists", 2024-09-10T18:55:15.999 INFO:teuthology.orchestra.run.smithi018.stdout: "up" 2024-09-10T18:55:15.999 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:15.999 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:15.999 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:15.999 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 2, 2024-09-10T18:55:15.999 INFO:teuthology.orchestra.run.smithi018.stdout: "uuid": "79a01c29-4f4d-43d1-b1ea-a3fcf014f84b", 2024-09-10T18:55:15.999 INFO:teuthology.orchestra.run.smithi018.stdout: "up": 1, 2024-09-10T18:55:15.999 INFO:teuthology.orchestra.run.smithi018.stdout: "in": 0, 2024-09-10T18:55:15.999 INFO:teuthology.orchestra.run.smithi018.stdout: "weight": 0, 2024-09-10T18:55:15.999 INFO:teuthology.orchestra.run.smithi018.stdout: "primary_affinity": 1, 2024-09-10T18:55:15.999 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_begin": 0, 2024-09-10T18:55:15.999 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_end": 0, 2024-09-10T18:55:15.999 INFO:teuthology.orchestra.run.smithi018.stdout: "up_from": 18, 2024-09-10T18:55:16.000 INFO:teuthology.orchestra.run.smithi018.stdout: "up_thru": 86, 2024-09-10T18:55:16.000 INFO:teuthology.orchestra.run.smithi018.stdout: "down_at": 0, 2024-09-10T18:55:16.000 INFO:teuthology.orchestra.run.smithi018.stdout: "lost_at": 0, 2024-09-10T18:55:16.000 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addrs": { 2024-09-10T18:55:16.000 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:55:16.000 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.000 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:55:16.000 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6809", 2024-09-10T18:55:16.000 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 4107623855 2024-09-10T18:55:16.000 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.000 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.000 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.000 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addrs": { 2024-09-10T18:55:16.001 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:55:16.001 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.001 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:55:16.001 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6810", 2024-09-10T18:55:16.001 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 4107623855 2024-09-10T18:55:16.001 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.001 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.001 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.001 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addrs": { 2024-09-10T18:55:16.001 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:55:16.001 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.001 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:55:16.001 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6812", 2024-09-10T18:55:16.001 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 4107623855 2024-09-10T18:55:16.002 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.002 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.002 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.002 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addrs": { 2024-09-10T18:55:16.002 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:55:16.002 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.002 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:55:16.002 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6811", 2024-09-10T18:55:16.002 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 4107623855 2024-09-10T18:55:16.002 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.002 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.002 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.002 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addr": "172.21.15.18:6809/4107623855", 2024-09-10T18:55:16.003 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addr": "172.21.15.18:6810/4107623855", 2024-09-10T18:55:16.003 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addr": "172.21.15.18:6812/4107623855", 2024-09-10T18:55:16.003 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addr": "172.21.15.18:6811/4107623855", 2024-09-10T18:55:16.003 INFO:teuthology.orchestra.run.smithi018.stdout: "state": [ 2024-09-10T18:55:16.003 INFO:teuthology.orchestra.run.smithi018.stdout: "exists", 2024-09-10T18:55:16.003 INFO:teuthology.orchestra.run.smithi018.stdout: "up" 2024-09-10T18:55:16.003 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.003 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.003 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.003 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 3, 2024-09-10T18:55:16.003 INFO:teuthology.orchestra.run.smithi018.stdout: "uuid": "1809e677-b98c-4924-982e-9297e37563af", 2024-09-10T18:55:16.003 INFO:teuthology.orchestra.run.smithi018.stdout: "up": 1, 2024-09-10T18:55:16.003 INFO:teuthology.orchestra.run.smithi018.stdout: "in": 1, 2024-09-10T18:55:16.004 INFO:teuthology.orchestra.run.smithi018.stdout: "weight": 1, 2024-09-10T18:55:16.004 INFO:teuthology.orchestra.run.smithi018.stdout: "primary_affinity": 1, 2024-09-10T18:55:16.004 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_begin": 0, 2024-09-10T18:55:16.004 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_end": 0, 2024-09-10T18:55:16.004 INFO:teuthology.orchestra.run.smithi018.stdout: "up_from": 25, 2024-09-10T18:55:16.004 INFO:teuthology.orchestra.run.smithi018.stdout: "up_thru": 209, 2024-09-10T18:55:16.004 INFO:teuthology.orchestra.run.smithi018.stdout: "down_at": 0, 2024-09-10T18:55:16.004 INFO:teuthology.orchestra.run.smithi018.stdout: "lost_at": 0, 2024-09-10T18:55:16.004 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addrs": { 2024-09-10T18:55:16.004 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:55:16.004 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.004 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:55:16.004 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6813", 2024-09-10T18:55:16.004 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 2732204253 2024-09-10T18:55:16.005 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.005 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.005 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.005 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addrs": { 2024-09-10T18:55:16.005 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:55:16.005 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.005 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:55:16.005 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6814", 2024-09-10T18:55:16.005 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 2732204253 2024-09-10T18:55:16.005 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.005 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.005 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.005 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addrs": { 2024-09-10T18:55:16.006 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:55:16.006 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.006 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:55:16.006 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6816", 2024-09-10T18:55:16.006 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 2732204253 2024-09-10T18:55:16.006 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.006 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.006 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.006 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addrs": { 2024-09-10T18:55:16.006 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:55:16.006 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.006 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:55:16.006 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6815", 2024-09-10T18:55:16.007 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 2732204253 2024-09-10T18:55:16.007 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.007 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.007 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.007 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addr": "172.21.15.18:6813/2732204253", 2024-09-10T18:55:16.007 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addr": "172.21.15.18:6814/2732204253", 2024-09-10T18:55:16.007 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addr": "172.21.15.18:6816/2732204253", 2024-09-10T18:55:16.007 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addr": "172.21.15.18:6815/2732204253", 2024-09-10T18:55:16.007 INFO:teuthology.orchestra.run.smithi018.stdout: "state": [ 2024-09-10T18:55:16.007 INFO:teuthology.orchestra.run.smithi018.stdout: "exists", 2024-09-10T18:55:16.007 INFO:teuthology.orchestra.run.smithi018.stdout: "up" 2024-09-10T18:55:16.007 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.007 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.007 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.008 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 4, 2024-09-10T18:55:16.008 INFO:teuthology.orchestra.run.smithi018.stdout: "uuid": "b682a300-bb53-4f36-8a9f-0753164ec4b7", 2024-09-10T18:55:16.008 INFO:teuthology.orchestra.run.smithi018.stdout: "up": 1, 2024-09-10T18:55:16.008 INFO:teuthology.orchestra.run.smithi018.stdout: "in": 1, 2024-09-10T18:55:16.008 INFO:teuthology.orchestra.run.smithi018.stdout: "weight": 1, 2024-09-10T18:55:16.008 INFO:teuthology.orchestra.run.smithi018.stdout: "primary_affinity": 1, 2024-09-10T18:55:16.008 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_begin": 0, 2024-09-10T18:55:16.008 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_end": 0, 2024-09-10T18:55:16.008 INFO:teuthology.orchestra.run.smithi018.stdout: "up_from": 30, 2024-09-10T18:55:16.008 INFO:teuthology.orchestra.run.smithi018.stdout: "up_thru": 206, 2024-09-10T18:55:16.008 INFO:teuthology.orchestra.run.smithi018.stdout: "down_at": 0, 2024-09-10T18:55:16.008 INFO:teuthology.orchestra.run.smithi018.stdout: "lost_at": 0, 2024-09-10T18:55:16.008 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addrs": { 2024-09-10T18:55:16.009 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:55:16.009 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.009 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:55:16.009 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6800", 2024-09-10T18:55:16.009 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 465860081 2024-09-10T18:55:16.009 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.009 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.010 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.010 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addrs": { 2024-09-10T18:55:16.010 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:55:16.010 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.010 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:55:16.010 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6801", 2024-09-10T18:55:16.010 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 465860081 2024-09-10T18:55:16.010 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.010 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.010 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.010 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addrs": { 2024-09-10T18:55:16.010 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:55:16.010 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.011 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:55:16.011 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6803", 2024-09-10T18:55:16.011 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 465860081 2024-09-10T18:55:16.011 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.011 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.011 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.011 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addrs": { 2024-09-10T18:55:16.011 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:55:16.011 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.011 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:55:16.011 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6802", 2024-09-10T18:55:16.011 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 465860081 2024-09-10T18:55:16.012 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.012 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.012 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.012 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addr": "172.21.15.103:6800/465860081", 2024-09-10T18:55:16.012 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addr": "172.21.15.103:6801/465860081", 2024-09-10T18:55:16.012 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addr": "172.21.15.103:6803/465860081", 2024-09-10T18:55:16.012 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addr": "172.21.15.103:6802/465860081", 2024-09-10T18:55:16.012 INFO:teuthology.orchestra.run.smithi018.stdout: "state": [ 2024-09-10T18:55:16.012 INFO:teuthology.orchestra.run.smithi018.stdout: "exists", 2024-09-10T18:55:16.012 INFO:teuthology.orchestra.run.smithi018.stdout: "up" 2024-09-10T18:55:16.012 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.012 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.012 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.013 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 5, 2024-09-10T18:55:16.013 INFO:teuthology.orchestra.run.smithi018.stdout: "uuid": "2bbfdb6c-0709-4fe4-8a4a-006f576f18e6", 2024-09-10T18:55:16.013 INFO:teuthology.orchestra.run.smithi018.stdout: "up": 1, 2024-09-10T18:55:16.013 INFO:teuthology.orchestra.run.smithi018.stdout: "in": 1, 2024-09-10T18:55:16.013 INFO:teuthology.orchestra.run.smithi018.stdout: "weight": 0.6995086669921875, 2024-09-10T18:55:16.013 INFO:teuthology.orchestra.run.smithi018.stdout: "primary_affinity": 1, 2024-09-10T18:55:16.013 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_begin": 0, 2024-09-10T18:55:16.013 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_end": 0, 2024-09-10T18:55:16.013 INFO:teuthology.orchestra.run.smithi018.stdout: "up_from": 37, 2024-09-10T18:55:16.013 INFO:teuthology.orchestra.run.smithi018.stdout: "up_thru": 206, 2024-09-10T18:55:16.013 INFO:teuthology.orchestra.run.smithi018.stdout: "down_at": 0, 2024-09-10T18:55:16.013 INFO:teuthology.orchestra.run.smithi018.stdout: "lost_at": 0, 2024-09-10T18:55:16.013 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addrs": { 2024-09-10T18:55:16.013 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:55:16.014 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.014 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:55:16.014 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6804", 2024-09-10T18:55:16.014 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3498719258 2024-09-10T18:55:16.014 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.014 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.014 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.014 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addrs": { 2024-09-10T18:55:16.014 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:55:16.014 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.014 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:55:16.014 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6805", 2024-09-10T18:55:16.014 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3498719258 2024-09-10T18:55:16.014 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.015 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.015 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.015 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addrs": { 2024-09-10T18:55:16.015 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:55:16.015 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.015 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:55:16.015 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6807", 2024-09-10T18:55:16.015 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3498719258 2024-09-10T18:55:16.015 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.015 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.015 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.015 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addrs": { 2024-09-10T18:55:16.015 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:55:16.015 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.015 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:55:16.016 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6806", 2024-09-10T18:55:16.016 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3498719258 2024-09-10T18:55:16.016 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.016 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.016 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.016 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addr": "172.21.15.103:6804/3498719258", 2024-09-10T18:55:16.016 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addr": "172.21.15.103:6805/3498719258", 2024-09-10T18:55:16.016 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addr": "172.21.15.103:6807/3498719258", 2024-09-10T18:55:16.016 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addr": "172.21.15.103:6806/3498719258", 2024-09-10T18:55:16.016 INFO:teuthology.orchestra.run.smithi018.stdout: "state": [ 2024-09-10T18:55:16.016 INFO:teuthology.orchestra.run.smithi018.stdout: "exists", 2024-09-10T18:55:16.016 INFO:teuthology.orchestra.run.smithi018.stdout: "up" 2024-09-10T18:55:16.016 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.016 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.017 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.017 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 6, 2024-09-10T18:55:16.017 INFO:teuthology.orchestra.run.smithi018.stdout: "uuid": "31f9711e-0ba8-4aeb-b852-7d93e8fcce33", 2024-09-10T18:55:16.017 INFO:teuthology.orchestra.run.smithi018.stdout: "up": 1, 2024-09-10T18:55:16.017 INFO:teuthology.orchestra.run.smithi018.stdout: "in": 1, 2024-09-10T18:55:16.017 INFO:teuthology.orchestra.run.smithi018.stdout: "weight": 1, 2024-09-10T18:55:16.017 INFO:teuthology.orchestra.run.smithi018.stdout: "primary_affinity": 1, 2024-09-10T18:55:16.017 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_begin": 0, 2024-09-10T18:55:16.017 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_end": 0, 2024-09-10T18:55:16.017 INFO:teuthology.orchestra.run.smithi018.stdout: "up_from": 43, 2024-09-10T18:55:16.017 INFO:teuthology.orchestra.run.smithi018.stdout: "up_thru": 218, 2024-09-10T18:55:16.017 INFO:teuthology.orchestra.run.smithi018.stdout: "down_at": 0, 2024-09-10T18:55:16.017 INFO:teuthology.orchestra.run.smithi018.stdout: "lost_at": 0, 2024-09-10T18:55:16.017 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addrs": { 2024-09-10T18:55:16.018 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:55:16.018 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.018 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:55:16.018 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6808", 2024-09-10T18:55:16.018 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 184353788 2024-09-10T18:55:16.018 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.018 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.018 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.018 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addrs": { 2024-09-10T18:55:16.018 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:55:16.018 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.018 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:55:16.018 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6809", 2024-09-10T18:55:16.018 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 184353788 2024-09-10T18:55:16.018 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.019 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.019 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.019 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addrs": { 2024-09-10T18:55:16.019 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:55:16.019 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.019 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:55:16.019 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6811", 2024-09-10T18:55:16.019 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 184353788 2024-09-10T18:55:16.019 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.019 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.019 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.019 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addrs": { 2024-09-10T18:55:16.019 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:55:16.019 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.019 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:55:16.020 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6810", 2024-09-10T18:55:16.020 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 184353788 2024-09-10T18:55:16.020 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.020 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.020 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.020 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addr": "172.21.15.103:6808/184353788", 2024-09-10T18:55:16.020 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addr": "172.21.15.103:6809/184353788", 2024-09-10T18:55:16.020 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addr": "172.21.15.103:6811/184353788", 2024-09-10T18:55:16.020 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addr": "172.21.15.103:6810/184353788", 2024-09-10T18:55:16.020 INFO:teuthology.orchestra.run.smithi018.stdout: "state": [ 2024-09-10T18:55:16.020 INFO:teuthology.orchestra.run.smithi018.stdout: "exists", 2024-09-10T18:55:16.020 INFO:teuthology.orchestra.run.smithi018.stdout: "up" 2024-09-10T18:55:16.020 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.020 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.021 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.021 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 7, 2024-09-10T18:55:16.021 INFO:teuthology.orchestra.run.smithi018.stdout: "uuid": "f12282f8-4747-45ff-8336-95440560a88f", 2024-09-10T18:55:16.021 INFO:teuthology.orchestra.run.smithi018.stdout: "up": 1, 2024-09-10T18:55:16.021 INFO:teuthology.orchestra.run.smithi018.stdout: "in": 1, 2024-09-10T18:55:16.021 INFO:teuthology.orchestra.run.smithi018.stdout: "weight": 1, 2024-09-10T18:55:16.021 INFO:teuthology.orchestra.run.smithi018.stdout: "primary_affinity": 1, 2024-09-10T18:55:16.021 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_begin": 0, 2024-09-10T18:55:16.021 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_end": 0, 2024-09-10T18:55:16.021 INFO:teuthology.orchestra.run.smithi018.stdout: "up_from": 52, 2024-09-10T18:55:16.021 INFO:teuthology.orchestra.run.smithi018.stdout: "up_thru": 206, 2024-09-10T18:55:16.021 INFO:teuthology.orchestra.run.smithi018.stdout: "down_at": 0, 2024-09-10T18:55:16.022 INFO:teuthology.orchestra.run.smithi018.stdout: "lost_at": 0, 2024-09-10T18:55:16.022 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addrs": { 2024-09-10T18:55:16.022 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:55:16.022 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.022 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:55:16.022 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6812", 2024-09-10T18:55:16.022 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 941024403 2024-09-10T18:55:16.022 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.022 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.022 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.022 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addrs": { 2024-09-10T18:55:16.022 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:55:16.023 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.023 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:55:16.023 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6813", 2024-09-10T18:55:16.023 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 941024403 2024-09-10T18:55:16.023 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.023 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.023 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.024 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addrs": { 2024-09-10T18:55:16.024 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:55:16.024 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.024 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:55:16.024 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6815", 2024-09-10T18:55:16.024 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 941024403 2024-09-10T18:55:16.024 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.024 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.024 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.024 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addrs": { 2024-09-10T18:55:16.024 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:55:16.025 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.025 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:55:16.025 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6814", 2024-09-10T18:55:16.025 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 941024403 2024-09-10T18:55:16.025 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.025 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.025 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.025 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addr": "172.21.15.103:6812/941024403", 2024-09-10T18:55:16.025 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addr": "172.21.15.103:6813/941024403", 2024-09-10T18:55:16.025 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addr": "172.21.15.103:6815/941024403", 2024-09-10T18:55:16.025 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addr": "172.21.15.103:6814/941024403", 2024-09-10T18:55:16.025 INFO:teuthology.orchestra.run.smithi018.stdout: "state": [ 2024-09-10T18:55:16.025 INFO:teuthology.orchestra.run.smithi018.stdout: "exists", 2024-09-10T18:55:16.025 INFO:teuthology.orchestra.run.smithi018.stdout: "up" 2024-09-10T18:55:16.025 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.026 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.026 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-09-10T18:55:16.026 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_xinfo": [ 2024-09-10T18:55:16.026 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.026 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 0, 2024-09-10T18:55:16.026 INFO:teuthology.orchestra.run.smithi018.stdout: "down_stamp": "0.000000", 2024-09-10T18:55:16.026 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_probability": 0, 2024-09-10T18:55:16.026 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_interval": 0, 2024-09-10T18:55:16.026 INFO:teuthology.orchestra.run.smithi018.stdout: "features": 4540138320759226367, 2024-09-10T18:55:16.026 INFO:teuthology.orchestra.run.smithi018.stdout: "old_weight": 0, 2024-09-10T18:55:16.026 INFO:teuthology.orchestra.run.smithi018.stdout: "last_purged_snaps_scrub": "2024-09-10T18:52:46.675816+0000", 2024-09-10T18:55:16.026 INFO:teuthology.orchestra.run.smithi018.stdout: "dead_epoch": 205 2024-09-10T18:55:16.026 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.026 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.027 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 1, 2024-09-10T18:55:16.027 INFO:teuthology.orchestra.run.smithi018.stdout: "down_stamp": "0.000000", 2024-09-10T18:55:16.027 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_probability": 0, 2024-09-10T18:55:16.027 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_interval": 0, 2024-09-10T18:55:16.027 INFO:teuthology.orchestra.run.smithi018.stdout: "features": 4540138320759226367, 2024-09-10T18:55:16.027 INFO:teuthology.orchestra.run.smithi018.stdout: "old_weight": 0, 2024-09-10T18:55:16.027 INFO:teuthology.orchestra.run.smithi018.stdout: "last_purged_snaps_scrub": "2024-09-10T18:54:18.517863+0000", 2024-09-10T18:55:16.027 INFO:teuthology.orchestra.run.smithi018.stdout: "dead_epoch": 0 2024-09-10T18:55:16.027 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.027 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.027 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 2, 2024-09-10T18:55:16.027 INFO:teuthology.orchestra.run.smithi018.stdout: "down_stamp": "0.000000", 2024-09-10T18:55:16.027 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_probability": 0, 2024-09-10T18:55:16.027 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_interval": 0, 2024-09-10T18:55:16.027 INFO:teuthology.orchestra.run.smithi018.stdout: "features": 4540138320759226367, 2024-09-10T18:55:16.028 INFO:teuthology.orchestra.run.smithi018.stdout: "old_weight": 65536, 2024-09-10T18:55:16.028 INFO:teuthology.orchestra.run.smithi018.stdout: "last_purged_snaps_scrub": "2024-09-10T18:54:35.797120+0000", 2024-09-10T18:55:16.028 INFO:teuthology.orchestra.run.smithi018.stdout: "dead_epoch": 0 2024-09-10T18:55:16.028 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.028 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.028 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 3, 2024-09-10T18:55:16.028 INFO:teuthology.orchestra.run.smithi018.stdout: "down_stamp": "0.000000", 2024-09-10T18:55:16.028 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_probability": 0, 2024-09-10T18:55:16.028 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_interval": 0, 2024-09-10T18:55:16.028 INFO:teuthology.orchestra.run.smithi018.stdout: "features": 4540138320759226367, 2024-09-10T18:55:16.028 INFO:teuthology.orchestra.run.smithi018.stdout: "old_weight": 0, 2024-09-10T18:55:16.028 INFO:teuthology.orchestra.run.smithi018.stdout: "last_purged_snaps_scrub": "2024-09-10T18:54:30.224222+0000", 2024-09-10T18:55:16.028 INFO:teuthology.orchestra.run.smithi018.stdout: "dead_epoch": 0 2024-09-10T18:55:16.029 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.029 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.029 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 4, 2024-09-10T18:55:16.029 INFO:teuthology.orchestra.run.smithi018.stdout: "down_stamp": "0.000000", 2024-09-10T18:55:16.029 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_probability": 0, 2024-09-10T18:55:16.029 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_interval": 0, 2024-09-10T18:55:16.029 INFO:teuthology.orchestra.run.smithi018.stdout: "features": 4540138320759226367, 2024-09-10T18:55:16.029 INFO:teuthology.orchestra.run.smithi018.stdout: "old_weight": 0, 2024-09-10T18:55:16.029 INFO:teuthology.orchestra.run.smithi018.stdout: "last_purged_snaps_scrub": "2024-09-10T18:54:44.490920+0000", 2024-09-10T18:55:16.029 INFO:teuthology.orchestra.run.smithi018.stdout: "dead_epoch": 0 2024-09-10T18:55:16.029 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.029 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.029 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 5, 2024-09-10T18:55:16.030 INFO:teuthology.orchestra.run.smithi018.stdout: "down_stamp": "0.000000", 2024-09-10T18:55:16.030 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_probability": 0, 2024-09-10T18:55:16.030 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_interval": 0, 2024-09-10T18:55:16.030 INFO:teuthology.orchestra.run.smithi018.stdout: "features": 4540138320759226367, 2024-09-10T18:55:16.030 INFO:teuthology.orchestra.run.smithi018.stdout: "old_weight": 0, 2024-09-10T18:55:16.030 INFO:teuthology.orchestra.run.smithi018.stdout: "last_purged_snaps_scrub": "2024-09-10T18:54:33.148741+0000", 2024-09-10T18:55:16.030 INFO:teuthology.orchestra.run.smithi018.stdout: "dead_epoch": 0 2024-09-10T18:55:16.030 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.030 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.030 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 6, 2024-09-10T18:55:16.030 INFO:teuthology.orchestra.run.smithi018.stdout: "down_stamp": "0.000000", 2024-09-10T18:55:16.030 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_probability": 0, 2024-09-10T18:55:16.030 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_interval": 0, 2024-09-10T18:55:16.031 INFO:teuthology.orchestra.run.smithi018.stdout: "features": 4540138320759226367, 2024-09-10T18:55:16.031 INFO:teuthology.orchestra.run.smithi018.stdout: "old_weight": 0, 2024-09-10T18:55:16.031 INFO:teuthology.orchestra.run.smithi018.stdout: "last_purged_snaps_scrub": "2024-09-10T18:54:59.166172+0000", 2024-09-10T18:55:16.031 INFO:teuthology.orchestra.run.smithi018.stdout: "dead_epoch": 0 2024-09-10T18:55:16.031 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.031 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.031 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 7, 2024-09-10T18:55:16.031 INFO:teuthology.orchestra.run.smithi018.stdout: "down_stamp": "0.000000", 2024-09-10T18:55:16.031 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_probability": 0, 2024-09-10T18:55:16.031 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_interval": 0, 2024-09-10T18:55:16.031 INFO:teuthology.orchestra.run.smithi018.stdout: "features": 4540138320759226367, 2024-09-10T18:55:16.031 INFO:teuthology.orchestra.run.smithi018.stdout: "old_weight": 0, 2024-09-10T18:55:16.031 INFO:teuthology.orchestra.run.smithi018.stdout: "last_purged_snaps_scrub": "2024-09-10T18:54:51.994817+0000", 2024-09-10T18:55:16.032 INFO:teuthology.orchestra.run.smithi018.stdout: "dead_epoch": 0 2024-09-10T18:55:16.032 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.032 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-09-10T18:55:16.032 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_upmap": [], 2024-09-10T18:55:16.032 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_upmap_items": [ 2024-09-10T18:55:16.032 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.032 INFO:teuthology.orchestra.run.smithi018.stdout: "pgid": "2.4", 2024-09-10T18:55:16.032 INFO:teuthology.orchestra.run.smithi018.stdout: "mappings": [ 2024-09-10T18:55:16.032 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.032 INFO:teuthology.orchestra.run.smithi018.stdout: "from": 7, 2024-09-10T18:55:16.032 INFO:teuthology.orchestra.run.smithi018.stdout: "to": 4 2024-09-10T18:55:16.032 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.032 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.033 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.033 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.033 INFO:teuthology.orchestra.run.smithi018.stdout: "pgid": "4.e", 2024-09-10T18:55:16.033 INFO:teuthology.orchestra.run.smithi018.stdout: "mappings": [ 2024-09-10T18:55:16.033 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.033 INFO:teuthology.orchestra.run.smithi018.stdout: "from": 1, 2024-09-10T18:55:16.033 INFO:teuthology.orchestra.run.smithi018.stdout: "to": 5 2024-09-10T18:55:16.033 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.033 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.033 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.033 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.033 INFO:teuthology.orchestra.run.smithi018.stdout: "pgid": "4.f", 2024-09-10T18:55:16.033 INFO:teuthology.orchestra.run.smithi018.stdout: "mappings": [ 2024-09-10T18:55:16.034 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.034 INFO:teuthology.orchestra.run.smithi018.stdout: "from": 1, 2024-09-10T18:55:16.034 INFO:teuthology.orchestra.run.smithi018.stdout: "to": 5 2024-09-10T18:55:16.034 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.034 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.034 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.034 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.034 INFO:teuthology.orchestra.run.smithi018.stdout: "pgid": "5.5", 2024-09-10T18:55:16.034 INFO:teuthology.orchestra.run.smithi018.stdout: "mappings": [ 2024-09-10T18:55:16.034 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.034 INFO:teuthology.orchestra.run.smithi018.stdout: "from": 0, 2024-09-10T18:55:16.034 INFO:teuthology.orchestra.run.smithi018.stdout: "to": 6 2024-09-10T18:55:16.034 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.034 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.034 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.035 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.035 INFO:teuthology.orchestra.run.smithi018.stdout: "pgid": "5.10", 2024-09-10T18:55:16.035 INFO:teuthology.orchestra.run.smithi018.stdout: "mappings": [ 2024-09-10T18:55:16.035 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.035 INFO:teuthology.orchestra.run.smithi018.stdout: "from": 7, 2024-09-10T18:55:16.035 INFO:teuthology.orchestra.run.smithi018.stdout: "to": 3 2024-09-10T18:55:16.035 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.035 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.035 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.035 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.035 INFO:teuthology.orchestra.run.smithi018.stdout: "pgid": "5.11", 2024-09-10T18:55:16.035 INFO:teuthology.orchestra.run.smithi018.stdout: "mappings": [ 2024-09-10T18:55:16.035 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.035 INFO:teuthology.orchestra.run.smithi018.stdout: "from": 4, 2024-09-10T18:55:16.036 INFO:teuthology.orchestra.run.smithi018.stdout: "to": 3 2024-09-10T18:55:16.036 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.036 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.036 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.036 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.036 INFO:teuthology.orchestra.run.smithi018.stdout: "pgid": "5.19", 2024-09-10T18:55:16.036 INFO:teuthology.orchestra.run.smithi018.stdout: "mappings": [ 2024-09-10T18:55:16.036 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.036 INFO:teuthology.orchestra.run.smithi018.stdout: "from": 1, 2024-09-10T18:55:16.036 INFO:teuthology.orchestra.run.smithi018.stdout: "to": 6 2024-09-10T18:55:16.036 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.036 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.036 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.037 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.037 INFO:teuthology.orchestra.run.smithi018.stdout: "pgid": "6.f", 2024-09-10T18:55:16.037 INFO:teuthology.orchestra.run.smithi018.stdout: "mappings": [ 2024-09-10T18:55:16.037 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.037 INFO:teuthology.orchestra.run.smithi018.stdout: "from": 3, 2024-09-10T18:55:16.037 INFO:teuthology.orchestra.run.smithi018.stdout: "to": 0 2024-09-10T18:55:16.037 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.037 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.037 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.037 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-09-10T18:55:16.037 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_temp": [ 2024-09-10T18:55:16.037 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:55:16.037 INFO:teuthology.orchestra.run.smithi018.stdout: "pgid": "8.9", 2024-09-10T18:55:16.037 INFO:teuthology.orchestra.run.smithi018.stdout: "osds": [ 2024-09-10T18:55:16.038 INFO:teuthology.orchestra.run.smithi018.stdout: 6, 2024-09-10T18:55:16.038 INFO:teuthology.orchestra.run.smithi018.stdout: 3, 2024-09-10T18:55:16.038 INFO:teuthology.orchestra.run.smithi018.stdout: 5 2024-09-10T18:55:16.038 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:55:16.038 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.038 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-09-10T18:55:16.038 INFO:teuthology.orchestra.run.smithi018.stdout: "primary_temp": [], 2024-09-10T18:55:16.038 INFO:teuthology.orchestra.run.smithi018.stdout: "blocklist": { 2024-09-10T18:55:16.038 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/3021537013": "2024-09-11T18:45:54.998616+0000", 2024-09-10T18:55:16.038 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/458572898": "2024-09-11T18:40:28.880042+0000", 2024-09-10T18:55:16.038 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/1859541419": "2024-09-11T18:40:28.880042+0000", 2024-09-10T18:55:16.038 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/3090642970": "2024-09-11T18:40:59.659411+0000", 2024-09-10T18:55:16.038 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/2875894312": "2024-09-11T18:40:59.659411+0000", 2024-09-10T18:55:16.039 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/178638059": "2024-09-11T18:40:59.659411+0000", 2024-09-10T18:55:16.039 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/692931402": "2024-09-11T18:45:54.998616+0000", 2024-09-10T18:55:16.039 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/2408360238": "2024-09-11T18:45:54.998616+0000", 2024-09-10T18:55:16.039 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/57192865": "2024-09-11T18:40:28.880042+0000", 2024-09-10T18:55:16.039 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:6800/2748756682": "2024-09-11T18:40:59.659411+0000", 2024-09-10T18:55:16.039 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:6800/732375116": "2024-09-11T18:45:54.998616+0000", 2024-09-10T18:55:16.039 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:6800/3869801150": "2024-09-11T18:40:28.880042+0000", 2024-09-10T18:55:16.039 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/2982361259": "2024-09-11T18:45:54.998616+0000", 2024-09-10T18:55:16.039 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/185700038": "2024-09-11T18:45:54.998616+0000" 2024-09-10T18:55:16.039 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.039 INFO:teuthology.orchestra.run.smithi018.stdout: "range_blocklist": {}, 2024-09-10T18:55:16.039 INFO:teuthology.orchestra.run.smithi018.stdout: "erasure_code_profiles": { 2024-09-10T18:55:16.039 INFO:teuthology.orchestra.run.smithi018.stdout: "default": { 2024-09-10T18:55:16.039 INFO:teuthology.orchestra.run.smithi018.stdout: "crush-failure-domain": "osd", 2024-09-10T18:55:16.040 INFO:teuthology.orchestra.run.smithi018.stdout: "k": "2", 2024-09-10T18:55:16.040 INFO:teuthology.orchestra.run.smithi018.stdout: "m": "1", 2024-09-10T18:55:16.040 INFO:teuthology.orchestra.run.smithi018.stdout: "plugin": "jerasure", 2024-09-10T18:55:16.040 INFO:teuthology.orchestra.run.smithi018.stdout: "technique": "reed_sol_van" 2024-09-10T18:55:16.040 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.040 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:55:16.040 INFO:teuthology.orchestra.run.smithi018.stdout: "removed_snaps_queue": [], 2024-09-10T18:55:16.040 INFO:teuthology.orchestra.run.smithi018.stdout: "new_removed_snaps": [], 2024-09-10T18:55:16.040 INFO:teuthology.orchestra.run.smithi018.stdout: "new_purged_snaps": [], 2024-09-10T18:55:16.040 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_node_flags": {}, 2024-09-10T18:55:16.040 INFO:teuthology.orchestra.run.smithi018.stdout: "device_class_flags": {}, 2024-09-10T18:55:16.040 INFO:teuthology.orchestra.run.smithi018.stdout: "stretch_mode": { 2024-09-10T18:55:16.040 INFO:teuthology.orchestra.run.smithi018.stdout: "stretch_mode_enabled": false, 2024-09-10T18:55:16.041 INFO:teuthology.orchestra.run.smithi018.stdout: "stretch_bucket_count": 0, 2024-09-10T18:55:16.041 INFO:teuthology.orchestra.run.smithi018.stdout: "degraded_stretch_mode": 0, 2024-09-10T18:55:16.041 INFO:teuthology.orchestra.run.smithi018.stdout: "recovering_stretch_mode": 0, 2024-09-10T18:55:16.041 INFO:teuthology.orchestra.run.smithi018.stdout: "stretch_mode_bucket": 0 2024-09-10T18:55:16.041 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:55:16.041 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T18:55:16.041 INFO:teuthology.orchestra.run.smithi018.stdout:ERROR: (22) Invalid argument 2024-09-10T18:55:16.041 INFO:teuthology.orchestra.run.smithi018.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:55:16.042 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:16.063 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:16.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:16.064 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:16.165 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:16.186 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:16.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:16.187 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:16.288 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:16.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:16.310 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:16.310 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:16.411 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:16.433 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:16.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:16.434 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:16.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:16 smithi018 ceph-mon[29336]: pgmap v443: 168 pgs: 1 active+clean+remapped, 43 active+undersized, 25 active+undersized+degraded, 99 active+clean; 468 KiB data, 5.1 GiB used, 621 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.48k op/s; 79/591 objects degraded (13.367%) 2024-09-10T18:55:16.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:16 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1120066895' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-10T18:55:16.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:16 smithi018 ceph-mon[40196]: pgmap v443: 168 pgs: 1 active+clean+remapped, 43 active+undersized, 25 active+undersized+degraded, 99 active+clean; 468 KiB data, 5.1 GiB used, 621 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.48k op/s; 79/591 objects degraded (13.367%) 2024-09-10T18:55:16.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:16 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1120066895' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-10T18:55:16.535 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:16.557 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:16.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:16.558 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:16.600 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_1 2024-09-10T18:55:16.600 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd pool rm unique_pool_1 unique_pool_1 --yes-i-really-really-mean-it 2024-09-10T18:55:16.609 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:16 smithi103 ceph-mon[33163]: pgmap v443: 168 pgs: 1 active+clean+remapped, 43 active+undersized, 25 active+undersized+degraded, 99 active+clean; 468 KiB data, 5.1 GiB used, 621 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.48k op/s; 79/591 objects degraded (13.367%) 2024-09-10T18:55:16.609 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:16 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1120066895' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-10T18:55:16.659 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:16.680 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:16.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:16.681 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:16.782 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:16.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:16.804 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:16.804 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:16.905 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:16.927 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:16.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:16.928 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:17.022 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:55:16 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:55:17.029 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:17.051 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:17.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:17.052 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:17.153 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:17.175 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:17.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:17.176 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:17.277 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:17.299 INFO:tasks.thrashosds.thrasher:j is {'epoch': 230, 'fsid': 'ba327f6c-6fa3-11ef-bcea-c7b262605968', 'created': '2024-09-10T18:40:07.669687+0000', 'modified': '2024-09-10T18:55:08.601033+0000', 'last_up_change': '2024-09-10T18:54:00.270083+0000', 'last_in_change': '2024-09-10T18:47:35.385915+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 8, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T18:43:08.162659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 21, 'pg_placement_num': 21, 'pg_placement_num_target': 21, 'pg_num_target': 21, 'pg_num_pending': 21, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '169', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '167', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': '.rgw.root', 'create_time': '2024-09-10T18:45:16.512219+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 3, 'pool_name': 'datapool', 'create_time': '2024-09-10T18:45:16.943626+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 60, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-10T18:45:18.450026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-10T18:45:20.494618+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-10T18:45:22.566992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}}, {'pool': 8, 'pool_name': 'unique_pool_1', 'create_time': '2024-09-10T18:51:36.493636+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '202', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': 'eb4c7e67-300c-4a1a-b9f5-bb028eeeabc2', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 193, 'down_at': 206, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6801', 'nonce': 2131279292}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6802', 'nonce': 2131279292}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6804', 'nonce': 2131279292}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6803', 'nonce': 2131279292}]}, 'public_addr': '172.21.15.18:6801/2131279292', 'cluster_addr': '172.21.15.18:6802/2131279292', 'heartbeat_back_addr': '172.21.15.18:6804/2131279292', 'heartbeat_front_addr': '172.21.15.18:6803/2131279292', 'state': ['exists']}, {'osd': 1, 'uuid': '63bf6a98-e349-4cc0-9502-a31693070c5b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 222, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6805', 'nonce': 3855640936}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6806', 'nonce': 3855640936}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6808', 'nonce': 3855640936}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6807', 'nonce': 3855640936}]}, 'public_addr': '172.21.15.18:6805/3855640936', 'cluster_addr': '172.21.15.18:6806/3855640936', 'heartbeat_back_addr': '172.21.15.18:6808/3855640936', 'heartbeat_front_addr': '172.21.15.18:6807/3855640936', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '79a01c29-4f4d-43d1-b1ea-a3fcf014f84b', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 86, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6809', 'nonce': 4107623855}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6810', 'nonce': 4107623855}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6812', 'nonce': 4107623855}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6811', 'nonce': 4107623855}]}, 'public_addr': '172.21.15.18:6809/4107623855', 'cluster_addr': '172.21.15.18:6810/4107623855', 'heartbeat_back_addr': '172.21.15.18:6812/4107623855', 'heartbeat_front_addr': '172.21.15.18:6811/4107623855', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '1809e677-b98c-4924-982e-9297e37563af', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 25, 'up_thru': 209, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6813', 'nonce': 2732204253}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6814', 'nonce': 2732204253}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6816', 'nonce': 2732204253}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6815', 'nonce': 2732204253}]}, 'public_addr': '172.21.15.18:6813/2732204253', 'cluster_addr': '172.21.15.18:6814/2732204253', 'heartbeat_back_addr': '172.21.15.18:6816/2732204253', 'heartbeat_front_addr': '172.21.15.18:6815/2732204253', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'b682a300-bb53-4f36-8a9f-0753164ec4b7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 30, 'up_thru': 206, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6800', 'nonce': 465860081}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6801', 'nonce': 465860081}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6803', 'nonce': 465860081}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6802', 'nonce': 465860081}]}, 'public_addr': '172.21.15.103:6800/465860081', 'cluster_addr': '172.21.15.103:6801/465860081', 'heartbeat_back_addr': '172.21.15.103:6803/465860081', 'heartbeat_front_addr': '172.21.15.103:6802/465860081', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '2bbfdb6c-0709-4fe4-8a4a-006f576f18e6', 'up': 1, 'in': 1, 'weight': 0.6995086669921875, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 37, 'up_thru': 206, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6804', 'nonce': 3498719258}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6805', 'nonce': 3498719258}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6807', 'nonce': 3498719258}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6806', 'nonce': 3498719258}]}, 'public_addr': '172.21.15.103:6804/3498719258', 'cluster_addr': '172.21.15.103:6805/3498719258', 'heartbeat_back_addr': '172.21.15.103:6807/3498719258', 'heartbeat_front_addr': '172.21.15.103:6806/3498719258', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '31f9711e-0ba8-4aeb-b852-7d93e8fcce33', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 43, 'up_thru': 218, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6808', 'nonce': 184353788}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6809', 'nonce': 184353788}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6811', 'nonce': 184353788}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6810', 'nonce': 184353788}]}, 'public_addr': '172.21.15.103:6808/184353788', 'cluster_addr': '172.21.15.103:6809/184353788', 'heartbeat_back_addr': '172.21.15.103:6811/184353788', 'heartbeat_front_addr': '172.21.15.103:6810/184353788', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'f12282f8-4747-45ff-8336-95440560a88f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 52, 'up_thru': 206, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6812', 'nonce': 941024403}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6813', 'nonce': 941024403}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6815', 'nonce': 941024403}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6814', 'nonce': 941024403}]}, 'public_addr': '172.21.15.103:6812/941024403', 'cluster_addr': '172.21.15.103:6813/941024403', 'heartbeat_back_addr': '172.21.15.103:6815/941024403', 'heartbeat_front_addr': '172.21.15.103:6814/941024403', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T18:52:46.675816+0000', 'dead_epoch': 205}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T18:54:18.517863+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-10T18:54:35.797120+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T18:54:30.224222+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T18:54:44.490920+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T18:54:33.148741+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T18:54:59.166172+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T18:54:51.994817+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '2.4', 'mappings': [{'from': 7, 'to': 4}]}, {'pgid': '4.e', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.f', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '5.5', 'mappings': [{'from': 0, 'to': 6}]}, {'pgid': '5.10', 'mappings': [{'from': 7, 'to': 3}]}, {'pgid': '5.11', 'mappings': [{'from': 4, 'to': 3}]}, {'pgid': '5.19', 'mappings': [{'from': 1, 'to': 6}]}, {'pgid': '6.f', 'mappings': [{'from': 3, 'to': 0}]}], 'pg_temp': [{'pgid': '8.9', 'osds': [6, 3, 5]}], 'primary_temp': [], 'blocklist': {'172.21.15.18:0/3021537013': '2024-09-11T18:45:54.998616+0000', '172.21.15.18:0/458572898': '2024-09-11T18:40:28.880042+0000', '172.21.15.18:0/1859541419': '2024-09-11T18:40:28.880042+0000', '172.21.15.18:0/3090642970': '2024-09-11T18:40:59.659411+0000', '172.21.15.18:0/2875894312': '2024-09-11T18:40:59.659411+0000', '172.21.15.18:0/178638059': '2024-09-11T18:40:59.659411+0000', '172.21.15.18:0/692931402': '2024-09-11T18:45:54.998616+0000', '172.21.15.18:0/2408360238': '2024-09-11T18:45:54.998616+0000', '172.21.15.18:0/57192865': '2024-09-11T18:40:28.880042+0000', '172.21.15.18:6800/2748756682': '2024-09-11T18:40:59.659411+0000', '172.21.15.18:6800/732375116': '2024-09-11T18:45:54.998616+0000', '172.21.15.18:6800/3869801150': '2024-09-11T18:40:28.880042+0000', '172.21.15.18:0/2982361259': '2024-09-11T18:45:54.998616+0000', '172.21.15.18:0/185700038': '2024-09-11T18:45:54.998616+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-10T18:55:17.299 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-10T18:55:17.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:17.300 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:17.301 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:17.381 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:55:17.382 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:55:17.401 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:17.425 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:17.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:17.429 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:17.530 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:17.552 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:17.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:17.553 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:17.654 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:17.676 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:17.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:17.677 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:17.778 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:17.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:17.801 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:17.801 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:17.845 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-10T18:55:17.855 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:55:17.902 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:17.927 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:17.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:17.929 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:18.029 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:18.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:18.055 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:18.055 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:18.156 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:18.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:18.179 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:18.179 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:18.280 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:18.306 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:18.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:18.307 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:18.363 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:18 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:18.363 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:18 smithi018 ceph-mon[29336]: pgmap v444: 168 pgs: 1 active+clean+remapped, 43 active+undersized, 25 active+undersized+degraded, 99 active+clean; 404 KiB data, 5.0 GiB used, 621 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 722 op/s; 78/588 objects degraded (13.265%) 2024-09-10T18:55:18.363 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:18 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:18.363 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:18 smithi018 ceph-mon[40196]: pgmap v444: 168 pgs: 1 active+clean+remapped, 43 active+undersized, 25 active+undersized+degraded, 99 active+clean; 404 KiB data, 5.0 GiB used, 621 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 722 op/s; 78/588 objects degraded (13.265%) 2024-09-10T18:55:18.408 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:18.433 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:18.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:18.434 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:18.535 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:18.561 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:18.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:18.562 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:18.663 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:18.686 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:18.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:18.687 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:18.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:18 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:18.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:18 smithi103 ceph-mon[33163]: pgmap v444: 168 pgs: 1 active+clean+remapped, 43 active+undersized, 25 active+undersized+degraded, 99 active+clean; 404 KiB data, 5.0 GiB used, 621 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 722 op/s; 78/588 objects degraded (13.265%) 2024-09-10T18:55:18.788 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:18.814 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:18.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:18.815 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:18.915 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:18.942 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:18.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:18.943 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:19.044 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:19.067 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:19.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:19.068 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:19.169 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:19.195 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:19.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:19.196 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:19.297 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:19.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:19.321 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:19.321 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:19.422 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:19.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:19 smithi018 ceph-mon[29336]: Health check update: Degraded data redundancy: 78/588 objects degraded (13.265%), 25 pgs degraded, 68 pgs undersized (PG_DEGRADED) 2024-09-10T18:55:19.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:19 smithi018 ceph-mon[40196]: Health check update: Degraded data redundancy: 78/588 objects degraded (13.265%), 25 pgs degraded, 68 pgs undersized (PG_DEGRADED) 2024-09-10T18:55:19.448 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:19.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:19.449 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:19.550 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:19.572 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:19.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:19.573 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:19.677 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:19.699 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:19.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:19.700 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:19.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:19 smithi103 ceph-mon[33163]: Health check update: Degraded data redundancy: 78/588 objects degraded (13.265%), 25 pgs degraded, 68 pgs undersized (PG_DEGRADED) 2024-09-10T18:55:19.797 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:55:19.801 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:19.827 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:19.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:19.828 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:19.929 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:19.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:19.956 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:19.956 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:20.057 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:20.083 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:20.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:20.084 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:20.184 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:20.208 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:20.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:20.209 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:20.310 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:20.335 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:20.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:20.337 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:20.437 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:20.460 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:20.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:20.462 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:20.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:20 smithi103 ceph-mon[33163]: pgmap v445: 168 pgs: 1 active+clean+remapped, 43 active+undersized, 25 active+undersized+degraded, 99 active+clean; 404 KiB data, 5.0 GiB used, 621 GiB / 626 GiB avail; 964 B/s rd, 0 B/s wr, 681 op/s; 78/588 objects degraded (13.265%) 2024-09-10T18:55:20.523 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:55:20 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:55:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:55:20.562 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:20.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:20.586 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:20.587 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:20.687 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:20.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:20 smithi018 ceph-mon[29336]: pgmap v445: 168 pgs: 1 active+clean+remapped, 43 active+undersized, 25 active+undersized+degraded, 99 active+clean; 404 KiB data, 5.0 GiB used, 621 GiB / 626 GiB avail; 964 B/s rd, 0 B/s wr, 681 op/s; 78/588 objects degraded (13.265%) 2024-09-10T18:55:20.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:20 smithi018 ceph-mon[40196]: pgmap v445: 168 pgs: 1 active+clean+remapped, 43 active+undersized, 25 active+undersized+degraded, 99 active+clean; 404 KiB data, 5.0 GiB used, 621 GiB / 626 GiB avail; 964 B/s rd, 0 B/s wr, 681 op/s; 78/588 objects degraded (13.265%) 2024-09-10T18:55:20.713 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:20.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:20.714 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:20.815 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:20.841 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:20.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:20.842 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:20.943 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:20.967 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:20.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:20.969 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:21.069 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:21.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:21.095 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:21.095 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:21.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:55:20 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:55:20] "GET /metrics HTTP/1.1" 200 224270 "" "Prometheus/2.43.0" 2024-09-10T18:55:21.196 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:21.219 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:21.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:21.220 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:21.321 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:21.345 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:21.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:21.346 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:21.447 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:21.470 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:21.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:21.471 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:21.572 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:21.597 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:21.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:21.598 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:21.699 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:21.722 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:21.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:21.723 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:21.824 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:21.847 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:21.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:21.848 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:21.949 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:21.973 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:21.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:21.974 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:22.075 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:22.098 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:22.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:22.099 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:22.200 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:22.223 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:22.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:22.224 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:22.299 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [0] live_osds: [1, 2, 3, 4, 5, 6, 7] 2024-09-10T18:55:22.300 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T18:55:22.300 INFO:tasks.thrashosds.thrasher:Reviving osd 0 2024-09-10T18:55:22.300 INFO:tasks.cephadm.osd.0:Restarting osd.0 (starting--it wasn't running)... 2024-09-10T18:55:22.300 DEBUG:teuthology.orchestra.run.smithi018:osd.0> sudo journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@osd.0.service 2024-09-10T18:55:22.303 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl start ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@osd.0 2024-09-10T18:55:22.325 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:22.349 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:22.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:22.350 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:22.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:22 smithi018 ceph-mon[29336]: pgmap v446: 168 pgs: 1 active+clean+remapped, 43 active+undersized, 25 active+undersized+degraded, 99 active+clean; 404 KiB data, 5.0 GiB used, 621 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 602 op/s; 78/588 objects degraded (13.265%) 2024-09-10T18:55:22.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:22 smithi018 ceph-mon[40196]: pgmap v446: 168 pgs: 1 active+clean+remapped, 43 active+undersized, 25 active+undersized+degraded, 99 active+clean; 404 KiB data, 5.0 GiB used, 621 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 602 op/s; 78/588 objects degraded (13.265%) 2024-09-10T18:55:22.445 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:55:22 smithi018 systemd[1]: Starting Ceph osd.0 for ba327f6c-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:55:22.451 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:22.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:22.477 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:22.477 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:22.578 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:22.602 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:22.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:22.603 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:22.704 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:22.726 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:22.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:22.728 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:22.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:22 smithi103 ceph-mon[33163]: pgmap v446: 168 pgs: 1 active+clean+remapped, 43 active+undersized, 25 active+undersized+degraded, 99 active+clean; 404 KiB data, 5.0 GiB used, 621 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 602 op/s; 78/588 objects degraded (13.265%) 2024-09-10T18:55:22.829 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:22.855 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:22.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:22.856 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:22.957 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:22.981 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:22.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:22.982 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:23.083 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:23.105 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:23.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:23.106 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:23.207 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:23.677 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is unset 2024-09-10T18:55:23.677 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:23 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3339421120' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:55:23.677 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:23 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3339421120' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:55:23.702 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:23.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:23.703 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:23.745 INFO:teuthology.orchestra.run.smithi018.stdout:ERROR: (22) Invalid argument 2024-09-10T18:55:23.746 INFO:teuthology.orchestra.run.smithi018.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:55:23.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:23 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3339421120' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:55:23.803 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:23.827 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:23.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:23.828 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:23.929 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:23.954 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:23.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:23.955 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:24.056 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:24.080 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:24.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:24.081 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:24.182 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:24.205 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:24.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:24.206 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:24.307 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:24.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:24.332 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:24.332 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:24.433 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:24.456 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:24.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:24.457 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:24.485 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-10T18:55:24.558 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:24.581 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:24.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:24.582 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:24.683 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:24.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:24 smithi018 ceph-mon[29336]: pgmap v447: 168 pgs: 1 active+clean+remapped, 43 active+undersized, 25 active+undersized+degraded, 99 active+clean; 404 KiB data, 5.0 GiB used, 621 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 602 op/s; 78/588 objects degraded (13.265%) 2024-09-10T18:55:24.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:24 smithi018 ceph-mon[29336]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:55:24.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:24 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3339421120' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:55:24.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:24 smithi018 ceph-mon[29336]: osdmap e231: 8 total, 7 up, 7 in 2024-09-10T18:55:24.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:24 smithi018 ceph-mon[29336]: osdmap e232: 8 total, 7 up, 7 in 2024-09-10T18:55:24.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:24 smithi018 ceph-mon[40196]: pgmap v447: 168 pgs: 1 active+clean+remapped, 43 active+undersized, 25 active+undersized+degraded, 99 active+clean; 404 KiB data, 5.0 GiB used, 621 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 602 op/s; 78/588 objects degraded (13.265%) 2024-09-10T18:55:24.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:24 smithi018 ceph-mon[40196]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:55:24.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:24 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3339421120' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:55:24.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:24 smithi018 ceph-mon[40196]: osdmap e231: 8 total, 7 up, 7 in 2024-09-10T18:55:24.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:24 smithi018 ceph-mon[40196]: osdmap e232: 8 total, 7 up, 7 in 2024-09-10T18:55:24.706 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:24.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:24.707 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:24.741 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:55:24.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:24 smithi103 ceph-mon[33163]: pgmap v447: 168 pgs: 1 active+clean+remapped, 43 active+undersized, 25 active+undersized+degraded, 99 active+clean; 404 KiB data, 5.0 GiB used, 621 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 602 op/s; 78/588 objects degraded (13.265%) 2024-09-10T18:55:24.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:24 smithi103 ceph-mon[33163]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:55:24.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:24 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3339421120' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:55:24.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:24 smithi103 ceph-mon[33163]: osdmap e231: 8 total, 7 up, 7 in 2024-09-10T18:55:24.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:24 smithi103 ceph-mon[33163]: osdmap e232: 8 total, 7 up, 7 in 2024-09-10T18:55:24.808 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:24.830 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:24.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:24.831 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:24.932 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:24.955 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:24.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:24.956 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:25.057 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:25.081 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:25.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:25.081 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:25.182 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:25.205 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:25.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:25.206 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:25.306 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:25.307 INFO:teuthology.orchestra.run.smithi018.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-10T18:55:25.325 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:55:25.325 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:55:25.326 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:25 smithi018 ceph-mon[40196]: 8.a scrub starts 2024-09-10T18:55:25.326 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:25 smithi018 ceph-mon[40196]: 8.a scrub ok 2024-09-10T18:55:25.326 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:25 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:25.326 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:25 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/759250631' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:55:25.326 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:25 smithi018 ceph-mon[29336]: 8.a scrub starts 2024-09-10T18:55:25.326 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:25 smithi018 ceph-mon[29336]: 8.a scrub ok 2024-09-10T18:55:25.326 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:25 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:25.326 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:25 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/759250631' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:55:25.328 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:25.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:25.329 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:25.344 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:55:25.344 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:55:25.360 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:55:25.360 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:55:25.376 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:55:25.376 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:55:25.392 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:55:25.392 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:55:25.396 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:25 smithi103 ceph-mon[33163]: 8.a scrub starts 2024-09-10T18:55:25.397 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:25 smithi103 ceph-mon[33163]: 8.a scrub ok 2024-09-10T18:55:25.397 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:25 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:25.401 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:25 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/759250631' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:55:25.408 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:55:25.408 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:55:25.423 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:55:25.423 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:55:25.430 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:25.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:25.458 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:25.458 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:25.558 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:25.580 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:25.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:25.581 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:25.682 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:25.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:25.705 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:25.706 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:25.806 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:25.829 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:25.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:25.830 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:25.931 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:25.952 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:25.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:25.954 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:26.054 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:26.077 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:26.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:26.078 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:26.179 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:26.200 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:26.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:26.201 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:26.302 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:26.312 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-10T18:55:26.312 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-10T18:55:26.323 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:26.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:26.324 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:26.425 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:26.447 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:26.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:26.448 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:26.549 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:26.571 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:26.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:26.572 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:26.620 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:26 smithi103 ceph-mon[33163]: pgmap v450: 168 pgs: 1 active+clean+remapped, 43 active+undersized, 25 active+undersized+degraded, 99 active+clean; 404 KiB data, 5.0 GiB used, 621 GiB / 626 GiB avail; 639 B/s rd, 0 op/s; 78/588 objects degraded (13.265%) 2024-09-10T18:55:26.620 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:26 smithi103 ceph-mon[33163]: 4.15 scrub starts 2024-09-10T18:55:26.620 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:26 smithi103 ceph-mon[33163]: 4.15 scrub ok 2024-09-10T18:55:26.620 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:26 smithi103 ceph-mon[33163]: 2.16 scrub starts 2024-09-10T18:55:26.621 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:26 smithi103 ceph-mon[33163]: 2.16 scrub ok 2024-09-10T18:55:26.673 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:26.685 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:26 smithi018 ceph-mon[29336]: pgmap v450: 168 pgs: 1 active+clean+remapped, 43 active+undersized, 25 active+undersized+degraded, 99 active+clean; 404 KiB data, 5.0 GiB used, 621 GiB / 626 GiB avail; 639 B/s rd, 0 op/s; 78/588 objects degraded (13.265%) 2024-09-10T18:55:26.685 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:26 smithi018 ceph-mon[29336]: 4.15 scrub starts 2024-09-10T18:55:26.685 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:26 smithi018 ceph-mon[29336]: 4.15 scrub ok 2024-09-10T18:55:26.685 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:26 smithi018 ceph-mon[29336]: 2.16 scrub starts 2024-09-10T18:55:26.685 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:26 smithi018 ceph-mon[29336]: 2.16 scrub ok 2024-09-10T18:55:26.685 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:26 smithi018 ceph-mon[40196]: pgmap v450: 168 pgs: 1 active+clean+remapped, 43 active+undersized, 25 active+undersized+degraded, 99 active+clean; 404 KiB data, 5.0 GiB used, 621 GiB / 626 GiB avail; 639 B/s rd, 0 op/s; 78/588 objects degraded (13.265%) 2024-09-10T18:55:26.685 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:26 smithi018 ceph-mon[40196]: 4.15 scrub starts 2024-09-10T18:55:26.685 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:26 smithi018 ceph-mon[40196]: 4.15 scrub ok 2024-09-10T18:55:26.685 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:26 smithi018 ceph-mon[40196]: 2.16 scrub starts 2024-09-10T18:55:26.686 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:26 smithi018 ceph-mon[40196]: 2.16 scrub ok 2024-09-10T18:55:26.697 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:26.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:26.698 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:26.799 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:26.823 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:26.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:26.824 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:26.918 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:55:26 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:55:26.924 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:26.944 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:55:26 smithi018 podman[130017]: 2024-09-10 18:55:26.586895941 +0000 UTC m=+1.248698378 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:55:26.944 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:55:26 smithi018 podman[130017]: 2024-09-10 18:55:26.882388238 +0000 UTC m=+1.544190675 container create 87a10e6aab049181a1a4373e2fe8949a0382dcba77668b3eb4652dfb887847ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-10T18:55:26.946 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:26.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:26.947 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:27.048 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:27.070 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:27.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:27.071 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:27.099 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-10T18:55:27.172 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:27.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:27.195 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:27.195 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:27.296 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:27.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:27.318 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:27.318 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:27.326 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:55:27 smithi018 podman[130017]: 2024-09-10 18:55:27.076538604 +0000 UTC m=+1.738341041 container init 87a10e6aab049181a1a4373e2fe8949a0382dcba77668b3eb4652dfb887847ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0-activate, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-10T18:55:27.326 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:55:27 smithi018 podman[130017]: 2024-09-10 18:55:27.080143555 +0000 UTC m=+1.741946000 container start 87a10e6aab049181a1a4373e2fe8949a0382dcba77668b3eb4652dfb887847ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0-activate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:55:27.326 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:55:27 smithi018 podman[130017]: 2024-09-10 18:55:27.176412429 +0000 UTC m=+1.838214885 container attach 87a10e6aab049181a1a4373e2fe8949a0382dcba77668b3eb4652dfb887847ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0-activate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True) 2024-09-10T18:55:27.326 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:27 smithi018 ceph-mon[29336]: 4.d scrub starts 2024-09-10T18:55:27.327 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:27 smithi018 ceph-mon[29336]: 4.d scrub ok 2024-09-10T18:55:27.327 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:27 smithi018 ceph-mon[29336]: 4.1f scrub starts 2024-09-10T18:55:27.327 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:27 smithi018 ceph-mon[29336]: 4.1f scrub ok 2024-09-10T18:55:27.327 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:27 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/122036562' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:55:27.327 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:27 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:55:27.327 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:27 smithi018 ceph-mon[29336]: 5.f scrub starts 2024-09-10T18:55:27.327 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:27 smithi018 ceph-mon[29336]: 5.f scrub ok 2024-09-10T18:55:27.333 INFO:teuthology.orchestra.run.smithi018.stderr:pool 'unique_pool_1' removed 2024-09-10T18:55:27.419 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:27.437 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T18:55:27.437 INFO:teuthology.orchestra.run.smithi103.stdout: "ops": [], 2024-09-10T18:55:27.438 INFO:teuthology.orchestra.run.smithi103.stdout: "num_ops": 0 2024-09-10T18:55:27.438 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T18:55:27.442 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:27.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:27.443 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:27.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:27 smithi103 ceph-mon[33163]: 4.d scrub starts 2024-09-10T18:55:27.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:27 smithi103 ceph-mon[33163]: 4.d scrub ok 2024-09-10T18:55:27.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:27 smithi103 ceph-mon[33163]: 4.1f scrub starts 2024-09-10T18:55:27.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:27 smithi103 ceph-mon[33163]: 4.1f scrub ok 2024-09-10T18:55:27.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:27 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/122036562' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:55:27.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:27 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:55:27.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:27 smithi103 ceph-mon[33163]: 5.f scrub starts 2024-09-10T18:55:27.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:27 smithi103 ceph-mon[33163]: 5.f scrub ok 2024-09-10T18:55:27.544 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:27.566 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:27.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:27.567 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:27.668 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:27.690 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:27.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:27.691 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:27.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:27 smithi018 ceph-mon[40196]: 4.d scrub starts 2024-09-10T18:55:27.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:27 smithi018 ceph-mon[40196]: 4.d scrub ok 2024-09-10T18:55:27.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:27 smithi018 ceph-mon[40196]: 4.1f scrub starts 2024-09-10T18:55:27.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:27 smithi018 ceph-mon[40196]: 4.1f scrub ok 2024-09-10T18:55:27.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:27 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/122036562' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:55:27.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:27 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:55:27.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:27 smithi018 ceph-mon[40196]: 5.f scrub starts 2024-09-10T18:55:27.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:27 smithi018 ceph-mon[40196]: 5.f scrub ok 2024-09-10T18:55:27.695 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:55:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0-activate[130151]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-10T18:55:27.695 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:55:27 smithi018 bash[130017]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-10T18:55:27.695 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:55:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0-activate[130151]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-09-10T18:55:27.695 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:55:27 smithi018 bash[130017]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-09-10T18:55:27.695 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:55:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0-activate[130151]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-09-10T18:55:27.695 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:55:27 smithi018 bash[130017]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-09-10T18:55:27.695 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:55:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0-activate[130151]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-10T18:55:27.695 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:55:27 smithi018 bash[130017]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-10T18:55:27.695 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:55:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0-activate[130151]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-09-10T18:55:27.695 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:55:27 smithi018 bash[130017]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-09-10T18:55:27.695 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:55:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0-activate[130151]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-10T18:55:27.696 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:55:27 smithi018 bash[130017]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-10T18:55:27.696 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:55:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0-activate[130151]: --> ceph-volume raw activate successful for osd ID: 0 2024-09-10T18:55:27.696 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:55:27 smithi018 bash[130017]: --> ceph-volume raw activate successful for osd ID: 0 2024-09-10T18:55:27.696 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:55:27 smithi018 podman[130017]: 2024-09-10 18:55:27.469106895 +0000 UTC m=+2.130909356 container died 87a10e6aab049181a1a4373e2fe8949a0382dcba77668b3eb4652dfb887847ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0-activate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-10T18:55:27.791 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:27.814 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:55:27.814 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:27.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:27.815 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:27.916 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:27.938 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:27.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:27.939 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:28.040 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:28.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:28.062 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:28.062 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:28.163 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:28.184 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:28.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:28.185 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:28.224 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-10T18:55:28.224 INFO:tasks.radosbench:Beginning radosbench... 2024-09-10T18:55:28.225 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_2 2024-09-10T18:55:28.225 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd pool create unique_pool_2 16 2024-09-10T18:55:28.286 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:28.307 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:28.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:28.308 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:28.313 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T18:55:28.409 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:28.431 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:28.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:28.432 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:28.478 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:28 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:28.478 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:28 smithi018 ceph-mon[40196]: pgmap v451: 168 pgs: 1 active+clean+remapped, 46 active+undersized, 22 active+undersized+degraded, 99 active+clean; 404 KiB data, 4.9 GiB used, 621 GiB / 626 GiB avail; 55 KiB/s rd, 0 B/s wr, 90 op/s; 78/588 objects degraded (13.265%) 2024-09-10T18:55:28.478 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:28 smithi018 ceph-mon[40196]: Health check update: Degraded data redundancy: 78/588 objects degraded (13.265%), 22 pgs degraded, 68 pgs undersized (PG_DEGRADED) 2024-09-10T18:55:28.478 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:28 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T18:55:28.478 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:28 smithi018 ceph-mon[40196]: osdmap e233: 8 total, 7 up, 7 in 2024-09-10T18:55:28.478 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:28 smithi018 ceph-mon[40196]: 2.9 scrub starts 2024-09-10T18:55:28.478 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:28 smithi018 ceph-mon[40196]: 5.1a scrub starts 2024-09-10T18:55:28.478 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:28 smithi018 ceph-mon[40196]: 5.1a scrub ok 2024-09-10T18:55:28.479 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:28 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:28.479 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:28 smithi018 ceph-mon[29336]: pgmap v451: 168 pgs: 1 active+clean+remapped, 46 active+undersized, 22 active+undersized+degraded, 99 active+clean; 404 KiB data, 4.9 GiB used, 621 GiB / 626 GiB avail; 55 KiB/s rd, 0 B/s wr, 90 op/s; 78/588 objects degraded (13.265%) 2024-09-10T18:55:28.479 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:28 smithi018 ceph-mon[29336]: Health check update: Degraded data redundancy: 78/588 objects degraded (13.265%), 22 pgs degraded, 68 pgs undersized (PG_DEGRADED) 2024-09-10T18:55:28.479 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:28 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T18:55:28.479 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:28 smithi018 ceph-mon[29336]: osdmap e233: 8 total, 7 up, 7 in 2024-09-10T18:55:28.479 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:28 smithi018 ceph-mon[29336]: 2.9 scrub starts 2024-09-10T18:55:28.479 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:28 smithi018 ceph-mon[29336]: 5.1a scrub starts 2024-09-10T18:55:28.479 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:28 smithi018 ceph-mon[29336]: 5.1a scrub ok 2024-09-10T18:55:28.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:28 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:28.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:28 smithi103 ceph-mon[33163]: pgmap v451: 168 pgs: 1 active+clean+remapped, 46 active+undersized, 22 active+undersized+degraded, 99 active+clean; 404 KiB data, 4.9 GiB used, 621 GiB / 626 GiB avail; 55 KiB/s rd, 0 B/s wr, 90 op/s; 78/588 objects degraded (13.265%) 2024-09-10T18:55:28.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:28 smithi103 ceph-mon[33163]: Health check update: Degraded data redundancy: 78/588 objects degraded (13.265%), 22 pgs degraded, 68 pgs undersized (PG_DEGRADED) 2024-09-10T18:55:28.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:28 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T18:55:28.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:28 smithi103 ceph-mon[33163]: osdmap e233: 8 total, 7 up, 7 in 2024-09-10T18:55:28.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:28 smithi103 ceph-mon[33163]: 2.9 scrub starts 2024-09-10T18:55:28.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:28 smithi103 ceph-mon[33163]: 5.1a scrub starts 2024-09-10T18:55:28.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:28 smithi103 ceph-mon[33163]: 5.1a scrub ok 2024-09-10T18:55:28.533 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:28.556 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:28.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:28.557 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:28.570 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-10T18:55:28.658 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:28.681 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:28.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:28.682 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:28.783 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:28.806 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:28.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:28.807 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:28.861 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:55:28.908 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:28.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:28.931 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:28.931 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:29.031 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:29.053 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:29.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:29.054 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:29.155 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:29.177 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:29.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:29.178 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:29.279 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:29.302 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:29.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:29.303 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:29.404 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:29.425 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:29.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:29.426 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:29.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:29 smithi018 ceph-mon[29336]: 4.9 scrub starts 2024-09-10T18:55:29.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:29 smithi018 ceph-mon[29336]: 4.9 scrub ok 2024-09-10T18:55:29.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:29 smithi018 ceph-mon[29336]: 4.a scrub starts 2024-09-10T18:55:29.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:29 smithi018 ceph-mon[29336]: 4.a scrub ok 2024-09-10T18:55:29.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:29 smithi018 ceph-mon[29336]: osdmap e234: 8 total, 7 up, 7 in 2024-09-10T18:55:29.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:29 smithi018 ceph-mon[29336]: 6.12 scrub starts 2024-09-10T18:55:29.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:29 smithi018 ceph-mon[29336]: 6.12 scrub ok 2024-09-10T18:55:29.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:29 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:55:29.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:29 smithi018 ceph-mon[40196]: 4.9 scrub starts 2024-09-10T18:55:29.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:29 smithi018 ceph-mon[40196]: 4.9 scrub ok 2024-09-10T18:55:29.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:29 smithi018 ceph-mon[40196]: 4.a scrub starts 2024-09-10T18:55:29.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:29 smithi018 ceph-mon[40196]: 4.a scrub ok 2024-09-10T18:55:29.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:29 smithi018 ceph-mon[40196]: osdmap e234: 8 total, 7 up, 7 in 2024-09-10T18:55:29.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:29 smithi018 ceph-mon[40196]: 6.12 scrub starts 2024-09-10T18:55:29.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:29 smithi018 ceph-mon[40196]: 6.12 scrub ok 2024-09-10T18:55:29.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:29 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:55:29.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:29 smithi103 ceph-mon[33163]: 4.9 scrub starts 2024-09-10T18:55:29.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:29 smithi103 ceph-mon[33163]: 4.9 scrub ok 2024-09-10T18:55:29.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:29 smithi103 ceph-mon[33163]: 4.a scrub starts 2024-09-10T18:55:29.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:29 smithi103 ceph-mon[33163]: 4.a scrub ok 2024-09-10T18:55:29.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:29 smithi103 ceph-mon[33163]: osdmap e234: 8 total, 7 up, 7 in 2024-09-10T18:55:29.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:29 smithi103 ceph-mon[33163]: 6.12 scrub starts 2024-09-10T18:55:29.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:29 smithi103 ceph-mon[33163]: 6.12 scrub ok 2024-09-10T18:55:29.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:29 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:55:29.527 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:29.552 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:29.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:29.553 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:29.654 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:29.676 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:29.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:29.677 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:29.778 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:29.798 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:29.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:29.799 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:29.900 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:29.921 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:29.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:29.922 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:29.944 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:55:29 smithi018 podman[130341]: 2024-09-10 18:55:29.497845936 +0000 UTC m=+2.056566752 container remove 87a10e6aab049181a1a4373e2fe8949a0382dcba77668b3eb4652dfb887847ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-10T18:55:30.023 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:30.047 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:55:30.049 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:30.050 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:55:30.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:30.050 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:30.151 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:30.174 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:30.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:30.176 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:30.276 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:30.300 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:30.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:30.301 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:30.319 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:55:30 smithi018 podman[130858]: 2024-09-10 18:55:30.062289609 +0000 UTC m=+0.021143768 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:55:30.319 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:55:30 smithi018 podman[130858]: 2024-09-10 18:55:30.219357125 +0000 UTC m=+0.178211284 container create e4449adc28b27c6b3584f3c0692e354e893cf1b14bf0ccb8ab08036ad1180169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-10T18:55:30.353 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:55:30 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:55:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:55:30.353 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:30 smithi103 ceph-mon[33163]: 1.f scrub starts 2024-09-10T18:55:30.354 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:30 smithi103 ceph-mon[33163]: 1.f scrub ok 2024-09-10T18:55:30.354 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:30 smithi103 ceph-mon[33163]: pgmap v454: 152 pgs: 41 active+undersized, 22 active+undersized+degraded, 89 active+clean; 404 KiB data, 4.9 GiB used, 621 GiB / 626 GiB avail; 73 KiB/s rd, 73 op/s; 78/588 objects degraded (13.265%) 2024-09-10T18:55:30.354 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:30 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:55:30.354 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:30 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:55:30.354 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:30 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:55:30.354 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:30 smithi103 ceph-mon[33163]: 2.2 scrub starts 2024-09-10T18:55:30.354 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:30 smithi103 ceph-mon[33163]: 2.2 scrub ok 2024-09-10T18:55:30.401 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:30.425 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:30.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:30.426 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:30.453 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T18:55:30.453 INFO:teuthology.orchestra.run.smithi103.stdout: "ops": [], 2024-09-10T18:55:30.453 INFO:teuthology.orchestra.run.smithi103.stdout: "complaint_time": 30, 2024-09-10T18:55:30.453 INFO:teuthology.orchestra.run.smithi103.stdout: "num_blocked_ops": 0 2024-09-10T18:55:30.453 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T18:55:30.527 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:30.550 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:30.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:30.551 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:30.593 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:30 smithi018 ceph-mon[40196]: 1.f scrub starts 2024-09-10T18:55:30.593 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:30 smithi018 ceph-mon[40196]: 1.f scrub ok 2024-09-10T18:55:30.593 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:30 smithi018 ceph-mon[40196]: pgmap v454: 152 pgs: 41 active+undersized, 22 active+undersized+degraded, 89 active+clean; 404 KiB data, 4.9 GiB used, 621 GiB / 626 GiB avail; 73 KiB/s rd, 73 op/s; 78/588 objects degraded (13.265%) 2024-09-10T18:55:30.593 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:30 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:55:30.593 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:30 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:55:30.593 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:30 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:55:30.593 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:30 smithi018 ceph-mon[40196]: 2.2 scrub starts 2024-09-10T18:55:30.593 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:30 smithi018 ceph-mon[40196]: 2.2 scrub ok 2024-09-10T18:55:30.594 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:30 smithi018 ceph-mon[29336]: 1.f scrub starts 2024-09-10T18:55:30.594 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:30 smithi018 ceph-mon[29336]: 1.f scrub ok 2024-09-10T18:55:30.594 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:30 smithi018 ceph-mon[29336]: pgmap v454: 152 pgs: 41 active+undersized, 22 active+undersized+degraded, 89 active+clean; 404 KiB data, 4.9 GiB used, 621 GiB / 626 GiB avail; 73 KiB/s rd, 73 op/s; 78/588 objects degraded (13.265%) 2024-09-10T18:55:30.594 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:30 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:55:30.594 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:30 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:55:30.594 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:30 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:55:30.594 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:30 smithi018 ceph-mon[29336]: 2.2 scrub starts 2024-09-10T18:55:30.594 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:30 smithi018 ceph-mon[29336]: 2.2 scrub ok 2024-09-10T18:55:30.652 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:30.683 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:30.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:30.684 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:30.785 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:30.808 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:30.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:30.809 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:30.910 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:30.932 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:30.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:30.933 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:31.034 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:31.057 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:31.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:31.058 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:31.075 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-10T18:55:31.159 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:31.181 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:31.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:31.182 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:31.283 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:31.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:31.306 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:31.307 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:31.324 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:55:31.328 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:55:30 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:55:30] "GET /metrics HTTP/1.1" 200 224131 "" "Prometheus/2.43.0" 2024-09-10T18:55:31.337 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is unset 2024-09-10T18:55:31.408 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:31.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:31.430 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:31.430 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:31.531 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:31.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:31.554 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:31.554 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:31.641 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:31 smithi018 ceph-mon[40196]: 5.1 scrub starts 2024-09-10T18:55:31.641 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:31 smithi018 ceph-mon[40196]: 5.1 scrub ok 2024-09-10T18:55:31.641 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:31 smithi018 ceph-mon[40196]: 6.1 scrub starts 2024-09-10T18:55:31.641 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:31 smithi018 ceph-mon[40196]: 6.1 scrub ok 2024-09-10T18:55:31.641 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:31 smithi018 ceph-mon[40196]: 4.17 scrub starts 2024-09-10T18:55:31.641 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:31 smithi018 ceph-mon[40196]: 4.17 scrub ok 2024-09-10T18:55:31.641 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:31 smithi018 ceph-mon[40196]: 2.b scrub starts 2024-09-10T18:55:31.641 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:31 smithi018 ceph-mon[40196]: 2.b scrub ok 2024-09-10T18:55:31.642 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:31 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/4181781858' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:55:31.642 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:31 smithi018 ceph-mon[29336]: 5.1 scrub starts 2024-09-10T18:55:31.642 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:31 smithi018 ceph-mon[29336]: 5.1 scrub ok 2024-09-10T18:55:31.642 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:31 smithi018 ceph-mon[29336]: 6.1 scrub starts 2024-09-10T18:55:31.642 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:31 smithi018 ceph-mon[29336]: 6.1 scrub ok 2024-09-10T18:55:31.642 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:31 smithi018 ceph-mon[29336]: 4.17 scrub starts 2024-09-10T18:55:31.642 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:31 smithi018 ceph-mon[29336]: 4.17 scrub ok 2024-09-10T18:55:31.642 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:31 smithi018 ceph-mon[29336]: 2.b scrub starts 2024-09-10T18:55:31.642 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:31 smithi018 ceph-mon[29336]: 2.b scrub ok 2024-09-10T18:55:31.642 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:31 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/4181781858' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:55:31.655 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:31.663 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:31 smithi103 ceph-mon[33163]: 5.1 scrub starts 2024-09-10T18:55:31.663 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:31 smithi103 ceph-mon[33163]: 5.1 scrub ok 2024-09-10T18:55:31.663 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:31 smithi103 ceph-mon[33163]: 6.1 scrub starts 2024-09-10T18:55:31.663 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:31 smithi103 ceph-mon[33163]: 6.1 scrub ok 2024-09-10T18:55:31.663 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:31 smithi103 ceph-mon[33163]: 4.17 scrub starts 2024-09-10T18:55:31.663 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:31 smithi103 ceph-mon[33163]: 4.17 scrub ok 2024-09-10T18:55:31.663 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:31 smithi103 ceph-mon[33163]: 2.b scrub starts 2024-09-10T18:55:31.663 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:31 smithi103 ceph-mon[33163]: 2.b scrub ok 2024-09-10T18:55:31.663 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:31 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/4181781858' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:55:31.678 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:31.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:31.679 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:31.780 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:31.802 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:31.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:31.803 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:31.904 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:31.926 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:31.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:31.928 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:32.028 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:32.050 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:32.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:32.051 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:32.152 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:32.178 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:32.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:32.179 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:32.280 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:32.303 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:32.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:32.305 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:32.406 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:32.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:32.430 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:32.430 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:32.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:32 smithi103 ceph-mon[33163]: 1.e scrub starts 2024-09-10T18:55:32.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:32 smithi103 ceph-mon[33163]: 1.e scrub ok 2024-09-10T18:55:32.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:32 smithi103 ceph-mon[33163]: pgmap v455: 152 pgs: 41 active+undersized, 22 active+undersized+degraded, 89 active+clean; 404 KiB data, 4.9 GiB used, 621 GiB / 626 GiB avail; 57 KiB/s rd, 56 op/s; 78/588 objects degraded (13.265%) 2024-09-10T18:55:32.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:32 smithi103 ceph-mon[33163]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:55:32.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:32 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/4181781858' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:55:32.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:32 smithi103 ceph-mon[33163]: osdmap e235: 8 total, 7 up, 7 in 2024-09-10T18:55:32.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:32 smithi103 ceph-mon[33163]: 1.8 scrub starts 2024-09-10T18:55:32.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:32 smithi103 ceph-mon[33163]: 1.8 scrub ok 2024-09-10T18:55:32.531 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:32.545 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:32 smithi018 ceph-mon[40196]: 1.e scrub starts 2024-09-10T18:55:32.546 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:32 smithi018 ceph-mon[40196]: 1.e scrub ok 2024-09-10T18:55:32.546 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:32 smithi018 ceph-mon[40196]: pgmap v455: 152 pgs: 41 active+undersized, 22 active+undersized+degraded, 89 active+clean; 404 KiB data, 4.9 GiB used, 621 GiB / 626 GiB avail; 57 KiB/s rd, 56 op/s; 78/588 objects degraded (13.265%) 2024-09-10T18:55:32.546 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:32 smithi018 ceph-mon[40196]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:55:32.546 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:32 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/4181781858' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:55:32.546 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:32 smithi018 ceph-mon[40196]: osdmap e235: 8 total, 7 up, 7 in 2024-09-10T18:55:32.546 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:32 smithi018 ceph-mon[40196]: 1.8 scrub starts 2024-09-10T18:55:32.546 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:32 smithi018 ceph-mon[40196]: 1.8 scrub ok 2024-09-10T18:55:32.546 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:32 smithi018 ceph-mon[29336]: 1.e scrub starts 2024-09-10T18:55:32.546 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:32 smithi018 ceph-mon[29336]: 1.e scrub ok 2024-09-10T18:55:32.546 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:32 smithi018 ceph-mon[29336]: pgmap v455: 152 pgs: 41 active+undersized, 22 active+undersized+degraded, 89 active+clean; 404 KiB data, 4.9 GiB used, 621 GiB / 626 GiB avail; 57 KiB/s rd, 56 op/s; 78/588 objects degraded (13.265%) 2024-09-10T18:55:32.547 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:32 smithi018 ceph-mon[29336]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:55:32.547 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:32 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/4181781858' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:55:32.547 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:32 smithi018 ceph-mon[29336]: osdmap e235: 8 total, 7 up, 7 in 2024-09-10T18:55:32.547 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:32 smithi018 ceph-mon[29336]: 1.8 scrub starts 2024-09-10T18:55:32.547 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:32 smithi018 ceph-mon[29336]: 1.8 scrub ok 2024-09-10T18:55:32.554 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:32.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:32.555 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:32.656 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:32.678 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:32.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:32.679 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:32.780 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:32.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:32.804 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:32.804 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:32.905 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:32.927 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:32.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:32.928 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:32.991 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T18:55:32.991 INFO:teuthology.orchestra.run.smithi103.stdout: "size": 20, 2024-09-10T18:55:32.992 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 600, 2024-09-10T18:55:32.992 INFO:teuthology.orchestra.run.smithi103.stdout: "ops": [ 2024-09-10T18:55:32.992 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:32.992 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14790.0:15123 7.1 7:8ab84aee:::benchmark_data_smithi018_78628_object15122:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-10T18:55:32.992 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:47:49.069832+0000", 2024-09-10T18:55:32.992 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 463.904969607, 2024-09-10T18:55:32.992 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.13743958000000001, 2024-09-10T18:55:32.992 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:32.992 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:55:32.992 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:55:32.992 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14790", 2024-09-10T18:55:32.992 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/4175225295", 2024-09-10T18:55:32.993 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 15123 2024-09-10T18:55:32.993 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:32.993 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:32.993 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:32.993 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:32.993 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.069832+0000", 2024-09-10T18:55:32.993 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:32.993 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:32.993 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:32.993 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:32.993 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.069832+0000", 2024-09-10T18:55:32.994 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:32.994 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:32.994 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:32.994 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:32.994 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.069831+0000", 2024-09-10T18:55:32.994 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:55:32.994 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:32.994 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:32.994 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:32.994 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.069870+0000", 2024-09-10T18:55:32.994 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.9096e-05 2024-09-10T18:55:32.994 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:32.994 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:32.995 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:32.995 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.069891+0000", 2024-09-10T18:55:32.995 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.1656999999999999e-05 2024-09-10T18:55:32.995 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:32.995 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:32.995 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:32.995 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.069897+0000", 2024-09-10T18:55:32.995 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.1809999999999998e-06 2024-09-10T18:55:32.995 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:32.995 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:32.995 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:32.995 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.069941+0000", 2024-09-10T18:55:32.995 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.3631000000000003e-05 2024-09-10T18:55:32.996 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:32.996 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:32.996 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:55:32.996 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.070064+0000", 2024-09-10T18:55:32.996 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000122488 2024-09-10T18:55:32.996 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:32.996 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:32.996 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,6", 2024-09-10T18:55:32.996 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.070205+0000", 2024-09-10T18:55:32.996 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00014103600000000001 2024-09-10T18:55:32.996 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:32.996 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:32.996 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:55:32.997 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.071177+0000", 2024-09-10T18:55:32.997 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00097243299999999998 2024-09-10T18:55:32.997 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:32.997 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:32.997 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:32.997 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.071494+0000", 2024-09-10T18:55:32.997 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000317292 2024-09-10T18:55:32.997 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:32.997 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:32.997 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:32.997 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.207096+0000", 2024-09-10T18:55:32.997 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.13560148699999999 2024-09-10T18:55:32.997 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:32.998 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:32.998 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:55:32.998 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.207191+0000", 2024-09-10T18:55:32.998 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.5328000000000006e-05 2024-09-10T18:55:32.998 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:32.998 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:32.998 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:32.998 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:47:49.207271+0000", 2024-09-10T18:55:32.998 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.0190000000000003e-05 2024-09-10T18:55:32.998 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:32.998 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:32.998 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:32.999 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:32.999 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:32.999 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:114343 8.2 8:4826fe8e:::benchmark_data_smithi018_106531_object114342:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e182)", 2024-09-10T18:55:32.999 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:52:52.257841+0000", 2024-09-10T18:55:32.999 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 160.71696014400001, 2024-09-10T18:55:32.999 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.18390023, 2024-09-10T18:55:32.999 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:32.999 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:55:32.999 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:55:32.999 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:55:32.999 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:55:32.999 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 114343 2024-09-10T18:55:32.999 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.000 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:33.000 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.000 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:33.000 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.257841+0000", 2024-09-10T18:55:33.000 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.000 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.000 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.000 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:33.000 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.257841+0000", 2024-09-10T18:55:33.000 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.000 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.000 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.000 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:33.001 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.257840+0000", 2024-09-10T18:55:33.001 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:55:33.001 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.001 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.001 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:33.001 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.257871+0000", 2024-09-10T18:55:33.001 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.1618000000000001e-05 2024-09-10T18:55:33.001 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.001 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.001 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:33.001 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.257883+0000", 2024-09-10T18:55:33.001 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.131e-05 2024-09-10T18:55:33.001 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.002 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.002 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:33.002 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.257888+0000", 2024-09-10T18:55:33.002 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.3310000000000003e-06 2024-09-10T18:55:33.002 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.002 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.002 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:33.002 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.257935+0000", 2024-09-10T18:55:33.002 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.7012000000000003e-05 2024-09-10T18:55:33.002 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.002 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.002 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:55:33.002 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.258049+0000", 2024-09-10T18:55:33.003 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00011372800000000001 2024-09-10T18:55:33.003 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.003 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.003 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,5", 2024-09-10T18:55:33.003 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.258174+0000", 2024-09-10T18:55:33.003 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00012501600000000001 2024-09-10T18:55:33.003 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.003 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.003 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:33.003 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.265469+0000", 2024-09-10T18:55:33.003 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0072950879999999999 2024-09-10T18:55:33.003 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.003 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.003 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:55:33.004 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.265475+0000", 2024-09-10T18:55:33.004 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.4520000000000002e-06 2024-09-10T18:55:33.004 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.004 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.004 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:33.004 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.441618+0000", 2024-09-10T18:55:33.004 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.17614328000000001 2024-09-10T18:55:33.004 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.004 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.004 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:55:33.005 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.441704+0000", 2024-09-10T18:55:33.005 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.5101999999999999e-05 2024-09-10T18:55:33.005 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.005 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.005 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:33.005 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.441741+0000", 2024-09-10T18:55:33.005 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.7876999999999997e-05 2024-09-10T18:55:33.005 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.005 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:33.005 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.005 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.005 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.006 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:114345 8.2 8:4a61445c:::benchmark_data_smithi018_106531_object114344:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e182)", 2024-09-10T18:55:33.006 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:52:52.259402+0000", 2024-09-10T18:55:33.006 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 160.715398986, 2024-09-10T18:55:33.006 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.18247158099999999, 2024-09-10T18:55:33.006 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:33.006 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:55:33.006 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:55:33.006 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:55:33.006 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:55:33.006 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 114345 2024-09-10T18:55:33.006 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.006 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:33.007 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.007 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:33.007 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.259402+0000", 2024-09-10T18:55:33.007 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.007 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.007 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.007 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:33.007 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.259402+0000", 2024-09-10T18:55:33.007 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.007 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.007 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.007 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:33.008 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.259401+0000", 2024-09-10T18:55:33.008 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:55:33.008 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.008 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.008 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:33.008 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.260238+0000", 2024-09-10T18:55:33.008 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00083640500000000003 2024-09-10T18:55:33.008 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.008 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.008 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:33.008 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.260250+0000", 2024-09-10T18:55:33.008 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.2139e-05 2024-09-10T18:55:33.008 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.009 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.009 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:33.009 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.260256+0000", 2024-09-10T18:55:33.009 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.3210000000000001e-06 2024-09-10T18:55:33.009 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.009 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.009 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:33.009 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.260358+0000", 2024-09-10T18:55:33.009 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000101879 2024-09-10T18:55:33.009 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.009 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.009 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:55:33.009 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.260479+0000", 2024-09-10T18:55:33.009 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00012159699999999999 2024-09-10T18:55:33.010 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.010 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.010 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,5", 2024-09-10T18:55:33.010 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.260623+0000", 2024-09-10T18:55:33.010 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00014367399999999999 2024-09-10T18:55:33.010 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.010 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.010 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:55:33.010 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.265480+0000", 2024-09-10T18:55:33.010 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0048571530000000003 2024-09-10T18:55:33.010 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.010 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.010 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:33.011 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.265516+0000", 2024-09-10T18:55:33.011 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.5441999999999997e-05 2024-09-10T18:55:33.011 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.011 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.011 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:33.011 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.441773+0000", 2024-09-10T18:55:33.011 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.17625696800000001 2024-09-10T18:55:33.011 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.011 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.011 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:55:33.011 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.441842+0000", 2024-09-10T18:55:33.011 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.9701000000000002e-05 2024-09-10T18:55:33.011 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.011 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.012 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:33.012 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.441874+0000", 2024-09-10T18:55:33.012 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.1606999999999997e-05 2024-09-10T18:55:33.012 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.012 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:33.012 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.012 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.012 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.012 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:114347 8.2 8:4644256c:::benchmark_data_smithi018_106531_object114346:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e182)", 2024-09-10T18:55:33.012 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:52:52.260269+0000", 2024-09-10T18:55:33.013 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 160.714532421, 2024-09-10T18:55:33.013 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.181733642, 2024-09-10T18:55:33.013 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:33.013 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:55:33.013 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:55:33.013 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:55:33.013 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:55:33.013 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 114347 2024-09-10T18:55:33.013 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.013 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:33.013 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.013 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:33.014 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.260269+0000", 2024-09-10T18:55:33.014 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.014 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.014 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.014 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:33.014 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.260269+0000", 2024-09-10T18:55:33.014 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.014 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.014 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.014 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:33.014 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.260268+0000", 2024-09-10T18:55:33.014 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:55:33.014 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.014 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.015 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:33.015 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.260297+0000", 2024-09-10T18:55:33.015 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.8452000000000001e-05 2024-09-10T18:55:33.015 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.015 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.015 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:33.015 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.260303+0000", 2024-09-10T18:55:33.015 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.2149999999999998e-06 2024-09-10T18:55:33.015 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.015 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.015 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:33.015 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.260307+0000", 2024-09-10T18:55:33.016 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.0080000000000004e-06 2024-09-10T18:55:33.016 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.016 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.016 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:33.016 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.262967+0000", 2024-09-10T18:55:33.016 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0026598939999999999 2024-09-10T18:55:33.016 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.016 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.016 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:55:33.016 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.263070+0000", 2024-09-10T18:55:33.016 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000102896 2024-09-10T18:55:33.016 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.016 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.017 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,5", 2024-09-10T18:55:33.017 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.263195+0000", 2024-09-10T18:55:33.017 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00012556000000000001 2024-09-10T18:55:33.017 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.017 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.017 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:55:33.017 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.265521+0000", 2024-09-10T18:55:33.017 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0023255089999999999 2024-09-10T18:55:33.017 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.017 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.017 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:33.017 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.269759+0000", 2024-09-10T18:55:33.017 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0042380619999999999 2024-09-10T18:55:33.017 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.018 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.018 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:33.018 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.441903+0000", 2024-09-10T18:55:33.018 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.172143869 2024-09-10T18:55:33.018 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.018 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.018 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:55:33.018 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.441966+0000", 2024-09-10T18:55:33.018 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.3577999999999993e-05 2024-09-10T18:55:33.018 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.018 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.018 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:33.019 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.442003+0000", 2024-09-10T18:55:33.019 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.6316000000000003e-05 2024-09-10T18:55:33.019 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.019 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:33.019 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.019 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.019 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.019 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:114350 8.2 8:42b27377:::benchmark_data_smithi018_106531_object114349:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e182)", 2024-09-10T18:55:33.019 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:52:52.261421+0000", 2024-09-10T18:55:33.019 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 160.713380543, 2024-09-10T18:55:33.019 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.18422898200000001, 2024-09-10T18:55:33.019 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:33.020 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:55:33.020 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:55:33.020 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:55:33.020 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:55:33.020 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 114350 2024-09-10T18:55:33.020 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.020 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:33.020 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.020 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:33.020 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.261421+0000", 2024-09-10T18:55:33.020 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.021 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.021 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.021 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:33.021 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.261421+0000", 2024-09-10T18:55:33.021 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.021 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.021 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.021 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:33.021 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.261420+0000", 2024-09-10T18:55:33.021 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:55:33.021 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.021 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.022 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:33.022 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.261516+0000", 2024-09-10T18:55:33.022 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.6298000000000005e-05 2024-09-10T18:55:33.022 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.022 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.022 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:33.022 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.261524+0000", 2024-09-10T18:55:33.022 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.1359999999999997e-06 2024-09-10T18:55:33.022 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.022 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.022 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:33.022 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.261528+0000", 2024-09-10T18:55:33.022 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.9709999999999998e-06 2024-09-10T18:55:33.023 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.023 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.023 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:33.023 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.265541+0000", 2024-09-10T18:55:33.023 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0040126679999999996 2024-09-10T18:55:33.023 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.023 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.023 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:55:33.023 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.265653+0000", 2024-09-10T18:55:33.023 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000112144 2024-09-10T18:55:33.023 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.023 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.023 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,5", 2024-09-10T18:55:33.023 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.265783+0000", 2024-09-10T18:55:33.024 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000129326 2024-09-10T18:55:33.024 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.024 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.024 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:55:33.024 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.268577+0000", 2024-09-10T18:55:33.024 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0027948999999999999 2024-09-10T18:55:33.024 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.024 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.024 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:33.024 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.269904+0000", 2024-09-10T18:55:33.024 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.001326977 2024-09-10T18:55:33.024 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.024 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.025 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:33.025 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.445358+0000", 2024-09-10T18:55:33.025 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.175453311 2024-09-10T18:55:33.025 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.025 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.025 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:55:33.025 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.445501+0000", 2024-09-10T18:55:33.025 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00014325000000000001 2024-09-10T18:55:33.025 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.025 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.025 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:33.025 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.445650+0000", 2024-09-10T18:55:33.025 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00014880699999999999 2024-09-10T18:55:33.026 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.026 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:33.026 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.026 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.026 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.026 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:114363 8.2 8:44557cd9:::benchmark_data_smithi018_106531_object114362:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e182)", 2024-09-10T18:55:33.026 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:52:52.272894+0000", 2024-09-10T18:55:33.026 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 160.70190690499999, 2024-09-10T18:55:33.026 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.18221031100000001, 2024-09-10T18:55:33.026 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:33.026 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:55:33.026 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:55:33.026 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:55:33.027 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:55:33.027 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 114363 2024-09-10T18:55:33.027 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.027 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:33.027 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.027 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:33.027 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.272894+0000", 2024-09-10T18:55:33.027 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.027 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.027 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.027 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:33.027 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.272894+0000", 2024-09-10T18:55:33.027 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.028 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.028 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.028 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:33.028 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.272893+0000", 2024-09-10T18:55:33.028 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:55:33.028 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.028 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.028 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:33.028 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.272954+0000", 2024-09-10T18:55:33.028 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.1280999999999998e-05 2024-09-10T18:55:33.028 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.028 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.028 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:33.028 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.272962+0000", 2024-09-10T18:55:33.029 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.259e-06 2024-09-10T18:55:33.029 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.029 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.029 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:33.029 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.272967+0000", 2024-09-10T18:55:33.029 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.104e-06 2024-09-10T18:55:33.029 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.029 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.029 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:33.029 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.272999+0000", 2024-09-10T18:55:33.029 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.2845999999999997e-05 2024-09-10T18:55:33.029 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.029 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.030 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:55:33.030 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.273081+0000", 2024-09-10T18:55:33.030 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.1428999999999996e-05 2024-09-10T18:55:33.030 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.030 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.030 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,5", 2024-09-10T18:55:33.030 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.273175+0000", 2024-09-10T18:55:33.030 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.4102000000000001e-05 2024-09-10T18:55:33.030 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.030 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.031 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:55:33.031 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.274856+0000", 2024-09-10T18:55:33.031 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0016813100000000001 2024-09-10T18:55:33.031 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.031 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.031 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:33.031 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.275243+0000", 2024-09-10T18:55:33.031 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00038718699999999999 2024-09-10T18:55:33.031 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.031 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.031 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:33.031 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.454809+0000", 2024-09-10T18:55:33.031 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.17956580799999999 2024-09-10T18:55:33.032 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.032 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.032 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:55:33.032 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.454986+0000", 2024-09-10T18:55:33.032 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00017709099999999999 2024-09-10T18:55:33.032 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.032 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.032 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:33.032 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:52.455105+0000", 2024-09-10T18:55:33.032 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00011843799999999999 2024-09-10T18:55:33.032 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.032 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:33.033 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.033 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.033 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.033 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_repop(client.24947.0:175712 8.f e201/158 8:f7635686:::benchmark_data_smithi018_106531_object29082:head v 201'11143, mlcod=201'11140)", 2024-09-10T18:55:33.033 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:53:44.313766+0000", 2024-09-10T18:55:33.033 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 108.66103520999999, 2024-09-10T18:55:33.033 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.191414907, 2024-09-10T18:55:33.033 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:33.033 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:55:33.033 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:33.033 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.033 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:33.033 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.313766+0000", 2024-09-10T18:55:33.034 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.034 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.034 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.034 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:33.034 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.313766+0000", 2024-09-10T18:55:33.034 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.034 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.034 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.034 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:33.034 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.313765+0000", 2024-09-10T18:55:33.034 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:55:33.034 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.034 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.035 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:33.035 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.313772+0000", 2024-09-10T18:55:33.035 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.1759999999999998e-06 2024-09-10T18:55:33.035 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.035 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.035 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:33.035 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.313782+0000", 2024-09-10T18:55:33.035 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.9820000000000003e-06 2024-09-10T18:55:33.035 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.035 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.035 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:33.035 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.313787+0000", 2024-09-10T18:55:33.035 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.1460000000000004e-06 2024-09-10T18:55:33.036 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.036 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.036 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:33.036 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.313836+0000", 2024-09-10T18:55:33.036 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.9026000000000001e-05 2024-09-10T18:55:33.036 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.036 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.036 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:55:33.036 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.313854+0000", 2024-09-10T18:55:33.036 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.8405000000000001e-05 2024-09-10T18:55:33.036 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.036 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.036 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:55:33.037 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.505159+0000", 2024-09-10T18:55:33.037 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.191304683 2024-09-10T18:55:33.037 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.037 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.037 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:33.037 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.505181+0000", 2024-09-10T18:55:33.037 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.226e-05 2024-09-10T18:55:33.037 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.037 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:33.037 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.037 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.037 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.037 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:175717 8.2 8:49893980:::benchmark_data_smithi018_106531_object29087:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e201)", 2024-09-10T18:55:33.038 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:53:44.314474+0000", 2024-09-10T18:55:33.038 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 108.660327747, 2024-09-10T18:55:33.038 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.20183859500000001, 2024-09-10T18:55:33.038 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:33.038 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:55:33.038 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:55:33.038 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:55:33.038 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:55:33.038 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 175717 2024-09-10T18:55:33.038 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.038 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:33.038 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.039 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:33.039 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.314474+0000", 2024-09-10T18:55:33.039 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.039 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.039 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.039 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:33.039 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.314474+0000", 2024-09-10T18:55:33.039 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.039 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.039 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.039 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:33.039 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.314473+0000", 2024-09-10T18:55:33.039 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:55:33.040 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.040 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.040 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:33.040 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.314479+0000", 2024-09-10T18:55:33.040 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.9800000000000003e-06 2024-09-10T18:55:33.040 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.040 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.040 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:33.040 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.499793+0000", 2024-09-10T18:55:33.040 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.18531483300000001 2024-09-10T18:55:33.040 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.040 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.041 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:33.041 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.499806+0000", 2024-09-10T18:55:33.041 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.2618e-05 2024-09-10T18:55:33.041 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.041 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.041 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:33.041 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.506800+0000", 2024-09-10T18:55:33.041 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0069936770000000002 2024-09-10T18:55:33.041 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.041 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.041 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:55:33.041 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.506926+0000", 2024-09-10T18:55:33.041 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00012590299999999999 2024-09-10T18:55:33.042 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.042 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.042 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,5", 2024-09-10T18:55:33.042 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.507045+0000", 2024-09-10T18:55:33.042 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000119877 2024-09-10T18:55:33.042 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.042 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.042 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:33.042 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.511577+0000", 2024-09-10T18:55:33.042 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0045318590000000001 2024-09-10T18:55:33.042 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.042 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.042 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:33.042 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.516190+0000", 2024-09-10T18:55:33.043 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0046131169999999999 2024-09-10T18:55:33.043 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.043 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.043 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:55:33.043 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.516201+0000", 2024-09-10T18:55:33.043 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0797e-05 2024-09-10T18:55:33.043 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.043 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.043 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:55:33.043 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.516277+0000", 2024-09-10T18:55:33.043 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.6242000000000006e-05 2024-09-10T18:55:33.043 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.043 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.044 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:33.044 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.516312+0000", 2024-09-10T18:55:33.044 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.4749000000000002e-05 2024-09-10T18:55:33.044 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.044 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:33.044 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.044 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.044 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.044 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_repop(client.24947.0:175721 8.f e201/158 8:f3506cce:::benchmark_data_smithi018_106531_object29091:head v 201'11144, mlcod=201'11141)", 2024-09-10T18:55:33.044 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:53:44.315017+0000", 2024-09-10T18:55:33.044 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 108.65978453300001, 2024-09-10T18:55:33.044 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.19018790099999999, 2024-09-10T18:55:33.045 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:33.045 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:55:33.045 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:33.045 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.045 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:33.045 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.315017+0000", 2024-09-10T18:55:33.045 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.045 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.045 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.045 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:33.045 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.315017+0000", 2024-09-10T18:55:33.045 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.046 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.046 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.046 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:33.046 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.315016+0000", 2024-09-10T18:55:33.046 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999995 2024-09-10T18:55:33.046 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.046 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.046 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:33.046 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.315022+0000", 2024-09-10T18:55:33.046 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.3789999999999997e-06 2024-09-10T18:55:33.046 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.046 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.046 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:33.047 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.499804+0000", 2024-09-10T18:55:33.047 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.184782101 2024-09-10T18:55:33.047 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.047 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.047 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:33.047 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.499812+0000", 2024-09-10T18:55:33.047 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.5739999999999996e-06 2024-09-10T18:55:33.047 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.047 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.047 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:33.047 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.502327+0000", 2024-09-10T18:55:33.047 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0025150250000000002 2024-09-10T18:55:33.048 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.048 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.048 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:55:33.048 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.502346+0000", 2024-09-10T18:55:33.048 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.8542999999999998e-05 2024-09-10T18:55:33.048 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.048 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.048 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:55:33.048 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.505183+0000", 2024-09-10T18:55:33.048 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0028373529999999999 2024-09-10T18:55:33.048 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.048 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.048 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:33.049 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.505205+0000", 2024-09-10T18:55:33.049 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.1495999999999999e-05 2024-09-10T18:55:33.049 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.049 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:33.049 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.049 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.049 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.049 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_repop(client.24947.0:175726 8.e e201/158 8:713a588e:::benchmark_data_smithi018_106531_object29096:head v 201'10855, mlcod=201'10853)", 2024-09-10T18:55:33.049 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:53:44.319131+0000", 2024-09-10T18:55:33.049 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 108.655670586, 2024-09-10T18:55:33.049 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.18626737800000001, 2024-09-10T18:55:33.049 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:33.050 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:55:33.050 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:33.050 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.050 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:33.050 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.319131+0000", 2024-09-10T18:55:33.050 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.050 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.050 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.050 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:33.050 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.319131+0000", 2024-09-10T18:55:33.050 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.050 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.050 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.050 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:33.051 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.319130+0000", 2024-09-10T18:55:33.051 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:55:33.051 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.051 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.051 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:33.051 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.319137+0000", 2024-09-10T18:55:33.051 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.1790000000000002e-06 2024-09-10T18:55:33.051 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.051 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.051 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:33.051 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.499790+0000", 2024-09-10T18:55:33.051 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.18065282499999999 2024-09-10T18:55:33.051 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.052 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.052 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:33.052 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.499815+0000", 2024-09-10T18:55:33.052 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.5091000000000001e-05 2024-09-10T18:55:33.052 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.052 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.052 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:33.052 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.500127+0000", 2024-09-10T18:55:33.052 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00031149699999999999 2024-09-10T18:55:33.052 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.052 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.053 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:55:33.053 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.500152+0000", 2024-09-10T18:55:33.053 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.5052e-05 2024-09-10T18:55:33.053 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.053 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.053 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:55:33.053 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.505370+0000", 2024-09-10T18:55:33.053 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0052185280000000001 2024-09-10T18:55:33.053 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.053 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.053 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:33.053 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:44.505398+0000", 2024-09-10T18:55:33.053 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.8016000000000001e-05 2024-09-10T18:55:33.054 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.054 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:33.054 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.054 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.054 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.054 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:176770 8.2 8:4125ff0a:::benchmark_data_smithi018_106531_object30140:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e202)", 2024-09-10T18:55:33.054 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:53:45.405406+0000", 2024-09-10T18:55:33.054 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 107.569395267, 2024-09-10T18:55:33.054 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.15057749400000001, 2024-09-10T18:55:33.054 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:33.054 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:55:33.054 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:55:33.055 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:55:33.055 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:55:33.055 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 176770 2024-09-10T18:55:33.055 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.055 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:33.055 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.055 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:33.055 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.405406+0000", 2024-09-10T18:55:33.055 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.055 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.055 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.055 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:33.055 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.405406+0000", 2024-09-10T18:55:33.055 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.056 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.056 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.056 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:33.056 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.405404+0000", 2024-09-10T18:55:33.056 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:55:33.056 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.056 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.056 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:33.056 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.405412+0000", 2024-09-10T18:55:33.056 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.0160000000000003e-06 2024-09-10T18:55:33.056 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.056 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.056 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:33.057 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.405423+0000", 2024-09-10T18:55:33.057 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.1073999999999999e-05 2024-09-10T18:55:33.057 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.057 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.057 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:33.057 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.405428+0000", 2024-09-10T18:55:33.057 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.5049999999999998e-06 2024-09-10T18:55:33.057 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.057 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.057 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:33.057 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.405471+0000", 2024-09-10T18:55:33.057 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.2828999999999999e-05 2024-09-10T18:55:33.058 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.058 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.058 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:55:33.058 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.405648+0000", 2024-09-10T18:55:33.058 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00017675500000000001 2024-09-10T18:55:33.058 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.058 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.058 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,5", 2024-09-10T18:55:33.058 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.405772+0000", 2024-09-10T18:55:33.058 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00012382800000000001 2024-09-10T18:55:33.058 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.058 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.058 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:55:33.059 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.407920+0000", 2024-09-10T18:55:33.059 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.002148666 2024-09-10T18:55:33.059 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.059 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.059 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:33.059 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.409617+0000", 2024-09-10T18:55:33.059 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0016963639999999999 2024-09-10T18:55:33.059 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.059 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.059 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:33.059 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.555865+0000", 2024-09-10T18:55:33.060 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.14624896300000001 2024-09-10T18:55:33.060 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.060 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.060 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:55:33.060 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.555946+0000", 2024-09-10T18:55:33.060 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.0518000000000004e-05 2024-09-10T18:55:33.060 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.060 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.060 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:33.060 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.555984+0000", 2024-09-10T18:55:33.060 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.7598000000000002e-05 2024-09-10T18:55:33.060 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.060 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:33.061 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.061 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.061 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.061 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_repop(client.24947.0:176776 8.e e202/158 8:7a552304:::benchmark_data_smithi018_106531_object30146:head v 202'10917, mlcod=202'10915)", 2024-09-10T18:55:33.061 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:53:45.410099+0000", 2024-09-10T18:55:33.061 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 107.56470272999999, 2024-09-10T18:55:33.061 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.127127568, 2024-09-10T18:55:33.061 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:33.061 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:55:33.061 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:33.061 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.061 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:33.062 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.410099+0000", 2024-09-10T18:55:33.062 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.062 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.062 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.062 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:33.062 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.410099+0000", 2024-09-10T18:55:33.062 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.062 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.062 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.062 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:33.062 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.410098+0000", 2024-09-10T18:55:33.062 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999995 2024-09-10T18:55:33.062 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.063 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.063 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:33.063 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.410102+0000", 2024-09-10T18:55:33.063 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.4419999999999998e-06 2024-09-10T18:55:33.063 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.063 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.063 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:33.063 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.410107+0000", 2024-09-10T18:55:33.063 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.7879999999999997e-06 2024-09-10T18:55:33.063 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.063 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.063 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:33.063 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.410109+0000", 2024-09-10T18:55:33.064 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.807e-06 2024-09-10T18:55:33.064 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.064 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.064 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:33.064 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.410142+0000", 2024-09-10T18:55:33.064 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.2644000000000001e-05 2024-09-10T18:55:33.064 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.064 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.064 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:55:33.064 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.410161+0000", 2024-09-10T18:55:33.064 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.8749999999999998e-05 2024-09-10T18:55:33.064 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.064 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.065 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:55:33.065 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.537156+0000", 2024-09-10T18:55:33.065 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.12699532699999999 2024-09-10T18:55:33.065 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.065 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.065 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:33.065 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.537226+0000", 2024-09-10T18:55:33.065 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.0167999999999996e-05 2024-09-10T18:55:33.065 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.065 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:33.065 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.065 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.066 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.066 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_repop(client.24947.0:176771 8.f e202/158 8:faf992bb:::benchmark_data_smithi018_106531_object30141:head v 202'11218, mlcod=202'11215)", 2024-09-10T18:55:33.066 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:53:45.411170+0000", 2024-09-10T18:55:33.066 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 107.563631831, 2024-09-10T18:55:33.066 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.135313199, 2024-09-10T18:55:33.066 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:33.066 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:55:33.066 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:33.066 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.066 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:33.066 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:33.068 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.411170+0000", 2024-09-10T18:55:33.068 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.068 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.068 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.068 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:33.068 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.411170+0000", 2024-09-10T18:55:33.068 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.068 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.068 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.069 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:33.069 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.411169+0000", 2024-09-10T18:55:33.069 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999995 2024-09-10T18:55:33.069 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.069 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.069 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:33.069 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.411174+0000", 2024-09-10T18:55:33.069 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.7940000000000002e-06 2024-09-10T18:55:33.069 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.069 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.069 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:33.069 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.411181+0000", 2024-09-10T18:55:33.069 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.798e-06 2024-09-10T18:55:33.070 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.070 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.070 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:33.070 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.411184+0000", 2024-09-10T18:55:33.070 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.591e-06 2024-09-10T18:55:33.070 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.070 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.070 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:33.070 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.537296+0000", 2024-09-10T18:55:33.070 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.12611148799999999 2024-09-10T18:55:33.070 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.070 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.070 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:55:33.071 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.537361+0000", 2024-09-10T18:55:33.071 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.4957999999999997e-05 2024-09-10T18:55:33.071 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.071 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.071 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:55:33.071 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.546454+0000", 2024-09-10T18:55:33.071 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0090930490000000006 2024-09-10T18:55:33.071 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.071 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.071 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:33.071 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.546483+0000", 2024-09-10T18:55:33.071 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.9051e-05 2024-09-10T18:55:33.071 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.072 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:33.072 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.072 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.072 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.072 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "MOSDPGPush(8.0 202/194 [PushOp(8:063997cd:::benchmark_data_smithi018_106531_object90066:head, version: 176'5542, data_included: [0~65536], data_size: 65536, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(8:063997cd:::benchmark_data_smithi018_106531_object90066:head@176'5542, size: 65536, copy_subset: [0~65536], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:65536, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-10T18:55:33.072 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:53:45.411744+0000", 2024-09-10T18:55:33.072 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 107.563057284, 2024-09-10T18:55:33.072 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.126052043, 2024-09-10T18:55:33.072 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:33.072 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "started", 2024-09-10T18:55:33.072 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:33.072 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.072 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:33.073 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.411744+0000", 2024-09-10T18:55:33.073 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.073 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.073 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.073 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:33.073 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.411744+0000", 2024-09-10T18:55:33.073 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.073 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.073 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.073 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:33.073 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.411744+0000", 2024-09-10T18:55:33.073 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999995 2024-09-10T18:55:33.073 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.074 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.074 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:33.074 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.411818+0000", 2024-09-10T18:55:33.074 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.4406000000000006e-05 2024-09-10T18:55:33.074 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.074 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.074 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:33.074 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.537028+0000", 2024-09-10T18:55:33.074 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.12521027100000001 2024-09-10T18:55:33.074 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.074 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.074 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:33.074 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.537049+0000", 2024-09-10T18:55:33.075 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.0622999999999999e-05 2024-09-10T18:55:33.075 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.075 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.075 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:33.075 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.537586+0000", 2024-09-10T18:55:33.075 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00053682100000000004 2024-09-10T18:55:33.075 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.075 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.075 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:55:33.075 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.537612+0000", 2024-09-10T18:55:33.075 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.6400000000000001e-05 2024-09-10T18:55:33.075 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.075 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.076 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:33.076 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:53:45.537796+0000", 2024-09-10T18:55:33.076 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000183983 2024-09-10T18:55:33.076 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.076 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:33.076 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.076 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.076 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.076 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:194897 8.2 8:40155cf6:::benchmark_data_smithi018_106531_object48267:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e205)", 2024-09-10T18:55:33.076 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:54:00.261384+0000", 2024-09-10T18:55:33.076 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 92.713417395999997, 2024-09-10T18:55:33.076 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.068155805, 2024-09-10T18:55:33.076 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:33.077 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "reached pg", 2024-09-10T18:55:33.077 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:55:33.077 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:55:33.077 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:55:33.077 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 194897 2024-09-10T18:55:33.077 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.077 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:33.077 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.077 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:33.077 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.261384+0000", 2024-09-10T18:55:33.077 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.077 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.077 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.078 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:33.078 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.261384+0000", 2024-09-10T18:55:33.078 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.078 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.078 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.078 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:33.078 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.261382+0000", 2024-09-10T18:55:33.078 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999981 2024-09-10T18:55:33.078 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.078 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.078 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:33.078 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.261389+0000", 2024-09-10T18:55:33.078 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.4549999999999998e-06 2024-09-10T18:55:33.078 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.079 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.079 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:33.079 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.261402+0000", 2024-09-10T18:55:33.079 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.2532e-05 2024-09-10T18:55:33.079 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.079 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.079 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:33.079 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.261408+0000", 2024-09-10T18:55:33.079 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.263e-06 2024-09-10T18:55:33.079 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.079 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.079 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:33.079 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.261451+0000", 2024-09-10T18:55:33.079 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.2805000000000001e-05 2024-09-10T18:55:33.080 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.080 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.080 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:55:33.080 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.261552+0000", 2024-09-10T18:55:33.080 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00010141600000000001 2024-09-10T18:55:33.080 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.080 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.080 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,5", 2024-09-10T18:55:33.080 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.261654+0000", 2024-09-10T18:55:33.080 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00010126 2024-09-10T18:55:33.080 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.080 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.080 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:33.081 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.267121+0000", 2024-09-10T18:55:33.081 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0054670539999999998 2024-09-10T18:55:33.081 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.081 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.081 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:55:33.081 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.269163+0000", 2024-09-10T18:55:33.081 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0020424430000000001 2024-09-10T18:55:33.081 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.081 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.081 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:33.081 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.297708+0000", 2024-09-10T18:55:33.081 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.028544488999999999 2024-09-10T18:55:33.082 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.082 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.082 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:55:33.082 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.297715+0000", 2024-09-10T18:55:33.082 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.3130000000000001e-06 2024-09-10T18:55:33.082 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.082 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.082 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:33.082 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.329466+0000", 2024-09-10T18:55:33.082 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.031751203 2024-09-10T18:55:33.082 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.082 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.083 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:33.083 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.329540+0000", 2024-09-10T18:55:33.083 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.3632000000000002e-05 2024-09-10T18:55:33.083 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.083 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:33.083 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.083 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.083 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.083 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:194911 8.e 8:7e5954cf:::benchmark_data_smithi018_106531_object48281:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e206)", 2024-09-10T18:55:33.083 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:54:00.294645+0000", 2024-09-10T18:55:33.083 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 92.680155948000007, 2024-09-10T18:55:33.083 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.037229145, 2024-09-10T18:55:33.084 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:33.084 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "reached pg", 2024-09-10T18:55:33.084 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:55:33.084 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:55:33.084 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:55:33.084 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 194911 2024-09-10T18:55:33.084 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.084 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:33.084 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.084 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:33.084 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294645+0000", 2024-09-10T18:55:33.084 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.084 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.084 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.085 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:33.085 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294645+0000", 2024-09-10T18:55:33.085 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.085 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.085 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.085 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:33.085 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294644+0000", 2024-09-10T18:55:33.085 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999981 2024-09-10T18:55:33.085 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.085 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.085 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:33.085 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294651+0000", 2024-09-10T18:55:33.086 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.7729999999999999e-06 2024-09-10T18:55:33.086 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.086 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.086 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:33.086 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294664+0000", 2024-09-10T18:55:33.086 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.2401999999999999e-05 2024-09-10T18:55:33.086 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.086 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.086 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:33.086 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294672+0000", 2024-09-10T18:55:33.086 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.2870000000000004e-06 2024-09-10T18:55:33.086 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.086 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.087 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:33.087 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.295300+0000", 2024-09-10T18:55:33.087 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00062796 2024-09-10T18:55:33.087 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.087 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.087 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:55:33.087 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.295307+0000", 2024-09-10T18:55:33.087 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.2779999999999996e-06 2024-09-10T18:55:33.087 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.087 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.087 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:33.087 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.330427+0000", 2024-09-10T18:55:33.087 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.03511982 2024-09-10T18:55:33.088 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.088 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.088 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:33.088 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.331875+0000", 2024-09-10T18:55:33.088 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.001447427 2024-09-10T18:55:33.088 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.088 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:33.088 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.088 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.088 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.088 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:194919 8.2 8:4d34d6e8:::benchmark_data_smithi018_106531_object48289:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e206)", 2024-09-10T18:55:33.088 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:54:00.294686+0000", 2024-09-10T18:55:33.089 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 92.680115478000005, 2024-09-10T18:55:33.089 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0404138540000001, 2024-09-10T18:55:33.089 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:33.089 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:55:33.089 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:55:33.089 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:55:33.089 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:55:33.089 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 194919 2024-09-10T18:55:33.089 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.089 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:33.089 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.089 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:33.089 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294686+0000", 2024-09-10T18:55:33.090 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.090 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.090 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.090 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:33.090 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294686+0000", 2024-09-10T18:55:33.090 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.090 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.090 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.090 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:33.090 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294685+0000", 2024-09-10T18:55:33.090 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999995 2024-09-10T18:55:33.091 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.091 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.091 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:33.091 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294689+0000", 2024-09-10T18:55:33.091 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.737e-06 2024-09-10T18:55:33.091 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.091 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.092 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:33.092 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294695+0000", 2024-09-10T18:55:33.092 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.3400000000000003e-06 2024-09-10T18:55:33.092 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.092 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.092 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:33.092 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294700+0000", 2024-09-10T18:55:33.092 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.1339999999999997e-06 2024-09-10T18:55:33.092 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.092 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.093 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:33.093 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.299920+0000", 2024-09-10T18:55:33.093 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.005220064 2024-09-10T18:55:33.093 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.093 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.093 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:55:33.093 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.299926+0000", 2024-09-10T18:55:33.093 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.9789999999999998e-06 2024-09-10T18:55:33.093 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.093 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.093 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:33.093 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.329569+0000", 2024-09-10T18:55:33.094 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.02964327 2024-09-10T18:55:33.094 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.094 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.094 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:55:33.094 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.329629+0000", 2024-09-10T18:55:33.094 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.0470000000000002e-05 2024-09-10T18:55:33.094 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.094 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.094 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 5", 2024-09-10T18:55:33.094 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.329727+0000", 2024-09-10T18:55:33.094 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.7558000000000003e-05 2024-09-10T18:55:33.094 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.094 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.095 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:33.095 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.334622+0000", 2024-09-10T18:55:33.095 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0048955750000000001 2024-09-10T18:55:33.095 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.095 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.095 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:55:33.095 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.334960+0000", 2024-09-10T18:55:33.095 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00033784 2024-09-10T18:55:33.095 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.095 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.095 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:55:33.095 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.335009+0000", 2024-09-10T18:55:33.095 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.8656000000000003e-05 2024-09-10T18:55:33.096 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.096 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.096 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:33.096 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.335100+0000", 2024-09-10T18:55:33.096 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.0801e-05 2024-09-10T18:55:33.096 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.096 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:33.096 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.096 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.096 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.096 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:194946 8.2 8:469a9e0a:::benchmark_data_smithi018_106531_object48316:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e206)", 2024-09-10T18:55:33.096 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:54:00.320261+0000", 2024-09-10T18:55:33.097 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 92.654540030000007, 2024-09-10T18:55:33.097 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0152080210000001, 2024-09-10T18:55:33.097 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:33.097 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:55:33.097 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:55:33.097 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:55:33.097 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:55:33.097 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 194946 2024-09-10T18:55:33.097 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.097 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:33.097 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.097 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:33.097 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320261+0000", 2024-09-10T18:55:33.097 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.098 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.098 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.098 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:33.098 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320261+0000", 2024-09-10T18:55:33.098 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.098 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.098 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.098 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:33.098 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320260+0000", 2024-09-10T18:55:33.098 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:55:33.098 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.098 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.098 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:33.099 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320266+0000", 2024-09-10T18:55:33.099 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.9780000000000002e-06 2024-09-10T18:55:33.099 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.099 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.099 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:33.099 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320275+0000", 2024-09-10T18:55:33.099 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.867e-06 2024-09-10T18:55:33.099 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.099 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.099 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:33.099 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320280+0000", 2024-09-10T18:55:33.099 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.4970000000000003e-06 2024-09-10T18:55:33.099 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.100 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.100 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:33.100 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320314+0000", 2024-09-10T18:55:33.100 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.4644999999999998e-05 2024-09-10T18:55:33.100 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.100 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.100 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:55:33.100 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320320+0000", 2024-09-10T18:55:33.100 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.2500000000000003e-06 2024-09-10T18:55:33.100 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.100 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.100 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:33.100 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.331780+0000", 2024-09-10T18:55:33.100 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.01145964 2024-09-10T18:55:33.101 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.101 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.101 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:55:33.101 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.331913+0000", 2024-09-10T18:55:33.101 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00013310700000000001 2024-09-10T18:55:33.101 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.101 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.101 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 5", 2024-09-10T18:55:33.101 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.332054+0000", 2024-09-10T18:55:33.101 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000141013 2024-09-10T18:55:33.101 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.101 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.102 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:55:33.102 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.335153+0000", 2024-09-10T18:55:33.102 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0030985829999999998 2024-09-10T18:55:33.102 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.102 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.102 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:33.102 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.335316+0000", 2024-09-10T18:55:33.102 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000162879 2024-09-10T18:55:33.102 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.102 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.102 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:55:33.103 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.335368+0000", 2024-09-10T18:55:33.103 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.2509999999999997e-05 2024-09-10T18:55:33.103 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.103 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.103 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:33.103 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.335469+0000", 2024-09-10T18:55:33.103 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00010118 2024-09-10T18:55:33.103 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.103 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:33.103 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.103 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.103 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.103 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:194947 8.e 8:7cb25091:::benchmark_data_smithi018_106531_object48317:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e206)", 2024-09-10T18:55:33.104 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:54:00.320324+0000", 2024-09-10T18:55:33.104 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 92.654477251000003, 2024-09-10T18:55:33.104 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0290095960000001, 2024-09-10T18:55:33.104 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:33.104 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:55:33.104 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:55:33.104 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:55:33.104 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:55:33.104 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 194947 2024-09-10T18:55:33.104 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.104 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:33.104 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.105 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:33.105 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320324+0000", 2024-09-10T18:55:33.105 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.105 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.105 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.105 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:33.105 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320324+0000", 2024-09-10T18:55:33.105 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.105 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.105 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.105 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:33.105 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320323+0000", 2024-09-10T18:55:33.106 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:55:33.106 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.106 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.106 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:33.106 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320329+0000", 2024-09-10T18:55:33.106 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.9649999999999996e-06 2024-09-10T18:55:33.106 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.106 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.106 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:33.106 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320339+0000", 2024-09-10T18:55:33.106 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0263e-05 2024-09-10T18:55:33.106 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.106 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.106 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:33.107 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320345+0000", 2024-09-10T18:55:33.107 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.0419999999999999e-06 2024-09-10T18:55:33.107 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.107 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.107 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:33.107 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320372+0000", 2024-09-10T18:55:33.107 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.6452e-05 2024-09-10T18:55:33.107 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.107 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.107 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:55:33.107 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320375+0000", 2024-09-10T18:55:33.107 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.5810000000000001e-06 2024-09-10T18:55:33.107 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.108 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.108 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:33.108 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.332371+0000", 2024-09-10T18:55:33.108 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.011995341 2024-09-10T18:55:33.108 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.108 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.108 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:55:33.108 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.332852+0000", 2024-09-10T18:55:33.108 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00048134099999999998 2024-09-10T18:55:33.108 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.108 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.108 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 7", 2024-09-10T18:55:33.108 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.333084+0000", 2024-09-10T18:55:33.109 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00023146899999999999 2024-09-10T18:55:33.109 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.109 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.109 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:55:33.109 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.346512+0000", 2024-09-10T18:55:33.109 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.013428697 2024-09-10T18:55:33.109 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.109 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.109 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:33.109 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.349159+0000", 2024-09-10T18:55:33.109 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0026464219999999998 2024-09-10T18:55:33.109 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.110 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.110 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:55:33.110 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.349241+0000", 2024-09-10T18:55:33.110 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.2155000000000002e-05 2024-09-10T18:55:33.110 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.110 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.110 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:33.110 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.349334+0000", 2024-09-10T18:55:33.110 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.2892000000000004e-05 2024-09-10T18:55:33.110 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.110 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:33.110 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.110 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.110 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.111 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.14415.0:1339 5.e 5:7759931f:::notify.3:head [watch reconnect cookie 94413763299104 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e206)", 2024-09-10T18:55:33.111 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:54:00.834762+0000", 2024-09-10T18:55:33.111 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 92.140039329999993, 2024-09-10T18:55:33.111 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.47194119299999998, 2024-09-10T18:55:33.111 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:33.111 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "started", 2024-09-10T18:55:33.111 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:55:33.111 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.14415", 2024-09-10T18:55:33.111 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/2965248470", 2024-09-10T18:55:33.111 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 1339 2024-09-10T18:55:33.111 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.111 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:33.111 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.111 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:33.112 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.834762+0000", 2024-09-10T18:55:33.112 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.112 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.112 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.112 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:33.112 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.834762+0000", 2024-09-10T18:55:33.112 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:33.112 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.112 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.112 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:33.112 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.834761+0000", 2024-09-10T18:55:33.112 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:55:33.113 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.113 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.113 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:33.113 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.834768+0000", 2024-09-10T18:55:33.113 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.7389999999999999e-06 2024-09-10T18:55:33.113 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.113 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.113 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:33.113 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.834777+0000", 2024-09-10T18:55:33.113 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.5459999999999992e-06 2024-09-10T18:55:33.113 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.113 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.113 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:33.114 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.834782+0000", 2024-09-10T18:55:33.114 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.9289999999999997e-06 2024-09-10T18:55:33.114 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.114 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.114 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:33.114 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.834833+0000", 2024-09-10T18:55:33.114 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.1635999999999998e-05 2024-09-10T18:55:33.114 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.114 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.114 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:55:33.114 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.834838+0000", 2024-09-10T18:55:33.114 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.8319999999999996e-06 2024-09-10T18:55:33.114 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.114 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.115 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:33.115 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.297583+0000", 2024-09-10T18:55:33.115 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.46274470899999998 2024-09-10T18:55:33.115 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.115 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.115 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:55:33.115 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.297740+0000", 2024-09-10T18:55:33.115 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00015732699999999999 2024-09-10T18:55:33.115 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:33.115 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:33.115 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:33.115 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.306703+0000", 2024-09-10T18:55:33.115 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0089629159999999996 2024-09-10T18:55:33.115 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.116 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:33.116 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.116 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:33.116 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:33.116 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T18:55:33.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:33.139 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:33.139 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:33.240 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:33.262 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:33.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:33.262 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:33.363 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:33.388 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:33.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:33.389 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:33.490 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:33.522 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:33.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:33.523 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:33.623 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:33.645 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:33.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:33.646 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:33.747 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:33.752 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-10T18:55:33.768 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:33.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:33.770 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:33.871 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:33.892 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:33.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:33.894 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:33.994 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:34.017 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:34.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:34.018 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:34.059 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:55:34.119 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:34.140 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:34.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:34.141 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:34.242 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:34.263 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:34.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:34.264 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:34.337 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:55:34 smithi018 podman[130858]: 2024-09-10 18:55:34.173145763 +0000 UTC m=+4.131999934 container init e4449adc28b27c6b3584f3c0692e354e893cf1b14bf0ccb8ab08036ad1180169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:55:34.337 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:55:34 smithi018 podman[130858]: 2024-09-10 18:55:34.177545926 +0000 UTC m=+4.136400085 container start e4449adc28b27c6b3584f3c0692e354e893cf1b14bf0ccb8ab08036ad1180169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-10T18:55:34.337 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:55:34 smithi018 ceph-osd[131340]: -- 172.21.15.18:0/1694895493 <== mon.0 v2:172.21.15.18:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556e255bad00 con 0x556e2558d400 2024-09-10T18:55:34.365 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:34.390 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:34.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:34.392 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:34.479 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:34 smithi103 ceph-mon[33163]: pgmap v457: 152 pgs: 41 active+undersized, 22 active+undersized+degraded, 89 active+clean; 404 KiB data, 4.9 GiB used, 621 GiB / 626 GiB avail; 682 B/s rd, 0 op/s; 78/588 objects degraded (13.265%) 2024-09-10T18:55:34.479 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:34 smithi103 ceph-mon[33163]: Health check update: Degraded data redundancy: 78/588 objects degraded (13.265%), 22 pgs degraded, 63 pgs undersized (PG_DEGRADED) 2024-09-10T18:55:34.479 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:34 smithi103 ceph-mon[33163]: osdmap e236: 8 total, 7 up, 7 in 2024-09-10T18:55:34.479 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:34 smithi103 ceph-mon[33163]: 4.14 scrub starts 2024-09-10T18:55:34.479 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:34 smithi103 ceph-mon[33163]: 4.14 scrub ok 2024-09-10T18:55:34.492 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:34.514 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:34.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:34.515 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:34.607 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:34 smithi018 ceph-mon[29336]: pgmap v457: 152 pgs: 41 active+undersized, 22 active+undersized+degraded, 89 active+clean; 404 KiB data, 4.9 GiB used, 621 GiB / 626 GiB avail; 682 B/s rd, 0 op/s; 78/588 objects degraded (13.265%) 2024-09-10T18:55:34.607 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:34 smithi018 ceph-mon[29336]: Health check update: Degraded data redundancy: 78/588 objects degraded (13.265%), 22 pgs degraded, 63 pgs undersized (PG_DEGRADED) 2024-09-10T18:55:34.607 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:34 smithi018 ceph-mon[29336]: osdmap e236: 8 total, 7 up, 7 in 2024-09-10T18:55:34.607 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:34 smithi018 ceph-mon[29336]: 4.14 scrub starts 2024-09-10T18:55:34.607 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:34 smithi018 ceph-mon[29336]: 4.14 scrub ok 2024-09-10T18:55:34.607 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:34 smithi018 ceph-mon[40196]: pgmap v457: 152 pgs: 41 active+undersized, 22 active+undersized+degraded, 89 active+clean; 404 KiB data, 4.9 GiB used, 621 GiB / 626 GiB avail; 682 B/s rd, 0 op/s; 78/588 objects degraded (13.265%) 2024-09-10T18:55:34.607 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:34 smithi018 ceph-mon[40196]: Health check update: Degraded data redundancy: 78/588 objects degraded (13.265%), 22 pgs degraded, 63 pgs undersized (PG_DEGRADED) 2024-09-10T18:55:34.607 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:34 smithi018 ceph-mon[40196]: osdmap e236: 8 total, 7 up, 7 in 2024-09-10T18:55:34.608 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:34 smithi018 ceph-mon[40196]: 4.14 scrub starts 2024-09-10T18:55:34.608 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:34 smithi018 ceph-mon[40196]: 4.14 scrub ok 2024-09-10T18:55:34.616 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:34.640 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:34.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:34.641 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:34.742 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-10T18:55:34.742 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:34.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:34.766 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:34.767 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:34.863 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:55:34 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0[130951]: 2024-09-10T18:55:34.709+0000 7fbd3239d440 -1 Falling back to public interface 2024-09-10T18:55:34.863 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:55:34 smithi018 bash[130858]: e4449adc28b27c6b3584f3c0692e354e893cf1b14bf0ccb8ab08036ad1180169 2024-09-10T18:55:34.863 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:55:34 smithi018 systemd[1]: Started Ceph osd.0 for ba327f6c-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:55:34.867 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:34.890 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:34.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:34.891 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:34.979 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:55:34.979 INFO:teuthology.orchestra.run.smithi018.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-10T18:55:34.979 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:55:34.984 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:55:34.984 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:55:34.992 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:35.002 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:55:35.002 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:55:35.014 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:35.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:35.015 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:35.019 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:55:35.019 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:55:35.043 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:55:35.043 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:55:35.059 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:55:35.059 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:55:35.072 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:55:35.076 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:55:35.076 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:55:35.115 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:35.137 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:35.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:35.138 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:35.239 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:35.263 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:35.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:35.264 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:35.347 INFO:teuthology.orchestra.run.smithi018.stderr:pool 'unique_pool_2' created 2024-09-10T18:55:35.364 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:35.386 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:35.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:35.387 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:35.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:35 smithi103 ceph-mon[33163]: 5.18 scrub starts 2024-09-10T18:55:35.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:35 smithi103 ceph-mon[33163]: 5.18 scrub ok 2024-09-10T18:55:35.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:35 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2245987968' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]: dispatch 2024-09-10T18:55:35.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:35 smithi103 ceph-mon[33163]: 1.14 scrub starts 2024-09-10T18:55:35.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:35 smithi103 ceph-mon[33163]: 1.14 scrub ok 2024-09-10T18:55:35.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:35 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2467313982' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:55:35.488 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:35.509 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:35.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:35.510 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:35.611 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:35.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:35.636 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:35.636 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:35.639 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T18:55:35.639 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:55:35.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:35 smithi018 ceph-mon[40196]: 5.18 scrub starts 2024-09-10T18:55:35.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:35 smithi018 ceph-mon[40196]: 5.18 scrub ok 2024-09-10T18:55:35.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:35 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2245987968' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]: dispatch 2024-09-10T18:55:35.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:35 smithi018 ceph-mon[40196]: 1.14 scrub starts 2024-09-10T18:55:35.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:35 smithi018 ceph-mon[40196]: 1.14 scrub ok 2024-09-10T18:55:35.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:35 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2467313982' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:55:35.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:35 smithi018 ceph-mon[29336]: 5.18 scrub starts 2024-09-10T18:55:35.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:35 smithi018 ceph-mon[29336]: 5.18 scrub ok 2024-09-10T18:55:35.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:35 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2245987968' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]: dispatch 2024-09-10T18:55:35.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:35 smithi018 ceph-mon[29336]: 1.14 scrub starts 2024-09-10T18:55:35.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:35 smithi018 ceph-mon[29336]: 1.14 scrub ok 2024-09-10T18:55:35.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:35 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2467313982' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:55:35.737 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:35.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:35.759 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:35.759 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:35.821 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-10T18:55:35.860 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:35.883 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:35.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:35.884 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:35.985 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:36.008 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:36.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:36.009 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:36.110 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-10T18:55:36.110 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-10T18:55:36.111 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:36.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:36.134 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:36.134 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:36.235 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:36.258 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:36.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:36.259 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:36.308 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-10T18:55:36.360 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:36.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:36.382 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:36.383 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:36.388 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:36 smithi018 ceph-mon[40196]: 2.11 scrub starts 2024-09-10T18:55:36.388 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:36 smithi018 ceph-mon[40196]: 2.11 scrub ok 2024-09-10T18:55:36.388 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:36 smithi018 ceph-mon[40196]: pgmap v459: 152 pgs: 41 active+undersized, 22 active+undersized+degraded, 89 active+clean; 404 KiB data, 4.9 GiB used, 621 GiB / 626 GiB avail; 774 B/s rd, 0 op/s; 78/588 objects degraded (13.265%) 2024-09-10T18:55:36.388 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:36 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2245987968' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]': finished 2024-09-10T18:55:36.388 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:36 smithi018 ceph-mon[40196]: osdmap e237: 8 total, 7 up, 7 in 2024-09-10T18:55:36.389 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:36 smithi018 ceph-mon[29336]: 2.11 scrub starts 2024-09-10T18:55:36.389 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:36 smithi018 ceph-mon[29336]: 2.11 scrub ok 2024-09-10T18:55:36.389 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:36 smithi018 ceph-mon[29336]: pgmap v459: 152 pgs: 41 active+undersized, 22 active+undersized+degraded, 89 active+clean; 404 KiB data, 4.9 GiB used, 621 GiB / 626 GiB avail; 774 B/s rd, 0 op/s; 78/588 objects degraded (13.265%) 2024-09-10T18:55:36.389 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:36 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2245987968' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]': finished 2024-09-10T18:55:36.389 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:36 smithi018 ceph-mon[29336]: osdmap e237: 8 total, 7 up, 7 in 2024-09-10T18:55:36.484 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:36.507 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:36.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:36.508 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:36.564 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:55:36.608 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:36.630 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:36 smithi103 ceph-mon[33163]: 2.11 scrub starts 2024-09-10T18:55:36.631 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:36 smithi103 ceph-mon[33163]: 2.11 scrub ok 2024-09-10T18:55:36.631 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:36 smithi103 ceph-mon[33163]: pgmap v459: 152 pgs: 41 active+undersized, 22 active+undersized+degraded, 89 active+clean; 404 KiB data, 4.9 GiB used, 621 GiB / 626 GiB avail; 774 B/s rd, 0 op/s; 78/588 objects degraded (13.265%) 2024-09-10T18:55:36.631 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:36 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2245987968' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]': finished 2024-09-10T18:55:36.631 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:36 smithi103 ceph-mon[33163]: osdmap e237: 8 total, 7 up, 7 in 2024-09-10T18:55:36.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:36.632 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:36.632 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:36.733 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:36.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:36.755 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:36.755 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:36.856 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:36.878 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:36.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:36.880 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:36.929 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:55:36 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:55:36.981 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:37.003 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:37.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:37.004 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:37.105 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:37.127 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:37.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:37.128 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:37.229 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:37.250 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:37.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:37.252 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:37.352 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:37.374 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:37.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:37.376 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:37.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:37 smithi018 ceph-mon[29336]: osdmap e238: 8 total, 7 up, 7 in 2024-09-10T18:55:37.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:37 smithi018 ceph-mon[29336]: 5.14 scrub starts 2024-09-10T18:55:37.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:37 smithi018 ceph-mon[29336]: 5.14 scrub ok 2024-09-10T18:55:37.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:37 smithi018 ceph-mon[40196]: osdmap e238: 8 total, 7 up, 7 in 2024-09-10T18:55:37.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:37 smithi018 ceph-mon[40196]: 5.14 scrub starts 2024-09-10T18:55:37.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:37 smithi018 ceph-mon[40196]: 5.14 scrub ok 2024-09-10T18:55:37.477 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:37.498 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:37.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:37.499 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:37.600 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:37.621 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:37.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:37.622 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:37.667 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:37 smithi103 ceph-mon[33163]: osdmap e238: 8 total, 7 up, 7 in 2024-09-10T18:55:37.667 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:37 smithi103 ceph-mon[33163]: 5.14 scrub starts 2024-09-10T18:55:37.667 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:37 smithi103 ceph-mon[33163]: 5.14 scrub ok 2024-09-10T18:55:37.723 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:37.747 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:37.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:37.748 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:37.767 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd pool application enable unique_pool_2 rados --yes-i-really-mean-it || true 2024-09-10T18:55:37.772 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:55:37.849 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:37.870 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:37.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:37.871 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:37.972 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:37.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:37.997 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:37.997 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:38.015 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:55:37 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0[130951]: 2024-09-10T18:55:37.965+0000 7fbd3239d440 -1 osd.0 205 log_to_monitors true 2024-09-10T18:55:38.098 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:38.111 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T18:55:38.121 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:38.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:38.122 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:38.166 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T18:55:38.166 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:55:38.222 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:38.245 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:38.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:38.246 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:38.347 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:38.375 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:38.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:38.376 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:38.476 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:38.496 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T18:55:38.496 INFO:teuthology.orchestra.run.smithi018.stdout: "ops": [], 2024-09-10T18:55:38.496 INFO:teuthology.orchestra.run.smithi018.stdout: "num_ops": 0 2024-09-10T18:55:38.496 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T18:55:38.499 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:38.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:38.500 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:38.601 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:38.623 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:38.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:38.624 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:38.694 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 18:55:38 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0[130951]: 2024-09-10T18:55:38.369+0000 7fbd27143640 -1 osd.0 205 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:55:38.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:38 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:38.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:38 smithi018 ceph-mon[40196]: pgmap v462: 168 pgs: 5 creating+peering, 44 active+undersized, 22 active+undersized+degraded, 97 active+clean; 608 KiB data, 4.9 GiB used, 621 GiB / 626 GiB avail; 78/591 objects degraded (13.198%) 2024-09-10T18:55:38.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:38 smithi018 ceph-mon[40196]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:55:38.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:38 smithi018 ceph-mon[40196]: osdmap e239: 8 total, 7 up, 7 in 2024-09-10T18:55:38.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:38 smithi018 ceph-mon[40196]: 5.9 scrub starts 2024-09-10T18:55:38.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:38 smithi018 ceph-mon[40196]: 5.9 scrub ok 2024-09-10T18:55:38.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:38 smithi018 ceph-mon[40196]: 1.a deep-scrub starts 2024-09-10T18:55:38.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:38 smithi018 ceph-mon[40196]: 1.a deep-scrub ok 2024-09-10T18:55:38.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:38 smithi018 ceph-mon[40196]: from='osd.0 v2:172.21.15.18:6801/4100931169' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T18:55:38.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:38 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:38.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:38 smithi018 ceph-mon[29336]: pgmap v462: 168 pgs: 5 creating+peering, 44 active+undersized, 22 active+undersized+degraded, 97 active+clean; 608 KiB data, 4.9 GiB used, 621 GiB / 626 GiB avail; 78/591 objects degraded (13.198%) 2024-09-10T18:55:38.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:38 smithi018 ceph-mon[29336]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:55:38.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:38 smithi018 ceph-mon[29336]: osdmap e239: 8 total, 7 up, 7 in 2024-09-10T18:55:38.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:38 smithi018 ceph-mon[29336]: 5.9 scrub starts 2024-09-10T18:55:38.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:38 smithi018 ceph-mon[29336]: 5.9 scrub ok 2024-09-10T18:55:38.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:38 smithi018 ceph-mon[29336]: 1.a deep-scrub starts 2024-09-10T18:55:38.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:38 smithi018 ceph-mon[29336]: 1.a deep-scrub ok 2024-09-10T18:55:38.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:38 smithi018 ceph-mon[29336]: from='osd.0 v2:172.21.15.18:6801/4100931169' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T18:55:38.716 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:38 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:38.716 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:38 smithi103 ceph-mon[33163]: pgmap v462: 168 pgs: 5 creating+peering, 44 active+undersized, 22 active+undersized+degraded, 97 active+clean; 608 KiB data, 4.9 GiB used, 621 GiB / 626 GiB avail; 78/591 objects degraded (13.198%) 2024-09-10T18:55:38.716 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:38 smithi103 ceph-mon[33163]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:55:38.716 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:38 smithi103 ceph-mon[33163]: osdmap e239: 8 total, 7 up, 7 in 2024-09-10T18:55:38.716 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:38 smithi103 ceph-mon[33163]: 5.9 scrub starts 2024-09-10T18:55:38.716 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:38 smithi103 ceph-mon[33163]: 5.9 scrub ok 2024-09-10T18:55:38.716 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:38 smithi103 ceph-mon[33163]: 1.a deep-scrub starts 2024-09-10T18:55:38.716 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:38 smithi103 ceph-mon[33163]: 1.a deep-scrub ok 2024-09-10T18:55:38.717 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:38 smithi103 ceph-mon[33163]: from='osd.0 v2:172.21.15.18:6801/4100931169' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T18:55:38.725 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:38.742 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-10T18:55:38.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:38.749 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:38.749 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:38.850 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:38.873 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:38.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:38.874 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:38.975 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:38.994 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:55:38.996 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:38.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:38.997 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:39.098 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:39.121 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:39.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:39.122 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:39.223 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:39.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:39.246 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:39.246 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:39.347 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:55:39.378 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:39.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:39.379 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:55:39.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:39 smithi018 ceph-mon[29336]: 6.e scrub starts 2024-09-10T18:55:39.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:39 smithi018 ceph-mon[29336]: 6.e scrub ok 2024-09-10T18:55:39.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:39 smithi018 ceph-mon[29336]: from='osd.0 v2:172.21.15.18:6801/4100931169' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T18:55:39.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:39 smithi018 ceph-mon[29336]: osdmap e240: 8 total, 7 up, 7 in 2024-09-10T18:55:39.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:39 smithi018 ceph-mon[29336]: from='osd.0 v2:172.21.15.18:6801/4100931169' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-09-10T18:55:39.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:39 smithi018 ceph-mon[29336]: Health check update: Degraded data redundancy: 78/591 objects degraded (13.198%), 22 pgs degraded, 63 pgs undersized (PG_DEGRADED) 2024-09-10T18:55:39.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:39 smithi018 ceph-mon[29336]: 5.6 scrub starts 2024-09-10T18:55:39.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:39 smithi018 ceph-mon[29336]: 5.6 scrub ok 2024-09-10T18:55:39.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:39 smithi018 ceph-mon[40196]: 6.e scrub starts 2024-09-10T18:55:39.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:39 smithi018 ceph-mon[40196]: 6.e scrub ok 2024-09-10T18:55:39.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:39 smithi018 ceph-mon[40196]: from='osd.0 v2:172.21.15.18:6801/4100931169' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T18:55:39.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:39 smithi018 ceph-mon[40196]: osdmap e240: 8 total, 7 up, 7 in 2024-09-10T18:55:39.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:39 smithi018 ceph-mon[40196]: from='osd.0 v2:172.21.15.18:6801/4100931169' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-09-10T18:55:39.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:39 smithi018 ceph-mon[40196]: Health check update: Degraded data redundancy: 78/591 objects degraded (13.198%), 22 pgs degraded, 63 pgs undersized (PG_DEGRADED) 2024-09-10T18:55:39.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:39 smithi018 ceph-mon[40196]: 5.6 scrub starts 2024-09-10T18:55:39.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:39 smithi018 ceph-mon[40196]: 5.6 scrub ok 2024-09-10T18:55:39.480 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:39.502 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:39.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:39.503 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:39.604 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:39.626 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:39.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:39.627 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:39.728 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:39.751 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:39.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:39.752 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:39.768 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:39 smithi103 ceph-mon[33163]: 6.e scrub starts 2024-09-10T18:55:39.769 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:39 smithi103 ceph-mon[33163]: 6.e scrub ok 2024-09-10T18:55:39.769 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:39 smithi103 ceph-mon[33163]: from='osd.0 v2:172.21.15.18:6801/4100931169' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T18:55:39.769 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:39 smithi103 ceph-mon[33163]: osdmap e240: 8 total, 7 up, 7 in 2024-09-10T18:55:39.769 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:39 smithi103 ceph-mon[33163]: from='osd.0 v2:172.21.15.18:6801/4100931169' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]: dispatch 2024-09-10T18:55:39.769 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:39 smithi103 ceph-mon[33163]: Health check update: Degraded data redundancy: 78/591 objects degraded (13.198%), 22 pgs degraded, 63 pgs undersized (PG_DEGRADED) 2024-09-10T18:55:39.769 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:39 smithi103 ceph-mon[33163]: 5.6 scrub starts 2024-09-10T18:55:39.769 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:39 smithi103 ceph-mon[33163]: 5.6 scrub ok 2024-09-10T18:55:39.820 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:55:39.853 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:39.876 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:39.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:39.877 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:39.978 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:40.001 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:40.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:40.003 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:40.103 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:40.125 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:40.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:40.126 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:40.227 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:55:40.249 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:40.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:40.250 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:55:40.289 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:55:40 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:55:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:55:40.352 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:40.374 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:40.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:40.375 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:40.476 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:40.498 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:40.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:40.499 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:40.561 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T18:55:40.561 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:55:40.562 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:40 smithi103 ceph-mon[33163]: pgmap v465: 168 pgs: 5 creating+peering, 44 active+undersized, 22 active+undersized+degraded, 97 active+clean; 608 KiB data, 4.8 GiB used, 621 GiB / 626 GiB avail; 78/591 objects degraded (13.198%) 2024-09-10T18:55:40.562 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:40 smithi103 ceph-mon[33163]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-10T18:55:40.562 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:40 smithi103 ceph-mon[33163]: osd.0 v2:172.21.15.18:6801/4100931169 boot 2024-09-10T18:55:40.562 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:40 smithi103 ceph-mon[33163]: osdmap e241: 8 total, 8 up, 7 in 2024-09-10T18:55:40.562 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:40 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:55:40.562 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:40 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:55:40.562 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:40 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:40.600 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:55:40.622 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:40.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:40.623 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:55:40.692 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:40 smithi018 ceph-mon[40196]: pgmap v465: 168 pgs: 5 creating+peering, 44 active+undersized, 22 active+undersized+degraded, 97 active+clean; 608 KiB data, 4.8 GiB used, 621 GiB / 626 GiB avail; 78/591 objects degraded (13.198%) 2024-09-10T18:55:40.692 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:40 smithi018 ceph-mon[40196]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-10T18:55:40.692 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:40 smithi018 ceph-mon[40196]: osd.0 v2:172.21.15.18:6801/4100931169 boot 2024-09-10T18:55:40.692 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:40 smithi018 ceph-mon[40196]: osdmap e241: 8 total, 8 up, 7 in 2024-09-10T18:55:40.692 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:40 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:55:40.693 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:40 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:55:40.693 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:40 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:40.693 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:40 smithi018 ceph-mon[29336]: pgmap v465: 168 pgs: 5 creating+peering, 44 active+undersized, 22 active+undersized+degraded, 97 active+clean; 608 KiB data, 4.8 GiB used, 621 GiB / 626 GiB avail; 78/591 objects degraded (13.198%) 2024-09-10T18:55:40.693 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:40 smithi018 ceph-mon[29336]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-10T18:55:40.693 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:40 smithi018 ceph-mon[29336]: osd.0 v2:172.21.15.18:6801/4100931169 boot 2024-09-10T18:55:40.693 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:40 smithi018 ceph-mon[29336]: osdmap e241: 8 total, 8 up, 7 in 2024-09-10T18:55:40.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:40 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:55:40.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:40 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:55:40.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:40 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:40.724 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:40.761 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:40.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:40.764 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:40.865 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:40.886 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:40.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:40.887 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:40.987 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:40.995 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:55:41.008 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:41.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:41.009 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:41.110 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:55:41.133 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:41.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:41.134 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:55:41.223 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-10T18:55:41.226 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:55:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:55:40] "GET /metrics HTTP/1.1" 200 223597 "" "Prometheus/2.43.0" 2024-09-10T18:55:41.235 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:41.287 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:41.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:41.289 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:41.390 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:55:41.392 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is set 2024-09-10T18:55:41.419 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:41.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:41.420 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:55:41.470 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:55:41.516 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:41 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:55:41.517 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:41 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:55:41.517 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:41 smithi018 ceph-mon[29336]: 6.1f deep-scrub starts 2024-09-10T18:55:41.517 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:41 smithi018 ceph-mon[29336]: 6.1f deep-scrub ok 2024-09-10T18:55:41.517 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:41 smithi018 ceph-mon[29336]: osdmap e242: 8 total, 8 up, 7 in 2024-09-10T18:55:41.517 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:41 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1191890424' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:55:41.517 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:41 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:55:41.517 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:41 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:55:41.517 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:41 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:55:41.518 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:41 smithi018 ceph-mon[40196]: 6.1f deep-scrub starts 2024-09-10T18:55:41.518 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:41 smithi018 ceph-mon[40196]: 6.1f deep-scrub ok 2024-09-10T18:55:41.518 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:41 smithi018 ceph-mon[40196]: osdmap e242: 8 total, 8 up, 7 in 2024-09-10T18:55:41.518 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:41 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1191890424' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:55:41.518 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:41 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:55:41.521 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:41.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:41 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:55:41.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:41 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:55:41.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:41 smithi103 ceph-mon[33163]: 6.1f deep-scrub starts 2024-09-10T18:55:41.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:41 smithi103 ceph-mon[33163]: 6.1f deep-scrub ok 2024-09-10T18:55:41.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:41 smithi103 ceph-mon[33163]: osdmap e242: 8 total, 8 up, 7 in 2024-09-10T18:55:41.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:41 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1191890424' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:55:41.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:41 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:55:41.544 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:41.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:41.545 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:41.646 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:41.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:41.668 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:41.668 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:41.769 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:41.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:41.792 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:41.792 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:41.893 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:41.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:41.917 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:41.917 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:42.018 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:42.039 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:42.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:42.040 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:42.141 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:42.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:42.165 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:42.165 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:42.266 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:42.288 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:42.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:42.289 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:42.390 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:42.413 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:42.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:42.414 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:42.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:42 smithi018 ceph-mon[29336]: pgmap v468: 168 pgs: 5 creating+peering, 44 active+undersized, 22 active+undersized+degraded, 97 active+clean; 608 KiB data, 4.8 GiB used, 621 GiB / 626 GiB avail; 78/591 objects degraded (13.198%) 2024-09-10T18:55:42.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:42 smithi018 ceph-mon[29336]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:55:42.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:42 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:55:42.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:42 smithi018 ceph-mon[29336]: osdmap e243: 8 total, 8 up, 7 in 2024-09-10T18:55:42.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:42 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:55:42.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:42 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:55:42.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:42 smithi018 ceph-mon[40196]: pgmap v468: 168 pgs: 5 creating+peering, 44 active+undersized, 22 active+undersized+degraded, 97 active+clean; 608 KiB data, 4.8 GiB used, 621 GiB / 626 GiB avail; 78/591 objects degraded (13.198%) 2024-09-10T18:55:42.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:42 smithi018 ceph-mon[40196]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:55:42.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:42 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:55:42.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:42 smithi018 ceph-mon[40196]: osdmap e243: 8 total, 8 up, 7 in 2024-09-10T18:55:42.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:42 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:55:42.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:42 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:55:42.516 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:42.539 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:42.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:42.540 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:42.540 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:42 smithi103 ceph-mon[33163]: pgmap v468: 168 pgs: 5 creating+peering, 44 active+undersized, 22 active+undersized+degraded, 97 active+clean; 608 KiB data, 4.8 GiB used, 621 GiB / 626 GiB avail; 78/591 objects degraded (13.198%) 2024-09-10T18:55:42.541 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:42 smithi103 ceph-mon[33163]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:55:42.541 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:42 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:55:42.541 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:42 smithi103 ceph-mon[33163]: osdmap e243: 8 total, 8 up, 7 in 2024-09-10T18:55:42.541 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:42 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:55:42.541 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:42 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:55:42.640 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:42.662 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:42.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:42.663 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:42.764 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:42.787 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:42.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:42.788 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:42.889 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:42.911 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:42.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:42.912 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:43.013 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:55:43.036 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:43.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:43.037 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:55:43.080 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T18:55:43.080 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:55:43.138 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:43.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:43.161 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:43.161 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:43.261 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:43.282 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:43.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:43.284 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:43.384 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:43.406 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:43.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:43.407 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:43.507 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:43.533 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:43.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:43.534 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:43.623 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:43 smithi103 ceph-mon[33163]: osdmap e244: 8 total, 8 up, 7 in 2024-09-10T18:55:43.634 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:43.647 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-10T18:55:43.653 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:43 smithi018 ceph-mon[40196]: osdmap e244: 8 total, 8 up, 7 in 2024-09-10T18:55:43.653 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:43 smithi018 ceph-mon[29336]: osdmap e244: 8 total, 8 up, 7 in 2024-09-10T18:55:43.655 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:43.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:43.657 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:43.757 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:43.779 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:43.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:43.780 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:43.881 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:43.898 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:55:43.905 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:43.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:43.906 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:44.007 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:44.030 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:44.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:44.032 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:44.133 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:44.150 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 6, 7, 0] 2024-09-10T18:55:44.150 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T18:55:44.155 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:44.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:44.156 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:44.257 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:44.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:44.281 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:44.281 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:44.383 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:44.404 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:44.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:44.405 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:44.442 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:44 smithi018 ceph-mon[29336]: pgmap v471: 168 pgs: 2 creating+peering, 12 active+undersized, 8 active+undersized+degraded, 146 active+clean; 608 KiB data, 4.8 GiB used, 621 GiB / 626 GiB avail; 767 B/s rd, 0 op/s; 30/591 objects degraded (5.076%); 0 B/s, 7 objects/s recovering 2024-09-10T18:55:44.442 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:44 smithi018 ceph-mon[29336]: Health check update: Degraded data redundancy: 30/591 objects degraded (5.076%), 8 pgs degraded, 19 pgs undersized (PG_DEGRADED) 2024-09-10T18:55:44.442 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:44 smithi018 ceph-mon[40196]: pgmap v471: 168 pgs: 2 creating+peering, 12 active+undersized, 8 active+undersized+degraded, 146 active+clean; 608 KiB data, 4.8 GiB used, 621 GiB / 626 GiB avail; 767 B/s rd, 0 op/s; 30/591 objects degraded (5.076%); 0 B/s, 7 objects/s recovering 2024-09-10T18:55:44.442 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:44 smithi018 ceph-mon[40196]: Health check update: Degraded data redundancy: 30/591 objects degraded (5.076%), 8 pgs degraded, 19 pgs undersized (PG_DEGRADED) 2024-09-10T18:55:44.507 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:44.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:44 smithi103 ceph-mon[33163]: pgmap v471: 168 pgs: 2 creating+peering, 12 active+undersized, 8 active+undersized+degraded, 146 active+clean; 608 KiB data, 4.8 GiB used, 621 GiB / 626 GiB avail; 767 B/s rd, 0 op/s; 30/591 objects degraded (5.076%); 0 B/s, 7 objects/s recovering 2024-09-10T18:55:44.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:44 smithi103 ceph-mon[33163]: Health check update: Degraded data redundancy: 30/591 objects degraded (5.076%), 8 pgs degraded, 19 pgs undersized (PG_DEGRADED) 2024-09-10T18:55:44.522 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-10T18:55:44.529 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:44.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:44.530 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:44.631 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:44.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:44.653 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:44.653 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:44.661 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:55:44.661 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:55:44.661 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:55:44.661 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:55:44.661 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:55:44.661 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:55:44.661 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:55:44.662 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:55:44.662 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:55:44.662 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:55:44.662 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:55:44.662 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:55:44.662 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:55:44.662 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:55:44.662 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:55:44.662 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:55:44.754 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:44.777 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:44.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:44.778 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:44.878 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:44.901 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:44.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:44.903 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:45.004 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:45.026 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:45.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:45.027 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:45.127 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:45.150 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:45.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:45.151 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:45.251 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:45.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:45.274 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:45.274 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:45.375 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:45.398 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:45.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:45.399 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:45.499 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:45.506 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T18:55:45.506 INFO:teuthology.orchestra.run.smithi103.stdout: "ops": [], 2024-09-10T18:55:45.506 INFO:teuthology.orchestra.run.smithi103.stdout: "complaint_time": 30, 2024-09-10T18:55:45.506 INFO:teuthology.orchestra.run.smithi103.stdout: "num_blocked_ops": 0 2024-09-10T18:55:45.506 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T18:55:45.524 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:45.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:45.525 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:45.625 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:45.649 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:45.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:45.650 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:45.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:45 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/493876296' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:55:45.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:45 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/493876296' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:55:45.750 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:45 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/493876296' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:55:45.751 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:45.774 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:45.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:45.775 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:45.855 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:55:45.876 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:45.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:45.901 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:45.901 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:46.002 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:46.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:46.025 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:46.025 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:46.126 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:46.141 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-10T18:55:46.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:46.149 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:46.149 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:46.250 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:46.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:46.274 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:46.274 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:46.375 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:46.398 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:46.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:46.399 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:46.399 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:46 smithi103 ceph-mon[33163]: pgmap v472: 168 pgs: 168 active+clean; 582 KiB data, 4.8 GiB used, 621 GiB / 626 GiB avail; 0 B/s, 8 objects/s recovering 2024-09-10T18:55:46.399 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:46 smithi103 ceph-mon[33163]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 30/591 objects degraded (5.076%), 8 pgs degraded, 19 pgs undersized) 2024-09-10T18:55:46.399 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:46 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2387964808' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:46.406 INFO:teuthology.orchestra.run.smithi018.stderr:enabled application 'rados' on pool 'unique_pool_2' 2024-09-10T18:55:46.407 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:55:46.500 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:46.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:46.522 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:46.523 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:46.623 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:46.644 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:46.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:46.646 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:46.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:46 smithi018 ceph-mon[40196]: pgmap v472: 168 pgs: 168 active+clean; 582 KiB data, 4.8 GiB used, 621 GiB / 626 GiB avail; 0 B/s, 8 objects/s recovering 2024-09-10T18:55:46.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:46 smithi018 ceph-mon[40196]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 30/591 objects degraded (5.076%), 8 pgs degraded, 19 pgs undersized) 2024-09-10T18:55:46.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:46 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2387964808' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:46.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:46 smithi018 ceph-mon[29336]: pgmap v472: 168 pgs: 168 active+clean; 582 KiB data, 4.8 GiB used, 621 GiB / 626 GiB avail; 0 B/s, 8 objects/s recovering 2024-09-10T18:55:46.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:46 smithi018 ceph-mon[29336]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 30/591 objects degraded (5.076%), 8 pgs degraded, 19 pgs undersized) 2024-09-10T18:55:46.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:46 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2387964808' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:46.746 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:46.749 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:55:46 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:55:46.768 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:46.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:46.769 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:46.870 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:46.892 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:46.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:46.893 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:46.994 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:47.020 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:47.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:47.021 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:47.122 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:47.150 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:47.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:47.151 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:47.251 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:47.277 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:47.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:47.278 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:47.379 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:47.403 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:47.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:47.404 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:47.505 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:47.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:47.529 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:47.530 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:47.575 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T18:55:47.576 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:47 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2387964808' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:47.576 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:47 smithi103 ceph-mon[33163]: osdmap e245: 8 total, 8 up, 7 in 2024-09-10T18:55:47.599 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:47 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2387964808' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:47.599 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:47 smithi018 ceph-mon[29336]: osdmap e245: 8 total, 8 up, 7 in 2024-09-10T18:55:47.599 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:47 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2387964808' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:47.599 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:47 smithi018 ceph-mon[40196]: osdmap e245: 8 total, 8 up, 7 in 2024-09-10T18:55:47.631 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:47.654 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:47.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:47.655 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:47.755 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:47.779 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:47.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:47.780 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:47.881 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:47.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:47.905 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:47.906 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:47.968 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:55:48.006 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:48.029 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:48.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:48.030 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:48.113 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T18:55:48.113 INFO:teuthology.orchestra.run.smithi103.stdout: "size": 20, 2024-09-10T18:55:48.113 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 600, 2024-09-10T18:55:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: "ops": [ 2024-09-10T18:55:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:115933 8.9 8:90b98b7c:::benchmark_data_smithi018_106531_object115932:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e183)", 2024-09-10T18:55:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:52:53.593001+0000", 2024-09-10T18:55:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 174.517259486, 2024-09-10T18:55:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.0933817010000002, 2024-09-10T18:55:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:55:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:55:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:55:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:55:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 115933 2024-09-10T18:55:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.593001+0000", 2024-09-10T18:55:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.593001+0000", 2024-09-10T18:55:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.593000+0000", 2024-09-10T18:55:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:55:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.593054+0000", 2024-09-10T18:55:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.4162000000000001e-05 2024-09-10T18:55:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.593066+0000", 2024-09-10T18:55:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.2206e-05 2024-09-10T18:55:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.593072+0000", 2024-09-10T18:55:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.643e-06 2024-09-10T18:55:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.593113+0000", 2024-09-10T18:55:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.0914999999999997e-05 2024-09-10T18:55:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:55:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.593120+0000", 2024-09-10T18:55:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.2620000000000001e-06 2024-09-10T18:55:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:54.576943+0000", 2024-09-10T18:55:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.98382287499999999 2024-09-10T18:55:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:55:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:54.576954+0000", 2024-09-10T18:55:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.1112e-05 2024-09-10T18:55:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.665292+0000", 2024-09-10T18:55:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0883377379999999 2024-09-10T18:55:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:55:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.665473+0000", 2024-09-10T18:55:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000180963 2024-09-10T18:55:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,3,5", 2024-09-10T18:55:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.667230+0000", 2024-09-10T18:55:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.001756608 2024-09-10T18:55:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:55:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.680606+0000", 2024-09-10T18:55:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.013376825 2024-09-10T18:55:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686161+0000", 2024-09-10T18:55:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0055545710000000003 2024-09-10T18:55:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686190+0000", 2024-09-10T18:55:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.8856999999999998e-05 2024-09-10T18:55:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686244+0000", 2024-09-10T18:55:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.4577000000000003e-05 2024-09-10T18:55:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:55:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686289+0000", 2024-09-10T18:55:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.4329999999999997e-05 2024-09-10T18:55:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686383+0000", 2024-09-10T18:55:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.4469999999999995e-05 2024-09-10T18:55:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:115942 8.9 8:966c7925:::benchmark_data_smithi018_106531_object115941:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e183)", 2024-09-10T18:55:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:52:53.598843+0000", 2024-09-10T18:55:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 174.51141808200001, 2024-09-10T18:55:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.0877665900000002, 2024-09-10T18:55:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:55:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:55:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:55:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:55:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 115942 2024-09-10T18:55:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.598843+0000", 2024-09-10T18:55:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.598843+0000", 2024-09-10T18:55:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.598841+0000", 2024-09-10T18:55:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:55:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.598871+0000", 2024-09-10T18:55:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.9961999999999999e-05 2024-09-10T18:55:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.598881+0000", 2024-09-10T18:55:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.3489999999999998e-06 2024-09-10T18:55:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.598885+0000", 2024-09-10T18:55:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.296e-06 2024-09-10T18:55:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.598944+0000", 2024-09-10T18:55:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.9110000000000002e-05 2024-09-10T18:55:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:55:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.598954+0000", 2024-09-10T18:55:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.7200000000000001e-06 2024-09-10T18:55:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:54.576976+0000", 2024-09-10T18:55:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.97802252300000003 2024-09-10T18:55:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:55:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:54.576983+0000", 2024-09-10T18:55:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.1149999999999996e-06 2024-09-10T18:55:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.671002+0000", 2024-09-10T18:55:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0940191109999999 2024-09-10T18:55:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:55:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.671130+0000", 2024-09-10T18:55:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00012751599999999999 2024-09-10T18:55:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,3,5", 2024-09-10T18:55:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.671299+0000", 2024-09-10T18:55:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00016920699999999999 2024-09-10T18:55:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:55:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.680614+0000", 2024-09-10T18:55:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0093144209999999998 2024-09-10T18:55:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686217+0000", 2024-09-10T18:55:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0056036649999999999 2024-09-10T18:55:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686416+0000", 2024-09-10T18:55:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00019841300000000001 2024-09-10T18:55:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686470+0000", 2024-09-10T18:55:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.4048999999999997e-05 2024-09-10T18:55:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:55:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686507+0000", 2024-09-10T18:55:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.6950999999999999e-05 2024-09-10T18:55:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686609+0000", 2024-09-10T18:55:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000102784 2024-09-10T18:55:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:115980 8.9 8:9c48a6ac:::benchmark_data_smithi018_106531_object115979:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e183)", 2024-09-10T18:55:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:52:53.631969+0000", 2024-09-10T18:55:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 174.47829236600001, 2024-09-10T18:55:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.0549517110000002, 2024-09-10T18:55:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:55:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:55:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:55:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:55:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 115980 2024-09-10T18:55:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.631969+0000", 2024-09-10T18:55:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.631969+0000", 2024-09-10T18:55:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.631967+0000", 2024-09-10T18:55:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:55:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.632008+0000", 2024-09-10T18:55:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.1397999999999999e-05 2024-09-10T18:55:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.632019+0000", 2024-09-10T18:55:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0875e-05 2024-09-10T18:55:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.632025+0000", 2024-09-10T18:55:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.8499999999999999e-06 2024-09-10T18:55:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.632091+0000", 2024-09-10T18:55:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.5895999999999994e-05 2024-09-10T18:55:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:55:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.632102+0000", 2024-09-10T18:55:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0992000000000001e-05 2024-09-10T18:55:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:54.577003+0000", 2024-09-10T18:55:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.94490056 2024-09-10T18:55:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:55:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:54.577009+0000", 2024-09-10T18:55:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.1990000000000003e-06 2024-09-10T18:55:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.673498+0000", 2024-09-10T18:55:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.096489402 2024-09-10T18:55:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:55:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.673643+0000", 2024-09-10T18:55:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00014505 2024-09-10T18:55:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,3,5", 2024-09-10T18:55:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.673801+0000", 2024-09-10T18:55:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000157608 2024-09-10T18:55:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:55:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.680622+0000", 2024-09-10T18:55:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0068210340000000001 2024-09-10T18:55:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686443+0000", 2024-09-10T18:55:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0058211000000000001 2024-09-10T18:55:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686659+0000", 2024-09-10T18:55:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000216276 2024-09-10T18:55:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686773+0000", 2024-09-10T18:55:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000113706 2024-09-10T18:55:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:55:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686812+0000", 2024-09-10T18:55:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.8617e-05 2024-09-10T18:55:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686920+0000", 2024-09-10T18:55:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000108651 2024-09-10T18:55:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:115987 8.9 8:94f7fdf8:::benchmark_data_smithi018_106531_object115986:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e183)", 2024-09-10T18:55:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:52:53.636486+0000", 2024-09-10T18:55:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 174.47377511900001, 2024-09-10T18:55:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.0506323740000001, 2024-09-10T18:55:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:55:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:55:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:55:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:55:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 115987 2024-09-10T18:55:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.636486+0000", 2024-09-10T18:55:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.141 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:48.141 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.636486+0000", 2024-09-10T18:55:48.144 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.144 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.144 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.144 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.636484+0000", 2024-09-10T18:55:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:55:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.636528+0000", 2024-09-10T18:55:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.4143000000000001e-05 2024-09-10T18:55:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.636542+0000", 2024-09-10T18:55:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.3889e-05 2024-09-10T18:55:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.146 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.146 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:48.148 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:48.148 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.636548+0000", 2024-09-10T18:55:48.148 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.0129999999999999e-06 2024-09-10T18:55:48.148 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.148 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.148 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.148 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.636644+0000", 2024-09-10T18:55:48.149 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.5265000000000004e-05 2024-09-10T18:55:48.149 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.149 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.149 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:55:48.149 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.636654+0000", 2024-09-10T18:55:48.149 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0414999999999999e-05 2024-09-10T18:55:48.149 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.149 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.149 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.149 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:54.577026+0000", 2024-09-10T18:55:48.149 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.94037243400000003 2024-09-10T18:55:48.149 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:55:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:54.577033+0000", 2024-09-10T18:55:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.4489999999999996e-06 2024-09-10T18:55:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.676009+0000", 2024-09-10T18:55:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.098976314 2024-09-10T18:55:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:55:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.676099+0000", 2024-09-10T18:55:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.9402999999999998e-05 2024-09-10T18:55:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,3,5", 2024-09-10T18:55:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.676215+0000", 2024-09-10T18:55:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000116106 2024-09-10T18:55:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:55:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.685890+0000", 2024-09-10T18:55:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0096749999999999996 2024-09-10T18:55:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686696+0000", 2024-09-10T18:55:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00080583300000000005 2024-09-10T18:55:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686956+0000", 2024-09-10T18:55:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00026063300000000002 2024-09-10T18:55:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686983+0000", 2024-09-10T18:55:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.6897999999999999e-05 2024-09-10T18:55:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:55:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.687024+0000", 2024-09-10T18:55:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.1143000000000002e-05 2024-09-10T18:55:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.687118+0000", 2024-09-10T18:55:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.3961000000000004e-05 2024-09-10T18:55:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:116013 8.9 8:923f941a:::benchmark_data_smithi018_106531_object116012:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e183)", 2024-09-10T18:55:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:52:53.661699+0000", 2024-09-10T18:55:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 174.44856188700001, 2024-09-10T18:55:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.0258877599999998, 2024-09-10T18:55:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:55:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:55:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:55:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:55:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 116013 2024-09-10T18:55:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.661699+0000", 2024-09-10T18:55:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.661699+0000", 2024-09-10T18:55:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.661698+0000", 2024-09-10T18:55:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:55:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.661753+0000", 2024-09-10T18:55:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.5143999999999999e-05 2024-09-10T18:55:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.661764+0000", 2024-09-10T18:55:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0651e-05 2024-09-10T18:55:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.661768+0000", 2024-09-10T18:55:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.3189999999999996e-06 2024-09-10T18:55:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.661810+0000", 2024-09-10T18:55:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.2271999999999998e-05 2024-09-10T18:55:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:55:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.661819+0000", 2024-09-10T18:55:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.6589999999999996e-06 2024-09-10T18:55:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:54.577071+0000", 2024-09-10T18:55:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.91525191500000003 2024-09-10T18:55:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:55:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:54.577077+0000", 2024-09-10T18:55:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.4160000000000001e-06 2024-09-10T18:55:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.678501+0000", 2024-09-10T18:55:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.101423915 2024-09-10T18:55:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:55:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.678633+0000", 2024-09-10T18:55:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00013232200000000001 2024-09-10T18:55:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,3,5", 2024-09-10T18:55:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.678805+0000", 2024-09-10T18:55:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000171103 2024-09-10T18:55:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:55:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.685940+0000", 2024-09-10T18:55:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0071355940000000003 2024-09-10T18:55:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686734+0000", 2024-09-10T18:55:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00079338399999999995 2024-09-10T18:55:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.687202+0000", 2024-09-10T18:55:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00046798199999999998 2024-09-10T18:55:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.687314+0000", 2024-09-10T18:55:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00011257200000000001 2024-09-10T18:55:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:55:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.687361+0000", 2024-09-10T18:55:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.7308000000000002e-05 2024-09-10T18:55:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.162 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:48.162 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.687587+0000", 2024-09-10T18:55:48.162 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000225443 2024-09-10T18:55:48.162 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.162 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:48.162 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.162 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.162 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.162 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:116021 8.9 8:985865c8:::benchmark_data_smithi018_106531_object116020:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e183)", 2024-09-10T18:55:48.162 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:52:53.671662+0000", 2024-09-10T18:55:48.162 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 174.43859872199999, 2024-09-10T18:55:48.162 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.0160958120000001, 2024-09-10T18:55:48.162 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:55:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:55:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:55:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:55:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 116021 2024-09-10T18:55:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.671662+0000", 2024-09-10T18:55:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.164 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:48.164 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.671662+0000", 2024-09-10T18:55:48.164 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.164 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.164 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.164 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:48.164 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.671661+0000", 2024-09-10T18:55:48.164 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:55:48.164 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.164 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.164 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:48.164 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.671709+0000", 2024-09-10T18:55:48.164 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.7559000000000003e-05 2024-09-10T18:55:48.164 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.165 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.165 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:48.165 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.671720+0000", 2024-09-10T18:55:48.165 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.1324e-05 2024-09-10T18:55:48.165 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.165 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.165 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:48.165 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.671724+0000", 2024-09-10T18:55:48.165 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.0980000000000004e-06 2024-09-10T18:55:48.165 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.165 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.165 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.165 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.671774+0000", 2024-09-10T18:55:48.166 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.0096000000000003e-05 2024-09-10T18:55:48.166 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.166 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.166 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:55:48.166 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.671781+0000", 2024-09-10T18:55:48.166 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.255e-06 2024-09-10T18:55:48.166 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.166 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.166 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.166 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:54.577099+0000", 2024-09-10T18:55:48.166 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.905317661 2024-09-10T18:55:48.166 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.166 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.167 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:55:48.167 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:54.577105+0000", 2024-09-10T18:55:48.167 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.3770000000000001e-06 2024-09-10T18:55:48.167 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.167 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.167 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.167 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.680666+0000", 2024-09-10T18:55:48.167 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.1035605239999999 2024-09-10T18:55:48.167 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.167 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.167 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:55:48.167 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.680802+0000", 2024-09-10T18:55:48.167 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000136503 2024-09-10T18:55:48.168 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.168 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.168 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,3,5", 2024-09-10T18:55:48.168 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.680991+0000", 2024-09-10T18:55:48.168 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000188786 2024-09-10T18:55:48.168 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.168 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.168 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:55:48.168 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.685946+0000", 2024-09-10T18:55:48.168 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.004954366 2024-09-10T18:55:48.168 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.168 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.687233+0000", 2024-09-10T18:55:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0012871029999999999 2024-09-10T18:55:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.687258+0000", 2024-09-10T18:55:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.5483e-05 2024-09-10T18:55:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.687626+0000", 2024-09-10T18:55:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00036776300000000001 2024-09-10T18:55:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:55:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.687669+0000", 2024-09-10T18:55:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.3053999999999998e-05 2024-09-10T18:55:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.687758+0000", 2024-09-10T18:55:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.9036999999999995e-05 2024-09-10T18:55:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:116041 8.9 8:9e49af6d:::benchmark_data_smithi018_106531_object116040:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e183)", 2024-09-10T18:55:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:52:53.703046+0000", 2024-09-10T18:55:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 174.40721544900001, 2024-09-10T18:55:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.993576947, 2024-09-10T18:55:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:55:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:55:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:55:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:55:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 116041 2024-09-10T18:55:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.703046+0000", 2024-09-10T18:55:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.703046+0000", 2024-09-10T18:55:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.703044+0000", 2024-09-10T18:55:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:55:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.703109+0000", 2024-09-10T18:55:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.4926999999999997e-05 2024-09-10T18:55:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.703121+0000", 2024-09-10T18:55:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.1892999999999999e-05 2024-09-10T18:55:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.703125+0000", 2024-09-10T18:55:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.3189999999999996e-06 2024-09-10T18:55:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.703177+0000", 2024-09-10T18:55:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.1378000000000002e-05 2024-09-10T18:55:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:55:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.703184+0000", 2024-09-10T18:55:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.7230000000000004e-06 2024-09-10T18:55:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:54.577248+0000", 2024-09-10T18:55:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.87406382299999996 2024-09-10T18:55:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:55:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:54.577257+0000", 2024-09-10T18:55:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.6649999999999992e-06 2024-09-10T18:55:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.682900+0000", 2024-09-10T18:55:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.105643361 2024-09-10T18:55:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:55:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.683022+0000", 2024-09-10T18:55:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00012169 2024-09-10T18:55:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,3,5", 2024-09-10T18:55:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.683166+0000", 2024-09-10T18:55:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000144189 2024-09-10T18:55:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.687288+0000", 2024-09-10T18:55:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0041215690000000003 2024-09-10T18:55:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:55:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.695411+0000", 2024-09-10T18:55:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0081232549999999994 2024-09-10T18:55:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.695694+0000", 2024-09-10T18:55:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000282959 2024-09-10T18:55:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.695730+0000", 2024-09-10T18:55:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.6159999999999999e-05 2024-09-10T18:55:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:55:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.696503+0000", 2024-09-10T18:55:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00077256700000000005 2024-09-10T18:55:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.696622+0000", 2024-09-10T18:55:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000119892 2024-09-10T18:55:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:194910 8.6 8:673b0092:::benchmark_data_smithi018_106531_object48280:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e206)", 2024-09-10T18:55:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:54:00.294785+0000", 2024-09-10T18:55:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 107.815475619, 2024-09-10T18:55:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.038020379, 2024-09-10T18:55:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "reached pg", 2024-09-10T18:55:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:55:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:55:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:55:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 194910 2024-09-10T18:55:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294785+0000", 2024-09-10T18:55:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294785+0000", 2024-09-10T18:55:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294784+0000", 2024-09-10T18:55:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:55:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294791+0000", 2024-09-10T18:55:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.37e-06 2024-09-10T18:55:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294800+0000", 2024-09-10T18:55:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.4690000000000003e-06 2024-09-10T18:55:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294806+0000", 2024-09-10T18:55:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.0430000000000004e-06 2024-09-10T18:55:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294976+0000", 2024-09-10T18:55:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00016977499999999999 2024-09-10T18:55:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:55:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294985+0000", 2024-09-10T18:55:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.8899999999999996e-06 2024-09-10T18:55:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.332737+0000", 2024-09-10T18:55:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0377524929999999 2024-09-10T18:55:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.332806+0000", 2024-09-10T18:55:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.8517999999999997e-05 2024-09-10T18:55:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:194916 8.6 8:6649d7fa:::benchmark_data_smithi018_106531_object48286:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e206)", 2024-09-10T18:55:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:54:00.294883+0000", 2024-09-10T18:55:48.185 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 107.81537760000001, 2024-09-10T18:55:48.185 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0521695499999999, 2024-09-10T18:55:48.185 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:48.185 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "reached pg", 2024-09-10T18:55:48.185 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:55:48.185 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:55:48.185 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:55:48.185 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 194916 2024-09-10T18:55:48.185 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.185 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:48.185 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294883+0000", 2024-09-10T18:55:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294883+0000", 2024-09-10T18:55:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294883+0000", 2024-09-10T18:55:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999995 2024-09-10T18:55:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294887+0000", 2024-09-10T18:55:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.143e-06 2024-09-10T18:55:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294896+0000", 2024-09-10T18:55:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.2799999999999992e-06 2024-09-10T18:55:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294902+0000", 2024-09-10T18:55:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.429e-06 2024-09-10T18:55:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.295026+0000", 2024-09-10T18:55:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00012407799999999999 2024-09-10T18:55:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:55:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.295031+0000", 2024-09-10T18:55:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.8679999999999998e-06 2024-09-10T18:55:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.332977+0000", 2024-09-10T18:55:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.037946563 2024-09-10T18:55:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for missing object", 2024-09-10T18:55:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.333062+0000", 2024-09-10T18:55:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.5266999999999995e-05 2024-09-10T18:55:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.346974+0000", 2024-09-10T18:55:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.013911468 2024-09-10T18:55:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.347053+0000", 2024-09-10T18:55:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.9085000000000006e-05 2024-09-10T18:55:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:194933 8.6 8:6cc0138d:::benchmark_data_smithi018_106531_object48303:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e206)", 2024-09-10T18:55:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:54:00.309381+0000", 2024-09-10T18:55:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 107.80087985900001, 2024-09-10T18:55:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0420561989999999, 2024-09-10T18:55:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:55:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:55:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:55:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:55:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 194933 2024-09-10T18:55:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.309381+0000", 2024-09-10T18:55:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.309381+0000", 2024-09-10T18:55:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.309380+0000", 2024-09-10T18:55:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:55:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.309386+0000", 2024-09-10T18:55:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.0750000000000003e-06 2024-09-10T18:55:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.309398+0000", 2024-09-10T18:55:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.2291e-05 2024-09-10T18:55:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.309403+0000", 2024-09-10T18:55:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.198e-06 2024-09-10T18:55:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.313851+0000", 2024-09-10T18:55:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0044473760000000003 2024-09-10T18:55:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:55:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.313858+0000", 2024-09-10T18:55:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.525e-06 2024-09-10T18:55:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.333087+0000", 2024-09-10T18:55:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0192285029999999 2024-09-10T18:55:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:55:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.333185+0000", 2024-09-10T18:55:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.8448000000000003e-05 2024-09-10T18:55:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 7", 2024-09-10T18:55:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.333548+0000", 2024-09-10T18:55:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00036276100000000002 2024-09-10T18:55:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:55:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.344236+0000", 2024-09-10T18:55:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.010687497000000001 2024-09-10T18:55:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.351170+0000", 2024-09-10T18:55:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0069349119999999997 2024-09-10T18:55:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:55:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.351337+0000", 2024-09-10T18:55:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00016637600000000001 2024-09-10T18:55:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.351437+0000", 2024-09-10T18:55:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00010045200000000001 2024-09-10T18:55:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:194942 8.6 8:6dedfc51:::benchmark_data_smithi018_106531_object48312:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e206)", 2024-09-10T18:55:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:54:00.317038+0000", 2024-09-10T18:55:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 107.793223153, 2024-09-10T18:55:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0348635230000001, 2024-09-10T18:55:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:55:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:55:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:55:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:55:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 194942 2024-09-10T18:55:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.317038+0000", 2024-09-10T18:55:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.317038+0000", 2024-09-10T18:55:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.317036+0000", 2024-09-10T18:55:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:55:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.317042+0000", 2024-09-10T18:55:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.682e-06 2024-09-10T18:55:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.317055+0000", 2024-09-10T18:55:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.2948e-05 2024-09-10T18:55:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.317062+0000", 2024-09-10T18:55:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.9299999999999997e-06 2024-09-10T18:55:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.317132+0000", 2024-09-10T18:55:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.0005000000000005e-05 2024-09-10T18:55:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:55:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.317140+0000", 2024-09-10T18:55:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.1850000000000004e-06 2024-09-10T18:55:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.339169+0000", 2024-09-10T18:55:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0220292820000001 2024-09-10T18:55:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:55:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.339295+0000", 2024-09-10T18:55:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000125339 2024-09-10T18:55:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 7", 2024-09-10T18:55:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.339424+0000", 2024-09-10T18:55:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000128775 2024-09-10T18:55:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:55:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.346717+0000", 2024-09-10T18:55:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0072938730000000002 2024-09-10T18:55:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.351712+0000", 2024-09-10T18:55:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0049943510000000002 2024-09-10T18:55:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:55:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.351824+0000", 2024-09-10T18:55:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000112222 2024-09-10T18:55:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.351901+0000", 2024-09-10T18:55:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.7371000000000002e-05 2024-09-10T18:55:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:194943 8.9 8:982e3ef1:::benchmark_data_smithi018_106531_object48313:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e206)", 2024-09-10T18:55:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:54:00.317188+0000", 2024-09-10T18:55:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 107.793073248, 2024-09-10T18:55:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.017647009, 2024-09-10T18:55:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:55:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:55:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:55:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:55:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 194943 2024-09-10T18:55:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.317188+0000", 2024-09-10T18:55:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.317188+0000", 2024-09-10T18:55:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.317187+0000", 2024-09-10T18:55:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:55:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.317192+0000", 2024-09-10T18:55:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.7600000000000002e-06 2024-09-10T18:55:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.317202+0000", 2024-09-10T18:55:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0264e-05 2024-09-10T18:55:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.317207+0000", 2024-09-10T18:55:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.3299999999999998e-06 2024-09-10T18:55:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.317262+0000", 2024-09-10T18:55:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.4280000000000002e-05 2024-09-10T18:55:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:55:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.317269+0000", 2024-09-10T18:55:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.4819999999999997e-06 2024-09-10T18:55:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.326257+0000", 2024-09-10T18:55:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0089881220000001 2024-09-10T18:55:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:55:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.326386+0000", 2024-09-10T18:55:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00012864800000000001 2024-09-10T18:55:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,5", 2024-09-10T18:55:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.326525+0000", 2024-09-10T18:55:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00013949500000000001 2024-09-10T18:55:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.331405+0000", 2024-09-10T18:55:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.004879492 2024-09-10T18:55:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.331554+0000", 2024-09-10T18:55:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00014887599999999999 2024-09-10T18:55:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:55:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.334694+0000", 2024-09-10T18:55:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0031405169999999998 2024-09-10T18:55:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:55:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.334736+0000", 2024-09-10T18:55:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.1347000000000002e-05 2024-09-10T18:55:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.334835+0000", 2024-09-10T18:55:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.9216999999999996e-05 2024-09-10T18:55:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:194948 8.6 8:60730bfc:::benchmark_data_smithi018_106531_object48318:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e206)", 2024-09-10T18:55:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:54:00.320523+0000", 2024-09-10T18:55:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 107.789737688, 2024-09-10T18:55:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0338989240000001, 2024-09-10T18:55:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:55:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:55:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:55:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:55:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 194948 2024-09-10T18:55:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320523+0000", 2024-09-10T18:55:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320523+0000", 2024-09-10T18:55:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320521+0000", 2024-09-10T18:55:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999981 2024-09-10T18:55:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320528+0000", 2024-09-10T18:55:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.4790000000000001e-06 2024-09-10T18:55:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320541+0000", 2024-09-10T18:55:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.2706e-05 2024-09-10T18:55:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320547+0000", 2024-09-10T18:55:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.9900000000000002e-06 2024-09-10T18:55:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320582+0000", 2024-09-10T18:55:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.4981999999999998e-05 2024-09-10T18:55:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:55:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320588+0000", 2024-09-10T18:55:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.1569999999999997e-06 2024-09-10T18:55:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.344263+0000", 2024-09-10T18:55:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0236751559999999 2024-09-10T18:55:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:55:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.344357+0000", 2024-09-10T18:55:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.3874000000000003e-05 2024-09-10T18:55:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 7", 2024-09-10T18:55:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.344462+0000", 2024-09-10T18:55:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000105306 2024-09-10T18:55:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:55:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.347052+0000", 2024-09-10T18:55:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0025898800000000001 2024-09-10T18:55:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.354312+0000", 2024-09-10T18:55:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.007260468 2024-09-10T18:55:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:55:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.354351+0000", 2024-09-10T18:55:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.8572999999999998e-05 2024-09-10T18:55:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.354422+0000", 2024-09-10T18:55:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.1250999999999999e-05 2024-09-10T18:55:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:194950 8.9 8:905debb1:::benchmark_data_smithi018_106531_object48320:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e206)", 2024-09-10T18:55:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:54:00.321242+0000", 2024-09-10T18:55:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 107.789019376, 2024-09-10T18:55:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0142816670000001, 2024-09-10T18:55:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:55:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:55:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:55:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:55:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 194950 2024-09-10T18:55:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.321242+0000", 2024-09-10T18:55:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.321242+0000", 2024-09-10T18:55:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.321241+0000", 2024-09-10T18:55:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999995 2024-09-10T18:55:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.321245+0000", 2024-09-10T18:55:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.1930000000000003e-06 2024-09-10T18:55:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.321253+0000", 2024-09-10T18:55:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.1680000000000004e-06 2024-09-10T18:55:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.321257+0000", 2024-09-10T18:55:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.4860000000000002e-06 2024-09-10T18:55:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.321303+0000", 2024-09-10T18:55:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.6282999999999997e-05 2024-09-10T18:55:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:55:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.321311+0000", 2024-09-10T18:55:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.2709999999999992e-06 2024-09-10T18:55:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.328706+0000", 2024-09-10T18:55:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.007394463 2024-09-10T18:55:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:55:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.328835+0000", 2024-09-10T18:55:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000129636 2024-09-10T18:55:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,5", 2024-09-10T18:55:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.328971+0000", 2024-09-10T18:55:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000135647 2024-09-10T18:55:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.334679+0000", 2024-09-10T18:55:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0057075310000000001 2024-09-10T18:55:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:55:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.334843+0000", 2024-09-10T18:55:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00016409700000000001 2024-09-10T18:55:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.335374+0000", 2024-09-10T18:55:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00053144499999999996 2024-09-10T18:55:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:55:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.335423+0000", 2024-09-10T18:55:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.8809e-05 2024-09-10T18:55:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.335523+0000", 2024-09-10T18:55:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00010024 2024-09-10T18:55:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:194951 8.6 8:6b51c5ec:::benchmark_data_smithi018_106531_object48321:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e206)", 2024-09-10T18:55:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:54:00.321286+0000", 2024-09-10T18:55:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 107.78897542599999, 2024-09-10T18:55:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.038199619, 2024-09-10T18:55:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:55:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:55:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:55:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:55:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 194951 2024-09-10T18:55:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.321286+0000", 2024-09-10T18:55:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.321286+0000", 2024-09-10T18:55:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.321285+0000", 2024-09-10T18:55:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999995 2024-09-10T18:55:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.321289+0000", 2024-09-10T18:55:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.4349999999999999e-06 2024-09-10T18:55:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.321297+0000", 2024-09-10T18:55:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.6939999999999995e-06 2024-09-10T18:55:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.321302+0000", 2024-09-10T18:55:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.6809999999999997e-06 2024-09-10T18:55:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.321345+0000", 2024-09-10T18:55:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.2984e-05 2024-09-10T18:55:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:55:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.321351+0000", 2024-09-10T18:55:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.3690000000000003e-06 2024-09-10T18:55:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.347098+0000", 2024-09-10T18:55:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.025746928 2024-09-10T18:55:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:55:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.347212+0000", 2024-09-10T18:55:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000114428 2024-09-10T18:55:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 7", 2024-09-10T18:55:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.347295+0000", 2024-09-10T18:55:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.2806999999999995e-05 2024-09-10T18:55:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.354456+0000", 2024-09-10T18:55:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0071609059999999999 2024-09-10T18:55:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:55:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.359376+0000", 2024-09-10T18:55:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0049194349999999998 2024-09-10T18:55:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:55:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.359415+0000", 2024-09-10T18:55:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.9453000000000003e-05 2024-09-10T18:55:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.359485+0000", 2024-09-10T18:55:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.0178000000000005e-05 2024-09-10T18:55:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:248413 8.9 8:9f1dcbb5:::benchmark_data_smithi018_106531_object101783:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e218)", 2024-09-10T18:55:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:54:39.652660+0000", 2024-09-10T18:55:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 68.457601179999997, 2024-09-10T18:55:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.002997237, 2024-09-10T18:55:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:55:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:55:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:55:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:55:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 248413 2024-09-10T18:55:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.652660+0000", 2024-09-10T18:55:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.652660+0000", 2024-09-10T18:55:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.652659+0000", 2024-09-10T18:55:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:55:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.652663+0000", 2024-09-10T18:55:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.9870000000000001e-06 2024-09-10T18:55:48.236 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.236 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.236 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:48.236 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.652670+0000", 2024-09-10T18:55:48.236 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.4440000000000002e-06 2024-09-10T18:55:48.236 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.236 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.236 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:48.236 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.652675+0000", 2024-09-10T18:55:48.236 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.2229999999999998e-06 2024-09-10T18:55:48.236 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.654639+0000", 2024-09-10T18:55:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00196356 2024-09-10T18:55:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:55:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.654647+0000", 2024-09-10T18:55:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.0150000000000003e-06 2024-09-10T18:55:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.633532+0000", 2024-09-10T18:55:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.97888465999999996 2024-09-10T18:55:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:55:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.633626+0000", 2024-09-10T18:55:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.4530999999999993e-05 2024-09-10T18:55:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,5", 2024-09-10T18:55:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.633707+0000", 2024-09-10T18:55:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.1034999999999999e-05 2024-09-10T18:55:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:55:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.639898+0000", 2024-09-10T18:55:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0061904899999999999 2024-09-10T18:55:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.655454+0000", 2024-09-10T18:55:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.015556367 2024-09-10T18:55:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.655517+0000", 2024-09-10T18:55:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.2558000000000007e-05 2024-09-10T18:55:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:55:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.655569+0000", 2024-09-10T18:55:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.2457999999999998e-05 2024-09-10T18:55:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.655657+0000", 2024-09-10T18:55:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.8018999999999999e-05 2024-09-10T18:55:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:248422 8.9 8:90ba02b9:::benchmark_data_smithi018_106531_object101792:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e218)", 2024-09-10T18:55:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:54:39.658256+0000", 2024-09-10T18:55:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 68.452005280999998, 2024-09-10T18:55:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.99755751800000003, 2024-09-10T18:55:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:55:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:55:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:55:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:55:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 248422 2024-09-10T18:55:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.658256+0000", 2024-09-10T18:55:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.658256+0000", 2024-09-10T18:55:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.658255+0000", 2024-09-10T18:55:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999995 2024-09-10T18:55:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.658259+0000", 2024-09-10T18:55:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.143e-06 2024-09-10T18:55:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.658267+0000", 2024-09-10T18:55:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.3480000000000004e-06 2024-09-10T18:55:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.658273+0000", 2024-09-10T18:55:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.1769999999999997e-06 2024-09-10T18:55:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.658316+0000", 2024-09-10T18:55:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.337e-05 2024-09-10T18:55:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:55:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.658324+0000", 2024-09-10T18:55:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.6040000000000004e-06 2024-09-10T18:55:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.635467+0000", 2024-09-10T18:55:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.97714298499999996 2024-09-10T18:55:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:55:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.635572+0000", 2024-09-10T18:55:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00010490899999999999 2024-09-10T18:55:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,5", 2024-09-10T18:55:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.635645+0000", 2024-09-10T18:55:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.3101000000000004e-05 2024-09-10T18:55:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:55:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.642303+0000", 2024-09-10T18:55:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0066582480000000003 2024-09-10T18:55:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.655702+0000", 2024-09-10T18:55:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.013399345999999999 2024-09-10T18:55:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.655738+0000", 2024-09-10T18:55:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.6174999999999998e-05 2024-09-10T18:55:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:55:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.655765+0000", 2024-09-10T18:55:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.6763000000000001e-05 2024-09-10T18:55:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.655813+0000", 2024-09-10T18:55:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.8047999999999998e-05 2024-09-10T18:55:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:248431 8.9 8:9ec8126b:::benchmark_data_smithi018_106531_object101801:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e218)", 2024-09-10T18:55:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:54:39.659488+0000", 2024-09-10T18:55:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 68.450773151000007, 2024-09-10T18:55:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.99650889399999998, 2024-09-10T18:55:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:55:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:55:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:55:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:55:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 248431 2024-09-10T18:55:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.659488+0000", 2024-09-10T18:55:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.659488+0000", 2024-09-10T18:55:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.659487+0000", 2024-09-10T18:55:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:55:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.250 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.250 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:48.250 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.659492+0000", 2024-09-10T18:55:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.8879999999999996e-06 2024-09-10T18:55:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.659504+0000", 2024-09-10T18:55:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.1799e-05 2024-09-10T18:55:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.659509+0000", 2024-09-10T18:55:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.3510000000000001e-06 2024-09-10T18:55:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.659552+0000", 2024-09-10T18:55:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.3344999999999999e-05 2024-09-10T18:55:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:55:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.659560+0000", 2024-09-10T18:55:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.0739999999999993e-06 2024-09-10T18:55:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.636881+0000", 2024-09-10T18:55:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.97732015900000002 2024-09-10T18:55:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:55:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.636955+0000", 2024-09-10T18:55:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.4685999999999996e-05 2024-09-10T18:55:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,5", 2024-09-10T18:55:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.637062+0000", 2024-09-10T18:55:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00010655800000000001 2024-09-10T18:55:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:55:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.642308+0000", 2024-09-10T18:55:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0052465350000000001 2024-09-10T18:55:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.655853+0000", 2024-09-10T18:55:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.013544700999999999 2024-09-10T18:55:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.655929+0000", 2024-09-10T18:55:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.5631999999999997e-05 2024-09-10T18:55:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:55:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.655951+0000", 2024-09-10T18:55:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.2388e-05 2024-09-10T18:55:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.655997+0000", 2024-09-10T18:55:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.5627999999999997e-05 2024-09-10T18:55:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:248432 8.9 8:9e11dac5:::benchmark_data_smithi018_106531_object101802:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e218)", 2024-09-10T18:55:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:54:39.661415+0000", 2024-09-10T18:55:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 68.448845933000001, 2024-09-10T18:55:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.99472695200000005, 2024-09-10T18:55:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:55:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:55:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:55:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:55:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 248432 2024-09-10T18:55:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.661415+0000", 2024-09-10T18:55:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.661415+0000", 2024-09-10T18:55:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.661414+0000", 2024-09-10T18:55:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:55:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.661420+0000", 2024-09-10T18:55:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.9540000000000001e-06 2024-09-10T18:55:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.661431+0000", 2024-09-10T18:55:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0813e-05 2024-09-10T18:55:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.661436+0000", 2024-09-10T18:55:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.7300000000000002e-06 2024-09-10T18:55:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.661592+0000", 2024-09-10T18:55:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00015532499999999999 2024-09-10T18:55:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:55:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.661599+0000", 2024-09-10T18:55:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.5530000000000004e-06 2024-09-10T18:55:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.638526+0000", 2024-09-10T18:55:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.97692656499999997 2024-09-10T18:55:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:55:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.638636+0000", 2024-09-10T18:55:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000109801 2024-09-10T18:55:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,5", 2024-09-10T18:55:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.638765+0000", 2024-09-10T18:55:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000129049 2024-09-10T18:55:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:55:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.644591+0000", 2024-09-10T18:55:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0058265979999999997 2024-09-10T18:55:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.656041+0000", 2024-09-10T18:55:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.011449456 2024-09-10T18:55:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.656074+0000", 2024-09-10T18:55:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.2951999999999999e-05 2024-09-10T18:55:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:55:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.656097+0000", 2024-09-10T18:55:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.3665000000000001e-05 2024-09-10T18:55:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.656142+0000", 2024-09-10T18:55:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.4719000000000003e-05 2024-09-10T18:55:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:248455 8.9 8:94535dab:::benchmark_data_smithi018_106531_object101825:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e218)", 2024-09-10T18:55:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:54:39.675441+0000", 2024-09-10T18:55:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 68.434820106999993, 2024-09-10T18:55:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.98094657600000001, 2024-09-10T18:55:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:55:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:55:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:55:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:55:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:55:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 248455 2024-09-10T18:55:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:55:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:55:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.675441+0000", 2024-09-10T18:55:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:55:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.675441+0000", 2024-09-10T18:55:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:55:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:55:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.675440+0000", 2024-09-10T18:55:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:55:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:55:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.675445+0000", 2024-09-10T18:55:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.3279999999999996e-06 2024-09-10T18:55:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:55:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.675454+0000", 2024-09-10T18:55:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.5879999999999998e-06 2024-09-10T18:55:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:55:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.675459+0000", 2024-09-10T18:55:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.13e-06 2024-09-10T18:55:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.675503+0000", 2024-09-10T18:55:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.4489e-05 2024-09-10T18:55:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:55:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.675510+0000", 2024-09-10T18:55:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.0169999999999999e-06 2024-09-10T18:55:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:55:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.639936+0000", 2024-09-10T18:55:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.96442608900000004 2024-09-10T18:55:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:55:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.640018+0000", 2024-09-10T18:55:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.1944e-05 2024-09-10T18:55:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,5", 2024-09-10T18:55:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.640089+0000", 2024-09-10T18:55:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.038e-05 2024-09-10T18:55:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:55:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.644594+0000", 2024-09-10T18:55:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0045056769999999996 2024-09-10T18:55:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.656185+0000", 2024-09-10T18:55:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.011590237 2024-09-10T18:55:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:55:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.656238+0000", 2024-09-10T18:55:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.3189999999999997e-05 2024-09-10T18:55:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:55:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.656286+0000", 2024-09-10T18:55:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.8539999999999999e-05 2024-09-10T18:55:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:55:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:55:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:55:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.656387+0000", 2024-09-10T18:55:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00010098 2024-09-10T18:55:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:55:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:55:48.271 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T18:55:48.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:48.271 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:48.272 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:48.328 DEBUG:teuthology.orchestra.run.smithi018:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_2 bench 90 write' 2024-09-10T18:55:48.329 INFO:tasks.thrashosds.thrasher:Shrinking pool unique_pool_2 2024-09-10T18:55:48.329 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool unique_pool_2 by 10 min 8 2024-09-10T18:55:48.329 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T18:55:48.332 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-10T18:55:48.332 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-10T18:55:48.372 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:48.381 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:hints = 1 2024-09-10T18:55:48.381 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-10T18:55:48.381 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Object prefix: benchmark_data_smithi018_133594 2024-09-10T18:55:48.382 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T18:55:48.382 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 0 0 0 0 0 0 - 0 2024-09-10T18:55:48.400 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:48.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:48.401 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:48.502 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:48.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:48 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:48.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:48 smithi018 ceph-mon[29336]: pgmap v474: 168 pgs: 168 active+clean; 582 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 0 B/s rd, 0 op/s; 0 B/s, 8 objects/s recovering 2024-09-10T18:55:48.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:48 smithi018 ceph-mon[29336]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T18:55:48.525 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:48 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:48.525 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:48 smithi018 ceph-mon[40196]: pgmap v474: 168 pgs: 168 active+clean; 582 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 0 B/s rd, 0 op/s; 0 B/s, 8 objects/s recovering 2024-09-10T18:55:48.526 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:48 smithi018 ceph-mon[40196]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T18:55:48.552 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:48.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:48.553 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:48.654 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:48.682 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:48.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:48.683 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:48.698 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:48 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:48.699 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:48 smithi103 ceph-mon[33163]: pgmap v474: 168 pgs: 168 active+clean; 582 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 0 B/s rd, 0 op/s; 0 B/s, 8 objects/s recovering 2024-09-10T18:55:48.699 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:48 smithi103 ceph-mon[33163]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T18:55:48.785 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:48.816 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:48.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:48.817 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:48.917 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:48.950 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:48.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:48.951 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:49.052 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:49.093 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:49.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:49.095 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:49.196 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:49.202 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-10T18:55:49.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:49.230 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:49.230 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:49.331 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:49.369 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:49.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:49.370 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:49.382 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 1 16 3462 3446 215.362 215.375 0.00419684 0.00462578 2024-09-10T18:55:49.471 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:49.508 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:49.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:49.509 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:49.610 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:49.642 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is set 2024-09-10T18:55:49.646 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:49.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:49.647 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:49.652 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:55:49.748 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:49.793 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:49.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:49.795 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:49.895 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:49.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:49 smithi103 ceph-mon[33163]: osdmap e246: 8 total, 8 up, 7 in 2024-09-10T18:55:49.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:49 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/19025122' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:55:49.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:49 smithi103 ceph-mon[33163]: pgmap v476: 168 pgs: 168 active+clean; 582 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 0 B/s, 3 objects/s recovering 2024-09-10T18:55:49.933 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:49.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:49.934 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:49.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:49 smithi018 ceph-mon[40196]: osdmap e246: 8 total, 8 up, 7 in 2024-09-10T18:55:49.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:49 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/19025122' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:55:49.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:49 smithi018 ceph-mon[40196]: pgmap v476: 168 pgs: 168 active+clean; 582 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 0 B/s, 3 objects/s recovering 2024-09-10T18:55:49.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:49 smithi018 ceph-mon[29336]: osdmap e246: 8 total, 8 up, 7 in 2024-09-10T18:55:49.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:49 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/19025122' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:55:49.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:49 smithi018 ceph-mon[29336]: pgmap v476: 168 pgs: 168 active+clean; 582 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 0 B/s, 3 objects/s recovering 2024-09-10T18:55:50.035 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:50.076 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:50.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:50.078 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:50.179 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:50.211 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:50.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:50.212 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:50.313 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:50.356 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:50.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:50.357 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:50.382 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 2 16 6798 6782 211.922 208.5 0.00525397 0.00470857 2024-09-10T18:55:50.458 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:50.488 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:50.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:50.489 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:50.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:55:50 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:55:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:55:50.531 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:55:50.532 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:55:50.567 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:55:50.567 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:55:50.590 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:50.600 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:55:50.600 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:55:50.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:50.632 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:50.632 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:50.634 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:55:50.635 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:55:50.668 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:55:50.668 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:55:50.702 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:55:50.702 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:55:50.734 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:50.735 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:55:50.735 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:55:50.770 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:55:50.770 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:55:50.771 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:50.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:50.773 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:50.873 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:55:50.874 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:50 smithi103 ceph-mon[33163]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:55:50.875 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:50 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/19025122' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:55:50.875 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:50 smithi103 ceph-mon[33163]: osdmap e247: 8 total, 8 up, 7 in 2024-09-10T18:55:50.875 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:50 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:55:50.875 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:50 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:55:50.875 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:50 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2878657305' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:55:50.894 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:50 smithi018 ceph-mon[29336]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:55:50.894 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:50 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/19025122' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:55:50.894 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:50 smithi018 ceph-mon[29336]: osdmap e247: 8 total, 8 up, 7 in 2024-09-10T18:55:50.894 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:50 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:55:50.894 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:50 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:55:50.894 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:50 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2878657305' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:55:50.895 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:50 smithi018 ceph-mon[40196]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:55:50.895 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:50 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/19025122' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:55:50.895 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:50 smithi018 ceph-mon[40196]: osdmap e247: 8 total, 8 up, 7 in 2024-09-10T18:55:50.895 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:50 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:55:50.895 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:50 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:55:50.895 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:50 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2878657305' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:55:50.912 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:50.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:50.913 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:55:51.014 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:51.049 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:51.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:51.050 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:51.133 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:55:51.151 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:51.176 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:55:50 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:55:50] "GET /metrics HTTP/1.1" 200 223597 "" "Prometheus/2.43.0" 2024-09-10T18:55:51.202 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:51.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:51.204 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:51.305 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:51.339 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:51.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:51.340 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:51.382 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 3 16 9779 9763 203.38 186.312 0.00264313 0.00490892 2024-09-10T18:55:51.442 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:51.482 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:51.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:51.483 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:51.517 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T18:55:51.517 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:55:51.584 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:51.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:51.626 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:51.626 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:51.727 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:51.763 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:51.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:51.764 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:51.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:51 smithi103 ceph-mon[33163]: osdmap e248: 8 total, 8 up, 7 in 2024-09-10T18:55:51.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:51 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:55:51.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:51 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:55:51.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:51 smithi103 ceph-mon[33163]: pgmap v479: 168 pgs: 168 active+clean; 582 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T18:55:51.864 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:51.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:51.899 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:51.899 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:51.927 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:51 smithi018 ceph-mon[40196]: osdmap e248: 8 total, 8 up, 7 in 2024-09-10T18:55:51.927 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:51 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:55:51.927 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:51 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:55:51.927 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:51 smithi018 ceph-mon[40196]: pgmap v479: 168 pgs: 168 active+clean; 582 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T18:55:51.927 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:51 smithi018 ceph-mon[29336]: osdmap e248: 8 total, 8 up, 7 in 2024-09-10T18:55:51.927 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:51 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:55:51.927 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:51 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:55:51.927 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:51 smithi018 ceph-mon[29336]: pgmap v479: 168 pgs: 168 active+clean; 582 KiB data, 4.7 GiB used, 621 GiB / 626 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T18:55:51.999 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:52.030 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:52.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:52.031 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:52.132 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:52.164 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:52.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:52.165 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:52.265 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:55:52.304 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:52.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:52.305 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:55:52.321 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-10T18:55:52.382 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 4 16 13033 13017 203.374 203.375 0.00493951 0.00490951 2024-09-10T18:55:52.406 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:52.433 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:52.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:52.434 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:52.535 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:52.570 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:52.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:52.571 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:52.650 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:52 smithi103 ceph-mon[33163]: osdmap e249: 8 total, 8 up, 7 in 2024-09-10T18:55:52.672 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:52.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:52 smithi018 ceph-mon[40196]: osdmap e249: 8 total, 8 up, 7 in 2024-09-10T18:55:52.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:52 smithi018 ceph-mon[29336]: osdmap e249: 8 total, 8 up, 7 in 2024-09-10T18:55:52.712 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:52.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:52.713 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:52.717 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:55:52.813 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:52.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:52.848 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:52.848 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:52.949 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:52.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:52.982 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:52.983 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:53.084 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:53.117 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:53.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:53.119 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:53.178 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-10T18:55:53.220 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:53.267 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:53.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:53.268 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:53.369 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:53.382 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 5 16 15948 15932 199.134 182.188 0.00519303 0.00501571 2024-09-10T18:55:53.401 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:53.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:53.402 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:53.503 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:53.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:53.559 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:53.559 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:53.660 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:53.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:53.695 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:53.695 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:53.744 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:53 smithi018 ceph-mon[29336]: pgmap v481: 168 pgs: 168 active+clean; 505 MiB data, 6.1 GiB used, 620 GiB / 626 GiB avail; 1.1 KiB/s rd, 110 MiB/s wr, 1.75k op/s 2024-09-10T18:55:53.744 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:53 smithi018 ceph-mon[29336]: osdmap e250: 8 total, 8 up, 7 in 2024-09-10T18:55:53.744 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:53 smithi018 ceph-mon[40196]: pgmap v481: 168 pgs: 168 active+clean; 505 MiB data, 6.1 GiB used, 620 GiB / 626 GiB avail; 1.1 KiB/s rd, 110 MiB/s wr, 1.75k op/s 2024-09-10T18:55:53.744 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:53 smithi018 ceph-mon[40196]: osdmap e250: 8 total, 8 up, 7 in 2024-09-10T18:55:53.796 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:53.836 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:53.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:53.837 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:53.915 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:53 smithi103 ceph-mon[33163]: pgmap v481: 168 pgs: 168 active+clean; 505 MiB data, 6.1 GiB used, 620 GiB / 626 GiB avail; 1.1 KiB/s rd, 110 MiB/s wr, 1.75k op/s 2024-09-10T18:55:53.915 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:53 smithi103 ceph-mon[33163]: osdmap e250: 8 total, 8 up, 7 in 2024-09-10T18:55:53.938 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:53.995 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T18:55:54.027 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:54.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:54.028 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:54.128 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:55:54.129 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:54.155 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:54.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:54.156 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:54.257 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:54.284 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:54.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:54.285 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:54.382 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 6 16 18251 18235 189.927 143.938 0.00801668 0.00525779 2024-09-10T18:55:54.386 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:54.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:54.436 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:54.436 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:54.537 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:54.554 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:55:54.560 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-09-10T18:55:54.590 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:54.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:54.591 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:54.692 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:55:54.734 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:54.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:54.735 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:55:54.836 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:54.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:54.876 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:54.876 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:54.900 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T18:55:54.900 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:55:54.977 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:55.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:55.010 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:55.010 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:55.111 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:55.138 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:55.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:55.139 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:55.240 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:55.278 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:55.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:55.279 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:55.379 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:55.382 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 7 16 20477 20461 182.668 139.125 0.00811458 0.0054673 2024-09-10T18:55:55.411 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:55.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:55.412 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:55.513 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:55:55.550 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:55.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:55.552 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:55:55.578 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-10T18:55:55.652 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:55:55.694 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:55.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:55.695 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:55:55.796 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:55.834 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:55.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:55.835 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:55.936 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:55.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:55 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:55:55.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:55 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:55:55.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:55 smithi018 ceph-mon[29336]: from='client.25108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:55:55.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:55 smithi018 ceph-mon[29336]: osdmap e251: 8 total, 8 up, 7 in 2024-09-10T18:55:55.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:55 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:55:55.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:55.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:55 smithi018 ceph-mon[29336]: pgmap v484: 168 pgs: 168 active+clean; 693 MiB data, 6.3 GiB used, 620 GiB / 626 GiB avail; 1.1 KiB/s rd, 151 MiB/s wr, 2.42k op/s 2024-09-10T18:55:55.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:55 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:55:55.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:55 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:55:55.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:55 smithi018 ceph-mon[40196]: from='client.25108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:55:55.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:55 smithi018 ceph-mon[40196]: osdmap e251: 8 total, 8 up, 7 in 2024-09-10T18:55:55.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:55 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:55:55.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:55.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:55 smithi018 ceph-mon[40196]: pgmap v484: 168 pgs: 168 active+clean; 693 MiB data, 6.3 GiB used, 620 GiB / 626 GiB avail; 1.1 KiB/s rd, 151 MiB/s wr, 2.42k op/s 2024-09-10T18:55:55.973 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:55.973 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:55:55.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:55.974 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:56.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:55 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:55:56.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:55 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:55:56.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:55 smithi103 ceph-mon[33163]: from='client.25108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:55:56.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:55 smithi103 ceph-mon[33163]: osdmap e251: 8 total, 8 up, 7 in 2024-09-10T18:55:56.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:55 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:55:56.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:56.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:55 smithi103 ceph-mon[33163]: pgmap v484: 168 pgs: 168 active+clean; 693 MiB data, 6.3 GiB used, 620 GiB / 626 GiB avail; 1.1 KiB/s rd, 151 MiB/s wr, 2.42k op/s 2024-09-10T18:55:56.075 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:56.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:56.106 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:56.106 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:56.207 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:56.237 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:56.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:56.238 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:56.339 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:56.381 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:56.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:56.383 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:56.383 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 8 16 22845 22829 178.331 148 0.00615043 0.00560152 2024-09-10T18:55:56.417 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":482,"stamp":"2024-09-10T18:55:53.617725+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":726735068,"num_objects":11276,"num_object_clones":0,"num_object_copies":33828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11276,"num_whiteouts":0,"num_read":1767,"num_read_kb":1519,"num_write":11796,"num_write_kb":709163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":2591,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13627,"ondisk_log_size":13627,"up":504,"acting":504,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":512,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":6609868,"kb_used_data":4170204,"kb_used_omap":14,"kb_used_meta":2439473,"kb_avail":649606196,"statfs":{"total":671965249536,"available":665196744704,"internally_reserved":0,"allocated":4270288896,"data_stored":4266064326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14826,"internal_metadata":2498020886},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,2,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":726138880,"num_objects":11080,"num_object_clones":0,"num_object_copies":33240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11080,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":11080,"num_write_kb":709120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.391136"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":234,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.648710+0000","last_change":"2024-09-10T18:52:55.776643+0000","last_active":"2024-09-10T18:55:49.648710+0000","last_peered":"2024-09-10T18:55:49.648710+0000","last_clean":"2024-09-10T18:55:49.648710+0000","last_became_active":"2024-09-10T18:45:24.265488+0000","last_became_peered":"2024-09-10T18:45:24.265488+0000","last_unstale":"2024-09-10T18:55:49.648710+0000","last_undegraded":"2024-09-10T18:55:49.648710+0000","last_fullsized":"2024-09-10T18:55:49.648710+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:55.776552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:52:55.776552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003664676,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":219,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:50.645930+0000","last_change":"2024-09-10T18:55:40.388273+0000","last_active":"2024-09-10T18:55:50.645930+0000","last_peered":"2024-09-10T18:55:50.645930+0000","last_clean":"2024-09-10T18:55:50.645930+0000","last_became_active":"2024-09-10T18:55:40.387702+0000","last_became_peered":"2024-09-10T18:55:40.387702+0000","last_unstale":"2024-09-10T18:55:50.645930+0000","last_undegraded":"2024-09-10T18:55:50.645930+0000","last_fullsized":"2024-09-10T18:55:50.645930+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:02.360246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.718270+0000","last_clean_scrub_stamp":"2024-09-10T18:52:02.360246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"232'27","reported_seq":329,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.647161+0000","last_change":"2024-09-10T18:55:40.416655+0000","last_active":"2024-09-10T18:55:49.647161+0000","last_peered":"2024-09-10T18:55:49.647161+0000","last_clean":"2024-09-10T18:55:49.647161+0000","last_became_active":"2024-09-10T18:55:40.394717+0000","last_became_peered":"2024-09-10T18:55:40.394717+0000","last_unstale":"2024-09-10T18:55:49.647161+0000","last_undegraded":"2024-09-10T18:55:49.647161+0000","last_fullsized":"2024-09-10T18:55:49.647161+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:51:34.023191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:34.023191+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027158859999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":244,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.640043+0000","last_change":"2024-09-10T18:55:33.526359+0000","last_active":"2024-09-10T18:55:49.640043+0000","last_peered":"2024-09-10T18:55:49.640043+0000","last_clean":"2024-09-10T18:55:49.640043+0000","last_became_active":"2024-09-10T18:45:22.065619+0000","last_became_peered":"2024-09-10T18:45:22.065619+0000","last_unstale":"2024-09-10T18:55:49.640043+0000","last_undegraded":"2024-09-10T18:55:49.640043+0000","last_fullsized":"2024-09-10T18:55:49.640043+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:33.526302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.057932+0000","last_clean_scrub_stamp":"2024-09-10T18:55:33.526302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:53.822783+0000","scrub_duration":0.001517556,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":267,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.649782+0000","last_change":"2024-09-10T18:55:40.402409+0000","last_active":"2024-09-10T18:55:49.649782+0000","last_peered":"2024-09-10T18:55:49.649782+0000","last_clean":"2024-09-10T18:55:49.649782+0000","last_became_active":"2024-09-10T18:55:40.402140+0000","last_became_peered":"2024-09-10T18:55:40.402140+0000","last_unstale":"2024-09-10T18:55:49.649782+0000","last_undegraded":"2024-09-10T18:55:49.649782+0000","last_fullsized":"2024-09-10T18:55:49.649782+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:03.790824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:53:03.790824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003204742,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"232'19","reported_seq":265,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.640121+0000","last_change":"2024-09-10T18:54:48.324177+0000","last_active":"2024-09-10T18:55:49.640121+0000","last_peered":"2024-09-10T18:55:49.640121+0000","last_clean":"2024-09-10T18:55:49.640121+0000","last_became_active":"2024-09-10T18:45:20.061266+0000","last_became_peered":"2024-09-10T18:45:20.061266+0000","last_unstale":"2024-09-10T18:55:49.640121+0000","last_undegraded":"2024-09-10T18:55:49.640121+0000","last_fullsized":"2024-09-10T18:55:49.640121+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'13","last_scrub_stamp":"2024-09-10T18:54:48.323985+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_clean_scrub_stamp":"2024-09-10T18:54:48.323985+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:08.681715+0000","scrub_duration":0.0080421699999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":81,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.648953+0000","last_change":"2024-09-10T18:54:29.866079+0000","last_active":"2024-09-10T18:55:49.648953+0000","last_peered":"2024-09-10T18:55:49.648953+0000","last_clean":"2024-09-10T18:55:49.648953+0000","last_became_active":"2024-09-10T18:52:58.104329+0000","last_became_peered":"2024-09-10T18:52:58.104329+0000","last_unstale":"2024-09-10T18:55:49.648953+0000","last_undegraded":"2024-09-10T18:55:49.648953+0000","last_fullsized":"2024-09-10T18:55:49.648953+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:29.865500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:01.135988+0000","last_clean_scrub_stamp":"2024-09-10T18:54:29.865500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0092801540000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,7],"acting":[6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":232,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.640012+0000","last_change":"2024-09-10T18:54:22.051769+0000","last_active":"2024-09-10T18:55:49.640012+0000","last_peered":"2024-09-10T18:55:49.640012+0000","last_clean":"2024-09-10T18:55:49.640012+0000","last_became_active":"2024-09-10T18:45:24.081002+0000","last_became_peered":"2024-09-10T18:45:24.081002+0000","last_unstale":"2024-09-10T18:55:49.640012+0000","last_undegraded":"2024-09-10T18:55:49.640012+0000","last_fullsized":"2024-09-10T18:55:49.640012+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:22.051692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:54:22.051692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013425862,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":261,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.653476+0000","last_change":"2024-09-10T18:55:40.390679+0000","last_active":"2024-09-10T18:55:49.653476+0000","last_peered":"2024-09-10T18:55:49.653476+0000","last_clean":"2024-09-10T18:55:49.653476+0000","last_became_active":"2024-09-10T18:55:40.390416+0000","last_became_peered":"2024-09-10T18:55:40.390416+0000","last_unstale":"2024-09-10T18:55:49.653476+0000","last_undegraded":"2024-09-10T18:55:49.653476+0000","last_fullsized":"2024-09-10T18:55:49.653476+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:51.188458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:51.188458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037631460000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"232'9","reported_seq":251,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.640111+0000","last_change":"2024-09-10T18:54:45.305098+0000","last_active":"2024-09-10T18:55:49.640111+0000","last_peered":"2024-09-10T18:55:49.640111+0000","last_clean":"2024-09-10T18:55:49.640111+0000","last_became_active":"2024-09-10T18:45:20.057567+0000","last_became_peered":"2024-09-10T18:45:20.057567+0000","last_unstale":"2024-09-10T18:55:49.640111+0000","last_undegraded":"2024-09-10T18:55:49.640111+0000","last_fullsized":"2024-09-10T18:55:49.640111+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"163'7","last_scrub_stamp":"2024-09-10T18:54:45.305023+0000","last_deep_scrub":"163'7","last_deep_scrub_stamp":"2024-09-10T18:51:54.261879+0000","last_clean_scrub_stamp":"2024-09-10T18:54:45.305023+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:12.958679+0000","scrub_duration":0.0088188659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":253,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.652477+0000","last_change":"2024-09-10T18:55:27.785971+0000","last_active":"2024-09-10T18:55:49.652477+0000","last_peered":"2024-09-10T18:55:49.652477+0000","last_clean":"2024-09-10T18:55:49.652477+0000","last_became_active":"2024-09-10T18:45:22.076575+0000","last_became_peered":"2024-09-10T18:45:22.076575+0000","last_unstale":"2024-09-10T18:55:49.652477+0000","last_undegraded":"2024-09-10T18:55:49.652477+0000","last_fullsized":"2024-09-10T18:55:49.652477+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:27.785914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:36.509971+0000","last_clean_scrub_stamp":"2024-09-10T18:55:27.785914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:43.986870+0000","scrub_duration":0.001369809,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":93,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.642906+0000","last_change":"2024-09-10T18:55:40.399739+0000","last_active":"2024-09-10T18:55:49.642906+0000","last_peered":"2024-09-10T18:55:49.642906+0000","last_clean":"2024-09-10T18:55:49.642906+0000","last_became_active":"2024-09-10T18:55:40.399236+0000","last_became_peered":"2024-09-10T18:55:40.399236+0000","last_unstale":"2024-09-10T18:55:49.642906+0000","last_undegraded":"2024-09-10T18:55:49.642906+0000","last_fullsized":"2024-09-10T18:55:49.642906+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:51.287440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:51.287440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":261,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.653167+0000","last_change":"2024-09-10T18:54:40.672896+0000","last_active":"2024-09-10T18:55:49.653167+0000","last_peered":"2024-09-10T18:55:49.653167+0000","last_clean":"2024-09-10T18:55:49.653167+0000","last_became_active":"2024-09-10T18:47:36.414393+0000","last_became_peered":"2024-09-10T18:47:36.414393+0000","last_unstale":"2024-09-10T18:55:49.653167+0000","last_undegraded":"2024-09-10T18:55:49.653167+0000","last_fullsized":"2024-09-10T18:55:49.653167+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:40.672810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:54:40.672810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0078573129999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"232'15","reported_seq":287,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.640277+0000","last_change":"2024-09-10T18:55:40.407535+0000","last_active":"2024-09-10T18:55:49.640277+0000","last_peered":"2024-09-10T18:55:49.640277+0000","last_clean":"2024-09-10T18:55:49.640277+0000","last_became_active":"2024-09-10T18:55:40.391571+0000","last_became_peered":"2024-09-10T18:55:40.391571+0000","last_unstale":"2024-09-10T18:55:49.640277+0000","last_undegraded":"2024-09-10T18:55:49.640277+0000","last_fullsized":"2024-09-10T18:55:49.640277+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:53:01.112771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:53:01.112771+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0055933110000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":253,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.649401+0000","last_change":"2024-09-10T18:55:40.393399+0000","last_active":"2024-09-10T18:55:49.649401+0000","last_peered":"2024-09-10T18:55:49.649401+0000","last_clean":"2024-09-10T18:55:49.649401+0000","last_became_active":"2024-09-10T18:55:40.393118+0000","last_became_peered":"2024-09-10T18:55:40.393118+0000","last_unstale":"2024-09-10T18:55:49.649401+0000","last_undegraded":"2024-09-10T18:55:49.649401+0000","last_fullsized":"2024-09-10T18:55:49.649401+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:59.201336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:59.201336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0032166959999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":194,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:50.643099+0000","last_change":"2024-09-10T18:55:40.397758+0000","last_active":"2024-09-10T18:55:50.643099+0000","last_peered":"2024-09-10T18:55:50.643099+0000","last_clean":"2024-09-10T18:55:50.643099+0000","last_became_active":"2024-09-10T18:55:40.397082+0000","last_became_peered":"2024-09-10T18:55:40.397082+0000","last_unstale":"2024-09-10T18:55:50.643099+0000","last_undegraded":"2024-09-10T18:55:50.643099+0000","last_fullsized":"2024-09-10T18:55:50.643099+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:59.425718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:59.425718+0000","last_clean_scrub_stamp":"2024-09-10T18:52:59.425718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"248'634","reported_seq":660,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:51.443145+0000","last_change":"2024-09-10T18:55:40.403238+0000","last_active":"2024-09-10T18:55:51.443145+0000","last_peered":"2024-09-10T18:55:51.443145+0000","last_clean":"2024-09-10T18:55:51.443145+0000","last_became_active":"2024-09-10T18:55:40.402912+0000","last_became_peered":"2024-09-10T18:55:40.402912+0000","last_unstale":"2024-09-10T18:55:51.443145+0000","last_undegraded":"2024-09-10T18:55:51.443145+0000","last_fullsized":"2024-09-10T18:55:51.443145+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:04.717705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41549824,"num_objects":634,"num_object_clones":0,"num_object_copies":1902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":634,"num_write_kb":40576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"70'81","reported_seq":94,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.647776+0000","last_change":"2024-09-10T18:55:31.654161+0000","last_active":"2024-09-10T18:55:49.647776+0000","last_peered":"2024-09-10T18:55:49.647776+0000","last_clean":"2024-09-10T18:55:49.647776+0000","last_became_active":"2024-09-10T18:52:54.619039+0000","last_became_peered":"2024-09-10T18:52:54.619039+0000","last_unstale":"2024-09-10T18:55:49.647776+0000","last_undegraded":"2024-09-10T18:55:49.647776+0000","last_fullsized":"2024-09-10T18:55:49.647776+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:55:31.654090+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:55:31.654090+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:48.911364+0000","scrub_duration":0.0025082630000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"232'39","reported_seq":331,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.641129+0000","last_change":"2024-09-10T18:55:25.483970+0000","last_active":"2024-09-10T18:55:49.641129+0000","last_peered":"2024-09-10T18:55:49.641129+0000","last_clean":"2024-09-10T18:55:49.641129+0000","last_became_active":"2024-09-10T18:47:36.409300+0000","last_became_peered":"2024-09-10T18:47:36.409300+0000","last_unstale":"2024-09-10T18:55:49.641129+0000","last_undegraded":"2024-09-10T18:55:49.641129+0000","last_fullsized":"2024-09-10T18:55:49.641129+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'27","last_scrub_stamp":"2024-09-10T18:55:25.483919+0000","last_deep_scrub":"78'25","last_deep_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_clean_scrub_stamp":"2024-09-10T18:55:25.483919+0000","objects_scrubbed":8,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:38.819002+0000","scrub_duration":0.0017172610000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":254,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.653900+0000","last_change":"2024-09-10T18:55:30.719533+0000","last_active":"2024-09-10T18:55:49.653900+0000","last_peered":"2024-09-10T18:55:49.653900+0000","last_clean":"2024-09-10T18:55:49.653900+0000","last_became_active":"2024-09-10T18:45:17.963732+0000","last_became_peered":"2024-09-10T18:45:17.963732+0000","last_unstale":"2024-09-10T18:55:49.653900+0000","last_undegraded":"2024-09-10T18:55:49.653900+0000","last_fullsized":"2024-09-10T18:55:49.653900+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:30.719475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:55:30.719475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:56.769674+0000","scrub_duration":0.00119068,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":243,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.644544+0000","last_change":"2024-09-10T18:55:40.392731+0000","last_active":"2024-09-10T18:55:49.644544+0000","last_peered":"2024-09-10T18:55:49.644544+0000","last_clean":"2024-09-10T18:55:49.644544+0000","last_became_active":"2024-09-10T18:55:40.392449+0000","last_became_peered":"2024-09-10T18:55:40.392449+0000","last_unstale":"2024-09-10T18:55:49.644544+0000","last_undegraded":"2024-09-10T18:55:49.644544+0000","last_fullsized":"2024-09-10T18:55:49.644544+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:39.055556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:04.512425+0000","last_clean_scrub_stamp":"2024-09-10T18:51:39.055556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016498730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":151,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:50.646069+0000","last_change":"2024-09-10T18:55:40.388451+0000","last_active":"2024-09-10T18:55:50.646069+0000","last_peered":"2024-09-10T18:55:50.646069+0000","last_clean":"2024-09-10T18:55:50.646069+0000","last_became_active":"2024-09-10T18:55:40.388154+0000","last_became_peered":"2024-09-10T18:55:40.388154+0000","last_unstale":"2024-09-10T18:55:50.646069+0000","last_undegraded":"2024-09-10T18:55:50.646069+0000","last_fullsized":"2024-09-10T18:55:50.646069+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:05.317828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:53:05.317828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015035510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1","version":"248'672","reported_seq":690,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:51.590217+0000","last_change":"2024-09-10T18:55:36.354324+0000","last_active":"2024-09-10T18:55:51.590217+0000","last_peered":"2024-09-10T18:55:51.590217+0000","last_clean":"2024-09-10T18:55:51.590217+0000","last_became_active":"2024-09-10T18:55:36.354136+0000","last_became_peered":"2024-09-10T18:55:36.354136+0000","last_unstale":"2024-09-10T18:55:51.590217+0000","last_undegraded":"2024-09-10T18:55:51.590217+0000","last_fullsized":"2024-09-10T18:55:51.590217+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:57.105359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44040192,"num_objects":672,"num_object_clones":0,"num_object_copies":2016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":672,"num_write_kb":43008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.9","version":"70'81","reported_seq":315,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.651669+0000","last_change":"2024-09-10T18:54:45.753884+0000","last_active":"2024-09-10T18:55:49.651669+0000","last_peered":"2024-09-10T18:55:49.651669+0000","last_clean":"2024-09-10T18:55:49.651669+0000","last_became_active":"2024-09-10T18:50:08.399879+0000","last_became_peered":"2024-09-10T18:50:08.399879+0000","last_unstale":"2024-09-10T18:55:49.651669+0000","last_undegraded":"2024-09-10T18:55:49.651669+0000","last_fullsized":"2024-09-10T18:55:49.651669+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:54:45.753792+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:54:45.753792+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:11.311909+0000","scrub_duration":0.0072404879999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"232'25","reported_seq":271,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.641274+0000","last_change":"2024-09-10T18:53:06.056470+0000","last_active":"2024-09-10T18:55:49.641274+0000","last_peered":"2024-09-10T18:55:49.641274+0000","last_clean":"2024-09-10T18:55:49.641274+0000","last_became_active":"2024-09-10T18:45:20.053428+0000","last_became_peered":"2024-09-10T18:45:20.053428+0000","last_unstale":"2024-09-10T18:55:49.641274+0000","last_undegraded":"2024-09-10T18:55:49.641274+0000","last_fullsized":"2024-09-10T18:55:49.641274+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"116'15","last_scrub_stamp":"2024-09-10T18:53:06.056392+0000","last_deep_scrub":"116'15","last_deep_scrub_stamp":"2024-09-10T18:49:12.374527+0000","last_clean_scrub_stamp":"2024-09-10T18:53:06.056392+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0078605010000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":240,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.645965+0000","last_change":"2024-09-10T18:52:45.139636+0000","last_active":"2024-09-10T18:55:49.645965+0000","last_peered":"2024-09-10T18:55:49.645965+0000","last_clean":"2024-09-10T18:55:49.645965+0000","last_became_active":"2024-09-10T18:45:17.966833+0000","last_became_peered":"2024-09-10T18:45:17.966833+0000","last_unstale":"2024-09-10T18:55:49.645965+0000","last_undegraded":"2024-09-10T18:55:49.645965+0000","last_fullsized":"2024-09-10T18:55:49.645965+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:45.139534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:52:45.139534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002710269,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"242'12","reported_seq":355,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:50.649927+0000","last_change":"2024-09-10T18:55:40.390386+0000","last_active":"2024-09-10T18:55:50.649927+0000","last_peered":"2024-09-10T18:55:50.649927+0000","last_clean":"2024-09-10T18:55:50.649927+0000","last_became_active":"2024-09-10T18:55:40.389996+0000","last_became_peered":"2024-09-10T18:55:40.389996+0000","last_unstale":"2024-09-10T18:55:50.649927+0000","last_undegraded":"2024-09-10T18:55:50.649927+0000","last_fullsized":"2024-09-10T18:55:50.649927+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"202'10","last_scrub_stamp":"2024-09-10T18:53:45.547686+0000","last_deep_scrub":"202'10","last_deep_scrub_stamp":"2024-09-10T18:53:45.547686+0000","last_clean_scrub_stamp":"2024-09-10T18:53:45.547686+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.121847446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":262,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.641578+0000","last_change":"2024-09-10T18:55:37.565414+0000","last_active":"2024-09-10T18:55:49.641578+0000","last_peered":"2024-09-10T18:55:49.641578+0000","last_clean":"2024-09-10T18:55:49.641578+0000","last_became_active":"2024-09-10T18:52:54.599288+0000","last_became_peered":"2024-09-10T18:52:54.599288+0000","last_unstale":"2024-09-10T18:55:49.641578+0000","last_undegraded":"2024-09-10T18:55:49.641578+0000","last_fullsized":"2024-09-10T18:55:49.641578+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:37.565359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:32.043666+0000","last_clean_scrub_stamp":"2024-09-10T18:55:37.565359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:04.559538+0000","scrub_duration":0.0038365510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"248'634","reported_seq":652,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:51.588331+0000","last_change":"2024-09-10T18:55:36.356387+0000","last_active":"2024-09-10T18:55:51.588331+0000","last_peered":"2024-09-10T18:55:51.588331+0000","last_clean":"2024-09-10T18:55:51.588331+0000","last_became_active":"2024-09-10T18:55:36.355584+0000","last_became_peered":"2024-09-10T18:55:36.355584+0000","last_unstale":"2024-09-10T18:55:51.588331+0000","last_undegraded":"2024-09-10T18:55:51.588331+0000","last_fullsized":"2024-09-10T18:55:51.588331+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:52.013473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41549824,"num_objects":634,"num_object_clones":0,"num_object_copies":1902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":634,"num_write_kb":40576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.a","version":"70'81","reported_seq":161,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.649300+0000","last_change":"2024-09-10T18:55:37.921571+0000","last_active":"2024-09-10T18:55:49.649300+0000","last_peered":"2024-09-10T18:55:49.649300+0000","last_clean":"2024-09-10T18:55:49.649300+0000","last_became_active":"2024-09-10T18:50:08.398345+0000","last_became_peered":"2024-09-10T18:50:08.398345+0000","last_unstale":"2024-09-10T18:55:49.649300+0000","last_undegraded":"2024-09-10T18:55:49.649300+0000","last_fullsized":"2024-09-10T18:55:49.649300+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:55:37.921519+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:55:37.921519+0000","last_clean_scrub_stamp":"2024-09-10T18:55:37.921519+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:05.293052+0000","scrub_duration":0.002222407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"232'34","reported_seq":255,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.650764+0000","last_change":"2024-09-10T18:53:00.188252+0000","last_active":"2024-09-10T18:55:49.650764+0000","last_peered":"2024-09-10T18:55:49.650764+0000","last_clean":"2024-09-10T18:55:49.650764+0000","last_became_active":"2024-09-10T18:46:56.742946+0000","last_became_peered":"2024-09-10T18:46:56.742946+0000","last_unstale":"2024-09-10T18:55:49.650764+0000","last_undegraded":"2024-09-10T18:55:49.650764+0000","last_fullsized":"2024-09-10T18:55:49.650764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"141'22","last_scrub_stamp":"2024-09-10T18:53:00.188166+0000","last_deep_scrub":"141'22","last_deep_scrub_stamp":"2024-09-10T18:50:57.119152+0000","last_clean_scrub_stamp":"2024-09-10T18:53:00.188166+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0091410329999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":243,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.642862+0000","last_change":"2024-09-10T18:55:27.457289+0000","last_active":"2024-09-10T18:55:49.642862+0000","last_peered":"2024-09-10T18:55:49.642862+0000","last_clean":"2024-09-10T18:55:49.642862+0000","last_became_active":"2024-09-10T18:45:17.965409+0000","last_became_peered":"2024-09-10T18:45:17.965409+0000","last_unstale":"2024-09-10T18:55:49.642862+0000","last_undegraded":"2024-09-10T18:55:49.642862+0000","last_fullsized":"2024-09-10T18:55:49.642862+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:51.054935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_clean_scrub_stamp":"2024-09-10T18:51:51.054935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004679758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"242'10","reported_seq":388,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:50.645547+0000","last_change":"2024-09-10T18:55:40.397697+0000","last_active":"2024-09-10T18:55:50.645547+0000","last_peered":"2024-09-10T18:55:50.645547+0000","last_clean":"2024-09-10T18:55:50.645547+0000","last_became_active":"2024-09-10T18:55:40.397246+0000","last_became_peered":"2024-09-10T18:55:40.397246+0000","last_unstale":"2024-09-10T18:55:50.645547+0000","last_undegraded":"2024-09-10T18:55:50.645547+0000","last_fullsized":"2024-09-10T18:55:50.645547+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:52:46.200266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:52:46.200266+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.049044943000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":251,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.649554+0000","last_change":"2024-09-10T18:55:40.393252+0000","last_active":"2024-09-10T18:55:49.649554+0000","last_peered":"2024-09-10T18:55:49.649554+0000","last_clean":"2024-09-10T18:55:49.649554+0000","last_became_active":"2024-09-10T18:55:40.392003+0000","last_became_peered":"2024-09-10T18:55:40.392003+0000","last_unstale":"2024-09-10T18:55:49.649554+0000","last_undegraded":"2024-09-10T18:55:49.649554+0000","last_fullsized":"2024-09-10T18:55:49.649554+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:54.218602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:54.218602+0000","last_clean_scrub_stamp":"2024-09-10T18:51:54.218602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0059638290000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"248'617","reported_seq":636,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:51.202855+0000","last_change":"2024-09-10T18:55:36.355594+0000","last_active":"2024-09-10T18:55:51.202855+0000","last_peered":"2024-09-10T18:55:51.202855+0000","last_clean":"2024-09-10T18:55:51.202855+0000","last_became_active":"2024-09-10T18:55:36.355300+0000","last_became_peered":"2024-09-10T18:55:36.355300+0000","last_unstale":"2024-09-10T18:55:51.202855+0000","last_undegraded":"2024-09-10T18:55:51.202855+0000","last_fullsized":"2024-09-10T18:55:51.202855+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:49.815521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40435712,"num_objects":617,"num_object_clones":0,"num_object_copies":1851,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":617,"num_write_kb":39488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.b","version":"70'81","reported_seq":193,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.644006+0000","last_change":"2024-09-10T18:55:40.402680+0000","last_active":"2024-09-10T18:55:49.644006+0000","last_peered":"2024-09-10T18:55:49.644006+0000","last_clean":"2024-09-10T18:55:49.644006+0000","last_became_active":"2024-09-10T18:55:40.401643+0000","last_became_peered":"2024-09-10T18:55:40.401643+0000","last_unstale":"2024-09-10T18:55:49.644006+0000","last_undegraded":"2024-09-10T18:55:49.644006+0000","last_fullsized":"2024-09-10T18:55:49.644006+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:53:04.265619+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:53:04.265619+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016066791,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"232'24","reported_seq":348,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.641791+0000","last_change":"2024-09-10T18:54:49.297032+0000","last_active":"2024-09-10T18:55:49.641791+0000","last_peered":"2024-09-10T18:55:49.641791+0000","last_clean":"2024-09-10T18:55:49.641791+0000","last_became_active":"2024-09-10T18:47:57.106935+0000","last_became_peered":"2024-09-10T18:47:57.106935+0000","last_unstale":"2024-09-10T18:55:49.641791+0000","last_undegraded":"2024-09-10T18:55:49.641791+0000","last_fullsized":"2024-09-10T18:55:49.641791+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"224'16","last_scrub_stamp":"2024-09-10T18:54:49.296957+0000","last_deep_scrub":"224'16","last_deep_scrub_stamp":"2024-09-10T18:54:49.296957+0000","last_clean_scrub_stamp":"2024-09-10T18:54:49.296957+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:06.425870+0000","scrub_duration":0.014223285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":268,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.650922+0000","last_change":"2024-09-10T18:55:40.397594+0000","last_active":"2024-09-10T18:55:49.650922+0000","last_peered":"2024-09-10T18:55:49.650922+0000","last_clean":"2024-09-10T18:55:49.650922+0000","last_became_active":"2024-09-10T18:55:40.396954+0000","last_became_peered":"2024-09-10T18:55:40.396954+0000","last_unstale":"2024-09-10T18:55:49.650922+0000","last_undegraded":"2024-09-10T18:55:49.650922+0000","last_fullsized":"2024-09-10T18:55:49.650922+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:43.419135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:53:43.419135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053276879999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":249,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.650901+0000","last_change":"2024-09-10T18:55:26.909204+0000","last_active":"2024-09-10T18:55:49.650901+0000","last_peered":"2024-09-10T18:55:49.650901+0000","last_clean":"2024-09-10T18:55:49.650901+0000","last_became_active":"2024-09-10T18:45:22.068918+0000","last_became_peered":"2024-09-10T18:45:22.068918+0000","last_unstale":"2024-09-10T18:55:49.650901+0000","last_undegraded":"2024-09-10T18:55:49.650901+0000","last_fullsized":"2024-09-10T18:55:49.650901+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:26.909143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_clean_scrub_stamp":"2024-09-10T18:55:26.909143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:32.278969+0000","scrub_duration":0.0013975070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":234,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.650229+0000","last_change":"2024-09-10T18:54:51.399064+0000","last_active":"2024-09-10T18:55:49.650229+0000","last_peered":"2024-09-10T18:55:49.650229+0000","last_clean":"2024-09-10T18:55:49.650229+0000","last_became_active":"2024-09-10T18:45:24.264538+0000","last_became_peered":"2024-09-10T18:45:24.264538+0000","last_unstale":"2024-09-10T18:55:49.650229+0000","last_undegraded":"2024-09-10T18:55:49.650229+0000","last_fullsized":"2024-09-10T18:55:49.650229+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:51.398981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:51.398981+0000","last_clean_scrub_stamp":"2024-09-10T18:54:51.398981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:19.907055+0000","scrub_duration":0.0027589680000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"248'622","reported_seq":640,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:51.574803+0000","last_change":"2024-09-10T18:55:36.355579+0000","last_active":"2024-09-10T18:55:51.574803+0000","last_peered":"2024-09-10T18:55:51.574803+0000","last_clean":"2024-09-10T18:55:51.574803+0000","last_became_active":"2024-09-10T18:55:36.355193+0000","last_became_peered":"2024-09-10T18:55:36.355193+0000","last_unstale":"2024-09-10T18:55:51.574803+0000","last_undegraded":"2024-09-10T18:55:51.574803+0000","last_fullsized":"2024-09-10T18:55:51.574803+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:39.304440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40763392,"num_objects":622,"num_object_clones":0,"num_object_copies":1866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":622,"num_write_kb":39808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"70'81","reported_seq":172,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.650354+0000","last_change":"2024-09-10T18:54:42.406852+0000","last_active":"2024-09-10T18:55:49.650354+0000","last_peered":"2024-09-10T18:55:49.650354+0000","last_clean":"2024-09-10T18:55:49.650354+0000","last_became_active":"2024-09-10T18:52:54.622187+0000","last_became_peered":"2024-09-10T18:52:54.622187+0000","last_unstale":"2024-09-10T18:55:49.650354+0000","last_undegraded":"2024-09-10T18:55:49.650354+0000","last_fullsized":"2024-09-10T18:55:49.650354+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:54:42.406761+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:54:42.406761+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037296899999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"232'35","reported_seq":285,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.639638+0000","last_change":"2024-09-10T18:52:59.095130+0000","last_active":"2024-09-10T18:55:49.639638+0000","last_peered":"2024-09-10T18:55:49.639638+0000","last_clean":"2024-09-10T18:55:49.639638+0000","last_became_active":"2024-09-10T18:45:20.052243+0000","last_became_peered":"2024-09-10T18:45:20.052243+0000","last_unstale":"2024-09-10T18:55:49.639638+0000","last_undegraded":"2024-09-10T18:55:49.639638+0000","last_fullsized":"2024-09-10T18:55:49.639638+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"143'21","last_scrub_stamp":"2024-09-10T18:52:59.095048+0000","last_deep_scrub":"143'21","last_deep_scrub_stamp":"2024-09-10T18:51:02.540216+0000","last_clean_scrub_stamp":"2024-09-10T18:52:59.095048+0000","objects_scrubbed":7,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013944524,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":260,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.648369+0000","last_change":"2024-09-10T18:54:53.087341+0000","last_active":"2024-09-10T18:55:49.648369+0000","last_peered":"2024-09-10T18:55:49.648369+0000","last_clean":"2024-09-10T18:55:49.648369+0000","last_became_active":"2024-09-10T18:47:36.424466+0000","last_became_peered":"2024-09-10T18:47:36.424466+0000","last_unstale":"2024-09-10T18:55:49.648369+0000","last_undegraded":"2024-09-10T18:55:49.648369+0000","last_fullsized":"2024-09-10T18:55:49.648369+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:53.087262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_clean_scrub_stamp":"2024-09-10T18:54:53.087262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:13.800823+0000","scrub_duration":0.010484613,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":371,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:50.651586+0000","last_change":"2024-09-10T18:53:02.761328+0000","last_active":"2024-09-10T18:55:50.651586+0000","last_peered":"2024-09-10T18:55:50.651586+0000","last_clean":"2024-09-10T18:55:50.651586+0000","last_became_active":"2024-09-10T18:45:22.074540+0000","last_became_peered":"2024-09-10T18:45:22.074540+0000","last_unstale":"2024-09-10T18:55:50.651586+0000","last_undegraded":"2024-09-10T18:55:50.651586+0000","last_fullsized":"2024-09-10T18:55:50.651586+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:53:02.761109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:53:02.761109+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012855039,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":265,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.653631+0000","last_change":"2024-09-10T18:54:49.759104+0000","last_active":"2024-09-10T18:55:49.653631+0000","last_peered":"2024-09-10T18:55:49.653631+0000","last_clean":"2024-09-10T18:55:49.653631+0000","last_became_active":"2024-09-10T18:47:36.414665+0000","last_became_peered":"2024-09-10T18:47:36.414665+0000","last_unstale":"2024-09-10T18:55:49.653631+0000","last_undegraded":"2024-09-10T18:55:49.653631+0000","last_fullsized":"2024-09-10T18:55:49.653631+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:49.758989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:54:49.758989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:06.243008+0000","scrub_duration":0.010517354,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.d","version":"248'579","reported_seq":597,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:51.214037+0000","last_change":"2024-09-10T18:55:36.359401+0000","last_active":"2024-09-10T18:55:51.214037+0000","last_peered":"2024-09-10T18:55:51.214037+0000","last_clean":"2024-09-10T18:55:51.214037+0000","last_became_active":"2024-09-10T18:55:36.359124+0000","last_became_peered":"2024-09-10T18:55:36.359124+0000","last_unstale":"2024-09-10T18:55:51.214037+0000","last_undegraded":"2024-09-10T18:55:51.214037+0000","last_fullsized":"2024-09-10T18:55:51.214037+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:51.170482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37945344,"num_objects":579,"num_object_clones":0,"num_object_copies":1737,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":579,"num_write_kb":37056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"70'81","reported_seq":156,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.640891+0000","last_change":"2024-09-10T18:52:54.120550+0000","last_active":"2024-09-10T18:55:49.640891+0000","last_peered":"2024-09-10T18:55:49.640891+0000","last_clean":"2024-09-10T18:55:49.640891+0000","last_became_active":"2024-09-10T18:52:14.584798+0000","last_became_peered":"2024-09-10T18:52:14.584798+0000","last_unstale":"2024-09-10T18:55:49.640891+0000","last_undegraded":"2024-09-10T18:55:49.640891+0000","last_fullsized":"2024-09-10T18:55:49.640891+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:52:54.120497+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:52:54.120497+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001089507,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"232'30","reported_seq":320,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.647786+0000","last_change":"2024-09-10T18:55:40.418263+0000","last_active":"2024-09-10T18:55:49.647786+0000","last_peered":"2024-09-10T18:55:49.647786+0000","last_clean":"2024-09-10T18:55:49.647786+0000","last_became_active":"2024-09-10T18:55:40.394879+0000","last_became_peered":"2024-09-10T18:55:40.394879+0000","last_unstale":"2024-09-10T18:55:49.647786+0000","last_undegraded":"2024-09-10T18:55:49.647786+0000","last_fullsized":"2024-09-10T18:55:49.647786+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'18","last_scrub_stamp":"2024-09-10T18:53:01.746777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:53:01.746777+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.020214684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":243,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.645140+0000","last_change":"2024-09-10T18:54:46.142189+0000","last_active":"2024-09-10T18:55:49.645140+0000","last_peered":"2024-09-10T18:55:49.645140+0000","last_clean":"2024-09-10T18:55:49.645140+0000","last_became_active":"2024-09-10T18:45:17.961695+0000","last_became_peered":"2024-09-10T18:45:17.961695+0000","last_unstale":"2024-09-10T18:55:49.645140+0000","last_undegraded":"2024-09-10T18:55:49.645140+0000","last_fullsized":"2024-09-10T18:55:49.645140+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:46.142129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:46.142129+0000","last_clean_scrub_stamp":"2024-09-10T18:54:46.142129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:55:53.768256+0000","scrub_duration":0.0015906060000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":244,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.639404+0000","last_change":"2024-09-10T18:55:29.533227+0000","last_active":"2024-09-10T18:55:49.639404+0000","last_peered":"2024-09-10T18:55:49.639404+0000","last_clean":"2024-09-10T18:55:49.639404+0000","last_became_active":"2024-09-10T18:45:22.074012+0000","last_became_peered":"2024-09-10T18:45:22.074012+0000","last_unstale":"2024-09-10T18:55:49.639404+0000","last_undegraded":"2024-09-10T18:55:49.639404+0000","last_fullsized":"2024-09-10T18:55:49.639404+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:29.533177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:55:29.533177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:56.213796+0000","scrub_duration":0.0012137999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":242,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.640204+0000","last_change":"2024-09-10T18:54:25.102613+0000","last_active":"2024-09-10T18:55:49.640204+0000","last_peered":"2024-09-10T18:55:49.640204+0000","last_clean":"2024-09-10T18:55:49.640204+0000","last_became_active":"2024-09-10T18:47:36.410836+0000","last_became_peered":"2024-09-10T18:47:36.410836+0000","last_unstale":"2024-09-10T18:55:49.640204+0000","last_undegraded":"2024-09-10T18:55:49.640204+0000","last_fullsized":"2024-09-10T18:55:49.640204+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:25.102358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_clean_scrub_stamp":"2024-09-10T18:54:25.102358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060015040000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.e","version":"248'566","reported_seq":585,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:51.211735+0000","last_change":"2024-09-10T18:55:36.358310+0000","last_active":"2024-09-10T18:55:51.211735+0000","last_peered":"2024-09-10T18:55:51.211735+0000","last_clean":"2024-09-10T18:55:51.211735+0000","last_became_active":"2024-09-10T18:55:36.357714+0000","last_became_peered":"2024-09-10T18:55:36.357714+0000","last_unstale":"2024-09-10T18:55:51.211735+0000","last_undegraded":"2024-09-10T18:55:51.211735+0000","last_fullsized":"2024-09-10T18:55:51.211735+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:44.330187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37093376,"num_objects":566,"num_object_clones":0,"num_object_copies":1698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":566,"num_write_kb":36224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"70'81","reported_seq":243,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.648060+0000","last_change":"2024-09-10T18:55:40.397243+0000","last_active":"2024-09-10T18:55:49.648060+0000","last_peered":"2024-09-10T18:55:49.648060+0000","last_clean":"2024-09-10T18:55:49.648060+0000","last_became_active":"2024-09-10T18:55:40.396828+0000","last_became_peered":"2024-09-10T18:55:40.396828+0000","last_unstale":"2024-09-10T18:55:49.648060+0000","last_undegraded":"2024-09-10T18:55:49.648060+0000","last_fullsized":"2024-09-10T18:55:49.648060+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:52:50.640292+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:52:50.640292+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043487049999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"232'29","reported_seq":268,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:50.645107+0000","last_change":"2024-09-10T18:55:40.411118+0000","last_active":"2024-09-10T18:55:50.645107+0000","last_peered":"2024-09-10T18:55:50.645107+0000","last_clean":"2024-09-10T18:55:50.645107+0000","last_became_active":"2024-09-10T18:55:40.391264+0000","last_became_peered":"2024-09-10T18:55:40.391264+0000","last_unstale":"2024-09-10T18:55:50.645107+0000","last_undegraded":"2024-09-10T18:55:50.645107+0000","last_fullsized":"2024-09-10T18:55:50.645107+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:53:38.384804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:53:38.384804+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":258,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.653562+0000","last_change":"2024-09-10T18:55:40.390590+0000","last_active":"2024-09-10T18:55:49.653562+0000","last_peered":"2024-09-10T18:55:49.653562+0000","last_clean":"2024-09-10T18:55:49.653562+0000","last_became_active":"2024-09-10T18:55:40.390100+0000","last_became_peered":"2024-09-10T18:55:40.390100+0000","last_unstale":"2024-09-10T18:55:49.653562+0000","last_undegraded":"2024-09-10T18:55:49.653562+0000","last_fullsized":"2024-09-10T18:55:49.653562+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:48.196065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:48.196065+0000","last_clean_scrub_stamp":"2024-09-10T18:51:48.196065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013037999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":253,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.647671+0000","last_change":"2024-09-10T18:55:40.388082+0000","last_active":"2024-09-10T18:55:49.647671+0000","last_peered":"2024-09-10T18:55:49.647671+0000","last_clean":"2024-09-10T18:55:49.647671+0000","last_became_active":"2024-09-10T18:55:40.387857+0000","last_became_peered":"2024-09-10T18:55:40.387857+0000","last_unstale":"2024-09-10T18:55:49.647671+0000","last_undegraded":"2024-09-10T18:55:49.647671+0000","last_fullsized":"2024-09-10T18:55:49.647671+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:53.548279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:53.548279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048693970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":244,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.642668+0000","last_change":"2024-09-10T18:55:30.482717+0000","last_active":"2024-09-10T18:55:49.642668+0000","last_peered":"2024-09-10T18:55:49.642668+0000","last_clean":"2024-09-10T18:55:49.642668+0000","last_became_active":"2024-09-10T18:47:36.422295+0000","last_became_peered":"2024-09-10T18:47:36.422295+0000","last_unstale":"2024-09-10T18:55:49.642668+0000","last_undegraded":"2024-09-10T18:55:49.642668+0000","last_fullsized":"2024-09-10T18:55:49.642668+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:30.482659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:49:18.281230+0000","last_clean_scrub_stamp":"2024-09-10T18:55:30.482659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:42.005520+0000","scrub_duration":0.0015073689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.f","version":"248'648","reported_seq":667,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:51.380191+0000","last_change":"2024-09-10T18:55:36.358334+0000","last_active":"2024-09-10T18:55:51.380191+0000","last_peered":"2024-09-10T18:55:51.380191+0000","last_clean":"2024-09-10T18:55:51.380191+0000","last_became_active":"2024-09-10T18:55:36.358064+0000","last_became_peered":"2024-09-10T18:55:36.358064+0000","last_unstale":"2024-09-10T18:55:51.380191+0000","last_undegraded":"2024-09-10T18:55:51.380191+0000","last_fullsized":"2024-09-10T18:55:51.380191+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":648,"log_dups_size":0,"ondisk_log_size":648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:01.132365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42467328,"num_objects":648,"num_object_clones":0,"num_object_copies":1944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":648,"num_write_kb":41472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"70'81","reported_seq":236,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.647835+0000","last_change":"2024-09-10T18:54:46.030156+0000","last_active":"2024-09-10T18:55:49.647835+0000","last_peered":"2024-09-10T18:55:49.647835+0000","last_clean":"2024-09-10T18:55:49.647835+0000","last_became_active":"2024-09-10T18:52:14.582796+0000","last_became_peered":"2024-09-10T18:52:14.582796+0000","last_unstale":"2024-09-10T18:55:49.647835+0000","last_undegraded":"2024-09-10T18:55:49.647835+0000","last_fullsized":"2024-09-10T18:55:49.647835+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:54:46.030076+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:54:46.030076+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:13.712910+0000","scrub_duration":0.0012983280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"232'25","reported_seq":325,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.645818+0000","last_change":"2024-09-10T18:55:40.415292+0000","last_active":"2024-09-10T18:55:49.645818+0000","last_peered":"2024-09-10T18:55:49.645818+0000","last_clean":"2024-09-10T18:55:49.645818+0000","last_became_active":"2024-09-10T18:55:40.400777+0000","last_became_peered":"2024-09-10T18:55:40.400777+0000","last_unstale":"2024-09-10T18:55:49.645818+0000","last_undegraded":"2024-09-10T18:55:49.645818+0000","last_fullsized":"2024-09-10T18:55:49.645818+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'15","last_scrub_stamp":"2024-09-10T18:52:04.182782+0000","last_deep_scrub":"83'15","last_deep_scrub_stamp":"2024-09-10T18:46:58.436473+0000","last_clean_scrub_stamp":"2024-09-10T18:52:04.182782+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0049024990000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":267,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.644899+0000","last_change":"2024-09-10T18:55:40.392113+0000","last_active":"2024-09-10T18:55:49.644899+0000","last_peered":"2024-09-10T18:55:49.644899+0000","last_clean":"2024-09-10T18:55:49.644899+0000","last_became_active":"2024-09-10T18:55:40.391805+0000","last_became_peered":"2024-09-10T18:55:40.391805+0000","last_unstale":"2024-09-10T18:55:49.644899+0000","last_undegraded":"2024-09-10T18:55:49.644899+0000","last_fullsized":"2024-09-10T18:55:49.644899+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:36.259868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:05.140027+0000","last_clean_scrub_stamp":"2024-09-10T18:53:36.259868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034952049999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"242'11","reported_seq":315,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:51.199382+0000","last_change":"2024-09-10T18:55:40.391903+0000","last_active":"2024-09-10T18:55:51.199382+0000","last_peered":"2024-09-10T18:55:51.199382+0000","last_clean":"2024-09-10T18:55:51.199382+0000","last_became_active":"2024-09-10T18:55:40.391094+0000","last_became_peered":"2024-09-10T18:55:40.391094+0000","last_unstale":"2024-09-10T18:55:51.199382+0000","last_undegraded":"2024-09-10T18:55:51.199382+0000","last_fullsized":"2024-09-10T18:55:51.199382+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:51:37.206224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:37.206224+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001760707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":210,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:50.642751+0000","last_change":"2024-09-10T18:55:40.398023+0000","last_active":"2024-09-10T18:55:50.642751+0000","last_peered":"2024-09-10T18:55:50.642751+0000","last_clean":"2024-09-10T18:55:50.642751+0000","last_became_active":"2024-09-10T18:55:40.397478+0000","last_became_peered":"2024-09-10T18:55:40.397478+0000","last_unstale":"2024-09-10T18:55:50.642751+0000","last_undegraded":"2024-09-10T18:55:50.642751+0000","last_fullsized":"2024-09-10T18:55:50.642751+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:55.515593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:52:55.515593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0072066109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.9","version":"249'1030","reported_seq":1047,"reported_epoch":249,"state":"active+clean","last_fresh":"2024-09-10T18:55:53.370096+0000","last_change":"2024-09-10T18:55:40.389308+0000","last_active":"2024-09-10T18:55:53.370096+0000","last_peered":"2024-09-10T18:55:53.370096+0000","last_clean":"2024-09-10T18:55:53.370096+0000","last_became_active":"2024-09-10T18:55:40.388325+0000","last_became_peered":"2024-09-10T18:55:40.388325+0000","last_unstale":"2024-09-10T18:55:53.370096+0000","last_undegraded":"2024-09-10T18:55:53.370096+0000","last_fullsized":"2024-09-10T18:55:53.370096+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":1030,"log_dups_size":0,"ondisk_log_size":1030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:42.480386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67502080,"num_objects":1030,"num_object_clones":0,"num_object_copies":3090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1030,"num_write_kb":65920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"70'81","reported_seq":175,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.649187+0000","last_change":"2024-09-10T18:55:40.392784+0000","last_active":"2024-09-10T18:55:49.649187+0000","last_peered":"2024-09-10T18:55:49.649187+0000","last_clean":"2024-09-10T18:55:49.649187+0000","last_became_active":"2024-09-10T18:55:40.392432+0000","last_became_peered":"2024-09-10T18:55:40.392432+0000","last_unstale":"2024-09-10T18:55:49.649187+0000","last_undegraded":"2024-09-10T18:55:49.649187+0000","last_fullsized":"2024-09-10T18:55:49.649187+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:52:50.091862+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:52:50.091862+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013302506,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"232'41","reported_seq":463,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:50.650043+0000","last_change":"2024-09-10T18:51:57.140326+0000","last_active":"2024-09-10T18:55:50.650043+0000","last_peered":"2024-09-10T18:55:50.650043+0000","last_clean":"2024-09-10T18:55:50.650043+0000","last_became_active":"2024-09-10T18:47:36.420185+0000","last_became_peered":"2024-09-10T18:47:36.420185+0000","last_unstale":"2024-09-10T18:55:50.650043+0000","last_undegraded":"2024-09-10T18:55:50.650043+0000","last_fullsized":"2024-09-10T18:55:50.650043+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'29","last_scrub_stamp":"2024-09-10T18:51:57.140236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:57.140236+0000","objects_scrubbed":10,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012827955,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":253,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.650338+0000","last_change":"2024-09-10T18:55:29.867732+0000","last_active":"2024-09-10T18:55:49.650338+0000","last_peered":"2024-09-10T18:55:49.650338+0000","last_clean":"2024-09-10T18:55:49.650338+0000","last_became_active":"2024-09-10T18:45:17.964494+0000","last_became_peered":"2024-09-10T18:45:17.964494+0000","last_unstale":"2024-09-10T18:55:49.650338+0000","last_undegraded":"2024-09-10T18:55:49.650338+0000","last_fullsized":"2024-09-10T18:55:49.650338+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:29.867683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:12.982599+0000","last_clean_scrub_stamp":"2024-09-10T18:55:29.867683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:52.216063+0000","scrub_duration":0.0015951909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":194,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.647592+0000","last_change":"2024-09-10T18:54:19.744213+0000","last_active":"2024-09-10T18:55:49.647592+0000","last_peered":"2024-09-10T18:55:49.647592+0000","last_clean":"2024-09-10T18:55:49.647592+0000","last_became_active":"2024-09-10T18:47:57.107478+0000","last_became_peered":"2024-09-10T18:47:57.107478+0000","last_unstale":"2024-09-10T18:55:49.647592+0000","last_undegraded":"2024-09-10T18:55:49.647592+0000","last_fullsized":"2024-09-10T18:55:49.647592+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:19.744135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:54:19.744135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0075590960000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":224,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.649496+0000","last_change":"2024-09-10T18:52:47.909320+0000","last_active":"2024-09-10T18:55:49.649496+0000","last_peered":"2024-09-10T18:55:49.649496+0000","last_clean":"2024-09-10T18:55:49.649496+0000","last_became_active":"2024-09-10T18:45:24.073307+0000","last_became_peered":"2024-09-10T18:45:24.073307+0000","last_unstale":"2024-09-10T18:55:49.649496+0000","last_undegraded":"2024-09-10T18:55:49.649496+0000","last_fullsized":"2024-09-10T18:55:49.649496+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:47.909248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:22.891084+0000","last_clean_scrub_stamp":"2024-09-10T18:52:47.909248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0077988500000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"248'599","reported_seq":618,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:51.434749+0000","last_change":"2024-09-10T18:55:36.355697+0000","last_active":"2024-09-10T18:55:51.434749+0000","last_peered":"2024-09-10T18:55:51.434749+0000","last_clean":"2024-09-10T18:55:51.434749+0000","last_became_active":"2024-09-10T18:55:36.355371+0000","last_became_peered":"2024-09-10T18:55:36.355371+0000","last_unstale":"2024-09-10T18:55:51.434749+0000","last_undegraded":"2024-09-10T18:55:51.434749+0000","last_fullsized":"2024-09-10T18:55:51.434749+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:36.148493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39256064,"num_objects":599,"num_object_clones":0,"num_object_copies":1797,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":599,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":599,"num_write_kb":38336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"70'81","reported_seq":332,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.652668+0000","last_change":"2024-09-10T18:55:40.390057+0000","last_active":"2024-09-10T18:55:49.652668+0000","last_peered":"2024-09-10T18:55:49.652668+0000","last_clean":"2024-09-10T18:55:49.652668+0000","last_became_active":"2024-09-10T18:55:40.389718+0000","last_became_peered":"2024-09-10T18:55:40.389718+0000","last_unstale":"2024-09-10T18:55:49.652668+0000","last_undegraded":"2024-09-10T18:55:49.652668+0000","last_fullsized":"2024-09-10T18:55:49.652668+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:52:54.096066+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:52:54.096066+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012293219999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"232'22","reported_seq":290,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.644370+0000","last_change":"2024-09-10T18:55:40.409309+0000","last_active":"2024-09-10T18:55:49.644370+0000","last_peered":"2024-09-10T18:55:49.644370+0000","last_clean":"2024-09-10T18:55:49.644370+0000","last_became_active":"2024-09-10T18:55:40.397422+0000","last_became_peered":"2024-09-10T18:55:40.397422+0000","last_unstale":"2024-09-10T18:55:49.644370+0000","last_undegraded":"2024-09-10T18:55:49.644370+0000","last_fullsized":"2024-09-10T18:55:49.644370+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'18","last_scrub_stamp":"2024-09-10T18:51:37.059477+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-10T18:47:10.514951+0000","last_clean_scrub_stamp":"2024-09-10T18:51:37.059477+0000","objects_scrubbed":5,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020902970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"84'6","reported_seq":892,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:50.649662+0000","last_change":"2024-09-10T18:55:40.403749+0000","last_active":"2024-09-10T18:55:50.649662+0000","last_peered":"2024-09-10T18:55:50.649662+0000","last_clean":"2024-09-10T18:55:50.649662+0000","last_became_active":"2024-09-10T18:55:40.403098+0000","last_became_peered":"2024-09-10T18:55:40.403098+0000","last_unstale":"2024-09-10T18:55:50.649662+0000","last_undegraded":"2024-09-10T18:55:50.649662+0000","last_fullsized":"2024-09-10T18:55:50.649662+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'6","last_scrub_stamp":"2024-09-10T18:51:49.096535+0000","last_deep_scrub":"84'6","last_deep_scrub_stamp":"2024-09-10T18:47:22.545450+0000","last_clean_scrub_stamp":"2024-09-10T18:51:49.096535+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037496439999999999,"objects_trimmed":0,"snaptrim_duration":0.0021933159999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":618,"num_read_kb":613,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":208,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.647068+0000","last_change":"2024-09-10T18:55:40.393812+0000","last_active":"2024-09-10T18:55:49.647068+0000","last_peered":"2024-09-10T18:55:49.647068+0000","last_clean":"2024-09-10T18:55:49.647068+0000","last_became_active":"2024-09-10T18:55:40.393389+0000","last_became_peered":"2024-09-10T18:55:40.393389+0000","last_unstale":"2024-09-10T18:55:49.647068+0000","last_undegraded":"2024-09-10T18:55:49.647068+0000","last_fullsized":"2024-09-10T18:55:49.647068+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:44.101575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:44.101575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012333190000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":214,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.650445+0000","last_change":"2024-09-10T18:55:38.900232+0000","last_active":"2024-09-10T18:55:49.650445+0000","last_peered":"2024-09-10T18:55:49.650445+0000","last_clean":"2024-09-10T18:55:49.650445+0000","last_became_active":"2024-09-10T18:47:36.419081+0000","last_became_peered":"2024-09-10T18:47:36.419081+0000","last_unstale":"2024-09-10T18:55:49.650445+0000","last_undegraded":"2024-09-10T18:55:49.650445+0000","last_fullsized":"2024-09-10T18:55:49.650445+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:38.900146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:55:38.900146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:40.649926+0000","scrub_duration":0.0048114799999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":235,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.652224+0000","last_change":"2024-09-10T18:53:07.150581+0000","last_active":"2024-09-10T18:55:49.652224+0000","last_peered":"2024-09-10T18:55:49.652224+0000","last_clean":"2024-09-10T18:55:49.652224+0000","last_became_active":"2024-09-10T18:45:24.264617+0000","last_became_peered":"2024-09-10T18:45:24.264617+0000","last_unstale":"2024-09-10T18:55:49.652224+0000","last_undegraded":"2024-09-10T18:55:49.652224+0000","last_fullsized":"2024-09-10T18:55:49.652224+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:07.150480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:07.150480+0000","last_clean_scrub_stamp":"2024-09-10T18:53:07.150480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.021644442,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"249'1000","reported_seq":1017,"reported_epoch":249,"state":"active+clean","last_fresh":"2024-09-10T18:55:53.348682+0000","last_change":"2024-09-10T18:55:40.391994+0000","last_active":"2024-09-10T18:55:53.348682+0000","last_peered":"2024-09-10T18:55:53.348682+0000","last_clean":"2024-09-10T18:55:53.348682+0000","last_became_active":"2024-09-10T18:55:40.391432+0000","last_became_peered":"2024-09-10T18:55:40.391432+0000","last_unstale":"2024-09-10T18:55:53.348682+0000","last_undegraded":"2024-09-10T18:55:53.348682+0000","last_fullsized":"2024-09-10T18:55:53.348682+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":1000,"log_dups_size":0,"ondisk_log_size":1000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:59.041462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65536000,"num_objects":1000,"num_object_clones":0,"num_object_copies":3000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1000,"num_write_kb":64000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"70'81","reported_seq":157,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.643411+0000","last_change":"2024-09-10T18:53:00.338829+0000","last_active":"2024-09-10T18:55:49.643411+0000","last_peered":"2024-09-10T18:55:49.643411+0000","last_clean":"2024-09-10T18:55:49.643411+0000","last_became_active":"2024-09-10T18:52:14.589009+0000","last_became_peered":"2024-09-10T18:52:14.589009+0000","last_unstale":"2024-09-10T18:55:49.643411+0000","last_undegraded":"2024-09-10T18:55:49.643411+0000","last_fullsized":"2024-09-10T18:55:49.643411+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:53:00.338611+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:53:00.338611+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023458709999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"232'24","reported_seq":273,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:50.646439+0000","last_change":"2024-09-10T18:55:40.419877+0000","last_active":"2024-09-10T18:55:50.646439+0000","last_peered":"2024-09-10T18:55:50.646439+0000","last_clean":"2024-09-10T18:55:50.646439+0000","last_became_active":"2024-09-10T18:55:40.397070+0000","last_became_peered":"2024-09-10T18:55:40.397070+0000","last_unstale":"2024-09-10T18:55:50.646439+0000","last_undegraded":"2024-09-10T18:55:50.646439+0000","last_fullsized":"2024-09-10T18:55:50.646439+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:52:01.328828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:52:01.328828+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"58'1","reported_seq":212,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:50.645195+0000","last_change":"2024-09-10T18:55:40.391005+0000","last_active":"2024-09-10T18:55:50.645195+0000","last_peered":"2024-09-10T18:55:50.645195+0000","last_clean":"2024-09-10T18:55:50.645195+0000","last_became_active":"2024-09-10T18:55:40.390022+0000","last_became_peered":"2024-09-10T18:55:40.390022+0000","last_unstale":"2024-09-10T18:55:50.645195+0000","last_undegraded":"2024-09-10T18:55:50.645195+0000","last_fullsized":"2024-09-10T18:55:50.645195+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'1","last_scrub_stamp":"2024-09-10T18:51:39.175290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:39.175290+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":216,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.650684+0000","last_change":"2024-09-10T18:55:40.397729+0000","last_active":"2024-09-10T18:55:49.650684+0000","last_peered":"2024-09-10T18:55:49.650684+0000","last_clean":"2024-09-10T18:55:49.650684+0000","last_became_active":"2024-09-10T18:55:40.397290+0000","last_became_peered":"2024-09-10T18:55:40.397290+0000","last_unstale":"2024-09-10T18:55:49.650684+0000","last_undegraded":"2024-09-10T18:55:49.650684+0000","last_fullsized":"2024-09-10T18:55:49.650684+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:28.415315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:53:28.415315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001771613,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":93,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.643652+0000","last_change":"2024-09-10T18:55:40.392772+0000","last_active":"2024-09-10T18:55:49.643652+0000","last_peered":"2024-09-10T18:55:49.643652+0000","last_clean":"2024-09-10T18:55:49.643652+0000","last_became_active":"2024-09-10T18:55:40.392494+0000","last_became_peered":"2024-09-10T18:55:40.392494+0000","last_unstale":"2024-09-10T18:55:49.643652+0000","last_undegraded":"2024-09-10T18:55:49.643652+0000","last_fullsized":"2024-09-10T18:55:49.643652+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:41.150673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:52:41.150673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":237,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.643944+0000","last_change":"2024-09-10T18:54:17.615248+0000","last_active":"2024-09-10T18:55:49.643944+0000","last_peered":"2024-09-10T18:55:49.643944+0000","last_clean":"2024-09-10T18:55:49.643944+0000","last_became_active":"2024-09-10T18:45:24.083689+0000","last_became_peered":"2024-09-10T18:45:24.083689+0000","last_unstale":"2024-09-10T18:55:49.643944+0000","last_undegraded":"2024-09-10T18:55:49.643944+0000","last_fullsized":"2024-09-10T18:55:49.643944+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:17.615191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:54:17.615191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011240523000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"248'710","reported_seq":733,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:51.590057+0000","last_change":"2024-09-10T18:55:40.395213+0000","last_active":"2024-09-10T18:55:51.590057+0000","last_peered":"2024-09-10T18:55:51.590057+0000","last_clean":"2024-09-10T18:55:51.590057+0000","last_became_active":"2024-09-10T18:55:40.394445+0000","last_became_peered":"2024-09-10T18:55:40.394445+0000","last_unstale":"2024-09-10T18:55:51.590057+0000","last_undegraded":"2024-09-10T18:55:51.590057+0000","last_fullsized":"2024-09-10T18:55:51.590057+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:47.412527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46530560,"num_objects":710,"num_object_clones":0,"num_object_copies":2130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":710,"num_write_kb":45440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"232'33","reported_seq":295,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.648229+0000","last_change":"2024-09-10T18:54:23.681122+0000","last_active":"2024-09-10T18:55:49.648229+0000","last_peered":"2024-09-10T18:55:49.648229+0000","last_clean":"2024-09-10T18:55:49.648229+0000","last_became_active":"2024-09-10T18:45:20.421119+0000","last_became_peered":"2024-09-10T18:45:20.421119+0000","last_unstale":"2024-09-10T18:55:49.648229+0000","last_undegraded":"2024-09-10T18:55:49.648229+0000","last_fullsized":"2024-09-10T18:55:49.648229+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"126'21","last_scrub_stamp":"2024-09-10T18:54:23.680929+0000","last_deep_scrub":"126'21","last_deep_scrub_stamp":"2024-09-10T18:50:03.380902+0000","last_clean_scrub_stamp":"2024-09-10T18:54:23.680929+0000","objects_scrubbed":7,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0083112350000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"60'2","reported_seq":253,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.650138+0000","last_change":"2024-09-10T18:54:19.313557+0000","last_active":"2024-09-10T18:55:49.650138+0000","last_peered":"2024-09-10T18:55:49.650138+0000","last_clean":"2024-09-10T18:55:49.650138+0000","last_became_active":"2024-09-10T18:45:17.961660+0000","last_became_peered":"2024-09-10T18:45:17.961660+0000","last_unstale":"2024-09-10T18:55:49.650138+0000","last_undegraded":"2024-09-10T18:55:49.650138+0000","last_fullsized":"2024-09-10T18:55:49.650138+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-10T18:54:19.313386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:54:19.313386+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056941359999999998,"objects_trimmed":0,"snaptrim_duration":0.0013942640000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":260,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.649612+0000","last_change":"2024-09-10T18:54:44.030368+0000","last_active":"2024-09-10T18:55:49.649612+0000","last_peered":"2024-09-10T18:55:49.649612+0000","last_clean":"2024-09-10T18:55:49.649612+0000","last_became_active":"2024-09-10T18:47:36.414636+0000","last_became_peered":"2024-09-10T18:47:36.414636+0000","last_unstale":"2024-09-10T18:55:49.649612+0000","last_undegraded":"2024-09-10T18:55:49.649612+0000","last_fullsized":"2024-09-10T18:55:49.649612+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:44.030238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:54:44.030238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0035207699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'81","reported_seq":323,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.651012+0000","last_change":"2024-09-10T18:55:40.397564+0000","last_active":"2024-09-10T18:55:49.651012+0000","last_peered":"2024-09-10T18:55:49.651012+0000","last_clean":"2024-09-10T18:55:49.651012+0000","last_became_active":"2024-09-10T18:55:40.397021+0000","last_became_peered":"2024-09-10T18:55:40.397021+0000","last_unstale":"2024-09-10T18:55:49.651012+0000","last_undegraded":"2024-09-10T18:55:49.651012+0000","last_fullsized":"2024-09-10T18:55:49.651012+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:52:56.085580+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:56.085580+0000","last_clean_scrub_stamp":"2024-09-10T18:52:56.085580+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012963819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":274,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.652989+0000","last_change":"2024-09-10T18:55:40.390348+0000","last_active":"2024-09-10T18:55:49.652989+0000","last_peered":"2024-09-10T18:55:49.652989+0000","last_clean":"2024-09-10T18:55:49.652989+0000","last_became_active":"2024-09-10T18:55:40.390040+0000","last_became_peered":"2024-09-10T18:55:40.390040+0000","last_unstale":"2024-09-10T18:55:49.652989+0000","last_undegraded":"2024-09-10T18:55:49.652989+0000","last_fullsized":"2024-09-10T18:55:49.652989+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:42.415663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:53:42.415663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007606036,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":244,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.651416+0000","last_change":"2024-09-10T18:54:48.995651+0000","last_active":"2024-09-10T18:55:49.651416+0000","last_peered":"2024-09-10T18:55:49.651416+0000","last_clean":"2024-09-10T18:55:49.651416+0000","last_became_active":"2024-09-10T18:45:24.073327+0000","last_became_peered":"2024-09-10T18:45:24.073327+0000","last_unstale":"2024-09-10T18:55:49.651416+0000","last_undegraded":"2024-09-10T18:55:49.651416+0000","last_fullsized":"2024-09-10T18:55:49.651416+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:48.995571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:54:48.995571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:55:58.078462+0000","scrub_duration":0.0029876989999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.7","version":"248'593","reported_seq":612,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:51.380034+0000","last_change":"2024-09-10T18:55:36.356374+0000","last_active":"2024-09-10T18:55:51.380034+0000","last_peered":"2024-09-10T18:55:51.380034+0000","last_clean":"2024-09-10T18:55:51.380034+0000","last_became_active":"2024-09-10T18:55:36.355945+0000","last_became_peered":"2024-09-10T18:55:36.355945+0000","last_unstale":"2024-09-10T18:55:51.380034+0000","last_undegraded":"2024-09-10T18:55:51.380034+0000","last_fullsized":"2024-09-10T18:55:51.380034+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:04.916365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38862848,"num_objects":593,"num_object_clones":0,"num_object_copies":1779,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":593,"num_write_kb":37952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.f","version":"70'81","reported_seq":248,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.648061+0000","last_change":"2024-09-10T18:55:29.018036+0000","last_active":"2024-09-10T18:55:49.648061+0000","last_peered":"2024-09-10T18:55:49.648061+0000","last_clean":"2024-09-10T18:55:49.648061+0000","last_became_active":"2024-09-10T18:52:16.598295+0000","last_became_peered":"2024-09-10T18:52:16.598295+0000","last_unstale":"2024-09-10T18:55:49.648061+0000","last_undegraded":"2024-09-10T18:55:49.648061+0000","last_fullsized":"2024-09-10T18:55:49.648061+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:55:29.017936+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:53.609885+0000","last_clean_scrub_stamp":"2024-09-10T18:55:29.017936+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:34.964703+0000","scrub_duration":0.001646813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"232'35","reported_seq":306,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.648177+0000","last_change":"2024-09-10T18:55:28.047394+0000","last_active":"2024-09-10T18:55:49.648177+0000","last_peered":"2024-09-10T18:55:49.648177+0000","last_clean":"2024-09-10T18:55:49.648177+0000","last_became_active":"2024-09-10T18:45:20.421097+0000","last_became_peered":"2024-09-10T18:45:20.421097+0000","last_unstale":"2024-09-10T18:55:49.648177+0000","last_undegraded":"2024-09-10T18:55:49.648177+0000","last_fullsized":"2024-09-10T18:55:49.648177+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'35","last_scrub_stamp":"2024-09-10T18:55:28.047336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:55:28.047336+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:33.623657+0000","scrub_duration":0.0019079699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":218,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.649769+0000","last_change":"2024-09-10T18:55:40.392220+0000","last_active":"2024-09-10T18:55:49.649769+0000","last_peered":"2024-09-10T18:55:49.649769+0000","last_clean":"2024-09-10T18:55:49.649769+0000","last_became_active":"2024-09-10T18:55:40.391849+0000","last_became_peered":"2024-09-10T18:55:40.391849+0000","last_unstale":"2024-09-10T18:55:49.649769+0000","last_undegraded":"2024-09-10T18:55:49.649769+0000","last_fullsized":"2024-09-10T18:55:49.649769+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:59.222933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:52:59.222933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012835052,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":169,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:50.644600+0000","last_change":"2024-09-10T18:55:40.397918+0000","last_active":"2024-09-10T18:55:50.644600+0000","last_peered":"2024-09-10T18:55:50.644600+0000","last_clean":"2024-09-10T18:55:50.644600+0000","last_became_active":"2024-09-10T18:55:40.397544+0000","last_became_peered":"2024-09-10T18:55:40.397544+0000","last_unstale":"2024-09-10T18:55:50.644600+0000","last_undegraded":"2024-09-10T18:55:50.644600+0000","last_fullsized":"2024-09-10T18:55:50.644600+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:40.126652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:40.126652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017754699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":248,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.651009+0000","last_change":"2024-09-10T18:54:20.686920+0000","last_active":"2024-09-10T18:55:49.651009+0000","last_peered":"2024-09-10T18:55:49.651009+0000","last_clean":"2024-09-10T18:55:49.651009+0000","last_became_active":"2024-09-10T18:47:36.412610+0000","last_became_peered":"2024-09-10T18:47:36.412610+0000","last_unstale":"2024-09-10T18:55:49.651009+0000","last_undegraded":"2024-09-10T18:55:49.651009+0000","last_fullsized":"2024-09-10T18:55:49.651009+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:20.686780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:54:20.686780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051641839999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"248'618","reported_seq":637,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:51.371642+0000","last_change":"2024-09-10T18:55:36.358103+0000","last_active":"2024-09-10T18:55:51.371642+0000","last_peered":"2024-09-10T18:55:51.371642+0000","last_clean":"2024-09-10T18:55:51.371642+0000","last_became_active":"2024-09-10T18:55:36.357828+0000","last_became_peered":"2024-09-10T18:55:36.357828+0000","last_unstale":"2024-09-10T18:55:51.371642+0000","last_undegraded":"2024-09-10T18:55:51.371642+0000","last_fullsized":"2024-09-10T18:55:51.371642+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:38.273211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40501248,"num_objects":618,"num_object_clones":0,"num_object_copies":1854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":618,"num_write_kb":39552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.e","version":"70'81","reported_seq":107,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.641409+0000","last_change":"2024-09-10T18:55:30.496965+0000","last_active":"2024-09-10T18:55:49.641409+0000","last_peered":"2024-09-10T18:55:49.641409+0000","last_clean":"2024-09-10T18:55:49.641409+0000","last_became_active":"2024-09-10T18:52:16.604236+0000","last_became_peered":"2024-09-10T18:52:16.604236+0000","last_unstale":"2024-09-10T18:55:49.641409+0000","last_undegraded":"2024-09-10T18:55:49.641409+0000","last_fullsized":"2024-09-10T18:55:49.641409+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:55:30.496910+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:55:30.496910+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:49.349532+0000","scrub_duration":0.0012110790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"232'19","reported_seq":245,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:50.645191+0000","last_change":"2024-09-10T18:55:40.408918+0000","last_active":"2024-09-10T18:55:50.645191+0000","last_peered":"2024-09-10T18:55:50.645191+0000","last_clean":"2024-09-10T18:55:50.645191+0000","last_became_active":"2024-09-10T18:55:40.395257+0000","last_became_peered":"2024-09-10T18:55:40.395257+0000","last_unstale":"2024-09-10T18:55:50.645191+0000","last_undegraded":"2024-09-10T18:55:50.645191+0000","last_fullsized":"2024-09-10T18:55:50.645191+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"130'13","last_scrub_stamp":"2024-09-10T18:51:58.427534+0000","last_deep_scrub":"130'13","last_deep_scrub_stamp":"2024-09-10T18:51:58.427534+0000","last_clean_scrub_stamp":"2024-09-10T18:51:58.427534+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.021357979999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":240,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.644446+0000","last_change":"2024-09-10T18:54:52.179168+0000","last_active":"2024-09-10T18:55:49.644446+0000","last_peered":"2024-09-10T18:55:49.644446+0000","last_clean":"2024-09-10T18:55:49.644446+0000","last_became_active":"2024-09-10T18:45:17.963057+0000","last_became_peered":"2024-09-10T18:45:17.963057+0000","last_unstale":"2024-09-10T18:55:49.644446+0000","last_undegraded":"2024-09-10T18:55:49.644446+0000","last_fullsized":"2024-09-10T18:55:49.644446+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:52.179074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:54:52.179074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:09.113325+0000","scrub_duration":0.001414738,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":206,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.640083+0000","last_change":"2024-09-10T18:55:40.392580+0000","last_active":"2024-09-10T18:55:49.640083+0000","last_peered":"2024-09-10T18:55:49.640083+0000","last_clean":"2024-09-10T18:55:49.640083+0000","last_became_active":"2024-09-10T18:55:40.390771+0000","last_became_peered":"2024-09-10T18:55:40.390771+0000","last_unstale":"2024-09-10T18:55:49.640083+0000","last_undegraded":"2024-09-10T18:55:49.640083+0000","last_fullsized":"2024-09-10T18:55:49.640083+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:35.325451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_clean_scrub_stamp":"2024-09-10T18:51:35.325451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018247820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":223,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:50.645281+0000","last_change":"2024-09-10T18:55:40.390877+0000","last_active":"2024-09-10T18:55:50.645281+0000","last_peered":"2024-09-10T18:55:50.645281+0000","last_clean":"2024-09-10T18:55:50.645281+0000","last_became_active":"2024-09-10T18:55:40.390451+0000","last_became_peered":"2024-09-10T18:55:40.390451+0000","last_unstale":"2024-09-10T18:55:50.645281+0000","last_undegraded":"2024-09-10T18:55:50.645281+0000","last_fullsized":"2024-09-10T18:55:50.645281+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:53.345184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:53.345184+0000","last_clean_scrub_stamp":"2024-09-10T18:51:53.345184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012977799999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.5","version":"249'978","reported_seq":995,"reported_epoch":249,"state":"active+clean","last_fresh":"2024-09-10T18:55:53.358287+0000","last_change":"2024-09-10T18:55:40.393743+0000","last_active":"2024-09-10T18:55:53.358287+0000","last_peered":"2024-09-10T18:55:53.358287+0000","last_clean":"2024-09-10T18:55:53.358287+0000","last_became_active":"2024-09-10T18:55:40.393456+0000","last_became_peered":"2024-09-10T18:55:40.393456+0000","last_unstale":"2024-09-10T18:55:53.358287+0000","last_undegraded":"2024-09-10T18:55:53.358287+0000","last_fullsized":"2024-09-10T18:55:53.358287+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":978,"log_dups_size":0,"ondisk_log_size":978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:49.754061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64094208,"num_objects":978,"num_object_clones":0,"num_object_copies":2934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":978,"num_write_kb":62592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.d","version":"70'81","reported_seq":98,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.652039+0000","last_change":"2024-09-10T18:53:01.067942+0000","last_active":"2024-09-10T18:55:49.652039+0000","last_peered":"2024-09-10T18:55:49.652039+0000","last_clean":"2024-09-10T18:55:49.652039+0000","last_became_active":"2024-09-10T18:52:16.605492+0000","last_became_peered":"2024-09-10T18:52:16.605492+0000","last_unstale":"2024-09-10T18:55:49.652039+0000","last_undegraded":"2024-09-10T18:55:49.652039+0000","last_fullsized":"2024-09-10T18:55:49.652039+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:53:01.067794+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:53:01.067794+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0033709180000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"232'27","reported_seq":161,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.650862+0000","last_change":"2024-09-10T18:54:22.720833+0000","last_active":"2024-09-10T18:55:49.650862+0000","last_peered":"2024-09-10T18:55:49.650862+0000","last_clean":"2024-09-10T18:55:49.650862+0000","last_became_active":"2024-09-10T18:52:54.579763+0000","last_became_peered":"2024-09-10T18:52:54.579763+0000","last_unstale":"2024-09-10T18:55:49.650862+0000","last_undegraded":"2024-09-10T18:55:49.650862+0000","last_fullsized":"2024-09-10T18:55:49.650862+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:54:22.720737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:54:22.720737+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014176382,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":191,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.649817+0000","last_change":"2024-09-10T18:52:44.851169+0000","last_active":"2024-09-10T18:55:49.649817+0000","last_peered":"2024-09-10T18:55:49.649817+0000","last_clean":"2024-09-10T18:55:49.649817+0000","last_became_active":"2024-09-10T18:47:36.420475+0000","last_became_peered":"2024-09-10T18:47:36.420475+0000","last_unstale":"2024-09-10T18:55:49.649817+0000","last_undegraded":"2024-09-10T18:55:49.649817+0000","last_fullsized":"2024-09-10T18:55:49.649817+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:44.851096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:05.603125+0000","last_clean_scrub_stamp":"2024-09-10T18:52:44.851096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0054082180000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"83'9","reported_seq":389,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:50.650652+0000","last_change":"2024-09-10T18:55:37.750865+0000","last_active":"2024-09-10T18:55:50.650652+0000","last_peered":"2024-09-10T18:55:50.650652+0000","last_clean":"2024-09-10T18:55:50.650652+0000","last_became_active":"2024-09-10T18:45:22.073383+0000","last_became_peered":"2024-09-10T18:45:22.073383+0000","last_unstale":"2024-09-10T18:55:50.650652+0000","last_undegraded":"2024-09-10T18:55:50.650652+0000","last_fullsized":"2024-09-10T18:55:50.650652+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'9","last_scrub_stamp":"2024-09-10T18:55:37.750784+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-09-10T18:47:02.582229+0000","last_clean_scrub_stamp":"2024-09-10T18:55:37.750784+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:06.868622+0000","scrub_duration":0.0029702869999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":251,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.649948+0000","last_change":"2024-09-10T18:55:40.391764+0000","last_active":"2024-09-10T18:55:49.649948+0000","last_peered":"2024-09-10T18:55:49.649948+0000","last_clean":"2024-09-10T18:55:49.649948+0000","last_became_active":"2024-09-10T18:55:40.389202+0000","last_became_peered":"2024-09-10T18:55:40.389202+0000","last_unstale":"2024-09-10T18:55:49.649948+0000","last_undegraded":"2024-09-10T18:55:49.649948+0000","last_fullsized":"2024-09-10T18:55:49.649948+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:48.093216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_clean_scrub_stamp":"2024-09-10T18:52:48.093216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0065930010000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.4","version":"248'580","reported_seq":599,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:51.433767+0000","last_change":"2024-09-10T18:55:36.350444+0000","last_active":"2024-09-10T18:55:51.433767+0000","last_peered":"2024-09-10T18:55:51.433767+0000","last_clean":"2024-09-10T18:55:51.433767+0000","last_became_active":"2024-09-10T18:55:36.350155+0000","last_became_peered":"2024-09-10T18:55:36.350155+0000","last_unstale":"2024-09-10T18:55:51.433767+0000","last_undegraded":"2024-09-10T18:55:51.433767+0000","last_fullsized":"2024-09-10T18:55:51.433767+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:41.679724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38010880,"num_objects":580,"num_object_clones":0,"num_object_copies":1740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":580,"num_write_kb":37120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.c","version":"70'81","reported_seq":194,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.650113+0000","last_change":"2024-09-10T18:55:40.391027+0000","last_active":"2024-09-10T18:55:49.650113+0000","last_peered":"2024-09-10T18:55:49.650113+0000","last_clean":"2024-09-10T18:55:49.650113+0000","last_became_active":"2024-09-10T18:55:40.390655+0000","last_became_peered":"2024-09-10T18:55:40.390655+0000","last_unstale":"2024-09-10T18:55:49.650113+0000","last_undegraded":"2024-09-10T18:55:49.650113+0000","last_fullsized":"2024-09-10T18:55:49.650113+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:53:05.814851+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:53:05.814851+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025032510000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"232'18","reported_seq":278,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.639562+0000","last_change":"2024-09-10T18:55:27.507437+0000","last_active":"2024-09-10T18:55:49.639562+0000","last_peered":"2024-09-10T18:55:49.639562+0000","last_clean":"2024-09-10T18:55:49.639562+0000","last_became_active":"2024-09-10T18:45:20.056869+0000","last_became_peered":"2024-09-10T18:45:20.056869+0000","last_unstale":"2024-09-10T18:55:49.639562+0000","last_undegraded":"2024-09-10T18:55:49.639562+0000","last_fullsized":"2024-09-10T18:55:49.639562+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'18","last_scrub_stamp":"2024-09-10T18:55:27.507364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:55:27.507364+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:56.211266+0000","scrub_duration":0.0030221390000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"56'2","reported_seq":262,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.641689+0000","last_change":"2024-09-10T18:55:40.392502+0000","last_active":"2024-09-10T18:55:49.641689+0000","last_peered":"2024-09-10T18:55:49.641689+0000","last_clean":"2024-09-10T18:55:49.641689+0000","last_became_active":"2024-09-10T18:55:40.392169+0000","last_became_peered":"2024-09-10T18:55:40.392169+0000","last_unstale":"2024-09-10T18:55:49.641689+0000","last_undegraded":"2024-09-10T18:55:49.641689+0000","last_fullsized":"2024-09-10T18:55:49.641689+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-10T18:51:42.424994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:42.424994+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001852226,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":179,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:50.642823+0000","last_change":"2024-09-10T18:55:40.398222+0000","last_active":"2024-09-10T18:55:50.642823+0000","last_peered":"2024-09-10T18:55:50.642823+0000","last_clean":"2024-09-10T18:55:50.642823+0000","last_became_active":"2024-09-10T18:55:40.397290+0000","last_became_peered":"2024-09-10T18:55:40.397290+0000","last_unstale":"2024-09-10T18:55:50.642823+0000","last_undegraded":"2024-09-10T18:55:50.642823+0000","last_fullsized":"2024-09-10T18:55:50.642823+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:54.342934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:02.711594+0000","last_clean_scrub_stamp":"2024-09-10T18:51:54.342934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005738529,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":234,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.648793+0000","last_change":"2024-09-10T18:54:53.352477+0000","last_active":"2024-09-10T18:55:49.648793+0000","last_peered":"2024-09-10T18:55:49.648793+0000","last_clean":"2024-09-10T18:55:49.648793+0000","last_became_active":"2024-09-10T18:45:24.077815+0000","last_became_peered":"2024-09-10T18:45:24.077815+0000","last_unstale":"2024-09-10T18:55:49.648793+0000","last_undegraded":"2024-09-10T18:55:49.648793+0000","last_fullsized":"2024-09-10T18:55:49.648793+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:53.352427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:53.352427+0000","last_clean_scrub_stamp":"2024-09-10T18:54:53.352427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:16.720823+0000","scrub_duration":0.0022835899999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.13","version":"70'81","reported_seq":166,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.643672+0000","last_change":"2024-09-10T18:53:06.263065+0000","last_active":"2024-09-10T18:55:49.643672+0000","last_peered":"2024-09-10T18:55:49.643672+0000","last_clean":"2024-09-10T18:55:49.643672+0000","last_became_active":"2024-09-10T18:52:16.602463+0000","last_became_peered":"2024-09-10T18:52:16.602463+0000","last_unstale":"2024-09-10T18:55:49.643672+0000","last_undegraded":"2024-09-10T18:55:49.643672+0000","last_fullsized":"2024-09-10T18:55:49.643672+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:53:06.262959+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:53:06.262959+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0036690479999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"232'19","reported_seq":248,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:50.646314+0000","last_change":"2024-09-10T18:55:40.414373+0000","last_active":"2024-09-10T18:55:50.646314+0000","last_peered":"2024-09-10T18:55:50.646314+0000","last_clean":"2024-09-10T18:55:50.646314+0000","last_became_active":"2024-09-10T18:55:40.393944+0000","last_became_peered":"2024-09-10T18:55:40.393944+0000","last_unstale":"2024-09-10T18:55:50.646314+0000","last_undegraded":"2024-09-10T18:55:50.646314+0000","last_fullsized":"2024-09-10T18:55:50.646314+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'13","last_scrub_stamp":"2024-09-10T18:51:56.389281+0000","last_deep_scrub":"76'13","last_deep_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_clean_scrub_stamp":"2024-09-10T18:51:56.389281+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":218,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.642220+0000","last_change":"2024-09-10T18:55:40.402772+0000","last_active":"2024-09-10T18:55:49.642220+0000","last_peered":"2024-09-10T18:55:49.642220+0000","last_clean":"2024-09-10T18:55:49.642220+0000","last_became_active":"2024-09-10T18:55:40.401037+0000","last_became_peered":"2024-09-10T18:55:40.401037+0000","last_unstale":"2024-09-10T18:55:49.642220+0000","last_undegraded":"2024-09-10T18:55:49.642220+0000","last_fullsized":"2024-09-10T18:55:49.642220+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:05.160144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:52:05.160144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041652920000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":236,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.649237+0000","last_change":"2024-09-10T18:53:00.755016+0000","last_active":"2024-09-10T18:55:49.649237+0000","last_peered":"2024-09-10T18:55:49.649237+0000","last_clean":"2024-09-10T18:55:49.649237+0000","last_became_active":"2024-09-10T18:45:22.069040+0000","last_became_peered":"2024-09-10T18:45:22.069040+0000","last_unstale":"2024-09-10T18:55:49.649237+0000","last_undegraded":"2024-09-10T18:55:49.649237+0000","last_fullsized":"2024-09-10T18:55:49.649237+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:00.754912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:00.754912+0000","last_clean_scrub_stamp":"2024-09-10T18:53:00.754912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0075269270000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"63'1","reported_seq":222,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.640734+0000","last_change":"2024-09-10T18:55:40.390346+0000","last_active":"2024-09-10T18:55:49.640734+0000","last_peered":"2024-09-10T18:55:49.640734+0000","last_clean":"2024-09-10T18:55:49.640734+0000","last_became_active":"2024-09-10T18:55:40.389882+0000","last_became_peered":"2024-09-10T18:55:40.389882+0000","last_unstale":"2024-09-10T18:55:49.640734+0000","last_undegraded":"2024-09-10T18:55:49.640734+0000","last_fullsized":"2024-09-10T18:55:49.640734+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-10T18:51:57.216086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:51:57.216086+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048352930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.12","version":"70'81","reported_seq":108,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.650125+0000","last_change":"2024-09-10T18:54:50.968303+0000","last_active":"2024-09-10T18:55:49.650125+0000","last_peered":"2024-09-10T18:55:49.650125+0000","last_clean":"2024-09-10T18:55:49.650125+0000","last_became_active":"2024-09-10T18:52:16.598921+0000","last_became_peered":"2024-09-10T18:52:16.598921+0000","last_unstale":"2024-09-10T18:55:49.650125+0000","last_undegraded":"2024-09-10T18:55:49.650125+0000","last_fullsized":"2024-09-10T18:55:49.650125+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:54:50.968202+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:54.127476+0000","last_clean_scrub_stamp":"2024-09-10T18:54:50.968202+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:07.866784+0000","scrub_duration":0.0074710000000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":253,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.649408+0000","last_change":"2024-09-10T18:55:35.136490+0000","last_active":"2024-09-10T18:55:49.649408+0000","last_peered":"2024-09-10T18:55:49.649408+0000","last_clean":"2024-09-10T18:55:49.649408+0000","last_became_active":"2024-09-10T18:45:17.964705+0000","last_became_peered":"2024-09-10T18:45:17.964705+0000","last_unstale":"2024-09-10T18:55:49.649408+0000","last_undegraded":"2024-09-10T18:55:49.649408+0000","last_fullsized":"2024-09-10T18:55:49.649408+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.136432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.136432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:38.251881+0000","scrub_duration":0.0015178839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"232'12","reported_seq":259,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.649463+0000","last_change":"2024-09-10T18:55:30.612864+0000","last_active":"2024-09-10T18:55:49.649463+0000","last_peered":"2024-09-10T18:55:49.649463+0000","last_clean":"2024-09-10T18:55:49.649463+0000","last_became_active":"2024-09-10T18:45:20.421746+0000","last_became_peered":"2024-09-10T18:45:20.421746+0000","last_unstale":"2024-09-10T18:55:49.649463+0000","last_undegraded":"2024-09-10T18:55:49.649463+0000","last_fullsized":"2024-09-10T18:55:49.649463+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'12","last_scrub_stamp":"2024-09-10T18:55:30.612808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:55:30.612808+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:55.373675+0000","scrub_duration":0.001922787,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":236,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.650539+0000","last_change":"2024-09-10T18:53:03.116972+0000","last_active":"2024-09-10T18:55:49.650539+0000","last_peered":"2024-09-10T18:55:49.650539+0000","last_clean":"2024-09-10T18:55:49.650539+0000","last_became_active":"2024-09-10T18:45:22.069653+0000","last_became_peered":"2024-09-10T18:45:22.069653+0000","last_unstale":"2024-09-10T18:55:49.650539+0000","last_undegraded":"2024-09-10T18:55:49.650539+0000","last_fullsized":"2024-09-10T18:55:49.650539+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:03.116899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:53:03.116899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010530331,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":245,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.652152+0000","last_change":"2024-09-10T18:54:47.741171+0000","last_active":"2024-09-10T18:55:49.652152+0000","last_peered":"2024-09-10T18:55:49.652152+0000","last_clean":"2024-09-10T18:55:49.652152+0000","last_became_active":"2024-09-10T18:45:24.264611+0000","last_became_peered":"2024-09-10T18:45:24.264611+0000","last_unstale":"2024-09-10T18:55:49.652152+0000","last_undegraded":"2024-09-10T18:55:49.652152+0000","last_fullsized":"2024-09-10T18:55:49.652152+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:47.741071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:54:47.741071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:55:54.401153+0000","scrub_duration":0.0053190620000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.11","version":"70'81","reported_seq":133,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:50.644507+0000","last_change":"2024-09-10T18:55:40.395720+0000","last_active":"2024-09-10T18:55:50.644507+0000","last_peered":"2024-09-10T18:55:50.644507+0000","last_clean":"2024-09-10T18:55:50.644507+0000","last_became_active":"2024-09-10T18:55:40.395019+0000","last_became_peered":"2024-09-10T18:55:40.395019+0000","last_unstale":"2024-09-10T18:55:50.644507+0000","last_undegraded":"2024-09-10T18:55:50.644507+0000","last_fullsized":"2024-09-10T18:55:50.644507+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:52:53.556903+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:52:53.556903+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004894626,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"232'20","reported_seq":312,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.649274+0000","last_change":"2024-09-10T18:55:33.667767+0000","last_active":"2024-09-10T18:55:49.649274+0000","last_peered":"2024-09-10T18:55:49.649274+0000","last_clean":"2024-09-10T18:55:49.649274+0000","last_became_active":"2024-09-10T18:47:36.426260+0000","last_became_peered":"2024-09-10T18:47:36.426260+0000","last_unstale":"2024-09-10T18:55:49.649274+0000","last_undegraded":"2024-09-10T18:55:49.649274+0000","last_fullsized":"2024-09-10T18:55:49.649274+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'20","last_scrub_stamp":"2024-09-10T18:55:33.667712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:55:33.667712+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:59.298753+0000","scrub_duration":0.0028302039999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":250,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.650269+0000","last_change":"2024-09-10T18:54:51.965707+0000","last_active":"2024-09-10T18:55:49.650269+0000","last_peered":"2024-09-10T18:55:49.650269+0000","last_clean":"2024-09-10T18:55:49.650269+0000","last_became_active":"2024-09-10T18:45:17.964870+0000","last_became_peered":"2024-09-10T18:45:17.964870+0000","last_unstale":"2024-09-10T18:55:49.650269+0000","last_undegraded":"2024-09-10T18:55:49.650269+0000","last_fullsized":"2024-09-10T18:55:49.650269+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:51.965637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:54:51.965637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:18.723317+0000","scrub_duration":0.011107304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"242'10","reported_seq":390,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:50.649541+0000","last_change":"2024-09-10T18:55:40.392039+0000","last_active":"2024-09-10T18:55:50.649541+0000","last_peered":"2024-09-10T18:55:50.649541+0000","last_clean":"2024-09-10T18:55:50.649541+0000","last_became_active":"2024-09-10T18:55:40.391439+0000","last_became_peered":"2024-09-10T18:55:40.391439+0000","last_unstale":"2024-09-10T18:55:50.649541+0000","last_undegraded":"2024-09-10T18:55:50.649541+0000","last_fullsized":"2024-09-10T18:55:50.649541+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:52:42.136982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:52:42.136982+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0098948820000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":192,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:50.645477+0000","last_change":"2024-09-10T18:55:40.394192+0000","last_active":"2024-09-10T18:55:50.645477+0000","last_peered":"2024-09-10T18:55:50.645477+0000","last_clean":"2024-09-10T18:55:50.645477+0000","last_became_active":"2024-09-10T18:55:40.390667+0000","last_became_peered":"2024-09-10T18:55:40.390667+0000","last_unstale":"2024-09-10T18:55:50.645477+0000","last_undegraded":"2024-09-10T18:55:50.645477+0000","last_fullsized":"2024-09-10T18:55:50.645477+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:57.476724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:57.476724+0000","last_clean_scrub_stamp":"2024-09-10T18:52:57.476724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.10","version":"70'81","reported_seq":98,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.647218+0000","last_change":"2024-09-10T18:52:57.175458+0000","last_active":"2024-09-10T18:55:49.647218+0000","last_peered":"2024-09-10T18:55:49.647218+0000","last_clean":"2024-09-10T18:55:49.647218+0000","last_became_active":"2024-09-10T18:52:16.601616+0000","last_became_peered":"2024-09-10T18:52:16.601616+0000","last_unstale":"2024-09-10T18:55:49.647218+0000","last_undegraded":"2024-09-10T18:55:49.647218+0000","last_fullsized":"2024-09-10T18:55:49.647218+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:52:57.175370+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:52:57.175370+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012721060000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"232'15","reported_seq":140,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.653302+0000","last_change":"2024-09-10T18:55:25.784395+0000","last_active":"2024-09-10T18:55:49.653302+0000","last_peered":"2024-09-10T18:55:49.653302+0000","last_clean":"2024-09-10T18:55:49.653302+0000","last_became_active":"2024-09-10T18:52:54.618780+0000","last_became_peered":"2024-09-10T18:52:54.618780+0000","last_unstale":"2024-09-10T18:55:49.653302+0000","last_undegraded":"2024-09-10T18:55:49.653302+0000","last_fullsized":"2024-09-10T18:55:49.653302+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T18:55:25.784333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:55:25.784333+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:42.282358+0000","scrub_duration":0.0015586210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":217,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:50.644736+0000","last_change":"2024-09-10T18:55:40.391735+0000","last_active":"2024-09-10T18:55:50.644736+0000","last_peered":"2024-09-10T18:55:50.644736+0000","last_clean":"2024-09-10T18:55:50.644736+0000","last_became_active":"2024-09-10T18:55:40.390697+0000","last_became_peered":"2024-09-10T18:55:40.390697+0000","last_unstale":"2024-09-10T18:55:50.644736+0000","last_undegraded":"2024-09-10T18:55:50.644736+0000","last_fullsized":"2024-09-10T18:55:50.644736+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:46.194176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:46.194176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"89'10","reported_seq":386,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:50.654368+0000","last_change":"2024-09-10T18:55:36.732976+0000","last_active":"2024-09-10T18:55:50.654368+0000","last_peered":"2024-09-10T18:55:50.654368+0000","last_clean":"2024-09-10T18:55:50.654368+0000","last_became_active":"2024-09-10T18:47:36.424575+0000","last_became_peered":"2024-09-10T18:47:36.424575+0000","last_unstale":"2024-09-10T18:55:50.654368+0000","last_undegraded":"2024-09-10T18:55:50.654368+0000","last_fullsized":"2024-09-10T18:55:50.654368+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'10","last_scrub_stamp":"2024-09-10T18:55:36.732908+0000","last_deep_scrub":"81'9","last_deep_scrub_stamp":"2024-09-10T18:46:54.800790+0000","last_clean_scrub_stamp":"2024-09-10T18:55:36.732908+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:39.212655+0000","scrub_duration":0.0016257330000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":259,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.641431+0000","last_change":"2024-09-10T18:55:40.392353+0000","last_active":"2024-09-10T18:55:49.641431+0000","last_peered":"2024-09-10T18:55:49.641431+0000","last_clean":"2024-09-10T18:55:49.641431+0000","last_became_active":"2024-09-10T18:55:40.391662+0000","last_became_peered":"2024-09-10T18:55:40.391662+0000","last_unstale":"2024-09-10T18:55:49.641431+0000","last_undegraded":"2024-09-10T18:55:49.641431+0000","last_fullsized":"2024-09-10T18:55:49.641431+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:42.196733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_clean_scrub_stamp":"2024-09-10T18:52:42.196733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039644839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"232'15","reported_seq":302,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.645005+0000","last_change":"2024-09-10T18:55:40.411162+0000","last_active":"2024-09-10T18:55:49.645005+0000","last_peered":"2024-09-10T18:55:49.645005+0000","last_clean":"2024-09-10T18:55:49.645005+0000","last_became_active":"2024-09-10T18:55:40.400788+0000","last_became_peered":"2024-09-10T18:55:40.400788+0000","last_unstale":"2024-09-10T18:55:49.645005+0000","last_undegraded":"2024-09-10T18:55:49.645005+0000","last_fullsized":"2024-09-10T18:55:49.645005+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:51:32.146622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:32.146622+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023792729999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":260,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.647964+0000","last_change":"2024-09-10T18:53:03.697342+0000","last_active":"2024-09-10T18:55:49.647964+0000","last_peered":"2024-09-10T18:55:49.647964+0000","last_clean":"2024-09-10T18:55:49.647964+0000","last_became_active":"2024-09-10T18:52:54.620353+0000","last_became_peered":"2024-09-10T18:52:54.620353+0000","last_unstale":"2024-09-10T18:55:49.647964+0000","last_undegraded":"2024-09-10T18:55:49.647964+0000","last_fullsized":"2024-09-10T18:55:49.647964+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:03.697178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:53:03.697178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010029720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":243,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.648827+0000","last_change":"2024-09-10T18:55:40.398505+0000","last_active":"2024-09-10T18:55:49.648827+0000","last_peered":"2024-09-10T18:55:49.648827+0000","last_clean":"2024-09-10T18:55:49.648827+0000","last_became_active":"2024-09-10T18:55:40.398243+0000","last_became_peered":"2024-09-10T18:55:40.398243+0000","last_unstale":"2024-09-10T18:55:49.648827+0000","last_undegraded":"2024-09-10T18:55:49.648827+0000","last_fullsized":"2024-09-10T18:55:49.648827+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:47.078574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:03.571786+0000","last_clean_scrub_stamp":"2024-09-10T18:51:47.078574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001570454,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":212,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:50.642676+0000","last_change":"2024-09-10T18:55:40.397599+0000","last_active":"2024-09-10T18:55:50.642676+0000","last_peered":"2024-09-10T18:55:50.642676+0000","last_clean":"2024-09-10T18:55:50.642676+0000","last_became_active":"2024-09-10T18:55:40.396932+0000","last_became_peered":"2024-09-10T18:55:40.396932+0000","last_unstale":"2024-09-10T18:55:50.642676+0000","last_undegraded":"2024-09-10T18:55:50.642676+0000","last_fullsized":"2024-09-10T18:55:50.642676+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:47.209023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:04.631558+0000","last_clean_scrub_stamp":"2024-09-10T18:51:47.209023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019148069999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"232'24","reported_seq":272,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.640185+0000","last_change":"2024-09-10T18:54:47.324126+0000","last_active":"2024-09-10T18:55:49.640185+0000","last_peered":"2024-09-10T18:55:49.640185+0000","last_clean":"2024-09-10T18:55:49.640185+0000","last_became_active":"2024-09-10T18:45:20.056885+0000","last_became_peered":"2024-09-10T18:45:20.056885+0000","last_unstale":"2024-09-10T18:55:49.640185+0000","last_undegraded":"2024-09-10T18:55:49.640185+0000","last_fullsized":"2024-09-10T18:55:49.640185+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'16","last_scrub_stamp":"2024-09-10T18:54:47.324044+0000","last_deep_scrub":"223'16","last_deep_scrub_stamp":"2024-09-10T18:54:47.324044+0000","last_clean_scrub_stamp":"2024-09-10T18:54:47.324044+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:03.104510+0000","scrub_duration":0.016282413999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":257,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.644299+0000","last_change":"2024-09-10T18:55:40.399504+0000","last_active":"2024-09-10T18:55:49.644299+0000","last_peered":"2024-09-10T18:55:49.644299+0000","last_clean":"2024-09-10T18:55:49.644299+0000","last_became_active":"2024-09-10T18:55:40.399198+0000","last_became_peered":"2024-09-10T18:55:40.399198+0000","last_unstale":"2024-09-10T18:55:49.644299+0000","last_undegraded":"2024-09-10T18:55:49.644299+0000","last_fullsized":"2024-09-10T18:55:49.644299+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:32.290320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:00.029389+0000","last_clean_scrub_stamp":"2024-09-10T18:53:32.290320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003302677,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":239,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.644850+0000","last_change":"2024-09-10T18:52:43.135942+0000","last_active":"2024-09-10T18:55:49.644850+0000","last_peered":"2024-09-10T18:55:49.644850+0000","last_clean":"2024-09-10T18:55:49.644850+0000","last_became_active":"2024-09-10T18:45:22.070065+0000","last_became_peered":"2024-09-10T18:45:22.070065+0000","last_unstale":"2024-09-10T18:55:49.644850+0000","last_undegraded":"2024-09-10T18:55:49.644850+0000","last_fullsized":"2024-09-10T18:55:49.644850+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:43.135170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:52:43.135170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011350617,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":241,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.647740+0000","last_change":"2024-09-10T18:55:40.396046+0000","last_active":"2024-09-10T18:55:49.647740+0000","last_peered":"2024-09-10T18:55:49.647740+0000","last_clean":"2024-09-10T18:55:49.647740+0000","last_became_active":"2024-09-10T18:55:40.395595+0000","last_became_peered":"2024-09-10T18:55:40.395595+0000","last_unstale":"2024-09-10T18:55:49.647740+0000","last_undegraded":"2024-09-10T18:55:49.647740+0000","last_fullsized":"2024-09-10T18:55:49.647740+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:05.098031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:52:05.098031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0067119739999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"232'8","reported_seq":250,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.647699+0000","last_change":"2024-09-10T18:52:56.832280+0000","last_active":"2024-09-10T18:55:49.647699+0000","last_peered":"2024-09-10T18:55:49.647699+0000","last_clean":"2024-09-10T18:55:49.647699+0000","last_became_active":"2024-09-10T18:45:20.059205+0000","last_became_peered":"2024-09-10T18:45:20.059205+0000","last_unstale":"2024-09-10T18:55:49.647699+0000","last_undegraded":"2024-09-10T18:55:49.647699+0000","last_fullsized":"2024-09-10T18:55:49.647699+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4","last_scrub_stamp":"2024-09-10T18:52:56.832176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:52:56.832176+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015287545,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":256,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.650587+0000","last_change":"2024-09-10T18:55:25.897258+0000","last_active":"2024-09-10T18:55:49.650587+0000","last_peered":"2024-09-10T18:55:49.650587+0000","last_clean":"2024-09-10T18:55:49.650587+0000","last_became_active":"2024-09-10T18:47:36.415866+0000","last_became_peered":"2024-09-10T18:47:36.415866+0000","last_unstale":"2024-09-10T18:55:49.650587+0000","last_undegraded":"2024-09-10T18:55:49.650587+0000","last_fullsized":"2024-09-10T18:55:49.650587+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:25.897205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:55:25.897205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:29.254051+0000","scrub_duration":0.001389787,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":256,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.649132+0000","last_change":"2024-09-10T18:54:27.808243+0000","last_active":"2024-09-10T18:55:49.649132+0000","last_peered":"2024-09-10T18:55:49.649132+0000","last_clean":"2024-09-10T18:55:49.649132+0000","last_became_active":"2024-09-10T18:52:58.103465+0000","last_became_peered":"2024-09-10T18:52:58.103465+0000","last_unstale":"2024-09-10T18:55:49.649132+0000","last_undegraded":"2024-09-10T18:55:49.649132+0000","last_fullsized":"2024-09-10T18:55:49.649132+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:27.808169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_clean_scrub_stamp":"2024-09-10T18:54:27.808169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:55:56.181665+0000","scrub_duration":0.0098845559999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":258,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.652399+0000","last_change":"2024-09-10T18:55:28.768157+0000","last_active":"2024-09-10T18:55:49.652399+0000","last_peered":"2024-09-10T18:55:49.652399+0000","last_clean":"2024-09-10T18:55:49.652399+0000","last_became_active":"2024-09-10T18:47:36.416187+0000","last_became_peered":"2024-09-10T18:47:36.416187+0000","last_unstale":"2024-09-10T18:55:49.652399+0000","last_undegraded":"2024-09-10T18:55:49.652399+0000","last_fullsized":"2024-09-10T18:55:49.652399+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:28.768091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:55.145638+0000","last_clean_scrub_stamp":"2024-09-10T18:55:28.768091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:54.162730+0000","scrub_duration":0.0027672999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.14","version":"70'81","reported_seq":268,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.651615+0000","last_change":"2024-09-10T18:55:34.791281+0000","last_active":"2024-09-10T18:55:49.651615+0000","last_peered":"2024-09-10T18:55:49.651615+0000","last_clean":"2024-09-10T18:55:49.651615+0000","last_became_active":"2024-09-10T18:52:16.600485+0000","last_became_peered":"2024-09-10T18:52:16.600485+0000","last_unstale":"2024-09-10T18:55:49.651615+0000","last_undegraded":"2024-09-10T18:55:49.651615+0000","last_fullsized":"2024-09-10T18:55:49.651615+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:55:34.791223+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:57.184383+0000","last_clean_scrub_stamp":"2024-09-10T18:55:34.791223+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:41.490465+0000","scrub_duration":0.0013477910000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"232'24","reported_seq":274,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.647410+0000","last_change":"2024-09-10T18:54:39.420770+0000","last_active":"2024-09-10T18:55:49.647410+0000","last_peered":"2024-09-10T18:55:49.647410+0000","last_clean":"2024-09-10T18:55:49.647410+0000","last_became_active":"2024-09-10T18:45:20.421479+0000","last_became_peered":"2024-09-10T18:45:20.421479+0000","last_unstale":"2024-09-10T18:55:49.647410+0000","last_undegraded":"2024-09-10T18:55:49.647410+0000","last_fullsized":"2024-09-10T18:55:49.647410+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"79'16","last_scrub_stamp":"2024-09-10T18:54:39.420695+0000","last_deep_scrub":"79'16","last_deep_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_clean_scrub_stamp":"2024-09-10T18:54:39.420695+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.052085645,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"56'1","reported_seq":271,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.648300+0000","last_change":"2024-09-10T18:54:52.114524+0000","last_active":"2024-09-10T18:55:49.648300+0000","last_peered":"2024-09-10T18:55:49.648300+0000","last_clean":"2024-09-10T18:55:49.648300+0000","last_became_active":"2024-09-10T18:47:36.421985+0000","last_became_peered":"2024-09-10T18:47:36.421985+0000","last_unstale":"2024-09-10T18:55:49.648300+0000","last_undegraded":"2024-09-10T18:55:49.648300+0000","last_fullsized":"2024-09-10T18:55:49.648300+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:54:52.114398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:54:52.114398+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:04.100907+0000","scrub_duration":0.0055628179999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":190,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.647209+0000","last_change":"2024-09-10T18:52:51.817600+0000","last_active":"2024-09-10T18:55:49.647209+0000","last_peered":"2024-09-10T18:55:49.647209+0000","last_clean":"2024-09-10T18:55:49.647209+0000","last_became_active":"2024-09-10T18:47:57.107149+0000","last_became_peered":"2024-09-10T18:47:57.107149+0000","last_unstale":"2024-09-10T18:55:49.647209+0000","last_undegraded":"2024-09-10T18:55:49.647209+0000","last_fullsized":"2024-09-10T18:55:49.647209+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:51.817521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:52:51.817521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001468528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":251,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.648748+0000","last_change":"2024-09-10T18:55:40.397793+0000","last_active":"2024-09-10T18:55:49.648748+0000","last_peered":"2024-09-10T18:55:49.648748+0000","last_clean":"2024-09-10T18:55:49.648748+0000","last_became_active":"2024-09-10T18:55:40.397523+0000","last_became_peered":"2024-09-10T18:55:40.397523+0000","last_unstale":"2024-09-10T18:55:49.648748+0000","last_undegraded":"2024-09-10T18:55:49.648748+0000","last_fullsized":"2024-09-10T18:55:49.648748+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:48.901462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:52:48.901462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039921779999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"78'2","reported_seq":264,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.651936+0000","last_change":"2024-09-10T18:54:27.784969+0000","last_active":"2024-09-10T18:55:49.651936+0000","last_peered":"2024-09-10T18:55:49.651936+0000","last_clean":"2024-09-10T18:55:49.651936+0000","last_became_active":"2024-09-10T18:47:36.418702+0000","last_became_peered":"2024-09-10T18:47:36.418702+0000","last_unstale":"2024-09-10T18:55:49.651936+0000","last_undegraded":"2024-09-10T18:55:49.651936+0000","last_fullsized":"2024-09-10T18:55:49.651936+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'2","last_scrub_stamp":"2024-09-10T18:54:27.784508+0000","last_deep_scrub":"78'2","last_deep_scrub_stamp":"2024-09-10T18:46:37.581130+0000","last_clean_scrub_stamp":"2024-09-10T18:54:27.784508+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:55:55.440546+0000","scrub_duration":0.011474105,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":240,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.647345+0000","last_change":"2024-09-10T18:52:52.152441+0000","last_active":"2024-09-10T18:55:49.647345+0000","last_peered":"2024-09-10T18:55:49.647345+0000","last_clean":"2024-09-10T18:55:49.647345+0000","last_became_active":"2024-09-10T18:45:17.962108+0000","last_became_peered":"2024-09-10T18:45:17.962108+0000","last_unstale":"2024-09-10T18:55:49.647345+0000","last_undegraded":"2024-09-10T18:55:49.647345+0000","last_fullsized":"2024-09-10T18:55:49.647345+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:52.152309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:52:52.152309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044316169999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"232'20","reported_seq":244,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:50.646178+0000","last_change":"2024-09-10T18:55:40.413383+0000","last_active":"2024-09-10T18:55:50.646178+0000","last_peered":"2024-09-10T18:55:50.646178+0000","last_clean":"2024-09-10T18:55:50.646178+0000","last_became_active":"2024-09-10T18:55:40.393341+0000","last_became_peered":"2024-09-10T18:55:40.393341+0000","last_unstale":"2024-09-10T18:55:50.646178+0000","last_undegraded":"2024-09-10T18:55:50.646178+0000","last_fullsized":"2024-09-10T18:55:50.646178+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:52:00.358298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:52:00.358298+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":23,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"190'9","reported_seq":385,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:50.652668+0000","last_change":"2024-09-10T18:53:07.746395+0000","last_active":"2024-09-10T18:55:50.652668+0000","last_peered":"2024-09-10T18:55:50.652668+0000","last_clean":"2024-09-10T18:55:50.652668+0000","last_became_active":"2024-09-10T18:45:22.073281+0000","last_became_peered":"2024-09-10T18:45:22.073281+0000","last_unstale":"2024-09-10T18:55:50.652668+0000","last_undegraded":"2024-09-10T18:55:50.652668+0000","last_fullsized":"2024-09-10T18:55:50.652668+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"190'9","last_scrub_stamp":"2024-09-10T18:53:07.746309+0000","last_deep_scrub":"190'9","last_deep_scrub_stamp":"2024-09-10T18:53:07.746309+0000","last_clean_scrub_stamp":"2024-09-10T18:53:07.746309+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0082424100000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":237,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.643656+0000","last_change":"2024-09-10T18:54:24.631566+0000","last_active":"2024-09-10T18:55:49.643656+0000","last_peered":"2024-09-10T18:55:49.643656+0000","last_clean":"2024-09-10T18:55:49.643656+0000","last_became_active":"2024-09-10T18:45:24.082318+0000","last_became_peered":"2024-09-10T18:45:24.082318+0000","last_unstale":"2024-09-10T18:55:49.643656+0000","last_undegraded":"2024-09-10T18:55:49.643656+0000","last_fullsized":"2024-09-10T18:55:49.643656+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:24.631469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:03.475299+0000","last_clean_scrub_stamp":"2024-09-10T18:54:24.631469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014197107,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":247,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.646991+0000","last_change":"2024-09-10T18:55:40.392817+0000","last_active":"2024-09-10T18:55:49.646991+0000","last_peered":"2024-09-10T18:55:49.646991+0000","last_clean":"2024-09-10T18:55:49.646991+0000","last_became_active":"2024-09-10T18:55:40.392524+0000","last_became_peered":"2024-09-10T18:55:40.392524+0000","last_unstale":"2024-09-10T18:55:49.646991+0000","last_undegraded":"2024-09-10T18:55:49.646991+0000","last_fullsized":"2024-09-10T18:55:49.646991+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:01.165584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:51:01.165584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0071542790000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"232'19","reported_seq":282,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.648216+0000","last_change":"2024-09-10T18:55:26.062315+0000","last_active":"2024-09-10T18:55:49.648216+0000","last_peered":"2024-09-10T18:55:49.648216+0000","last_clean":"2024-09-10T18:55:49.648216+0000","last_became_active":"2024-09-10T18:45:20.058717+0000","last_became_peered":"2024-09-10T18:45:20.058717+0000","last_unstale":"2024-09-10T18:55:49.648216+0000","last_undegraded":"2024-09-10T18:55:49.648216+0000","last_fullsized":"2024-09-10T18:55:49.648216+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T18:55:26.062256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:55:26.062256+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:50.509567+0000","scrub_duration":0.001573893,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":200,"reported_epoch":248,"state":"active+clean","last_fresh":"2024-09-10T18:55:50.646510+0000","last_change":"2024-09-10T18:55:40.398399+0000","last_active":"2024-09-10T18:55:50.646510+0000","last_peered":"2024-09-10T18:55:50.646510+0000","last_clean":"2024-09-10T18:55:50.646510+0000","last_became_active":"2024-09-10T18:55:40.396437+0000","last_became_peered":"2024-09-10T18:55:40.396437+0000","last_unstale":"2024-09-10T18:55:50.646510+0000","last_undegraded":"2024-09-10T18:55:50.646510+0000","last_fullsized":"2024-09-10T18:55:50.646510+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:52.592184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:52:52.592184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015375409999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":232,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.641642+0000","last_change":"2024-09-10T18:54:21.072224+0000","last_active":"2024-09-10T18:55:49.641642+0000","last_peered":"2024-09-10T18:55:49.641642+0000","last_clean":"2024-09-10T18:55:49.641642+0000","last_became_active":"2024-09-10T18:45:24.267251+0000","last_became_peered":"2024-09-10T18:45:24.267251+0000","last_unstale":"2024-09-10T18:55:49.641642+0000","last_undegraded":"2024-09-10T18:55:49.641642+0000","last_fullsized":"2024-09-10T18:55:49.641642+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:21.071910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:54:21.071910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007087388,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"56'1","reported_seq":252,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.648368+0000","last_change":"2024-09-10T18:54:42.956938+0000","last_active":"2024-09-10T18:55:49.648368+0000","last_peered":"2024-09-10T18:55:49.648368+0000","last_clean":"2024-09-10T18:55:49.648368+0000","last_became_active":"2024-09-10T18:45:17.966286+0000","last_became_peered":"2024-09-10T18:45:17.966286+0000","last_unstale":"2024-09-10T18:55:49.648368+0000","last_undegraded":"2024-09-10T18:55:49.648368+0000","last_fullsized":"2024-09-10T18:55:49.648368+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:52:42.673300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:52:42.673300+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011239885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"232'27","reported_seq":294,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.644628+0000","last_change":"2024-09-10T18:55:40.421161+0000","last_active":"2024-09-10T18:55:49.644628+0000","last_peered":"2024-09-10T18:55:49.644628+0000","last_clean":"2024-09-10T18:55:49.644628+0000","last_became_active":"2024-09-10T18:55:40.401788+0000","last_became_peered":"2024-09-10T18:55:40.401788+0000","last_unstale":"2024-09-10T18:55:49.644628+0000","last_undegraded":"2024-09-10T18:55:49.644628+0000","last_fullsized":"2024-09-10T18:55:49.644628+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:51:59.160774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:51:59.160774+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030853120000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":243,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.643926+0000","last_change":"2024-09-10T18:55:40.398176+0000","last_active":"2024-09-10T18:55:49.643926+0000","last_peered":"2024-09-10T18:55:49.643926+0000","last_clean":"2024-09-10T18:55:49.643926+0000","last_became_active":"2024-09-10T18:55:40.397870+0000","last_became_peered":"2024-09-10T18:55:40.397870+0000","last_unstale":"2024-09-10T18:55:49.643926+0000","last_undegraded":"2024-09-10T18:55:49.643926+0000","last_fullsized":"2024-09-10T18:55:49.643926+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:54.152908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:54.152908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0079641550000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":244,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.649090+0000","last_change":"2024-09-10T18:55:39.815156+0000","last_active":"2024-09-10T18:55:49.649090+0000","last_peered":"2024-09-10T18:55:49.649090+0000","last_clean":"2024-09-10T18:55:49.649090+0000","last_became_active":"2024-09-10T18:52:54.624256+0000","last_became_peered":"2024-09-10T18:52:54.624256+0000","last_unstale":"2024-09-10T18:55:49.649090+0000","last_undegraded":"2024-09-10T18:55:49.649090+0000","last_fullsized":"2024-09-10T18:55:49.649090+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:39.815087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:39.815087+0000","last_clean_scrub_stamp":"2024-09-10T18:55:39.815087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:41.582754+0000","scrub_duration":0.0018555970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":240,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.648963+0000","last_change":"2024-09-10T18:54:46.433255+0000","last_active":"2024-09-10T18:55:49.648963+0000","last_peered":"2024-09-10T18:55:49.648963+0000","last_clean":"2024-09-10T18:55:49.648963+0000","last_became_active":"2024-09-10T18:45:17.963257+0000","last_became_peered":"2024-09-10T18:45:17.963257+0000","last_unstale":"2024-09-10T18:55:49.648963+0000","last_undegraded":"2024-09-10T18:55:49.648963+0000","last_fullsized":"2024-09-10T18:55:49.648963+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:46.433200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:54:46.433200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:05.812296+0000","scrub_duration":0.0013571259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"232'18","reported_seq":270,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.649026+0000","last_change":"2024-09-10T18:54:21.298808+0000","last_active":"2024-09-10T18:55:49.649026+0000","last_peered":"2024-09-10T18:55:49.649026+0000","last_clean":"2024-09-10T18:55:49.649026+0000","last_became_active":"2024-09-10T18:45:20.058391+0000","last_became_peered":"2024-09-10T18:45:20.058391+0000","last_unstale":"2024-09-10T18:55:49.649026+0000","last_undegraded":"2024-09-10T18:55:49.649026+0000","last_fullsized":"2024-09-10T18:55:49.649026+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:54:21.298731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:54:21.298731+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060418820000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":254,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-10T18:55:49.640892+0000","last_change":"2024-09-10T18:54:53.316597+0000","last_active":"2024-09-10T18:55:49.640892+0000","last_peered":"2024-09-10T18:55:49.640892+0000","last_clean":"2024-09-10T18:55:49.640892+0000","last_became_active":"2024-09-10T18:47:36.416131+0000","last_became_peered":"2024-09-10T18:47:36.416131+0000","last_unstale":"2024-09-10T18:55:49.640892+0000","last_undegraded":"2024-09-10T18:55:49.640892+0000","last_fullsized":"2024-09-10T18:55:49.640892+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:53.316517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:54:53.316517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:02.238593+0000","scrub_duration":0.0090189510000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":726138880,"num_objects":11080,"num_object_clones":0,"num_object_copies":33240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11080,"num_write_kb":709120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2065563648,"data_stored":2065563648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11080,"ondisk_log_size":11080,"up":48,"acting":48,"num_store_stats":7},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1120,"num_read_kb":877,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":1458,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":751,"ondisk_log_size":751,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":618,"num_read_kb":613,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1701,"ondisk_log_size":1701,"up":63,"acting":63,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299523,"num_pgs":81,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":683392,"kb_used_data":373544,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93061760,"statfs":{"total":95995035648,"available":95295242240,"internally_reserved":0,"allocated":382509056,"data_stored":381915368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:53:20 2024","interfaces":[{"interface":"back","average":{"1min":1.746,"5min":1.2689999999999999,"15min":0.873},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":11.85,"5min":21.231000000000002,"15min":31.805},"last":0.373},{"interface":"front","average":{"1min":1.595,"5min":1.4770000000000001,"15min":0.92300000000000004},"min":{"1min":0.22800000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":11.484999999999999,"5min":21.422999999999998,"15min":31.212},"last":0.46200000000000002}]},{"osd":1,"last update":"Tue Sep 10 18:55:26 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.022,"15min":0.81699999999999995},"min":{"1min":0.25,"5min":0.23100000000000001,"15min":0.221},"max":{"1min":4.702,"5min":8.6189999999999998,"15min":21.34},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":1.0589999999999999,"15min":0.82899999999999996},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":4.7759999999999998,"5min":8.5899999999999999,"15min":21.248000000000001},"last":0.39100000000000001}]},{"osd":2,"last update":"Tue Sep 10 18:55:26 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.755,"15min":0.626},"min":{"1min":0.28199999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":4.516,"5min":4.9749999999999996,"15min":5.1399999999999997},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.876,"15min":0.69399999999999995},"min":{"1min":0.32300000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.0449999999999999,"5min":3.9830000000000001,"15min":8.8859999999999992},"last":0.41999999999999998}]},{"osd":3,"last update":"Tue Sep 10 18:55:26 2024","interfaces":[{"interface":"back","average":{"1min":1.742,"5min":1.361,"15min":1.036},"min":{"1min":0.26100000000000001,"5min":0.251,"15min":0.23799999999999999},"max":{"1min":7.5899999999999999,"5min":11.282,"15min":31.478000000000002},"last":0.38100000000000001},{"interface":"front","average":{"1min":1.167,"5min":1.3220000000000001,"15min":1.095},"min":{"1min":0.34499999999999997,"5min":0.29499999999999998,"15min":0.254},"max":{"1min":6.1470000000000002,"5min":10.185,"15min":33.731999999999999},"last":0.48499999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:55:26 2024","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.1699999999999999,"15min":0.66700000000000004},"min":{"1min":0.28000000000000003,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":7.6890000000000001,"5min":7.6890000000000001,"15min":7.6890000000000001},"last":0.309},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.2310000000000001,"15min":0.68600000000000005},"min":{"1min":0.23000000000000001,"5min":0.22,"15min":0.22},"max":{"1min":2.8969999999999998,"5min":6.7270000000000003,"15min":6.7270000000000003},"last":0.46999999999999997}]},{"osd":5,"last update":"Tue Sep 10 18:55:26 2024","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":2.5499999999999998,"15min":1.141},"min":{"1min":0.27600000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":7.7789999999999999,"5min":113.255,"15min":113.255},"last":0.437},{"interface":"front","average":{"1min":1.079,"5min":2.2799999999999998,"15min":1.0489999999999999},"min":{"1min":0.29599999999999999,"5min":0.25900000000000001,"15min":0.253},"max":{"1min":3.0049999999999999,"5min":113.851,"15min":113.851},"last":0.45300000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:55:26 2024","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":2.1829999999999998,"15min":1.0549999999999999},"min":{"1min":0.25,"5min":0.247,"15min":0.247},"max":{"1min":5.7110000000000003,"5min":87.716999999999999,"15min":87.716999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":2.0289999999999999,"5min":2.157,"15min":1.0409999999999999},"min":{"1min":0.317,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":7.6609999999999996,"5min":87.738,"15min":87.738},"last":0.40899999999999997}]}]},{"osd":6,"up_from":43,"seq":184683593864,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":466240,"kb_used_data":156408,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93278912,"statfs":{"total":95995035648,"available":95517605888,"internally_reserved":0,"allocated":160161792,"data_stored":159552918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:53:51 2024","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.583,"15min":0.89600000000000002},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":3.1819999999999999,"5min":46.384999999999998,"15min":46.384999999999998},"last":0.28299999999999997},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.77,"15min":0.94899999999999995},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.22600000000000001},"max":{"1min":3.3199999999999998,"5min":48.372999999999998,"15min":48.372999999999998},"last":0.64600000000000002}]},{"osd":1,"last update":"Tue Sep 10 18:54:51 2024","interfaces":[{"interface":"back","average":{"1min":1.95,"5min":1.341,"15min":0.85499999999999998},"min":{"1min":0.251,"5min":0.22600000000000001,"15min":0.219},"max":{"1min":9.2509999999999994,"5min":9.2509999999999994,"15min":9.2509999999999994},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.6000000000000001,"5min":1.143,"15min":0.752},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.214},"max":{"1min":9.1750000000000007,"5min":9.1750000000000007,"15min":9.1750000000000007},"last":0.77400000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:54:51 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.93500000000000005,"15min":0.69699999999999995},"min":{"1min":0.29499999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":2.4169999999999998,"5min":4.0119999999999996,"15min":4.4740000000000002},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.3080000000000001,"5min":0.93500000000000005,"15min":0.66300000000000003},"min":{"1min":0.316,"5min":0.29199999999999998,"15min":0.27200000000000002},"max":{"1min":4.8150000000000004,"5min":4.8150000000000004,"15min":4.8150000000000004},"last":0.89400000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:54:51 2024","interfaces":[{"interface":"back","average":{"1min":1.6659999999999999,"5min":1.1619999999999999,"15min":0.76700000000000002},"min":{"1min":0.28399999999999997,"5min":0.27600000000000002,"15min":0.25700000000000001},"max":{"1min":10.891999999999999,"5min":10.891999999999999,"15min":10.891999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":1.655,"5min":1.3169999999999999,"15min":0.89300000000000002},"min":{"1min":0.36499999999999999,"5min":0.253,"15min":0.253},"max":{"1min":10.657,"5min":10.657,"15min":10.657},"last":0.753}]},{"osd":4,"last update":"Tue Sep 10 18:54:51 2024","interfaces":[{"interface":"back","average":{"1min":1.8420000000000001,"5min":2.8100000000000001,"15min":1.228},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.20999999999999999},"max":{"1min":5.335,"5min":165.03299999999999,"15min":165.03299999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.159,"5min":2.5179999999999998,"15min":1.127},"min":{"1min":0.255,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":3.2869999999999999,"5min":163.959,"15min":163.959},"last":0.53900000000000003}]},{"osd":5,"last update":"Tue Sep 10 18:54:51 2024","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.6080000000000001,"15min":0.83999999999999997},"min":{"1min":0.27100000000000002,"5min":0.26700000000000002,"15min":0.24099999999999999},"max":{"1min":3.3159999999999998,"5min":43.811,"15min":43.811},"last":0.69899999999999995},{"interface":"front","average":{"1min":1.8200000000000001,"5min":1.659,"15min":0.86499999999999999},"min":{"1min":0.28199999999999997,"5min":0.27800000000000002,"15min":0.23999999999999999},"max":{"1min":4.7599999999999998,"5min":42.314,"15min":42.314},"last":0.97699999999999998}]},{"osd":7,"last update":"Tue Sep 10 18:55:21 2024","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.1439999999999999,"15min":0.73399999999999999},"min":{"1min":0.309,"5min":0.29799999999999999,"15min":0.26200000000000001},"max":{"1min":5.077,"5min":5.3680000000000003,"15min":5.3680000000000003},"last":1.889},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.2110000000000001,"15min":0.77600000000000002},"min":{"1min":0.34100000000000003,"5min":0.32500000000000001,"15min":0.29099999999999998},"max":{"1min":4.0620000000000003,"5min":7.0350000000000001,"15min":7.0350000000000001},"last":1.9350000000000001}]}]},{"osd":5,"up_from":37,"seq":158913790092,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":685492,"kb_used_data":383964,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93059660,"statfs":{"total":95995035648,"available":95293091840,"internally_reserved":0,"allocated":393179136,"data_stored":392596984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:53:32 2024","interfaces":[{"interface":"back","average":{"1min":6.4000000000000004,"5min":2.8849999999999998,"15min":1.3169999999999999},"min":{"1min":0.27400000000000002,"5min":0.183,"15min":0.183},"max":{"1min":73.674000000000007,"5min":73.674000000000007,"15min":73.674000000000007},"last":0.23999999999999999},{"interface":"front","average":{"1min":6.6959999999999997,"5min":2.8260000000000001,"15min":1.321},"min":{"1min":0.27900000000000003,"5min":0.219,"15min":0.219},"max":{"1min":73.488,"5min":73.488,"15min":73.488},"last":0.33700000000000002}]},{"osd":1,"last update":"Tue Sep 10 18:55:33 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":1.0529999999999999,"15min":0.76300000000000001},"min":{"1min":0.20399999999999999,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":2.5630000000000002,"5min":12.920999999999999,"15min":12.920999999999999},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.423,"15min":0.91800000000000004},"min":{"1min":0.23599999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":8.9090000000000007,"5min":19.474,"15min":19.474},"last":1.254}]},{"osd":2,"last update":"Tue Sep 10 18:55:33 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.90500000000000003,"15min":0.68700000000000006},"min":{"1min":0.30099999999999999,"5min":0.27900000000000003,"15min":0.249},"max":{"1min":3.218,"5min":4.6319999999999997,"15min":7.8029999999999999},"last":2.669},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.88500000000000001,"15min":0.71999999999999997},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":2.6269999999999998,"5min":4.8680000000000003,"15min":6.7210000000000001},"last":2.766}]},{"osd":3,"last update":"Tue Sep 10 18:55:33 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":1.742,"15min":1.2809999999999999},"min":{"1min":0.315,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":3.153,"5min":32.479999999999997,"15min":66.593999999999994},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.71699999999999997,"5min":2.093,"15min":1.4099999999999999},"min":{"1min":0.34000000000000002,"5min":0.245,"15min":0.245},"max":{"1min":2.6019999999999999,"5min":32.171999999999997,"15min":65.939999999999998},"last":0.60599999999999998}]},{"osd":4,"last update":"Tue Sep 10 18:55:33 2024","interfaces":[{"interface":"back","average":{"1min":1.6739999999999999,"5min":1.48,"15min":0.77600000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.20899999999999999},"max":{"1min":9.0310000000000006,"5min":9.0310000000000006,"15min":9.0310000000000006},"last":0.30199999999999999},{"interface":"front","average":{"1min":1.55,"5min":1.3480000000000001,"15min":0.73799999999999999},"min":{"1min":0.255,"5min":0.24399999999999999,"15min":0.23400000000000001},"max":{"1min":11.182,"5min":11.182,"15min":11.182},"last":0.26500000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:54:57 2024","interfaces":[{"interface":"back","average":{"1min":1.8540000000000001,"5min":1.6020000000000001,"15min":0.86299999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":9.0069999999999997,"5min":11.17,"15min":11.17},"last":0.39600000000000002},{"interface":"front","average":{"1min":2.0190000000000001,"5min":1.621,"15min":0.89600000000000002},"min":{"1min":0.41699999999999998,"5min":0.32900000000000001,"15min":0.31},"max":{"1min":7.3719999999999999,"5min":11.18,"15min":11.18},"last":0.38400000000000001}]},{"osd":7,"last update":"Tue Sep 10 18:55:19 2024","interfaces":[{"interface":"back","average":{"1min":1.8660000000000001,"5min":1.4510000000000001,"15min":0.84399999999999997},"min":{"1min":0.33200000000000002,"5min":0.29299999999999998,"15min":0.27600000000000002},"max":{"1min":5.3410000000000002,"5min":5.7249999999999996,"15min":10.782999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":2.0059999999999998,"5min":1.2370000000000001,"15min":0.77900000000000003},"min":{"1min":0.30399999999999999,"5min":0.24099999999999999,"15min":0.23599999999999999},"max":{"1min":4.2430000000000003,"5min":5.9500000000000002,"15min":11.846},"last":0.372}]}]},{"osd":4,"up_from":30,"seq":128849019026,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":379452,"kb_used_data":82088,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93365700,"statfs":{"total":95995035648,"available":95606476800,"internally_reserved":0,"allocated":84058112,"data_stored":83457922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:53:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.5900000000000001,"15min":0.88100000000000001},"min":{"1min":0.215,"5min":0.21099999999999999,"15min":0.20699999999999999},"max":{"1min":2.4249999999999998,"5min":55.776000000000003,"15min":55.776000000000003},"last":0.443},{"interface":"front","average":{"1min":1.181,"5min":1.607,"15min":0.88900000000000001},"min":{"1min":0.26200000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":4.1429999999999998,"5min":55.737000000000002,"15min":55.737000000000002},"last":0.47599999999999998}]},{"osd":1,"last update":"Tue Sep 10 18:55:17 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.5269999999999999,"15min":0.96999999999999997},"min":{"1min":0.33700000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":5.7300000000000004,"5min":45.984999999999999,"15min":45.984999999999999},"last":3.9769999999999999},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.425,"15min":0.86799999999999999},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":5.5910000000000002,"5min":43.590000000000003,"15min":43.590000000000003},"last":1.921}]},{"osd":2,"last update":"Tue Sep 10 18:55:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":0.82999999999999996,"15min":0.64200000000000002},"min":{"1min":0.27300000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":5.6189999999999998,"5min":5.6189999999999998,"15min":5.6189999999999998},"last":0.38900000000000001},{"interface":"front","average":{"1min":1.028,"5min":0.92900000000000005,"15min":0.68100000000000005},"min":{"1min":0.28000000000000003,"5min":0.27900000000000003,"15min":0.27100000000000002},"max":{"1min":4.5129999999999999,"5min":6.3540000000000001,"15min":6.3540000000000001},"last":0.60699999999999998}]},{"osd":3,"last update":"Tue Sep 10 18:55:17 2024","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.6080000000000001,"15min":1.073},"min":{"1min":0.249,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":4.4800000000000004,"5min":43.826000000000001,"15min":43.826000000000001},"last":4.5140000000000002},{"interface":"front","average":{"1min":1.2,"5min":1.486,"15min":1.014},"min":{"1min":0.33000000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":4.1210000000000004,"5min":43.570999999999998,"15min":43.570999999999998},"last":0.55500000000000005}]},{"osd":5,"last update":"Tue Sep 10 18:55:41 2024","interfaces":[{"interface":"back","average":{"1min":1.5800000000000001,"5min":1.4359999999999999,"15min":0.76000000000000001},"min":{"1min":0.23200000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":10.912000000000001,"5min":10.912000000000001,"15min":10.912000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.244,"15min":0.68799999999999994},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":5.6429999999999998,"5min":5.6429999999999998,"15min":5.6429999999999998},"last":0.50800000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:55:01 2024","interfaces":[{"interface":"back","average":{"1min":1.7669999999999999,"5min":1.056,"15min":0.66200000000000003},"min":{"1min":0.28000000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":8.4030000000000005,"5min":8.4030000000000005,"15min":8.4030000000000005},"last":0.62},{"interface":"front","average":{"1min":1.8460000000000001,"5min":1.3080000000000001,"15min":0.73999999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.25800000000000001},"max":{"1min":6.8710000000000004,"5min":7.7699999999999996,"15min":7.7699999999999996},"last":0.57799999999999996}]},{"osd":7,"last update":"Tue Sep 10 18:55:28 2024","interfaces":[{"interface":"back","average":{"1min":1.8740000000000001,"5min":1.248,"15min":0.77000000000000002},"min":{"1min":0.33000000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":6.4630000000000001,"5min":6.4630000000000001,"15min":6.4630000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":1.538,"5min":1.194,"15min":0.72099999999999997},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":4.5449999999999999,"5min":6.3949999999999996,"15min":6.3949999999999996},"last":0.64200000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182551,"num_pgs":80,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":882700,"kb_used_data":292796,"kb_used_omap":1,"kb_used_meta":589886,"kb_avail":92862452,"statfs":{"total":95995035648,"available":95091150848,"internally_reserved":0,"allocated":299823104,"data_stored":299209164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":604043608},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,2,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:53:47 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":2.6099999999999999,"15min":1.214},"min":{"1min":0.182,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":2.544,"5min":83.126000000000005,"15min":83.126000000000005},"last":1.1200000000000001},{"interface":"front","average":{"1min":0.95199999999999996,"5min":2.5699999999999998,"15min":1.2150000000000001},"min":{"1min":0.19,"5min":0.184,"15min":0.184},"max":{"1min":6.2800000000000002,"5min":81.632999999999996,"15min":81.632999999999996},"last":0.81299999999999994}]},{"osd":1,"last update":"Tue Sep 10 18:55:50 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":1.2969999999999999,"15min":0.996},"min":{"1min":0.16200000000000001,"5min":0.16200000000000001,"15min":0.16200000000000001},"max":{"1min":1.425,"5min":58.215000000000003,"15min":58.215000000000003},"last":1.425},{"interface":"front","average":{"1min":0.42699999999999999,"5min":1.3080000000000001,"15min":1.02},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":2.0649999999999999,"5min":57.896999999999998,"15min":57.896999999999998},"last":0.377}]},{"osd":2,"last update":"Tue Sep 10 18:55:50 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.63,"15min":0.67800000000000005},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":2.133,"5min":3.9140000000000001,"15min":5.8220000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.378,"5min":0.52800000000000002,"15min":0.61299999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.95299999999999996,"5min":4.6289999999999996,"15min":7.0979999999999999},"last":0.95299999999999996}]},{"osd":4,"last update":"Tue Sep 10 18:55:18 2024","interfaces":[{"interface":"back","average":{"1min":2.0990000000000002,"5min":1.419,"15min":0.93500000000000005},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":29.013000000000002,"5min":29.013000000000002,"15min":29.013000000000002},"last":1},{"interface":"front","average":{"1min":2.2650000000000001,"5min":1.399,"15min":0.93400000000000005},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":28.826000000000001,"5min":28.826000000000001,"15min":28.826000000000001},"last":1.0269999999999999}]},{"osd":5,"last update":"Tue Sep 10 18:55:41 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":1.145,"15min":0.90100000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":3.1829999999999998,"5min":6.1989999999999998,"15min":6.1989999999999998},"last":1.2929999999999999},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.1910000000000001,"15min":0.92600000000000005},"min":{"1min":0.36299999999999999,"5min":0.255,"15min":0.255},"max":{"1min":3.1949999999999998,"5min":6.1870000000000003,"15min":9.2509999999999994},"last":0.69699999999999995}]},{"osd":6,"last update":"Tue Sep 10 18:55:00 2024","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.3120000000000001,"15min":0.92600000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":3.8069999999999999,"5min":9.3239999999999998,"15min":9.3239999999999998},"last":0.73099999999999998},{"interface":"front","average":{"1min":1.274,"5min":1.302,"15min":0.90600000000000003},"min":{"1min":0.35299999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":5.2670000000000003,"5min":7.1559999999999997,"15min":7.1559999999999997},"last":1.3560000000000001}]},{"osd":7,"last update":"Tue Sep 10 18:55:29 2024","interfaces":[{"interface":"back","average":{"1min":1.6479999999999999,"5min":1.206,"15min":0.91000000000000003},"min":{"1min":0.438,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":5.2569999999999997,"5min":6.0839999999999996,"15min":6.0839999999999996},"last":1.333},{"interface":"front","average":{"1min":1.423,"5min":1.2989999999999999,"15min":0.98199999999999998},"min":{"1min":0.49299999999999999,"5min":0.39300000000000002,"15min":0.33100000000000002},"max":{"1min":5.274,"5min":5.274,"15min":9.2949999999999999},"last":0.76100000000000001}]}]},{"osd":2,"up_from":0,"seq":77309411483,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834575007,"num_pgs":80,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":680780,"kb_used_data":383424,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93064372,"statfs":{"total":95995035648,"available":95297916928,"internally_reserved":0,"allocated":392626176,"data_stored":392008179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:52:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":1.278,"15min":0.64200000000000002},"min":{"1min":0.215,"5min":0.193,"15min":0.193},"max":{"1min":1.913,"5min":13.531000000000001,"15min":13.531000000000001},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.93600000000000005,"5min":1.8260000000000001,"15min":0.83999999999999997},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":4.3630000000000004,"5min":16.206,"15min":16.206},"last":0.88300000000000001}]},{"osd":2,"last update":"Tue Sep 10 18:55:28 2024","interfaces":[{"interface":"back","average":{"1min":0.28599999999999998,"5min":0.68200000000000005,"15min":0.54600000000000004},"min":{"1min":0.19600000000000001,"5min":0.186,"15min":0.186},"max":{"1min":0.51100000000000001,"5min":2.8159999999999998,"15min":3.8479999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.63900000000000001,"15min":0.58599999999999997},"min":{"1min":0.23000000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.79500000000000004,"5min":3.754,"15min":7.093},"last":0.28899999999999998}]},{"osd":3,"last update":"Tue Sep 10 18:54:56 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":1.365,"15min":0.91100000000000003},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":4.2119999999999997,"5min":12.714,"15min":12.714},"last":1.1579999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":1.1919999999999999,"15min":0.92700000000000005},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":1.3839999999999999,"5min":12.696,"15min":12.696},"last":0.82499999999999996}]},{"osd":4,"last update":"Tue Sep 10 18:55:18 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":1.298,"15min":0.93300000000000005},"min":{"1min":0.29799999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":3.222,"5min":15.965999999999999,"15min":15.965999999999999},"last":1.012},{"interface":"front","average":{"1min":0.67200000000000004,"5min":1.073,"15min":0.79300000000000004},"min":{"1min":0.28100000000000003,"5min":0.246,"15min":0.246},"max":{"1min":2.109,"5min":3.1659999999999999,"15min":3.9540000000000002},"last":0.97199999999999998}]},{"osd":5,"last update":"Tue Sep 10 18:55:40 2024","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.4099999999999999,"15min":0.96499999999999997},"min":{"1min":0.28899999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":10.334,"5min":10.334,"15min":10.334},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.72099999999999997,"5min":1.4099999999999999,"15min":1.036},"min":{"1min":0.29699999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":4.609,"5min":16.039999999999999,"15min":16.039999999999999},"last":0.85899999999999999}]},{"osd":6,"last update":"Tue Sep 10 18:54:56 2024","interfaces":[{"interface":"back","average":{"1min":2.1040000000000001,"5min":1.8300000000000001,"15min":1.2450000000000001},"min":{"1min":0.315,"5min":0.308,"15min":0.308},"max":{"1min":7.6760000000000002,"5min":33.290999999999997,"15min":38.039999999999999},"last":1.21},{"interface":"front","average":{"1min":1.929,"5min":2.02,"15min":1.369},"min":{"1min":0.33400000000000002,"5min":0.32800000000000001,"15min":0.30599999999999999},"max":{"1min":7.7169999999999996,"5min":33.310000000000002,"15min":38.100999999999999},"last":1.0349999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:55:28 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.226,"15min":0.93100000000000005},"min":{"1min":0.38500000000000001,"5min":0.315,"15min":0.315},"max":{"1min":4.5449999999999999,"5min":4.5449999999999999,"15min":4.6539999999999999},"last":1.238},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.0469999999999999,"15min":0.91100000000000003},"min":{"1min":0.36199999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":4.9029999999999996,"5min":4.9029999999999996,"15min":4.9029999999999996},"last":0.90400000000000003}]}]},{"osd":0,"up_from":241,"seq":1035087118340,"num_pgs":74,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2831812,"kb_used_data":2497980,"kb_used_omap":4,"kb_used_meta":333755,"kb_avail":90913340,"statfs":{"total":95995035648,"available":93095260160,"internally_reserved":0,"allocated":2557931520,"data_stored":2557323791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":341765638},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9410000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.867}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9019999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":331939840,"data_stored":331939840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":406061056,"data_stored":406061056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":385482752,"data_stored":385482752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":110166016,"data_stored":110166016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":244252672,"data_stored":244252672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":164560896,"data_stored":164560896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":423100416,"data_stored":423100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T18:55:56.425 INFO:tasks.cephadm.ceph_manager.ceph:would drop below min_pgs, proj 6, currently 16 2024-09-10T18:55:56.484 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:56.520 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:56.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:56.521 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:56.622 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:56.663 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:56.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:56.664 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:56.765 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:56.801 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:56.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:56.802 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:56.903 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:56.934 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:56.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:56.935 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:57.022 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:55:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:55:57.036 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:55:57.072 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:57.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:57.073 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:55:57.174 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:57.206 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:57.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:57.207 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:57.308 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:57.320 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:55:57.344 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:57.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:57.345 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:57.383 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 9 16 25390 25374 176.188 159.062 0.00377351 0.00566937 2024-09-10T18:55:57.446 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:57.482 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:57.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:57.483 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:57.583 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:57.605 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:57.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:57.606 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:57.707 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:57.739 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:57.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:57.740 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:57.840 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:57.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:57.863 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:57.863 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:57.964 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:57.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:57.996 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:57.996 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:58.097 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:58.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:58.129 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:58.129 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:58.230 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:58.266 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:58.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:58.267 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:58.368 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:55:58.383 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 10 16 27343 27327 170.772 122.062 0.00594584 0.00584982 2024-09-10T18:55:58.409 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:58.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:58.410 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:55:58.510 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:58 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:58.510 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:58 smithi103 ceph-mon[33163]: pgmap v485: 168 pgs: 168 active+clean; 1.3 GiB data, 8.5 GiB used, 617 GiB / 626 GiB avail; 1.7 KiB/s rd, 226 MiB/s wr, 3.62k op/s 2024-09-10T18:55:58.511 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:58.552 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:58.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:58.553 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:58.633 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:58 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:58.633 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:58 smithi018 ceph-mon[29336]: pgmap v485: 168 pgs: 168 active+clean; 1.3 GiB data, 8.5 GiB used, 617 GiB / 626 GiB avail; 1.7 KiB/s rd, 226 MiB/s wr, 3.62k op/s 2024-09-10T18:55:58.634 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:58 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:58.634 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:58 smithi018 ceph-mon[40196]: pgmap v485: 168 pgs: 168 active+clean; 1.3 GiB data, 8.5 GiB used, 617 GiB / 626 GiB avail; 1.7 KiB/s rd, 226 MiB/s wr, 3.62k op/s 2024-09-10T18:55:58.654 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:58.687 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:58.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:58.688 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:58.788 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:55:58.824 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:58.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:58.825 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:55:58.925 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:55:58.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:58.966 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:58.967 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:55:59.067 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:55:59.089 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:59.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:59.090 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:55:59.191 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:55:59.226 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:59.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:59.227 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:55:59.328 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:55:59.364 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:59.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:59.365 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:55:59.383 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 11 16 29343 29327 166.61 125 0.00865775 0.00599664 2024-09-10T18:55:59.467 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:55:59.507 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:55:59.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:59.508 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:55:59.609 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:55:59.641 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:59.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:59.642 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:55:59.743 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:59.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:59.778 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:59.778 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:59.834 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T18:55:59.834 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:55:59.878 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:55:59.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:55:59.916 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:55:59.916 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:55:59.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:59 smithi018 ceph-mon[40196]: osdmap e252: 8 total, 8 up, 7 in 2024-09-10T18:55:59.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:55:59 smithi018 ceph-mon[40196]: pgmap v487: 168 pgs: 168 active+clean; 1.5 GiB data, 8.7 GiB used, 617 GiB / 626 GiB avail; 852 B/s rd, 164 MiB/s wr, 2.62k op/s 2024-09-10T18:55:59.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:59 smithi018 ceph-mon[29336]: osdmap e252: 8 total, 8 up, 7 in 2024-09-10T18:55:59.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:55:59 smithi018 ceph-mon[29336]: pgmap v487: 168 pgs: 168 active+clean; 1.5 GiB data, 8.7 GiB used, 617 GiB / 626 GiB avail; 852 B/s rd, 164 MiB/s wr, 2.62k op/s 2024-09-10T18:55:59.989 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:59 smithi103 ceph-mon[33163]: osdmap e252: 8 total, 8 up, 7 in 2024-09-10T18:55:59.989 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:55:59 smithi103 ceph-mon[33163]: pgmap v487: 168 pgs: 168 active+clean; 1.5 GiB data, 8.7 GiB used, 617 GiB / 626 GiB avail; 852 B/s rd, 164 MiB/s wr, 2.62k op/s 2024-09-10T18:56:00.017 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:00.061 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:00.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:00.062 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:00.163 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:00.208 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:00.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:00.209 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:00.310 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:00.341 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:00.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:00.342 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:00.383 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 12 16 31515 31499 164.038 135.75 0.00704177 0.0060846 2024-09-10T18:56:00.443 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:00.489 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:00.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:00.490 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:00.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:56:00 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:56:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:56:00.591 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:00.619 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:00.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:00.620 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:00.661 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:56:00.661 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:56:00.669 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is unset 2024-09-10T18:56:00.703 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:56:00.703 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:56:00.721 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:00.728 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-10T18:56:00.741 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:56:00.741 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:56:00.753 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:00.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:00.753 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:00.773 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:56:00.773 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:56:00.809 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:56:00.809 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:56:00.840 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:56:00.840 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:56:00.854 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:00.876 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:00 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:56:00.877 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:00 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:56:00.877 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:00 smithi018 ceph-mon[29336]: osdmap e253: 8 total, 8 up, 7 in 2024-09-10T18:56:00.877 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:00 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3228601032' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:56:00.877 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:00 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3467994826' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:56:00.877 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:00 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:56:00.877 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:00 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:56:00.877 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:00 smithi018 ceph-mon[40196]: osdmap e253: 8 total, 8 up, 7 in 2024-09-10T18:56:00.877 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:00 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3228601032' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:56:00.878 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:00 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3467994826' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:56:00.878 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:56:00.878 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:56:00.884 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:00.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:00.885 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:00.905 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:56:00.905 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:56:00.986 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:01.014 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:01.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:01.015 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:01.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:00 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:56:01.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:00 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:56:01.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:00 smithi103 ceph-mon[33163]: osdmap e253: 8 total, 8 up, 7 in 2024-09-10T18:56:01.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:00 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3228601032' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:56:01.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:00 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3467994826' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:56:01.116 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:01.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:01.146 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:01.147 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:01.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:56:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:56:00] "GET /metrics HTTP/1.1" 200 224107 "" "Prometheus/2.43.0" 2024-09-10T18:56:01.248 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:01.271 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:01.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:01.272 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:01.373 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:01.383 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 13 16 33345 33329 160.217 114.375 0.00631345 0.0062373 2024-09-10T18:56:01.406 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:01.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:01.408 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:01.426 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 6, 7, 0] 2024-09-10T18:56:01.426 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T18:56:01.426 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-10T18:56:01.427 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd dump -f json-pretty 2024-09-10T18:56:01.508 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:01.551 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:01.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:01.552 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:01.653 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:01.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:01.701 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:01.701 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:01.802 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:01.847 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:01.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:01.848 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:01.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:01 smithi018 ceph-mon[40196]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:56:01.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:01 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3228601032' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:56:01.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:01 smithi018 ceph-mon[40196]: osdmap e254: 8 total, 8 up, 7 in 2024-09-10T18:56:01.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:01 smithi018 ceph-mon[40196]: 5.1b scrub starts 2024-09-10T18:56:01.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:01 smithi018 ceph-mon[40196]: 5.1b scrub ok 2024-09-10T18:56:01.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:01 smithi018 ceph-mon[40196]: pgmap v490: 168 pgs: 168 active+clean; 1.5 GiB data, 8.8 GiB used, 617 GiB / 626 GiB avail; 852 B/s rd, 132 MiB/s wr, 2.12k op/s 2024-09-10T18:56:01.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:01 smithi018 ceph-mon[29336]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:56:01.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:01 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3228601032' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:56:01.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:01 smithi018 ceph-mon[29336]: osdmap e254: 8 total, 8 up, 7 in 2024-09-10T18:56:01.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:01 smithi018 ceph-mon[29336]: 5.1b scrub starts 2024-09-10T18:56:01.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:01 smithi018 ceph-mon[29336]: 5.1b scrub ok 2024-09-10T18:56:01.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:01 smithi018 ceph-mon[29336]: pgmap v490: 168 pgs: 168 active+clean; 1.5 GiB data, 8.8 GiB used, 617 GiB / 626 GiB avail; 852 B/s rd, 132 MiB/s wr, 2.12k op/s 2024-09-10T18:56:01.948 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:01.984 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:01.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:01.987 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:02.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:01 smithi103 ceph-mon[33163]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:56:02.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:01 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3228601032' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:56:02.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:01 smithi103 ceph-mon[33163]: osdmap e254: 8 total, 8 up, 7 in 2024-09-10T18:56:02.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:01 smithi103 ceph-mon[33163]: 5.1b scrub starts 2024-09-10T18:56:02.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:01 smithi103 ceph-mon[33163]: 5.1b scrub ok 2024-09-10T18:56:02.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:01 smithi103 ceph-mon[33163]: pgmap v490: 168 pgs: 168 active+clean; 1.5 GiB data, 8.8 GiB used, 617 GiB / 626 GiB avail; 852 B/s rd, 132 MiB/s wr, 2.12k op/s 2024-09-10T18:56:02.087 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:02.120 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:02.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:02.121 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:02.222 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:02.254 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:02.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:02.256 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:02.357 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:02.378 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:02.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:02.379 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:02.383 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 14 16 35273 35257 157.379 120.5 0.0645217 0.00633478 2024-09-10T18:56:02.480 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:02.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:02.524 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:02.524 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:02.625 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:02.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:02.652 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:02.653 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:02.753 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:02.792 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:02.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:02.794 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:02.895 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:02.933 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:02.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:02.934 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:02.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:02 smithi018 ceph-mon[40196]: 2.19 scrub starts 2024-09-10T18:56:02.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:02 smithi018 ceph-mon[40196]: 2.19 scrub ok 2024-09-10T18:56:02.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:02 smithi018 ceph-mon[40196]: 2.5 scrub starts 2024-09-10T18:56:02.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:02 smithi018 ceph-mon[40196]: 2.5 scrub ok 2024-09-10T18:56:02.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:02 smithi018 ceph-mon[40196]: 2.f scrub starts 2024-09-10T18:56:02.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:02 smithi018 ceph-mon[40196]: 2.f scrub ok 2024-09-10T18:56:02.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:02 smithi018 ceph-mon[40196]: 6.d scrub starts 2024-09-10T18:56:02.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:02 smithi018 ceph-mon[40196]: 6.d scrub ok 2024-09-10T18:56:02.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:02 smithi018 ceph-mon[29336]: 2.19 scrub starts 2024-09-10T18:56:02.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:02 smithi018 ceph-mon[29336]: 2.19 scrub ok 2024-09-10T18:56:02.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:02 smithi018 ceph-mon[29336]: 2.5 scrub starts 2024-09-10T18:56:02.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:02 smithi018 ceph-mon[29336]: 2.5 scrub ok 2024-09-10T18:56:02.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:02 smithi018 ceph-mon[29336]: 2.f scrub starts 2024-09-10T18:56:02.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:02 smithi018 ceph-mon[29336]: 2.f scrub ok 2024-09-10T18:56:02.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:02 smithi018 ceph-mon[29336]: 6.d scrub starts 2024-09-10T18:56:02.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:02 smithi018 ceph-mon[29336]: 6.d scrub ok 2024-09-10T18:56:03.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:02 smithi103 ceph-mon[33163]: 2.19 scrub starts 2024-09-10T18:56:03.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:02 smithi103 ceph-mon[33163]: 2.19 scrub ok 2024-09-10T18:56:03.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:02 smithi103 ceph-mon[33163]: 2.5 scrub starts 2024-09-10T18:56:03.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:02 smithi103 ceph-mon[33163]: 2.5 scrub ok 2024-09-10T18:56:03.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:02 smithi103 ceph-mon[33163]: 2.f scrub starts 2024-09-10T18:56:03.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:02 smithi103 ceph-mon[33163]: 2.f scrub ok 2024-09-10T18:56:03.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:02 smithi103 ceph-mon[33163]: 6.d scrub starts 2024-09-10T18:56:03.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:02 smithi103 ceph-mon[33163]: 6.d scrub ok 2024-09-10T18:56:03.035 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:03.069 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:03.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:03.070 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:03.171 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:03.210 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:03.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:03.211 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:03.312 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:03.345 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:03.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:03.346 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:03.384 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 15 16 37279 37263 155.245 125.375 0.00635455 0.00643622 2024-09-10T18:56:03.447 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:03.469 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:03.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:03.470 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:03.571 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:03.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:03.603 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:03.604 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:03.704 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:03.743 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:03.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:03.744 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:03.845 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:03.878 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:03.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:03.879 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:03.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:03 smithi018 ceph-mon[40196]: 5.2 scrub starts 2024-09-10T18:56:03.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:03 smithi018 ceph-mon[40196]: 5.2 scrub ok 2024-09-10T18:56:03.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:03 smithi018 ceph-mon[40196]: 4.12 scrub starts 2024-09-10T18:56:03.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:03 smithi018 ceph-mon[40196]: 4.12 scrub ok 2024-09-10T18:56:03.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:03 smithi018 ceph-mon[40196]: pgmap v491: 168 pgs: 168 active+clean; 2.0 GiB data, 10 GiB used, 615 GiB / 626 GiB avail; 852 B/s rd, 110 MiB/s wr, 1.76k op/s 2024-09-10T18:56:03.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:03 smithi018 ceph-mon[29336]: 5.2 scrub starts 2024-09-10T18:56:03.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:03 smithi018 ceph-mon[29336]: 5.2 scrub ok 2024-09-10T18:56:03.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:03 smithi018 ceph-mon[29336]: 4.12 scrub starts 2024-09-10T18:56:03.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:03 smithi018 ceph-mon[29336]: 4.12 scrub ok 2024-09-10T18:56:03.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:03 smithi018 ceph-mon[29336]: pgmap v491: 168 pgs: 168 active+clean; 2.0 GiB data, 10 GiB used, 615 GiB / 626 GiB avail; 852 B/s rd, 110 MiB/s wr, 1.76k op/s 2024-09-10T18:56:03.980 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:04.019 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:04.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:04.020 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:04.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:03 smithi103 ceph-mon[33163]: 5.2 scrub starts 2024-09-10T18:56:04.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:03 smithi103 ceph-mon[33163]: 5.2 scrub ok 2024-09-10T18:56:04.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:03 smithi103 ceph-mon[33163]: 4.12 scrub starts 2024-09-10T18:56:04.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:03 smithi103 ceph-mon[33163]: 4.12 scrub ok 2024-09-10T18:56:04.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:03 smithi103 ceph-mon[33163]: pgmap v491: 168 pgs: 168 active+clean; 2.0 GiB data, 10 GiB used, 615 GiB / 626 GiB avail; 852 B/s rd, 110 MiB/s wr, 1.76k op/s 2024-09-10T18:56:04.121 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:04.158 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:04.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:04.159 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:04.260 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:04.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:04.296 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:04.296 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:04.384 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 16 16 39125 39109 152.752 115.375 0.0101401 0.00654159 2024-09-10T18:56:04.396 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:04.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:04.430 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:04.430 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:04.531 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:04.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:04.563 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:04.564 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:04.665 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:04.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:04.727 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:04.727 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:04.773 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:56:04.810 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:56:04.828 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:04.876 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:04.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:04.877 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:04.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:04 smithi018 ceph-mon[29336]: 4.19 scrub starts 2024-09-10T18:56:04.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:04 smithi018 ceph-mon[29336]: 4.19 scrub ok 2024-09-10T18:56:04.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:04 smithi018 ceph-mon[40196]: 4.19 scrub starts 2024-09-10T18:56:04.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:04 smithi018 ceph-mon[40196]: 4.19 scrub ok 2024-09-10T18:56:04.977 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:05.020 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:05.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:05.021 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:05.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:04 smithi103 ceph-mon[33163]: 4.19 scrub starts 2024-09-10T18:56:05.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:04 smithi103 ceph-mon[33163]: 4.19 scrub ok 2024-09-10T18:56:05.122 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:05.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:05.154 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:05.154 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:05.255 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:05.294 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:05.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:05.295 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:05.384 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 17 16 41106 41090 151.049 123.812 0.00895393 0.00661543 2024-09-10T18:56:05.396 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:05.429 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:05.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:05.431 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:05.531 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:05.573 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:05.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:05.574 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:05.675 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:05.715 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:05.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:05.716 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:05.817 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:05.851 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:05.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:05.853 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:05.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:05 smithi018 ceph-mon[40196]: 2.1 scrub starts 2024-09-10T18:56:05.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:05 smithi018 ceph-mon[40196]: 2.1 scrub ok 2024-09-10T18:56:05.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:05 smithi018 ceph-mon[40196]: 6.10 scrub starts 2024-09-10T18:56:05.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:05 smithi018 ceph-mon[40196]: 6.10 scrub ok 2024-09-10T18:56:05.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:05 smithi018 ceph-mon[40196]: 5.13 scrub starts 2024-09-10T18:56:05.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:05 smithi018 ceph-mon[40196]: 5.13 scrub ok 2024-09-10T18:56:05.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:05 smithi018 ceph-mon[40196]: pgmap v492: 168 pgs: 168 active+clean; 2.1 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 774 B/s rd, 98 MiB/s wr, 1.57k op/s 2024-09-10T18:56:05.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:05 smithi018 ceph-mon[29336]: 2.1 scrub starts 2024-09-10T18:56:05.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:05 smithi018 ceph-mon[29336]: 2.1 scrub ok 2024-09-10T18:56:05.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:05 smithi018 ceph-mon[29336]: 6.10 scrub starts 2024-09-10T18:56:05.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:05 smithi018 ceph-mon[29336]: 6.10 scrub ok 2024-09-10T18:56:05.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:05 smithi018 ceph-mon[29336]: 5.13 scrub starts 2024-09-10T18:56:05.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:05 smithi018 ceph-mon[29336]: 5.13 scrub ok 2024-09-10T18:56:05.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:05 smithi018 ceph-mon[29336]: pgmap v492: 168 pgs: 168 active+clean; 2.1 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 774 B/s rd, 98 MiB/s wr, 1.57k op/s 2024-09-10T18:56:05.954 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:05.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:05.984 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:05.984 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:06.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:05 smithi103 ceph-mon[33163]: 2.1 scrub starts 2024-09-10T18:56:06.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:05 smithi103 ceph-mon[33163]: 2.1 scrub ok 2024-09-10T18:56:06.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:05 smithi103 ceph-mon[33163]: 6.10 scrub starts 2024-09-10T18:56:06.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:05 smithi103 ceph-mon[33163]: 6.10 scrub ok 2024-09-10T18:56:06.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:05 smithi103 ceph-mon[33163]: 5.13 scrub starts 2024-09-10T18:56:06.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:05 smithi103 ceph-mon[33163]: 5.13 scrub ok 2024-09-10T18:56:06.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:05 smithi103 ceph-mon[33163]: pgmap v492: 168 pgs: 168 active+clean; 2.1 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 774 B/s rd, 98 MiB/s wr, 1.57k op/s 2024-09-10T18:56:06.085 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:06.118 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:06.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:06.119 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:06.220 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:06.233 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-10T18:56:06.254 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:06.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:06.255 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:06.356 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:06.386 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 18 16 43119 43103 149.632 125.812 0.00959129 0.00667811 2024-09-10T18:56:06.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:06.408 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:06.408 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:06.509 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:06.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:06.541 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:06.541 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:06.642 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:06.676 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:06.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:06.677 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:06.738 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T18:56:06.780 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:06.830 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:06.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:06.831 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:06.932 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:06.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:06 smithi018 ceph-mon[40196]: 2.1a scrub starts 2024-09-10T18:56:06.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:06 smithi018 ceph-mon[40196]: 2.1a scrub ok 2024-09-10T18:56:06.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:06 smithi018 ceph-mon[40196]: 4.7 scrub starts 2024-09-10T18:56:06.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:06 smithi018 ceph-mon[40196]: 4.7 scrub ok 2024-09-10T18:56:06.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:06 smithi018 ceph-mon[29336]: 2.1a scrub starts 2024-09-10T18:56:06.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:06 smithi018 ceph-mon[29336]: 2.1a scrub ok 2024-09-10T18:56:06.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:06 smithi018 ceph-mon[29336]: 4.7 scrub starts 2024-09-10T18:56:06.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:06 smithi018 ceph-mon[29336]: 4.7 scrub ok 2024-09-10T18:56:06.960 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:06.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:06.961 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:07.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:06 smithi103 ceph-mon[33163]: 2.1a scrub starts 2024-09-10T18:56:07.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:06 smithi103 ceph-mon[33163]: 2.1a scrub ok 2024-09-10T18:56:07.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:06 smithi103 ceph-mon[33163]: 4.7 scrub starts 2024-09-10T18:56:07.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:06 smithi103 ceph-mon[33163]: 4.7 scrub ok 2024-09-10T18:56:07.023 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:56:06 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:56:07.061 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:07.107 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:07.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:07.108 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:07.208 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:07.248 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:07.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:07.249 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:07.350 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:07.386 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 19 16 44964 44948 147.825 115.312 0.0116325 0.00675977 2024-09-10T18:56:07.389 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:07.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:07.390 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:07.491 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:07.529 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:07.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:07.531 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:07.631 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:07.665 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:07.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:07.666 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:07.766 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:07.795 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:07.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:07.796 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:07.897 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:07.935 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:07.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:07.937 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:07.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:07 smithi018 ceph-mon[40196]: 2.1d scrub starts 2024-09-10T18:56:07.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:07 smithi018 ceph-mon[40196]: 2.1d scrub ok 2024-09-10T18:56:07.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:07 smithi018 ceph-mon[40196]: 3.1 scrub starts 2024-09-10T18:56:07.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:07 smithi018 ceph-mon[40196]: 3.1 scrub ok 2024-09-10T18:56:07.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:07 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:07.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:07 smithi018 ceph-mon[40196]: pgmap v493: 168 pgs: 168 active+clean; 2.6 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1.1 KiB/s rd, 143 MiB/s wr, 2.28k op/s 2024-09-10T18:56:07.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:07 smithi018 ceph-mon[29336]: 2.1d scrub starts 2024-09-10T18:56:07.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:07 smithi018 ceph-mon[29336]: 2.1d scrub ok 2024-09-10T18:56:07.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:07 smithi018 ceph-mon[29336]: 3.1 scrub starts 2024-09-10T18:56:07.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:07 smithi018 ceph-mon[29336]: 3.1 scrub ok 2024-09-10T18:56:07.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:07 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:07.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:07 smithi018 ceph-mon[29336]: pgmap v493: 168 pgs: 168 active+clean; 2.6 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1.1 KiB/s rd, 143 MiB/s wr, 2.28k op/s 2024-09-10T18:56:08.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:07 smithi103 ceph-mon[33163]: 2.1d scrub starts 2024-09-10T18:56:08.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:07 smithi103 ceph-mon[33163]: 2.1d scrub ok 2024-09-10T18:56:08.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:07 smithi103 ceph-mon[33163]: 3.1 scrub starts 2024-09-10T18:56:08.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:07 smithi103 ceph-mon[33163]: 3.1 scrub ok 2024-09-10T18:56:08.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:07 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:08.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:07 smithi103 ceph-mon[33163]: pgmap v493: 168 pgs: 168 active+clean; 2.6 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1.1 KiB/s rd, 143 MiB/s wr, 2.28k op/s 2024-09-10T18:56:08.038 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:08.086 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:08.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:08.087 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:08.188 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:08.219 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:08.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:08.220 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:08.321 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:08.355 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:08.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:08.356 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:08.386 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:2024-09-10T18:56:08.385717+0000 min lat: 0.00128005 max lat: 0.122347 avg lat: 0.00685294 2024-09-10T18:56:08.386 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T18:56:08.386 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 20 16 46684 46668 145.809 107.5 0.00733885 0.00685294 2024-09-10T18:56:08.456 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:08.491 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:08.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:08.492 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:08.593 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:08.628 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:08.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:08.629 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:08.730 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:08.779 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:08.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:08.780 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:08.881 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:08.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:08.924 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:08.925 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:08.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:08 smithi018 ceph-mon[29336]: 6.11 scrub starts 2024-09-10T18:56:08.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:08 smithi018 ceph-mon[29336]: 6.11 scrub ok 2024-09-10T18:56:08.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:08 smithi018 ceph-mon[29336]: 1.2 scrub starts 2024-09-10T18:56:08.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:08 smithi018 ceph-mon[29336]: 1.2 scrub ok 2024-09-10T18:56:08.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:08 smithi018 ceph-mon[29336]: 6.19 scrub starts 2024-09-10T18:56:08.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:08 smithi018 ceph-mon[29336]: 6.19 scrub ok 2024-09-10T18:56:08.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:08 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:56:08.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:08 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:56:08.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:08 smithi018 ceph-mon[29336]: 6.14 scrub starts 2024-09-10T18:56:08.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:08 smithi018 ceph-mon[29336]: osdmap e255: 8 total, 8 up, 7 in 2024-09-10T18:56:08.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:08 smithi018 ceph-mon[29336]: 6.14 scrub ok 2024-09-10T18:56:08.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:08 smithi018 ceph-mon[29336]: 5.15 scrub starts 2024-09-10T18:56:08.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:08 smithi018 ceph-mon[29336]: 5.15 scrub ok 2024-09-10T18:56:08.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:08 smithi018 ceph-mon[29336]: osdmap e256: 8 total, 8 up, 7 in 2024-09-10T18:56:08.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:08 smithi018 ceph-mon[40196]: 6.11 scrub starts 2024-09-10T18:56:08.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:08 smithi018 ceph-mon[40196]: 6.11 scrub ok 2024-09-10T18:56:08.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:08 smithi018 ceph-mon[40196]: 1.2 scrub starts 2024-09-10T18:56:08.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:08 smithi018 ceph-mon[40196]: 1.2 scrub ok 2024-09-10T18:56:08.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:08 smithi018 ceph-mon[40196]: 6.19 scrub starts 2024-09-10T18:56:08.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:08 smithi018 ceph-mon[40196]: 6.19 scrub ok 2024-09-10T18:56:08.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:08 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:56:08.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:08 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:56:08.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:08 smithi018 ceph-mon[40196]: 6.14 scrub starts 2024-09-10T18:56:08.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:08 smithi018 ceph-mon[40196]: osdmap e255: 8 total, 8 up, 7 in 2024-09-10T18:56:08.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:08 smithi018 ceph-mon[40196]: 6.14 scrub ok 2024-09-10T18:56:08.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:08 smithi018 ceph-mon[40196]: 5.15 scrub starts 2024-09-10T18:56:08.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:08 smithi018 ceph-mon[40196]: 5.15 scrub ok 2024-09-10T18:56:08.947 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:08 smithi018 ceph-mon[40196]: osdmap e256: 8 total, 8 up, 7 in 2024-09-10T18:56:09.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:08 smithi103 ceph-mon[33163]: 6.11 scrub starts 2024-09-10T18:56:09.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:08 smithi103 ceph-mon[33163]: 6.11 scrub ok 2024-09-10T18:56:09.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:08 smithi103 ceph-mon[33163]: 1.2 scrub starts 2024-09-10T18:56:09.024 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:08 smithi103 ceph-mon[33163]: 1.2 scrub ok 2024-09-10T18:56:09.024 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:08 smithi103 ceph-mon[33163]: 6.19 scrub starts 2024-09-10T18:56:09.024 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:08 smithi103 ceph-mon[33163]: 6.19 scrub ok 2024-09-10T18:56:09.024 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:08 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:56:09.024 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:08 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:56:09.024 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:08 smithi103 ceph-mon[33163]: 6.14 scrub starts 2024-09-10T18:56:09.024 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:08 smithi103 ceph-mon[33163]: osdmap e255: 8 total, 8 up, 7 in 2024-09-10T18:56:09.024 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:08 smithi103 ceph-mon[33163]: 6.14 scrub ok 2024-09-10T18:56:09.025 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:08 smithi103 ceph-mon[33163]: 5.15 scrub starts 2024-09-10T18:56:09.025 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:08 smithi103 ceph-mon[33163]: 5.15 scrub ok 2024-09-10T18:56:09.025 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:08 smithi103 ceph-mon[33163]: osdmap e256: 8 total, 8 up, 7 in 2024-09-10T18:56:09.025 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:09.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:09.061 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:09.061 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:09.161 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:09.198 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:09.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:09.199 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:09.299 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:09.323 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:09.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:09.324 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:09.386 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 21 16 48327 48311 143.755 102.688 0.00696469 0.00695091 2024-09-10T18:56:09.424 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:09.450 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:09.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:09.451 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:09.552 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:09.593 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:09.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:09.594 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:09.695 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:09.714 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T18:56:09.714 INFO:teuthology.orchestra.run.smithi018.stdout: "ops": [ 2024-09-10T18:56:09.714 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.714 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.15354.0:48455 9.0 e256/241 9:0ba4b5d2:::benchmark_data_smithi018_133594_object48454:head v 256'3025, mlcod=256'3022)", 2024-09-10T18:56:09.714 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:56:09.484738+0000", 2024-09-10T18:56:09.714 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 0.0086277809999999993, 2024-09-10T18:56:09.714 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.008657896, 2024-09-10T18:56:09.714 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:56:09.714 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "started", 2024-09-10T18:56:09.714 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:56:09.715 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.715 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:56:09.715 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.484738+0000", 2024-09-10T18:56:09.715 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:09.715 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.715 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.715 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:56:09.715 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.484738+0000", 2024-09-10T18:56:09.715 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:09.715 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.715 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.715 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:56:09.715 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.484737+0000", 2024-09-10T18:56:09.715 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:56:09.715 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.716 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.716 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:56:09.716 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.484763+0000", 2024-09-10T18:56:09.716 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.5867000000000002e-05 2024-09-10T18:56:09.716 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.716 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.716 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:56:09.716 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.484774+0000", 2024-09-10T18:56:09.716 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1323e-05 2024-09-10T18:56:09.716 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.716 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.716 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:56:09.716 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.484780+0000", 2024-09-10T18:56:09.717 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.0669999999999997e-06 2024-09-10T18:56:09.717 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.717 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.717 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:56:09.717 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.484893+0000", 2024-09-10T18:56:09.717 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00011226999999999999 2024-09-10T18:56:09.717 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.717 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.717 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:56:09.717 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.484914+0000", 2024-09-10T18:56:09.717 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.1027e-05 2024-09-10T18:56:09.717 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.718 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:09.718 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.718 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.718 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.718 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.15354.0:48462 9.8 e256/241 9:11af8aab:::benchmark_data_smithi018_133594_object48461:head v 256'3157, mlcod=256'3154)", 2024-09-10T18:56:09.718 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:56:09.489007+0000", 2024-09-10T18:56:09.718 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 0.0043584050000000001, 2024-09-10T18:56:09.718 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0044515450000000003, 2024-09-10T18:56:09.718 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:56:09.718 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "started", 2024-09-10T18:56:09.718 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:56:09.718 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.719 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:56:09.719 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.489007+0000", 2024-09-10T18:56:09.719 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:09.719 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.719 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.719 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:56:09.719 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.489007+0000", 2024-09-10T18:56:09.719 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:09.719 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.719 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.719 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:56:09.719 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.489006+0000", 2024-09-10T18:56:09.720 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:56:09.720 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.720 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.720 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:56:09.720 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.489035+0000", 2024-09-10T18:56:09.720 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.8534000000000002e-05 2024-09-10T18:56:09.720 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.720 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.720 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:56:09.720 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.489052+0000", 2024-09-10T18:56:09.720 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.7414e-05 2024-09-10T18:56:09.720 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.720 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.721 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:56:09.721 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.489058+0000", 2024-09-10T18:56:09.721 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.3910000000000003e-06 2024-09-10T18:56:09.721 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.721 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.721 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:56:09.721 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.489133+0000", 2024-09-10T18:56:09.721 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.4271999999999996e-05 2024-09-10T18:56:09.721 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.722 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.722 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:56:09.722 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.489153+0000", 2024-09-10T18:56:09.722 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.0656999999999999e-05 2024-09-10T18:56:09.722 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.722 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:09.722 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.722 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.722 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.722 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15354.0:48469 9.5 9:a3f82311:::benchmark_data_smithi018_133594_object48468:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e256)", 2024-09-10T18:56:09.722 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:56:09.490892+0000", 2024-09-10T18:56:09.722 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 0.0024734710000000001, 2024-09-10T18:56:09.722 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0026292199999999998, 2024-09-10T18:56:09.722 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:56:09.723 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "waiting for sub ops", 2024-09-10T18:56:09.723 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:56:09.723 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15354", 2024-09-10T18:56:09.723 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T18:56:09.723 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 48469 2024-09-10T18:56:09.723 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.723 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:56:09.723 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.723 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:56:09.723 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.490892+0000", 2024-09-10T18:56:09.723 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:09.723 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.723 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.723 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:56:09.723 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.490892+0000", 2024-09-10T18:56:09.724 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:09.724 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.724 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.724 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:56:09.724 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.490890+0000", 2024-09-10T18:56:09.724 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T18:56:09.724 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.724 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.724 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:56:09.724 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.490915+0000", 2024-09-10T18:56:09.724 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.4635999999999999e-05 2024-09-10T18:56:09.724 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.724 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.724 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:56:09.725 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.490929+0000", 2024-09-10T18:56:09.725 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.3805000000000001e-05 2024-09-10T18:56:09.725 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.725 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.725 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:56:09.725 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.490939+0000", 2024-09-10T18:56:09.725 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.6690000000000001e-06 2024-09-10T18:56:09.725 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.725 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.725 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:56:09.725 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.491001+0000", 2024-09-10T18:56:09.725 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.2854000000000005e-05 2024-09-10T18:56:09.725 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.725 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.726 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:56:09.726 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.491112+0000", 2024-09-10T18:56:09.726 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000111019 2024-09-10T18:56:09.726 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.726 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.726 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,7", 2024-09-10T18:56:09.726 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.491266+0000", 2024-09-10T18:56:09.726 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00015397800000000001 2024-09-10T18:56:09.726 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.726 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:09.726 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.726 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.726 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.726 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15354.0:48470 9.5 9:a4cc83e2:::benchmark_data_smithi018_133594_object48469:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e256)", 2024-09-10T18:56:09.726 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:56:09.490956+0000", 2024-09-10T18:56:09.727 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 0.0024096880000000001, 2024-09-10T18:56:09.727 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0026373590000000001, 2024-09-10T18:56:09.727 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:56:09.727 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "waiting for sub ops", 2024-09-10T18:56:09.727 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:56:09.727 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15354", 2024-09-10T18:56:09.727 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T18:56:09.727 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 48470 2024-09-10T18:56:09.727 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.727 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:56:09.727 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.727 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:56:09.727 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.490956+0000", 2024-09-10T18:56:09.727 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:09.727 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.727 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.728 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:56:09.728 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.490956+0000", 2024-09-10T18:56:09.728 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:09.728 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.728 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.728 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:56:09.728 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.490955+0000", 2024-09-10T18:56:09.728 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T18:56:09.728 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.728 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.728 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:56:09.728 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.490988+0000", 2024-09-10T18:56:09.729 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.2790999999999998e-05 2024-09-10T18:56:09.729 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.729 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.729 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:56:09.729 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.490996+0000", 2024-09-10T18:56:09.729 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.1359999999999997e-06 2024-09-10T18:56:09.729 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.729 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.729 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:56:09.729 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.491000+0000", 2024-09-10T18:56:09.729 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.2289999999999997e-06 2024-09-10T18:56:09.729 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.729 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.730 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:56:09.730 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.492950+0000", 2024-09-10T18:56:09.730 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.001949159 2024-09-10T18:56:09.730 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.730 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.730 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:56:09.730 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.493120+0000", 2024-09-10T18:56:09.730 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000170624 2024-09-10T18:56:09.730 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.730 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.730 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,7", 2024-09-10T18:56:09.730 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.493284+0000", 2024-09-10T18:56:09.730 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000164203 2024-09-10T18:56:09.730 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.730 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:09.731 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.731 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.731 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.731 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.15354.0:48463 9.0 e256/241 9:0932a600:::benchmark_data_smithi018_133594_object48462:head v 256'3026, mlcod=256'3022)", 2024-09-10T18:56:09.731 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:56:09.490965+0000", 2024-09-10T18:56:09.731 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 0.0024003779999999999, 2024-09-10T18:56:09.731 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0026958059999999998, 2024-09-10T18:56:09.731 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:56:09.731 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "started", 2024-09-10T18:56:09.731 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:56:09.731 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.731 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:56:09.731 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.490965+0000", 2024-09-10T18:56:09.732 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:09.732 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.732 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.732 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:56:09.732 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.490965+0000", 2024-09-10T18:56:09.732 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:09.732 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.732 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.732 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:56:09.732 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.490964+0000", 2024-09-10T18:56:09.732 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:56:09.732 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.732 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.733 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:56:09.733 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.491085+0000", 2024-09-10T18:56:09.733 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000120839 2024-09-10T18:56:09.733 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.733 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.733 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:56:09.733 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.491097+0000", 2024-09-10T18:56:09.733 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1626e-05 2024-09-10T18:56:09.733 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.733 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.733 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:56:09.734 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.491103+0000", 2024-09-10T18:56:09.734 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.1290000000000001e-06 2024-09-10T18:56:09.734 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.734 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.734 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:56:09.734 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.492149+0000", 2024-09-10T18:56:09.734 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.001046209 2024-09-10T18:56:09.734 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.734 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.734 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:56:09.734 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.492201+0000", 2024-09-10T18:56:09.734 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.1972000000000003e-05 2024-09-10T18:56:09.734 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.734 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:09.735 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.735 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.735 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.735 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop_reply(client.15354.0:48469 9.5 e256/241)", 2024-09-10T18:56:09.735 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:56:09.493322+0000", 2024-09-10T18:56:09.735 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 4.3894999999999999e-05, 2024-09-10T18:56:09.735 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00039001500000000001, 2024-09-10T18:56:09.735 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:56:09.735 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "queued for pg", 2024-09-10T18:56:09.735 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:56:09.735 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.735 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:56:09.736 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.493322+0000", 2024-09-10T18:56:09.736 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:09.736 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.736 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.736 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:56:09.736 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.493322+0000", 2024-09-10T18:56:09.736 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:09.736 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.736 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.736 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:56:09.737 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.493321+0000", 2024-09-10T18:56:09.737 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:56:09.737 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.737 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.737 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:56:09.737 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.493326+0000", 2024-09-10T18:56:09.737 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.361e-06 2024-09-10T18:56:09.737 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.737 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.737 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:56:09.737 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.493348+0000", 2024-09-10T18:56:09.737 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.1478e-05 2024-09-10T18:56:09.738 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.738 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.738 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:56:09.738 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:09.493353+0000", 2024-09-10T18:56:09.738 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.3900000000000001e-06 2024-09-10T18:56:09.738 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.738 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:09.738 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.738 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.738 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-09-10T18:56:09.738 INFO:teuthology.orchestra.run.smithi018.stdout: "num_ops": 6 2024-09-10T18:56:09.738 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T18:56:09.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:09.740 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:09.740 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:09.841 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:09.881 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:09.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:09.882 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:09.925 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:09 smithi018 ceph-mon[40196]: pgmap v496: 168 pgs: 168 active+clean; 2.7 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1.1 KiB/s rd, 157 MiB/s wr, 2.51k op/s 2024-09-10T18:56:09.925 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:09 smithi018 ceph-mon[29336]: pgmap v496: 168 pgs: 168 active+clean; 2.7 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1.1 KiB/s rd, 157 MiB/s wr, 2.51k op/s 2024-09-10T18:56:09.926 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:56:09.927 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:56:09.927 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T18:56:09.927 INFO:teuthology.orchestra.run.smithi018.stdout: "epoch": 256, 2024-09-10T18:56:09.927 INFO:teuthology.orchestra.run.smithi018.stdout: "fsid": "ba327f6c-6fa3-11ef-bcea-c7b262605968", 2024-09-10T18:56:09.927 INFO:teuthology.orchestra.run.smithi018.stdout: "created": "2024-09-10T18:40:07.669687+0000", 2024-09-10T18:56:09.927 INFO:teuthology.orchestra.run.smithi018.stdout: "modified": "2024-09-10T18:56:08.618936+0000", 2024-09-10T18:56:09.927 INFO:teuthology.orchestra.run.smithi018.stdout: "last_up_change": "2024-09-10T18:55:39.357104+0000", 2024-09-10T18:56:09.927 INFO:teuthology.orchestra.run.smithi018.stdout: "last_in_change": "2024-09-10T18:47:35.385915+0000", 2024-09-10T18:56:09.927 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-10T18:56:09.928 INFO:teuthology.orchestra.run.smithi018.stdout: "flags_num": 5804032, 2024-09-10T18:56:09.928 INFO:teuthology.orchestra.run.smithi018.stdout: "flags_set": [ 2024-09-10T18:56:09.928 INFO:teuthology.orchestra.run.smithi018.stdout: "nodeep-scrub", 2024-09-10T18:56:09.928 INFO:teuthology.orchestra.run.smithi018.stdout: "pglog_hardlimit", 2024-09-10T18:56:09.928 INFO:teuthology.orchestra.run.smithi018.stdout: "purged_snapdirs", 2024-09-10T18:56:09.928 INFO:teuthology.orchestra.run.smithi018.stdout: "recovery_deletes", 2024-09-10T18:56:09.928 INFO:teuthology.orchestra.run.smithi018.stdout: "sortbitwise" 2024-09-10T18:56:09.928 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-09-10T18:56:09.928 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_version": 19, 2024-09-10T18:56:09.928 INFO:teuthology.orchestra.run.smithi018.stdout: "full_ratio": 0.94999998807907104, 2024-09-10T18:56:09.928 INFO:teuthology.orchestra.run.smithi018.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-10T18:56:09.928 INFO:teuthology.orchestra.run.smithi018.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-10T18:56:09.928 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_snapshot": "", 2024-09-10T18:56:09.929 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_max": 9, 2024-09-10T18:56:09.929 INFO:teuthology.orchestra.run.smithi018.stdout: "max_osd": 8, 2024-09-10T18:56:09.929 INFO:teuthology.orchestra.run.smithi018.stdout: "require_min_compat_client": "luminous", 2024-09-10T18:56:09.929 INFO:teuthology.orchestra.run.smithi018.stdout: "min_compat_client": "luminous", 2024-09-10T18:56:09.929 INFO:teuthology.orchestra.run.smithi018.stdout: "require_osd_release": "quincy", 2024-09-10T18:56:09.929 INFO:teuthology.orchestra.run.smithi018.stdout: "pools": [ 2024-09-10T18:56:09.929 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.929 INFO:teuthology.orchestra.run.smithi018.stdout: "pool": 1, 2024-09-10T18:56:09.929 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_name": ".mgr", 2024-09-10T18:56:09.929 INFO:teuthology.orchestra.run.smithi018.stdout: "create_time": "2024-09-10T18:43:08.162659+0000", 2024-09-10T18:56:09.929 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-09-10T18:56:09.929 INFO:teuthology.orchestra.run.smithi018.stdout: "flags_names": "hashpspool", 2024-09-10T18:56:09.930 INFO:teuthology.orchestra.run.smithi018.stdout: "type": 1, 2024-09-10T18:56:09.930 INFO:teuthology.orchestra.run.smithi018.stdout: "size": 3, 2024-09-10T18:56:09.930 INFO:teuthology.orchestra.run.smithi018.stdout: "min_size": 2, 2024-09-10T18:56:09.930 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_rule": 0, 2024-09-10T18:56:09.930 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_count": 0, 2024-09-10T18:56:09.930 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_target": 0, 2024-09-10T18:56:09.930 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T18:56:09.930 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T18:56:09.930 INFO:teuthology.orchestra.run.smithi018.stdout: "object_hash": 2, 2024-09-10T18:56:09.930 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscale_mode": "off", 2024-09-10T18:56:09.930 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num": 21, 2024-09-10T18:56:09.930 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num": 21, 2024-09-10T18:56:09.930 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num_target": 21, 2024-09-10T18:56:09.930 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_target": 21, 2024-09-10T18:56:09.931 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_pending": 21, 2024-09-10T18:56:09.931 INFO:teuthology.orchestra.run.smithi018.stdout: "last_pg_merge_meta": { 2024-09-10T18:56:09.931 INFO:teuthology.orchestra.run.smithi018.stdout: "source_pgid": "0.0", 2024-09-10T18:56:09.931 INFO:teuthology.orchestra.run.smithi018.stdout: "ready_epoch": 0, 2024-09-10T18:56:09.931 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_started": 0, 2024-09-10T18:56:09.931 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_clean": 0, 2024-09-10T18:56:09.931 INFO:teuthology.orchestra.run.smithi018.stdout: "source_version": "0'0", 2024-09-10T18:56:09.931 INFO:teuthology.orchestra.run.smithi018.stdout: "target_version": "0'0" 2024-09-10T18:56:09.931 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.931 INFO:teuthology.orchestra.run.smithi018.stdout: "last_change": "169", 2024-09-10T18:56:09.931 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend": "0", 2024-09-10T18:56:09.931 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T18:56:09.931 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_preluminous": "167", 2024-09-10T18:56:09.932 INFO:teuthology.orchestra.run.smithi018.stdout: "auid": 0, 2024-09-10T18:56:09.932 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_mode": "selfmanaged", 2024-09-10T18:56:09.932 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_seq": 0, 2024-09-10T18:56:09.932 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_epoch": 0, 2024-09-10T18:56:09.932 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_snaps": [], 2024-09-10T18:56:09.932 INFO:teuthology.orchestra.run.smithi018.stdout: "removed_snaps": "[]", 2024-09-10T18:56:09.932 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_bytes": 0, 2024-09-10T18:56:09.932 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_objects": 0, 2024-09-10T18:56:09.932 INFO:teuthology.orchestra.run.smithi018.stdout: "tiers": [], 2024-09-10T18:56:09.932 INFO:teuthology.orchestra.run.smithi018.stdout: "tier_of": -1, 2024-09-10T18:56:09.932 INFO:teuthology.orchestra.run.smithi018.stdout: "read_tier": -1, 2024-09-10T18:56:09.932 INFO:teuthology.orchestra.run.smithi018.stdout: "write_tier": -1, 2024-09-10T18:56:09.933 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_mode": "none", 2024-09-10T18:56:09.933 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_bytes": 0, 2024-09-10T18:56:09.933 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_objects": 0, 2024-09-10T18:56:09.933 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T18:56:09.933 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T18:56:09.933 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T18:56:09.933 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_flush_age": 0, 2024-09-10T18:56:09.933 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_evict_age": 0, 2024-09-10T18:56:09.933 INFO:teuthology.orchestra.run.smithi018.stdout: "erasure_code_profile": "", 2024-09-10T18:56:09.933 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_params": { 2024-09-10T18:56:09.933 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "none" 2024-09-10T18:56:09.933 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.934 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_period": 0, 2024-09-10T18:56:09.934 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_count": 0, 2024-09-10T18:56:09.934 INFO:teuthology.orchestra.run.smithi018.stdout: "use_gmt_hitset": true, 2024-09-10T18:56:09.934 INFO:teuthology.orchestra.run.smithi018.stdout: "min_read_recency_for_promote": 0, 2024-09-10T18:56:09.934 INFO:teuthology.orchestra.run.smithi018.stdout: "min_write_recency_for_promote": 0, 2024-09-10T18:56:09.934 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T18:56:09.934 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_search_last_n": 0, 2024-09-10T18:56:09.934 INFO:teuthology.orchestra.run.smithi018.stdout: "grade_table": [], 2024-09-10T18:56:09.934 INFO:teuthology.orchestra.run.smithi018.stdout: "stripe_width": 0, 2024-09-10T18:56:09.934 INFO:teuthology.orchestra.run.smithi018.stdout: "expected_num_objects": 0, 2024-09-10T18:56:09.934 INFO:teuthology.orchestra.run.smithi018.stdout: "fast_read": false, 2024-09-10T18:56:09.934 INFO:teuthology.orchestra.run.smithi018.stdout: "options": { 2024-09-10T18:56:09.935 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_max": 32, 2024-09-10T18:56:09.935 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_min": 1 2024-09-10T18:56:09.935 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.935 INFO:teuthology.orchestra.run.smithi018.stdout: "application_metadata": { 2024-09-10T18:56:09.935 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": {} 2024-09-10T18:56:09.935 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.935 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.935 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.935 INFO:teuthology.orchestra.run.smithi018.stdout: "pool": 2, 2024-09-10T18:56:09.935 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_name": ".rgw.root", 2024-09-10T18:56:09.935 INFO:teuthology.orchestra.run.smithi018.stdout: "create_time": "2024-09-10T18:45:16.512219+0000", 2024-09-10T18:56:09.936 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-09-10T18:56:09.936 INFO:teuthology.orchestra.run.smithi018.stdout: "flags_names": "hashpspool", 2024-09-10T18:56:09.936 INFO:teuthology.orchestra.run.smithi018.stdout: "type": 1, 2024-09-10T18:56:09.936 INFO:teuthology.orchestra.run.smithi018.stdout: "size": 3, 2024-09-10T18:56:09.936 INFO:teuthology.orchestra.run.smithi018.stdout: "min_size": 2, 2024-09-10T18:56:09.936 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_rule": 0, 2024-09-10T18:56:09.936 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_count": 0, 2024-09-10T18:56:09.936 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_target": 0, 2024-09-10T18:56:09.936 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T18:56:09.936 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T18:56:09.936 INFO:teuthology.orchestra.run.smithi018.stdout: "object_hash": 2, 2024-09-10T18:56:09.936 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscale_mode": "off", 2024-09-10T18:56:09.936 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num": 32, 2024-09-10T18:56:09.937 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num": 32, 2024-09-10T18:56:09.937 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num_target": 32, 2024-09-10T18:56:09.937 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_target": 32, 2024-09-10T18:56:09.937 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_pending": 32, 2024-09-10T18:56:09.937 INFO:teuthology.orchestra.run.smithi018.stdout: "last_pg_merge_meta": { 2024-09-10T18:56:09.937 INFO:teuthology.orchestra.run.smithi018.stdout: "source_pgid": "0.0", 2024-09-10T18:56:09.937 INFO:teuthology.orchestra.run.smithi018.stdout: "ready_epoch": 0, 2024-09-10T18:56:09.937 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_started": 0, 2024-09-10T18:56:09.938 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_clean": 0, 2024-09-10T18:56:09.938 INFO:teuthology.orchestra.run.smithi018.stdout: "source_version": "0'0", 2024-09-10T18:56:09.938 INFO:teuthology.orchestra.run.smithi018.stdout: "target_version": "0'0" 2024-09-10T18:56:09.938 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.938 INFO:teuthology.orchestra.run.smithi018.stdout: "last_change": "57", 2024-09-10T18:56:09.938 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend": "0", 2024-09-10T18:56:09.938 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T18:56:09.938 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T18:56:09.938 INFO:teuthology.orchestra.run.smithi018.stdout: "auid": 0, 2024-09-10T18:56:09.938 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_mode": "selfmanaged", 2024-09-10T18:56:09.938 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_seq": 0, 2024-09-10T18:56:09.938 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_epoch": 0, 2024-09-10T18:56:09.938 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_snaps": [], 2024-09-10T18:56:09.938 INFO:teuthology.orchestra.run.smithi018.stdout: "removed_snaps": "[]", 2024-09-10T18:56:09.938 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_bytes": 0, 2024-09-10T18:56:09.939 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_objects": 0, 2024-09-10T18:56:09.939 INFO:teuthology.orchestra.run.smithi018.stdout: "tiers": [], 2024-09-10T18:56:09.939 INFO:teuthology.orchestra.run.smithi018.stdout: "tier_of": -1, 2024-09-10T18:56:09.939 INFO:teuthology.orchestra.run.smithi018.stdout: "read_tier": -1, 2024-09-10T18:56:09.939 INFO:teuthology.orchestra.run.smithi018.stdout: "write_tier": -1, 2024-09-10T18:56:09.939 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_mode": "none", 2024-09-10T18:56:09.939 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_bytes": 0, 2024-09-10T18:56:09.939 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_objects": 0, 2024-09-10T18:56:09.939 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T18:56:09.939 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T18:56:09.939 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T18:56:09.939 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_flush_age": 0, 2024-09-10T18:56:09.939 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_evict_age": 0, 2024-09-10T18:56:09.940 INFO:teuthology.orchestra.run.smithi018.stdout: "erasure_code_profile": "", 2024-09-10T18:56:09.940 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_params": { 2024-09-10T18:56:09.940 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "none" 2024-09-10T18:56:09.940 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.940 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_period": 0, 2024-09-10T18:56:09.940 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_count": 0, 2024-09-10T18:56:09.940 INFO:teuthology.orchestra.run.smithi018.stdout: "use_gmt_hitset": true, 2024-09-10T18:56:09.940 INFO:teuthology.orchestra.run.smithi018.stdout: "min_read_recency_for_promote": 0, 2024-09-10T18:56:09.940 INFO:teuthology.orchestra.run.smithi018.stdout: "min_write_recency_for_promote": 0, 2024-09-10T18:56:09.940 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T18:56:09.940 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_search_last_n": 0, 2024-09-10T18:56:09.940 INFO:teuthology.orchestra.run.smithi018.stdout: "grade_table": [], 2024-09-10T18:56:09.941 INFO:teuthology.orchestra.run.smithi018.stdout: "stripe_width": 0, 2024-09-10T18:56:09.941 INFO:teuthology.orchestra.run.smithi018.stdout: "expected_num_objects": 0, 2024-09-10T18:56:09.941 INFO:teuthology.orchestra.run.smithi018.stdout: "fast_read": false, 2024-09-10T18:56:09.941 INFO:teuthology.orchestra.run.smithi018.stdout: "options": {}, 2024-09-10T18:56:09.941 INFO:teuthology.orchestra.run.smithi018.stdout: "application_metadata": { 2024-09-10T18:56:09.941 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw": {} 2024-09-10T18:56:09.941 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.941 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.941 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.941 INFO:teuthology.orchestra.run.smithi018.stdout: "pool": 3, 2024-09-10T18:56:09.941 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_name": "datapool", 2024-09-10T18:56:09.941 INFO:teuthology.orchestra.run.smithi018.stdout: "create_time": "2024-09-10T18:45:16.943626+0000", 2024-09-10T18:56:09.941 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 8193, 2024-09-10T18:56:09.941 INFO:teuthology.orchestra.run.smithi018.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-10T18:56:09.942 INFO:teuthology.orchestra.run.smithi018.stdout: "type": 1, 2024-09-10T18:56:09.942 INFO:teuthology.orchestra.run.smithi018.stdout: "size": 3, 2024-09-10T18:56:09.942 INFO:teuthology.orchestra.run.smithi018.stdout: "min_size": 2, 2024-09-10T18:56:09.942 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_rule": 0, 2024-09-10T18:56:09.942 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_count": 0, 2024-09-10T18:56:09.942 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_target": 0, 2024-09-10T18:56:09.942 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T18:56:09.942 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T18:56:09.942 INFO:teuthology.orchestra.run.smithi018.stdout: "object_hash": 2, 2024-09-10T18:56:09.942 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscale_mode": "off", 2024-09-10T18:56:09.942 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num": 3, 2024-09-10T18:56:09.942 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num": 3, 2024-09-10T18:56:09.942 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num_target": 3, 2024-09-10T18:56:09.943 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_target": 3, 2024-09-10T18:56:09.943 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_pending": 3, 2024-09-10T18:56:09.943 INFO:teuthology.orchestra.run.smithi018.stdout: "last_pg_merge_meta": { 2024-09-10T18:56:09.943 INFO:teuthology.orchestra.run.smithi018.stdout: "source_pgid": "0.0", 2024-09-10T18:56:09.943 INFO:teuthology.orchestra.run.smithi018.stdout: "ready_epoch": 0, 2024-09-10T18:56:09.943 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_started": 0, 2024-09-10T18:56:09.943 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_clean": 0, 2024-09-10T18:56:09.943 INFO:teuthology.orchestra.run.smithi018.stdout: "source_version": "0'0", 2024-09-10T18:56:09.943 INFO:teuthology.orchestra.run.smithi018.stdout: "target_version": "0'0" 2024-09-10T18:56:09.943 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.943 INFO:teuthology.orchestra.run.smithi018.stdout: "last_change": "60", 2024-09-10T18:56:09.943 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend": "0", 2024-09-10T18:56:09.943 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T18:56:09.943 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T18:56:09.944 INFO:teuthology.orchestra.run.smithi018.stdout: "auid": 0, 2024-09-10T18:56:09.944 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_mode": "selfmanaged", 2024-09-10T18:56:09.944 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_seq": 3, 2024-09-10T18:56:09.944 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_epoch": 60, 2024-09-10T18:56:09.944 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_snaps": [], 2024-09-10T18:56:09.944 INFO:teuthology.orchestra.run.smithi018.stdout: "removed_snaps": "[]", 2024-09-10T18:56:09.944 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_bytes": 0, 2024-09-10T18:56:09.944 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_objects": 0, 2024-09-10T18:56:09.944 INFO:teuthology.orchestra.run.smithi018.stdout: "tiers": [], 2024-09-10T18:56:09.944 INFO:teuthology.orchestra.run.smithi018.stdout: "tier_of": -1, 2024-09-10T18:56:09.944 INFO:teuthology.orchestra.run.smithi018.stdout: "read_tier": -1, 2024-09-10T18:56:09.944 INFO:teuthology.orchestra.run.smithi018.stdout: "write_tier": -1, 2024-09-10T18:56:09.944 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_mode": "none", 2024-09-10T18:56:09.944 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_bytes": 0, 2024-09-10T18:56:09.945 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_objects": 0, 2024-09-10T18:56:09.945 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T18:56:09.945 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T18:56:09.945 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T18:56:09.945 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_flush_age": 0, 2024-09-10T18:56:09.945 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_evict_age": 0, 2024-09-10T18:56:09.945 INFO:teuthology.orchestra.run.smithi018.stdout: "erasure_code_profile": "", 2024-09-10T18:56:09.945 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_params": { 2024-09-10T18:56:09.945 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "none" 2024-09-10T18:56:09.945 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.945 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_period": 0, 2024-09-10T18:56:09.945 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_count": 0, 2024-09-10T18:56:09.945 INFO:teuthology.orchestra.run.smithi018.stdout: "use_gmt_hitset": true, 2024-09-10T18:56:09.946 INFO:teuthology.orchestra.run.smithi018.stdout: "min_read_recency_for_promote": 0, 2024-09-10T18:56:09.946 INFO:teuthology.orchestra.run.smithi018.stdout: "min_write_recency_for_promote": 0, 2024-09-10T18:56:09.946 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T18:56:09.946 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_search_last_n": 0, 2024-09-10T18:56:09.946 INFO:teuthology.orchestra.run.smithi018.stdout: "grade_table": [], 2024-09-10T18:56:09.946 INFO:teuthology.orchestra.run.smithi018.stdout: "stripe_width": 0, 2024-09-10T18:56:09.946 INFO:teuthology.orchestra.run.smithi018.stdout: "expected_num_objects": 0, 2024-09-10T18:56:09.946 INFO:teuthology.orchestra.run.smithi018.stdout: "fast_read": false, 2024-09-10T18:56:09.946 INFO:teuthology.orchestra.run.smithi018.stdout: "options": {}, 2024-09-10T18:56:09.946 INFO:teuthology.orchestra.run.smithi018.stdout: "application_metadata": { 2024-09-10T18:56:09.946 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd": {} 2024-09-10T18:56:09.946 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.946 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.946 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.946 INFO:teuthology.orchestra.run.smithi018.stdout: "pool": 4, 2024-09-10T18:56:09.947 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_name": "default.rgw.log", 2024-09-10T18:56:09.947 INFO:teuthology.orchestra.run.smithi018.stdout: "create_time": "2024-09-10T18:45:18.450026+0000", 2024-09-10T18:56:09.947 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-09-10T18:56:09.947 INFO:teuthology.orchestra.run.smithi018.stdout: "flags_names": "hashpspool", 2024-09-10T18:56:09.947 INFO:teuthology.orchestra.run.smithi018.stdout: "type": 1, 2024-09-10T18:56:09.947 INFO:teuthology.orchestra.run.smithi018.stdout: "size": 3, 2024-09-10T18:56:09.947 INFO:teuthology.orchestra.run.smithi018.stdout: "min_size": 2, 2024-09-10T18:56:09.947 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_rule": 0, 2024-09-10T18:56:09.947 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_count": 0, 2024-09-10T18:56:09.947 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_target": 0, 2024-09-10T18:56:09.947 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T18:56:09.947 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T18:56:09.947 INFO:teuthology.orchestra.run.smithi018.stdout: "object_hash": 2, 2024-09-10T18:56:09.948 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscale_mode": "off", 2024-09-10T18:56:09.948 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num": 32, 2024-09-10T18:56:09.948 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num": 32, 2024-09-10T18:56:09.948 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num_target": 32, 2024-09-10T18:56:09.948 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_target": 32, 2024-09-10T18:56:09.948 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_pending": 32, 2024-09-10T18:56:09.948 INFO:teuthology.orchestra.run.smithi018.stdout: "last_pg_merge_meta": { 2024-09-10T18:56:09.948 INFO:teuthology.orchestra.run.smithi018.stdout: "source_pgid": "0.0", 2024-09-10T18:56:09.948 INFO:teuthology.orchestra.run.smithi018.stdout: "ready_epoch": 0, 2024-09-10T18:56:09.948 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_started": 0, 2024-09-10T18:56:09.948 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_clean": 0, 2024-09-10T18:56:09.948 INFO:teuthology.orchestra.run.smithi018.stdout: "source_version": "0'0", 2024-09-10T18:56:09.948 INFO:teuthology.orchestra.run.smithi018.stdout: "target_version": "0'0" 2024-09-10T18:56:09.948 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.949 INFO:teuthology.orchestra.run.smithi018.stdout: "last_change": "59", 2024-09-10T18:56:09.949 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend": "0", 2024-09-10T18:56:09.949 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T18:56:09.949 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T18:56:09.949 INFO:teuthology.orchestra.run.smithi018.stdout: "auid": 0, 2024-09-10T18:56:09.949 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_mode": "selfmanaged", 2024-09-10T18:56:09.949 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_seq": 0, 2024-09-10T18:56:09.949 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_epoch": 0, 2024-09-10T18:56:09.949 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_snaps": [], 2024-09-10T18:56:09.949 INFO:teuthology.orchestra.run.smithi018.stdout: "removed_snaps": "[]", 2024-09-10T18:56:09.949 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_bytes": 0, 2024-09-10T18:56:09.949 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_objects": 0, 2024-09-10T18:56:09.950 INFO:teuthology.orchestra.run.smithi018.stdout: "tiers": [], 2024-09-10T18:56:09.950 INFO:teuthology.orchestra.run.smithi018.stdout: "tier_of": -1, 2024-09-10T18:56:09.950 INFO:teuthology.orchestra.run.smithi018.stdout: "read_tier": -1, 2024-09-10T18:56:09.950 INFO:teuthology.orchestra.run.smithi018.stdout: "write_tier": -1, 2024-09-10T18:56:09.950 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_mode": "none", 2024-09-10T18:56:09.950 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_bytes": 0, 2024-09-10T18:56:09.950 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_objects": 0, 2024-09-10T18:56:09.950 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T18:56:09.950 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T18:56:09.950 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T18:56:09.950 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_flush_age": 0, 2024-09-10T18:56:09.950 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_evict_age": 0, 2024-09-10T18:56:09.951 INFO:teuthology.orchestra.run.smithi018.stdout: "erasure_code_profile": "", 2024-09-10T18:56:09.951 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_params": { 2024-09-10T18:56:09.951 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "none" 2024-09-10T18:56:09.951 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.951 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_period": 0, 2024-09-10T18:56:09.951 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_count": 0, 2024-09-10T18:56:09.951 INFO:teuthology.orchestra.run.smithi018.stdout: "use_gmt_hitset": true, 2024-09-10T18:56:09.951 INFO:teuthology.orchestra.run.smithi018.stdout: "min_read_recency_for_promote": 0, 2024-09-10T18:56:09.951 INFO:teuthology.orchestra.run.smithi018.stdout: "min_write_recency_for_promote": 0, 2024-09-10T18:56:09.951 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T18:56:09.951 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_search_last_n": 0, 2024-09-10T18:56:09.951 INFO:teuthology.orchestra.run.smithi018.stdout: "grade_table": [], 2024-09-10T18:56:09.952 INFO:teuthology.orchestra.run.smithi018.stdout: "stripe_width": 0, 2024-09-10T18:56:09.952 INFO:teuthology.orchestra.run.smithi018.stdout: "expected_num_objects": 0, 2024-09-10T18:56:09.952 INFO:teuthology.orchestra.run.smithi018.stdout: "fast_read": false, 2024-09-10T18:56:09.952 INFO:teuthology.orchestra.run.smithi018.stdout: "options": {}, 2024-09-10T18:56:09.952 INFO:teuthology.orchestra.run.smithi018.stdout: "application_metadata": { 2024-09-10T18:56:09.952 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw": {} 2024-09-10T18:56:09.952 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.952 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.952 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.952 INFO:teuthology.orchestra.run.smithi018.stdout: "pool": 5, 2024-09-10T18:56:09.952 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_name": "default.rgw.control", 2024-09-10T18:56:09.953 INFO:teuthology.orchestra.run.smithi018.stdout: "create_time": "2024-09-10T18:45:20.494618+0000", 2024-09-10T18:56:09.953 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-09-10T18:56:09.953 INFO:teuthology.orchestra.run.smithi018.stdout: "flags_names": "hashpspool", 2024-09-10T18:56:09.953 INFO:teuthology.orchestra.run.smithi018.stdout: "type": 1, 2024-09-10T18:56:09.953 INFO:teuthology.orchestra.run.smithi018.stdout: "size": 3, 2024-09-10T18:56:09.953 INFO:teuthology.orchestra.run.smithi018.stdout: "min_size": 2, 2024-09-10T18:56:09.953 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_rule": 0, 2024-09-10T18:56:09.953 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_count": 0, 2024-09-10T18:56:09.954 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_target": 0, 2024-09-10T18:56:09.954 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T18:56:09.954 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T18:56:09.954 INFO:teuthology.orchestra.run.smithi018.stdout: "object_hash": 2, 2024-09-10T18:56:09.954 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscale_mode": "off", 2024-09-10T18:56:09.954 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num": 32, 2024-09-10T18:56:09.954 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num": 32, 2024-09-10T18:56:09.954 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num_target": 32, 2024-09-10T18:56:09.954 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_target": 32, 2024-09-10T18:56:09.954 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_pending": 32, 2024-09-10T18:56:09.954 INFO:teuthology.orchestra.run.smithi018.stdout: "last_pg_merge_meta": { 2024-09-10T18:56:09.954 INFO:teuthology.orchestra.run.smithi018.stdout: "source_pgid": "0.0", 2024-09-10T18:56:09.955 INFO:teuthology.orchestra.run.smithi018.stdout: "ready_epoch": 0, 2024-09-10T18:56:09.955 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_started": 0, 2024-09-10T18:56:09.955 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_clean": 0, 2024-09-10T18:56:09.955 INFO:teuthology.orchestra.run.smithi018.stdout: "source_version": "0'0", 2024-09-10T18:56:09.955 INFO:teuthology.orchestra.run.smithi018.stdout: "target_version": "0'0" 2024-09-10T18:56:09.955 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.955 INFO:teuthology.orchestra.run.smithi018.stdout: "last_change": "61", 2024-09-10T18:56:09.955 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend": "0", 2024-09-10T18:56:09.955 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T18:56:09.955 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T18:56:09.955 INFO:teuthology.orchestra.run.smithi018.stdout: "auid": 0, 2024-09-10T18:56:09.955 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_mode": "selfmanaged", 2024-09-10T18:56:09.956 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_seq": 0, 2024-09-10T18:56:09.956 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_epoch": 0, 2024-09-10T18:56:09.956 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_snaps": [], 2024-09-10T18:56:09.956 INFO:teuthology.orchestra.run.smithi018.stdout: "removed_snaps": "[]", 2024-09-10T18:56:09.956 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_bytes": 0, 2024-09-10T18:56:09.956 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_objects": 0, 2024-09-10T18:56:09.956 INFO:teuthology.orchestra.run.smithi018.stdout: "tiers": [], 2024-09-10T18:56:09.956 INFO:teuthology.orchestra.run.smithi018.stdout: "tier_of": -1, 2024-09-10T18:56:09.956 INFO:teuthology.orchestra.run.smithi018.stdout: "read_tier": -1, 2024-09-10T18:56:09.956 INFO:teuthology.orchestra.run.smithi018.stdout: "write_tier": -1, 2024-09-10T18:56:09.956 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_mode": "none", 2024-09-10T18:56:09.956 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_bytes": 0, 2024-09-10T18:56:09.957 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_objects": 0, 2024-09-10T18:56:09.957 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T18:56:09.957 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T18:56:09.957 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T18:56:09.957 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_flush_age": 0, 2024-09-10T18:56:09.957 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_evict_age": 0, 2024-09-10T18:56:09.957 INFO:teuthology.orchestra.run.smithi018.stdout: "erasure_code_profile": "", 2024-09-10T18:56:09.957 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_params": { 2024-09-10T18:56:09.957 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "none" 2024-09-10T18:56:09.957 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.957 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_period": 0, 2024-09-10T18:56:09.957 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_count": 0, 2024-09-10T18:56:09.957 INFO:teuthology.orchestra.run.smithi018.stdout: "use_gmt_hitset": true, 2024-09-10T18:56:09.958 INFO:teuthology.orchestra.run.smithi018.stdout: "min_read_recency_for_promote": 0, 2024-09-10T18:56:09.958 INFO:teuthology.orchestra.run.smithi018.stdout: "min_write_recency_for_promote": 0, 2024-09-10T18:56:09.958 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T18:56:09.958 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_search_last_n": 0, 2024-09-10T18:56:09.958 INFO:teuthology.orchestra.run.smithi018.stdout: "grade_table": [], 2024-09-10T18:56:09.958 INFO:teuthology.orchestra.run.smithi018.stdout: "stripe_width": 0, 2024-09-10T18:56:09.958 INFO:teuthology.orchestra.run.smithi018.stdout: "expected_num_objects": 0, 2024-09-10T18:56:09.958 INFO:teuthology.orchestra.run.smithi018.stdout: "fast_read": false, 2024-09-10T18:56:09.958 INFO:teuthology.orchestra.run.smithi018.stdout: "options": {}, 2024-09-10T18:56:09.958 INFO:teuthology.orchestra.run.smithi018.stdout: "application_metadata": { 2024-09-10T18:56:09.958 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw": {} 2024-09-10T18:56:09.958 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.958 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.958 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.959 INFO:teuthology.orchestra.run.smithi018.stdout: "pool": 6, 2024-09-10T18:56:09.959 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_name": "default.rgw.meta", 2024-09-10T18:56:09.959 INFO:teuthology.orchestra.run.smithi018.stdout: "create_time": "2024-09-10T18:45:22.566992+0000", 2024-09-10T18:56:09.959 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-09-10T18:56:09.959 INFO:teuthology.orchestra.run.smithi018.stdout: "flags_names": "hashpspool", 2024-09-10T18:56:09.959 INFO:teuthology.orchestra.run.smithi018.stdout: "type": 1, 2024-09-10T18:56:09.959 INFO:teuthology.orchestra.run.smithi018.stdout: "size": 3, 2024-09-10T18:56:09.959 INFO:teuthology.orchestra.run.smithi018.stdout: "min_size": 2, 2024-09-10T18:56:09.959 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_rule": 0, 2024-09-10T18:56:09.959 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_count": 0, 2024-09-10T18:56:09.959 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_target": 0, 2024-09-10T18:56:09.959 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T18:56:09.959 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T18:56:09.960 INFO:teuthology.orchestra.run.smithi018.stdout: "object_hash": 2, 2024-09-10T18:56:09.960 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscale_mode": "off", 2024-09-10T18:56:09.960 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num": 32, 2024-09-10T18:56:09.960 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num": 32, 2024-09-10T18:56:09.960 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num_target": 32, 2024-09-10T18:56:09.960 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_target": 32, 2024-09-10T18:56:09.960 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_pending": 32, 2024-09-10T18:56:09.960 INFO:teuthology.orchestra.run.smithi018.stdout: "last_pg_merge_meta": { 2024-09-10T18:56:09.960 INFO:teuthology.orchestra.run.smithi018.stdout: "source_pgid": "0.0", 2024-09-10T18:56:09.960 INFO:teuthology.orchestra.run.smithi018.stdout: "ready_epoch": 0, 2024-09-10T18:56:09.960 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_started": 0, 2024-09-10T18:56:09.960 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_clean": 0, 2024-09-10T18:56:09.960 INFO:teuthology.orchestra.run.smithi018.stdout: "source_version": "0'0", 2024-09-10T18:56:09.960 INFO:teuthology.orchestra.run.smithi018.stdout: "target_version": "0'0" 2024-09-10T18:56:09.961 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.961 INFO:teuthology.orchestra.run.smithi018.stdout: "last_change": "63", 2024-09-10T18:56:09.961 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend": "0", 2024-09-10T18:56:09.961 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T18:56:09.961 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T18:56:09.961 INFO:teuthology.orchestra.run.smithi018.stdout: "auid": 0, 2024-09-10T18:56:09.961 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_mode": "selfmanaged", 2024-09-10T18:56:09.961 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_seq": 0, 2024-09-10T18:56:09.961 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_epoch": 0, 2024-09-10T18:56:09.961 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_snaps": [], 2024-09-10T18:56:09.961 INFO:teuthology.orchestra.run.smithi018.stdout: "removed_snaps": "[]", 2024-09-10T18:56:09.961 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_bytes": 0, 2024-09-10T18:56:09.961 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_objects": 0, 2024-09-10T18:56:09.962 INFO:teuthology.orchestra.run.smithi018.stdout: "tiers": [], 2024-09-10T18:56:09.962 INFO:teuthology.orchestra.run.smithi018.stdout: "tier_of": -1, 2024-09-10T18:56:09.962 INFO:teuthology.orchestra.run.smithi018.stdout: "read_tier": -1, 2024-09-10T18:56:09.962 INFO:teuthology.orchestra.run.smithi018.stdout: "write_tier": -1, 2024-09-10T18:56:09.962 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_mode": "none", 2024-09-10T18:56:09.962 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_bytes": 0, 2024-09-10T18:56:09.962 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_objects": 0, 2024-09-10T18:56:09.962 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T18:56:09.962 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T18:56:09.962 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T18:56:09.962 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_flush_age": 0, 2024-09-10T18:56:09.962 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_evict_age": 0, 2024-09-10T18:56:09.962 INFO:teuthology.orchestra.run.smithi018.stdout: "erasure_code_profile": "", 2024-09-10T18:56:09.962 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_params": { 2024-09-10T18:56:09.962 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "none" 2024-09-10T18:56:09.963 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.963 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_period": 0, 2024-09-10T18:56:09.963 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_count": 0, 2024-09-10T18:56:09.963 INFO:teuthology.orchestra.run.smithi018.stdout: "use_gmt_hitset": true, 2024-09-10T18:56:09.963 INFO:teuthology.orchestra.run.smithi018.stdout: "min_read_recency_for_promote": 0, 2024-09-10T18:56:09.963 INFO:teuthology.orchestra.run.smithi018.stdout: "min_write_recency_for_promote": 0, 2024-09-10T18:56:09.963 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T18:56:09.963 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_search_last_n": 0, 2024-09-10T18:56:09.963 INFO:teuthology.orchestra.run.smithi018.stdout: "grade_table": [], 2024-09-10T18:56:09.963 INFO:teuthology.orchestra.run.smithi018.stdout: "stripe_width": 0, 2024-09-10T18:56:09.963 INFO:teuthology.orchestra.run.smithi018.stdout: "expected_num_objects": 0, 2024-09-10T18:56:09.963 INFO:teuthology.orchestra.run.smithi018.stdout: "fast_read": false, 2024-09-10T18:56:09.963 INFO:teuthology.orchestra.run.smithi018.stdout: "options": { 2024-09-10T18:56:09.963 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscale_bias": 4 2024-09-10T18:56:09.964 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.964 INFO:teuthology.orchestra.run.smithi018.stdout: "application_metadata": { 2024-09-10T18:56:09.964 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw": {} 2024-09-10T18:56:09.964 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.964 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.964 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.965 INFO:teuthology.orchestra.run.smithi018.stdout: "pool": 9, 2024-09-10T18:56:09.965 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_name": "unique_pool_2", 2024-09-10T18:56:09.965 INFO:teuthology.orchestra.run.smithi018.stdout: "create_time": "2024-09-10T18:55:34.972834+0000", 2024-09-10T18:56:09.965 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-09-10T18:56:09.965 INFO:teuthology.orchestra.run.smithi018.stdout: "flags_names": "hashpspool", 2024-09-10T18:56:09.965 INFO:teuthology.orchestra.run.smithi018.stdout: "type": 1, 2024-09-10T18:56:09.965 INFO:teuthology.orchestra.run.smithi018.stdout: "size": 3, 2024-09-10T18:56:09.965 INFO:teuthology.orchestra.run.smithi018.stdout: "min_size": 2, 2024-09-10T18:56:09.965 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_rule": 0, 2024-09-10T18:56:09.965 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_count": 0, 2024-09-10T18:56:09.965 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_target": 0, 2024-09-10T18:56:09.965 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T18:56:09.965 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T18:56:09.966 INFO:teuthology.orchestra.run.smithi018.stdout: "object_hash": 2, 2024-09-10T18:56:09.966 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscale_mode": "off", 2024-09-10T18:56:09.966 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num": 16, 2024-09-10T18:56:09.966 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num": 16, 2024-09-10T18:56:09.966 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num_target": 16, 2024-09-10T18:56:09.966 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_target": 16, 2024-09-10T18:56:09.966 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_pending": 16, 2024-09-10T18:56:09.966 INFO:teuthology.orchestra.run.smithi018.stdout: "last_pg_merge_meta": { 2024-09-10T18:56:09.966 INFO:teuthology.orchestra.run.smithi018.stdout: "source_pgid": "0.0", 2024-09-10T18:56:09.966 INFO:teuthology.orchestra.run.smithi018.stdout: "ready_epoch": 0, 2024-09-10T18:56:09.966 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_started": 0, 2024-09-10T18:56:09.966 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_clean": 0, 2024-09-10T18:56:09.966 INFO:teuthology.orchestra.run.smithi018.stdout: "source_version": "0'0", 2024-09-10T18:56:09.966 INFO:teuthology.orchestra.run.smithi018.stdout: "target_version": "0'0" 2024-09-10T18:56:09.966 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.967 INFO:teuthology.orchestra.run.smithi018.stdout: "last_change": "245", 2024-09-10T18:56:09.967 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend": "0", 2024-09-10T18:56:09.967 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T18:56:09.967 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T18:56:09.967 INFO:teuthology.orchestra.run.smithi018.stdout: "auid": 0, 2024-09-10T18:56:09.967 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_mode": "selfmanaged", 2024-09-10T18:56:09.967 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_seq": 0, 2024-09-10T18:56:09.967 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_epoch": 0, 2024-09-10T18:56:09.967 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_snaps": [], 2024-09-10T18:56:09.967 INFO:teuthology.orchestra.run.smithi018.stdout: "removed_snaps": "[]", 2024-09-10T18:56:09.967 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_bytes": 0, 2024-09-10T18:56:09.967 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_objects": 0, 2024-09-10T18:56:09.967 INFO:teuthology.orchestra.run.smithi018.stdout: "tiers": [], 2024-09-10T18:56:09.967 INFO:teuthology.orchestra.run.smithi018.stdout: "tier_of": -1, 2024-09-10T18:56:09.968 INFO:teuthology.orchestra.run.smithi018.stdout: "read_tier": -1, 2024-09-10T18:56:09.968 INFO:teuthology.orchestra.run.smithi018.stdout: "write_tier": -1, 2024-09-10T18:56:09.968 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_mode": "none", 2024-09-10T18:56:09.968 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_bytes": 0, 2024-09-10T18:56:09.968 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_objects": 0, 2024-09-10T18:56:09.968 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T18:56:09.968 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T18:56:09.968 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T18:56:09.968 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_flush_age": 0, 2024-09-10T18:56:09.968 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_evict_age": 0, 2024-09-10T18:56:09.968 INFO:teuthology.orchestra.run.smithi018.stdout: "erasure_code_profile": "", 2024-09-10T18:56:09.968 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_params": { 2024-09-10T18:56:09.968 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "none" 2024-09-10T18:56:09.969 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.969 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_period": 0, 2024-09-10T18:56:09.969 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_count": 0, 2024-09-10T18:56:09.969 INFO:teuthology.orchestra.run.smithi018.stdout: "use_gmt_hitset": true, 2024-09-10T18:56:09.969 INFO:teuthology.orchestra.run.smithi018.stdout: "min_read_recency_for_promote": 0, 2024-09-10T18:56:09.969 INFO:teuthology.orchestra.run.smithi018.stdout: "min_write_recency_for_promote": 0, 2024-09-10T18:56:09.969 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T18:56:09.969 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_search_last_n": 0, 2024-09-10T18:56:09.969 INFO:teuthology.orchestra.run.smithi018.stdout: "grade_table": [], 2024-09-10T18:56:09.969 INFO:teuthology.orchestra.run.smithi018.stdout: "stripe_width": 0, 2024-09-10T18:56:09.969 INFO:teuthology.orchestra.run.smithi018.stdout: "expected_num_objects": 0, 2024-09-10T18:56:09.969 INFO:teuthology.orchestra.run.smithi018.stdout: "fast_read": false, 2024-09-10T18:56:09.969 INFO:teuthology.orchestra.run.smithi018.stdout: "options": {}, 2024-09-10T18:56:09.969 INFO:teuthology.orchestra.run.smithi018.stdout: "application_metadata": { 2024-09-10T18:56:09.970 INFO:teuthology.orchestra.run.smithi018.stdout: "rados": {} 2024-09-10T18:56:09.970 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.970 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.970 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-09-10T18:56:09.970 INFO:teuthology.orchestra.run.smithi018.stdout: "osds": [ 2024-09-10T18:56:09.970 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.970 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 0, 2024-09-10T18:56:09.970 INFO:teuthology.orchestra.run.smithi018.stdout: "uuid": "eb4c7e67-300c-4a1a-b9f5-bb028eeeabc2", 2024-09-10T18:56:09.970 INFO:teuthology.orchestra.run.smithi018.stdout: "up": 1, 2024-09-10T18:56:09.970 INFO:teuthology.orchestra.run.smithi018.stdout: "in": 1, 2024-09-10T18:56:09.970 INFO:teuthology.orchestra.run.smithi018.stdout: "weight": 1, 2024-09-10T18:56:09.970 INFO:teuthology.orchestra.run.smithi018.stdout: "primary_affinity": 1, 2024-09-10T18:56:09.970 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_begin": 8, 2024-09-10T18:56:09.971 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_end": 205, 2024-09-10T18:56:09.971 INFO:teuthology.orchestra.run.smithi018.stdout: "up_from": 241, 2024-09-10T18:56:09.971 INFO:teuthology.orchestra.run.smithi018.stdout: "up_thru": 241, 2024-09-10T18:56:09.971 INFO:teuthology.orchestra.run.smithi018.stdout: "down_at": 206, 2024-09-10T18:56:09.971 INFO:teuthology.orchestra.run.smithi018.stdout: "lost_at": 0, 2024-09-10T18:56:09.971 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addrs": { 2024-09-10T18:56:09.971 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:56:09.971 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.971 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:56:09.971 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6801", 2024-09-10T18:56:09.971 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 4100931169 2024-09-10T18:56:09.972 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.972 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:09.972 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.972 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addrs": { 2024-09-10T18:56:09.972 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:56:09.972 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.972 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:56:09.972 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6802", 2024-09-10T18:56:09.972 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 4100931169 2024-09-10T18:56:09.973 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.973 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:09.973 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.973 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addrs": { 2024-09-10T18:56:09.973 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:56:09.973 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.973 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:56:09.973 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6804", 2024-09-10T18:56:09.973 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 4100931169 2024-09-10T18:56:09.973 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.973 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:09.973 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.973 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addrs": { 2024-09-10T18:56:09.974 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:56:09.974 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.974 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:56:09.974 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6803", 2024-09-10T18:56:09.974 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 4100931169 2024-09-10T18:56:09.974 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.974 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:09.974 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.974 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addr": "172.21.15.18:6801/4100931169", 2024-09-10T18:56:09.974 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addr": "172.21.15.18:6802/4100931169", 2024-09-10T18:56:09.974 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addr": "172.21.15.18:6804/4100931169", 2024-09-10T18:56:09.974 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addr": "172.21.15.18:6803/4100931169", 2024-09-10T18:56:09.975 INFO:teuthology.orchestra.run.smithi018.stdout: "state": [ 2024-09-10T18:56:09.975 INFO:teuthology.orchestra.run.smithi018.stdout: "exists", 2024-09-10T18:56:09.975 INFO:teuthology.orchestra.run.smithi018.stdout: "up" 2024-09-10T18:56:09.975 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:09.975 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.975 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.975 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 1, 2024-09-10T18:56:09.975 INFO:teuthology.orchestra.run.smithi018.stdout: "uuid": "63bf6a98-e349-4cc0-9502-a31693070c5b", 2024-09-10T18:56:09.975 INFO:teuthology.orchestra.run.smithi018.stdout: "up": 1, 2024-09-10T18:56:09.975 INFO:teuthology.orchestra.run.smithi018.stdout: "in": 1, 2024-09-10T18:56:09.975 INFO:teuthology.orchestra.run.smithi018.stdout: "weight": 1, 2024-09-10T18:56:09.975 INFO:teuthology.orchestra.run.smithi018.stdout: "primary_affinity": 1, 2024-09-10T18:56:09.975 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_begin": 0, 2024-09-10T18:56:09.976 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_end": 0, 2024-09-10T18:56:09.976 INFO:teuthology.orchestra.run.smithi018.stdout: "up_from": 13, 2024-09-10T18:56:09.976 INFO:teuthology.orchestra.run.smithi018.stdout: "up_thru": 241, 2024-09-10T18:56:09.976 INFO:teuthology.orchestra.run.smithi018.stdout: "down_at": 0, 2024-09-10T18:56:09.976 INFO:teuthology.orchestra.run.smithi018.stdout: "lost_at": 0, 2024-09-10T18:56:09.976 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addrs": { 2024-09-10T18:56:09.976 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:56:09.976 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.976 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:56:09.976 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6805", 2024-09-10T18:56:09.976 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3855640936 2024-09-10T18:56:09.976 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.977 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:09.977 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.977 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addrs": { 2024-09-10T18:56:09.977 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:56:09.977 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.977 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:56:09.977 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6806", 2024-09-10T18:56:09.977 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3855640936 2024-09-10T18:56:09.977 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.977 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:09.977 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.977 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addrs": { 2024-09-10T18:56:09.977 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:56:09.978 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.978 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:56:09.978 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6808", 2024-09-10T18:56:09.978 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3855640936 2024-09-10T18:56:09.978 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.978 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:09.978 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.978 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addrs": { 2024-09-10T18:56:09.978 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:56:09.978 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.978 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:56:09.978 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6807", 2024-09-10T18:56:09.979 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3855640936 2024-09-10T18:56:09.979 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.979 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:09.979 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.979 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addr": "172.21.15.18:6805/3855640936", 2024-09-10T18:56:09.979 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addr": "172.21.15.18:6806/3855640936", 2024-09-10T18:56:09.979 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addr": "172.21.15.18:6808/3855640936", 2024-09-10T18:56:09.979 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addr": "172.21.15.18:6807/3855640936", 2024-09-10T18:56:09.979 INFO:teuthology.orchestra.run.smithi018.stdout: "state": [ 2024-09-10T18:56:09.979 INFO:teuthology.orchestra.run.smithi018.stdout: "exists", 2024-09-10T18:56:09.979 INFO:teuthology.orchestra.run.smithi018.stdout: "up" 2024-09-10T18:56:09.979 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:09.980 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.980 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.980 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 2, 2024-09-10T18:56:09.980 INFO:teuthology.orchestra.run.smithi018.stdout: "uuid": "79a01c29-4f4d-43d1-b1ea-a3fcf014f84b", 2024-09-10T18:56:09.980 INFO:teuthology.orchestra.run.smithi018.stdout: "up": 1, 2024-09-10T18:56:09.980 INFO:teuthology.orchestra.run.smithi018.stdout: "in": 0, 2024-09-10T18:56:09.980 INFO:teuthology.orchestra.run.smithi018.stdout: "weight": 0, 2024-09-10T18:56:09.980 INFO:teuthology.orchestra.run.smithi018.stdout: "primary_affinity": 1, 2024-09-10T18:56:09.980 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_begin": 0, 2024-09-10T18:56:09.980 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_end": 0, 2024-09-10T18:56:09.980 INFO:teuthology.orchestra.run.smithi018.stdout: "up_from": 18, 2024-09-10T18:56:09.981 INFO:teuthology.orchestra.run.smithi018.stdout: "up_thru": 86, 2024-09-10T18:56:09.981 INFO:teuthology.orchestra.run.smithi018.stdout: "down_at": 0, 2024-09-10T18:56:09.981 INFO:teuthology.orchestra.run.smithi018.stdout: "lost_at": 0, 2024-09-10T18:56:09.981 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addrs": { 2024-09-10T18:56:09.981 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:56:09.981 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.981 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:56:09.981 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6809", 2024-09-10T18:56:09.981 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 4107623855 2024-09-10T18:56:09.981 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.981 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:09.981 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.981 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addrs": { 2024-09-10T18:56:09.982 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:56:09.982 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.982 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:56:09.982 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6810", 2024-09-10T18:56:09.982 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 4107623855 2024-09-10T18:56:09.982 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.982 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:09.982 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.982 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addrs": { 2024-09-10T18:56:09.982 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:56:09.982 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.982 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:56:09.982 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6812", 2024-09-10T18:56:09.983 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 4107623855 2024-09-10T18:56:09.983 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.983 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:09.983 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.983 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addrs": { 2024-09-10T18:56:09.983 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:56:09.983 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.983 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:56:09.983 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6811", 2024-09-10T18:56:09.983 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 4107623855 2024-09-10T18:56:09.983 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.984 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:09.984 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.984 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addr": "172.21.15.18:6809/4107623855", 2024-09-10T18:56:09.984 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addr": "172.21.15.18:6810/4107623855", 2024-09-10T18:56:09.984 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addr": "172.21.15.18:6812/4107623855", 2024-09-10T18:56:09.984 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addr": "172.21.15.18:6811/4107623855", 2024-09-10T18:56:09.984 INFO:teuthology.orchestra.run.smithi018.stdout: "state": [ 2024-09-10T18:56:09.984 INFO:teuthology.orchestra.run.smithi018.stdout: "exists", 2024-09-10T18:56:09.984 INFO:teuthology.orchestra.run.smithi018.stdout: "up" 2024-09-10T18:56:09.984 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:09.984 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.984 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.984 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 3, 2024-09-10T18:56:09.985 INFO:teuthology.orchestra.run.smithi018.stdout: "uuid": "1809e677-b98c-4924-982e-9297e37563af", 2024-09-10T18:56:09.985 INFO:teuthology.orchestra.run.smithi018.stdout: "up": 1, 2024-09-10T18:56:09.985 INFO:teuthology.orchestra.run.smithi018.stdout: "in": 1, 2024-09-10T18:56:09.985 INFO:teuthology.orchestra.run.smithi018.stdout: "weight": 1, 2024-09-10T18:56:09.985 INFO:teuthology.orchestra.run.smithi018.stdout: "primary_affinity": 1, 2024-09-10T18:56:09.985 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_begin": 0, 2024-09-10T18:56:09.985 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_end": 0, 2024-09-10T18:56:09.985 INFO:teuthology.orchestra.run.smithi018.stdout: "up_from": 25, 2024-09-10T18:56:09.985 INFO:teuthology.orchestra.run.smithi018.stdout: "up_thru": 241, 2024-09-10T18:56:09.985 INFO:teuthology.orchestra.run.smithi018.stdout: "down_at": 0, 2024-09-10T18:56:09.985 INFO:teuthology.orchestra.run.smithi018.stdout: "lost_at": 0, 2024-09-10T18:56:09.985 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addrs": { 2024-09-10T18:56:09.985 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:56:09.985 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.986 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:56:09.986 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6813", 2024-09-10T18:56:09.986 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 2732204253 2024-09-10T18:56:09.986 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.986 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:09.986 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.986 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addrs": { 2024-09-10T18:56:09.986 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:56:09.986 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.986 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:56:09.986 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6814", 2024-09-10T18:56:09.986 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 2732204253 2024-09-10T18:56:09.986 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.986 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:09.987 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.987 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addrs": { 2024-09-10T18:56:09.987 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:56:09.987 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.987 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:56:09.987 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6816", 2024-09-10T18:56:09.987 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 2732204253 2024-09-10T18:56:09.987 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.987 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:09.987 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.987 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addrs": { 2024-09-10T18:56:09.987 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:56:09.987 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.988 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:56:09.988 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6815", 2024-09-10T18:56:09.988 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 2732204253 2024-09-10T18:56:09.988 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.988 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:09.988 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.988 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addr": "172.21.15.18:6813/2732204253", 2024-09-10T18:56:09.988 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addr": "172.21.15.18:6814/2732204253", 2024-09-10T18:56:09.988 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addr": "172.21.15.18:6816/2732204253", 2024-09-10T18:56:09.988 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addr": "172.21.15.18:6815/2732204253", 2024-09-10T18:56:09.988 INFO:teuthology.orchestra.run.smithi018.stdout: "state": [ 2024-09-10T18:56:09.988 INFO:teuthology.orchestra.run.smithi018.stdout: "exists", 2024-09-10T18:56:09.988 INFO:teuthology.orchestra.run.smithi018.stdout: "up" 2024-09-10T18:56:09.988 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:09.989 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.989 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.989 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 4, 2024-09-10T18:56:09.989 INFO:teuthology.orchestra.run.smithi018.stdout: "uuid": "b682a300-bb53-4f36-8a9f-0753164ec4b7", 2024-09-10T18:56:09.989 INFO:teuthology.orchestra.run.smithi018.stdout: "up": 1, 2024-09-10T18:56:09.989 INFO:teuthology.orchestra.run.smithi018.stdout: "in": 1, 2024-09-10T18:56:09.989 INFO:teuthology.orchestra.run.smithi018.stdout: "weight": 1, 2024-09-10T18:56:09.989 INFO:teuthology.orchestra.run.smithi018.stdout: "primary_affinity": 1, 2024-09-10T18:56:09.989 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_begin": 0, 2024-09-10T18:56:09.989 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_end": 0, 2024-09-10T18:56:09.989 INFO:teuthology.orchestra.run.smithi018.stdout: "up_from": 30, 2024-09-10T18:56:09.989 INFO:teuthology.orchestra.run.smithi018.stdout: "up_thru": 241, 2024-09-10T18:56:09.989 INFO:teuthology.orchestra.run.smithi018.stdout: "down_at": 0, 2024-09-10T18:56:09.990 INFO:teuthology.orchestra.run.smithi018.stdout: "lost_at": 0, 2024-09-10T18:56:09.990 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addrs": { 2024-09-10T18:56:09.990 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:56:09.990 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.990 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:56:09.990 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6800", 2024-09-10T18:56:09.990 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 465860081 2024-09-10T18:56:09.990 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.990 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:09.990 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.990 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addrs": { 2024-09-10T18:56:09.990 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:56:09.990 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.990 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:56:09.991 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6801", 2024-09-10T18:56:09.991 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 465860081 2024-09-10T18:56:09.991 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.991 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:09.991 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.991 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addrs": { 2024-09-10T18:56:09.991 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:56:09.991 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.991 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:56:09.991 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6803", 2024-09-10T18:56:09.991 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 465860081 2024-09-10T18:56:09.991 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.991 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:09.991 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.991 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addrs": { 2024-09-10T18:56:09.992 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:56:09.992 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.992 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:56:09.992 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6802", 2024-09-10T18:56:09.992 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 465860081 2024-09-10T18:56:09.992 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.992 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:09.992 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.992 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addr": "172.21.15.103:6800/465860081", 2024-09-10T18:56:09.992 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addr": "172.21.15.103:6801/465860081", 2024-09-10T18:56:09.992 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addr": "172.21.15.103:6803/465860081", 2024-09-10T18:56:09.992 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addr": "172.21.15.103:6802/465860081", 2024-09-10T18:56:09.993 INFO:teuthology.orchestra.run.smithi018.stdout: "state": [ 2024-09-10T18:56:09.993 INFO:teuthology.orchestra.run.smithi018.stdout: "exists", 2024-09-10T18:56:09.993 INFO:teuthology.orchestra.run.smithi018.stdout: "up" 2024-09-10T18:56:09.993 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:09.993 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.993 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.993 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 5, 2024-09-10T18:56:09.993 INFO:teuthology.orchestra.run.smithi018.stdout: "uuid": "2bbfdb6c-0709-4fe4-8a4a-006f576f18e6", 2024-09-10T18:56:09.993 INFO:teuthology.orchestra.run.smithi018.stdout: "up": 1, 2024-09-10T18:56:09.993 INFO:teuthology.orchestra.run.smithi018.stdout: "in": 1, 2024-09-10T18:56:09.993 INFO:teuthology.orchestra.run.smithi018.stdout: "weight": 0.6995086669921875, 2024-09-10T18:56:09.993 INFO:teuthology.orchestra.run.smithi018.stdout: "primary_affinity": 1, 2024-09-10T18:56:09.993 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_begin": 0, 2024-09-10T18:56:09.994 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_end": 0, 2024-09-10T18:56:09.994 INFO:teuthology.orchestra.run.smithi018.stdout: "up_from": 37, 2024-09-10T18:56:09.994 INFO:teuthology.orchestra.run.smithi018.stdout: "up_thru": 241, 2024-09-10T18:56:09.994 INFO:teuthology.orchestra.run.smithi018.stdout: "down_at": 0, 2024-09-10T18:56:09.994 INFO:teuthology.orchestra.run.smithi018.stdout: "lost_at": 0, 2024-09-10T18:56:09.994 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addrs": { 2024-09-10T18:56:09.994 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:56:09.994 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.994 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:56:09.994 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6804", 2024-09-10T18:56:09.994 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3498719258 2024-09-10T18:56:09.994 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.994 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:09.995 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.995 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addrs": { 2024-09-10T18:56:09.995 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:56:09.995 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.995 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:56:09.995 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6805", 2024-09-10T18:56:09.995 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3498719258 2024-09-10T18:56:09.995 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.995 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:09.995 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.995 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addrs": { 2024-09-10T18:56:09.995 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:56:09.995 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.996 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:56:09.996 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6807", 2024-09-10T18:56:09.996 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3498719258 2024-09-10T18:56:09.996 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.996 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:09.996 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.996 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addrs": { 2024-09-10T18:56:09.996 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:56:09.996 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.996 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:56:09.996 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6806", 2024-09-10T18:56:09.996 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3498719258 2024-09-10T18:56:09.996 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.997 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:09.997 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.997 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addr": "172.21.15.103:6804/3498719258", 2024-09-10T18:56:09.997 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addr": "172.21.15.103:6805/3498719258", 2024-09-10T18:56:09.997 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addr": "172.21.15.103:6807/3498719258", 2024-09-10T18:56:09.997 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addr": "172.21.15.103:6806/3498719258", 2024-09-10T18:56:09.997 INFO:teuthology.orchestra.run.smithi018.stdout: "state": [ 2024-09-10T18:56:09.997 INFO:teuthology.orchestra.run.smithi018.stdout: "exists", 2024-09-10T18:56:09.997 INFO:teuthology.orchestra.run.smithi018.stdout: "up" 2024-09-10T18:56:09.997 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:09.997 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.997 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.997 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 6, 2024-09-10T18:56:09.998 INFO:teuthology.orchestra.run.smithi018.stdout: "uuid": "31f9711e-0ba8-4aeb-b852-7d93e8fcce33", 2024-09-10T18:56:09.998 INFO:teuthology.orchestra.run.smithi018.stdout: "up": 1, 2024-09-10T18:56:09.998 INFO:teuthology.orchestra.run.smithi018.stdout: "in": 1, 2024-09-10T18:56:09.998 INFO:teuthology.orchestra.run.smithi018.stdout: "weight": 1, 2024-09-10T18:56:09.998 INFO:teuthology.orchestra.run.smithi018.stdout: "primary_affinity": 1, 2024-09-10T18:56:09.998 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_begin": 0, 2024-09-10T18:56:09.998 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_end": 0, 2024-09-10T18:56:09.998 INFO:teuthology.orchestra.run.smithi018.stdout: "up_from": 43, 2024-09-10T18:56:09.998 INFO:teuthology.orchestra.run.smithi018.stdout: "up_thru": 241, 2024-09-10T18:56:09.998 INFO:teuthology.orchestra.run.smithi018.stdout: "down_at": 0, 2024-09-10T18:56:09.998 INFO:teuthology.orchestra.run.smithi018.stdout: "lost_at": 0, 2024-09-10T18:56:09.998 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addrs": { 2024-09-10T18:56:09.998 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:56:09.999 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.999 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:56:09.999 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6808", 2024-09-10T18:56:09.999 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 184353788 2024-09-10T18:56:09.999 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:09.999 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:09.999 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:09.999 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addrs": { 2024-09-10T18:56:09.999 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:56:09.999 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:09.999 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:56:09.999 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6809", 2024-09-10T18:56:09.999 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 184353788 2024-09-10T18:56:10.000 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:10.000 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:10.000 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:10.000 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addrs": { 2024-09-10T18:56:10.000 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:56:10.000 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:10.000 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:56:10.000 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6811", 2024-09-10T18:56:10.000 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 184353788 2024-09-10T18:56:10.000 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:10.000 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:10.000 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:10.000 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addrs": { 2024-09-10T18:56:10.001 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:56:10.001 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:10.001 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:56:10.001 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6810", 2024-09-10T18:56:10.001 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 184353788 2024-09-10T18:56:10.001 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:10.001 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:10.001 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:10.001 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addr": "172.21.15.103:6808/184353788", 2024-09-10T18:56:10.001 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addr": "172.21.15.103:6809/184353788", 2024-09-10T18:56:10.001 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addr": "172.21.15.103:6811/184353788", 2024-09-10T18:56:10.001 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addr": "172.21.15.103:6810/184353788", 2024-09-10T18:56:10.001 INFO:teuthology.orchestra.run.smithi018.stdout: "state": [ 2024-09-10T18:56:10.002 INFO:teuthology.orchestra.run.smithi018.stdout: "exists", 2024-09-10T18:56:10.002 INFO:teuthology.orchestra.run.smithi018.stdout: "up" 2024-09-10T18:56:10.002 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:10.002 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:10.002 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:10.002 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 7, 2024-09-10T18:56:10.002 INFO:teuthology.orchestra.run.smithi018.stdout: "uuid": "f12282f8-4747-45ff-8336-95440560a88f", 2024-09-10T18:56:10.002 INFO:teuthology.orchestra.run.smithi018.stdout: "up": 1, 2024-09-10T18:56:10.002 INFO:teuthology.orchestra.run.smithi018.stdout: "in": 1, 2024-09-10T18:56:10.002 INFO:teuthology.orchestra.run.smithi018.stdout: "weight": 1, 2024-09-10T18:56:10.002 INFO:teuthology.orchestra.run.smithi018.stdout: "primary_affinity": 1, 2024-09-10T18:56:10.002 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_begin": 0, 2024-09-10T18:56:10.003 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_end": 0, 2024-09-10T18:56:10.003 INFO:teuthology.orchestra.run.smithi018.stdout: "up_from": 52, 2024-09-10T18:56:10.003 INFO:teuthology.orchestra.run.smithi018.stdout: "up_thru": 241, 2024-09-10T18:56:10.003 INFO:teuthology.orchestra.run.smithi018.stdout: "down_at": 0, 2024-09-10T18:56:10.003 INFO:teuthology.orchestra.run.smithi018.stdout: "lost_at": 0, 2024-09-10T18:56:10.003 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addrs": { 2024-09-10T18:56:10.003 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:56:10.003 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:10.003 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:56:10.003 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6812", 2024-09-10T18:56:10.003 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 941024403 2024-09-10T18:56:10.003 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:10.003 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:10.003 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:10.004 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addrs": { 2024-09-10T18:56:10.004 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:56:10.004 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:10.004 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:56:10.004 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6813", 2024-09-10T18:56:10.004 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 941024403 2024-09-10T18:56:10.004 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:10.004 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:10.004 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:10.004 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:10.005 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addrs": { 2024-09-10T18:56:10.005 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:56:10.005 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:10.005 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:56:10.005 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6815", 2024-09-10T18:56:10.005 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 941024403 2024-09-10T18:56:10.005 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:10.006 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:10.006 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:10.006 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addrs": { 2024-09-10T18:56:10.006 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T18:56:10.006 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:10.006 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T18:56:10.006 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6814", 2024-09-10T18:56:10.006 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 941024403 2024-09-10T18:56:10.006 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:10.006 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:10.006 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:10.006 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addr": "172.21.15.103:6812/941024403", 2024-09-10T18:56:10.006 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addr": "172.21.15.103:6813/941024403", 2024-09-10T18:56:10.007 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addr": "172.21.15.103:6815/941024403", 2024-09-10T18:56:10.007 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addr": "172.21.15.103:6814/941024403", 2024-09-10T18:56:10.007 INFO:teuthology.orchestra.run.smithi018.stdout: "state": [ 2024-09-10T18:56:10.007 INFO:teuthology.orchestra.run.smithi018.stdout: "exists", 2024-09-10T18:56:10.007 INFO:teuthology.orchestra.run.smithi018.stdout: "up" 2024-09-10T18:56:10.007 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:10.007 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:10.007 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-09-10T18:56:10.007 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_xinfo": [ 2024-09-10T18:56:10.007 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:10.007 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 0, 2024-09-10T18:56:10.007 INFO:teuthology.orchestra.run.smithi018.stdout: "down_stamp": "0.000000", 2024-09-10T18:56:10.007 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_probability": 0, 2024-09-10T18:56:10.007 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_interval": 0, 2024-09-10T18:56:10.008 INFO:teuthology.orchestra.run.smithi018.stdout: "features": 4540138320759226367, 2024-09-10T18:56:10.008 INFO:teuthology.orchestra.run.smithi018.stdout: "old_weight": 0, 2024-09-10T18:56:10.008 INFO:teuthology.orchestra.run.smithi018.stdout: "last_purged_snaps_scrub": "2024-09-10T18:55:38.936154+0000", 2024-09-10T18:56:10.008 INFO:teuthology.orchestra.run.smithi018.stdout: "dead_epoch": 205 2024-09-10T18:56:10.008 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:10.008 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:10.008 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 1, 2024-09-10T18:56:10.008 INFO:teuthology.orchestra.run.smithi018.stdout: "down_stamp": "0.000000", 2024-09-10T18:56:10.008 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_probability": 0, 2024-09-10T18:56:10.008 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_interval": 0, 2024-09-10T18:56:10.008 INFO:teuthology.orchestra.run.smithi018.stdout: "features": 4540138320759226367, 2024-09-10T18:56:10.008 INFO:teuthology.orchestra.run.smithi018.stdout: "old_weight": 0, 2024-09-10T18:56:10.008 INFO:teuthology.orchestra.run.smithi018.stdout: "last_purged_snaps_scrub": "2024-09-10T18:55:41.440820+0000", 2024-09-10T18:56:10.009 INFO:teuthology.orchestra.run.smithi018.stdout: "dead_epoch": 0 2024-09-10T18:56:10.009 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:10.009 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:10.009 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 2, 2024-09-10T18:56:10.009 INFO:teuthology.orchestra.run.smithi018.stdout: "down_stamp": "0.000000", 2024-09-10T18:56:10.009 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_probability": 0, 2024-09-10T18:56:10.009 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_interval": 0, 2024-09-10T18:56:10.009 INFO:teuthology.orchestra.run.smithi018.stdout: "features": 4540138320759226367, 2024-09-10T18:56:10.009 INFO:teuthology.orchestra.run.smithi018.stdout: "old_weight": 65536, 2024-09-10T18:56:10.009 INFO:teuthology.orchestra.run.smithi018.stdout: "last_purged_snaps_scrub": "2024-09-10T18:55:54.206256+0000", 2024-09-10T18:56:10.009 INFO:teuthology.orchestra.run.smithi018.stdout: "dead_epoch": 0 2024-09-10T18:56:10.009 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:10.009 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:10.010 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 3, 2024-09-10T18:56:10.010 INFO:teuthology.orchestra.run.smithi018.stdout: "down_stamp": "0.000000", 2024-09-10T18:56:10.010 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_probability": 0, 2024-09-10T18:56:10.010 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_interval": 0, 2024-09-10T18:56:10.010 INFO:teuthology.orchestra.run.smithi018.stdout: "features": 4540138320759226367, 2024-09-10T18:56:10.010 INFO:teuthology.orchestra.run.smithi018.stdout: "old_weight": 0, 2024-09-10T18:56:10.010 INFO:teuthology.orchestra.run.smithi018.stdout: "last_purged_snaps_scrub": "2024-09-10T18:55:59.094206+0000", 2024-09-10T18:56:10.010 INFO:teuthology.orchestra.run.smithi018.stdout: "dead_epoch": 0 2024-09-10T18:56:10.010 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:10.010 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:10.010 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 4, 2024-09-10T18:56:10.010 INFO:teuthology.orchestra.run.smithi018.stdout: "down_stamp": "0.000000", 2024-09-10T18:56:10.010 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_probability": 0, 2024-09-10T18:56:10.011 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_interval": 0, 2024-09-10T18:56:10.011 INFO:teuthology.orchestra.run.smithi018.stdout: "features": 4540138320759226367, 2024-09-10T18:56:10.011 INFO:teuthology.orchestra.run.smithi018.stdout: "old_weight": 0, 2024-09-10T18:56:10.011 INFO:teuthology.orchestra.run.smithi018.stdout: "last_purged_snaps_scrub": "2024-09-10T18:55:50.745647+0000", 2024-09-10T18:56:10.011 INFO:teuthology.orchestra.run.smithi018.stdout: "dead_epoch": 0 2024-09-10T18:56:10.011 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:10.011 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:10.011 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 5, 2024-09-10T18:56:10.011 INFO:teuthology.orchestra.run.smithi018.stdout: "down_stamp": "0.000000", 2024-09-10T18:56:10.011 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_probability": 0, 2024-09-10T18:56:10.011 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_interval": 0, 2024-09-10T18:56:10.011 INFO:teuthology.orchestra.run.smithi018.stdout: "features": 4540138320759226367, 2024-09-10T18:56:10.011 INFO:teuthology.orchestra.run.smithi018.stdout: "old_weight": 0, 2024-09-10T18:56:10.011 INFO:teuthology.orchestra.run.smithi018.stdout: "last_purged_snaps_scrub": "2024-09-10T18:55:49.900930+0000", 2024-09-10T18:56:10.012 INFO:teuthology.orchestra.run.smithi018.stdout: "dead_epoch": 0 2024-09-10T18:56:10.012 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:10.012 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:10.012 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 6, 2024-09-10T18:56:10.012 INFO:teuthology.orchestra.run.smithi018.stdout: "down_stamp": "0.000000", 2024-09-10T18:56:10.012 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_probability": 0, 2024-09-10T18:56:10.012 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_interval": 0, 2024-09-10T18:56:10.012 INFO:teuthology.orchestra.run.smithi018.stdout: "features": 4540138320759226367, 2024-09-10T18:56:10.012 INFO:teuthology.orchestra.run.smithi018.stdout: "old_weight": 0, 2024-09-10T18:56:10.012 INFO:teuthology.orchestra.run.smithi018.stdout: "last_purged_snaps_scrub": "2024-09-10T18:56:07.561754+0000", 2024-09-10T18:56:10.012 INFO:teuthology.orchestra.run.smithi018.stdout: "dead_epoch": 0 2024-09-10T18:56:10.012 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:10.012 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:10.013 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 7, 2024-09-10T18:56:10.013 INFO:teuthology.orchestra.run.smithi018.stdout: "down_stamp": "0.000000", 2024-09-10T18:56:10.013 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_probability": 0, 2024-09-10T18:56:10.013 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_interval": 0, 2024-09-10T18:56:10.013 INFO:teuthology.orchestra.run.smithi018.stdout: "features": 4540138320759226367, 2024-09-10T18:56:10.013 INFO:teuthology.orchestra.run.smithi018.stdout: "old_weight": 0, 2024-09-10T18:56:10.013 INFO:teuthology.orchestra.run.smithi018.stdout: "last_purged_snaps_scrub": "2024-09-10T18:54:51.994817+0000", 2024-09-10T18:56:10.013 INFO:teuthology.orchestra.run.smithi018.stdout: "dead_epoch": 0 2024-09-10T18:56:10.013 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:10.013 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-09-10T18:56:10.013 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_upmap": [], 2024-09-10T18:56:10.013 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_upmap_items": [ 2024-09-10T18:56:10.013 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:10.014 INFO:teuthology.orchestra.run.smithi018.stdout: "pgid": "2.4", 2024-09-10T18:56:10.014 INFO:teuthology.orchestra.run.smithi018.stdout: "mappings": [ 2024-09-10T18:56:10.014 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:10.014 INFO:teuthology.orchestra.run.smithi018.stdout: "from": 7, 2024-09-10T18:56:10.014 INFO:teuthology.orchestra.run.smithi018.stdout: "to": 4 2024-09-10T18:56:10.014 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:10.014 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:10.014 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:10.014 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:10.014 INFO:teuthology.orchestra.run.smithi018.stdout: "pgid": "4.e", 2024-09-10T18:56:10.014 INFO:teuthology.orchestra.run.smithi018.stdout: "mappings": [ 2024-09-10T18:56:10.014 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:10.014 INFO:teuthology.orchestra.run.smithi018.stdout: "from": 1, 2024-09-10T18:56:10.014 INFO:teuthology.orchestra.run.smithi018.stdout: "to": 5 2024-09-10T18:56:10.014 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:10.015 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:10.015 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:10.015 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:10.015 INFO:teuthology.orchestra.run.smithi018.stdout: "pgid": "4.f", 2024-09-10T18:56:10.015 INFO:teuthology.orchestra.run.smithi018.stdout: "mappings": [ 2024-09-10T18:56:10.015 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:10.015 INFO:teuthology.orchestra.run.smithi018.stdout: "from": 1, 2024-09-10T18:56:10.015 INFO:teuthology.orchestra.run.smithi018.stdout: "to": 5 2024-09-10T18:56:10.015 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:10.015 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:10.015 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:10.015 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:10.015 INFO:teuthology.orchestra.run.smithi018.stdout: "pgid": "5.5", 2024-09-10T18:56:10.015 INFO:teuthology.orchestra.run.smithi018.stdout: "mappings": [ 2024-09-10T18:56:10.016 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:10.016 INFO:teuthology.orchestra.run.smithi018.stdout: "from": 0, 2024-09-10T18:56:10.016 INFO:teuthology.orchestra.run.smithi018.stdout: "to": 6 2024-09-10T18:56:10.016 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:10.016 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:10.016 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:10.016 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:10.016 INFO:teuthology.orchestra.run.smithi018.stdout: "pgid": "5.10", 2024-09-10T18:56:10.016 INFO:teuthology.orchestra.run.smithi018.stdout: "mappings": [ 2024-09-10T18:56:10.016 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:10.016 INFO:teuthology.orchestra.run.smithi018.stdout: "from": 7, 2024-09-10T18:56:10.016 INFO:teuthology.orchestra.run.smithi018.stdout: "to": 3 2024-09-10T18:56:10.016 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:10.017 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:10.017 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:10.017 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:10.017 INFO:teuthology.orchestra.run.smithi018.stdout: "pgid": "5.11", 2024-09-10T18:56:10.017 INFO:teuthology.orchestra.run.smithi018.stdout: "mappings": [ 2024-09-10T18:56:10.017 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:10.017 INFO:teuthology.orchestra.run.smithi018.stdout: "from": 4, 2024-09-10T18:56:10.017 INFO:teuthology.orchestra.run.smithi018.stdout: "to": 3 2024-09-10T18:56:10.017 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:10.017 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:10.017 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:10.017 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:10.018 INFO:teuthology.orchestra.run.smithi018.stdout: "pgid": "5.19", 2024-09-10T18:56:10.018 INFO:teuthology.orchestra.run.smithi018.stdout: "mappings": [ 2024-09-10T18:56:10.018 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:10.018 INFO:teuthology.orchestra.run.smithi018.stdout: "from": 1, 2024-09-10T18:56:10.018 INFO:teuthology.orchestra.run.smithi018.stdout: "to": 6 2024-09-10T18:56:10.018 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:10.018 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:10.018 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:10.018 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:10.018 INFO:teuthology.orchestra.run.smithi018.stdout: "pgid": "6.f", 2024-09-10T18:56:10.018 INFO:teuthology.orchestra.run.smithi018.stdout: "mappings": [ 2024-09-10T18:56:10.018 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:10.018 INFO:teuthology.orchestra.run.smithi018.stdout: "from": 3, 2024-09-10T18:56:10.019 INFO:teuthology.orchestra.run.smithi018.stdout: "to": 0 2024-09-10T18:56:10.019 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:10.019 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:10.019 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:10.019 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-09-10T18:56:10.019 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_temp": [], 2024-09-10T18:56:10.019 INFO:teuthology.orchestra.run.smithi018.stdout: "primary_temp": [], 2024-09-10T18:56:10.019 INFO:teuthology.orchestra.run.smithi018.stdout: "blocklist": { 2024-09-10T18:56:10.019 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/3021537013": "2024-09-11T18:45:54.998616+0000", 2024-09-10T18:56:10.019 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/458572898": "2024-09-11T18:40:28.880042+0000", 2024-09-10T18:56:10.019 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/1859541419": "2024-09-11T18:40:28.880042+0000", 2024-09-10T18:56:10.019 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/3090642970": "2024-09-11T18:40:59.659411+0000", 2024-09-10T18:56:10.020 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/2875894312": "2024-09-11T18:40:59.659411+0000", 2024-09-10T18:56:10.020 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/178638059": "2024-09-11T18:40:59.659411+0000", 2024-09-10T18:56:10.020 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/692931402": "2024-09-11T18:45:54.998616+0000", 2024-09-10T18:56:10.020 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/2408360238": "2024-09-11T18:45:54.998616+0000", 2024-09-10T18:56:10.020 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/57192865": "2024-09-11T18:40:28.880042+0000", 2024-09-10T18:56:10.020 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:6800/2748756682": "2024-09-11T18:40:59.659411+0000", 2024-09-10T18:56:10.020 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:6800/732375116": "2024-09-11T18:45:54.998616+0000", 2024-09-10T18:56:10.020 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:6800/3869801150": "2024-09-11T18:40:28.880042+0000", 2024-09-10T18:56:10.020 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/2982361259": "2024-09-11T18:45:54.998616+0000", 2024-09-10T18:56:10.020 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/185700038": "2024-09-11T18:45:54.998616+0000" 2024-09-10T18:56:10.020 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:10.020 INFO:teuthology.orchestra.run.smithi018.stdout: "range_blocklist": {}, 2024-09-10T18:56:10.020 INFO:teuthology.orchestra.run.smithi018.stdout: "erasure_code_profiles": { 2024-09-10T18:56:10.021 INFO:teuthology.orchestra.run.smithi018.stdout: "default": { 2024-09-10T18:56:10.021 INFO:teuthology.orchestra.run.smithi018.stdout: "crush-failure-domain": "osd", 2024-09-10T18:56:10.021 INFO:teuthology.orchestra.run.smithi018.stdout: "k": "2", 2024-09-10T18:56:10.021 INFO:teuthology.orchestra.run.smithi018.stdout: "m": "1", 2024-09-10T18:56:10.021 INFO:teuthology.orchestra.run.smithi018.stdout: "plugin": "jerasure", 2024-09-10T18:56:10.021 INFO:teuthology.orchestra.run.smithi018.stdout: "technique": "reed_sol_van" 2024-09-10T18:56:10.021 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:10.021 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:10.021 INFO:teuthology.orchestra.run.smithi018.stdout: "removed_snaps_queue": [], 2024-09-10T18:56:10.021 INFO:teuthology.orchestra.run.smithi018.stdout: "new_removed_snaps": [], 2024-09-10T18:56:10.021 INFO:teuthology.orchestra.run.smithi018.stdout: "new_purged_snaps": [], 2024-09-10T18:56:10.021 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_node_flags": {}, 2024-09-10T18:56:10.022 INFO:teuthology.orchestra.run.smithi018.stdout: "device_class_flags": {}, 2024-09-10T18:56:10.022 INFO:teuthology.orchestra.run.smithi018.stdout: "stretch_mode": { 2024-09-10T18:56:10.022 INFO:teuthology.orchestra.run.smithi018.stdout: "stretch_mode_enabled": false, 2024-09-10T18:56:10.022 INFO:teuthology.orchestra.run.smithi018.stdout: "stretch_bucket_count": 0, 2024-09-10T18:56:10.022 INFO:teuthology.orchestra.run.smithi018.stdout: "degraded_stretch_mode": 0, 2024-09-10T18:56:10.022 INFO:teuthology.orchestra.run.smithi018.stdout: "recovering_stretch_mode": 0, 2024-09-10T18:56:10.022 INFO:teuthology.orchestra.run.smithi018.stdout: "stretch_mode_bucket": 0 2024-09-10T18:56:10.022 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:10.022 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T18:56:10.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:09 smithi103 ceph-mon[33163]: pgmap v496: 168 pgs: 168 active+clean; 2.7 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1.1 KiB/s rd, 157 MiB/s wr, 2.51k op/s 2024-09-10T18:56:10.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:10.063 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:10.063 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:10.164 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:10.205 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:10.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:10.206 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:10.307 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:10.346 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:10.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:10.347 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:10.386 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 22 16 49932 49916 141.78 100.312 0.012633 0.00704807 2024-09-10T18:56:10.447 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:10.481 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:10.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:10.482 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:10.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:56:10 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:56:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:56:10.583 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:10.602 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-10T18:56:10.608 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:10.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:10.609 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:10.710 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:10.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:10.767 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:10.767 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:10.868 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:10.898 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:10.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:10.899 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:10.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:10 smithi018 ceph-mon[40196]: 5.c scrub starts 2024-09-10T18:56:10.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:10 smithi018 ceph-mon[40196]: 5.c scrub ok 2024-09-10T18:56:10.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:10 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2945295223' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-10T18:56:10.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:10 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:56:10.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:10 smithi018 ceph-mon[29336]: 5.c scrub starts 2024-09-10T18:56:10.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:10 smithi018 ceph-mon[29336]: 5.c scrub ok 2024-09-10T18:56:10.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:10 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2945295223' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-10T18:56:10.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:10 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:56:11.000 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:11.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:10 smithi103 ceph-mon[33163]: 5.c scrub starts 2024-09-10T18:56:11.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:10 smithi103 ceph-mon[33163]: 5.c scrub ok 2024-09-10T18:56:11.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:10 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2945295223' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-10T18:56:11.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:10 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:56:11.050 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:56:11.051 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:11.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:11.053 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:11.154 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:11.186 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:11.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:11.188 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:11.281 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:56:10 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:56:10] "GET /metrics HTTP/1.1" 200 224181 "" "Prometheus/2.43.0" 2024-09-10T18:56:11.288 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:11.331 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:11.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:11.332 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:11.386 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 23 16 51643 51627 140.265 106.938 0.00873839 0.00712417 2024-09-10T18:56:11.434 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:11.463 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:11.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:11.465 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:11.565 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:11.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:11.605 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:11.605 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:11.706 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:11.751 INFO:tasks.thrashosds.thrasher:j is {'epoch': 256, 'fsid': 'ba327f6c-6fa3-11ef-bcea-c7b262605968', 'created': '2024-09-10T18:40:07.669687+0000', 'modified': '2024-09-10T18:56:08.618936+0000', 'last_up_change': '2024-09-10T18:55:39.357104+0000', 'last_in_change': '2024-09-10T18:47:35.385915+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 19, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 9, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T18:43:08.162659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 21, 'pg_placement_num': 21, 'pg_placement_num_target': 21, 'pg_num_target': 21, 'pg_num_pending': 21, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '169', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '167', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': '.rgw.root', 'create_time': '2024-09-10T18:45:16.512219+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 3, 'pool_name': 'datapool', 'create_time': '2024-09-10T18:45:16.943626+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 60, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-10T18:45:18.450026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-10T18:45:20.494618+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-10T18:45:22.566992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}}, {'pool': 9, 'pool_name': 'unique_pool_2', 'create_time': '2024-09-10T18:55:34.972834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '245', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': 'eb4c7e67-300c-4a1a-b9f5-bb028eeeabc2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 8, 'last_clean_end': 205, 'up_from': 241, 'up_thru': 241, 'down_at': 206, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6801', 'nonce': 4100931169}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6802', 'nonce': 4100931169}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6804', 'nonce': 4100931169}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6803', 'nonce': 4100931169}]}, 'public_addr': '172.21.15.18:6801/4100931169', 'cluster_addr': '172.21.15.18:6802/4100931169', 'heartbeat_back_addr': '172.21.15.18:6804/4100931169', 'heartbeat_front_addr': '172.21.15.18:6803/4100931169', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '63bf6a98-e349-4cc0-9502-a31693070c5b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 241, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6805', 'nonce': 3855640936}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6806', 'nonce': 3855640936}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6808', 'nonce': 3855640936}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6807', 'nonce': 3855640936}]}, 'public_addr': '172.21.15.18:6805/3855640936', 'cluster_addr': '172.21.15.18:6806/3855640936', 'heartbeat_back_addr': '172.21.15.18:6808/3855640936', 'heartbeat_front_addr': '172.21.15.18:6807/3855640936', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '79a01c29-4f4d-43d1-b1ea-a3fcf014f84b', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 86, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6809', 'nonce': 4107623855}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6810', 'nonce': 4107623855}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6812', 'nonce': 4107623855}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6811', 'nonce': 4107623855}]}, 'public_addr': '172.21.15.18:6809/4107623855', 'cluster_addr': '172.21.15.18:6810/4107623855', 'heartbeat_back_addr': '172.21.15.18:6812/4107623855', 'heartbeat_front_addr': '172.21.15.18:6811/4107623855', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '1809e677-b98c-4924-982e-9297e37563af', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 25, 'up_thru': 241, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6813', 'nonce': 2732204253}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6814', 'nonce': 2732204253}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6816', 'nonce': 2732204253}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6815', 'nonce': 2732204253}]}, 'public_addr': '172.21.15.18:6813/2732204253', 'cluster_addr': '172.21.15.18:6814/2732204253', 'heartbeat_back_addr': '172.21.15.18:6816/2732204253', 'heartbeat_front_addr': '172.21.15.18:6815/2732204253', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'b682a300-bb53-4f36-8a9f-0753164ec4b7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 30, 'up_thru': 241, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6800', 'nonce': 465860081}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6801', 'nonce': 465860081}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6803', 'nonce': 465860081}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6802', 'nonce': 465860081}]}, 'public_addr': '172.21.15.103:6800/465860081', 'cluster_addr': '172.21.15.103:6801/465860081', 'heartbeat_back_addr': '172.21.15.103:6803/465860081', 'heartbeat_front_addr': '172.21.15.103:6802/465860081', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '2bbfdb6c-0709-4fe4-8a4a-006f576f18e6', 'up': 1, 'in': 1, 'weight': 0.6995086669921875, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 37, 'up_thru': 241, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6804', 'nonce': 3498719258}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6805', 'nonce': 3498719258}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6807', 'nonce': 3498719258}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6806', 'nonce': 3498719258}]}, 'public_addr': '172.21.15.103:6804/3498719258', 'cluster_addr': '172.21.15.103:6805/3498719258', 'heartbeat_back_addr': '172.21.15.103:6807/3498719258', 'heartbeat_front_addr': '172.21.15.103:6806/3498719258', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '31f9711e-0ba8-4aeb-b852-7d93e8fcce33', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 43, 'up_thru': 241, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6808', 'nonce': 184353788}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6809', 'nonce': 184353788}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6811', 'nonce': 184353788}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6810', 'nonce': 184353788}]}, 'public_addr': '172.21.15.103:6808/184353788', 'cluster_addr': '172.21.15.103:6809/184353788', 'heartbeat_back_addr': '172.21.15.103:6811/184353788', 'heartbeat_front_addr': '172.21.15.103:6810/184353788', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'f12282f8-4747-45ff-8336-95440560a88f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 52, 'up_thru': 241, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6812', 'nonce': 941024403}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6813', 'nonce': 941024403}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6815', 'nonce': 941024403}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6814', 'nonce': 941024403}]}, 'public_addr': '172.21.15.103:6812/941024403', 'cluster_addr': '172.21.15.103:6813/941024403', 'heartbeat_back_addr': '172.21.15.103:6815/941024403', 'heartbeat_front_addr': '172.21.15.103:6814/941024403', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T18:55:38.936154+0000', 'dead_epoch': 205}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T18:55:41.440820+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-10T18:55:54.206256+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T18:55:59.094206+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T18:55:50.745647+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T18:55:49.900930+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T18:56:07.561754+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T18:54:51.994817+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '2.4', 'mappings': [{'from': 7, 'to': 4}]}, {'pgid': '4.e', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.f', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '5.5', 'mappings': [{'from': 0, 'to': 6}]}, {'pgid': '5.10', 'mappings': [{'from': 7, 'to': 3}]}, {'pgid': '5.11', 'mappings': [{'from': 4, 'to': 3}]}, {'pgid': '5.19', 'mappings': [{'from': 1, 'to': 6}]}, {'pgid': '6.f', 'mappings': [{'from': 3, 'to': 0}]}], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.18:0/3021537013': '2024-09-11T18:45:54.998616+0000', '172.21.15.18:0/458572898': '2024-09-11T18:40:28.880042+0000', '172.21.15.18:0/1859541419': '2024-09-11T18:40:28.880042+0000', '172.21.15.18:0/3090642970': '2024-09-11T18:40:59.659411+0000', '172.21.15.18:0/2875894312': '2024-09-11T18:40:59.659411+0000', '172.21.15.18:0/178638059': '2024-09-11T18:40:59.659411+0000', '172.21.15.18:0/692931402': '2024-09-11T18:45:54.998616+0000', '172.21.15.18:0/2408360238': '2024-09-11T18:45:54.998616+0000', '172.21.15.18:0/57192865': '2024-09-11T18:40:28.880042+0000', '172.21.15.18:6800/2748756682': '2024-09-11T18:40:59.659411+0000', '172.21.15.18:6800/732375116': '2024-09-11T18:45:54.998616+0000', '172.21.15.18:6800/3869801150': '2024-09-11T18:40:28.880042+0000', '172.21.15.18:0/2982361259': '2024-09-11T18:45:54.998616+0000', '172.21.15.18:0/185700038': '2024-09-11T18:45:54.998616+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-10T18:56:11.751 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T18:56:11.753 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:11.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:11.757 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:11.857 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:11.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:11.905 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:11.908 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:11.998 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:11 smithi018 ceph-mon[29336]: 6.13 scrub starts 2024-09-10T18:56:11.998 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:11 smithi018 ceph-mon[29336]: 6.13 scrub ok 2024-09-10T18:56:11.998 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:11 smithi018 ceph-mon[29336]: 6.a scrub starts 2024-09-10T18:56:11.998 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:11 smithi018 ceph-mon[29336]: 6.a scrub ok 2024-09-10T18:56:11.998 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:11 smithi018 ceph-mon[29336]: pgmap v497: 168 pgs: 168 active+clean; 2.7 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 511 B/s rd, 91 MiB/s wr, 1.45k op/s 2024-09-10T18:56:11.999 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:11 smithi018 ceph-mon[40196]: 6.13 scrub starts 2024-09-10T18:56:11.999 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:11 smithi018 ceph-mon[40196]: 6.13 scrub ok 2024-09-10T18:56:11.999 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:11 smithi018 ceph-mon[40196]: 6.a scrub starts 2024-09-10T18:56:11.999 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:11 smithi018 ceph-mon[40196]: 6.a scrub ok 2024-09-10T18:56:11.999 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:11 smithi018 ceph-mon[40196]: pgmap v497: 168 pgs: 168 active+clean; 2.7 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 511 B/s rd, 91 MiB/s wr, 1.45k op/s 2024-09-10T18:56:12.009 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:12.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:11 smithi103 ceph-mon[33163]: 6.13 scrub starts 2024-09-10T18:56:12.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:11 smithi103 ceph-mon[33163]: 6.13 scrub ok 2024-09-10T18:56:12.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:11 smithi103 ceph-mon[33163]: 6.a scrub starts 2024-09-10T18:56:12.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:11 smithi103 ceph-mon[33163]: 6.a scrub ok 2024-09-10T18:56:12.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:11 smithi103 ceph-mon[33163]: pgmap v497: 168 pgs: 168 active+clean; 2.7 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 511 B/s rd, 91 MiB/s wr, 1.45k op/s 2024-09-10T18:56:12.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:12.066 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:12.066 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:12.167 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:12.217 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:12.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:12.218 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:12.319 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:12.351 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:12.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:12.352 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:12.386 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 24 16 53309 53293 138.759 104.125 0.00786085 0.00720209 2024-09-10T18:56:12.453 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:12.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:12.485 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:12.486 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:12.491 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:56:12.586 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:12.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:12.625 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:12.625 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:12.726 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:12.759 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:12.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:12.760 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:12.861 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:12.894 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:12.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:12.895 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:12.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:12 smithi018 ceph-mon[40196]: 1.0 scrub starts 2024-09-10T18:56:12.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:12 smithi018 ceph-mon[40196]: 1.0 scrub ok 2024-09-10T18:56:12.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:12 smithi018 ceph-mon[40196]: 1.1 scrub starts 2024-09-10T18:56:12.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:12 smithi018 ceph-mon[40196]: 1.1 scrub ok 2024-09-10T18:56:12.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:12 smithi018 ceph-mon[29336]: 1.0 scrub starts 2024-09-10T18:56:12.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:12 smithi018 ceph-mon[29336]: 1.0 scrub ok 2024-09-10T18:56:12.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:12 smithi018 ceph-mon[29336]: 1.1 scrub starts 2024-09-10T18:56:12.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:12 smithi018 ceph-mon[29336]: 1.1 scrub ok 2024-09-10T18:56:12.996 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:13.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:12 smithi103 ceph-mon[33163]: 1.0 scrub starts 2024-09-10T18:56:13.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:12 smithi103 ceph-mon[33163]: 1.0 scrub ok 2024-09-10T18:56:13.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:12 smithi103 ceph-mon[33163]: 1.1 scrub starts 2024-09-10T18:56:13.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:12 smithi103 ceph-mon[33163]: 1.1 scrub ok 2024-09-10T18:56:13.029 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:13.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:13.030 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:13.134 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:13.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:13.185 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:13.185 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:13.286 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:13.350 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:13.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:13.352 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:13.386 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 25 16 54968 54952 137.356 103.688 0.00878628 0.00727539 2024-09-10T18:56:13.452 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:13.486 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:13.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:13.487 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:13.588 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:13.617 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:13.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:13.618 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:13.719 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:13.750 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:13.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:13.751 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:13.852 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:13.888 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:13.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:13.889 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:13.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:13 smithi018 ceph-mon[40196]: 6.6 scrub starts 2024-09-10T18:56:13.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:13 smithi018 ceph-mon[40196]: 6.6 scrub ok 2024-09-10T18:56:13.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:13 smithi018 ceph-mon[40196]: 1.6 scrub starts 2024-09-10T18:56:13.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:13 smithi018 ceph-mon[40196]: 1.6 scrub ok 2024-09-10T18:56:13.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:13 smithi018 ceph-mon[40196]: pgmap v498: 168 pgs: 168 active+clean; 3.1 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.1 KiB/s rd, 130 MiB/s wr, 2.08k op/s 2024-09-10T18:56:13.948 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:13 smithi018 ceph-mon[29336]: 6.6 scrub starts 2024-09-10T18:56:13.948 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:13 smithi018 ceph-mon[29336]: 6.6 scrub ok 2024-09-10T18:56:13.948 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:13 smithi018 ceph-mon[29336]: 1.6 scrub starts 2024-09-10T18:56:13.948 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:13 smithi018 ceph-mon[29336]: 1.6 scrub ok 2024-09-10T18:56:13.948 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:13 smithi018 ceph-mon[29336]: pgmap v498: 168 pgs: 168 active+clean; 3.1 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.1 KiB/s rd, 130 MiB/s wr, 2.08k op/s 2024-09-10T18:56:13.990 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:14.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:13 smithi103 ceph-mon[33163]: 6.6 scrub starts 2024-09-10T18:56:14.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:13 smithi103 ceph-mon[33163]: 6.6 scrub ok 2024-09-10T18:56:14.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:13 smithi103 ceph-mon[33163]: 1.6 scrub starts 2024-09-10T18:56:14.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:13 smithi103 ceph-mon[33163]: 1.6 scrub ok 2024-09-10T18:56:14.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:13 smithi103 ceph-mon[33163]: pgmap v498: 168 pgs: 168 active+clean; 3.1 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.1 KiB/s rd, 130 MiB/s wr, 2.08k op/s 2024-09-10T18:56:14.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:14.030 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:14.031 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:14.131 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:14.172 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:14.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:14.173 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:14.274 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:14.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:14.306 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:14.306 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:14.386 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 26 16 56641 56625 136.094 104.562 0.00961488 0.00734261 2024-09-10T18:56:14.407 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:14.446 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:14.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:14.447 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:14.548 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:14.579 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:14.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:14.580 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:14.681 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:14.718 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:14.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:14.719 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:14.820 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:14.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:14.861 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:14.862 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:14.962 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:14.993 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:14.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:14.994 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:15.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:14 smithi103 ceph-mon[33163]: 5.10 scrub starts 2024-09-10T18:56:15.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:14 smithi103 ceph-mon[33163]: 5.10 scrub ok 2024-09-10T18:56:15.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:14 smithi103 ceph-mon[33163]: 5.1d scrub starts 2024-09-10T18:56:15.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:14 smithi103 ceph-mon[33163]: 5.1d scrub ok 2024-09-10T18:56:15.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:14 smithi103 ceph-mon[33163]: 2.c scrub starts 2024-09-10T18:56:15.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:14 smithi103 ceph-mon[33163]: 2.c scrub ok 2024-09-10T18:56:15.059 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:14 smithi018 ceph-mon[40196]: 5.10 scrub starts 2024-09-10T18:56:15.060 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:14 smithi018 ceph-mon[40196]: 5.10 scrub ok 2024-09-10T18:56:15.060 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:14 smithi018 ceph-mon[40196]: 5.1d scrub starts 2024-09-10T18:56:15.060 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:14 smithi018 ceph-mon[40196]: 5.1d scrub ok 2024-09-10T18:56:15.060 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:14 smithi018 ceph-mon[40196]: 2.c scrub starts 2024-09-10T18:56:15.060 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:14 smithi018 ceph-mon[40196]: 2.c scrub ok 2024-09-10T18:56:15.061 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:14 smithi018 ceph-mon[29336]: 5.10 scrub starts 2024-09-10T18:56:15.061 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:14 smithi018 ceph-mon[29336]: 5.10 scrub ok 2024-09-10T18:56:15.061 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:14 smithi018 ceph-mon[29336]: 5.1d scrub starts 2024-09-10T18:56:15.061 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:14 smithi018 ceph-mon[29336]: 5.1d scrub ok 2024-09-10T18:56:15.061 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:14 smithi018 ceph-mon[29336]: 2.c scrub starts 2024-09-10T18:56:15.061 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:14 smithi018 ceph-mon[29336]: 2.c scrub ok 2024-09-10T18:56:15.095 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:15.144 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:15.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:15.145 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:15.150 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:56:15.150 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:56:15.185 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:56:15.185 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:56:15.215 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:56:15.216 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:56:15.246 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:15.252 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:56:15.252 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:56:15.276 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:15.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:15.277 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:15.285 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:56:15.286 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:56:15.318 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:56:15.319 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:56:15.359 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:56:15.359 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:56:15.377 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:15.386 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 27 16 58217 58201 134.701 98.5 0.0269643 0.00741758 2024-09-10T18:56:15.396 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:56:15.396 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:56:15.415 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:15.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:15.416 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:15.517 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:15.559 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:15.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:15.560 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:15.661 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:15.705 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:15.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:15.706 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:15.807 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:15.852 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:15.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:15.853 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:15.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:15 smithi018 ceph-mon[40196]: 2.9 scrub starts 2024-09-10T18:56:15.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:15 smithi018 ceph-mon[40196]: 2.9 scrub ok 2024-09-10T18:56:15.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:15 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3359589609' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:56:15.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:15 smithi018 ceph-mon[40196]: pgmap v499: 168 pgs: 168 active+clean; 3.2 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 639 B/s rd, 80 MiB/s wr, 1.28k op/s 2024-09-10T18:56:15.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:15 smithi018 ceph-mon[29336]: 2.9 scrub starts 2024-09-10T18:56:15.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:15 smithi018 ceph-mon[29336]: 2.9 scrub ok 2024-09-10T18:56:15.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:15 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3359589609' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:56:15.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:15 smithi018 ceph-mon[29336]: pgmap v499: 168 pgs: 168 active+clean; 3.2 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 639 B/s rd, 80 MiB/s wr, 1.28k op/s 2024-09-10T18:56:15.953 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:15.996 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:15.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:15.997 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:16.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:15 smithi103 ceph-mon[33163]: 2.9 scrub starts 2024-09-10T18:56:16.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:15 smithi103 ceph-mon[33163]: 2.9 scrub ok 2024-09-10T18:56:16.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:15 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3359589609' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:56:16.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:15 smithi103 ceph-mon[33163]: pgmap v499: 168 pgs: 168 active+clean; 3.2 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 639 B/s rd, 80 MiB/s wr, 1.28k op/s 2024-09-10T18:56:16.098 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:16.130 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:16.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:16.131 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:16.232 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:16.264 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:16.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:16.265 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:16.366 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:16.386 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 28 16 59735 59719 133.278 94.875 0.00664412 0.00749734 2024-09-10T18:56:16.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:16.408 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:16.408 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:16.509 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:16.548 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:16.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:16.549 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:16.650 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:16.685 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:16.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:16.687 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:16.777 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is unset 2024-09-10T18:56:16.788 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:16.834 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:16.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:16.835 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:16.936 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:16.971 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:16.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:16.972 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:17.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:16 smithi103 ceph-mon[33163]: 6.1b scrub starts 2024-09-10T18:56:17.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:16 smithi103 ceph-mon[33163]: 6.1b scrub ok 2024-09-10T18:56:17.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:16 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:56:17.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:16 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:56:17.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:16 smithi103 ceph-mon[33163]: osdmap e257: 8 total, 8 up, 7 in 2024-09-10T18:56:17.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:16 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2157109052' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:56:17.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:16 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:56:17.023 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:56:16 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:56:17.073 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:17.116 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:17.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:17.117 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:17.137 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:16 smithi018 ceph-mon[40196]: 6.1b scrub starts 2024-09-10T18:56:17.137 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:16 smithi018 ceph-mon[40196]: 6.1b scrub ok 2024-09-10T18:56:17.138 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:16 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:56:17.138 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:16 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:56:17.138 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:16 smithi018 ceph-mon[40196]: osdmap e257: 8 total, 8 up, 7 in 2024-09-10T18:56:17.138 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:16 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2157109052' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:56:17.138 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:16 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:56:17.138 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:16 smithi018 ceph-mon[29336]: 6.1b scrub starts 2024-09-10T18:56:17.138 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:16 smithi018 ceph-mon[29336]: 6.1b scrub ok 2024-09-10T18:56:17.138 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:16 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:56:17.138 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:16 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:56:17.138 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:16 smithi018 ceph-mon[29336]: osdmap e257: 8 total, 8 up, 7 in 2024-09-10T18:56:17.138 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:16 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2157109052' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:56:17.138 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:16 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:56:17.218 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:17.249 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:17.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:17.250 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:17.351 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:17.387 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 29 16 61368 61352 132.202 102.062 0.0168937 0.00755845 2024-09-10T18:56:17.390 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:17.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:17.392 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:17.492 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:17.523 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:17.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:17.524 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:17.625 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:17.664 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:17.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:17.665 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:17.765 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:17.794 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:17.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:17.795 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:17.896 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:17.936 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:17.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:17.937 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:17.982 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:56:18.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:17 smithi103 ceph-mon[33163]: 4.0 scrub starts 2024-09-10T18:56:18.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:17 smithi103 ceph-mon[33163]: 4.0 scrub ok 2024-09-10T18:56:18.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:17 smithi103 ceph-mon[33163]: 5.8 scrub starts 2024-09-10T18:56:18.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:17 smithi103 ceph-mon[33163]: 5.8 scrub ok 2024-09-10T18:56:18.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:17 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:18.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:17 smithi103 ceph-mon[33163]: 2.18 scrub starts 2024-09-10T18:56:18.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:17 smithi103 ceph-mon[33163]: 2.18 scrub ok 2024-09-10T18:56:18.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:17 smithi103 ceph-mon[33163]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:56:18.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:17 smithi103 ceph-mon[33163]: Cluster is now healthy 2024-09-10T18:56:18.024 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:17 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:56:18.024 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:17 smithi103 ceph-mon[33163]: osdmap e258: 8 total, 8 up, 7 in 2024-09-10T18:56:18.024 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:17 smithi103 ceph-mon[33163]: pgmap v502: 168 pgs: 168 active+clean; 3.6 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 117 MiB/s wr, 1.88k op/s 2024-09-10T18:56:18.038 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:18.085 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:18.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:18.087 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:18.109 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:17 smithi018 ceph-mon[40196]: 4.0 scrub starts 2024-09-10T18:56:18.110 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:17 smithi018 ceph-mon[40196]: 4.0 scrub ok 2024-09-10T18:56:18.110 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:17 smithi018 ceph-mon[40196]: 5.8 scrub starts 2024-09-10T18:56:18.110 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:17 smithi018 ceph-mon[40196]: 5.8 scrub ok 2024-09-10T18:56:18.110 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:17 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:18.110 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:17 smithi018 ceph-mon[40196]: 2.18 scrub starts 2024-09-10T18:56:18.110 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:17 smithi018 ceph-mon[40196]: 2.18 scrub ok 2024-09-10T18:56:18.110 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:17 smithi018 ceph-mon[40196]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:56:18.110 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:17 smithi018 ceph-mon[40196]: Cluster is now healthy 2024-09-10T18:56:18.110 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:17 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:56:18.110 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:17 smithi018 ceph-mon[40196]: osdmap e258: 8 total, 8 up, 7 in 2024-09-10T18:56:18.110 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:17 smithi018 ceph-mon[40196]: pgmap v502: 168 pgs: 168 active+clean; 3.6 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 117 MiB/s wr, 1.88k op/s 2024-09-10T18:56:18.111 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:17 smithi018 ceph-mon[29336]: 4.0 scrub starts 2024-09-10T18:56:18.111 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:17 smithi018 ceph-mon[29336]: 4.0 scrub ok 2024-09-10T18:56:18.111 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:17 smithi018 ceph-mon[29336]: 5.8 scrub starts 2024-09-10T18:56:18.111 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:17 smithi018 ceph-mon[29336]: 5.8 scrub ok 2024-09-10T18:56:18.111 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:17 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:18.111 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:17 smithi018 ceph-mon[29336]: 2.18 scrub starts 2024-09-10T18:56:18.111 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:17 smithi018 ceph-mon[29336]: 2.18 scrub ok 2024-09-10T18:56:18.111 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:17 smithi018 ceph-mon[29336]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:56:18.111 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:17 smithi018 ceph-mon[29336]: Cluster is now healthy 2024-09-10T18:56:18.111 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:17 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:56:18.111 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:17 smithi018 ceph-mon[29336]: osdmap e258: 8 total, 8 up, 7 in 2024-09-10T18:56:18.111 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:17 smithi018 ceph-mon[29336]: pgmap v502: 168 pgs: 168 active+clean; 3.6 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 117 MiB/s wr, 1.88k op/s 2024-09-10T18:56:18.188 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:18.222 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:18.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:18.223 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:18.324 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:18.356 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:18.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:18.357 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:18.373 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T18:56:18.387 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 30 16 62996 62980 131.186 101.75 0.00872727 0.00761772 2024-09-10T18:56:18.457 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:18.482 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:18.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:18.483 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:18.584 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:18.640 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:18.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:18.641 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:18.742 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:18.782 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:18.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:18.783 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:18.884 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:18.916 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:18.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:18.917 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:18.977 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:18 smithi018 ceph-mon[40196]: 5.a scrub starts 2024-09-10T18:56:18.977 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:18 smithi018 ceph-mon[40196]: 5.a scrub ok 2024-09-10T18:56:18.977 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:18 smithi018 ceph-mon[29336]: 5.a scrub starts 2024-09-10T18:56:18.977 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:18 smithi018 ceph-mon[29336]: 5.a scrub ok 2024-09-10T18:56:19.018 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:19.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:18 smithi103 ceph-mon[33163]: 5.a scrub starts 2024-09-10T18:56:19.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:18 smithi103 ceph-mon[33163]: 5.a scrub ok 2024-09-10T18:56:19.052 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:19.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:19.053 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:19.154 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:19.188 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:19.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:19.189 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:19.290 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:19.326 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:19.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:19.327 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:19.387 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 31 15 64438 64423 129.863 90.1875 0.0125366 0.0076959 2024-09-10T18:56:19.428 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:19.468 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:19.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:19.469 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:19.570 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:19.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:19.593 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:19.593 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:19.694 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:19.729 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:19.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:19.730 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:19.831 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:19.863 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:19.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:19.864 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:19.965 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:19.971 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-10T18:56:19.996 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:19.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:19.998 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:20.099 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:20.121 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:20.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:20.122 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:20.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:19 smithi018 ceph-mon[40196]: 5.0 scrub starts 2024-09-10T18:56:20.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:19 smithi018 ceph-mon[40196]: 5.0 scrub ok 2024-09-10T18:56:20.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:19 smithi018 ceph-mon[40196]: 1.d scrub starts 2024-09-10T18:56:20.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:19 smithi018 ceph-mon[40196]: 1.d scrub ok 2024-09-10T18:56:20.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:19 smithi018 ceph-mon[40196]: 2.14 scrub starts 2024-09-10T18:56:20.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:19 smithi018 ceph-mon[40196]: 2.14 scrub ok 2024-09-10T18:56:20.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:19 smithi018 ceph-mon[40196]: 4.4 scrub starts 2024-09-10T18:56:20.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:19 smithi018 ceph-mon[40196]: 4.4 scrub ok 2024-09-10T18:56:20.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:19 smithi018 ceph-mon[40196]: pgmap v503: 168 pgs: 168 active+clean; 3.7 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 129 MiB/s wr, 2.07k op/s 2024-09-10T18:56:20.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:19 smithi018 ceph-mon[29336]: 5.0 scrub starts 2024-09-10T18:56:20.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:19 smithi018 ceph-mon[29336]: 5.0 scrub ok 2024-09-10T18:56:20.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:19 smithi018 ceph-mon[29336]: 1.d scrub starts 2024-09-10T18:56:20.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:19 smithi018 ceph-mon[29336]: 1.d scrub ok 2024-09-10T18:56:20.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:19 smithi018 ceph-mon[29336]: 2.14 scrub starts 2024-09-10T18:56:20.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:19 smithi018 ceph-mon[29336]: 2.14 scrub ok 2024-09-10T18:56:20.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:19 smithi018 ceph-mon[29336]: 4.4 scrub starts 2024-09-10T18:56:20.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:19 smithi018 ceph-mon[29336]: 4.4 scrub ok 2024-09-10T18:56:20.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:19 smithi018 ceph-mon[29336]: pgmap v503: 168 pgs: 168 active+clean; 3.7 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 129 MiB/s wr, 2.07k op/s 2024-09-10T18:56:20.223 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:20.238 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:19 smithi103 ceph-mon[33163]: 5.0 scrub starts 2024-09-10T18:56:20.238 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:19 smithi103 ceph-mon[33163]: 5.0 scrub ok 2024-09-10T18:56:20.238 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:19 smithi103 ceph-mon[33163]: 1.d scrub starts 2024-09-10T18:56:20.238 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:19 smithi103 ceph-mon[33163]: 1.d scrub ok 2024-09-10T18:56:20.238 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:19 smithi103 ceph-mon[33163]: 2.14 scrub starts 2024-09-10T18:56:20.238 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:19 smithi103 ceph-mon[33163]: 2.14 scrub ok 2024-09-10T18:56:20.238 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:19 smithi103 ceph-mon[33163]: 4.4 scrub starts 2024-09-10T18:56:20.238 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:19 smithi103 ceph-mon[33163]: 4.4 scrub ok 2024-09-10T18:56:20.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:19 smithi103 ceph-mon[33163]: pgmap v503: 168 pgs: 168 active+clean; 3.7 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 129 MiB/s wr, 2.07k op/s 2024-09-10T18:56:20.253 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:20.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:20.254 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:20.355 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:20.376 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:20.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:20.377 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:20.387 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 32 16 65648 65632 128.167 75.5625 0.00859891 0.00778568 2024-09-10T18:56:20.478 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:20.507 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:20.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:20.508 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:20.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:56:20 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:56:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:56:20.609 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:20.646 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:20.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:20.647 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:20.748 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:20.784 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:20.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:20.785 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:20.886 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:20.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:20.933 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:20.933 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:21.034 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:21.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:21.060 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:21.060 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:21.161 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:21.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:20 smithi018 ceph-mon[29336]: 5.1f scrub starts 2024-09-10T18:56:21.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:20 smithi018 ceph-mon[29336]: 5.1f scrub ok 2024-09-10T18:56:21.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:20 smithi018 ceph-mon[29336]: 5.e scrub starts 2024-09-10T18:56:21.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:20 smithi018 ceph-mon[29336]: 5.e scrub ok 2024-09-10T18:56:21.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:20 smithi018 ceph-mon[40196]: 5.1f scrub starts 2024-09-10T18:56:21.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:20 smithi018 ceph-mon[40196]: 5.1f scrub ok 2024-09-10T18:56:21.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:20 smithi018 ceph-mon[40196]: 5.e scrub starts 2024-09-10T18:56:21.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:20 smithi018 ceph-mon[40196]: 5.e scrub ok 2024-09-10T18:56:21.195 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:56:20 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:56:20] "GET /metrics HTTP/1.1" 200 224181 "" "Prometheus/2.43.0" 2024-09-10T18:56:21.197 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:21.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:21.198 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:21.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:20 smithi103 ceph-mon[33163]: 5.1f scrub starts 2024-09-10T18:56:21.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:20 smithi103 ceph-mon[33163]: 5.1f scrub ok 2024-09-10T18:56:21.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:20 smithi103 ceph-mon[33163]: 5.e scrub starts 2024-09-10T18:56:21.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:20 smithi103 ceph-mon[33163]: 5.e scrub ok 2024-09-10T18:56:21.298 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:21.321 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:21.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:21.322 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:21.387 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 33 16 66830 66814 126.521 73.875 0.00799469 0.00789842 2024-09-10T18:56:21.424 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:21.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:21.460 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:21.460 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:21.561 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:21.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:21.589 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:21.589 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:21.690 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:21.722 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:21.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:21.723 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:21.824 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:21.853 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:21.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:21.854 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:21.955 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:21.983 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:21.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:21.984 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:22.085 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:22.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:22.126 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:22.126 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:22.163 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:21 smithi018 ceph-mon[40196]: 4.b scrub starts 2024-09-10T18:56:22.163 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:21 smithi018 ceph-mon[40196]: 4.b scrub ok 2024-09-10T18:56:22.163 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:21 smithi018 ceph-mon[40196]: 5.5 scrub starts 2024-09-10T18:56:22.163 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:21 smithi018 ceph-mon[40196]: 5.5 scrub ok 2024-09-10T18:56:22.163 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:21 smithi018 ceph-mon[40196]: 6.17 scrub starts 2024-09-10T18:56:22.163 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:21 smithi018 ceph-mon[40196]: 6.17 scrub ok 2024-09-10T18:56:22.164 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:21 smithi018 ceph-mon[40196]: 1.10 scrub starts 2024-09-10T18:56:22.164 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:21 smithi018 ceph-mon[40196]: 1.10 scrub ok 2024-09-10T18:56:22.164 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:21 smithi018 ceph-mon[40196]: pgmap v504: 168 pgs: 168 active+clean; 3.7 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 639 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-10T18:56:22.164 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:21 smithi018 ceph-mon[29336]: 4.b scrub starts 2024-09-10T18:56:22.164 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:21 smithi018 ceph-mon[29336]: 4.b scrub ok 2024-09-10T18:56:22.164 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:21 smithi018 ceph-mon[29336]: 5.5 scrub starts 2024-09-10T18:56:22.164 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:21 smithi018 ceph-mon[29336]: 5.5 scrub ok 2024-09-10T18:56:22.164 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:21 smithi018 ceph-mon[29336]: 6.17 scrub starts 2024-09-10T18:56:22.164 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:21 smithi018 ceph-mon[29336]: 6.17 scrub ok 2024-09-10T18:56:22.164 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:21 smithi018 ceph-mon[29336]: 1.10 scrub starts 2024-09-10T18:56:22.165 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:21 smithi018 ceph-mon[29336]: 1.10 scrub ok 2024-09-10T18:56:22.165 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:21 smithi018 ceph-mon[29336]: pgmap v504: 168 pgs: 168 active+clean; 3.7 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 639 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-10T18:56:22.227 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:22.254 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:22.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:22.255 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:22.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:21 smithi103 ceph-mon[33163]: 4.b scrub starts 2024-09-10T18:56:22.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:21 smithi103 ceph-mon[33163]: 4.b scrub ok 2024-09-10T18:56:22.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:21 smithi103 ceph-mon[33163]: 5.5 scrub starts 2024-09-10T18:56:22.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:21 smithi103 ceph-mon[33163]: 5.5 scrub ok 2024-09-10T18:56:22.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:21 smithi103 ceph-mon[33163]: 6.17 scrub starts 2024-09-10T18:56:22.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:21 smithi103 ceph-mon[33163]: 6.17 scrub ok 2024-09-10T18:56:22.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:21 smithi103 ceph-mon[33163]: 1.10 scrub starts 2024-09-10T18:56:22.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:21 smithi103 ceph-mon[33163]: 1.10 scrub ok 2024-09-10T18:56:22.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:21 smithi103 ceph-mon[33163]: pgmap v504: 168 pgs: 168 active+clean; 3.7 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 639 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-10T18:56:22.356 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:22.387 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 34 16 68255 68239 125.419 89.0625 0.0142626 0.00796821 2024-09-10T18:56:22.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:22.389 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:22.389 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:22.490 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:22.514 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:22.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:22.515 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:22.616 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:22.653 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:22.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:22.654 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:22.755 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:22.788 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:22.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:22.789 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:22.889 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:22.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:22.930 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:22.931 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:23.031 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:23.064 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:23.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:23.065 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:23.166 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:23.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:22 smithi018 ceph-mon[40196]: 6.5 scrub starts 2024-09-10T18:56:23.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:22 smithi018 ceph-mon[40196]: 6.5 scrub ok 2024-09-10T18:56:23.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:22 smithi018 ceph-mon[40196]: 5.3 deep-scrub starts 2024-09-10T18:56:23.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:22 smithi018 ceph-mon[40196]: 5.3 deep-scrub ok 2024-09-10T18:56:23.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:22 smithi018 ceph-mon[40196]: 4.5 deep-scrub starts 2024-09-10T18:56:23.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:22 smithi018 ceph-mon[40196]: 4.5 deep-scrub ok 2024-09-10T18:56:23.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:22 smithi018 ceph-mon[29336]: 6.5 scrub starts 2024-09-10T18:56:23.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:22 smithi018 ceph-mon[29336]: 6.5 scrub ok 2024-09-10T18:56:23.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:22 smithi018 ceph-mon[29336]: 5.3 deep-scrub starts 2024-09-10T18:56:23.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:22 smithi018 ceph-mon[29336]: 5.3 deep-scrub ok 2024-09-10T18:56:23.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:22 smithi018 ceph-mon[29336]: 4.5 deep-scrub starts 2024-09-10T18:56:23.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:22 smithi018 ceph-mon[29336]: 4.5 deep-scrub ok 2024-09-10T18:56:23.205 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:23.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:23.206 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:23.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:22 smithi103 ceph-mon[33163]: 6.5 scrub starts 2024-09-10T18:56:23.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:22 smithi103 ceph-mon[33163]: 6.5 scrub ok 2024-09-10T18:56:23.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:22 smithi103 ceph-mon[33163]: 5.3 deep-scrub starts 2024-09-10T18:56:23.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:22 smithi103 ceph-mon[33163]: 5.3 deep-scrub ok 2024-09-10T18:56:23.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:22 smithi103 ceph-mon[33163]: 4.5 deep-scrub starts 2024-09-10T18:56:23.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:22 smithi103 ceph-mon[33163]: 4.5 deep-scrub ok 2024-09-10T18:56:23.307 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:23.310 INFO:teuthology.orchestra.run.smithi018.stdout:ERROR: (22) Invalid argument 2024-09-10T18:56:23.310 INFO:teuthology.orchestra.run.smithi018.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:56:23.339 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:23.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:23.340 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:23.387 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 35 16 69721 69705 124.454 91.625 0.00974671 0.00803022 2024-09-10T18:56:23.441 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:23.468 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:23.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:23.469 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:23.570 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:23.603 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:23.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:23.604 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:23.705 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:23.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:23.728 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:23.729 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:23.829 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:23.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:23.863 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:23.863 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:23.964 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:24.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:24.002 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:24.002 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:24.103 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:24.125 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:24.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:24.126 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:24.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:23 smithi018 ceph-mon[40196]: 5.19 scrub starts 2024-09-10T18:56:24.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:23 smithi018 ceph-mon[40196]: 5.19 scrub ok 2024-09-10T18:56:24.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:23 smithi018 ceph-mon[40196]: 4.f scrub starts 2024-09-10T18:56:24.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:23 smithi018 ceph-mon[40196]: 4.f scrub ok 2024-09-10T18:56:24.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:23 smithi018 ceph-mon[40196]: 3.0 scrub starts 2024-09-10T18:56:24.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:23 smithi018 ceph-mon[40196]: 3.0 scrub ok 2024-09-10T18:56:24.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:23 smithi018 ceph-mon[40196]: pgmap v505: 168 pgs: 168 active+clean; 4.0 GiB data, 16 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 108 MiB/s wr, 1.74k op/s 2024-09-10T18:56:24.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:23 smithi018 ceph-mon[29336]: 5.19 scrub starts 2024-09-10T18:56:24.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:23 smithi018 ceph-mon[29336]: 5.19 scrub ok 2024-09-10T18:56:24.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:23 smithi018 ceph-mon[29336]: 4.f scrub starts 2024-09-10T18:56:24.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:23 smithi018 ceph-mon[29336]: 4.f scrub ok 2024-09-10T18:56:24.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:23 smithi018 ceph-mon[29336]: 3.0 scrub starts 2024-09-10T18:56:24.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:23 smithi018 ceph-mon[29336]: 3.0 scrub ok 2024-09-10T18:56:24.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:23 smithi018 ceph-mon[29336]: pgmap v505: 168 pgs: 168 active+clean; 4.0 GiB data, 16 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 108 MiB/s wr, 1.74k op/s 2024-09-10T18:56:24.227 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:24.262 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:24.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:24.263 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:24.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:23 smithi103 ceph-mon[33163]: 5.19 scrub starts 2024-09-10T18:56:24.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:23 smithi103 ceph-mon[33163]: 5.19 scrub ok 2024-09-10T18:56:24.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:23 smithi103 ceph-mon[33163]: 4.f scrub starts 2024-09-10T18:56:24.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:23 smithi103 ceph-mon[33163]: 4.f scrub ok 2024-09-10T18:56:24.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:23 smithi103 ceph-mon[33163]: 3.0 scrub starts 2024-09-10T18:56:24.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:23 smithi103 ceph-mon[33163]: 3.0 scrub ok 2024-09-10T18:56:24.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:23 smithi103 ceph-mon[33163]: pgmap v505: 168 pgs: 168 active+clean; 4.0 GiB data, 16 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 108 MiB/s wr, 1.74k op/s 2024-09-10T18:56:24.364 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:24.387 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 36 16 70796 70780 122.863 67.1875 0.00814459 0.00813404 2024-09-10T18:56:24.403 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:24.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:24.404 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:24.505 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:24.537 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:24.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:24.538 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:24.639 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:24.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:24.708 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:24.708 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:24.808 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:24.852 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:24.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:24.854 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:24.955 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:24.995 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:24.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:24.996 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:25.097 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:25.126 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:25.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:25.127 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:25.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:24 smithi018 ceph-mon[40196]: 2.13 scrub starts 2024-09-10T18:56:25.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:24 smithi018 ceph-mon[40196]: 2.13 scrub ok 2024-09-10T18:56:25.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:24 smithi018 ceph-mon[40196]: 5.11 scrub starts 2024-09-10T18:56:25.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:24 smithi018 ceph-mon[40196]: 5.11 scrub ok 2024-09-10T18:56:25.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:24 smithi018 ceph-mon[40196]: 4.1c deep-scrub starts 2024-09-10T18:56:25.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:24 smithi018 ceph-mon[40196]: 4.1c deep-scrub ok 2024-09-10T18:56:25.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:24 smithi018 ceph-mon[40196]: 2.1e scrub starts 2024-09-10T18:56:25.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:24 smithi018 ceph-mon[40196]: 2.1e scrub ok 2024-09-10T18:56:25.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:24 smithi018 ceph-mon[29336]: 2.13 scrub starts 2024-09-10T18:56:25.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:24 smithi018 ceph-mon[29336]: 2.13 scrub ok 2024-09-10T18:56:25.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:24 smithi018 ceph-mon[29336]: 5.11 scrub starts 2024-09-10T18:56:25.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:24 smithi018 ceph-mon[29336]: 5.11 scrub ok 2024-09-10T18:56:25.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:24 smithi018 ceph-mon[29336]: 4.1c deep-scrub starts 2024-09-10T18:56:25.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:24 smithi018 ceph-mon[29336]: 4.1c deep-scrub ok 2024-09-10T18:56:25.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:24 smithi018 ceph-mon[29336]: 2.1e scrub starts 2024-09-10T18:56:25.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:24 smithi018 ceph-mon[29336]: 2.1e scrub ok 2024-09-10T18:56:25.228 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:25.263 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:25.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:25.265 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:25.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:24 smithi103 ceph-mon[33163]: 2.13 scrub starts 2024-09-10T18:56:25.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:24 smithi103 ceph-mon[33163]: 2.13 scrub ok 2024-09-10T18:56:25.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:24 smithi103 ceph-mon[33163]: 5.11 scrub starts 2024-09-10T18:56:25.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:24 smithi103 ceph-mon[33163]: 5.11 scrub ok 2024-09-10T18:56:25.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:24 smithi103 ceph-mon[33163]: 4.1c deep-scrub starts 2024-09-10T18:56:25.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:24 smithi103 ceph-mon[33163]: 4.1c deep-scrub ok 2024-09-10T18:56:25.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:24 smithi103 ceph-mon[33163]: 2.1e scrub starts 2024-09-10T18:56:25.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:24 smithi103 ceph-mon[33163]: 2.1e scrub ok 2024-09-10T18:56:25.366 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:25.387 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 37 16 72162 72146 121.85 85.375 0.0200913 0.00820192 2024-09-10T18:56:25.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:25.401 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:25.402 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:25.502 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:25.542 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:25.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:25.543 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:25.644 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:25.679 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:25.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:25.680 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:25.781 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:25.820 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:25.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:25.821 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:25.895 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:56:25.922 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:25.973 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:25.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:25.974 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:26.074 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:26.118 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:26.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:26.119 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:26.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:25 smithi018 ceph-mon[40196]: 2.0 scrub starts 2024-09-10T18:56:26.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:25 smithi018 ceph-mon[40196]: 5.b scrub starts 2024-09-10T18:56:26.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:25 smithi018 ceph-mon[40196]: 2.0 scrub ok 2024-09-10T18:56:26.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:25 smithi018 ceph-mon[40196]: 5.b scrub ok 2024-09-10T18:56:26.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:25 smithi018 ceph-mon[40196]: 2.e scrub starts 2024-09-10T18:56:26.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:25 smithi018 ceph-mon[40196]: 2.e scrub ok 2024-09-10T18:56:26.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:25 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:56:26.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:25 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:56:26.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:25 smithi018 ceph-mon[40196]: pgmap v506: 168 pgs: 168 active+clean; 4.1 GiB data, 16 GiB used, 609 GiB / 626 GiB avail; 1.1 KiB/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-10T18:56:26.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:25 smithi018 ceph-mon[29336]: 2.0 scrub starts 2024-09-10T18:56:26.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:25 smithi018 ceph-mon[29336]: 5.b scrub starts 2024-09-10T18:56:26.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:25 smithi018 ceph-mon[29336]: 2.0 scrub ok 2024-09-10T18:56:26.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:25 smithi018 ceph-mon[29336]: 5.b scrub ok 2024-09-10T18:56:26.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:25 smithi018 ceph-mon[29336]: 2.e scrub starts 2024-09-10T18:56:26.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:25 smithi018 ceph-mon[29336]: 2.e scrub ok 2024-09-10T18:56:26.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:25 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:56:26.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:25 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:56:26.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:25 smithi018 ceph-mon[29336]: pgmap v506: 168 pgs: 168 active+clean; 4.1 GiB data, 16 GiB used, 609 GiB / 626 GiB avail; 1.1 KiB/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-10T18:56:26.220 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:26.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:26.258 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:26.258 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:26.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:25 smithi103 ceph-mon[33163]: 2.0 scrub starts 2024-09-10T18:56:26.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:25 smithi103 ceph-mon[33163]: 5.b scrub starts 2024-09-10T18:56:26.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:25 smithi103 ceph-mon[33163]: 2.0 scrub ok 2024-09-10T18:56:26.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:25 smithi103 ceph-mon[33163]: 5.b scrub ok 2024-09-10T18:56:26.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:25 smithi103 ceph-mon[33163]: 2.e scrub starts 2024-09-10T18:56:26.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:25 smithi103 ceph-mon[33163]: 2.e scrub ok 2024-09-10T18:56:26.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:25 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:56:26.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:25 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:56:26.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:25 smithi103 ceph-mon[33163]: pgmap v506: 168 pgs: 168 active+clean; 4.1 GiB data, 16 GiB used, 609 GiB / 626 GiB avail; 1.1 KiB/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-10T18:56:26.359 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:26.388 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 38 16 73708 73692 121.186 96.625 0.0103365 0.00824686 2024-09-10T18:56:26.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:26.389 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:26.390 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:26.490 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:26.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:26.521 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:26.521 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:26.546 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-10T18:56:26.622 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:26.668 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:26.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:26.669 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:26.770 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:26.803 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:26.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:26.815 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:26.916 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:26.963 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:26.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:26.964 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:27.022 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:56:26 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:56:27.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:26 smithi103 ceph-mon[33163]: 4.16 scrub starts 2024-09-10T18:56:27.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:26 smithi103 ceph-mon[33163]: 4.16 scrub ok 2024-09-10T18:56:27.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:26 smithi103 ceph-mon[33163]: 2.17 scrub starts 2024-09-10T18:56:27.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:26 smithi103 ceph-mon[33163]: 2.17 scrub ok 2024-09-10T18:56:27.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:26 smithi103 ceph-mon[33163]: 2.10 scrub starts 2024-09-10T18:56:27.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:26 smithi103 ceph-mon[33163]: 2.10 scrub ok 2024-09-10T18:56:27.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:26 smithi103 ceph-mon[33163]: 5.17 scrub starts 2024-09-10T18:56:27.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:26 smithi103 ceph-mon[33163]: 5.17 scrub ok 2024-09-10T18:56:27.065 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:27.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:27.111 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:27.111 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:27.115 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:26 smithi018 ceph-mon[40196]: 4.16 scrub starts 2024-09-10T18:56:27.115 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:26 smithi018 ceph-mon[40196]: 4.16 scrub ok 2024-09-10T18:56:27.115 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:26 smithi018 ceph-mon[40196]: 2.17 scrub starts 2024-09-10T18:56:27.115 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:26 smithi018 ceph-mon[40196]: 2.17 scrub ok 2024-09-10T18:56:27.115 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:26 smithi018 ceph-mon[40196]: 2.10 scrub starts 2024-09-10T18:56:27.115 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:26 smithi018 ceph-mon[40196]: 2.10 scrub ok 2024-09-10T18:56:27.115 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:26 smithi018 ceph-mon[40196]: 5.17 scrub starts 2024-09-10T18:56:27.116 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:26 smithi018 ceph-mon[40196]: 5.17 scrub ok 2024-09-10T18:56:27.116 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:26 smithi018 ceph-mon[29336]: 4.16 scrub starts 2024-09-10T18:56:27.116 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:26 smithi018 ceph-mon[29336]: 4.16 scrub ok 2024-09-10T18:56:27.116 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:26 smithi018 ceph-mon[29336]: 2.17 scrub starts 2024-09-10T18:56:27.116 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:26 smithi018 ceph-mon[29336]: 2.17 scrub ok 2024-09-10T18:56:27.116 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:26 smithi018 ceph-mon[29336]: 2.10 scrub starts 2024-09-10T18:56:27.116 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:26 smithi018 ceph-mon[29336]: 2.10 scrub ok 2024-09-10T18:56:27.116 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:26 smithi018 ceph-mon[29336]: 5.17 scrub starts 2024-09-10T18:56:27.116 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:26 smithi018 ceph-mon[29336]: 5.17 scrub ok 2024-09-10T18:56:27.212 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:27.251 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:27.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:27.252 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:27.352 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:27.389 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 39 16 75012 74996 120.164 81.5 0.0083039 0.00831671 2024-09-10T18:56:27.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:27.412 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:27.412 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:27.513 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:27.551 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:56:27.555 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-09-10T18:56:27.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:27.557 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:27.557 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:27.658 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:27.705 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:27.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:27.706 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:27.807 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:27.850 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:27.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:27.851 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:27.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:27 smithi018 ceph-mon[29336]: 4.1a scrub starts 2024-09-10T18:56:27.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:27 smithi018 ceph-mon[29336]: 4.1a scrub ok 2024-09-10T18:56:27.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:27 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:27.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:27 smithi018 ceph-mon[29336]: pgmap v507: 168 pgs: 168 active+clean; 4.5 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 977 B/s rd, 84 MiB/s wr, 1.35k op/s 2024-09-10T18:56:27.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:27 smithi018 ceph-mon[40196]: 4.1a scrub starts 2024-09-10T18:56:27.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:27 smithi018 ceph-mon[40196]: 4.1a scrub ok 2024-09-10T18:56:27.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:27 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:27.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:27 smithi018 ceph-mon[40196]: pgmap v507: 168 pgs: 168 active+clean; 4.5 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 977 B/s rd, 84 MiB/s wr, 1.35k op/s 2024-09-10T18:56:27.953 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:27.979 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:27.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:27.980 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:28.081 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:28.111 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:56:28.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:28.121 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:28.121 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:28.222 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:28.257 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:28.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:28.258 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:28.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:27 smithi103 ceph-mon[33163]: 4.1a scrub starts 2024-09-10T18:56:28.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:27 smithi103 ceph-mon[33163]: 4.1a scrub ok 2024-09-10T18:56:28.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:27 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:28.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:27 smithi103 ceph-mon[33163]: pgmap v507: 168 pgs: 168 active+clean; 4.5 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 977 B/s rd, 84 MiB/s wr, 1.35k op/s 2024-09-10T18:56:28.358 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:28.389 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:2024-09-10T18:56:28.389087+0000 min lat: 0.00128005 max lat: 0.159628 avg lat: 0.00838332 2024-09-10T18:56:28.389 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T18:56:28.389 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 40 16 76329 76313 119.217 82.3125 0.00664204 0.00838332 2024-09-10T18:56:28.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:28.394 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:28.395 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:28.495 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:28.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:28.524 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:28.524 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:28.625 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:28.649 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:28.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:28.650 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:28.672 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":507,"stamp":"2024-09-10T18:56:27.244770+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4788984028,"num_objects":73261,"num_object_clones":0,"num_object_copies":219783,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73261,"num_whiteouts":0,"num_read":1801,"num_read_kb":1553,"num_write":73781,"num_write_kb":4676203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":2591,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66620,"ondisk_log_size":66620,"up":504,"acting":504,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":511,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":18329756,"kb_used_data":15869412,"kb_used_omap":14,"kb_used_meta":2460273,"kb_avail":637886308,"statfs":{"total":671965249536,"available":653195579392,"internally_reserved":0,"allocated":16250277888,"data_stored":16245681898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14826,"internal_metadata":2519320086},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":925892608,"num_objects":14128,"num_object_clones":0,"num_object_copies":42384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14128,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":14128,"num_write_kb":904192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.475999"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":254,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.759386+0000","last_change":"2024-09-10T18:56:14.876181+0000","last_active":"2024-09-10T18:56:15.759386+0000","last_peered":"2024-09-10T18:56:15.759386+0000","last_clean":"2024-09-10T18:56:15.759386+0000","last_became_active":"2024-09-10T18:45:24.265488+0000","last_became_peered":"2024-09-10T18:45:24.265488+0000","last_unstale":"2024-09-10T18:56:15.759386+0000","last_undegraded":"2024-09-10T18:56:15.759386+0000","last_fullsized":"2024-09-10T18:56:15.759386+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:14.875872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:14.875872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:42.755291+0000","scrub_duration":0.0037478699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":228,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.752495+0000","last_change":"2024-09-10T18:55:40.388273+0000","last_active":"2024-09-10T18:56:15.752495+0000","last_peered":"2024-09-10T18:56:15.752495+0000","last_clean":"2024-09-10T18:56:15.752495+0000","last_became_active":"2024-09-10T18:55:40.387702+0000","last_became_peered":"2024-09-10T18:55:40.387702+0000","last_unstale":"2024-09-10T18:56:15.752495+0000","last_undegraded":"2024-09-10T18:56:15.752495+0000","last_fullsized":"2024-09-10T18:56:15.752495+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:02.360246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.718270+0000","last_clean_scrub_stamp":"2024-09-10T18:52:02.360246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"232'27","reported_seq":349,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.758590+0000","last_change":"2024-09-10T18:56:02.821637+0000","last_active":"2024-09-10T18:56:15.758590+0000","last_peered":"2024-09-10T18:56:15.758590+0000","last_clean":"2024-09-10T18:56:15.758590+0000","last_became_active":"2024-09-10T18:55:40.394717+0000","last_became_peered":"2024-09-10T18:55:40.394717+0000","last_unstale":"2024-09-10T18:56:15.758590+0000","last_undegraded":"2024-09-10T18:56:15.758590+0000","last_fullsized":"2024-09-10T18:56:15.758590+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'27","last_scrub_stamp":"2024-09-10T18:56:02.821551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:02.821551+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:30.084130+0000","scrub_duration":0.0083110159999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":254,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.762655+0000","last_change":"2024-09-10T18:55:33.526359+0000","last_active":"2024-09-10T18:56:15.762655+0000","last_peered":"2024-09-10T18:56:15.762655+0000","last_clean":"2024-09-10T18:56:15.762655+0000","last_became_active":"2024-09-10T18:45:22.065619+0000","last_became_peered":"2024-09-10T18:45:22.065619+0000","last_unstale":"2024-09-10T18:56:15.762655+0000","last_undegraded":"2024-09-10T18:56:15.762655+0000","last_fullsized":"2024-09-10T18:56:15.762655+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:33.526302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.057932+0000","last_clean_scrub_stamp":"2024-09-10T18:55:33.526302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:53.822783+0000","scrub_duration":0.001517556,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":287,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:23.885678+0000","last_change":"2024-09-10T18:56:23.885640+0000","last_active":"2024-09-10T18:56:23.885678+0000","last_peered":"2024-09-10T18:56:23.885678+0000","last_clean":"2024-09-10T18:56:23.885678+0000","last_became_active":"2024-09-10T18:55:40.402140+0000","last_became_peered":"2024-09-10T18:55:40.402140+0000","last_unstale":"2024-09-10T18:56:23.885678+0000","last_undegraded":"2024-09-10T18:56:23.885678+0000","last_fullsized":"2024-09-10T18:56:23.885678+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:23.885562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:23.885562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:39.695161+0000","scrub_duration":0.0031033800000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"232'19","reported_seq":275,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.762713+0000","last_change":"2024-09-10T18:54:48.324177+0000","last_active":"2024-09-10T18:56:15.762713+0000","last_peered":"2024-09-10T18:56:15.762713+0000","last_clean":"2024-09-10T18:56:15.762713+0000","last_became_active":"2024-09-10T18:45:20.061266+0000","last_became_peered":"2024-09-10T18:45:20.061266+0000","last_unstale":"2024-09-10T18:56:15.762713+0000","last_undegraded":"2024-09-10T18:56:15.762713+0000","last_fullsized":"2024-09-10T18:56:15.762713+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'13","last_scrub_stamp":"2024-09-10T18:54:48.323985+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_clean_scrub_stamp":"2024-09-10T18:54:48.323985+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0080421699999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":101,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:22.163546+0000","last_change":"2024-09-10T18:56:22.163195+0000","last_active":"2024-09-10T18:56:22.163546+0000","last_peered":"2024-09-10T18:56:22.163546+0000","last_clean":"2024-09-10T18:56:22.163546+0000","last_became_active":"2024-09-10T18:52:58.104329+0000","last_became_peered":"2024-09-10T18:52:58.104329+0000","last_unstale":"2024-09-10T18:56:22.163546+0000","last_undegraded":"2024-09-10T18:56:22.163546+0000","last_fullsized":"2024-09-10T18:56:22.163546+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:22.163121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:01.135988+0000","last_clean_scrub_stamp":"2024-09-10T18:56:22.163121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:28.635584+0000","scrub_duration":0.002916054,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,7],"acting":[6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":242,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.762798+0000","last_change":"2024-09-10T18:54:22.051769+0000","last_active":"2024-09-10T18:56:15.762798+0000","last_peered":"2024-09-10T18:56:15.762798+0000","last_clean":"2024-09-10T18:56:15.762798+0000","last_became_active":"2024-09-10T18:45:24.081002+0000","last_became_peered":"2024-09-10T18:45:24.081002+0000","last_unstale":"2024-09-10T18:56:15.762798+0000","last_undegraded":"2024-09-10T18:56:15.762798+0000","last_fullsized":"2024-09-10T18:56:15.762798+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:22.051692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:54:22.051692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013425862,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":281,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.758253+0000","last_change":"2024-09-10T18:56:06.064127+0000","last_active":"2024-09-10T18:56:15.758253+0000","last_peered":"2024-09-10T18:56:15.758253+0000","last_clean":"2024-09-10T18:56:15.758253+0000","last_became_active":"2024-09-10T18:55:40.390416+0000","last_became_peered":"2024-09-10T18:55:40.390416+0000","last_unstale":"2024-09-10T18:56:15.758253+0000","last_undegraded":"2024-09-10T18:56:15.758253+0000","last_fullsized":"2024-09-10T18:56:15.758253+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:06.064071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:06.064071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:35.887377+0000","scrub_duration":0.0025507189999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"232'9","reported_seq":261,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.763092+0000","last_change":"2024-09-10T18:54:45.305098+0000","last_active":"2024-09-10T18:56:15.763092+0000","last_peered":"2024-09-10T18:56:15.763092+0000","last_clean":"2024-09-10T18:56:15.763092+0000","last_became_active":"2024-09-10T18:45:20.057567+0000","last_became_peered":"2024-09-10T18:45:20.057567+0000","last_unstale":"2024-09-10T18:56:15.763092+0000","last_undegraded":"2024-09-10T18:56:15.763092+0000","last_fullsized":"2024-09-10T18:56:15.763092+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"163'7","last_scrub_stamp":"2024-09-10T18:54:45.305023+0000","last_deep_scrub":"163'7","last_deep_scrub_stamp":"2024-09-10T18:51:54.261879+0000","last_clean_scrub_stamp":"2024-09-10T18:54:45.305023+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0088188659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":263,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.757762+0000","last_change":"2024-09-10T18:55:27.785971+0000","last_active":"2024-09-10T18:56:15.757762+0000","last_peered":"2024-09-10T18:56:15.757762+0000","last_clean":"2024-09-10T18:56:15.757762+0000","last_became_active":"2024-09-10T18:45:22.076575+0000","last_became_peered":"2024-09-10T18:45:22.076575+0000","last_unstale":"2024-09-10T18:56:15.757762+0000","last_undegraded":"2024-09-10T18:56:15.757762+0000","last_fullsized":"2024-09-10T18:56:15.757762+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:27.785914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:36.509971+0000","last_clean_scrub_stamp":"2024-09-10T18:55:27.785914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:43.986870+0000","scrub_duration":0.001369809,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":113,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.757523+0000","last_change":"2024-09-10T18:56:07.553751+0000","last_active":"2024-09-10T18:56:15.757523+0000","last_peered":"2024-09-10T18:56:15.757523+0000","last_clean":"2024-09-10T18:56:15.757523+0000","last_became_active":"2024-09-10T18:55:40.399236+0000","last_became_peered":"2024-09-10T18:55:40.399236+0000","last_unstale":"2024-09-10T18:56:15.757523+0000","last_undegraded":"2024-09-10T18:56:15.757523+0000","last_fullsized":"2024-09-10T18:56:15.757523+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:07.553696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:07.553696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:16.468669+0000","scrub_duration":0.0091042339999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":271,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.756391+0000","last_change":"2024-09-10T18:54:40.672896+0000","last_active":"2024-09-10T18:56:15.756391+0000","last_peered":"2024-09-10T18:56:15.756391+0000","last_clean":"2024-09-10T18:56:15.756391+0000","last_became_active":"2024-09-10T18:47:36.414393+0000","last_became_peered":"2024-09-10T18:47:36.414393+0000","last_unstale":"2024-09-10T18:56:15.756391+0000","last_undegraded":"2024-09-10T18:56:15.756391+0000","last_fullsized":"2024-09-10T18:56:15.756391+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:40.672810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:54:40.672810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0078573129999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"232'15","reported_seq":297,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.762892+0000","last_change":"2024-09-10T18:55:40.407535+0000","last_active":"2024-09-10T18:56:15.762892+0000","last_peered":"2024-09-10T18:56:15.762892+0000","last_clean":"2024-09-10T18:56:15.762892+0000","last_became_active":"2024-09-10T18:55:40.391571+0000","last_became_peered":"2024-09-10T18:55:40.391571+0000","last_unstale":"2024-09-10T18:56:15.762892+0000","last_undegraded":"2024-09-10T18:56:15.762892+0000","last_fullsized":"2024-09-10T18:56:15.762892+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'9","last_scrub_stamp":"2024-09-10T18:53:01.112771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:53:01.112771+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0055933110000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":273,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.752590+0000","last_change":"2024-09-10T18:56:00.729907+0000","last_active":"2024-09-10T18:56:15.752590+0000","last_peered":"2024-09-10T18:56:15.752590+0000","last_clean":"2024-09-10T18:56:15.752590+0000","last_became_active":"2024-09-10T18:55:40.393118+0000","last_became_peered":"2024-09-10T18:55:40.393118+0000","last_unstale":"2024-09-10T18:56:15.752590+0000","last_undegraded":"2024-09-10T18:56:15.752590+0000","last_fullsized":"2024-09-10T18:56:15.752590+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:00.729699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:00.729699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:24.926828+0000","scrub_duration":0.0047811370000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":203,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.752684+0000","last_change":"2024-09-10T18:55:40.397758+0000","last_active":"2024-09-10T18:56:15.752684+0000","last_peered":"2024-09-10T18:56:15.752684+0000","last_clean":"2024-09-10T18:56:15.752684+0000","last_became_active":"2024-09-10T18:55:40.397082+0000","last_became_peered":"2024-09-10T18:55:40.397082+0000","last_unstale":"2024-09-10T18:56:15.752684+0000","last_undegraded":"2024-09-10T18:56:15.752684+0000","last_fullsized":"2024-09-10T18:56:15.752684+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:59.425718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:59.425718+0000","last_clean_scrub_stamp":"2024-09-10T18:52:59.425718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"258'4698","reported_seq":4734,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:26.477298+0000","last_change":"2024-09-10T18:55:40.403238+0000","last_active":"2024-09-10T18:56:26.477298+0000","last_peered":"2024-09-10T18:56:26.477298+0000","last_clean":"2024-09-10T18:56:26.477298+0000","last_became_active":"2024-09-10T18:55:40.402912+0000","last_became_peered":"2024-09-10T18:55:40.402912+0000","last_unstale":"2024-09-10T18:56:26.477298+0000","last_undegraded":"2024-09-10T18:56:26.477298+0000","last_fullsized":"2024-09-10T18:56:26.477298+0000","mapping_epoch":241,"log_start":"249'900","ondisk_log_start":"249'900","created":237,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3798,"log_dups_size":0,"ondisk_log_size":3798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:04.717705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":307888128,"num_objects":4698,"num_object_clones":0,"num_object_copies":14094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4698,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4698,"num_write_kb":300672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"70'81","reported_seq":104,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.759110+0000","last_change":"2024-09-10T18:55:31.654161+0000","last_active":"2024-09-10T18:56:15.759110+0000","last_peered":"2024-09-10T18:56:15.759110+0000","last_clean":"2024-09-10T18:56:15.759110+0000","last_became_active":"2024-09-10T18:52:54.619039+0000","last_became_peered":"2024-09-10T18:52:54.619039+0000","last_unstale":"2024-09-10T18:56:15.759110+0000","last_undegraded":"2024-09-10T18:56:15.759110+0000","last_fullsized":"2024-09-10T18:56:15.759110+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:55:31.654090+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:55:31.654090+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:48.911364+0000","scrub_duration":0.0025082630000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"232'39","reported_seq":341,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.763509+0000","last_change":"2024-09-10T18:55:25.483970+0000","last_active":"2024-09-10T18:56:15.763509+0000","last_peered":"2024-09-10T18:56:15.763509+0000","last_clean":"2024-09-10T18:56:15.763509+0000","last_became_active":"2024-09-10T18:47:36.409300+0000","last_became_peered":"2024-09-10T18:47:36.409300+0000","last_unstale":"2024-09-10T18:56:15.763509+0000","last_undegraded":"2024-09-10T18:56:15.763509+0000","last_fullsized":"2024-09-10T18:56:15.763509+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'27","last_scrub_stamp":"2024-09-10T18:55:25.483919+0000","last_deep_scrub":"78'25","last_deep_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_clean_scrub_stamp":"2024-09-10T18:55:25.483919+0000","objects_scrubbed":8,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:38.819002+0000","scrub_duration":0.0017172610000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":264,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.757413+0000","last_change":"2024-09-10T18:55:30.719533+0000","last_active":"2024-09-10T18:56:15.757413+0000","last_peered":"2024-09-10T18:56:15.757413+0000","last_clean":"2024-09-10T18:56:15.757413+0000","last_became_active":"2024-09-10T18:45:17.963732+0000","last_became_peered":"2024-09-10T18:45:17.963732+0000","last_unstale":"2024-09-10T18:56:15.757413+0000","last_undegraded":"2024-09-10T18:56:15.757413+0000","last_fullsized":"2024-09-10T18:56:15.757413+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:30.719475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:55:30.719475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:56.769674+0000","scrub_duration":0.00119068,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":263,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.757945+0000","last_change":"2024-09-10T18:56:09.623123+0000","last_active":"2024-09-10T18:56:15.757945+0000","last_peered":"2024-09-10T18:56:15.757945+0000","last_clean":"2024-09-10T18:56:15.757945+0000","last_became_active":"2024-09-10T18:55:40.392449+0000","last_became_peered":"2024-09-10T18:55:40.392449+0000","last_unstale":"2024-09-10T18:56:15.757945+0000","last_undegraded":"2024-09-10T18:56:15.757945+0000","last_fullsized":"2024-09-10T18:56:15.757945+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:09.622841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:04.512425+0000","last_clean_scrub_stamp":"2024-09-10T18:56:09.622841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:35.116868+0000","scrub_duration":0.010581503000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":160,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.752626+0000","last_change":"2024-09-10T18:55:40.388451+0000","last_active":"2024-09-10T18:56:15.752626+0000","last_peered":"2024-09-10T18:56:15.752626+0000","last_clean":"2024-09-10T18:56:15.752626+0000","last_became_active":"2024-09-10T18:55:40.388154+0000","last_became_peered":"2024-09-10T18:55:40.388154+0000","last_unstale":"2024-09-10T18:56:15.752626+0000","last_undegraded":"2024-09-10T18:56:15.752626+0000","last_fullsized":"2024-09-10T18:56:15.752626+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:05.317828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:53:05.317828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015035510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1","version":"258'4580","reported_seq":4608,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:26.609789+0000","last_change":"2024-09-10T18:55:36.354324+0000","last_active":"2024-09-10T18:56:26.609789+0000","last_peered":"2024-09-10T18:56:26.609789+0000","last_clean":"2024-09-10T18:56:26.609789+0000","last_became_active":"2024-09-10T18:55:36.354136+0000","last_became_peered":"2024-09-10T18:55:36.354136+0000","last_unstale":"2024-09-10T18:56:26.609789+0000","last_undegraded":"2024-09-10T18:56:26.609789+0000","last_fullsized":"2024-09-10T18:56:26.609789+0000","mapping_epoch":237,"log_start":"249'800","ondisk_log_start":"249'800","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3780,"log_dups_size":0,"ondisk_log_size":3780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:57.105359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":300154880,"num_objects":4580,"num_object_clones":0,"num_object_copies":13740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4580,"num_write_kb":293120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.9","version":"70'81","reported_seq":325,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.756330+0000","last_change":"2024-09-10T18:54:45.753884+0000","last_active":"2024-09-10T18:56:15.756330+0000","last_peered":"2024-09-10T18:56:15.756330+0000","last_clean":"2024-09-10T18:56:15.756330+0000","last_became_active":"2024-09-10T18:50:08.399879+0000","last_became_peered":"2024-09-10T18:50:08.399879+0000","last_unstale":"2024-09-10T18:56:15.756330+0000","last_undegraded":"2024-09-10T18:56:15.756330+0000","last_fullsized":"2024-09-10T18:56:15.756330+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:54:45.753792+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:54:45.753792+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0072404879999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"232'25","reported_seq":281,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.763577+0000","last_change":"2024-09-10T18:53:06.056470+0000","last_active":"2024-09-10T18:56:15.763577+0000","last_peered":"2024-09-10T18:56:15.763577+0000","last_clean":"2024-09-10T18:56:15.763577+0000","last_became_active":"2024-09-10T18:45:20.053428+0000","last_became_peered":"2024-09-10T18:45:20.053428+0000","last_unstale":"2024-09-10T18:56:15.763577+0000","last_undegraded":"2024-09-10T18:56:15.763577+0000","last_fullsized":"2024-09-10T18:56:15.763577+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"116'15","last_scrub_stamp":"2024-09-10T18:53:06.056392+0000","last_deep_scrub":"116'15","last_deep_scrub_stamp":"2024-09-10T18:49:12.374527+0000","last_clean_scrub_stamp":"2024-09-10T18:53:06.056392+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0078605010000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":250,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.758759+0000","last_change":"2024-09-10T18:52:45.139636+0000","last_active":"2024-09-10T18:56:15.758759+0000","last_peered":"2024-09-10T18:56:15.758759+0000","last_clean":"2024-09-10T18:56:15.758759+0000","last_became_active":"2024-09-10T18:45:17.966833+0000","last_became_peered":"2024-09-10T18:45:17.966833+0000","last_unstale":"2024-09-10T18:56:15.758759+0000","last_undegraded":"2024-09-10T18:56:15.758759+0000","last_fullsized":"2024-09-10T18:56:15.758759+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:45.139534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:52:45.139534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002710269,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"242'12","reported_seq":372,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:21.204732+0000","last_change":"2024-09-10T18:55:40.390386+0000","last_active":"2024-09-10T18:56:21.204732+0000","last_peered":"2024-09-10T18:56:21.204732+0000","last_clean":"2024-09-10T18:56:21.204732+0000","last_became_active":"2024-09-10T18:55:40.389996+0000","last_became_peered":"2024-09-10T18:55:40.389996+0000","last_unstale":"2024-09-10T18:56:21.204732+0000","last_undegraded":"2024-09-10T18:56:21.204732+0000","last_fullsized":"2024-09-10T18:56:21.204732+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"202'10","last_scrub_stamp":"2024-09-10T18:53:45.547686+0000","last_deep_scrub":"202'10","last_deep_scrub_stamp":"2024-09-10T18:53:45.547686+0000","last_clean_scrub_stamp":"2024-09-10T18:53:45.547686+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.121847446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":272,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.763591+0000","last_change":"2024-09-10T18:55:37.565414+0000","last_active":"2024-09-10T18:56:15.763591+0000","last_peered":"2024-09-10T18:56:15.763591+0000","last_clean":"2024-09-10T18:56:15.763591+0000","last_became_active":"2024-09-10T18:52:54.599288+0000","last_became_peered":"2024-09-10T18:52:54.599288+0000","last_unstale":"2024-09-10T18:56:15.763591+0000","last_undegraded":"2024-09-10T18:56:15.763591+0000","last_fullsized":"2024-09-10T18:56:15.763591+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:37.565359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:32.043666+0000","last_clean_scrub_stamp":"2024-09-10T18:55:37.565359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:04.559538+0000","scrub_duration":0.0038365510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"258'4548","reported_seq":4576,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:26.587153+0000","last_change":"2024-09-10T18:55:36.356387+0000","last_active":"2024-09-10T18:56:26.587153+0000","last_peered":"2024-09-10T18:56:26.587153+0000","last_clean":"2024-09-10T18:56:26.587153+0000","last_became_active":"2024-09-10T18:55:36.355584+0000","last_became_peered":"2024-09-10T18:55:36.355584+0000","last_unstale":"2024-09-10T18:56:26.587153+0000","last_undegraded":"2024-09-10T18:56:26.587153+0000","last_fullsized":"2024-09-10T18:56:26.587153+0000","mapping_epoch":237,"log_start":"249'700","ondisk_log_start":"249'700","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3848,"log_dups_size":0,"ondisk_log_size":3848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:52.013473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":298057728,"num_objects":4548,"num_object_clones":0,"num_object_copies":13644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4548,"num_write_kb":291072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.a","version":"70'81","reported_seq":171,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.752080+0000","last_change":"2024-09-10T18:55:37.921571+0000","last_active":"2024-09-10T18:56:15.752080+0000","last_peered":"2024-09-10T18:56:15.752080+0000","last_clean":"2024-09-10T18:56:15.752080+0000","last_became_active":"2024-09-10T18:50:08.398345+0000","last_became_peered":"2024-09-10T18:50:08.398345+0000","last_unstale":"2024-09-10T18:56:15.752080+0000","last_undegraded":"2024-09-10T18:56:15.752080+0000","last_fullsized":"2024-09-10T18:56:15.752080+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:55:37.921519+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:55:37.921519+0000","last_clean_scrub_stamp":"2024-09-10T18:55:37.921519+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:05.293052+0000","scrub_duration":0.002222407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"232'34","reported_seq":275,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:22.727781+0000","last_change":"2024-09-10T18:56:22.727738+0000","last_active":"2024-09-10T18:56:22.727781+0000","last_peered":"2024-09-10T18:56:22.727781+0000","last_clean":"2024-09-10T18:56:22.727781+0000","last_became_active":"2024-09-10T18:46:56.742946+0000","last_became_peered":"2024-09-10T18:46:56.742946+0000","last_unstale":"2024-09-10T18:56:22.727781+0000","last_undegraded":"2024-09-10T18:56:22.727781+0000","last_fullsized":"2024-09-10T18:56:22.727781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'34","last_scrub_stamp":"2024-09-10T18:56:22.727671+0000","last_deep_scrub":"141'22","last_deep_scrub_stamp":"2024-09-10T18:50:57.119152+0000","last_clean_scrub_stamp":"2024-09-10T18:56:22.727671+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:28.158538+0000","scrub_duration":0.0079020229999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":263,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.757827+0000","last_change":"2024-09-10T18:56:14.634516+0000","last_active":"2024-09-10T18:56:15.757827+0000","last_peered":"2024-09-10T18:56:15.757827+0000","last_clean":"2024-09-10T18:56:15.757827+0000","last_became_active":"2024-09-10T18:45:17.965409+0000","last_became_peered":"2024-09-10T18:45:17.965409+0000","last_unstale":"2024-09-10T18:56:15.757827+0000","last_undegraded":"2024-09-10T18:56:15.757827+0000","last_fullsized":"2024-09-10T18:56:15.757827+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:14.634438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_clean_scrub_stamp":"2024-09-10T18:56:14.634438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:42.900347+0000","scrub_duration":0.0058886299999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"242'10","reported_seq":415,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:21.204435+0000","last_change":"2024-09-10T18:56:19.581070+0000","last_active":"2024-09-10T18:56:21.204435+0000","last_peered":"2024-09-10T18:56:21.204435+0000","last_clean":"2024-09-10T18:56:21.204435+0000","last_became_active":"2024-09-10T18:55:40.397246+0000","last_became_peered":"2024-09-10T18:55:40.397246+0000","last_unstale":"2024-09-10T18:56:21.204435+0000","last_undegraded":"2024-09-10T18:56:21.204435+0000","last_fullsized":"2024-09-10T18:56:21.204435+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"242'10","last_scrub_stamp":"2024-09-10T18:56:19.581012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:19.581012+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:38.538372+0000","scrub_duration":0.0013423529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":271,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.753135+0000","last_change":"2024-09-10T18:56:01.731623+0000","last_active":"2024-09-10T18:56:15.753135+0000","last_peered":"2024-09-10T18:56:15.753135+0000","last_clean":"2024-09-10T18:56:15.753135+0000","last_became_active":"2024-09-10T18:55:40.392003+0000","last_became_peered":"2024-09-10T18:55:40.392003+0000","last_unstale":"2024-09-10T18:56:15.753135+0000","last_undegraded":"2024-09-10T18:56:15.753135+0000","last_fullsized":"2024-09-10T18:56:15.753135+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:01.731528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:54.218602+0000","last_clean_scrub_stamp":"2024-09-10T18:56:01.731528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:03.271161+0000","scrub_duration":0.003574701,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"258'4694","reported_seq":4723,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:26.264665+0000","last_change":"2024-09-10T18:55:36.355594+0000","last_active":"2024-09-10T18:56:26.264665+0000","last_peered":"2024-09-10T18:56:26.264665+0000","last_clean":"2024-09-10T18:56:26.264665+0000","last_became_active":"2024-09-10T18:55:36.355300+0000","last_became_peered":"2024-09-10T18:55:36.355300+0000","last_unstale":"2024-09-10T18:56:26.264665+0000","last_undegraded":"2024-09-10T18:56:26.264665+0000","last_fullsized":"2024-09-10T18:56:26.264665+0000","mapping_epoch":237,"log_start":"249'900","ondisk_log_start":"249'900","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3794,"log_dups_size":0,"ondisk_log_size":3794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:49.815521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":307625984,"num_objects":4694,"num_object_clones":0,"num_object_copies":14082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4694,"num_write_kb":300416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.b","version":"70'81","reported_seq":203,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.763214+0000","last_change":"2024-09-10T18:55:40.402680+0000","last_active":"2024-09-10T18:56:15.763214+0000","last_peered":"2024-09-10T18:56:15.763214+0000","last_clean":"2024-09-10T18:56:15.763214+0000","last_became_active":"2024-09-10T18:55:40.401643+0000","last_became_peered":"2024-09-10T18:55:40.401643+0000","last_unstale":"2024-09-10T18:56:15.763214+0000","last_undegraded":"2024-09-10T18:56:15.763214+0000","last_fullsized":"2024-09-10T18:56:15.763214+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:53:04.265619+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:53:04.265619+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016066791,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"232'24","reported_seq":358,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.763719+0000","last_change":"2024-09-10T18:54:49.297032+0000","last_active":"2024-09-10T18:56:15.763719+0000","last_peered":"2024-09-10T18:56:15.763719+0000","last_clean":"2024-09-10T18:56:15.763719+0000","last_became_active":"2024-09-10T18:47:57.106935+0000","last_became_peered":"2024-09-10T18:47:57.106935+0000","last_unstale":"2024-09-10T18:56:15.763719+0000","last_undegraded":"2024-09-10T18:56:15.763719+0000","last_fullsized":"2024-09-10T18:56:15.763719+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"224'16","last_scrub_stamp":"2024-09-10T18:54:49.296957+0000","last_deep_scrub":"224'16","last_deep_scrub_stamp":"2024-09-10T18:54:49.296957+0000","last_clean_scrub_stamp":"2024-09-10T18:54:49.296957+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014223285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":278,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.755581+0000","last_change":"2024-09-10T18:55:40.397594+0000","last_active":"2024-09-10T18:56:15.755581+0000","last_peered":"2024-09-10T18:56:15.755581+0000","last_clean":"2024-09-10T18:56:15.755581+0000","last_became_active":"2024-09-10T18:55:40.396954+0000","last_became_peered":"2024-09-10T18:55:40.396954+0000","last_unstale":"2024-09-10T18:56:15.755581+0000","last_undegraded":"2024-09-10T18:56:15.755581+0000","last_fullsized":"2024-09-10T18:56:15.755581+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:43.419135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:53:43.419135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053276879999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":259,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.752542+0000","last_change":"2024-09-10T18:55:26.909204+0000","last_active":"2024-09-10T18:56:15.752542+0000","last_peered":"2024-09-10T18:56:15.752542+0000","last_clean":"2024-09-10T18:56:15.752542+0000","last_became_active":"2024-09-10T18:45:22.068918+0000","last_became_peered":"2024-09-10T18:45:22.068918+0000","last_unstale":"2024-09-10T18:56:15.752542+0000","last_undegraded":"2024-09-10T18:56:15.752542+0000","last_fullsized":"2024-09-10T18:56:15.752542+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:26.909143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_clean_scrub_stamp":"2024-09-10T18:55:26.909143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:32.278969+0000","scrub_duration":0.0013975070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":244,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.762857+0000","last_change":"2024-09-10T18:54:51.399064+0000","last_active":"2024-09-10T18:56:15.762857+0000","last_peered":"2024-09-10T18:56:15.762857+0000","last_clean":"2024-09-10T18:56:15.762857+0000","last_became_active":"2024-09-10T18:45:24.264538+0000","last_became_peered":"2024-09-10T18:45:24.264538+0000","last_unstale":"2024-09-10T18:56:15.762857+0000","last_undegraded":"2024-09-10T18:56:15.762857+0000","last_fullsized":"2024-09-10T18:56:15.762857+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:51.398981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:51.398981+0000","last_clean_scrub_stamp":"2024-09-10T18:54:51.398981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:19.907055+0000","scrub_duration":0.0027589680000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"258'4652","reported_seq":4680,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:26.557005+0000","last_change":"2024-09-10T18:55:36.355579+0000","last_active":"2024-09-10T18:56:26.557005+0000","last_peered":"2024-09-10T18:56:26.557005+0000","last_clean":"2024-09-10T18:56:26.557005+0000","last_became_active":"2024-09-10T18:55:36.355193+0000","last_became_peered":"2024-09-10T18:55:36.355193+0000","last_unstale":"2024-09-10T18:56:26.557005+0000","last_undegraded":"2024-09-10T18:56:26.557005+0000","last_fullsized":"2024-09-10T18:56:26.557005+0000","mapping_epoch":237,"log_start":"249'900","ondisk_log_start":"249'900","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3752,"log_dups_size":0,"ondisk_log_size":3752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:39.304440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":304873472,"num_objects":4652,"num_object_clones":0,"num_object_copies":13956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4652,"num_write_kb":297728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"70'81","reported_seq":182,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.763097+0000","last_change":"2024-09-10T18:54:42.406852+0000","last_active":"2024-09-10T18:56:15.763097+0000","last_peered":"2024-09-10T18:56:15.763097+0000","last_clean":"2024-09-10T18:56:15.763097+0000","last_became_active":"2024-09-10T18:52:54.622187+0000","last_became_peered":"2024-09-10T18:52:54.622187+0000","last_unstale":"2024-09-10T18:56:15.763097+0000","last_undegraded":"2024-09-10T18:56:15.763097+0000","last_fullsized":"2024-09-10T18:56:15.763097+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:54:42.406761+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:54:42.406761+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037296899999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"232'35","reported_seq":295,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.762755+0000","last_change":"2024-09-10T18:52:59.095130+0000","last_active":"2024-09-10T18:56:15.762755+0000","last_peered":"2024-09-10T18:56:15.762755+0000","last_clean":"2024-09-10T18:56:15.762755+0000","last_became_active":"2024-09-10T18:45:20.052243+0000","last_became_peered":"2024-09-10T18:45:20.052243+0000","last_unstale":"2024-09-10T18:56:15.762755+0000","last_undegraded":"2024-09-10T18:56:15.762755+0000","last_fullsized":"2024-09-10T18:56:15.762755+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"143'21","last_scrub_stamp":"2024-09-10T18:52:59.095048+0000","last_deep_scrub":"143'21","last_deep_scrub_stamp":"2024-09-10T18:51:02.540216+0000","last_clean_scrub_stamp":"2024-09-10T18:52:59.095048+0000","objects_scrubbed":7,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013944524,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":270,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.756743+0000","last_change":"2024-09-10T18:54:53.087341+0000","last_active":"2024-09-10T18:56:15.756743+0000","last_peered":"2024-09-10T18:56:15.756743+0000","last_clean":"2024-09-10T18:56:15.756743+0000","last_became_active":"2024-09-10T18:47:36.424466+0000","last_became_peered":"2024-09-10T18:47:36.424466+0000","last_unstale":"2024-09-10T18:56:15.756743+0000","last_undegraded":"2024-09-10T18:56:15.756743+0000","last_fullsized":"2024-09-10T18:56:15.756743+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:53.087262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_clean_scrub_stamp":"2024-09-10T18:54:53.087262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010484613,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":398,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:21.332979+0000","last_change":"2024-09-10T18:56:17.914259+0000","last_active":"2024-09-10T18:56:21.332979+0000","last_peered":"2024-09-10T18:56:21.332979+0000","last_clean":"2024-09-10T18:56:21.332979+0000","last_became_active":"2024-09-10T18:45:22.074540+0000","last_became_peered":"2024-09-10T18:45:22.074540+0000","last_unstale":"2024-09-10T18:56:21.332979+0000","last_undegraded":"2024-09-10T18:56:21.332979+0000","last_fullsized":"2024-09-10T18:56:21.332979+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:56:17.914175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:17.914175+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:25.944051+0000","scrub_duration":0.0026666329999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":275,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.757085+0000","last_change":"2024-09-10T18:54:49.759104+0000","last_active":"2024-09-10T18:56:15.757085+0000","last_peered":"2024-09-10T18:56:15.757085+0000","last_clean":"2024-09-10T18:56:15.757085+0000","last_became_active":"2024-09-10T18:47:36.414665+0000","last_became_peered":"2024-09-10T18:47:36.414665+0000","last_unstale":"2024-09-10T18:56:15.757085+0000","last_undegraded":"2024-09-10T18:56:15.757085+0000","last_fullsized":"2024-09-10T18:56:15.757085+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:49.758989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:54:49.758989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010517354,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.d","version":"258'4469","reported_seq":4498,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:26.240926+0000","last_change":"2024-09-10T18:55:36.359401+0000","last_active":"2024-09-10T18:56:26.240926+0000","last_peered":"2024-09-10T18:56:26.240926+0000","last_clean":"2024-09-10T18:56:26.240926+0000","last_became_active":"2024-09-10T18:55:36.359124+0000","last_became_peered":"2024-09-10T18:55:36.359124+0000","last_unstale":"2024-09-10T18:56:26.240926+0000","last_undegraded":"2024-09-10T18:56:26.240926+0000","last_fullsized":"2024-09-10T18:56:26.240926+0000","mapping_epoch":237,"log_start":"249'700","ondisk_log_start":"249'700","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3769,"log_dups_size":0,"ondisk_log_size":3769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:51.170482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":292880384,"num_objects":4469,"num_object_clones":0,"num_object_copies":13407,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4469,"num_write_kb":286016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"70'81","reported_seq":166,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.763142+0000","last_change":"2024-09-10T18:52:54.120550+0000","last_active":"2024-09-10T18:56:15.763142+0000","last_peered":"2024-09-10T18:56:15.763142+0000","last_clean":"2024-09-10T18:56:15.763142+0000","last_became_active":"2024-09-10T18:52:14.584798+0000","last_became_peered":"2024-09-10T18:52:14.584798+0000","last_unstale":"2024-09-10T18:56:15.763142+0000","last_undegraded":"2024-09-10T18:56:15.763142+0000","last_fullsized":"2024-09-10T18:56:15.763142+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:52:54.120497+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:52:54.120497+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001089507,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"232'30","reported_seq":340,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.876557+0000","last_change":"2024-09-10T18:56:15.876275+0000","last_active":"2024-09-10T18:56:15.876557+0000","last_peered":"2024-09-10T18:56:15.876557+0000","last_clean":"2024-09-10T18:56:15.876557+0000","last_became_active":"2024-09-10T18:55:40.394879+0000","last_became_peered":"2024-09-10T18:55:40.394879+0000","last_unstale":"2024-09-10T18:56:15.876557+0000","last_undegraded":"2024-09-10T18:56:15.876557+0000","last_fullsized":"2024-09-10T18:56:15.876557+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'30","last_scrub_stamp":"2024-09-10T18:56:15.875924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:15.875924+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:32.745165+0000","scrub_duration":0.0075153879999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":253,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.768645+0000","last_change":"2024-09-10T18:54:46.142189+0000","last_active":"2024-09-10T18:56:15.768645+0000","last_peered":"2024-09-10T18:56:15.768645+0000","last_clean":"2024-09-10T18:56:15.768645+0000","last_became_active":"2024-09-10T18:45:17.961695+0000","last_became_peered":"2024-09-10T18:45:17.961695+0000","last_unstale":"2024-09-10T18:56:15.768645+0000","last_undegraded":"2024-09-10T18:56:15.768645+0000","last_fullsized":"2024-09-10T18:56:15.768645+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:46.142129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:46.142129+0000","last_clean_scrub_stamp":"2024-09-10T18:54:46.142129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015906060000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":254,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.762619+0000","last_change":"2024-09-10T18:55:29.533227+0000","last_active":"2024-09-10T18:56:15.762619+0000","last_peered":"2024-09-10T18:56:15.762619+0000","last_clean":"2024-09-10T18:56:15.762619+0000","last_became_active":"2024-09-10T18:45:22.074012+0000","last_became_peered":"2024-09-10T18:45:22.074012+0000","last_unstale":"2024-09-10T18:56:15.762619+0000","last_undegraded":"2024-09-10T18:56:15.762619+0000","last_fullsized":"2024-09-10T18:56:15.762619+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:29.533177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:55:29.533177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:56.213796+0000","scrub_duration":0.0012137999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":252,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.762906+0000","last_change":"2024-09-10T18:54:25.102613+0000","last_active":"2024-09-10T18:56:15.762906+0000","last_peered":"2024-09-10T18:56:15.762906+0000","last_clean":"2024-09-10T18:56:15.762906+0000","last_became_active":"2024-09-10T18:47:36.410836+0000","last_became_peered":"2024-09-10T18:47:36.410836+0000","last_unstale":"2024-09-10T18:56:15.762906+0000","last_undegraded":"2024-09-10T18:56:15.762906+0000","last_fullsized":"2024-09-10T18:56:15.762906+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:25.102358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_clean_scrub_stamp":"2024-09-10T18:54:25.102358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060015040000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.e","version":"258'4572","reported_seq":4601,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:26.266202+0000","last_change":"2024-09-10T18:55:36.358310+0000","last_active":"2024-09-10T18:56:26.266202+0000","last_peered":"2024-09-10T18:56:26.266202+0000","last_clean":"2024-09-10T18:56:26.266202+0000","last_became_active":"2024-09-10T18:55:36.357714+0000","last_became_peered":"2024-09-10T18:55:36.357714+0000","last_unstale":"2024-09-10T18:56:26.266202+0000","last_undegraded":"2024-09-10T18:56:26.266202+0000","last_fullsized":"2024-09-10T18:56:26.266202+0000","mapping_epoch":237,"log_start":"249'800","ondisk_log_start":"249'800","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3772,"log_dups_size":0,"ondisk_log_size":3772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:44.330187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":299630592,"num_objects":4572,"num_object_clones":0,"num_object_copies":13716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4572,"num_write_kb":292608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"70'81","reported_seq":263,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.756967+0000","last_change":"2024-09-10T18:56:12.184278+0000","last_active":"2024-09-10T18:56:15.756967+0000","last_peered":"2024-09-10T18:56:15.756967+0000","last_clean":"2024-09-10T18:56:15.756967+0000","last_became_active":"2024-09-10T18:55:40.396828+0000","last_became_peered":"2024-09-10T18:55:40.396828+0000","last_unstale":"2024-09-10T18:56:15.756967+0000","last_undegraded":"2024-09-10T18:56:15.756967+0000","last_fullsized":"2024-09-10T18:56:15.756967+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:12.184127+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:12.184127+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:33.805848+0000","scrub_duration":0.0040368189999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"232'29","reported_seq":277,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.753029+0000","last_change":"2024-09-10T18:55:40.411118+0000","last_active":"2024-09-10T18:56:15.753029+0000","last_peered":"2024-09-10T18:56:15.753029+0000","last_clean":"2024-09-10T18:56:15.753029+0000","last_became_active":"2024-09-10T18:55:40.391264+0000","last_became_peered":"2024-09-10T18:55:40.391264+0000","last_unstale":"2024-09-10T18:56:15.753029+0000","last_undegraded":"2024-09-10T18:56:15.753029+0000","last_fullsized":"2024-09-10T18:56:15.753029+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:53:38.384804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:53:38.384804+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":278,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.758313+0000","last_change":"2024-09-10T18:56:01.022373+0000","last_active":"2024-09-10T18:56:15.758313+0000","last_peered":"2024-09-10T18:56:15.758313+0000","last_clean":"2024-09-10T18:56:15.758313+0000","last_became_active":"2024-09-10T18:55:40.390100+0000","last_became_peered":"2024-09-10T18:55:40.390100+0000","last_unstale":"2024-09-10T18:56:15.758313+0000","last_undegraded":"2024-09-10T18:56:15.758313+0000","last_fullsized":"2024-09-10T18:56:15.758313+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:01.022282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:48.196065+0000","last_clean_scrub_stamp":"2024-09-10T18:56:01.022282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:30.771501+0000","scrub_duration":0.0051685999999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":273,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.755105+0000","last_change":"2024-09-10T18:56:02.162637+0000","last_active":"2024-09-10T18:56:15.755105+0000","last_peered":"2024-09-10T18:56:15.755105+0000","last_clean":"2024-09-10T18:56:15.755105+0000","last_became_active":"2024-09-10T18:55:40.387857+0000","last_became_peered":"2024-09-10T18:55:40.387857+0000","last_unstale":"2024-09-10T18:56:15.755105+0000","last_undegraded":"2024-09-10T18:56:15.755105+0000","last_fullsized":"2024-09-10T18:56:15.755105+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:02.162551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:02.162551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:15.298292+0000","scrub_duration":0.0027989849999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":254,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.757608+0000","last_change":"2024-09-10T18:55:30.482717+0000","last_active":"2024-09-10T18:56:15.757608+0000","last_peered":"2024-09-10T18:56:15.757608+0000","last_clean":"2024-09-10T18:56:15.757608+0000","last_became_active":"2024-09-10T18:47:36.422295+0000","last_became_peered":"2024-09-10T18:47:36.422295+0000","last_unstale":"2024-09-10T18:56:15.757608+0000","last_undegraded":"2024-09-10T18:56:15.757608+0000","last_fullsized":"2024-09-10T18:56:15.757608+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:30.482659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:49:18.281230+0000","last_clean_scrub_stamp":"2024-09-10T18:55:30.482659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:42.005520+0000","scrub_duration":0.0015073689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.f","version":"258'4598","reported_seq":4627,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:26.380996+0000","last_change":"2024-09-10T18:55:36.358334+0000","last_active":"2024-09-10T18:56:26.380996+0000","last_peered":"2024-09-10T18:56:26.380996+0000","last_clean":"2024-09-10T18:56:26.380996+0000","last_became_active":"2024-09-10T18:55:36.358064+0000","last_became_peered":"2024-09-10T18:55:36.358064+0000","last_unstale":"2024-09-10T18:56:26.380996+0000","last_undegraded":"2024-09-10T18:56:26.380996+0000","last_fullsized":"2024-09-10T18:56:26.380996+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":4598,"log_dups_size":0,"ondisk_log_size":4598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:01.132365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":301334528,"num_objects":4598,"num_object_clones":0,"num_object_copies":13794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4598,"num_write_kb":294272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"70'81","reported_seq":246,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.755419+0000","last_change":"2024-09-10T18:54:46.030156+0000","last_active":"2024-09-10T18:56:15.755419+0000","last_peered":"2024-09-10T18:56:15.755419+0000","last_clean":"2024-09-10T18:56:15.755419+0000","last_became_active":"2024-09-10T18:52:14.582796+0000","last_became_peered":"2024-09-10T18:52:14.582796+0000","last_unstale":"2024-09-10T18:56:15.755419+0000","last_undegraded":"2024-09-10T18:56:15.755419+0000","last_fullsized":"2024-09-10T18:56:15.755419+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:54:46.030076+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:54:46.030076+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012983280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"232'25","reported_seq":335,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.758635+0000","last_change":"2024-09-10T18:55:40.415292+0000","last_active":"2024-09-10T18:56:15.758635+0000","last_peered":"2024-09-10T18:56:15.758635+0000","last_clean":"2024-09-10T18:56:15.758635+0000","last_became_active":"2024-09-10T18:55:40.400777+0000","last_became_peered":"2024-09-10T18:55:40.400777+0000","last_unstale":"2024-09-10T18:56:15.758635+0000","last_undegraded":"2024-09-10T18:56:15.758635+0000","last_fullsized":"2024-09-10T18:56:15.758635+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'15","last_scrub_stamp":"2024-09-10T18:52:04.182782+0000","last_deep_scrub":"83'15","last_deep_scrub_stamp":"2024-09-10T18:46:58.436473+0000","last_clean_scrub_stamp":"2024-09-10T18:52:04.182782+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0049024990000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":277,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.760274+0000","last_change":"2024-09-10T18:55:40.392113+0000","last_active":"2024-09-10T18:56:15.760274+0000","last_peered":"2024-09-10T18:56:15.760274+0000","last_clean":"2024-09-10T18:56:15.760274+0000","last_became_active":"2024-09-10T18:55:40.391805+0000","last_became_peered":"2024-09-10T18:55:40.391805+0000","last_unstale":"2024-09-10T18:56:15.760274+0000","last_undegraded":"2024-09-10T18:56:15.760274+0000","last_fullsized":"2024-09-10T18:56:15.760274+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:36.259868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:05.140027+0000","last_clean_scrub_stamp":"2024-09-10T18:53:36.259868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034952049999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"258'12","reported_seq":341,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:21.099131+0000","last_change":"2024-09-10T18:56:21.098913+0000","last_active":"2024-09-10T18:56:21.099131+0000","last_peered":"2024-09-10T18:56:21.099131+0000","last_clean":"2024-09-10T18:56:21.099131+0000","last_became_active":"2024-09-10T18:55:40.391094+0000","last_became_peered":"2024-09-10T18:55:40.391094+0000","last_unstale":"2024-09-10T18:56:21.099131+0000","last_undegraded":"2024-09-10T18:56:21.099131+0000","last_fullsized":"2024-09-10T18:56:21.099131+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'12","last_scrub_stamp":"2024-09-10T18:56:21.098770+0000","last_deep_scrub":"258'12","last_deep_scrub_stamp":"2024-09-10T18:56:21.098770+0000","last_clean_scrub_stamp":"2024-09-10T18:56:21.098770+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:32.905276+0000","scrub_duration":0.0034993049999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":219,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.752314+0000","last_change":"2024-09-10T18:55:40.398023+0000","last_active":"2024-09-10T18:56:15.752314+0000","last_peered":"2024-09-10T18:56:15.752314+0000","last_clean":"2024-09-10T18:56:15.752314+0000","last_became_active":"2024-09-10T18:55:40.397478+0000","last_became_peered":"2024-09-10T18:55:40.397478+0000","last_unstale":"2024-09-10T18:56:15.752314+0000","last_undegraded":"2024-09-10T18:56:15.752314+0000","last_fullsized":"2024-09-10T18:56:15.752314+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:55.515593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:52:55.515593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0072066109999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.9","version":"258'4388","reported_seq":4414,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:23.376023+0000","last_change":"2024-09-10T18:55:40.389308+0000","last_active":"2024-09-10T18:56:23.376023+0000","last_peered":"2024-09-10T18:56:23.376023+0000","last_clean":"2024-09-10T18:56:23.376023+0000","last_became_active":"2024-09-10T18:55:40.388325+0000","last_became_peered":"2024-09-10T18:55:40.388325+0000","last_unstale":"2024-09-10T18:56:23.376023+0000","last_undegraded":"2024-09-10T18:56:23.376023+0000","last_fullsized":"2024-09-10T18:56:23.376023+0000","mapping_epoch":241,"log_start":"247'300","ondisk_log_start":"247'300","created":237,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":4088,"log_dups_size":0,"ondisk_log_size":4088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:42.480386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":287571968,"num_objects":4388,"num_object_clones":0,"num_object_copies":13164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4388,"num_write_kb":280832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"70'81","reported_seq":195,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.753774+0000","last_change":"2024-09-10T18:56:11.765628+0000","last_active":"2024-09-10T18:56:15.753774+0000","last_peered":"2024-09-10T18:56:15.753774+0000","last_clean":"2024-09-10T18:56:15.753774+0000","last_became_active":"2024-09-10T18:55:40.392432+0000","last_became_peered":"2024-09-10T18:55:40.392432+0000","last_unstale":"2024-09-10T18:56:15.753774+0000","last_undegraded":"2024-09-10T18:56:15.753774+0000","last_fullsized":"2024-09-10T18:56:15.753774+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:11.765541+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:11.765541+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:27.470910+0000","scrub_duration":0.002140374,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"232'41","reported_seq":490,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:21.333778+0000","last_change":"2024-09-10T18:56:18.672331+0000","last_active":"2024-09-10T18:56:21.333778+0000","last_peered":"2024-09-10T18:56:21.333778+0000","last_clean":"2024-09-10T18:56:21.333778+0000","last_became_active":"2024-09-10T18:47:36.420185+0000","last_became_peered":"2024-09-10T18:47:36.420185+0000","last_unstale":"2024-09-10T18:56:21.333778+0000","last_undegraded":"2024-09-10T18:56:21.333778+0000","last_fullsized":"2024-09-10T18:56:21.333778+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'41","last_scrub_stamp":"2024-09-10T18:56:18.672139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:18.672139+0000","objects_scrubbed":10,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:41.547337+0000","scrub_duration":0.020233260999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":263,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.752537+0000","last_change":"2024-09-10T18:55:29.867732+0000","last_active":"2024-09-10T18:56:15.752537+0000","last_peered":"2024-09-10T18:56:15.752537+0000","last_clean":"2024-09-10T18:56:15.752537+0000","last_became_active":"2024-09-10T18:45:17.964494+0000","last_became_peered":"2024-09-10T18:45:17.964494+0000","last_unstale":"2024-09-10T18:56:15.752537+0000","last_undegraded":"2024-09-10T18:56:15.752537+0000","last_fullsized":"2024-09-10T18:56:15.752537+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:29.867683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:12.982599+0000","last_clean_scrub_stamp":"2024-09-10T18:55:29.867683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:52.216063+0000","scrub_duration":0.0015951909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":214,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:20.224715+0000","last_change":"2024-09-10T18:56:20.224668+0000","last_active":"2024-09-10T18:56:20.224715+0000","last_peered":"2024-09-10T18:56:20.224715+0000","last_clean":"2024-09-10T18:56:20.224715+0000","last_became_active":"2024-09-10T18:47:57.107478+0000","last_became_peered":"2024-09-10T18:47:57.107478+0000","last_unstale":"2024-09-10T18:56:20.224715+0000","last_undegraded":"2024-09-10T18:56:20.224715+0000","last_fullsized":"2024-09-10T18:56:20.224715+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:20.224562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:20.224562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:48.420895+0000","scrub_duration":0.008542315,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":244,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.758669+0000","last_change":"2024-09-10T18:56:11.878274+0000","last_active":"2024-09-10T18:56:15.758669+0000","last_peered":"2024-09-10T18:56:15.758669+0000","last_clean":"2024-09-10T18:56:15.758669+0000","last_became_active":"2024-09-10T18:45:24.073307+0000","last_became_peered":"2024-09-10T18:45:24.073307+0000","last_unstale":"2024-09-10T18:56:15.758669+0000","last_undegraded":"2024-09-10T18:56:15.758669+0000","last_fullsized":"2024-09-10T18:56:15.758669+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:11.878181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:22.891084+0000","last_clean_scrub_stamp":"2024-09-10T18:56:11.878181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:34.367581+0000","scrub_duration":0.0037255349999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"258'4696","reported_seq":4725,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:26.457532+0000","last_change":"2024-09-10T18:55:36.355697+0000","last_active":"2024-09-10T18:56:26.457532+0000","last_peered":"2024-09-10T18:56:26.457532+0000","last_clean":"2024-09-10T18:56:26.457532+0000","last_became_active":"2024-09-10T18:55:36.355371+0000","last_became_peered":"2024-09-10T18:55:36.355371+0000","last_unstale":"2024-09-10T18:56:26.457532+0000","last_undegraded":"2024-09-10T18:56:26.457532+0000","last_fullsized":"2024-09-10T18:56:26.457532+0000","mapping_epoch":237,"log_start":"249'900","ondisk_log_start":"249'900","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3796,"log_dups_size":0,"ondisk_log_size":3796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:36.148493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":307757056,"num_objects":4696,"num_object_clones":0,"num_object_copies":14088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4696,"num_write_kb":300544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"70'81","reported_seq":352,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.757950+0000","last_change":"2024-09-10T18:56:07.028113+0000","last_active":"2024-09-10T18:56:15.757950+0000","last_peered":"2024-09-10T18:56:15.757950+0000","last_clean":"2024-09-10T18:56:15.757950+0000","last_became_active":"2024-09-10T18:55:40.389718+0000","last_became_peered":"2024-09-10T18:55:40.389718+0000","last_unstale":"2024-09-10T18:56:15.757950+0000","last_undegraded":"2024-09-10T18:56:15.757950+0000","last_fullsized":"2024-09-10T18:56:15.757950+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:07.028031+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:07.028031+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:26.021371+0000","scrub_duration":0.00655837,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"232'22","reported_seq":310,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.760879+0000","last_change":"2024-09-10T18:56:05.550938+0000","last_active":"2024-09-10T18:56:15.760879+0000","last_peered":"2024-09-10T18:56:15.760879+0000","last_clean":"2024-09-10T18:56:15.760879+0000","last_became_active":"2024-09-10T18:55:40.397422+0000","last_became_peered":"2024-09-10T18:55:40.397422+0000","last_unstale":"2024-09-10T18:56:15.760879+0000","last_undegraded":"2024-09-10T18:56:15.760879+0000","last_fullsized":"2024-09-10T18:56:15.760879+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'22","last_scrub_stamp":"2024-09-10T18:56:05.550846+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-10T18:47:10.514951+0000","last_clean_scrub_stamp":"2024-09-10T18:56:05.550846+0000","objects_scrubbed":5,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:34.381969+0000","scrub_duration":0.0079381009999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"84'6","reported_seq":946,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:24.579578+0000","last_change":"2024-09-10T18:56:22.799839+0000","last_active":"2024-09-10T18:56:24.579578+0000","last_peered":"2024-09-10T18:56:24.579578+0000","last_clean":"2024-09-10T18:56:24.579578+0000","last_became_active":"2024-09-10T18:55:40.403098+0000","last_became_peered":"2024-09-10T18:55:40.403098+0000","last_unstale":"2024-09-10T18:56:24.579578+0000","last_undegraded":"2024-09-10T18:56:24.579578+0000","last_fullsized":"2024-09-10T18:56:24.579578+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'6","last_scrub_stamp":"2024-09-10T18:56:22.799414+0000","last_deep_scrub":"84'6","last_deep_scrub_stamp":"2024-09-10T18:47:22.545450+0000","last_clean_scrub_stamp":"2024-09-10T18:56:22.799414+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:35.469589+0000","scrub_duration":0.004952756,"objects_trimmed":0,"snaptrim_duration":0.0021933159999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":652,"num_read_kb":647,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":228,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.758507+0000","last_change":"2024-09-10T18:56:03.791304+0000","last_active":"2024-09-10T18:56:15.758507+0000","last_peered":"2024-09-10T18:56:15.758507+0000","last_clean":"2024-09-10T18:56:15.758507+0000","last_became_active":"2024-09-10T18:55:40.393389+0000","last_became_peered":"2024-09-10T18:55:40.393389+0000","last_unstale":"2024-09-10T18:56:15.758507+0000","last_undegraded":"2024-09-10T18:56:15.758507+0000","last_fullsized":"2024-09-10T18:56:15.758507+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:03.791228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:03.791228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:15.631676+0000","scrub_duration":0.004016899,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":224,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.755912+0000","last_change":"2024-09-10T18:55:38.900232+0000","last_active":"2024-09-10T18:56:15.755912+0000","last_peered":"2024-09-10T18:56:15.755912+0000","last_clean":"2024-09-10T18:56:15.755912+0000","last_became_active":"2024-09-10T18:47:36.419081+0000","last_became_peered":"2024-09-10T18:47:36.419081+0000","last_unstale":"2024-09-10T18:56:15.755912+0000","last_undegraded":"2024-09-10T18:56:15.755912+0000","last_fullsized":"2024-09-10T18:56:15.755912+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:38.900146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:55:38.900146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:40.649926+0000","scrub_duration":0.0048114799999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":255,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:21.087017+0000","last_change":"2024-09-10T18:56:21.086977+0000","last_active":"2024-09-10T18:56:21.087017+0000","last_peered":"2024-09-10T18:56:21.087017+0000","last_clean":"2024-09-10T18:56:21.087017+0000","last_became_active":"2024-09-10T18:45:24.264617+0000","last_became_peered":"2024-09-10T18:45:24.264617+0000","last_unstale":"2024-09-10T18:56:21.087017+0000","last_undegraded":"2024-09-10T18:56:21.087017+0000","last_fullsized":"2024-09-10T18:56:21.087017+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:21.086881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:07.150480+0000","last_clean_scrub_stamp":"2024-09-10T18:56:21.086881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:50.076246+0000","scrub_duration":0.005097926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"258'4387","reported_seq":4413,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:23.400975+0000","last_change":"2024-09-10T18:55:40.391994+0000","last_active":"2024-09-10T18:56:23.400975+0000","last_peered":"2024-09-10T18:56:23.400975+0000","last_clean":"2024-09-10T18:56:23.400975+0000","last_became_active":"2024-09-10T18:55:40.391432+0000","last_became_peered":"2024-09-10T18:55:40.391432+0000","last_unstale":"2024-09-10T18:56:23.400975+0000","last_undegraded":"2024-09-10T18:56:23.400975+0000","last_fullsized":"2024-09-10T18:56:23.400975+0000","mapping_epoch":241,"log_start":"246'300","ondisk_log_start":"246'300","created":237,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":4087,"log_dups_size":0,"ondisk_log_size":4087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:59.041462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":287506432,"num_objects":4387,"num_object_clones":0,"num_object_copies":13161,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4387,"num_write_kb":280768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"70'81","reported_seq":167,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.762818+0000","last_change":"2024-09-10T18:53:00.338829+0000","last_active":"2024-09-10T18:56:15.762818+0000","last_peered":"2024-09-10T18:56:15.762818+0000","last_clean":"2024-09-10T18:56:15.762818+0000","last_became_active":"2024-09-10T18:52:14.589009+0000","last_became_peered":"2024-09-10T18:52:14.589009+0000","last_unstale":"2024-09-10T18:56:15.762818+0000","last_undegraded":"2024-09-10T18:56:15.762818+0000","last_fullsized":"2024-09-10T18:56:15.762818+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:53:00.338611+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:53:00.338611+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023458709999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"232'24","reported_seq":282,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.752045+0000","last_change":"2024-09-10T18:55:40.419877+0000","last_active":"2024-09-10T18:56:15.752045+0000","last_peered":"2024-09-10T18:56:15.752045+0000","last_clean":"2024-09-10T18:56:15.752045+0000","last_became_active":"2024-09-10T18:55:40.397070+0000","last_became_peered":"2024-09-10T18:55:40.397070+0000","last_unstale":"2024-09-10T18:56:15.752045+0000","last_undegraded":"2024-09-10T18:56:15.752045+0000","last_fullsized":"2024-09-10T18:56:15.752045+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:52:01.328828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:52:01.328828+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"58'1","reported_seq":231,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.754424+0000","last_change":"2024-09-10T18:56:06.131199+0000","last_active":"2024-09-10T18:56:15.754424+0000","last_peered":"2024-09-10T18:56:15.754424+0000","last_clean":"2024-09-10T18:56:15.754424+0000","last_became_active":"2024-09-10T18:55:40.390022+0000","last_became_peered":"2024-09-10T18:55:40.390022+0000","last_unstale":"2024-09-10T18:56:15.754424+0000","last_undegraded":"2024-09-10T18:56:15.754424+0000","last_fullsized":"2024-09-10T18:56:15.754424+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'1","last_scrub_stamp":"2024-09-10T18:56:06.131098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:06.131098+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:09.358910+0000","scrub_duration":0.016770345999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":236,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:24.154085+0000","last_change":"2024-09-10T18:56:24.154041+0000","last_active":"2024-09-10T18:56:24.154085+0000","last_peered":"2024-09-10T18:56:24.154085+0000","last_clean":"2024-09-10T18:56:24.154085+0000","last_became_active":"2024-09-10T18:55:40.397290+0000","last_became_peered":"2024-09-10T18:55:40.397290+0000","last_unstale":"2024-09-10T18:56:24.154085+0000","last_undegraded":"2024-09-10T18:56:24.154085+0000","last_fullsized":"2024-09-10T18:56:24.154085+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:24.153973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:24.153973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:37.952634+0000","scrub_duration":0.084106958999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":103,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.760618+0000","last_change":"2024-09-10T18:55:40.392772+0000","last_active":"2024-09-10T18:56:15.760618+0000","last_peered":"2024-09-10T18:56:15.760618+0000","last_clean":"2024-09-10T18:56:15.760618+0000","last_became_active":"2024-09-10T18:55:40.392494+0000","last_became_peered":"2024-09-10T18:55:40.392494+0000","last_unstale":"2024-09-10T18:56:15.760618+0000","last_undegraded":"2024-09-10T18:56:15.760618+0000","last_fullsized":"2024-09-10T18:56:15.760618+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:41.150673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:52:41.150673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":247,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.764843+0000","last_change":"2024-09-10T18:54:17.615248+0000","last_active":"2024-09-10T18:56:15.764843+0000","last_peered":"2024-09-10T18:56:15.764843+0000","last_clean":"2024-09-10T18:56:15.764843+0000","last_became_active":"2024-09-10T18:45:24.083689+0000","last_became_peered":"2024-09-10T18:45:24.083689+0000","last_unstale":"2024-09-10T18:56:15.764843+0000","last_undegraded":"2024-09-10T18:56:15.764843+0000","last_fullsized":"2024-09-10T18:56:15.764843+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:17.615191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:54:17.615191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011240523000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"258'4735","reported_seq":4770,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:26.625337+0000","last_change":"2024-09-10T18:55:40.395213+0000","last_active":"2024-09-10T18:56:26.625337+0000","last_peered":"2024-09-10T18:56:26.625337+0000","last_clean":"2024-09-10T18:56:26.625337+0000","last_became_active":"2024-09-10T18:55:40.394445+0000","last_became_peered":"2024-09-10T18:55:40.394445+0000","last_unstale":"2024-09-10T18:56:26.625337+0000","last_undegraded":"2024-09-10T18:56:26.625337+0000","last_fullsized":"2024-09-10T18:56:26.625337+0000","mapping_epoch":241,"log_start":"249'900","ondisk_log_start":"249'900","created":237,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3835,"log_dups_size":0,"ondisk_log_size":3835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:47.412527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":310312960,"num_objects":4735,"num_object_clones":0,"num_object_copies":14205,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4735,"num_write_kb":303040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"232'33","reported_seq":315,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:21.207074+0000","last_change":"2024-09-10T18:56:21.207045+0000","last_active":"2024-09-10T18:56:21.207074+0000","last_peered":"2024-09-10T18:56:21.207074+0000","last_clean":"2024-09-10T18:56:21.207074+0000","last_became_active":"2024-09-10T18:45:20.421119+0000","last_became_peered":"2024-09-10T18:45:20.421119+0000","last_unstale":"2024-09-10T18:56:21.207074+0000","last_undegraded":"2024-09-10T18:56:21.207074+0000","last_fullsized":"2024-09-10T18:56:21.207074+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'33","last_scrub_stamp":"2024-09-10T18:56:21.206960+0000","last_deep_scrub":"232'33","last_deep_scrub_stamp":"2024-09-10T18:56:21.206960+0000","last_clean_scrub_stamp":"2024-09-10T18:56:21.206960+0000","objects_scrubbed":7,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:29.526268+0000","scrub_duration":0.0042638809999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"60'2","reported_seq":263,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.759637+0000","last_change":"2024-09-10T18:54:19.313557+0000","last_active":"2024-09-10T18:56:15.759637+0000","last_peered":"2024-09-10T18:56:15.759637+0000","last_clean":"2024-09-10T18:56:15.759637+0000","last_became_active":"2024-09-10T18:45:17.961660+0000","last_became_peered":"2024-09-10T18:45:17.961660+0000","last_unstale":"2024-09-10T18:56:15.759637+0000","last_undegraded":"2024-09-10T18:56:15.759637+0000","last_fullsized":"2024-09-10T18:56:15.759637+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-10T18:54:19.313386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:54:19.313386+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056941359999999998,"objects_trimmed":0,"snaptrim_duration":0.0013942640000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":270,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.752758+0000","last_change":"2024-09-10T18:54:44.030368+0000","last_active":"2024-09-10T18:56:15.752758+0000","last_peered":"2024-09-10T18:56:15.752758+0000","last_clean":"2024-09-10T18:56:15.752758+0000","last_became_active":"2024-09-10T18:47:36.414636+0000","last_became_peered":"2024-09-10T18:47:36.414636+0000","last_unstale":"2024-09-10T18:56:15.752758+0000","last_undegraded":"2024-09-10T18:56:15.752758+0000","last_fullsized":"2024-09-10T18:56:15.752758+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:44.030238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:54:44.030238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0035207699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'81","reported_seq":343,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.756557+0000","last_change":"2024-09-10T18:56:11.056259+0000","last_active":"2024-09-10T18:56:15.756557+0000","last_peered":"2024-09-10T18:56:15.756557+0000","last_clean":"2024-09-10T18:56:15.756557+0000","last_became_active":"2024-09-10T18:55:40.397021+0000","last_became_peered":"2024-09-10T18:55:40.397021+0000","last_unstale":"2024-09-10T18:56:15.756557+0000","last_undegraded":"2024-09-10T18:56:15.756557+0000","last_fullsized":"2024-09-10T18:56:15.756557+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:11.056156+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:56.085580+0000","last_clean_scrub_stamp":"2024-09-10T18:56:11.056156+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:24.032506+0000","scrub_duration":0.0022215939999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":284,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.756188+0000","last_change":"2024-09-10T18:55:40.390348+0000","last_active":"2024-09-10T18:56:15.756188+0000","last_peered":"2024-09-10T18:56:15.756188+0000","last_clean":"2024-09-10T18:56:15.756188+0000","last_became_active":"2024-09-10T18:55:40.390040+0000","last_became_peered":"2024-09-10T18:55:40.390040+0000","last_unstale":"2024-09-10T18:56:15.756188+0000","last_undegraded":"2024-09-10T18:56:15.756188+0000","last_fullsized":"2024-09-10T18:56:15.756188+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:42.415663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:53:42.415663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007606036,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":254,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.752854+0000","last_change":"2024-09-10T18:54:48.995651+0000","last_active":"2024-09-10T18:56:15.752854+0000","last_peered":"2024-09-10T18:56:15.752854+0000","last_clean":"2024-09-10T18:56:15.752854+0000","last_became_active":"2024-09-10T18:45:24.073327+0000","last_became_peered":"2024-09-10T18:45:24.073327+0000","last_unstale":"2024-09-10T18:56:15.752854+0000","last_undegraded":"2024-09-10T18:56:15.752854+0000","last_fullsized":"2024-09-10T18:56:15.752854+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:48.995571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:54:48.995571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029876989999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.7","version":"258'4518","reported_seq":4547,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:26.359406+0000","last_change":"2024-09-10T18:55:36.356374+0000","last_active":"2024-09-10T18:56:26.359406+0000","last_peered":"2024-09-10T18:56:26.359406+0000","last_clean":"2024-09-10T18:56:26.359406+0000","last_became_active":"2024-09-10T18:55:36.355945+0000","last_became_peered":"2024-09-10T18:55:36.355945+0000","last_unstale":"2024-09-10T18:56:26.359406+0000","last_undegraded":"2024-09-10T18:56:26.359406+0000","last_fullsized":"2024-09-10T18:56:26.359406+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":4518,"log_dups_size":0,"ondisk_log_size":4518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:04.916365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":296091648,"num_objects":4518,"num_object_clones":0,"num_object_copies":13554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4518,"num_write_kb":289152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.f","version":"70'81","reported_seq":258,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.756658+0000","last_change":"2024-09-10T18:55:29.018036+0000","last_active":"2024-09-10T18:56:15.756658+0000","last_peered":"2024-09-10T18:56:15.756658+0000","last_clean":"2024-09-10T18:56:15.756658+0000","last_became_active":"2024-09-10T18:52:16.598295+0000","last_became_peered":"2024-09-10T18:52:16.598295+0000","last_unstale":"2024-09-10T18:56:15.756658+0000","last_undegraded":"2024-09-10T18:56:15.756658+0000","last_fullsized":"2024-09-10T18:56:15.756658+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:55:29.017936+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:53.609885+0000","last_clean_scrub_stamp":"2024-09-10T18:55:29.017936+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:34.964703+0000","scrub_duration":0.001646813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"232'35","reported_seq":316,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.756482+0000","last_change":"2024-09-10T18:55:28.047394+0000","last_active":"2024-09-10T18:56:15.756482+0000","last_peered":"2024-09-10T18:56:15.756482+0000","last_clean":"2024-09-10T18:56:15.756482+0000","last_became_active":"2024-09-10T18:45:20.421097+0000","last_became_peered":"2024-09-10T18:45:20.421097+0000","last_unstale":"2024-09-10T18:56:15.756482+0000","last_undegraded":"2024-09-10T18:56:15.756482+0000","last_fullsized":"2024-09-10T18:56:15.756482+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'35","last_scrub_stamp":"2024-09-10T18:55:28.047336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:55:28.047336+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:33.623657+0000","scrub_duration":0.0019079699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":238,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.752857+0000","last_change":"2024-09-10T18:56:13.760318+0000","last_active":"2024-09-10T18:56:15.752857+0000","last_peered":"2024-09-10T18:56:15.752857+0000","last_clean":"2024-09-10T18:56:15.752857+0000","last_became_active":"2024-09-10T18:55:40.391849+0000","last_became_peered":"2024-09-10T18:55:40.391849+0000","last_unstale":"2024-09-10T18:56:15.752857+0000","last_undegraded":"2024-09-10T18:56:15.752857+0000","last_fullsized":"2024-09-10T18:56:15.752857+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:13.760238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:13.760238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:40.211689+0000","scrub_duration":0.048619793000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":178,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.752820+0000","last_change":"2024-09-10T18:55:40.397918+0000","last_active":"2024-09-10T18:56:15.752820+0000","last_peered":"2024-09-10T18:56:15.752820+0000","last_clean":"2024-09-10T18:56:15.752820+0000","last_became_active":"2024-09-10T18:55:40.397544+0000","last_became_peered":"2024-09-10T18:55:40.397544+0000","last_unstale":"2024-09-10T18:56:15.752820+0000","last_undegraded":"2024-09-10T18:56:15.752820+0000","last_fullsized":"2024-09-10T18:56:15.752820+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:40.126652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:51:40.126652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017754699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":258,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.756473+0000","last_change":"2024-09-10T18:54:20.686920+0000","last_active":"2024-09-10T18:56:15.756473+0000","last_peered":"2024-09-10T18:56:15.756473+0000","last_clean":"2024-09-10T18:56:15.756473+0000","last_became_active":"2024-09-10T18:47:36.412610+0000","last_became_peered":"2024-09-10T18:47:36.412610+0000","last_unstale":"2024-09-10T18:56:15.756473+0000","last_undegraded":"2024-09-10T18:56:15.756473+0000","last_fullsized":"2024-09-10T18:56:15.756473+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:20.686780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:54:20.686780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051641839999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"258'4666","reported_seq":4695,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:26.399259+0000","last_change":"2024-09-10T18:55:36.358103+0000","last_active":"2024-09-10T18:56:26.399259+0000","last_peered":"2024-09-10T18:56:26.399259+0000","last_clean":"2024-09-10T18:56:26.399259+0000","last_became_active":"2024-09-10T18:55:36.357828+0000","last_became_peered":"2024-09-10T18:55:36.357828+0000","last_unstale":"2024-09-10T18:56:26.399259+0000","last_undegraded":"2024-09-10T18:56:26.399259+0000","last_fullsized":"2024-09-10T18:56:26.399259+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":4666,"log_dups_size":0,"ondisk_log_size":4666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:38.273211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":305790976,"num_objects":4666,"num_object_clones":0,"num_object_copies":13998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4666,"num_write_kb":298624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.e","version":"70'81","reported_seq":117,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.763424+0000","last_change":"2024-09-10T18:55:30.496965+0000","last_active":"2024-09-10T18:56:15.763424+0000","last_peered":"2024-09-10T18:56:15.763424+0000","last_clean":"2024-09-10T18:56:15.763424+0000","last_became_active":"2024-09-10T18:52:16.604236+0000","last_became_peered":"2024-09-10T18:52:16.604236+0000","last_unstale":"2024-09-10T18:56:15.763424+0000","last_undegraded":"2024-09-10T18:56:15.763424+0000","last_fullsized":"2024-09-10T18:56:15.763424+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:55:30.496910+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:55:30.496910+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:49.349532+0000","scrub_duration":0.0012110790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"232'19","reported_seq":264,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:20.061668+0000","last_change":"2024-09-10T18:56:20.061554+0000","last_active":"2024-09-10T18:56:20.061668+0000","last_peered":"2024-09-10T18:56:20.061668+0000","last_clean":"2024-09-10T18:56:20.061668+0000","last_became_active":"2024-09-10T18:55:40.395257+0000","last_became_peered":"2024-09-10T18:55:40.395257+0000","last_unstale":"2024-09-10T18:56:20.061668+0000","last_undegraded":"2024-09-10T18:56:20.061668+0000","last_fullsized":"2024-09-10T18:56:20.061668+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T18:56:20.061420+0000","last_deep_scrub":"130'13","last_deep_scrub_stamp":"2024-09-10T18:51:58.427534+0000","last_clean_scrub_stamp":"2024-09-10T18:56:20.061420+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:44.377480+0000","scrub_duration":0.0065809780000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":250,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.766179+0000","last_change":"2024-09-10T18:54:52.179168+0000","last_active":"2024-09-10T18:56:15.766179+0000","last_peered":"2024-09-10T18:56:15.766179+0000","last_clean":"2024-09-10T18:56:15.766179+0000","last_became_active":"2024-09-10T18:45:17.963057+0000","last_became_peered":"2024-09-10T18:45:17.963057+0000","last_unstale":"2024-09-10T18:56:15.766179+0000","last_undegraded":"2024-09-10T18:56:15.766179+0000","last_fullsized":"2024-09-10T18:56:15.766179+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:52.179074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:54:52.179074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001414738,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":226,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:17.661955+0000","last_change":"2024-09-10T18:56:17.661917+0000","last_active":"2024-09-10T18:56:17.661955+0000","last_peered":"2024-09-10T18:56:17.661955+0000","last_clean":"2024-09-10T18:56:17.661955+0000","last_became_active":"2024-09-10T18:55:40.390771+0000","last_became_peered":"2024-09-10T18:55:40.390771+0000","last_unstale":"2024-09-10T18:56:17.661955+0000","last_undegraded":"2024-09-10T18:56:17.661955+0000","last_fullsized":"2024-09-10T18:56:17.661955+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:17.661832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_clean_scrub_stamp":"2024-09-10T18:56:17.661832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:37.630708+0000","scrub_duration":0.0041460890000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":232,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.754499+0000","last_change":"2024-09-10T18:55:40.390877+0000","last_active":"2024-09-10T18:56:15.754499+0000","last_peered":"2024-09-10T18:56:15.754499+0000","last_clean":"2024-09-10T18:56:15.754499+0000","last_became_active":"2024-09-10T18:55:40.390451+0000","last_became_peered":"2024-09-10T18:55:40.390451+0000","last_unstale":"2024-09-10T18:56:15.754499+0000","last_undegraded":"2024-09-10T18:56:15.754499+0000","last_fullsized":"2024-09-10T18:56:15.754499+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:53.345184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:53.345184+0000","last_clean_scrub_stamp":"2024-09-10T18:51:53.345184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012977799999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.5","version":"258'4328","reported_seq":4352,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:23.385260+0000","last_change":"2024-09-10T18:55:40.393743+0000","last_active":"2024-09-10T18:56:23.385260+0000","last_peered":"2024-09-10T18:56:23.385260+0000","last_clean":"2024-09-10T18:56:23.385260+0000","last_became_active":"2024-09-10T18:55:40.393456+0000","last_became_peered":"2024-09-10T18:55:40.393456+0000","last_unstale":"2024-09-10T18:56:23.385260+0000","last_undegraded":"2024-09-10T18:56:23.385260+0000","last_fullsized":"2024-09-10T18:56:23.385260+0000","mapping_epoch":241,"log_start":"246'200","ondisk_log_start":"246'200","created":237,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":4128,"log_dups_size":0,"ondisk_log_size":4128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:49.754061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":283639808,"num_objects":4328,"num_object_clones":0,"num_object_copies":12984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4328,"num_write_kb":276992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.d","version":"70'81","reported_seq":118,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:18.041207+0000","last_change":"2024-09-10T18:56:18.041177+0000","last_active":"2024-09-10T18:56:18.041207+0000","last_peered":"2024-09-10T18:56:18.041207+0000","last_clean":"2024-09-10T18:56:18.041207+0000","last_became_active":"2024-09-10T18:52:16.605492+0000","last_became_peered":"2024-09-10T18:52:16.605492+0000","last_unstale":"2024-09-10T18:56:18.041207+0000","last_undegraded":"2024-09-10T18:56:18.041207+0000","last_fullsized":"2024-09-10T18:56:18.041207+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:18.041116+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:18.041116+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:33.909201+0000","scrub_duration":0.0034400989999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"232'27","reported_seq":171,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.755478+0000","last_change":"2024-09-10T18:54:22.720833+0000","last_active":"2024-09-10T18:56:15.755478+0000","last_peered":"2024-09-10T18:56:15.755478+0000","last_clean":"2024-09-10T18:56:15.755478+0000","last_became_active":"2024-09-10T18:52:54.579763+0000","last_became_peered":"2024-09-10T18:52:54.579763+0000","last_unstale":"2024-09-10T18:56:15.755478+0000","last_undegraded":"2024-09-10T18:56:15.755478+0000","last_fullsized":"2024-09-10T18:56:15.755478+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:54:22.720737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:54:22.720737+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014176382,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":211,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:24.902328+0000","last_change":"2024-09-10T18:56:24.902286+0000","last_active":"2024-09-10T18:56:24.902328+0000","last_peered":"2024-09-10T18:56:24.902328+0000","last_clean":"2024-09-10T18:56:24.902328+0000","last_became_active":"2024-09-10T18:47:36.420475+0000","last_became_peered":"2024-09-10T18:47:36.420475+0000","last_unstale":"2024-09-10T18:56:24.902328+0000","last_undegraded":"2024-09-10T18:56:24.902328+0000","last_fullsized":"2024-09-10T18:56:24.902328+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:24.902205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:05.603125+0000","last_clean_scrub_stamp":"2024-09-10T18:56:24.902205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:26.786922+0000","scrub_duration":0.0076874830000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"83'9","reported_seq":406,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:21.208959+0000","last_change":"2024-09-10T18:55:37.750865+0000","last_active":"2024-09-10T18:56:21.208959+0000","last_peered":"2024-09-10T18:56:21.208959+0000","last_clean":"2024-09-10T18:56:21.208959+0000","last_became_active":"2024-09-10T18:45:22.073383+0000","last_became_peered":"2024-09-10T18:45:22.073383+0000","last_unstale":"2024-09-10T18:56:21.208959+0000","last_undegraded":"2024-09-10T18:56:21.208959+0000","last_fullsized":"2024-09-10T18:56:21.208959+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'9","last_scrub_stamp":"2024-09-10T18:55:37.750784+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-09-10T18:47:02.582229+0000","last_clean_scrub_stamp":"2024-09-10T18:55:37.750784+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:06.868622+0000","scrub_duration":0.0029702869999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":271,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.752204+0000","last_change":"2024-09-10T18:56:10.775911+0000","last_active":"2024-09-10T18:56:15.752204+0000","last_peered":"2024-09-10T18:56:15.752204+0000","last_clean":"2024-09-10T18:56:15.752204+0000","last_became_active":"2024-09-10T18:55:40.389202+0000","last_became_peered":"2024-09-10T18:55:40.389202+0000","last_unstale":"2024-09-10T18:56:15.752204+0000","last_undegraded":"2024-09-10T18:56:15.752204+0000","last_fullsized":"2024-09-10T18:56:15.752204+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:10.775814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_clean_scrub_stamp":"2024-09-10T18:56:10.775814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:12.073187+0000","scrub_duration":0.0031802459999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.4","version":"258'4536","reported_seq":4565,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:26.455884+0000","last_change":"2024-09-10T18:55:36.350444+0000","last_active":"2024-09-10T18:56:26.455884+0000","last_peered":"2024-09-10T18:56:26.455884+0000","last_clean":"2024-09-10T18:56:26.455884+0000","last_became_active":"2024-09-10T18:55:36.350155+0000","last_became_peered":"2024-09-10T18:55:36.350155+0000","last_unstale":"2024-09-10T18:56:26.455884+0000","last_undegraded":"2024-09-10T18:56:26.455884+0000","last_fullsized":"2024-09-10T18:56:26.455884+0000","mapping_epoch":237,"log_start":"249'700","ondisk_log_start":"249'700","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3836,"log_dups_size":0,"ondisk_log_size":3836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:41.679724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":297271296,"num_objects":4536,"num_object_clones":0,"num_object_copies":13608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4536,"num_write_kb":290304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.c","version":"70'81","reported_seq":204,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.762387+0000","last_change":"2024-09-10T18:55:40.391027+0000","last_active":"2024-09-10T18:56:15.762387+0000","last_peered":"2024-09-10T18:56:15.762387+0000","last_clean":"2024-09-10T18:56:15.762387+0000","last_became_active":"2024-09-10T18:55:40.390655+0000","last_became_peered":"2024-09-10T18:55:40.390655+0000","last_unstale":"2024-09-10T18:56:15.762387+0000","last_undegraded":"2024-09-10T18:56:15.762387+0000","last_fullsized":"2024-09-10T18:56:15.762387+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:53:05.814851+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:53:05.814851+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025032510000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"232'18","reported_seq":288,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.762749+0000","last_change":"2024-09-10T18:55:27.507437+0000","last_active":"2024-09-10T18:56:15.762749+0000","last_peered":"2024-09-10T18:56:15.762749+0000","last_clean":"2024-09-10T18:56:15.762749+0000","last_became_active":"2024-09-10T18:45:20.056869+0000","last_became_peered":"2024-09-10T18:45:20.056869+0000","last_unstale":"2024-09-10T18:56:15.762749+0000","last_undegraded":"2024-09-10T18:56:15.762749+0000","last_fullsized":"2024-09-10T18:56:15.762749+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'18","last_scrub_stamp":"2024-09-10T18:55:27.507364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:55:27.507364+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:56.211266+0000","scrub_duration":0.0030221390000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"56'2","reported_seq":282,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.763670+0000","last_change":"2024-09-10T18:56:01.630150+0000","last_active":"2024-09-10T18:56:15.763670+0000","last_peered":"2024-09-10T18:56:15.763670+0000","last_clean":"2024-09-10T18:56:15.763670+0000","last_became_active":"2024-09-10T18:55:40.392169+0000","last_became_peered":"2024-09-10T18:55:40.392169+0000","last_unstale":"2024-09-10T18:56:15.763670+0000","last_undegraded":"2024-09-10T18:56:15.763670+0000","last_fullsized":"2024-09-10T18:56:15.763670+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-10T18:56:01.630091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:01.630091+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:04.396927+0000","scrub_duration":0.001578238,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":198,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:16.020148+0000","last_change":"2024-09-10T18:56:16.019721+0000","last_active":"2024-09-10T18:56:16.020148+0000","last_peered":"2024-09-10T18:56:16.020148+0000","last_clean":"2024-09-10T18:56:16.020148+0000","last_became_active":"2024-09-10T18:55:40.397290+0000","last_became_peered":"2024-09-10T18:55:40.397290+0000","last_unstale":"2024-09-10T18:56:16.020148+0000","last_undegraded":"2024-09-10T18:56:16.020148+0000","last_fullsized":"2024-09-10T18:56:16.020148+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:16.019357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:02.711594+0000","last_clean_scrub_stamp":"2024-09-10T18:56:16.019357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:32.894810+0000","scrub_duration":0.0021619629999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":244,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.759639+0000","last_change":"2024-09-10T18:54:53.352477+0000","last_active":"2024-09-10T18:56:15.759639+0000","last_peered":"2024-09-10T18:56:15.759639+0000","last_clean":"2024-09-10T18:56:15.759639+0000","last_became_active":"2024-09-10T18:45:24.077815+0000","last_became_peered":"2024-09-10T18:45:24.077815+0000","last_unstale":"2024-09-10T18:56:15.759639+0000","last_undegraded":"2024-09-10T18:56:15.759639+0000","last_fullsized":"2024-09-10T18:56:15.759639+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:53.352427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:53.352427+0000","last_clean_scrub_stamp":"2024-09-10T18:54:53.352427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022835899999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.13","version":"70'81","reported_seq":176,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.762964+0000","last_change":"2024-09-10T18:53:06.263065+0000","last_active":"2024-09-10T18:56:15.762964+0000","last_peered":"2024-09-10T18:56:15.762964+0000","last_clean":"2024-09-10T18:56:15.762964+0000","last_became_active":"2024-09-10T18:52:16.602463+0000","last_became_peered":"2024-09-10T18:52:16.602463+0000","last_unstale":"2024-09-10T18:56:15.762964+0000","last_undegraded":"2024-09-10T18:56:15.762964+0000","last_fullsized":"2024-09-10T18:56:15.762964+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:53:06.262959+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:53:06.262959+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0036690479999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"232'19","reported_seq":257,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.751896+0000","last_change":"2024-09-10T18:55:40.414373+0000","last_active":"2024-09-10T18:56:15.751896+0000","last_peered":"2024-09-10T18:56:15.751896+0000","last_clean":"2024-09-10T18:56:15.751896+0000","last_became_active":"2024-09-10T18:55:40.393944+0000","last_became_peered":"2024-09-10T18:55:40.393944+0000","last_unstale":"2024-09-10T18:56:15.751896+0000","last_undegraded":"2024-09-10T18:56:15.751896+0000","last_fullsized":"2024-09-10T18:56:15.751896+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'13","last_scrub_stamp":"2024-09-10T18:51:56.389281+0000","last_deep_scrub":"76'13","last_deep_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_clean_scrub_stamp":"2024-09-10T18:51:56.389281+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":238,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:25.820781+0000","last_change":"2024-09-10T18:56:25.820426+0000","last_active":"2024-09-10T18:56:25.820781+0000","last_peered":"2024-09-10T18:56:25.820781+0000","last_clean":"2024-09-10T18:56:25.820781+0000","last_became_active":"2024-09-10T18:55:40.401037+0000","last_became_peered":"2024-09-10T18:55:40.401037+0000","last_unstale":"2024-09-10T18:56:25.820781+0000","last_undegraded":"2024-09-10T18:56:25.820781+0000","last_fullsized":"2024-09-10T18:56:25.820781+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:25.820339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:25.820339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:27.095822+0000","scrub_duration":0.0048533279999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":256,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:25.863076+0000","last_change":"2024-09-10T18:56:25.862941+0000","last_active":"2024-09-10T18:56:25.863076+0000","last_peered":"2024-09-10T18:56:25.863076+0000","last_clean":"2024-09-10T18:56:25.863076+0000","last_became_active":"2024-09-10T18:45:22.069040+0000","last_became_peered":"2024-09-10T18:45:22.069040+0000","last_unstale":"2024-09-10T18:56:25.863076+0000","last_undegraded":"2024-09-10T18:56:25.863076+0000","last_fullsized":"2024-09-10T18:56:25.863076+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:25.862853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:00.754912+0000","last_clean_scrub_stamp":"2024-09-10T18:56:25.862853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:28.931296+0000","scrub_duration":0.0033952399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"63'1","reported_seq":242,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.763306+0000","last_change":"2024-09-10T18:56:07.774408+0000","last_active":"2024-09-10T18:56:15.763306+0000","last_peered":"2024-09-10T18:56:15.763306+0000","last_clean":"2024-09-10T18:56:15.763306+0000","last_became_active":"2024-09-10T18:55:40.389882+0000","last_became_peered":"2024-09-10T18:55:40.389882+0000","last_unstale":"2024-09-10T18:56:15.763306+0000","last_undegraded":"2024-09-10T18:56:15.763306+0000","last_fullsized":"2024-09-10T18:56:15.763306+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-10T18:56:07.774334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:07.774334+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:12.791580+0000","scrub_duration":0.090386770000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.12","version":"70'81","reported_seq":118,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.752355+0000","last_change":"2024-09-10T18:54:50.968303+0000","last_active":"2024-09-10T18:56:15.752355+0000","last_peered":"2024-09-10T18:56:15.752355+0000","last_clean":"2024-09-10T18:56:15.752355+0000","last_became_active":"2024-09-10T18:52:16.598921+0000","last_became_peered":"2024-09-10T18:52:16.598921+0000","last_unstale":"2024-09-10T18:56:15.752355+0000","last_undegraded":"2024-09-10T18:56:15.752355+0000","last_fullsized":"2024-09-10T18:56:15.752355+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:54:50.968202+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:54.127476+0000","last_clean_scrub_stamp":"2024-09-10T18:54:50.968202+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0074710000000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":263,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.755318+0000","last_change":"2024-09-10T18:55:35.136490+0000","last_active":"2024-09-10T18:56:15.755318+0000","last_peered":"2024-09-10T18:56:15.755318+0000","last_clean":"2024-09-10T18:56:15.755318+0000","last_became_active":"2024-09-10T18:45:17.964705+0000","last_became_peered":"2024-09-10T18:45:17.964705+0000","last_unstale":"2024-09-10T18:56:15.755318+0000","last_undegraded":"2024-09-10T18:56:15.755318+0000","last_fullsized":"2024-09-10T18:56:15.755318+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.136432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.136432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:38.251881+0000","scrub_duration":0.0015178839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"232'12","reported_seq":269,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.758582+0000","last_change":"2024-09-10T18:55:30.612864+0000","last_active":"2024-09-10T18:56:15.758582+0000","last_peered":"2024-09-10T18:56:15.758582+0000","last_clean":"2024-09-10T18:56:15.758582+0000","last_became_active":"2024-09-10T18:45:20.421746+0000","last_became_peered":"2024-09-10T18:45:20.421746+0000","last_unstale":"2024-09-10T18:56:15.758582+0000","last_undegraded":"2024-09-10T18:56:15.758582+0000","last_fullsized":"2024-09-10T18:56:15.758582+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'12","last_scrub_stamp":"2024-09-10T18:55:30.612808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:55:30.612808+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:55.373675+0000","scrub_duration":0.001922787,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":246,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.756081+0000","last_change":"2024-09-10T18:53:03.116972+0000","last_active":"2024-09-10T18:56:15.756081+0000","last_peered":"2024-09-10T18:56:15.756081+0000","last_clean":"2024-09-10T18:56:15.756081+0000","last_became_active":"2024-09-10T18:45:22.069653+0000","last_became_peered":"2024-09-10T18:45:22.069653+0000","last_unstale":"2024-09-10T18:56:15.756081+0000","last_undegraded":"2024-09-10T18:56:15.756081+0000","last_fullsized":"2024-09-10T18:56:15.756081+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:03.116899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:53:03.116899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010530331,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":255,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.757890+0000","last_change":"2024-09-10T18:54:47.741171+0000","last_active":"2024-09-10T18:56:15.757890+0000","last_peered":"2024-09-10T18:56:15.757890+0000","last_clean":"2024-09-10T18:56:15.757890+0000","last_became_active":"2024-09-10T18:45:24.264611+0000","last_became_peered":"2024-09-10T18:45:24.264611+0000","last_unstale":"2024-09-10T18:56:15.757890+0000","last_undegraded":"2024-09-10T18:56:15.757890+0000","last_fullsized":"2024-09-10T18:56:15.757890+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:47.741071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:54:47.741071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053190620000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.11","version":"70'81","reported_seq":142,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.754235+0000","last_change":"2024-09-10T18:55:40.395720+0000","last_active":"2024-09-10T18:56:15.754235+0000","last_peered":"2024-09-10T18:56:15.754235+0000","last_clean":"2024-09-10T18:56:15.754235+0000","last_became_active":"2024-09-10T18:55:40.395019+0000","last_became_peered":"2024-09-10T18:55:40.395019+0000","last_unstale":"2024-09-10T18:56:15.754235+0000","last_undegraded":"2024-09-10T18:56:15.754235+0000","last_fullsized":"2024-09-10T18:56:15.754235+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:52:53.556903+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:52:53.556903+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004894626,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"232'20","reported_seq":322,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.760848+0000","last_change":"2024-09-10T18:55:33.667767+0000","last_active":"2024-09-10T18:56:15.760848+0000","last_peered":"2024-09-10T18:56:15.760848+0000","last_clean":"2024-09-10T18:56:15.760848+0000","last_became_active":"2024-09-10T18:47:36.426260+0000","last_became_peered":"2024-09-10T18:47:36.426260+0000","last_unstale":"2024-09-10T18:56:15.760848+0000","last_undegraded":"2024-09-10T18:56:15.760848+0000","last_fullsized":"2024-09-10T18:56:15.760848+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'20","last_scrub_stamp":"2024-09-10T18:55:33.667712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:55:33.667712+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:59.298753+0000","scrub_duration":0.0028302039999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":260,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.752473+0000","last_change":"2024-09-10T18:54:51.965707+0000","last_active":"2024-09-10T18:56:15.752473+0000","last_peered":"2024-09-10T18:56:15.752473+0000","last_clean":"2024-09-10T18:56:15.752473+0000","last_became_active":"2024-09-10T18:45:17.964870+0000","last_became_peered":"2024-09-10T18:45:17.964870+0000","last_unstale":"2024-09-10T18:56:15.752473+0000","last_undegraded":"2024-09-10T18:56:15.752473+0000","last_fullsized":"2024-09-10T18:56:15.752473+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:51.965637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:54:51.965637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:18.723317+0000","scrub_duration":0.011107304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"242'10","reported_seq":417,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:21.206486+0000","last_change":"2024-09-10T18:56:07.787017+0000","last_active":"2024-09-10T18:56:21.206486+0000","last_peered":"2024-09-10T18:56:21.206486+0000","last_clean":"2024-09-10T18:56:21.206486+0000","last_became_active":"2024-09-10T18:55:40.391439+0000","last_became_peered":"2024-09-10T18:55:40.391439+0000","last_unstale":"2024-09-10T18:56:21.206486+0000","last_undegraded":"2024-09-10T18:56:21.206486+0000","last_fullsized":"2024-09-10T18:56:21.206486+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"242'10","last_scrub_stamp":"2024-09-10T18:56:07.786933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:07.786933+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:30.536318+0000","scrub_duration":0.0054185049999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":201,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.751614+0000","last_change":"2024-09-10T18:55:40.394192+0000","last_active":"2024-09-10T18:56:15.751614+0000","last_peered":"2024-09-10T18:56:15.751614+0000","last_clean":"2024-09-10T18:56:15.751614+0000","last_became_active":"2024-09-10T18:55:40.390667+0000","last_became_peered":"2024-09-10T18:55:40.390667+0000","last_unstale":"2024-09-10T18:56:15.751614+0000","last_undegraded":"2024-09-10T18:56:15.751614+0000","last_fullsized":"2024-09-10T18:56:15.751614+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:57.476724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:57.476724+0000","last_clean_scrub_stamp":"2024-09-10T18:52:57.476724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.10","version":"70'81","reported_seq":118,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:20.743662+0000","last_change":"2024-09-10T18:56:20.743618+0000","last_active":"2024-09-10T18:56:20.743662+0000","last_peered":"2024-09-10T18:56:20.743662+0000","last_clean":"2024-09-10T18:56:20.743662+0000","last_became_active":"2024-09-10T18:52:16.601616+0000","last_became_peered":"2024-09-10T18:52:16.601616+0000","last_unstale":"2024-09-10T18:56:20.743662+0000","last_undegraded":"2024-09-10T18:56:20.743662+0000","last_fullsized":"2024-09-10T18:56:20.743662+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:20.743543+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:20.743543+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:24.943073+0000","scrub_duration":0.0043223280000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"232'15","reported_seq":150,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.758138+0000","last_change":"2024-09-10T18:55:25.784395+0000","last_active":"2024-09-10T18:56:15.758138+0000","last_peered":"2024-09-10T18:56:15.758138+0000","last_clean":"2024-09-10T18:56:15.758138+0000","last_became_active":"2024-09-10T18:52:54.618780+0000","last_became_peered":"2024-09-10T18:52:54.618780+0000","last_unstale":"2024-09-10T18:56:15.758138+0000","last_undegraded":"2024-09-10T18:56:15.758138+0000","last_fullsized":"2024-09-10T18:56:15.758138+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T18:55:25.784333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:55:25.784333+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:42.282358+0000","scrub_duration":0.0015586210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":236,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:23.096555+0000","last_change":"2024-09-10T18:56:23.096461+0000","last_active":"2024-09-10T18:56:23.096555+0000","last_peered":"2024-09-10T18:56:23.096555+0000","last_clean":"2024-09-10T18:56:23.096555+0000","last_became_active":"2024-09-10T18:55:40.390697+0000","last_became_peered":"2024-09-10T18:55:40.390697+0000","last_unstale":"2024-09-10T18:56:23.096555+0000","last_undegraded":"2024-09-10T18:56:23.096555+0000","last_fullsized":"2024-09-10T18:56:23.096555+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:23.096373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:23.096373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:52.573724+0000","scrub_duration":0.0057615890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"89'10","reported_seq":403,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:21.334646+0000","last_change":"2024-09-10T18:55:36.732976+0000","last_active":"2024-09-10T18:56:21.334646+0000","last_peered":"2024-09-10T18:56:21.334646+0000","last_clean":"2024-09-10T18:56:21.334646+0000","last_became_active":"2024-09-10T18:47:36.424575+0000","last_became_peered":"2024-09-10T18:47:36.424575+0000","last_unstale":"2024-09-10T18:56:21.334646+0000","last_undegraded":"2024-09-10T18:56:21.334646+0000","last_fullsized":"2024-09-10T18:56:21.334646+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'10","last_scrub_stamp":"2024-09-10T18:55:36.732908+0000","last_deep_scrub":"81'9","last_deep_scrub_stamp":"2024-09-10T18:46:54.800790+0000","last_clean_scrub_stamp":"2024-09-10T18:55:36.732908+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:39.212655+0000","scrub_duration":0.0016257330000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":279,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:20.584938+0000","last_change":"2024-09-10T18:56:20.584899+0000","last_active":"2024-09-10T18:56:20.584938+0000","last_peered":"2024-09-10T18:56:20.584938+0000","last_clean":"2024-09-10T18:56:20.584938+0000","last_became_active":"2024-09-10T18:55:40.391662+0000","last_became_peered":"2024-09-10T18:55:40.391662+0000","last_unstale":"2024-09-10T18:56:20.584938+0000","last_undegraded":"2024-09-10T18:56:20.584938+0000","last_fullsized":"2024-09-10T18:56:20.584938+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:20.584808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_clean_scrub_stamp":"2024-09-10T18:56:20.584808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:43.647662+0000","scrub_duration":0.0035961249999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"232'15","reported_seq":322,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.758460+0000","last_change":"2024-09-10T18:56:02.500313+0000","last_active":"2024-09-10T18:56:15.758460+0000","last_peered":"2024-09-10T18:56:15.758460+0000","last_clean":"2024-09-10T18:56:15.758460+0000","last_became_active":"2024-09-10T18:55:40.400788+0000","last_became_peered":"2024-09-10T18:55:40.400788+0000","last_unstale":"2024-09-10T18:56:15.758460+0000","last_undegraded":"2024-09-10T18:56:15.758460+0000","last_fullsized":"2024-09-10T18:56:15.758460+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T18:56:02.500101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:02.500101+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:26.361106+0000","scrub_duration":0.0097839320000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":280,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:18.272245+0000","last_change":"2024-09-10T18:56:18.272207+0000","last_active":"2024-09-10T18:56:18.272245+0000","last_peered":"2024-09-10T18:56:18.272245+0000","last_clean":"2024-09-10T18:56:18.272245+0000","last_became_active":"2024-09-10T18:52:54.620353+0000","last_became_peered":"2024-09-10T18:52:54.620353+0000","last_unstale":"2024-09-10T18:56:18.272245+0000","last_undegraded":"2024-09-10T18:56:18.272245+0000","last_fullsized":"2024-09-10T18:56:18.272245+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:18.272126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:18.272126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:19.841179+0000","scrub_duration":0.0035259570000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":263,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.759708+0000","last_change":"2024-09-10T18:56:04.753218+0000","last_active":"2024-09-10T18:56:15.759708+0000","last_peered":"2024-09-10T18:56:15.759708+0000","last_clean":"2024-09-10T18:56:15.759708+0000","last_became_active":"2024-09-10T18:55:40.398243+0000","last_became_peered":"2024-09-10T18:55:40.398243+0000","last_unstale":"2024-09-10T18:56:15.759708+0000","last_undegraded":"2024-09-10T18:56:15.759708+0000","last_fullsized":"2024-09-10T18:56:15.759708+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:04.753133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:03.571786+0000","last_clean_scrub_stamp":"2024-09-10T18:56:04.753133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:32.426506+0000","scrub_duration":0.0069575230000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":231,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.752232+0000","last_change":"2024-09-10T18:56:04.126848+0000","last_active":"2024-09-10T18:56:15.752232+0000","last_peered":"2024-09-10T18:56:15.752232+0000","last_clean":"2024-09-10T18:56:15.752232+0000","last_became_active":"2024-09-10T18:55:40.396932+0000","last_became_peered":"2024-09-10T18:55:40.396932+0000","last_unstale":"2024-09-10T18:56:15.752232+0000","last_undegraded":"2024-09-10T18:56:15.752232+0000","last_fullsized":"2024-09-10T18:56:15.752232+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:04.126516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:04.631558+0000","last_clean_scrub_stamp":"2024-09-10T18:56:04.126516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:25.286369+0000","scrub_duration":0.010034405999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"232'24","reported_seq":282,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.763184+0000","last_change":"2024-09-10T18:54:47.324126+0000","last_active":"2024-09-10T18:56:15.763184+0000","last_peered":"2024-09-10T18:56:15.763184+0000","last_clean":"2024-09-10T18:56:15.763184+0000","last_became_active":"2024-09-10T18:45:20.056885+0000","last_became_peered":"2024-09-10T18:45:20.056885+0000","last_unstale":"2024-09-10T18:56:15.763184+0000","last_undegraded":"2024-09-10T18:56:15.763184+0000","last_fullsized":"2024-09-10T18:56:15.763184+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'16","last_scrub_stamp":"2024-09-10T18:54:47.324044+0000","last_deep_scrub":"223'16","last_deep_scrub_stamp":"2024-09-10T18:54:47.324044+0000","last_clean_scrub_stamp":"2024-09-10T18:54:47.324044+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016282413999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":267,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.766100+0000","last_change":"2024-09-10T18:55:40.399504+0000","last_active":"2024-09-10T18:56:15.766100+0000","last_peered":"2024-09-10T18:56:15.766100+0000","last_clean":"2024-09-10T18:56:15.766100+0000","last_became_active":"2024-09-10T18:55:40.399198+0000","last_became_peered":"2024-09-10T18:55:40.399198+0000","last_unstale":"2024-09-10T18:56:15.766100+0000","last_undegraded":"2024-09-10T18:56:15.766100+0000","last_fullsized":"2024-09-10T18:56:15.766100+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:32.290320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:00.029389+0000","last_clean_scrub_stamp":"2024-09-10T18:53:32.290320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003302677,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":249,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.758337+0000","last_change":"2024-09-10T18:52:43.135942+0000","last_active":"2024-09-10T18:56:15.758337+0000","last_peered":"2024-09-10T18:56:15.758337+0000","last_clean":"2024-09-10T18:56:15.758337+0000","last_became_active":"2024-09-10T18:45:22.070065+0000","last_became_peered":"2024-09-10T18:45:22.070065+0000","last_unstale":"2024-09-10T18:56:15.758337+0000","last_undegraded":"2024-09-10T18:56:15.758337+0000","last_fullsized":"2024-09-10T18:56:15.758337+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:43.135170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:52:43.135170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011350617,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":261,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.760579+0000","last_change":"2024-09-10T18:56:06.792096+0000","last_active":"2024-09-10T18:56:15.760579+0000","last_peered":"2024-09-10T18:56:15.760579+0000","last_clean":"2024-09-10T18:56:15.760579+0000","last_became_active":"2024-09-10T18:55:40.395595+0000","last_became_peered":"2024-09-10T18:55:40.395595+0000","last_unstale":"2024-09-10T18:56:15.760579+0000","last_undegraded":"2024-09-10T18:56:15.760579+0000","last_fullsized":"2024-09-10T18:56:15.760579+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:06.791860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:06.791860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:26.074630+0000","scrub_duration":0.0046852380000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"232'8","reported_seq":260,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.760344+0000","last_change":"2024-09-10T18:52:56.832280+0000","last_active":"2024-09-10T18:56:15.760344+0000","last_peered":"2024-09-10T18:56:15.760344+0000","last_clean":"2024-09-10T18:56:15.760344+0000","last_became_active":"2024-09-10T18:45:20.059205+0000","last_became_peered":"2024-09-10T18:45:20.059205+0000","last_unstale":"2024-09-10T18:56:15.760344+0000","last_undegraded":"2024-09-10T18:56:15.760344+0000","last_fullsized":"2024-09-10T18:56:15.760344+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4","last_scrub_stamp":"2024-09-10T18:52:56.832176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:52:56.832176+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.015287545,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":266,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.756148+0000","last_change":"2024-09-10T18:55:25.897258+0000","last_active":"2024-09-10T18:56:15.756148+0000","last_peered":"2024-09-10T18:56:15.756148+0000","last_clean":"2024-09-10T18:56:15.756148+0000","last_became_active":"2024-09-10T18:47:36.415866+0000","last_became_peered":"2024-09-10T18:47:36.415866+0000","last_unstale":"2024-09-10T18:56:15.756148+0000","last_undegraded":"2024-09-10T18:56:15.756148+0000","last_fullsized":"2024-09-10T18:56:15.756148+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:25.897205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:55:25.897205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:29.254051+0000","scrub_duration":0.001389787,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":276,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:23.195297+0000","last_change":"2024-09-10T18:56:23.195250+0000","last_active":"2024-09-10T18:56:23.195297+0000","last_peered":"2024-09-10T18:56:23.195297+0000","last_clean":"2024-09-10T18:56:23.195297+0000","last_became_active":"2024-09-10T18:52:58.103465+0000","last_became_peered":"2024-09-10T18:52:58.103465+0000","last_unstale":"2024-09-10T18:56:23.195297+0000","last_undegraded":"2024-09-10T18:56:23.195297+0000","last_fullsized":"2024-09-10T18:56:23.195297+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:23.195060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_clean_scrub_stamp":"2024-09-10T18:56:23.195060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:23.450987+0000","scrub_duration":0.0047613259999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":268,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.757693+0000","last_change":"2024-09-10T18:55:28.768157+0000","last_active":"2024-09-10T18:56:15.757693+0000","last_peered":"2024-09-10T18:56:15.757693+0000","last_clean":"2024-09-10T18:56:15.757693+0000","last_became_active":"2024-09-10T18:47:36.416187+0000","last_became_peered":"2024-09-10T18:47:36.416187+0000","last_unstale":"2024-09-10T18:56:15.757693+0000","last_undegraded":"2024-09-10T18:56:15.757693+0000","last_fullsized":"2024-09-10T18:56:15.757693+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:28.768091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:55.145638+0000","last_clean_scrub_stamp":"2024-09-10T18:55:28.768091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:54.162730+0000","scrub_duration":0.0027672999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.14","version":"70'81","reported_seq":278,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.755840+0000","last_change":"2024-09-10T18:55:34.791281+0000","last_active":"2024-09-10T18:56:15.755840+0000","last_peered":"2024-09-10T18:56:15.755840+0000","last_clean":"2024-09-10T18:56:15.755840+0000","last_became_active":"2024-09-10T18:52:16.600485+0000","last_became_peered":"2024-09-10T18:52:16.600485+0000","last_unstale":"2024-09-10T18:56:15.755840+0000","last_undegraded":"2024-09-10T18:56:15.755840+0000","last_fullsized":"2024-09-10T18:56:15.755840+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:55:34.791223+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:57.184383+0000","last_clean_scrub_stamp":"2024-09-10T18:55:34.791223+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:41.490465+0000","scrub_duration":0.0013477910000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"232'24","reported_seq":284,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.758909+0000","last_change":"2024-09-10T18:54:39.420770+0000","last_active":"2024-09-10T18:56:15.758909+0000","last_peered":"2024-09-10T18:56:15.758909+0000","last_clean":"2024-09-10T18:56:15.758909+0000","last_became_active":"2024-09-10T18:45:20.421479+0000","last_became_peered":"2024-09-10T18:45:20.421479+0000","last_unstale":"2024-09-10T18:56:15.758909+0000","last_undegraded":"2024-09-10T18:56:15.758909+0000","last_fullsized":"2024-09-10T18:56:15.758909+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"79'16","last_scrub_stamp":"2024-09-10T18:54:39.420695+0000","last_deep_scrub":"79'16","last_deep_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_clean_scrub_stamp":"2024-09-10T18:54:39.420695+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.052085645,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"56'1","reported_seq":291,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:25.211227+0000","last_change":"2024-09-10T18:56:25.211179+0000","last_active":"2024-09-10T18:56:25.211227+0000","last_peered":"2024-09-10T18:56:25.211227+0000","last_clean":"2024-09-10T18:56:25.211227+0000","last_became_active":"2024-09-10T18:47:36.421985+0000","last_became_peered":"2024-09-10T18:47:36.421985+0000","last_unstale":"2024-09-10T18:56:25.211227+0000","last_undegraded":"2024-09-10T18:56:25.211227+0000","last_fullsized":"2024-09-10T18:56:25.211227+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:56:25.211095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:25.211095+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:47.313472+0000","scrub_duration":0.0025468880000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":210,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.759340+0000","last_change":"2024-09-10T18:56:12.852846+0000","last_active":"2024-09-10T18:56:15.759340+0000","last_peered":"2024-09-10T18:56:15.759340+0000","last_clean":"2024-09-10T18:56:15.759340+0000","last_became_active":"2024-09-10T18:47:57.107149+0000","last_became_peered":"2024-09-10T18:47:57.107149+0000","last_unstale":"2024-09-10T18:56:15.759340+0000","last_undegraded":"2024-09-10T18:56:15.759340+0000","last_fullsized":"2024-09-10T18:56:15.759340+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:12.852769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:12.852769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:40.529459+0000","scrub_duration":0.002947716,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":271,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.759555+0000","last_change":"2024-09-10T18:56:09.903930+0000","last_active":"2024-09-10T18:56:15.759555+0000","last_peered":"2024-09-10T18:56:15.759555+0000","last_clean":"2024-09-10T18:56:15.759555+0000","last_became_active":"2024-09-10T18:55:40.397523+0000","last_became_peered":"2024-09-10T18:55:40.397523+0000","last_unstale":"2024-09-10T18:56:15.759555+0000","last_undegraded":"2024-09-10T18:56:15.759555+0000","last_fullsized":"2024-09-10T18:56:15.759555+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:09.903846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:09.903846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:27.318802+0000","scrub_duration":0.011721984,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"78'2","reported_seq":274,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.756114+0000","last_change":"2024-09-10T18:54:27.784969+0000","last_active":"2024-09-10T18:56:15.756114+0000","last_peered":"2024-09-10T18:56:15.756114+0000","last_clean":"2024-09-10T18:56:15.756114+0000","last_became_active":"2024-09-10T18:47:36.418702+0000","last_became_peered":"2024-09-10T18:47:36.418702+0000","last_unstale":"2024-09-10T18:56:15.756114+0000","last_undegraded":"2024-09-10T18:56:15.756114+0000","last_fullsized":"2024-09-10T18:56:15.756114+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'2","last_scrub_stamp":"2024-09-10T18:54:27.784508+0000","last_deep_scrub":"78'2","last_deep_scrub_stamp":"2024-09-10T18:46:37.581130+0000","last_clean_scrub_stamp":"2024-09-10T18:54:27.784508+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011474105,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":260,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:16.743920+0000","last_change":"2024-09-10T18:56:16.743865+0000","last_active":"2024-09-10T18:56:16.743920+0000","last_peered":"2024-09-10T18:56:16.743920+0000","last_clean":"2024-09-10T18:56:16.743920+0000","last_became_active":"2024-09-10T18:45:17.962108+0000","last_became_peered":"2024-09-10T18:45:17.962108+0000","last_unstale":"2024-09-10T18:56:16.743920+0000","last_undegraded":"2024-09-10T18:56:16.743920+0000","last_fullsized":"2024-09-10T18:56:16.743920+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:16.743764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:16.743764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:26.799158+0000","scrub_duration":0.0031221030000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"232'20","reported_seq":253,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.751761+0000","last_change":"2024-09-10T18:55:40.413383+0000","last_active":"2024-09-10T18:56:15.751761+0000","last_peered":"2024-09-10T18:56:15.751761+0000","last_clean":"2024-09-10T18:56:15.751761+0000","last_became_active":"2024-09-10T18:55:40.393341+0000","last_became_peered":"2024-09-10T18:55:40.393341+0000","last_unstale":"2024-09-10T18:56:15.751761+0000","last_undegraded":"2024-09-10T18:56:15.751761+0000","last_fullsized":"2024-09-10T18:56:15.751761+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'10","last_scrub_stamp":"2024-09-10T18:52:00.358298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:52:00.358298+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":23,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"190'9","reported_seq":412,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:21.204461+0000","last_change":"2024-09-10T18:56:19.259959+0000","last_active":"2024-09-10T18:56:21.204461+0000","last_peered":"2024-09-10T18:56:21.204461+0000","last_clean":"2024-09-10T18:56:21.204461+0000","last_became_active":"2024-09-10T18:45:22.073281+0000","last_became_peered":"2024-09-10T18:45:22.073281+0000","last_unstale":"2024-09-10T18:56:21.204461+0000","last_undegraded":"2024-09-10T18:56:21.204461+0000","last_fullsized":"2024-09-10T18:56:21.204461+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"190'9","last_scrub_stamp":"2024-09-10T18:56:19.259869+0000","last_deep_scrub":"190'9","last_deep_scrub_stamp":"2024-09-10T18:53:07.746309+0000","last_clean_scrub_stamp":"2024-09-10T18:56:19.259869+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:22.578672+0000","scrub_duration":0.0020087130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":247,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.765334+0000","last_change":"2024-09-10T18:54:24.631566+0000","last_active":"2024-09-10T18:56:15.765334+0000","last_peered":"2024-09-10T18:56:15.765334+0000","last_clean":"2024-09-10T18:56:15.765334+0000","last_became_active":"2024-09-10T18:45:24.082318+0000","last_became_peered":"2024-09-10T18:45:24.082318+0000","last_unstale":"2024-09-10T18:56:15.765334+0000","last_undegraded":"2024-09-10T18:56:15.765334+0000","last_fullsized":"2024-09-10T18:56:15.765334+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:24.631469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:03.475299+0000","last_clean_scrub_stamp":"2024-09-10T18:54:24.631469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014197107,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":267,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.758429+0000","last_change":"2024-09-10T18:56:00.878293+0000","last_active":"2024-09-10T18:56:15.758429+0000","last_peered":"2024-09-10T18:56:15.758429+0000","last_clean":"2024-09-10T18:56:15.758429+0000","last_became_active":"2024-09-10T18:55:40.392524+0000","last_became_peered":"2024-09-10T18:55:40.392524+0000","last_unstale":"2024-09-10T18:56:15.758429+0000","last_undegraded":"2024-09-10T18:56:15.758429+0000","last_fullsized":"2024-09-10T18:56:15.758429+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:00.877849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:00.877849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:25.495379+0000","scrub_duration":0.0096660570000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"232'19","reported_seq":292,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.756674+0000","last_change":"2024-09-10T18:55:26.062315+0000","last_active":"2024-09-10T18:56:15.756674+0000","last_peered":"2024-09-10T18:56:15.756674+0000","last_clean":"2024-09-10T18:56:15.756674+0000","last_became_active":"2024-09-10T18:45:20.058717+0000","last_became_peered":"2024-09-10T18:45:20.058717+0000","last_unstale":"2024-09-10T18:56:15.756674+0000","last_undegraded":"2024-09-10T18:56:15.756674+0000","last_fullsized":"2024-09-10T18:56:15.756674+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T18:55:26.062256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:55:26.062256+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:50.509567+0000","scrub_duration":0.001573893,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":209,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.752125+0000","last_change":"2024-09-10T18:55:40.398399+0000","last_active":"2024-09-10T18:56:15.752125+0000","last_peered":"2024-09-10T18:56:15.752125+0000","last_clean":"2024-09-10T18:56:15.752125+0000","last_became_active":"2024-09-10T18:55:40.396437+0000","last_became_peered":"2024-09-10T18:55:40.396437+0000","last_unstale":"2024-09-10T18:56:15.752125+0000","last_undegraded":"2024-09-10T18:56:15.752125+0000","last_fullsized":"2024-09-10T18:56:15.752125+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:52.592184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:52:52.592184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015375409999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":242,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.763636+0000","last_change":"2024-09-10T18:54:21.072224+0000","last_active":"2024-09-10T18:56:15.763636+0000","last_peered":"2024-09-10T18:56:15.763636+0000","last_clean":"2024-09-10T18:56:15.763636+0000","last_became_active":"2024-09-10T18:45:24.267251+0000","last_became_peered":"2024-09-10T18:45:24.267251+0000","last_unstale":"2024-09-10T18:56:15.763636+0000","last_undegraded":"2024-09-10T18:56:15.763636+0000","last_fullsized":"2024-09-10T18:56:15.763636+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:21.071910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:54:21.071910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007087388,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"56'1","reported_seq":272,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.756412+0000","last_change":"2024-09-10T18:56:05.159063+0000","last_active":"2024-09-10T18:56:15.756412+0000","last_peered":"2024-09-10T18:56:15.756412+0000","last_clean":"2024-09-10T18:56:15.756412+0000","last_became_active":"2024-09-10T18:45:17.966286+0000","last_became_peered":"2024-09-10T18:45:17.966286+0000","last_unstale":"2024-09-10T18:56:15.756412+0000","last_undegraded":"2024-09-10T18:56:15.756412+0000","last_fullsized":"2024-09-10T18:56:15.756412+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:56:05.158970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:05.158970+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:21.053145+0000","scrub_duration":0.0055166130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"258'34","reported_seq":321,"reported_epoch":258,"state":"active+clean","last_fresh":"2024-09-10T18:56:23.839232+0000","last_change":"2024-09-10T18:56:23.839043+0000","last_active":"2024-09-10T18:56:23.839232+0000","last_peered":"2024-09-10T18:56:23.839232+0000","last_clean":"2024-09-10T18:56:23.839232+0000","last_became_active":"2024-09-10T18:55:40.401788+0000","last_became_peered":"2024-09-10T18:55:40.401788+0000","last_unstale":"2024-09-10T18:56:23.839232+0000","last_undegraded":"2024-09-10T18:56:23.839232+0000","last_fullsized":"2024-09-10T18:56:23.839232+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'34","last_scrub_stamp":"2024-09-10T18:56:23.838951+0000","last_deep_scrub":"258'34","last_deep_scrub_stamp":"2024-09-10T18:56:23.838951+0000","last_clean_scrub_stamp":"2024-09-10T18:56:23.838951+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:49.645990+0000","scrub_duration":0.035929888,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":263,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.765795+0000","last_change":"2024-09-10T18:56:13.611288+0000","last_active":"2024-09-10T18:56:15.765795+0000","last_peered":"2024-09-10T18:56:15.765795+0000","last_clean":"2024-09-10T18:56:15.765795+0000","last_became_active":"2024-09-10T18:55:40.397870+0000","last_became_peered":"2024-09-10T18:55:40.397870+0000","last_unstale":"2024-09-10T18:56:15.765795+0000","last_undegraded":"2024-09-10T18:56:15.765795+0000","last_fullsized":"2024-09-10T18:56:15.765795+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:13.610976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:13.610976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:40.012534+0000","scrub_duration":0.010999553,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":254,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.758188+0000","last_change":"2024-09-10T18:55:39.815156+0000","last_active":"2024-09-10T18:56:15.758188+0000","last_peered":"2024-09-10T18:56:15.758188+0000","last_clean":"2024-09-10T18:56:15.758188+0000","last_became_active":"2024-09-10T18:52:54.624256+0000","last_became_peered":"2024-09-10T18:52:54.624256+0000","last_unstale":"2024-09-10T18:56:15.758188+0000","last_undegraded":"2024-09-10T18:56:15.758188+0000","last_fullsized":"2024-09-10T18:56:15.758188+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:39.815087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:39.815087+0000","last_clean_scrub_stamp":"2024-09-10T18:55:39.815087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:56:41.582754+0000","scrub_duration":0.0018555970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":250,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.762414+0000","last_change":"2024-09-10T18:54:46.433255+0000","last_active":"2024-09-10T18:56:15.762414+0000","last_peered":"2024-09-10T18:56:15.762414+0000","last_clean":"2024-09-10T18:56:15.762414+0000","last_became_active":"2024-09-10T18:45:17.963257+0000","last_became_peered":"2024-09-10T18:45:17.963257+0000","last_unstale":"2024-09-10T18:56:15.762414+0000","last_undegraded":"2024-09-10T18:56:15.762414+0000","last_fullsized":"2024-09-10T18:56:15.762414+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:46.433200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:54:46.433200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013571259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"232'18","reported_seq":280,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.758062+0000","last_change":"2024-09-10T18:54:21.298808+0000","last_active":"2024-09-10T18:56:15.758062+0000","last_peered":"2024-09-10T18:56:15.758062+0000","last_clean":"2024-09-10T18:56:15.758062+0000","last_became_active":"2024-09-10T18:45:20.058391+0000","last_became_peered":"2024-09-10T18:45:20.058391+0000","last_unstale":"2024-09-10T18:56:15.758062+0000","last_undegraded":"2024-09-10T18:56:15.758062+0000","last_fullsized":"2024-09-10T18:56:15.758062+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:54:21.298731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:54:21.298731+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060418820000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":264,"reported_epoch":257,"state":"active+clean","last_fresh":"2024-09-10T18:56:15.763427+0000","last_change":"2024-09-10T18:54:53.316597+0000","last_active":"2024-09-10T18:56:15.763427+0000","last_peered":"2024-09-10T18:56:15.763427+0000","last_clean":"2024-09-10T18:56:15.763427+0000","last_became_active":"2024-09-10T18:47:36.416131+0000","last_became_peered":"2024-09-10T18:47:36.416131+0000","last_unstale":"2024-09-10T18:56:15.763427+0000","last_undegraded":"2024-09-10T18:56:15.763427+0000","last_fullsized":"2024-09-10T18:56:15.763427+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:53.316517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:54:53.316517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0090189510000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":4788387840,"num_objects":73065,"num_object_clones":0,"num_object_copies":219195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73065,"num_write_kb":4676160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":14420934656,"data_stored":14420934656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64065,"ondisk_log_size":64065,"up":48,"acting":48,"num_store_stats":7},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80,"ondisk_log_size":80,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1120,"num_read_kb":877,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":1458,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":12014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":758,"ondisk_log_size":758,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":652,"num_read_kb":647,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1701,"ondisk_log_size":1701,"up":63,"acting":63,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":52,"seq":223338299530,"num_pgs":81,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3519016,"kb_used_data":3217504,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":90226136,"statfs":{"total":95995035648,"available":92391563264,"internally_reserved":0,"allocated":3294724096,"data_stored":3294077348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:53:20 2024","interfaces":[{"interface":"back","average":{"1min":1.746,"5min":1.2689999999999999,"15min":0.873},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":11.85,"5min":21.231000000000002,"15min":31.805},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.595,"5min":1.4770000000000001,"15min":0.92300000000000004},"min":{"1min":0.22800000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":11.484999999999999,"5min":21.422999999999998,"15min":31.212},"last":0.91000000000000003}]},{"osd":1,"last update":"Tue Sep 10 18:55:26 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.022,"15min":0.81699999999999995},"min":{"1min":0.25,"5min":0.23100000000000001,"15min":0.221},"max":{"1min":4.702,"5min":8.6189999999999998,"15min":21.34},"last":1.5429999999999999},{"interface":"front","average":{"1min":0.89600000000000002,"5min":1.0589999999999999,"15min":0.82899999999999996},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":4.7759999999999998,"5min":8.5899999999999999,"15min":21.248000000000001},"last":0.95499999999999996}]},{"osd":2,"last update":"Tue Sep 10 18:55:26 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.755,"15min":0.626},"min":{"1min":0.28199999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":4.516,"5min":4.9749999999999996,"15min":5.1399999999999997},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.876,"15min":0.69399999999999995},"min":{"1min":0.32300000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.0449999999999999,"5min":3.9830000000000001,"15min":8.8859999999999992},"last":1.0289999999999999}]},{"osd":3,"last update":"Tue Sep 10 18:55:26 2024","interfaces":[{"interface":"back","average":{"1min":1.742,"5min":1.361,"15min":1.036},"min":{"1min":0.26100000000000001,"5min":0.251,"15min":0.23799999999999999},"max":{"1min":7.5899999999999999,"5min":11.282,"15min":31.478000000000002},"last":0.46700000000000003},{"interface":"front","average":{"1min":1.167,"5min":1.3220000000000001,"15min":1.095},"min":{"1min":0.34499999999999997,"5min":0.29499999999999998,"15min":0.254},"max":{"1min":6.1470000000000002,"5min":10.185,"15min":33.731999999999999},"last":1.0029999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:55:26 2024","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.1699999999999999,"15min":0.66700000000000004},"min":{"1min":0.28000000000000003,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":7.6890000000000001,"5min":7.6890000000000001,"15min":7.6890000000000001},"last":0.875},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.2310000000000001,"15min":0.68600000000000005},"min":{"1min":0.23000000000000001,"5min":0.22,"15min":0.22},"max":{"1min":2.8969999999999998,"5min":6.7270000000000003,"15min":6.7270000000000003},"last":0.93400000000000005}]},{"osd":5,"last update":"Tue Sep 10 18:55:26 2024","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":2.5499999999999998,"15min":1.141},"min":{"1min":0.27600000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":7.7789999999999999,"5min":113.255,"15min":113.255},"last":2.5219999999999998},{"interface":"front","average":{"1min":1.079,"5min":2.2799999999999998,"15min":1.0489999999999999},"min":{"1min":0.29599999999999999,"5min":0.25900000000000001,"15min":0.253},"max":{"1min":3.0049999999999999,"5min":113.851,"15min":113.851},"last":1.4359999999999999}]},{"osd":6,"last update":"Tue Sep 10 18:55:26 2024","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":2.1829999999999998,"15min":1.0549999999999999},"min":{"1min":0.25,"5min":0.247,"15min":0.247},"max":{"1min":5.7110000000000003,"5min":87.716999999999999,"15min":87.716999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":2.0289999999999999,"5min":2.157,"15min":1.0409999999999999},"min":{"1min":0.317,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":7.6609999999999996,"5min":87.738,"15min":87.738},"last":0.41899999999999998}]}]},{"osd":6,"up_from":43,"seq":184683593871,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1486456,"kb_used_data":1180784,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":92258696,"statfs":{"total":95995035648,"available":94472904704,"internally_reserved":0,"allocated":1209122816,"data_stored":1208460882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:53:51 2024","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.583,"15min":0.89600000000000002},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":3.1819999999999999,"5min":46.384999999999998,"15min":46.384999999999998},"last":0.20899999999999999},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.77,"15min":0.94899999999999995},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.22600000000000001},"max":{"1min":3.3199999999999998,"5min":48.372999999999998,"15min":48.372999999999998},"last":0.46899999999999997}]},{"osd":1,"last update":"Tue Sep 10 18:55:54 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":1.252,"15min":0.873},"min":{"1min":0.34499999999999997,"5min":0.251,"15min":0.219},"max":{"1min":3.3370000000000002,"5min":9.2509999999999994,"15min":9.2509999999999994},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.1619999999999999,"15min":0.78600000000000003},"min":{"1min":0.33200000000000002,"5min":0.23799999999999999,"15min":0.214},"max":{"1min":4.8140000000000001,"5min":9.1750000000000007,"15min":9.1750000000000007},"last":0.28699999999999998}]},{"osd":2,"last update":"Tue Sep 10 18:55:54 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94399999999999995,"15min":0.72799999999999998},"min":{"1min":0.33200000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":4.1029999999999998,"5min":4.1029999999999998,"15min":4.4740000000000002},"last":0.438},{"interface":"front","average":{"1min":0.753,"5min":0.94799999999999995,"15min":0.68100000000000005},"min":{"1min":0.34999999999999998,"5min":0.29199999999999998,"15min":0.27200000000000002},"max":{"1min":3.2949999999999999,"5min":4.8150000000000004,"15min":4.8150000000000004},"last":0.38400000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:55:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.127,"15min":0.80400000000000005},"min":{"1min":0.35999999999999999,"5min":0.28399999999999997,"15min":0.25700000000000001},"max":{"1min":4.8479999999999999,"5min":10.891999999999999,"15min":10.891999999999999},"last":0.318},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.26,"15min":0.92500000000000004},"min":{"1min":0.28699999999999998,"5min":0.253,"15min":0.253},"max":{"1min":4.1260000000000003,"5min":10.657,"15min":10.657},"last":0.64900000000000002}]},{"osd":4,"last update":"Tue Sep 10 18:55:54 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":2.7869999999999999,"15min":1.25},"min":{"1min":0.23999999999999999,"5min":0.23499999999999999,"15min":0.20999999999999999},"max":{"1min":3.4100000000000001,"5min":165.03299999999999,"15min":165.03299999999999},"last":1.3759999999999999},{"interface":"front","average":{"1min":0.92100000000000004,"5min":2.5920000000000001,"15min":1.163},"min":{"1min":0.26500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":4.9450000000000003,"5min":163.959,"15min":163.959},"last":1.0580000000000001}]},{"osd":5,"last update":"Tue Sep 10 18:55:54 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.655,"15min":0.877},"min":{"1min":0.29299999999999998,"5min":0.26700000000000002,"15min":0.24099999999999999},"max":{"1min":4.3849999999999998,"5min":43.811,"15min":43.811},"last":4.1159999999999997},{"interface":"front","average":{"1min":1.022,"5min":1.716,"15min":0.90500000000000003},"min":{"1min":0.31,"5min":0.27800000000000002,"15min":0.23999999999999999},"max":{"1min":5.5609999999999999,"5min":42.314,"15min":42.314},"last":1.351}]},{"osd":7,"last update":"Tue Sep 10 18:56:23 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":1.1579999999999999,"15min":0.76500000000000001},"min":{"1min":0.34399999999999997,"5min":0.29799999999999999,"15min":0.26200000000000001},"max":{"1min":4.1580000000000004,"5min":5.3680000000000003,"15min":5.3680000000000003},"last":4.1580000000000004},{"interface":"front","average":{"1min":1.036,"5min":1.2709999999999999,"15min":0.81200000000000006},"min":{"1min":0.313,"5min":0.313,"15min":0.29099999999999998},"max":{"1min":3.2759999999999998,"5min":7.0350000000000001,"15min":7.0350000000000001},"last":3.2759999999999998}]}]},{"osd":5,"up_from":37,"seq":158913790099,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2138204,"kb_used_data":1828372,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":91606948,"statfs":{"total":95995035648,"available":93805514752,"internally_reserved":0,"allocated":1872252928,"data_stored":1871617716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:53:32 2024","interfaces":[{"interface":"back","average":{"1min":6.4000000000000004,"5min":2.8849999999999998,"15min":1.3169999999999999},"min":{"1min":0.27400000000000002,"5min":0.183,"15min":0.183},"max":{"1min":73.674000000000007,"5min":73.674000000000007,"15min":73.674000000000007},"last":0.36299999999999999},{"interface":"front","average":{"1min":6.6959999999999997,"5min":2.8260000000000001,"15min":1.321},"min":{"1min":0.27900000000000003,"5min":0.219,"15min":0.219},"max":{"1min":73.488,"5min":73.488,"15min":73.488},"last":0.316}]},{"osd":1,"last update":"Tue Sep 10 18:55:33 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":1.0529999999999999,"15min":0.76300000000000001},"min":{"1min":0.20399999999999999,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":2.5630000000000002,"5min":12.920999999999999,"15min":12.920999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.423,"15min":0.91800000000000004},"min":{"1min":0.23599999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":8.9090000000000007,"5min":19.474,"15min":19.474},"last":0.435}]},{"osd":2,"last update":"Tue Sep 10 18:55:33 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.90500000000000003,"15min":0.68700000000000006},"min":{"1min":0.30099999999999999,"5min":0.27900000000000003,"15min":0.249},"max":{"1min":3.218,"5min":4.6319999999999997,"15min":7.8029999999999999},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.88500000000000001,"15min":0.71999999999999997},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":2.6269999999999998,"5min":4.8680000000000003,"15min":6.7210000000000001},"last":0.29999999999999999}]},{"osd":3,"last update":"Tue Sep 10 18:55:33 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":1.742,"15min":1.2809999999999999},"min":{"1min":0.315,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":3.153,"5min":32.479999999999997,"15min":66.593999999999994},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":2.093,"15min":1.4099999999999999},"min":{"1min":0.34000000000000002,"5min":0.245,"15min":0.245},"max":{"1min":2.6019999999999999,"5min":32.171999999999997,"15min":65.939999999999998},"last":0.45800000000000002}]},{"osd":4,"last update":"Tue Sep 10 18:55:33 2024","interfaces":[{"interface":"back","average":{"1min":1.6739999999999999,"5min":1.48,"15min":0.77600000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.20899999999999999},"max":{"1min":9.0310000000000006,"5min":9.0310000000000006,"15min":9.0310000000000006},"last":0.35299999999999998},{"interface":"front","average":{"1min":1.55,"5min":1.3480000000000001,"15min":0.73799999999999999},"min":{"1min":0.255,"5min":0.24399999999999999,"15min":0.23400000000000001},"max":{"1min":11.182,"5min":11.182,"15min":11.182},"last":0.378}]},{"osd":6,"last update":"Tue Sep 10 18:55:59 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":1.4399999999999999,"15min":0.871},"min":{"1min":0.30299999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.3560000000000001,"5min":10.385,"15min":11.17},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.46400000000000002,"5min":1.431,"15min":0.89600000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.66700000000000004,"5min":7.3719999999999999,"15min":11.18},"last":0.41299999999999998}]},{"osd":7,"last update":"Tue Sep 10 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":1.4410000000000001,"15min":0.871},"min":{"1min":0.33300000000000002,"5min":0.29299999999999998,"15min":0.27600000000000002},"max":{"1min":3.2770000000000001,"5min":5.7249999999999996,"15min":10.782999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":1.204,"15min":0.78700000000000003},"min":{"1min":0.309,"5min":0.24099999999999999,"15min":0.23599999999999999},"max":{"1min":1.956,"5min":5.9500000000000002,"15min":11.846},"last":0.40200000000000002}]}]},{"osd":4,"up_from":30,"seq":128849019033,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1160308,"kb_used_data":862956,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":92584844,"statfs":{"total":95995035648,"available":94806880256,"internally_reserved":0,"allocated":883666944,"data_stored":883008413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:53:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.5900000000000001,"15min":0.88100000000000001},"min":{"1min":0.215,"5min":0.21099999999999999,"15min":0.20699999999999999},"max":{"1min":2.4249999999999998,"5min":55.776000000000003,"15min":55.776000000000003},"last":2.0489999999999999},{"interface":"front","average":{"1min":1.181,"5min":1.607,"15min":0.88900000000000001},"min":{"1min":0.26200000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":4.1429999999999998,"5min":55.737000000000002,"15min":55.737000000000002},"last":2.0699999999999998}]},{"osd":1,"last update":"Tue Sep 10 18:56:18 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.5509999999999999,"15min":0.998},"min":{"1min":0.23799999999999999,"5min":0.22800000000000001,"15min":0.22600000000000001},"max":{"1min":3.9769999999999999,"5min":45.984999999999999,"15min":45.984999999999999},"last":1.274},{"interface":"front","average":{"1min":0.69899999999999995,"5min":1.363,"15min":0.88},"min":{"1min":0.27600000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.921,"5min":43.590000000000003,"15min":43.590000000000003},"last":2.0950000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:56:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.81999999999999995,"15min":0.64300000000000002},"min":{"1min":0.315,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.353,"5min":5.6189999999999998,"15min":5.6189999999999998},"last":2.0259999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.93799999999999994,"15min":0.68300000000000005},"min":{"1min":0.29899999999999999,"5min":0.28000000000000003,"15min":0.27100000000000002},"max":{"1min":1.9730000000000001,"5min":6.3540000000000001,"15min":6.3540000000000001},"last":0.47699999999999998}]},{"osd":3,"last update":"Tue Sep 10 18:56:18 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":1.5660000000000001,"15min":1.0980000000000001},"min":{"1min":0.31,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":4.5140000000000002,"5min":43.826000000000001,"15min":43.826000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":1.4359999999999999,"15min":1.0209999999999999},"min":{"1min":0.29899999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":2.6930000000000001,"5min":43.570999999999998,"15min":43.570999999999998},"last":1.7769999999999999}]},{"osd":5,"last update":"Tue Sep 10 18:55:41 2024","interfaces":[{"interface":"back","average":{"1min":1.5800000000000001,"5min":1.4359999999999999,"15min":0.76000000000000001},"min":{"1min":0.23200000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":10.912000000000001,"5min":10.912000000000001,"15min":10.912000000000001},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.244,"15min":0.68799999999999994},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":5.6429999999999998,"5min":5.6429999999999998,"15min":5.6429999999999998},"last":2.1269999999999998}]},{"osd":6,"last update":"Tue Sep 10 18:56:05 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.129,"15min":0.69899999999999995},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27600000000000002},"max":{"1min":5.5529999999999999,"5min":8.4030000000000005,"15min":8.4030000000000005},"last":1.972},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.341,"15min":0.77900000000000003},"min":{"1min":0.26600000000000001,"5min":0.26400000000000001,"15min":0.25800000000000001},"max":{"1min":4.0999999999999996,"5min":7.7699999999999996,"15min":7.7699999999999996},"last":1.242}]},{"osd":7,"last update":"Tue Sep 10 18:55:28 2024","interfaces":[{"interface":"back","average":{"1min":1.8740000000000001,"5min":1.248,"15min":0.77000000000000002},"min":{"1min":0.33000000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":6.4630000000000001,"5min":6.4630000000000001,"15min":6.4630000000000001},"last":2.0019999999999998},{"interface":"front","average":{"1min":1.538,"5min":1.194,"15min":0.72099999999999997},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":4.5449999999999999,"5min":6.3949999999999996,"15min":6.3949999999999996},"last":1.3620000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182558,"num_pgs":80,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3220988,"kb_used_data":2626932,"kb_used_omap":1,"kb_used_meta":594046,"kb_avail":90524164,"statfs":{"total":95995035648,"available":92696743936,"internally_reserved":0,"allocated":2689978368,"data_stored":2689311368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":608303448},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:53:47 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":2.6099999999999999,"15min":1.214},"min":{"1min":0.182,"5min":0.16700000000000001,"15min":0.16700000000000001},"max":{"1min":2.544,"5min":83.126000000000005,"15min":83.126000000000005},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.95199999999999996,"5min":2.5699999999999998,"15min":1.2150000000000001},"min":{"1min":0.19,"5min":0.184,"15min":0.184},"max":{"1min":6.2800000000000002,"5min":81.632999999999996,"15min":81.632999999999996},"last":0.379}]},{"osd":1,"last update":"Tue Sep 10 18:55:50 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":1.2969999999999999,"15min":0.996},"min":{"1min":0.16200000000000001,"5min":0.16200000000000001,"15min":0.16200000000000001},"max":{"1min":1.425,"5min":58.215000000000003,"15min":58.215000000000003},"last":1.2030000000000001},{"interface":"front","average":{"1min":0.42699999999999999,"5min":1.3080000000000001,"15min":1.02},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":2.0649999999999999,"5min":57.896999999999998,"15min":57.896999999999998},"last":0.35399999999999998}]},{"osd":2,"last update":"Tue Sep 10 18:55:50 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.63,"15min":0.67800000000000005},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":2.133,"5min":3.9140000000000001,"15min":5.8220000000000001},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.378,"5min":0.52800000000000002,"15min":0.61299999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.95299999999999996,"5min":4.6289999999999996,"15min":7.0979999999999999},"last":0.502}]},{"osd":4,"last update":"Tue Sep 10 18:56:21 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":1.272,"15min":0.95899999999999996},"min":{"1min":0.31,"5min":0.247,"15min":0.247},"max":{"1min":2.7690000000000001,"5min":29.013000000000002,"15min":29.013000000000002},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.90400000000000003,"5min":1.22,"15min":0.95599999999999996},"min":{"1min":0.28799999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":2.8359999999999999,"5min":28.826000000000001,"15min":28.826000000000001},"last":0.69299999999999995}]},{"osd":5,"last update":"Tue Sep 10 18:55:41 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":1.145,"15min":0.90100000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":3.1829999999999998,"5min":6.1989999999999998,"15min":6.1989999999999998},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.1910000000000001,"15min":0.92600000000000005},"min":{"1min":0.36299999999999999,"5min":0.255,"15min":0.255},"max":{"1min":3.1949999999999998,"5min":6.1870000000000003,"15min":9.2509999999999994},"last":0.85199999999999998}]},{"osd":6,"last update":"Tue Sep 10 18:56:03 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":1.093,"15min":0.94999999999999996},"min":{"1min":0.33600000000000002,"5min":0.309,"15min":0.309},"max":{"1min":2.9089999999999998,"5min":5.0149999999999997,"15min":9.3239999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.95199999999999996,"5min":1.077,"15min":0.93200000000000005},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":3.355,"5min":6.4900000000000002,"15min":7.1559999999999997},"last":0.71999999999999997}]},{"osd":7,"last update":"Tue Sep 10 18:55:29 2024","interfaces":[{"interface":"back","average":{"1min":1.6479999999999999,"5min":1.206,"15min":0.91000000000000003},"min":{"1min":0.438,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":5.2569999999999997,"5min":6.0839999999999996,"15min":6.0839999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":1.423,"5min":1.2989999999999999,"15min":0.98199999999999998},"min":{"1min":0.49299999999999999,"5min":0.39300000000000002,"15min":0.33100000000000002},"max":{"1min":5.274,"5min":5.274,"15min":9.2949999999999999},"last":0.56200000000000006}]}]},{"osd":2,"up_from":0,"seq":77309411490,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834575014,"num_pgs":80,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3218176,"kb_used_data":2912504,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":90526976,"statfs":{"total":95995035648,"available":92699623424,"internally_reserved":0,"allocated":2982404096,"data_stored":2981733039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:52:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":1.278,"15min":0.64200000000000002},"min":{"1min":0.215,"5min":0.193,"15min":0.193},"max":{"1min":1.913,"5min":13.531000000000001,"15min":13.531000000000001},"last":2.863},{"interface":"front","average":{"1min":0.93600000000000005,"5min":1.8260000000000001,"15min":0.83999999999999997},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":4.3630000000000004,"5min":16.206,"15min":16.206},"last":0.25800000000000001}]},{"osd":2,"last update":"Tue Sep 10 18:55:28 2024","interfaces":[{"interface":"back","average":{"1min":0.28599999999999998,"5min":0.68200000000000005,"15min":0.54600000000000004},"min":{"1min":0.19600000000000001,"5min":0.186,"15min":0.186},"max":{"1min":0.51100000000000001,"5min":2.8159999999999998,"15min":3.8479999999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.63900000000000001,"15min":0.58599999999999997},"min":{"1min":0.23000000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.79500000000000004,"5min":3.754,"15min":7.093},"last":2.1389999999999998}]},{"osd":3,"last update":"Tue Sep 10 18:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.0269999999999999,"15min":0.95499999999999996},"min":{"1min":0.25,"5min":0.185,"15min":0.185},"max":{"1min":10.840999999999999,"5min":10.840999999999999,"15min":12.714},"last":2.2000000000000002},{"interface":"front","average":{"1min":1.2390000000000001,"5min":0.92300000000000004,"15min":0.98099999999999998},"min":{"1min":0.23699999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":13.226000000000001,"5min":13.226000000000001,"15min":13.226000000000001},"last":3.698}]},{"osd":4,"last update":"Tue Sep 10 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":1.2150000000000001,"15min":0.95399999999999996},"min":{"1min":0.373,"5min":0.29799999999999999,"15min":0.26200000000000001},"max":{"1min":4.4089999999999998,"5min":15.965999999999999,"15min":15.965999999999999},"last":2.9470000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":1.034,"15min":0.80900000000000005},"min":{"1min":0.28799999999999998,"5min":0.28100000000000003,"15min":0.246},"max":{"1min":2.544,"5min":3.1659999999999999,"15min":3.9540000000000002},"last":0.54700000000000004}]},{"osd":5,"last update":"Tue Sep 10 18:55:40 2024","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.4099999999999999,"15min":0.96499999999999997},"min":{"1min":0.28899999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":10.334,"5min":10.334,"15min":10.334},"last":2.2450000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":1.4099999999999999,"15min":1.036},"min":{"1min":0.29699999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":4.609,"5min":16.039999999999999,"15min":16.039999999999999},"last":2.8999999999999999}]},{"osd":6,"last update":"Tue Sep 10 18:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.8440000000000001,"15min":1.29},"min":{"1min":0.34699999999999998,"5min":0.308,"15min":0.308},"max":{"1min":10.282999999999999,"5min":33.290999999999997,"15min":38.039999999999999},"last":2.2229999999999999},{"interface":"front","average":{"1min":1.3779999999999999,"5min":2.016,"15min":1.4159999999999999},"min":{"1min":0.38200000000000001,"5min":0.33400000000000002,"15min":0.30599999999999999},"max":{"1min":10.404999999999999,"5min":33.310000000000002,"15min":38.100999999999999},"last":2.968}]},{"osd":7,"last update":"Tue Sep 10 18:55:28 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.226,"15min":0.93100000000000005},"min":{"1min":0.38500000000000001,"5min":0.315,"15min":0.315},"max":{"1min":4.5449999999999999,"5min":4.5449999999999999,"15min":4.6539999999999999},"last":2.266},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.0469999999999999,"15min":0.91100000000000003},"min":{"1min":0.36199999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":4.9029999999999996,"5min":4.9029999999999996,"15min":4.9029999999999996},"last":2.9249999999999998}]}]},{"osd":0,"up_from":241,"seq":1035087118346,"num_pgs":74,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3586608,"kb_used_data":3240360,"kb_used_omap":4,"kb_used_meta":346235,"kb_avail":90158544,"statfs":{"total":95995035648,"available":92322349056,"internally_reserved":0,"allocated":3318128640,"data_stored":3317473132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":354545158},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1443430400,"data_stored":1443430400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3023962112,"data_stored":3023962112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2730491904,"data_stored":2730491904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":897515520,"data_stored":897515520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1802829824,"data_stored":1802829824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1212481536,"data_stored":1212481536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3310223360,"data_stored":3310223360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T18:56:28.685 INFO:tasks.thrashosds.thrasher:Setting 2.11 to [6, 7, 5] 2024-09-10T18:56:28.688 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '2.11', '6', '7', '5'] 2024-09-10T18:56:28.689 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd pg-upmap 2.11 6 7 5 2024-09-10T18:56:28.751 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:28.780 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:28.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:28.782 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:28.882 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:28.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:28.934 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:28.934 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:28.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:28 smithi018 ceph-mon[40196]: 5.4 scrub starts 2024-09-10T18:56:28.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:28 smithi018 ceph-mon[40196]: 5.4 scrub ok 2024-09-10T18:56:28.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:28 smithi018 ceph-mon[40196]: 4.1e scrub starts 2024-09-10T18:56:28.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:28 smithi018 ceph-mon[40196]: 4.1e scrub ok 2024-09-10T18:56:28.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:28 smithi018 ceph-mon[40196]: from='client.25141 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:56:28.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:28 smithi018 ceph-mon[40196]: 4.2 scrub starts 2024-09-10T18:56:28.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:28 smithi018 ceph-mon[40196]: 4.2 scrub ok 2024-09-10T18:56:28.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:28 smithi018 ceph-mon[29336]: 5.4 scrub starts 2024-09-10T18:56:28.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:28 smithi018 ceph-mon[29336]: 5.4 scrub ok 2024-09-10T18:56:28.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:28 smithi018 ceph-mon[29336]: 4.1e scrub starts 2024-09-10T18:56:28.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:28 smithi018 ceph-mon[29336]: 4.1e scrub ok 2024-09-10T18:56:28.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:28 smithi018 ceph-mon[29336]: from='client.25141 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:56:28.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:28 smithi018 ceph-mon[29336]: 4.2 scrub starts 2024-09-10T18:56:28.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:28 smithi018 ceph-mon[29336]: 4.2 scrub ok 2024-09-10T18:56:29.035 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:29.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:29.086 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:29.086 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:29.187 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:29.226 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:29.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:29.227 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:29.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:28 smithi103 ceph-mon[33163]: 5.4 scrub starts 2024-09-10T18:56:29.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:28 smithi103 ceph-mon[33163]: 5.4 scrub ok 2024-09-10T18:56:29.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:28 smithi103 ceph-mon[33163]: 4.1e scrub starts 2024-09-10T18:56:29.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:28 smithi103 ceph-mon[33163]: 4.1e scrub ok 2024-09-10T18:56:29.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:28 smithi103 ceph-mon[33163]: from='client.25141 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:56:29.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:28 smithi103 ceph-mon[33163]: 4.2 scrub starts 2024-09-10T18:56:29.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:28 smithi103 ceph-mon[33163]: 4.2 scrub ok 2024-09-10T18:56:29.328 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:29.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:29.370 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:29.371 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:29.389 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 41 15 77585 77570 118.226 78.5625 0.0168796 0.00845352 2024-09-10T18:56:29.472 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:29.506 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:29.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:29.507 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:29.607 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:29.642 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:29.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:29.643 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:29.744 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:29.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:29.782 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:29.782 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:29.882 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:29.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:29.931 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:29.931 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:30.032 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:30.071 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:30.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:30.072 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:30.173 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:30.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:29 smithi018 ceph-mon[29336]: 5.d scrub starts 2024-09-10T18:56:30.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:29 smithi018 ceph-mon[29336]: 5.d scrub ok 2024-09-10T18:56:30.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:29 smithi018 ceph-mon[29336]: 1.7 scrub starts 2024-09-10T18:56:30.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:29 smithi018 ceph-mon[29336]: 1.7 scrub ok 2024-09-10T18:56:30.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:29 smithi018 ceph-mon[29336]: 4.1 scrub starts 2024-09-10T18:56:30.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:29 smithi018 ceph-mon[29336]: 4.1 scrub ok 2024-09-10T18:56:30.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:29 smithi018 ceph-mon[29336]: pgmap v508: 168 pgs: 168 active+clean; 4.5 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 852 B/s rd, 80 MiB/s wr, 1.28k op/s 2024-09-10T18:56:30.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:29 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:56:30.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:29 smithi018 ceph-mon[40196]: 5.d scrub starts 2024-09-10T18:56:30.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:29 smithi018 ceph-mon[40196]: 5.d scrub ok 2024-09-10T18:56:30.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:29 smithi018 ceph-mon[40196]: 1.7 scrub starts 2024-09-10T18:56:30.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:29 smithi018 ceph-mon[40196]: 1.7 scrub ok 2024-09-10T18:56:30.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:29 smithi018 ceph-mon[40196]: 4.1 scrub starts 2024-09-10T18:56:30.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:29 smithi018 ceph-mon[40196]: 4.1 scrub ok 2024-09-10T18:56:30.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:29 smithi018 ceph-mon[40196]: pgmap v508: 168 pgs: 168 active+clean; 4.5 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 852 B/s rd, 80 MiB/s wr, 1.28k op/s 2024-09-10T18:56:30.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:29 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:56:30.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:30.221 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:30.221 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:30.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:29 smithi103 ceph-mon[33163]: 5.d scrub starts 2024-09-10T18:56:30.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:29 smithi103 ceph-mon[33163]: 5.d scrub ok 2024-09-10T18:56:30.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:29 smithi103 ceph-mon[33163]: 1.7 scrub starts 2024-09-10T18:56:30.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:29 smithi103 ceph-mon[33163]: 1.7 scrub ok 2024-09-10T18:56:30.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:29 smithi103 ceph-mon[33163]: 4.1 scrub starts 2024-09-10T18:56:30.240 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:29 smithi103 ceph-mon[33163]: 4.1 scrub ok 2024-09-10T18:56:30.240 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:29 smithi103 ceph-mon[33163]: pgmap v508: 168 pgs: 168 active+clean; 4.5 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 852 B/s rd, 80 MiB/s wr, 1.28k op/s 2024-09-10T18:56:30.240 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:29 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:56:30.322 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:30.349 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:30.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:30.351 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:30.390 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 42 15 78946 78931 117.434 85.0625 0.018988 0.00851029 2024-09-10T18:56:30.452 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:30.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:30.491 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:30.491 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:30.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:56:30 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:56:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:56:30.592 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:30.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:30.631 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:30.631 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:30.732 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:30.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:30.763 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:30.763 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:30.864 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:30.909 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:30.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:30.910 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:31.011 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:31.044 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:31.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:31.045 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:31.146 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:31.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:31.184 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:31.184 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:31.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:56:30 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:56:30] "GET /metrics HTTP/1.1" 200 224230 "" "Prometheus/2.43.0" 2024-09-10T18:56:31.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:30 smithi018 ceph-mon[29336]: 2.7 scrub starts 2024-09-10T18:56:31.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:30 smithi018 ceph-mon[29336]: 2.7 scrub ok 2024-09-10T18:56:31.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:30 smithi018 ceph-mon[29336]: 2.a scrub starts 2024-09-10T18:56:31.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:30 smithi018 ceph-mon[29336]: 2.a scrub ok 2024-09-10T18:56:31.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:30 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:31.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:30 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:31.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:30 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:31.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:30 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:31.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:30 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:31.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:30 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:31.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:30 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:31.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:30 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:31.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:30 smithi018 ceph-mon[29336]: Adjusting osd_memory_target on smithi103 to 3248M 2024-09-10T18:56:31.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:30 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:56:31.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:30 smithi018 ceph-mon[40196]: 2.7 scrub starts 2024-09-10T18:56:31.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:30 smithi018 ceph-mon[40196]: 2.7 scrub ok 2024-09-10T18:56:31.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:30 smithi018 ceph-mon[40196]: 2.a scrub starts 2024-09-10T18:56:31.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:30 smithi018 ceph-mon[40196]: 2.a scrub ok 2024-09-10T18:56:31.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:30 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:31.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:30 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:31.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:30 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:31.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:30 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:31.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:30 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:31.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:30 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:31.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:30 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:31.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:30 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:31.197 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:30 smithi018 ceph-mon[40196]: Adjusting osd_memory_target on smithi103 to 3248M 2024-09-10T18:56:31.197 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:30 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:56:31.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:30 smithi103 ceph-mon[33163]: 2.7 scrub starts 2024-09-10T18:56:31.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:30 smithi103 ceph-mon[33163]: 2.7 scrub ok 2024-09-10T18:56:31.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:30 smithi103 ceph-mon[33163]: 2.a scrub starts 2024-09-10T18:56:31.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:30 smithi103 ceph-mon[33163]: 2.a scrub ok 2024-09-10T18:56:31.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:30 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:31.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:30 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:31.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:30 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:31.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:30 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:31.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:30 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:31.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:30 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:31.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:30 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:31.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:30 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:31.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:30 smithi103 ceph-mon[33163]: Adjusting osd_memory_target on smithi103 to 3248M 2024-09-10T18:56:31.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:30 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:56:31.285 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:31.319 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:31.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:31.320 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:31.390 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 43 16 80265 80249 116.618 82.375 0.0083859 0.00857037 2024-09-10T18:56:31.421 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:31.459 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:31.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:31.461 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:31.561 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:31.598 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:31.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:31.599 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:31.700 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:31.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:31.742 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:31.742 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:31.843 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:31.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:31.875 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:31.876 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:31.881 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:56:31.976 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:32.024 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:32.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:32.025 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:32.126 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:32.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:32.158 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:32.158 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:32.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:31 smithi018 ceph-mon[29336]: 2.8 scrub starts 2024-09-10T18:56:32.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:31 smithi018 ceph-mon[29336]: 2.8 scrub ok 2024-09-10T18:56:32.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:31 smithi018 ceph-mon[29336]: 4.6 scrub starts 2024-09-10T18:56:32.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:31 smithi018 ceph-mon[29336]: 4.6 scrub ok 2024-09-10T18:56:32.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:31 smithi018 ceph-mon[29336]: 5.7 scrub starts 2024-09-10T18:56:32.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:31 smithi018 ceph-mon[29336]: 4.10 scrub starts 2024-09-10T18:56:32.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:31 smithi018 ceph-mon[29336]: 5.7 scrub ok 2024-09-10T18:56:32.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:31 smithi018 ceph-mon[29336]: 4.10 scrub ok 2024-09-10T18:56:32.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:31 smithi018 ceph-mon[29336]: pgmap v509: 168 pgs: 168 active+clean; 4.5 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 852 B/s rd, 72 MiB/s wr, 1.15k op/s 2024-09-10T18:56:32.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:31 smithi018 ceph-mon[40196]: 2.8 scrub starts 2024-09-10T18:56:32.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:31 smithi018 ceph-mon[40196]: 2.8 scrub ok 2024-09-10T18:56:32.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:31 smithi018 ceph-mon[40196]: 4.6 scrub starts 2024-09-10T18:56:32.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:31 smithi018 ceph-mon[40196]: 4.6 scrub ok 2024-09-10T18:56:32.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:31 smithi018 ceph-mon[40196]: 5.7 scrub starts 2024-09-10T18:56:32.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:31 smithi018 ceph-mon[40196]: 4.10 scrub starts 2024-09-10T18:56:32.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:31 smithi018 ceph-mon[40196]: 5.7 scrub ok 2024-09-10T18:56:32.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:31 smithi018 ceph-mon[40196]: 4.10 scrub ok 2024-09-10T18:56:32.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:31 smithi018 ceph-mon[40196]: pgmap v509: 168 pgs: 168 active+clean; 4.5 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 852 B/s rd, 72 MiB/s wr, 1.15k op/s 2024-09-10T18:56:32.259 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:32.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:31 smithi103 ceph-mon[33163]: 2.8 scrub starts 2024-09-10T18:56:32.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:31 smithi103 ceph-mon[33163]: 2.8 scrub ok 2024-09-10T18:56:32.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:31 smithi103 ceph-mon[33163]: 4.6 scrub starts 2024-09-10T18:56:32.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:31 smithi103 ceph-mon[33163]: 4.6 scrub ok 2024-09-10T18:56:32.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:31 smithi103 ceph-mon[33163]: 5.7 scrub starts 2024-09-10T18:56:32.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:31 smithi103 ceph-mon[33163]: 4.10 scrub starts 2024-09-10T18:56:32.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:31 smithi103 ceph-mon[33163]: 5.7 scrub ok 2024-09-10T18:56:32.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:31 smithi103 ceph-mon[33163]: 4.10 scrub ok 2024-09-10T18:56:32.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:31 smithi103 ceph-mon[33163]: pgmap v509: 168 pgs: 168 active+clean; 4.5 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 852 B/s rd, 72 MiB/s wr, 1.15k op/s 2024-09-10T18:56:32.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:32.297 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:32.297 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:32.390 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 44 16 81757 81741 116.087 93.25 0.0108759 0.00860934 2024-09-10T18:56:32.398 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:32.434 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:32.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:32.435 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:32.535 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:32.569 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:32.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:32.571 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:32.671 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:32.703 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:32.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:32.704 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:32.804 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:32.852 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:32.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:32.853 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:32.954 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:32.955 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:32 smithi018 ceph-mon[40196]: 2.1f deep-scrub starts 2024-09-10T18:56:32.955 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:32 smithi018 ceph-mon[40196]: 2.1f deep-scrub ok 2024-09-10T18:56:32.955 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:32 smithi018 ceph-mon[40196]: 1.5 deep-scrub starts 2024-09-10T18:56:32.956 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:32 smithi018 ceph-mon[40196]: 1.5 deep-scrub ok 2024-09-10T18:56:32.956 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:32 smithi018 ceph-mon[40196]: 5.12 scrub starts 2024-09-10T18:56:32.956 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:32 smithi018 ceph-mon[40196]: 5.12 scrub ok 2024-09-10T18:56:32.956 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:32 smithi018 ceph-mon[29336]: 2.1f deep-scrub starts 2024-09-10T18:56:32.956 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:32 smithi018 ceph-mon[29336]: 2.1f deep-scrub ok 2024-09-10T18:56:32.956 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:32 smithi018 ceph-mon[29336]: 1.5 deep-scrub starts 2024-09-10T18:56:32.956 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:32 smithi018 ceph-mon[29336]: 1.5 deep-scrub ok 2024-09-10T18:56:32.956 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:32 smithi018 ceph-mon[29336]: 5.12 scrub starts 2024-09-10T18:56:32.956 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:32 smithi018 ceph-mon[29336]: 5.12 scrub ok 2024-09-10T18:56:32.982 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:32.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:32.984 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:33.085 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:33.129 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:33.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:33.130 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:33.231 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:33.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:32 smithi103 ceph-mon[33163]: 2.1f deep-scrub starts 2024-09-10T18:56:33.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:32 smithi103 ceph-mon[33163]: 2.1f deep-scrub ok 2024-09-10T18:56:33.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:32 smithi103 ceph-mon[33163]: 1.5 deep-scrub starts 2024-09-10T18:56:33.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:32 smithi103 ceph-mon[33163]: 1.5 deep-scrub ok 2024-09-10T18:56:33.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:32 smithi103 ceph-mon[33163]: 5.12 scrub starts 2024-09-10T18:56:33.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:32 smithi103 ceph-mon[33163]: 5.12 scrub ok 2024-09-10T18:56:33.274 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:33.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:33.275 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:33.376 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:33.390 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 45 16 83129 83113 115.413 85.75 0.0109434 0.00865961 2024-09-10T18:56:33.409 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:33.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:33.410 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:33.511 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:33.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:33.560 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:33.561 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:33.662 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:33.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:33.699 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:33.699 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:33.800 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:33.837 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:33.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:33.838 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:33.939 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:33.969 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:33.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:33.970 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:34.071 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:34.098 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:34.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:34.099 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:34.175 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[29336]: 6.9 scrub starts 2024-09-10T18:56:34.175 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[29336]: 6.9 scrub ok 2024-09-10T18:56:34.176 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[29336]: 4.c scrub starts 2024-09-10T18:56:34.176 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[29336]: 4.c scrub ok 2024-09-10T18:56:34.176 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[29336]: 5.16 scrub starts 2024-09-10T18:56:34.176 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[29336]: 5.16 scrub ok 2024-09-10T18:56:34.176 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:34.176 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:34.176 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:34.176 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:34.176 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:34.176 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:34.177 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:34.177 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:34.177 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[29336]: Adjusting osd_memory_target on smithi018 to 3249M 2024-09-10T18:56:34.177 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:56:34.177 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:56:34.177 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:56:34.177 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:56:34.177 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[29336]: pgmap v510: 168 pgs: 168 active+clean; 4.9 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.2 KiB/s rd, 100 MiB/s wr, 1.59k op/s 2024-09-10T18:56:34.177 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[29336]: 1.c deep-scrub starts 2024-09-10T18:56:34.177 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[29336]: 1.c deep-scrub ok 2024-09-10T18:56:34.178 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[40196]: 6.9 scrub starts 2024-09-10T18:56:34.178 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[40196]: 6.9 scrub ok 2024-09-10T18:56:34.178 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[40196]: 4.c scrub starts 2024-09-10T18:56:34.178 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[40196]: 4.c scrub ok 2024-09-10T18:56:34.178 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[40196]: 5.16 scrub starts 2024-09-10T18:56:34.178 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[40196]: 5.16 scrub ok 2024-09-10T18:56:34.178 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:34.178 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:34.178 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:34.178 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:34.178 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:34.179 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:34.179 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:34.179 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:34.179 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[40196]: Adjusting osd_memory_target on smithi018 to 3249M 2024-09-10T18:56:34.179 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:56:34.179 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:56:34.179 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:56:34.179 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:56:34.179 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[40196]: pgmap v510: 168 pgs: 168 active+clean; 4.9 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.2 KiB/s rd, 100 MiB/s wr, 1.59k op/s 2024-09-10T18:56:34.179 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[40196]: 1.c deep-scrub starts 2024-09-10T18:56:34.179 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:33 smithi018 ceph-mon[40196]: 1.c deep-scrub ok 2024-09-10T18:56:34.190 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:56:34.190 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:56:34.199 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:34.210 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:56:34.210 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:56:34.225 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:34.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:34.227 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:34.243 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:56:34.243 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:56:34.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:33 smithi103 ceph-mon[33163]: 6.9 scrub starts 2024-09-10T18:56:34.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:33 smithi103 ceph-mon[33163]: 6.9 scrub ok 2024-09-10T18:56:34.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:33 smithi103 ceph-mon[33163]: 4.c scrub starts 2024-09-10T18:56:34.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:33 smithi103 ceph-mon[33163]: 4.c scrub ok 2024-09-10T18:56:34.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:33 smithi103 ceph-mon[33163]: 5.16 scrub starts 2024-09-10T18:56:34.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:33 smithi103 ceph-mon[33163]: 5.16 scrub ok 2024-09-10T18:56:34.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:33 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:34.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:33 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:34.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:33 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:34.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:33 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:34.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:33 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:34.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:33 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:34.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:33 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:34.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:33 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:56:34.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:33 smithi103 ceph-mon[33163]: Adjusting osd_memory_target on smithi018 to 3249M 2024-09-10T18:56:34.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:33 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:56:34.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:33 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:56:34.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:33 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:56:34.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:33 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:56:34.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:33 smithi103 ceph-mon[33163]: pgmap v510: 168 pgs: 168 active+clean; 4.9 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.2 KiB/s rd, 100 MiB/s wr, 1.59k op/s 2024-09-10T18:56:34.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:33 smithi103 ceph-mon[33163]: 1.c deep-scrub starts 2024-09-10T18:56:34.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:33 smithi103 ceph-mon[33163]: 1.c deep-scrub ok 2024-09-10T18:56:34.281 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:56:34.288 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:56:34.288 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:56:34.327 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:34.337 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:56:34.337 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:56:34.356 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:34.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:34.357 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:34.374 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:56:34.374 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:56:34.391 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 46 16 84445 84429 114.692 82.25 0.00420176 0.00871374 2024-09-10T18:56:34.410 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:56:34.410 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:56:34.455 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:56:34.455 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:56:34.458 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:34.504 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:34.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:34.506 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:34.608 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:34.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:34.646 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:34.646 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:34.747 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:34.782 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:34.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:34.783 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:34.884 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:34.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:34.917 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:34.917 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:35.018 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:35.019 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is set 2024-09-10T18:56:35.045 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:35.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:35.046 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:35.121 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:34 smithi018 ceph-mon[40196]: 5.1e scrub starts 2024-09-10T18:56:35.121 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:34 smithi018 ceph-mon[40196]: 5.1e scrub ok 2024-09-10T18:56:35.121 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:34 smithi018 ceph-mon[40196]: 2.3 scrub starts 2024-09-10T18:56:35.121 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:34 smithi018 ceph-mon[40196]: 2.3 scrub ok 2024-09-10T18:56:35.121 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:34 smithi018 ceph-mon[40196]: 1.b scrub starts 2024-09-10T18:56:35.121 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:34 smithi018 ceph-mon[40196]: 1.b scrub ok 2024-09-10T18:56:35.121 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:34 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1182609182' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:56:35.121 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:34 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1655282956' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:56:35.121 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:34 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:56:35.121 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:34 smithi018 ceph-mon[29336]: 5.1e scrub starts 2024-09-10T18:56:35.122 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:34 smithi018 ceph-mon[29336]: 5.1e scrub ok 2024-09-10T18:56:35.122 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:34 smithi018 ceph-mon[29336]: 2.3 scrub starts 2024-09-10T18:56:35.122 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:34 smithi018 ceph-mon[29336]: 2.3 scrub ok 2024-09-10T18:56:35.122 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:34 smithi018 ceph-mon[29336]: 1.b scrub starts 2024-09-10T18:56:35.122 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:34 smithi018 ceph-mon[29336]: 1.b scrub ok 2024-09-10T18:56:35.122 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:34 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1182609182' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:56:35.122 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:34 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1655282956' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:56:35.122 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:34 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:56:35.147 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:35.179 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:35.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:35.180 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:35.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:34 smithi103 ceph-mon[33163]: 5.1e scrub starts 2024-09-10T18:56:35.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:34 smithi103 ceph-mon[33163]: 5.1e scrub ok 2024-09-10T18:56:35.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:34 smithi103 ceph-mon[33163]: 2.3 scrub starts 2024-09-10T18:56:35.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:34 smithi103 ceph-mon[33163]: 2.3 scrub ok 2024-09-10T18:56:35.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:34 smithi103 ceph-mon[33163]: 1.b scrub starts 2024-09-10T18:56:35.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:34 smithi103 ceph-mon[33163]: 1.b scrub ok 2024-09-10T18:56:35.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:34 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1182609182' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:56:35.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:34 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1655282956' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:56:35.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:34 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:56:35.281 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:35.316 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:35.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:35.317 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:35.391 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 47 16 85954 85938 114.258 94.3125 0.00593372 0.00874792 2024-09-10T18:56:35.417 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:35.447 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:35.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:35.448 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:35.549 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:35.593 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:35.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:35.594 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:35.695 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:35.732 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:35.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:35.733 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:35.834 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:35.879 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:35.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:35.880 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:35.981 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:36.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:36.021 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:36.021 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:36.122 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:36.146 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:35 smithi018 ceph-mon[29336]: 4.a scrub starts 2024-09-10T18:56:36.146 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:35 smithi018 ceph-mon[29336]: 4.a scrub ok 2024-09-10T18:56:36.146 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:35 smithi018 ceph-mon[29336]: 6.7 scrub starts 2024-09-10T18:56:36.146 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:35 smithi018 ceph-mon[29336]: 6.7 scrub ok 2024-09-10T18:56:36.146 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:35 smithi018 ceph-mon[29336]: 1.13 scrub starts 2024-09-10T18:56:36.146 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:35 smithi018 ceph-mon[29336]: 1.13 scrub ok 2024-09-10T18:56:36.146 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:35 smithi018 ceph-mon[29336]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:56:36.147 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:35 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:56:36.147 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:35 smithi018 ceph-mon[29336]: osdmap e259: 8 total, 8 up, 7 in 2024-09-10T18:56:36.147 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:35 smithi018 ceph-mon[29336]: pgmap v512: 168 pgs: 168 active+clean; 4.9 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1023 B/s rd, 84 MiB/s wr, 1.34k op/s 2024-09-10T18:56:36.147 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:35 smithi018 ceph-mon[40196]: 4.a scrub starts 2024-09-10T18:56:36.147 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:35 smithi018 ceph-mon[40196]: 4.a scrub ok 2024-09-10T18:56:36.147 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:35 smithi018 ceph-mon[40196]: 6.7 scrub starts 2024-09-10T18:56:36.147 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:35 smithi018 ceph-mon[40196]: 6.7 scrub ok 2024-09-10T18:56:36.147 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:35 smithi018 ceph-mon[40196]: 1.13 scrub starts 2024-09-10T18:56:36.147 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:35 smithi018 ceph-mon[40196]: 1.13 scrub ok 2024-09-10T18:56:36.147 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:35 smithi018 ceph-mon[40196]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:56:36.148 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:35 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:56:36.148 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:35 smithi018 ceph-mon[40196]: osdmap e259: 8 total, 8 up, 7 in 2024-09-10T18:56:36.148 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:35 smithi018 ceph-mon[40196]: pgmap v512: 168 pgs: 168 active+clean; 4.9 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1023 B/s rd, 84 MiB/s wr, 1.34k op/s 2024-09-10T18:56:36.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:36.153 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:36.153 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:36.254 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:36.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:35 smithi103 ceph-mon[33163]: 4.a scrub starts 2024-09-10T18:56:36.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:35 smithi103 ceph-mon[33163]: 4.a scrub ok 2024-09-10T18:56:36.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:35 smithi103 ceph-mon[33163]: 6.7 scrub starts 2024-09-10T18:56:36.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:35 smithi103 ceph-mon[33163]: 6.7 scrub ok 2024-09-10T18:56:36.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:35 smithi103 ceph-mon[33163]: 1.13 scrub starts 2024-09-10T18:56:36.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:35 smithi103 ceph-mon[33163]: 1.13 scrub ok 2024-09-10T18:56:36.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:35 smithi103 ceph-mon[33163]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:56:36.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:35 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:56:36.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:35 smithi103 ceph-mon[33163]: osdmap e259: 8 total, 8 up, 7 in 2024-09-10T18:56:36.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:35 smithi103 ceph-mon[33163]: pgmap v512: 168 pgs: 168 active+clean; 4.9 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1023 B/s rd, 84 MiB/s wr, 1.34k op/s 2024-09-10T18:56:36.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:36.293 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:36.293 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:36.391 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 48 16 87488 87472 113.875 95.875 0.0089155 0.00877681 2024-09-10T18:56:36.393 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:36.431 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:36.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:36.432 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:36.533 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:36.564 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:36.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:36.565 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:36.666 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:36.697 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:36.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:36.698 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:36.798 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:36.829 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:36.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:36.830 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:36.931 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:36.969 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:36.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:36.971 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:37.022 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:56:36 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:56:37.072 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:37.110 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:37.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:37.111 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:37.212 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:37.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:37.243 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:37.243 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:37.344 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:37.381 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:37.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:37.382 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:37.391 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 49 16 89024 89008 113.51 96 0.0101897 0.00880556 2024-09-10T18:56:37.484 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:37.527 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:37.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:37.528 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:37.629 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:37.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:37.671 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:37.671 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:37.772 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:37.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:37.818 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:37.818 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:37.864 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T18:56:37.919 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:37.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:37.962 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:37.962 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:38.063 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:38.091 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:38.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:38.092 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:38.193 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:38.226 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:38.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:38.227 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:38.328 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:38.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:38.360 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:38.360 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:38.391 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 50 16 90510 90494 113.097 92.875 0.00928526 0.00883745 2024-09-10T18:56:38.461 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:38.494 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:38.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:38.495 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:38.558 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:38 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:38.558 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:38 smithi018 ceph-mon[29336]: pgmap v513: 168 pgs: 168 active+clean; 5.3 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1023 B/s rd, 86 MiB/s wr, 1.37k op/s 2024-09-10T18:56:38.559 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:38 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:38.559 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:38 smithi018 ceph-mon[40196]: pgmap v513: 168 pgs: 168 active+clean; 5.3 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1023 B/s rd, 86 MiB/s wr, 1.37k op/s 2024-09-10T18:56:38.596 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:38.630 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:38.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:38.631 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:38.732 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:38.769 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:38.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:38.770 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:38.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:38 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:38.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:38 smithi103 ceph-mon[33163]: pgmap v513: 168 pgs: 168 active+clean; 5.3 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1023 B/s rd, 86 MiB/s wr, 1.37k op/s 2024-09-10T18:56:38.870 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:38.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:38.915 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:38.915 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:39.016 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:39.048 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:39.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:39.049 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:39.150 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:39.195 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:39.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:39.196 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:39.296 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:39.341 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:39.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:39.343 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:39.391 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 51 16 91827 91811 112.493 82.3125 0.0229093 0.00888431 2024-09-10T18:56:39.444 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:39.484 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:39.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:39.486 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:39.522 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T18:56:39.522 INFO:teuthology.orchestra.run.smithi018.stdout: "size": 20, 2024-09-10T18:56:39.522 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 600, 2024-09-10T18:56:39.522 INFO:teuthology.orchestra.run.smithi018.stdout: "ops": [ 2024-09-10T18:56:39.522 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.522 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15354.0:33170 9.b 9:dcae3347:::benchmark_data_smithi018_133594_object33169:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-10T18:56:39.522 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:56:01.248575+0000", 2024-09-10T18:56:39.523 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 38.269357835000001, 2024-09-10T18:56:39.523 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.074615603000000003, 2024-09-10T18:56:39.523 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:56:39.523 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:56:39.523 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:56:39.523 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15354", 2024-09-10T18:56:39.523 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T18:56:39.523 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 33170 2024-09-10T18:56:39.523 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.523 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:56:39.523 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.523 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:56:39.523 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.248575+0000", 2024-09-10T18:56:39.524 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.524 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.524 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.524 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:56:39.524 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.248575+0000", 2024-09-10T18:56:39.524 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.524 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.524 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.524 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:56:39.524 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.248574+0000", 2024-09-10T18:56:39.524 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T18:56:39.524 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.524 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.525 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:56:39.525 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.248598+0000", 2024-09-10T18:56:39.525 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.4085000000000001e-05 2024-09-10T18:56:39.525 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.525 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.525 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:56:39.525 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.248663+0000", 2024-09-10T18:56:39.525 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.5271000000000006e-05 2024-09-10T18:56:39.525 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.525 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.525 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:56:39.525 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.248669+0000", 2024-09-10T18:56:39.525 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.5790000000000003e-06 2024-09-10T18:56:39.526 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.526 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.526 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:56:39.526 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.250323+0000", 2024-09-10T18:56:39.526 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.001653772 2024-09-10T18:56:39.526 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.526 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.526 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:56:39.526 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.250437+0000", 2024-09-10T18:56:39.526 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00011397500000000001 2024-09-10T18:56:39.526 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.526 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.526 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 5,6", 2024-09-10T18:56:39.527 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.250561+0000", 2024-09-10T18:56:39.527 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000124151 2024-09-10T18:56:39.527 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.527 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.527 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:56:39.527 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.252977+0000", 2024-09-10T18:56:39.527 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0024163129999999998 2024-09-10T18:56:39.527 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.527 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.527 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:56:39.527 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.254196+0000", 2024-09-10T18:56:39.527 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.001218815 2024-09-10T18:56:39.527 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.528 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.528 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:56:39.529 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.323116+0000", 2024-09-10T18:56:39.529 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.068920115000000004 2024-09-10T18:56:39.529 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.529 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.529 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:56:39.529 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.323152+0000", 2024-09-10T18:56:39.529 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.5627000000000003e-05 2024-09-10T18:56:39.529 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.529 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.529 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:56:39.529 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.323191+0000", 2024-09-10T18:56:39.529 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.9263e-05 2024-09-10T18:56:39.529 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.530 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:39.530 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.530 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.530 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.530 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15354.0:33172 9.b 9:d407521b:::benchmark_data_smithi018_133594_object33171:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-10T18:56:39.530 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:56:01.249104+0000", 2024-09-10T18:56:39.530 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 38.268829171, 2024-09-10T18:56:39.530 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.074179585000000006, 2024-09-10T18:56:39.530 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:56:39.530 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:56:39.530 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:56:39.530 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15354", 2024-09-10T18:56:39.530 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T18:56:39.530 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 33172 2024-09-10T18:56:39.531 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.531 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:56:39.531 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.531 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:56:39.531 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.249104+0000", 2024-09-10T18:56:39.531 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.531 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.531 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.531 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:56:39.531 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.249104+0000", 2024-09-10T18:56:39.531 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.531 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.531 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.531 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:56:39.532 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.249103+0000", 2024-09-10T18:56:39.532 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:56:39.532 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.532 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.532 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:56:39.532 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.249122+0000", 2024-09-10T18:56:39.532 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.9136000000000001e-05 2024-09-10T18:56:39.532 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.532 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.532 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:56:39.532 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.249133+0000", 2024-09-10T18:56:39.532 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.063e-05 2024-09-10T18:56:39.532 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.533 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.533 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:56:39.533 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.249139+0000", 2024-09-10T18:56:39.533 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.9150000000000001e-06 2024-09-10T18:56:39.533 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.533 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.533 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:56:39.533 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.251707+0000", 2024-09-10T18:56:39.533 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0025686440000000001 2024-09-10T18:56:39.533 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.533 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.533 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:56:39.533 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.251806+0000", 2024-09-10T18:56:39.533 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.8443999999999995e-05 2024-09-10T18:56:39.534 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.534 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.534 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 5,6", 2024-09-10T18:56:39.534 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.251925+0000", 2024-09-10T18:56:39.534 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000119144 2024-09-10T18:56:39.534 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.534 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.534 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:56:39.534 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.253141+0000", 2024-09-10T18:56:39.534 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0012160350000000001 2024-09-10T18:56:39.534 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.534 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.534 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:56:39.535 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.254258+0000", 2024-09-10T18:56:39.535 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.001116508 2024-09-10T18:56:39.535 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.535 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.535 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:56:39.535 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.323221+0000", 2024-09-10T18:56:39.535 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.068963362 2024-09-10T18:56:39.535 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.535 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.535 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:56:39.535 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.323254+0000", 2024-09-10T18:56:39.535 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.3047999999999999e-05 2024-09-10T18:56:39.535 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.535 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.536 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:56:39.536 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.323284+0000", 2024-09-10T18:56:39.536 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.9612e-05 2024-09-10T18:56:39.536 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.536 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:39.536 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.536 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.536 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.536 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15354.0:33181 9.9 9:993c318a:::benchmark_data_smithi018_133594_object33180:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-10T18:56:39.536 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:56:01.254494+0000", 2024-09-10T18:56:39.536 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 38.263438973, 2024-09-10T18:56:39.536 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.070119944000000003, 2024-09-10T18:56:39.536 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:56:39.536 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:56:39.537 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:56:39.537 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15354", 2024-09-10T18:56:39.537 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T18:56:39.537 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 33181 2024-09-10T18:56:39.538 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.538 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:56:39.538 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.538 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:56:39.538 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.254494+0000", 2024-09-10T18:56:39.538 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.538 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.538 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.538 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:56:39.538 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.254494+0000", 2024-09-10T18:56:39.538 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.538 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.538 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.539 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:56:39.539 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.254493+0000", 2024-09-10T18:56:39.539 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T18:56:39.539 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.539 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.539 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:56:39.539 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.254517+0000", 2024-09-10T18:56:39.539 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.4219e-05 2024-09-10T18:56:39.539 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.539 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.539 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:56:39.539 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.254527+0000", 2024-09-10T18:56:39.539 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.4490000000000005e-06 2024-09-10T18:56:39.540 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.540 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.540 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:56:39.540 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.254532+0000", 2024-09-10T18:56:39.540 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.1000000000000003e-06 2024-09-10T18:56:39.540 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.540 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.540 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:56:39.540 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.254586+0000", 2024-09-10T18:56:39.540 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.4360999999999997e-05 2024-09-10T18:56:39.540 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.540 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.540 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:56:39.541 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.254689+0000", 2024-09-10T18:56:39.541 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000102434 2024-09-10T18:56:39.541 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.541 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.541 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 5,7", 2024-09-10T18:56:39.541 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.254819+0000", 2024-09-10T18:56:39.541 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000130164 2024-09-10T18:56:39.541 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.541 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.541 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:56:39.541 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.323856+0000", 2024-09-10T18:56:39.541 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.069037507999999997 2024-09-10T18:56:39.542 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.542 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.542 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:56:39.542 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.323935+0000", 2024-09-10T18:56:39.542 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.9301999999999994e-05 2024-09-10T18:56:39.542 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.542 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.542 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:56:39.542 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.324521+0000", 2024-09-10T18:56:39.542 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00058586300000000005 2024-09-10T18:56:39.542 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.542 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.542 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:56:39.543 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.324576+0000", 2024-09-10T18:56:39.543 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.4503000000000003e-05 2024-09-10T18:56:39.543 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.543 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.543 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:56:39.543 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.324614+0000", 2024-09-10T18:56:39.543 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.8328000000000003e-05 2024-09-10T18:56:39.543 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.543 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:39.543 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.543 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.543 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.543 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15354.0:33184 9.b 9:d079fc62:::benchmark_data_smithi018_133594_object33183:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-10T18:56:39.544 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:56:01.254626+0000", 2024-09-10T18:56:39.544 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 38.263307443000002, 2024-09-10T18:56:39.544 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.070632997000000003, 2024-09-10T18:56:39.544 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:56:39.544 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:56:39.544 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:56:39.544 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15354", 2024-09-10T18:56:39.544 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T18:56:39.544 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 33184 2024-09-10T18:56:39.544 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.544 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:56:39.544 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.544 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:56:39.545 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.254626+0000", 2024-09-10T18:56:39.545 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.545 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.545 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.545 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:56:39.545 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.254626+0000", 2024-09-10T18:56:39.545 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.545 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.545 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.545 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:56:39.545 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.254625+0000", 2024-09-10T18:56:39.545 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T18:56:39.545 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.546 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.546 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:56:39.546 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.254644+0000", 2024-09-10T18:56:39.546 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.8658e-05 2024-09-10T18:56:39.546 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.546 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.546 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:56:39.546 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.254656+0000", 2024-09-10T18:56:39.546 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.2520999999999999e-05 2024-09-10T18:56:39.546 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.546 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.546 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:56:39.546 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.254660+0000", 2024-09-10T18:56:39.547 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.4120000000000001e-06 2024-09-10T18:56:39.547 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.547 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.547 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:56:39.547 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.254698+0000", 2024-09-10T18:56:39.547 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.7914999999999999e-05 2024-09-10T18:56:39.547 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.547 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.547 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:56:39.547 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.254771+0000", 2024-09-10T18:56:39.547 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.3243000000000003e-05 2024-09-10T18:56:39.547 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.547 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.548 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 5,6", 2024-09-10T18:56:39.548 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.321901+0000", 2024-09-10T18:56:39.548 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.067129935000000002 2024-09-10T18:56:39.548 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.548 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.548 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:56:39.548 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.325094+0000", 2024-09-10T18:56:39.548 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0031935359999999999 2024-09-10T18:56:39.548 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.548 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.548 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:56:39.548 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.325102+0000", 2024-09-10T18:56:39.548 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.8590000000000005e-06 2024-09-10T18:56:39.549 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.549 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.549 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:56:39.549 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.325143+0000", 2024-09-10T18:56:39.549 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.1112000000000002e-05 2024-09-10T18:56:39.549 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.549 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.549 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:56:39.549 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.325222+0000", 2024-09-10T18:56:39.549 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.8567999999999997e-05 2024-09-10T18:56:39.549 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.550 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.550 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:56:39.550 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.325259+0000", 2024-09-10T18:56:39.550 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.6947999999999999e-05 2024-09-10T18:56:39.550 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.550 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:39.550 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.550 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.550 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.550 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15354.0:33185 9.5 9:a57b67f3:::benchmark_data_smithi018_133594_object33184:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-10T18:56:39.550 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:56:01.254842+0000", 2024-09-10T18:56:39.550 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 38.263090834000003, 2024-09-10T18:56:39.550 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.073261001000000006, 2024-09-10T18:56:39.551 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:56:39.551 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:56:39.551 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:56:39.551 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15354", 2024-09-10T18:56:39.551 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T18:56:39.551 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 33185 2024-09-10T18:56:39.551 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.551 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:56:39.551 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.551 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:56:39.551 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.254842+0000", 2024-09-10T18:56:39.551 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.551 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.551 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.552 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:56:39.552 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.254842+0000", 2024-09-10T18:56:39.552 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.552 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.552 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.552 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:56:39.552 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.254842+0000", 2024-09-10T18:56:39.552 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:56:39.552 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.552 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.552 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:56:39.552 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.254864+0000", 2024-09-10T18:56:39.553 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.2799999999999999e-05 2024-09-10T18:56:39.553 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.553 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.553 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:56:39.553 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.321626+0000", 2024-09-10T18:56:39.553 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.066761305000000007 2024-09-10T18:56:39.553 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.553 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.553 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:56:39.553 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.321639+0000", 2024-09-10T18:56:39.554 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.2941e-05 2024-09-10T18:56:39.554 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.554 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.554 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:56:39.554 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.321687+0000", 2024-09-10T18:56:39.554 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.8730999999999998e-05 2024-09-10T18:56:39.554 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.554 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.554 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:56:39.554 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.321836+0000", 2024-09-10T18:56:39.554 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00014818700000000001 2024-09-10T18:56:39.554 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.555 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.555 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,7", 2024-09-10T18:56:39.555 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.322000+0000", 2024-09-10T18:56:39.555 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00016414399999999999 2024-09-10T18:56:39.555 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.555 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.555 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:56:39.555 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.327983+0000", 2024-09-10T18:56:39.555 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00598323 2024-09-10T18:56:39.555 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.555 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.555 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:56:39.555 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.328020+0000", 2024-09-10T18:56:39.556 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.6888999999999999e-05 2024-09-10T18:56:39.556 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.556 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.556 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:56:39.556 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.328025+0000", 2024-09-10T18:56:39.556 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.5249999999999996e-06 2024-09-10T18:56:39.556 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.556 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.556 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:56:39.556 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.328069+0000", 2024-09-10T18:56:39.556 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.3383999999999997e-05 2024-09-10T18:56:39.556 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.556 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.557 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:56:39.557 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.328103+0000", 2024-09-10T18:56:39.557 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.4699000000000001e-05 2024-09-10T18:56:39.557 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.557 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:39.557 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.557 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.557 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.557 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.15354.0:33176 9.0 e254/241 9:025ffe61:::benchmark_data_smithi018_133594_object33175:head v 254'2055, mlcod=254'2050)", 2024-09-10T18:56:39.557 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:56:01.254879+0000", 2024-09-10T18:56:39.557 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 38.263053948, 2024-09-10T18:56:39.557 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.074207705999999998, 2024-09-10T18:56:39.557 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:56:39.557 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:56:39.558 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:56:39.558 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.558 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:56:39.558 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.254879+0000", 2024-09-10T18:56:39.558 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.558 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.558 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.558 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:56:39.558 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.254879+0000", 2024-09-10T18:56:39.558 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.558 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.558 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.559 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:56:39.559 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.254879+0000", 2024-09-10T18:56:39.559 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T18:56:39.559 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.559 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.559 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:56:39.559 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.254897+0000", 2024-09-10T18:56:39.559 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.8717000000000001e-05 2024-09-10T18:56:39.559 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.559 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.559 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:56:39.559 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.321628+0000", 2024-09-10T18:56:39.559 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.066730359000000003 2024-09-10T18:56:39.560 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.560 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.560 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:56:39.560 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.321639+0000", 2024-09-10T18:56:39.560 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1412e-05 2024-09-10T18:56:39.560 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.560 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.560 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:56:39.560 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.322282+0000", 2024-09-10T18:56:39.560 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00064259099999999997 2024-09-10T18:56:39.560 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.560 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.560 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:56:39.561 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.322300+0000", 2024-09-10T18:56:39.561 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.806e-05 2024-09-10T18:56:39.561 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.561 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.561 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:56:39.561 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.329006+0000", 2024-09-10T18:56:39.561 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0067056529999999998 2024-09-10T18:56:39.561 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.561 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.561 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:56:39.561 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:01.329087+0000", 2024-09-10T18:56:39.561 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.1442999999999999e-05 2024-09-10T18:56:39.562 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.562 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:39.562 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.562 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.562 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.562 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15354.0:35262 9.5 9:ac4b914e:::benchmark_data_smithi018_133594_object35261:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-10T18:56:39.562 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:56:02.324880+0000", 2024-09-10T18:56:39.562 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 37.193053008, 2024-09-10T18:56:39.562 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.059060420000000002, 2024-09-10T18:56:39.562 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:56:39.562 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:56:39.562 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:56:39.562 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15354", 2024-09-10T18:56:39.563 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T18:56:39.563 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 35262 2024-09-10T18:56:39.563 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.563 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:56:39.563 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.563 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:56:39.563 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:02.324880+0000", 2024-09-10T18:56:39.563 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.563 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.563 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.563 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:56:39.563 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:02.324880+0000", 2024-09-10T18:56:39.563 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.563 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.564 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.564 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:56:39.564 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:02.324879+0000", 2024-09-10T18:56:39.564 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T18:56:39.564 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.564 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.564 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:56:39.564 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:02.324906+0000", 2024-09-10T18:56:39.564 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.6871000000000001e-05 2024-09-10T18:56:39.564 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.564 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.564 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:56:39.565 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:02.324916+0000", 2024-09-10T18:56:39.565 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0729e-05 2024-09-10T18:56:39.565 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.565 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.565 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:56:39.565 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:02.324922+0000", 2024-09-10T18:56:39.565 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.5840000000000002e-06 2024-09-10T18:56:39.565 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.565 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.565 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:56:39.565 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:02.325000+0000", 2024-09-10T18:56:39.565 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.7615999999999997e-05 2024-09-10T18:56:39.565 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.566 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.566 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:56:39.566 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:02.325116+0000", 2024-09-10T18:56:39.566 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00011670199999999999 2024-09-10T18:56:39.566 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.566 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.566 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,7", 2024-09-10T18:56:39.566 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:02.325230+0000", 2024-09-10T18:56:39.566 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000113596 2024-09-10T18:56:39.566 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.566 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.566 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:56:39.566 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:02.380473+0000", 2024-09-10T18:56:39.567 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.055242807999999997 2024-09-10T18:56:39.567 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.567 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.567 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:56:39.567 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:02.380863+0000", 2024-09-10T18:56:39.567 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00039041599999999998 2024-09-10T18:56:39.567 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.567 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.567 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:56:39.568 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:02.383851+0000", 2024-09-10T18:56:39.568 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0029881059999999999 2024-09-10T18:56:39.568 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.568 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.568 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:56:39.568 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:02.383903+0000", 2024-09-10T18:56:39.568 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.1492000000000001e-05 2024-09-10T18:56:39.568 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.569 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.569 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:56:39.569 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:02.383941+0000", 2024-09-10T18:56:39.569 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.7845000000000001e-05 2024-09-10T18:56:39.569 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.569 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:39.569 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.569 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.569 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.569 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15354.0:35264 9.9 9:95b082b5:::benchmark_data_smithi018_133594_object35263:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-10T18:56:39.569 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:56:02.325089+0000", 2024-09-10T18:56:39.569 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 37.192843734999997, 2024-09-10T18:56:39.569 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.055941493000000002, 2024-09-10T18:56:39.570 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:56:39.570 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:56:39.570 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:56:39.570 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15354", 2024-09-10T18:56:39.570 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T18:56:39.570 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 35264 2024-09-10T18:56:39.570 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.570 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:56:39.570 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.570 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:56:39.570 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:02.325089+0000", 2024-09-10T18:56:39.570 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.571 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.571 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.571 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:56:39.571 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:02.325089+0000", 2024-09-10T18:56:39.571 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.571 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.571 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.571 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:56:39.571 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:02.325089+0000", 2024-09-10T18:56:39.571 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:56:39.571 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.571 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.571 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:56:39.572 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:02.325110+0000", 2024-09-10T18:56:39.572 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.1180000000000001e-05 2024-09-10T18:56:39.572 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.572 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.572 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:56:39.572 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:02.325119+0000", 2024-09-10T18:56:39.572 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.4380000000000001e-06 2024-09-10T18:56:39.572 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.572 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.572 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:56:39.572 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:02.325125+0000", 2024-09-10T18:56:39.572 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.8470000000000001e-06 2024-09-10T18:56:39.572 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.573 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.573 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:56:39.573 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:02.325154+0000", 2024-09-10T18:56:39.573 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.8864e-05 2024-09-10T18:56:39.573 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.573 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.573 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:56:39.573 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:02.325224+0000", 2024-09-10T18:56:39.573 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.9748999999999999e-05 2024-09-10T18:56:39.573 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.573 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.573 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 5,7", 2024-09-10T18:56:39.574 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:02.325340+0000", 2024-09-10T18:56:39.574 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00011618 2024-09-10T18:56:39.574 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.574 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.574 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:56:39.574 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:02.378993+0000", 2024-09-10T18:56:39.574 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.053653081999999998 2024-09-10T18:56:39.574 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.574 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.574 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:56:39.574 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:02.379026+0000", 2024-09-10T18:56:39.574 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.2539999999999997e-05 2024-09-10T18:56:39.574 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.574 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.575 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:56:39.575 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:02.380914+0000", 2024-09-10T18:56:39.575 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00188817 2024-09-10T18:56:39.575 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.575 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.575 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:56:39.575 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:02.380989+0000", 2024-09-10T18:56:39.575 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.5426999999999995e-05 2024-09-10T18:56:39.575 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.575 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.575 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:56:39.575 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:02.381031+0000", 2024-09-10T18:56:39.575 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.1792000000000002e-05 2024-09-10T18:56:39.575 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.576 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:39.576 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.576 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.576 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.576 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15354.0:37403 9.5 9:a852ebd3:::benchmark_data_smithi018_133594_object37402:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-10T18:56:39.576 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:56:03.438872+0000", 2024-09-10T18:56:39.576 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 36.079061453999998, 2024-09-10T18:56:39.576 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.065573553000000007, 2024-09-10T18:56:39.576 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:56:39.576 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:56:39.576 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:56:39.576 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15354", 2024-09-10T18:56:39.576 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T18:56:39.576 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 37403 2024-09-10T18:56:39.577 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.577 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:56:39.577 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.577 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:56:39.577 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:03.438872+0000", 2024-09-10T18:56:39.577 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.577 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.577 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.577 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:56:39.577 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:03.438872+0000", 2024-09-10T18:56:39.577 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.577 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.577 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.577 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:56:39.578 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:03.438870+0000", 2024-09-10T18:56:39.578 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T18:56:39.578 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.578 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.578 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:56:39.578 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:03.438896+0000", 2024-09-10T18:56:39.578 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.5341e-05 2024-09-10T18:56:39.578 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.578 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.578 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:56:39.578 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:03.438908+0000", 2024-09-10T18:56:39.578 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.2771e-05 2024-09-10T18:56:39.578 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.579 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.579 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:56:39.579 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:03.438915+0000", 2024-09-10T18:56:39.579 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.4230000000000002e-06 2024-09-10T18:56:39.579 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.579 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.579 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:56:39.579 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:03.438966+0000", 2024-09-10T18:56:39.579 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.1277999999999999e-05 2024-09-10T18:56:39.579 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.579 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.579 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:56:39.579 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:03.439090+0000", 2024-09-10T18:56:39.580 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00012437099999999999 2024-09-10T18:56:39.580 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.580 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.580 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,7", 2024-09-10T18:56:39.580 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:03.439327+0000", 2024-09-10T18:56:39.580 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00023683800000000001 2024-09-10T18:56:39.580 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.580 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.580 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:56:39.580 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:03.441029+0000", 2024-09-10T18:56:39.580 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0017019170000000001 2024-09-10T18:56:39.580 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.580 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.580 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:56:39.581 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:03.441374+0000", 2024-09-10T18:56:39.581 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00034509900000000002 2024-09-10T18:56:39.581 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.581 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.581 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:56:39.581 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:03.504324+0000", 2024-09-10T18:56:39.581 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.062950200999999997 2024-09-10T18:56:39.581 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.581 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.581 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:56:39.581 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:03.504409+0000", 2024-09-10T18:56:39.581 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.4109000000000004e-05 2024-09-10T18:56:39.582 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.582 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.582 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:56:39.582 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:03.504445+0000", 2024-09-10T18:56:39.582 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.6801000000000002e-05 2024-09-10T18:56:39.582 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.582 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:39.582 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.582 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.582 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.583 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:56:39.583 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15354.0:37624 9.5 9:a7947812:::benchmark_data_smithi018_133594_object37623:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-10T18:56:39.583 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:56:03.604085+0000", 2024-09-10T18:56:39.583 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 35.913848530999999, 2024-09-10T18:56:39.583 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.073508252999999996, 2024-09-10T18:56:39.583 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:56:39.583 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:56:39.583 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:56:39.584 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15354", 2024-09-10T18:56:39.584 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T18:56:39.584 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 37624 2024-09-10T18:56:39.584 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.584 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:56:39.584 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.584 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:56:39.584 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:03.604085+0000", 2024-09-10T18:56:39.584 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.584 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.584 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.584 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:56:39.584 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:03.604085+0000", 2024-09-10T18:56:39.585 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.585 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.585 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.585 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:56:39.585 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:03.604083+0000", 2024-09-10T18:56:39.585 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999981 2024-09-10T18:56:39.585 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.585 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.585 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:56:39.585 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:03.604119+0000", 2024-09-10T18:56:39.585 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.5948999999999998e-05 2024-09-10T18:56:39.585 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.585 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.586 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:56:39.586 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:03.604130+0000", 2024-09-10T18:56:39.586 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1235e-05 2024-09-10T18:56:39.586 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.586 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.586 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:56:39.586 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:03.604136+0000", 2024-09-10T18:56:39.586 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.7130000000000002e-06 2024-09-10T18:56:39.586 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.586 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.586 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:56:39.586 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:03.604274+0000", 2024-09-10T18:56:39.586 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00013799999999999999 2024-09-10T18:56:39.587 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.587 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.587 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:56:39.587 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:03.604383+0000", 2024-09-10T18:56:39.587 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000109016 2024-09-10T18:56:39.587 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.587 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.587 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,7", 2024-09-10T18:56:39.587 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:03.604512+0000", 2024-09-10T18:56:39.587 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000129309 2024-09-10T18:56:39.587 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.587 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.587 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:56:39.588 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:03.606157+0000", 2024-09-10T18:56:39.588 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0016450670000000001 2024-09-10T18:56:39.588 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.588 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.588 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:56:39.588 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:03.607068+0000", 2024-09-10T18:56:39.588 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00091120999999999997 2024-09-10T18:56:39.588 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.588 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.588 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:56:39.588 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:03.676692+0000", 2024-09-10T18:56:39.588 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.069623190000000001 2024-09-10T18:56:39.588 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.589 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.589 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:56:39.589 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:03.677429+0000", 2024-09-10T18:56:39.589 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000737639 2024-09-10T18:56:39.589 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.589 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.589 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:56:39.589 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:03.677593+0000", 2024-09-10T18:56:39.589 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000163785 2024-09-10T18:56:39.589 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.589 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:39.589 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.589 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.590 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.590 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15354.0:45460 9.9 9:9bab22b2:::benchmark_data_smithi018_133594_object45459:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-10T18:56:39.590 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:56:07.650674+0000", 2024-09-10T18:56:39.590 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 31.867258941999999, 2024-09-10T18:56:39.590 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.121705859, 2024-09-10T18:56:39.590 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:56:39.590 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:56:39.590 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:56:39.590 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15354", 2024-09-10T18:56:39.590 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T18:56:39.590 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 45460 2024-09-10T18:56:39.590 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.590 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:56:39.591 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.591 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:56:39.591 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.650674+0000", 2024-09-10T18:56:39.591 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.591 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.591 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.591 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:56:39.591 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.650674+0000", 2024-09-10T18:56:39.591 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.591 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.591 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.591 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:56:39.591 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.650673+0000", 2024-09-10T18:56:39.591 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T18:56:39.592 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.592 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.592 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:56:39.592 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.650698+0000", 2024-09-10T18:56:39.592 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.499e-05 2024-09-10T18:56:39.592 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.592 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.592 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:56:39.592 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.650711+0000", 2024-09-10T18:56:39.592 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.2773e-05 2024-09-10T18:56:39.592 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.592 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.592 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:56:39.593 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.650718+0000", 2024-09-10T18:56:39.593 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.0720000000000002e-06 2024-09-10T18:56:39.593 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.593 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.593 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:56:39.593 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.651304+0000", 2024-09-10T18:56:39.593 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00058651600000000003 2024-09-10T18:56:39.593 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.593 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.593 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:56:39.593 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.651417+0000", 2024-09-10T18:56:39.593 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000112419 2024-09-10T18:56:39.594 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.594 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.594 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 5,7", 2024-09-10T18:56:39.594 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.651545+0000", 2024-09-10T18:56:39.594 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000128319 2024-09-10T18:56:39.594 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.594 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.594 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:56:39.594 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.655387+0000", 2024-09-10T18:56:39.594 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0038421710000000001 2024-09-10T18:56:39.594 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.594 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.594 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:56:39.594 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.659465+0000", 2024-09-10T18:56:39.595 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0040783479999999999 2024-09-10T18:56:39.595 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.595 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.595 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:56:39.595 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.772309+0000", 2024-09-10T18:56:39.595 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.112843367 2024-09-10T18:56:39.595 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.595 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.595 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:56:39.595 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.772353+0000", 2024-09-10T18:56:39.595 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.4348000000000003e-05 2024-09-10T18:56:39.595 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.596 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.596 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:56:39.596 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.772380+0000", 2024-09-10T18:56:39.596 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.7047e-05 2024-09-10T18:56:39.596 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.596 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:39.596 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.596 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.596 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.596 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15354.0:45461 9.9 9:98841762:::benchmark_data_smithi018_133594_object45460:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-10T18:56:39.596 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:56:07.650733+0000", 2024-09-10T18:56:39.596 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 31.867200421, 2024-09-10T18:56:39.596 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.12179393099999999, 2024-09-10T18:56:39.597 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:56:39.597 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:56:39.597 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:56:39.597 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15354", 2024-09-10T18:56:39.597 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T18:56:39.597 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 45461 2024-09-10T18:56:39.598 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.598 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:56:39.598 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.598 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:56:39.598 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.650733+0000", 2024-09-10T18:56:39.598 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.598 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.598 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.598 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:56:39.598 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.650733+0000", 2024-09-10T18:56:39.598 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.599 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.599 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.599 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:56:39.599 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.650732+0000", 2024-09-10T18:56:39.599 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T18:56:39.599 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.599 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.599 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:56:39.599 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.650823+0000", 2024-09-10T18:56:39.599 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.0909000000000007e-05 2024-09-10T18:56:39.599 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.599 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.599 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:56:39.600 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.650833+0000", 2024-09-10T18:56:39.600 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.7100000000000002e-06 2024-09-10T18:56:39.600 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.600 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.600 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:56:39.600 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.650838+0000", 2024-09-10T18:56:39.600 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.3179999999999997e-06 2024-09-10T18:56:39.600 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.600 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.600 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:56:39.600 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.653275+0000", 2024-09-10T18:56:39.600 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0024367410000000001 2024-09-10T18:56:39.600 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.601 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.601 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:56:39.601 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.653391+0000", 2024-09-10T18:56:39.601 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000116025 2024-09-10T18:56:39.601 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.601 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.601 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 5,7", 2024-09-10T18:56:39.601 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.653526+0000", 2024-09-10T18:56:39.601 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000135461 2024-09-10T18:56:39.601 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.601 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.601 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:56:39.601 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.659433+0000", 2024-09-10T18:56:39.602 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0059069129999999997 2024-09-10T18:56:39.602 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.602 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.602 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:56:39.602 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.659512+0000", 2024-09-10T18:56:39.602 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.8532e-05 2024-09-10T18:56:39.602 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.602 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.602 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:56:39.602 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.772423+0000", 2024-09-10T18:56:39.603 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.112911149 2024-09-10T18:56:39.603 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.603 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.603 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:56:39.603 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.772491+0000", 2024-09-10T18:56:39.603 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.7979999999999996e-05 2024-09-10T18:56:39.603 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.603 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.603 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:56:39.603 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.772527+0000", 2024-09-10T18:56:39.603 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.5732999999999998e-05 2024-09-10T18:56:39.604 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.604 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:39.604 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.604 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.604 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.604 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15354.0:45466 9.9 9:92b7bc49:::benchmark_data_smithi018_133594_object45465:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-10T18:56:39.604 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:56:07.651862+0000", 2024-09-10T18:56:39.604 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 31.866070829999998, 2024-09-10T18:56:39.604 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.120778205, 2024-09-10T18:56:39.604 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:56:39.604 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:56:39.604 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:56:39.604 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15354", 2024-09-10T18:56:39.605 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T18:56:39.605 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 45466 2024-09-10T18:56:39.605 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.605 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:56:39.605 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.605 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:56:39.605 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.651862+0000", 2024-09-10T18:56:39.605 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.605 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.605 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.605 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:56:39.605 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.651862+0000", 2024-09-10T18:56:39.606 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.606 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.606 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.606 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:56:39.606 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.651862+0000", 2024-09-10T18:56:39.606 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:56:39.606 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.606 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.606 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:56:39.606 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.651881+0000", 2024-09-10T18:56:39.606 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.9772999999999999e-05 2024-09-10T18:56:39.606 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.607 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.607 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:56:39.607 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.651889+0000", 2024-09-10T18:56:39.607 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.8369999999999997e-06 2024-09-10T18:56:39.607 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.607 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.607 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:56:39.607 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.651894+0000", 2024-09-10T18:56:39.608 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.7029999999999997e-06 2024-09-10T18:56:39.608 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.608 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.608 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:56:39.608 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.655514+0000", 2024-09-10T18:56:39.608 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0036203199999999998 2024-09-10T18:56:39.608 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.608 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.608 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:56:39.608 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.655616+0000", 2024-09-10T18:56:39.608 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000102098 2024-09-10T18:56:39.608 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.609 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.609 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 5,7", 2024-09-10T18:56:39.609 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.655735+0000", 2024-09-10T18:56:39.609 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000118354 2024-09-10T18:56:39.609 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.609 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.609 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:56:39.609 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.659480+0000", 2024-09-10T18:56:39.609 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.003745511 2024-09-10T18:56:39.609 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.609 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.609 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:56:39.609 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.662302+0000", 2024-09-10T18:56:39.610 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0028218029999999999 2024-09-10T18:56:39.610 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.610 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.610 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:56:39.610 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.772567+0000", 2024-09-10T18:56:39.610 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.110265212 2024-09-10T18:56:39.610 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.610 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.610 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:56:39.610 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.772607+0000", 2024-09-10T18:56:39.610 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.9947000000000002e-05 2024-09-10T18:56:39.610 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.610 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.611 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:56:39.611 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.772641+0000", 2024-09-10T18:56:39.611 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.3427999999999999e-05 2024-09-10T18:56:39.611 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.611 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:39.611 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.611 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.611 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.611 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15354.0:45471 9.9 9:9c39bc4b:::benchmark_data_smithi018_133594_object45470:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-10T18:56:39.611 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:56:07.654147+0000", 2024-09-10T18:56:39.611 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 31.863786592, 2024-09-10T18:56:39.611 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.11869291, 2024-09-10T18:56:39.611 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:56:39.612 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:56:39.612 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:56:39.612 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15354", 2024-09-10T18:56:39.612 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T18:56:39.612 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 45471 2024-09-10T18:56:39.612 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.612 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:56:39.612 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.612 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:56:39.612 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.654147+0000", 2024-09-10T18:56:39.612 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.612 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.613 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.613 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:56:39.613 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.654147+0000", 2024-09-10T18:56:39.613 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.613 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.613 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.613 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:56:39.613 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.654145+0000", 2024-09-10T18:56:39.613 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T18:56:39.613 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.613 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.613 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:56:39.613 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.654172+0000", 2024-09-10T18:56:39.614 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:39.615 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.6652000000000002e-05 2024-09-10T18:56:39.615 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.615 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.615 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:56:39.615 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.654194+0000", 2024-09-10T18:56:39.615 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.2810999999999999e-05 2024-09-10T18:56:39.615 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.615 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.615 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:56:39.616 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.654201+0000", 2024-09-10T18:56:39.616 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.246e-06 2024-09-10T18:56:39.616 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.616 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.616 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:56:39.616 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.657398+0000", 2024-09-10T18:56:39.616 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0031977120000000001 2024-09-10T18:56:39.616 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.616 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.616 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:56:39.616 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.657497+0000", 2024-09-10T18:56:39.616 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.8945999999999998e-05 2024-09-10T18:56:39.617 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.617 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.617 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 5,7", 2024-09-10T18:56:39.617 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.657616+0000", 2024-09-10T18:56:39.617 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000118583 2024-09-10T18:56:39.617 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.617 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.617 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:56:39.617 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.661252+0000", 2024-09-10T18:56:39.617 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0036361739999999998 2024-09-10T18:56:39.617 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.618 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.618 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:56:39.618 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.662330+0000", 2024-09-10T18:56:39.618 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.001078044 2024-09-10T18:56:39.618 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.618 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.618 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:56:39.618 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.772734+0000", 2024-09-10T18:56:39.618 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.110403693 2024-09-10T18:56:39.618 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.618 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.618 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:56:39.618 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.772803+0000", 2024-09-10T18:56:39.619 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.9632e-05 2024-09-10T18:56:39.619 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.619 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.619 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:56:39.619 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.772840+0000", 2024-09-10T18:56:39.619 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.6040000000000001e-05 2024-09-10T18:56:39.619 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.619 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:39.619 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.619 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.619 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.619 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15354.0:45478 9.9 9:99c8c6f6:::benchmark_data_smithi018_133594_object45477:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-10T18:56:39.619 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:56:07.657175+0000", 2024-09-10T18:56:39.619 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 31.860757934999999, 2024-09-10T18:56:39.620 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.116097618, 2024-09-10T18:56:39.620 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:56:39.620 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:56:39.620 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:56:39.620 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15354", 2024-09-10T18:56:39.620 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T18:56:39.620 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 45478 2024-09-10T18:56:39.620 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.620 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:56:39.620 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.620 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:56:39.620 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.657175+0000", 2024-09-10T18:56:39.620 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.620 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.621 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.621 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:56:39.621 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.657175+0000", 2024-09-10T18:56:39.621 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.621 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.621 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.621 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:56:39.621 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.657174+0000", 2024-09-10T18:56:39.621 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T18:56:39.621 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.621 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.621 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:56:39.621 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.657198+0000", 2024-09-10T18:56:39.622 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.3672999999999998e-05 2024-09-10T18:56:39.622 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.622 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.622 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:56:39.622 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.657207+0000", 2024-09-10T18:56:39.622 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.3829999999999998e-06 2024-09-10T18:56:39.622 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.622 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.622 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:56:39.622 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.657212+0000", 2024-09-10T18:56:39.622 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.8470000000000003e-06 2024-09-10T18:56:39.622 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.622 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.622 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:56:39.623 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.659535+0000", 2024-09-10T18:56:39.623 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0023229380000000001 2024-09-10T18:56:39.623 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.623 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.623 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:56:39.623 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.659647+0000", 2024-09-10T18:56:39.623 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000112154 2024-09-10T18:56:39.623 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.623 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.623 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 5,7", 2024-09-10T18:56:39.623 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.659768+0000", 2024-09-10T18:56:39.623 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000121138 2024-09-10T18:56:39.623 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.624 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.624 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:56:39.624 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.661404+0000", 2024-09-10T18:56:39.624 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0016355499999999999 2024-09-10T18:56:39.624 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.624 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.624 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:56:39.624 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.664514+0000", 2024-09-10T18:56:39.624 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0031101000000000002 2024-09-10T18:56:39.624 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.624 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.624 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:56:39.624 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.773022+0000", 2024-09-10T18:56:39.625 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.108508041 2024-09-10T18:56:39.625 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.625 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.625 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:56:39.625 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.773236+0000", 2024-09-10T18:56:39.625 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00021395499999999999 2024-09-10T18:56:39.625 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.625 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.625 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:56:39.625 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.773273+0000", 2024-09-10T18:56:39.625 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.7123999999999999e-05 2024-09-10T18:56:39.625 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.625 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:39.625 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.626 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.626 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.626 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15354.0:45490 9.5 9:adb8ff9c:::benchmark_data_smithi018_133594_object45489:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-10T18:56:39.626 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:56:07.665367+0000", 2024-09-10T18:56:39.626 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 31.852566504999999, 2024-09-10T18:56:39.626 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.115380485, 2024-09-10T18:56:39.626 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:56:39.626 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:56:39.626 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:56:39.626 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15354", 2024-09-10T18:56:39.626 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T18:56:39.626 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 45490 2024-09-10T18:56:39.626 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.627 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:56:39.627 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.627 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:56:39.627 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.665367+0000", 2024-09-10T18:56:39.627 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.627 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.627 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.627 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:56:39.627 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.665367+0000", 2024-09-10T18:56:39.627 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.627 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.627 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.627 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:56:39.627 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.665365+0000", 2024-09-10T18:56:39.628 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999981 2024-09-10T18:56:39.628 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.628 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.628 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:56:39.628 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.665400+0000", 2024-09-10T18:56:39.628 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.4774000000000003e-05 2024-09-10T18:56:39.628 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.628 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.628 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:56:39.628 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.665412+0000", 2024-09-10T18:56:39.628 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.2799000000000001e-05 2024-09-10T18:56:39.628 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.628 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.629 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:56:39.629 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.665419+0000", 2024-09-10T18:56:39.629 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.5080000000000002e-06 2024-09-10T18:56:39.629 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.629 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.629 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:56:39.629 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.665472+0000", 2024-09-10T18:56:39.629 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.3093999999999997e-05 2024-09-10T18:56:39.629 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.629 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.629 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:56:39.629 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.772300+0000", 2024-09-10T18:56:39.630 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.106827952 2024-09-10T18:56:39.630 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.630 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.630 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,7", 2024-09-10T18:56:39.630 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.772461+0000", 2024-09-10T18:56:39.630 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000161259 2024-09-10T18:56:39.630 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.630 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.630 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:56:39.630 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.777063+0000", 2024-09-10T18:56:39.630 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0046014430000000002 2024-09-10T18:56:39.630 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.630 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.631 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:56:39.631 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.777172+0000", 2024-09-10T18:56:39.631 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00010945100000000001 2024-09-10T18:56:39.631 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.631 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.631 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:56:39.631 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.780650+0000", 2024-09-10T18:56:39.631 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.003478054 2024-09-10T18:56:39.631 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.631 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.631 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:56:39.631 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.780710+0000", 2024-09-10T18:56:39.631 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.0161000000000002e-05 2024-09-10T18:56:39.632 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.632 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.632 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:56:39.632 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.780747+0000", 2024-09-10T18:56:39.632 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.6801000000000002e-05 2024-09-10T18:56:39.632 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.632 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:39.632 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.632 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.632 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.632 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop_reply(client.15354.0:45460 9.9 e254/241 ondisk, result = 0)", 2024-09-10T18:56:39.632 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:56:07.665746+0000", 2024-09-10T18:56:39.633 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 31.852187338, 2024-09-10T18:56:39.633 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.10664362500000001, 2024-09-10T18:56:39.633 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:56:39.633 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "started", 2024-09-10T18:56:39.633 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:56:39.633 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.633 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:56:39.633 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.665746+0000", 2024-09-10T18:56:39.633 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.633 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.633 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.633 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:56:39.633 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.665746+0000", 2024-09-10T18:56:39.634 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.634 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.634 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.634 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:56:39.634 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.665745+0000", 2024-09-10T18:56:39.634 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:56:39.634 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.634 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.634 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:56:39.634 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.665751+0000", 2024-09-10T18:56:39.634 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.8490000000000002e-06 2024-09-10T18:56:39.634 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.634 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.634 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:56:39.635 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.772197+0000", 2024-09-10T18:56:39.635 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.10644598399999999 2024-09-10T18:56:39.635 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.635 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.635 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:56:39.635 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.772223+0000", 2024-09-10T18:56:39.635 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.6047e-05 2024-09-10T18:56:39.635 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.635 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.635 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:56:39.635 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.772294+0000", 2024-09-10T18:56:39.635 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.1048000000000001e-05 2024-09-10T18:56:39.635 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.636 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.636 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:56:39.636 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.772302+0000", 2024-09-10T18:56:39.636 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.1680000000000004e-06 2024-09-10T18:56:39.636 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.636 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.636 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:56:39.636 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.772390+0000", 2024-09-10T18:56:39.636 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.7429999999999995e-05 2024-09-10T18:56:39.636 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.636 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:39.636 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.636 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.637 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.637 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15354.0:45497 9.5 9:abdacfb0:::benchmark_data_smithi018_133594_object45496:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-10T18:56:39.637 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:56:07.666694+0000", 2024-09-10T18:56:39.637 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 31.851238787, 2024-09-10T18:56:39.637 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.114169545, 2024-09-10T18:56:39.637 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:56:39.637 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:56:39.637 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:56:39.637 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15354", 2024-09-10T18:56:39.637 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T18:56:39.637 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 45497 2024-09-10T18:56:39.637 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.637 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:56:39.638 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.638 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:56:39.638 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.666694+0000", 2024-09-10T18:56:39.638 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.638 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.638 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.638 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:56:39.638 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.666694+0000", 2024-09-10T18:56:39.638 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.638 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.638 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.638 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:56:39.638 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.666693+0000", 2024-09-10T18:56:39.639 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T18:56:39.639 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.639 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.639 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:56:39.639 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.666720+0000", 2024-09-10T18:56:39.639 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.7107e-05 2024-09-10T18:56:39.639 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.639 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.639 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:56:39.639 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.772201+0000", 2024-09-10T18:56:39.639 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.10548047000000001 2024-09-10T18:56:39.639 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.639 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.640 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:56:39.640 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.772227+0000", 2024-09-10T18:56:39.640 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.6550000000000002e-05 2024-09-10T18:56:39.640 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.640 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.640 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:56:39.640 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.774684+0000", 2024-09-10T18:56:39.640 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00245699 2024-09-10T18:56:39.640 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.640 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.640 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:56:39.640 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.774786+0000", 2024-09-10T18:56:39.640 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000101993 2024-09-10T18:56:39.641 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.641 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.641 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,7", 2024-09-10T18:56:39.641 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.774965+0000", 2024-09-10T18:56:39.641 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00017895300000000001 2024-09-10T18:56:39.641 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.641 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.641 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:56:39.641 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.777068+0000", 2024-09-10T18:56:39.641 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.002103316 2024-09-10T18:56:39.641 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.641 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.641 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:56:39.642 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.777216+0000", 2024-09-10T18:56:39.642 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00014776599999999999 2024-09-10T18:56:39.642 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.642 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.642 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:56:39.642 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.780792+0000", 2024-09-10T18:56:39.642 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0035761759999999999 2024-09-10T18:56:39.642 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.642 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.642 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:56:39.642 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.780834+0000", 2024-09-10T18:56:39.642 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.1448e-05 2024-09-10T18:56:39.642 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.643 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.643 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:56:39.643 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.780864+0000", 2024-09-10T18:56:39.643 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.0162e-05 2024-09-10T18:56:39.643 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.643 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:39.643 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.643 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.643 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.643 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.15354.0:45500 9.0 e254/241 9:0c70411f:::benchmark_data_smithi018_133594_object45499:head v 254'2850, mlcod=254'2848)", 2024-09-10T18:56:39.643 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:56:07.670061+0000", 2024-09-10T18:56:39.643 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 31.847872707000001, 2024-09-10T18:56:39.643 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.10938732800000001, 2024-09-10T18:56:39.643 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:56:39.644 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:56:39.644 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:56:39.644 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.644 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:56:39.644 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.670061+0000", 2024-09-10T18:56:39.644 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.644 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.644 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.644 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:56:39.644 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.670061+0000", 2024-09-10T18:56:39.644 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.644 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.644 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.645 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:56:39.645 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.670060+0000", 2024-09-10T18:56:39.645 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:56:39.645 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.645 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.645 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:56:39.645 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.670085+0000", 2024-09-10T18:56:39.645 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.5384e-05 2024-09-10T18:56:39.645 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.645 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.645 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:56:39.645 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.772240+0000", 2024-09-10T18:56:39.645 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.10215550900000001 2024-09-10T18:56:39.646 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.646 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.646 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:56:39.646 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.772261+0000", 2024-09-10T18:56:39.646 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.0361999999999999e-05 2024-09-10T18:56:39.646 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.646 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.646 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:56:39.646 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.772395+0000", 2024-09-10T18:56:39.646 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000134647 2024-09-10T18:56:39.646 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.646 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.646 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:56:39.647 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.772417+0000", 2024-09-10T18:56:39.647 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.1058000000000001e-05 2024-09-10T18:56:39.647 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.647 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.647 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:56:39.647 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.778720+0000", 2024-09-10T18:56:39.647 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0063033919999999997 2024-09-10T18:56:39.647 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.647 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.647 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:56:39.647 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:07.779448+0000", 2024-09-10T18:56:39.647 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00072792099999999997 2024-09-10T18:56:39.647 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.648 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:39.648 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.648 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.648 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.648 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15354.0:55508 9.b 9:dbf3e7fb:::benchmark_data_smithi018_133594_object55507:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e256)", 2024-09-10T18:56:39.648 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:56:13.709978+0000", 2024-09-10T18:56:39.648 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 25.807954994999999, 2024-09-10T18:56:39.648 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.054288626, 2024-09-10T18:56:39.648 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:56:39.648 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:56:39.648 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:56:39.648 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15354", 2024-09-10T18:56:39.648 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T18:56:39.649 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 55508 2024-09-10T18:56:39.649 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.649 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:56:39.649 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.649 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:56:39.649 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:13.709978+0000", 2024-09-10T18:56:39.649 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.649 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.649 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.649 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:56:39.649 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:13.709978+0000", 2024-09-10T18:56:39.649 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:56:39.649 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.649 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.650 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:56:39.650 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:13.709977+0000", 2024-09-10T18:56:39.650 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999981 2024-09-10T18:56:39.650 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.650 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.650 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:56:39.650 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:13.710001+0000", 2024-09-10T18:56:39.650 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.4643000000000001e-05 2024-09-10T18:56:39.650 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.650 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.650 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:56:39.650 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:13.710018+0000", 2024-09-10T18:56:39.650 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.6824999999999999e-05 2024-09-10T18:56:39.651 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.651 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.651 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:56:39.651 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:13.710046+0000", 2024-09-10T18:56:39.651 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.8017999999999998e-05 2024-09-10T18:56:39.651 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.651 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.651 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:56:39.651 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:13.710098+0000", 2024-09-10T18:56:39.651 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.2389999999999998e-05 2024-09-10T18:56:39.651 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.651 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.651 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:56:39.652 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:13.710210+0000", 2024-09-10T18:56:39.652 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000111513 2024-09-10T18:56:39.652 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.652 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.652 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 5,6", 2024-09-10T18:56:39.652 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:13.710330+0000", 2024-09-10T18:56:39.652 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000120034 2024-09-10T18:56:39.652 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.652 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.652 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:56:39.652 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:13.712342+0000", 2024-09-10T18:56:39.652 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.002012364 2024-09-10T18:56:39.652 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.653 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.653 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:56:39.653 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:13.740475+0000", 2024-09-10T18:56:39.653 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.028132957 2024-09-10T18:56:39.653 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.653 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.653 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:56:39.653 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:13.764005+0000", 2024-09-10T18:56:39.653 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.023529377000000001 2024-09-10T18:56:39.653 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.653 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.653 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:56:39.653 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:13.764182+0000", 2024-09-10T18:56:39.653 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00017731499999999999 2024-09-10T18:56:39.654 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:56:39.654 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:56:39.654 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:56:39.654 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:56:13.764267+0000", 2024-09-10T18:56:39.654 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.4872999999999999e-05 2024-09-10T18:56:39.654 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.654 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:39.654 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.654 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:56:39.654 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:56:39.654 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T18:56:39.658 INFO:teuthology.orchestra.run.smithi018.stderr:set 2.11 pg_upmap mapping to [6,7,5] 2024-09-10T18:56:39.684 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:39.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:39.685 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:39.786 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:39.829 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:39.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:39.830 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:39.848 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:39 smithi018 ceph-mon[29336]: osdmap e260: 8 total, 8 up, 7 in 2024-09-10T18:56:39.848 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:39 smithi018 ceph-mon[29336]: pgmap v515: 168 pgs: 168 active+clean; 5.4 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.2 KiB/s rd, 109 MiB/s wr, 1.74k op/s 2024-09-10T18:56:39.848 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:39 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3377122698' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "2.11", "id": [6, 7, 5]}]: dispatch 2024-09-10T18:56:39.848 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:39 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "2.11", "id": [6, 7, 5]}]: dispatch 2024-09-10T18:56:39.848 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:39 smithi018 ceph-mon[40196]: osdmap e260: 8 total, 8 up, 7 in 2024-09-10T18:56:39.848 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:39 smithi018 ceph-mon[40196]: pgmap v515: 168 pgs: 168 active+clean; 5.4 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.2 KiB/s rd, 109 MiB/s wr, 1.74k op/s 2024-09-10T18:56:39.848 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:39 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3377122698' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "2.11", "id": [6, 7, 5]}]: dispatch 2024-09-10T18:56:39.848 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:39 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "2.11", "id": [6, 7, 5]}]: dispatch 2024-09-10T18:56:39.931 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:39.972 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:39.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:39.974 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:40.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:39 smithi103 ceph-mon[33163]: osdmap e260: 8 total, 8 up, 7 in 2024-09-10T18:56:40.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:39 smithi103 ceph-mon[33163]: pgmap v515: 168 pgs: 168 active+clean; 5.4 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.2 KiB/s rd, 109 MiB/s wr, 1.74k op/s 2024-09-10T18:56:40.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:39 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3377122698' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "2.11", "id": [6, 7, 5]}]: dispatch 2024-09-10T18:56:40.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:39 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "2.11", "id": [6, 7, 5]}]: dispatch 2024-09-10T18:56:40.074 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:40.105 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:40.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:40.106 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:40.207 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:40.236 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:40.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:40.237 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:40.338 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:40.369 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:40.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:40.371 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:40.391 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 52 16 93208 93192 111.99 86.3125 0.0135466 0.0089247 2024-09-10T18:56:40.442 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-10T18:56:40.471 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:40.506 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:40.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:40.508 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:40.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:56:40 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:56:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:56:40.609 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:40.638 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:40.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:40.639 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:40.740 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:40.778 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:40.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:40.779 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:40.880 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:40.923 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:40.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:40.925 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:40.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:40 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "2.11", "id": [6, 7, 5]}]': finished 2024-09-10T18:56:40.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:40 smithi018 ceph-mon[29336]: osdmap e261: 8 total, 8 up, 7 in 2024-09-10T18:56:40.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:40 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:56:40.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:40 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:56:40.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:40 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "2.11", "id": [6, 7, 5]}]': finished 2024-09-10T18:56:40.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:40 smithi018 ceph-mon[40196]: osdmap e261: 8 total, 8 up, 7 in 2024-09-10T18:56:40.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:40 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:56:40.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:40 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:56:41.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:40 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "2.11", "id": [6, 7, 5]}]': finished 2024-09-10T18:56:41.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:40 smithi103 ceph-mon[33163]: osdmap e261: 8 total, 8 up, 7 in 2024-09-10T18:56:41.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:40 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:56:41.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:40 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:56:41.026 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:41.057 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:41.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:41.058 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:41.158 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:41.194 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:41.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:41.194 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:41.285 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:56:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:56:40] "GET /metrics HTTP/1.1" 200 224226 "" "Prometheus/2.43.0" 2024-09-10T18:56:41.295 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:41.339 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:41.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:41.340 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:41.370 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-10T18:56:41.391 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 53 16 94637 94621 111.562 89.3125 0.00921192 0.0089589 2024-09-10T18:56:41.441 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:41.468 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:41.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:41.469 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:41.570 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:41.625 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:41.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:41.626 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:41.727 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:41.757 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:41.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:41.758 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:41.859 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:41.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:41.905 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:41.905 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:41.911 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:41 smithi018 ceph-mon[29336]: osdmap e262: 8 total, 8 up, 7 in 2024-09-10T18:56:41.911 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:41 smithi018 ceph-mon[29336]: pgmap v518: 168 pgs: 168 active+clean; 5.4 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 853 B/s rd, 76 MiB/s wr, 1.21k op/s 2024-09-10T18:56:41.911 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:41 smithi018 ceph-mon[40196]: osdmap e262: 8 total, 8 up, 7 in 2024-09-10T18:56:41.911 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:41 smithi018 ceph-mon[40196]: pgmap v518: 168 pgs: 168 active+clean; 5.4 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 853 B/s rd, 76 MiB/s wr, 1.21k op/s 2024-09-10T18:56:42.006 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:42.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:41 smithi103 ceph-mon[33163]: osdmap e262: 8 total, 8 up, 7 in 2024-09-10T18:56:42.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:41 smithi103 ceph-mon[33163]: pgmap v518: 168 pgs: 168 active+clean; 5.4 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 853 B/s rd, 76 MiB/s wr, 1.21k op/s 2024-09-10T18:56:42.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:42.040 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:42.040 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:42.141 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:42.176 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:42.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:42.177 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:42.278 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:42.313 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:42.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:42.314 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:42.391 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 54 16 96118 96102 111.21 92.5625 0.00422228 0.00898754 2024-09-10T18:56:42.415 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:42.445 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:42.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:42.446 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:42.547 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:42.580 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:42.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:42.581 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:42.682 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:42.710 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:42.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:42.711 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:42.811 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:42.859 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:42.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:42.860 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:42.961 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:42.995 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:42.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:42.996 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:43.097 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:43.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:43.143 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:43.143 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:43.207 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:56:43.244 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:43.269 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:43.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:43.270 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:43.371 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:43.395 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 55 16 97546 97530 110.804 89.25 0.0100193 0.00902024 2024-09-10T18:56:43.403 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:43.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:43.405 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:43.506 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:43.548 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:43.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:43.550 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:43.652 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:43.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:43.698 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:43.698 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:43.799 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:43.830 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:43.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:43.831 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:43.932 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:43.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:43.972 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:43.973 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:44.073 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:44.104 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:44.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:44.105 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:44.205 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:44.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:44.253 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:44.253 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:44.354 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:44.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:44.393 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:44.394 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:44.394 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:44 smithi018 ceph-mon[40196]: pgmap v519: 168 pgs: 168 active+clean; 5.7 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 853 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-10T18:56:44.394 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:44 smithi018 ceph-mon[29336]: pgmap v519: 168 pgs: 168 active+clean; 5.7 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 853 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-10T18:56:44.395 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 56 16 98996 98980 110.443 90.625 0.00984483 0.00904973 2024-09-10T18:56:44.494 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:44.525 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:44.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:44.526 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:44.627 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:44.660 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:44.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:44.661 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:44.724 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:44 smithi103 ceph-mon[33163]: pgmap v519: 168 pgs: 168 active+clean; 5.7 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 853 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-10T18:56:44.763 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:44.801 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:44.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:44.802 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:44.903 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:44.954 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:44.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:44.955 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:45.056 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:45.071 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:56:45.103 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:45.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:45.104 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:45.109 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:56:45.109 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:56:45.143 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:56:45.143 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:56:45.172 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:56:45.172 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:56:45.201 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:56:45.201 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:56:45.205 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:45.235 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:56:45.235 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:56:45.243 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:45.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:45.244 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:45.269 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:56:45.269 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:56:45.303 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:56:45.303 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:56:45.342 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:56:45.342 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:56:45.345 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:45.370 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:45 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1154123284' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:56:45.371 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:45 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1154123284' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:56:45.387 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:45.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:45.389 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:45.396 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 57 16 100352 100336 109.991 84.75 0.0212734 0.00908619 2024-09-10T18:56:45.437 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:45 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1154123284' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:56:45.490 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:45.525 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:45.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:45.526 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:45.626 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:45.664 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:45.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:45.665 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:45.766 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:45.802 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:45.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:45.803 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:45.904 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:45.936 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:45.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:45.937 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:46.038 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:46.070 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:46.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:46.071 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:46.172 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:46.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:46.207 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:46.207 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:46.308 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:46.344 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:46.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:46.346 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:46.396 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 58 16 101693 101677 109.539 83.8125 0.0106493 0.00912469 2024-09-10T18:56:46.446 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:46.481 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:46.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:46.482 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:46.583 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:46.609 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:46 smithi103 ceph-mon[33163]: pgmap v520: 168 pgs: 168 active+clean; 5.8 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 773 B/s rd, 67 MiB/s wr, 1.07k op/s 2024-09-10T18:56:46.618 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:46.619 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:46 smithi018 ceph-mon[40196]: pgmap v520: 168 pgs: 168 active+clean; 5.8 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 773 B/s rd, 67 MiB/s wr, 1.07k op/s 2024-09-10T18:56:46.620 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:46 smithi018 ceph-mon[29336]: pgmap v520: 168 pgs: 168 active+clean; 5.8 GiB data, 21 GiB used, 604 GiB / 626 GiB avail; 773 B/s rd, 67 MiB/s wr, 1.07k op/s 2024-09-10T18:56:46.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:46.620 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:46.721 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:46.745 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:46.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:46.746 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:46.846 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:46.878 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:46.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:46.879 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:46.980 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:47.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:47.017 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:47.017 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:47.022 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:56:46 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:56:47.118 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:47.154 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:47.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:47.155 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:47.256 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:47.277 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:47.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:47.279 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:47.380 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:47.396 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 59 16 102839 102823 108.897 71.625 0.0169816 0.00917851 2024-09-10T18:56:47.411 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:47.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:47.412 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:47.513 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:47.549 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:47.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:47.551 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:47.651 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:47.675 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:47.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:47.676 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:47.690 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 6, 7, 0] 2024-09-10T18:56:47.690 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T18:56:47.691 INFO:tasks.thrashosds.thrasher:Removing osd 7, in_osds are: [0, 1, 3, 4, 5, 6, 7] 2024-09-10T18:56:47.691 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd out 7 2024-09-10T18:56:47.777 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:47.803 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:47.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:47.804 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:47.905 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:47.932 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:47.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:47.933 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:48.034 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:48.081 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:48.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:48.082 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:48.183 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:48.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:48.217 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:48.218 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:48.318 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:48.358 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is set 2024-09-10T18:56:48.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:48.359 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:48.359 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:48.386 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:48 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:48.386 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:48 smithi018 ceph-mon[29336]: pgmap v521: 168 pgs: 168 active+clean; 6.2 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 1.2 KiB/s rd, 100 MiB/s wr, 1.61k op/s 2024-09-10T18:56:48.386 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:48 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2025506404' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:56:48.386 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:48 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:56:48.386 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:48 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:48.386 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:48 smithi018 ceph-mon[40196]: pgmap v521: 168 pgs: 168 active+clean; 6.2 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 1.2 KiB/s rd, 100 MiB/s wr, 1.61k op/s 2024-09-10T18:56:48.386 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:48 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2025506404' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:56:48.386 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:48 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:56:48.396 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:2024-09-10T18:56:48.395937+0000 min lat: 0.00128005 max lat: 0.159628 avg lat: 0.00922151 2024-09-10T18:56:48.396 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T18:56:48.396 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 60 16 104095 104079 108.39 78.5 0.00891716 0.00922151 2024-09-10T18:56:48.460 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:48.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:48.501 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:48.502 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:48.602 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:48.635 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:48.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:48.636 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:48.737 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:48.768 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:48.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:48.769 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:48.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:48 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:48.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:48 smithi103 ceph-mon[33163]: pgmap v521: 168 pgs: 168 active+clean; 6.2 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 1.2 KiB/s rd, 100 MiB/s wr, 1.61k op/s 2024-09-10T18:56:48.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:48 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2025506404' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:56:48.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:48 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:56:48.870 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:48.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:48.901 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:48.901 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:49.002 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:49.039 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:49.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:49.041 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:49.141 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:49.180 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:49.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:49.181 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:49.282 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:49.321 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:49.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:49.323 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:49.396 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 61 16 105617 105601 108.172 95.125 0.0121744 0.00924003 2024-09-10T18:56:49.424 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:49.471 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:49.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:49.472 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:49.550 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:49 smithi018 ceph-mon[40196]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:56:49.550 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:49 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:56:49.550 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:49 smithi018 ceph-mon[40196]: osdmap e263: 8 total, 8 up, 7 in 2024-09-10T18:56:49.550 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:49 smithi018 ceph-mon[40196]: osdmap e264: 8 total, 8 up, 7 in 2024-09-10T18:56:49.550 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:49 smithi018 ceph-mon[29336]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:56:49.550 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:49 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:56:49.550 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:49 smithi018 ceph-mon[29336]: osdmap e263: 8 total, 8 up, 7 in 2024-09-10T18:56:49.551 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:49 smithi018 ceph-mon[29336]: osdmap e264: 8 total, 8 up, 7 in 2024-09-10T18:56:49.573 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:49.604 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:49.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:49.606 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:49.706 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:49.746 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:49.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:49.747 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:49.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:49 smithi103 ceph-mon[33163]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:56:49.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:49 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:56:49.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:49 smithi103 ceph-mon[33163]: osdmap e263: 8 total, 8 up, 7 in 2024-09-10T18:56:49.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:49 smithi103 ceph-mon[33163]: osdmap e264: 8 total, 8 up, 7 in 2024-09-10T18:56:49.848 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:49.884 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:49.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:49.886 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:49.986 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:50.026 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:50.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:50.027 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:50.128 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:50.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:50.178 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:50.178 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:50.279 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:50.317 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:50.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:50.319 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:50.396 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 62 16 107107 107091 107.93 93.125 0.013771 0.00926135 2024-09-10T18:56:50.420 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:50.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:50.454 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:50.454 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:50.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:50 smithi103 ceph-mon[33163]: pgmap v524: 168 pgs: 168 active+clean; 6.2 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 1.2 KiB/s rd, 110 MiB/s wr, 1.76k op/s 2024-09-10T18:56:50.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:56:50 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:56:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:56:50.556 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:50.585 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:50.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:50.587 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:50.687 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:50.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:50 smithi018 ceph-mon[29336]: pgmap v524: 168 pgs: 168 active+clean; 6.2 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 1.2 KiB/s rd, 110 MiB/s wr, 1.76k op/s 2024-09-10T18:56:50.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:50 smithi018 ceph-mon[40196]: pgmap v524: 168 pgs: 168 active+clean; 6.2 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 1.2 KiB/s rd, 110 MiB/s wr, 1.76k op/s 2024-09-10T18:56:50.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:50.730 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:50.731 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:50.832 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:50.875 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:50.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:50.876 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:50.977 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:51.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:51.012 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:51.013 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:51.049 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:56:51.056 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:56:50 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:56:50] "GET /metrics HTTP/1.1" 200 224226 "" "Prometheus/2.43.0" 2024-09-10T18:56:51.113 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:51.140 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:51.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:51.141 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:51.242 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:51.263 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-10T18:56:51.275 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:51.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:51.276 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:51.369 INFO:teuthology.orchestra.run.smithi018.stdout:ERROR: (22) Invalid argument 2024-09-10T18:56:51.369 INFO:teuthology.orchestra.run.smithi018.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:56:51.377 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:51.396 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 63 16 108499 108483 107.597 87 0.0110548 0.0092891 2024-09-10T18:56:51.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:51.441 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:51.441 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:51.542 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:51.573 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:51.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:51.575 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:51.675 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:51.733 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:51.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:51.734 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:51.835 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:51.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:51.870 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:51.870 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:51.971 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:52.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:52.001 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:52.002 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:52.102 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:52.132 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:52.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:52.133 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:52.234 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:52.273 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:52.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:52.274 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:52.377 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:52.396 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 64 16 109991 109975 107.373 93.25 0.0121992 0.00930872 2024-09-10T18:56:52.417 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:52.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:52.421 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:52.522 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:52.553 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:52.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:52.555 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:52.655 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:52.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:52.694 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:52.694 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:52.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:52 smithi018 ceph-mon[40196]: pgmap v525: 168 pgs: 168 active+clean; 6.2 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 639 B/s rd, 64 MiB/s wr, 1.03k op/s 2024-09-10T18:56:52.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:52 smithi018 ceph-mon[29336]: pgmap v525: 168 pgs: 168 active+clean; 6.2 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 639 B/s rd, 64 MiB/s wr, 1.03k op/s 2024-09-10T18:56:52.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:52 smithi103 ceph-mon[33163]: pgmap v525: 168 pgs: 168 active+clean; 6.2 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 639 B/s rd, 64 MiB/s wr, 1.03k op/s 2024-09-10T18:56:52.779 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T18:56:52.795 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:52.871 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:52.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:52.872 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:52.972 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:53.020 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:53.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:53.021 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:53.121 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:53.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:53.166 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:53.167 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:53.267 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:53.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:53.309 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:53.310 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:53.397 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 65 16 111428 111412 107.103 89.8125 0.0121865 0.00933255 2024-09-10T18:56:53.411 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:53.445 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:53.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:53.446 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:53.547 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:53.585 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:53.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:53.586 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:53.687 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:53.720 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:53.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:53.721 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:53.822 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:53.859 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:53.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:53.860 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:53.961 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:53.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:54.000 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:54.000 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:54.101 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:54.130 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:54.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:54.131 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:54.231 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:54.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:54.254 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:54.254 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:54.355 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:54.391 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:54.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:54.392 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:54.396 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 66 16 112692 112676 106.677 79 0.00962353 0.00936999 2024-09-10T18:56:54.493 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:54.519 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:54.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:54.520 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:54.621 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:54.660 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:54.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:54.661 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:54.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:54 smithi018 ceph-mon[40196]: pgmap v526: 168 pgs: 168 active+clean; 6.6 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1.2 KiB/s rd, 98 MiB/s wr, 1.57k op/s 2024-09-10T18:56:54.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:54 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:56:54.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:54 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:56:54.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:54 smithi018 ceph-mon[29336]: pgmap v526: 168 pgs: 168 active+clean; 6.6 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1.2 KiB/s rd, 98 MiB/s wr, 1.57k op/s 2024-09-10T18:56:54.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:54 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:56:54.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:54 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:56:54.758 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-10T18:56:54.762 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:54.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:54 smithi103 ceph-mon[33163]: pgmap v526: 168 pgs: 168 active+clean; 6.6 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1.2 KiB/s rd, 98 MiB/s wr, 1.57k op/s 2024-09-10T18:56:54.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:54 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:56:54.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:54 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:56:54.793 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:54.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:54.794 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:54.895 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:54.939 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:54.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:54.940 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:55.041 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:55.091 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:55.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:55.092 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:55.193 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:55.214 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:55.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:55.216 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:55.316 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:55.343 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:55.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:55.344 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:55.397 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 67 16 113758 113742 106.079 66.625 0.00441334 0.00942242 2024-09-10T18:56:55.445 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:55.485 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:55.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:55.487 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:55.587 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:55.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:55.632 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:55.633 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:55.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:55 smithi018 ceph-mon[29336]: osdmap e265: 8 total, 8 up, 7 in 2024-09-10T18:56:55.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:56:55.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:55 smithi018 ceph-mon[40196]: osdmap e265: 8 total, 8 up, 7 in 2024-09-10T18:56:55.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:56:55.734 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:55.756 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:55.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:55.757 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:55.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:55 smithi103 ceph-mon[33163]: osdmap e265: 8 total, 8 up, 7 in 2024-09-10T18:56:55.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:56:55.858 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:55.888 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:55.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:55.889 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:55.989 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:56.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:56.032 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:56.032 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:56.133 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:56.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:56.169 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:56.169 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:56.270 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:56.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:56.304 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:56.305 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:56.397 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 68 16 115182 115166 105.828 89 0.0055439 0.00944481 2024-09-10T18:56:56.405 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:56.438 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:56.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:56.439 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:56.471 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:56:56.471 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:56:56.540 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:56.593 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:56.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:56.594 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:56.619 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:56 smithi018 ceph-mon[40196]: pgmap v528: 168 pgs: 168 active+clean; 6.7 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 741 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-10T18:56:56.619 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:56 smithi018 ceph-mon[29336]: pgmap v528: 168 pgs: 168 active+clean; 6.7 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 741 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-10T18:56:56.695 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:56.716 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:56 smithi103 ceph-mon[33163]: pgmap v528: 168 pgs: 168 active+clean; 6.7 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 741 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-10T18:56:56.717 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:56:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=sqlstore.transactions t=2024-09-10T18:56:56.43793008Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-10T18:56:56.717 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 18:56:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=cleanup t=2024-09-10T18:56:56.43925393Z level=info msg="Completed cleanup jobs" duration=12.616147ms 2024-09-10T18:56:56.734 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:56.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:56.735 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:56.836 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:56.871 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:56.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:56.872 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:56.973 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:57.021 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:57.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:57.022 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:57.022 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:56:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:56:57.123 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:57.160 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:57.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:57.162 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:57.262 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:57.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:57.313 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:57.314 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:57.397 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 69 16 116577 116561 105.558 87.1875 0.00777357 0.00946877 2024-09-10T18:56:57.415 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:57.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:57 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 4]}]: dispatch 2024-09-10T18:56:57.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:57 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 4]}]: dispatch 2024-09-10T18:56:57.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:57 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 4]}]: dispatch 2024-09-10T18:56:57.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:57 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 4]}]: dispatch 2024-09-10T18:56:57.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:57.457 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:57.457 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:57.558 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:57.591 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:57.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:57.592 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:57.693 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:57.730 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:57.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:57.731 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:57.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:57 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 4]}]: dispatch 2024-09-10T18:56:57.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:57 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 4]}]: dispatch 2024-09-10T18:56:57.832 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:57.868 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:57.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:57.869 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:57.969 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:58.001 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:58.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:58.002 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:58.103 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:56:58.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:58.135 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:58.135 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:56:58.236 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:58.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:58.271 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:58.272 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:58.372 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:58.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:58.397 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:58.398 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:56:58.398 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 70 16 118079 118063 105.391 93.875 0.00337959 0.00948175 2024-09-10T18:56:58.499 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:58.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:58.537 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:58.537 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:58.638 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:58.677 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:58.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:58.679 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:58.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:58 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:58.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:58 smithi018 ceph-mon[40196]: pgmap v529: 168 pgs: 168 active+clean; 7.0 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 1.2 KiB/s rd, 90 MiB/s wr, 1.44k op/s 2024-09-10T18:56:58.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:58 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 4]}]': finished 2024-09-10T18:56:58.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:58 smithi018 ceph-mon[40196]: osdmap e266: 8 total, 8 up, 7 in 2024-09-10T18:56:58.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:58 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:58.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:58 smithi018 ceph-mon[29336]: pgmap v529: 168 pgs: 168 active+clean; 7.0 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 1.2 KiB/s rd, 90 MiB/s wr, 1.44k op/s 2024-09-10T18:56:58.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:58 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 4]}]': finished 2024-09-10T18:56:58.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:58 smithi018 ceph-mon[29336]: osdmap e266: 8 total, 8 up, 7 in 2024-09-10T18:56:58.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:58 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:58.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:58 smithi103 ceph-mon[33163]: pgmap v529: 168 pgs: 168 active+clean; 7.0 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 1.2 KiB/s rd, 90 MiB/s wr, 1.44k op/s 2024-09-10T18:56:58.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:58 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 4]}]': finished 2024-09-10T18:56:58.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:58 smithi103 ceph-mon[33163]: osdmap e266: 8 total, 8 up, 7 in 2024-09-10T18:56:58.780 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:56:58.818 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:58.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:58.819 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:56:58.920 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:56:58.958 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:58.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:58.959 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:56:59.060 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:59.099 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:59.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:59.100 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:59.201 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:59.251 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:59.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:59.252 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:59.352 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:56:59.387 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:59.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:59.388 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:56:59.397 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 71 15 119460 119445 105.123 86.375 0.00556431 0.00950849 2024-09-10T18:56:59.490 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:56:59.536 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:59.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:59.537 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:56:59.558 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:59 smithi018 ceph-mon[29336]: osdmap e267: 8 total, 8 up, 7 in 2024-09-10T18:56:59.559 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:59 smithi018 ceph-mon[29336]: osdmap e268: 8 total, 8 up, 7 in 2024-09-10T18:56:59.559 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:59 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:56:59.559 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:56:59 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:56:59.559 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:59 smithi018 ceph-mon[40196]: osdmap e267: 8 total, 8 up, 7 in 2024-09-10T18:56:59.559 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:59 smithi018 ceph-mon[40196]: osdmap e268: 8 total, 8 up, 7 in 2024-09-10T18:56:59.559 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:59 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:56:59.559 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:56:59 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:56:59.638 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:56:59.681 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:59.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:59.683 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:56:59.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:59 smithi103 ceph-mon[33163]: osdmap e267: 8 total, 8 up, 7 in 2024-09-10T18:56:59.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:59 smithi103 ceph-mon[33163]: osdmap e268: 8 total, 8 up, 7 in 2024-09-10T18:56:59.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:59 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:56:59.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:56:59 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:56:59.783 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:56:59.823 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:56:59.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:59.824 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:56:59.925 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:56:59.959 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:56:59.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:56:59.961 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:00.061 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:00.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:00.094 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:00.094 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:00.195 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:00.224 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:00.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:00.225 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:00.326 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:00.342 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:57:00.397 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 72 16 120762 120746 104.792 81.3125 0.0138795 0.00953774 2024-09-10T18:57:00.398 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:00.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:00.401 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:00.502 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:00.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:00 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:57:00.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:00 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:57:00.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:00 smithi103 ceph-mon[33163]: pgmap v533: 168 pgs: 168 active+clean; 7.1 GiB data, 25 GiB used, 600 GiB / 626 GiB avail; 1.0 KiB/s rd, 84 MiB/s wr, 1.35k op/s 2024-09-10T18:57:00.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:00 smithi103 ceph-mon[33163]: osdmap e269: 8 total, 8 up, 7 in 2024-09-10T18:57:00.523 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:57:00 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:57:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:57:00.527 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:00.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:00.528 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:00.609 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:00 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:57:00.609 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:00 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:57:00.609 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:00 smithi018 ceph-mon[40196]: pgmap v533: 168 pgs: 168 active+clean; 7.1 GiB data, 25 GiB used, 600 GiB / 626 GiB avail; 1.0 KiB/s rd, 84 MiB/s wr, 1.35k op/s 2024-09-10T18:57:00.609 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:00 smithi018 ceph-mon[40196]: osdmap e269: 8 total, 8 up, 7 in 2024-09-10T18:57:00.609 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:00 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:57:00.609 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:00 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:57:00.609 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:00 smithi018 ceph-mon[29336]: pgmap v533: 168 pgs: 168 active+clean; 7.1 GiB data, 25 GiB used, 600 GiB / 626 GiB avail; 1.0 KiB/s rd, 84 MiB/s wr, 1.35k op/s 2024-09-10T18:57:00.610 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:00 smithi018 ceph-mon[29336]: osdmap e269: 8 total, 8 up, 7 in 2024-09-10T18:57:00.629 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:00.656 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:00.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:00.657 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:00.758 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:00.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:00.793 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:00.793 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:00.894 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:00.939 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:00.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:00.940 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:01.041 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:01.079 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:01.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:01.081 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:01.182 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:01.213 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:01.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:01.214 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:01.223 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:01 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mon-a[29332]: 2024-09-10T18:57:01.010+0000 7f4eb0f2a640 -1 mon.a@0(leader).osd e269 definitely_dead 0 2024-09-10T18:57:01.223 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:57:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:57:00] "GET /metrics HTTP/1.1" 200 224249 "" "Prometheus/2.43.0" 2024-09-10T18:57:01.315 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:01.346 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:01.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:01.347 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:01.397 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 73 16 121917 121901 104.345 72.1875 0.0147957 0.00957882 2024-09-10T18:57:01.424 INFO:teuthology.orchestra.run.smithi018.stderr:marked out osd.7. 2024-09-10T18:57:01.448 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:01.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:01.471 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:01.471 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:01.474 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:01 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1059328719' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["7"]}]: dispatch 2024-09-10T18:57:01.474 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:01 smithi018 ceph-mon[29336]: Client client.admin marked osd.7 out, while it was still marked up 2024-09-10T18:57:01.474 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:01 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3827214299' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:57:01.475 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:01 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1059328719' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["7"]}]: dispatch 2024-09-10T18:57:01.475 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:01 smithi018 ceph-mon[40196]: Client client.admin marked osd.7 out, while it was still marked up 2024-09-10T18:57:01.475 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:01 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3827214299' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:57:01.572 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:01.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:01.597 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:01.597 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:01.698 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:01.720 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:01.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:01.722 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:01.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:01 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1059328719' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["7"]}]: dispatch 2024-09-10T18:57:01.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:01 smithi103 ceph-mon[33163]: Client client.admin marked osd.7 out, while it was still marked up 2024-09-10T18:57:01.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:01 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3827214299' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:57:01.823 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:01.844 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:01.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:01.845 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:01.946 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:01.968 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:01.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:01.969 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:02.055 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:57:02.055 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:57:02.055 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:57:02.055 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:57:02.055 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:57:02.055 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:57:02.056 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:57:02.056 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:57:02.056 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:57:02.056 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:57:02.056 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:57:02.056 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:57:02.056 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:57:02.056 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:57:02.057 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:57:02.057 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:57:02.069 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:02.092 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:02.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:02.093 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:02.194 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:02.216 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:02.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:02.217 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:02.320 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:02.343 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:02.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:02.344 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:02.398 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 74 16 121946 121930 102.96 1.8125 0.0546025 0.00958161 2024-09-10T18:57:02.426 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is unset 2024-09-10T18:57:02.445 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:02.476 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:02.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:02.477 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:02.578 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:02.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:02.601 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:02.601 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:02.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:02 smithi018 ceph-mon[29336]: pgmap v535: 168 pgs: 168 active+clean; 7.1 GiB data, 25 GiB used, 600 GiB / 626 GiB avail; 18 MiB/s wr, 295 op/s 2024-09-10T18:57:02.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:02 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:57:02.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:02 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:57:02.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:02 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1059328719' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["7"]}]': finished 2024-09-10T18:57:02.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:02 smithi018 ceph-mon[29336]: osdmap e270: 8 total, 8 up, 6 in 2024-09-10T18:57:02.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:02 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2171998636' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:57:02.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:02 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:57:02.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:02 smithi018 ceph-mon[40196]: pgmap v535: 168 pgs: 168 active+clean; 7.1 GiB data, 25 GiB used, 600 GiB / 626 GiB avail; 18 MiB/s wr, 295 op/s 2024-09-10T18:57:02.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:02 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:57:02.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:02 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:57:02.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:02 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1059328719' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["7"]}]': finished 2024-09-10T18:57:02.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:02 smithi018 ceph-mon[40196]: osdmap e270: 8 total, 8 up, 6 in 2024-09-10T18:57:02.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:02 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2171998636' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:57:02.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:02 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:57:02.701 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:02.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:02.724 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:02.725 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:02.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:02 smithi103 ceph-mon[33163]: pgmap v535: 168 pgs: 168 active+clean; 7.1 GiB data, 25 GiB used, 600 GiB / 626 GiB avail; 18 MiB/s wr, 295 op/s 2024-09-10T18:57:02.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:02 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:57:02.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:02 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:57:02.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:02 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1059328719' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["7"]}]': finished 2024-09-10T18:57:02.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:02 smithi103 ceph-mon[33163]: osdmap e270: 8 total, 8 up, 6 in 2024-09-10T18:57:02.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:02 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2171998636' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:57:02.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:02 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:57:02.825 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:02.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:02.848 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:02.848 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:02.949 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:02.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:02.972 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:02.972 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:03.073 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:03.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:03.096 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:03.097 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:03.197 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:03.218 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:03.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:03.220 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:03.320 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:03.344 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:03.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:03.345 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:03.398 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 75 16 121946 121930 101.587 0 - 0.00958161 2024-09-10T18:57:03.449 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:03.475 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:03.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:03.476 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:03.577 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:03.608 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:03.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:03.609 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:03.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:03 smithi018 ceph-mon[40196]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:57:03.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:03 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:57:03.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:03 smithi018 ceph-mon[40196]: osdmap e271: 8 total, 8 up, 6 in 2024-09-10T18:57:03.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:03 smithi018 ceph-mon[40196]: osdmap e272: 8 total, 8 up, 6 in 2024-09-10T18:57:03.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:03 smithi018 ceph-mon[29336]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:57:03.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:03 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:57:03.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:03 smithi018 ceph-mon[29336]: osdmap e271: 8 total, 8 up, 6 in 2024-09-10T18:57:03.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:03 smithi018 ceph-mon[29336]: osdmap e272: 8 total, 8 up, 6 in 2024-09-10T18:57:03.709 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:03.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:03.743 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:03.743 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:03.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:03 smithi103 ceph-mon[33163]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:57:03.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:03 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:57:03.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:03 smithi103 ceph-mon[33163]: osdmap e271: 8 total, 8 up, 6 in 2024-09-10T18:57:03.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:03 smithi103 ceph-mon[33163]: osdmap e272: 8 total, 8 up, 6 in 2024-09-10T18:57:03.844 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:03.891 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:03.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:03.892 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:03.993 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:04.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:04.028 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:04.028 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:04.129 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:04.179 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:04.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:04.181 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:04.281 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:04.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:04.331 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:04.332 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:04.398 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 76 16 122830 122814 100.977 27.625 0.0182756 0.0098988 2024-09-10T18:57:04.433 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:04.464 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:04.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:04.466 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:04.567 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:04.568 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[29336]: 6.8 scrub starts 2024-09-10T18:57:04.568 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[29336]: 6.8 scrub ok 2024-09-10T18:57:04.568 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[29336]: pgmap v538: 168 pgs: 40 peering, 128 active+clean; 7.4 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 5.4 KiB/s rd, 74 MiB/s wr, 1.18k op/s; 23 KiB/s, 0 objects/s recovering 2024-09-10T18:57:04.568 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[29336]: 6.0 scrub starts 2024-09-10T18:57:04.569 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[29336]: 6.0 scrub ok 2024-09-10T18:57:04.569 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[29336]: 9.a starting backfill to osd.6 from (0'0,0'0] MAX to 269'7717 2024-09-10T18:57:04.569 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[29336]: 9.9 starting backfill to osd.3 from (0'0,0'0] MAX to 269'7600 2024-09-10T18:57:04.569 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[29336]: 9.e starting backfill to osd.0 from (0'0,0'0] MAX to 269'7676 2024-09-10T18:57:04.569 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[29336]: 9.d starting backfill to osd.4 from (0'0,0'0] MAX to 269'7451 2024-09-10T18:57:04.569 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[29336]: 9.5 starting backfill to osd.3 from (0'0,0'0] MAX to 269'7522 2024-09-10T18:57:04.569 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[29336]: 9.c starting backfill to osd.0 from (0'0,0'0] MAX to 269'7661 2024-09-10T18:57:04.569 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[29336]: 9.3 starting backfill to osd.5 from (0'0,0'0] MAX to 269'7687 2024-09-10T18:57:04.569 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[29336]: 9.7 starting backfill to osd.1 from (0'0,0'0] MAX to 269'7462 2024-09-10T18:57:04.569 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[29336]: 9.1 starting backfill to osd.4 from (0'0,0'0] MAX to 269'7594 2024-09-10T18:57:04.569 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[29336]: Health check failed: Reduced data availability: 5 pgs inactive, 8 pgs peering (PG_AVAILABILITY) 2024-09-10T18:57:04.570 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[29336]: 9.8 starting backfill to osd.1 from (0'0,0'0] MAX to 269'7708 2024-09-10T18:57:04.570 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[29336]: 9.2 starting backfill to osd.6 from (0'0,0'0] MAX to 269'7573 2024-09-10T18:57:04.570 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[29336]: 6.1c scrub starts 2024-09-10T18:57:04.570 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:57:04.570 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:57:04.570 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[40196]: 6.8 scrub starts 2024-09-10T18:57:04.570 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[40196]: 6.8 scrub ok 2024-09-10T18:57:04.570 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[40196]: pgmap v538: 168 pgs: 40 peering, 128 active+clean; 7.4 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 5.4 KiB/s rd, 74 MiB/s wr, 1.18k op/s; 23 KiB/s, 0 objects/s recovering 2024-09-10T18:57:04.570 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[40196]: 6.0 scrub starts 2024-09-10T18:57:04.571 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[40196]: 6.0 scrub ok 2024-09-10T18:57:04.571 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[40196]: 9.a starting backfill to osd.6 from (0'0,0'0] MAX to 269'7717 2024-09-10T18:57:04.571 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[40196]: 9.9 starting backfill to osd.3 from (0'0,0'0] MAX to 269'7600 2024-09-10T18:57:04.571 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[40196]: 9.e starting backfill to osd.0 from (0'0,0'0] MAX to 269'7676 2024-09-10T18:57:04.571 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[40196]: 9.d starting backfill to osd.4 from (0'0,0'0] MAX to 269'7451 2024-09-10T18:57:04.571 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[40196]: 9.5 starting backfill to osd.3 from (0'0,0'0] MAX to 269'7522 2024-09-10T18:57:04.571 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[40196]: 9.c starting backfill to osd.0 from (0'0,0'0] MAX to 269'7661 2024-09-10T18:57:04.571 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[40196]: 9.3 starting backfill to osd.5 from (0'0,0'0] MAX to 269'7687 2024-09-10T18:57:04.571 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[40196]: 9.7 starting backfill to osd.1 from (0'0,0'0] MAX to 269'7462 2024-09-10T18:57:04.571 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[40196]: 9.1 starting backfill to osd.4 from (0'0,0'0] MAX to 269'7594 2024-09-10T18:57:04.571 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[40196]: Health check failed: Reduced data availability: 5 pgs inactive, 8 pgs peering (PG_AVAILABILITY) 2024-09-10T18:57:04.571 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[40196]: 9.8 starting backfill to osd.1 from (0'0,0'0] MAX to 269'7708 2024-09-10T18:57:04.571 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[40196]: 9.2 starting backfill to osd.6 from (0'0,0'0] MAX to 269'7573 2024-09-10T18:57:04.572 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[40196]: 6.1c scrub starts 2024-09-10T18:57:04.572 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:57:04.572 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:04 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:57:04.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:04.599 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:04.600 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:04.701 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:04.750 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:04.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:04.751 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:04.767 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T18:57:04.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:04 smithi103 ceph-mon[33163]: 6.8 scrub starts 2024-09-10T18:57:04.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:04 smithi103 ceph-mon[33163]: 6.8 scrub ok 2024-09-10T18:57:04.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:04 smithi103 ceph-mon[33163]: pgmap v538: 168 pgs: 40 peering, 128 active+clean; 7.4 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 5.4 KiB/s rd, 74 MiB/s wr, 1.18k op/s; 23 KiB/s, 0 objects/s recovering 2024-09-10T18:57:04.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:04 smithi103 ceph-mon[33163]: 6.0 scrub starts 2024-09-10T18:57:04.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:04 smithi103 ceph-mon[33163]: 6.0 scrub ok 2024-09-10T18:57:04.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:04 smithi103 ceph-mon[33163]: 9.a starting backfill to osd.6 from (0'0,0'0] MAX to 269'7717 2024-09-10T18:57:04.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:04 smithi103 ceph-mon[33163]: 9.9 starting backfill to osd.3 from (0'0,0'0] MAX to 269'7600 2024-09-10T18:57:04.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:04 smithi103 ceph-mon[33163]: 9.e starting backfill to osd.0 from (0'0,0'0] MAX to 269'7676 2024-09-10T18:57:04.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:04 smithi103 ceph-mon[33163]: 9.d starting backfill to osd.4 from (0'0,0'0] MAX to 269'7451 2024-09-10T18:57:04.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:04 smithi103 ceph-mon[33163]: 9.5 starting backfill to osd.3 from (0'0,0'0] MAX to 269'7522 2024-09-10T18:57:04.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:04 smithi103 ceph-mon[33163]: 9.c starting backfill to osd.0 from (0'0,0'0] MAX to 269'7661 2024-09-10T18:57:04.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:04 smithi103 ceph-mon[33163]: 9.3 starting backfill to osd.5 from (0'0,0'0] MAX to 269'7687 2024-09-10T18:57:04.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:04 smithi103 ceph-mon[33163]: 9.7 starting backfill to osd.1 from (0'0,0'0] MAX to 269'7462 2024-09-10T18:57:04.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:04 smithi103 ceph-mon[33163]: 9.1 starting backfill to osd.4 from (0'0,0'0] MAX to 269'7594 2024-09-10T18:57:04.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:04 smithi103 ceph-mon[33163]: Health check failed: Reduced data availability: 5 pgs inactive, 8 pgs peering (PG_AVAILABILITY) 2024-09-10T18:57:04.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:04 smithi103 ceph-mon[33163]: 9.8 starting backfill to osd.1 from (0'0,0'0] MAX to 269'7708 2024-09-10T18:57:04.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:04 smithi103 ceph-mon[33163]: 9.2 starting backfill to osd.6 from (0'0,0'0] MAX to 269'7573 2024-09-10T18:57:04.775 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:04 smithi103 ceph-mon[33163]: 6.1c scrub starts 2024-09-10T18:57:04.775 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:04 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:57:04.775 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:04 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:57:04.851 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:04.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:04.910 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:04.910 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:05.010 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:05.040 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:05.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:05.041 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:05.142 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:05.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:05.188 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:05.189 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:05.289 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:05.337 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:05.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:05.338 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:05.398 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 77 16 123789 123773 100.444 59.9375 0.0140507 0.00995111 2024-09-10T18:57:05.440 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:05.471 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:05.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:05.472 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:05.573 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:05.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:05.621 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:05.621 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:05.722 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:05.759 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:05.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:05.760 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:05.861 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:05.894 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:05.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:05.895 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:05.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:05 smithi018 ceph-mon[29336]: osdmap e273: 8 total, 8 up, 6 in 2024-09-10T18:57:05.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:05 smithi018 ceph-mon[29336]: pgmap v541: 168 pgs: 2 activating+remapped, 40 peering, 126 active+clean; 7.4 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 11 KiB/s rd, 96 MiB/s wr, 1.54k op/s; 15122/366369 objects misplaced (4.128%); 52 KiB/s, 5 objects/s recovering 2024-09-10T18:57:05.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:05 smithi018 ceph-mon[40196]: osdmap e273: 8 total, 8 up, 6 in 2024-09-10T18:57:05.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:05 smithi018 ceph-mon[40196]: pgmap v541: 168 pgs: 2 activating+remapped, 40 peering, 126 active+clean; 7.4 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 11 KiB/s rd, 96 MiB/s wr, 1.54k op/s; 15122/366369 objects misplaced (4.128%); 52 KiB/s, 5 objects/s recovering 2024-09-10T18:57:05.996 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:06.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:05 smithi103 ceph-mon[33163]: osdmap e273: 8 total, 8 up, 6 in 2024-09-10T18:57:06.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:05 smithi103 ceph-mon[33163]: pgmap v541: 168 pgs: 2 activating+remapped, 40 peering, 126 active+clean; 7.4 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 11 KiB/s rd, 96 MiB/s wr, 1.54k op/s; 15122/366369 objects misplaced (4.128%); 52 KiB/s, 5 objects/s recovering 2024-09-10T18:57:06.045 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:06.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:06.046 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:06.147 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:06.171 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:06.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:06.172 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:06.273 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:06.307 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:06.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:06.308 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:06.398 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 78 16 124817 124801 99.9798 64.25 0.00747052 0.00999754 2024-09-10T18:57:06.409 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:06.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:06.433 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:06.433 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:06.534 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:06.568 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:06.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:06.569 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:06.669 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:06.714 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:06.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:06.715 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:06.816 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:06.839 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-10T18:57:06.856 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:06.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:06.857 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:06.958 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:06.988 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:06.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:06.989 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:07.022 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:57:06 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:57:07.040 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T18:57:07.041 INFO:teuthology.orchestra.run.smithi018.stdout: "ops": [], 2024-09-10T18:57:07.041 INFO:teuthology.orchestra.run.smithi018.stdout: "complaint_time": 30, 2024-09-10T18:57:07.041 INFO:teuthology.orchestra.run.smithi018.stdout: "num_blocked_ops": 0 2024-09-10T18:57:07.041 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T18:57:07.089 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:07.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:07.137 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:07.137 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:07.238 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:07.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:07.272 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:07.272 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:07.373 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:07.398 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 79 16 125742 125726 99.446 57.8125 0.0101084 0.0100509 2024-09-10T18:57:07.403 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:07.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:07.404 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:07.505 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:07.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:07.549 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:07.549 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:07.650 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:07.695 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:07.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:07.696 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:07.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:07 smithi018 ceph-mon[40196]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-10T18:57:07.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:07 smithi018 ceph-mon[40196]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 5 pgs inactive, 8 pgs peering) 2024-09-10T18:57:07.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:07 smithi018 ceph-mon[29336]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-10T18:57:07.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:07 smithi018 ceph-mon[29336]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 5 pgs inactive, 8 pgs peering) 2024-09-10T18:57:07.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:07 smithi103 ceph-mon[33163]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-10T18:57:07.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:07 smithi103 ceph-mon[33163]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 5 pgs inactive, 8 pgs peering) 2024-09-10T18:57:07.797 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:07.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:07.842 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:07.842 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:07.889 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-10T18:57:07.944 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:07.963 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:57:07.969 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:07.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:07.970 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:07.976 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:57:08.070 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:08.098 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:08.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:08.099 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:08.200 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:08.253 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:08.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:08.254 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:08.355 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:08.375 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:08 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:08.375 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:08 smithi018 ceph-mon[29336]: pgmap v542: 168 pgs: 6 active+remapped+backfilling, 2 active+remapped+backfill_wait, 2 activating+remapped, 1 active+remapped+backfill_toofull, 157 active+clean; 7.6 GiB data, 22 GiB used, 515 GiB / 536 GiB avail; 9.4 KiB/s rd, 78 MiB/s wr, 1.26k op/s; 82902/373767 objects misplaced (22.180%); 26 MiB/s, 427 objects/s recovering 2024-09-10T18:57:08.399 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:2024-09-10T18:57:08.398335+0000 min lat: 0.00128005 max lat: 2.17684 avg lat: 0.0100983 2024-09-10T18:57:08.399 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T18:57:08.399 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 80 16 126739 126723 98.9818 62.3125 0.012123 0.0100983 2024-09-10T18:57:08.407 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:08.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:08.408 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:08.509 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:08.539 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:08.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:08.541 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:08.641 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:08.674 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:08.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:08.675 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:08.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:08 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:08.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:08 smithi018 ceph-mon[40196]: pgmap v542: 168 pgs: 6 active+remapped+backfilling, 2 active+remapped+backfill_wait, 2 activating+remapped, 1 active+remapped+backfill_toofull, 157 active+clean; 7.6 GiB data, 22 GiB used, 515 GiB / 536 GiB avail; 9.4 KiB/s rd, 78 MiB/s wr, 1.26k op/s; 82902/373767 objects misplaced (22.180%); 26 MiB/s, 427 objects/s recovering 2024-09-10T18:57:08.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:08 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:08.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:08 smithi103 ceph-mon[33163]: pgmap v542: 168 pgs: 6 active+remapped+backfilling, 2 active+remapped+backfill_wait, 2 activating+remapped, 1 active+remapped+backfill_toofull, 157 active+clean; 7.6 GiB data, 22 GiB used, 515 GiB / 536 GiB avail; 9.4 KiB/s rd, 78 MiB/s wr, 1.26k op/s; 82902/373767 objects misplaced (22.180%); 26 MiB/s, 427 objects/s recovering 2024-09-10T18:57:08.777 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:08.809 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:08.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:08.810 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:08.866 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6] out_osds: [2, 7] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 6, 7, 0] 2024-09-10T18:57:08.866 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T18:57:08.867 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_2 2024-09-10T18:57:08.867 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T18:57:08.911 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:08.954 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:08.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:08.955 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:09.056 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:09.116 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:09.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:09.117 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:09.218 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:09.269 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:09.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:09.271 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:09.371 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:09.399 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 81 16 127768 127752 98.5533 64.3125 0.0164765 0.0101423 2024-09-10T18:57:09.420 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:09.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:09.421 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:09.522 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:09.548 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:09.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:09.549 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:09.650 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:09.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:09.681 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:09.681 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:09.782 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:09.824 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:09.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:09.825 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:09.926 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:09.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:09 smithi018 ceph-mon[29336]: pgmap v543: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 7.6 GiB data, 22 GiB used, 515 GiB / 536 GiB avail; 4.4 KiB/s rd, 37 MiB/s wr, 592 op/s; 82481/376440 objects misplaced (21.911%); 31 MiB/s, 512 objects/s recovering 2024-09-10T18:57:09.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:09 smithi018 ceph-mon[40196]: pgmap v543: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 7.6 GiB data, 22 GiB used, 515 GiB / 536 GiB avail; 4.4 KiB/s rd, 37 MiB/s wr, 592 op/s; 82481/376440 objects misplaced (21.911%); 31 MiB/s, 512 objects/s recovering 2024-09-10T18:57:09.969 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:09.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:09.970 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:10.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:09 smithi103 ceph-mon[33163]: pgmap v543: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 7.6 GiB data, 22 GiB used, 515 GiB / 536 GiB avail; 4.4 KiB/s rd, 37 MiB/s wr, 592 op/s; 82481/376440 objects misplaced (21.911%); 31 MiB/s, 512 objects/s recovering 2024-09-10T18:57:10.071 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:10.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:10.123 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:10.123 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:10.224 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:10.261 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:10.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:10.262 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:10.363 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:10.399 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 82 16 128840 128824 98.1684 67 0.018418 0.0101819 2024-09-10T18:57:10.403 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:10.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:10.405 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:10.468 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:57:10.505 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:10.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:57:10 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:57:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:57:10.535 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:10.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:10.536 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:10.637 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:10.666 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:10.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:10.667 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:10.768 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:10.839 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:10.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:10.840 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:10.941 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:10.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:10.985 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:10.985 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:11.086 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:11.135 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:11.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:11.136 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:11.235 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:11 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:57:11.235 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:11 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:57:11.235 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:11 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:57:11.235 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:11 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:57:11.235 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:57:10 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:57:10] "GET /metrics HTTP/1.1" 200 224417 "" "Prometheus/2.43.0" 2024-09-10T18:57:11.237 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:11.278 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:11.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:11.279 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:11.380 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:11.399 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 83 16 129769 129753 97.6852 58.0625 0.0237198 0.0102325 2024-09-10T18:57:11.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:11.412 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:11.412 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:11.512 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:11.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:11 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:57:11.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:11 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:57:11.554 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:11.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:11.555 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:11.656 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:11.684 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:11.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:11.685 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:11.786 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:11.829 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:11.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:11.830 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:11.930 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:11.977 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:11.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:11.978 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:12.022 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:57:12.079 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:12.107 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is unset 2024-09-10T18:57:12.110 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:12.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:12.111 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:12.212 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:12.257 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:12.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:12.258 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:12.289 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:12 smithi018 ceph-mon[40196]: pgmap v544: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 7.7 GiB data, 22 GiB used, 515 GiB / 536 GiB avail; 3.7 KiB/s rd, 34 MiB/s wr, 546 op/s; 81878/377385 objects misplaced (21.696%); 34 MiB/s, 548 objects/s recovering 2024-09-10T18:57:12.289 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:12 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1026771092' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:57:12.290 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:12 smithi018 ceph-mon[29336]: pgmap v544: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 7.7 GiB data, 22 GiB used, 515 GiB / 536 GiB avail; 3.7 KiB/s rd, 34 MiB/s wr, 546 op/s; 81878/377385 objects misplaced (21.696%); 34 MiB/s, 548 objects/s recovering 2024-09-10T18:57:12.290 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:12 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1026771092' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:57:12.360 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:12.389 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:12.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:12.390 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:12.399 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 84 16 130727 130711 97.2351 59.875 0.0134775 0.0102794 2024-09-10T18:57:12.492 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:12.521 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:12.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:12.522 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:12.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:12 smithi103 ceph-mon[33163]: pgmap v544: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 7.7 GiB data, 22 GiB used, 515 GiB / 536 GiB avail; 3.7 KiB/s rd, 34 MiB/s wr, 546 op/s; 81878/377385 objects misplaced (21.696%); 34 MiB/s, 548 objects/s recovering 2024-09-10T18:57:12.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:12 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1026771092' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:57:12.623 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:12.654 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:12.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:12.655 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:12.756 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:12.802 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:12.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:12.803 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:12.903 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:12.953 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:12.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:12.954 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:13.055 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:13.087 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:13.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:13.088 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:13.189 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:13.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:13.223 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:13.223 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:13.324 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:13.360 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:13.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:13.361 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:13.399 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 85 16 131786 131770 96.8698 66.1875 0.00709062 0.0103188 2024-09-10T18:57:13.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:13 smithi018 ceph-mon[29336]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:57:13.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:13 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1026771092' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:57:13.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:13 smithi018 ceph-mon[29336]: osdmap e274: 8 total, 8 up, 6 in 2024-09-10T18:57:13.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:13 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1716038481' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:57:13.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:13 smithi018 ceph-mon[40196]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:57:13.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:13 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1026771092' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:57:13.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:13 smithi018 ceph-mon[40196]: osdmap e274: 8 total, 8 up, 6 in 2024-09-10T18:57:13.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:13 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1716038481' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:57:13.462 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:13.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:13.495 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:13.495 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:13.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:13 smithi103 ceph-mon[33163]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:57:13.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:13 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1026771092' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:57:13.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:13 smithi103 ceph-mon[33163]: osdmap e274: 8 total, 8 up, 6 in 2024-09-10T18:57:13.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:13 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1716038481' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:57:13.596 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:13.619 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:13.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:13.620 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:13.721 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:13.762 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:13.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:13.763 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:13.865 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:13.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:13.886 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:13.887 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:13.987 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:14.019 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:14.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:14.020 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:14.121 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:14.162 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:14.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:14.163 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:14.264 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:14.300 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:14.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:14.302 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:14.399 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 86 16 132626 132610 96.3539 52.5 0.0119692 0.0103739 2024-09-10T18:57:14.403 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:14.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:14 smithi103 ceph-mon[33163]: pgmap v546: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 7.9 GiB data, 23 GiB used, 514 GiB / 536 GiB avail; 1.9 KiB/s rd, 54 MiB/s wr, 861 op/s; 81102/388572 objects misplaced (20.872%); 54 MiB/s, 872 objects/s recovering 2024-09-10T18:57:14.441 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:14.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:14.442 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:14.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:14 smithi018 ceph-mon[40196]: pgmap v546: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 7.9 GiB data, 23 GiB used, 514 GiB / 536 GiB avail; 1.9 KiB/s rd, 54 MiB/s wr, 861 op/s; 81102/388572 objects misplaced (20.872%); 54 MiB/s, 872 objects/s recovering 2024-09-10T18:57:14.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:14 smithi018 ceph-mon[29336]: pgmap v546: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 7.9 GiB data, 23 GiB used, 514 GiB / 536 GiB avail; 1.9 KiB/s rd, 54 MiB/s wr, 861 op/s; 81102/388572 objects misplaced (20.872%); 54 MiB/s, 872 objects/s recovering 2024-09-10T18:57:14.543 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:14.586 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:14.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:14.587 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:14.688 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:14.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:14.718 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:14.718 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:14.818 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:14.860 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:14.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:14.861 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:14.962 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:14.993 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:14.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:14.994 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:15.095 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:15.116 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:15.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:15.117 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:15.218 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:15.253 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:15.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:15.255 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:15.355 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:15.379 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:15.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:15.380 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:15.399 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 87 16 133618 133602 95.959 62 0.00679185 0.0104129 2024-09-10T18:57:15.481 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:15.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:15.525 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:15.525 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:15.626 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:15.669 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:15.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:15.670 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:15.711 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:57:15.711 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:57:15.711 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:57:15.711 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:57:15.711 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:57:15.712 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:57:15.712 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:57:15.712 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:57:15.712 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:57:15.712 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:57:15.712 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:57:15.712 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:57:15.712 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:57:15.712 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:57:15.712 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:57:15.712 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:57:15.771 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:15.803 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:15.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:15.804 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:15.905 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:15.946 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:15.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:15.948 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:16.049 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:16.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:16.081 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:16.081 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:16.182 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:16.214 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:16.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:16.215 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:16.316 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:16.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:16.359 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:16.360 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:16.400 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 88 16 134615 134599 95.5766 62.3125 0.013418 0.0104581 2024-09-10T18:57:16.460 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:16.512 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:16.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:16.513 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:16.614 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:16.645 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:16.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:16.646 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:16.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:16 smithi018 ceph-mon[40196]: pgmap v547: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 8.0 GiB data, 23 GiB used, 514 GiB / 536 GiB avail; 1.6 KiB/s rd, 52 MiB/s wr, 838 op/s; 80756/391482 objects misplaced (20.628%); 53 MiB/s, 846 objects/s recovering 2024-09-10T18:57:16.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:16 smithi018 ceph-mon[29336]: pgmap v547: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 8.0 GiB data, 23 GiB used, 514 GiB / 536 GiB avail; 1.6 KiB/s rd, 52 MiB/s wr, 838 op/s; 80756/391482 objects misplaced (20.628%); 53 MiB/s, 846 objects/s recovering 2024-09-10T18:57:16.738 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:16 smithi103 ceph-mon[33163]: pgmap v547: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 8.0 GiB data, 23 GiB used, 514 GiB / 536 GiB avail; 1.6 KiB/s rd, 52 MiB/s wr, 838 op/s; 80756/391482 objects misplaced (20.628%); 53 MiB/s, 846 objects/s recovering 2024-09-10T18:57:16.747 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:16.781 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:16.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:16.782 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:16.883 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:16.904 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:16.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:16.905 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:17.006 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:17.022 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:57:16 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:57:17.036 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:17.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:17.038 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:17.138 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:17.183 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:17.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:17.184 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:17.285 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:17.330 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:17.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:17.331 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:17.400 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 89 16 135690 135674 95.2576 67.1875 0.009237 0.0104934 2024-09-10T18:57:17.432 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:17.464 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:17.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:17.465 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:17.565 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:17.611 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:17.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:17.612 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:17.713 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:17.754 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:17.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:17.755 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:17.856 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:17.889 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:17.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:17.890 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:17.991 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:18.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:18.042 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:18.042 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:18.143 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:18.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:18.180 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:18.180 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:18.281 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:18.312 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:18.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:18.317 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:18.401 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Total time run: 90.0084 2024-09-10T18:57:18.401 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Total writes made: 136749 2024-09-10T18:57:18.401 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Write size: 65536 2024-09-10T18:57:18.401 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Object size: 65536 2024-09-10T18:57:18.402 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Bandwidth (MB/sec): 94.9557 2024-09-10T18:57:18.402 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Stddev Bandwidth: 36.6149 2024-09-10T18:57:18.402 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Max bandwidth (MB/sec): 215.375 2024-09-10T18:57:18.402 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Min bandwidth (MB/sec): 0 2024-09-10T18:57:18.402 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Average IOPS: 1519 2024-09-10T18:57:18.402 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Stddev IOPS: 585.839 2024-09-10T18:57:18.402 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Max IOPS: 3446 2024-09-10T18:57:18.402 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Min IOPS: 0 2024-09-10T18:57:18.402 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Average Latency(s): 0.0105273 2024-09-10T18:57:18.402 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Stddev Latency(s): 0.0241829 2024-09-10T18:57:18.402 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Max latency(s): 2.17684 2024-09-10T18:57:18.402 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Min latency(s): 0.00128005 2024-09-10T18:57:18.407 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Cleaning up (deleting benchmark objects) 2024-09-10T18:57:18.417 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:18.453 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:18.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:18.454 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:18.530 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:18 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:18.530 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:18 smithi018 ceph-mon[29336]: pgmap v548: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 8.2 GiB data, 23 GiB used, 513 GiB / 536 GiB avail; 1023 B/s rd, 62 MiB/s wr, 988 op/s; 79235/403416 objects misplaced (19.641%); 62 MiB/s, 989 objects/s recovering 2024-09-10T18:57:18.531 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:18 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:18.531 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:18 smithi018 ceph-mon[40196]: pgmap v548: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 8.2 GiB data, 23 GiB used, 513 GiB / 536 GiB avail; 1023 B/s rd, 62 MiB/s wr, 988 op/s; 79235/403416 objects misplaced (19.641%); 62 MiB/s, 989 objects/s recovering 2024-09-10T18:57:18.555 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:18.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:18.583 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:18.583 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:18.684 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:18.714 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:18.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:18.715 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:18.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:18 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:18.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:18 smithi103 ceph-mon[33163]: pgmap v548: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 8.2 GiB data, 23 GiB used, 513 GiB / 536 GiB avail; 1023 B/s rd, 62 MiB/s wr, 988 op/s; 79235/403416 objects misplaced (19.641%); 62 MiB/s, 989 objects/s recovering 2024-09-10T18:57:18.817 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:18.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:18.862 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:18.863 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:18.963 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:18.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:18.994 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:18.994 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:19.095 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:19.100 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T18:57:19.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:19.138 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:19.138 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:19.239 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:19.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:19.295 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:19.295 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:19.396 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:19.425 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:19.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:19.426 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:19.527 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:19.553 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-10T18:57:19.572 INFO:teuthology.orchestra.run.smithi018.stdout:ERROR: (22) Invalid argument 2024-09-10T18:57:19.573 INFO:teuthology.orchestra.run.smithi018.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:57:19.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:19.597 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:19.597 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:19.698 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:19.725 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:19.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:19.726 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:19.827 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:19.878 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:19 smithi018 ceph-mon[40196]: pgmap v549: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 8.3 GiB data, 24 GiB used, 513 GiB / 536 GiB avail; 1023 B/s rd, 62 MiB/s wr, 993 op/s; 78784/406098 objects misplaced (19.400%); 62 MiB/s, 984 objects/s recovering 2024-09-10T18:57:19.878 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:19 smithi018 ceph-mon[29336]: pgmap v549: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 8.3 GiB data, 24 GiB used, 513 GiB / 536 GiB avail; 1023 B/s rd, 62 MiB/s wr, 993 op/s; 78784/406098 objects misplaced (19.400%); 62 MiB/s, 984 objects/s recovering 2024-09-10T18:57:19.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:19.885 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:19.885 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:19.986 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:20.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:19 smithi103 ceph-mon[33163]: pgmap v549: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 8.3 GiB data, 24 GiB used, 513 GiB / 536 GiB avail; 1023 B/s rd, 62 MiB/s wr, 993 op/s; 78784/406098 objects misplaced (19.400%); 62 MiB/s, 984 objects/s recovering 2024-09-10T18:57:20.033 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:20.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:20.035 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:20.136 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:20.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:20.167 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:20.167 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:20.268 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:20.299 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-10T18:57:20.323 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:20.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:20.327 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:20.428 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:20.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:20.486 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:20.486 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:20.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:57:20 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:57:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:57:20.587 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:20.611 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:20.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:20.613 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:20.714 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:20.760 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:20.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:20.764 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:20.865 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:20.871 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:57:20.871 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:57:20.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:20.941 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:20.941 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:21.042 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:21.072 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:21.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:21.073 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:21.174 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:21.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:57:20 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:57:20] "GET /metrics HTTP/1.1" 200 224417 "" "Prometheus/2.43.0" 2024-09-10T18:57:21.215 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:57:21.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:21.223 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:21.223 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:21.297 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:57:21.300 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-09-10T18:57:21.324 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:21.353 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:21.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:21.354 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:21.455 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:21.481 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:21.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:21.482 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:21.583 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:21.624 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:21.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:21.625 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:21.726 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:21.762 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:21.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:21.763 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:21.864 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:21.894 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:21.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:21.895 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:21.995 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:22.018 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:22.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:22.019 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:22.120 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:22.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:22.151 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:22.151 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:22.252 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:22.280 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:22.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:22.281 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:22.382 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:22.415 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:22.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:22.416 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:22.517 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:22.548 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:22.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:22.549 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:22.650 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:22.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:22.679 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:22.679 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:22.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:22 smithi018 ceph-mon[40196]: from='client.25189 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:57:22.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:22 smithi018 ceph-mon[40196]: pgmap v550: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 8.2 GiB data, 24 GiB used, 513 GiB / 536 GiB avail; 1021 B/s rd, 61 MiB/s wr, 992 op/s; 77849/406050 objects misplaced (19.172%); 62 MiB/s, 987 objects/s recovering 2024-09-10T18:57:22.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:22 smithi018 ceph-mon[29336]: from='client.25189 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:57:22.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:22 smithi018 ceph-mon[29336]: pgmap v550: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 8.2 GiB data, 24 GiB used, 513 GiB / 536 GiB avail; 1021 B/s rd, 61 MiB/s wr, 992 op/s; 77849/406050 objects misplaced (19.172%); 62 MiB/s, 987 objects/s recovering 2024-09-10T18:57:22.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:22 smithi103 ceph-mon[33163]: from='client.25189 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:57:22.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:22 smithi103 ceph-mon[33163]: pgmap v550: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 8.2 GiB data, 24 GiB used, 513 GiB / 536 GiB avail; 1021 B/s rd, 61 MiB/s wr, 992 op/s; 77849/406050 objects misplaced (19.172%); 62 MiB/s, 987 objects/s recovering 2024-09-10T18:57:22.780 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:22.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:22.813 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:22.813 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:22.914 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:22.943 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:22.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:22.944 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:23.045 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:23.070 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:23.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:23.071 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:23.172 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:23.201 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:23.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:23.203 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:23.303 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:23.324 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:23.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:23.325 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:23.426 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:23.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:23.456 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:23.456 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:23.557 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:23.588 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:23.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:23.589 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:23.690 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:23.719 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:23.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:23.721 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:23.821 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:23.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:23.859 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:23.859 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:23.960 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:24.001 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:24.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:24.002 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:24.102 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:24.127 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:24.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:24.128 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:24.228 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:24.268 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:24.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:24.269 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:24.370 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:24.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:24.402 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:24.402 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:24.503 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:24.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:24.536 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:24.536 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:24.637 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:24.679 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:24.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:24.680 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:24.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:24 smithi018 ceph-mon[29336]: pgmap v551: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 8.2 GiB data, 24 GiB used, 513 GiB / 536 GiB avail; 1006 B/s rd, 41 MiB/s wr, 895 op/s; 74060/403074 objects misplaced (18.374%); 56 MiB/s, 888 objects/s recovering 2024-09-10T18:57:24.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:24 smithi018 ceph-mon[40196]: pgmap v551: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 8.2 GiB data, 24 GiB used, 513 GiB / 536 GiB avail; 1006 B/s rd, 41 MiB/s wr, 895 op/s; 74060/403074 objects misplaced (18.374%); 56 MiB/s, 888 objects/s recovering 2024-09-10T18:57:24.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:24 smithi103 ceph-mon[33163]: pgmap v551: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 8.2 GiB data, 24 GiB used, 513 GiB / 536 GiB avail; 1006 B/s rd, 41 MiB/s wr, 895 op/s; 74060/403074 objects misplaced (18.374%); 56 MiB/s, 888 objects/s recovering 2024-09-10T18:57:24.782 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:24.802 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:24.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:24.804 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:24.904 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:24.948 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:24.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:24.949 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:25.050 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:25.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:25.090 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:25.090 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:25.191 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:25.217 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:25.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:25.218 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:25.318 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:25.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:25.366 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:25.366 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:25.467 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:25.503 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:25.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:25.505 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:25.605 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:25.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:25.629 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:25.630 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:25.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:25 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:57:25.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:25 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:57:25.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:25 smithi018 ceph-mon[40196]: osdmap e275: 8 total, 8 up, 6 in 2024-09-10T18:57:25.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:25 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:57:25.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:25 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:57:25.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:25 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:57:25.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:25 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:57:25.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:25 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:57:25.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:25 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:57:25.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:25 smithi018 ceph-mon[29336]: osdmap e275: 8 total, 8 up, 6 in 2024-09-10T18:57:25.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:25 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:57:25.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:25 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:57:25.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:25 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:57:25.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:25 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:57:25.731 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:25.761 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:25.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:25.762 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:25.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:25 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:57:25.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:25 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:57:25.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:25 smithi103 ceph-mon[33163]: osdmap e275: 8 total, 8 up, 6 in 2024-09-10T18:57:25.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:25 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:57:25.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:25 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:57:25.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:25 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:57:25.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:25 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:57:25.863 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:25.910 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:25.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:25.911 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:26.012 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:26.042 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:26.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:26.043 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:26.144 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:26.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:26.166 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:26.167 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:26.267 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:26.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:26.313 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:26.313 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:26.414 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:26.445 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:26.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:26.446 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:26.547 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:26.578 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:26.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:26.579 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:26.680 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:26.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:26 smithi018 ceph-mon[40196]: pgmap v553: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 8.1 GiB data, 24 GiB used, 513 GiB / 536 GiB avail; 1.1 KiB/s rd, 40 MiB/s wr, 1k op/s; 72691/400143 objects misplaced (18.166%); 61 MiB/s, 983 objects/s recovering 2024-09-10T18:57:26.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:26 smithi018 ceph-mon[40196]: osdmap e276: 8 total, 8 up, 6 in 2024-09-10T18:57:26.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:26 smithi018 ceph-mon[29336]: pgmap v553: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 8.1 GiB data, 24 GiB used, 513 GiB / 536 GiB avail; 1.1 KiB/s rd, 40 MiB/s wr, 1k op/s; 72691/400143 objects misplaced (18.166%); 61 MiB/s, 983 objects/s recovering 2024-09-10T18:57:26.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:26 smithi018 ceph-mon[29336]: osdmap e276: 8 total, 8 up, 6 in 2024-09-10T18:57:26.712 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:26.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:26.713 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:26.749 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:26 smithi103 ceph-mon[33163]: pgmap v553: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 8.1 GiB data, 24 GiB used, 513 GiB / 536 GiB avail; 1.1 KiB/s rd, 40 MiB/s wr, 1k op/s; 72691/400143 objects misplaced (18.166%); 61 MiB/s, 983 objects/s recovering 2024-09-10T18:57:26.749 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:26 smithi103 ceph-mon[33163]: osdmap e276: 8 total, 8 up, 6 in 2024-09-10T18:57:26.814 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:26.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:26.846 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:26.846 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:26.947 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:26.980 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:26.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:26.981 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:27.022 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:57:26 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:57:27.082 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:27.123 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:27.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:27.124 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:27.224 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:27.257 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:27.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:27.258 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:27.359 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:27.390 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:27.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:27.391 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:27.492 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:27.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:27.523 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:27.523 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:27.624 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:27.647 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:27.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:27.648 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:27.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:27 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:57:27.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:27 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:57:27.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:27 smithi018 ceph-mon[40196]: osdmap e277: 8 total, 8 up, 6 in 2024-09-10T18:57:27.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:27 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:57:27.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:27 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:57:27.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:27 smithi018 ceph-mon[29336]: osdmap e277: 8 total, 8 up, 6 in 2024-09-10T18:57:27.749 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:27.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:27 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:57:27.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:27 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:57:27.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:27 smithi103 ceph-mon[33163]: osdmap e277: 8 total, 8 up, 6 in 2024-09-10T18:57:27.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:27.782 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:27.782 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:27.883 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:27.926 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:27.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:27.927 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:28.027 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:28.062 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:28.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:28.063 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:28.163 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:28.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:28.197 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:28.197 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:28.298 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:28.328 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:28.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:28.329 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:28.429 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:28.470 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:28.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:28.471 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:28.572 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:28.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:28.594 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:28.595 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:28.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:28 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:28.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:28 smithi018 ceph-mon[40196]: pgmap v556: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 7.9 GiB data, 23 GiB used, 513 GiB / 536 GiB avail; 1.8 KiB/s rd, 15 MiB/s wr, 1.43k op/s; 67179/388770 objects misplaced (17.280%); 79 MiB/s, 1.26k objects/s recovering 2024-09-10T18:57:28.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:28 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:28.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:28 smithi018 ceph-mon[29336]: pgmap v556: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 7.9 GiB data, 23 GiB used, 513 GiB / 536 GiB avail; 1.8 KiB/s rd, 15 MiB/s wr, 1.43k op/s; 67179/388770 objects misplaced (17.280%); 79 MiB/s, 1.26k objects/s recovering 2024-09-10T18:57:28.696 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:28.725 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:28.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:28.727 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:28.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:28 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:28.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:28 smithi103 ceph-mon[33163]: pgmap v556: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 7.9 GiB data, 23 GiB used, 513 GiB / 536 GiB avail; 1.8 KiB/s rd, 15 MiB/s wr, 1.43k op/s; 67179/388770 objects misplaced (17.280%); 79 MiB/s, 1.26k objects/s recovering 2024-09-10T18:57:28.827 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:28.859 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:28.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:28.860 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:28.961 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:29.003 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:29.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:29.004 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:29.105 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:29.144 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:29.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:29.145 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:29.246 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:29.276 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:29.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:29.277 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:29.378 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:29.420 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:29.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:29.421 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:29.522 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:29.553 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:29.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:29.554 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:29.655 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:29.706 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:29.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:29.707 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:29.808 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:29.842 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:29.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:29.843 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:29.944 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:29.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:29 smithi018 ceph-mon[29336]: pgmap v557: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 7.8 GiB data, 23 GiB used, 513 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 936 op/s; 65964/386214 objects misplaced (17.080%); 54 MiB/s, 858 objects/s recovering 2024-09-10T18:57:29.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:29 smithi018 ceph-mon[40196]: pgmap v557: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 7.8 GiB data, 23 GiB used, 513 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 936 op/s; 65964/386214 objects misplaced (17.080%); 54 MiB/s, 858 objects/s recovering 2024-09-10T18:57:29.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:29.974 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:29.974 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:30.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:29 smithi103 ceph-mon[33163]: pgmap v557: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 7.8 GiB data, 23 GiB used, 513 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 936 op/s; 65964/386214 objects misplaced (17.080%); 54 MiB/s, 858 objects/s recovering 2024-09-10T18:57:30.075 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:30.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:30.108 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:30.108 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:30.209 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:30.239 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:30.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:30.240 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:30.341 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:30.389 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:30.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:30.391 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:30.491 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:30.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:57:30 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:57:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:57:30.523 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:30.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:30.524 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:30.625 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:30.652 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:30.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:30.653 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:30.754 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:30.787 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:30.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:30.788 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:30.889 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:30.906 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":549,"stamp":"2024-09-10T18:57:19.272535+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8859222537,"num_objects":135366,"num_object_clones":0,"num_object_copies":406098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":78784,"num_objects_unfound":0,"num_objects_dirty":135366,"num_whiteouts":0,"num_read":1874,"num_read_kb":1648,"num_write":135951,"num_write_kb":8652612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13508,"num_bytes_recovered":870966578,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":60267,"ondisk_log_size":60267,"up":504,"acting":504,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":511,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":562470912,"kb_used":24689044,"kb_used_data":21971428,"kb_used_omap":12,"kb_used_meta":2717555,"kb_avail":537781868,"statfs":{"total":575970213888,"available":550688632832,"internally_reserved":0,"allocated":22498742272,"data_stored":22496147518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13122,"internal_metadata":2782776510},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,3,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":647888896,"num_objects":9886,"num_object_clones":0,"num_object_copies":29658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-3697,"num_objects_unfound":0,"num_objects_dirty":9886,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":9928,"num_write_kb":634048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9851,"num_bytes_recovered":645595136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.005940"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":280,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.674060+0000","last_change":"2024-09-10T18:57:02.462265+0000","last_active":"2024-09-10T18:57:04.674060+0000","last_peered":"2024-09-10T18:57:04.674060+0000","last_clean":"2024-09-10T18:57:04.674060+0000","last_became_active":"2024-09-10T18:57:02.460451+0000","last_became_peered":"2024-09-10T18:57:02.460451+0000","last_unstale":"2024-09-10T18:57:04.674060+0000","last_undegraded":"2024-09-10T18:57:04.674060+0000","last_fullsized":"2024-09-10T18:57:04.674060+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:14.875872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:14.875872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:30.666906+0000","scrub_duration":0.0037478699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":264,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.665941+0000","last_change":"2024-09-10T18:57:02.442207+0000","last_active":"2024-09-10T18:57:04.665941+0000","last_peered":"2024-09-10T18:57:04.665941+0000","last_clean":"2024-09-10T18:57:04.665941+0000","last_became_active":"2024-09-10T18:57:02.440648+0000","last_became_peered":"2024-09-10T18:57:02.440648+0000","last_unstale":"2024-09-10T18:57:04.665941+0000","last_undegraded":"2024-09-10T18:57:04.665941+0000","last_fullsized":"2024-09-10T18:57:04.665941+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:31.221724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:56:31.221724+0000","last_clean_scrub_stamp":"2024-09-10T18:56:31.221724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:35.953504+0000","scrub_duration":0.0051032730000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"232'27","reported_seq":365,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.670502+0000","last_change":"2024-09-10T18:56:02.821637+0000","last_active":"2024-09-10T18:57:04.670502+0000","last_peered":"2024-09-10T18:57:04.670502+0000","last_clean":"2024-09-10T18:57:04.670502+0000","last_became_active":"2024-09-10T18:55:40.394717+0000","last_became_peered":"2024-09-10T18:55:40.394717+0000","last_unstale":"2024-09-10T18:57:04.670502+0000","last_undegraded":"2024-09-10T18:57:04.670502+0000","last_fullsized":"2024-09-10T18:57:04.670502+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'27","last_scrub_stamp":"2024-09-10T18:56:02.821551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:02.821551+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:30.084130+0000","scrub_duration":0.0083110159999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":270,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.660015+0000","last_change":"2024-09-10T18:55:33.526359+0000","last_active":"2024-09-10T18:57:04.660015+0000","last_peered":"2024-09-10T18:57:04.660015+0000","last_clean":"2024-09-10T18:57:04.660015+0000","last_became_active":"2024-09-10T18:45:22.065619+0000","last_became_peered":"2024-09-10T18:45:22.065619+0000","last_unstale":"2024-09-10T18:57:04.660015+0000","last_undegraded":"2024-09-10T18:57:04.660015+0000","last_fullsized":"2024-09-10T18:57:04.660015+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:33.526302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.057932+0000","last_clean_scrub_stamp":"2024-09-10T18:55:33.526302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001517556,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":313,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.675896+0000","last_change":"2024-09-10T18:57:02.457742+0000","last_active":"2024-09-10T18:57:04.675896+0000","last_peered":"2024-09-10T18:57:04.675896+0000","last_clean":"2024-09-10T18:57:04.675896+0000","last_became_active":"2024-09-10T18:57:02.457322+0000","last_became_peered":"2024-09-10T18:57:02.457322+0000","last_unstale":"2024-09-10T18:57:04.675896+0000","last_undegraded":"2024-09-10T18:57:04.675896+0000","last_fullsized":"2024-09-10T18:57:04.675896+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:23.885562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:23.885562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:26.898168+0000","scrub_duration":0.0031033800000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"232'19","reported_seq":291,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.660109+0000","last_change":"2024-09-10T18:54:48.324177+0000","last_active":"2024-09-10T18:57:04.660109+0000","last_peered":"2024-09-10T18:57:04.660109+0000","last_clean":"2024-09-10T18:57:04.660109+0000","last_became_active":"2024-09-10T18:45:20.061266+0000","last_became_peered":"2024-09-10T18:45:20.061266+0000","last_unstale":"2024-09-10T18:57:04.660109+0000","last_undegraded":"2024-09-10T18:57:04.660109+0000","last_fullsized":"2024-09-10T18:57:04.660109+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'13","last_scrub_stamp":"2024-09-10T18:54:48.323985+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_clean_scrub_stamp":"2024-09-10T18:54:48.323985+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0080421699999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.663094+0000","last_change":"2024-09-10T18:57:02.446964+0000","last_active":"2024-09-10T18:57:04.663094+0000","last_peered":"2024-09-10T18:57:04.663094+0000","last_clean":"2024-09-10T18:57:04.663094+0000","last_became_active":"2024-09-10T18:57:02.446008+0000","last_became_peered":"2024-09-10T18:57:02.446008+0000","last_unstale":"2024-09-10T18:57:04.663094+0000","last_undegraded":"2024-09-10T18:57:04.663094+0000","last_fullsized":"2024-09-10T18:57:04.663094+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:22.163121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:01.135988+0000","last_clean_scrub_stamp":"2024-09-10T18:56:22.163121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:36.622434+0000","scrub_duration":0.002631478,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":258,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.660780+0000","last_change":"2024-09-10T18:54:22.051769+0000","last_active":"2024-09-10T18:57:04.660780+0000","last_peered":"2024-09-10T18:57:04.660780+0000","last_clean":"2024-09-10T18:57:04.660780+0000","last_became_active":"2024-09-10T18:45:24.081002+0000","last_became_peered":"2024-09-10T18:45:24.081002+0000","last_unstale":"2024-09-10T18:57:04.660780+0000","last_undegraded":"2024-09-10T18:57:04.660780+0000","last_fullsized":"2024-09-10T18:57:04.660780+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:22.051692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:54:22.051692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013425862,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":262,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.676624+0000","last_change":"2024-09-10T18:57:02.452754+0000","last_active":"2024-09-10T18:57:04.676624+0000","last_peered":"2024-09-10T18:57:04.676624+0000","last_clean":"2024-09-10T18:57:04.676624+0000","last_became_active":"2024-09-10T18:57:02.452259+0000","last_became_peered":"2024-09-10T18:57:02.452259+0000","last_unstale":"2024-09-10T18:57:04.676624+0000","last_undegraded":"2024-09-10T18:57:04.676624+0000","last_fullsized":"2024-09-10T18:57:04.676624+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:06.064071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:06.064071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:28.915864+0000","scrub_duration":0.0037631460000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1b","version":"232'9","reported_seq":277,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.661086+0000","last_change":"2024-09-10T18:54:45.305098+0000","last_active":"2024-09-10T18:57:04.661086+0000","last_peered":"2024-09-10T18:57:04.661086+0000","last_clean":"2024-09-10T18:57:04.661086+0000","last_became_active":"2024-09-10T18:45:20.057567+0000","last_became_peered":"2024-09-10T18:45:20.057567+0000","last_unstale":"2024-09-10T18:57:04.661086+0000","last_undegraded":"2024-09-10T18:57:04.661086+0000","last_fullsized":"2024-09-10T18:57:04.661086+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"163'7","last_scrub_stamp":"2024-09-10T18:54:45.305023+0000","last_deep_scrub":"163'7","last_deep_scrub_stamp":"2024-09-10T18:51:54.261879+0000","last_clean_scrub_stamp":"2024-09-10T18:54:45.305023+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0088188659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":16,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.660723+0000","last_change":"2024-09-10T18:57:02.439756+0000","last_active":"2024-09-10T18:57:04.660723+0000","last_peered":"2024-09-10T18:57:04.660723+0000","last_clean":"2024-09-10T18:57:04.660723+0000","last_became_active":"2024-09-10T18:57:02.439440+0000","last_became_peered":"2024-09-10T18:57:02.439440+0000","last_unstale":"2024-09-10T18:57:04.660723+0000","last_undegraded":"2024-09-10T18:57:04.660723+0000","last_fullsized":"2024-09-10T18:57:04.660723+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:27.785914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:36.509971+0000","last_clean_scrub_stamp":"2024-09-10T18:55:27.785914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":129,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.663547+0000","last_change":"2024-09-10T18:56:07.553751+0000","last_active":"2024-09-10T18:57:04.663547+0000","last_peered":"2024-09-10T18:57:04.663547+0000","last_clean":"2024-09-10T18:57:04.663547+0000","last_became_active":"2024-09-10T18:55:40.399236+0000","last_became_peered":"2024-09-10T18:55:40.399236+0000","last_unstale":"2024-09-10T18:57:04.663547+0000","last_undegraded":"2024-09-10T18:57:04.663547+0000","last_fullsized":"2024-09-10T18:57:04.663547+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:07.553696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:07.553696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:16.468669+0000","scrub_duration":0.0091042339999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":69,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.672152+0000","last_change":"2024-09-10T18:57:02.440789+0000","last_active":"2024-09-10T18:57:04.672152+0000","last_peered":"2024-09-10T18:57:04.672152+0000","last_clean":"2024-09-10T18:57:04.672152+0000","last_became_active":"2024-09-10T18:57:02.440497+0000","last_became_peered":"2024-09-10T18:57:02.440497+0000","last_unstale":"2024-09-10T18:57:04.672152+0000","last_undegraded":"2024-09-10T18:57:04.672152+0000","last_fullsized":"2024-09-10T18:57:04.672152+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:40.672810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:54:40.672810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016221689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"232'15","reported_seq":323,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.661077+0000","last_change":"2024-09-10T18:56:26.480197+0000","last_active":"2024-09-10T18:57:04.661077+0000","last_peered":"2024-09-10T18:57:04.661077+0000","last_clean":"2024-09-10T18:57:04.661077+0000","last_became_active":"2024-09-10T18:55:40.391571+0000","last_became_peered":"2024-09-10T18:55:40.391571+0000","last_unstale":"2024-09-10T18:57:04.661077+0000","last_undegraded":"2024-09-10T18:57:04.661077+0000","last_fullsized":"2024-09-10T18:57:04.661077+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T18:56:26.480105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:26.480105+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:35.172927+0000","scrub_duration":0.0027886920000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":299,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.671788+0000","last_change":"2024-09-10T18:57:02.431878+0000","last_active":"2024-09-10T18:57:04.671788+0000","last_peered":"2024-09-10T18:57:04.671788+0000","last_clean":"2024-09-10T18:57:04.671788+0000","last_became_active":"2024-09-10T18:57:02.431585+0000","last_became_peered":"2024-09-10T18:57:02.431585+0000","last_unstale":"2024-09-10T18:57:04.671788+0000","last_undegraded":"2024-09-10T18:57:04.671788+0000","last_fullsized":"2024-09-10T18:57:04.671788+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:00.729699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:00.729699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047811370000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":229,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.664756+0000","last_change":"2024-09-10T18:57:02.444127+0000","last_active":"2024-09-10T18:57:04.664756+0000","last_peered":"2024-09-10T18:57:04.664756+0000","last_clean":"2024-09-10T18:57:04.664756+0000","last_became_active":"2024-09-10T18:57:02.442905+0000","last_became_peered":"2024-09-10T18:57:02.442905+0000","last_unstale":"2024-09-10T18:57:04.664756+0000","last_undegraded":"2024-09-10T18:57:04.664756+0000","last_fullsized":"2024-09-10T18:57:04.664756+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:59.425718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:59.425718+0000","last_clean_scrub_stamp":"2024-09-10T18:52:59.425718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"274'8507","reported_seq":8559,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-10T18:57:16.504612+0000","last_change":"2024-09-10T18:55:40.403238+0000","last_active":"2024-09-10T18:57:16.504612+0000","last_peered":"2024-09-10T18:57:16.504612+0000","last_clean":"2024-09-10T18:57:16.504612+0000","last_became_active":"2024-09-10T18:55:40.402912+0000","last_became_peered":"2024-09-10T18:55:40.402912+0000","last_unstale":"2024-09-10T18:57:16.504612+0000","last_undegraded":"2024-09-10T18:57:16.504612+0000","last_fullsized":"2024-09-10T18:57:16.504612+0000","mapping_epoch":241,"log_start":"259'5414","ondisk_log_start":"259'5414","created":237,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3093,"log_dups_size":0,"ondisk_log_size":3093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":557514752,"num_objects":8507,"num_object_clones":0,"num_object_copies":25521,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8507,"num_write_kb":544448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"70'81","reported_seq":120,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.670143+0000","last_change":"2024-09-10T18:55:31.654161+0000","last_active":"2024-09-10T18:57:04.670143+0000","last_peered":"2024-09-10T18:57:04.670143+0000","last_clean":"2024-09-10T18:57:04.670143+0000","last_became_active":"2024-09-10T18:52:54.619039+0000","last_became_peered":"2024-09-10T18:52:54.619039+0000","last_unstale":"2024-09-10T18:57:04.670143+0000","last_undegraded":"2024-09-10T18:57:04.670143+0000","last_fullsized":"2024-09-10T18:57:04.670143+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:55:31.654090+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:55:31.654090+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025082630000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"232'39","reported_seq":357,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.661986+0000","last_change":"2024-09-10T18:55:25.483970+0000","last_active":"2024-09-10T18:57:04.661986+0000","last_peered":"2024-09-10T18:57:04.661986+0000","last_clean":"2024-09-10T18:57:04.661986+0000","last_became_active":"2024-09-10T18:47:36.409300+0000","last_became_peered":"2024-09-10T18:47:36.409300+0000","last_unstale":"2024-09-10T18:57:04.661986+0000","last_undegraded":"2024-09-10T18:57:04.661986+0000","last_fullsized":"2024-09-10T18:57:04.661986+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'27","last_scrub_stamp":"2024-09-10T18:55:25.483919+0000","last_deep_scrub":"78'25","last_deep_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_clean_scrub_stamp":"2024-09-10T18:55:25.483919+0000","objects_scrubbed":8,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017172610000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":16,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.661217+0000","last_change":"2024-09-10T18:57:02.437548+0000","last_active":"2024-09-10T18:57:04.661217+0000","last_peered":"2024-09-10T18:57:04.661217+0000","last_clean":"2024-09-10T18:57:04.661217+0000","last_became_active":"2024-09-10T18:57:02.437260+0000","last_became_peered":"2024-09-10T18:57:02.437260+0000","last_unstale":"2024-09-10T18:57:04.661217+0000","last_undegraded":"2024-09-10T18:57:04.661217+0000","last_fullsized":"2024-09-10T18:57:04.661217+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:30.719475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:55:30.719475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":279,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.666906+0000","last_change":"2024-09-10T18:56:09.623123+0000","last_active":"2024-09-10T18:57:04.666906+0000","last_peered":"2024-09-10T18:57:04.666906+0000","last_clean":"2024-09-10T18:57:04.666906+0000","last_became_active":"2024-09-10T18:55:40.392449+0000","last_became_peered":"2024-09-10T18:55:40.392449+0000","last_unstale":"2024-09-10T18:57:04.666906+0000","last_undegraded":"2024-09-10T18:57:04.666906+0000","last_fullsized":"2024-09-10T18:57:04.666906+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:09.622841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:04.512425+0000","last_clean_scrub_stamp":"2024-09-10T18:56:09.622841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:35.116868+0000","scrub_duration":0.010581503000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":176,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.666267+0000","last_change":"2024-09-10T18:55:40.388451+0000","last_active":"2024-09-10T18:57:04.666267+0000","last_peered":"2024-09-10T18:57:04.666267+0000","last_clean":"2024-09-10T18:57:04.666267+0000","last_became_active":"2024-09-10T18:55:40.388154+0000","last_became_peered":"2024-09-10T18:55:40.388154+0000","last_unstale":"2024-09-10T18:57:04.666267+0000","last_undegraded":"2024-09-10T18:57:04.666267+0000","last_fullsized":"2024-09-10T18:57:04.666267+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:05.317828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:53:05.317828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015035510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1","version":"274'8385","reported_seq":8452,"reported_epoch":274,"state":"active+remapped+backfill_wait","last_fresh":"2024-09-10T18:57:16.704875+0000","last_change":"2024-09-10T18:57:03.564873+0000","last_active":"2024-09-10T18:57:16.704875+0000","last_peered":"2024-09-10T18:57:16.704875+0000","last_clean":"2024-09-10T18:57:01.406183+0000","last_became_active":"2024-09-10T18:57:03.519482+0000","last_became_peered":"2024-09-10T18:57:03.519482+0000","last_unstale":"2024-09-10T18:57:16.704875+0000","last_undegraded":"2024-09-10T18:57:16.704875+0000","last_fullsized":"2024-09-10T18:57:16.704875+0000","mapping_epoch":271,"log_start":"258'4924","ondisk_log_start":"258'4924","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3461,"log_dups_size":0,"ondisk_log_size":3461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":549519360,"num_objects":8385,"num_object_clones":0,"num_object_copies":25155,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":8382,"num_objects_unfound":0,"num_objects_dirty":8385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8385,"num_write_kb":536640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,7],"avail_no_missing":["3","6","7"],"object_location_counts":[{"shards":"3,6,7","objects":8385}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.9","version":"70'81","reported_seq":174,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.672938+0000","last_change":"2024-09-10T18:57:02.465293+0000","last_active":"2024-09-10T18:57:04.672938+0000","last_peered":"2024-09-10T18:57:04.672938+0000","last_clean":"2024-09-10T18:57:04.672938+0000","last_became_active":"2024-09-10T18:57:02.463750+0000","last_became_peered":"2024-09-10T18:57:02.463750+0000","last_unstale":"2024-09-10T18:57:04.672938+0000","last_undegraded":"2024-09-10T18:57:04.672938+0000","last_fullsized":"2024-09-10T18:57:04.672938+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:54:45.753792+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:54:45.753792+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016709736999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":107339,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"232'25","reported_seq":307,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.661563+0000","last_change":"2024-09-10T18:56:32.437605+0000","last_active":"2024-09-10T18:57:04.661563+0000","last_peered":"2024-09-10T18:57:04.661563+0000","last_clean":"2024-09-10T18:57:04.661563+0000","last_became_active":"2024-09-10T18:45:20.053428+0000","last_became_peered":"2024-09-10T18:45:20.053428+0000","last_unstale":"2024-09-10T18:57:04.661563+0000","last_undegraded":"2024-09-10T18:57:04.661563+0000","last_fullsized":"2024-09-10T18:57:04.661563+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'25","last_scrub_stamp":"2024-09-10T18:56:32.437503+0000","last_deep_scrub":"116'15","last_deep_scrub_stamp":"2024-09-10T18:49:12.374527+0000","last_clean_scrub_stamp":"2024-09-10T18:56:32.437503+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:46.997961+0000","scrub_duration":0.0039673679999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":286,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.664198+0000","last_change":"2024-09-10T18:57:02.441472+0000","last_active":"2024-09-10T18:57:04.664198+0000","last_peered":"2024-09-10T18:57:04.664198+0000","last_clean":"2024-09-10T18:57:04.664198+0000","last_became_active":"2024-09-10T18:57:02.440442+0000","last_became_peered":"2024-09-10T18:57:02.440442+0000","last_unstale":"2024-09-10T18:57:04.664198+0000","last_undegraded":"2024-09-10T18:57:04.664198+0000","last_fullsized":"2024-09-10T18:57:04.664198+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:29.802937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:29.802937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:37.558176+0000","scrub_duration":0.0044818080000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"273'13","reported_seq":280,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-10T18:57:12.163426+0000","last_change":"2024-09-10T18:57:02.463853+0000","last_active":"2024-09-10T18:57:12.163426+0000","last_peered":"2024-09-10T18:57:12.163426+0000","last_clean":"2024-09-10T18:57:12.163426+0000","last_became_active":"2024-09-10T18:57:02.448258+0000","last_became_peered":"2024-09-10T18:57:02.448258+0000","last_unstale":"2024-09-10T18:57:12.163426+0000","last_undegraded":"2024-09-10T18:57:12.163426+0000","last_fullsized":"2024-09-10T18:57:12.163426+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"242'12","last_scrub_stamp":"2024-09-10T18:56:27.995682+0000","last_deep_scrub":"202'10","last_deep_scrub_stamp":"2024-09-10T18:53:45.547686+0000","last_clean_scrub_stamp":"2024-09-10T18:56:27.995682+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:57.194386+0000","scrub_duration":0.0025134200000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":298,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.663702+0000","last_change":"2024-09-10T18:57:02.440361+0000","last_active":"2024-09-10T18:57:04.663702+0000","last_peered":"2024-09-10T18:57:04.663702+0000","last_clean":"2024-09-10T18:57:04.663702+0000","last_became_active":"2024-09-10T18:57:02.439477+0000","last_became_peered":"2024-09-10T18:57:02.439477+0000","last_unstale":"2024-09-10T18:57:04.663702+0000","last_undegraded":"2024-09-10T18:57:04.663702+0000","last_fullsized":"2024-09-10T18:57:04.663702+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:37.565359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:32.043666+0000","last_clean_scrub_stamp":"2024-09-10T18:55:37.565359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038365510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"274'8438","reported_seq":11636,"reported_epoch":274,"state":"active+remapped+backfilling","last_fresh":"2024-09-10T18:57:16.731000+0000","last_change":"2024-09-10T18:57:03.593637+0000","last_active":"2024-09-10T18:57:16.731000+0000","last_peered":"2024-09-10T18:57:16.731000+0000","last_clean":"2024-09-10T18:57:01.384356+0000","last_became_active":"2024-09-10T18:57:03.516732+0000","last_became_peered":"2024-09-10T18:57:03.516732+0000","last_unstale":"2024-09-10T18:57:16.731000+0000","last_undegraded":"2024-09-10T18:57:16.731000+0000","last_fullsized":"2024-09-10T18:57:16.731000+0000","mapping_epoch":271,"log_start":"258'5003","ondisk_log_start":"258'5003","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3435,"log_dups_size":0,"ondisk_log_size":3435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":552992768,"num_objects":8438,"num_object_clones":0,"num_object_copies":25314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6792,"num_objects_unfound":0,"num_objects_dirty":8438,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8438,"num_write_kb":540032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1570,"num_bytes_recovered":102891520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,7],"avail_no_missing":["3","1","7"],"object_location_counts":[{"shards":"1,3,7","objects":8438}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.a","version":"70'81","reported_seq":187,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.671340+0000","last_change":"2024-09-10T18:55:37.921571+0000","last_active":"2024-09-10T18:57:04.671340+0000","last_peered":"2024-09-10T18:57:04.671340+0000","last_clean":"2024-09-10T18:57:04.671340+0000","last_became_active":"2024-09-10T18:50:08.398345+0000","last_became_peered":"2024-09-10T18:50:08.398345+0000","last_unstale":"2024-09-10T18:57:04.671340+0000","last_undegraded":"2024-09-10T18:57:04.671340+0000","last_fullsized":"2024-09-10T18:57:04.671340+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:55:37.921519+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:55:37.921519+0000","last_clean_scrub_stamp":"2024-09-10T18:55:37.921519+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002222407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"232'34","reported_seq":291,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.673278+0000","last_change":"2024-09-10T18:56:22.727738+0000","last_active":"2024-09-10T18:57:04.673278+0000","last_peered":"2024-09-10T18:57:04.673278+0000","last_clean":"2024-09-10T18:57:04.673278+0000","last_became_active":"2024-09-10T18:46:56.742946+0000","last_became_peered":"2024-09-10T18:46:56.742946+0000","last_unstale":"2024-09-10T18:57:04.673278+0000","last_undegraded":"2024-09-10T18:57:04.673278+0000","last_fullsized":"2024-09-10T18:57:04.673278+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'34","last_scrub_stamp":"2024-09-10T18:56:22.727671+0000","last_deep_scrub":"141'22","last_deep_scrub_stamp":"2024-09-10T18:50:57.119152+0000","last_clean_scrub_stamp":"2024-09-10T18:56:22.727671+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:28.158538+0000","scrub_duration":0.0079020229999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":289,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.663786+0000","last_change":"2024-09-10T18:57:02.447072+0000","last_active":"2024-09-10T18:57:04.663786+0000","last_peered":"2024-09-10T18:57:04.663786+0000","last_clean":"2024-09-10T18:57:04.663786+0000","last_became_active":"2024-09-10T18:57:02.445848+0000","last_became_peered":"2024-09-10T18:57:02.445848+0000","last_unstale":"2024-09-10T18:57:04.663786+0000","last_undegraded":"2024-09-10T18:57:04.663786+0000","last_fullsized":"2024-09-10T18:57:04.663786+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:14.634438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_clean_scrub_stamp":"2024-09-10T18:56:14.634438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:17.658238+0000","scrub_duration":0.0058886299999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"242'10","reported_seq":441,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-10T18:57:12.160715+0000","last_change":"2024-09-10T18:56:19.581070+0000","last_active":"2024-09-10T18:57:12.160715+0000","last_peered":"2024-09-10T18:57:12.160715+0000","last_clean":"2024-09-10T18:57:12.160715+0000","last_became_active":"2024-09-10T18:55:40.397246+0000","last_became_peered":"2024-09-10T18:55:40.397246+0000","last_unstale":"2024-09-10T18:57:12.160715+0000","last_undegraded":"2024-09-10T18:57:12.160715+0000","last_fullsized":"2024-09-10T18:57:12.160715+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"242'10","last_scrub_stamp":"2024-09-10T18:56:19.581012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:19.581012+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:38.538372+0000","scrub_duration":0.0013423529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":287,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.672059+0000","last_change":"2024-09-10T18:56:01.731623+0000","last_active":"2024-09-10T18:57:04.672059+0000","last_peered":"2024-09-10T18:57:04.672059+0000","last_clean":"2024-09-10T18:57:04.672059+0000","last_became_active":"2024-09-10T18:55:40.392003+0000","last_became_peered":"2024-09-10T18:55:40.392003+0000","last_unstale":"2024-09-10T18:57:04.672059+0000","last_undegraded":"2024-09-10T18:57:04.672059+0000","last_fullsized":"2024-09-10T18:57:04.672059+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:01.731528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:54.218602+0000","last_clean_scrub_stamp":"2024-09-10T18:56:01.731528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003574701,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"274'8539","reported_seq":12412,"reported_epoch":274,"state":"active+remapped+backfilling","last_fresh":"2024-09-10T18:57:16.728053+0000","last_change":"2024-09-10T18:57:03.521123+0000","last_active":"2024-09-10T18:57:16.728053+0000","last_peered":"2024-09-10T18:57:16.728053+0000","last_clean":"2024-09-10T18:57:01.383738+0000","last_became_active":"2024-09-10T18:57:03.512968+0000","last_became_peered":"2024-09-10T18:57:03.512968+0000","last_unstale":"2024-09-10T18:57:16.728053+0000","last_undegraded":"2024-09-10T18:57:16.728053+0000","last_fullsized":"2024-09-10T18:57:16.728053+0000","mapping_epoch":271,"log_start":"258'5117","ondisk_log_start":"258'5117","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3422,"log_dups_size":0,"ondisk_log_size":3422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":559611904,"num_objects":8539,"num_object_clones":0,"num_object_copies":25617,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6520,"num_objects_unfound":0,"num_objects_dirty":8539,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8539,"num_write_kb":546496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1908,"num_bytes_recovered":125042688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,7],"avail_no_missing":["3","1","7"],"object_location_counts":[{"shards":"1,3,7","objects":8539}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.b","version":"70'81","reported_seq":229,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.663785+0000","last_change":"2024-09-10T18:56:33.793393+0000","last_active":"2024-09-10T18:57:04.663785+0000","last_peered":"2024-09-10T18:57:04.663785+0000","last_clean":"2024-09-10T18:57:04.663785+0000","last_became_active":"2024-09-10T18:55:40.401643+0000","last_became_peered":"2024-09-10T18:55:40.401643+0000","last_unstale":"2024-09-10T18:57:04.663785+0000","last_undegraded":"2024-09-10T18:57:04.663785+0000","last_fullsized":"2024-09-10T18:57:04.663785+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:33.793296+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:33.793296+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:58:03.666307+0000","scrub_duration":0.0045437230000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"232'24","reported_seq":374,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.663772+0000","last_change":"2024-09-10T18:54:49.297032+0000","last_active":"2024-09-10T18:57:04.663772+0000","last_peered":"2024-09-10T18:57:04.663772+0000","last_clean":"2024-09-10T18:57:04.663772+0000","last_became_active":"2024-09-10T18:47:57.106935+0000","last_became_peered":"2024-09-10T18:47:57.106935+0000","last_unstale":"2024-09-10T18:57:04.663772+0000","last_undegraded":"2024-09-10T18:57:04.663772+0000","last_fullsized":"2024-09-10T18:57:04.663772+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"224'16","last_scrub_stamp":"2024-09-10T18:54:49.296957+0000","last_deep_scrub":"224'16","last_deep_scrub_stamp":"2024-09-10T18:54:49.296957+0000","last_clean_scrub_stamp":"2024-09-10T18:54:49.296957+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014223285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":269,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.664961+0000","last_change":"2024-09-10T18:57:02.444295+0000","last_active":"2024-09-10T18:57:04.664961+0000","last_peered":"2024-09-10T18:57:04.664961+0000","last_clean":"2024-09-10T18:57:04.664961+0000","last_became_active":"2024-09-10T18:57:02.443816+0000","last_became_peered":"2024-09-10T18:57:02.443816+0000","last_unstale":"2024-09-10T18:57:04.664961+0000","last_undegraded":"2024-09-10T18:57:04.664961+0000","last_fullsized":"2024-09-10T18:57:04.664961+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:29.994533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:29.994533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:59.492557+0000","scrub_duration":0.0053276879999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":275,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.673539+0000","last_change":"2024-09-10T18:55:26.909204+0000","last_active":"2024-09-10T18:57:04.673539+0000","last_peered":"2024-09-10T18:57:04.673539+0000","last_clean":"2024-09-10T18:57:04.673539+0000","last_became_active":"2024-09-10T18:45:22.068918+0000","last_became_peered":"2024-09-10T18:45:22.068918+0000","last_unstale":"2024-09-10T18:57:04.673539+0000","last_undegraded":"2024-09-10T18:57:04.673539+0000","last_fullsized":"2024-09-10T18:57:04.673539+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:26.909143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_clean_scrub_stamp":"2024-09-10T18:55:26.909143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013975070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":260,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.676581+0000","last_change":"2024-09-10T18:54:51.399064+0000","last_active":"2024-09-10T18:57:04.676581+0000","last_peered":"2024-09-10T18:57:04.676581+0000","last_clean":"2024-09-10T18:57:04.676581+0000","last_became_active":"2024-09-10T18:45:24.264538+0000","last_became_peered":"2024-09-10T18:45:24.264538+0000","last_unstale":"2024-09-10T18:57:04.676581+0000","last_undegraded":"2024-09-10T18:57:04.676581+0000","last_fullsized":"2024-09-10T18:57:04.676581+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:51.398981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:51.398981+0000","last_clean_scrub_stamp":"2024-09-10T18:54:51.398981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027589680000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"274'8479","reported_seq":11076,"reported_epoch":274,"state":"active+remapped+backfilling","last_fresh":"2024-09-10T18:57:16.733759+0000","last_change":"2024-09-10T18:57:03.547472+0000","last_active":"2024-09-10T18:57:16.733759+0000","last_peered":"2024-09-10T18:57:16.733759+0000","last_clean":"2024-09-10T18:57:01.397751+0000","last_became_active":"2024-09-10T18:57:03.511108+0000","last_became_peered":"2024-09-10T18:57:03.511108+0000","last_unstale":"2024-09-10T18:57:16.733759+0000","last_undegraded":"2024-09-10T18:57:16.733759+0000","last_fullsized":"2024-09-10T18:57:16.733759+0000","mapping_epoch":271,"log_start":"258'5091","ondisk_log_start":"258'5091","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3388,"log_dups_size":0,"ondisk_log_size":3388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":555679744,"num_objects":8479,"num_object_clones":0,"num_object_copies":25437,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7136,"num_objects_unfound":0,"num_objects_dirty":8479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8479,"num_write_kb":542656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1270,"num_bytes_recovered":83230720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,7],"avail_no_missing":["3","4","7"],"object_location_counts":[{"shards":"3,4,7","objects":8479}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"70'81","reported_seq":208,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.679755+0000","last_change":"2024-09-10T18:57:02.460160+0000","last_active":"2024-09-10T18:57:04.679755+0000","last_peered":"2024-09-10T18:57:04.679755+0000","last_clean":"2024-09-10T18:57:04.679755+0000","last_became_active":"2024-09-10T18:57:02.459708+0000","last_became_peered":"2024-09-10T18:57:02.459708+0000","last_unstale":"2024-09-10T18:57:04.679755+0000","last_undegraded":"2024-09-10T18:57:04.679755+0000","last_fullsized":"2024-09-10T18:57:04.679755+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:54:42.406761+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:54:42.406761+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037296899999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"232'35","reported_seq":321,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.660590+0000","last_change":"2024-09-10T18:56:28.475224+0000","last_active":"2024-09-10T18:57:04.660590+0000","last_peered":"2024-09-10T18:57:04.660590+0000","last_clean":"2024-09-10T18:57:04.660590+0000","last_became_active":"2024-09-10T18:45:20.052243+0000","last_became_peered":"2024-09-10T18:45:20.052243+0000","last_unstale":"2024-09-10T18:57:04.660590+0000","last_undegraded":"2024-09-10T18:57:04.660590+0000","last_fullsized":"2024-09-10T18:57:04.660590+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'35","last_scrub_stamp":"2024-09-10T18:56:28.475142+0000","last_deep_scrub":"143'21","last_deep_scrub_stamp":"2024-09-10T18:51:02.540216+0000","last_clean_scrub_stamp":"2024-09-10T18:56:28.475142+0000","objects_scrubbed":7,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:45.992194+0000","scrub_duration":0.0026978649999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":306,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.676508+0000","last_change":"2024-09-10T18:57:02.453766+0000","last_active":"2024-09-10T18:57:04.676508+0000","last_peered":"2024-09-10T18:57:04.676508+0000","last_clean":"2024-09-10T18:57:04.676508+0000","last_became_active":"2024-09-10T18:57:02.453404+0000","last_became_peered":"2024-09-10T18:57:02.453404+0000","last_unstale":"2024-09-10T18:57:04.676508+0000","last_undegraded":"2024-09-10T18:57:04.676508+0000","last_fullsized":"2024-09-10T18:57:04.676508+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:29.226803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_clean_scrub_stamp":"2024-09-10T18:56:29.226803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:44.655540+0000","scrub_duration":0.0086984899999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":424,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-10T18:57:12.170506+0000","last_change":"2024-09-10T18:56:17.914259+0000","last_active":"2024-09-10T18:57:12.170506+0000","last_peered":"2024-09-10T18:57:12.170506+0000","last_clean":"2024-09-10T18:57:12.170506+0000","last_became_active":"2024-09-10T18:45:22.074540+0000","last_became_peered":"2024-09-10T18:45:22.074540+0000","last_unstale":"2024-09-10T18:57:12.170506+0000","last_undegraded":"2024-09-10T18:57:12.170506+0000","last_fullsized":"2024-09-10T18:57:12.170506+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:56:17.914175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:17.914175+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:25.944051+0000","scrub_duration":0.0026666329999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":63,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.675923+0000","last_change":"2024-09-10T18:57:02.439140+0000","last_active":"2024-09-10T18:57:04.675923+0000","last_peered":"2024-09-10T18:57:04.675923+0000","last_clean":"2024-09-10T18:57:04.675923+0000","last_became_active":"2024-09-10T18:57:02.437100+0000","last_became_peered":"2024-09-10T18:57:02.437100+0000","last_unstale":"2024-09-10T18:57:04.675923+0000","last_undegraded":"2024-09-10T18:57:04.675923+0000","last_fullsized":"2024-09-10T18:57:04.675923+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:49.758989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:54:49.758989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013292270000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.d","version":"274'8257","reported_seq":12090,"reported_epoch":274,"state":"active+remapped+backfilling","last_fresh":"2024-09-10T18:57:16.569976+0000","last_change":"2024-09-10T18:57:03.488325+0000","last_active":"2024-09-10T18:57:16.569976+0000","last_peered":"2024-09-10T18:57:16.569976+0000","last_clean":"2024-09-10T18:57:01.349249+0000","last_became_active":"2024-09-10T18:57:03.483778+0000","last_became_peered":"2024-09-10T18:57:03.483778+0000","last_unstale":"2024-09-10T18:57:16.569976+0000","last_undegraded":"2024-09-10T18:57:16.569976+0000","last_fullsized":"2024-09-10T18:57:16.569976+0000","mapping_epoch":271,"log_start":"259'5251","ondisk_log_start":"259'5251","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3006,"log_dups_size":0,"ondisk_log_size":3006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":541130752,"num_objects":8257,"num_object_clones":0,"num_object_copies":24771,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6277,"num_objects_unfound":0,"num_objects_dirty":8257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8257,"num_write_kb":528448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1890,"num_bytes_recovered":123863040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,7],"avail_no_missing":["1","5","7"],"object_location_counts":[{"shards":"1,5,7","objects":8257}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"70'81","reported_seq":192,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.661410+0000","last_change":"2024-09-10T18:56:31.481362+0000","last_active":"2024-09-10T18:57:04.661410+0000","last_peered":"2024-09-10T18:57:04.661410+0000","last_clean":"2024-09-10T18:57:04.661410+0000","last_became_active":"2024-09-10T18:52:14.584798+0000","last_became_peered":"2024-09-10T18:52:14.584798+0000","last_unstale":"2024-09-10T18:57:04.661410+0000","last_undegraded":"2024-09-10T18:57:04.661410+0000","last_fullsized":"2024-09-10T18:57:04.661410+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:31.481303+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:56:31.481303+0000","last_clean_scrub_stamp":"2024-09-10T18:56:31.481303+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:39.982424+0000","scrub_duration":0.0011170959999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"232'30","reported_seq":384,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.670046+0000","last_change":"2024-09-10T18:57:02.490987+0000","last_active":"2024-09-10T18:57:04.670046+0000","last_peered":"2024-09-10T18:57:04.670046+0000","last_clean":"2024-09-10T18:57:04.670046+0000","last_became_active":"2024-09-10T18:57:02.455615+0000","last_became_peered":"2024-09-10T18:57:02.455615+0000","last_unstale":"2024-09-10T18:57:04.670046+0000","last_undegraded":"2024-09-10T18:57:04.670046+0000","last_fullsized":"2024-09-10T18:57:04.670046+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'30","last_scrub_stamp":"2024-09-10T18:56:15.875924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:15.875924+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:41.337141+0000","scrub_duration":0.0075153879999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":269,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.671163+0000","last_change":"2024-09-10T18:54:46.142189+0000","last_active":"2024-09-10T18:57:04.671163+0000","last_peered":"2024-09-10T18:57:04.671163+0000","last_clean":"2024-09-10T18:57:04.671163+0000","last_became_active":"2024-09-10T18:45:17.961695+0000","last_became_peered":"2024-09-10T18:45:17.961695+0000","last_unstale":"2024-09-10T18:57:04.671163+0000","last_undegraded":"2024-09-10T18:57:04.671163+0000","last_fullsized":"2024-09-10T18:57:04.671163+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:46.142129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:46.142129+0000","last_clean_scrub_stamp":"2024-09-10T18:54:46.142129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015906060000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":280,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.660359+0000","last_change":"2024-09-10T18:57:02.448422+0000","last_active":"2024-09-10T18:57:04.660359+0000","last_peered":"2024-09-10T18:57:04.660359+0000","last_clean":"2024-09-10T18:57:04.660359+0000","last_became_active":"2024-09-10T18:57:02.448126+0000","last_became_peered":"2024-09-10T18:57:02.448126+0000","last_unstale":"2024-09-10T18:57:04.660359+0000","last_undegraded":"2024-09-10T18:57:04.660359+0000","last_fullsized":"2024-09-10T18:57:04.660359+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:29.533177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:55:29.533177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012137999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":268,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.660889+0000","last_change":"2024-09-10T18:54:25.102613+0000","last_active":"2024-09-10T18:57:04.660889+0000","last_peered":"2024-09-10T18:57:04.660889+0000","last_clean":"2024-09-10T18:57:04.660889+0000","last_became_active":"2024-09-10T18:47:36.410836+0000","last_became_peered":"2024-09-10T18:47:36.410836+0000","last_unstale":"2024-09-10T18:57:04.660889+0000","last_undegraded":"2024-09-10T18:57:04.660889+0000","last_fullsized":"2024-09-10T18:57:04.660889+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:25.102358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_clean_scrub_stamp":"2024-09-10T18:54:25.102358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060015040000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.e","version":"274'8461","reported_seq":13056,"reported_epoch":274,"state":"active+remapped+backfilling","last_fresh":"2024-09-10T18:57:16.243676+0000","last_change":"2024-09-10T18:57:03.506052+0000","last_active":"2024-09-10T18:57:16.243676+0000","last_peered":"2024-09-10T18:57:16.243676+0000","last_clean":"2024-09-10T18:57:01.406214+0000","last_became_active":"2024-09-10T18:57:03.494480+0000","last_became_peered":"2024-09-10T18:57:03.494480+0000","last_unstale":"2024-09-10T18:57:16.243676+0000","last_undegraded":"2024-09-10T18:57:16.243676+0000","last_fullsized":"2024-09-10T18:57:16.243676+0000","mapping_epoch":271,"log_start":"258'4762","ondisk_log_start":"258'4762","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3699,"log_dups_size":0,"ondisk_log_size":3699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554500096,"num_objects":8461,"num_object_clones":0,"num_object_copies":25383,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6048,"num_objects_unfound":0,"num_objects_dirty":8461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8461,"num_write_kb":541504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2269,"num_bytes_recovered":148701184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,7],"avail_no_missing":["4","1","7"],"object_location_counts":[{"shards":"1,4,7","objects":8461}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"70'81","reported_seq":279,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.676185+0000","last_change":"2024-09-10T18:56:12.184278+0000","last_active":"2024-09-10T18:57:04.676185+0000","last_peered":"2024-09-10T18:57:04.676185+0000","last_clean":"2024-09-10T18:57:04.676185+0000","last_became_active":"2024-09-10T18:55:40.396828+0000","last_became_peered":"2024-09-10T18:55:40.396828+0000","last_unstale":"2024-09-10T18:57:04.676185+0000","last_undegraded":"2024-09-10T18:57:04.676185+0000","last_fullsized":"2024-09-10T18:57:04.676185+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:12.184127+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:12.184127+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:33.805848+0000","scrub_duration":0.0040368189999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"232'29","reported_seq":321,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.665095+0000","last_change":"2024-09-10T18:57:02.473700+0000","last_active":"2024-09-10T18:57:04.665095+0000","last_peered":"2024-09-10T18:57:04.665095+0000","last_clean":"2024-09-10T18:57:04.665095+0000","last_became_active":"2024-09-10T18:57:02.442862+0000","last_became_peered":"2024-09-10T18:57:02.442862+0000","last_unstale":"2024-09-10T18:57:04.665095+0000","last_undegraded":"2024-09-10T18:57:04.665095+0000","last_fullsized":"2024-09-10T18:57:04.665095+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:53:38.384804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:53:38.384804+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":259,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.666752+0000","last_change":"2024-09-10T18:57:02.444015+0000","last_active":"2024-09-10T18:57:04.666752+0000","last_peered":"2024-09-10T18:57:04.666752+0000","last_clean":"2024-09-10T18:57:04.666752+0000","last_became_active":"2024-09-10T18:57:02.443522+0000","last_became_peered":"2024-09-10T18:57:02.443522+0000","last_unstale":"2024-09-10T18:57:04.666752+0000","last_undegraded":"2024-09-10T18:57:04.666752+0000","last_fullsized":"2024-09-10T18:57:04.666752+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:01.022282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:48.196065+0000","last_clean_scrub_stamp":"2024-09-10T18:56:01.022282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:25.531126+0000","scrub_duration":0.0013037999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":289,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.675411+0000","last_change":"2024-09-10T18:56:02.162637+0000","last_active":"2024-09-10T18:57:04.675411+0000","last_peered":"2024-09-10T18:57:04.675411+0000","last_clean":"2024-09-10T18:57:04.675411+0000","last_became_active":"2024-09-10T18:55:40.387857+0000","last_became_peered":"2024-09-10T18:55:40.387857+0000","last_unstale":"2024-09-10T18:57:04.675411+0000","last_undegraded":"2024-09-10T18:57:04.675411+0000","last_fullsized":"2024-09-10T18:57:04.675411+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:02.162551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:02.162551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:15.298292+0000","scrub_duration":0.0027989849999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":280,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.663626+0000","last_change":"2024-09-10T18:57:02.445448+0000","last_active":"2024-09-10T18:57:04.663626+0000","last_peered":"2024-09-10T18:57:04.663626+0000","last_clean":"2024-09-10T18:57:04.663626+0000","last_became_active":"2024-09-10T18:57:02.442479+0000","last_became_peered":"2024-09-10T18:57:02.442479+0000","last_unstale":"2024-09-10T18:57:04.663626+0000","last_undegraded":"2024-09-10T18:57:04.663626+0000","last_fullsized":"2024-09-10T18:57:04.663626+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:30.482659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:49:18.281230+0000","last_clean_scrub_stamp":"2024-09-10T18:55:30.482659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015073689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.f","version":"274'8484","reported_seq":8529,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-10T18:57:16.417863+0000","last_change":"2024-09-10T18:55:36.358334+0000","last_active":"2024-09-10T18:57:16.417863+0000","last_peered":"2024-09-10T18:57:16.417863+0000","last_clean":"2024-09-10T18:57:16.417863+0000","last_became_active":"2024-09-10T18:55:36.358064+0000","last_became_peered":"2024-09-10T18:55:36.358064+0000","last_unstale":"2024-09-10T18:57:16.417863+0000","last_undegraded":"2024-09-10T18:57:16.417863+0000","last_fullsized":"2024-09-10T18:57:16.417863+0000","mapping_epoch":237,"log_start":"257'3669","ondisk_log_start":"257'3669","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":4815,"log_dups_size":0,"ondisk_log_size":4815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":556007424,"num_objects":8484,"num_object_clones":0,"num_object_copies":25452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8484,"num_write_kb":542976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"70'81","reported_seq":282,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.675900+0000","last_change":"2024-09-10T18:57:02.453204+0000","last_active":"2024-09-10T18:57:04.675900+0000","last_peered":"2024-09-10T18:57:04.675900+0000","last_clean":"2024-09-10T18:57:04.675900+0000","last_became_active":"2024-09-10T18:57:02.452659+0000","last_became_peered":"2024-09-10T18:57:02.452659+0000","last_unstale":"2024-09-10T18:57:04.675900+0000","last_undegraded":"2024-09-10T18:57:04.675900+0000","last_fullsized":"2024-09-10T18:57:04.675900+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:28.264003+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:28.264003+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:35.588963+0000","scrub_duration":0.001853637,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"232'25","reported_seq":361,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.664069+0000","last_change":"2024-09-10T18:56:27.835783+0000","last_active":"2024-09-10T18:57:04.664069+0000","last_peered":"2024-09-10T18:57:04.664069+0000","last_clean":"2024-09-10T18:57:04.664069+0000","last_became_active":"2024-09-10T18:55:40.400777+0000","last_became_peered":"2024-09-10T18:55:40.400777+0000","last_unstale":"2024-09-10T18:57:04.664069+0000","last_undegraded":"2024-09-10T18:57:04.664069+0000","last_fullsized":"2024-09-10T18:57:04.664069+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'25","last_scrub_stamp":"2024-09-10T18:56:27.835688+0000","last_deep_scrub":"83'15","last_deep_scrub_stamp":"2024-09-10T18:46:58.436473+0000","last_clean_scrub_stamp":"2024-09-10T18:56:27.835688+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:31.667888+0000","scrub_duration":0.0047372150000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":303,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.667229+0000","last_change":"2024-09-10T18:57:02.448298+0000","last_active":"2024-09-10T18:57:04.667229+0000","last_peered":"2024-09-10T18:57:04.667229+0000","last_clean":"2024-09-10T18:57:04.667229+0000","last_became_active":"2024-09-10T18:57:02.447940+0000","last_became_peered":"2024-09-10T18:57:02.447940+0000","last_unstale":"2024-09-10T18:57:04.667229+0000","last_undegraded":"2024-09-10T18:57:04.667229+0000","last_fullsized":"2024-09-10T18:57:04.667229+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:36.259868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:05.140027+0000","last_clean_scrub_stamp":"2024-09-10T18:53:36.259868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034952049999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"258'12","reported_seq":368,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-10T18:57:12.157201+0000","last_change":"2024-09-10T18:56:21.098913+0000","last_active":"2024-09-10T18:57:12.157201+0000","last_peered":"2024-09-10T18:57:12.157201+0000","last_clean":"2024-09-10T18:57:12.157201+0000","last_became_active":"2024-09-10T18:55:40.391094+0000","last_became_peered":"2024-09-10T18:55:40.391094+0000","last_unstale":"2024-09-10T18:57:12.157201+0000","last_undegraded":"2024-09-10T18:57:12.157201+0000","last_fullsized":"2024-09-10T18:57:12.157201+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'12","last_scrub_stamp":"2024-09-10T18:56:21.098770+0000","last_deep_scrub":"258'12","last_deep_scrub_stamp":"2024-09-10T18:56:21.098770+0000","last_clean_scrub_stamp":"2024-09-10T18:56:21.098770+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:32.905276+0000","scrub_duration":0.0034993049999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":245,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.665177+0000","last_change":"2024-09-10T18:57:03.353073+0000","last_active":"2024-09-10T18:57:04.665177+0000","last_peered":"2024-09-10T18:57:04.665177+0000","last_clean":"2024-09-10T18:57:04.665177+0000","last_became_active":"2024-09-10T18:55:40.397478+0000","last_became_peered":"2024-09-10T18:55:40.397478+0000","last_unstale":"2024-09-10T18:57:04.665177+0000","last_undegraded":"2024-09-10T18:57:04.665177+0000","last_fullsized":"2024-09-10T18:57:04.665177+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:57:03.353014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:57:03.353014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:58:32.307182+0000","scrub_duration":0.0014199639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.9","version":"274'8545","reported_seq":11204,"reported_epoch":274,"state":"active+remapped+backfilling","last_fresh":"2024-09-10T18:57:18.493924+0000","last_change":"2024-09-10T18:57:03.534667+0000","last_active":"2024-09-10T18:57:18.493924+0000","last_peered":"2024-09-10T18:57:18.493924+0000","last_clean":"2024-09-10T18:57:01.393416+0000","last_became_active":"2024-09-10T18:57:03.528506+0000","last_became_peered":"2024-09-10T18:57:03.528506+0000","last_unstale":"2024-09-10T18:57:18.493924+0000","last_undegraded":"2024-09-10T18:57:18.493924+0000","last_fullsized":"2024-09-10T18:57:18.493924+0000","mapping_epoch":271,"log_start":"258'5130","ondisk_log_start":"258'5130","created":237,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3415,"log_dups_size":0,"ondisk_log_size":3415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":559742976,"num_objects":8541,"num_object_clones":0,"num_object_copies":25623,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7159,"num_objects_unfound":0,"num_objects_dirty":8541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8545,"num_write_kb":546752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1302,"num_bytes_recovered":85327872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,7],"avail_no_missing":["0","5","7"],"object_location_counts":[{"shards":"0,5,7","objects":8541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"70'81","reported_seq":211,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.672613+0000","last_change":"2024-09-10T18:56:11.765628+0000","last_active":"2024-09-10T18:57:04.672613+0000","last_peered":"2024-09-10T18:57:04.672613+0000","last_clean":"2024-09-10T18:57:04.672613+0000","last_became_active":"2024-09-10T18:55:40.392432+0000","last_became_peered":"2024-09-10T18:55:40.392432+0000","last_unstale":"2024-09-10T18:57:04.672613+0000","last_undegraded":"2024-09-10T18:57:04.672613+0000","last_fullsized":"2024-09-10T18:57:04.672613+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:11.765541+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:11.765541+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:27.470910+0000","scrub_duration":0.002140374,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"232'41","reported_seq":516,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-10T18:57:12.159669+0000","last_change":"2024-09-10T18:56:18.672331+0000","last_active":"2024-09-10T18:57:12.159669+0000","last_peered":"2024-09-10T18:57:12.159669+0000","last_clean":"2024-09-10T18:57:12.159669+0000","last_became_active":"2024-09-10T18:47:36.420185+0000","last_became_peered":"2024-09-10T18:47:36.420185+0000","last_unstale":"2024-09-10T18:57:12.159669+0000","last_undegraded":"2024-09-10T18:57:12.159669+0000","last_fullsized":"2024-09-10T18:57:12.159669+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'41","last_scrub_stamp":"2024-09-10T18:56:18.672139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:18.672139+0000","objects_scrubbed":10,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:41.547337+0000","scrub_duration":0.020233260999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":279,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.671783+0000","last_change":"2024-09-10T18:55:29.867732+0000","last_active":"2024-09-10T18:57:04.671783+0000","last_peered":"2024-09-10T18:57:04.671783+0000","last_clean":"2024-09-10T18:57:04.671783+0000","last_became_active":"2024-09-10T18:45:17.964494+0000","last_became_peered":"2024-09-10T18:45:17.964494+0000","last_unstale":"2024-09-10T18:57:04.671783+0000","last_undegraded":"2024-09-10T18:57:04.671783+0000","last_fullsized":"2024-09-10T18:57:04.671783+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:29.867683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:12.982599+0000","last_clean_scrub_stamp":"2024-09-10T18:55:29.867683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015951909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":230,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.675760+0000","last_change":"2024-09-10T18:56:20.224668+0000","last_active":"2024-09-10T18:57:04.675760+0000","last_peered":"2024-09-10T18:57:04.675760+0000","last_clean":"2024-09-10T18:57:04.675760+0000","last_became_active":"2024-09-10T18:47:57.107478+0000","last_became_peered":"2024-09-10T18:47:57.107478+0000","last_unstale":"2024-09-10T18:57:04.675760+0000","last_undegraded":"2024-09-10T18:57:04.675760+0000","last_fullsized":"2024-09-10T18:57:04.675760+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:20.224562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:20.224562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:48.420895+0000","scrub_duration":0.008542315,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":270,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.675029+0000","last_change":"2024-09-10T18:57:02.462009+0000","last_active":"2024-09-10T18:57:04.675029+0000","last_peered":"2024-09-10T18:57:04.675029+0000","last_clean":"2024-09-10T18:57:04.675029+0000","last_became_active":"2024-09-10T18:57:02.460594+0000","last_became_peered":"2024-09-10T18:57:02.460594+0000","last_unstale":"2024-09-10T18:57:04.675029+0000","last_undegraded":"2024-09-10T18:57:04.675029+0000","last_fullsized":"2024-09-10T18:57:04.675029+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:11.878181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:22.891084+0000","last_clean_scrub_stamp":"2024-09-10T18:56:11.878181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:27.306269+0000","scrub_duration":0.0037255349999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"274'8523","reported_seq":8581,"reported_epoch":274,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-10T18:57:16.556569+0000","last_change":"2024-09-10T18:57:03.537753+0000","last_active":"2024-09-10T18:57:16.556569+0000","last_peered":"2024-09-10T18:57:16.556569+0000","last_clean":"2024-09-10T18:57:01.368396+0000","last_became_active":"2024-09-10T18:57:03.517319+0000","last_became_peered":"2024-09-10T18:57:03.517319+0000","last_unstale":"2024-09-10T18:57:16.556569+0000","last_undegraded":"2024-09-10T18:57:16.556569+0000","last_fullsized":"2024-09-10T18:57:16.556569+0000","mapping_epoch":271,"log_start":"259'5517","ondisk_log_start":"259'5517","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3006,"log_dups_size":0,"ondisk_log_size":3006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":558563328,"num_objects":8523,"num_object_clones":0,"num_object_copies":25569,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":8523,"num_objects_unfound":0,"num_objects_dirty":8523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8523,"num_write_kb":545472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,7],"avail_no_missing":["1","3","7"],"object_location_counts":[{"shards":"1,3,7","objects":8523}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"70'81","reported_seq":81,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.665287+0000","last_change":"2024-09-10T18:57:02.438557+0000","last_active":"2024-09-10T18:57:04.665287+0000","last_peered":"2024-09-10T18:57:04.665287+0000","last_clean":"2024-09-10T18:57:04.665287+0000","last_became_active":"2024-09-10T18:57:02.437944+0000","last_became_peered":"2024-09-10T18:57:02.437944+0000","last_unstale":"2024-09-10T18:57:04.665287+0000","last_undegraded":"2024-09-10T18:57:04.665287+0000","last_fullsized":"2024-09-10T18:57:04.665287+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:07.028031+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:07.028031+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019876809999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":5,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":53670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"232'22","reported_seq":326,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.671172+0000","last_change":"2024-09-10T18:56:05.550938+0000","last_active":"2024-09-10T18:57:04.671172+0000","last_peered":"2024-09-10T18:57:04.671172+0000","last_clean":"2024-09-10T18:57:04.671172+0000","last_became_active":"2024-09-10T18:55:40.397422+0000","last_became_peered":"2024-09-10T18:55:40.397422+0000","last_unstale":"2024-09-10T18:57:04.671172+0000","last_undegraded":"2024-09-10T18:57:04.671172+0000","last_fullsized":"2024-09-10T18:57:04.671172+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'22","last_scrub_stamp":"2024-09-10T18:56:05.550846+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-10T18:47:10.514951+0000","last_clean_scrub_stamp":"2024-09-10T18:56:05.550846+0000","objects_scrubbed":5,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:34.381969+0000","scrub_duration":0.0079381009999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"84'6","reported_seq":1012,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-10T18:57:14.748316+0000","last_change":"2024-09-10T18:56:22.799839+0000","last_active":"2024-09-10T18:57:14.748316+0000","last_peered":"2024-09-10T18:57:14.748316+0000","last_clean":"2024-09-10T18:57:14.748316+0000","last_became_active":"2024-09-10T18:55:40.403098+0000","last_became_peered":"2024-09-10T18:55:40.403098+0000","last_unstale":"2024-09-10T18:57:14.748316+0000","last_undegraded":"2024-09-10T18:57:14.748316+0000","last_fullsized":"2024-09-10T18:57:14.748316+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'6","last_scrub_stamp":"2024-09-10T18:56:22.799414+0000","last_deep_scrub":"84'6","last_deep_scrub_stamp":"2024-09-10T18:47:22.545450+0000","last_clean_scrub_stamp":"2024-09-10T18:56:22.799414+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:35.469589+0000","scrub_duration":0.004952756,"objects_trimmed":0,"snaptrim_duration":0.0021933159999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":702,"num_read_kb":697,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":244,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.670222+0000","last_change":"2024-09-10T18:56:03.791304+0000","last_active":"2024-09-10T18:57:04.670222+0000","last_peered":"2024-09-10T18:57:04.670222+0000","last_clean":"2024-09-10T18:57:04.670222+0000","last_became_active":"2024-09-10T18:55:40.393389+0000","last_became_peered":"2024-09-10T18:55:40.393389+0000","last_unstale":"2024-09-10T18:57:04.670222+0000","last_undegraded":"2024-09-10T18:57:04.670222+0000","last_fullsized":"2024-09-10T18:57:04.670222+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:03.791228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:03.791228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:15.631676+0000","scrub_duration":0.004016899,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":250,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.673055+0000","last_change":"2024-09-10T18:57:02.439247+0000","last_active":"2024-09-10T18:57:04.673055+0000","last_peered":"2024-09-10T18:57:04.673055+0000","last_clean":"2024-09-10T18:57:04.673055+0000","last_became_active":"2024-09-10T18:57:02.438945+0000","last_became_peered":"2024-09-10T18:57:02.438945+0000","last_unstale":"2024-09-10T18:57:04.673055+0000","last_undegraded":"2024-09-10T18:57:04.673055+0000","last_fullsized":"2024-09-10T18:57:04.673055+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:38.900146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:55:38.900146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048114799999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":16,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.676487+0000","last_change":"2024-09-10T18:57:02.454154+0000","last_active":"2024-09-10T18:57:04.676487+0000","last_peered":"2024-09-10T18:57:04.676487+0000","last_clean":"2024-09-10T18:57:04.676487+0000","last_became_active":"2024-09-10T18:57:02.453865+0000","last_became_peered":"2024-09-10T18:57:02.453865+0000","last_unstale":"2024-09-10T18:57:04.676487+0000","last_undegraded":"2024-09-10T18:57:04.676487+0000","last_fullsized":"2024-09-10T18:57:04.676487+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:21.086881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:07.150480+0000","last_clean_scrub_stamp":"2024-09-10T18:56:21.086881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:39.726811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.b","version":"274'8472","reported_seq":8514,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-10T18:57:18.495435+0000","last_change":"2024-09-10T18:55:40.391994+0000","last_active":"2024-09-10T18:57:18.495435+0000","last_peered":"2024-09-10T18:57:18.495435+0000","last_clean":"2024-09-10T18:57:18.495435+0000","last_became_active":"2024-09-10T18:55:40.391432+0000","last_became_peered":"2024-09-10T18:55:40.391432+0000","last_unstale":"2024-09-10T18:57:18.495435+0000","last_undegraded":"2024-09-10T18:57:18.495435+0000","last_fullsized":"2024-09-10T18:57:18.495435+0000","mapping_epoch":241,"log_start":"258'5022","ondisk_log_start":"258'5022","created":237,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3450,"log_dups_size":0,"ondisk_log_size":3450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554172416,"num_objects":8456,"num_object_clones":0,"num_object_copies":25368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8472,"num_write_kb":541696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"70'81","reported_seq":183,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.663280+0000","last_change":"2024-09-10T18:53:00.338829+0000","last_active":"2024-09-10T18:57:04.663280+0000","last_peered":"2024-09-10T18:57:04.663280+0000","last_clean":"2024-09-10T18:57:04.663280+0000","last_became_active":"2024-09-10T18:52:14.589009+0000","last_became_peered":"2024-09-10T18:52:14.589009+0000","last_unstale":"2024-09-10T18:57:04.663280+0000","last_undegraded":"2024-09-10T18:57:04.663280+0000","last_fullsized":"2024-09-10T18:57:04.663280+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:53:00.338611+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:53:00.338611+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023458709999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"232'24","reported_seq":332,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.665928+0000","last_change":"2024-09-10T18:57:02.478395+0000","last_active":"2024-09-10T18:57:04.665928+0000","last_peered":"2024-09-10T18:57:04.665928+0000","last_clean":"2024-09-10T18:57:04.665928+0000","last_became_active":"2024-09-10T18:57:02.447673+0000","last_became_peered":"2024-09-10T18:57:02.447673+0000","last_unstale":"2024-09-10T18:57:04.665928+0000","last_undegraded":"2024-09-10T18:57:04.665928+0000","last_fullsized":"2024-09-10T18:57:04.665928+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T18:56:30.183414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:30.183414+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:45.278323+0000","scrub_duration":0.0047701100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"58'1","reported_seq":247,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.666967+0000","last_change":"2024-09-10T18:56:06.131199+0000","last_active":"2024-09-10T18:57:04.666967+0000","last_peered":"2024-09-10T18:57:04.666967+0000","last_clean":"2024-09-10T18:57:04.666967+0000","last_became_active":"2024-09-10T18:55:40.390022+0000","last_became_peered":"2024-09-10T18:55:40.390022+0000","last_unstale":"2024-09-10T18:57:04.666967+0000","last_undegraded":"2024-09-10T18:57:04.666967+0000","last_fullsized":"2024-09-10T18:57:04.666967+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'1","last_scrub_stamp":"2024-09-10T18:56:06.131098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:06.131098+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016770345999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":217,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.662934+0000","last_change":"2024-09-10T18:57:02.447491+0000","last_active":"2024-09-10T18:57:04.662934+0000","last_peered":"2024-09-10T18:57:04.662934+0000","last_clean":"2024-09-10T18:57:04.662934+0000","last_became_active":"2024-09-10T18:57:02.447195+0000","last_became_peered":"2024-09-10T18:57:02.447195+0000","last_unstale":"2024-09-10T18:57:04.662934+0000","last_undegraded":"2024-09-10T18:57:04.662934+0000","last_fullsized":"2024-09-10T18:57:04.662934+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:24.153973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:24.153973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:41.056293+0000","scrub_duration":0.001771613,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":129,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.668818+0000","last_change":"2024-09-10T18:56:30.801802+0000","last_active":"2024-09-10T18:57:04.668818+0000","last_peered":"2024-09-10T18:57:04.668818+0000","last_clean":"2024-09-10T18:57:04.668818+0000","last_became_active":"2024-09-10T18:55:40.392494+0000","last_became_peered":"2024-09-10T18:55:40.392494+0000","last_unstale":"2024-09-10T18:57:04.668818+0000","last_undegraded":"2024-09-10T18:57:04.668818+0000","last_fullsized":"2024-09-10T18:57:04.668818+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:30.801727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:30.801727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:36.335518+0000","scrub_duration":0.0027881669999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":263,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.666832+0000","last_change":"2024-09-10T18:54:17.615248+0000","last_active":"2024-09-10T18:57:04.666832+0000","last_peered":"2024-09-10T18:57:04.666832+0000","last_clean":"2024-09-10T18:57:04.666832+0000","last_became_active":"2024-09-10T18:45:24.083689+0000","last_became_peered":"2024-09-10T18:45:24.083689+0000","last_unstale":"2024-09-10T18:57:04.666832+0000","last_undegraded":"2024-09-10T18:57:04.666832+0000","last_fullsized":"2024-09-10T18:57:04.666832+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:17.615191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:54:17.615191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011240523000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"274'8479","reported_seq":8545,"reported_epoch":274,"state":"active+remapped+backfill_wait","last_fresh":"2024-09-10T18:57:16.715692+0000","last_change":"2024-09-10T18:57:03.591383+0000","last_active":"2024-09-10T18:57:16.715692+0000","last_peered":"2024-09-10T18:57:16.715692+0000","last_clean":"2024-09-10T18:57:01.434103+0000","last_became_active":"2024-09-10T18:57:03.556257+0000","last_became_peered":"2024-09-10T18:57:03.556257+0000","last_unstale":"2024-09-10T18:57:16.715692+0000","last_undegraded":"2024-09-10T18:57:16.715692+0000","last_fullsized":"2024-09-10T18:57:16.715692+0000","mapping_epoch":271,"log_start":"258'5038","ondisk_log_start":"258'5038","created":237,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3441,"log_dups_size":0,"ondisk_log_size":3441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":555679744,"num_objects":8479,"num_object_clones":0,"num_object_copies":25437,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":8479,"num_objects_unfound":0,"num_objects_dirty":8479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8479,"num_write_kb":542656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,7],"avail_no_missing":["3","0","7"],"object_location_counts":[{"shards":"0,3,7","objects":8479}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"232'33","reported_seq":357,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.676558+0000","last_change":"2024-09-10T18:57:02.468556+0000","last_active":"2024-09-10T18:57:04.676558+0000","last_peered":"2024-09-10T18:57:04.676558+0000","last_clean":"2024-09-10T18:57:04.676558+0000","last_became_active":"2024-09-10T18:57:02.435499+0000","last_became_peered":"2024-09-10T18:57:02.435499+0000","last_unstale":"2024-09-10T18:57:04.676558+0000","last_undegraded":"2024-09-10T18:57:04.676558+0000","last_fullsized":"2024-09-10T18:57:04.676558+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'33","last_scrub_stamp":"2024-09-10T18:56:21.206960+0000","last_deep_scrub":"232'33","last_deep_scrub_stamp":"2024-09-10T18:56:21.206960+0000","last_clean_scrub_stamp":"2024-09-10T18:56:21.206960+0000","objects_scrubbed":7,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:25.204748+0000","scrub_duration":0.0042638809999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"60'2","reported_seq":279,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.676060+0000","last_change":"2024-09-10T18:54:19.313557+0000","last_active":"2024-09-10T18:57:04.676060+0000","last_peered":"2024-09-10T18:57:04.676060+0000","last_clean":"2024-09-10T18:57:04.676060+0000","last_became_active":"2024-09-10T18:45:17.961660+0000","last_became_peered":"2024-09-10T18:45:17.961660+0000","last_unstale":"2024-09-10T18:57:04.676060+0000","last_undegraded":"2024-09-10T18:57:04.676060+0000","last_fullsized":"2024-09-10T18:57:04.676060+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-10T18:54:19.313386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:54:19.313386+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056941359999999998,"objects_trimmed":0,"snaptrim_duration":0.0013942640000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":306,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.671975+0000","last_change":"2024-09-10T18:57:02.446028+0000","last_active":"2024-09-10T18:57:04.671975+0000","last_peered":"2024-09-10T18:57:04.671975+0000","last_clean":"2024-09-10T18:57:04.671975+0000","last_became_active":"2024-09-10T18:57:02.445712+0000","last_became_peered":"2024-09-10T18:57:02.445712+0000","last_unstale":"2024-09-10T18:57:04.671975+0000","last_undegraded":"2024-09-10T18:57:04.671975+0000","last_fullsized":"2024-09-10T18:57:04.671975+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:33.559651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:33.559651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:53.748604+0000","scrub_duration":0.0036787759999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'81","reported_seq":81,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.664414+0000","last_change":"2024-09-10T18:57:02.440391+0000","last_active":"2024-09-10T18:57:04.664414+0000","last_peered":"2024-09-10T18:57:04.664414+0000","last_clean":"2024-09-10T18:57:04.664414+0000","last_became_active":"2024-09-10T18:57:02.440063+0000","last_became_peered":"2024-09-10T18:57:02.440063+0000","last_unstale":"2024-09-10T18:57:04.664414+0000","last_undegraded":"2024-09-10T18:57:04.664414+0000","last_fullsized":"2024-09-10T18:57:04.664414+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:11.056156+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:56.085580+0000","last_clean_scrub_stamp":"2024-09-10T18:56:11.056156+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019876809999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26834,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":9,"num_write":5,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":53669,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":275,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.672024+0000","last_change":"2024-09-10T18:57:02.432025+0000","last_active":"2024-09-10T18:57:04.672024+0000","last_peered":"2024-09-10T18:57:04.672024+0000","last_clean":"2024-09-10T18:57:04.672024+0000","last_became_active":"2024-09-10T18:57:02.431556+0000","last_became_peered":"2024-09-10T18:57:02.431556+0000","last_unstale":"2024-09-10T18:57:04.672024+0000","last_undegraded":"2024-09-10T18:57:04.672024+0000","last_fullsized":"2024-09-10T18:57:04.672024+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:27.012123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:27.012123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:47.899019+0000","scrub_duration":0.007606036,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":280,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.673763+0000","last_change":"2024-09-10T18:56:34.565177+0000","last_active":"2024-09-10T18:57:04.673763+0000","last_peered":"2024-09-10T18:57:04.673763+0000","last_clean":"2024-09-10T18:57:04.673763+0000","last_became_active":"2024-09-10T18:45:24.073327+0000","last_became_peered":"2024-09-10T18:45:24.073327+0000","last_unstale":"2024-09-10T18:57:04.673763+0000","last_undegraded":"2024-09-10T18:57:04.673763+0000","last_fullsized":"2024-09-10T18:57:04.673763+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:34.564993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:34.564993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:38.282914+0000","scrub_duration":0.0031904390000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.7","version":"274'8256","reported_seq":11216,"reported_epoch":274,"state":"active+remapped+backfilling","last_fresh":"2024-09-10T18:57:16.436843+0000","last_change":"2024-09-10T18:57:03.580330+0000","last_active":"2024-09-10T18:57:16.436843+0000","last_peered":"2024-09-10T18:57:16.436843+0000","last_clean":"2024-09-10T18:57:01.402074+0000","last_became_active":"2024-09-10T18:57:03.547000+0000","last_became_peered":"2024-09-10T18:57:03.547000+0000","last_unstale":"2024-09-10T18:57:16.436843+0000","last_undegraded":"2024-09-10T18:57:16.436843+0000","last_fullsized":"2024-09-10T18:57:16.436843+0000","mapping_epoch":271,"log_start":"256'3475","ondisk_log_start":"256'3475","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":4781,"log_dups_size":0,"ondisk_log_size":4781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":541065216,"num_objects":8256,"num_object_clones":0,"num_object_copies":24768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6723,"num_objects_unfound":0,"num_objects_dirty":8256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8256,"num_write_kb":528384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1452,"num_bytes_recovered":95158272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,7],"avail_no_missing":["5","3","7"],"object_location_counts":[{"shards":"3,5,7","objects":8256}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.f","version":"70'81","reported_seq":274,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.676258+0000","last_change":"2024-09-10T18:55:29.018036+0000","last_active":"2024-09-10T18:57:04.676258+0000","last_peered":"2024-09-10T18:57:04.676258+0000","last_clean":"2024-09-10T18:57:04.676258+0000","last_became_active":"2024-09-10T18:52:16.598295+0000","last_became_peered":"2024-09-10T18:52:16.598295+0000","last_unstale":"2024-09-10T18:57:04.676258+0000","last_undegraded":"2024-09-10T18:57:04.676258+0000","last_fullsized":"2024-09-10T18:57:04.676258+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:55:29.017936+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:53.609885+0000","last_clean_scrub_stamp":"2024-09-10T18:55:29.017936+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001646813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"232'35","reported_seq":372,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.675628+0000","last_change":"2024-09-10T18:57:02.466701+0000","last_active":"2024-09-10T18:57:04.675628+0000","last_peered":"2024-09-10T18:57:04.675628+0000","last_clean":"2024-09-10T18:57:04.675628+0000","last_became_active":"2024-09-10T18:57:02.436059+0000","last_became_peered":"2024-09-10T18:57:02.436059+0000","last_unstale":"2024-09-10T18:57:04.675628+0000","last_undegraded":"2024-09-10T18:57:04.675628+0000","last_fullsized":"2024-09-10T18:57:04.675628+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'35","last_scrub_stamp":"2024-09-10T18:56:34.373736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:34.373736+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:39.907320+0000","scrub_duration":0.016527315000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":254,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.672095+0000","last_change":"2024-09-10T18:56:13.760318+0000","last_active":"2024-09-10T18:57:04.672095+0000","last_peered":"2024-09-10T18:57:04.672095+0000","last_clean":"2024-09-10T18:57:04.672095+0000","last_became_active":"2024-09-10T18:55:40.391849+0000","last_became_peered":"2024-09-10T18:55:40.391849+0000","last_unstale":"2024-09-10T18:57:04.672095+0000","last_undegraded":"2024-09-10T18:57:04.672095+0000","last_fullsized":"2024-09-10T18:57:04.672095+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:13.760238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:13.760238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:40.211689+0000","scrub_duration":0.048619793000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":214,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.664861+0000","last_change":"2024-09-10T18:57:02.451716+0000","last_active":"2024-09-10T18:57:04.664861+0000","last_peered":"2024-09-10T18:57:04.664861+0000","last_clean":"2024-09-10T18:57:04.664861+0000","last_became_active":"2024-09-10T18:57:02.451332+0000","last_became_peered":"2024-09-10T18:57:02.451332+0000","last_unstale":"2024-09-10T18:57:04.664861+0000","last_undegraded":"2024-09-10T18:57:04.664861+0000","last_fullsized":"2024-09-10T18:57:04.664861+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:24.158403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:24.158403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:52.796397+0000","scrub_duration":0.02305596,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":73,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.669651+0000","last_change":"2024-09-10T18:57:03.015513+0000","last_active":"2024-09-10T18:57:04.669651+0000","last_peered":"2024-09-10T18:57:04.669651+0000","last_clean":"2024-09-10T18:57:04.669651+0000","last_became_active":"2024-09-10T18:57:02.468866+0000","last_became_peered":"2024-09-10T18:57:02.468866+0000","last_unstale":"2024-09-10T18:57:04.669651+0000","last_undegraded":"2024-09-10T18:57:04.669651+0000","last_fullsized":"2024-09-10T18:57:04.669651+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:57:03.015457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:57:03.015457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:58:11.403200+0000","scrub_duration":0.001545795,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.6","version":"274'8507","reported_seq":8552,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-10T18:57:16.415230+0000","last_change":"2024-09-10T18:55:36.358103+0000","last_active":"2024-09-10T18:57:16.415230+0000","last_peered":"2024-09-10T18:57:16.415230+0000","last_clean":"2024-09-10T18:57:16.415230+0000","last_became_active":"2024-09-10T18:55:36.357828+0000","last_became_peered":"2024-09-10T18:55:36.357828+0000","last_unstale":"2024-09-10T18:57:16.415230+0000","last_undegraded":"2024-09-10T18:57:16.415230+0000","last_fullsized":"2024-09-10T18:57:16.415230+0000","mapping_epoch":237,"log_start":"256'3727","ondisk_log_start":"256'3727","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":4780,"log_dups_size":0,"ondisk_log_size":4780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":557514752,"num_objects":8507,"num_object_clones":0,"num_object_copies":25521,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8507,"num_write_kb":544448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.e","version":"70'81","reported_seq":143,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.663532+0000","last_change":"2024-09-10T18:57:02.450954+0000","last_active":"2024-09-10T18:57:04.663532+0000","last_peered":"2024-09-10T18:57:04.663532+0000","last_clean":"2024-09-10T18:57:04.663532+0000","last_became_active":"2024-09-10T18:57:02.449115+0000","last_became_peered":"2024-09-10T18:57:02.449115+0000","last_unstale":"2024-09-10T18:57:04.663532+0000","last_undegraded":"2024-09-10T18:57:04.663532+0000","last_fullsized":"2024-09-10T18:57:04.663532+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:55:30.496910+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:55:30.496910+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012110790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"232'19","reported_seq":280,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.665375+0000","last_change":"2024-09-10T18:56:20.061554+0000","last_active":"2024-09-10T18:57:04.665375+0000","last_peered":"2024-09-10T18:57:04.665375+0000","last_clean":"2024-09-10T18:57:04.665375+0000","last_became_active":"2024-09-10T18:55:40.395257+0000","last_became_peered":"2024-09-10T18:55:40.395257+0000","last_unstale":"2024-09-10T18:57:04.665375+0000","last_undegraded":"2024-09-10T18:57:04.665375+0000","last_fullsized":"2024-09-10T18:57:04.665375+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T18:56:20.061420+0000","last_deep_scrub":"130'13","last_deep_scrub_stamp":"2024-09-10T18:51:58.427534+0000","last_clean_scrub_stamp":"2024-09-10T18:56:20.061420+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:44.377480+0000","scrub_duration":0.0065809780000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":266,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.669324+0000","last_change":"2024-09-10T18:54:52.179168+0000","last_active":"2024-09-10T18:57:04.669324+0000","last_peered":"2024-09-10T18:57:04.669324+0000","last_clean":"2024-09-10T18:57:04.669324+0000","last_became_active":"2024-09-10T18:45:17.963057+0000","last_became_peered":"2024-09-10T18:45:17.963057+0000","last_unstale":"2024-09-10T18:57:04.669324+0000","last_undegraded":"2024-09-10T18:57:04.669324+0000","last_fullsized":"2024-09-10T18:57:04.669324+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:52.179074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:54:52.179074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001414738,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":242,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.660798+0000","last_change":"2024-09-10T18:56:17.661917+0000","last_active":"2024-09-10T18:57:04.660798+0000","last_peered":"2024-09-10T18:57:04.660798+0000","last_clean":"2024-09-10T18:57:04.660798+0000","last_became_active":"2024-09-10T18:55:40.390771+0000","last_became_peered":"2024-09-10T18:55:40.390771+0000","last_unstale":"2024-09-10T18:57:04.660798+0000","last_undegraded":"2024-09-10T18:57:04.660798+0000","last_fullsized":"2024-09-10T18:57:04.660798+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:17.661832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_clean_scrub_stamp":"2024-09-10T18:56:17.661832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:37.630708+0000","scrub_duration":0.0041460890000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":268,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.666391+0000","last_change":"2024-09-10T18:57:02.453927+0000","last_active":"2024-09-10T18:57:04.666391+0000","last_peered":"2024-09-10T18:57:04.666391+0000","last_clean":"2024-09-10T18:57:04.666391+0000","last_became_active":"2024-09-10T18:57:02.452387+0000","last_became_peered":"2024-09-10T18:57:02.452387+0000","last_unstale":"2024-09-10T18:57:04.666391+0000","last_undegraded":"2024-09-10T18:57:04.666391+0000","last_fullsized":"2024-09-10T18:57:04.666391+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:32.217641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:53.345184+0000","last_clean_scrub_stamp":"2024-09-10T18:56:32.217641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:45.792578+0000","scrub_duration":0.0059447470000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.5","version":"274'8494","reported_seq":11788,"reported_epoch":274,"state":"active+remapped+backfilling","last_fresh":"2024-09-10T18:57:18.499062+0000","last_change":"2024-09-10T18:57:03.528081+0000","last_active":"2024-09-10T18:57:18.499062+0000","last_peered":"2024-09-10T18:57:18.499062+0000","last_clean":"2024-09-10T18:57:01.419226+0000","last_became_active":"2024-09-10T18:57:03.516871+0000","last_became_peered":"2024-09-10T18:57:03.516871+0000","last_unstale":"2024-09-10T18:57:18.499062+0000","last_undegraded":"2024-09-10T18:57:18.499062+0000","last_fullsized":"2024-09-10T18:57:18.499062+0000","mapping_epoch":271,"log_start":"258'5052","ondisk_log_start":"258'5052","created":237,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3442,"log_dups_size":0,"ondisk_log_size":3442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":555220992,"num_objects":8472,"num_object_clones":0,"num_object_copies":25416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6745,"num_objects_unfound":0,"num_objects_dirty":8472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8494,"num_write_kb":542912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1620,"num_bytes_recovered":106168320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,7],"avail_no_missing":["0","1","7"],"object_location_counts":[{"shards":"0,1,7","objects":8472}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.d","version":"70'81","reported_seq":81,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.661771+0000","last_change":"2024-09-10T18:57:02.439985+0000","last_active":"2024-09-10T18:57:04.661771+0000","last_peered":"2024-09-10T18:57:04.661771+0000","last_clean":"2024-09-10T18:57:04.661771+0000","last_became_active":"2024-09-10T18:57:02.439652+0000","last_became_peered":"2024-09-10T18:57:02.439652+0000","last_unstale":"2024-09-10T18:57:04.661771+0000","last_undegraded":"2024-09-10T18:57:04.661771+0000","last_fullsized":"2024-09-10T18:57:04.661771+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:18.041116+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:18.041116+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014613810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"232'27","reported_seq":175,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.675545+0000","last_change":"2024-09-10T18:57:02.491285+0000","last_active":"2024-09-10T18:57:04.675545+0000","last_peered":"2024-09-10T18:57:04.675545+0000","last_clean":"2024-09-10T18:57:04.675545+0000","last_became_active":"2024-09-10T18:57:02.454653+0000","last_became_peered":"2024-09-10T18:57:02.454653+0000","last_unstale":"2024-09-10T18:57:04.675545+0000","last_undegraded":"2024-09-10T18:57:04.675545+0000","last_fullsized":"2024-09-10T18:57:04.675545+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:54:22.720737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:54:22.720737+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014176382,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":33,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3],"acting":[6,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":237,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.675165+0000","last_change":"2024-09-10T18:57:02.463092+0000","last_active":"2024-09-10T18:57:04.675165+0000","last_peered":"2024-09-10T18:57:04.675165+0000","last_clean":"2024-09-10T18:57:04.675165+0000","last_became_active":"2024-09-10T18:57:02.461708+0000","last_became_peered":"2024-09-10T18:57:02.461708+0000","last_unstale":"2024-09-10T18:57:04.675165+0000","last_undegraded":"2024-09-10T18:57:04.675165+0000","last_fullsized":"2024-09-10T18:57:04.675165+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:24.902205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:05.603125+0000","last_clean_scrub_stamp":"2024-09-10T18:56:24.902205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:26.437336+0000","scrub_duration":0.0076874830000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"273'10","reported_seq":91,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-10T18:57:12.103601+0000","last_change":"2024-09-10T18:57:02.476362+0000","last_active":"2024-09-10T18:57:12.103601+0000","last_peered":"2024-09-10T18:57:12.103601+0000","last_clean":"2024-09-10T18:57:12.103601+0000","last_became_active":"2024-09-10T18:57:02.441890+0000","last_became_peered":"2024-09-10T18:57:02.441890+0000","last_unstale":"2024-09-10T18:57:12.103601+0000","last_undegraded":"2024-09-10T18:57:12.103601+0000","last_fullsized":"2024-09-10T18:57:12.103601+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'9","last_scrub_stamp":"2024-09-10T18:55:37.750784+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-09-10T18:47:02.582229+0000","last_clean_scrub_stamp":"2024-09-10T18:55:37.750784+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":287,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.671464+0000","last_change":"2024-09-10T18:56:10.775911+0000","last_active":"2024-09-10T18:57:04.671464+0000","last_peered":"2024-09-10T18:57:04.671464+0000","last_clean":"2024-09-10T18:57:04.671464+0000","last_became_active":"2024-09-10T18:55:40.389202+0000","last_became_peered":"2024-09-10T18:55:40.389202+0000","last_unstale":"2024-09-10T18:57:04.671464+0000","last_undegraded":"2024-09-10T18:57:04.671464+0000","last_fullsized":"2024-09-10T18:57:04.671464+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:10.775814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_clean_scrub_stamp":"2024-09-10T18:56:10.775814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0031802459999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.4","version":"274'8386","reported_seq":8431,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-10T18:57:16.553652+0000","last_change":"2024-09-10T18:55:36.350444+0000","last_active":"2024-09-10T18:57:16.553652+0000","last_peered":"2024-09-10T18:57:16.553652+0000","last_clean":"2024-09-10T18:57:16.553652+0000","last_became_active":"2024-09-10T18:55:36.350155+0000","last_became_peered":"2024-09-10T18:55:36.350155+0000","last_unstale":"2024-09-10T18:57:16.553652+0000","last_undegraded":"2024-09-10T18:57:16.553652+0000","last_fullsized":"2024-09-10T18:57:16.553652+0000","mapping_epoch":237,"log_start":"259'5312","ondisk_log_start":"259'5312","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3074,"log_dups_size":0,"ondisk_log_size":3074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":549584896,"num_objects":8386,"num_object_clones":0,"num_object_copies":25158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8386,"num_write_kb":536704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.c","version":"70'81","reported_seq":230,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.676472+0000","last_change":"2024-09-10T18:56:33.779212+0000","last_active":"2024-09-10T18:57:04.676472+0000","last_peered":"2024-09-10T18:57:04.676472+0000","last_clean":"2024-09-10T18:57:04.676472+0000","last_became_active":"2024-09-10T18:55:40.390655+0000","last_became_peered":"2024-09-10T18:55:40.390655+0000","last_unstale":"2024-09-10T18:57:04.676472+0000","last_undegraded":"2024-09-10T18:57:04.676472+0000","last_fullsized":"2024-09-10T18:57:04.676472+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:33.779154+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:56:33.779154+0000","last_clean_scrub_stamp":"2024-09-10T18:56:33.779154+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:54.692016+0000","scrub_duration":0.0075121179999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"232'18","reported_seq":304,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.660506+0000","last_change":"2024-09-10T18:55:27.507437+0000","last_active":"2024-09-10T18:57:04.660506+0000","last_peered":"2024-09-10T18:57:04.660506+0000","last_clean":"2024-09-10T18:57:04.660506+0000","last_became_active":"2024-09-10T18:45:20.056869+0000","last_became_peered":"2024-09-10T18:45:20.056869+0000","last_unstale":"2024-09-10T18:57:04.660506+0000","last_undegraded":"2024-09-10T18:57:04.660506+0000","last_fullsized":"2024-09-10T18:57:04.660506+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'18","last_scrub_stamp":"2024-09-10T18:55:27.507364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:55:27.507364+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030221390000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"56'2","reported_seq":314,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.661892+0000","last_change":"2024-09-10T18:57:02.456385+0000","last_active":"2024-09-10T18:57:04.661892+0000","last_peered":"2024-09-10T18:57:04.661892+0000","last_clean":"2024-09-10T18:57:04.661892+0000","last_became_active":"2024-09-10T18:57:02.434475+0000","last_became_peered":"2024-09-10T18:57:02.434475+0000","last_unstale":"2024-09-10T18:57:04.661892+0000","last_undegraded":"2024-09-10T18:57:04.661892+0000","last_fullsized":"2024-09-10T18:57:04.661892+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-10T18:56:01.630091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:01.630091+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001578238,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":214,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.665350+0000","last_change":"2024-09-10T18:56:16.019721+0000","last_active":"2024-09-10T18:57:04.665350+0000","last_peered":"2024-09-10T18:57:04.665350+0000","last_clean":"2024-09-10T18:57:04.665350+0000","last_became_active":"2024-09-10T18:55:40.397290+0000","last_became_peered":"2024-09-10T18:55:40.397290+0000","last_unstale":"2024-09-10T18:57:04.665350+0000","last_undegraded":"2024-09-10T18:57:04.665350+0000","last_fullsized":"2024-09-10T18:57:04.665350+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:16.019357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:02.711594+0000","last_clean_scrub_stamp":"2024-09-10T18:56:16.019357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:32.894810+0000","scrub_duration":0.0021619629999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":270,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.674211+0000","last_change":"2024-09-10T18:57:02.461727+0000","last_active":"2024-09-10T18:57:04.674211+0000","last_peered":"2024-09-10T18:57:04.674211+0000","last_clean":"2024-09-10T18:57:04.674211+0000","last_became_active":"2024-09-10T18:57:02.459985+0000","last_became_peered":"2024-09-10T18:57:02.459985+0000","last_unstale":"2024-09-10T18:57:04.674211+0000","last_undegraded":"2024-09-10T18:57:04.674211+0000","last_fullsized":"2024-09-10T18:57:04.674211+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:53.352427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:53.352427+0000","last_clean_scrub_stamp":"2024-09-10T18:54:53.352427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022835899999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.13","version":"70'81","reported_seq":202,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.663426+0000","last_change":"2024-09-10T18:56:34.760114+0000","last_active":"2024-09-10T18:57:04.663426+0000","last_peered":"2024-09-10T18:57:04.663426+0000","last_clean":"2024-09-10T18:57:04.663426+0000","last_became_active":"2024-09-10T18:52:16.602463+0000","last_became_peered":"2024-09-10T18:52:16.602463+0000","last_unstale":"2024-09-10T18:57:04.663426+0000","last_undegraded":"2024-09-10T18:57:04.663426+0000","last_fullsized":"2024-09-10T18:57:04.663426+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:34.760031+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:34.760031+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:37.136778+0000","scrub_duration":0.002735595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"232'19","reported_seq":303,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.665864+0000","last_change":"2024-09-10T18:57:02.485836+0000","last_active":"2024-09-10T18:57:04.665864+0000","last_peered":"2024-09-10T18:57:04.665864+0000","last_clean":"2024-09-10T18:57:04.665864+0000","last_became_active":"2024-09-10T18:57:02.453001+0000","last_became_peered":"2024-09-10T18:57:02.453001+0000","last_unstale":"2024-09-10T18:57:04.665864+0000","last_undegraded":"2024-09-10T18:57:04.665864+0000","last_fullsized":"2024-09-10T18:57:04.665864+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T18:56:25.122070+0000","last_deep_scrub":"76'13","last_deep_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_clean_scrub_stamp":"2024-09-10T18:56:25.122070+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:51.452723+0000","scrub_duration":0.0061142590000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":254,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.662799+0000","last_change":"2024-09-10T18:56:25.820426+0000","last_active":"2024-09-10T18:57:04.662799+0000","last_peered":"2024-09-10T18:57:04.662799+0000","last_clean":"2024-09-10T18:57:04.662799+0000","last_became_active":"2024-09-10T18:55:40.401037+0000","last_became_peered":"2024-09-10T18:55:40.401037+0000","last_unstale":"2024-09-10T18:57:04.662799+0000","last_undegraded":"2024-09-10T18:57:04.662799+0000","last_fullsized":"2024-09-10T18:57:04.662799+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:25.820339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:25.820339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:27.095822+0000","scrub_duration":0.0048533279999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":282,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.674924+0000","last_change":"2024-09-10T18:57:02.455638+0000","last_active":"2024-09-10T18:57:04.674924+0000","last_peered":"2024-09-10T18:57:04.674924+0000","last_clean":"2024-09-10T18:57:04.674924+0000","last_became_active":"2024-09-10T18:57:02.455341+0000","last_became_peered":"2024-09-10T18:57:02.455341+0000","last_unstale":"2024-09-10T18:57:04.674924+0000","last_undegraded":"2024-09-10T18:57:04.674924+0000","last_fullsized":"2024-09-10T18:57:04.674924+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:25.862853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:00.754912+0000","last_clean_scrub_stamp":"2024-09-10T18:56:25.862853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:33.286261+0000","scrub_duration":0.0033952399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"63'1","reported_seq":272,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.661390+0000","last_change":"2024-09-10T18:57:02.456685+0000","last_active":"2024-09-10T18:57:04.661390+0000","last_peered":"2024-09-10T18:57:04.661390+0000","last_clean":"2024-09-10T18:57:04.661390+0000","last_became_active":"2024-09-10T18:57:02.435896+0000","last_became_peered":"2024-09-10T18:57:02.435896+0000","last_unstale":"2024-09-10T18:57:04.661390+0000","last_undegraded":"2024-09-10T18:57:04.661390+0000","last_fullsized":"2024-09-10T18:57:04.661390+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-10T18:56:07.774334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:07.774334+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.090386770000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.12","version":"70'81","reported_seq":134,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.671557+0000","last_change":"2024-09-10T18:54:50.968303+0000","last_active":"2024-09-10T18:57:04.671557+0000","last_peered":"2024-09-10T18:57:04.671557+0000","last_clean":"2024-09-10T18:57:04.671557+0000","last_became_active":"2024-09-10T18:52:16.598921+0000","last_became_peered":"2024-09-10T18:52:16.598921+0000","last_unstale":"2024-09-10T18:57:04.671557+0000","last_undegraded":"2024-09-10T18:57:04.671557+0000","last_fullsized":"2024-09-10T18:57:04.671557+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:54:50.968202+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:54.127476+0000","last_clean_scrub_stamp":"2024-09-10T18:54:50.968202+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0074710000000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":299,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.677398+0000","last_change":"2024-09-10T18:57:02.442633+0000","last_active":"2024-09-10T18:57:04.677398+0000","last_peered":"2024-09-10T18:57:04.677398+0000","last_clean":"2024-09-10T18:57:04.677398+0000","last_became_active":"2024-09-10T18:57:02.442251+0000","last_became_peered":"2024-09-10T18:57:02.442251+0000","last_unstale":"2024-09-10T18:57:04.677398+0000","last_undegraded":"2024-09-10T18:57:04.677398+0000","last_fullsized":"2024-09-10T18:57:04.677398+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.136432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.136432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015178839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"232'12","reported_seq":303,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.675168+0000","last_change":"2024-09-10T18:57:02.483803+0000","last_active":"2024-09-10T18:57:04.675168+0000","last_peered":"2024-09-10T18:57:04.675168+0000","last_clean":"2024-09-10T18:57:04.675168+0000","last_became_active":"2024-09-10T18:57:02.460430+0000","last_became_peered":"2024-09-10T18:57:02.460430+0000","last_unstale":"2024-09-10T18:57:04.675168+0000","last_undegraded":"2024-09-10T18:57:04.675168+0000","last_fullsized":"2024-09-10T18:57:04.675168+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'12","last_scrub_stamp":"2024-09-10T18:55:30.612808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:55:30.612808+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001922787,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":272,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.673247+0000","last_change":"2024-09-10T18:56:32.602499+0000","last_active":"2024-09-10T18:57:04.673247+0000","last_peered":"2024-09-10T18:57:04.673247+0000","last_clean":"2024-09-10T18:57:04.673247+0000","last_became_active":"2024-09-10T18:45:22.069653+0000","last_became_peered":"2024-09-10T18:45:22.069653+0000","last_unstale":"2024-09-10T18:57:04.673247+0000","last_undegraded":"2024-09-10T18:57:04.673247+0000","last_fullsized":"2024-09-10T18:57:04.673247+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:32.602448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:32.602448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:54.657009+0000","scrub_duration":0.0052503430000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":16,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.676419+0000","last_change":"2024-09-10T18:57:02.438506+0000","last_active":"2024-09-10T18:57:04.676419+0000","last_peered":"2024-09-10T18:57:04.676419+0000","last_clean":"2024-09-10T18:57:04.676419+0000","last_became_active":"2024-09-10T18:57:02.437979+0000","last_became_peered":"2024-09-10T18:57:02.437979+0000","last_unstale":"2024-09-10T18:57:04.676419+0000","last_undegraded":"2024-09-10T18:57:04.676419+0000","last_fullsized":"2024-09-10T18:57:04.676419+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:47.741071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:54:47.741071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.11","version":"70'81","reported_seq":158,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.666537+0000","last_change":"2024-09-10T18:55:40.395720+0000","last_active":"2024-09-10T18:57:04.666537+0000","last_peered":"2024-09-10T18:57:04.666537+0000","last_clean":"2024-09-10T18:57:04.666537+0000","last_became_active":"2024-09-10T18:55:40.395019+0000","last_became_peered":"2024-09-10T18:55:40.395019+0000","last_unstale":"2024-09-10T18:57:04.666537+0000","last_undegraded":"2024-09-10T18:57:04.666537+0000","last_fullsized":"2024-09-10T18:57:04.666537+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:52:53.556903+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:52:53.556903+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004894626,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"232'20","reported_seq":360,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.676156+0000","last_change":"2024-09-10T18:57:02.468721+0000","last_active":"2024-09-10T18:57:04.676156+0000","last_peered":"2024-09-10T18:57:04.676156+0000","last_clean":"2024-09-10T18:57:04.676156+0000","last_became_active":"2024-09-10T18:57:02.444437+0000","last_became_peered":"2024-09-10T18:57:02.444437+0000","last_unstale":"2024-09-10T18:57:04.676156+0000","last_undegraded":"2024-09-10T18:57:04.676156+0000","last_fullsized":"2024-09-10T18:57:04.676156+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'20","last_scrub_stamp":"2024-09-10T18:55:33.667712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:55:33.667712+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0028302039999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":286,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.671711+0000","last_change":"2024-09-10T18:57:02.446113+0000","last_active":"2024-09-10T18:57:04.671711+0000","last_peered":"2024-09-10T18:57:04.671711+0000","last_clean":"2024-09-10T18:57:04.671711+0000","last_became_active":"2024-09-10T18:57:02.445710+0000","last_became_peered":"2024-09-10T18:57:02.445710+0000","last_unstale":"2024-09-10T18:57:04.671711+0000","last_undegraded":"2024-09-10T18:57:04.671711+0000","last_fullsized":"2024-09-10T18:57:04.671711+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:51.965637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:54:51.965637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011107304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"242'10","reported_seq":443,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-10T18:57:12.163562+0000","last_change":"2024-09-10T18:56:07.787017+0000","last_active":"2024-09-10T18:57:12.163562+0000","last_peered":"2024-09-10T18:57:12.163562+0000","last_clean":"2024-09-10T18:57:12.163562+0000","last_became_active":"2024-09-10T18:55:40.391439+0000","last_became_peered":"2024-09-10T18:55:40.391439+0000","last_unstale":"2024-09-10T18:57:12.163562+0000","last_undegraded":"2024-09-10T18:57:12.163562+0000","last_fullsized":"2024-09-10T18:57:12.163562+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"242'10","last_scrub_stamp":"2024-09-10T18:56:07.786933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:07.786933+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:30.536318+0000","scrub_duration":0.0054185049999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":227,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.664666+0000","last_change":"2024-09-10T18:57:02.454114+0000","last_active":"2024-09-10T18:57:04.664666+0000","last_peered":"2024-09-10T18:57:04.664666+0000","last_clean":"2024-09-10T18:57:04.664666+0000","last_became_active":"2024-09-10T18:57:02.452949+0000","last_became_peered":"2024-09-10T18:57:02.452949+0000","last_unstale":"2024-09-10T18:57:04.664666+0000","last_undegraded":"2024-09-10T18:57:04.664666+0000","last_fullsized":"2024-09-10T18:57:04.664666+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:57.476724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:57.476724+0000","last_clean_scrub_stamp":"2024-09-10T18:52:57.476724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.10","version":"70'81","reported_seq":144,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.671104+0000","last_change":"2024-09-10T18:57:02.444347+0000","last_active":"2024-09-10T18:57:04.671104+0000","last_peered":"2024-09-10T18:57:04.671104+0000","last_clean":"2024-09-10T18:57:04.671104+0000","last_became_active":"2024-09-10T18:57:02.443782+0000","last_became_peered":"2024-09-10T18:57:02.443782+0000","last_unstale":"2024-09-10T18:57:04.671104+0000","last_undegraded":"2024-09-10T18:57:04.671104+0000","last_fullsized":"2024-09-10T18:57:04.671104+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:20.743543+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:20.743543+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:35.491530+0000","scrub_duration":0.0043223280000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"232'15","reported_seq":138,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.674119+0000","last_change":"2024-09-10T18:57:02.487296+0000","last_active":"2024-09-10T18:57:04.674119+0000","last_peered":"2024-09-10T18:57:04.674119+0000","last_clean":"2024-09-10T18:57:04.674119+0000","last_became_active":"2024-09-10T18:57:02.460705+0000","last_became_peered":"2024-09-10T18:57:02.460705+0000","last_unstale":"2024-09-10T18:57:04.674119+0000","last_undegraded":"2024-09-10T18:57:04.674119+0000","last_fullsized":"2024-09-10T18:57:04.674119+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T18:55:25.784333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:55:25.784333+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":252,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.664964+0000","last_change":"2024-09-10T18:56:23.096461+0000","last_active":"2024-09-10T18:57:04.664964+0000","last_peered":"2024-09-10T18:57:04.664964+0000","last_clean":"2024-09-10T18:57:04.664964+0000","last_became_active":"2024-09-10T18:55:40.390697+0000","last_became_peered":"2024-09-10T18:55:40.390697+0000","last_unstale":"2024-09-10T18:57:04.664964+0000","last_undegraded":"2024-09-10T18:57:04.664964+0000","last_fullsized":"2024-09-10T18:57:04.664964+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:23.096373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:23.096373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:52.573724+0000","scrub_duration":0.0057615890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"273'11","reported_seq":443,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-10T18:57:12.177685+0000","last_change":"2024-09-10T18:57:02.473032+0000","last_active":"2024-09-10T18:57:12.177685+0000","last_peered":"2024-09-10T18:57:12.177685+0000","last_clean":"2024-09-10T18:57:12.177685+0000","last_became_active":"2024-09-10T18:57:02.445268+0000","last_became_peered":"2024-09-10T18:57:02.445268+0000","last_unstale":"2024-09-10T18:57:12.177685+0000","last_undegraded":"2024-09-10T18:57:12.177685+0000","last_fullsized":"2024-09-10T18:57:12.177685+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'10","last_scrub_stamp":"2024-09-10T18:55:36.732908+0000","last_deep_scrub":"81'9","last_deep_scrub_stamp":"2024-09-10T18:46:54.800790+0000","last_clean_scrub_stamp":"2024-09-10T18:55:36.732908+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016257330000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":295,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.661667+0000","last_change":"2024-09-10T18:56:20.584899+0000","last_active":"2024-09-10T18:57:04.661667+0000","last_peered":"2024-09-10T18:57:04.661667+0000","last_clean":"2024-09-10T18:57:04.661667+0000","last_became_active":"2024-09-10T18:55:40.391662+0000","last_became_peered":"2024-09-10T18:55:40.391662+0000","last_unstale":"2024-09-10T18:57:04.661667+0000","last_undegraded":"2024-09-10T18:57:04.661667+0000","last_fullsized":"2024-09-10T18:57:04.661667+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:20.584808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_clean_scrub_stamp":"2024-09-10T18:56:20.584808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:43.647662+0000","scrub_duration":0.0035961249999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"232'15","reported_seq":338,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.663909+0000","last_change":"2024-09-10T18:56:02.500313+0000","last_active":"2024-09-10T18:57:04.663909+0000","last_peered":"2024-09-10T18:57:04.663909+0000","last_clean":"2024-09-10T18:57:04.663909+0000","last_became_active":"2024-09-10T18:55:40.400788+0000","last_became_peered":"2024-09-10T18:55:40.400788+0000","last_unstale":"2024-09-10T18:57:04.663909+0000","last_undegraded":"2024-09-10T18:57:04.663909+0000","last_fullsized":"2024-09-10T18:57:04.663909+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T18:56:02.500101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:02.500101+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:26.361106+0000","scrub_duration":0.0097839320000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":306,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.675929+0000","last_change":"2024-09-10T18:57:02.453289+0000","last_active":"2024-09-10T18:57:04.675929+0000","last_peered":"2024-09-10T18:57:04.675929+0000","last_clean":"2024-09-10T18:57:04.675929+0000","last_became_active":"2024-09-10T18:57:02.452986+0000","last_became_peered":"2024-09-10T18:57:02.452986+0000","last_unstale":"2024-09-10T18:57:04.675929+0000","last_undegraded":"2024-09-10T18:57:04.675929+0000","last_fullsized":"2024-09-10T18:57:04.675929+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:18.272126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:18.272126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:34.206060+0000","scrub_duration":0.0035259570000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":279,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.674324+0000","last_change":"2024-09-10T18:56:04.753218+0000","last_active":"2024-09-10T18:57:04.674324+0000","last_peered":"2024-09-10T18:57:04.674324+0000","last_clean":"2024-09-10T18:57:04.674324+0000","last_became_active":"2024-09-10T18:55:40.398243+0000","last_became_peered":"2024-09-10T18:55:40.398243+0000","last_unstale":"2024-09-10T18:57:04.674324+0000","last_undegraded":"2024-09-10T18:57:04.674324+0000","last_fullsized":"2024-09-10T18:57:04.674324+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:04.753133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:03.571786+0000","last_clean_scrub_stamp":"2024-09-10T18:56:04.753133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:32.426506+0000","scrub_duration":0.0069575230000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":247,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.665253+0000","last_change":"2024-09-10T18:56:04.126848+0000","last_active":"2024-09-10T18:57:04.665253+0000","last_peered":"2024-09-10T18:57:04.665253+0000","last_clean":"2024-09-10T18:57:04.665253+0000","last_became_active":"2024-09-10T18:55:40.396932+0000","last_became_peered":"2024-09-10T18:55:40.396932+0000","last_unstale":"2024-09-10T18:57:04.665253+0000","last_undegraded":"2024-09-10T18:57:04.665253+0000","last_fullsized":"2024-09-10T18:57:04.665253+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:04.126516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:04.631558+0000","last_clean_scrub_stamp":"2024-09-10T18:56:04.126516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:25.286369+0000","scrub_duration":0.010034405999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"232'24","reported_seq":298,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.661212+0000","last_change":"2024-09-10T18:54:47.324126+0000","last_active":"2024-09-10T18:57:04.661212+0000","last_peered":"2024-09-10T18:57:04.661212+0000","last_clean":"2024-09-10T18:57:04.661212+0000","last_became_active":"2024-09-10T18:45:20.056885+0000","last_became_peered":"2024-09-10T18:45:20.056885+0000","last_unstale":"2024-09-10T18:57:04.661212+0000","last_undegraded":"2024-09-10T18:57:04.661212+0000","last_fullsized":"2024-09-10T18:57:04.661212+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'16","last_scrub_stamp":"2024-09-10T18:54:47.324044+0000","last_deep_scrub":"223'16","last_deep_scrub_stamp":"2024-09-10T18:54:47.324044+0000","last_clean_scrub_stamp":"2024-09-10T18:54:47.324044+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016282413999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":283,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.667893+0000","last_change":"2024-09-10T18:55:40.399504+0000","last_active":"2024-09-10T18:57:04.667893+0000","last_peered":"2024-09-10T18:57:04.667893+0000","last_clean":"2024-09-10T18:57:04.667893+0000","last_became_active":"2024-09-10T18:55:40.399198+0000","last_became_peered":"2024-09-10T18:55:40.399198+0000","last_unstale":"2024-09-10T18:57:04.667893+0000","last_undegraded":"2024-09-10T18:57:04.667893+0000","last_fullsized":"2024-09-10T18:57:04.667893+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:32.290320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:00.029389+0000","last_clean_scrub_stamp":"2024-09-10T18:53:32.290320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003302677,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":275,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.663765+0000","last_change":"2024-09-10T18:56:31.783026+0000","last_active":"2024-09-10T18:57:04.663765+0000","last_peered":"2024-09-10T18:57:04.663765+0000","last_clean":"2024-09-10T18:57:04.663765+0000","last_became_active":"2024-09-10T18:45:22.070065+0000","last_became_peered":"2024-09-10T18:45:22.070065+0000","last_unstale":"2024-09-10T18:57:04.663765+0000","last_undegraded":"2024-09-10T18:57:04.663765+0000","last_fullsized":"2024-09-10T18:57:04.663765+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:31.782918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:31.782918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:48.901788+0000","scrub_duration":0.0043032130000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":277,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.672860+0000","last_change":"2024-09-10T18:56:06.792096+0000","last_active":"2024-09-10T18:57:04.672860+0000","last_peered":"2024-09-10T18:57:04.672860+0000","last_clean":"2024-09-10T18:57:04.672860+0000","last_became_active":"2024-09-10T18:55:40.395595+0000","last_became_peered":"2024-09-10T18:55:40.395595+0000","last_unstale":"2024-09-10T18:57:04.672860+0000","last_undegraded":"2024-09-10T18:57:04.672860+0000","last_fullsized":"2024-09-10T18:57:04.672860+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:06.791860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:06.791860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:26.074630+0000","scrub_duration":0.0046852380000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"232'8","reported_seq":286,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.669864+0000","last_change":"2024-09-10T18:56:30.802774+0000","last_active":"2024-09-10T18:57:04.669864+0000","last_peered":"2024-09-10T18:57:04.669864+0000","last_clean":"2024-09-10T18:57:04.669864+0000","last_became_active":"2024-09-10T18:45:20.059205+0000","last_became_peered":"2024-09-10T18:45:20.059205+0000","last_unstale":"2024-09-10T18:57:04.669864+0000","last_undegraded":"2024-09-10T18:57:04.669864+0000","last_fullsized":"2024-09-10T18:57:04.669864+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'8","last_scrub_stamp":"2024-09-10T18:56:30.802679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:30.802679+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:46.492044+0000","scrub_duration":0.0034579609999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":282,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.673316+0000","last_change":"2024-09-10T18:55:25.897258+0000","last_active":"2024-09-10T18:57:04.673316+0000","last_peered":"2024-09-10T18:57:04.673316+0000","last_clean":"2024-09-10T18:57:04.673316+0000","last_became_active":"2024-09-10T18:47:36.415866+0000","last_became_peered":"2024-09-10T18:47:36.415866+0000","last_unstale":"2024-09-10T18:57:04.673316+0000","last_undegraded":"2024-09-10T18:57:04.673316+0000","last_fullsized":"2024-09-10T18:57:04.673316+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:25.897205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:55:25.897205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001389787,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":302,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.677074+0000","last_change":"2024-09-10T18:57:02.454548+0000","last_active":"2024-09-10T18:57:04.677074+0000","last_peered":"2024-09-10T18:57:04.677074+0000","last_clean":"2024-09-10T18:57:04.677074+0000","last_became_active":"2024-09-10T18:57:02.453974+0000","last_became_peered":"2024-09-10T18:57:02.453974+0000","last_unstale":"2024-09-10T18:57:04.677074+0000","last_undegraded":"2024-09-10T18:57:04.677074+0000","last_fullsized":"2024-09-10T18:57:04.677074+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:23.195060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_clean_scrub_stamp":"2024-09-10T18:56:23.195060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:50.879571+0000","scrub_duration":0.0047613259999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":63,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.660704+0000","last_change":"2024-09-10T18:57:02.447992+0000","last_active":"2024-09-10T18:57:04.660704+0000","last_peered":"2024-09-10T18:57:04.660704+0000","last_clean":"2024-09-10T18:57:04.660704+0000","last_became_active":"2024-09-10T18:57:02.447566+0000","last_became_peered":"2024-09-10T18:57:02.447566+0000","last_unstale":"2024-09-10T18:57:04.660704+0000","last_undegraded":"2024-09-10T18:57:04.660704+0000","last_fullsized":"2024-09-10T18:57:04.660704+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:28.768091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:55.145638+0000","last_clean_scrub_stamp":"2024-09-10T18:55:28.768091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020285519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.14","version":"70'81","reported_seq":174,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.661876+0000","last_change":"2024-09-10T18:57:02.442850+0000","last_active":"2024-09-10T18:57:04.661876+0000","last_peered":"2024-09-10T18:57:04.661876+0000","last_clean":"2024-09-10T18:57:04.661876+0000","last_became_active":"2024-09-10T18:57:02.442457+0000","last_became_peered":"2024-09-10T18:57:02.442457+0000","last_unstale":"2024-09-10T18:57:04.661876+0000","last_undegraded":"2024-09-10T18:57:04.661876+0000","last_fullsized":"2024-09-10T18:57:04.661876+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:55:34.791223+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:57.184383+0000","last_clean_scrub_stamp":"2024-09-10T18:55:34.791223+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016709736999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17890,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":7,"num_write":3,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":35780,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.11","version":"232'24","reported_seq":322,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.672476+0000","last_change":"2024-09-10T18:57:02.490512+0000","last_active":"2024-09-10T18:57:04.672476+0000","last_peered":"2024-09-10T18:57:04.672476+0000","last_clean":"2024-09-10T18:57:04.672476+0000","last_became_active":"2024-09-10T18:57:02.461909+0000","last_became_peered":"2024-09-10T18:57:02.461909+0000","last_unstale":"2024-09-10T18:57:04.672476+0000","last_undegraded":"2024-09-10T18:57:04.672476+0000","last_fullsized":"2024-09-10T18:57:04.672476+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"79'16","last_scrub_stamp":"2024-09-10T18:54:39.420695+0000","last_deep_scrub":"79'16","last_deep_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_clean_scrub_stamp":"2024-09-10T18:54:39.420695+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.052085645,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"56'1","reported_seq":307,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.676441+0000","last_change":"2024-09-10T18:56:25.211179+0000","last_active":"2024-09-10T18:57:04.676441+0000","last_peered":"2024-09-10T18:57:04.676441+0000","last_clean":"2024-09-10T18:57:04.676441+0000","last_became_active":"2024-09-10T18:47:36.421985+0000","last_became_peered":"2024-09-10T18:47:36.421985+0000","last_unstale":"2024-09-10T18:57:04.676441+0000","last_undegraded":"2024-09-10T18:57:04.676441+0000","last_fullsized":"2024-09-10T18:57:04.676441+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:56:25.211095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:25.211095+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:47.313472+0000","scrub_duration":0.0025468880000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":17,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.659997+0000","last_change":"2024-09-10T18:57:02.435423+0000","last_active":"2024-09-10T18:57:04.659997+0000","last_peered":"2024-09-10T18:57:04.659997+0000","last_clean":"2024-09-10T18:57:04.659997+0000","last_became_active":"2024-09-10T18:57:02.435124+0000","last_became_peered":"2024-09-10T18:57:02.435124+0000","last_unstale":"2024-09-10T18:57:04.659997+0000","last_undegraded":"2024-09-10T18:57:04.659997+0000","last_fullsized":"2024-09-10T18:57:04.659997+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:12.852769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:12.852769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:15.945077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":287,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.674138+0000","last_change":"2024-09-10T18:56:09.903930+0000","last_active":"2024-09-10T18:57:04.674138+0000","last_peered":"2024-09-10T18:57:04.674138+0000","last_clean":"2024-09-10T18:57:04.674138+0000","last_became_active":"2024-09-10T18:55:40.397523+0000","last_became_peered":"2024-09-10T18:55:40.397523+0000","last_unstale":"2024-09-10T18:57:04.674138+0000","last_undegraded":"2024-09-10T18:57:04.674138+0000","last_fullsized":"2024-09-10T18:57:04.674138+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:09.903846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:09.903846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:27.318802+0000","scrub_duration":0.011721984,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"78'2","reported_seq":53,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.671866+0000","last_change":"2024-09-10T18:57:03.514045+0000","last_active":"2024-09-10T18:57:04.671866+0000","last_peered":"2024-09-10T18:57:04.671866+0000","last_clean":"2024-09-10T18:57:04.671866+0000","last_became_active":"2024-09-10T18:57:02.447198+0000","last_became_peered":"2024-09-10T18:57:02.447198+0000","last_unstale":"2024-09-10T18:57:04.671866+0000","last_undegraded":"2024-09-10T18:57:04.671866+0000","last_fullsized":"2024-09-10T18:57:04.671866+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'2","last_scrub_stamp":"2024-09-10T18:54:27.784508+0000","last_deep_scrub":"78'2","last_deep_scrub_stamp":"2024-09-10T18:46:37.581130+0000","last_clean_scrub_stamp":"2024-09-10T18:54:27.784508+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":286,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.671229+0000","last_change":"2024-09-10T18:57:02.444124+0000","last_active":"2024-09-10T18:57:04.671229+0000","last_peered":"2024-09-10T18:57:04.671229+0000","last_clean":"2024-09-10T18:57:04.671229+0000","last_became_active":"2024-09-10T18:57:02.443353+0000","last_became_peered":"2024-09-10T18:57:02.443353+0000","last_unstale":"2024-09-10T18:57:04.671229+0000","last_undegraded":"2024-09-10T18:57:04.671229+0000","last_fullsized":"2024-09-10T18:57:04.671229+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:16.743764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:16.743764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:20.137200+0000","scrub_duration":0.0031221030000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"232'20","reported_seq":301,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.665729+0000","last_change":"2024-09-10T18:57:02.486956+0000","last_active":"2024-09-10T18:57:04.665729+0000","last_peered":"2024-09-10T18:57:04.665729+0000","last_clean":"2024-09-10T18:57:04.665729+0000","last_became_active":"2024-09-10T18:57:02.452473+0000","last_became_peered":"2024-09-10T18:57:02.452473+0000","last_unstale":"2024-09-10T18:57:04.665729+0000","last_undegraded":"2024-09-10T18:57:04.665729+0000","last_fullsized":"2024-09-10T18:57:04.665729+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'20","last_scrub_stamp":"2024-09-10T18:56:27.158686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:27.158686+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:55.303652+0000","scrub_duration":0.0090371180000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":23,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"273'10","reported_seq":452,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-10T18:57:12.103868+0000","last_change":"2024-09-10T18:57:02.474747+0000","last_active":"2024-09-10T18:57:12.103868+0000","last_peered":"2024-09-10T18:57:12.103868+0000","last_clean":"2024-09-10T18:57:12.103868+0000","last_became_active":"2024-09-10T18:57:02.441763+0000","last_became_peered":"2024-09-10T18:57:02.441763+0000","last_unstale":"2024-09-10T18:57:12.103868+0000","last_undegraded":"2024-09-10T18:57:12.103868+0000","last_fullsized":"2024-09-10T18:57:12.103868+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"190'9","last_scrub_stamp":"2024-09-10T18:56:19.259869+0000","last_deep_scrub":"190'9","last_deep_scrub_stamp":"2024-09-10T18:53:07.746309+0000","last_clean_scrub_stamp":"2024-09-10T18:56:19.259869+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:29.454283+0000","scrub_duration":0.0020087130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":263,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.667315+0000","last_change":"2024-09-10T18:54:24.631566+0000","last_active":"2024-09-10T18:57:04.667315+0000","last_peered":"2024-09-10T18:57:04.667315+0000","last_clean":"2024-09-10T18:57:04.667315+0000","last_became_active":"2024-09-10T18:45:24.082318+0000","last_became_peered":"2024-09-10T18:45:24.082318+0000","last_unstale":"2024-09-10T18:57:04.667315+0000","last_undegraded":"2024-09-10T18:57:04.667315+0000","last_fullsized":"2024-09-10T18:57:04.667315+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:24.631469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:03.475299+0000","last_clean_scrub_stamp":"2024-09-10T18:54:24.631469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014197107,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":283,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.670146+0000","last_change":"2024-09-10T18:56:00.878293+0000","last_active":"2024-09-10T18:57:04.670146+0000","last_peered":"2024-09-10T18:57:04.670146+0000","last_clean":"2024-09-10T18:57:04.670146+0000","last_became_active":"2024-09-10T18:55:40.392524+0000","last_became_peered":"2024-09-10T18:55:40.392524+0000","last_unstale":"2024-09-10T18:57:04.670146+0000","last_undegraded":"2024-09-10T18:57:04.670146+0000","last_fullsized":"2024-09-10T18:57:04.670146+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:00.877849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:00.877849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:25.495379+0000","scrub_duration":0.0096660570000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"232'19","reported_seq":308,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.676329+0000","last_change":"2024-09-10T18:55:26.062315+0000","last_active":"2024-09-10T18:57:04.676329+0000","last_peered":"2024-09-10T18:57:04.676329+0000","last_clean":"2024-09-10T18:57:04.676329+0000","last_became_active":"2024-09-10T18:45:20.058717+0000","last_became_peered":"2024-09-10T18:45:20.058717+0000","last_unstale":"2024-09-10T18:57:04.676329+0000","last_undegraded":"2024-09-10T18:57:04.676329+0000","last_fullsized":"2024-09-10T18:57:04.676329+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T18:55:26.062256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:55:26.062256+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001573893,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":245,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.665950+0000","last_change":"2024-09-10T18:57:02.454464+0000","last_active":"2024-09-10T18:57:04.665950+0000","last_peered":"2024-09-10T18:57:04.665950+0000","last_clean":"2024-09-10T18:57:04.665950+0000","last_became_active":"2024-09-10T18:57:02.453464+0000","last_became_peered":"2024-09-10T18:57:02.453464+0000","last_unstale":"2024-09-10T18:57:04.665950+0000","last_undegraded":"2024-09-10T18:57:04.665950+0000","last_fullsized":"2024-09-10T18:57:04.665950+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:33.248024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:33.248024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:33.937538+0000","scrub_duration":0.0060527929999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":258,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.663721+0000","last_change":"2024-09-10T18:54:21.072224+0000","last_active":"2024-09-10T18:57:04.663721+0000","last_peered":"2024-09-10T18:57:04.663721+0000","last_clean":"2024-09-10T18:57:04.663721+0000","last_became_active":"2024-09-10T18:45:24.267251+0000","last_became_peered":"2024-09-10T18:45:24.267251+0000","last_unstale":"2024-09-10T18:57:04.663721+0000","last_undegraded":"2024-09-10T18:57:04.663721+0000","last_fullsized":"2024-09-10T18:57:04.663721+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:21.071910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:54:21.071910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007087388,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"56'1","reported_seq":302,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.675891+0000","last_change":"2024-09-10T18:57:02.449727+0000","last_active":"2024-09-10T18:57:04.675891+0000","last_peered":"2024-09-10T18:57:04.675891+0000","last_clean":"2024-09-10T18:57:04.675891+0000","last_became_active":"2024-09-10T18:57:02.435758+0000","last_became_peered":"2024-09-10T18:57:02.435758+0000","last_unstale":"2024-09-10T18:57:04.675891+0000","last_undegraded":"2024-09-10T18:57:04.675891+0000","last_fullsized":"2024-09-10T18:57:04.675891+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:56:05.158970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:05.158970+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:25.743275+0000","scrub_duration":0.0055166130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"258'34","reported_seq":337,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.666991+0000","last_change":"2024-09-10T18:56:23.839043+0000","last_active":"2024-09-10T18:57:04.666991+0000","last_peered":"2024-09-10T18:57:04.666991+0000","last_clean":"2024-09-10T18:57:04.666991+0000","last_became_active":"2024-09-10T18:55:40.401788+0000","last_became_peered":"2024-09-10T18:55:40.401788+0000","last_unstale":"2024-09-10T18:57:04.666991+0000","last_undegraded":"2024-09-10T18:57:04.666991+0000","last_fullsized":"2024-09-10T18:57:04.666991+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'34","last_scrub_stamp":"2024-09-10T18:56:23.838951+0000","last_deep_scrub":"258'34","last_deep_scrub_stamp":"2024-09-10T18:56:23.838951+0000","last_clean_scrub_stamp":"2024-09-10T18:56:23.838951+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:49.645990+0000","scrub_duration":0.035929888,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":279,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.667526+0000","last_change":"2024-09-10T18:56:13.611288+0000","last_active":"2024-09-10T18:57:04.667526+0000","last_peered":"2024-09-10T18:57:04.667526+0000","last_clean":"2024-09-10T18:57:04.667526+0000","last_became_active":"2024-09-10T18:55:40.397870+0000","last_became_peered":"2024-09-10T18:55:40.397870+0000","last_unstale":"2024-09-10T18:57:04.667526+0000","last_undegraded":"2024-09-10T18:57:04.667526+0000","last_fullsized":"2024-09-10T18:57:04.667526+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:13.610976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:13.610976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:57:40.012534+0000","scrub_duration":0.010999553,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":270,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.674811+0000","last_change":"2024-09-10T18:55:39.815156+0000","last_active":"2024-09-10T18:57:04.674811+0000","last_peered":"2024-09-10T18:57:04.674811+0000","last_clean":"2024-09-10T18:57:04.674811+0000","last_became_active":"2024-09-10T18:52:54.624256+0000","last_became_peered":"2024-09-10T18:52:54.624256+0000","last_unstale":"2024-09-10T18:57:04.674811+0000","last_undegraded":"2024-09-10T18:57:04.674811+0000","last_fullsized":"2024-09-10T18:57:04.674811+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:39.815087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:39.815087+0000","last_clean_scrub_stamp":"2024-09-10T18:55:39.815087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018555970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":286,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.674591+0000","last_change":"2024-09-10T18:57:02.463213+0000","last_active":"2024-09-10T18:57:04.674591+0000","last_peered":"2024-09-10T18:57:04.674591+0000","last_clean":"2024-09-10T18:57:04.674591+0000","last_became_active":"2024-09-10T18:57:02.460855+0000","last_became_peered":"2024-09-10T18:57:02.460855+0000","last_unstale":"2024-09-10T18:57:04.674591+0000","last_undegraded":"2024-09-10T18:57:04.674591+0000","last_fullsized":"2024-09-10T18:57:04.674591+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:46.433200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:54:46.433200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013571259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"232'18","reported_seq":296,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.674286+0000","last_change":"2024-09-10T18:54:21.298808+0000","last_active":"2024-09-10T18:57:04.674286+0000","last_peered":"2024-09-10T18:57:04.674286+0000","last_clean":"2024-09-10T18:57:04.674286+0000","last_became_active":"2024-09-10T18:45:20.058391+0000","last_became_peered":"2024-09-10T18:45:20.058391+0000","last_unstale":"2024-09-10T18:57:04.674286+0000","last_undegraded":"2024-09-10T18:57:04.674286+0000","last_fullsized":"2024-09-10T18:57:04.674286+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:54:21.298731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:54:21.298731+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060418820000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":290,"reported_epoch":273,"state":"active+clean","last_fresh":"2024-09-10T18:57:04.661493+0000","last_change":"2024-09-10T18:57:02.450561+0000","last_active":"2024-09-10T18:57:04.661493+0000","last_peered":"2024-09-10T18:57:04.661493+0000","last_clean":"2024-09-10T18:57:04.661493+0000","last_became_active":"2024-09-10T18:57:02.449135+0000","last_became_peered":"2024-09-10T18:57:02.449135+0000","last_unstale":"2024-09-10T18:57:04.661493+0000","last_undegraded":"2024-09-10T18:57:04.661493+0000","last_fullsized":"2024-09-10T18:57:04.661493+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:53.316517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:54:53.316517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0090189510000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":8858501120,"num_objects":135170,"num_object_clones":0,"num_object_copies":405510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":78784,"num_objects_unfound":0,"num_objects_dirty":135170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135212,"num_write_kb":8652224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13286,"num_bytes_recovered":870711296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":27454078976,"data_stored":27454078976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57708,"ondisk_log_size":57708,"up":48,"acting":48,"num_store_stats":7},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84,"ondisk_log_size":84,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1118,"num_read_kb":875,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":2712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":499712,"data_stored":13268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":758,"ondisk_log_size":758,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":702,"num_read_kb":697,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":715597,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":47,"num_write":23,"num_write_kb":345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":250458,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1701,"ondisk_log_size":1701,"up":63,"acting":63,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":0,"seq":223338299540,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":43,"seq":184683593881,"num_pgs":86,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2535660,"kb_used_data":2234148,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":91209492,"statfs":{"total":95995035648,"available":93398519808,"internally_reserved":0,"allocated":2287767552,"data_stored":2287337869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:56:43 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":5.6040000000000001,"5min":5.6040000000000001,"15min":5.6040000000000001},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":5.6299999999999999,"5min":5.6299999999999999,"15min":5.6299999999999999},"last":0.36299999999999999}]},{"osd":1,"last update":"Tue Sep 10 18:56:57 2024","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3779999999999999,"15min":0.93100000000000005},"min":{"1min":0.247,"5min":0.247,"15min":0.219},"max":{"1min":18.664000000000001,"5min":18.664000000000001,"15min":18.664000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.1779999999999999,"15min":0.82399999999999995},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.214},"max":{"1min":6.266,"5min":9.1750000000000007,"15min":9.1750000000000007},"last":0.72599999999999998}]},{"osd":2,"last update":"Tue Sep 10 18:56:57 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.91800000000000004,"15min":0.73599999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":2.9460000000000002,"5min":4.1029999999999998,"15min":4.4740000000000002},"last":0.312},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.95899999999999996,"15min":0.68600000000000005},"min":{"1min":0.32400000000000001,"5min":0.29199999999999998,"15min":0.27200000000000002},"max":{"1min":2.5670000000000002,"5min":4.8150000000000004,"15min":4.8150000000000004},"last":0.376}]},{"osd":3,"last update":"Tue Sep 10 18:56:57 2024","interfaces":[{"interface":"back","average":{"1min":1.698,"5min":1.302,"15min":0.88500000000000001},"min":{"1min":0.318,"5min":0.28399999999999997,"15min":0.25700000000000001},"max":{"1min":18.817,"5min":18.817,"15min":18.817},"last":2.0640000000000001},{"interface":"front","average":{"1min":1.673,"5min":1.3939999999999999,"15min":1.002},"min":{"1min":0.28699999999999998,"5min":0.253,"15min":0.253},"max":{"1min":18.678999999999998,"5min":18.678999999999998,"15min":18.678999999999998},"last":0.433}]},{"osd":4,"last update":"Tue Sep 10 18:56:57 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":2.7959999999999998,"15min":1.2569999999999999},"min":{"1min":0.25600000000000001,"5min":0.23499999999999999,"15min":0.20999999999999999},"max":{"1min":1.3759999999999999,"5min":165.03299999999999,"15min":165.03299999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.69499999999999995,"5min":2.6099999999999999,"15min":1.1830000000000001},"min":{"1min":0.28100000000000003,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":2.766,"5min":163.959,"15min":163.959},"last":0.32200000000000001}]},{"osd":5,"last update":"Tue Sep 10 18:56:57 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.7230000000000001,"15min":0.91300000000000003},"min":{"1min":0.32600000000000001,"5min":0.27100000000000002,"15min":0.24099999999999999},"max":{"1min":4.1159999999999997,"5min":43.811,"15min":43.811},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.69599999999999995,"5min":1.748,"15min":0.92400000000000004},"min":{"1min":0.32800000000000001,"5min":0.27800000000000002,"15min":0.23999999999999999},"max":{"1min":1.3660000000000001,"5min":42.314,"15min":42.314},"last":0.42299999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:56:23 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":1.1579999999999999,"15min":0.76500000000000001},"min":{"1min":0.34399999999999997,"5min":0.29799999999999999,"15min":0.26200000000000001},"max":{"1min":4.1580000000000004,"5min":5.3680000000000003,"15min":5.3680000000000003},"last":0.61299999999999999},{"interface":"front","average":{"1min":1.036,"5min":1.2709999999999999,"15min":0.81200000000000006},"min":{"1min":0.313,"5min":0.313,"15min":0.29099999999999998},"max":{"1min":3.2759999999999998,"5min":7.0350000000000001,"15min":7.0350000000000001},"last":0.63}]}]},{"osd":5,"up_from":37,"seq":158913790109,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3606432,"kb_used_data":3309080,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":90138720,"statfs":{"total":95995035648,"available":92302049280,"internally_reserved":0,"allocated":3388497920,"data_stored":3388078790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:56:42 2024","interfaces":[{"interface":"back","average":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":15.032999999999999,"5min":15.032999999999999,"15min":15.032999999999999},"last":2.1619999999999999},{"interface":"front","average":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":15.007,"5min":15.007,"15min":15.007},"last":2.1779999999999999}]},{"osd":1,"last update":"Tue Sep 10 18:56:37 2024","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.254,"15min":0.82799999999999996},"min":{"1min":0.254,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":15.084,"5min":15.084,"15min":15.084},"last":0.36799999999999999},{"interface":"front","average":{"1min":1.9330000000000001,"5min":1.7,"15min":1.0109999999999999},"min":{"1min":0.36499999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":15.092000000000001,"5min":19.474,"15min":19.474},"last":0.41899999999999998}]},{"osd":2,"last update":"Tue Sep 10 18:56:37 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.92100000000000004,"15min":0.69699999999999995},"min":{"1min":0.28899999999999998,"5min":0.27900000000000003,"15min":0.249},"max":{"1min":2.669,"5min":4.6319999999999997,"15min":7.8029999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.93200000000000005,"15min":0.72999999999999998},"min":{"1min":0.29999999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":2.766,"5min":4.8680000000000003,"15min":6.7210000000000001},"last":0.53700000000000003}]},{"osd":3,"last update":"Tue Sep 10 18:56:37 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":1.72,"15min":1.2989999999999999},"min":{"1min":0.29099999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":2.7810000000000001,"5min":32.479999999999997,"15min":66.593999999999994},"last":0.497},{"interface":"front","average":{"1min":0.76600000000000001,"5min":1.984,"15min":1.425},"min":{"1min":0.32300000000000001,"5min":0.318,"15min":0.245},"max":{"1min":3.1920000000000002,"5min":32.171999999999997,"15min":65.939999999999998},"last":0.58499999999999996}]},{"osd":4,"last update":"Tue Sep 10 18:56:37 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":1.5549999999999999,"15min":0.80500000000000005},"min":{"1min":0.26000000000000001,"5min":0.23999999999999999,"15min":0.20899999999999999},"max":{"1min":2.9940000000000002,"5min":9.0310000000000006,"15min":9.0310000000000006},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.46999999999999997,"5min":1.3240000000000001,"15min":0.74199999999999999},"min":{"1min":0.26500000000000001,"5min":0.247,"15min":0.23400000000000001},"max":{"1min":1.4690000000000001,"5min":11.182,"15min":11.182},"last":0.46200000000000002}]},{"osd":6,"last update":"Tue Sep 10 18:56:59 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":1.5149999999999999,"15min":0.90100000000000002},"min":{"1min":0.29499999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":3.069,"5min":10.385,"15min":11.17},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.627,"5min":1.4319999999999999,"15min":0.90700000000000003},"min":{"1min":0.27200000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.5369999999999999,"5min":7.3719999999999999,"15min":11.18},"last":0.52100000000000002}]},{"osd":7,"last update":"Tue Sep 10 18:56:22 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":1.4410000000000001,"15min":0.871},"min":{"1min":0.33300000000000002,"5min":0.29299999999999998,"15min":0.27600000000000002},"max":{"1min":3.2770000000000001,"5min":5.7249999999999996,"15min":10.782999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":1.204,"15min":0.78700000000000003},"min":{"1min":0.309,"5min":0.24099999999999999,"15min":0.23599999999999999},"max":{"1min":1.956,"5min":5.9500000000000002,"15min":11.846},"last":0.56699999999999995}]}]},{"osd":4,"up_from":30,"seq":128849019043,"num_pgs":83,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2026024,"kb_used_data":1728672,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":91719128,"statfs":{"total":95995035648,"available":93920387072,"internally_reserved":0,"allocated":1770160128,"data_stored":1769734107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:56:41 2024","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":11.872999999999999,"5min":11.872999999999999,"15min":11.872999999999999},"last":1.071},{"interface":"front","average":{"1min":1.466,"5min":1.466,"15min":1.466},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":11.852,"5min":11.852,"15min":11.852},"last":1.097}]},{"osd":1,"last update":"Tue Sep 10 18:56:18 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.5509999999999999,"15min":0.998},"min":{"1min":0.23799999999999999,"5min":0.22800000000000001,"15min":0.22600000000000001},"max":{"1min":3.9769999999999999,"5min":45.984999999999999,"15min":45.984999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":1.363,"15min":0.88},"min":{"1min":0.27600000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.921,"5min":43.590000000000003,"15min":43.590000000000003},"last":2.21}]},{"osd":2,"last update":"Tue Sep 10 18:56:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.81999999999999995,"15min":0.64300000000000002},"min":{"1min":0.315,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.353,"5min":5.6189999999999998,"15min":5.6189999999999998},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.93799999999999994,"15min":0.68300000000000005},"min":{"1min":0.29899999999999999,"5min":0.28000000000000003,"15min":0.27100000000000002},"max":{"1min":1.9730000000000001,"5min":6.3540000000000001,"15min":6.3540000000000001},"last":0.70299999999999996}]},{"osd":3,"last update":"Tue Sep 10 18:56:18 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":1.5660000000000001,"15min":1.0980000000000001},"min":{"1min":0.31,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":4.5140000000000002,"5min":43.826000000000001,"15min":43.826000000000001},"last":5.5590000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":1.4359999999999999,"15min":1.0209999999999999},"min":{"1min":0.29899999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":2.6930000000000001,"5min":43.570999999999998,"15min":43.570999999999998},"last":0.65900000000000003}]},{"osd":5,"last update":"Tue Sep 10 18:56:41 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":1.4430000000000001,"15min":0.77500000000000002},"min":{"1min":0.25700000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":2.3170000000000002,"5min":10.912000000000001,"15min":10.912000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.64600000000000002,"5min":1.2689999999999999,"15min":0.70699999999999996},"min":{"1min":0.25700000000000001,"5min":0.219,"15min":0.219},"max":{"1min":2.1269999999999998,"5min":5.6429999999999998,"15min":5.6429999999999998},"last":0.94099999999999995}]},{"osd":6,"last update":"Tue Sep 10 18:57:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":1.1339999999999999,"15min":0.70899999999999996},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.972,"5min":8.4030000000000005,"15min":8.4030000000000005},"last":0.876},{"interface":"front","average":{"1min":0.51800000000000002,"5min":1.3129999999999999,"15min":0.78800000000000003},"min":{"1min":0.29699999999999999,"5min":0.26400000000000001,"15min":0.25800000000000001},"max":{"1min":2.3460000000000001,"5min":7.7699999999999996,"15min":7.7699999999999996},"last":0.68400000000000005}]},{"osd":7,"last update":"Tue Sep 10 18:56:31 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":1.3140000000000001,"15min":0.79400000000000004},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":4.5250000000000004,"5min":6.4630000000000001,"15min":6.4630000000000001},"last":1.1479999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":1.2569999999999999,"15min":0.75},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":2.831,"5min":6.3949999999999996,"15min":6.3949999999999996},"last":0.73799999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182568,"num_pgs":89,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5912624,"kb_used_data":5014888,"kb_used_omap":1,"kb_used_meta":897726,"kb_avail":87832528,"statfs":{"total":95995035648,"available":89940508672,"internally_reserved":0,"allocated":5135245312,"data_stored":5134810563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":919271768},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:56:39 2024","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":4.3970000000000002,"5min":4.3970000000000002,"15min":4.3970000000000002},"last":5.8550000000000004},{"interface":"front","average":{"1min":1.516,"5min":1.516,"15min":1.516},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":7.1260000000000003,"5min":7.1260000000000003,"15min":7.1260000000000003},"last":9.6940000000000008}]},{"osd":1,"last update":"Tue Sep 10 18:56:54 2024","interfaces":[{"interface":"back","average":{"1min":2.8170000000000002,"5min":1.0980000000000001,"15min":1.157},"min":{"1min":0.32200000000000001,"5min":0.16200000000000001,"15min":0.16200000000000001},"max":{"1min":7.9189999999999996,"5min":7.9189999999999996,"15min":58.215000000000003},"last":5.7889999999999997},{"interface":"front","average":{"1min":2.109,"5min":0.97399999999999998,"15min":1.133},"min":{"1min":0.23499999999999999,"5min":0.188,"15min":0.188},"max":{"1min":7.2309999999999999,"5min":7.2309999999999999,"15min":57.896999999999998},"last":3.5430000000000001}]},{"osd":2,"last update":"Tue Sep 10 18:56:54 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.68000000000000005,"15min":0.71199999999999997},"min":{"1min":0.27100000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":5.1150000000000002,"5min":5.1150000000000002,"15min":5.8220000000000001},"last":3.508},{"interface":"front","average":{"1min":1.2989999999999999,"5min":0.70099999999999996,"15min":0.67000000000000004},"min":{"1min":0.27400000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":2.907,"5min":4.6289999999999996,"15min":7.0979999999999999},"last":2.8519999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:56:21 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":1.272,"15min":0.95899999999999996},"min":{"1min":0.31,"5min":0.247,"15min":0.247},"max":{"1min":2.7690000000000001,"5min":29.013000000000002,"15min":29.013000000000002},"last":3.1400000000000001},{"interface":"front","average":{"1min":0.90400000000000003,"5min":1.22,"15min":0.95599999999999996},"min":{"1min":0.28799999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":2.8359999999999999,"5min":28.826000000000001,"15min":28.826000000000001},"last":2.895}]},{"osd":5,"last update":"Tue Sep 10 18:56:44 2024","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.2889999999999999,"15min":0.96399999999999997},"min":{"1min":0.33900000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":4.5259999999999998,"5min":6.1989999999999998,"15min":6.1989999999999998},"last":5.8220000000000001},{"interface":"front","average":{"1min":1.04,"5min":1.1819999999999999,"15min":0.95699999999999996},"min":{"1min":0.313,"5min":0.255,"15min":0.255},"max":{"1min":5.2069999999999999,"5min":6.1870000000000003,"15min":9.2509999999999994},"last":3.6709999999999998}]},{"osd":6,"last update":"Tue Sep 10 18:57:04 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.167,"15min":0.997},"min":{"1min":0.317,"5min":0.309,"15min":0.309},"max":{"1min":5.2370000000000001,"5min":5.2370000000000001,"15min":9.3239999999999998},"last":3.7000000000000002},{"interface":"front","average":{"1min":1.839,"5min":1.3080000000000001,"15min":1.0169999999999999},"min":{"1min":0.61299999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":4.9610000000000003,"5min":6.4900000000000002,"15min":7.1559999999999997},"last":3.2120000000000002}]},{"osd":7,"last update":"Tue Sep 10 18:56:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.3160000000000001,"15min":0.96099999999999997},"min":{"1min":0.35799999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":4.4930000000000003,"5min":6.0839999999999996,"15min":6.0839999999999996},"last":5.7549999999999999},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.294,"15min":1.0189999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.33100000000000002},"max":{"1min":5.2859999999999996,"5min":5.2859999999999996,"15min":9.2949999999999999},"last":3.6299999999999999}]}]},{"osd":2,"up_from":0,"seq":77309411500,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834575024,"num_pgs":100,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6051272,"kb_used_data":5465536,"kb_used_omap":1,"kb_used_meta":585726,"kb_avail":87693880,"statfs":{"total":95995035648,"available":89798533120,"internally_reserved":0,"allocated":5596708864,"data_stored":5596250892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":599783768},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,3,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:56:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":4.1120000000000001,"5min":4.1120000000000001,"15min":4.1120000000000001},"last":0.40899999999999997},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":3.4870000000000001,"5min":3.4870000000000001,"15min":3.4870000000000001},"last":0.29899999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:56:30 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.70799999999999996,"15min":0.58499999999999996},"min":{"1min":0.25700000000000001,"5min":0.186,"15min":0.186},"max":{"1min":3.516,"5min":3.516,"15min":3.8479999999999999},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.60999999999999999,"15min":0.61599999999999999},"min":{"1min":0.23899999999999999,"5min":0.215,"15min":0.215},"max":{"1min":3.649,"5min":3.649,"15min":7.093},"last":0.42899999999999999}]},{"osd":3,"last update":"Tue Sep 10 18:57:00 2024","interfaces":[{"interface":"back","average":{"1min":4.415,"5min":1.7230000000000001,"15min":1.2190000000000001},"min":{"1min":0.34699999999999998,"5min":0.185,"15min":0.185},"max":{"1min":47.781999999999996,"5min":47.781999999999996,"15min":47.781999999999996},"last":1.2450000000000001},{"interface":"front","average":{"1min":4.2960000000000003,"5min":1.6000000000000001,"15min":1.2390000000000001},"min":{"1min":0.317,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":47.828000000000003,"5min":47.828000000000003,"15min":47.828000000000003},"last":1.752}]},{"osd":4,"last update":"Tue Sep 10 18:56:19 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":1.2150000000000001,"15min":0.95399999999999996},"min":{"1min":0.373,"5min":0.29799999999999999,"15min":0.26200000000000001},"max":{"1min":4.4089999999999998,"5min":15.965999999999999,"15min":15.965999999999999},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":1.034,"15min":0.80900000000000005},"min":{"1min":0.28799999999999998,"5min":0.28100000000000003,"15min":0.246},"max":{"1min":2.544,"5min":3.1659999999999999,"15min":3.9540000000000002},"last":0.52300000000000002}]},{"osd":5,"last update":"Tue Sep 10 18:56:42 2024","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.452,"15min":1.014},"min":{"1min":0.36299999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":4.2130000000000001,"5min":10.334,"15min":10.334},"last":0.66900000000000004},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.4330000000000001,"15min":1.0720000000000001},"min":{"1min":0.315,"5min":0.29699999999999999,"15min":0.28799999999999998},"max":{"1min":4.1820000000000004,"5min":16.039999999999999,"15min":16.039999999999999},"last":0.90000000000000002}]},{"osd":6,"last update":"Tue Sep 10 18:57:00 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.889,"15min":1.329},"min":{"1min":0.378,"5min":0.308,"15min":0.308},"max":{"1min":4.2999999999999998,"5min":33.290999999999997,"15min":38.039999999999999},"last":0.502},{"interface":"front","average":{"1min":1.2649999999999999,"5min":2.0779999999999998,"15min":1.4550000000000001},"min":{"1min":0.35399999999999998,"5min":0.33400000000000002,"15min":0.30599999999999999},"max":{"1min":3.246,"5min":33.310000000000002,"15min":38.100999999999999},"last":0.875}]},{"osd":7,"last update":"Tue Sep 10 18:56:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.2370000000000001,"15min":0.96799999999999997},"min":{"1min":0.373,"5min":0.35299999999999998,"15min":0.315},"max":{"1min":3.7090000000000001,"5min":4.5449999999999999,"15min":4.6539999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":1.133,"5min":1.046,"15min":0.94399999999999995},"min":{"1min":0.38200000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":4.4400000000000004,"5min":4.9029999999999996,"15min":4.9029999999999996},"last":0.84599999999999997}]}]},{"osd":0,"up_from":241,"seq":1035087118357,"num_pgs":89,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4557032,"kb_used_data":4219104,"kb_used_omap":4,"kb_used_meta":337915,"kb_avail":89188120,"statfs":{"total":95995035648,"available":91328634880,"internally_reserved":0,"allocated":4320362496,"data_stored":4319935297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":346025478},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 18:56:40 2024","interfaces":[{"interface":"back","average":{"1min":2.7799999999999998,"5min":2.7799999999999998,"15min":2.7799999999999998},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":34.692,"5min":34.692,"15min":34.692},"last":2.4889999999999999},{"interface":"front","average":{"1min":2.8199999999999998,"5min":2.8199999999999998,"15min":2.8199999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":34.588999999999999,"5min":34.588999999999999,"15min":34.588999999999999},"last":5.2759999999999998}]},{"osd":2,"last update":"Tue Sep 10 18:56:40 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":3.6219999999999999,"5min":3.6219999999999999,"15min":3.6219999999999999},"last":1.875},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.495}]},{"osd":3,"last update":"Tue Sep 10 18:56:40 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":2.3769999999999998,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":0.46700000000000003},{"interface":"front","average":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":4.8120000000000003,"5min":4.8120000000000003,"15min":4.8120000000000003},"last":3.9220000000000002}]},{"osd":4,"last update":"Tue Sep 10 18:56:40 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":2.5139999999999998},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.7890000000000001,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":1.1870000000000001}]},{"osd":5,"last update":"Tue Sep 10 18:56:40 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":2.7559999999999998,"5min":2.7559999999999998,"15min":2.7559999999999998},"last":1.157},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.921}]},{"osd":6,"last update":"Tue Sep 10 18:56:40 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":2.8170000000000002,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":1.21},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":1.9490000000000001}]},{"osd":7,"last update":"Tue Sep 10 18:56:40 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.9019999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":2.4550000000000001},{"interface":"front","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.972,"5min":2.972,"15min":2.972},"last":0.435}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":2668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3085631488,"data_stored":3085631488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5638127616,"data_stored":5638127616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5161287680,"data_stored":5161287680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1782317056,"data_stored":1782317056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3421634560,"data_stored":3421634560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2312765440,"data_stored":2312765440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":6052315136,"data_stored":6052315136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T18:57:30.915 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd pool set unique_pool_2 pgp_num 16 2024-09-10T18:57:30.938 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:30.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:30.939 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:31.039 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:31.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:31.093 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:31.093 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:31.194 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:31.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:57:30 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:57:30] "GET /metrics HTTP/1.1" 200 224420 "" "Prometheus/2.43.0" 2024-09-10T18:57:31.221 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:31.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:31.222 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:31.323 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:31.373 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:31.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:31.374 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:31.475 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:31.502 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:31.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:31.503 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:31.604 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:31.646 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:31.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:31.647 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:31.748 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:31.775 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:31.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:31.776 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:31.877 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:31.904 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:31.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:31.905 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:32.006 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:32.050 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:32.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:32.051 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:32.152 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:32.179 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:32.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:32.181 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:32.281 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:32.308 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:32.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:32.310 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:32.410 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:32.449 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:32.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:32.451 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:32.551 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:32.579 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:32.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:32.580 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:32.604 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:32 smithi018 ceph-mon[29336]: pgmap v558: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 7.8 GiB data, 23 GiB used, 513 GiB / 536 GiB avail; 738 B/s rd, 0 B/s wr, 720 op/s; 65001/385176 objects misplaced (16.876%); 46 MiB/s, 734 objects/s recovering 2024-09-10T18:57:32.604 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:32 smithi018 ceph-mon[40196]: pgmap v558: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 7.8 GiB data, 23 GiB used, 513 GiB / 536 GiB avail; 738 B/s rd, 0 B/s wr, 720 op/s; 65001/385176 objects misplaced (16.876%); 46 MiB/s, 734 objects/s recovering 2024-09-10T18:57:32.680 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:32.712 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:57:32.713 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:57:32.730 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:32.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:32.731 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:32.749 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:57:32.749 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:57:32.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:32 smithi103 ceph-mon[33163]: pgmap v558: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 7.8 GiB data, 23 GiB used, 513 GiB / 536 GiB avail; 738 B/s rd, 0 B/s wr, 720 op/s; 65001/385176 objects misplaced (16.876%); 46 MiB/s, 734 objects/s recovering 2024-09-10T18:57:32.785 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:57:32.785 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:57:32.823 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:57:32.824 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:57:32.832 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:32.862 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:57:32.862 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:57:32.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:32.879 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:32.880 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:32.895 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:57:32.895 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:57:32.931 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:57:32.931 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:57:32.963 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:57:32.963 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:57:32.981 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:33.006 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:33.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:33.007 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:33.108 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:33.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:33.136 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:33.136 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:33.237 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:33.315 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:33.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:33.316 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:33.417 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:33.451 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:33.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:33.452 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:33.552 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:33.583 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:33.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:33.584 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:33.685 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:33.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:33 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/4239327413' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:57:33.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:33 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:57:33.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:33 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/4239327413' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:57:33.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:33 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:57:33.712 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:33.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:33.713 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:33.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:33 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/4239327413' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:57:33.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:33 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:57:33.814 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:33.843 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:33.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:33.845 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:33.945 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:33.966 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:33.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:33.967 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:34.068 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:34.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:34.103 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:34.103 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:34.204 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:34.239 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:34.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:34.240 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:34.341 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:34.372 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:34.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:34.373 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:34.474 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:34.497 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:34.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:34.498 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:34.599 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:34.646 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:34.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:34.647 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:34.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:34 smithi018 ceph-mon[40196]: pgmap v559: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 7.6 GiB data, 23 GiB used, 514 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.10k op/s; 59897/373629 objects misplaced (16.031%); 64 MiB/s, 1.03k objects/s recovering 2024-09-10T18:57:34.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:34 smithi018 ceph-mon[29336]: pgmap v559: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 7.6 GiB data, 23 GiB used, 514 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.10k op/s; 59897/373629 objects misplaced (16.031%); 64 MiB/s, 1.03k objects/s recovering 2024-09-10T18:57:34.748 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:34.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:34 smithi103 ceph-mon[33163]: pgmap v559: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 7.6 GiB data, 23 GiB used, 514 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.10k op/s; 59897/373629 objects misplaced (16.031%); 64 MiB/s, 1.03k objects/s recovering 2024-09-10T18:57:34.783 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:34.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:34.784 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:34.885 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:34.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:34.917 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:34.917 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:35.018 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:35.058 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:35.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:35.059 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:35.160 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:35.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:35.195 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:35.195 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:35.296 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:35.339 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:35.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:35.340 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:35.441 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:35.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:35.497 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:35.498 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:35.598 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:35.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:35.629 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:35.629 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:35.730 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:35.773 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:35.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:35.774 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:35.874 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:35.897 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:35.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:35.898 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:35.999 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:36.039 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:36.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:36.040 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:36.141 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:36.173 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:36.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:36.174 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:36.275 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:36.315 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:36.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:36.316 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:36.417 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:36.447 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:36.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:36.448 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:36.549 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:36.591 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:36.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:36.592 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:36.693 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:36.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:36 smithi018 ceph-mon[40196]: pgmap v560: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 7.5 GiB data, 23 GiB used, 514 GiB / 536 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 983 op/s; 58573/370845 objects misplaced (15.794%); 58 MiB/s, 920 objects/s recovering 2024-09-10T18:57:36.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:36 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:36.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:36 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:57:36.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:36 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:57:36.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:36 smithi018 ceph-mon[29336]: pgmap v560: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 7.5 GiB data, 23 GiB used, 514 GiB / 536 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 983 op/s; 58573/370845 objects misplaced (15.794%); 58 MiB/s, 920 objects/s recovering 2024-09-10T18:57:36.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:36 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:36.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:36 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:57:36.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:36 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:57:36.726 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:36.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:36.728 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:36.761 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:36 smithi103 ceph-mon[33163]: pgmap v560: 168 pgs: 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 1 active+remapped+backfill_toofull, 157 active+clean; 7.5 GiB data, 23 GiB used, 514 GiB / 536 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 983 op/s; 58573/370845 objects misplaced (15.794%); 58 MiB/s, 920 objects/s recovering 2024-09-10T18:57:36.761 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:36 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:36.761 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:36 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:57:36.761 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:36 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:57:36.828 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:36.859 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:36.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:36.860 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:36.961 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:37.011 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:37.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:37.012 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:37.022 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:57:36 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:57:37.113 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:37.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:37.136 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:37.136 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:37.237 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:37.270 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:37.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:37.271 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:37.372 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:37.404 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:37.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:37.406 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:37.507 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:37.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:37.539 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:37.539 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:37.640 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:37.682 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:37.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:37.683 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:37.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:37 smithi018 ceph-mon[40196]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-10T18:57:37.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:37 smithi018 ceph-mon[40196]: Cluster is now healthy 2024-09-10T18:57:37.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:37 smithi018 ceph-mon[29336]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-10T18:57:37.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:37 smithi018 ceph-mon[29336]: Cluster is now healthy 2024-09-10T18:57:37.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:37 smithi103 ceph-mon[33163]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-10T18:57:37.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:37 smithi103 ceph-mon[33163]: Cluster is now healthy 2024-09-10T18:57:37.785 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:37.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:37.817 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:37.817 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:37.918 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:37.951 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:37.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:37.952 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:38.053 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:38.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:38.090 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:38.091 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:38.191 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:38.214 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:38.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:38.214 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:38.315 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:38.344 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:38.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:38.345 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:38.446 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:38.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:38.484 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:38.485 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:38.585 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:38.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:38.615 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:38.615 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:38.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:38 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:38.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:38 smithi018 ceph-mon[40196]: pgmap v561: 168 pgs: 9 active+remapped+backfilling, 2 active+remapped+backfill_wait, 157 active+clean; 7.3 GiB data, 23 GiB used, 514 GiB / 536 GiB avail; 1002 B/s rd, 0 B/s wr, 957 op/s; 52733/359478 objects misplaced (14.669%); 60 MiB/s, 958 objects/s recovering 2024-09-10T18:57:38.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:38 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:38.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:38 smithi018 ceph-mon[29336]: pgmap v561: 168 pgs: 9 active+remapped+backfilling, 2 active+remapped+backfill_wait, 157 active+clean; 7.3 GiB data, 23 GiB used, 514 GiB / 536 GiB avail; 1002 B/s rd, 0 B/s wr, 957 op/s; 52733/359478 objects misplaced (14.669%); 60 MiB/s, 958 objects/s recovering 2024-09-10T18:57:38.716 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:38.760 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:38.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:38.761 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:38.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:38 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:38.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:38 smithi103 ceph-mon[33163]: pgmap v561: 168 pgs: 9 active+remapped+backfilling, 2 active+remapped+backfill_wait, 157 active+clean; 7.3 GiB data, 23 GiB used, 514 GiB / 536 GiB avail; 1002 B/s rd, 0 B/s wr, 957 op/s; 52733/359478 objects misplaced (14.669%); 60 MiB/s, 958 objects/s recovering 2024-09-10T18:57:38.862 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:38.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:38.905 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:38.905 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:39.006 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:39.041 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:39.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:39.043 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:39.143 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:39.174 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:39.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:39.175 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:39.276 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:39.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:39.307 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:39.308 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:39.408 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:39.430 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:39.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:39.431 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:39.532 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:39.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:39.576 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:39.577 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:39.678 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:39.699 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:39.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:39.701 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:39.801 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:39.846 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:39.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:39.848 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:39.949 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:40.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:40.003 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:40.003 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:40.104 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:40.134 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:40.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:40.135 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:40.235 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:40.271 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:40.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:40.272 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:40.373 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:40.403 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:40.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:40.404 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:40.505 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:40.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:57:40 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:57:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:57:40.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:40 smithi103 ceph-mon[33163]: pgmap v562: 168 pgs: 9 active+remapped+backfilling, 2 active+remapped+backfill_wait, 157 active+clean; 7.2 GiB data, 22 GiB used, 514 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 888 op/s; 51483/356799 objects misplaced (14.429%); 55 MiB/s, 886 objects/s recovering 2024-09-10T18:57:40.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:40 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:57:40.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:40 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:57:40.550 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:40.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:40.551 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:40.652 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:40.684 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:40.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:40.685 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:40.786 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:40.811 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:40.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:40.812 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:40.913 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:40.941 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:40.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:40.942 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:40.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:40 smithi018 ceph-mon[40196]: pgmap v562: 168 pgs: 9 active+remapped+backfilling, 2 active+remapped+backfill_wait, 157 active+clean; 7.2 GiB data, 22 GiB used, 514 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 888 op/s; 51483/356799 objects misplaced (14.429%); 55 MiB/s, 886 objects/s recovering 2024-09-10T18:57:40.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:40 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:57:40.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:40 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:57:40.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:40 smithi018 ceph-mon[29336]: pgmap v562: 168 pgs: 9 active+remapped+backfilling, 2 active+remapped+backfill_wait, 157 active+clean; 7.2 GiB data, 22 GiB used, 514 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 888 op/s; 51483/356799 objects misplaced (14.429%); 55 MiB/s, 886 objects/s recovering 2024-09-10T18:57:40.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:40 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:57:40.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:40 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:57:41.044 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:41.075 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:41.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:41.076 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:41.177 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:41.222 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:41.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:41.223 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:41.324 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:41.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:41.362 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:41.363 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:41.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:57:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:57:40] "GET /metrics HTTP/1.1" 200 224426 "" "Prometheus/2.43.0" 2024-09-10T18:57:41.548 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:41.585 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:41.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:41.586 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:41.686 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:41.716 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:41.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:41.717 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:41.818 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:41.845 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:41.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:41.846 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:41.947 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:41.977 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:41.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:41.978 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:42.078 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:42.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:42.108 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:42.108 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:42.209 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:42.232 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:42.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:42.233 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:42.334 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:42.372 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:42.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:42.373 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:42.474 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:42.506 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:42.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:42.507 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:42.608 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:42.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:42.651 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:42.651 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:42.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:42 smithi018 ceph-mon[29336]: pgmap v563: 168 pgs: 9 active+remapped+backfilling, 2 active+remapped+backfill_wait, 157 active+clean; 7.2 GiB data, 22 GiB used, 514 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 842 op/s; 50578/355881 objects misplaced (14.212%); 56 MiB/s, 889 objects/s recovering 2024-09-10T18:57:42.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:42 smithi018 ceph-mon[40196]: pgmap v563: 168 pgs: 9 active+remapped+backfilling, 2 active+remapped+backfill_wait, 157 active+clean; 7.2 GiB data, 22 GiB used, 514 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 842 op/s; 50578/355881 objects misplaced (14.212%); 56 MiB/s, 889 objects/s recovering 2024-09-10T18:57:42.753 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:42.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:42 smithi103 ceph-mon[33163]: pgmap v563: 168 pgs: 9 active+remapped+backfilling, 2 active+remapped+backfill_wait, 157 active+clean; 7.2 GiB data, 22 GiB used, 514 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 842 op/s; 50578/355881 objects misplaced (14.212%); 56 MiB/s, 889 objects/s recovering 2024-09-10T18:57:42.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:42.799 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:42.799 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:42.825 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:57:42.900 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:42.928 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:42.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:42.929 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:43.030 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:43.076 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:43.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:43.077 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:43.178 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:43.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:43.205 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:43.206 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:43.306 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:43.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:43.337 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:43.337 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:43.438 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:43.469 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is set 2024-09-10T18:57:43.479 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:43.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:43.481 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:43.581 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:43.619 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:43.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:43.620 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:43.721 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:43.750 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:43.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:43.752 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:43.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:43 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1017632592' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:57:43.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:43 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:57:43.853 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:43.883 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:43.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:43.885 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:43.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:43 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1017632592' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:57:43.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:43 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:57:43.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:43 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1017632592' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:57:43.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:43 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:57:43.985 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:44.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:44.017 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:44.017 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:44.118 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:44.151 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:44.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:44.152 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:44.253 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:44.295 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:44.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:44.296 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:44.397 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:44.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:44.441 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:44.441 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:44.457 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T18:57:44.542 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:44.571 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:44.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:44.571 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:44.648 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:44 smithi018 ceph-mon[29336]: pgmap v564: 168 pgs: 9 active+remapped+backfilling, 2 active+remapped+backfill_wait, 157 active+clean; 7.0 GiB data, 22 GiB used, 514 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.11k op/s; 45628/345378 objects misplaced (13.211%); 69 MiB/s, 1.10k objects/s recovering 2024-09-10T18:57:44.648 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:44 smithi018 ceph-mon[29336]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:57:44.648 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:44 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:57:44.648 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:44 smithi018 ceph-mon[29336]: osdmap e278: 8 total, 8 up, 6 in 2024-09-10T18:57:44.649 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:44 smithi018 ceph-mon[40196]: pgmap v564: 168 pgs: 9 active+remapped+backfilling, 2 active+remapped+backfill_wait, 157 active+clean; 7.0 GiB data, 22 GiB used, 514 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.11k op/s; 45628/345378 objects misplaced (13.211%); 69 MiB/s, 1.10k objects/s recovering 2024-09-10T18:57:44.649 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:44 smithi018 ceph-mon[40196]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:57:44.649 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:44 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:57:44.649 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:44 smithi018 ceph-mon[40196]: osdmap e278: 8 total, 8 up, 6 in 2024-09-10T18:57:44.672 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:44.697 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:44.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:44.698 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:44.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:44 smithi103 ceph-mon[33163]: pgmap v564: 168 pgs: 9 active+remapped+backfilling, 2 active+remapped+backfill_wait, 157 active+clean; 7.0 GiB data, 22 GiB used, 514 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.11k op/s; 45628/345378 objects misplaced (13.211%); 69 MiB/s, 1.10k objects/s recovering 2024-09-10T18:57:44.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:44 smithi103 ceph-mon[33163]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:57:44.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:44 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:57:44.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:44 smithi103 ceph-mon[33163]: osdmap e278: 8 total, 8 up, 6 in 2024-09-10T18:57:44.799 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:44.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:44.866 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:44.866 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:44.967 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:44.971 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T18:57:44.971 INFO:teuthology.orchestra.run.smithi018.stdout: "ops": [], 2024-09-10T18:57:44.971 INFO:teuthology.orchestra.run.smithi018.stdout: "num_ops": 0 2024-09-10T18:57:44.971 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T18:57:44.995 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:44.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:44.996 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:45.097 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:45.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:45.146 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:45.146 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:45.247 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:45.277 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:45.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:45.278 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:45.379 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:45.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:45.409 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:45.410 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:45.511 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:45.543 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:45.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:45.544 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:45.645 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:45.679 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:45.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:45.680 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:45.781 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:45.810 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:45.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:45.811 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:45.912 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:45.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:45.957 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:45.957 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:46.058 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:46.097 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:46.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:46.098 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:46.199 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:46.242 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:46.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:46.243 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:46.344 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:46.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:46.389 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:46.390 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:46.490 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:46.520 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:46.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:46.521 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:46.563 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-10T18:57:46.621 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:46.650 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:46.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:46.651 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:46.752 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:46.770 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:46 smithi103 ceph-mon[33163]: pgmap v566: 168 pgs: 9 active+remapped+backfilling, 2 active+remapped+backfill_wait, 157 active+clean; 7.0 GiB data, 22 GiB used, 514 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 929 op/s; 44364/342972 objects misplaced (12.935%); 64 MiB/s, 1.02k objects/s recovering 2024-09-10T18:57:46.810 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:46.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:46.811 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:46.912 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:46.933 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:46.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:46.934 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:46.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:46 smithi018 ceph-mon[40196]: pgmap v566: 168 pgs: 9 active+remapped+backfilling, 2 active+remapped+backfill_wait, 157 active+clean; 7.0 GiB data, 22 GiB used, 514 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 929 op/s; 44364/342972 objects misplaced (12.935%); 64 MiB/s, 1.02k objects/s recovering 2024-09-10T18:57:46.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:46 smithi018 ceph-mon[29336]: pgmap v566: 168 pgs: 9 active+remapped+backfilling, 2 active+remapped+backfill_wait, 157 active+clean; 7.0 GiB data, 22 GiB used, 514 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 929 op/s; 44364/342972 objects misplaced (12.935%); 64 MiB/s, 1.02k objects/s recovering 2024-09-10T18:57:47.022 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:57:46 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:57:47.036 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:47.066 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:47.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:47.067 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:47.168 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:47.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:47.210 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:47.210 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:47.311 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:47.353 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:47.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:47.354 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:47.455 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:47.487 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:47.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:47.488 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:47.589 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:47.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:47.618 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:47.618 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:47.719 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:47.752 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:47.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:47.753 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:47.853 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:47.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:47.908 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:47.908 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:48.009 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:48.030 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:48.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:48.031 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:48.132 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:48.159 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:48.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:48.160 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:48.261 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:48.300 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:48.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:48.301 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:48.402 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:48.424 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:48.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:48.425 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:48.526 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:48.559 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:48.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:48.560 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:48.661 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:48.693 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:48.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:48.694 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:48.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:48 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:48.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:48 smithi103 ceph-mon[33163]: pgmap v567: 168 pgs: 9 active+remapped+backfilling, 2 active+remapped+backfill_wait, 157 active+clean; 6.7 GiB data, 22 GiB used, 515 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 948 op/s; 38214/331047 objects misplaced (11.543%); 67 MiB/s, 1.07k objects/s recovering 2024-09-10T18:57:48.796 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:48.817 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:48.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:48.818 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:48.919 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:48.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:48 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:48.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:48 smithi018 ceph-mon[40196]: pgmap v567: 168 pgs: 9 active+remapped+backfilling, 2 active+remapped+backfill_wait, 157 active+clean; 6.7 GiB data, 22 GiB used, 515 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 948 op/s; 38214/331047 objects misplaced (11.543%); 67 MiB/s, 1.07k objects/s recovering 2024-09-10T18:57:48.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:48 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:48.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:48 smithi018 ceph-mon[29336]: pgmap v567: 168 pgs: 9 active+remapped+backfilling, 2 active+remapped+backfill_wait, 157 active+clean; 6.7 GiB data, 22 GiB used, 515 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 948 op/s; 38214/331047 objects misplaced (11.543%); 67 MiB/s, 1.07k objects/s recovering 2024-09-10T18:57:48.963 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:48.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:48.964 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:49.065 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:49.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:49.104 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:49.105 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:49.205 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:49.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:49.239 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:49.239 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:49.340 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:49.371 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:49.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:49.372 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:49.473 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:49.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:49.509 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:49.509 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:49.610 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:49.639 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:49.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:49.641 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:49.741 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:49.801 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:49.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:49.802 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:49.903 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:49.929 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:49.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:49.930 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:49.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:49 smithi018 ceph-mon[29336]: pgmap v568: 168 pgs: 9 active+remapped+backfilling, 2 active+remapped+backfill_wait, 157 active+clean; 6.7 GiB data, 22 GiB used, 515 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 949 op/s; 37051/328323 objects misplaced (11.285%); 67 MiB/s, 1.07k objects/s recovering 2024-09-10T18:57:49.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:49 smithi018 ceph-mon[40196]: pgmap v568: 168 pgs: 9 active+remapped+backfilling, 2 active+remapped+backfill_wait, 157 active+clean; 6.7 GiB data, 22 GiB used, 515 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 949 op/s; 37051/328323 objects misplaced (11.285%); 67 MiB/s, 1.07k objects/s recovering 2024-09-10T18:57:50.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:49 smithi103 ceph-mon[33163]: pgmap v568: 168 pgs: 9 active+remapped+backfilling, 2 active+remapped+backfill_wait, 157 active+clean; 6.7 GiB data, 22 GiB used, 515 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 949 op/s; 37051/328323 objects misplaced (11.285%); 67 MiB/s, 1.07k objects/s recovering 2024-09-10T18:57:50.031 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:50.063 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:50.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:50.064 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:50.165 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:50.190 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:50.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:50.191 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:50.292 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:50.324 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:50.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:50.325 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:50.426 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:50.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:50.455 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:50.455 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:50.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:57:50 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:57:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:57:50.556 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:50.597 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:50.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:50.598 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:50.699 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:50.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:50.721 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:50.722 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:50.822 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:50.853 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:50.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:50.854 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:50.955 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:50.985 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:50.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:50.986 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:51.087 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:51.108 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:51.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:51.109 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:51.210 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:51.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:51.237 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:51.237 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:51.338 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:51.359 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:51.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:51.360 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:51.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:57:50 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:57:50] "GET /metrics HTTP/1.1" 200 224426 "" "Prometheus/2.43.0" 2024-09-10T18:57:51.460 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:51.486 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:51.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:51.487 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:51.587 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:51.612 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:51.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:51.613 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:51.714 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:51.737 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:51.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:51.738 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:51.839 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:51.876 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:51.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:51.877 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:51.978 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:52.019 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:52.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:52.020 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:52.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:51 smithi103 ceph-mon[33163]: osdmap e279: 8 total, 8 up, 6 in 2024-09-10T18:57:52.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:51 smithi103 ceph-mon[33163]: pgmap v570: 168 pgs: 1 peering, 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 157 active+clean; 6.7 GiB data, 22 GiB used, 515 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 740 op/s; 36530/327600 objects misplaced (11.151%); 55 MiB/s, 877 objects/s recovering 2024-09-10T18:57:52.121 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:52.151 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:52.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:52.152 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:52.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:51 smithi018 ceph-mon[40196]: osdmap e279: 8 total, 8 up, 6 in 2024-09-10T18:57:52.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:51 smithi018 ceph-mon[40196]: pgmap v570: 168 pgs: 1 peering, 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 157 active+clean; 6.7 GiB data, 22 GiB used, 515 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 740 op/s; 36530/327600 objects misplaced (11.151%); 55 MiB/s, 877 objects/s recovering 2024-09-10T18:57:52.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:51 smithi018 ceph-mon[29336]: osdmap e279: 8 total, 8 up, 6 in 2024-09-10T18:57:52.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:51 smithi018 ceph-mon[29336]: pgmap v570: 168 pgs: 1 peering, 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 157 active+clean; 6.7 GiB data, 22 GiB used, 515 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 740 op/s; 36530/327600 objects misplaced (11.151%); 55 MiB/s, 877 objects/s recovering 2024-09-10T18:57:52.253 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:52.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:52.275 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:52.276 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:52.376 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:52.409 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:52.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:52.410 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:52.511 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:52.542 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:52.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:52.543 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:52.644 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:52.665 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:52.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:52.666 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:52.767 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:52.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:52.809 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:52.810 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:52.911 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:52.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:52.942 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:52.942 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:53.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:52 smithi103 ceph-mon[33163]: osdmap e280: 8 total, 8 up, 6 in 2024-09-10T18:57:53.043 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:53.064 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:53.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:53.066 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:53.167 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:53.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:52 smithi018 ceph-mon[40196]: osdmap e280: 8 total, 8 up, 6 in 2024-09-10T18:57:53.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:52 smithi018 ceph-mon[29336]: osdmap e280: 8 total, 8 up, 6 in 2024-09-10T18:57:53.197 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:53.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:53.198 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:53.299 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:53.337 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:53.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:53.338 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:53.439 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:53.470 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:53.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:53.471 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:53.572 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:53.610 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:53.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:53.612 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:53.712 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:53.742 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:53.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:53.743 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:53.844 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:53.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:53.889 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:53.889 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:53.990 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:54.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:54.020 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:54.020 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:54.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:53 smithi103 ceph-mon[33163]: pgmap v572: 168 pgs: 1 peering, 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 157 active+clean; 6.5 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.00k op/s; 31584/318849 objects misplaced (9.906%); 78 MiB/s, 1.25k objects/s recovering 2024-09-10T18:57:54.122 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:54.161 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:54.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:54.162 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:54.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:53 smithi018 ceph-mon[40196]: pgmap v572: 168 pgs: 1 peering, 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 157 active+clean; 6.5 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.00k op/s; 31584/318849 objects misplaced (9.906%); 78 MiB/s, 1.25k objects/s recovering 2024-09-10T18:57:54.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:53 smithi018 ceph-mon[29336]: pgmap v572: 168 pgs: 1 peering, 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 157 active+clean; 6.5 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.00k op/s; 31584/318849 objects misplaced (9.906%); 78 MiB/s, 1.25k objects/s recovering 2024-09-10T18:57:54.263 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:54.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:54.293 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:54.293 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:54.394 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:54.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:54.418 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:54.418 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:54.519 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:54.557 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:54.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:54.558 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:54.659 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:54.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:54.681 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:54.681 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:54.782 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:54.825 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:54.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:54.826 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:54.927 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:54.960 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:54.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:54.961 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:55.061 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:55.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:55.104 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:55.104 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:55.205 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:55.235 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:55.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:55.236 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:55.337 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:55.363 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:55.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:55.364 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:55.464 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:55.487 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:55.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:55.489 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:55.589 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:55.631 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:55.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:55.632 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:55.733 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:55.775 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:55.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:55.776 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:55.877 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:55.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:55.899 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:55.899 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:56.000 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:56.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:56.031 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:56.031 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:56.132 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:56.176 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:56.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:56.177 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:56.278 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:56.300 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:56.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:56.301 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:56.402 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:56.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:56.442 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:56.442 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:56.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:56 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:57:56.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:56 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:57:56.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:56 smithi018 ceph-mon[40196]: pgmap v573: 168 pgs: 1 peering, 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 157 active+clean; 6.4 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 602 op/s; 30268/316599 objects misplaced (9.560%); 50 MiB/s, 801 objects/s recovering 2024-09-10T18:57:56.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:56 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:57:56.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:56 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:57:56.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:56 smithi018 ceph-mon[29336]: pgmap v573: 168 pgs: 1 peering, 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 157 active+clean; 6.4 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 602 op/s; 30268/316599 objects misplaced (9.560%); 50 MiB/s, 801 objects/s recovering 2024-09-10T18:57:56.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:56 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:57:56.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:56 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:57:56.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:56 smithi103 ceph-mon[33163]: pgmap v573: 168 pgs: 1 peering, 8 active+remapped+backfilling, 2 active+remapped+backfill_wait, 157 active+clean; 6.4 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 602 op/s; 30268/316599 objects misplaced (9.560%); 50 MiB/s, 801 objects/s recovering 2024-09-10T18:57:56.543 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:56.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:56.569 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:56.569 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:56.670 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:56.708 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:56.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:56.709 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:56.810 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:57:56.854 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:56.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:56.855 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:57:56.956 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:56.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:56.988 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:56.989 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:57.089 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:57.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:57.113 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:57.114 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:57.214 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:57.241 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:57.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:57.242 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:57.272 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:57:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:57:57.342 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:57.372 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:57.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:57.373 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:57.474 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:57.504 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:57.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:57.505 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:57.593 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-10T18:57:57.606 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:57.679 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:57.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:57.680 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:57.709 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:57:57.781 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:57.803 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:57.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:57.804 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:57.905 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:57.938 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:57.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:57.939 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:58.040 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:58.061 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:58.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:58.062 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:58.163 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:58.192 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:58.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:58.193 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:58.294 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:58.321 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:58.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:58.322 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:58.422 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:57:58.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:58 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:58.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:58 smithi018 ceph-mon[29336]: osdmap e281: 8 total, 8 up, 6 in 2024-09-10T18:57:58.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:58 smithi018 ceph-mon[29336]: pgmap v575: 168 pgs: 1 active+clean+remapped, 7 active+remapped+backfilling, 2 active+remapped+backfill_wait, 158 active+clean; 6.2 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.10k op/s; 26064/306417 objects misplaced (8.506%); 83 MiB/s, 1.32k objects/s recovering 2024-09-10T18:57:58.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:58 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:58.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:58 smithi018 ceph-mon[40196]: osdmap e281: 8 total, 8 up, 6 in 2024-09-10T18:57:58.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:58 smithi018 ceph-mon[40196]: pgmap v575: 168 pgs: 1 active+clean+remapped, 7 active+remapped+backfilling, 2 active+remapped+backfill_wait, 158 active+clean; 6.2 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.10k op/s; 26064/306417 objects misplaced (8.506%); 83 MiB/s, 1.32k objects/s recovering 2024-09-10T18:57:58.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:58.445 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:58.446 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:57:58.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:58 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:58.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:58 smithi103 ceph-mon[33163]: osdmap e281: 8 total, 8 up, 6 in 2024-09-10T18:57:58.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:58 smithi103 ceph-mon[33163]: pgmap v575: 168 pgs: 1 active+clean+remapped, 7 active+remapped+backfilling, 2 active+remapped+backfill_wait, 158 active+clean; 6.2 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.10k op/s; 26064/306417 objects misplaced (8.506%); 83 MiB/s, 1.32k objects/s recovering 2024-09-10T18:57:58.547 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:57:58.571 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:58.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:58.573 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:57:58.673 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:58.707 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:58.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:58.708 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:58.809 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:58.833 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:58.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:58.834 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:58.935 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:58.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:58.960 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:58.961 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:59.061 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:57:59.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:59.095 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:59.095 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:57:59.196 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:59.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:59.233 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:59.234 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:59.334 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:59.362 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:59.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:59.363 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:57:59.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:59 smithi018 ceph-mon[29336]: osdmap e282: 8 total, 8 up, 6 in 2024-09-10T18:57:59.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:59 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:57:59.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:57:59 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:57:59.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:59 smithi018 ceph-mon[40196]: osdmap e282: 8 total, 8 up, 6 in 2024-09-10T18:57:59.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:59 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:57:59.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:57:59 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:57:59.464 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:57:59.491 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:59.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:59.492 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:57:59.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:59 smithi103 ceph-mon[33163]: osdmap e282: 8 total, 8 up, 6 in 2024-09-10T18:57:59.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:59 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:57:59.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:57:59 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:57:59.593 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:57:59.638 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:59.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:59.639 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:57:59.739 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:57:59.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:59.783 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:57:59.784 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:57:59.884 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:57:59.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:59.915 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:57:59.915 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:00.015 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:00.052 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:00.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:00.053 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:00.154 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:00.189 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:00.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:00.190 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:00.290 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:00.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:00.318 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:00.318 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:00.419 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:00.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:00 smithi018 ceph-mon[29336]: osdmap e283: 8 total, 8 up, 6 in 2024-09-10T18:58:00.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:00 smithi018 ceph-mon[29336]: pgmap v578: 168 pgs: 1 active+clean+remapped, 7 active+remapped+backfilling, 2 active+remapped+backfill_wait, 158 active+clean; 6.2 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 797 op/s; 24737/304503 objects misplaced (8.124%); 57 MiB/s, 912 objects/s recovering 2024-09-10T18:58:00.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:00 smithi018 ceph-mon[40196]: osdmap e283: 8 total, 8 up, 6 in 2024-09-10T18:58:00.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:00 smithi018 ceph-mon[40196]: pgmap v578: 168 pgs: 1 active+clean+remapped, 7 active+remapped+backfilling, 2 active+remapped+backfill_wait, 158 active+clean; 6.2 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 797 op/s; 24737/304503 objects misplaced (8.124%); 57 MiB/s, 912 objects/s recovering 2024-09-10T18:58:00.447 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:00.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:00.448 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:00.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:00 smithi103 ceph-mon[33163]: osdmap e283: 8 total, 8 up, 6 in 2024-09-10T18:58:00.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:00 smithi103 ceph-mon[33163]: pgmap v578: 168 pgs: 1 active+clean+remapped, 7 active+remapped+backfilling, 2 active+remapped+backfill_wait, 158 active+clean; 6.2 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 797 op/s; 24737/304503 objects misplaced (8.124%); 57 MiB/s, 912 objects/s recovering 2024-09-10T18:58:00.523 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:58:00 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:58:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:58:00.549 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:00.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:00.573 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:00.574 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:00.674 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:00.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:00.715 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:00.715 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:00.816 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:00.854 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:00.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:00.855 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:00.956 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:00.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:00.991 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:00.991 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:01.092 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:01.119 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:01.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:01.120 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:01.221 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:01.266 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:01.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:01.267 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:01.367 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:01.404 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:01.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:01.405 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:01.413 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:58:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:58:00] "GET /metrics HTTP/1.1" 200 224471 "" "Prometheus/2.43.0" 2024-09-10T18:58:01.506 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:01.531 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:01.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:01.532 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:01.633 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:01.670 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:01.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:01.671 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:01.772 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:01.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:01.809 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:01.809 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:01.910 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:01.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:01.932 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:01.932 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:02.033 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:02.056 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:02.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:02.057 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:02.158 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:02.169 INFO:teuthology.orchestra.run.smithi018.stderr:set pool 9 pgp_num to 16 2024-09-10T18:58:02.203 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:02.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:02.205 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:02.305 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:02.349 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:02.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:02.350 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:02.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:02 smithi018 ceph-mon[29336]: osdmap e284: 8 total, 8 up, 6 in 2024-09-10T18:58:02.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:02 smithi018 ceph-mon[29336]: pgmap v580: 168 pgs: 1 active+clean+remapped, 7 active+remapped+backfilling, 2 active+remapped+backfill_wait, 158 active+clean; 6.2 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 0 B/s wr, 202 op/s; 24737/303852 objects misplaced (8.141%); 17 MiB/s, 276 objects/s recovering 2024-09-10T18:58:02.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:02 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2802427616' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]: dispatch 2024-09-10T18:58:02.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:02 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]: dispatch 2024-09-10T18:58:02.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:02 smithi018 ceph-mon[40196]: osdmap e284: 8 total, 8 up, 6 in 2024-09-10T18:58:02.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:02 smithi018 ceph-mon[40196]: pgmap v580: 168 pgs: 1 active+clean+remapped, 7 active+remapped+backfilling, 2 active+remapped+backfill_wait, 158 active+clean; 6.2 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 0 B/s wr, 202 op/s; 24737/303852 objects misplaced (8.141%); 17 MiB/s, 276 objects/s recovering 2024-09-10T18:58:02.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:02 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2802427616' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]: dispatch 2024-09-10T18:58:02.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:02 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]: dispatch 2024-09-10T18:58:02.452 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:02.473 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:02.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:02.474 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:02.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:02 smithi103 ceph-mon[33163]: osdmap e284: 8 total, 8 up, 6 in 2024-09-10T18:58:02.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:02 smithi103 ceph-mon[33163]: pgmap v580: 168 pgs: 1 active+clean+remapped, 7 active+remapped+backfilling, 2 active+remapped+backfill_wait, 158 active+clean; 6.2 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 0 B/s wr, 202 op/s; 24737/303852 objects misplaced (8.141%); 17 MiB/s, 276 objects/s recovering 2024-09-10T18:58:02.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:02 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2802427616' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]: dispatch 2024-09-10T18:58:02.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:02 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]: dispatch 2024-09-10T18:58:02.575 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:02.622 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:02.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:02.623 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:02.724 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:02.749 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:02.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:02.750 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:02.851 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:02.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:02.892 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:02.893 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:02.994 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:03.025 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:03.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:03.026 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:03.126 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:03.153 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:03.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:03.154 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:03.255 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:03.298 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:03.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:03.299 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:03.400 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:03.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:03.422 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:03.422 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:03.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:03 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]': finished 2024-09-10T18:58:03.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:03 smithi018 ceph-mon[29336]: osdmap e285: 8 total, 8 up, 6 in 2024-09-10T18:58:03.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:03 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]': finished 2024-09-10T18:58:03.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:03 smithi018 ceph-mon[40196]: osdmap e285: 8 total, 8 up, 6 in 2024-09-10T18:58:03.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:03 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]': finished 2024-09-10T18:58:03.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:03 smithi103 ceph-mon[33163]: osdmap e285: 8 total, 8 up, 6 in 2024-09-10T18:58:03.523 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:03.562 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:03.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:03.563 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:03.664 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:03.690 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:03.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:03.691 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:03.792 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:03.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:03.820 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:03.820 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:03.921 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:03.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:03.971 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:03.971 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:04.072 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:04.112 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:04.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:04.114 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:04.192 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:58:04.215 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:04.240 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:04.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:04.241 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:04.342 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:04.370 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:04.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:04.372 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:04.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:04 smithi018 ceph-mon[29336]: pgmap v582: 168 pgs: 1 peering, 7 active+remapped+backfilling, 160 active+clean; 6.0 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 982 B/s rd, 0 B/s wr, 604 op/s; 18274/296976 objects misplaced (6.153%); 85 MiB/s, 1.36k objects/s recovering 2024-09-10T18:58:04.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:04 smithi018 ceph-mon[40196]: pgmap v582: 168 pgs: 1 peering, 7 active+remapped+backfilling, 160 active+clean; 6.0 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 982 B/s rd, 0 B/s wr, 604 op/s; 18274/296976 objects misplaced (6.153%); 85 MiB/s, 1.36k objects/s recovering 2024-09-10T18:58:04.473 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:04.498 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:04.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:04.500 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:04.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:04 smithi103 ceph-mon[33163]: pgmap v582: 168 pgs: 1 peering, 7 active+remapped+backfilling, 160 active+clean; 6.0 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 982 B/s rd, 0 B/s wr, 604 op/s; 18274/296976 objects misplaced (6.153%); 85 MiB/s, 1.36k objects/s recovering 2024-09-10T18:58:04.601 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:04.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:04.633 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:04.634 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:04.734 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:04.772 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:04.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:04.773 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:04.874 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:04.916 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:04.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:04.917 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:05.018 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:05.059 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:05.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:05.060 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:05.161 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:05.182 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:05.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:05.183 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:05.284 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:05.311 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:05.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:05.313 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:05.413 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:05.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:05.442 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:05.443 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:05.543 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:05.567 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:05.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:05.568 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:05.669 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:05.691 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:05.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:05.692 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:05.793 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:05.816 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:05.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:05.817 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:05.918 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:05.950 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:05.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:05.951 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:06.051 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:06.084 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:06.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:06.086 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:06.187 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:06.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:06.223 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:06.223 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:06.324 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:06.351 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:06.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:06.352 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:06.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:06 smithi018 ceph-mon[40196]: osdmap e286: 8 total, 8 up, 6 in 2024-09-10T18:58:06.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:06 smithi018 ceph-mon[40196]: pgmap v584: 168 pgs: 1 peering, 7 active+remapped+backfilling, 160 active+clean; 6.0 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 532 op/s; 16540/294936 objects misplaced (5.608%); 79 MiB/s, 1.26k objects/s recovering 2024-09-10T18:58:06.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:06 smithi018 ceph-mon[29336]: osdmap e286: 8 total, 8 up, 6 in 2024-09-10T18:58:06.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:06 smithi018 ceph-mon[29336]: pgmap v584: 168 pgs: 1 peering, 7 active+remapped+backfilling, 160 active+clean; 6.0 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 532 op/s; 16540/294936 objects misplaced (5.608%); 79 MiB/s, 1.26k objects/s recovering 2024-09-10T18:58:06.452 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:06.486 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:06.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:06.487 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:06.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:06 smithi103 ceph-mon[33163]: osdmap e286: 8 total, 8 up, 6 in 2024-09-10T18:58:06.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:06 smithi103 ceph-mon[33163]: pgmap v584: 168 pgs: 1 peering, 7 active+remapped+backfilling, 160 active+clean; 6.0 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 532 op/s; 16540/294936 objects misplaced (5.608%); 79 MiB/s, 1.26k objects/s recovering 2024-09-10T18:58:06.588 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:06.609 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:06.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:06.610 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:06.711 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:06.734 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:06.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:06.735 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:06.836 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:06.858 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:06.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:06.859 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:06.960 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:06.983 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:06.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:06.984 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:07.085 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:07.283 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:58:06 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:58:07.328 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:07.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:07.329 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:07.430 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:07.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:07 smithi018 ceph-mon[40196]: osdmap e287: 8 total, 8 up, 6 in 2024-09-10T18:58:07.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:07 smithi018 ceph-mon[29336]: osdmap e287: 8 total, 8 up, 6 in 2024-09-10T18:58:07.452 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:07.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:07.453 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:07.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:07 smithi103 ceph-mon[33163]: osdmap e287: 8 total, 8 up, 6 in 2024-09-10T18:58:07.554 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:07.595 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:07.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:07.596 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:07.697 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:07.722 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:07.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:07.723 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:07.823 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:07.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:07.860 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:07.860 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:07.961 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:07.985 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:07.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:07.986 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:08.087 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:08.109 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:08.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:08.110 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:08.211 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:08.258 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:08.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:08.259 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:08.360 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:08.376 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:08 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:58:08.376 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:08 smithi018 ceph-mon[40196]: osdmap e288: 8 total, 8 up, 6 in 2024-09-10T18:58:08.376 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:08 smithi018 ceph-mon[40196]: pgmap v587: 168 pgs: 6 active+remapped+backfilling, 162 active+clean; 5.8 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 987 B/s rd, 0 B/s wr, 667 op/s; 11516/286617 objects misplaced (4.018%); 75 MiB/s, 1.20k objects/s recovering 2024-09-10T18:58:08.376 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:08 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:58:08.376 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:08 smithi018 ceph-mon[29336]: osdmap e288: 8 total, 8 up, 6 in 2024-09-10T18:58:08.377 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:08 smithi018 ceph-mon[29336]: pgmap v587: 168 pgs: 6 active+remapped+backfilling, 162 active+clean; 5.8 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 987 B/s rd, 0 B/s wr, 667 op/s; 11516/286617 objects misplaced (4.018%); 75 MiB/s, 1.20k objects/s recovering 2024-09-10T18:58:08.382 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:08.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:08.383 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:08.484 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:08.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:08 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:58:08.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:08 smithi103 ceph-mon[33163]: osdmap e288: 8 total, 8 up, 6 in 2024-09-10T18:58:08.525 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:08 smithi103 ceph-mon[33163]: pgmap v587: 168 pgs: 6 active+remapped+backfilling, 162 active+clean; 5.8 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 987 B/s rd, 0 B/s wr, 667 op/s; 11516/286617 objects misplaced (4.018%); 75 MiB/s, 1.20k objects/s recovering 2024-09-10T18:58:08.527 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:08.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:08.528 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:08.629 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:08.670 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:08.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:08.671 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:08.772 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:08.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:08.796 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:08.797 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:08.897 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:08.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:08.923 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:08.923 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:09.024 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:09.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:09.062 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:09.062 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:09.163 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:09.191 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:09.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:09.192 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:09.293 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:09.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:09.316 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:09.316 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:09.417 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:09.443 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:09.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:09.444 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:09.545 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:09.568 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:09.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:09.569 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:09.670 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:09.696 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:09.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:09.696 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:09.797 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:09.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:09.838 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:09.838 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:09.939 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:09.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:09 smithi018 ceph-mon[29336]: pgmap v588: 168 pgs: 5 active+remapped+backfilling, 163 active+clean; 5.8 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 686 op/s; 10398/284625 objects misplaced (3.653%); 76 MiB/s, 1.21k objects/s recovering 2024-09-10T18:58:09.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:09 smithi018 ceph-mon[40196]: pgmap v588: 168 pgs: 5 active+remapped+backfilling, 163 active+clean; 5.8 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 686 op/s; 10398/284625 objects misplaced (3.653%); 76 MiB/s, 1.21k objects/s recovering 2024-09-10T18:58:09.960 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:09.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:09.961 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:10.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:09 smithi103 ceph-mon[33163]: pgmap v588: 168 pgs: 5 active+remapped+backfilling, 163 active+clean; 5.8 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 686 op/s; 10398/284625 objects misplaced (3.653%); 76 MiB/s, 1.21k objects/s recovering 2024-09-10T18:58:10.062 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:10.101 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:10.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:10.102 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:10.203 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:10.244 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:10.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:10.245 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:10.346 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:10.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:10.373 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:10.374 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:10.475 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:10.504 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:10.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:10.505 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:10.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:58:10 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:58:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:58:10.606 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:10.646 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:10.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:10.648 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:10.748 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:10.778 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:10.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:10.779 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:10.880 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:10.901 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:10.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:10.902 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:10.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:10 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:58:10.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:10 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:58:11.003 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:11.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:10 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:58:11.024 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:11.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:11.025 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:11.126 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:11.151 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:11.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:11.152 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:11.253 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:11.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:11.277 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:11.277 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:11.378 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:11.400 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:11.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:11.405 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:11.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:58:10 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:58:10] "GET /metrics HTTP/1.1" 200 224525 "" "Prometheus/2.43.0" 2024-09-10T18:58:11.505 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:11.535 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:11.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:11.538 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:11.639 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:11.660 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:11.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:11.661 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:11.762 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:11.787 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:11.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:11.788 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:11.889 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:11.930 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:11.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:11.931 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:11.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:11 smithi018 ceph-mon[29336]: osdmap e289: 8 total, 8 up, 6 in 2024-09-10T18:58:11.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:11 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:58:11.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:11 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:58:11.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:11 smithi018 ceph-mon[29336]: pgmap v590: 168 pgs: 5 active+remapped+backfilling, 163 active+clean; 5.8 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 608 op/s; 10398/283992 objects misplaced (3.661%); 59 MiB/s, 941 objects/s recovering 2024-09-10T18:58:11.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:11 smithi018 ceph-mon[40196]: osdmap e289: 8 total, 8 up, 6 in 2024-09-10T18:58:11.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:11 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:58:11.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:11 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:58:11.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:11 smithi018 ceph-mon[40196]: pgmap v590: 168 pgs: 5 active+remapped+backfilling, 163 active+clean; 5.8 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 608 op/s; 10398/283992 objects misplaced (3.661%); 59 MiB/s, 941 objects/s recovering 2024-09-10T18:58:12.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:11 smithi103 ceph-mon[33163]: osdmap e289: 8 total, 8 up, 6 in 2024-09-10T18:58:12.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:11 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:58:12.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:11 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:58:12.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:11 smithi103 ceph-mon[33163]: pgmap v590: 168 pgs: 5 active+remapped+backfilling, 163 active+clean; 5.8 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 608 op/s; 10398/283992 objects misplaced (3.661%); 59 MiB/s, 941 objects/s recovering 2024-09-10T18:58:12.032 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:12.061 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:12.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:12.062 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:12.163 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:12.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:12.202 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:12.202 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:12.303 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:12.343 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:12.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:12.344 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:12.445 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:12.466 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:12.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:12.467 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:12.568 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:12.605 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:12.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:12.606 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:12.706 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:12.742 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:12.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:12.743 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:12.843 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:12.872 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:12.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:12.873 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:12.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:12 smithi018 ceph-mon[40196]: osdmap e290: 8 total, 8 up, 6 in 2024-09-10T18:58:12.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:12 smithi018 ceph-mon[29336]: osdmap e290: 8 total, 8 up, 6 in 2024-09-10T18:58:12.974 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:13.001 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:13.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:13.003 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:13.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:12 smithi103 ceph-mon[33163]: osdmap e290: 8 total, 8 up, 6 in 2024-09-10T18:58:13.104 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:13.130 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:13.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:13.131 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:13.232 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:13.273 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:13.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:13.274 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:13.375 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:13.402 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:13.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:13.403 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:13.504 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:13.530 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:13.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:13.531 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:13.632 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:13.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:13.662 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:13.662 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:13.762 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:13.789 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:13.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:13.790 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:13.891 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:13.917 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:13.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:13.918 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:14.019 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:14.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:13 smithi103 ceph-mon[33163]: pgmap v592: 168 pgs: 1 peering, 3 active+remapped+backfilling, 164 active+clean; 5.6 GiB data, 21 GiB used, 516 GiB / 536 GiB avail; 830 B/s rd, 0 B/s wr, 548 op/s; 6803/276483 objects misplaced (2.461%); 44 MiB/s, 706 objects/s recovering 2024-09-10T18:58:14.046 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:14.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:14.047 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:14.148 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:14.156 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:13 smithi018 ceph-mon[40196]: pgmap v592: 168 pgs: 1 peering, 3 active+remapped+backfilling, 164 active+clean; 5.6 GiB data, 21 GiB used, 516 GiB / 536 GiB avail; 830 B/s rd, 0 B/s wr, 548 op/s; 6803/276483 objects misplaced (2.461%); 44 MiB/s, 706 objects/s recovering 2024-09-10T18:58:14.156 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:13 smithi018 ceph-mon[29336]: pgmap v592: 168 pgs: 1 peering, 3 active+remapped+backfilling, 164 active+clean; 5.6 GiB data, 21 GiB used, 516 GiB / 536 GiB avail; 830 B/s rd, 0 B/s wr, 548 op/s; 6803/276483 objects misplaced (2.461%); 44 MiB/s, 706 objects/s recovering 2024-09-10T18:58:14.177 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:58:14.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:14.201 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:14.201 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:14.302 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:14.329 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:14.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:14.330 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:14.431 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:14.456 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:14.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:14.458 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:14.559 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:14.597 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:14.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:14.599 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:14.699 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:14.740 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:14.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:14.741 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:14.842 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:14.877 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:14.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:14.879 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:14.979 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:15.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:15.002 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:15.002 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:15.103 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:15.124 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:15.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:15.125 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:15.226 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:15.247 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:15.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:15.249 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:15.349 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:15.377 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:15.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:15.378 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:15.479 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:15.506 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:15.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:15.508 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:15.609 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:15.630 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:15.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:15.631 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:15.731 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:15.763 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:15.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:15.764 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:15.865 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:15.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:15.905 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:15.906 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:16.006 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:16.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:15 smithi103 ceph-mon[33163]: osdmap e291: 8 total, 8 up, 6 in 2024-09-10T18:58:16.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:15 smithi103 ceph-mon[33163]: pgmap v594: 168 pgs: 1 peering, 3 active+remapped+backfilling, 164 active+clean; 5.6 GiB data, 21 GiB used, 516 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 587 op/s; 5997/274059 objects misplaced (2.188%); 42 MiB/s, 678 objects/s recovering 2024-09-10T18:58:16.032 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:16.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:16.033 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:16.086 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:15 smithi018 ceph-mon[40196]: osdmap e291: 8 total, 8 up, 6 in 2024-09-10T18:58:16.086 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:15 smithi018 ceph-mon[40196]: pgmap v594: 168 pgs: 1 peering, 3 active+remapped+backfilling, 164 active+clean; 5.6 GiB data, 21 GiB used, 516 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 587 op/s; 5997/274059 objects misplaced (2.188%); 42 MiB/s, 678 objects/s recovering 2024-09-10T18:58:16.086 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:15 smithi018 ceph-mon[29336]: osdmap e291: 8 total, 8 up, 6 in 2024-09-10T18:58:16.086 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:15 smithi018 ceph-mon[29336]: pgmap v594: 168 pgs: 1 peering, 3 active+remapped+backfilling, 164 active+clean; 5.6 GiB data, 21 GiB used, 516 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 587 op/s; 5997/274059 objects misplaced (2.188%); 42 MiB/s, 678 objects/s recovering 2024-09-10T18:58:16.134 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:16.175 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:16.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:16.176 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:16.277 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:16.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:16.320 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:16.320 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:16.421 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:16.448 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:16.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:16.448 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:16.549 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:16.584 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:16.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:16.585 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:16.686 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:16.712 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:16.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:16.713 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:16.814 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:16.856 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:16.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:16.858 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:16.958 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:16.985 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:16.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:16.986 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:17.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:16 smithi103 ceph-mon[33163]: osdmap e292: 8 total, 8 up, 6 in 2024-09-10T18:58:17.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:16 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:58:17.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:16 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:58:17.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:16 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:58:17.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:16 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:58:17.023 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:58:16 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:58:17.087 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:17.109 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:17.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:17.110 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:17.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:16 smithi018 ceph-mon[40196]: osdmap e292: 8 total, 8 up, 6 in 2024-09-10T18:58:17.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:16 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:58:17.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:16 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:58:17.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:16 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:58:17.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:16 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:58:17.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:16 smithi018 ceph-mon[29336]: osdmap e292: 8 total, 8 up, 6 in 2024-09-10T18:58:17.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:16 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:58:17.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:16 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:58:17.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:16 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:58:17.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:16 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:58:17.210 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:17.236 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:17.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:17.237 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:17.340 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:17.365 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:17.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:17.366 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:17.467 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:17.505 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:17.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:17.506 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:17.607 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:17.649 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:17.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:17.650 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:17.751 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:17.793 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:17.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:17.794 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:17.895 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:17.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:17.950 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:17.950 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:18.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:17 smithi103 ceph-mon[33163]: osdmap e293: 8 total, 8 up, 6 in 2024-09-10T18:58:18.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:17 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:58:18.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:17 smithi103 ceph-mon[33163]: pgmap v597: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 5.3 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 909 B/s rd, 0 B/s wr, 1.05k op/s; 3425/262062 objects misplaced (1.307%); 62 MiB/s, 994 objects/s recovering 2024-09-10T18:58:18.046 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:17 smithi018 ceph-mon[29336]: osdmap e293: 8 total, 8 up, 6 in 2024-09-10T18:58:18.046 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:17 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:58:18.046 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:17 smithi018 ceph-mon[29336]: pgmap v597: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 5.3 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 909 B/s rd, 0 B/s wr, 1.05k op/s; 3425/262062 objects misplaced (1.307%); 62 MiB/s, 994 objects/s recovering 2024-09-10T18:58:18.047 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:17 smithi018 ceph-mon[40196]: osdmap e293: 8 total, 8 up, 6 in 2024-09-10T18:58:18.047 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:17 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:58:18.047 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:17 smithi018 ceph-mon[40196]: pgmap v597: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 5.3 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 909 B/s rd, 0 B/s wr, 1.05k op/s; 3425/262062 objects misplaced (1.307%); 62 MiB/s, 994 objects/s recovering 2024-09-10T18:58:18.051 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:18.075 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:18.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:18.076 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:18.177 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:18.202 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:18.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:18.204 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:18.305 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:18.348 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:18.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:18.350 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:18.451 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:18.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:18.473 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:18.473 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:18.574 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:18.600 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:18.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:18.601 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:18.702 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:18.727 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:18.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:18.728 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:18.829 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:18.853 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:18.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:18.854 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:18.954 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:19.001 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:19.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:19.002 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:19.104 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:19.128 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:19.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:19.129 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:19.225 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6] out_osds: [2, 7] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 6, 7, 0] 2024-09-10T18:58:19.226 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T18:58:19.226 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_2 2024-09-10T18:58:19.226 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T18:58:19.229 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:19.321 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:19.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:19.322 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:19.422 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:19.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:19.455 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:19.455 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:19.556 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:19.582 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:19.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:19.583 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:19.684 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:19.705 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:19.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:19.706 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:19.806 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:19.835 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:19.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:19.836 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:19.937 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:19.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:19 smithi018 ceph-mon[40196]: pgmap v598: 168 pgs: 2 active+remapped+backfilling, 166 active+clean; 5.3 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 957 op/s; 3407/259260 objects misplaced (1.314%); 32 MiB/s, 512 objects/s recovering 2024-09-10T18:58:19.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:19 smithi018 ceph-mon[29336]: pgmap v598: 168 pgs: 2 active+remapped+backfilling, 166 active+clean; 5.3 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 957 op/s; 3407/259260 objects misplaced (1.314%); 32 MiB/s, 512 objects/s recovering 2024-09-10T18:58:19.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:19.958 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:19.959 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:20.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:19 smithi103 ceph-mon[33163]: pgmap v598: 168 pgs: 2 active+remapped+backfilling, 166 active+clean; 5.3 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 957 op/s; 3407/259260 objects misplaced (1.314%); 32 MiB/s, 512 objects/s recovering 2024-09-10T18:58:20.059 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:20.086 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:20.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:20.087 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:20.188 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:20.215 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:20.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:20.216 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:20.316 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:20.344 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:20.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:20.345 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:20.446 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:20.479 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:20.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:20.480 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:20.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:58:20 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:58:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:58:20.580 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:20.620 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:20.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:20.621 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:20.722 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:20.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:20.768 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:20.768 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:20.869 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:20.907 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:20.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:20.909 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:21.009 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:21.040 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:21.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:21.042 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:21.143 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:21.169 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:21.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:21.170 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:21.271 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:21.313 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:21.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:21.315 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:21.416 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:21.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:58:20 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:58:20] "GET /metrics HTTP/1.1" 200 224525 "" "Prometheus/2.43.0" 2024-09-10T18:58:21.453 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:21.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:21.454 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:21.555 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:21.579 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:21.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:21.580 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:21.681 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:21.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:21.719 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:21.719 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:21.820 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:21.845 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:21.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:21.847 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:21.947 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:21.975 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:21.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:21.976 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:22.076 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:22.116 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:22.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:22.117 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:22.218 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:22.261 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:22.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:22.262 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:22.363 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:22.387 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:22.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:22.388 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:22.460 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:58:22.461 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:58:22.489 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:22.494 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:58:22.494 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:58:22.514 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:22.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:22.515 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:22.526 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:58:22.526 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:58:22.599 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:58:22.600 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:58:22.615 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:22.632 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:22 smithi018 ceph-mon[40196]: pgmap v599: 168 pgs: 2 active+remapped+backfilling, 166 active+clean; 5.2 GiB data, 20 GiB used, 517 GiB / 536 GiB avail; 774 B/s rd, 0 B/s wr, 802 op/s; 3407/258156 objects misplaced (1.320%); 22 MiB/s, 347 objects/s recovering 2024-09-10T18:58:22.633 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:58:22.633 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:58:22.633 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:22 smithi018 ceph-mon[29336]: pgmap v599: 168 pgs: 2 active+remapped+backfilling, 166 active+clean; 5.2 GiB data, 20 GiB used, 517 GiB / 536 GiB avail; 774 B/s rd, 0 B/s wr, 802 op/s; 3407/258156 objects misplaced (1.320%); 22 MiB/s, 347 objects/s recovering 2024-09-10T18:58:22.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:22.656 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:22.656 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:22.666 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:58:22.667 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:58:22.697 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:58:22.697 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:58:22.725 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:58:22.725 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:58:22.757 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:22.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:22 smithi103 ceph-mon[33163]: pgmap v599: 168 pgs: 2 active+remapped+backfilling, 166 active+clean; 5.2 GiB data, 20 GiB used, 517 GiB / 536 GiB avail; 774 B/s rd, 0 B/s wr, 802 op/s; 3407/258156 objects misplaced (1.320%); 22 MiB/s, 347 objects/s recovering 2024-09-10T18:58:22.782 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:22.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:22.783 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:22.884 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:22.921 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:22.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:22.922 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:23.023 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:23.055 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:23.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:23.057 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:23.158 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:23.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:23.194 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:23.195 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:23.295 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:23.320 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:23.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:23.321 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:23.422 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:23.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:23.467 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:23.467 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:23.568 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:23.605 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:23.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:23.607 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:23.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:23 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1288506484' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:58:23.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:23 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1288506484' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:58:23.708 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:23.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:23.734 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:23.734 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:23.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:23 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1288506484' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:58:23.835 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:23.860 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:23.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:23.861 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:23.962 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:23.986 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:23.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:23.987 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:24.049 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:58:24.088 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:24.113 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:24.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:24.115 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:24.215 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:24.242 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:24.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:24.243 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:24.344 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:24.372 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:24.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:24.373 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:24.403 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is set 2024-09-10T18:58:24.474 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:24.503 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:24.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:24.504 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:24.605 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:24.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:24.646 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:24.646 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:24.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:24 smithi018 ceph-mon[40196]: pgmap v600: 168 pgs: 2 active+remapped+backfilling, 166 active+clean; 5.0 GiB data, 19 GiB used, 517 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.22k op/s; 1979/244797 objects misplaced (0.808%); 27 MiB/s, 439 objects/s recovering 2024-09-10T18:58:24.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:24 smithi018 ceph-mon[40196]: osdmap e294: 8 total, 8 up, 6 in 2024-09-10T18:58:24.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:24 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/4123371540' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:58:24.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:24 smithi018 ceph-mon[29336]: pgmap v600: 168 pgs: 2 active+remapped+backfilling, 166 active+clean; 5.0 GiB data, 19 GiB used, 517 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.22k op/s; 1979/244797 objects misplaced (0.808%); 27 MiB/s, 439 objects/s recovering 2024-09-10T18:58:24.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:24 smithi018 ceph-mon[29336]: osdmap e294: 8 total, 8 up, 6 in 2024-09-10T18:58:24.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:24 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/4123371540' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:58:24.747 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:24.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:24 smithi103 ceph-mon[33163]: pgmap v600: 168 pgs: 2 active+remapped+backfilling, 166 active+clean; 5.0 GiB data, 19 GiB used, 517 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.22k op/s; 1979/244797 objects misplaced (0.808%); 27 MiB/s, 439 objects/s recovering 2024-09-10T18:58:24.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:24 smithi103 ceph-mon[33163]: osdmap e294: 8 total, 8 up, 6 in 2024-09-10T18:58:24.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:24 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/4123371540' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:58:24.785 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:24.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:24.786 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:24.886 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:24.925 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:24.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:24.926 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:25.027 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:25.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:25.066 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:25.067 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:25.167 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:25.194 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:25.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:25.196 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:25.296 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:25.319 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:25.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:25.320 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:25.421 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:25.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:25.444 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:25.444 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:25.546 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:25.572 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:25.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:25.573 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:25.674 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:25.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:25 smithi018 ceph-mon[29336]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:58:25.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:25 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/4123371540' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:58:25.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:25 smithi018 ceph-mon[29336]: osdmap e295: 8 total, 8 up, 6 in 2024-09-10T18:58:25.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:25 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:58:25.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:25 smithi018 ceph-mon[40196]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:58:25.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:25 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/4123371540' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:58:25.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:25 smithi018 ceph-mon[40196]: osdmap e295: 8 total, 8 up, 6 in 2024-09-10T18:58:25.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:25 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:58:25.707 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:25.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:25.708 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:25.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:25 smithi103 ceph-mon[33163]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:58:25.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:25 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/4123371540' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:58:25.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:25 smithi103 ceph-mon[33163]: osdmap e295: 8 total, 8 up, 6 in 2024-09-10T18:58:25.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:25 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:58:25.809 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:25.833 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:25.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:25.834 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:25.935 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:25.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:25.956 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:25.956 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:26.057 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:26.076 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T18:58:26.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:26.100 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:26.100 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:26.201 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:26.245 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:26.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:26.246 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:26.347 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:26.373 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:26.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:26.375 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:26.475 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:26.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:26.503 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:26.503 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:26.604 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:26.628 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:26.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:26.629 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:26.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:26 smithi018 ceph-mon[40196]: pgmap v603: 168 pgs: 2 active+remapped+backfilling, 166 active+clean; 4.9 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 859 op/s; 1979/241437 objects misplaced (0.820%); 9.7 MiB/s, 155 objects/s recovering 2024-09-10T18:58:26.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:26 smithi018 ceph-mon[29336]: pgmap v603: 168 pgs: 2 active+remapped+backfilling, 166 active+clean; 4.9 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 859 op/s; 1979/241437 objects misplaced (0.820%); 9.7 MiB/s, 155 objects/s recovering 2024-09-10T18:58:26.730 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:26.756 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:26.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:26.757 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:26.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:26 smithi103 ceph-mon[33163]: pgmap v603: 168 pgs: 2 active+remapped+backfilling, 166 active+clean; 4.9 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 859 op/s; 1979/241437 objects misplaced (0.820%); 9.7 MiB/s, 155 objects/s recovering 2024-09-10T18:58:26.861 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:26.897 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:26.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:26.898 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:26.999 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:27.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:27.041 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:27.041 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:27.142 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:27.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:27.173 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:27.174 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:27.272 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:58:26 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:58:27.274 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:27.317 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:27.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:27.318 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:27.419 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:27.439 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:27.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:27.440 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:27.541 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:27.591 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:27.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:27.592 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:27.693 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:27.718 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:27.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:27.719 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:27.820 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:27.843 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:27.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:27.844 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:27.945 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:27.984 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:27.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:27.985 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:28.086 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:28.112 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:28.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:28.113 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:28.214 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:28.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:28.257 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:28.257 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:28.358 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:28.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:28.406 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:28.407 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:28.507 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:28.532 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:28.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:28.533 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:28.551 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:28 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:58:28.551 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:28 smithi018 ceph-mon[29336]: pgmap v604: 168 pgs: 1 active+remapped+backfilling, 167 active+clean; 4.6 GiB data, 18 GiB used, 518 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.27k op/s; 559/228846 objects misplaced (0.244%); 20 MiB/s, 321 objects/s recovering 2024-09-10T18:58:28.551 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:28 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:58:28.551 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:28 smithi018 ceph-mon[40196]: pgmap v604: 168 pgs: 1 active+remapped+backfilling, 167 active+clean; 4.6 GiB data, 18 GiB used, 518 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.27k op/s; 559/228846 objects misplaced (0.244%); 20 MiB/s, 321 objects/s recovering 2024-09-10T18:58:28.596 INFO:teuthology.orchestra.run.smithi018.stdout:ERROR: (22) Invalid argument 2024-09-10T18:58:28.596 INFO:teuthology.orchestra.run.smithi018.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:58:28.634 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:28.657 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:28.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:28.658 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:28.759 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:28.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:28 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:58:28.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:28 smithi103 ceph-mon[33163]: pgmap v604: 168 pgs: 1 active+remapped+backfilling, 167 active+clean; 4.6 GiB data, 18 GiB used, 518 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.27k op/s; 559/228846 objects misplaced (0.244%); 20 MiB/s, 321 objects/s recovering 2024-09-10T18:58:28.784 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:28.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:28.785 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:28.886 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:28.909 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:28.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:28.910 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:29.012 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:29.059 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:29.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:29.060 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:29.160 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:29.186 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:29.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:29.187 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:29.288 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:29.313 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:29.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:29.314 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:29.415 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:29.458 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:29.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:29.459 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:29.560 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:29.600 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:29.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:29.601 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:29.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:29 smithi018 ceph-mon[40196]: pgmap v605: 168 pgs: 1 active+remapped+backfilling, 167 active+clean; 4.6 GiB data, 18 GiB used, 518 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.34k op/s; 559/225933 objects misplaced (0.247%); 20 MiB/s, 321 objects/s recovering 2024-09-10T18:58:29.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:29 smithi018 ceph-mon[29336]: pgmap v605: 168 pgs: 1 active+remapped+backfilling, 167 active+clean; 4.6 GiB data, 18 GiB used, 518 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.34k op/s; 559/225933 objects misplaced (0.247%); 20 MiB/s, 321 objects/s recovering 2024-09-10T18:58:29.701 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:29.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:29.744 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:29.744 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:29.755 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-10T18:58:29.845 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:29.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:29.890 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:29.890 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:29.991 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:30.017 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:30.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:30.018 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:30.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:29 smithi103 ceph-mon[33163]: pgmap v605: 168 pgs: 1 active+remapped+backfilling, 167 active+clean; 4.6 GiB data, 18 GiB used, 518 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.34k op/s; 559/225933 objects misplaced (0.247%); 20 MiB/s, 321 objects/s recovering 2024-09-10T18:58:30.120 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:30.142 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:30.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:30.143 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:30.243 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:30.268 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:30.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:30.269 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:30.334 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:58:30.370 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:30.433 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:30.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:30.434 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:30.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:58:30 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:58:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:58:30.535 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:30.559 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:30.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:30.560 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:30.661 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:30.685 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:30.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:30.686 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:30.738 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:30 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:58:30.738 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:30 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:58:30.739 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:58:30.739 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:30 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:58:30.740 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:30 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:58:30.748 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-09-10T18:58:30.786 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:30.812 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:30.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:30.813 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:30.914 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:30.955 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:30.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:30.956 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:31.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:30 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:58:31.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:30 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:58:31.060 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:31.097 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:31.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:31.098 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:31.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:58:30 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:58:30] "GET /metrics HTTP/1.1" 200 224474 "" "Prometheus/2.43.0" 2024-09-10T18:58:31.199 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:31.220 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:31.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:31.221 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:31.322 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:31.349 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:31.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:31.350 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:31.451 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:31.477 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:31.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:31.478 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:31.579 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:31.605 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:31.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:31.607 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:31.708 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:31.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:31.749 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:31.750 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:31.851 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:31.895 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:31.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:31.896 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:31.997 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:32.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:31 smithi103 ceph-mon[33163]: osdmap e296: 8 total, 8 up, 6 in 2024-09-10T18:58:32.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:31 smithi103 ceph-mon[33163]: from='client.25213 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:58:32.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:31 smithi103 ceph-mon[33163]: pgmap v607: 168 pgs: 1 active+remapped+backfilling, 167 active+clean; 4.6 GiB data, 18 GiB used, 519 GiB / 536 GiB avail; 645 B/s rd, 0 B/s wr, 843 op/s; 559/224751 objects misplaced (0.249%); 11 MiB/s, 169 objects/s recovering 2024-09-10T18:58:32.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:32.037 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:32.037 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:32.047 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:31 smithi018 ceph-mon[29336]: osdmap e296: 8 total, 8 up, 6 in 2024-09-10T18:58:32.047 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:31 smithi018 ceph-mon[29336]: from='client.25213 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:58:32.048 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:31 smithi018 ceph-mon[29336]: pgmap v607: 168 pgs: 1 active+remapped+backfilling, 167 active+clean; 4.6 GiB data, 18 GiB used, 519 GiB / 536 GiB avail; 645 B/s rd, 0 B/s wr, 843 op/s; 559/224751 objects misplaced (0.249%); 11 MiB/s, 169 objects/s recovering 2024-09-10T18:58:32.048 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:31 smithi018 ceph-mon[40196]: osdmap e296: 8 total, 8 up, 6 in 2024-09-10T18:58:32.048 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:31 smithi018 ceph-mon[40196]: from='client.25213 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:58:32.048 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:31 smithi018 ceph-mon[40196]: pgmap v607: 168 pgs: 1 active+remapped+backfilling, 167 active+clean; 4.6 GiB data, 18 GiB used, 519 GiB / 536 GiB avail; 645 B/s rd, 0 B/s wr, 843 op/s; 559/224751 objects misplaced (0.249%); 11 MiB/s, 169 objects/s recovering 2024-09-10T18:58:32.138 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:32.153 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-10T18:58:32.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:32.175 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:32.175 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:32.276 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:32.341 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:32.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:32.342 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:32.443 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:32.494 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:32.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:32.495 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:32.540 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:58:32.596 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:32.622 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:32.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:32.623 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:32.724 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:32.748 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:32.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:32.750 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:32.851 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:32.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:32.888 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:32.888 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:32.989 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:33.014 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:33.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:33.015 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:33.116 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:33.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:33.139 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:33.139 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:33.240 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:33.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:33.263 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:33.263 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:33.364 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:33.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:33.388 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:33.388 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:33.489 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:33.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:33.512 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:33.512 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:33.613 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:33.635 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:33.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:33.636 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:33.737 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:33.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:33.774 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:33.774 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:33.875 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:33.910 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:33.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:33.911 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:34.012 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:34.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:33 smithi103 ceph-mon[33163]: osdmap e297: 8 total, 8 up, 6 in 2024-09-10T18:58:34.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:33 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:58:34.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:33 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:58:34.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:33 smithi103 ceph-mon[33163]: pgmap v609: 168 pgs: 1 active+remapped+backfilling, 167 active+clean; 4.3 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.31k op/s; 45/210090 objects misplaced (0.021%); 14 MiB/s, 228 objects/s recovering 2024-09-10T18:58:34.050 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:34.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:34.051 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:34.152 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:34.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:34.189 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:34.189 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:34.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:33 smithi018 ceph-mon[29336]: osdmap e297: 8 total, 8 up, 6 in 2024-09-10T18:58:34.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:33 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:58:34.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:33 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:58:34.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:33 smithi018 ceph-mon[29336]: pgmap v609: 168 pgs: 1 active+remapped+backfilling, 167 active+clean; 4.3 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.31k op/s; 45/210090 objects misplaced (0.021%); 14 MiB/s, 228 objects/s recovering 2024-09-10T18:58:34.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:33 smithi018 ceph-mon[40196]: osdmap e297: 8 total, 8 up, 6 in 2024-09-10T18:58:34.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:33 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:58:34.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:33 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:58:34.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:33 smithi018 ceph-mon[40196]: pgmap v609: 168 pgs: 1 active+remapped+backfilling, 167 active+clean; 4.3 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.31k op/s; 45/210090 objects misplaced (0.021%); 14 MiB/s, 228 objects/s recovering 2024-09-10T18:58:34.290 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:34.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:34.327 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:34.327 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:34.411 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":606,"stamp":"2024-09-10T18:58:30.688692+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4923392545,"num_objects":75311,"num_object_clones":0,"num_object_copies":225933,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":559,"num_objects_unfound":0,"num_objects_dirty":75311,"num_whiteouts":0,"num_read":1945,"num_read_kb":1719,"num_write":199124,"num_write_kb":8752325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75413,"num_bytes_recovered":4927841610,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":60340,"ondisk_log_size":60340,"up":504,"acting":504,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":510,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":562470912,"kb_used":18841612,"kb_used_data":14953176,"kb_used_omap":12,"kb_used_meta":3887795,"kb_avail":543629300,"statfs":{"total":575970213888,"available":556676403200,"internally_reserved":0,"allocated":15312052224,"data_stored":15309805782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13122,"internal_metadata":3981102270},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-412090368,"num_objects":-6288,"num_object_clones":0,"num_object_copies":-18864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-1420,"num_objects_unfound":0,"num_objects_dirty":-6288,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":6288,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1345,"num_bytes_recovered":88145920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.370486"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":301,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.409097+0000","last_change":"2024-09-10T18:57:02.462265+0000","last_active":"2024-09-10T18:58:23.409097+0000","last_peered":"2024-09-10T18:58:23.409097+0000","last_clean":"2024-09-10T18:58:23.409097+0000","last_became_active":"2024-09-10T18:57:02.460451+0000","last_became_peered":"2024-09-10T18:57:02.460451+0000","last_unstale":"2024-09-10T18:58:23.409097+0000","last_undegraded":"2024-09-10T18:58:23.409097+0000","last_fullsized":"2024-09-10T18:58:23.409097+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:14.875872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:14.875872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037478699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":285,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.399564+0000","last_change":"2024-09-10T18:57:02.442207+0000","last_active":"2024-09-10T18:58:23.399564+0000","last_peered":"2024-09-10T18:58:23.399564+0000","last_clean":"2024-09-10T18:58:23.399564+0000","last_became_active":"2024-09-10T18:57:02.440648+0000","last_became_peered":"2024-09-10T18:57:02.440648+0000","last_unstale":"2024-09-10T18:58:23.399564+0000","last_undegraded":"2024-09-10T18:58:23.399564+0000","last_fullsized":"2024-09-10T18:58:23.399564+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:31.221724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:56:31.221724+0000","last_clean_scrub_stamp":"2024-09-10T18:56:31.221724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051032730000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"232'27","reported_seq":386,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.403407+0000","last_change":"2024-09-10T18:56:02.821637+0000","last_active":"2024-09-10T18:58:23.403407+0000","last_peered":"2024-09-10T18:58:23.403407+0000","last_clean":"2024-09-10T18:58:23.403407+0000","last_became_active":"2024-09-10T18:55:40.394717+0000","last_became_peered":"2024-09-10T18:55:40.394717+0000","last_unstale":"2024-09-10T18:58:23.403407+0000","last_undegraded":"2024-09-10T18:58:23.403407+0000","last_fullsized":"2024-09-10T18:58:23.403407+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'27","last_scrub_stamp":"2024-09-10T18:56:02.821551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:02.821551+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0083110159999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":291,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.409833+0000","last_change":"2024-09-10T18:55:33.526359+0000","last_active":"2024-09-10T18:58:23.409833+0000","last_peered":"2024-09-10T18:58:23.409833+0000","last_clean":"2024-09-10T18:58:23.409833+0000","last_became_active":"2024-09-10T18:45:22.065619+0000","last_became_peered":"2024-09-10T18:45:22.065619+0000","last_unstale":"2024-09-10T18:58:23.409833+0000","last_undegraded":"2024-09-10T18:58:23.409833+0000","last_fullsized":"2024-09-10T18:58:23.409833+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:33.526302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.057932+0000","last_clean_scrub_stamp":"2024-09-10T18:55:33.526302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001517556,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":334,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.405699+0000","last_change":"2024-09-10T18:57:02.457742+0000","last_active":"2024-09-10T18:58:23.405699+0000","last_peered":"2024-09-10T18:58:23.405699+0000","last_clean":"2024-09-10T18:58:23.405699+0000","last_became_active":"2024-09-10T18:57:02.457322+0000","last_became_peered":"2024-09-10T18:57:02.457322+0000","last_unstale":"2024-09-10T18:58:23.405699+0000","last_undegraded":"2024-09-10T18:58:23.405699+0000","last_fullsized":"2024-09-10T18:58:23.405699+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:23.885562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:23.885562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0031033800000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"232'19","reported_seq":312,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.409661+0000","last_change":"2024-09-10T18:54:48.324177+0000","last_active":"2024-09-10T18:58:23.409661+0000","last_peered":"2024-09-10T18:58:23.409661+0000","last_clean":"2024-09-10T18:58:23.409661+0000","last_became_active":"2024-09-10T18:45:20.061266+0000","last_became_peered":"2024-09-10T18:45:20.061266+0000","last_unstale":"2024-09-10T18:58:23.409661+0000","last_undegraded":"2024-09-10T18:58:23.409661+0000","last_fullsized":"2024-09-10T18:58:23.409661+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'13","last_scrub_stamp":"2024-09-10T18:54:48.323985+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_clean_scrub_stamp":"2024-09-10T18:54:48.323985+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0080421699999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":34,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.400928+0000","last_change":"2024-09-10T18:57:02.446964+0000","last_active":"2024-09-10T18:58:23.400928+0000","last_peered":"2024-09-10T18:58:23.400928+0000","last_clean":"2024-09-10T18:58:23.400928+0000","last_became_active":"2024-09-10T18:57:02.446008+0000","last_became_peered":"2024-09-10T18:57:02.446008+0000","last_unstale":"2024-09-10T18:58:23.400928+0000","last_undegraded":"2024-09-10T18:58:23.400928+0000","last_fullsized":"2024-09-10T18:58:23.400928+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:22.163121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:01.135988+0000","last_clean_scrub_stamp":"2024-09-10T18:56:22.163121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002631478,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":279,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.409508+0000","last_change":"2024-09-10T18:54:22.051769+0000","last_active":"2024-09-10T18:58:23.409508+0000","last_peered":"2024-09-10T18:58:23.409508+0000","last_clean":"2024-09-10T18:58:23.409508+0000","last_became_active":"2024-09-10T18:45:24.081002+0000","last_became_peered":"2024-09-10T18:45:24.081002+0000","last_unstale":"2024-09-10T18:58:23.409508+0000","last_undegraded":"2024-09-10T18:58:23.409508+0000","last_fullsized":"2024-09-10T18:58:23.409508+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:22.051692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:54:22.051692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013425862,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":283,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.406534+0000","last_change":"2024-09-10T18:57:02.452754+0000","last_active":"2024-09-10T18:58:23.406534+0000","last_peered":"2024-09-10T18:58:23.406534+0000","last_clean":"2024-09-10T18:58:23.406534+0000","last_became_active":"2024-09-10T18:57:02.452259+0000","last_became_peered":"2024-09-10T18:57:02.452259+0000","last_unstale":"2024-09-10T18:58:23.406534+0000","last_undegraded":"2024-09-10T18:58:23.406534+0000","last_fullsized":"2024-09-10T18:58:23.406534+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:06.064071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:06.064071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037631460000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1b","version":"232'9","reported_seq":298,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.409636+0000","last_change":"2024-09-10T18:54:45.305098+0000","last_active":"2024-09-10T18:58:23.409636+0000","last_peered":"2024-09-10T18:58:23.409636+0000","last_clean":"2024-09-10T18:58:23.409636+0000","last_became_active":"2024-09-10T18:45:20.057567+0000","last_became_peered":"2024-09-10T18:45:20.057567+0000","last_unstale":"2024-09-10T18:58:23.409636+0000","last_undegraded":"2024-09-10T18:58:23.409636+0000","last_fullsized":"2024-09-10T18:58:23.409636+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"163'7","last_scrub_stamp":"2024-09-10T18:54:45.305023+0000","last_deep_scrub":"163'7","last_deep_scrub_stamp":"2024-09-10T18:51:54.261879+0000","last_clean_scrub_stamp":"2024-09-10T18:54:45.305023+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0088188659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":37,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.409434+0000","last_change":"2024-09-10T18:57:02.439756+0000","last_active":"2024-09-10T18:58:23.409434+0000","last_peered":"2024-09-10T18:58:23.409434+0000","last_clean":"2024-09-10T18:58:23.409434+0000","last_became_active":"2024-09-10T18:57:02.439440+0000","last_became_peered":"2024-09-10T18:57:02.439440+0000","last_unstale":"2024-09-10T18:58:23.409434+0000","last_undegraded":"2024-09-10T18:58:23.409434+0000","last_fullsized":"2024-09-10T18:58:23.409434+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:27.785914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:36.509971+0000","last_clean_scrub_stamp":"2024-09-10T18:55:27.785914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":150,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.401140+0000","last_change":"2024-09-10T18:56:07.553751+0000","last_active":"2024-09-10T18:58:23.401140+0000","last_peered":"2024-09-10T18:58:23.401140+0000","last_clean":"2024-09-10T18:58:23.401140+0000","last_became_active":"2024-09-10T18:55:40.399236+0000","last_became_peered":"2024-09-10T18:55:40.399236+0000","last_unstale":"2024-09-10T18:58:23.401140+0000","last_undegraded":"2024-09-10T18:58:23.401140+0000","last_fullsized":"2024-09-10T18:58:23.401140+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:07.553696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:07.553696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0091042339999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":90,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.393736+0000","last_change":"2024-09-10T18:57:02.440789+0000","last_active":"2024-09-10T18:58:23.393736+0000","last_peered":"2024-09-10T18:58:23.393736+0000","last_clean":"2024-09-10T18:58:23.393736+0000","last_became_active":"2024-09-10T18:57:02.440497+0000","last_became_peered":"2024-09-10T18:57:02.440497+0000","last_unstale":"2024-09-10T18:58:23.393736+0000","last_undegraded":"2024-09-10T18:58:23.393736+0000","last_fullsized":"2024-09-10T18:58:23.393736+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:40.672810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:54:40.672810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016221689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"232'15","reported_seq":344,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.409867+0000","last_change":"2024-09-10T18:56:26.480197+0000","last_active":"2024-09-10T18:58:23.409867+0000","last_peered":"2024-09-10T18:58:23.409867+0000","last_clean":"2024-09-10T18:58:23.409867+0000","last_became_active":"2024-09-10T18:55:40.391571+0000","last_became_peered":"2024-09-10T18:55:40.391571+0000","last_unstale":"2024-09-10T18:58:23.409867+0000","last_undegraded":"2024-09-10T18:58:23.409867+0000","last_fullsized":"2024-09-10T18:58:23.409867+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T18:56:26.480105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:26.480105+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027886920000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":320,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.395643+0000","last_change":"2024-09-10T18:57:02.431878+0000","last_active":"2024-09-10T18:58:23.395643+0000","last_peered":"2024-09-10T18:58:23.395643+0000","last_clean":"2024-09-10T18:58:23.395643+0000","last_became_active":"2024-09-10T18:57:02.431585+0000","last_became_peered":"2024-09-10T18:57:02.431585+0000","last_unstale":"2024-09-10T18:58:23.395643+0000","last_undegraded":"2024-09-10T18:58:23.395643+0000","last_fullsized":"2024-09-10T18:58:23.395643+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:00.729699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:00.729699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047811370000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":250,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.398608+0000","last_change":"2024-09-10T18:57:02.444127+0000","last_active":"2024-09-10T18:58:23.398608+0000","last_peered":"2024-09-10T18:58:23.398608+0000","last_clean":"2024-09-10T18:58:23.398608+0000","last_became_active":"2024-09-10T18:57:02.442905+0000","last_became_peered":"2024-09-10T18:57:02.442905+0000","last_unstale":"2024-09-10T18:58:23.398608+0000","last_undegraded":"2024-09-10T18:58:23.398608+0000","last_fullsized":"2024-09-10T18:58:23.398608+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:59.425718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:59.425718+0000","last_clean_scrub_stamp":"2024-09-10T18:52:59.425718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"295'12497","reported_seq":12570,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-10T18:58:26.653136+0000","last_change":"2024-09-10T18:55:40.403238+0000","last_active":"2024-09-10T18:58:26.653136+0000","last_peered":"2024-09-10T18:58:26.653136+0000","last_clean":"2024-09-10T18:58:26.653136+0000","last_became_active":"2024-09-10T18:55:40.402912+0000","last_became_peered":"2024-09-10T18:55:40.402912+0000","last_unstale":"2024-09-10T18:58:26.653136+0000","last_undegraded":"2024-09-10T18:58:26.653136+0000","last_fullsized":"2024-09-10T18:58:26.653136+0000","mapping_epoch":241,"log_start":"277'9414","ondisk_log_start":"277'9414","created":237,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3083,"log_dups_size":0,"ondisk_log_size":3083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":312410112,"num_objects":4767,"num_object_clones":0,"num_object_copies":14301,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12497,"num_write_kb":552448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"70'81","reported_seq":141,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.404229+0000","last_change":"2024-09-10T18:55:31.654161+0000","last_active":"2024-09-10T18:58:23.404229+0000","last_peered":"2024-09-10T18:58:23.404229+0000","last_clean":"2024-09-10T18:58:23.404229+0000","last_became_active":"2024-09-10T18:52:54.619039+0000","last_became_peered":"2024-09-10T18:52:54.619039+0000","last_unstale":"2024-09-10T18:58:23.404229+0000","last_undegraded":"2024-09-10T18:58:23.404229+0000","last_fullsized":"2024-09-10T18:58:23.404229+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:55:31.654090+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:55:31.654090+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025082630000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"232'39","reported_seq":378,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.410681+0000","last_change":"2024-09-10T18:55:25.483970+0000","last_active":"2024-09-10T18:58:23.410681+0000","last_peered":"2024-09-10T18:58:23.410681+0000","last_clean":"2024-09-10T18:58:23.410681+0000","last_became_active":"2024-09-10T18:47:36.409300+0000","last_became_peered":"2024-09-10T18:47:36.409300+0000","last_unstale":"2024-09-10T18:58:23.410681+0000","last_undegraded":"2024-09-10T18:58:23.410681+0000","last_fullsized":"2024-09-10T18:58:23.410681+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'27","last_scrub_stamp":"2024-09-10T18:55:25.483919+0000","last_deep_scrub":"78'25","last_deep_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_clean_scrub_stamp":"2024-09-10T18:55:25.483919+0000","objects_scrubbed":8,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017172610000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":37,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.409788+0000","last_change":"2024-09-10T18:57:02.437548+0000","last_active":"2024-09-10T18:58:23.409788+0000","last_peered":"2024-09-10T18:58:23.409788+0000","last_clean":"2024-09-10T18:58:23.409788+0000","last_became_active":"2024-09-10T18:57:02.437260+0000","last_became_peered":"2024-09-10T18:57:02.437260+0000","last_unstale":"2024-09-10T18:58:23.409788+0000","last_undegraded":"2024-09-10T18:58:23.409788+0000","last_fullsized":"2024-09-10T18:58:23.409788+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:30.719475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:55:30.719475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":300,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.403615+0000","last_change":"2024-09-10T18:56:09.623123+0000","last_active":"2024-09-10T18:58:23.403615+0000","last_peered":"2024-09-10T18:58:23.403615+0000","last_clean":"2024-09-10T18:58:23.403615+0000","last_became_active":"2024-09-10T18:55:40.392449+0000","last_became_peered":"2024-09-10T18:55:40.392449+0000","last_unstale":"2024-09-10T18:58:23.403615+0000","last_undegraded":"2024-09-10T18:58:23.403615+0000","last_fullsized":"2024-09-10T18:58:23.403615+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:09.622841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:04.512425+0000","last_clean_scrub_stamp":"2024-09-10T18:56:09.622841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010581503000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":197,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.399715+0000","last_change":"2024-09-10T18:55:40.388451+0000","last_active":"2024-09-10T18:58:23.399715+0000","last_peered":"2024-09-10T18:58:23.399715+0000","last_clean":"2024-09-10T18:58:23.399715+0000","last_became_active":"2024-09-10T18:55:40.388154+0000","last_became_peered":"2024-09-10T18:55:40.388154+0000","last_unstale":"2024-09-10T18:58:23.399715+0000","last_undegraded":"2024-09-10T18:58:23.399715+0000","last_fullsized":"2024-09-10T18:58:23.399715+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:05.317828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:53:05.317828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015035510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1","version":"295'12303","reported_seq":23898,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-10T18:58:26.856393+0000","last_change":"2024-09-10T18:58:24.432220+0000","last_active":"2024-09-10T18:58:26.856393+0000","last_peered":"2024-09-10T18:58:26.856393+0000","last_clean":"2024-09-10T18:58:26.856393+0000","last_became_active":"2024-09-10T18:58:24.405312+0000","last_became_peered":"2024-09-10T18:58:24.405312+0000","last_unstale":"2024-09-10T18:58:26.856393+0000","last_undegraded":"2024-09-10T18:58:26.856393+0000","last_fullsized":"2024-09-10T18:58:26.856393+0000","mapping_epoch":294,"log_start":"276'8924","ondisk_log_start":"276'8924","created":237,"last_epoch_clean":295,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3379,"log_dups_size":0,"ondisk_log_size":3379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":307953664,"num_objects":4699,"num_object_clones":0,"num_object_copies":14097,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12303,"num_write_kb":544064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5750,"num_bytes_recovered":376832000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.9","version":"70'81","reported_seq":195,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.404562+0000","last_change":"2024-09-10T18:57:02.465293+0000","last_active":"2024-09-10T18:58:23.404562+0000","last_peered":"2024-09-10T18:58:23.404562+0000","last_clean":"2024-09-10T18:58:23.404562+0000","last_became_active":"2024-09-10T18:57:02.463750+0000","last_became_peered":"2024-09-10T18:57:02.463750+0000","last_unstale":"2024-09-10T18:58:23.404562+0000","last_undegraded":"2024-09-10T18:58:23.404562+0000","last_fullsized":"2024-09-10T18:58:23.404562+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:54:45.753792+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:54:45.753792+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016709736999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":107339,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"232'25","reported_seq":328,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.410572+0000","last_change":"2024-09-10T18:56:32.437605+0000","last_active":"2024-09-10T18:58:23.410572+0000","last_peered":"2024-09-10T18:58:23.410572+0000","last_clean":"2024-09-10T18:58:23.410572+0000","last_became_active":"2024-09-10T18:45:20.053428+0000","last_became_peered":"2024-09-10T18:45:20.053428+0000","last_unstale":"2024-09-10T18:58:23.410572+0000","last_undegraded":"2024-09-10T18:58:23.410572+0000","last_fullsized":"2024-09-10T18:58:23.410572+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'25","last_scrub_stamp":"2024-09-10T18:56:32.437503+0000","last_deep_scrub":"116'15","last_deep_scrub_stamp":"2024-09-10T18:49:12.374527+0000","last_clean_scrub_stamp":"2024-09-10T18:56:32.437503+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039673679999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":307,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.401681+0000","last_change":"2024-09-10T18:57:02.441472+0000","last_active":"2024-09-10T18:58:23.401681+0000","last_peered":"2024-09-10T18:58:23.401681+0000","last_clean":"2024-09-10T18:58:23.401681+0000","last_became_active":"2024-09-10T18:57:02.440442+0000","last_became_peered":"2024-09-10T18:57:02.440442+0000","last_unstale":"2024-09-10T18:58:23.401681+0000","last_undegraded":"2024-09-10T18:58:23.401681+0000","last_fullsized":"2024-09-10T18:58:23.401681+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:29.802937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:29.802937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044818080000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"273'13","reported_seq":315,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-10T18:58:24.439346+0000","last_change":"2024-09-10T18:57:02.463853+0000","last_active":"2024-09-10T18:58:24.439346+0000","last_peered":"2024-09-10T18:58:24.439346+0000","last_clean":"2024-09-10T18:58:24.439346+0000","last_became_active":"2024-09-10T18:57:02.448258+0000","last_became_peered":"2024-09-10T18:57:02.448258+0000","last_unstale":"2024-09-10T18:58:24.439346+0000","last_undegraded":"2024-09-10T18:58:24.439346+0000","last_fullsized":"2024-09-10T18:58:24.439346+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"242'12","last_scrub_stamp":"2024-09-10T18:56:27.995682+0000","last_deep_scrub":"202'10","last_deep_scrub_stamp":"2024-09-10T18:53:45.547686+0000","last_clean_scrub_stamp":"2024-09-10T18:56:27.995682+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025134200000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":319,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.411059+0000","last_change":"2024-09-10T18:57:02.440361+0000","last_active":"2024-09-10T18:58:23.411059+0000","last_peered":"2024-09-10T18:58:23.411059+0000","last_clean":"2024-09-10T18:58:23.411059+0000","last_became_active":"2024-09-10T18:57:02.439477+0000","last_became_peered":"2024-09-10T18:57:02.439477+0000","last_unstale":"2024-09-10T18:58:23.411059+0000","last_undegraded":"2024-09-10T18:58:23.411059+0000","last_fullsized":"2024-09-10T18:58:23.411059+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:37.565359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:32.043666+0000","last_clean_scrub_stamp":"2024-09-10T18:55:37.565359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038365510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"295'12313","reported_seq":26973,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-10T18:58:26.851056+0000","last_change":"2024-09-10T18:58:02.162234+0000","last_active":"2024-09-10T18:58:26.851056+0000","last_peered":"2024-09-10T18:58:26.851056+0000","last_clean":"2024-09-10T18:58:26.851056+0000","last_became_active":"2024-09-10T18:58:02.141866+0000","last_became_peered":"2024-09-10T18:58:02.141866+0000","last_unstale":"2024-09-10T18:58:26.851056+0000","last_undegraded":"2024-09-10T18:58:26.851056+0000","last_fullsized":"2024-09-10T18:58:26.851056+0000","mapping_epoch":284,"log_start":"275'8903","ondisk_log_start":"275'8903","created":237,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3410,"log_dups_size":0,"ondisk_log_size":3410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":310575104,"num_objects":4739,"num_object_clones":0,"num_object_copies":14217,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12313,"num_write_kb":545664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7284,"num_bytes_recovered":477364224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.a","version":"70'81","reported_seq":208,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.393093+0000","last_change":"2024-09-10T18:55:37.921571+0000","last_active":"2024-09-10T18:58:23.393093+0000","last_peered":"2024-09-10T18:58:23.393093+0000","last_clean":"2024-09-10T18:58:23.393093+0000","last_became_active":"2024-09-10T18:50:08.398345+0000","last_became_peered":"2024-09-10T18:50:08.398345+0000","last_unstale":"2024-09-10T18:58:23.393093+0000","last_undegraded":"2024-09-10T18:58:23.393093+0000","last_fullsized":"2024-09-10T18:58:23.393093+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:55:37.921519+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:55:37.921519+0000","last_clean_scrub_stamp":"2024-09-10T18:55:37.921519+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002222407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"232'34","reported_seq":312,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.393200+0000","last_change":"2024-09-10T18:56:22.727738+0000","last_active":"2024-09-10T18:58:23.393200+0000","last_peered":"2024-09-10T18:58:23.393200+0000","last_clean":"2024-09-10T18:58:23.393200+0000","last_became_active":"2024-09-10T18:46:56.742946+0000","last_became_peered":"2024-09-10T18:46:56.742946+0000","last_unstale":"2024-09-10T18:58:23.393200+0000","last_undegraded":"2024-09-10T18:58:23.393200+0000","last_fullsized":"2024-09-10T18:58:23.393200+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'34","last_scrub_stamp":"2024-09-10T18:56:22.727671+0000","last_deep_scrub":"141'22","last_deep_scrub_stamp":"2024-09-10T18:50:57.119152+0000","last_clean_scrub_stamp":"2024-09-10T18:56:22.727671+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0079020229999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":310,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.401354+0000","last_change":"2024-09-10T18:57:02.447072+0000","last_active":"2024-09-10T18:58:23.401354+0000","last_peered":"2024-09-10T18:58:23.401354+0000","last_clean":"2024-09-10T18:58:23.401354+0000","last_became_active":"2024-09-10T18:57:02.445848+0000","last_became_peered":"2024-09-10T18:57:02.445848+0000","last_unstale":"2024-09-10T18:58:23.401354+0000","last_undegraded":"2024-09-10T18:58:23.401354+0000","last_fullsized":"2024-09-10T18:58:23.401354+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:14.634438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_clean_scrub_stamp":"2024-09-10T18:56:14.634438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058886299999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"242'10","reported_seq":476,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-10T18:58:24.405598+0000","last_change":"2024-09-10T18:56:19.581070+0000","last_active":"2024-09-10T18:58:24.405598+0000","last_peered":"2024-09-10T18:58:24.405598+0000","last_clean":"2024-09-10T18:58:24.405598+0000","last_became_active":"2024-09-10T18:55:40.397246+0000","last_became_peered":"2024-09-10T18:55:40.397246+0000","last_unstale":"2024-09-10T18:58:24.405598+0000","last_undegraded":"2024-09-10T18:58:24.405598+0000","last_fullsized":"2024-09-10T18:58:24.405598+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"242'10","last_scrub_stamp":"2024-09-10T18:56:19.581012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:19.581012+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013423529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":308,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.393814+0000","last_change":"2024-09-10T18:56:01.731623+0000","last_active":"2024-09-10T18:58:23.393814+0000","last_peered":"2024-09-10T18:58:23.393814+0000","last_clean":"2024-09-10T18:58:23.393814+0000","last_became_active":"2024-09-10T18:55:40.392003+0000","last_became_peered":"2024-09-10T18:55:40.392003+0000","last_unstale":"2024-09-10T18:58:23.393814+0000","last_undegraded":"2024-09-10T18:58:23.393814+0000","last_fullsized":"2024-09-10T18:58:23.393814+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:01.731528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:54.218602+0000","last_clean_scrub_stamp":"2024-09-10T18:56:01.731528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003574701,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"295'12559","reported_seq":27934,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-10T18:58:26.849601+0000","last_change":"2024-09-10T18:57:59.139381+0000","last_active":"2024-09-10T18:58:26.849601+0000","last_peered":"2024-09-10T18:58:26.849601+0000","last_clean":"2024-09-10T18:58:26.849601+0000","last_became_active":"2024-09-10T18:57:59.111754+0000","last_became_peered":"2024-09-10T18:57:59.111754+0000","last_unstale":"2024-09-10T18:58:26.849601+0000","last_undegraded":"2024-09-10T18:58:26.849601+0000","last_fullsized":"2024-09-10T18:58:26.849601+0000","mapping_epoch":282,"log_start":"276'9117","ondisk_log_start":"276'9117","created":237,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3442,"log_dups_size":0,"ondisk_log_size":3442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":310444032,"num_objects":4737,"num_object_clones":0,"num_object_copies":14211,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12559,"num_write_kb":553472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7642,"num_bytes_recovered":500826112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.b","version":"70'81","reported_seq":250,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.403602+0000","last_change":"2024-09-10T18:56:33.793393+0000","last_active":"2024-09-10T18:58:23.403602+0000","last_peered":"2024-09-10T18:58:23.403602+0000","last_clean":"2024-09-10T18:58:23.403602+0000","last_became_active":"2024-09-10T18:55:40.401643+0000","last_became_peered":"2024-09-10T18:55:40.401643+0000","last_unstale":"2024-09-10T18:58:23.403602+0000","last_undegraded":"2024-09-10T18:58:23.403602+0000","last_fullsized":"2024-09-10T18:58:23.403602+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:33.793296+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:33.793296+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045437230000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"232'24","reported_seq":395,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.411239+0000","last_change":"2024-09-10T18:54:49.297032+0000","last_active":"2024-09-10T18:58:23.411239+0000","last_peered":"2024-09-10T18:58:23.411239+0000","last_clean":"2024-09-10T18:58:23.411239+0000","last_became_active":"2024-09-10T18:47:57.106935+0000","last_became_peered":"2024-09-10T18:47:57.106935+0000","last_unstale":"2024-09-10T18:58:23.411239+0000","last_undegraded":"2024-09-10T18:58:23.411239+0000","last_fullsized":"2024-09-10T18:58:23.411239+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"224'16","last_scrub_stamp":"2024-09-10T18:54:49.296957+0000","last_deep_scrub":"224'16","last_deep_scrub_stamp":"2024-09-10T18:54:49.296957+0000","last_clean_scrub_stamp":"2024-09-10T18:54:49.296957+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014223285,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":290,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.398787+0000","last_change":"2024-09-10T18:57:02.444295+0000","last_active":"2024-09-10T18:58:23.398787+0000","last_peered":"2024-09-10T18:58:23.398787+0000","last_clean":"2024-09-10T18:58:23.398787+0000","last_became_active":"2024-09-10T18:57:02.443816+0000","last_became_peered":"2024-09-10T18:57:02.443816+0000","last_unstale":"2024-09-10T18:58:23.398787+0000","last_undegraded":"2024-09-10T18:58:23.398787+0000","last_fullsized":"2024-09-10T18:58:23.398787+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:29.994533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:29.994533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053276879999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":296,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.393324+0000","last_change":"2024-09-10T18:55:26.909204+0000","last_active":"2024-09-10T18:58:23.393324+0000","last_peered":"2024-09-10T18:58:23.393324+0000","last_clean":"2024-09-10T18:58:23.393324+0000","last_became_active":"2024-09-10T18:45:22.068918+0000","last_became_peered":"2024-09-10T18:45:22.068918+0000","last_unstale":"2024-09-10T18:58:23.393324+0000","last_undegraded":"2024-09-10T18:58:23.393324+0000","last_fullsized":"2024-09-10T18:58:23.393324+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:26.909143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_clean_scrub_stamp":"2024-09-10T18:55:26.909143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013975070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":281,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.409116+0000","last_change":"2024-09-10T18:54:51.399064+0000","last_active":"2024-09-10T18:58:23.409116+0000","last_peered":"2024-09-10T18:58:23.409116+0000","last_clean":"2024-09-10T18:58:23.409116+0000","last_became_active":"2024-09-10T18:45:24.264538+0000","last_became_peered":"2024-09-10T18:45:24.264538+0000","last_unstale":"2024-09-10T18:58:23.409116+0000","last_undegraded":"2024-09-10T18:58:23.409116+0000","last_fullsized":"2024-09-10T18:58:23.409116+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:51.398981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:51.398981+0000","last_clean_scrub_stamp":"2024-09-10T18:54:51.398981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027589680000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"295'12429","reported_seq":26674,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-10T18:58:26.860461+0000","last_change":"2024-09-10T18:58:11.814916+0000","last_active":"2024-09-10T18:58:26.860461+0000","last_peered":"2024-09-10T18:58:26.860461+0000","last_clean":"2024-09-10T18:58:26.860461+0000","last_became_active":"2024-09-10T18:58:11.753019+0000","last_became_peered":"2024-09-10T18:58:11.753019+0000","last_unstale":"2024-09-10T18:58:26.860461+0000","last_undegraded":"2024-09-10T18:58:26.860461+0000","last_fullsized":"2024-09-10T18:58:26.860461+0000","mapping_epoch":289,"log_start":"275'8991","ondisk_log_start":"275'8991","created":237,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3438,"log_dups_size":0,"ondisk_log_size":3438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":313196544,"num_objects":4779,"num_object_clones":0,"num_object_copies":14337,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12429,"num_write_kb":550656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7076,"num_bytes_recovered":463667200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"70'81","reported_seq":229,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.409311+0000","last_change":"2024-09-10T18:57:02.460160+0000","last_active":"2024-09-10T18:58:23.409311+0000","last_peered":"2024-09-10T18:58:23.409311+0000","last_clean":"2024-09-10T18:58:23.409311+0000","last_became_active":"2024-09-10T18:57:02.459708+0000","last_became_peered":"2024-09-10T18:57:02.459708+0000","last_unstale":"2024-09-10T18:58:23.409311+0000","last_undegraded":"2024-09-10T18:58:23.409311+0000","last_fullsized":"2024-09-10T18:58:23.409311+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:54:42.406761+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:54:42.406761+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037296899999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"232'35","reported_seq":342,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.409604+0000","last_change":"2024-09-10T18:56:28.475224+0000","last_active":"2024-09-10T18:58:23.409604+0000","last_peered":"2024-09-10T18:58:23.409604+0000","last_clean":"2024-09-10T18:58:23.409604+0000","last_became_active":"2024-09-10T18:45:20.052243+0000","last_became_peered":"2024-09-10T18:45:20.052243+0000","last_unstale":"2024-09-10T18:58:23.409604+0000","last_undegraded":"2024-09-10T18:58:23.409604+0000","last_fullsized":"2024-09-10T18:58:23.409604+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'35","last_scrub_stamp":"2024-09-10T18:56:28.475142+0000","last_deep_scrub":"143'21","last_deep_scrub_stamp":"2024-09-10T18:51:02.540216+0000","last_clean_scrub_stamp":"2024-09-10T18:56:28.475142+0000","objects_scrubbed":7,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026978649999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":327,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.407061+0000","last_change":"2024-09-10T18:57:02.453766+0000","last_active":"2024-09-10T18:58:23.407061+0000","last_peered":"2024-09-10T18:58:23.407061+0000","last_clean":"2024-09-10T18:58:23.407061+0000","last_became_active":"2024-09-10T18:57:02.453404+0000","last_became_peered":"2024-09-10T18:57:02.453404+0000","last_unstale":"2024-09-10T18:58:23.407061+0000","last_undegraded":"2024-09-10T18:58:23.407061+0000","last_fullsized":"2024-09-10T18:58:23.407061+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:29.226803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_clean_scrub_stamp":"2024-09-10T18:56:29.226803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0086984899999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":459,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-10T18:58:24.400743+0000","last_change":"2024-09-10T18:56:17.914259+0000","last_active":"2024-09-10T18:58:24.400743+0000","last_peered":"2024-09-10T18:58:24.400743+0000","last_clean":"2024-09-10T18:58:24.400743+0000","last_became_active":"2024-09-10T18:45:22.074540+0000","last_became_peered":"2024-09-10T18:45:22.074540+0000","last_unstale":"2024-09-10T18:58:24.400743+0000","last_undegraded":"2024-09-10T18:58:24.400743+0000","last_fullsized":"2024-09-10T18:58:24.400743+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:56:17.914175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:17.914175+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026666329999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":84,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.406132+0000","last_change":"2024-09-10T18:57:02.439140+0000","last_active":"2024-09-10T18:58:23.406132+0000","last_peered":"2024-09-10T18:58:23.406132+0000","last_clean":"2024-09-10T18:58:23.406132+0000","last_became_active":"2024-09-10T18:57:02.437100+0000","last_became_peered":"2024-09-10T18:57:02.437100+0000","last_unstale":"2024-09-10T18:58:23.406132+0000","last_undegraded":"2024-09-10T18:58:23.406132+0000","last_fullsized":"2024-09-10T18:58:23.406132+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:49.758989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:54:49.758989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013292270000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.d","version":"295'12085","reported_seq":27033,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-10T18:58:26.653766+0000","last_change":"2024-09-10T18:57:58.141549+0000","last_active":"2024-09-10T18:58:26.653766+0000","last_peered":"2024-09-10T18:58:26.653766+0000","last_clean":"2024-09-10T18:58:26.653766+0000","last_became_active":"2024-09-10T18:57:58.118967+0000","last_became_peered":"2024-09-10T18:57:58.118967+0000","last_unstale":"2024-09-10T18:58:26.653766+0000","last_undegraded":"2024-09-10T18:58:26.653766+0000","last_fullsized":"2024-09-10T18:58:26.653766+0000","mapping_epoch":281,"log_start":"277'9051","ondisk_log_start":"277'9051","created":237,"last_epoch_clean":282,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3034,"log_dups_size":0,"ondisk_log_size":3034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":305004568,"num_objects":4655,"num_object_clones":0,"num_object_copies":13965,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4655,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12085,"num_write_kb":535617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7430,"num_bytes_recovered":486866968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"70'81","reported_seq":213,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.410244+0000","last_change":"2024-09-10T18:56:31.481362+0000","last_active":"2024-09-10T18:58:23.410244+0000","last_peered":"2024-09-10T18:58:23.410244+0000","last_clean":"2024-09-10T18:58:23.410244+0000","last_became_active":"2024-09-10T18:52:14.584798+0000","last_became_peered":"2024-09-10T18:52:14.584798+0000","last_unstale":"2024-09-10T18:58:23.410244+0000","last_undegraded":"2024-09-10T18:58:23.410244+0000","last_fullsized":"2024-09-10T18:58:23.410244+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:31.481303+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:56:31.481303+0000","last_clean_scrub_stamp":"2024-09-10T18:56:31.481303+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0011170959999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"232'30","reported_seq":405,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.405120+0000","last_change":"2024-09-10T18:57:02.490987+0000","last_active":"2024-09-10T18:58:23.405120+0000","last_peered":"2024-09-10T18:58:23.405120+0000","last_clean":"2024-09-10T18:58:23.405120+0000","last_became_active":"2024-09-10T18:57:02.455615+0000","last_became_peered":"2024-09-10T18:57:02.455615+0000","last_unstale":"2024-09-10T18:58:23.405120+0000","last_undegraded":"2024-09-10T18:58:23.405120+0000","last_fullsized":"2024-09-10T18:58:23.405120+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'30","last_scrub_stamp":"2024-09-10T18:56:15.875924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:15.875924+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0075153879999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":290,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.411170+0000","last_change":"2024-09-10T18:54:46.142189+0000","last_active":"2024-09-10T18:58:23.411170+0000","last_peered":"2024-09-10T18:58:23.411170+0000","last_clean":"2024-09-10T18:58:23.411170+0000","last_became_active":"2024-09-10T18:45:17.961695+0000","last_became_peered":"2024-09-10T18:45:17.961695+0000","last_unstale":"2024-09-10T18:58:23.411170+0000","last_undegraded":"2024-09-10T18:58:23.411170+0000","last_fullsized":"2024-09-10T18:58:23.411170+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:46.142129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:46.142129+0000","last_clean_scrub_stamp":"2024-09-10T18:54:46.142129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015906060000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":301,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.409383+0000","last_change":"2024-09-10T18:57:02.448422+0000","last_active":"2024-09-10T18:58:23.409383+0000","last_peered":"2024-09-10T18:58:23.409383+0000","last_clean":"2024-09-10T18:58:23.409383+0000","last_became_active":"2024-09-10T18:57:02.448126+0000","last_became_peered":"2024-09-10T18:57:02.448126+0000","last_unstale":"2024-09-10T18:58:23.409383+0000","last_undegraded":"2024-09-10T18:58:23.409383+0000","last_fullsized":"2024-09-10T18:58:23.409383+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:29.533177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:55:29.533177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012137999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":289,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.409758+0000","last_change":"2024-09-10T18:54:25.102613+0000","last_active":"2024-09-10T18:58:23.409758+0000","last_peered":"2024-09-10T18:58:23.409758+0000","last_clean":"2024-09-10T18:58:23.409758+0000","last_became_active":"2024-09-10T18:47:36.410836+0000","last_became_peered":"2024-09-10T18:47:36.410836+0000","last_unstale":"2024-09-10T18:58:23.409758+0000","last_undegraded":"2024-09-10T18:58:23.409758+0000","last_fullsized":"2024-09-10T18:58:23.409758+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:25.102358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_clean_scrub_stamp":"2024-09-10T18:54:25.102358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060015040000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.e","version":"295'12413","reported_seq":28050,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-10T18:58:26.204768+0000","last_change":"2024-09-10T18:57:51.717681+0000","last_active":"2024-09-10T18:58:26.204768+0000","last_peered":"2024-09-10T18:58:26.204768+0000","last_clean":"2024-09-10T18:58:26.204768+0000","last_became_active":"2024-09-10T18:57:51.701782+0000","last_became_peered":"2024-09-10T18:57:51.701782+0000","last_unstale":"2024-09-10T18:58:26.204768+0000","last_undegraded":"2024-09-10T18:58:26.204768+0000","last_fullsized":"2024-09-10T18:58:26.204768+0000","mapping_epoch":279,"log_start":"274'8762","ondisk_log_start":"274'8762","created":237,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3651,"log_dups_size":0,"ondisk_log_size":3651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":315555840,"num_objects":4815,"num_object_clones":0,"num_object_copies":14445,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12413,"num_write_kb":551296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7773,"num_bytes_recovered":509411328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"70'81","reported_seq":300,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.408179+0000","last_change":"2024-09-10T18:56:12.184278+0000","last_active":"2024-09-10T18:58:23.408179+0000","last_peered":"2024-09-10T18:58:23.408179+0000","last_clean":"2024-09-10T18:58:23.408179+0000","last_became_active":"2024-09-10T18:55:40.396828+0000","last_became_peered":"2024-09-10T18:55:40.396828+0000","last_unstale":"2024-09-10T18:58:23.408179+0000","last_undegraded":"2024-09-10T18:58:23.408179+0000","last_fullsized":"2024-09-10T18:58:23.408179+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:12.184127+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:12.184127+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040368189999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"232'29","reported_seq":342,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.397887+0000","last_change":"2024-09-10T18:57:02.473700+0000","last_active":"2024-09-10T18:58:23.397887+0000","last_peered":"2024-09-10T18:58:23.397887+0000","last_clean":"2024-09-10T18:58:23.397887+0000","last_became_active":"2024-09-10T18:57:02.442862+0000","last_became_peered":"2024-09-10T18:57:02.442862+0000","last_unstale":"2024-09-10T18:58:23.397887+0000","last_undegraded":"2024-09-10T18:58:23.397887+0000","last_fullsized":"2024-09-10T18:58:23.397887+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'19","last_scrub_stamp":"2024-09-10T18:53:38.384804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:53:38.384804+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":280,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.400857+0000","last_change":"2024-09-10T18:57:02.444015+0000","last_active":"2024-09-10T18:58:23.400857+0000","last_peered":"2024-09-10T18:58:23.400857+0000","last_clean":"2024-09-10T18:58:23.400857+0000","last_became_active":"2024-09-10T18:57:02.443522+0000","last_became_peered":"2024-09-10T18:57:02.443522+0000","last_unstale":"2024-09-10T18:58:23.400857+0000","last_undegraded":"2024-09-10T18:58:23.400857+0000","last_fullsized":"2024-09-10T18:58:23.400857+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:01.022282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:48.196065+0000","last_clean_scrub_stamp":"2024-09-10T18:56:01.022282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013037999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":310,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.406800+0000","last_change":"2024-09-10T18:56:02.162637+0000","last_active":"2024-09-10T18:58:23.406800+0000","last_peered":"2024-09-10T18:58:23.406800+0000","last_clean":"2024-09-10T18:58:23.406800+0000","last_became_active":"2024-09-10T18:55:40.387857+0000","last_became_peered":"2024-09-10T18:55:40.387857+0000","last_unstale":"2024-09-10T18:58:23.406800+0000","last_undegraded":"2024-09-10T18:58:23.406800+0000","last_fullsized":"2024-09-10T18:58:23.406800+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:02.162551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:02.162551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027989849999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":301,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.401215+0000","last_change":"2024-09-10T18:57:02.445448+0000","last_active":"2024-09-10T18:58:23.401215+0000","last_peered":"2024-09-10T18:58:23.401215+0000","last_clean":"2024-09-10T18:58:23.401215+0000","last_became_active":"2024-09-10T18:57:02.442479+0000","last_became_peered":"2024-09-10T18:57:02.442479+0000","last_unstale":"2024-09-10T18:58:23.401215+0000","last_undegraded":"2024-09-10T18:58:23.401215+0000","last_fullsized":"2024-09-10T18:58:23.401215+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:30.482659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:49:18.281230+0000","last_clean_scrub_stamp":"2024-09-10T18:55:30.482659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015073689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.f","version":"295'12408","reported_seq":12473,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-10T18:58:26.457074+0000","last_change":"2024-09-10T18:55:36.358334+0000","last_active":"2024-09-10T18:58:26.457074+0000","last_peered":"2024-09-10T18:58:26.457074+0000","last_clean":"2024-09-10T18:58:26.457074+0000","last_became_active":"2024-09-10T18:55:36.358064+0000","last_became_peered":"2024-09-10T18:55:36.358064+0000","last_unstale":"2024-09-10T18:58:26.457074+0000","last_undegraded":"2024-09-10T18:58:26.457074+0000","last_fullsized":"2024-09-10T18:58:26.457074+0000","mapping_epoch":237,"log_start":"269'7569","ondisk_log_start":"269'7569","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":4839,"log_dups_size":0,"ondisk_log_size":4839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":316276736,"num_objects":4826,"num_object_clones":0,"num_object_copies":14478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12408,"num_write_kb":551488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"70'81","reported_seq":303,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.406733+0000","last_change":"2024-09-10T18:57:02.453204+0000","last_active":"2024-09-10T18:58:23.406733+0000","last_peered":"2024-09-10T18:58:23.406733+0000","last_clean":"2024-09-10T18:58:23.406733+0000","last_became_active":"2024-09-10T18:57:02.452659+0000","last_became_peered":"2024-09-10T18:57:02.452659+0000","last_unstale":"2024-09-10T18:58:23.406733+0000","last_undegraded":"2024-09-10T18:58:23.406733+0000","last_fullsized":"2024-09-10T18:58:23.406733+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:28.264003+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:28.264003+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001853637,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"232'25","reported_seq":382,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.401563+0000","last_change":"2024-09-10T18:56:27.835783+0000","last_active":"2024-09-10T18:58:23.401563+0000","last_peered":"2024-09-10T18:58:23.401563+0000","last_clean":"2024-09-10T18:58:23.401563+0000","last_became_active":"2024-09-10T18:55:40.400777+0000","last_became_peered":"2024-09-10T18:55:40.400777+0000","last_unstale":"2024-09-10T18:58:23.401563+0000","last_undegraded":"2024-09-10T18:58:23.401563+0000","last_fullsized":"2024-09-10T18:58:23.401563+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'25","last_scrub_stamp":"2024-09-10T18:56:27.835688+0000","last_deep_scrub":"83'15","last_deep_scrub_stamp":"2024-09-10T18:46:58.436473+0000","last_clean_scrub_stamp":"2024-09-10T18:56:27.835688+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047372150000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":324,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.404192+0000","last_change":"2024-09-10T18:57:02.448298+0000","last_active":"2024-09-10T18:58:23.404192+0000","last_peered":"2024-09-10T18:58:23.404192+0000","last_clean":"2024-09-10T18:58:23.404192+0000","last_became_active":"2024-09-10T18:57:02.447940+0000","last_became_peered":"2024-09-10T18:57:02.447940+0000","last_unstale":"2024-09-10T18:58:23.404192+0000","last_undegraded":"2024-09-10T18:58:23.404192+0000","last_fullsized":"2024-09-10T18:58:23.404192+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:36.259868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:05.140027+0000","last_clean_scrub_stamp":"2024-09-10T18:53:36.259868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034952049999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"258'12","reported_seq":403,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-10T18:58:24.400183+0000","last_change":"2024-09-10T18:56:21.098913+0000","last_active":"2024-09-10T18:58:24.400183+0000","last_peered":"2024-09-10T18:58:24.400183+0000","last_clean":"2024-09-10T18:58:24.400183+0000","last_became_active":"2024-09-10T18:55:40.391094+0000","last_became_peered":"2024-09-10T18:55:40.391094+0000","last_unstale":"2024-09-10T18:58:24.400183+0000","last_undegraded":"2024-09-10T18:58:24.400183+0000","last_fullsized":"2024-09-10T18:58:24.400183+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'12","last_scrub_stamp":"2024-09-10T18:56:21.098770+0000","last_deep_scrub":"258'12","last_deep_scrub_stamp":"2024-09-10T18:56:21.098770+0000","last_clean_scrub_stamp":"2024-09-10T18:56:21.098770+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034993049999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":266,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.399013+0000","last_change":"2024-09-10T18:57:03.353073+0000","last_active":"2024-09-10T18:58:23.399013+0000","last_peered":"2024-09-10T18:58:23.399013+0000","last_clean":"2024-09-10T18:58:23.399013+0000","last_became_active":"2024-09-10T18:55:40.397478+0000","last_became_peered":"2024-09-10T18:55:40.397478+0000","last_unstale":"2024-09-10T18:58:23.399013+0000","last_undegraded":"2024-09-10T18:58:23.399013+0000","last_fullsized":"2024-09-10T18:58:23.399013+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:57:03.353014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:57:03.353014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:58:32.307182+0000","scrub_duration":0.0014199639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.9","version":"295'12535","reported_seq":26314,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-10T18:58:28.582177+0000","last_change":"2024-09-10T18:58:15.769166+0000","last_active":"2024-09-10T18:58:28.582177+0000","last_peered":"2024-09-10T18:58:28.582177+0000","last_clean":"2024-09-10T18:58:28.582177+0000","last_became_active":"2024-09-10T18:58:15.716019+0000","last_became_peered":"2024-09-10T18:58:15.716019+0000","last_unstale":"2024-09-10T18:58:28.582177+0000","last_undegraded":"2024-09-10T18:58:28.582177+0000","last_fullsized":"2024-09-10T18:58:28.582177+0000","mapping_epoch":291,"log_start":"277'9130","ondisk_log_start":"277'9130","created":237,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3405,"log_dups_size":0,"ondisk_log_size":3405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":298254336,"num_objects":4551,"num_object_clones":0,"num_object_copies":13653,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12535,"num_write_kb":546752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6845,"num_bytes_recovered":448593920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"70'81","reported_seq":232,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.393430+0000","last_change":"2024-09-10T18:56:11.765628+0000","last_active":"2024-09-10T18:58:23.393430+0000","last_peered":"2024-09-10T18:58:23.393430+0000","last_clean":"2024-09-10T18:58:23.393430+0000","last_became_active":"2024-09-10T18:55:40.392432+0000","last_became_peered":"2024-09-10T18:55:40.392432+0000","last_unstale":"2024-09-10T18:58:23.393430+0000","last_undegraded":"2024-09-10T18:58:23.393430+0000","last_fullsized":"2024-09-10T18:58:23.393430+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:11.765541+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:11.765541+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002140374,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"232'41","reported_seq":551,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-10T18:58:24.405879+0000","last_change":"2024-09-10T18:56:18.672331+0000","last_active":"2024-09-10T18:58:24.405879+0000","last_peered":"2024-09-10T18:58:24.405879+0000","last_clean":"2024-09-10T18:58:24.405879+0000","last_became_active":"2024-09-10T18:47:36.420185+0000","last_became_peered":"2024-09-10T18:47:36.420185+0000","last_unstale":"2024-09-10T18:58:24.405879+0000","last_undegraded":"2024-09-10T18:58:24.405879+0000","last_fullsized":"2024-09-10T18:58:24.405879+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'41","last_scrub_stamp":"2024-09-10T18:56:18.672139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:18.672139+0000","objects_scrubbed":10,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.020233260999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":300,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.393660+0000","last_change":"2024-09-10T18:55:29.867732+0000","last_active":"2024-09-10T18:58:23.393660+0000","last_peered":"2024-09-10T18:58:23.393660+0000","last_clean":"2024-09-10T18:58:23.393660+0000","last_became_active":"2024-09-10T18:45:17.964494+0000","last_became_peered":"2024-09-10T18:45:17.964494+0000","last_unstale":"2024-09-10T18:58:23.393660+0000","last_undegraded":"2024-09-10T18:58:23.393660+0000","last_fullsized":"2024-09-10T18:58:23.393660+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:29.867683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:12.982599+0000","last_clean_scrub_stamp":"2024-09-10T18:55:29.867683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015951909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":251,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.406393+0000","last_change":"2024-09-10T18:56:20.224668+0000","last_active":"2024-09-10T18:58:23.406393+0000","last_peered":"2024-09-10T18:58:23.406393+0000","last_clean":"2024-09-10T18:58:23.406393+0000","last_became_active":"2024-09-10T18:47:57.107478+0000","last_became_peered":"2024-09-10T18:47:57.107478+0000","last_unstale":"2024-09-10T18:58:23.406393+0000","last_undegraded":"2024-09-10T18:58:23.406393+0000","last_fullsized":"2024-09-10T18:58:23.406393+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:20.224562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:20.224562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.008542315,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":291,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.405454+0000","last_change":"2024-09-10T18:57:02.462009+0000","last_active":"2024-09-10T18:58:23.405454+0000","last_peered":"2024-09-10T18:58:23.405454+0000","last_clean":"2024-09-10T18:58:23.405454+0000","last_became_active":"2024-09-10T18:57:02.460594+0000","last_became_peered":"2024-09-10T18:57:02.460594+0000","last_unstale":"2024-09-10T18:58:23.405454+0000","last_undegraded":"2024-09-10T18:58:23.405454+0000","last_fullsized":"2024-09-10T18:58:23.405454+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:11.878181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:22.891084+0000","last_clean_scrub_stamp":"2024-09-10T18:56:11.878181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037255349999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"295'12506","reported_seq":25709,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-10T18:58:26.664412+0000","last_change":"2024-09-10T18:58:11.767121+0000","last_active":"2024-09-10T18:58:26.664412+0000","last_peered":"2024-09-10T18:58:26.664412+0000","last_clean":"2024-09-10T18:58:26.664412+0000","last_became_active":"2024-09-10T18:58:11.743985+0000","last_became_peered":"2024-09-10T18:58:11.743985+0000","last_unstale":"2024-09-10T18:58:26.664412+0000","last_undegraded":"2024-09-10T18:58:26.664412+0000","last_fullsized":"2024-09-10T18:58:26.664412+0000","mapping_epoch":289,"log_start":"277'9417","ondisk_log_start":"277'9417","created":237,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3089,"log_dups_size":0,"ondisk_log_size":3089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":312999936,"num_objects":4776,"num_object_clones":0,"num_object_copies":14328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12506,"num_write_kb":553024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6554,"num_bytes_recovered":429522944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"70'81","reported_seq":102,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.397513+0000","last_change":"2024-09-10T18:57:02.438557+0000","last_active":"2024-09-10T18:58:23.397513+0000","last_peered":"2024-09-10T18:58:23.397513+0000","last_clean":"2024-09-10T18:58:23.397513+0000","last_became_active":"2024-09-10T18:57:02.437944+0000","last_became_peered":"2024-09-10T18:57:02.437944+0000","last_unstale":"2024-09-10T18:58:23.397513+0000","last_undegraded":"2024-09-10T18:58:23.397513+0000","last_fullsized":"2024-09-10T18:58:23.397513+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:07.028031+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:07.028031+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019876809999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":5,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":53670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"232'22","reported_seq":347,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.410296+0000","last_change":"2024-09-10T18:56:05.550938+0000","last_active":"2024-09-10T18:58:23.410296+0000","last_peered":"2024-09-10T18:58:23.410296+0000","last_clean":"2024-09-10T18:58:23.410296+0000","last_became_active":"2024-09-10T18:55:40.397422+0000","last_became_peered":"2024-09-10T18:55:40.397422+0000","last_unstale":"2024-09-10T18:58:23.410296+0000","last_undegraded":"2024-09-10T18:58:23.410296+0000","last_fullsized":"2024-09-10T18:58:23.410296+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'22","last_scrub_stamp":"2024-09-10T18:56:05.550846+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-10T18:47:10.514951+0000","last_clean_scrub_stamp":"2024-09-10T18:56:05.550846+0000","objects_scrubbed":5,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0079381009999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"84'6","reported_seq":1103,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-10T18:58:25.232174+0000","last_change":"2024-09-10T18:56:22.799839+0000","last_active":"2024-09-10T18:58:25.232174+0000","last_peered":"2024-09-10T18:58:25.232174+0000","last_clean":"2024-09-10T18:58:25.232174+0000","last_became_active":"2024-09-10T18:55:40.403098+0000","last_became_peered":"2024-09-10T18:55:40.403098+0000","last_unstale":"2024-09-10T18:58:25.232174+0000","last_undegraded":"2024-09-10T18:58:25.232174+0000","last_fullsized":"2024-09-10T18:58:25.232174+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'6","last_scrub_stamp":"2024-09-10T18:56:22.799414+0000","last_deep_scrub":"84'6","last_deep_scrub_stamp":"2024-09-10T18:47:22.545450+0000","last_clean_scrub_stamp":"2024-09-10T18:56:22.799414+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004952756,"objects_trimmed":0,"snaptrim_duration":0.0021933159999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":772,"num_read_kb":767,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":265,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.403250+0000","last_change":"2024-09-10T18:56:03.791304+0000","last_active":"2024-09-10T18:58:23.403250+0000","last_peered":"2024-09-10T18:58:23.403250+0000","last_clean":"2024-09-10T18:58:23.403250+0000","last_became_active":"2024-09-10T18:55:40.393389+0000","last_became_peered":"2024-09-10T18:55:40.393389+0000","last_unstale":"2024-09-10T18:58:23.403250+0000","last_undegraded":"2024-09-10T18:58:23.403250+0000","last_fullsized":"2024-09-10T18:58:23.403250+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:03.791228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:03.791228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004016899,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":271,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.393158+0000","last_change":"2024-09-10T18:57:02.439247+0000","last_active":"2024-09-10T18:58:23.393158+0000","last_peered":"2024-09-10T18:58:23.393158+0000","last_clean":"2024-09-10T18:58:23.393158+0000","last_became_active":"2024-09-10T18:57:02.438945+0000","last_became_peered":"2024-09-10T18:57:02.438945+0000","last_unstale":"2024-09-10T18:58:23.393158+0000","last_undegraded":"2024-09-10T18:58:23.393158+0000","last_fullsized":"2024-09-10T18:58:23.393158+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:38.900146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:55:38.900146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048114799999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":37,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.406467+0000","last_change":"2024-09-10T18:57:02.454154+0000","last_active":"2024-09-10T18:58:23.406467+0000","last_peered":"2024-09-10T18:58:23.406467+0000","last_clean":"2024-09-10T18:58:23.406467+0000","last_became_active":"2024-09-10T18:57:02.453865+0000","last_became_peered":"2024-09-10T18:57:02.453865+0000","last_unstale":"2024-09-10T18:58:23.406467+0000","last_undegraded":"2024-09-10T18:58:23.406467+0000","last_fullsized":"2024-09-10T18:58:23.406467+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:21.086881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:07.150480+0000","last_clean_scrub_stamp":"2024-09-10T18:56:21.086881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.b","version":"295'12502","reported_seq":12565,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-10T18:58:28.595949+0000","last_change":"2024-09-10T18:55:40.391994+0000","last_active":"2024-09-10T18:58:28.595949+0000","last_peered":"2024-09-10T18:58:28.595949+0000","last_clean":"2024-09-10T18:58:28.595949+0000","last_became_active":"2024-09-10T18:55:40.391432+0000","last_became_peered":"2024-09-10T18:55:40.391432+0000","last_unstale":"2024-09-10T18:58:28.595949+0000","last_undegraded":"2024-09-10T18:58:28.595949+0000","last_fullsized":"2024-09-10T18:58:28.595949+0000","mapping_epoch":241,"log_start":"277'9122","ondisk_log_start":"277'9122","created":237,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3380,"log_dups_size":0,"ondisk_log_size":3380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":290062336,"num_objects":4426,"num_object_clones":0,"num_object_copies":13278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12502,"num_write_kb":541696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"70'81","reported_seq":204,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.402187+0000","last_change":"2024-09-10T18:53:00.338829+0000","last_active":"2024-09-10T18:58:23.402187+0000","last_peered":"2024-09-10T18:58:23.402187+0000","last_clean":"2024-09-10T18:58:23.402187+0000","last_became_active":"2024-09-10T18:52:14.589009+0000","last_became_peered":"2024-09-10T18:52:14.589009+0000","last_unstale":"2024-09-10T18:58:23.402187+0000","last_undegraded":"2024-09-10T18:58:23.402187+0000","last_fullsized":"2024-09-10T18:58:23.402187+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:53:00.338611+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:53:00.338611+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023458709999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"232'24","reported_seq":353,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.399762+0000","last_change":"2024-09-10T18:57:02.478395+0000","last_active":"2024-09-10T18:58:23.399762+0000","last_peered":"2024-09-10T18:58:23.399762+0000","last_clean":"2024-09-10T18:58:23.399762+0000","last_became_active":"2024-09-10T18:57:02.447673+0000","last_became_peered":"2024-09-10T18:57:02.447673+0000","last_unstale":"2024-09-10T18:58:23.399762+0000","last_undegraded":"2024-09-10T18:58:23.399762+0000","last_fullsized":"2024-09-10T18:58:23.399762+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T18:56:30.183414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:30.183414+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047701100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"58'1","reported_seq":268,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.399622+0000","last_change":"2024-09-10T18:56:06.131199+0000","last_active":"2024-09-10T18:58:23.399622+0000","last_peered":"2024-09-10T18:58:23.399622+0000","last_clean":"2024-09-10T18:58:23.399622+0000","last_became_active":"2024-09-10T18:55:40.390022+0000","last_became_peered":"2024-09-10T18:55:40.390022+0000","last_unstale":"2024-09-10T18:58:23.399622+0000","last_undegraded":"2024-09-10T18:58:23.399622+0000","last_fullsized":"2024-09-10T18:58:23.399622+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'1","last_scrub_stamp":"2024-09-10T18:56:06.131098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:06.131098+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016770345999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":238,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.401108+0000","last_change":"2024-09-10T18:57:02.447491+0000","last_active":"2024-09-10T18:58:23.401108+0000","last_peered":"2024-09-10T18:58:23.401108+0000","last_clean":"2024-09-10T18:58:23.401108+0000","last_became_active":"2024-09-10T18:57:02.447195+0000","last_became_peered":"2024-09-10T18:57:02.447195+0000","last_unstale":"2024-09-10T18:58:23.401108+0000","last_undegraded":"2024-09-10T18:58:23.401108+0000","last_fullsized":"2024-09-10T18:58:23.401108+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:24.153973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:24.153973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001771613,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":150,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.409952+0000","last_change":"2024-09-10T18:56:30.801802+0000","last_active":"2024-09-10T18:58:23.409952+0000","last_peered":"2024-09-10T18:58:23.409952+0000","last_clean":"2024-09-10T18:58:23.409952+0000","last_became_active":"2024-09-10T18:55:40.392494+0000","last_became_peered":"2024-09-10T18:55:40.392494+0000","last_unstale":"2024-09-10T18:58:23.409952+0000","last_undegraded":"2024-09-10T18:58:23.409952+0000","last_fullsized":"2024-09-10T18:58:23.409952+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:30.801727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:30.801727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027881669999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":284,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.403543+0000","last_change":"2024-09-10T18:54:17.615248+0000","last_active":"2024-09-10T18:58:23.403543+0000","last_peered":"2024-09-10T18:58:23.403543+0000","last_clean":"2024-09-10T18:58:23.403543+0000","last_became_active":"2024-09-10T18:45:24.083689+0000","last_became_peered":"2024-09-10T18:45:24.083689+0000","last_unstale":"2024-09-10T18:58:23.403543+0000","last_undegraded":"2024-09-10T18:58:23.403543+0000","last_fullsized":"2024-09-10T18:58:23.403543+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:17.615191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:54:17.615191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011240523000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"295'12577","reported_seq":22082,"reported_epoch":295,"state":"active+remapped+backfilling","last_fresh":"2024-09-10T18:58:26.854513+0000","last_change":"2024-09-10T18:58:01.092773+0000","last_active":"2024-09-10T18:58:26.854513+0000","last_peered":"2024-09-10T18:58:26.854513+0000","last_clean":"2024-09-10T18:57:01.434103+0000","last_became_active":"2024-09-10T18:57:03.556257+0000","last_became_peered":"2024-09-10T18:57:03.556257+0000","last_unstale":"2024-09-10T18:58:26.854513+0000","last_undegraded":"2024-09-10T18:58:26.854513+0000","last_fullsized":"2024-09-10T18:58:26.854513+0000","mapping_epoch":271,"log_start":"276'9138","ondisk_log_start":"276'9138","created":237,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3439,"log_dups_size":0,"ondisk_log_size":3439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":301137920,"num_objects":4595,"num_object_clones":0,"num_object_copies":13785,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":559,"num_objects_unfound":0,"num_objects_dirty":4595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12577,"num_write_kb":549504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4708,"num_bytes_recovered":308543488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,7],"avail_no_missing":["3","0","7"],"object_location_counts":[{"shards":"0,3,7","objects":4595}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"232'33","reported_seq":378,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.406523+0000","last_change":"2024-09-10T18:57:02.468556+0000","last_active":"2024-09-10T18:58:23.406523+0000","last_peered":"2024-09-10T18:58:23.406523+0000","last_clean":"2024-09-10T18:58:23.406523+0000","last_became_active":"2024-09-10T18:57:02.435499+0000","last_became_peered":"2024-09-10T18:57:02.435499+0000","last_unstale":"2024-09-10T18:58:23.406523+0000","last_undegraded":"2024-09-10T18:58:23.406523+0000","last_fullsized":"2024-09-10T18:58:23.406523+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'33","last_scrub_stamp":"2024-09-10T18:56:21.206960+0000","last_deep_scrub":"232'33","last_deep_scrub_stamp":"2024-09-10T18:56:21.206960+0000","last_clean_scrub_stamp":"2024-09-10T18:56:21.206960+0000","objects_scrubbed":7,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0042638809999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"60'2","reported_seq":300,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.404177+0000","last_change":"2024-09-10T18:54:19.313557+0000","last_active":"2024-09-10T18:58:23.404177+0000","last_peered":"2024-09-10T18:58:23.404177+0000","last_clean":"2024-09-10T18:58:23.404177+0000","last_became_active":"2024-09-10T18:45:17.961660+0000","last_became_peered":"2024-09-10T18:45:17.961660+0000","last_unstale":"2024-09-10T18:58:23.404177+0000","last_undegraded":"2024-09-10T18:58:23.404177+0000","last_fullsized":"2024-09-10T18:58:23.404177+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-10T18:54:19.313386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:54:19.313386+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056941359999999998,"objects_trimmed":0,"snaptrim_duration":0.0013942640000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":327,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.395759+0000","last_change":"2024-09-10T18:57:02.446028+0000","last_active":"2024-09-10T18:58:23.395759+0000","last_peered":"2024-09-10T18:58:23.395759+0000","last_clean":"2024-09-10T18:58:23.395759+0000","last_became_active":"2024-09-10T18:57:02.445712+0000","last_became_peered":"2024-09-10T18:57:02.445712+0000","last_unstale":"2024-09-10T18:58:23.395759+0000","last_undegraded":"2024-09-10T18:58:23.395759+0000","last_fullsized":"2024-09-10T18:58:23.395759+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:33.559651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:33.559651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0036787759999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'81","reported_seq":102,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.398452+0000","last_change":"2024-09-10T18:57:02.440391+0000","last_active":"2024-09-10T18:58:23.398452+0000","last_peered":"2024-09-10T18:58:23.398452+0000","last_clean":"2024-09-10T18:58:23.398452+0000","last_became_active":"2024-09-10T18:57:02.440063+0000","last_became_peered":"2024-09-10T18:57:02.440063+0000","last_unstale":"2024-09-10T18:58:23.398452+0000","last_undegraded":"2024-09-10T18:58:23.398452+0000","last_fullsized":"2024-09-10T18:58:23.398452+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:11.056156+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:56.085580+0000","last_clean_scrub_stamp":"2024-09-10T18:56:11.056156+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019876809999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26834,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":9,"num_write":5,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":53669,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":296,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.393918+0000","last_change":"2024-09-10T18:57:02.432025+0000","last_active":"2024-09-10T18:58:23.393918+0000","last_peered":"2024-09-10T18:58:23.393918+0000","last_clean":"2024-09-10T18:58:23.393918+0000","last_became_active":"2024-09-10T18:57:02.431556+0000","last_became_peered":"2024-09-10T18:57:02.431556+0000","last_unstale":"2024-09-10T18:58:23.393918+0000","last_undegraded":"2024-09-10T18:58:23.393918+0000","last_fullsized":"2024-09-10T18:58:23.393918+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:27.012123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:27.012123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007606036,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":301,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.393607+0000","last_change":"2024-09-10T18:56:34.565177+0000","last_active":"2024-09-10T18:58:23.393607+0000","last_peered":"2024-09-10T18:58:23.393607+0000","last_clean":"2024-09-10T18:58:23.393607+0000","last_became_active":"2024-09-10T18:45:24.073327+0000","last_became_peered":"2024-09-10T18:45:24.073327+0000","last_unstale":"2024-09-10T18:58:23.393607+0000","last_undegraded":"2024-09-10T18:58:23.393607+0000","last_fullsized":"2024-09-10T18:58:23.393607+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:34.564993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:34.564993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0031904390000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.7","version":"295'12123","reported_seq":26232,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-10T18:58:26.446964+0000","last_change":"2024-09-10T18:58:06.200926+0000","last_active":"2024-09-10T18:58:26.446964+0000","last_peered":"2024-09-10T18:58:26.446964+0000","last_clean":"2024-09-10T18:58:26.446964+0000","last_became_active":"2024-09-10T18:58:06.157679+0000","last_became_peered":"2024-09-10T18:58:06.157679+0000","last_unstale":"2024-09-10T18:58:26.446964+0000","last_undegraded":"2024-09-10T18:58:26.446964+0000","last_fullsized":"2024-09-10T18:58:26.446964+0000","mapping_epoch":286,"log_start":"268'7275","ondisk_log_start":"268'7275","created":237,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":4848,"log_dups_size":0,"ondisk_log_size":4848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":305725440,"num_objects":4665,"num_object_clones":0,"num_object_copies":13995,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12123,"num_write_kb":537216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7010,"num_bytes_recovered":459407360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.f","version":"70'81","reported_seq":295,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.406905+0000","last_change":"2024-09-10T18:55:29.018036+0000","last_active":"2024-09-10T18:58:23.406905+0000","last_peered":"2024-09-10T18:58:23.406905+0000","last_clean":"2024-09-10T18:58:23.406905+0000","last_became_active":"2024-09-10T18:52:16.598295+0000","last_became_peered":"2024-09-10T18:52:16.598295+0000","last_unstale":"2024-09-10T18:58:23.406905+0000","last_undegraded":"2024-09-10T18:58:23.406905+0000","last_fullsized":"2024-09-10T18:58:23.406905+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:55:29.017936+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:53.609885+0000","last_clean_scrub_stamp":"2024-09-10T18:55:29.017936+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001646813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"232'35","reported_seq":393,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.406985+0000","last_change":"2024-09-10T18:57:02.466701+0000","last_active":"2024-09-10T18:58:23.406985+0000","last_peered":"2024-09-10T18:58:23.406985+0000","last_clean":"2024-09-10T18:58:23.406985+0000","last_became_active":"2024-09-10T18:57:02.436059+0000","last_became_peered":"2024-09-10T18:57:02.436059+0000","last_unstale":"2024-09-10T18:58:23.406985+0000","last_undegraded":"2024-09-10T18:58:23.406985+0000","last_fullsized":"2024-09-10T18:58:23.406985+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'35","last_scrub_stamp":"2024-09-10T18:56:34.373736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:34.373736+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016527315000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":275,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.393840+0000","last_change":"2024-09-10T18:56:13.760318+0000","last_active":"2024-09-10T18:58:23.393840+0000","last_peered":"2024-09-10T18:58:23.393840+0000","last_clean":"2024-09-10T18:58:23.393840+0000","last_became_active":"2024-09-10T18:55:40.391849+0000","last_became_peered":"2024-09-10T18:55:40.391849+0000","last_unstale":"2024-09-10T18:58:23.393840+0000","last_undegraded":"2024-09-10T18:58:23.393840+0000","last_fullsized":"2024-09-10T18:58:23.393840+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:13.760238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:13.760238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.048619793000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":235,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.397525+0000","last_change":"2024-09-10T18:57:02.451716+0000","last_active":"2024-09-10T18:58:23.397525+0000","last_peered":"2024-09-10T18:58:23.397525+0000","last_clean":"2024-09-10T18:58:23.397525+0000","last_became_active":"2024-09-10T18:57:02.451332+0000","last_became_peered":"2024-09-10T18:57:02.451332+0000","last_unstale":"2024-09-10T18:58:23.397525+0000","last_undegraded":"2024-09-10T18:58:23.397525+0000","last_fullsized":"2024-09-10T18:58:23.397525+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:24.158403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:24.158403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.02305596,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":94,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.404889+0000","last_change":"2024-09-10T18:57:03.015513+0000","last_active":"2024-09-10T18:58:23.404889+0000","last_peered":"2024-09-10T18:58:23.404889+0000","last_clean":"2024-09-10T18:58:23.404889+0000","last_became_active":"2024-09-10T18:57:02.468866+0000","last_became_peered":"2024-09-10T18:57:02.468866+0000","last_unstale":"2024-09-10T18:58:23.404889+0000","last_undegraded":"2024-09-10T18:58:23.404889+0000","last_fullsized":"2024-09-10T18:58:23.404889+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:57:03.015457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:57:03.015457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001545795,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.6","version":"295'12471","reported_seq":12537,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-10T18:58:26.452066+0000","last_change":"2024-09-10T18:55:36.358103+0000","last_active":"2024-09-10T18:58:26.452066+0000","last_peered":"2024-09-10T18:58:26.452066+0000","last_clean":"2024-09-10T18:58:26.452066+0000","last_became_active":"2024-09-10T18:55:36.357828+0000","last_became_peered":"2024-09-10T18:55:36.357828+0000","last_unstale":"2024-09-10T18:58:26.452066+0000","last_undegraded":"2024-09-10T18:58:26.452066+0000","last_fullsized":"2024-09-10T18:58:26.452066+0000","mapping_epoch":237,"log_start":"269'7627","ondisk_log_start":"269'7627","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":4844,"log_dups_size":0,"ondisk_log_size":4844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":312541184,"num_objects":4769,"num_object_clones":0,"num_object_copies":14307,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12471,"num_write_kb":551680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.e","version":"70'81","reported_seq":164,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.411155+0000","last_change":"2024-09-10T18:57:02.450954+0000","last_active":"2024-09-10T18:58:23.411155+0000","last_peered":"2024-09-10T18:58:23.411155+0000","last_clean":"2024-09-10T18:58:23.411155+0000","last_became_active":"2024-09-10T18:57:02.449115+0000","last_became_peered":"2024-09-10T18:57:02.449115+0000","last_unstale":"2024-09-10T18:58:23.411155+0000","last_undegraded":"2024-09-10T18:58:23.411155+0000","last_fullsized":"2024-09-10T18:58:23.411155+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:55:30.496910+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:55:30.496910+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012110790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"232'19","reported_seq":301,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.401085+0000","last_change":"2024-09-10T18:56:20.061554+0000","last_active":"2024-09-10T18:58:23.401085+0000","last_peered":"2024-09-10T18:58:23.401085+0000","last_clean":"2024-09-10T18:58:23.401085+0000","last_became_active":"2024-09-10T18:55:40.395257+0000","last_became_peered":"2024-09-10T18:55:40.395257+0000","last_unstale":"2024-09-10T18:58:23.401085+0000","last_undegraded":"2024-09-10T18:58:23.401085+0000","last_fullsized":"2024-09-10T18:58:23.401085+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T18:56:20.061420+0000","last_deep_scrub":"130'13","last_deep_scrub_stamp":"2024-09-10T18:51:58.427534+0000","last_clean_scrub_stamp":"2024-09-10T18:56:20.061420+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0065809780000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":287,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.410450+0000","last_change":"2024-09-10T18:54:52.179168+0000","last_active":"2024-09-10T18:58:23.410450+0000","last_peered":"2024-09-10T18:58:23.410450+0000","last_clean":"2024-09-10T18:58:23.410450+0000","last_became_active":"2024-09-10T18:45:17.963057+0000","last_became_peered":"2024-09-10T18:45:17.963057+0000","last_unstale":"2024-09-10T18:58:23.410450+0000","last_undegraded":"2024-09-10T18:58:23.410450+0000","last_fullsized":"2024-09-10T18:58:23.410450+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:52.179074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:54:52.179074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001414738,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":263,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.410010+0000","last_change":"2024-09-10T18:56:17.661917+0000","last_active":"2024-09-10T18:58:23.410010+0000","last_peered":"2024-09-10T18:58:23.410010+0000","last_clean":"2024-09-10T18:58:23.410010+0000","last_became_active":"2024-09-10T18:55:40.390771+0000","last_became_peered":"2024-09-10T18:55:40.390771+0000","last_unstale":"2024-09-10T18:58:23.410010+0000","last_undegraded":"2024-09-10T18:58:23.410010+0000","last_fullsized":"2024-09-10T18:58:23.410010+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:17.661832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_clean_scrub_stamp":"2024-09-10T18:56:17.661832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041460890000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":289,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.397934+0000","last_change":"2024-09-10T18:57:02.453927+0000","last_active":"2024-09-10T18:58:23.397934+0000","last_peered":"2024-09-10T18:58:23.397934+0000","last_clean":"2024-09-10T18:58:23.397934+0000","last_became_active":"2024-09-10T18:57:02.452387+0000","last_became_peered":"2024-09-10T18:57:02.452387+0000","last_unstale":"2024-09-10T18:58:23.397934+0000","last_undegraded":"2024-09-10T18:58:23.397934+0000","last_fullsized":"2024-09-10T18:58:23.397934+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:32.217641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:53.345184+0000","last_clean_scrub_stamp":"2024-09-10T18:56:32.217641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0059447470000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.5","version":"295'12399","reported_seq":26726,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-10T18:58:28.593714+0000","last_change":"2024-09-10T18:58:07.213556+0000","last_active":"2024-09-10T18:58:28.593714+0000","last_peered":"2024-09-10T18:58:28.593714+0000","last_clean":"2024-09-10T18:58:28.593714+0000","last_became_active":"2024-09-10T18:58:07.161586+0000","last_became_peered":"2024-09-10T18:58:07.161586+0000","last_unstale":"2024-09-10T18:58:28.593714+0000","last_undegraded":"2024-09-10T18:58:28.593714+0000","last_fullsized":"2024-09-10T18:58:28.593714+0000","mapping_epoch":287,"log_start":"276'8952","ondisk_log_start":"276'8952","created":237,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3447,"log_dups_size":0,"ondisk_log_size":3447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":299302912,"num_objects":4567,"num_object_clones":0,"num_object_copies":13701,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12399,"num_write_kb":542912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7119,"num_bytes_recovered":466550784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.d","version":"70'81","reported_seq":102,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.410601+0000","last_change":"2024-09-10T18:57:02.439985+0000","last_active":"2024-09-10T18:58:23.410601+0000","last_peered":"2024-09-10T18:58:23.410601+0000","last_clean":"2024-09-10T18:58:23.410601+0000","last_became_active":"2024-09-10T18:57:02.439652+0000","last_became_peered":"2024-09-10T18:57:02.439652+0000","last_unstale":"2024-09-10T18:58:23.410601+0000","last_undegraded":"2024-09-10T18:58:23.410601+0000","last_fullsized":"2024-09-10T18:58:23.410601+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:18.041116+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:18.041116+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014613810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"232'27","reported_seq":196,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.405783+0000","last_change":"2024-09-10T18:57:02.491285+0000","last_active":"2024-09-10T18:58:23.405783+0000","last_peered":"2024-09-10T18:58:23.405783+0000","last_clean":"2024-09-10T18:58:23.405783+0000","last_became_active":"2024-09-10T18:57:02.454653+0000","last_became_peered":"2024-09-10T18:57:02.454653+0000","last_unstale":"2024-09-10T18:58:23.405783+0000","last_undegraded":"2024-09-10T18:58:23.405783+0000","last_fullsized":"2024-09-10T18:58:23.405783+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'15","last_scrub_stamp":"2024-09-10T18:54:22.720737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:54:22.720737+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014176382,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":33,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3],"acting":[6,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":258,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.405578+0000","last_change":"2024-09-10T18:57:02.463092+0000","last_active":"2024-09-10T18:58:23.405578+0000","last_peered":"2024-09-10T18:58:23.405578+0000","last_clean":"2024-09-10T18:58:23.405578+0000","last_became_active":"2024-09-10T18:57:02.461708+0000","last_became_peered":"2024-09-10T18:57:02.461708+0000","last_unstale":"2024-09-10T18:58:23.405578+0000","last_undegraded":"2024-09-10T18:58:23.405578+0000","last_fullsized":"2024-09-10T18:58:23.405578+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:24.902205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:05.603125+0000","last_clean_scrub_stamp":"2024-09-10T18:56:24.902205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0076874830000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"273'10","reported_seq":126,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-10T18:58:24.399062+0000","last_change":"2024-09-10T18:57:02.476362+0000","last_active":"2024-09-10T18:58:24.399062+0000","last_peered":"2024-09-10T18:58:24.399062+0000","last_clean":"2024-09-10T18:58:24.399062+0000","last_became_active":"2024-09-10T18:57:02.441890+0000","last_became_peered":"2024-09-10T18:57:02.441890+0000","last_unstale":"2024-09-10T18:58:24.399062+0000","last_undegraded":"2024-09-10T18:58:24.399062+0000","last_fullsized":"2024-09-10T18:58:24.399062+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'9","last_scrub_stamp":"2024-09-10T18:55:37.750784+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-09-10T18:47:02.582229+0000","last_clean_scrub_stamp":"2024-09-10T18:55:37.750784+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":308,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.393168+0000","last_change":"2024-09-10T18:56:10.775911+0000","last_active":"2024-09-10T18:58:23.393168+0000","last_peered":"2024-09-10T18:58:23.393168+0000","last_clean":"2024-09-10T18:58:23.393168+0000","last_became_active":"2024-09-10T18:55:40.389202+0000","last_became_peered":"2024-09-10T18:55:40.389202+0000","last_unstale":"2024-09-10T18:58:23.393168+0000","last_undegraded":"2024-09-10T18:58:23.393168+0000","last_fullsized":"2024-09-10T18:58:23.393168+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:10.775814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_clean_scrub_stamp":"2024-09-10T18:56:10.775814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0031802459999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.4","version":"295'12265","reported_seq":12331,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-10T18:58:26.674025+0000","last_change":"2024-09-10T18:55:36.350444+0000","last_active":"2024-09-10T18:58:26.674025+0000","last_peered":"2024-09-10T18:58:26.674025+0000","last_clean":"2024-09-10T18:58:26.674025+0000","last_became_active":"2024-09-10T18:55:36.350155+0000","last_became_peered":"2024-09-10T18:55:36.350155+0000","last_unstale":"2024-09-10T18:58:26.674025+0000","last_undegraded":"2024-09-10T18:58:26.674025+0000","last_fullsized":"2024-09-10T18:58:26.674025+0000","mapping_epoch":237,"log_start":"277'9212","ondisk_log_start":"277'9212","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3053,"log_dups_size":0,"ondisk_log_size":3053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":311230464,"num_objects":4749,"num_object_clones":0,"num_object_copies":14247,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12265,"num_write_kb":544448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.c","version":"70'81","reported_seq":251,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.408900+0000","last_change":"2024-09-10T18:56:33.779212+0000","last_active":"2024-09-10T18:58:23.408900+0000","last_peered":"2024-09-10T18:58:23.408900+0000","last_clean":"2024-09-10T18:58:23.408900+0000","last_became_active":"2024-09-10T18:55:40.390655+0000","last_became_peered":"2024-09-10T18:55:40.390655+0000","last_unstale":"2024-09-10T18:58:23.408900+0000","last_undegraded":"2024-09-10T18:58:23.408900+0000","last_fullsized":"2024-09-10T18:58:23.408900+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:33.779154+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:56:33.779154+0000","last_clean_scrub_stamp":"2024-09-10T18:56:33.779154+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0075121179999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"232'18","reported_seq":325,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.409522+0000","last_change":"2024-09-10T18:55:27.507437+0000","last_active":"2024-09-10T18:58:23.409522+0000","last_peered":"2024-09-10T18:58:23.409522+0000","last_clean":"2024-09-10T18:58:23.409522+0000","last_became_active":"2024-09-10T18:45:20.056869+0000","last_became_peered":"2024-09-10T18:45:20.056869+0000","last_unstale":"2024-09-10T18:58:23.409522+0000","last_undegraded":"2024-09-10T18:58:23.409522+0000","last_fullsized":"2024-09-10T18:58:23.409522+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'18","last_scrub_stamp":"2024-09-10T18:55:27.507364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:55:27.507364+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030221390000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"56'2","reported_seq":335,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.411013+0000","last_change":"2024-09-10T18:57:02.456385+0000","last_active":"2024-09-10T18:58:23.411013+0000","last_peered":"2024-09-10T18:58:23.411013+0000","last_clean":"2024-09-10T18:58:23.411013+0000","last_became_active":"2024-09-10T18:57:02.434475+0000","last_became_peered":"2024-09-10T18:57:02.434475+0000","last_unstale":"2024-09-10T18:58:23.411013+0000","last_undegraded":"2024-09-10T18:58:23.411013+0000","last_fullsized":"2024-09-10T18:58:23.411013+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-10T18:56:01.630091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:01.630091+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001578238,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":235,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.398919+0000","last_change":"2024-09-10T18:56:16.019721+0000","last_active":"2024-09-10T18:58:23.398919+0000","last_peered":"2024-09-10T18:58:23.398919+0000","last_clean":"2024-09-10T18:58:23.398919+0000","last_became_active":"2024-09-10T18:55:40.397290+0000","last_became_peered":"2024-09-10T18:55:40.397290+0000","last_unstale":"2024-09-10T18:58:23.398919+0000","last_undegraded":"2024-09-10T18:58:23.398919+0000","last_fullsized":"2024-09-10T18:58:23.398919+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:16.019357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:02.711594+0000","last_clean_scrub_stamp":"2024-09-10T18:56:16.019357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021619629999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":291,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.411329+0000","last_change":"2024-09-10T18:57:02.461727+0000","last_active":"2024-09-10T18:58:23.411329+0000","last_peered":"2024-09-10T18:58:23.411329+0000","last_clean":"2024-09-10T18:58:23.411329+0000","last_became_active":"2024-09-10T18:57:02.459985+0000","last_became_peered":"2024-09-10T18:57:02.459985+0000","last_unstale":"2024-09-10T18:58:23.411329+0000","last_undegraded":"2024-09-10T18:58:23.411329+0000","last_fullsized":"2024-09-10T18:58:23.411329+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:53.352427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:53.352427+0000","last_clean_scrub_stamp":"2024-09-10T18:54:53.352427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022835899999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.13","version":"70'81","reported_seq":223,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.403030+0000","last_change":"2024-09-10T18:56:34.760114+0000","last_active":"2024-09-10T18:58:23.403030+0000","last_peered":"2024-09-10T18:58:23.403030+0000","last_clean":"2024-09-10T18:58:23.403030+0000","last_became_active":"2024-09-10T18:52:16.602463+0000","last_became_peered":"2024-09-10T18:52:16.602463+0000","last_unstale":"2024-09-10T18:58:23.403030+0000","last_undegraded":"2024-09-10T18:58:23.403030+0000","last_fullsized":"2024-09-10T18:58:23.403030+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:34.760031+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:34.760031+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002735595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"232'19","reported_seq":324,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.399626+0000","last_change":"2024-09-10T18:57:02.485836+0000","last_active":"2024-09-10T18:58:23.399626+0000","last_peered":"2024-09-10T18:58:23.399626+0000","last_clean":"2024-09-10T18:58:23.399626+0000","last_became_active":"2024-09-10T18:57:02.453001+0000","last_became_peered":"2024-09-10T18:57:02.453001+0000","last_unstale":"2024-09-10T18:58:23.399626+0000","last_undegraded":"2024-09-10T18:58:23.399626+0000","last_fullsized":"2024-09-10T18:58:23.399626+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T18:56:25.122070+0000","last_deep_scrub":"76'13","last_deep_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_clean_scrub_stamp":"2024-09-10T18:56:25.122070+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0061142590000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":275,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.400964+0000","last_change":"2024-09-10T18:56:25.820426+0000","last_active":"2024-09-10T18:58:23.400964+0000","last_peered":"2024-09-10T18:58:23.400964+0000","last_clean":"2024-09-10T18:58:23.400964+0000","last_became_active":"2024-09-10T18:55:40.401037+0000","last_became_peered":"2024-09-10T18:55:40.401037+0000","last_unstale":"2024-09-10T18:58:23.400964+0000","last_undegraded":"2024-09-10T18:58:23.400964+0000","last_fullsized":"2024-09-10T18:58:23.400964+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:25.820339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:25.820339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048533279999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":303,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.411593+0000","last_change":"2024-09-10T18:57:02.455638+0000","last_active":"2024-09-10T18:58:23.411593+0000","last_peered":"2024-09-10T18:58:23.411593+0000","last_clean":"2024-09-10T18:58:23.411593+0000","last_became_active":"2024-09-10T18:57:02.455341+0000","last_became_peered":"2024-09-10T18:57:02.455341+0000","last_unstale":"2024-09-10T18:58:23.411593+0000","last_undegraded":"2024-09-10T18:58:23.411593+0000","last_fullsized":"2024-09-10T18:58:23.411593+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:25.862853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:00.754912+0000","last_clean_scrub_stamp":"2024-09-10T18:56:25.862853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0033952399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"63'1","reported_seq":293,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.410275+0000","last_change":"2024-09-10T18:57:02.456685+0000","last_active":"2024-09-10T18:58:23.410275+0000","last_peered":"2024-09-10T18:58:23.410275+0000","last_clean":"2024-09-10T18:58:23.410275+0000","last_became_active":"2024-09-10T18:57:02.435896+0000","last_became_peered":"2024-09-10T18:57:02.435896+0000","last_unstale":"2024-09-10T18:58:23.410275+0000","last_undegraded":"2024-09-10T18:58:23.410275+0000","last_fullsized":"2024-09-10T18:58:23.410275+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-10T18:56:07.774334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:07.774334+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.090386770000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.12","version":"70'81","reported_seq":155,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.393338+0000","last_change":"2024-09-10T18:54:50.968303+0000","last_active":"2024-09-10T18:58:23.393338+0000","last_peered":"2024-09-10T18:58:23.393338+0000","last_clean":"2024-09-10T18:58:23.393338+0000","last_became_active":"2024-09-10T18:52:16.598921+0000","last_became_peered":"2024-09-10T18:52:16.598921+0000","last_unstale":"2024-09-10T18:58:23.393338+0000","last_undegraded":"2024-09-10T18:58:23.393338+0000","last_fullsized":"2024-09-10T18:58:23.393338+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:54:50.968202+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:54.127476+0000","last_clean_scrub_stamp":"2024-09-10T18:54:50.968202+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0074710000000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":320,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.406532+0000","last_change":"2024-09-10T18:57:02.442633+0000","last_active":"2024-09-10T18:58:23.406532+0000","last_peered":"2024-09-10T18:58:23.406532+0000","last_clean":"2024-09-10T18:58:23.406532+0000","last_became_active":"2024-09-10T18:57:02.442251+0000","last_became_peered":"2024-09-10T18:57:02.442251+0000","last_unstale":"2024-09-10T18:58:23.406532+0000","last_undegraded":"2024-09-10T18:58:23.406532+0000","last_fullsized":"2024-09-10T18:58:23.406532+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.136432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.136432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015178839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"232'12","reported_seq":324,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.411875+0000","last_change":"2024-09-10T18:57:02.483803+0000","last_active":"2024-09-10T18:58:23.411875+0000","last_peered":"2024-09-10T18:58:23.411875+0000","last_clean":"2024-09-10T18:58:23.411875+0000","last_became_active":"2024-09-10T18:57:02.460430+0000","last_became_peered":"2024-09-10T18:57:02.460430+0000","last_unstale":"2024-09-10T18:58:23.411875+0000","last_undegraded":"2024-09-10T18:58:23.411875+0000","last_fullsized":"2024-09-10T18:58:23.411875+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'12","last_scrub_stamp":"2024-09-10T18:55:30.612808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:55:30.612808+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001922787,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":293,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.393529+0000","last_change":"2024-09-10T18:56:32.602499+0000","last_active":"2024-09-10T18:58:23.393529+0000","last_peered":"2024-09-10T18:58:23.393529+0000","last_clean":"2024-09-10T18:58:23.393529+0000","last_became_active":"2024-09-10T18:45:22.069653+0000","last_became_peered":"2024-09-10T18:45:22.069653+0000","last_unstale":"2024-09-10T18:58:23.393529+0000","last_undegraded":"2024-09-10T18:58:23.393529+0000","last_fullsized":"2024-09-10T18:58:23.393529+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:32.602448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:32.602448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0052503430000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":37,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.406454+0000","last_change":"2024-09-10T18:57:02.438506+0000","last_active":"2024-09-10T18:58:23.406454+0000","last_peered":"2024-09-10T18:58:23.406454+0000","last_clean":"2024-09-10T18:58:23.406454+0000","last_became_active":"2024-09-10T18:57:02.437979+0000","last_became_peered":"2024-09-10T18:57:02.437979+0000","last_unstale":"2024-09-10T18:58:23.406454+0000","last_undegraded":"2024-09-10T18:58:23.406454+0000","last_fullsized":"2024-09-10T18:58:23.406454+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:47.741071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:54:47.741071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.11","version":"70'81","reported_seq":179,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.399320+0000","last_change":"2024-09-10T18:55:40.395720+0000","last_active":"2024-09-10T18:58:23.399320+0000","last_peered":"2024-09-10T18:58:23.399320+0000","last_clean":"2024-09-10T18:58:23.399320+0000","last_became_active":"2024-09-10T18:55:40.395019+0000","last_became_peered":"2024-09-10T18:55:40.395019+0000","last_unstale":"2024-09-10T18:58:23.399320+0000","last_undegraded":"2024-09-10T18:58:23.399320+0000","last_fullsized":"2024-09-10T18:58:23.399320+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:52:53.556903+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:52:53.556903+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004894626,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"232'20","reported_seq":381,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.406163+0000","last_change":"2024-09-10T18:57:02.468721+0000","last_active":"2024-09-10T18:58:23.406163+0000","last_peered":"2024-09-10T18:58:23.406163+0000","last_clean":"2024-09-10T18:58:23.406163+0000","last_became_active":"2024-09-10T18:57:02.444437+0000","last_became_peered":"2024-09-10T18:57:02.444437+0000","last_unstale":"2024-09-10T18:58:23.406163+0000","last_undegraded":"2024-09-10T18:58:23.406163+0000","last_fullsized":"2024-09-10T18:58:23.406163+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'20","last_scrub_stamp":"2024-09-10T18:55:33.667712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:55:33.667712+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0028302039999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":307,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.393586+0000","last_change":"2024-09-10T18:57:02.446113+0000","last_active":"2024-09-10T18:58:23.393586+0000","last_peered":"2024-09-10T18:58:23.393586+0000","last_clean":"2024-09-10T18:58:23.393586+0000","last_became_active":"2024-09-10T18:57:02.445710+0000","last_became_peered":"2024-09-10T18:57:02.445710+0000","last_unstale":"2024-09-10T18:58:23.393586+0000","last_undegraded":"2024-09-10T18:58:23.393586+0000","last_fullsized":"2024-09-10T18:58:23.393586+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:51.965637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:54:51.965637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011107304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"242'10","reported_seq":478,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-10T18:58:24.439550+0000","last_change":"2024-09-10T18:56:07.787017+0000","last_active":"2024-09-10T18:58:24.439550+0000","last_peered":"2024-09-10T18:58:24.439550+0000","last_clean":"2024-09-10T18:58:24.439550+0000","last_became_active":"2024-09-10T18:55:40.391439+0000","last_became_peered":"2024-09-10T18:55:40.391439+0000","last_unstale":"2024-09-10T18:58:24.439550+0000","last_undegraded":"2024-09-10T18:58:24.439550+0000","last_fullsized":"2024-09-10T18:58:24.439550+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"242'10","last_scrub_stamp":"2024-09-10T18:56:07.786933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:07.786933+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0054185049999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":248,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.399288+0000","last_change":"2024-09-10T18:57:02.454114+0000","last_active":"2024-09-10T18:58:23.399288+0000","last_peered":"2024-09-10T18:58:23.399288+0000","last_clean":"2024-09-10T18:58:23.399288+0000","last_became_active":"2024-09-10T18:57:02.452949+0000","last_became_peered":"2024-09-10T18:57:02.452949+0000","last_unstale":"2024-09-10T18:58:23.399288+0000","last_undegraded":"2024-09-10T18:58:23.399288+0000","last_fullsized":"2024-09-10T18:58:23.399288+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:52:57.476724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:57.476724+0000","last_clean_scrub_stamp":"2024-09-10T18:52:57.476724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.10","version":"70'81","reported_seq":165,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.392816+0000","last_change":"2024-09-10T18:57:02.444347+0000","last_active":"2024-09-10T18:58:23.392816+0000","last_peered":"2024-09-10T18:58:23.392816+0000","last_clean":"2024-09-10T18:58:23.392816+0000","last_became_active":"2024-09-10T18:57:02.443782+0000","last_became_peered":"2024-09-10T18:57:02.443782+0000","last_unstale":"2024-09-10T18:58:23.392816+0000","last_undegraded":"2024-09-10T18:58:23.392816+0000","last_fullsized":"2024-09-10T18:58:23.392816+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:20.743543+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:20.743543+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043223280000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"232'15","reported_seq":159,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.404024+0000","last_change":"2024-09-10T18:57:02.487296+0000","last_active":"2024-09-10T18:58:23.404024+0000","last_peered":"2024-09-10T18:58:23.404024+0000","last_clean":"2024-09-10T18:58:23.404024+0000","last_became_active":"2024-09-10T18:57:02.460705+0000","last_became_peered":"2024-09-10T18:57:02.460705+0000","last_unstale":"2024-09-10T18:58:23.404024+0000","last_undegraded":"2024-09-10T18:58:23.404024+0000","last_fullsized":"2024-09-10T18:58:23.404024+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T18:55:25.784333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:55:25.784333+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":273,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.397675+0000","last_change":"2024-09-10T18:56:23.096461+0000","last_active":"2024-09-10T18:58:23.397675+0000","last_peered":"2024-09-10T18:58:23.397675+0000","last_clean":"2024-09-10T18:58:23.397675+0000","last_became_active":"2024-09-10T18:55:40.390697+0000","last_became_peered":"2024-09-10T18:55:40.390697+0000","last_unstale":"2024-09-10T18:58:23.397675+0000","last_undegraded":"2024-09-10T18:58:23.397675+0000","last_fullsized":"2024-09-10T18:58:23.397675+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:23.096373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:23.096373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0057615890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"273'11","reported_seq":478,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-10T18:58:24.400793+0000","last_change":"2024-09-10T18:57:02.473032+0000","last_active":"2024-09-10T18:58:24.400793+0000","last_peered":"2024-09-10T18:58:24.400793+0000","last_clean":"2024-09-10T18:58:24.400793+0000","last_became_active":"2024-09-10T18:57:02.445268+0000","last_became_peered":"2024-09-10T18:57:02.445268+0000","last_unstale":"2024-09-10T18:58:24.400793+0000","last_undegraded":"2024-09-10T18:58:24.400793+0000","last_fullsized":"2024-09-10T18:58:24.400793+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'10","last_scrub_stamp":"2024-09-10T18:55:36.732908+0000","last_deep_scrub":"81'9","last_deep_scrub_stamp":"2024-09-10T18:46:54.800790+0000","last_clean_scrub_stamp":"2024-09-10T18:55:36.732908+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016257330000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":316,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.410783+0000","last_change":"2024-09-10T18:56:20.584899+0000","last_active":"2024-09-10T18:58:23.410783+0000","last_peered":"2024-09-10T18:58:23.410783+0000","last_clean":"2024-09-10T18:58:23.410783+0000","last_became_active":"2024-09-10T18:55:40.391662+0000","last_became_peered":"2024-09-10T18:55:40.391662+0000","last_unstale":"2024-09-10T18:58:23.410783+0000","last_undegraded":"2024-09-10T18:58:23.410783+0000","last_fullsized":"2024-09-10T18:58:23.410783+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:20.584808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_clean_scrub_stamp":"2024-09-10T18:56:20.584808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0035961249999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"232'15","reported_seq":359,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.401428+0000","last_change":"2024-09-10T18:56:02.500313+0000","last_active":"2024-09-10T18:58:23.401428+0000","last_peered":"2024-09-10T18:58:23.401428+0000","last_clean":"2024-09-10T18:58:23.401428+0000","last_became_active":"2024-09-10T18:55:40.400788+0000","last_became_peered":"2024-09-10T18:55:40.400788+0000","last_unstale":"2024-09-10T18:58:23.401428+0000","last_undegraded":"2024-09-10T18:58:23.401428+0000","last_fullsized":"2024-09-10T18:58:23.401428+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T18:56:02.500101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:02.500101+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0097839320000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":327,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.406621+0000","last_change":"2024-09-10T18:57:02.453289+0000","last_active":"2024-09-10T18:58:23.406621+0000","last_peered":"2024-09-10T18:58:23.406621+0000","last_clean":"2024-09-10T18:58:23.406621+0000","last_became_active":"2024-09-10T18:57:02.452986+0000","last_became_peered":"2024-09-10T18:57:02.452986+0000","last_unstale":"2024-09-10T18:58:23.406621+0000","last_undegraded":"2024-09-10T18:58:23.406621+0000","last_fullsized":"2024-09-10T18:58:23.406621+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:18.272126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:18.272126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0035259570000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":300,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.411455+0000","last_change":"2024-09-10T18:56:04.753218+0000","last_active":"2024-09-10T18:58:23.411455+0000","last_peered":"2024-09-10T18:58:23.411455+0000","last_clean":"2024-09-10T18:58:23.411455+0000","last_became_active":"2024-09-10T18:55:40.398243+0000","last_became_peered":"2024-09-10T18:55:40.398243+0000","last_unstale":"2024-09-10T18:58:23.411455+0000","last_undegraded":"2024-09-10T18:58:23.411455+0000","last_fullsized":"2024-09-10T18:58:23.411455+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:04.753133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:03.571786+0000","last_clean_scrub_stamp":"2024-09-10T18:56:04.753133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0069575230000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":268,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.399089+0000","last_change":"2024-09-10T18:56:04.126848+0000","last_active":"2024-09-10T18:58:23.399089+0000","last_peered":"2024-09-10T18:58:23.399089+0000","last_clean":"2024-09-10T18:58:23.399089+0000","last_became_active":"2024-09-10T18:55:40.396932+0000","last_became_peered":"2024-09-10T18:55:40.396932+0000","last_unstale":"2024-09-10T18:58:23.399089+0000","last_undegraded":"2024-09-10T18:58:23.399089+0000","last_fullsized":"2024-09-10T18:58:23.399089+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:04.126516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:04.631558+0000","last_clean_scrub_stamp":"2024-09-10T18:56:04.126516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010034405999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"232'24","reported_seq":319,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.409714+0000","last_change":"2024-09-10T18:54:47.324126+0000","last_active":"2024-09-10T18:58:23.409714+0000","last_peered":"2024-09-10T18:58:23.409714+0000","last_clean":"2024-09-10T18:58:23.409714+0000","last_became_active":"2024-09-10T18:45:20.056885+0000","last_became_peered":"2024-09-10T18:45:20.056885+0000","last_unstale":"2024-09-10T18:58:23.409714+0000","last_undegraded":"2024-09-10T18:58:23.409714+0000","last_fullsized":"2024-09-10T18:58:23.409714+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'16","last_scrub_stamp":"2024-09-10T18:54:47.324044+0000","last_deep_scrub":"223'16","last_deep_scrub_stamp":"2024-09-10T18:54:47.324044+0000","last_clean_scrub_stamp":"2024-09-10T18:54:47.324044+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016282413999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":304,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.410470+0000","last_change":"2024-09-10T18:55:40.399504+0000","last_active":"2024-09-10T18:58:23.410470+0000","last_peered":"2024-09-10T18:58:23.410470+0000","last_clean":"2024-09-10T18:58:23.410470+0000","last_became_active":"2024-09-10T18:55:40.399198+0000","last_became_peered":"2024-09-10T18:55:40.399198+0000","last_unstale":"2024-09-10T18:58:23.410470+0000","last_undegraded":"2024-09-10T18:58:23.410470+0000","last_fullsized":"2024-09-10T18:58:23.410470+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:32.290320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:00.029389+0000","last_clean_scrub_stamp":"2024-09-10T18:53:32.290320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003302677,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":296,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.401281+0000","last_change":"2024-09-10T18:56:31.783026+0000","last_active":"2024-09-10T18:58:23.401281+0000","last_peered":"2024-09-10T18:58:23.401281+0000","last_clean":"2024-09-10T18:58:23.401281+0000","last_became_active":"2024-09-10T18:45:22.070065+0000","last_became_peered":"2024-09-10T18:45:22.070065+0000","last_unstale":"2024-09-10T18:58:23.401281+0000","last_undegraded":"2024-09-10T18:58:23.401281+0000","last_fullsized":"2024-09-10T18:58:23.401281+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:31.782918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:31.782918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043032130000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":298,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.404735+0000","last_change":"2024-09-10T18:56:06.792096+0000","last_active":"2024-09-10T18:58:23.404735+0000","last_peered":"2024-09-10T18:58:23.404735+0000","last_clean":"2024-09-10T18:58:23.404735+0000","last_became_active":"2024-09-10T18:55:40.395595+0000","last_became_peered":"2024-09-10T18:55:40.395595+0000","last_unstale":"2024-09-10T18:58:23.404735+0000","last_undegraded":"2024-09-10T18:58:23.404735+0000","last_fullsized":"2024-09-10T18:58:23.404735+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:06.791860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:06.791860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046852380000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"232'8","reported_seq":307,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.405046+0000","last_change":"2024-09-10T18:56:30.802774+0000","last_active":"2024-09-10T18:58:23.405046+0000","last_peered":"2024-09-10T18:58:23.405046+0000","last_clean":"2024-09-10T18:58:23.405046+0000","last_became_active":"2024-09-10T18:45:20.059205+0000","last_became_peered":"2024-09-10T18:45:20.059205+0000","last_unstale":"2024-09-10T18:58:23.405046+0000","last_undegraded":"2024-09-10T18:58:23.405046+0000","last_fullsized":"2024-09-10T18:58:23.405046+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'8","last_scrub_stamp":"2024-09-10T18:56:30.802679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:30.802679+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034579609999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":303,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.393597+0000","last_change":"2024-09-10T18:55:25.897258+0000","last_active":"2024-09-10T18:58:23.393597+0000","last_peered":"2024-09-10T18:58:23.393597+0000","last_clean":"2024-09-10T18:58:23.393597+0000","last_became_active":"2024-09-10T18:47:36.415866+0000","last_became_peered":"2024-09-10T18:47:36.415866+0000","last_unstale":"2024-09-10T18:58:23.393597+0000","last_undegraded":"2024-09-10T18:58:23.393597+0000","last_fullsized":"2024-09-10T18:58:23.393597+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:25.897205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:55:25.897205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001389787,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":323,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.405983+0000","last_change":"2024-09-10T18:57:02.454548+0000","last_active":"2024-09-10T18:58:23.405983+0000","last_peered":"2024-09-10T18:58:23.405983+0000","last_clean":"2024-09-10T18:58:23.405983+0000","last_became_active":"2024-09-10T18:57:02.453974+0000","last_became_peered":"2024-09-10T18:57:02.453974+0000","last_unstale":"2024-09-10T18:58:23.405983+0000","last_undegraded":"2024-09-10T18:58:23.405983+0000","last_fullsized":"2024-09-10T18:58:23.405983+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:23.195060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_clean_scrub_stamp":"2024-09-10T18:56:23.195060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047613259999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":84,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.409352+0000","last_change":"2024-09-10T18:57:02.447992+0000","last_active":"2024-09-10T18:58:23.409352+0000","last_peered":"2024-09-10T18:58:23.409352+0000","last_clean":"2024-09-10T18:58:23.409352+0000","last_became_active":"2024-09-10T18:57:02.447566+0000","last_became_peered":"2024-09-10T18:57:02.447566+0000","last_unstale":"2024-09-10T18:58:23.409352+0000","last_undegraded":"2024-09-10T18:58:23.409352+0000","last_fullsized":"2024-09-10T18:58:23.409352+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:28.768091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:55.145638+0000","last_clean_scrub_stamp":"2024-09-10T18:55:28.768091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020285519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.14","version":"70'81","reported_seq":195,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.410482+0000","last_change":"2024-09-10T18:57:02.442850+0000","last_active":"2024-09-10T18:58:23.410482+0000","last_peered":"2024-09-10T18:58:23.410482+0000","last_clean":"2024-09-10T18:58:23.410482+0000","last_became_active":"2024-09-10T18:57:02.442457+0000","last_became_peered":"2024-09-10T18:57:02.442457+0000","last_unstale":"2024-09-10T18:58:23.410482+0000","last_undegraded":"2024-09-10T18:58:23.410482+0000","last_fullsized":"2024-09-10T18:58:23.410482+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:55:34.791223+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:57.184383+0000","last_clean_scrub_stamp":"2024-09-10T18:55:34.791223+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016709736999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17890,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":7,"num_write":3,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":35780,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.11","version":"232'24","reported_seq":343,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.404476+0000","last_change":"2024-09-10T18:57:02.490512+0000","last_active":"2024-09-10T18:58:23.404476+0000","last_peered":"2024-09-10T18:58:23.404476+0000","last_clean":"2024-09-10T18:58:23.404476+0000","last_became_active":"2024-09-10T18:57:02.461909+0000","last_became_peered":"2024-09-10T18:57:02.461909+0000","last_unstale":"2024-09-10T18:58:23.404476+0000","last_undegraded":"2024-09-10T18:58:23.404476+0000","last_fullsized":"2024-09-10T18:58:23.404476+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"79'16","last_scrub_stamp":"2024-09-10T18:54:39.420695+0000","last_deep_scrub":"79'16","last_deep_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_clean_scrub_stamp":"2024-09-10T18:54:39.420695+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.052085645,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"56'1","reported_seq":328,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.407081+0000","last_change":"2024-09-10T18:56:25.211179+0000","last_active":"2024-09-10T18:58:23.407081+0000","last_peered":"2024-09-10T18:58:23.407081+0000","last_clean":"2024-09-10T18:58:23.407081+0000","last_became_active":"2024-09-10T18:47:36.421985+0000","last_became_peered":"2024-09-10T18:47:36.421985+0000","last_unstale":"2024-09-10T18:58:23.407081+0000","last_undegraded":"2024-09-10T18:58:23.407081+0000","last_fullsized":"2024-09-10T18:58:23.407081+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:56:25.211095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:25.211095+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025468880000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":38,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.409466+0000","last_change":"2024-09-10T18:57:02.435423+0000","last_active":"2024-09-10T18:58:23.409466+0000","last_peered":"2024-09-10T18:58:23.409466+0000","last_clean":"2024-09-10T18:58:23.409466+0000","last_became_active":"2024-09-10T18:57:02.435124+0000","last_became_peered":"2024-09-10T18:57:02.435124+0000","last_unstale":"2024-09-10T18:58:23.409466+0000","last_undegraded":"2024-09-10T18:58:23.409466+0000","last_fullsized":"2024-09-10T18:58:23.409466+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:12.852769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:12.852769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":308,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.411250+0000","last_change":"2024-09-10T18:56:09.903930+0000","last_active":"2024-09-10T18:58:23.411250+0000","last_peered":"2024-09-10T18:58:23.411250+0000","last_clean":"2024-09-10T18:58:23.411250+0000","last_became_active":"2024-09-10T18:55:40.397523+0000","last_became_peered":"2024-09-10T18:55:40.397523+0000","last_unstale":"2024-09-10T18:58:23.411250+0000","last_undegraded":"2024-09-10T18:58:23.411250+0000","last_fullsized":"2024-09-10T18:58:23.411250+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:09.903846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:09.903846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011721984,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"78'2","reported_seq":74,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.393377+0000","last_change":"2024-09-10T18:57:03.514045+0000","last_active":"2024-09-10T18:58:23.393377+0000","last_peered":"2024-09-10T18:58:23.393377+0000","last_clean":"2024-09-10T18:58:23.393377+0000","last_became_active":"2024-09-10T18:57:02.447198+0000","last_became_peered":"2024-09-10T18:57:02.447198+0000","last_unstale":"2024-09-10T18:58:23.393377+0000","last_undegraded":"2024-09-10T18:58:23.393377+0000","last_fullsized":"2024-09-10T18:58:23.393377+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'2","last_scrub_stamp":"2024-09-10T18:54:27.784508+0000","last_deep_scrub":"78'2","last_deep_scrub_stamp":"2024-09-10T18:46:37.581130+0000","last_clean_scrub_stamp":"2024-09-10T18:54:27.784508+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":307,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.392909+0000","last_change":"2024-09-10T18:57:02.444124+0000","last_active":"2024-09-10T18:58:23.392909+0000","last_peered":"2024-09-10T18:58:23.392909+0000","last_clean":"2024-09-10T18:58:23.392909+0000","last_became_active":"2024-09-10T18:57:02.443353+0000","last_became_peered":"2024-09-10T18:57:02.443353+0000","last_unstale":"2024-09-10T18:58:23.392909+0000","last_undegraded":"2024-09-10T18:58:23.392909+0000","last_fullsized":"2024-09-10T18:58:23.392909+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:16.743764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:16.743764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0031221030000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"232'20","reported_seq":322,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.399428+0000","last_change":"2024-09-10T18:57:02.486956+0000","last_active":"2024-09-10T18:58:23.399428+0000","last_peered":"2024-09-10T18:58:23.399428+0000","last_clean":"2024-09-10T18:58:23.399428+0000","last_became_active":"2024-09-10T18:57:02.452473+0000","last_became_peered":"2024-09-10T18:57:02.452473+0000","last_unstale":"2024-09-10T18:58:23.399428+0000","last_undegraded":"2024-09-10T18:58:23.399428+0000","last_fullsized":"2024-09-10T18:58:23.399428+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'20","last_scrub_stamp":"2024-09-10T18:56:27.158686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:27.158686+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0090371180000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":23,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"273'10","reported_seq":487,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-10T18:58:24.400602+0000","last_change":"2024-09-10T18:57:02.474747+0000","last_active":"2024-09-10T18:58:24.400602+0000","last_peered":"2024-09-10T18:58:24.400602+0000","last_clean":"2024-09-10T18:58:24.400602+0000","last_became_active":"2024-09-10T18:57:02.441763+0000","last_became_peered":"2024-09-10T18:57:02.441763+0000","last_unstale":"2024-09-10T18:58:24.400602+0000","last_undegraded":"2024-09-10T18:58:24.400602+0000","last_fullsized":"2024-09-10T18:58:24.400602+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"190'9","last_scrub_stamp":"2024-09-10T18:56:19.259869+0000","last_deep_scrub":"190'9","last_deep_scrub_stamp":"2024-09-10T18:53:07.746309+0000","last_clean_scrub_stamp":"2024-09-10T18:56:19.259869+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020087130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":284,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.407705+0000","last_change":"2024-09-10T18:54:24.631566+0000","last_active":"2024-09-10T18:58:23.407705+0000","last_peered":"2024-09-10T18:58:23.407705+0000","last_clean":"2024-09-10T18:58:23.407705+0000","last_became_active":"2024-09-10T18:45:24.082318+0000","last_became_peered":"2024-09-10T18:45:24.082318+0000","last_unstale":"2024-09-10T18:58:23.407705+0000","last_undegraded":"2024-09-10T18:58:23.407705+0000","last_fullsized":"2024-09-10T18:58:23.407705+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:24.631469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:03.475299+0000","last_clean_scrub_stamp":"2024-09-10T18:54:24.631469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014197107,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":304,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.403151+0000","last_change":"2024-09-10T18:56:00.878293+0000","last_active":"2024-09-10T18:58:23.403151+0000","last_peered":"2024-09-10T18:58:23.403151+0000","last_clean":"2024-09-10T18:58:23.403151+0000","last_became_active":"2024-09-10T18:55:40.392524+0000","last_became_peered":"2024-09-10T18:55:40.392524+0000","last_unstale":"2024-09-10T18:58:23.403151+0000","last_undegraded":"2024-09-10T18:58:23.403151+0000","last_fullsized":"2024-09-10T18:58:23.403151+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:00.877849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:00.877849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0096660570000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"232'19","reported_seq":329,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.406986+0000","last_change":"2024-09-10T18:55:26.062315+0000","last_active":"2024-09-10T18:58:23.406986+0000","last_peered":"2024-09-10T18:58:23.406986+0000","last_clean":"2024-09-10T18:58:23.406986+0000","last_became_active":"2024-09-10T18:45:20.058717+0000","last_became_peered":"2024-09-10T18:45:20.058717+0000","last_unstale":"2024-09-10T18:58:23.406986+0000","last_undegraded":"2024-09-10T18:58:23.406986+0000","last_fullsized":"2024-09-10T18:58:23.406986+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T18:55:26.062256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:55:26.062256+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001573893,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":266,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.401687+0000","last_change":"2024-09-10T18:57:02.454464+0000","last_active":"2024-09-10T18:58:23.401687+0000","last_peered":"2024-09-10T18:58:23.401687+0000","last_clean":"2024-09-10T18:58:23.401687+0000","last_became_active":"2024-09-10T18:57:02.453464+0000","last_became_peered":"2024-09-10T18:57:02.453464+0000","last_unstale":"2024-09-10T18:58:23.401687+0000","last_undegraded":"2024-09-10T18:58:23.401687+0000","last_fullsized":"2024-09-10T18:58:23.401687+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:33.248024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:33.248024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060527929999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":279,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.411120+0000","last_change":"2024-09-10T18:54:21.072224+0000","last_active":"2024-09-10T18:58:23.411120+0000","last_peered":"2024-09-10T18:58:23.411120+0000","last_clean":"2024-09-10T18:58:23.411120+0000","last_became_active":"2024-09-10T18:45:24.267251+0000","last_became_peered":"2024-09-10T18:45:24.267251+0000","last_unstale":"2024-09-10T18:58:23.411120+0000","last_undegraded":"2024-09-10T18:58:23.411120+0000","last_fullsized":"2024-09-10T18:58:23.411120+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:21.071910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:54:21.071910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007087388,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"56'1","reported_seq":323,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.407292+0000","last_change":"2024-09-10T18:57:02.449727+0000","last_active":"2024-09-10T18:58:23.407292+0000","last_peered":"2024-09-10T18:58:23.407292+0000","last_clean":"2024-09-10T18:58:23.407292+0000","last_became_active":"2024-09-10T18:57:02.435758+0000","last_became_peered":"2024-09-10T18:57:02.435758+0000","last_unstale":"2024-09-10T18:58:23.407292+0000","last_undegraded":"2024-09-10T18:58:23.407292+0000","last_fullsized":"2024-09-10T18:58:23.407292+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:56:05.158970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:05.158970+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0055166130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"258'34","reported_seq":358,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.403730+0000","last_change":"2024-09-10T18:56:23.839043+0000","last_active":"2024-09-10T18:58:23.403730+0000","last_peered":"2024-09-10T18:58:23.403730+0000","last_clean":"2024-09-10T18:58:23.403730+0000","last_became_active":"2024-09-10T18:55:40.401788+0000","last_became_peered":"2024-09-10T18:55:40.401788+0000","last_unstale":"2024-09-10T18:58:23.403730+0000","last_undegraded":"2024-09-10T18:58:23.403730+0000","last_fullsized":"2024-09-10T18:58:23.403730+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'34","last_scrub_stamp":"2024-09-10T18:56:23.838951+0000","last_deep_scrub":"258'34","last_deep_scrub_stamp":"2024-09-10T18:56:23.838951+0000","last_clean_scrub_stamp":"2024-09-10T18:56:23.838951+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.035929888,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":300,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.410108+0000","last_change":"2024-09-10T18:56:13.611288+0000","last_active":"2024-09-10T18:58:23.410108+0000","last_peered":"2024-09-10T18:58:23.410108+0000","last_clean":"2024-09-10T18:58:23.410108+0000","last_became_active":"2024-09-10T18:55:40.397870+0000","last_became_peered":"2024-09-10T18:55:40.397870+0000","last_unstale":"2024-09-10T18:58:23.410108+0000","last_undegraded":"2024-09-10T18:58:23.410108+0000","last_fullsized":"2024-09-10T18:58:23.410108+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:13.610976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:13.610976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010999553,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":291,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.409386+0000","last_change":"2024-09-10T18:55:39.815156+0000","last_active":"2024-09-10T18:58:23.409386+0000","last_peered":"2024-09-10T18:58:23.409386+0000","last_clean":"2024-09-10T18:58:23.409386+0000","last_became_active":"2024-09-10T18:52:54.624256+0000","last_became_peered":"2024-09-10T18:52:54.624256+0000","last_unstale":"2024-09-10T18:58:23.409386+0000","last_undegraded":"2024-09-10T18:58:23.409386+0000","last_fullsized":"2024-09-10T18:58:23.409386+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:39.815087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:39.815087+0000","last_clean_scrub_stamp":"2024-09-10T18:55:39.815087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018555970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":307,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.411713+0000","last_change":"2024-09-10T18:57:02.463213+0000","last_active":"2024-09-10T18:58:23.411713+0000","last_peered":"2024-09-10T18:58:23.411713+0000","last_clean":"2024-09-10T18:58:23.411713+0000","last_became_active":"2024-09-10T18:57:02.460855+0000","last_became_peered":"2024-09-10T18:57:02.460855+0000","last_unstale":"2024-09-10T18:58:23.411713+0000","last_undegraded":"2024-09-10T18:58:23.411713+0000","last_fullsized":"2024-09-10T18:58:23.411713+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:46.433200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:54:46.433200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013571259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"232'18","reported_seq":317,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.405874+0000","last_change":"2024-09-10T18:54:21.298808+0000","last_active":"2024-09-10T18:58:23.405874+0000","last_peered":"2024-09-10T18:58:23.405874+0000","last_clean":"2024-09-10T18:58:23.405874+0000","last_became_active":"2024-09-10T18:45:20.058391+0000","last_became_peered":"2024-09-10T18:45:20.058391+0000","last_unstale":"2024-09-10T18:58:23.405874+0000","last_undegraded":"2024-09-10T18:58:23.405874+0000","last_fullsized":"2024-09-10T18:58:23.405874+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'12","last_scrub_stamp":"2024-09-10T18:54:21.298731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:54:21.298731+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060418820000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":311,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-10T18:58:23.410399+0000","last_change":"2024-09-10T18:57:02.450561+0000","last_active":"2024-09-10T18:58:23.410399+0000","last_peered":"2024-09-10T18:58:23.410399+0000","last_clean":"2024-09-10T18:58:23.410399+0000","last_became_active":"2024-09-10T18:57:02.449135+0000","last_became_peered":"2024-09-10T18:57:02.449135+0000","last_unstale":"2024-09-10T18:58:23.410399+0000","last_undegraded":"2024-09-10T18:58:23.410399+0000","last_fullsized":"2024-09-10T18:58:23.410399+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:53.316517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:54:53.316517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0090189510000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":4922671128,"num_objects":75115,"num_object_clones":0,"num_object_copies":225345,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":559,"num_objects_unfound":0,"num_objects_dirty":75115,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":198385,"num_write_kb":8751937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75191,"num_bytes_recovered":4927586328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18545852416,"data_stored":18545836128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57781,"ondisk_log_size":57781,"up":48,"acting":48,"num_store_stats":7},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84,"ondisk_log_size":84,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1118,"num_read_kb":875,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":2712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":499712,"data_stored":13268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":758,"ondisk_log_size":758,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":772,"num_read_kb":767,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":715597,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":47,"num_write":23,"num_write_kb":345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":250458,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1701,"ondisk_log_size":1701,"up":63,"acting":63,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":0,"seq":223338299554,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":43,"seq":184683593895,"num_pgs":86,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2438188,"kb_used_data":1848288,"kb_used_omap":1,"kb_used_meta":589886,"kb_avail":91306964,"statfs":{"total":95995035648,"available":93498331136,"internally_reserved":0,"allocated":1892646912,"data_stored":1892277317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":604043608},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:57:44 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":0.91800000000000004,"15min":0.90000000000000002},"min":{"1min":0.28899999999999998,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":6.085,"5min":6.085,"15min":6.085},"last":0.32500000000000001},{"interface":"front","average":{"1min":1.798,"5min":1.111,"15min":0.997},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":18.632999999999999,"5min":18.632999999999999,"15min":18.632999999999999},"last":0.34999999999999998}]},{"osd":1,"last update":"Tue Sep 10 18:57:57 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":1.266,"15min":0.96099999999999997},"min":{"1min":0.29699999999999999,"5min":0.247,"15min":0.219},"max":{"1min":3.8300000000000001,"5min":18.664000000000001,"15min":18.664000000000001},"last":0.38300000000000001},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.155,"15min":0.86299999999999999},"min":{"1min":0.30399999999999999,"5min":0.23300000000000001,"15min":0.214},"max":{"1min":3.5190000000000001,"5min":9.1750000000000007,"15min":9.1750000000000007},"last":0.35999999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:57:57 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.80600000000000005,"15min":0.746},"min":{"1min":0.28999999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":2.173,"5min":4.1029999999999998,"15min":4.4740000000000002},"last":0.37},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.84899999999999998,"15min":0.68799999999999994},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.27200000000000002},"max":{"1min":2.3860000000000001,"5min":4.8150000000000004,"15min":4.8150000000000004},"last":0.29599999999999999}]},{"osd":3,"last update":"Tue Sep 10 18:57:57 2024","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.371,"15min":0.94699999999999995},"min":{"1min":0.35899999999999999,"5min":0.28399999999999997,"15min":0.25700000000000001},"max":{"1min":4.609,"5min":18.817,"15min":18.817},"last":76.977999999999994},{"interface":"front","average":{"1min":1.401,"5min":1.389,"15min":1.0620000000000001},"min":{"1min":0.375,"5min":0.253,"15min":0.253},"max":{"1min":5.673,"5min":18.678999999999998,"15min":18.678999999999998},"last":76.879999999999995}]},{"osd":4,"last update":"Tue Sep 10 18:57:57 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":2.7410000000000001,"15min":1.2729999999999999},"min":{"1min":0.23899999999999999,"5min":0.23499999999999999,"15min":0.20999999999999999},"max":{"1min":2.2599999999999998,"5min":165.03299999999999,"15min":165.03299999999999},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":2.5750000000000002,"15min":1.1950000000000001},"min":{"1min":0.29799999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.919,"5min":163.959,"15min":163.959},"last":0.27000000000000002}]},{"osd":5,"last update":"Tue Sep 10 18:57:57 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":1.4890000000000001,"15min":0.92100000000000004},"min":{"1min":0.313,"5min":0.27100000000000002,"15min":0.24099999999999999},"max":{"1min":1.1200000000000001,"5min":43.811,"15min":43.811},"last":0.313},{"interface":"front","average":{"1min":0.74199999999999999,"5min":1.645,"15min":0.94499999999999995},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.23999999999999999},"max":{"1min":3.4359999999999999,"5min":42.314,"15min":42.314},"last":0.432}]},{"osd":7,"last update":"Tue Sep 10 18:57:28 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":1.1519999999999999,"15min":0.78800000000000003},"min":{"1min":0.33600000000000002,"5min":0.29799999999999999,"15min":0.26200000000000001},"max":{"1min":2.98,"5min":5.3680000000000003,"15min":5.3680000000000003},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.76500000000000001,"5min":1.272,"15min":0.82999999999999996},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.8480000000000001,"5min":7.0350000000000001,"15min":7.0350000000000001},"last":0.41699999999999998}]}]},{"osd":5,"up_from":37,"seq":158913790123,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2738976,"kb_used_data":2149080,"kb_used_omap":1,"kb_used_meta":589886,"kb_avail":91006176,"statfs":{"total":95995035648,"available":93190324224,"internally_reserved":0,"allocated":2200657920,"data_stored":2200294806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":604043608},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:57:44 2024","interfaces":[{"interface":"back","average":{"1min":10.287000000000001,"5min":3.2959999999999998,"15min":2.1309999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":82.465000000000003,"5min":82.465000000000003,"15min":82.465000000000003},"last":0.73299999999999998},{"interface":"front","average":{"1min":10.266,"5min":3.4700000000000002,"15min":2.3370000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":82.519000000000005,"5min":82.519000000000005,"15min":82.519000000000005},"last":0.75}]},{"osd":1,"last update":"Tue Sep 10 18:57:40 2024","interfaces":[{"interface":"back","average":{"1min":10.069000000000001,"5min":3.1230000000000002,"15min":1.462},"min":{"1min":0.25700000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":67.582999999999998,"5min":67.582999999999998,"15min":67.582999999999998},"last":0.251},{"interface":"front","average":{"1min":9.4079999999999995,"5min":3.4529999999999998,"15min":1.6020000000000001},"min":{"1min":0.38700000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":67.603999999999999,"5min":67.603999999999999,"15min":67.603999999999999},"last":0.309}]},{"osd":2,"last update":"Tue Sep 10 18:57:40 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.94899999999999995,"15min":0.71399999999999997},"min":{"1min":0.28999999999999998,"5min":0.27900000000000003,"15min":0.249},"max":{"1min":3.2410000000000001,"5min":4.6319999999999997,"15min":7.8029999999999999},"last":1.579},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.90500000000000003,"15min":0.73099999999999998},"min":{"1min":0.28100000000000003,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.512,"5min":4.8680000000000003,"15min":6.7210000000000001},"last":1.911}]},{"osd":3,"last update":"Tue Sep 10 18:57:40 2024","interfaces":[{"interface":"back","average":{"1min":20.542000000000002,"5min":5.4109999999999996,"15min":2.633},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26000000000000001},"max":{"1min":123.34399999999999,"5min":123.34399999999999,"15min":123.34399999999999},"last":1.9299999999999999},{"interface":"front","average":{"1min":12.853,"5min":4.0629999999999997,"15min":2.2450000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.245},"max":{"1min":82.415000000000006,"5min":82.415000000000006,"15min":82.415000000000006},"last":0.75800000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:57:40 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":1.5189999999999999,"15min":0.82499999999999996},"min":{"1min":0.26300000000000001,"5min":0.23999999999999999,"15min":0.20899999999999999},"max":{"1min":1.8180000000000001,"5min":9.0310000000000006,"15min":9.0310000000000006},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":1.3340000000000001,"15min":0.76100000000000001},"min":{"1min":0.29199999999999998,"5min":0.247,"15min":0.23400000000000001},"max":{"1min":2.012,"5min":11.182,"15min":11.182},"last":0.27100000000000002}]},{"osd":6,"last update":"Tue Sep 10 18:58:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":1.3899999999999999,"15min":0.91400000000000003},"min":{"1min":0.29899999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.534,"5min":10.385,"15min":11.17},"last":0.376},{"interface":"front","average":{"1min":0.51000000000000001,"5min":1.29,"15min":0.91000000000000003},"min":{"1min":0.29699999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.5029999999999999,"5min":7.3719999999999999,"15min":11.18},"last":0.35699999999999998}]},{"osd":7,"last update":"Tue Sep 10 18:58:25 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":1.22,"15min":0.91000000000000003},"min":{"1min":0.313,"5min":0.29499999999999998,"15min":0.27600000000000002},"max":{"1min":1.117,"5min":5.7249999999999996,"15min":10.782999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.98799999999999999,"15min":0.79400000000000004},"min":{"1min":0.33300000000000002,"5min":0.24099999999999999,"15min":0.23599999999999999},"max":{"1min":1.105,"5min":5.9500000000000002,"15min":11.846},"last":0.38900000000000001}]}]},{"osd":4,"up_from":30,"seq":128849019057,"num_pgs":83,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2127976,"kb_used_data":1542240,"kb_used_omap":1,"kb_used_meta":585726,"kb_avail":91617176,"statfs":{"total":95995035648,"available":93815988224,"internally_reserved":0,"allocated":1579253760,"data_stored":1578883755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":599783768},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:57:41 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":1.298,"15min":1.3899999999999999},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.8859999999999999,"5min":11.872999999999999,"15min":11.872999999999999},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":1.333,"15min":1.421},"min":{"1min":0.38300000000000001,"5min":0.308,"15min":0.308},"max":{"1min":1.9159999999999999,"5min":11.852,"15min":11.852},"last":0.308}]},{"osd":1,"last update":"Tue Sep 10 18:58:24 2024","interfaces":[{"interface":"back","average":{"1min":7.9329999999999998,"5min":3.0049999999999999,"15min":1.5349999999999999},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":125.15600000000001,"5min":125.15600000000001,"15min":125.15600000000001},"last":0.23799999999999999},{"interface":"front","average":{"1min":8.0329999999999995,"5min":2.9900000000000002,"15min":1.474},"min":{"1min":0.35399999999999998,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":125.14700000000001,"5min":125.14700000000001,"15min":125.14700000000001},"last":0.39200000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:58:24 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.746,"15min":0.66000000000000003},"min":{"1min":0.28799999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.2669999999999999,"5min":5.6189999999999998,"15min":5.6189999999999998},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.79200000000000004,"15min":0.69399999999999995},"min":{"1min":0.32100000000000001,"5min":0.28000000000000003,"15min":0.27100000000000002},"max":{"1min":1.6639999999999999,"5min":6.3540000000000001,"15min":6.3540000000000001},"last":0.32100000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:58:24 2024","interfaces":[{"interface":"back","average":{"1min":9.6590000000000007,"5min":3.5270000000000001,"15min":1.837},"min":{"1min":0.311,"5min":0.249,"15min":0.23799999999999999},"max":{"1min":111.514,"5min":111.514,"15min":111.514},"last":0.376},{"interface":"front","average":{"1min":7.1369999999999996,"5min":2.6899999999999999,"15min":1.506},"min":{"1min":0.27200000000000002,"5min":0.26900000000000002,"15min":0.25700000000000001},"max":{"1min":110.914,"5min":110.914,"15min":110.914},"last":0.27300000000000002}]},{"osd":5,"last update":"Tue Sep 10 18:57:41 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":1.27,"15min":0.78100000000000003},"min":{"1min":0.23699999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.95099999999999996,"5min":10.912000000000001,"15min":10.912000000000001},"last":9.7309999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":1.1910000000000001,"15min":0.72899999999999998},"min":{"1min":0.24399999999999999,"5min":0.219,"15min":0.219},"max":{"1min":1.6279999999999999,"5min":5.6429999999999998,"15min":5.6429999999999998},"last":10.147}]},{"osd":6,"last update":"Tue Sep 10 18:58:06 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":1.0800000000000001,"15min":0.72099999999999997},"min":{"1min":0.27700000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.6200000000000001,"5min":8.4030000000000005,"15min":8.4030000000000005},"last":0.221},{"interface":"front","average":{"1min":0.51700000000000002,"5min":1.111,"15min":0.79600000000000004},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.25800000000000001},"max":{"1min":1.341,"5min":7.7699999999999996,"15min":7.7699999999999996},"last":0.28999999999999998}]},{"osd":7,"last update":"Tue Sep 10 18:57:36 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":1.3049999999999999,"15min":0.80800000000000005},"min":{"1min":0.309,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.8879999999999999,"5min":6.4630000000000001,"15min":6.4630000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.71999999999999997,"5min":1.2390000000000001,"15min":0.76800000000000002},"min":{"1min":0.28899999999999998,"5min":0.251,"15min":0.251},"max":{"1min":2.4489999999999998,"5min":6.3949999999999996,"15min":6.3949999999999996},"last":0.28000000000000003}]}]},{"osd":3,"up_from":25,"seq":107374182582,"num_pgs":89,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4212780,"kb_used_data":3323300,"kb_used_omap":1,"kb_used_meta":889406,"kb_avail":89532372,"statfs":{"total":95995035648,"available":91681148928,"internally_reserved":0,"allocated":3403059200,"data_stored":3402684539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":910752088},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:57:40 2024","interfaces":[{"interface":"back","average":{"1min":2.1299999999999999,"5min":1.4419999999999999,"15min":1.3280000000000001},"min":{"1min":0.25800000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":5.8550000000000004,"5min":5.8550000000000004,"15min":5.8550000000000004},"last":0.66300000000000003},{"interface":"front","average":{"1min":2.4119999999999999,"5min":1.6950000000000001,"15min":1.575},"min":{"1min":0.60499999999999998,"5min":0.252,"15min":0.252},"max":{"1min":9.6940000000000008,"5min":9.6940000000000008,"15min":9.6940000000000008},"last":1.139}]},{"osd":1,"last update":"Tue Sep 10 18:57:57 2024","interfaces":[{"interface":"back","average":{"1min":3.2749999999999999,"5min":1.631,"15min":1.3480000000000001},"min":{"1min":0.41799999999999998,"5min":0.16200000000000001,"15min":0.16200000000000001},"max":{"1min":10.417999999999999,"5min":10.417999999999999,"15min":58.215000000000003},"last":1.4490000000000001},{"interface":"front","average":{"1min":2.1600000000000001,"5min":1.218,"15min":1.2490000000000001},"min":{"1min":0.41199999999999998,"5min":0.188,"15min":0.188},"max":{"1min":4.6820000000000004,"5min":7.2309999999999999,"15min":57.896999999999998},"last":1.202}]},{"osd":2,"last update":"Tue Sep 10 18:57:57 2024","interfaces":[{"interface":"back","average":{"1min":1.569,"5min":0.85799999999999998,"15min":0.78700000000000003},"min":{"1min":0.26400000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":4.4649999999999999,"5min":5.1150000000000002,"15min":5.8220000000000001},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.53,"5min":0.90400000000000003,"15min":0.74299999999999999},"min":{"1min":0.26200000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":4.923,"5min":4.923,"15min":7.0979999999999999},"last":0.79200000000000004}]},{"osd":4,"last update":"Tue Sep 10 18:57:23 2024","interfaces":[{"interface":"back","average":{"1min":2.121,"5min":1.538,"15min":1.0640000000000001},"min":{"1min":0.36499999999999999,"5min":0.247,"15min":0.247},"max":{"1min":4.8460000000000001,"5min":29.013000000000002,"15min":29.013000000000002},"last":0.89700000000000002},{"interface":"front","average":{"1min":2.3780000000000001,"5min":1.554,"15min":1.0760000000000001},"min":{"1min":0.32400000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":4.891,"5min":28.826000000000001,"15min":28.826000000000001},"last":0.94099999999999995}]},{"osd":5,"last update":"Tue Sep 10 18:57:46 2024","interfaces":[{"interface":"back","average":{"1min":2.2730000000000001,"5min":1.55,"15min":1.079},"min":{"1min":0.59299999999999997,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":5.8220000000000001,"5min":6.1989999999999998,"15min":6.1989999999999998},"last":1.369},{"interface":"front","average":{"1min":1.9299999999999999,"5min":1.3360000000000001,"15min":1.048},"min":{"1min":0.56399999999999995,"5min":0.255,"15min":0.255},"max":{"1min":4.4950000000000001,"5min":6.1870000000000003,"15min":9.2509999999999994},"last":1.294}]},{"osd":6,"last update":"Tue Sep 10 18:58:07 2024","interfaces":[{"interface":"back","average":{"1min":1.923,"5min":1.335,"15min":1.0880000000000001},"min":{"1min":0.53100000000000003,"5min":0.309,"15min":0.309},"max":{"1min":4.7160000000000002,"5min":5.2370000000000001,"15min":9.3239999999999998},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.6779999999999999,"5min":1.393,"15min":1.0920000000000001},"min":{"1min":0.308,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":4.8209999999999997,"5min":5.2670000000000003,"15min":7.1559999999999997},"last":0.86299999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:57:33 2024","interfaces":[{"interface":"back","average":{"1min":2.5419999999999998,"5min":1.6559999999999999,"15min":1.0960000000000001},"min":{"1min":0.92200000000000004,"5min":0.35799999999999998,"15min":0.34899999999999998},"max":{"1min":5.7549999999999999,"5min":6.0839999999999996,"15min":6.0839999999999996},"last":1.3919999999999999},{"interface":"front","average":{"1min":2.266,"5min":1.5640000000000001,"15min":1.1319999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.33100000000000002},"max":{"1min":6.6029999999999998,"5min":6.6029999999999998,"15min":9.2949999999999999},"last":1.3380000000000001}]}]},{"osd":2,"up_from":0,"seq":77309411514,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834575038,"num_pgs":100,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4531336,"kb_used_data":3640064,"kb_used_omap":1,"kb_used_meta":890814,"kb_avail":89213816,"statfs":{"total":95995035648,"available":91354947584,"internally_reserved":0,"allocated":3727425536,"data_stored":3727023580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":912193880},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:57:42 2024","interfaces":[{"interface":"back","average":{"1min":2.0299999999999998,"5min":1.403,"15min":1.2989999999999999},"min":{"1min":0.20300000000000001,"5min":0.193,"15min":0.193},"max":{"1min":14.192,"5min":14.192,"15min":14.192},"last":1.5229999999999999},{"interface":"front","average":{"1min":2.137,"5min":1.323,"15min":1.1870000000000001},"min":{"1min":0.26500000000000001,"5min":0.217,"15min":0.217},"max":{"1min":11.502000000000001,"5min":11.502000000000001,"15min":11.502000000000001},"last":3.1640000000000001}]},{"osd":2,"last update":"Tue Sep 10 18:57:31 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":0.77100000000000002,"15min":0.63800000000000001},"min":{"1min":0.24199999999999999,"5min":0.186,"15min":0.186},"max":{"1min":4.1719999999999997,"5min":4.1719999999999997,"15min":4.1719999999999997},"last":0.46600000000000003},{"interface":"front","average":{"1min":1.228,"5min":0.76700000000000002,"15min":0.67200000000000004},"min":{"1min":0.26000000000000001,"5min":0.215,"15min":0.215},"max":{"1min":5.6929999999999996,"5min":5.6929999999999996,"15min":7.093},"last":0.39900000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:58:01 2024","interfaces":[{"interface":"back","average":{"1min":5.21,"5min":2.5379999999999998,"15min":1.5349999999999999},"min":{"1min":0.26100000000000001,"5min":0.185,"15min":0.185},"max":{"1min":48.015000000000001,"5min":48.015000000000001,"15min":48.015000000000001},"last":0.432},{"interface":"front","average":{"1min":3.3759999999999999,"5min":2.121,"15min":1.4359999999999999},"min":{"1min":0.218,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":19.651,"5min":47.828000000000003,"15min":47.828000000000003},"last":3.1789999999999998}]},{"osd":4,"last update":"Tue Sep 10 18:58:23 2024","interfaces":[{"interface":"back","average":{"1min":1.944,"5min":1.224,"15min":1.0820000000000001},"min":{"1min":0.30599999999999999,"5min":0.29799999999999999,"15min":0.26200000000000001},"max":{"1min":9.125,"5min":9.125,"15min":15.965999999999999},"last":1.407},{"interface":"front","average":{"1min":1.9330000000000001,"5min":1.2769999999999999,"15min":0.95899999999999996},"min":{"1min":0.252,"5min":0.252,"15min":0.246},"max":{"1min":7.0940000000000003,"5min":7.0940000000000003,"15min":7.0940000000000003},"last":0.5}]},{"osd":5,"last update":"Tue Sep 10 18:57:42 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.452,"15min":1.0600000000000001},"min":{"1min":0.30499999999999999,"5min":0.28899999999999998,"15min":0.28499999999999998},"max":{"1min":5.4450000000000003,"5min":10.334,"15min":10.334},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.2310000000000001,"15min":1.0940000000000001},"min":{"1min":0.378,"5min":0.29699999999999999,"15min":0.28799999999999998},"max":{"1min":3.4420000000000002,"5min":7.9969999999999999,"15min":16.039999999999999},"last":1.4970000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:58:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.8520000000000001,"15min":1.3560000000000001},"min":{"1min":0.33100000000000002,"5min":0.315,"15min":0.308},"max":{"1min":2.875,"5min":33.290999999999997,"15min":38.039999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.8109999999999999,"15min":1.4750000000000001},"min":{"1min":0.39300000000000002,"5min":0.33400000000000002,"15min":0.30599999999999999},"max":{"1min":3.3940000000000001,"5min":33.310000000000002,"15min":38.100999999999999},"last":1.4390000000000001}]},{"osd":7,"last update":"Tue Sep 10 18:57:31 2024","interfaces":[{"interface":"back","average":{"1min":1.6299999999999999,"5min":1.3759999999999999,"15min":1.0329999999999999},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.315},"max":{"1min":5.6559999999999997,"5min":5.6559999999999997,"15min":5.6559999999999997},"last":0.48399999999999999},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1339999999999999,"15min":0.97499999999999998},"min":{"1min":0.38900000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":3.226,"5min":4.9029999999999996,"15min":4.9029999999999996},"last":1.4670000000000001}]}]},{"osd":0,"up_from":241,"seq":1035087118371,"num_pgs":89,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2792356,"kb_used_data":2450204,"kb_used_omap":4,"kb_used_meta":342075,"kb_avail":90952796,"statfs":{"total":95995035648,"available":93135663104,"internally_reserved":0,"allocated":2509008896,"data_stored":2508641785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":350285318},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 18:57:40 2024","interfaces":[{"interface":"back","average":{"1min":2.9300000000000002,"5min":2.8100000000000001,"15min":2.79},"min":{"1min":0.32900000000000001,"5min":0.189,"15min":0.189},"max":{"1min":13.617000000000001,"5min":34.692,"15min":34.692},"last":0.88300000000000001},{"interface":"front","average":{"1min":2.431,"5min":2.742,"15min":2.794},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":9.7100000000000009,"5min":34.588999999999999,"15min":34.588999999999999},"last":1.841}]},{"osd":2,"last update":"Tue Sep 10 18:57:40 2024","interfaces":[{"interface":"back","average":{"1min":1.883,"5min":1.079,"15min":0.94499999999999995},"min":{"1min":0.22,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":13.305,"5min":13.305,"15min":13.305},"last":0.60699999999999998},{"interface":"front","average":{"1min":1.9079999999999999,"5min":0.93600000000000005,"15min":0.77400000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":13.693,"5min":13.693,"15min":13.693},"last":1.7609999999999999}]},{"osd":3,"last update":"Tue Sep 10 18:57:40 2024","interfaces":[{"interface":"back","average":{"1min":2.3420000000000001,"5min":1.2450000000000001,"15min":1.0620000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":21.108000000000001,"5min":21.108000000000001,"15min":21.108000000000001},"last":3.1360000000000001},{"interface":"front","average":{"1min":3.657,"5min":1.978,"15min":1.698},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":11.629,"5min":11.629,"15min":11.629},"last":5.2190000000000003}]},{"osd":4,"last update":"Tue Sep 10 18:57:40 2024","interfaces":[{"interface":"back","average":{"1min":1.603,"5min":0.89500000000000002,"15min":0.77700000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":13.161,"5min":13.161,"15min":13.161},"last":0.56399999999999995},{"interface":"front","average":{"1min":1.3480000000000001,"5min":0.99199999999999999,"15min":0.93300000000000005},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":4.6020000000000003,"5min":4.6020000000000003,"15min":4.6020000000000003},"last":2.657}]},{"osd":5,"last update":"Tue Sep 10 18:57:40 2024","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.113,"15min":1.044},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":4.5759999999999996,"5min":4.5759999999999996,"15min":4.5759999999999996},"last":2.621},{"interface":"front","average":{"1min":1.9279999999999999,"5min":1.008,"15min":0.85499999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":13.202,"5min":13.202,"15min":13.202},"last":0.68200000000000005}]},{"osd":6,"last update":"Tue Sep 10 18:57:40 2024","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.0109999999999999,"15min":0.94499999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":4.5309999999999997,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":2.581},{"interface":"front","average":{"1min":1.571,"5min":0.871,"15min":0.755},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":13.279,"5min":13.279,"15min":13.279},"last":0.63400000000000001}]},{"osd":7,"last update":"Tue Sep 10 18:57:40 2024","interfaces":[{"interface":"back","average":{"1min":1.9039999999999999,"5min":1.042,"15min":0.89800000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":13.233000000000001,"5min":13.233000000000001,"15min":13.233000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.2130000000000001,"15min":1.169},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":10.141999999999999,"5min":10.141999999999999,"15min":10.141999999999999},"last":1.8080000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":2668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2086928384,"data_stored":2086928384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3683192832,"data_stored":3683188760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3399876608,"data_stored":3399876608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1564086272,"data_stored":1564082200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2161643520,"data_stored":2161639448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1863581696,"data_stored":1863581696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3786543104,"data_stored":3786539032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T18:58:34.420 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd pool set unique_pool_2 pgp_num 16 2024-09-10T18:58:34.428 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:34.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:34.515 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:34.515 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:34.616 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:34.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:34.663 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:34.663 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:34.764 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:34.788 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:34.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:34.790 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:34.890 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:34.914 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:34.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:34.915 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:34.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:34 smithi018 ceph-mon[40196]: osdmap e298: 8 total, 8 up, 6 in 2024-09-10T18:58:34.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:34 smithi018 ceph-mon[29336]: osdmap e298: 8 total, 8 up, 6 in 2024-09-10T18:58:35.016 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:35.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:34 smithi103 ceph-mon[33163]: osdmap e298: 8 total, 8 up, 6 in 2024-09-10T18:58:35.040 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:35.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:35.041 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:35.142 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:35.165 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:35.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:35.166 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:35.266 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:35.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:35.293 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:35.293 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:35.394 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:35.418 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:35.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:35.420 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:35.521 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:35.561 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:35.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:35.562 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:35.663 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:35.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:35.690 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:35.690 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:35.791 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:35.812 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:35.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:35.813 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:35.914 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:35.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:35 smithi018 ceph-mon[40196]: pgmap v611: 168 pgs: 1 active+remapped+backfilling, 167 active+clean; 4.2 GiB data, 17 GiB used, 520 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 1.06k op/s; 45/206892 objects misplaced (0.022%); 5.1 MiB/s, 81 objects/s recovering 2024-09-10T18:58:35.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:35 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3306140219' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:58:35.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:35 smithi018 ceph-mon[29336]: pgmap v611: 168 pgs: 1 active+remapped+backfilling, 167 active+clean; 4.2 GiB data, 17 GiB used, 520 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 1.06k op/s; 45/206892 objects misplaced (0.022%); 5.1 MiB/s, 81 objects/s recovering 2024-09-10T18:58:35.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:35 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3306140219' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:58:35.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:35.954 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:35.954 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:36.021 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:58:36.022 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:58:36.022 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:58:36.022 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:58:36.022 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:58:36.022 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:58:36.022 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:58:36.022 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:58:36.022 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:58:36.022 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:58:36.022 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:58:36.022 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:58:36.022 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:58:36.023 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:58:36.023 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:58:36.023 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:58:36.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:35 smithi103 ceph-mon[33163]: pgmap v611: 168 pgs: 1 active+remapped+backfilling, 167 active+clean; 4.2 GiB data, 17 GiB used, 520 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 1.06k op/s; 45/206892 objects misplaced (0.022%); 5.1 MiB/s, 81 objects/s recovering 2024-09-10T18:58:36.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:35 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3306140219' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:58:36.056 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:36.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:36.095 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:36.095 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:36.196 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:36.224 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:36.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:36.226 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:36.326 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:36.354 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:36.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:36.355 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:36.456 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:36.482 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:36.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:36.483 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:36.583 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:36.627 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:36.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:36.629 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:36.730 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:36.757 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:36.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:36.758 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:36.859 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:36.883 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:36.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:36.884 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:36.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:36 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:58:36.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:36 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:58:36.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:36 smithi018 ceph-mon[40196]: osdmap e299: 8 total, 8 up, 6 in 2024-09-10T18:58:36.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:36 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:58:36.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:36 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:58:36.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:36 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:58:36.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:36 smithi018 ceph-mon[29336]: osdmap e299: 8 total, 8 up, 6 in 2024-09-10T18:58:36.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:36 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:58:36.985 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:37.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:37.007 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:37.007 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:37.022 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:58:36 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:58:37.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:36 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:58:37.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:36 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:58:37.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:36 smithi103 ceph-mon[33163]: osdmap e299: 8 total, 8 up, 6 in 2024-09-10T18:58:37.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:36 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:58:37.109 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:37.154 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:37.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:37.155 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:37.256 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:37.287 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:37.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:37.288 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:37.388 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:37.415 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:37.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:37.416 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:37.517 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:37.562 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:37.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:37.563 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:37.664 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:37.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:37.703 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:37.703 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:37.804 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:37.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:37.850 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:37.850 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:37.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:37 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:58:37.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:37 smithi018 ceph-mon[40196]: pgmap v613: 168 pgs: 168 active+clean; 3.9 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.69k op/s; 5.5 MiB/s, 88 objects/s recovering 2024-09-10T18:58:37.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:37 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:58:37.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:37 smithi018 ceph-mon[29336]: pgmap v613: 168 pgs: 168 active+clean; 3.9 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.69k op/s; 5.5 MiB/s, 88 objects/s recovering 2024-09-10T18:58:37.951 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:37.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:37.977 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:37.977 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:38.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:37 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:58:38.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:37 smithi103 ceph-mon[33163]: pgmap v613: 168 pgs: 168 active+clean; 3.9 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.69k op/s; 5.5 MiB/s, 88 objects/s recovering 2024-09-10T18:58:38.078 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:38.103 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:38.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:38.104 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:38.205 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:38.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:38.247 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:38.247 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:38.348 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:38.390 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:38.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:38.391 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:38.492 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:38.527 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:38.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:38.528 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:38.546 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:58:38.630 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:38.635 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:58:38.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:38.693 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:38.693 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:38.794 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:38.822 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:38.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:38.823 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:38.923 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:38.975 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:38.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:38.976 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:39.077 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:39.105 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:39.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:39.106 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:39.207 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:39.245 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:39.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:39.247 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:39.347 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:39.375 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:39.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:39.376 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:39.378 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T18:58:39.476 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:39.535 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:39.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:39.537 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:39.638 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:39.692 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:39.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:39.693 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:39.794 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:39.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:39.842 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:39.842 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:39.943 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:39.969 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:39.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:39.971 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:40.072 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:40.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:40.118 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:40.118 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:40.219 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:40.258 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:40.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:40.259 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:40.359 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:40.391 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:40.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:40.392 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:40.492 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:40.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:40 smithi103 ceph-mon[33163]: pgmap v614: 168 pgs: 168 active+clean; 3.9 GiB data, 16 GiB used, 521 GiB / 536 GiB avail; 773 B/s rd, 0 B/s wr, 970 op/s; 426 KiB/s, 6 objects/s recovering 2024-09-10T18:58:40.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:40 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:58:40.523 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:58:40 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:58:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:58:40.527 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:40.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:40.528 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:40.629 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:40.667 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:40.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:40.668 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:40.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:40 smithi018 ceph-mon[40196]: pgmap v614: 168 pgs: 168 active+clean; 3.9 GiB data, 16 GiB used, 521 GiB / 536 GiB avail; 773 B/s rd, 0 B/s wr, 970 op/s; 426 KiB/s, 6 objects/s recovering 2024-09-10T18:58:40.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:40 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:58:40.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:40 smithi018 ceph-mon[29336]: pgmap v614: 168 pgs: 168 active+clean; 3.9 GiB data, 16 GiB used, 521 GiB / 536 GiB avail; 773 B/s rd, 0 B/s wr, 970 op/s; 426 KiB/s, 6 objects/s recovering 2024-09-10T18:58:40.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:40 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:58:40.768 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:40.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:40.809 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:40.809 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:40.910 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:40.950 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:40.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:40.952 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:41.052 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:41.080 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:41.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:41.081 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:41.182 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:41.243 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:41.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:41.244 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:41.345 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:41.368 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:41.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:41.369 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:41.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:58:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:58:40] "GET /metrics HTTP/1.1" 200 224464 "" "Prometheus/2.43.0" 2024-09-10T18:58:41.470 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:41.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:41.494 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:41.495 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:41.595 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:41.621 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:41.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:41.622 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:41.722 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:41.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:41.751 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:41.751 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:41.852 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:41.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:41.879 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:41.880 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:41.981 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:42.005 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:42.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:42.006 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:42.107 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:42.135 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:42.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:42.136 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:42.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:41 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:58:42.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:41 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:58:42.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:41 smithi018 ceph-mon[29336]: pgmap v615: 168 pgs: 168 active+clean; 3.8 GiB data, 16 GiB used, 521 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 855 op/s; 352 KiB/s, 5 objects/s recovering 2024-09-10T18:58:42.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:41 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:58:42.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:41 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:58:42.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:41 smithi018 ceph-mon[40196]: pgmap v615: 168 pgs: 168 active+clean; 3.8 GiB data, 16 GiB used, 521 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 855 op/s; 352 KiB/s, 5 objects/s recovering 2024-09-10T18:58:42.237 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:42.258 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:42.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:42.259 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:42.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:41 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:58:42.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:41 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:58:42.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:41 smithi103 ceph-mon[33163]: pgmap v615: 168 pgs: 168 active+clean; 3.8 GiB data, 16 GiB used, 521 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 855 op/s; 352 KiB/s, 5 objects/s recovering 2024-09-10T18:58:42.360 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:42.386 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:42.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:42.387 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:42.488 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:42.528 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:42.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:42.529 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:42.630 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:42.667 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:42.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:42.669 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:42.769 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:42.818 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:42.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:42.819 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:42.920 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:42.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:42.962 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:42.962 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:43.063 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:43.089 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:43.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:43.090 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:43.191 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:43.218 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:43.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:43.220 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:43.321 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:43.345 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:43.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:43.346 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:43.447 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:43.471 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:43.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:43.472 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:43.573 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:43.612 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:43.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:43.614 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:43.715 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:43.754 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:43.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:43.755 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:43.856 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:43.880 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:43.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:43.882 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:43.982 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:44.007 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:44.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:44.008 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:44.109 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:44.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:44.158 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:44.158 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:44.259 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:44.286 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:44.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:44.287 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:44.388 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:44.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:44.415 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:44.416 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:44.480 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T18:58:44.480 INFO:teuthology.orchestra.run.smithi018.stdout: "size": 20, 2024-09-10T18:58:44.480 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 600, 2024-09-10T18:58:44.480 INFO:teuthology.orchestra.run.smithi018.stdout: "ops": [] 2024-09-10T18:58:44.480 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T18:58:44.517 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:44.562 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:44.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:44.563 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:44.664 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:44.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:44.691 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:44.691 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:44.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:44 smithi018 ceph-mon[40196]: pgmap v616: 168 pgs: 168 active+clean; 3.5 GiB data, 15 GiB used, 522 GiB / 536 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.12k op/s; 293 KiB/s, 4 objects/s recovering 2024-09-10T18:58:44.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:44 smithi018 ceph-mon[29336]: pgmap v616: 168 pgs: 168 active+clean; 3.5 GiB data, 15 GiB used, 522 GiB / 536 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.12k op/s; 293 KiB/s, 4 objects/s recovering 2024-09-10T18:58:44.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:44 smithi103 ceph-mon[33163]: pgmap v616: 168 pgs: 168 active+clean; 3.5 GiB data, 15 GiB used, 522 GiB / 536 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.12k op/s; 293 KiB/s, 4 objects/s recovering 2024-09-10T18:58:44.793 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:44.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:44.835 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:44.835 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:44.937 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:44.975 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:44.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:44.977 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:45.078 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:45.106 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:45.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:45.107 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:45.208 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:45.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:45.234 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:45.234 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:45.335 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:45.359 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:45.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:45.360 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:45.461 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:45.509 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:45.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:45.510 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:45.611 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:45.690 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:45.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:45.691 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:45.791 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:45.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:45.834 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:45.834 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:45.935 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:45.961 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:45.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:45.962 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:46.063 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:46.099 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:46.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:46.100 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:46.152 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-10T18:58:46.202 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:46.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:46.251 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:46.251 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:46.352 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:46.378 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:46.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:46.379 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:46.479 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:46.504 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:46.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:46.505 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:46.533 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:58:46.605 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:46.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:46.650 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:46.650 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:46.656 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:46 smithi018 ceph-mon[29336]: pgmap v617: 168 pgs: 168 active+clean; 3.5 GiB data, 15 GiB used, 522 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.20k op/s; 281 KiB/s, 4 objects/s recovering 2024-09-10T18:58:46.656 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:46 smithi018 ceph-mon[40196]: pgmap v617: 168 pgs: 168 active+clean; 3.5 GiB data, 15 GiB used, 522 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.20k op/s; 281 KiB/s, 4 objects/s recovering 2024-09-10T18:58:46.752 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:46.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:46 smithi103 ceph-mon[33163]: pgmap v617: 168 pgs: 168 active+clean; 3.5 GiB data, 15 GiB used, 522 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.20k op/s; 281 KiB/s, 4 objects/s recovering 2024-09-10T18:58:46.801 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:46.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:46.802 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:46.903 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:46.933 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:46.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:46.934 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:47.035 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:47.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:47.075 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:47.075 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:47.159 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:58:46 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:58:47.175 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:47.217 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:47.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:47.218 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:47.319 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:47.356 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:47.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:47.357 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:47.399 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is unset 2024-09-10T18:58:47.458 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:47.486 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:47.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:47.487 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:47.588 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:47.627 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:47.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:47.628 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:47.693 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:47 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/560930243' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:58:47.693 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:47 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:58:47.693 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:47 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/560930243' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:58:47.693 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:47 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:58:47.729 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:47.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:47 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/560930243' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:58:47.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:47 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:58:47.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:47.774 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:47.774 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:47.875 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:47.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:47.916 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:47.916 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:48.017 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:48.051 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:48.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:48.052 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:48.153 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:48.190 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:48.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:48.191 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:48.292 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:48.319 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:48.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:48.320 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:48.421 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:48.442 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:48.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:48.443 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:48.544 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:48.570 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:48.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:48.571 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:48.671 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:48.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:48 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:58:48.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:48 smithi018 ceph-mon[40196]: pgmap v618: 168 pgs: 168 active+clean; 3.1 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.51k op/s; 242 KiB/s, 3 objects/s recovering 2024-09-10T18:58:48.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:48 smithi018 ceph-mon[40196]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:58:48.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:48 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:58:48.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:48 smithi018 ceph-mon[40196]: osdmap e300: 8 total, 8 up, 6 in 2024-09-10T18:58:48.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:48 smithi018 ceph-mon[40196]: 4.8 scrub starts 2024-09-10T18:58:48.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:48 smithi018 ceph-mon[40196]: 4.8 scrub ok 2024-09-10T18:58:48.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:48 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:58:48.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:48 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:58:48.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:48 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:58:48.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:48 smithi018 ceph-mon[29336]: pgmap v618: 168 pgs: 168 active+clean; 3.1 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.51k op/s; 242 KiB/s, 3 objects/s recovering 2024-09-10T18:58:48.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:48 smithi018 ceph-mon[29336]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:58:48.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:48 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:58:48.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:48 smithi018 ceph-mon[29336]: osdmap e300: 8 total, 8 up, 6 in 2024-09-10T18:58:48.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:48 smithi018 ceph-mon[29336]: 4.8 scrub starts 2024-09-10T18:58:48.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:48 smithi018 ceph-mon[29336]: 4.8 scrub ok 2024-09-10T18:58:48.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:48 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:58:48.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:48 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:58:48.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:48.697 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:48.697 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:48.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:48 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:58:48.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:48 smithi103 ceph-mon[33163]: pgmap v618: 168 pgs: 168 active+clean; 3.1 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.51k op/s; 242 KiB/s, 3 objects/s recovering 2024-09-10T18:58:48.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:48 smithi103 ceph-mon[33163]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:58:48.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:48 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:58:48.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:48 smithi103 ceph-mon[33163]: osdmap e300: 8 total, 8 up, 6 in 2024-09-10T18:58:48.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:48 smithi103 ceph-mon[33163]: 4.8 scrub starts 2024-09-10T18:58:48.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:48 smithi103 ceph-mon[33163]: 4.8 scrub ok 2024-09-10T18:58:48.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:48 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:58:48.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:48 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:58:48.798 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:48.838 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:48.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:48.839 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:48.940 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:48.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:48.987 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:48.987 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:49.088 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:49.114 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:49.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:49.115 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:49.215 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:49.241 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:49.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:49.242 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:49.343 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:49.370 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:49.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:49.371 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:49.472 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:49.496 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:49.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:49.498 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:49.598 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:49.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:49.638 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:49.638 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:49.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:49 smithi018 ceph-mon[40196]: osdmap e301: 8 total, 8 up, 6 in 2024-09-10T18:58:49.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:49 smithi018 ceph-mon[40196]: 6.1c scrub starts 2024-09-10T18:58:49.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:49 smithi018 ceph-mon[40196]: 6.1c scrub ok 2024-09-10T18:58:49.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:49 smithi018 ceph-mon[40196]: 6.2 scrub starts 2024-09-10T18:58:49.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:49 smithi018 ceph-mon[40196]: 6.2 scrub ok 2024-09-10T18:58:49.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:49 smithi018 ceph-mon[29336]: osdmap e301: 8 total, 8 up, 6 in 2024-09-10T18:58:49.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:49 smithi018 ceph-mon[29336]: 6.1c scrub starts 2024-09-10T18:58:49.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:49 smithi018 ceph-mon[29336]: 6.1c scrub ok 2024-09-10T18:58:49.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:49 smithi018 ceph-mon[29336]: 6.2 scrub starts 2024-09-10T18:58:49.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:49 smithi018 ceph-mon[29336]: 6.2 scrub ok 2024-09-10T18:58:49.739 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:49.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:49.765 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:49.765 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:49.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:49 smithi103 ceph-mon[33163]: osdmap e301: 8 total, 8 up, 6 in 2024-09-10T18:58:49.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:49 smithi103 ceph-mon[33163]: 6.1c scrub starts 2024-09-10T18:58:49.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:49 smithi103 ceph-mon[33163]: 6.1c scrub ok 2024-09-10T18:58:49.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:49 smithi103 ceph-mon[33163]: 6.2 scrub starts 2024-09-10T18:58:49.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:49 smithi103 ceph-mon[33163]: 6.2 scrub ok 2024-09-10T18:58:49.866 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:49.913 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:49.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:49.914 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:50.015 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:50.055 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:50.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:50.056 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:50.157 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:50.209 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:50.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:50.210 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:50.311 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:50.338 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:50.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:50.340 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:50.440 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:50.481 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:50.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:50.482 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:50.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:58:50 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:58:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:58:50.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:50 smithi103 ceph-mon[33163]: 1.11 scrub starts 2024-09-10T18:58:50.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:50 smithi103 ceph-mon[33163]: 1.11 scrub ok 2024-09-10T18:58:50.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:50 smithi103 ceph-mon[33163]: 1.3 scrub starts 2024-09-10T18:58:50.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:50 smithi103 ceph-mon[33163]: 1.3 scrub ok 2024-09-10T18:58:50.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:50 smithi103 ceph-mon[33163]: pgmap v621: 168 pgs: 168 active+clean; 3.1 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.62k op/s 2024-09-10T18:58:50.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:50 smithi103 ceph-mon[33163]: 2.1c scrub starts 2024-09-10T18:58:50.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:50 smithi103 ceph-mon[33163]: 2.1c scrub ok 2024-09-10T18:58:50.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:50 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3664224892' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]: dispatch 2024-09-10T18:58:50.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:50 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]: dispatch 2024-09-10T18:58:50.582 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:50.609 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:50.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:50.610 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:50.652 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:50 smithi018 ceph-mon[40196]: 1.11 scrub starts 2024-09-10T18:58:50.652 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:50 smithi018 ceph-mon[40196]: 1.11 scrub ok 2024-09-10T18:58:50.652 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:50 smithi018 ceph-mon[40196]: 1.3 scrub starts 2024-09-10T18:58:50.652 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:50 smithi018 ceph-mon[40196]: 1.3 scrub ok 2024-09-10T18:58:50.652 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:50 smithi018 ceph-mon[40196]: pgmap v621: 168 pgs: 168 active+clean; 3.1 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.62k op/s 2024-09-10T18:58:50.652 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:50 smithi018 ceph-mon[40196]: 2.1c scrub starts 2024-09-10T18:58:50.652 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:50 smithi018 ceph-mon[40196]: 2.1c scrub ok 2024-09-10T18:58:50.653 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:50 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3664224892' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]: dispatch 2024-09-10T18:58:50.653 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:50 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]: dispatch 2024-09-10T18:58:50.653 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:50 smithi018 ceph-mon[29336]: 1.11 scrub starts 2024-09-10T18:58:50.653 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:50 smithi018 ceph-mon[29336]: 1.11 scrub ok 2024-09-10T18:58:50.653 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:50 smithi018 ceph-mon[29336]: 1.3 scrub starts 2024-09-10T18:58:50.653 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:50 smithi018 ceph-mon[29336]: 1.3 scrub ok 2024-09-10T18:58:50.653 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:50 smithi018 ceph-mon[29336]: pgmap v621: 168 pgs: 168 active+clean; 3.1 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.62k op/s 2024-09-10T18:58:50.653 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:50 smithi018 ceph-mon[29336]: 2.1c scrub starts 2024-09-10T18:58:50.653 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:50 smithi018 ceph-mon[29336]: 2.1c scrub ok 2024-09-10T18:58:50.653 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:50 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3664224892' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]: dispatch 2024-09-10T18:58:50.653 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:50 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]: dispatch 2024-09-10T18:58:50.704 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:58:50.704 INFO:teuthology.orchestra.run.smithi018.stderr:set pool 9 pgp_num to 16 2024-09-10T18:58:50.711 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:50.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:50.786 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:50.786 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:50.887 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:50.915 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:50.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:50.916 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:51.017 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:51.061 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:51.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:51.062 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:51.163 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:51.189 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:51.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:51.190 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:51.249 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:58:50 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:58:50] "GET /metrics HTTP/1.1" 200 224464 "" "Prometheus/2.43.0" 2024-09-10T18:58:51.290 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:51.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:51.327 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:51.328 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:51.429 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:51.453 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:51.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:51.454 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:51.504 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:51 smithi018 ceph-mon[29336]: 6.f scrub starts 2024-09-10T18:58:51.504 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:51 smithi018 ceph-mon[29336]: 6.f scrub ok 2024-09-10T18:58:51.504 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:51 smithi018 ceph-mon[29336]: 2.4 scrub starts 2024-09-10T18:58:51.504 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:51 smithi018 ceph-mon[29336]: 2.4 scrub ok 2024-09-10T18:58:51.504 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:51 smithi018 ceph-mon[29336]: 2.12 scrub starts 2024-09-10T18:58:51.504 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:51 smithi018 ceph-mon[29336]: 2.12 scrub ok 2024-09-10T18:58:51.504 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:51 smithi018 ceph-mon[29336]: 6.3 scrub starts 2024-09-10T18:58:51.505 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:51 smithi018 ceph-mon[29336]: 6.3 scrub ok 2024-09-10T18:58:51.505 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:51 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]': finished 2024-09-10T18:58:51.505 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:51 smithi018 ceph-mon[29336]: osdmap e302: 8 total, 8 up, 6 in 2024-09-10T18:58:51.505 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:51 smithi018 ceph-mon[40196]: 6.f scrub starts 2024-09-10T18:58:51.505 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:51 smithi018 ceph-mon[40196]: 6.f scrub ok 2024-09-10T18:58:51.505 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:51 smithi018 ceph-mon[40196]: 2.4 scrub starts 2024-09-10T18:58:51.505 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:51 smithi018 ceph-mon[40196]: 2.4 scrub ok 2024-09-10T18:58:51.505 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:51 smithi018 ceph-mon[40196]: 2.12 scrub starts 2024-09-10T18:58:51.505 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:51 smithi018 ceph-mon[40196]: 2.12 scrub ok 2024-09-10T18:58:51.505 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:51 smithi018 ceph-mon[40196]: 6.3 scrub starts 2024-09-10T18:58:51.505 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:51 smithi018 ceph-mon[40196]: 6.3 scrub ok 2024-09-10T18:58:51.506 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:51 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]': finished 2024-09-10T18:58:51.506 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:51 smithi018 ceph-mon[40196]: osdmap e302: 8 total, 8 up, 6 in 2024-09-10T18:58:51.555 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:51.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:51.596 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:51.596 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:51.697 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:51.723 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:51.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:51.724 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:51.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:51 smithi103 ceph-mon[33163]: 6.f scrub starts 2024-09-10T18:58:51.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:51 smithi103 ceph-mon[33163]: 6.f scrub ok 2024-09-10T18:58:51.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:51 smithi103 ceph-mon[33163]: 2.4 scrub starts 2024-09-10T18:58:51.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:51 smithi103 ceph-mon[33163]: 2.4 scrub ok 2024-09-10T18:58:51.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:51 smithi103 ceph-mon[33163]: 2.12 scrub starts 2024-09-10T18:58:51.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:51 smithi103 ceph-mon[33163]: 2.12 scrub ok 2024-09-10T18:58:51.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:51 smithi103 ceph-mon[33163]: 6.3 scrub starts 2024-09-10T18:58:51.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:51 smithi103 ceph-mon[33163]: 6.3 scrub ok 2024-09-10T18:58:51.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:51 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]': finished 2024-09-10T18:58:51.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:51 smithi103 ceph-mon[33163]: osdmap e302: 8 total, 8 up, 6 in 2024-09-10T18:58:51.825 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:51.852 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:51.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:51.853 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:51.954 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:51.998 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:51.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:51.999 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:52.099 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:52.136 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-10T18:58:52.138 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:52.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:52.139 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:52.240 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:52.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:52.299 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:52.299 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:52.400 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:52.423 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:52.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:52.424 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:52.525 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:52.550 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:52.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:52.551 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:52.633 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:52 smithi018 ceph-mon[29336]: 2.15 scrub starts 2024-09-10T18:58:52.633 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:52 smithi018 ceph-mon[29336]: 2.15 scrub ok 2024-09-10T18:58:52.633 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:52 smithi018 ceph-mon[29336]: pgmap v623: 168 pgs: 168 active+clean; 3.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 1.20k op/s 2024-09-10T18:58:52.634 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:52 smithi018 ceph-mon[29336]: 6.1a scrub starts 2024-09-10T18:58:52.634 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:52 smithi018 ceph-mon[29336]: 6.1a scrub ok 2024-09-10T18:58:52.634 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:52 smithi018 ceph-mon[40196]: 2.15 scrub starts 2024-09-10T18:58:52.634 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:52 smithi018 ceph-mon[40196]: 2.15 scrub ok 2024-09-10T18:58:52.634 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:52 smithi018 ceph-mon[40196]: pgmap v623: 168 pgs: 168 active+clean; 3.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 1.20k op/s 2024-09-10T18:58:52.634 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:52 smithi018 ceph-mon[40196]: 6.1a scrub starts 2024-09-10T18:58:52.634 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:52 smithi018 ceph-mon[40196]: 6.1a scrub ok 2024-09-10T18:58:52.653 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:52.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:52.693 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:52.694 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:52.713 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:58:52.713 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:58:52.713 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:58:52.713 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:58:52.714 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:58:52.714 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:58:52.714 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:58:52.714 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:58:52.714 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:58:52.714 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:58:52.725 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:58:52.726 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:58:52.761 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:58:52.762 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:58:52.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:52 smithi103 ceph-mon[33163]: 2.15 scrub starts 2024-09-10T18:58:52.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:52 smithi103 ceph-mon[33163]: 2.15 scrub ok 2024-09-10T18:58:52.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:52 smithi103 ceph-mon[33163]: pgmap v623: 168 pgs: 168 active+clean; 3.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 1.20k op/s 2024-09-10T18:58:52.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:52 smithi103 ceph-mon[33163]: 6.1a scrub starts 2024-09-10T18:58:52.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:52 smithi103 ceph-mon[33163]: 6.1a scrub ok 2024-09-10T18:58:52.790 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:58:52.790 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:58:52.794 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:52.837 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:52.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:52.838 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:52.939 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:52.965 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:52.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:52.966 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:53.066 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:53.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:53.093 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:53.094 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:53.194 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:53.229 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:53.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:53.233 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:53.334 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:53.373 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:53.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:53.377 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:53.478 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:53.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:53.501 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:53.502 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:53.602 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:53.645 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:53.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:53.646 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:53.746 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:53.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:53.770 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:53.770 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:53.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:53 smithi103 ceph-mon[33163]: 6.18 scrub starts 2024-09-10T18:58:53.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:53 smithi103 ceph-mon[33163]: 6.18 scrub ok 2024-09-10T18:58:53.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:53 smithi103 ceph-mon[33163]: 6.4 scrub starts 2024-09-10T18:58:53.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:53 smithi103 ceph-mon[33163]: 6.4 scrub ok 2024-09-10T18:58:53.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:53 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2060027268' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:58:53.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:53 smithi103 ceph-mon[33163]: 1.12 scrub starts 2024-09-10T18:58:53.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:53 smithi103 ceph-mon[33163]: 1.12 scrub ok 2024-09-10T18:58:53.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:53 smithi103 ceph-mon[33163]: 4.1d scrub starts 2024-09-10T18:58:53.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:53 smithi103 ceph-mon[33163]: 4.1d scrub ok 2024-09-10T18:58:53.871 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:53.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:53.912 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:53.912 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:53.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:53 smithi018 ceph-mon[29336]: 6.18 scrub starts 2024-09-10T18:58:53.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:53 smithi018 ceph-mon[29336]: 6.18 scrub ok 2024-09-10T18:58:53.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:53 smithi018 ceph-mon[29336]: 6.4 scrub starts 2024-09-10T18:58:53.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:53 smithi018 ceph-mon[29336]: 6.4 scrub ok 2024-09-10T18:58:53.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:53 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2060027268' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:58:53.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:53 smithi018 ceph-mon[29336]: 1.12 scrub starts 2024-09-10T18:58:53.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:53 smithi018 ceph-mon[29336]: 1.12 scrub ok 2024-09-10T18:58:53.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:53 smithi018 ceph-mon[29336]: 4.1d scrub starts 2024-09-10T18:58:53.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:53 smithi018 ceph-mon[29336]: 4.1d scrub ok 2024-09-10T18:58:53.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:53 smithi018 ceph-mon[40196]: 6.18 scrub starts 2024-09-10T18:58:53.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:53 smithi018 ceph-mon[40196]: 6.18 scrub ok 2024-09-10T18:58:53.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:53 smithi018 ceph-mon[40196]: 6.4 scrub starts 2024-09-10T18:58:53.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:53 smithi018 ceph-mon[40196]: 6.4 scrub ok 2024-09-10T18:58:53.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:53 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2060027268' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:58:53.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:53 smithi018 ceph-mon[40196]: 1.12 scrub starts 2024-09-10T18:58:53.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:53 smithi018 ceph-mon[40196]: 1.12 scrub ok 2024-09-10T18:58:53.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:53 smithi018 ceph-mon[40196]: 4.1d scrub starts 2024-09-10T18:58:53.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:53 smithi018 ceph-mon[40196]: 4.1d scrub ok 2024-09-10T18:58:54.014 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:54.038 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:54.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:54.039 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:54.140 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:54.165 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:54.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:54.166 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:54.267 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:54.292 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:54.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:54.293 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:54.394 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:54.429 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:54.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:54.430 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:54.531 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:54.569 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:54.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:54.570 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:54.671 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:54.706 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:54.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:54.708 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:54.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:54 smithi103 ceph-mon[33163]: pgmap v624: 168 pgs: 168 active+clean; 2.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 1.11k op/s 2024-09-10T18:58:54.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:54 smithi103 ceph-mon[33163]: 2.16 scrub starts 2024-09-10T18:58:54.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:54 smithi103 ceph-mon[33163]: 2.16 scrub ok 2024-09-10T18:58:54.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:54 smithi103 ceph-mon[33163]: 6.15 scrub starts 2024-09-10T18:58:54.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:54 smithi103 ceph-mon[33163]: 6.15 scrub ok 2024-09-10T18:58:54.808 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:54.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:54.833 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:54.833 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:54.934 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:54.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:54 smithi018 ceph-mon[40196]: pgmap v624: 168 pgs: 168 active+clean; 2.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 1.11k op/s 2024-09-10T18:58:54.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:54 smithi018 ceph-mon[40196]: 2.16 scrub starts 2024-09-10T18:58:54.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:54 smithi018 ceph-mon[40196]: 2.16 scrub ok 2024-09-10T18:58:54.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:54 smithi018 ceph-mon[40196]: 6.15 scrub starts 2024-09-10T18:58:54.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:54 smithi018 ceph-mon[40196]: 6.15 scrub ok 2024-09-10T18:58:54.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:54 smithi018 ceph-mon[29336]: pgmap v624: 168 pgs: 168 active+clean; 2.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 1.11k op/s 2024-09-10T18:58:54.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:54 smithi018 ceph-mon[29336]: 2.16 scrub starts 2024-09-10T18:58:54.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:54 smithi018 ceph-mon[29336]: 2.16 scrub ok 2024-09-10T18:58:54.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:54 smithi018 ceph-mon[29336]: 6.15 scrub starts 2024-09-10T18:58:54.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:54 smithi018 ceph-mon[29336]: 6.15 scrub ok 2024-09-10T18:58:54.956 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:54.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:54.957 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:55.058 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:55.101 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:55.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:55.102 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:55.203 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:55.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:55.226 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:55.226 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:55.327 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:55.365 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:55.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:55.367 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:55.467 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:55.499 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:55.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:55.500 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:55.601 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:55.623 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:55.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:55.624 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:55.725 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:55.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:55.760 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:55.760 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:55.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:55 smithi103 ceph-mon[33163]: 5.f scrub starts 2024-09-10T18:58:55.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:55 smithi103 ceph-mon[33163]: 5.f scrub ok 2024-09-10T18:58:55.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:58:55.861 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:55.893 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:55.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:55.894 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:55.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:55 smithi018 ceph-mon[40196]: 5.f scrub starts 2024-09-10T18:58:55.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:55 smithi018 ceph-mon[40196]: 5.f scrub ok 2024-09-10T18:58:55.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:58:55.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:55 smithi018 ceph-mon[29336]: 5.f scrub starts 2024-09-10T18:58:55.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:55 smithi018 ceph-mon[29336]: 5.f scrub ok 2024-09-10T18:58:55.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:58:55.995 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:56.031 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:56.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:56.032 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:56.133 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:56.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:56.156 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:56.156 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:56.257 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:56.294 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:56.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:56.296 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:56.396 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:56.438 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:56.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:56.440 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:56.540 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:56.584 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:56.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:56.585 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:56.686 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:56.712 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:56.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:56.713 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:56.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:56 smithi103 ceph-mon[33163]: pgmap v625: 168 pgs: 168 active+clean; 2.6 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 645 B/s rd, 0 B/s wr, 1.00k op/s 2024-09-10T18:58:56.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:56 smithi103 ceph-mon[33163]: 6.1e scrub starts 2024-09-10T18:58:56.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:56 smithi103 ceph-mon[33163]: 6.1e scrub ok 2024-09-10T18:58:56.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:56 smithi103 ceph-mon[33163]: 3.2 scrub starts 2024-09-10T18:58:56.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:56 smithi103 ceph-mon[33163]: 3.2 scrub ok 2024-09-10T18:58:56.788 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:56 smithi018 ceph-mon[40196]: pgmap v625: 168 pgs: 168 active+clean; 2.6 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 645 B/s rd, 0 B/s wr, 1.00k op/s 2024-09-10T18:58:56.789 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:56 smithi018 ceph-mon[40196]: 6.1e scrub starts 2024-09-10T18:58:56.789 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:56 smithi018 ceph-mon[40196]: 6.1e scrub ok 2024-09-10T18:58:56.789 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:56 smithi018 ceph-mon[40196]: 3.2 scrub starts 2024-09-10T18:58:56.789 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:56 smithi018 ceph-mon[40196]: 3.2 scrub ok 2024-09-10T18:58:56.789 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:56 smithi018 ceph-mon[29336]: pgmap v625: 168 pgs: 168 active+clean; 2.6 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 645 B/s rd, 0 B/s wr, 1.00k op/s 2024-09-10T18:58:56.789 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:56 smithi018 ceph-mon[29336]: 6.1e scrub starts 2024-09-10T18:58:56.789 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:56 smithi018 ceph-mon[29336]: 6.1e scrub ok 2024-09-10T18:58:56.789 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:56 smithi018 ceph-mon[29336]: 3.2 scrub starts 2024-09-10T18:58:56.789 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:56 smithi018 ceph-mon[29336]: 3.2 scrub ok 2024-09-10T18:58:56.814 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:56.839 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:56.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:56.840 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:56.940 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:56.965 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:56.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:56.966 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:57.066 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:57.089 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:57.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:57.090 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:57.191 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:57.233 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:57.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:57.234 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:57.272 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:58:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:58:57.334 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:57.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:57.361 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:57.361 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:57.462 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:57.512 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:57.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:57.513 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:57.614 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:57.636 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:57.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:57.637 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:57.738 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:57.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:57 smithi103 ceph-mon[33163]: 5.1c scrub starts 2024-09-10T18:58:57.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:57 smithi103 ceph-mon[33163]: 5.1c scrub ok 2024-09-10T18:58:57.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:57 smithi103 ceph-mon[33163]: 1.4 scrub starts 2024-09-10T18:58:57.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:57 smithi103 ceph-mon[33163]: 1.4 scrub ok 2024-09-10T18:58:57.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:57 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.7", "id": [1, 6]}]: dispatch 2024-09-10T18:58:57.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:57 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.15", "id": [1, 6]}]: dispatch 2024-09-10T18:58:57.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:57 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.1a", "id": [1, 3]}]: dispatch 2024-09-10T18:58:57.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:57 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.7", "id": [1, 6]}]: dispatch 2024-09-10T18:58:57.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:57 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.15", "id": [1, 6]}]: dispatch 2024-09-10T18:58:57.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:57 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.1a", "id": [1, 3]}]: dispatch 2024-09-10T18:58:57.775 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:57.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:57.776 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:57.877 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:57.911 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:57.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:57.913 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:57.924 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:57 smithi018 ceph-mon[29336]: 5.1c scrub starts 2024-09-10T18:58:57.924 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:57 smithi018 ceph-mon[29336]: 5.1c scrub ok 2024-09-10T18:58:57.924 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:57 smithi018 ceph-mon[29336]: 1.4 scrub starts 2024-09-10T18:58:57.924 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:57 smithi018 ceph-mon[29336]: 1.4 scrub ok 2024-09-10T18:58:57.924 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:57 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.7", "id": [1, 6]}]: dispatch 2024-09-10T18:58:57.925 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:57 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.15", "id": [1, 6]}]: dispatch 2024-09-10T18:58:57.925 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:57 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.1a", "id": [1, 3]}]: dispatch 2024-09-10T18:58:57.925 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:57 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.7", "id": [1, 6]}]: dispatch 2024-09-10T18:58:57.925 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:57 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.15", "id": [1, 6]}]: dispatch 2024-09-10T18:58:57.925 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:57 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.1a", "id": [1, 3]}]: dispatch 2024-09-10T18:58:57.925 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:57 smithi018 ceph-mon[40196]: 5.1c scrub starts 2024-09-10T18:58:57.925 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:57 smithi018 ceph-mon[40196]: 5.1c scrub ok 2024-09-10T18:58:57.925 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:57 smithi018 ceph-mon[40196]: 1.4 scrub starts 2024-09-10T18:58:57.925 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:57 smithi018 ceph-mon[40196]: 1.4 scrub ok 2024-09-10T18:58:57.925 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:57 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.7", "id": [1, 6]}]: dispatch 2024-09-10T18:58:57.925 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:57 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.15", "id": [1, 6]}]: dispatch 2024-09-10T18:58:57.926 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:57 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.1a", "id": [1, 3]}]: dispatch 2024-09-10T18:58:57.926 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:57 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.7", "id": [1, 6]}]: dispatch 2024-09-10T18:58:57.926 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:57 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.15", "id": [1, 6]}]: dispatch 2024-09-10T18:58:57.926 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:57 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.1a", "id": [1, 3]}]: dispatch 2024-09-10T18:58:58.013 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:58.039 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:58.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:58.040 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:58.141 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:58.167 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:58.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:58.169 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:58:58.184 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:58:58.269 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:58.320 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:58.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:58.321 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:58.422 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:58.462 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:58.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:58.464 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:58.565 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:58.605 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:58.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:58.606 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:58.707 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:58.751 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:58.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:58.752 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:58.756 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:58 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:58:58.756 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:58 smithi018 ceph-mon[29336]: pgmap v626: 168 pgs: 168 active+clean; 2.3 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.47k op/s 2024-09-10T18:58:58.756 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:58 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.7", "id": [1, 6]}]': finished 2024-09-10T18:58:58.756 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:58 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.15", "id": [1, 6]}]': finished 2024-09-10T18:58:58.757 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:58 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.1a", "id": [1, 3]}]': finished 2024-09-10T18:58:58.757 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:58 smithi018 ceph-mon[29336]: osdmap e303: 8 total, 8 up, 6 in 2024-09-10T18:58:58.757 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:58 smithi018 ceph-mon[29336]: 9.6 scrub starts 2024-09-10T18:58:58.757 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:58 smithi018 ceph-mon[29336]: 2.1b scrub starts 2024-09-10T18:58:58.757 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:58 smithi018 ceph-mon[29336]: 2.1b scrub ok 2024-09-10T18:58:58.757 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:58 smithi018 ceph-mon[29336]: 9.6 scrub ok 2024-09-10T18:58:58.757 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:58 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:58:58.757 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:58 smithi018 ceph-mon[40196]: pgmap v626: 168 pgs: 168 active+clean; 2.3 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.47k op/s 2024-09-10T18:58:58.757 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:58 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.7", "id": [1, 6]}]': finished 2024-09-10T18:58:58.757 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:58 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.15", "id": [1, 6]}]': finished 2024-09-10T18:58:58.758 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:58 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.1a", "id": [1, 3]}]': finished 2024-09-10T18:58:58.758 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:58 smithi018 ceph-mon[40196]: osdmap e303: 8 total, 8 up, 6 in 2024-09-10T18:58:58.758 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:58 smithi018 ceph-mon[40196]: 9.6 scrub starts 2024-09-10T18:58:58.758 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:58 smithi018 ceph-mon[40196]: 2.1b scrub starts 2024-09-10T18:58:58.758 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:58 smithi018 ceph-mon[40196]: 2.1b scrub ok 2024-09-10T18:58:58.758 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:58 smithi018 ceph-mon[40196]: 9.6 scrub ok 2024-09-10T18:58:58.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:58 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:58:58.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:58 smithi103 ceph-mon[33163]: pgmap v626: 168 pgs: 168 active+clean; 2.3 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.47k op/s 2024-09-10T18:58:58.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:58 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.7", "id": [1, 6]}]': finished 2024-09-10T18:58:58.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:58 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.15", "id": [1, 6]}]': finished 2024-09-10T18:58:58.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:58 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.1a", "id": [1, 3]}]': finished 2024-09-10T18:58:58.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:58 smithi103 ceph-mon[33163]: osdmap e303: 8 total, 8 up, 6 in 2024-09-10T18:58:58.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:58 smithi103 ceph-mon[33163]: 9.6 scrub starts 2024-09-10T18:58:58.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:58 smithi103 ceph-mon[33163]: 2.1b scrub starts 2024-09-10T18:58:58.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:58 smithi103 ceph-mon[33163]: 2.1b scrub ok 2024-09-10T18:58:58.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:58 smithi103 ceph-mon[33163]: 9.6 scrub ok 2024-09-10T18:58:58.853 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:58.878 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:58.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:58.880 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:58.981 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:58:59.017 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:59.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:59.018 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:58:59.057 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6] out_osds: [2, 7] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 6, 7, 0] 2024-09-10T18:58:59.058 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T18:58:59.058 INFO:tasks.thrashosds.thrasher:Shrinking pool unique_pool_2 2024-09-10T18:58:59.058 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool unique_pool_2 by 10 min 8 2024-09-10T18:58:59.058 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T18:58:59.120 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:58:59.166 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:59.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:59.167 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:58:59.268 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:58:59.290 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:59.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:59.292 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:58:59.392 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:58:59.418 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:59.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:59.419 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:58:59.520 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:58:59.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:59.574 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:59.574 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:58:59.675 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:58:59.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:59 smithi018 ceph-mon[29336]: 6.16 scrub starts 2024-09-10T18:58:59.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:59 smithi018 ceph-mon[29336]: 6.16 scrub ok 2024-09-10T18:58:59.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:59 smithi018 ceph-mon[29336]: osdmap e304: 8 total, 8 up, 6 in 2024-09-10T18:58:59.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:59 smithi018 ceph-mon[29336]: 5.6 scrub starts 2024-09-10T18:58:59.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:59 smithi018 ceph-mon[29336]: 5.6 scrub ok 2024-09-10T18:58:59.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:59 smithi018 ceph-mon[29336]: 4.13 scrub starts 2024-09-10T18:58:59.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:58:59 smithi018 ceph-mon[29336]: 4.13 scrub ok 2024-09-10T18:58:59.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:59 smithi018 ceph-mon[40196]: 6.16 scrub starts 2024-09-10T18:58:59.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:59 smithi018 ceph-mon[40196]: 6.16 scrub ok 2024-09-10T18:58:59.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:59 smithi018 ceph-mon[40196]: osdmap e304: 8 total, 8 up, 6 in 2024-09-10T18:58:59.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:59 smithi018 ceph-mon[40196]: 5.6 scrub starts 2024-09-10T18:58:59.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:59 smithi018 ceph-mon[40196]: 5.6 scrub ok 2024-09-10T18:58:59.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:59 smithi018 ceph-mon[40196]: 4.13 scrub starts 2024-09-10T18:58:59.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:58:59 smithi018 ceph-mon[40196]: 4.13 scrub ok 2024-09-10T18:58:59.700 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:59.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:59.701 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:58:59.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:59 smithi103 ceph-mon[33163]: 6.16 scrub starts 2024-09-10T18:58:59.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:59 smithi103 ceph-mon[33163]: 6.16 scrub ok 2024-09-10T18:58:59.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:59 smithi103 ceph-mon[33163]: osdmap e304: 8 total, 8 up, 6 in 2024-09-10T18:58:59.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:59 smithi103 ceph-mon[33163]: 5.6 scrub starts 2024-09-10T18:58:59.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:59 smithi103 ceph-mon[33163]: 5.6 scrub ok 2024-09-10T18:58:59.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:59 smithi103 ceph-mon[33163]: 4.13 scrub starts 2024-09-10T18:58:59.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:58:59 smithi103 ceph-mon[33163]: 4.13 scrub ok 2024-09-10T18:58:59.802 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:58:59.826 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:58:59.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:59.827 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:58:59.927 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:58:59.953 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:58:59.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:58:59.954 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:00.055 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:00.081 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:00.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:00.082 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:00.151 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T18:59:00.183 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:00.238 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:00.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:00.239 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:00.340 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:00.361 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:00.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:00.362 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:00.399 INFO:teuthology.orchestra.run.smithi018.stdout:ERROR: (22) Invalid argument 2024-09-10T18:59:00.399 INFO:teuthology.orchestra.run.smithi018.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:59:00.463 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:00.494 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:59:00 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:59:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:59:00.508 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:00.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:00.509 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:00.610 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:00.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:00.638 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:00.638 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:00.739 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:00.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:00 smithi103 ceph-mon[33163]: 4.3 scrub starts 2024-09-10T18:59:00.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:00 smithi103 ceph-mon[33163]: 4.3 scrub ok 2024-09-10T18:59:00.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:00 smithi103 ceph-mon[33163]: 6.1d scrub starts 2024-09-10T18:59:00.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:00 smithi103 ceph-mon[33163]: 6.1d scrub ok 2024-09-10T18:59:00.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:00 smithi103 ceph-mon[33163]: pgmap v629: 168 pgs: 168 active+clean; 2.3 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.56k op/s 2024-09-10T18:59:00.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:00 smithi103 ceph-mon[33163]: 1.f scrub starts 2024-09-10T18:59:00.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:00 smithi103 ceph-mon[33163]: 1.f scrub ok 2024-09-10T18:59:00.774 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:00.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:00.775 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:00.876 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:00.902 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:00.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:00.903 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:00.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:00 smithi018 ceph-mon[40196]: 4.3 scrub starts 2024-09-10T18:59:00.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:00 smithi018 ceph-mon[40196]: 4.3 scrub ok 2024-09-10T18:59:00.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:00 smithi018 ceph-mon[40196]: 6.1d scrub starts 2024-09-10T18:59:00.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:00 smithi018 ceph-mon[40196]: 6.1d scrub ok 2024-09-10T18:59:00.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:00 smithi018 ceph-mon[40196]: pgmap v629: 168 pgs: 168 active+clean; 2.3 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.56k op/s 2024-09-10T18:59:00.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:00 smithi018 ceph-mon[40196]: 1.f scrub starts 2024-09-10T18:59:00.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:00 smithi018 ceph-mon[40196]: 1.f scrub ok 2024-09-10T18:59:00.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:00 smithi018 ceph-mon[29336]: 4.3 scrub starts 2024-09-10T18:59:00.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:00 smithi018 ceph-mon[29336]: 4.3 scrub ok 2024-09-10T18:59:00.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:00 smithi018 ceph-mon[29336]: 6.1d scrub starts 2024-09-10T18:59:00.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:00 smithi018 ceph-mon[29336]: 6.1d scrub ok 2024-09-10T18:59:00.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:00 smithi018 ceph-mon[29336]: pgmap v629: 168 pgs: 168 active+clean; 2.3 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.56k op/s 2024-09-10T18:59:00.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:00 smithi018 ceph-mon[29336]: 1.f scrub starts 2024-09-10T18:59:00.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:00 smithi018 ceph-mon[29336]: 1.f scrub ok 2024-09-10T18:59:01.003 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:01.046 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:01.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:01.047 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:01.148 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:01.193 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:01.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:01.194 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:01.295 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:01.321 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:01.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:01.323 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:01.423 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:01.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:59:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:59:00] "GET /metrics HTTP/1.1" 200 224449 "" "Prometheus/2.43.0" 2024-09-10T18:59:01.461 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:01.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:01.463 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:01.563 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:01.585 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:01.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:01.587 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:01.597 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-10T18:59:01.687 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:01.711 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:01.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:01.713 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:01.757 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:01 smithi018 ceph-mon[40196]: 9.7 scrub starts 2024-09-10T18:59:01.757 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:01 smithi018 ceph-mon[40196]: 9.7 scrub ok 2024-09-10T18:59:01.757 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:01 smithi018 ceph-mon[29336]: 9.7 scrub starts 2024-09-10T18:59:01.757 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:01 smithi018 ceph-mon[29336]: 9.7 scrub ok 2024-09-10T18:59:01.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:01 smithi103 ceph-mon[33163]: 9.7 scrub starts 2024-09-10T18:59:01.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:01 smithi103 ceph-mon[33163]: 9.7 scrub ok 2024-09-10T18:59:01.814 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:01.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:01.854 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:01.854 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:01.955 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:01.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:01.980 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:01.981 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:02.081 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:02.107 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:02.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:02.108 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:02.209 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:02.232 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:02.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:02.233 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:02.284 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:59:02.334 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:02.384 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:02.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:02.386 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:02.487 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:02.512 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:02.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:02.513 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:02.614 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:02.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:02.653 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:02.653 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:02.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:02 smithi018 ceph-mon[40196]: pgmap v630: 168 pgs: 168 active+clean; 2.2 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 989 op/s 2024-09-10T18:59:02.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:02 smithi018 ceph-mon[29336]: pgmap v630: 168 pgs: 168 active+clean; 2.2 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 989 op/s 2024-09-10T18:59:02.707 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:59:02.710 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:59:02.754 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:02.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:02 smithi103 ceph-mon[33163]: pgmap v630: 168 pgs: 168 active+clean; 2.2 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 989 op/s 2024-09-10T18:59:02.802 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:02.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:02.806 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:02.907 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:02.945 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:02.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:02.946 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:03.047 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:03.109 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:03.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:03.110 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:03.211 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:03.257 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:03.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:03.258 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:03.359 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:03.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:03.400 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:03.400 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:03.501 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:03.526 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:03.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:03.527 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:03.628 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:03.672 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:03.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:03.674 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:03.774 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:03.813 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:03.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:03.814 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:03.915 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:03.951 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:03.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:03.953 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:04.053 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:04.093 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:04.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:04.094 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:04.195 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:04.237 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:04.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:04.238 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:04.339 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:04.362 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:04.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:04.363 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:04.464 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:04.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:04.489 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:04.489 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:04.520 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is unset 2024-09-10T18:59:04.590 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:04.634 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:04.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:04.635 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:04.735 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:04.765 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:04.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:04.766 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:04.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:04 smithi103 ceph-mon[33163]: pgmap v631: 168 pgs: 168 active+clean; 1.9 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.44k op/s; 0 B/s, 0 objects/s recovering 2024-09-10T18:59:04.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:04 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3676668042' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:59:04.867 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:04.906 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:04.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:04.907 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:04.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:04 smithi018 ceph-mon[29336]: pgmap v631: 168 pgs: 168 active+clean; 1.9 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.44k op/s; 0 B/s, 0 objects/s recovering 2024-09-10T18:59:04.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:04 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3676668042' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:59:04.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:04 smithi018 ceph-mon[40196]: pgmap v631: 168 pgs: 168 active+clean; 1.9 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.44k op/s; 0 B/s, 0 objects/s recovering 2024-09-10T18:59:04.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:04 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3676668042' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:59:05.008 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:05.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:05.048 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:05.049 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:05.149 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:05.177 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:05.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:05.178 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:05.279 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:05.305 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:05.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:05.306 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:05.407 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:05.428 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:05.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:05.429 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:05.530 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:05.556 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:05.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:05.557 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:05.658 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:05.684 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:05.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:05.685 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:05.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:05 smithi103 ceph-mon[33163]: 2.2 scrub starts 2024-09-10T18:59:05.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:05 smithi103 ceph-mon[33163]: 2.2 scrub ok 2024-09-10T18:59:05.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:05 smithi103 ceph-mon[33163]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:59:05.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:05 smithi103 ceph-mon[33163]: Cluster is now healthy 2024-09-10T18:59:05.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:05 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3676668042' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:59:05.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:05 smithi103 ceph-mon[33163]: osdmap e305: 8 total, 8 up, 6 in 2024-09-10T18:59:05.786 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:05.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:05.829 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:05.830 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:05.930 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:05.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:05 smithi018 ceph-mon[40196]: 2.2 scrub starts 2024-09-10T18:59:05.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:05 smithi018 ceph-mon[40196]: 2.2 scrub ok 2024-09-10T18:59:05.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:05 smithi018 ceph-mon[40196]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:59:05.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:05 smithi018 ceph-mon[40196]: Cluster is now healthy 2024-09-10T18:59:05.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:05 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3676668042' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:59:05.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:05 smithi018 ceph-mon[40196]: osdmap e305: 8 total, 8 up, 6 in 2024-09-10T18:59:05.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:05 smithi018 ceph-mon[29336]: 2.2 scrub starts 2024-09-10T18:59:05.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:05 smithi018 ceph-mon[29336]: 2.2 scrub ok 2024-09-10T18:59:05.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:05 smithi018 ceph-mon[29336]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:59:05.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:05 smithi018 ceph-mon[29336]: Cluster is now healthy 2024-09-10T18:59:05.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:05 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3676668042' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:59:05.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:05 smithi018 ceph-mon[29336]: osdmap e305: 8 total, 8 up, 6 in 2024-09-10T18:59:05.970 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:05.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:05.971 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:06.072 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:06.118 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:06.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:06.119 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:06.220 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:06.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:06.259 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:06.260 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:06.360 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:06.387 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:06.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:06.388 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:06.489 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:06.530 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:06.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:06.531 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:06.632 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:06.659 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:06.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:06.660 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:06.760 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:06.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:06 smithi103 ceph-mon[33163]: 2.6 scrub starts 2024-09-10T18:59:06.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:06 smithi103 ceph-mon[33163]: 2.6 scrub ok 2024-09-10T18:59:06.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:06 smithi103 ceph-mon[33163]: pgmap v633: 168 pgs: 168 active+clean; 1.9 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 651 B/s rd, 0 B/s wr, 958 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T18:59:06.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:06 smithi103 ceph-mon[33163]: 2.11 scrub starts 2024-09-10T18:59:06.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:06 smithi103 ceph-mon[33163]: 2.11 scrub ok 2024-09-10T18:59:06.783 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:06.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:06.784 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:06.876 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:06 smithi018 ceph-mon[29336]: 2.6 scrub starts 2024-09-10T18:59:06.876 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:06 smithi018 ceph-mon[29336]: 2.6 scrub ok 2024-09-10T18:59:06.876 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:06 smithi018 ceph-mon[29336]: pgmap v633: 168 pgs: 168 active+clean; 1.9 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 651 B/s rd, 0 B/s wr, 958 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T18:59:06.876 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:06 smithi018 ceph-mon[29336]: 2.11 scrub starts 2024-09-10T18:59:06.877 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:06 smithi018 ceph-mon[29336]: 2.11 scrub ok 2024-09-10T18:59:06.877 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:06 smithi018 ceph-mon[40196]: 2.6 scrub starts 2024-09-10T18:59:06.877 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:06 smithi018 ceph-mon[40196]: 2.6 scrub ok 2024-09-10T18:59:06.877 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:06 smithi018 ceph-mon[40196]: pgmap v633: 168 pgs: 168 active+clean; 1.9 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 651 B/s rd, 0 B/s wr, 958 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T18:59:06.877 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:06 smithi018 ceph-mon[40196]: 2.11 scrub starts 2024-09-10T18:59:06.877 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:06 smithi018 ceph-mon[40196]: 2.11 scrub ok 2024-09-10T18:59:06.885 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:06.926 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:06.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:06.927 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:07.028 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:07.064 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:07.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:07.065 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:07.166 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:07.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:07.191 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:07.191 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:07.272 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:59:06 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:59:07.291 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:07.315 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:07.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:07.316 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:07.417 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:07.440 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:07.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:07.441 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:07.542 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:07.566 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:07.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:07.567 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:07.668 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:07.703 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:07.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:07.704 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:07.800 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:07 smithi018 ceph-mon[40196]: 9.9 scrub starts 2024-09-10T18:59:07.800 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:07 smithi018 ceph-mon[40196]: 9.9 scrub ok 2024-09-10T18:59:07.800 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:07 smithi018 ceph-mon[40196]: 4.e scrub starts 2024-09-10T18:59:07.800 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:07 smithi018 ceph-mon[40196]: 4.e scrub ok 2024-09-10T18:59:07.801 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:07 smithi018 ceph-mon[29336]: 9.9 scrub starts 2024-09-10T18:59:07.801 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:07 smithi018 ceph-mon[29336]: 9.9 scrub ok 2024-09-10T18:59:07.804 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:07 smithi018 ceph-mon[29336]: 4.e scrub starts 2024-09-10T18:59:07.804 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:07 smithi018 ceph-mon[29336]: 4.e scrub ok 2024-09-10T18:59:07.805 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:07.828 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:07.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:07.830 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:07.930 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:07.973 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:07.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:07.974 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:08.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:07 smithi103 ceph-mon[33163]: 9.9 scrub starts 2024-09-10T18:59:08.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:07 smithi103 ceph-mon[33163]: 9.9 scrub ok 2024-09-10T18:59:08.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:07 smithi103 ceph-mon[33163]: 4.e scrub starts 2024-09-10T18:59:08.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:07 smithi103 ceph-mon[33163]: 4.e scrub ok 2024-09-10T18:59:08.075 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:08.112 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:08.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:08.113 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:08.214 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:08.240 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:08.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:08.241 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:08.342 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:08.381 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:08.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:08.382 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:08.483 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:08.589 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:08.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:08.590 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:08.691 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:08.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:08 smithi018 ceph-mon[40196]: 9.5 scrub starts 2024-09-10T18:59:08.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:08 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:59:08.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:08 smithi018 ceph-mon[40196]: pgmap v634: 168 pgs: 168 active+clean; 1.5 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.48k op/s; 0 B/s, 0 objects/s recovering 2024-09-10T18:59:08.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:08 smithi018 ceph-mon[40196]: 9.5 scrub ok 2024-09-10T18:59:08.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:08 smithi018 ceph-mon[29336]: 9.5 scrub starts 2024-09-10T18:59:08.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:08 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:59:08.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:08 smithi018 ceph-mon[29336]: pgmap v634: 168 pgs: 168 active+clean; 1.5 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.48k op/s; 0 B/s, 0 objects/s recovering 2024-09-10T18:59:08.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:08 smithi018 ceph-mon[29336]: 9.5 scrub ok 2024-09-10T18:59:08.733 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:08.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:08.734 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:08.751 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-10T18:59:08.836 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:08.860 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:08.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:08.861 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:08.962 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:08.986 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:08.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:08.987 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:09.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:08 smithi103 ceph-mon[33163]: 9.5 scrub starts 2024-09-10T18:59:09.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:08 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:59:09.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:08 smithi103 ceph-mon[33163]: pgmap v634: 168 pgs: 168 active+clean; 1.5 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.48k op/s; 0 B/s, 0 objects/s recovering 2024-09-10T18:59:09.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:08 smithi103 ceph-mon[33163]: 9.5 scrub ok 2024-09-10T18:59:09.088 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:09.140 INFO:teuthology.orchestra.run.smithi018.stdout:ERROR: (22) Invalid argument 2024-09-10T18:59:09.140 INFO:teuthology.orchestra.run.smithi018.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:59:09.140 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:09.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:09.141 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:09.242 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:09.267 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:09.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:09.268 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:09.369 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:09.415 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:09.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:09.416 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:09.517 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:09.560 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:09.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:09.561 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:09.662 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:09.687 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:09.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:09.688 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:09.789 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:09.812 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:09.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:09.813 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:09.914 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:09.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:09 smithi018 ceph-mon[29336]: 2.d scrub starts 2024-09-10T18:59:09.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:09 smithi018 ceph-mon[29336]: 2.d scrub ok 2024-09-10T18:59:09.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:09 smithi018 ceph-mon[29336]: pgmap v635: 168 pgs: 168 active+clean; 1.5 GiB data, 8.3 GiB used, 528 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.32k op/s; 0 B/s, 0 objects/s recovering 2024-09-10T18:59:09.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:09 smithi018 ceph-mon[40196]: 2.d scrub starts 2024-09-10T18:59:09.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:09 smithi018 ceph-mon[40196]: 2.d scrub ok 2024-09-10T18:59:09.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:09 smithi018 ceph-mon[40196]: pgmap v635: 168 pgs: 168 active+clean; 1.5 GiB data, 8.3 GiB used, 528 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.32k op/s; 0 B/s, 0 objects/s recovering 2024-09-10T18:59:09.952 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:09.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:09.953 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:10.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:09 smithi103 ceph-mon[33163]: 2.d scrub starts 2024-09-10T18:59:10.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:09 smithi103 ceph-mon[33163]: 2.d scrub ok 2024-09-10T18:59:10.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:09 smithi103 ceph-mon[33163]: pgmap v635: 168 pgs: 168 active+clean; 1.5 GiB data, 8.3 GiB used, 528 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.32k op/s; 0 B/s, 0 objects/s recovering 2024-09-10T18:59:10.054 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:10.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:10.094 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:10.094 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:10.188 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-10T18:59:10.195 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:10.219 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:10.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:10.220 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:10.321 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:10.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:10.366 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:10.366 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:10.467 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:10.491 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:10.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:10.492 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:10.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:59:10 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:59:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:59:10.593 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:10.633 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:10.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:10.635 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:10.736 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:10.779 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:10.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:10.780 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:10.881 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:10.929 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:10.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:10.931 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:10.954 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:10 smithi018 ceph-mon[40196]: 9.f scrub starts 2024-09-10T18:59:10.954 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:10 smithi018 ceph-mon[40196]: 4.1f scrub starts 2024-09-10T18:59:10.954 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:10 smithi018 ceph-mon[40196]: 4.1f scrub ok 2024-09-10T18:59:10.954 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:10 smithi018 ceph-mon[40196]: 4.18 scrub starts 2024-09-10T18:59:10.954 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:10 smithi018 ceph-mon[40196]: 4.18 scrub ok 2024-09-10T18:59:10.954 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:10 smithi018 ceph-mon[40196]: 9.f scrub ok 2024-09-10T18:59:10.954 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:10 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:59:10.954 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:10 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:59:10.955 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:10 smithi018 ceph-mon[29336]: 9.f scrub starts 2024-09-10T18:59:10.955 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:10 smithi018 ceph-mon[29336]: 4.1f scrub starts 2024-09-10T18:59:10.955 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:10 smithi018 ceph-mon[29336]: 4.1f scrub ok 2024-09-10T18:59:10.955 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:10 smithi018 ceph-mon[29336]: 4.18 scrub starts 2024-09-10T18:59:10.955 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:10 smithi018 ceph-mon[29336]: 4.18 scrub ok 2024-09-10T18:59:10.955 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:10 smithi018 ceph-mon[29336]: 9.f scrub ok 2024-09-10T18:59:10.955 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:10 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:59:10.955 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:10 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:59:11.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:10 smithi103 ceph-mon[33163]: 9.f scrub starts 2024-09-10T18:59:11.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:10 smithi103 ceph-mon[33163]: 4.1f scrub starts 2024-09-10T18:59:11.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:10 smithi103 ceph-mon[33163]: 4.1f scrub ok 2024-09-10T18:59:11.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:10 smithi103 ceph-mon[33163]: 4.18 scrub starts 2024-09-10T18:59:11.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:10 smithi103 ceph-mon[33163]: 4.18 scrub ok 2024-09-10T18:59:11.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:10 smithi103 ceph-mon[33163]: 9.f scrub ok 2024-09-10T18:59:11.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:10 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:59:11.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:10 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:59:11.031 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:11.056 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:11.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:11.057 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:11.158 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:11.182 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:11.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:11.183 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:11.238 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:59:10 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:59:10] "GET /metrics HTTP/1.1" 200 224467 "" "Prometheus/2.43.0" 2024-09-10T18:59:11.284 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:11.308 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:11.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:11.309 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:11.410 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:11.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:11.436 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:11.436 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:11.537 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:11.563 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:11.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:11.565 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:11.665 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:11.695 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:11.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:11.697 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:11.797 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:11.838 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:11.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:11.839 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:11.940 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:11.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:11 smithi018 ceph-mon[29336]: 9.b scrub starts 2024-09-10T18:59:11.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:11 smithi018 ceph-mon[29336]: 9.b scrub ok 2024-09-10T18:59:11.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:11 smithi018 ceph-mon[29336]: 6.d scrub starts 2024-09-10T18:59:11.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:11 smithi018 ceph-mon[29336]: 6.d scrub ok 2024-09-10T18:59:11.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:11 smithi018 ceph-mon[29336]: 4.11 scrub starts 2024-09-10T18:59:11.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:11 smithi018 ceph-mon[29336]: 4.11 scrub ok 2024-09-10T18:59:11.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:11 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2137281217' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:59:11.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:11 smithi018 ceph-mon[29336]: pgmap v636: 168 pgs: 168 active+clean; 1.4 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.33k op/s; 0 B/s, 0 objects/s recovering 2024-09-10T18:59:11.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:11 smithi018 ceph-mon[40196]: 9.b scrub starts 2024-09-10T18:59:11.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:11 smithi018 ceph-mon[40196]: 9.b scrub ok 2024-09-10T18:59:11.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:11 smithi018 ceph-mon[40196]: 6.d scrub starts 2024-09-10T18:59:11.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:11 smithi018 ceph-mon[40196]: 6.d scrub ok 2024-09-10T18:59:11.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:11 smithi018 ceph-mon[40196]: 4.11 scrub starts 2024-09-10T18:59:11.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:11 smithi018 ceph-mon[40196]: 4.11 scrub ok 2024-09-10T18:59:11.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:11 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2137281217' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:59:11.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:11 smithi018 ceph-mon[40196]: pgmap v636: 168 pgs: 168 active+clean; 1.4 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.33k op/s; 0 B/s, 0 objects/s recovering 2024-09-10T18:59:11.978 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:11.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:11.979 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:12.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:11 smithi103 ceph-mon[33163]: 9.b scrub starts 2024-09-10T18:59:12.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:11 smithi103 ceph-mon[33163]: 9.b scrub ok 2024-09-10T18:59:12.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:11 smithi103 ceph-mon[33163]: 6.d scrub starts 2024-09-10T18:59:12.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:11 smithi103 ceph-mon[33163]: 6.d scrub ok 2024-09-10T18:59:12.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:11 smithi103 ceph-mon[33163]: 4.11 scrub starts 2024-09-10T18:59:12.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:11 smithi103 ceph-mon[33163]: 4.11 scrub ok 2024-09-10T18:59:12.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:11 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2137281217' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:59:12.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:11 smithi103 ceph-mon[33163]: pgmap v636: 168 pgs: 168 active+clean; 1.4 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.33k op/s; 0 B/s, 0 objects/s recovering 2024-09-10T18:59:12.080 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:12.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:12.107 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:12.107 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:12.208 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:12.247 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:12.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:12.249 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:12.349 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:12.375 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:12.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:12.376 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:12.477 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:12.513 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:12.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:12.514 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:12.539 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:59:12.540 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:59:12.540 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:59:12.540 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:59:12.540 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:59:12.540 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:59:12.540 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:59:12.541 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:59:12.541 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:59:12.541 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:59:12.541 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:59:12.541 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:59:12.541 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:59:12.541 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:59:12.541 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:59:12.541 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:59:12.615 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:12.636 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:12.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:12.637 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:12.738 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:12.780 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:12.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:12.781 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:12.882 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:12.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:12.905 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:12.905 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:12.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:12 smithi018 ceph-mon[40196]: 1.a scrub starts 2024-09-10T18:59:12.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:12 smithi018 ceph-mon[40196]: 1.a scrub ok 2024-09-10T18:59:12.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:12 smithi018 ceph-mon[40196]: 5.9 scrub starts 2024-09-10T18:59:12.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:12 smithi018 ceph-mon[40196]: 5.9 scrub ok 2024-09-10T18:59:12.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:12 smithi018 ceph-mon[40196]: 4.1b scrub starts 2024-09-10T18:59:12.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:12 smithi018 ceph-mon[40196]: 4.1b scrub ok 2024-09-10T18:59:12.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:12 smithi018 ceph-mon[40196]: 6.b scrub starts 2024-09-10T18:59:12.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:12 smithi018 ceph-mon[40196]: 6.b scrub ok 2024-09-10T18:59:12.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:12 smithi018 ceph-mon[29336]: 1.a scrub starts 2024-09-10T18:59:12.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:12 smithi018 ceph-mon[29336]: 1.a scrub ok 2024-09-10T18:59:12.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:12 smithi018 ceph-mon[29336]: 5.9 scrub starts 2024-09-10T18:59:12.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:12 smithi018 ceph-mon[29336]: 5.9 scrub ok 2024-09-10T18:59:12.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:12 smithi018 ceph-mon[29336]: 4.1b scrub starts 2024-09-10T18:59:12.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:12 smithi018 ceph-mon[29336]: 4.1b scrub ok 2024-09-10T18:59:12.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:12 smithi018 ceph-mon[29336]: 6.b scrub starts 2024-09-10T18:59:12.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:12 smithi018 ceph-mon[29336]: 6.b scrub ok 2024-09-10T18:59:13.006 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:13.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:12 smithi103 ceph-mon[33163]: 1.a scrub starts 2024-09-10T18:59:13.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:12 smithi103 ceph-mon[33163]: 1.a scrub ok 2024-09-10T18:59:13.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:12 smithi103 ceph-mon[33163]: 5.9 scrub starts 2024-09-10T18:59:13.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:12 smithi103 ceph-mon[33163]: 5.9 scrub ok 2024-09-10T18:59:13.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:12 smithi103 ceph-mon[33163]: 4.1b scrub starts 2024-09-10T18:59:13.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:12 smithi103 ceph-mon[33163]: 4.1b scrub ok 2024-09-10T18:59:13.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:12 smithi103 ceph-mon[33163]: 6.b scrub starts 2024-09-10T18:59:13.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:12 smithi103 ceph-mon[33163]: 6.b scrub ok 2024-09-10T18:59:13.032 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:13.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:13.033 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:13.134 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:13.157 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:13.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:13.158 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:13.259 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:13.298 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:13.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:13.299 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:13.400 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:13.436 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:13.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:13.437 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:13.541 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:13.582 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:13.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:13.583 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:13.683 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:13.720 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:13.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:13.721 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:13.822 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:13.848 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:13.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:13.849 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:13.950 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:13.974 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:13.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:13.975 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:14.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:13 smithi103 ceph-mon[33163]: 1.2 scrub starts 2024-09-10T18:59:14.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:13 smithi103 ceph-mon[33163]: 1.2 scrub ok 2024-09-10T18:59:14.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:13 smithi103 ceph-mon[33163]: pgmap v637: 168 pgs: 168 active+clean; 1.1 GiB data, 7.2 GiB used, 529 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.36k op/s 2024-09-10T18:59:14.076 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:14.112 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:14.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:14.114 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:14.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:13 smithi018 ceph-mon[40196]: 1.2 scrub starts 2024-09-10T18:59:14.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:13 smithi018 ceph-mon[40196]: 1.2 scrub ok 2024-09-10T18:59:14.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:13 smithi018 ceph-mon[40196]: pgmap v637: 168 pgs: 168 active+clean; 1.1 GiB data, 7.2 GiB used, 529 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.36k op/s 2024-09-10T18:59:14.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:13 smithi018 ceph-mon[29336]: 1.2 scrub starts 2024-09-10T18:59:14.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:13 smithi018 ceph-mon[29336]: 1.2 scrub ok 2024-09-10T18:59:14.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:13 smithi018 ceph-mon[29336]: pgmap v637: 168 pgs: 168 active+clean; 1.1 GiB data, 7.2 GiB used, 529 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.36k op/s 2024-09-10T18:59:14.215 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:14.239 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:14.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:14.240 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:14.340 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:14.383 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:14.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:14.384 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:14.485 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:14.509 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:14.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:14.510 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:14.611 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:14.646 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:14.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:14.647 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:14.747 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:14.784 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:14.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:14.785 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:14.886 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:14.927 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:14.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:14.928 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:15.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:14 smithi103 ceph-mon[33163]: 6.c scrub starts 2024-09-10T18:59:15.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:14 smithi103 ceph-mon[33163]: 6.c scrub ok 2024-09-10T18:59:15.028 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:15.054 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:15.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:15.055 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:15.155 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:15.194 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:15.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:14 smithi018 ceph-mon[29336]: 6.c scrub starts 2024-09-10T18:59:15.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:14 smithi018 ceph-mon[29336]: 6.c scrub ok 2024-09-10T18:59:15.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:14 smithi018 ceph-mon[40196]: 6.c scrub starts 2024-09-10T18:59:15.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:14 smithi018 ceph-mon[40196]: 6.c scrub ok 2024-09-10T18:59:15.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:15.195 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:15.296 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:15.332 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:15.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:15.334 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:15.434 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:15.460 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:15.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:15.461 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:15.562 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:15.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:15.603 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:15.603 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:15.704 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:15.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:15.741 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:15.742 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:15.842 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:15.889 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T18:59:15.892 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-09-10T18:59:15.912 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:15.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:15.913 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:16.014 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:16.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:15 smithi103 ceph-mon[33163]: 9.d scrub starts 2024-09-10T18:59:16.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:15 smithi103 ceph-mon[33163]: pgmap v638: 168 pgs: 168 active+clean; 1.0 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 0 B/s rd, 0 B/s wr, 1.25k op/s 2024-09-10T18:59:16.047 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:16.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:16.049 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:16.150 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:16.189 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:16.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:16.190 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:16.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:15 smithi018 ceph-mon[40196]: 9.d scrub starts 2024-09-10T18:59:16.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:15 smithi018 ceph-mon[40196]: pgmap v638: 168 pgs: 168 active+clean; 1.0 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 0 B/s rd, 0 B/s wr, 1.25k op/s 2024-09-10T18:59:16.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:15 smithi018 ceph-mon[29336]: 9.d scrub starts 2024-09-10T18:59:16.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:15 smithi018 ceph-mon[29336]: pgmap v638: 168 pgs: 168 active+clean; 1.0 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 0 B/s rd, 0 B/s wr, 1.25k op/s 2024-09-10T18:59:16.291 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:16.340 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:16.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:16.341 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:16.442 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:16.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:16.487 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:16.487 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:16.588 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:16.624 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:16.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:16.625 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:16.726 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:16.750 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:16.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:16.752 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:16.852 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:16.877 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:16.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:16.877 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:16.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:16 smithi018 ceph-mon[40196]: 1.0 scrub starts 2024-09-10T18:59:16.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:16 smithi018 ceph-mon[40196]: 1.0 scrub ok 2024-09-10T18:59:16.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:16 smithi018 ceph-mon[40196]: 5.2 scrub starts 2024-09-10T18:59:16.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:16 smithi018 ceph-mon[40196]: 5.2 scrub ok 2024-09-10T18:59:16.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:16 smithi018 ceph-mon[40196]: 9.d scrub ok 2024-09-10T18:59:16.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:16 smithi018 ceph-mon[40196]: from='client.25391 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:59:16.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:16 smithi018 ceph-mon[40196]: 9.1 scrub starts 2024-09-10T18:59:16.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:16 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:59:16.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:16 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:59:16.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:16 smithi018 ceph-mon[40196]: 6.1 scrub starts 2024-09-10T18:59:16.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:16 smithi018 ceph-mon[40196]: 6.1 scrub ok 2024-09-10T18:59:16.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:16 smithi018 ceph-mon[40196]: 9.1 scrub ok 2024-09-10T18:59:16.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:16 smithi018 ceph-mon[29336]: 1.0 scrub starts 2024-09-10T18:59:16.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:16 smithi018 ceph-mon[29336]: 1.0 scrub ok 2024-09-10T18:59:16.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:16 smithi018 ceph-mon[29336]: 5.2 scrub starts 2024-09-10T18:59:16.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:16 smithi018 ceph-mon[29336]: 5.2 scrub ok 2024-09-10T18:59:16.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:16 smithi018 ceph-mon[29336]: 9.d scrub ok 2024-09-10T18:59:16.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:16 smithi018 ceph-mon[29336]: from='client.25391 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:59:16.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:16 smithi018 ceph-mon[29336]: 9.1 scrub starts 2024-09-10T18:59:16.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:16 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:59:16.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:16 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:59:16.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:16 smithi018 ceph-mon[29336]: 6.1 scrub starts 2024-09-10T18:59:16.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:16 smithi018 ceph-mon[29336]: 6.1 scrub ok 2024-09-10T18:59:16.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:16 smithi018 ceph-mon[29336]: 9.1 scrub ok 2024-09-10T18:59:16.978 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:16.999 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:59:17.001 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:17.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:17.002 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:17.022 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:59:16 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:59:17.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:16 smithi103 ceph-mon[33163]: 1.0 scrub starts 2024-09-10T18:59:17.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:16 smithi103 ceph-mon[33163]: 1.0 scrub ok 2024-09-10T18:59:17.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:16 smithi103 ceph-mon[33163]: 5.2 scrub starts 2024-09-10T18:59:17.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:16 smithi103 ceph-mon[33163]: 5.2 scrub ok 2024-09-10T18:59:17.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:16 smithi103 ceph-mon[33163]: 9.d scrub ok 2024-09-10T18:59:17.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:16 smithi103 ceph-mon[33163]: from='client.25391 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:59:17.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:16 smithi103 ceph-mon[33163]: 9.1 scrub starts 2024-09-10T18:59:17.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:16 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:59:17.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:16 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:59:17.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:16 smithi103 ceph-mon[33163]: 6.1 scrub starts 2024-09-10T18:59:17.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:16 smithi103 ceph-mon[33163]: 6.1 scrub ok 2024-09-10T18:59:17.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:16 smithi103 ceph-mon[33163]: 9.1 scrub ok 2024-09-10T18:59:17.102 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:17.152 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:17.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:17.153 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:17.254 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:17.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:17.298 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:17.298 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:17.399 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:17.436 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:17.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:17.437 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:17.538 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:17.563 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:59:17.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:17.597 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:17.597 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:17.612 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":638,"stamp":"2024-09-10T18:59:15.345929+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1122146638,"num_objects":17309,"num_object_clones":0,"num_object_copies":51927,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17309,"num_whiteouts":0,"num_read":1985,"num_read_kb":1754,"num_write":257121,"num_write_kb":8752251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75945,"num_bytes_recovered":4962587556,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":61043,"ondisk_log_size":61043,"up":504,"acting":504,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":504,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":562470912,"kb_used":7396248,"kb_used_data":3475112,"kb_used_omap":12,"kb_used_meta":3921075,"kb_avail":555074664,"statfs":{"total":575970213888,"available":568396455936,"internally_reserved":0,"allocated":3558514688,"data_stored":3555952778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13122,"internal_metadata":4015180990},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,8],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-887711955,"num_objects":-13545,"num_object_clones":0,"num_object_copies":-40635,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-13545,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":13540,"num_write_kb":-74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":-53670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.829159"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":311,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.494648+0000","last_change":"2024-09-10T18:57:02.462265+0000","last_active":"2024-09-10T18:58:58.494648+0000","last_peered":"2024-09-10T18:58:58.494648+0000","last_clean":"2024-09-10T18:58:58.494648+0000","last_became_active":"2024-09-10T18:57:02.460451+0000","last_became_peered":"2024-09-10T18:57:02.460451+0000","last_unstale":"2024-09-10T18:58:58.494648+0000","last_undegraded":"2024-09-10T18:58:58.494648+0000","last_fullsized":"2024-09-10T18:58:58.494648+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:14.875872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:14.875872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037478699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":295,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.498343+0000","last_change":"2024-09-10T18:57:02.442207+0000","last_active":"2024-09-10T18:58:58.498343+0000","last_peered":"2024-09-10T18:58:58.498343+0000","last_clean":"2024-09-10T18:58:58.498343+0000","last_became_active":"2024-09-10T18:57:02.440648+0000","last_became_peered":"2024-09-10T18:57:02.440648+0000","last_unstale":"2024-09-10T18:58:58.498343+0000","last_undegraded":"2024-09-10T18:58:58.498343+0000","last_fullsized":"2024-09-10T18:58:58.498343+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:31.221724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:56:31.221724+0000","last_clean_scrub_stamp":"2024-09-10T18:56:31.221724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051032730000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"232'27","reported_seq":396,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.494032+0000","last_change":"2024-09-10T18:56:02.821637+0000","last_active":"2024-09-10T18:58:58.494032+0000","last_peered":"2024-09-10T18:58:58.494032+0000","last_clean":"2024-09-10T18:58:58.494032+0000","last_became_active":"2024-09-10T18:55:40.394717+0000","last_became_peered":"2024-09-10T18:55:40.394717+0000","last_unstale":"2024-09-10T18:58:58.494032+0000","last_undegraded":"2024-09-10T18:58:58.494032+0000","last_fullsized":"2024-09-10T18:58:58.494032+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'27","last_scrub_stamp":"2024-09-10T18:56:02.821551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:02.821551+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0083110159999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":301,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.482771+0000","last_change":"2024-09-10T18:55:33.526359+0000","last_active":"2024-09-10T18:58:58.482771+0000","last_peered":"2024-09-10T18:58:58.482771+0000","last_clean":"2024-09-10T18:58:58.482771+0000","last_became_active":"2024-09-10T18:45:22.065619+0000","last_became_peered":"2024-09-10T18:45:22.065619+0000","last_unstale":"2024-09-10T18:58:58.482771+0000","last_undegraded":"2024-09-10T18:58:58.482771+0000","last_fullsized":"2024-09-10T18:58:58.482771+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:33.526302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.057932+0000","last_clean_scrub_stamp":"2024-09-10T18:55:33.526302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001517556,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":344,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.496543+0000","last_change":"2024-09-10T18:57:02.457742+0000","last_active":"2024-09-10T18:58:58.496543+0000","last_peered":"2024-09-10T18:58:58.496543+0000","last_clean":"2024-09-10T18:58:58.496543+0000","last_became_active":"2024-09-10T18:57:02.457322+0000","last_became_peered":"2024-09-10T18:57:02.457322+0000","last_unstale":"2024-09-10T18:58:58.496543+0000","last_undegraded":"2024-09-10T18:58:58.496543+0000","last_fullsized":"2024-09-10T18:58:58.496543+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:23.885562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:23.885562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0031033800000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"232'19","reported_seq":332,"reported_epoch":305,"state":"active+clean","last_fresh":"2024-09-10T18:59:09.642792+0000","last_change":"2024-09-10T18:59:09.642743+0000","last_active":"2024-09-10T18:59:09.642792+0000","last_peered":"2024-09-10T18:59:09.642792+0000","last_clean":"2024-09-10T18:59:09.642792+0000","last_became_active":"2024-09-10T18:45:20.061266+0000","last_became_peered":"2024-09-10T18:45:20.061266+0000","last_unstale":"2024-09-10T18:59:09.642792+0000","last_undegraded":"2024-09-10T18:59:09.642792+0000","last_fullsized":"2024-09-10T18:59:09.642792+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T18:59:09.642642+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_clean_scrub_stamp":"2024-09-10T18:59:09.642642+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:31.669430+0000","scrub_duration":0.0095920940000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":44,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.490178+0000","last_change":"2024-09-10T18:57:02.446964+0000","last_active":"2024-09-10T18:58:58.490178+0000","last_peered":"2024-09-10T18:58:58.490178+0000","last_clean":"2024-09-10T18:58:58.490178+0000","last_became_active":"2024-09-10T18:57:02.446008+0000","last_became_peered":"2024-09-10T18:57:02.446008+0000","last_unstale":"2024-09-10T18:58:58.490178+0000","last_undegraded":"2024-09-10T18:58:58.490178+0000","last_fullsized":"2024-09-10T18:58:58.490178+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:22.163121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:01.135988+0000","last_clean_scrub_stamp":"2024-09-10T18:56:22.163121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002631478,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":299,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.483322+0000","last_change":"2024-09-10T18:58:51.682046+0000","last_active":"2024-09-10T18:58:58.483322+0000","last_peered":"2024-09-10T18:58:58.483322+0000","last_clean":"2024-09-10T18:58:58.483322+0000","last_became_active":"2024-09-10T18:45:24.081002+0000","last_became_peered":"2024-09-10T18:45:24.081002+0000","last_unstale":"2024-09-10T18:58:58.483322+0000","last_undegraded":"2024-09-10T18:58:58.483322+0000","last_fullsized":"2024-09-10T18:58:58.483322+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:51.681994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:58:51.681994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:16.065023+0000","scrub_duration":0.0051776249999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":293,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.483723+0000","last_change":"2024-09-10T18:57:02.452754+0000","last_active":"2024-09-10T18:58:58.483723+0000","last_peered":"2024-09-10T18:58:58.483723+0000","last_clean":"2024-09-10T18:58:58.483723+0000","last_became_active":"2024-09-10T18:57:02.452259+0000","last_became_peered":"2024-09-10T18:57:02.452259+0000","last_unstale":"2024-09-10T18:58:58.483723+0000","last_undegraded":"2024-09-10T18:58:58.483723+0000","last_fullsized":"2024-09-10T18:58:58.483723+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:06.064071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:06.064071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037631460000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1b","version":"232'9","reported_seq":308,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.483832+0000","last_change":"2024-09-10T18:54:45.305098+0000","last_active":"2024-09-10T18:58:58.483832+0000","last_peered":"2024-09-10T18:58:58.483832+0000","last_clean":"2024-09-10T18:58:58.483832+0000","last_became_active":"2024-09-10T18:45:20.057567+0000","last_became_peered":"2024-09-10T18:45:20.057567+0000","last_unstale":"2024-09-10T18:58:58.483832+0000","last_undegraded":"2024-09-10T18:58:58.483832+0000","last_fullsized":"2024-09-10T18:58:58.483832+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"163'7","last_scrub_stamp":"2024-09-10T18:54:45.305023+0000","last_deep_scrub":"163'7","last_deep_scrub_stamp":"2024-09-10T18:51:54.261879+0000","last_clean_scrub_stamp":"2024-09-10T18:54:45.305023+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0088188659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":57,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.504530+0000","last_change":"2024-09-10T18:58:58.504293+0000","last_active":"2024-09-10T18:58:58.504530+0000","last_peered":"2024-09-10T18:58:58.504530+0000","last_clean":"2024-09-10T18:58:58.504530+0000","last_became_active":"2024-09-10T18:58:58.503924+0000","last_became_peered":"2024-09-10T18:58:58.503924+0000","last_unstale":"2024-09-10T18:58:58.504530+0000","last_undegraded":"2024-09-10T18:58:58.504530+0000","last_fullsized":"2024-09-10T18:58:58.504530+0000","mapping_epoch":303,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":304,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:27.785914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:36.509971+0000","last_clean_scrub_stamp":"2024-09-10T18:55:27.785914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":160,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.490374+0000","last_change":"2024-09-10T18:56:07.553751+0000","last_active":"2024-09-10T18:58:58.490374+0000","last_peered":"2024-09-10T18:58:58.490374+0000","last_clean":"2024-09-10T18:58:58.490374+0000","last_became_active":"2024-09-10T18:55:40.399236+0000","last_became_peered":"2024-09-10T18:55:40.399236+0000","last_unstale":"2024-09-10T18:58:58.490374+0000","last_undegraded":"2024-09-10T18:58:58.490374+0000","last_fullsized":"2024-09-10T18:58:58.490374+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:07.553696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:07.553696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0091042339999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":110,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.485832+0000","last_change":"2024-09-10T18:58:49.506212+0000","last_active":"2024-09-10T18:58:58.485832+0000","last_peered":"2024-09-10T18:58:58.485832+0000","last_clean":"2024-09-10T18:58:58.485832+0000","last_became_active":"2024-09-10T18:57:02.440497+0000","last_became_peered":"2024-09-10T18:57:02.440497+0000","last_unstale":"2024-09-10T18:58:58.485832+0000","last_undegraded":"2024-09-10T18:58:58.485832+0000","last_fullsized":"2024-09-10T18:58:58.485832+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:49.506160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:58:49.506160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:06.899561+0000","scrub_duration":0.018244065,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"232'15","reported_seq":354,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.483526+0000","last_change":"2024-09-10T18:56:26.480197+0000","last_active":"2024-09-10T18:58:58.483526+0000","last_peered":"2024-09-10T18:58:58.483526+0000","last_clean":"2024-09-10T18:58:58.483526+0000","last_became_active":"2024-09-10T18:55:40.391571+0000","last_became_peered":"2024-09-10T18:55:40.391571+0000","last_unstale":"2024-09-10T18:58:58.483526+0000","last_undegraded":"2024-09-10T18:58:58.483526+0000","last_fullsized":"2024-09-10T18:58:58.483526+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T18:56:26.480105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:26.480105+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027886920000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":330,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.485485+0000","last_change":"2024-09-10T18:57:02.431878+0000","last_active":"2024-09-10T18:58:58.485485+0000","last_peered":"2024-09-10T18:58:58.485485+0000","last_clean":"2024-09-10T18:58:58.485485+0000","last_became_active":"2024-09-10T18:57:02.431585+0000","last_became_peered":"2024-09-10T18:57:02.431585+0000","last_unstale":"2024-09-10T18:58:58.485485+0000","last_undegraded":"2024-09-10T18:58:58.485485+0000","last_fullsized":"2024-09-10T18:58:58.485485+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:00.729699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:00.729699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047811370000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":270,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.488963+0000","last_change":"2024-09-10T18:58:51.561127+0000","last_active":"2024-09-10T18:58:58.488963+0000","last_peered":"2024-09-10T18:58:58.488963+0000","last_clean":"2024-09-10T18:58:58.488963+0000","last_became_active":"2024-09-10T18:57:02.442905+0000","last_became_peered":"2024-09-10T18:57:02.442905+0000","last_unstale":"2024-09-10T18:58:58.488963+0000","last_undegraded":"2024-09-10T18:58:58.488963+0000","last_fullsized":"2024-09-10T18:58:58.488963+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:51.561010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:59.425718+0000","last_clean_scrub_stamp":"2024-09-10T18:58:51.561010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:59:55.288796+0000","scrub_duration":0.032561917000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"305'16186","reported_seq":16269,"reported_epoch":305,"state":"active+clean","last_fresh":"2024-09-10T18:59:11.731244+0000","last_change":"2024-09-10T18:55:40.403238+0000","last_active":"2024-09-10T18:59:11.731244+0000","last_peered":"2024-09-10T18:59:11.731244+0000","last_clean":"2024-09-10T18:59:11.731244+0000","last_became_active":"2024-09-10T18:55:40.402912+0000","last_became_peered":"2024-09-10T18:55:40.402912+0000","last_unstale":"2024-09-10T18:59:11.731244+0000","last_undegraded":"2024-09-10T18:59:11.731244+0000","last_fullsized":"2024-09-10T18:59:11.731244+0000","mapping_epoch":241,"log_start":"296'13014","ondisk_log_start":"296'13014","created":237,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3172,"log_dups_size":0,"ondisk_log_size":3172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70647808,"num_objects":1078,"num_object_clones":0,"num_object_copies":3234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16186,"num_write_kb":552448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"70'81","reported_seq":151,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.496593+0000","last_change":"2024-09-10T18:55:31.654161+0000","last_active":"2024-09-10T18:58:58.496593+0000","last_peered":"2024-09-10T18:58:58.496593+0000","last_clean":"2024-09-10T18:58:58.496593+0000","last_became_active":"2024-09-10T18:52:54.619039+0000","last_became_peered":"2024-09-10T18:52:54.619039+0000","last_unstale":"2024-09-10T18:58:58.496593+0000","last_undegraded":"2024-09-10T18:58:58.496593+0000","last_fullsized":"2024-09-10T18:58:58.496593+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:55:31.654090+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:55:31.654090+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025082630000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"232'39","reported_seq":388,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.485158+0000","last_change":"2024-09-10T18:55:25.483970+0000","last_active":"2024-09-10T18:58:58.485158+0000","last_peered":"2024-09-10T18:58:58.485158+0000","last_clean":"2024-09-10T18:58:58.485158+0000","last_became_active":"2024-09-10T18:47:36.409300+0000","last_became_peered":"2024-09-10T18:47:36.409300+0000","last_unstale":"2024-09-10T18:58:58.485158+0000","last_undegraded":"2024-09-10T18:58:58.485158+0000","last_fullsized":"2024-09-10T18:58:58.485158+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'27","last_scrub_stamp":"2024-09-10T18:55:25.483919+0000","last_deep_scrub":"78'25","last_deep_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_clean_scrub_stamp":"2024-09-10T18:55:25.483919+0000","objects_scrubbed":8,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017172610000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":47,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.483930+0000","last_change":"2024-09-10T18:57:02.437548+0000","last_active":"2024-09-10T18:58:58.483930+0000","last_peered":"2024-09-10T18:58:58.483930+0000","last_clean":"2024-09-10T18:58:58.483930+0000","last_became_active":"2024-09-10T18:57:02.437260+0000","last_became_peered":"2024-09-10T18:57:02.437260+0000","last_unstale":"2024-09-10T18:58:58.483930+0000","last_undegraded":"2024-09-10T18:58:58.483930+0000","last_fullsized":"2024-09-10T18:58:58.483930+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:30.719475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:55:30.719475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":310,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.492741+0000","last_change":"2024-09-10T18:56:09.623123+0000","last_active":"2024-09-10T18:58:58.492741+0000","last_peered":"2024-09-10T18:58:58.492741+0000","last_clean":"2024-09-10T18:58:58.492741+0000","last_became_active":"2024-09-10T18:55:40.392449+0000","last_became_peered":"2024-09-10T18:55:40.392449+0000","last_unstale":"2024-09-10T18:58:58.492741+0000","last_undegraded":"2024-09-10T18:58:58.492741+0000","last_fullsized":"2024-09-10T18:58:58.492741+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:09.622841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:04.512425+0000","last_clean_scrub_stamp":"2024-09-10T18:56:09.622841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010581503000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":217,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.498267+0000","last_change":"2024-09-10T18:58:49.524134+0000","last_active":"2024-09-10T18:58:58.498267+0000","last_peered":"2024-09-10T18:58:58.498267+0000","last_clean":"2024-09-10T18:58:58.498267+0000","last_became_active":"2024-09-10T18:55:40.388154+0000","last_became_peered":"2024-09-10T18:55:40.388154+0000","last_unstale":"2024-09-10T18:58:58.498267+0000","last_undegraded":"2024-09-10T18:58:58.498267+0000","last_fullsized":"2024-09-10T18:58:58.498267+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:49.524015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:58:49.524015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:16.118448+0000","scrub_duration":0.005180479,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1","version":"305'15966","reported_seq":27571,"reported_epoch":305,"state":"active+clean","last_fresh":"2024-09-10T18:59:11.932687+0000","last_change":"2024-09-10T18:58:24.432220+0000","last_active":"2024-09-10T18:59:11.932687+0000","last_peered":"2024-09-10T18:59:11.932687+0000","last_clean":"2024-09-10T18:59:11.932687+0000","last_became_active":"2024-09-10T18:58:24.405312+0000","last_became_peered":"2024-09-10T18:58:24.405312+0000","last_unstale":"2024-09-10T18:59:11.932687+0000","last_undegraded":"2024-09-10T18:59:11.932687+0000","last_fullsized":"2024-09-10T18:59:11.932687+0000","mapping_epoch":294,"log_start":"296'12624","ondisk_log_start":"296'12624","created":237,"last_epoch_clean":295,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3342,"log_dups_size":0,"ondisk_log_size":3342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67895296,"num_objects":1036,"num_object_clones":0,"num_object_copies":3108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1036,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15966,"num_write_kb":544064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5750,"num_bytes_recovered":376832000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.9","version":"70'81","reported_seq":205,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.494384+0000","last_change":"2024-09-10T18:57:02.465293+0000","last_active":"2024-09-10T18:58:58.494384+0000","last_peered":"2024-09-10T18:58:58.494384+0000","last_clean":"2024-09-10T18:58:58.494384+0000","last_became_active":"2024-09-10T18:57:02.463750+0000","last_became_peered":"2024-09-10T18:57:02.463750+0000","last_unstale":"2024-09-10T18:58:58.494384+0000","last_undegraded":"2024-09-10T18:58:58.494384+0000","last_fullsized":"2024-09-10T18:58:58.494384+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:54:45.753792+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:54:45.753792+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016709736999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":21,"num_write":10,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":107339,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"232'25","reported_seq":338,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.484700+0000","last_change":"2024-09-10T18:56:32.437605+0000","last_active":"2024-09-10T18:58:58.484700+0000","last_peered":"2024-09-10T18:58:58.484700+0000","last_clean":"2024-09-10T18:58:58.484700+0000","last_became_active":"2024-09-10T18:45:20.053428+0000","last_became_peered":"2024-09-10T18:45:20.053428+0000","last_unstale":"2024-09-10T18:58:58.484700+0000","last_undegraded":"2024-09-10T18:58:58.484700+0000","last_fullsized":"2024-09-10T18:58:58.484700+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'25","last_scrub_stamp":"2024-09-10T18:56:32.437503+0000","last_deep_scrub":"116'15","last_deep_scrub_stamp":"2024-09-10T18:49:12.374527+0000","last_clean_scrub_stamp":"2024-09-10T18:56:32.437503+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039673679999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":317,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.492331+0000","last_change":"2024-09-10T18:57:02.441472+0000","last_active":"2024-09-10T18:58:58.492331+0000","last_peered":"2024-09-10T18:58:58.492331+0000","last_clean":"2024-09-10T18:58:58.492331+0000","last_became_active":"2024-09-10T18:57:02.440442+0000","last_became_peered":"2024-09-10T18:57:02.440442+0000","last_unstale":"2024-09-10T18:58:58.492331+0000","last_undegraded":"2024-09-10T18:58:58.492331+0000","last_fullsized":"2024-09-10T18:58:58.492331+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:29.802937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:29.802937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044818080000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"273'13","reported_seq":334,"reported_epoch":305,"state":"active+clean","last_fresh":"2024-09-10T18:59:06.238599+0000","last_change":"2024-09-10T18:57:02.463853+0000","last_active":"2024-09-10T18:59:06.238599+0000","last_peered":"2024-09-10T18:59:06.238599+0000","last_clean":"2024-09-10T18:59:06.238599+0000","last_became_active":"2024-09-10T18:57:02.448258+0000","last_became_peered":"2024-09-10T18:57:02.448258+0000","last_unstale":"2024-09-10T18:59:06.238599+0000","last_undegraded":"2024-09-10T18:59:06.238599+0000","last_fullsized":"2024-09-10T18:59:06.238599+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"242'12","last_scrub_stamp":"2024-09-10T18:56:27.995682+0000","last_deep_scrub":"202'10","last_deep_scrub_stamp":"2024-09-10T18:53:45.547686+0000","last_clean_scrub_stamp":"2024-09-10T18:56:27.995682+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025134200000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":329,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.485506+0000","last_change":"2024-09-10T18:57:02.440361+0000","last_active":"2024-09-10T18:58:58.485506+0000","last_peered":"2024-09-10T18:58:58.485506+0000","last_clean":"2024-09-10T18:58:58.485506+0000","last_became_active":"2024-09-10T18:57:02.439477+0000","last_became_peered":"2024-09-10T18:57:02.439477+0000","last_unstale":"2024-09-10T18:58:58.485506+0000","last_undegraded":"2024-09-10T18:58:58.485506+0000","last_fullsized":"2024-09-10T18:58:58.485506+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:37.565359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:32.043666+0000","last_clean_scrub_stamp":"2024-09-10T18:55:37.565359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038365510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"305'15947","reported_seq":30616,"reported_epoch":305,"state":"active+clean","last_fresh":"2024-09-10T18:59:11.911659+0000","last_change":"2024-09-10T18:58:02.162234+0000","last_active":"2024-09-10T18:59:11.911659+0000","last_peered":"2024-09-10T18:59:11.911659+0000","last_clean":"2024-09-10T18:59:11.911659+0000","last_became_active":"2024-09-10T18:58:02.141866+0000","last_became_peered":"2024-09-10T18:58:02.141866+0000","last_unstale":"2024-09-10T18:59:11.911659+0000","last_undegraded":"2024-09-10T18:59:11.911659+0000","last_fullsized":"2024-09-10T18:59:11.911659+0000","mapping_epoch":284,"log_start":"295'12603","ondisk_log_start":"295'12603","created":237,"last_epoch_clean":285,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3344,"log_dups_size":0,"ondisk_log_size":3344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":72417280,"num_objects":1105,"num_object_clones":0,"num_object_copies":3315,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15947,"num_write_kb":545664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7284,"num_bytes_recovered":477364224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.a","version":"70'81","reported_seq":218,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.485516+0000","last_change":"2024-09-10T18:55:37.921571+0000","last_active":"2024-09-10T18:58:58.485516+0000","last_peered":"2024-09-10T18:58:58.485516+0000","last_clean":"2024-09-10T18:58:58.485516+0000","last_became_active":"2024-09-10T18:50:08.398345+0000","last_became_peered":"2024-09-10T18:50:08.398345+0000","last_unstale":"2024-09-10T18:58:58.485516+0000","last_undegraded":"2024-09-10T18:58:58.485516+0000","last_fullsized":"2024-09-10T18:58:58.485516+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:55:37.921519+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:55:37.921519+0000","last_clean_scrub_stamp":"2024-09-10T18:55:37.921519+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002222407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"232'34","reported_seq":322,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.486939+0000","last_change":"2024-09-10T18:56:22.727738+0000","last_active":"2024-09-10T18:58:58.486939+0000","last_peered":"2024-09-10T18:58:58.486939+0000","last_clean":"2024-09-10T18:58:58.486939+0000","last_became_active":"2024-09-10T18:46:56.742946+0000","last_became_peered":"2024-09-10T18:46:56.742946+0000","last_unstale":"2024-09-10T18:58:58.486939+0000","last_undegraded":"2024-09-10T18:58:58.486939+0000","last_fullsized":"2024-09-10T18:58:58.486939+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'34","last_scrub_stamp":"2024-09-10T18:56:22.727671+0000","last_deep_scrub":"141'22","last_deep_scrub_stamp":"2024-09-10T18:50:57.119152+0000","last_clean_scrub_stamp":"2024-09-10T18:56:22.727671+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0079020229999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":320,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.490588+0000","last_change":"2024-09-10T18:57:02.447072+0000","last_active":"2024-09-10T18:58:58.490588+0000","last_peered":"2024-09-10T18:58:58.490588+0000","last_clean":"2024-09-10T18:58:58.490588+0000","last_became_active":"2024-09-10T18:57:02.445848+0000","last_became_peered":"2024-09-10T18:57:02.445848+0000","last_unstale":"2024-09-10T18:58:58.490588+0000","last_undegraded":"2024-09-10T18:58:58.490588+0000","last_fullsized":"2024-09-10T18:58:58.490588+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:14.634438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_clean_scrub_stamp":"2024-09-10T18:56:14.634438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058886299999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"242'10","reported_seq":495,"reported_epoch":305,"state":"active+clean","last_fresh":"2024-09-10T18:59:06.238518+0000","last_change":"2024-09-10T18:56:19.581070+0000","last_active":"2024-09-10T18:59:06.238518+0000","last_peered":"2024-09-10T18:59:06.238518+0000","last_clean":"2024-09-10T18:59:06.238518+0000","last_became_active":"2024-09-10T18:55:40.397246+0000","last_became_peered":"2024-09-10T18:55:40.397246+0000","last_unstale":"2024-09-10T18:59:06.238518+0000","last_undegraded":"2024-09-10T18:59:06.238518+0000","last_fullsized":"2024-09-10T18:59:06.238518+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"242'10","last_scrub_stamp":"2024-09-10T18:56:19.581012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:19.581012+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013423529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":328,"reported_epoch":305,"state":"active+clean","last_fresh":"2024-09-10T18:59:10.477458+0000","last_change":"2024-09-10T18:59:10.477418+0000","last_active":"2024-09-10T18:59:10.477458+0000","last_peered":"2024-09-10T18:59:10.477458+0000","last_clean":"2024-09-10T18:59:10.477458+0000","last_became_active":"2024-09-10T18:55:40.392003+0000","last_became_peered":"2024-09-10T18:55:40.392003+0000","last_unstale":"2024-09-10T18:59:10.477458+0000","last_undegraded":"2024-09-10T18:59:10.477458+0000","last_fullsized":"2024-09-10T18:59:10.477458+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:10.477346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:54.218602+0000","last_clean_scrub_stamp":"2024-09-10T18:59:10.477346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:29.628862+0000","scrub_duration":0.004313005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"305'16200","reported_seq":31585,"reported_epoch":305,"state":"active+clean","last_fresh":"2024-09-10T18:59:11.919719+0000","last_change":"2024-09-10T18:57:59.139381+0000","last_active":"2024-09-10T18:59:11.919719+0000","last_peered":"2024-09-10T18:59:11.919719+0000","last_clean":"2024-09-10T18:59:11.919719+0000","last_became_active":"2024-09-10T18:57:59.111754+0000","last_became_peered":"2024-09-10T18:57:59.111754+0000","last_unstale":"2024-09-10T18:59:11.919719+0000","last_undegraded":"2024-09-10T18:59:11.919719+0000","last_fullsized":"2024-09-10T18:59:11.919719+0000","mapping_epoch":282,"log_start":"295'12817","ondisk_log_start":"295'12817","created":237,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3383,"log_dups_size":0,"ondisk_log_size":3383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71827456,"num_objects":1096,"num_object_clones":0,"num_object_copies":3288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16200,"num_write_kb":553472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7642,"num_bytes_recovered":500826112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.b","version":"70'81","reported_seq":260,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.491082+0000","last_change":"2024-09-10T18:56:33.793393+0000","last_active":"2024-09-10T18:58:58.491082+0000","last_peered":"2024-09-10T18:58:58.491082+0000","last_clean":"2024-09-10T18:58:58.491082+0000","last_became_active":"2024-09-10T18:55:40.401643+0000","last_became_peered":"2024-09-10T18:55:40.401643+0000","last_unstale":"2024-09-10T18:58:58.491082+0000","last_undegraded":"2024-09-10T18:58:58.491082+0000","last_fullsized":"2024-09-10T18:58:58.491082+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:33.793296+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:33.793296+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045437230000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"232'24","reported_seq":415,"reported_epoch":305,"state":"active+clean","last_fresh":"2024-09-10T18:59:06.595633+0000","last_change":"2024-09-10T18:59:06.595605+0000","last_active":"2024-09-10T18:59:06.595633+0000","last_peered":"2024-09-10T18:59:06.595633+0000","last_clean":"2024-09-10T18:59:06.595633+0000","last_became_active":"2024-09-10T18:47:57.106935+0000","last_became_peered":"2024-09-10T18:47:57.106935+0000","last_unstale":"2024-09-10T18:59:06.595633+0000","last_undegraded":"2024-09-10T18:59:06.595633+0000","last_fullsized":"2024-09-10T18:59:06.595633+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T18:59:06.595554+0000","last_deep_scrub":"224'16","last_deep_scrub_stamp":"2024-09-10T18:54:49.296957+0000","last_clean_scrub_stamp":"2024-09-10T18:59:06.595554+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:19.274166+0000","scrub_duration":0.0020642540000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":300,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.489141+0000","last_change":"2024-09-10T18:57:02.444295+0000","last_active":"2024-09-10T18:58:58.489141+0000","last_peered":"2024-09-10T18:58:58.489141+0000","last_clean":"2024-09-10T18:58:58.489141+0000","last_became_active":"2024-09-10T18:57:02.443816+0000","last_became_peered":"2024-09-10T18:57:02.443816+0000","last_unstale":"2024-09-10T18:58:58.489141+0000","last_undegraded":"2024-09-10T18:58:58.489141+0000","last_fullsized":"2024-09-10T18:58:58.489141+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:29.994533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:29.994533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053276879999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":316,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.487175+0000","last_change":"2024-09-10T18:58:54.548235+0000","last_active":"2024-09-10T18:58:58.487175+0000","last_peered":"2024-09-10T18:58:58.487175+0000","last_clean":"2024-09-10T18:58:58.487175+0000","last_became_active":"2024-09-10T18:45:22.068918+0000","last_became_peered":"2024-09-10T18:45:22.068918+0000","last_unstale":"2024-09-10T18:58:58.487175+0000","last_undegraded":"2024-09-10T18:58:58.487175+0000","last_fullsized":"2024-09-10T18:58:58.487175+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:54.548153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_clean_scrub_stamp":"2024-09-10T18:58:54.548153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:06.044612+0000","scrub_duration":0.0013051650000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":291,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.497396+0000","last_change":"2024-09-10T18:54:51.399064+0000","last_active":"2024-09-10T18:58:58.497396+0000","last_peered":"2024-09-10T18:58:58.497396+0000","last_clean":"2024-09-10T18:58:58.497396+0000","last_became_active":"2024-09-10T18:45:24.264538+0000","last_became_peered":"2024-09-10T18:45:24.264538+0000","last_unstale":"2024-09-10T18:58:58.497396+0000","last_undegraded":"2024-09-10T18:58:58.497396+0000","last_fullsized":"2024-09-10T18:58:58.497396+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:54:51.398981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:51.398981+0000","last_clean_scrub_stamp":"2024-09-10T18:54:51.398981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027589680000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"305'16118","reported_seq":30373,"reported_epoch":305,"state":"active+clean","last_fresh":"2024-09-10T18:59:11.912938+0000","last_change":"2024-09-10T18:58:11.814916+0000","last_active":"2024-09-10T18:59:11.912938+0000","last_peered":"2024-09-10T18:59:11.912938+0000","last_clean":"2024-09-10T18:59:11.912938+0000","last_became_active":"2024-09-10T18:58:11.753019+0000","last_became_peered":"2024-09-10T18:58:11.753019+0000","last_unstale":"2024-09-10T18:59:11.912938+0000","last_undegraded":"2024-09-10T18:59:11.912938+0000","last_fullsized":"2024-09-10T18:59:11.912938+0000","mapping_epoch":289,"log_start":"295'12691","ondisk_log_start":"295'12691","created":237,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3427,"log_dups_size":0,"ondisk_log_size":3427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71434240,"num_objects":1090,"num_object_clones":0,"num_object_copies":3270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16118,"num_write_kb":550656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7076,"num_bytes_recovered":463667200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"70'81","reported_seq":249,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.497549+0000","last_change":"2024-09-10T18:58:56.913127+0000","last_active":"2024-09-10T18:58:58.497549+0000","last_peered":"2024-09-10T18:58:58.497549+0000","last_clean":"2024-09-10T18:58:58.497549+0000","last_became_active":"2024-09-10T18:57:02.459708+0000","last_became_peered":"2024-09-10T18:57:02.459708+0000","last_unstale":"2024-09-10T18:58:58.497549+0000","last_undegraded":"2024-09-10T18:58:58.497549+0000","last_fullsized":"2024-09-10T18:58:58.497549+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:58:56.912936+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:58:56.912936+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:21.078455+0000","scrub_duration":0.0045874109999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"232'35","reported_seq":352,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.483489+0000","last_change":"2024-09-10T18:56:28.475224+0000","last_active":"2024-09-10T18:58:58.483489+0000","last_peered":"2024-09-10T18:58:58.483489+0000","last_clean":"2024-09-10T18:58:58.483489+0000","last_became_active":"2024-09-10T18:45:20.052243+0000","last_became_peered":"2024-09-10T18:45:20.052243+0000","last_unstale":"2024-09-10T18:58:58.483489+0000","last_undegraded":"2024-09-10T18:58:58.483489+0000","last_fullsized":"2024-09-10T18:58:58.483489+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'35","last_scrub_stamp":"2024-09-10T18:56:28.475142+0000","last_deep_scrub":"143'21","last_deep_scrub_stamp":"2024-09-10T18:51:02.540216+0000","last_clean_scrub_stamp":"2024-09-10T18:56:28.475142+0000","objects_scrubbed":7,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026978649999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":337,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.484955+0000","last_change":"2024-09-10T18:57:02.453766+0000","last_active":"2024-09-10T18:58:58.484955+0000","last_peered":"2024-09-10T18:58:58.484955+0000","last_clean":"2024-09-10T18:58:58.484955+0000","last_became_active":"2024-09-10T18:57:02.453404+0000","last_became_peered":"2024-09-10T18:57:02.453404+0000","last_unstale":"2024-09-10T18:58:58.484955+0000","last_undegraded":"2024-09-10T18:58:58.484955+0000","last_fullsized":"2024-09-10T18:58:58.484955+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:29.226803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_clean_scrub_stamp":"2024-09-10T18:56:29.226803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0086984899999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":478,"reported_epoch":305,"state":"active+clean","last_fresh":"2024-09-10T18:59:06.240822+0000","last_change":"2024-09-10T18:56:17.914259+0000","last_active":"2024-09-10T18:59:06.240822+0000","last_peered":"2024-09-10T18:59:06.240822+0000","last_clean":"2024-09-10T18:59:06.240822+0000","last_became_active":"2024-09-10T18:45:22.074540+0000","last_became_peered":"2024-09-10T18:45:22.074540+0000","last_unstale":"2024-09-10T18:59:06.240822+0000","last_undegraded":"2024-09-10T18:59:06.240822+0000","last_fullsized":"2024-09-10T18:59:06.240822+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:56:17.914175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:17.914175+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026666329999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":104,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.482901+0000","last_change":"2024-09-10T18:58:50.638774+0000","last_active":"2024-09-10T18:58:58.482901+0000","last_peered":"2024-09-10T18:58:58.482901+0000","last_clean":"2024-09-10T18:58:58.482901+0000","last_became_active":"2024-09-10T18:57:02.437100+0000","last_became_peered":"2024-09-10T18:57:02.437100+0000","last_unstale":"2024-09-10T18:58:58.482901+0000","last_undegraded":"2024-09-10T18:58:58.482901+0000","last_fullsized":"2024-09-10T18:58:58.482901+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:50.638720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:58:50.638720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T18:59:52.477163+0000","scrub_duration":0.003753546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.d","version":"305'15656","reported_seq":30614,"reported_epoch":305,"state":"active+clean","last_fresh":"2024-09-10T18:59:11.671912+0000","last_change":"2024-09-10T18:57:58.141549+0000","last_active":"2024-09-10T18:59:11.671912+0000","last_peered":"2024-09-10T18:59:11.671912+0000","last_clean":"2024-09-10T18:59:11.671912+0000","last_became_active":"2024-09-10T18:57:58.118967+0000","last_became_peered":"2024-09-10T18:57:58.118967+0000","last_unstale":"2024-09-10T18:59:11.671912+0000","last_undegraded":"2024-09-10T18:59:11.671912+0000","last_fullsized":"2024-09-10T18:59:11.671912+0000","mapping_epoch":281,"log_start":"296'12551","ondisk_log_start":"296'12551","created":237,"last_epoch_clean":282,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3105,"log_dups_size":0,"ondisk_log_size":3105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70975512,"num_objects":1084,"num_object_clones":0,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15656,"num_write_kb":535617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7430,"num_bytes_recovered":486866968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"70'81","reported_seq":223,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.484619+0000","last_change":"2024-09-10T18:56:31.481362+0000","last_active":"2024-09-10T18:58:58.484619+0000","last_peered":"2024-09-10T18:58:58.484619+0000","last_clean":"2024-09-10T18:58:58.484619+0000","last_became_active":"2024-09-10T18:52:14.584798+0000","last_became_peered":"2024-09-10T18:52:14.584798+0000","last_unstale":"2024-09-10T18:58:58.484619+0000","last_undegraded":"2024-09-10T18:58:58.484619+0000","last_fullsized":"2024-09-10T18:58:58.484619+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:31.481303+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:56:31.481303+0000","last_clean_scrub_stamp":"2024-09-10T18:56:31.481303+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0011170959999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"232'30","reported_seq":415,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.497063+0000","last_change":"2024-09-10T18:57:02.490987+0000","last_active":"2024-09-10T18:58:58.497063+0000","last_peered":"2024-09-10T18:58:58.497063+0000","last_clean":"2024-09-10T18:58:58.497063+0000","last_became_active":"2024-09-10T18:57:02.455615+0000","last_became_peered":"2024-09-10T18:57:02.455615+0000","last_unstale":"2024-09-10T18:58:58.497063+0000","last_undegraded":"2024-09-10T18:58:58.497063+0000","last_fullsized":"2024-09-10T18:58:58.497063+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'30","last_scrub_stamp":"2024-09-10T18:56:15.875924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:15.875924+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0075153879999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":310,"reported_epoch":305,"state":"active+clean","last_fresh":"2024-09-10T18:59:05.247147+0000","last_change":"2024-09-10T18:59:05.246945+0000","last_active":"2024-09-10T18:59:05.247147+0000","last_peered":"2024-09-10T18:59:05.247147+0000","last_clean":"2024-09-10T18:59:05.247147+0000","last_became_active":"2024-09-10T18:45:17.961695+0000","last_became_peered":"2024-09-10T18:45:17.961695+0000","last_unstale":"2024-09-10T18:59:05.247147+0000","last_undegraded":"2024-09-10T18:59:05.247147+0000","last_fullsized":"2024-09-10T18:59:05.247147+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:05.246629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:46.142129+0000","last_clean_scrub_stamp":"2024-09-10T18:59:05.246629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:18.444804+0000","scrub_duration":0.0072716029999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":311,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.483369+0000","last_change":"2024-09-10T18:57:02.448422+0000","last_active":"2024-09-10T18:58:58.483369+0000","last_peered":"2024-09-10T18:58:58.483369+0000","last_clean":"2024-09-10T18:58:58.483369+0000","last_became_active":"2024-09-10T18:57:02.448126+0000","last_became_peered":"2024-09-10T18:57:02.448126+0000","last_unstale":"2024-09-10T18:58:58.483369+0000","last_undegraded":"2024-09-10T18:58:58.483369+0000","last_fullsized":"2024-09-10T18:58:58.483369+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:29.533177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:55:29.533177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012137999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":309,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.483474+0000","last_change":"2024-09-10T18:58:48.633501+0000","last_active":"2024-09-10T18:58:58.483474+0000","last_peered":"2024-09-10T18:58:58.483474+0000","last_clean":"2024-09-10T18:58:58.483474+0000","last_became_active":"2024-09-10T18:47:36.410836+0000","last_became_peered":"2024-09-10T18:47:36.410836+0000","last_unstale":"2024-09-10T18:58:58.483474+0000","last_undegraded":"2024-09-10T18:58:58.483474+0000","last_fullsized":"2024-09-10T18:58:58.483474+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:48.633423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_clean_scrub_stamp":"2024-09-10T18:58:48.633423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:14.559342+0000","scrub_duration":0.0079262829999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.e","version":"305'16050","reported_seq":31697,"reported_epoch":305,"state":"active+clean","last_fresh":"2024-09-10T18:59:11.238114+0000","last_change":"2024-09-10T18:57:51.717681+0000","last_active":"2024-09-10T18:59:11.238114+0000","last_peered":"2024-09-10T18:59:11.238114+0000","last_clean":"2024-09-10T18:59:11.238114+0000","last_became_active":"2024-09-10T18:57:51.701782+0000","last_became_peered":"2024-09-10T18:57:51.701782+0000","last_unstale":"2024-09-10T18:59:11.238114+0000","last_undegraded":"2024-09-10T18:59:11.238114+0000","last_fullsized":"2024-09-10T18:59:11.238114+0000","mapping_epoch":279,"log_start":"295'12362","ondisk_log_start":"295'12362","created":237,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3688,"log_dups_size":0,"ondisk_log_size":3688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":77201408,"num_objects":1178,"num_object_clones":0,"num_object_copies":3534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16050,"num_write_kb":551296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7773,"num_bytes_recovered":509411328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"70'81","reported_seq":310,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.484370+0000","last_change":"2024-09-10T18:56:12.184278+0000","last_active":"2024-09-10T18:58:58.484370+0000","last_peered":"2024-09-10T18:58:58.484370+0000","last_clean":"2024-09-10T18:58:58.484370+0000","last_became_active":"2024-09-10T18:55:40.396828+0000","last_became_peered":"2024-09-10T18:55:40.396828+0000","last_unstale":"2024-09-10T18:58:58.484370+0000","last_undegraded":"2024-09-10T18:58:58.484370+0000","last_fullsized":"2024-09-10T18:58:58.484370+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:12.184127+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:12.184127+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040368189999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"232'29","reported_seq":362,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.837396+0000","last_change":"2024-09-10T18:58:58.837250+0000","last_active":"2024-09-10T18:58:58.837396+0000","last_peered":"2024-09-10T18:58:58.837396+0000","last_clean":"2024-09-10T18:58:58.837396+0000","last_became_active":"2024-09-10T18:57:02.442862+0000","last_became_peered":"2024-09-10T18:57:02.442862+0000","last_unstale":"2024-09-10T18:58:58.837396+0000","last_undegraded":"2024-09-10T18:58:58.837396+0000","last_fullsized":"2024-09-10T18:58:58.837396+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'29","last_scrub_stamp":"2024-09-10T18:58:58.837163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:58:58.837163+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:12.101443+0000","scrub_duration":0.019169703999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":290,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.484559+0000","last_change":"2024-09-10T18:57:02.444015+0000","last_active":"2024-09-10T18:58:58.484559+0000","last_peered":"2024-09-10T18:58:58.484559+0000","last_clean":"2024-09-10T18:58:58.484559+0000","last_became_active":"2024-09-10T18:57:02.443522+0000","last_became_peered":"2024-09-10T18:57:02.443522+0000","last_unstale":"2024-09-10T18:58:58.484559+0000","last_undegraded":"2024-09-10T18:58:58.484559+0000","last_fullsized":"2024-09-10T18:58:58.484559+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:01.022282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:48.196065+0000","last_clean_scrub_stamp":"2024-09-10T18:56:01.022282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013037999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":320,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.483288+0000","last_change":"2024-09-10T18:56:02.162637+0000","last_active":"2024-09-10T18:58:58.483288+0000","last_peered":"2024-09-10T18:58:58.483288+0000","last_clean":"2024-09-10T18:58:58.483288+0000","last_became_active":"2024-09-10T18:55:40.387857+0000","last_became_peered":"2024-09-10T18:55:40.387857+0000","last_unstale":"2024-09-10T18:58:58.483288+0000","last_undegraded":"2024-09-10T18:58:58.483288+0000","last_fullsized":"2024-09-10T18:58:58.483288+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:02.162551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:02.162551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027989849999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":311,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.490446+0000","last_change":"2024-09-10T18:57:02.445448+0000","last_active":"2024-09-10T18:58:58.490446+0000","last_peered":"2024-09-10T18:58:58.490446+0000","last_clean":"2024-09-10T18:58:58.490446+0000","last_became_active":"2024-09-10T18:57:02.442479+0000","last_became_peered":"2024-09-10T18:57:02.442479+0000","last_unstale":"2024-09-10T18:58:58.490446+0000","last_undegraded":"2024-09-10T18:58:58.490446+0000","last_fullsized":"2024-09-10T18:58:58.490446+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:30.482659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:49:18.281230+0000","last_clean_scrub_stamp":"2024-09-10T18:55:30.482659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015073689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.f","version":"305'16059","reported_seq":16199,"reported_epoch":305,"state":"active+clean","last_fresh":"2024-09-10T18:59:11.457438+0000","last_change":"2024-09-10T18:59:10.020045+0000","last_active":"2024-09-10T18:59:11.457438+0000","last_peered":"2024-09-10T18:59:11.457438+0000","last_clean":"2024-09-10T18:59:11.457438+0000","last_became_active":"2024-09-10T18:55:36.358064+0000","last_became_peered":"2024-09-10T18:55:36.358064+0000","last_unstale":"2024-09-10T18:59:11.457438+0000","last_undegraded":"2024-09-10T18:59:11.457438+0000","last_fullsized":"2024-09-10T18:59:11.457438+0000","mapping_epoch":237,"log_start":"288'11269","ondisk_log_start":"288'11269","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"305'15928","last_scrub_stamp":"2024-09-10T18:59:10.019945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:59:10.019945+0000","objects_scrubbed":1332,"log_size":4790,"log_dups_size":0,"ondisk_log_size":4790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:25.221537+0000","scrub_duration":0.56148255400000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":77004800,"num_objects":1175,"num_object_clones":0,"num_object_copies":3525,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16059,"num_write_kb":551488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"70'81","reported_seq":313,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.484524+0000","last_change":"2024-09-10T18:57:02.453204+0000","last_active":"2024-09-10T18:58:58.484524+0000","last_peered":"2024-09-10T18:58:58.484524+0000","last_clean":"2024-09-10T18:58:58.484524+0000","last_became_active":"2024-09-10T18:57:02.452659+0000","last_became_peered":"2024-09-10T18:57:02.452659+0000","last_unstale":"2024-09-10T18:58:58.484524+0000","last_undegraded":"2024-09-10T18:58:58.484524+0000","last_fullsized":"2024-09-10T18:58:58.484524+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:28.264003+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:28.264003+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001853637,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"232'25","reported_seq":392,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.492219+0000","last_change":"2024-09-10T18:56:27.835783+0000","last_active":"2024-09-10T18:58:58.492219+0000","last_peered":"2024-09-10T18:58:58.492219+0000","last_clean":"2024-09-10T18:58:58.492219+0000","last_became_active":"2024-09-10T18:55:40.400777+0000","last_became_peered":"2024-09-10T18:55:40.400777+0000","last_unstale":"2024-09-10T18:58:58.492219+0000","last_undegraded":"2024-09-10T18:58:58.492219+0000","last_fullsized":"2024-09-10T18:58:58.492219+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'25","last_scrub_stamp":"2024-09-10T18:56:27.835688+0000","last_deep_scrub":"83'15","last_deep_scrub_stamp":"2024-09-10T18:46:58.436473+0000","last_clean_scrub_stamp":"2024-09-10T18:56:27.835688+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047372150000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":344,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.493199+0000","last_change":"2024-09-10T18:58:50.137466+0000","last_active":"2024-09-10T18:58:58.493199+0000","last_peered":"2024-09-10T18:58:58.493199+0000","last_clean":"2024-09-10T18:58:58.493199+0000","last_became_active":"2024-09-10T18:57:02.447940+0000","last_became_peered":"2024-09-10T18:57:02.447940+0000","last_unstale":"2024-09-10T18:58:58.493199+0000","last_undegraded":"2024-09-10T18:58:58.493199+0000","last_fullsized":"2024-09-10T18:58:58.493199+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:50.137393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:05.140027+0000","last_clean_scrub_stamp":"2024-09-10T18:58:50.137393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:07.786488+0000","scrub_duration":0.014454461,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"258'12","reported_seq":422,"reported_epoch":305,"state":"active+clean","last_fresh":"2024-09-10T18:59:06.238699+0000","last_change":"2024-09-10T18:56:21.098913+0000","last_active":"2024-09-10T18:59:06.238699+0000","last_peered":"2024-09-10T18:59:06.238699+0000","last_clean":"2024-09-10T18:59:06.238699+0000","last_became_active":"2024-09-10T18:55:40.391094+0000","last_became_peered":"2024-09-10T18:55:40.391094+0000","last_unstale":"2024-09-10T18:59:06.238699+0000","last_undegraded":"2024-09-10T18:59:06.238699+0000","last_fullsized":"2024-09-10T18:59:06.238699+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'12","last_scrub_stamp":"2024-09-10T18:56:21.098770+0000","last_deep_scrub":"258'12","last_deep_scrub_stamp":"2024-09-10T18:56:21.098770+0000","last_clean_scrub_stamp":"2024-09-10T18:56:21.098770+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034993049999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":276,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.489342+0000","last_change":"2024-09-10T18:57:03.353073+0000","last_active":"2024-09-10T18:58:58.489342+0000","last_peered":"2024-09-10T18:58:58.489342+0000","last_clean":"2024-09-10T18:58:58.489342+0000","last_became_active":"2024-09-10T18:55:40.397478+0000","last_became_peered":"2024-09-10T18:55:40.397478+0000","last_unstale":"2024-09-10T18:58:58.489342+0000","last_undegraded":"2024-09-10T18:58:58.489342+0000","last_fullsized":"2024-09-10T18:58:58.489342+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:57:03.353014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:57:03.353014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014199639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.9","version":"305'16132","reported_seq":29992,"reported_epoch":305,"state":"active+clean","last_fresh":"2024-09-10T18:59:13.627205+0000","last_change":"2024-09-10T18:59:06.506789+0000","last_active":"2024-09-10T18:59:13.627205+0000","last_peered":"2024-09-10T18:59:13.627205+0000","last_clean":"2024-09-10T18:59:13.627205+0000","last_became_active":"2024-09-10T18:58:15.716019+0000","last_became_peered":"2024-09-10T18:58:15.716019+0000","last_unstale":"2024-09-10T18:59:13.627205+0000","last_undegraded":"2024-09-10T18:59:13.627205+0000","last_fullsized":"2024-09-10T18:59:13.627205+0000","mapping_epoch":291,"log_start":"295'12430","ondisk_log_start":"295'12430","created":237,"last_epoch_clean":292,"parent":"0.0","parent_split_bits":0,"last_scrub":"305'15576","last_scrub_stamp":"2024-09-10T18:59:06.506706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:59:06.506706+0000","objects_scrubbed":1526,"log_size":3702,"log_dups_size":0,"ondisk_log_size":3702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:29.132448+0000","scrub_duration":0.63256196099999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62521344,"num_objects":954,"num_object_clones":0,"num_object_copies":2862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16132,"num_write_kb":546752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6845,"num_bytes_recovered":448593920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"70'81","reported_seq":242,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.484656+0000","last_change":"2024-09-10T18:56:11.765628+0000","last_active":"2024-09-10T18:58:58.484656+0000","last_peered":"2024-09-10T18:58:58.484656+0000","last_clean":"2024-09-10T18:58:58.484656+0000","last_became_active":"2024-09-10T18:55:40.392432+0000","last_became_peered":"2024-09-10T18:55:40.392432+0000","last_unstale":"2024-09-10T18:58:58.484656+0000","last_undegraded":"2024-09-10T18:58:58.484656+0000","last_fullsized":"2024-09-10T18:58:58.484656+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:11.765541+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:11.765541+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002140374,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"232'41","reported_seq":570,"reported_epoch":305,"state":"active+clean","last_fresh":"2024-09-10T18:59:06.243937+0000","last_change":"2024-09-10T18:56:18.672331+0000","last_active":"2024-09-10T18:59:06.243937+0000","last_peered":"2024-09-10T18:59:06.243937+0000","last_clean":"2024-09-10T18:59:06.243937+0000","last_became_active":"2024-09-10T18:47:36.420185+0000","last_became_peered":"2024-09-10T18:47:36.420185+0000","last_unstale":"2024-09-10T18:59:06.243937+0000","last_undegraded":"2024-09-10T18:59:06.243937+0000","last_fullsized":"2024-09-10T18:59:06.243937+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'41","last_scrub_stamp":"2024-09-10T18:56:18.672139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:18.672139+0000","objects_scrubbed":10,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.020233260999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":320,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:59:04.434090+0000","last_change":"2024-09-10T18:59:04.434037+0000","last_active":"2024-09-10T18:59:04.434090+0000","last_peered":"2024-09-10T18:59:04.434090+0000","last_clean":"2024-09-10T18:59:04.434090+0000","last_became_active":"2024-09-10T18:45:17.964494+0000","last_became_peered":"2024-09-10T18:45:17.964494+0000","last_unstale":"2024-09-10T18:59:04.434090+0000","last_undegraded":"2024-09-10T18:59:04.434090+0000","last_fullsized":"2024-09-10T18:59:04.434090+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:04.433964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:12.982599+0000","last_clean_scrub_stamp":"2024-09-10T18:59:04.433964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:11.584403+0000","scrub_duration":0.0024364220000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":261,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.483213+0000","last_change":"2024-09-10T18:56:20.224668+0000","last_active":"2024-09-10T18:58:58.483213+0000","last_peered":"2024-09-10T18:58:58.483213+0000","last_clean":"2024-09-10T18:58:58.483213+0000","last_became_active":"2024-09-10T18:47:57.107478+0000","last_became_peered":"2024-09-10T18:47:57.107478+0000","last_unstale":"2024-09-10T18:58:58.483213+0000","last_undegraded":"2024-09-10T18:58:58.483213+0000","last_fullsized":"2024-09-10T18:58:58.483213+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:20.224562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:20.224562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.008542315,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":301,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.496221+0000","last_change":"2024-09-10T18:57:02.462009+0000","last_active":"2024-09-10T18:58:58.496221+0000","last_peered":"2024-09-10T18:58:58.496221+0000","last_clean":"2024-09-10T18:58:58.496221+0000","last_became_active":"2024-09-10T18:57:02.460594+0000","last_became_peered":"2024-09-10T18:57:02.460594+0000","last_unstale":"2024-09-10T18:58:58.496221+0000","last_undegraded":"2024-09-10T18:58:58.496221+0000","last_fullsized":"2024-09-10T18:58:58.496221+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:11.878181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:22.891084+0000","last_clean_scrub_stamp":"2024-09-10T18:56:11.878181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037255349999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"305'16194","reported_seq":29407,"reported_epoch":305,"state":"active+clean","last_fresh":"2024-09-10T18:59:11.746117+0000","last_change":"2024-09-10T18:58:11.767121+0000","last_active":"2024-09-10T18:59:11.746117+0000","last_peered":"2024-09-10T18:59:11.746117+0000","last_clean":"2024-09-10T18:59:11.746117+0000","last_became_active":"2024-09-10T18:58:11.743985+0000","last_became_peered":"2024-09-10T18:58:11.743985+0000","last_unstale":"2024-09-10T18:59:11.746117+0000","last_undegraded":"2024-09-10T18:59:11.746117+0000","last_fullsized":"2024-09-10T18:59:11.746117+0000","mapping_epoch":289,"log_start":"296'13017","ondisk_log_start":"296'13017","created":237,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3177,"log_dups_size":0,"ondisk_log_size":3177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":1088,"num_object_clones":0,"num_object_copies":3264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16194,"num_write_kb":553024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6554,"num_bytes_recovered":429522944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"70'81","reported_seq":122,"reported_epoch":305,"state":"active+clean","last_fresh":"2024-09-10T18:59:11.875753+0000","last_change":"2024-09-10T18:59:11.875713+0000","last_active":"2024-09-10T18:59:11.875753+0000","last_peered":"2024-09-10T18:59:11.875753+0000","last_clean":"2024-09-10T18:59:11.875753+0000","last_became_active":"2024-09-10T18:57:02.437944+0000","last_became_peered":"2024-09-10T18:57:02.437944+0000","last_unstale":"2024-09-10T18:59:11.875753+0000","last_undegraded":"2024-09-10T18:59:11.875753+0000","last_fullsized":"2024-09-10T18:59:11.875753+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:11.875625+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:59:11.875625+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:13.558213+0000","scrub_duration":0.0040920360000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"232'22","reported_seq":357,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.494851+0000","last_change":"2024-09-10T18:56:05.550938+0000","last_active":"2024-09-10T18:58:58.494851+0000","last_peered":"2024-09-10T18:58:58.494851+0000","last_clean":"2024-09-10T18:58:58.494851+0000","last_became_active":"2024-09-10T18:55:40.397422+0000","last_became_peered":"2024-09-10T18:55:40.397422+0000","last_unstale":"2024-09-10T18:58:58.494851+0000","last_undegraded":"2024-09-10T18:58:58.494851+0000","last_fullsized":"2024-09-10T18:58:58.494851+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'22","last_scrub_stamp":"2024-09-10T18:56:05.550846+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-10T18:47:10.514951+0000","last_clean_scrub_stamp":"2024-09-10T18:56:05.550846+0000","objects_scrubbed":5,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0079381009999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"84'6","reported_seq":1158,"reported_epoch":305,"state":"active+clean","last_fresh":"2024-09-10T18:59:10.447771+0000","last_change":"2024-09-10T18:56:22.799839+0000","last_active":"2024-09-10T18:59:10.447771+0000","last_peered":"2024-09-10T18:59:10.447771+0000","last_clean":"2024-09-10T18:59:10.447771+0000","last_became_active":"2024-09-10T18:55:40.403098+0000","last_became_peered":"2024-09-10T18:55:40.403098+0000","last_unstale":"2024-09-10T18:59:10.447771+0000","last_undegraded":"2024-09-10T18:59:10.447771+0000","last_fullsized":"2024-09-10T18:59:10.447771+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'6","last_scrub_stamp":"2024-09-10T18:56:22.799414+0000","last_deep_scrub":"84'6","last_deep_scrub_stamp":"2024-09-10T18:47:22.545450+0000","last_clean_scrub_stamp":"2024-09-10T18:56:22.799414+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004952756,"objects_trimmed":0,"snaptrim_duration":0.0021933159999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":817,"num_read_kb":812,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":275,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.493877+0000","last_change":"2024-09-10T18:56:03.791304+0000","last_active":"2024-09-10T18:58:58.493877+0000","last_peered":"2024-09-10T18:58:58.493877+0000","last_clean":"2024-09-10T18:58:58.493877+0000","last_became_active":"2024-09-10T18:55:40.393389+0000","last_became_peered":"2024-09-10T18:55:40.393389+0000","last_unstale":"2024-09-10T18:58:58.493877+0000","last_undegraded":"2024-09-10T18:58:58.493877+0000","last_fullsized":"2024-09-10T18:58:58.493877+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:03.791228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:03.791228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004016899,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":291,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.522392+0000","last_change":"2024-09-10T18:58:58.522346+0000","last_active":"2024-09-10T18:58:58.522392+0000","last_peered":"2024-09-10T18:58:58.522392+0000","last_clean":"2024-09-10T18:58:58.522392+0000","last_became_active":"2024-09-10T18:57:02.438945+0000","last_became_peered":"2024-09-10T18:57:02.438945+0000","last_unstale":"2024-09-10T18:58:58.522392+0000","last_undegraded":"2024-09-10T18:58:58.522392+0000","last_fullsized":"2024-09-10T18:58:58.522392+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:58.522295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:58:58.522295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:06.243103+0000","scrub_duration":0.0059618290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":47,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.483716+0000","last_change":"2024-09-10T18:57:02.454154+0000","last_active":"2024-09-10T18:58:58.483716+0000","last_peered":"2024-09-10T18:58:58.483716+0000","last_clean":"2024-09-10T18:58:58.483716+0000","last_became_active":"2024-09-10T18:57:02.453865+0000","last_became_peered":"2024-09-10T18:57:02.453865+0000","last_unstale":"2024-09-10T18:58:58.483716+0000","last_undegraded":"2024-09-10T18:58:58.483716+0000","last_fullsized":"2024-09-10T18:58:58.483716+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:21.086881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:07.150480+0000","last_clean_scrub_stamp":"2024-09-10T18:56:21.086881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.b","version":"305'16043","reported_seq":16172,"reported_epoch":305,"state":"active+clean","last_fresh":"2024-09-10T18:59:13.629372+0000","last_change":"2024-09-10T18:59:10.290839+0000","last_active":"2024-09-10T18:59:13.629372+0000","last_peered":"2024-09-10T18:59:13.629372+0000","last_clean":"2024-09-10T18:59:13.629372+0000","last_became_active":"2024-09-10T18:55:40.391432+0000","last_became_peered":"2024-09-10T18:55:40.391432+0000","last_unstale":"2024-09-10T18:59:13.629372+0000","last_undegraded":"2024-09-10T18:59:13.629372+0000","last_fullsized":"2024-09-10T18:59:13.629372+0000","mapping_epoch":241,"log_start":"295'12422","ondisk_log_start":"295'12422","created":237,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"305'15787","last_scrub_stamp":"2024-09-10T18:59:10.290714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:59:10.290714+0000","objects_scrubbed":1158,"log_size":3621,"log_dups_size":0,"ondisk_log_size":3621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:36.990040+0000","scrub_duration":0.42113059000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57999360,"num_objects":885,"num_object_clones":0,"num_object_copies":2655,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16043,"num_write_kb":541696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"70'81","reported_seq":224,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.490345+0000","last_change":"2024-09-10T18:58:49.141636+0000","last_active":"2024-09-10T18:58:58.490345+0000","last_peered":"2024-09-10T18:58:58.490345+0000","last_clean":"2024-09-10T18:58:58.490345+0000","last_became_active":"2024-09-10T18:52:14.589009+0000","last_became_peered":"2024-09-10T18:52:14.589009+0000","last_unstale":"2024-09-10T18:58:58.490345+0000","last_undegraded":"2024-09-10T18:58:58.490345+0000","last_fullsized":"2024-09-10T18:58:58.490345+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:58:49.141013+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:58:49.141013+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:15.628763+0000","scrub_duration":0.0036869540000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"232'24","reported_seq":363,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.486305+0000","last_change":"2024-09-10T18:57:02.478395+0000","last_active":"2024-09-10T18:58:58.486305+0000","last_peered":"2024-09-10T18:58:58.486305+0000","last_clean":"2024-09-10T18:58:58.486305+0000","last_became_active":"2024-09-10T18:57:02.447673+0000","last_became_peered":"2024-09-10T18:57:02.447673+0000","last_unstale":"2024-09-10T18:58:58.486305+0000","last_undegraded":"2024-09-10T18:58:58.486305+0000","last_fullsized":"2024-09-10T18:58:58.486305+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T18:56:30.183414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:30.183414+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047701100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"58'1","reported_seq":278,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.488311+0000","last_change":"2024-09-10T18:56:06.131199+0000","last_active":"2024-09-10T18:58:58.488311+0000","last_peered":"2024-09-10T18:58:58.488311+0000","last_clean":"2024-09-10T18:58:58.488311+0000","last_became_active":"2024-09-10T18:55:40.390022+0000","last_became_peered":"2024-09-10T18:55:40.390022+0000","last_unstale":"2024-09-10T18:58:58.488311+0000","last_undegraded":"2024-09-10T18:58:58.488311+0000","last_fullsized":"2024-09-10T18:58:58.488311+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'1","last_scrub_stamp":"2024-09-10T18:56:06.131098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:06.131098+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016770345999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":248,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.490272+0000","last_change":"2024-09-10T18:57:02.447491+0000","last_active":"2024-09-10T18:58:58.490272+0000","last_peered":"2024-09-10T18:58:58.490272+0000","last_clean":"2024-09-10T18:58:58.490272+0000","last_became_active":"2024-09-10T18:57:02.447195+0000","last_became_peered":"2024-09-10T18:57:02.447195+0000","last_unstale":"2024-09-10T18:58:58.490272+0000","last_undegraded":"2024-09-10T18:58:58.490272+0000","last_fullsized":"2024-09-10T18:58:58.490272+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:24.153973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:24.153973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001771613,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.499145+0000","last_change":"2024-09-10T18:58:58.498916+0000","last_active":"2024-09-10T18:58:58.499145+0000","last_peered":"2024-09-10T18:58:58.499145+0000","last_clean":"2024-09-10T18:58:58.499145+0000","last_became_active":"2024-09-10T18:58:58.498629+0000","last_became_peered":"2024-09-10T18:58:58.498629+0000","last_unstale":"2024-09-10T18:58:58.499145+0000","last_undegraded":"2024-09-10T18:58:58.499145+0000","last_fullsized":"2024-09-10T18:58:58.499145+0000","mapping_epoch":303,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":304,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:30.801727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:30.801727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,4],"acting":[6,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":304,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.492742+0000","last_change":"2024-09-10T18:58:52.136201+0000","last_active":"2024-09-10T18:58:58.492742+0000","last_peered":"2024-09-10T18:58:58.492742+0000","last_clean":"2024-09-10T18:58:58.492742+0000","last_became_active":"2024-09-10T18:45:24.083689+0000","last_became_peered":"2024-09-10T18:45:24.083689+0000","last_unstale":"2024-09-10T18:58:58.492742+0000","last_undegraded":"2024-09-10T18:58:58.492742+0000","last_fullsized":"2024-09-10T18:58:58.492742+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:52.136095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:58:52.136095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:02.933124+0000","scrub_duration":0.021731822000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"305'16142","reported_seq":26732,"reported_epoch":305,"state":"active+clean","last_fresh":"2024-09-10T18:59:11.911532+0000","last_change":"2024-09-10T18:58:33.729848+0000","last_active":"2024-09-10T18:59:11.911532+0000","last_peered":"2024-09-10T18:59:11.911532+0000","last_clean":"2024-09-10T18:59:11.911532+0000","last_became_active":"2024-09-10T18:58:33.712029+0000","last_became_peered":"2024-09-10T18:58:33.712029+0000","last_unstale":"2024-09-10T18:59:11.911532+0000","last_undegraded":"2024-09-10T18:59:11.911532+0000","last_fullsized":"2024-09-10T18:59:11.911532+0000","mapping_epoch":297,"log_start":"295'12738","ondisk_log_start":"295'12738","created":237,"last_epoch_clean":298,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3404,"log_dups_size":0,"ondisk_log_size":3404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67502080,"num_objects":1030,"num_object_clones":0,"num_object_copies":3090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16142,"num_write_kb":549504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5239,"num_bytes_recovered":343343104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"232'33","reported_seq":388,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.483646+0000","last_change":"2024-09-10T18:57:02.468556+0000","last_active":"2024-09-10T18:58:58.483646+0000","last_peered":"2024-09-10T18:58:58.483646+0000","last_clean":"2024-09-10T18:58:58.483646+0000","last_became_active":"2024-09-10T18:57:02.435499+0000","last_became_peered":"2024-09-10T18:57:02.435499+0000","last_unstale":"2024-09-10T18:58:58.483646+0000","last_undegraded":"2024-09-10T18:58:58.483646+0000","last_fullsized":"2024-09-10T18:58:58.483646+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'33","last_scrub_stamp":"2024-09-10T18:56:21.206960+0000","last_deep_scrub":"232'33","last_deep_scrub_stamp":"2024-09-10T18:56:21.206960+0000","last_clean_scrub_stamp":"2024-09-10T18:56:21.206960+0000","objects_scrubbed":7,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0042638809999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"60'2","reported_seq":320,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.496926+0000","last_change":"2024-09-10T18:58:55.931801+0000","last_active":"2024-09-10T18:58:58.496926+0000","last_peered":"2024-09-10T18:58:58.496926+0000","last_clean":"2024-09-10T18:58:58.496926+0000","last_became_active":"2024-09-10T18:45:17.961660+0000","last_became_peered":"2024-09-10T18:45:17.961660+0000","last_unstale":"2024-09-10T18:58:58.496926+0000","last_undegraded":"2024-09-10T18:58:58.496926+0000","last_fullsized":"2024-09-10T18:58:58.496926+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-10T18:58:55.931717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:58:55.931717+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:13.291828+0000","scrub_duration":0.0040038909999999999,"objects_trimmed":0,"snaptrim_duration":0.0013942640000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":337,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.485684+0000","last_change":"2024-09-10T18:57:02.446028+0000","last_active":"2024-09-10T18:58:58.485684+0000","last_peered":"2024-09-10T18:58:58.485684+0000","last_clean":"2024-09-10T18:58:58.485684+0000","last_became_active":"2024-09-10T18:57:02.445712+0000","last_became_peered":"2024-09-10T18:57:02.445712+0000","last_unstale":"2024-09-10T18:58:58.485684+0000","last_undegraded":"2024-09-10T18:58:58.485684+0000","last_fullsized":"2024-09-10T18:58:58.485684+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:33.559651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:33.559651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0036787759999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'81","reported_seq":112,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.488814+0000","last_change":"2024-09-10T18:57:02.440391+0000","last_active":"2024-09-10T18:58:58.488814+0000","last_peered":"2024-09-10T18:58:58.488814+0000","last_clean":"2024-09-10T18:58:58.488814+0000","last_became_active":"2024-09-10T18:57:02.440063+0000","last_became_peered":"2024-09-10T18:57:02.440063+0000","last_unstale":"2024-09-10T18:58:58.488814+0000","last_undegraded":"2024-09-10T18:58:58.488814+0000","last_fullsized":"2024-09-10T18:58:58.488814+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:11.056156+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:56.085580+0000","last_clean_scrub_stamp":"2024-09-10T18:56:11.056156+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019876809999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26834,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":9,"num_write":5,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":53669,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":306,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.485910+0000","last_change":"2024-09-10T18:57:02.432025+0000","last_active":"2024-09-10T18:58:58.485910+0000","last_peered":"2024-09-10T18:58:58.485910+0000","last_clean":"2024-09-10T18:58:58.485910+0000","last_became_active":"2024-09-10T18:57:02.431556+0000","last_became_peered":"2024-09-10T18:57:02.431556+0000","last_unstale":"2024-09-10T18:58:58.485910+0000","last_undegraded":"2024-09-10T18:58:58.485910+0000","last_fullsized":"2024-09-10T18:58:58.485910+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:27.012123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:27.012123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007606036,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":311,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.487336+0000","last_change":"2024-09-10T18:56:34.565177+0000","last_active":"2024-09-10T18:58:58.487336+0000","last_peered":"2024-09-10T18:58:58.487336+0000","last_clean":"2024-09-10T18:58:58.487336+0000","last_became_active":"2024-09-10T18:45:24.073327+0000","last_became_peered":"2024-09-10T18:45:24.073327+0000","last_unstale":"2024-09-10T18:58:58.487336+0000","last_undegraded":"2024-09-10T18:58:58.487336+0000","last_fullsized":"2024-09-10T18:58:58.487336+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:34.564993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:34.564993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0031904390000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.7","version":"305'15640","reported_seq":29849,"reported_epoch":305,"state":"active+clean","last_fresh":"2024-09-10T18:59:11.466037+0000","last_change":"2024-09-10T18:59:01.247187+0000","last_active":"2024-09-10T18:59:11.466037+0000","last_peered":"2024-09-10T18:59:11.466037+0000","last_clean":"2024-09-10T18:59:11.466037+0000","last_became_active":"2024-09-10T18:58:06.157679+0000","last_became_peered":"2024-09-10T18:58:06.157679+0000","last_unstale":"2024-09-10T18:59:11.466037+0000","last_undegraded":"2024-09-10T18:59:11.466037+0000","last_fullsized":"2024-09-10T18:59:11.466037+0000","mapping_epoch":286,"log_start":"285'10875","ondisk_log_start":"285'10875","created":237,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"304'14817","last_scrub_stamp":"2024-09-10T18:59:01.247106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:59:01.247106+0000","objects_scrubbed":2001,"log_size":4765,"log_dups_size":0,"ondisk_log_size":4765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:20.751192+0000","scrub_duration":0.76863930199999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75235328,"num_objects":1148,"num_object_clones":0,"num_object_copies":3444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15640,"num_write_kb":537216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7010,"num_bytes_recovered":459407360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.f","version":"70'81","reported_seq":315,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:59.582666+0000","last_change":"2024-09-10T18:58:59.582622+0000","last_active":"2024-09-10T18:58:59.582666+0000","last_peered":"2024-09-10T18:58:59.582666+0000","last_clean":"2024-09-10T18:58:59.582666+0000","last_became_active":"2024-09-10T18:52:16.598295+0000","last_became_peered":"2024-09-10T18:52:16.598295+0000","last_unstale":"2024-09-10T18:58:59.582666+0000","last_undegraded":"2024-09-10T18:58:59.582666+0000","last_fullsized":"2024-09-10T18:58:59.582666+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:58:59.582565+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:53.609885+0000","last_clean_scrub_stamp":"2024-09-10T18:58:59.582565+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:10.859477+0000","scrub_duration":0.0046448920000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"232'35","reported_seq":403,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.483431+0000","last_change":"2024-09-10T18:57:02.466701+0000","last_active":"2024-09-10T18:58:58.483431+0000","last_peered":"2024-09-10T18:58:58.483431+0000","last_clean":"2024-09-10T18:58:58.483431+0000","last_became_active":"2024-09-10T18:57:02.436059+0000","last_became_peered":"2024-09-10T18:57:02.436059+0000","last_unstale":"2024-09-10T18:58:58.483431+0000","last_undegraded":"2024-09-10T18:58:58.483431+0000","last_fullsized":"2024-09-10T18:58:58.483431+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'35","last_scrub_stamp":"2024-09-10T18:56:34.373736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:34.373736+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016527315000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":285,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.485976+0000","last_change":"2024-09-10T18:56:13.760318+0000","last_active":"2024-09-10T18:58:58.485976+0000","last_peered":"2024-09-10T18:58:58.485976+0000","last_clean":"2024-09-10T18:58:58.485976+0000","last_became_active":"2024-09-10T18:55:40.391849+0000","last_became_peered":"2024-09-10T18:55:40.391849+0000","last_unstale":"2024-09-10T18:58:58.485976+0000","last_undegraded":"2024-09-10T18:58:58.485976+0000","last_fullsized":"2024-09-10T18:58:58.485976+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:13.760238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:13.760238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.048619793000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":245,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.483379+0000","last_change":"2024-09-10T18:57:02.451716+0000","last_active":"2024-09-10T18:58:58.483379+0000","last_peered":"2024-09-10T18:58:58.483379+0000","last_clean":"2024-09-10T18:58:58.483379+0000","last_became_active":"2024-09-10T18:57:02.451332+0000","last_became_peered":"2024-09-10T18:57:02.451332+0000","last_unstale":"2024-09-10T18:58:58.483379+0000","last_undegraded":"2024-09-10T18:58:58.483379+0000","last_fullsized":"2024-09-10T18:58:58.483379+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:24.158403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:24.158403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.02305596,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":104,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.496821+0000","last_change":"2024-09-10T18:57:03.015513+0000","last_active":"2024-09-10T18:58:58.496821+0000","last_peered":"2024-09-10T18:58:58.496821+0000","last_clean":"2024-09-10T18:58:58.496821+0000","last_became_active":"2024-09-10T18:57:02.468866+0000","last_became_peered":"2024-09-10T18:57:02.468866+0000","last_unstale":"2024-09-10T18:58:58.496821+0000","last_undegraded":"2024-09-10T18:58:58.496821+0000","last_fullsized":"2024-09-10T18:58:58.496821+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:57:03.015457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:57:03.015457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001545795,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.6","version":"305'16129","reported_seq":16305,"reported_epoch":305,"state":"active+clean","last_fresh":"2024-09-10T18:59:11.466783+0000","last_change":"2024-09-10T18:58:58.181715+0000","last_active":"2024-09-10T18:59:11.466783+0000","last_peered":"2024-09-10T18:59:11.466783+0000","last_clean":"2024-09-10T18:59:11.466783+0000","last_became_active":"2024-09-10T18:55:36.357828+0000","last_became_peered":"2024-09-10T18:55:36.357828+0000","last_unstale":"2024-09-10T18:59:11.466783+0000","last_undegraded":"2024-09-10T18:59:11.466783+0000","last_fullsized":"2024-09-10T18:59:11.466783+0000","mapping_epoch":237,"log_start":"288'11327","ondisk_log_start":"288'11327","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"303'15015","last_scrub_stamp":"2024-09-10T18:58:58.181622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:58:58.181622+0000","objects_scrubbed":2247,"log_size":4802,"log_dups_size":0,"ondisk_log_size":4802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:02.739827+0000","scrub_duration":0.69115638300000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":72810496,"num_objects":1111,"num_object_clones":0,"num_object_copies":3333,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16129,"num_write_kb":551680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.e","version":"70'81","reported_seq":174,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.485268+0000","last_change":"2024-09-10T18:57:02.450954+0000","last_active":"2024-09-10T18:58:58.485268+0000","last_peered":"2024-09-10T18:58:58.485268+0000","last_clean":"2024-09-10T18:58:58.485268+0000","last_became_active":"2024-09-10T18:57:02.449115+0000","last_became_peered":"2024-09-10T18:57:02.449115+0000","last_unstale":"2024-09-10T18:58:58.485268+0000","last_undegraded":"2024-09-10T18:58:58.485268+0000","last_fullsized":"2024-09-10T18:58:58.485268+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:55:30.496910+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:55:30.496910+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012110790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"232'19","reported_seq":311,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.483881+0000","last_change":"2024-09-10T18:56:20.061554+0000","last_active":"2024-09-10T18:58:58.483881+0000","last_peered":"2024-09-10T18:58:58.483881+0000","last_clean":"2024-09-10T18:58:58.483881+0000","last_became_active":"2024-09-10T18:55:40.395257+0000","last_became_peered":"2024-09-10T18:55:40.395257+0000","last_unstale":"2024-09-10T18:58:58.483881+0000","last_undegraded":"2024-09-10T18:58:58.483881+0000","last_fullsized":"2024-09-10T18:58:58.483881+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T18:56:20.061420+0000","last_deep_scrub":"130'13","last_deep_scrub_stamp":"2024-09-10T18:51:58.427534+0000","last_clean_scrub_stamp":"2024-09-10T18:56:20.061420+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0065809780000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":307,"reported_epoch":305,"state":"active+clean","last_fresh":"2024-09-10T18:59:09.206749+0000","last_change":"2024-09-10T18:59:09.206400+0000","last_active":"2024-09-10T18:59:09.206749+0000","last_peered":"2024-09-10T18:59:09.206749+0000","last_clean":"2024-09-10T18:59:09.206749+0000","last_became_active":"2024-09-10T18:45:17.963057+0000","last_became_peered":"2024-09-10T18:45:17.963057+0000","last_unstale":"2024-09-10T18:59:09.206749+0000","last_undegraded":"2024-09-10T18:59:09.206749+0000","last_fullsized":"2024-09-10T18:59:09.206749+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:09.206317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:59:09.206317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:20.174093+0000","scrub_duration":0.018986050000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":273,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.483398+0000","last_change":"2024-09-10T18:56:17.661917+0000","last_active":"2024-09-10T18:58:58.483398+0000","last_peered":"2024-09-10T18:58:58.483398+0000","last_clean":"2024-09-10T18:58:58.483398+0000","last_became_active":"2024-09-10T18:55:40.390771+0000","last_became_peered":"2024-09-10T18:55:40.390771+0000","last_unstale":"2024-09-10T18:58:58.483398+0000","last_undegraded":"2024-09-10T18:58:58.483398+0000","last_fullsized":"2024-09-10T18:58:58.483398+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:17.661832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_clean_scrub_stamp":"2024-09-10T18:56:17.661832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041460890000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":299,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.488040+0000","last_change":"2024-09-10T18:57:02.453927+0000","last_active":"2024-09-10T18:58:58.488040+0000","last_peered":"2024-09-10T18:58:58.488040+0000","last_clean":"2024-09-10T18:58:58.488040+0000","last_became_active":"2024-09-10T18:57:02.452387+0000","last_became_peered":"2024-09-10T18:57:02.452387+0000","last_unstale":"2024-09-10T18:58:58.488040+0000","last_undegraded":"2024-09-10T18:58:58.488040+0000","last_fullsized":"2024-09-10T18:58:58.488040+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:32.217641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:53.345184+0000","last_clean_scrub_stamp":"2024-09-10T18:56:32.217641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0059447470000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.5","version":"305'15987","reported_seq":30394,"reported_epoch":305,"state":"active+clean","last_fresh":"2024-09-10T18:59:13.611299+0000","last_change":"2024-09-10T18:59:07.490651+0000","last_active":"2024-09-10T18:59:13.611299+0000","last_peered":"2024-09-10T18:59:13.611299+0000","last_clean":"2024-09-10T18:59:13.611299+0000","last_became_active":"2024-09-10T18:58:07.161586+0000","last_became_peered":"2024-09-10T18:58:07.161586+0000","last_unstale":"2024-09-10T18:59:13.611299+0000","last_undegraded":"2024-09-10T18:59:13.611299+0000","last_fullsized":"2024-09-10T18:59:13.611299+0000","mapping_epoch":287,"log_start":"295'12352","ondisk_log_start":"295'12352","created":237,"last_epoch_clean":288,"parent":"0.0","parent_split_bits":0,"last_scrub":"305'15476","last_scrub_stamp":"2024-09-10T18:59:07.489750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:59:07.489750+0000","objects_scrubbed":1516,"log_size":3635,"log_dups_size":0,"ondisk_log_size":3635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:16.877296+0000","scrub_duration":0.64902760800000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64159744,"num_objects":979,"num_object_clones":0,"num_object_copies":2937,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15987,"num_write_kb":542912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7119,"num_bytes_recovered":466550784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.d","version":"70'81","reported_seq":112,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.485003+0000","last_change":"2024-09-10T18:57:02.439985+0000","last_active":"2024-09-10T18:58:58.485003+0000","last_peered":"2024-09-10T18:58:58.485003+0000","last_clean":"2024-09-10T18:58:58.485003+0000","last_became_active":"2024-09-10T18:57:02.439652+0000","last_became_peered":"2024-09-10T18:57:02.439652+0000","last_unstale":"2024-09-10T18:58:58.485003+0000","last_undegraded":"2024-09-10T18:58:58.485003+0000","last_fullsized":"2024-09-10T18:58:58.485003+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:18.041116+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:18.041116+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014613810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"232'27","reported_seq":216,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.483797+0000","last_change":"2024-09-10T18:58:47.595348+0000","last_active":"2024-09-10T18:58:58.483797+0000","last_peered":"2024-09-10T18:58:58.483797+0000","last_clean":"2024-09-10T18:58:58.483797+0000","last_became_active":"2024-09-10T18:57:02.454653+0000","last_became_peered":"2024-09-10T18:57:02.454653+0000","last_unstale":"2024-09-10T18:58:58.483797+0000","last_undegraded":"2024-09-10T18:58:58.483797+0000","last_fullsized":"2024-09-10T18:58:58.483797+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'27","last_scrub_stamp":"2024-09-10T18:58:47.595256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:58:47.595256+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:08.737359+0000","scrub_duration":0.0093202389999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":33,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3],"acting":[6,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":268,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.496363+0000","last_change":"2024-09-10T18:57:02.463092+0000","last_active":"2024-09-10T18:58:58.496363+0000","last_peered":"2024-09-10T18:58:58.496363+0000","last_clean":"2024-09-10T18:58:58.496363+0000","last_became_active":"2024-09-10T18:57:02.461708+0000","last_became_peered":"2024-09-10T18:57:02.461708+0000","last_unstale":"2024-09-10T18:58:58.496363+0000","last_undegraded":"2024-09-10T18:58:58.496363+0000","last_fullsized":"2024-09-10T18:58:58.496363+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:24.902205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:05.603125+0000","last_clean_scrub_stamp":"2024-09-10T18:56:24.902205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0076874830000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"273'10","reported_seq":145,"reported_epoch":305,"state":"active+clean","last_fresh":"2024-09-10T18:59:06.238409+0000","last_change":"2024-09-10T18:57:02.476362+0000","last_active":"2024-09-10T18:59:06.238409+0000","last_peered":"2024-09-10T18:59:06.238409+0000","last_clean":"2024-09-10T18:59:06.238409+0000","last_became_active":"2024-09-10T18:57:02.441890+0000","last_became_peered":"2024-09-10T18:57:02.441890+0000","last_unstale":"2024-09-10T18:59:06.238409+0000","last_undegraded":"2024-09-10T18:59:06.238409+0000","last_fullsized":"2024-09-10T18:59:06.238409+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'9","last_scrub_stamp":"2024-09-10T18:55:37.750784+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-09-10T18:47:02.582229+0000","last_clean_scrub_stamp":"2024-09-10T18:55:37.750784+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":318,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.485648+0000","last_change":"2024-09-10T18:56:10.775911+0000","last_active":"2024-09-10T18:58:58.485648+0000","last_peered":"2024-09-10T18:58:58.485648+0000","last_clean":"2024-09-10T18:58:58.485648+0000","last_became_active":"2024-09-10T18:55:40.389202+0000","last_became_peered":"2024-09-10T18:55:40.389202+0000","last_unstale":"2024-09-10T18:58:58.485648+0000","last_undegraded":"2024-09-10T18:58:58.485648+0000","last_fullsized":"2024-09-10T18:58:58.485648+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:10.775814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_clean_scrub_stamp":"2024-09-10T18:56:10.775814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0031802459999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.4","version":"305'15938","reported_seq":16014,"reported_epoch":305,"state":"active+clean","last_fresh":"2024-09-10T18:59:11.737508+0000","last_change":"2024-09-10T18:55:36.350444+0000","last_active":"2024-09-10T18:59:11.737508+0000","last_peered":"2024-09-10T18:59:11.737508+0000","last_clean":"2024-09-10T18:59:11.737508+0000","last_became_active":"2024-09-10T18:55:36.350155+0000","last_became_peered":"2024-09-10T18:55:36.350155+0000","last_unstale":"2024-09-10T18:59:11.737508+0000","last_undegraded":"2024-09-10T18:59:11.737508+0000","last_fullsized":"2024-09-10T18:59:11.737508+0000","mapping_epoch":237,"log_start":"298'12812","ondisk_log_start":"298'12812","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:35.336477+0000","last_clean_scrub_stamp":"2024-09-10T18:55:35.336477+0000","objects_scrubbed":0,"log_size":3126,"log_dups_size":0,"ondisk_log_size":3126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70516736,"num_objects":1076,"num_object_clones":0,"num_object_copies":3228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1076,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15938,"num_write_kb":544448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.c","version":"70'81","reported_seq":261,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.497293+0000","last_change":"2024-09-10T18:56:33.779212+0000","last_active":"2024-09-10T18:58:58.497293+0000","last_peered":"2024-09-10T18:58:58.497293+0000","last_clean":"2024-09-10T18:58:58.497293+0000","last_became_active":"2024-09-10T18:55:40.390655+0000","last_became_peered":"2024-09-10T18:55:40.390655+0000","last_unstale":"2024-09-10T18:58:58.497293+0000","last_undegraded":"2024-09-10T18:58:58.497293+0000","last_fullsized":"2024-09-10T18:58:58.497293+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:33.779154+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:56:33.779154+0000","last_clean_scrub_stamp":"2024-09-10T18:56:33.779154+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0075121179999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"232'18","reported_seq":335,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.483620+0000","last_change":"2024-09-10T18:55:27.507437+0000","last_active":"2024-09-10T18:58:58.483620+0000","last_peered":"2024-09-10T18:58:58.483620+0000","last_clean":"2024-09-10T18:58:58.483620+0000","last_became_active":"2024-09-10T18:45:20.056869+0000","last_became_peered":"2024-09-10T18:45:20.056869+0000","last_unstale":"2024-09-10T18:58:58.483620+0000","last_undegraded":"2024-09-10T18:58:58.483620+0000","last_fullsized":"2024-09-10T18:58:58.483620+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'18","last_scrub_stamp":"2024-09-10T18:55:27.507364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:55:27.507364+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030221390000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"56'2","reported_seq":345,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.485229+0000","last_change":"2024-09-10T18:57:02.456385+0000","last_active":"2024-09-10T18:58:58.485229+0000","last_peered":"2024-09-10T18:58:58.485229+0000","last_clean":"2024-09-10T18:58:58.485229+0000","last_became_active":"2024-09-10T18:57:02.434475+0000","last_became_peered":"2024-09-10T18:57:02.434475+0000","last_unstale":"2024-09-10T18:58:58.485229+0000","last_undegraded":"2024-09-10T18:58:58.485229+0000","last_fullsized":"2024-09-10T18:58:58.485229+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-10T18:56:01.630091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:01.630091+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001578238,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":245,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.489268+0000","last_change":"2024-09-10T18:56:16.019721+0000","last_active":"2024-09-10T18:58:58.489268+0000","last_peered":"2024-09-10T18:58:58.489268+0000","last_clean":"2024-09-10T18:58:58.489268+0000","last_became_active":"2024-09-10T18:55:40.397290+0000","last_became_peered":"2024-09-10T18:55:40.397290+0000","last_unstale":"2024-09-10T18:58:58.489268+0000","last_undegraded":"2024-09-10T18:58:58.489268+0000","last_fullsized":"2024-09-10T18:58:58.489268+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:16.019357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:02.711594+0000","last_clean_scrub_stamp":"2024-09-10T18:56:16.019357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021619629999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":311,"reported_epoch":305,"state":"active+clean","last_fresh":"2024-09-10T18:59:11.827221+0000","last_change":"2024-09-10T18:59:11.827181+0000","last_active":"2024-09-10T18:59:11.827221+0000","last_peered":"2024-09-10T18:59:11.827221+0000","last_clean":"2024-09-10T18:59:11.827221+0000","last_became_active":"2024-09-10T18:57:02.459985+0000","last_became_peered":"2024-09-10T18:57:02.459985+0000","last_unstale":"2024-09-10T18:59:11.827221+0000","last_undegraded":"2024-09-10T18:59:11.827221+0000","last_fullsized":"2024-09-10T18:59:11.827221+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:11.827101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:53.352427+0000","last_clean_scrub_stamp":"2024-09-10T18:59:11.827101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:37.818697+0000","scrub_duration":0.0034557899999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.13","version":"70'81","reported_seq":233,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.490798+0000","last_change":"2024-09-10T18:56:34.760114+0000","last_active":"2024-09-10T18:58:58.490798+0000","last_peered":"2024-09-10T18:58:58.490798+0000","last_clean":"2024-09-10T18:58:58.490798+0000","last_became_active":"2024-09-10T18:52:16.602463+0000","last_became_peered":"2024-09-10T18:52:16.602463+0000","last_unstale":"2024-09-10T18:58:58.490798+0000","last_undegraded":"2024-09-10T18:58:58.490798+0000","last_fullsized":"2024-09-10T18:58:58.490798+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:34.760031+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:34.760031+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002735595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"232'19","reported_seq":334,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.486166+0000","last_change":"2024-09-10T18:57:02.485836+0000","last_active":"2024-09-10T18:58:58.486166+0000","last_peered":"2024-09-10T18:58:58.486166+0000","last_clean":"2024-09-10T18:58:58.486166+0000","last_became_active":"2024-09-10T18:57:02.453001+0000","last_became_peered":"2024-09-10T18:57:02.453001+0000","last_unstale":"2024-09-10T18:58:58.486166+0000","last_undegraded":"2024-09-10T18:58:58.486166+0000","last_fullsized":"2024-09-10T18:58:58.486166+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T18:56:25.122070+0000","last_deep_scrub":"76'13","last_deep_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_clean_scrub_stamp":"2024-09-10T18:56:25.122070+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0061142590000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":285,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.489790+0000","last_change":"2024-09-10T18:56:25.820426+0000","last_active":"2024-09-10T18:58:58.489790+0000","last_peered":"2024-09-10T18:58:58.489790+0000","last_clean":"2024-09-10T18:58:58.489790+0000","last_became_active":"2024-09-10T18:55:40.401037+0000","last_became_peered":"2024-09-10T18:55:40.401037+0000","last_unstale":"2024-09-10T18:58:58.489790+0000","last_undegraded":"2024-09-10T18:58:58.489790+0000","last_fullsized":"2024-09-10T18:58:58.489790+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:25.820339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:25.820339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048533279999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":313,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.496225+0000","last_change":"2024-09-10T18:57:02.455638+0000","last_active":"2024-09-10T18:58:58.496225+0000","last_peered":"2024-09-10T18:58:58.496225+0000","last_clean":"2024-09-10T18:58:58.496225+0000","last_became_active":"2024-09-10T18:57:02.455341+0000","last_became_peered":"2024-09-10T18:57:02.455341+0000","last_unstale":"2024-09-10T18:58:58.496225+0000","last_undegraded":"2024-09-10T18:58:58.496225+0000","last_fullsized":"2024-09-10T18:58:58.496225+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:25.862853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:00.754912+0000","last_clean_scrub_stamp":"2024-09-10T18:56:25.862853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0033952399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"63'1","reported_seq":303,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.484556+0000","last_change":"2024-09-10T18:57:02.456685+0000","last_active":"2024-09-10T18:58:58.484556+0000","last_peered":"2024-09-10T18:58:58.484556+0000","last_clean":"2024-09-10T18:58:58.484556+0000","last_became_active":"2024-09-10T18:57:02.435896+0000","last_became_peered":"2024-09-10T18:57:02.435896+0000","last_unstale":"2024-09-10T18:58:58.484556+0000","last_undegraded":"2024-09-10T18:58:58.484556+0000","last_fullsized":"2024-09-10T18:58:58.484556+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-10T18:56:07.774334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:07.774334+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.090386770000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.12","version":"70'81","reported_seq":175,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.485838+0000","last_change":"2024-09-10T18:58:52.543604+0000","last_active":"2024-09-10T18:58:58.485838+0000","last_peered":"2024-09-10T18:58:58.485838+0000","last_clean":"2024-09-10T18:58:58.485838+0000","last_became_active":"2024-09-10T18:52:16.598921+0000","last_became_peered":"2024-09-10T18:52:16.598921+0000","last_unstale":"2024-09-10T18:58:58.485838+0000","last_undegraded":"2024-09-10T18:58:58.485838+0000","last_fullsized":"2024-09-10T18:58:58.485838+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:58:52.543544+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:54.127476+0000","last_clean_scrub_stamp":"2024-09-10T18:58:52.543544+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:02.310835+0000","scrub_duration":0.003920644,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":340,"reported_epoch":305,"state":"active+clean","last_fresh":"2024-09-10T18:59:05.556418+0000","last_change":"2024-09-10T18:59:05.556377+0000","last_active":"2024-09-10T18:59:05.556418+0000","last_peered":"2024-09-10T18:59:05.556418+0000","last_clean":"2024-09-10T18:59:05.556418+0000","last_became_active":"2024-09-10T18:57:02.442251+0000","last_became_peered":"2024-09-10T18:57:02.442251+0000","last_unstale":"2024-09-10T18:59:05.556418+0000","last_undegraded":"2024-09-10T18:59:05.556418+0000","last_fullsized":"2024-09-10T18:59:05.556418+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:05.556292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:59:05.556292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:29.350561+0000","scrub_duration":0.0026781280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"232'12","reported_seq":334,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.496549+0000","last_change":"2024-09-10T18:57:02.483803+0000","last_active":"2024-09-10T18:58:58.496549+0000","last_peered":"2024-09-10T18:58:58.496549+0000","last_clean":"2024-09-10T18:58:58.496549+0000","last_became_active":"2024-09-10T18:57:02.460430+0000","last_became_peered":"2024-09-10T18:57:02.460430+0000","last_unstale":"2024-09-10T18:58:58.496549+0000","last_undegraded":"2024-09-10T18:58:58.496549+0000","last_fullsized":"2024-09-10T18:58:58.496549+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'12","last_scrub_stamp":"2024-09-10T18:55:30.612808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:55:30.612808+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001922787,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":303,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.486748+0000","last_change":"2024-09-10T18:56:32.602499+0000","last_active":"2024-09-10T18:58:58.486748+0000","last_peered":"2024-09-10T18:58:58.486748+0000","last_clean":"2024-09-10T18:58:58.486748+0000","last_became_active":"2024-09-10T18:45:22.069653+0000","last_became_peered":"2024-09-10T18:45:22.069653+0000","last_unstale":"2024-09-10T18:58:58.486748+0000","last_undegraded":"2024-09-10T18:58:58.486748+0000","last_fullsized":"2024-09-10T18:58:58.486748+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:32.602448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:32.602448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0052503430000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":57,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.483359+0000","last_change":"2024-09-10T18:58:53.608110+0000","last_active":"2024-09-10T18:58:58.483359+0000","last_peered":"2024-09-10T18:58:58.483359+0000","last_clean":"2024-09-10T18:58:58.483359+0000","last_became_active":"2024-09-10T18:57:02.437979+0000","last_became_peered":"2024-09-10T18:57:02.437979+0000","last_unstale":"2024-09-10T18:58:58.483359+0000","last_undegraded":"2024-09-10T18:58:58.483359+0000","last_fullsized":"2024-09-10T18:58:58.483359+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:53.608029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:58:53.608029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:13.844448+0000","scrub_duration":0.0058174170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.11","version":"70'81","reported_seq":199,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.488138+0000","last_change":"2024-09-10T18:58:48.552866+0000","last_active":"2024-09-10T18:58:58.488138+0000","last_peered":"2024-09-10T18:58:58.488138+0000","last_clean":"2024-09-10T18:58:58.488138+0000","last_became_active":"2024-09-10T18:55:40.395019+0000","last_became_peered":"2024-09-10T18:55:40.395019+0000","last_unstale":"2024-09-10T18:58:58.488138+0000","last_undegraded":"2024-09-10T18:58:58.488138+0000","last_fullsized":"2024-09-10T18:58:58.488138+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:58:48.552631+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:58:48.552631+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:14.476031+0000","scrub_duration":0.012534919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"232'20","reported_seq":391,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.496766+0000","last_change":"2024-09-10T18:57:02.468721+0000","last_active":"2024-09-10T18:58:58.496766+0000","last_peered":"2024-09-10T18:58:58.496766+0000","last_clean":"2024-09-10T18:58:58.496766+0000","last_became_active":"2024-09-10T18:57:02.444437+0000","last_became_peered":"2024-09-10T18:57:02.444437+0000","last_unstale":"2024-09-10T18:58:58.496766+0000","last_undegraded":"2024-09-10T18:58:58.496766+0000","last_fullsized":"2024-09-10T18:58:58.496766+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'20","last_scrub_stamp":"2024-09-10T18:55:33.667712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:55:33.667712+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0028302039999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":327,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.486071+0000","last_change":"2024-09-10T18:58:50.541776+0000","last_active":"2024-09-10T18:58:58.486071+0000","last_peered":"2024-09-10T18:58:58.486071+0000","last_clean":"2024-09-10T18:58:58.486071+0000","last_became_active":"2024-09-10T18:57:02.445710+0000","last_became_peered":"2024-09-10T18:57:02.445710+0000","last_unstale":"2024-09-10T18:58:58.486071+0000","last_undegraded":"2024-09-10T18:58:58.486071+0000","last_fullsized":"2024-09-10T18:58:58.486071+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:50.541691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:58:50.541691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:20.462191+0000","scrub_duration":0.0089012770000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"304'11","reported_seq":512,"reported_epoch":305,"state":"active+clean","last_fresh":"2024-09-10T18:59:06.238593+0000","last_change":"2024-09-10T18:58:58.492043+0000","last_active":"2024-09-10T18:59:06.238593+0000","last_peered":"2024-09-10T18:59:06.238593+0000","last_clean":"2024-09-10T18:59:06.238593+0000","last_became_active":"2024-09-10T18:58:58.489370+0000","last_became_peered":"2024-09-10T18:58:58.489370+0000","last_unstale":"2024-09-10T18:59:06.238593+0000","last_undegraded":"2024-09-10T18:59:06.238593+0000","last_fullsized":"2024-09-10T18:59:06.238593+0000","mapping_epoch":303,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":304,"parent":"0.0","parent_split_bits":0,"last_scrub":"242'10","last_scrub_stamp":"2024-09-10T18:56:07.786933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:07.786933+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0054185049999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":268,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.485599+0000","last_change":"2024-09-10T18:58:57.812623+0000","last_active":"2024-09-10T18:58:58.485599+0000","last_peered":"2024-09-10T18:58:58.485599+0000","last_clean":"2024-09-10T18:58:58.485599+0000","last_became_active":"2024-09-10T18:57:02.452949+0000","last_became_peered":"2024-09-10T18:57:02.452949+0000","last_unstale":"2024-09-10T18:58:58.485599+0000","last_undegraded":"2024-09-10T18:58:58.485599+0000","last_fullsized":"2024-09-10T18:58:58.485599+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:57.812395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:57.476724+0000","last_clean_scrub_stamp":"2024-09-10T18:58:57.812395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:05.269799+0000","scrub_duration":0.0050128610000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.10","version":"70'81","reported_seq":175,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.483926+0000","last_change":"2024-09-10T18:57:02.444347+0000","last_active":"2024-09-10T18:58:58.483926+0000","last_peered":"2024-09-10T18:58:58.483926+0000","last_clean":"2024-09-10T18:58:58.483926+0000","last_became_active":"2024-09-10T18:57:02.443782+0000","last_became_peered":"2024-09-10T18:57:02.443782+0000","last_unstale":"2024-09-10T18:58:58.483926+0000","last_undegraded":"2024-09-10T18:58:58.483926+0000","last_fullsized":"2024-09-10T18:58:58.483926+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:20.743543+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:20.743543+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043223280000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"232'15","reported_seq":169,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.494837+0000","last_change":"2024-09-10T18:57:02.487296+0000","last_active":"2024-09-10T18:58:58.494837+0000","last_peered":"2024-09-10T18:58:58.494837+0000","last_clean":"2024-09-10T18:58:58.494837+0000","last_became_active":"2024-09-10T18:57:02.460705+0000","last_became_peered":"2024-09-10T18:57:02.460705+0000","last_unstale":"2024-09-10T18:58:58.494837+0000","last_undegraded":"2024-09-10T18:58:58.494837+0000","last_fullsized":"2024-09-10T18:58:58.494837+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T18:55:25.784333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:55:25.784333+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":283,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.483503+0000","last_change":"2024-09-10T18:56:23.096461+0000","last_active":"2024-09-10T18:58:58.483503+0000","last_peered":"2024-09-10T18:58:58.483503+0000","last_clean":"2024-09-10T18:58:58.483503+0000","last_became_active":"2024-09-10T18:55:40.390697+0000","last_became_peered":"2024-09-10T18:55:40.390697+0000","last_unstale":"2024-09-10T18:58:58.483503+0000","last_undegraded":"2024-09-10T18:58:58.483503+0000","last_fullsized":"2024-09-10T18:58:58.483503+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:23.096373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:23.096373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0057615890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"273'11","reported_seq":497,"reported_epoch":305,"state":"active+clean","last_fresh":"2024-09-10T18:59:06.244638+0000","last_change":"2024-09-10T18:57:02.473032+0000","last_active":"2024-09-10T18:59:06.244638+0000","last_peered":"2024-09-10T18:59:06.244638+0000","last_clean":"2024-09-10T18:59:06.244638+0000","last_became_active":"2024-09-10T18:57:02.445268+0000","last_became_peered":"2024-09-10T18:57:02.445268+0000","last_unstale":"2024-09-10T18:59:06.244638+0000","last_undegraded":"2024-09-10T18:59:06.244638+0000","last_fullsized":"2024-09-10T18:59:06.244638+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'10","last_scrub_stamp":"2024-09-10T18:55:36.732908+0000","last_deep_scrub":"81'9","last_deep_scrub_stamp":"2024-09-10T18:46:54.800790+0000","last_clean_scrub_stamp":"2024-09-10T18:55:36.732908+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016257330000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":326,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.485003+0000","last_change":"2024-09-10T18:56:20.584899+0000","last_active":"2024-09-10T18:58:58.485003+0000","last_peered":"2024-09-10T18:58:58.485003+0000","last_clean":"2024-09-10T18:58:58.485003+0000","last_became_active":"2024-09-10T18:55:40.391662+0000","last_became_peered":"2024-09-10T18:55:40.391662+0000","last_unstale":"2024-09-10T18:58:58.485003+0000","last_undegraded":"2024-09-10T18:58:58.485003+0000","last_fullsized":"2024-09-10T18:58:58.485003+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:20.584808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_clean_scrub_stamp":"2024-09-10T18:56:20.584808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0035961249999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"232'15","reported_seq":369,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.492071+0000","last_change":"2024-09-10T18:56:02.500313+0000","last_active":"2024-09-10T18:58:58.492071+0000","last_peered":"2024-09-10T18:58:58.492071+0000","last_clean":"2024-09-10T18:58:58.492071+0000","last_became_active":"2024-09-10T18:55:40.400788+0000","last_became_peered":"2024-09-10T18:55:40.400788+0000","last_unstale":"2024-09-10T18:58:58.492071+0000","last_undegraded":"2024-09-10T18:58:58.492071+0000","last_fullsized":"2024-09-10T18:58:58.492071+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T18:56:02.500101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:02.500101+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0097839320000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":337,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.483249+0000","last_change":"2024-09-10T18:57:02.453289+0000","last_active":"2024-09-10T18:58:58.483249+0000","last_peered":"2024-09-10T18:58:58.483249+0000","last_clean":"2024-09-10T18:58:58.483249+0000","last_became_active":"2024-09-10T18:57:02.452986+0000","last_became_peered":"2024-09-10T18:57:02.452986+0000","last_unstale":"2024-09-10T18:58:58.483249+0000","last_undegraded":"2024-09-10T18:58:58.483249+0000","last_fullsized":"2024-09-10T18:58:58.483249+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:18.272126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:18.272126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0035259570000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":310,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.494914+0000","last_change":"2024-09-10T18:56:04.753218+0000","last_active":"2024-09-10T18:58:58.494914+0000","last_peered":"2024-09-10T18:58:58.494914+0000","last_clean":"2024-09-10T18:58:58.494914+0000","last_became_active":"2024-09-10T18:55:40.398243+0000","last_became_peered":"2024-09-10T18:55:40.398243+0000","last_unstale":"2024-09-10T18:58:58.494914+0000","last_undegraded":"2024-09-10T18:58:58.494914+0000","last_fullsized":"2024-09-10T18:58:58.494914+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:04.753133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:03.571786+0000","last_clean_scrub_stamp":"2024-09-10T18:56:04.753133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0069575230000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":278,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.489414+0000","last_change":"2024-09-10T18:56:04.126848+0000","last_active":"2024-09-10T18:58:58.489414+0000","last_peered":"2024-09-10T18:58:58.489414+0000","last_clean":"2024-09-10T18:58:58.489414+0000","last_became_active":"2024-09-10T18:55:40.396932+0000","last_became_peered":"2024-09-10T18:55:40.396932+0000","last_unstale":"2024-09-10T18:58:58.489414+0000","last_undegraded":"2024-09-10T18:58:58.489414+0000","last_fullsized":"2024-09-10T18:58:58.489414+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:04.126516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:04.631558+0000","last_clean_scrub_stamp":"2024-09-10T18:56:04.126516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010034405999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"232'24","reported_seq":339,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.698169+0000","last_change":"2024-09-10T18:58:58.698097+0000","last_active":"2024-09-10T18:58:58.698169+0000","last_peered":"2024-09-10T18:58:58.698169+0000","last_clean":"2024-09-10T18:58:58.698169+0000","last_became_active":"2024-09-10T18:45:20.056885+0000","last_became_peered":"2024-09-10T18:45:20.056885+0000","last_unstale":"2024-09-10T18:58:58.698169+0000","last_undegraded":"2024-09-10T18:58:58.698169+0000","last_fullsized":"2024-09-10T18:58:58.698169+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T18:58:58.698003+0000","last_deep_scrub":"223'16","last_deep_scrub_stamp":"2024-09-10T18:54:47.324044+0000","last_clean_scrub_stamp":"2024-09-10T18:58:58.698003+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:15.915468+0000","scrub_duration":0.006955676,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":324,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.491984+0000","last_change":"2024-09-10T18:58:51.138423+0000","last_active":"2024-09-10T18:58:58.491984+0000","last_peered":"2024-09-10T18:58:58.491984+0000","last_clean":"2024-09-10T18:58:58.491984+0000","last_became_active":"2024-09-10T18:55:40.399198+0000","last_became_peered":"2024-09-10T18:55:40.399198+0000","last_unstale":"2024-09-10T18:58:58.491984+0000","last_undegraded":"2024-09-10T18:58:58.491984+0000","last_fullsized":"2024-09-10T18:58:58.491984+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:51.138240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:00.029389+0000","last_clean_scrub_stamp":"2024-09-10T18:58:51.138240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:01.134622+0000","scrub_duration":0.0086738059999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":306,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.491920+0000","last_change":"2024-09-10T18:56:31.783026+0000","last_active":"2024-09-10T18:58:58.491920+0000","last_peered":"2024-09-10T18:58:58.491920+0000","last_clean":"2024-09-10T18:58:58.491920+0000","last_became_active":"2024-09-10T18:45:22.070065+0000","last_became_peered":"2024-09-10T18:45:22.070065+0000","last_unstale":"2024-09-10T18:58:58.491920+0000","last_undegraded":"2024-09-10T18:58:58.491920+0000","last_fullsized":"2024-09-10T18:58:58.491920+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:31.782918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:31.782918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043032130000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":308,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.494550+0000","last_change":"2024-09-10T18:56:06.792096+0000","last_active":"2024-09-10T18:58:58.494550+0000","last_peered":"2024-09-10T18:58:58.494550+0000","last_clean":"2024-09-10T18:58:58.494550+0000","last_became_active":"2024-09-10T18:55:40.395595+0000","last_became_peered":"2024-09-10T18:55:40.395595+0000","last_unstale":"2024-09-10T18:58:58.494550+0000","last_undegraded":"2024-09-10T18:58:58.494550+0000","last_fullsized":"2024-09-10T18:58:58.494550+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:06.791860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:06.791860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046852380000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"232'8","reported_seq":317,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.496977+0000","last_change":"2024-09-10T18:56:30.802774+0000","last_active":"2024-09-10T18:58:58.496977+0000","last_peered":"2024-09-10T18:58:58.496977+0000","last_clean":"2024-09-10T18:58:58.496977+0000","last_became_active":"2024-09-10T18:45:20.059205+0000","last_became_peered":"2024-09-10T18:45:20.059205+0000","last_unstale":"2024-09-10T18:58:58.496977+0000","last_undegraded":"2024-09-10T18:58:58.496977+0000","last_fullsized":"2024-09-10T18:58:58.496977+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'8","last_scrub_stamp":"2024-09-10T18:56:30.802679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:30.802679+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034579609999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":323,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.486802+0000","last_change":"2024-09-10T18:58:53.527829+0000","last_active":"2024-09-10T18:58:58.486802+0000","last_peered":"2024-09-10T18:58:58.486802+0000","last_clean":"2024-09-10T18:58:58.486802+0000","last_became_active":"2024-09-10T18:47:36.415866+0000","last_became_peered":"2024-09-10T18:47:36.415866+0000","last_unstale":"2024-09-10T18:58:58.486802+0000","last_undegraded":"2024-09-10T18:58:58.486802+0000","last_fullsized":"2024-09-10T18:58:58.486802+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:53.527746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:58:53.527746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:14.990880+0000","scrub_duration":0.0058191349999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":333,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.484112+0000","last_change":"2024-09-10T18:57:02.454548+0000","last_active":"2024-09-10T18:58:58.484112+0000","last_peered":"2024-09-10T18:58:58.484112+0000","last_clean":"2024-09-10T18:58:58.484112+0000","last_became_active":"2024-09-10T18:57:02.453974+0000","last_became_peered":"2024-09-10T18:57:02.453974+0000","last_unstale":"2024-09-10T18:58:58.484112+0000","last_undegraded":"2024-09-10T18:58:58.484112+0000","last_fullsized":"2024-09-10T18:58:58.484112+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:23.195060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_clean_scrub_stamp":"2024-09-10T18:56:23.195060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047613259999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":94,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.482848+0000","last_change":"2024-09-10T18:57:02.447992+0000","last_active":"2024-09-10T18:58:58.482848+0000","last_peered":"2024-09-10T18:58:58.482848+0000","last_clean":"2024-09-10T18:58:58.482848+0000","last_became_active":"2024-09-10T18:57:02.447566+0000","last_became_peered":"2024-09-10T18:57:02.447566+0000","last_unstale":"2024-09-10T18:58:58.482848+0000","last_undegraded":"2024-09-10T18:58:58.482848+0000","last_fullsized":"2024-09-10T18:58:58.482848+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:28.768091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:55.145638+0000","last_clean_scrub_stamp":"2024-09-10T18:55:28.768091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020285519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.14","version":"70'81","reported_seq":205,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.484718+0000","last_change":"2024-09-10T18:57:02.442850+0000","last_active":"2024-09-10T18:58:58.484718+0000","last_peered":"2024-09-10T18:58:58.484718+0000","last_clean":"2024-09-10T18:58:58.484718+0000","last_became_active":"2024-09-10T18:57:02.442457+0000","last_became_peered":"2024-09-10T18:57:02.442457+0000","last_unstale":"2024-09-10T18:58:58.484718+0000","last_undegraded":"2024-09-10T18:58:58.484718+0000","last_fullsized":"2024-09-10T18:58:58.484718+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:55:34.791223+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:57.184383+0000","last_clean_scrub_stamp":"2024-09-10T18:55:34.791223+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016709736999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17890,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":7,"num_write":3,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":35780,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.11","version":"232'24","reported_seq":363,"reported_epoch":305,"state":"active+clean","last_fresh":"2024-09-10T18:59:10.879726+0000","last_change":"2024-09-10T18:59:10.879686+0000","last_active":"2024-09-10T18:59:10.879726+0000","last_peered":"2024-09-10T18:59:10.879726+0000","last_clean":"2024-09-10T18:59:10.879726+0000","last_became_active":"2024-09-10T18:57:02.461909+0000","last_became_peered":"2024-09-10T18:57:02.461909+0000","last_unstale":"2024-09-10T18:59:10.879726+0000","last_undegraded":"2024-09-10T18:59:10.879726+0000","last_fullsized":"2024-09-10T18:59:10.879726+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T18:59:10.879597+0000","last_deep_scrub":"79'16","last_deep_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_clean_scrub_stamp":"2024-09-10T18:59:10.879597+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:35.960275+0000","scrub_duration":0.0090530790000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"56'1","reported_seq":338,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.484884+0000","last_change":"2024-09-10T18:56:25.211179+0000","last_active":"2024-09-10T18:58:58.484884+0000","last_peered":"2024-09-10T18:58:58.484884+0000","last_clean":"2024-09-10T18:58:58.484884+0000","last_became_active":"2024-09-10T18:47:36.421985+0000","last_became_peered":"2024-09-10T18:47:36.421985+0000","last_unstale":"2024-09-10T18:58:58.484884+0000","last_undegraded":"2024-09-10T18:58:58.484884+0000","last_fullsized":"2024-09-10T18:58:58.484884+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:56:25.211095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:25.211095+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025468880000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":48,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.482705+0000","last_change":"2024-09-10T18:57:02.435423+0000","last_active":"2024-09-10T18:58:58.482705+0000","last_peered":"2024-09-10T18:58:58.482705+0000","last_clean":"2024-09-10T18:58:58.482705+0000","last_became_active":"2024-09-10T18:57:02.435124+0000","last_became_peered":"2024-09-10T18:57:02.435124+0000","last_unstale":"2024-09-10T18:58:58.482705+0000","last_undegraded":"2024-09-10T18:58:58.482705+0000","last_fullsized":"2024-09-10T18:58:58.482705+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:12.852769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:12.852769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":318,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.494724+0000","last_change":"2024-09-10T18:56:09.903930+0000","last_active":"2024-09-10T18:58:58.494724+0000","last_peered":"2024-09-10T18:58:58.494724+0000","last_clean":"2024-09-10T18:58:58.494724+0000","last_became_active":"2024-09-10T18:55:40.397523+0000","last_became_peered":"2024-09-10T18:55:40.397523+0000","last_unstale":"2024-09-10T18:58:58.494724+0000","last_undegraded":"2024-09-10T18:58:58.494724+0000","last_fullsized":"2024-09-10T18:58:58.494724+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:09.903846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:09.903846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011721984,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"78'2","reported_seq":94,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.485687+0000","last_change":"2024-09-10T18:58:48.472078+0000","last_active":"2024-09-10T18:58:58.485687+0000","last_peered":"2024-09-10T18:58:58.485687+0000","last_clean":"2024-09-10T18:58:58.485687+0000","last_became_active":"2024-09-10T18:57:02.447198+0000","last_became_peered":"2024-09-10T18:57:02.447198+0000","last_unstale":"2024-09-10T18:58:58.485687+0000","last_undegraded":"2024-09-10T18:58:58.485687+0000","last_fullsized":"2024-09-10T18:58:58.485687+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'2","last_scrub_stamp":"2024-09-10T18:58:48.472024+0000","last_deep_scrub":"78'2","last_deep_scrub_stamp":"2024-09-10T18:46:37.581130+0000","last_clean_scrub_stamp":"2024-09-10T18:58:48.472024+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:12.653675+0000","scrub_duration":0.022088824,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":317,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.484011+0000","last_change":"2024-09-10T18:57:02.444124+0000","last_active":"2024-09-10T18:58:58.484011+0000","last_peered":"2024-09-10T18:58:58.484011+0000","last_clean":"2024-09-10T18:58:58.484011+0000","last_became_active":"2024-09-10T18:57:02.443353+0000","last_became_peered":"2024-09-10T18:57:02.443353+0000","last_unstale":"2024-09-10T18:58:58.484011+0000","last_undegraded":"2024-09-10T18:58:58.484011+0000","last_fullsized":"2024-09-10T18:58:58.484011+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:16.743764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:16.743764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0031221030000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"232'20","reported_seq":332,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.485740+0000","last_change":"2024-09-10T18:57:02.486956+0000","last_active":"2024-09-10T18:58:58.485740+0000","last_peered":"2024-09-10T18:58:58.485740+0000","last_clean":"2024-09-10T18:58:58.485740+0000","last_became_active":"2024-09-10T18:57:02.452473+0000","last_became_peered":"2024-09-10T18:57:02.452473+0000","last_unstale":"2024-09-10T18:58:58.485740+0000","last_undegraded":"2024-09-10T18:58:58.485740+0000","last_fullsized":"2024-09-10T18:58:58.485740+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'20","last_scrub_stamp":"2024-09-10T18:56:27.158686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:27.158686+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0090371180000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":23,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"273'10","reported_seq":506,"reported_epoch":305,"state":"active+clean","last_fresh":"2024-09-10T18:59:06.238409+0000","last_change":"2024-09-10T18:57:02.474747+0000","last_active":"2024-09-10T18:59:06.238409+0000","last_peered":"2024-09-10T18:59:06.238409+0000","last_clean":"2024-09-10T18:59:06.238409+0000","last_became_active":"2024-09-10T18:57:02.441763+0000","last_became_peered":"2024-09-10T18:57:02.441763+0000","last_unstale":"2024-09-10T18:59:06.238409+0000","last_undegraded":"2024-09-10T18:59:06.238409+0000","last_fullsized":"2024-09-10T18:59:06.238409+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"190'9","last_scrub_stamp":"2024-09-10T18:56:19.259869+0000","last_deep_scrub":"190'9","last_deep_scrub_stamp":"2024-09-10T18:53:07.746309+0000","last_clean_scrub_stamp":"2024-09-10T18:56:19.259869+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020087130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":304,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:59.150840+0000","last_change":"2024-09-10T18:58:59.150773+0000","last_active":"2024-09-10T18:58:59.150840+0000","last_peered":"2024-09-10T18:58:59.150840+0000","last_clean":"2024-09-10T18:58:59.150840+0000","last_became_active":"2024-09-10T18:45:24.082318+0000","last_became_peered":"2024-09-10T18:45:24.082318+0000","last_unstale":"2024-09-10T18:58:59.150840+0000","last_undegraded":"2024-09-10T18:58:59.150840+0000","last_fullsized":"2024-09-10T18:58:59.150840+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:59.148958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:03.475299+0000","last_clean_scrub_stamp":"2024-09-10T18:58:59.148958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:15.018567+0000","scrub_duration":0.0041797229999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":314,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.493807+0000","last_change":"2024-09-10T18:56:00.878293+0000","last_active":"2024-09-10T18:58:58.493807+0000","last_peered":"2024-09-10T18:58:58.493807+0000","last_clean":"2024-09-10T18:58:58.493807+0000","last_became_active":"2024-09-10T18:55:40.392524+0000","last_became_peered":"2024-09-10T18:55:40.392524+0000","last_unstale":"2024-09-10T18:58:58.493807+0000","last_undegraded":"2024-09-10T18:58:58.493807+0000","last_fullsized":"2024-09-10T18:58:58.493807+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:00.877849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:00.877849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0096660570000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"232'19","reported_seq":349,"reported_epoch":305,"state":"active+clean","last_fresh":"2024-09-10T18:59:09.544317+0000","last_change":"2024-09-10T18:59:09.544279+0000","last_active":"2024-09-10T18:59:09.544317+0000","last_peered":"2024-09-10T18:59:09.544317+0000","last_clean":"2024-09-10T18:59:09.544317+0000","last_became_active":"2024-09-10T18:45:20.058717+0000","last_became_peered":"2024-09-10T18:45:20.058717+0000","last_unstale":"2024-09-10T18:59:09.544317+0000","last_undegraded":"2024-09-10T18:59:09.544317+0000","last_fullsized":"2024-09-10T18:59:09.544317+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T18:59:09.544194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:59:09.544194+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:10.025449+0000","scrub_duration":0.0043401830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":276,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.486387+0000","last_change":"2024-09-10T18:57:02.454464+0000","last_active":"2024-09-10T18:58:58.486387+0000","last_peered":"2024-09-10T18:58:58.486387+0000","last_clean":"2024-09-10T18:58:58.486387+0000","last_became_active":"2024-09-10T18:57:02.453464+0000","last_became_peered":"2024-09-10T18:57:02.453464+0000","last_unstale":"2024-09-10T18:58:58.486387+0000","last_undegraded":"2024-09-10T18:58:58.486387+0000","last_fullsized":"2024-09-10T18:58:58.486387+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:33.248024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:33.248024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060527929999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":299,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.485580+0000","last_change":"2024-09-10T18:58:55.726863+0000","last_active":"2024-09-10T18:58:58.485580+0000","last_peered":"2024-09-10T18:58:58.485580+0000","last_clean":"2024-09-10T18:58:58.485580+0000","last_became_active":"2024-09-10T18:45:24.267251+0000","last_became_peered":"2024-09-10T18:45:24.267251+0000","last_unstale":"2024-09-10T18:58:58.485580+0000","last_undegraded":"2024-09-10T18:58:58.485580+0000","last_fullsized":"2024-09-10T18:58:58.485580+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:55.726799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:58:55.726799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:20.336523+0000","scrub_duration":0.001154721,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"56'1","reported_seq":333,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.483549+0000","last_change":"2024-09-10T18:57:02.449727+0000","last_active":"2024-09-10T18:58:58.483549+0000","last_peered":"2024-09-10T18:58:58.483549+0000","last_clean":"2024-09-10T18:58:58.483549+0000","last_became_active":"2024-09-10T18:57:02.435758+0000","last_became_peered":"2024-09-10T18:57:02.435758+0000","last_unstale":"2024-09-10T18:58:58.483549+0000","last_undegraded":"2024-09-10T18:58:58.483549+0000","last_fullsized":"2024-09-10T18:58:58.483549+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:56:05.158970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:05.158970+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0055166130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"258'34","reported_seq":368,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.492830+0000","last_change":"2024-09-10T18:56:23.839043+0000","last_active":"2024-09-10T18:58:58.492830+0000","last_peered":"2024-09-10T18:58:58.492830+0000","last_clean":"2024-09-10T18:58:58.492830+0000","last_became_active":"2024-09-10T18:55:40.401788+0000","last_became_peered":"2024-09-10T18:55:40.401788+0000","last_unstale":"2024-09-10T18:58:58.492830+0000","last_undegraded":"2024-09-10T18:58:58.492830+0000","last_fullsized":"2024-09-10T18:58:58.492830+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'34","last_scrub_stamp":"2024-09-10T18:56:23.838951+0000","last_deep_scrub":"258'34","last_deep_scrub_stamp":"2024-09-10T18:56:23.838951+0000","last_clean_scrub_stamp":"2024-09-10T18:56:23.838951+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.035929888,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":310,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.491281+0000","last_change":"2024-09-10T18:56:13.611288+0000","last_active":"2024-09-10T18:58:58.491281+0000","last_peered":"2024-09-10T18:58:58.491281+0000","last_clean":"2024-09-10T18:58:58.491281+0000","last_became_active":"2024-09-10T18:55:40.397870+0000","last_became_peered":"2024-09-10T18:55:40.397870+0000","last_unstale":"2024-09-10T18:58:58.491281+0000","last_undegraded":"2024-09-10T18:58:58.491281+0000","last_fullsized":"2024-09-10T18:58:58.491281+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:13.610976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:13.610976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010999553,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":301,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.496103+0000","last_change":"2024-09-10T18:55:39.815156+0000","last_active":"2024-09-10T18:58:58.496103+0000","last_peered":"2024-09-10T18:58:58.496103+0000","last_clean":"2024-09-10T18:58:58.496103+0000","last_became_active":"2024-09-10T18:52:54.624256+0000","last_became_peered":"2024-09-10T18:52:54.624256+0000","last_unstale":"2024-09-10T18:58:58.496103+0000","last_undegraded":"2024-09-10T18:58:58.496103+0000","last_fullsized":"2024-09-10T18:58:58.496103+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:55:39.815087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:55:39.815087+0000","last_clean_scrub_stamp":"2024-09-10T18:55:39.815087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018555970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":327,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.495161+0000","last_change":"2024-09-10T18:58:57.919890+0000","last_active":"2024-09-10T18:58:58.495161+0000","last_peered":"2024-09-10T18:58:58.495161+0000","last_clean":"2024-09-10T18:58:58.495161+0000","last_became_active":"2024-09-10T18:57:02.460855+0000","last_became_peered":"2024-09-10T18:57:02.460855+0000","last_unstale":"2024-09-10T18:58:58.495161+0000","last_undegraded":"2024-09-10T18:58:58.495161+0000","last_fullsized":"2024-09-10T18:58:58.495161+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:57.919515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:58:57.919515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:20.230331+0000","scrub_duration":0.01355234,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"232'18","reported_seq":337,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.494986+0000","last_change":"2024-09-10T18:58:52.890712+0000","last_active":"2024-09-10T18:58:58.494986+0000","last_peered":"2024-09-10T18:58:58.494986+0000","last_clean":"2024-09-10T18:58:58.494986+0000","last_became_active":"2024-09-10T18:45:20.058391+0000","last_became_peered":"2024-09-10T18:45:20.058391+0000","last_unstale":"2024-09-10T18:58:58.494986+0000","last_undegraded":"2024-09-10T18:58:58.494986+0000","last_fullsized":"2024-09-10T18:58:58.494986+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'18","last_scrub_stamp":"2024-09-10T18:58:52.890618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:58:52.890618+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:21.226487+0000","scrub_duration":0.016012082,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":331,"reported_epoch":304,"state":"active+clean","last_fresh":"2024-09-10T18:58:58.484609+0000","last_change":"2024-09-10T18:58:56.690660+0000","last_active":"2024-09-10T18:58:58.484609+0000","last_peered":"2024-09-10T18:58:58.484609+0000","last_clean":"2024-09-10T18:58:58.484609+0000","last_became_active":"2024-09-10T18:57:02.449135+0000","last_became_peered":"2024-09-10T18:57:02.449135+0000","last_unstale":"2024-09-10T18:58:58.484609+0000","last_undegraded":"2024-09-10T18:58:58.484609+0000","last_fullsized":"2024-09-10T18:58:58.484609+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:56.690606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:58:56.690606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:20.364483+0000","scrub_duration":0.0060631599999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":1121452056,"num_objects":17113,"num_object_clones":0,"num_object_copies":51339,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17113,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":256387,"num_write_kb":8751937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75722,"num_bytes_recovered":4962385944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6186614784,"data_stored":6186598496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58483,"ondisk_log_size":58483,"up":48,"acting":48,"num_store_stats":7},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85,"ondisk_log_size":85,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1118,"num_read_kb":875,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":2712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":499712,"data_stored":13268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":758,"ondisk_log_size":758,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":817,"num_read_kb":812,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":688762,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":37,"num_write":18,"num_write_kb":271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":196788,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1701,"ondisk_log_size":1701,"up":63,"acting":63,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":0,"seq":223338299563,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":43,"seq":184683593904,"num_pgs":88,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1047328,"kb_used_data":453272,"kb_used_omap":1,"kb_used_meta":594046,"kb_avail":92697824,"statfs":{"total":95995035648,"available":94922571776,"internally_reserved":0,"allocated":464150528,"data_stored":463728355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":608303448},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:58:47 2024","interfaces":[{"interface":"back","average":{"1min":1.8540000000000001,"5min":1.111,"15min":0.96499999999999997},"min":{"1min":0.23000000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":27.545000000000002,"5min":27.545000000000002,"15min":27.545000000000002},"last":0.376},{"interface":"front","average":{"1min":1.881,"5min":1.2989999999999999,"15min":1.0589999999999999},"min":{"1min":0.26800000000000002,"5min":0.219,"15min":0.219},"max":{"1min":27.568000000000001,"5min":27.568000000000001,"15min":27.568000000000001},"last":0.52200000000000002}]},{"osd":1,"last update":"Tue Sep 10 18:58:59 2024","interfaces":[{"interface":"back","average":{"1min":4.7750000000000004,"5min":1.956,"15min":1.2470000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.219},"max":{"1min":45.311,"5min":45.311,"15min":45.311},"last":0.53500000000000003},{"interface":"front","average":{"1min":4.5620000000000003,"5min":1.841,"15min":1.1359999999999999},"min":{"1min":0.25900000000000001,"5min":0.23300000000000001,"15min":0.214},"max":{"1min":45.027999999999999,"5min":45.027999999999999,"15min":45.027999999999999},"last":0.438}]},{"osd":2,"last update":"Tue Sep 10 18:58:59 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.67200000000000004,"15min":0.74099999999999999},"min":{"1min":0.27200000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.60199999999999998,"5min":4.1029999999999998,"15min":4.4740000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.71499999999999997,"15min":0.68400000000000005},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.27200000000000002},"max":{"1min":1.6399999999999999,"5min":4.8150000000000004,"15min":4.8150000000000004},"last":0.39200000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:58:59 2024","interfaces":[{"interface":"back","average":{"1min":6.7110000000000003,"5min":2.5059999999999998,"15min":1.3620000000000001},"min":{"1min":0.29599999999999999,"5min":0.28399999999999997,"15min":0.25700000000000001},"max":{"1min":76.977999999999994,"5min":76.977999999999994,"15min":76.977999999999994},"last":0.76900000000000002},{"interface":"front","average":{"1min":6.8630000000000004,"5min":2.516,"15min":1.4850000000000001},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.253},"max":{"1min":76.879999999999995,"5min":76.879999999999995,"15min":76.879999999999995},"last":0.47699999999999998}]},{"osd":4,"last update":"Tue Sep 10 18:58:59 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.82799999999999996,"15min":1.274},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.20999999999999999},"max":{"1min":1.8899999999999999,"5min":5.335,"15min":165.03299999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.73699999999999999,"15min":1.1930000000000001},"min":{"1min":0.26000000000000001,"5min":0.255,"15min":0.22500000000000001},"max":{"1min":0.50600000000000001,"5min":4.9450000000000003,"15min":163.959},"last":0.40500000000000003}]},{"osd":5,"last update":"Tue Sep 10 18:58:59 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.81999999999999995,"15min":0.92700000000000005},"min":{"1min":0.28599999999999998,"5min":0.27100000000000002,"15min":0.24099999999999999},"max":{"1min":1.623,"5min":4.3849999999999998,"15min":43.811},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.93999999999999995,"15min":0.94499999999999995},"min":{"1min":0.29399999999999998,"5min":0.27300000000000002,"15min":0.23999999999999999},"max":{"1min":0.77900000000000003,"5min":5.5609999999999999,"15min":42.314},"last":0.42299999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:58:30 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.97399999999999998,"15min":0.79400000000000004},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.26200000000000001},"max":{"1min":1.355,"5min":5.2160000000000002,"15min":5.3680000000000003},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":1.1200000000000001,"15min":0.83699999999999997},"min":{"1min":0.29699999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":2.3519999999999999,"5min":7.0350000000000001,"15min":7.0350000000000001},"last":0.496}]}]},{"osd":5,"up_from":37,"seq":158913790132,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1130072,"kb_used_data":540176,"kb_used_omap":1,"kb_used_meta":589886,"kb_avail":92615080,"statfs":{"total":95995035648,"available":94837841920,"internally_reserved":0,"allocated":553140224,"data_stored":552724532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":604043608},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:58:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":3.113,"15min":2.0699999999999998},"min":{"1min":0.25900000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":2.056,"5min":82.465000000000003,"15min":82.465000000000003},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":3.2530000000000001,"15min":2.2650000000000001},"min":{"1min":0.33200000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.0670000000000002,"5min":82.519000000000005,"15min":82.519000000000005},"last":0.80300000000000005}]},{"osd":1,"last update":"Tue Sep 10 18:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":2.8690000000000002,"15min":1.4750000000000001},"min":{"1min":0.251,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":3.5840000000000001,"5min":67.582999999999998,"15min":67.582999999999998},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.85299999999999998,"5min":2.9649999999999999,"15min":1.623},"min":{"1min":0.308,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":3.6459999999999999,"5min":67.603999999999999,"15min":67.603999999999999},"last":0.36399999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.88,"15min":0.72299999999999998},"min":{"1min":0.29999999999999999,"5min":0.28899999999999998,"15min":0.249},"max":{"1min":1.579,"5min":4.4969999999999999,"15min":7.8029999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81599999999999995,"15min":0.748},"min":{"1min":0.32800000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":2.1629999999999998,"5min":4.1550000000000002,"15min":6.7210000000000001},"last":0.27100000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":4.9260000000000002,"15min":2.653},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26000000000000001},"max":{"1min":2.5630000000000002,"5min":123.34399999999999,"15min":123.34399999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":3.2890000000000001,"15min":2.2559999999999998},"min":{"1min":0.34000000000000002,"5min":0.30199999999999999,"15min":0.245},"max":{"1min":1.1519999999999999,"5min":82.415000000000006,"15min":82.415000000000006},"last":1.3480000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:58:42 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":1.21,"15min":0.82599999999999996},"min":{"1min":0.26400000000000001,"5min":0.23999999999999999,"15min":0.20899999999999999},"max":{"1min":0.86599999999999999,"5min":9.0310000000000006,"15min":9.0310000000000006},"last":0.312},{"interface":"front","average":{"1min":0.36299999999999999,"5min":1.0600000000000001,"15min":0.75800000000000001},"min":{"1min":0.27100000000000002,"5min":0.255,"15min":0.23400000000000001},"max":{"1min":0.47899999999999998,"5min":11.182,"15min":11.182},"last":0.28399999999999997}]},{"osd":6,"last update":"Tue Sep 10 18:59:02 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.85799999999999998,"15min":0.91600000000000004},"min":{"1min":0.23999999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.89800000000000002,"5min":9.0069999999999997,"15min":11.17},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.81599999999999995,"15min":0.91100000000000003},"min":{"1min":0.32200000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.871,"5min":7.3719999999999999,"15min":11.18},"last":0.41999999999999998}]},{"osd":7,"last update":"Tue Sep 10 18:58:25 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":1.22,"15min":0.91000000000000003},"min":{"1min":0.313,"5min":0.29499999999999998,"15min":0.27600000000000002},"max":{"1min":1.117,"5min":5.7249999999999996,"15min":10.782999999999999},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.98799999999999999,"15min":0.79400000000000004},"min":{"1min":0.33300000000000002,"5min":0.24099999999999999,"15min":0.23599999999999999},"max":{"1min":1.105,"5min":5.9500000000000002,"15min":11.846},"last":0.377}]}]},{"osd":4,"up_from":30,"seq":128849019066,"num_pgs":83,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":973472,"kb_used_data":379416,"kb_used_omap":1,"kb_used_meta":594046,"kb_avail":92771680,"statfs":{"total":95995035648,"available":94998200320,"internally_reserved":0,"allocated":388521984,"data_stored":388099401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":608303448},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:58:46 2024","interfaces":[{"interface":"back","average":{"1min":2.6030000000000002,"5min":1.5309999999999999,"15min":1.468},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":46.234999999999999,"5min":46.234999999999999,"15min":46.234999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":2.673,"5min":1.5740000000000001,"15min":1.502},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":46.264000000000003,"5min":46.264000000000003,"15min":46.264000000000003},"last":0.60999999999999999}]},{"osd":1,"last update":"Tue Sep 10 18:58:24 2024","interfaces":[{"interface":"back","average":{"1min":7.9329999999999998,"5min":3.0049999999999999,"15min":1.5349999999999999},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":125.15600000000001,"5min":125.15600000000001,"15min":125.15600000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":8.0329999999999995,"5min":2.9900000000000002,"15min":1.474},"min":{"1min":0.35399999999999998,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":125.14700000000001,"5min":125.14700000000001,"15min":125.14700000000001},"last":0.23699999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:58:24 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.746,"15min":0.66000000000000003},"min":{"1min":0.28799999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.2669999999999999,"5min":5.6189999999999998,"15min":5.6189999999999998},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.79200000000000004,"15min":0.69399999999999995},"min":{"1min":0.32100000000000001,"5min":0.28000000000000003,"15min":0.27100000000000002},"max":{"1min":1.6639999999999999,"5min":6.3540000000000001,"15min":6.3540000000000001},"last":0.32500000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:58:24 2024","interfaces":[{"interface":"back","average":{"1min":9.6590000000000007,"5min":3.5270000000000001,"15min":1.837},"min":{"1min":0.311,"5min":0.249,"15min":0.23799999999999999},"max":{"1min":111.514,"5min":111.514,"15min":111.514},"last":0.28299999999999997},{"interface":"front","average":{"1min":7.1369999999999996,"5min":2.6899999999999999,"15min":1.506},"min":{"1min":0.27200000000000002,"5min":0.26900000000000002,"15min":0.25700000000000001},"max":{"1min":110.914,"5min":110.914,"15min":110.914},"last":0.38}]},{"osd":5,"last update":"Tue Sep 10 18:58:46 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.099,"15min":0.82499999999999996},"min":{"1min":0.23100000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":9.7309999999999999,"5min":10.912000000000001,"15min":10.912000000000001},"last":0.33700000000000002},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0649999999999999,"15min":0.77500000000000002},"min":{"1min":0.23799999999999999,"5min":0.219,"15min":0.219},"max":{"1min":10.147,"5min":10.147,"15min":10.147},"last":0.27000000000000002}]},{"osd":6,"last update":"Tue Sep 10 18:59:06 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.874,"15min":0.72299999999999998},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":1.4930000000000001,"5min":8.4030000000000005,"15min":8.4030000000000005},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.84199999999999997,"15min":0.79400000000000004},"min":{"1min":0.27500000000000002,"5min":0.26300000000000001,"15min":0.25800000000000001},"max":{"1min":0.52500000000000002,"5min":6.8710000000000004,"15min":7.7699999999999996},"last":0.30599999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:58:41 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":1.0409999999999999,"15min":0.80800000000000005},"min":{"1min":0.29099999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.516,"5min":6.4630000000000001,"15min":6.4630000000000001},"last":0.316},{"interface":"front","average":{"1min":0.48099999999999998,"5min":1.0009999999999999,"15min":0.77000000000000002},"min":{"1min":0.27500000000000002,"5min":0.251,"15min":0.251},"max":{"1min":1.6930000000000001,"5min":6.3949999999999996,"15min":6.3949999999999996},"last":0.35199999999999998}]}]},{"osd":3,"up_from":25,"seq":107374182591,"num_pgs":90,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1672100,"kb_used_data":770204,"kb_used_omap":1,"kb_used_meta":901886,"kb_avail":92073052,"statfs":{"total":95995035648,"available":94282805248,"internally_reserved":0,"allocated":788688896,"data_stored":788261657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":923531608},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:58:40 2024","interfaces":[{"interface":"back","average":{"1min":18.138000000000002,"5min":4.8159999999999998,"15min":2.452},"min":{"1min":0.26000000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":290.721,"5min":290.721,"15min":290.721},"last":0.49299999999999999},{"interface":"front","average":{"1min":18.018000000000001,"5min":4.9950000000000001,"15min":2.6749999999999998},"min":{"1min":0.27600000000000002,"5min":0.252,"15min":0.252},"max":{"1min":293.59300000000002,"5min":293.59300000000002,"15min":293.59300000000002},"last":2.4740000000000002}]},{"osd":1,"last update":"Tue Sep 10 18:59:00 2024","interfaces":[{"interface":"back","average":{"1min":2.444,"5min":1.9239999999999999,"15min":1.4830000000000001},"min":{"1min":0.46100000000000002,"5min":0.16200000000000001,"15min":0.16200000000000001},"max":{"1min":7.4000000000000004,"5min":10.417999999999999,"15min":58.215000000000003},"last":0.56299999999999994},{"interface":"front","average":{"1min":2.4670000000000001,"5min":1.546,"15min":1.385},"min":{"1min":0.20000000000000001,"5min":0.188,"15min":0.188},"max":{"1min":6.7850000000000001,"5min":7.2309999999999999,"15min":57.896999999999998},"last":2.5819999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:59:00 2024","interfaces":[{"interface":"back","average":{"1min":2.1469999999999998,"5min":1.151,"15min":0.90100000000000002},"min":{"1min":0.36599999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":5.6680000000000001,"5min":5.6680000000000001,"15min":5.8220000000000001},"last":2.5150000000000001},{"interface":"front","average":{"1min":1.6479999999999999,"5min":1.075,"15min":0.82399999999999995},"min":{"1min":0.22600000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":5.1399999999999997,"5min":5.1399999999999997,"15min":7.0979999999999999},"last":2.7090000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:58:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.5680000000000001,"15min":1.1020000000000001},"min":{"1min":0.26400000000000001,"5min":0.247,"15min":0.247},"max":{"1min":4.7549999999999999,"5min":29.013000000000002,"15min":29.013000000000002},"last":2.7509999999999999},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.6519999999999999,"15min":1.1140000000000001},"min":{"1min":0.28000000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":4.7800000000000002,"5min":28.826000000000001,"15min":28.826000000000001},"last":2.7829999999999999}]},{"osd":5,"last update":"Tue Sep 10 18:58:51 2024","interfaces":[{"interface":"back","average":{"1min":1.869,"5min":1.694,"15min":1.1679999999999999},"min":{"1min":0.379,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":7.2300000000000004,"5min":7.2300000000000004,"15min":7.2300000000000004},"last":0.52200000000000002},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.4139999999999999,"15min":1.111},"min":{"1min":0.27100000000000002,"5min":0.255,"15min":0.255},"max":{"1min":3.774,"5min":6.1870000000000003,"15min":9.2509999999999994},"last":2.8740000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:59:07 2024","interfaces":[{"interface":"back","average":{"1min":1.7729999999999999,"5min":1.399,"15min":1.1679999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":5.4480000000000004,"5min":5.4480000000000004,"15min":9.3239999999999998},"last":2.9060000000000001},{"interface":"front","average":{"1min":1.921,"5min":1.532,"15min":1.1830000000000001},"min":{"1min":0.46400000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":5.0789999999999997,"5min":5.2670000000000003,"15min":7.1559999999999997},"last":2.8460000000000001}]},{"osd":7,"last update":"Tue Sep 10 18:58:34 2024","interfaces":[{"interface":"back","average":{"1min":1.6950000000000001,"5min":1.7210000000000001,"15min":1.173},"min":{"1min":0.40500000000000003,"5min":0.35799999999999998,"15min":0.34899999999999998},"max":{"1min":6.1180000000000003,"5min":6.1180000000000003,"15min":6.1180000000000003},"last":0.54300000000000004},{"interface":"front","average":{"1min":1.502,"5min":1.554,"15min":1.1930000000000001},"min":{"1min":0.372,"5min":0.36899999999999999,"15min":0.33100000000000002},"max":{"1min":4.5460000000000003,"5min":6.6029999999999998,"15min":9.2949999999999999},"last":2.9279999999999999}]}]},{"osd":2,"up_from":0,"seq":77309411523,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834575047,"num_pgs":97,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1787968,"kb_used_data":884664,"kb_used_omap":1,"kb_used_meta":903294,"kb_avail":91957184,"statfs":{"total":95995035648,"available":94164156416,"internally_reserved":0,"allocated":905895936,"data_stored":905441402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":924973400},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,8],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:58:45 2024","interfaces":[{"interface":"back","average":{"1min":7.7569999999999997,"5min":2.7050000000000001,"15min":1.7330000000000001},"min":{"1min":0.20999999999999999,"5min":0.193,"15min":0.193},"max":{"1min":111.014,"5min":111.014,"15min":111.014},"last":0.72999999999999998},{"interface":"front","average":{"1min":8.2569999999999997,"5min":2.75,"15min":1.663},"min":{"1min":0.27400000000000002,"5min":0.217,"15min":0.217},"max":{"1min":111.072,"5min":111.072,"15min":111.072},"last":0.77600000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:58:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":0.83499999999999996,"15min":0.68300000000000005},"min":{"1min":0.23300000000000001,"5min":0.186,"15min":0.186},"max":{"1min":3.4900000000000002,"5min":4.1719999999999997,"15min":4.1719999999999997},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.78000000000000003,"15min":0.69899999999999995},"min":{"1min":0.24199999999999999,"5min":0.215,"15min":0.215},"max":{"1min":2.1829999999999998,"5min":5.6929999999999996,"15min":7.093},"last":0.65800000000000003}]},{"osd":3,"last update":"Tue Sep 10 18:59:02 2024","interfaces":[{"interface":"back","average":{"1min":6.5359999999999996,"5min":3.6059999999999999,"15min":1.9399999999999999},"min":{"1min":0.29399999999999998,"5min":0.185,"15min":0.185},"max":{"1min":92.069000000000003,"5min":92.069000000000003,"15min":92.069000000000003},"last":0.59499999999999997},{"interface":"front","average":{"1min":6.851,"5min":3.2549999999999999,"15min":1.865},"min":{"1min":0.37,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":88.658000000000001,"5min":88.658000000000001,"15min":88.658000000000001},"last":2.8879999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:58:23 2024","interfaces":[{"interface":"back","average":{"1min":1.944,"5min":1.224,"15min":1.0820000000000001},"min":{"1min":0.30599999999999999,"5min":0.29799999999999999,"15min":0.26200000000000001},"max":{"1min":9.125,"5min":9.125,"15min":15.965999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":1.9330000000000001,"5min":1.2769999999999999,"15min":0.95899999999999996},"min":{"1min":0.252,"5min":0.252,"15min":0.246},"max":{"1min":7.0940000000000003,"5min":7.0940000000000003,"15min":7.0940000000000003},"last":0.66900000000000004}]},{"osd":5,"last update":"Tue Sep 10 18:58:45 2024","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.4279999999999999,"15min":1.099},"min":{"1min":0.308,"5min":0.28899999999999998,"15min":0.28499999999999998},"max":{"1min":6.4690000000000003,"5min":10.334,"15min":10.334},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.302,"5min":1.169,"15min":1.137},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":4.1890000000000001,"5min":7.9969999999999999,"15min":16.039999999999999},"last":0.46300000000000002}]},{"osd":6,"last update":"Tue Sep 10 18:59:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.3420000000000001,"15min":1.3839999999999999},"min":{"1min":0.34100000000000003,"5min":0.315,"15min":0.308},"max":{"1min":3.181,"5min":10.282999999999999,"15min":38.039999999999999},"last":0.626},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.399,"15min":1.5269999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":4.4829999999999997,"5min":10.404999999999999,"15min":38.100999999999999},"last":0.48799999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:58:32 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":1.3049999999999999,"15min":1.048},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.315},"max":{"1min":2.6600000000000001,"5min":5.6559999999999997,"15min":5.6559999999999997},"last":0.68400000000000005},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.1499999999999999,"15min":1.0189999999999999},"min":{"1min":0.39900000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":4.3369999999999997,"5min":4.9029999999999996,"15min":4.9029999999999996},"last":0.55400000000000005}]}]},{"osd":0,"up_from":241,"seq":1035087118380,"num_pgs":83,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":785308,"kb_used_data":447380,"kb_used_omap":4,"kb_used_meta":337915,"kb_avail":92959844,"statfs":{"total":95995035648,"available":95190880256,"internally_reserved":0,"allocated":458117120,"data_stored":457697431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":346025478},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 18:58:44 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":2.504,"15min":2.6880000000000002},"min":{"1min":0.17100000000000001,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":5.085,"5min":34.692,"15min":34.692},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.4339999999999999,"5min":2.4649999999999999,"15min":2.7010000000000001},"min":{"1min":0.23899999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":5.7320000000000002,"5min":34.588999999999999,"15min":34.588999999999999},"last":0.85999999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:58:44 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":1.0600000000000001,"15min":0.93799999999999994},"min":{"1min":0.222,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":2.7069999999999999,"5min":13.305,"15min":13.305},"last":0.5},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.085,"15min":0.82399999999999995},"min":{"1min":0.30599999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":6.5899999999999999,"5min":13.693,"15min":13.693},"last":0.45800000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:58:44 2024","interfaces":[{"interface":"back","average":{"1min":1.8180000000000001,"5min":1.4139999999999999,"15min":1.1180000000000001},"min":{"1min":0.35999999999999999,"5min":0.251,"15min":0.251},"max":{"1min":7.2350000000000003,"5min":21.108000000000001,"15min":21.108000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":1.877,"5min":2.0419999999999998,"15min":1.72},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":6.5309999999999997,"5min":11.629,"15min":11.629},"last":7.5140000000000002}]},{"osd":4,"last update":"Tue Sep 10 18:58:44 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.90800000000000003,"15min":0.78100000000000003},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":2.6560000000000001,"5min":13.161,"15min":13.161},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.94499999999999995,"5min":1.0009999999999999,"15min":0.93600000000000005},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.657,"5min":4.6020000000000003,"15min":4.6020000000000003},"last":0.64100000000000001}]},{"osd":5,"last update":"Tue Sep 10 18:58:44 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.113,"15min":1.044},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":2.621,"5min":4.5759999999999996,"15min":4.5759999999999996},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.83399999999999996,"5min":1.0189999999999999,"15min":0.85899999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":2.6190000000000002,"5min":13.202,"15min":13.202},"last":0.69299999999999995}]},{"osd":6,"last update":"Tue Sep 10 18:58:44 2024","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.038,"15min":0.95399999999999996},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":2.581,"5min":4.5309999999999997,"15min":4.5309999999999997},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.873,"5min":0.90700000000000003,"15min":0.76700000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":2.774,"5min":13.279,"15min":13.279},"last":0.755}]},{"osd":7,"last update":"Tue Sep 10 18:58:44 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":1.0449999999999999,"15min":0.89900000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":2.7320000000000002,"5min":13.233000000000001,"15min":13.233000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2370000000000001,"15min":1.177},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":5.7690000000000001,"5min":10.141999999999999,"15min":10.141999999999999},"last":0.59199999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":2668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":428998656,"data_stored":428998656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":865931264,"data_stored":865927192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":779943936,"data_stored":779943936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":370348032,"data_stored":370343960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":513019904,"data_stored":513015832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":433061888,"data_stored":433061888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2795311104,"data_stored":2795307032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T18:59:17.621 INFO:tasks.cephadm.ceph_manager.ceph:would drop below min_pgs, proj 6, currently 16 2024-09-10T18:59:17.698 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:17.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:17.749 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:17.749 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:17.850 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:17.888 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:17 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:59:17.888 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:17 smithi018 ceph-mon[40196]: 4.15 scrub starts 2024-09-10T18:59:17.888 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:17 smithi018 ceph-mon[40196]: 4.15 scrub ok 2024-09-10T18:59:17.888 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:17 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:17.888 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:17 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:59:17.888 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:17 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:59:17.888 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:17 smithi018 ceph-mon[40196]: pgmap v639: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 752 MiB data, 6.1 GiB used, 530 GiB / 536 GiB avail; 426 B/s rd, 0 B/s wr, 1.55k op/s 2024-09-10T18:59:17.889 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:17 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:59:17.889 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:17 smithi018 ceph-mon[29336]: 4.15 scrub starts 2024-09-10T18:59:17.889 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:17 smithi018 ceph-mon[29336]: 4.15 scrub ok 2024-09-10T18:59:17.889 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:17 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:17.889 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:17 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:59:17.889 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:17 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:59:17.889 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:17 smithi018 ceph-mon[29336]: pgmap v639: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 752 MiB data, 6.1 GiB used, 530 GiB / 536 GiB avail; 426 B/s rd, 0 B/s wr, 1.55k op/s 2024-09-10T18:59:17.902 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:17.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:17.903 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:18.004 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:18.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:17 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:59:18.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:17 smithi103 ceph-mon[33163]: 4.15 scrub starts 2024-09-10T18:59:18.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:17 smithi103 ceph-mon[33163]: 4.15 scrub ok 2024-09-10T18:59:18.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:17 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:18.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:17 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:59:18.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:17 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:59:18.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:17 smithi103 ceph-mon[33163]: pgmap v639: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 752 MiB data, 6.1 GiB used, 530 GiB / 536 GiB avail; 426 B/s rd, 0 B/s wr, 1.55k op/s 2024-09-10T18:59:18.045 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:18.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:18.046 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:18.146 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:18.173 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:18.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:18.174 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:18.274 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:18.303 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:18.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:18.304 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:18.405 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:18.431 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:18.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:18.432 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:18.533 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:18.557 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:18.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:18.558 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:18.659 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:18.683 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:18.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:18.684 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:18.698 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T18:59:18.785 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:18.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:18.811 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:18.811 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:18.912 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:18.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:18 smithi018 ceph-mon[40196]: 4.5 scrub starts 2024-09-10T18:59:18.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:18 smithi018 ceph-mon[40196]: 4.5 scrub ok 2024-09-10T18:59:18.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:18 smithi018 ceph-mon[40196]: 6.1f deep-scrub starts 2024-09-10T18:59:18.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:18 smithi018 ceph-mon[40196]: 6.1f deep-scrub ok 2024-09-10T18:59:18.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:18 smithi018 ceph-mon[29336]: 4.5 scrub starts 2024-09-10T18:59:18.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:18 smithi018 ceph-mon[29336]: 4.5 scrub ok 2024-09-10T18:59:18.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:18 smithi018 ceph-mon[29336]: 6.1f deep-scrub starts 2024-09-10T18:59:18.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:18 smithi018 ceph-mon[29336]: 6.1f deep-scrub ok 2024-09-10T18:59:18.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:18.952 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:18.952 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:19.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:18 smithi103 ceph-mon[33163]: 4.5 scrub starts 2024-09-10T18:59:19.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:18 smithi103 ceph-mon[33163]: 4.5 scrub ok 2024-09-10T18:59:19.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:18 smithi103 ceph-mon[33163]: 6.1f deep-scrub starts 2024-09-10T18:59:19.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:18 smithi103 ceph-mon[33163]: 6.1f deep-scrub ok 2024-09-10T18:59:19.052 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:19.095 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:19.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:19.096 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:19.196 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:19.241 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:19.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:19.242 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:19.343 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:19.385 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:19.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:19.386 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:19.487 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:19.510 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:19.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:19.511 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:19.611 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:19.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:19.660 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:19.660 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:19.761 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:19.787 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:19.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:19.788 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:19.888 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:19.909 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:19.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:19.911 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:20.012 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:20.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:19 smithi103 ceph-mon[33163]: pgmap v640: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 677 MiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 0 B/s wr, 1.19k op/s 2024-09-10T18:59:20.048 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:20.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:20.049 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:20.150 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:20.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:20.193 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:20.193 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:20.193 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:19 smithi018 ceph-mon[40196]: pgmap v640: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 677 MiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 0 B/s wr, 1.19k op/s 2024-09-10T18:59:20.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:19 smithi018 ceph-mon[29336]: pgmap v640: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 677 MiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 0 B/s wr, 1.19k op/s 2024-09-10T18:59:20.294 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:20.319 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:20.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:20.320 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:20.421 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:20.446 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:20.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:20.447 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:20.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:59:20 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:59:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:59:20.548 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:20.601 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:59:20.610 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:20.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:20.611 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:20.712 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:20.753 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:20.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:20.754 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:20.787 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:20 smithi018 ceph-mon[29336]: 3.1 deep-scrub starts 2024-09-10T18:59:20.788 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:20 smithi018 ceph-mon[29336]: 3.1 deep-scrub ok 2024-09-10T18:59:20.788 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:20 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:59:20.788 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:20 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:59:20.788 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:20 smithi018 ceph-mon[29336]: osdmap e306: 8 total, 8 up, 6 in 2024-09-10T18:59:20.788 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:20 smithi018 ceph-mon[29336]: 9.4 scrub starts 2024-09-10T18:59:20.788 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:20 smithi018 ceph-mon[29336]: 9.4 scrub ok 2024-09-10T18:59:20.788 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:20 smithi018 ceph-mon[40196]: 3.1 deep-scrub starts 2024-09-10T18:59:20.788 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:20 smithi018 ceph-mon[40196]: 3.1 deep-scrub ok 2024-09-10T18:59:20.789 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:20 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:59:20.789 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:20 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:59:20.789 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:20 smithi018 ceph-mon[40196]: osdmap e306: 8 total, 8 up, 6 in 2024-09-10T18:59:20.789 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:20 smithi018 ceph-mon[40196]: 9.4 scrub starts 2024-09-10T18:59:20.789 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:20 smithi018 ceph-mon[40196]: 9.4 scrub ok 2024-09-10T18:59:20.856 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:20.883 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:20.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:20.884 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:20.985 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:21.020 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:21.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:21.021 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:21.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:20 smithi103 ceph-mon[33163]: 3.1 deep-scrub starts 2024-09-10T18:59:21.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:20 smithi103 ceph-mon[33163]: 3.1 deep-scrub ok 2024-09-10T18:59:21.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:20 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:59:21.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:20 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:59:21.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:20 smithi103 ceph-mon[33163]: osdmap e306: 8 total, 8 up, 6 in 2024-09-10T18:59:21.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:20 smithi103 ceph-mon[33163]: 9.4 scrub starts 2024-09-10T18:59:21.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:20 smithi103 ceph-mon[33163]: 9.4 scrub ok 2024-09-10T18:59:21.122 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:21.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:21.145 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:21.146 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:21.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:59:20 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:59:20] "GET /metrics HTTP/1.1" 200 224467 "" "Prometheus/2.43.0" 2024-09-10T18:59:21.246 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:21.271 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:21.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:21.272 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:21.373 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:21.398 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:21.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:21.399 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:21.500 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:21.539 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:21.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:21.541 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:21.641 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:21.680 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:21.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:21.681 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:21.782 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:21.806 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:21.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:21.807 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:21.908 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:21.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:21.945 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:21.946 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:22.047 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:22.075 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:22.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:22.076 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:22.125 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:21 smithi018 ceph-mon[40196]: 2.1a deep-scrub starts 2024-09-10T18:59:22.125 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:21 smithi018 ceph-mon[40196]: 2.1a deep-scrub ok 2024-09-10T18:59:22.125 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:21 smithi018 ceph-mon[40196]: pgmap v642: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 649 MiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 0 B/s wr, 1.30k op/s 2024-09-10T18:59:22.125 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:21 smithi018 ceph-mon[29336]: 2.1a deep-scrub starts 2024-09-10T18:59:22.125 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:21 smithi018 ceph-mon[29336]: 2.1a deep-scrub ok 2024-09-10T18:59:22.126 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:21 smithi018 ceph-mon[29336]: pgmap v642: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 649 MiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 0 B/s wr, 1.30k op/s 2024-09-10T18:59:22.177 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:22.247 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:22.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:22.248 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:22.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:21 smithi103 ceph-mon[33163]: 2.1a deep-scrub starts 2024-09-10T18:59:22.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:21 smithi103 ceph-mon[33163]: 2.1a deep-scrub ok 2024-09-10T18:59:22.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:21 smithi103 ceph-mon[33163]: pgmap v642: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 649 MiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 0 B/s wr, 1.30k op/s 2024-09-10T18:59:22.349 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:22.387 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:22.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:22.388 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:22.489 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:22.528 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:22.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:22.529 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:22.621 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6] out_osds: [2, 7] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 6, 7, 0] 2024-09-10T18:59:22.622 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T18:59:22.622 INFO:tasks.thrashosds.thrasher:Reweighting osd 0 to 0.8862214262041657 2024-09-10T18:59:22.622 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd reweight 0 0.8862214262041657 2024-09-10T18:59:22.630 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:22.679 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:22.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:22.680 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:22.781 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:22.806 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:22.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:22.807 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:22.908 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:22.927 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T18:59:22.927 INFO:teuthology.orchestra.run.smithi018.stdout: "size": 20, 2024-09-10T18:59:22.927 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 600, 2024-09-10T18:59:22.927 INFO:teuthology.orchestra.run.smithi018.stdout: "ops": [ 2024-09-10T18:59:22.927 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.927 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.24947.0:147763 8.0 8:07a311f3:::benchmark_data_smithi018_106531_object1133:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e192)", 2024-09-10T18:59:22.928 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:53:20.159423+0000", 2024-09-10T18:59:22.928 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 362.76285324999998, 2024-09-10T18:59:22.928 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.0556502540000001, 2024-09-10T18:59:22.928 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:59:22.928 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "reached pg", 2024-09-10T18:59:22.928 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:59:22.928 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.24947", 2024-09-10T18:59:22.928 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:59:22.928 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 147763 2024-09-10T18:59:22.928 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.928 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:59:22.928 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.928 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:59:22.929 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.159423+0000", 2024-09-10T18:59:22.929 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:22.929 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.929 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.929 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:59:22.929 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.159423+0000", 2024-09-10T18:59:22.929 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:22.929 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.929 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.929 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:59:22.929 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.159421+0000", 2024-09-10T18:59:22.929 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T18:59:22.929 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.930 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.930 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:59:22.930 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.159427+0000", 2024-09-10T18:59:22.930 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.7339999999999996e-06 2024-09-10T18:59:22.930 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.930 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.930 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:59:22.930 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.159442+0000", 2024-09-10T18:59:22.930 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.4694e-05 2024-09-10T18:59:22.930 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.930 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.930 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:59:22.930 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.159447+0000", 2024-09-10T18:59:22.931 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.5849999999999999e-06 2024-09-10T18:59:22.931 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.931 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.931 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:22.931 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.159506+0000", 2024-09-10T18:59:22.931 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.8298000000000003e-05 2024-09-10T18:59:22.931 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.931 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.931 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:59:22.931 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.159605+0000", 2024-09-10T18:59:22.931 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.9652000000000001e-05 2024-09-10T18:59:22.931 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.931 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.932 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,5", 2024-09-10T18:59:22.932 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.159715+0000", 2024-09-10T18:59:22.932 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000109737 2024-09-10T18:59:22.932 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.932 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.932 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:22.932 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.162735+0000", 2024-09-10T18:59:22.932 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0030194950000000001 2024-09-10T18:59:22.932 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.932 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.932 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:59:22.932 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.165322+0000", 2024-09-10T18:59:22.932 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0025876530000000001 2024-09-10T18:59:22.933 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.933 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.933 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:22.933 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.178232+0000", 2024-09-10T18:59:22.933 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.012909532 2024-09-10T18:59:22.933 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.933 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.933 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T18:59:22.933 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.178241+0000", 2024-09-10T18:59:22.933 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.5670000000000001e-06 2024-09-10T18:59:22.933 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.933 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.933 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:22.934 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:21.163026+0000", 2024-09-10T18:59:22.934 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.98478463500000002 2024-09-10T18:59:22.934 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.934 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.934 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T18:59:22.934 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:21.163042+0000", 2024-09-10T18:59:22.934 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.5546e-05 2024-09-10T18:59:22.934 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.934 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.934 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:22.934 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.214983+0000", 2024-09-10T18:59:22.934 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.051941719 2024-09-10T18:59:22.934 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.935 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.935 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:59:22.935 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.215073+0000", 2024-09-10T18:59:22.935 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.9677000000000002e-05 2024-09-10T18:59:22.935 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:22.935 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:59:22.935 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:22.935 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.935 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.935 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.24947.0:147774 8.0 8:09df9c28:::benchmark_data_smithi018_106531_object1144:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e193)", 2024-09-10T18:59:22.935 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:53:20.171505+0000", 2024-09-10T18:59:22.935 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 362.75077122699997, 2024-09-10T18:59:22.935 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.0502639039999999, 2024-09-10T18:59:22.935 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:59:22.936 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:59:22.936 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:59:22.936 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.24947", 2024-09-10T18:59:22.936 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:59:22.936 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 147774 2024-09-10T18:59:22.936 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.936 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:59:22.936 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.936 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:59:22.936 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.171505+0000", 2024-09-10T18:59:22.936 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:22.936 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.936 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.937 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:59:22.937 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.171505+0000", 2024-09-10T18:59:22.937 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:22.937 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.937 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.937 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:59:22.937 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.171503+0000", 2024-09-10T18:59:22.937 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999981 2024-09-10T18:59:22.937 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.937 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.937 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:59:22.937 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.171510+0000", 2024-09-10T18:59:22.937 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.9410000000000001e-06 2024-09-10T18:59:22.937 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.938 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.938 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:59:22.938 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.171522+0000", 2024-09-10T18:59:22.938 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.2121e-05 2024-09-10T18:59:22.938 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.938 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.938 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:59:22.938 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.171528+0000", 2024-09-10T18:59:22.938 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.4189999999999999e-06 2024-09-10T18:59:22.938 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.938 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.938 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:22.938 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.178407+0000", 2024-09-10T18:59:22.938 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0068784390000000001 2024-09-10T18:59:22.939 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.939 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.939 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T18:59:22.939 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.178413+0000", 2024-09-10T18:59:22.939 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.7810000000000002e-06 2024-09-10T18:59:22.939 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.939 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.939 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:22.939 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:21.163073+0000", 2024-09-10T18:59:22.939 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.98465997999999999 2024-09-10T18:59:22.939 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.939 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.939 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T18:59:22.940 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:21.163080+0000", 2024-09-10T18:59:22.940 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.6279999999999996e-06 2024-09-10T18:59:22.940 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.940 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.940 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:22.940 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.215333+0000", 2024-09-10T18:59:22.940 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.052253103 2024-09-10T18:59:22.940 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.940 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.940 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:59:22.940 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.215423+0000", 2024-09-10T18:59:22.940 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.9197999999999995e-05 2024-09-10T18:59:22.940 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.941 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.941 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,4,5", 2024-09-10T18:59:22.941 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.215651+0000", 2024-09-10T18:59:22.941 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000228495 2024-09-10T18:59:22.941 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.941 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.941 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:59:22.941 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.221438+0000", 2024-09-10T18:59:22.941 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0057872879999999998 2024-09-10T18:59:22.941 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.941 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.941 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:22.941 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.221501+0000", 2024-09-10T18:59:22.942 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.2972999999999995e-05 2024-09-10T18:59:22.943 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.943 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.943 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:22.943 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.221557+0000", 2024-09-10T18:59:22.943 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.5408000000000002e-05 2024-09-10T18:59:22.943 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.943 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.943 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:22.943 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.221624+0000", 2024-09-10T18:59:22.943 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.7217000000000004e-05 2024-09-10T18:59:22.943 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.943 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.944 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:59:22.944 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.221682+0000", 2024-09-10T18:59:22.944 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.7720999999999998e-05 2024-09-10T18:59:22.944 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.944 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.944 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:59:22.944 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.221769+0000", 2024-09-10T18:59:22.944 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.6873000000000007e-05 2024-09-10T18:59:22.944 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:22.944 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:59:22.944 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:22.944 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.944 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.945 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.24947.0:147775 8.0 8:0cde921e:::benchmark_data_smithi018_106531_object1145:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e193)", 2024-09-10T18:59:22.945 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:53:20.171731+0000", 2024-09-10T18:59:22.945 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 362.75054500499999, 2024-09-10T18:59:22.945 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.0502509459999998, 2024-09-10T18:59:22.945 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:59:22.945 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:59:22.945 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:59:22.945 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.24947", 2024-09-10T18:59:22.945 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:59:22.945 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 147775 2024-09-10T18:59:22.945 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.945 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:59:22.945 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.946 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:59:22.946 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.171731+0000", 2024-09-10T18:59:22.946 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:22.946 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.946 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.946 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:59:22.946 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.171731+0000", 2024-09-10T18:59:22.946 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:22.946 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.946 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.946 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:59:22.946 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.171730+0000", 2024-09-10T18:59:22.946 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:59:22.946 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.947 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.947 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:59:22.947 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.171736+0000", 2024-09-10T18:59:22.947 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.7509999999999997e-06 2024-09-10T18:59:22.947 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.947 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.947 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:59:22.947 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.171746+0000", 2024-09-10T18:59:22.947 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0139999999999999e-05 2024-09-10T18:59:22.947 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.947 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.947 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:59:22.947 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.171752+0000", 2024-09-10T18:59:22.947 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.0360000000000003e-06 2024-09-10T18:59:22.948 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.948 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.948 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:22.948 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.178520+0000", 2024-09-10T18:59:22.948 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0067677079999999999 2024-09-10T18:59:22.948 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.948 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.948 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T18:59:22.948 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.178527+0000", 2024-09-10T18:59:22.948 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.8450000000000003e-06 2024-09-10T18:59:22.948 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.948 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.948 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:22.949 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:21.163224+0000", 2024-09-10T18:59:22.949 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.98469684400000002 2024-09-10T18:59:22.949 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.949 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.949 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T18:59:22.949 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:21.163233+0000", 2024-09-10T18:59:22.949 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.9789999999999999e-06 2024-09-10T18:59:22.949 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.949 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.949 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:22.949 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.217861+0000", 2024-09-10T18:59:22.949 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0546280320000001 2024-09-10T18:59:22.949 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.949 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.950 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:59:22.950 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.217997+0000", 2024-09-10T18:59:22.950 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00013612499999999999 2024-09-10T18:59:22.950 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.950 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.950 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,4,5", 2024-09-10T18:59:22.950 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.218137+0000", 2024-09-10T18:59:22.950 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000139667 2024-09-10T18:59:22.950 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.950 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.950 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:59:22.950 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.221567+0000", 2024-09-10T18:59:22.950 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0034302249999999999 2024-09-10T18:59:22.950 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.950 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.951 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:22.951 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.221813+0000", 2024-09-10T18:59:22.951 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00024533600000000002 2024-09-10T18:59:22.951 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.951 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.951 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:22.951 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.221847+0000", 2024-09-10T18:59:22.951 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.472e-05 2024-09-10T18:59:22.951 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.951 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.951 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:22.951 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.221880+0000", 2024-09-10T18:59:22.951 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.2762999999999998e-05 2024-09-10T18:59:22.951 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.952 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.952 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:59:22.952 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.221910+0000", 2024-09-10T18:59:22.952 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.0236e-05 2024-09-10T18:59:22.952 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.952 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.952 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:59:22.952 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.221982+0000", 2024-09-10T18:59:22.952 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.1450000000000002e-05 2024-09-10T18:59:22.952 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:22.952 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:59:22.952 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:22.952 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.953 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.953 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.24947.0:147853 8.0 8:0a426a63:::benchmark_data_smithi018_106531_object1223:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e193)", 2024-09-10T18:59:22.953 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:53:20.229784+0000", 2024-09-10T18:59:22.953 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 362.69249196099997, 2024-09-10T18:59:22.953 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.998631295, 2024-09-10T18:59:22.953 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:59:22.953 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:59:22.953 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:59:22.953 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.24947", 2024-09-10T18:59:22.953 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:59:22.953 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 147853 2024-09-10T18:59:22.953 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.953 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:59:22.953 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.953 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:59:22.954 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.229784+0000", 2024-09-10T18:59:22.954 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:22.954 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.954 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.954 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:59:22.954 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.229784+0000", 2024-09-10T18:59:22.954 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:22.954 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.954 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.954 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:59:22.954 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.229783+0000", 2024-09-10T18:59:22.954 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:59:22.954 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.954 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.955 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:59:22.955 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.229788+0000", 2024-09-10T18:59:22.955 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.0529999999999997e-06 2024-09-10T18:59:22.955 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.955 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.955 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:59:22.955 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.229799+0000", 2024-09-10T18:59:22.955 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0973e-05 2024-09-10T18:59:22.955 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.955 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.955 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:59:22.955 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.229805+0000", 2024-09-10T18:59:22.956 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.9939999999999997e-06 2024-09-10T18:59:22.956 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.956 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.956 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:22.956 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.229851+0000", 2024-09-10T18:59:22.956 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.6112000000000002e-05 2024-09-10T18:59:22.956 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.956 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.956 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T18:59:22.956 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.229858+0000", 2024-09-10T18:59:22.956 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.8650000000000003e-06 2024-09-10T18:59:22.956 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.956 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.957 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:22.957 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:21.163260+0000", 2024-09-10T18:59:22.957 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.93340213100000002 2024-09-10T18:59:22.957 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.957 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.957 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T18:59:22.957 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:21.163332+0000", 2024-09-10T18:59:22.957 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.1589999999999997e-05 2024-09-10T18:59:22.957 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.957 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.957 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:22.957 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.219807+0000", 2024-09-10T18:59:22.957 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.05647519 2024-09-10T18:59:22.957 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.957 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.958 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:59:22.958 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.219939+0000", 2024-09-10T18:59:22.958 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000132235 2024-09-10T18:59:22.958 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.958 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.958 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,4,5", 2024-09-10T18:59:22.958 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.220112+0000", 2024-09-10T18:59:22.958 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00017242000000000001 2024-09-10T18:59:22.958 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.958 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.958 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:59:22.958 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.221575+0000", 2024-09-10T18:59:22.958 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0014636149999999999 2024-09-10T18:59:22.959 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.959 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.959 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:22.959 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.222037+0000", 2024-09-10T18:59:22.959 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00046180400000000001 2024-09-10T18:59:22.959 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.959 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.959 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:22.959 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.228178+0000", 2024-09-10T18:59:22.959 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0061407759999999997 2024-09-10T18:59:22.959 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.959 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.959 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:22.960 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.228244+0000", 2024-09-10T18:59:22.960 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.6038999999999996e-05 2024-09-10T18:59:22.960 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.960 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.960 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:59:22.960 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.228300+0000", 2024-09-10T18:59:22.960 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.5807000000000003e-05 2024-09-10T18:59:22.960 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.960 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.960 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:59:22.960 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.228415+0000", 2024-09-10T18:59:22.960 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000115622 2024-09-10T18:59:22.960 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:22.961 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:59:22.961 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:22.961 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.961 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.961 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.24947.0:147855 8.0 8:0edd2ba6:::benchmark_data_smithi018_106531_object1225:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e193)", 2024-09-10T18:59:22.961 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:53:20.230013+0000", 2024-09-10T18:59:22.961 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 362.69226277299998, 2024-09-10T18:59:22.961 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.9987049720000001, 2024-09-10T18:59:22.961 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:59:22.961 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:59:22.961 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:59:22.961 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.24947", 2024-09-10T18:59:22.961 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:59:22.962 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 147855 2024-09-10T18:59:22.962 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.962 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:59:22.962 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.962 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:59:22.962 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.230013+0000", 2024-09-10T18:59:22.962 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:22.962 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.962 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.962 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:59:22.962 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.230013+0000", 2024-09-10T18:59:22.962 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:22.963 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.963 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.963 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:59:22.963 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.230012+0000", 2024-09-10T18:59:22.963 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:59:22.963 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.963 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.963 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:59:22.963 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.230017+0000", 2024-09-10T18:59:22.963 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.8029999999999996e-06 2024-09-10T18:59:22.963 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.963 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.963 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:59:22.963 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.230042+0000", 2024-09-10T18:59:22.964 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.5279e-05 2024-09-10T18:59:22.964 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.964 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.964 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:59:22.964 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.230049+0000", 2024-09-10T18:59:22.964 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.4200000000000004e-06 2024-09-10T18:59:22.964 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.964 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.964 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:22.964 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.230078+0000", 2024-09-10T18:59:22.964 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.9040999999999999e-05 2024-09-10T18:59:22.964 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.964 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.965 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T18:59:22.965 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.230085+0000", 2024-09-10T18:59:22.965 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.5360000000000004e-06 2024-09-10T18:59:22.965 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.965 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.965 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:22.965 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:21.163511+0000", 2024-09-10T18:59:22.965 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.93342508000000002 2024-09-10T18:59:22.965 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.965 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.965 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T18:59:22.965 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:21.163518+0000", 2024-09-10T18:59:22.965 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.6249999999999998e-06 2024-09-10T18:59:22.965 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.966 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.966 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:22.966 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.222067+0000", 2024-09-10T18:59:22.966 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0585492059999999 2024-09-10T18:59:22.966 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.966 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.966 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:59:22.966 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.222148+0000", 2024-09-10T18:59:22.966 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.0760000000000006e-05 2024-09-10T18:59:22.966 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.966 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.966 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,4,5", 2024-09-10T18:59:22.966 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.222227+0000", 2024-09-10T18:59:22.966 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.8949999999999995e-05 2024-09-10T18:59:22.967 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.967 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.967 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:22.967 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.225481+0000", 2024-09-10T18:59:22.967 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0032537030000000002 2024-09-10T18:59:22.967 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.967 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.967 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:59:22.967 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.225540+0000", 2024-09-10T18:59:22.967 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.8989000000000001e-05 2024-09-10T18:59:22.967 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.967 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.967 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:22.967 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.228506+0000", 2024-09-10T18:59:22.968 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0029657970000000001 2024-09-10T18:59:22.968 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.968 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.968 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:22.968 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.228567+0000", 2024-09-10T18:59:22.968 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.1639999999999999e-05 2024-09-10T18:59:22.968 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.968 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.968 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:59:22.968 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.228619+0000", 2024-09-10T18:59:22.968 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.1588999999999997e-05 2024-09-10T18:59:22.968 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.968 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.969 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:59:22.969 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.228718+0000", 2024-09-10T18:59:22.969 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.9319000000000002e-05 2024-09-10T18:59:22.969 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:22.969 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:59:22.969 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:22.969 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.969 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.969 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.24947.0:147859 8.0 8:0c272c4f:::benchmark_data_smithi018_106531_object1229:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e193)", 2024-09-10T18:59:22.969 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:53:20.233507+0000", 2024-09-10T18:59:22.969 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 362.68876916900001, 2024-09-10T18:59:22.969 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.9994049280000001, 2024-09-10T18:59:22.969 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:59:22.969 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:59:22.970 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:59:22.970 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.24947", 2024-09-10T18:59:22.970 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:59:22.970 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 147859 2024-09-10T18:59:22.970 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.970 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:59:22.970 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.970 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:59:22.970 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.233507+0000", 2024-09-10T18:59:22.970 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:22.970 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.970 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.970 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:59:22.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:22.971 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:22 smithi018 ceph-mon[40196]: 2.1d deep-scrub starts 2024-09-10T18:59:22.971 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:22 smithi018 ceph-mon[40196]: 2.1d deep-scrub ok 2024-09-10T18:59:22.971 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:22 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:59:22.971 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:22 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:59:22.971 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:22 smithi018 ceph-mon[29336]: 2.1d deep-scrub starts 2024-09-10T18:59:22.971 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:22 smithi018 ceph-mon[29336]: 2.1d deep-scrub ok 2024-09-10T18:59:22.971 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:22 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:59:22.972 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:22 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:59:22.972 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:22.972 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:22.973 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.233507+0000", 2024-09-10T18:59:22.973 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:22.973 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.973 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.973 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:59:22.973 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.233506+0000", 2024-09-10T18:59:22.973 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:59:22.973 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.973 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.973 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:59:22.974 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.233510+0000", 2024-09-10T18:59:22.974 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.301e-06 2024-09-10T18:59:22.974 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.974 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.974 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:59:22.974 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.233520+0000", 2024-09-10T18:59:22.974 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.7580000000000004e-06 2024-09-10T18:59:22.974 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.974 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.974 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:59:22.974 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.233527+0000", 2024-09-10T18:59:22.974 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.3520000000000001e-06 2024-09-10T18:59:22.974 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.975 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.975 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:22.975 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.233578+0000", 2024-09-10T18:59:22.975 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.0584999999999998e-05 2024-09-10T18:59:22.975 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.975 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.975 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T18:59:22.975 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.233587+0000", 2024-09-10T18:59:22.975 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.3640000000000005e-06 2024-09-10T18:59:22.975 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.975 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.975 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:22.975 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:21.163543+0000", 2024-09-10T18:59:22.976 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.929955333 2024-09-10T18:59:22.976 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.976 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.976 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T18:59:22.976 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:21.163550+0000", 2024-09-10T18:59:22.976 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.8199999999999997e-06 2024-09-10T18:59:22.976 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.976 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.976 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:22.976 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.223161+0000", 2024-09-10T18:59:22.976 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0596107109999999 2024-09-10T18:59:22.976 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.977 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.977 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:59:22.977 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.223277+0000", 2024-09-10T18:59:22.977 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000115556 2024-09-10T18:59:22.977 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.977 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.977 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,4,5", 2024-09-10T18:59:22.977 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.223405+0000", 2024-09-10T18:59:22.977 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00012836600000000001 2024-09-10T18:59:22.977 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.977 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.977 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:22.977 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.225529+0000", 2024-09-10T18:59:22.978 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0021234610000000001 2024-09-10T18:59:22.978 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.978 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.978 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:59:22.978 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.225547+0000", 2024-09-10T18:59:22.978 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.8011000000000001e-05 2024-09-10T18:59:22.978 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.978 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.978 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:22.978 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.228820+0000", 2024-09-10T18:59:22.978 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0032735849999999999 2024-09-10T18:59:22.978 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.978 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.979 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:22.979 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.232727+0000", 2024-09-10T18:59:22.979 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0039069229999999996 2024-09-10T18:59:22.979 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.979 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.979 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:59:22.979 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.232785+0000", 2024-09-10T18:59:22.979 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.7936999999999998e-05 2024-09-10T18:59:22.979 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.979 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.979 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:59:22.979 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.232912+0000", 2024-09-10T18:59:22.979 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000126637 2024-09-10T18:59:22.979 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:22.980 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:59:22.980 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:22.980 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.980 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.980 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.24947.0:147864 8.0 8:06a71dd4:::benchmark_data_smithi018_106531_object1234:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e193)", 2024-09-10T18:59:22.980 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:53:20.235925+0000", 2024-09-10T18:59:22.980 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 362.68635130899997, 2024-09-10T18:59:22.980 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.997209016, 2024-09-10T18:59:22.980 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:59:22.980 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:59:22.980 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:59:22.980 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.24947", 2024-09-10T18:59:22.980 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:59:22.980 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 147864 2024-09-10T18:59:22.981 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.981 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:59:22.981 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.981 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:59:22.981 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.235925+0000", 2024-09-10T18:59:22.981 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:22.981 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.981 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.981 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:59:22.981 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.235925+0000", 2024-09-10T18:59:22.981 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:22.981 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.981 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.981 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:59:22.982 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.235924+0000", 2024-09-10T18:59:22.982 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:59:22.982 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.982 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.982 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:59:22.982 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.235929+0000", 2024-09-10T18:59:22.982 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.9740000000000001e-06 2024-09-10T18:59:22.982 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.982 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.982 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:59:22.982 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.235936+0000", 2024-09-10T18:59:22.982 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.6310000000000003e-06 2024-09-10T18:59:22.982 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.983 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.983 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:59:22.983 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.235943+0000", 2024-09-10T18:59:22.983 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.7190000000000004e-06 2024-09-10T18:59:22.983 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.983 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.983 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:22.983 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.236002+0000", 2024-09-10T18:59:22.983 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.8823000000000002e-05 2024-09-10T18:59:22.983 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.983 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.983 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T18:59:22.983 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.236010+0000", 2024-09-10T18:59:22.983 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.6279999999999996e-06 2024-09-10T18:59:22.984 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.984 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.984 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:22.984 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:21.163572+0000", 2024-09-10T18:59:22.984 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.92756287699999995 2024-09-10T18:59:22.984 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.984 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.984 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T18:59:22.984 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:21.163579+0000", 2024-09-10T18:59:22.984 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.4400000000000002e-06 2024-09-10T18:59:22.984 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.984 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.984 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:22.984 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.224411+0000", 2024-09-10T18:59:22.985 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0608317759999999 2024-09-10T18:59:22.985 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.985 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.985 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:59:22.985 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.224502+0000", 2024-09-10T18:59:22.985 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.1003999999999998e-05 2024-09-10T18:59:22.985 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.985 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.985 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,4,5", 2024-09-10T18:59:22.985 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.224573+0000", 2024-09-10T18:59:22.985 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.1497999999999999e-05 2024-09-10T18:59:22.985 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.985 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.985 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:59:22.986 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.228084+0000", 2024-09-10T18:59:22.986 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0035104239999999998 2024-09-10T18:59:22.986 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.986 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.986 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:22.986 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.228127+0000", 2024-09-10T18:59:22.986 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.3328000000000003e-05 2024-09-10T18:59:22.986 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.986 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.986 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:22.986 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.228868+0000", 2024-09-10T18:59:22.986 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00074106300000000003 2024-09-10T18:59:22.986 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.986 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.986 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:22.987 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.232984+0000", 2024-09-10T18:59:22.987 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0041164330000000001 2024-09-10T18:59:22.987 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.987 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.987 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:59:22.987 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.233033+0000", 2024-09-10T18:59:22.987 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.8093000000000002e-05 2024-09-10T18:59:22.987 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.987 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.987 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:59:22.987 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.233134+0000", 2024-09-10T18:59:22.987 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00010108 2024-09-10T18:59:22.987 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:22.988 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:59:22.988 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:22.988 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.988 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.988 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.24947.0:147889 8.0 8:0f922230:::benchmark_data_smithi018_106531_object1259:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e193)", 2024-09-10T18:59:22.988 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:53:20.251963+0000", 2024-09-10T18:59:22.988 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 362.670313239, 2024-09-10T18:59:22.988 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.9814712810000001, 2024-09-10T18:59:22.988 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:59:22.988 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:59:22.988 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:59:22.988 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.24947", 2024-09-10T18:59:22.988 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:59:22.988 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 147889 2024-09-10T18:59:22.989 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.989 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:59:22.989 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.989 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:59:22.989 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.251963+0000", 2024-09-10T18:59:22.989 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:22.989 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.989 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.989 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:59:22.989 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.251963+0000", 2024-09-10T18:59:22.989 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:22.989 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.989 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.990 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:59:22.990 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.251961+0000", 2024-09-10T18:59:22.990 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T18:59:22.990 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.990 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.990 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:59:22.990 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.251967+0000", 2024-09-10T18:59:22.990 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.9669999999999998e-06 2024-09-10T18:59:22.990 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.990 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.990 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:59:22.990 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.251988+0000", 2024-09-10T18:59:22.990 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.0282999999999999e-05 2024-09-10T18:59:22.990 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.991 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.991 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:59:22.991 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.251994+0000", 2024-09-10T18:59:22.991 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.6220000000000003e-06 2024-09-10T18:59:22.991 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.991 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.991 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:22.991 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.252053+0000", 2024-09-10T18:59:22.991 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.9144999999999997e-05 2024-09-10T18:59:22.991 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.991 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.991 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T18:59:22.991 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.252063+0000", 2024-09-10T18:59:22.991 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.9059999999999997e-06 2024-09-10T18:59:22.992 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.992 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.992 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:22.992 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:21.163603+0000", 2024-09-10T18:59:22.992 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.91153938800000001 2024-09-10T18:59:22.992 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.992 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.992 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T18:59:22.992 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:21.163609+0000", 2024-09-10T18:59:22.992 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.3600000000000001e-06 2024-09-10T18:59:22.992 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.992 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.992 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:22.992 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.225592+0000", 2024-09-10T18:59:22.993 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0619829220000001 2024-09-10T18:59:22.993 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.993 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.993 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:59:22.993 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.225717+0000", 2024-09-10T18:59:22.993 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00012458900000000001 2024-09-10T18:59:22.993 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.993 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.993 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,4,5", 2024-09-10T18:59:22.993 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.225796+0000", 2024-09-10T18:59:22.993 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.9055999999999997e-05 2024-09-10T18:59:22.993 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.993 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.994 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:59:22.994 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.228188+0000", 2024-09-10T18:59:22.994 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0023922990000000001 2024-09-10T18:59:22.994 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.994 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.994 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:22.994 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.228460+0000", 2024-09-10T18:59:22.994 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000272303 2024-09-10T18:59:22.994 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.994 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.994 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:22.994 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.228918+0000", 2024-09-10T18:59:22.994 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00045776500000000001 2024-09-10T18:59:22.994 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.994 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.995 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:22.995 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.233230+0000", 2024-09-10T18:59:22.995 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0043122070000000002 2024-09-10T18:59:22.995 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.995 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.995 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:59:22.995 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.233301+0000", 2024-09-10T18:59:22.995 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.0673999999999995e-05 2024-09-10T18:59:22.995 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.995 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.995 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:59:22.995 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.233434+0000", 2024-09-10T18:59:22.995 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00013303300000000001 2024-09-10T18:59:22.995 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:22.996 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:59:22.996 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:22.996 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.996 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.996 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.24947.0:147891 8.0 8:015f3556:::benchmark_data_smithi018_106531_object1261:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e193)", 2024-09-10T18:59:22.996 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:53:20.254432+0000", 2024-09-10T18:59:22.996 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 362.66784402399998, 2024-09-10T18:59:22.996 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.9792919260000001, 2024-09-10T18:59:22.996 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:59:22.996 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:59:22.996 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:59:22.996 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.24947", 2024-09-10T18:59:22.996 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:59:22.997 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 147891 2024-09-10T18:59:22.997 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.997 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:59:22.997 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.997 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:59:22.997 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.254432+0000", 2024-09-10T18:59:22.997 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:22.997 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.997 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.997 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:59:22.997 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.254432+0000", 2024-09-10T18:59:22.997 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:22.997 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.998 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.998 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:59:22.998 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.254430+0000", 2024-09-10T18:59:22.998 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T18:59:22.998 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.998 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.998 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:59:22.998 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.254437+0000", 2024-09-10T18:59:22.998 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.126e-06 2024-09-10T18:59:22.998 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.998 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.998 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:59:22.998 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.254451+0000", 2024-09-10T18:59:22.999 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.3611999999999999e-05 2024-09-10T18:59:22.999 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.999 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.999 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:59:22.999 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.254458+0000", 2024-09-10T18:59:22.999 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.4409999999999998e-06 2024-09-10T18:59:22.999 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.999 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:22.999 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:22.999 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.254519+0000", 2024-09-10T18:59:22.999 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.1671e-05 2024-09-10T18:59:22.999 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:22.999 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.000 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T18:59:23.000 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.254528+0000", 2024-09-10T18:59:23.000 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.619e-06 2024-09-10T18:59:23.000 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.000 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.000 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:23.000 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:21.163653+0000", 2024-09-10T18:59:23.000 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.90912540500000005 2024-09-10T18:59:23.000 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.000 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.000 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T18:59:23.000 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:21.163660+0000", 2024-09-10T18:59:23.000 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.9450000000000004e-06 2024-09-10T18:59:23.000 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.001 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.001 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:23.001 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.226856+0000", 2024-09-10T18:59:23.001 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0631953970000001 2024-09-10T18:59:23.001 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.001 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.001 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:59:23.001 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.226966+0000", 2024-09-10T18:59:23.001 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000110639 2024-09-10T18:59:23.001 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.001 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.001 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,4,5", 2024-09-10T18:59:23.001 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.227069+0000", 2024-09-10T18:59:23.002 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000102926 2024-09-10T18:59:23.002 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.002 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.002 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:59:23.002 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.228192+0000", 2024-09-10T18:59:23.002 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0011233009999999999 2024-09-10T18:59:23.002 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.002 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.002 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:23.002 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.228762+0000", 2024-09-10T18:59:23.002 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00056986000000000005 2024-09-10T18:59:23.002 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.002 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.003 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:23.003 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.230567+0000", 2024-09-10T18:59:23.003 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0018048929999999999 2024-09-10T18:59:23.003 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.003 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.003 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:23.003 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.233572+0000", 2024-09-10T18:59:23.003 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0030052339999999999 2024-09-10T18:59:23.003 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.003 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.003 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:59:23.003 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.233625+0000", 2024-09-10T18:59:23.003 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.3071000000000001e-05 2024-09-10T18:59:23.003 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.003 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.004 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:59:23.004 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.233724+0000", 2024-09-10T18:59:23.004 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.8324000000000003e-05 2024-09-10T18:59:23.004 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:23.004 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:59:23.004 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:23.004 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.004 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.004 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.24947.0:147908 8.0 8:03905bbd:::benchmark_data_smithi018_106531_object1278:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e193)", 2024-09-10T18:59:23.004 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:53:20.267062+0000", 2024-09-10T18:59:23.004 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 362.65521386900002, 2024-09-10T18:59:23.004 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.9668400660000001, 2024-09-10T18:59:23.004 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:59:23.004 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:59:23.005 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:59:23.005 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.24947", 2024-09-10T18:59:23.005 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:59:23.005 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 147908 2024-09-10T18:59:23.005 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.005 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:59:23.005 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.005 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:59:23.005 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.267062+0000", 2024-09-10T18:59:23.005 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:23.005 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.005 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.005 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:59:23.005 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.267062+0000", 2024-09-10T18:59:23.006 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:23.006 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.006 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.006 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:59:23.006 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.267060+0000", 2024-09-10T18:59:23.006 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999981 2024-09-10T18:59:23.006 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.006 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.006 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:59:23.006 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.267068+0000", 2024-09-10T18:59:23.006 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.5510000000000002e-06 2024-09-10T18:59:23.006 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.006 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.006 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:59:23.007 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.267085+0000", 2024-09-10T18:59:23.007 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.6801000000000001e-05 2024-09-10T18:59:23.007 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.007 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.007 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:59:23.007 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.267091+0000", 2024-09-10T18:59:23.007 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.635e-06 2024-09-10T18:59:23.007 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.007 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.007 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:23.007 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.267151+0000", 2024-09-10T18:59:23.007 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.0112999999999999e-05 2024-09-10T18:59:23.007 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.007 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.008 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T18:59:23.008 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.267209+0000", 2024-09-10T18:59:23.008 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.7370999999999997e-05 2024-09-10T18:59:23.008 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.008 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.008 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:23.008 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:21.163681+0000", 2024-09-10T18:59:23.008 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.89647246800000002 2024-09-10T18:59:23.008 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.008 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.008 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T18:59:23.008 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:21.163687+0000", 2024-09-10T18:59:23.008 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.0190000000000003e-06 2024-09-10T18:59:23.008 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.009 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.009 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:23.009 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.228934+0000", 2024-09-10T18:59:23.009 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0652464070000001 2024-09-10T18:59:23.009 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.009 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.009 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:59:23.009 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.229055+0000", 2024-09-10T18:59:23.009 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00012128 2024-09-10T18:59:23.009 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.009 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.009 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,4,5", 2024-09-10T18:59:23.009 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.229192+0000", 2024-09-10T18:59:23.009 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000137661 2024-09-10T18:59:23.010 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.010 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.010 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:23.010 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.232699+0000", 2024-09-10T18:59:23.010 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0035064279999999998 2024-09-10T18:59:23.010 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.010 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.010 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:23.010 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.233191+0000", 2024-09-10T18:59:23.010 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00049255900000000005 2024-09-10T18:59:23.010 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.010 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.010 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:59:23.011 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.233467+0000", 2024-09-10T18:59:23.011 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00027529200000000001 2024-09-10T18:59:23.011 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.011 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.011 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:23.011 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.233765+0000", 2024-09-10T18:59:23.011 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00029826899999999999 2024-09-10T18:59:23.011 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.011 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.011 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:59:23.011 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.233809+0000", 2024-09-10T18:59:23.011 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.3869000000000003e-05 2024-09-10T18:59:23.011 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.012 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.012 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:59:23.012 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.233902+0000", 2024-09-10T18:59:23.012 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.3200000000000002e-05 2024-09-10T18:59:23.012 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:23.012 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:59:23.012 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:23.012 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.012 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.012 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.24947.0:147909 8.0 8:0f27c56a:::benchmark_data_smithi018_106531_object1279:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e193)", 2024-09-10T18:59:23.012 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:53:20.267209+0000", 2024-09-10T18:59:23.012 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 362.65506682400002, 2024-09-10T18:59:23.012 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.9668770850000001, 2024-09-10T18:59:23.013 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:59:23.013 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:59:23.013 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:59:23.013 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.24947", 2024-09-10T18:59:23.013 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:59:23.013 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 147909 2024-09-10T18:59:23.013 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.013 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:59:23.013 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.013 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:59:23.013 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.267209+0000", 2024-09-10T18:59:23.013 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:23.013 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.013 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.014 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:59:23.014 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.267209+0000", 2024-09-10T18:59:23.014 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:23.014 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.014 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.014 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:59:23.014 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.267208+0000", 2024-09-10T18:59:23.014 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:59:23.014 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.014 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.014 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:59:23.014 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.267214+0000", 2024-09-10T18:59:23.015 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.2310000000000004e-06 2024-09-10T18:59:23.015 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.015 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.015 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:59:23.015 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.267223+0000", 2024-09-10T18:59:23.015 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.0780000000000002e-06 2024-09-10T18:59:23.016 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.016 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.016 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:59:23.016 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.267228+0000", 2024-09-10T18:59:23.016 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.3499999999999996e-06 2024-09-10T18:59:23.016 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.016 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.016 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:23.016 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.267277+0000", 2024-09-10T18:59:23.016 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.9285e-05 2024-09-10T18:59:23.016 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.016 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.016 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T18:59:23.017 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.267282+0000", 2024-09-10T18:59:23.017 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.8339999999999998e-06 2024-09-10T18:59:23.017 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.017 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.017 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:23.017 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:21.163708+0000", 2024-09-10T18:59:23.017 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.89642588700000003 2024-09-10T18:59:23.017 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.017 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.017 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T18:59:23.017 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:21.163714+0000", 2024-09-10T18:59:23.017 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.7710000000000003e-06 2024-09-10T18:59:23.017 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.017 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.018 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:23.018 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.230593+0000", 2024-09-10T18:59:23.018 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.066878953 2024-09-10T18:59:23.018 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.018 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.018 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:59:23.018 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.230718+0000", 2024-09-10T18:59:23.018 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00012566500000000001 2024-09-10T18:59:23.018 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.018 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.018 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,4,5", 2024-09-10T18:59:23.018 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.230870+0000", 2024-09-10T18:59:23.018 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00015144299999999999 2024-09-10T18:59:23.019 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.019 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.019 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:23.019 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.232948+0000", 2024-09-10T18:59:23.019 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0020781409999999999 2024-09-10T18:59:23.019 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.019 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.019 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:59:23.019 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.233475+0000", 2024-09-10T18:59:23.019 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00052665600000000002 2024-09-10T18:59:23.019 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.019 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.019 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:23.020 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.233525+0000", 2024-09-10T18:59:23.020 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.0798999999999999e-05 2024-09-10T18:59:23.020 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.020 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.020 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:23.020 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.233938+0000", 2024-09-10T18:59:23.020 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000412311 2024-09-10T18:59:23.020 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.020 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.020 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:59:23.020 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.233994+0000", 2024-09-10T18:59:23.020 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.6038000000000001e-05 2024-09-10T18:59:23.020 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.020 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.021 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:59:23.021 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.234086+0000", 2024-09-10T18:59:23.021 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.2368999999999996e-05 2024-09-10T18:59:23.021 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:23.021 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:59:23.021 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:23.021 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.021 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.021 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.24947.0:147918 8.0 8:01f60d7a:::benchmark_data_smithi018_106531_object1288:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e193)", 2024-09-10T18:59:23.021 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:53:20.274157+0000", 2024-09-10T18:59:23.021 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 362.648118582, 2024-09-10T18:59:23.021 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.9672496530000001, 2024-09-10T18:59:23.021 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:59:23.021 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:59:23.022 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:59:23.022 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.24947", 2024-09-10T18:59:23.022 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:59:23.022 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 147918 2024-09-10T18:59:23.022 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.022 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:59:23.022 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.022 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:59:23.022 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.274157+0000", 2024-09-10T18:59:23.022 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:23.022 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.022 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.022 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:59:23.022 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.274157+0000", 2024-09-10T18:59:23.022 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:23.023 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.023 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.023 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:59:23.023 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.274157+0000", 2024-09-10T18:59:23.023 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T18:59:23.023 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.023 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.023 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:59:23.023 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.274162+0000", 2024-09-10T18:59:23.023 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.0300000000000001e-06 2024-09-10T18:59:23.023 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.023 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.023 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:59:23.023 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.274173+0000", 2024-09-10T18:59:23.024 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1236000000000001e-05 2024-09-10T18:59:23.024 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.024 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.024 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:59:23.024 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.274180+0000", 2024-09-10T18:59:23.024 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.126e-06 2024-09-10T18:59:23.024 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.024 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.024 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:23.024 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.274234+0000", 2024-09-10T18:59:23.024 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.4384e-05 2024-09-10T18:59:23.024 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.024 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.024 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T18:59:23.025 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.274245+0000", 2024-09-10T18:59:23.025 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0896e-05 2024-09-10T18:59:23.025 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.025 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.025 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:23.025 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:21.163746+0000", 2024-09-10T18:59:23.025 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.88950111600000004 2024-09-10T18:59:23.025 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.025 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.025 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T18:59:23.025 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:21.163753+0000", 2024-09-10T18:59:23.025 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.6120000000000004e-06 2024-09-10T18:59:23.025 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.025 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.025 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:23.026 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.234114+0000", 2024-09-10T18:59:23.026 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0703605350000001 2024-09-10T18:59:23.026 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.026 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.026 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:59:23.026 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.234218+0000", 2024-09-10T18:59:23.026 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000104885 2024-09-10T18:59:23.026 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.026 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.026 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,4,5", 2024-09-10T18:59:23.026 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.234333+0000", 2024-09-10T18:59:23.026 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000114949 2024-09-10T18:59:23.026 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.027 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.027 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:23.027 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.241160+0000", 2024-09-10T18:59:23.027 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0068267049999999998 2024-09-10T18:59:23.027 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.027 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.027 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:23.027 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.241195+0000", 2024-09-10T18:59:23.027 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.4953000000000002e-05 2024-09-10T18:59:23.027 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.027 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.027 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:23.027 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.241229+0000", 2024-09-10T18:59:23.027 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.3662999999999999e-05 2024-09-10T18:59:23.027 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.028 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.028 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:59:23.028 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.241307+0000", 2024-09-10T18:59:23.028 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.8022999999999995e-05 2024-09-10T18:59:23.028 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.028 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.028 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:59:23.028 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.241337+0000", 2024-09-10T18:59:23.028 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.9830999999999999e-05 2024-09-10T18:59:23.028 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.028 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.028 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:59:23.028 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.241407+0000", 2024-09-10T18:59:23.028 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.0401000000000006e-05 2024-09-10T18:59:23.029 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:23.029 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:59:23.029 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:23.029 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.029 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.029 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.24947.0:147926 8.0 8:0fd9af3c:::benchmark_data_smithi018_106531_object1296:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e193)", 2024-09-10T18:59:23.029 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:53:20.280867+0000", 2024-09-10T18:59:23.029 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 362.64140876300002, 2024-09-10T18:59:23.029 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.9607530280000001, 2024-09-10T18:59:23.029 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:59:23.029 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:59:23.029 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:59:23.029 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.24947", 2024-09-10T18:59:23.029 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:59:23.029 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 147926 2024-09-10T18:59:23.030 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.030 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:59:23.030 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.030 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:59:23.030 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.280867+0000", 2024-09-10T18:59:23.030 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:23.030 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.030 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.030 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:59:23.030 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.280867+0000", 2024-09-10T18:59:23.030 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:23.030 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.030 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.031 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:59:23.031 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.280866+0000", 2024-09-10T18:59:23.031 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T18:59:23.031 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.031 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.031 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:59:23.031 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.280873+0000", 2024-09-10T18:59:23.031 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.9229999999999996e-06 2024-09-10T18:59:23.031 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.031 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.031 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:59:23.031 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.280890+0000", 2024-09-10T18:59:23.031 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.768e-05 2024-09-10T18:59:23.031 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.031 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.031 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:59:23.032 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.280897+0000", 2024-09-10T18:59:23.032 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.8759999999999999e-06 2024-09-10T18:59:23.032 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.032 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.032 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:23.032 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.280948+0000", 2024-09-10T18:59:23.032 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.1023999999999998e-05 2024-09-10T18:59:23.032 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.032 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.032 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T18:59:23.032 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.280958+0000", 2024-09-10T18:59:23.032 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0222e-05 2024-09-10T18:59:23.032 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.032 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.033 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:23.033 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:21.163775+0000", 2024-09-10T18:59:23.033 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.88281693100000003 2024-09-10T18:59:23.033 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.033 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.033 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T18:59:23.033 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:21.163782+0000", 2024-09-10T18:59:23.033 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.1279999999999996e-06 2024-09-10T18:59:23.033 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.033 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.033 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:23.033 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.236380+0000", 2024-09-10T18:59:23.033 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0725980100000001 2024-09-10T18:59:23.033 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.034 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.034 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:59:23.034 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.236495+0000", 2024-09-10T18:59:23.034 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000115012 2024-09-10T18:59:23.034 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.034 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.034 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,4,5", 2024-09-10T18:59:23.034 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.236619+0000", 2024-09-10T18:59:23.034 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000124523 2024-09-10T18:59:23.034 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.034 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.034 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:23.034 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.241263+0000", 2024-09-10T18:59:23.034 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0046439719999999997 2024-09-10T18:59:23.034 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.034 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.035 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:23.035 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.241298+0000", 2024-09-10T18:59:23.035 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.5145999999999999e-05 2024-09-10T18:59:23.035 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.035 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.035 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:59:23.035 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.241420+0000", 2024-09-10T18:59:23.035 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000122247 2024-09-10T18:59:23.035 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.035 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.035 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:23.035 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.241463+0000", 2024-09-10T18:59:23.035 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.2336999999999998e-05 2024-09-10T18:59:23.035 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.035 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.036 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:59:23.036 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.241512+0000", 2024-09-10T18:59:23.036 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.9546999999999998e-05 2024-09-10T18:59:23.036 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.036 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.036 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:59:23.036 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.241620+0000", 2024-09-10T18:59:23.036 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00010786200000000001 2024-09-10T18:59:23.036 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:23.036 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:59:23.036 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:23.036 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.036 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.037 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.24947.0:147973 8.0 8:0a309262:::benchmark_data_smithi018_106531_object1343:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e193)", 2024-09-10T18:59:23.037 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:53:20.328297+0000", 2024-09-10T18:59:23.037 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 362.59397890700001, 2024-09-10T18:59:23.037 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.9246313719999999, 2024-09-10T18:59:23.037 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:59:23.037 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:59:23.037 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:59:23.037 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.24947", 2024-09-10T18:59:23.037 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:59:23.037 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 147973 2024-09-10T18:59:23.037 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.037 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:59:23.037 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.037 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:59:23.038 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.328297+0000", 2024-09-10T18:59:23.038 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:23.038 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.038 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.038 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:59:23.038 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.328297+0000", 2024-09-10T18:59:23.038 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:23.038 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.038 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.038 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:59:23.038 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.328296+0000", 2024-09-10T18:59:23.038 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T18:59:23.038 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.038 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.038 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:59:23.039 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.328302+0000", 2024-09-10T18:59:23.039 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.1330000000000004e-06 2024-09-10T18:59:23.039 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.039 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.039 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:59:23.039 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.328314+0000", 2024-09-10T18:59:23.039 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.2591e-05 2024-09-10T18:59:23.039 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.039 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.039 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:59:23.039 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.328320+0000", 2024-09-10T18:59:23.039 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.0009999999999998e-06 2024-09-10T18:59:23.039 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.039 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.040 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:23.040 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.328372+0000", 2024-09-10T18:59:23.040 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.2015999999999998e-05 2024-09-10T18:59:23.040 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.040 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.040 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T18:59:23.040 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.328380+0000", 2024-09-10T18:59:23.040 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.7870000000000002e-06 2024-09-10T18:59:23.040 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.040 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.040 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:23.040 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:21.163804+0000", 2024-09-10T18:59:23.040 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.835423319 2024-09-10T18:59:23.041 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.041 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.041 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T18:59:23.041 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:21.163810+0000", 2024-09-10T18:59:23.041 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.7469999999999999e-06 2024-09-10T18:59:23.041 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.041 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.041 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:23.041 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.238780+0000", 2024-09-10T18:59:23.041 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.074969286 2024-09-10T18:59:23.041 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.041 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.041 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:59:23.041 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.238896+0000", 2024-09-10T18:59:23.042 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00011636700000000001 2024-09-10T18:59:23.042 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.042 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.042 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,4,5", 2024-09-10T18:59:23.042 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.239041+0000", 2024-09-10T18:59:23.042 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00014488200000000001 2024-09-10T18:59:23.042 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.042 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.042 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:59:23.042 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.243877+0000", 2024-09-10T18:59:23.042 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0048363879999999996 2024-09-10T18:59:23.042 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.042 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.043 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:23.043 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.248499+0000", 2024-09-10T18:59:23.043 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0046214730000000001 2024-09-10T18:59:23.043 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.043 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.043 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:23.043 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.248563+0000", 2024-09-10T18:59:23.043 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.3993999999999998e-05 2024-09-10T18:59:23.043 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.043 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.043 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:23.043 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.252759+0000", 2024-09-10T18:59:23.043 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.004195898 2024-09-10T18:59:23.043 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.043 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.044 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:59:23.044 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.252812+0000", 2024-09-10T18:59:23.044 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.3164000000000001e-05 2024-09-10T18:59:23.044 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.044 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.044 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:59:23.044 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.252928+0000", 2024-09-10T18:59:23.044 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000116655 2024-09-10T18:59:23.044 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:23.044 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:59:23.044 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:23.044 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.044 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.044 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.24947.0:148000 8.0 8:04489dcc:::benchmark_data_smithi018_106531_object1370:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e193)", 2024-09-10T18:59:23.045 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:53:20.365422+0000", 2024-09-10T18:59:23.045 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 362.556853929, 2024-09-10T18:59:23.045 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.8948281439999999, 2024-09-10T18:59:23.045 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:59:23.045 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:59:23.045 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:59:23.045 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.24947", 2024-09-10T18:59:23.045 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:59:23.045 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 148000 2024-09-10T18:59:23.045 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.045 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:59:23.045 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.045 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:59:23.045 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.365422+0000", 2024-09-10T18:59:23.045 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:23.046 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.046 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.046 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:59:23.046 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.365422+0000", 2024-09-10T18:59:23.046 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:23.046 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.046 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.046 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:59:23.046 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.365421+0000", 2024-09-10T18:59:23.046 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T18:59:23.046 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.046 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.046 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:59:23.046 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.365427+0000", 2024-09-10T18:59:23.047 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.7939999999999999e-06 2024-09-10T18:59:23.047 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.047 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.047 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:59:23.047 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.365437+0000", 2024-09-10T18:59:23.047 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.011e-05 2024-09-10T18:59:23.047 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.047 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.047 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:59:23.047 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.365441+0000", 2024-09-10T18:59:23.047 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.4070000000000003e-06 2024-09-10T18:59:23.047 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.047 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.047 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:23.047 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.365492+0000", 2024-09-10T18:59:23.048 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.0603999999999999e-05 2024-09-10T18:59:23.048 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.048 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.048 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T18:59:23.048 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.365499+0000", 2024-09-10T18:59:23.048 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.3710000000000002e-06 2024-09-10T18:59:23.048 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.048 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.048 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:23.048 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:21.163834+0000", 2024-09-10T18:59:23.048 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.79833463800000004 2024-09-10T18:59:23.048 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.048 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.048 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T18:59:23.048 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:21.163840+0000", 2024-09-10T18:59:23.049 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.37e-06 2024-09-10T18:59:23.049 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.049 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.049 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:23.049 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.241663+0000", 2024-09-10T18:59:23.049 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.077823253 2024-09-10T18:59:23.049 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.049 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.049 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:59:23.049 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.241787+0000", 2024-09-10T18:59:23.049 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00012356699999999999 2024-09-10T18:59:23.049 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.049 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.049 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,4,5", 2024-09-10T18:59:23.050 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.241920+0000", 2024-09-10T18:59:23.050 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00013296 2024-09-10T18:59:23.050 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.050 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.050 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:23.050 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.248529+0000", 2024-09-10T18:59:23.050 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0066094470000000001 2024-09-10T18:59:23.050 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.050 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.050 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:59:23.050 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.252706+0000", 2024-09-10T18:59:23.050 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0041763369999999996 2024-09-10T18:59:23.050 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.050 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.050 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:23.051 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.253060+0000", 2024-09-10T18:59:23.051 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00035445799999999999 2024-09-10T18:59:23.051 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.051 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.051 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:23.051 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.260070+0000", 2024-09-10T18:59:23.051 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0070096840000000004 2024-09-10T18:59:23.051 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.051 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.051 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:59:23.051 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.260122+0000", 2024-09-10T18:59:23.051 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.2726999999999998e-05 2024-09-10T18:59:23.051 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.051 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.052 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:59:23.052 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.260250+0000", 2024-09-10T18:59:23.052 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00012763799999999999 2024-09-10T18:59:23.052 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:23.052 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:59:23.052 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:23.052 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.052 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.052 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.24947.0:148003 8.0 8:0cc5e256:::benchmark_data_smithi018_106531_object1373:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e193)", 2024-09-10T18:59:23.052 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:53:20.370303+0000", 2024-09-10T18:59:23.052 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 362.55197267400001, 2024-09-10T18:59:23.052 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.895113284, 2024-09-10T18:59:23.052 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:59:23.052 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:59:23.053 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:59:23.053 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.24947", 2024-09-10T18:59:23.053 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:59:23.053 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 148003 2024-09-10T18:59:23.053 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.053 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:59:23.053 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.053 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:59:23.053 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.370303+0000", 2024-09-10T18:59:23.053 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:23.053 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.053 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.053 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:59:23.053 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.370303+0000", 2024-09-10T18:59:23.053 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:23.054 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.054 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.054 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:59:23.054 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.370302+0000", 2024-09-10T18:59:23.054 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:59:23.054 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.054 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.054 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:59:23.054 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.370307+0000", 2024-09-10T18:59:23.054 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.8770000000000001e-06 2024-09-10T18:59:23.054 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.054 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.054 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:59:23.054 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.370317+0000", 2024-09-10T18:59:23.054 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0081999999999999e-05 2024-09-10T18:59:23.055 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.055 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.055 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:59:23.055 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.370322+0000", 2024-09-10T18:59:23.055 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.3200000000000001e-06 2024-09-10T18:59:23.055 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.055 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.055 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:23.056 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.370370+0000", 2024-09-10T18:59:23.056 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.8208000000000003e-05 2024-09-10T18:59:23.056 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.056 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.056 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T18:59:23.056 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:20.370377+0000", 2024-09-10T18:59:23.056 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.1679999999999998e-06 2024-09-10T18:59:23.056 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.056 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.056 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:23.056 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:21.164669+0000", 2024-09-10T18:59:23.057 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.79429215099999995 2024-09-10T18:59:23.057 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.057 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.057 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T18:59:23.057 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:21.164679+0000", 2024-09-10T18:59:23.057 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.5289999999999997e-06 2024-09-10T18:59:23.057 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.057 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.057 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:23.057 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.243906+0000", 2024-09-10T18:59:23.057 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0792277429999999 2024-09-10T18:59:23.057 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.058 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.058 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:59:23.058 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.244036+0000", 2024-09-10T18:59:23.058 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000129889 2024-09-10T18:59:23.058 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.058 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.058 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,4,5", 2024-09-10T18:59:23.058 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.244155+0000", 2024-09-10T18:59:23.058 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000118634 2024-09-10T18:59:23.058 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.058 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.058 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:23.058 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.252694+0000", 2024-09-10T18:59:23.058 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0085393970000000007 2024-09-10T18:59:23.059 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.059 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.059 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:59:23.059 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.253002+0000", 2024-09-10T18:59:23.059 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00030721900000000001 2024-09-10T18:59:23.059 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.059 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.059 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:23.059 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.260291+0000", 2024-09-10T18:59:23.059 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.007289167 2024-09-10T18:59:23.059 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.059 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.059 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:23.059 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.265241+0000", 2024-09-10T18:59:23.059 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0049506869999999996 2024-09-10T18:59:23.060 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.060 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.060 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:59:23.060 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.265296+0000", 2024-09-10T18:59:23.060 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.5084000000000003e-05 2024-09-10T18:59:23.060 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.060 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.060 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:59:23.060 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:53:22.265417+0000", 2024-09-10T18:59:23.060 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00012006900000000001 2024-09-10T18:59:23.060 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:23.060 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:59:23.060 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:23.061 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.061 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.061 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15354.0:174900 9.2 9:468890ee:::benchmark_data_smithi018_133594_object38148:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e283)", 2024-09-10T18:59:23.061 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:58:01.119449+0000", 2024-09-10T18:59:23.061 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 81.802826745000004, 2024-09-10T18:59:23.061 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.042442463, 2024-09-10T18:59:23.061 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:59:23.061 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "reached pg", 2024-09-10T18:59:23.061 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:59:23.061 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15354", 2024-09-10T18:59:23.061 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T18:59:23.061 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 174900 2024-09-10T18:59:23.061 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.061 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:59:23.062 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.062 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:59:23.062 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:01.119449+0000", 2024-09-10T18:59:23.062 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:23.062 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.062 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.062 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:59:23.062 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:01.119449+0000", 2024-09-10T18:59:23.062 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:23.062 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.062 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.062 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:59:23.062 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:01.119448+0000", 2024-09-10T18:59:23.063 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T18:59:23.063 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.063 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.063 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:59:23.063 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:01.119454+0000", 2024-09-10T18:59:23.063 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.1970000000000001e-06 2024-09-10T18:59:23.063 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.063 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.063 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:59:23.063 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:01.119465+0000", 2024-09-10T18:59:23.063 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0530000000000001e-05 2024-09-10T18:59:23.063 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.063 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.064 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:59:23.064 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:01.119470+0000", 2024-09-10T18:59:23.064 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.7459999999999997e-06 2024-09-10T18:59:23.064 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.064 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.064 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:23.064 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:01.119513+0000", 2024-09-10T18:59:23.064 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.3096000000000002e-05 2024-09-10T18:59:23.064 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.064 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.064 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:59:23.064 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:01.119630+0000", 2024-09-10T18:59:23.064 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00011681200000000001 2024-09-10T18:59:23.064 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.065 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.065 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,6,7", 2024-09-10T18:59:23.065 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:01.119748+0000", 2024-09-10T18:59:23.065 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000117439 2024-09-10T18:59:23.065 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.065 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.065 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:59:23.065 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:01.123411+0000", 2024-09-10T18:59:23.065 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0036635470000000001 2024-09-10T18:59:23.065 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.065 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.065 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:23.065 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:01.127947+0000", 2024-09-10T18:59:23.065 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0045355370000000001 2024-09-10T18:59:23.066 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.066 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.066 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:23.066 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:01.127985+0000", 2024-09-10T18:59:23.066 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.8559999999999997e-05 2024-09-10T18:59:23.066 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.066 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.066 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:23.066 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:01.136874+0000", 2024-09-10T18:59:23.066 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0088886529999999998 2024-09-10T18:59:23.066 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.066 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.066 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T18:59:23.066 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:01.136882+0000", 2024-09-10T18:59:23.067 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.2479999999999996e-06 2024-09-10T18:59:23.067 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.067 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.067 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:23.067 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:02.161830+0000", 2024-09-10T18:59:23.067 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0249477549999999 2024-09-10T18:59:23.067 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.067 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.067 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:59:23.067 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:02.161892+0000", 2024-09-10T18:59:23.067 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.1606999999999995e-05 2024-09-10T18:59:23.067 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:23.067 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:59:23.067 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:23.067 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.067 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.068 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15354.0:181814 9.c 9:38be6ce0:::benchmark_data_smithi018_133594_object45062:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e288)", 2024-09-10T18:59:23.068 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:58:10.683245+0000", 2024-09-10T18:59:23.068 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 72.239031334000003, 2024-09-10T18:59:23.068 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1025302299999999, 2024-09-10T18:59:23.068 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:59:23.068 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "reached pg", 2024-09-10T18:59:23.068 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:59:23.068 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15354", 2024-09-10T18:59:23.068 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T18:59:23.068 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 181814 2024-09-10T18:59:23.068 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.068 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:59:23.068 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.068 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:59:23.068 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:10.683245+0000", 2024-09-10T18:59:23.069 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:23.069 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.069 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.069 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:59:23.069 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:10.683245+0000", 2024-09-10T18:59:23.069 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:23.069 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.069 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.069 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:59:23.069 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:10.683243+0000", 2024-09-10T18:59:23.069 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:59:23.069 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.069 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.070 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:59:23.070 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:10.683250+0000", 2024-09-10T18:59:23.070 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.122e-06 2024-09-10T18:59:23.070 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.070 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.070 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:59:23.070 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:10.683260+0000", 2024-09-10T18:59:23.070 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0355e-05 2024-09-10T18:59:23.070 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.070 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.070 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:59:23.070 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:10.683265+0000", 2024-09-10T18:59:23.070 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.5609999999999998e-06 2024-09-10T18:59:23.070 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.070 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.071 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:23.071 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:10.683306+0000", 2024-09-10T18:59:23.071 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.0351e-05 2024-09-10T18:59:23.071 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.071 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.071 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:59:23.071 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:10.683421+0000", 2024-09-10T18:59:23.071 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000114728 2024-09-10T18:59:23.071 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.071 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.071 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 0,4,7", 2024-09-10T18:59:23.071 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:10.683558+0000", 2024-09-10T18:59:23.071 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000137596 2024-09-10T18:59:23.071 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.072 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.072 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:23.072 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:10.690210+0000", 2024-09-10T18:59:23.072 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0066515309999999996 2024-09-10T18:59:23.072 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.072 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.072 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:59:23.072 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:10.690404+0000", 2024-09-10T18:59:23.072 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00019470199999999999 2024-09-10T18:59:23.072 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.072 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.072 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:23.072 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:10.702956+0000", 2024-09-10T18:59:23.073 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.012551860999999999 2024-09-10T18:59:23.073 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.073 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.073 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T18:59:23.073 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:10.702963+0000", 2024-09-10T18:59:23.073 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.2990000000000002e-06 2024-09-10T18:59:23.073 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.073 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.073 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:23.073 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:11.775086+0000", 2024-09-10T18:59:23.073 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0721239229999999 2024-09-10T18:59:23.073 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.073 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.074 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for missing object", 2024-09-10T18:59:23.074 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:11.775185+0000", 2024-09-10T18:59:23.074 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.8564e-05 2024-09-10T18:59:23.074 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.074 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.074 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:23.074 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:11.785707+0000", 2024-09-10T18:59:23.074 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.010521849999999999 2024-09-10T18:59:23.074 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.074 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.074 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:59:23.074 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:11.785775+0000", 2024-09-10T18:59:23.074 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.7937999999999999e-05 2024-09-10T18:59:23.074 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:23.075 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:59:23.075 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:23.075 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.075 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.075 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15354.0:181829 9.c 9:38557f3d:::benchmark_data_smithi018_133594_object45077:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-10T18:59:23.075 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:58:10.700071+0000", 2024-09-10T18:59:23.075 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 72.222204902000001, 2024-09-10T18:59:23.075 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.105001795, 2024-09-10T18:59:23.075 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:59:23.075 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:59:23.075 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:59:23.075 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15354", 2024-09-10T18:59:23.075 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T18:59:23.076 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 181829 2024-09-10T18:59:23.076 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.076 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:59:23.076 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.076 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:59:23.076 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:10.700071+0000", 2024-09-10T18:59:23.076 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:23.076 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.076 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.076 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:59:23.076 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:10.700071+0000", 2024-09-10T18:59:23.076 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:23.076 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.077 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.077 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:59:23.077 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:10.700042+0000", 2024-09-10T18:59:23.077 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999714 2024-09-10T18:59:23.077 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.077 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.077 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:59:23.077 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:10.700100+0000", 2024-09-10T18:59:23.077 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.7873e-05 2024-09-10T18:59:23.077 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.077 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.077 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:59:23.077 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:10.700143+0000", 2024-09-10T18:59:23.077 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.3228e-05 2024-09-10T18:59:23.078 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.078 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.078 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:59:23.078 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:10.700167+0000", 2024-09-10T18:59:23.078 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.3958e-05 2024-09-10T18:59:23.078 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.078 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.078 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:23.078 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:10.703022+0000", 2024-09-10T18:59:23.078 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0028549169999999998 2024-09-10T18:59:23.078 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.078 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.078 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T18:59:23.078 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:10.703029+0000", 2024-09-10T18:59:23.079 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.2400000000000004e-06 2024-09-10T18:59:23.079 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.079 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.079 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:23.079 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:11.775753+0000", 2024-09-10T18:59:23.079 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.07272485 2024-09-10T18:59:23.079 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.079 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.079 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:59:23.079 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:11.775887+0000", 2024-09-10T18:59:23.079 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000133373 2024-09-10T18:59:23.079 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.079 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.079 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 0,4", 2024-09-10T18:59:23.080 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:11.776017+0000", 2024-09-10T18:59:23.080 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00013060799999999999 2024-09-10T18:59:23.080 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.080 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.080 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:23.080 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:11.785315+0000", 2024-09-10T18:59:23.080 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0092974279999999999 2024-09-10T18:59:23.080 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.080 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.080 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:23.080 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:11.785368+0000", 2024-09-10T18:59:23.080 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.3616000000000003e-05 2024-09-10T18:59:23.080 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.080 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.081 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:59:23.081 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:11.804955+0000", 2024-09-10T18:59:23.081 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.019586292000000002 2024-09-10T18:59:23.081 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.081 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.081 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:59:23.081 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:11.804998+0000", 2024-09-10T18:59:23.081 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.3191e-05 2024-09-10T18:59:23.081 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.081 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.081 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:59:23.081 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:11.805073+0000", 2024-09-10T18:59:23.081 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.4892000000000001e-05 2024-09-10T18:59:23.081 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:23.081 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:59:23.081 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:23.082 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.082 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.082 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15354.0:181834 9.c 9:3f1a29da:::benchmark_data_smithi018_133594_object45082:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e289)", 2024-09-10T18:59:23.082 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:58:10.703553+0000", 2024-09-10T18:59:23.082 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 72.218722651999997, 2024-09-10T18:59:23.082 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.101840352, 2024-09-10T18:59:23.082 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T18:59:23.082 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:59:23.082 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T18:59:23.082 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15354", 2024-09-10T18:59:23.082 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T18:59:23.082 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 181834 2024-09-10T18:59:23.082 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.083 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T18:59:23.083 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.083 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T18:59:23.083 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:10.703553+0000", 2024-09-10T18:59:23.083 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:23.083 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.083 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.083 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T18:59:23.083 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:10.703553+0000", 2024-09-10T18:59:23.083 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T18:59:23.083 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.083 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.084 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T18:59:23.084 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:10.703552+0000", 2024-09-10T18:59:23.084 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T18:59:23.084 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.084 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.084 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T18:59:23.084 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:10.703558+0000", 2024-09-10T18:59:23.084 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.0109999999999997e-06 2024-09-10T18:59:23.084 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.084 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.084 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T18:59:23.084 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:10.703568+0000", 2024-09-10T18:59:23.084 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0084999999999999e-05 2024-09-10T18:59:23.084 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.084 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.085 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T18:59:23.085 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:10.703574+0000", 2024-09-10T18:59:23.085 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.6019999999999999e-06 2024-09-10T18:59:23.085 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.085 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.085 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:23.085 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:10.704266+0000", 2024-09-10T18:59:23.085 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000692247 2024-09-10T18:59:23.085 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.085 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.085 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T18:59:23.085 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:10.704272+0000", 2024-09-10T18:59:23.085 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.0850000000000002e-06 2024-09-10T18:59:23.085 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.085 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.086 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T18:59:23.086 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:11.779894+0000", 2024-09-10T18:59:23.086 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0756217160000001 2024-09-10T18:59:23.086 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.086 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.086 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T18:59:23.086 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:11.780086+0000", 2024-09-10T18:59:23.086 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000192301 2024-09-10T18:59:23.086 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.086 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.086 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 0,4", 2024-09-10T18:59:23.086 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:11.780207+0000", 2024-09-10T18:59:23.086 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00012076800000000001 2024-09-10T18:59:23.086 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.086 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.087 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:23.087 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:11.785399+0000", 2024-09-10T18:59:23.087 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0051918590000000001 2024-09-10T18:59:23.087 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.087 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.087 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:23.087 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:11.785812+0000", 2024-09-10T18:59:23.087 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00041343899999999998 2024-09-10T18:59:23.087 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.087 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.087 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T18:59:23.087 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:11.805085+0000", 2024-09-10T18:59:23.087 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.019272437999999999 2024-09-10T18:59:23.087 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.087 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.088 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T18:59:23.088 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:11.805141+0000", 2024-09-10T18:59:23.088 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.6101999999999999e-05 2024-09-10T18:59:23.088 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T18:59:23.088 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T18:59:23.088 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T18:59:23.088 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:11.805394+0000", 2024-09-10T18:59:23.088 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000252871 2024-09-10T18:59:23.088 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:23.088 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:59:23.088 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:23.088 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T18:59:23.088 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T18:59:23.088 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T18:59:23.089 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:23.131 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:23.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:23.133 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:23.233 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:23.257 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:23.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:23.258 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:23.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:22 smithi103 ceph-mon[33163]: 2.1d deep-scrub starts 2024-09-10T18:59:23.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:22 smithi103 ceph-mon[33163]: 2.1d deep-scrub ok 2024-09-10T18:59:23.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:22 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:59:23.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:22 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:59:23.359 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:23.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:23.397 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:23.397 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:23.498 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:23.523 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:23.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:23.524 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:23.625 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:23.647 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:23.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:23.649 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:23.749 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:23.778 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:23.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:23.778 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:23.795 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:23 smithi018 ceph-mon[40196]: 5.1f scrub starts 2024-09-10T18:59:23.795 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:23 smithi018 ceph-mon[40196]: 5.1f scrub ok 2024-09-10T18:59:23.795 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:23 smithi018 ceph-mon[40196]: 5.1b scrub starts 2024-09-10T18:59:23.795 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:23 smithi018 ceph-mon[40196]: 5.1b scrub ok 2024-09-10T18:59:23.795 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:23 smithi018 ceph-mon[40196]: osdmap e307: 8 total, 8 up, 6 in 2024-09-10T18:59:23.795 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:23 smithi018 ceph-mon[40196]: 9.c scrub starts 2024-09-10T18:59:23.796 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:23 smithi018 ceph-mon[40196]: 9.c scrub ok 2024-09-10T18:59:23.796 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:23 smithi018 ceph-mon[40196]: 9.a scrub starts 2024-09-10T18:59:23.796 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:23 smithi018 ceph-mon[40196]: pgmap v644: 168 pgs: 168 active+clean; 337 MiB data, 4.9 GiB used, 532 GiB / 536 GiB avail; 127 B/s rd, 0 B/s wr, 1.47k op/s 2024-09-10T18:59:23.796 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:23 smithi018 ceph-mon[40196]: 9.a scrub ok 2024-09-10T18:59:23.796 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:23 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/367756598' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:59:23.796 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:23 smithi018 ceph-mon[29336]: 5.1f scrub starts 2024-09-10T18:59:23.796 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:23 smithi018 ceph-mon[29336]: 5.1f scrub ok 2024-09-10T18:59:23.796 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:23 smithi018 ceph-mon[29336]: 5.1b scrub starts 2024-09-10T18:59:23.796 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:23 smithi018 ceph-mon[29336]: 5.1b scrub ok 2024-09-10T18:59:23.809 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is set 2024-09-10T18:59:23.879 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:23.904 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:23.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:23.906 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:24.006 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:24.033 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:24.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:24.034 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:24.134 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:24.175 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:24.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:24.176 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:24.193 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:23 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:59:24.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:23 smithi018 ceph-mon[29336]: osdmap e307: 8 total, 8 up, 6 in 2024-09-10T18:59:24.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:23 smithi018 ceph-mon[29336]: 9.c scrub starts 2024-09-10T18:59:24.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:23 smithi018 ceph-mon[29336]: 9.c scrub ok 2024-09-10T18:59:24.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:23 smithi018 ceph-mon[29336]: 9.a scrub starts 2024-09-10T18:59:24.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:23 smithi018 ceph-mon[29336]: pgmap v644: 168 pgs: 168 active+clean; 337 MiB data, 4.9 GiB used, 532 GiB / 536 GiB avail; 127 B/s rd, 0 B/s wr, 1.47k op/s 2024-09-10T18:59:24.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:23 smithi018 ceph-mon[29336]: 9.a scrub ok 2024-09-10T18:59:24.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:23 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/367756598' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:59:24.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:23 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:59:24.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:23 smithi103 ceph-mon[33163]: 5.1f scrub starts 2024-09-10T18:59:24.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:23 smithi103 ceph-mon[33163]: 5.1f scrub ok 2024-09-10T18:59:24.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:23 smithi103 ceph-mon[33163]: 5.1b scrub starts 2024-09-10T18:59:24.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:23 smithi103 ceph-mon[33163]: 5.1b scrub ok 2024-09-10T18:59:24.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:23 smithi103 ceph-mon[33163]: osdmap e307: 8 total, 8 up, 6 in 2024-09-10T18:59:24.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:23 smithi103 ceph-mon[33163]: 9.c scrub starts 2024-09-10T18:59:24.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:23 smithi103 ceph-mon[33163]: 9.c scrub ok 2024-09-10T18:59:24.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:23 smithi103 ceph-mon[33163]: 9.a scrub starts 2024-09-10T18:59:24.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:23 smithi103 ceph-mon[33163]: pgmap v644: 168 pgs: 168 active+clean; 337 MiB data, 4.9 GiB used, 532 GiB / 536 GiB avail; 127 B/s rd, 0 B/s wr, 1.47k op/s 2024-09-10T18:59:24.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:23 smithi103 ceph-mon[33163]: 9.a scrub ok 2024-09-10T18:59:24.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:23 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/367756598' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:59:24.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:23 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T18:59:24.276 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:24.317 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:24.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:24.318 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:24.419 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:24.462 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:24.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:24.463 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:24.552 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-10T18:59:24.564 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:24.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:24.618 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:24.618 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:24.719 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:24.768 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:24.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:24.770 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:24.868 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:59:24.870 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:24.902 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:24.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:24.903 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:24.996 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:24 smithi103 ceph-mon[33163]: 1.6 scrub starts 2024-09-10T18:59:24.996 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:24 smithi103 ceph-mon[33163]: 1.6 scrub ok 2024-09-10T18:59:24.996 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:24 smithi103 ceph-mon[33163]: 1.e scrub starts 2024-09-10T18:59:24.996 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:24 smithi103 ceph-mon[33163]: 1.e scrub ok 2024-09-10T18:59:24.996 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:24 smithi103 ceph-mon[33163]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:59:24.996 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:24 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:59:24.996 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:24 smithi103 ceph-mon[33163]: osdmap e308: 8 total, 8 up, 6 in 2024-09-10T18:59:25.005 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:25.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:25.044 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:25.045 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:25.145 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:25.161 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:24 smithi018 ceph-mon[29336]: 1.6 scrub starts 2024-09-10T18:59:25.161 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:24 smithi018 ceph-mon[29336]: 1.6 scrub ok 2024-09-10T18:59:25.161 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:24 smithi018 ceph-mon[29336]: 1.e scrub starts 2024-09-10T18:59:25.161 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:24 smithi018 ceph-mon[29336]: 1.e scrub ok 2024-09-10T18:59:25.161 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:24 smithi018 ceph-mon[29336]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:59:25.161 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:24 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:59:25.161 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:24 smithi018 ceph-mon[29336]: osdmap e308: 8 total, 8 up, 6 in 2024-09-10T18:59:25.161 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:24 smithi018 ceph-mon[40196]: 1.6 scrub starts 2024-09-10T18:59:25.161 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:24 smithi018 ceph-mon[40196]: 1.6 scrub ok 2024-09-10T18:59:25.161 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:24 smithi018 ceph-mon[40196]: 1.e scrub starts 2024-09-10T18:59:25.162 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:24 smithi018 ceph-mon[40196]: 1.e scrub ok 2024-09-10T18:59:25.162 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:24 smithi018 ceph-mon[40196]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:59:25.162 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:24 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T18:59:25.162 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:24 smithi018 ceph-mon[40196]: osdmap e308: 8 total, 8 up, 6 in 2024-09-10T18:59:25.190 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:25.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:25.191 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:25.291 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:25.334 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:25.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:25.335 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:25.392 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Removed 136749 objects 2024-09-10T18:59:25.392 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Clean up completed and total clean up time :126.985 2024-09-10T18:59:25.437 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:25.459 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:25.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:25.460 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:25.561 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:25.582 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:25.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:25.583 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:25.684 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:25.705 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:25.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:25.706 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:25.807 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:25.828 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:25.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:25.829 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:25.930 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:25.954 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:25.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:25.955 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:26.055 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:26.078 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:26.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:26.079 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:26.180 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:26.202 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:26.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:26.203 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:26.304 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:26.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:26.326 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:26.326 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:26.392 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:26 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:59:26.392 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:26 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:59:26.392 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:26 smithi018 ceph-mon[29336]: pgmap v646: 168 pgs: 168 active+clean; 264 MiB data, 4.7 GiB used, 532 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 1.10k op/s 2024-09-10T18:59:26.392 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:26 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:59:26.392 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:26 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:59:26.392 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:26 smithi018 ceph-mon[40196]: pgmap v646: 168 pgs: 168 active+clean; 264 MiB data, 4.7 GiB used, 532 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 1.10k op/s 2024-09-10T18:59:26.394 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:26 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T18:59:26.394 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:26 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:59:26.394 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:26 smithi103 ceph-mon[33163]: pgmap v646: 168 pgs: 168 active+clean; 264 MiB data, 4.7 GiB used, 532 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 1.10k op/s 2024-09-10T18:59:26.427 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:26.449 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:26.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:26.450 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:26.551 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:26.574 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:26.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:26.575 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:26.676 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:26.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:26.699 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:26.699 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:26.755 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T18:59:26.755 INFO:teuthology.orchestra.run.smithi103.stdout: "ops": [], 2024-09-10T18:59:26.755 INFO:teuthology.orchestra.run.smithi103.stdout: "num_ops": 0 2024-09-10T18:59:26.755 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T18:59:26.800 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:26.821 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:26.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:26.824 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:26.925 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:26.951 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:26.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:26.952 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:27.022 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:59:26 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:59:27.053 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:27.074 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:27.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:27.075 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:27.176 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:27.202 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:27.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:27.203 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:27.304 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:27.329 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:27.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:27.330 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:27.431 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:27.442 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-10T18:59:27.455 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:27.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:27.456 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:27.557 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:27.578 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:27.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:27.579 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:27.680 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:27.702 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:27.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:27.703 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:27.706 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:59:27.804 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:27.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:27.826 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:27.826 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:27.927 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:27.949 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:27.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:27.950 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:27.974 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_2 2024-09-10T18:59:27.975 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd pool rm unique_pool_2 unique_pool_2 --yes-i-really-really-mean-it 2024-09-10T18:59:28.051 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:28.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:28.076 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:28.077 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:28.115 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:59:28.178 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:28.201 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:28.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:28.202 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:28.303 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:28.326 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:28.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:28.327 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:28.428 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:28.452 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:28.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:28.453 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:28.554 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:28.576 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:28.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:28.577 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:28.614 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:28 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:59:28.614 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:28 smithi018 ceph-mon[40196]: pgmap v647: 168 pgs: 168 active+clean; 28 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.37k op/s 2024-09-10T18:59:28.614 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:28 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:59:28.614 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:28 smithi018 ceph-mon[29336]: pgmap v647: 168 pgs: 168 active+clean; 28 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.37k op/s 2024-09-10T18:59:28.678 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:28.705 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:28.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:28.706 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:28.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:28 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:59:28.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:28 smithi103 ceph-mon[33163]: pgmap v647: 168 pgs: 168 active+clean; 28 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.37k op/s 2024-09-10T18:59:28.807 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:28.829 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:28.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:28.830 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:28.930 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:28.952 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:28.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:28.953 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:29.054 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:29.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:29.076 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:29.077 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:29.177 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:29.201 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:29.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:29.202 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:29.265 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:59:29.265 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:59:29.273 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:59:29.273 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:59:29.292 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:59:29.292 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:59:29.302 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:29.307 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T18:59:29.307 INFO:teuthology.orchestra.run.smithi103.stdout: "ops": [], 2024-09-10T18:59:29.307 INFO:teuthology.orchestra.run.smithi103.stdout: "complaint_time": 30, 2024-09-10T18:59:29.307 INFO:teuthology.orchestra.run.smithi103.stdout: "num_blocked_ops": 0 2024-09-10T18:59:29.307 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T18:59:29.311 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:59:29.311 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:59:29.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:29.326 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:29.326 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:29.327 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:59:29.328 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:59:29.342 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:59:29.343 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:59:29.358 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:59:29.358 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:59:29.374 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:59:29.374 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:59:29.427 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:29.450 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:29.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:29.452 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:29.552 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:29.568 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-10T18:59:29.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:29.576 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:29.577 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:29.678 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:29.701 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:29.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:29.702 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:29.803 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:29.826 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:29.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:29.827 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:29.916 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-10T18:59:29.927 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:29.938 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:29 smithi103 ceph-mon[33163]: osdmap e309: 8 total, 8 up, 6 in 2024-09-10T18:59:29.938 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:29 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/559644118' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:59:29.938 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:29 smithi103 ceph-mon[33163]: pgmap v649: 168 pgs: 168 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 780 B/s rd, 0 B/s wr, 821 op/s 2024-09-10T18:59:29.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:29 smithi018 ceph-mon[29336]: osdmap e309: 8 total, 8 up, 6 in 2024-09-10T18:59:29.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:29 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/559644118' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:59:29.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:29 smithi018 ceph-mon[29336]: pgmap v649: 168 pgs: 168 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 780 B/s rd, 0 B/s wr, 821 op/s 2024-09-10T18:59:29.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:29 smithi018 ceph-mon[40196]: osdmap e309: 8 total, 8 up, 6 in 2024-09-10T18:59:29.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:29 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/559644118' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:59:29.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:29 smithi018 ceph-mon[40196]: pgmap v649: 168 pgs: 168 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 780 B/s rd, 0 B/s wr, 821 op/s 2024-09-10T18:59:29.959 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:29.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:29.960 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:30.061 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:30.084 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:30.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:30.085 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:30.162 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:59:30.186 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:30.209 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:30.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:30.210 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:30.311 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:30.334 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:30.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:30.335 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:30.436 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:30.458 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:30.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:30.459 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:30.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:59:30 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:59:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:59:30.560 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:30.583 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:30.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:30.584 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:30.601 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:59:30.685 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:30.707 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:30.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:30.708 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:30.809 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:30.832 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:30.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:30.833 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:30.933 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:30.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:30.956 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:30.956 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:31.025 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:59:30 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:59:30] "GET /metrics HTTP/1.1" 200 224419 "" "Prometheus/2.43.0" 2024-09-10T18:59:31.057 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:31.080 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:31.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:31.081 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:31.181 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:31.204 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:31.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:31.205 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:31.305 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:31.328 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:31.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:31.329 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:31.430 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:31.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:31.453 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:31.453 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:31.554 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:31.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:31.577 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:31.577 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:31.678 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:31.721 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:31.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:31.722 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:31.823 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:31.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:31.846 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:31.846 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:31.857 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T18:59:31.857 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:59:31.947 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:31.971 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:31.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:31.972 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:32.073 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:32.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:32.095 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:32.095 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:32.196 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:32.219 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:32.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:32.220 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:32.321 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:32.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:32.344 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:32.344 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:32.441 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:32 smithi103 ceph-mon[33163]: pgmap v650: 168 pgs: 168 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 673 op/s 2024-09-10T18:59:32.445 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:32.466 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-10T18:59:32.467 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:32.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:32.468 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:32.508 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:59:32.569 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:32.592 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:32.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:32.593 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:32.693 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:32.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:32 smithi018 ceph-mon[29336]: pgmap v650: 168 pgs: 168 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 673 op/s 2024-09-10T18:59:32.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:32 smithi018 ceph-mon[40196]: pgmap v650: 168 pgs: 168 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 673 op/s 2024-09-10T18:59:32.710 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:59:32.717 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:32.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:32.718 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:32.819 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:32.842 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:32.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:32.843 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:32.944 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:32.967 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:32.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:32.968 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:33.069 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:33.091 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:33.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:33.092 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:33.193 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:33.215 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:33.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:33.216 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:33.317 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:33.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:33.340 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:33.340 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:33.440 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:33.462 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:33.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:33.463 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:33.564 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:33.586 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:33.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:33.587 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:33.647 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:33 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:59:33.648 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:33 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:59:33.648 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:33 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:59:33.648 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:33 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:59:33.687 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:33.690 INFO:teuthology.orchestra.run.smithi018.stderr:reweighted osd.0 to 0.886221 (e2df) 2024-09-10T18:59:33.718 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:33.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:33.719 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:33.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:33 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:59:33.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:33 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:59:33.819 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:33.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:33.841 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:33.841 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:33.942 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:33.963 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:33.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:33.964 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:34.064 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:34.085 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:34.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:34.086 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:34.187 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:34.208 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:34.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:34.209 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:34.310 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:34.333 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:34.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:34.334 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:34.435 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:34.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:34.461 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:34.461 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:34.469 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T18:59:34.469 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:59:34.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:34 smithi103 ceph-mon[33163]: pgmap v651: 168 pgs: 168 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 442 op/s 2024-09-10T18:59:34.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:34 smithi103 ceph-mon[33163]: osdmap e310: 8 total, 8 up, 6 in 2024-09-10T18:59:34.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:34 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1280479588' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 0.8862214262041657}]: dispatch 2024-09-10T18:59:34.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:34 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1280479588' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 0.8862214262041657}]': finished 2024-09-10T18:59:34.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:34 smithi103 ceph-mon[33163]: osdmap e311: 8 total, 8 up, 6 in 2024-09-10T18:59:34.470 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:34 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1507625184' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:59:34.535 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T18:59:34.562 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:34.584 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:34.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:34.585 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:34.686 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:34.693 INFO:teuthology.orchestra.run.smithi018.stderr:pool 'unique_pool_2' removed 2024-09-10T18:59:34.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:34 smithi018 ceph-mon[29336]: pgmap v651: 168 pgs: 168 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 442 op/s 2024-09-10T18:59:34.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:34 smithi018 ceph-mon[29336]: osdmap e310: 8 total, 8 up, 6 in 2024-09-10T18:59:34.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:34 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1280479588' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 0.8862214262041657}]: dispatch 2024-09-10T18:59:34.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:34 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1280479588' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 0.8862214262041657}]': finished 2024-09-10T18:59:34.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:34 smithi018 ceph-mon[29336]: osdmap e311: 8 total, 8 up, 6 in 2024-09-10T18:59:34.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:34 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1507625184' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:59:34.700 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:34 smithi018 ceph-mon[40196]: pgmap v651: 168 pgs: 168 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 442 op/s 2024-09-10T18:59:34.700 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:34 smithi018 ceph-mon[40196]: osdmap e310: 8 total, 8 up, 6 in 2024-09-10T18:59:34.700 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:34 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1280479588' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 0.8862214262041657}]: dispatch 2024-09-10T18:59:34.700 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:34 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1280479588' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 0.8862214262041657}]': finished 2024-09-10T18:59:34.701 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:34 smithi018 ceph-mon[40196]: osdmap e311: 8 total, 8 up, 6 in 2024-09-10T18:59:34.701 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:34 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1507625184' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:59:34.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:34.734 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:34.734 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:34.835 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:34.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:34.857 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:34.857 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:34.958 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:34.981 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:34.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:34.982 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:35.083 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:35.105 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:35.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:35.106 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:35.207 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:35.229 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:35.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:35.230 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:35.239 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-10T18:59:35.330 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:35.352 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:35.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:35.353 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:35.454 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:35.477 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:35.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:35.478 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:35.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:35 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:59:35.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:35 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:59:35.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:35 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1507625184' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T18:59:35.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:35 smithi103 ceph-mon[33163]: osdmap e312: 8 total, 8 up, 6 in 2024-09-10T18:59:35.529 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:59:35.578 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:35.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:35.601 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:35.601 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:35.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:35 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:59:35.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:35 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:59:35.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:35 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1507625184' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T18:59:35.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:35 smithi018 ceph-mon[29336]: osdmap e312: 8 total, 8 up, 6 in 2024-09-10T18:59:35.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:35 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:59:35.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:35 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:59:35.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:35 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1507625184' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T18:59:35.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:35 smithi018 ceph-mon[40196]: osdmap e312: 8 total, 8 up, 6 in 2024-09-10T18:59:35.702 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:35.725 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:35.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:35.726 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:35.826 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:35.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:35.848 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:35.849 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:35.949 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:35.973 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:35.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:35.974 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:36.075 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:36.099 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:36.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:36.100 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:36.200 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:36.224 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:36.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:36.225 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:36.326 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:36.348 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:36.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:36.349 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:36.450 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:36.473 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:36.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:36.475 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:36.575 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:36.597 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:36.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:36.598 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:36.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:36 smithi018 ceph-mon[29336]: pgmap v655: 152 pgs: 152 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 682 B/s rd, 0 op/s 2024-09-10T18:59:36.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:36 smithi018 ceph-mon[40196]: pgmap v655: 152 pgs: 152 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 682 B/s rd, 0 op/s 2024-09-10T18:59:36.699 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:36.721 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:36.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:36.722 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:36.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:36 smithi103 ceph-mon[33163]: pgmap v655: 152 pgs: 152 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 682 B/s rd, 0 op/s 2024-09-10T18:59:36.823 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:36.844 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:36.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:36.846 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:36.848 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-10T18:59:36.849 INFO:tasks.radosbench:Beginning radosbench... 2024-09-10T18:59:36.849 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_3 2024-09-10T18:59:36.849 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd pool create unique_pool_3 16 2024-09-10T18:59:36.947 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:36.968 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:36.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:36.970 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:37.069 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:59:36 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:59:37.070 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:37.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:37.098 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:37.098 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:37.103 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:59:37.149 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:59:37.177 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T18:59:37.177 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:59:37.199 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:37.220 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:37.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:37.221 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:37.322 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:37.344 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:37.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:37.345 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:37.419 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is set 2024-09-10T18:59:37.446 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:37.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:37.470 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:37.470 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:37.560 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:37 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1739872170' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:59:37.561 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:37 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1739872170' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:59:37.571 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:37.594 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:37.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:37.595 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:37.696 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:37.720 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:37.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:37.721 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:37.765 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:37 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1739872170' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:59:37.794 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-10T18:59:37.822 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:37.844 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:37.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:37.845 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:37.946 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:37.968 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:37.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:37.969 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:38.038 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:59:38.070 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:38.091 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:38.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:38.092 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:38.193 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:38.216 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:38.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:38.217 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:38.318 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:38.341 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:38.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:38.342 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:38.443 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:38.466 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:38.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:38.467 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:38.568 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:38.589 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:38.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:38.590 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:38.619 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:38 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:59:38.620 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:38 smithi018 ceph-mon[40196]: pgmap v656: 152 pgs: 152 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-10T18:59:38.620 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:38 smithi018 ceph-mon[40196]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:59:38.620 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:38 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1739872170' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:59:38.620 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:38 smithi018 ceph-mon[40196]: osdmap e313: 8 total, 8 up, 6 in 2024-09-10T18:59:38.620 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:38 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:59:38.620 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:38 smithi018 ceph-mon[29336]: pgmap v656: 152 pgs: 152 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-10T18:59:38.620 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:38 smithi018 ceph-mon[29336]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:59:38.620 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:38 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1739872170' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:59:38.620 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:38 smithi018 ceph-mon[29336]: osdmap e313: 8 total, 8 up, 6 in 2024-09-10T18:59:38.691 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:38.716 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:38.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:38.718 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:38.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:38 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:59:38.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:38 smithi103 ceph-mon[33163]: pgmap v656: 152 pgs: 152 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-10T18:59:38.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:38 smithi103 ceph-mon[33163]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:59:38.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:38 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1739872170' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T18:59:38.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:38 smithi103 ceph-mon[33163]: osdmap e313: 8 total, 8 up, 6 in 2024-09-10T18:59:38.818 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:38.842 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:38.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:38.843 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:38.943 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:38.965 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:38.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:38.966 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:39.067 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:39.089 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:39.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:39.090 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:39.191 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:39.213 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:39.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:39.276 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:39.377 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:39.399 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:39.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:39.400 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:39.501 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:39.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:39.526 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:39.526 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:39.627 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:39.644 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T18:59:39.644 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:59:39.648 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:39.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:39.649 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:39.750 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:39.772 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:39.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:39.773 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:39.874 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:39.897 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:39.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:39.898 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:39.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:39 smithi018 ceph-mon[29336]: osdmap e314: 8 total, 8 up, 6 in 2024-09-10T18:59:39.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:39 smithi018 ceph-mon[29336]: pgmap v659: 152 pgs: 152 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 722 B/s rd, 0 op/s 2024-09-10T18:59:39.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:39 smithi018 ceph-mon[40196]: osdmap e314: 8 total, 8 up, 6 in 2024-09-10T18:59:39.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:39 smithi018 ceph-mon[40196]: pgmap v659: 152 pgs: 152 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 722 B/s rd, 0 op/s 2024-09-10T18:59:39.999 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:40.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:39 smithi103 ceph-mon[33163]: osdmap e314: 8 total, 8 up, 6 in 2024-09-10T18:59:40.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:39 smithi103 ceph-mon[33163]: pgmap v659: 152 pgs: 152 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 722 B/s rd, 0 op/s 2024-09-10T18:59:40.023 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:40.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:40.024 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:40.125 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:40.148 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:40.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:40.149 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:40.250 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:40.263 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-10T18:59:40.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:40.273 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:40.273 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:40.340 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6] out_osds: [2, 7] dead_osds: [] live_osds: [1, 2, 3, 4, 5, 6, 7, 0] 2024-09-10T18:59:40.341 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd reweight 0 1 2024-09-10T18:59:40.350 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-10T18:59:40.374 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:40.407 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:40.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:40.409 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:40.509 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:40.517 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:59:40.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:59:40 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:59:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:59:40.531 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:40.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:40.532 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:40.633 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:40.655 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:40.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:40.656 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:40.756 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:40.777 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:40.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:40.779 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:40.841 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:40 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:59:40.842 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:40 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:59:40.879 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:40.902 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:40.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:40.903 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:40.968 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:40 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:59:40.985 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:59:40.985 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:59:40.985 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:59:40.985 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:59:40.985 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:59:40.985 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:59:40.985 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:59:40.986 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:59:40.986 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:59:40.986 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:59:40.986 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:59:40.986 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:59:40.986 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:59:40.986 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:59:40.987 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:59:40.987 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:59:41.004 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:41.025 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:41.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:41.027 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:41.120 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:59:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:59:40] "GET /metrics HTTP/1.1" 200 222033 "" "Prometheus/2.43.0" 2024-09-10T18:59:41.127 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:41.148 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:41.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:41.149 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:41.250 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:41.275 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:41.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:41.276 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:41.377 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:41.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:41.400 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:41.400 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:41.501 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:41.522 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:41.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:41.525 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:41.625 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:41.648 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:41.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:41.649 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:41.707 INFO:teuthology.orchestra.run.smithi018.stderr:pool 'unique_pool_3' created 2024-09-10T18:59:41.751 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:41.765 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:41 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2885974485' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:59:41.765 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:41 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/971840337' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]: dispatch 2024-09-10T18:59:41.765 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:41 smithi018 ceph-mon[40196]: pgmap v660: 152 pgs: 152 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 768 B/s rd, 0 op/s; 51 B/s, 1 objects/s recovering 2024-09-10T18:59:41.766 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:41 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2885974485' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:59:41.766 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:41 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/971840337' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]: dispatch 2024-09-10T18:59:41.766 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:41 smithi018 ceph-mon[29336]: pgmap v660: 152 pgs: 152 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 768 B/s rd, 0 op/s; 51 B/s, 1 objects/s recovering 2024-09-10T18:59:41.774 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:41.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:41.775 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:41.876 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:41.897 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:41.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:41.898 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:41.899 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:41 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2885974485' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:59:41.899 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:41 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/971840337' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]: dispatch 2024-09-10T18:59:41.899 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:41 smithi103 ceph-mon[33163]: pgmap v660: 152 pgs: 152 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 768 B/s rd, 0 op/s; 51 B/s, 1 objects/s recovering 2024-09-10T18:59:41.999 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:42.022 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:42.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:42.024 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:42.108 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T18:59:42.108 INFO:teuthology.orchestra.run.smithi103.stdout: "ops": [], 2024-09-10T18:59:42.109 INFO:teuthology.orchestra.run.smithi103.stdout: "num_ops": 0 2024-09-10T18:59:42.109 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T18:59:42.124 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:42.148 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:42.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:42.149 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:42.250 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:42.273 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:42.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:42.274 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:42.375 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:42.377 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd pool application enable unique_pool_3 rados --yes-i-really-mean-it || true 2024-09-10T18:59:42.441 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:42.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:42.442 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:42.444 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:59:42.446 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:59:42.542 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:42.565 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:42.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:42.566 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:42.667 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:42.690 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:42.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:42.691 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:42.791 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:42.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:42.816 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:42.816 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:42.915 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:42 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/971840337' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]': finished 2024-09-10T18:59:42.916 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:42 smithi018 ceph-mon[40196]: osdmap e315: 8 total, 8 up, 6 in 2024-09-10T18:59:42.916 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:42 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/971840337' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]': finished 2024-09-10T18:59:42.916 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:42 smithi018 ceph-mon[29336]: osdmap e315: 8 total, 8 up, 6 in 2024-09-10T18:59:42.916 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:42.940 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:42.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:42.941 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:42.974 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:42 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/971840337' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]': finished 2024-09-10T18:59:42.974 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:42 smithi103 ceph-mon[33163]: osdmap e315: 8 total, 8 up, 6 in 2024-09-10T18:59:42.999 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-10T18:59:43.042 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:43.063 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:43.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:43.064 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:43.165 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:43.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:43.189 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:43.189 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:43.247 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:59:43.290 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:43.314 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:43.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:43.316 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:43.416 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:43.439 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:43.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:43.440 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:43.541 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:43.564 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:43.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:43.566 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:43.667 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:43.689 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:43.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:43.690 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:43.767 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T18:59:43.791 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:43.814 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:43.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:43.815 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:43.916 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:43.938 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:43.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:43.939 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:44.024 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:43 smithi103 ceph-mon[33163]: osdmap e316: 8 total, 8 up, 6 in 2024-09-10T18:59:44.024 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:43 smithi103 ceph-mon[33163]: pgmap v663: 168 pgs: 1 creating+peering, 15 unknown, 152 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 862 B/s rd, 0 op/s 2024-09-10T18:59:44.024 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:43 smithi103 ceph-mon[33163]: osdmap e317: 8 total, 8 up, 6 in 2024-09-10T18:59:44.040 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:44.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:44.062 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:44.062 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:44.163 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:44.184 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:44.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:44.185 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:44.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:43 smithi018 ceph-mon[29336]: osdmap e316: 8 total, 8 up, 6 in 2024-09-10T18:59:44.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:43 smithi018 ceph-mon[29336]: pgmap v663: 168 pgs: 1 creating+peering, 15 unknown, 152 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 862 B/s rd, 0 op/s 2024-09-10T18:59:44.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:43 smithi018 ceph-mon[29336]: osdmap e317: 8 total, 8 up, 6 in 2024-09-10T18:59:44.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:43 smithi018 ceph-mon[40196]: osdmap e316: 8 total, 8 up, 6 in 2024-09-10T18:59:44.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:43 smithi018 ceph-mon[40196]: pgmap v663: 168 pgs: 1 creating+peering, 15 unknown, 152 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 862 B/s rd, 0 op/s 2024-09-10T18:59:44.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:43 smithi018 ceph-mon[40196]: osdmap e317: 8 total, 8 up, 6 in 2024-09-10T18:59:44.265 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:59:44.286 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:44.312 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:44.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:44.313 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:44.414 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:44.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:44.437 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:44.437 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:44.538 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:44.561 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:44.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:44.562 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:44.662 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:44.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:44.685 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:44.685 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:44.786 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:44.787 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T18:59:44.787 INFO:teuthology.orchestra.run.smithi103.stdout: "ops": [], 2024-09-10T18:59:44.787 INFO:teuthology.orchestra.run.smithi103.stdout: "complaint_time": 30, 2024-09-10T18:59:44.787 INFO:teuthology.orchestra.run.smithi103.stdout: "num_blocked_ops": 0 2024-09-10T18:59:44.787 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T18:59:44.809 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:44.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:44.810 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:44.813 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:44 smithi018 ceph-mon[40196]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:59:44.814 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:44 smithi018 ceph-mon[29336]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:59:44.910 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:44.933 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:44.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:44.934 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:45.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:44 smithi103 ceph-mon[33163]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:59:45.035 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:45.056 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:45.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:45.057 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:45.158 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:45.180 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:45.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:45.181 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:45.281 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:45.305 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:45.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:45.306 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:45.407 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:45.414 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-10T18:59:45.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:45.429 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:45.430 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:45.531 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:45.555 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:45.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:45.556 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:45.657 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:45.669 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:59:45.681 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:45.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:45.682 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:45.782 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:45.805 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:45.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:45.806 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:45.907 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:45.929 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:45.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:45.930 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:45.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:45 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:59:45.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:45 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:59:45.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:45 smithi018 ceph-mon[29336]: osdmap e318: 8 total, 8 up, 6 in 2024-09-10T18:59:45.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:45 smithi018 ceph-mon[29336]: pgmap v666: 168 pgs: 1 creating+peering, 12 unknown, 155 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:45.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:45 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:59:45.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:45 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:59:45.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:45 smithi018 ceph-mon[40196]: osdmap e318: 8 total, 8 up, 6 in 2024-09-10T18:59:45.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:45 smithi018 ceph-mon[40196]: pgmap v666: 168 pgs: 1 creating+peering, 12 unknown, 155 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:46.011 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:45 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:59:46.012 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:45 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:59:46.012 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:45 smithi103 ceph-mon[33163]: osdmap e318: 8 total, 8 up, 6 in 2024-09-10T18:59:46.012 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:45 smithi103 ceph-mon[33163]: pgmap v666: 168 pgs: 1 creating+peering, 12 unknown, 155 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:46.031 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:46.054 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:46.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:46.055 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:46.156 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:46.178 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:46.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:46.179 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:46.280 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:46.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:46.303 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:46.304 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:46.405 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:46.427 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:46.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:46.428 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:46.529 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:46.551 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:46.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:46.552 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:46.631 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:59:46.653 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:46.675 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:46.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:46.676 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:46.731 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is unset 2024-09-10T18:59:46.777 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:46.799 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:46.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:46.800 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:46.891 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:46 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1686727005' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:59:46.892 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:46 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:59:46.892 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:46 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1686727005' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:59:46.892 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:46 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:59:46.901 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:46.923 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:46.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:46.924 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:47.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:46 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1686727005' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:59:47.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:46 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T18:59:47.023 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:59:46 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:59:47.024 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:47.046 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:47.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:47.047 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:47.148 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:47.169 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:47.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:47.170 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:47.271 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:47.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:47.296 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:47.296 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:47.397 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:47.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:47.420 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:47.421 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:47.522 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:47.530 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T18:59:47.530 INFO:teuthology.orchestra.run.smithi103.stdout: "size": 20, 2024-09-10T18:59:47.530 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 600, 2024-09-10T18:59:47.530 INFO:teuthology.orchestra.run.smithi103.stdout: "ops": [ 2024-09-10T18:59:47.530 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.530 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:115933 8.9 8:90b98b7c:::benchmark_data_smithi018_106531_object115932:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e183)", 2024-09-10T18:59:47.531 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:52:53.593001+0000", 2024-09-10T18:59:47.531 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 413.89799393499999, 2024-09-10T18:59:47.531 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.0933817010000002, 2024-09-10T18:59:47.531 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:59:47.531 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:59:47.531 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:59:47.531 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:59:47.531 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:59:47.531 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 115933 2024-09-10T18:59:47.531 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.531 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:59:47.531 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.532 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:59:47.532 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.593001+0000", 2024-09-10T18:59:47.532 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.532 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.532 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.532 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:59:47.532 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.593001+0000", 2024-09-10T18:59:47.532 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.532 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.532 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.532 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:59:47.532 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.593000+0000", 2024-09-10T18:59:47.533 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:59:47.533 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.533 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.533 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:59:47.533 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.593054+0000", 2024-09-10T18:59:47.533 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.4162000000000001e-05 2024-09-10T18:59:47.533 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.533 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.533 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:59:47.533 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.593066+0000", 2024-09-10T18:59:47.533 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.2206e-05 2024-09-10T18:59:47.533 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.534 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.534 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:59:47.534 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.593072+0000", 2024-09-10T18:59:47.534 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.643e-06 2024-09-10T18:59:47.534 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.534 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.534 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.534 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.593113+0000", 2024-09-10T18:59:47.534 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.0914999999999997e-05 2024-09-10T18:59:47.534 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.534 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.534 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:59:47.534 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.593120+0000", 2024-09-10T18:59:47.535 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.2620000000000001e-06 2024-09-10T18:59:47.535 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.535 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.535 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.535 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:54.576943+0000", 2024-09-10T18:59:47.535 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.98382287499999999 2024-09-10T18:59:47.535 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.535 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.535 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:59:47.535 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:54.576954+0000", 2024-09-10T18:59:47.535 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.1112e-05 2024-09-10T18:59:47.535 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.535 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.536 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.536 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.665292+0000", 2024-09-10T18:59:47.536 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0883377379999999 2024-09-10T18:59:47.536 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.536 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.536 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:59:47.536 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.665473+0000", 2024-09-10T18:59:47.536 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000180963 2024-09-10T18:59:47.536 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.536 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.536 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,3,5", 2024-09-10T18:59:47.536 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.667230+0000", 2024-09-10T18:59:47.537 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.001756608 2024-09-10T18:59:47.537 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.537 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.537 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:59:47.537 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.680606+0000", 2024-09-10T18:59:47.537 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.013376825 2024-09-10T18:59:47.537 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.537 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.537 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.537 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686161+0000", 2024-09-10T18:59:47.537 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0055545710000000003 2024-09-10T18:59:47.537 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.537 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.538 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.538 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686190+0000", 2024-09-10T18:59:47.538 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.8856999999999998e-05 2024-09-10T18:59:47.538 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.538 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.538 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.538 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686244+0000", 2024-09-10T18:59:47.538 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.4577000000000003e-05 2024-09-10T18:59:47.538 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.538 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.538 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:59:47.538 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686289+0000", 2024-09-10T18:59:47.538 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.4329999999999997e-05 2024-09-10T18:59:47.538 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.539 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.539 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:59:47.539 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686383+0000", 2024-09-10T18:59:47.539 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.4469999999999995e-05 2024-09-10T18:59:47.539 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.539 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:59:47.539 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.539 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.539 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.539 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:115942 8.9 8:966c7925:::benchmark_data_smithi018_106531_object115941:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e183)", 2024-09-10T18:59:47.539 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:52:53.598843+0000", 2024-09-10T18:59:47.539 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 413.89215253100002, 2024-09-10T18:59:47.539 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.0877665900000002, 2024-09-10T18:59:47.539 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:59:47.540 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:59:47.540 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:59:47.540 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:59:47.540 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:59:47.540 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 115942 2024-09-10T18:59:47.540 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.540 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:59:47.540 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.540 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:59:47.540 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.598843+0000", 2024-09-10T18:59:47.540 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.540 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.540 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.541 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:59:47.541 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.598843+0000", 2024-09-10T18:59:47.541 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.541 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.541 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.541 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:59:47.541 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.598841+0000", 2024-09-10T18:59:47.541 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:59:47.541 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.541 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.541 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:59:47.541 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.598871+0000", 2024-09-10T18:59:47.541 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.9961999999999999e-05 2024-09-10T18:59:47.542 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.542 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.542 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:59:47.542 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.598881+0000", 2024-09-10T18:59:47.542 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.3489999999999998e-06 2024-09-10T18:59:47.542 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.542 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.542 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:59:47.542 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.598885+0000", 2024-09-10T18:59:47.542 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.296e-06 2024-09-10T18:59:47.542 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.542 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.542 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.542 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.598944+0000", 2024-09-10T18:59:47.543 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.9110000000000002e-05 2024-09-10T18:59:47.543 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.543 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.543 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:59:47.543 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.598954+0000", 2024-09-10T18:59:47.543 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.7200000000000001e-06 2024-09-10T18:59:47.543 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.543 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.543 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.543 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:54.576976+0000", 2024-09-10T18:59:47.543 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.97802252300000003 2024-09-10T18:59:47.543 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.544 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.544 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:59:47.544 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:54.576983+0000", 2024-09-10T18:59:47.544 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.1149999999999996e-06 2024-09-10T18:59:47.544 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.544 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.544 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.544 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.671002+0000", 2024-09-10T18:59:47.544 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0940191109999999 2024-09-10T18:59:47.544 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.544 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.544 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:59:47.544 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.671130+0000", 2024-09-10T18:59:47.544 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00012751599999999999 2024-09-10T18:59:47.545 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.545 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.545 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,3,5", 2024-09-10T18:59:47.545 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.671299+0000", 2024-09-10T18:59:47.545 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00016920699999999999 2024-09-10T18:59:47.545 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.545 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.545 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:59:47.545 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.680614+0000", 2024-09-10T18:59:47.545 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0093144209999999998 2024-09-10T18:59:47.545 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.545 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.546 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.546 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686217+0000", 2024-09-10T18:59:47.546 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0056036649999999999 2024-09-10T18:59:47.546 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.546 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.546 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.546 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686416+0000", 2024-09-10T18:59:47.546 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00019841300000000001 2024-09-10T18:59:47.546 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.546 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.546 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.546 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686470+0000", 2024-09-10T18:59:47.546 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.4048999999999997e-05 2024-09-10T18:59:47.547 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.547 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.547 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:59:47.547 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686507+0000", 2024-09-10T18:59:47.547 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.6950999999999999e-05 2024-09-10T18:59:47.547 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.547 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.547 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:59:47.547 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686609+0000", 2024-09-10T18:59:47.547 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000102784 2024-09-10T18:59:47.547 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.547 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:59:47.547 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.548 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.548 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.548 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:115980 8.9 8:9c48a6ac:::benchmark_data_smithi018_106531_object115979:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e183)", 2024-09-10T18:59:47.548 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:52:53.631969+0000", 2024-09-10T18:59:47.548 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 413.85902681499999, 2024-09-10T18:59:47.548 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.0549517110000002, 2024-09-10T18:59:47.548 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:59:47.548 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:59:47.548 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:59:47.548 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:59:47.548 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:59:47.548 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 115980 2024-09-10T18:59:47.548 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.549 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:59:47.549 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.549 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:59:47.549 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.631969+0000", 2024-09-10T18:59:47.549 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.549 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.549 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.549 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:59:47.549 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.631969+0000", 2024-09-10T18:59:47.549 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.549 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.549 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.549 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:59:47.550 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.631967+0000", 2024-09-10T18:59:47.550 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:59:47.550 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.550 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.550 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:59:47.550 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.632008+0000", 2024-09-10T18:59:47.550 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.1397999999999999e-05 2024-09-10T18:59:47.550 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.550 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.550 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:59:47.550 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.632019+0000", 2024-09-10T18:59:47.550 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0875e-05 2024-09-10T18:59:47.550 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.551 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.551 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:59:47.551 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.632025+0000", 2024-09-10T18:59:47.551 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.8499999999999999e-06 2024-09-10T18:59:47.551 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.551 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.551 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.551 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.632091+0000", 2024-09-10T18:59:47.551 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.5895999999999994e-05 2024-09-10T18:59:47.551 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.551 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.551 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:59:47.551 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.632102+0000", 2024-09-10T18:59:47.551 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0992000000000001e-05 2024-09-10T18:59:47.552 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.552 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.552 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.552 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:54.577003+0000", 2024-09-10T18:59:47.552 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.94490056 2024-09-10T18:59:47.552 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.552 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.552 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:59:47.552 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:54.577009+0000", 2024-09-10T18:59:47.552 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.1990000000000003e-06 2024-09-10T18:59:47.552 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.552 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.552 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.552 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.673498+0000", 2024-09-10T18:59:47.553 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.096489402 2024-09-10T18:59:47.553 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.553 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.553 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:59:47.553 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.673643+0000", 2024-09-10T18:59:47.553 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00014505 2024-09-10T18:59:47.553 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.553 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.553 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,3,5", 2024-09-10T18:59:47.553 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.673801+0000", 2024-09-10T18:59:47.553 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000157608 2024-09-10T18:59:47.553 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.553 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.554 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:59:47.554 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.680622+0000", 2024-09-10T18:59:47.554 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0068210340000000001 2024-09-10T18:59:47.554 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.554 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.554 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.554 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686443+0000", 2024-09-10T18:59:47.554 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0058211000000000001 2024-09-10T18:59:47.554 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.554 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.554 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.554 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686659+0000", 2024-09-10T18:59:47.554 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000216276 2024-09-10T18:59:47.555 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.555 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.555 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.555 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686773+0000", 2024-09-10T18:59:47.555 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000113706 2024-09-10T18:59:47.555 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.555 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.555 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:59:47.555 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686812+0000", 2024-09-10T18:59:47.555 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.8617e-05 2024-09-10T18:59:47.555 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.555 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.555 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:59:47.556 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686920+0000", 2024-09-10T18:59:47.556 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000108651 2024-09-10T18:59:47.556 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.556 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:59:47.556 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.556 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.556 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.556 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:115987 8.9 8:94f7fdf8:::benchmark_data_smithi018_106531_object115986:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e183)", 2024-09-10T18:59:47.556 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:52:53.636486+0000", 2024-09-10T18:59:47.556 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 413.85450956800003, 2024-09-10T18:59:47.556 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.0506323740000001, 2024-09-10T18:59:47.556 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:59:47.556 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:59:47.557 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:59:47.557 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:59:47.557 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:59:47.557 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 115987 2024-09-10T18:59:47.557 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.557 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:59:47.557 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.557 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:59:47.557 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.636486+0000", 2024-09-10T18:59:47.557 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.557 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.557 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.557 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:59:47.558 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.636486+0000", 2024-09-10T18:59:47.558 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.558 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.558 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.558 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:59:47.558 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.636484+0000", 2024-09-10T18:59:47.558 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:59:47.558 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.558 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.558 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:59:47.558 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.636528+0000", 2024-09-10T18:59:47.558 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.4143000000000001e-05 2024-09-10T18:59:47.558 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.559 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.559 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:59:47.559 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.636542+0000", 2024-09-10T18:59:47.559 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.3889e-05 2024-09-10T18:59:47.559 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.559 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.559 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:59:47.559 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.636548+0000", 2024-09-10T18:59:47.559 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.0129999999999999e-06 2024-09-10T18:59:47.559 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.559 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.559 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.559 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.636644+0000", 2024-09-10T18:59:47.560 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.5265000000000004e-05 2024-09-10T18:59:47.560 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.560 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.560 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:59:47.560 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.636654+0000", 2024-09-10T18:59:47.560 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0414999999999999e-05 2024-09-10T18:59:47.560 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.560 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.560 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.560 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:54.577026+0000", 2024-09-10T18:59:47.560 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.94037243400000003 2024-09-10T18:59:47.560 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.560 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.561 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:59:47.561 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:54.577033+0000", 2024-09-10T18:59:47.561 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.4489999999999996e-06 2024-09-10T18:59:47.561 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.561 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.561 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.561 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.676009+0000", 2024-09-10T18:59:47.561 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.098976314 2024-09-10T18:59:47.561 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.561 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.561 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:59:47.561 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.676099+0000", 2024-09-10T18:59:47.561 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.9402999999999998e-05 2024-09-10T18:59:47.562 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.562 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.562 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,3,5", 2024-09-10T18:59:47.562 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.676215+0000", 2024-09-10T18:59:47.562 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000116106 2024-09-10T18:59:47.562 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.562 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.562 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:59:47.562 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.685890+0000", 2024-09-10T18:59:47.562 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0096749999999999996 2024-09-10T18:59:47.562 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.562 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.562 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.563 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686696+0000", 2024-09-10T18:59:47.563 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00080583300000000005 2024-09-10T18:59:47.563 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.563 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.563 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.563 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686956+0000", 2024-09-10T18:59:47.563 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00026063300000000002 2024-09-10T18:59:47.563 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.563 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.563 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.563 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686983+0000", 2024-09-10T18:59:47.563 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.6897999999999999e-05 2024-09-10T18:59:47.563 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.564 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.564 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:59:47.564 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.687024+0000", 2024-09-10T18:59:47.564 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.1143000000000002e-05 2024-09-10T18:59:47.564 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.564 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.564 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:59:47.564 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.687118+0000", 2024-09-10T18:59:47.564 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.3961000000000004e-05 2024-09-10T18:59:47.564 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.564 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:59:47.564 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.565 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.565 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.565 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:116013 8.9 8:923f941a:::benchmark_data_smithi018_106531_object116012:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e183)", 2024-09-10T18:59:47.565 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:52:53.661699+0000", 2024-09-10T18:59:47.565 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 413.82929633600003, 2024-09-10T18:59:47.565 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.0258877599999998, 2024-09-10T18:59:47.565 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:59:47.565 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:59:47.565 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:59:47.565 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:59:47.565 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:59:47.565 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 116013 2024-09-10T18:59:47.566 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.566 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:59:47.566 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.566 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:59:47.566 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.661699+0000", 2024-09-10T18:59:47.566 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.566 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.566 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.566 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:59:47.566 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.661699+0000", 2024-09-10T18:59:47.566 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.566 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.567 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.567 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:59:47.567 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.661698+0000", 2024-09-10T18:59:47.567 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:59:47.567 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.567 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.567 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:59:47.567 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.661753+0000", 2024-09-10T18:59:47.567 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.5143999999999999e-05 2024-09-10T18:59:47.567 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.567 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.567 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:59:47.567 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.661764+0000", 2024-09-10T18:59:47.567 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0651e-05 2024-09-10T18:59:47.568 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.568 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.568 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:59:47.568 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.661768+0000", 2024-09-10T18:59:47.568 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.3189999999999996e-06 2024-09-10T18:59:47.568 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.568 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.568 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.568 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.661810+0000", 2024-09-10T18:59:47.568 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.2271999999999998e-05 2024-09-10T18:59:47.568 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.568 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.568 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:59:47.569 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.661819+0000", 2024-09-10T18:59:47.569 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.6589999999999996e-06 2024-09-10T18:59:47.569 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.569 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.569 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.569 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:54.577071+0000", 2024-09-10T18:59:47.569 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.91525191500000003 2024-09-10T18:59:47.569 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.569 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.569 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:59:47.569 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:54.577077+0000", 2024-09-10T18:59:47.569 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.4160000000000001e-06 2024-09-10T18:59:47.569 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.570 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.570 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.570 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.678501+0000", 2024-09-10T18:59:47.570 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.101423915 2024-09-10T18:59:47.570 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.570 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.570 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:59:47.570 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.678633+0000", 2024-09-10T18:59:47.570 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00013232200000000001 2024-09-10T18:59:47.570 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.570 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.571 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,3,5", 2024-09-10T18:59:47.571 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.678805+0000", 2024-09-10T18:59:47.571 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000171103 2024-09-10T18:59:47.571 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.571 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.571 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:59:47.571 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.685940+0000", 2024-09-10T18:59:47.571 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0071355940000000003 2024-09-10T18:59:47.571 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.571 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.571 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.571 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.686734+0000", 2024-09-10T18:59:47.571 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00079338399999999995 2024-09-10T18:59:47.572 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.572 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.572 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.572 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.687202+0000", 2024-09-10T18:59:47.572 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00046798199999999998 2024-09-10T18:59:47.572 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.572 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.572 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.572 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.687314+0000", 2024-09-10T18:59:47.572 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00011257200000000001 2024-09-10T18:59:47.572 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.572 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.572 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:59:47.572 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.687361+0000", 2024-09-10T18:59:47.573 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.7308000000000002e-05 2024-09-10T18:59:47.573 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.573 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.573 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:59:47.573 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.687587+0000", 2024-09-10T18:59:47.573 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000225443 2024-09-10T18:59:47.573 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.573 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:59:47.573 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.573 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.573 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.573 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:116021 8.9 8:985865c8:::benchmark_data_smithi018_106531_object116020:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e183)", 2024-09-10T18:59:47.574 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:52:53.671662+0000", 2024-09-10T18:59:47.574 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 413.81933317099998, 2024-09-10T18:59:47.574 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.0160958120000001, 2024-09-10T18:59:47.574 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:59:47.574 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:59:47.574 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:59:47.574 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:59:47.574 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:59:47.574 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 116021 2024-09-10T18:59:47.574 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.574 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:59:47.574 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.574 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:59:47.575 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.671662+0000", 2024-09-10T18:59:47.575 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.575 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.575 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.575 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:59:47.575 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.671662+0000", 2024-09-10T18:59:47.575 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.575 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.575 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.575 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:59:47.575 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.671661+0000", 2024-09-10T18:59:47.575 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:59:47.576 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.576 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.576 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:59:47.576 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.671709+0000", 2024-09-10T18:59:47.576 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.7559000000000003e-05 2024-09-10T18:59:47.576 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.576 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.576 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:59:47.576 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.671720+0000", 2024-09-10T18:59:47.576 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.1324e-05 2024-09-10T18:59:47.576 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.576 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.576 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:59:47.577 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.671724+0000", 2024-09-10T18:59:47.577 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.0980000000000004e-06 2024-09-10T18:59:47.577 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.577 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.577 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.577 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.671774+0000", 2024-09-10T18:59:47.577 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.0096000000000003e-05 2024-09-10T18:59:47.577 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.577 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.577 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:59:47.577 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.671781+0000", 2024-09-10T18:59:47.577 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.255e-06 2024-09-10T18:59:47.578 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.578 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.578 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.578 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:54.577099+0000", 2024-09-10T18:59:47.578 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.905317661 2024-09-10T18:59:47.578 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.578 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.578 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:59:47.578 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:54.577105+0000", 2024-09-10T18:59:47.578 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.3770000000000001e-06 2024-09-10T18:59:47.578 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.578 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.578 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.579 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.680666+0000", 2024-09-10T18:59:47.579 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.1035605239999999 2024-09-10T18:59:47.579 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.579 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.579 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:59:47.579 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.680802+0000", 2024-09-10T18:59:47.579 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000136503 2024-09-10T18:59:47.579 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.579 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.579 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,3,5", 2024-09-10T18:59:47.579 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.680991+0000", 2024-09-10T18:59:47.579 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000188786 2024-09-10T18:59:47.579 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.580 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.580 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:59:47.580 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.685946+0000", 2024-09-10T18:59:47.580 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.004954366 2024-09-10T18:59:47.580 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.580 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.580 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.580 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.687233+0000", 2024-09-10T18:59:47.580 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0012871029999999999 2024-09-10T18:59:47.580 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.580 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.580 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.580 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.687258+0000", 2024-09-10T18:59:47.581 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.5483e-05 2024-09-10T18:59:47.581 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.581 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.581 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.581 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.687626+0000", 2024-09-10T18:59:47.581 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00036776300000000001 2024-09-10T18:59:47.581 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.581 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.581 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:59:47.581 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.687669+0000", 2024-09-10T18:59:47.581 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.3053999999999998e-05 2024-09-10T18:59:47.581 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.582 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.582 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:59:47.582 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.687758+0000", 2024-09-10T18:59:47.582 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.9036999999999995e-05 2024-09-10T18:59:47.582 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.582 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:59:47.582 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.582 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.582 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.582 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:116041 8.9 8:9e49af6d:::benchmark_data_smithi018_106531_object116040:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e183)", 2024-09-10T18:59:47.582 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:52:53.703046+0000", 2024-09-10T18:59:47.582 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 413.78794989800002, 2024-09-10T18:59:47.582 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.993576947, 2024-09-10T18:59:47.583 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:59:47.583 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:59:47.583 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:59:47.583 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:59:47.583 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:59:47.583 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 116041 2024-09-10T18:59:47.583 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.583 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:59:47.583 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.583 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:59:47.583 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.703046+0000", 2024-09-10T18:59:47.583 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.583 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.584 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.584 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:59:47.584 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.703046+0000", 2024-09-10T18:59:47.584 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.584 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.584 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.584 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:59:47.584 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.703044+0000", 2024-09-10T18:59:47.584 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:59:47.584 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.584 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.584 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:59:47.584 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.703109+0000", 2024-09-10T18:59:47.584 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.4926999999999997e-05 2024-09-10T18:59:47.585 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.585 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.585 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:59:47.585 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.703121+0000", 2024-09-10T18:59:47.585 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.1892999999999999e-05 2024-09-10T18:59:47.585 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.585 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.585 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:59:47.585 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.703125+0000", 2024-09-10T18:59:47.585 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.3189999999999996e-06 2024-09-10T18:59:47.585 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.585 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.586 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.586 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.703177+0000", 2024-09-10T18:59:47.586 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.1378000000000002e-05 2024-09-10T18:59:47.586 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.586 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.586 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:59:47.586 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:53.703184+0000", 2024-09-10T18:59:47.586 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.7230000000000004e-06 2024-09-10T18:59:47.586 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.586 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.586 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.586 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:54.577248+0000", 2024-09-10T18:59:47.586 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.87406382299999996 2024-09-10T18:59:47.587 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.587 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.587 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:59:47.587 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:54.577257+0000", 2024-09-10T18:59:47.587 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.6649999999999992e-06 2024-09-10T18:59:47.587 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.587 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.587 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.587 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.682900+0000", 2024-09-10T18:59:47.587 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.105643361 2024-09-10T18:59:47.587 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.587 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.587 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:59:47.588 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.683022+0000", 2024-09-10T18:59:47.588 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00012169 2024-09-10T18:59:47.588 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.588 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.588 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,3,5", 2024-09-10T18:59:47.588 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.683166+0000", 2024-09-10T18:59:47.588 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000144189 2024-09-10T18:59:47.588 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.588 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.588 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.588 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.687288+0000", 2024-09-10T18:59:47.588 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0041215690000000003 2024-09-10T18:59:47.588 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.589 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.589 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:59:47.589 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.695411+0000", 2024-09-10T18:59:47.589 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0081232549999999994 2024-09-10T18:59:47.589 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.589 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.589 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.589 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.695694+0000", 2024-09-10T18:59:47.589 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000282959 2024-09-10T18:59:47.589 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.589 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.589 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.590 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.695730+0000", 2024-09-10T18:59:47.590 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.6159999999999999e-05 2024-09-10T18:59:47.590 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.590 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.590 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:59:47.590 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.696503+0000", 2024-09-10T18:59:47.590 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00077256700000000005 2024-09-10T18:59:47.590 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.590 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.590 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:59:47.590 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:52:55.696622+0000", 2024-09-10T18:59:47.590 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000119892 2024-09-10T18:59:47.590 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.590 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:59:47.591 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.591 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.591 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.591 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:194910 8.6 8:673b0092:::benchmark_data_smithi018_106531_object48280:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e206)", 2024-09-10T18:59:47.591 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:54:00.294785+0000", 2024-09-10T18:59:47.591 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 347.19621006800003, 2024-09-10T18:59:47.591 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.038020379, 2024-09-10T18:59:47.591 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:59:47.591 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "reached pg", 2024-09-10T18:59:47.591 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:59:47.591 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:59:47.591 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:59:47.592 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 194910 2024-09-10T18:59:47.592 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.592 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:59:47.592 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.592 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:59:47.592 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294785+0000", 2024-09-10T18:59:47.592 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.592 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.592 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.592 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:59:47.592 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294785+0000", 2024-09-10T18:59:47.592 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.593 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.593 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.593 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:59:47.593 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294784+0000", 2024-09-10T18:59:47.593 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:59:47.593 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.593 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.593 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:59:47.593 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294791+0000", 2024-09-10T18:59:47.593 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.37e-06 2024-09-10T18:59:47.593 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.593 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.593 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:59:47.594 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294800+0000", 2024-09-10T18:59:47.594 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.4690000000000003e-06 2024-09-10T18:59:47.594 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.594 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.594 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:59:47.594 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294806+0000", 2024-09-10T18:59:47.594 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.0430000000000004e-06 2024-09-10T18:59:47.594 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.594 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.594 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.594 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294976+0000", 2024-09-10T18:59:47.594 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00016977499999999999 2024-09-10T18:59:47.594 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.595 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.595 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:59:47.595 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294985+0000", 2024-09-10T18:59:47.595 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.8899999999999996e-06 2024-09-10T18:59:47.595 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.595 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.595 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.595 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.332737+0000", 2024-09-10T18:59:47.595 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0377524929999999 2024-09-10T18:59:47.595 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.595 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.595 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:59:47.595 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.332806+0000", 2024-09-10T18:59:47.595 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.8517999999999997e-05 2024-09-10T18:59:47.596 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.596 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:59:47.596 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.596 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.596 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.596 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:194916 8.6 8:6649d7fa:::benchmark_data_smithi018_106531_object48286:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e206)", 2024-09-10T18:59:47.596 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:54:00.294883+0000", 2024-09-10T18:59:47.596 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 347.19611204900002, 2024-09-10T18:59:47.596 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0521695499999999, 2024-09-10T18:59:47.596 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:59:47.596 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "reached pg", 2024-09-10T18:59:47.596 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:59:47.596 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:59:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:59:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 194916 2024-09-10T18:59:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:59:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:59:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294883+0000", 2024-09-10T18:59:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:59:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294883+0000", 2024-09-10T18:59:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:59:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294883+0000", 2024-09-10T18:59:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999995 2024-09-10T18:59:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:59:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294887+0000", 2024-09-10T18:59:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.143e-06 2024-09-10T18:59:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:59:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294896+0000", 2024-09-10T18:59:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.2799999999999992e-06 2024-09-10T18:59:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:59:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294902+0000", 2024-09-10T18:59:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.429e-06 2024-09-10T18:59:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.295026+0000", 2024-09-10T18:59:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00012407799999999999 2024-09-10T18:59:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:59:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.295031+0000", 2024-09-10T18:59:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.8679999999999998e-06 2024-09-10T18:59:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.332977+0000", 2024-09-10T18:59:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.037946563 2024-09-10T18:59:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for missing object", 2024-09-10T18:59:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.333062+0000", 2024-09-10T18:59:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.5266999999999995e-05 2024-09-10T18:59:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.346974+0000", 2024-09-10T18:59:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.013911468 2024-09-10T18:59:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:59:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.347053+0000", 2024-09-10T18:59:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.9085000000000006e-05 2024-09-10T18:59:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:59:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:194933 8.6 8:6cc0138d:::benchmark_data_smithi018_106531_object48303:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e206)", 2024-09-10T18:59:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:54:00.309381+0000", 2024-09-10T18:59:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 347.18161430800001, 2024-09-10T18:59:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0420561989999999, 2024-09-10T18:59:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:59:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:59:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:59:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:59:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:59:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 194933 2024-09-10T18:59:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:59:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:59:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.309381+0000", 2024-09-10T18:59:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:59:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.309381+0000", 2024-09-10T18:59:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:59:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.309380+0000", 2024-09-10T18:59:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:59:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:59:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.309386+0000", 2024-09-10T18:59:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.0750000000000003e-06 2024-09-10T18:59:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:59:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.309398+0000", 2024-09-10T18:59:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.2291e-05 2024-09-10T18:59:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:59:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.309403+0000", 2024-09-10T18:59:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.198e-06 2024-09-10T18:59:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.313851+0000", 2024-09-10T18:59:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0044473760000000003 2024-09-10T18:59:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:59:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.313858+0000", 2024-09-10T18:59:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.525e-06 2024-09-10T18:59:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.333087+0000", 2024-09-10T18:59:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0192285029999999 2024-09-10T18:59:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:59:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.333185+0000", 2024-09-10T18:59:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.8448000000000003e-05 2024-09-10T18:59:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 7", 2024-09-10T18:59:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.333548+0000", 2024-09-10T18:59:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00036276100000000002 2024-09-10T18:59:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:59:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.344236+0000", 2024-09-10T18:59:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.010687497000000001 2024-09-10T18:59:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.351170+0000", 2024-09-10T18:59:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0069349119999999997 2024-09-10T18:59:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:59:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.351337+0000", 2024-09-10T18:59:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00016637600000000001 2024-09-10T18:59:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:59:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.351437+0000", 2024-09-10T18:59:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00010045200000000001 2024-09-10T18:59:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:59:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:194942 8.6 8:6dedfc51:::benchmark_data_smithi018_106531_object48312:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e206)", 2024-09-10T18:59:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:54:00.317038+0000", 2024-09-10T18:59:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 347.17395760199997, 2024-09-10T18:59:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0348635230000001, 2024-09-10T18:59:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:59:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:59:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:59:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:59:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:59:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 194942 2024-09-10T18:59:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:59:47.610 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.610 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:59:47.610 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.317038+0000", 2024-09-10T18:59:47.610 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.610 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.610 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.610 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:59:47.610 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.317038+0000", 2024-09-10T18:59:47.610 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.610 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.610 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.611 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:59:47.611 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.317036+0000", 2024-09-10T18:59:47.611 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:59:47.611 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.611 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.611 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:59:47.611 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.317042+0000", 2024-09-10T18:59:47.611 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.682e-06 2024-09-10T18:59:47.611 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.611 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.611 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:59:47.611 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.317055+0000", 2024-09-10T18:59:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.2948e-05 2024-09-10T18:59:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:59:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.317062+0000", 2024-09-10T18:59:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.9299999999999997e-06 2024-09-10T18:59:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.317132+0000", 2024-09-10T18:59:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.0005000000000005e-05 2024-09-10T18:59:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:59:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.317140+0000", 2024-09-10T18:59:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.1850000000000004e-06 2024-09-10T18:59:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.339169+0000", 2024-09-10T18:59:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0220292820000001 2024-09-10T18:59:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:59:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.339295+0000", 2024-09-10T18:59:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000125339 2024-09-10T18:59:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 7", 2024-09-10T18:59:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.339424+0000", 2024-09-10T18:59:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000128775 2024-09-10T18:59:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:59:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.346717+0000", 2024-09-10T18:59:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0072938730000000002 2024-09-10T18:59:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.351712+0000", 2024-09-10T18:59:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0049943510000000002 2024-09-10T18:59:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:59:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.351824+0000", 2024-09-10T18:59:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000112222 2024-09-10T18:59:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:59:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.351901+0000", 2024-09-10T18:59:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.7371000000000002e-05 2024-09-10T18:59:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:59:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:194943 8.9 8:982e3ef1:::benchmark_data_smithi018_106531_object48313:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e206)", 2024-09-10T18:59:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:54:00.317188+0000", 2024-09-10T18:59:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 347.17380769699997, 2024-09-10T18:59:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.017647009, 2024-09-10T18:59:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:59:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:59:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:59:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:59:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:59:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 194943 2024-09-10T18:59:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:59:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:59:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.317188+0000", 2024-09-10T18:59:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:59:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.317188+0000", 2024-09-10T18:59:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:59:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.317187+0000", 2024-09-10T18:59:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:59:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:59:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.317192+0000", 2024-09-10T18:59:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.7600000000000002e-06 2024-09-10T18:59:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:59:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.317202+0000", 2024-09-10T18:59:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0264e-05 2024-09-10T18:59:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:59:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.317207+0000", 2024-09-10T18:59:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.3299999999999998e-06 2024-09-10T18:59:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.317262+0000", 2024-09-10T18:59:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.4280000000000002e-05 2024-09-10T18:59:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:59:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.317269+0000", 2024-09-10T18:59:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.4819999999999997e-06 2024-09-10T18:59:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.326257+0000", 2024-09-10T18:59:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0089881220000001 2024-09-10T18:59:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.621 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:59:47.621 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.326386+0000", 2024-09-10T18:59:47.621 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00012864800000000001 2024-09-10T18:59:47.621 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.621 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.621 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,5", 2024-09-10T18:59:47.621 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.326525+0000", 2024-09-10T18:59:47.621 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00013949500000000001 2024-09-10T18:59:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.331405+0000", 2024-09-10T18:59:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.004879492 2024-09-10T18:59:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.331554+0000", 2024-09-10T18:59:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00014887599999999999 2024-09-10T18:59:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:59:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.334694+0000", 2024-09-10T18:59:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0031405169999999998 2024-09-10T18:59:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:59:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.334736+0000", 2024-09-10T18:59:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.1347000000000002e-05 2024-09-10T18:59:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:59:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.334835+0000", 2024-09-10T18:59:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.9216999999999996e-05 2024-09-10T18:59:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:59:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:194948 8.6 8:60730bfc:::benchmark_data_smithi018_106531_object48318:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e206)", 2024-09-10T18:59:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:54:00.320523+0000", 2024-09-10T18:59:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 347.17047213699999, 2024-09-10T18:59:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0338989240000001, 2024-09-10T18:59:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:59:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:59:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:59:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:59:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:59:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 194948 2024-09-10T18:59:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:59:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:59:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320523+0000", 2024-09-10T18:59:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:59:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320523+0000", 2024-09-10T18:59:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:59:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320521+0000", 2024-09-10T18:59:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999981 2024-09-10T18:59:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:59:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320528+0000", 2024-09-10T18:59:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.4790000000000001e-06 2024-09-10T18:59:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:59:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320541+0000", 2024-09-10T18:59:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.2706e-05 2024-09-10T18:59:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:59:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320547+0000", 2024-09-10T18:59:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.9900000000000002e-06 2024-09-10T18:59:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.628 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.628 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320582+0000", 2024-09-10T18:59:47.628 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.4981999999999998e-05 2024-09-10T18:59:47.628 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.628 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.628 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:59:47.628 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320588+0000", 2024-09-10T18:59:47.628 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.1569999999999997e-06 2024-09-10T18:59:47.628 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.628 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.628 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.344263+0000", 2024-09-10T18:59:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0236751559999999 2024-09-10T18:59:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:59:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.344357+0000", 2024-09-10T18:59:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.3874000000000003e-05 2024-09-10T18:59:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 7", 2024-09-10T18:59:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.344462+0000", 2024-09-10T18:59:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000105306 2024-09-10T18:59:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:59:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.347052+0000", 2024-09-10T18:59:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0025898800000000001 2024-09-10T18:59:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.354312+0000", 2024-09-10T18:59:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.007260468 2024-09-10T18:59:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:59:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.354351+0000", 2024-09-10T18:59:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.8572999999999998e-05 2024-09-10T18:59:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:59:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.354422+0000", 2024-09-10T18:59:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.1250999999999999e-05 2024-09-10T18:59:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:59:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.632 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.632 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.632 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:194950 8.9 8:905debb1:::benchmark_data_smithi018_106531_object48320:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e206)", 2024-09-10T18:59:47.632 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:54:00.321242+0000", 2024-09-10T18:59:47.632 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 347.16975382499999, 2024-09-10T18:59:47.632 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0142816670000001, 2024-09-10T18:59:47.632 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:59:47.632 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:59:47.632 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:59:47.632 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:59:47.632 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:59:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 194950 2024-09-10T18:59:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:59:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:59:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.321242+0000", 2024-09-10T18:59:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:59:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.321242+0000", 2024-09-10T18:59:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.634 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.634 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.634 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:59:47.634 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.321241+0000", 2024-09-10T18:59:47.634 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999995 2024-09-10T18:59:47.634 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.634 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.634 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:59:47.634 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.321245+0000", 2024-09-10T18:59:47.634 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.1930000000000003e-06 2024-09-10T18:59:47.634 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.634 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.635 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:59:47.635 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.321253+0000", 2024-09-10T18:59:47.635 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.1680000000000004e-06 2024-09-10T18:59:47.635 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.635 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.635 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:59:47.635 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.321257+0000", 2024-09-10T18:59:47.635 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.4860000000000002e-06 2024-09-10T18:59:47.635 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.635 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.635 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.636 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.321303+0000", 2024-09-10T18:59:47.636 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.6282999999999997e-05 2024-09-10T18:59:47.636 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.636 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.636 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:59:47.636 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.321311+0000", 2024-09-10T18:59:47.636 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.2709999999999992e-06 2024-09-10T18:59:47.636 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.636 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.636 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.636 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.328706+0000", 2024-09-10T18:59:47.636 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.007394463 2024-09-10T18:59:47.637 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.637 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.637 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:59:47.637 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.328835+0000", 2024-09-10T18:59:47.637 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000129636 2024-09-10T18:59:47.637 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.637 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.637 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,5", 2024-09-10T18:59:47.637 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.328971+0000", 2024-09-10T18:59:47.637 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000135647 2024-09-10T18:59:47.637 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.638 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.638 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.638 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.334679+0000", 2024-09-10T18:59:47.638 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0057075310000000001 2024-09-10T18:59:47.638 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.638 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.638 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:59:47.638 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.334843+0000", 2024-09-10T18:59:47.638 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00016409700000000001 2024-09-10T18:59:47.638 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.638 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.638 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.638 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.335374+0000", 2024-09-10T18:59:47.639 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00053144499999999996 2024-09-10T18:59:47.639 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.639 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.639 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:59:47.639 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.335423+0000", 2024-09-10T18:59:47.639 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.8809e-05 2024-09-10T18:59:47.639 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.639 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.639 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:59:47.639 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.335523+0000", 2024-09-10T18:59:47.639 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00010024 2024-09-10T18:59:47.639 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.640 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:59:47.640 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.640 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.640 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.640 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:194951 8.6 8:6b51c5ec:::benchmark_data_smithi018_106531_object48321:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e206)", 2024-09-10T18:59:47.640 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:54:00.321286+0000", 2024-09-10T18:59:47.640 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 347.16970987500002, 2024-09-10T18:59:47.640 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.038199619, 2024-09-10T18:59:47.640 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:59:47.640 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:59:47.640 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:59:47.641 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:59:47.641 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:59:47.641 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 194951 2024-09-10T18:59:47.641 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.641 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:59:47.641 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.641 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:59:47.641 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.321286+0000", 2024-09-10T18:59:47.641 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.641 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.641 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.641 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:59:47.642 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.321286+0000", 2024-09-10T18:59:47.642 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.642 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.642 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.642 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:59:47.642 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.321285+0000", 2024-09-10T18:59:47.642 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999995 2024-09-10T18:59:47.642 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.642 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.642 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:59:47.642 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.321289+0000", 2024-09-10T18:59:47.643 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.4349999999999999e-06 2024-09-10T18:59:47.643 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.643 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.643 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:59:47.643 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.321297+0000", 2024-09-10T18:59:47.643 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.6939999999999995e-06 2024-09-10T18:59:47.643 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.643 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.643 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:59:47.643 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.321302+0000", 2024-09-10T18:59:47.643 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.6809999999999997e-06 2024-09-10T18:59:47.643 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.644 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.644 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.644 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.321345+0000", 2024-09-10T18:59:47.644 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.2984e-05 2024-09-10T18:59:47.644 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.644 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.644 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:59:47.644 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.321351+0000", 2024-09-10T18:59:47.644 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.3690000000000003e-06 2024-09-10T18:59:47.644 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.644 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.645 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.645 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.347098+0000", 2024-09-10T18:59:47.645 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.025746928 2024-09-10T18:59:47.645 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.645 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.645 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:59:47.645 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.347212+0000", 2024-09-10T18:59:47.645 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000114428 2024-09-10T18:59:47.645 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.645 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.645 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 7", 2024-09-10T18:59:47.645 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.347295+0000", 2024-09-10T18:59:47.646 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.2806999999999995e-05 2024-09-10T18:59:47.646 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.646 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.646 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.646 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.354456+0000", 2024-09-10T18:59:47.646 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0071609059999999999 2024-09-10T18:59:47.646 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.646 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.646 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:59:47.646 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.359376+0000", 2024-09-10T18:59:47.646 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0049194349999999998 2024-09-10T18:59:47.646 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.646 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.647 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:59:47.647 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.359415+0000", 2024-09-10T18:59:47.647 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.9453000000000003e-05 2024-09-10T18:59:47.647 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.647 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.647 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:59:47.647 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.359485+0000", 2024-09-10T18:59:47.647 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.0178000000000005e-05 2024-09-10T18:59:47.647 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.647 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:59:47.647 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.647 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.647 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.648 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:248413 8.9 8:9f1dcbb5:::benchmark_data_smithi018_106531_object101783:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e218)", 2024-09-10T18:59:47.648 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:54:39.652660+0000", 2024-09-10T18:59:47.648 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 307.83833562900003, 2024-09-10T18:59:47.648 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.002997237, 2024-09-10T18:59:47.648 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:59:47.648 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:59:47.648 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:59:47.648 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:59:47.648 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:59:47.648 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 248413 2024-09-10T18:59:47.648 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.648 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:59:47.649 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.649 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:59:47.649 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.652660+0000", 2024-09-10T18:59:47.649 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.649 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.649 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.649 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:59:47.649 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.652660+0000", 2024-09-10T18:59:47.649 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.649 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.649 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.649 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:59:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.652659+0000", 2024-09-10T18:59:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:59:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:59:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.652663+0000", 2024-09-10T18:59:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.9870000000000001e-06 2024-09-10T18:59:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:59:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.652670+0000", 2024-09-10T18:59:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.4440000000000002e-06 2024-09-10T18:59:47.650 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.651 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.651 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:59:47.651 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.652675+0000", 2024-09-10T18:59:47.651 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.2229999999999998e-06 2024-09-10T18:59:47.651 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.651 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.651 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.651 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.654639+0000", 2024-09-10T18:59:47.651 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00196356 2024-09-10T18:59:47.651 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.651 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.651 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:59:47.651 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.654647+0000", 2024-09-10T18:59:47.652 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.0150000000000003e-06 2024-09-10T18:59:47.652 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.652 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.652 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.652 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.633532+0000", 2024-09-10T18:59:47.652 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.97888465999999996 2024-09-10T18:59:47.652 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.652 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.652 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:59:47.652 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.633626+0000", 2024-09-10T18:59:47.652 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.4530999999999993e-05 2024-09-10T18:59:47.652 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.653 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.653 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,5", 2024-09-10T18:59:47.653 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.633707+0000", 2024-09-10T18:59:47.653 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.1034999999999999e-05 2024-09-10T18:59:47.653 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.653 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.653 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:59:47.653 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.639898+0000", 2024-09-10T18:59:47.653 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0061904899999999999 2024-09-10T18:59:47.653 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.653 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.654 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.654 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.655454+0000", 2024-09-10T18:59:47.654 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.015556367 2024-09-10T18:59:47.654 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.654 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.654 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.654 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.655517+0000", 2024-09-10T18:59:47.654 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.2558000000000007e-05 2024-09-10T18:59:47.654 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.654 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.654 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:59:47.654 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.655569+0000", 2024-09-10T18:59:47.655 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.2457999999999998e-05 2024-09-10T18:59:47.655 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.655 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.655 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:59:47.655 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.655657+0000", 2024-09-10T18:59:47.655 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.8018999999999999e-05 2024-09-10T18:59:47.655 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.655 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:59:47.655 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.655 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.655 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.655 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:248422 8.9 8:90ba02b9:::benchmark_data_smithi018_106531_object101792:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e218)", 2024-09-10T18:59:47.655 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:54:39.658256+0000", 2024-09-10T18:59:47.656 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 307.83273973000001, 2024-09-10T18:59:47.656 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.99755751800000003, 2024-09-10T18:59:47.656 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:59:47.656 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:59:47.656 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:59:47.656 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:59:47.656 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:59:47.656 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 248422 2024-09-10T18:59:47.656 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.656 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:59:47.656 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.656 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:59:47.657 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.658256+0000", 2024-09-10T18:59:47.657 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.657 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.657 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.657 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:59:47.657 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.658256+0000", 2024-09-10T18:59:47.657 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.657 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.657 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.657 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:59:47.657 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.658255+0000", 2024-09-10T18:59:47.658 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999995 2024-09-10T18:59:47.658 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.658 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.658 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:59:47.658 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.658259+0000", 2024-09-10T18:59:47.658 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.143e-06 2024-09-10T18:59:47.658 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.658 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.658 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:59:47.658 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.658267+0000", 2024-09-10T18:59:47.658 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.3480000000000004e-06 2024-09-10T18:59:47.659 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.659 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.659 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:59:47.659 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.658273+0000", 2024-09-10T18:59:47.659 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.1769999999999997e-06 2024-09-10T18:59:47.659 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.659 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.659 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.659 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.658316+0000", 2024-09-10T18:59:47.659 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.337e-05 2024-09-10T18:59:47.659 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.660 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.660 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:59:47.660 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.658324+0000", 2024-09-10T18:59:47.660 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.6040000000000004e-06 2024-09-10T18:59:47.660 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.660 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.660 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.660 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.635467+0000", 2024-09-10T18:59:47.660 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.97714298499999996 2024-09-10T18:59:47.660 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.660 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.661 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:59:47.661 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.635572+0000", 2024-09-10T18:59:47.661 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00010490899999999999 2024-09-10T18:59:47.661 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.661 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.661 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,5", 2024-09-10T18:59:47.661 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.635645+0000", 2024-09-10T18:59:47.661 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.3101000000000004e-05 2024-09-10T18:59:47.661 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.661 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.661 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:59:47.662 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.642303+0000", 2024-09-10T18:59:47.662 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0066582480000000003 2024-09-10T18:59:47.662 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.662 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.662 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.662 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.655702+0000", 2024-09-10T18:59:47.662 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.013399345999999999 2024-09-10T18:59:47.662 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.662 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.662 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.662 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.655738+0000", 2024-09-10T18:59:47.663 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.6174999999999998e-05 2024-09-10T18:59:47.663 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.663 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.663 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:59:47.663 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.655765+0000", 2024-09-10T18:59:47.663 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.6763000000000001e-05 2024-09-10T18:59:47.663 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.663 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.663 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:59:47.663 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.655813+0000", 2024-09-10T18:59:47.663 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.8047999999999998e-05 2024-09-10T18:59:47.664 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.664 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:59:47.664 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.664 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.664 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.664 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:248431 8.9 8:9ec8126b:::benchmark_data_smithi018_106531_object101801:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e218)", 2024-09-10T18:59:47.664 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:54:39.659488+0000", 2024-09-10T18:59:47.664 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 307.83150760000001, 2024-09-10T18:59:47.664 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.99650889399999998, 2024-09-10T18:59:47.664 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:59:47.664 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:59:47.664 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:59:47.665 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:59:47.665 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:59:47.665 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 248431 2024-09-10T18:59:47.665 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.665 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:59:47.665 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.665 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:59:47.665 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.659488+0000", 2024-09-10T18:59:47.665 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.665 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.665 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.666 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:59:47.666 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.659488+0000", 2024-09-10T18:59:47.666 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.666 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.666 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.666 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:59:47.666 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.659487+0000", 2024-09-10T18:59:47.666 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:59:47.666 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.666 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.667 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:59:47.667 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.659492+0000", 2024-09-10T18:59:47.668 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.8879999999999996e-06 2024-09-10T18:59:47.668 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.668 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.668 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:59:47.668 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.659504+0000", 2024-09-10T18:59:47.668 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.1799e-05 2024-09-10T18:59:47.669 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.669 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.669 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:59:47.669 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.659509+0000", 2024-09-10T18:59:47.669 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.3510000000000001e-06 2024-09-10T18:59:47.669 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.669 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.669 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.669 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.659552+0000", 2024-09-10T18:59:47.669 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.3344999999999999e-05 2024-09-10T18:59:47.670 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.670 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.670 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:59:47.670 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.659560+0000", 2024-09-10T18:59:47.670 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.0739999999999993e-06 2024-09-10T18:59:47.670 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.670 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.670 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.670 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.636881+0000", 2024-09-10T18:59:47.671 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.97732015900000002 2024-09-10T18:59:47.671 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.671 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.671 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:59:47.671 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.636955+0000", 2024-09-10T18:59:47.671 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.4685999999999996e-05 2024-09-10T18:59:47.671 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.671 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.671 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,5", 2024-09-10T18:59:47.671 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.637062+0000", 2024-09-10T18:59:47.671 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00010655800000000001 2024-09-10T18:59:47.671 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.672 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.672 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:59:47.672 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.642308+0000", 2024-09-10T18:59:47.672 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0052465350000000001 2024-09-10T18:59:47.672 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.672 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.672 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.672 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.655853+0000", 2024-09-10T18:59:47.672 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.013544700999999999 2024-09-10T18:59:47.672 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.672 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.673 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.673 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.655929+0000", 2024-09-10T18:59:47.673 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.5631999999999997e-05 2024-09-10T18:59:47.673 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.673 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.673 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:59:47.673 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.655951+0000", 2024-09-10T18:59:47.673 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.2388e-05 2024-09-10T18:59:47.673 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.673 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.673 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:59:47.674 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.655997+0000", 2024-09-10T18:59:47.674 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.5627999999999997e-05 2024-09-10T18:59:47.674 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.674 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:59:47.674 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.674 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.674 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.674 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:248432 8.9 8:9e11dac5:::benchmark_data_smithi018_106531_object101802:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e218)", 2024-09-10T18:59:47.674 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:54:39.661415+0000", 2024-09-10T18:59:47.674 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 307.82958038200002, 2024-09-10T18:59:47.674 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.99472695200000005, 2024-09-10T18:59:47.675 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:59:47.675 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:59:47.675 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:59:47.675 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:59:47.675 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:59:47.675 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 248432 2024-09-10T18:59:47.675 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.675 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:59:47.675 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.675 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:59:47.675 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.661415+0000", 2024-09-10T18:59:47.675 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.676 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.676 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.676 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:59:47.676 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.661415+0000", 2024-09-10T18:59:47.676 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.676 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.676 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.676 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:59:47.676 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.661414+0000", 2024-09-10T18:59:47.676 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T18:59:47.676 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.677 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.677 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:59:47.677 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.661420+0000", 2024-09-10T18:59:47.677 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.9540000000000001e-06 2024-09-10T18:59:47.677 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.677 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.677 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:59:47.677 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.661431+0000", 2024-09-10T18:59:47.677 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0813e-05 2024-09-10T18:59:47.677 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.677 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.677 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:59:47.678 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.661436+0000", 2024-09-10T18:59:47.678 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.7300000000000002e-06 2024-09-10T18:59:47.678 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.678 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.678 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.678 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.661592+0000", 2024-09-10T18:59:47.678 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00015532499999999999 2024-09-10T18:59:47.678 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.678 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.678 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:59:47.679 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.661599+0000", 2024-09-10T18:59:47.679 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.5530000000000004e-06 2024-09-10T18:59:47.679 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.679 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.679 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.679 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.638526+0000", 2024-09-10T18:59:47.679 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.97692656499999997 2024-09-10T18:59:47.679 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.679 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.679 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:59:47.679 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.638636+0000", 2024-09-10T18:59:47.680 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000109801 2024-09-10T18:59:47.680 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.680 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.680 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,5", 2024-09-10T18:59:47.680 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.638765+0000", 2024-09-10T18:59:47.680 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000129049 2024-09-10T18:59:47.680 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.680 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.680 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:59:47.680 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.644591+0000", 2024-09-10T18:59:47.680 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0058265979999999997 2024-09-10T18:59:47.680 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.681 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.681 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.681 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.656041+0000", 2024-09-10T18:59:47.681 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.011449456 2024-09-10T18:59:47.681 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.681 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.681 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.681 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.656074+0000", 2024-09-10T18:59:47.681 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.2951999999999999e-05 2024-09-10T18:59:47.681 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.681 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.682 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:59:47.682 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.656097+0000", 2024-09-10T18:59:47.682 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.3665000000000001e-05 2024-09-10T18:59:47.682 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.682 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.682 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:59:47.682 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.656142+0000", 2024-09-10T18:59:47.682 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.4719000000000003e-05 2024-09-10T18:59:47.682 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.682 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:59:47.682 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.682 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.683 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.683 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:248455 8.9 8:94535dab:::benchmark_data_smithi018_106531_object101825:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e218)", 2024-09-10T18:59:47.683 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:54:39.675441+0000", 2024-09-10T18:59:47.683 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 307.815554556, 2024-09-10T18:59:47.683 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.98094657600000001, 2024-09-10T18:59:47.683 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T18:59:47.683 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T18:59:47.683 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T18:59:47.683 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T18:59:47.683 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T18:59:47.683 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 248455 2024-09-10T18:59:47.684 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.684 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T18:59:47.684 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.684 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T18:59:47.684 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.675441+0000", 2024-09-10T18:59:47.684 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.684 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.684 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.684 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T18:59:47.684 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.675441+0000", 2024-09-10T18:59:47.684 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T18:59:47.684 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.685 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.685 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T18:59:47.685 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.675440+0000", 2024-09-10T18:59:47.685 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T18:59:47.685 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.685 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.685 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T18:59:47.685 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.675445+0000", 2024-09-10T18:59:47.685 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.3279999999999996e-06 2024-09-10T18:59:47.685 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.685 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.686 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T18:59:47.686 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.675454+0000", 2024-09-10T18:59:47.686 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.5879999999999998e-06 2024-09-10T18:59:47.686 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.686 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.686 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T18:59:47.686 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.675459+0000", 2024-09-10T18:59:47.686 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.13e-06 2024-09-10T18:59:47.686 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.686 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.686 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.687 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.675503+0000", 2024-09-10T18:59:47.687 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.4489e-05 2024-09-10T18:59:47.687 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.687 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.687 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T18:59:47.687 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:39.675510+0000", 2024-09-10T18:59:47.687 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.0169999999999999e-06 2024-09-10T18:59:47.687 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.687 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.687 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T18:59:47.687 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.639936+0000", 2024-09-10T18:59:47.688 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.96442608900000004 2024-09-10T18:59:47.688 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.688 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.688 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T18:59:47.688 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.640018+0000", 2024-09-10T18:59:47.688 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.1944e-05 2024-09-10T18:59:47.688 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.688 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.688 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,5", 2024-09-10T18:59:47.688 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.640089+0000", 2024-09-10T18:59:47.688 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.038e-05 2024-09-10T18:59:47.688 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.689 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.689 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T18:59:47.689 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.644594+0000", 2024-09-10T18:59:47.689 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0045056769999999996 2024-09-10T18:59:47.689 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.689 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.689 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.689 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.656185+0000", 2024-09-10T18:59:47.689 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.011590237 2024-09-10T18:59:47.689 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.689 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.689 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T18:59:47.690 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.656238+0000", 2024-09-10T18:59:47.690 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.3189999999999997e-05 2024-09-10T18:59:47.690 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.690 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.690 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T18:59:47.690 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.656286+0000", 2024-09-10T18:59:47.690 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.8539999999999999e-05 2024-09-10T18:59:47.690 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T18:59:47.690 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T18:59:47.690 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T18:59:47.690 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:40.656387+0000", 2024-09-10T18:59:47.691 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00010098 2024-09-10T18:59:47.691 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.691 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:59:47.691 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.691 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T18:59:47.691 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T18:59:47.691 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T18:59:47.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:47.693 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:47.693 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:47.742 INFO:teuthology.orchestra.run.smithi018.stderr:reweighted osd.0 to 1 (10000) 2024-09-10T18:59:47.794 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:47.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:47.816 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:47.816 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:47.917 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:47.938 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:47.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:47.940 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:47.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:47 smithi018 ceph-mon[29336]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:59:47.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:47 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:59:47.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:47 smithi018 ceph-mon[29336]: osdmap e319: 8 total, 8 up, 6 in 2024-09-10T18:59:47.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:47 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2228623598' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-10T18:59:47.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:47 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-10T18:59:47.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:47 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:59:47.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:47 smithi018 ceph-mon[29336]: pgmap v668: 168 pgs: 168 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:59:47.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:47 smithi018 ceph-mon[40196]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:59:47.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:47 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:59:47.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:47 smithi018 ceph-mon[40196]: osdmap e319: 8 total, 8 up, 6 in 2024-09-10T18:59:47.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:47 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2228623598' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-10T18:59:47.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:47 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-10T18:59:47.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:47 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:59:47.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:47 smithi018 ceph-mon[40196]: pgmap v668: 168 pgs: 168 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:59:48.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:47 smithi103 ceph-mon[33163]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:59:48.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:47 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T18:59:48.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:47 smithi103 ceph-mon[33163]: osdmap e319: 8 total, 8 up, 6 in 2024-09-10T18:59:48.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:47 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2228623598' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-10T18:59:48.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:47 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-10T18:59:48.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:47 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:59:48.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:47 smithi103 ceph-mon[33163]: pgmap v668: 168 pgs: 168 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:59:48.041 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:48.062 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:48.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:48.063 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:48.164 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:48.191 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:48.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:48.192 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:48.293 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:48.320 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:48.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:48.322 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:48.412 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-10T18:59:48.422 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:48.448 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:48.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:48.450 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:48.551 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:48.575 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:48.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:48.576 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:48.661 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:59:48.677 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:48.699 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:48.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:48.701 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:48.802 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:48.826 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:48.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:48.827 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:48.928 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:48.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:48.950 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:48.951 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:49.007 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:48 smithi103 ceph-mon[33163]: 6.10 scrub starts 2024-09-10T18:59:49.007 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:48 smithi103 ceph-mon[33163]: 6.10 scrub ok 2024-09-10T18:59:49.007 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:48 smithi103 ceph-mon[33163]: 4.d scrub starts 2024-09-10T18:59:49.007 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:48 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-10T18:59:49.007 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:48 smithi103 ceph-mon[33163]: osdmap e320: 8 total, 8 up, 6 in 2024-09-10T18:59:49.007 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:48 smithi103 ceph-mon[33163]: 4.d scrub ok 2024-09-10T18:59:49.007 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:48 smithi103 ceph-mon[33163]: 6.19 scrub starts 2024-09-10T18:59:49.008 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:48 smithi103 ceph-mon[33163]: 6.19 scrub ok 2024-09-10T18:59:49.008 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:48 smithi018 ceph-mon[40196]: 6.10 scrub starts 2024-09-10T18:59:49.008 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:48 smithi018 ceph-mon[40196]: 6.10 scrub ok 2024-09-10T18:59:49.008 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:48 smithi018 ceph-mon[40196]: 4.d scrub starts 2024-09-10T18:59:49.008 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:48 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-10T18:59:49.008 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:48 smithi018 ceph-mon[40196]: osdmap e320: 8 total, 8 up, 6 in 2024-09-10T18:59:49.008 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:48 smithi018 ceph-mon[40196]: 4.d scrub ok 2024-09-10T18:59:49.009 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:48 smithi018 ceph-mon[40196]: 6.19 scrub starts 2024-09-10T18:59:49.009 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:48 smithi018 ceph-mon[40196]: 6.19 scrub ok 2024-09-10T18:59:49.009 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:48 smithi018 ceph-mon[29336]: 6.10 scrub starts 2024-09-10T18:59:49.009 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:48 smithi018 ceph-mon[29336]: 6.10 scrub ok 2024-09-10T18:59:49.009 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:48 smithi018 ceph-mon[29336]: 4.d scrub starts 2024-09-10T18:59:49.009 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:48 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-10T18:59:49.009 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:48 smithi018 ceph-mon[29336]: osdmap e320: 8 total, 8 up, 6 in 2024-09-10T18:59:49.009 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:48 smithi018 ceph-mon[29336]: 4.d scrub ok 2024-09-10T18:59:49.009 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:48 smithi018 ceph-mon[29336]: 6.19 scrub starts 2024-09-10T18:59:49.009 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:48 smithi018 ceph-mon[29336]: 6.19 scrub ok 2024-09-10T18:59:49.052 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:49.075 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:49.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:49.076 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:49.176 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:49.199 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:49.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:49.200 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:49.301 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:49.322 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:49.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:49.323 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:49.424 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:49.445 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:49.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:49.446 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:49.547 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:49.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:49.569 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:49.569 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:49.670 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:49.693 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-10T18:59:49.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:49.695 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:49.695 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:49.796 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:49.819 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:49.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:49.820 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:49.861 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:49 smithi018 ceph-mon[40196]: 2.14 scrub starts 2024-09-10T18:59:49.862 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:49 smithi018 ceph-mon[40196]: 2.14 scrub ok 2024-09-10T18:59:49.862 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:49 smithi018 ceph-mon[40196]: osdmap e321: 8 total, 8 up, 6 in 2024-09-10T18:59:49.862 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:49 smithi018 ceph-mon[40196]: pgmap v671: 168 pgs: 3 peering, 165 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail 2024-09-10T18:59:49.862 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:49 smithi018 ceph-mon[29336]: 2.14 scrub starts 2024-09-10T18:59:49.862 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:49 smithi018 ceph-mon[29336]: 2.14 scrub ok 2024-09-10T18:59:49.862 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:49 smithi018 ceph-mon[29336]: osdmap e321: 8 total, 8 up, 6 in 2024-09-10T18:59:49.862 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:49 smithi018 ceph-mon[29336]: pgmap v671: 168 pgs: 3 peering, 165 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail 2024-09-10T18:59:49.921 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:49.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:49.945 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:49.945 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:49.948 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:49 smithi103 ceph-mon[33163]: 2.14 scrub starts 2024-09-10T18:59:49.948 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:49 smithi103 ceph-mon[33163]: 2.14 scrub ok 2024-09-10T18:59:49.948 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:49 smithi103 ceph-mon[33163]: osdmap e321: 8 total, 8 up, 6 in 2024-09-10T18:59:49.948 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:49 smithi103 ceph-mon[33163]: pgmap v671: 168 pgs: 3 peering, 165 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail 2024-09-10T18:59:50.046 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:50.068 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:50.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:50.069 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:50.170 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:50.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:50.193 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:50.193 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:50.199 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T18:59:50.199 INFO:teuthology.orchestra.run.smithi103.stdout: "ops": [], 2024-09-10T18:59:50.199 INFO:teuthology.orchestra.run.smithi103.stdout: "num_ops": 0 2024-09-10T18:59:50.199 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T18:59:50.294 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:50.316 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:50.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:50.317 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:50.391 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd reweight 1 1 2024-09-10T18:59:50.417 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:50.439 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:50.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:50.440 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:50.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 18:59:50 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:59:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:59:50.540 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:50.562 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:50.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:50.563 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:50.664 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:50.685 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:50.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:50.686 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:50.750 INFO:teuthology.orchestra.run.smithi018.stderr:enabled application 'rados' on pool 'unique_pool_3' 2024-09-10T18:59:50.775 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T18:59:50.775 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T18:59:50.787 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:50.795 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T18:59:50.795 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T18:59:50.808 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:50.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:50.809 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:50.810 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T18:59:50.810 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T18:59:50.825 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T18:59:50.826 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T18:59:50.841 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T18:59:50.841 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T18:59:50.857 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T18:59:50.857 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T18:59:50.870 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-10T18:59:50.872 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T18:59:50.873 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T18:59:50.888 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T18:59:50.889 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T18:59:50.910 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:50.933 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:50.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:50.934 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:50.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:50 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:59:50.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:50 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:59:50.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:50 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3597462325' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:50.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:50 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:59:50.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:50 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:59:50.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:50 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3597462325' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:51.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:50 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:59:51.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:50 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:59:51.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:50 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3597462325' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:51.035 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:51.057 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:51.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:51.058 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:51.121 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:59:51.159 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:51.180 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:51.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:51.181 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:51.281 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:51.305 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:51.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:51.307 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:51.307 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 18:59:50 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:18:59:50] "GET /metrics HTTP/1.1" 200 222033 "" "Prometheus/2.43.0" 2024-09-10T18:59:51.408 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:51.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:51.433 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:51.433 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:51.534 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:51.557 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:51.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:51.558 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:51.659 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:51.660 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:59:51.685 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:51.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:51.686 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:51.699 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:59:51.787 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:51.809 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:51.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:51.810 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:51.911 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:51.936 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:51 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3597462325' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:59:51.936 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:51 smithi018 ceph-mon[40196]: osdmap e322: 8 total, 8 up, 6 in 2024-09-10T18:59:51.936 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:51 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3174789244' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:59:51.936 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:51 smithi018 ceph-mon[40196]: 1.8 scrub starts 2024-09-10T18:59:51.936 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:51 smithi018 ceph-mon[40196]: 1.8 scrub ok 2024-09-10T18:59:51.936 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:51 smithi018 ceph-mon[40196]: 2.9 scrub starts 2024-09-10T18:59:51.936 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:51 smithi018 ceph-mon[40196]: 2.9 scrub ok 2024-09-10T18:59:51.936 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:51 smithi018 ceph-mon[40196]: pgmap v673: 168 pgs: 3 peering, 165 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 2 B/s, 0 objects/s recovering 2024-09-10T18:59:51.937 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:51 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3597462325' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:59:51.937 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:51 smithi018 ceph-mon[29336]: osdmap e322: 8 total, 8 up, 6 in 2024-09-10T18:59:51.937 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:51 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3174789244' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:59:51.937 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:51 smithi018 ceph-mon[29336]: 1.8 scrub starts 2024-09-10T18:59:51.937 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:51 smithi018 ceph-mon[29336]: 1.8 scrub ok 2024-09-10T18:59:51.937 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:51 smithi018 ceph-mon[29336]: 2.9 scrub starts 2024-09-10T18:59:51.937 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:51 smithi018 ceph-mon[29336]: 2.9 scrub ok 2024-09-10T18:59:51.937 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:51 smithi018 ceph-mon[29336]: pgmap v673: 168 pgs: 3 peering, 165 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 2 B/s, 0 objects/s recovering 2024-09-10T18:59:51.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:51.938 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:51.938 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:52.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:51 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3597462325' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:59:52.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:51 smithi103 ceph-mon[33163]: osdmap e322: 8 total, 8 up, 6 in 2024-09-10T18:59:52.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:51 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3174789244' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T18:59:52.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:51 smithi103 ceph-mon[33163]: 1.8 scrub starts 2024-09-10T18:59:52.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:51 smithi103 ceph-mon[33163]: 1.8 scrub ok 2024-09-10T18:59:52.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:51 smithi103 ceph-mon[33163]: 2.9 scrub starts 2024-09-10T18:59:52.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:51 smithi103 ceph-mon[33163]: 2.9 scrub ok 2024-09-10T18:59:52.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:51 smithi103 ceph-mon[33163]: pgmap v673: 168 pgs: 3 peering, 165 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 2 B/s, 0 objects/s recovering 2024-09-10T18:59:52.039 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:52.061 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:52.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:52.062 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:52.163 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:52.187 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:52.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:52.188 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:52.289 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:52.312 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:52.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:52.313 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:52.413 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:52.438 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:52.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:52.439 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:52.540 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:52.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:52.564 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:52.564 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:52.665 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:52.687 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:52.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:52.688 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:52.789 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:52.813 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:52.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:52.814 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:52.915 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:52.937 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:52.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:52.938 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:52.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:52 smithi018 ceph-mon[29336]: 6.a scrub starts 2024-09-10T18:59:52.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:52 smithi018 ceph-mon[29336]: 6.a scrub ok 2024-09-10T18:59:52.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:52 smithi018 ceph-mon[29336]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T18:59:52.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:52 smithi018 ceph-mon[29336]: 4.17 scrub starts 2024-09-10T18:59:52.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:52 smithi018 ceph-mon[29336]: 4.17 scrub ok 2024-09-10T18:59:52.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:52 smithi018 ceph-mon[40196]: 6.a scrub starts 2024-09-10T18:59:52.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:52 smithi018 ceph-mon[40196]: 6.a scrub ok 2024-09-10T18:59:52.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:52 smithi018 ceph-mon[40196]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T18:59:52.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:52 smithi018 ceph-mon[40196]: 4.17 scrub starts 2024-09-10T18:59:52.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:52 smithi018 ceph-mon[40196]: 4.17 scrub ok 2024-09-10T18:59:52.981 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:52 smithi103 ceph-mon[33163]: 6.a scrub starts 2024-09-10T18:59:52.981 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:52 smithi103 ceph-mon[33163]: 6.a scrub ok 2024-09-10T18:59:52.981 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:52 smithi103 ceph-mon[33163]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T18:59:52.981 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:52 smithi103 ceph-mon[33163]: 4.17 scrub starts 2024-09-10T18:59:52.981 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:52 smithi103 ceph-mon[33163]: 4.17 scrub ok 2024-09-10T18:59:53.039 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:53.065 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:53.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:53.066 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:53.166 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:53.192 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:53.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:53.193 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:53.209 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T18:59:53.209 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:59:53.294 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:53.317 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:53.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:53.317 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:53.418 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:53.439 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:53.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:53.440 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:53.447 DEBUG:teuthology.orchestra.run.smithi018:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_3 bench 90 write' 2024-09-10T18:59:53.450 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-10T18:59:53.450 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-10T18:59:53.541 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:53.562 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:53.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:53.563 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:53.594 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:hints = 1 2024-09-10T18:59:53.594 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-10T18:59:53.594 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Object prefix: benchmark_data_smithi018_157943 2024-09-10T18:59:53.594 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T18:59:53.595 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 0 0 0 0 0 0 - 0 2024-09-10T18:59:53.651 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T18:59:53.663 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:53.697 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:53.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:53.698 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:53.798 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:53.838 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:53.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:53.839 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:53.892 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-10T18:59:53.939 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:53.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:53 smithi018 ceph-mon[40196]: 5.14 scrub starts 2024-09-10T18:59:53.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:53 smithi018 ceph-mon[40196]: 5.14 scrub ok 2024-09-10T18:59:53.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:53 smithi018 ceph-mon[40196]: 4.12 scrub starts 2024-09-10T18:59:53.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:53 smithi018 ceph-mon[40196]: 4.12 scrub ok 2024-09-10T18:59:53.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:53 smithi018 ceph-mon[40196]: pgmap v674: 168 pgs: 3 peering, 165 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 57 B/s, 1 objects/s recovering 2024-09-10T18:59:53.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:53 smithi018 ceph-mon[29336]: 5.14 scrub starts 2024-09-10T18:59:53.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:53 smithi018 ceph-mon[29336]: 5.14 scrub ok 2024-09-10T18:59:53.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:53 smithi018 ceph-mon[29336]: 4.12 scrub starts 2024-09-10T18:59:53.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:53 smithi018 ceph-mon[29336]: 4.12 scrub ok 2024-09-10T18:59:53.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:53 smithi018 ceph-mon[29336]: pgmap v674: 168 pgs: 3 peering, 165 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 57 B/s, 1 objects/s recovering 2024-09-10T18:59:53.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:53.981 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:53.981 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:54.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:53 smithi103 ceph-mon[33163]: 5.14 scrub starts 2024-09-10T18:59:54.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:53 smithi103 ceph-mon[33163]: 5.14 scrub ok 2024-09-10T18:59:54.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:53 smithi103 ceph-mon[33163]: 4.12 scrub starts 2024-09-10T18:59:54.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:53 smithi103 ceph-mon[33163]: 4.12 scrub ok 2024-09-10T18:59:54.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:53 smithi103 ceph-mon[33163]: pgmap v674: 168 pgs: 3 peering, 165 active+clean; 652 KiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 57 B/s, 1 objects/s recovering 2024-09-10T18:59:54.082 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:54.118 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:54.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:54.119 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:54.220 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:54.257 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:54.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:54.258 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:54.338 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:59:54.358 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:54.396 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:54.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:54.397 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:54.498 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:54.534 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:54.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:54.535 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:54.594 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 1 16 3738 3722 232.612 232.625 0.00407873 0.0042872 2024-09-10T18:59:54.636 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:54.668 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:54.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:54.669 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:54.770 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:54.805 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:54.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:54.806 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:54.906 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:54.941 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:54.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:54.943 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:54.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:54 smithi018 ceph-mon[29336]: 2.18 scrub starts 2024-09-10T18:59:54.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:54 smithi018 ceph-mon[29336]: 2.18 scrub ok 2024-09-10T18:59:54.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:54 smithi018 ceph-mon[29336]: 1.7 scrub starts 2024-09-10T18:59:54.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:54 smithi018 ceph-mon[29336]: 1.7 scrub ok 2024-09-10T18:59:54.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:54 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:59:54.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:54 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:59:54.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:54 smithi018 ceph-mon[40196]: 2.18 scrub starts 2024-09-10T18:59:54.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:54 smithi018 ceph-mon[40196]: 2.18 scrub ok 2024-09-10T18:59:54.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:54 smithi018 ceph-mon[40196]: 1.7 scrub starts 2024-09-10T18:59:54.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:54 smithi018 ceph-mon[40196]: 1.7 scrub ok 2024-09-10T18:59:54.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:54 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:59:54.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:54 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:59:55.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:54 smithi103 ceph-mon[33163]: 2.18 scrub starts 2024-09-10T18:59:55.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:54 smithi103 ceph-mon[33163]: 2.18 scrub ok 2024-09-10T18:59:55.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:54 smithi103 ceph-mon[33163]: 1.7 scrub starts 2024-09-10T18:59:55.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:54 smithi103 ceph-mon[33163]: 1.7 scrub ok 2024-09-10T18:59:55.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:54 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:59:55.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:54 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:59:55.044 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:55.075 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:55.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:55.076 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:55.177 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:55.215 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:55.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:55.216 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:55.243 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:59:55.317 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:55.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:55.356 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:55.356 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:55.457 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:55.499 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:55.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:55.500 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:55.595 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 2 15 6824 6809 212.748 192.938 0.00718644 0.00468647 2024-09-10T18:59:55.601 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:55.642 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:55.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:55.643 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:55.744 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:55.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:55.780 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:55.781 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:55.881 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:55.905 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:55 smithi103 ceph-mon[33163]: 2.5 scrub starts 2024-09-10T18:59:55.905 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:55 smithi103 ceph-mon[33163]: 2.5 scrub ok 2024-09-10T18:59:55.905 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:55 smithi103 ceph-mon[33163]: osdmap e323: 8 total, 8 up, 6 in 2024-09-10T18:59:55.905 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:59:55.905 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:55 smithi103 ceph-mon[33163]: pgmap v676: 168 pgs: 168 active+clean; 4.1 MiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 5.1 KiB/s rd, 531 KiB/s wr, 17 op/s; 51 B/s, 4 objects/s recovering 2024-09-10T18:59:55.919 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:55.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:55.921 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:56.021 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:56.055 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:56.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:56.056 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:56.109 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:55 smithi018 ceph-mon[40196]: 2.5 scrub starts 2024-09-10T18:59:56.109 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:55 smithi018 ceph-mon[40196]: 2.5 scrub ok 2024-09-10T18:59:56.109 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:55 smithi018 ceph-mon[40196]: osdmap e323: 8 total, 8 up, 6 in 2024-09-10T18:59:56.109 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:59:56.109 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:55 smithi018 ceph-mon[40196]: pgmap v676: 168 pgs: 168 active+clean; 4.1 MiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 5.1 KiB/s rd, 531 KiB/s wr, 17 op/s; 51 B/s, 4 objects/s recovering 2024-09-10T18:59:56.110 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:55 smithi018 ceph-mon[29336]: 2.5 scrub starts 2024-09-10T18:59:56.110 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:55 smithi018 ceph-mon[29336]: 2.5 scrub ok 2024-09-10T18:59:56.110 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:55 smithi018 ceph-mon[29336]: osdmap e323: 8 total, 8 up, 6 in 2024-09-10T18:59:56.110 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:59:56.110 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:55 smithi018 ceph-mon[29336]: pgmap v676: 168 pgs: 168 active+clean; 4.1 MiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 5.1 KiB/s rd, 531 KiB/s wr, 17 op/s; 51 B/s, 4 objects/s recovering 2024-09-10T18:59:56.157 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:56.193 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:56.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:56.194 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:56.248 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T18:59:56.248 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T18:59:56.295 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:56.336 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:56.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:56.337 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:56.438 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:56.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:56.474 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:56.475 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:56.575 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:56.595 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 3 16 9761 9745 202.987 183.5 0.00465728 0.00491652 2024-09-10T18:59:56.615 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:56.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:56.616 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:56.717 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:56.752 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:56.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:56.754 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:56.793 INFO:teuthology.orchestra.run.smithi018.stderr:reweighted osd.1 to 1 (10000) 2024-09-10T18:59:56.795 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is unset 2024-09-10T18:59:56.854 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:56.890 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:56.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:56.891 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:56.991 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:57.022 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 18:59:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T18:59:57.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:56 smithi103 ceph-mon[33163]: 5.8 scrub starts 2024-09-10T18:59:57.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:56 smithi103 ceph-mon[33163]: 5.8 scrub ok 2024-09-10T18:59:57.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:56 smithi103 ceph-mon[33163]: 5.15 scrub starts 2024-09-10T18:59:57.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:56 smithi103 ceph-mon[33163]: 5.15 scrub ok 2024-09-10T18:59:57.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:56 smithi103 ceph-mon[33163]: 6.12 scrub starts 2024-09-10T18:59:57.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:56 smithi103 ceph-mon[33163]: 6.12 scrub ok 2024-09-10T18:59:57.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:56 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2249021478' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:59:57.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:56 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:59:57.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:56 smithi103 ceph-mon[33163]: 1.9 scrub starts 2024-09-10T18:59:57.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:56 smithi103 ceph-mon[33163]: 1.9 scrub ok 2024-09-10T18:59:57.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:56 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/4054919974' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-10T18:59:57.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:56 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-10T18:59:57.026 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:57.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:57.027 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:57.128 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:57.167 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:57.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:57.168 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:57.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:56 smithi018 ceph-mon[29336]: 5.8 scrub starts 2024-09-10T18:59:57.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:56 smithi018 ceph-mon[29336]: 5.8 scrub ok 2024-09-10T18:59:57.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:56 smithi018 ceph-mon[29336]: 5.15 scrub starts 2024-09-10T18:59:57.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:56 smithi018 ceph-mon[29336]: 5.15 scrub ok 2024-09-10T18:59:57.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:56 smithi018 ceph-mon[29336]: 6.12 scrub starts 2024-09-10T18:59:57.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:56 smithi018 ceph-mon[29336]: 6.12 scrub ok 2024-09-10T18:59:57.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:56 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2249021478' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:59:57.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:56 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:59:57.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:56 smithi018 ceph-mon[29336]: 1.9 scrub starts 2024-09-10T18:59:57.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:56 smithi018 ceph-mon[29336]: 1.9 scrub ok 2024-09-10T18:59:57.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:56 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/4054919974' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-10T18:59:57.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:56 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-10T18:59:57.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:56 smithi018 ceph-mon[40196]: 5.8 scrub starts 2024-09-10T18:59:57.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:56 smithi018 ceph-mon[40196]: 5.8 scrub ok 2024-09-10T18:59:57.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:56 smithi018 ceph-mon[40196]: 5.15 scrub starts 2024-09-10T18:59:57.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:56 smithi018 ceph-mon[40196]: 5.15 scrub ok 2024-09-10T18:59:57.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:56 smithi018 ceph-mon[40196]: 6.12 scrub starts 2024-09-10T18:59:57.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:56 smithi018 ceph-mon[40196]: 6.12 scrub ok 2024-09-10T18:59:57.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:56 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2249021478' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:59:57.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:56 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T18:59:57.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:56 smithi018 ceph-mon[40196]: 1.9 scrub starts 2024-09-10T18:59:57.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:56 smithi018 ceph-mon[40196]: 1.9 scrub ok 2024-09-10T18:59:57.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:56 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/4054919974' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-10T18:59:57.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:56 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-10T18:59:57.269 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:57.294 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:57.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:57.295 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:57.396 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:57.435 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:57.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:57.437 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:57.537 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:57.565 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:57.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:57.566 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:57.595 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 4 16 12173 12157 189.925 150.75 0.00811631 0.00525908 2024-09-10T18:59:57.667 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:57.703 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:57.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:57.704 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:57.805 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:57.829 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:57.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:57.830 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:57.931 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:57.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:57 smithi018 ceph-mon[29336]: 5.3 scrub starts 2024-09-10T18:59:57.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:57 smithi018 ceph-mon[29336]: 5.3 scrub ok 2024-09-10T18:59:57.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:57 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T18:59:57.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:57 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T18:59:57.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:57 smithi018 ceph-mon[29336]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:59:57.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:57 smithi018 ceph-mon[29336]: Cluster is now healthy 2024-09-10T18:59:57.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:57 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:59:57.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:57 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-10T18:59:57.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:57 smithi018 ceph-mon[29336]: osdmap e324: 8 total, 8 up, 6 in 2024-09-10T18:59:57.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:57 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:59:57.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:57 smithi018 ceph-mon[29336]: pgmap v678: 168 pgs: 168 active+clean; 474 MiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 2.7 KiB/s rd, 72 MiB/s wr, 1.15k op/s; 0 B/s, 4 objects/s recovering 2024-09-10T18:59:57.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:57 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "10.c", "id": [4, 1]}]: dispatch 2024-09-10T18:59:57.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:57 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "10.c", "id": [4, 1]}]: dispatch 2024-09-10T18:59:57.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:57 smithi018 ceph-mon[40196]: 5.3 scrub starts 2024-09-10T18:59:57.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:57 smithi018 ceph-mon[40196]: 5.3 scrub ok 2024-09-10T18:59:57.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:57 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T18:59:57.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:57 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T18:59:57.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:57 smithi018 ceph-mon[40196]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:59:57.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:57 smithi018 ceph-mon[40196]: Cluster is now healthy 2024-09-10T18:59:57.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:57 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:59:57.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:57 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-10T18:59:57.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:57 smithi018 ceph-mon[40196]: osdmap e324: 8 total, 8 up, 6 in 2024-09-10T18:59:57.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:57 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:59:57.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:57 smithi018 ceph-mon[40196]: pgmap v678: 168 pgs: 168 active+clean; 474 MiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 2.7 KiB/s rd, 72 MiB/s wr, 1.15k op/s; 0 B/s, 4 objects/s recovering 2024-09-10T18:59:57.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:57 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "10.c", "id": [4, 1]}]: dispatch 2024-09-10T18:59:57.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:57 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "10.c", "id": [4, 1]}]: dispatch 2024-09-10T18:59:57.952 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:57.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:57.953 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:58.054 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:58.076 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:58.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:58.077 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:58.178 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T18:59:58.199 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:58.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:58.200 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T18:59:58.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:57 smithi103 ceph-mon[33163]: 5.3 scrub starts 2024-09-10T18:59:58.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:57 smithi103 ceph-mon[33163]: 5.3 scrub ok 2024-09-10T18:59:58.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:57 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T18:59:58.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:57 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T18:59:58.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:57 smithi103 ceph-mon[33163]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T18:59:58.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:57 smithi103 ceph-mon[33163]: Cluster is now healthy 2024-09-10T18:59:58.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:57 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T18:59:58.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:57 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-10T18:59:58.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:57 smithi103 ceph-mon[33163]: osdmap e324: 8 total, 8 up, 6 in 2024-09-10T18:59:58.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:57 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:59:58.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:57 smithi103 ceph-mon[33163]: pgmap v678: 168 pgs: 168 active+clean; 474 MiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 2.7 KiB/s rd, 72 MiB/s wr, 1.15k op/s; 0 B/s, 4 objects/s recovering 2024-09-10T18:59:58.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:57 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "10.c", "id": [4, 1]}]: dispatch 2024-09-10T18:59:58.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:57 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "10.c", "id": [4, 1]}]: dispatch 2024-09-10T18:59:58.301 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:58.323 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:58.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:58.324 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:58.425 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:58.448 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:58.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:58.449 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:58.550 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:58.572 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:58.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:58.573 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:58.595 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 5 16 12967 12951 161.865 49.625 0.00143259 0.00543672 2024-09-10T18:59:58.674 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T18:59:58.700 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:58.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:58.701 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T18:59:58.802 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:58.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:58.828 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:58.828 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T18:59:58.929 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:58.951 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-10T18:59:58.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:58.952 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:58.953 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:59.054 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T18:59:59.078 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:59.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:59.079 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T18:59:59.161 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:58 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "10.c", "id": [4, 1]}]': finished 2024-09-10T18:59:59.161 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:58 smithi018 ceph-mon[40196]: osdmap e325: 8 total, 8 up, 6 in 2024-09-10T18:59:59.161 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:58 smithi018 ceph-mon[40196]: 5.1e scrub starts 2024-09-10T18:59:59.161 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:58 smithi018 ceph-mon[40196]: 5.1e scrub ok 2024-09-10T18:59:59.161 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:58 smithi018 ceph-mon[40196]: 5.1 deep-scrub starts 2024-09-10T18:59:59.161 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:58 smithi018 ceph-mon[40196]: 5.1 deep-scrub ok 2024-09-10T18:59:59.161 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:58 smithi018 ceph-mon[40196]: osdmap e326: 8 total, 8 up, 6 in 2024-09-10T18:59:59.162 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:58 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "10.c", "id": [4, 1]}]': finished 2024-09-10T18:59:59.162 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:58 smithi018 ceph-mon[29336]: osdmap e325: 8 total, 8 up, 6 in 2024-09-10T18:59:59.162 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:58 smithi018 ceph-mon[29336]: 5.1e scrub starts 2024-09-10T18:59:59.164 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:58 smithi018 ceph-mon[29336]: 5.1e scrub ok 2024-09-10T18:59:59.164 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:58 smithi018 ceph-mon[29336]: 5.1 deep-scrub starts 2024-09-10T18:59:59.164 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:58 smithi018 ceph-mon[29336]: 5.1 deep-scrub ok 2024-09-10T18:59:59.164 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:58 smithi018 ceph-mon[29336]: osdmap e326: 8 total, 8 up, 6 in 2024-09-10T18:59:59.179 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T18:59:59.190 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd reweight 3 1 2024-09-10T18:59:59.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:59.204 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:59.204 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T18:59:59.254 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:59:59.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:58 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "10.c", "id": [4, 1]}]': finished 2024-09-10T18:59:59.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:58 smithi103 ceph-mon[33163]: osdmap e325: 8 total, 8 up, 6 in 2024-09-10T18:59:59.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:58 smithi103 ceph-mon[33163]: 5.1e scrub starts 2024-09-10T18:59:59.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:58 smithi103 ceph-mon[33163]: 5.1e scrub ok 2024-09-10T18:59:59.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:58 smithi103 ceph-mon[33163]: 5.1 deep-scrub starts 2024-09-10T18:59:59.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:58 smithi103 ceph-mon[33163]: 5.1 deep-scrub ok 2024-09-10T18:59:59.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:58 smithi103 ceph-mon[33163]: osdmap e326: 8 total, 8 up, 6 in 2024-09-10T18:59:59.305 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T18:59:59.327 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:59.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:59.328 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T18:59:59.429 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T18:59:59.452 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T18:59:59.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:59.453 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T18:59:59.554 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T18:59:59.581 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:59.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:59.582 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T18:59:59.595 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 6 16 12967 12951 134.889 0 - 0.00543672 2024-09-10T18:59:59.682 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T18:59:59.708 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T18:59:59.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:59.908 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:00.009 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:00.046 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:00.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:00.048 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:00.148 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:00.181 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:00.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:00.182 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:00.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:59 smithi018 ceph-mon[29336]: pgmap v681: 168 pgs: 168 active+clean; 619 MiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 134 MiB/s wr, 2.14k op/s 2024-09-10T19:00:00.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:59 smithi018 ceph-mon[29336]: 2.10 scrub starts 2024-09-10T19:00:00.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:59 smithi018 ceph-mon[29336]: 2.10 scrub ok 2024-09-10T19:00:00.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 18:59:59 smithi018 ceph-mon[29336]: osdmap e327: 8 total, 8 up, 6 in 2024-09-10T19:00:00.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:59 smithi018 ceph-mon[40196]: pgmap v681: 168 pgs: 168 active+clean; 619 MiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 134 MiB/s wr, 2.14k op/s 2024-09-10T19:00:00.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:59 smithi018 ceph-mon[40196]: 2.10 scrub starts 2024-09-10T19:00:00.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:59 smithi018 ceph-mon[40196]: 2.10 scrub ok 2024-09-10T19:00:00.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 18:59:59 smithi018 ceph-mon[40196]: osdmap e327: 8 total, 8 up, 6 in 2024-09-10T19:00:00.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:59 smithi103 ceph-mon[33163]: pgmap v681: 168 pgs: 168 active+clean; 619 MiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 134 MiB/s wr, 2.14k op/s 2024-09-10T19:00:00.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:59 smithi103 ceph-mon[33163]: 2.10 scrub starts 2024-09-10T19:00:00.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:59 smithi103 ceph-mon[33163]: 2.10 scrub ok 2024-09-10T19:00:00.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 18:59:59 smithi103 ceph-mon[33163]: osdmap e327: 8 total, 8 up, 6 in 2024-09-10T19:00:00.284 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:00.328 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:00.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:00.329 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:00.430 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:00.464 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:00.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:00.465 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:00.480 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-10T19:00:00.481 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T19:00:00.481 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T19:00:00.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:00:00 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:00:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:00:00.523 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T19:00:00.523 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T19:00:00.556 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T19:00:00.556 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T19:00:00.567 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:00.581 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T19:00:00.581 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T19:00:00.596 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 7 16 14716 14700 131.231 54.6562 0.0060553 0.00761162 2024-09-10T19:00:00.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:00.616 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:00.616 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:00.616 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T19:00:00.616 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T19:00:00.646 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T19:00:00.646 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T19:00:00.679 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T19:00:00.679 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T19:00:00.711 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T19:00:00.712 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T19:00:00.717 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:00.758 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:00.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:00.759 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:00.860 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:00.888 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:00.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:00.889 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:00.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:00 smithi018 ceph-mon[40196]: 2.1f deep-scrub starts 2024-09-10T19:00:00.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:00 smithi018 ceph-mon[40196]: 2.1f deep-scrub ok 2024-09-10T19:00:00.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:00 smithi018 ceph-mon[40196]: overall HEALTH_OK 2024-09-10T19:00:00.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:00 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/428079087' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:00:00.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:00 smithi018 ceph-mon[29336]: 2.1f deep-scrub starts 2024-09-10T19:00:00.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:00 smithi018 ceph-mon[29336]: 2.1f deep-scrub ok 2024-09-10T19:00:00.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:00 smithi018 ceph-mon[29336]: overall HEALTH_OK 2024-09-10T19:00:00.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:00 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/428079087' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:00:00.989 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:01.030 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:01.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:01.031 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:01.132 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:01.173 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:01.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:01.174 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:01.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:00 smithi103 ceph-mon[33163]: 2.1f deep-scrub starts 2024-09-10T19:00:01.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:00 smithi103 ceph-mon[33163]: 2.1f deep-scrub ok 2024-09-10T19:00:01.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:00 smithi103 ceph-mon[33163]: overall HEALTH_OK 2024-09-10T19:00:01.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:00 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/428079087' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:00:01.275 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:01.306 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:01.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:01.307 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:01.408 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:01.442 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:01.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:01.443 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:01.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:00:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:00:00] "GET /metrics HTTP/1.1" 200 224432 "" "Prometheus/2.43.0" 2024-09-10T19:00:01.543 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:01.581 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:01.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:01.582 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:01.595 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 8 16 17232 17216 134.481 157.25 0.00445484 0.0074291 2024-09-10T19:00:01.683 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:01.714 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:01.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:01.716 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:01.816 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:01.840 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:01.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:01.840 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:01.941 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:01.975 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:01.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:01.976 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:02.077 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:02.106 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:02.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:02.107 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:02.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:01 smithi018 ceph-mon[40196]: 1.1 scrub starts 2024-09-10T19:00:02.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:01 smithi018 ceph-mon[40196]: pgmap v683: 168 pgs: 168 active+clean; 710 MiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 52 MiB/s wr, 823 op/s 2024-09-10T19:00:02.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:01 smithi018 ceph-mon[29336]: 1.1 scrub starts 2024-09-10T19:00:02.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:01 smithi018 ceph-mon[29336]: pgmap v683: 168 pgs: 168 active+clean; 710 MiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 52 MiB/s wr, 823 op/s 2024-09-10T19:00:02.208 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:02.249 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:02.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:02.250 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:02.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:01 smithi103 ceph-mon[33163]: 1.1 scrub starts 2024-09-10T19:00:02.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:01 smithi103 ceph-mon[33163]: pgmap v683: 168 pgs: 168 active+clean; 710 MiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 52 MiB/s wr, 823 op/s 2024-09-10T19:00:02.350 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:02.383 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:02.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:02.384 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:02.485 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:02.506 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:02.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:02.507 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:02.595 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 9 15 19278 19263 133.753 127.938 0.00573535 0.00746997 2024-09-10T19:00:02.608 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:02.645 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:02.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:02.646 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:02.747 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:02.769 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:02.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:02.770 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:02.870 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:02.905 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:02.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:02.906 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:03.007 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:03.041 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:03.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:03.042 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:03.142 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:03.180 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:03.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:03.182 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:03.282 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:03.307 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:03.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:03.308 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:03.408 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:03.430 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:03.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:03.431 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:03.532 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:03.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:03.566 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:03.566 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:03.596 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 10 16 21128 21112 131.933 115.562 0.00429362 0.00757376 2024-09-10T19:00:03.668 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:03.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:03 smithi018 ceph-mon[40196]: 4.f scrub starts 2024-09-10T19:00:03.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:03 smithi018 ceph-mon[29336]: 4.f scrub starts 2024-09-10T19:00:03.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:03.703 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:03.703 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:03.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:03 smithi103 ceph-mon[33163]: 4.f scrub starts 2024-09-10T19:00:03.804 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:03.842 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:03.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:03.843 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:03.944 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:03.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:03.986 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:03.986 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:04.087 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:04.138 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:04.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:04.139 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:04.240 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:04.285 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:04.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:04.286 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:04.387 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:04.419 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:04.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:04.420 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:04.521 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:04.558 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:04.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:04.559 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:04.596 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 11 16 23662 23646 134.335 158.375 0.00452535 0.00743794 2024-09-10T19:00:04.661 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:04.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:04 smithi018 ceph-mon[40196]: pgmap v684: 168 pgs: 168 active+clean; 980 MiB data, 6.8 GiB used, 530 GiB / 536 GiB avail; 853 B/s rd, 84 MiB/s wr, 1.35k op/s 2024-09-10T19:00:04.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:04 smithi018 ceph-mon[40196]: osdmap e328: 8 total, 8 up, 6 in 2024-09-10T19:00:04.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:04 smithi018 ceph-mon[29336]: pgmap v684: 168 pgs: 168 active+clean; 980 MiB data, 6.8 GiB used, 530 GiB / 536 GiB avail; 853 B/s rd, 84 MiB/s wr, 1.35k op/s 2024-09-10T19:00:04.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:04 smithi018 ceph-mon[29336]: osdmap e328: 8 total, 8 up, 6 in 2024-09-10T19:00:04.696 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:04.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:04.697 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:04.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:04 smithi103 ceph-mon[33163]: pgmap v684: 168 pgs: 168 active+clean; 980 MiB data, 6.8 GiB used, 530 GiB / 536 GiB avail; 853 B/s rd, 84 MiB/s wr, 1.35k op/s 2024-09-10T19:00:04.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:04 smithi103 ceph-mon[33163]: osdmap e328: 8 total, 8 up, 6 in 2024-09-10T19:00:04.798 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:04.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:04.836 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:04.836 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:04.937 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:04.970 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:04.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:04.971 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:05.072 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:05.107 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:05.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:05.108 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:05.208 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:05.232 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:00:05.232 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:00:05.238 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:05.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:05.239 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:05.299 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T19:00:05.299 INFO:teuthology.orchestra.run.smithi018.stdout: "ops": [ 2024-09-10T19:00:05.300 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.300 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:25217 10.5 10:a1e187f0:::benchmark_data_smithi018_157943_object25216:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e328)", 2024-09-10T19:00:05.300 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:05.286792+0000", 2024-09-10T19:00:05.300 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 0.0080675169999999997, 2024-09-10T19:00:05.300 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.008101703, 2024-09-10T19:00:05.300 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:00:05.300 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "waiting for sub ops", 2024-09-10T19:00:05.300 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:00:05.300 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:00:05.300 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:00:05.300 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 25217 2024-09-10T19:00:05.300 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.301 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:00:05.301 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.301 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:00:05.301 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.286792+0000", 2024-09-10T19:00:05.301 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:00:05.301 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.301 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.301 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:00:05.301 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.286792+0000", 2024-09-10T19:00:05.301 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:00:05.301 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.301 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.301 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:00:05.302 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.286791+0000", 2024-09-10T19:00:05.302 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T19:00:05.302 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.302 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.302 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:00:05.302 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.286820+0000", 2024-09-10T19:00:05.302 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.9043e-05 2024-09-10T19:00:05.302 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.302 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.302 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:00:05.302 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.286832+0000", 2024-09-10T19:00:05.302 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1891e-05 2024-09-10T19:00:05.302 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.302 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.303 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:00:05.303 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.286838+0000", 2024-09-10T19:00:05.303 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.5640000000000002e-06 2024-09-10T19:00:05.303 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.303 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.303 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:00:05.303 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.287840+0000", 2024-09-10T19:00:05.303 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0010018189999999999 2024-09-10T19:00:05.303 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.303 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.303 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:00:05.303 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.288157+0000", 2024-09-10T19:00:05.303 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00031692599999999998 2024-09-10T19:00:05.304 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.304 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.304 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,4", 2024-09-10T19:00:05.304 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.288311+0000", 2024-09-10T19:00:05.304 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00015455199999999999 2024-09-10T19:00:05.304 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.304 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.304 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:00:05.304 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.292743+0000", 2024-09-10T19:00:05.304 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.004431974 2024-09-10T19:00:05.304 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:00:05.304 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:00:05.304 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:00:05.305 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.305 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.305 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:25219 10.c 10:3fa6c50f:::benchmark_data_smithi018_157943_object25218:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e328)", 2024-09-10T19:00:05.305 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:05.286855+0000", 2024-09-10T19:00:05.305 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 0.0080046010000000001, 2024-09-10T19:00:05.305 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0081385710000000007, 2024-09-10T19:00:05.305 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:00:05.305 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "waiting for sub ops", 2024-09-10T19:00:05.305 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:00:05.305 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:00:05.305 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:00:05.305 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 25219 2024-09-10T19:00:05.305 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.305 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:00:05.306 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.306 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:00:05.306 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.286855+0000", 2024-09-10T19:00:05.306 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:00:05.306 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.306 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.306 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:00:05.306 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.286855+0000", 2024-09-10T19:00:05.306 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:00:05.306 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.306 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.306 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:00:05.306 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.286854+0000", 2024-09-10T19:00:05.306 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T19:00:05.307 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.307 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.307 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:00:05.307 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.286878+0000", 2024-09-10T19:00:05.307 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.4108999999999999e-05 2024-09-10T19:00:05.307 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.307 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.307 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:00:05.307 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.286886+0000", 2024-09-10T19:00:05.307 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.7889999999999995e-06 2024-09-10T19:00:05.307 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.307 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.307 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:00:05.308 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.286891+0000", 2024-09-10T19:00:05.308 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.5789999999999997e-06 2024-09-10T19:00:05.308 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.308 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.308 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:00:05.308 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.288459+0000", 2024-09-10T19:00:05.308 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.001567976 2024-09-10T19:00:05.308 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.308 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.308 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:00:05.308 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.288576+0000", 2024-09-10T19:00:05.308 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00011696000000000001 2024-09-10T19:00:05.308 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.308 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.309 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,3", 2024-09-10T19:00:05.309 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.288722+0000", 2024-09-10T19:00:05.309 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00014662900000000001 2024-09-10T19:00:05.309 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.309 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.309 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:00:05.309 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.294786+0000", 2024-09-10T19:00:05.309 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0060639789999999997 2024-09-10T19:00:05.309 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.309 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.309 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:00:05.309 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.294799+0000", 2024-09-10T19:00:05.309 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.3232e-05 2024-09-10T19:00:05.309 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:00:05.310 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:00:05.310 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:00:05.310 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.310 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.310 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:25220 10.c 10:34a11d9d:::benchmark_data_smithi018_157943_object25219:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e328)", 2024-09-10T19:00:05.310 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:05.286903+0000", 2024-09-10T19:00:05.310 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 0.0079558779999999996, 2024-09-10T19:00:05.310 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0081910560000000004, 2024-09-10T19:00:05.310 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:00:05.310 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "waiting for sub ops", 2024-09-10T19:00:05.310 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:00:05.310 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:00:05.310 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:00:05.310 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 25220 2024-09-10T19:00:05.311 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.311 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:00:05.311 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.311 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:00:05.311 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.286903+0000", 2024-09-10T19:00:05.311 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:00:05.311 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.311 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.311 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:00:05.311 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.286903+0000", 2024-09-10T19:00:05.311 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:00:05.311 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.311 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.311 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:00:05.312 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.286903+0000", 2024-09-10T19:00:05.312 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T19:00:05.312 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.312 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.312 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:00:05.312 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.286930+0000", 2024-09-10T19:00:05.312 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.7008999999999998e-05 2024-09-10T19:00:05.312 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.312 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.312 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:00:05.312 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.286938+0000", 2024-09-10T19:00:05.312 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.106e-06 2024-09-10T19:00:05.312 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.313 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.313 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:00:05.313 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.286942+0000", 2024-09-10T19:00:05.313 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.233e-06 2024-09-10T19:00:05.313 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.313 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.313 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:00:05.313 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.289726+0000", 2024-09-10T19:00:05.313 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0027835360000000001 2024-09-10T19:00:05.313 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.313 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.313 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:00:05.313 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.289857+0000", 2024-09-10T19:00:05.313 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000130858 2024-09-10T19:00:05.314 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.314 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.314 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,3", 2024-09-10T19:00:05.314 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.290032+0000", 2024-09-10T19:00:05.314 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00017499799999999999 2024-09-10T19:00:05.314 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.314 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.314 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:00:05.314 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.294898+0000", 2024-09-10T19:00:05.314 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0048664219999999996 2024-09-10T19:00:05.314 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:00:05.314 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:00:05.314 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:00:05.314 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.315 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.315 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.15573.0:25223 10.1 e328/315 10:888d7ca6:::benchmark_data_smithi018_157943_object25222:head v 328'1591, mlcod=328'1589)", 2024-09-10T19:00:05.315 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:05.290553+0000", 2024-09-10T19:00:05.315 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 0.004306435, 2024-09-10T19:00:05.315 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0046218159999999999, 2024-09-10T19:00:05.315 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:00:05.315 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "started", 2024-09-10T19:00:05.315 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:00:05.315 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.315 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:00:05.315 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.290553+0000", 2024-09-10T19:00:05.315 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:00:05.315 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.316 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.316 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:00:05.316 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.290553+0000", 2024-09-10T19:00:05.316 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:00:05.316 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.316 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.316 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:00:05.316 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.290553+0000", 2024-09-10T19:00:05.316 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T19:00:05.316 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.316 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.316 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:00:05.316 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.290589+0000", 2024-09-10T19:00:05.317 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.6238000000000001e-05 2024-09-10T19:00:05.317 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.317 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.317 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:00:05.317 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.290595+0000", 2024-09-10T19:00:05.317 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.0530000000000003e-06 2024-09-10T19:00:05.317 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.317 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.317 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:00:05.317 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.290598+0000", 2024-09-10T19:00:05.317 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.6600000000000001e-06 2024-09-10T19:00:05.317 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.317 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.317 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:00:05.318 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.290650+0000", 2024-09-10T19:00:05.318 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.1617999999999999e-05 2024-09-10T19:00:05.318 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.318 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.318 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:00:05.318 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.290667+0000", 2024-09-10T19:00:05.318 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.6473e-05 2024-09-10T19:00:05.318 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:00:05.318 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:00:05.318 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:00:05.318 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.318 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.318 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.15573.0:25228 10.1 e328/315 10:82906ded:::benchmark_data_smithi018_157943_object25227:head v 328'1592, mlcod=328'1589)", 2024-09-10T19:00:05.318 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:05.290916+0000", 2024-09-10T19:00:05.318 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 0.0039432149999999999, 2024-09-10T19:00:05.319 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0043172380000000002, 2024-09-10T19:00:05.319 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:00:05.319 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "started", 2024-09-10T19:00:05.319 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:00:05.319 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.319 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:00:05.319 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.290916+0000", 2024-09-10T19:00:05.319 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:00:05.319 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.319 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.319 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:00:05.319 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.290916+0000", 2024-09-10T19:00:05.320 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:00:05.320 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.320 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.320 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:00:05.320 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.290915+0000", 2024-09-10T19:00:05.320 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T19:00:05.320 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.320 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.320 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:00:05.320 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291003+0000", 2024-09-10T19:00:05.320 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.7425999999999999e-05 2024-09-10T19:00:05.320 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.320 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.320 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:00:05.321 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291012+0000", 2024-09-10T19:00:05.321 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.3130000000000004e-06 2024-09-10T19:00:05.321 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.321 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.321 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:00:05.321 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291017+0000", 2024-09-10T19:00:05.321 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.0220000000000004e-06 2024-09-10T19:00:05.321 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.321 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.321 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:00:05.321 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291705+0000", 2024-09-10T19:00:05.321 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00068816099999999996 2024-09-10T19:00:05.321 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.321 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.322 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:00:05.322 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291742+0000", 2024-09-10T19:00:05.322 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.6201000000000001e-05 2024-09-10T19:00:05.322 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:00:05.322 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:00:05.322 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:00:05.322 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.322 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.322 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:25224 10.5 10:aed93dc3:::benchmark_data_smithi018_157943_object25223:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e328)", 2024-09-10T19:00:05.322 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:05.291203+0000", 2024-09-10T19:00:05.322 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 0.0036558229999999999, 2024-09-10T19:00:05.322 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0040918960000000002, 2024-09-10T19:00:05.322 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:00:05.322 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "waiting for sub ops", 2024-09-10T19:00:05.323 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:00:05.323 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:00:05.323 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:00:05.323 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 25224 2024-09-10T19:00:05.323 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.323 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:00:05.323 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.323 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:00:05.323 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291203+0000", 2024-09-10T19:00:05.323 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:00:05.323 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.323 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.323 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:00:05.323 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291203+0000", 2024-09-10T19:00:05.324 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:00:05.324 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.324 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.324 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:00:05.324 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291202+0000", 2024-09-10T19:00:05.324 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T19:00:05.324 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.324 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.324 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:00:05.324 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291228+0000", 2024-09-10T19:00:05.324 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.5850000000000002e-05 2024-09-10T19:00:05.324 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.324 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.325 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:00:05.325 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291240+0000", 2024-09-10T19:00:05.325 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.2018000000000001e-05 2024-09-10T19:00:05.325 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.325 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.325 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:00:05.325 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291247+0000", 2024-09-10T19:00:05.325 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.6340000000000003e-06 2024-09-10T19:00:05.325 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.325 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.325 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:00:05.325 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.292769+0000", 2024-09-10T19:00:05.325 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0015225709999999999 2024-09-10T19:00:05.325 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.326 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.326 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:00:05.326 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.292890+0000", 2024-09-10T19:00:05.326 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000120895 2024-09-10T19:00:05.326 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.326 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.326 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,4", 2024-09-10T19:00:05.326 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.293030+0000", 2024-09-10T19:00:05.326 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00014038500000000001 2024-09-10T19:00:05.326 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:00:05.326 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:00:05.326 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:00:05.326 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.326 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.327 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:25225 10.5 10:a68c9fcf:::benchmark_data_smithi018_157943_object25224:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e328)", 2024-09-10T19:00:05.327 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:05.291255+0000", 2024-09-10T19:00:05.327 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 0.0036039280000000002, 2024-09-10T19:00:05.327 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0041149170000000001, 2024-09-10T19:00:05.327 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:00:05.327 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "waiting for sub ops", 2024-09-10T19:00:05.327 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:00:05.328 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:00:05.328 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:00:05.328 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 25225 2024-09-10T19:00:05.328 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.328 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:00:05.328 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.328 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:00:05.328 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291255+0000", 2024-09-10T19:00:05.328 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:00:05.328 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.328 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.328 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:00:05.328 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291255+0000", 2024-09-10T19:00:05.329 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:00:05.329 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.329 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.329 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:00:05.329 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291255+0000", 2024-09-10T19:00:05.329 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T19:00:05.329 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.329 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.329 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:00:05.329 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291275+0000", 2024-09-10T19:00:05.329 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.0072e-05 2024-09-10T19:00:05.329 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.329 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.329 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:00:05.330 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291281+0000", 2024-09-10T19:00:05.330 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.8719999999999999e-06 2024-09-10T19:00:05.330 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.330 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.330 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:00:05.330 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291285+0000", 2024-09-10T19:00:05.330 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.8380000000000004e-06 2024-09-10T19:00:05.330 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.330 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.330 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:00:05.330 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.294391+0000", 2024-09-10T19:00:05.330 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0031064830000000002 2024-09-10T19:00:05.330 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.331 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.331 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:00:05.331 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.294511+0000", 2024-09-10T19:00:05.331 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000119707 2024-09-10T19:00:05.331 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.331 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.331 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,4", 2024-09-10T19:00:05.331 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.294761+0000", 2024-09-10T19:00:05.331 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00025014299999999998 2024-09-10T19:00:05.331 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:00:05.331 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:00:05.331 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:00:05.331 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.332 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.332 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:25226 10.5 10.c2f5e975 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e328)", 2024-09-10T19:00:05.332 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:05.291289+0000", 2024-09-10T19:00:05.332 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 0.0035701040000000002, 2024-09-10T19:00:05.332 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0041530539999999998, 2024-09-10T19:00:05.332 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:00:05.332 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "queued for pg", 2024-09-10T19:00:05.332 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:00:05.332 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:00:05.332 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:00:05.332 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 25226 2024-09-10T19:00:05.332 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.332 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:00:05.332 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.333 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:00:05.333 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291289+0000", 2024-09-10T19:00:05.333 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:00:05.333 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.333 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.333 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:00:05.333 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291289+0000", 2024-09-10T19:00:05.333 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:00:05.333 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.333 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.333 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:00:05.333 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291289+0000", 2024-09-10T19:00:05.333 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T19:00:05.334 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.334 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.334 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:00:05.334 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291309+0000", 2024-09-10T19:00:05.334 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.9859000000000001e-05 2024-09-10T19:00:05.334 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.334 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.334 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:00:05.334 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291314+0000", 2024-09-10T19:00:05.334 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.3709999999999999e-06 2024-09-10T19:00:05.334 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.334 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.334 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:00:05.335 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291318+0000", 2024-09-10T19:00:05.335 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.737e-06 2024-09-10T19:00:05.335 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:00:05.335 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:00:05.335 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:00:05.335 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.335 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.335 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:25227 10.4 10:2514a2ce:::benchmark_data_smithi018_157943_object25226:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e328)", 2024-09-10T19:00:05.335 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:05.291322+0000", 2024-09-10T19:00:05.335 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 0.0035372110000000002, 2024-09-10T19:00:05.335 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0041795679999999998, 2024-09-10T19:00:05.335 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:00:05.335 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "waiting for sub ops", 2024-09-10T19:00:05.336 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:00:05.336 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:00:05.336 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:00:05.336 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 25227 2024-09-10T19:00:05.336 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.336 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:00:05.336 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.336 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:00:05.336 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291322+0000", 2024-09-10T19:00:05.336 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:00:05.336 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.336 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.336 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:00:05.336 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291322+0000", 2024-09-10T19:00:05.336 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:00:05.337 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.337 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.337 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:00:05.337 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291322+0000", 2024-09-10T19:00:05.337 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T19:00:05.337 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.337 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.337 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:00:05.337 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291341+0000", 2024-09-10T19:00:05.337 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.8805e-05 2024-09-10T19:00:05.337 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.337 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.337 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:00:05.337 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291346+0000", 2024-09-10T19:00:05.338 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.3759999999999999e-06 2024-09-10T19:00:05.338 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.338 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.338 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:00:05.338 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291349+0000", 2024-09-10T19:00:05.338 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.4599999999999999e-06 2024-09-10T19:00:05.338 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.338 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.338 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:00:05.338 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291568+0000", 2024-09-10T19:00:05.338 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00021851800000000001 2024-09-10T19:00:05.338 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.338 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.339 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:00:05.339 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291694+0000", 2024-09-10T19:00:05.339 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00012653600000000001 2024-09-10T19:00:05.339 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.339 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.339 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,4", 2024-09-10T19:00:05.339 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291897+0000", 2024-09-10T19:00:05.339 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00020217600000000001 2024-09-10T19:00:05.339 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.339 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.339 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:00:05.339 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.294830+0000", 2024-09-10T19:00:05.339 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0029337439999999998 2024-09-10T19:00:05.339 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:00:05.340 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:00:05.340 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:00:05.340 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.340 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.340 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:25229 10.4 10:23dc1242:::benchmark_data_smithi018_157943_object25228:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e328)", 2024-09-10T19:00:05.340 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:05.291360+0000", 2024-09-10T19:00:05.340 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 0.0034995199999999999, 2024-09-10T19:00:05.340 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0042267030000000001, 2024-09-10T19:00:05.340 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:00:05.340 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "waiting for sub ops", 2024-09-10T19:00:05.340 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:00:05.340 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:00:05.340 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:00:05.341 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 25229 2024-09-10T19:00:05.341 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.341 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:00:05.341 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.341 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:00:05.341 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291360+0000", 2024-09-10T19:00:05.341 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:00:05.341 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.341 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.341 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:00:05.341 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291360+0000", 2024-09-10T19:00:05.341 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:00:05.341 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.342 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.342 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:00:05.342 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291359+0000", 2024-09-10T19:00:05.342 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T19:00:05.342 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.342 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.342 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:00:05.342 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291398+0000", 2024-09-10T19:00:05.342 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.8185000000000001e-05 2024-09-10T19:00:05.342 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.342 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.342 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:00:05.342 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291403+0000", 2024-09-10T19:00:05.343 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.8000000000000004e-06 2024-09-10T19:00:05.343 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.343 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.343 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:00:05.343 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.291407+0000", 2024-09-10T19:00:05.343 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.5810000000000001e-06 2024-09-10T19:00:05.343 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.343 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.343 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:00:05.343 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.293318+0000", 2024-09-10T19:00:05.343 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0019108319999999999 2024-09-10T19:00:05.343 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.343 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.343 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:00:05.344 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.293445+0000", 2024-09-10T19:00:05.344 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00012701300000000001 2024-09-10T19:00:05.344 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.344 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.344 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,4", 2024-09-10T19:00:05.344 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.293576+0000", 2024-09-10T19:00:05.344 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000131278 2024-09-10T19:00:05.344 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.344 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.344 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:00:05.344 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.294859+0000", 2024-09-10T19:00:05.344 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0012826879999999999 2024-09-10T19:00:05.344 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:00:05.344 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:00:05.345 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:00:05.345 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.345 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.345 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.15573.0:25230 10.9 e328/315 10:937e273e:::benchmark_data_smithi018_157943_object25229:head v 328'1614, mlcod=328'1611)", 2024-09-10T19:00:05.345 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:05.292105+0000", 2024-09-10T19:00:05.345 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 0.0027542479999999999, 2024-09-10T19:00:05.345 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0035403129999999998, 2024-09-10T19:00:05.345 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:00:05.345 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "started", 2024-09-10T19:00:05.345 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:00:05.345 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.345 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:00:05.345 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.292105+0000", 2024-09-10T19:00:05.345 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:00:05.346 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.346 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.346 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:00:05.346 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.292105+0000", 2024-09-10T19:00:05.346 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:00:05.346 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.346 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.346 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:00:05.346 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.292104+0000", 2024-09-10T19:00:05.346 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T19:00:05.346 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.346 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.346 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:00:05.347 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.292175+0000", 2024-09-10T19:00:05.347 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.0077e-05 2024-09-10T19:00:05.347 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.347 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.347 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:00:05.347 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.292187+0000", 2024-09-10T19:00:05.347 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.2962000000000001e-05 2024-09-10T19:00:05.347 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.347 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.347 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:00:05.347 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.292193+0000", 2024-09-10T19:00:05.347 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.3369999999999999e-06 2024-09-10T19:00:05.347 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.348 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.348 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:00:05.348 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.292381+0000", 2024-09-10T19:00:05.348 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000188662 2024-09-10T19:00:05.348 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.348 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.348 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:00:05.348 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.292400+0000", 2024-09-10T19:00:05.348 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.8399000000000001e-05 2024-09-10T19:00:05.348 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:00:05.348 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:00:05.348 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:00:05.348 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.349 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.349 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.15573.0:25231 10.1 e328/315 10:8818a97d:::benchmark_data_smithi018_157943_object25230:head v 328'1593, mlcod=328'1589)", 2024-09-10T19:00:05.349 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:05.292718+0000", 2024-09-10T19:00:05.349 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 0.0021415380000000001, 2024-09-10T19:00:05.349 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.002964253, 2024-09-10T19:00:05.349 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:00:05.349 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "started", 2024-09-10T19:00:05.349 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:00:05.349 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.349 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:00:05.349 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.292718+0000", 2024-09-10T19:00:05.349 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:00:05.349 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.349 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.350 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:00:05.350 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.292718+0000", 2024-09-10T19:00:05.350 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:00:05.350 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.350 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.350 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:00:05.350 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.292717+0000", 2024-09-10T19:00:05.350 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T19:00:05.350 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.350 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.350 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:00:05.350 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.292798+0000", 2024-09-10T19:00:05.350 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.1403999999999995e-05 2024-09-10T19:00:05.350 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.351 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.351 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:00:05.351 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.292821+0000", 2024-09-10T19:00:05.351 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.2218e-05 2024-09-10T19:00:05.351 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.351 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.351 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:00:05.351 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.292826+0000", 2024-09-10T19:00:05.351 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.9420000000000003e-06 2024-09-10T19:00:05.351 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.351 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.351 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:00:05.351 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.293431+0000", 2024-09-10T19:00:05.352 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00060502500000000003 2024-09-10T19:00:05.352 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.352 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.352 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:00:05.352 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.293450+0000", 2024-09-10T19:00:05.352 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.9524000000000001e-05 2024-09-10T19:00:05.352 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:00:05.352 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:00:05.352 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:00:05.352 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.352 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.352 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop_reply(client.15573.0:25224 10.5 e328/315)", 2024-09-10T19:00:05.352 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:05.294286+0000", 2024-09-10T19:00:05.352 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 0.00057326899999999995, 2024-09-10T19:00:05.353 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0014263469999999999, 2024-09-10T19:00:05.353 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:00:05.353 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "queued for pg", 2024-09-10T19:00:05.353 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:00:05.353 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.353 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:00:05.353 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.294286+0000", 2024-09-10T19:00:05.353 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:00:05.353 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.353 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.353 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:00:05.353 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.294286+0000", 2024-09-10T19:00:05.353 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:00:05.353 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.354 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.354 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:00:05.354 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.294285+0000", 2024-09-10T19:00:05.354 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T19:00:05.354 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.354 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.354 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:00:05.354 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.294293+0000", 2024-09-10T19:00:05.354 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.1130000000000003e-06 2024-09-10T19:00:05.354 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.354 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.354 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:00:05.354 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.294303+0000", 2024-09-10T19:00:05.355 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0256e-05 2024-09-10T19:00:05.355 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.355 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.355 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:00:05.355 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.294308+0000", 2024-09-10T19:00:05.355 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.3499999999999996e-06 2024-09-10T19:00:05.355 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:00:05.355 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:00:05.355 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:00:05.355 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.355 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.355 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop_reply(client.15573.0:25217 10.5 e328/315)", 2024-09-10T19:00:05.355 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:05.295157+0000", 2024-09-10T19:00:05.355 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 4294967295.9997025, 2024-09-10T19:00:05.356 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00059982299999999998, 2024-09-10T19:00:05.356 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:00:05.356 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "queued for pg", 2024-09-10T19:00:05.356 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:00:05.356 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.356 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:00:05.356 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.295157+0000", 2024-09-10T19:00:05.356 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:00:05.356 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:05.357 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.357 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.357 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:00:05.357 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.295157+0000", 2024-09-10T19:00:05.357 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:00:05.357 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.358 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.358 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:00:05.358 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.295156+0000", 2024-09-10T19:00:05.358 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T19:00:05.358 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.358 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.358 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:00:05.358 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.295162+0000", 2024-09-10T19:00:05.358 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.0050000000000001e-06 2024-09-10T19:00:05.358 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.358 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.358 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:00:05.358 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.295172+0000", 2024-09-10T19:00:05.358 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0327e-05 2024-09-10T19:00:05.359 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.359 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.359 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:00:05.359 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.295178+0000", 2024-09-10T19:00:05.359 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.6359999999999999e-06 2024-09-10T19:00:05.359 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:00:05.359 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:00:05.359 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:00:05.359 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.359 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.359 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop_reply(client.15573.0:25219 10.c e328/326)", 2024-09-10T19:00:05.359 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:05.295198+0000", 2024-09-10T19:00:05.359 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 4294967295.999661, 2024-09-10T19:00:05.360 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000607371, 2024-09-10T19:00:05.360 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:00:05.360 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "queued for pg", 2024-09-10T19:00:05.360 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:00:05.360 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.360 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:00:05.360 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.295198+0000", 2024-09-10T19:00:05.360 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:00:05.360 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.360 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.360 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:00:05.360 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.295198+0000", 2024-09-10T19:00:05.360 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:00:05.361 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.361 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.361 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:00:05.361 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.295198+0000", 2024-09-10T19:00:05.361 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T19:00:05.361 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.361 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.361 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:00:05.361 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.295201+0000", 2024-09-10T19:00:05.361 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.6900000000000001e-06 2024-09-10T19:00:05.361 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.361 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.361 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:00:05.362 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.295206+0000", 2024-09-10T19:00:05.362 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.6690000000000003e-06 2024-09-10T19:00:05.362 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.362 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.362 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:00:05.362 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.295210+0000", 2024-09-10T19:00:05.362 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.8430000000000003e-06 2024-09-10T19:00:05.362 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:00:05.362 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:00:05.362 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:00:05.362 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.362 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.362 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.15573.0:25233 10.6 e328/315 10:673e4c05:::benchmark_data_smithi018_157943_object25232:head v 328'1557, mlcod=328'1555)", 2024-09-10T19:00:05.363 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:05.295234+0000", 2024-09-10T19:00:05.363 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 4294967295.9996257, 2024-09-10T19:00:05.363 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00062317899999999996, 2024-09-10T19:00:05.363 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:00:05.363 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "started", 2024-09-10T19:00:05.363 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:00:05.363 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.363 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:00:05.363 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.295234+0000", 2024-09-10T19:00:05.363 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:00:05.363 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.363 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.363 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:00:05.363 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.295234+0000", 2024-09-10T19:00:05.364 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:00:05.364 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.364 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.364 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:00:05.364 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.295233+0000", 2024-09-10T19:00:05.364 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T19:00:05.364 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.364 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.364 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:00:05.364 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.295278+0000", 2024-09-10T19:00:05.364 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.5009999999999998e-05 2024-09-10T19:00:05.364 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.364 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.364 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:00:05.365 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.295286+0000", 2024-09-10T19:00:05.365 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.8730000000000007e-06 2024-09-10T19:00:05.365 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.365 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.365 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:00:05.365 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.295290+0000", 2024-09-10T19:00:05.365 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.1300000000000003e-06 2024-09-10T19:00:05.365 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.365 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.365 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:00:05.365 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.295325+0000", 2024-09-10T19:00:05.365 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.4696999999999997e-05 2024-09-10T19:00:05.365 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.365 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.366 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:00:05.366 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.295342+0000", 2024-09-10T19:00:05.366 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.7125e-05 2024-09-10T19:00:05.366 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:00:05.366 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:00:05.366 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:00:05.366 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.366 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.366 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:25234 10.5 10.480a625 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e328)", 2024-09-10T19:00:05.366 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:05.295308+0000", 2024-09-10T19:00:05.366 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 4294967295.9995513, 2024-09-10T19:00:05.366 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00060377200000000003, 2024-09-10T19:00:05.366 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:00:05.366 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "queued for pg", 2024-09-10T19:00:05.366 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:00:05.367 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:00:05.367 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:00:05.367 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 25234 2024-09-10T19:00:05.367 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.367 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:00:05.367 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.367 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:00:05.367 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.295308+0000", 2024-09-10T19:00:05.367 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:00:05.367 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.367 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.367 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:00:05.367 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.295308+0000", 2024-09-10T19:00:05.367 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:00:05.368 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.368 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.368 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:00:05.368 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.295306+0000", 2024-09-10T19:00:05.368 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T19:00:05.368 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.368 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.368 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:00:05.368 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.295329+0000", 2024-09-10T19:00:05.368 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.2912999999999999e-05 2024-09-10T19:00:05.368 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.368 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.368 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:00:05.368 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.295338+0000", 2024-09-10T19:00:05.369 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.9660000000000002e-06 2024-09-10T19:00:05.369 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:00:05.369 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:00:05.369 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:00:05.369 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:05.295343+0000", 2024-09-10T19:00:05.369 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.1669999999999998e-06 2024-09-10T19:00:05.369 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:00:05.369 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:00:05.369 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:00:05.369 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:00:05.369 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-09-10T19:00:05.369 INFO:teuthology.orchestra.run.smithi018.stdout: "num_ops": 17 2024-09-10T19:00:05.369 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T19:00:05.412 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:05.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:05.413 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:05.514 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:05.552 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:05.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:05.553 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:05.596 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 12 16 25937 25921 134.989 142.188 0.00416389 0.00740242 2024-09-10T19:00:05.645 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:05 smithi018 ceph-mon[40196]: 5.1a scrub starts 2024-09-10T19:00:05.645 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:05 smithi018 ceph-mon[40196]: 5.7 scrub starts 2024-09-10T19:00:05.645 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:05 smithi018 ceph-mon[40196]: 1.10 scrub starts 2024-09-10T19:00:05.645 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:05 smithi018 ceph-mon[40196]: 1.10 scrub ok 2024-09-10T19:00:05.645 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:05 smithi018 ceph-mon[29336]: 5.1a scrub starts 2024-09-10T19:00:05.645 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:05 smithi018 ceph-mon[29336]: 5.7 scrub starts 2024-09-10T19:00:05.645 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:05 smithi018 ceph-mon[29336]: 1.10 scrub starts 2024-09-10T19:00:05.645 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:05 smithi018 ceph-mon[29336]: 1.10 scrub ok 2024-09-10T19:00:05.654 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:05.695 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:05.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:05.696 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:05.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:05 smithi103 ceph-mon[33163]: 5.1a scrub starts 2024-09-10T19:00:05.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:05 smithi103 ceph-mon[33163]: 5.7 scrub starts 2024-09-10T19:00:05.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:05 smithi103 ceph-mon[33163]: 1.10 scrub starts 2024-09-10T19:00:05.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:05 smithi103 ceph-mon[33163]: 1.10 scrub ok 2024-09-10T19:00:05.796 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:05.829 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:05.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:05.830 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:05.931 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:05.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:05.969 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:05.969 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:06.070 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:06.104 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:06.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:06.105 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:06.206 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:06.241 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:06.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:06.242 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:06.343 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:06.380 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:06.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:06.381 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:06.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:06 smithi018 ceph-mon[29336]: pgmap v686: 168 pgs: 1 active+recovering+undersized+remapped, 167 active+clean; 1.1 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 767 B/s rd, 76 MiB/s wr, 1.22k op/s; 450/54651 objects misplaced (0.823%); 3.1 MiB/s, 50 objects/s recovering 2024-09-10T19:00:06.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:06 smithi018 ceph-mon[40196]: pgmap v686: 168 pgs: 1 active+recovering+undersized+remapped, 167 active+clean; 1.1 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 767 B/s rd, 76 MiB/s wr, 1.22k op/s; 450/54651 objects misplaced (0.823%); 3.1 MiB/s, 50 objects/s recovering 2024-09-10T19:00:06.481 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:06.516 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:06.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:06.520 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:06.588 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-10T19:00:06.596 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 13 16 28348 28332 136.193 150.688 0.00797993 0.0073375 2024-09-10T19:00:06.620 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:06.657 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:06.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:06.658 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:06.759 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:06.764 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T19:00:06.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:06 smithi103 ceph-mon[33163]: pgmap v686: 168 pgs: 1 active+recovering+undersized+remapped, 167 active+clean; 1.1 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 767 B/s rd, 76 MiB/s wr, 1.22k op/s; 450/54651 objects misplaced (0.823%); 3.1 MiB/s, 50 objects/s recovering 2024-09-10T19:00:06.816 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:06.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:06.818 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:06.919 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:06.948 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:06.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:06.949 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:07.050 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:07.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:07.088 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:07.089 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:07.189 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:07.228 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:07.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:07.229 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:07.272 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:00:06 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:00:07.330 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:07.374 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:07.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:07.375 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:07.476 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:07.504 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:07.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:07.505 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:07.596 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 14 16 30237 30221 134.897 118.062 0.00185892 0.00730127 2024-09-10T19:00:07.606 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:07.629 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:07.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:07.629 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:07.730 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:07.752 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:07.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:07.753 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:07.854 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:07.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:07.877 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:07.877 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:07.978 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:08.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:08.002 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:08.002 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:08.103 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:08.126 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:08.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:08.127 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:08.227 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:08.251 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:08.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:08.252 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:08.353 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:08.376 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:08.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:08.377 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:08.478 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:08.509 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:08.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:08.510 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:08.596 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 15 16 30510 30494 127.042 17.0625 0.00787087 0.00786617 2024-09-10T19:00:08.611 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:08.649 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:08.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:08.652 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:08.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:08 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:00:08.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:08 smithi018 ceph-mon[40196]: 4.14 scrub starts 2024-09-10T19:00:08.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:08 smithi018 ceph-mon[40196]: 4.14 scrub ok 2024-09-10T19:00:08.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:08 smithi018 ceph-mon[40196]: osdmap e329: 8 total, 8 up, 6 in 2024-09-10T19:00:08.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:08 smithi018 ceph-mon[40196]: pgmap v688: 168 pgs: 1 active+recovering+undersized+remapped, 167 active+clean; 1.6 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 1.3 KiB/s rd, 134 MiB/s wr, 2.14k op/s; 450/79506 objects misplaced (0.566%); 2.7 MiB/s, 43 objects/s recovering 2024-09-10T19:00:08.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:08 smithi018 ceph-mon[40196]: 4.2 scrub starts 2024-09-10T19:00:08.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:08 smithi018 ceph-mon[40196]: 4.2 scrub ok 2024-09-10T19:00:08.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:08 smithi018 ceph-mon[40196]: 6.7 scrub starts 2024-09-10T19:00:08.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:08 smithi018 ceph-mon[40196]: 6.7 scrub ok 2024-09-10T19:00:08.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:08 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:00:08.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:08 smithi018 ceph-mon[29336]: 4.14 scrub starts 2024-09-10T19:00:08.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:08 smithi018 ceph-mon[29336]: 4.14 scrub ok 2024-09-10T19:00:08.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:08 smithi018 ceph-mon[29336]: osdmap e329: 8 total, 8 up, 6 in 2024-09-10T19:00:08.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:08 smithi018 ceph-mon[29336]: pgmap v688: 168 pgs: 1 active+recovering+undersized+remapped, 167 active+clean; 1.6 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 1.3 KiB/s rd, 134 MiB/s wr, 2.14k op/s; 450/79506 objects misplaced (0.566%); 2.7 MiB/s, 43 objects/s recovering 2024-09-10T19:00:08.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:08 smithi018 ceph-mon[29336]: 4.2 scrub starts 2024-09-10T19:00:08.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:08 smithi018 ceph-mon[29336]: 4.2 scrub ok 2024-09-10T19:00:08.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:08 smithi018 ceph-mon[29336]: 6.7 scrub starts 2024-09-10T19:00:08.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:08 smithi018 ceph-mon[29336]: 6.7 scrub ok 2024-09-10T19:00:08.753 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:08.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:08 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:00:08.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:08 smithi103 ceph-mon[33163]: 4.14 scrub starts 2024-09-10T19:00:08.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:08 smithi103 ceph-mon[33163]: 4.14 scrub ok 2024-09-10T19:00:08.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:08 smithi103 ceph-mon[33163]: osdmap e329: 8 total, 8 up, 6 in 2024-09-10T19:00:08.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:08 smithi103 ceph-mon[33163]: pgmap v688: 168 pgs: 1 active+recovering+undersized+remapped, 167 active+clean; 1.6 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 1.3 KiB/s rd, 134 MiB/s wr, 2.14k op/s; 450/79506 objects misplaced (0.566%); 2.7 MiB/s, 43 objects/s recovering 2024-09-10T19:00:08.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:08 smithi103 ceph-mon[33163]: 4.2 scrub starts 2024-09-10T19:00:08.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:08 smithi103 ceph-mon[33163]: 4.2 scrub ok 2024-09-10T19:00:08.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:08 smithi103 ceph-mon[33163]: 6.7 scrub starts 2024-09-10T19:00:08.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:08 smithi103 ceph-mon[33163]: 6.7 scrub ok 2024-09-10T19:00:08.785 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:08.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:08.786 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:08.887 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:08.924 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:08.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:08.926 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:09.026 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:09.057 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:09.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:09.058 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:09.159 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:09.195 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:09.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:09.196 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:09.297 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:09.328 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:09.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:09.329 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:09.430 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:09.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:09.465 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:09.465 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:09.566 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:09.597 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 16 16 32476 32460 126.781 122.875 0.00482232 0.00788247 2024-09-10T19:00:09.602 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:09.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:09.604 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:09.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:09 smithi018 ceph-mon[40196]: 6.e scrub starts 2024-09-10T19:00:09.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:09 smithi018 ceph-mon[40196]: 6.e scrub ok 2024-09-10T19:00:09.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:09 smithi018 ceph-mon[40196]: 2.1 scrub starts 2024-09-10T19:00:09.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:09 smithi018 ceph-mon[40196]: 2.1 scrub ok 2024-09-10T19:00:09.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:09 smithi018 ceph-mon[40196]: osdmap e330: 8 total, 8 up, 6 in 2024-09-10T19:00:09.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:09 smithi018 ceph-mon[40196]: 4.7 scrub starts 2024-09-10T19:00:09.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:09 smithi018 ceph-mon[40196]: 4.7 scrub ok 2024-09-10T19:00:09.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:09 smithi018 ceph-mon[29336]: 6.e scrub starts 2024-09-10T19:00:09.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:09 smithi018 ceph-mon[29336]: 6.e scrub ok 2024-09-10T19:00:09.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:09 smithi018 ceph-mon[29336]: 2.1 scrub starts 2024-09-10T19:00:09.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:09 smithi018 ceph-mon[29336]: 2.1 scrub ok 2024-09-10T19:00:09.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:09 smithi018 ceph-mon[29336]: osdmap e330: 8 total, 8 up, 6 in 2024-09-10T19:00:09.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:09 smithi018 ceph-mon[29336]: 4.7 scrub starts 2024-09-10T19:00:09.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:09 smithi018 ceph-mon[29336]: 4.7 scrub ok 2024-09-10T19:00:09.705 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:09.740 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:09.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:09.741 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:09.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:09 smithi103 ceph-mon[33163]: 6.e scrub starts 2024-09-10T19:00:09.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:09 smithi103 ceph-mon[33163]: 6.e scrub ok 2024-09-10T19:00:09.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:09 smithi103 ceph-mon[33163]: 2.1 scrub starts 2024-09-10T19:00:09.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:09 smithi103 ceph-mon[33163]: 2.1 scrub ok 2024-09-10T19:00:09.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:09 smithi103 ceph-mon[33163]: osdmap e330: 8 total, 8 up, 6 in 2024-09-10T19:00:09.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:09 smithi103 ceph-mon[33163]: 4.7 scrub starts 2024-09-10T19:00:09.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:09 smithi103 ceph-mon[33163]: 4.7 scrub ok 2024-09-10T19:00:09.842 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:09.881 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:09.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:09.883 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:09.984 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:10.022 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:10.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:10.023 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:10.123 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:10.162 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:10.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:10.163 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:10.264 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:10.298 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:10.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:10.299 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:10.400 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:10.429 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:10.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:10.431 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:10.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:00:10 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:00:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:00:10.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:10 smithi103 ceph-mon[33163]: 5.18 scrub starts 2024-09-10T19:00:10.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:10 smithi103 ceph-mon[33163]: 5.18 scrub ok 2024-09-10T19:00:10.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:10 smithi103 ceph-mon[33163]: pgmap v690: 168 pgs: 1 active+recovering+undersized+remapped, 167 active+clean; 1.7 GiB data, 9.1 GiB used, 527 GiB / 536 GiB avail; 853 B/s rd, 129 MiB/s wr, 2.07k op/s; 450/84843 objects misplaced (0.530%); 3.5 MiB/s, 55 objects/s recovering 2024-09-10T19:00:10.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:10 smithi103 ceph-mon[33163]: 5.c scrub starts 2024-09-10T19:00:10.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:10 smithi103 ceph-mon[33163]: 5.c scrub ok 2024-09-10T19:00:10.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:10 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:00:10.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:10 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:00:10.531 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:10.569 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:10.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:10.569 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:10.597 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 17 16 34740 34724 127.646 141.5 0.00409954 0.0078285 2024-09-10T19:00:10.670 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:10.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:10 smithi018 ceph-mon[40196]: 5.18 scrub starts 2024-09-10T19:00:10.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:10 smithi018 ceph-mon[40196]: 5.18 scrub ok 2024-09-10T19:00:10.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:10 smithi018 ceph-mon[40196]: pgmap v690: 168 pgs: 1 active+recovering+undersized+remapped, 167 active+clean; 1.7 GiB data, 9.1 GiB used, 527 GiB / 536 GiB avail; 853 B/s rd, 129 MiB/s wr, 2.07k op/s; 450/84843 objects misplaced (0.530%); 3.5 MiB/s, 55 objects/s recovering 2024-09-10T19:00:10.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:10 smithi018 ceph-mon[40196]: 5.c scrub starts 2024-09-10T19:00:10.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:10 smithi018 ceph-mon[40196]: 5.c scrub ok 2024-09-10T19:00:10.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:10 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:00:10.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:10 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:00:10.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:10 smithi018 ceph-mon[29336]: 5.18 scrub starts 2024-09-10T19:00:10.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:10 smithi018 ceph-mon[29336]: 5.18 scrub ok 2024-09-10T19:00:10.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:10 smithi018 ceph-mon[29336]: pgmap v690: 168 pgs: 1 active+recovering+undersized+remapped, 167 active+clean; 1.7 GiB data, 9.1 GiB used, 527 GiB / 536 GiB avail; 853 B/s rd, 129 MiB/s wr, 2.07k op/s; 450/84843 objects misplaced (0.530%); 3.5 MiB/s, 55 objects/s recovering 2024-09-10T19:00:10.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:10 smithi018 ceph-mon[29336]: 5.c scrub starts 2024-09-10T19:00:10.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:10 smithi018 ceph-mon[29336]: 5.c scrub ok 2024-09-10T19:00:10.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:10 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:00:10.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:10 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:00:10.708 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:10.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:10.710 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:10.810 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:10.845 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:10.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:10.846 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:10.851 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:00:10.947 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:10.987 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:10.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:10.988 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:11.089 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:11.124 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:11.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:11.125 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:11.142 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:00:10 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:00:10] "GET /metrics HTTP/1.1" 200 224606 "" "Prometheus/2.43.0" 2024-09-10T19:00:11.225 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:11.258 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:11.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:11.259 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:11.359 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:11.408 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:11 smithi018 ceph-mon[29336]: 2.b scrub starts 2024-09-10T19:00:11.408 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:11 smithi018 ceph-mon[29336]: 2.b scrub ok 2024-09-10T19:00:11.408 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:11 smithi018 ceph-mon[29336]: 2.19 scrub starts 2024-09-10T19:00:11.408 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:11 smithi018 ceph-mon[29336]: 2.19 scrub ok 2024-09-10T19:00:11.408 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:11 smithi018 ceph-mon[40196]: 2.b scrub starts 2024-09-10T19:00:11.408 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:11 smithi018 ceph-mon[40196]: 2.b scrub ok 2024-09-10T19:00:11.408 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:11 smithi018 ceph-mon[40196]: 2.19 scrub starts 2024-09-10T19:00:11.409 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:11 smithi018 ceph-mon[40196]: 2.19 scrub ok 2024-09-10T19:00:11.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:11.409 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:11.409 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:11.510 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:11.540 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:11.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:11.541 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:11.601 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 18 16 36650 36634 127.155 119.375 0.00568142 0.00785871 2024-09-10T19:00:11.609 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:00:11.642 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:11.688 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:11.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:11.689 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:11.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:11 smithi103 ceph-mon[33163]: 2.b scrub starts 2024-09-10T19:00:11.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:11 smithi103 ceph-mon[33163]: 2.b scrub ok 2024-09-10T19:00:11.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:11 smithi103 ceph-mon[33163]: 2.19 scrub starts 2024-09-10T19:00:11.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:11 smithi103 ceph-mon[33163]: 2.19 scrub ok 2024-09-10T19:00:11.791 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:11.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:11.829 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:11.829 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:11.930 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:11.977 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:11.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:11.978 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:12.079 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:12.117 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:12.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:12.118 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:12.218 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:12.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:12.266 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:12.266 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:12.366 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:12.386 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is set 2024-09-10T19:00:12.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:12.413 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:12.413 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:12.514 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:12.547 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:12.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:12.548 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:12.601 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 19 16 38451 38435 126.386 112.562 0.00722215 0.00790714 2024-09-10T19:00:12.649 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:12.688 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:12.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:12.689 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:12.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:12 smithi018 ceph-mon[40196]: pgmap v691: 168 pgs: 1 active+recovering+undersized+remapped, 167 active+clean; 1.8 GiB data, 9.5 GiB used, 527 GiB / 536 GiB avail; 667 B/s rd, 95 MiB/s wr, 1.52k op/s; 450/89685 objects misplaced (0.502%) 2024-09-10T19:00:12.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:12 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3565544702' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T19:00:12.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:12 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T19:00:12.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:12 smithi018 ceph-mon[29336]: pgmap v691: 168 pgs: 1 active+recovering+undersized+remapped, 167 active+clean; 1.8 GiB data, 9.5 GiB used, 527 GiB / 536 GiB avail; 667 B/s rd, 95 MiB/s wr, 1.52k op/s; 450/89685 objects misplaced (0.502%) 2024-09-10T19:00:12.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:12 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3565544702' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T19:00:12.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:12 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T19:00:12.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:12 smithi103 ceph-mon[33163]: pgmap v691: 168 pgs: 1 active+recovering+undersized+remapped, 167 active+clean; 1.8 GiB data, 9.5 GiB used, 527 GiB / 536 GiB avail; 667 B/s rd, 95 MiB/s wr, 1.52k op/s; 450/89685 objects misplaced (0.502%) 2024-09-10T19:00:12.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:12 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3565544702' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T19:00:12.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:12 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T19:00:12.790 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:12.823 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:12.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:12.825 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:12.926 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:12.966 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:12.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:12.967 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:13.068 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:13.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:13.106 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:13.106 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:13.207 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:13.246 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:13.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:13.247 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:13.347 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:13.388 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:13.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:13.390 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:13.390 INFO:teuthology.orchestra.run.smithi018.stderr:reweighted osd.3 to 1 (10000) 2024-09-10T19:00:13.491 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:13.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:13.529 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:13.529 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:13.601 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:2024-09-10T19:00:13.601150+0000 min lat: 0.00113481 max lat: 2.06434 avg lat: 0.00790885 2024-09-10T19:00:13.601 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T19:00:13.601 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 20 16 40466 40450 126.363 125.938 0.00896991 0.00790885 2024-09-10T19:00:13.630 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:13.661 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:13 smithi018 ceph-mon[40196]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:00:13.661 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:13 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T19:00:13.661 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:13 smithi018 ceph-mon[40196]: osdmap e331: 8 total, 8 up, 6 in 2024-09-10T19:00:13.661 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:13 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/257763258' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-10T19:00:13.661 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:13 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-10T19:00:13.661 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:13 smithi018 ceph-mon[29336]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:00:13.662 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:13 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T19:00:13.662 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:13 smithi018 ceph-mon[29336]: osdmap e331: 8 total, 8 up, 6 in 2024-09-10T19:00:13.662 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:13 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/257763258' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-10T19:00:13.662 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:13 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-10T19:00:13.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:13.667 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:13.667 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:13.768 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:13.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:13 smithi103 ceph-mon[33163]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:00:13.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:13 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T19:00:13.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:13 smithi103 ceph-mon[33163]: osdmap e331: 8 total, 8 up, 6 in 2024-09-10T19:00:13.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:13 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/257763258' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-10T19:00:13.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:13 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-10T19:00:13.798 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:13.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:13.799 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:13.899 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:13.935 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:13.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:13.936 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:14.037 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:14.073 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:14.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:14.075 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:14.175 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:14.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:14.213 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:14.214 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:14.314 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:14.352 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:14.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:14.353 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:14.454 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:14.498 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:14.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:14.499 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:14.600 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:14.601 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 21 16 42541 42525 126.521 129.688 0.00832086 0.00789859 2024-09-10T19:00:14.643 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:14.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:14.645 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:14.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:14 smithi018 ceph-mon[29336]: pgmap v693: 168 pgs: 168 active+clean; 2.2 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 850 B/s rd, 94 MiB/s wr, 1.50k op/s; 4.7 MiB/s, 74 objects/s recovering 2024-09-10T19:00:14.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:14 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-10T19:00:14.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:14 smithi018 ceph-mon[29336]: osdmap e332: 8 total, 8 up, 6 in 2024-09-10T19:00:14.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:14 smithi018 ceph-mon[40196]: pgmap v693: 168 pgs: 168 active+clean; 2.2 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 850 B/s rd, 94 MiB/s wr, 1.50k op/s; 4.7 MiB/s, 74 objects/s recovering 2024-09-10T19:00:14.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:14 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-10T19:00:14.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:14 smithi018 ceph-mon[40196]: osdmap e332: 8 total, 8 up, 6 in 2024-09-10T19:00:14.746 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:14.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:14 smithi103 ceph-mon[33163]: pgmap v693: 168 pgs: 168 active+clean; 2.2 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 850 B/s rd, 94 MiB/s wr, 1.50k op/s; 4.7 MiB/s, 74 objects/s recovering 2024-09-10T19:00:14.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:14 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-10T19:00:14.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:14 smithi103 ceph-mon[33163]: osdmap e332: 8 total, 8 up, 6 in 2024-09-10T19:00:14.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:14.789 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:14.789 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:14.890 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:14.923 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:14.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:14.924 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:15.025 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:15.062 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:15.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:15.063 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:15.164 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:15.197 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:15.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:15.199 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:15.299 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:15.335 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:15.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:15.336 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:15.436 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:15.478 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:15.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:15.479 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:15.579 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:15.601 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 22 16 44567 44551 126.525 126.625 0.00672604 0.00789883 2024-09-10T19:00:15.617 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:15.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:15.618 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:15.719 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:15.757 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:15.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:15.758 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:15.858 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:15.907 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:15.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:15.908 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:16.009 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:16.046 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:16.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:16.047 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:16.148 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:16.182 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:16.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:16.183 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:16.283 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:16.316 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:16.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:16.317 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:16.418 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:16.448 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:16.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:16.449 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:16.456 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd reweight 4 1 2024-09-10T19:00:16.550 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:16.587 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:16.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:16.589 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:16.601 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 23 16 46525 46509 126.344 122.375 0.00872428 0.00791025 2024-09-10T19:00:16.674 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:16 smithi018 ceph-mon[29336]: pgmap v695: 168 pgs: 168 active+clean; 2.3 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 729 B/s rd, 97 MiB/s wr, 1.55k op/s; 4.0 MiB/s, 64 objects/s recovering 2024-09-10T19:00:16.675 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:16 smithi018 ceph-mon[40196]: pgmap v695: 168 pgs: 168 active+clean; 2.3 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 729 B/s rd, 97 MiB/s wr, 1.55k op/s; 4.0 MiB/s, 64 objects/s recovering 2024-09-10T19:00:16.690 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:16.716 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:16.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:16.717 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:16.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:16 smithi103 ceph-mon[33163]: pgmap v695: 168 pgs: 168 active+clean; 2.3 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 729 B/s rd, 97 MiB/s wr, 1.55k op/s; 4.0 MiB/s, 64 objects/s recovering 2024-09-10T19:00:16.818 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:16.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:16.854 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:16.855 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:16.955 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:16.978 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:16.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:16.979 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:17.079 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:17.102 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:17.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:17.103 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:17.172 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T19:00:17.172 INFO:teuthology.orchestra.run.smithi018.stdout: "ops": [], 2024-09-10T19:00:17.172 INFO:teuthology.orchestra.run.smithi018.stdout: "complaint_time": 30, 2024-09-10T19:00:17.172 INFO:teuthology.orchestra.run.smithi018.stdout: "num_blocked_ops": 0 2024-09-10T19:00:17.172 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T19:00:17.203 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:17.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:17.243 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:17.243 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:17.272 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:00:16 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:00:17.344 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:17.373 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:17.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:17.374 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:17.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:17 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:00:17.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:17 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:00:17.456 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T19:00:17.456 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T19:00:17.475 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:17.492 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T19:00:17.492 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T19:00:17.517 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:17.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:17.518 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:17.525 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T19:00:17.525 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T19:00:17.554 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T19:00:17.555 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T19:00:17.587 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T19:00:17.588 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T19:00:17.602 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 24 16 48001 47985 124.918 92.25 0.00865126 0.00800021 2024-09-10T19:00:17.619 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:17.620 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T19:00:17.620 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T19:00:17.659 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T19:00:17.660 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T19:00:17.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:17.672 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:17.672 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:17.691 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T19:00:17.691 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T19:00:17.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:17 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:00:17.773 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:17.808 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:17.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:17.809 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:17.894 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-10T19:00:17.910 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:17.946 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:17.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:17.947 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:18.048 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:18.089 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:18.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:18.090 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:18.191 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:18.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:18.232 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:18.232 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:18.332 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:18.366 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:18.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:18.366 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:18.415 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-10T19:00:18.467 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:18.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:18.511 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:18.511 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:18.603 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 25 16 49687 49671 124.136 105.375 0.00774 0.00805013 2024-09-10T19:00:18.612 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:18.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:18.668 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:18.668 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:18.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:18 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:00:18.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:18 smithi018 ceph-mon[29336]: pgmap v696: 168 pgs: 168 active+clean; 2.8 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 1.2 KiB/s rd, 134 MiB/s wr, 2.15k op/s; 3.5 MiB/s, 56 objects/s recovering 2024-09-10T19:00:18.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:18 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2627687050' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:00:18.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:18 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:00:18.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:18 smithi018 ceph-mon[40196]: pgmap v696: 168 pgs: 168 active+clean; 2.8 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 1.2 KiB/s rd, 134 MiB/s wr, 2.15k op/s; 3.5 MiB/s, 56 objects/s recovering 2024-09-10T19:00:18.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:18 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2627687050' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:00:18.769 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:18.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:18 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:00:18.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:18 smithi103 ceph-mon[33163]: pgmap v696: 168 pgs: 168 active+clean; 2.8 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 1.2 KiB/s rd, 134 MiB/s wr, 2.15k op/s; 3.5 MiB/s, 56 objects/s recovering 2024-09-10T19:00:18.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:18 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2627687050' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:00:18.799 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:18.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:18.800 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:18.884 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:00:18.901 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:18.951 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:00:18.967 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:18.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:18.968 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:19.069 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:19.105 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:19.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:19.106 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:19.206 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:19.247 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:19.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:19.248 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:19.349 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:19.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:19.385 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:19.385 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:19.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:19 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:19.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:19 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:00:19.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:19 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:00:19.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:19 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:19.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:19 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:00:19.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:19 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:00:19.486 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:19.524 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:19.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:19.525 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:19.603 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 26 16 51404 51388 123.489 107.312 0.010931 0.00809271 2024-09-10T19:00:19.626 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:19.663 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:19.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:19.664 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:19.764 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:19.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:19 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:19.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:19 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:00:19.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:19 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:00:19.805 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:19.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:19.807 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:19.908 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:19.942 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:19.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:19.943 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:20.044 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:20.088 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:20.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:20.089 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:20.189 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:20.220 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:20.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:20.221 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:20.322 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:20.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:20.356 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:20.356 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:20.457 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:20.494 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:20.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:20.496 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:20.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:20 smithi103 ceph-mon[33163]: pgmap v697: 168 pgs: 168 active+clean; 2.9 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 1.2 KiB/s rd, 134 MiB/s wr, 2.15k op/s; 3.5 MiB/s, 56 objects/s recovering 2024-09-10T19:00:20.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:00:20 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:00:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:00:20.597 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:20.603 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 27 16 53242 53226 123.17 114.875 0.00827473 0.0081144 2024-09-10T19:00:20.637 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:20.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:20.638 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:20.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:20 smithi018 ceph-mon[29336]: pgmap v697: 168 pgs: 168 active+clean; 2.9 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 1.2 KiB/s rd, 134 MiB/s wr, 2.15k op/s; 3.5 MiB/s, 56 objects/s recovering 2024-09-10T19:00:20.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:20 smithi018 ceph-mon[40196]: pgmap v697: 168 pgs: 168 active+clean; 2.9 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 1.2 KiB/s rd, 134 MiB/s wr, 2.15k op/s; 3.5 MiB/s, 56 objects/s recovering 2024-09-10T19:00:20.697 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:00:20.739 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:20.775 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:20.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:20.776 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:20.877 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:20.914 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:20.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:20.915 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:20.937 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T19:00:21.015 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:21.062 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:21.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:21.064 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:21.165 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:21.200 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:21.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:21.201 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:21.249 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:00:20 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:00:20] "GET /metrics HTTP/1.1" 200 224606 "" "Prometheus/2.43.0" 2024-09-10T19:00:21.302 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:21.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:21.346 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:21.346 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:21.447 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:21.481 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:21.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:21.483 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:21.583 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:21.604 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 28 16 54948 54932 122.576 106.625 0.00743572 0.00815267 2024-09-10T19:00:21.621 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:21.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:21.622 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:21.723 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:21.757 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:21.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:21.758 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:21.859 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:21.893 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:21.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:21.894 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:21.994 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:22.028 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:22.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:22.029 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:22.130 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:22.165 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:22.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:22.167 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:22.267 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:22.292 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:22.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:22.293 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:22.393 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:22.429 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:22.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:22.430 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:22.531 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:22.541 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:22 smithi018 ceph-mon[29336]: pgmap v698: 168 pgs: 168 active+clean; 3.0 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 569 B/s rd, 92 MiB/s wr, 1.47k op/s 2024-09-10T19:00:22.542 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:22 smithi018 ceph-mon[40196]: pgmap v698: 168 pgs: 168 active+clean; 3.0 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 569 B/s rd, 92 MiB/s wr, 1.47k op/s 2024-09-10T19:00:22.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:22.572 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:22.572 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:22.604 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 29 16 56624 56608 121.961 104.75 0.00969602 0.00819476 2024-09-10T19:00:22.672 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:22.695 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:22.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:22.696 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:22.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:22 smithi103 ceph-mon[33163]: pgmap v698: 168 pgs: 168 active+clean; 3.0 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 569 B/s rd, 92 MiB/s wr, 1.47k op/s 2024-09-10T19:00:22.796 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:22.833 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:22.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:22.835 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:22.935 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:22.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:22.966 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:22.966 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:23.067 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:23.100 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:23.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:23.101 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:23.202 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:23.236 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:23.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:23.237 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:23.338 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:23.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:23.380 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:23.380 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:23.480 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:23.515 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:23.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:23.517 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:23.604 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 30 16 58208 58192 121.195 99 0.0109688 0.00824649 2024-09-10T19:00:23.618 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:23.651 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:23.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:23.652 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:23.752 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:23.790 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:23.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:23.791 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:23.892 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:23.921 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:23.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:23.922 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:24.023 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:24.062 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:24.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:24.063 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:24.164 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:24.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:24.197 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:24.198 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:24.298 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:24.334 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:24.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:24.336 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:24.428 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is set 2024-09-10T19:00:24.436 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:24.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:24.480 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:24.480 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:24.581 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:24.604 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 31 16 59664 59648 120.221 91 0.0534304 0.00831221 2024-09-10T19:00:24.615 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:24.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:24.616 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:24.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:24 smithi018 ceph-mon[40196]: pgmap v699: 168 pgs: 168 active+clean; 3.3 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 1023 B/s rd, 115 MiB/s wr, 1.85k op/s 2024-09-10T19:00:24.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:24 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3627305449' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:00:24.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:24 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:00:24.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:24 smithi018 ceph-mon[29336]: pgmap v699: 168 pgs: 168 active+clean; 3.3 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 1023 B/s rd, 115 MiB/s wr, 1.85k op/s 2024-09-10T19:00:24.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:24 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3627305449' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:00:24.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:24 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:00:24.717 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:24.739 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:24.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:24.740 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:24.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:24 smithi103 ceph-mon[33163]: pgmap v699: 168 pgs: 168 active+clean; 3.3 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 1023 B/s rd, 115 MiB/s wr, 1.85k op/s 2024-09-10T19:00:24.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:24 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3627305449' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:00:24.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:24 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:00:24.840 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:24.876 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:24.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:24.877 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:24.978 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:25.013 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:25.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:25.015 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:25.115 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:25.159 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:25.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:25.160 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:25.261 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:25.297 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:25.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:25.298 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:25.399 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:25.420 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:25.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:25.421 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:25.522 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:25.555 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:25.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:25.557 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:25.604 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 32 16 61051 61035 119.173 86.6875 0.00658258 0.00838646 2024-09-10T19:00:25.657 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:25.693 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:25 smithi018 ceph-mon[29336]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:00:25.693 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:25 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T19:00:25.693 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:25 smithi018 ceph-mon[29336]: osdmap e333: 8 total, 8 up, 6 in 2024-09-10T19:00:25.693 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:25 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2378462805' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-10T19:00:25.693 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:25 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:00:25.693 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:25 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:00:25.693 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:25 smithi018 ceph-mon[40196]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:00:25.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:25 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T19:00:25.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:25 smithi018 ceph-mon[40196]: osdmap e333: 8 total, 8 up, 6 in 2024-09-10T19:00:25.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:25 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2378462805' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-10T19:00:25.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:25 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:00:25.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:25 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:00:25.695 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:25.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:25.696 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:25.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:25 smithi103 ceph-mon[33163]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:00:25.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:25 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T19:00:25.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:25 smithi103 ceph-mon[33163]: osdmap e333: 8 total, 8 up, 6 in 2024-09-10T19:00:25.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:25 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2378462805' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-10T19:00:25.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:25 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:00:25.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:25 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:00:25.797 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:25.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:25.832 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:25.833 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:25.873 INFO:teuthology.orchestra.run.smithi018.stderr:reweighted osd.4 to 1 (10000) 2024-09-10T19:00:25.934 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:25.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:25.970 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:25.970 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:26.071 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:26.106 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:26.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:26.107 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:26.208 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:26.243 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:26.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:26.244 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:26.328 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:00:26.344 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:26.377 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:26.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:26.378 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:26.478 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:26.516 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:26.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:26.517 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:26.604 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 33 16 62796 62780 118.867 109.062 0.00962992 0.00840819 2024-09-10T19:00:26.618 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:26.651 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:26.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:26.653 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:26.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:26 smithi018 ceph-mon[29336]: pgmap v701: 168 pgs: 168 active+clean; 3.4 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 1023 B/s rd, 114 MiB/s wr, 1.82k op/s 2024-09-10T19:00:26.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:26 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2378462805' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-10T19:00:26.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:26 smithi018 ceph-mon[29336]: osdmap e334: 8 total, 8 up, 6 in 2024-09-10T19:00:26.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:26 smithi018 ceph-mon[40196]: pgmap v701: 168 pgs: 168 active+clean; 3.4 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 1023 B/s rd, 114 MiB/s wr, 1.82k op/s 2024-09-10T19:00:26.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:26 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2378462805' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-10T19:00:26.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:26 smithi018 ceph-mon[40196]: osdmap e334: 8 total, 8 up, 6 in 2024-09-10T19:00:26.754 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:26.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:26 smithi103 ceph-mon[33163]: pgmap v701: 168 pgs: 168 active+clean; 3.4 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 1023 B/s rd, 114 MiB/s wr, 1.82k op/s 2024-09-10T19:00:26.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:26 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2378462805' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-10T19:00:26.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:26 smithi103 ceph-mon[33163]: osdmap e334: 8 total, 8 up, 6 in 2024-09-10T19:00:26.788 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:26.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:26.789 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:26.890 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:26.940 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:26.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:26.941 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:27.042 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:27.079 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:27.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:27.081 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:27.181 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:27.218 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:27.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:27.220 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:27.272 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:00:26 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:00:27.321 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:27.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:27.360 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:27.360 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:27.461 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:27.499 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:27.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:27.500 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:27.517 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd reweight 5 1 2024-09-10T19:00:27.601 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:27.605 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 34 16 64526 64510 118.545 108.125 0.00980974 0.00843013 2024-09-10T19:00:27.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:27.649 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:27.649 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:27.750 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:27.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:27.773 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:27.773 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:27.874 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:27.913 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:27.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:27.914 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:28.015 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:28.055 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:28.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:28.056 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:28.157 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:28.194 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:28.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:28.195 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:28.295 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:28.332 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:28.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:28.334 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:28.434 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:28.468 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:28.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:28.470 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:28.570 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:28.603 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:28.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:28.604 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:28.606 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 35 16 65954 65938 117.708 89.25 0.00771475 0.00848868 2024-09-10T19:00:28.704 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:28.734 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:28.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:28.735 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:28.744 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:28 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:00:28.744 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:28 smithi018 ceph-mon[29336]: pgmap v703: 168 pgs: 168 active+clean; 3.8 GiB data, 15 GiB used, 521 GiB / 536 GiB avail; 1.2 KiB/s rd, 116 MiB/s wr, 1.86k op/s 2024-09-10T19:00:28.744 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:28 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:00:28.744 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:28 smithi018 ceph-mon[40196]: pgmap v703: 168 pgs: 168 active+clean; 3.8 GiB data, 15 GiB used, 521 GiB / 536 GiB avail; 1.2 KiB/s rd, 116 MiB/s wr, 1.86k op/s 2024-09-10T19:00:28.745 INFO:teuthology.orchestra.run.smithi018.stdout:ERROR: (22) Invalid argument 2024-09-10T19:00:28.745 INFO:teuthology.orchestra.run.smithi018.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T19:00:28.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:28 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:00:28.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:28 smithi103 ceph-mon[33163]: pgmap v703: 168 pgs: 168 active+clean; 3.8 GiB data, 15 GiB used, 521 GiB / 536 GiB avail; 1.2 KiB/s rd, 116 MiB/s wr, 1.86k op/s 2024-09-10T19:00:28.836 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:28.877 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:28.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:28.878 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:28.926 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-10T19:00:28.979 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:29.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:29.016 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:29.016 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:29.117 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:29.163 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:29.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:29.165 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:29.266 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:29.302 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:29.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:29.303 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:29.404 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:29.448 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:29.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:29.449 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:29.550 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:29.571 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:29.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:29.572 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:29.606 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 36 16 67466 67450 117.063 94.5 0.0104013 0.00853789 2024-09-10T19:00:29.673 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:29.709 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:29.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:29.710 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:29.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:29 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:00:29.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:29 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:00:29.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:29 smithi103 ceph-mon[33163]: osdmap e335: 8 total, 8 up, 6 in 2024-09-10T19:00:29.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:29 smithi103 ceph-mon[33163]: osdmap e336: 8 total, 8 up, 6 in 2024-09-10T19:00:29.811 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:29.845 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:29.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:29.847 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:29.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:29 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:00:29.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:29 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:00:29.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:29 smithi018 ceph-mon[40196]: osdmap e335: 8 total, 8 up, 6 in 2024-09-10T19:00:29.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:29 smithi018 ceph-mon[40196]: osdmap e336: 8 total, 8 up, 6 in 2024-09-10T19:00:29.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:29 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:00:29.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:29 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:00:29.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:29 smithi018 ceph-mon[29336]: osdmap e335: 8 total, 8 up, 6 in 2024-09-10T19:00:29.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:29 smithi018 ceph-mon[29336]: osdmap e336: 8 total, 8 up, 6 in 2024-09-10T19:00:29.948 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:29.981 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:29.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:29.982 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:30.083 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:30.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:30.121 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:30.122 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:30.223 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:30.263 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:30.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:30.264 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:30.287 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-10T19:00:30.365 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:30.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:30.436 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:30.436 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:30.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:30 smithi103 ceph-mon[33163]: pgmap v706: 168 pgs: 168 active+clean; 3.9 GiB data, 16 GiB used, 521 GiB / 536 GiB avail; 1.0 KiB/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-10T19:00:30.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:00:30 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:00:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:00:30.537 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:30.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:30.581 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:30.581 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:30.606 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 37 15 69027 69012 116.537 97.625 0.0105244 0.00857653 2024-09-10T19:00:30.683 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:30.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:30 smithi018 ceph-mon[40196]: pgmap v706: 168 pgs: 168 active+clean; 3.9 GiB data, 16 GiB used, 521 GiB / 536 GiB avail; 1.0 KiB/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-10T19:00:30.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:30 smithi018 ceph-mon[29336]: pgmap v706: 168 pgs: 168 active+clean; 3.9 GiB data, 16 GiB used, 521 GiB / 536 GiB avail; 1.0 KiB/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-10T19:00:30.717 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:30.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:30.718 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:30.819 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:30.855 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:30.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:30.856 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:30.957 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:30.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:30.994 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:30.994 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:31.095 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:31.137 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:31.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:31.138 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:31.238 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:31.275 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:31.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:31.276 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:31.306 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:00:30 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:00:30] "GET /metrics HTTP/1.1" 200 224617 "" "Prometheus/2.43.0" 2024-09-10T19:00:31.377 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:31.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:31.423 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:31.423 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:31.524 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:31.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:31.561 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:31.561 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:31.562 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:31 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3935339854' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:00:31.563 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:31 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3935339854' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:00:31.606 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 38 16 70576 70560 116.016 96.75 0.00649813 0.00861366 2024-09-10T19:00:31.662 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:31.695 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:31.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:31.696 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:31.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:31 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3935339854' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:00:31.797 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:31.835 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:31.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:31.836 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:31.936 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:31.975 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:31.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:31.977 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:32.078 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:32.087 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T19:00:32.087 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T19:00:32.087 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T19:00:32.087 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T19:00:32.087 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T19:00:32.088 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T19:00:32.088 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T19:00:32.088 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T19:00:32.088 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T19:00:32.088 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T19:00:32.088 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T19:00:32.088 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T19:00:32.088 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T19:00:32.088 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T19:00:32.088 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T19:00:32.088 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T19:00:32.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:32.113 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:32.113 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:32.214 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:32.246 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:32.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:32.247 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:32.348 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:32.374 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:32.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:32.375 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:32.476 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:32.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:32.511 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:32.511 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:32.607 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 39 16 72247 72231 115.719 104.438 0.00877677 0.00863685 2024-09-10T19:00:32.612 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:32.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:32.649 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:32.649 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:32.750 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:32.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:32 smithi103 ceph-mon[33163]: pgmap v707: 168 pgs: 168 active+clean; 4.0 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 853 B/s rd, 97 MiB/s wr, 1.56k op/s 2024-09-10T19:00:32.785 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:32.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:32.787 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:32.887 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:32.918 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:32.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:32.919 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:32.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:32 smithi018 ceph-mon[40196]: pgmap v707: 168 pgs: 168 active+clean; 4.0 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 853 B/s rd, 97 MiB/s wr, 1.56k op/s 2024-09-10T19:00:32.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:32 smithi018 ceph-mon[29336]: pgmap v707: 168 pgs: 168 active+clean; 4.0 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 853 B/s rd, 97 MiB/s wr, 1.56k op/s 2024-09-10T19:00:33.020 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:33.053 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:33.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:33.054 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:33.155 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:33.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:33.191 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:33.191 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:33.292 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:33.329 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:33.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:33.330 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:33.431 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:33.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:33.466 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:33.467 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:33.567 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:33.600 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:33.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:33.601 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:33.606 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:2024-09-10T19:00:33.606523+0000 min lat: 0.00113481 max lat: 2.06434 avg lat: 0.00865178 2024-09-10T19:00:33.606 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T19:00:33.607 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 40 16 73973 73957 115.523 107.875 0.00939213 0.00865178 2024-09-10T19:00:33.702 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:33.738 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:33.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:33.740 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:33.840 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:33.876 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:33.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:33.877 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:33.977 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:34.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:34.017 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:34.018 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:34.118 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:34.155 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:34.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:34.156 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:34.238 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:00:34.258 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:34.286 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:34.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:34.287 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:34.318 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:00:34.330 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:00:34.387 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:34.438 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:34.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:34.440 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:34.540 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:34.572 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:34.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:34.573 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:34.607 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 41 16 75613 75597 115.203 102.5 0.00644754 0.00867572 2024-09-10T19:00:34.675 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:34.710 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:34.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:34.711 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:34.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:34 smithi103 ceph-mon[33163]: pgmap v708: 168 pgs: 168 active+clean; 4.2 GiB data, 17 GiB used, 520 GiB / 536 GiB avail; 1.3 KiB/s rd, 112 MiB/s wr, 1.79k op/s 2024-09-10T19:00:34.811 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:34.820 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:34 smithi018 ceph-mon[40196]: pgmap v708: 168 pgs: 168 active+clean; 4.2 GiB data, 17 GiB used, 520 GiB / 536 GiB avail; 1.3 KiB/s rd, 112 MiB/s wr, 1.79k op/s 2024-09-10T19:00:34.821 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:34 smithi018 ceph-mon[29336]: pgmap v708: 168 pgs: 168 active+clean; 4.2 GiB data, 17 GiB used, 520 GiB / 536 GiB avail; 1.3 KiB/s rd, 112 MiB/s wr, 1.79k op/s 2024-09-10T19:00:34.842 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:34.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:34.843 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:34.944 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:34.982 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:34.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:34.983 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:35.084 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:35.131 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:35.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:35.132 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:35.233 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:35.264 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:35.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:35.265 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:35.366 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:35.401 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:35.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:35.402 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:35.502 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:35.535 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:35.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:35.536 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:35.607 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 42 15 77340 77325 115.031 108 0.0049358 0.00868927 2024-09-10T19:00:35.637 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:35.673 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:35.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:35.674 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:35.775 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:35.814 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:35.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:35.815 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:35.915 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:35.950 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:35.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:35.951 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:36.052 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:36.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:36.092 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:36.092 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:36.193 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:36.230 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:36.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:36.231 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:36.272 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T19:00:36.332 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:36.366 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:36.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:36.367 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:36.467 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:36.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:36.509 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:36.509 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:36.583 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:36 smithi018 ceph-mon[29336]: pgmap v709: 168 pgs: 168 active+clean; 4.3 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 639 B/s rd, 73 MiB/s wr, 1.16k op/s 2024-09-10T19:00:36.584 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:36 smithi018 ceph-mon[40196]: pgmap v709: 168 pgs: 168 active+clean; 4.3 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 639 B/s rd, 73 MiB/s wr, 1.16k op/s 2024-09-10T19:00:36.608 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 43 16 79011 78995 114.783 104.375 0.00547039 0.00870748 2024-09-10T19:00:36.610 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:36.662 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:36.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:36.663 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:36.764 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:36.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:36 smithi103 ceph-mon[33163]: pgmap v709: 168 pgs: 168 active+clean; 4.3 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 639 B/s rd, 73 MiB/s wr, 1.16k op/s 2024-09-10T19:00:36.800 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:36.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:36.801 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:36.902 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:36.934 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:36.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:36.936 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:37.036 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:37.074 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:37.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:37.075 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:37.176 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:37.203 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:37.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:37.204 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:37.272 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:00:36 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:00:37.304 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:37.344 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:37.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:37.345 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:37.446 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:37.482 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:37.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:37.483 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:37.584 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:37.608 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 44 16 80537 80521 114.341 95.375 0.0136919 0.00874096 2024-09-10T19:00:37.627 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:37.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:37.628 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:37.729 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:37.765 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:37.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:37.766 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:37.866 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:37.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:37.905 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:37.905 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:38.006 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:38.039 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:38.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:38.041 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:38.141 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:38.171 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:38.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:38.172 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:38.273 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:38.294 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:38.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:38.295 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:38.396 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:38.432 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:38.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:38.433 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:38.534 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:38.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:38.575 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:38.575 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:38.608 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 45 15 82018 82003 113.858 92.625 0.00788169 0.00877867 2024-09-10T19:00:38.676 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:38.706 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:38 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:00:38.706 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:38 smithi018 ceph-mon[29336]: pgmap v710: 168 pgs: 168 active+clean; 4.8 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 1.1 KiB/s rd, 113 MiB/s wr, 1.81k op/s 2024-09-10T19:00:38.707 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:38 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:00:38.707 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:38 smithi018 ceph-mon[40196]: pgmap v710: 168 pgs: 168 active+clean; 4.8 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 1.1 KiB/s rd, 113 MiB/s wr, 1.81k op/s 2024-09-10T19:00:38.713 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:38.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:38.714 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:38.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:38 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:00:38.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:38 smithi103 ceph-mon[33163]: pgmap v710: 168 pgs: 168 active+clean; 4.8 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 1.1 KiB/s rd, 113 MiB/s wr, 1.81k op/s 2024-09-10T19:00:38.815 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:38.847 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:38.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:38.848 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:38.949 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:39.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:39.009 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:39.010 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:39.110 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:39.137 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:39.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:39.138 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:39.239 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:39.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:39.272 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:39.273 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:39.373 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:39.415 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:39.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:39.416 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:39.517 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:39.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:39.559 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:39.559 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:39.608 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 46 16 83474 83458 113.36 90.9375 0.00568656 0.0088153 2024-09-10T19:00:39.660 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:39.681 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:39.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:39.682 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:39.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:39 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/610793089' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:00:39.783 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:39.821 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:39.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:39.822 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:39.911 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is unset 2024-09-10T19:00:39.912 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:39 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/610793089' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:00:39.912 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:39 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/610793089' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:00:39.924 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:39.960 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:39.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:39.962 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:40.063 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:40.096 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:40.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:40.097 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:40.198 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:40.237 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:40.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:40.238 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:40.328 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:00:40.339 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:40.388 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:40.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:40.389 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:40.490 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:40.492 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:00:40 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:00:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:00:40.530 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:40.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:40.531 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:40.608 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 47 16 84912 84896 112.86 89.875 0.018387 0.00885527 2024-09-10T19:00:40.632 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:40.675 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:40.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:40.676 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:40.693 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:40 smithi018 ceph-mon[29336]: pgmap v711: 168 pgs: 168 active+clean; 4.8 GiB data, 19 GiB used, 517 GiB / 536 GiB avail; 960 B/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-10T19:00:40.693 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:40 smithi018 ceph-mon[29336]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:00:40.693 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:40 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/610793089' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T19:00:40.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:40 smithi018 ceph-mon[29336]: osdmap e337: 8 total, 8 up, 6 in 2024-09-10T19:00:40.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:40 smithi018 ceph-mon[29336]: 5.7 scrub starts 2024-09-10T19:00:40.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:40 smithi018 ceph-mon[29336]: 4.6 scrub starts 2024-09-10T19:00:40.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:40 smithi018 ceph-mon[29336]: 4.6 scrub ok 2024-09-10T19:00:40.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:40 smithi018 ceph-mon[29336]: 5.7 scrub ok 2024-09-10T19:00:40.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:40 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:00:40.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:40 smithi018 ceph-mon[40196]: pgmap v711: 168 pgs: 168 active+clean; 4.8 GiB data, 19 GiB used, 517 GiB / 536 GiB avail; 960 B/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-10T19:00:40.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:40 smithi018 ceph-mon[40196]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:00:40.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:40 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/610793089' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T19:00:40.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:40 smithi018 ceph-mon[40196]: osdmap e337: 8 total, 8 up, 6 in 2024-09-10T19:00:40.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:40 smithi018 ceph-mon[40196]: 5.7 scrub starts 2024-09-10T19:00:40.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:40 smithi018 ceph-mon[40196]: 4.6 scrub starts 2024-09-10T19:00:40.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:40 smithi018 ceph-mon[40196]: 4.6 scrub ok 2024-09-10T19:00:40.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:40 smithi018 ceph-mon[40196]: 5.7 scrub ok 2024-09-10T19:00:40.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:40 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:00:40.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:40 smithi103 ceph-mon[33163]: pgmap v711: 168 pgs: 168 active+clean; 4.8 GiB data, 19 GiB used, 517 GiB / 536 GiB avail; 960 B/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-10T19:00:40.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:40 smithi103 ceph-mon[33163]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:00:40.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:40 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/610793089' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T19:00:40.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:40 smithi103 ceph-mon[33163]: osdmap e337: 8 total, 8 up, 6 in 2024-09-10T19:00:40.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:40 smithi103 ceph-mon[33163]: 5.7 scrub starts 2024-09-10T19:00:40.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:40 smithi103 ceph-mon[33163]: 4.6 scrub starts 2024-09-10T19:00:40.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:40 smithi103 ceph-mon[33163]: 4.6 scrub ok 2024-09-10T19:00:40.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:40 smithi103 ceph-mon[33163]: 5.7 scrub ok 2024-09-10T19:00:40.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:40 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:00:40.777 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:40.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:40.818 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:40.819 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:40.920 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:40.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:40.959 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:40.959 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:41.060 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:41.095 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:41.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:41.096 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:41.197 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:41.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:41.239 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:41.240 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:41.340 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:41.379 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:41.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:41.381 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:41.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:00:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:00:40] "GET /metrics HTTP/1.1" 200 224660 "" "Prometheus/2.43.0" 2024-09-10T19:00:41.481 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:41.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:41.518 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:41.518 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:41.519 INFO:teuthology.orchestra.run.smithi018.stderr:reweighted osd.5 to 1 (10000) 2024-09-10T19:00:41.609 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 48 16 86445 86429 112.504 95.8125 0.00275703 0.00887079 2024-09-10T19:00:41.618 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:41.640 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:41.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:41.641 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:41.742 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:41.763 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:41.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:41.764 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:41.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:41 smithi103 ceph-mon[33163]: 1.1 scrub starts 2024-09-10T19:00:41.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:41 smithi103 ceph-mon[33163]: 1.1 scrub ok 2024-09-10T19:00:41.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:41 smithi103 ceph-mon[33163]: 3.0 scrub starts 2024-09-10T19:00:41.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:41 smithi103 ceph-mon[33163]: 3.0 scrub ok 2024-09-10T19:00:41.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:41 smithi103 ceph-mon[33163]: 4.b scrub starts 2024-09-10T19:00:41.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:41 smithi103 ceph-mon[33163]: 4.b scrub ok 2024-09-10T19:00:41.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:41 smithi103 ceph-mon[33163]: 5.1a scrub starts 2024-09-10T19:00:41.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:41 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2183217013' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-10T19:00:41.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:41 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-10T19:00:41.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:41 smithi103 ceph-mon[33163]: 5.1a scrub ok 2024-09-10T19:00:41.865 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:41.886 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:41.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:41.887 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:41.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:41 smithi018 ceph-mon[40196]: 1.1 scrub starts 2024-09-10T19:00:41.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:41 smithi018 ceph-mon[40196]: 1.1 scrub ok 2024-09-10T19:00:41.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:41 smithi018 ceph-mon[40196]: 3.0 scrub starts 2024-09-10T19:00:41.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:41 smithi018 ceph-mon[40196]: 3.0 scrub ok 2024-09-10T19:00:41.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:41 smithi018 ceph-mon[40196]: 4.b scrub starts 2024-09-10T19:00:41.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:41 smithi018 ceph-mon[40196]: 4.b scrub ok 2024-09-10T19:00:41.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:41 smithi018 ceph-mon[40196]: 5.1a scrub starts 2024-09-10T19:00:41.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:41 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2183217013' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-10T19:00:41.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:41 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-10T19:00:41.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:41 smithi018 ceph-mon[40196]: 5.1a scrub ok 2024-09-10T19:00:41.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:41 smithi018 ceph-mon[29336]: 1.1 scrub starts 2024-09-10T19:00:41.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:41 smithi018 ceph-mon[29336]: 1.1 scrub ok 2024-09-10T19:00:41.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:41 smithi018 ceph-mon[29336]: 3.0 scrub starts 2024-09-10T19:00:41.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:41 smithi018 ceph-mon[29336]: 3.0 scrub ok 2024-09-10T19:00:41.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:41 smithi018 ceph-mon[29336]: 4.b scrub starts 2024-09-10T19:00:41.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:41 smithi018 ceph-mon[29336]: 4.b scrub ok 2024-09-10T19:00:41.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:41 smithi018 ceph-mon[29336]: 5.1a scrub starts 2024-09-10T19:00:41.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:41 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2183217013' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-10T19:00:41.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:41 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-10T19:00:41.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:41 smithi018 ceph-mon[29336]: 5.1a scrub ok 2024-09-10T19:00:41.988 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:42.009 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:42.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:42.010 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:42.110 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:42.132 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:42.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:42.133 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:42.233 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:42.258 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:42.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:42.259 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:42.360 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:42.383 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:42.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:42.384 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:42.447 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T19:00:42.448 INFO:teuthology.orchestra.run.smithi018.stdout: "ops": [], 2024-09-10T19:00:42.448 INFO:teuthology.orchestra.run.smithi018.stdout: "num_ops": 0 2024-09-10T19:00:42.448 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T19:00:42.485 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:42.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:42.511 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:42.512 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:42.512 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:42 smithi018 ceph-mon[29336]: pgmap v713: 168 pgs: 168 active+clean; 4.9 GiB data, 19 GiB used, 517 GiB / 536 GiB avail; 1023 B/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-10T19:00:42.513 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:42 smithi018 ceph-mon[29336]: 4.f scrub starts 2024-09-10T19:00:42.513 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:42 smithi018 ceph-mon[29336]: 4.f scrub ok 2024-09-10T19:00:42.513 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:42 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-10T19:00:42.513 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:42 smithi018 ceph-mon[29336]: osdmap e338: 8 total, 8 up, 6 in 2024-09-10T19:00:42.513 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:42 smithi018 ceph-mon[29336]: 6.5 scrub starts 2024-09-10T19:00:42.513 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:42 smithi018 ceph-mon[29336]: 6.5 scrub ok 2024-09-10T19:00:42.609 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 49 16 86445 86429 110.209 0 - 0.00887079 2024-09-10T19:00:42.612 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:42.633 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:42.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:42.634 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:42.735 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:42.757 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:42.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:42.758 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:42.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:42 smithi103 ceph-mon[33163]: pgmap v713: 168 pgs: 168 active+clean; 4.9 GiB data, 19 GiB used, 517 GiB / 536 GiB avail; 1023 B/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-10T19:00:42.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:42 smithi103 ceph-mon[33163]: 4.f scrub starts 2024-09-10T19:00:42.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:42 smithi103 ceph-mon[33163]: 4.f scrub ok 2024-09-10T19:00:42.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:42 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-10T19:00:42.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:42 smithi103 ceph-mon[33163]: osdmap e338: 8 total, 8 up, 6 in 2024-09-10T19:00:42.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:42 smithi103 ceph-mon[33163]: 6.5 scrub starts 2024-09-10T19:00:42.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:42 smithi103 ceph-mon[33163]: 6.5 scrub ok 2024-09-10T19:00:42.858 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:42.879 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:42.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:42.880 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:42.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:42 smithi018 ceph-mon[40196]: pgmap v713: 168 pgs: 168 active+clean; 4.9 GiB data, 19 GiB used, 517 GiB / 536 GiB avail; 1023 B/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-10T19:00:42.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:42 smithi018 ceph-mon[40196]: 4.f scrub starts 2024-09-10T19:00:42.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:42 smithi018 ceph-mon[40196]: 4.f scrub ok 2024-09-10T19:00:42.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:42 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-10T19:00:42.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:42 smithi018 ceph-mon[40196]: osdmap e338: 8 total, 8 up, 6 in 2024-09-10T19:00:42.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:42 smithi018 ceph-mon[40196]: 6.5 scrub starts 2024-09-10T19:00:42.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:42 smithi018 ceph-mon[40196]: 6.5 scrub ok 2024-09-10T19:00:42.981 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:43.005 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:43.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:43.006 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:43.107 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:43.128 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:43.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:43.129 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:43.230 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:43.238 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-10T19:00:43.251 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:43.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:43.252 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:43.353 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:43.374 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:43.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:43.375 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:43.476 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:43.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:43.498 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:43.498 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:43.598 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:43.609 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 50 16 86445 86429 108.005 0 - 0.00887079 2024-09-10T19:00:43.622 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:43.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:43.623 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:43.647 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:43 smithi018 ceph-mon[29336]: osdmap e339: 8 total, 8 up, 6 in 2024-09-10T19:00:43.647 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:43 smithi018 ceph-mon[29336]: 5.b scrub starts 2024-09-10T19:00:43.647 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:43 smithi018 ceph-mon[29336]: 5.b scrub ok 2024-09-10T19:00:43.647 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:43 smithi018 ceph-mon[29336]: osdmap e340: 8 total, 8 up, 6 in 2024-09-10T19:00:43.648 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:43 smithi018 ceph-mon[40196]: osdmap e339: 8 total, 8 up, 6 in 2024-09-10T19:00:43.648 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:43 smithi018 ceph-mon[40196]: 5.b scrub starts 2024-09-10T19:00:43.648 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:43 smithi018 ceph-mon[40196]: 5.b scrub ok 2024-09-10T19:00:43.648 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:43 smithi018 ceph-mon[40196]: osdmap e340: 8 total, 8 up, 6 in 2024-09-10T19:00:43.723 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:43.745 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:43.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:43.746 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:43.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:43 smithi103 ceph-mon[33163]: osdmap e339: 8 total, 8 up, 6 in 2024-09-10T19:00:43.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:43 smithi103 ceph-mon[33163]: 5.b scrub starts 2024-09-10T19:00:43.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:43 smithi103 ceph-mon[33163]: 5.b scrub ok 2024-09-10T19:00:43.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:43 smithi103 ceph-mon[33163]: osdmap e340: 8 total, 8 up, 6 in 2024-09-10T19:00:43.847 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:43.885 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:43.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:43.886 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:43.973 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-10T19:00:43.986 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:44.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:44.026 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:44.026 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:44.127 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:44.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:44.177 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:44.177 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:44.278 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:44.316 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:44.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:44.317 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:44.418 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:44.455 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:44.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:44.456 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:44.557 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:44.592 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:44.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:44.593 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:44.609 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 51 16 87545 87529 107.235 22.9167 0.00945227 0.00932036 2024-09-10T19:00:44.694 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:44.721 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:44.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:44.721 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:44.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:44 smithi103 ceph-mon[33163]: pgmap v716: 168 pgs: 5 peering, 163 active+clean; 5.2 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 853 B/s rd, 79 MiB/s wr, 1.26k op/s 2024-09-10T19:00:44.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:44 smithi103 ceph-mon[33163]: 10.5 starting backfill to osd.5 from (0'0,0'0] MAX to 337'5337 2024-09-10T19:00:44.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:44 smithi103 ceph-mon[33163]: 10.7 starting backfill to osd.5 from (0'0,0'0] MAX to 337'5418 2024-09-10T19:00:44.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:44 smithi103 ceph-mon[33163]: 10.9 starting backfill to osd.5 from (0'0,0'0] MAX to 337'5402 2024-09-10T19:00:44.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:44 smithi103 ceph-mon[33163]: 4.a scrub starts 2024-09-10T19:00:44.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:44 smithi103 ceph-mon[33163]: 4.a scrub ok 2024-09-10T19:00:44.822 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:44.863 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:44.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:44.864 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:44.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:44 smithi018 ceph-mon[29336]: pgmap v716: 168 pgs: 5 peering, 163 active+clean; 5.2 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 853 B/s rd, 79 MiB/s wr, 1.26k op/s 2024-09-10T19:00:44.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:44 smithi018 ceph-mon[29336]: 10.5 starting backfill to osd.5 from (0'0,0'0] MAX to 337'5337 2024-09-10T19:00:44.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:44 smithi018 ceph-mon[29336]: 10.7 starting backfill to osd.5 from (0'0,0'0] MAX to 337'5418 2024-09-10T19:00:44.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:44 smithi018 ceph-mon[29336]: 10.9 starting backfill to osd.5 from (0'0,0'0] MAX to 337'5402 2024-09-10T19:00:44.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:44 smithi018 ceph-mon[29336]: 4.a scrub starts 2024-09-10T19:00:44.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:44 smithi018 ceph-mon[29336]: 4.a scrub ok 2024-09-10T19:00:44.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:44 smithi018 ceph-mon[40196]: pgmap v716: 168 pgs: 5 peering, 163 active+clean; 5.2 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 853 B/s rd, 79 MiB/s wr, 1.26k op/s 2024-09-10T19:00:44.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:44 smithi018 ceph-mon[40196]: 10.5 starting backfill to osd.5 from (0'0,0'0] MAX to 337'5337 2024-09-10T19:00:44.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:44 smithi018 ceph-mon[40196]: 10.7 starting backfill to osd.5 from (0'0,0'0] MAX to 337'5418 2024-09-10T19:00:44.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:44 smithi018 ceph-mon[40196]: 10.9 starting backfill to osd.5 from (0'0,0'0] MAX to 337'5402 2024-09-10T19:00:44.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:44 smithi018 ceph-mon[40196]: 4.a scrub starts 2024-09-10T19:00:44.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:44 smithi018 ceph-mon[40196]: 4.a scrub ok 2024-09-10T19:00:44.965 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:45.002 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:45.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:45.003 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:45.103 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:45.139 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:45.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:45.140 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:45.241 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:45.278 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:45.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:45.279 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:45.380 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:45.422 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:45.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:45.423 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:45.510 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd reweight 6 1 2024-09-10T19:00:45.523 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:45.610 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 52 16 89029 89013 106.957 92.75 0.0136086 0.00934489 2024-09-10T19:00:45.613 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:45.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:45.614 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:45.715 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:45.751 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:45.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:45.752 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:45.852 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:45.884 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:45.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:45.885 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:45.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:45 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:00:45.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:45 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:00:45.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:45 smithi018 ceph-mon[29336]: osdmap e341: 8 total, 8 up, 6 in 2024-09-10T19:00:45.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:45 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:00:45.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:45 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:00:45.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:45 smithi018 ceph-mon[40196]: osdmap e341: 8 total, 8 up, 6 in 2024-09-10T19:00:45.986 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:46.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:45 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:00:46.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:45 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:00:46.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:45 smithi103 ceph-mon[33163]: osdmap e341: 8 total, 8 up, 6 in 2024-09-10T19:00:46.040 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:46.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:46.041 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:46.076 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:00:46.142 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:46.170 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:46.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:46.171 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:46.271 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:46.305 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:46.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:46.306 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:46.407 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:46.440 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:46.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:46.441 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:46.542 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:46.582 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:46.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:46.584 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:46.609 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 53 16 90462 90446 106.628 89.5625 0.00447309 0.0093739 2024-09-10T19:00:46.685 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:46.709 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:46.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:46.710 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:46.811 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:46.848 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:46.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:46.849 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:46.852 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:46 smithi018 ceph-mon[40196]: pgmap v719: 168 pgs: 1 activating+remapped, 5 peering, 162 active+clean; 5.3 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 1.2 KiB/s rd, 87 MiB/s wr, 1.39k op/s; 5337/259560 objects misplaced (2.056%); 0 B/s, 1 objects/s recovering 2024-09-10T19:00:46.852 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:46 smithi018 ceph-mon[29336]: pgmap v719: 168 pgs: 1 activating+remapped, 5 peering, 162 active+clean; 5.3 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 1.2 KiB/s rd, 87 MiB/s wr, 1.39k op/s; 5337/259560 objects misplaced (2.056%); 0 B/s, 1 objects/s recovering 2024-09-10T19:00:46.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:46 smithi103 ceph-mon[33163]: pgmap v719: 168 pgs: 1 activating+remapped, 5 peering, 162 active+clean; 5.3 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 1.2 KiB/s rd, 87 MiB/s wr, 1.39k op/s; 5337/259560 objects misplaced (2.056%); 0 B/s, 1 objects/s recovering 2024-09-10T19:00:46.950 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:46.986 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:46.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:46.987 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:47.086 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:00:47.088 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:00:47.088 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:47.127 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:47.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:47.128 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:47.229 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:47.268 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:47.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:47.269 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:47.272 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:00:46 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:00:47.370 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:47.408 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:47.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:47.409 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:47.490 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T19:00:47.490 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T19:00:47.510 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:47.523 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T19:00:47.523 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T19:00:47.546 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:47.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:47.548 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:47.558 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T19:00:47.559 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T19:00:47.596 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T19:00:47.596 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T19:00:47.610 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 54 16 91849 91833 106.259 86.6875 0.00934743 0.00940659 2024-09-10T19:00:47.628 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T19:00:47.628 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T19:00:47.648 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:47.658 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T19:00:47.658 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T19:00:47.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:47.693 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:47.693 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:47.697 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T19:00:47.698 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T19:00:47.735 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T19:00:47.735 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T19:00:47.749 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:47 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2011312003' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:00:47.749 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:47 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2011312003' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:00:47.794 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:47.832 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:47.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:47.833 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:47.934 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:47.968 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:47.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:47.969 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:48.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:47 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2011312003' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:00:48.070 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:48.106 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:48.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:48.107 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:48.207 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:48.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:48.242 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:48.242 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:48.343 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:48.377 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:48.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:48.378 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:48.478 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:48.515 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:48.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:48.516 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:48.609 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 55 16 93247 93231 105.916 87.375 0.0149667 0.00943624 2024-09-10T19:00:48.617 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:48.653 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:48.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:48.654 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:48.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:48 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:00:48.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:48 smithi018 ceph-mon[29336]: pgmap v720: 168 pgs: 2 active+remapped+backfilling, 1 activating+remapped, 165 active+clean; 5.5 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 1.7 KiB/s rd, 54 MiB/s wr, 861 op/s; 15693/269490 objects misplaced (5.823%); 10 MiB/s, 168 objects/s recovering 2024-09-10T19:00:48.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:48 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:00:48.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:48 smithi018 ceph-mon[40196]: pgmap v720: 168 pgs: 2 active+remapped+backfilling, 1 activating+remapped, 165 active+clean; 5.5 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 1.7 KiB/s rd, 54 MiB/s wr, 861 op/s; 15693/269490 objects misplaced (5.823%); 10 MiB/s, 168 objects/s recovering 2024-09-10T19:00:48.755 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:48.792 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:48.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:48.793 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:48.893 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:48.934 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:48.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:48.935 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:49.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:48 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:00:49.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:48 smithi103 ceph-mon[33163]: pgmap v720: 168 pgs: 2 active+remapped+backfilling, 1 activating+remapped, 165 active+clean; 5.5 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 1.7 KiB/s rd, 54 MiB/s wr, 861 op/s; 15693/269490 objects misplaced (5.823%); 10 MiB/s, 168 objects/s recovering 2024-09-10T19:00:49.037 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:49.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:49.078 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:49.078 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:49.179 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:49.213 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:49.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:49.214 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:49.315 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:49.351 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:49.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:49.352 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:49.453 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:49.488 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:49.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:49.489 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:49.590 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:49.609 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 56 16 94597 94581 105.531 84.375 0.012547 0.00947092 2024-09-10T19:00:49.629 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:49.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:49.631 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:49.731 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:49.767 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:49.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:49.768 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:49.869 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:49.881 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:49 smithi103 ceph-mon[33163]: 5.19 scrub starts 2024-09-10T19:00:49.881 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:49 smithi103 ceph-mon[33163]: osdmap e342: 8 total, 8 up, 6 in 2024-09-10T19:00:49.909 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:49.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:49.911 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:49.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:49 smithi018 ceph-mon[40196]: 5.19 scrub starts 2024-09-10T19:00:49.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:49 smithi018 ceph-mon[40196]: osdmap e342: 8 total, 8 up, 6 in 2024-09-10T19:00:49.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:49 smithi018 ceph-mon[29336]: 5.19 scrub starts 2024-09-10T19:00:49.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:49 smithi018 ceph-mon[29336]: osdmap e342: 8 total, 8 up, 6 in 2024-09-10T19:00:50.012 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:50.048 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:50.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:50.049 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:50.150 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:50.190 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:50.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:50.191 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:50.292 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:50.343 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:50.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:50.344 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:50.445 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:50.484 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:50.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:50.486 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:50.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:00:50 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:00:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:00:50.587 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:50.610 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 57 16 95923 95907 105.133 82.875 0.00549735 0.00950346 2024-09-10T19:00:50.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:50.611 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:50.611 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:50.712 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:50.748 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:50.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:50.749 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:50.850 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:50.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:50.893 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:50.894 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:50.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:50 smithi018 ceph-mon[29336]: pgmap v722: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 5.5 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 853 B/s rd, 56 MiB/s wr, 897 op/s; 15443/273276 objects misplaced (5.651%); 17 MiB/s, 272 objects/s recovering 2024-09-10T19:00:50.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:50 smithi018 ceph-mon[29336]: 1.13 scrub starts 2024-09-10T19:00:50.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:50 smithi018 ceph-mon[40196]: pgmap v722: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 5.5 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 853 B/s rd, 56 MiB/s wr, 897 op/s; 15443/273276 objects misplaced (5.651%); 17 MiB/s, 272 objects/s recovering 2024-09-10T19:00:50.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:50 smithi018 ceph-mon[40196]: 1.13 scrub starts 2024-09-10T19:00:50.995 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:51.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:50 smithi103 ceph-mon[33163]: pgmap v722: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 5.5 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 853 B/s rd, 56 MiB/s wr, 897 op/s; 15443/273276 objects misplaced (5.651%); 17 MiB/s, 272 objects/s recovering 2024-09-10T19:00:51.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:50 smithi103 ceph-mon[33163]: 1.13 scrub starts 2024-09-10T19:00:51.031 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:51.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:51.032 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:51.133 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:51.155 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:51.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:51.156 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:51.257 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:51.291 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:51.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:51.292 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:51.393 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:51.430 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:51.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:51.431 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:51.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:00:50 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:00:50] "GET /metrics HTTP/1.1" 200 224660 "" "Prometheus/2.43.0" 2024-09-10T19:00:51.531 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:51.562 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:51.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:51.563 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:51.610 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 58 16 97131 97115 104.622 75.5 0.00732364 0.00955336 2024-09-10T19:00:51.664 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:51.688 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:51.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:51.689 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:51.790 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:51.822 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:51.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:51.823 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:51.924 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:51.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:51 smithi018 ceph-mon[29336]: 4.9 scrub starts 2024-09-10T19:00:51.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:51 smithi018 ceph-mon[40196]: 4.9 scrub starts 2024-09-10T19:00:51.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:51.967 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:51.967 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:52.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:51 smithi103 ceph-mon[33163]: 4.9 scrub starts 2024-09-10T19:00:52.067 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:52.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:52.112 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:52.112 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:52.213 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:52.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:52.250 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:52.250 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:52.351 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:52.387 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:52.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:52.388 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:52.489 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:52.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:52.530 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:52.530 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:52.610 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 59 16 98515 98499 104.315 86.5 0.0113736 0.00958212 2024-09-10T19:00:52.631 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:52.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:52.669 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:52.670 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:52.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:52 smithi018 ceph-mon[29336]: pgmap v723: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 5.6 GiB data, 22 GiB used, 515 GiB / 536 GiB avail; 650 B/s rd, 49 MiB/s wr, 789 op/s; 15443/275763 objects misplaced (5.600%); 13 MiB/s, 207 objects/s recovering 2024-09-10T19:00:52.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:52 smithi018 ceph-mon[29336]: 2.a scrub starts 2024-09-10T19:00:52.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:52 smithi018 ceph-mon[40196]: pgmap v723: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 5.6 GiB data, 22 GiB used, 515 GiB / 536 GiB avail; 650 B/s rd, 49 MiB/s wr, 789 op/s; 15443/275763 objects misplaced (5.600%); 13 MiB/s, 207 objects/s recovering 2024-09-10T19:00:52.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:52 smithi018 ceph-mon[40196]: 2.a scrub starts 2024-09-10T19:00:52.771 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:52.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:52.823 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:52.823 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:52.924 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:52.960 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:52.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:52.963 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:53.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:52 smithi103 ceph-mon[33163]: pgmap v723: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 5.6 GiB data, 22 GiB used, 515 GiB / 536 GiB avail; 650 B/s rd, 49 MiB/s wr, 789 op/s; 15443/275763 objects misplaced (5.600%); 13 MiB/s, 207 objects/s recovering 2024-09-10T19:00:53.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:52 smithi103 ceph-mon[33163]: 2.a scrub starts 2024-09-10T19:00:53.064 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:53.099 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:53.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:53.100 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:53.201 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:53.236 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:53.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:53.237 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:53.338 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:53.379 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:53.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:53.380 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:53.481 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:53.507 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:53.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:53.508 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:53.609 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:53.610 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:2024-09-10T19:00:53.610058+0000 min lat: 0.00113481 max lat: 2.35071 avg lat: 0.00960581 2024-09-10T19:00:53.610 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T19:00:53.610 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 60 16 99937 99921 104.057 88.875 0.00915643 0.00960581 2024-09-10T19:00:53.650 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:53.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:53.651 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:53.752 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:53.790 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:53.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:53.791 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:53.868 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:53 smithi018 ceph-mon[40196]: 4.1c scrub starts 2024-09-10T19:00:53.868 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:53 smithi018 ceph-mon[29336]: 4.1c scrub starts 2024-09-10T19:00:53.892 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:53.929 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:53.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:53.930 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:54.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:53 smithi103 ceph-mon[33163]: 4.1c scrub starts 2024-09-10T19:00:54.031 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:54.072 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:54.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:54.073 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:54.174 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:54.206 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:54.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:54.207 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:54.308 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:54.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:54.354 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:54.354 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:54.381 INFO:teuthology.orchestra.run.smithi018.stdout:ERROR: (22) Invalid argument 2024-09-10T19:00:54.382 INFO:teuthology.orchestra.run.smithi018.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T19:00:54.455 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:54.476 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T19:00:54.487 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:54.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:54.488 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:54.589 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:54.601 INFO:teuthology.orchestra.run.smithi018.stderr:reweighted osd.6 to 1 (10000) 2024-09-10T19:00:54.610 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 61 16 101324 101308 103.772 86.6875 0.0114155 0.00963052 2024-09-10T19:00:54.644 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:54.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:54.645 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:54.746 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:54.779 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:54.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:54.780 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:54.881 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:54.921 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:54.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:54.922 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:54.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:54 smithi018 ceph-mon[29336]: pgmap v724: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 5.9 GiB data, 22 GiB used, 514 GiB / 536 GiB avail; 1.1 KiB/s rd, 71 MiB/s wr, 1.13k op/s; 14514/289623 objects misplaced (5.011%); 23 MiB/s, 372 objects/s recovering 2024-09-10T19:00:54.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:54 smithi018 ceph-mon[29336]: 1.d scrub starts 2024-09-10T19:00:54.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:54 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2972177403' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-10T19:00:54.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:54 smithi018 ceph-mon[40196]: pgmap v724: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 5.9 GiB data, 22 GiB used, 514 GiB / 536 GiB avail; 1.1 KiB/s rd, 71 MiB/s wr, 1.13k op/s; 14514/289623 objects misplaced (5.011%); 23 MiB/s, 372 objects/s recovering 2024-09-10T19:00:54.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:54 smithi018 ceph-mon[40196]: 1.d scrub starts 2024-09-10T19:00:54.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:54 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2972177403' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-10T19:00:55.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:54 smithi103 ceph-mon[33163]: pgmap v724: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 5.9 GiB data, 22 GiB used, 514 GiB / 536 GiB avail; 1.1 KiB/s rd, 71 MiB/s wr, 1.13k op/s; 14514/289623 objects misplaced (5.011%); 23 MiB/s, 372 objects/s recovering 2024-09-10T19:00:55.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:54 smithi103 ceph-mon[33163]: 1.d scrub starts 2024-09-10T19:00:55.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:54 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2972177403' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-10T19:00:55.022 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:55.056 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:55.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:55.057 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:55.158 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:55.194 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:55.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:55.195 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:55.296 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:55.331 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:55.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:55.333 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:55.433 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:55.457 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:55.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:55.458 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:55.560 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:55.599 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:55.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:55.600 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:55.611 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 62 16 102688 102672 103.473 85.25 0.013404 0.00965972 2024-09-10T19:00:55.627 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is unset 2024-09-10T19:00:55.700 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:55.744 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:55.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:55.745 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:55.846 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:55.879 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:55.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:55.880 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:55.881 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:55 smithi018 ceph-mon[29336]: 5.1d scrub starts 2024-09-10T19:00:55.881 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:55 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2972177403' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-10T19:00:55.881 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:55 smithi018 ceph-mon[29336]: osdmap e343: 8 total, 8 up, 6 in 2024-09-10T19:00:55.881 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:00:55.881 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:55 smithi018 ceph-mon[29336]: pgmap v726: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 6.0 GiB data, 23 GiB used, 514 GiB / 536 GiB avail; 639 B/s rd, 62 MiB/s wr, 998 op/s; 14129/293445 objects misplaced (4.815%); 24 MiB/s, 381 objects/s recovering 2024-09-10T19:00:55.881 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:55 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/227062856' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:00:55.881 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:55 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:00:55.881 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:55 smithi018 ceph-mon[40196]: 5.1d scrub starts 2024-09-10T19:00:55.881 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:55 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2972177403' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-10T19:00:55.882 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:55 smithi018 ceph-mon[40196]: osdmap e343: 8 total, 8 up, 6 in 2024-09-10T19:00:55.882 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:00:55.882 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:55 smithi018 ceph-mon[40196]: pgmap v726: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 6.0 GiB data, 23 GiB used, 514 GiB / 536 GiB avail; 639 B/s rd, 62 MiB/s wr, 998 op/s; 14129/293445 objects misplaced (4.815%); 24 MiB/s, 381 objects/s recovering 2024-09-10T19:00:55.882 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:55 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/227062856' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:00:55.882 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:55 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:00:55.887 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-10T19:00:55.887 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T19:00:55.981 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:56.021 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:56.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:55 smithi103 ceph-mon[33163]: 5.1d scrub starts 2024-09-10T19:00:56.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:55 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2972177403' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-10T19:00:56.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:55 smithi103 ceph-mon[33163]: osdmap e343: 8 total, 8 up, 6 in 2024-09-10T19:00:56.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:00:56.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:55 smithi103 ceph-mon[33163]: pgmap v726: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 6.0 GiB data, 23 GiB used, 514 GiB / 536 GiB avail; 639 B/s rd, 62 MiB/s wr, 998 op/s; 14129/293445 objects misplaced (4.815%); 24 MiB/s, 381 objects/s recovering 2024-09-10T19:00:56.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:55 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/227062856' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:00:56.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:55 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:00:56.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:56.023 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:56.124 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:56.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:56.159 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:56.159 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:56.260 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:56.285 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:56.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:56.286 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:56.387 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:56.408 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:56.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:56.409 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:56.510 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:56.532 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:56.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:56.533 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:56.611 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 63 15 103566 103551 102.702 54.9375 0.0226147 0.00973238 2024-09-10T19:00:56.633 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:56.671 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:56.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:56.672 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:56.773 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:56.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:56.817 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:56.817 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:56.918 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:56.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:56 smithi018 ceph-mon[40196]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T19:00:56.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:56 smithi018 ceph-mon[40196]: Cluster is now healthy 2024-09-10T19:00:56.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:56 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T19:00:56.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:56 smithi018 ceph-mon[40196]: osdmap e344: 8 total, 8 up, 6 in 2024-09-10T19:00:56.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:56 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:00:56.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:56 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:00:56.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:56 smithi018 ceph-mon[40196]: 2.0 scrub starts 2024-09-10T19:00:56.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:56 smithi018 ceph-mon[29336]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T19:00:56.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:56 smithi018 ceph-mon[29336]: Cluster is now healthy 2024-09-10T19:00:56.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:56 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T19:00:56.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:56 smithi018 ceph-mon[29336]: osdmap e344: 8 total, 8 up, 6 in 2024-09-10T19:00:56.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:56 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:00:56.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:56 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:00:56.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:56 smithi018 ceph-mon[29336]: 2.0 scrub starts 2024-09-10T19:00:56.950 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:56.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:56.951 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:56.963 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:56 smithi103 ceph-mon[33163]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T19:00:56.963 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:56 smithi103 ceph-mon[33163]: Cluster is now healthy 2024-09-10T19:00:56.963 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:56 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T19:00:56.963 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:56 smithi103 ceph-mon[33163]: osdmap e344: 8 total, 8 up, 6 in 2024-09-10T19:00:56.963 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:56 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:00:56.963 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:56 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:00:56.963 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:56 smithi103 ceph-mon[33163]: 2.0 scrub starts 2024-09-10T19:00:57.052 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:57.091 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:57.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:57.092 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:57.192 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:57.230 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:57.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:57.231 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:57.247 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-10T19:00:57.272 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:00:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:00:57.321 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:00:57.332 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:00:57.352 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:00:57.365 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:57.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:57.366 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:00:57.466 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:57.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:57.513 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:57.513 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:57.611 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 64 16 104956 104940 102.454 86.8125 0.00638541 0.00975589 2024-09-10T19:00:57.614 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:57.665 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:57.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:57.666 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:57.740 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:57 smithi018 ceph-mon[29336]: osdmap e345: 8 total, 8 up, 6 in 2024-09-10T19:00:57.741 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:57 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:00:57.741 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:57 smithi018 ceph-mon[29336]: pgmap v729: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 6.3 GiB data, 24 GiB used, 513 GiB / 536 GiB avail; 1.7 KiB/s rd, 116 MiB/s wr, 1.86k op/s; 13241/309204 objects misplaced (4.282%); 40 MiB/s, 640 objects/s recovering 2024-09-10T19:00:57.741 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:57 smithi018 ceph-mon[29336]: 4.4 scrub starts 2024-09-10T19:00:57.741 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:57 smithi018 ceph-mon[40196]: osdmap e345: 8 total, 8 up, 6 in 2024-09-10T19:00:57.741 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:57 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:00:57.741 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:57 smithi018 ceph-mon[40196]: pgmap v729: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 6.3 GiB data, 24 GiB used, 513 GiB / 536 GiB avail; 1.7 KiB/s rd, 116 MiB/s wr, 1.86k op/s; 13241/309204 objects misplaced (4.282%); 40 MiB/s, 640 objects/s recovering 2024-09-10T19:00:57.741 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:57 smithi018 ceph-mon[40196]: 4.4 scrub starts 2024-09-10T19:00:57.767 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:57.810 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:57.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:57.811 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:57.912 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:00:57.947 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:57.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:57.948 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:00:58.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:57 smithi103 ceph-mon[33163]: osdmap e345: 8 total, 8 up, 6 in 2024-09-10T19:00:58.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:57 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:00:58.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:57 smithi103 ceph-mon[33163]: pgmap v729: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 6.3 GiB data, 24 GiB used, 513 GiB / 536 GiB avail; 1.7 KiB/s rd, 116 MiB/s wr, 1.86k op/s; 13241/309204 objects misplaced (4.282%); 40 MiB/s, 640 objects/s recovering 2024-09-10T19:00:58.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:57 smithi103 ceph-mon[33163]: 4.4 scrub starts 2024-09-10T19:00:58.048 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:58.085 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:58.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:58.086 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:58.187 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:58.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:58.226 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:58.226 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:58.327 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:00:58.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:58.368 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:58.369 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:00:58.470 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:58.504 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:58.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:58.505 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:58.606 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:00:58.611 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 65 16 106464 106448 102.328 94.25 0.00508323 0.00976854 2024-09-10T19:00:58.647 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:58.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:58.648 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:00:58.749 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:58.785 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:58.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:58.786 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:58.802 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-10T19:00:58.839 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:58 smithi018 ceph-mon[29336]: 1.14 scrub starts 2024-09-10T19:00:58.840 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:58 smithi018 ceph-mon[40196]: 1.14 scrub starts 2024-09-10T19:00:58.886 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:58.924 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:58.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:58.925 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:59.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:58 smithi103 ceph-mon[33163]: 1.14 scrub starts 2024-09-10T19:00:59.025 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:59.077 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:59.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:59.078 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:59.179 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:59.201 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:59.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:59.202 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:00:59.303 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:59.338 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:59.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:59.339 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:59.440 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:00:59.472 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:59.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:59.473 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:00:59.574 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:00:59.611 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 66 16 107801 107785 102.043 83.5625 0.00908486 0.00979576 2024-09-10T19:00:59.611 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:59.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:59.612 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:00:59.713 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:00:59.760 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:00:59.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:59.762 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:00:59.862 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:00:59.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:59.900 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:00:59.900 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:00.001 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:00.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:59 smithi103 ceph-mon[33163]: osdmap e346: 8 total, 8 up, 6 in 2024-09-10T19:01:00.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:59 smithi103 ceph-mon[33163]: pgmap v731: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 6.4 GiB data, 24 GiB used, 512 GiB / 536 GiB avail; 1.0 KiB/s rd, 84 MiB/s wr, 1.35k op/s; 12891/312852 objects misplaced (4.120%); 27 MiB/s, 432 objects/s recovering 2024-09-10T19:01:00.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:00:59 smithi103 ceph-mon[33163]: 5.12 scrub starts 2024-09-10T19:01:00.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:00.041 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:00.041 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:00.062 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:59 smithi018 ceph-mon[29336]: osdmap e346: 8 total, 8 up, 6 in 2024-09-10T19:01:00.062 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:59 smithi018 ceph-mon[29336]: pgmap v731: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 6.4 GiB data, 24 GiB used, 512 GiB / 536 GiB avail; 1.0 KiB/s rd, 84 MiB/s wr, 1.35k op/s; 12891/312852 objects misplaced (4.120%); 27 MiB/s, 432 objects/s recovering 2024-09-10T19:01:00.062 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:00:59 smithi018 ceph-mon[29336]: 5.12 scrub starts 2024-09-10T19:01:00.062 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:59 smithi018 ceph-mon[40196]: osdmap e346: 8 total, 8 up, 6 in 2024-09-10T19:01:00.062 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:59 smithi018 ceph-mon[40196]: pgmap v731: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 6.4 GiB data, 24 GiB used, 512 GiB / 536 GiB avail; 1.0 KiB/s rd, 84 MiB/s wr, 1.35k op/s; 12891/312852 objects misplaced (4.120%); 27 MiB/s, 432 objects/s recovering 2024-09-10T19:01:00.062 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:00:59 smithi018 ceph-mon[40196]: 5.12 scrub starts 2024-09-10T19:01:00.142 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:00.192 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:00.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:00.193 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:00.294 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:00.332 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:00.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:00.333 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:00.434 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:00.472 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:00.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:00.473 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:00.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:01:00 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:01:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:01:00.575 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:00.611 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 67 15 109209 109194 101.835 88.0625 0.0106029 0.0098158 2024-09-10T19:01:00.613 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:00.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:00.614 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:00.715 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:00.752 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:00.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:00.753 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:00.854 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:00.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:00.900 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:00.900 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:01.001 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:01.041 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:01.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:01.042 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:01.143 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:01.167 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:01:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:01:00] "GET /metrics HTTP/1.1" 200 224736 "" "Prometheus/2.43.0" 2024-09-10T19:01:01.199 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:01.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:01.200 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:01.301 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:01.342 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:01.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:01.343 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:01.444 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:01.477 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:01.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:01.478 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:01.579 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:01.584 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T19:01:01.584 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T19:01:01.584 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T19:01:01.584 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T19:01:01.584 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T19:01:01.584 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T19:01:01.584 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T19:01:01.585 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T19:01:01.590 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T19:01:01.590 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T19:01:01.610 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:01.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:01.611 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:01.611 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 68 16 110506 110490 101.528 81 0.0064619 0.0098447 2024-09-10T19:01:01.665 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T19:01:01.665 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T19:01:01.675 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:01:01.697 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:01:01.703 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T19:01:01.704 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T19:01:01.711 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:01.743 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T19:01:01.743 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T19:01:01.771 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:01.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:01.772 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:01.862 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T19:01:01.871 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-09-10T19:01:01.873 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:01.910 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:01.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:01.911 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:02.011 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:02.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:01 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:01:02.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:01 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:01:02.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:01 smithi103 ceph-mon[33163]: osdmap e347: 8 total, 8 up, 6 in 2024-09-10T19:01:02.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:01 smithi103 ceph-mon[33163]: 2.f deep-scrub starts 2024-09-10T19:01:02.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:01 smithi103 ceph-mon[33163]: pgmap v733: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 6.4 GiB data, 24 GiB used, 512 GiB / 536 GiB avail; 889 B/s rd, 83 MiB/s wr, 1.32k op/s; 12327/316320 objects misplaced (3.897%); 31 MiB/s, 496 objects/s recovering 2024-09-10T19:01:02.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:01 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/4207082281' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:01:02.048 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:02.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:02.049 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:02.065 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:01 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:01:02.065 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:01 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:01:02.065 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:01 smithi018 ceph-mon[40196]: osdmap e347: 8 total, 8 up, 6 in 2024-09-10T19:01:02.065 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:01 smithi018 ceph-mon[40196]: 2.f deep-scrub starts 2024-09-10T19:01:02.065 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:01 smithi018 ceph-mon[40196]: pgmap v733: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 6.4 GiB data, 24 GiB used, 512 GiB / 536 GiB avail; 889 B/s rd, 83 MiB/s wr, 1.32k op/s; 12327/316320 objects misplaced (3.897%); 31 MiB/s, 496 objects/s recovering 2024-09-10T19:01:02.065 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:01 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/4207082281' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:01:02.065 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:01 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:01:02.065 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:01 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:01:02.065 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:01 smithi018 ceph-mon[29336]: osdmap e347: 8 total, 8 up, 6 in 2024-09-10T19:01:02.065 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:01 smithi018 ceph-mon[29336]: 2.f deep-scrub starts 2024-09-10T19:01:02.065 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:01 smithi018 ceph-mon[29336]: pgmap v733: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 6.4 GiB data, 24 GiB used, 512 GiB / 536 GiB avail; 889 B/s rd, 83 MiB/s wr, 1.32k op/s; 12327/316320 objects misplaced (3.897%); 31 MiB/s, 496 objects/s recovering 2024-09-10T19:01:02.066 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:01 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/4207082281' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:01:02.150 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:02.187 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:02.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:02.188 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:02.289 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:02.328 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:02.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:02.329 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:02.430 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:02.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:02.470 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:02.471 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:02.571 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:02.609 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:02.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:02.610 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:02.611 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 69 16 111829 111813 101.255 82.6875 0.013885 0.00987191 2024-09-10T19:01:02.711 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:02.751 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:02.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:02.753 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:02.853 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:02.889 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:02.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:02.891 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:02.991 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:03.017 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:02 smithi018 ceph-mon[40196]: from='client.25348 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:01:03.018 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:02 smithi018 ceph-mon[29336]: from='client.25348 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:01:03.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:02 smithi103 ceph-mon[33163]: from='client.25348 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:01:03.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:03.029 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:03.029 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:03.130 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:03.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:03.166 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:03.166 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:03.266 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:03.302 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:03.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:03.303 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:03.404 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:03.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:03.445 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:03.445 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:03.546 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":733,"stamp":"2024-09-10T19:01:01.379772+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6897884862,"num_objects":105440,"num_object_clones":0,"num_object_copies":316320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":12327,"num_objects_unfound":0,"num_objects_dirty":105440,"num_whiteouts":0,"num_read":2073,"num_read_kb":1828,"num_write":105963,"num_write_kb":6735708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7937,"num_bytes_recovered":502899562,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":59864,"ondisk_log_size":59864,"up":504,"acting":504,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":507,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":562470912,"kb_used":25687040,"kb_used_data":20888244,"kb_used_omap":12,"kb_used_meta":4798707,"kb_avail":536783872,"statfs":{"total":575970213888,"available":549666684928,"internally_reserved":0,"allocated":21389561856,"data_stored":21385743356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13122,"internal_metadata":4913876158},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":499712000,"num_objects":7625,"num_object_clones":0,"num_object_copies":22875,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-1802,"num_objects_unfound":0,"num_objects_dirty":7625,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":7625,"num_write_kb":488000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2860,"num_bytes_recovered":187432960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.756190"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":351,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.717333+0000","last_change":"2024-09-10T18:57:02.462265+0000","last_active":"2024-09-10T19:00:58.717333+0000","last_peered":"2024-09-10T19:00:58.717333+0000","last_clean":"2024-09-10T19:00:58.717333+0000","last_became_active":"2024-09-10T18:57:02.460451+0000","last_became_peered":"2024-09-10T18:57:02.460451+0000","last_unstale":"2024-09-10T19:00:58.717333+0000","last_undegraded":"2024-09-10T19:00:58.717333+0000","last_fullsized":"2024-09-10T19:00:58.717333+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:14.875872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:14.875872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037478699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":345,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.617348+0000","last_change":"2024-09-10T18:59:59.848814+0000","last_active":"2024-09-10T19:00:55.617348+0000","last_peered":"2024-09-10T19:00:55.617348+0000","last_clean":"2024-09-10T19:00:55.617348+0000","last_became_active":"2024-09-10T18:57:02.440648+0000","last_became_peered":"2024-09-10T18:57:02.440648+0000","last_unstale":"2024-09-10T19:00:55.617348+0000","last_undegraded":"2024-09-10T19:00:55.617348+0000","last_fullsized":"2024-09-10T19:00:55.617348+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:59.848700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:59.848700+0000","last_clean_scrub_stamp":"2024-09-10T18:59:59.848700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:27.993110+0000","scrub_duration":0.0047956400000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"232'27","reported_seq":436,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.718390+0000","last_change":"2024-09-10T18:56:02.821637+0000","last_active":"2024-09-10T19:00:58.718390+0000","last_peered":"2024-09-10T19:00:58.718390+0000","last_clean":"2024-09-10T19:00:58.718390+0000","last_became_active":"2024-09-10T18:55:40.394717+0000","last_became_peered":"2024-09-10T18:55:40.394717+0000","last_unstale":"2024-09-10T19:00:58.718390+0000","last_undegraded":"2024-09-10T19:00:58.718390+0000","last_fullsized":"2024-09-10T19:00:58.718390+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'27","last_scrub_stamp":"2024-09-10T18:56:02.821551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:02.821551+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0083110159999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":352,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.718475+0000","last_change":"2024-09-10T19:00:08.841172+0000","last_active":"2024-09-10T19:00:58.718475+0000","last_peered":"2024-09-10T19:00:58.718475+0000","last_clean":"2024-09-10T19:00:58.718475+0000","last_became_active":"2024-09-10T18:45:22.065619+0000","last_became_peered":"2024-09-10T18:45:22.065619+0000","last_unstale":"2024-09-10T19:00:58.718475+0000","last_undegraded":"2024-09-10T19:00:58.718475+0000","last_fullsized":"2024-09-10T19:00:58.718475+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:08.841125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.057932+0000","last_clean_scrub_stamp":"2024-09-10T19:00:08.841125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:12.343664+0000","scrub_duration":0.001059857,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":414,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.717650+0000","last_change":"2024-09-10T19:00:42.519131+0000","last_active":"2024-09-10T19:00:58.717650+0000","last_peered":"2024-09-10T19:00:58.717650+0000","last_clean":"2024-09-10T19:00:58.717650+0000","last_became_active":"2024-09-10T19:00:42.518824+0000","last_became_peered":"2024-09-10T19:00:42.518824+0000","last_unstale":"2024-09-10T19:00:58.717650+0000","last_undegraded":"2024-09-10T19:00:58.717650+0000","last_fullsized":"2024-09-10T19:00:58.717650+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:23.885562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:23.885562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0031033800000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"232'19","reported_seq":373,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.718368+0000","last_change":"2024-09-10T18:59:09.642743+0000","last_active":"2024-09-10T19:00:58.718368+0000","last_peered":"2024-09-10T19:00:58.718368+0000","last_clean":"2024-09-10T19:00:58.718368+0000","last_became_active":"2024-09-10T18:45:20.061266+0000","last_became_peered":"2024-09-10T18:45:20.061266+0000","last_unstale":"2024-09-10T19:00:58.718368+0000","last_undegraded":"2024-09-10T19:00:58.718368+0000","last_fullsized":"2024-09-10T19:00:58.718368+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T18:59:09.642642+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_clean_scrub_stamp":"2024-09-10T18:59:09.642642+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0095920940000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":86,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.624961+0000","last_change":"2024-09-10T19:00:48.452343+0000","last_active":"2024-09-10T19:00:55.624961+0000","last_peered":"2024-09-10T19:00:55.624961+0000","last_clean":"2024-09-10T19:00:55.624961+0000","last_became_active":"2024-09-10T18:57:02.446008+0000","last_became_peered":"2024-09-10T18:57:02.446008+0000","last_unstale":"2024-09-10T19:00:55.624961+0000","last_undegraded":"2024-09-10T19:00:55.624961+0000","last_fullsized":"2024-09-10T19:00:55.624961+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:22.163121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:01.135988+0000","last_clean_scrub_stamp":"2024-09-10T18:56:22.163121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002631478,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":340,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.722946+0000","last_change":"2024-09-10T18:58:51.682046+0000","last_active":"2024-09-10T19:00:58.722946+0000","last_peered":"2024-09-10T19:00:58.722946+0000","last_clean":"2024-09-10T19:00:58.722946+0000","last_became_active":"2024-09-10T18:45:24.081002+0000","last_became_peered":"2024-09-10T18:45:24.081002+0000","last_unstale":"2024-09-10T19:00:58.722946+0000","last_undegraded":"2024-09-10T19:00:58.722946+0000","last_fullsized":"2024-09-10T19:00:58.722946+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:51.681994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:58:51.681994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051776249999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":342,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.596537+0000","last_change":"2024-09-10T18:59:21.502068+0000","last_active":"2024-09-10T19:00:54.596537+0000","last_peered":"2024-09-10T19:00:54.596537+0000","last_clean":"2024-09-10T19:00:54.596537+0000","last_became_active":"2024-09-10T18:57:02.452259+0000","last_became_peered":"2024-09-10T18:57:02.452259+0000","last_unstale":"2024-09-10T19:00:54.596537+0000","last_undegraded":"2024-09-10T19:00:54.596537+0000","last_fullsized":"2024-09-10T19:00:54.596537+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:21.501984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:21.501984+0000","last_clean_scrub_stamp":"2024-09-10T18:59:21.501984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014628558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1b","version":"232'9","reported_seq":359,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.721522+0000","last_change":"2024-09-10T18:59:11.622597+0000","last_active":"2024-09-10T19:00:58.721522+0000","last_peered":"2024-09-10T19:00:58.721522+0000","last_clean":"2024-09-10T19:00:58.721522+0000","last_became_active":"2024-09-10T18:45:20.057567+0000","last_became_peered":"2024-09-10T18:45:20.057567+0000","last_unstale":"2024-09-10T19:00:58.721522+0000","last_undegraded":"2024-09-10T19:00:58.721522+0000","last_fullsized":"2024-09-10T19:00:58.721522+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'9","last_scrub_stamp":"2024-09-10T18:59:11.622509+0000","last_deep_scrub":"163'7","last_deep_scrub_stamp":"2024-09-10T18:51:54.261879+0000","last_clean_scrub_stamp":"2024-09-10T18:59:11.622509+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056120609999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":111,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.722878+0000","last_change":"2024-09-10T19:00:40.947163+0000","last_active":"2024-09-10T19:00:58.722878+0000","last_peered":"2024-09-10T19:00:58.722878+0000","last_clean":"2024-09-10T19:00:58.722878+0000","last_became_active":"2024-09-10T18:58:58.503924+0000","last_became_peered":"2024-09-10T18:58:58.503924+0000","last_unstale":"2024-09-10T19:00:58.722878+0000","last_undegraded":"2024-09-10T19:00:58.722878+0000","last_fullsized":"2024-09-10T19:00:58.722878+0000","mapping_epoch":303,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":304,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:40.947066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:36.509971+0000","last_clean_scrub_stamp":"2024-09-10T19:00:40.947066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:02:09.283651+0000","scrub_duration":0.012293715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":15,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.593746+0000","last_change":"2024-09-10T19:00:42.516918+0000","last_active":"2024-09-10T19:00:54.593746+0000","last_peered":"2024-09-10T19:00:54.593746+0000","last_clean":"2024-09-10T19:00:54.593746+0000","last_became_active":"2024-09-10T19:00:42.515840+0000","last_became_peered":"2024-09-10T19:00:42.515840+0000","last_unstale":"2024-09-10T19:00:54.593746+0000","last_undegraded":"2024-09-10T19:00:54.593746+0000","last_fullsized":"2024-09-10T19:00:54.593746+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:48.230459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:59:48.230459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:58.120725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":148,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.594853+0000","last_change":"2024-09-10T18:58:49.506212+0000","last_active":"2024-09-10T19:00:54.594853+0000","last_peered":"2024-09-10T19:00:54.594853+0000","last_clean":"2024-09-10T19:00:54.594853+0000","last_became_active":"2024-09-10T18:57:02.440497+0000","last_became_peered":"2024-09-10T18:57:02.440497+0000","last_unstale":"2024-09-10T19:00:54.594853+0000","last_undegraded":"2024-09-10T19:00:54.594853+0000","last_fullsized":"2024-09-10T19:00:54.594853+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:49.506160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:58:49.506160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.018244065,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"232'15","reported_seq":395,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.726765+0000","last_change":"2024-09-10T18:56:26.480197+0000","last_active":"2024-09-10T19:00:58.726765+0000","last_peered":"2024-09-10T19:00:58.726765+0000","last_clean":"2024-09-10T19:00:58.726765+0000","last_became_active":"2024-09-10T18:55:40.391571+0000","last_became_peered":"2024-09-10T18:55:40.391571+0000","last_unstale":"2024-09-10T19:00:58.726765+0000","last_undegraded":"2024-09-10T19:00:58.726765+0000","last_fullsized":"2024-09-10T19:00:58.726765+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T18:56:26.480105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:26.480105+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027886920000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":378,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.595271+0000","last_change":"2024-09-10T18:59:22.643043+0000","last_active":"2024-09-10T19:00:54.595271+0000","last_peered":"2024-09-10T19:00:54.595271+0000","last_clean":"2024-09-10T19:00:54.595271+0000","last_became_active":"2024-09-10T18:57:02.431585+0000","last_became_peered":"2024-09-10T18:57:02.431585+0000","last_unstale":"2024-09-10T19:00:54.595271+0000","last_undegraded":"2024-09-10T19:00:54.595271+0000","last_fullsized":"2024-09-10T19:00:54.595271+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:22.642961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:59:22.642961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.018189153999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":34,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.616147+0000","last_change":"2024-09-10T18:59:48.738143+0000","last_active":"2024-09-10T19:00:55.616147+0000","last_peered":"2024-09-10T19:00:55.616147+0000","last_clean":"2024-09-10T19:00:55.616147+0000","last_became_active":"2024-09-10T18:59:48.737788+0000","last_became_peered":"2024-09-10T18:59:48.737788+0000","last_unstale":"2024-09-10T19:00:55.616147+0000","last_undegraded":"2024-09-10T19:00:55.616147+0000","last_fullsized":"2024-09-10T19:00:55.616147+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:51.561010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:59.425718+0000","last_clean_scrub_stamp":"2024-09-10T18:58:51.561010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3","version":"344'6431","reported_seq":6468,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:56.129525+0000","last_change":"2024-09-10T18:59:43.007892+0000","last_active":"2024-09-10T19:00:56.129525+0000","last_peered":"2024-09-10T19:00:56.129525+0000","last_clean":"2024-09-10T19:00:56.129525+0000","last_became_active":"2024-09-10T18:59:43.007585+0000","last_became_peered":"2024-09-10T18:59:43.007585+0000","last_unstale":"2024-09-10T19:00:56.129525+0000","last_undegraded":"2024-09-10T19:00:56.129525+0000","last_fullsized":"2024-09-10T19:00:56.129525+0000","mapping_epoch":315,"log_start":"332'2799","ondisk_log_start":"332'2799","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3632,"log_dups_size":0,"ondisk_log_size":3632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:56.658856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":421462016,"num_objects":6431,"num_object_clones":0,"num_object_copies":19293,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6431,"num_write_kb":411584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.8","version":"70'81","reported_seq":19,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.594317+0000","last_change":"2024-09-10T19:00:42.527220+0000","last_active":"2024-09-10T19:00:54.594317+0000","last_peered":"2024-09-10T19:00:54.594317+0000","last_clean":"2024-09-10T19:00:54.594317+0000","last_became_active":"2024-09-10T19:00:42.517072+0000","last_became_peered":"2024-09-10T19:00:42.517072+0000","last_unstale":"2024-09-10T19:00:54.594317+0000","last_undegraded":"2024-09-10T19:00:54.594317+0000","last_fullsized":"2024-09-10T19:00:54.594317+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:51.167302+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:59:51.167302+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015031440000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":262688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"232'39","reported_seq":439,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.722485+0000","last_change":"2024-09-10T18:59:47.745859+0000","last_active":"2024-09-10T19:00:58.722485+0000","last_peered":"2024-09-10T19:00:58.722485+0000","last_clean":"2024-09-10T19:00:58.722485+0000","last_became_active":"2024-09-10T18:47:36.409300+0000","last_became_peered":"2024-09-10T18:47:36.409300+0000","last_unstale":"2024-09-10T19:00:58.722485+0000","last_undegraded":"2024-09-10T19:00:58.722485+0000","last_fullsized":"2024-09-10T19:00:58.722485+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'39","last_scrub_stamp":"2024-09-10T18:59:47.745766+0000","last_deep_scrub":"78'25","last_deep_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_clean_scrub_stamp":"2024-09-10T18:59:47.745766+0000","objects_scrubbed":8,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:03.642076+0000","scrub_duration":0.011279000000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":98,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.721880+0000","last_change":"2024-09-10T19:00:09.867278+0000","last_active":"2024-09-10T19:00:58.721880+0000","last_peered":"2024-09-10T19:00:58.721880+0000","last_clean":"2024-09-10T19:00:58.721880+0000","last_became_active":"2024-09-10T18:57:02.437260+0000","last_became_peered":"2024-09-10T18:57:02.437260+0000","last_unstale":"2024-09-10T19:00:58.721880+0000","last_undegraded":"2024-09-10T19:00:58.721880+0000","last_fullsized":"2024-09-10T19:00:58.721880+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:09.867183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:00:09.867183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:10.289388+0000","scrub_duration":0.0026871460000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":359,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.623065+0000","last_change":"2024-09-10T19:00:09.470422+0000","last_active":"2024-09-10T19:00:55.623065+0000","last_peered":"2024-09-10T19:00:55.623065+0000","last_clean":"2024-09-10T19:00:55.623065+0000","last_became_active":"2024-09-10T18:55:40.392449+0000","last_became_peered":"2024-09-10T18:55:40.392449+0000","last_unstale":"2024-09-10T19:00:55.623065+0000","last_undegraded":"2024-09-10T19:00:55.623065+0000","last_fullsized":"2024-09-10T19:00:55.623065+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:09.470293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:04.512425+0000","last_clean_scrub_stamp":"2024-09-10T19:00:09.470293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:13.293960+0000","scrub_duration":0.0073759949999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":257,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.617476+0000","last_change":"2024-09-10T18:58:49.524134+0000","last_active":"2024-09-10T19:00:55.617476+0000","last_peered":"2024-09-10T19:00:55.617476+0000","last_clean":"2024-09-10T19:00:55.617476+0000","last_became_active":"2024-09-10T18:55:40.388154+0000","last_became_peered":"2024-09-10T18:55:40.388154+0000","last_unstale":"2024-09-10T19:00:55.617476+0000","last_undegraded":"2024-09-10T19:00:55.617476+0000","last_fullsized":"2024-09-10T19:00:55.617476+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:49.524015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:58:49.524015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005180479,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2","version":"345'6603","reported_seq":6641,"reported_epoch":345,"state":"active+clean","last_fresh":"2024-09-10T19:00:56.793174+0000","last_change":"2024-09-10T18:59:42.709056+0000","last_active":"2024-09-10T19:00:56.793174+0000","last_peered":"2024-09-10T19:00:56.793174+0000","last_clean":"2024-09-10T19:00:56.793174+0000","last_became_active":"2024-09-10T18:59:42.708732+0000","last_became_peered":"2024-09-10T18:59:42.708732+0000","last_unstale":"2024-09-10T19:00:56.793174+0000","last_undegraded":"2024-09-10T19:00:56.793174+0000","last_fullsized":"2024-09-10T19:00:56.793174+0000","mapping_epoch":315,"log_start":"332'3100","ondisk_log_start":"332'3100","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3503,"log_dups_size":0,"ondisk_log_size":3503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":432734208,"num_objects":6603,"num_object_clones":0,"num_object_copies":19809,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6603,"num_write_kb":422592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.9","version":"70'81","reported_seq":255,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.720611+0000","last_change":"2024-09-10T18:59:56.164487+0000","last_active":"2024-09-10T19:00:58.720611+0000","last_peered":"2024-09-10T19:00:58.720611+0000","last_clean":"2024-09-10T19:00:58.720611+0000","last_became_active":"2024-09-10T18:57:02.463750+0000","last_became_peered":"2024-09-10T18:57:02.463750+0000","last_unstale":"2024-09-10T19:00:58.720611+0000","last_undegraded":"2024-09-10T19:00:58.720611+0000","last_fullsized":"2024-09-10T19:00:58.720611+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:56.164384+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:59:56.164384+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:11.692666+0000","scrub_duration":0.0041056720000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"232'25","reported_seq":379,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.721782+0000","last_change":"2024-09-10T18:56:32.437605+0000","last_active":"2024-09-10T19:00:58.721782+0000","last_peered":"2024-09-10T19:00:58.721782+0000","last_clean":"2024-09-10T19:00:58.721782+0000","last_became_active":"2024-09-10T18:45:20.053428+0000","last_became_peered":"2024-09-10T18:45:20.053428+0000","last_unstale":"2024-09-10T19:00:58.721782+0000","last_undegraded":"2024-09-10T19:00:58.721782+0000","last_fullsized":"2024-09-10T19:00:58.721782+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'25","last_scrub_stamp":"2024-09-10T18:56:32.437503+0000","last_deep_scrub":"116'15","last_deep_scrub_stamp":"2024-09-10T18:49:12.374527+0000","last_clean_scrub_stamp":"2024-09-10T18:56:32.437503+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039673679999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":359,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.633882+0000","last_change":"2024-09-10T19:00:51.441092+0000","last_active":"2024-09-10T19:00:55.633882+0000","last_peered":"2024-09-10T19:00:55.633882+0000","last_clean":"2024-09-10T19:00:55.633882+0000","last_became_active":"2024-09-10T18:57:02.440442+0000","last_became_peered":"2024-09-10T18:57:02.440442+0000","last_unstale":"2024-09-10T19:00:55.633882+0000","last_undegraded":"2024-09-10T19:00:55.633882+0000","last_fullsized":"2024-09-10T19:00:55.633882+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:29.802937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:29.802937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044818080000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"273'13","reported_seq":393,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.628770+0000","last_change":"2024-09-10T18:57:02.463853+0000","last_active":"2024-09-10T19:00:55.628770+0000","last_peered":"2024-09-10T19:00:55.628770+0000","last_clean":"2024-09-10T19:00:55.628770+0000","last_became_active":"2024-09-10T18:57:02.448258+0000","last_became_peered":"2024-09-10T18:57:02.448258+0000","last_unstale":"2024-09-10T19:00:55.628770+0000","last_undegraded":"2024-09-10T19:00:55.628770+0000","last_fullsized":"2024-09-10T19:00:55.628770+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"242'12","last_scrub_stamp":"2024-09-10T18:56:27.995682+0000","last_deep_scrub":"202'10","last_deep_scrub_stamp":"2024-09-10T18:53:45.547686+0000","last_clean_scrub_stamp":"2024-09-10T18:56:27.995682+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025134200000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":390,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.725547+0000","last_change":"2024-09-10T19:00:42.515842+0000","last_active":"2024-09-10T19:00:58.725547+0000","last_peered":"2024-09-10T19:00:58.725547+0000","last_clean":"2024-09-10T19:00:58.725547+0000","last_became_active":"2024-09-10T19:00:42.515523+0000","last_became_peered":"2024-09-10T19:00:42.515523+0000","last_unstale":"2024-09-10T19:00:58.725547+0000","last_undegraded":"2024-09-10T19:00:58.725547+0000","last_fullsized":"2024-09-10T19:00:58.725547+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:07.803916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:32.043666+0000","last_clean_scrub_stamp":"2024-09-10T19:00:07.803916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:26.903851+0000","scrub_duration":0.0011072429999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1","version":"347'6841","reported_seq":6881,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-10T19:01:01.311299+0000","last_change":"2024-09-10T18:59:42.711934+0000","last_active":"2024-09-10T19:01:01.311299+0000","last_peered":"2024-09-10T19:01:01.311299+0000","last_clean":"2024-09-10T19:01:01.311299+0000","last_became_active":"2024-09-10T18:59:42.711662+0000","last_became_peered":"2024-09-10T18:59:42.711662+0000","last_unstale":"2024-09-10T19:01:01.311299+0000","last_undegraded":"2024-09-10T19:01:01.311299+0000","last_fullsized":"2024-09-10T19:01:01.311299+0000","mapping_epoch":315,"log_start":"332'3300","ondisk_log_start":"332'3300","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3541,"log_dups_size":0,"ondisk_log_size":3541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":448331776,"num_objects":6841,"num_object_clones":0,"num_object_copies":20523,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6841,"num_write_kb":437824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1],"acting":[4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"70'81","reported_seq":266,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.593769+0000","last_change":"2024-09-10T18:59:11.493175+0000","last_active":"2024-09-10T19:00:54.593769+0000","last_peered":"2024-09-10T19:00:54.593769+0000","last_clean":"2024-09-10T19:00:54.593769+0000","last_became_active":"2024-09-10T18:50:08.398345+0000","last_became_peered":"2024-09-10T18:50:08.398345+0000","last_unstale":"2024-09-10T19:00:54.593769+0000","last_undegraded":"2024-09-10T19:00:54.593769+0000","last_fullsized":"2024-09-10T19:00:54.593769+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:11.493086+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:55:37.921519+0000","last_clean_scrub_stamp":"2024-09-10T18:59:11.493086+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013358479999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"232'34","reported_seq":373,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.594722+0000","last_change":"2024-09-10T19:00:41.482710+0000","last_active":"2024-09-10T19:00:54.594722+0000","last_peered":"2024-09-10T19:00:54.594722+0000","last_clean":"2024-09-10T19:00:54.594722+0000","last_became_active":"2024-09-10T18:46:56.742946+0000","last_became_peered":"2024-09-10T18:46:56.742946+0000","last_unstale":"2024-09-10T19:00:54.594722+0000","last_undegraded":"2024-09-10T19:00:54.594722+0000","last_fullsized":"2024-09-10T19:00:54.594722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'34","last_scrub_stamp":"2024-09-10T19:00:41.482619+0000","last_deep_scrub":"141'22","last_deep_scrub_stamp":"2024-09-10T18:50:57.119152+0000","last_clean_scrub_stamp":"2024-09-10T19:00:41.482619+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:02:06.038176+0000","scrub_duration":0.0069363309999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":369,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.625436+0000","last_change":"2024-09-10T18:59:51.286189+0000","last_active":"2024-09-10T19:00:55.625436+0000","last_peered":"2024-09-10T19:00:55.625436+0000","last_clean":"2024-09-10T19:00:55.625436+0000","last_became_active":"2024-09-10T18:57:02.445848+0000","last_became_peered":"2024-09-10T18:57:02.445848+0000","last_unstale":"2024-09-10T19:00:55.625436+0000","last_undegraded":"2024-09-10T19:00:55.625436+0000","last_fullsized":"2024-09-10T19:00:55.625436+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:51.286131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_clean_scrub_stamp":"2024-09-10T18:59:51.286131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:18.689647+0000","scrub_duration":0.0015180669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"242'10","reported_seq":558,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-10T19:01:00.746435+0000","last_change":"2024-09-10T18:56:19.581070+0000","last_active":"2024-09-10T19:01:00.746435+0000","last_peered":"2024-09-10T19:01:00.746435+0000","last_clean":"2024-09-10T19:01:00.746435+0000","last_became_active":"2024-09-10T18:55:40.397246+0000","last_became_peered":"2024-09-10T18:55:40.397246+0000","last_unstale":"2024-09-10T19:01:00.746435+0000","last_undegraded":"2024-09-10T19:01:00.746435+0000","last_fullsized":"2024-09-10T19:01:00.746435+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"242'10","last_scrub_stamp":"2024-09-10T18:56:19.581012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:19.581012+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013423529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":366,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.595227+0000","last_change":"2024-09-10T18:59:10.477418+0000","last_active":"2024-09-10T19:00:54.595227+0000","last_peered":"2024-09-10T19:00:54.595227+0000","last_clean":"2024-09-10T19:00:54.595227+0000","last_became_active":"2024-09-10T18:55:40.392003+0000","last_became_peered":"2024-09-10T18:55:40.392003+0000","last_unstale":"2024-09-10T19:00:54.595227+0000","last_undegraded":"2024-09-10T19:00:54.595227+0000","last_fullsized":"2024-09-10T19:00:54.595227+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:10.477346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:54.218602+0000","last_clean_scrub_stamp":"2024-09-10T18:59:10.477346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004313005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"344'6515","reported_seq":6552,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:56.258286+0000","last_change":"2024-09-10T18:59:42.715764+0000","last_active":"2024-09-10T19:00:56.258286+0000","last_peered":"2024-09-10T19:00:56.258286+0000","last_clean":"2024-09-10T19:00:56.258286+0000","last_became_active":"2024-09-10T18:59:42.714808+0000","last_became_peered":"2024-09-10T18:59:42.714808+0000","last_unstale":"2024-09-10T19:00:56.258286+0000","last_undegraded":"2024-09-10T19:00:56.258286+0000","last_fullsized":"2024-09-10T19:00:56.258286+0000","mapping_epoch":315,"log_start":"332'3000","ondisk_log_start":"332'3000","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3515,"log_dups_size":0,"ondisk_log_size":3515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":426967040,"num_objects":6515,"num_object_clones":0,"num_object_copies":19545,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6515,"num_write_kb":416960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.b","version":"70'81","reported_seq":309,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.623395+0000","last_change":"2024-09-10T19:00:42.519244+0000","last_active":"2024-09-10T19:00:55.623395+0000","last_peered":"2024-09-10T19:00:55.623395+0000","last_clean":"2024-09-10T19:00:55.623395+0000","last_became_active":"2024-09-10T19:00:42.518107+0000","last_became_peered":"2024-09-10T19:00:42.518107+0000","last_unstale":"2024-09-10T19:00:55.623395+0000","last_undegraded":"2024-09-10T19:00:55.623395+0000","last_fullsized":"2024-09-10T19:00:55.623395+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:33.793296+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:33.793296+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045437230000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"232'24","reported_seq":456,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.725812+0000","last_change":"2024-09-10T18:59:06.595605+0000","last_active":"2024-09-10T19:00:58.725812+0000","last_peered":"2024-09-10T19:00:58.725812+0000","last_clean":"2024-09-10T19:00:58.725812+0000","last_became_active":"2024-09-10T18:47:57.106935+0000","last_became_peered":"2024-09-10T18:47:57.106935+0000","last_unstale":"2024-09-10T19:00:58.725812+0000","last_undegraded":"2024-09-10T19:00:58.725812+0000","last_fullsized":"2024-09-10T19:00:58.725812+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T18:59:06.595554+0000","last_deep_scrub":"224'16","last_deep_scrub_stamp":"2024-09-10T18:54:49.296957+0000","last_clean_scrub_stamp":"2024-09-10T18:59:06.595554+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020642540000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":340,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.616848+0000","last_change":"2024-09-10T18:57:02.444295+0000","last_active":"2024-09-10T19:00:55.616848+0000","last_peered":"2024-09-10T19:00:55.616848+0000","last_clean":"2024-09-10T19:00:55.616848+0000","last_became_active":"2024-09-10T18:57:02.443816+0000","last_became_peered":"2024-09-10T18:57:02.443816+0000","last_unstale":"2024-09-10T19:00:55.616848+0000","last_undegraded":"2024-09-10T19:00:55.616848+0000","last_fullsized":"2024-09-10T19:00:55.616848+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:29.994533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:29.994533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053276879999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":354,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.594860+0000","last_change":"2024-09-10T18:58:54.548235+0000","last_active":"2024-09-10T19:00:54.594860+0000","last_peered":"2024-09-10T19:00:54.594860+0000","last_clean":"2024-09-10T19:00:54.594860+0000","last_became_active":"2024-09-10T18:45:22.068918+0000","last_became_peered":"2024-09-10T18:45:22.068918+0000","last_unstale":"2024-09-10T19:00:54.594860+0000","last_undegraded":"2024-09-10T19:00:54.594860+0000","last_fullsized":"2024-09-10T19:00:54.594860+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:54.548153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_clean_scrub_stamp":"2024-09-10T18:58:54.548153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013051650000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":341,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.718429+0000","last_change":"2024-09-10T18:59:13.892307+0000","last_active":"2024-09-10T19:00:58.718429+0000","last_peered":"2024-09-10T19:00:58.718429+0000","last_clean":"2024-09-10T19:00:58.718429+0000","last_became_active":"2024-09-10T18:45:24.264538+0000","last_became_peered":"2024-09-10T18:45:24.264538+0000","last_unstale":"2024-09-10T19:00:58.718429+0000","last_undegraded":"2024-09-10T19:00:58.718429+0000","last_fullsized":"2024-09-10T19:00:58.718429+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:13.892129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:51.398981+0000","last_clean_scrub_stamp":"2024-09-10T18:59:13.892129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007612214,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"344'6445","reported_seq":6482,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:56.262213+0000","last_change":"2024-09-10T18:59:43.008094+0000","last_active":"2024-09-10T19:00:56.262213+0000","last_peered":"2024-09-10T19:00:56.262213+0000","last_clean":"2024-09-10T19:00:56.262213+0000","last_became_active":"2024-09-10T18:59:43.007838+0000","last_became_peered":"2024-09-10T18:59:43.007838+0000","last_unstale":"2024-09-10T19:00:56.262213+0000","last_undegraded":"2024-09-10T19:00:56.262213+0000","last_fullsized":"2024-09-10T19:00:56.262213+0000","mapping_epoch":315,"log_start":"332'2900","ondisk_log_start":"332'2900","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3545,"log_dups_size":0,"ondisk_log_size":3545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":422379520,"num_objects":6445,"num_object_clones":0,"num_object_copies":19335,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6445,"num_write_kb":412480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"70'81","reported_seq":299,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.718607+0000","last_change":"2024-09-10T19:00:42.520084+0000","last_active":"2024-09-10T19:00:58.718607+0000","last_peered":"2024-09-10T19:00:58.718607+0000","last_clean":"2024-09-10T19:00:58.718607+0000","last_became_active":"2024-09-10T19:00:42.519742+0000","last_became_peered":"2024-09-10T19:00:42.519742+0000","last_unstale":"2024-09-10T19:00:58.718607+0000","last_undegraded":"2024-09-10T19:00:58.718607+0000","last_fullsized":"2024-09-10T19:00:58.718607+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:58:56.912936+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:58:56.912936+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045874109999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"232'35","reported_seq":393,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.722629+0000","last_change":"2024-09-10T18:56:28.475224+0000","last_active":"2024-09-10T19:00:58.722629+0000","last_peered":"2024-09-10T19:00:58.722629+0000","last_clean":"2024-09-10T19:00:58.722629+0000","last_became_active":"2024-09-10T18:45:20.052243+0000","last_became_peered":"2024-09-10T18:45:20.052243+0000","last_unstale":"2024-09-10T19:00:58.722629+0000","last_undegraded":"2024-09-10T19:00:58.722629+0000","last_fullsized":"2024-09-10T19:00:58.722629+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'35","last_scrub_stamp":"2024-09-10T18:56:28.475142+0000","last_deep_scrub":"143'21","last_deep_scrub_stamp":"2024-09-10T18:51:02.540216+0000","last_clean_scrub_stamp":"2024-09-10T18:56:28.475142+0000","objects_scrubbed":7,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026978649999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":376,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.600888+0000","last_change":"2024-09-10T18:57:02.453766+0000","last_active":"2024-09-10T19:00:54.600888+0000","last_peered":"2024-09-10T19:00:54.600888+0000","last_clean":"2024-09-10T19:00:54.600888+0000","last_became_active":"2024-09-10T18:57:02.453404+0000","last_became_peered":"2024-09-10T18:57:02.453404+0000","last_unstale":"2024-09-10T19:00:54.600888+0000","last_undegraded":"2024-09-10T19:00:54.600888+0000","last_fullsized":"2024-09-10T19:00:54.600888+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:29.226803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_clean_scrub_stamp":"2024-09-10T18:56:29.226803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0086984899999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":539,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.716813+0000","last_change":"2024-09-10T18:56:17.914259+0000","last_active":"2024-09-10T19:00:58.716813+0000","last_peered":"2024-09-10T19:00:58.716813+0000","last_clean":"2024-09-10T19:00:58.716813+0000","last_became_active":"2024-09-10T18:45:22.074540+0000","last_became_peered":"2024-09-10T18:45:22.074540+0000","last_unstale":"2024-09-10T19:00:58.716813+0000","last_undegraded":"2024-09-10T19:00:58.716813+0000","last_fullsized":"2024-09-10T19:00:58.716813+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:56:17.914175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:17.914175+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026666329999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":143,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.596745+0000","last_change":"2024-09-10T18:58:50.638774+0000","last_active":"2024-09-10T19:00:54.596745+0000","last_peered":"2024-09-10T19:00:54.596745+0000","last_clean":"2024-09-10T19:00:54.596745+0000","last_became_active":"2024-09-10T18:57:02.437100+0000","last_became_peered":"2024-09-10T18:57:02.437100+0000","last_unstale":"2024-09-10T19:00:54.596745+0000","last_undegraded":"2024-09-10T19:00:54.596745+0000","last_fullsized":"2024-09-10T19:00:54.596745+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:50.638720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:58:50.638720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003753546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.e","version":"347'6803","reported_seq":6843,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-10T19:01:01.314045+0000","last_change":"2024-09-10T18:59:42.713933+0000","last_active":"2024-09-10T19:01:01.314045+0000","last_peered":"2024-09-10T19:01:01.314045+0000","last_clean":"2024-09-10T19:01:01.314045+0000","last_became_active":"2024-09-10T18:59:42.713605+0000","last_became_peered":"2024-09-10T18:59:42.713605+0000","last_unstale":"2024-09-10T19:01:01.314045+0000","last_undegraded":"2024-09-10T19:01:01.314045+0000","last_fullsized":"2024-09-10T19:01:01.314045+0000","mapping_epoch":315,"log_start":"332'3200","ondisk_log_start":"332'3200","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3603,"log_dups_size":0,"ondisk_log_size":3603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:09.861584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":445841408,"num_objects":6803,"num_object_clones":0,"num_object_copies":20409,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6803,"num_write_kb":435392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"70'81","reported_seq":264,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.721905+0000","last_change":"2024-09-10T18:56:31.481362+0000","last_active":"2024-09-10T19:00:58.721905+0000","last_peered":"2024-09-10T19:00:58.721905+0000","last_clean":"2024-09-10T19:00:58.721905+0000","last_became_active":"2024-09-10T18:52:14.584798+0000","last_became_peered":"2024-09-10T18:52:14.584798+0000","last_unstale":"2024-09-10T19:00:58.721905+0000","last_undegraded":"2024-09-10T19:00:58.721905+0000","last_fullsized":"2024-09-10T19:00:58.721905+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:31.481303+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:56:31.481303+0000","last_clean_scrub_stamp":"2024-09-10T18:56:31.481303+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0011170959999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"232'30","reported_seq":511,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.717916+0000","last_change":"2024-09-10T18:59:48.754412+0000","last_active":"2024-09-10T19:00:58.717916+0000","last_peered":"2024-09-10T19:00:58.717916+0000","last_clean":"2024-09-10T19:00:58.717916+0000","last_became_active":"2024-09-10T18:59:48.742377+0000","last_became_peered":"2024-09-10T18:59:48.742377+0000","last_unstale":"2024-09-10T19:00:58.717916+0000","last_undegraded":"2024-09-10T19:00:58.717916+0000","last_fullsized":"2024-09-10T19:00:58.717916+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'30","last_scrub_stamp":"2024-09-10T18:56:15.875924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:15.875924+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0075153879999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":349,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.623889+0000","last_change":"2024-09-10T18:59:05.246945+0000","last_active":"2024-09-10T19:00:55.623889+0000","last_peered":"2024-09-10T19:00:55.623889+0000","last_clean":"2024-09-10T19:00:55.623889+0000","last_became_active":"2024-09-10T18:45:17.961695+0000","last_became_peered":"2024-09-10T18:45:17.961695+0000","last_unstale":"2024-09-10T19:00:55.623889+0000","last_undegraded":"2024-09-10T19:00:55.623889+0000","last_fullsized":"2024-09-10T19:00:55.623889+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:05.246629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:46.142129+0000","last_clean_scrub_stamp":"2024-09-10T18:59:05.246629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0072716029999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":362,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.722360+0000","last_change":"2024-09-10T18:59:57.903394+0000","last_active":"2024-09-10T19:00:58.722360+0000","last_peered":"2024-09-10T19:00:58.722360+0000","last_clean":"2024-09-10T19:00:58.722360+0000","last_became_active":"2024-09-10T18:57:02.448126+0000","last_became_peered":"2024-09-10T18:57:02.448126+0000","last_unstale":"2024-09-10T19:00:58.722360+0000","last_undegraded":"2024-09-10T19:00:58.722360+0000","last_fullsized":"2024-09-10T19:00:58.722360+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:57.903311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:57.903311+0000","last_clean_scrub_stamp":"2024-09-10T18:59:57.903311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:09.007138+0000","scrub_duration":0.011228738,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":350,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.726687+0000","last_change":"2024-09-10T18:58:48.633501+0000","last_active":"2024-09-10T19:00:58.726687+0000","last_peered":"2024-09-10T19:00:58.726687+0000","last_clean":"2024-09-10T19:00:58.726687+0000","last_became_active":"2024-09-10T18:47:36.410836+0000","last_became_peered":"2024-09-10T18:47:36.410836+0000","last_unstale":"2024-09-10T19:00:58.726687+0000","last_undegraded":"2024-09-10T19:00:58.726687+0000","last_fullsized":"2024-09-10T19:00:58.726687+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:48.633423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_clean_scrub_stamp":"2024-09-10T18:58:48.633423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0079262829999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.d","version":"344'6501","reported_seq":6538,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:56.131853+0000","last_change":"2024-09-10T18:59:42.717015+0000","last_active":"2024-09-10T19:00:56.131853+0000","last_peered":"2024-09-10T19:00:56.131853+0000","last_clean":"2024-09-10T19:00:56.131853+0000","last_became_active":"2024-09-10T18:59:42.716708+0000","last_became_peered":"2024-09-10T18:59:42.716708+0000","last_unstale":"2024-09-10T19:00:56.131853+0000","last_undegraded":"2024-09-10T19:00:56.131853+0000","last_fullsized":"2024-09-10T19:00:56.131853+0000","mapping_epoch":315,"log_start":"332'3000","ondisk_log_start":"332'3000","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3501,"log_dups_size":0,"ondisk_log_size":3501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":426049536,"num_objects":6501,"num_object_clones":0,"num_object_copies":19503,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6501,"num_write_kb":416064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"70'81","reported_seq":359,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.596904+0000","last_change":"2024-09-10T18:59:23.481986+0000","last_active":"2024-09-10T19:00:54.596904+0000","last_peered":"2024-09-10T19:00:54.596904+0000","last_clean":"2024-09-10T19:00:54.596904+0000","last_became_active":"2024-09-10T18:55:40.396828+0000","last_became_peered":"2024-09-10T18:55:40.396828+0000","last_unstale":"2024-09-10T19:00:54.596904+0000","last_undegraded":"2024-09-10T19:00:54.596904+0000","last_fullsized":"2024-09-10T19:00:54.596904+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:23.481902+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:59:23.481902+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0052121340000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"232'29","reported_seq":402,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.623174+0000","last_change":"2024-09-10T18:58:58.837250+0000","last_active":"2024-09-10T19:00:55.623174+0000","last_peered":"2024-09-10T19:00:55.623174+0000","last_clean":"2024-09-10T19:00:55.623174+0000","last_became_active":"2024-09-10T18:57:02.442862+0000","last_became_peered":"2024-09-10T18:57:02.442862+0000","last_unstale":"2024-09-10T19:00:55.623174+0000","last_undegraded":"2024-09-10T19:00:55.623174+0000","last_fullsized":"2024-09-10T19:00:55.623174+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'29","last_scrub_stamp":"2024-09-10T18:58:58.837163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:58:58.837163+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.019169703999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":340,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.624006+0000","last_change":"2024-09-10T18:59:53.858914+0000","last_active":"2024-09-10T19:00:55.624006+0000","last_peered":"2024-09-10T19:00:55.624006+0000","last_clean":"2024-09-10T19:00:55.624006+0000","last_became_active":"2024-09-10T18:57:02.443522+0000","last_became_peered":"2024-09-10T18:57:02.443522+0000","last_unstale":"2024-09-10T19:00:55.624006+0000","last_undegraded":"2024-09-10T19:00:55.624006+0000","last_fullsized":"2024-09-10T19:00:55.624006+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:53.858807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:48.196065+0000","last_clean_scrub_stamp":"2024-09-10T18:59:53.858807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:17.457534+0000","scrub_duration":0.0044256060000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":369,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.595239+0000","last_change":"2024-09-10T18:59:15.511792+0000","last_active":"2024-09-10T19:00:54.595239+0000","last_peered":"2024-09-10T19:00:54.595239+0000","last_clean":"2024-09-10T19:00:54.595239+0000","last_became_active":"2024-09-10T18:55:40.387857+0000","last_became_peered":"2024-09-10T18:55:40.387857+0000","last_unstale":"2024-09-10T19:00:54.595239+0000","last_undegraded":"2024-09-10T19:00:54.595239+0000","last_fullsized":"2024-09-10T19:00:54.595239+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:15.511743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:59:15.511743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001893837,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":360,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.625250+0000","last_change":"2024-09-10T18:59:16.235312+0000","last_active":"2024-09-10T19:00:55.625250+0000","last_peered":"2024-09-10T19:00:55.625250+0000","last_clean":"2024-09-10T19:00:55.625250+0000","last_became_active":"2024-09-10T18:57:02.442479+0000","last_became_peered":"2024-09-10T18:57:02.442479+0000","last_unstale":"2024-09-10T19:00:55.625250+0000","last_undegraded":"2024-09-10T19:00:55.625250+0000","last_fullsized":"2024-09-10T19:00:55.625250+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:16.235235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:49:18.281230+0000","last_clean_scrub_stamp":"2024-09-10T18:59:16.235235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034192950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"345'6347","reported_seq":7972,"reported_epoch":345,"state":"active+clean","last_fresh":"2024-09-10T19:00:56.778480+0000","last_change":"2024-09-10T19:00:08.469551+0000","last_active":"2024-09-10T19:00:56.778480+0000","last_peered":"2024-09-10T19:00:56.778480+0000","last_clean":"2024-09-10T19:00:56.778480+0000","last_became_active":"2024-09-10T19:00:08.465795+0000","last_became_peered":"2024-09-10T19:00:08.465795+0000","last_unstale":"2024-09-10T19:00:56.778480+0000","last_undegraded":"2024-09-10T19:00:56.778480+0000","last_fullsized":"2024-09-10T19:00:56.778480+0000","mapping_epoch":329,"log_start":"332'2800","ondisk_log_start":"332'2800","created":315,"last_epoch_clean":330,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3547,"log_dups_size":0,"ondisk_log_size":3547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":415956992,"num_objects":6347,"num_object_clones":0,"num_object_copies":19041,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6347,"num_write_kb":406208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":784,"num_bytes_recovered":51380224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"70'81","reported_seq":362,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.601558+0000","last_change":"2024-09-10T18:59:53.550489+0000","last_active":"2024-09-10T19:00:54.601558+0000","last_peered":"2024-09-10T19:00:54.601558+0000","last_clean":"2024-09-10T19:00:54.601558+0000","last_became_active":"2024-09-10T18:57:02.452659+0000","last_became_peered":"2024-09-10T18:57:02.452659+0000","last_unstale":"2024-09-10T19:00:54.601558+0000","last_undegraded":"2024-09-10T19:00:54.601558+0000","last_fullsized":"2024-09-10T19:00:54.601558+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:53.550433+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:59:53.550433+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:00.096622+0000","scrub_duration":0.0013042399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"232'25","reported_seq":463,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.633689+0000","last_change":"2024-09-10T19:00:42.531653+0000","last_active":"2024-09-10T19:00:55.633689+0000","last_peered":"2024-09-10T19:00:55.633689+0000","last_clean":"2024-09-10T19:00:55.633689+0000","last_became_active":"2024-09-10T19:00:42.517353+0000","last_became_peered":"2024-09-10T19:00:42.517353+0000","last_unstale":"2024-09-10T19:00:55.633689+0000","last_undegraded":"2024-09-10T19:00:55.633689+0000","last_fullsized":"2024-09-10T19:00:55.633689+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'25","last_scrub_stamp":"2024-09-10T19:00:07.390135+0000","last_deep_scrub":"83'15","last_deep_scrub_stamp":"2024-09-10T18:46:58.436473+0000","last_clean_scrub_stamp":"2024-09-10T19:00:07.390135+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:12.882057+0000","scrub_duration":0.002339874,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":383,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.623412+0000","last_change":"2024-09-10T18:58:50.137466+0000","last_active":"2024-09-10T19:00:55.623412+0000","last_peered":"2024-09-10T19:00:55.623412+0000","last_clean":"2024-09-10T19:00:55.623412+0000","last_became_active":"2024-09-10T18:57:02.447940+0000","last_became_peered":"2024-09-10T18:57:02.447940+0000","last_unstale":"2024-09-10T19:00:55.623412+0000","last_undegraded":"2024-09-10T19:00:55.623412+0000","last_fullsized":"2024-09-10T19:00:55.623412+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:50.137393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:05.140027+0000","last_clean_scrub_stamp":"2024-09-10T18:58:50.137393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014454461,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"339'13","reported_seq":508,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:56.259507+0000","last_change":"2024-09-10T19:00:42.528804+0000","last_active":"2024-09-10T19:00:56.259507+0000","last_peered":"2024-09-10T19:00:56.259507+0000","last_clean":"2024-09-10T19:00:56.259507+0000","last_became_active":"2024-09-10T19:00:42.520360+0000","last_became_peered":"2024-09-10T19:00:42.520360+0000","last_unstale":"2024-09-10T19:00:56.259507+0000","last_undegraded":"2024-09-10T19:00:56.259507+0000","last_fullsized":"2024-09-10T19:00:56.259507+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'12","last_scrub_stamp":"2024-09-10T18:59:55.943869+0000","last_deep_scrub":"258'12","last_deep_scrub_stamp":"2024-09-10T18:56:21.098770+0000","last_clean_scrub_stamp":"2024-09-10T18:59:55.943869+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:17.541448+0000","scrub_duration":0.020416790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":326,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.619394+0000","last_change":"2024-09-10T19:00:42.515515+0000","last_active":"2024-09-10T19:00:55.619394+0000","last_peered":"2024-09-10T19:00:55.619394+0000","last_clean":"2024-09-10T19:00:55.619394+0000","last_became_active":"2024-09-10T19:00:42.515144+0000","last_became_peered":"2024-09-10T19:00:42.515144+0000","last_unstale":"2024-09-10T19:00:55.619394+0000","last_undegraded":"2024-09-10T19:00:55.619394+0000","last_fullsized":"2024-09-10T19:00:55.619394+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:57:03.353014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:57:03.353014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014199639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.a","version":"344'6631","reported_seq":6678,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:56.126021+0000","last_change":"2024-09-10T18:59:48.739701+0000","last_active":"2024-09-10T19:00:56.126021+0000","last_peered":"2024-09-10T19:00:56.126021+0000","last_clean":"2024-09-10T19:00:56.126021+0000","last_became_active":"2024-09-10T18:59:48.739415+0000","last_became_peered":"2024-09-10T18:59:48.739415+0000","last_unstale":"2024-09-10T19:00:56.126021+0000","last_undegraded":"2024-09-10T19:00:56.126021+0000","last_fullsized":"2024-09-10T19:00:56.126021+0000","mapping_epoch":320,"log_start":"332'3100","ondisk_log_start":"332'3100","created":315,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3531,"log_dups_size":0,"ondisk_log_size":3531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":434569216,"num_objects":6631,"num_object_clones":0,"num_object_copies":19893,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6631,"num_write_kb":424384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"70'81","reported_seq":293,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.593621+0000","last_change":"2024-09-10T19:00:40.472680+0000","last_active":"2024-09-10T19:00:54.593621+0000","last_peered":"2024-09-10T19:00:54.593621+0000","last_clean":"2024-09-10T19:00:54.593621+0000","last_became_active":"2024-09-10T18:55:40.392432+0000","last_became_peered":"2024-09-10T18:55:40.392432+0000","last_unstale":"2024-09-10T19:00:54.593621+0000","last_undegraded":"2024-09-10T19:00:54.593621+0000","last_fullsized":"2024-09-10T19:00:54.593621+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:00:40.472595+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:00:40.472595+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:44.413314+0000","scrub_duration":0.0072897099999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"232'41","reported_seq":630,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:56.259652+0000","last_change":"2024-09-10T18:56:18.672331+0000","last_active":"2024-09-10T19:00:56.259652+0000","last_peered":"2024-09-10T19:00:56.259652+0000","last_clean":"2024-09-10T19:00:56.259652+0000","last_became_active":"2024-09-10T18:47:36.420185+0000","last_became_peered":"2024-09-10T18:47:36.420185+0000","last_unstale":"2024-09-10T19:00:56.259652+0000","last_undegraded":"2024-09-10T19:00:56.259652+0000","last_fullsized":"2024-09-10T19:00:56.259652+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'41","last_scrub_stamp":"2024-09-10T18:56:18.672139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:18.672139+0000","objects_scrubbed":10,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.020233260999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":358,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.593286+0000","last_change":"2024-09-10T18:59:04.434037+0000","last_active":"2024-09-10T19:00:54.593286+0000","last_peered":"2024-09-10T19:00:54.593286+0000","last_clean":"2024-09-10T19:00:54.593286+0000","last_became_active":"2024-09-10T18:45:17.964494+0000","last_became_peered":"2024-09-10T18:45:17.964494+0000","last_unstale":"2024-09-10T19:00:54.593286+0000","last_undegraded":"2024-09-10T19:00:54.593286+0000","last_fullsized":"2024-09-10T19:00:54.593286+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:04.433964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:12.982599+0000","last_clean_scrub_stamp":"2024-09-10T18:59:04.433964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024364220000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":300,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.596287+0000","last_change":"2024-09-10T18:56:20.224668+0000","last_active":"2024-09-10T19:00:54.596287+0000","last_peered":"2024-09-10T19:00:54.596287+0000","last_clean":"2024-09-10T19:00:54.596287+0000","last_became_active":"2024-09-10T18:47:57.107478+0000","last_became_peered":"2024-09-10T18:47:57.107478+0000","last_unstale":"2024-09-10T19:00:54.596287+0000","last_undegraded":"2024-09-10T19:00:54.596287+0000","last_fullsized":"2024-09-10T19:00:54.596287+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:20.224562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:20.224562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.008542315,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":341,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.717359+0000","last_change":"2024-09-10T18:57:02.462009+0000","last_active":"2024-09-10T19:00:58.717359+0000","last_peered":"2024-09-10T19:00:58.717359+0000","last_clean":"2024-09-10T19:00:58.717359+0000","last_became_active":"2024-09-10T18:57:02.460594+0000","last_became_peered":"2024-09-10T18:57:02.460594+0000","last_unstale":"2024-09-10T19:00:58.717359+0000","last_undegraded":"2024-09-10T19:00:58.717359+0000","last_fullsized":"2024-09-10T19:00:58.717359+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:11.878181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:22.891084+0000","last_clean_scrub_stamp":"2024-09-10T18:56:11.878181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037255349999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"347'6866","reported_seq":12319,"reported_epoch":347,"state":"active+remapped+backfilling","last_fresh":"2024-09-10T19:01:01.023580+0000","last_change":"2024-09-10T19:00:43.855834+0000","last_active":"2024-09-10T19:01:01.023580+0000","last_peered":"2024-09-10T19:01:01.023580+0000","last_clean":"2024-09-10T19:00:41.499200+0000","last_became_active":"2024-09-10T19:00:43.833848+0000","last_became_peered":"2024-09-10T19:00:43.833848+0000","last_unstale":"2024-09-10T19:01:01.023580+0000","last_undegraded":"2024-09-10T19:01:01.023580+0000","last_fullsized":"2024-09-10T19:01:01.023580+0000","mapping_epoch":339,"log_start":"332'3200","ondisk_log_start":"332'3200","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3666,"log_dups_size":0,"ondisk_log_size":3666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:09.902486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":449970176,"num_objects":6866,"num_object_clones":0,"num_object_copies":20598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3843,"num_objects_unfound":0,"num_objects_dirty":6866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6866,"num_write_kb":439424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2701,"num_bytes_recovered":177012736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[3,4,0],"avail_no_missing":["3","0","4"],"object_location_counts":[{"shards":"0,3,4","objects":6866}],"blocked_by":[],"up_primary":5,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"70'81","reported_seq":162,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.627660+0000","last_change":"2024-09-10T18:59:11.875713+0000","last_active":"2024-09-10T19:00:55.627660+0000","last_peered":"2024-09-10T19:00:55.627660+0000","last_clean":"2024-09-10T19:00:55.627660+0000","last_became_active":"2024-09-10T18:57:02.437944+0000","last_became_peered":"2024-09-10T18:57:02.437944+0000","last_unstale":"2024-09-10T19:00:55.627660+0000","last_undegraded":"2024-09-10T19:00:55.627660+0000","last_fullsized":"2024-09-10T19:00:55.627660+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:11.875625+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:59:11.875625+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040920360000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"232'22","reported_seq":406,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.624336+0000","last_change":"2024-09-10T19:00:08.468265+0000","last_active":"2024-09-10T19:00:55.624336+0000","last_peered":"2024-09-10T19:00:55.624336+0000","last_clean":"2024-09-10T19:00:55.624336+0000","last_became_active":"2024-09-10T18:55:40.397422+0000","last_became_peered":"2024-09-10T18:55:40.397422+0000","last_unstale":"2024-09-10T19:00:55.624336+0000","last_undegraded":"2024-09-10T19:00:55.624336+0000","last_fullsized":"2024-09-10T19:00:55.624336+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'22","last_scrub_stamp":"2024-09-10T19:00:08.468154+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-10T18:47:10.514951+0000","last_clean_scrub_stamp":"2024-09-10T19:00:08.468154+0000","objects_scrubbed":5,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:28.777079+0000","scrub_duration":0.0024987170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"84'6","reported_seq":1312,"reported_epoch":345,"state":"active+clean","last_fresh":"2024-09-10T19:00:56.621339+0000","last_change":"2024-09-10T19:00:40.504252+0000","last_active":"2024-09-10T19:00:56.621339+0000","last_peered":"2024-09-10T19:00:56.621339+0000","last_clean":"2024-09-10T19:00:56.621339+0000","last_became_active":"2024-09-10T18:55:40.403098+0000","last_became_peered":"2024-09-10T18:55:40.403098+0000","last_unstale":"2024-09-10T19:00:56.621339+0000","last_undegraded":"2024-09-10T19:00:56.621339+0000","last_fullsized":"2024-09-10T19:00:56.621339+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'6","last_scrub_stamp":"2024-09-10T19:00:40.504174+0000","last_deep_scrub":"84'6","last_deep_scrub_stamp":"2024-09-10T18:47:22.545450+0000","last_clean_scrub_stamp":"2024-09-10T19:00:40.504174+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:02:03.384361+0000","scrub_duration":0.0052333889999999997,"objects_trimmed":0,"snaptrim_duration":0.0021933159999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":922,"num_read_kb":917,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":325,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.718304+0000","last_change":"2024-09-10T19:00:08.141664+0000","last_active":"2024-09-10T19:00:58.718304+0000","last_peered":"2024-09-10T19:00:58.718304+0000","last_clean":"2024-09-10T19:00:58.718304+0000","last_became_active":"2024-09-10T18:55:40.393389+0000","last_became_peered":"2024-09-10T18:55:40.393389+0000","last_unstale":"2024-09-10T19:00:58.718304+0000","last_undegraded":"2024-09-10T19:00:58.718304+0000","last_fullsized":"2024-09-10T19:00:58.718304+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:08.141603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:00:08.141603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:09.834900+0000","scrub_duration":0.001270729,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":329,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.595886+0000","last_change":"2024-09-10T18:58:58.522346+0000","last_active":"2024-09-10T19:00:54.595886+0000","last_peered":"2024-09-10T19:00:54.595886+0000","last_clean":"2024-09-10T19:00:54.595886+0000","last_became_active":"2024-09-10T18:57:02.438945+0000","last_became_peered":"2024-09-10T18:57:02.438945+0000","last_unstale":"2024-09-10T19:00:54.595886+0000","last_undegraded":"2024-09-10T19:00:54.595886+0000","last_fullsized":"2024-09-10T19:00:54.595886+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:58.522295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:58:58.522295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0059618290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":96,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.596459+0000","last_change":"2024-09-10T19:00:41.794168+0000","last_active":"2024-09-10T19:00:54.596459+0000","last_peered":"2024-09-10T19:00:54.596459+0000","last_clean":"2024-09-10T19:00:54.596459+0000","last_became_active":"2024-09-10T18:57:02.453865+0000","last_became_peered":"2024-09-10T18:57:02.453865+0000","last_unstale":"2024-09-10T19:00:54.596459+0000","last_undegraded":"2024-09-10T19:00:54.596459+0000","last_fullsized":"2024-09-10T19:00:54.596459+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:41.794116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:07.150480+0000","last_clean_scrub_stamp":"2024-09-10T19:00:41.794116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:45.662042+0000","scrub_duration":0.001296148,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.8","version":"345'6688","reported_seq":6726,"reported_epoch":345,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.691925+0000","last_change":"2024-09-10T18:59:42.712702+0000","last_active":"2024-09-10T19:00:58.691925+0000","last_peered":"2024-09-10T19:00:58.691925+0000","last_clean":"2024-09-10T19:00:58.691925+0000","last_became_active":"2024-09-10T18:59:42.712397+0000","last_became_peered":"2024-09-10T18:59:42.712397+0000","last_unstale":"2024-09-10T19:00:58.691925+0000","last_undegraded":"2024-09-10T19:00:58.691925+0000","last_fullsized":"2024-09-10T19:00:58.691925+0000","mapping_epoch":315,"log_start":"332'3000","ondisk_log_start":"332'3000","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3688,"log_dups_size":0,"ondisk_log_size":3688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":438304768,"num_objects":6688,"num_object_clones":0,"num_object_copies":20064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6688,"num_write_kb":428032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,4],"acting":[0,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"70'81","reported_seq":263,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.623007+0000","last_change":"2024-09-10T18:58:49.141636+0000","last_active":"2024-09-10T19:00:55.623007+0000","last_peered":"2024-09-10T19:00:55.623007+0000","last_clean":"2024-09-10T19:00:55.623007+0000","last_became_active":"2024-09-10T18:52:14.589009+0000","last_became_peered":"2024-09-10T18:52:14.589009+0000","last_unstale":"2024-09-10T19:00:55.623007+0000","last_undegraded":"2024-09-10T19:00:55.623007+0000","last_fullsized":"2024-09-10T19:00:55.623007+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:58:49.141013+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:58:49.141013+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0036869540000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"232'24","reported_seq":82,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.616966+0000","last_change":"2024-09-10T19:00:42.532827+0000","last_active":"2024-09-10T19:00:55.616966+0000","last_peered":"2024-09-10T19:00:55.616966+0000","last_clean":"2024-09-10T19:00:55.616966+0000","last_became_active":"2024-09-10T19:00:42.516521+0000","last_became_peered":"2024-09-10T19:00:42.516521+0000","last_unstale":"2024-09-10T19:00:55.616966+0000","last_undegraded":"2024-09-10T19:00:55.616966+0000","last_fullsized":"2024-09-10T19:00:55.616966+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T19:00:39.748509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:00:39.748509+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:02:03.931650+0000","scrub_duration":0.0027783730000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"305'2","reported_seq":329,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.622189+0000","last_change":"2024-09-10T18:59:18.897245+0000","last_active":"2024-09-10T19:00:55.622189+0000","last_peered":"2024-09-10T19:00:55.622189+0000","last_clean":"2024-09-10T19:00:55.622189+0000","last_became_active":"2024-09-10T18:55:40.390022+0000","last_became_peered":"2024-09-10T18:55:40.390022+0000","last_unstale":"2024-09-10T19:00:55.622189+0000","last_undegraded":"2024-09-10T19:00:55.622189+0000","last_fullsized":"2024-09-10T19:00:55.622189+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"305'2","last_scrub_stamp":"2024-09-10T18:59:18.897059+0000","last_deep_scrub":"305'2","last_deep_scrub_stamp":"2024-09-10T18:59:18.897059+0000","last_clean_scrub_stamp":"2024-09-10T18:59:18.897059+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016452700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":290,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:56.482770+0000","last_change":"2024-09-10T19:00:56.482753+0000","last_active":"2024-09-10T19:00:56.482770+0000","last_peered":"2024-09-10T19:00:56.482770+0000","last_clean":"2024-09-10T19:00:56.482770+0000","last_became_active":"2024-09-10T18:57:02.447195+0000","last_became_peered":"2024-09-10T18:57:02.447195+0000","last_unstale":"2024-09-10T19:00:56.482770+0000","last_undegraded":"2024-09-10T19:00:56.482770+0000","last_fullsized":"2024-09-10T19:00:56.482770+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:24.153973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:24.153973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001771613,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":15,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.594430+0000","last_change":"2024-09-10T19:00:42.518887+0000","last_active":"2024-09-10T19:00:54.594430+0000","last_peered":"2024-09-10T19:00:54.594430+0000","last_clean":"2024-09-10T19:00:54.594430+0000","last_became_active":"2024-09-10T19:00:42.518566+0000","last_became_peered":"2024-09-10T19:00:42.518566+0000","last_unstale":"2024-09-10T19:00:54.594430+0000","last_undegraded":"2024-09-10T19:00:54.594430+0000","last_fullsized":"2024-09-10T19:00:54.594430+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:39.752722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:00:39.752722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:44.640996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":343,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.622984+0000","last_change":"2024-09-10T18:58:52.136201+0000","last_active":"2024-09-10T19:00:55.622984+0000","last_peered":"2024-09-10T19:00:55.622984+0000","last_clean":"2024-09-10T19:00:55.622984+0000","last_became_active":"2024-09-10T18:45:24.083689+0000","last_became_peered":"2024-09-10T18:45:24.083689+0000","last_unstale":"2024-09-10T19:00:55.622984+0000","last_undegraded":"2024-09-10T19:00:55.622984+0000","last_fullsized":"2024-09-10T19:00:55.622984+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:52.136095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:58:52.136095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.021731822000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"344'6516","reported_seq":6553,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:56.058090+0000","last_change":"2024-09-10T18:59:42.717031+0000","last_active":"2024-09-10T19:00:56.058090+0000","last_peered":"2024-09-10T19:00:56.058090+0000","last_clean":"2024-09-10T19:00:56.058090+0000","last_became_active":"2024-09-10T18:59:42.716730+0000","last_became_peered":"2024-09-10T18:59:42.716730+0000","last_unstale":"2024-09-10T19:00:56.058090+0000","last_undegraded":"2024-09-10T19:00:56.058090+0000","last_fullsized":"2024-09-10T19:00:56.058090+0000","mapping_epoch":315,"log_start":"332'3000","ondisk_log_start":"332'3000","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3516,"log_dups_size":0,"ondisk_log_size":3516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":427032576,"num_objects":6516,"num_object_clones":0,"num_object_copies":19548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6516,"num_write_kb":417024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"232'33","reported_seq":437,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.596472+0000","last_change":"2024-09-10T18:59:17.515699+0000","last_active":"2024-09-10T19:00:54.596472+0000","last_peered":"2024-09-10T19:00:54.596472+0000","last_clean":"2024-09-10T19:00:54.596472+0000","last_became_active":"2024-09-10T18:57:02.435499+0000","last_became_peered":"2024-09-10T18:57:02.435499+0000","last_unstale":"2024-09-10T19:00:54.596472+0000","last_undegraded":"2024-09-10T19:00:54.596472+0000","last_fullsized":"2024-09-10T19:00:54.596472+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'33","last_scrub_stamp":"2024-09-10T18:59:17.515615+0000","last_deep_scrub":"232'33","last_deep_scrub_stamp":"2024-09-10T18:56:21.206960+0000","last_clean_scrub_stamp":"2024-09-10T18:59:17.515615+0000","objects_scrubbed":7,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.009416209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"60'2","reported_seq":360,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.717850+0000","last_change":"2024-09-10T18:58:55.931801+0000","last_active":"2024-09-10T19:00:58.717850+0000","last_peered":"2024-09-10T19:00:58.717850+0000","last_clean":"2024-09-10T19:00:58.717850+0000","last_became_active":"2024-09-10T18:45:17.961660+0000","last_became_peered":"2024-09-10T18:45:17.961660+0000","last_unstale":"2024-09-10T19:00:58.717850+0000","last_undegraded":"2024-09-10T19:00:58.717850+0000","last_fullsized":"2024-09-10T19:00:58.717850+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-10T18:58:55.931717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:58:55.931717+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040038909999999999,"objects_trimmed":0,"snaptrim_duration":0.0013942640000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":375,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.594786+0000","last_change":"2024-09-10T18:57:02.446028+0000","last_active":"2024-09-10T19:00:54.594786+0000","last_peered":"2024-09-10T19:00:54.594786+0000","last_clean":"2024-09-10T19:00:54.594786+0000","last_became_active":"2024-09-10T18:57:02.445712+0000","last_became_peered":"2024-09-10T18:57:02.445712+0000","last_unstale":"2024-09-10T19:00:54.594786+0000","last_undegraded":"2024-09-10T19:00:54.594786+0000","last_fullsized":"2024-09-10T19:00:54.594786+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:33.559651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:33.559651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0036787759999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'81","reported_seq":162,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.616321+0000","last_change":"2024-09-10T18:59:14.873293+0000","last_active":"2024-09-10T19:00:55.616321+0000","last_peered":"2024-09-10T19:00:55.616321+0000","last_clean":"2024-09-10T19:00:55.616321+0000","last_became_active":"2024-09-10T18:57:02.440063+0000","last_became_peered":"2024-09-10T18:57:02.440063+0000","last_unstale":"2024-09-10T19:00:55.616321+0000","last_undegraded":"2024-09-10T19:00:55.616321+0000","last_fullsized":"2024-09-10T19:00:55.616321+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:14.873083+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:56.085580+0000","last_clean_scrub_stamp":"2024-09-10T18:59:14.873083+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0063632760000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":344,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.594935+0000","last_change":"2024-09-10T18:57:02.432025+0000","last_active":"2024-09-10T19:00:54.594935+0000","last_peered":"2024-09-10T19:00:54.594935+0000","last_clean":"2024-09-10T19:00:54.594935+0000","last_became_active":"2024-09-10T18:57:02.431556+0000","last_became_peered":"2024-09-10T18:57:02.431556+0000","last_unstale":"2024-09-10T19:00:54.594935+0000","last_undegraded":"2024-09-10T19:00:54.594935+0000","last_fullsized":"2024-09-10T19:00:54.594935+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:27.012123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:27.012123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007606036,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":359,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.595069+0000","last_change":"2024-09-10T19:00:07.624756+0000","last_active":"2024-09-10T19:00:54.595069+0000","last_peered":"2024-09-10T19:00:54.595069+0000","last_clean":"2024-09-10T19:00:54.595069+0000","last_became_active":"2024-09-10T18:45:24.073327+0000","last_became_peered":"2024-09-10T18:45:24.073327+0000","last_unstale":"2024-09-10T19:00:54.595069+0000","last_undegraded":"2024-09-10T19:00:54.595069+0000","last_fullsized":"2024-09-10T19:00:54.595069+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:07.624701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:00:07.624701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:29.473904+0000","scrub_duration":0.001137962,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"345'6512","reported_seq":6550,"reported_epoch":345,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.710200+0000","last_change":"2024-09-10T18:59:43.009491+0000","last_active":"2024-09-10T19:00:58.710200+0000","last_peered":"2024-09-10T19:00:58.710200+0000","last_clean":"2024-09-10T19:00:58.710200+0000","last_became_active":"2024-09-10T18:59:43.009210+0000","last_became_peered":"2024-09-10T18:59:43.009210+0000","last_unstale":"2024-09-10T19:00:58.710200+0000","last_undegraded":"2024-09-10T19:00:58.710200+0000","last_fullsized":"2024-09-10T19:00:58.710200+0000","mapping_epoch":315,"log_start":"332'2800","ondisk_log_start":"332'2800","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3712,"log_dups_size":0,"ondisk_log_size":3712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:06.741150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":426770432,"num_objects":6512,"num_object_clones":0,"num_object_copies":19536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6512,"num_write_kb":416768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"70'81","reported_seq":354,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.601116+0000","last_change":"2024-09-10T18:58:59.582622+0000","last_active":"2024-09-10T19:00:54.601116+0000","last_peered":"2024-09-10T19:00:54.601116+0000","last_clean":"2024-09-10T19:00:54.601116+0000","last_became_active":"2024-09-10T18:52:16.598295+0000","last_became_peered":"2024-09-10T18:52:16.598295+0000","last_unstale":"2024-09-10T19:00:54.601116+0000","last_undegraded":"2024-09-10T19:00:54.601116+0000","last_fullsized":"2024-09-10T19:00:54.601116+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:58:59.582565+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:53.609885+0000","last_clean_scrub_stamp":"2024-09-10T18:58:59.582565+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046448920000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"232'35","reported_seq":452,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.595402+0000","last_change":"2024-09-10T19:00:43.819111+0000","last_active":"2024-09-10T19:00:54.595402+0000","last_peered":"2024-09-10T19:00:54.595402+0000","last_clean":"2024-09-10T19:00:54.595402+0000","last_became_active":"2024-09-10T18:57:02.436059+0000","last_became_peered":"2024-09-10T18:57:02.436059+0000","last_unstale":"2024-09-10T19:00:54.595402+0000","last_undegraded":"2024-09-10T19:00:54.595402+0000","last_fullsized":"2024-09-10T19:00:54.595402+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'35","last_scrub_stamp":"2024-09-10T19:00:43.819004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:00:43.819004+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:02:08.320454+0000","scrub_duration":0.0050234040000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":323,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.595004+0000","last_change":"2024-09-10T18:56:13.760318+0000","last_active":"2024-09-10T19:00:54.595004+0000","last_peered":"2024-09-10T19:00:54.595004+0000","last_clean":"2024-09-10T19:00:54.595004+0000","last_became_active":"2024-09-10T18:55:40.391849+0000","last_became_peered":"2024-09-10T18:55:40.391849+0000","last_unstale":"2024-09-10T19:00:54.595004+0000","last_undegraded":"2024-09-10T19:00:54.595004+0000","last_fullsized":"2024-09-10T19:00:54.595004+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:13.760238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:13.760238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.048619793000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":305,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.622900+0000","last_change":"2024-09-10T19:00:42.738386+0000","last_active":"2024-09-10T19:00:55.622900+0000","last_peered":"2024-09-10T19:00:55.622900+0000","last_clean":"2024-09-10T19:00:55.622900+0000","last_became_active":"2024-09-10T19:00:42.514709+0000","last_became_peered":"2024-09-10T19:00:42.514709+0000","last_unstale":"2024-09-10T19:00:55.622900+0000","last_undegraded":"2024-09-10T19:00:55.622900+0000","last_fullsized":"2024-09-10T19:00:55.622900+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:42.738331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:00:42.738331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:02:08.495978+0000","scrub_duration":0.001211314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":144,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.716736+0000","last_change":"2024-09-10T18:57:03.015513+0000","last_active":"2024-09-10T19:00:58.716736+0000","last_peered":"2024-09-10T19:00:58.716736+0000","last_clean":"2024-09-10T19:00:58.716736+0000","last_became_active":"2024-09-10T18:57:02.468866+0000","last_became_peered":"2024-09-10T18:57:02.468866+0000","last_unstale":"2024-09-10T19:00:58.716736+0000","last_undegraded":"2024-09-10T19:00:58.716736+0000","last_fullsized":"2024-09-10T19:00:58.716736+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:57:03.015457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:57:03.015457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001545795,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.5","version":"345'6600","reported_seq":10690,"reported_epoch":345,"state":"active+remapped+backfilling","last_fresh":"2024-09-10T19:00:58.706715+0000","last_change":"2024-09-10T19:00:43.858699+0000","last_active":"2024-09-10T19:00:58.706715+0000","last_peered":"2024-09-10T19:00:58.706715+0000","last_clean":"2024-09-10T19:00:41.500440+0000","last_became_active":"2024-09-10T19:00:43.839525+0000","last_became_peered":"2024-09-10T19:00:43.839525+0000","last_unstale":"2024-09-10T19:00:58.706715+0000","last_undegraded":"2024-09-10T19:00:58.706715+0000","last_fullsized":"2024-09-10T19:00:58.706715+0000","mapping_epoch":339,"log_start":"332'2900","ondisk_log_start":"332'2900","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3700,"log_dups_size":0,"ondisk_log_size":3700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":432537600,"num_objects":6600,"num_object_clones":0,"num_object_copies":19800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4352,"num_objects_unfound":0,"num_objects_dirty":6600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6600,"num_write_kb":422400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2020,"num_bytes_recovered":132382720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,4],"avail_no_missing":["0","3","4"],"object_location_counts":[{"shards":"0,3,4","objects":6600}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.e","version":"70'81","reported_seq":225,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.725315+0000","last_change":"2024-09-10T18:59:23.592755+0000","last_active":"2024-09-10T19:00:58.725315+0000","last_peered":"2024-09-10T19:00:58.725315+0000","last_clean":"2024-09-10T19:00:58.725315+0000","last_became_active":"2024-09-10T18:57:02.449115+0000","last_became_peered":"2024-09-10T18:57:02.449115+0000","last_unstale":"2024-09-10T19:00:58.725315+0000","last_undegraded":"2024-09-10T19:00:58.725315+0000","last_fullsized":"2024-09-10T19:00:58.725315+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:23.592672+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:59:23.592672+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038976470000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"232'19","reported_seq":361,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.623255+0000","last_change":"2024-09-10T19:00:40.748808+0000","last_active":"2024-09-10T19:00:55.623255+0000","last_peered":"2024-09-10T19:00:55.623255+0000","last_clean":"2024-09-10T19:00:55.623255+0000","last_became_active":"2024-09-10T18:55:40.395257+0000","last_became_peered":"2024-09-10T18:55:40.395257+0000","last_unstale":"2024-09-10T19:00:55.623255+0000","last_undegraded":"2024-09-10T19:00:55.623255+0000","last_fullsized":"2024-09-10T19:00:55.623255+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T19:00:40.748713+0000","last_deep_scrub":"130'13","last_deep_scrub_stamp":"2024-09-10T18:51:58.427534+0000","last_clean_scrub_stamp":"2024-09-10T19:00:40.748713+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:02:08.421967+0000","scrub_duration":0.0043426029999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":346,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.623315+0000","last_change":"2024-09-10T18:59:09.206400+0000","last_active":"2024-09-10T19:00:55.623315+0000","last_peered":"2024-09-10T19:00:55.623315+0000","last_clean":"2024-09-10T19:00:55.623315+0000","last_became_active":"2024-09-10T18:45:17.963057+0000","last_became_peered":"2024-09-10T18:45:17.963057+0000","last_unstale":"2024-09-10T19:00:55.623315+0000","last_undegraded":"2024-09-10T19:00:55.623315+0000","last_fullsized":"2024-09-10T19:00:55.623315+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:09.206317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:59:09.206317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.018986050000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":314,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.726545+0000","last_change":"2024-09-10T18:56:17.661917+0000","last_active":"2024-09-10T19:00:58.726545+0000","last_peered":"2024-09-10T19:00:58.726545+0000","last_clean":"2024-09-10T19:00:58.726545+0000","last_became_active":"2024-09-10T18:55:40.390771+0000","last_became_peered":"2024-09-10T18:55:40.390771+0000","last_unstale":"2024-09-10T19:00:58.726545+0000","last_undegraded":"2024-09-10T19:00:58.726545+0000","last_fullsized":"2024-09-10T19:00:58.726545+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:17.661832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_clean_scrub_stamp":"2024-09-10T18:56:17.661832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041460890000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":339,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.621827+0000","last_change":"2024-09-10T18:57:02.453927+0000","last_active":"2024-09-10T19:00:55.621827+0000","last_peered":"2024-09-10T19:00:55.621827+0000","last_clean":"2024-09-10T19:00:55.621827+0000","last_became_active":"2024-09-10T18:57:02.452387+0000","last_became_peered":"2024-09-10T18:57:02.452387+0000","last_unstale":"2024-09-10T19:00:55.621827+0000","last_undegraded":"2024-09-10T19:00:55.621827+0000","last_fullsized":"2024-09-10T19:00:55.621827+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:32.217641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:53.345184+0000","last_clean_scrub_stamp":"2024-09-10T18:56:32.217641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0059447470000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"344'6443","reported_seq":6478,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:56.543032+0000","last_change":"2024-09-10T18:59:42.715562+0000","last_active":"2024-09-10T19:00:56.543032+0000","last_peered":"2024-09-10T19:00:56.543032+0000","last_clean":"2024-09-10T19:00:56.543032+0000","last_became_active":"2024-09-10T18:59:42.715275+0000","last_became_peered":"2024-09-10T18:59:42.715275+0000","last_unstale":"2024-09-10T19:00:56.543032+0000","last_undegraded":"2024-09-10T19:00:56.543032+0000","last_fullsized":"2024-09-10T19:00:56.543032+0000","mapping_epoch":315,"log_start":"332'2844","ondisk_log_start":"332'2844","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3599,"log_dups_size":0,"ondisk_log_size":3599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:58.015138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":422248448,"num_objects":6443,"num_object_clones":0,"num_object_copies":19329,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6443,"num_write_kb":412352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.d","version":"70'81","reported_seq":156,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.722299+0000","last_change":"2024-09-10T19:00:54.102663+0000","last_active":"2024-09-10T19:00:58.722299+0000","last_peered":"2024-09-10T19:00:58.722299+0000","last_clean":"2024-09-10T19:00:58.722299+0000","last_became_active":"2024-09-10T18:57:02.439652+0000","last_became_peered":"2024-09-10T18:57:02.439652+0000","last_unstale":"2024-09-10T19:00:58.722299+0000","last_undegraded":"2024-09-10T19:00:58.722299+0000","last_fullsized":"2024-09-10T19:00:58.722299+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:18.041116+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:18.041116+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014613810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"232'27","reported_seq":31,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.596169+0000","last_change":"2024-09-10T19:00:42.533625+0000","last_active":"2024-09-10T19:00:54.596169+0000","last_peered":"2024-09-10T19:00:54.596169+0000","last_clean":"2024-09-10T19:00:54.596169+0000","last_became_active":"2024-09-10T19:00:42.515903+0000","last_became_peered":"2024-09-10T19:00:42.515903+0000","last_unstale":"2024-09-10T19:00:54.596169+0000","last_undegraded":"2024-09-10T19:00:54.596169+0000","last_fullsized":"2024-09-10T19:00:54.596169+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'27","last_scrub_stamp":"2024-09-10T18:58:47.595256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:58:47.595256+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014176382,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":33,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":308,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.717504+0000","last_change":"2024-09-10T18:57:02.463092+0000","last_active":"2024-09-10T19:00:58.717504+0000","last_peered":"2024-09-10T19:00:58.717504+0000","last_clean":"2024-09-10T19:00:58.717504+0000","last_became_active":"2024-09-10T18:57:02.461708+0000","last_became_peered":"2024-09-10T18:57:02.461708+0000","last_unstale":"2024-09-10T19:00:58.717504+0000","last_undegraded":"2024-09-10T19:00:58.717504+0000","last_fullsized":"2024-09-10T19:00:58.717504+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:24.902205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:05.603125+0000","last_clean_scrub_stamp":"2024-09-10T18:56:24.902205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0076874830000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"273'10","reported_seq":215,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.624998+0000","last_change":"2024-09-10T18:59:11.568683+0000","last_active":"2024-09-10T19:00:55.624998+0000","last_peered":"2024-09-10T19:00:55.624998+0000","last_clean":"2024-09-10T19:00:55.624998+0000","last_became_active":"2024-09-10T18:57:02.441890+0000","last_became_peered":"2024-09-10T18:57:02.441890+0000","last_unstale":"2024-09-10T19:00:55.624998+0000","last_undegraded":"2024-09-10T19:00:55.624998+0000","last_fullsized":"2024-09-10T19:00:55.624998+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'10","last_scrub_stamp":"2024-09-10T18:59:11.568599+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-09-10T18:47:02.582229+0000","last_clean_scrub_stamp":"2024-09-10T18:59:11.568599+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0054658950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":366,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.593688+0000","last_change":"2024-09-10T18:59:51.487816+0000","last_active":"2024-09-10T19:00:54.593688+0000","last_peered":"2024-09-10T19:00:54.593688+0000","last_clean":"2024-09-10T19:00:54.593688+0000","last_became_active":"2024-09-10T18:55:40.389202+0000","last_became_peered":"2024-09-10T18:55:40.389202+0000","last_unstale":"2024-09-10T19:00:54.593688+0000","last_undegraded":"2024-09-10T19:00:54.593688+0000","last_fullsized":"2024-09-10T19:00:54.593688+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:51.487763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_clean_scrub_stamp":"2024-09-10T18:59:51.487763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:04.421280+0000","scrub_duration":0.001471904,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"344'6502","reported_seq":10881,"reported_epoch":344,"state":"active+remapped+backfilling","last_fresh":"2024-09-10T19:00:56.264103+0000","last_change":"2024-09-10T19:00:43.876105+0000","last_active":"2024-09-10T19:00:56.264103+0000","last_peered":"2024-09-10T19:00:56.264103+0000","last_clean":"2024-09-10T19:00:41.508090+0000","last_became_active":"2024-09-10T19:00:43.837692+0000","last_became_peered":"2024-09-10T19:00:43.837692+0000","last_unstale":"2024-09-10T19:00:56.264103+0000","last_undegraded":"2024-09-10T19:00:56.264103+0000","last_fullsized":"2024-09-10T19:00:56.264103+0000","mapping_epoch":339,"log_start":"332'3000","ondisk_log_start":"332'3000","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3502,"log_dups_size":0,"ondisk_log_size":3502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:00:58.153082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":426115072,"num_objects":6502,"num_object_clones":0,"num_object_copies":19506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4132,"num_objects_unfound":0,"num_objects_dirty":6502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6502,"num_write_kb":416128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2164,"num_bytes_recovered":141819904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,4],"avail_no_missing":["6","3","4"],"object_location_counts":[{"shards":"3,4,6","objects":6502}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.c","version":"70'81","reported_seq":301,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.718211+0000","last_change":"2024-09-10T18:56:33.779212+0000","last_active":"2024-09-10T19:00:58.718211+0000","last_peered":"2024-09-10T19:00:58.718211+0000","last_clean":"2024-09-10T19:00:58.718211+0000","last_became_active":"2024-09-10T18:55:40.390655+0000","last_became_peered":"2024-09-10T18:55:40.390655+0000","last_unstale":"2024-09-10T19:00:58.718211+0000","last_undegraded":"2024-09-10T19:00:58.718211+0000","last_fullsized":"2024-09-10T19:00:58.718211+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:33.779154+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:56:33.779154+0000","last_clean_scrub_stamp":"2024-09-10T18:56:33.779154+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0075121179999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"232'18","reported_seq":379,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.722558+0000","last_change":"2024-09-10T19:00:51.009315+0000","last_active":"2024-09-10T19:00:58.722558+0000","last_peered":"2024-09-10T19:00:58.722558+0000","last_clean":"2024-09-10T19:00:58.722558+0000","last_became_active":"2024-09-10T18:45:20.056869+0000","last_became_peered":"2024-09-10T18:45:20.056869+0000","last_unstale":"2024-09-10T19:00:58.722558+0000","last_undegraded":"2024-09-10T19:00:58.722558+0000","last_fullsized":"2024-09-10T19:00:58.722558+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'18","last_scrub_stamp":"2024-09-10T18:55:27.507364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:55:27.507364+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030221390000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"56'2","reported_seq":389,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-10T19:01:01.185585+0000","last_change":"2024-09-10T19:01:01.185585+0000","last_active":"2024-09-10T19:01:01.185585+0000","last_peered":"2024-09-10T19:01:01.185585+0000","last_clean":"2024-09-10T19:01:01.185585+0000","last_became_active":"2024-09-10T18:57:02.434475+0000","last_became_peered":"2024-09-10T18:57:02.434475+0000","last_unstale":"2024-09-10T19:01:01.185585+0000","last_undegraded":"2024-09-10T19:01:01.185585+0000","last_fullsized":"2024-09-10T19:01:01.185585+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-10T18:56:01.630091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:01.630091+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001578238,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":305,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.619318+0000","last_change":"2024-09-10T19:00:42.515054+0000","last_active":"2024-09-10T19:00:55.619318+0000","last_peered":"2024-09-10T19:00:55.619318+0000","last_clean":"2024-09-10T19:00:55.619318+0000","last_became_active":"2024-09-10T19:00:42.513796+0000","last_became_peered":"2024-09-10T19:00:42.513796+0000","last_unstale":"2024-09-10T19:00:55.619318+0000","last_undegraded":"2024-09-10T19:00:55.619318+0000","last_fullsized":"2024-09-10T19:00:55.619318+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:54.899134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:02.711594+0000","last_clean_scrub_stamp":"2024-09-10T18:59:54.899134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:15.700320+0000","scrub_duration":0.0051739739999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":351,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.717427+0000","last_change":"2024-09-10T18:59:11.827181+0000","last_active":"2024-09-10T19:00:58.717427+0000","last_peered":"2024-09-10T19:00:58.717427+0000","last_clean":"2024-09-10T19:00:58.717427+0000","last_became_active":"2024-09-10T18:57:02.459985+0000","last_became_peered":"2024-09-10T18:57:02.459985+0000","last_unstale":"2024-09-10T19:00:58.717427+0000","last_undegraded":"2024-09-10T19:00:58.717427+0000","last_fullsized":"2024-09-10T19:00:58.717427+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:11.827101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:53.352427+0000","last_clean_scrub_stamp":"2024-09-10T18:59:11.827101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034557899999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.13","version":"70'81","reported_seq":275,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.623161+0000","last_change":"2024-09-10T19:00:49.472063+0000","last_active":"2024-09-10T19:00:55.623161+0000","last_peered":"2024-09-10T19:00:55.623161+0000","last_clean":"2024-09-10T19:00:55.623161+0000","last_became_active":"2024-09-10T18:52:16.602463+0000","last_became_peered":"2024-09-10T18:52:16.602463+0000","last_unstale":"2024-09-10T19:00:55.623161+0000","last_undegraded":"2024-09-10T19:00:55.623161+0000","last_fullsized":"2024-09-10T19:00:55.623161+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:34.760031+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:34.760031+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002735595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"232'19","reported_seq":374,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.619118+0000","last_change":"2024-09-10T18:57:02.485836+0000","last_active":"2024-09-10T19:00:55.619118+0000","last_peered":"2024-09-10T19:00:55.619118+0000","last_clean":"2024-09-10T19:00:55.619118+0000","last_became_active":"2024-09-10T18:57:02.453001+0000","last_became_peered":"2024-09-10T18:57:02.453001+0000","last_unstale":"2024-09-10T19:00:55.619118+0000","last_undegraded":"2024-09-10T19:00:55.619118+0000","last_fullsized":"2024-09-10T19:00:55.619118+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T18:56:25.122070+0000","last_deep_scrub":"76'13","last_deep_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_clean_scrub_stamp":"2024-09-10T18:56:25.122070+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0061142590000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":334,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.620378+0000","last_change":"2024-09-10T18:59:59.385405+0000","last_active":"2024-09-10T19:00:55.620378+0000","last_peered":"2024-09-10T19:00:55.620378+0000","last_clean":"2024-09-10T19:00:55.620378+0000","last_became_active":"2024-09-10T18:55:40.401037+0000","last_became_peered":"2024-09-10T18:55:40.401037+0000","last_unstale":"2024-09-10T19:00:55.620378+0000","last_undegraded":"2024-09-10T19:00:55.620378+0000","last_fullsized":"2024-09-10T19:00:55.620378+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:59.385349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:59:59.385349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:08.461235+0000","scrub_duration":0.00163496,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":353,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.718943+0000","last_change":"2024-09-10T18:57:02.455638+0000","last_active":"2024-09-10T19:00:58.718943+0000","last_peered":"2024-09-10T19:00:58.718943+0000","last_clean":"2024-09-10T19:00:58.718943+0000","last_became_active":"2024-09-10T18:57:02.455341+0000","last_became_peered":"2024-09-10T18:57:02.455341+0000","last_unstale":"2024-09-10T19:00:58.718943+0000","last_undegraded":"2024-09-10T19:00:58.718943+0000","last_fullsized":"2024-09-10T19:00:58.718943+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:25.862853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:00.754912+0000","last_clean_scrub_stamp":"2024-09-10T18:56:25.862853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0033952399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"63'1","reported_seq":372,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.721496+0000","last_change":"2024-09-10T18:59:48.746423+0000","last_active":"2024-09-10T19:00:58.721496+0000","last_peered":"2024-09-10T19:00:58.721496+0000","last_clean":"2024-09-10T19:00:58.721496+0000","last_became_active":"2024-09-10T18:59:48.741275+0000","last_became_peered":"2024-09-10T18:59:48.741275+0000","last_unstale":"2024-09-10T19:00:58.721496+0000","last_undegraded":"2024-09-10T19:00:58.721496+0000","last_fullsized":"2024-09-10T19:00:58.721496+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-10T18:56:07.774334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:07.774334+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.090386770000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":52,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.12","version":"70'81","reported_seq":213,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.593580+0000","last_change":"2024-09-10T18:58:52.543604+0000","last_active":"2024-09-10T19:00:54.593580+0000","last_peered":"2024-09-10T19:00:54.593580+0000","last_clean":"2024-09-10T19:00:54.593580+0000","last_became_active":"2024-09-10T18:52:16.598921+0000","last_became_peered":"2024-09-10T18:52:16.598921+0000","last_unstale":"2024-09-10T19:00:54.593580+0000","last_undegraded":"2024-09-10T19:00:54.593580+0000","last_fullsized":"2024-09-10T19:00:54.593580+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:58:52.543544+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:54.127476+0000","last_clean_scrub_stamp":"2024-09-10T18:58:52.543544+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003920644,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":379,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.595443+0000","last_change":"2024-09-10T18:59:05.556377+0000","last_active":"2024-09-10T19:00:54.595443+0000","last_peered":"2024-09-10T19:00:54.595443+0000","last_clean":"2024-09-10T19:00:54.595443+0000","last_became_active":"2024-09-10T18:57:02.442251+0000","last_became_peered":"2024-09-10T18:57:02.442251+0000","last_unstale":"2024-09-10T19:00:54.595443+0000","last_undegraded":"2024-09-10T19:00:54.595443+0000","last_fullsized":"2024-09-10T19:00:54.595443+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:05.556292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:59:05.556292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026781280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"232'12","reported_seq":384,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.719178+0000","last_change":"2024-09-10T18:59:52.198399+0000","last_active":"2024-09-10T19:00:58.719178+0000","last_peered":"2024-09-10T19:00:58.719178+0000","last_clean":"2024-09-10T19:00:58.719178+0000","last_became_active":"2024-09-10T18:57:02.460430+0000","last_became_peered":"2024-09-10T18:57:02.460430+0000","last_unstale":"2024-09-10T19:00:58.719178+0000","last_undegraded":"2024-09-10T19:00:58.719178+0000","last_fullsized":"2024-09-10T19:00:58.719178+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'12","last_scrub_stamp":"2024-09-10T18:59:52.198340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:59:52.198340+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:08.662513+0000","scrub_duration":0.001790591,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":341,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.596083+0000","last_change":"2024-09-10T18:56:32.602499+0000","last_active":"2024-09-10T19:00:54.596083+0000","last_peered":"2024-09-10T19:00:54.596083+0000","last_clean":"2024-09-10T19:00:54.596083+0000","last_became_active":"2024-09-10T18:45:22.069653+0000","last_became_peered":"2024-09-10T18:45:22.069653+0000","last_unstale":"2024-09-10T19:00:54.596083+0000","last_undegraded":"2024-09-10T19:00:54.596083+0000","last_fullsized":"2024-09-10T19:00:54.596083+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:32.602448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:32.602448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0052503430000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":106,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.596374+0000","last_change":"2024-09-10T19:00:42.519802+0000","last_active":"2024-09-10T19:00:54.596374+0000","last_peered":"2024-09-10T19:00:54.596374+0000","last_clean":"2024-09-10T19:00:54.596374+0000","last_became_active":"2024-09-10T19:00:42.519492+0000","last_became_peered":"2024-09-10T19:00:42.519492+0000","last_unstale":"2024-09-10T19:00:54.596374+0000","last_undegraded":"2024-09-10T19:00:54.596374+0000","last_fullsized":"2024-09-10T19:00:54.596374+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:53.608029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:58:53.608029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058174170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.11","version":"70'81","reported_seq":249,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.622014+0000","last_change":"2024-09-10T19:00:42.517609+0000","last_active":"2024-09-10T19:00:55.622014+0000","last_peered":"2024-09-10T19:00:55.622014+0000","last_clean":"2024-09-10T19:00:55.622014+0000","last_became_active":"2024-09-10T19:00:42.517311+0000","last_became_peered":"2024-09-10T19:00:42.517311+0000","last_unstale":"2024-09-10T19:00:55.622014+0000","last_undegraded":"2024-09-10T19:00:55.622014+0000","last_fullsized":"2024-09-10T19:00:55.622014+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:58:48.552631+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:58:48.552631+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012534919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"232'20","reported_seq":441,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.717880+0000","last_change":"2024-09-10T19:00:07.147063+0000","last_active":"2024-09-10T19:00:58.717880+0000","last_peered":"2024-09-10T19:00:58.717880+0000","last_clean":"2024-09-10T19:00:58.717880+0000","last_became_active":"2024-09-10T18:57:02.444437+0000","last_became_peered":"2024-09-10T18:57:02.444437+0000","last_unstale":"2024-09-10T19:00:58.717880+0000","last_undegraded":"2024-09-10T19:00:58.717880+0000","last_fullsized":"2024-09-10T19:00:58.717880+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'20","last_scrub_stamp":"2024-09-10T19:00:07.146928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:00:07.146928+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:22.782553+0000","scrub_duration":0.0071404040000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":365,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.593362+0000","last_change":"2024-09-10T18:58:50.541776+0000","last_active":"2024-09-10T19:00:54.593362+0000","last_peered":"2024-09-10T19:00:54.593362+0000","last_clean":"2024-09-10T19:00:54.593362+0000","last_became_active":"2024-09-10T18:57:02.445710+0000","last_became_peered":"2024-09-10T18:57:02.445710+0000","last_unstale":"2024-09-10T19:00:54.593362+0000","last_undegraded":"2024-09-10T19:00:54.593362+0000","last_fullsized":"2024-09-10T19:00:54.593362+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:50.541691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:58:50.541691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0089012770000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"304'11","reported_seq":581,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.628913+0000","last_change":"2024-09-10T18:59:55.525278+0000","last_active":"2024-09-10T19:00:55.628913+0000","last_peered":"2024-09-10T19:00:55.628913+0000","last_clean":"2024-09-10T19:00:55.628913+0000","last_became_active":"2024-09-10T18:58:58.489370+0000","last_became_peered":"2024-09-10T18:58:58.489370+0000","last_unstale":"2024-09-10T19:00:55.628913+0000","last_undegraded":"2024-09-10T19:00:55.628913+0000","last_fullsized":"2024-09-10T19:00:55.628913+0000","mapping_epoch":303,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":304,"parent":"0.0","parent_split_bits":0,"last_scrub":"304'11","last_scrub_stamp":"2024-09-10T18:59:55.525185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:59:55.525185+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:07.259492+0000","scrub_duration":0.0020778110000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":34,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.616225+0000","last_change":"2024-09-10T18:59:48.745646+0000","last_active":"2024-09-10T19:00:55.616225+0000","last_peered":"2024-09-10T19:00:55.616225+0000","last_clean":"2024-09-10T19:00:55.616225+0000","last_became_active":"2024-09-10T18:59:48.744923+0000","last_became_peered":"2024-09-10T18:59:48.744923+0000","last_unstale":"2024-09-10T19:00:55.616225+0000","last_undegraded":"2024-09-10T19:00:55.616225+0000","last_fullsized":"2024-09-10T19:00:55.616225+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:57.812395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:57.476724+0000","last_clean_scrub_stamp":"2024-09-10T18:58:57.812395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.10","version":"70'81","reported_seq":223,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.596341+0000","last_change":"2024-09-10T19:00:04.612412+0000","last_active":"2024-09-10T19:00:54.596341+0000","last_peered":"2024-09-10T19:00:54.596341+0000","last_clean":"2024-09-10T19:00:54.596341+0000","last_became_active":"2024-09-10T18:57:02.443782+0000","last_became_peered":"2024-09-10T18:57:02.443782+0000","last_unstale":"2024-09-10T19:00:54.596341+0000","last_undegraded":"2024-09-10T19:00:54.596341+0000","last_fullsized":"2024-09-10T19:00:54.596341+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:00:04.612324+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:00:04.612324+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:27.149851+0000","scrub_duration":0.0025961159999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"232'15","reported_seq":25,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.595560+0000","last_change":"2024-09-10T19:00:42.530883+0000","last_active":"2024-09-10T19:00:54.595560+0000","last_peered":"2024-09-10T19:00:54.595560+0000","last_clean":"2024-09-10T19:00:54.595560+0000","last_became_active":"2024-09-10T19:00:42.518672+0000","last_became_peered":"2024-09-10T19:00:42.518672+0000","last_unstale":"2024-09-10T19:00:54.595560+0000","last_undegraded":"2024-09-10T19:00:54.595560+0000","last_fullsized":"2024-09-10T19:00:54.595560+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T18:59:16.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:59:16.949954+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":323,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.623040+0000","last_change":"2024-09-10T18:56:23.096461+0000","last_active":"2024-09-10T19:00:55.623040+0000","last_peered":"2024-09-10T19:00:55.623040+0000","last_clean":"2024-09-10T19:00:55.623040+0000","last_became_active":"2024-09-10T18:55:40.390697+0000","last_became_peered":"2024-09-10T18:55:40.390697+0000","last_unstale":"2024-09-10T19:00:55.623040+0000","last_undegraded":"2024-09-10T19:00:55.623040+0000","last_fullsized":"2024-09-10T19:00:55.623040+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:23.096373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:23.096373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0057615890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"273'11","reported_seq":568,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.718299+0000","last_change":"2024-09-10T18:59:53.158730+0000","last_active":"2024-09-10T19:00:58.718299+0000","last_peered":"2024-09-10T19:00:58.718299+0000","last_clean":"2024-09-10T19:00:58.718299+0000","last_became_active":"2024-09-10T18:57:02.445268+0000","last_became_peered":"2024-09-10T18:57:02.445268+0000","last_unstale":"2024-09-10T19:00:58.718299+0000","last_undegraded":"2024-09-10T19:00:58.718299+0000","last_fullsized":"2024-09-10T19:00:58.718299+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'11","last_scrub_stamp":"2024-09-10T18:59:53.158664+0000","last_deep_scrub":"81'9","last_deep_scrub_stamp":"2024-09-10T18:46:54.800790+0000","last_clean_scrub_stamp":"2024-09-10T18:59:53.158664+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017427930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":367,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.730367+0000","last_change":"2024-09-10T18:56:20.584899+0000","last_active":"2024-09-10T19:00:58.730367+0000","last_peered":"2024-09-10T19:00:58.730367+0000","last_clean":"2024-09-10T19:00:58.730367+0000","last_became_active":"2024-09-10T18:55:40.391662+0000","last_became_peered":"2024-09-10T18:55:40.391662+0000","last_unstale":"2024-09-10T19:00:58.730367+0000","last_undegraded":"2024-09-10T19:00:58.730367+0000","last_fullsized":"2024-09-10T19:00:58.730367+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:20.584808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_clean_scrub_stamp":"2024-09-10T18:56:20.584808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0035961249999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"232'15","reported_seq":418,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.633561+0000","last_change":"2024-09-10T18:59:53.320831+0000","last_active":"2024-09-10T19:00:55.633561+0000","last_peered":"2024-09-10T19:00:55.633561+0000","last_clean":"2024-09-10T19:00:55.633561+0000","last_became_active":"2024-09-10T18:55:40.400788+0000","last_became_peered":"2024-09-10T18:55:40.400788+0000","last_unstale":"2024-09-10T19:00:55.633561+0000","last_undegraded":"2024-09-10T19:00:55.633561+0000","last_fullsized":"2024-09-10T19:00:55.633561+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T18:59:53.320757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:59:53.320757+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:15.356065+0000","scrub_duration":0.001889578,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":396,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.595913+0000","last_change":"2024-09-10T19:00:42.518606+0000","last_active":"2024-09-10T19:00:54.595913+0000","last_peered":"2024-09-10T19:00:54.595913+0000","last_clean":"2024-09-10T19:00:54.595913+0000","last_became_active":"2024-09-10T19:00:42.518310+0000","last_became_peered":"2024-09-10T19:00:42.518310+0000","last_unstale":"2024-09-10T19:00:54.595913+0000","last_undegraded":"2024-09-10T19:00:54.595913+0000","last_fullsized":"2024-09-10T19:00:54.595913+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:48.622701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:59:48.622701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:02.081467+0000","scrub_duration":0.001531279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":350,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.717548+0000","last_change":"2024-09-10T18:56:04.753218+0000","last_active":"2024-09-10T19:00:58.717548+0000","last_peered":"2024-09-10T19:00:58.717548+0000","last_clean":"2024-09-10T19:00:58.717548+0000","last_became_active":"2024-09-10T18:55:40.398243+0000","last_became_peered":"2024-09-10T18:55:40.398243+0000","last_unstale":"2024-09-10T19:00:58.717548+0000","last_undegraded":"2024-09-10T19:00:58.717548+0000","last_fullsized":"2024-09-10T19:00:58.717548+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:04.753133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:03.571786+0000","last_clean_scrub_stamp":"2024-09-10T18:56:04.753133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0069575230000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":338,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.619468+0000","last_change":"2024-09-10T19:00:42.517459+0000","last_active":"2024-09-10T19:00:55.619468+0000","last_peered":"2024-09-10T19:00:55.619468+0000","last_clean":"2024-09-10T19:00:55.619468+0000","last_became_active":"2024-09-10T19:00:42.516833+0000","last_became_peered":"2024-09-10T19:00:42.516833+0000","last_unstale":"2024-09-10T19:00:55.619468+0000","last_undegraded":"2024-09-10T19:00:55.619468+0000","last_fullsized":"2024-09-10T19:00:55.619468+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:46.863758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:04.631558+0000","last_clean_scrub_stamp":"2024-09-10T18:59:46.863758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0031907379999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"232'24","reported_seq":380,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.721645+0000","last_change":"2024-09-10T18:58:58.698097+0000","last_active":"2024-09-10T19:00:58.721645+0000","last_peered":"2024-09-10T19:00:58.721645+0000","last_clean":"2024-09-10T19:00:58.721645+0000","last_became_active":"2024-09-10T18:45:20.056885+0000","last_became_peered":"2024-09-10T18:45:20.056885+0000","last_unstale":"2024-09-10T19:00:58.721645+0000","last_undegraded":"2024-09-10T19:00:58.721645+0000","last_fullsized":"2024-09-10T19:00:58.721645+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T18:58:58.698003+0000","last_deep_scrub":"223'16","last_deep_scrub_stamp":"2024-09-10T18:54:47.324044+0000","last_clean_scrub_stamp":"2024-09-10T18:58:58.698003+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.006955676,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":363,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.623239+0000","last_change":"2024-09-10T18:58:51.138423+0000","last_active":"2024-09-10T19:00:55.623239+0000","last_peered":"2024-09-10T19:00:55.623239+0000","last_clean":"2024-09-10T19:00:55.623239+0000","last_became_active":"2024-09-10T18:55:40.399198+0000","last_became_peered":"2024-09-10T18:55:40.399198+0000","last_unstale":"2024-09-10T19:00:55.623239+0000","last_undegraded":"2024-09-10T19:00:55.623239+0000","last_fullsized":"2024-09-10T19:00:55.623239+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:51.138240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:00.029389+0000","last_clean_scrub_stamp":"2024-09-10T18:58:51.138240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0086738059999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":345,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.633429+0000","last_change":"2024-09-10T18:56:31.783026+0000","last_active":"2024-09-10T19:00:55.633429+0000","last_peered":"2024-09-10T19:00:55.633429+0000","last_clean":"2024-09-10T19:00:55.633429+0000","last_became_active":"2024-09-10T18:45:22.070065+0000","last_became_peered":"2024-09-10T18:45:22.070065+0000","last_unstale":"2024-09-10T19:00:55.633429+0000","last_undegraded":"2024-09-10T19:00:55.633429+0000","last_fullsized":"2024-09-10T19:00:55.633429+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:31.782918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:31.782918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043032130000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":348,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.719053+0000","last_change":"2024-09-10T18:56:06.792096+0000","last_active":"2024-09-10T19:00:58.719053+0000","last_peered":"2024-09-10T19:00:58.719053+0000","last_clean":"2024-09-10T19:00:58.719053+0000","last_became_active":"2024-09-10T18:55:40.395595+0000","last_became_peered":"2024-09-10T18:55:40.395595+0000","last_unstale":"2024-09-10T19:00:58.719053+0000","last_undegraded":"2024-09-10T19:00:58.719053+0000","last_fullsized":"2024-09-10T19:00:58.719053+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:06.791860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:06.791860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046852380000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"232'8","reported_seq":357,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.717827+0000","last_change":"2024-09-10T18:56:30.802774+0000","last_active":"2024-09-10T19:00:58.717827+0000","last_peered":"2024-09-10T19:00:58.717827+0000","last_clean":"2024-09-10T19:00:58.717827+0000","last_became_active":"2024-09-10T18:45:20.059205+0000","last_became_peered":"2024-09-10T18:45:20.059205+0000","last_unstale":"2024-09-10T19:00:58.717827+0000","last_undegraded":"2024-09-10T19:00:58.717827+0000","last_fullsized":"2024-09-10T19:00:58.717827+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'8","last_scrub_stamp":"2024-09-10T18:56:30.802679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:30.802679+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034579609999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":361,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.596156+0000","last_change":"2024-09-10T18:58:53.527829+0000","last_active":"2024-09-10T19:00:54.596156+0000","last_peered":"2024-09-10T19:00:54.596156+0000","last_clean":"2024-09-10T19:00:54.596156+0000","last_became_active":"2024-09-10T18:47:36.415866+0000","last_became_peered":"2024-09-10T18:47:36.415866+0000","last_unstale":"2024-09-10T19:00:54.596156+0000","last_undegraded":"2024-09-10T19:00:54.596156+0000","last_fullsized":"2024-09-10T19:00:54.596156+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:53.527746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:58:53.527746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058191349999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":372,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.595139+0000","last_change":"2024-09-10T18:57:02.454548+0000","last_active":"2024-09-10T19:00:54.595139+0000","last_peered":"2024-09-10T19:00:54.595139+0000","last_clean":"2024-09-10T19:00:54.595139+0000","last_became_active":"2024-09-10T18:57:02.453974+0000","last_became_peered":"2024-09-10T18:57:02.453974+0000","last_unstale":"2024-09-10T19:00:54.595139+0000","last_undegraded":"2024-09-10T19:00:54.595139+0000","last_fullsized":"2024-09-10T19:00:54.595139+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:23.195060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_clean_scrub_stamp":"2024-09-10T18:56:23.195060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047613259999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":145,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.722802+0000","last_change":"2024-09-10T18:59:55.812370+0000","last_active":"2024-09-10T19:00:58.722802+0000","last_peered":"2024-09-10T19:00:58.722802+0000","last_clean":"2024-09-10T19:00:58.722802+0000","last_became_active":"2024-09-10T18:57:02.447566+0000","last_became_peered":"2024-09-10T18:57:02.447566+0000","last_unstale":"2024-09-10T19:00:58.722802+0000","last_undegraded":"2024-09-10T19:00:58.722802+0000","last_fullsized":"2024-09-10T19:00:58.722802+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:55.812312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:55.145638+0000","last_clean_scrub_stamp":"2024-09-10T18:59:55.812312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0055522669999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.14","version":"70'81","reported_seq":249,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.721694+0000","last_change":"2024-09-10T19:00:58.100610+0000","last_active":"2024-09-10T19:00:58.721694+0000","last_peered":"2024-09-10T19:00:58.721694+0000","last_clean":"2024-09-10T19:00:58.721694+0000","last_became_active":"2024-09-10T18:57:02.442457+0000","last_became_peered":"2024-09-10T18:57:02.442457+0000","last_unstale":"2024-09-10T19:00:58.721694+0000","last_undegraded":"2024-09-10T19:00:58.721694+0000","last_fullsized":"2024-09-10T19:00:58.721694+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:55:34.791223+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:57.184383+0000","last_clean_scrub_stamp":"2024-09-10T18:55:34.791223+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016709736999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17890,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":7,"num_write":3,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":35780,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.11","version":"232'24","reported_seq":403,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.718686+0000","last_change":"2024-09-10T18:59:10.879686+0000","last_active":"2024-09-10T19:00:58.718686+0000","last_peered":"2024-09-10T19:00:58.718686+0000","last_clean":"2024-09-10T19:00:58.718686+0000","last_became_active":"2024-09-10T18:57:02.461909+0000","last_became_peered":"2024-09-10T18:57:02.461909+0000","last_unstale":"2024-09-10T19:00:58.718686+0000","last_undegraded":"2024-09-10T19:00:58.718686+0000","last_fullsized":"2024-09-10T19:00:58.718686+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T18:59:10.879597+0000","last_deep_scrub":"79'16","last_deep_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_clean_scrub_stamp":"2024-09-10T18:59:10.879597+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0090530790000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"56'1","reported_seq":377,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.601026+0000","last_change":"2024-09-10T18:56:25.211179+0000","last_active":"2024-09-10T19:00:54.601026+0000","last_peered":"2024-09-10T19:00:54.601026+0000","last_clean":"2024-09-10T19:00:54.601026+0000","last_became_active":"2024-09-10T18:47:36.421985+0000","last_became_peered":"2024-09-10T18:47:36.421985+0000","last_unstale":"2024-09-10T19:00:54.601026+0000","last_undegraded":"2024-09-10T19:00:54.601026+0000","last_fullsized":"2024-09-10T19:00:54.601026+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:56:25.211095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:25.211095+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025468880000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":89,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.718543+0000","last_change":"2024-09-10T18:57:02.435423+0000","last_active":"2024-09-10T19:00:58.718543+0000","last_peered":"2024-09-10T19:00:58.718543+0000","last_clean":"2024-09-10T19:00:58.718543+0000","last_became_active":"2024-09-10T18:57:02.435124+0000","last_became_peered":"2024-09-10T18:57:02.435124+0000","last_unstale":"2024-09-10T19:00:58.718543+0000","last_undegraded":"2024-09-10T19:00:58.718543+0000","last_fullsized":"2024-09-10T19:00:58.718543+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:12.852769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:12.852769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":358,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.717395+0000","last_change":"2024-09-10T18:56:09.903930+0000","last_active":"2024-09-10T19:00:58.717395+0000","last_peered":"2024-09-10T19:00:58.717395+0000","last_clean":"2024-09-10T19:00:58.717395+0000","last_became_active":"2024-09-10T18:55:40.397523+0000","last_became_peered":"2024-09-10T18:55:40.397523+0000","last_unstale":"2024-09-10T19:00:58.717395+0000","last_undegraded":"2024-09-10T19:00:58.717395+0000","last_fullsized":"2024-09-10T19:00:58.717395+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:09.903846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:09.903846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011721984,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"78'2","reported_seq":132,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.594719+0000","last_change":"2024-09-10T18:58:48.472078+0000","last_active":"2024-09-10T19:00:54.594719+0000","last_peered":"2024-09-10T19:00:54.594719+0000","last_clean":"2024-09-10T19:00:54.594719+0000","last_became_active":"2024-09-10T18:57:02.447198+0000","last_became_peered":"2024-09-10T18:57:02.447198+0000","last_unstale":"2024-09-10T19:00:54.594719+0000","last_undegraded":"2024-09-10T19:00:54.594719+0000","last_fullsized":"2024-09-10T19:00:54.594719+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'2","last_scrub_stamp":"2024-09-10T18:58:48.472024+0000","last_deep_scrub":"78'2","last_deep_scrub_stamp":"2024-09-10T18:46:37.581130+0000","last_clean_scrub_stamp":"2024-09-10T18:58:48.472024+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.022088824,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":365,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.596460+0000","last_change":"2024-09-10T18:59:53.505658+0000","last_active":"2024-09-10T19:00:54.596460+0000","last_peered":"2024-09-10T19:00:54.596460+0000","last_clean":"2024-09-10T19:00:54.596460+0000","last_became_active":"2024-09-10T18:57:02.443353+0000","last_became_peered":"2024-09-10T18:57:02.443353+0000","last_unstale":"2024-09-10T19:00:54.596460+0000","last_undegraded":"2024-09-10T19:00:54.596460+0000","last_fullsized":"2024-09-10T19:00:54.596460+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:53.505561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:59:53.505561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:19.130152+0000","scrub_duration":0.001238605,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"232'20","reported_seq":372,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.619155+0000","last_change":"2024-09-10T18:57:02.486956+0000","last_active":"2024-09-10T19:00:55.619155+0000","last_peered":"2024-09-10T19:00:55.619155+0000","last_clean":"2024-09-10T19:00:55.619155+0000","last_became_active":"2024-09-10T18:57:02.452473+0000","last_became_peered":"2024-09-10T18:57:02.452473+0000","last_unstale":"2024-09-10T19:00:55.619155+0000","last_undegraded":"2024-09-10T19:00:55.619155+0000","last_fullsized":"2024-09-10T19:00:55.619155+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'20","last_scrub_stamp":"2024-09-10T18:56:27.158686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:27.158686+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0090371180000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":23,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"273'10","reported_seq":576,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.626999+0000","last_change":"2024-09-10T18:59:22.516714+0000","last_active":"2024-09-10T19:00:55.626999+0000","last_peered":"2024-09-10T19:00:55.626999+0000","last_clean":"2024-09-10T19:00:55.626999+0000","last_became_active":"2024-09-10T18:57:02.441763+0000","last_became_peered":"2024-09-10T18:57:02.441763+0000","last_unstale":"2024-09-10T19:00:55.626999+0000","last_undegraded":"2024-09-10T19:00:55.626999+0000","last_fullsized":"2024-09-10T19:00:55.626999+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'10","last_scrub_stamp":"2024-09-10T18:59:22.516625+0000","last_deep_scrub":"190'9","last_deep_scrub_stamp":"2024-09-10T18:53:07.746309+0000","last_clean_scrub_stamp":"2024-09-10T18:59:22.516625+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021912820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":343,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.622789+0000","last_change":"2024-09-10T18:58:59.150773+0000","last_active":"2024-09-10T19:00:55.622789+0000","last_peered":"2024-09-10T19:00:55.622789+0000","last_clean":"2024-09-10T19:00:55.622789+0000","last_became_active":"2024-09-10T18:45:24.082318+0000","last_became_peered":"2024-09-10T18:45:24.082318+0000","last_unstale":"2024-09-10T19:00:55.622789+0000","last_undegraded":"2024-09-10T19:00:55.622789+0000","last_fullsized":"2024-09-10T19:00:55.622789+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:59.148958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:03.475299+0000","last_clean_scrub_stamp":"2024-09-10T18:58:59.148958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041797229999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":364,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.718014+0000","last_change":"2024-09-10T19:00:10.182129+0000","last_active":"2024-09-10T19:00:58.718014+0000","last_peered":"2024-09-10T19:00:58.718014+0000","last_clean":"2024-09-10T19:00:58.718014+0000","last_became_active":"2024-09-10T18:55:40.392524+0000","last_became_peered":"2024-09-10T18:55:40.392524+0000","last_unstale":"2024-09-10T19:00:58.718014+0000","last_undegraded":"2024-09-10T19:00:58.718014+0000","last_fullsized":"2024-09-10T19:00:58.718014+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:10.181982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:00:10.181982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:31.010155+0000","scrub_duration":0.0029510420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"232'19","reported_seq":388,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.601070+0000","last_change":"2024-09-10T18:59:09.544279+0000","last_active":"2024-09-10T19:00:54.601070+0000","last_peered":"2024-09-10T19:00:54.601070+0000","last_clean":"2024-09-10T19:00:54.601070+0000","last_became_active":"2024-09-10T18:45:20.058717+0000","last_became_peered":"2024-09-10T18:45:20.058717+0000","last_unstale":"2024-09-10T19:00:54.601070+0000","last_undegraded":"2024-09-10T19:00:54.601070+0000","last_fullsized":"2024-09-10T19:00:54.601070+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T18:59:09.544194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:59:09.544194+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043401830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":326,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.619201+0000","last_change":"2024-09-10T18:59:57.862722+0000","last_active":"2024-09-10T19:00:55.619201+0000","last_peered":"2024-09-10T19:00:55.619201+0000","last_clean":"2024-09-10T19:00:55.619201+0000","last_became_active":"2024-09-10T18:57:02.453464+0000","last_became_peered":"2024-09-10T18:57:02.453464+0000","last_unstale":"2024-09-10T19:00:55.619201+0000","last_undegraded":"2024-09-10T19:00:55.619201+0000","last_fullsized":"2024-09-10T19:00:55.619201+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:57.862666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:59:57.862666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:06.449870+0000","scrub_duration":0.001262242,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":340,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.725652+0000","last_change":"2024-09-10T18:58:55.726863+0000","last_active":"2024-09-10T19:00:58.725652+0000","last_peered":"2024-09-10T19:00:58.725652+0000","last_clean":"2024-09-10T19:00:58.725652+0000","last_became_active":"2024-09-10T18:45:24.267251+0000","last_became_peered":"2024-09-10T18:45:24.267251+0000","last_unstale":"2024-09-10T19:00:58.725652+0000","last_undegraded":"2024-09-10T19:00:58.725652+0000","last_fullsized":"2024-09-10T19:00:58.725652+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:55.726799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:58:55.726799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001154721,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"306'2","reported_seq":383,"reported_epoch":343,"state":"active+clean","last_fresh":"2024-09-10T19:00:54.595497+0000","last_change":"2024-09-10T18:59:20.534927+0000","last_active":"2024-09-10T19:00:54.595497+0000","last_peered":"2024-09-10T19:00:54.595497+0000","last_clean":"2024-09-10T19:00:54.595497+0000","last_became_active":"2024-09-10T18:57:02.435758+0000","last_became_peered":"2024-09-10T18:57:02.435758+0000","last_unstale":"2024-09-10T19:00:54.595497+0000","last_undegraded":"2024-09-10T19:00:54.595497+0000","last_fullsized":"2024-09-10T19:00:54.595497+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"306'2","last_scrub_stamp":"2024-09-10T18:59:20.534869+0000","last_deep_scrub":"306'2","last_deep_scrub_stamp":"2024-09-10T18:59:20.534869+0000","last_clean_scrub_stamp":"2024-09-10T18:59:20.534869+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0072446200000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"258'34","reported_seq":462,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.623151+0000","last_change":"2024-09-10T19:00:52.459800+0000","last_active":"2024-09-10T19:00:55.623151+0000","last_peered":"2024-09-10T19:00:55.623151+0000","last_clean":"2024-09-10T19:00:55.623151+0000","last_became_active":"2024-09-10T18:59:48.744293+0000","last_became_peered":"2024-09-10T18:59:48.744293+0000","last_unstale":"2024-09-10T19:00:55.623151+0000","last_undegraded":"2024-09-10T19:00:55.623151+0000","last_fullsized":"2024-09-10T19:00:55.623151+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'34","last_scrub_stamp":"2024-09-10T18:56:23.838951+0000","last_deep_scrub":"258'34","last_deep_scrub_stamp":"2024-09-10T18:56:23.838951+0000","last_clean_scrub_stamp":"2024-09-10T18:56:23.838951+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.035929888,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":352,"reported_epoch":344,"state":"active+clean","last_fresh":"2024-09-10T19:00:55.622964+0000","last_change":"2024-09-10T19:00:54.441149+0000","last_active":"2024-09-10T19:00:55.622964+0000","last_peered":"2024-09-10T19:00:55.622964+0000","last_clean":"2024-09-10T19:00:55.622964+0000","last_became_active":"2024-09-10T18:55:40.397870+0000","last_became_peered":"2024-09-10T18:55:40.397870+0000","last_unstale":"2024-09-10T19:00:55.622964+0000","last_undegraded":"2024-09-10T19:00:55.622964+0000","last_fullsized":"2024-09-10T19:00:55.622964+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:13.610976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:13.610976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010999553,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":361,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.718679+0000","last_change":"2024-09-10T19:00:42.520936+0000","last_active":"2024-09-10T19:00:58.718679+0000","last_peered":"2024-09-10T19:00:58.718679+0000","last_clean":"2024-09-10T19:00:58.718679+0000","last_became_active":"2024-09-10T19:00:42.520239+0000","last_became_peered":"2024-09-10T19:00:42.520239+0000","last_unstale":"2024-09-10T19:00:58.718679+0000","last_undegraded":"2024-09-10T19:00:58.718679+0000","last_fullsized":"2024-09-10T19:00:58.718679+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:17.985545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:17.985545+0000","last_clean_scrub_stamp":"2024-09-10T18:59:17.985545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0086180830000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":367,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.717812+0000","last_change":"2024-09-10T18:58:57.919890+0000","last_active":"2024-09-10T19:00:58.717812+0000","last_peered":"2024-09-10T19:00:58.717812+0000","last_clean":"2024-09-10T19:00:58.717812+0000","last_became_active":"2024-09-10T18:57:02.460855+0000","last_became_peered":"2024-09-10T18:57:02.460855+0000","last_unstale":"2024-09-10T19:00:58.717812+0000","last_undegraded":"2024-09-10T19:00:58.717812+0000","last_fullsized":"2024-09-10T19:00:58.717812+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:57.919515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:58:57.919515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.01355234,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"232'18","reported_seq":377,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.718133+0000","last_change":"2024-09-10T18:58:52.890712+0000","last_active":"2024-09-10T19:00:58.718133+0000","last_peered":"2024-09-10T19:00:58.718133+0000","last_clean":"2024-09-10T19:00:58.718133+0000","last_became_active":"2024-09-10T18:45:20.058391+0000","last_became_peered":"2024-09-10T18:45:20.058391+0000","last_unstale":"2024-09-10T19:00:58.718133+0000","last_undegraded":"2024-09-10T19:00:58.718133+0000","last_fullsized":"2024-09-10T19:00:58.718133+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'18","last_scrub_stamp":"2024-09-10T18:58:52.890618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:58:52.890618+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016012082,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":372,"reported_epoch":346,"state":"active+clean","last_fresh":"2024-09-10T19:00:58.721643+0000","last_change":"2024-09-10T18:58:56.690660+0000","last_active":"2024-09-10T19:00:58.721643+0000","last_peered":"2024-09-10T19:00:58.721643+0000","last_clean":"2024-09-10T19:00:58.721643+0000","last_became_active":"2024-09-10T18:57:02.449135+0000","last_became_peered":"2024-09-10T18:57:02.449135+0000","last_unstale":"2024-09-10T19:00:58.721643+0000","last_undegraded":"2024-09-10T19:00:58.721643+0000","last_fullsized":"2024-09-10T19:00:58.721643+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:56.690606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:58:56.690606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060631599999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":6897270784,"num_objects":105244,"num_object_clones":0,"num_object_copies":315732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":12327,"num_objects_unfound":0,"num_objects_dirty":105244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105244,"num_write_kb":6735616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7669,"num_bytes_recovered":502595584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":21443575808,"data_stored":21443575808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57301,"ondisk_log_size":57301,"up":48,"acting":48,"num_store_stats":6},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":421,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1118,"num_read_kb":875,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":3372,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":499712,"data_stored":13268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":758,"ondisk_log_size":758,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":922,"num_read_kb":917,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":608258,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":7,"num_write":3,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":298468,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1701,"ondisk_log_size":1701,"up":63,"acting":63,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":0,"seq":223338299585,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":43,"seq":184683593925,"num_pgs":86,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3785852,"kb_used_data":2900016,"kb_used_omap":1,"kb_used_meta":885822,"kb_avail":89959300,"statfs":{"total":95995035648,"available":92118323200,"internally_reserved":0,"allocated":2969616384,"data_stored":2968987851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":907082072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:00:54 2024","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.1830000000000001,"15min":0.98899999999999999},"min":{"1min":0.34399999999999997,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":5.0910000000000002,"5min":27.545000000000002,"15min":27.545000000000002},"last":0.63800000000000001},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.3149999999999999,"15min":1.0649999999999999},"min":{"1min":0.27900000000000003,"5min":0.219,"15min":0.219},"max":{"1min":4.6180000000000003,"5min":27.568000000000001,"15min":27.568000000000001},"last":1.1779999999999999}]},{"osd":1,"last update":"Tue Sep 10 19:00:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":1.7,"15min":1.2589999999999999},"min":{"1min":0.30499999999999999,"5min":0.22900000000000001,"15min":0.219},"max":{"1min":1.9099999999999999,"5min":45.311,"15min":45.311},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.746,"5min":1.6699999999999999,"15min":1.155},"min":{"1min":0.32200000000000001,"5min":0.23300000000000001,"15min":0.214},"max":{"1min":2.1930000000000001,"5min":45.027999999999999,"15min":45.027999999999999},"last":0.65800000000000003}]},{"osd":2,"last update":"Tue Sep 10 19:00:00 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.63600000000000001,"15min":0.749},"min":{"1min":0.28799999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":2.0579999999999998,"5min":4.1029999999999998,"15min":4.4740000000000002},"last":1.2230000000000001},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.54300000000000004,"15min":0.68100000000000005},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.74199999999999999,"5min":3.2949999999999999,"15min":4.8150000000000004},"last":0.70899999999999996}]},{"osd":3,"last update":"Tue Sep 10 19:00:00 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":2.3370000000000002,"15min":1.385},"min":{"1min":0.35399999999999998,"5min":0.29599999999999999,"15min":0.25700000000000001},"max":{"1min":2.3740000000000001,"5min":76.977999999999994,"15min":76.977999999999994},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.67800000000000005,"5min":2.3210000000000002,"15min":1.4970000000000001},"min":{"1min":0.33600000000000002,"5min":0.27500000000000002,"15min":0.253},"max":{"1min":1.74,"5min":76.879999999999995,"15min":76.879999999999995},"last":1.202}]},{"osd":4,"last update":"Tue Sep 10 19:00:00 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.56200000000000006,"15min":1.2809999999999999},"min":{"1min":0.25900000000000001,"5min":0.23400000000000001,"15min":0.20999999999999999},"max":{"1min":2.4169999999999998,"5min":3.4100000000000001,"15min":165.03299999999999},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.61799999999999999,"15min":1.2050000000000001},"min":{"1min":0.28100000000000003,"5min":0.26000000000000001,"15min":0.22500000000000001},"max":{"1min":1.944,"5min":4.9450000000000003,"15min":163.959},"last":0.56200000000000006}]},{"osd":5,"last update":"Tue Sep 10 19:00:00 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.69999999999999996,"15min":0.93500000000000005},"min":{"1min":0.29399999999999998,"5min":0.28599999999999998,"15min":0.24099999999999999},"max":{"1min":1.744,"5min":4.3849999999999998,"15min":43.811},"last":1.605},{"interface":"front","average":{"1min":0.433,"5min":0.66300000000000003,"15min":0.94599999999999995},"min":{"1min":0.309,"5min":0.27300000000000002,"15min":0.23999999999999999},"max":{"1min":0.78100000000000003,"5min":5.5609999999999999,"15min":42.314},"last":0.42099999999999999}]},{"osd":7,"last update":"Tue Sep 10 19:00:32 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.67500000000000004,"15min":0.80300000000000005},"min":{"1min":0.33700000000000002,"5min":0.27900000000000003,"15min":0.26200000000000001},"max":{"1min":1.599,"5min":4.1580000000000004,"15min":5.3680000000000003},"last":1.1479999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.70499999999999996,"15min":0.84599999999999997},"min":{"1min":0.309,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.4910000000000001,"5min":3.2759999999999998,"15min":7.0350000000000001},"last":0.48599999999999999}]}]},{"osd":5,"up_from":37,"seq":158913790153,"num_pgs":84,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3371260,"kb_used_data":2483440,"kb_used_omap":1,"kb_used_meta":887806,"kb_avail":90373892,"statfs":{"total":95995035648,"available":92542865408,"internally_reserved":0,"allocated":2543042560,"data_stored":2542413056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":909113688},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:00:49 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":2.7530000000000001,"15min":1.95},"min":{"1min":0.33000000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":2.1600000000000001,"5min":82.465000000000003,"15min":82.465000000000003},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":2.7970000000000002,"15min":2.113},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":1.645,"5min":82.519000000000005,"15min":82.519000000000005},"last":2.9289999999999998}]},{"osd":1,"last update":"Tue Sep 10 19:00:49 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":2.8490000000000002,"15min":1.524},"min":{"1min":0.313,"5min":0.20999999999999999,"15min":0.19500000000000001},"max":{"1min":3.851,"5min":67.582999999999998,"15min":67.582999999999998},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":2.7879999999999998,"15min":1.667},"min":{"1min":0.35099999999999998,"5min":0.29699999999999999,"15min":0.20399999999999999},"max":{"1min":1.7250000000000001,"5min":67.603999999999999,"15min":67.603999999999999},"last":2.9670000000000001}]},{"osd":2,"last update":"Tue Sep 10 19:00:49 2024","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":0.73999999999999999,"15min":0.76000000000000001},"min":{"1min":0.30299999999999999,"5min":0.27200000000000002,"15min":0.249},"max":{"1min":7.8879999999999999,"5min":7.8879999999999999,"15min":7.8879999999999999},"last":1.016},{"interface":"front","average":{"1min":0.996,"5min":0.70599999999999996,"15min":0.77400000000000002},"min":{"1min":0.28499999999999998,"5min":0.27100000000000002,"15min":0.24399999999999999},"max":{"1min":6.7549999999999999,"5min":6.7549999999999999,"15min":6.7549999999999999},"last":0.504}]},{"osd":3,"last update":"Tue Sep 10 19:00:49 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":4.7930000000000001,"15min":2.7029999999999998},"min":{"1min":0.27900000000000003,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":3.5510000000000002,"5min":123.34399999999999,"15min":123.34399999999999},"last":1.054},{"interface":"front","average":{"1min":0.80300000000000005,"5min":3.1579999999999999,"15min":2.2810000000000001},"min":{"1min":0.36599999999999999,"5min":0.27300000000000002,"15min":0.245},"max":{"1min":2.7719999999999998,"5min":82.415000000000006,"15min":82.415000000000006},"last":0.52500000000000002}]},{"osd":4,"last update":"Tue Sep 10 19:00:49 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.58199999999999996,"15min":0.84199999999999997},"min":{"1min":0.27500000000000002,"5min":0.221,"15min":0.20899999999999999},"max":{"1min":1.706,"5min":2.9940000000000002,"15min":9.0310000000000006},"last":2.9990000000000001},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.54100000000000004,"15min":0.78400000000000003},"min":{"1min":0.27300000000000002,"5min":0.23899999999999999,"15min":0.23400000000000001},"max":{"1min":2.742,"5min":2.742,"15min":11.182},"last":1.107}]},{"osd":6,"last update":"Tue Sep 10 19:00:05 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.57499999999999996,"15min":0.91700000000000004},"min":{"1min":0.28000000000000003,"5min":0.23999999999999999,"15min":0.23400000000000001},"max":{"1min":0.91200000000000003,"5min":3.069,"15min":11.17},"last":4.1200000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.501,"15min":0.91000000000000003},"min":{"1min":0.309,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.59599999999999997,"5min":1.5369999999999999,"15min":11.18},"last":1.129}]},{"osd":7,"last update":"Tue Sep 10 19:00:28 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.66700000000000004,"15min":0.90800000000000003},"min":{"1min":0.318,"5min":0.28999999999999998,"15min":0.27600000000000002},"max":{"1min":1.3520000000000001,"5min":3.665,"15min":10.782999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53500000000000003,"15min":0.79800000000000004},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.23599999999999999},"max":{"1min":1.246,"5min":1.956,"15min":11.846},"last":0.54300000000000004}]}]},{"osd":4,"up_from":30,"seq":128849019088,"num_pgs":85,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6599452,"kb_used_data":5713292,"kb_used_omap":1,"kb_used_meta":886142,"kb_avail":87145700,"statfs":{"total":95995035648,"available":89237196800,"internally_reserved":0,"allocated":5850411008,"data_stored":5849774366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":907409752},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.4990000000000001,"15min":1.4570000000000001},"min":{"1min":0.39100000000000001,"5min":0.223,"15min":0.223},"max":{"1min":3.6400000000000001,"5min":46.234999999999999,"15min":46.234999999999999},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.099,"5min":1.423,"15min":1.4510000000000001},"min":{"1min":0.36599999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":3.609,"5min":46.264000000000003,"15min":46.264000000000003},"last":1.5669999999999999}]},{"osd":1,"last update":"Tue Sep 10 19:00:27 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":2.3330000000000002,"15min":1.556},"min":{"1min":0.33700000000000002,"5min":0.223,"15min":0.223},"max":{"1min":3.6749999999999998,"5min":125.15600000000001,"15min":125.15600000000001},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":2.4140000000000001,"15min":1.498},"min":{"1min":0.26400000000000001,"5min":0.23699999999999999,"15min":0.23100000000000001},"max":{"1min":2.069,"5min":125.14700000000001,"15min":125.14700000000001},"last":1.48}]},{"osd":2,"last update":"Tue Sep 10 19:00:27 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.65100000000000002,"15min":0.68000000000000005},"min":{"1min":0.38100000000000001,"5min":0.26200000000000001,"15min":0.25900000000000001},"max":{"1min":2.1560000000000001,"5min":2.8879999999999999,"15min":5.6189999999999998},"last":1.5900000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.30499999999999999,"5min":0.29899999999999999,"15min":0.27100000000000002},"max":{"1min":1.603,"5min":12.782999999999999,"15min":12.782999999999999},"last":0.67500000000000004}]},{"osd":3,"last update":"Tue Sep 10 19:00:27 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":3.5150000000000001,"15min":2.0670000000000002},"min":{"1min":0.33000000000000002,"5min":0.28299999999999997,"15min":0.23799999999999999},"max":{"1min":3.1880000000000002,"5min":111.514,"15min":111.514},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.53700000000000003,"5min":2.5670000000000002,"15min":1.6879999999999999},"min":{"1min":0.28399999999999997,"5min":0.26900000000000002,"15min":0.25700000000000001},"max":{"1min":0.875,"5min":110.914,"15min":110.914},"last":0.48499999999999999}]},{"osd":5,"last update":"Tue Sep 10 19:00:51 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.628,"15min":0.85499999999999998},"min":{"1min":0.24099999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":2.6800000000000002,"5min":9.7309999999999999,"15min":10.912000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.68200000000000005,"15min":0.79700000000000004},"min":{"1min":0.312,"5min":0.216,"15min":0.216},"max":{"1min":2.0379999999999998,"5min":10.147,"15min":10.147},"last":1.6359999999999999}]},{"osd":6,"last update":"Tue Sep 10 19:00:07 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.60099999999999998,"15min":0.72199999999999998},"min":{"1min":0.26500000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.80500000000000005,"5min":5.5529999999999999,"15min":8.4030000000000005},"last":1.6140000000000001},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.55000000000000004,"15min":0.79400000000000004},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.92700000000000005,"5min":4.0999999999999996,"15min":7.7699999999999996},"last":3.665}]},{"osd":7,"last update":"Tue Sep 10 19:00:46 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.63700000000000001,"15min":0.81599999999999995},"min":{"1min":0.33600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.1829999999999998,"5min":4.5250000000000004,"15min":6.4630000000000001},"last":1.516},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.67700000000000005,"15min":0.79700000000000004},"min":{"1min":0.28199999999999997,"5min":0.251,"15min":0.251},"max":{"1min":3.589,"5min":3.589,"15min":6.3949999999999996},"last":0.53300000000000003}]}]},{"osd":3,"up_from":25,"seq":107374182613,"num_pgs":83,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3939472,"kb_used_data":3037576,"kb_used_omap":1,"kb_used_meta":901886,"kb_avail":89805680,"statfs":{"total":95995035648,"available":91961016320,"internally_reserved":0,"allocated":3110477824,"data_stored":3109837979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":923531608},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":4.7220000000000004,"15min":2.4209999999999998},"min":{"1min":0.23200000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":2.6989999999999998,"5min":290.721,"15min":290.721},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.879,"5min":4.9530000000000003,"15min":2.661},"min":{"1min":0.17499999999999999,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":3.1880000000000002,"5min":293.59300000000002,"15min":293.59300000000002},"last":0.93400000000000005}]},{"osd":1,"last update":"Tue Sep 10 19:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":2.024,"15min":1.536},"min":{"1min":0.22900000000000001,"5min":0.16200000000000001,"15min":0.16200000000000001},"max":{"1min":10.645,"5min":10.645,"15min":58.215000000000003},"last":1.05},{"interface":"front","average":{"1min":1.6200000000000001,"5min":1.756,"15min":1.4650000000000001},"min":{"1min":0.26000000000000001,"5min":0.188,"15min":0.188},"max":{"1min":13.263999999999999,"5min":13.263999999999999,"15min":57.896999999999998},"last":0.33300000000000002}]},{"osd":2,"last update":"Tue Sep 10 19:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.2330000000000001,"15min":0.93999999999999995},"min":{"1min":0.30099999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":4.5670000000000002,"5min":5.6680000000000001,"15min":5.8220000000000001},"last":0.621},{"interface":"front","average":{"1min":0.55300000000000005,"5min":1.081,"15min":0.83099999999999996},"min":{"1min":0.23799999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":2.7090000000000001,"5min":5.1399999999999997,"15min":7.0979999999999999},"last":1.2529999999999999}]},{"osd":4,"last update":"Tue Sep 10 19:00:30 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":1.337,"15min":1.1990000000000001},"min":{"1min":0.33400000000000002,"5min":0.26400000000000001,"15min":0.247},"max":{"1min":2.3130000000000002,"5min":5.0490000000000004,"15min":29.013000000000002},"last":1.306},{"interface":"front","average":{"1min":0.501,"5min":1.4039999999999999,"15min":1.21},"min":{"1min":0.31900000000000001,"5min":0.28000000000000003,"15min":0.26800000000000002},"max":{"1min":0.90000000000000002,"5min":8.8130000000000006,"15min":28.826000000000001},"last":1.345}]},{"osd":5,"last update":"Tue Sep 10 19:00:54 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":1.4970000000000001,"15min":1.22},"min":{"1min":0.39800000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":2.4409999999999998,"5min":7.2300000000000004,"15min":7.2300000000000004},"last":1.3280000000000001},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.365,"15min":1.1930000000000001},"min":{"1min":0.29099999999999998,"5min":0.27100000000000002,"15min":0.255},"max":{"1min":2.9849999999999999,"5min":5.4139999999999997,"15min":9.2509999999999994},"last":0.64900000000000002}]},{"osd":6,"last update":"Tue Sep 10 19:00:09 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.4179999999999999,"15min":1.21},"min":{"1min":0.372,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":4.6289999999999996,"5min":5.4480000000000004,"15min":9.3239999999999998},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.63500000000000001,"5min":1.405,"15min":1.1879999999999999},"min":{"1min":0.35799999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":2.8460000000000001,"5min":5.0789999999999997,"15min":7.1559999999999997},"last":1.2829999999999999}]},{"osd":7,"last update":"Tue Sep 10 19:00:40 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":1.6120000000000001,"15min":1.27},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.34899999999999998},"max":{"1min":0.98599999999999999,"5min":8.7989999999999995,"15min":8.7989999999999995},"last":1.992},{"interface":"front","average":{"1min":0.83099999999999996,"5min":1.526,"15min":1.298},"min":{"1min":0.39600000000000002,"5min":0.36899999999999999,"15min":0.33100000000000002},"max":{"1min":2.2890000000000001,"5min":6.6029999999999998,"15min":9.2949999999999999},"last":0.67700000000000005}]}]},{"osd":2,"up_from":0,"seq":77309411545,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834575068,"num_pgs":87,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2976988,"kb_used_data":2073676,"kb_used_omap":1,"kb_used_meta":903294,"kb_avail":90768164,"statfs":{"total":95995035648,"available":92946599936,"internally_reserved":0,"allocated":2123444224,"data_stored":2122787402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":924973400},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:00:50 2024","interfaces":[{"interface":"back","average":{"1min":2.5139999999999998,"5min":2.8820000000000001,"15min":1.792},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":45.786000000000001,"5min":111.014,"15min":111.014},"last":1.0569999999999999},{"interface":"front","average":{"1min":2.5600000000000001,"5min":2.9980000000000002,"15min":1.746},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":46.311999999999998,"5min":111.072,"15min":111.072},"last":1.0309999999999999}]},{"osd":2,"last update":"Tue Sep 10 19:00:36 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.92700000000000005,"15min":0.73199999999999998},"min":{"1min":0.23799999999999999,"5min":0.23300000000000001,"15min":0.186},"max":{"1min":1.1990000000000001,"5min":4.1719999999999997,"15min":4.1719999999999997},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.90600000000000003,"15min":0.76000000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.215},"max":{"1min":2.8029999999999999,"5min":5.6929999999999996,"15min":7.093},"last":0.42599999999999999}]},{"osd":3,"last update":"Tue Sep 10 19:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":3.661,"15min":1.978},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.185},"max":{"1min":6.1559999999999997,"5min":92.069000000000003,"15min":92.069000000000003},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.76300000000000001,"5min":3.3050000000000002,"15min":1.8879999999999999},"min":{"1min":0.27800000000000002,"5min":0.218,"15min":0.19900000000000001},"max":{"1min":2.8879999999999999,"5min":88.658000000000001,"15min":88.658000000000001},"last":0.39400000000000002}]},{"osd":4,"last update":"Tue Sep 10 19:00:27 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":1.159,"15min":1.135},"min":{"1min":0.28100000000000003,"5min":0.27800000000000002,"15min":0.26200000000000001},"max":{"1min":1.7330000000000001,"5min":9.125,"15min":15.965999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":1.1539999999999999,"15min":1.002},"min":{"1min":0.34999999999999998,"5min":0.252,"15min":0.246},"max":{"1min":0.98499999999999999,"5min":7.0940000000000003,"15min":7.0940000000000003},"last":0.30599999999999999}]},{"osd":5,"last update":"Tue Sep 10 19:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":1.1579999999999999,"15min":1.1499999999999999},"min":{"1min":0.33400000000000002,"5min":0.30499999999999999,"15min":0.28499999999999998},"max":{"1min":3.0779999999999998,"5min":6.4690000000000003,"15min":10.334},"last":0.503},{"interface":"front","average":{"1min":0.64800000000000002,"5min":1.0609999999999999,"15min":1.175},"min":{"1min":0.30499999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.486,"5min":7.0590000000000002,"15min":16.039999999999999},"last":0.60099999999999998}]},{"osd":6,"last update":"Tue Sep 10 19:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.653,"5min":1.252,"15min":1.452},"min":{"1min":0.379,"5min":0.33100000000000002,"15min":0.308},"max":{"1min":19.125,"5min":19.125,"15min":38.039999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":1.1459999999999999,"15min":1.526},"min":{"1min":0.36199999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":2.7090000000000001,"5min":10.404999999999999,"15min":38.100999999999999},"last":0.35999999999999999}]},{"osd":7,"last update":"Tue Sep 10 19:00:36 2024","interfaces":[{"interface":"back","average":{"1min":1.5489999999999999,"5min":1.2829999999999999,"15min":1.143},"min":{"1min":0.371,"5min":0.33100000000000002,"15min":0.315},"max":{"1min":19.169,"5min":19.169,"15min":19.169},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":1.0640000000000001,"15min":1.05},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.32800000000000001},"max":{"1min":0.91300000000000003,"5min":4.46,"15min":4.9029999999999996},"last":0.57699999999999996}]}]},{"osd":0,"up_from":241,"seq":1035087118401,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5014016,"kb_used_data":4680244,"kb_used_omap":4,"kb_used_meta":333755,"kb_avail":88731136,"statfs":{"total":95995035648,"available":90860683264,"internally_reserved":0,"allocated":4792569856,"data_stored":4791942702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":341765638},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 19:00:48 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.8899999999999999,"15min":2.4830000000000001},"min":{"1min":0.25700000000000001,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":6.9050000000000002,"5min":34.692,"15min":34.692},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.996,"5min":1.8080000000000001,"15min":2.4820000000000002},"min":{"1min":0.28199999999999997,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":7.5960000000000001,"5min":34.588999999999999,"15min":34.588999999999999},"last":0.371}]},{"osd":2,"last update":"Tue Sep 10 19:00:48 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.0589999999999999,"15min":0.93799999999999994},"min":{"1min":0.29999999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":6.8639999999999999,"5min":13.305,"15min":13.305},"last":0.75},{"interface":"front","average":{"1min":0.57999999999999996,"5min":1.036,"15min":0.80800000000000005},"min":{"1min":0.30599999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.5600000000000001,"5min":13.693,"15min":13.693},"last":0.46000000000000002}]},{"osd":3,"last update":"Tue Sep 10 19:00:48 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":1.4430000000000001,"15min":1.1279999999999999},"min":{"1min":0.39600000000000002,"5min":0.251,"15min":0.251},"max":{"1min":2.2309999999999999,"5min":21.108000000000001,"15min":21.108000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.97599999999999998,"5min":1.9830000000000001,"15min":1.7},"min":{"1min":0.35599999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":2.3159999999999998,"5min":11.629,"15min":11.629},"last":0.63}]},{"osd":4,"last update":"Tue Sep 10 19:00:48 2024","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.0760000000000001,"15min":0.83699999999999997},"min":{"1min":0.39800000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":3.1960000000000002,"5min":13.161,"15min":13.161},"last":1.095},{"interface":"front","average":{"1min":1.583,"5min":1.1850000000000001,"15min":0.997},"min":{"1min":0.47399999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.4590000000000001,"5min":5.0490000000000004,"15min":5.0490000000000004},"last":0.72499999999999998}]},{"osd":5,"last update":"Tue Sep 10 19:00:48 2024","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.2330000000000001,"15min":1.0840000000000001},"min":{"1min":0.47399999999999998,"5min":0.31,"15min":0.31},"max":{"1min":3.3679999999999999,"5min":5.125,"15min":5.125},"last":0.54800000000000004},{"interface":"front","average":{"1min":1.25,"5min":1.1419999999999999,"15min":0.90000000000000002},"min":{"1min":0.438,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":3.1179999999999999,"5min":13.202,"15min":13.202},"last":0.99299999999999999}]},{"osd":6,"last update":"Tue Sep 10 19:00:48 2024","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.198,"15min":1.008},"min":{"1min":0.504,"5min":0.312,"15min":0.312},"max":{"1min":3.4009999999999998,"5min":5.0899999999999999,"15min":5.0899999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":1.415,"5min":1.1020000000000001,"15min":0.83199999999999996},"min":{"1min":0.41899999999999998,"5min":0.313,"15min":0.313},"max":{"1min":3.3780000000000001,"5min":13.279,"15min":13.279},"last":3.5259999999999998}]},{"osd":7,"last update":"Tue Sep 10 19:00:48 2024","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.181,"15min":0.94499999999999995},"min":{"1min":0.45900000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":3.3679999999999999,"5min":13.233000000000001,"15min":13.233000000000001},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.135,"15min":1.143},"min":{"1min":0.439,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.7229999999999999,"5min":10.141999999999999,"15min":10.141999999999999},"last":0.91000000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":2668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4805820416,"data_stored":4805820416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2129199104,"data_stored":2129199104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3123773440,"data_stored":3123773440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5858394112,"data_stored":5858394112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2542665728,"data_stored":2542665728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2983723008,"data_stored":2983723008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T19:01:03.554 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T19:01:03.554 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:03.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:03.594 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:03.595 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:03.611 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 70 16 113290 113274 101.113 91.3125 0.00608074 0.00988583 2024-09-10T19:01:03.696 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:03.738 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:03.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:03.739 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:03.840 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:03.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:03.885 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:03.886 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:03.986 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:04.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:03 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:01:04.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:03 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:01:04.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:03 smithi103 ceph-mon[33163]: osdmap e348: 8 total, 8 up, 6 in 2024-09-10T19:01:04.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:03 smithi103 ceph-mon[33163]: pgmap v735: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 6.7 GiB data, 25 GiB used, 511 GiB / 536 GiB avail; 853 B/s rd, 70 MiB/s wr, 1.12k op/s; 11680/329421 objects misplaced (3.546%); 24 MiB/s, 384 objects/s recovering 2024-09-10T19:01:04.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:03 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:01:04.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:03 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:01:04.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:03 smithi018 ceph-mon[29336]: osdmap e348: 8 total, 8 up, 6 in 2024-09-10T19:01:04.026 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:03 smithi018 ceph-mon[29336]: pgmap v735: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 6.7 GiB data, 25 GiB used, 511 GiB / 536 GiB avail; 853 B/s rd, 70 MiB/s wr, 1.12k op/s; 11680/329421 objects misplaced (3.546%); 24 MiB/s, 384 objects/s recovering 2024-09-10T19:01:04.027 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:03 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:01:04.027 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:03 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:01:04.027 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:03 smithi018 ceph-mon[40196]: osdmap e348: 8 total, 8 up, 6 in 2024-09-10T19:01:04.027 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:03 smithi018 ceph-mon[40196]: pgmap v735: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 6.7 GiB data, 25 GiB used, 511 GiB / 536 GiB avail; 853 B/s rd, 70 MiB/s wr, 1.12k op/s; 11680/329421 objects misplaced (3.546%); 24 MiB/s, 384 objects/s recovering 2024-09-10T19:01:04.029 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:04.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:04.031 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:04.132 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:04.165 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:04.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:04.166 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:04.267 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:04.304 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:04.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:04.305 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:04.405 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:04.445 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:04.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:04.447 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:04.547 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:04.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:04.585 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:04.586 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:04.611 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 71 15 114713 114698 100.942 89 0.019182 0.00990244 2024-09-10T19:01:04.686 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:04.727 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:04.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:04.729 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:04.830 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:04.865 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:04.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:04.866 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:04.916 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T19:01:04.967 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:05.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:05.012 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:05.012 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:05.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:04 smithi103 ceph-mon[33163]: 1.b deep-scrub starts 2024-09-10T19:01:05.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:04 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:01:05.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:04 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:01:05.112 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:05.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:05.163 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:05.163 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:05.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:04 smithi018 ceph-mon[29336]: 1.b deep-scrub starts 2024-09-10T19:01:05.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:04 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:01:05.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:04 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:01:05.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:04 smithi018 ceph-mon[40196]: 1.b deep-scrub starts 2024-09-10T19:01:05.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:04 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:01:05.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:04 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:01:05.264 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:05.296 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:05.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:05.297 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:05.398 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:05.438 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:05.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:05.439 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:05.539 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:05.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:05.582 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:05.582 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:05.612 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 72 16 116046 116030 100.696 83.25 0.0076493 0.00992671 2024-09-10T19:01:05.683 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:05.722 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:05.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:05.723 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:05.824 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:05.858 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:05.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:05.859 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:05.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:05 smithi018 ceph-mon[40196]: osdmap e349: 8 total, 8 up, 6 in 2024-09-10T19:01:05.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:05 smithi018 ceph-mon[40196]: pgmap v737: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 6.8 GiB data, 25 GiB used, 511 GiB / 536 GiB avail; 853 B/s rd, 71 MiB/s wr, 1.13k op/s; 11248/333195 objects misplaced (3.376%); 25 MiB/s, 393 objects/s recovering 2024-09-10T19:01:05.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:05 smithi018 ceph-mon[29336]: osdmap e349: 8 total, 8 up, 6 in 2024-09-10T19:01:05.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:05 smithi018 ceph-mon[29336]: pgmap v737: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 6.8 GiB data, 25 GiB used, 511 GiB / 536 GiB avail; 853 B/s rd, 71 MiB/s wr, 1.13k op/s; 11248/333195 objects misplaced (3.376%); 25 MiB/s, 393 objects/s recovering 2024-09-10T19:01:05.960 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:05.992 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:05.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:05.993 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:06.093 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:06.126 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:06.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:06.127 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:06.228 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:06.264 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:06.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:06.266 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:06.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:05 smithi103 ceph-mon[33163]: osdmap e349: 8 total, 8 up, 6 in 2024-09-10T19:01:06.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:05 smithi103 ceph-mon[33163]: pgmap v737: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 6.8 GiB data, 25 GiB used, 511 GiB / 536 GiB avail; 853 B/s rd, 71 MiB/s wr, 1.13k op/s; 11248/333195 objects misplaced (3.376%); 25 MiB/s, 393 objects/s recovering 2024-09-10T19:01:06.366 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:06.406 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:06.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:06.407 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:06.508 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:06.541 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:06.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:06.542 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:06.612 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 73 16 117391 117375 100.468 84.0625 0.00261933 0.00994295 2024-09-10T19:01:06.642 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:06.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:06.664 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:06.664 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:06.765 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:06.790 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:06.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:06.791 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:06.892 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:06.916 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:06.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:06.917 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:07.018 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:07.052 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:07.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:07.053 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:07.153 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:07.189 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:07.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:07.190 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:07.272 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:01:06 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:01:07.290 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:07.332 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:07.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:07.332 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:07.433 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:07.462 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:07.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:07.463 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:07.563 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:07.600 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:07.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:07.601 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:07.612 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 74 16 118334 118318 99.9072 58.9375 0.00995565 0.010005 2024-09-10T19:01:07.702 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:07.731 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:07.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:07.732 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:07.833 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:07.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:07.873 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:07.873 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:07.973 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:08.006 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:08.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:08.007 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:08.108 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:08.144 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:08.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:08.145 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:08.246 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:08.286 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:08.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:08.287 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:08.387 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:08.413 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:08.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:08.414 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:08.515 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:08.552 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:08.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:08.554 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:08.612 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 75 15 119520 119505 99.5642 74.1875 0.00800903 0.0100395 2024-09-10T19:01:08.654 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:08.691 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:08.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:08.692 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:08.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:08 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:08.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:08 smithi018 ceph-mon[40196]: 5.10 scrub starts 2024-09-10T19:01:08.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:08 smithi018 ceph-mon[40196]: pgmap v738: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 7.1 GiB data, 27 GiB used, 510 GiB / 536 GiB avail; 1.5 KiB/s rd, 116 MiB/s wr, 1.86k op/s; 9973/350037 objects misplaced (2.849%); 38 MiB/s, 608 objects/s recovering 2024-09-10T19:01:08.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:08 smithi018 ceph-mon[40196]: 2.15 scrub starts 2024-09-10T19:01:08.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:08 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:08.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:08 smithi018 ceph-mon[29336]: 5.10 scrub starts 2024-09-10T19:01:08.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:08 smithi018 ceph-mon[29336]: pgmap v738: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 7.1 GiB data, 27 GiB used, 510 GiB / 536 GiB avail; 1.5 KiB/s rd, 116 MiB/s wr, 1.86k op/s; 9973/350037 objects misplaced (2.849%); 38 MiB/s, 608 objects/s recovering 2024-09-10T19:01:08.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:08 smithi018 ceph-mon[29336]: 2.15 scrub starts 2024-09-10T19:01:08.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:08 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:08.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:08 smithi103 ceph-mon[33163]: 5.10 scrub starts 2024-09-10T19:01:08.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:08 smithi103 ceph-mon[33163]: pgmap v738: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 7.1 GiB data, 27 GiB used, 510 GiB / 536 GiB avail; 1.5 KiB/s rd, 116 MiB/s wr, 1.86k op/s; 9973/350037 objects misplaced (2.849%); 38 MiB/s, 608 objects/s recovering 2024-09-10T19:01:08.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:08 smithi103 ceph-mon[33163]: 2.15 scrub starts 2024-09-10T19:01:08.793 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:08.816 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:08.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:08.817 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:08.918 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:08.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:08.958 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:08.958 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:09.058 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:09.097 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:09.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:09.098 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:09.199 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:09.234 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:09.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:09.235 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:09.336 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:09.371 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:09.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:09.372 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:09.473 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:09.496 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:09.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:09.497 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:09.597 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:09.612 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 76 16 120798 120782 99.3042 79.8125 0.00495046 0.0100656 2024-09-10T19:01:09.632 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:09.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:09.633 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:09.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:09 smithi018 ceph-mon[40196]: 6.14 scrub starts 2024-09-10T19:01:09.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:09 smithi018 ceph-mon[40196]: osdmap e350: 8 total, 8 up, 6 in 2024-09-10T19:01:09.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:09 smithi018 ceph-mon[29336]: 6.14 scrub starts 2024-09-10T19:01:09.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:09 smithi018 ceph-mon[29336]: osdmap e350: 8 total, 8 up, 6 in 2024-09-10T19:01:09.734 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:09.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:09 smithi103 ceph-mon[33163]: 6.14 scrub starts 2024-09-10T19:01:09.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:09 smithi103 ceph-mon[33163]: osdmap e350: 8 total, 8 up, 6 in 2024-09-10T19:01:09.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:09.774 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:09.774 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:09.875 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:09.912 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:09.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:09.913 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:10.014 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:10.049 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:10.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:10.050 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:10.151 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:10.190 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:10.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:10.191 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:10.292 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:10.324 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:10.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:10.325 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:10.426 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:10.449 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:10.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:10.450 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:10.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:10 smithi103 ceph-mon[33163]: 4.1a scrub starts 2024-09-10T19:01:10.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:10 smithi103 ceph-mon[33163]: 6.4 scrub starts 2024-09-10T19:01:10.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:10 smithi103 ceph-mon[33163]: pgmap v740: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 7.2 GiB data, 27 GiB used, 510 GiB / 536 GiB avail; 771 B/s rd, 76 MiB/s wr, 1.21k op/s; 9388/353523 objects misplaced (2.656%); 30 MiB/s, 475 objects/s recovering 2024-09-10T19:01:10.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:10 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:01:10.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:10 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:01:10.523 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:01:10 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:01:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:01:10.551 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:10.589 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:10.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:10.590 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:10.612 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 77 16 122054 122038 99.034 78.5 0.00562721 0.0100937 2024-09-10T19:01:10.691 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:10.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:10 smithi018 ceph-mon[40196]: 4.1a scrub starts 2024-09-10T19:01:10.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:10 smithi018 ceph-mon[40196]: 6.4 scrub starts 2024-09-10T19:01:10.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:10 smithi018 ceph-mon[40196]: pgmap v740: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 7.2 GiB data, 27 GiB used, 510 GiB / 536 GiB avail; 771 B/s rd, 76 MiB/s wr, 1.21k op/s; 9388/353523 objects misplaced (2.656%); 30 MiB/s, 475 objects/s recovering 2024-09-10T19:01:10.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:10 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:01:10.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:10 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:01:10.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:10 smithi018 ceph-mon[29336]: 4.1a scrub starts 2024-09-10T19:01:10.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:10 smithi018 ceph-mon[29336]: 6.4 scrub starts 2024-09-10T19:01:10.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:10 smithi018 ceph-mon[29336]: pgmap v740: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 7.2 GiB data, 27 GiB used, 510 GiB / 536 GiB avail; 771 B/s rd, 76 MiB/s wr, 1.21k op/s; 9388/353523 objects misplaced (2.656%); 30 MiB/s, 475 objects/s recovering 2024-09-10T19:01:10.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:10 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:01:10.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:10 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:01:10.724 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:10.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:10.725 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:10.826 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:10.867 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:10.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:10.868 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:10.968 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:11.003 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:11.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:11.004 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:11.105 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:11.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:11.143 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:11.143 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:11.244 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:11.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:11.285 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:11.285 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:11.386 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:11.414 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:11.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:11.416 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:11.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:01:10 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:01:10] "GET /metrics HTTP/1.1" 200 224743 "" "Prometheus/2.43.0" 2024-09-10T19:01:11.517 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:11.556 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:11.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:11.557 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:11.612 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 78 16 123360 123344 98.8108 81.625 0.00340111 0.0101155 2024-09-10T19:01:11.658 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:11.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:11.697 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:11.697 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:11.798 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:11.834 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:11.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:11.835 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:11.936 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:11.972 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:11.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:11.973 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:12.074 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:12.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:12.111 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:12.111 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:12.212 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:12.235 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:12.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:12.236 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:12.337 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:12.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:12.375 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:12.375 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:12.476 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:12.517 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:12.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:12.518 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:12.612 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 79 16 124753 124737 98.662 87.0625 0.00839575 0.0101315 2024-09-10T19:01:12.620 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:12.658 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:12.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:12.659 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:12.760 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:12.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:12 smithi103 ceph-mon[33163]: pgmap v741: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 7.2 GiB data, 27 GiB used, 509 GiB / 536 GiB avail; 639 B/s rd, 71 MiB/s wr, 1.14k op/s; 8593/356808 objects misplaced (2.408%); 32 MiB/s, 508 objects/s recovering 2024-09-10T19:01:12.794 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:12.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:12.796 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:12.897 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:12.932 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:12.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:12.933 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:12.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:12 smithi018 ceph-mon[40196]: pgmap v741: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 7.2 GiB data, 27 GiB used, 509 GiB / 536 GiB avail; 639 B/s rd, 71 MiB/s wr, 1.14k op/s; 8593/356808 objects misplaced (2.408%); 32 MiB/s, 508 objects/s recovering 2024-09-10T19:01:12.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:12 smithi018 ceph-mon[29336]: pgmap v741: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 7.2 GiB data, 27 GiB used, 509 GiB / 536 GiB avail; 639 B/s rd, 71 MiB/s wr, 1.14k op/s; 8593/356808 objects misplaced (2.408%); 32 MiB/s, 508 objects/s recovering 2024-09-10T19:01:13.034 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:13.069 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:13.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:13.070 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:13.171 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:13.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:13.212 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:13.212 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:13.313 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:13.353 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:13.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:13.354 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:13.455 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:13.489 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:13.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:13.490 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:13.591 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:13.612 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:2024-09-10T19:01:13.612316+0000 min lat: 0.00113481 max lat: 2.35071 avg lat: 0.0101496 2024-09-10T19:01:13.612 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T19:01:13.612 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 80 15 126107 126092 98.4872 84.6875 0.0064737 0.0101496 2024-09-10T19:01:13.624 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:13.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:13.625 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:13.725 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:13.766 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:13.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:13.767 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:13.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:13 smithi103 ceph-mon[33163]: 5.a scrub starts 2024-09-10T19:01:13.799 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:13 smithi018 ceph-mon[29336]: 5.a scrub starts 2024-09-10T19:01:13.800 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:13 smithi018 ceph-mon[40196]: 5.a scrub starts 2024-09-10T19:01:13.868 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:13.925 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:13.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:13.926 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:14.027 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:14.064 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:14.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:14.065 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:14.165 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:14.203 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:14.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:14.204 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:14.305 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:14.350 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:14.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:14.351 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:14.452 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:14.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:14.498 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:14.498 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:14.600 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:14.613 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 81 16 127473 127457 98.3234 85.3125 0.0155669 0.0101658 2024-09-10T19:01:14.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:14.638 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:14.638 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:14.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:14 smithi018 ceph-mon[40196]: pgmap v742: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 7.5 GiB data, 28 GiB used, 509 GiB / 536 GiB avail; 1.2 KiB/s rd, 85 MiB/s wr, 1.36k op/s; 7510/368253 objects misplaced (2.039%); 33 MiB/s, 531 objects/s recovering 2024-09-10T19:01:14.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:14 smithi018 ceph-mon[29336]: pgmap v742: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 7.5 GiB data, 28 GiB used, 509 GiB / 536 GiB avail; 1.2 KiB/s rd, 85 MiB/s wr, 1.36k op/s; 7510/368253 objects misplaced (2.039%); 33 MiB/s, 531 objects/s recovering 2024-09-10T19:01:14.739 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:14.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:14 smithi103 ceph-mon[33163]: pgmap v742: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 7.5 GiB data, 28 GiB used, 509 GiB / 536 GiB avail; 1.2 KiB/s rd, 85 MiB/s wr, 1.36k op/s; 7510/368253 objects misplaced (2.039%); 33 MiB/s, 531 objects/s recovering 2024-09-10T19:01:14.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:14.783 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:14.783 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:14.884 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:14.920 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:14.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:14.921 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:15.022 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:15.067 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:15.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:15.068 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:15.077 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:01:15.145 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T19:01:15.145 INFO:teuthology.orchestra.run.smithi018.stdout: "size": 20, 2024-09-10T19:01:15.145 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 600, 2024-09-10T19:01:15.146 INFO:teuthology.orchestra.run.smithi018.stdout: "ops": [ 2024-09-10T19:01:15.146 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.146 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.24947.0:115928 8.a 8:5420eb55:::benchmark_data_smithi018_106531_object115927:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e182)", 2024-09-10T19:01:15.146 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:52:53.580101+0000", 2024-09-10T19:01:15.146 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 501.55853231499998, 2024-09-10T19:01:15.146 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.056086187, 2024-09-10T19:01:15.146 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:01:15.146 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:01:15.146 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:01:15.146 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.24947", 2024-09-10T19:01:15.146 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T19:01:15.146 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 115928 2024-09-10T19:01:15.146 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.146 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:01:15.147 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.147 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:01:15.147 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.580101+0000", 2024-09-10T19:01:15.147 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.147 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.147 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.147 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:01:15.147 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.580101+0000", 2024-09-10T19:01:15.147 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.147 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.147 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.147 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:01:15.148 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.580099+0000", 2024-09-10T19:01:15.148 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T19:01:15.148 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.148 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.148 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:01:15.148 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.580474+0000", 2024-09-10T19:01:15.148 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00037428400000000001 2024-09-10T19:01:15.148 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.148 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.148 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:01:15.148 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.580488+0000", 2024-09-10T19:01:15.149 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.4428e-05 2024-09-10T19:01:15.149 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.149 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.149 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:01:15.149 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.580652+0000", 2024-09-10T19:01:15.149 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000163528 2024-09-10T19:01:15.149 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.149 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.149 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.149 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.585602+0000", 2024-09-10T19:01:15.149 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0049509269999999999 2024-09-10T19:01:15.150 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.150 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.150 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:01:15.150 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.585613+0000", 2024-09-10T19:01:15.150 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0916e-05 2024-09-10T19:01:15.150 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.150 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.150 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.150 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:54.617015+0000", 2024-09-10T19:01:15.150 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0314019210000001 2024-09-10T19:01:15.150 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.150 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.150 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:01:15.150 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:54.617030+0000", 2024-09-10T19:01:15.150 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.4482e-05 2024-09-10T19:01:15.151 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.151 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.151 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.151 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.628678+0000", 2024-09-10T19:01:15.151 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0116481690000001 2024-09-10T19:01:15.151 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.151 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.151 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:01:15.151 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.628837+0000", 2024-09-10T19:01:15.151 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00015945099999999999 2024-09-10T19:01:15.151 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.151 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.151 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,4,5", 2024-09-10T19:01:15.151 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.629304+0000", 2024-09-10T19:01:15.152 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00046692899999999999 2024-09-10T19:01:15.152 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.152 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.152 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.152 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.633362+0000", 2024-09-10T19:01:15.152 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0040577089999999996 2024-09-10T19:01:15.152 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.152 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.152 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.152 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.633422+0000", 2024-09-10T19:01:15.152 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.9514e-05 2024-09-10T19:01:15.152 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.152 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.152 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.153 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.636000+0000", 2024-09-10T19:01:15.153 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.002577986 2024-09-10T19:01:15.153 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.153 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.153 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:01:15.153 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.636054+0000", 2024-09-10T19:01:15.153 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.4061999999999998e-05 2024-09-10T19:01:15.153 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.153 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.153 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:01:15.153 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.636096+0000", 2024-09-10T19:01:15.153 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.2700000000000001e-05 2024-09-10T19:01:15.153 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.154 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.154 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:01:15.154 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.636187+0000", 2024-09-10T19:01:15.154 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.0780999999999997e-05 2024-09-10T19:01:15.154 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.154 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:01:15.154 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.154 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.154 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.154 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.24947.0:115929 8.a 8:5a107eae:::benchmark_data_smithi018_106531_object115928:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e182)", 2024-09-10T19:01:15.154 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:52:53.580660+0000", 2024-09-10T19:01:15.154 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 501.557973484, 2024-09-10T19:01:15.154 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.0559732880000001, 2024-09-10T19:01:15.154 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:01:15.155 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:01:15.155 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:01:15.155 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.24947", 2024-09-10T19:01:15.155 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T19:01:15.155 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 115929 2024-09-10T19:01:15.155 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.155 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:01:15.155 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.155 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:01:15.155 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.580660+0000", 2024-09-10T19:01:15.155 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.155 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.155 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.156 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:01:15.156 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.580660+0000", 2024-09-10T19:01:15.156 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.156 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.156 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.156 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:01:15.156 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.580659+0000", 2024-09-10T19:01:15.156 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T19:01:15.156 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.156 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.156 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:01:15.156 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.580877+0000", 2024-09-10T19:01:15.156 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00021799499999999999 2024-09-10T19:01:15.156 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.157 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.157 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:01:15.157 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.581279+0000", 2024-09-10T19:01:15.157 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00040172899999999998 2024-09-10T19:01:15.157 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.157 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.157 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:01:15.157 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.581286+0000", 2024-09-10T19:01:15.157 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.0979999999999996e-06 2024-09-10T19:01:15.157 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.157 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.157 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.157 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.585645+0000", 2024-09-10T19:01:15.157 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0043590030000000002 2024-09-10T19:01:15.158 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.158 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.158 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:01:15.158 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.585653+0000", 2024-09-10T19:01:15.158 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.9729999999999997e-06 2024-09-10T19:01:15.158 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.158 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.158 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.158 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:54.617054+0000", 2024-09-10T19:01:15.158 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0314011700000001 2024-09-10T19:01:15.158 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.158 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.158 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:01:15.159 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:54.617061+0000", 2024-09-10T19:01:15.159 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.3300000000000001e-06 2024-09-10T19:01:15.159 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.159 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.159 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.159 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.633452+0000", 2024-09-10T19:01:15.159 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.016390833 2024-09-10T19:01:15.159 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.159 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.159 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:01:15.159 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.633606+0000", 2024-09-10T19:01:15.159 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00015381999999999999 2024-09-10T19:01:15.159 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.159 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.159 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,4,5", 2024-09-10T19:01:15.160 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.633783+0000", 2024-09-10T19:01:15.160 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00017709299999999999 2024-09-10T19:01:15.160 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.160 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.160 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.160 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.636044+0000", 2024-09-10T19:01:15.160 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0022610640000000001 2024-09-10T19:01:15.160 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.160 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.160 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.160 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.636233+0000", 2024-09-10T19:01:15.160 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000188739 2024-09-10T19:01:15.160 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.161 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.161 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:01:15.161 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.636420+0000", 2024-09-10T19:01:15.161 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00018738399999999999 2024-09-10T19:01:15.161 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.161 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.161 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.161 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.636475+0000", 2024-09-10T19:01:15.161 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.4879000000000001e-05 2024-09-10T19:01:15.161 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.161 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.161 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:01:15.161 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.636527+0000", 2024-09-10T19:01:15.162 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.1987000000000002e-05 2024-09-10T19:01:15.162 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.162 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.162 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:01:15.162 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.636633+0000", 2024-09-10T19:01:15.162 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000105927 2024-09-10T19:01:15.162 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.162 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:01:15.162 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.162 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.162 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.162 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.24947.0:115931 8.a 8:5c47ed93:::benchmark_data_smithi018_106531_object115930:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e183)", 2024-09-10T19:01:15.162 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:52:53.590366+0000", 2024-09-10T19:01:15.162 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 501.54826675300001, 2024-09-10T19:01:15.163 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.0741490819999999, 2024-09-10T19:01:15.163 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:01:15.163 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:01:15.163 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:01:15.163 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.24947", 2024-09-10T19:01:15.163 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T19:01:15.163 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 115931 2024-09-10T19:01:15.163 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.163 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:01:15.163 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.163 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:01:15.163 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.590366+0000", 2024-09-10T19:01:15.163 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.163 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.163 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.164 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:01:15.164 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.590366+0000", 2024-09-10T19:01:15.164 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.164 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.164 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.164 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:01:15.164 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.590365+0000", 2024-09-10T19:01:15.164 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T19:01:15.164 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.164 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.164 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:01:15.164 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.590390+0000", 2024-09-10T19:01:15.164 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.4545999999999998e-05 2024-09-10T19:01:15.164 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.165 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.165 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:01:15.165 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.590405+0000", 2024-09-10T19:01:15.165 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.4952e-05 2024-09-10T19:01:15.165 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.165 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.165 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:01:15.165 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.590411+0000", 2024-09-10T19:01:15.165 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.4559999999999997e-06 2024-09-10T19:01:15.165 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.165 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.165 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.166 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.590474+0000", 2024-09-10T19:01:15.166 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.3231000000000005e-05 2024-09-10T19:01:15.166 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.166 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.166 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:01:15.166 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.590484+0000", 2024-09-10T19:01:15.166 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0006999999999999e-05 2024-09-10T19:01:15.166 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.166 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.166 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.166 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:54.617089+0000", 2024-09-10T19:01:15.166 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.026605134 2024-09-10T19:01:15.166 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.166 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.167 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:01:15.167 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:54.617096+0000", 2024-09-10T19:01:15.167 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.866e-06 2024-09-10T19:01:15.167 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.167 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.167 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.167 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.636654+0000", 2024-09-10T19:01:15.167 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.019557585 2024-09-10T19:01:15.167 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.167 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.167 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:01:15.167 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.636733+0000", 2024-09-10T19:01:15.167 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.9066000000000005e-05 2024-09-10T19:01:15.167 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.167 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.168 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,4,5", 2024-09-10T19:01:15.168 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.636829+0000", 2024-09-10T19:01:15.168 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.6119000000000006e-05 2024-09-10T19:01:15.168 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.168 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.168 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.168 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.664235+0000", 2024-09-10T19:01:15.168 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.027405763999999999 2024-09-10T19:01:15.168 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.168 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.168 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:01:15.168 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.664250+0000", 2024-09-10T19:01:15.168 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.5218999999999999e-05 2024-09-10T19:01:15.168 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.169 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.169 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.169 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.664298+0000", 2024-09-10T19:01:15.169 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.8056999999999997e-05 2024-09-10T19:01:15.169 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.169 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.169 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.169 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.664364+0000", 2024-09-10T19:01:15.169 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.5344000000000003e-05 2024-09-10T19:01:15.169 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.169 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.170 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:01:15.170 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.664413+0000", 2024-09-10T19:01:15.170 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.9968e-05 2024-09-10T19:01:15.170 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.170 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.170 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:01:15.170 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.664516+0000", 2024-09-10T19:01:15.170 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000102097 2024-09-10T19:01:15.170 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.170 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:01:15.170 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.171 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.171 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.171 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.24947.0:115952 8.a 8:50b01674:::benchmark_data_smithi018_106531_object115951:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e183)", 2024-09-10T19:01:15.171 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:52:53.610166+0000", 2024-09-10T19:01:15.171 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 501.52846748500002, 2024-09-10T19:01:15.171 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.0609831989999998, 2024-09-10T19:01:15.171 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:01:15.171 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:01:15.171 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:01:15.171 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.24947", 2024-09-10T19:01:15.171 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T19:01:15.171 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 115952 2024-09-10T19:01:15.171 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.171 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:01:15.172 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.172 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:01:15.172 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.610166+0000", 2024-09-10T19:01:15.172 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.172 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.172 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.172 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:01:15.172 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.610166+0000", 2024-09-10T19:01:15.172 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.172 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.172 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.172 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:01:15.172 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.610164+0000", 2024-09-10T19:01:15.172 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T19:01:15.173 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.173 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.173 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:01:15.173 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.610192+0000", 2024-09-10T19:01:15.173 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.7613000000000001e-05 2024-09-10T19:01:15.173 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.173 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.173 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:01:15.173 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.610210+0000", 2024-09-10T19:01:15.173 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.8071999999999999e-05 2024-09-10T19:01:15.173 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.173 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:15.174 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.174 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:01:15.174 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.610217+0000", 2024-09-10T19:01:15.175 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.9550000000000003e-06 2024-09-10T19:01:15.175 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.175 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.175 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.175 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.610277+0000", 2024-09-10T19:01:15.175 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.0344999999999999e-05 2024-09-10T19:01:15.175 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.175 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.175 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:01:15.175 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.610288+0000", 2024-09-10T19:01:15.175 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0710000000000001e-05 2024-09-10T19:01:15.175 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.175 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.176 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.176 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:54.617117+0000", 2024-09-10T19:01:15.176 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0068289189999999 2024-09-10T19:01:15.176 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.176 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.176 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:01:15.176 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:54.617123+0000", 2024-09-10T19:01:15.176 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.674e-06 2024-09-10T19:01:15.176 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.176 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.176 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.176 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.661662+0000", 2024-09-10T19:01:15.176 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0445389920000001 2024-09-10T19:01:15.177 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.177 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.177 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:01:15.177 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.661812+0000", 2024-09-10T19:01:15.177 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00014981099999999999 2024-09-10T19:01:15.177 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.177 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.177 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,4,5", 2024-09-10T19:01:15.177 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.661962+0000", 2024-09-10T19:01:15.177 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00014974500000000001 2024-09-10T19:01:15.177 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.177 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.177 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.178 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.664329+0000", 2024-09-10T19:01:15.178 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0023671030000000002 2024-09-10T19:01:15.178 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.178 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.178 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.178 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.664557+0000", 2024-09-10T19:01:15.178 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00022776200000000001 2024-09-10T19:01:15.178 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.178 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.178 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:01:15.178 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.670860+0000", 2024-09-10T19:01:15.178 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0063028540000000001 2024-09-10T19:01:15.178 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.179 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.179 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.179 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.670909+0000", 2024-09-10T19:01:15.179 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.9119000000000002e-05 2024-09-10T19:01:15.179 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.179 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.179 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:01:15.179 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.671025+0000", 2024-09-10T19:01:15.179 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00011587900000000001 2024-09-10T19:01:15.179 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.179 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.179 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:01:15.179 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.671149+0000", 2024-09-10T19:01:15.179 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00012427900000000001 2024-09-10T19:01:15.180 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.180 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:01:15.180 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.180 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.180 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.180 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.24947.0:115968 8.a 8:5dcb8429:::benchmark_data_smithi018_106531_object115967:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e183)", 2024-09-10T19:01:15.180 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:52:53.621562+0000", 2024-09-10T19:01:15.180 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 501.51707168500002, 2024-09-10T19:01:15.180 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.0499422599999999, 2024-09-10T19:01:15.180 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:01:15.180 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:01:15.181 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:01:15.181 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.24947", 2024-09-10T19:01:15.181 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T19:01:15.181 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 115968 2024-09-10T19:01:15.181 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.181 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:01:15.181 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.181 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:01:15.181 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.621562+0000", 2024-09-10T19:01:15.181 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.181 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.181 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.181 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:01:15.182 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.621562+0000", 2024-09-10T19:01:15.182 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.182 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.182 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.182 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:01:15.182 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.621560+0000", 2024-09-10T19:01:15.182 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T19:01:15.182 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.182 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.182 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:01:15.182 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.621586+0000", 2024-09-10T19:01:15.182 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.6268e-05 2024-09-10T19:01:15.182 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.182 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.182 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:01:15.183 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.621599+0000", 2024-09-10T19:01:15.183 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.2449999999999999e-05 2024-09-10T19:01:15.183 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.183 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.183 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:01:15.183 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.621604+0000", 2024-09-10T19:01:15.183 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.5990000000000001e-06 2024-09-10T19:01:15.183 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.183 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.183 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.183 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.621702+0000", 2024-09-10T19:01:15.183 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.7917000000000005e-05 2024-09-10T19:01:15.183 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.183 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.184 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:01:15.184 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.621712+0000", 2024-09-10T19:01:15.184 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.3840000000000003e-06 2024-09-10T19:01:15.184 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.184 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.184 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.184 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:54.617143+0000", 2024-09-10T19:01:15.184 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.99543151200000002 2024-09-10T19:01:15.184 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.184 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.184 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:01:15.184 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:54.617150+0000", 2024-09-10T19:01:15.184 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.533e-06 2024-09-10T19:01:15.184 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.185 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.185 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.185 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.664600+0000", 2024-09-10T19:01:15.185 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0474503909999999 2024-09-10T19:01:15.185 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.185 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.185 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:01:15.185 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.664725+0000", 2024-09-10T19:01:15.185 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00012484799999999999 2024-09-10T19:01:15.185 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.185 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.185 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,4,5", 2024-09-10T19:01:15.185 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.664932+0000", 2024-09-10T19:01:15.185 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000207231 2024-09-10T19:01:15.185 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.186 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.186 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:01:15.186 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.671166+0000", 2024-09-10T19:01:15.186 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0062342480000000004 2024-09-10T19:01:15.186 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.186 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.186 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.186 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.671204+0000", 2024-09-10T19:01:15.186 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.7614000000000003e-05 2024-09-10T19:01:15.186 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.186 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.186 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.186 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.671237+0000", 2024-09-10T19:01:15.186 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.3256999999999997e-05 2024-09-10T19:01:15.186 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.187 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.187 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.187 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.671267+0000", 2024-09-10T19:01:15.187 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.0247999999999999e-05 2024-09-10T19:01:15.187 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.187 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.187 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:01:15.187 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.671313+0000", 2024-09-10T19:01:15.187 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.5765e-05 2024-09-10T19:01:15.187 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.187 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.187 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:01:15.187 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.671504+0000", 2024-09-10T19:01:15.188 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00019061300000000001 2024-09-10T19:01:15.188 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.188 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:01:15.188 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.188 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.188 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.188 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.24947.0:115969 8.a 8:589675ca:::benchmark_data_smithi018_106531_object115968:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e183)", 2024-09-10T19:01:15.188 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:52:53.621617+0000", 2024-09-10T19:01:15.188 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 501.51701640900001, 2024-09-10T19:01:15.188 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.0619245199999998, 2024-09-10T19:01:15.188 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:01:15.188 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:01:15.189 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:01:15.189 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.24947", 2024-09-10T19:01:15.189 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T19:01:15.189 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 115969 2024-09-10T19:01:15.189 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.189 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:01:15.189 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.189 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:01:15.189 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.621617+0000", 2024-09-10T19:01:15.189 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.189 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.189 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.190 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:01:15.190 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.621617+0000", 2024-09-10T19:01:15.190 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.190 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.190 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.190 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:01:15.190 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.621616+0000", 2024-09-10T19:01:15.190 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T19:01:15.190 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.190 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.190 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:01:15.191 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.621638+0000", 2024-09-10T19:01:15.191 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.2214000000000001e-05 2024-09-10T19:01:15.191 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.191 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.191 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:01:15.191 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.621646+0000", 2024-09-10T19:01:15.191 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.0299999999999996e-06 2024-09-10T19:01:15.191 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.191 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.191 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:01:15.191 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.621650+0000", 2024-09-10T19:01:15.191 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.7049999999999998e-06 2024-09-10T19:01:15.192 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.192 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.192 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.192 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.621732+0000", 2024-09-10T19:01:15.192 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.2174000000000003e-05 2024-09-10T19:01:15.192 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.192 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.192 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:01:15.192 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.621739+0000", 2024-09-10T19:01:15.192 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.5899999999999996e-06 2024-09-10T19:01:15.192 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.192 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.193 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.193 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:54.617179+0000", 2024-09-10T19:01:15.193 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.99543985300000004 2024-09-10T19:01:15.193 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.193 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.193 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:01:15.193 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:54.617186+0000", 2024-09-10T19:01:15.193 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.2030000000000003e-06 2024-09-10T19:01:15.193 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.193 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.193 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.193 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.667446+0000", 2024-09-10T19:01:15.193 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0502600440000001 2024-09-10T19:01:15.193 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.194 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.194 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:01:15.194 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.667758+0000", 2024-09-10T19:01:15.194 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00031175400000000002 2024-09-10T19:01:15.194 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.194 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.194 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,4,5", 2024-09-10T19:01:15.194 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.667931+0000", 2024-09-10T19:01:15.194 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00017359 2024-09-10T19:01:15.194 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.194 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.194 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.194 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.671570+0000", 2024-09-10T19:01:15.194 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0036383489999999999 2024-09-10T19:01:15.195 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.195 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.195 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.195 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.671618+0000", 2024-09-10T19:01:15.195 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.7781000000000002e-05 2024-09-10T19:01:15.195 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.195 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.195 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.195 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.677149+0000", 2024-09-10T19:01:15.195 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0055318720000000002 2024-09-10T19:01:15.195 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.195 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.195 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:01:15.195 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.683384+0000", 2024-09-10T19:01:15.196 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0062348330000000004 2024-09-10T19:01:15.196 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.196 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.196 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:01:15.196 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.683435+0000", 2024-09-10T19:01:15.196 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.0929e-05 2024-09-10T19:01:15.196 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.196 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.196 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:01:15.196 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.683541+0000", 2024-09-10T19:01:15.196 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000106153 2024-09-10T19:01:15.196 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.196 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:01:15.197 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.197 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.197 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.197 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.24947.0:115975 8.a 8:59048f38:::benchmark_data_smithi018_106531_object115974:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e183)", 2024-09-10T19:01:15.197 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:52:53.626419+0000", 2024-09-10T19:01:15.197 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 501.51221429899999, 2024-09-10T19:01:15.197 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.057331843, 2024-09-10T19:01:15.197 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:01:15.197 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:01:15.197 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:01:15.197 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.24947", 2024-09-10T19:01:15.197 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T19:01:15.197 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 115975 2024-09-10T19:01:15.198 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.198 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:01:15.198 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.198 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:01:15.198 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.626419+0000", 2024-09-10T19:01:15.198 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.198 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.198 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.198 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:01:15.198 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.626419+0000", 2024-09-10T19:01:15.198 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.198 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.198 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.199 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:01:15.199 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.626418+0000", 2024-09-10T19:01:15.199 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T19:01:15.199 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.199 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.199 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:01:15.199 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.626452+0000", 2024-09-10T19:01:15.199 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.4986e-05 2024-09-10T19:01:15.199 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.199 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.199 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:01:15.199 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.626469+0000", 2024-09-10T19:01:15.199 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.6015000000000002e-05 2024-09-10T19:01:15.200 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.200 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.200 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:01:15.200 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.626474+0000", 2024-09-10T19:01:15.200 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.8540000000000002e-06 2024-09-10T19:01:15.200 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.200 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.200 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.200 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.626525+0000", 2024-09-10T19:01:15.200 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.0571000000000001e-05 2024-09-10T19:01:15.200 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.200 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.201 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:01:15.201 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.626535+0000", 2024-09-10T19:01:15.201 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.7159999999999998e-06 2024-09-10T19:01:15.201 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.201 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.201 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.201 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:54.617208+0000", 2024-09-10T19:01:15.201 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.99067301600000002 2024-09-10T19:01:15.201 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.201 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.201 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:01:15.201 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:54.617214+0000", 2024-09-10T19:01:15.201 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.5769999999999999e-06 2024-09-10T19:01:15.201 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.202 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.202 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.202 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.671675+0000", 2024-09-10T19:01:15.202 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0544608659999999 2024-09-10T19:01:15.202 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.202 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.202 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:01:15.202 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.671809+0000", 2024-09-10T19:01:15.202 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0001337 2024-09-10T19:01:15.202 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.202 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.202 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,4,5", 2024-09-10T19:01:15.202 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.672010+0000", 2024-09-10T19:01:15.202 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00020144800000000001 2024-09-10T19:01:15.203 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.203 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.203 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:01:15.203 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.204 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.677181+0000", 2024-09-10T19:01:15.204 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0051704940000000003 2024-09-10T19:01:15.204 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.204 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.204 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.204 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.677212+0000", 2024-09-10T19:01:15.204 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.1239999999999999e-05 2024-09-10T19:01:15.204 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.204 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.204 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.204 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.677244+0000", 2024-09-10T19:01:15.204 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.1921999999999997e-05 2024-09-10T19:01:15.204 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.205 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.205 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:01:15.205 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.683555+0000", 2024-09-10T19:01:15.205 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0063106530000000003 2024-09-10T19:01:15.205 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.205 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.205 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:01:15.205 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.683647+0000", 2024-09-10T19:01:15.205 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.2014000000000004e-05 2024-09-10T19:01:15.205 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.205 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.205 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:01:15.205 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.683751+0000", 2024-09-10T19:01:15.205 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000104212 2024-09-10T19:01:15.206 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.206 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:01:15.206 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.206 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.206 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.206 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.24947.0:115984 8.a 8:544a87bb:::benchmark_data_smithi018_106531_object115983:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e183)", 2024-09-10T19:01:15.206 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:52:53.635053+0000", 2024-09-10T19:01:15.206 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 501.50358051400002, 2024-09-10T19:01:15.206 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.049048564, 2024-09-10T19:01:15.206 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:01:15.206 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:01:15.206 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:01:15.206 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.24947", 2024-09-10T19:01:15.207 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T19:01:15.207 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 115984 2024-09-10T19:01:15.207 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.207 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:01:15.207 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.207 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:01:15.207 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.635053+0000", 2024-09-10T19:01:15.207 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.207 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.207 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.207 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:01:15.207 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.635053+0000", 2024-09-10T19:01:15.207 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.207 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.207 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.208 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:01:15.208 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.635051+0000", 2024-09-10T19:01:15.208 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999981 2024-09-10T19:01:15.208 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.208 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.208 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:01:15.208 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.635090+0000", 2024-09-10T19:01:15.208 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.8859999999999997e-05 2024-09-10T19:01:15.208 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.208 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.208 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:01:15.208 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.635108+0000", 2024-09-10T19:01:15.208 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.7876999999999998e-05 2024-09-10T19:01:15.209 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.209 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.209 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:01:15.209 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.635115+0000", 2024-09-10T19:01:15.209 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.9210000000000003e-06 2024-09-10T19:01:15.209 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.209 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.209 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.209 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.635188+0000", 2024-09-10T19:01:15.209 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.3081e-05 2024-09-10T19:01:15.209 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.209 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.209 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:01:15.210 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.635197+0000", 2024-09-10T19:01:15.210 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.4890000000000001e-06 2024-09-10T19:01:15.210 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.210 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.210 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.210 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:54.617383+0000", 2024-09-10T19:01:15.210 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.98218564399999997 2024-09-10T19:01:15.210 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.210 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.210 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:01:15.210 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:54.617393+0000", 2024-09-10T19:01:15.210 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0346999999999999e-05 2024-09-10T19:01:15.211 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.211 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.211 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.211 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.674749+0000", 2024-09-10T19:01:15.211 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0573558780000001 2024-09-10T19:01:15.211 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.211 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.211 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:01:15.211 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.674884+0000", 2024-09-10T19:01:15.211 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000134559 2024-09-10T19:01:15.211 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.211 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.211 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,4,5", 2024-09-10T19:01:15.212 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.675089+0000", 2024-09-10T19:01:15.212 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00020513899999999999 2024-09-10T19:01:15.212 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.212 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.212 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.212 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.683798+0000", 2024-09-10T19:01:15.212 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0087089860000000002 2024-09-10T19:01:15.212 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.212 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.212 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:01:15.212 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.683806+0000", 2024-09-10T19:01:15.212 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.8499999999999994e-06 2024-09-10T19:01:15.212 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.213 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.213 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.213 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.683850+0000", 2024-09-10T19:01:15.213 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.4452000000000001e-05 2024-09-10T19:01:15.213 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.213 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.213 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.213 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.683943+0000", 2024-09-10T19:01:15.213 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.3156000000000001e-05 2024-09-10T19:01:15.213 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.213 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.213 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:01:15.214 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.683997+0000", 2024-09-10T19:01:15.214 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.3346000000000001e-05 2024-09-10T19:01:15.214 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.214 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.214 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:01:15.214 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.684101+0000", 2024-09-10T19:01:15.214 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000104782 2024-09-10T19:01:15.214 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.214 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:01:15.214 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.214 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.216 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.216 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.24947.0:116059 8.a 8:54dcc1e5:::benchmark_data_smithi018_106531_object116058:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e183)", 2024-09-10T19:01:15.216 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:52:53.748462+0000", 2024-09-10T19:01:15.216 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 501.39017105699998, 2024-09-10T19:01:15.216 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.9359218249999999, 2024-09-10T19:01:15.216 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:01:15.216 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:01:15.216 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:01:15.216 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.24947", 2024-09-10T19:01:15.217 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T19:01:15.217 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 116059 2024-09-10T19:01:15.217 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.217 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:01:15.217 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.217 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:01:15.217 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.748462+0000", 2024-09-10T19:01:15.217 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.217 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.217 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.217 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:01:15.217 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.748462+0000", 2024-09-10T19:01:15.217 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.217 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.218 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.218 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:01:15.218 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.748461+0000", 2024-09-10T19:01:15.218 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T19:01:15.218 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.218 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.218 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:01:15.218 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.748500+0000", 2024-09-10T19:01:15.218 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.9198e-05 2024-09-10T19:01:15.218 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.218 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.218 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:01:15.218 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.748511+0000", 2024-09-10T19:01:15.218 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0319000000000001e-05 2024-09-10T19:01:15.219 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.219 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.219 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:01:15.219 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.748515+0000", 2024-09-10T19:01:15.219 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.5730000000000001e-06 2024-09-10T19:01:15.219 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.219 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.219 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.219 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.748569+0000", 2024-09-10T19:01:15.219 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.3671000000000002e-05 2024-09-10T19:01:15.219 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.219 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.219 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:01:15.220 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:53.748576+0000", 2024-09-10T19:01:15.220 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.4290000000000004e-06 2024-09-10T19:01:15.220 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.220 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.220 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.220 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:54.617439+0000", 2024-09-10T19:01:15.220 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.86886286000000001 2024-09-10T19:01:15.220 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.220 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.220 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:01:15.220 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:54.617447+0000", 2024-09-10T19:01:15.220 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.0919999999999998e-06 2024-09-10T19:01:15.221 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.221 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.221 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.221 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.677265+0000", 2024-09-10T19:01:15.221 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.059817394 2024-09-10T19:01:15.221 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.221 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.221 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:01:15.221 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.677372+0000", 2024-09-10T19:01:15.221 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000107496 2024-09-10T19:01:15.221 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.221 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.221 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,4,5", 2024-09-10T19:01:15.222 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.677519+0000", 2024-09-10T19:01:15.222 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00014644699999999999 2024-09-10T19:01:15.222 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.222 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.222 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:01:15.222 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.683811+0000", 2024-09-10T19:01:15.222 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0062920090000000003 2024-09-10T19:01:15.222 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.222 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.222 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.222 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.683883+0000", 2024-09-10T19:01:15.222 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.2263999999999998e-05 2024-09-10T19:01:15.223 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.223 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.223 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.223 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.684152+0000", 2024-09-10T19:01:15.223 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00026858600000000002 2024-09-10T19:01:15.223 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.223 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.223 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.223 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.684213+0000", 2024-09-10T19:01:15.223 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.1704000000000004e-05 2024-09-10T19:01:15.223 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.223 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.223 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:01:15.223 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.684291+0000", 2024-09-10T19:01:15.224 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.8088000000000002e-05 2024-09-10T19:01:15.224 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.224 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.224 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:01:15.224 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:52:55.684384+0000", 2024-09-10T19:01:15.224 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.2606000000000001e-05 2024-09-10T19:01:15.224 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.224 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:01:15.224 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.224 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.224 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.224 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15354.0:121944 9.a 9:5da60af9:::benchmark_data_smithi018_133594_object121943:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e270)", 2024-09-10T19:01:15.224 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:57:01.468640+0000", 2024-09-10T19:01:15.224 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 253.669993428, 2024-09-10T19:01:15.225 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.0828585020000001, 2024-09-10T19:01:15.225 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:01:15.225 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:01:15.225 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:01:15.225 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15354", 2024-09-10T19:01:15.225 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:01:15.225 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 121944 2024-09-10T19:01:15.225 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.225 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:01:15.225 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.225 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:01:15.225 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:01.468640+0000", 2024-09-10T19:01:15.225 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.225 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.225 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.226 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:01:15.226 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:01.468640+0000", 2024-09-10T19:01:15.226 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.226 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.226 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.226 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:01:15.226 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:01.468638+0000", 2024-09-10T19:01:15.226 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999981 2024-09-10T19:01:15.226 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.226 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.226 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:01:15.226 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:01.468668+0000", 2024-09-10T19:01:15.226 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.9788e-05 2024-09-10T19:01:15.226 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.227 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.227 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:01:15.227 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:01.468682+0000", 2024-09-10T19:01:15.227 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.3730000000000001e-05 2024-09-10T19:01:15.227 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.227 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.227 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:01:15.227 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:01.468689+0000", 2024-09-10T19:01:15.227 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.8420000000000005e-06 2024-09-10T19:01:15.227 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.227 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.227 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.227 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:01.468980+0000", 2024-09-10T19:01:15.227 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00029055699999999999 2024-09-10T19:01:15.228 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.228 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.228 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:01:15.228 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:01.468990+0000", 2024-09-10T19:01:15.228 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0339999999999999e-05 2024-09-10T19:01:15.228 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.228 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.228 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.228 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:02.422407+0000", 2024-09-10T19:01:15.228 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.95341685300000001 2024-09-10T19:01:15.228 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.228 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.228 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:01:15.228 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:02.422415+0000", 2024-09-10T19:01:15.229 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.3669999999999999e-06 2024-09-10T19:01:15.229 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.229 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.229 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.229 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:03.525114+0000", 2024-09-10T19:01:15.229 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1026991399999999 2024-09-10T19:01:15.229 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.229 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.229 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:01:15.229 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:03.525255+0000", 2024-09-10T19:01:15.229 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00014135800000000001 2024-09-10T19:01:15.229 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.229 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.229 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,6,7", 2024-09-10T19:01:15.229 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:03.525693+0000", 2024-09-10T19:01:15.230 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00043795699999999999 2024-09-10T19:01:15.230 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.230 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.230 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.230 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:03.537237+0000", 2024-09-10T19:01:15.230 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0115438 2024-09-10T19:01:15.230 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.230 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.230 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.230 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:03.537465+0000", 2024-09-10T19:01:15.230 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00022770900000000001 2024-09-10T19:01:15.230 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.230 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.230 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.231 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:03.537519+0000", 2024-09-10T19:01:15.231 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.4184000000000002e-05 2024-09-10T19:01:15.231 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.231 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.231 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:01:15.231 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:03.551333+0000", 2024-09-10T19:01:15.231 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.013814180000000001 2024-09-10T19:01:15.231 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.231 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.231 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:01:15.231 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:03.551389+0000", 2024-09-10T19:01:15.231 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.6261999999999998e-05 2024-09-10T19:01:15.231 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.231 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.232 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:01:15.232 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:03.551498+0000", 2024-09-10T19:01:15.232 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00010918599999999999 2024-09-10T19:01:15.232 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.232 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:01:15.232 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.232 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.232 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.232 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:30066 10.c 10:3545e195:::benchmark_data_smithi018_157943_object30065:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e329)", 2024-09-10T19:01:15.232 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:07.349273+0000", 2024-09-10T19:01:15.232 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 67.789360246000001, 2024-09-10T19:01:15.232 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1219290630000001, 2024-09-10T19:01:15.232 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:01:15.233 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:01:15.233 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:01:15.233 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:01:15.233 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:01:15.233 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 30066 2024-09-10T19:01:15.233 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.233 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:01:15.233 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.233 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:01:15.233 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.349273+0000", 2024-09-10T19:01:15.233 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.233 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.233 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.234 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:01:15.234 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.349273+0000", 2024-09-10T19:01:15.234 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.234 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.234 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.234 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:01:15.234 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.349271+0000", 2024-09-10T19:01:15.234 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999981 2024-09-10T19:01:15.234 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.234 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.234 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:01:15.234 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.349303+0000", 2024-09-10T19:01:15.235 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.2063000000000001e-05 2024-09-10T19:01:15.235 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.235 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.235 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:01:15.235 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.349316+0000", 2024-09-10T19:01:15.235 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.2388000000000001e-05 2024-09-10T19:01:15.235 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.235 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.235 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:01:15.235 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.349322+0000", 2024-09-10T19:01:15.235 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.037e-06 2024-09-10T19:01:15.235 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.235 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.236 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.236 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.355006+0000", 2024-09-10T19:01:15.236 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0056842400000000001 2024-09-10T19:01:15.236 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.236 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.236 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:01:15.236 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.355015+0000", 2024-09-10T19:01:15.236 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.7569999999999993e-06 2024-09-10T19:01:15.236 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.236 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.236 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.236 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.467715+0000", 2024-09-10T19:01:15.236 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.112699814 2024-09-10T19:01:15.237 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.237 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.237 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:01:15.237 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.467852+0000", 2024-09-10T19:01:15.237 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000137835 2024-09-10T19:01:15.237 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.237 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.237 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 0,3", 2024-09-10T19:01:15.237 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.468052+0000", 2024-09-10T19:01:15.237 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00019955999999999999 2024-09-10T19:01:15.237 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.237 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.237 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:01:15.237 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.471004+0000", 2024-09-10T19:01:15.237 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0029518389999999999 2024-09-10T19:01:15.238 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.238 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.238 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.238 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.471065+0000", 2024-09-10T19:01:15.238 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.1542000000000001e-05 2024-09-10T19:01:15.238 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.238 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.238 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.238 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.471123+0000", 2024-09-10T19:01:15.238 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.7457999999999998e-05 2024-09-10T19:01:15.238 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.238 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.238 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:01:15.238 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.471173+0000", 2024-09-10T19:01:15.238 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.0084999999999999e-05 2024-09-10T19:01:15.239 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.239 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.239 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:01:15.239 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.471202+0000", 2024-09-10T19:01:15.239 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.9176999999999999e-05 2024-09-10T19:01:15.239 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.239 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:01:15.239 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.239 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.239 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.239 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:30069 10.c 10:30e91e09:::benchmark_data_smithi018_157943_object30068:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e329)", 2024-09-10T19:01:15.239 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:07.355152+0000", 2024-09-10T19:01:15.239 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 67.783480951000001, 2024-09-10T19:01:15.240 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1176073280000001, 2024-09-10T19:01:15.240 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:01:15.240 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:01:15.240 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:01:15.240 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:01:15.240 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:01:15.240 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 30069 2024-09-10T19:01:15.240 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.240 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:01:15.240 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.240 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:01:15.240 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.355152+0000", 2024-09-10T19:01:15.240 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.240 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.240 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.241 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:01:15.241 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.355152+0000", 2024-09-10T19:01:15.241 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.241 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.241 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.241 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:01:15.241 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.355151+0000", 2024-09-10T19:01:15.241 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999981 2024-09-10T19:01:15.241 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.241 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.241 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:01:15.241 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.355183+0000", 2024-09-10T19:01:15.241 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.2786000000000001e-05 2024-09-10T19:01:15.241 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.242 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.242 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:01:15.242 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.355196+0000", 2024-09-10T19:01:15.242 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.2388000000000001e-05 2024-09-10T19:01:15.242 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.242 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.242 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:01:15.242 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.355202+0000", 2024-09-10T19:01:15.242 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.4690000000000002e-06 2024-09-10T19:01:15.242 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.242 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.242 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.242 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.355380+0000", 2024-09-10T19:01:15.242 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00017756900000000001 2024-09-10T19:01:15.242 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.243 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.243 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:01:15.243 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.355388+0000", 2024-09-10T19:01:15.243 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.6799999999999999e-06 2024-09-10T19:01:15.243 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.243 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.243 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.243 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.469672+0000", 2024-09-10T19:01:15.243 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.114283312 2024-09-10T19:01:15.243 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.243 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.243 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:01:15.243 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.469808+0000", 2024-09-10T19:01:15.243 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00013658999999999999 2024-09-10T19:01:15.244 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.244 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.244 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 0,3", 2024-09-10T19:01:15.244 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.469964+0000", 2024-09-10T19:01:15.244 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000155835 2024-09-10T19:01:15.244 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.244 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.244 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.244 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.472575+0000", 2024-09-10T19:01:15.244 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0026112900000000001 2024-09-10T19:01:15.244 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.244 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.244 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:01:15.245 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.472591+0000", 2024-09-10T19:01:15.245 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.5607000000000002e-05 2024-09-10T19:01:15.245 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.245 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.245 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.245 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.472671+0000", 2024-09-10T19:01:15.245 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.0150999999999999e-05 2024-09-10T19:01:15.245 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.245 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.245 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:01:15.245 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.472719+0000", 2024-09-10T19:01:15.245 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.7922e-05 2024-09-10T19:01:15.246 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.246 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.246 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:01:15.246 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.472760+0000", 2024-09-10T19:01:15.246 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.0445000000000003e-05 2024-09-10T19:01:15.246 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.246 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:01:15.246 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.246 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.246 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.247 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:30086 10.c 10:3336df4e:::benchmark_data_smithi018_157943_object30085:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e329)", 2024-09-10T19:01:15.247 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:07.357870+0000", 2024-09-10T19:01:15.247 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 67.780763113000006, 2024-09-10T19:01:15.248 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1176519279999999, 2024-09-10T19:01:15.248 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:01:15.248 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:01:15.248 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:01:15.248 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:01:15.248 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:01:15.248 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 30086 2024-09-10T19:01:15.248 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.248 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:01:15.248 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.248 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:01:15.248 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.357870+0000", 2024-09-10T19:01:15.249 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.249 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.249 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.249 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:01:15.249 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.357870+0000", 2024-09-10T19:01:15.249 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.249 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.249 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.249 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:01:15.249 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.357869+0000", 2024-09-10T19:01:15.249 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T19:01:15.249 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.250 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.250 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:01:15.250 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.357892+0000", 2024-09-10T19:01:15.250 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.3281999999999998e-05 2024-09-10T19:01:15.250 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.250 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.250 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:01:15.250 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.357903+0000", 2024-09-10T19:01:15.250 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0423e-05 2024-09-10T19:01:15.250 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.250 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.250 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:01:15.250 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.357909+0000", 2024-09-10T19:01:15.250 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.0560000000000001e-06 2024-09-10T19:01:15.251 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.251 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.251 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.251 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.358223+0000", 2024-09-10T19:01:15.251 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00031434699999999998 2024-09-10T19:01:15.251 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.251 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.251 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:01:15.251 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.358232+0000", 2024-09-10T19:01:15.251 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.1370000000000008e-06 2024-09-10T19:01:15.251 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.251 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.251 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.251 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.471296+0000", 2024-09-10T19:01:15.252 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1130639929999999 2024-09-10T19:01:15.252 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.252 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.252 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:01:15.252 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.471427+0000", 2024-09-10T19:01:15.252 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000130412 2024-09-10T19:01:15.252 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.252 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.252 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 0,3", 2024-09-10T19:01:15.252 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.471573+0000", 2024-09-10T19:01:15.252 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00014568099999999999 2024-09-10T19:01:15.252 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.253 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.253 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.253 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.474058+0000", 2024-09-10T19:01:15.253 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0024856930000000002 2024-09-10T19:01:15.253 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.253 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.253 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.253 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.474129+0000", 2024-09-10T19:01:15.253 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.0660000000000004e-05 2024-09-10T19:01:15.253 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.253 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.253 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:01:15.253 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.475424+0000", 2024-09-10T19:01:15.254 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0012947290000000001 2024-09-10T19:01:15.254 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.254 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.254 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:01:15.254 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.475482+0000", 2024-09-10T19:01:15.254 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.8479e-05 2024-09-10T19:01:15.254 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.254 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.254 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:01:15.254 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.475522+0000", 2024-09-10T19:01:15.254 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.9991999999999999e-05 2024-09-10T19:01:15.254 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.254 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:01:15.255 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.255 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.255 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.255 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:30091 10.c 10:321e1c33:::benchmark_data_smithi018_157943_object30090:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e329)", 2024-09-10T19:01:15.255 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:07.363674+0000", 2024-09-10T19:01:15.255 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 67.774959448999994, 2024-09-10T19:01:15.255 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.113559628, 2024-09-10T19:01:15.255 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:01:15.255 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:01:15.255 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:01:15.255 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:01:15.255 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:01:15.255 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 30091 2024-09-10T19:01:15.255 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.256 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:01:15.256 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.256 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:01:15.256 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.363674+0000", 2024-09-10T19:01:15.256 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.256 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.256 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.256 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:01:15.256 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.363674+0000", 2024-09-10T19:01:15.256 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.256 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.256 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.256 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:01:15.257 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.363672+0000", 2024-09-10T19:01:15.257 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T19:01:15.257 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.257 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.257 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:01:15.257 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.363703+0000", 2024-09-10T19:01:15.257 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.0219e-05 2024-09-10T19:01:15.257 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.257 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.257 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:01:15.257 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.363718+0000", 2024-09-10T19:01:15.257 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.5041999999999999e-05 2024-09-10T19:01:15.257 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.258 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.258 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:01:15.258 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.363723+0000", 2024-09-10T19:01:15.258 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.4199999999999998e-06 2024-09-10T19:01:15.258 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.258 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.258 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.258 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.364936+0000", 2024-09-10T19:01:15.258 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0012127939999999999 2024-09-10T19:01:15.258 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.258 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.258 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:01:15.259 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.364963+0000", 2024-09-10T19:01:15.259 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.7033e-05 2024-09-10T19:01:15.259 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.259 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.259 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.259 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.472794+0000", 2024-09-10T19:01:15.259 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1078312450000001 2024-09-10T19:01:15.259 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.259 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.259 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:01:15.259 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.472926+0000", 2024-09-10T19:01:15.260 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000131326 2024-09-10T19:01:15.260 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.260 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.260 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 0,3", 2024-09-10T19:01:15.260 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.473065+0000", 2024-09-10T19:01:15.260 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00013973300000000001 2024-09-10T19:01:15.260 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.260 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.260 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.260 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.477097+0000", 2024-09-10T19:01:15.260 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0040313700000000003 2024-09-10T19:01:15.260 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.260 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.261 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:01:15.261 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.477104+0000", 2024-09-10T19:01:15.261 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.0690000000000004e-06 2024-09-10T19:01:15.261 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.261 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.261 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.261 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.477147+0000", 2024-09-10T19:01:15.261 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.2840000000000003e-05 2024-09-10T19:01:15.261 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.261 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.261 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:01:15.261 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.477194+0000", 2024-09-10T19:01:15.261 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.7070999999999997e-05 2024-09-10T19:01:15.262 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.262 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.262 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:01:15.262 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.477233+0000", 2024-09-10T19:01:15.262 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.9762000000000003e-05 2024-09-10T19:01:15.262 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.262 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:01:15.262 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.262 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.262 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.262 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:30094 10.c 10:34d9c766:::benchmark_data_smithi018_157943_object30093:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e329)", 2024-09-10T19:01:15.262 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:07.363968+0000", 2024-09-10T19:01:15.262 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 67.774665479000006, 2024-09-10T19:01:15.262 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1134059510000001, 2024-09-10T19:01:15.262 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:01:15.263 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:01:15.263 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:01:15.263 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:01:15.263 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:01:15.263 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 30094 2024-09-10T19:01:15.263 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.263 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:01:15.263 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.263 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:01:15.263 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.363968+0000", 2024-09-10T19:01:15.263 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.263 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.263 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.264 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:01:15.264 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.363968+0000", 2024-09-10T19:01:15.264 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.264 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.264 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.264 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:01:15.264 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.363967+0000", 2024-09-10T19:01:15.264 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T19:01:15.264 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.264 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.264 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:01:15.265 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.364007+0000", 2024-09-10T19:01:15.265 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.9898000000000003e-05 2024-09-10T19:01:15.265 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.265 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.265 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:01:15.265 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.364021+0000", 2024-09-10T19:01:15.265 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.3842000000000001e-05 2024-09-10T19:01:15.265 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.265 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.265 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:01:15.265 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.364027+0000", 2024-09-10T19:01:15.265 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.0449999999999997e-06 2024-09-10T19:01:15.265 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.266 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.266 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.266 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.365009+0000", 2024-09-10T19:01:15.266 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00098269799999999995 2024-09-10T19:01:15.266 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.266 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.266 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:01:15.266 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.365017+0000", 2024-09-10T19:01:15.266 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.8809999999999996e-06 2024-09-10T19:01:15.266 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.266 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.266 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.266 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.474180+0000", 2024-09-10T19:01:15.267 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.109162371 2024-09-10T19:01:15.267 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.267 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.267 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:01:15.267 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.474309+0000", 2024-09-10T19:01:15.267 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000129286 2024-09-10T19:01:15.267 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.267 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.267 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 0,3", 2024-09-10T19:01:15.267 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.474489+0000", 2024-09-10T19:01:15.267 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00018039999999999999 2024-09-10T19:01:15.267 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.267 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.268 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:01:15.268 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.477245+0000", 2024-09-10T19:01:15.268 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.002755773 2024-09-10T19:01:15.268 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.268 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.268 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.268 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.477275+0000", 2024-09-10T19:01:15.268 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.9607e-05 2024-09-10T19:01:15.268 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.268 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.268 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.268 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.477303+0000", 2024-09-10T19:01:15.269 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.8107999999999999e-05 2024-09-10T19:01:15.269 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.269 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.269 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:01:15.269 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.477342+0000", 2024-09-10T19:01:15.269 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.9595000000000003e-05 2024-09-10T19:01:15.269 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.269 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.269 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:01:15.269 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.477374+0000", 2024-09-10T19:01:15.269 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.1374999999999997e-05 2024-09-10T19:01:15.269 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.270 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:01:15.270 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.270 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.270 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.270 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:30141 10.c 10:3ad97324:::benchmark_data_smithi018_157943_object30140:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e329)", 2024-09-10T19:01:15.270 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:07.385387+0000", 2024-09-10T19:01:15.270 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 67.753245747999998, 2024-09-10T19:01:15.270 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.093648599, 2024-09-10T19:01:15.270 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:01:15.270 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:01:15.270 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:01:15.270 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:01:15.270 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:01:15.270 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 30141 2024-09-10T19:01:15.271 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.271 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:01:15.271 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.271 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:01:15.271 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.385387+0000", 2024-09-10T19:01:15.271 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.271 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.271 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.271 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:01:15.271 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.385387+0000", 2024-09-10T19:01:15.271 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.271 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.271 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.272 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:01:15.272 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.385386+0000", 2024-09-10T19:01:15.272 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999981 2024-09-10T19:01:15.272 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.272 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.272 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:01:15.272 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.385411+0000", 2024-09-10T19:01:15.272 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.5398999999999999e-05 2024-09-10T19:01:15.272 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.272 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.272 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:01:15.272 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.385428+0000", 2024-09-10T19:01:15.272 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.7357e-05 2024-09-10T19:01:15.273 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.273 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.273 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:01:15.273 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.385435+0000", 2024-09-10T19:01:15.273 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.0190000000000001e-06 2024-09-10T19:01:15.273 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.273 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.273 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.273 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.385498+0000", 2024-09-10T19:01:15.273 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.2858e-05 2024-09-10T19:01:15.273 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.273 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.273 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:01:15.274 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.385508+0000", 2024-09-10T19:01:15.274 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.6129999999999993e-06 2024-09-10T19:01:15.274 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.274 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.274 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.274 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.475579+0000", 2024-09-10T19:01:15.274 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.090070761 2024-09-10T19:01:15.274 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.274 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.274 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:01:15.274 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.475723+0000", 2024-09-10T19:01:15.274 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00014458099999999999 2024-09-10T19:01:15.274 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.275 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.275 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 0,3", 2024-09-10T19:01:15.275 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.475849+0000", 2024-09-10T19:01:15.275 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00012578399999999999 2024-09-10T19:01:15.275 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.275 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.275 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:01:15.275 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.478826+0000", 2024-09-10T19:01:15.275 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.002977436 2024-09-10T19:01:15.275 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.275 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.275 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.275 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.478883+0000", 2024-09-10T19:01:15.276 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.6851000000000002e-05 2024-09-10T19:01:15.276 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.276 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.276 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.276 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.478928+0000", 2024-09-10T19:01:15.276 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.5136999999999998e-05 2024-09-10T19:01:15.276 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.276 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.276 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:01:15.276 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.478998+0000", 2024-09-10T19:01:15.276 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.9381999999999994e-05 2024-09-10T19:01:15.276 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.276 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.277 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:01:15.277 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.479036+0000", 2024-09-10T19:01:15.277 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.8368999999999998e-05 2024-09-10T19:01:15.277 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.277 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:01:15.277 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.277 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.277 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.277 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:30157 10.c 10:36dc0055:::benchmark_data_smithi018_157943_object30156:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e329)", 2024-09-10T19:01:15.277 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:07.392968+0000", 2024-09-10T19:01:15.277 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 67.745665611000007, 2024-09-10T19:01:15.277 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0948556270000001, 2024-09-10T19:01:15.277 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:01:15.278 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:01:15.278 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:01:15.278 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:01:15.278 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:01:15.278 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 30157 2024-09-10T19:01:15.278 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.278 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:01:15.278 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.278 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:01:15.278 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.392968+0000", 2024-09-10T19:01:15.278 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.278 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.278 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.279 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:01:15.279 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.392968+0000", 2024-09-10T19:01:15.279 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.279 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.279 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.279 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:01:15.279 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.392967+0000", 2024-09-10T19:01:15.279 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T19:01:15.279 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.279 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.279 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:01:15.279 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.393004+0000", 2024-09-10T19:01:15.279 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.7685999999999998e-05 2024-09-10T19:01:15.280 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.280 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.280 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:01:15.280 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.393018+0000", 2024-09-10T19:01:15.280 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.4029000000000001e-05 2024-09-10T19:01:15.280 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.280 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.280 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:01:15.280 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.393024+0000", 2024-09-10T19:01:15.280 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.6949999999999997e-06 2024-09-10T19:01:15.280 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.280 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.280 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.281 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.393087+0000", 2024-09-10T19:01:15.281 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.2749000000000005e-05 2024-09-10T19:01:15.281 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.281 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.281 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:01:15.281 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.393096+0000", 2024-09-10T19:01:15.281 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.6379999999999999e-06 2024-09-10T19:01:15.281 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.281 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.281 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.281 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.480350+0000", 2024-09-10T19:01:15.281 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0872540820000001 2024-09-10T19:01:15.281 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.282 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.282 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:01:15.282 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.480467+0000", 2024-09-10T19:01:15.282 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000116823 2024-09-10T19:01:15.282 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.282 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.282 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 0,3", 2024-09-10T19:01:15.282 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.480624+0000", 2024-09-10T19:01:15.282 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00015658499999999999 2024-09-10T19:01:15.282 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.282 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.282 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:01:15.282 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.482009+0000", 2024-09-10T19:01:15.283 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0013854640000000001 2024-09-10T19:01:15.283 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.283 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.283 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.283 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.487698+0000", 2024-09-10T19:01:15.283 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0056892269999999998 2024-09-10T19:01:15.283 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.283 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.283 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.283 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.487736+0000", 2024-09-10T19:01:15.283 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.7500999999999999e-05 2024-09-10T19:01:15.283 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.284 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.284 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:01:15.284 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.487788+0000", 2024-09-10T19:01:15.284 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.1637e-05 2024-09-10T19:01:15.284 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.284 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.284 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:01:15.284 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.487823+0000", 2024-09-10T19:01:15.284 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.5621000000000003e-05 2024-09-10T19:01:15.284 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.284 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:01:15.284 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.285 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.285 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.285 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:30176 10.c 10:3c44fac3:::benchmark_data_smithi018_157943_object30175:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e329)", 2024-09-10T19:01:15.285 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:07.401052+0000", 2024-09-10T19:01:15.285 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 67.737581055000007, 2024-09-10T19:01:15.285 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0946845709999999, 2024-09-10T19:01:15.285 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:01:15.285 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:01:15.285 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:01:15.285 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:01:15.285 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:01:15.285 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 30176 2024-09-10T19:01:15.285 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.285 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:01:15.286 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.286 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:01:15.286 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.401052+0000", 2024-09-10T19:01:15.286 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.286 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.286 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.286 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:01:15.286 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.401052+0000", 2024-09-10T19:01:15.286 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.286 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.286 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.286 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:01:15.286 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.401050+0000", 2024-09-10T19:01:15.287 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999981 2024-09-10T19:01:15.287 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.287 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.287 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:01:15.287 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.401083+0000", 2024-09-10T19:01:15.287 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.3191000000000001e-05 2024-09-10T19:01:15.287 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.287 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.287 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:01:15.287 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.401098+0000", 2024-09-10T19:01:15.287 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.5031000000000001e-05 2024-09-10T19:01:15.287 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.287 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.287 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:01:15.288 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.401106+0000", 2024-09-10T19:01:15.288 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.5399999999999998e-06 2024-09-10T19:01:15.288 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.288 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.288 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.288 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.401170+0000", 2024-09-10T19:01:15.288 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.3587e-05 2024-09-10T19:01:15.288 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.288 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.288 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:01:15.288 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.401181+0000", 2024-09-10T19:01:15.288 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1703e-05 2024-09-10T19:01:15.288 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.289 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.289 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.289 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.486342+0000", 2024-09-10T19:01:15.289 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0851604100000001 2024-09-10T19:01:15.289 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.289 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.289 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:01:15.289 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.486454+0000", 2024-09-10T19:01:15.289 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00011186 2024-09-10T19:01:15.289 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.289 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.289 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 0,3", 2024-09-10T19:01:15.289 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.486588+0000", 2024-09-10T19:01:15.290 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00013417600000000001 2024-09-10T19:01:15.290 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.290 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.290 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:01:15.290 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.492612+0000", 2024-09-10T19:01:15.290 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0060245369999999999 2024-09-10T19:01:15.290 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.290 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.290 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.290 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.492655+0000", 2024-09-10T19:01:15.290 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.2815000000000003e-05 2024-09-10T19:01:15.290 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.290 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.290 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.291 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.495655+0000", 2024-09-10T19:01:15.291 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0030003600000000001 2024-09-10T19:01:15.291 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.291 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.291 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:01:15.292 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.495699+0000", 2024-09-10T19:01:15.292 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.3615000000000002e-05 2024-09-10T19:01:15.292 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.293 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.293 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:01:15.293 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.495737+0000", 2024-09-10T19:01:15.293 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.7666000000000002e-05 2024-09-10T19:01:15.293 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.293 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:01:15.293 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.293 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.293 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.293 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:30184 10.c 10:3a927544:::benchmark_data_smithi018_157943_object30183:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e329)", 2024-09-10T19:01:15.293 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:07.405082+0000", 2024-09-10T19:01:15.293 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 67.733551460000001, 2024-09-10T19:01:15.294 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.093739512, 2024-09-10T19:01:15.294 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:01:15.294 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:01:15.294 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:01:15.294 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:01:15.294 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:01:15.294 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 30184 2024-09-10T19:01:15.294 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.294 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:01:15.294 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.294 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:01:15.294 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.405082+0000", 2024-09-10T19:01:15.294 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.294 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.295 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.295 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:01:15.295 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.405082+0000", 2024-09-10T19:01:15.295 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.295 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.295 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.295 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:01:15.295 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.405074+0000", 2024-09-10T19:01:15.295 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999924 2024-09-10T19:01:15.295 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.295 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.295 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:01:15.295 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.405106+0000", 2024-09-10T19:01:15.296 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.2462000000000002e-05 2024-09-10T19:01:15.296 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.296 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.296 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:01:15.296 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.405122+0000", 2024-09-10T19:01:15.296 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.5829e-05 2024-09-10T19:01:15.296 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.296 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.296 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:01:15.296 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.405130+0000", 2024-09-10T19:01:15.296 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.6550000000000004e-06 2024-09-10T19:01:15.296 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.296 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.297 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.297 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.405171+0000", 2024-09-10T19:01:15.297 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.1498999999999997e-05 2024-09-10T19:01:15.297 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.297 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.297 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:01:15.297 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.405178+0000", 2024-09-10T19:01:15.297 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.2840000000000003e-06 2024-09-10T19:01:15.297 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.297 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.297 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.297 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.487997+0000", 2024-09-10T19:01:15.297 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0828189509999999 2024-09-10T19:01:15.298 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.298 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.298 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:01:15.298 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.488121+0000", 2024-09-10T19:01:15.298 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000124354 2024-09-10T19:01:15.298 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.298 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.298 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 0,3", 2024-09-10T19:01:15.298 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.488427+0000", 2024-09-10T19:01:15.298 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00030601599999999999 2024-09-10T19:01:15.298 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.298 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.298 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:01:15.299 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.492619+0000", 2024-09-10T19:01:15.299 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0041922590000000003 2024-09-10T19:01:15.299 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.299 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.299 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.299 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.492709+0000", 2024-09-10T19:01:15.299 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.9599999999999996e-05 2024-09-10T19:01:15.299 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.299 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.299 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.299 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.498721+0000", 2024-09-10T19:01:15.299 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0060124610000000002 2024-09-10T19:01:15.299 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.300 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.300 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:01:15.300 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.498775+0000", 2024-09-10T19:01:15.300 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.3381000000000003e-05 2024-09-10T19:01:15.300 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.300 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.300 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:01:15.300 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.498821+0000", 2024-09-10T19:01:15.300 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.6520999999999997e-05 2024-09-10T19:01:15.300 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.300 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:01:15.300 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.300 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.301 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.301 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:30187 10.c 10:3f41b8ee:::benchmark_data_smithi018_157943_object30186:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e329)", 2024-09-10T19:01:15.301 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:07.406336+0000", 2024-09-10T19:01:15.301 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 67.732297353000007, 2024-09-10T19:01:15.301 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.092607122, 2024-09-10T19:01:15.301 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:01:15.301 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:01:15.301 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:01:15.301 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:01:15.301 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:01:15.301 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 30187 2024-09-10T19:01:15.301 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.301 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:01:15.301 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.302 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:01:15.302 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.406336+0000", 2024-09-10T19:01:15.302 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.302 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.302 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.302 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:01:15.302 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.406336+0000", 2024-09-10T19:01:15.302 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:01:15.302 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.302 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.302 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:01:15.302 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.406335+0000", 2024-09-10T19:01:15.302 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T19:01:15.302 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.303 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.303 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:01:15.303 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.406357+0000", 2024-09-10T19:01:15.303 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.2090999999999999e-05 2024-09-10T19:01:15.303 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.303 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.303 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:01:15.303 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.406368+0000", 2024-09-10T19:01:15.303 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1077000000000001e-05 2024-09-10T19:01:15.303 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.303 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.303 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:01:15.303 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.406373+0000", 2024-09-10T19:01:15.304 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.2259999999999996e-06 2024-09-10T19:01:15.304 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.304 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.304 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.304 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.406409+0000", 2024-09-10T19:01:15.304 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.5908999999999998e-05 2024-09-10T19:01:15.304 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.304 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.304 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:01:15.304 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.406416+0000", 2024-09-10T19:01:15.304 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.8210000000000004e-06 2024-09-10T19:01:15.304 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.305 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.305 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:01:15.305 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.489738+0000", 2024-09-10T19:01:15.305 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0833220569999999 2024-09-10T19:01:15.305 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.305 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.305 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:01:15.305 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.489880+0000", 2024-09-10T19:01:15.305 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00014216899999999999 2024-09-10T19:01:15.305 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.305 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.305 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 0,3", 2024-09-10T19:01:15.305 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.490067+0000", 2024-09-10T19:01:15.306 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00018631999999999999 2024-09-10T19:01:15.306 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.306 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.306 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:01:15.306 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.492661+0000", 2024-09-10T19:01:15.306 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0025938049999999998 2024-09-10T19:01:15.306 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.306 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.306 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.306 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.495613+0000", 2024-09-10T19:01:15.306 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.002952391 2024-09-10T19:01:15.306 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.306 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.307 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:01:15.307 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.498866+0000", 2024-09-10T19:01:15.307 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0032528399999999999 2024-09-10T19:01:15.307 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.307 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.307 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:01:15.307 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.498911+0000", 2024-09-10T19:01:15.307 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.5664999999999997e-05 2024-09-10T19:01:15.307 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:01:15.307 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:01:15.307 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:01:15.307 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.498943+0000", 2024-09-10T19:01:15.307 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.1563000000000003e-05 2024-09-10T19:01:15.308 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.308 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:01:15.308 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.308 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:01:15.308 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:01:15.308 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T19:01:15.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:15.309 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:15.309 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:15.410 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:15.449 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:15.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:15.450 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:15.480 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is set 2024-09-10T19:01:15.551 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:15.585 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:15.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:15.586 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:15.613 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 82 16 128784 128768 98.1235 81.9375 0.00579891 0.0101871 2024-09-10T19:01:15.687 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:15.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:15 smithi018 ceph-mon[29336]: 2.4 scrub starts 2024-09-10T19:01:15.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:15 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/295964618' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T19:01:15.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:15 smithi018 ceph-mon[40196]: 2.4 scrub starts 2024-09-10T19:01:15.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:15 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/295964618' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T19:01:15.727 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:15.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:15.728 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:15.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:15 smithi103 ceph-mon[33163]: 2.4 scrub starts 2024-09-10T19:01:15.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:15 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/295964618' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T19:01:15.828 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:15.870 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:15.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:15.871 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:15.972 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:16.008 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:16.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:16.009 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:16.111 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:16.145 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:16.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:16.146 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:16.246 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:16.282 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:16.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:16.284 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:16.384 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:16.425 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:16.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:16.426 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:16.526 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:16.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:16.564 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:16.564 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:16.614 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 83 16 130267 130251 98.058 92.6875 0.0121839 0.010194 2024-09-10T19:01:16.665 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:16.702 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:16.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:16.703 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:16.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:16 smithi103 ceph-mon[33163]: pgmap v743: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 7.6 GiB data, 28 GiB used, 508 GiB / 536 GiB avail; 1023 B/s rd, 80 MiB/s wr, 1.29k op/s; 6983/371841 objects misplaced (1.878%); 33 MiB/s, 524 objects/s recovering 2024-09-10T19:01:16.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:16 smithi103 ceph-mon[33163]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:01:16.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:16 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/295964618' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T19:01:16.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:16 smithi103 ceph-mon[33163]: osdmap e351: 8 total, 8 up, 6 in 2024-09-10T19:01:16.804 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:16.838 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:16.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:16.840 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:16.940 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:16.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:16 smithi018 ceph-mon[40196]: pgmap v743: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 7.6 GiB data, 28 GiB used, 508 GiB / 536 GiB avail; 1023 B/s rd, 80 MiB/s wr, 1.29k op/s; 6983/371841 objects misplaced (1.878%); 33 MiB/s, 524 objects/s recovering 2024-09-10T19:01:16.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:16 smithi018 ceph-mon[40196]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:01:16.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:16 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/295964618' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T19:01:16.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:16 smithi018 ceph-mon[40196]: osdmap e351: 8 total, 8 up, 6 in 2024-09-10T19:01:16.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:16 smithi018 ceph-mon[29336]: pgmap v743: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 7.6 GiB data, 28 GiB used, 508 GiB / 536 GiB avail; 1023 B/s rd, 80 MiB/s wr, 1.29k op/s; 6983/371841 objects misplaced (1.878%); 33 MiB/s, 524 objects/s recovering 2024-09-10T19:01:16.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:16 smithi018 ceph-mon[29336]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:01:16.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:16 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/295964618' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T19:01:16.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:16 smithi018 ceph-mon[29336]: osdmap e351: 8 total, 8 up, 6 in 2024-09-10T19:01:16.980 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:16.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:16.981 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:17.082 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:17.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:17.118 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:17.118 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:17.219 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:17.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:17.260 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:17.260 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:17.272 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:01:16 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:01:17.361 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:17.400 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:17.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:17.401 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:17.502 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:17.536 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:17.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:17.537 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:17.614 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 84 16 131677 131661 97.9396 88.125 0.0059949 0.0102062 2024-09-10T19:01:17.638 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:17.672 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:17.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:17.673 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:17.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:17 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:01:17.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:17 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:01:17.774 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:17.813 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:17.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:17.815 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:17.823 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:17 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:01:17.823 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:17 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:01:17.824 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:17 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:01:17.824 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:17 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:01:17.915 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:17.952 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:17.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:17.953 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:18.054 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:18.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:18.089 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:18.089 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:18.190 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:18.231 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:18.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:18.232 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:18.333 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:18.368 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:18.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:18.369 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:18.470 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:18.507 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:18.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:18.510 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:18.611 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:18.614 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 85 16 133046 133030 97.794 85.5625 0.00920018 0.0102215 2024-09-10T19:01:18.649 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:18.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:18.650 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:18.751 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:18.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:18 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:18.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:18 smithi103 ceph-mon[33163]: pgmap v745: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 7.9 GiB data, 29 GiB used, 507 GiB / 536 GiB avail; 1.2 KiB/s rd, 93 MiB/s wr, 1.48k op/s; 5337/388590 objects misplaced (1.373%); 39 MiB/s, 623 objects/s recovering 2024-09-10T19:01:18.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:18 smithi103 ceph-mon[33163]: osdmap e352: 8 total, 8 up, 6 in 2024-09-10T19:01:18.787 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:18.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:18.789 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:18.890 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:18.905 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:18 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:18.905 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:18 smithi018 ceph-mon[29336]: pgmap v745: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 7.9 GiB data, 29 GiB used, 507 GiB / 536 GiB avail; 1.2 KiB/s rd, 93 MiB/s wr, 1.48k op/s; 5337/388590 objects misplaced (1.373%); 39 MiB/s, 623 objects/s recovering 2024-09-10T19:01:18.905 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:18 smithi018 ceph-mon[29336]: osdmap e352: 8 total, 8 up, 6 in 2024-09-10T19:01:18.905 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:18 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:18.905 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:18 smithi018 ceph-mon[40196]: pgmap v745: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 7.9 GiB data, 29 GiB used, 507 GiB / 536 GiB avail; 1.2 KiB/s rd, 93 MiB/s wr, 1.48k op/s; 5337/388590 objects misplaced (1.373%); 39 MiB/s, 623 objects/s recovering 2024-09-10T19:01:18.905 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:18 smithi018 ceph-mon[40196]: osdmap e352: 8 total, 8 up, 6 in 2024-09-10T19:01:18.927 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:18.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:18.928 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:19.029 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:19.064 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:19.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:19.065 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:19.166 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:19.204 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:19.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:19.205 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:19.306 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:19.345 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:19.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:19.346 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:19.447 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:19.521 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:19.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:19.521 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:19.614 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 86 16 134394 134378 97.6364 84.25 0.0114784 0.0102379 2024-09-10T19:01:19.622 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:19.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:19.660 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:19.660 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:19.761 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:19.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:19 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:01:19.803 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:19.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:19.805 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:19.858 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:19 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:01:19.858 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:19 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:01:19.906 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:19.943 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:19.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:19.944 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:20.045 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:20.084 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:20.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:20.085 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:20.186 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:20.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:20.220 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:20.221 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:20.322 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:20.355 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:20.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:20.356 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:20.457 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:20.494 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:01:20 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:01:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:01:20.496 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:20.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:20.497 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:20.598 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:20.614 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 87 16 135755 135739 97.4918 85.0625 0.012032 0.0102529 2024-09-10T19:01:20.635 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:20.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:20.636 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:20.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:20 smithi018 ceph-mon[40196]: pgmap v747: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 8.0 GiB data, 30 GiB used, 507 GiB / 536 GiB avail; 1.2 KiB/s rd, 93 MiB/s wr, 1.49k op/s; 4765/392487 objects misplaced (1.214%); 34 MiB/s, 550 objects/s recovering 2024-09-10T19:01:20.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:20 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:01:20.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:20 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:01:20.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:20 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:01:20.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:20 smithi018 ceph-mon[29336]: pgmap v747: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 8.0 GiB data, 30 GiB used, 507 GiB / 536 GiB avail; 1.2 KiB/s rd, 93 MiB/s wr, 1.49k op/s; 4765/392487 objects misplaced (1.214%); 34 MiB/s, 550 objects/s recovering 2024-09-10T19:01:20.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:20 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:01:20.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:20 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:01:20.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:20 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:01:20.737 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:20.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:20 smithi103 ceph-mon[33163]: pgmap v747: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 8.0 GiB data, 30 GiB used, 507 GiB / 536 GiB avail; 1.2 KiB/s rd, 93 MiB/s wr, 1.49k op/s; 4765/392487 objects misplaced (1.214%); 34 MiB/s, 550 objects/s recovering 2024-09-10T19:01:20.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:20 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:01:20.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:20 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:01:20.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:20 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:01:20.773 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:20.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:20.774 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:20.820 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-10T19:01:20.875 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:20.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:20.936 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:20.936 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:20.957 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-10T19:01:21.038 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:21.076 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:21.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:21.077 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:21.079 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:01:20 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:01:20] "GET /metrics HTTP/1.1" 200 224743 "" "Prometheus/2.43.0" 2024-09-10T19:01:21.178 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:21.217 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:21.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:21.218 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:21.319 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:21.355 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:21.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:21.356 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:21.457 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:21.493 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:21.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:21.494 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:21.595 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:21.618 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 88 16 137105 137089 97.3393 84.375 0.0116234 0.0102689 2024-09-10T19:01:21.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:21.673 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:21.674 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:21.774 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:21.807 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:21.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:21.808 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:21.909 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:21.952 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:21.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:21.954 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:22.054 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:22.085 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:22.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:22.086 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:22.187 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:22.221 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:22.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:22.222 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:22.246 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T19:01:22.248 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-09-10T19:01:22.323 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:22.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:22.362 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:22.362 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:22.463 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:22.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:22.501 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:22.501 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:22.601 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:22.617 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 89 16 138468 138452 97.2027 85.1875 0.00620383 0.0102807 2024-09-10T19:01:22.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:22.640 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:22.640 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:22.740 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:22.761 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:22.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:22.763 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:22.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:22 smithi103 ceph-mon[33163]: pgmap v748: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 8.1 GiB data, 30 GiB used, 506 GiB / 536 GiB avail; 639 B/s rd, 73 MiB/s wr, 1.17k op/s; 4052/396336 objects misplaced (1.022%); 31 MiB/s, 500 objects/s recovering 2024-09-10T19:01:22.863 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:22.886 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:22.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:22.887 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:22.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:22 smithi018 ceph-mon[29336]: pgmap v748: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 8.1 GiB data, 30 GiB used, 506 GiB / 536 GiB avail; 639 B/s rd, 73 MiB/s wr, 1.17k op/s; 4052/396336 objects misplaced (1.022%); 31 MiB/s, 500 objects/s recovering 2024-09-10T19:01:22.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:22 smithi018 ceph-mon[40196]: pgmap v748: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 8.1 GiB data, 30 GiB used, 506 GiB / 536 GiB avail; 639 B/s rd, 73 MiB/s wr, 1.17k op/s; 4052/396336 objects misplaced (1.022%); 31 MiB/s, 500 objects/s recovering 2024-09-10T19:01:22.988 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:23.011 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:23.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:23.012 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:23.113 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:23.136 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:23.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:23.137 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:23.237 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:23.261 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:23.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:23.262 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:23.362 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:23.385 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:23.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:23.386 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:23.487 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:23.510 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:23.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:23.511 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:23.612 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:23.617 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Total time run: 90.0114 2024-09-10T19:01:23.617 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Total writes made: 138655 2024-09-10T19:01:23.617 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Write size: 65536 2024-09-10T19:01:23.617 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Object size: 65536 2024-09-10T19:01:23.617 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Bandwidth (MB/sec): 96.276 2024-09-10T19:01:23.617 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Stddev Bandwidth: 35.902 2024-09-10T19:01:23.617 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Max bandwidth (MB/sec): 232.625 2024-09-10T19:01:23.618 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Min bandwidth (MB/sec): 0 2024-09-10T19:01:23.618 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Average IOPS: 1540 2024-09-10T19:01:23.618 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Stddev IOPS: 574.454 2024-09-10T19:01:23.618 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Max IOPS: 3722 2024-09-10T19:01:23.618 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Min IOPS: 0 2024-09-10T19:01:23.618 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Average Latency(s): 0.010383 2024-09-10T19:01:23.618 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Stddev Latency(s): 0.036919 2024-09-10T19:01:23.618 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Max latency(s): 2.35071 2024-09-10T19:01:23.618 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Min latency(s): 0.00113481 2024-09-10T19:01:23.621 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Cleaning up (deleting benchmark objects) 2024-09-10T19:01:23.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:23.641 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:23.641 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:23.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:23 smithi018 ceph-mon[40196]: from='client.25541 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:01:23.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:23 smithi018 ceph-mon[40196]: osdmap e353: 8 total, 8 up, 6 in 2024-09-10T19:01:23.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:23 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:01:23.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:23 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:01:23.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:23 smithi018 ceph-mon[29336]: from='client.25541 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:01:23.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:23 smithi018 ceph-mon[29336]: osdmap e353: 8 total, 8 up, 6 in 2024-09-10T19:01:23.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:23 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:01:23.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:23 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:01:23.742 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:23.771 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:23.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:23.772 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:23.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:23 smithi103 ceph-mon[33163]: from='client.25541 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:01:23.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:23 smithi103 ceph-mon[33163]: osdmap e353: 8 total, 8 up, 6 in 2024-09-10T19:01:23.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:23 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:01:23.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:23 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:01:23.873 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:23.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:23.907 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:23.908 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:24.008 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:24.043 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:24.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:24.044 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:24.144 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:24.181 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:24.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:24.182 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:24.282 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:24.315 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:24.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:24.316 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:24.416 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:24.438 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:24.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:24.439 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:24.540 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:24.567 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:24.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:24.568 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:24.669 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:24.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:24 smithi018 ceph-mon[29336]: pgmap v750: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 8.3 GiB data, 31 GiB used, 506 GiB / 536 GiB avail; 1.3 KiB/s rd, 99 MiB/s wr, 1.58k op/s; 3260/409305 objects misplaced (0.796%); 33 MiB/s, 524 objects/s recovering 2024-09-10T19:01:24.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:24 smithi018 ceph-mon[29336]: osdmap e354: 8 total, 8 up, 6 in 2024-09-10T19:01:24.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:24 smithi018 ceph-mon[40196]: pgmap v750: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 8.3 GiB data, 31 GiB used, 506 GiB / 536 GiB avail; 1.3 KiB/s rd, 99 MiB/s wr, 1.58k op/s; 3260/409305 objects misplaced (0.796%); 33 MiB/s, 524 objects/s recovering 2024-09-10T19:01:24.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:24 smithi018 ceph-mon[40196]: osdmap e354: 8 total, 8 up, 6 in 2024-09-10T19:01:24.700 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:24.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:24.701 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:24.802 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:24.824 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:24.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:24.825 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:24.925 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:24.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:24.947 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:24.948 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:25.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:24 smithi103 ceph-mon[33163]: pgmap v750: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 8.3 GiB data, 31 GiB used, 506 GiB / 536 GiB avail; 1.3 KiB/s rd, 99 MiB/s wr, 1.58k op/s; 3260/409305 objects misplaced (0.796%); 33 MiB/s, 524 objects/s recovering 2024-09-10T19:01:25.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:24 smithi103 ceph-mon[33163]: osdmap e354: 8 total, 8 up, 6 in 2024-09-10T19:01:25.048 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:25.071 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:25.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:25.072 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:25.172 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:25.194 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:25.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:25.195 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:25.296 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:25.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:25.319 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:25.319 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:25.419 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:25.443 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:25.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:25.444 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:25.544 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:25.576 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:25.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:25.577 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:25.678 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:25.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:25 smithi018 ceph-mon[29336]: osdmap e355: 8 total, 8 up, 6 in 2024-09-10T19:01:25.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:25 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:01:25.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:25 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:01:25.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:25 smithi018 ceph-mon[29336]: osdmap e356: 8 total, 8 up, 6 in 2024-09-10T19:01:25.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:25 smithi018 ceph-mon[40196]: osdmap e355: 8 total, 8 up, 6 in 2024-09-10T19:01:25.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:25 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:01:25.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:25 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:01:25.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:25 smithi018 ceph-mon[40196]: osdmap e356: 8 total, 8 up, 6 in 2024-09-10T19:01:25.714 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:25.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:25.715 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:25.795 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":748,"stamp":"2024-09-10T19:01:21.386338+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8645861054,"num_objects":132112,"num_object_clones":0,"num_object_copies":396336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4052,"num_objects_unfound":0,"num_objects_dirty":132112,"num_whiteouts":0,"num_read":2093,"num_read_kb":1848,"num_write":132635,"num_write_kb":8442716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18090,"num_bytes_recovered":1168286570,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":60036,"ondisk_log_size":60036,"up":504,"acting":504,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":507,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":562470912,"kb_used":31605908,"kb_used_data":26793932,"kb_used_omap":12,"kb_used_meta":4811187,"kb_avail":530865004,"statfs":{"total":575970213888,"available":543605764096,"internally_reserved":0,"allocated":27436986368,"data_stored":27434316620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13122,"internal_metadata":4926655678},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,3,6,9,7,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":613482496,"num_objects":9361,"num_object_clones":0,"num_object_copies":28083,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-3458,"num_objects_unfound":0,"num_objects_dirty":9361,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":9361,"num_write_kb":599104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":262602752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002964"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":356,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.490344+0000","last_change":"2024-09-10T18:57:02.462265+0000","last_active":"2024-09-10T19:01:15.490344+0000","last_peered":"2024-09-10T19:01:15.490344+0000","last_clean":"2024-09-10T19:01:15.490344+0000","last_became_active":"2024-09-10T18:57:02.460451+0000","last_became_peered":"2024-09-10T18:57:02.460451+0000","last_unstale":"2024-09-10T19:01:15.490344+0000","last_undegraded":"2024-09-10T19:01:15.490344+0000","last_fullsized":"2024-09-10T19:01:15.490344+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:14.875872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:14.875872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037478699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":352,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.479049+0000","last_change":"2024-09-10T18:59:59.848814+0000","last_active":"2024-09-10T19:01:15.479049+0000","last_peered":"2024-09-10T19:01:15.479049+0000","last_clean":"2024-09-10T19:01:15.479049+0000","last_became_active":"2024-09-10T18:57:02.440648+0000","last_became_peered":"2024-09-10T18:57:02.440648+0000","last_unstale":"2024-09-10T19:01:15.479049+0000","last_undegraded":"2024-09-10T19:01:15.479049+0000","last_fullsized":"2024-09-10T19:01:15.479049+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:59.848700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:59.848700+0000","last_clean_scrub_stamp":"2024-09-10T18:59:59.848700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:27.993110+0000","scrub_duration":0.0047956400000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"232'27","reported_seq":441,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.491690+0000","last_change":"2024-09-10T18:56:02.821637+0000","last_active":"2024-09-10T19:01:15.491690+0000","last_peered":"2024-09-10T19:01:15.491690+0000","last_clean":"2024-09-10T19:01:15.491690+0000","last_became_active":"2024-09-10T18:55:40.394717+0000","last_became_peered":"2024-09-10T18:55:40.394717+0000","last_unstale":"2024-09-10T19:01:15.491690+0000","last_undegraded":"2024-09-10T19:01:15.491690+0000","last_fullsized":"2024-09-10T19:01:15.491690+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'27","last_scrub_stamp":"2024-09-10T18:56:02.821551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:02.821551+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0083110159999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":357,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.475435+0000","last_change":"2024-09-10T19:00:08.841172+0000","last_active":"2024-09-10T19:01:15.475435+0000","last_peered":"2024-09-10T19:01:15.475435+0000","last_clean":"2024-09-10T19:01:15.475435+0000","last_became_active":"2024-09-10T18:45:22.065619+0000","last_became_peered":"2024-09-10T18:45:22.065619+0000","last_unstale":"2024-09-10T19:01:15.475435+0000","last_undegraded":"2024-09-10T19:01:15.475435+0000","last_fullsized":"2024-09-10T19:01:15.475435+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:08.841125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.057932+0000","last_clean_scrub_stamp":"2024-09-10T19:00:08.841125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001059857,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":419,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.491083+0000","last_change":"2024-09-10T19:00:42.519131+0000","last_active":"2024-09-10T19:01:15.491083+0000","last_peered":"2024-09-10T19:01:15.491083+0000","last_clean":"2024-09-10T19:01:15.491083+0000","last_became_active":"2024-09-10T19:00:42.518824+0000","last_became_peered":"2024-09-10T19:00:42.518824+0000","last_unstale":"2024-09-10T19:01:15.491083+0000","last_undegraded":"2024-09-10T19:01:15.491083+0000","last_fullsized":"2024-09-10T19:01:15.491083+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:23.885562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:23.885562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0031033800000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"232'19","reported_seq":378,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.475362+0000","last_change":"2024-09-10T18:59:09.642743+0000","last_active":"2024-09-10T19:01:15.475362+0000","last_peered":"2024-09-10T19:01:15.475362+0000","last_clean":"2024-09-10T19:01:15.475362+0000","last_became_active":"2024-09-10T18:45:20.061266+0000","last_became_peered":"2024-09-10T18:45:20.061266+0000","last_unstale":"2024-09-10T19:01:15.475362+0000","last_undegraded":"2024-09-10T19:01:15.475362+0000","last_fullsized":"2024-09-10T19:01:15.475362+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T18:59:09.642642+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_clean_scrub_stamp":"2024-09-10T18:59:09.642642+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0095920940000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":92,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.726246+0000","last_change":"2024-09-10T19:00:48.452343+0000","last_active":"2024-09-10T19:01:08.726246+0000","last_peered":"2024-09-10T19:01:08.726246+0000","last_clean":"2024-09-10T19:01:08.726246+0000","last_became_active":"2024-09-10T18:57:02.446008+0000","last_became_peered":"2024-09-10T18:57:02.446008+0000","last_unstale":"2024-09-10T19:01:08.726246+0000","last_undegraded":"2024-09-10T19:01:08.726246+0000","last_fullsized":"2024-09-10T19:01:08.726246+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:22.163121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:01.135988+0000","last_clean_scrub_stamp":"2024-09-10T18:56:22.163121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002631478,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":345,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.476303+0000","last_change":"2024-09-10T18:58:51.682046+0000","last_active":"2024-09-10T19:01:15.476303+0000","last_peered":"2024-09-10T19:01:15.476303+0000","last_clean":"2024-09-10T19:01:15.476303+0000","last_became_active":"2024-09-10T18:45:24.081002+0000","last_became_peered":"2024-09-10T18:45:24.081002+0000","last_unstale":"2024-09-10T19:01:15.476303+0000","last_undegraded":"2024-09-10T19:01:15.476303+0000","last_fullsized":"2024-09-10T19:01:15.476303+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:51.681994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:58:51.681994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051776249999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":349,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.730639+0000","last_change":"2024-09-10T18:59:21.502068+0000","last_active":"2024-09-10T19:01:08.730639+0000","last_peered":"2024-09-10T19:01:08.730639+0000","last_clean":"2024-09-10T19:01:08.730639+0000","last_became_active":"2024-09-10T18:57:02.452259+0000","last_became_peered":"2024-09-10T18:57:02.452259+0000","last_unstale":"2024-09-10T19:01:08.730639+0000","last_undegraded":"2024-09-10T19:01:08.730639+0000","last_fullsized":"2024-09-10T19:01:08.730639+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:21.501984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:21.501984+0000","last_clean_scrub_stamp":"2024-09-10T18:59:21.501984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014628558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1b","version":"232'9","reported_seq":364,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.475764+0000","last_change":"2024-09-10T18:59:11.622597+0000","last_active":"2024-09-10T19:01:15.475764+0000","last_peered":"2024-09-10T19:01:15.475764+0000","last_clean":"2024-09-10T19:01:15.475764+0000","last_became_active":"2024-09-10T18:45:20.057567+0000","last_became_peered":"2024-09-10T18:45:20.057567+0000","last_unstale":"2024-09-10T19:01:15.475764+0000","last_undegraded":"2024-09-10T19:01:15.475764+0000","last_fullsized":"2024-09-10T19:01:15.475764+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'9","last_scrub_stamp":"2024-09-10T18:59:11.622509+0000","last_deep_scrub":"163'7","last_deep_scrub_stamp":"2024-09-10T18:51:54.261879+0000","last_clean_scrub_stamp":"2024-09-10T18:59:11.622509+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056120609999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":116,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.476230+0000","last_change":"2024-09-10T19:00:40.947163+0000","last_active":"2024-09-10T19:01:15.476230+0000","last_peered":"2024-09-10T19:01:15.476230+0000","last_clean":"2024-09-10T19:01:15.476230+0000","last_became_active":"2024-09-10T18:58:58.503924+0000","last_became_peered":"2024-09-10T18:58:58.503924+0000","last_unstale":"2024-09-10T19:01:15.476230+0000","last_undegraded":"2024-09-10T19:01:15.476230+0000","last_fullsized":"2024-09-10T19:01:15.476230+0000","mapping_epoch":303,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":304,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:40.947066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:36.509971+0000","last_clean_scrub_stamp":"2024-09-10T19:00:40.947066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:02:09.283651+0000","scrub_duration":0.012293715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":22,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.731661+0000","last_change":"2024-09-10T19:00:42.516918+0000","last_active":"2024-09-10T19:01:08.731661+0000","last_peered":"2024-09-10T19:01:08.731661+0000","last_clean":"2024-09-10T19:01:08.731661+0000","last_became_active":"2024-09-10T19:00:42.515840+0000","last_became_peered":"2024-09-10T19:00:42.515840+0000","last_unstale":"2024-09-10T19:01:08.731661+0000","last_undegraded":"2024-09-10T19:01:08.731661+0000","last_fullsized":"2024-09-10T19:01:08.731661+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:48.230459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:59:48.230459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":155,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.731931+0000","last_change":"2024-09-10T18:58:49.506212+0000","last_active":"2024-09-10T19:01:08.731931+0000","last_peered":"2024-09-10T19:01:08.731931+0000","last_clean":"2024-09-10T19:01:08.731931+0000","last_became_active":"2024-09-10T18:57:02.440497+0000","last_became_peered":"2024-09-10T18:57:02.440497+0000","last_unstale":"2024-09-10T19:01:08.731931+0000","last_undegraded":"2024-09-10T19:01:08.731931+0000","last_fullsized":"2024-09-10T19:01:08.731931+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:49.506160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:58:49.506160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.018244065,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"232'15","reported_seq":403,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.476608+0000","last_change":"2024-09-10T19:01:08.981911+0000","last_active":"2024-09-10T19:01:15.476608+0000","last_peered":"2024-09-10T19:01:15.476608+0000","last_clean":"2024-09-10T19:01:15.476608+0000","last_became_active":"2024-09-10T18:55:40.391571+0000","last_became_peered":"2024-09-10T18:55:40.391571+0000","last_unstale":"2024-09-10T19:01:15.476608+0000","last_undegraded":"2024-09-10T19:01:15.476608+0000","last_fullsized":"2024-09-10T19:01:15.476608+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T18:56:26.480105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:26.480105+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027886920000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":385,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.732395+0000","last_change":"2024-09-10T18:59:22.643043+0000","last_active":"2024-09-10T19:01:08.732395+0000","last_peered":"2024-09-10T19:01:08.732395+0000","last_clean":"2024-09-10T19:01:08.732395+0000","last_became_active":"2024-09-10T18:57:02.431585+0000","last_became_peered":"2024-09-10T18:57:02.431585+0000","last_unstale":"2024-09-10T19:01:08.732395+0000","last_undegraded":"2024-09-10T19:01:08.732395+0000","last_fullsized":"2024-09-10T19:01:08.732395+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:22.642961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:59:22.642961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.018189153999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":41,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.477332+0000","last_change":"2024-09-10T18:59:48.738143+0000","last_active":"2024-09-10T19:01:15.477332+0000","last_peered":"2024-09-10T19:01:15.477332+0000","last_clean":"2024-09-10T19:01:15.477332+0000","last_became_active":"2024-09-10T18:59:48.737788+0000","last_became_peered":"2024-09-10T18:59:48.737788+0000","last_unstale":"2024-09-10T19:01:15.477332+0000","last_undegraded":"2024-09-10T19:01:15.477332+0000","last_fullsized":"2024-09-10T19:01:15.477332+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:51.561010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:59.425718+0000","last_clean_scrub_stamp":"2024-09-10T18:58:51.561010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3","version":"351'8072","reported_seq":8116,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:16.537046+0000","last_change":"2024-09-10T18:59:43.007892+0000","last_active":"2024-09-10T19:01:16.537046+0000","last_peered":"2024-09-10T19:01:16.537046+0000","last_clean":"2024-09-10T19:01:16.537046+0000","last_became_active":"2024-09-10T18:59:43.007585+0000","last_became_peered":"2024-09-10T18:59:43.007585+0000","last_unstale":"2024-09-10T19:01:16.537046+0000","last_undegraded":"2024-09-10T19:01:16.537046+0000","last_fullsized":"2024-09-10T19:01:16.537046+0000","mapping_epoch":315,"log_start":"336'4499","ondisk_log_start":"336'4499","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3573,"log_dups_size":0,"ondisk_log_size":3573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":529006592,"num_objects":8072,"num_object_clones":0,"num_object_copies":24216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8072,"num_write_kb":516608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.8","version":"70'81","reported_seq":26,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.731626+0000","last_change":"2024-09-10T19:00:42.527220+0000","last_active":"2024-09-10T19:01:08.731626+0000","last_peered":"2024-09-10T19:01:08.731626+0000","last_clean":"2024-09-10T19:01:08.731626+0000","last_became_active":"2024-09-10T19:00:42.517072+0000","last_became_peered":"2024-09-10T19:00:42.517072+0000","last_unstale":"2024-09-10T19:01:08.731626+0000","last_undegraded":"2024-09-10T19:01:08.731626+0000","last_fullsized":"2024-09-10T19:01:08.731626+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:51.167302+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:59:51.167302+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015031440000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":262688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"232'39","reported_seq":444,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.477459+0000","last_change":"2024-09-10T18:59:47.745859+0000","last_active":"2024-09-10T19:01:15.477459+0000","last_peered":"2024-09-10T19:01:15.477459+0000","last_clean":"2024-09-10T19:01:15.477459+0000","last_became_active":"2024-09-10T18:47:36.409300+0000","last_became_peered":"2024-09-10T18:47:36.409300+0000","last_unstale":"2024-09-10T19:01:15.477459+0000","last_undegraded":"2024-09-10T19:01:15.477459+0000","last_fullsized":"2024-09-10T19:01:15.477459+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'39","last_scrub_stamp":"2024-09-10T18:59:47.745766+0000","last_deep_scrub":"78'25","last_deep_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_clean_scrub_stamp":"2024-09-10T18:59:47.745766+0000","objects_scrubbed":8,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011279000000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":103,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.476807+0000","last_change":"2024-09-10T19:00:09.867278+0000","last_active":"2024-09-10T19:01:15.476807+0000","last_peered":"2024-09-10T19:01:15.476807+0000","last_clean":"2024-09-10T19:01:15.476807+0000","last_became_active":"2024-09-10T18:57:02.437260+0000","last_became_peered":"2024-09-10T18:57:02.437260+0000","last_unstale":"2024-09-10T19:01:15.476807+0000","last_undegraded":"2024-09-10T19:01:15.476807+0000","last_fullsized":"2024-09-10T19:01:15.476807+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:09.867183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:00:09.867183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026871460000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":365,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.725158+0000","last_change":"2024-09-10T19:00:09.470422+0000","last_active":"2024-09-10T19:01:08.725158+0000","last_peered":"2024-09-10T19:01:08.725158+0000","last_clean":"2024-09-10T19:01:08.725158+0000","last_became_active":"2024-09-10T18:55:40.392449+0000","last_became_peered":"2024-09-10T18:55:40.392449+0000","last_unstale":"2024-09-10T19:01:08.725158+0000","last_undegraded":"2024-09-10T19:01:08.725158+0000","last_fullsized":"2024-09-10T19:01:08.725158+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:09.470293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:04.512425+0000","last_clean_scrub_stamp":"2024-09-10T19:00:09.470293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0073759949999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":264,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.479175+0000","last_change":"2024-09-10T18:58:49.524134+0000","last_active":"2024-09-10T19:01:15.479175+0000","last_peered":"2024-09-10T19:01:15.479175+0000","last_clean":"2024-09-10T19:01:15.479175+0000","last_became_active":"2024-09-10T18:55:40.388154+0000","last_became_peered":"2024-09-10T18:55:40.388154+0000","last_unstale":"2024-09-10T19:01:15.479175+0000","last_undegraded":"2024-09-10T19:01:15.479175+0000","last_fullsized":"2024-09-10T19:01:15.479175+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:49.524015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:58:49.524015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005180479,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2","version":"351'8311","reported_seq":8355,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:16.775877+0000","last_change":"2024-09-10T18:59:42.709056+0000","last_active":"2024-09-10T19:01:16.775877+0000","last_peered":"2024-09-10T19:01:16.775877+0000","last_clean":"2024-09-10T19:01:16.775877+0000","last_became_active":"2024-09-10T18:59:42.708732+0000","last_became_peered":"2024-09-10T18:59:42.708732+0000","last_unstale":"2024-09-10T19:01:16.775877+0000","last_undegraded":"2024-09-10T19:01:16.775877+0000","last_fullsized":"2024-09-10T19:01:16.775877+0000","mapping_epoch":315,"log_start":"336'4800","ondisk_log_start":"336'4800","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3511,"log_dups_size":0,"ondisk_log_size":3511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":544669696,"num_objects":8311,"num_object_clones":0,"num_object_copies":24933,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8311,"num_write_kb":531904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.9","version":"70'81","reported_seq":260,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.491852+0000","last_change":"2024-09-10T18:59:56.164487+0000","last_active":"2024-09-10T19:01:15.491852+0000","last_peered":"2024-09-10T19:01:15.491852+0000","last_clean":"2024-09-10T19:01:15.491852+0000","last_became_active":"2024-09-10T18:57:02.463750+0000","last_became_peered":"2024-09-10T18:57:02.463750+0000","last_unstale":"2024-09-10T19:01:15.491852+0000","last_undegraded":"2024-09-10T19:01:15.491852+0000","last_fullsized":"2024-09-10T19:01:15.491852+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:56.164384+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:59:56.164384+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041056720000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"232'25","reported_seq":384,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.476622+0000","last_change":"2024-09-10T18:56:32.437605+0000","last_active":"2024-09-10T19:01:15.476622+0000","last_peered":"2024-09-10T19:01:15.476622+0000","last_clean":"2024-09-10T19:01:15.476622+0000","last_became_active":"2024-09-10T18:45:20.053428+0000","last_became_peered":"2024-09-10T18:45:20.053428+0000","last_unstale":"2024-09-10T19:01:15.476622+0000","last_undegraded":"2024-09-10T19:01:15.476622+0000","last_fullsized":"2024-09-10T19:01:15.476622+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'25","last_scrub_stamp":"2024-09-10T18:56:32.437503+0000","last_deep_scrub":"116'15","last_deep_scrub_stamp":"2024-09-10T18:49:12.374527+0000","last_clean_scrub_stamp":"2024-09-10T18:56:32.437503+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039673679999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":365,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.724998+0000","last_change":"2024-09-10T19:00:51.441092+0000","last_active":"2024-09-10T19:01:08.724998+0000","last_peered":"2024-09-10T19:01:08.724998+0000","last_clean":"2024-09-10T19:01:08.724998+0000","last_became_active":"2024-09-10T18:57:02.440442+0000","last_became_peered":"2024-09-10T18:57:02.440442+0000","last_unstale":"2024-09-10T19:01:08.724998+0000","last_undegraded":"2024-09-10T19:01:08.724998+0000","last_fullsized":"2024-09-10T19:01:08.724998+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:29.802937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:29.802937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044818080000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"273'13","reported_seq":404,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.479623+0000","last_change":"2024-09-10T18:57:02.463853+0000","last_active":"2024-09-10T19:01:15.479623+0000","last_peered":"2024-09-10T19:01:15.479623+0000","last_clean":"2024-09-10T19:01:15.479623+0000","last_became_active":"2024-09-10T18:57:02.448258+0000","last_became_peered":"2024-09-10T18:57:02.448258+0000","last_unstale":"2024-09-10T19:01:15.479623+0000","last_undegraded":"2024-09-10T19:01:15.479623+0000","last_fullsized":"2024-09-10T19:01:15.479623+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"242'12","last_scrub_stamp":"2024-09-10T18:56:27.995682+0000","last_deep_scrub":"202'10","last_deep_scrub_stamp":"2024-09-10T18:53:45.547686+0000","last_clean_scrub_stamp":"2024-09-10T18:56:27.995682+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025134200000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":395,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.475900+0000","last_change":"2024-09-10T19:00:42.515842+0000","last_active":"2024-09-10T19:01:15.475900+0000","last_peered":"2024-09-10T19:01:15.475900+0000","last_clean":"2024-09-10T19:01:15.475900+0000","last_became_active":"2024-09-10T19:00:42.515523+0000","last_became_peered":"2024-09-10T19:00:42.515523+0000","last_unstale":"2024-09-10T19:01:15.475900+0000","last_undegraded":"2024-09-10T19:01:15.475900+0000","last_fullsized":"2024-09-10T19:01:15.475900+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:07.803916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:32.043666+0000","last_clean_scrub_stamp":"2024-09-10T19:00:07.803916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:26.903851+0000","scrub_duration":0.0011072429999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1","version":"352'8526","reported_seq":8571,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-10T19:01:21.345325+0000","last_change":"2024-09-10T18:59:42.711934+0000","last_active":"2024-09-10T19:01:21.345325+0000","last_peered":"2024-09-10T19:01:21.345325+0000","last_clean":"2024-09-10T19:01:21.345325+0000","last_became_active":"2024-09-10T18:59:42.711662+0000","last_became_peered":"2024-09-10T18:59:42.711662+0000","last_unstale":"2024-09-10T19:01:21.345325+0000","last_undegraded":"2024-09-10T19:01:21.345325+0000","last_fullsized":"2024-09-10T19:01:21.345325+0000","mapping_epoch":315,"log_start":"336'4900","ondisk_log_start":"336'4900","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3626,"log_dups_size":0,"ondisk_log_size":3626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":558759936,"num_objects":8526,"num_object_clones":0,"num_object_copies":25578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8526,"num_write_kb":545664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1],"acting":[4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"70'81","reported_seq":273,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.732928+0000","last_change":"2024-09-10T18:59:11.493175+0000","last_active":"2024-09-10T19:01:08.732928+0000","last_peered":"2024-09-10T19:01:08.732928+0000","last_clean":"2024-09-10T19:01:08.732928+0000","last_became_active":"2024-09-10T18:50:08.398345+0000","last_became_peered":"2024-09-10T18:50:08.398345+0000","last_unstale":"2024-09-10T19:01:08.732928+0000","last_undegraded":"2024-09-10T19:01:08.732928+0000","last_fullsized":"2024-09-10T19:01:08.732928+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:11.493086+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:55:37.921519+0000","last_clean_scrub_stamp":"2024-09-10T18:59:11.493086+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013358479999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"232'34","reported_seq":380,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.732608+0000","last_change":"2024-09-10T19:00:41.482710+0000","last_active":"2024-09-10T19:01:08.732608+0000","last_peered":"2024-09-10T19:01:08.732608+0000","last_clean":"2024-09-10T19:01:08.732608+0000","last_became_active":"2024-09-10T18:46:56.742946+0000","last_became_peered":"2024-09-10T18:46:56.742946+0000","last_unstale":"2024-09-10T19:01:08.732608+0000","last_undegraded":"2024-09-10T19:01:08.732608+0000","last_fullsized":"2024-09-10T19:01:08.732608+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'34","last_scrub_stamp":"2024-09-10T19:00:41.482619+0000","last_deep_scrub":"141'22","last_deep_scrub_stamp":"2024-09-10T18:50:57.119152+0000","last_clean_scrub_stamp":"2024-09-10T19:00:41.482619+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:02:06.038176+0000","scrub_duration":0.0069363309999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":375,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.726465+0000","last_change":"2024-09-10T18:59:51.286189+0000","last_active":"2024-09-10T19:01:08.726465+0000","last_peered":"2024-09-10T19:01:08.726465+0000","last_clean":"2024-09-10T19:01:08.726465+0000","last_became_active":"2024-09-10T18:57:02.445848+0000","last_became_peered":"2024-09-10T18:57:02.445848+0000","last_unstale":"2024-09-10T19:01:08.726465+0000","last_undegraded":"2024-09-10T19:01:08.726465+0000","last_fullsized":"2024-09-10T19:01:08.726465+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:51.286131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_clean_scrub_stamp":"2024-09-10T18:59:51.286131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:18.689647+0000","scrub_duration":0.0015180669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"242'10","reported_seq":567,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-10T19:01:17.482696+0000","last_change":"2024-09-10T18:56:19.581070+0000","last_active":"2024-09-10T19:01:17.482696+0000","last_peered":"2024-09-10T19:01:17.482696+0000","last_clean":"2024-09-10T19:01:17.482696+0000","last_became_active":"2024-09-10T18:55:40.397246+0000","last_became_peered":"2024-09-10T18:55:40.397246+0000","last_unstale":"2024-09-10T19:01:17.482696+0000","last_undegraded":"2024-09-10T19:01:17.482696+0000","last_fullsized":"2024-09-10T19:01:17.482696+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"242'10","last_scrub_stamp":"2024-09-10T18:56:19.581012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:19.581012+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013423529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":373,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.731999+0000","last_change":"2024-09-10T18:59:10.477418+0000","last_active":"2024-09-10T19:01:08.731999+0000","last_peered":"2024-09-10T19:01:08.731999+0000","last_clean":"2024-09-10T19:01:08.731999+0000","last_became_active":"2024-09-10T18:55:40.392003+0000","last_became_peered":"2024-09-10T18:55:40.392003+0000","last_unstale":"2024-09-10T19:01:08.731999+0000","last_undegraded":"2024-09-10T19:01:08.731999+0000","last_fullsized":"2024-09-10T19:01:08.731999+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:10.477346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:54.218602+0000","last_clean_scrub_stamp":"2024-09-10T18:59:10.477346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004313005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"351'8173","reported_seq":8217,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:16.453390+0000","last_change":"2024-09-10T18:59:42.715764+0000","last_active":"2024-09-10T19:01:16.453390+0000","last_peered":"2024-09-10T19:01:16.453390+0000","last_clean":"2024-09-10T19:01:16.453390+0000","last_became_active":"2024-09-10T18:59:42.714808+0000","last_became_peered":"2024-09-10T18:59:42.714808+0000","last_unstale":"2024-09-10T19:01:16.453390+0000","last_undegraded":"2024-09-10T19:01:16.453390+0000","last_fullsized":"2024-09-10T19:01:16.453390+0000","mapping_epoch":315,"log_start":"336'4600","ondisk_log_start":"336'4600","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3573,"log_dups_size":0,"ondisk_log_size":3573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":535625728,"num_objects":8173,"num_object_clones":0,"num_object_copies":24519,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8173,"num_write_kb":523072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.b","version":"70'81","reported_seq":318,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.725276+0000","last_change":"2024-09-10T19:01:04.457442+0000","last_active":"2024-09-10T19:01:08.725276+0000","last_peered":"2024-09-10T19:01:08.725276+0000","last_clean":"2024-09-10T19:01:08.725276+0000","last_became_active":"2024-09-10T19:00:42.518107+0000","last_became_peered":"2024-09-10T19:00:42.518107+0000","last_unstale":"2024-09-10T19:01:08.725276+0000","last_undegraded":"2024-09-10T19:01:08.725276+0000","last_fullsized":"2024-09-10T19:01:08.725276+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:33.793296+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:33.793296+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045437230000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"232'24","reported_seq":461,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.476468+0000","last_change":"2024-09-10T18:59:06.595605+0000","last_active":"2024-09-10T19:01:15.476468+0000","last_peered":"2024-09-10T19:01:15.476468+0000","last_clean":"2024-09-10T19:01:15.476468+0000","last_became_active":"2024-09-10T18:47:57.106935+0000","last_became_peered":"2024-09-10T18:47:57.106935+0000","last_unstale":"2024-09-10T19:01:15.476468+0000","last_undegraded":"2024-09-10T19:01:15.476468+0000","last_fullsized":"2024-09-10T19:01:15.476468+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T18:59:06.595554+0000","last_deep_scrub":"224'16","last_deep_scrub_stamp":"2024-09-10T18:54:49.296957+0000","last_clean_scrub_stamp":"2024-09-10T18:59:06.595554+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020642540000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":347,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.477689+0000","last_change":"2024-09-10T18:57:02.444295+0000","last_active":"2024-09-10T19:01:15.477689+0000","last_peered":"2024-09-10T19:01:15.477689+0000","last_clean":"2024-09-10T19:01:15.477689+0000","last_became_active":"2024-09-10T18:57:02.443816+0000","last_became_peered":"2024-09-10T18:57:02.443816+0000","last_unstale":"2024-09-10T19:01:15.477689+0000","last_undegraded":"2024-09-10T19:01:15.477689+0000","last_fullsized":"2024-09-10T19:01:15.477689+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:29.994533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:29.994533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053276879999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":361,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.732740+0000","last_change":"2024-09-10T18:58:54.548235+0000","last_active":"2024-09-10T19:01:08.732740+0000","last_peered":"2024-09-10T19:01:08.732740+0000","last_clean":"2024-09-10T19:01:08.732740+0000","last_became_active":"2024-09-10T18:45:22.068918+0000","last_became_peered":"2024-09-10T18:45:22.068918+0000","last_unstale":"2024-09-10T19:01:08.732740+0000","last_undegraded":"2024-09-10T19:01:08.732740+0000","last_fullsized":"2024-09-10T19:01:08.732740+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:54.548153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_clean_scrub_stamp":"2024-09-10T18:58:54.548153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013051650000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":346,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.491179+0000","last_change":"2024-09-10T18:59:13.892307+0000","last_active":"2024-09-10T19:01:15.491179+0000","last_peered":"2024-09-10T19:01:15.491179+0000","last_clean":"2024-09-10T19:01:15.491179+0000","last_became_active":"2024-09-10T18:45:24.264538+0000","last_became_peered":"2024-09-10T18:45:24.264538+0000","last_unstale":"2024-09-10T19:01:15.491179+0000","last_undegraded":"2024-09-10T19:01:15.491179+0000","last_fullsized":"2024-09-10T19:01:15.491179+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:13.892129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:51.398981+0000","last_clean_scrub_stamp":"2024-09-10T18:59:13.892129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007612214,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"351'8115","reported_seq":8159,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:16.473381+0000","last_change":"2024-09-10T18:59:43.008094+0000","last_active":"2024-09-10T19:01:16.473381+0000","last_peered":"2024-09-10T19:01:16.473381+0000","last_clean":"2024-09-10T19:01:16.473381+0000","last_became_active":"2024-09-10T18:59:43.007838+0000","last_became_peered":"2024-09-10T18:59:43.007838+0000","last_unstale":"2024-09-10T19:01:16.473381+0000","last_undegraded":"2024-09-10T19:01:16.473381+0000","last_fullsized":"2024-09-10T19:01:16.473381+0000","mapping_epoch":315,"log_start":"336'4600","ondisk_log_start":"336'4600","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3515,"log_dups_size":0,"ondisk_log_size":3515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":531824640,"num_objects":8115,"num_object_clones":0,"num_object_copies":24345,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8115,"num_write_kb":519360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"70'81","reported_seq":304,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.491584+0000","last_change":"2024-09-10T19:00:42.520084+0000","last_active":"2024-09-10T19:01:15.491584+0000","last_peered":"2024-09-10T19:01:15.491584+0000","last_clean":"2024-09-10T19:01:15.491584+0000","last_became_active":"2024-09-10T19:00:42.519742+0000","last_became_peered":"2024-09-10T19:00:42.519742+0000","last_unstale":"2024-09-10T19:01:15.491584+0000","last_undegraded":"2024-09-10T19:01:15.491584+0000","last_fullsized":"2024-09-10T19:01:15.491584+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:58:56.912936+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:58:56.912936+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045874109999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"232'35","reported_seq":398,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.476045+0000","last_change":"2024-09-10T18:56:28.475224+0000","last_active":"2024-09-10T19:01:15.476045+0000","last_peered":"2024-09-10T19:01:15.476045+0000","last_clean":"2024-09-10T19:01:15.476045+0000","last_became_active":"2024-09-10T18:45:20.052243+0000","last_became_peered":"2024-09-10T18:45:20.052243+0000","last_unstale":"2024-09-10T19:01:15.476045+0000","last_undegraded":"2024-09-10T19:01:15.476045+0000","last_fullsized":"2024-09-10T19:01:15.476045+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'35","last_scrub_stamp":"2024-09-10T18:56:28.475142+0000","last_deep_scrub":"143'21","last_deep_scrub_stamp":"2024-09-10T18:51:02.540216+0000","last_clean_scrub_stamp":"2024-09-10T18:56:28.475142+0000","objects_scrubbed":7,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026978649999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":383,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.730318+0000","last_change":"2024-09-10T18:57:02.453766+0000","last_active":"2024-09-10T19:01:08.730318+0000","last_peered":"2024-09-10T19:01:08.730318+0000","last_clean":"2024-09-10T19:01:08.730318+0000","last_became_active":"2024-09-10T18:57:02.453404+0000","last_became_peered":"2024-09-10T18:57:02.453404+0000","last_unstale":"2024-09-10T19:01:08.730318+0000","last_undegraded":"2024-09-10T19:01:08.730318+0000","last_fullsized":"2024-09-10T19:01:08.730318+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:29.226803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_clean_scrub_stamp":"2024-09-10T18:56:29.226803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0086984899999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":548,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:16.263376+0000","last_change":"2024-09-10T18:56:17.914259+0000","last_active":"2024-09-10T19:01:16.263376+0000","last_peered":"2024-09-10T19:01:16.263376+0000","last_clean":"2024-09-10T19:01:16.263376+0000","last_became_active":"2024-09-10T18:45:22.074540+0000","last_became_peered":"2024-09-10T18:45:22.074540+0000","last_unstale":"2024-09-10T19:01:16.263376+0000","last_undegraded":"2024-09-10T19:01:16.263376+0000","last_fullsized":"2024-09-10T19:01:16.263376+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:56:17.914175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:17.914175+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026666329999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":150,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.728296+0000","last_change":"2024-09-10T18:58:50.638774+0000","last_active":"2024-09-10T19:01:08.728296+0000","last_peered":"2024-09-10T19:01:08.728296+0000","last_clean":"2024-09-10T19:01:08.728296+0000","last_became_active":"2024-09-10T18:57:02.437100+0000","last_became_peered":"2024-09-10T18:57:02.437100+0000","last_unstale":"2024-09-10T19:01:08.728296+0000","last_undegraded":"2024-09-10T19:01:08.728296+0000","last_fullsized":"2024-09-10T19:01:08.728296+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:50.638720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:58:50.638720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003753546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.e","version":"352'8458","reported_seq":8503,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-10T19:01:21.338175+0000","last_change":"2024-09-10T18:59:42.713933+0000","last_active":"2024-09-10T19:01:21.338175+0000","last_peered":"2024-09-10T19:01:21.338175+0000","last_clean":"2024-09-10T19:01:21.338175+0000","last_became_active":"2024-09-10T18:59:42.713605+0000","last_became_peered":"2024-09-10T18:59:42.713605+0000","last_unstale":"2024-09-10T19:01:21.338175+0000","last_undegraded":"2024-09-10T19:01:21.338175+0000","last_fullsized":"2024-09-10T19:01:21.338175+0000","mapping_epoch":315,"log_start":"336'4900","ondisk_log_start":"336'4900","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3558,"log_dups_size":0,"ondisk_log_size":3558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554303488,"num_objects":8458,"num_object_clones":0,"num_object_copies":25374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8458,"num_write_kb":541312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"70'81","reported_seq":269,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.475532+0000","last_change":"2024-09-10T18:56:31.481362+0000","last_active":"2024-09-10T19:01:15.475532+0000","last_peered":"2024-09-10T19:01:15.475532+0000","last_clean":"2024-09-10T19:01:15.475532+0000","last_became_active":"2024-09-10T18:52:14.584798+0000","last_became_peered":"2024-09-10T18:52:14.584798+0000","last_unstale":"2024-09-10T19:01:15.475532+0000","last_undegraded":"2024-09-10T19:01:15.475532+0000","last_fullsized":"2024-09-10T19:01:15.475532+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:31.481303+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:56:31.481303+0000","last_clean_scrub_stamp":"2024-09-10T18:56:31.481303+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0011170959999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"232'30","reported_seq":516,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.490306+0000","last_change":"2024-09-10T18:59:48.754412+0000","last_active":"2024-09-10T19:01:15.490306+0000","last_peered":"2024-09-10T19:01:15.490306+0000","last_clean":"2024-09-10T19:01:15.490306+0000","last_became_active":"2024-09-10T18:59:48.742377+0000","last_became_peered":"2024-09-10T18:59:48.742377+0000","last_unstale":"2024-09-10T19:01:15.490306+0000","last_undegraded":"2024-09-10T19:01:15.490306+0000","last_fullsized":"2024-09-10T19:01:15.490306+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'30","last_scrub_stamp":"2024-09-10T18:56:15.875924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:15.875924+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0075153879999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":355,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.725283+0000","last_change":"2024-09-10T18:59:05.246945+0000","last_active":"2024-09-10T19:01:08.725283+0000","last_peered":"2024-09-10T19:01:08.725283+0000","last_clean":"2024-09-10T19:01:08.725283+0000","last_became_active":"2024-09-10T18:45:17.961695+0000","last_became_peered":"2024-09-10T18:45:17.961695+0000","last_unstale":"2024-09-10T19:01:08.725283+0000","last_undegraded":"2024-09-10T19:01:08.725283+0000","last_fullsized":"2024-09-10T19:01:08.725283+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:05.246629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:46.142129+0000","last_clean_scrub_stamp":"2024-09-10T18:59:05.246629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0072716029999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":367,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.475832+0000","last_change":"2024-09-10T18:59:57.903394+0000","last_active":"2024-09-10T19:01:15.475832+0000","last_peered":"2024-09-10T19:01:15.475832+0000","last_clean":"2024-09-10T19:01:15.475832+0000","last_became_active":"2024-09-10T18:57:02.448126+0000","last_became_peered":"2024-09-10T18:57:02.448126+0000","last_unstale":"2024-09-10T19:01:15.475832+0000","last_undegraded":"2024-09-10T19:01:15.475832+0000","last_fullsized":"2024-09-10T19:01:15.475832+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:57.903311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:57.903311+0000","last_clean_scrub_stamp":"2024-09-10T18:59:57.903311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011228738,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":355,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.476546+0000","last_change":"2024-09-10T18:58:48.633501+0000","last_active":"2024-09-10T19:01:15.476546+0000","last_peered":"2024-09-10T19:01:15.476546+0000","last_clean":"2024-09-10T19:01:15.476546+0000","last_became_active":"2024-09-10T18:47:36.410836+0000","last_became_peered":"2024-09-10T18:47:36.410836+0000","last_unstale":"2024-09-10T19:01:15.476546+0000","last_undegraded":"2024-09-10T19:01:15.476546+0000","last_fullsized":"2024-09-10T19:01:15.476546+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:48.633423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_clean_scrub_stamp":"2024-09-10T18:58:48.633423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0079262829999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.d","version":"351'8249","reported_seq":8293,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:16.446997+0000","last_change":"2024-09-10T18:59:42.717015+0000","last_active":"2024-09-10T19:01:16.446997+0000","last_peered":"2024-09-10T19:01:16.446997+0000","last_clean":"2024-09-10T19:01:16.446997+0000","last_became_active":"2024-09-10T18:59:42.716708+0000","last_became_peered":"2024-09-10T18:59:42.716708+0000","last_unstale":"2024-09-10T19:01:16.446997+0000","last_undegraded":"2024-09-10T19:01:16.446997+0000","last_fullsized":"2024-09-10T19:01:16.446997+0000","mapping_epoch":315,"log_start":"336'4700","ondisk_log_start":"336'4700","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3549,"log_dups_size":0,"ondisk_log_size":3549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540606464,"num_objects":8249,"num_object_clones":0,"num_object_copies":24747,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8249,"num_write_kb":527936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"70'81","reported_seq":366,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.729152+0000","last_change":"2024-09-10T18:59:23.481986+0000","last_active":"2024-09-10T19:01:08.729152+0000","last_peered":"2024-09-10T19:01:08.729152+0000","last_clean":"2024-09-10T19:01:08.729152+0000","last_became_active":"2024-09-10T18:55:40.396828+0000","last_became_peered":"2024-09-10T18:55:40.396828+0000","last_unstale":"2024-09-10T19:01:08.729152+0000","last_undegraded":"2024-09-10T19:01:08.729152+0000","last_fullsized":"2024-09-10T19:01:08.729152+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:23.481902+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:59:23.481902+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0052121340000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"232'29","reported_seq":409,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.478004+0000","last_change":"2024-09-10T18:58:58.837250+0000","last_active":"2024-09-10T19:01:15.478004+0000","last_peered":"2024-09-10T19:01:15.478004+0000","last_clean":"2024-09-10T19:01:15.478004+0000","last_became_active":"2024-09-10T18:57:02.442862+0000","last_became_peered":"2024-09-10T18:57:02.442862+0000","last_unstale":"2024-09-10T19:01:15.478004+0000","last_undegraded":"2024-09-10T19:01:15.478004+0000","last_fullsized":"2024-09-10T19:01:15.478004+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'29","last_scrub_stamp":"2024-09-10T18:58:58.837163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:58:58.837163+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.019169703999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":347,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.480177+0000","last_change":"2024-09-10T18:59:53.858914+0000","last_active":"2024-09-10T19:01:15.480177+0000","last_peered":"2024-09-10T19:01:15.480177+0000","last_clean":"2024-09-10T19:01:15.480177+0000","last_became_active":"2024-09-10T18:57:02.443522+0000","last_became_peered":"2024-09-10T18:57:02.443522+0000","last_unstale":"2024-09-10T19:01:15.480177+0000","last_undegraded":"2024-09-10T19:01:15.480177+0000","last_fullsized":"2024-09-10T19:01:15.480177+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:53.858807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:48.196065+0000","last_clean_scrub_stamp":"2024-09-10T18:59:53.858807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044256060000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":376,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.727555+0000","last_change":"2024-09-10T18:59:15.511792+0000","last_active":"2024-09-10T19:01:08.727555+0000","last_peered":"2024-09-10T19:01:08.727555+0000","last_clean":"2024-09-10T19:01:08.727555+0000","last_became_active":"2024-09-10T18:55:40.387857+0000","last_became_peered":"2024-09-10T18:55:40.387857+0000","last_unstale":"2024-09-10T19:01:08.727555+0000","last_undegraded":"2024-09-10T19:01:08.727555+0000","last_fullsized":"2024-09-10T19:01:08.727555+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:15.511743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:59:15.511743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001893837,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":366,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.726352+0000","last_change":"2024-09-10T18:59:16.235312+0000","last_active":"2024-09-10T19:01:08.726352+0000","last_peered":"2024-09-10T19:01:08.726352+0000","last_clean":"2024-09-10T19:01:08.726352+0000","last_became_active":"2024-09-10T18:57:02.442479+0000","last_became_peered":"2024-09-10T18:57:02.442479+0000","last_unstale":"2024-09-10T19:01:08.726352+0000","last_undegraded":"2024-09-10T19:01:08.726352+0000","last_fullsized":"2024-09-10T19:01:08.726352+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:16.235235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:49:18.281230+0000","last_clean_scrub_stamp":"2024-09-10T18:59:16.235235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034192950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"351'8019","reported_seq":9650,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:16.781989+0000","last_change":"2024-09-10T19:00:08.469551+0000","last_active":"2024-09-10T19:01:16.781989+0000","last_peered":"2024-09-10T19:01:16.781989+0000","last_clean":"2024-09-10T19:01:16.781989+0000","last_became_active":"2024-09-10T19:00:08.465795+0000","last_became_peered":"2024-09-10T19:00:08.465795+0000","last_unstale":"2024-09-10T19:01:16.781989+0000","last_undegraded":"2024-09-10T19:01:16.781989+0000","last_fullsized":"2024-09-10T19:01:16.781989+0000","mapping_epoch":329,"log_start":"336'4500","ondisk_log_start":"336'4500","created":315,"last_epoch_clean":330,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3519,"log_dups_size":0,"ondisk_log_size":3519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":525533184,"num_objects":8019,"num_object_clones":0,"num_object_copies":24057,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8019,"num_write_kb":513216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":784,"num_bytes_recovered":51380224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"70'81","reported_seq":369,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.730888+0000","last_change":"2024-09-10T18:59:53.550489+0000","last_active":"2024-09-10T19:01:08.730888+0000","last_peered":"2024-09-10T19:01:08.730888+0000","last_clean":"2024-09-10T19:01:08.730888+0000","last_became_active":"2024-09-10T18:57:02.452659+0000","last_became_peered":"2024-09-10T18:57:02.452659+0000","last_unstale":"2024-09-10T19:01:08.730888+0000","last_undegraded":"2024-09-10T19:01:08.730888+0000","last_fullsized":"2024-09-10T19:01:08.730888+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:53.550433+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:59:53.550433+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013042399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"232'25","reported_seq":469,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.724748+0000","last_change":"2024-09-10T19:00:42.531653+0000","last_active":"2024-09-10T19:01:08.724748+0000","last_peered":"2024-09-10T19:01:08.724748+0000","last_clean":"2024-09-10T19:01:08.724748+0000","last_became_active":"2024-09-10T19:00:42.517353+0000","last_became_peered":"2024-09-10T19:00:42.517353+0000","last_unstale":"2024-09-10T19:01:08.724748+0000","last_undegraded":"2024-09-10T19:01:08.724748+0000","last_fullsized":"2024-09-10T19:01:08.724748+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'25","last_scrub_stamp":"2024-09-10T19:00:07.390135+0000","last_deep_scrub":"83'15","last_deep_scrub_stamp":"2024-09-10T18:46:58.436473+0000","last_clean_scrub_stamp":"2024-09-10T19:00:07.390135+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002339874,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":392,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:14.526249+0000","last_change":"2024-09-10T19:01:14.526223+0000","last_active":"2024-09-10T19:01:14.526249+0000","last_peered":"2024-09-10T19:01:14.526249+0000","last_clean":"2024-09-10T19:01:14.526249+0000","last_became_active":"2024-09-10T18:57:02.447940+0000","last_became_peered":"2024-09-10T18:57:02.447940+0000","last_unstale":"2024-09-10T19:01:14.526249+0000","last_undegraded":"2024-09-10T19:01:14.526249+0000","last_fullsized":"2024-09-10T19:01:14.526249+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:50.137393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:05.140027+0000","last_clean_scrub_stamp":"2024-09-10T18:58:50.137393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014454461,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"339'13","reported_seq":519,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:16.262685+0000","last_change":"2024-09-10T19:00:42.528804+0000","last_active":"2024-09-10T19:01:16.262685+0000","last_peered":"2024-09-10T19:01:16.262685+0000","last_clean":"2024-09-10T19:01:16.262685+0000","last_became_active":"2024-09-10T19:00:42.520360+0000","last_became_peered":"2024-09-10T19:00:42.520360+0000","last_unstale":"2024-09-10T19:01:16.262685+0000","last_undegraded":"2024-09-10T19:01:16.262685+0000","last_fullsized":"2024-09-10T19:01:16.262685+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'12","last_scrub_stamp":"2024-09-10T18:59:55.943869+0000","last_deep_scrub":"258'12","last_deep_scrub_stamp":"2024-09-10T18:56:21.098770+0000","last_clean_scrub_stamp":"2024-09-10T18:59:55.943869+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:17.541448+0000","scrub_duration":0.020416790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":333,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.478143+0000","last_change":"2024-09-10T19:00:42.515515+0000","last_active":"2024-09-10T19:01:15.478143+0000","last_peered":"2024-09-10T19:01:15.478143+0000","last_clean":"2024-09-10T19:01:15.478143+0000","last_became_active":"2024-09-10T19:00:42.515144+0000","last_became_peered":"2024-09-10T19:00:42.515144+0000","last_unstale":"2024-09-10T19:01:15.478143+0000","last_undegraded":"2024-09-10T19:01:15.478143+0000","last_fullsized":"2024-09-10T19:01:15.478143+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:57:03.353014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:57:03.353014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014199639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.a","version":"351'8263","reported_seq":8317,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:16.486997+0000","last_change":"2024-09-10T18:59:48.739701+0000","last_active":"2024-09-10T19:01:16.486997+0000","last_peered":"2024-09-10T19:01:16.486997+0000","last_clean":"2024-09-10T19:01:16.486997+0000","last_became_active":"2024-09-10T18:59:48.739415+0000","last_became_peered":"2024-09-10T18:59:48.739415+0000","last_unstale":"2024-09-10T19:01:16.486997+0000","last_undegraded":"2024-09-10T19:01:16.486997+0000","last_fullsized":"2024-09-10T19:01:16.486997+0000","mapping_epoch":320,"log_start":"336'4700","ondisk_log_start":"336'4700","created":315,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3563,"log_dups_size":0,"ondisk_log_size":3563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":541523968,"num_objects":8263,"num_object_clones":0,"num_object_copies":24789,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8263,"num_write_kb":528832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"70'81","reported_seq":300,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.731238+0000","last_change":"2024-09-10T19:00:40.472680+0000","last_active":"2024-09-10T19:01:08.731238+0000","last_peered":"2024-09-10T19:01:08.731238+0000","last_clean":"2024-09-10T19:01:08.731238+0000","last_became_active":"2024-09-10T18:55:40.392432+0000","last_became_peered":"2024-09-10T18:55:40.392432+0000","last_unstale":"2024-09-10T19:01:08.731238+0000","last_undegraded":"2024-09-10T19:01:08.731238+0000","last_fullsized":"2024-09-10T19:01:08.731238+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:00:40.472595+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:00:40.472595+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:44.413314+0000","scrub_duration":0.0072897099999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"232'41","reported_seq":643,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.481990+0000","last_change":"2024-09-10T19:00:57.500215+0000","last_active":"2024-09-10T19:01:15.481990+0000","last_peered":"2024-09-10T19:01:15.481990+0000","last_clean":"2024-09-10T19:01:15.481990+0000","last_became_active":"2024-09-10T18:47:36.420185+0000","last_became_peered":"2024-09-10T18:47:36.420185+0000","last_unstale":"2024-09-10T19:01:15.481990+0000","last_undegraded":"2024-09-10T19:01:15.481990+0000","last_fullsized":"2024-09-10T19:01:15.481990+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'41","last_scrub_stamp":"2024-09-10T18:56:18.672139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:18.672139+0000","objects_scrubbed":10,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.020233260999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":365,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.732217+0000","last_change":"2024-09-10T18:59:04.434037+0000","last_active":"2024-09-10T19:01:08.732217+0000","last_peered":"2024-09-10T19:01:08.732217+0000","last_clean":"2024-09-10T19:01:08.732217+0000","last_became_active":"2024-09-10T18:45:17.964494+0000","last_became_peered":"2024-09-10T18:45:17.964494+0000","last_unstale":"2024-09-10T19:01:08.732217+0000","last_undegraded":"2024-09-10T19:01:08.732217+0000","last_fullsized":"2024-09-10T19:01:08.732217+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:04.433964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:12.982599+0000","last_clean_scrub_stamp":"2024-09-10T18:59:04.433964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024364220000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":307,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.730371+0000","last_change":"2024-09-10T18:56:20.224668+0000","last_active":"2024-09-10T19:01:08.730371+0000","last_peered":"2024-09-10T19:01:08.730371+0000","last_clean":"2024-09-10T19:01:08.730371+0000","last_became_active":"2024-09-10T18:47:57.107478+0000","last_became_peered":"2024-09-10T18:47:57.107478+0000","last_unstale":"2024-09-10T19:01:08.730371+0000","last_undegraded":"2024-09-10T19:01:08.730371+0000","last_fullsized":"2024-09-10T19:01:08.730371+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:20.224562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:20.224562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.008542315,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":346,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.490902+0000","last_change":"2024-09-10T18:57:02.462009+0000","last_active":"2024-09-10T19:01:15.490902+0000","last_peered":"2024-09-10T19:01:15.490902+0000","last_clean":"2024-09-10T19:01:15.490902+0000","last_became_active":"2024-09-10T18:57:02.460594+0000","last_became_peered":"2024-09-10T18:57:02.460594+0000","last_unstale":"2024-09-10T19:01:15.490902+0000","last_undegraded":"2024-09-10T19:01:15.490902+0000","last_fullsized":"2024-09-10T19:01:15.490902+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:11.878181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:22.891084+0000","last_clean_scrub_stamp":"2024-09-10T18:56:11.878181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037255349999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"352'8508","reported_seq":20828,"reported_epoch":352,"state":"active+remapped+backfilling","last_fresh":"2024-09-10T19:01:21.034249+0000","last_change":"2024-09-10T19:00:43.855834+0000","last_active":"2024-09-10T19:01:21.034249+0000","last_peered":"2024-09-10T19:01:21.034249+0000","last_clean":"2024-09-10T19:00:41.499200+0000","last_became_active":"2024-09-10T19:00:43.833848+0000","last_became_peered":"2024-09-10T19:00:43.833848+0000","last_unstale":"2024-09-10T19:01:21.034249+0000","last_undegraded":"2024-09-10T19:01:21.034249+0000","last_fullsized":"2024-09-10T19:01:21.034249+0000","mapping_epoch":339,"log_start":"336'4800","ondisk_log_start":"336'4800","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3708,"log_dups_size":0,"ondisk_log_size":3708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":557580288,"num_objects":8508,"num_object_clones":0,"num_object_copies":25524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":958,"num_objects_unfound":0,"num_objects_dirty":8508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8508,"num_write_kb":544512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6132,"num_bytes_recovered":401866752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[3,4,0],"avail_no_missing":["3","0","4"],"object_location_counts":[{"shards":"0,3,4","objects":8508}],"blocked_by":[],"up_primary":5,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"70'81","reported_seq":169,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.482414+0000","last_change":"2024-09-10T18:59:11.875713+0000","last_active":"2024-09-10T19:01:15.482414+0000","last_peered":"2024-09-10T19:01:15.482414+0000","last_clean":"2024-09-10T19:01:15.482414+0000","last_became_active":"2024-09-10T18:57:02.437944+0000","last_became_peered":"2024-09-10T18:57:02.437944+0000","last_unstale":"2024-09-10T19:01:15.482414+0000","last_undegraded":"2024-09-10T19:01:15.482414+0000","last_fullsized":"2024-09-10T19:01:15.482414+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:11.875625+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:59:11.875625+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040920360000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"232'22","reported_seq":412,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.726063+0000","last_change":"2024-09-10T19:00:08.468265+0000","last_active":"2024-09-10T19:01:08.726063+0000","last_peered":"2024-09-10T19:01:08.726063+0000","last_clean":"2024-09-10T19:01:08.726063+0000","last_became_active":"2024-09-10T18:55:40.397422+0000","last_became_peered":"2024-09-10T18:55:40.397422+0000","last_unstale":"2024-09-10T19:01:08.726063+0000","last_undegraded":"2024-09-10T19:01:08.726063+0000","last_fullsized":"2024-09-10T19:01:08.726063+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'22","last_scrub_stamp":"2024-09-10T19:00:08.468154+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-10T18:47:10.514951+0000","last_clean_scrub_stamp":"2024-09-10T19:00:08.468154+0000","objects_scrubbed":5,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:28.777079+0000","scrub_duration":0.0024987170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"84'6","reported_seq":1338,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.686084+0000","last_change":"2024-09-10T19:00:40.504252+0000","last_active":"2024-09-10T19:01:15.686084+0000","last_peered":"2024-09-10T19:01:15.686084+0000","last_clean":"2024-09-10T19:01:15.686084+0000","last_became_active":"2024-09-10T18:55:40.403098+0000","last_became_peered":"2024-09-10T18:55:40.403098+0000","last_unstale":"2024-09-10T19:01:15.686084+0000","last_undegraded":"2024-09-10T19:01:15.686084+0000","last_fullsized":"2024-09-10T19:01:15.686084+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'6","last_scrub_stamp":"2024-09-10T19:00:40.504174+0000","last_deep_scrub":"84'6","last_deep_scrub_stamp":"2024-09-10T18:47:22.545450+0000","last_clean_scrub_stamp":"2024-09-10T19:00:40.504174+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:02:03.384361+0000","scrub_duration":0.0052333889999999997,"objects_trimmed":0,"snaptrim_duration":0.0021933159999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":942,"num_read_kb":937,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":330,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.491601+0000","last_change":"2024-09-10T19:00:08.141664+0000","last_active":"2024-09-10T19:01:15.491601+0000","last_peered":"2024-09-10T19:01:15.491601+0000","last_clean":"2024-09-10T19:01:15.491601+0000","last_became_active":"2024-09-10T18:55:40.393389+0000","last_became_peered":"2024-09-10T18:55:40.393389+0000","last_unstale":"2024-09-10T19:01:15.491601+0000","last_undegraded":"2024-09-10T19:01:15.491601+0000","last_fullsized":"2024-09-10T19:01:15.491601+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:08.141603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:00:08.141603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001270729,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":336,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.733191+0000","last_change":"2024-09-10T18:58:58.522346+0000","last_active":"2024-09-10T19:01:08.733191+0000","last_peered":"2024-09-10T19:01:08.733191+0000","last_clean":"2024-09-10T19:01:08.733191+0000","last_became_active":"2024-09-10T18:57:02.438945+0000","last_became_peered":"2024-09-10T18:57:02.438945+0000","last_unstale":"2024-09-10T19:01:08.733191+0000","last_undegraded":"2024-09-10T19:01:08.733191+0000","last_fullsized":"2024-09-10T19:01:08.733191+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:58.522295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:58:58.522295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0059618290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":103,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.730569+0000","last_change":"2024-09-10T19:00:41.794168+0000","last_active":"2024-09-10T19:01:08.730569+0000","last_peered":"2024-09-10T19:01:08.730569+0000","last_clean":"2024-09-10T19:01:08.730569+0000","last_became_active":"2024-09-10T18:57:02.453865+0000","last_became_peered":"2024-09-10T18:57:02.453865+0000","last_unstale":"2024-09-10T19:01:08.730569+0000","last_undegraded":"2024-09-10T19:01:08.730569+0000","last_fullsized":"2024-09-10T19:01:08.730569+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:41.794116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:07.150480+0000","last_clean_scrub_stamp":"2024-09-10T19:00:41.794116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:45.662042+0000","scrub_duration":0.001296148,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.8","version":"352'8295","reported_seq":8340,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-10T19:01:18.690939+0000","last_change":"2024-09-10T18:59:42.712702+0000","last_active":"2024-09-10T19:01:18.690939+0000","last_peered":"2024-09-10T19:01:18.690939+0000","last_clean":"2024-09-10T19:01:18.690939+0000","last_became_active":"2024-09-10T18:59:42.712397+0000","last_became_peered":"2024-09-10T18:59:42.712397+0000","last_unstale":"2024-09-10T19:01:18.690939+0000","last_undegraded":"2024-09-10T19:01:18.690939+0000","last_fullsized":"2024-09-10T19:01:18.690939+0000","mapping_epoch":315,"log_start":"336'4600","ondisk_log_start":"336'4600","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3695,"log_dups_size":0,"ondisk_log_size":3695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":543621120,"num_objects":8295,"num_object_clones":0,"num_object_copies":24885,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8295,"num_write_kb":530880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,4],"acting":[0,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"70'81","reported_seq":269,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.724896+0000","last_change":"2024-09-10T18:58:49.141636+0000","last_active":"2024-09-10T19:01:08.724896+0000","last_peered":"2024-09-10T19:01:08.724896+0000","last_clean":"2024-09-10T19:01:08.724896+0000","last_became_active":"2024-09-10T18:52:14.589009+0000","last_became_peered":"2024-09-10T18:52:14.589009+0000","last_unstale":"2024-09-10T19:01:08.724896+0000","last_undegraded":"2024-09-10T19:01:08.724896+0000","last_fullsized":"2024-09-10T19:01:08.724896+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:58:49.141013+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:58:49.141013+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0036869540000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"232'24","reported_seq":89,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.478857+0000","last_change":"2024-09-10T19:00:42.532827+0000","last_active":"2024-09-10T19:01:15.478857+0000","last_peered":"2024-09-10T19:01:15.478857+0000","last_clean":"2024-09-10T19:01:15.478857+0000","last_became_active":"2024-09-10T19:00:42.516521+0000","last_became_peered":"2024-09-10T19:00:42.516521+0000","last_unstale":"2024-09-10T19:01:15.478857+0000","last_undegraded":"2024-09-10T19:01:15.478857+0000","last_fullsized":"2024-09-10T19:01:15.478857+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T19:00:39.748509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:00:39.748509+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:02:03.931650+0000","scrub_duration":0.0027783730000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"305'2","reported_seq":336,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.479866+0000","last_change":"2024-09-10T18:59:18.897245+0000","last_active":"2024-09-10T19:01:15.479866+0000","last_peered":"2024-09-10T19:01:15.479866+0000","last_clean":"2024-09-10T19:01:15.479866+0000","last_became_active":"2024-09-10T18:55:40.390022+0000","last_became_peered":"2024-09-10T18:55:40.390022+0000","last_unstale":"2024-09-10T19:01:15.479866+0000","last_undegraded":"2024-09-10T19:01:15.479866+0000","last_fullsized":"2024-09-10T19:01:15.479866+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"305'2","last_scrub_stamp":"2024-09-10T18:59:18.897059+0000","last_deep_scrub":"305'2","last_deep_scrub_stamp":"2024-09-10T18:59:18.897059+0000","last_clean_scrub_stamp":"2024-09-10T18:59:18.897059+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016452700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":296,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.727473+0000","last_change":"2024-09-10T19:00:56.482753+0000","last_active":"2024-09-10T19:01:08.727473+0000","last_peered":"2024-09-10T19:01:08.727473+0000","last_clean":"2024-09-10T19:01:08.727473+0000","last_became_active":"2024-09-10T18:57:02.447195+0000","last_became_peered":"2024-09-10T18:57:02.447195+0000","last_unstale":"2024-09-10T19:01:08.727473+0000","last_undegraded":"2024-09-10T19:01:08.727473+0000","last_fullsized":"2024-09-10T19:01:08.727473+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:24.153973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:24.153973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001771613,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":22,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.732332+0000","last_change":"2024-09-10T19:00:42.518887+0000","last_active":"2024-09-10T19:01:08.732332+0000","last_peered":"2024-09-10T19:01:08.732332+0000","last_clean":"2024-09-10T19:01:08.732332+0000","last_became_active":"2024-09-10T19:00:42.518566+0000","last_became_peered":"2024-09-10T19:00:42.518566+0000","last_unstale":"2024-09-10T19:01:08.732332+0000","last_undegraded":"2024-09-10T19:01:08.732332+0000","last_fullsized":"2024-09-10T19:01:08.732332+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:39.752722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:00:39.752722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:44.640996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":352,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:09.387189+0000","last_change":"2024-09-10T19:01:09.387164+0000","last_active":"2024-09-10T19:01:09.387189+0000","last_peered":"2024-09-10T19:01:09.387189+0000","last_clean":"2024-09-10T19:01:09.387189+0000","last_became_active":"2024-09-10T18:45:24.083689+0000","last_became_peered":"2024-09-10T18:45:24.083689+0000","last_unstale":"2024-09-10T19:01:09.387189+0000","last_undegraded":"2024-09-10T19:01:09.387189+0000","last_fullsized":"2024-09-10T19:01:09.387189+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:52.136095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:58:52.136095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.021731822000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"351'8252","reported_seq":8296,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:16.462382+0000","last_change":"2024-09-10T18:59:42.717031+0000","last_active":"2024-09-10T19:01:16.462382+0000","last_peered":"2024-09-10T19:01:16.462382+0000","last_clean":"2024-09-10T19:01:16.462382+0000","last_became_active":"2024-09-10T18:59:42.716730+0000","last_became_peered":"2024-09-10T18:59:42.716730+0000","last_unstale":"2024-09-10T19:01:16.462382+0000","last_undegraded":"2024-09-10T19:01:16.462382+0000","last_fullsized":"2024-09-10T19:01:16.462382+0000","mapping_epoch":315,"log_start":"336'4700","ondisk_log_start":"336'4700","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3552,"log_dups_size":0,"ondisk_log_size":3552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540803072,"num_objects":8252,"num_object_clones":0,"num_object_copies":24756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8252,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8252,"num_write_kb":528128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"232'33","reported_seq":444,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.730609+0000","last_change":"2024-09-10T18:59:17.515699+0000","last_active":"2024-09-10T19:01:08.730609+0000","last_peered":"2024-09-10T19:01:08.730609+0000","last_clean":"2024-09-10T19:01:08.730609+0000","last_became_active":"2024-09-10T18:57:02.435499+0000","last_became_peered":"2024-09-10T18:57:02.435499+0000","last_unstale":"2024-09-10T19:01:08.730609+0000","last_undegraded":"2024-09-10T19:01:08.730609+0000","last_fullsized":"2024-09-10T19:01:08.730609+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'33","last_scrub_stamp":"2024-09-10T18:59:17.515615+0000","last_deep_scrub":"232'33","last_deep_scrub_stamp":"2024-09-10T18:56:21.206960+0000","last_clean_scrub_stamp":"2024-09-10T18:59:17.515615+0000","objects_scrubbed":7,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.009416209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"60'2","reported_seq":365,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.490489+0000","last_change":"2024-09-10T18:58:55.931801+0000","last_active":"2024-09-10T19:01:15.490489+0000","last_peered":"2024-09-10T19:01:15.490489+0000","last_clean":"2024-09-10T19:01:15.490489+0000","last_became_active":"2024-09-10T18:45:17.961660+0000","last_became_peered":"2024-09-10T18:45:17.961660+0000","last_unstale":"2024-09-10T19:01:15.490489+0000","last_undegraded":"2024-09-10T19:01:15.490489+0000","last_fullsized":"2024-09-10T19:01:15.490489+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-10T18:58:55.931717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:58:55.931717+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040038909999999999,"objects_trimmed":0,"snaptrim_duration":0.0013942640000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":382,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.733365+0000","last_change":"2024-09-10T18:57:02.446028+0000","last_active":"2024-09-10T19:01:08.733365+0000","last_peered":"2024-09-10T19:01:08.733365+0000","last_clean":"2024-09-10T19:01:08.733365+0000","last_became_active":"2024-09-10T18:57:02.445712+0000","last_became_peered":"2024-09-10T18:57:02.445712+0000","last_unstale":"2024-09-10T19:01:08.733365+0000","last_undegraded":"2024-09-10T19:01:08.733365+0000","last_fullsized":"2024-09-10T19:01:08.733365+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:33.559651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:33.559651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0036787759999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'81","reported_seq":169,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.477421+0000","last_change":"2024-09-10T18:59:14.873293+0000","last_active":"2024-09-10T19:01:15.477421+0000","last_peered":"2024-09-10T19:01:15.477421+0000","last_clean":"2024-09-10T19:01:15.477421+0000","last_became_active":"2024-09-10T18:57:02.440063+0000","last_became_peered":"2024-09-10T18:57:02.440063+0000","last_unstale":"2024-09-10T19:01:15.477421+0000","last_undegraded":"2024-09-10T19:01:15.477421+0000","last_fullsized":"2024-09-10T19:01:15.477421+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:14.873083+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:56.085580+0000","last_clean_scrub_stamp":"2024-09-10T18:59:14.873083+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0063632760000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":351,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.732005+0000","last_change":"2024-09-10T18:57:02.432025+0000","last_active":"2024-09-10T19:01:08.732005+0000","last_peered":"2024-09-10T19:01:08.732005+0000","last_clean":"2024-09-10T19:01:08.732005+0000","last_became_active":"2024-09-10T18:57:02.431556+0000","last_became_peered":"2024-09-10T18:57:02.431556+0000","last_unstale":"2024-09-10T19:01:08.732005+0000","last_undegraded":"2024-09-10T19:01:08.732005+0000","last_fullsized":"2024-09-10T19:01:08.732005+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:27.012123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:27.012123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007606036,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":366,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.732945+0000","last_change":"2024-09-10T19:00:07.624756+0000","last_active":"2024-09-10T19:01:08.732945+0000","last_peered":"2024-09-10T19:01:08.732945+0000","last_clean":"2024-09-10T19:01:08.732945+0000","last_became_active":"2024-09-10T18:45:24.073327+0000","last_became_peered":"2024-09-10T18:45:24.073327+0000","last_unstale":"2024-09-10T19:01:08.732945+0000","last_undegraded":"2024-09-10T19:01:08.732945+0000","last_fullsized":"2024-09-10T19:01:08.732945+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:07.624701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:00:07.624701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:29.473904+0000","scrub_duration":0.001137962,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"352'8206","reported_seq":8251,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-10T19:01:18.726824+0000","last_change":"2024-09-10T18:59:43.009491+0000","last_active":"2024-09-10T19:01:18.726824+0000","last_peered":"2024-09-10T19:01:18.726824+0000","last_clean":"2024-09-10T19:01:18.726824+0000","last_became_active":"2024-09-10T18:59:43.009210+0000","last_became_peered":"2024-09-10T18:59:43.009210+0000","last_unstale":"2024-09-10T19:01:18.726824+0000","last_undegraded":"2024-09-10T19:01:18.726824+0000","last_fullsized":"2024-09-10T19:01:18.726824+0000","mapping_epoch":315,"log_start":"336'4500","ondisk_log_start":"336'4500","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3706,"log_dups_size":0,"ondisk_log_size":3706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":537788416,"num_objects":8206,"num_object_clones":0,"num_object_copies":24618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8206,"num_write_kb":525184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"70'81","reported_seq":361,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.730535+0000","last_change":"2024-09-10T18:58:59.582622+0000","last_active":"2024-09-10T19:01:08.730535+0000","last_peered":"2024-09-10T19:01:08.730535+0000","last_clean":"2024-09-10T19:01:08.730535+0000","last_became_active":"2024-09-10T18:52:16.598295+0000","last_became_peered":"2024-09-10T18:52:16.598295+0000","last_unstale":"2024-09-10T19:01:08.730535+0000","last_undegraded":"2024-09-10T19:01:08.730535+0000","last_fullsized":"2024-09-10T19:01:08.730535+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:58:59.582565+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:53.609885+0000","last_clean_scrub_stamp":"2024-09-10T18:58:59.582565+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046448920000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"232'35","reported_seq":459,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.727806+0000","last_change":"2024-09-10T19:00:43.819111+0000","last_active":"2024-09-10T19:01:08.727806+0000","last_peered":"2024-09-10T19:01:08.727806+0000","last_clean":"2024-09-10T19:01:08.727806+0000","last_became_active":"2024-09-10T18:57:02.436059+0000","last_became_peered":"2024-09-10T18:57:02.436059+0000","last_unstale":"2024-09-10T19:01:08.727806+0000","last_undegraded":"2024-09-10T19:01:08.727806+0000","last_fullsized":"2024-09-10T19:01:08.727806+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'35","last_scrub_stamp":"2024-09-10T19:00:43.819004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:00:43.819004+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:02:08.320454+0000","scrub_duration":0.0050234040000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":330,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.732077+0000","last_change":"2024-09-10T18:56:13.760318+0000","last_active":"2024-09-10T19:01:08.732077+0000","last_peered":"2024-09-10T19:01:08.732077+0000","last_clean":"2024-09-10T19:01:08.732077+0000","last_became_active":"2024-09-10T18:55:40.391849+0000","last_became_peered":"2024-09-10T18:55:40.391849+0000","last_unstale":"2024-09-10T19:01:08.732077+0000","last_undegraded":"2024-09-10T19:01:08.732077+0000","last_fullsized":"2024-09-10T19:01:08.732077+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:13.760238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:13.760238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.048619793000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":312,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.477275+0000","last_change":"2024-09-10T19:00:42.738386+0000","last_active":"2024-09-10T19:01:15.477275+0000","last_peered":"2024-09-10T19:01:15.477275+0000","last_clean":"2024-09-10T19:01:15.477275+0000","last_became_active":"2024-09-10T19:00:42.514709+0000","last_became_peered":"2024-09-10T19:00:42.514709+0000","last_unstale":"2024-09-10T19:01:15.477275+0000","last_undegraded":"2024-09-10T19:01:15.477275+0000","last_fullsized":"2024-09-10T19:01:15.477275+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:42.738331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:00:42.738331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:02:08.495978+0000","scrub_duration":0.001211314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":149,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.489718+0000","last_change":"2024-09-10T18:57:03.015513+0000","last_active":"2024-09-10T19:01:15.489718+0000","last_peered":"2024-09-10T19:01:15.489718+0000","last_clean":"2024-09-10T19:01:15.489718+0000","last_became_active":"2024-09-10T18:57:02.468866+0000","last_became_peered":"2024-09-10T18:57:02.468866+0000","last_unstale":"2024-09-10T19:01:15.489718+0000","last_undegraded":"2024-09-10T19:01:15.489718+0000","last_fullsized":"2024-09-10T19:01:15.489718+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:57:03.015457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:57:03.015457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001545795,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.5","version":"352'8214","reported_seq":18005,"reported_epoch":352,"state":"active+remapped+backfilling","last_fresh":"2024-09-10T19:01:18.721746+0000","last_change":"2024-09-10T19:00:43.858699+0000","last_active":"2024-09-10T19:01:18.721746+0000","last_peered":"2024-09-10T19:01:18.721746+0000","last_clean":"2024-09-10T19:00:41.500440+0000","last_became_active":"2024-09-10T19:00:43.839525+0000","last_became_peered":"2024-09-10T19:00:43.839525+0000","last_unstale":"2024-09-10T19:01:18.721746+0000","last_undegraded":"2024-09-10T19:01:18.721746+0000","last_fullsized":"2024-09-10T19:01:18.721746+0000","mapping_epoch":339,"log_start":"336'4500","ondisk_log_start":"336'4500","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3714,"log_dups_size":0,"ondisk_log_size":3714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":538312704,"num_objects":8214,"num_object_clones":0,"num_object_copies":24642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2236,"num_objects_unfound":0,"num_objects_dirty":8214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8214,"num_write_kb":525696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4867,"num_bytes_recovered":318963712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,4],"avail_no_missing":["0","3","4"],"object_location_counts":[{"shards":"0,3,4","objects":8214}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.e","version":"70'81","reported_seq":230,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.475641+0000","last_change":"2024-09-10T18:59:23.592755+0000","last_active":"2024-09-10T19:01:15.475641+0000","last_peered":"2024-09-10T19:01:15.475641+0000","last_clean":"2024-09-10T19:01:15.475641+0000","last_became_active":"2024-09-10T18:57:02.449115+0000","last_became_peered":"2024-09-10T18:57:02.449115+0000","last_unstale":"2024-09-10T19:01:15.475641+0000","last_undegraded":"2024-09-10T19:01:15.475641+0000","last_fullsized":"2024-09-10T19:01:15.475641+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:23.592672+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:59:23.592672+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038976470000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"232'19","reported_seq":368,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.478167+0000","last_change":"2024-09-10T19:00:40.748808+0000","last_active":"2024-09-10T19:01:15.478167+0000","last_peered":"2024-09-10T19:01:15.478167+0000","last_clean":"2024-09-10T19:01:15.478167+0000","last_became_active":"2024-09-10T18:55:40.395257+0000","last_became_peered":"2024-09-10T18:55:40.395257+0000","last_unstale":"2024-09-10T19:01:15.478167+0000","last_undegraded":"2024-09-10T19:01:15.478167+0000","last_fullsized":"2024-09-10T19:01:15.478167+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T19:00:40.748713+0000","last_deep_scrub":"130'13","last_deep_scrub_stamp":"2024-09-10T18:51:58.427534+0000","last_clean_scrub_stamp":"2024-09-10T19:00:40.748713+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:02:08.421967+0000","scrub_duration":0.0043426029999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":352,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.725501+0000","last_change":"2024-09-10T18:59:09.206400+0000","last_active":"2024-09-10T19:01:08.725501+0000","last_peered":"2024-09-10T19:01:08.725501+0000","last_clean":"2024-09-10T19:01:08.725501+0000","last_became_active":"2024-09-10T18:45:17.963057+0000","last_became_peered":"2024-09-10T18:45:17.963057+0000","last_unstale":"2024-09-10T19:01:08.725501+0000","last_undegraded":"2024-09-10T19:01:08.725501+0000","last_fullsized":"2024-09-10T19:01:08.725501+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:09.206317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:59:09.206317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.018986050000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":322,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.476374+0000","last_change":"2024-09-10T19:01:12.880399+0000","last_active":"2024-09-10T19:01:15.476374+0000","last_peered":"2024-09-10T19:01:15.476374+0000","last_clean":"2024-09-10T19:01:15.476374+0000","last_became_active":"2024-09-10T18:55:40.390771+0000","last_became_peered":"2024-09-10T18:55:40.390771+0000","last_unstale":"2024-09-10T19:01:15.476374+0000","last_undegraded":"2024-09-10T19:01:15.476374+0000","last_fullsized":"2024-09-10T19:01:15.476374+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:17.661832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_clean_scrub_stamp":"2024-09-10T18:56:17.661832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041460890000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":346,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.479788+0000","last_change":"2024-09-10T18:57:02.453927+0000","last_active":"2024-09-10T19:01:15.479788+0000","last_peered":"2024-09-10T19:01:15.479788+0000","last_clean":"2024-09-10T19:01:15.479788+0000","last_became_active":"2024-09-10T18:57:02.452387+0000","last_became_peered":"2024-09-10T18:57:02.452387+0000","last_unstale":"2024-09-10T19:01:15.479788+0000","last_undegraded":"2024-09-10T19:01:15.479788+0000","last_fullsized":"2024-09-10T19:01:15.479788+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:32.217641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:53.345184+0000","last_clean_scrub_stamp":"2024-09-10T18:56:32.217641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0059447470000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"351'8134","reported_seq":8178,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:16.523155+0000","last_change":"2024-09-10T18:59:42.715562+0000","last_active":"2024-09-10T19:01:16.523155+0000","last_peered":"2024-09-10T19:01:16.523155+0000","last_clean":"2024-09-10T19:01:16.523155+0000","last_became_active":"2024-09-10T18:59:42.715275+0000","last_became_peered":"2024-09-10T18:59:42.715275+0000","last_unstale":"2024-09-10T19:01:16.523155+0000","last_undegraded":"2024-09-10T19:01:16.523155+0000","last_fullsized":"2024-09-10T19:01:16.523155+0000","mapping_epoch":315,"log_start":"336'4544","ondisk_log_start":"336'4544","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3590,"log_dups_size":0,"ondisk_log_size":3590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":533069824,"num_objects":8134,"num_object_clones":0,"num_object_copies":24402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8134,"num_write_kb":520576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.d","version":"70'81","reported_seq":161,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.476707+0000","last_change":"2024-09-10T19:00:54.102663+0000","last_active":"2024-09-10T19:01:15.476707+0000","last_peered":"2024-09-10T19:01:15.476707+0000","last_clean":"2024-09-10T19:01:15.476707+0000","last_became_active":"2024-09-10T18:57:02.439652+0000","last_became_peered":"2024-09-10T18:57:02.439652+0000","last_unstale":"2024-09-10T19:01:15.476707+0000","last_undegraded":"2024-09-10T19:01:15.476707+0000","last_fullsized":"2024-09-10T19:01:15.476707+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:18.041116+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:18.041116+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014613810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"232'27","reported_seq":38,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.731704+0000","last_change":"2024-09-10T19:00:42.533625+0000","last_active":"2024-09-10T19:01:08.731704+0000","last_peered":"2024-09-10T19:01:08.731704+0000","last_clean":"2024-09-10T19:01:08.731704+0000","last_became_active":"2024-09-10T19:00:42.515903+0000","last_became_peered":"2024-09-10T19:00:42.515903+0000","last_unstale":"2024-09-10T19:01:08.731704+0000","last_undegraded":"2024-09-10T19:01:08.731704+0000","last_fullsized":"2024-09-10T19:01:08.731704+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'27","last_scrub_stamp":"2024-09-10T18:58:47.595256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:58:47.595256+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014176382,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":33,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":313,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.491003+0000","last_change":"2024-09-10T18:57:02.463092+0000","last_active":"2024-09-10T19:01:15.491003+0000","last_peered":"2024-09-10T19:01:15.491003+0000","last_clean":"2024-09-10T19:01:15.491003+0000","last_became_active":"2024-09-10T18:57:02.461708+0000","last_became_peered":"2024-09-10T18:57:02.461708+0000","last_unstale":"2024-09-10T19:01:15.491003+0000","last_undegraded":"2024-09-10T19:01:15.491003+0000","last_fullsized":"2024-09-10T19:01:15.491003+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:24.902205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:05.603125+0000","last_clean_scrub_stamp":"2024-09-10T18:56:24.902205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0076874830000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"273'10","reported_seq":226,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.480576+0000","last_change":"2024-09-10T18:59:11.568683+0000","last_active":"2024-09-10T19:01:15.480576+0000","last_peered":"2024-09-10T19:01:15.480576+0000","last_clean":"2024-09-10T19:01:15.480576+0000","last_became_active":"2024-09-10T18:57:02.441890+0000","last_became_peered":"2024-09-10T18:57:02.441890+0000","last_unstale":"2024-09-10T19:01:15.480576+0000","last_undegraded":"2024-09-10T19:01:15.480576+0000","last_fullsized":"2024-09-10T19:01:15.480576+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'10","last_scrub_stamp":"2024-09-10T18:59:11.568599+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-09-10T18:47:02.582229+0000","last_clean_scrub_stamp":"2024-09-10T18:59:11.568599+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0054658950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":373,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.732790+0000","last_change":"2024-09-10T18:59:51.487816+0000","last_active":"2024-09-10T19:01:08.732790+0000","last_peered":"2024-09-10T19:01:08.732790+0000","last_clean":"2024-09-10T19:01:08.732790+0000","last_became_active":"2024-09-10T18:55:40.389202+0000","last_became_peered":"2024-09-10T18:55:40.389202+0000","last_unstale":"2024-09-10T19:01:08.732790+0000","last_undegraded":"2024-09-10T19:01:08.732790+0000","last_fullsized":"2024-09-10T19:01:08.732790+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:51.487763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_clean_scrub_stamp":"2024-09-10T18:59:51.487763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001471904,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"351'8121","reported_seq":20259,"reported_epoch":351,"state":"active+remapped+backfilling","last_fresh":"2024-09-10T19:01:16.487028+0000","last_change":"2024-09-10T19:00:43.876105+0000","last_active":"2024-09-10T19:01:16.487028+0000","last_peered":"2024-09-10T19:01:16.487028+0000","last_clean":"2024-09-10T19:00:41.508090+0000","last_became_active":"2024-09-10T19:00:43.837692+0000","last_became_peered":"2024-09-10T19:00:43.837692+0000","last_unstale":"2024-09-10T19:01:16.487028+0000","last_undegraded":"2024-09-10T19:01:16.487028+0000","last_fullsized":"2024-09-10T19:01:16.487028+0000","mapping_epoch":339,"log_start":"336'4600","ondisk_log_start":"336'4600","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3521,"log_dups_size":0,"ondisk_log_size":3521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":532217856,"num_objects":8121,"num_object_clones":0,"num_object_copies":24363,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":858,"num_objects_unfound":0,"num_objects_dirty":8121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8121,"num_write_kb":519744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6039,"num_bytes_recovered":395771904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,4],"avail_no_missing":["6","3","4"],"object_location_counts":[{"shards":"3,4,6","objects":8121}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.c","version":"70'81","reported_seq":306,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.490893+0000","last_change":"2024-09-10T18:56:33.779212+0000","last_active":"2024-09-10T19:01:15.490893+0000","last_peered":"2024-09-10T19:01:15.490893+0000","last_clean":"2024-09-10T19:01:15.490893+0000","last_became_active":"2024-09-10T18:55:40.390655+0000","last_became_peered":"2024-09-10T18:55:40.390655+0000","last_unstale":"2024-09-10T19:01:15.490893+0000","last_undegraded":"2024-09-10T19:01:15.490893+0000","last_fullsized":"2024-09-10T19:01:15.490893+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:33.779154+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:56:33.779154+0000","last_clean_scrub_stamp":"2024-09-10T18:56:33.779154+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0075121179999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"232'18","reported_seq":384,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.475972+0000","last_change":"2024-09-10T19:00:51.009315+0000","last_active":"2024-09-10T19:01:15.475972+0000","last_peered":"2024-09-10T19:01:15.475972+0000","last_clean":"2024-09-10T19:01:15.475972+0000","last_became_active":"2024-09-10T18:45:20.056869+0000","last_became_peered":"2024-09-10T18:45:20.056869+0000","last_unstale":"2024-09-10T19:01:15.475972+0000","last_undegraded":"2024-09-10T19:01:15.475972+0000","last_fullsized":"2024-09-10T19:01:15.475972+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'18","last_scrub_stamp":"2024-09-10T18:55:27.507364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:55:27.507364+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030221390000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"56'2","reported_seq":394,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.477530+0000","last_change":"2024-09-10T19:01:01.185585+0000","last_active":"2024-09-10T19:01:15.477530+0000","last_peered":"2024-09-10T19:01:15.477530+0000","last_clean":"2024-09-10T19:01:15.477530+0000","last_became_active":"2024-09-10T18:57:02.434475+0000","last_became_peered":"2024-09-10T18:57:02.434475+0000","last_unstale":"2024-09-10T19:01:15.477530+0000","last_undegraded":"2024-09-10T19:01:15.477530+0000","last_fullsized":"2024-09-10T19:01:15.477530+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-10T18:56:01.630091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:01.630091+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001578238,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":312,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.478054+0000","last_change":"2024-09-10T19:00:42.515054+0000","last_active":"2024-09-10T19:01:15.478054+0000","last_peered":"2024-09-10T19:01:15.478054+0000","last_clean":"2024-09-10T19:01:15.478054+0000","last_became_active":"2024-09-10T19:00:42.513796+0000","last_became_peered":"2024-09-10T19:00:42.513796+0000","last_unstale":"2024-09-10T19:01:15.478054+0000","last_undegraded":"2024-09-10T19:01:15.478054+0000","last_fullsized":"2024-09-10T19:01:15.478054+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:54.899134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:02.711594+0000","last_clean_scrub_stamp":"2024-09-10T18:59:54.899134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051739739999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":356,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.490445+0000","last_change":"2024-09-10T18:59:11.827181+0000","last_active":"2024-09-10T19:01:15.490445+0000","last_peered":"2024-09-10T19:01:15.490445+0000","last_clean":"2024-09-10T19:01:15.490445+0000","last_became_active":"2024-09-10T18:57:02.459985+0000","last_became_peered":"2024-09-10T18:57:02.459985+0000","last_unstale":"2024-09-10T19:01:15.490445+0000","last_undegraded":"2024-09-10T19:01:15.490445+0000","last_fullsized":"2024-09-10T19:01:15.490445+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:11.827101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:53.352427+0000","last_clean_scrub_stamp":"2024-09-10T18:59:11.827101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034557899999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.13","version":"70'81","reported_seq":281,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.725059+0000","last_change":"2024-09-10T19:00:49.472063+0000","last_active":"2024-09-10T19:01:08.725059+0000","last_peered":"2024-09-10T19:01:08.725059+0000","last_clean":"2024-09-10T19:01:08.725059+0000","last_became_active":"2024-09-10T18:52:16.602463+0000","last_became_peered":"2024-09-10T18:52:16.602463+0000","last_unstale":"2024-09-10T19:01:08.725059+0000","last_undegraded":"2024-09-10T19:01:08.725059+0000","last_fullsized":"2024-09-10T19:01:08.725059+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:34.760031+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:34.760031+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002735595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"232'19","reported_seq":381,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.480247+0000","last_change":"2024-09-10T18:57:02.485836+0000","last_active":"2024-09-10T19:01:15.480247+0000","last_peered":"2024-09-10T19:01:15.480247+0000","last_clean":"2024-09-10T19:01:15.480247+0000","last_became_active":"2024-09-10T18:57:02.453001+0000","last_became_peered":"2024-09-10T18:57:02.453001+0000","last_unstale":"2024-09-10T19:01:15.480247+0000","last_undegraded":"2024-09-10T19:01:15.480247+0000","last_fullsized":"2024-09-10T19:01:15.480247+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T18:56:25.122070+0000","last_deep_scrub":"76'13","last_deep_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_clean_scrub_stamp":"2024-09-10T18:56:25.122070+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0061142590000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":340,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.727340+0000","last_change":"2024-09-10T18:59:59.385405+0000","last_active":"2024-09-10T19:01:08.727340+0000","last_peered":"2024-09-10T19:01:08.727340+0000","last_clean":"2024-09-10T19:01:08.727340+0000","last_became_active":"2024-09-10T18:55:40.401037+0000","last_became_peered":"2024-09-10T18:55:40.401037+0000","last_unstale":"2024-09-10T19:01:08.727340+0000","last_undegraded":"2024-09-10T19:01:08.727340+0000","last_fullsized":"2024-09-10T19:01:08.727340+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:59.385349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:59:59.385349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00163496,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":358,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.492452+0000","last_change":"2024-09-10T18:57:02.455638+0000","last_active":"2024-09-10T19:01:15.492452+0000","last_peered":"2024-09-10T19:01:15.492452+0000","last_clean":"2024-09-10T19:01:15.492452+0000","last_became_active":"2024-09-10T18:57:02.455341+0000","last_became_peered":"2024-09-10T18:57:02.455341+0000","last_unstale":"2024-09-10T19:01:15.492452+0000","last_undegraded":"2024-09-10T19:01:15.492452+0000","last_fullsized":"2024-09-10T19:01:15.492452+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:25.862853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:00.754912+0000","last_clean_scrub_stamp":"2024-09-10T18:56:25.862853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0033952399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"63'1","reported_seq":380,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.476344+0000","last_change":"2024-09-10T19:01:08.024679+0000","last_active":"2024-09-10T19:01:15.476344+0000","last_peered":"2024-09-10T19:01:15.476344+0000","last_clean":"2024-09-10T19:01:15.476344+0000","last_became_active":"2024-09-10T18:59:48.741275+0000","last_became_peered":"2024-09-10T18:59:48.741275+0000","last_unstale":"2024-09-10T19:01:15.476344+0000","last_undegraded":"2024-09-10T19:01:15.476344+0000","last_fullsized":"2024-09-10T19:01:15.476344+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-10T18:56:07.774334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:07.774334+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.090386770000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":52,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.12","version":"70'81","reported_seq":220,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.732613+0000","last_change":"2024-09-10T18:58:52.543604+0000","last_active":"2024-09-10T19:01:08.732613+0000","last_peered":"2024-09-10T19:01:08.732613+0000","last_clean":"2024-09-10T19:01:08.732613+0000","last_became_active":"2024-09-10T18:52:16.598921+0000","last_became_peered":"2024-09-10T18:52:16.598921+0000","last_unstale":"2024-09-10T19:01:08.732613+0000","last_undegraded":"2024-09-10T19:01:08.732613+0000","last_fullsized":"2024-09-10T19:01:08.732613+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:58:52.543544+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:54.127476+0000","last_clean_scrub_stamp":"2024-09-10T18:58:52.543544+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003920644,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":386,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.727793+0000","last_change":"2024-09-10T18:59:05.556377+0000","last_active":"2024-09-10T19:01:08.727793+0000","last_peered":"2024-09-10T19:01:08.727793+0000","last_clean":"2024-09-10T19:01:08.727793+0000","last_became_active":"2024-09-10T18:57:02.442251+0000","last_became_peered":"2024-09-10T18:57:02.442251+0000","last_unstale":"2024-09-10T19:01:08.727793+0000","last_undegraded":"2024-09-10T19:01:08.727793+0000","last_fullsized":"2024-09-10T19:01:08.727793+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:05.556292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:59:05.556292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026781280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"232'12","reported_seq":389,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.492662+0000","last_change":"2024-09-10T18:59:52.198399+0000","last_active":"2024-09-10T19:01:15.492662+0000","last_peered":"2024-09-10T19:01:15.492662+0000","last_clean":"2024-09-10T19:01:15.492662+0000","last_became_active":"2024-09-10T18:57:02.460430+0000","last_became_peered":"2024-09-10T18:57:02.460430+0000","last_unstale":"2024-09-10T19:01:15.492662+0000","last_undegraded":"2024-09-10T19:01:15.492662+0000","last_fullsized":"2024-09-10T19:01:15.492662+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'12","last_scrub_stamp":"2024-09-10T18:59:52.198340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:59:52.198340+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001790591,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":348,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.733341+0000","last_change":"2024-09-10T18:56:32.602499+0000","last_active":"2024-09-10T19:01:08.733341+0000","last_peered":"2024-09-10T19:01:08.733341+0000","last_clean":"2024-09-10T19:01:08.733341+0000","last_became_active":"2024-09-10T18:45:22.069653+0000","last_became_peered":"2024-09-10T18:45:22.069653+0000","last_unstale":"2024-09-10T19:01:08.733341+0000","last_undegraded":"2024-09-10T19:01:08.733341+0000","last_fullsized":"2024-09-10T19:01:08.733341+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:32.602448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:32.602448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0052503430000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":113,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.730498+0000","last_change":"2024-09-10T19:00:42.519802+0000","last_active":"2024-09-10T19:01:08.730498+0000","last_peered":"2024-09-10T19:01:08.730498+0000","last_clean":"2024-09-10T19:01:08.730498+0000","last_became_active":"2024-09-10T19:00:42.519492+0000","last_became_peered":"2024-09-10T19:00:42.519492+0000","last_unstale":"2024-09-10T19:01:08.730498+0000","last_undegraded":"2024-09-10T19:01:08.730498+0000","last_fullsized":"2024-09-10T19:01:08.730498+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:53.608029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:58:53.608029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058174170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.11","version":"70'81","reported_seq":256,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.479459+0000","last_change":"2024-09-10T19:00:42.517609+0000","last_active":"2024-09-10T19:01:15.479459+0000","last_peered":"2024-09-10T19:01:15.479459+0000","last_clean":"2024-09-10T19:01:15.479459+0000","last_became_active":"2024-09-10T19:00:42.517311+0000","last_became_peered":"2024-09-10T19:00:42.517311+0000","last_unstale":"2024-09-10T19:01:15.479459+0000","last_undegraded":"2024-09-10T19:01:15.479459+0000","last_fullsized":"2024-09-10T19:01:15.479459+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:58:48.552631+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:58:48.552631+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012534919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"232'20","reported_seq":446,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.490498+0000","last_change":"2024-09-10T19:00:07.147063+0000","last_active":"2024-09-10T19:01:15.490498+0000","last_peered":"2024-09-10T19:01:15.490498+0000","last_clean":"2024-09-10T19:01:15.490498+0000","last_became_active":"2024-09-10T18:57:02.444437+0000","last_became_peered":"2024-09-10T18:57:02.444437+0000","last_unstale":"2024-09-10T19:01:15.490498+0000","last_undegraded":"2024-09-10T19:01:15.490498+0000","last_fullsized":"2024-09-10T19:01:15.490498+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'20","last_scrub_stamp":"2024-09-10T19:00:07.146928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:00:07.146928+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:22.782553+0000","scrub_duration":0.0071404040000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":372,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.732345+0000","last_change":"2024-09-10T18:58:50.541776+0000","last_active":"2024-09-10T19:01:08.732345+0000","last_peered":"2024-09-10T19:01:08.732345+0000","last_clean":"2024-09-10T19:01:08.732345+0000","last_became_active":"2024-09-10T18:57:02.445710+0000","last_became_peered":"2024-09-10T18:57:02.445710+0000","last_unstale":"2024-09-10T19:01:08.732345+0000","last_undegraded":"2024-09-10T19:01:08.732345+0000","last_fullsized":"2024-09-10T19:01:08.732345+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:50.541691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:58:50.541691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0089012770000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"304'11","reported_seq":592,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.479764+0000","last_change":"2024-09-10T18:59:55.525278+0000","last_active":"2024-09-10T19:01:15.479764+0000","last_peered":"2024-09-10T19:01:15.479764+0000","last_clean":"2024-09-10T19:01:15.479764+0000","last_became_active":"2024-09-10T18:58:58.489370+0000","last_became_peered":"2024-09-10T18:58:58.489370+0000","last_unstale":"2024-09-10T19:01:15.479764+0000","last_undegraded":"2024-09-10T19:01:15.479764+0000","last_fullsized":"2024-09-10T19:01:15.479764+0000","mapping_epoch":303,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":304,"parent":"0.0","parent_split_bits":0,"last_scrub":"304'11","last_scrub_stamp":"2024-09-10T18:59:55.525185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:59:55.525185+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020778110000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":41,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.478681+0000","last_change":"2024-09-10T18:59:48.745646+0000","last_active":"2024-09-10T19:01:15.478681+0000","last_peered":"2024-09-10T19:01:15.478681+0000","last_clean":"2024-09-10T19:01:15.478681+0000","last_became_active":"2024-09-10T18:59:48.744923+0000","last_became_peered":"2024-09-10T18:59:48.744923+0000","last_unstale":"2024-09-10T19:01:15.478681+0000","last_undegraded":"2024-09-10T19:01:15.478681+0000","last_fullsized":"2024-09-10T19:01:15.478681+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:57.812395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:57.476724+0000","last_clean_scrub_stamp":"2024-09-10T18:58:57.812395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.10","version":"70'81","reported_seq":230,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.731845+0000","last_change":"2024-09-10T19:00:04.612412+0000","last_active":"2024-09-10T19:01:08.731845+0000","last_peered":"2024-09-10T19:01:08.731845+0000","last_clean":"2024-09-10T19:01:08.731845+0000","last_became_active":"2024-09-10T18:57:02.443782+0000","last_became_peered":"2024-09-10T18:57:02.443782+0000","last_unstale":"2024-09-10T19:01:08.731845+0000","last_undegraded":"2024-09-10T19:01:08.731845+0000","last_fullsized":"2024-09-10T19:01:08.731845+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:00:04.612324+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:00:04.612324+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:27.149851+0000","scrub_duration":0.0025961159999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"232'15","reported_seq":32,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.732352+0000","last_change":"2024-09-10T19:00:42.530883+0000","last_active":"2024-09-10T19:01:08.732352+0000","last_peered":"2024-09-10T19:01:08.732352+0000","last_clean":"2024-09-10T19:01:08.732352+0000","last_became_active":"2024-09-10T19:00:42.518672+0000","last_became_peered":"2024-09-10T19:00:42.518672+0000","last_unstale":"2024-09-10T19:01:08.732352+0000","last_undegraded":"2024-09-10T19:01:08.732352+0000","last_fullsized":"2024-09-10T19:01:08.732352+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T18:59:16.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:59:16.949954+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":330,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.477352+0000","last_change":"2024-09-10T18:56:23.096461+0000","last_active":"2024-09-10T19:01:15.477352+0000","last_peered":"2024-09-10T19:01:15.477352+0000","last_clean":"2024-09-10T19:01:15.477352+0000","last_became_active":"2024-09-10T18:55:40.390697+0000","last_became_peered":"2024-09-10T18:55:40.390697+0000","last_unstale":"2024-09-10T19:01:15.477352+0000","last_undegraded":"2024-09-10T19:01:15.477352+0000","last_fullsized":"2024-09-10T19:01:15.477352+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:23.096373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:23.096373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0057615890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"273'11","reported_seq":577,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:16.263538+0000","last_change":"2024-09-10T18:59:53.158730+0000","last_active":"2024-09-10T19:01:16.263538+0000","last_peered":"2024-09-10T19:01:16.263538+0000","last_clean":"2024-09-10T19:01:16.263538+0000","last_became_active":"2024-09-10T18:57:02.445268+0000","last_became_peered":"2024-09-10T18:57:02.445268+0000","last_unstale":"2024-09-10T19:01:16.263538+0000","last_undegraded":"2024-09-10T19:01:16.263538+0000","last_fullsized":"2024-09-10T19:01:16.263538+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'11","last_scrub_stamp":"2024-09-10T18:59:53.158664+0000","last_deep_scrub":"81'9","last_deep_scrub_stamp":"2024-09-10T18:46:54.800790+0000","last_clean_scrub_stamp":"2024-09-10T18:59:53.158664+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017427930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":372,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.476243+0000","last_change":"2024-09-10T18:56:20.584899+0000","last_active":"2024-09-10T19:01:15.476243+0000","last_peered":"2024-09-10T19:01:15.476243+0000","last_clean":"2024-09-10T19:01:15.476243+0000","last_became_active":"2024-09-10T18:55:40.391662+0000","last_became_peered":"2024-09-10T18:55:40.391662+0000","last_unstale":"2024-09-10T19:01:15.476243+0000","last_undegraded":"2024-09-10T19:01:15.476243+0000","last_fullsized":"2024-09-10T19:01:15.476243+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:20.584808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_clean_scrub_stamp":"2024-09-10T18:56:20.584808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0035961249999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"232'15","reported_seq":424,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.724595+0000","last_change":"2024-09-10T18:59:53.320831+0000","last_active":"2024-09-10T19:01:08.724595+0000","last_peered":"2024-09-10T19:01:08.724595+0000","last_clean":"2024-09-10T19:01:08.724595+0000","last_became_active":"2024-09-10T18:55:40.400788+0000","last_became_peered":"2024-09-10T18:55:40.400788+0000","last_unstale":"2024-09-10T19:01:08.724595+0000","last_undegraded":"2024-09-10T19:01:08.724595+0000","last_fullsized":"2024-09-10T19:01:08.724595+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T18:59:53.320757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:59:53.320757+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001889578,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":403,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.728654+0000","last_change":"2024-09-10T19:00:42.518606+0000","last_active":"2024-09-10T19:01:08.728654+0000","last_peered":"2024-09-10T19:01:08.728654+0000","last_clean":"2024-09-10T19:01:08.728654+0000","last_became_active":"2024-09-10T19:00:42.518310+0000","last_became_peered":"2024-09-10T19:00:42.518310+0000","last_unstale":"2024-09-10T19:01:08.728654+0000","last_undegraded":"2024-09-10T19:01:08.728654+0000","last_fullsized":"2024-09-10T19:01:08.728654+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:48.622701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:59:48.622701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001531279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":355,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.490644+0000","last_change":"2024-09-10T18:56:04.753218+0000","last_active":"2024-09-10T19:01:15.490644+0000","last_peered":"2024-09-10T19:01:15.490644+0000","last_clean":"2024-09-10T19:01:15.490644+0000","last_became_active":"2024-09-10T18:55:40.398243+0000","last_became_peered":"2024-09-10T18:55:40.398243+0000","last_unstale":"2024-09-10T19:01:15.490644+0000","last_undegraded":"2024-09-10T19:01:15.490644+0000","last_fullsized":"2024-09-10T19:01:15.490644+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:04.753133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:03.571786+0000","last_clean_scrub_stamp":"2024-09-10T18:56:04.753133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0069575230000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":345,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.478227+0000","last_change":"2024-09-10T19:00:42.517459+0000","last_active":"2024-09-10T19:01:15.478227+0000","last_peered":"2024-09-10T19:01:15.478227+0000","last_clean":"2024-09-10T19:01:15.478227+0000","last_became_active":"2024-09-10T19:00:42.516833+0000","last_became_peered":"2024-09-10T19:00:42.516833+0000","last_unstale":"2024-09-10T19:01:15.478227+0000","last_undegraded":"2024-09-10T19:01:15.478227+0000","last_fullsized":"2024-09-10T19:01:15.478227+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:46.863758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:04.631558+0000","last_clean_scrub_stamp":"2024-09-10T18:59:46.863758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0031907379999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"232'24","reported_seq":385,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.475848+0000","last_change":"2024-09-10T18:58:58.698097+0000","last_active":"2024-09-10T19:01:15.475848+0000","last_peered":"2024-09-10T19:01:15.475848+0000","last_clean":"2024-09-10T19:01:15.475848+0000","last_became_active":"2024-09-10T18:45:20.056885+0000","last_became_peered":"2024-09-10T18:45:20.056885+0000","last_unstale":"2024-09-10T19:01:15.475848+0000","last_undegraded":"2024-09-10T19:01:15.475848+0000","last_fullsized":"2024-09-10T19:01:15.475848+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T18:58:58.698003+0000","last_deep_scrub":"223'16","last_deep_scrub_stamp":"2024-09-10T18:54:47.324044+0000","last_clean_scrub_stamp":"2024-09-10T18:58:58.698003+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.006955676,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":372,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.725424+0000","last_change":"2024-09-10T19:01:07.432016+0000","last_active":"2024-09-10T19:01:08.725424+0000","last_peered":"2024-09-10T19:01:08.725424+0000","last_clean":"2024-09-10T19:01:08.725424+0000","last_became_active":"2024-09-10T18:55:40.399198+0000","last_became_peered":"2024-09-10T18:55:40.399198+0000","last_unstale":"2024-09-10T19:01:08.725424+0000","last_undegraded":"2024-09-10T19:01:08.725424+0000","last_fullsized":"2024-09-10T19:01:08.725424+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:51.138240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:00.029389+0000","last_clean_scrub_stamp":"2024-09-10T18:58:51.138240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0086738059999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":354,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.724436+0000","last_change":"2024-09-10T19:00:59.501368+0000","last_active":"2024-09-10T19:01:08.724436+0000","last_peered":"2024-09-10T19:01:08.724436+0000","last_clean":"2024-09-10T19:01:08.724436+0000","last_became_active":"2024-09-10T18:45:22.070065+0000","last_became_peered":"2024-09-10T18:45:22.070065+0000","last_unstale":"2024-09-10T19:01:08.724436+0000","last_undegraded":"2024-09-10T19:01:08.724436+0000","last_fullsized":"2024-09-10T19:01:08.724436+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:31.782918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:31.782918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043032130000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":353,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.492217+0000","last_change":"2024-09-10T18:56:06.792096+0000","last_active":"2024-09-10T19:01:15.492217+0000","last_peered":"2024-09-10T19:01:15.492217+0000","last_clean":"2024-09-10T19:01:15.492217+0000","last_became_active":"2024-09-10T18:55:40.395595+0000","last_became_peered":"2024-09-10T18:55:40.395595+0000","last_unstale":"2024-09-10T19:01:15.492217+0000","last_undegraded":"2024-09-10T19:01:15.492217+0000","last_fullsized":"2024-09-10T19:01:15.492217+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:06.791860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:06.791860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046852380000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"232'8","reported_seq":362,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.489872+0000","last_change":"2024-09-10T18:56:30.802774+0000","last_active":"2024-09-10T19:01:15.489872+0000","last_peered":"2024-09-10T19:01:15.489872+0000","last_clean":"2024-09-10T19:01:15.489872+0000","last_became_active":"2024-09-10T18:45:20.059205+0000","last_became_peered":"2024-09-10T18:45:20.059205+0000","last_unstale":"2024-09-10T19:01:15.489872+0000","last_undegraded":"2024-09-10T19:01:15.489872+0000","last_fullsized":"2024-09-10T19:01:15.489872+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'8","last_scrub_stamp":"2024-09-10T18:56:30.802679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:30.802679+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034579609999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":368,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.733416+0000","last_change":"2024-09-10T18:58:53.527829+0000","last_active":"2024-09-10T19:01:08.733416+0000","last_peered":"2024-09-10T19:01:08.733416+0000","last_clean":"2024-09-10T19:01:08.733416+0000","last_became_active":"2024-09-10T18:47:36.415866+0000","last_became_peered":"2024-09-10T18:47:36.415866+0000","last_unstale":"2024-09-10T19:01:08.733416+0000","last_undegraded":"2024-09-10T19:01:08.733416+0000","last_fullsized":"2024-09-10T19:01:08.733416+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:53.527746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:58:53.527746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058191349999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":379,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.727502+0000","last_change":"2024-09-10T18:57:02.454548+0000","last_active":"2024-09-10T19:01:08.727502+0000","last_peered":"2024-09-10T19:01:08.727502+0000","last_clean":"2024-09-10T19:01:08.727502+0000","last_became_active":"2024-09-10T18:57:02.453974+0000","last_became_peered":"2024-09-10T18:57:02.453974+0000","last_unstale":"2024-09-10T19:01:08.727502+0000","last_undegraded":"2024-09-10T19:01:08.727502+0000","last_fullsized":"2024-09-10T19:01:08.727502+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:23.195060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_clean_scrub_stamp":"2024-09-10T18:56:23.195060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047613259999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":150,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.476160+0000","last_change":"2024-09-10T18:59:55.812370+0000","last_active":"2024-09-10T19:01:15.476160+0000","last_peered":"2024-09-10T19:01:15.476160+0000","last_clean":"2024-09-10T19:01:15.476160+0000","last_became_active":"2024-09-10T18:57:02.447566+0000","last_became_peered":"2024-09-10T18:57:02.447566+0000","last_unstale":"2024-09-10T19:01:15.476160+0000","last_undegraded":"2024-09-10T19:01:15.476160+0000","last_fullsized":"2024-09-10T19:01:15.476160+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:55.812312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:55.145638+0000","last_clean_scrub_stamp":"2024-09-10T18:59:55.812312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0055522669999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.14","version":"70'81","reported_seq":254,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.476542+0000","last_change":"2024-09-10T19:00:58.100610+0000","last_active":"2024-09-10T19:01:15.476542+0000","last_peered":"2024-09-10T19:01:15.476542+0000","last_clean":"2024-09-10T19:01:15.476542+0000","last_became_active":"2024-09-10T18:57:02.442457+0000","last_became_peered":"2024-09-10T18:57:02.442457+0000","last_unstale":"2024-09-10T19:01:15.476542+0000","last_undegraded":"2024-09-10T19:01:15.476542+0000","last_fullsized":"2024-09-10T19:01:15.476542+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:55:34.791223+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:57.184383+0000","last_clean_scrub_stamp":"2024-09-10T18:55:34.791223+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016709736999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17890,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":7,"num_write":3,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":35780,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.11","version":"232'24","reported_seq":408,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.491845+0000","last_change":"2024-09-10T18:59:10.879686+0000","last_active":"2024-09-10T19:01:15.491845+0000","last_peered":"2024-09-10T19:01:15.491845+0000","last_clean":"2024-09-10T19:01:15.491845+0000","last_became_active":"2024-09-10T18:57:02.461909+0000","last_became_peered":"2024-09-10T18:57:02.461909+0000","last_unstale":"2024-09-10T19:01:15.491845+0000","last_undegraded":"2024-09-10T19:01:15.491845+0000","last_fullsized":"2024-09-10T19:01:15.491845+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T18:59:10.879597+0000","last_deep_scrub":"79'16","last_deep_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_clean_scrub_stamp":"2024-09-10T18:59:10.879597+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0090530790000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"56'1","reported_seq":384,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.730388+0000","last_change":"2024-09-10T18:56:25.211179+0000","last_active":"2024-09-10T19:01:08.730388+0000","last_peered":"2024-09-10T19:01:08.730388+0000","last_clean":"2024-09-10T19:01:08.730388+0000","last_became_active":"2024-09-10T18:47:36.421985+0000","last_became_peered":"2024-09-10T18:47:36.421985+0000","last_unstale":"2024-09-10T19:01:08.730388+0000","last_undegraded":"2024-09-10T19:01:08.730388+0000","last_fullsized":"2024-09-10T19:01:08.730388+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:56:25.211095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:25.211095+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025468880000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":97,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.475502+0000","last_change":"2024-09-10T19:01:07.015846+0000","last_active":"2024-09-10T19:01:15.475502+0000","last_peered":"2024-09-10T19:01:15.475502+0000","last_clean":"2024-09-10T19:01:15.475502+0000","last_became_active":"2024-09-10T18:57:02.435124+0000","last_became_peered":"2024-09-10T18:57:02.435124+0000","last_unstale":"2024-09-10T19:01:15.475502+0000","last_undegraded":"2024-09-10T19:01:15.475502+0000","last_fullsized":"2024-09-10T19:01:15.475502+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:12.852769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:12.852769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":363,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.490601+0000","last_change":"2024-09-10T18:56:09.903930+0000","last_active":"2024-09-10T19:01:15.490601+0000","last_peered":"2024-09-10T19:01:15.490601+0000","last_clean":"2024-09-10T19:01:15.490601+0000","last_became_active":"2024-09-10T18:55:40.397523+0000","last_became_peered":"2024-09-10T18:55:40.397523+0000","last_unstale":"2024-09-10T19:01:15.490601+0000","last_undegraded":"2024-09-10T19:01:15.490601+0000","last_fullsized":"2024-09-10T19:01:15.490601+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:09.903846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:09.903846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011721984,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"78'2","reported_seq":139,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.731795+0000","last_change":"2024-09-10T18:58:48.472078+0000","last_active":"2024-09-10T19:01:08.731795+0000","last_peered":"2024-09-10T19:01:08.731795+0000","last_clean":"2024-09-10T19:01:08.731795+0000","last_became_active":"2024-09-10T18:57:02.447198+0000","last_became_peered":"2024-09-10T18:57:02.447198+0000","last_unstale":"2024-09-10T19:01:08.731795+0000","last_undegraded":"2024-09-10T19:01:08.731795+0000","last_fullsized":"2024-09-10T19:01:08.731795+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'2","last_scrub_stamp":"2024-09-10T18:58:48.472024+0000","last_deep_scrub":"78'2","last_deep_scrub_stamp":"2024-09-10T18:46:37.581130+0000","last_clean_scrub_stamp":"2024-09-10T18:58:48.472024+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.022088824,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":372,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.731962+0000","last_change":"2024-09-10T18:59:53.505658+0000","last_active":"2024-09-10T19:01:08.731962+0000","last_peered":"2024-09-10T19:01:08.731962+0000","last_clean":"2024-09-10T19:01:08.731962+0000","last_became_active":"2024-09-10T18:57:02.443353+0000","last_became_peered":"2024-09-10T18:57:02.443353+0000","last_unstale":"2024-09-10T19:01:08.731962+0000","last_undegraded":"2024-09-10T19:01:08.731962+0000","last_fullsized":"2024-09-10T19:01:08.731962+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:53.505561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:59:53.505561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:19.130152+0000","scrub_duration":0.001238605,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"232'20","reported_seq":379,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.479073+0000","last_change":"2024-09-10T18:57:02.486956+0000","last_active":"2024-09-10T19:01:15.479073+0000","last_peered":"2024-09-10T19:01:15.479073+0000","last_clean":"2024-09-10T19:01:15.479073+0000","last_became_active":"2024-09-10T18:57:02.452473+0000","last_became_peered":"2024-09-10T18:57:02.452473+0000","last_unstale":"2024-09-10T19:01:15.479073+0000","last_undegraded":"2024-09-10T19:01:15.479073+0000","last_fullsized":"2024-09-10T19:01:15.479073+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'20","last_scrub_stamp":"2024-09-10T18:56:27.158686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:27.158686+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0090371180000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":23,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"273'10","reported_seq":587,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.487913+0000","last_change":"2024-09-10T18:59:22.516714+0000","last_active":"2024-09-10T19:01:15.487913+0000","last_peered":"2024-09-10T19:01:15.487913+0000","last_clean":"2024-09-10T19:01:15.487913+0000","last_became_active":"2024-09-10T18:57:02.441763+0000","last_became_peered":"2024-09-10T18:57:02.441763+0000","last_unstale":"2024-09-10T19:01:15.487913+0000","last_undegraded":"2024-09-10T19:01:15.487913+0000","last_fullsized":"2024-09-10T19:01:15.487913+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'10","last_scrub_stamp":"2024-09-10T18:59:22.516625+0000","last_deep_scrub":"190'9","last_deep_scrub_stamp":"2024-09-10T18:53:07.746309+0000","last_clean_scrub_stamp":"2024-09-10T18:59:22.516625+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021912820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":349,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.724666+0000","last_change":"2024-09-10T18:58:59.150773+0000","last_active":"2024-09-10T19:01:08.724666+0000","last_peered":"2024-09-10T19:01:08.724666+0000","last_clean":"2024-09-10T19:01:08.724666+0000","last_became_active":"2024-09-10T18:45:24.082318+0000","last_became_peered":"2024-09-10T18:45:24.082318+0000","last_unstale":"2024-09-10T19:01:08.724666+0000","last_undegraded":"2024-09-10T19:01:08.724666+0000","last_fullsized":"2024-09-10T19:01:08.724666+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:59.148958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:03.475299+0000","last_clean_scrub_stamp":"2024-09-10T18:58:59.148958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041797229999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":369,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.490654+0000","last_change":"2024-09-10T19:00:10.182129+0000","last_active":"2024-09-10T19:01:15.490654+0000","last_peered":"2024-09-10T19:01:15.490654+0000","last_clean":"2024-09-10T19:01:15.490654+0000","last_became_active":"2024-09-10T18:55:40.392524+0000","last_became_peered":"2024-09-10T18:55:40.392524+0000","last_unstale":"2024-09-10T19:01:15.490654+0000","last_undegraded":"2024-09-10T19:01:15.490654+0000","last_fullsized":"2024-09-10T19:01:15.490654+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:10.181982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:00:10.181982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:31.010155+0000","scrub_duration":0.0029510420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"232'19","reported_seq":395,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.730460+0000","last_change":"2024-09-10T18:59:09.544279+0000","last_active":"2024-09-10T19:01:08.730460+0000","last_peered":"2024-09-10T19:01:08.730460+0000","last_clean":"2024-09-10T19:01:08.730460+0000","last_became_active":"2024-09-10T18:45:20.058717+0000","last_became_peered":"2024-09-10T18:45:20.058717+0000","last_unstale":"2024-09-10T19:01:08.730460+0000","last_undegraded":"2024-09-10T19:01:08.730460+0000","last_fullsized":"2024-09-10T19:01:08.730460+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T18:59:09.544194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:59:09.544194+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043401830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":333,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.479270+0000","last_change":"2024-09-10T18:59:57.862722+0000","last_active":"2024-09-10T19:01:15.479270+0000","last_peered":"2024-09-10T19:01:15.479270+0000","last_clean":"2024-09-10T19:01:15.479270+0000","last_became_active":"2024-09-10T18:57:02.453464+0000","last_became_peered":"2024-09-10T18:57:02.453464+0000","last_unstale":"2024-09-10T19:01:15.479270+0000","last_undegraded":"2024-09-10T19:01:15.479270+0000","last_fullsized":"2024-09-10T19:01:15.479270+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:57.862666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:59:57.862666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001262242,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":345,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.475979+0000","last_change":"2024-09-10T18:58:55.726863+0000","last_active":"2024-09-10T19:01:15.475979+0000","last_peered":"2024-09-10T19:01:15.475979+0000","last_clean":"2024-09-10T19:01:15.475979+0000","last_became_active":"2024-09-10T18:45:24.267251+0000","last_became_peered":"2024-09-10T18:45:24.267251+0000","last_unstale":"2024-09-10T19:01:15.475979+0000","last_undegraded":"2024-09-10T19:01:15.475979+0000","last_fullsized":"2024-09-10T19:01:15.475979+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:55.726799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:58:55.726799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001154721,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"306'2","reported_seq":390,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.727983+0000","last_change":"2024-09-10T18:59:20.534927+0000","last_active":"2024-09-10T19:01:08.727983+0000","last_peered":"2024-09-10T19:01:08.727983+0000","last_clean":"2024-09-10T19:01:08.727983+0000","last_became_active":"2024-09-10T18:57:02.435758+0000","last_became_peered":"2024-09-10T18:57:02.435758+0000","last_unstale":"2024-09-10T19:01:08.727983+0000","last_undegraded":"2024-09-10T19:01:08.727983+0000","last_fullsized":"2024-09-10T19:01:08.727983+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"306'2","last_scrub_stamp":"2024-09-10T18:59:20.534869+0000","last_deep_scrub":"306'2","last_deep_scrub_stamp":"2024-09-10T18:59:20.534869+0000","last_clean_scrub_stamp":"2024-09-10T18:59:20.534869+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0072446200000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"258'34","reported_seq":468,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.725247+0000","last_change":"2024-09-10T19:00:52.459800+0000","last_active":"2024-09-10T19:01:08.725247+0000","last_peered":"2024-09-10T19:01:08.725247+0000","last_clean":"2024-09-10T19:01:08.725247+0000","last_became_active":"2024-09-10T18:59:48.744293+0000","last_became_peered":"2024-09-10T18:59:48.744293+0000","last_unstale":"2024-09-10T19:01:08.725247+0000","last_undegraded":"2024-09-10T19:01:08.725247+0000","last_fullsized":"2024-09-10T19:01:08.725247+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'34","last_scrub_stamp":"2024-09-10T18:56:23.838951+0000","last_deep_scrub":"258'34","last_deep_scrub_stamp":"2024-09-10T18:56:23.838951+0000","last_clean_scrub_stamp":"2024-09-10T18:56:23.838951+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.035929888,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":358,"reported_epoch":350,"state":"active+clean","last_fresh":"2024-09-10T19:01:08.725551+0000","last_change":"2024-09-10T19:00:54.441149+0000","last_active":"2024-09-10T19:01:08.725551+0000","last_peered":"2024-09-10T19:01:08.725551+0000","last_clean":"2024-09-10T19:01:08.725551+0000","last_became_active":"2024-09-10T18:55:40.397870+0000","last_became_peered":"2024-09-10T18:55:40.397870+0000","last_unstale":"2024-09-10T19:01:08.725551+0000","last_undegraded":"2024-09-10T19:01:08.725551+0000","last_fullsized":"2024-09-10T19:01:08.725551+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:13.610976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:13.610976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010999553,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":366,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.492815+0000","last_change":"2024-09-10T19:00:42.520936+0000","last_active":"2024-09-10T19:01:15.492815+0000","last_peered":"2024-09-10T19:01:15.492815+0000","last_clean":"2024-09-10T19:01:15.492815+0000","last_became_active":"2024-09-10T19:00:42.520239+0000","last_became_peered":"2024-09-10T19:00:42.520239+0000","last_unstale":"2024-09-10T19:01:15.492815+0000","last_undegraded":"2024-09-10T19:01:15.492815+0000","last_fullsized":"2024-09-10T19:01:15.492815+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:17.985545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:17.985545+0000","last_clean_scrub_stamp":"2024-09-10T18:59:17.985545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0086180830000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":372,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.490857+0000","last_change":"2024-09-10T18:58:57.919890+0000","last_active":"2024-09-10T19:01:15.490857+0000","last_peered":"2024-09-10T19:01:15.490857+0000","last_clean":"2024-09-10T19:01:15.490857+0000","last_became_active":"2024-09-10T18:57:02.460855+0000","last_became_peered":"2024-09-10T18:57:02.460855+0000","last_unstale":"2024-09-10T19:01:15.490857+0000","last_undegraded":"2024-09-10T19:01:15.490857+0000","last_fullsized":"2024-09-10T19:01:15.490857+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:57.919515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:58:57.919515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.01355234,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"232'18","reported_seq":382,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.491348+0000","last_change":"2024-09-10T18:58:52.890712+0000","last_active":"2024-09-10T19:01:15.491348+0000","last_peered":"2024-09-10T19:01:15.491348+0000","last_clean":"2024-09-10T19:01:15.491348+0000","last_became_active":"2024-09-10T18:45:20.058391+0000","last_became_peered":"2024-09-10T18:45:20.058391+0000","last_unstale":"2024-09-10T19:01:15.491348+0000","last_undegraded":"2024-09-10T19:01:15.491348+0000","last_fullsized":"2024-09-10T19:01:15.491348+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'18","last_scrub_stamp":"2024-09-10T18:58:52.890618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:58:52.890618+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016012082,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":377,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-10T19:01:15.476463+0000","last_change":"2024-09-10T18:58:56.690660+0000","last_active":"2024-09-10T19:01:15.476463+0000","last_peered":"2024-09-10T19:01:15.476463+0000","last_clean":"2024-09-10T19:01:15.476463+0000","last_became_active":"2024-09-10T18:57:02.449135+0000","last_became_peered":"2024-09-10T18:57:02.449135+0000","last_unstale":"2024-09-10T19:01:15.476463+0000","last_undegraded":"2024-09-10T19:01:15.476463+0000","last_fullsized":"2024-09-10T19:01:15.476463+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:56.690606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:58:56.690606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060631599999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":8645246976,"num_objects":131916,"num_object_clones":0,"num_object_copies":395748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4052,"num_objects_unfound":0,"num_objects_dirty":131916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":131916,"num_write_kb":8442624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17822,"num_bytes_recovered":1167982592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":27520991232,"data_stored":27520991232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57473,"ondisk_log_size":57473,"up":48,"acting":48,"num_store_stats":6},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":421,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1118,"num_read_kb":875,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":3372,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":499712,"data_stored":13268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":758,"ondisk_log_size":758,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":942,"num_read_kb":937,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":608258,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":7,"num_write":3,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":298468,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1701,"ondisk_log_size":1701,"up":63,"acting":63,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":0,"seq":223338299589,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":43,"seq":184683593929,"num_pgs":86,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4535312,"kb_used_data":3640796,"kb_used_omap":1,"kb_used_meta":894142,"kb_avail":89209840,"statfs":{"total":95995035648,"available":91350876160,"internally_reserved":0,"allocated":3728175104,"data_stored":3727666384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":915601752},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:00:54 2024","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.1830000000000001,"15min":0.98899999999999999},"min":{"1min":0.34399999999999997,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":5.0910000000000002,"5min":27.545000000000002,"15min":27.545000000000002},"last":49.353999999999999},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.3149999999999999,"15min":1.0649999999999999},"min":{"1min":0.27900000000000003,"5min":0.219,"15min":0.219},"max":{"1min":4.6180000000000003,"5min":27.568000000000001,"15min":27.568000000000001},"last":49.341999999999999}]},{"osd":1,"last update":"Tue Sep 10 19:01:03 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":1.7370000000000001,"15min":1.2889999999999999},"min":{"1min":0.253,"5min":0.22900000000000001,"15min":0.219},"max":{"1min":4.6200000000000001,"5min":45.311,"15min":45.311},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.84599999999999997,"5min":1.645,"15min":1.181},"min":{"1min":0.26300000000000001,"5min":0.23300000000000001,"15min":0.214},"max":{"1min":3.5289999999999999,"5min":45.027999999999999,"15min":45.027999999999999},"last":0.45400000000000001}]},{"osd":2,"last update":"Tue Sep 10 19:01:03 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.64800000000000002,"15min":0.78400000000000003},"min":{"1min":0.32700000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":3.4009999999999998,"5min":3.4009999999999998,"15min":4.4740000000000002},"last":0.441},{"interface":"front","average":{"1min":1.0660000000000001,"5min":0.60499999999999998,"15min":0.71899999999999997},"min":{"1min":0.28499999999999998,"5min":0.249,"15min":0.249},"max":{"1min":4.6580000000000004,"5min":4.6580000000000004,"15min":4.8150000000000004},"last":0.59399999999999997}]},{"osd":3,"last update":"Tue Sep 10 19:01:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":2.3410000000000002,"15min":1.4239999999999999},"min":{"1min":0.34200000000000003,"5min":0.29599999999999999,"15min":0.25700000000000001},"max":{"1min":5.0220000000000002,"5min":76.977999999999994,"15min":76.977999999999994},"last":47.325000000000003},{"interface":"front","average":{"1min":0.88600000000000001,"5min":2.2999999999999998,"15min":1.522},"min":{"1min":0.29799999999999999,"5min":0.27500000000000002,"15min":0.253},"max":{"1min":2.2080000000000002,"5min":76.879999999999995,"15min":76.879999999999995},"last":46.793999999999997}]},{"osd":4,"last update":"Tue Sep 10 19:01:03 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":0.64000000000000001,"15min":1.3280000000000001},"min":{"1min":0.25,"5min":0.23400000000000001,"15min":0.20999999999999999},"max":{"1min":4.7220000000000004,"5min":4.7220000000000004,"15min":165.03299999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.625,"15min":1.2430000000000001},"min":{"1min":0.26200000000000001,"5min":0.26000000000000001,"15min":0.22500000000000001},"max":{"1min":3.5619999999999998,"5min":3.5619999999999998,"15min":163.959},"last":0.30299999999999999}]},{"osd":5,"last update":"Tue Sep 10 19:01:03 2024","interfaces":[{"interface":"back","average":{"1min":12.706,"5min":3.0459999999999998,"15min":1.7549999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.24099999999999999},"max":{"1min":288.56999999999999,"5min":288.56999999999999,"15min":288.56999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":12.539,"5min":2.9660000000000002,"15min":1.7529999999999999},"min":{"1min":0.29799999999999999,"5min":0.27300000000000002,"15min":0.23999999999999999},"max":{"1min":291.05399999999997,"5min":291.05399999999997,"15min":291.05399999999997},"last":0.53500000000000003}]},{"osd":7,"last update":"Tue Sep 10 19:00:32 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.67500000000000004,"15min":0.80300000000000005},"min":{"1min":0.33700000000000002,"5min":0.27900000000000003,"15min":0.26200000000000001},"max":{"1min":1.599,"5min":4.1580000000000004,"15min":5.3680000000000003},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.70499999999999996,"15min":0.84599999999999997},"min":{"1min":0.309,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.4910000000000001,"5min":3.2759999999999998,"15min":7.0350000000000001},"last":0.56999999999999995}]}]},{"osd":5,"up_from":37,"seq":158913790157,"num_pgs":84,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4723472,"kb_used_data":3839452,"kb_used_omap":1,"kb_used_meta":883646,"kb_avail":89021680,"statfs":{"total":95995035648,"available":91158200320,"internally_reserved":0,"allocated":3931598848,"data_stored":3931089157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":904853848},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:00:49 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":2.7530000000000001,"15min":1.95},"min":{"1min":0.33000000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":2.1600000000000001,"5min":82.465000000000003,"15min":82.465000000000003},"last":2.5800000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":2.7970000000000002,"15min":2.113},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":1.645,"5min":82.519000000000005,"15min":82.519000000000005},"last":0.26500000000000001}]},{"osd":1,"last update":"Tue Sep 10 19:00:49 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":2.8490000000000002,"15min":1.524},"min":{"1min":0.313,"5min":0.20999999999999999,"15min":0.19500000000000001},"max":{"1min":3.851,"5min":67.582999999999998,"15min":67.582999999999998},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.73399999999999999,"5min":2.7879999999999998,"15min":1.667},"min":{"1min":0.35099999999999998,"5min":0.29699999999999999,"15min":0.20399999999999999},"max":{"1min":1.7250000000000001,"5min":67.603999999999999,"15min":67.603999999999999},"last":0.29899999999999999}]},{"osd":2,"last update":"Tue Sep 10 19:00:49 2024","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":0.73999999999999999,"15min":0.76000000000000001},"min":{"1min":0.30299999999999999,"5min":0.27200000000000002,"15min":0.249},"max":{"1min":7.8879999999999999,"5min":7.8879999999999999,"15min":7.8879999999999999},"last":1.649},{"interface":"front","average":{"1min":0.996,"5min":0.70599999999999996,"15min":0.77400000000000002},"min":{"1min":0.28499999999999998,"5min":0.27100000000000002,"15min":0.24399999999999999},"max":{"1min":6.7549999999999999,"5min":6.7549999999999999,"15min":6.7549999999999999},"last":0.90400000000000003}]},{"osd":3,"last update":"Tue Sep 10 19:00:49 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":4.7930000000000001,"15min":2.7029999999999998},"min":{"1min":0.27900000000000003,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":3.5510000000000002,"5min":123.34399999999999,"15min":123.34399999999999},"last":2.2000000000000002},{"interface":"front","average":{"1min":0.80300000000000005,"5min":3.1579999999999999,"15min":2.2810000000000001},"min":{"1min":0.36599999999999999,"5min":0.27300000000000002,"15min":0.245},"max":{"1min":2.7719999999999998,"5min":82.415000000000006,"15min":82.415000000000006},"last":1.365}]},{"osd":4,"last update":"Tue Sep 10 19:00:49 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.58199999999999996,"15min":0.84199999999999997},"min":{"1min":0.27500000000000002,"5min":0.221,"15min":0.20899999999999999},"max":{"1min":1.706,"5min":2.9940000000000002,"15min":9.0310000000000006},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.54100000000000004,"15min":0.78400000000000003},"min":{"1min":0.27300000000000002,"5min":0.23899999999999999,"15min":0.23400000000000001},"max":{"1min":2.742,"5min":2.742,"15min":11.182},"last":1.798}]},{"osd":6,"last update":"Tue Sep 10 19:01:10 2024","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":0.69899999999999995,"15min":0.96599999999999997},"min":{"1min":0.30199999999999999,"5min":0.23999999999999999,"15min":0.23400000000000001},"max":{"1min":4.1200000000000001,"5min":4.1200000000000001,"15min":11.17},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.0669999999999999,"5min":0.622,"15min":0.94999999999999996},"min":{"1min":0.38500000000000001,"5min":0.27200000000000002,"15min":0.26700000000000002},"max":{"1min":3.1930000000000001,"5min":3.1930000000000001,"15min":11.18},"last":1.8260000000000001}]},{"osd":7,"last update":"Tue Sep 10 19:00:28 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.66700000000000004,"15min":0.90800000000000003},"min":{"1min":0.318,"5min":0.28999999999999998,"15min":0.27600000000000002},"max":{"1min":1.3520000000000001,"5min":3.665,"15min":10.782999999999999},"last":1.615},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53500000000000003,"15min":0.79800000000000004},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.23599999999999999},"max":{"1min":1.246,"5min":1.956,"15min":11.846},"last":0.873}]}]},{"osd":4,"up_from":30,"seq":128849019092,"num_pgs":85,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7962020,"kb_used_data":7071708,"kb_used_omap":1,"kb_used_meta":890302,"kb_avail":85783132,"statfs":{"total":95995035648,"available":87841927168,"internally_reserved":0,"allocated":7241428992,"data_stored":7241060562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":911669592},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.4990000000000001,"15min":1.4570000000000001},"min":{"1min":0.39100000000000001,"5min":0.223,"15min":0.223},"max":{"1min":3.6400000000000001,"5min":46.234999999999999,"15min":46.234999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":1.099,"5min":1.423,"15min":1.4510000000000001},"min":{"1min":0.36599999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":3.609,"5min":46.264000000000003,"15min":46.264000000000003},"last":0.96899999999999997}]},{"osd":1,"last update":"Tue Sep 10 19:00:27 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":2.3330000000000002,"15min":1.556},"min":{"1min":0.33700000000000002,"5min":0.223,"15min":0.223},"max":{"1min":3.6749999999999998,"5min":125.15600000000001,"15min":125.15600000000001},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.66000000000000003,"5min":2.4140000000000001,"15min":1.498},"min":{"1min":0.26400000000000001,"5min":0.23699999999999999,"15min":0.23100000000000001},"max":{"1min":2.069,"5min":125.14700000000001,"15min":125.14700000000001},"last":0.90100000000000002}]},{"osd":2,"last update":"Tue Sep 10 19:00:27 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.65100000000000002,"15min":0.68000000000000005},"min":{"1min":0.38100000000000001,"5min":0.26200000000000001,"15min":0.25900000000000001},"max":{"1min":2.1560000000000001,"5min":2.8879999999999999,"15min":5.6189999999999998},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.30499999999999999,"5min":0.29899999999999999,"15min":0.27100000000000002},"max":{"1min":1.603,"5min":12.782999999999999,"15min":12.782999999999999},"last":1.401}]},{"osd":3,"last update":"Tue Sep 10 19:00:27 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":3.5150000000000001,"15min":2.0670000000000002},"min":{"1min":0.33000000000000002,"5min":0.28299999999999997,"15min":0.23799999999999999},"max":{"1min":3.1880000000000002,"5min":111.514,"15min":111.514},"last":1.4530000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":2.5670000000000002,"15min":1.6879999999999999},"min":{"1min":0.28399999999999997,"5min":0.26900000000000002,"15min":0.25700000000000001},"max":{"1min":0.875,"5min":110.914,"15min":110.914},"last":0.27400000000000002}]},{"osd":5,"last update":"Tue Sep 10 19:00:51 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.628,"15min":0.85499999999999998},"min":{"1min":0.24099999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":2.6800000000000002,"5min":9.7309999999999999,"15min":10.912000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.68200000000000005,"15min":0.79700000000000004},"min":{"1min":0.312,"5min":0.216,"15min":0.216},"max":{"1min":2.0379999999999998,"5min":10.147,"15min":10.147},"last":0.98799999999999999}]},{"osd":6,"last update":"Tue Sep 10 19:01:12 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.57099999999999995,"15min":0.749},"min":{"1min":0.33600000000000002,"5min":0.221,"15min":0.221},"max":{"1min":2.5110000000000001,"5min":2.5110000000000001,"15min":8.4030000000000005},"last":0.86699999999999999},{"interface":"front","average":{"1min":2.1190000000000002,"5min":0.77800000000000002,"15min":0.90900000000000003},"min":{"1min":0.32900000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":21.254000000000001,"5min":21.254000000000001,"15min":21.254000000000001},"last":4.8949999999999996}]},{"osd":7,"last update":"Tue Sep 10 19:00:46 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.63700000000000001,"15min":0.81599999999999995},"min":{"1min":0.33600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.1829999999999998,"5min":4.5250000000000004,"15min":6.4630000000000001},"last":1.008},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.67700000000000005,"15min":0.79700000000000004},"min":{"1min":0.28199999999999997,"5min":0.251,"15min":0.251},"max":{"1min":3.589,"5min":3.589,"15min":6.3949999999999996},"last":1.3660000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182617,"num_pgs":83,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4668268,"kb_used_data":3774692,"kb_used_omap":1,"kb_used_meta":893566,"kb_avail":89076884,"statfs":{"total":95995035648,"available":91214729216,"internally_reserved":0,"allocated":3865284608,"data_stored":3864907884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":915011928},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":4.7220000000000004,"15min":2.4209999999999998},"min":{"1min":0.23200000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":2.6989999999999998,"5min":290.721,"15min":290.721},"last":0.23599999999999999},{"interface":"front","average":{"1min":0.879,"5min":4.9530000000000003,"15min":2.661},"min":{"1min":0.17499999999999999,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":3.1880000000000002,"5min":293.59300000000002,"15min":293.59300000000002},"last":0.307}]},{"osd":1,"last update":"Tue Sep 10 19:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":2.1419999999999999,"15min":1.573},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.16200000000000001},"max":{"1min":3.173,"5min":10.645,"15min":58.215000000000003},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":1.8260000000000001,"15min":1.4910000000000001},"min":{"1min":0.22700000000000001,"5min":0.20000000000000001,"15min":0.188},"max":{"1min":2.3239999999999998,"5min":13.263999999999999,"15min":57.896999999999998},"last":0.40899999999999997}]},{"osd":2,"last update":"Tue Sep 10 19:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":1.286,"15min":0.95999999999999996},"min":{"1min":0.255,"5min":0.255,"15min":0.22800000000000001},"max":{"1min":2.23,"5min":5.6680000000000001,"15min":5.8220000000000001},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":1.1220000000000001,"15min":0.84299999999999997},"min":{"1min":0.254,"5min":0.22600000000000001,"15min":0.21199999999999999},"max":{"1min":1.526,"5min":5.1399999999999997,"15min":7.0979999999999999},"last":0.91600000000000004}]},{"osd":4,"last update":"Tue Sep 10 19:00:30 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":1.337,"15min":1.1990000000000001},"min":{"1min":0.33400000000000002,"5min":0.26400000000000001,"15min":0.247},"max":{"1min":2.3130000000000002,"5min":5.0490000000000004,"15min":29.013000000000002},"last":2.117},{"interface":"front","average":{"1min":0.501,"5min":1.4039999999999999,"15min":1.21},"min":{"1min":0.31900000000000001,"5min":0.28000000000000003,"15min":0.26800000000000002},"max":{"1min":0.90000000000000002,"5min":8.8130000000000006,"15min":28.826000000000001},"last":0.97199999999999998}]},{"osd":5,"last update":"Tue Sep 10 19:00:54 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":1.4970000000000001,"15min":1.22},"min":{"1min":0.39800000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":2.4409999999999998,"5min":7.2300000000000004,"15min":7.2300000000000004},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.365,"15min":1.1930000000000001},"min":{"1min":0.29099999999999998,"5min":0.27100000000000002,"15min":0.255},"max":{"1min":2.9849999999999999,"5min":5.4139999999999997,"15min":9.2509999999999994},"last":0.49399999999999999}]},{"osd":6,"last update":"Tue Sep 10 19:01:09 2024","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.504,"15min":1.2629999999999999},"min":{"1min":0.39500000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":3.431,"5min":5.4480000000000004,"15min":9.3239999999999998},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.04,"5min":1.4219999999999999,"15min":1.2210000000000001},"min":{"1min":0.34999999999999998,"5min":0.308,"15min":0.29399999999999998},"max":{"1min":2.794,"5min":5.0789999999999997,"15min":7.1559999999999997},"last":0.94699999999999995}]},{"osd":7,"last update":"Tue Sep 10 19:00:40 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":1.6120000000000001,"15min":1.27},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.34899999999999998},"max":{"1min":0.98599999999999999,"5min":8.7989999999999995,"15min":8.7989999999999995},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.83099999999999996,"5min":1.526,"15min":1.298},"min":{"1min":0.39600000000000002,"5min":0.36899999999999999,"15min":0.33100000000000002},"max":{"1min":2.2890000000000001,"5min":6.6029999999999998,"15min":9.2949999999999999},"last":0.53400000000000003}]}]},{"osd":2,"up_from":0,"seq":77309411549,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834575072,"num_pgs":87,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3515520,"kb_used_data":2612216,"kb_used_omap":1,"kb_used_meta":903294,"kb_avail":90229632,"statfs":{"total":95995035648,"available":92395143168,"internally_reserved":0,"allocated":2674909184,"data_stored":2674372175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":924973400},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:00:50 2024","interfaces":[{"interface":"back","average":{"1min":2.5139999999999998,"5min":2.8820000000000001,"15min":1.792},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":45.786000000000001,"5min":111.014,"15min":111.014},"last":0.27000000000000002},{"interface":"front","average":{"1min":2.5600000000000001,"5min":2.9980000000000002,"15min":1.746},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":46.311999999999998,"5min":111.072,"15min":111.072},"last":0.877}]},{"osd":2,"last update":"Tue Sep 10 19:00:36 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.92700000000000005,"15min":0.73199999999999998},"min":{"1min":0.23799999999999999,"5min":0.23300000000000001,"15min":0.186},"max":{"1min":1.1990000000000001,"5min":4.1719999999999997,"15min":4.1719999999999997},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.90600000000000003,"15min":0.76000000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.215},"max":{"1min":2.8029999999999999,"5min":5.6929999999999996,"15min":7.093},"last":0.48199999999999998}]},{"osd":3,"last update":"Tue Sep 10 19:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":3.718,"15min":2.0419999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.185},"max":{"1min":6.9560000000000004,"5min":92.069000000000003,"15min":92.069000000000003},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":3.1930000000000001,"15min":1.9019999999999999},"min":{"1min":0.23300000000000001,"5min":0.218,"15min":0.19900000000000001},"max":{"1min":2.1110000000000002,"5min":88.658000000000001,"15min":88.658000000000001},"last":0.85999999999999999}]},{"osd":4,"last update":"Tue Sep 10 19:00:27 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":1.159,"15min":1.135},"min":{"1min":0.28100000000000003,"5min":0.27800000000000002,"15min":0.26200000000000001},"max":{"1min":1.7330000000000001,"5min":9.125,"15min":15.965999999999999},"last":0.307},{"interface":"front","average":{"1min":0.56599999999999995,"5min":1.1539999999999999,"15min":1.002},"min":{"1min":0.34999999999999998,"5min":0.252,"15min":0.246},"max":{"1min":0.98499999999999999,"5min":7.0940000000000003,"15min":7.0940000000000003},"last":1.087}]},{"osd":5,"last update":"Tue Sep 10 19:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":1.1579999999999999,"15min":1.1499999999999999},"min":{"1min":0.33400000000000002,"5min":0.30499999999999999,"15min":0.28499999999999998},"max":{"1min":3.0779999999999998,"5min":6.4690000000000003,"15min":10.334},"last":3.4630000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":1.0609999999999999,"15min":1.175},"min":{"1min":0.30499999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.486,"5min":7.0590000000000002,"15min":16.039999999999999},"last":2.4910000000000001}]},{"osd":6,"last update":"Tue Sep 10 19:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.2809999999999999,"15min":1.506},"min":{"1min":0.34200000000000003,"5min":0.33100000000000002,"15min":0.308},"max":{"1min":7.2850000000000001,"5min":19.125,"15min":38.039999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":1.51,"5min":1.173,"15min":1.5820000000000001},"min":{"1min":0.32600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":7.2380000000000004,"5min":7.2380000000000004,"15min":38.100999999999999},"last":0.51100000000000001}]},{"osd":7,"last update":"Tue Sep 10 19:00:36 2024","interfaces":[{"interface":"back","average":{"1min":1.5489999999999999,"5min":1.2829999999999999,"15min":1.143},"min":{"1min":0.371,"5min":0.33100000000000002,"15min":0.315},"max":{"1min":19.169,"5min":19.169,"15min":19.169},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":1.0640000000000001,"15min":1.05},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.32800000000000001},"max":{"1min":0.91300000000000003,"5min":4.46,"15min":4.9029999999999996},"last":0.42299999999999999}]}]},{"osd":0,"up_from":241,"seq":1035087118405,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6201316,"kb_used_data":5855068,"kb_used_omap":4,"kb_used_meta":346235,"kb_avail":87543836,"statfs":{"total":95995035648,"available":89644888064,"internally_reserved":0,"allocated":5995589632,"data_stored":5995220458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":354545158},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,3,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 19:00:48 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.8899999999999999,"15min":2.4830000000000001},"min":{"1min":0.25700000000000001,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":6.9050000000000002,"5min":34.692,"15min":34.692},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.996,"5min":1.8080000000000001,"15min":2.4820000000000002},"min":{"1min":0.28199999999999997,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":7.5960000000000001,"5min":34.588999999999999,"15min":34.588999999999999},"last":0.64000000000000001}]},{"osd":2,"last update":"Tue Sep 10 19:00:48 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.0589999999999999,"15min":0.93799999999999994},"min":{"1min":0.29999999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":6.8639999999999999,"5min":13.305,"15min":13.305},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":1.036,"15min":0.80800000000000005},"min":{"1min":0.30599999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.5600000000000001,"5min":13.693,"15min":13.693},"last":0.49399999999999999}]},{"osd":3,"last update":"Tue Sep 10 19:00:48 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":1.4430000000000001,"15min":1.1279999999999999},"min":{"1min":0.39600000000000002,"5min":0.251,"15min":0.251},"max":{"1min":2.2309999999999999,"5min":21.108000000000001,"15min":21.108000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.97599999999999998,"5min":1.9830000000000001,"15min":1.7},"min":{"1min":0.35599999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":2.3159999999999998,"5min":11.629,"15min":11.629},"last":2.8700000000000001}]},{"osd":4,"last update":"Tue Sep 10 19:00:48 2024","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.0760000000000001,"15min":0.83699999999999997},"min":{"1min":0.39800000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":3.1960000000000002,"5min":13.161,"15min":13.161},"last":0.60499999999999998},{"interface":"front","average":{"1min":1.583,"5min":1.1850000000000001,"15min":0.997},"min":{"1min":0.47399999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.4590000000000001,"5min":5.0490000000000004,"15min":5.0490000000000004},"last":2.9159999999999999}]},{"osd":5,"last update":"Tue Sep 10 19:00:48 2024","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.2330000000000001,"15min":1.0840000000000001},"min":{"1min":0.47399999999999998,"5min":0.31,"15min":0.31},"max":{"1min":3.3679999999999999,"5min":5.125,"15min":5.125},"last":4.5819999999999999},{"interface":"front","average":{"1min":1.25,"5min":1.1419999999999999,"15min":0.90000000000000002},"min":{"1min":0.438,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":3.1179999999999999,"5min":13.202,"15min":13.202},"last":1.367}]},{"osd":6,"last update":"Tue Sep 10 19:00:48 2024","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.198,"15min":1.008},"min":{"1min":0.504,"5min":0.312,"15min":0.312},"max":{"1min":3.4009999999999998,"5min":5.0899999999999999,"15min":5.0899999999999999},"last":2.96},{"interface":"front","average":{"1min":1.415,"5min":1.1020000000000001,"15min":0.83199999999999996},"min":{"1min":0.41899999999999998,"5min":0.313,"15min":0.313},"max":{"1min":3.3780000000000001,"5min":13.279,"15min":13.279},"last":1.3120000000000001}]},{"osd":7,"last update":"Tue Sep 10 19:00:48 2024","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.181,"15min":0.94499999999999995},"min":{"1min":0.45900000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":3.3679999999999999,"5min":13.233000000000001,"15min":13.233000000000001},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.135,"15min":1.143},"min":{"1min":0.439,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.7229999999999999,"5min":10.141999999999999,"15min":10.141999999999999},"last":0.93700000000000006}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":2668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6006374400,"data_stored":6006374400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2684026880,"data_stored":2684026880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3879141376,"data_stored":3879141376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7269056512,"data_stored":7269056512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3931963392,"data_stored":3931963392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3750428672,"data_stored":3750428672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T19:01:25.802 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph status --format=json 2024-09-10T19:01:25.815 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:25.846 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:25.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:25.847 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:25.948 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:25.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:25.980 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:25.980 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:26.025 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:25 smithi103 ceph-mon[33163]: osdmap e355: 8 total, 8 up, 6 in 2024-09-10T19:01:26.025 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:25 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:01:26.025 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:25 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:01:26.025 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:25 smithi103 ceph-mon[33163]: osdmap e356: 8 total, 8 up, 6 in 2024-09-10T19:01:26.081 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:26.103 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:01:26.140 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:26.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:26.141 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:26.156 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T19:01:26.156 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T19:01:26.193 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T19:01:26.193 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T19:01:26.226 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T19:01:26.226 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T19:01:26.242 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:26.259 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T19:01:26.259 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T19:01:26.280 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:26.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:26.281 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:26.292 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T19:01:26.292 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T19:01:26.326 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T19:01:26.326 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T19:01:26.357 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T19:01:26.357 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T19:01:26.382 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:26.385 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T19:01:26.386 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T19:01:26.421 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:26.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:26.422 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:26.523 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:26.558 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:26.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:26.559 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:26.660 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:26.692 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:26.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:26.693 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:26.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:26 smithi018 ceph-mon[29336]: pgmap v753: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 8.4 GiB data, 31 GiB used, 505 GiB / 536 GiB avail; 853 B/s rd, 69 MiB/s wr, 1.11k op/s; 2275/412305 objects misplaced (0.552%); 28 MiB/s, 446 objects/s recovering 2024-09-10T19:01:26.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:26 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1527206515' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:01:26.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:26 smithi018 ceph-mon[40196]: pgmap v753: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 8.4 GiB data, 31 GiB used, 505 GiB / 536 GiB avail; 853 B/s rd, 69 MiB/s wr, 1.11k op/s; 2275/412305 objects misplaced (0.552%); 28 MiB/s, 446 objects/s recovering 2024-09-10T19:01:26.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:26 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1527206515' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:01:26.793 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:26.833 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:26.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:26.834 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:26.935 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:26.971 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:26.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:26.972 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:26.994 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:26 smithi103 ceph-mon[33163]: pgmap v753: 168 pgs: 3 active+remapped+backfilling, 165 active+clean; 8.4 GiB data, 31 GiB used, 505 GiB / 536 GiB avail; 853 B/s rd, 69 MiB/s wr, 1.11k op/s; 2275/412305 objects misplaced (0.552%); 28 MiB/s, 446 objects/s recovering 2024-09-10T19:01:26.994 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:26 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1527206515' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:01:27.073 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:27.110 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:27.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:27.112 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:27.212 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:27.248 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:27.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:27.249 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:27.272 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:01:26 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:01:27.349 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:27.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:27.387 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:27.387 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:27.488 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:27.522 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:27.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:27.524 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:27.624 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:27.656 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:27.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:27.657 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:27.758 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:27.793 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:27.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:27.794 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:27.895 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:27.933 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:01:27.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:27.933 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:27.934 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:28.034 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:28.067 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:28.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:28.068 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:28.169 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:28.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:28.201 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:28.201 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:28.302 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:28.336 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:28.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:28.337 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:28.438 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:28.476 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:28.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:28.477 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:28.544 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:01:28.577 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:28.609 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:28.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:28.610 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:28.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:28 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:28.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:28 smithi018 ceph-mon[29336]: pgmap v755: 168 pgs: 1 active+remapped+backfilling, 167 active+clean; 8.3 GiB data, 31 GiB used, 505 GiB / 536 GiB avail; 1.2 KiB/s rd, 31 MiB/s wr, 1.01k op/s; 1251/409005 objects misplaced (0.306%); 27 MiB/s, 430 objects/s recovering 2024-09-10T19:01:28.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:28 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:28.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:28 smithi018 ceph-mon[40196]: pgmap v755: 168 pgs: 1 active+remapped+backfilling, 167 active+clean; 8.3 GiB data, 31 GiB used, 505 GiB / 536 GiB avail; 1.2 KiB/s rd, 31 MiB/s wr, 1.01k op/s; 1251/409005 objects misplaced (0.306%); 27 MiB/s, 430 objects/s recovering 2024-09-10T19:01:28.711 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:28.736 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:28.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:28.737 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:28.838 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:28.860 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:28.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:28.861 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:28.962 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:28.984 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:28.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:28.985 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:29.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:28 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:29.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:28 smithi103 ceph-mon[33163]: pgmap v755: 168 pgs: 1 active+remapped+backfilling, 167 active+clean; 8.3 GiB data, 31 GiB used, 505 GiB / 536 GiB avail; 1.2 KiB/s rd, 31 MiB/s wr, 1.01k op/s; 1251/409005 objects misplaced (0.306%); 27 MiB/s, 430 objects/s recovering 2024-09-10T19:01:29.086 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:29.109 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:29.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:29.110 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:29.211 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:29.234 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:29.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:29.235 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:29.336 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:29.360 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:29.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:29.361 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:29.462 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:29.487 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:29.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:29.488 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:29.589 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:29.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:29.624 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:29.624 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:29.725 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:29.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:29.764 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:29.764 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:29.862 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:29 smithi018 ceph-mon[40196]: osdmap e357: 8 total, 8 up, 6 in 2024-09-10T19:01:29.862 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:29 smithi018 ceph-mon[29336]: osdmap e357: 8 total, 8 up, 6 in 2024-09-10T19:01:29.865 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:29.899 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T19:01:29.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:29.903 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:29.903 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:29.912 INFO:teuthology.orchestra.run.smithi018.stdout:ERROR: (22) Invalid argument 2024-09-10T19:01:29.912 INFO:teuthology.orchestra.run.smithi018.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T19:01:30.004 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:30.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:29 smithi103 ceph-mon[33163]: osdmap e357: 8 total, 8 up, 6 in 2024-09-10T19:01:30.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:30.077 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:30.077 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:30.178 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:30.226 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:30.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:30.227 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:30.328 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:30.364 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:30.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:30.365 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:30.466 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:30.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:30.500 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:30.500 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:30.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:01:30 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:01:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:01:30.600 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:30.636 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:30.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:30.637 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:30.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:30 smithi018 ceph-mon[40196]: pgmap v757: 168 pgs: 1 peering, 167 active+clean; 8.2 GiB data, 31 GiB used, 505 GiB / 536 GiB avail; 1.0 KiB/s rd, 26 MiB/s wr, 1.04k op/s; 35 MiB/s, 566 objects/s recovering 2024-09-10T19:01:30.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:30 smithi018 ceph-mon[40196]: osdmap e358: 8 total, 8 up, 6 in 2024-09-10T19:01:30.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:30 smithi018 ceph-mon[29336]: pgmap v757: 168 pgs: 1 peering, 167 active+clean; 8.2 GiB data, 31 GiB used, 505 GiB / 536 GiB avail; 1.0 KiB/s rd, 26 MiB/s wr, 1.04k op/s; 35 MiB/s, 566 objects/s recovering 2024-09-10T19:01:30.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:30 smithi018 ceph-mon[29336]: osdmap e358: 8 total, 8 up, 6 in 2024-09-10T19:01:30.738 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:30.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:30.760 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:30.760 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:30.861 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:30.900 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:30.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:30.901 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:31.001 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:31.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:30 smithi103 ceph-mon[33163]: pgmap v757: 168 pgs: 1 peering, 167 active+clean; 8.2 GiB data, 31 GiB used, 505 GiB / 536 GiB avail; 1.0 KiB/s rd, 26 MiB/s wr, 1.04k op/s; 35 MiB/s, 566 objects/s recovering 2024-09-10T19:01:31.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:30 smithi103 ceph-mon[33163]: osdmap e358: 8 total, 8 up, 6 in 2024-09-10T19:01:31.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:31.041 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:31.042 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:31.142 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:31.174 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:31.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:31.175 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:31.276 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:31.313 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:31.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:31.314 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:31.415 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:31.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:01:30 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:01:30] "GET /metrics HTTP/1.1" 200 224718 "" "Prometheus/2.43.0" 2024-09-10T19:01:31.450 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:31.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:31.451 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:31.552 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:31.585 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:31.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:31.588 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:31.689 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:31.729 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:31.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:31.730 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:31.830 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:31.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:31.868 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:31.868 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:31.969 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:32.002 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:32.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:32.003 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:32.061 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-10T19:01:32.104 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:32.140 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:32.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:32.141 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:32.242 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:32.280 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:32.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:32.281 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:32.382 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:32.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:32.440 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:32.440 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:32.541 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:32.576 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:32.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:32.578 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:32.678 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:32.680 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:32 smithi018 ceph-mon[29336]: pgmap v759: 168 pgs: 1 peering, 167 active+clean; 8.2 GiB data, 30 GiB used, 506 GiB / 536 GiB avail; 1023 B/s rd, 14 MiB/s wr, 986 op/s; 24 MiB/s, 376 objects/s recovering 2024-09-10T19:01:32.680 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:32 smithi018 ceph-mon[40196]: pgmap v759: 168 pgs: 1 peering, 167 active+clean; 8.2 GiB data, 30 GiB used, 506 GiB / 536 GiB avail; 1023 B/s rd, 14 MiB/s wr, 986 op/s; 24 MiB/s, 376 objects/s recovering 2024-09-10T19:01:32.724 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:32.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:32.725 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:32.826 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:32.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:32.854 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:32.855 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:32.955 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:32.987 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:32.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:32.988 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:33.024 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:32 smithi103 ceph-mon[33163]: pgmap v759: 168 pgs: 1 peering, 167 active+clean; 8.2 GiB data, 30 GiB used, 506 GiB / 536 GiB avail; 1023 B/s rd, 14 MiB/s wr, 986 op/s; 24 MiB/s, 376 objects/s recovering 2024-09-10T19:01:33.089 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:33.137 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:33.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:33.138 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:33.156 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T19:01:33.156 INFO:teuthology.orchestra.run.smithi018.stdout:{"fsid":"ba327f6c-6fa3-11ef-bcea-c7b262605968","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"noscrub flag(s) set","count":7},"muted":false}},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1184,"monmap":{"epoch":3,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":358,"num_osds":8,"num_up_osds":8,"osd_up_since":1725994539,"num_in_osds":6,"osd_in_since":1725994621,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":167},{"state_name":"peering","count":1}],"num_pgs":168,"num_pools":7,"num_objects":134274,"data_bytes":8787484374,"bytes_used":32704327680,"bytes_avail":543265886208,"bytes_total":575970213888,"inactive_pgs_ratio":0.0059523810632526875,"recovering_objects_per_sec":376,"recovering_bytes_per_sec":24656338,"recovering_keys_per_sec":0,"num_objects_recovered":2259,"num_bytes_recovered":147980288,"num_keys_recovered":0,"read_bytes_sec":1023,"write_bytes_sec":15014548,"read_op_per_sec":0,"write_op_per_sec":985},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"services":{"dashboard":"https://172.21.15.18:8443/","prometheus":"http://172.21.15.18:9283/"}},"servicemap":{"epoch":15,"modified":"2024-09-10T18:56:39.251082+0000","services":{"rgw":{"daemons":{"summary":"","14460":{"start_epoch":12,"start_stamp":"2024-09-10T18:45:25.199225+0000","gid":14460,"addr":"172.21.15.18:0/3895312980","metadata":{"arch":"x86_64","ceph_release":"quincy","ceph_version":"ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable)","ceph_version_short":"17.2.7-1606-g5be57395","container_hostname":"smithi018","container_image":"quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16","cpu":"Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz","distro":"centos","distro_description":"CentOS Stream 9","distro_version":"9","frontend_config#0":"beast port=80","frontend_type#0":"beast","hostname":"smithi018","id":"foo.a","kernel_description":"#1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024","kernel_version":"5.14.0-505.el9.x86_64","mem_swap_kb":"0","mem_total_kb":"32493496","num_handles":"1","os":"Linux","pid":"2","realm_id":"","realm_name":"","zone_id":"0a27a0b0-e8a8-41c2-bb1b-8a59ee973f43","zone_name":"default","zonegroup_id":"48d110f7-3055-40a1-811b-d1f5b65fef07","zonegroup_name":"default"},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T19:01:33.239 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:33.278 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:33.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:33.279 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:33.379 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:33.418 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:33.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:33.419 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:33.520 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:33.556 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:33.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:33.557 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:33.658 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:33.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:33 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3233052890' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-10T19:01:33.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:33 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3233052890' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-10T19:01:33.709 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:33.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:33.710 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:33.747 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:01:33.754 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:01:33.811 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:33.844 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:33.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:33.845 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:33.945 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:33.979 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:33.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:33.980 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:34.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:33 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3233052890' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-10T19:01:34.081 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:34.120 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:34.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:34.121 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:34.221 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:34.251 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:34.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:34.252 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:34.353 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:34.389 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:34.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:34.390 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:34.491 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:34.522 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:34.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:34.523 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:34.588 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is set 2024-09-10T19:01:34.624 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:34.664 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:34.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:34.665 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:34.766 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:34.801 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:34.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:34.802 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:34.866 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:34 smithi018 ceph-mon[29336]: pgmap v760: 168 pgs: 1 peering, 167 active+clean; 7.9 GiB data, 30 GiB used, 506 GiB / 536 GiB avail; 1.4 KiB/s rd, 11 MiB/s wr, 1.38k op/s; 18 MiB/s, 286 objects/s recovering 2024-09-10T19:01:34.866 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:34 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3274044890' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:01:34.867 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:34 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:01:34.867 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:34 smithi018 ceph-mon[40196]: pgmap v760: 168 pgs: 1 peering, 167 active+clean; 7.9 GiB data, 30 GiB used, 506 GiB / 536 GiB avail; 1.4 KiB/s rd, 11 MiB/s wr, 1.38k op/s; 18 MiB/s, 286 objects/s recovering 2024-09-10T19:01:34.867 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:34 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3274044890' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:01:34.867 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:34 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:01:34.903 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:34.908 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2024-09-10T19:01:34.908 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T19:01:34.934 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:34.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:34.934 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:35.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:34 smithi103 ceph-mon[33163]: pgmap v760: 168 pgs: 1 peering, 167 active+clean; 7.9 GiB data, 30 GiB used, 506 GiB / 536 GiB avail; 1.4 KiB/s rd, 11 MiB/s wr, 1.38k op/s; 18 MiB/s, 286 objects/s recovering 2024-09-10T19:01:35.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:34 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3274044890' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:01:35.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:34 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:01:35.036 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:35.072 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:35.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:35.073 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:35.173 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:35.217 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:35.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:35.219 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:35.319 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:35.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:35.353 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:35.353 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:35.454 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:35.487 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:35.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:35.488 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:35.589 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:35.626 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:35.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:35.627 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:35.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:35 smithi018 ceph-mon[40196]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:01:35.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:35 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T19:01:35.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:35 smithi018 ceph-mon[40196]: osdmap e359: 8 total, 8 up, 6 in 2024-09-10T19:01:35.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:35 smithi018 ceph-mon[29336]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:01:35.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:35 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T19:01:35.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:35 smithi018 ceph-mon[29336]: osdmap e359: 8 total, 8 up, 6 in 2024-09-10T19:01:35.728 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:35.761 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:35.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:35.762 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:35.863 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:35.905 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:35.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:35.906 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:36.006 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:36.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:35 smithi103 ceph-mon[33163]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:01:36.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:35 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T19:01:36.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:35 smithi103 ceph-mon[33163]: osdmap e359: 8 total, 8 up, 6 in 2024-09-10T19:01:36.048 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:36.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:36.049 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:36.150 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:36.155 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:01:36.176 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:36.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:36.177 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:36.278 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:36.313 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:36.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:36.314 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:36.414 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:36.456 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:36.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:36.457 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:36.557 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:36.587 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:36.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:36.589 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:36.689 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:36.693 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:36 smithi018 ceph-mon[40196]: pgmap v762: 168 pgs: 168 active+clean; 7.8 GiB data, 30 GiB used, 507 GiB / 536 GiB avail; 748 B/s rd, 0 B/s wr, 1.21k op/s; 11 MiB/s, 179 objects/s recovering 2024-09-10T19:01:36.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:36 smithi018 ceph-mon[29336]: pgmap v762: 168 pgs: 168 active+clean; 7.8 GiB data, 30 GiB used, 507 GiB / 536 GiB avail; 748 B/s rd, 0 B/s wr, 1.21k op/s; 11 MiB/s, 179 objects/s recovering 2024-09-10T19:01:36.724 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:36.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:36.725 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:36.826 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:36.864 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:36.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:36.865 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:36.966 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:37.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:37.006 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:37.006 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:37.008 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:36 smithi103 ceph-mon[33163]: pgmap v762: 168 pgs: 168 active+clean; 7.8 GiB data, 30 GiB used, 507 GiB / 536 GiB avail; 748 B/s rd, 0 B/s wr, 1.21k op/s; 11 MiB/s, 179 objects/s recovering 2024-09-10T19:01:37.107 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:37.140 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:37.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:37.141 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:37.242 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:37.272 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:01:37 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:01:37.275 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:37.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:37.276 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:37.376 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:37.412 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:37.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:37.413 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:37.514 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:37.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:37.545 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:37.545 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:37.646 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:37.676 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:37.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:37.677 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:37.778 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:37.811 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:37.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:37.812 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:37.869 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-10T19:01:37.913 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:37.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:37.983 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:37.983 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:38.083 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:38.148 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:38.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:38.149 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:38.249 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:38.290 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:38.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:38.291 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:38.392 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:38.421 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:38.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:38.422 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:38.523 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:38.568 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:38.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:38.569 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:38.657 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:38 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:38.657 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:38 smithi018 ceph-mon[29336]: pgmap v763: 168 pgs: 168 active+clean; 7.4 GiB data, 28 GiB used, 508 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.74k op/s 2024-09-10T19:01:38.658 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:38 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:38.658 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:38 smithi018 ceph-mon[40196]: pgmap v763: 168 pgs: 168 active+clean; 7.4 GiB data, 28 GiB used, 508 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.74k op/s 2024-09-10T19:01:38.670 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:38.707 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:38.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:38.708 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:38.809 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:38.831 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T19:01:38.831 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T19:01:38.842 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:38.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:38.843 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:38.862 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T19:01:38.863 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T19:01:38.893 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T19:01:38.893 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T19:01:38.921 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T19:01:38.921 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T19:01:38.944 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:38.952 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T19:01:38.953 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T19:01:38.984 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:38.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:38.985 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:38.989 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T19:01:38.989 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T19:01:39.018 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T19:01:39.018 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T19:01:39.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:38 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:39.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:38 smithi103 ceph-mon[33163]: pgmap v763: 168 pgs: 168 active+clean; 7.4 GiB data, 28 GiB used, 508 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.74k op/s 2024-09-10T19:01:39.052 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T19:01:39.052 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T19:01:39.086 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:39.117 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:39.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:39.118 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:39.218 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:39.253 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:39.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:39.254 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:39.355 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:39.388 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:39.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:39.389 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:39.490 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:39.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:39.525 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:39.525 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:39.626 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:39.662 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:39.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:39.663 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:39.764 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:39.798 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:39.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:39.799 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:39.900 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:39.930 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:39.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:39.932 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:39.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:39 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1272099752' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:01:39.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:39 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1272099752' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:01:40.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:39 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1272099752' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:01:40.032 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:40.068 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:40.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:40.069 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:40.170 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:40.207 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:40.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:40.208 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:40.309 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:40.346 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:40.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:40.347 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:40.448 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:40.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:40.487 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:40.488 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:40.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:01:40 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:01:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:01:40.588 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:40.626 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:40.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:40.627 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:40.728 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:40.760 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:40.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:40.762 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:40.862 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:40.902 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:40.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:40.903 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:40.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:40 smithi018 ceph-mon[29336]: pgmap v764: 168 pgs: 168 active+clean; 7.3 GiB data, 28 GiB used, 508 GiB / 536 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.58k op/s 2024-09-10T19:01:40.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:40 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:01:40.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:40 smithi018 ceph-mon[40196]: pgmap v764: 168 pgs: 168 active+clean; 7.3 GiB data, 28 GiB used, 508 GiB / 536 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.58k op/s 2024-09-10T19:01:40.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:40 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:01:41.004 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:41.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:40 smithi103 ceph-mon[33163]: pgmap v764: 168 pgs: 168 active+clean; 7.3 GiB data, 28 GiB used, 508 GiB / 536 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.58k op/s 2024-09-10T19:01:41.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:40 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:01:41.040 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:41.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:41.041 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:41.141 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:41.176 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:41.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:41.178 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:41.279 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:41.311 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:41.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:41.312 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:41.413 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:41.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:01:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:01:40] "GET /metrics HTTP/1.1" 200 224671 "" "Prometheus/2.43.0" 2024-09-10T19:01:41.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:41.454 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:41.454 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:41.554 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:41.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:41.589 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:41.590 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:41.690 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:41.724 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:41.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:41.725 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:41.826 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:41.861 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:41.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:41.862 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:41.963 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:41.995 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:41.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:41.996 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:42.097 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:42.134 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:42.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:42.135 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:42.236 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:42.257 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:42.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:42.258 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:42.359 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:42.390 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:42.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:42.391 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:42.492 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:42.525 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:42.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:42.526 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:42.626 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:42.661 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:42.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:42.662 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:42.763 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:42.791 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T19:01:42.797 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:42.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:42.798 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:42.899 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:42.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:42.930 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:42.930 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:42.943 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:42 smithi018 ceph-mon[40196]: pgmap v765: 168 pgs: 168 active+clean; 7.2 GiB data, 28 GiB used, 509 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.58k op/s 2024-09-10T19:01:42.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:42 smithi018 ceph-mon[29336]: pgmap v765: 168 pgs: 168 active+clean; 7.2 GiB data, 28 GiB used, 509 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.58k op/s 2024-09-10T19:01:43.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:42 smithi103 ceph-mon[33163]: pgmap v765: 168 pgs: 168 active+clean; 7.2 GiB data, 28 GiB used, 509 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.58k op/s 2024-09-10T19:01:43.031 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:43.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:43.068 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:43.068 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:43.169 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:43.200 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:43.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:43.201 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:43.302 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:43.341 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:43.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:43.343 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:43.444 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:43.474 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:43.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:43.475 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:43.576 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:43.614 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:43.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:43.615 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:43.716 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:43.754 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:43.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:43.756 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:43.857 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:43.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:43.881 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:43.882 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:43.982 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:44.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:44.022 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:44.022 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:44.123 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:44.159 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:44.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:44.160 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:44.260 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:44.292 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:44.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:44.293 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:44.394 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:44.432 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:44.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:44.434 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:44.534 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:44.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:44.571 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:44.571 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:44.672 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:44.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:44.707 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:44.707 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:44.808 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:44.838 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:44.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:44.839 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:44.939 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:44 smithi018 ceph-mon[29336]: pgmap v766: 168 pgs: 168 active+clean; 6.9 GiB data, 27 GiB used, 510 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.69k op/s 2024-09-10T19:01:44.940 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:44 smithi018 ceph-mon[40196]: pgmap v766: 168 pgs: 168 active+clean; 6.9 GiB data, 27 GiB used, 510 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.69k op/s 2024-09-10T19:01:44.940 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:44.974 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:44.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:44.975 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:45.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:44 smithi103 ceph-mon[33163]: pgmap v766: 168 pgs: 168 active+clean; 6.9 GiB data, 27 GiB used, 510 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.69k op/s 2024-09-10T19:01:45.076 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:45.109 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:45.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:45.110 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:45.211 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:45.245 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:45.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:45.247 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:45.347 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:45.382 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:45.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:45.383 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:45.484 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:45.511 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:45.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:45.512 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:45.613 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:45.647 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:45.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:45.648 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:45.749 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:45.785 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:45.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:45.786 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:45.887 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:45.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:45.921 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:45.921 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:45.938 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T19:01:45.938 INFO:teuthology.orchestra.run.smithi018.stdout: "ops": [], 2024-09-10T19:01:45.938 INFO:teuthology.orchestra.run.smithi018.stdout: "complaint_time": 30, 2024-09-10T19:01:45.938 INFO:teuthology.orchestra.run.smithi018.stdout: "num_blocked_ops": 0 2024-09-10T19:01:45.938 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T19:01:46.022 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:46.045 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T19:01:46.048 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-09-10T19:01:46.052 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:46.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:46.054 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:46.154 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:46.189 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:46.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:46.191 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:46.291 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:46.327 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:46.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:46.328 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:46.393 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:01:46.428 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:46.460 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:46.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:46.461 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:46.562 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:46.596 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:46.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:46.597 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:46.698 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:46.735 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:46.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:46.737 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:46.837 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:46.869 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:46.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:46.870 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:46.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:46 smithi018 ceph-mon[29336]: pgmap v767: 168 pgs: 168 active+clean; 6.8 GiB data, 26 GiB used, 510 GiB / 536 GiB avail; 946 B/s rd, 0 B/s wr, 1.59k op/s 2024-09-10T19:01:46.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:46 smithi018 ceph-mon[40196]: pgmap v767: 168 pgs: 168 active+clean; 6.8 GiB data, 26 GiB used, 510 GiB / 536 GiB avail; 946 B/s rd, 0 B/s wr, 1.59k op/s 2024-09-10T19:01:46.971 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:47.004 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:47.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:47.005 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:47.018 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:46 smithi103 ceph-mon[33163]: pgmap v767: 168 pgs: 168 active+clean; 6.8 GiB data, 26 GiB used, 510 GiB / 536 GiB avail; 946 B/s rd, 0 B/s wr, 1.59k op/s 2024-09-10T19:01:47.106 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:47.142 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:47.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:47.143 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:47.175 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-10T19:01:47.243 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:47.272 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:01:47 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:01:47.280 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:47.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:47.281 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:47.382 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:47.426 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:47.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:47.428 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:47.528 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:47.572 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:47.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:47.573 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:47.652 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":767,"stamp":"2024-09-10T19:01:45.394359+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":7251254998,"num_objects":110833,"num_object_clones":0,"num_object_copies":332499,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110833,"num_whiteouts":0,"num_read":2119,"num_read_kb":1874,"num_write":167394,"num_write_kb":8874013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22243,"num_bytes_recovered":1440392042,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":60057,"ondisk_log_size":60057,"up":504,"acting":504,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":507,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":562470912,"kb_used":27784792,"kb_used_data":22965224,"kb_used_omap":12,"kb_used_meta":4819507,"kb_avail":534686120,"statfs":{"total":575970213888,"available":547518586880,"internally_reserved":0,"allocated":23516389376,"data_stored":23513888714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13122,"internal_metadata":4935175358},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,6,7,5,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1127743488,"num_objects":-17208,"num_object_clones":0,"num_object_copies":-51624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-17208,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":17208,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.815289"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":363,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.558567+0000","last_change":"2024-09-10T18:57:02.462265+0000","last_active":"2024-09-10T19:01:29.558567+0000","last_peered":"2024-09-10T19:01:29.558567+0000","last_clean":"2024-09-10T19:01:29.558567+0000","last_became_active":"2024-09-10T18:57:02.460451+0000","last_became_peered":"2024-09-10T18:57:02.460451+0000","last_unstale":"2024-09-10T19:01:29.558567+0000","last_undegraded":"2024-09-10T19:01:29.558567+0000","last_fullsized":"2024-09-10T19:01:29.558567+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:14.875872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:14.875872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037478699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":359,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.556951+0000","last_change":"2024-09-10T18:59:59.848814+0000","last_active":"2024-09-10T19:01:29.556951+0000","last_peered":"2024-09-10T19:01:29.556951+0000","last_clean":"2024-09-10T19:01:29.556951+0000","last_became_active":"2024-09-10T18:57:02.440648+0000","last_became_peered":"2024-09-10T18:57:02.440648+0000","last_unstale":"2024-09-10T19:01:29.556951+0000","last_undegraded":"2024-09-10T19:01:29.556951+0000","last_fullsized":"2024-09-10T19:01:29.556951+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:59.848700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:59.848700+0000","last_clean_scrub_stamp":"2024-09-10T18:59:59.848700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047956400000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"232'27","reported_seq":448,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.558318+0000","last_change":"2024-09-10T18:56:02.821637+0000","last_active":"2024-09-10T19:01:29.558318+0000","last_peered":"2024-09-10T19:01:29.558318+0000","last_clean":"2024-09-10T19:01:29.558318+0000","last_became_active":"2024-09-10T18:55:40.394717+0000","last_became_peered":"2024-09-10T18:55:40.394717+0000","last_unstale":"2024-09-10T19:01:29.558318+0000","last_undegraded":"2024-09-10T19:01:29.558318+0000","last_fullsized":"2024-09-10T19:01:29.558318+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'27","last_scrub_stamp":"2024-09-10T18:56:02.821551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:02.821551+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0083110159999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":364,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.560470+0000","last_change":"2024-09-10T19:00:08.841172+0000","last_active":"2024-09-10T19:01:29.560470+0000","last_peered":"2024-09-10T19:01:29.560470+0000","last_clean":"2024-09-10T19:01:29.560470+0000","last_became_active":"2024-09-10T18:45:22.065619+0000","last_became_peered":"2024-09-10T18:45:22.065619+0000","last_unstale":"2024-09-10T19:01:29.560470+0000","last_undegraded":"2024-09-10T19:01:29.560470+0000","last_fullsized":"2024-09-10T19:01:29.560470+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:08.841125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.057932+0000","last_clean_scrub_stamp":"2024-09-10T19:00:08.841125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001059857,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":426,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.559426+0000","last_change":"2024-09-10T19:00:42.519131+0000","last_active":"2024-09-10T19:01:29.559426+0000","last_peered":"2024-09-10T19:01:29.559426+0000","last_clean":"2024-09-10T19:01:29.559426+0000","last_became_active":"2024-09-10T19:00:42.518824+0000","last_became_peered":"2024-09-10T19:00:42.518824+0000","last_unstale":"2024-09-10T19:01:29.559426+0000","last_undegraded":"2024-09-10T19:01:29.559426+0000","last_fullsized":"2024-09-10T19:01:29.559426+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:23.885562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:23.885562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0031033800000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"232'19","reported_seq":385,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.560442+0000","last_change":"2024-09-10T18:59:09.642743+0000","last_active":"2024-09-10T19:01:29.560442+0000","last_peered":"2024-09-10T19:01:29.560442+0000","last_clean":"2024-09-10T19:01:29.560442+0000","last_became_active":"2024-09-10T18:45:20.061266+0000","last_became_peered":"2024-09-10T18:45:20.061266+0000","last_unstale":"2024-09-10T19:01:29.560442+0000","last_undegraded":"2024-09-10T19:01:29.560442+0000","last_fullsized":"2024-09-10T19:01:29.560442+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T18:59:09.642642+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_clean_scrub_stamp":"2024-09-10T18:59:09.642642+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0095920940000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":100,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.554671+0000","last_change":"2024-09-10T19:00:48.452343+0000","last_active":"2024-09-10T19:01:29.554671+0000","last_peered":"2024-09-10T19:01:29.554671+0000","last_clean":"2024-09-10T19:01:29.554671+0000","last_became_active":"2024-09-10T18:57:02.446008+0000","last_became_peered":"2024-09-10T18:57:02.446008+0000","last_unstale":"2024-09-10T19:01:29.554671+0000","last_undegraded":"2024-09-10T19:01:29.554671+0000","last_fullsized":"2024-09-10T19:01:29.554671+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:22.163121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:01.135988+0000","last_clean_scrub_stamp":"2024-09-10T18:56:22.163121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002631478,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":352,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.561747+0000","last_change":"2024-09-10T18:58:51.682046+0000","last_active":"2024-09-10T19:01:29.561747+0000","last_peered":"2024-09-10T19:01:29.561747+0000","last_clean":"2024-09-10T19:01:29.561747+0000","last_became_active":"2024-09-10T18:45:24.081002+0000","last_became_peered":"2024-09-10T18:45:24.081002+0000","last_unstale":"2024-09-10T19:01:29.561747+0000","last_undegraded":"2024-09-10T19:01:29.561747+0000","last_fullsized":"2024-09-10T19:01:29.561747+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:51.681994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:58:51.681994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051776249999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":357,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.564735+0000","last_change":"2024-09-10T18:59:21.502068+0000","last_active":"2024-09-10T19:01:29.564735+0000","last_peered":"2024-09-10T19:01:29.564735+0000","last_clean":"2024-09-10T19:01:29.564735+0000","last_became_active":"2024-09-10T18:57:02.452259+0000","last_became_peered":"2024-09-10T18:57:02.452259+0000","last_unstale":"2024-09-10T19:01:29.564735+0000","last_undegraded":"2024-09-10T19:01:29.564735+0000","last_fullsized":"2024-09-10T19:01:29.564735+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:21.501984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:21.501984+0000","last_clean_scrub_stamp":"2024-09-10T18:59:21.501984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014628558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1b","version":"232'9","reported_seq":371,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.561932+0000","last_change":"2024-09-10T18:59:11.622597+0000","last_active":"2024-09-10T19:01:29.561932+0000","last_peered":"2024-09-10T19:01:29.561932+0000","last_clean":"2024-09-10T19:01:29.561932+0000","last_became_active":"2024-09-10T18:45:20.057567+0000","last_became_peered":"2024-09-10T18:45:20.057567+0000","last_unstale":"2024-09-10T19:01:29.561932+0000","last_undegraded":"2024-09-10T19:01:29.561932+0000","last_fullsized":"2024-09-10T19:01:29.561932+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'9","last_scrub_stamp":"2024-09-10T18:59:11.622509+0000","last_deep_scrub":"163'7","last_deep_scrub_stamp":"2024-09-10T18:51:54.261879+0000","last_clean_scrub_stamp":"2024-09-10T18:59:11.622509+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056120609999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":123,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.561657+0000","last_change":"2024-09-10T19:00:40.947163+0000","last_active":"2024-09-10T19:01:29.561657+0000","last_peered":"2024-09-10T19:01:29.561657+0000","last_clean":"2024-09-10T19:01:29.561657+0000","last_became_active":"2024-09-10T18:58:58.503924+0000","last_became_peered":"2024-09-10T18:58:58.503924+0000","last_unstale":"2024-09-10T19:01:29.561657+0000","last_undegraded":"2024-09-10T19:01:29.561657+0000","last_fullsized":"2024-09-10T19:01:29.561657+0000","mapping_epoch":303,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":304,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:40.947066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:36.509971+0000","last_clean_scrub_stamp":"2024-09-10T19:00:40.947066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:02:09.283651+0000","scrub_duration":0.012293715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":30,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.559426+0000","last_change":"2024-09-10T19:00:42.516918+0000","last_active":"2024-09-10T19:01:29.559426+0000","last_peered":"2024-09-10T19:01:29.559426+0000","last_clean":"2024-09-10T19:01:29.559426+0000","last_became_active":"2024-09-10T19:00:42.515840+0000","last_became_peered":"2024-09-10T19:00:42.515840+0000","last_unstale":"2024-09-10T19:01:29.559426+0000","last_undegraded":"2024-09-10T19:01:29.559426+0000","last_fullsized":"2024-09-10T19:01:29.559426+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:48.230459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:59:48.230459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":163,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.559869+0000","last_change":"2024-09-10T18:58:49.506212+0000","last_active":"2024-09-10T19:01:29.559869+0000","last_peered":"2024-09-10T19:01:29.559869+0000","last_clean":"2024-09-10T19:01:29.559869+0000","last_became_active":"2024-09-10T18:57:02.440497+0000","last_became_peered":"2024-09-10T18:57:02.440497+0000","last_unstale":"2024-09-10T19:01:29.559869+0000","last_undegraded":"2024-09-10T19:01:29.559869+0000","last_fullsized":"2024-09-10T19:01:29.559869+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:49.506160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:58:49.506160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.018244065,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"232'15","reported_seq":410,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.562063+0000","last_change":"2024-09-10T19:01:08.981911+0000","last_active":"2024-09-10T19:01:29.562063+0000","last_peered":"2024-09-10T19:01:29.562063+0000","last_clean":"2024-09-10T19:01:29.562063+0000","last_became_active":"2024-09-10T18:55:40.391571+0000","last_became_peered":"2024-09-10T18:55:40.391571+0000","last_unstale":"2024-09-10T19:01:29.562063+0000","last_undegraded":"2024-09-10T19:01:29.562063+0000","last_fullsized":"2024-09-10T19:01:29.562063+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T18:56:26.480105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:26.480105+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027886920000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":393,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.559432+0000","last_change":"2024-09-10T18:59:22.643043+0000","last_active":"2024-09-10T19:01:29.559432+0000","last_peered":"2024-09-10T19:01:29.559432+0000","last_clean":"2024-09-10T19:01:29.559432+0000","last_became_active":"2024-09-10T18:57:02.431585+0000","last_became_peered":"2024-09-10T18:57:02.431585+0000","last_unstale":"2024-09-10T19:01:29.559432+0000","last_undegraded":"2024-09-10T19:01:29.559432+0000","last_fullsized":"2024-09-10T19:01:29.559432+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:22.642961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:59:22.642961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.018189153999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":48,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.555753+0000","last_change":"2024-09-10T18:59:48.738143+0000","last_active":"2024-09-10T19:01:29.555753+0000","last_peered":"2024-09-10T19:01:29.555753+0000","last_clean":"2024-09-10T19:01:29.555753+0000","last_became_active":"2024-09-10T18:59:48.737788+0000","last_became_peered":"2024-09-10T18:59:48.737788+0000","last_unstale":"2024-09-10T19:01:29.555753+0000","last_undegraded":"2024-09-10T19:01:29.555753+0000","last_fullsized":"2024-09-10T19:01:29.555753+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:51.561010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:59.425718+0000","last_clean_scrub_stamp":"2024-09-10T18:58:51.561010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3","version":"359'10191","reported_seq":10243,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:41.566516+0000","last_change":"2024-09-10T18:59:43.007892+0000","last_active":"2024-09-10T19:01:41.566516+0000","last_peered":"2024-09-10T19:01:41.566516+0000","last_clean":"2024-09-10T19:01:41.566516+0000","last_became_active":"2024-09-10T18:59:43.007585+0000","last_became_peered":"2024-09-10T18:59:43.007585+0000","last_unstale":"2024-09-10T19:01:41.566516+0000","last_undegraded":"2024-09-10T19:01:41.566516+0000","last_fullsized":"2024-09-10T19:01:41.566516+0000","mapping_epoch":315,"log_start":"345'6599","ondisk_log_start":"345'6599","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3592,"log_dups_size":0,"ondisk_log_size":3592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":455540736,"num_objects":6951,"num_object_clones":0,"num_object_copies":20853,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10191,"num_write_kb":548544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.8","version":"70'81","reported_seq":34,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.558912+0000","last_change":"2024-09-10T19:00:42.527220+0000","last_active":"2024-09-10T19:01:29.558912+0000","last_peered":"2024-09-10T19:01:29.558912+0000","last_clean":"2024-09-10T19:01:29.558912+0000","last_became_active":"2024-09-10T19:00:42.517072+0000","last_became_peered":"2024-09-10T19:00:42.517072+0000","last_unstale":"2024-09-10T19:01:29.558912+0000","last_undegraded":"2024-09-10T19:01:29.558912+0000","last_fullsized":"2024-09-10T19:01:29.558912+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:51.167302+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:59:51.167302+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015031440000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":262688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"232'39","reported_seq":451,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.562786+0000","last_change":"2024-09-10T18:59:47.745859+0000","last_active":"2024-09-10T19:01:29.562786+0000","last_peered":"2024-09-10T19:01:29.562786+0000","last_clean":"2024-09-10T19:01:29.562786+0000","last_became_active":"2024-09-10T18:47:36.409300+0000","last_became_peered":"2024-09-10T18:47:36.409300+0000","last_unstale":"2024-09-10T19:01:29.562786+0000","last_undegraded":"2024-09-10T19:01:29.562786+0000","last_fullsized":"2024-09-10T19:01:29.562786+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'39","last_scrub_stamp":"2024-09-10T18:59:47.745766+0000","last_deep_scrub":"78'25","last_deep_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_clean_scrub_stamp":"2024-09-10T18:59:47.745766+0000","objects_scrubbed":8,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011279000000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":110,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.562276+0000","last_change":"2024-09-10T19:00:09.867278+0000","last_active":"2024-09-10T19:01:29.562276+0000","last_peered":"2024-09-10T19:01:29.562276+0000","last_clean":"2024-09-10T19:01:29.562276+0000","last_became_active":"2024-09-10T18:57:02.437260+0000","last_became_peered":"2024-09-10T18:57:02.437260+0000","last_unstale":"2024-09-10T19:01:29.562276+0000","last_undegraded":"2024-09-10T19:01:29.562276+0000","last_fullsized":"2024-09-10T19:01:29.562276+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:09.867183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:00:09.867183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026871460000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":373,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.556007+0000","last_change":"2024-09-10T19:00:09.470422+0000","last_active":"2024-09-10T19:01:29.556007+0000","last_peered":"2024-09-10T19:01:29.556007+0000","last_clean":"2024-09-10T19:01:29.556007+0000","last_became_active":"2024-09-10T18:55:40.392449+0000","last_became_peered":"2024-09-10T18:55:40.392449+0000","last_unstale":"2024-09-10T19:01:29.556007+0000","last_undegraded":"2024-09-10T19:01:29.556007+0000","last_fullsized":"2024-09-10T19:01:29.556007+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:09.470293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:04.512425+0000","last_clean_scrub_stamp":"2024-09-10T19:00:09.470293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0073759949999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":271,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.557419+0000","last_change":"2024-09-10T18:58:49.524134+0000","last_active":"2024-09-10T19:01:29.557419+0000","last_peered":"2024-09-10T19:01:29.557419+0000","last_clean":"2024-09-10T19:01:29.557419+0000","last_became_active":"2024-09-10T18:55:40.388154+0000","last_became_peered":"2024-09-10T18:55:40.388154+0000","last_unstale":"2024-09-10T19:01:29.557419+0000","last_undegraded":"2024-09-10T19:01:29.557419+0000","last_fullsized":"2024-09-10T19:01:29.557419+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:49.524015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:58:49.524015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005180479,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2","version":"359'10593","reported_seq":10645,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:41.805372+0000","last_change":"2024-09-10T18:59:42.709056+0000","last_active":"2024-09-10T19:01:41.805372+0000","last_peered":"2024-09-10T19:01:41.805372+0000","last_clean":"2024-09-10T19:01:41.805372+0000","last_became_active":"2024-09-10T18:59:42.708732+0000","last_became_peered":"2024-09-10T18:59:42.708732+0000","last_unstale":"2024-09-10T19:01:41.805372+0000","last_undegraded":"2024-09-10T19:01:41.805372+0000","last_fullsized":"2024-09-10T19:01:41.805372+0000","mapping_epoch":315,"log_start":"347'7100","ondisk_log_start":"347'7100","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3493,"log_dups_size":0,"ondisk_log_size":3493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":460390400,"num_objects":7025,"num_object_clones":0,"num_object_copies":21075,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10593,"num_write_kb":563776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.9","version":"70'81","reported_seq":267,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.558498+0000","last_change":"2024-09-10T18:59:56.164487+0000","last_active":"2024-09-10T19:01:29.558498+0000","last_peered":"2024-09-10T19:01:29.558498+0000","last_clean":"2024-09-10T19:01:29.558498+0000","last_became_active":"2024-09-10T18:57:02.463750+0000","last_became_peered":"2024-09-10T18:57:02.463750+0000","last_unstale":"2024-09-10T19:01:29.558498+0000","last_undegraded":"2024-09-10T19:01:29.558498+0000","last_fullsized":"2024-09-10T19:01:29.558498+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:56.164384+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:59:56.164384+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041056720000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"232'25","reported_seq":391,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.562650+0000","last_change":"2024-09-10T18:56:32.437605+0000","last_active":"2024-09-10T19:01:29.562650+0000","last_peered":"2024-09-10T19:01:29.562650+0000","last_clean":"2024-09-10T19:01:29.562650+0000","last_became_active":"2024-09-10T18:45:20.053428+0000","last_became_peered":"2024-09-10T18:45:20.053428+0000","last_unstale":"2024-09-10T19:01:29.562650+0000","last_undegraded":"2024-09-10T19:01:29.562650+0000","last_fullsized":"2024-09-10T19:01:29.562650+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'25","last_scrub_stamp":"2024-09-10T18:56:32.437503+0000","last_deep_scrub":"116'15","last_deep_scrub_stamp":"2024-09-10T18:49:12.374527+0000","last_clean_scrub_stamp":"2024-09-10T18:56:32.437503+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039673679999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":373,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.555466+0000","last_change":"2024-09-10T19:00:51.441092+0000","last_active":"2024-09-10T19:01:29.555466+0000","last_peered":"2024-09-10T19:01:29.555466+0000","last_clean":"2024-09-10T19:01:29.555466+0000","last_became_active":"2024-09-10T18:57:02.440442+0000","last_became_peered":"2024-09-10T18:57:02.440442+0000","last_unstale":"2024-09-10T19:01:29.555466+0000","last_undegraded":"2024-09-10T19:01:29.555466+0000","last_fullsized":"2024-09-10T19:01:29.555466+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:29.802937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:29.802937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044818080000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"273'13","reported_seq":417,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:36.268603+0000","last_change":"2024-09-10T18:57:02.463853+0000","last_active":"2024-09-10T19:01:36.268603+0000","last_peered":"2024-09-10T19:01:36.268603+0000","last_clean":"2024-09-10T19:01:36.268603+0000","last_became_active":"2024-09-10T18:57:02.448258+0000","last_became_peered":"2024-09-10T18:57:02.448258+0000","last_unstale":"2024-09-10T19:01:36.268603+0000","last_undegraded":"2024-09-10T19:01:36.268603+0000","last_fullsized":"2024-09-10T19:01:36.268603+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"242'12","last_scrub_stamp":"2024-09-10T18:56:27.995682+0000","last_deep_scrub":"202'10","last_deep_scrub_stamp":"2024-09-10T18:53:45.547686+0000","last_clean_scrub_stamp":"2024-09-10T18:56:27.995682+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025134200000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":402,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.562929+0000","last_change":"2024-09-10T19:00:42.515842+0000","last_active":"2024-09-10T19:01:29.562929+0000","last_peered":"2024-09-10T19:01:29.562929+0000","last_clean":"2024-09-10T19:01:29.562929+0000","last_became_active":"2024-09-10T19:00:42.515523+0000","last_became_peered":"2024-09-10T19:00:42.515523+0000","last_unstale":"2024-09-10T19:01:29.562929+0000","last_undegraded":"2024-09-10T19:01:29.562929+0000","last_fullsized":"2024-09-10T19:01:29.562929+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:07.803916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:32.043666+0000","last_clean_scrub_stamp":"2024-09-10T19:00:07.803916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0011072429999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1","version":"359'10316","reported_seq":10364,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:41.275361+0000","last_change":"2024-09-10T18:59:42.711934+0000","last_active":"2024-09-10T19:01:41.275361+0000","last_peered":"2024-09-10T19:01:41.275361+0000","last_clean":"2024-09-10T19:01:41.275361+0000","last_became_active":"2024-09-10T18:59:42.711662+0000","last_became_peered":"2024-09-10T18:59:42.711662+0000","last_unstale":"2024-09-10T19:01:41.275361+0000","last_undegraded":"2024-09-10T19:01:41.275361+0000","last_fullsized":"2024-09-10T19:01:41.275361+0000","mapping_epoch":315,"log_start":"346'6700","ondisk_log_start":"346'6700","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3616,"log_dups_size":0,"ondisk_log_size":3616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":453902336,"num_objects":6926,"num_object_clones":0,"num_object_copies":20778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":551744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1],"acting":[4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"70'81","reported_seq":281,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.560411+0000","last_change":"2024-09-10T18:59:11.493175+0000","last_active":"2024-09-10T19:01:29.560411+0000","last_peered":"2024-09-10T19:01:29.560411+0000","last_clean":"2024-09-10T19:01:29.560411+0000","last_became_active":"2024-09-10T18:50:08.398345+0000","last_became_peered":"2024-09-10T18:50:08.398345+0000","last_unstale":"2024-09-10T19:01:29.560411+0000","last_undegraded":"2024-09-10T19:01:29.560411+0000","last_fullsized":"2024-09-10T19:01:29.560411+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:11.493086+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:55:37.921519+0000","last_clean_scrub_stamp":"2024-09-10T18:59:11.493086+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013358479999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"232'34","reported_seq":388,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.560504+0000","last_change":"2024-09-10T19:00:41.482710+0000","last_active":"2024-09-10T19:01:29.560504+0000","last_peered":"2024-09-10T19:01:29.560504+0000","last_clean":"2024-09-10T19:01:29.560504+0000","last_became_active":"2024-09-10T18:46:56.742946+0000","last_became_peered":"2024-09-10T18:46:56.742946+0000","last_unstale":"2024-09-10T19:01:29.560504+0000","last_undegraded":"2024-09-10T19:01:29.560504+0000","last_fullsized":"2024-09-10T19:01:29.560504+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'34","last_scrub_stamp":"2024-09-10T19:00:41.482619+0000","last_deep_scrub":"141'22","last_deep_scrub_stamp":"2024-09-10T18:50:57.119152+0000","last_clean_scrub_stamp":"2024-09-10T19:00:41.482619+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:02:06.038176+0000","scrub_duration":0.0069363309999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":383,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.555103+0000","last_change":"2024-09-10T18:59:51.286189+0000","last_active":"2024-09-10T19:01:29.555103+0000","last_peered":"2024-09-10T19:01:29.555103+0000","last_clean":"2024-09-10T19:01:29.555103+0000","last_became_active":"2024-09-10T18:57:02.445848+0000","last_became_peered":"2024-09-10T18:57:02.445848+0000","last_unstale":"2024-09-10T19:01:29.555103+0000","last_undegraded":"2024-09-10T19:01:29.555103+0000","last_fullsized":"2024-09-10T19:01:29.555103+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:51.286131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_clean_scrub_stamp":"2024-09-10T18:59:51.286131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015180669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"242'10","reported_seq":578,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:36.267187+0000","last_change":"2024-09-10T18:56:19.581070+0000","last_active":"2024-09-10T19:01:36.267187+0000","last_peered":"2024-09-10T19:01:36.267187+0000","last_clean":"2024-09-10T19:01:36.267187+0000","last_became_active":"2024-09-10T18:55:40.397246+0000","last_became_peered":"2024-09-10T18:55:40.397246+0000","last_unstale":"2024-09-10T19:01:36.267187+0000","last_undegraded":"2024-09-10T19:01:36.267187+0000","last_fullsized":"2024-09-10T19:01:36.267187+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"242'10","last_scrub_stamp":"2024-09-10T18:56:19.581012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:19.581012+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013423529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":381,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.559782+0000","last_change":"2024-09-10T18:59:10.477418+0000","last_active":"2024-09-10T19:01:29.559782+0000","last_peered":"2024-09-10T19:01:29.559782+0000","last_clean":"2024-09-10T19:01:29.559782+0000","last_became_active":"2024-09-10T18:55:40.392003+0000","last_became_peered":"2024-09-10T18:55:40.392003+0000","last_unstale":"2024-09-10T19:01:29.559782+0000","last_undegraded":"2024-09-10T19:01:29.559782+0000","last_fullsized":"2024-09-10T19:01:29.559782+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:10.477346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:54.218602+0000","last_clean_scrub_stamp":"2024-09-10T18:59:10.477346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004313005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"359'10452","reported_seq":10504,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:41.486920+0000","last_change":"2024-09-10T18:59:42.715764+0000","last_active":"2024-09-10T19:01:41.486920+0000","last_peered":"2024-09-10T19:01:41.486920+0000","last_clean":"2024-09-10T19:01:41.486920+0000","last_became_active":"2024-09-10T18:59:42.714808+0000","last_became_peered":"2024-09-10T18:59:42.714808+0000","last_unstale":"2024-09-10T19:01:41.486920+0000","last_undegraded":"2024-09-10T19:01:41.486920+0000","last_fullsized":"2024-09-10T19:01:41.486920+0000","mapping_epoch":315,"log_start":"347'6900","ondisk_log_start":"347'6900","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3552,"log_dups_size":0,"ondisk_log_size":3552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":463863808,"num_objects":7078,"num_object_clones":0,"num_object_copies":21234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10452,"num_write_kb":560960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.b","version":"70'81","reported_seq":326,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.555476+0000","last_change":"2024-09-10T19:01:04.457442+0000","last_active":"2024-09-10T19:01:29.555476+0000","last_peered":"2024-09-10T19:01:29.555476+0000","last_clean":"2024-09-10T19:01:29.555476+0000","last_became_active":"2024-09-10T19:00:42.518107+0000","last_became_peered":"2024-09-10T19:00:42.518107+0000","last_unstale":"2024-09-10T19:01:29.555476+0000","last_undegraded":"2024-09-10T19:01:29.555476+0000","last_fullsized":"2024-09-10T19:01:29.555476+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:33.793296+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:33.793296+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045437230000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"232'24","reported_seq":468,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.563205+0000","last_change":"2024-09-10T18:59:06.595605+0000","last_active":"2024-09-10T19:01:29.563205+0000","last_peered":"2024-09-10T19:01:29.563205+0000","last_clean":"2024-09-10T19:01:29.563205+0000","last_became_active":"2024-09-10T18:47:57.106935+0000","last_became_peered":"2024-09-10T18:47:57.106935+0000","last_unstale":"2024-09-10T19:01:29.563205+0000","last_undegraded":"2024-09-10T19:01:29.563205+0000","last_fullsized":"2024-09-10T19:01:29.563205+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T18:59:06.595554+0000","last_deep_scrub":"224'16","last_deep_scrub_stamp":"2024-09-10T18:54:49.296957+0000","last_clean_scrub_stamp":"2024-09-10T18:59:06.595554+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020642540000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":354,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.556243+0000","last_change":"2024-09-10T18:57:02.444295+0000","last_active":"2024-09-10T19:01:29.556243+0000","last_peered":"2024-09-10T19:01:29.556243+0000","last_clean":"2024-09-10T19:01:29.556243+0000","last_became_active":"2024-09-10T18:57:02.443816+0000","last_became_peered":"2024-09-10T18:57:02.443816+0000","last_unstale":"2024-09-10T19:01:29.556243+0000","last_undegraded":"2024-09-10T19:01:29.556243+0000","last_fullsized":"2024-09-10T19:01:29.556243+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:29.994533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:29.994533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053276879999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":369,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.560605+0000","last_change":"2024-09-10T18:58:54.548235+0000","last_active":"2024-09-10T19:01:29.560605+0000","last_peered":"2024-09-10T19:01:29.560605+0000","last_clean":"2024-09-10T19:01:29.560605+0000","last_became_active":"2024-09-10T18:45:22.068918+0000","last_became_peered":"2024-09-10T18:45:22.068918+0000","last_unstale":"2024-09-10T19:01:29.560605+0000","last_undegraded":"2024-09-10T19:01:29.560605+0000","last_fullsized":"2024-09-10T19:01:29.560605+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:54.548153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_clean_scrub_stamp":"2024-09-10T18:58:54.548153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013051650000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":353,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.559443+0000","last_change":"2024-09-10T18:59:13.892307+0000","last_active":"2024-09-10T19:01:29.559443+0000","last_peered":"2024-09-10T19:01:29.559443+0000","last_clean":"2024-09-10T19:01:29.559443+0000","last_became_active":"2024-09-10T18:45:24.264538+0000","last_became_peered":"2024-09-10T18:45:24.264538+0000","last_unstale":"2024-09-10T19:01:29.559443+0000","last_undegraded":"2024-09-10T19:01:29.559443+0000","last_fullsized":"2024-09-10T19:01:29.559443+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:13.892129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:51.398981+0000","last_clean_scrub_stamp":"2024-09-10T18:59:13.892129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007612214,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"359'10376","reported_seq":10428,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:41.481384+0000","last_change":"2024-09-10T18:59:43.008094+0000","last_active":"2024-09-10T19:01:41.481384+0000","last_peered":"2024-09-10T19:01:41.481384+0000","last_clean":"2024-09-10T19:01:41.481384+0000","last_became_active":"2024-09-10T18:59:43.007838+0000","last_became_peered":"2024-09-10T18:59:43.007838+0000","last_unstale":"2024-09-10T19:01:41.481384+0000","last_undegraded":"2024-09-10T19:01:41.481384+0000","last_fullsized":"2024-09-10T19:01:41.481384+0000","mapping_epoch":315,"log_start":"346'6800","ondisk_log_start":"346'6800","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3576,"log_dups_size":0,"ondisk_log_size":3576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":454819840,"num_objects":6940,"num_object_clones":0,"num_object_copies":20820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10376,"num_write_kb":554112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"70'81","reported_seq":311,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.559578+0000","last_change":"2024-09-10T19:00:42.520084+0000","last_active":"2024-09-10T19:01:29.559578+0000","last_peered":"2024-09-10T19:01:29.559578+0000","last_clean":"2024-09-10T19:01:29.559578+0000","last_became_active":"2024-09-10T19:00:42.519742+0000","last_became_peered":"2024-09-10T19:00:42.519742+0000","last_unstale":"2024-09-10T19:01:29.559578+0000","last_undegraded":"2024-09-10T19:01:29.559578+0000","last_fullsized":"2024-09-10T19:01:29.559578+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:58:56.912936+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:58:56.912936+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045874109999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"232'35","reported_seq":405,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.561989+0000","last_change":"2024-09-10T18:56:28.475224+0000","last_active":"2024-09-10T19:01:29.561989+0000","last_peered":"2024-09-10T19:01:29.561989+0000","last_clean":"2024-09-10T19:01:29.561989+0000","last_became_active":"2024-09-10T18:45:20.052243+0000","last_became_peered":"2024-09-10T18:45:20.052243+0000","last_unstale":"2024-09-10T19:01:29.561989+0000","last_undegraded":"2024-09-10T19:01:29.561989+0000","last_fullsized":"2024-09-10T19:01:29.561989+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'35","last_scrub_stamp":"2024-09-10T18:56:28.475142+0000","last_deep_scrub":"143'21","last_deep_scrub_stamp":"2024-09-10T18:51:02.540216+0000","last_clean_scrub_stamp":"2024-09-10T18:56:28.475142+0000","objects_scrubbed":7,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026978649999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":391,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.564441+0000","last_change":"2024-09-10T18:57:02.453766+0000","last_active":"2024-09-10T19:01:29.564441+0000","last_peered":"2024-09-10T19:01:29.564441+0000","last_clean":"2024-09-10T19:01:29.564441+0000","last_became_active":"2024-09-10T18:57:02.453404+0000","last_became_peered":"2024-09-10T18:57:02.453404+0000","last_unstale":"2024-09-10T19:01:29.564441+0000","last_undegraded":"2024-09-10T19:01:29.564441+0000","last_fullsized":"2024-09-10T19:01:29.564441+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:29.226803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_clean_scrub_stamp":"2024-09-10T18:56:29.226803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0086984899999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":559,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.588710+0000","last_change":"2024-09-10T18:56:17.914259+0000","last_active":"2024-09-10T19:01:34.588710+0000","last_peered":"2024-09-10T19:01:34.588710+0000","last_clean":"2024-09-10T19:01:34.588710+0000","last_became_active":"2024-09-10T18:45:22.074540+0000","last_became_peered":"2024-09-10T18:45:22.074540+0000","last_unstale":"2024-09-10T19:01:34.588710+0000","last_undegraded":"2024-09-10T19:01:34.588710+0000","last_fullsized":"2024-09-10T19:01:34.588710+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:56:17.914175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:17.914175+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026666329999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":158,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.563944+0000","last_change":"2024-09-10T18:58:50.638774+0000","last_active":"2024-09-10T19:01:29.563944+0000","last_peered":"2024-09-10T19:01:29.563944+0000","last_clean":"2024-09-10T19:01:29.563944+0000","last_became_active":"2024-09-10T18:57:02.437100+0000","last_became_peered":"2024-09-10T18:57:02.437100+0000","last_unstale":"2024-09-10T19:01:29.563944+0000","last_undegraded":"2024-09-10T19:01:29.563944+0000","last_fullsized":"2024-09-10T19:01:29.563944+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:50.638720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:58:50.638720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003753546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.e","version":"359'10253","reported_seq":10305,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:41.304854+0000","last_change":"2024-09-10T18:59:42.713933+0000","last_active":"2024-09-10T19:01:41.304854+0000","last_peered":"2024-09-10T19:01:41.304854+0000","last_clean":"2024-09-10T19:01:41.304854+0000","last_became_active":"2024-09-10T18:59:42.713605+0000","last_became_peered":"2024-09-10T18:59:42.713605+0000","last_unstale":"2024-09-10T19:01:41.304854+0000","last_undegraded":"2024-09-10T19:01:41.304854+0000","last_fullsized":"2024-09-10T19:01:41.304854+0000","mapping_epoch":315,"log_start":"346'6700","ondisk_log_start":"346'6700","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3553,"log_dups_size":0,"ondisk_log_size":3553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":452788224,"num_objects":6909,"num_object_clones":0,"num_object_copies":20727,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10253,"num_write_kb":549184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"70'81","reported_seq":276,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.562441+0000","last_change":"2024-09-10T18:56:31.481362+0000","last_active":"2024-09-10T19:01:29.562441+0000","last_peered":"2024-09-10T19:01:29.562441+0000","last_clean":"2024-09-10T19:01:29.562441+0000","last_became_active":"2024-09-10T18:52:14.584798+0000","last_became_peered":"2024-09-10T18:52:14.584798+0000","last_unstale":"2024-09-10T19:01:29.562441+0000","last_undegraded":"2024-09-10T19:01:29.562441+0000","last_fullsized":"2024-09-10T19:01:29.562441+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:31.481303+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:56:31.481303+0000","last_clean_scrub_stamp":"2024-09-10T18:56:31.481303+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0011170959999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"232'30","reported_seq":523,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.558310+0000","last_change":"2024-09-10T18:59:48.754412+0000","last_active":"2024-09-10T19:01:29.558310+0000","last_peered":"2024-09-10T19:01:29.558310+0000","last_clean":"2024-09-10T19:01:29.558310+0000","last_became_active":"2024-09-10T18:59:48.742377+0000","last_became_peered":"2024-09-10T18:59:48.742377+0000","last_unstale":"2024-09-10T19:01:29.558310+0000","last_undegraded":"2024-09-10T19:01:29.558310+0000","last_fullsized":"2024-09-10T19:01:29.558310+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'30","last_scrub_stamp":"2024-09-10T18:56:15.875924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:15.875924+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0075153879999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":363,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.556841+0000","last_change":"2024-09-10T18:59:05.246945+0000","last_active":"2024-09-10T19:01:29.556841+0000","last_peered":"2024-09-10T19:01:29.556841+0000","last_clean":"2024-09-10T19:01:29.556841+0000","last_became_active":"2024-09-10T18:45:17.961695+0000","last_became_peered":"2024-09-10T18:45:17.961695+0000","last_unstale":"2024-09-10T19:01:29.556841+0000","last_undegraded":"2024-09-10T19:01:29.556841+0000","last_fullsized":"2024-09-10T19:01:29.556841+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:05.246629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:46.142129+0000","last_clean_scrub_stamp":"2024-09-10T18:59:05.246629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0072716029999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":374,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.561647+0000","last_change":"2024-09-10T18:59:57.903394+0000","last_active":"2024-09-10T19:01:29.561647+0000","last_peered":"2024-09-10T19:01:29.561647+0000","last_clean":"2024-09-10T19:01:29.561647+0000","last_became_active":"2024-09-10T18:57:02.448126+0000","last_became_peered":"2024-09-10T18:57:02.448126+0000","last_unstale":"2024-09-10T19:01:29.561647+0000","last_undegraded":"2024-09-10T19:01:29.561647+0000","last_fullsized":"2024-09-10T19:01:29.561647+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:57.903311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:57.903311+0000","last_clean_scrub_stamp":"2024-09-10T18:59:57.903311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011228738,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":362,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.561998+0000","last_change":"2024-09-10T18:58:48.633501+0000","last_active":"2024-09-10T19:01:29.561998+0000","last_peered":"2024-09-10T19:01:29.561998+0000","last_clean":"2024-09-10T19:01:29.561998+0000","last_became_active":"2024-09-10T18:47:36.410836+0000","last_became_peered":"2024-09-10T18:47:36.410836+0000","last_unstale":"2024-09-10T19:01:29.561998+0000","last_undegraded":"2024-09-10T19:01:29.561998+0000","last_fullsized":"2024-09-10T19:01:29.561998+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:48.633423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_clean_scrub_stamp":"2024-09-10T18:58:48.633423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0079262829999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.d","version":"359'10501","reported_seq":10554,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:41.488460+0000","last_change":"2024-09-10T18:59:42.717015+0000","last_active":"2024-09-10T19:01:41.488460+0000","last_peered":"2024-09-10T19:01:41.488460+0000","last_clean":"2024-09-10T19:01:41.488460+0000","last_became_active":"2024-09-10T18:59:42.716708+0000","last_became_peered":"2024-09-10T18:59:42.716708+0000","last_unstale":"2024-09-10T19:01:41.488460+0000","last_undegraded":"2024-09-10T19:01:41.488460+0000","last_fullsized":"2024-09-10T19:01:41.488460+0000","mapping_epoch":315,"log_start":"347'7000","ondisk_log_start":"347'7000","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3501,"log_dups_size":0,"ondisk_log_size":3501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":460718104,"num_objects":7031,"num_object_clones":0,"num_object_copies":21093,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7031,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10501,"num_write_kb":560961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"70'81","reported_seq":374,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.564445+0000","last_change":"2024-09-10T18:59:23.481986+0000","last_active":"2024-09-10T19:01:29.564445+0000","last_peered":"2024-09-10T19:01:29.564445+0000","last_clean":"2024-09-10T19:01:29.564445+0000","last_became_active":"2024-09-10T18:55:40.396828+0000","last_became_peered":"2024-09-10T18:55:40.396828+0000","last_unstale":"2024-09-10T19:01:29.564445+0000","last_undegraded":"2024-09-10T19:01:29.564445+0000","last_fullsized":"2024-09-10T19:01:29.564445+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:23.481902+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:59:23.481902+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0052121340000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"232'29","reported_seq":416,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.556240+0000","last_change":"2024-09-10T18:58:58.837250+0000","last_active":"2024-09-10T19:01:29.556240+0000","last_peered":"2024-09-10T19:01:29.556240+0000","last_clean":"2024-09-10T19:01:29.556240+0000","last_became_active":"2024-09-10T18:57:02.442862+0000","last_became_peered":"2024-09-10T18:57:02.442862+0000","last_unstale":"2024-09-10T19:01:29.556240+0000","last_undegraded":"2024-09-10T19:01:29.556240+0000","last_fullsized":"2024-09-10T19:01:29.556240+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'29","last_scrub_stamp":"2024-09-10T18:58:58.837163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:58:58.837163+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.019169703999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":354,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.556601+0000","last_change":"2024-09-10T18:59:53.858914+0000","last_active":"2024-09-10T19:01:29.556601+0000","last_peered":"2024-09-10T19:01:29.556601+0000","last_clean":"2024-09-10T19:01:29.556601+0000","last_became_active":"2024-09-10T18:57:02.443522+0000","last_became_peered":"2024-09-10T18:57:02.443522+0000","last_unstale":"2024-09-10T19:01:29.556601+0000","last_undegraded":"2024-09-10T19:01:29.556601+0000","last_fullsized":"2024-09-10T19:01:29.556601+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:53.858807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:48.196065+0000","last_clean_scrub_stamp":"2024-09-10T18:59:53.858807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044256060000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":384,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.563794+0000","last_change":"2024-09-10T18:59:15.511792+0000","last_active":"2024-09-10T19:01:29.563794+0000","last_peered":"2024-09-10T19:01:29.563794+0000","last_clean":"2024-09-10T19:01:29.563794+0000","last_became_active":"2024-09-10T18:55:40.387857+0000","last_became_peered":"2024-09-10T18:55:40.387857+0000","last_unstale":"2024-09-10T19:01:29.563794+0000","last_undegraded":"2024-09-10T19:01:29.563794+0000","last_fullsized":"2024-09-10T19:01:29.563794+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:15.511743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:59:15.511743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001893837,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":374,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.554895+0000","last_change":"2024-09-10T18:59:16.235312+0000","last_active":"2024-09-10T19:01:29.554895+0000","last_peered":"2024-09-10T19:01:29.554895+0000","last_clean":"2024-09-10T19:01:29.554895+0000","last_became_active":"2024-09-10T18:57:02.442479+0000","last_became_peered":"2024-09-10T18:57:02.442479+0000","last_unstale":"2024-09-10T19:01:29.554895+0000","last_undegraded":"2024-09-10T19:01:29.554895+0000","last_fullsized":"2024-09-10T19:01:29.554895+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:16.235235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:49:18.281230+0000","last_clean_scrub_stamp":"2024-09-10T18:59:16.235235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034192950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"359'10242","reported_seq":11881,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:41.798005+0000","last_change":"2024-09-10T19:00:08.469551+0000","last_active":"2024-09-10T19:01:41.798005+0000","last_peered":"2024-09-10T19:01:41.798005+0000","last_clean":"2024-09-10T19:01:41.798005+0000","last_became_active":"2024-09-10T19:00:08.465795+0000","last_became_peered":"2024-09-10T19:00:08.465795+0000","last_unstale":"2024-09-10T19:01:41.798005+0000","last_undegraded":"2024-09-10T19:01:41.798005+0000","last_fullsized":"2024-09-10T19:01:41.798005+0000","mapping_epoch":329,"log_start":"346'6700","ondisk_log_start":"346'6700","created":315,"last_epoch_clean":330,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3542,"log_dups_size":0,"ondisk_log_size":3542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":444334080,"num_objects":6780,"num_object_clones":0,"num_object_copies":20340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10242,"num_write_kb":544704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":784,"num_bytes_recovered":51380224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"70'81","reported_seq":377,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.564814+0000","last_change":"2024-09-10T18:59:53.550489+0000","last_active":"2024-09-10T19:01:29.564814+0000","last_peered":"2024-09-10T19:01:29.564814+0000","last_clean":"2024-09-10T19:01:29.564814+0000","last_became_active":"2024-09-10T18:57:02.452659+0000","last_became_peered":"2024-09-10T18:57:02.452659+0000","last_unstale":"2024-09-10T19:01:29.564814+0000","last_undegraded":"2024-09-10T19:01:29.564814+0000","last_fullsized":"2024-09-10T19:01:29.564814+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:53.550433+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:59:53.550433+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013042399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"232'25","reported_seq":477,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.555238+0000","last_change":"2024-09-10T19:00:42.531653+0000","last_active":"2024-09-10T19:01:29.555238+0000","last_peered":"2024-09-10T19:01:29.555238+0000","last_clean":"2024-09-10T19:01:29.555238+0000","last_became_active":"2024-09-10T19:00:42.517353+0000","last_became_peered":"2024-09-10T19:00:42.517353+0000","last_unstale":"2024-09-10T19:01:29.555238+0000","last_undegraded":"2024-09-10T19:01:29.555238+0000","last_fullsized":"2024-09-10T19:01:29.555238+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'25","last_scrub_stamp":"2024-09-10T19:00:07.390135+0000","last_deep_scrub":"83'15","last_deep_scrub_stamp":"2024-09-10T18:46:58.436473+0000","last_clean_scrub_stamp":"2024-09-10T19:00:07.390135+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002339874,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":400,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.556331+0000","last_change":"2024-09-10T19:01:14.526223+0000","last_active":"2024-09-10T19:01:29.556331+0000","last_peered":"2024-09-10T19:01:29.556331+0000","last_clean":"2024-09-10T19:01:29.556331+0000","last_became_active":"2024-09-10T18:57:02.447940+0000","last_became_peered":"2024-09-10T18:57:02.447940+0000","last_unstale":"2024-09-10T19:01:29.556331+0000","last_undegraded":"2024-09-10T19:01:29.556331+0000","last_fullsized":"2024-09-10T19:01:29.556331+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:50.137393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:05.140027+0000","last_clean_scrub_stamp":"2024-09-10T18:58:50.137393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014454461,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"339'13","reported_seq":531,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:36.267443+0000","last_change":"2024-09-10T19:00:42.528804+0000","last_active":"2024-09-10T19:01:36.267443+0000","last_peered":"2024-09-10T19:01:36.267443+0000","last_clean":"2024-09-10T19:01:36.267443+0000","last_became_active":"2024-09-10T19:00:42.520360+0000","last_became_peered":"2024-09-10T19:00:42.520360+0000","last_unstale":"2024-09-10T19:01:36.267443+0000","last_undegraded":"2024-09-10T19:01:36.267443+0000","last_fullsized":"2024-09-10T19:01:36.267443+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'12","last_scrub_stamp":"2024-09-10T18:59:55.943869+0000","last_deep_scrub":"258'12","last_deep_scrub_stamp":"2024-09-10T18:56:21.098770+0000","last_clean_scrub_stamp":"2024-09-10T18:59:55.943869+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.020416790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":340,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.556464+0000","last_change":"2024-09-10T19:00:42.515515+0000","last_active":"2024-09-10T19:01:29.556464+0000","last_peered":"2024-09-10T19:01:29.556464+0000","last_clean":"2024-09-10T19:01:29.556464+0000","last_became_active":"2024-09-10T19:00:42.515144+0000","last_became_peered":"2024-09-10T19:00:42.515144+0000","last_unstale":"2024-09-10T19:01:29.556464+0000","last_undegraded":"2024-09-10T19:01:29.556464+0000","last_fullsized":"2024-09-10T19:01:29.556464+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:57:03.353014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:57:03.353014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014199639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.a","version":"359'10581","reported_seq":10643,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:41.500599+0000","last_change":"2024-09-10T18:59:48.739701+0000","last_active":"2024-09-10T19:01:41.500599+0000","last_peered":"2024-09-10T19:01:41.500599+0000","last_clean":"2024-09-10T19:01:41.500599+0000","last_became_active":"2024-09-10T18:59:48.739415+0000","last_became_peered":"2024-09-10T18:59:48.739415+0000","last_unstale":"2024-09-10T19:01:41.500599+0000","last_undegraded":"2024-09-10T19:01:41.500599+0000","last_fullsized":"2024-09-10T19:01:41.500599+0000","mapping_epoch":320,"log_start":"347'7000","ondisk_log_start":"347'7000","created":315,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3581,"log_dups_size":0,"ondisk_log_size":3581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":465108992,"num_objects":7097,"num_object_clones":0,"num_object_copies":21291,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7097,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10581,"num_write_kb":565696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"70'81","reported_seq":308,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.559058+0000","last_change":"2024-09-10T19:00:40.472680+0000","last_active":"2024-09-10T19:01:29.559058+0000","last_peered":"2024-09-10T19:01:29.559058+0000","last_clean":"2024-09-10T19:01:29.559058+0000","last_became_active":"2024-09-10T18:55:40.392432+0000","last_became_peered":"2024-09-10T18:55:40.392432+0000","last_unstale":"2024-09-10T19:01:29.559058+0000","last_undegraded":"2024-09-10T19:01:29.559058+0000","last_fullsized":"2024-09-10T19:01:29.559058+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:00:40.472595+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:00:40.472595+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:44.413314+0000","scrub_duration":0.0072897099999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"232'41","reported_seq":656,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:36.267836+0000","last_change":"2024-09-10T19:00:57.500215+0000","last_active":"2024-09-10T19:01:36.267836+0000","last_peered":"2024-09-10T19:01:36.267836+0000","last_clean":"2024-09-10T19:01:36.267836+0000","last_became_active":"2024-09-10T18:47:36.420185+0000","last_became_peered":"2024-09-10T18:47:36.420185+0000","last_unstale":"2024-09-10T19:01:36.267836+0000","last_undegraded":"2024-09-10T19:01:36.267836+0000","last_fullsized":"2024-09-10T19:01:36.267836+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'41","last_scrub_stamp":"2024-09-10T18:56:18.672139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:18.672139+0000","objects_scrubbed":10,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.020233260999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":373,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.560262+0000","last_change":"2024-09-10T18:59:04.434037+0000","last_active":"2024-09-10T19:01:29.560262+0000","last_peered":"2024-09-10T19:01:29.560262+0000","last_clean":"2024-09-10T19:01:29.560262+0000","last_became_active":"2024-09-10T18:45:17.964494+0000","last_became_peered":"2024-09-10T18:45:17.964494+0000","last_unstale":"2024-09-10T19:01:29.560262+0000","last_undegraded":"2024-09-10T19:01:29.560262+0000","last_fullsized":"2024-09-10T19:01:29.560262+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:04.433964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:12.982599+0000","last_clean_scrub_stamp":"2024-09-10T18:59:04.433964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024364220000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":315,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.564321+0000","last_change":"2024-09-10T18:56:20.224668+0000","last_active":"2024-09-10T19:01:29.564321+0000","last_peered":"2024-09-10T19:01:29.564321+0000","last_clean":"2024-09-10T19:01:29.564321+0000","last_became_active":"2024-09-10T18:47:57.107478+0000","last_became_peered":"2024-09-10T18:47:57.107478+0000","last_unstale":"2024-09-10T19:01:29.564321+0000","last_undegraded":"2024-09-10T19:01:29.564321+0000","last_fullsized":"2024-09-10T19:01:29.564321+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:20.224562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:20.224562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.008542315,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":353,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.559248+0000","last_change":"2024-09-10T18:57:02.462009+0000","last_active":"2024-09-10T19:01:29.559248+0000","last_peered":"2024-09-10T19:01:29.559248+0000","last_clean":"2024-09-10T19:01:29.559248+0000","last_became_active":"2024-09-10T18:57:02.460594+0000","last_became_peered":"2024-09-10T18:57:02.460594+0000","last_unstale":"2024-09-10T19:01:29.559248+0000","last_undegraded":"2024-09-10T19:01:29.559248+0000","last_fullsized":"2024-09-10T19:01:29.559248+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:11.878181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:22.891084+0000","last_clean_scrub_stamp":"2024-09-10T18:56:11.878181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037255349999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"359'10390","reported_seq":24665,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:41.580185+0000","last_change":"2024-09-10T19:01:25.571972+0000","last_active":"2024-09-10T19:01:41.580185+0000","last_peered":"2024-09-10T19:01:41.580185+0000","last_clean":"2024-09-10T19:01:41.580185+0000","last_became_active":"2024-09-10T19:01:25.526168+0000","last_became_peered":"2024-09-10T19:01:25.526168+0000","last_unstale":"2024-09-10T19:01:41.580185+0000","last_undegraded":"2024-09-10T19:01:41.580185+0000","last_fullsized":"2024-09-10T19:01:41.580185+0000","mapping_epoch":355,"log_start":"346'6738","ondisk_log_start":"346'6738","created":315,"last_epoch_clean":356,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3652,"log_dups_size":0,"ondisk_log_size":3652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":452067328,"num_objects":6898,"num_object_clones":0,"num_object_copies":20694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10390,"num_write_kb":553216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7098,"num_bytes_recovered":465108992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"70'81","reported_seq":176,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.557316+0000","last_change":"2024-09-10T18:59:11.875713+0000","last_active":"2024-09-10T19:01:29.557316+0000","last_peered":"2024-09-10T19:01:29.557316+0000","last_clean":"2024-09-10T19:01:29.557316+0000","last_became_active":"2024-09-10T18:57:02.437944+0000","last_became_peered":"2024-09-10T18:57:02.437944+0000","last_unstale":"2024-09-10T19:01:29.557316+0000","last_undegraded":"2024-09-10T19:01:29.557316+0000","last_fullsized":"2024-09-10T19:01:29.557316+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:11.875625+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:59:11.875625+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040920360000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"232'22","reported_seq":420,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.556351+0000","last_change":"2024-09-10T19:00:08.468265+0000","last_active":"2024-09-10T19:01:29.556351+0000","last_peered":"2024-09-10T19:01:29.556351+0000","last_clean":"2024-09-10T19:01:29.556351+0000","last_became_active":"2024-09-10T18:55:40.397422+0000","last_became_peered":"2024-09-10T18:55:40.397422+0000","last_unstale":"2024-09-10T19:01:29.556351+0000","last_undegraded":"2024-09-10T19:01:29.556351+0000","last_fullsized":"2024-09-10T19:01:29.556351+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'22","last_scrub_stamp":"2024-09-10T19:00:08.468154+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-10T18:47:10.514951+0000","last_clean_scrub_stamp":"2024-09-10T19:00:08.468154+0000","objects_scrubbed":5,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024987170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"84'6","reported_seq":1371,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:40.729154+0000","last_change":"2024-09-10T19:00:40.504252+0000","last_active":"2024-09-10T19:01:40.729154+0000","last_peered":"2024-09-10T19:01:40.729154+0000","last_clean":"2024-09-10T19:01:40.729154+0000","last_became_active":"2024-09-10T18:55:40.403098+0000","last_became_peered":"2024-09-10T18:55:40.403098+0000","last_unstale":"2024-09-10T19:01:40.729154+0000","last_undegraded":"2024-09-10T19:01:40.729154+0000","last_fullsized":"2024-09-10T19:01:40.729154+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'6","last_scrub_stamp":"2024-09-10T19:00:40.504174+0000","last_deep_scrub":"84'6","last_deep_scrub_stamp":"2024-09-10T18:47:22.545450+0000","last_clean_scrub_stamp":"2024-09-10T19:00:40.504174+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:02:03.384361+0000","scrub_duration":0.0052333889999999997,"objects_trimmed":0,"snaptrim_duration":0.0021933159999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":967,"num_read_kb":962,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":337,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.558271+0000","last_change":"2024-09-10T19:00:08.141664+0000","last_active":"2024-09-10T19:01:29.558271+0000","last_peered":"2024-09-10T19:01:29.558271+0000","last_clean":"2024-09-10T19:01:29.558271+0000","last_became_active":"2024-09-10T18:55:40.393389+0000","last_became_peered":"2024-09-10T18:55:40.393389+0000","last_unstale":"2024-09-10T19:01:29.558271+0000","last_undegraded":"2024-09-10T19:01:29.558271+0000","last_fullsized":"2024-09-10T19:01:29.558271+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:08.141603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:00:08.141603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001270729,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":344,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.560534+0000","last_change":"2024-09-10T18:58:58.522346+0000","last_active":"2024-09-10T19:01:29.560534+0000","last_peered":"2024-09-10T19:01:29.560534+0000","last_clean":"2024-09-10T19:01:29.560534+0000","last_became_active":"2024-09-10T18:57:02.438945+0000","last_became_peered":"2024-09-10T18:57:02.438945+0000","last_unstale":"2024-09-10T19:01:29.560534+0000","last_undegraded":"2024-09-10T19:01:29.560534+0000","last_fullsized":"2024-09-10T19:01:29.560534+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:58.522295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:58:58.522295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0059618290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":111,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.564641+0000","last_change":"2024-09-10T19:00:41.794168+0000","last_active":"2024-09-10T19:01:29.564641+0000","last_peered":"2024-09-10T19:01:29.564641+0000","last_clean":"2024-09-10T19:01:29.564641+0000","last_became_active":"2024-09-10T18:57:02.453865+0000","last_became_peered":"2024-09-10T18:57:02.453865+0000","last_unstale":"2024-09-10T19:01:29.564641+0000","last_undegraded":"2024-09-10T19:01:29.564641+0000","last_fullsized":"2024-09-10T19:01:29.564641+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:41.794116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:07.150480+0000","last_clean_scrub_stamp":"2024-09-10T19:00:41.794116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:45.662042+0000","scrub_duration":0.001296148,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.8","version":"359'10606","reported_seq":10658,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:43.710059+0000","last_change":"2024-09-10T18:59:42.712702+0000","last_active":"2024-09-10T19:01:43.710059+0000","last_peered":"2024-09-10T19:01:43.710059+0000","last_clean":"2024-09-10T19:01:43.710059+0000","last_became_active":"2024-09-10T18:59:42.712397+0000","last_became_peered":"2024-09-10T18:59:42.712397+0000","last_unstale":"2024-09-10T19:01:43.710059+0000","last_undegraded":"2024-09-10T19:01:43.710059+0000","last_fullsized":"2024-09-10T19:01:43.710059+0000","mapping_epoch":315,"log_start":"347'6900","ondisk_log_start":"347'6900","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3706,"log_dups_size":0,"ondisk_log_size":3706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":435552256,"num_objects":6646,"num_object_clones":0,"num_object_copies":19938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10606,"num_write_kb":552064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,4],"acting":[0,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"70'81","reported_seq":277,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.555127+0000","last_change":"2024-09-10T18:58:49.141636+0000","last_active":"2024-09-10T19:01:29.555127+0000","last_peered":"2024-09-10T19:01:29.555127+0000","last_clean":"2024-09-10T19:01:29.555127+0000","last_became_active":"2024-09-10T18:52:14.589009+0000","last_became_peered":"2024-09-10T18:52:14.589009+0000","last_unstale":"2024-09-10T19:01:29.555127+0000","last_undegraded":"2024-09-10T19:01:29.555127+0000","last_fullsized":"2024-09-10T19:01:29.555127+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:58:49.141013+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:58:49.141013+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0036869540000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"232'24","reported_seq":96,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.557547+0000","last_change":"2024-09-10T19:00:42.532827+0000","last_active":"2024-09-10T19:01:29.557547+0000","last_peered":"2024-09-10T19:01:29.557547+0000","last_clean":"2024-09-10T19:01:29.557547+0000","last_became_active":"2024-09-10T19:00:42.516521+0000","last_became_peered":"2024-09-10T19:00:42.516521+0000","last_unstale":"2024-09-10T19:01:29.557547+0000","last_undegraded":"2024-09-10T19:01:29.557547+0000","last_fullsized":"2024-09-10T19:01:29.557547+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T19:00:39.748509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:00:39.748509+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:02:03.931650+0000","scrub_duration":0.0027783730000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"305'2","reported_seq":343,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.557449+0000","last_change":"2024-09-10T18:59:18.897245+0000","last_active":"2024-09-10T19:01:29.557449+0000","last_peered":"2024-09-10T19:01:29.557449+0000","last_clean":"2024-09-10T19:01:29.557449+0000","last_became_active":"2024-09-10T18:55:40.390022+0000","last_became_peered":"2024-09-10T18:55:40.390022+0000","last_unstale":"2024-09-10T19:01:29.557449+0000","last_undegraded":"2024-09-10T19:01:29.557449+0000","last_fullsized":"2024-09-10T19:01:29.557449+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"305'2","last_scrub_stamp":"2024-09-10T18:59:18.897059+0000","last_deep_scrub":"305'2","last_deep_scrub_stamp":"2024-09-10T18:59:18.897059+0000","last_clean_scrub_stamp":"2024-09-10T18:59:18.897059+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016452700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":304,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.554655+0000","last_change":"2024-09-10T19:00:56.482753+0000","last_active":"2024-09-10T19:01:29.554655+0000","last_peered":"2024-09-10T19:01:29.554655+0000","last_clean":"2024-09-10T19:01:29.554655+0000","last_became_active":"2024-09-10T18:57:02.447195+0000","last_became_peered":"2024-09-10T18:57:02.447195+0000","last_unstale":"2024-09-10T19:01:29.554655+0000","last_undegraded":"2024-09-10T19:01:29.554655+0000","last_fullsized":"2024-09-10T19:01:29.554655+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:24.153973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:24.153973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001771613,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":30,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.560074+0000","last_change":"2024-09-10T19:00:42.518887+0000","last_active":"2024-09-10T19:01:29.560074+0000","last_peered":"2024-09-10T19:01:29.560074+0000","last_clean":"2024-09-10T19:01:29.560074+0000","last_became_active":"2024-09-10T19:00:42.518566+0000","last_became_peered":"2024-09-10T19:00:42.518566+0000","last_unstale":"2024-09-10T19:01:29.560074+0000","last_undegraded":"2024-09-10T19:01:29.560074+0000","last_fullsized":"2024-09-10T19:01:29.560074+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:39.752722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:00:39.752722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:01:44.640996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":360,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.555929+0000","last_change":"2024-09-10T19:01:09.387164+0000","last_active":"2024-09-10T19:01:29.555929+0000","last_peered":"2024-09-10T19:01:29.555929+0000","last_clean":"2024-09-10T19:01:29.555929+0000","last_became_active":"2024-09-10T18:45:24.083689+0000","last_became_peered":"2024-09-10T18:45:24.083689+0000","last_unstale":"2024-09-10T19:01:29.555929+0000","last_undegraded":"2024-09-10T19:01:29.555929+0000","last_fullsized":"2024-09-10T19:01:29.555929+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:52.136095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:58:52.136095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.021731822000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"359'10534","reported_seq":10586,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:41.467266+0000","last_change":"2024-09-10T18:59:42.717031+0000","last_active":"2024-09-10T19:01:41.467266+0000","last_peered":"2024-09-10T19:01:41.467266+0000","last_clean":"2024-09-10T19:01:41.467266+0000","last_became_active":"2024-09-10T18:59:42.716730+0000","last_became_peered":"2024-09-10T18:59:42.716730+0000","last_unstale":"2024-09-10T19:01:41.467266+0000","last_undegraded":"2024-09-10T19:01:41.467266+0000","last_fullsized":"2024-09-10T19:01:41.467266+0000","mapping_epoch":315,"log_start":"347'7000","ondisk_log_start":"347'7000","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3534,"log_dups_size":0,"ondisk_log_size":3534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":464388096,"num_objects":7086,"num_object_clones":0,"num_object_copies":21258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10534,"num_write_kb":563840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"232'33","reported_seq":452,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.564710+0000","last_change":"2024-09-10T18:59:17.515699+0000","last_active":"2024-09-10T19:01:29.564710+0000","last_peered":"2024-09-10T19:01:29.564710+0000","last_clean":"2024-09-10T19:01:29.564710+0000","last_became_active":"2024-09-10T18:57:02.435499+0000","last_became_peered":"2024-09-10T18:57:02.435499+0000","last_unstale":"2024-09-10T19:01:29.564710+0000","last_undegraded":"2024-09-10T19:01:29.564710+0000","last_fullsized":"2024-09-10T19:01:29.564710+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'33","last_scrub_stamp":"2024-09-10T18:59:17.515615+0000","last_deep_scrub":"232'33","last_deep_scrub_stamp":"2024-09-10T18:56:21.206960+0000","last_clean_scrub_stamp":"2024-09-10T18:59:17.515615+0000","objects_scrubbed":7,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.009416209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"60'2","reported_seq":372,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.558718+0000","last_change":"2024-09-10T18:58:55.931801+0000","last_active":"2024-09-10T19:01:29.558718+0000","last_peered":"2024-09-10T19:01:29.558718+0000","last_clean":"2024-09-10T19:01:29.558718+0000","last_became_active":"2024-09-10T18:45:17.961660+0000","last_became_peered":"2024-09-10T18:45:17.961660+0000","last_unstale":"2024-09-10T19:01:29.558718+0000","last_undegraded":"2024-09-10T19:01:29.558718+0000","last_fullsized":"2024-09-10T19:01:29.558718+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-10T18:58:55.931717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:58:55.931717+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040038909999999999,"objects_trimmed":0,"snaptrim_duration":0.0013942640000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":390,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.559750+0000","last_change":"2024-09-10T18:57:02.446028+0000","last_active":"2024-09-10T19:01:29.559750+0000","last_peered":"2024-09-10T19:01:29.559750+0000","last_clean":"2024-09-10T19:01:29.559750+0000","last_became_active":"2024-09-10T18:57:02.445712+0000","last_became_peered":"2024-09-10T18:57:02.445712+0000","last_unstale":"2024-09-10T19:01:29.559750+0000","last_undegraded":"2024-09-10T19:01:29.559750+0000","last_fullsized":"2024-09-10T19:01:29.559750+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:33.559651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:33.559651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0036787759999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'81","reported_seq":176,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.555839+0000","last_change":"2024-09-10T18:59:14.873293+0000","last_active":"2024-09-10T19:01:29.555839+0000","last_peered":"2024-09-10T19:01:29.555839+0000","last_clean":"2024-09-10T19:01:29.555839+0000","last_became_active":"2024-09-10T18:57:02.440063+0000","last_became_peered":"2024-09-10T18:57:02.440063+0000","last_unstale":"2024-09-10T19:01:29.555839+0000","last_undegraded":"2024-09-10T19:01:29.555839+0000","last_fullsized":"2024-09-10T19:01:29.555839+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:14.873083+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:56.085580+0000","last_clean_scrub_stamp":"2024-09-10T18:59:14.873083+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0063632760000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":359,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.559980+0000","last_change":"2024-09-10T18:57:02.432025+0000","last_active":"2024-09-10T19:01:29.559980+0000","last_peered":"2024-09-10T19:01:29.559980+0000","last_clean":"2024-09-10T19:01:29.559980+0000","last_became_active":"2024-09-10T18:57:02.431556+0000","last_became_peered":"2024-09-10T18:57:02.431556+0000","last_unstale":"2024-09-10T19:01:29.559980+0000","last_undegraded":"2024-09-10T19:01:29.559980+0000","last_fullsized":"2024-09-10T19:01:29.559980+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:27.012123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:27.012123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007606036,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":374,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.560757+0000","last_change":"2024-09-10T19:00:07.624756+0000","last_active":"2024-09-10T19:01:29.560757+0000","last_peered":"2024-09-10T19:01:29.560757+0000","last_clean":"2024-09-10T19:01:29.560757+0000","last_became_active":"2024-09-10T18:45:24.073327+0000","last_became_peered":"2024-09-10T18:45:24.073327+0000","last_unstale":"2024-09-10T19:01:29.560757+0000","last_undegraded":"2024-09-10T19:01:29.560757+0000","last_fullsized":"2024-09-10T19:01:29.560757+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:07.624701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:00:07.624701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001137962,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"359'10448","reported_seq":10500,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:43.751472+0000","last_change":"2024-09-10T18:59:43.009491+0000","last_active":"2024-09-10T19:01:43.751472+0000","last_peered":"2024-09-10T19:01:43.751472+0000","last_clean":"2024-09-10T19:01:43.751472+0000","last_became_active":"2024-09-10T18:59:43.009210+0000","last_became_peered":"2024-09-10T18:59:43.009210+0000","last_unstale":"2024-09-10T19:01:43.751472+0000","last_undegraded":"2024-09-10T19:01:43.751472+0000","last_fullsized":"2024-09-10T19:01:43.751472+0000","mapping_epoch":315,"log_start":"346'6700","ondisk_log_start":"346'6700","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3748,"log_dups_size":0,"ondisk_log_size":3748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436994048,"num_objects":6668,"num_object_clones":0,"num_object_copies":20004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6668,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10448,"num_write_kb":547712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"70'81","reported_seq":369,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.564578+0000","last_change":"2024-09-10T18:58:59.582622+0000","last_active":"2024-09-10T19:01:29.564578+0000","last_peered":"2024-09-10T19:01:29.564578+0000","last_clean":"2024-09-10T19:01:29.564578+0000","last_became_active":"2024-09-10T18:52:16.598295+0000","last_became_peered":"2024-09-10T18:52:16.598295+0000","last_unstale":"2024-09-10T19:01:29.564578+0000","last_undegraded":"2024-09-10T19:01:29.564578+0000","last_fullsized":"2024-09-10T19:01:29.564578+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:58:59.582565+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:53.609885+0000","last_clean_scrub_stamp":"2024-09-10T18:58:59.582565+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046448920000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"232'35","reported_seq":467,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.564236+0000","last_change":"2024-09-10T19:00:43.819111+0000","last_active":"2024-09-10T19:01:29.564236+0000","last_peered":"2024-09-10T19:01:29.564236+0000","last_clean":"2024-09-10T19:01:29.564236+0000","last_became_active":"2024-09-10T18:57:02.436059+0000","last_became_peered":"2024-09-10T18:57:02.436059+0000","last_unstale":"2024-09-10T19:01:29.564236+0000","last_undegraded":"2024-09-10T19:01:29.564236+0000","last_fullsized":"2024-09-10T19:01:29.564236+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'35","last_scrub_stamp":"2024-09-10T19:00:43.819004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:00:43.819004+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:02:08.320454+0000","scrub_duration":0.0050234040000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":338,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.560084+0000","last_change":"2024-09-10T18:56:13.760318+0000","last_active":"2024-09-10T19:01:29.560084+0000","last_peered":"2024-09-10T19:01:29.560084+0000","last_clean":"2024-09-10T19:01:29.560084+0000","last_became_active":"2024-09-10T18:55:40.391849+0000","last_became_peered":"2024-09-10T18:55:40.391849+0000","last_unstale":"2024-09-10T19:01:29.560084+0000","last_undegraded":"2024-09-10T19:01:29.560084+0000","last_fullsized":"2024-09-10T19:01:29.560084+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:13.760238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:13.760238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.048619793000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":319,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.555859+0000","last_change":"2024-09-10T19:00:42.738386+0000","last_active":"2024-09-10T19:01:29.555859+0000","last_peered":"2024-09-10T19:01:29.555859+0000","last_clean":"2024-09-10T19:01:29.555859+0000","last_became_active":"2024-09-10T19:00:42.514709+0000","last_became_peered":"2024-09-10T19:00:42.514709+0000","last_unstale":"2024-09-10T19:01:29.555859+0000","last_undegraded":"2024-09-10T19:01:29.555859+0000","last_fullsized":"2024-09-10T19:01:29.555859+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:42.738331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:00:42.738331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:02:08.495978+0000","scrub_duration":0.001211314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":156,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.558140+0000","last_change":"2024-09-10T18:57:03.015513+0000","last_active":"2024-09-10T19:01:29.558140+0000","last_peered":"2024-09-10T19:01:29.558140+0000","last_clean":"2024-09-10T19:01:29.558140+0000","last_became_active":"2024-09-10T18:57:02.468866+0000","last_became_peered":"2024-09-10T18:57:02.468866+0000","last_unstale":"2024-09-10T19:01:29.558140+0000","last_undegraded":"2024-09-10T19:01:29.558140+0000","last_fullsized":"2024-09-10T19:01:29.558140+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:57:03.015457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:57:03.015457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001545795,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.5","version":"359'10476","reported_seq":24889,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:43.759400+0000","last_change":"2024-09-10T19:01:29.582446+0000","last_active":"2024-09-10T19:01:43.759400+0000","last_peered":"2024-09-10T19:01:43.759400+0000","last_clean":"2024-09-10T19:01:43.759400+0000","last_became_active":"2024-09-10T19:01:29.559761+0000","last_became_peered":"2024-09-10T19:01:29.559761+0000","last_unstale":"2024-09-10T19:01:43.759400+0000","last_undegraded":"2024-09-10T19:01:43.759400+0000","last_fullsized":"2024-09-10T19:01:43.759400+0000","mapping_epoch":357,"log_start":"347'6800","ondisk_log_start":"347'6800","created":315,"last_epoch_clean":358,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3676,"log_dups_size":0,"ondisk_log_size":3676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436994048,"num_objects":6668,"num_object_clones":0,"num_object_copies":20004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6668,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10476,"num_write_kb":548608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7168,"num_bytes_recovered":469762048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.e","version":"70'81","reported_seq":237,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.563078+0000","last_change":"2024-09-10T18:59:23.592755+0000","last_active":"2024-09-10T19:01:29.563078+0000","last_peered":"2024-09-10T19:01:29.563078+0000","last_clean":"2024-09-10T19:01:29.563078+0000","last_became_active":"2024-09-10T18:57:02.449115+0000","last_became_peered":"2024-09-10T18:57:02.449115+0000","last_unstale":"2024-09-10T19:01:29.563078+0000","last_undegraded":"2024-09-10T19:01:29.563078+0000","last_fullsized":"2024-09-10T19:01:29.563078+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:23.592672+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:59:23.592672+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038976470000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"232'19","reported_seq":375,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.556422+0000","last_change":"2024-09-10T19:00:40.748808+0000","last_active":"2024-09-10T19:01:29.556422+0000","last_peered":"2024-09-10T19:01:29.556422+0000","last_clean":"2024-09-10T19:01:29.556422+0000","last_became_active":"2024-09-10T18:55:40.395257+0000","last_became_peered":"2024-09-10T18:55:40.395257+0000","last_unstale":"2024-09-10T19:01:29.556422+0000","last_undegraded":"2024-09-10T19:01:29.556422+0000","last_fullsized":"2024-09-10T19:01:29.556422+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T19:00:40.748713+0000","last_deep_scrub":"130'13","last_deep_scrub_stamp":"2024-09-10T18:51:58.427534+0000","last_clean_scrub_stamp":"2024-09-10T19:00:40.748713+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:02:08.421967+0000","scrub_duration":0.0043426029999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":360,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.555829+0000","last_change":"2024-09-10T18:59:09.206400+0000","last_active":"2024-09-10T19:01:29.555829+0000","last_peered":"2024-09-10T19:01:29.555829+0000","last_clean":"2024-09-10T19:01:29.555829+0000","last_became_active":"2024-09-10T18:45:17.963057+0000","last_became_peered":"2024-09-10T18:45:17.963057+0000","last_unstale":"2024-09-10T19:01:29.555829+0000","last_undegraded":"2024-09-10T19:01:29.555829+0000","last_fullsized":"2024-09-10T19:01:29.555829+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:09.206317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:59:09.206317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.018986050000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":329,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.561849+0000","last_change":"2024-09-10T19:01:12.880399+0000","last_active":"2024-09-10T19:01:29.561849+0000","last_peered":"2024-09-10T19:01:29.561849+0000","last_clean":"2024-09-10T19:01:29.561849+0000","last_became_active":"2024-09-10T18:55:40.390771+0000","last_became_peered":"2024-09-10T18:55:40.390771+0000","last_unstale":"2024-09-10T19:01:29.561849+0000","last_undegraded":"2024-09-10T19:01:29.561849+0000","last_fullsized":"2024-09-10T19:01:29.561849+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:17.661832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_clean_scrub_stamp":"2024-09-10T18:56:17.661832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041460890000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":353,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.556950+0000","last_change":"2024-09-10T18:57:02.453927+0000","last_active":"2024-09-10T19:01:29.556950+0000","last_peered":"2024-09-10T19:01:29.556950+0000","last_clean":"2024-09-10T19:01:29.556950+0000","last_became_active":"2024-09-10T18:57:02.452387+0000","last_became_peered":"2024-09-10T18:57:02.452387+0000","last_unstale":"2024-09-10T19:01:29.556950+0000","last_undegraded":"2024-09-10T19:01:29.556950+0000","last_fullsized":"2024-09-10T19:01:29.556950+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:32.217641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:53.345184+0000","last_clean_scrub_stamp":"2024-09-10T18:56:32.217641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0059447470000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"359'10340","reported_seq":10392,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:41.578683+0000","last_change":"2024-09-10T18:59:42.715562+0000","last_active":"2024-09-10T19:01:41.578683+0000","last_peered":"2024-09-10T19:01:41.578683+0000","last_clean":"2024-09-10T19:01:41.578683+0000","last_became_active":"2024-09-10T18:59:42.715275+0000","last_became_peered":"2024-09-10T18:59:42.715275+0000","last_unstale":"2024-09-10T19:01:41.578683+0000","last_undegraded":"2024-09-10T19:01:41.578683+0000","last_fullsized":"2024-09-10T19:01:41.578683+0000","mapping_epoch":315,"log_start":"346'6744","ondisk_log_start":"346'6744","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3596,"log_dups_size":0,"ondisk_log_size":3596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":454819840,"num_objects":6940,"num_object_clones":0,"num_object_copies":20820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10340,"num_write_kb":552960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.d","version":"70'81","reported_seq":168,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.562716+0000","last_change":"2024-09-10T19:00:54.102663+0000","last_active":"2024-09-10T19:01:29.562716+0000","last_peered":"2024-09-10T19:01:29.562716+0000","last_clean":"2024-09-10T19:01:29.562716+0000","last_became_active":"2024-09-10T18:57:02.439652+0000","last_became_peered":"2024-09-10T18:57:02.439652+0000","last_unstale":"2024-09-10T19:01:29.562716+0000","last_undegraded":"2024-09-10T19:01:29.562716+0000","last_fullsized":"2024-09-10T19:01:29.562716+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:18.041116+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:18.041116+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014613810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"232'27","reported_seq":46,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.558899+0000","last_change":"2024-09-10T19:00:42.533625+0000","last_active":"2024-09-10T19:01:29.558899+0000","last_peered":"2024-09-10T19:01:29.558899+0000","last_clean":"2024-09-10T19:01:29.558899+0000","last_became_active":"2024-09-10T19:00:42.515903+0000","last_became_peered":"2024-09-10T19:00:42.515903+0000","last_unstale":"2024-09-10T19:01:29.558899+0000","last_undegraded":"2024-09-10T19:01:29.558899+0000","last_fullsized":"2024-09-10T19:01:29.558899+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'27","last_scrub_stamp":"2024-09-10T18:58:47.595256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:58:47.595256+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014176382,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":33,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":320,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.559359+0000","last_change":"2024-09-10T18:57:02.463092+0000","last_active":"2024-09-10T19:01:29.559359+0000","last_peered":"2024-09-10T19:01:29.559359+0000","last_clean":"2024-09-10T19:01:29.559359+0000","last_became_active":"2024-09-10T18:57:02.461708+0000","last_became_peered":"2024-09-10T18:57:02.461708+0000","last_unstale":"2024-09-10T19:01:29.559359+0000","last_undegraded":"2024-09-10T19:01:29.559359+0000","last_fullsized":"2024-09-10T19:01:29.559359+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:24.902205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:05.603125+0000","last_clean_scrub_stamp":"2024-09-10T18:56:24.902205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0076874830000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"273'10","reported_seq":239,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:36.266381+0000","last_change":"2024-09-10T18:59:11.568683+0000","last_active":"2024-09-10T19:01:36.266381+0000","last_peered":"2024-09-10T19:01:36.266381+0000","last_clean":"2024-09-10T19:01:36.266381+0000","last_became_active":"2024-09-10T18:57:02.441890+0000","last_became_peered":"2024-09-10T18:57:02.441890+0000","last_unstale":"2024-09-10T19:01:36.266381+0000","last_undegraded":"2024-09-10T19:01:36.266381+0000","last_fullsized":"2024-09-10T19:01:36.266381+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'10","last_scrub_stamp":"2024-09-10T18:59:11.568599+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-09-10T18:47:02.582229+0000","last_clean_scrub_stamp":"2024-09-10T18:59:11.568599+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0054658950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":381,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.560179+0000","last_change":"2024-09-10T18:59:51.487816+0000","last_active":"2024-09-10T19:01:29.560179+0000","last_peered":"2024-09-10T19:01:29.560179+0000","last_clean":"2024-09-10T19:01:29.560179+0000","last_became_active":"2024-09-10T18:55:40.389202+0000","last_became_peered":"2024-09-10T18:55:40.389202+0000","last_unstale":"2024-09-10T19:01:29.560179+0000","last_undegraded":"2024-09-10T19:01:29.560179+0000","last_fullsized":"2024-09-10T19:01:29.560179+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:51.487763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_clean_scrub_stamp":"2024-09-10T18:59:51.487763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001471904,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"359'10376","reported_seq":24306,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:41.462845+0000","last_change":"2024-09-10T19:01:23.521153+0000","last_active":"2024-09-10T19:01:41.462845+0000","last_peered":"2024-09-10T19:01:41.462845+0000","last_clean":"2024-09-10T19:01:41.462845+0000","last_became_active":"2024-09-10T19:01:23.517785+0000","last_became_peered":"2024-09-10T19:01:23.517785+0000","last_unstale":"2024-09-10T19:01:41.462845+0000","last_undegraded":"2024-09-10T19:01:41.462845+0000","last_fullsized":"2024-09-10T19:01:41.462845+0000","mapping_epoch":353,"log_start":"346'6800","ondisk_log_start":"346'6800","created":315,"last_epoch_clean":354,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3576,"log_dups_size":0,"ondisk_log_size":3576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458358784,"num_objects":6994,"num_object_clones":0,"num_object_copies":20982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10376,"num_write_kb":555840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6925,"num_bytes_recovered":453836800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.c","version":"70'81","reported_seq":313,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.559278+0000","last_change":"2024-09-10T18:56:33.779212+0000","last_active":"2024-09-10T19:01:29.559278+0000","last_peered":"2024-09-10T19:01:29.559278+0000","last_clean":"2024-09-10T19:01:29.559278+0000","last_became_active":"2024-09-10T18:55:40.390655+0000","last_became_peered":"2024-09-10T18:55:40.390655+0000","last_unstale":"2024-09-10T19:01:29.559278+0000","last_undegraded":"2024-09-10T19:01:29.559278+0000","last_fullsized":"2024-09-10T19:01:29.559278+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:33.779154+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:56:33.779154+0000","last_clean_scrub_stamp":"2024-09-10T18:56:33.779154+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0075121179999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"232'18","reported_seq":391,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.561813+0000","last_change":"2024-09-10T19:00:51.009315+0000","last_active":"2024-09-10T19:01:29.561813+0000","last_peered":"2024-09-10T19:01:29.561813+0000","last_clean":"2024-09-10T19:01:29.561813+0000","last_became_active":"2024-09-10T18:45:20.056869+0000","last_became_peered":"2024-09-10T18:45:20.056869+0000","last_unstale":"2024-09-10T19:01:29.561813+0000","last_undegraded":"2024-09-10T19:01:29.561813+0000","last_fullsized":"2024-09-10T19:01:29.561813+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'18","last_scrub_stamp":"2024-09-10T18:55:27.507364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:55:27.507364+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030221390000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"56'2","reported_seq":401,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.563190+0000","last_change":"2024-09-10T19:01:01.185585+0000","last_active":"2024-09-10T19:01:29.563190+0000","last_peered":"2024-09-10T19:01:29.563190+0000","last_clean":"2024-09-10T19:01:29.563190+0000","last_became_active":"2024-09-10T18:57:02.434475+0000","last_became_peered":"2024-09-10T18:57:02.434475+0000","last_unstale":"2024-09-10T19:01:29.563190+0000","last_undegraded":"2024-09-10T19:01:29.563190+0000","last_fullsized":"2024-09-10T19:01:29.563190+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-10T18:56:01.630091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:01.630091+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001578238,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":319,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.556387+0000","last_change":"2024-09-10T19:00:42.515054+0000","last_active":"2024-09-10T19:01:29.556387+0000","last_peered":"2024-09-10T19:01:29.556387+0000","last_clean":"2024-09-10T19:01:29.556387+0000","last_became_active":"2024-09-10T19:00:42.513796+0000","last_became_peered":"2024-09-10T19:00:42.513796+0000","last_unstale":"2024-09-10T19:01:29.556387+0000","last_undegraded":"2024-09-10T19:01:29.556387+0000","last_fullsized":"2024-09-10T19:01:29.556387+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:54.899134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:02.711594+0000","last_clean_scrub_stamp":"2024-09-10T18:59:54.899134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051739739999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":363,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.558805+0000","last_change":"2024-09-10T18:59:11.827181+0000","last_active":"2024-09-10T19:01:29.558805+0000","last_peered":"2024-09-10T19:01:29.558805+0000","last_clean":"2024-09-10T19:01:29.558805+0000","last_became_active":"2024-09-10T18:57:02.459985+0000","last_became_peered":"2024-09-10T18:57:02.459985+0000","last_unstale":"2024-09-10T19:01:29.558805+0000","last_undegraded":"2024-09-10T19:01:29.558805+0000","last_fullsized":"2024-09-10T19:01:29.558805+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:11.827101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:53.352427+0000","last_clean_scrub_stamp":"2024-09-10T18:59:11.827101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034557899999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.13","version":"70'81","reported_seq":289,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.555269+0000","last_change":"2024-09-10T19:00:49.472063+0000","last_active":"2024-09-10T19:01:29.555269+0000","last_peered":"2024-09-10T19:01:29.555269+0000","last_clean":"2024-09-10T19:01:29.555269+0000","last_became_active":"2024-09-10T18:52:16.602463+0000","last_became_peered":"2024-09-10T18:52:16.602463+0000","last_unstale":"2024-09-10T19:01:29.555269+0000","last_undegraded":"2024-09-10T19:01:29.555269+0000","last_fullsized":"2024-09-10T19:01:29.555269+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:34.760031+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:34.760031+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002735595,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"232'19","reported_seq":388,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.557646+0000","last_change":"2024-09-10T18:57:02.485836+0000","last_active":"2024-09-10T19:01:29.557646+0000","last_peered":"2024-09-10T19:01:29.557646+0000","last_clean":"2024-09-10T19:01:29.557646+0000","last_became_active":"2024-09-10T18:57:02.453001+0000","last_became_peered":"2024-09-10T18:57:02.453001+0000","last_unstale":"2024-09-10T19:01:29.557646+0000","last_undegraded":"2024-09-10T19:01:29.557646+0000","last_fullsized":"2024-09-10T19:01:29.557646+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T18:56:25.122070+0000","last_deep_scrub":"76'13","last_deep_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_clean_scrub_stamp":"2024-09-10T18:56:25.122070+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0061142590000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":348,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.554579+0000","last_change":"2024-09-10T18:59:59.385405+0000","last_active":"2024-09-10T19:01:29.554579+0000","last_peered":"2024-09-10T19:01:29.554579+0000","last_clean":"2024-09-10T19:01:29.554579+0000","last_became_active":"2024-09-10T18:55:40.401037+0000","last_became_peered":"2024-09-10T18:55:40.401037+0000","last_unstale":"2024-09-10T19:01:29.554579+0000","last_undegraded":"2024-09-10T19:01:29.554579+0000","last_fullsized":"2024-09-10T19:01:29.554579+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:59.385349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:59:59.385349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00163496,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":365,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.559448+0000","last_change":"2024-09-10T18:57:02.455638+0000","last_active":"2024-09-10T19:01:29.559448+0000","last_peered":"2024-09-10T19:01:29.559448+0000","last_clean":"2024-09-10T19:01:29.559448+0000","last_became_active":"2024-09-10T18:57:02.455341+0000","last_became_peered":"2024-09-10T18:57:02.455341+0000","last_unstale":"2024-09-10T19:01:29.559448+0000","last_undegraded":"2024-09-10T19:01:29.559448+0000","last_fullsized":"2024-09-10T19:01:29.559448+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:25.862853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:00.754912+0000","last_clean_scrub_stamp":"2024-09-10T18:56:25.862853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0033952399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"63'1","reported_seq":387,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.562395+0000","last_change":"2024-09-10T19:01:08.024679+0000","last_active":"2024-09-10T19:01:29.562395+0000","last_peered":"2024-09-10T19:01:29.562395+0000","last_clean":"2024-09-10T19:01:29.562395+0000","last_became_active":"2024-09-10T18:59:48.741275+0000","last_became_peered":"2024-09-10T18:59:48.741275+0000","last_unstale":"2024-09-10T19:01:29.562395+0000","last_undegraded":"2024-09-10T19:01:29.562395+0000","last_fullsized":"2024-09-10T19:01:29.562395+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-10T18:56:07.774334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:07.774334+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.090386770000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":52,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.12","version":"70'81","reported_seq":228,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.559882+0000","last_change":"2024-09-10T18:58:52.543604+0000","last_active":"2024-09-10T19:01:29.559882+0000","last_peered":"2024-09-10T19:01:29.559882+0000","last_clean":"2024-09-10T19:01:29.559882+0000","last_became_active":"2024-09-10T18:52:16.598921+0000","last_became_peered":"2024-09-10T18:52:16.598921+0000","last_unstale":"2024-09-10T19:01:29.559882+0000","last_undegraded":"2024-09-10T19:01:29.559882+0000","last_fullsized":"2024-09-10T19:01:29.559882+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:58:52.543544+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:54.127476+0000","last_clean_scrub_stamp":"2024-09-10T18:58:52.543544+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003920644,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":394,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.563870+0000","last_change":"2024-09-10T18:59:05.556377+0000","last_active":"2024-09-10T19:01:29.563870+0000","last_peered":"2024-09-10T19:01:29.563870+0000","last_clean":"2024-09-10T19:01:29.563870+0000","last_became_active":"2024-09-10T18:57:02.442251+0000","last_became_peered":"2024-09-10T18:57:02.442251+0000","last_unstale":"2024-09-10T19:01:29.563870+0000","last_undegraded":"2024-09-10T19:01:29.563870+0000","last_fullsized":"2024-09-10T19:01:29.563870+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:05.556292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:59:05.556292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026781280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"232'12","reported_seq":396,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.559533+0000","last_change":"2024-09-10T18:59:52.198399+0000","last_active":"2024-09-10T19:01:29.559533+0000","last_peered":"2024-09-10T19:01:29.559533+0000","last_clean":"2024-09-10T19:01:29.559533+0000","last_became_active":"2024-09-10T18:57:02.460430+0000","last_became_peered":"2024-09-10T18:57:02.460430+0000","last_unstale":"2024-09-10T19:01:29.559533+0000","last_undegraded":"2024-09-10T19:01:29.559533+0000","last_fullsized":"2024-09-10T19:01:29.559533+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'12","last_scrub_stamp":"2024-09-10T18:59:52.198340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:59:52.198340+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001790591,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":356,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.560760+0000","last_change":"2024-09-10T18:56:32.602499+0000","last_active":"2024-09-10T19:01:29.560760+0000","last_peered":"2024-09-10T19:01:29.560760+0000","last_clean":"2024-09-10T19:01:29.560760+0000","last_became_active":"2024-09-10T18:45:22.069653+0000","last_became_peered":"2024-09-10T18:45:22.069653+0000","last_unstale":"2024-09-10T19:01:29.560760+0000","last_undegraded":"2024-09-10T19:01:29.560760+0000","last_fullsized":"2024-09-10T19:01:29.560760+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:32.602448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:32.602448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0052503430000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":121,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.564522+0000","last_change":"2024-09-10T19:00:42.519802+0000","last_active":"2024-09-10T19:01:29.564522+0000","last_peered":"2024-09-10T19:01:29.564522+0000","last_clean":"2024-09-10T19:01:29.564522+0000","last_became_active":"2024-09-10T19:00:42.519492+0000","last_became_peered":"2024-09-10T19:00:42.519492+0000","last_unstale":"2024-09-10T19:01:29.564522+0000","last_undegraded":"2024-09-10T19:01:29.564522+0000","last_fullsized":"2024-09-10T19:01:29.564522+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:53.608029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:58:53.608029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058174170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.11","version":"70'81","reported_seq":263,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.557227+0000","last_change":"2024-09-10T19:00:42.517609+0000","last_active":"2024-09-10T19:01:29.557227+0000","last_peered":"2024-09-10T19:01:29.557227+0000","last_clean":"2024-09-10T19:01:29.557227+0000","last_became_active":"2024-09-10T19:00:42.517311+0000","last_became_peered":"2024-09-10T19:00:42.517311+0000","last_unstale":"2024-09-10T19:01:29.557227+0000","last_undegraded":"2024-09-10T19:01:29.557227+0000","last_fullsized":"2024-09-10T19:01:29.557227+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:58:48.552631+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:58:48.552631+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012534919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"232'20","reported_seq":453,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.559236+0000","last_change":"2024-09-10T19:00:07.147063+0000","last_active":"2024-09-10T19:01:29.559236+0000","last_peered":"2024-09-10T19:01:29.559236+0000","last_clean":"2024-09-10T19:01:29.559236+0000","last_became_active":"2024-09-10T18:57:02.444437+0000","last_became_peered":"2024-09-10T18:57:02.444437+0000","last_unstale":"2024-09-10T19:01:29.559236+0000","last_undegraded":"2024-09-10T19:01:29.559236+0000","last_fullsized":"2024-09-10T19:01:29.559236+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'20","last_scrub_stamp":"2024-09-10T19:00:07.146928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:00:07.146928+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0071404040000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":380,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.559580+0000","last_change":"2024-09-10T18:58:50.541776+0000","last_active":"2024-09-10T19:01:29.559580+0000","last_peered":"2024-09-10T19:01:29.559580+0000","last_clean":"2024-09-10T19:01:29.559580+0000","last_became_active":"2024-09-10T18:57:02.445710+0000","last_became_peered":"2024-09-10T18:57:02.445710+0000","last_unstale":"2024-09-10T19:01:29.559580+0000","last_undegraded":"2024-09-10T19:01:29.559580+0000","last_fullsized":"2024-09-10T19:01:29.559580+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:50.541691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:58:50.541691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0089012770000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"304'11","reported_seq":605,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:36.268434+0000","last_change":"2024-09-10T18:59:55.525278+0000","last_active":"2024-09-10T19:01:36.268434+0000","last_peered":"2024-09-10T19:01:36.268434+0000","last_clean":"2024-09-10T19:01:36.268434+0000","last_became_active":"2024-09-10T18:58:58.489370+0000","last_became_peered":"2024-09-10T18:58:58.489370+0000","last_unstale":"2024-09-10T19:01:36.268434+0000","last_undegraded":"2024-09-10T19:01:36.268434+0000","last_fullsized":"2024-09-10T19:01:36.268434+0000","mapping_epoch":303,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":304,"parent":"0.0","parent_split_bits":0,"last_scrub":"304'11","last_scrub_stamp":"2024-09-10T18:59:55.525185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:59:55.525185+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020778110000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":48,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.556949+0000","last_change":"2024-09-10T18:59:48.745646+0000","last_active":"2024-09-10T19:01:29.556949+0000","last_peered":"2024-09-10T19:01:29.556949+0000","last_clean":"2024-09-10T19:01:29.556949+0000","last_became_active":"2024-09-10T18:59:48.744923+0000","last_became_peered":"2024-09-10T18:59:48.744923+0000","last_unstale":"2024-09-10T19:01:29.556949+0000","last_undegraded":"2024-09-10T19:01:29.556949+0000","last_fullsized":"2024-09-10T19:01:29.556949+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:57.812395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:57.476724+0000","last_clean_scrub_stamp":"2024-09-10T18:58:57.812395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.10","version":"70'81","reported_seq":238,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.559013+0000","last_change":"2024-09-10T19:00:04.612412+0000","last_active":"2024-09-10T19:01:29.559013+0000","last_peered":"2024-09-10T19:01:29.559013+0000","last_clean":"2024-09-10T19:01:29.559013+0000","last_became_active":"2024-09-10T18:57:02.443782+0000","last_became_peered":"2024-09-10T18:57:02.443782+0000","last_unstale":"2024-09-10T19:01:29.559013+0000","last_undegraded":"2024-09-10T19:01:29.559013+0000","last_fullsized":"2024-09-10T19:01:29.559013+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:00:04.612324+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:00:04.612324+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025961159999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"232'15","reported_seq":40,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.560075+0000","last_change":"2024-09-10T19:00:42.530883+0000","last_active":"2024-09-10T19:01:29.560075+0000","last_peered":"2024-09-10T19:01:29.560075+0000","last_clean":"2024-09-10T19:01:29.560075+0000","last_became_active":"2024-09-10T19:00:42.518672+0000","last_became_peered":"2024-09-10T19:00:42.518672+0000","last_unstale":"2024-09-10T19:01:29.560075+0000","last_undegraded":"2024-09-10T19:01:29.560075+0000","last_fullsized":"2024-09-10T19:01:29.560075+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T18:59:16.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:59:16.949954+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":337,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.556010+0000","last_change":"2024-09-10T18:56:23.096461+0000","last_active":"2024-09-10T19:01:29.556010+0000","last_peered":"2024-09-10T19:01:29.556010+0000","last_clean":"2024-09-10T19:01:29.556010+0000","last_became_active":"2024-09-10T18:55:40.390697+0000","last_became_peered":"2024-09-10T18:55:40.390697+0000","last_unstale":"2024-09-10T19:01:29.556010+0000","last_undegraded":"2024-09-10T19:01:29.556010+0000","last_fullsized":"2024-09-10T19:01:29.556010+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:23.096373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:23.096373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0057615890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"273'11","reported_seq":588,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.590932+0000","last_change":"2024-09-10T18:59:53.158730+0000","last_active":"2024-09-10T19:01:34.590932+0000","last_peered":"2024-09-10T19:01:34.590932+0000","last_clean":"2024-09-10T19:01:34.590932+0000","last_became_active":"2024-09-10T18:57:02.445268+0000","last_became_peered":"2024-09-10T18:57:02.445268+0000","last_unstale":"2024-09-10T19:01:34.590932+0000","last_undegraded":"2024-09-10T19:01:34.590932+0000","last_fullsized":"2024-09-10T19:01:34.590932+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'11","last_scrub_stamp":"2024-09-10T18:59:53.158664+0000","last_deep_scrub":"81'9","last_deep_scrub_stamp":"2024-09-10T18:46:54.800790+0000","last_clean_scrub_stamp":"2024-09-10T18:59:53.158664+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017427930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":379,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.562870+0000","last_change":"2024-09-10T18:56:20.584899+0000","last_active":"2024-09-10T19:01:29.562870+0000","last_peered":"2024-09-10T19:01:29.562870+0000","last_clean":"2024-09-10T19:01:29.562870+0000","last_became_active":"2024-09-10T18:55:40.391662+0000","last_became_peered":"2024-09-10T18:55:40.391662+0000","last_unstale":"2024-09-10T19:01:29.562870+0000","last_undegraded":"2024-09-10T19:01:29.562870+0000","last_fullsized":"2024-09-10T19:01:29.562870+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:20.584808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_clean_scrub_stamp":"2024-09-10T18:56:20.584808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0035961249999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"232'15","reported_seq":432,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.555087+0000","last_change":"2024-09-10T18:59:53.320831+0000","last_active":"2024-09-10T19:01:29.555087+0000","last_peered":"2024-09-10T19:01:29.555087+0000","last_clean":"2024-09-10T19:01:29.555087+0000","last_became_active":"2024-09-10T18:55:40.400788+0000","last_became_peered":"2024-09-10T18:55:40.400788+0000","last_unstale":"2024-09-10T19:01:29.555087+0000","last_undegraded":"2024-09-10T19:01:29.555087+0000","last_fullsized":"2024-09-10T19:01:29.555087+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T18:59:53.320757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:59:53.320757+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001889578,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":411,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.564160+0000","last_change":"2024-09-10T19:00:42.518606+0000","last_active":"2024-09-10T19:01:29.564160+0000","last_peered":"2024-09-10T19:01:29.564160+0000","last_clean":"2024-09-10T19:01:29.564160+0000","last_became_active":"2024-09-10T19:00:42.518310+0000","last_became_peered":"2024-09-10T19:00:42.518310+0000","last_unstale":"2024-09-10T19:01:29.564160+0000","last_undegraded":"2024-09-10T19:01:29.564160+0000","last_fullsized":"2024-09-10T19:01:29.564160+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:48.622701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:59:48.622701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001531279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":362,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.558889+0000","last_change":"2024-09-10T18:56:04.753218+0000","last_active":"2024-09-10T19:01:29.558889+0000","last_peered":"2024-09-10T19:01:29.558889+0000","last_clean":"2024-09-10T19:01:29.558889+0000","last_became_active":"2024-09-10T18:55:40.398243+0000","last_became_peered":"2024-09-10T18:55:40.398243+0000","last_unstale":"2024-09-10T19:01:29.558889+0000","last_undegraded":"2024-09-10T19:01:29.558889+0000","last_fullsized":"2024-09-10T19:01:29.558889+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:04.753133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:03.571786+0000","last_clean_scrub_stamp":"2024-09-10T18:56:04.753133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0069575230000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":352,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.556538+0000","last_change":"2024-09-10T19:00:42.517459+0000","last_active":"2024-09-10T19:01:29.556538+0000","last_peered":"2024-09-10T19:01:29.556538+0000","last_clean":"2024-09-10T19:01:29.556538+0000","last_became_active":"2024-09-10T19:00:42.516833+0000","last_became_peered":"2024-09-10T19:00:42.516833+0000","last_unstale":"2024-09-10T19:01:29.556538+0000","last_undegraded":"2024-09-10T19:01:29.556538+0000","last_fullsized":"2024-09-10T19:01:29.556538+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:46.863758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:04.631558+0000","last_clean_scrub_stamp":"2024-09-10T18:59:46.863758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0031907379999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"232'24","reported_seq":392,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.562184+0000","last_change":"2024-09-10T18:58:58.698097+0000","last_active":"2024-09-10T19:01:29.562184+0000","last_peered":"2024-09-10T19:01:29.562184+0000","last_clean":"2024-09-10T19:01:29.562184+0000","last_became_active":"2024-09-10T18:45:20.056885+0000","last_became_peered":"2024-09-10T18:45:20.056885+0000","last_unstale":"2024-09-10T19:01:29.562184+0000","last_undegraded":"2024-09-10T19:01:29.562184+0000","last_fullsized":"2024-09-10T19:01:29.562184+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T18:58:58.698003+0000","last_deep_scrub":"223'16","last_deep_scrub_stamp":"2024-09-10T18:54:47.324044+0000","last_clean_scrub_stamp":"2024-09-10T18:58:58.698003+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.006955676,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":380,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.555759+0000","last_change":"2024-09-10T19:01:07.432016+0000","last_active":"2024-09-10T19:01:29.555759+0000","last_peered":"2024-09-10T19:01:29.555759+0000","last_clean":"2024-09-10T19:01:29.555759+0000","last_became_active":"2024-09-10T18:55:40.399198+0000","last_became_peered":"2024-09-10T18:55:40.399198+0000","last_unstale":"2024-09-10T19:01:29.555759+0000","last_undegraded":"2024-09-10T19:01:29.555759+0000","last_fullsized":"2024-09-10T19:01:29.555759+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:51.138240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:00.029389+0000","last_clean_scrub_stamp":"2024-09-10T18:58:51.138240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0086738059999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":362,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.554883+0000","last_change":"2024-09-10T19:00:59.501368+0000","last_active":"2024-09-10T19:01:29.554883+0000","last_peered":"2024-09-10T19:01:29.554883+0000","last_clean":"2024-09-10T19:01:29.554883+0000","last_became_active":"2024-09-10T18:45:22.070065+0000","last_became_peered":"2024-09-10T18:45:22.070065+0000","last_unstale":"2024-09-10T19:01:29.554883+0000","last_undegraded":"2024-09-10T19:01:29.554883+0000","last_fullsized":"2024-09-10T19:01:29.554883+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:31.782918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:31.782918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043032130000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":360,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.558660+0000","last_change":"2024-09-10T18:56:06.792096+0000","last_active":"2024-09-10T19:01:29.558660+0000","last_peered":"2024-09-10T19:01:29.558660+0000","last_clean":"2024-09-10T19:01:29.558660+0000","last_became_active":"2024-09-10T18:55:40.395595+0000","last_became_peered":"2024-09-10T18:55:40.395595+0000","last_unstale":"2024-09-10T19:01:29.558660+0000","last_undegraded":"2024-09-10T19:01:29.558660+0000","last_fullsized":"2024-09-10T19:01:29.558660+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:06.791860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:06.791860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046852380000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"232'8","reported_seq":369,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.558235+0000","last_change":"2024-09-10T18:56:30.802774+0000","last_active":"2024-09-10T19:01:29.558235+0000","last_peered":"2024-09-10T19:01:29.558235+0000","last_clean":"2024-09-10T19:01:29.558235+0000","last_became_active":"2024-09-10T18:45:20.059205+0000","last_became_peered":"2024-09-10T18:45:20.059205+0000","last_unstale":"2024-09-10T19:01:29.558235+0000","last_undegraded":"2024-09-10T19:01:29.558235+0000","last_fullsized":"2024-09-10T19:01:29.558235+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'8","last_scrub_stamp":"2024-09-10T18:56:30.802679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:30.802679+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034579609999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":376,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.560761+0000","last_change":"2024-09-10T18:58:53.527829+0000","last_active":"2024-09-10T19:01:29.560761+0000","last_peered":"2024-09-10T19:01:29.560761+0000","last_clean":"2024-09-10T19:01:29.560761+0000","last_became_active":"2024-09-10T18:47:36.415866+0000","last_became_peered":"2024-09-10T18:47:36.415866+0000","last_unstale":"2024-09-10T19:01:29.560761+0000","last_undegraded":"2024-09-10T19:01:29.560761+0000","last_fullsized":"2024-09-10T19:01:29.560761+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:53.527746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:58:53.527746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058191349999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":387,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.563676+0000","last_change":"2024-09-10T18:57:02.454548+0000","last_active":"2024-09-10T19:01:29.563676+0000","last_peered":"2024-09-10T19:01:29.563676+0000","last_clean":"2024-09-10T19:01:29.563676+0000","last_became_active":"2024-09-10T18:57:02.453974+0000","last_became_peered":"2024-09-10T18:57:02.453974+0000","last_unstale":"2024-09-10T19:01:29.563676+0000","last_undegraded":"2024-09-10T19:01:29.563676+0000","last_fullsized":"2024-09-10T19:01:29.563676+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:23.195060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_clean_scrub_stamp":"2024-09-10T18:56:23.195060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047613259999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":157,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.561571+0000","last_change":"2024-09-10T18:59:55.812370+0000","last_active":"2024-09-10T19:01:29.561571+0000","last_peered":"2024-09-10T19:01:29.561571+0000","last_clean":"2024-09-10T19:01:29.561571+0000","last_became_active":"2024-09-10T18:57:02.447566+0000","last_became_peered":"2024-09-10T18:57:02.447566+0000","last_unstale":"2024-09-10T19:01:29.561571+0000","last_undegraded":"2024-09-10T19:01:29.561571+0000","last_fullsized":"2024-09-10T19:01:29.561571+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:55.812312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:55.145638+0000","last_clean_scrub_stamp":"2024-09-10T18:59:55.812312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0055522669999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.14","version":"70'81","reported_seq":261,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.562553+0000","last_change":"2024-09-10T19:00:58.100610+0000","last_active":"2024-09-10T19:01:29.562553+0000","last_peered":"2024-09-10T19:01:29.562553+0000","last_clean":"2024-09-10T19:01:29.562553+0000","last_became_active":"2024-09-10T18:57:02.442457+0000","last_became_peered":"2024-09-10T18:57:02.442457+0000","last_unstale":"2024-09-10T19:01:29.562553+0000","last_undegraded":"2024-09-10T19:01:29.562553+0000","last_fullsized":"2024-09-10T19:01:29.562553+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:55:34.791223+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:57.184383+0000","last_clean_scrub_stamp":"2024-09-10T18:55:34.791223+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016709736999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17890,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":7,"num_write":3,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":35780,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.11","version":"232'24","reported_seq":415,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.558488+0000","last_change":"2024-09-10T18:59:10.879686+0000","last_active":"2024-09-10T19:01:29.558488+0000","last_peered":"2024-09-10T19:01:29.558488+0000","last_clean":"2024-09-10T19:01:29.558488+0000","last_became_active":"2024-09-10T18:57:02.461909+0000","last_became_peered":"2024-09-10T18:57:02.461909+0000","last_unstale":"2024-09-10T19:01:29.558488+0000","last_undegraded":"2024-09-10T19:01:29.558488+0000","last_fullsized":"2024-09-10T19:01:29.558488+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T18:59:10.879597+0000","last_deep_scrub":"79'16","last_deep_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_clean_scrub_stamp":"2024-09-10T18:59:10.879597+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0090530790000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"56'1","reported_seq":392,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.564424+0000","last_change":"2024-09-10T18:56:25.211179+0000","last_active":"2024-09-10T19:01:29.564424+0000","last_peered":"2024-09-10T19:01:29.564424+0000","last_clean":"2024-09-10T19:01:29.564424+0000","last_became_active":"2024-09-10T18:47:36.421985+0000","last_became_peered":"2024-09-10T18:47:36.421985+0000","last_unstale":"2024-09-10T19:01:29.564424+0000","last_undegraded":"2024-09-10T19:01:29.564424+0000","last_fullsized":"2024-09-10T19:01:29.564424+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:56:25.211095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:25.211095+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025468880000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":104,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.560840+0000","last_change":"2024-09-10T19:01:07.015846+0000","last_active":"2024-09-10T19:01:29.560840+0000","last_peered":"2024-09-10T19:01:29.560840+0000","last_clean":"2024-09-10T19:01:29.560840+0000","last_became_active":"2024-09-10T18:57:02.435124+0000","last_became_peered":"2024-09-10T18:57:02.435124+0000","last_unstale":"2024-09-10T19:01:29.560840+0000","last_undegraded":"2024-09-10T19:01:29.560840+0000","last_fullsized":"2024-09-10T19:01:29.560840+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:12.852769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:12.852769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":370,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.558653+0000","last_change":"2024-09-10T18:56:09.903930+0000","last_active":"2024-09-10T19:01:29.558653+0000","last_peered":"2024-09-10T19:01:29.558653+0000","last_clean":"2024-09-10T19:01:29.558653+0000","last_became_active":"2024-09-10T18:55:40.397523+0000","last_became_peered":"2024-09-10T18:55:40.397523+0000","last_unstale":"2024-09-10T19:01:29.558653+0000","last_undegraded":"2024-09-10T19:01:29.558653+0000","last_fullsized":"2024-09-10T19:01:29.558653+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:09.903846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:09.903846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011721984,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"78'2","reported_seq":147,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.559756+0000","last_change":"2024-09-10T18:58:48.472078+0000","last_active":"2024-09-10T19:01:29.559756+0000","last_peered":"2024-09-10T19:01:29.559756+0000","last_clean":"2024-09-10T19:01:29.559756+0000","last_became_active":"2024-09-10T18:57:02.447198+0000","last_became_peered":"2024-09-10T18:57:02.447198+0000","last_unstale":"2024-09-10T19:01:29.559756+0000","last_undegraded":"2024-09-10T19:01:29.559756+0000","last_fullsized":"2024-09-10T19:01:29.559756+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'2","last_scrub_stamp":"2024-09-10T18:58:48.472024+0000","last_deep_scrub":"78'2","last_deep_scrub_stamp":"2024-09-10T18:46:37.581130+0000","last_clean_scrub_stamp":"2024-09-10T18:58:48.472024+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.022088824,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":380,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.559168+0000","last_change":"2024-09-10T18:59:53.505658+0000","last_active":"2024-09-10T19:01:29.559168+0000","last_peered":"2024-09-10T19:01:29.559168+0000","last_clean":"2024-09-10T19:01:29.559168+0000","last_became_active":"2024-09-10T18:57:02.443353+0000","last_became_peered":"2024-09-10T18:57:02.443353+0000","last_unstale":"2024-09-10T19:01:29.559168+0000","last_undegraded":"2024-09-10T19:01:29.559168+0000","last_fullsized":"2024-09-10T19:01:29.559168+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:53.505561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:59:53.505561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001238605,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"232'20","reported_seq":386,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.557601+0000","last_change":"2024-09-10T18:57:02.486956+0000","last_active":"2024-09-10T19:01:29.557601+0000","last_peered":"2024-09-10T19:01:29.557601+0000","last_clean":"2024-09-10T19:01:29.557601+0000","last_became_active":"2024-09-10T18:57:02.452473+0000","last_became_peered":"2024-09-10T18:57:02.452473+0000","last_unstale":"2024-09-10T19:01:29.557601+0000","last_undegraded":"2024-09-10T19:01:29.557601+0000","last_fullsized":"2024-09-10T19:01:29.557601+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'20","last_scrub_stamp":"2024-09-10T18:56:27.158686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:27.158686+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0090371180000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":23,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"273'10","reported_seq":600,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:36.267725+0000","last_change":"2024-09-10T18:59:22.516714+0000","last_active":"2024-09-10T19:01:36.267725+0000","last_peered":"2024-09-10T19:01:36.267725+0000","last_clean":"2024-09-10T19:01:36.267725+0000","last_became_active":"2024-09-10T18:57:02.441763+0000","last_became_peered":"2024-09-10T18:57:02.441763+0000","last_unstale":"2024-09-10T19:01:36.267725+0000","last_undegraded":"2024-09-10T19:01:36.267725+0000","last_fullsized":"2024-09-10T19:01:36.267725+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'10","last_scrub_stamp":"2024-09-10T18:59:22.516625+0000","last_deep_scrub":"190'9","last_deep_scrub_stamp":"2024-09-10T18:53:07.746309+0000","last_clean_scrub_stamp":"2024-09-10T18:59:22.516625+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021912820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":357,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.555329+0000","last_change":"2024-09-10T18:58:59.150773+0000","last_active":"2024-09-10T19:01:29.555329+0000","last_peered":"2024-09-10T19:01:29.555329+0000","last_clean":"2024-09-10T19:01:29.555329+0000","last_became_active":"2024-09-10T18:45:24.082318+0000","last_became_peered":"2024-09-10T18:45:24.082318+0000","last_unstale":"2024-09-10T19:01:29.555329+0000","last_undegraded":"2024-09-10T19:01:29.555329+0000","last_fullsized":"2024-09-10T19:01:29.555329+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:59.148958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:03.475299+0000","last_clean_scrub_stamp":"2024-09-10T18:58:59.148958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041797229999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":376,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.558181+0000","last_change":"2024-09-10T19:00:10.182129+0000","last_active":"2024-09-10T19:01:29.558181+0000","last_peered":"2024-09-10T19:01:29.558181+0000","last_clean":"2024-09-10T19:01:29.558181+0000","last_became_active":"2024-09-10T18:55:40.392524+0000","last_became_peered":"2024-09-10T18:55:40.392524+0000","last_unstale":"2024-09-10T19:01:29.558181+0000","last_undegraded":"2024-09-10T19:01:29.558181+0000","last_fullsized":"2024-09-10T19:01:29.558181+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:10.181982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:00:10.181982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029510420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"232'19","reported_seq":403,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.564563+0000","last_change":"2024-09-10T18:59:09.544279+0000","last_active":"2024-09-10T19:01:29.564563+0000","last_peered":"2024-09-10T19:01:29.564563+0000","last_clean":"2024-09-10T19:01:29.564563+0000","last_became_active":"2024-09-10T18:45:20.058717+0000","last_became_peered":"2024-09-10T18:45:20.058717+0000","last_unstale":"2024-09-10T19:01:29.564563+0000","last_undegraded":"2024-09-10T19:01:29.564563+0000","last_fullsized":"2024-09-10T19:01:29.564563+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T18:59:09.544194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:59:09.544194+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043401830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":340,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.557667+0000","last_change":"2024-09-10T18:59:57.862722+0000","last_active":"2024-09-10T19:01:29.557667+0000","last_peered":"2024-09-10T19:01:29.557667+0000","last_clean":"2024-09-10T19:01:29.557667+0000","last_became_active":"2024-09-10T18:57:02.453464+0000","last_became_peered":"2024-09-10T18:57:02.453464+0000","last_unstale":"2024-09-10T19:01:29.557667+0000","last_undegraded":"2024-09-10T19:01:29.557667+0000","last_fullsized":"2024-09-10T19:01:29.557667+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:57.862666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:59:57.862666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001262242,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":352,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.563080+0000","last_change":"2024-09-10T18:58:55.726863+0000","last_active":"2024-09-10T19:01:29.563080+0000","last_peered":"2024-09-10T19:01:29.563080+0000","last_clean":"2024-09-10T19:01:29.563080+0000","last_became_active":"2024-09-10T18:45:24.267251+0000","last_became_peered":"2024-09-10T18:45:24.267251+0000","last_unstale":"2024-09-10T19:01:29.563080+0000","last_undegraded":"2024-09-10T19:01:29.563080+0000","last_fullsized":"2024-09-10T19:01:29.563080+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:55.726799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:58:55.726799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001154721,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"306'2","reported_seq":398,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.564484+0000","last_change":"2024-09-10T18:59:20.534927+0000","last_active":"2024-09-10T19:01:29.564484+0000","last_peered":"2024-09-10T19:01:29.564484+0000","last_clean":"2024-09-10T19:01:29.564484+0000","last_became_active":"2024-09-10T18:57:02.435758+0000","last_became_peered":"2024-09-10T18:57:02.435758+0000","last_unstale":"2024-09-10T19:01:29.564484+0000","last_undegraded":"2024-09-10T19:01:29.564484+0000","last_fullsized":"2024-09-10T19:01:29.564484+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"306'2","last_scrub_stamp":"2024-09-10T18:59:20.534869+0000","last_deep_scrub":"306'2","last_deep_scrub_stamp":"2024-09-10T18:59:20.534869+0000","last_clean_scrub_stamp":"2024-09-10T18:59:20.534869+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0072446200000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"258'34","reported_seq":476,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.556090+0000","last_change":"2024-09-10T19:00:52.459800+0000","last_active":"2024-09-10T19:01:29.556090+0000","last_peered":"2024-09-10T19:01:29.556090+0000","last_clean":"2024-09-10T19:01:29.556090+0000","last_became_active":"2024-09-10T18:59:48.744293+0000","last_became_peered":"2024-09-10T18:59:48.744293+0000","last_unstale":"2024-09-10T19:01:29.556090+0000","last_undegraded":"2024-09-10T19:01:29.556090+0000","last_fullsized":"2024-09-10T19:01:29.556090+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'34","last_scrub_stamp":"2024-09-10T18:56:23.838951+0000","last_deep_scrub":"258'34","last_deep_scrub_stamp":"2024-09-10T18:56:23.838951+0000","last_clean_scrub_stamp":"2024-09-10T18:56:23.838951+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.035929888,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":366,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.555226+0000","last_change":"2024-09-10T19:00:54.441149+0000","last_active":"2024-09-10T19:01:29.555226+0000","last_peered":"2024-09-10T19:01:29.555226+0000","last_clean":"2024-09-10T19:01:29.555226+0000","last_became_active":"2024-09-10T18:55:40.397870+0000","last_became_peered":"2024-09-10T18:55:40.397870+0000","last_unstale":"2024-09-10T19:01:29.555226+0000","last_undegraded":"2024-09-10T19:01:29.555226+0000","last_fullsized":"2024-09-10T19:01:29.555226+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:13.610976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:13.610976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010999553,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":373,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.559401+0000","last_change":"2024-09-10T19:00:42.520936+0000","last_active":"2024-09-10T19:01:29.559401+0000","last_peered":"2024-09-10T19:01:29.559401+0000","last_clean":"2024-09-10T19:01:29.559401+0000","last_became_active":"2024-09-10T19:00:42.520239+0000","last_became_peered":"2024-09-10T19:00:42.520239+0000","last_unstale":"2024-09-10T19:01:29.559401+0000","last_undegraded":"2024-09-10T19:01:29.559401+0000","last_fullsized":"2024-09-10T19:01:29.559401+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:17.985545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:17.985545+0000","last_clean_scrub_stamp":"2024-09-10T18:59:17.985545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0086180830000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":379,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.559044+0000","last_change":"2024-09-10T18:58:57.919890+0000","last_active":"2024-09-10T19:01:29.559044+0000","last_peered":"2024-09-10T19:01:29.559044+0000","last_clean":"2024-09-10T19:01:29.559044+0000","last_became_active":"2024-09-10T18:57:02.460855+0000","last_became_peered":"2024-09-10T18:57:02.460855+0000","last_unstale":"2024-09-10T19:01:29.559044+0000","last_undegraded":"2024-09-10T19:01:29.559044+0000","last_fullsized":"2024-09-10T19:01:29.559044+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:57.919515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:58:57.919515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.01355234,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"232'18","reported_seq":389,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.559338+0000","last_change":"2024-09-10T18:58:52.890712+0000","last_active":"2024-09-10T19:01:29.559338+0000","last_peered":"2024-09-10T19:01:29.559338+0000","last_clean":"2024-09-10T19:01:29.559338+0000","last_became_active":"2024-09-10T18:45:20.058391+0000","last_became_peered":"2024-09-10T18:45:20.058391+0000","last_unstale":"2024-09-10T19:01:29.559338+0000","last_undegraded":"2024-09-10T19:01:29.559338+0000","last_fullsized":"2024-09-10T19:01:29.559338+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'18","last_scrub_stamp":"2024-09-10T18:58:52.890618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:58:52.890618+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016012082,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":384,"reported_epoch":358,"state":"active+clean","last_fresh":"2024-09-10T19:01:29.562560+0000","last_change":"2024-09-10T18:58:56.690660+0000","last_active":"2024-09-10T19:01:29.562560+0000","last_peered":"2024-09-10T19:01:29.562560+0000","last_clean":"2024-09-10T19:01:29.562560+0000","last_became_active":"2024-09-10T18:57:02.449135+0000","last_became_peered":"2024-09-10T18:57:02.449135+0000","last_unstale":"2024-09-10T19:01:29.562560+0000","last_undegraded":"2024-09-10T19:01:29.562560+0000","last_fullsized":"2024-09-10T19:01:29.562560+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:56.690606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:58:56.690606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060631599999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":7250640920,"num_objects":110637,"num_object_clones":0,"num_object_copies":331911,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110637,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":166675,"num_write_kb":8873921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21975,"num_bytes_recovered":1440088064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":23339282432,"data_stored":23339270216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57494,"ondisk_log_size":57494,"up":48,"acting":48,"num_store_stats":6},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":421,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1118,"num_read_kb":875,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":3372,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":499712,"data_stored":13268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":758,"ondisk_log_size":758,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":967,"num_read_kb":962,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":608258,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":7,"num_write":3,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":298468,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1701,"ondisk_log_size":1701,"up":63,"acting":63,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":0,"seq":223338299593,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":43,"seq":184683593934,"num_pgs":86,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4065604,"kb_used_data":3163132,"kb_used_omap":1,"kb_used_meta":902462,"kb_avail":89679548,"statfs":{"total":95995035648,"available":91831857152,"internally_reserved":0,"allocated":3239047168,"data_stored":3238631131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":924121432},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:00:54 2024","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.1830000000000001,"15min":0.98899999999999999},"min":{"1min":0.34399999999999997,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":5.0910000000000002,"5min":27.545000000000002,"15min":27.545000000000002},"last":0.31900000000000001},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.3149999999999999,"15min":1.0649999999999999},"min":{"1min":0.27900000000000003,"5min":0.219,"15min":0.219},"max":{"1min":4.6180000000000003,"5min":27.568000000000001,"15min":27.568000000000001},"last":2.2679999999999998}]},{"osd":1,"last update":"Tue Sep 10 19:01:03 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":1.7370000000000001,"15min":1.2889999999999999},"min":{"1min":0.253,"5min":0.22900000000000001,"15min":0.219},"max":{"1min":4.6200000000000001,"5min":45.311,"15min":45.311},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.84599999999999997,"5min":1.645,"15min":1.181},"min":{"1min":0.26300000000000001,"5min":0.23300000000000001,"15min":0.214},"max":{"1min":3.5289999999999999,"5min":45.027999999999999,"15min":45.027999999999999},"last":0.74299999999999999}]},{"osd":2,"last update":"Tue Sep 10 19:01:03 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.64800000000000002,"15min":0.78400000000000003},"min":{"1min":0.32700000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":3.4009999999999998,"5min":3.4009999999999998,"15min":4.4740000000000002},"last":2.2320000000000002},{"interface":"front","average":{"1min":1.0660000000000001,"5min":0.60499999999999998,"15min":0.71899999999999997},"min":{"1min":0.28499999999999998,"5min":0.249,"15min":0.249},"max":{"1min":4.6580000000000004,"5min":4.6580000000000004,"15min":4.8150000000000004},"last":0.62}]},{"osd":3,"last update":"Tue Sep 10 19:01:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":2.3410000000000002,"15min":1.4239999999999999},"min":{"1min":0.34200000000000003,"5min":0.29599999999999999,"15min":0.25700000000000001},"max":{"1min":5.0220000000000002,"5min":76.977999999999994,"15min":76.977999999999994},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.88600000000000001,"5min":2.2999999999999998,"15min":1.522},"min":{"1min":0.29799999999999999,"5min":0.27500000000000002,"15min":0.253},"max":{"1min":2.2080000000000002,"5min":76.879999999999995,"15min":76.879999999999995},"last":2.2949999999999999}]},{"osd":4,"last update":"Tue Sep 10 19:01:03 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":0.64000000000000001,"15min":1.3280000000000001},"min":{"1min":0.25,"5min":0.23400000000000001,"15min":0.20999999999999999},"max":{"1min":4.7220000000000004,"5min":4.7220000000000004,"15min":165.03299999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.625,"15min":1.2430000000000001},"min":{"1min":0.26200000000000001,"5min":0.26000000000000001,"15min":0.22500000000000001},"max":{"1min":3.5619999999999998,"5min":3.5619999999999998,"15min":163.959},"last":0.58299999999999996}]},{"osd":5,"last update":"Tue Sep 10 19:01:03 2024","interfaces":[{"interface":"back","average":{"1min":12.706,"5min":3.0459999999999998,"15min":1.7549999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.24099999999999999},"max":{"1min":288.56999999999999,"5min":288.56999999999999,"15min":288.56999999999999},"last":2.3780000000000001},{"interface":"front","average":{"1min":12.539,"5min":2.9660000000000002,"15min":1.7529999999999999},"min":{"1min":0.29799999999999999,"5min":0.27300000000000002,"15min":0.23999999999999999},"max":{"1min":291.05399999999997,"5min":291.05399999999997,"15min":291.05399999999997},"last":0.66400000000000003}]},{"osd":7,"last update":"Tue Sep 10 19:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.67800000000000005,"15min":0.83499999999999996},"min":{"1min":0.32200000000000001,"5min":0.27900000000000003,"15min":0.26200000000000001},"max":{"1min":2.2839999999999998,"5min":2.98,"15min":5.3680000000000003},"last":2.3279999999999998},{"interface":"front","average":{"1min":1.1379999999999999,"5min":0.72499999999999998,"15min":0.88900000000000001},"min":{"1min":0.31,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":4.6820000000000004,"5min":4.6820000000000004,"15min":7.0350000000000001},"last":0.64500000000000002}]}]},{"osd":5,"up_from":37,"seq":158913790162,"num_pgs":84,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4500096,"kb_used_data":3608120,"kb_used_omap":1,"kb_used_meta":891966,"kb_avail":89245056,"statfs":{"total":95995035648,"available":91386937344,"internally_reserved":0,"allocated":3694714880,"data_stored":3694301944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":913373528},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:00:49 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":2.7530000000000001,"15min":1.95},"min":{"1min":0.33000000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":2.1600000000000001,"5min":82.465000000000003,"15min":82.465000000000003},"last":4.0650000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":2.7970000000000002,"15min":2.113},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":1.645,"5min":82.519000000000005,"15min":82.519000000000005},"last":2.8260000000000001}]},{"osd":1,"last update":"Tue Sep 10 19:00:49 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":2.8490000000000002,"15min":1.524},"min":{"1min":0.313,"5min":0.20999999999999999,"15min":0.19500000000000001},"max":{"1min":3.851,"5min":67.582999999999998,"15min":67.582999999999998},"last":4.0179999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":2.7879999999999998,"15min":1.667},"min":{"1min":0.35099999999999998,"5min":0.29699999999999999,"15min":0.20399999999999999},"max":{"1min":1.7250000000000001,"5min":67.603999999999999,"15min":67.603999999999999},"last":2.3700000000000001}]},{"osd":2,"last update":"Tue Sep 10 19:00:49 2024","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":0.73999999999999999,"15min":0.76000000000000001},"min":{"1min":0.30299999999999999,"5min":0.27200000000000002,"15min":0.249},"max":{"1min":7.8879999999999999,"5min":7.8879999999999999,"15min":7.8879999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.996,"5min":0.70599999999999996,"15min":0.77400000000000002},"min":{"1min":0.28499999999999998,"5min":0.27100000000000002,"15min":0.24399999999999999},"max":{"1min":6.7549999999999999,"5min":6.7549999999999999,"15min":6.7549999999999999},"last":0.28199999999999997}]},{"osd":3,"last update":"Tue Sep 10 19:00:49 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":4.7930000000000001,"15min":2.7029999999999998},"min":{"1min":0.27900000000000003,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":3.5510000000000002,"5min":123.34399999999999,"15min":123.34399999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.80300000000000005,"5min":3.1579999999999999,"15min":2.2810000000000001},"min":{"1min":0.36599999999999999,"5min":0.27300000000000002,"15min":0.245},"max":{"1min":2.7719999999999998,"5min":82.415000000000006,"15min":82.415000000000006},"last":0.312}]},{"osd":4,"last update":"Tue Sep 10 19:00:49 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.58199999999999996,"15min":0.84199999999999997},"min":{"1min":0.27500000000000002,"5min":0.221,"15min":0.20899999999999999},"max":{"1min":1.706,"5min":2.9940000000000002,"15min":9.0310000000000006},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.54100000000000004,"15min":0.78400000000000003},"min":{"1min":0.27300000000000002,"5min":0.23899999999999999,"15min":0.23400000000000001},"max":{"1min":2.742,"5min":2.742,"15min":11.182},"last":0.61099999999999999}]},{"osd":6,"last update":"Tue Sep 10 19:01:10 2024","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":0.69899999999999995,"15min":0.96599999999999997},"min":{"1min":0.30199999999999999,"5min":0.23999999999999999,"15min":0.23400000000000001},"max":{"1min":4.1200000000000001,"5min":4.1200000000000001,"15min":11.17},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.0669999999999999,"5min":0.622,"15min":0.94999999999999996},"min":{"1min":0.38500000000000001,"5min":0.27200000000000002,"15min":0.26700000000000002},"max":{"1min":3.1930000000000001,"5min":3.1930000000000001,"15min":11.18},"last":0.63200000000000001}]},{"osd":7,"last update":"Tue Sep 10 19:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":0.69399999999999995,"15min":0.94399999999999995},"min":{"1min":0.308,"5min":0.28999999999999998,"15min":0.27600000000000002},"max":{"1min":2.6859999999999999,"5min":3.665,"15min":10.782999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.61299999999999999,"15min":0.83199999999999996},"min":{"1min":0.34699999999999998,"5min":0.26900000000000002,"15min":0.23599999999999999},"max":{"1min":2.6200000000000001,"5min":2.6200000000000001,"15min":11.846},"last":0.41099999999999998}]}]},{"osd":4,"up_from":30,"seq":128849019096,"num_pgs":85,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6864316,"kb_used_data":5969844,"kb_used_omap":1,"kb_used_meta":894462,"kb_avail":86880836,"statfs":{"total":95995035648,"available":88965976064,"internally_reserved":0,"allocated":6113120256,"data_stored":6112711433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":915929432},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:00:51 2024","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.4990000000000001,"15min":1.4570000000000001},"min":{"1min":0.39100000000000001,"5min":0.223,"15min":0.223},"max":{"1min":3.6400000000000001,"5min":46.234999999999999,"15min":46.234999999999999},"last":0.438},{"interface":"front","average":{"1min":1.099,"5min":1.423,"15min":1.4510000000000001},"min":{"1min":0.36599999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":3.609,"5min":46.264000000000003,"15min":46.264000000000003},"last":0.54700000000000004}]},{"osd":1,"last update":"Tue Sep 10 19:01:30 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":2.2829999999999999,"15min":1.5660000000000001},"min":{"1min":0.24199999999999999,"5min":0.223,"15min":0.223},"max":{"1min":2.254,"5min":125.15600000000001,"15min":125.15600000000001},"last":0.20899999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":2.4129999999999998,"15min":1.502},"min":{"1min":0.30199999999999999,"5min":0.23699999999999999,"15min":0.23100000000000001},"max":{"1min":1.48,"5min":125.14700000000001,"15min":125.14700000000001},"last":0.57099999999999995}]},{"osd":2,"last update":"Tue Sep 10 19:01:30 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67400000000000004,"15min":0.68200000000000005},"min":{"1min":0.28499999999999998,"5min":0.26200000000000001,"15min":0.25900000000000001},"max":{"1min":1.5900000000000001,"5min":2.8879999999999999,"15min":5.6189999999999998},"last":0.38200000000000001},{"interface":"front","average":{"1min":1.2310000000000001,"5min":0.872,"15min":0.79100000000000004},"min":{"1min":0.34699999999999998,"5min":0.29999999999999999,"15min":0.27100000000000002},"max":{"1min":3.673,"5min":12.782999999999999,"15min":12.782999999999999},"last":0.28299999999999997}]},{"osd":3,"last update":"Tue Sep 10 19:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":3.6030000000000002,"15min":2.1230000000000002},"min":{"1min":0.377,"5min":0.28299999999999997,"15min":0.23799999999999999},"max":{"1min":3.556,"5min":111.514,"15min":111.514},"last":2.2149999999999999},{"interface":"front","average":{"1min":0.86399999999999999,"5min":2.6099999999999999,"15min":1.7130000000000001},"min":{"1min":0.27400000000000002,"5min":0.26900000000000002,"15min":0.25700000000000001},"max":{"1min":2.3730000000000002,"5min":110.914,"15min":110.914},"last":0.33000000000000002}]},{"osd":5,"last update":"Tue Sep 10 19:00:51 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.628,"15min":0.85499999999999998},"min":{"1min":0.24099999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":2.6800000000000002,"5min":9.7309999999999999,"15min":10.912000000000001},"last":1.8220000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.68200000000000005,"15min":0.79700000000000004},"min":{"1min":0.312,"5min":0.216,"15min":0.216},"max":{"1min":2.0379999999999998,"5min":10.147,"15min":10.147},"last":1.522}]},{"osd":6,"last update":"Tue Sep 10 19:01:12 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.57099999999999995,"15min":0.749},"min":{"1min":0.33600000000000002,"5min":0.221,"15min":0.221},"max":{"1min":2.5110000000000001,"5min":2.5110000000000001,"15min":8.4030000000000005},"last":1.982},{"interface":"front","average":{"1min":2.1190000000000002,"5min":0.77800000000000002,"15min":0.90900000000000003},"min":{"1min":0.32900000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":21.254000000000001,"5min":21.254000000000001,"15min":21.254000000000001},"last":0.64400000000000002}]},{"osd":7,"last update":"Tue Sep 10 19:00:46 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.63700000000000001,"15min":0.81599999999999995},"min":{"1min":0.33600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.1829999999999998,"5min":4.5250000000000004,"15min":6.4630000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.67700000000000005,"15min":0.79700000000000004},"min":{"1min":0.28199999999999997,"5min":0.251,"15min":0.251},"max":{"1min":3.589,"5min":3.589,"15min":6.3949999999999996},"last":0.311}]}]},{"osd":3,"up_from":25,"seq":107374182621,"num_pgs":83,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4039552,"kb_used_data":3150136,"kb_used_omap":1,"kb_used_meta":889406,"kb_avail":89705600,"statfs":{"total":95995035648,"available":91858534400,"internally_reserved":0,"allocated":3225739264,"data_stored":3225326219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":910752088},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:00:45 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":4.7220000000000004,"15min":2.4209999999999998},"min":{"1min":0.23200000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":2.6989999999999998,"5min":290.721,"15min":290.721},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.879,"5min":4.9530000000000003,"15min":2.661},"min":{"1min":0.17499999999999999,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":3.1880000000000002,"5min":293.59300000000002,"15min":293.59300000000002},"last":0.24399999999999999}]},{"osd":1,"last update":"Tue Sep 10 19:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":2.1419999999999999,"15min":1.573},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.16200000000000001},"max":{"1min":3.173,"5min":10.645,"15min":58.215000000000003},"last":0.317},{"interface":"front","average":{"1min":0.77400000000000002,"5min":1.8260000000000001,"15min":1.4910000000000001},"min":{"1min":0.22700000000000001,"5min":0.20000000000000001,"15min":0.188},"max":{"1min":2.3239999999999998,"5min":13.263999999999999,"15min":57.896999999999998},"last":0.27500000000000002}]},{"osd":2,"last update":"Tue Sep 10 19:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":1.286,"15min":0.95999999999999996},"min":{"1min":0.255,"5min":0.255,"15min":0.22800000000000001},"max":{"1min":2.23,"5min":5.6680000000000001,"15min":5.8220000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":1.1220000000000001,"15min":0.84299999999999997},"min":{"1min":0.254,"5min":0.22600000000000001,"15min":0.21199999999999999},"max":{"1min":1.526,"5min":5.1399999999999997,"15min":7.0979999999999999},"last":0.47399999999999998}]},{"osd":4,"last update":"Tue Sep 10 19:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.423,"15min":1.2529999999999999},"min":{"1min":0.439,"5min":0.26400000000000001,"15min":0.247},"max":{"1min":3.4249999999999998,"5min":5.0490000000000004,"15min":29.013000000000002},"last":0.78100000000000003},{"interface":"front","average":{"1min":1.103,"5min":1.4430000000000001,"15min":1.248},"min":{"1min":0.39700000000000002,"5min":0.28000000000000003,"15min":0.26800000000000002},"max":{"1min":3.3879999999999999,"5min":8.8130000000000006,"15min":28.826000000000001},"last":1.296}]},{"osd":5,"last update":"Tue Sep 10 19:00:54 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":1.4970000000000001,"15min":1.22},"min":{"1min":0.39800000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":2.4409999999999998,"5min":7.2300000000000004,"15min":7.2300000000000004},"last":1.647},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.365,"15min":1.1930000000000001},"min":{"1min":0.29099999999999998,"5min":0.27100000000000002,"15min":0.255},"max":{"1min":2.9849999999999999,"5min":5.4139999999999997,"15min":9.2509999999999994},"last":1.6579999999999999}]},{"osd":6,"last update":"Tue Sep 10 19:01:09 2024","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.504,"15min":1.2629999999999999},"min":{"1min":0.39500000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":3.431,"5min":5.4480000000000004,"15min":9.3239999999999998},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.04,"5min":1.4219999999999999,"15min":1.2210000000000001},"min":{"1min":0.34999999999999998,"5min":0.308,"15min":0.29399999999999998},"max":{"1min":2.794,"5min":5.0789999999999997,"15min":7.1559999999999997},"last":1.3440000000000001}]},{"osd":7,"last update":"Tue Sep 10 19:01:40 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":1.5289999999999999,"15min":1.294},"min":{"1min":0.39700000000000002,"5min":0.35599999999999998,"15min":0.34899999999999998},"max":{"1min":2.7290000000000001,"5min":8.7989999999999995,"15min":8.7989999999999995},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":1.46,"15min":1.3140000000000001},"min":{"1min":0.42999999999999999,"5min":0.36899999999999999,"15min":0.33100000000000002},"max":{"1min":2.266,"5min":6.6029999999999998,"15min":9.2949999999999999},"last":0.91300000000000003}]}]},{"osd":2,"up_from":0,"seq":77309411553,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834575077,"num_pgs":87,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3137052,"kb_used_data":2242068,"kb_used_omap":1,"kb_used_meta":894974,"kb_avail":90608100,"statfs":{"total":95995035648,"available":92782694400,"internally_reserved":0,"allocated":2295877632,"data_stored":2295437378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":916453720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:00:50 2024","interfaces":[{"interface":"back","average":{"1min":2.5139999999999998,"5min":2.8820000000000001,"15min":1.792},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":45.786000000000001,"5min":111.014,"15min":111.014},"last":1.617},{"interface":"front","average":{"1min":2.5600000000000001,"5min":2.9980000000000002,"15min":1.746},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":46.311999999999998,"5min":111.072,"15min":111.072},"last":1.6919999999999999}]},{"osd":2,"last update":"Tue Sep 10 19:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.873,"15min":0.752},"min":{"1min":0.26500000000000001,"5min":0.23300000000000001,"15min":0.186},"max":{"1min":1.845,"5min":4.1719999999999997,"15min":4.1719999999999997},"last":0.71499999999999997},{"interface":"front","average":{"1min":1.073,"5min":0.95599999999999996,"15min":0.80500000000000005},"min":{"1min":0.25900000000000001,"5min":0.23899999999999999,"15min":0.215},"max":{"1min":4.2560000000000002,"5min":5.6929999999999996,"15min":7.093},"last":0.92600000000000005}]},{"osd":3,"last update":"Tue Sep 10 19:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":3.718,"15min":2.0419999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.185},"max":{"1min":6.9560000000000004,"5min":92.069000000000003,"15min":92.069000000000003},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.68100000000000005,"5min":3.1930000000000001,"15min":1.9019999999999999},"min":{"1min":0.23300000000000001,"5min":0.218,"15min":0.19900000000000001},"max":{"1min":2.1110000000000002,"5min":88.658000000000001,"15min":88.658000000000001},"last":0.36899999999999999}]},{"osd":4,"last update":"Tue Sep 10 19:01:28 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":1.155,"15min":1.151},"min":{"1min":0.307,"5min":0.27800000000000002,"15min":0.26200000000000001},"max":{"1min":2.2010000000000001,"5min":9.125,"15min":15.965999999999999},"last":2.359},{"interface":"front","average":{"1min":0.70899999999999996,"5min":1.141,"15min":1.0089999999999999},"min":{"1min":0.29899999999999999,"5min":0.252,"15min":0.246},"max":{"1min":1.7549999999999999,"5min":7.0940000000000003,"15min":7.0940000000000003},"last":2.2549999999999999}]},{"osd":5,"last update":"Tue Sep 10 19:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":1.1579999999999999,"15min":1.1499999999999999},"min":{"1min":0.33400000000000002,"5min":0.30499999999999999,"15min":0.28499999999999998},"max":{"1min":3.0779999999999998,"5min":6.4690000000000003,"15min":10.334},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":1.0609999999999999,"15min":1.175},"min":{"1min":0.30499999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.486,"5min":7.0590000000000002,"15min":16.039999999999999},"last":1.9570000000000001}]},{"osd":6,"last update":"Tue Sep 10 19:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.2809999999999999,"15min":1.506},"min":{"1min":0.34200000000000003,"5min":0.33100000000000002,"15min":0.308},"max":{"1min":7.2850000000000001,"5min":19.125,"15min":38.039999999999999},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.51,"5min":1.173,"15min":1.5820000000000001},"min":{"1min":0.32600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":7.2380000000000004,"5min":7.2380000000000004,"15min":38.100999999999999},"last":0.77700000000000002}]},{"osd":7,"last update":"Tue Sep 10 19:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":1.2330000000000001,"15min":1.163},"min":{"1min":0.39800000000000002,"5min":0.33100000000000002,"15min":0.315},"max":{"1min":4.1710000000000003,"5min":19.169,"15min":19.169},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.995,"15min":1.0589999999999999},"min":{"1min":0.39600000000000002,"5min":0.36499999999999999,"15min":0.32800000000000001},"max":{"1min":2.2410000000000001,"5min":4.46,"15min":4.9029999999999996},"last":0.753}]}]},{"osd":0,"up_from":241,"seq":1035087118410,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5178172,"kb_used_data":4831924,"kb_used_omap":4,"kb_used_meta":346235,"kb_avail":88566980,"statfs":{"total":95995035648,"available":90692587520,"internally_reserved":0,"allocated":4947890176,"data_stored":4947480609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":354545158},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,4,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 19:00:48 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.8899999999999999,"15min":2.4830000000000001},"min":{"1min":0.25700000000000001,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":6.9050000000000002,"5min":34.692,"15min":34.692},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.996,"5min":1.8080000000000001,"15min":2.4820000000000002},"min":{"1min":0.28199999999999997,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":7.5960000000000001,"5min":34.588999999999999,"15min":34.588999999999999},"last":0.34599999999999997}]},{"osd":2,"last update":"Tue Sep 10 19:00:48 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.0589999999999999,"15min":0.93799999999999994},"min":{"1min":0.29999999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":6.8639999999999999,"5min":13.305,"15min":13.305},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":1.036,"15min":0.80800000000000005},"min":{"1min":0.30599999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.5600000000000001,"5min":13.693,"15min":13.693},"last":0.377}]},{"osd":3,"last update":"Tue Sep 10 19:00:48 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":1.4430000000000001,"15min":1.1279999999999999},"min":{"1min":0.39600000000000002,"5min":0.251,"15min":0.251},"max":{"1min":2.2309999999999999,"5min":21.108000000000001,"15min":21.108000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.97599999999999998,"5min":1.9830000000000001,"15min":1.7},"min":{"1min":0.35599999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":2.3159999999999998,"5min":11.629,"15min":11.629},"last":0.90300000000000002}]},{"osd":4,"last update":"Tue Sep 10 19:00:48 2024","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.0760000000000001,"15min":0.83699999999999997},"min":{"1min":0.39800000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":3.1960000000000002,"5min":13.161,"15min":13.161},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.583,"5min":1.1850000000000001,"15min":0.997},"min":{"1min":0.47399999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.4590000000000001,"5min":5.0490000000000004,"15min":5.0490000000000004},"last":4.4020000000000001}]},{"osd":5,"last update":"Tue Sep 10 19:00:48 2024","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.2330000000000001,"15min":1.0840000000000001},"min":{"1min":0.47399999999999998,"5min":0.31,"15min":0.31},"max":{"1min":3.3679999999999999,"5min":5.125,"15min":5.125},"last":1.097},{"interface":"front","average":{"1min":1.25,"5min":1.1419999999999999,"15min":0.90000000000000002},"min":{"1min":0.438,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":3.1179999999999999,"5min":13.202,"15min":13.202},"last":0.495}]},{"osd":6,"last update":"Tue Sep 10 19:00:48 2024","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.198,"15min":1.008},"min":{"1min":0.504,"5min":0.312,"15min":0.312},"max":{"1min":3.4009999999999998,"5min":5.0899999999999999,"15min":5.0899999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":1.415,"5min":1.1020000000000001,"15min":0.83199999999999996},"min":{"1min":0.41899999999999998,"5min":0.313,"15min":0.313},"max":{"1min":3.3780000000000001,"5min":13.279,"15min":13.279},"last":0.997}]},{"osd":7,"last update":"Tue Sep 10 19:00:48 2024","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.181,"15min":0.94499999999999995},"min":{"1min":0.45900000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":3.3679999999999999,"5min":13.233000000000001,"15min":13.233000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.135,"15min":1.143},"min":{"1min":0.439,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.7229999999999999,"5min":10.141999999999999,"15min":10.141999999999999},"last":0.41499999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":2668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4933750784,"data_stored":4933746712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2279342080,"data_stored":2279342080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3198287872,"data_stored":3198287872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6071980032,"data_stored":6071975960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3636789248,"data_stored":3636789248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3219132416,"data_stored":3219128344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T19:01:47.674 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:47.706 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:47.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:47.707 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:47.808 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:47.813 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:01:47.821 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:01:47.852 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:47.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:47.853 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:47.923 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:47 smithi018 ceph-mon[29336]: from='client.15681 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:01:47.924 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:47 smithi018 ceph-mon[40196]: from='client.15681 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:01:47.954 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:48.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:48.004 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:48.004 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:48.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:47 smithi103 ceph-mon[33163]: from='client.15681 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:01:48.105 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:48.144 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:48.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:48.145 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:48.246 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:48.279 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:48.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:48.280 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:48.381 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:48.423 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:48.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:48.424 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:48.525 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:48.558 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:48.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:48.559 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:48.660 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:48.692 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:48.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:48.693 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:48.794 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:48.830 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:48.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:48.831 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:48.867 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:48 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:48.868 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:48 smithi018 ceph-mon[40196]: pgmap v768: 168 pgs: 168 active+clean; 6.3 GiB data, 25 GiB used, 511 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2.01k op/s 2024-09-10T19:01:48.868 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:48 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:48.868 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:48 smithi018 ceph-mon[29336]: pgmap v768: 168 pgs: 168 active+clean; 6.3 GiB data, 25 GiB used, 511 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2.01k op/s 2024-09-10T19:01:48.933 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:48.965 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:48.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:48.966 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:49.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:48 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:49.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:48 smithi103 ceph-mon[33163]: pgmap v768: 168 pgs: 168 active+clean; 6.3 GiB data, 25 GiB used, 511 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2.01k op/s 2024-09-10T19:01:49.066 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:49.100 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:49.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:49.101 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:49.202 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:49.237 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:49.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:49.238 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:49.339 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:49.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:49.376 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:49.376 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:49.477 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:49.508 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:49.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:49.509 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:49.610 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:49.644 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:49.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:49.645 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:49.746 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:49.779 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:49.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:49.780 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:49.881 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:49.915 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:49.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:49.917 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:49.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:49 smithi018 ceph-mon[40196]: pgmap v769: 168 pgs: 168 active+clean; 6.2 GiB data, 25 GiB used, 512 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 1.58k op/s 2024-09-10T19:01:49.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:49 smithi018 ceph-mon[29336]: pgmap v769: 168 pgs: 168 active+clean; 6.2 GiB data, 25 GiB used, 512 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 1.58k op/s 2024-09-10T19:01:50.018 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:50.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:49 smithi103 ceph-mon[33163]: pgmap v769: 168 pgs: 168 active+clean; 6.2 GiB data, 25 GiB used, 512 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 1.58k op/s 2024-09-10T19:01:50.052 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:50.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:50.053 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:50.154 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:50.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:50.189 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:50.189 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:50.290 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:50.330 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:50.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:50.331 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:50.431 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:50.472 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:50.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:50.473 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:50.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:01:50 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:01:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:01:50.574 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:50.606 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:50.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:50.607 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:50.660 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T19:01:50.708 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:50.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:50.743 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:50.743 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:50.844 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:50.886 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:50.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:50.889 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:50.990 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:51.025 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:51.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:51.026 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:51.127 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:51.157 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:51.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:51.159 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:51.260 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:51.264 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:01:50 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:01:50] "GET /metrics HTTP/1.1" 200 224671 "" "Prometheus/2.43.0" 2024-09-10T19:01:51.293 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:51.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:51.294 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:51.394 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:51.431 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:51.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:51.432 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:51.533 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:51.568 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:51.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:51.569 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:51.670 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:51.711 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:51.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:51.713 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:51.813 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:51.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:51.853 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:51.853 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:51.954 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:51.991 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:51.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:51.992 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:52.093 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:52.131 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:52.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:52.133 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:52.233 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:52.266 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:52.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:52.267 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:52.368 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:52.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:52.424 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:52.424 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:52.525 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:52.538 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:52 smithi018 ceph-mon[29336]: pgmap v770: 168 pgs: 168 active+clean; 6.2 GiB data, 24 GiB used, 512 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 1.54k op/s 2024-09-10T19:01:52.538 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:52 smithi018 ceph-mon[40196]: pgmap v770: 168 pgs: 168 active+clean; 6.2 GiB data, 24 GiB used, 512 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 1.54k op/s 2024-09-10T19:01:52.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:52.553 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:52.554 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:52.654 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:52.693 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:52.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:52.694 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:52.702 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T19:01:52.702 INFO:teuthology.orchestra.run.smithi018.stdout: "size": 20, 2024-09-10T19:01:52.702 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 600, 2024-09-10T19:01:52.702 INFO:teuthology.orchestra.run.smithi018.stdout: "ops": [] 2024-09-10T19:01:52.702 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T19:01:52.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:52 smithi103 ceph-mon[33163]: pgmap v770: 168 pgs: 168 active+clean; 6.2 GiB data, 24 GiB used, 512 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 1.54k op/s 2024-09-10T19:01:52.795 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:52.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:52.829 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:52.829 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:52.930 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:52.960 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:52.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:52.961 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:53.062 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:53.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:53.097 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:53.097 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:53.198 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:53.231 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:53.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:53.232 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:53.332 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:53.369 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:53.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:53.370 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:53.471 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:53.474 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-10T19:01:53.481 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is unset 2024-09-10T19:01:53.508 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:53.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:53.509 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:53.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:53 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/7913349' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:01:53.524 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:53 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:01:53.524 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:53 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/7913349' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:01:53.524 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:53 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:01:53.610 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:53.653 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:53.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:53.654 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:53.755 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:53.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:53 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/7913349' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:01:53.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:53 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:01:53.793 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:53.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:53.795 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:53.895 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:53.937 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:53.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:53.938 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:54.039 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:54.085 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:54.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:54.086 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:54.186 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:54.228 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:54.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:54.229 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:54.329 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:54.369 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:54.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:54.370 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:54.471 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:54.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:54.508 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:54.508 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:54.527 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:54 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:01:54.527 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:54 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:01:54.527 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:54 smithi018 ceph-mon[40196]: pgmap v771: 168 pgs: 168 active+clean; 5.8 GiB data, 24 GiB used, 513 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.96k op/s 2024-09-10T19:01:54.528 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:54 smithi018 ceph-mon[40196]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:01:54.528 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:54 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T19:01:54.528 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:54 smithi018 ceph-mon[40196]: osdmap e360: 8 total, 8 up, 6 in 2024-09-10T19:01:54.528 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:54 smithi018 ceph-mon[40196]: 2.c scrub starts 2024-09-10T19:01:54.528 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:54 smithi018 ceph-mon[40196]: 4.9 scrub starts 2024-09-10T19:01:54.528 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:54 smithi018 ceph-mon[40196]: 4.9 scrub ok 2024-09-10T19:01:54.528 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:54 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/4207437484' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:01:54.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:54 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:01:54.528 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:54 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:01:54.529 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:54 smithi018 ceph-mon[29336]: pgmap v771: 168 pgs: 168 active+clean; 5.8 GiB data, 24 GiB used, 513 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.96k op/s 2024-09-10T19:01:54.529 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:54 smithi018 ceph-mon[29336]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:01:54.529 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:54 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T19:01:54.529 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:54 smithi018 ceph-mon[29336]: osdmap e360: 8 total, 8 up, 6 in 2024-09-10T19:01:54.529 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:54 smithi018 ceph-mon[29336]: 2.c scrub starts 2024-09-10T19:01:54.529 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:54 smithi018 ceph-mon[29336]: 4.9 scrub starts 2024-09-10T19:01:54.529 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:54 smithi018 ceph-mon[29336]: 4.9 scrub ok 2024-09-10T19:01:54.529 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:54 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/4207437484' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:01:54.609 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:54.645 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:54.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:54.647 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:54.747 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:54.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:54 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:01:54.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:54 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:01:54.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:54 smithi103 ceph-mon[33163]: pgmap v771: 168 pgs: 168 active+clean; 5.8 GiB data, 24 GiB used, 513 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.96k op/s 2024-09-10T19:01:54.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:54 smithi103 ceph-mon[33163]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:01:54.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:54 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T19:01:54.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:54 smithi103 ceph-mon[33163]: osdmap e360: 8 total, 8 up, 6 in 2024-09-10T19:01:54.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:54 smithi103 ceph-mon[33163]: 2.c scrub starts 2024-09-10T19:01:54.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:54 smithi103 ceph-mon[33163]: 4.9 scrub starts 2024-09-10T19:01:54.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:54 smithi103 ceph-mon[33163]: 4.9 scrub ok 2024-09-10T19:01:54.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:54 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/4207437484' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:01:54.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:54.783 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:54.783 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:54.884 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:54.917 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:54.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:54.918 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:55.018 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:55.055 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:55.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:55.056 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:55.156 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:55.186 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:55.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:55.187 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:55.288 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:55.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:55.322 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:55.322 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:55.365 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T19:01:55.365 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T19:01:55.365 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T19:01:55.365 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T19:01:55.366 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T19:01:55.366 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T19:01:55.366 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T19:01:55.366 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T19:01:55.366 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T19:01:55.366 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T19:01:55.366 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T19:01:55.366 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T19:01:55.366 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T19:01:55.367 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T19:01:55.367 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T19:01:55.367 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T19:01:55.422 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:55.457 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:55.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:55.458 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:55.559 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:55.597 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:55.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:55.598 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:55.699 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:55.736 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:55.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:55.737 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:55.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:55 smithi103 ceph-mon[33163]: 2.c scrub ok 2024-09-10T19:01:55.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:55 smithi103 ceph-mon[33163]: 5.19 scrub starts 2024-09-10T19:01:55.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:55 smithi103 ceph-mon[33163]: 5.19 scrub ok 2024-09-10T19:01:55.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:01:55.772 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 19:01:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=sqlstore.transactions t=2024-09-10T19:01:55.65437169Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-10T19:01:55.838 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:55.868 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:55.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:55.869 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:55.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:55 smithi018 ceph-mon[40196]: 2.c scrub ok 2024-09-10T19:01:55.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:55 smithi018 ceph-mon[40196]: 5.19 scrub starts 2024-09-10T19:01:55.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:55 smithi018 ceph-mon[40196]: 5.19 scrub ok 2024-09-10T19:01:55.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:01:55.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:55 smithi018 ceph-mon[29336]: 2.c scrub ok 2024-09-10T19:01:55.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:55 smithi018 ceph-mon[29336]: 5.19 scrub starts 2024-09-10T19:01:55.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:55 smithi018 ceph-mon[29336]: 5.19 scrub ok 2024-09-10T19:01:55.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:01:55.970 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:56.002 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:56.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:56.003 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:56.024 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:01:56.096 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:01:56.103 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:56.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:56.135 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:56.135 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:56.236 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:56.275 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:56.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:56.277 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:56.377 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:56.412 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:56.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:56.413 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:56.514 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:56.547 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:56.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:56.549 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:56.597 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:56 smithi018 ceph-mon[40196]: 2.7 scrub starts 2024-09-10T19:01:56.597 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:56 smithi018 ceph-mon[40196]: 2.7 scrub ok 2024-09-10T19:01:56.597 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:56 smithi018 ceph-mon[40196]: pgmap v773: 168 pgs: 168 active+clean; 5.7 GiB data, 23 GiB used, 513 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.74k op/s 2024-09-10T19:01:56.598 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:56 smithi018 ceph-mon[29336]: 2.7 scrub starts 2024-09-10T19:01:56.598 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:56 smithi018 ceph-mon[29336]: 2.7 scrub ok 2024-09-10T19:01:56.598 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:56 smithi018 ceph-mon[29336]: pgmap v773: 168 pgs: 168 active+clean; 5.7 GiB data, 23 GiB used, 513 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.74k op/s 2024-09-10T19:01:56.650 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:56.688 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:56.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:56.689 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:56.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:56 smithi103 ceph-mon[33163]: 2.7 scrub starts 2024-09-10T19:01:56.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:56 smithi103 ceph-mon[33163]: 2.7 scrub ok 2024-09-10T19:01:56.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:56 smithi103 ceph-mon[33163]: pgmap v773: 168 pgs: 168 active+clean; 5.7 GiB data, 23 GiB used, 513 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.74k op/s 2024-09-10T19:01:56.790 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:56.823 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:56.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:56.825 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:56.926 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:56.960 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:56.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:56.961 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:57.062 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:57.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:57.095 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:57.095 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:57.196 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:57.218 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-10T19:01:57.233 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:57.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:57.235 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:57.335 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:57.380 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:57.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:57.381 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:57.482 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:57.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:01:57 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:01:57.494 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:57 smithi103 ceph-mon[33163]: 5.4 scrub starts 2024-09-10T19:01:57.494 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:57 smithi103 ceph-mon[33163]: 5.4 scrub ok 2024-09-10T19:01:57.495 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:57 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:01:57.495 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:57 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:01:57.495 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:57 smithi103 ceph-mon[33163]: 5.0 scrub starts 2024-09-10T19:01:57.495 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:57 smithi103 ceph-mon[33163]: 5.0 scrub ok 2024-09-10T19:01:57.495 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:57 smithi103 ceph-mon[33163]: 1.13 scrub starts 2024-09-10T19:01:57.495 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:57 smithi103 ceph-mon[33163]: 1.13 scrub ok 2024-09-10T19:01:57.495 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:57 smithi103 ceph-mon[33163]: osdmap e361: 8 total, 8 up, 6 in 2024-09-10T19:01:57.495 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:57 smithi103 ceph-mon[33163]: 2.3 scrub starts 2024-09-10T19:01:57.495 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:57 smithi103 ceph-mon[33163]: 2.3 scrub ok 2024-09-10T19:01:57.512 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:57.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:57.513 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:57.614 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:57.647 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:57.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:57.648 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:57.752 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:57.788 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:57.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:57.789 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:57.889 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:01:57.923 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:57.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:57.924 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:01:57.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:57 smithi018 ceph-mon[40196]: 5.4 scrub starts 2024-09-10T19:01:57.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:57 smithi018 ceph-mon[40196]: 5.4 scrub ok 2024-09-10T19:01:57.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:57 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:01:57.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:57 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:01:57.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:57 smithi018 ceph-mon[40196]: 5.0 scrub starts 2024-09-10T19:01:57.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:57 smithi018 ceph-mon[40196]: 5.0 scrub ok 2024-09-10T19:01:57.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:57 smithi018 ceph-mon[40196]: 1.13 scrub starts 2024-09-10T19:01:57.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:57 smithi018 ceph-mon[40196]: 1.13 scrub ok 2024-09-10T19:01:57.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:57 smithi018 ceph-mon[40196]: osdmap e361: 8 total, 8 up, 6 in 2024-09-10T19:01:57.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:57 smithi018 ceph-mon[40196]: 2.3 scrub starts 2024-09-10T19:01:57.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:57 smithi018 ceph-mon[40196]: 2.3 scrub ok 2024-09-10T19:01:57.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:57 smithi018 ceph-mon[29336]: 5.4 scrub starts 2024-09-10T19:01:57.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:57 smithi018 ceph-mon[29336]: 5.4 scrub ok 2024-09-10T19:01:57.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:57 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:01:57.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:57 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:01:57.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:57 smithi018 ceph-mon[29336]: 5.0 scrub starts 2024-09-10T19:01:57.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:57 smithi018 ceph-mon[29336]: 5.0 scrub ok 2024-09-10T19:01:57.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:57 smithi018 ceph-mon[29336]: 1.13 scrub starts 2024-09-10T19:01:57.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:57 smithi018 ceph-mon[29336]: 1.13 scrub ok 2024-09-10T19:01:57.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:57 smithi018 ceph-mon[29336]: osdmap e361: 8 total, 8 up, 6 in 2024-09-10T19:01:57.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:57 smithi018 ceph-mon[29336]: 2.3 scrub starts 2024-09-10T19:01:57.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:57 smithi018 ceph-mon[29336]: 2.3 scrub ok 2024-09-10T19:01:58.025 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:58.059 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T19:01:58.061 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:58.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:58.062 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:58.163 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:58.194 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:58.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:58.195 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:58.296 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:01:58.335 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:58.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:58.337 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:01:58.438 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:58.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:58.477 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:58.477 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:58.577 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:58.640 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:58.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:58.641 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:58.689 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:01:58.742 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:58.766 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:58 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:58.766 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:58 smithi018 ceph-mon[40196]: pgmap v775: 168 pgs: 168 active+clean; 5.3 GiB data, 22 GiB used, 514 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.98k op/s 2024-09-10T19:01:58.766 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:58 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:58.766 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:58 smithi018 ceph-mon[29336]: pgmap v775: 168 pgs: 168 active+clean; 5.3 GiB data, 22 GiB used, 514 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.98k op/s 2024-09-10T19:01:58.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:58 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:58.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:58 smithi103 ceph-mon[33163]: pgmap v775: 168 pgs: 168 active+clean; 5.3 GiB data, 22 GiB used, 514 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.98k op/s 2024-09-10T19:01:58.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:58.773 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:58.773 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:58.874 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:01:58.915 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:58.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:58.916 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:01:59.017 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:59.054 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:59.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:59.055 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:01:59.156 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:59.178 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:59.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:59.181 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:59.282 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:59.310 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:59.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:59.311 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:59.412 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:01:59.446 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:59.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:59.447 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:01:59.548 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:01:59.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:59.589 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:59.589 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:01:59.690 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:01:59.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:59.727 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:01:59.727 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:01:59.732 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:59 smithi018 ceph-mon[40196]: 2.17 scrub starts 2024-09-10T19:01:59.732 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:59 smithi018 ceph-mon[40196]: 2.17 scrub ok 2024-09-10T19:01:59.732 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:59 smithi018 ceph-mon[40196]: 5.16 scrub starts 2024-09-10T19:01:59.732 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:59 smithi018 ceph-mon[40196]: 5.16 scrub ok 2024-09-10T19:01:59.732 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:59 smithi018 ceph-mon[40196]: 6.9 scrub starts 2024-09-10T19:01:59.732 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:01:59 smithi018 ceph-mon[40196]: 6.9 scrub ok 2024-09-10T19:01:59.733 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:59 smithi018 ceph-mon[29336]: 2.17 scrub starts 2024-09-10T19:01:59.733 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:59 smithi018 ceph-mon[29336]: 2.17 scrub ok 2024-09-10T19:01:59.733 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:59 smithi018 ceph-mon[29336]: 5.16 scrub starts 2024-09-10T19:01:59.733 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:59 smithi018 ceph-mon[29336]: 5.16 scrub ok 2024-09-10T19:01:59.733 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:59 smithi018 ceph-mon[29336]: 6.9 scrub starts 2024-09-10T19:01:59.733 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:01:59 smithi018 ceph-mon[29336]: 6.9 scrub ok 2024-09-10T19:01:59.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:59 smithi103 ceph-mon[33163]: 2.17 scrub starts 2024-09-10T19:01:59.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:59 smithi103 ceph-mon[33163]: 2.17 scrub ok 2024-09-10T19:01:59.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:59 smithi103 ceph-mon[33163]: 5.16 scrub starts 2024-09-10T19:01:59.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:59 smithi103 ceph-mon[33163]: 5.16 scrub ok 2024-09-10T19:01:59.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:59 smithi103 ceph-mon[33163]: 6.9 scrub starts 2024-09-10T19:01:59.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:01:59 smithi103 ceph-mon[33163]: 6.9 scrub ok 2024-09-10T19:01:59.828 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:01:59.837 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:01:59.861 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:59.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:59.862 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:01:59.889 INFO:teuthology.orchestra.run.smithi018.stdout:ERROR: (22) Invalid argument 2024-09-10T19:01:59.889 INFO:teuthology.orchestra.run.smithi018.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T19:01:59.962 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:01:59.969 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T19:01:59.972 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-09-10T19:01:59.988 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:01:59.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:01:59.989 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:02:00.089 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:00.124 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:00.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:00.125 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:00.227 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:00.264 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:00.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:00.265 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:00.366 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:00.396 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:00.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:00.398 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:00.498 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:00.511 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:02:00 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:02:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:02:00.530 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:00.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:00.531 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:00.632 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:00.664 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:00.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:00.665 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:00.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:00 smithi018 ceph-mon[40196]: pgmap v776: 168 pgs: 168 active+clean; 5.2 GiB data, 22 GiB used, 515 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2.03k op/s 2024-09-10T19:02:00.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:00 smithi018 ceph-mon[40196]: 5.d scrub starts 2024-09-10T19:02:00.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:00 smithi018 ceph-mon[40196]: 5.d scrub ok 2024-09-10T19:02:00.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:00 smithi018 ceph-mon[29336]: pgmap v776: 168 pgs: 168 active+clean; 5.2 GiB data, 22 GiB used, 515 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2.03k op/s 2024-09-10T19:02:00.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:00 smithi018 ceph-mon[29336]: 5.d scrub starts 2024-09-10T19:02:00.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:00 smithi018 ceph-mon[29336]: 5.d scrub ok 2024-09-10T19:02:00.766 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:02:00.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:00 smithi103 ceph-mon[33163]: pgmap v776: 168 pgs: 168 active+clean; 5.2 GiB data, 22 GiB used, 515 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2.03k op/s 2024-09-10T19:02:00.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:00 smithi103 ceph-mon[33163]: 5.d scrub starts 2024-09-10T19:02:00.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:00 smithi103 ceph-mon[33163]: 5.d scrub ok 2024-09-10T19:02:00.800 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:00.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:00.801 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:02:00.902 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:00.936 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:00.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:00.937 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:01.038 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:02:01.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:01.076 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:01.076 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:02:01.177 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:02:01.205 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:01.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:01.206 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:02:01.307 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:02:01.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:01.344 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:01.344 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:02:01.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:02:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:02:00] "GET /metrics HTTP/1.1" 200 224626 "" "Prometheus/2.43.0" 2024-09-10T19:02:01.444 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:01.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:01.486 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:01.486 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:01.587 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:01.627 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:01.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:01.628 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:01.729 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:01.763 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:01.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:01.764 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:01.865 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:01.897 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:01.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:01.898 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:01.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:01 smithi018 ceph-mon[40196]: from='client.15696 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:02:01.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:01 smithi018 ceph-mon[40196]: 6.11 scrub starts 2024-09-10T19:02:01.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:01 smithi018 ceph-mon[40196]: 6.11 scrub ok 2024-09-10T19:02:01.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:01 smithi018 ceph-mon[40196]: 1.d scrub starts 2024-09-10T19:02:01.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:01 smithi018 ceph-mon[40196]: 1.d scrub ok 2024-09-10T19:02:01.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:01 smithi018 ceph-mon[29336]: from='client.15696 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:02:01.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:01 smithi018 ceph-mon[29336]: 6.11 scrub starts 2024-09-10T19:02:01.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:01 smithi018 ceph-mon[29336]: 6.11 scrub ok 2024-09-10T19:02:01.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:01 smithi018 ceph-mon[29336]: 1.d scrub starts 2024-09-10T19:02:01.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:01 smithi018 ceph-mon[29336]: 1.d scrub ok 2024-09-10T19:02:01.999 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:02.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:01 smithi103 ceph-mon[33163]: from='client.15696 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:02:02.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:01 smithi103 ceph-mon[33163]: 6.11 scrub starts 2024-09-10T19:02:02.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:01 smithi103 ceph-mon[33163]: 6.11 scrub ok 2024-09-10T19:02:02.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:01 smithi103 ceph-mon[33163]: 1.d scrub starts 2024-09-10T19:02:02.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:01 smithi103 ceph-mon[33163]: 1.d scrub ok 2024-09-10T19:02:02.026 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:02.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:02.027 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:02.128 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:02.161 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:02.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:02.162 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:02.263 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:02.296 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:02.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:02.297 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:02.398 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:02.434 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:02.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:02.435 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:02.536 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:02.558 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:02.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:02.560 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:02.660 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:02.695 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:02.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:02.696 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:02.796 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:02.833 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:02.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:02.834 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:02.886 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:02 smithi018 ceph-mon[40196]: 5.5 scrub starts 2024-09-10T19:02:02.886 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:02 smithi018 ceph-mon[40196]: 5.5 scrub ok 2024-09-10T19:02:02.886 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:02 smithi018 ceph-mon[40196]: 2.a scrub starts 2024-09-10T19:02:02.886 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:02 smithi018 ceph-mon[40196]: 2.a scrub ok 2024-09-10T19:02:02.886 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:02 smithi018 ceph-mon[40196]: pgmap v777: 168 pgs: 168 active+clean; 5.1 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 1.39k op/s 2024-09-10T19:02:02.886 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:02 smithi018 ceph-mon[29336]: 5.5 scrub starts 2024-09-10T19:02:02.887 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:02 smithi018 ceph-mon[29336]: 5.5 scrub ok 2024-09-10T19:02:02.887 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:02 smithi018 ceph-mon[29336]: 2.a scrub starts 2024-09-10T19:02:02.887 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:02 smithi018 ceph-mon[29336]: 2.a scrub ok 2024-09-10T19:02:02.887 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:02 smithi018 ceph-mon[29336]: pgmap v777: 168 pgs: 168 active+clean; 5.1 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 1.39k op/s 2024-09-10T19:02:02.935 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:02.970 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:02.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:02.971 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:03.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:02 smithi103 ceph-mon[33163]: 5.5 scrub starts 2024-09-10T19:02:03.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:02 smithi103 ceph-mon[33163]: 5.5 scrub ok 2024-09-10T19:02:03.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:02 smithi103 ceph-mon[33163]: 2.a scrub starts 2024-09-10T19:02:03.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:02 smithi103 ceph-mon[33163]: 2.a scrub ok 2024-09-10T19:02:03.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:02 smithi103 ceph-mon[33163]: pgmap v777: 168 pgs: 168 active+clean; 5.1 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 1.39k op/s 2024-09-10T19:02:03.072 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:03.112 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:03.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:03.113 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:03.214 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:03.235 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:03.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:03.237 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:03.337 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:03.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:03.369 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:03.369 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:03.470 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:03.507 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:03.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:03.508 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:03.608 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:03.645 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:03.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:03.646 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:03.747 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:03.784 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:03.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:03.785 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:03.886 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:03.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:03.926 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:03.926 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:03.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:03 smithi018 ceph-mon[40196]: 4.1c scrub starts 2024-09-10T19:02:03.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:03 smithi018 ceph-mon[40196]: 4.1c scrub ok 2024-09-10T19:02:03.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:03 smithi018 ceph-mon[40196]: 4.16 scrub starts 2024-09-10T19:02:03.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:03 smithi018 ceph-mon[40196]: 4.16 scrub ok 2024-09-10T19:02:03.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:03 smithi018 ceph-mon[40196]: 1.14 scrub starts 2024-09-10T19:02:03.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:03 smithi018 ceph-mon[40196]: 1.14 scrub ok 2024-09-10T19:02:03.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:03 smithi018 ceph-mon[29336]: 4.1c scrub starts 2024-09-10T19:02:03.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:03 smithi018 ceph-mon[29336]: 4.1c scrub ok 2024-09-10T19:02:03.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:03 smithi018 ceph-mon[29336]: 4.16 scrub starts 2024-09-10T19:02:03.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:03 smithi018 ceph-mon[29336]: 4.16 scrub ok 2024-09-10T19:02:03.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:03 smithi018 ceph-mon[29336]: 1.14 scrub starts 2024-09-10T19:02:03.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:03 smithi018 ceph-mon[29336]: 1.14 scrub ok 2024-09-10T19:02:04.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:03 smithi103 ceph-mon[33163]: 4.1c scrub starts 2024-09-10T19:02:04.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:03 smithi103 ceph-mon[33163]: 4.1c scrub ok 2024-09-10T19:02:04.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:03 smithi103 ceph-mon[33163]: 4.16 scrub starts 2024-09-10T19:02:04.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:03 smithi103 ceph-mon[33163]: 4.16 scrub ok 2024-09-10T19:02:04.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:03 smithi103 ceph-mon[33163]: 1.14 scrub starts 2024-09-10T19:02:04.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:03 smithi103 ceph-mon[33163]: 1.14 scrub ok 2024-09-10T19:02:04.027 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:04.056 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:04.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:04.057 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:04.158 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:04.196 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:04.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:04.197 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:04.298 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:02:04.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:04.330 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:04.330 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:02:04.431 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:02:04.465 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:04.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:04.466 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:02:04.567 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:04.601 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:04.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:04.603 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:04.704 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:04.748 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:04.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:04.749 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:04.850 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:02:04.884 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:04.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:04.885 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:02:04.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:04 smithi018 ceph-mon[40196]: 5.1d scrub starts 2024-09-10T19:02:04.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:04 smithi018 ceph-mon[40196]: 5.1d scrub ok 2024-09-10T19:02:04.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:04 smithi018 ceph-mon[40196]: pgmap v778: 168 pgs: 168 active+clean; 4.8 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.69k op/s 2024-09-10T19:02:04.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:04 smithi018 ceph-mon[40196]: 2.13 scrub starts 2024-09-10T19:02:04.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:04 smithi018 ceph-mon[40196]: 2.13 scrub ok 2024-09-10T19:02:04.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:04 smithi018 ceph-mon[29336]: 5.1d scrub starts 2024-09-10T19:02:04.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:04 smithi018 ceph-mon[29336]: 5.1d scrub ok 2024-09-10T19:02:04.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:04 smithi018 ceph-mon[29336]: pgmap v778: 168 pgs: 168 active+clean; 4.8 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.69k op/s 2024-09-10T19:02:04.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:04 smithi018 ceph-mon[29336]: 2.13 scrub starts 2024-09-10T19:02:04.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:04 smithi018 ceph-mon[29336]: 2.13 scrub ok 2024-09-10T19:02:04.986 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:05.020 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:05.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:05.021 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:05.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:04 smithi103 ceph-mon[33163]: 5.1d scrub starts 2024-09-10T19:02:05.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:04 smithi103 ceph-mon[33163]: 5.1d scrub ok 2024-09-10T19:02:05.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:04 smithi103 ceph-mon[33163]: pgmap v778: 168 pgs: 168 active+clean; 4.8 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.69k op/s 2024-09-10T19:02:05.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:04 smithi103 ceph-mon[33163]: 2.13 scrub starts 2024-09-10T19:02:05.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:04 smithi103 ceph-mon[33163]: 2.13 scrub ok 2024-09-10T19:02:05.122 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:02:05.160 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:05.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:05.161 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:02:05.262 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:05.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:05.297 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:05.298 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:05.399 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:05.438 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:05.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:05.439 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:05.540 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:05.577 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:05.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:05.579 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:05.679 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:05.714 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:05.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:05.715 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:05.816 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:05.851 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:05.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:05.853 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:05.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:05 smithi018 ceph-mon[40196]: 4.1e scrub starts 2024-09-10T19:02:05.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:05 smithi018 ceph-mon[40196]: 4.1e scrub ok 2024-09-10T19:02:05.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:05 smithi018 ceph-mon[29336]: 4.1e scrub starts 2024-09-10T19:02:05.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:05 smithi018 ceph-mon[29336]: 4.1e scrub ok 2024-09-10T19:02:05.954 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:05.986 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:05.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:05.987 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:06.022 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":776,"stamp":"2024-09-10T19:01:59.400157+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5551054550,"num_objects":84890,"num_object_clones":0,"num_object_copies":254670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84890,"num_whiteouts":0,"num_read":2134,"num_read_kb":1889,"num_write":193337,"num_write_kb":8874013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22243,"num_bytes_recovered":1440392042,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":59900,"ondisk_log_size":59900,"up":504,"acting":504,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":507,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":562470912,"kb_used":22648004,"kb_used_data":17844504,"kb_used_omap":12,"kb_used_meta":4802867,"kb_avail":539822908,"statfs":{"total":575970213888,"available":552778657792,"internally_reserved":0,"allocated":18272772096,"data_stored":18270334614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13122,"internal_metadata":4918135998},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,5,3,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1063452672,"num_objects":-16227,"num_object_clones":0,"num_object_copies":-48681,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-16227,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":16227,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003778"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":364,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.588955+0000","last_change":"2024-09-10T18:57:02.462265+0000","last_active":"2024-09-10T19:01:34.588955+0000","last_peered":"2024-09-10T19:01:34.588955+0000","last_clean":"2024-09-10T19:01:34.588955+0000","last_became_active":"2024-09-10T18:57:02.460451+0000","last_became_peered":"2024-09-10T18:57:02.460451+0000","last_unstale":"2024-09-10T19:01:34.588955+0000","last_undegraded":"2024-09-10T19:01:34.588955+0000","last_fullsized":"2024-09-10T19:01:34.588955+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:14.875872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:14.875872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037478699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":361,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.484022+0000","last_change":"2024-09-10T18:59:59.848814+0000","last_active":"2024-09-10T19:01:53.484022+0000","last_peered":"2024-09-10T19:01:53.484022+0000","last_clean":"2024-09-10T19:01:53.484022+0000","last_became_active":"2024-09-10T18:57:02.440648+0000","last_became_peered":"2024-09-10T18:57:02.440648+0000","last_unstale":"2024-09-10T19:01:53.484022+0000","last_undegraded":"2024-09-10T19:01:53.484022+0000","last_fullsized":"2024-09-10T19:01:53.484022+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:59.848700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:59.848700+0000","last_clean_scrub_stamp":"2024-09-10T18:59:59.848700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047956400000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"232'27","reported_seq":449,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.589103+0000","last_change":"2024-09-10T18:56:02.821637+0000","last_active":"2024-09-10T19:01:34.589103+0000","last_peered":"2024-09-10T19:01:34.589103+0000","last_clean":"2024-09-10T19:01:34.589103+0000","last_became_active":"2024-09-10T18:55:40.394717+0000","last_became_peered":"2024-09-10T18:55:40.394717+0000","last_unstale":"2024-09-10T19:01:34.589103+0000","last_undegraded":"2024-09-10T19:01:34.589103+0000","last_fullsized":"2024-09-10T19:01:34.589103+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'27","last_scrub_stamp":"2024-09-10T18:56:02.821551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:02.821551+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0083110159999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":365,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.583297+0000","last_change":"2024-09-10T19:00:08.841172+0000","last_active":"2024-09-10T19:01:34.583297+0000","last_peered":"2024-09-10T19:01:34.583297+0000","last_clean":"2024-09-10T19:01:34.583297+0000","last_became_active":"2024-09-10T18:45:22.065619+0000","last_became_peered":"2024-09-10T18:45:22.065619+0000","last_unstale":"2024-09-10T19:01:34.583297+0000","last_undegraded":"2024-09-10T19:01:34.583297+0000","last_fullsized":"2024-09-10T19:01:34.583297+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:08.841125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.057932+0000","last_clean_scrub_stamp":"2024-09-10T19:00:08.841125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001059857,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":427,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.592164+0000","last_change":"2024-09-10T19:00:42.519131+0000","last_active":"2024-09-10T19:01:34.592164+0000","last_peered":"2024-09-10T19:01:34.592164+0000","last_clean":"2024-09-10T19:01:34.592164+0000","last_became_active":"2024-09-10T19:00:42.518824+0000","last_became_peered":"2024-09-10T19:00:42.518824+0000","last_unstale":"2024-09-10T19:01:34.592164+0000","last_undegraded":"2024-09-10T19:01:34.592164+0000","last_fullsized":"2024-09-10T19:01:34.592164+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:23.885562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:23.885562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0031033800000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"232'19","reported_seq":386,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.582652+0000","last_change":"2024-09-10T18:59:09.642743+0000","last_active":"2024-09-10T19:01:34.582652+0000","last_peered":"2024-09-10T19:01:34.582652+0000","last_clean":"2024-09-10T19:01:34.582652+0000","last_became_active":"2024-09-10T18:45:20.061266+0000","last_became_peered":"2024-09-10T18:45:20.061266+0000","last_unstale":"2024-09-10T19:01:34.582652+0000","last_undegraded":"2024-09-10T19:01:34.582652+0000","last_fullsized":"2024-09-10T19:01:34.582652+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T18:59:09.642642+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_clean_scrub_stamp":"2024-09-10T18:59:09.642642+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0095920940000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":112,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:54.384808+0000","last_change":"2024-09-10T19:01:54.384697+0000","last_active":"2024-09-10T19:01:54.384808+0000","last_peered":"2024-09-10T19:01:54.384808+0000","last_clean":"2024-09-10T19:01:54.384808+0000","last_became_active":"2024-09-10T18:57:02.446008+0000","last_became_peered":"2024-09-10T18:57:02.446008+0000","last_unstale":"2024-09-10T19:01:54.384808+0000","last_undegraded":"2024-09-10T19:01:54.384808+0000","last_fullsized":"2024-09-10T19:01:54.384808+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:01:54.384604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:01.135988+0000","last_clean_scrub_stamp":"2024-09-10T19:01:54.384604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:09.487091+0000","scrub_duration":0.010950134,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":353,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.582606+0000","last_change":"2024-09-10T18:58:51.682046+0000","last_active":"2024-09-10T19:01:34.582606+0000","last_peered":"2024-09-10T19:01:34.582606+0000","last_clean":"2024-09-10T19:01:34.582606+0000","last_became_active":"2024-09-10T18:45:24.081002+0000","last_became_peered":"2024-09-10T18:45:24.081002+0000","last_unstale":"2024-09-10T19:01:34.582606+0000","last_undegraded":"2024-09-10T19:01:34.582606+0000","last_fullsized":"2024-09-10T19:01:34.582606+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:51.681994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:58:51.681994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051776249999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":359,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.480363+0000","last_change":"2024-09-10T18:59:21.502068+0000","last_active":"2024-09-10T19:01:53.480363+0000","last_peered":"2024-09-10T19:01:53.480363+0000","last_clean":"2024-09-10T19:01:53.480363+0000","last_became_active":"2024-09-10T18:57:02.452259+0000","last_became_peered":"2024-09-10T18:57:02.452259+0000","last_unstale":"2024-09-10T19:01:53.480363+0000","last_undegraded":"2024-09-10T19:01:53.480363+0000","last_fullsized":"2024-09-10T19:01:53.480363+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:21.501984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:21.501984+0000","last_clean_scrub_stamp":"2024-09-10T18:59:21.501984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014628558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1b","version":"232'9","reported_seq":372,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.582672+0000","last_change":"2024-09-10T18:59:11.622597+0000","last_active":"2024-09-10T19:01:34.582672+0000","last_peered":"2024-09-10T19:01:34.582672+0000","last_clean":"2024-09-10T19:01:34.582672+0000","last_became_active":"2024-09-10T18:45:20.057567+0000","last_became_peered":"2024-09-10T18:45:20.057567+0000","last_unstale":"2024-09-10T19:01:34.582672+0000","last_undegraded":"2024-09-10T19:01:34.582672+0000","last_fullsized":"2024-09-10T19:01:34.582672+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'9","last_scrub_stamp":"2024-09-10T18:59:11.622509+0000","last_deep_scrub":"163'7","last_deep_scrub_stamp":"2024-09-10T18:51:54.261879+0000","last_clean_scrub_stamp":"2024-09-10T18:59:11.622509+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056120609999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":124,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.582535+0000","last_change":"2024-09-10T19:00:40.947163+0000","last_active":"2024-09-10T19:01:34.582535+0000","last_peered":"2024-09-10T19:01:34.582535+0000","last_clean":"2024-09-10T19:01:34.582535+0000","last_became_active":"2024-09-10T18:58:58.503924+0000","last_became_peered":"2024-09-10T18:58:58.503924+0000","last_unstale":"2024-09-10T19:01:34.582535+0000","last_undegraded":"2024-09-10T19:01:34.582535+0000","last_fullsized":"2024-09-10T19:01:34.582535+0000","mapping_epoch":303,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":304,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:40.947066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:36.509971+0000","last_clean_scrub_stamp":"2024-09-10T19:00:40.947066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:02:09.283651+0000","scrub_duration":0.012293715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":32,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.477921+0000","last_change":"2024-09-10T19:00:42.516918+0000","last_active":"2024-09-10T19:01:53.477921+0000","last_peered":"2024-09-10T19:01:53.477921+0000","last_clean":"2024-09-10T19:01:53.477921+0000","last_became_active":"2024-09-10T19:00:42.515840+0000","last_became_peered":"2024-09-10T19:00:42.515840+0000","last_unstale":"2024-09-10T19:01:53.477921+0000","last_undegraded":"2024-09-10T19:01:53.477921+0000","last_fullsized":"2024-09-10T19:01:53.477921+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:48.230459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:59:48.230459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":165,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.477142+0000","last_change":"2024-09-10T18:58:49.506212+0000","last_active":"2024-09-10T19:01:53.477142+0000","last_peered":"2024-09-10T19:01:53.477142+0000","last_clean":"2024-09-10T19:01:53.477142+0000","last_became_active":"2024-09-10T18:57:02.440497+0000","last_became_peered":"2024-09-10T18:57:02.440497+0000","last_unstale":"2024-09-10T19:01:53.477142+0000","last_undegraded":"2024-09-10T19:01:53.477142+0000","last_fullsized":"2024-09-10T19:01:53.477142+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:49.506160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:58:49.506160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.018244065,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"232'15","reported_seq":411,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.582869+0000","last_change":"2024-09-10T19:01:08.981911+0000","last_active":"2024-09-10T19:01:34.582869+0000","last_peered":"2024-09-10T19:01:34.582869+0000","last_clean":"2024-09-10T19:01:34.582869+0000","last_became_active":"2024-09-10T18:55:40.391571+0000","last_became_peered":"2024-09-10T18:55:40.391571+0000","last_unstale":"2024-09-10T19:01:34.582869+0000","last_undegraded":"2024-09-10T19:01:34.582869+0000","last_fullsized":"2024-09-10T19:01:34.582869+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T18:56:26.480105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:26.480105+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027886920000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":395,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.476546+0000","last_change":"2024-09-10T18:59:22.643043+0000","last_active":"2024-09-10T19:01:53.476546+0000","last_peered":"2024-09-10T19:01:53.476546+0000","last_clean":"2024-09-10T19:01:53.476546+0000","last_became_active":"2024-09-10T18:57:02.431585+0000","last_became_peered":"2024-09-10T18:57:02.431585+0000","last_unstale":"2024-09-10T19:01:53.476546+0000","last_undegraded":"2024-09-10T19:01:53.476546+0000","last_fullsized":"2024-09-10T19:01:53.476546+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:22.642961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:59:22.642961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.018189153999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":50,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.484460+0000","last_change":"2024-09-10T18:59:48.738143+0000","last_active":"2024-09-10T19:01:53.484460+0000","last_peered":"2024-09-10T19:01:53.484460+0000","last_clean":"2024-09-10T19:01:53.484460+0000","last_became_active":"2024-09-10T18:59:48.737788+0000","last_became_peered":"2024-09-10T18:59:48.737788+0000","last_unstale":"2024-09-10T19:01:53.484460+0000","last_undegraded":"2024-09-10T19:01:53.484460+0000","last_fullsized":"2024-09-10T19:01:53.484460+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:51.561010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:59.425718+0000","last_clean_scrub_stamp":"2024-09-10T18:58:51.561010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3","version":"361'11855","reported_seq":11909,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-09-10T19:01:56.579076+0000","last_change":"2024-09-10T18:59:43.007892+0000","last_active":"2024-09-10T19:01:56.579076+0000","last_peered":"2024-09-10T19:01:56.579076+0000","last_clean":"2024-09-10T19:01:56.579076+0000","last_became_active":"2024-09-10T18:59:43.007585+0000","last_became_peered":"2024-09-10T18:59:43.007585+0000","last_unstale":"2024-09-10T19:01:56.579076+0000","last_undegraded":"2024-09-10T19:01:56.579076+0000","last_fullsized":"2024-09-10T19:01:56.579076+0000","mapping_epoch":315,"log_start":"352'8199","ondisk_log_start":"352'8199","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3656,"log_dups_size":0,"ondisk_log_size":3656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":346488832,"num_objects":5287,"num_object_clones":0,"num_object_copies":15861,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11855,"num_write_kb":548544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.8","version":"70'81","reported_seq":36,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.476817+0000","last_change":"2024-09-10T19:00:42.527220+0000","last_active":"2024-09-10T19:01:53.476817+0000","last_peered":"2024-09-10T19:01:53.476817+0000","last_clean":"2024-09-10T19:01:53.476817+0000","last_became_active":"2024-09-10T19:00:42.517072+0000","last_became_peered":"2024-09-10T19:00:42.517072+0000","last_unstale":"2024-09-10T19:01:53.476817+0000","last_undegraded":"2024-09-10T19:01:53.476817+0000","last_fullsized":"2024-09-10T19:01:53.476817+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:51.167302+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:59:51.167302+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015031440000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":262688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"232'39","reported_seq":452,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.585595+0000","last_change":"2024-09-10T18:59:47.745859+0000","last_active":"2024-09-10T19:01:34.585595+0000","last_peered":"2024-09-10T19:01:34.585595+0000","last_clean":"2024-09-10T19:01:34.585595+0000","last_became_active":"2024-09-10T18:47:36.409300+0000","last_became_peered":"2024-09-10T18:47:36.409300+0000","last_unstale":"2024-09-10T19:01:34.585595+0000","last_undegraded":"2024-09-10T19:01:34.585595+0000","last_fullsized":"2024-09-10T19:01:34.585595+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'39","last_scrub_stamp":"2024-09-10T18:59:47.745766+0000","last_deep_scrub":"78'25","last_deep_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_clean_scrub_stamp":"2024-09-10T18:59:47.745766+0000","objects_scrubbed":8,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011279000000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":111,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.582894+0000","last_change":"2024-09-10T19:00:09.867278+0000","last_active":"2024-09-10T19:01:34.582894+0000","last_peered":"2024-09-10T19:01:34.582894+0000","last_clean":"2024-09-10T19:01:34.582894+0000","last_became_active":"2024-09-10T18:57:02.437260+0000","last_became_peered":"2024-09-10T18:57:02.437260+0000","last_unstale":"2024-09-10T19:01:34.582894+0000","last_undegraded":"2024-09-10T19:01:34.582894+0000","last_fullsized":"2024-09-10T19:01:34.582894+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:09.867183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:00:09.867183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026871460000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":375,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.483406+0000","last_change":"2024-09-10T19:00:09.470422+0000","last_active":"2024-09-10T19:01:53.483406+0000","last_peered":"2024-09-10T19:01:53.483406+0000","last_clean":"2024-09-10T19:01:53.483406+0000","last_became_active":"2024-09-10T18:55:40.392449+0000","last_became_peered":"2024-09-10T18:55:40.392449+0000","last_unstale":"2024-09-10T19:01:53.483406+0000","last_undegraded":"2024-09-10T19:01:53.483406+0000","last_fullsized":"2024-09-10T19:01:53.483406+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:09.470293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:04.512425+0000","last_clean_scrub_stamp":"2024-09-10T19:00:09.470293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0073759949999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":273,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.484176+0000","last_change":"2024-09-10T18:58:49.524134+0000","last_active":"2024-09-10T19:01:53.484176+0000","last_peered":"2024-09-10T19:01:53.484176+0000","last_clean":"2024-09-10T19:01:53.484176+0000","last_became_active":"2024-09-10T18:55:40.388154+0000","last_became_peered":"2024-09-10T18:55:40.388154+0000","last_unstale":"2024-09-10T19:01:53.484176+0000","last_undegraded":"2024-09-10T19:01:53.484176+0000","last_fullsized":"2024-09-10T19:01:53.484176+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:49.524015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:58:49.524015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005180479,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2","version":"361'12228","reported_seq":12282,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-09-10T19:01:56.826126+0000","last_change":"2024-09-10T18:59:42.709056+0000","last_active":"2024-09-10T19:01:56.826126+0000","last_peered":"2024-09-10T19:01:56.826126+0000","last_clean":"2024-09-10T19:01:56.826126+0000","last_became_active":"2024-09-10T18:59:42.708732+0000","last_became_peered":"2024-09-10T18:59:42.708732+0000","last_unstale":"2024-09-10T19:01:56.826126+0000","last_undegraded":"2024-09-10T19:01:56.826126+0000","last_fullsized":"2024-09-10T19:01:56.826126+0000","mapping_epoch":315,"log_start":"352'8700","ondisk_log_start":"352'8700","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3528,"log_dups_size":0,"ondisk_log_size":3528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":353239040,"num_objects":5390,"num_object_clones":0,"num_object_copies":16170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12228,"num_write_kb":563776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.9","version":"70'81","reported_seq":268,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.589189+0000","last_change":"2024-09-10T18:59:56.164487+0000","last_active":"2024-09-10T19:01:34.589189+0000","last_peered":"2024-09-10T19:01:34.589189+0000","last_clean":"2024-09-10T19:01:34.589189+0000","last_became_active":"2024-09-10T18:57:02.463750+0000","last_became_peered":"2024-09-10T18:57:02.463750+0000","last_unstale":"2024-09-10T19:01:34.589189+0000","last_undegraded":"2024-09-10T19:01:34.589189+0000","last_fullsized":"2024-09-10T19:01:34.589189+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:56.164384+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:59:56.164384+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041056720000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"232'25","reported_seq":392,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.583890+0000","last_change":"2024-09-10T18:56:32.437605+0000","last_active":"2024-09-10T19:01:34.583890+0000","last_peered":"2024-09-10T19:01:34.583890+0000","last_clean":"2024-09-10T19:01:34.583890+0000","last_became_active":"2024-09-10T18:45:20.053428+0000","last_became_peered":"2024-09-10T18:45:20.053428+0000","last_unstale":"2024-09-10T19:01:34.583890+0000","last_undegraded":"2024-09-10T19:01:34.583890+0000","last_fullsized":"2024-09-10T19:01:34.583890+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'25","last_scrub_stamp":"2024-09-10T18:56:32.437503+0000","last_deep_scrub":"116'15","last_deep_scrub_stamp":"2024-09-10T18:49:12.374527+0000","last_clean_scrub_stamp":"2024-09-10T18:56:32.437503+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039673679999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":375,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.481684+0000","last_change":"2024-09-10T19:00:51.441092+0000","last_active":"2024-09-10T19:01:53.481684+0000","last_peered":"2024-09-10T19:01:53.481684+0000","last_clean":"2024-09-10T19:01:53.481684+0000","last_became_active":"2024-09-10T18:57:02.440442+0000","last_became_peered":"2024-09-10T18:57:02.440442+0000","last_unstale":"2024-09-10T19:01:53.481684+0000","last_undegraded":"2024-09-10T19:01:53.481684+0000","last_fullsized":"2024-09-10T19:01:53.481684+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:29.802937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:29.802937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044818080000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"273'13","reported_seq":422,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:56.274452+0000","last_change":"2024-09-10T18:57:02.463853+0000","last_active":"2024-09-10T19:01:56.274452+0000","last_peered":"2024-09-10T19:01:56.274452+0000","last_clean":"2024-09-10T19:01:56.274452+0000","last_became_active":"2024-09-10T18:57:02.448258+0000","last_became_peered":"2024-09-10T18:57:02.448258+0000","last_unstale":"2024-09-10T19:01:56.274452+0000","last_undegraded":"2024-09-10T19:01:56.274452+0000","last_fullsized":"2024-09-10T19:01:56.274452+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"242'12","last_scrub_stamp":"2024-09-10T18:56:27.995682+0000","last_deep_scrub":"202'10","last_deep_scrub_stamp":"2024-09-10T18:53:45.547686+0000","last_clean_scrub_stamp":"2024-09-10T18:56:27.995682+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025134200000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":403,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.584285+0000","last_change":"2024-09-10T19:00:42.515842+0000","last_active":"2024-09-10T19:01:34.584285+0000","last_peered":"2024-09-10T19:01:34.584285+0000","last_clean":"2024-09-10T19:01:34.584285+0000","last_became_active":"2024-09-10T19:00:42.515523+0000","last_became_peered":"2024-09-10T19:00:42.515523+0000","last_unstale":"2024-09-10T19:01:34.584285+0000","last_undegraded":"2024-09-10T19:01:34.584285+0000","last_fullsized":"2024-09-10T19:01:34.584285+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:07.803916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:32.043666+0000","last_clean_scrub_stamp":"2024-09-10T19:00:07.803916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0011072429999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1","version":"360'12004","reported_seq":12057,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:56.363184+0000","last_change":"2024-09-10T18:59:42.711934+0000","last_active":"2024-09-10T19:01:56.363184+0000","last_peered":"2024-09-10T19:01:56.363184+0000","last_clean":"2024-09-10T19:01:56.363184+0000","last_became_active":"2024-09-10T18:59:42.711662+0000","last_became_peered":"2024-09-10T18:59:42.711662+0000","last_unstale":"2024-09-10T19:01:56.363184+0000","last_undegraded":"2024-09-10T19:01:56.363184+0000","last_fullsized":"2024-09-10T19:01:56.363184+0000","mapping_epoch":315,"log_start":"352'8400","ondisk_log_start":"352'8400","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3604,"log_dups_size":0,"ondisk_log_size":3604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":343277568,"num_objects":5238,"num_object_clones":0,"num_object_copies":15714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12004,"num_write_kb":551744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1],"acting":[4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"70'81","reported_seq":283,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.477622+0000","last_change":"2024-09-10T18:59:11.493175+0000","last_active":"2024-09-10T19:01:53.477622+0000","last_peered":"2024-09-10T19:01:53.477622+0000","last_clean":"2024-09-10T19:01:53.477622+0000","last_became_active":"2024-09-10T18:50:08.398345+0000","last_became_peered":"2024-09-10T18:50:08.398345+0000","last_unstale":"2024-09-10T19:01:53.477622+0000","last_undegraded":"2024-09-10T19:01:53.477622+0000","last_fullsized":"2024-09-10T19:01:53.477622+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:11.493086+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:55:37.921519+0000","last_clean_scrub_stamp":"2024-09-10T18:59:11.493086+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013358479999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"232'34","reported_seq":390,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.476940+0000","last_change":"2024-09-10T19:00:41.482710+0000","last_active":"2024-09-10T19:01:53.476940+0000","last_peered":"2024-09-10T19:01:53.476940+0000","last_clean":"2024-09-10T19:01:53.476940+0000","last_became_active":"2024-09-10T18:46:56.742946+0000","last_became_peered":"2024-09-10T18:46:56.742946+0000","last_unstale":"2024-09-10T19:01:53.476940+0000","last_undegraded":"2024-09-10T19:01:53.476940+0000","last_fullsized":"2024-09-10T19:01:53.476940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'34","last_scrub_stamp":"2024-09-10T19:00:41.482619+0000","last_deep_scrub":"141'22","last_deep_scrub_stamp":"2024-09-10T18:50:57.119152+0000","last_clean_scrub_stamp":"2024-09-10T19:00:41.482619+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:02:06.038176+0000","scrub_duration":0.0069363309999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":385,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.481543+0000","last_change":"2024-09-10T18:59:51.286189+0000","last_active":"2024-09-10T19:01:53.481543+0000","last_peered":"2024-09-10T19:01:53.481543+0000","last_clean":"2024-09-10T19:01:53.481543+0000","last_became_active":"2024-09-10T18:57:02.445848+0000","last_became_peered":"2024-09-10T18:57:02.445848+0000","last_unstale":"2024-09-10T19:01:53.481543+0000","last_undegraded":"2024-09-10T19:01:53.481543+0000","last_fullsized":"2024-09-10T19:01:53.481543+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:51.286131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_clean_scrub_stamp":"2024-09-10T18:59:51.286131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015180669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"242'10","reported_seq":582,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.474520+0000","last_change":"2024-09-10T18:56:19.581070+0000","last_active":"2024-09-10T19:01:53.474520+0000","last_peered":"2024-09-10T19:01:53.474520+0000","last_clean":"2024-09-10T19:01:53.474520+0000","last_became_active":"2024-09-10T18:55:40.397246+0000","last_became_peered":"2024-09-10T18:55:40.397246+0000","last_unstale":"2024-09-10T19:01:53.474520+0000","last_undegraded":"2024-09-10T19:01:53.474520+0000","last_fullsized":"2024-09-10T19:01:53.474520+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"242'10","last_scrub_stamp":"2024-09-10T18:56:19.581012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:19.581012+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013423529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":383,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.477205+0000","last_change":"2024-09-10T18:59:10.477418+0000","last_active":"2024-09-10T19:01:53.477205+0000","last_peered":"2024-09-10T19:01:53.477205+0000","last_clean":"2024-09-10T19:01:53.477205+0000","last_became_active":"2024-09-10T18:55:40.392003+0000","last_became_peered":"2024-09-10T18:55:40.392003+0000","last_unstale":"2024-09-10T19:01:53.477205+0000","last_undegraded":"2024-09-10T19:01:53.477205+0000","last_fullsized":"2024-09-10T19:01:53.477205+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:10.477346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:54.218602+0000","last_clean_scrub_stamp":"2024-09-10T18:59:10.477346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004313005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"361'12074","reported_seq":12128,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-09-10T19:01:56.496325+0000","last_change":"2024-09-10T18:59:42.715764+0000","last_active":"2024-09-10T19:01:56.496325+0000","last_peered":"2024-09-10T19:01:56.496325+0000","last_clean":"2024-09-10T19:01:56.496325+0000","last_became_active":"2024-09-10T18:59:42.714808+0000","last_became_peered":"2024-09-10T18:59:42.714808+0000","last_unstale":"2024-09-10T19:01:56.496325+0000","last_undegraded":"2024-09-10T19:01:56.496325+0000","last_fullsized":"2024-09-10T19:01:56.496325+0000","mapping_epoch":315,"log_start":"352'8500","ondisk_log_start":"352'8500","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3574,"log_dups_size":0,"ondisk_log_size":3574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":357564416,"num_objects":5456,"num_object_clones":0,"num_object_copies":16368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12074,"num_write_kb":560960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.b","version":"70'81","reported_seq":328,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.479657+0000","last_change":"2024-09-10T19:01:04.457442+0000","last_active":"2024-09-10T19:01:53.479657+0000","last_peered":"2024-09-10T19:01:53.479657+0000","last_clean":"2024-09-10T19:01:53.479657+0000","last_became_active":"2024-09-10T19:00:42.518107+0000","last_became_peered":"2024-09-10T19:00:42.518107+0000","last_unstale":"2024-09-10T19:01:53.479657+0000","last_undegraded":"2024-09-10T19:01:53.479657+0000","last_fullsized":"2024-09-10T19:01:53.479657+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:33.793296+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:33.793296+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045437230000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"232'24","reported_seq":469,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.585095+0000","last_change":"2024-09-10T18:59:06.595605+0000","last_active":"2024-09-10T19:01:34.585095+0000","last_peered":"2024-09-10T19:01:34.585095+0000","last_clean":"2024-09-10T19:01:34.585095+0000","last_became_active":"2024-09-10T18:47:57.106935+0000","last_became_peered":"2024-09-10T18:47:57.106935+0000","last_unstale":"2024-09-10T19:01:34.585095+0000","last_undegraded":"2024-09-10T19:01:34.585095+0000","last_fullsized":"2024-09-10T19:01:34.585095+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T18:59:06.595554+0000","last_deep_scrub":"224'16","last_deep_scrub_stamp":"2024-09-10T18:54:49.296957+0000","last_clean_scrub_stamp":"2024-09-10T18:59:06.595554+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020642540000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":356,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.485424+0000","last_change":"2024-09-10T18:57:02.444295+0000","last_active":"2024-09-10T19:01:53.485424+0000","last_peered":"2024-09-10T19:01:53.485424+0000","last_clean":"2024-09-10T19:01:53.485424+0000","last_became_active":"2024-09-10T18:57:02.443816+0000","last_became_peered":"2024-09-10T18:57:02.443816+0000","last_unstale":"2024-09-10T19:01:53.485424+0000","last_undegraded":"2024-09-10T19:01:53.485424+0000","last_fullsized":"2024-09-10T19:01:53.485424+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:29.994533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:29.994533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053276879999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":371,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.477082+0000","last_change":"2024-09-10T18:58:54.548235+0000","last_active":"2024-09-10T19:01:53.477082+0000","last_peered":"2024-09-10T19:01:53.477082+0000","last_clean":"2024-09-10T19:01:53.477082+0000","last_became_active":"2024-09-10T18:45:22.068918+0000","last_became_peered":"2024-09-10T18:45:22.068918+0000","last_unstale":"2024-09-10T19:01:53.477082+0000","last_undegraded":"2024-09-10T19:01:53.477082+0000","last_fullsized":"2024-09-10T19:01:53.477082+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:54.548153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_clean_scrub_stamp":"2024-09-10T18:58:54.548153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013051650000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":354,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.591055+0000","last_change":"2024-09-10T18:59:13.892307+0000","last_active":"2024-09-10T19:01:34.591055+0000","last_peered":"2024-09-10T19:01:34.591055+0000","last_clean":"2024-09-10T19:01:34.591055+0000","last_became_active":"2024-09-10T18:45:24.264538+0000","last_became_peered":"2024-09-10T18:45:24.264538+0000","last_unstale":"2024-09-10T19:01:34.591055+0000","last_undegraded":"2024-09-10T19:01:34.591055+0000","last_fullsized":"2024-09-10T19:01:34.591055+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:13.892129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:51.398981+0000","last_clean_scrub_stamp":"2024-09-10T18:59:13.892129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007612214,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"361'11998","reported_seq":12051,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-09-10T19:01:56.517200+0000","last_change":"2024-09-10T18:59:43.008094+0000","last_active":"2024-09-10T19:01:56.517200+0000","last_peered":"2024-09-10T19:01:56.517200+0000","last_clean":"2024-09-10T19:01:56.517200+0000","last_became_active":"2024-09-10T18:59:43.007838+0000","last_became_peered":"2024-09-10T18:59:43.007838+0000","last_unstale":"2024-09-10T19:01:56.517200+0000","last_undegraded":"2024-09-10T19:01:56.517200+0000","last_fullsized":"2024-09-10T19:01:56.517200+0000","mapping_epoch":315,"log_start":"352'8500","ondisk_log_start":"352'8500","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3498,"log_dups_size":0,"ondisk_log_size":3498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348520448,"num_objects":5318,"num_object_clones":0,"num_object_copies":15954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11998,"num_write_kb":554112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"70'81","reported_seq":312,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.591135+0000","last_change":"2024-09-10T19:00:42.520084+0000","last_active":"2024-09-10T19:01:34.591135+0000","last_peered":"2024-09-10T19:01:34.591135+0000","last_clean":"2024-09-10T19:01:34.591135+0000","last_became_active":"2024-09-10T19:00:42.519742+0000","last_became_peered":"2024-09-10T19:00:42.519742+0000","last_unstale":"2024-09-10T19:01:34.591135+0000","last_undegraded":"2024-09-10T19:01:34.591135+0000","last_fullsized":"2024-09-10T19:01:34.591135+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:58:56.912936+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:58:56.912936+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045874109999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"232'35","reported_seq":406,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.583173+0000","last_change":"2024-09-10T18:56:28.475224+0000","last_active":"2024-09-10T19:01:34.583173+0000","last_peered":"2024-09-10T19:01:34.583173+0000","last_clean":"2024-09-10T19:01:34.583173+0000","last_became_active":"2024-09-10T18:45:20.052243+0000","last_became_peered":"2024-09-10T18:45:20.052243+0000","last_unstale":"2024-09-10T19:01:34.583173+0000","last_undegraded":"2024-09-10T19:01:34.583173+0000","last_fullsized":"2024-09-10T19:01:34.583173+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'35","last_scrub_stamp":"2024-09-10T18:56:28.475142+0000","last_deep_scrub":"143'21","last_deep_scrub_stamp":"2024-09-10T18:51:02.540216+0000","last_clean_scrub_stamp":"2024-09-10T18:56:28.475142+0000","objects_scrubbed":7,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026978649999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":403,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:55.150399+0000","last_change":"2024-09-10T19:01:55.150294+0000","last_active":"2024-09-10T19:01:55.150399+0000","last_peered":"2024-09-10T19:01:55.150399+0000","last_clean":"2024-09-10T19:01:55.150399+0000","last_became_active":"2024-09-10T18:57:02.453404+0000","last_became_peered":"2024-09-10T18:57:02.453404+0000","last_unstale":"2024-09-10T19:01:55.150399+0000","last_undegraded":"2024-09-10T19:01:55.150399+0000","last_fullsized":"2024-09-10T19:01:55.150399+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:01:55.150215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_clean_scrub_stamp":"2024-09-10T19:01:55.150215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:02:58.303982+0000","scrub_duration":0.0060613990000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":563,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:51.270617+0000","last_change":"2024-09-10T18:56:17.914259+0000","last_active":"2024-09-10T19:01:51.270617+0000","last_peered":"2024-09-10T19:01:51.270617+0000","last_clean":"2024-09-10T19:01:51.270617+0000","last_became_active":"2024-09-10T18:45:22.074540+0000","last_became_peered":"2024-09-10T18:45:22.074540+0000","last_unstale":"2024-09-10T19:01:51.270617+0000","last_undegraded":"2024-09-10T19:01:51.270617+0000","last_fullsized":"2024-09-10T19:01:51.270617+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T18:56:17.914175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:17.914175+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026666329999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":160,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.479581+0000","last_change":"2024-09-10T18:58:50.638774+0000","last_active":"2024-09-10T19:01:53.479581+0000","last_peered":"2024-09-10T19:01:53.479581+0000","last_clean":"2024-09-10T19:01:53.479581+0000","last_became_active":"2024-09-10T18:57:02.437100+0000","last_became_peered":"2024-09-10T18:57:02.437100+0000","last_unstale":"2024-09-10T19:01:53.479581+0000","last_undegraded":"2024-09-10T19:01:53.479581+0000","last_fullsized":"2024-09-10T19:01:53.479581+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:50.638720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:58:50.638720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003753546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.e","version":"360'11941","reported_seq":11994,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:56.335456+0000","last_change":"2024-09-10T18:59:42.713933+0000","last_active":"2024-09-10T19:01:56.335456+0000","last_peered":"2024-09-10T19:01:56.335456+0000","last_clean":"2024-09-10T19:01:56.335456+0000","last_became_active":"2024-09-10T18:59:42.713605+0000","last_became_peered":"2024-09-10T18:59:42.713605+0000","last_unstale":"2024-09-10T19:01:56.335456+0000","last_undegraded":"2024-09-10T19:01:56.335456+0000","last_fullsized":"2024-09-10T19:01:56.335456+0000","mapping_epoch":315,"log_start":"352'8400","ondisk_log_start":"352'8400","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3541,"log_dups_size":0,"ondisk_log_size":3541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":342163456,"num_objects":5221,"num_object_clones":0,"num_object_copies":15663,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11941,"num_write_kb":549184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"70'81","reported_seq":277,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.583310+0000","last_change":"2024-09-10T18:56:31.481362+0000","last_active":"2024-09-10T19:01:34.583310+0000","last_peered":"2024-09-10T19:01:34.583310+0000","last_clean":"2024-09-10T19:01:34.583310+0000","last_became_active":"2024-09-10T18:52:14.584798+0000","last_became_peered":"2024-09-10T18:52:14.584798+0000","last_unstale":"2024-09-10T19:01:34.583310+0000","last_undegraded":"2024-09-10T19:01:34.583310+0000","last_fullsized":"2024-09-10T19:01:34.583310+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:31.481303+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:56:31.481303+0000","last_clean_scrub_stamp":"2024-09-10T18:56:31.481303+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0011170959999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"232'30","reported_seq":524,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.588898+0000","last_change":"2024-09-10T18:59:48.754412+0000","last_active":"2024-09-10T19:01:34.588898+0000","last_peered":"2024-09-10T19:01:34.588898+0000","last_clean":"2024-09-10T19:01:34.588898+0000","last_became_active":"2024-09-10T18:59:48.742377+0000","last_became_peered":"2024-09-10T18:59:48.742377+0000","last_unstale":"2024-09-10T19:01:34.588898+0000","last_undegraded":"2024-09-10T19:01:34.588898+0000","last_fullsized":"2024-09-10T19:01:34.588898+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'30","last_scrub_stamp":"2024-09-10T18:56:15.875924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:15.875924+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0075153879999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":365,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.479457+0000","last_change":"2024-09-10T18:59:05.246945+0000","last_active":"2024-09-10T19:01:53.479457+0000","last_peered":"2024-09-10T19:01:53.479457+0000","last_clean":"2024-09-10T19:01:53.479457+0000","last_became_active":"2024-09-10T18:45:17.961695+0000","last_became_peered":"2024-09-10T18:45:17.961695+0000","last_unstale":"2024-09-10T19:01:53.479457+0000","last_undegraded":"2024-09-10T19:01:53.479457+0000","last_fullsized":"2024-09-10T19:01:53.479457+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:05.246629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:46.142129+0000","last_clean_scrub_stamp":"2024-09-10T18:59:05.246629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0072716029999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":375,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.582956+0000","last_change":"2024-09-10T18:59:57.903394+0000","last_active":"2024-09-10T19:01:34.582956+0000","last_peered":"2024-09-10T19:01:34.582956+0000","last_clean":"2024-09-10T19:01:34.582956+0000","last_became_active":"2024-09-10T18:57:02.448126+0000","last_became_peered":"2024-09-10T18:57:02.448126+0000","last_unstale":"2024-09-10T19:01:34.582956+0000","last_undegraded":"2024-09-10T19:01:34.582956+0000","last_fullsized":"2024-09-10T19:01:34.582956+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:57.903311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:57.903311+0000","last_clean_scrub_stamp":"2024-09-10T18:59:57.903311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011228738,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":363,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.583011+0000","last_change":"2024-09-10T18:58:48.633501+0000","last_active":"2024-09-10T19:01:34.583011+0000","last_peered":"2024-09-10T19:01:34.583011+0000","last_clean":"2024-09-10T19:01:34.583011+0000","last_became_active":"2024-09-10T18:47:36.410836+0000","last_became_peered":"2024-09-10T18:47:36.410836+0000","last_unstale":"2024-09-10T19:01:34.583011+0000","last_undegraded":"2024-09-10T19:01:34.583011+0000","last_fullsized":"2024-09-10T19:01:34.583011+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:48.633423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_clean_scrub_stamp":"2024-09-10T18:58:48.633423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0079262829999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.d","version":"361'12091","reported_seq":12146,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-09-10T19:01:56.512384+0000","last_change":"2024-09-10T18:59:42.717015+0000","last_active":"2024-09-10T19:01:56.512384+0000","last_peered":"2024-09-10T19:01:56.512384+0000","last_clean":"2024-09-10T19:01:56.512384+0000","last_became_active":"2024-09-10T18:59:42.716708+0000","last_became_peered":"2024-09-10T18:59:42.716708+0000","last_unstale":"2024-09-10T19:01:56.512384+0000","last_undegraded":"2024-09-10T19:01:56.512384+0000","last_fullsized":"2024-09-10T19:01:56.512384+0000","mapping_epoch":315,"log_start":"352'8600","ondisk_log_start":"352'8600","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3491,"log_dups_size":0,"ondisk_log_size":3491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":356515864,"num_objects":5441,"num_object_clones":0,"num_object_copies":16323,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5441,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12091,"num_write_kb":560961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"70'81","reported_seq":376,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.479881+0000","last_change":"2024-09-10T18:59:23.481986+0000","last_active":"2024-09-10T19:01:53.479881+0000","last_peered":"2024-09-10T19:01:53.479881+0000","last_clean":"2024-09-10T19:01:53.479881+0000","last_became_active":"2024-09-10T18:55:40.396828+0000","last_became_peered":"2024-09-10T18:55:40.396828+0000","last_unstale":"2024-09-10T19:01:53.479881+0000","last_undegraded":"2024-09-10T19:01:53.479881+0000","last_fullsized":"2024-09-10T19:01:53.479881+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:23.481902+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:59:23.481902+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0052121340000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"232'29","reported_seq":418,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.484112+0000","last_change":"2024-09-10T18:58:58.837250+0000","last_active":"2024-09-10T19:01:53.484112+0000","last_peered":"2024-09-10T19:01:53.484112+0000","last_clean":"2024-09-10T19:01:53.484112+0000","last_became_active":"2024-09-10T18:57:02.442862+0000","last_became_peered":"2024-09-10T18:57:02.442862+0000","last_unstale":"2024-09-10T19:01:53.484112+0000","last_undegraded":"2024-09-10T19:01:53.484112+0000","last_fullsized":"2024-09-10T19:01:53.484112+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'29","last_scrub_stamp":"2024-09-10T18:58:58.837163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:58:58.837163+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.019169703999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":356,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.494331+0000","last_change":"2024-09-10T18:59:53.858914+0000","last_active":"2024-09-10T19:01:53.494331+0000","last_peered":"2024-09-10T19:01:53.494331+0000","last_clean":"2024-09-10T19:01:53.494331+0000","last_became_active":"2024-09-10T18:57:02.443522+0000","last_became_peered":"2024-09-10T18:57:02.443522+0000","last_unstale":"2024-09-10T19:01:53.494331+0000","last_undegraded":"2024-09-10T19:01:53.494331+0000","last_fullsized":"2024-09-10T19:01:53.494331+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:53.858807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:48.196065+0000","last_clean_scrub_stamp":"2024-09-10T18:59:53.858807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044256060000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":386,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.479299+0000","last_change":"2024-09-10T18:59:15.511792+0000","last_active":"2024-09-10T19:01:53.479299+0000","last_peered":"2024-09-10T19:01:53.479299+0000","last_clean":"2024-09-10T19:01:53.479299+0000","last_became_active":"2024-09-10T18:55:40.387857+0000","last_became_peered":"2024-09-10T18:55:40.387857+0000","last_unstale":"2024-09-10T19:01:53.479299+0000","last_undegraded":"2024-09-10T19:01:53.479299+0000","last_fullsized":"2024-09-10T19:01:53.479299+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:15.511743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:59:15.511743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001893837,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":376,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.481445+0000","last_change":"2024-09-10T18:59:16.235312+0000","last_active":"2024-09-10T19:01:53.481445+0000","last_peered":"2024-09-10T19:01:53.481445+0000","last_clean":"2024-09-10T19:01:53.481445+0000","last_became_active":"2024-09-10T18:57:02.442479+0000","last_became_peered":"2024-09-10T18:57:02.442479+0000","last_unstale":"2024-09-10T19:01:53.481445+0000","last_undegraded":"2024-09-10T19:01:53.481445+0000","last_fullsized":"2024-09-10T19:01:53.481445+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:16.235235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:49:18.281230+0000","last_clean_scrub_stamp":"2024-09-10T18:59:16.235235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034192950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"361'11767","reported_seq":13407,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-09-10T19:01:56.807293+0000","last_change":"2024-09-10T19:00:08.469551+0000","last_active":"2024-09-10T19:01:56.807293+0000","last_peered":"2024-09-10T19:01:56.807293+0000","last_clean":"2024-09-10T19:01:56.807293+0000","last_became_active":"2024-09-10T19:00:08.465795+0000","last_became_peered":"2024-09-10T19:00:08.465795+0000","last_unstale":"2024-09-10T19:01:56.807293+0000","last_undegraded":"2024-09-10T19:01:56.807293+0000","last_fullsized":"2024-09-10T19:01:56.807293+0000","mapping_epoch":329,"log_start":"352'8300","ondisk_log_start":"352'8300","created":315,"last_epoch_clean":330,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3467,"log_dups_size":0,"ondisk_log_size":3467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":344391680,"num_objects":5255,"num_object_clones":0,"num_object_copies":15765,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11767,"num_write_kb":544704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":784,"num_bytes_recovered":51380224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"70'81","reported_seq":379,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.480435+0000","last_change":"2024-09-10T18:59:53.550489+0000","last_active":"2024-09-10T19:01:53.480435+0000","last_peered":"2024-09-10T19:01:53.480435+0000","last_clean":"2024-09-10T19:01:53.480435+0000","last_became_active":"2024-09-10T18:57:02.452659+0000","last_became_peered":"2024-09-10T18:57:02.452659+0000","last_unstale":"2024-09-10T19:01:53.480435+0000","last_undegraded":"2024-09-10T19:01:53.480435+0000","last_fullsized":"2024-09-10T19:01:53.480435+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:53.550433+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:59:53.550433+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013042399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"232'25","reported_seq":479,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.481611+0000","last_change":"2024-09-10T19:00:42.531653+0000","last_active":"2024-09-10T19:01:53.481611+0000","last_peered":"2024-09-10T19:01:53.481611+0000","last_clean":"2024-09-10T19:01:53.481611+0000","last_became_active":"2024-09-10T19:00:42.517353+0000","last_became_peered":"2024-09-10T19:00:42.517353+0000","last_unstale":"2024-09-10T19:01:53.481611+0000","last_undegraded":"2024-09-10T19:01:53.481611+0000","last_fullsized":"2024-09-10T19:01:53.481611+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'25","last_scrub_stamp":"2024-09-10T19:00:07.390135+0000","last_deep_scrub":"83'15","last_deep_scrub_stamp":"2024-09-10T18:46:58.436473+0000","last_clean_scrub_stamp":"2024-09-10T19:00:07.390135+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002339874,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":402,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.483851+0000","last_change":"2024-09-10T19:01:14.526223+0000","last_active":"2024-09-10T19:01:53.483851+0000","last_peered":"2024-09-10T19:01:53.483851+0000","last_clean":"2024-09-10T19:01:53.483851+0000","last_became_active":"2024-09-10T18:57:02.447940+0000","last_became_peered":"2024-09-10T18:57:02.447940+0000","last_unstale":"2024-09-10T19:01:53.483851+0000","last_undegraded":"2024-09-10T19:01:53.483851+0000","last_fullsized":"2024-09-10T19:01:53.483851+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:50.137393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:05.140027+0000","last_clean_scrub_stamp":"2024-09-10T18:58:50.137393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014454461,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"339'13","reported_seq":536,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:56.270918+0000","last_change":"2024-09-10T19:00:42.528804+0000","last_active":"2024-09-10T19:01:56.270918+0000","last_peered":"2024-09-10T19:01:56.270918+0000","last_clean":"2024-09-10T19:01:56.270918+0000","last_became_active":"2024-09-10T19:00:42.520360+0000","last_became_peered":"2024-09-10T19:00:42.520360+0000","last_unstale":"2024-09-10T19:01:56.270918+0000","last_undegraded":"2024-09-10T19:01:56.270918+0000","last_fullsized":"2024-09-10T19:01:56.270918+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'12","last_scrub_stamp":"2024-09-10T18:59:55.943869+0000","last_deep_scrub":"258'12","last_deep_scrub_stamp":"2024-09-10T18:56:21.098770+0000","last_clean_scrub_stamp":"2024-09-10T18:59:55.943869+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.020416790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":342,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.485650+0000","last_change":"2024-09-10T19:00:42.515515+0000","last_active":"2024-09-10T19:01:53.485650+0000","last_peered":"2024-09-10T19:01:53.485650+0000","last_clean":"2024-09-10T19:01:53.485650+0000","last_became_active":"2024-09-10T19:00:42.515144+0000","last_became_peered":"2024-09-10T19:00:42.515144+0000","last_unstale":"2024-09-10T19:01:53.485650+0000","last_undegraded":"2024-09-10T19:01:53.485650+0000","last_fullsized":"2024-09-10T19:01:53.485650+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:57:03.353014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:57:03.353014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014199639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.a","version":"361'12229","reported_seq":12293,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-09-10T19:01:56.504352+0000","last_change":"2024-09-10T18:59:48.739701+0000","last_active":"2024-09-10T19:01:56.504352+0000","last_peered":"2024-09-10T19:01:56.504352+0000","last_clean":"2024-09-10T19:01:56.504352+0000","last_became_active":"2024-09-10T18:59:48.739415+0000","last_became_peered":"2024-09-10T18:59:48.739415+0000","last_unstale":"2024-09-10T19:01:56.504352+0000","last_undegraded":"2024-09-10T19:01:56.504352+0000","last_fullsized":"2024-09-10T19:01:56.504352+0000","mapping_epoch":320,"log_start":"352'8700","ondisk_log_start":"352'8700","created":315,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3529,"log_dups_size":0,"ondisk_log_size":3529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":357105664,"num_objects":5449,"num_object_clones":0,"num_object_copies":16347,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12229,"num_write_kb":565696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"70'81","reported_seq":310,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.476421+0000","last_change":"2024-09-10T19:00:40.472680+0000","last_active":"2024-09-10T19:01:53.476421+0000","last_peered":"2024-09-10T19:01:53.476421+0000","last_clean":"2024-09-10T19:01:53.476421+0000","last_became_active":"2024-09-10T18:55:40.392432+0000","last_became_peered":"2024-09-10T18:55:40.392432+0000","last_unstale":"2024-09-10T19:01:53.476421+0000","last_undegraded":"2024-09-10T19:01:53.476421+0000","last_fullsized":"2024-09-10T19:01:53.476421+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:00:40.472595+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:00:40.472595+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0072897099999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"232'41","reported_seq":661,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:56.270374+0000","last_change":"2024-09-10T19:00:57.500215+0000","last_active":"2024-09-10T19:01:56.270374+0000","last_peered":"2024-09-10T19:01:56.270374+0000","last_clean":"2024-09-10T19:01:56.270374+0000","last_became_active":"2024-09-10T18:47:36.420185+0000","last_became_peered":"2024-09-10T18:47:36.420185+0000","last_unstale":"2024-09-10T19:01:56.270374+0000","last_undegraded":"2024-09-10T19:01:56.270374+0000","last_fullsized":"2024-09-10T19:01:56.270374+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'41","last_scrub_stamp":"2024-09-10T18:56:18.672139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:18.672139+0000","objects_scrubbed":10,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.020233260999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":375,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.476949+0000","last_change":"2024-09-10T18:59:04.434037+0000","last_active":"2024-09-10T19:01:53.476949+0000","last_peered":"2024-09-10T19:01:53.476949+0000","last_clean":"2024-09-10T19:01:53.476949+0000","last_became_active":"2024-09-10T18:45:17.964494+0000","last_became_peered":"2024-09-10T18:45:17.964494+0000","last_unstale":"2024-09-10T19:01:53.476949+0000","last_undegraded":"2024-09-10T19:01:53.476949+0000","last_fullsized":"2024-09-10T19:01:53.476949+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:04.433964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:12.982599+0000","last_clean_scrub_stamp":"2024-09-10T18:59:04.433964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024364220000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":317,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.479933+0000","last_change":"2024-09-10T18:56:20.224668+0000","last_active":"2024-09-10T19:01:53.479933+0000","last_peered":"2024-09-10T19:01:53.479933+0000","last_clean":"2024-09-10T19:01:53.479933+0000","last_became_active":"2024-09-10T18:47:57.107478+0000","last_became_peered":"2024-09-10T18:47:57.107478+0000","last_unstale":"2024-09-10T19:01:53.479933+0000","last_undegraded":"2024-09-10T19:01:53.479933+0000","last_fullsized":"2024-09-10T19:01:53.479933+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:20.224562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:20.224562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.008542315,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":354,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.591900+0000","last_change":"2024-09-10T18:57:02.462009+0000","last_active":"2024-09-10T19:01:34.591900+0000","last_peered":"2024-09-10T19:01:34.591900+0000","last_clean":"2024-09-10T19:01:34.591900+0000","last_became_active":"2024-09-10T18:57:02.460594+0000","last_became_peered":"2024-09-10T18:57:02.460594+0000","last_unstale":"2024-09-10T19:01:34.591900+0000","last_undegraded":"2024-09-10T19:01:34.591900+0000","last_fullsized":"2024-09-10T19:01:34.591900+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:11.878181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:22.891084+0000","last_clean_scrub_stamp":"2024-09-10T18:56:11.878181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037255349999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"361'11997","reported_seq":26274,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-09-10T19:01:56.592091+0000","last_change":"2024-09-10T19:01:25.571972+0000","last_active":"2024-09-10T19:01:56.592091+0000","last_peered":"2024-09-10T19:01:56.592091+0000","last_clean":"2024-09-10T19:01:56.592091+0000","last_became_active":"2024-09-10T19:01:25.526168+0000","last_became_peered":"2024-09-10T19:01:25.526168+0000","last_unstale":"2024-09-10T19:01:56.592091+0000","last_undegraded":"2024-09-10T19:01:56.592091+0000","last_fullsized":"2024-09-10T19:01:56.592091+0000","mapping_epoch":355,"log_start":"352'8338","ondisk_log_start":"352'8338","created":315,"last_epoch_clean":356,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3659,"log_dups_size":0,"ondisk_log_size":3659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":346750976,"num_objects":5291,"num_object_clones":0,"num_object_copies":15873,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11997,"num_write_kb":553216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7098,"num_bytes_recovered":465108992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"70'81","reported_seq":178,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.485621+0000","last_change":"2024-09-10T18:59:11.875713+0000","last_active":"2024-09-10T19:01:53.485621+0000","last_peered":"2024-09-10T19:01:53.485621+0000","last_clean":"2024-09-10T19:01:53.485621+0000","last_became_active":"2024-09-10T18:57:02.437944+0000","last_became_peered":"2024-09-10T18:57:02.437944+0000","last_unstale":"2024-09-10T19:01:53.485621+0000","last_undegraded":"2024-09-10T19:01:53.485621+0000","last_fullsized":"2024-09-10T19:01:53.485621+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:11.875625+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:59:11.875625+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040920360000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"232'22","reported_seq":422,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.479929+0000","last_change":"2024-09-10T19:00:08.468265+0000","last_active":"2024-09-10T19:01:53.479929+0000","last_peered":"2024-09-10T19:01:53.479929+0000","last_clean":"2024-09-10T19:01:53.479929+0000","last_became_active":"2024-09-10T18:55:40.397422+0000","last_became_peered":"2024-09-10T18:55:40.397422+0000","last_unstale":"2024-09-10T19:01:53.479929+0000","last_undegraded":"2024-09-10T19:01:53.479929+0000","last_fullsized":"2024-09-10T19:01:53.479929+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'22","last_scrub_stamp":"2024-09-10T19:00:08.468154+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-10T18:47:10.514951+0000","last_clean_scrub_stamp":"2024-09-10T19:00:08.468154+0000","objects_scrubbed":5,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024987170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"84'6","reported_seq":1388,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-09-10T19:01:56.494395+0000","last_change":"2024-09-10T19:00:40.504252+0000","last_active":"2024-09-10T19:01:56.494395+0000","last_peered":"2024-09-10T19:01:56.494395+0000","last_clean":"2024-09-10T19:01:56.494395+0000","last_became_active":"2024-09-10T18:55:40.403098+0000","last_became_peered":"2024-09-10T18:55:40.403098+0000","last_unstale":"2024-09-10T19:01:56.494395+0000","last_undegraded":"2024-09-10T19:01:56.494395+0000","last_fullsized":"2024-09-10T19:01:56.494395+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'6","last_scrub_stamp":"2024-09-10T19:00:40.504174+0000","last_deep_scrub":"84'6","last_deep_scrub_stamp":"2024-09-10T18:47:22.545450+0000","last_clean_scrub_stamp":"2024-09-10T19:00:40.504174+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:02:03.384361+0000","scrub_duration":0.0052333889999999997,"objects_trimmed":0,"snaptrim_duration":0.0021933159999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":982,"num_read_kb":977,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":338,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.588797+0000","last_change":"2024-09-10T19:00:08.141664+0000","last_active":"2024-09-10T19:01:34.588797+0000","last_peered":"2024-09-10T19:01:34.588797+0000","last_clean":"2024-09-10T19:01:34.588797+0000","last_became_active":"2024-09-10T18:55:40.393389+0000","last_became_peered":"2024-09-10T18:55:40.393389+0000","last_unstale":"2024-09-10T19:01:34.588797+0000","last_undegraded":"2024-09-10T19:01:34.588797+0000","last_fullsized":"2024-09-10T19:01:34.588797+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:08.141603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:00:08.141603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001270729,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":346,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.478262+0000","last_change":"2024-09-10T18:58:58.522346+0000","last_active":"2024-09-10T19:01:53.478262+0000","last_peered":"2024-09-10T19:01:53.478262+0000","last_clean":"2024-09-10T19:01:53.478262+0000","last_became_active":"2024-09-10T18:57:02.438945+0000","last_became_peered":"2024-09-10T18:57:02.438945+0000","last_unstale":"2024-09-10T19:01:53.478262+0000","last_undegraded":"2024-09-10T19:01:53.478262+0000","last_fullsized":"2024-09-10T19:01:53.478262+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:58.522295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:58:58.522295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0059618290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":113,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.480208+0000","last_change":"2024-09-10T19:00:41.794168+0000","last_active":"2024-09-10T19:01:53.480208+0000","last_peered":"2024-09-10T19:01:53.480208+0000","last_clean":"2024-09-10T19:01:53.480208+0000","last_became_active":"2024-09-10T18:57:02.453865+0000","last_became_peered":"2024-09-10T18:57:02.453865+0000","last_unstale":"2024-09-10T19:01:53.480208+0000","last_undegraded":"2024-09-10T19:01:53.480208+0000","last_fullsized":"2024-09-10T19:01:53.480208+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:41.794116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:07.150480+0000","last_clean_scrub_stamp":"2024-09-10T19:00:41.794116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001296148,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.8","version":"361'12211","reported_seq":12265,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-09-10T19:01:58.737249+0000","last_change":"2024-09-10T18:59:42.712702+0000","last_active":"2024-09-10T19:01:58.737249+0000","last_peered":"2024-09-10T19:01:58.737249+0000","last_clean":"2024-09-10T19:01:58.737249+0000","last_became_active":"2024-09-10T18:59:42.712397+0000","last_became_peered":"2024-09-10T18:59:42.712397+0000","last_unstale":"2024-09-10T19:01:58.737249+0000","last_undegraded":"2024-09-10T19:01:58.737249+0000","last_fullsized":"2024-09-10T19:01:58.737249+0000","mapping_epoch":315,"log_start":"352'8500","ondisk_log_start":"352'8500","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3711,"log_dups_size":0,"ondisk_log_size":3711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330366976,"num_objects":5041,"num_object_clones":0,"num_object_copies":15123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12211,"num_write_kb":552064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,4],"acting":[0,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"70'81","reported_seq":279,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.479289+0000","last_change":"2024-09-10T18:58:49.141636+0000","last_active":"2024-09-10T19:01:53.479289+0000","last_peered":"2024-09-10T19:01:53.479289+0000","last_clean":"2024-09-10T19:01:53.479289+0000","last_became_active":"2024-09-10T18:52:14.589009+0000","last_became_peered":"2024-09-10T18:52:14.589009+0000","last_unstale":"2024-09-10T19:01:53.479289+0000","last_undegraded":"2024-09-10T19:01:53.479289+0000","last_fullsized":"2024-09-10T19:01:53.479289+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:58:49.141013+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:58:49.141013+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0036869540000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"232'24","reported_seq":98,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.484101+0000","last_change":"2024-09-10T19:00:42.532827+0000","last_active":"2024-09-10T19:01:53.484101+0000","last_peered":"2024-09-10T19:01:53.484101+0000","last_clean":"2024-09-10T19:01:53.484101+0000","last_became_active":"2024-09-10T19:00:42.516521+0000","last_became_peered":"2024-09-10T19:00:42.516521+0000","last_unstale":"2024-09-10T19:01:53.484101+0000","last_undegraded":"2024-09-10T19:01:53.484101+0000","last_fullsized":"2024-09-10T19:01:53.484101+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T19:00:39.748509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:00:39.748509+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:02:03.931650+0000","scrub_duration":0.0027783730000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"305'2","reported_seq":345,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.484401+0000","last_change":"2024-09-10T18:59:18.897245+0000","last_active":"2024-09-10T19:01:53.484401+0000","last_peered":"2024-09-10T19:01:53.484401+0000","last_clean":"2024-09-10T19:01:53.484401+0000","last_became_active":"2024-09-10T18:55:40.390022+0000","last_became_peered":"2024-09-10T18:55:40.390022+0000","last_unstale":"2024-09-10T19:01:53.484401+0000","last_undegraded":"2024-09-10T19:01:53.484401+0000","last_fullsized":"2024-09-10T19:01:53.484401+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"305'2","last_scrub_stamp":"2024-09-10T18:59:18.897059+0000","last_deep_scrub":"305'2","last_deep_scrub_stamp":"2024-09-10T18:59:18.897059+0000","last_clean_scrub_stamp":"2024-09-10T18:59:18.897059+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016452700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":306,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.475738+0000","last_change":"2024-09-10T19:00:56.482753+0000","last_active":"2024-09-10T19:01:53.475738+0000","last_peered":"2024-09-10T19:01:53.475738+0000","last_clean":"2024-09-10T19:01:53.475738+0000","last_became_active":"2024-09-10T18:57:02.447195+0000","last_became_peered":"2024-09-10T18:57:02.447195+0000","last_unstale":"2024-09-10T19:01:53.475738+0000","last_undegraded":"2024-09-10T19:01:53.475738+0000","last_fullsized":"2024-09-10T19:01:53.475738+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:24.153973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:24.153973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001771613,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":32,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.476795+0000","last_change":"2024-09-10T19:00:42.518887+0000","last_active":"2024-09-10T19:01:53.476795+0000","last_peered":"2024-09-10T19:01:53.476795+0000","last_clean":"2024-09-10T19:01:53.476795+0000","last_became_active":"2024-09-10T19:00:42.518566+0000","last_became_peered":"2024-09-10T19:00:42.518566+0000","last_unstale":"2024-09-10T19:01:53.476795+0000","last_undegraded":"2024-09-10T19:01:53.476795+0000","last_fullsized":"2024-09-10T19:01:53.476795+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:39.752722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:00:39.752722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":362,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.483332+0000","last_change":"2024-09-10T19:01:09.387164+0000","last_active":"2024-09-10T19:01:53.483332+0000","last_peered":"2024-09-10T19:01:53.483332+0000","last_clean":"2024-09-10T19:01:53.483332+0000","last_became_active":"2024-09-10T18:45:24.083689+0000","last_became_peered":"2024-09-10T18:45:24.083689+0000","last_unstale":"2024-09-10T19:01:53.483332+0000","last_undegraded":"2024-09-10T19:01:53.483332+0000","last_fullsized":"2024-09-10T19:01:53.483332+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:52.136095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:58:52.136095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.021731822000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"361'12148","reported_seq":12201,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-09-10T19:01:56.506589+0000","last_change":"2024-09-10T18:59:42.717031+0000","last_active":"2024-09-10T19:01:56.506589+0000","last_peered":"2024-09-10T19:01:56.506589+0000","last_clean":"2024-09-10T19:01:56.506589+0000","last_became_active":"2024-09-10T18:59:42.716730+0000","last_became_peered":"2024-09-10T18:59:42.716730+0000","last_unstale":"2024-09-10T19:01:56.506589+0000","last_undegraded":"2024-09-10T19:01:56.506589+0000","last_fullsized":"2024-09-10T19:01:56.506589+0000","mapping_epoch":315,"log_start":"352'8600","ondisk_log_start":"352'8600","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3548,"log_dups_size":0,"ondisk_log_size":3548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358612992,"num_objects":5472,"num_object_clones":0,"num_object_copies":16416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12148,"num_write_kb":563840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"232'33","reported_seq":454,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.480283+0000","last_change":"2024-09-10T18:59:17.515699+0000","last_active":"2024-09-10T19:01:53.480283+0000","last_peered":"2024-09-10T19:01:53.480283+0000","last_clean":"2024-09-10T19:01:53.480283+0000","last_became_active":"2024-09-10T18:57:02.435499+0000","last_became_peered":"2024-09-10T18:57:02.435499+0000","last_unstale":"2024-09-10T19:01:53.480283+0000","last_undegraded":"2024-09-10T19:01:53.480283+0000","last_fullsized":"2024-09-10T19:01:53.480283+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'33","last_scrub_stamp":"2024-09-10T18:59:17.515615+0000","last_deep_scrub":"232'33","last_deep_scrub_stamp":"2024-09-10T18:56:21.206960+0000","last_clean_scrub_stamp":"2024-09-10T18:59:17.515615+0000","objects_scrubbed":7,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.009416209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"60'2","reported_seq":373,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.589060+0000","last_change":"2024-09-10T18:58:55.931801+0000","last_active":"2024-09-10T19:01:34.589060+0000","last_peered":"2024-09-10T19:01:34.589060+0000","last_clean":"2024-09-10T19:01:34.589060+0000","last_became_active":"2024-09-10T18:45:17.961660+0000","last_became_peered":"2024-09-10T18:45:17.961660+0000","last_unstale":"2024-09-10T19:01:34.589060+0000","last_undegraded":"2024-09-10T19:01:34.589060+0000","last_fullsized":"2024-09-10T19:01:34.589060+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-10T18:58:55.931717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:58:55.931717+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040038909999999999,"objects_trimmed":0,"snaptrim_duration":0.0013942640000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":402,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-09-10T19:01:56.581181+0000","last_change":"2024-09-10T19:01:56.581104+0000","last_active":"2024-09-10T19:01:56.581181+0000","last_peered":"2024-09-10T19:01:56.581181+0000","last_clean":"2024-09-10T19:01:56.581181+0000","last_became_active":"2024-09-10T18:57:02.445712+0000","last_became_peered":"2024-09-10T18:57:02.445712+0000","last_unstale":"2024-09-10T19:01:56.581181+0000","last_undegraded":"2024-09-10T19:01:56.581181+0000","last_fullsized":"2024-09-10T19:01:56.581181+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:01:56.581018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:01:56.581018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:02.923984+0000","scrub_duration":0.0048884530000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'81","reported_seq":178,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.485135+0000","last_change":"2024-09-10T18:59:14.873293+0000","last_active":"2024-09-10T19:01:53.485135+0000","last_peered":"2024-09-10T19:01:53.485135+0000","last_clean":"2024-09-10T19:01:53.485135+0000","last_became_active":"2024-09-10T18:57:02.440063+0000","last_became_peered":"2024-09-10T18:57:02.440063+0000","last_unstale":"2024-09-10T19:01:53.485135+0000","last_undegraded":"2024-09-10T19:01:53.485135+0000","last_fullsized":"2024-09-10T19:01:53.485135+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:14.873083+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:56.085580+0000","last_clean_scrub_stamp":"2024-09-10T18:59:14.873083+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0063632760000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":371,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:55.620830+0000","last_change":"2024-09-10T19:01:55.620765+0000","last_active":"2024-09-10T19:01:55.620830+0000","last_peered":"2024-09-10T19:01:55.620830+0000","last_clean":"2024-09-10T19:01:55.620830+0000","last_became_active":"2024-09-10T18:57:02.431556+0000","last_became_peered":"2024-09-10T18:57:02.431556+0000","last_unstale":"2024-09-10T19:01:55.620830+0000","last_undegraded":"2024-09-10T19:01:55.620830+0000","last_fullsized":"2024-09-10T19:01:55.620830+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:01:55.620696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:01:55.620696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:02:57.091592+0000","scrub_duration":0.0068284349999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":376,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.477166+0000","last_change":"2024-09-10T19:00:07.624756+0000","last_active":"2024-09-10T19:01:53.477166+0000","last_peered":"2024-09-10T19:01:53.477166+0000","last_clean":"2024-09-10T19:01:53.477166+0000","last_became_active":"2024-09-10T18:45:24.073327+0000","last_became_peered":"2024-09-10T18:45:24.073327+0000","last_unstale":"2024-09-10T19:01:53.477166+0000","last_undegraded":"2024-09-10T19:01:53.477166+0000","last_fullsized":"2024-09-10T19:01:53.477166+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:07.624701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:00:07.624701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001137962,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"361'12065","reported_seq":12119,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-09-10T19:01:58.746323+0000","last_change":"2024-09-10T18:59:43.009491+0000","last_active":"2024-09-10T19:01:58.746323+0000","last_peered":"2024-09-10T19:01:58.746323+0000","last_clean":"2024-09-10T19:01:58.746323+0000","last_became_active":"2024-09-10T18:59:43.009210+0000","last_became_peered":"2024-09-10T18:59:43.009210+0000","last_unstale":"2024-09-10T19:01:58.746323+0000","last_undegraded":"2024-09-10T19:01:58.746323+0000","last_fullsized":"2024-09-10T19:01:58.746323+0000","mapping_epoch":315,"log_start":"352'8400","ondisk_log_start":"352'8400","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3665,"log_dups_size":0,"ondisk_log_size":3665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":331022336,"num_objects":5051,"num_object_clones":0,"num_object_copies":15153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12065,"num_write_kb":547712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"70'81","reported_seq":371,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.480175+0000","last_change":"2024-09-10T18:58:59.582622+0000","last_active":"2024-09-10T19:01:53.480175+0000","last_peered":"2024-09-10T19:01:53.480175+0000","last_clean":"2024-09-10T19:01:53.480175+0000","last_became_active":"2024-09-10T18:52:16.598295+0000","last_became_peered":"2024-09-10T18:52:16.598295+0000","last_unstale":"2024-09-10T19:01:53.480175+0000","last_undegraded":"2024-09-10T19:01:53.480175+0000","last_fullsized":"2024-09-10T19:01:53.480175+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:58:59.582565+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:53.609885+0000","last_clean_scrub_stamp":"2024-09-10T18:58:59.582565+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046448920000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"232'35","reported_seq":469,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.479465+0000","last_change":"2024-09-10T19:00:43.819111+0000","last_active":"2024-09-10T19:01:53.479465+0000","last_peered":"2024-09-10T19:01:53.479465+0000","last_clean":"2024-09-10T19:01:53.479465+0000","last_became_active":"2024-09-10T18:57:02.436059+0000","last_became_peered":"2024-09-10T18:57:02.436059+0000","last_unstale":"2024-09-10T19:01:53.479465+0000","last_undegraded":"2024-09-10T19:01:53.479465+0000","last_fullsized":"2024-09-10T19:01:53.479465+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'35","last_scrub_stamp":"2024-09-10T19:00:43.819004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:00:43.819004+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:02:08.320454+0000","scrub_duration":0.0050234040000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":350,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.609306+0000","last_change":"2024-09-10T19:01:53.609239+0000","last_active":"2024-09-10T19:01:53.609306+0000","last_peered":"2024-09-10T19:01:53.609306+0000","last_clean":"2024-09-10T19:01:53.609306+0000","last_became_active":"2024-09-10T18:55:40.391849+0000","last_became_peered":"2024-09-10T18:55:40.391849+0000","last_unstale":"2024-09-10T19:01:53.609306+0000","last_undegraded":"2024-09-10T19:01:53.609306+0000","last_fullsized":"2024-09-10T19:01:53.609306+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:01:53.609184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:01:53.609184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:10.204226+0000","scrub_duration":0.024484227000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":321,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.483764+0000","last_change":"2024-09-10T19:00:42.738386+0000","last_active":"2024-09-10T19:01:53.483764+0000","last_peered":"2024-09-10T19:01:53.483764+0000","last_clean":"2024-09-10T19:01:53.483764+0000","last_became_active":"2024-09-10T19:00:42.514709+0000","last_became_peered":"2024-09-10T19:00:42.514709+0000","last_unstale":"2024-09-10T19:01:53.483764+0000","last_undegraded":"2024-09-10T19:01:53.483764+0000","last_fullsized":"2024-09-10T19:01:53.483764+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:42.738331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:00:42.738331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:02:08.495978+0000","scrub_duration":0.001211314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":157,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.588630+0000","last_change":"2024-09-10T18:57:03.015513+0000","last_active":"2024-09-10T19:01:34.588630+0000","last_peered":"2024-09-10T19:01:34.588630+0000","last_clean":"2024-09-10T19:01:34.588630+0000","last_became_active":"2024-09-10T18:57:02.468866+0000","last_became_peered":"2024-09-10T18:57:02.468866+0000","last_unstale":"2024-09-10T19:01:34.588630+0000","last_undegraded":"2024-09-10T19:01:34.588630+0000","last_fullsized":"2024-09-10T19:01:34.588630+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:57:03.015457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:57:03.015457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001545795,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.5","version":"361'12021","reported_seq":26436,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-09-10T19:01:58.725506+0000","last_change":"2024-09-10T19:01:29.582446+0000","last_active":"2024-09-10T19:01:58.725506+0000","last_peered":"2024-09-10T19:01:58.725506+0000","last_clean":"2024-09-10T19:01:58.725506+0000","last_became_active":"2024-09-10T19:01:29.559761+0000","last_became_peered":"2024-09-10T19:01:29.559761+0000","last_unstale":"2024-09-10T19:01:58.725506+0000","last_undegraded":"2024-09-10T19:01:58.725506+0000","last_fullsized":"2024-09-10T19:01:58.725506+0000","mapping_epoch":357,"log_start":"352'8300","ondisk_log_start":"352'8300","created":315,"last_epoch_clean":358,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3721,"log_dups_size":0,"ondisk_log_size":3721,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":335740928,"num_objects":5123,"num_object_clones":0,"num_object_copies":15369,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12021,"num_write_kb":548608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7168,"num_bytes_recovered":469762048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.e","version":"70'81","reported_seq":238,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.584076+0000","last_change":"2024-09-10T18:59:23.592755+0000","last_active":"2024-09-10T19:01:34.584076+0000","last_peered":"2024-09-10T19:01:34.584076+0000","last_clean":"2024-09-10T19:01:34.584076+0000","last_became_active":"2024-09-10T18:57:02.449115+0000","last_became_peered":"2024-09-10T18:57:02.449115+0000","last_unstale":"2024-09-10T19:01:34.584076+0000","last_undegraded":"2024-09-10T19:01:34.584076+0000","last_fullsized":"2024-09-10T19:01:34.584076+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:23.592672+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:59:23.592672+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038976470000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"232'19","reported_seq":377,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.484450+0000","last_change":"2024-09-10T19:00:40.748808+0000","last_active":"2024-09-10T19:01:53.484450+0000","last_peered":"2024-09-10T19:01:53.484450+0000","last_clean":"2024-09-10T19:01:53.484450+0000","last_became_active":"2024-09-10T18:55:40.395257+0000","last_became_peered":"2024-09-10T18:55:40.395257+0000","last_unstale":"2024-09-10T19:01:53.484450+0000","last_undegraded":"2024-09-10T19:01:53.484450+0000","last_fullsized":"2024-09-10T19:01:53.484450+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T19:00:40.748713+0000","last_deep_scrub":"130'13","last_deep_scrub_stamp":"2024-09-10T18:51:58.427534+0000","last_clean_scrub_stamp":"2024-09-10T19:00:40.748713+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:02:08.421967+0000","scrub_duration":0.0043426029999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":362,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.480834+0000","last_change":"2024-09-10T18:59:09.206400+0000","last_active":"2024-09-10T19:01:53.480834+0000","last_peered":"2024-09-10T19:01:53.480834+0000","last_clean":"2024-09-10T19:01:53.480834+0000","last_became_active":"2024-09-10T18:45:17.963057+0000","last_became_peered":"2024-09-10T18:45:17.963057+0000","last_unstale":"2024-09-10T19:01:53.480834+0000","last_undegraded":"2024-09-10T19:01:53.480834+0000","last_fullsized":"2024-09-10T19:01:53.480834+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:09.206317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:59:09.206317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.018986050000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":330,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.582673+0000","last_change":"2024-09-10T19:01:12.880399+0000","last_active":"2024-09-10T19:01:34.582673+0000","last_peered":"2024-09-10T19:01:34.582673+0000","last_clean":"2024-09-10T19:01:34.582673+0000","last_became_active":"2024-09-10T18:55:40.390771+0000","last_became_peered":"2024-09-10T18:55:40.390771+0000","last_unstale":"2024-09-10T19:01:34.582673+0000","last_undegraded":"2024-09-10T19:01:34.582673+0000","last_fullsized":"2024-09-10T19:01:34.582673+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:17.661832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_clean_scrub_stamp":"2024-09-10T18:56:17.661832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041460890000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":365,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-09-10T19:01:58.603469+0000","last_change":"2024-09-10T19:01:58.603370+0000","last_active":"2024-09-10T19:01:58.603469+0000","last_peered":"2024-09-10T19:01:58.603469+0000","last_clean":"2024-09-10T19:01:58.603469+0000","last_became_active":"2024-09-10T18:57:02.452387+0000","last_became_peered":"2024-09-10T18:57:02.452387+0000","last_unstale":"2024-09-10T19:01:58.603469+0000","last_undegraded":"2024-09-10T19:01:58.603469+0000","last_fullsized":"2024-09-10T19:01:58.603469+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:01:58.601096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:53.345184+0000","last_clean_scrub_stamp":"2024-09-10T19:01:58.601096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:02:59.732936+0000","scrub_duration":0.0026721589999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"361'11978","reported_seq":12032,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-09-10T19:01:56.588200+0000","last_change":"2024-09-10T18:59:42.715562+0000","last_active":"2024-09-10T19:01:56.588200+0000","last_peered":"2024-09-10T19:01:56.588200+0000","last_clean":"2024-09-10T19:01:56.588200+0000","last_became_active":"2024-09-10T18:59:42.715275+0000","last_became_peered":"2024-09-10T18:59:42.715275+0000","last_unstale":"2024-09-10T19:01:56.588200+0000","last_undegraded":"2024-09-10T19:01:56.588200+0000","last_fullsized":"2024-09-10T19:01:56.588200+0000","mapping_epoch":315,"log_start":"352'8344","ondisk_log_start":"352'8344","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3634,"log_dups_size":0,"ondisk_log_size":3634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":347471872,"num_objects":5302,"num_object_clones":0,"num_object_copies":15906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11978,"num_write_kb":552960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.d","version":"70'81","reported_seq":169,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.585458+0000","last_change":"2024-09-10T19:00:54.102663+0000","last_active":"2024-09-10T19:01:34.585458+0000","last_peered":"2024-09-10T19:01:34.585458+0000","last_clean":"2024-09-10T19:01:34.585458+0000","last_became_active":"2024-09-10T18:57:02.439652+0000","last_became_peered":"2024-09-10T18:57:02.439652+0000","last_unstale":"2024-09-10T19:01:34.585458+0000","last_undegraded":"2024-09-10T19:01:34.585458+0000","last_fullsized":"2024-09-10T19:01:34.585458+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:18.041116+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:56:18.041116+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014613810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"232'27","reported_seq":48,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.476929+0000","last_change":"2024-09-10T19:00:42.533625+0000","last_active":"2024-09-10T19:01:53.476929+0000","last_peered":"2024-09-10T19:01:53.476929+0000","last_clean":"2024-09-10T19:01:53.476929+0000","last_became_active":"2024-09-10T19:00:42.515903+0000","last_became_peered":"2024-09-10T19:00:42.515903+0000","last_unstale":"2024-09-10T19:01:53.476929+0000","last_undegraded":"2024-09-10T19:01:53.476929+0000","last_fullsized":"2024-09-10T19:01:53.476929+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'27","last_scrub_stamp":"2024-09-10T18:58:47.595256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:58:47.595256+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014176382,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":33,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":321,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.592037+0000","last_change":"2024-09-10T18:57:02.463092+0000","last_active":"2024-09-10T19:01:34.592037+0000","last_peered":"2024-09-10T19:01:34.592037+0000","last_clean":"2024-09-10T19:01:34.592037+0000","last_became_active":"2024-09-10T18:57:02.461708+0000","last_became_peered":"2024-09-10T18:57:02.461708+0000","last_unstale":"2024-09-10T19:01:34.592037+0000","last_undegraded":"2024-09-10T19:01:34.592037+0000","last_fullsized":"2024-09-10T19:01:34.592037+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:24.902205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:05.603125+0000","last_clean_scrub_stamp":"2024-09-10T18:56:24.902205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0076874830000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"273'10","reported_seq":244,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:56.271545+0000","last_change":"2024-09-10T18:59:11.568683+0000","last_active":"2024-09-10T19:01:56.271545+0000","last_peered":"2024-09-10T19:01:56.271545+0000","last_clean":"2024-09-10T19:01:56.271545+0000","last_became_active":"2024-09-10T18:57:02.441890+0000","last_became_peered":"2024-09-10T18:57:02.441890+0000","last_unstale":"2024-09-10T19:01:56.271545+0000","last_undegraded":"2024-09-10T19:01:56.271545+0000","last_fullsized":"2024-09-10T19:01:56.271545+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'10","last_scrub_stamp":"2024-09-10T18:59:11.568599+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-09-10T18:47:02.582229+0000","last_clean_scrub_stamp":"2024-09-10T18:59:11.568599+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0054658950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":383,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.477497+0000","last_change":"2024-09-10T18:59:51.487816+0000","last_active":"2024-09-10T19:01:53.477497+0000","last_peered":"2024-09-10T19:01:53.477497+0000","last_clean":"2024-09-10T19:01:53.477497+0000","last_became_active":"2024-09-10T18:55:40.389202+0000","last_became_peered":"2024-09-10T18:55:40.389202+0000","last_unstale":"2024-09-10T19:01:53.477497+0000","last_undegraded":"2024-09-10T19:01:53.477497+0000","last_fullsized":"2024-09-10T19:01:53.477497+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:51.487763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_clean_scrub_stamp":"2024-09-10T18:59:51.487763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001471904,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"361'12011","reported_seq":25943,"reported_epoch":361,"state":"active+clean","last_fresh":"2024-09-10T19:01:56.501041+0000","last_change":"2024-09-10T19:01:23.521153+0000","last_active":"2024-09-10T19:01:56.501041+0000","last_peered":"2024-09-10T19:01:56.501041+0000","last_clean":"2024-09-10T19:01:56.501041+0000","last_became_active":"2024-09-10T19:01:23.517785+0000","last_became_peered":"2024-09-10T19:01:23.517785+0000","last_unstale":"2024-09-10T19:01:56.501041+0000","last_undegraded":"2024-09-10T19:01:56.501041+0000","last_fullsized":"2024-09-10T19:01:56.501041+0000","mapping_epoch":353,"log_start":"352'8500","ondisk_log_start":"352'8500","created":315,"last_epoch_clean":354,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3511,"log_dups_size":0,"ondisk_log_size":3511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":351207424,"num_objects":5359,"num_object_clones":0,"num_object_copies":16077,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12011,"num_write_kb":555840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6925,"num_bytes_recovered":453836800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.c","version":"70'81","reported_seq":314,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.590928+0000","last_change":"2024-09-10T18:56:33.779212+0000","last_active":"2024-09-10T19:01:34.590928+0000","last_peered":"2024-09-10T19:01:34.590928+0000","last_clean":"2024-09-10T19:01:34.590928+0000","last_became_active":"2024-09-10T18:55:40.390655+0000","last_became_peered":"2024-09-10T18:55:40.390655+0000","last_unstale":"2024-09-10T19:01:34.590928+0000","last_undegraded":"2024-09-10T19:01:34.590928+0000","last_fullsized":"2024-09-10T19:01:34.590928+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:33.779154+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:56:33.779154+0000","last_clean_scrub_stamp":"2024-09-10T18:56:33.779154+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0075121179999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"232'18","reported_seq":402,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.948651+0000","last_change":"2024-09-10T19:01:53.948548+0000","last_active":"2024-09-10T19:01:53.948651+0000","last_peered":"2024-09-10T19:01:53.948651+0000","last_clean":"2024-09-10T19:01:53.948651+0000","last_became_active":"2024-09-10T18:45:20.056869+0000","last_became_peered":"2024-09-10T18:45:20.056869+0000","last_unstale":"2024-09-10T19:01:53.948651+0000","last_undegraded":"2024-09-10T19:01:53.948651+0000","last_fullsized":"2024-09-10T19:01:53.948651+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'18","last_scrub_stamp":"2024-09-10T19:01:53.948309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:01:53.948309+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:00.880791+0000","scrub_duration":0.010806517,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"56'2","reported_seq":402,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.584691+0000","last_change":"2024-09-10T19:01:01.185585+0000","last_active":"2024-09-10T19:01:34.584691+0000","last_peered":"2024-09-10T19:01:34.584691+0000","last_clean":"2024-09-10T19:01:34.584691+0000","last_became_active":"2024-09-10T18:57:02.434475+0000","last_became_peered":"2024-09-10T18:57:02.434475+0000","last_unstale":"2024-09-10T19:01:34.584691+0000","last_undegraded":"2024-09-10T19:01:34.584691+0000","last_fullsized":"2024-09-10T19:01:34.584691+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-10T18:56:01.630091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:01.630091+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001578238,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":321,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.485568+0000","last_change":"2024-09-10T19:00:42.515054+0000","last_active":"2024-09-10T19:01:53.485568+0000","last_peered":"2024-09-10T19:01:53.485568+0000","last_clean":"2024-09-10T19:01:53.485568+0000","last_became_active":"2024-09-10T19:00:42.513796+0000","last_became_peered":"2024-09-10T19:00:42.513796+0000","last_unstale":"2024-09-10T19:01:53.485568+0000","last_undegraded":"2024-09-10T19:01:53.485568+0000","last_fullsized":"2024-09-10T19:01:53.485568+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:54.899134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:02.711594+0000","last_clean_scrub_stamp":"2024-09-10T18:59:54.899134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051739739999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":364,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.589859+0000","last_change":"2024-09-10T18:59:11.827181+0000","last_active":"2024-09-10T19:01:34.589859+0000","last_peered":"2024-09-10T19:01:34.589859+0000","last_clean":"2024-09-10T19:01:34.589859+0000","last_became_active":"2024-09-10T18:57:02.459985+0000","last_became_peered":"2024-09-10T18:57:02.459985+0000","last_unstale":"2024-09-10T19:01:34.589859+0000","last_undegraded":"2024-09-10T19:01:34.589859+0000","last_fullsized":"2024-09-10T19:01:34.589859+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:11.827101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:53.352427+0000","last_clean_scrub_stamp":"2024-09-10T18:59:11.827101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034557899999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.13","version":"70'81","reported_seq":301,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:56.385160+0000","last_change":"2024-09-10T19:01:56.385098+0000","last_active":"2024-09-10T19:01:56.385160+0000","last_peered":"2024-09-10T19:01:56.385160+0000","last_clean":"2024-09-10T19:01:56.385160+0000","last_became_active":"2024-09-10T18:52:16.602463+0000","last_became_peered":"2024-09-10T18:52:16.602463+0000","last_unstale":"2024-09-10T19:01:56.385160+0000","last_undegraded":"2024-09-10T19:01:56.385160+0000","last_fullsized":"2024-09-10T19:01:56.385160+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:01:56.384747+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:01:56.384747+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:22.562132+0000","scrub_duration":0.0049177420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"232'19","reported_seq":390,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.484909+0000","last_change":"2024-09-10T18:57:02.485836+0000","last_active":"2024-09-10T19:01:53.484909+0000","last_peered":"2024-09-10T19:01:53.484909+0000","last_clean":"2024-09-10T19:01:53.484909+0000","last_became_active":"2024-09-10T18:57:02.453001+0000","last_became_peered":"2024-09-10T18:57:02.453001+0000","last_unstale":"2024-09-10T19:01:53.484909+0000","last_undegraded":"2024-09-10T19:01:53.484909+0000","last_fullsized":"2024-09-10T19:01:53.484909+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T18:56:25.122070+0000","last_deep_scrub":"76'13","last_deep_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_clean_scrub_stamp":"2024-09-10T18:56:25.122070+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0061142590000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":350,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.475654+0000","last_change":"2024-09-10T18:59:59.385405+0000","last_active":"2024-09-10T19:01:53.475654+0000","last_peered":"2024-09-10T19:01:53.475654+0000","last_clean":"2024-09-10T19:01:53.475654+0000","last_became_active":"2024-09-10T18:55:40.401037+0000","last_became_peered":"2024-09-10T18:55:40.401037+0000","last_unstale":"2024-09-10T19:01:53.475654+0000","last_undegraded":"2024-09-10T19:01:53.475654+0000","last_fullsized":"2024-09-10T19:01:53.475654+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:59.385349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:59:59.385349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00163496,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":366,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.592073+0000","last_change":"2024-09-10T18:57:02.455638+0000","last_active":"2024-09-10T19:01:34.592073+0000","last_peered":"2024-09-10T19:01:34.592073+0000","last_clean":"2024-09-10T19:01:34.592073+0000","last_became_active":"2024-09-10T18:57:02.455341+0000","last_became_peered":"2024-09-10T18:57:02.455341+0000","last_unstale":"2024-09-10T19:01:34.592073+0000","last_undegraded":"2024-09-10T19:01:34.592073+0000","last_fullsized":"2024-09-10T19:01:34.592073+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:25.862853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:00.754912+0000","last_clean_scrub_stamp":"2024-09-10T18:56:25.862853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0033952399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"63'1","reported_seq":388,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.583291+0000","last_change":"2024-09-10T19:01:08.024679+0000","last_active":"2024-09-10T19:01:34.583291+0000","last_peered":"2024-09-10T19:01:34.583291+0000","last_clean":"2024-09-10T19:01:34.583291+0000","last_became_active":"2024-09-10T18:59:48.741275+0000","last_became_peered":"2024-09-10T18:59:48.741275+0000","last_unstale":"2024-09-10T19:01:34.583291+0000","last_undegraded":"2024-09-10T19:01:34.583291+0000","last_fullsized":"2024-09-10T19:01:34.583291+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'1","last_scrub_stamp":"2024-09-10T18:56:07.774334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:07.774334+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.090386770000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":52,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.12","version":"70'81","reported_seq":230,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.477333+0000","last_change":"2024-09-10T18:58:52.543604+0000","last_active":"2024-09-10T19:01:53.477333+0000","last_peered":"2024-09-10T19:01:53.477333+0000","last_clean":"2024-09-10T19:01:53.477333+0000","last_became_active":"2024-09-10T18:52:16.598921+0000","last_became_peered":"2024-09-10T18:52:16.598921+0000","last_unstale":"2024-09-10T19:01:53.477333+0000","last_undegraded":"2024-09-10T19:01:53.477333+0000","last_fullsized":"2024-09-10T19:01:53.477333+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:58:52.543544+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:54.127476+0000","last_clean_scrub_stamp":"2024-09-10T18:58:52.543544+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003920644,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":396,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.479493+0000","last_change":"2024-09-10T18:59:05.556377+0000","last_active":"2024-09-10T19:01:53.479493+0000","last_peered":"2024-09-10T19:01:53.479493+0000","last_clean":"2024-09-10T19:01:53.479493+0000","last_became_active":"2024-09-10T18:57:02.442251+0000","last_became_peered":"2024-09-10T18:57:02.442251+0000","last_unstale":"2024-09-10T19:01:53.479493+0000","last_undegraded":"2024-09-10T19:01:53.479493+0000","last_fullsized":"2024-09-10T19:01:53.479493+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:05.556292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:59:05.556292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026781280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"232'12","reported_seq":397,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.592253+0000","last_change":"2024-09-10T18:59:52.198399+0000","last_active":"2024-09-10T19:01:34.592253+0000","last_peered":"2024-09-10T19:01:34.592253+0000","last_clean":"2024-09-10T19:01:34.592253+0000","last_became_active":"2024-09-10T18:57:02.460430+0000","last_became_peered":"2024-09-10T18:57:02.460430+0000","last_unstale":"2024-09-10T19:01:34.592253+0000","last_undegraded":"2024-09-10T19:01:34.592253+0000","last_fullsized":"2024-09-10T19:01:34.592253+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'12","last_scrub_stamp":"2024-09-10T18:59:52.198340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:59:52.198340+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001790591,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":358,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.478369+0000","last_change":"2024-09-10T18:56:32.602499+0000","last_active":"2024-09-10T19:01:53.478369+0000","last_peered":"2024-09-10T19:01:53.478369+0000","last_clean":"2024-09-10T19:01:53.478369+0000","last_became_active":"2024-09-10T18:45:22.069653+0000","last_became_peered":"2024-09-10T18:45:22.069653+0000","last_unstale":"2024-09-10T19:01:53.478369+0000","last_undegraded":"2024-09-10T19:01:53.478369+0000","last_fullsized":"2024-09-10T19:01:53.478369+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:32.602448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:32.602448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0052503430000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":123,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.480078+0000","last_change":"2024-09-10T19:00:42.519802+0000","last_active":"2024-09-10T19:01:53.480078+0000","last_peered":"2024-09-10T19:01:53.480078+0000","last_clean":"2024-09-10T19:01:53.480078+0000","last_became_active":"2024-09-10T19:00:42.519492+0000","last_became_peered":"2024-09-10T19:00:42.519492+0000","last_unstale":"2024-09-10T19:01:53.480078+0000","last_undegraded":"2024-09-10T19:01:53.480078+0000","last_fullsized":"2024-09-10T19:01:53.480078+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:53.608029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:58:53.608029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058174170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.11","version":"70'81","reported_seq":265,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.484169+0000","last_change":"2024-09-10T19:00:42.517609+0000","last_active":"2024-09-10T19:01:53.484169+0000","last_peered":"2024-09-10T19:01:53.484169+0000","last_clean":"2024-09-10T19:01:53.484169+0000","last_became_active":"2024-09-10T19:00:42.517311+0000","last_became_peered":"2024-09-10T19:00:42.517311+0000","last_unstale":"2024-09-10T19:01:53.484169+0000","last_undegraded":"2024-09-10T19:01:53.484169+0000","last_fullsized":"2024-09-10T19:01:53.484169+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:58:48.552631+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:58:48.552631+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012534919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"232'20","reported_seq":454,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.590660+0000","last_change":"2024-09-10T19:00:07.147063+0000","last_active":"2024-09-10T19:01:34.590660+0000","last_peered":"2024-09-10T19:01:34.590660+0000","last_clean":"2024-09-10T19:01:34.590660+0000","last_became_active":"2024-09-10T18:57:02.444437+0000","last_became_peered":"2024-09-10T18:57:02.444437+0000","last_unstale":"2024-09-10T19:01:34.590660+0000","last_undegraded":"2024-09-10T19:01:34.590660+0000","last_fullsized":"2024-09-10T19:01:34.590660+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'20","last_scrub_stamp":"2024-09-10T19:00:07.146928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:00:07.146928+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0071404040000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":382,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.477088+0000","last_change":"2024-09-10T18:58:50.541776+0000","last_active":"2024-09-10T19:01:53.477088+0000","last_peered":"2024-09-10T19:01:53.477088+0000","last_clean":"2024-09-10T19:01:53.477088+0000","last_became_active":"2024-09-10T18:57:02.445710+0000","last_became_peered":"2024-09-10T18:57:02.445710+0000","last_unstale":"2024-09-10T19:01:53.477088+0000","last_undegraded":"2024-09-10T19:01:53.477088+0000","last_fullsized":"2024-09-10T19:01:53.477088+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:50.541691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:58:50.541691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0089012770000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"304'11","reported_seq":610,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:56.271160+0000","last_change":"2024-09-10T18:59:55.525278+0000","last_active":"2024-09-10T19:01:56.271160+0000","last_peered":"2024-09-10T19:01:56.271160+0000","last_clean":"2024-09-10T19:01:56.271160+0000","last_became_active":"2024-09-10T18:58:58.489370+0000","last_became_peered":"2024-09-10T18:58:58.489370+0000","last_unstale":"2024-09-10T19:01:56.271160+0000","last_undegraded":"2024-09-10T19:01:56.271160+0000","last_fullsized":"2024-09-10T19:01:56.271160+0000","mapping_epoch":303,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":304,"parent":"0.0","parent_split_bits":0,"last_scrub":"304'11","last_scrub_stamp":"2024-09-10T18:59:55.525185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:59:55.525185+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020778110000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":50,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.483947+0000","last_change":"2024-09-10T18:59:48.745646+0000","last_active":"2024-09-10T19:01:53.483947+0000","last_peered":"2024-09-10T19:01:53.483947+0000","last_clean":"2024-09-10T19:01:53.483947+0000","last_became_active":"2024-09-10T18:59:48.744923+0000","last_became_peered":"2024-09-10T18:59:48.744923+0000","last_unstale":"2024-09-10T19:01:53.483947+0000","last_undegraded":"2024-09-10T19:01:53.483947+0000","last_fullsized":"2024-09-10T19:01:53.483947+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:57.812395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:57.476724+0000","last_clean_scrub_stamp":"2024-09-10T18:58:57.812395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.10","version":"70'81","reported_seq":240,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.477110+0000","last_change":"2024-09-10T19:00:04.612412+0000","last_active":"2024-09-10T19:01:53.477110+0000","last_peered":"2024-09-10T19:01:53.477110+0000","last_clean":"2024-09-10T19:01:53.477110+0000","last_became_active":"2024-09-10T18:57:02.443782+0000","last_became_peered":"2024-09-10T18:57:02.443782+0000","last_unstale":"2024-09-10T19:01:53.477110+0000","last_undegraded":"2024-09-10T19:01:53.477110+0000","last_fullsized":"2024-09-10T19:01:53.477110+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:00:04.612324+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:00:04.612324+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025961159999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"232'15","reported_seq":42,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.477481+0000","last_change":"2024-09-10T19:00:42.530883+0000","last_active":"2024-09-10T19:01:53.477481+0000","last_peered":"2024-09-10T19:01:53.477481+0000","last_clean":"2024-09-10T19:01:53.477481+0000","last_became_active":"2024-09-10T19:00:42.518672+0000","last_became_peered":"2024-09-10T19:00:42.518672+0000","last_unstale":"2024-09-10T19:01:53.477481+0000","last_undegraded":"2024-09-10T19:01:53.477481+0000","last_fullsized":"2024-09-10T19:01:53.477481+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T18:59:16.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:59:16.949954+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":339,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.483903+0000","last_change":"2024-09-10T18:56:23.096461+0000","last_active":"2024-09-10T19:01:53.483903+0000","last_peered":"2024-09-10T19:01:53.483903+0000","last_clean":"2024-09-10T19:01:53.483903+0000","last_became_active":"2024-09-10T18:55:40.390697+0000","last_became_peered":"2024-09-10T18:55:40.390697+0000","last_unstale":"2024-09-10T19:01:53.483903+0000","last_undegraded":"2024-09-10T19:01:53.483903+0000","last_fullsized":"2024-09-10T19:01:53.483903+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:23.096373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:23.096373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0057615890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"273'11","reported_seq":592,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:51.272044+0000","last_change":"2024-09-10T18:59:53.158730+0000","last_active":"2024-09-10T19:01:51.272044+0000","last_peered":"2024-09-10T19:01:51.272044+0000","last_clean":"2024-09-10T19:01:51.272044+0000","last_became_active":"2024-09-10T18:57:02.445268+0000","last_became_peered":"2024-09-10T18:57:02.445268+0000","last_unstale":"2024-09-10T19:01:51.272044+0000","last_undegraded":"2024-09-10T19:01:51.272044+0000","last_fullsized":"2024-09-10T19:01:51.272044+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'11","last_scrub_stamp":"2024-09-10T18:59:53.158664+0000","last_deep_scrub":"81'9","last_deep_scrub_stamp":"2024-09-10T18:46:54.800790+0000","last_clean_scrub_stamp":"2024-09-10T18:59:53.158664+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017427930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":380,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.584356+0000","last_change":"2024-09-10T18:56:20.584899+0000","last_active":"2024-09-10T19:01:34.584356+0000","last_peered":"2024-09-10T19:01:34.584356+0000","last_clean":"2024-09-10T19:01:34.584356+0000","last_became_active":"2024-09-10T18:55:40.391662+0000","last_became_peered":"2024-09-10T18:55:40.391662+0000","last_unstale":"2024-09-10T19:01:34.584356+0000","last_undegraded":"2024-09-10T19:01:34.584356+0000","last_fullsized":"2024-09-10T19:01:34.584356+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:20.584808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_clean_scrub_stamp":"2024-09-10T18:56:20.584808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0035961249999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"232'15","reported_seq":434,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.481455+0000","last_change":"2024-09-10T18:59:53.320831+0000","last_active":"2024-09-10T19:01:53.481455+0000","last_peered":"2024-09-10T19:01:53.481455+0000","last_clean":"2024-09-10T19:01:53.481455+0000","last_became_active":"2024-09-10T18:55:40.400788+0000","last_became_peered":"2024-09-10T18:55:40.400788+0000","last_unstale":"2024-09-10T19:01:53.481455+0000","last_undegraded":"2024-09-10T19:01:53.481455+0000","last_fullsized":"2024-09-10T19:01:53.481455+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T18:59:53.320757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:59:53.320757+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001889578,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":413,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.479791+0000","last_change":"2024-09-10T19:00:42.518606+0000","last_active":"2024-09-10T19:01:53.479791+0000","last_peered":"2024-09-10T19:01:53.479791+0000","last_clean":"2024-09-10T19:01:53.479791+0000","last_became_active":"2024-09-10T19:00:42.518310+0000","last_became_peered":"2024-09-10T19:00:42.518310+0000","last_unstale":"2024-09-10T19:01:53.479791+0000","last_undegraded":"2024-09-10T19:01:53.479791+0000","last_fullsized":"2024-09-10T19:01:53.479791+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:48.622701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:59:48.622701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001531279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":363,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.590429+0000","last_change":"2024-09-10T18:56:04.753218+0000","last_active":"2024-09-10T19:01:34.590429+0000","last_peered":"2024-09-10T19:01:34.590429+0000","last_clean":"2024-09-10T19:01:34.590429+0000","last_became_active":"2024-09-10T18:55:40.398243+0000","last_became_peered":"2024-09-10T18:55:40.398243+0000","last_unstale":"2024-09-10T19:01:34.590429+0000","last_undegraded":"2024-09-10T19:01:34.590429+0000","last_fullsized":"2024-09-10T19:01:34.590429+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:04.753133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:03.571786+0000","last_clean_scrub_stamp":"2024-09-10T18:56:04.753133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0069575230000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":354,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.485735+0000","last_change":"2024-09-10T19:00:42.517459+0000","last_active":"2024-09-10T19:01:53.485735+0000","last_peered":"2024-09-10T19:01:53.485735+0000","last_clean":"2024-09-10T19:01:53.485735+0000","last_became_active":"2024-09-10T19:00:42.516833+0000","last_became_peered":"2024-09-10T19:00:42.516833+0000","last_unstale":"2024-09-10T19:01:53.485735+0000","last_undegraded":"2024-09-10T19:01:53.485735+0000","last_fullsized":"2024-09-10T19:01:53.485735+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:46.863758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:04.631558+0000","last_clean_scrub_stamp":"2024-09-10T18:59:46.863758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0031907379999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"232'24","reported_seq":393,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.582754+0000","last_change":"2024-09-10T18:58:58.698097+0000","last_active":"2024-09-10T19:01:34.582754+0000","last_peered":"2024-09-10T19:01:34.582754+0000","last_clean":"2024-09-10T19:01:34.582754+0000","last_became_active":"2024-09-10T18:45:20.056885+0000","last_became_peered":"2024-09-10T18:45:20.056885+0000","last_unstale":"2024-09-10T19:01:34.582754+0000","last_undegraded":"2024-09-10T19:01:34.582754+0000","last_fullsized":"2024-09-10T19:01:34.582754+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T18:58:58.698003+0000","last_deep_scrub":"223'16","last_deep_scrub_stamp":"2024-09-10T18:54:47.324044+0000","last_clean_scrub_stamp":"2024-09-10T18:58:58.698003+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.006955676,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":382,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.480763+0000","last_change":"2024-09-10T19:01:07.432016+0000","last_active":"2024-09-10T19:01:53.480763+0000","last_peered":"2024-09-10T19:01:53.480763+0000","last_clean":"2024-09-10T19:01:53.480763+0000","last_became_active":"2024-09-10T18:55:40.399198+0000","last_became_peered":"2024-09-10T18:55:40.399198+0000","last_unstale":"2024-09-10T19:01:53.480763+0000","last_undegraded":"2024-09-10T19:01:53.480763+0000","last_fullsized":"2024-09-10T19:01:53.480763+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:51.138240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:00.029389+0000","last_clean_scrub_stamp":"2024-09-10T18:58:51.138240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0086738059999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":364,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.481467+0000","last_change":"2024-09-10T19:00:59.501368+0000","last_active":"2024-09-10T19:01:53.481467+0000","last_peered":"2024-09-10T19:01:53.481467+0000","last_clean":"2024-09-10T19:01:53.481467+0000","last_became_active":"2024-09-10T18:45:22.070065+0000","last_became_peered":"2024-09-10T18:45:22.070065+0000","last_unstale":"2024-09-10T19:01:53.481467+0000","last_undegraded":"2024-09-10T19:01:53.481467+0000","last_fullsized":"2024-09-10T19:01:53.481467+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:31.782918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:31.782918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043032130000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":361,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.589274+0000","last_change":"2024-09-10T18:56:06.792096+0000","last_active":"2024-09-10T19:01:34.589274+0000","last_peered":"2024-09-10T19:01:34.589274+0000","last_clean":"2024-09-10T19:01:34.589274+0000","last_became_active":"2024-09-10T18:55:40.395595+0000","last_became_peered":"2024-09-10T18:55:40.395595+0000","last_unstale":"2024-09-10T19:01:34.589274+0000","last_undegraded":"2024-09-10T19:01:34.589274+0000","last_fullsized":"2024-09-10T19:01:34.589274+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:06.791860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:06.791860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0046852380000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"232'8","reported_seq":370,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.588793+0000","last_change":"2024-09-10T18:56:30.802774+0000","last_active":"2024-09-10T19:01:34.588793+0000","last_peered":"2024-09-10T19:01:34.588793+0000","last_clean":"2024-09-10T19:01:34.588793+0000","last_became_active":"2024-09-10T18:45:20.059205+0000","last_became_peered":"2024-09-10T18:45:20.059205+0000","last_unstale":"2024-09-10T19:01:34.588793+0000","last_undegraded":"2024-09-10T19:01:34.588793+0000","last_fullsized":"2024-09-10T19:01:34.588793+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'8","last_scrub_stamp":"2024-09-10T18:56:30.802679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:30.802679+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034579609999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":378,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.478404+0000","last_change":"2024-09-10T18:58:53.527829+0000","last_active":"2024-09-10T19:01:53.478404+0000","last_peered":"2024-09-10T19:01:53.478404+0000","last_clean":"2024-09-10T19:01:53.478404+0000","last_became_active":"2024-09-10T18:47:36.415866+0000","last_became_peered":"2024-09-10T18:47:36.415866+0000","last_unstale":"2024-09-10T19:01:53.478404+0000","last_undegraded":"2024-09-10T19:01:53.478404+0000","last_fullsized":"2024-09-10T19:01:53.478404+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:53.527746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:58:53.527746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058191349999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":389,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.479268+0000","last_change":"2024-09-10T18:57:02.454548+0000","last_active":"2024-09-10T19:01:53.479268+0000","last_peered":"2024-09-10T19:01:53.479268+0000","last_clean":"2024-09-10T19:01:53.479268+0000","last_became_active":"2024-09-10T18:57:02.453974+0000","last_became_peered":"2024-09-10T18:57:02.453974+0000","last_unstale":"2024-09-10T19:01:53.479268+0000","last_undegraded":"2024-09-10T19:01:53.479268+0000","last_fullsized":"2024-09-10T19:01:53.479268+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:23.195060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_clean_scrub_stamp":"2024-09-10T18:56:23.195060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047613259999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":158,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.582455+0000","last_change":"2024-09-10T18:59:55.812370+0000","last_active":"2024-09-10T19:01:34.582455+0000","last_peered":"2024-09-10T19:01:34.582455+0000","last_clean":"2024-09-10T19:01:34.582455+0000","last_became_active":"2024-09-10T18:57:02.447566+0000","last_became_peered":"2024-09-10T18:57:02.447566+0000","last_unstale":"2024-09-10T19:01:34.582455+0000","last_undegraded":"2024-09-10T19:01:34.582455+0000","last_fullsized":"2024-09-10T19:01:34.582455+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:55.812312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:55.145638+0000","last_clean_scrub_stamp":"2024-09-10T18:59:55.812312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0055522669999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.14","version":"70'81","reported_seq":262,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.583494+0000","last_change":"2024-09-10T19:00:58.100610+0000","last_active":"2024-09-10T19:01:34.583494+0000","last_peered":"2024-09-10T19:01:34.583494+0000","last_clean":"2024-09-10T19:01:34.583494+0000","last_became_active":"2024-09-10T18:57:02.442457+0000","last_became_peered":"2024-09-10T18:57:02.442457+0000","last_unstale":"2024-09-10T19:01:34.583494+0000","last_undegraded":"2024-09-10T19:01:34.583494+0000","last_fullsized":"2024-09-10T19:01:34.583494+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:55:34.791223+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:57.184383+0000","last_clean_scrub_stamp":"2024-09-10T18:55:34.791223+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016709736999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17890,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":7,"num_write":3,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":35780,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.11","version":"232'24","reported_seq":416,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.589131+0000","last_change":"2024-09-10T18:59:10.879686+0000","last_active":"2024-09-10T19:01:34.589131+0000","last_peered":"2024-09-10T19:01:34.589131+0000","last_clean":"2024-09-10T19:01:34.589131+0000","last_became_active":"2024-09-10T18:57:02.461909+0000","last_became_peered":"2024-09-10T18:57:02.461909+0000","last_unstale":"2024-09-10T19:01:34.589131+0000","last_undegraded":"2024-09-10T19:01:34.589131+0000","last_fullsized":"2024-09-10T19:01:34.589131+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T18:59:10.879597+0000","last_deep_scrub":"79'16","last_deep_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_clean_scrub_stamp":"2024-09-10T18:59:10.879597+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0090530790000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"56'1","reported_seq":394,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.480084+0000","last_change":"2024-09-10T18:56:25.211179+0000","last_active":"2024-09-10T19:01:53.480084+0000","last_peered":"2024-09-10T19:01:53.480084+0000","last_clean":"2024-09-10T19:01:53.480084+0000","last_became_active":"2024-09-10T18:47:36.421985+0000","last_became_peered":"2024-09-10T18:47:36.421985+0000","last_unstale":"2024-09-10T19:01:53.480084+0000","last_undegraded":"2024-09-10T19:01:53.480084+0000","last_fullsized":"2024-09-10T19:01:53.480084+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T18:56:25.211095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:56:25.211095+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025468880000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":105,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.583419+0000","last_change":"2024-09-10T19:01:07.015846+0000","last_active":"2024-09-10T19:01:34.583419+0000","last_peered":"2024-09-10T19:01:34.583419+0000","last_clean":"2024-09-10T19:01:34.583419+0000","last_became_active":"2024-09-10T18:57:02.435124+0000","last_became_peered":"2024-09-10T18:57:02.435124+0000","last_unstale":"2024-09-10T19:01:34.583419+0000","last_undegraded":"2024-09-10T19:01:34.583419+0000","last_fullsized":"2024-09-10T19:01:34.583419+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:12.852769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:12.852769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":371,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.589574+0000","last_change":"2024-09-10T18:56:09.903930+0000","last_active":"2024-09-10T19:01:34.589574+0000","last_peered":"2024-09-10T19:01:34.589574+0000","last_clean":"2024-09-10T19:01:34.589574+0000","last_became_active":"2024-09-10T18:55:40.397523+0000","last_became_peered":"2024-09-10T18:55:40.397523+0000","last_unstale":"2024-09-10T19:01:34.589574+0000","last_undegraded":"2024-09-10T19:01:34.589574+0000","last_fullsized":"2024-09-10T19:01:34.589574+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:09.903846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:56:09.903846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011721984,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"78'2","reported_seq":149,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.476812+0000","last_change":"2024-09-10T18:58:48.472078+0000","last_active":"2024-09-10T19:01:53.476812+0000","last_peered":"2024-09-10T19:01:53.476812+0000","last_clean":"2024-09-10T19:01:53.476812+0000","last_became_active":"2024-09-10T18:57:02.447198+0000","last_became_peered":"2024-09-10T18:57:02.447198+0000","last_unstale":"2024-09-10T19:01:53.476812+0000","last_undegraded":"2024-09-10T19:01:53.476812+0000","last_fullsized":"2024-09-10T19:01:53.476812+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'2","last_scrub_stamp":"2024-09-10T18:58:48.472024+0000","last_deep_scrub":"78'2","last_deep_scrub_stamp":"2024-09-10T18:46:37.581130+0000","last_clean_scrub_stamp":"2024-09-10T18:58:48.472024+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.022088824,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":382,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.477195+0000","last_change":"2024-09-10T18:59:53.505658+0000","last_active":"2024-09-10T19:01:53.477195+0000","last_peered":"2024-09-10T19:01:53.477195+0000","last_clean":"2024-09-10T19:01:53.477195+0000","last_became_active":"2024-09-10T18:57:02.443353+0000","last_became_peered":"2024-09-10T18:57:02.443353+0000","last_unstale":"2024-09-10T19:01:53.477195+0000","last_undegraded":"2024-09-10T19:01:53.477195+0000","last_fullsized":"2024-09-10T19:01:53.477195+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:53.505561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:59:53.505561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001238605,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"232'20","reported_seq":388,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.484393+0000","last_change":"2024-09-10T18:57:02.486956+0000","last_active":"2024-09-10T19:01:53.484393+0000","last_peered":"2024-09-10T19:01:53.484393+0000","last_clean":"2024-09-10T19:01:53.484393+0000","last_became_active":"2024-09-10T18:57:02.452473+0000","last_became_peered":"2024-09-10T18:57:02.452473+0000","last_unstale":"2024-09-10T19:01:53.484393+0000","last_undegraded":"2024-09-10T19:01:53.484393+0000","last_fullsized":"2024-09-10T19:01:53.484393+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'20","last_scrub_stamp":"2024-09-10T18:56:27.158686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:27.158686+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0090371180000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":23,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"273'10","reported_seq":605,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:56.270463+0000","last_change":"2024-09-10T18:59:22.516714+0000","last_active":"2024-09-10T19:01:56.270463+0000","last_peered":"2024-09-10T19:01:56.270463+0000","last_clean":"2024-09-10T19:01:56.270463+0000","last_became_active":"2024-09-10T18:57:02.441763+0000","last_became_peered":"2024-09-10T18:57:02.441763+0000","last_unstale":"2024-09-10T19:01:56.270463+0000","last_undegraded":"2024-09-10T19:01:56.270463+0000","last_fullsized":"2024-09-10T19:01:56.270463+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'10","last_scrub_stamp":"2024-09-10T18:59:22.516625+0000","last_deep_scrub":"190'9","last_deep_scrub_stamp":"2024-09-10T18:53:07.746309+0000","last_clean_scrub_stamp":"2024-09-10T18:59:22.516625+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021912820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":359,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.480037+0000","last_change":"2024-09-10T18:58:59.150773+0000","last_active":"2024-09-10T19:01:53.480037+0000","last_peered":"2024-09-10T19:01:53.480037+0000","last_clean":"2024-09-10T19:01:53.480037+0000","last_became_active":"2024-09-10T18:45:24.082318+0000","last_became_peered":"2024-09-10T18:45:24.082318+0000","last_unstale":"2024-09-10T19:01:53.480037+0000","last_undegraded":"2024-09-10T19:01:53.480037+0000","last_fullsized":"2024-09-10T19:01:53.480037+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:59.148958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:03.475299+0000","last_clean_scrub_stamp":"2024-09-10T18:58:59.148958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041797229999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":377,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.588659+0000","last_change":"2024-09-10T19:00:10.182129+0000","last_active":"2024-09-10T19:01:34.588659+0000","last_peered":"2024-09-10T19:01:34.588659+0000","last_clean":"2024-09-10T19:01:34.588659+0000","last_became_active":"2024-09-10T18:55:40.392524+0000","last_became_peered":"2024-09-10T18:55:40.392524+0000","last_unstale":"2024-09-10T19:01:34.588659+0000","last_undegraded":"2024-09-10T19:01:34.588659+0000","last_fullsized":"2024-09-10T19:01:34.588659+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:10.181982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:00:10.181982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029510420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"232'19","reported_seq":405,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.480084+0000","last_change":"2024-09-10T18:59:09.544279+0000","last_active":"2024-09-10T19:01:53.480084+0000","last_peered":"2024-09-10T19:01:53.480084+0000","last_clean":"2024-09-10T19:01:53.480084+0000","last_became_active":"2024-09-10T18:45:20.058717+0000","last_became_peered":"2024-09-10T18:45:20.058717+0000","last_unstale":"2024-09-10T19:01:53.480084+0000","last_undegraded":"2024-09-10T19:01:53.480084+0000","last_fullsized":"2024-09-10T19:01:53.480084+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T18:59:09.544194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:59:09.544194+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043401830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":342,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.485001+0000","last_change":"2024-09-10T18:59:57.862722+0000","last_active":"2024-09-10T19:01:53.485001+0000","last_peered":"2024-09-10T19:01:53.485001+0000","last_clean":"2024-09-10T19:01:53.485001+0000","last_became_active":"2024-09-10T18:57:02.453464+0000","last_became_peered":"2024-09-10T18:57:02.453464+0000","last_unstale":"2024-09-10T19:01:53.485001+0000","last_undegraded":"2024-09-10T19:01:53.485001+0000","last_fullsized":"2024-09-10T19:01:53.485001+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:57.862666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:59:57.862666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001262242,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":353,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.584991+0000","last_change":"2024-09-10T18:58:55.726863+0000","last_active":"2024-09-10T19:01:34.584991+0000","last_peered":"2024-09-10T19:01:34.584991+0000","last_clean":"2024-09-10T19:01:34.584991+0000","last_became_active":"2024-09-10T18:45:24.267251+0000","last_became_peered":"2024-09-10T18:45:24.267251+0000","last_unstale":"2024-09-10T19:01:34.584991+0000","last_undegraded":"2024-09-10T19:01:34.584991+0000","last_fullsized":"2024-09-10T19:01:34.584991+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:55.726799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:58:55.726799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001154721,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"306'2","reported_seq":400,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.479709+0000","last_change":"2024-09-10T18:59:20.534927+0000","last_active":"2024-09-10T19:01:53.479709+0000","last_peered":"2024-09-10T19:01:53.479709+0000","last_clean":"2024-09-10T19:01:53.479709+0000","last_became_active":"2024-09-10T18:57:02.435758+0000","last_became_peered":"2024-09-10T18:57:02.435758+0000","last_unstale":"2024-09-10T19:01:53.479709+0000","last_undegraded":"2024-09-10T19:01:53.479709+0000","last_fullsized":"2024-09-10T19:01:53.479709+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"306'2","last_scrub_stamp":"2024-09-10T18:59:20.534869+0000","last_deep_scrub":"306'2","last_deep_scrub_stamp":"2024-09-10T18:59:20.534869+0000","last_clean_scrub_stamp":"2024-09-10T18:59:20.534869+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0072446200000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"258'34","reported_seq":478,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.483491+0000","last_change":"2024-09-10T19:00:52.459800+0000","last_active":"2024-09-10T19:01:53.483491+0000","last_peered":"2024-09-10T19:01:53.483491+0000","last_clean":"2024-09-10T19:01:53.483491+0000","last_became_active":"2024-09-10T18:59:48.744293+0000","last_became_peered":"2024-09-10T18:59:48.744293+0000","last_unstale":"2024-09-10T19:01:53.483491+0000","last_undegraded":"2024-09-10T19:01:53.483491+0000","last_fullsized":"2024-09-10T19:01:53.483491+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'34","last_scrub_stamp":"2024-09-10T18:56:23.838951+0000","last_deep_scrub":"258'34","last_deep_scrub_stamp":"2024-09-10T18:56:23.838951+0000","last_clean_scrub_stamp":"2024-09-10T18:56:23.838951+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.035929888,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":368,"reported_epoch":360,"state":"active+clean","last_fresh":"2024-09-10T19:01:53.480201+0000","last_change":"2024-09-10T19:00:54.441149+0000","last_active":"2024-09-10T19:01:53.480201+0000","last_peered":"2024-09-10T19:01:53.480201+0000","last_clean":"2024-09-10T19:01:53.480201+0000","last_became_active":"2024-09-10T18:55:40.397870+0000","last_became_peered":"2024-09-10T18:55:40.397870+0000","last_unstale":"2024-09-10T19:01:53.480201+0000","last_undegraded":"2024-09-10T19:01:53.480201+0000","last_fullsized":"2024-09-10T19:01:53.480201+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:13.610976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:13.610976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010999553,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":374,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.591894+0000","last_change":"2024-09-10T19:00:42.520936+0000","last_active":"2024-09-10T19:01:34.591894+0000","last_peered":"2024-09-10T19:01:34.591894+0000","last_clean":"2024-09-10T19:01:34.591894+0000","last_became_active":"2024-09-10T19:00:42.520239+0000","last_became_peered":"2024-09-10T19:00:42.520239+0000","last_unstale":"2024-09-10T19:01:34.591894+0000","last_undegraded":"2024-09-10T19:01:34.591894+0000","last_fullsized":"2024-09-10T19:01:34.591894+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:17.985545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:17.985545+0000","last_clean_scrub_stamp":"2024-09-10T18:59:17.985545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0086180830000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":380,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.590615+0000","last_change":"2024-09-10T18:58:57.919890+0000","last_active":"2024-09-10T19:01:34.590615+0000","last_peered":"2024-09-10T19:01:34.590615+0000","last_clean":"2024-09-10T19:01:34.590615+0000","last_became_active":"2024-09-10T18:57:02.460855+0000","last_became_peered":"2024-09-10T18:57:02.460855+0000","last_unstale":"2024-09-10T19:01:34.590615+0000","last_undegraded":"2024-09-10T19:01:34.590615+0000","last_fullsized":"2024-09-10T19:01:34.590615+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:57.919515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:58:57.919515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.01355234,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"232'18","reported_seq":390,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.591598+0000","last_change":"2024-09-10T18:58:52.890712+0000","last_active":"2024-09-10T19:01:34.591598+0000","last_peered":"2024-09-10T19:01:34.591598+0000","last_clean":"2024-09-10T19:01:34.591598+0000","last_became_active":"2024-09-10T18:45:20.058391+0000","last_became_peered":"2024-09-10T18:45:20.058391+0000","last_unstale":"2024-09-10T19:01:34.591598+0000","last_undegraded":"2024-09-10T19:01:34.591598+0000","last_fullsized":"2024-09-10T19:01:34.591598+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'18","last_scrub_stamp":"2024-09-10T18:58:52.890618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:58:52.890618+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016012082,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":385,"reported_epoch":359,"state":"active+clean","last_fresh":"2024-09-10T19:01:34.583418+0000","last_change":"2024-09-10T18:58:56.690660+0000","last_active":"2024-09-10T19:01:34.583418+0000","last_peered":"2024-09-10T19:01:34.583418+0000","last_clean":"2024-09-10T19:01:34.583418+0000","last_became_active":"2024-09-10T18:57:02.449135+0000","last_became_peered":"2024-09-10T18:57:02.449135+0000","last_unstale":"2024-09-10T19:01:34.583418+0000","last_undegraded":"2024-09-10T19:01:34.583418+0000","last_fullsized":"2024-09-10T19:01:34.583418+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:56.690606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:58:56.690606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060631599999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":5550440472,"num_objects":84694,"num_object_clones":0,"num_object_copies":254082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84694,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":192618,"num_write_kb":8873921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21975,"num_bytes_recovered":1440088064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18147586048,"data_stored":18147573832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57337,"ondisk_log_size":57337,"up":48,"acting":48,"num_store_stats":6},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":421,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1118,"num_read_kb":875,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":3372,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":499712,"data_stored":13268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":758,"ondisk_log_size":758,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":982,"num_read_kb":977,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":608258,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":7,"num_write":3,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":298468,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1701,"ondisk_log_size":1701,"up":63,"acting":63,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":0,"seq":223338299596,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":43,"seq":184683593937,"num_pgs":86,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3328848,"kb_used_data":2434632,"kb_used_omap":1,"kb_used_meta":894142,"kb_avail":90416304,"statfs":{"total":95995035648,"available":92586295296,"internally_reserved":0,"allocated":2493063168,"data_stored":2492641941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":915601752},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:00:54 2024","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.1830000000000001,"15min":0.98899999999999999},"min":{"1min":0.34399999999999997,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":5.0910000000000002,"5min":27.545000000000002,"15min":27.545000000000002},"last":0.63700000000000001},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.3149999999999999,"15min":1.0649999999999999},"min":{"1min":0.27900000000000003,"5min":0.219,"15min":0.219},"max":{"1min":4.6180000000000003,"5min":27.568000000000001,"15min":27.568000000000001},"last":0.79000000000000004}]},{"osd":1,"last update":"Tue Sep 10 19:01:03 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":1.7370000000000001,"15min":1.2889999999999999},"min":{"1min":0.253,"5min":0.22900000000000001,"15min":0.219},"max":{"1min":4.6200000000000001,"5min":45.311,"15min":45.311},"last":1.7929999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":1.645,"15min":1.181},"min":{"1min":0.26300000000000001,"5min":0.23300000000000001,"15min":0.214},"max":{"1min":3.5289999999999999,"5min":45.027999999999999,"15min":45.027999999999999},"last":1.587}]},{"osd":2,"last update":"Tue Sep 10 19:01:03 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.64800000000000002,"15min":0.78400000000000003},"min":{"1min":0.32700000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":3.4009999999999998,"5min":3.4009999999999998,"15min":4.4740000000000002},"last":0.69899999999999995},{"interface":"front","average":{"1min":1.0660000000000001,"5min":0.60499999999999998,"15min":0.71899999999999997},"min":{"1min":0.28499999999999998,"5min":0.249,"15min":0.249},"max":{"1min":4.6580000000000004,"5min":4.6580000000000004,"15min":4.8150000000000004},"last":1.7729999999999999}]},{"osd":3,"last update":"Tue Sep 10 19:01:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":2.3410000000000002,"15min":1.4239999999999999},"min":{"1min":0.34200000000000003,"5min":0.29599999999999999,"15min":0.25700000000000001},"max":{"1min":5.0220000000000002,"5min":76.977999999999994,"15min":76.977999999999994},"last":1.657},{"interface":"front","average":{"1min":0.88600000000000001,"5min":2.2999999999999998,"15min":1.522},"min":{"1min":0.29799999999999999,"5min":0.27500000000000002,"15min":0.253},"max":{"1min":2.2080000000000002,"5min":76.879999999999995,"15min":76.879999999999995},"last":0.51100000000000001}]},{"osd":4,"last update":"Tue Sep 10 19:01:03 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":0.64000000000000001,"15min":1.3280000000000001},"min":{"1min":0.25,"5min":0.23400000000000001,"15min":0.20999999999999999},"max":{"1min":4.7220000000000004,"5min":4.7220000000000004,"15min":165.03299999999999},"last":1.716},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.625,"15min":1.2430000000000001},"min":{"1min":0.26200000000000001,"5min":0.26000000000000001,"15min":0.22500000000000001},"max":{"1min":3.5619999999999998,"5min":3.5619999999999998,"15min":163.959},"last":1.6890000000000001}]},{"osd":5,"last update":"Tue Sep 10 19:01:03 2024","interfaces":[{"interface":"back","average":{"1min":12.706,"5min":3.0459999999999998,"15min":1.7549999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.24099999999999999},"max":{"1min":288.56999999999999,"5min":288.56999999999999,"15min":288.56999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":12.539,"5min":2.9660000000000002,"15min":1.7529999999999999},"min":{"1min":0.29799999999999999,"5min":0.27300000000000002,"15min":0.23999999999999999},"max":{"1min":291.05399999999997,"5min":291.05399999999997,"15min":291.05399999999997},"last":1.833}]},{"osd":7,"last update":"Tue Sep 10 19:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.67800000000000005,"15min":0.83499999999999996},"min":{"1min":0.32200000000000001,"5min":0.27900000000000003,"15min":0.26200000000000001},"max":{"1min":2.2839999999999998,"5min":2.98,"15min":5.3680000000000003},"last":0.65000000000000002},{"interface":"front","average":{"1min":1.1379999999999999,"5min":0.72499999999999998,"15min":0.88900000000000001},"min":{"1min":0.31,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":4.6820000000000004,"5min":4.6820000000000004,"15min":7.0350000000000001},"last":1.7490000000000001}]}]},{"osd":5,"up_from":37,"seq":158913790165,"num_pgs":84,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3613724,"kb_used_data":2721560,"kb_used_omap":1,"kb_used_meta":891966,"kb_avail":90131428,"statfs":{"total":95995035648,"available":92294582272,"internally_reserved":0,"allocated":2786877440,"data_stored":2786506474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":913373528},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:01:50 2024","interfaces":[{"interface":"back","average":{"1min":2.0720000000000001,"5min":2.8580000000000001,"15min":1.9850000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":12.401,"5min":82.465000000000003,"15min":82.465000000000003},"last":0.34799999999999998},{"interface":"front","average":{"1min":2.3109999999999999,"5min":2.9049999999999998,"15min":2.149},"min":{"1min":0.26500000000000001,"5min":0.218,"15min":0.218},"max":{"1min":12.492000000000001,"5min":82.519000000000005,"15min":82.519000000000005},"last":1.1950000000000001}]},{"osd":1,"last update":"Tue Sep 10 19:01:50 2024","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":2.7469999999999999,"15min":1.5600000000000001},"min":{"1min":0.29099999999999998,"5min":0.20999999999999999,"15min":0.19500000000000001},"max":{"1min":4.0179999999999998,"5min":67.582999999999998,"15min":67.582999999999998},"last":0.41899999999999998},{"interface":"front","average":{"1min":1.278,"5min":2.657,"15min":1.7190000000000001},"min":{"1min":0.29899999999999999,"5min":0.29699999999999999,"15min":0.20399999999999999},"max":{"1min":3.298,"5min":67.603999999999999,"15min":67.603999999999999},"last":1.2589999999999999}]},{"osd":2,"last update":"Tue Sep 10 19:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.33700000000000002,"5min":0.27200000000000002,"15min":0.249},"max":{"1min":2.1030000000000002,"5min":7.8879999999999999,"15min":7.8879999999999999},"last":0.505},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.752,"15min":0.80200000000000005},"min":{"1min":0.28199999999999997,"5min":0.27100000000000002,"15min":0.24399999999999999},"max":{"1min":2.8479999999999999,"5min":6.7549999999999999,"15min":6.7549999999999999},"last":0.316}]},{"osd":3,"last update":"Tue Sep 10 19:01:50 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":4.8780000000000001,"15min":2.75},"min":{"1min":0.27700000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":5.2750000000000004,"5min":123.34399999999999,"15min":123.34399999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":1.524,"5min":3.3100000000000001,"15min":2.3479999999999999},"min":{"1min":0.312,"5min":0.27300000000000002,"15min":0.245},"max":{"1min":5.4580000000000002,"5min":82.415000000000006,"15min":82.415000000000006},"last":0.28599999999999998}]},{"osd":4,"last update":"Tue Sep 10 19:01:50 2024","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":0.70899999999999996,"15min":0.91400000000000003},"min":{"1min":0.36399999999999999,"5min":0.221,"15min":0.20899999999999999},"max":{"1min":3.9820000000000002,"5min":3.9820000000000002,"15min":9.0310000000000006},"last":1.29},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.63700000000000001,"15min":0.81999999999999995},"min":{"1min":0.317,"5min":0.23899999999999999,"15min":0.23400000000000001},"max":{"1min":2.125,"5min":2.742,"15min":11.182},"last":1.8799999999999999}]},{"osd":6,"last update":"Tue Sep 10 19:01:10 2024","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":0.69899999999999995,"15min":0.96599999999999997},"min":{"1min":0.30199999999999999,"5min":0.23999999999999999,"15min":0.23400000000000001},"max":{"1min":4.1200000000000001,"5min":4.1200000000000001,"15min":11.17},"last":0.36499999999999999},{"interface":"front","average":{"1min":1.0669999999999999,"5min":0.622,"15min":0.94999999999999996},"min":{"1min":0.38500000000000001,"5min":0.27200000000000002,"15min":0.26700000000000002},"max":{"1min":3.1930000000000001,"5min":3.1930000000000001,"15min":11.18},"last":0.56100000000000005}]},{"osd":7,"last update":"Tue Sep 10 19:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":0.69399999999999995,"15min":0.94399999999999995},"min":{"1min":0.308,"5min":0.28999999999999998,"15min":0.27600000000000002},"max":{"1min":2.6859999999999999,"5min":3.665,"15min":10.782999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.61299999999999999,"15min":0.83199999999999996},"min":{"1min":0.34699999999999998,"5min":0.26900000000000002,"15min":0.23599999999999999},"max":{"1min":2.6200000000000001,"5min":2.6200000000000001,"15min":11.846},"last":0.39100000000000001}]}]},{"osd":4,"up_from":30,"seq":128849019099,"num_pgs":85,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5638792,"kb_used_data":4752448,"kb_used_omap":1,"kb_used_meta":886142,"kb_avail":88106360,"statfs":{"total":95995035648,"available":90220912640,"internally_reserved":0,"allocated":4866506752,"data_stored":4866092739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":907409752},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:01:55 2024","interfaces":[{"interface":"back","average":{"1min":1.7569999999999999,"5min":1.5629999999999999,"15min":1.4790000000000001},"min":{"1min":0.37,"5min":0.223,"15min":0.223},"max":{"1min":10.601000000000001,"5min":46.234999999999999,"15min":46.234999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.806,"5min":1.4910000000000001,"15min":1.474},"min":{"1min":0.50700000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":10.634,"5min":46.264000000000003,"15min":46.264000000000003},"last":0.75800000000000001}]},{"osd":1,"last update":"Tue Sep 10 19:01:30 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":2.2829999999999999,"15min":1.5660000000000001},"min":{"1min":0.24199999999999999,"5min":0.223,"15min":0.223},"max":{"1min":2.254,"5min":125.15600000000001,"15min":125.15600000000001},"last":1.0960000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":2.4129999999999998,"15min":1.502},"min":{"1min":0.30199999999999999,"5min":0.23699999999999999,"15min":0.23100000000000001},"max":{"1min":1.48,"5min":125.14700000000001,"15min":125.14700000000001},"last":0.39700000000000002}]},{"osd":2,"last update":"Tue Sep 10 19:01:30 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67400000000000004,"15min":0.68200000000000005},"min":{"1min":0.28499999999999998,"5min":0.26200000000000001,"15min":0.25900000000000001},"max":{"1min":1.5900000000000001,"5min":2.8879999999999999,"15min":5.6189999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":1.2310000000000001,"5min":0.872,"15min":0.79100000000000004},"min":{"1min":0.34699999999999998,"5min":0.29999999999999999,"15min":0.27100000000000002},"max":{"1min":3.673,"5min":12.782999999999999,"15min":12.782999999999999},"last":0.63100000000000001}]},{"osd":3,"last update":"Tue Sep 10 19:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":3.6030000000000002,"15min":2.1230000000000002},"min":{"1min":0.377,"5min":0.28299999999999997,"15min":0.23799999999999999},"max":{"1min":3.556,"5min":111.514,"15min":111.514},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.86399999999999999,"5min":2.6099999999999999,"15min":1.7130000000000001},"min":{"1min":0.27400000000000002,"5min":0.26900000000000002,"15min":0.25700000000000001},"max":{"1min":2.3730000000000002,"5min":110.914,"15min":110.914},"last":1.0740000000000001}]},{"osd":5,"last update":"Tue Sep 10 19:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.71299999999999997,"15min":0.89300000000000002},"min":{"1min":0.36299999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":2.6699999999999999,"5min":9.7309999999999999,"15min":10.912000000000001},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.024,"5min":0.75800000000000001,"15min":0.83599999999999997},"min":{"1min":0.33100000000000002,"5min":0.216,"15min":0.216},"max":{"1min":3.0019999999999998,"5min":10.147,"15min":10.147},"last":0.33100000000000002}]},{"osd":6,"last update":"Tue Sep 10 19:01:12 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.57099999999999995,"15min":0.749},"min":{"1min":0.33600000000000002,"5min":0.221,"15min":0.221},"max":{"1min":2.5110000000000001,"5min":2.5110000000000001,"15min":8.4030000000000005},"last":0.78400000000000003},{"interface":"front","average":{"1min":2.1190000000000002,"5min":0.77800000000000002,"15min":0.90900000000000003},"min":{"1min":0.32900000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":21.254000000000001,"5min":21.254000000000001,"15min":21.254000000000001},"last":1.0489999999999999}]},{"osd":7,"last update":"Tue Sep 10 19:01:46 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.60899999999999999,"15min":0.82999999999999996},"min":{"1min":0.40600000000000003,"5min":0.29099999999999998,"15min":0.27600000000000002},"max":{"1min":1.516,"5min":2.8879999999999999,"15min":6.4630000000000001},"last":0.42599999999999999},{"interface":"front","average":{"1min":1.05,"5min":0.71099999999999997,"15min":0.83699999999999997},"min":{"1min":0.311,"5min":0.27500000000000002,"15min":0.251},"max":{"1min":3.1219999999999999,"5min":3.589,"15min":6.3949999999999996},"last":0.57799999999999996}]}]},{"osd":3,"up_from":25,"seq":107374182624,"num_pgs":83,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3338764,"kb_used_data":2441028,"kb_used_omap":1,"kb_used_meta":897726,"kb_avail":90406388,"statfs":{"total":95995035648,"available":92576141312,"internally_reserved":0,"allocated":2499612672,"data_stored":2499194437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":919271768},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:01:50 2024","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":4.7140000000000004,"15min":2.4180000000000001},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":5.6349999999999998,"5min":290.721,"15min":290.721},"last":1.6060000000000001},{"interface":"front","average":{"1min":0.82099999999999995,"5min":4.8140000000000001,"15min":2.6150000000000002},"min":{"1min":0.188,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":2.335,"5min":293.59300000000002,"15min":293.59300000000002},"last":2.2549999999999999}]},{"osd":1,"last update":"Tue Sep 10 19:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":2.1419999999999999,"15min":1.573},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.16200000000000001},"max":{"1min":3.173,"5min":10.645,"15min":58.215000000000003},"last":0.246},{"interface":"front","average":{"1min":0.77400000000000002,"5min":1.8260000000000001,"15min":1.4910000000000001},"min":{"1min":0.22700000000000001,"5min":0.20000000000000001,"15min":0.188},"max":{"1min":2.3239999999999998,"5min":13.263999999999999,"15min":57.896999999999998},"last":2.1699999999999999}]},{"osd":2,"last update":"Tue Sep 10 19:01:05 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":1.286,"15min":0.95999999999999996},"min":{"1min":0.255,"5min":0.255,"15min":0.22800000000000001},"max":{"1min":2.23,"5min":5.6680000000000001,"15min":5.8220000000000001},"last":2.2759999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":1.1220000000000001,"15min":0.84299999999999997},"min":{"1min":0.254,"5min":0.22600000000000001,"15min":0.21199999999999999},"max":{"1min":1.526,"5min":5.1399999999999997,"15min":7.0979999999999999},"last":0.68999999999999995}]},{"osd":4,"last update":"Tue Sep 10 19:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.423,"15min":1.2529999999999999},"min":{"1min":0.439,"5min":0.26400000000000001,"15min":0.247},"max":{"1min":3.4249999999999998,"5min":5.0490000000000004,"15min":29.013000000000002},"last":0.71499999999999997},{"interface":"front","average":{"1min":1.103,"5min":1.4430000000000001,"15min":1.248},"min":{"1min":0.39700000000000002,"5min":0.28000000000000003,"15min":0.26800000000000002},"max":{"1min":3.3879999999999999,"5min":8.8130000000000006,"15min":28.826000000000001},"last":0.65600000000000003}]},{"osd":5,"last update":"Tue Sep 10 19:01:54 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.3999999999999999,"15min":1.25},"min":{"1min":0.35399999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":2.6890000000000001,"5min":7.2300000000000004,"15min":7.2300000000000004},"last":0.59899999999999998},{"interface":"front","average":{"1min":1,"5min":1.357,"15min":1.2230000000000001},"min":{"1min":0.28299999999999997,"5min":0.27100000000000002,"15min":0.255},"max":{"1min":4.2789999999999999,"5min":5.4139999999999997,"15min":9.2509999999999994},"last":4.2789999999999999}]},{"osd":6,"last update":"Tue Sep 10 19:01:09 2024","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.504,"15min":1.2629999999999999},"min":{"1min":0.39500000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":3.431,"5min":5.4480000000000004,"15min":9.3239999999999998},"last":2.206},{"interface":"front","average":{"1min":1.04,"5min":1.4219999999999999,"15min":1.2210000000000001},"min":{"1min":0.34999999999999998,"5min":0.308,"15min":0.29399999999999998},"max":{"1min":2.794,"5min":5.0789999999999997,"15min":7.1559999999999997},"last":0.52400000000000002}]},{"osd":7,"last update":"Tue Sep 10 19:01:40 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":1.5289999999999999,"15min":1.294},"min":{"1min":0.39700000000000002,"5min":0.35599999999999998,"15min":0.34899999999999998},"max":{"1min":2.7290000000000001,"5min":8.7989999999999995,"15min":8.7989999999999995},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":1.46,"15min":1.3140000000000001},"min":{"1min":0.42999999999999999,"5min":0.36899999999999999,"15min":0.33100000000000002},"max":{"1min":2.266,"5min":6.6029999999999998,"15min":9.2949999999999999},"last":2.2320000000000002}]}]},{"osd":2,"up_from":0,"seq":77309411556,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834575080,"num_pgs":87,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2620008,"kb_used_data":1724960,"kb_used_omap":1,"kb_used_meta":894974,"kb_avail":91125144,"statfs":{"total":95995035648,"available":93312147456,"internally_reserved":0,"allocated":1766359040,"data_stored":1765913596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":916453720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:01:51 2024","interfaces":[{"interface":"back","average":{"1min":2.7410000000000001,"5min":3.181,"15min":1.891},"min":{"1min":0.158,"5min":0.158,"15min":0.158},"max":{"1min":25.370999999999999,"5min":111.014,"15min":111.014},"last":0.61599999999999999},{"interface":"front","average":{"1min":2.9199999999999999,"5min":3.3580000000000001,"15min":1.8660000000000001},"min":{"1min":0.217,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":27.809999999999999,"5min":111.072,"15min":111.072},"last":0.502}]},{"osd":2,"last update":"Tue Sep 10 19:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.873,"15min":0.752},"min":{"1min":0.26500000000000001,"5min":0.23300000000000001,"15min":0.186},"max":{"1min":1.845,"5min":4.1719999999999997,"15min":4.1719999999999997},"last":0.32300000000000001},{"interface":"front","average":{"1min":1.073,"5min":0.95599999999999996,"15min":0.80500000000000005},"min":{"1min":0.25900000000000001,"5min":0.23899999999999999,"15min":0.215},"max":{"1min":4.2560000000000002,"5min":5.6929999999999996,"15min":7.093},"last":0.51600000000000001}]},{"osd":3,"last update":"Tue Sep 10 19:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":3.718,"15min":2.0419999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.185},"max":{"1min":6.9560000000000004,"5min":92.069000000000003,"15min":92.069000000000003},"last":0.248},{"interface":"front","average":{"1min":0.68100000000000005,"5min":3.1930000000000001,"15min":1.9019999999999999},"min":{"1min":0.23300000000000001,"5min":0.218,"15min":0.19900000000000001},"max":{"1min":2.1110000000000002,"5min":88.658000000000001,"15min":88.658000000000001},"last":0.55000000000000004}]},{"osd":4,"last update":"Tue Sep 10 19:01:28 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":1.155,"15min":1.151},"min":{"1min":0.307,"5min":0.27800000000000002,"15min":0.26200000000000001},"max":{"1min":2.2010000000000001,"5min":9.125,"15min":15.965999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":1.141,"15min":1.0089999999999999},"min":{"1min":0.29899999999999999,"5min":0.252,"15min":0.246},"max":{"1min":1.7549999999999999,"5min":7.0940000000000003,"15min":7.0940000000000003},"last":0.53400000000000003}]},{"osd":5,"last update":"Tue Sep 10 19:01:51 2024","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":1.1899999999999999,"15min":1.21},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.28499999999999998},"max":{"1min":4.226,"5min":6.4690000000000003,"15min":10.334},"last":0.45100000000000001},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.075,"15min":1.2170000000000001},"min":{"1min":0.46800000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":3.5089999999999999,"5min":7.0590000000000002,"15min":16.039999999999999},"last":0.58399999999999996}]},{"osd":6,"last update":"Tue Sep 10 19:01:05 2024","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.2809999999999999,"15min":1.506},"min":{"1min":0.34200000000000003,"5min":0.33100000000000002,"15min":0.308},"max":{"1min":7.2850000000000001,"5min":19.125,"15min":38.039999999999999},"last":1.403},{"interface":"front","average":{"1min":1.51,"5min":1.173,"15min":1.5820000000000001},"min":{"1min":0.32600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":7.2380000000000004,"5min":7.2380000000000004,"15min":38.100999999999999},"last":1.3420000000000001}]},{"osd":7,"last update":"Tue Sep 10 19:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":1.2330000000000001,"15min":1.163},"min":{"1min":0.39800000000000002,"5min":0.33100000000000002,"15min":0.315},"max":{"1min":4.1710000000000003,"5min":19.169,"15min":19.169},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.995,"15min":1.0589999999999999},"min":{"1min":0.39600000000000002,"5min":0.36499999999999999,"15min":0.32800000000000001},"max":{"1min":2.2410000000000001,"5min":4.46,"15min":4.9029999999999996},"last":1.1439999999999999}]}]},{"osd":0,"up_from":241,"seq":1035087118413,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4107868,"kb_used_data":3769876,"kb_used_omap":4,"kb_used_meta":337915,"kb_avail":89637284,"statfs":{"total":95995035648,"available":91788578816,"internally_reserved":0,"allocated":3860353024,"data_stored":3859985427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":346025478},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 19:01:49 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.534,"15min":2.3639999999999999},"min":{"1min":0.22800000000000001,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":4.5810000000000004,"5min":13.617000000000001,"15min":34.692},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.875,"5min":1.419,"15min":2.3530000000000002},"min":{"1min":0.253,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":3.2040000000000002,"5min":9.7100000000000009,"15min":34.588999999999999},"last":1.002}]},{"osd":2,"last update":"Tue Sep 10 19:01:49 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":1.0640000000000001,"15min":0.93999999999999995},"min":{"1min":0.23799999999999999,"5min":0.22,"15min":0.20699999999999999},"max":{"1min":4.6130000000000004,"5min":13.305,"15min":13.305},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.79000000000000004,"5min":1.0549999999999999,"15min":0.81399999999999995},"min":{"1min":0.27800000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":3.0960000000000001,"5min":13.693,"15min":13.693},"last":1.0289999999999999}]},{"osd":3,"last update":"Tue Sep 10 19:01:49 2024","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.4930000000000001,"15min":1.145},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":3.2589999999999999,"5min":21.108000000000001,"15min":21.108000000000001},"last":1.276},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.9359999999999999,"15min":1.6839999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":6.2949999999999999,"5min":11.629,"15min":11.629},"last":0.69299999999999995}]},{"osd":4,"last update":"Tue Sep 10 19:01:49 2024","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.212,"15min":0.88200000000000001},"min":{"1min":0.36699999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":4.2830000000000004,"5min":13.161,"15min":13.161},"last":2.4780000000000002},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.2989999999999999,"15min":1.0349999999999999},"min":{"1min":0.34100000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":4.4020000000000001,"5min":5.0490000000000004,"15min":5.0490000000000004},"last":0.77600000000000002}]},{"osd":5,"last update":"Tue Sep 10 19:01:49 2024","interfaces":[{"interface":"back","average":{"1min":7.0060000000000002,"5min":2.4319999999999999,"15min":1.484},"min":{"1min":0.36399999999999999,"5min":0.31,"15min":0.31},"max":{"1min":97.296999999999997,"5min":97.296999999999997,"15min":97.296999999999997},"last":1.0880000000000001},{"interface":"front","average":{"1min":6.391,"5min":2.2639999999999998,"15min":1.274},"min":{"1min":0.48299999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":99.221999999999994,"5min":99.221999999999994,"15min":99.221999999999994},"last":0.879}]},{"osd":6,"last update":"Tue Sep 10 19:01:49 2024","interfaces":[{"interface":"back","average":{"1min":1.524,"5min":1.3200000000000001,"15min":1.048},"min":{"1min":0.39000000000000001,"5min":0.312,"15min":0.312},"max":{"1min":7.351,"5min":7.351,"15min":7.351},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.516,"5min":1.266,"15min":0.88600000000000001},"min":{"1min":0.40100000000000002,"5min":0.313,"15min":0.313},"max":{"1min":4.3849999999999998,"5min":13.279,"15min":13.279},"last":0.81599999999999995}]},{"osd":7,"last update":"Tue Sep 10 19:01:49 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.2509999999999999,"15min":0.96799999999999997},"min":{"1min":0.46899999999999997,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":4.3380000000000001,"5min":13.233000000000001,"15min":13.233000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":2.9430000000000001,"5min":1.494,"15min":1.262},"min":{"1min":0.40100000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":38.636000000000003,"5min":38.636000000000003,"15min":38.636000000000003},"last":1.242}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":2668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3847884800,"data_stored":3847880728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1748631552,"data_stored":1748631552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2461138944,"data_stored":2461138944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4834791424,"data_stored":4834787352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2784559104,"data_stored":2784559104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2470580224,"data_stored":2470576152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T19:02:06.030 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-10T19:02:06.030 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T19:02:06.031 INFO:tasks.thrashosds.thrasher:Killing osd 7, live_osds are [1, 2, 3, 4, 5, 6, 7, 0] 2024-09-10T19:02:06.031 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-10T19:02:06.031 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@osd.7 2024-09-10T19:02:06.032 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:05 smithi103 ceph-mon[33163]: 4.1e scrub starts 2024-09-10T19:02:06.032 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:05 smithi103 ceph-mon[33163]: 4.1e scrub ok 2024-09-10T19:02:06.088 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:06.127 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:06.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:06.128 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:06.229 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:06.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:06.271 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:06.271 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:06.372 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:06.404 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:06.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:06.405 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:06.506 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:06.522 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:02:06 smithi103 systemd[1]: Stopping Ceph osd.7 for ba327f6c-6fa3-11ef-bcea-c7b262605968... 2024-09-10T19:02:06.537 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:06.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:06.538 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:06.570 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:06 smithi018 ceph-mon[40196]: pgmap v779: 168 pgs: 168 active+clean; 4.7 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.67k op/s 2024-09-10T19:02:06.571 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:06 smithi018 ceph-mon[40196]: 1.12 scrub starts 2024-09-10T19:02:06.571 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:06 smithi018 ceph-mon[40196]: 1.12 scrub ok 2024-09-10T19:02:06.571 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:06 smithi018 ceph-mon[40196]: 2.f scrub starts 2024-09-10T19:02:06.571 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:06 smithi018 ceph-mon[40196]: 2.f scrub ok 2024-09-10T19:02:06.571 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:06 smithi018 ceph-mon[29336]: pgmap v779: 168 pgs: 168 active+clean; 4.7 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.67k op/s 2024-09-10T19:02:06.572 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:06 smithi018 ceph-mon[29336]: 1.12 scrub starts 2024-09-10T19:02:06.572 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:06 smithi018 ceph-mon[29336]: 1.12 scrub ok 2024-09-10T19:02:06.572 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:06 smithi018 ceph-mon[29336]: 2.f scrub starts 2024-09-10T19:02:06.572 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:06 smithi018 ceph-mon[29336]: 2.f scrub ok 2024-09-10T19:02:06.639 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:06.674 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:06.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:06.675 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:06.776 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:06.811 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:06.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:06.813 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:06.913 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:06.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:06.950 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:06.951 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:07.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:06 smithi103 ceph-mon[33163]: pgmap v779: 168 pgs: 168 active+clean; 4.7 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.67k op/s 2024-09-10T19:02:07.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:06 smithi103 ceph-mon[33163]: 1.12 scrub starts 2024-09-10T19:02:07.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:06 smithi103 ceph-mon[33163]: 1.12 scrub ok 2024-09-10T19:02:07.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:06 smithi103 ceph-mon[33163]: 2.f scrub starts 2024-09-10T19:02:07.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:06 smithi103 ceph-mon[33163]: 2.f scrub ok 2024-09-10T19:02:07.023 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:02:06 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[55499]: 2024-09-10T19:02:06.584+0000 7f5cdab76640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:02:07.023 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:02:06 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[55499]: 2024-09-10T19:02:06.584+0000 7f5cdab76640 -1 osd.7 361 *** Got signal Terminated *** 2024-09-10T19:02:07.023 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:02:06 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[55499]: 2024-09-10T19:02:06.584+0000 7f5cdab76640 -1 osd.7 361 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T19:02:07.051 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:07.083 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:07.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:07.084 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:07.185 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:07.224 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:07.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:07.225 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:07.326 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:07.364 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:07.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:07.365 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:07.466 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:07.481 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-10T19:02:07.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:07.498 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:07.498 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:07.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:02:07 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:02:07.599 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:07.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:07.643 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:07.643 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:07.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:07 smithi018 ceph-mon[40196]: 2.0 scrub starts 2024-09-10T19:02:07.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:07 smithi018 ceph-mon[40196]: 2.0 scrub ok 2024-09-10T19:02:07.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:07 smithi018 ceph-mon[40196]: osd.7 marked itself down and dead 2024-09-10T19:02:07.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:07 smithi018 ceph-mon[29336]: 2.0 scrub starts 2024-09-10T19:02:07.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:07 smithi018 ceph-mon[29336]: 2.0 scrub ok 2024-09-10T19:02:07.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:07 smithi018 ceph-mon[29336]: osd.7 marked itself down and dead 2024-09-10T19:02:07.744 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:07.791 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:07.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:07.792 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:07.893 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:07.928 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:07.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:07.929 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:08.017 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:07 smithi103 ceph-mon[33163]: 2.0 scrub starts 2024-09-10T19:02:08.017 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:07 smithi103 ceph-mon[33163]: 2.0 scrub ok 2024-09-10T19:02:08.017 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:07 smithi103 ceph-mon[33163]: osd.7 marked itself down and dead 2024-09-10T19:02:08.031 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:08.064 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:08.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:08.065 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:08.166 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:08.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:08.199 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:08.200 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:08.272 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:02:08 smithi103 podman[129868]: 2024-09-10 19:02:08.016749897 +0000 UTC m=+1.598619518 container died 9a1f0477ea5b1bcebe484b0586c825668a67642ce526107aaf3caca0c8c61cd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-10T19:02:08.301 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:08.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:08.345 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:08.345 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:08.446 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:08.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:08.487 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:08.487 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:08.588 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:08.617 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:08 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:08.617 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:08 smithi103 ceph-mon[33163]: pgmap v780: 168 pgs: 168 active+clean; 4.3 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 750 B/s rd, 0 B/s wr, 2.04k op/s 2024-09-10T19:02:08.617 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:08 smithi103 ceph-mon[33163]: osdmap e362: 8 total, 7 up, 6 in 2024-09-10T19:02:08.617 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:02:08 smithi103 podman[129868]: 2024-09-10 19:02:08.605128806 +0000 UTC m=+2.186998428 container cleanup 9a1f0477ea5b1bcebe484b0586c825668a67642ce526107aaf3caca0c8c61cd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True) 2024-09-10T19:02:08.617 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:02:08 smithi103 bash[129868]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7 2024-09-10T19:02:08.623 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:08.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:08.624 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:08.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:08 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:08.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:08 smithi018 ceph-mon[29336]: pgmap v780: 168 pgs: 168 active+clean; 4.3 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 750 B/s rd, 0 B/s wr, 2.04k op/s 2024-09-10T19:02:08.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:08 smithi018 ceph-mon[29336]: osdmap e362: 8 total, 7 up, 6 in 2024-09-10T19:02:08.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:08 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:08.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:08 smithi018 ceph-mon[40196]: pgmap v780: 168 pgs: 168 active+clean; 4.3 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 750 B/s rd, 0 B/s wr, 2.04k op/s 2024-09-10T19:02:08.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:08 smithi018 ceph-mon[40196]: osdmap e362: 8 total, 7 up, 6 in 2024-09-10T19:02:08.725 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:08.760 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:08.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:08.761 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:08.862 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:08.922 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:02:08.922 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:08.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:08.923 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:09.025 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:09.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:09.063 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:09.063 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:09.164 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:09.206 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:09.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:09.207 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:09.308 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:09.346 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:09.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:09.347 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:09.448 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:09.486 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:09.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:09.487 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:09.503 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:02:09 smithi103 podman[129950]: 2024-09-10 19:02:09.024366869 +0000 UTC m=+1.004644354 container remove 9a1f0477ea5b1bcebe484b0586c825668a67642ce526107aaf3caca0c8c61cd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T19:02:09.588 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:09.629 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:09.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:09.630 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:09.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:09 smithi018 ceph-mon[29336]: 5.f scrub starts 2024-09-10T19:02:09.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:09 smithi018 ceph-mon[29336]: 5.f scrub ok 2024-09-10T19:02:09.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:09 smithi018 ceph-mon[29336]: 5.11 scrub starts 2024-09-10T19:02:09.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:09 smithi018 ceph-mon[29336]: 5.11 scrub ok 2024-09-10T19:02:09.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:09 smithi018 ceph-mon[29336]: 2.e scrub starts 2024-09-10T19:02:09.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:09 smithi018 ceph-mon[29336]: 2.e scrub ok 2024-09-10T19:02:09.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:09 smithi018 ceph-mon[40196]: 5.f scrub starts 2024-09-10T19:02:09.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:09 smithi018 ceph-mon[40196]: 5.f scrub ok 2024-09-10T19:02:09.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:09 smithi018 ceph-mon[40196]: 5.11 scrub starts 2024-09-10T19:02:09.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:09 smithi018 ceph-mon[40196]: 5.11 scrub ok 2024-09-10T19:02:09.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:09 smithi018 ceph-mon[40196]: 2.e scrub starts 2024-09-10T19:02:09.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:09 smithi018 ceph-mon[40196]: 2.e scrub ok 2024-09-10T19:02:09.731 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:09.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:09.766 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:09.766 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:09.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:09 smithi103 ceph-mon[33163]: 5.f scrub starts 2024-09-10T19:02:09.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:09 smithi103 ceph-mon[33163]: 5.f scrub ok 2024-09-10T19:02:09.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:09 smithi103 ceph-mon[33163]: 5.11 scrub starts 2024-09-10T19:02:09.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:09 smithi103 ceph-mon[33163]: 5.11 scrub ok 2024-09-10T19:02:09.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:09 smithi103 ceph-mon[33163]: 2.e scrub starts 2024-09-10T19:02:09.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:09 smithi103 ceph-mon[33163]: 2.e scrub ok 2024-09-10T19:02:09.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:02:09 smithi103 podman[130078]: 2024-09-10 19:02:09.407695357 +0000 UTC m=+0.027263652 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:02:09.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:02:09 smithi103 podman[130078]: 2024-09-10 19:02:09.533943179 +0000 UTC m=+0.153511473 container create 21f2eca2f3ef1ee06d3205e8092aba23e6893741ff9083649d7479ae2f0a277a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-10T19:02:09.867 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:09.909 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:09.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:09.910 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:10.007 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T19:02:10.007 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T19:02:10.007 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T19:02:10.007 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T19:02:10.007 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T19:02:10.007 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T19:02:10.007 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T19:02:10.008 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T19:02:10.008 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T19:02:10.008 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T19:02:10.008 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T19:02:10.008 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T19:02:10.008 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T19:02:10.008 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T19:02:10.008 INFO:teuthology.orchestra.run.smithi018.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-10T19:02:10.011 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:10.030 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:02:09 smithi103 podman[130078]: 2024-09-10 19:02:09.806992122 +0000 UTC m=+0.426560418 container init 21f2eca2f3ef1ee06d3205e8092aba23e6893741ff9083649d7479ae2f0a277a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-10T19:02:10.030 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:02:09 smithi103 podman[130078]: 2024-09-10 19:02:09.812412339 +0000 UTC m=+0.431980635 container start 21f2eca2f3ef1ee06d3205e8092aba23e6893741ff9083649d7479ae2f0a277a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-10T19:02:10.030 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:02:09 smithi103 podman[130078]: 2024-09-10 19:02:09.921393412 +0000 UTC m=+0.540961708 container attach 21f2eca2f3ef1ee06d3205e8092aba23e6893741ff9083649d7479ae2f0a277a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-10T19:02:10.049 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:10.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:10.050 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:10.151 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:10.186 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:10.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:10.187 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:10.288 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:10.319 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:10.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:10.320 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:10.376 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:02:10 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T19:02:10.090+0000 7f66d46e1640 -1 mgr get_metadata_python Requested missing service osd.7 2024-09-10T19:02:10.421 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:10.455 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:10.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:10.456 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:10.522 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:02:10 smithi103 podman[130078]: 2024-09-10 19:02:10.105683874 +0000 UTC m=+0.725252170 container died 21f2eca2f3ef1ee06d3205e8092aba23e6893741ff9083649d7479ae2f0a277a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T19:02:10.523 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:02:10 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:02:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:02:10.557 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:10.592 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:10.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:10.593 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:10.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:10 smithi018 ceph-mon[40196]: 6.6 scrub starts 2024-09-10T19:02:10.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:10 smithi018 ceph-mon[40196]: 6.6 scrub ok 2024-09-10T19:02:10.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:10 smithi018 ceph-mon[40196]: pgmap v782: 168 pgs: 168 active+clean; 4.2 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 307 B/s rd, 0 B/s wr, 1.65k op/s 2024-09-10T19:02:10.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:10 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1993769412' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:02:10.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:10 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:02:10.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:10 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/842582913' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:02:10.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:10 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:02:10.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:10 smithi018 ceph-mon[29336]: 6.6 scrub starts 2024-09-10T19:02:10.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:10 smithi018 ceph-mon[29336]: 6.6 scrub ok 2024-09-10T19:02:10.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:10 smithi018 ceph-mon[29336]: pgmap v782: 168 pgs: 168 active+clean; 4.2 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 307 B/s rd, 0 B/s wr, 1.65k op/s 2024-09-10T19:02:10.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:10 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1993769412' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:02:10.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:10 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:02:10.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:10 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/842582913' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:02:10.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:10 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:02:10.695 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:10.733 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:10.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:10.734 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:10.835 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:10.845 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:10 smithi103 ceph-mon[33163]: 6.6 scrub starts 2024-09-10T19:02:10.845 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:10 smithi103 ceph-mon[33163]: 6.6 scrub ok 2024-09-10T19:02:10.845 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:10 smithi103 ceph-mon[33163]: pgmap v782: 168 pgs: 168 active+clean; 4.2 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 307 B/s rd, 0 B/s wr, 1.65k op/s 2024-09-10T19:02:10.845 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:10 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1993769412' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:02:10.845 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:10 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:02:10.845 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:10 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/842582913' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:02:10.845 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:10 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:02:10.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:10.872 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:10.872 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:10.880 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@osd.7.service' 2024-09-10T19:02:10.973 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:10.996 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:02:10.996 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-10T19:02:10.997 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.7 2024-09-10T19:02:10.997 INFO:tasks.thrashosds.thrasher:remote for osd 7 is ubuntu@smithi103.front.sepia.ceph.com 2024-09-10T19:02:10.997 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -n osd.7 --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-7 --journal-path /var/lib/ceph/osd/ceph-7/journal --op list-pgs 2024-09-10T19:02:11.007 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:11.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:11.009 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:11.010 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:02:10 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:02:10] "GET /metrics HTTP/1.1" 200 209516 "" "Prometheus/2.43.0" 2024-09-10T19:02:11.110 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:11.142 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:11.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:11.143 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:11.157 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-10T19:02:11.157 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-10T19:02:11.233 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is unset 2024-09-10T19:02:11.244 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:11.277 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:11.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:11.278 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:11.343 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/osd.7/config 2024-09-10T19:02:11.378 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:11.413 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:11.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:11.414 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:11.514 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:11.555 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:11.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:11.556 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:11.657 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:11.699 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:11.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:11.700 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:11.801 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:11.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:11.836 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:11.836 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:11.875 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:11 smithi103 ceph-mon[33163]: 5.6 scrub starts 2024-09-10T19:02:11.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:11 smithi103 ceph-mon[33163]: 5.6 scrub ok 2024-09-10T19:02:11.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:11 smithi103 ceph-mon[33163]: 6.3 scrub starts 2024-09-10T19:02:11.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:11 smithi103 ceph-mon[33163]: 6.3 scrub ok 2024-09-10T19:02:11.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:11 smithi103 ceph-mon[33163]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T19:02:11.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:11 smithi103 ceph-mon[33163]: Cluster is now healthy 2024-09-10T19:02:11.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:11 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T19:02:11.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:11 smithi103 ceph-mon[33163]: osdmap e363: 8 total, 7 up, 6 in 2024-09-10T19:02:11.938 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:11.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:11 smithi018 ceph-mon[40196]: 5.6 scrub starts 2024-09-10T19:02:11.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:11 smithi018 ceph-mon[40196]: 5.6 scrub ok 2024-09-10T19:02:11.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:11 smithi018 ceph-mon[40196]: 6.3 scrub starts 2024-09-10T19:02:11.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:11 smithi018 ceph-mon[40196]: 6.3 scrub ok 2024-09-10T19:02:11.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:11 smithi018 ceph-mon[40196]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T19:02:11.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:11 smithi018 ceph-mon[40196]: Cluster is now healthy 2024-09-10T19:02:11.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:11 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T19:02:11.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:11 smithi018 ceph-mon[40196]: osdmap e363: 8 total, 7 up, 6 in 2024-09-10T19:02:11.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:11 smithi018 ceph-mon[29336]: 5.6 scrub starts 2024-09-10T19:02:11.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:11 smithi018 ceph-mon[29336]: 5.6 scrub ok 2024-09-10T19:02:11.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:11 smithi018 ceph-mon[29336]: 6.3 scrub starts 2024-09-10T19:02:11.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:11 smithi018 ceph-mon[29336]: 6.3 scrub ok 2024-09-10T19:02:11.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:11 smithi018 ceph-mon[29336]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T19:02:11.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:11 smithi018 ceph-mon[29336]: Cluster is now healthy 2024-09-10T19:02:11.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:11 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T19:02:11.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:11 smithi018 ceph-mon[29336]: osdmap e363: 8 total, 7 up, 6 in 2024-09-10T19:02:11.972 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:11.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:11.973 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:12.074 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:12.104 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:12.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:12.105 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:12.206 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:12.248 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:12.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:12.249 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:12.350 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:12.384 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:12.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:12.385 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:12.485 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:12.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:12.523 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:12.523 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:12.624 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:12.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:12.664 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:12.664 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:12.765 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:12.805 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:12.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:12.807 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:12.891 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:12 smithi103 ceph-mon[33163]: pgmap v784: 168 pgs: 168 active+clean; 4.1 GiB data, 18 GiB used, 518 GiB / 536 GiB avail; 0 B/s rd, 0 B/s wr, 1.34k op/s 2024-09-10T19:02:12.908 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:12.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:12 smithi018 ceph-mon[29336]: pgmap v784: 168 pgs: 168 active+clean; 4.1 GiB data, 18 GiB used, 518 GiB / 536 GiB avail; 0 B/s rd, 0 B/s wr, 1.34k op/s 2024-09-10T19:02:12.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:12 smithi018 ceph-mon[40196]: pgmap v784: 168 pgs: 168 active+clean; 4.1 GiB data, 18 GiB used, 518 GiB / 536 GiB avail; 0 B/s rd, 0 B/s wr, 1.34k op/s 2024-09-10T19:02:12.945 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:12.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:12.946 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:13.046 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:13.077 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:13.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:13.078 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:13.158 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T19:02:13.179 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:13.221 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:13.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:13.223 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:13.323 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:13.365 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:13.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:13.366 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:13.467 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:13.498 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:13.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:13.499 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:13.600 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:13.648 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:13.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:13.649 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:13.668 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:13 smithi018 ceph-mon[40196]: 4.1f scrub starts 2024-09-10T19:02:13.668 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:13 smithi018 ceph-mon[40196]: 4.1f scrub ok 2024-09-10T19:02:13.668 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:13 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:02:13.668 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:13 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:02:13.668 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:13 smithi018 ceph-mon[40196]: 4.4 scrub starts 2024-09-10T19:02:13.668 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:13 smithi018 ceph-mon[40196]: 4.4 scrub ok 2024-09-10T19:02:13.669 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:13 smithi018 ceph-mon[40196]: osdmap e364: 8 total, 7 up, 6 in 2024-09-10T19:02:13.669 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:13 smithi018 ceph-mon[29336]: 4.1f scrub starts 2024-09-10T19:02:13.669 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:13 smithi018 ceph-mon[29336]: 4.1f scrub ok 2024-09-10T19:02:13.669 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:13 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:02:13.669 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:13 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:02:13.669 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:13 smithi018 ceph-mon[29336]: 4.4 scrub starts 2024-09-10T19:02:13.669 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:13 smithi018 ceph-mon[29336]: 4.4 scrub ok 2024-09-10T19:02:13.669 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:13 smithi018 ceph-mon[29336]: osdmap e364: 8 total, 7 up, 6 in 2024-09-10T19:02:13.716 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T19:02:13.717 INFO:teuthology.orchestra.run.smithi018.stdout: "ops": [], 2024-09-10T19:02:13.717 INFO:teuthology.orchestra.run.smithi018.stdout: "complaint_time": 30, 2024-09-10T19:02:13.717 INFO:teuthology.orchestra.run.smithi018.stdout: "num_blocked_ops": 0 2024-09-10T19:02:13.717 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T19:02:13.750 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:13.790 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:13.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:13.792 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:13.892 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:13.927 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:13.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:13.928 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:13.972 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:13 smithi103 ceph-mon[33163]: 4.1f scrub starts 2024-09-10T19:02:13.972 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:13 smithi103 ceph-mon[33163]: 4.1f scrub ok 2024-09-10T19:02:13.972 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:13 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:02:13.973 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:13 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:02:13.973 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:13 smithi103 ceph-mon[33163]: 4.4 scrub starts 2024-09-10T19:02:13.973 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:13 smithi103 ceph-mon[33163]: 4.4 scrub ok 2024-09-10T19:02:13.973 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:13 smithi103 ceph-mon[33163]: osdmap e364: 8 total, 7 up, 6 in 2024-09-10T19:02:14.029 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:14.052 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:14.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:14.053 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:14.154 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:14.189 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:14.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:14.190 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:14.291 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:14.326 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:14.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:14.327 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:14.428 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:14.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:14.459 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:14.460 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:14.560 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:14.596 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:14.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:14.597 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:14.698 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:14.709 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:02:14.733 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:14.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:14.734 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:14.834 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:14.875 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:14.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:14.876 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:14.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:14 smithi018 ceph-mon[40196]: 2.1c scrub starts 2024-09-10T19:02:14.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:14 smithi018 ceph-mon[40196]: 2.1c scrub ok 2024-09-10T19:02:14.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:14 smithi018 ceph-mon[40196]: 1.f scrub starts 2024-09-10T19:02:14.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:14 smithi018 ceph-mon[40196]: 1.f scrub ok 2024-09-10T19:02:14.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:14 smithi018 ceph-mon[40196]: pgmap v786: 168 pgs: 168 active+clean; 3.8 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 1.35k op/s 2024-09-10T19:02:14.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:14 smithi018 ceph-mon[40196]: 2.8 scrub starts 2024-09-10T19:02:14.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:14 smithi018 ceph-mon[40196]: 2.8 scrub ok 2024-09-10T19:02:14.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:14 smithi018 ceph-mon[40196]: 6.15 scrub starts 2024-09-10T19:02:14.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:14 smithi018 ceph-mon[40196]: 6.15 scrub ok 2024-09-10T19:02:14.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:14 smithi018 ceph-mon[40196]: 5.10 scrub starts 2024-09-10T19:02:14.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:14 smithi018 ceph-mon[40196]: 5.10 scrub ok 2024-09-10T19:02:14.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:14 smithi018 ceph-mon[29336]: 2.1c scrub starts 2024-09-10T19:02:14.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:14 smithi018 ceph-mon[29336]: 2.1c scrub ok 2024-09-10T19:02:14.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:14 smithi018 ceph-mon[29336]: 1.f scrub starts 2024-09-10T19:02:14.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:14 smithi018 ceph-mon[29336]: 1.f scrub ok 2024-09-10T19:02:14.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:14 smithi018 ceph-mon[29336]: pgmap v786: 168 pgs: 168 active+clean; 3.8 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 1.35k op/s 2024-09-10T19:02:14.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:14 smithi018 ceph-mon[29336]: 2.8 scrub starts 2024-09-10T19:02:14.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:14 smithi018 ceph-mon[29336]: 2.8 scrub ok 2024-09-10T19:02:14.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:14 smithi018 ceph-mon[29336]: 6.15 scrub starts 2024-09-10T19:02:14.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:14 smithi018 ceph-mon[29336]: 6.15 scrub ok 2024-09-10T19:02:14.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:14 smithi018 ceph-mon[29336]: 5.10 scrub starts 2024-09-10T19:02:14.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:14 smithi018 ceph-mon[29336]: 5.10 scrub ok 2024-09-10T19:02:14.976 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:15.010 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:15.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:15.011 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:15.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:14 smithi103 ceph-mon[33163]: 2.1c scrub starts 2024-09-10T19:02:15.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:14 smithi103 ceph-mon[33163]: 2.1c scrub ok 2024-09-10T19:02:15.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:14 smithi103 ceph-mon[33163]: 1.f scrub starts 2024-09-10T19:02:15.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:14 smithi103 ceph-mon[33163]: 1.f scrub ok 2024-09-10T19:02:15.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:14 smithi103 ceph-mon[33163]: pgmap v786: 168 pgs: 168 active+clean; 3.8 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 1.35k op/s 2024-09-10T19:02:15.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:14 smithi103 ceph-mon[33163]: 2.8 scrub starts 2024-09-10T19:02:15.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:14 smithi103 ceph-mon[33163]: 2.8 scrub ok 2024-09-10T19:02:15.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:14 smithi103 ceph-mon[33163]: 6.15 scrub starts 2024-09-10T19:02:15.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:14 smithi103 ceph-mon[33163]: 6.15 scrub ok 2024-09-10T19:02:15.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:14 smithi103 ceph-mon[33163]: 5.10 scrub starts 2024-09-10T19:02:15.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:14 smithi103 ceph-mon[33163]: 5.10 scrub ok 2024-09-10T19:02:15.112 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:15.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:15.145 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:15.145 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:15.188 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-10T19:02:15.192 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-10T19:02:15.246 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:15.284 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:15.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:15.285 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:15.386 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:15.453 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:15.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:15.454 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:15.555 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:15.606 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:15.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:15.607 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:15.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:15 smithi018 ceph-mon[40196]: 2.2 scrub starts 2024-09-10T19:02:15.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:15 smithi018 ceph-mon[40196]: 2.2 scrub ok 2024-09-10T19:02:15.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:15 smithi018 ceph-mon[40196]: 6.0 scrub starts 2024-09-10T19:02:15.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:15 smithi018 ceph-mon[40196]: 6.0 scrub ok 2024-09-10T19:02:15.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:15 smithi018 ceph-mon[40196]: 6.14 deep-scrub starts 2024-09-10T19:02:15.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:15 smithi018 ceph-mon[40196]: 6.14 deep-scrub ok 2024-09-10T19:02:15.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:15 smithi018 ceph-mon[29336]: 2.2 scrub starts 2024-09-10T19:02:15.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:15 smithi018 ceph-mon[29336]: 2.2 scrub ok 2024-09-10T19:02:15.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:15 smithi018 ceph-mon[29336]: 6.0 scrub starts 2024-09-10T19:02:15.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:15 smithi018 ceph-mon[29336]: 6.0 scrub ok 2024-09-10T19:02:15.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:15 smithi018 ceph-mon[29336]: 6.14 deep-scrub starts 2024-09-10T19:02:15.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:15 smithi018 ceph-mon[29336]: 6.14 deep-scrub ok 2024-09-10T19:02:15.708 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:15.747 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:15.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:15.748 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:15.849 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:15.887 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:15.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:15.888 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:15.989 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:16.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:15 smithi103 ceph-mon[33163]: 2.2 scrub starts 2024-09-10T19:02:16.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:15 smithi103 ceph-mon[33163]: 2.2 scrub ok 2024-09-10T19:02:16.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:15 smithi103 ceph-mon[33163]: 6.0 scrub starts 2024-09-10T19:02:16.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:15 smithi103 ceph-mon[33163]: 6.0 scrub ok 2024-09-10T19:02:16.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:15 smithi103 ceph-mon[33163]: 6.14 deep-scrub starts 2024-09-10T19:02:16.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:15 smithi103 ceph-mon[33163]: 6.14 deep-scrub ok 2024-09-10T19:02:16.025 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:16.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:16.026 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:16.127 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:16.164 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:16.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:16.165 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:16.266 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:16.302 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:16.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:16.303 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:16.404 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:16.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:16.440 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:16.440 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:16.541 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:16.579 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:16.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:16.580 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:16.681 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:16.712 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:16.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:16.713 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:16.814 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:16.847 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:16.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:16.849 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:16.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:16 smithi103 ceph-mon[33163]: 5.12 scrub starts 2024-09-10T19:02:16.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:16 smithi103 ceph-mon[33163]: 5.12 scrub ok 2024-09-10T19:02:16.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:16 smithi103 ceph-mon[33163]: pgmap v787: 168 pgs: 168 active+clean; 3.7 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 652 B/s rd, 0 B/s wr, 1.22k op/s 2024-09-10T19:02:16.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:16 smithi103 ceph-mon[33163]: 6.1c deep-scrub starts 2024-09-10T19:02:16.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:16 smithi103 ceph-mon[33163]: 6.1c deep-scrub ok 2024-09-10T19:02:16.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:16 smithi103 ceph-mon[33163]: 4.5 scrub starts 2024-09-10T19:02:16.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:16 smithi103 ceph-mon[33163]: 4.5 scrub ok 2024-09-10T19:02:16.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:16 smithi018 ceph-mon[40196]: 5.12 scrub starts 2024-09-10T19:02:16.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:16 smithi018 ceph-mon[40196]: 5.12 scrub ok 2024-09-10T19:02:16.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:16 smithi018 ceph-mon[40196]: pgmap v787: 168 pgs: 168 active+clean; 3.7 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 652 B/s rd, 0 B/s wr, 1.22k op/s 2024-09-10T19:02:16.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:16 smithi018 ceph-mon[40196]: 6.1c deep-scrub starts 2024-09-10T19:02:16.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:16 smithi018 ceph-mon[40196]: 6.1c deep-scrub ok 2024-09-10T19:02:16.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:16 smithi018 ceph-mon[40196]: 4.5 scrub starts 2024-09-10T19:02:16.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:16 smithi018 ceph-mon[40196]: 4.5 scrub ok 2024-09-10T19:02:16.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:16 smithi018 ceph-mon[29336]: 5.12 scrub starts 2024-09-10T19:02:16.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:16 smithi018 ceph-mon[29336]: 5.12 scrub ok 2024-09-10T19:02:16.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:16 smithi018 ceph-mon[29336]: pgmap v787: 168 pgs: 168 active+clean; 3.7 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 652 B/s rd, 0 B/s wr, 1.22k op/s 2024-09-10T19:02:16.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:16 smithi018 ceph-mon[29336]: 6.1c deep-scrub starts 2024-09-10T19:02:16.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:16 smithi018 ceph-mon[29336]: 6.1c deep-scrub ok 2024-09-10T19:02:16.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:16 smithi018 ceph-mon[29336]: 4.5 scrub starts 2024-09-10T19:02:16.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:16 smithi018 ceph-mon[29336]: 4.5 scrub ok 2024-09-10T19:02:16.949 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:16.986 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:16.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:16.987 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:17.087 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:17.109 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:17.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:17.110 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:17.211 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:17.245 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:17.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:17.248 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:17.272 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:02:17 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:02:17.349 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:17.385 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:17.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:17.386 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:17.487 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:17.522 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:17.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:17.523 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:17.624 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:17.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:17.661 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:17.661 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:17.761 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:02:17.762 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:17.796 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:17.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:17.797 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:17.815 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:02:17.897 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:17.906 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:17 smithi018 ceph-mon[29336]: 6.13 deep-scrub starts 2024-09-10T19:02:17.906 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:17 smithi018 ceph-mon[29336]: 6.13 deep-scrub ok 2024-09-10T19:02:17.906 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:17 smithi018 ceph-mon[29336]: 2.16 scrub starts 2024-09-10T19:02:17.906 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:17 smithi018 ceph-mon[29336]: 2.16 scrub ok 2024-09-10T19:02:17.906 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:17 smithi018 ceph-mon[29336]: 5.2 scrub starts 2024-09-10T19:02:17.906 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:17 smithi018 ceph-mon[29336]: 5.2 scrub ok 2024-09-10T19:02:17.906 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:17 smithi018 ceph-mon[29336]: 2.1e scrub starts 2024-09-10T19:02:17.906 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:17 smithi018 ceph-mon[29336]: 2.1e scrub ok 2024-09-10T19:02:17.906 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:17 smithi018 ceph-mon[40196]: 6.13 deep-scrub starts 2024-09-10T19:02:17.907 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:17 smithi018 ceph-mon[40196]: 6.13 deep-scrub ok 2024-09-10T19:02:17.907 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:17 smithi018 ceph-mon[40196]: 2.16 scrub starts 2024-09-10T19:02:17.907 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:17 smithi018 ceph-mon[40196]: 2.16 scrub ok 2024-09-10T19:02:17.907 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:17 smithi018 ceph-mon[40196]: 5.2 scrub starts 2024-09-10T19:02:17.907 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:17 smithi018 ceph-mon[40196]: 5.2 scrub ok 2024-09-10T19:02:17.907 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:17 smithi018 ceph-mon[40196]: 2.1e scrub starts 2024-09-10T19:02:17.907 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:17 smithi018 ceph-mon[40196]: 2.1e scrub ok 2024-09-10T19:02:17.942 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:17.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:17.943 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:18.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:17 smithi103 ceph-mon[33163]: 6.13 deep-scrub starts 2024-09-10T19:02:18.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:17 smithi103 ceph-mon[33163]: 6.13 deep-scrub ok 2024-09-10T19:02:18.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:17 smithi103 ceph-mon[33163]: 2.16 scrub starts 2024-09-10T19:02:18.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:17 smithi103 ceph-mon[33163]: 2.16 scrub ok 2024-09-10T19:02:18.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:17 smithi103 ceph-mon[33163]: 5.2 scrub starts 2024-09-10T19:02:18.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:17 smithi103 ceph-mon[33163]: 5.2 scrub ok 2024-09-10T19:02:18.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:17 smithi103 ceph-mon[33163]: 2.1e scrub starts 2024-09-10T19:02:18.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:17 smithi103 ceph-mon[33163]: 2.1e scrub ok 2024-09-10T19:02:18.043 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:18.077 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:18.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:18.078 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:18.179 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:18.210 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:18.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:18.211 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:18.312 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:18.344 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:18.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:18.345 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:18.448 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:18.480 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:18.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:18.481 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:18.582 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:18.616 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:18.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:18.617 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:18.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:18 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:18.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:18 smithi018 ceph-mon[40196]: 1.b deep-scrub starts 2024-09-10T19:02:18.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:18 smithi018 ceph-mon[40196]: 1.b deep-scrub ok 2024-09-10T19:02:18.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:18 smithi018 ceph-mon[40196]: pgmap v788: 168 pgs: 168 active+clean; 3.3 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.79k op/s 2024-09-10T19:02:18.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:18 smithi018 ceph-mon[40196]: 1.11 scrub starts 2024-09-10T19:02:18.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:18 smithi018 ceph-mon[40196]: 1.11 scrub ok 2024-09-10T19:02:18.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:18 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:02:18.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:18 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:02:18.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:18 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:18.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:18 smithi018 ceph-mon[29336]: 1.b deep-scrub starts 2024-09-10T19:02:18.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:18 smithi018 ceph-mon[29336]: 1.b deep-scrub ok 2024-09-10T19:02:18.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:18 smithi018 ceph-mon[29336]: pgmap v788: 168 pgs: 168 active+clean; 3.3 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.79k op/s 2024-09-10T19:02:18.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:18 smithi018 ceph-mon[29336]: 1.11 scrub starts 2024-09-10T19:02:18.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:18 smithi018 ceph-mon[29336]: 1.11 scrub ok 2024-09-10T19:02:18.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:18 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:02:18.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:18 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:02:18.718 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:18.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:18.755 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:18.755 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:18.856 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:18.896 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:18.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:18.897 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:18.998 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:19.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:18 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:19.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:18 smithi103 ceph-mon[33163]: 1.b deep-scrub starts 2024-09-10T19:02:19.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:18 smithi103 ceph-mon[33163]: 1.b deep-scrub ok 2024-09-10T19:02:19.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:18 smithi103 ceph-mon[33163]: pgmap v788: 168 pgs: 168 active+clean; 3.3 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.79k op/s 2024-09-10T19:02:19.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:18 smithi103 ceph-mon[33163]: 1.11 scrub starts 2024-09-10T19:02:19.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:18 smithi103 ceph-mon[33163]: 1.11 scrub ok 2024-09-10T19:02:19.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:18 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:02:19.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:18 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:02:19.040 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:19.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:19.041 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:19.141 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:19.175 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:19.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:19.176 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:19.277 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:19.313 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:19.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:19.314 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:19.414 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:19.450 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:19.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:19.451 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:19.552 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:19.586 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:19.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:19.587 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:19.688 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:19.720 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:19.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:19.721 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:19.822 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:19.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:19.861 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:19.861 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:19.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:19 smithi018 ceph-mon[40196]: osdmap e365: 8 total, 7 up, 6 in 2024-09-10T19:02:19.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:19 smithi018 ceph-mon[40196]: 4.8 deep-scrub starts 2024-09-10T19:02:19.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:19 smithi018 ceph-mon[40196]: 4.8 deep-scrub ok 2024-09-10T19:02:19.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:19 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2020642095' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:02:19.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:19 smithi018 ceph-mon[40196]: 2.11 scrub starts 2024-09-10T19:02:19.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:19 smithi018 ceph-mon[40196]: 2.11 scrub ok 2024-09-10T19:02:19.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:19 smithi018 ceph-mon[40196]: 2.15 scrub starts 2024-09-10T19:02:19.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:19 smithi018 ceph-mon[40196]: 2.15 scrub ok 2024-09-10T19:02:19.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:19 smithi018 ceph-mon[40196]: pgmap v790: 168 pgs: 168 active+clean; 3.2 GiB data, 16 GiB used, 521 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.85k op/s 2024-09-10T19:02:19.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:19 smithi018 ceph-mon[29336]: osdmap e365: 8 total, 7 up, 6 in 2024-09-10T19:02:19.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:19 smithi018 ceph-mon[29336]: 4.8 deep-scrub starts 2024-09-10T19:02:19.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:19 smithi018 ceph-mon[29336]: 4.8 deep-scrub ok 2024-09-10T19:02:19.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:19 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2020642095' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:02:19.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:19 smithi018 ceph-mon[29336]: 2.11 scrub starts 2024-09-10T19:02:19.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:19 smithi018 ceph-mon[29336]: 2.11 scrub ok 2024-09-10T19:02:19.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:19 smithi018 ceph-mon[29336]: 2.15 scrub starts 2024-09-10T19:02:19.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:19 smithi018 ceph-mon[29336]: 2.15 scrub ok 2024-09-10T19:02:19.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:19 smithi018 ceph-mon[29336]: pgmap v790: 168 pgs: 168 active+clean; 3.2 GiB data, 16 GiB used, 521 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.85k op/s 2024-09-10T19:02:19.962 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:19.999 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:19 smithi103 ceph-mon[33163]: osdmap e365: 8 total, 7 up, 6 in 2024-09-10T19:02:19.999 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:19 smithi103 ceph-mon[33163]: 4.8 deep-scrub starts 2024-09-10T19:02:19.999 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:19 smithi103 ceph-mon[33163]: 4.8 deep-scrub ok 2024-09-10T19:02:19.999 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:19 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2020642095' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:02:19.999 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:19 smithi103 ceph-mon[33163]: 2.11 scrub starts 2024-09-10T19:02:19.999 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:19 smithi103 ceph-mon[33163]: 2.11 scrub ok 2024-09-10T19:02:20.000 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:19 smithi103 ceph-mon[33163]: 2.15 scrub starts 2024-09-10T19:02:20.000 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:19 smithi103 ceph-mon[33163]: 2.15 scrub ok 2024-09-10T19:02:20.000 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:19 smithi103 ceph-mon[33163]: pgmap v790: 168 pgs: 168 active+clean; 3.2 GiB data, 16 GiB used, 521 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.85k op/s 2024-09-10T19:02:20.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:20.012 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:20.012 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:20.113 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:20.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:20.148 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:20.148 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:20.249 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:20.272 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:02:20 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:02:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:02:20.289 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:20.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:20.290 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:20.391 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:20.415 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T19:02:20.415 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T19:02:20.416 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T19:02:20.416 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T19:02:20.416 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T19:02:20.416 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T19:02:20.416 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T19:02:20.416 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T19:02:20.416 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T19:02:20.416 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T19:02:20.416 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T19:02:20.416 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T19:02:20.416 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T19:02:20.417 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T19:02:20.417 INFO:teuthology.orchestra.run.smithi018.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-10T19:02:20.426 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:20.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:20.427 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:20.528 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:20.567 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:20.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:20.568 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:20.669 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:20.703 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:20.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:20.704 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:20.805 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:20.837 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:20.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:20.838 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:20.938 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:20.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:20.977 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:20.977 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:21.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:20 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:02:21.024 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:20 smithi103 ceph-mon[33163]: 4.19 scrub starts 2024-09-10T19:02:21.024 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:20 smithi103 ceph-mon[33163]: 4.19 scrub ok 2024-09-10T19:02:21.024 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:20 smithi103 ceph-mon[33163]: 6.4 scrub starts 2024-09-10T19:02:21.024 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:20 smithi103 ceph-mon[33163]: 6.4 scrub ok 2024-09-10T19:02:21.059 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:20 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:02:21.059 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:20 smithi018 ceph-mon[40196]: 4.19 scrub starts 2024-09-10T19:02:21.059 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:20 smithi018 ceph-mon[40196]: 4.19 scrub ok 2024-09-10T19:02:21.059 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:20 smithi018 ceph-mon[40196]: 6.4 scrub starts 2024-09-10T19:02:21.060 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:20 smithi018 ceph-mon[40196]: 6.4 scrub ok 2024-09-10T19:02:21.060 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:20 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:02:21.060 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:20 smithi018 ceph-mon[29336]: 4.19 scrub starts 2024-09-10T19:02:21.060 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:20 smithi018 ceph-mon[29336]: 4.19 scrub ok 2024-09-10T19:02:21.060 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:20 smithi018 ceph-mon[29336]: 6.4 scrub starts 2024-09-10T19:02:21.060 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:20 smithi018 ceph-mon[29336]: 6.4 scrub ok 2024-09-10T19:02:21.060 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:02:20 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:02:20] "GET /metrics HTTP/1.1" 200 209516 "" "Prometheus/2.43.0" 2024-09-10T19:02:21.078 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:21.109 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:21.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:21.110 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:21.200 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-10T19:02:21.201 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-10T19:02:21.211 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:21.248 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:21.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:21.249 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:21.350 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:21.394 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:21.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:21.395 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:21.495 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:21.516 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:21.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:21.517 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:21.618 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:21.665 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:21.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:21.666 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:21.767 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:21.814 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:21.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:21.815 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:21.916 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:21.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:21.952 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:21.952 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:22.053 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:22.089 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:22.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:22.090 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:22.191 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:22.230 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:22.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:22.231 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:22.332 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:22.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:22.374 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:22.374 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:22.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:22 smithi018 ceph-mon[40196]: 2.12 scrub starts 2024-09-10T19:02:22.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:22 smithi018 ceph-mon[40196]: 2.12 scrub ok 2024-09-10T19:02:22.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:22 smithi018 ceph-mon[40196]: 6.1b scrub starts 2024-09-10T19:02:22.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:22 smithi018 ceph-mon[40196]: 6.1b scrub ok 2024-09-10T19:02:22.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:22 smithi018 ceph-mon[40196]: 2.4 scrub starts 2024-09-10T19:02:22.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:22 smithi018 ceph-mon[40196]: 2.4 scrub ok 2024-09-10T19:02:22.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:22 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:02:22.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:22 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:02:22.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:22 smithi018 ceph-mon[40196]: pgmap v791: 168 pgs: 168 active+clean; 3.1 GiB data, 15 GiB used, 521 GiB / 536 GiB avail; 591 B/s rd, 0 B/s wr, 1.18k op/s 2024-09-10T19:02:22.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:22 smithi018 ceph-mon[40196]: 1.6 scrub starts 2024-09-10T19:02:22.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:22 smithi018 ceph-mon[40196]: 1.6 scrub ok 2024-09-10T19:02:22.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:22 smithi018 ceph-mon[29336]: 2.12 scrub starts 2024-09-10T19:02:22.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:22 smithi018 ceph-mon[29336]: 2.12 scrub ok 2024-09-10T19:02:22.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:22 smithi018 ceph-mon[29336]: 6.1b scrub starts 2024-09-10T19:02:22.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:22 smithi018 ceph-mon[29336]: 6.1b scrub ok 2024-09-10T19:02:22.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:22 smithi018 ceph-mon[29336]: 2.4 scrub starts 2024-09-10T19:02:22.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:22 smithi018 ceph-mon[29336]: 2.4 scrub ok 2024-09-10T19:02:22.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:22 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:02:22.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:22 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:02:22.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:22 smithi018 ceph-mon[29336]: pgmap v791: 168 pgs: 168 active+clean; 3.1 GiB data, 15 GiB used, 521 GiB / 536 GiB avail; 591 B/s rd, 0 B/s wr, 1.18k op/s 2024-09-10T19:02:22.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:22 smithi018 ceph-mon[29336]: 1.6 scrub starts 2024-09-10T19:02:22.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:22 smithi018 ceph-mon[29336]: 1.6 scrub ok 2024-09-10T19:02:22.476 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:22.509 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:22.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:22.510 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:22.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:22 smithi103 ceph-mon[33163]: 2.12 scrub starts 2024-09-10T19:02:22.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:22 smithi103 ceph-mon[33163]: 2.12 scrub ok 2024-09-10T19:02:22.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:22 smithi103 ceph-mon[33163]: 6.1b scrub starts 2024-09-10T19:02:22.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:22 smithi103 ceph-mon[33163]: 6.1b scrub ok 2024-09-10T19:02:22.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:22 smithi103 ceph-mon[33163]: 2.4 scrub starts 2024-09-10T19:02:22.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:22 smithi103 ceph-mon[33163]: 2.4 scrub ok 2024-09-10T19:02:22.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:22 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:02:22.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:22 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:02:22.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:22 smithi103 ceph-mon[33163]: pgmap v791: 168 pgs: 168 active+clean; 3.1 GiB data, 15 GiB used, 521 GiB / 536 GiB avail; 591 B/s rd, 0 B/s wr, 1.18k op/s 2024-09-10T19:02:22.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:22 smithi103 ceph-mon[33163]: 1.6 scrub starts 2024-09-10T19:02:22.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:22 smithi103 ceph-mon[33163]: 1.6 scrub ok 2024-09-10T19:02:22.611 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:22.649 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:22.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:22.651 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:22.751 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:22.787 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:22.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:22.788 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:22.812 INFO:tasks.thrashosds.thrasher:No PGs found for osd.7 2024-09-10T19:02:22.889 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:22.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:22.926 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:22.926 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:23.027 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:23.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:23.065 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:23.065 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:23.166 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:23.226 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T19:02:23.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:23.282 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:23.282 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:23.344 INFO:teuthology.orchestra.run.smithi018.stdout:ERROR: (22) Invalid argument 2024-09-10T19:02:23.344 INFO:teuthology.orchestra.run.smithi018.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T19:02:23.383 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:23.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:23.432 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:23.432 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:23.533 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:23.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:23.579 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:23.580 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:23.680 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:23.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:23 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:02:23.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:23 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:02:23.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:23 smithi018 ceph-mon[29336]: 4.1a scrub starts 2024-09-10T19:02:23.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:23 smithi018 ceph-mon[29336]: 4.1a scrub ok 2024-09-10T19:02:23.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:23 smithi018 ceph-mon[29336]: 5.13 scrub starts 2024-09-10T19:02:23.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:23 smithi018 ceph-mon[29336]: 5.13 scrub ok 2024-09-10T19:02:23.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:23 smithi018 ceph-mon[29336]: osdmap e366: 8 total, 7 up, 6 in 2024-09-10T19:02:23.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:23 smithi018 ceph-mon[29336]: 6.16 scrub starts 2024-09-10T19:02:23.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:23 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:02:23.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:23 smithi018 ceph-mon[29336]: 6.16 scrub ok 2024-09-10T19:02:23.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:23 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:02:23.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:23 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:02:23.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:23 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:02:23.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:23 smithi018 ceph-mon[40196]: 4.1a scrub starts 2024-09-10T19:02:23.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:23 smithi018 ceph-mon[40196]: 4.1a scrub ok 2024-09-10T19:02:23.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:23 smithi018 ceph-mon[40196]: 5.13 scrub starts 2024-09-10T19:02:23.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:23 smithi018 ceph-mon[40196]: 5.13 scrub ok 2024-09-10T19:02:23.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:23 smithi018 ceph-mon[40196]: osdmap e366: 8 total, 7 up, 6 in 2024-09-10T19:02:23.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:23 smithi018 ceph-mon[40196]: 6.16 scrub starts 2024-09-10T19:02:23.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:23 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:02:23.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:23 smithi018 ceph-mon[40196]: 6.16 scrub ok 2024-09-10T19:02:23.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:23 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:02:23.715 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:23.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:23.716 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:23.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:23 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:02:23.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:23 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:02:23.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:23 smithi103 ceph-mon[33163]: 4.1a scrub starts 2024-09-10T19:02:23.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:23 smithi103 ceph-mon[33163]: 4.1a scrub ok 2024-09-10T19:02:23.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:23 smithi103 ceph-mon[33163]: 5.13 scrub starts 2024-09-10T19:02:23.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:23 smithi103 ceph-mon[33163]: 5.13 scrub ok 2024-09-10T19:02:23.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:23 smithi103 ceph-mon[33163]: osdmap e366: 8 total, 7 up, 6 in 2024-09-10T19:02:23.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:23 smithi103 ceph-mon[33163]: 6.16 scrub starts 2024-09-10T19:02:23.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:23 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:02:23.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:23 smithi103 ceph-mon[33163]: 6.16 scrub ok 2024-09-10T19:02:23.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:23 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:02:23.818 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:23.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:23.858 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:23.858 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:23.958 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:23.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:23.994 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:23.995 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:24.095 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:24.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:24.135 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:24.135 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:24.236 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:24.242 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-10T19:02:24.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:24.273 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:24.274 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:24.325 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is set 2024-09-10T19:02:24.375 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:24.415 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:24.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:24.416 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:24.517 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:24.554 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:24.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:24.555 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:24.656 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:24.684 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:24.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:24.685 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:24.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:24 smithi018 ceph-mon[40196]: 5.17 scrub starts 2024-09-10T19:02:24.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:24 smithi018 ceph-mon[40196]: 5.17 scrub ok 2024-09-10T19:02:24.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:24 smithi018 ceph-mon[40196]: 6.1d scrub starts 2024-09-10T19:02:24.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:24 smithi018 ceph-mon[40196]: 6.1d scrub ok 2024-09-10T19:02:24.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:24 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/43333569' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T19:02:24.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:24 smithi018 ceph-mon[40196]: pgmap v793: 168 pgs: 168 active+clean; 2.8 GiB data, 15 GiB used, 522 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.78k op/s 2024-09-10T19:02:24.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:24 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:02:24.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:24 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:02:24.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:24 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:02:24.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:24 smithi018 ceph-mon[29336]: 5.17 scrub starts 2024-09-10T19:02:24.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:24 smithi018 ceph-mon[29336]: 5.17 scrub ok 2024-09-10T19:02:24.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:24 smithi018 ceph-mon[29336]: 6.1d scrub starts 2024-09-10T19:02:24.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:24 smithi018 ceph-mon[29336]: 6.1d scrub ok 2024-09-10T19:02:24.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:24 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/43333569' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T19:02:24.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:24 smithi018 ceph-mon[29336]: pgmap v793: 168 pgs: 168 active+clean; 2.8 GiB data, 15 GiB used, 522 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.78k op/s 2024-09-10T19:02:24.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:24 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:02:24.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:24 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:02:24.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:24 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:02:24.731 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:02:24.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:24 smithi103 ceph-mon[33163]: 5.17 scrub starts 2024-09-10T19:02:24.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:24 smithi103 ceph-mon[33163]: 5.17 scrub ok 2024-09-10T19:02:24.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:24 smithi103 ceph-mon[33163]: 6.1d scrub starts 2024-09-10T19:02:24.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:24 smithi103 ceph-mon[33163]: 6.1d scrub ok 2024-09-10T19:02:24.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:24 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/43333569' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T19:02:24.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:24 smithi103 ceph-mon[33163]: pgmap v793: 168 pgs: 168 active+clean; 2.8 GiB data, 15 GiB used, 522 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.78k op/s 2024-09-10T19:02:24.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:24 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:02:24.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:24 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:02:24.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:24 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:02:24.786 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:24.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:24.822 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:24.822 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:24.923 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:24.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:24.959 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:24.960 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:25.061 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:25.088 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:25.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:25.089 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:25.190 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:25.225 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:25.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:25.226 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:25.326 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:25.362 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:25.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:25.364 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:25.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:25 smithi018 ceph-mon[40196]: 1.a scrub starts 2024-09-10T19:02:25.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:25 smithi018 ceph-mon[40196]: 1.a scrub ok 2024-09-10T19:02:25.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:25 smithi018 ceph-mon[40196]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:02:25.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:25 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/43333569' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T19:02:25.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:25 smithi018 ceph-mon[40196]: osdmap e367: 8 total, 7 up, 6 in 2024-09-10T19:02:25.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:25 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:02:25.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:25 smithi018 ceph-mon[29336]: 1.a scrub starts 2024-09-10T19:02:25.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:25 smithi018 ceph-mon[29336]: 1.a scrub ok 2024-09-10T19:02:25.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:25 smithi018 ceph-mon[29336]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:02:25.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:25 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/43333569' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T19:02:25.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:25 smithi018 ceph-mon[29336]: osdmap e367: 8 total, 7 up, 6 in 2024-09-10T19:02:25.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:25 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:02:25.445 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:02:25 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T19:02:25.088+0000 7f66d46e1640 -1 mgr get_metadata_python Requested missing service osd.7 2024-09-10T19:02:25.464 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:25.491 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:25.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:25.492 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:25.499 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:02:25.593 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:25.629 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:25.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:25.630 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:25.731 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:25.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:25.767 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:25.767 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:25.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:25 smithi103 ceph-mon[33163]: 1.a scrub starts 2024-09-10T19:02:25.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:25 smithi103 ceph-mon[33163]: 1.a scrub ok 2024-09-10T19:02:25.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:25 smithi103 ceph-mon[33163]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:02:25.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:25 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/43333569' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T19:02:25.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:25 smithi103 ceph-mon[33163]: osdmap e367: 8 total, 7 up, 6 in 2024-09-10T19:02:25.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:25 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:02:25.868 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:25.904 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:25.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:25.905 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:26.006 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:26.039 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:26.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:26.040 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:26.140 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:26.178 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:26.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:26.179 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:26.280 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:26.316 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:26.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:26.317 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:26.418 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:26.452 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:26.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:26.453 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:26.523 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:26 smithi018 ceph-mon[40196]: pgmap v795: 168 pgs: 168 active+clean; 2.7 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 754 B/s rd, 0 B/s wr, 1.39k op/s 2024-09-10T19:02:26.523 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:26 smithi018 ceph-mon[29336]: pgmap v795: 168 pgs: 168 active+clean; 2.7 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 754 B/s rd, 0 B/s wr, 1.39k op/s 2024-09-10T19:02:26.554 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:26.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:26.588 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:26.589 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:26.690 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:26.736 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:26.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:26.737 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:26.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:26 smithi103 ceph-mon[33163]: pgmap v795: 168 pgs: 168 active+clean; 2.7 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 754 B/s rd, 0 B/s wr, 1.39k op/s 2024-09-10T19:02:26.837 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:26.865 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:26.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:26.866 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:26.967 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:27.002 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:27.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:27.003 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:27.104 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:27.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:27.147 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:27.148 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:27.248 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:27.286 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:27.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:27.287 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:27.327 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:02:27 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:02:27.388 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:27.426 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:27.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:27.427 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:27.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:27 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:02:27.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:27 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:02:27.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:27 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:02:27.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:27 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:02:27.527 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:27.575 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:27.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:27.576 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:27.677 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:27.709 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:27.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:27.710 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:27.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:27 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:02:27.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:27 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:02:27.811 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:27.813 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6] out_osds: [2, 7] dead_osds: [7] live_osds: [1, 2, 3, 4, 5, 6, 0] 2024-09-10T19:02:27.813 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T19:02:27.814 INFO:tasks.thrashosds.thrasher:Shrinking pool unique_pool_3 2024-09-10T19:02:27.814 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool unique_pool_3 by 10 min 8 2024-09-10T19:02:27.814 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T19:02:27.841 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:27.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:27.843 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:27.879 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T19:02:27.879 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T19:02:27.920 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T19:02:27.920 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T19:02:27.943 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:27.955 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T19:02:27.955 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T19:02:27.981 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-10T19:02:27.992 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T19:02:27.992 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T19:02:27.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:27.993 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:27.993 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:28.021 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T19:02:28.021 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T19:02:28.062 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T19:02:28.063 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T19:02:28.094 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:28.101 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T19:02:28.101 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T19:02:28.103 INFO:teuthology.orchestra.run.smithi018.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-10T19:02:28.119 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:28.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:28.120 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:28.221 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:28.251 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:28.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:28.252 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:28.353 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:28.399 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:28.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:28.400 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:28.500 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:28.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:28 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:28.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:28 smithi103 ceph-mon[33163]: osdmap e368: 8 total, 7 up, 6 in 2024-09-10T19:02:28.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:28 smithi103 ceph-mon[33163]: pgmap v797: 168 pgs: 168 active+clean; 2.3 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2.27k op/s 2024-09-10T19:02:28.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:28 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2039692876' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:02:28.536 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:28.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:28.537 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:28.638 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:28.674 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:28.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:28.675 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:28.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:28 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:28.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:28 smithi018 ceph-mon[40196]: osdmap e368: 8 total, 7 up, 6 in 2024-09-10T19:02:28.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:28 smithi018 ceph-mon[40196]: pgmap v797: 168 pgs: 168 active+clean; 2.3 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2.27k op/s 2024-09-10T19:02:28.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:28 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2039692876' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:02:28.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:28 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:28.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:28 smithi018 ceph-mon[29336]: osdmap e368: 8 total, 7 up, 6 in 2024-09-10T19:02:28.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:28 smithi018 ceph-mon[29336]: pgmap v797: 168 pgs: 168 active+clean; 2.3 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2.27k op/s 2024-09-10T19:02:28.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:28 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2039692876' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:02:28.775 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:28.799 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-10T19:02:28.799 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-10T19:02:28.819 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:28.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:28.822 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:28.875 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:02:28.912 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:02:28.923 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:28.964 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:28.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:28.965 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:29.066 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:29.099 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:29.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:29.101 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:29.201 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:29.238 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:29.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:29.239 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:29.340 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:29.372 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:29.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:29.373 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:29.474 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:29.532 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:29.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:29.533 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:29.634 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:29.664 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:29.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:29.665 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:29.763 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T19:02:29.763 INFO:teuthology.orchestra.run.smithi103.stdout: "ops": [ 2024-09-10T19:02:29.763 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:29.764 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_repop(client.15573.0:245459 10.6 e368/315 10:6a057d5b:::benchmark_data_smithi018_157943_object106801:head v 368'15306, mlcod=368'15302)", 2024-09-10T19:02:29.764 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:02:29.760627+0000", 2024-09-10T19:02:29.764 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0010209570000000001, 2024-09-10T19:02:29.764 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0010475459999999999, 2024-09-10T19:02:29.764 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:29.764 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "started", 2024-09-10T19:02:29.764 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:29.764 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:29.764 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:29.764 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:29.760627+0000", 2024-09-10T19:02:29.764 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:29.764 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:29.765 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:29.765 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:29.765 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:29.760627+0000", 2024-09-10T19:02:29.765 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:29.765 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:29.765 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:29.765 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:29.765 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:29.760626+0000", 2024-09-10T19:02:29.765 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999995 2024-09-10T19:02:29.765 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:29.765 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:29.765 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:29.766 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:29.760631+0000", 2024-09-10T19:02:29.766 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.3029999999999999e-06 2024-09-10T19:02:29.766 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:29.766 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:29.766 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:29.766 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:29.760640+0000", 2024-09-10T19:02:29.766 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.6049999999999998e-06 2024-09-10T19:02:29.766 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:29.766 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:29.766 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:29.766 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:29.760645+0000", 2024-09-10T19:02:29.766 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.1839999999999998e-06 2024-09-10T19:02:29.766 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:29.766 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:29.767 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:29.767 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:29.760751+0000", 2024-09-10T19:02:29.767 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00010614400000000001 2024-09-10T19:02:29.767 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:29.767 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:29.767 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:02:29.767 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:29.760806+0000", 2024-09-10T19:02:29.767 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.4830999999999997e-05 2024-09-10T19:02:29.767 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:29.767 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:29.767 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:29.767 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:29.767 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:29.768 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15573.0:245457 10.1 10:80e83a9f:::benchmark_data_smithi018_157943_object106799:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e368)", 2024-09-10T19:02:29.768 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:02:29.758206+0000", 2024-09-10T19:02:29.768 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0034418629999999999, 2024-09-10T19:02:29.768 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0035323279999999999, 2024-09-10T19:02:29.768 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:29.768 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "waiting for sub ops", 2024-09-10T19:02:29.768 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:29.768 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15573", 2024-09-10T19:02:29.768 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:02:29.768 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 245457 2024-09-10T19:02:29.768 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:29.768 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:29.769 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:29.769 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:29.769 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:29.758206+0000", 2024-09-10T19:02:29.769 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:29.769 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:29.769 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:29.769 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:29.769 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:29.758206+0000", 2024-09-10T19:02:29.769 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:29.769 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:29.769 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:29.769 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:29.770 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:29.758204+0000", 2024-09-10T19:02:29.770 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T19:02:29.770 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:29.770 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:29.770 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:29.770 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:29.758210+0000", 2024-09-10T19:02:29.770 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.7080000000000002e-06 2024-09-10T19:02:29.770 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:29.770 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:29.770 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:29.770 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:29.758221+0000", 2024-09-10T19:02:29.770 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0881e-05 2024-09-10T19:02:29.770 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:29.771 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:29.771 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:29.771 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:29.758227+0000", 2024-09-10T19:02:29.771 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.8289999999999996e-06 2024-09-10T19:02:29.771 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:29.771 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:29.771 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:29.771 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:29.758486+0000", 2024-09-10T19:02:29.771 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00025933199999999998 2024-09-10T19:02:29.771 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:29.771 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:29.771 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:02:29.771 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:29.758598+0000", 2024-09-10T19:02:29.772 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00011191700000000001 2024-09-10T19:02:29.772 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:29.772 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:29.772 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1", 2024-09-10T19:02:29.772 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:29.758698+0000", 2024-09-10T19:02:29.772 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000100664 2024-09-10T19:02:29.772 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:29.772 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:29.772 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:29.772 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:29.772 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:29.772 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_repop(client.15573.0:245456 10.3 e368/315 10:c2e49d6e:::benchmark_data_smithi018_157943_object106798:head v 368'15208, mlcod=368'15205)", 2024-09-10T19:02:29.772 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:02:29.759085+0000", 2024-09-10T19:02:29.772 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0025625299999999999, 2024-09-10T19:02:29.773 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0027203639999999999, 2024-09-10T19:02:29.773 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:29.773 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "started", 2024-09-10T19:02:29.773 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:29.773 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:29.773 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:29.773 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:29.759085+0000", 2024-09-10T19:02:29.773 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:29.773 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:29.773 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:29.773 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:29.773 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:29.759085+0000", 2024-09-10T19:02:29.774 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:29.774 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:29.774 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:29.774 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:29.774 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:29.759085+0000", 2024-09-10T19:02:29.774 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999995 2024-09-10T19:02:29.774 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:29.774 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:29.774 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:29.774 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:29.759090+0000", 2024-09-10T19:02:29.774 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.5219999999999998e-06 2024-09-10T19:02:29.774 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:29.774 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:29.774 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:29.775 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:29.759099+0000", 2024-09-10T19:02:29.775 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.3379999999999994e-06 2024-09-10T19:02:29.775 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:29.775 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:29.775 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:29.775 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:29.759104+0000", 2024-09-10T19:02:29.775 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.9389999999999996e-06 2024-09-10T19:02:29.775 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:29.775 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:29.775 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:29.775 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:29.759137+0000", 2024-09-10T19:02:29.775 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.2332000000000001e-05 2024-09-10T19:02:29.775 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:29.775 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:29.775 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:02:29.776 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:29.759154+0000", 2024-09-10T19:02:29.776 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.7850999999999999e-05 2024-09-10T19:02:29.776 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:29.776 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:29.776 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:29.776 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:29.776 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-10T19:02:29.776 INFO:teuthology.orchestra.run.smithi103.stdout: "num_ops": 3 2024-09-10T19:02:29.776 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T19:02:29.776 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:29.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:29.816 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:29.816 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:29.917 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:29.951 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:29.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:29.952 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:30.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:29 smithi103 ceph-mon[33163]: pgmap v798: 168 pgs: 168 active+clean; 2.2 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 719 B/s rd, 0 B/s wr, 1.35k op/s 2024-09-10T19:02:30.034 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:29 smithi018 ceph-mon[40196]: pgmap v798: 168 pgs: 168 active+clean; 2.2 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 719 B/s rd, 0 B/s wr, 1.35k op/s 2024-09-10T19:02:30.035 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:29 smithi018 ceph-mon[29336]: pgmap v798: 168 pgs: 168 active+clean; 2.2 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 719 B/s rd, 0 B/s wr, 1.35k op/s 2024-09-10T19:02:30.052 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:30.085 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:30.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:30.086 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:30.186 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:30.216 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:30.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:30.217 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:30.319 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:30.355 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:30.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:30.357 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:30.458 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:30.495 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:30.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:30.496 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:30.524 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:02:30 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:02:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:02:30.587 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-10T19:02:30.597 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:30.638 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:30.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:30.639 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:30.740 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:30.779 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:30.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:30.781 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:30.800 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T19:02:30.881 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:30.929 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:30.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:30.933 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:31.017 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:02:31.034 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:31.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:31.082 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:31.083 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:31.183 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:31.217 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:31.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:31.218 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:31.319 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:31.351 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:31.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:31.353 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:31.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:02:30 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:02:30] "GET /metrics HTTP/1.1" 200 209549 "" "Prometheus/2.43.0" 2024-09-10T19:02:31.453 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:31.490 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:31.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:31.491 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:31.592 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:31.624 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:31.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:31.625 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:31.670 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:02:31.726 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:31.757 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:31.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:31.758 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:31.859 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:31.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:31.893 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:31.893 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:31.994 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:32.023 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:32.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:32.024 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:32.125 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:32.163 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:32.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:32.164 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:32.265 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:32.313 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:32.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:32.314 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:32.415 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:32.464 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:32.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:32.465 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:32.565 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:32.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:32.601 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:32.602 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:32.702 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:32.728 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:32.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:32.729 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:32.745 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:32 smithi103 ceph-mon[33163]: pgmap v799: 168 pgs: 168 active+clean; 2.2 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 1.20k op/s 2024-09-10T19:02:32.830 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:32.832 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:32 smithi018 ceph-mon[40196]: pgmap v799: 168 pgs: 168 active+clean; 2.2 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 1.20k op/s 2024-09-10T19:02:32.836 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:32 smithi018 ceph-mon[29336]: pgmap v799: 168 pgs: 168 active+clean; 2.2 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 1.20k op/s 2024-09-10T19:02:32.877 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:32.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:32.878 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:32.979 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:33.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:33.012 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:33.012 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:33.053 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T19:02:33.053 INFO:teuthology.orchestra.run.smithi103.stdout: "ops": [], 2024-09-10T19:02:33.053 INFO:teuthology.orchestra.run.smithi103.stdout: "complaint_time": 30, 2024-09-10T19:02:33.053 INFO:teuthology.orchestra.run.smithi103.stdout: "num_blocked_ops": 0 2024-09-10T19:02:33.053 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T19:02:33.113 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:33.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:33.150 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:33.150 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:33.251 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:33.287 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:33.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:33.288 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:33.388 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:33.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:33.431 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:33.431 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:33.532 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:33.572 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:33.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:33.573 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:33.674 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:33.713 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:33.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:33.714 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:33.732 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-10T19:02:33.815 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:33.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:33.848 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:33.848 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:33.949 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:33.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:33.994 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:33.994 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:34.095 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:34.134 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:02:34.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:34.148 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:34.148 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:34.215 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T19:02:34.218 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-09-10T19:02:34.248 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:34.287 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:34.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:34.288 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:34.389 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:34.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:34.430 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:34.431 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:34.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:34 smithi103 ceph-mon[33163]: pgmap v800: 168 pgs: 168 active+clean; 1.8 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.64k op/s 2024-09-10T19:02:34.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:34 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/44623057' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:02:34.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:34 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:02:34.532 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:34.566 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:34.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:34.566 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:34.667 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:34.715 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:34 smithi018 ceph-mon[40196]: pgmap v800: 168 pgs: 168 active+clean; 1.8 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.64k op/s 2024-09-10T19:02:34.715 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:34 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/44623057' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:02:34.715 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:34 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:02:34.716 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:34 smithi018 ceph-mon[29336]: pgmap v800: 168 pgs: 168 active+clean; 1.8 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.64k op/s 2024-09-10T19:02:34.716 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:34 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/44623057' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:02:34.716 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:34 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:02:34.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:34.717 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:34.717 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:34.761 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is set 2024-09-10T19:02:34.818 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:34.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:34.863 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:34.863 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:34.964 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:35.004 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:35.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:35.005 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:35.106 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:35.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:35.143 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:35.143 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:35.196 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":800,"stamp":"2024-09-10T19:02:33.414010+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1939413236,"num_objects":29781,"num_object_clones":0,"num_object_copies":89343,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29781,"num_whiteouts":0,"num_read":2165,"num_read_kb":1917,"num_write":248443,"num_write_kb":8873964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22243,"num_bytes_recovered":1440356262,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":59717,"ondisk_log_size":59717,"up":504,"acting":504,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":507,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":562470912,"kb_used":12016340,"kb_used_data":7208992,"kb_used_omap":12,"kb_used_meta":4807027,"kb_avail":550454572,"statfs":{"total":575970213888,"available":563665481728,"internally_reserved":0,"allocated":7382007808,"data_stored":7379540434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13122,"internal_metadata":4922395838},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,8,3,4,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-974979072,"num_objects":-14877,"num_object_clones":0,"num_object_copies":-44631,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-14877,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":14877,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.098284"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":382,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.328414+0000","last_change":"2024-09-10T19:02:21.219606+0000","last_active":"2024-09-10T19:02:24.328414+0000","last_peered":"2024-09-10T19:02:24.328414+0000","last_clean":"2024-09-10T19:02:24.328414+0000","last_became_active":"2024-09-10T18:57:02.460451+0000","last_became_peered":"2024-09-10T18:57:02.460451+0000","last_unstale":"2024-09-10T19:02:24.328414+0000","last_undegraded":"2024-09-10T19:02:24.328414+0000","last_fullsized":"2024-09-10T19:02:24.328414+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:21.219520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:02:21.219520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:26.516003+0000","scrub_duration":0.0056254590000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":368,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.321016+0000","last_change":"2024-09-10T18:59:59.848814+0000","last_active":"2024-09-10T19:02:24.321016+0000","last_peered":"2024-09-10T19:02:24.321016+0000","last_clean":"2024-09-10T19:02:24.321016+0000","last_became_active":"2024-09-10T18:57:02.440648+0000","last_became_peered":"2024-09-10T18:57:02.440648+0000","last_unstale":"2024-09-10T19:02:24.321016+0000","last_undegraded":"2024-09-10T19:02:24.321016+0000","last_fullsized":"2024-09-10T19:02:24.321016+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:59.848700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:59.848700+0000","last_clean_scrub_stamp":"2024-09-10T18:59:59.848700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047956400000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"232'27","reported_seq":467,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.329132+0000","last_change":"2024-09-10T19:02:20.251547+0000","last_active":"2024-09-10T19:02:24.329132+0000","last_peered":"2024-09-10T19:02:24.329132+0000","last_clean":"2024-09-10T19:02:24.329132+0000","last_became_active":"2024-09-10T18:55:40.394717+0000","last_became_peered":"2024-09-10T18:55:40.394717+0000","last_unstale":"2024-09-10T19:02:24.329132+0000","last_undegraded":"2024-09-10T19:02:24.329132+0000","last_fullsized":"2024-09-10T19:02:24.329132+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'27","last_scrub_stamp":"2024-09-10T19:02:20.251456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:20.251456+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:27.983556+0000","scrub_duration":0.011781700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":373,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.327461+0000","last_change":"2024-09-10T19:00:08.841172+0000","last_active":"2024-09-10T19:02:24.327461+0000","last_peered":"2024-09-10T19:02:24.327461+0000","last_clean":"2024-09-10T19:02:24.327461+0000","last_became_active":"2024-09-10T18:45:22.065619+0000","last_became_peered":"2024-09-10T18:45:22.065619+0000","last_unstale":"2024-09-10T19:02:24.327461+0000","last_undegraded":"2024-09-10T19:02:24.327461+0000","last_fullsized":"2024-09-10T19:02:24.327461+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:08.841125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.057932+0000","last_clean_scrub_stamp":"2024-09-10T19:00:08.841125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001059857,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":445,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.328471+0000","last_change":"2024-09-10T19:02:17.284387+0000","last_active":"2024-09-10T19:02:24.328471+0000","last_peered":"2024-09-10T19:02:24.328471+0000","last_clean":"2024-09-10T19:02:24.328471+0000","last_became_active":"2024-09-10T19:00:42.518824+0000","last_became_peered":"2024-09-10T19:00:42.518824+0000","last_unstale":"2024-09-10T19:02:24.328471+0000","last_undegraded":"2024-09-10T19:02:24.328471+0000","last_fullsized":"2024-09-10T19:02:24.328471+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:17.284305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:17.284305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:27.881734+0000","scrub_duration":0.0018984430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"232'19","reported_seq":394,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.327396+0000","last_change":"2024-09-10T18:59:09.642743+0000","last_active":"2024-09-10T19:02:24.327396+0000","last_peered":"2024-09-10T19:02:24.327396+0000","last_clean":"2024-09-10T19:02:24.327396+0000","last_became_active":"2024-09-10T18:45:20.061266+0000","last_became_peered":"2024-09-10T18:45:20.061266+0000","last_unstale":"2024-09-10T19:02:24.327396+0000","last_undegraded":"2024-09-10T19:02:24.327396+0000","last_fullsized":"2024-09-10T19:02:24.327396+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T18:59:09.642642+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_clean_scrub_stamp":"2024-09-10T18:59:09.642642+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0095920940000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":119,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.327640+0000","last_change":"2024-09-10T19:01:54.384697+0000","last_active":"2024-09-10T19:02:24.327640+0000","last_peered":"2024-09-10T19:02:24.327640+0000","last_clean":"2024-09-10T19:02:24.327640+0000","last_became_active":"2024-09-10T18:57:02.446008+0000","last_became_peered":"2024-09-10T18:57:02.446008+0000","last_unstale":"2024-09-10T19:02:24.327640+0000","last_undegraded":"2024-09-10T19:02:24.327640+0000","last_fullsized":"2024-09-10T19:02:24.327640+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:01:54.384604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:01.135988+0000","last_clean_scrub_stamp":"2024-09-10T19:01:54.384604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:09.487091+0000","scrub_duration":0.010950134,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":361,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.325545+0000","last_change":"2024-09-10T18:58:51.682046+0000","last_active":"2024-09-10T19:02:24.325545+0000","last_peered":"2024-09-10T19:02:24.325545+0000","last_clean":"2024-09-10T19:02:24.325545+0000","last_became_active":"2024-09-10T18:45:24.081002+0000","last_became_peered":"2024-09-10T18:45:24.081002+0000","last_unstale":"2024-09-10T19:02:24.325545+0000","last_undegraded":"2024-09-10T19:02:24.325545+0000","last_fullsized":"2024-09-10T19:02:24.325545+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:51.681994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:58:51.681994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051776249999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":366,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.328354+0000","last_change":"2024-09-10T18:59:21.502068+0000","last_active":"2024-09-10T19:02:24.328354+0000","last_peered":"2024-09-10T19:02:24.328354+0000","last_clean":"2024-09-10T19:02:24.328354+0000","last_became_active":"2024-09-10T18:57:02.452259+0000","last_became_peered":"2024-09-10T18:57:02.452259+0000","last_unstale":"2024-09-10T19:02:24.328354+0000","last_undegraded":"2024-09-10T19:02:24.328354+0000","last_fullsized":"2024-09-10T19:02:24.328354+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:21.501984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:21.501984+0000","last_clean_scrub_stamp":"2024-09-10T18:59:21.501984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014628558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1b","version":"232'9","reported_seq":380,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.325707+0000","last_change":"2024-09-10T18:59:11.622597+0000","last_active":"2024-09-10T19:02:24.325707+0000","last_peered":"2024-09-10T19:02:24.325707+0000","last_clean":"2024-09-10T19:02:24.325707+0000","last_became_active":"2024-09-10T18:45:20.057567+0000","last_became_peered":"2024-09-10T18:45:20.057567+0000","last_unstale":"2024-09-10T19:02:24.325707+0000","last_undegraded":"2024-09-10T19:02:24.325707+0000","last_fullsized":"2024-09-10T19:02:24.325707+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'9","last_scrub_stamp":"2024-09-10T18:59:11.622509+0000","last_deep_scrub":"163'7","last_deep_scrub_stamp":"2024-09-10T18:51:54.261879+0000","last_clean_scrub_stamp":"2024-09-10T18:59:11.622509+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056120609999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":132,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.325455+0000","last_change":"2024-09-10T19:00:40.947163+0000","last_active":"2024-09-10T19:02:24.325455+0000","last_peered":"2024-09-10T19:02:24.325455+0000","last_clean":"2024-09-10T19:02:24.325455+0000","last_became_active":"2024-09-10T18:58:58.503924+0000","last_became_peered":"2024-09-10T18:58:58.503924+0000","last_unstale":"2024-09-10T19:02:24.325455+0000","last_undegraded":"2024-09-10T19:02:24.325455+0000","last_fullsized":"2024-09-10T19:02:24.325455+0000","mapping_epoch":303,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":304,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:40.947066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:36.509971+0000","last_clean_scrub_stamp":"2024-09-10T19:00:40.947066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012293715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":39,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.325880+0000","last_change":"2024-09-10T19:00:42.516918+0000","last_active":"2024-09-10T19:02:24.325880+0000","last_peered":"2024-09-10T19:02:24.325880+0000","last_clean":"2024-09-10T19:02:24.325880+0000","last_became_active":"2024-09-10T19:00:42.515840+0000","last_became_peered":"2024-09-10T19:00:42.515840+0000","last_unstale":"2024-09-10T19:02:24.325880+0000","last_undegraded":"2024-09-10T19:02:24.325880+0000","last_fullsized":"2024-09-10T19:02:24.325880+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:48.230459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:59:48.230459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":182,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.324080+0000","last_change":"2024-09-10T19:02:12.656894+0000","last_active":"2024-09-10T19:02:24.324080+0000","last_peered":"2024-09-10T19:02:24.324080+0000","last_clean":"2024-09-10T19:02:24.324080+0000","last_became_active":"2024-09-10T18:57:02.440497+0000","last_became_peered":"2024-09-10T18:57:02.440497+0000","last_unstale":"2024-09-10T19:02:24.324080+0000","last_undegraded":"2024-09-10T19:02:24.324080+0000","last_fullsized":"2024-09-10T19:02:24.324080+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:12.656811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:12.656811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:13.925956+0000","scrub_duration":0.0082345850000000009,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"232'15","reported_seq":429,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.325772+0000","last_change":"2024-09-10T19:02:22.113293+0000","last_active":"2024-09-10T19:02:24.325772+0000","last_peered":"2024-09-10T19:02:24.325772+0000","last_clean":"2024-09-10T19:02:24.325772+0000","last_became_active":"2024-09-10T18:55:40.391571+0000","last_became_peered":"2024-09-10T18:55:40.391571+0000","last_unstale":"2024-09-10T19:02:24.325772+0000","last_undegraded":"2024-09-10T19:02:24.325772+0000","last_fullsized":"2024-09-10T19:02:24.325772+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T19:02:22.113140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:22.113140+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:23.371300+0000","scrub_duration":0.0061707400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":402,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.321251+0000","last_change":"2024-09-10T18:59:22.643043+0000","last_active":"2024-09-10T19:02:24.321251+0000","last_peered":"2024-09-10T19:02:24.321251+0000","last_clean":"2024-09-10T19:02:24.321251+0000","last_became_active":"2024-09-10T18:57:02.431585+0000","last_became_peered":"2024-09-10T18:57:02.431585+0000","last_unstale":"2024-09-10T19:02:24.321251+0000","last_undegraded":"2024-09-10T19:02:24.321251+0000","last_fullsized":"2024-09-10T19:02:24.321251+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:22.642961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:59:22.642961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.018189153999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":57,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.326931+0000","last_change":"2024-09-10T18:59:48.738143+0000","last_active":"2024-09-10T19:02:24.326931+0000","last_peered":"2024-09-10T19:02:24.326931+0000","last_clean":"2024-09-10T19:02:24.326931+0000","last_became_active":"2024-09-10T18:59:48.737788+0000","last_became_peered":"2024-09-10T18:59:48.737788+0000","last_unstale":"2024-09-10T19:02:24.326931+0000","last_undegraded":"2024-09-10T19:02:24.326931+0000","last_fullsized":"2024-09-10T19:02:24.326931+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:51.561010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:59.425718+0000","last_clean_scrub_stamp":"2024-09-10T18:58:51.561010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3","version":"368'15409","reported_seq":15470,"reported_epoch":368,"state":"active+clean","last_fresh":"2024-09-10T19:02:31.606909+0000","last_change":"2024-09-10T18:59:43.007892+0000","last_active":"2024-09-10T19:02:31.606909+0000","last_peered":"2024-09-10T19:02:31.606909+0000","last_clean":"2024-09-10T19:02:31.606909+0000","last_became_active":"2024-09-10T18:59:43.007585+0000","last_became_peered":"2024-09-10T18:59:43.007585+0000","last_unstale":"2024-09-10T19:02:31.606909+0000","last_undegraded":"2024-09-10T19:02:31.606909+0000","last_fullsized":"2024-09-10T19:02:31.606909+0000","mapping_epoch":315,"log_start":"360'11799","ondisk_log_start":"360'11799","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3610,"log_dups_size":0,"ondisk_log_size":3610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113573888,"num_objects":1733,"num_object_clones":0,"num_object_copies":5199,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15409,"num_write_kb":548544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.8","version":"70'81","reported_seq":43,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.324694+0000","last_change":"2024-09-10T19:00:42.527220+0000","last_active":"2024-09-10T19:02:24.324694+0000","last_peered":"2024-09-10T19:02:24.324694+0000","last_clean":"2024-09-10T19:02:24.324694+0000","last_became_active":"2024-09-10T19:00:42.517072+0000","last_became_peered":"2024-09-10T19:00:42.517072+0000","last_unstale":"2024-09-10T19:02:24.324694+0000","last_undegraded":"2024-09-10T19:02:24.324694+0000","last_fullsized":"2024-09-10T19:02:24.324694+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:51.167302+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:59:51.167302+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015031440000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":262688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"232'39","reported_seq":460,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.326587+0000","last_change":"2024-09-10T18:59:47.745859+0000","last_active":"2024-09-10T19:02:24.326587+0000","last_peered":"2024-09-10T19:02:24.326587+0000","last_clean":"2024-09-10T19:02:24.326587+0000","last_became_active":"2024-09-10T18:47:36.409300+0000","last_became_peered":"2024-09-10T18:47:36.409300+0000","last_unstale":"2024-09-10T19:02:24.326587+0000","last_undegraded":"2024-09-10T19:02:24.326587+0000","last_fullsized":"2024-09-10T19:02:24.326587+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'39","last_scrub_stamp":"2024-09-10T18:59:47.745766+0000","last_deep_scrub":"78'25","last_deep_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_clean_scrub_stamp":"2024-09-10T18:59:47.745766+0000","objects_scrubbed":8,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011279000000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":119,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.325944+0000","last_change":"2024-09-10T19:00:09.867278+0000","last_active":"2024-09-10T19:02:24.325944+0000","last_peered":"2024-09-10T19:02:24.325944+0000","last_clean":"2024-09-10T19:02:24.325944+0000","last_became_active":"2024-09-10T18:57:02.437260+0000","last_became_peered":"2024-09-10T18:57:02.437260+0000","last_unstale":"2024-09-10T19:02:24.325944+0000","last_undegraded":"2024-09-10T19:02:24.325944+0000","last_fullsized":"2024-09-10T19:02:24.325944+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:09.867183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:00:09.867183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026871460000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":382,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.329278+0000","last_change":"2024-09-10T19:00:09.470422+0000","last_active":"2024-09-10T19:02:24.329278+0000","last_peered":"2024-09-10T19:02:24.329278+0000","last_clean":"2024-09-10T19:02:24.329278+0000","last_became_active":"2024-09-10T18:55:40.392449+0000","last_became_peered":"2024-09-10T18:55:40.392449+0000","last_unstale":"2024-09-10T19:02:24.329278+0000","last_undegraded":"2024-09-10T19:02:24.329278+0000","last_fullsized":"2024-09-10T19:02:24.329278+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:09.470293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:04.512425+0000","last_clean_scrub_stamp":"2024-09-10T19:00:09.470293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0073759949999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":280,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.321141+0000","last_change":"2024-09-10T18:58:49.524134+0000","last_active":"2024-09-10T19:02:24.321141+0000","last_peered":"2024-09-10T19:02:24.321141+0000","last_clean":"2024-09-10T19:02:24.321141+0000","last_became_active":"2024-09-10T18:55:40.388154+0000","last_became_peered":"2024-09-10T18:55:40.388154+0000","last_unstale":"2024-09-10T19:02:24.321141+0000","last_undegraded":"2024-09-10T19:02:24.321141+0000","last_fullsized":"2024-09-10T19:02:24.321141+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:49.524015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:58:49.524015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005180479,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2","version":"368'15852","reported_seq":15913,"reported_epoch":368,"state":"active+clean","last_fresh":"2024-09-10T19:02:31.823121+0000","last_change":"2024-09-10T18:59:42.709056+0000","last_active":"2024-09-10T19:02:31.823121+0000","last_peered":"2024-09-10T19:02:31.823121+0000","last_clean":"2024-09-10T19:02:31.823121+0000","last_became_active":"2024-09-10T18:59:42.708732+0000","last_became_peered":"2024-09-10T18:59:42.708732+0000","last_unstale":"2024-09-10T19:02:31.823121+0000","last_undegraded":"2024-09-10T19:02:31.823121+0000","last_fullsized":"2024-09-10T19:02:31.823121+0000","mapping_epoch":315,"log_start":"361'12400","ondisk_log_start":"361'12400","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3452,"log_dups_size":0,"ondisk_log_size":3452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115736576,"num_objects":1766,"num_object_clones":0,"num_object_copies":5298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15852,"num_write_kb":563776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.9","version":"70'81","reported_seq":276,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.325394+0000","last_change":"2024-09-10T18:59:56.164487+0000","last_active":"2024-09-10T19:02:24.325394+0000","last_peered":"2024-09-10T19:02:24.325394+0000","last_clean":"2024-09-10T19:02:24.325394+0000","last_became_active":"2024-09-10T18:57:02.463750+0000","last_became_peered":"2024-09-10T18:57:02.463750+0000","last_unstale":"2024-09-10T19:02:24.325394+0000","last_undegraded":"2024-09-10T19:02:24.325394+0000","last_fullsized":"2024-09-10T19:02:24.325394+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:56.164384+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:59:56.164384+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041056720000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"232'25","reported_seq":400,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.326290+0000","last_change":"2024-09-10T18:56:32.437605+0000","last_active":"2024-09-10T19:02:24.326290+0000","last_peered":"2024-09-10T19:02:24.326290+0000","last_clean":"2024-09-10T19:02:24.326290+0000","last_became_active":"2024-09-10T18:45:20.053428+0000","last_became_peered":"2024-09-10T18:45:20.053428+0000","last_unstale":"2024-09-10T19:02:24.326290+0000","last_undegraded":"2024-09-10T19:02:24.326290+0000","last_fullsized":"2024-09-10T19:02:24.326290+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'25","last_scrub_stamp":"2024-09-10T18:56:32.437503+0000","last_deep_scrub":"116'15","last_deep_scrub_stamp":"2024-09-10T18:49:12.374527+0000","last_clean_scrub_stamp":"2024-09-10T18:56:32.437503+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039673679999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":392,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.328507+0000","last_change":"2024-09-10T19:02:01.343291+0000","last_active":"2024-09-10T19:02:24.328507+0000","last_peered":"2024-09-10T19:02:24.328507+0000","last_clean":"2024-09-10T19:02:24.328507+0000","last_became_active":"2024-09-10T18:57:02.440442+0000","last_became_peered":"2024-09-10T18:57:02.440442+0000","last_unstale":"2024-09-10T19:02:24.328507+0000","last_undegraded":"2024-09-10T19:02:24.328507+0000","last_fullsized":"2024-09-10T19:02:24.328507+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:01.343214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:01.343214+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:20.056326+0000","scrub_duration":0.0072015969999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"273'13","reported_seq":446,"reported_epoch":368,"state":"active+clean","last_fresh":"2024-09-10T19:02:27.323358+0000","last_change":"2024-09-10T19:01:59.557656+0000","last_active":"2024-09-10T19:02:27.323358+0000","last_peered":"2024-09-10T19:02:27.323358+0000","last_clean":"2024-09-10T19:02:27.323358+0000","last_became_active":"2024-09-10T18:57:02.448258+0000","last_became_peered":"2024-09-10T18:57:02.448258+0000","last_unstale":"2024-09-10T19:02:27.323358+0000","last_undegraded":"2024-09-10T19:02:27.323358+0000","last_fullsized":"2024-09-10T19:02:27.323358+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'13","last_scrub_stamp":"2024-09-10T19:01:59.557500+0000","last_deep_scrub":"202'10","last_deep_scrub_stamp":"2024-09-10T18:53:45.547686+0000","last_clean_scrub_stamp":"2024-09-10T19:01:59.557500+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:16.615418+0000","scrub_duration":0.0027644549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":411,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.326417+0000","last_change":"2024-09-10T19:00:42.515842+0000","last_active":"2024-09-10T19:02:24.326417+0000","last_peered":"2024-09-10T19:02:24.326417+0000","last_clean":"2024-09-10T19:02:24.326417+0000","last_became_active":"2024-09-10T19:00:42.515523+0000","last_became_peered":"2024-09-10T19:00:42.515523+0000","last_unstale":"2024-09-10T19:02:24.326417+0000","last_undegraded":"2024-09-10T19:02:24.326417+0000","last_fullsized":"2024-09-10T19:02:24.326417+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:07.803916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:32.043666+0000","last_clean_scrub_stamp":"2024-09-10T19:00:07.803916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0011072429999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1","version":"368'15438","reported_seq":15499,"reported_epoch":368,"state":"active+clean","last_fresh":"2024-09-10T19:02:31.382467+0000","last_change":"2024-09-10T18:59:42.711934+0000","last_active":"2024-09-10T19:02:31.382467+0000","last_peered":"2024-09-10T19:02:31.382467+0000","last_clean":"2024-09-10T19:02:31.382467+0000","last_became_active":"2024-09-10T18:59:42.711662+0000","last_became_peered":"2024-09-10T18:59:42.711662+0000","last_unstale":"2024-09-10T19:02:31.382467+0000","last_undegraded":"2024-09-10T19:02:31.382467+0000","last_fullsized":"2024-09-10T19:02:31.382467+0000","mapping_epoch":315,"log_start":"360'11900","ondisk_log_start":"360'11900","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3538,"log_dups_size":0,"ondisk_log_size":3538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118226944,"num_objects":1804,"num_object_clones":0,"num_object_copies":5412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15438,"num_write_kb":551744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1],"acting":[4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"70'81","reported_seq":300,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.325209+0000","last_change":"2024-09-10T19:02:23.723122+0000","last_active":"2024-09-10T19:02:24.325209+0000","last_peered":"2024-09-10T19:02:24.325209+0000","last_clean":"2024-09-10T19:02:24.325209+0000","last_became_active":"2024-09-10T18:50:08.398345+0000","last_became_peered":"2024-09-10T18:50:08.398345+0000","last_unstale":"2024-09-10T19:02:24.325209+0000","last_undegraded":"2024-09-10T19:02:24.325209+0000","last_fullsized":"2024-09-10T19:02:24.325209+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:23.723036+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:55:37.921519+0000","last_clean_scrub_stamp":"2024-09-10T19:02:23.723036+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:30.139717+0000","scrub_duration":0.0054938950000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"232'34","reported_seq":397,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.324048+0000","last_change":"2024-09-10T19:00:41.482710+0000","last_active":"2024-09-10T19:02:24.324048+0000","last_peered":"2024-09-10T19:02:24.324048+0000","last_clean":"2024-09-10T19:02:24.324048+0000","last_became_active":"2024-09-10T18:46:56.742946+0000","last_became_peered":"2024-09-10T18:46:56.742946+0000","last_unstale":"2024-09-10T19:02:24.324048+0000","last_undegraded":"2024-09-10T19:02:24.324048+0000","last_fullsized":"2024-09-10T19:02:24.324048+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'34","last_scrub_stamp":"2024-09-10T19:00:41.482619+0000","last_deep_scrub":"141'22","last_deep_scrub_stamp":"2024-09-10T18:50:57.119152+0000","last_clean_scrub_stamp":"2024-09-10T19:00:41.482619+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0069363309999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":392,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.328023+0000","last_change":"2024-09-10T18:59:51.286189+0000","last_active":"2024-09-10T19:02:24.328023+0000","last_peered":"2024-09-10T19:02:24.328023+0000","last_clean":"2024-09-10T19:02:24.328023+0000","last_became_active":"2024-09-10T18:57:02.445848+0000","last_became_peered":"2024-09-10T18:57:02.445848+0000","last_unstale":"2024-09-10T19:02:24.328023+0000","last_undegraded":"2024-09-10T19:02:24.328023+0000","last_fullsized":"2024-09-10T19:02:24.328023+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:51.286131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_clean_scrub_stamp":"2024-09-10T18:59:51.286131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015180669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"242'10","reported_seq":597,"reported_epoch":368,"state":"active+clean","last_fresh":"2024-09-10T19:02:27.320580+0000","last_change":"2024-09-10T18:56:19.581070+0000","last_active":"2024-09-10T19:02:27.320580+0000","last_peered":"2024-09-10T19:02:27.320580+0000","last_clean":"2024-09-10T19:02:27.320580+0000","last_became_active":"2024-09-10T18:55:40.397246+0000","last_became_peered":"2024-09-10T18:55:40.397246+0000","last_unstale":"2024-09-10T19:02:27.320580+0000","last_undegraded":"2024-09-10T19:02:27.320580+0000","last_fullsized":"2024-09-10T19:02:27.320580+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"242'10","last_scrub_stamp":"2024-09-10T18:56:19.581012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:56:19.581012+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013423529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":390,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.323728+0000","last_change":"2024-09-10T18:59:10.477418+0000","last_active":"2024-09-10T19:02:24.323728+0000","last_peered":"2024-09-10T19:02:24.323728+0000","last_clean":"2024-09-10T19:02:24.323728+0000","last_became_active":"2024-09-10T18:55:40.392003+0000","last_became_peered":"2024-09-10T18:55:40.392003+0000","last_unstale":"2024-09-10T19:02:24.323728+0000","last_undegraded":"2024-09-10T19:02:24.323728+0000","last_fullsized":"2024-09-10T19:02:24.323728+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:10.477346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:54.218602+0000","last_clean_scrub_stamp":"2024-09-10T18:59:10.477346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004313005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"368'15654","reported_seq":15715,"reported_epoch":368,"state":"active+clean","last_fresh":"2024-09-10T19:02:31.519117+0000","last_change":"2024-09-10T18:59:42.715764+0000","last_active":"2024-09-10T19:02:31.519117+0000","last_peered":"2024-09-10T19:02:31.519117+0000","last_clean":"2024-09-10T19:02:31.519117+0000","last_became_active":"2024-09-10T18:59:42.714808+0000","last_became_peered":"2024-09-10T18:59:42.714808+0000","last_unstale":"2024-09-10T19:02:31.519117+0000","last_undegraded":"2024-09-10T19:02:31.519117+0000","last_fullsized":"2024-09-10T19:02:31.519117+0000","mapping_epoch":315,"log_start":"361'12100","ondisk_log_start":"361'12100","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3554,"log_dups_size":0,"ondisk_log_size":3554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":122945536,"num_objects":1876,"num_object_clones":0,"num_object_copies":5628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15654,"num_write_kb":560960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.b","version":"70'81","reported_seq":345,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.330208+0000","last_change":"2024-09-10T19:02:17.370846+0000","last_active":"2024-09-10T19:02:24.330208+0000","last_peered":"2024-09-10T19:02:24.330208+0000","last_clean":"2024-09-10T19:02:24.330208+0000","last_became_active":"2024-09-10T19:00:42.518107+0000","last_became_peered":"2024-09-10T19:00:42.518107+0000","last_unstale":"2024-09-10T19:02:24.330208+0000","last_undegraded":"2024-09-10T19:02:24.330208+0000","last_fullsized":"2024-09-10T19:02:24.330208+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:17.370627+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T19:02:17.370627+0000","last_clean_scrub_stamp":"2024-09-10T19:02:17.370627+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:33.584446+0000","scrub_duration":0.0044409599999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"232'24","reported_seq":477,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.326646+0000","last_change":"2024-09-10T18:59:06.595605+0000","last_active":"2024-09-10T19:02:24.326646+0000","last_peered":"2024-09-10T19:02:24.326646+0000","last_clean":"2024-09-10T19:02:24.326646+0000","last_became_active":"2024-09-10T18:47:57.106935+0000","last_became_peered":"2024-09-10T18:47:57.106935+0000","last_unstale":"2024-09-10T19:02:24.326646+0000","last_undegraded":"2024-09-10T19:02:24.326646+0000","last_fullsized":"2024-09-10T19:02:24.326646+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T18:59:06.595554+0000","last_deep_scrub":"224'16","last_deep_scrub_stamp":"2024-09-10T18:54:49.296957+0000","last_clean_scrub_stamp":"2024-09-10T18:59:06.595554+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020642540000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":373,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.330203+0000","last_change":"2024-09-10T19:02:13.720500+0000","last_active":"2024-09-10T19:02:24.330203+0000","last_peered":"2024-09-10T19:02:24.330203+0000","last_clean":"2024-09-10T19:02:24.330203+0000","last_became_active":"2024-09-10T18:57:02.443816+0000","last_became_peered":"2024-09-10T18:57:02.443816+0000","last_unstale":"2024-09-10T19:02:24.330203+0000","last_undegraded":"2024-09-10T19:02:24.330203+0000","last_fullsized":"2024-09-10T19:02:24.330203+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:13.720416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:13.720416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:36.657529+0000","scrub_duration":0.014226275,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":388,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.324174+0000","last_change":"2024-09-10T19:02:07.623139+0000","last_active":"2024-09-10T19:02:24.324174+0000","last_peered":"2024-09-10T19:02:24.324174+0000","last_clean":"2024-09-10T19:02:24.324174+0000","last_became_active":"2024-09-10T18:45:22.068918+0000","last_became_peered":"2024-09-10T18:45:22.068918+0000","last_unstale":"2024-09-10T19:02:24.324174+0000","last_undegraded":"2024-09-10T19:02:24.324174+0000","last_fullsized":"2024-09-10T19:02:24.324174+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:07.623054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_clean_scrub_stamp":"2024-09-10T19:02:07.623054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:10.526309+0000","scrub_duration":0.003868888,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":362,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.329477+0000","last_change":"2024-09-10T18:59:13.892307+0000","last_active":"2024-09-10T19:02:24.329477+0000","last_peered":"2024-09-10T19:02:24.329477+0000","last_clean":"2024-09-10T19:02:24.329477+0000","last_became_active":"2024-09-10T18:45:24.264538+0000","last_became_peered":"2024-09-10T18:45:24.264538+0000","last_unstale":"2024-09-10T19:02:24.329477+0000","last_undegraded":"2024-09-10T19:02:24.329477+0000","last_fullsized":"2024-09-10T19:02:24.329477+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:13.892129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:51.398981+0000","last_clean_scrub_stamp":"2024-09-10T18:59:13.892129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007612214,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"368'15504","reported_seq":15565,"reported_epoch":368,"state":"active+clean","last_fresh":"2024-09-10T19:02:31.536848+0000","last_change":"2024-09-10T18:59:43.008094+0000","last_active":"2024-09-10T19:02:31.536848+0000","last_peered":"2024-09-10T19:02:31.536848+0000","last_clean":"2024-09-10T19:02:31.536848+0000","last_became_active":"2024-09-10T18:59:43.007838+0000","last_became_peered":"2024-09-10T18:59:43.007838+0000","last_unstale":"2024-09-10T19:02:31.536848+0000","last_undegraded":"2024-09-10T19:02:31.536848+0000","last_fullsized":"2024-09-10T19:02:31.536848+0000","mapping_epoch":315,"log_start":"361'12000","ondisk_log_start":"361'12000","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3504,"log_dups_size":0,"ondisk_log_size":3504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118751232,"num_objects":1812,"num_object_clones":0,"num_object_copies":5436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1812,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15504,"num_write_kb":554112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"70'81","reported_seq":320,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.329598+0000","last_change":"2024-09-10T19:00:42.520084+0000","last_active":"2024-09-10T19:02:24.329598+0000","last_peered":"2024-09-10T19:02:24.329598+0000","last_clean":"2024-09-10T19:02:24.329598+0000","last_became_active":"2024-09-10T19:00:42.519742+0000","last_became_peered":"2024-09-10T19:00:42.519742+0000","last_unstale":"2024-09-10T19:02:24.329598+0000","last_undegraded":"2024-09-10T19:02:24.329598+0000","last_fullsized":"2024-09-10T19:02:24.329598+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:58:56.912936+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:58:56.912936+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0045874109999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"232'35","reported_seq":414,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.329905+0000","last_change":"2024-09-10T18:56:28.475224+0000","last_active":"2024-09-10T19:02:24.329905+0000","last_peered":"2024-09-10T19:02:24.329905+0000","last_clean":"2024-09-10T19:02:24.329905+0000","last_became_active":"2024-09-10T18:45:20.052243+0000","last_became_peered":"2024-09-10T18:45:20.052243+0000","last_unstale":"2024-09-10T19:02:24.329905+0000","last_undegraded":"2024-09-10T19:02:24.329905+0000","last_fullsized":"2024-09-10T19:02:24.329905+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'35","last_scrub_stamp":"2024-09-10T18:56:28.475142+0000","last_deep_scrub":"143'21","last_deep_scrub_stamp":"2024-09-10T18:51:02.540216+0000","last_clean_scrub_stamp":"2024-09-10T18:56:28.475142+0000","objects_scrubbed":7,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026978649999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":410,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.329193+0000","last_change":"2024-09-10T19:01:55.150294+0000","last_active":"2024-09-10T19:02:24.329193+0000","last_peered":"2024-09-10T19:02:24.329193+0000","last_clean":"2024-09-10T19:02:24.329193+0000","last_became_active":"2024-09-10T18:57:02.453404+0000","last_became_peered":"2024-09-10T18:57:02.453404+0000","last_unstale":"2024-09-10T19:02:24.329193+0000","last_undegraded":"2024-09-10T19:02:24.329193+0000","last_fullsized":"2024-09-10T19:02:24.329193+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:01:55.150215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_clean_scrub_stamp":"2024-09-10T19:01:55.150215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:02:58.303982+0000","scrub_duration":0.0060613990000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":588,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:26.277732+0000","last_change":"2024-09-10T19:01:56.290088+0000","last_active":"2024-09-10T19:02:26.277732+0000","last_peered":"2024-09-10T19:02:26.277732+0000","last_clean":"2024-09-10T19:02:26.277732+0000","last_became_active":"2024-09-10T18:45:22.074540+0000","last_became_peered":"2024-09-10T18:45:22.074540+0000","last_unstale":"2024-09-10T19:02:26.277732+0000","last_undegraded":"2024-09-10T19:02:26.277732+0000","last_fullsized":"2024-09-10T19:02:26.277732+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T19:01:56.289997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:01:56.289997+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:11.516466+0000","scrub_duration":0.0040368549999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":177,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.327838+0000","last_change":"2024-09-10T19:02:10.106717+0000","last_active":"2024-09-10T19:02:24.327838+0000","last_peered":"2024-09-10T19:02:24.327838+0000","last_clean":"2024-09-10T19:02:24.327838+0000","last_became_active":"2024-09-10T18:57:02.437100+0000","last_became_peered":"2024-09-10T18:57:02.437100+0000","last_unstale":"2024-09-10T19:02:24.327838+0000","last_undegraded":"2024-09-10T19:02:24.327838+0000","last_fullsized":"2024-09-10T19:02:24.327838+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:10.106577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:02:10.106577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:16.542552+0000","scrub_duration":0.0035912439999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.e","version":"368'15356","reported_seq":15417,"reported_epoch":368,"state":"active+clean","last_fresh":"2024-09-10T19:02:31.413642+0000","last_change":"2024-09-10T18:59:42.713933+0000","last_active":"2024-09-10T19:02:31.413642+0000","last_peered":"2024-09-10T19:02:31.413642+0000","last_clean":"2024-09-10T19:02:31.413642+0000","last_became_active":"2024-09-10T18:59:42.713605+0000","last_became_peered":"2024-09-10T18:59:42.713605+0000","last_unstale":"2024-09-10T19:02:31.413642+0000","last_undegraded":"2024-09-10T19:02:31.413642+0000","last_fullsized":"2024-09-10T19:02:31.413642+0000","mapping_epoch":315,"log_start":"360'11800","ondisk_log_start":"360'11800","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3556,"log_dups_size":0,"ondisk_log_size":3556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118358016,"num_objects":1806,"num_object_clones":0,"num_object_copies":5418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15356,"num_write_kb":549184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"70'81","reported_seq":285,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.326141+0000","last_change":"2024-09-10T18:56:31.481362+0000","last_active":"2024-09-10T19:02:24.326141+0000","last_peered":"2024-09-10T19:02:24.326141+0000","last_clean":"2024-09-10T19:02:24.326141+0000","last_became_active":"2024-09-10T18:52:14.584798+0000","last_became_peered":"2024-09-10T18:52:14.584798+0000","last_unstale":"2024-09-10T19:02:24.326141+0000","last_undegraded":"2024-09-10T19:02:24.326141+0000","last_fullsized":"2024-09-10T19:02:24.326141+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:31.481303+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:56:31.481303+0000","last_clean_scrub_stamp":"2024-09-10T18:56:31.481303+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0011170959999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"232'30","reported_seq":532,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.325006+0000","last_change":"2024-09-10T18:59:48.754412+0000","last_active":"2024-09-10T19:02:24.325006+0000","last_peered":"2024-09-10T19:02:24.325006+0000","last_clean":"2024-09-10T19:02:24.325006+0000","last_became_active":"2024-09-10T18:59:48.742377+0000","last_became_peered":"2024-09-10T18:59:48.742377+0000","last_unstale":"2024-09-10T19:02:24.325006+0000","last_undegraded":"2024-09-10T19:02:24.325006+0000","last_fullsized":"2024-09-10T19:02:24.325006+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'30","last_scrub_stamp":"2024-09-10T18:56:15.875924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:15.875924+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0075153879999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":372,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.328895+0000","last_change":"2024-09-10T18:59:05.246945+0000","last_active":"2024-09-10T19:02:24.328895+0000","last_peered":"2024-09-10T19:02:24.328895+0000","last_clean":"2024-09-10T19:02:24.328895+0000","last_became_active":"2024-09-10T18:45:17.961695+0000","last_became_peered":"2024-09-10T18:45:17.961695+0000","last_unstale":"2024-09-10T19:02:24.328895+0000","last_undegraded":"2024-09-10T19:02:24.328895+0000","last_fullsized":"2024-09-10T19:02:24.328895+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:05.246629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:46.142129+0000","last_clean_scrub_stamp":"2024-09-10T18:59:05.246629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0072716029999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":383,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.329724+0000","last_change":"2024-09-10T18:59:57.903394+0000","last_active":"2024-09-10T19:02:24.329724+0000","last_peered":"2024-09-10T19:02:24.329724+0000","last_clean":"2024-09-10T19:02:24.329724+0000","last_became_active":"2024-09-10T18:57:02.448126+0000","last_became_peered":"2024-09-10T18:57:02.448126+0000","last_unstale":"2024-09-10T19:02:24.329724+0000","last_undegraded":"2024-09-10T19:02:24.329724+0000","last_fullsized":"2024-09-10T19:02:24.329724+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:57.903311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:57.903311+0000","last_clean_scrub_stamp":"2024-09-10T18:59:57.903311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011228738,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":371,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.326064+0000","last_change":"2024-09-10T18:58:48.633501+0000","last_active":"2024-09-10T19:02:24.326064+0000","last_peered":"2024-09-10T19:02:24.326064+0000","last_clean":"2024-09-10T19:02:24.326064+0000","last_became_active":"2024-09-10T18:47:36.410836+0000","last_became_peered":"2024-09-10T18:47:36.410836+0000","last_unstale":"2024-09-10T19:02:24.326064+0000","last_undegraded":"2024-09-10T19:02:24.326064+0000","last_fullsized":"2024-09-10T19:02:24.326064+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:48.633423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_clean_scrub_stamp":"2024-09-10T18:58:48.633423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0079262829999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.d","version":"368'15707","reported_seq":15769,"reported_epoch":368,"state":"active+clean","last_fresh":"2024-09-10T19:02:31.522715+0000","last_change":"2024-09-10T18:59:42.717015+0000","last_active":"2024-09-10T19:02:31.522715+0000","last_peered":"2024-09-10T19:02:31.522715+0000","last_clean":"2024-09-10T19:02:31.522715+0000","last_became_active":"2024-09-10T18:59:42.716708+0000","last_became_peered":"2024-09-10T18:59:42.716708+0000","last_unstale":"2024-09-10T19:02:31.522715+0000","last_undegraded":"2024-09-10T19:02:31.522715+0000","last_fullsized":"2024-09-10T19:02:31.522715+0000","mapping_epoch":315,"log_start":"361'12200","ondisk_log_start":"361'12200","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3507,"log_dups_size":0,"ondisk_log_size":3507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":119537688,"num_objects":1825,"num_object_clones":0,"num_object_copies":5475,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1825,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15707,"num_write_kb":560961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"70'81","reported_seq":393,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.329095+0000","last_change":"2024-09-10T19:02:21.983253+0000","last_active":"2024-09-10T19:02:24.329095+0000","last_peered":"2024-09-10T19:02:24.329095+0000","last_clean":"2024-09-10T19:02:24.329095+0000","last_became_active":"2024-09-10T18:55:40.396828+0000","last_became_peered":"2024-09-10T18:55:40.396828+0000","last_unstale":"2024-09-10T19:02:24.329095+0000","last_undegraded":"2024-09-10T19:02:24.329095+0000","last_fullsized":"2024-09-10T19:02:24.329095+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:21.983118+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:02:21.983118+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:37.401129+0000","scrub_duration":0.0056724799999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"232'29","reported_seq":425,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.324567+0000","last_change":"2024-09-10T18:58:58.837250+0000","last_active":"2024-09-10T19:02:24.324567+0000","last_peered":"2024-09-10T19:02:24.324567+0000","last_clean":"2024-09-10T19:02:24.324567+0000","last_became_active":"2024-09-10T18:57:02.442862+0000","last_became_peered":"2024-09-10T18:57:02.442862+0000","last_unstale":"2024-09-10T19:02:24.324567+0000","last_undegraded":"2024-09-10T19:02:24.324567+0000","last_fullsized":"2024-09-10T19:02:24.324567+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'29","last_scrub_stamp":"2024-09-10T18:58:58.837163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:58:58.837163+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.019169703999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":363,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.323501+0000","last_change":"2024-09-10T18:59:53.858914+0000","last_active":"2024-09-10T19:02:24.323501+0000","last_peered":"2024-09-10T19:02:24.323501+0000","last_clean":"2024-09-10T19:02:24.323501+0000","last_became_active":"2024-09-10T18:57:02.443522+0000","last_became_peered":"2024-09-10T18:57:02.443522+0000","last_unstale":"2024-09-10T19:02:24.323501+0000","last_undegraded":"2024-09-10T19:02:24.323501+0000","last_fullsized":"2024-09-10T19:02:24.323501+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:53.858807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:48.196065+0000","last_clean_scrub_stamp":"2024-09-10T18:59:53.858807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044256060000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":403,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.327674+0000","last_change":"2024-09-10T19:02:17.064956+0000","last_active":"2024-09-10T19:02:24.327674+0000","last_peered":"2024-09-10T19:02:24.327674+0000","last_clean":"2024-09-10T19:02:24.327674+0000","last_became_active":"2024-09-10T18:55:40.387857+0000","last_became_peered":"2024-09-10T18:55:40.387857+0000","last_unstale":"2024-09-10T19:02:24.327674+0000","last_undegraded":"2024-09-10T19:02:24.327674+0000","last_fullsized":"2024-09-10T19:02:24.327674+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:17.064877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:02:17.064877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:30.026713+0000","scrub_duration":0.0089490170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":383,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.327829+0000","last_change":"2024-09-10T18:59:16.235312+0000","last_active":"2024-09-10T19:02:24.327829+0000","last_peered":"2024-09-10T19:02:24.327829+0000","last_clean":"2024-09-10T19:02:24.327829+0000","last_became_active":"2024-09-10T18:57:02.442479+0000","last_became_peered":"2024-09-10T18:57:02.442479+0000","last_unstale":"2024-09-10T19:02:24.327829+0000","last_undegraded":"2024-09-10T19:02:24.327829+0000","last_fullsized":"2024-09-10T19:02:24.327829+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:16.235235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:49:18.281230+0000","last_clean_scrub_stamp":"2024-09-10T18:59:16.235235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034192950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"368'15277","reported_seq":16925,"reported_epoch":368,"state":"active+clean","last_fresh":"2024-09-10T19:02:31.778716+0000","last_change":"2024-09-10T19:00:08.469551+0000","last_active":"2024-09-10T19:02:31.778716+0000","last_peered":"2024-09-10T19:02:31.778716+0000","last_clean":"2024-09-10T19:02:31.778716+0000","last_became_active":"2024-09-10T19:00:08.465795+0000","last_became_peered":"2024-09-10T19:00:08.465795+0000","last_unstale":"2024-09-10T19:02:31.778716+0000","last_undegraded":"2024-09-10T19:02:31.778716+0000","last_fullsized":"2024-09-10T19:02:31.778716+0000","mapping_epoch":329,"log_start":"361'11800","ondisk_log_start":"361'11800","created":315,"last_epoch_clean":330,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3477,"log_dups_size":0,"ondisk_log_size":3477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":114360320,"num_objects":1745,"num_object_clones":0,"num_object_copies":5235,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15277,"num_write_kb":544704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":784,"num_bytes_recovered":51380224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"70'81","reported_seq":386,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.329495+0000","last_change":"2024-09-10T18:59:53.550489+0000","last_active":"2024-09-10T19:02:24.329495+0000","last_peered":"2024-09-10T19:02:24.329495+0000","last_clean":"2024-09-10T19:02:24.329495+0000","last_became_active":"2024-09-10T18:57:02.452659+0000","last_became_peered":"2024-09-10T18:57:02.452659+0000","last_unstale":"2024-09-10T19:02:24.329495+0000","last_undegraded":"2024-09-10T19:02:24.329495+0000","last_fullsized":"2024-09-10T19:02:24.329495+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:53.550433+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:59:53.550433+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013042399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"232'25","reported_seq":486,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.327767+0000","last_change":"2024-09-10T19:00:42.531653+0000","last_active":"2024-09-10T19:02:24.327767+0000","last_peered":"2024-09-10T19:02:24.327767+0000","last_clean":"2024-09-10T19:02:24.327767+0000","last_became_active":"2024-09-10T19:00:42.517353+0000","last_became_peered":"2024-09-10T19:00:42.517353+0000","last_unstale":"2024-09-10T19:02:24.327767+0000","last_undegraded":"2024-09-10T19:02:24.327767+0000","last_fullsized":"2024-09-10T19:02:24.327767+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'25","last_scrub_stamp":"2024-09-10T19:00:07.390135+0000","last_deep_scrub":"83'15","last_deep_scrub_stamp":"2024-09-10T18:46:58.436473+0000","last_clean_scrub_stamp":"2024-09-10T19:00:07.390135+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002339874,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":419,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.329577+0000","last_change":"2024-09-10T19:02:21.267129+0000","last_active":"2024-09-10T19:02:24.329577+0000","last_peered":"2024-09-10T19:02:24.329577+0000","last_clean":"2024-09-10T19:02:24.329577+0000","last_became_active":"2024-09-10T18:57:02.447940+0000","last_became_peered":"2024-09-10T18:57:02.447940+0000","last_unstale":"2024-09-10T19:02:24.329577+0000","last_undegraded":"2024-09-10T19:02:24.329577+0000","last_fullsized":"2024-09-10T19:02:24.329577+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:21.266828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:05.140027+0000","last_clean_scrub_stamp":"2024-09-10T19:02:21.266828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:23.752272+0000","scrub_duration":0.011278888000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"339'13","reported_seq":549,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:26.278701+0000","last_change":"2024-09-10T19:00:42.528804+0000","last_active":"2024-09-10T19:02:26.278701+0000","last_peered":"2024-09-10T19:02:26.278701+0000","last_clean":"2024-09-10T19:02:26.278701+0000","last_became_active":"2024-09-10T19:00:42.520360+0000","last_became_peered":"2024-09-10T19:00:42.520360+0000","last_unstale":"2024-09-10T19:02:26.278701+0000","last_undegraded":"2024-09-10T19:02:26.278701+0000","last_fullsized":"2024-09-10T19:02:26.278701+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'12","last_scrub_stamp":"2024-09-10T18:59:55.943869+0000","last_deep_scrub":"258'12","last_deep_scrub_stamp":"2024-09-10T18:56:21.098770+0000","last_clean_scrub_stamp":"2024-09-10T18:59:55.943869+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.020416790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":359,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.330439+0000","last_change":"2024-09-10T19:02:14.738026+0000","last_active":"2024-09-10T19:02:24.330439+0000","last_peered":"2024-09-10T19:02:24.330439+0000","last_clean":"2024-09-10T19:02:24.330439+0000","last_became_active":"2024-09-10T19:00:42.515144+0000","last_became_peered":"2024-09-10T19:00:42.515144+0000","last_unstale":"2024-09-10T19:02:24.330439+0000","last_undegraded":"2024-09-10T19:02:24.330439+0000","last_fullsized":"2024-09-10T19:02:24.330439+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:14.737936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:02:14.737936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:27.822745+0000","scrub_duration":0.003039609,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.a","version":"368'15846","reported_seq":15917,"reported_epoch":368,"state":"active+clean","last_fresh":"2024-09-10T19:02:31.547901+0000","last_change":"2024-09-10T18:59:48.739701+0000","last_active":"2024-09-10T19:02:31.547901+0000","last_peered":"2024-09-10T19:02:31.547901+0000","last_clean":"2024-09-10T19:02:31.547901+0000","last_became_active":"2024-09-10T18:59:48.739415+0000","last_became_peered":"2024-09-10T18:59:48.739415+0000","last_unstale":"2024-09-10T19:02:31.547901+0000","last_undegraded":"2024-09-10T19:02:31.547901+0000","last_fullsized":"2024-09-10T19:02:31.547901+0000","mapping_epoch":320,"log_start":"361'12300","ondisk_log_start":"361'12300","created":315,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3546,"log_dups_size":0,"ondisk_log_size":3546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":120061952,"num_objects":1832,"num_object_clones":0,"num_object_copies":5496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15846,"num_write_kb":565696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"70'81","reported_seq":317,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.325504+0000","last_change":"2024-09-10T19:00:40.472680+0000","last_active":"2024-09-10T19:02:24.325504+0000","last_peered":"2024-09-10T19:02:24.325504+0000","last_clean":"2024-09-10T19:02:24.325504+0000","last_became_active":"2024-09-10T18:55:40.392432+0000","last_became_peered":"2024-09-10T18:55:40.392432+0000","last_unstale":"2024-09-10T19:02:24.325504+0000","last_undegraded":"2024-09-10T19:02:24.325504+0000","last_fullsized":"2024-09-10T19:02:24.325504+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:00:40.472595+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:00:40.472595+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0072897099999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"232'41","reported_seq":685,"reported_epoch":368,"state":"active+clean","last_fresh":"2024-09-10T19:02:27.330765+0000","last_change":"2024-09-10T19:02:12.270711+0000","last_active":"2024-09-10T19:02:27.330765+0000","last_peered":"2024-09-10T19:02:27.330765+0000","last_clean":"2024-09-10T19:02:27.330765+0000","last_became_active":"2024-09-10T18:47:36.420185+0000","last_became_peered":"2024-09-10T18:47:36.420185+0000","last_unstale":"2024-09-10T19:02:27.330765+0000","last_undegraded":"2024-09-10T19:02:27.330765+0000","last_fullsized":"2024-09-10T19:02:27.330765+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'41","last_scrub_stamp":"2024-09-10T19:02:12.270610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:12.270610+0000","objects_scrubbed":10,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:41.719724+0000","scrub_duration":0.0028300339999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":392,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.321732+0000","last_change":"2024-09-10T19:02:13.666158+0000","last_active":"2024-09-10T19:02:24.321732+0000","last_peered":"2024-09-10T19:02:24.321732+0000","last_clean":"2024-09-10T19:02:24.321732+0000","last_became_active":"2024-09-10T18:45:17.964494+0000","last_became_peered":"2024-09-10T18:45:17.964494+0000","last_unstale":"2024-09-10T19:02:24.321732+0000","last_undegraded":"2024-09-10T19:02:24.321732+0000","last_fullsized":"2024-09-10T19:02:24.321732+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:13.665999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:12.982599+0000","last_clean_scrub_stamp":"2024-09-10T19:02:13.665999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:15.216961+0000","scrub_duration":0.007648719,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":334,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.328045+0000","last_change":"2024-09-10T19:02:01.110434+0000","last_active":"2024-09-10T19:02:24.328045+0000","last_peered":"2024-09-10T19:02:24.328045+0000","last_clean":"2024-09-10T19:02:24.328045+0000","last_became_active":"2024-09-10T18:47:57.107478+0000","last_became_peered":"2024-09-10T18:47:57.107478+0000","last_unstale":"2024-09-10T19:02:24.328045+0000","last_undegraded":"2024-09-10T19:02:24.328045+0000","last_fullsized":"2024-09-10T19:02:24.328045+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:01.110335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:02:01.110335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:12.298320+0000","scrub_duration":0.0062803349999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":372,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.328209+0000","last_change":"2024-09-10T19:02:09.267386+0000","last_active":"2024-09-10T19:02:24.328209+0000","last_peered":"2024-09-10T19:02:24.328209+0000","last_clean":"2024-09-10T19:02:24.328209+0000","last_became_active":"2024-09-10T18:57:02.460594+0000","last_became_peered":"2024-09-10T18:57:02.460594+0000","last_unstale":"2024-09-10T19:02:24.328209+0000","last_undegraded":"2024-09-10T19:02:24.328209+0000","last_fullsized":"2024-09-10T19:02:24.328209+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:09.267253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:22.891084+0000","last_clean_scrub_stamp":"2024-09-10T19:02:09.267253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:28.464433+0000","scrub_duration":0.0067979290000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"368'15524","reported_seq":29808,"reported_epoch":368,"state":"active+clean","last_fresh":"2024-09-10T19:02:31.594132+0000","last_change":"2024-09-10T19:01:25.571972+0000","last_active":"2024-09-10T19:02:31.594132+0000","last_peered":"2024-09-10T19:02:31.594132+0000","last_clean":"2024-09-10T19:02:31.594132+0000","last_became_active":"2024-09-10T19:01:25.526168+0000","last_became_peered":"2024-09-10T19:01:25.526168+0000","last_unstale":"2024-09-10T19:02:31.594132+0000","last_undegraded":"2024-09-10T19:02:31.594132+0000","last_fullsized":"2024-09-10T19:02:31.594132+0000","mapping_epoch":355,"log_start":"360'11938","ondisk_log_start":"360'11938","created":315,"last_epoch_clean":356,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3586,"log_dups_size":0,"ondisk_log_size":3586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115605504,"num_objects":1764,"num_object_clones":0,"num_object_copies":5292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15524,"num_write_kb":553216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7098,"num_bytes_recovered":465108992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"70'81","reported_seq":185,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.331812+0000","last_change":"2024-09-10T18:59:11.875713+0000","last_active":"2024-09-10T19:02:24.331812+0000","last_peered":"2024-09-10T19:02:24.331812+0000","last_clean":"2024-09-10T19:02:24.331812+0000","last_became_active":"2024-09-10T18:57:02.437944+0000","last_became_peered":"2024-09-10T18:57:02.437944+0000","last_unstale":"2024-09-10T19:02:24.331812+0000","last_undegraded":"2024-09-10T19:02:24.331812+0000","last_fullsized":"2024-09-10T19:02:24.331812+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:11.875625+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:59:11.875625+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040920360000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"232'22","reported_seq":429,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.329044+0000","last_change":"2024-09-10T19:00:08.468265+0000","last_active":"2024-09-10T19:02:24.329044+0000","last_peered":"2024-09-10T19:02:24.329044+0000","last_clean":"2024-09-10T19:02:24.329044+0000","last_became_active":"2024-09-10T18:55:40.397422+0000","last_became_peered":"2024-09-10T18:55:40.397422+0000","last_unstale":"2024-09-10T19:02:24.329044+0000","last_undegraded":"2024-09-10T19:02:24.329044+0000","last_fullsized":"2024-09-10T19:02:24.329044+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'22","last_scrub_stamp":"2024-09-10T19:00:08.468154+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-10T18:47:10.514951+0000","last_clean_scrub_stamp":"2024-09-10T19:00:08.468154+0000","objects_scrubbed":5,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024987170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":25,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"84'6","reported_seq":1430,"reported_epoch":368,"state":"active+clean","last_fresh":"2024-09-10T19:02:30.829164+0000","last_change":"2024-09-10T19:00:40.504252+0000","last_active":"2024-09-10T19:02:30.829164+0000","last_peered":"2024-09-10T19:02:30.829164+0000","last_clean":"2024-09-10T19:02:30.829164+0000","last_became_active":"2024-09-10T18:55:40.403098+0000","last_became_peered":"2024-09-10T18:55:40.403098+0000","last_unstale":"2024-09-10T19:02:30.829164+0000","last_undegraded":"2024-09-10T19:02:30.829164+0000","last_fullsized":"2024-09-10T19:02:30.829164+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'6","last_scrub_stamp":"2024-09-10T19:00:40.504174+0000","last_deep_scrub":"84'6","last_deep_scrub_stamp":"2024-09-10T18:47:22.545450+0000","last_clean_scrub_stamp":"2024-09-10T19:00:40.504174+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0052333889999999997,"objects_trimmed":0,"snaptrim_duration":0.0021933159999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1017,"num_read_kb":1012,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":346,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.325060+0000","last_change":"2024-09-10T19:00:08.141664+0000","last_active":"2024-09-10T19:02:24.325060+0000","last_peered":"2024-09-10T19:02:24.325060+0000","last_clean":"2024-09-10T19:02:24.325060+0000","last_became_active":"2024-09-10T18:55:40.393389+0000","last_became_peered":"2024-09-10T18:55:40.393389+0000","last_unstale":"2024-09-10T19:02:24.325060+0000","last_undegraded":"2024-09-10T19:02:24.325060+0000","last_fullsized":"2024-09-10T19:02:24.325060+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:08.141603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:00:08.141603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001270729,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":363,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.323989+0000","last_change":"2024-09-10T19:02:09.600216+0000","last_active":"2024-09-10T19:02:24.323989+0000","last_peered":"2024-09-10T19:02:24.323989+0000","last_clean":"2024-09-10T19:02:24.323989+0000","last_became_active":"2024-09-10T18:57:02.438945+0000","last_became_peered":"2024-09-10T18:57:02.438945+0000","last_unstale":"2024-09-10T19:02:24.323989+0000","last_undegraded":"2024-09-10T19:02:24.323989+0000","last_fullsized":"2024-09-10T19:02:24.323989+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:09.600134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:02:09.600134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:13.909394+0000","scrub_duration":0.003812368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":120,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.328203+0000","last_change":"2024-09-10T19:00:41.794168+0000","last_active":"2024-09-10T19:02:24.328203+0000","last_peered":"2024-09-10T19:02:24.328203+0000","last_clean":"2024-09-10T19:02:24.328203+0000","last_became_active":"2024-09-10T18:57:02.453865+0000","last_became_peered":"2024-09-10T18:57:02.453865+0000","last_unstale":"2024-09-10T19:02:24.328203+0000","last_undegraded":"2024-09-10T19:02:24.328203+0000","last_fullsized":"2024-09-10T19:02:24.328203+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:41.794116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:07.150480+0000","last_clean_scrub_stamp":"2024-09-10T19:00:41.794116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001296148,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.8","version":"368'15223","reported_seq":15284,"reported_epoch":368,"state":"active+clean","last_fresh":"2024-09-10T19:02:28.787481+0000","last_change":"2024-09-10T18:59:42.712702+0000","last_active":"2024-09-10T19:02:28.787481+0000","last_peered":"2024-09-10T19:02:28.787481+0000","last_clean":"2024-09-10T19:02:28.787481+0000","last_became_active":"2024-09-10T18:59:42.712397+0000","last_became_peered":"2024-09-10T18:59:42.712397+0000","last_unstale":"2024-09-10T19:02:28.787481+0000","last_undegraded":"2024-09-10T19:02:28.787481+0000","last_fullsized":"2024-09-10T19:02:28.787481+0000","mapping_epoch":315,"log_start":"359'11500","ondisk_log_start":"359'11500","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3723,"log_dups_size":0,"ondisk_log_size":3723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":132972544,"num_objects":2029,"num_object_clones":0,"num_object_copies":6087,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15223,"num_write_kb":552064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,4],"acting":[0,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"70'81","reported_seq":286,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.329862+0000","last_change":"2024-09-10T18:58:49.141636+0000","last_active":"2024-09-10T19:02:24.329862+0000","last_peered":"2024-09-10T19:02:24.329862+0000","last_clean":"2024-09-10T19:02:24.329862+0000","last_became_active":"2024-09-10T18:52:14.589009+0000","last_became_peered":"2024-09-10T18:52:14.589009+0000","last_unstale":"2024-09-10T19:02:24.329862+0000","last_undegraded":"2024-09-10T19:02:24.329862+0000","last_fullsized":"2024-09-10T19:02:24.329862+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:58:49.141013+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:58:49.141013+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0036869540000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"232'24","reported_seq":105,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.321804+0000","last_change":"2024-09-10T19:00:42.532827+0000","last_active":"2024-09-10T19:02:24.321804+0000","last_peered":"2024-09-10T19:02:24.321804+0000","last_clean":"2024-09-10T19:02:24.321804+0000","last_became_active":"2024-09-10T19:00:42.516521+0000","last_became_peered":"2024-09-10T19:00:42.516521+0000","last_unstale":"2024-09-10T19:02:24.321804+0000","last_undegraded":"2024-09-10T19:02:24.321804+0000","last_fullsized":"2024-09-10T19:02:24.321804+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T19:00:39.748509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:00:39.748509+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027783730000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"305'2","reported_seq":352,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.323984+0000","last_change":"2024-09-10T18:59:18.897245+0000","last_active":"2024-09-10T19:02:24.323984+0000","last_peered":"2024-09-10T19:02:24.323984+0000","last_clean":"2024-09-10T19:02:24.323984+0000","last_became_active":"2024-09-10T18:55:40.390022+0000","last_became_peered":"2024-09-10T18:55:40.390022+0000","last_unstale":"2024-09-10T19:02:24.323984+0000","last_undegraded":"2024-09-10T19:02:24.323984+0000","last_fullsized":"2024-09-10T19:02:24.323984+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"305'2","last_scrub_stamp":"2024-09-10T18:59:18.897059+0000","last_deep_scrub":"305'2","last_deep_scrub_stamp":"2024-09-10T18:59:18.897059+0000","last_clean_scrub_stamp":"2024-09-10T18:59:18.897059+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016452700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":323,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.327723+0000","last_change":"2024-09-10T19:02:06.273188+0000","last_active":"2024-09-10T19:02:24.327723+0000","last_peered":"2024-09-10T19:02:24.327723+0000","last_clean":"2024-09-10T19:02:24.327723+0000","last_became_active":"2024-09-10T18:57:02.447195+0000","last_became_peered":"2024-09-10T18:57:02.447195+0000","last_unstale":"2024-09-10T19:02:24.327723+0000","last_undegraded":"2024-09-10T19:02:24.327723+0000","last_fullsized":"2024-09-10T19:02:24.327723+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:06.272690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:06.272690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:14.146873+0000","scrub_duration":0.0032789619999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":39,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.323782+0000","last_change":"2024-09-10T19:00:42.518887+0000","last_active":"2024-09-10T19:02:24.323782+0000","last_peered":"2024-09-10T19:02:24.323782+0000","last_clean":"2024-09-10T19:02:24.323782+0000","last_became_active":"2024-09-10T19:00:42.518566+0000","last_became_peered":"2024-09-10T19:00:42.518566+0000","last_unstale":"2024-09-10T19:02:24.323782+0000","last_undegraded":"2024-09-10T19:02:24.323782+0000","last_fullsized":"2024-09-10T19:02:24.323782+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:39.752722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:00:39.752722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":379,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.329212+0000","last_change":"2024-09-10T19:02:20.272847+0000","last_active":"2024-09-10T19:02:24.329212+0000","last_peered":"2024-09-10T19:02:24.329212+0000","last_clean":"2024-09-10T19:02:24.329212+0000","last_became_active":"2024-09-10T18:45:24.083689+0000","last_became_peered":"2024-09-10T18:45:24.083689+0000","last_unstale":"2024-09-10T19:02:24.329212+0000","last_undegraded":"2024-09-10T19:02:24.329212+0000","last_fullsized":"2024-09-10T19:02:24.329212+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:20.272756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:02:20.272756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:26.962724+0000","scrub_duration":0.0046676249999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"368'15752","reported_seq":15813,"reported_epoch":368,"state":"active+clean","last_fresh":"2024-09-10T19:02:31.537108+0000","last_change":"2024-09-10T18:59:42.717031+0000","last_active":"2024-09-10T19:02:31.537108+0000","last_peered":"2024-09-10T19:02:31.537108+0000","last_clean":"2024-09-10T19:02:31.537108+0000","last_became_active":"2024-09-10T18:59:42.716730+0000","last_became_peered":"2024-09-10T18:59:42.716730+0000","last_unstale":"2024-09-10T19:02:31.537108+0000","last_undegraded":"2024-09-10T19:02:31.537108+0000","last_fullsized":"2024-09-10T19:02:31.537108+0000","mapping_epoch":315,"log_start":"361'12200","ondisk_log_start":"361'12200","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3552,"log_dups_size":0,"ondisk_log_size":3552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":122421248,"num_objects":1868,"num_object_clones":0,"num_object_copies":5604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15752,"num_write_kb":563840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"232'33","reported_seq":471,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.328277+0000","last_change":"2024-09-10T19:02:16.059463+0000","last_active":"2024-09-10T19:02:24.328277+0000","last_peered":"2024-09-10T19:02:24.328277+0000","last_clean":"2024-09-10T19:02:24.328277+0000","last_became_active":"2024-09-10T18:57:02.435499+0000","last_became_peered":"2024-09-10T18:57:02.435499+0000","last_unstale":"2024-09-10T19:02:24.328277+0000","last_undegraded":"2024-09-10T19:02:24.328277+0000","last_fullsized":"2024-09-10T19:02:24.328277+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'33","last_scrub_stamp":"2024-09-10T19:02:16.059377+0000","last_deep_scrub":"232'33","last_deep_scrub_stamp":"2024-09-10T18:56:21.206960+0000","last_clean_scrub_stamp":"2024-09-10T19:02:16.059377+0000","objects_scrubbed":7,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:23.273294+0000","scrub_duration":0.0042245340000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"60'2","reported_seq":381,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.327836+0000","last_change":"2024-09-10T18:58:55.931801+0000","last_active":"2024-09-10T19:02:24.327836+0000","last_peered":"2024-09-10T19:02:24.327836+0000","last_clean":"2024-09-10T19:02:24.327836+0000","last_became_active":"2024-09-10T18:45:17.961660+0000","last_became_peered":"2024-09-10T18:45:17.961660+0000","last_unstale":"2024-09-10T19:02:24.327836+0000","last_undegraded":"2024-09-10T19:02:24.327836+0000","last_fullsized":"2024-09-10T19:02:24.327836+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-10T18:58:55.931717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:58:55.931717+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040038909999999999,"objects_trimmed":0,"snaptrim_duration":0.0013942640000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":409,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.321577+0000","last_change":"2024-09-10T19:01:56.581104+0000","last_active":"2024-09-10T19:02:24.321577+0000","last_peered":"2024-09-10T19:02:24.321577+0000","last_clean":"2024-09-10T19:02:24.321577+0000","last_became_active":"2024-09-10T18:57:02.445712+0000","last_became_peered":"2024-09-10T18:57:02.445712+0000","last_unstale":"2024-09-10T19:02:24.321577+0000","last_undegraded":"2024-09-10T19:02:24.321577+0000","last_fullsized":"2024-09-10T19:02:24.321577+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:01:56.581018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:01:56.581018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:02.923984+0000","scrub_duration":0.0048884530000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'81","reported_seq":185,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.327022+0000","last_change":"2024-09-10T18:59:14.873293+0000","last_active":"2024-09-10T19:02:24.327022+0000","last_peered":"2024-09-10T19:02:24.327022+0000","last_clean":"2024-09-10T19:02:24.327022+0000","last_became_active":"2024-09-10T18:57:02.440063+0000","last_became_peered":"2024-09-10T18:57:02.440063+0000","last_unstale":"2024-09-10T19:02:24.327022+0000","last_undegraded":"2024-09-10T19:02:24.327022+0000","last_fullsized":"2024-09-10T19:02:24.327022+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:14.873083+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:56.085580+0000","last_clean_scrub_stamp":"2024-09-10T18:59:14.873083+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0063632760000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":378,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.324156+0000","last_change":"2024-09-10T19:01:55.620765+0000","last_active":"2024-09-10T19:02:24.324156+0000","last_peered":"2024-09-10T19:02:24.324156+0000","last_clean":"2024-09-10T19:02:24.324156+0000","last_became_active":"2024-09-10T18:57:02.431556+0000","last_became_peered":"2024-09-10T18:57:02.431556+0000","last_unstale":"2024-09-10T19:02:24.324156+0000","last_undegraded":"2024-09-10T19:02:24.324156+0000","last_fullsized":"2024-09-10T19:02:24.324156+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:01:55.620696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:01:55.620696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:02:57.091592+0000","scrub_duration":0.0068284349999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":383,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.324353+0000","last_change":"2024-09-10T19:00:07.624756+0000","last_active":"2024-09-10T19:02:24.324353+0000","last_peered":"2024-09-10T19:02:24.324353+0000","last_clean":"2024-09-10T19:02:24.324353+0000","last_became_active":"2024-09-10T18:45:24.073327+0000","last_became_peered":"2024-09-10T18:45:24.073327+0000","last_unstale":"2024-09-10T19:02:24.324353+0000","last_undegraded":"2024-09-10T19:02:24.324353+0000","last_fullsized":"2024-09-10T19:02:24.324353+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:07.624701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:00:07.624701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001137962,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"368'14982","reported_seq":15043,"reported_epoch":368,"state":"active+clean","last_fresh":"2024-09-10T19:02:28.790352+0000","last_change":"2024-09-10T18:59:43.009491+0000","last_active":"2024-09-10T19:02:28.790352+0000","last_peered":"2024-09-10T19:02:28.790352+0000","last_clean":"2024-09-10T19:02:28.790352+0000","last_became_active":"2024-09-10T18:59:43.009210+0000","last_became_peered":"2024-09-10T18:59:43.009210+0000","last_unstale":"2024-09-10T19:02:28.790352+0000","last_undegraded":"2024-09-10T19:02:28.790352+0000","last_fullsized":"2024-09-10T19:02:28.790352+0000","mapping_epoch":315,"log_start":"359'11300","ondisk_log_start":"359'11300","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3682,"log_dups_size":0,"ondisk_log_size":3682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":139853824,"num_objects":2134,"num_object_clones":0,"num_object_copies":6402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14982,"num_write_kb":547712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"70'81","reported_seq":388,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.329298+0000","last_change":"2024-09-10T19:02:13.085183+0000","last_active":"2024-09-10T19:02:24.329298+0000","last_peered":"2024-09-10T19:02:24.329298+0000","last_clean":"2024-09-10T19:02:24.329298+0000","last_became_active":"2024-09-10T18:52:16.598295+0000","last_became_peered":"2024-09-10T18:52:16.598295+0000","last_unstale":"2024-09-10T19:02:24.329298+0000","last_undegraded":"2024-09-10T19:02:24.329298+0000","last_fullsized":"2024-09-10T19:02:24.329298+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:13.085003+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:53.609885+0000","last_clean_scrub_stamp":"2024-09-10T19:02:13.085003+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:36.454967+0000","scrub_duration":0.00305172,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"232'35","reported_seq":476,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.328529+0000","last_change":"2024-09-10T19:00:43.819111+0000","last_active":"2024-09-10T19:02:24.328529+0000","last_peered":"2024-09-10T19:02:24.328529+0000","last_clean":"2024-09-10T19:02:24.328529+0000","last_became_active":"2024-09-10T18:57:02.436059+0000","last_became_peered":"2024-09-10T18:57:02.436059+0000","last_unstale":"2024-09-10T19:02:24.328529+0000","last_undegraded":"2024-09-10T19:02:24.328529+0000","last_fullsized":"2024-09-10T19:02:24.328529+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'35","last_scrub_stamp":"2024-09-10T19:00:43.819004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:00:43.819004+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0050234040000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":357,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.324230+0000","last_change":"2024-09-10T19:01:53.609239+0000","last_active":"2024-09-10T19:02:24.324230+0000","last_peered":"2024-09-10T19:02:24.324230+0000","last_clean":"2024-09-10T19:02:24.324230+0000","last_became_active":"2024-09-10T18:55:40.391849+0000","last_became_peered":"2024-09-10T18:55:40.391849+0000","last_unstale":"2024-09-10T19:02:24.324230+0000","last_undegraded":"2024-09-10T19:02:24.324230+0000","last_fullsized":"2024-09-10T19:02:24.324230+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:01:53.609184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:01:53.609184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:10.204226+0000","scrub_duration":0.024484227000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":328,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.324273+0000","last_change":"2024-09-10T19:00:42.738386+0000","last_active":"2024-09-10T19:02:24.324273+0000","last_peered":"2024-09-10T19:02:24.324273+0000","last_clean":"2024-09-10T19:02:24.324273+0000","last_became_active":"2024-09-10T19:00:42.514709+0000","last_became_peered":"2024-09-10T19:00:42.514709+0000","last_unstale":"2024-09-10T19:02:24.324273+0000","last_undegraded":"2024-09-10T19:02:24.324273+0000","last_fullsized":"2024-09-10T19:02:24.324273+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:42.738331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:00:42.738331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001211314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":165,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.324752+0000","last_change":"2024-09-10T18:57:03.015513+0000","last_active":"2024-09-10T19:02:24.324752+0000","last_peered":"2024-09-10T19:02:24.324752+0000","last_clean":"2024-09-10T19:02:24.324752+0000","last_became_active":"2024-09-10T18:57:02.468866+0000","last_became_peered":"2024-09-10T18:57:02.468866+0000","last_unstale":"2024-09-10T19:02:24.324752+0000","last_undegraded":"2024-09-10T19:02:24.324752+0000","last_fullsized":"2024-09-10T19:02:24.324752+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:57:03.015457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:57:03.015457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001545795,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.5","version":"368'15094","reported_seq":29516,"reported_epoch":368,"state":"active+clean","last_fresh":"2024-09-10T19:02:28.767379+0000","last_change":"2024-09-10T19:01:29.582446+0000","last_active":"2024-09-10T19:02:28.767379+0000","last_peered":"2024-09-10T19:02:28.767379+0000","last_clean":"2024-09-10T19:02:28.767379+0000","last_became_active":"2024-09-10T19:01:29.559761+0000","last_became_peered":"2024-09-10T19:01:29.559761+0000","last_unstale":"2024-09-10T19:02:28.767379+0000","last_undegraded":"2024-09-10T19:02:28.767379+0000","last_fullsized":"2024-09-10T19:02:28.767379+0000","mapping_epoch":357,"log_start":"359'11400","ondisk_log_start":"359'11400","created":315,"last_epoch_clean":358,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3694,"log_dups_size":0,"ondisk_log_size":3694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134348800,"num_objects":2050,"num_object_clones":0,"num_object_copies":6150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15094,"num_write_kb":548608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7168,"num_bytes_recovered":469762048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.e","version":"70'81","reported_seq":246,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.325894+0000","last_change":"2024-09-10T18:59:23.592755+0000","last_active":"2024-09-10T19:02:24.325894+0000","last_peered":"2024-09-10T19:02:24.325894+0000","last_clean":"2024-09-10T19:02:24.325894+0000","last_became_active":"2024-09-10T18:57:02.449115+0000","last_became_peered":"2024-09-10T18:57:02.449115+0000","last_unstale":"2024-09-10T19:02:24.325894+0000","last_undegraded":"2024-09-10T19:02:24.325894+0000","last_fullsized":"2024-09-10T19:02:24.325894+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:23.592672+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:59:23.592672+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038976470000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"232'19","reported_seq":384,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.324719+0000","last_change":"2024-09-10T19:00:40.748808+0000","last_active":"2024-09-10T19:02:24.324719+0000","last_peered":"2024-09-10T19:02:24.324719+0000","last_clean":"2024-09-10T19:02:24.324719+0000","last_became_active":"2024-09-10T18:55:40.395257+0000","last_became_peered":"2024-09-10T18:55:40.395257+0000","last_unstale":"2024-09-10T19:02:24.324719+0000","last_undegraded":"2024-09-10T19:02:24.324719+0000","last_fullsized":"2024-09-10T19:02:24.324719+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T19:00:40.748713+0000","last_deep_scrub":"130'13","last_deep_scrub_stamp":"2024-09-10T18:51:58.427534+0000","last_clean_scrub_stamp":"2024-09-10T19:00:40.748713+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043426029999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":369,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.329315+0000","last_change":"2024-09-10T18:59:09.206400+0000","last_active":"2024-09-10T19:02:24.329315+0000","last_peered":"2024-09-10T19:02:24.329315+0000","last_clean":"2024-09-10T19:02:24.329315+0000","last_became_active":"2024-09-10T18:45:17.963057+0000","last_became_peered":"2024-09-10T18:45:17.963057+0000","last_unstale":"2024-09-10T19:02:24.329315+0000","last_undegraded":"2024-09-10T19:02:24.329315+0000","last_fullsized":"2024-09-10T19:02:24.329315+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:09.206317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:59:09.206317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.018986050000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":338,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.325695+0000","last_change":"2024-09-10T19:01:12.880399+0000","last_active":"2024-09-10T19:02:24.325695+0000","last_peered":"2024-09-10T19:02:24.325695+0000","last_clean":"2024-09-10T19:02:24.325695+0000","last_became_active":"2024-09-10T18:55:40.390771+0000","last_became_peered":"2024-09-10T18:55:40.390771+0000","last_unstale":"2024-09-10T19:02:24.325695+0000","last_undegraded":"2024-09-10T19:02:24.325695+0000","last_fullsized":"2024-09-10T19:02:24.325695+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:17.661832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_clean_scrub_stamp":"2024-09-10T18:56:17.661832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041460890000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":372,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.323607+0000","last_change":"2024-09-10T19:01:58.603370+0000","last_active":"2024-09-10T19:02:24.323607+0000","last_peered":"2024-09-10T19:02:24.323607+0000","last_clean":"2024-09-10T19:02:24.323607+0000","last_became_active":"2024-09-10T18:57:02.452387+0000","last_became_peered":"2024-09-10T18:57:02.452387+0000","last_unstale":"2024-09-10T19:02:24.323607+0000","last_undegraded":"2024-09-10T19:02:24.323607+0000","last_fullsized":"2024-09-10T19:02:24.323607+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:01:58.601096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:53.345184+0000","last_clean_scrub_stamp":"2024-09-10T19:01:58.601096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:02:59.732936+0000","scrub_duration":0.0026721589999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.6","version":"368'15514","reported_seq":15575,"reported_epoch":368,"state":"active+clean","last_fresh":"2024-09-10T19:02:31.589800+0000","last_change":"2024-09-10T18:59:42.715562+0000","last_active":"2024-09-10T19:02:31.589800+0000","last_peered":"2024-09-10T19:02:31.589800+0000","last_clean":"2024-09-10T19:02:31.589800+0000","last_became_active":"2024-09-10T18:59:42.715275+0000","last_became_peered":"2024-09-10T18:59:42.715275+0000","last_unstale":"2024-09-10T19:02:31.589800+0000","last_undegraded":"2024-09-10T19:02:31.589800+0000","last_fullsized":"2024-09-10T19:02:31.589800+0000","mapping_epoch":315,"log_start":"360'11844","ondisk_log_start":"360'11844","created":315,"last_epoch_clean":316,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3670,"log_dups_size":0,"ondisk_log_size":3670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115736576,"num_objects":1766,"num_object_clones":0,"num_object_copies":5298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15514,"num_write_kb":552960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.d","version":"70'81","reported_seq":187,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.326463+0000","last_change":"2024-09-10T19:02:01.044879+0000","last_active":"2024-09-10T19:02:24.326463+0000","last_peered":"2024-09-10T19:02:24.326463+0000","last_clean":"2024-09-10T19:02:24.326463+0000","last_became_active":"2024-09-10T18:57:02.439652+0000","last_became_peered":"2024-09-10T18:57:02.439652+0000","last_unstale":"2024-09-10T19:02:24.326463+0000","last_undegraded":"2024-09-10T19:02:24.326463+0000","last_fullsized":"2024-09-10T19:02:24.326463+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:01.044143+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:02:01.044143+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:28.935452+0000","scrub_duration":0.014088588000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"365'34","reported_seq":72,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.324753+0000","last_change":"2024-09-10T19:02:18.712390+0000","last_active":"2024-09-10T19:02:24.324753+0000","last_peered":"2024-09-10T19:02:24.324753+0000","last_clean":"2024-09-10T19:02:24.324753+0000","last_became_active":"2024-09-10T19:00:42.515903+0000","last_became_peered":"2024-09-10T19:00:42.515903+0000","last_unstale":"2024-09-10T19:02:24.324753+0000","last_undegraded":"2024-09-10T19:02:24.324753+0000","last_fullsized":"2024-09-10T19:02:24.324753+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"365'34","last_scrub_stamp":"2024-09-10T19:02:18.712306+0000","last_deep_scrub":"365'34","last_deep_scrub_stamp":"2024-09-10T19:02:18.712306+0000","last_clean_scrub_stamp":"2024-09-10T19:02:18.712306+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:29.349251+0000","scrub_duration":0.017099106999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":33,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":339,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.328349+0000","last_change":"2024-09-10T19:02:08.306579+0000","last_active":"2024-09-10T19:02:24.328349+0000","last_peered":"2024-09-10T19:02:24.328349+0000","last_clean":"2024-09-10T19:02:24.328349+0000","last_became_active":"2024-09-10T18:57:02.461708+0000","last_became_peered":"2024-09-10T18:57:02.461708+0000","last_unstale":"2024-09-10T19:02:24.328349+0000","last_undegraded":"2024-09-10T19:02:24.328349+0000","last_fullsized":"2024-09-10T19:02:24.328349+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:08.306490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:05.603125+0000","last_clean_scrub_stamp":"2024-09-10T19:02:08.306490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:19.815600+0000","scrub_duration":0.0043377579999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"273'10","reported_seq":258,"reported_epoch":368,"state":"active+clean","last_fresh":"2024-09-10T19:02:27.325701+0000","last_change":"2024-09-10T18:59:11.568683+0000","last_active":"2024-09-10T19:02:27.325701+0000","last_peered":"2024-09-10T19:02:27.325701+0000","last_clean":"2024-09-10T19:02:27.325701+0000","last_became_active":"2024-09-10T18:57:02.441890+0000","last_became_peered":"2024-09-10T18:57:02.441890+0000","last_unstale":"2024-09-10T19:02:27.325701+0000","last_undegraded":"2024-09-10T19:02:27.325701+0000","last_fullsized":"2024-09-10T19:02:27.325701+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'10","last_scrub_stamp":"2024-09-10T18:59:11.568599+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-09-10T18:47:02.582229+0000","last_clean_scrub_stamp":"2024-09-10T18:59:11.568599+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0054658950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":390,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.325095+0000","last_change":"2024-09-10T18:59:51.487816+0000","last_active":"2024-09-10T19:02:24.325095+0000","last_peered":"2024-09-10T19:02:24.325095+0000","last_clean":"2024-09-10T19:02:24.325095+0000","last_became_active":"2024-09-10T18:55:40.389202+0000","last_became_peered":"2024-09-10T18:55:40.389202+0000","last_unstale":"2024-09-10T19:02:24.325095+0000","last_undegraded":"2024-09-10T19:02:24.325095+0000","last_fullsized":"2024-09-10T19:02:24.325095+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:51.487763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_clean_scrub_stamp":"2024-09-10T18:59:51.487763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001471904,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"368'15595","reported_seq":29532,"reported_epoch":368,"state":"active+clean","last_fresh":"2024-09-10T19:02:31.542643+0000","last_change":"2024-09-10T19:01:23.521153+0000","last_active":"2024-09-10T19:02:31.542643+0000","last_peered":"2024-09-10T19:02:31.542643+0000","last_clean":"2024-09-10T19:02:31.542643+0000","last_became_active":"2024-09-10T19:01:23.517785+0000","last_became_peered":"2024-09-10T19:01:23.517785+0000","last_unstale":"2024-09-10T19:02:31.542643+0000","last_undegraded":"2024-09-10T19:02:31.542643+0000","last_fullsized":"2024-09-10T19:02:31.542643+0000","mapping_epoch":353,"log_start":"361'12100","ondisk_log_start":"361'12100","created":315,"last_epoch_clean":354,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:41.696116+0000","last_clean_scrub_stamp":"2024-09-10T18:59:41.696116+0000","objects_scrubbed":0,"log_size":3495,"log_dups_size":0,"ondisk_log_size":3495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":116326400,"num_objects":1775,"num_object_clones":0,"num_object_copies":5325,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15595,"num_write_kb":555840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6925,"num_bytes_recovered":453836800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.c","version":"70'81","reported_seq":322,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.329272+0000","last_change":"2024-09-10T18:56:33.779212+0000","last_active":"2024-09-10T19:02:24.329272+0000","last_peered":"2024-09-10T19:02:24.329272+0000","last_clean":"2024-09-10T19:02:24.329272+0000","last_became_active":"2024-09-10T18:55:40.390655+0000","last_became_peered":"2024-09-10T18:55:40.390655+0000","last_unstale":"2024-09-10T19:02:24.329272+0000","last_undegraded":"2024-09-10T19:02:24.329272+0000","last_fullsized":"2024-09-10T19:02:24.329272+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:56:33.779154+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:56:33.779154+0000","last_clean_scrub_stamp":"2024-09-10T18:56:33.779154+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0075121179999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"232'18","reported_seq":410,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.329829+0000","last_change":"2024-09-10T19:01:53.948548+0000","last_active":"2024-09-10T19:02:24.329829+0000","last_peered":"2024-09-10T19:02:24.329829+0000","last_clean":"2024-09-10T19:02:24.329829+0000","last_became_active":"2024-09-10T18:45:20.056869+0000","last_became_peered":"2024-09-10T18:45:20.056869+0000","last_unstale":"2024-09-10T19:02:24.329829+0000","last_undegraded":"2024-09-10T19:02:24.329829+0000","last_fullsized":"2024-09-10T19:02:24.329829+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'18","last_scrub_stamp":"2024-09-10T19:01:53.948309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:01:53.948309+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:00.880791+0000","scrub_duration":0.010806517,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"56'2","reported_seq":420,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.326417+0000","last_change":"2024-09-10T19:02:06.065853+0000","last_active":"2024-09-10T19:02:24.326417+0000","last_peered":"2024-09-10T19:02:24.326417+0000","last_clean":"2024-09-10T19:02:24.326417+0000","last_became_active":"2024-09-10T18:57:02.434475+0000","last_became_peered":"2024-09-10T18:57:02.434475+0000","last_unstale":"2024-09-10T19:02:24.326417+0000","last_undegraded":"2024-09-10T19:02:24.326417+0000","last_fullsized":"2024-09-10T19:02:24.326417+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-10T19:02:06.065768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:06.065768+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:13.722627+0000","scrub_duration":0.0050047989999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":328,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.330346+0000","last_change":"2024-09-10T19:00:42.515054+0000","last_active":"2024-09-10T19:02:24.330346+0000","last_peered":"2024-09-10T19:02:24.330346+0000","last_clean":"2024-09-10T19:02:24.330346+0000","last_became_active":"2024-09-10T19:00:42.513796+0000","last_became_peered":"2024-09-10T19:00:42.513796+0000","last_unstale":"2024-09-10T19:02:24.330346+0000","last_undegraded":"2024-09-10T19:02:24.330346+0000","last_fullsized":"2024-09-10T19:02:24.330346+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:54.899134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:02.711594+0000","last_clean_scrub_stamp":"2024-09-10T18:59:54.899134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051739739999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":372,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.328582+0000","last_change":"2024-09-10T18:59:11.827181+0000","last_active":"2024-09-10T19:02:24.328582+0000","last_peered":"2024-09-10T19:02:24.328582+0000","last_clean":"2024-09-10T19:02:24.328582+0000","last_became_active":"2024-09-10T18:57:02.459985+0000","last_became_peered":"2024-09-10T18:57:02.459985+0000","last_unstale":"2024-09-10T19:02:24.328582+0000","last_undegraded":"2024-09-10T19:02:24.328582+0000","last_fullsized":"2024-09-10T19:02:24.328582+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:11.827101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:53.352427+0000","last_clean_scrub_stamp":"2024-09-10T18:59:11.827101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034557899999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.13","version":"70'81","reported_seq":308,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.330009+0000","last_change":"2024-09-10T19:01:56.385098+0000","last_active":"2024-09-10T19:02:24.330009+0000","last_peered":"2024-09-10T19:02:24.330009+0000","last_clean":"2024-09-10T19:02:24.330009+0000","last_became_active":"2024-09-10T18:52:16.602463+0000","last_became_peered":"2024-09-10T18:52:16.602463+0000","last_unstale":"2024-09-10T19:02:24.330009+0000","last_undegraded":"2024-09-10T19:02:24.330009+0000","last_fullsized":"2024-09-10T19:02:24.330009+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:01:56.384747+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:01:56.384747+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:22.562132+0000","scrub_duration":0.0049177420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"232'19","reported_seq":407,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.322106+0000","last_change":"2024-09-10T19:02:02.627107+0000","last_active":"2024-09-10T19:02:24.322106+0000","last_peered":"2024-09-10T19:02:24.322106+0000","last_clean":"2024-09-10T19:02:24.322106+0000","last_became_active":"2024-09-10T18:57:02.453001+0000","last_became_peered":"2024-09-10T18:57:02.453001+0000","last_unstale":"2024-09-10T19:02:24.322106+0000","last_undegraded":"2024-09-10T19:02:24.322106+0000","last_fullsized":"2024-09-10T19:02:24.322106+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T19:02:02.627031+0000","last_deep_scrub":"76'13","last_deep_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_clean_scrub_stamp":"2024-09-10T19:02:02.627031+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:25.149324+0000","scrub_duration":0.00634981,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":357,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.327541+0000","last_change":"2024-09-10T18:59:59.385405+0000","last_active":"2024-09-10T19:02:24.327541+0000","last_peered":"2024-09-10T19:02:24.327541+0000","last_clean":"2024-09-10T19:02:24.327541+0000","last_became_active":"2024-09-10T18:55:40.401037+0000","last_became_peered":"2024-09-10T18:55:40.401037+0000","last_unstale":"2024-09-10T19:02:24.327541+0000","last_undegraded":"2024-09-10T19:02:24.327541+0000","last_fullsized":"2024-09-10T19:02:24.327541+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:59.385349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:59:59.385349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00163496,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":384,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.327778+0000","last_change":"2024-09-10T19:02:23.261573+0000","last_active":"2024-09-10T19:02:24.327778+0000","last_peered":"2024-09-10T19:02:24.327778+0000","last_clean":"2024-09-10T19:02:24.327778+0000","last_became_active":"2024-09-10T18:57:02.455341+0000","last_became_peered":"2024-09-10T18:57:02.455341+0000","last_unstale":"2024-09-10T19:02:24.327778+0000","last_undegraded":"2024-09-10T19:02:24.327778+0000","last_fullsized":"2024-09-10T19:02:24.327778+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:23.261482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:00.754912+0000","last_clean_scrub_stamp":"2024-09-10T19:02:23.261482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:26.833629+0000","scrub_duration":0.0076325380000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"364'2","reported_seq":407,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.325863+0000","last_change":"2024-09-10T19:02:15.059462+0000","last_active":"2024-09-10T19:02:24.325863+0000","last_peered":"2024-09-10T19:02:24.325863+0000","last_clean":"2024-09-10T19:02:24.325863+0000","last_became_active":"2024-09-10T18:59:48.741275+0000","last_became_peered":"2024-09-10T18:59:48.741275+0000","last_unstale":"2024-09-10T19:02:24.325863+0000","last_undegraded":"2024-09-10T19:02:24.325863+0000","last_fullsized":"2024-09-10T19:02:24.325863+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"364'2","last_scrub_stamp":"2024-09-10T19:02:15.059320+0000","last_deep_scrub":"364'2","last_deep_scrub_stamp":"2024-09-10T19:02:15.059320+0000","last_clean_scrub_stamp":"2024-09-10T19:02:15.059320+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:27.449988+0000","scrub_duration":0.020873968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":52,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.12","version":"70'81","reported_seq":247,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.324535+0000","last_change":"2024-09-10T19:02:05.565674+0000","last_active":"2024-09-10T19:02:24.324535+0000","last_peered":"2024-09-10T19:02:24.324535+0000","last_clean":"2024-09-10T19:02:24.324535+0000","last_became_active":"2024-09-10T18:52:16.598921+0000","last_became_peered":"2024-09-10T18:52:16.598921+0000","last_unstale":"2024-09-10T19:02:24.324535+0000","last_undegraded":"2024-09-10T19:02:24.324535+0000","last_fullsized":"2024-09-10T19:02:24.324535+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:05.565587+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:54.127476+0000","last_clean_scrub_stamp":"2024-09-10T19:02:05.565587+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:11.403928+0000","scrub_duration":0.0051538479999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":413,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.328595+0000","last_change":"2024-09-10T19:02:18.973390+0000","last_active":"2024-09-10T19:02:24.328595+0000","last_peered":"2024-09-10T19:02:24.328595+0000","last_clean":"2024-09-10T19:02:24.328595+0000","last_became_active":"2024-09-10T18:57:02.442251+0000","last_became_peered":"2024-09-10T18:57:02.442251+0000","last_unstale":"2024-09-10T19:02:24.328595+0000","last_undegraded":"2024-09-10T19:02:24.328595+0000","last_fullsized":"2024-09-10T19:02:24.328595+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:18.973341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:18.973341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:33.006119+0000","scrub_duration":0.001038347,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"232'12","reported_seq":405,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.327984+0000","last_change":"2024-09-10T18:59:52.198399+0000","last_active":"2024-09-10T19:02:24.327984+0000","last_peered":"2024-09-10T19:02:24.327984+0000","last_clean":"2024-09-10T19:02:24.327984+0000","last_became_active":"2024-09-10T18:57:02.460430+0000","last_became_peered":"2024-09-10T18:57:02.460430+0000","last_unstale":"2024-09-10T19:02:24.327984+0000","last_undegraded":"2024-09-10T19:02:24.327984+0000","last_fullsized":"2024-09-10T19:02:24.327984+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'12","last_scrub_stamp":"2024-09-10T18:59:52.198340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:59:52.198340+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001790591,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":375,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.324196+0000","last_change":"2024-09-10T19:01:58.529083+0000","last_active":"2024-09-10T19:02:24.324196+0000","last_peered":"2024-09-10T19:02:24.324196+0000","last_clean":"2024-09-10T19:02:24.324196+0000","last_became_active":"2024-09-10T18:45:22.069653+0000","last_became_peered":"2024-09-10T18:45:22.069653+0000","last_unstale":"2024-09-10T19:02:24.324196+0000","last_undegraded":"2024-09-10T19:02:24.324196+0000","last_fullsized":"2024-09-10T19:02:24.324196+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:01:58.528538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:01:58.528538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:24.448033+0000","scrub_duration":0.0057370839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":140,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.328130+0000","last_change":"2024-09-10T19:02:14.037294+0000","last_active":"2024-09-10T19:02:24.328130+0000","last_peered":"2024-09-10T19:02:24.328130+0000","last_clean":"2024-09-10T19:02:24.328130+0000","last_became_active":"2024-09-10T19:00:42.519492+0000","last_became_peered":"2024-09-10T19:00:42.519492+0000","last_unstale":"2024-09-10T19:02:24.328130+0000","last_undegraded":"2024-09-10T19:02:24.328130+0000","last_fullsized":"2024-09-10T19:02:24.328130+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:14.037238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:02:14.037238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:33.201810+0000","scrub_duration":0.001210221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.11","version":"70'81","reported_seq":282,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.323787+0000","last_change":"2024-09-10T19:02:17.778329+0000","last_active":"2024-09-10T19:02:24.323787+0000","last_peered":"2024-09-10T19:02:24.323787+0000","last_clean":"2024-09-10T19:02:24.323787+0000","last_became_active":"2024-09-10T19:00:42.517311+0000","last_became_peered":"2024-09-10T19:00:42.517311+0000","last_unstale":"2024-09-10T19:02:24.323787+0000","last_undegraded":"2024-09-10T19:02:24.323787+0000","last_fullsized":"2024-09-10T19:02:24.323787+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:17.778227+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:02:17.778227+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:21.466825+0000","scrub_duration":0.0070248109999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"232'20","reported_seq":462,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.328874+0000","last_change":"2024-09-10T19:00:07.147063+0000","last_active":"2024-09-10T19:02:24.328874+0000","last_peered":"2024-09-10T19:02:24.328874+0000","last_clean":"2024-09-10T19:02:24.328874+0000","last_became_active":"2024-09-10T18:57:02.444437+0000","last_became_peered":"2024-09-10T18:57:02.444437+0000","last_unstale":"2024-09-10T19:02:24.328874+0000","last_undegraded":"2024-09-10T19:02:24.328874+0000","last_fullsized":"2024-09-10T19:02:24.328874+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'20","last_scrub_stamp":"2024-09-10T19:00:07.146928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:00:07.146928+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0071404040000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":399,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.321873+0000","last_change":"2024-09-10T19:02:20.755424+0000","last_active":"2024-09-10T19:02:24.321873+0000","last_peered":"2024-09-10T19:02:24.321873+0000","last_clean":"2024-09-10T19:02:24.321873+0000","last_became_active":"2024-09-10T18:57:02.445710+0000","last_became_peered":"2024-09-10T18:57:02.445710+0000","last_unstale":"2024-09-10T19:02:24.321873+0000","last_undegraded":"2024-09-10T19:02:24.321873+0000","last_fullsized":"2024-09-10T19:02:24.321873+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:20.755253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:20.755253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:37.147814+0000","scrub_duration":0.0023404490000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"304'11","reported_seq":624,"reported_epoch":368,"state":"active+clean","last_fresh":"2024-09-10T19:02:27.323481+0000","last_change":"2024-09-10T18:59:55.525278+0000","last_active":"2024-09-10T19:02:27.323481+0000","last_peered":"2024-09-10T19:02:27.323481+0000","last_clean":"2024-09-10T19:02:27.323481+0000","last_became_active":"2024-09-10T18:58:58.489370+0000","last_became_peered":"2024-09-10T18:58:58.489370+0000","last_unstale":"2024-09-10T19:02:27.323481+0000","last_undegraded":"2024-09-10T19:02:27.323481+0000","last_fullsized":"2024-09-10T19:02:27.323481+0000","mapping_epoch":303,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":304,"parent":"0.0","parent_split_bits":0,"last_scrub":"304'11","last_scrub_stamp":"2024-09-10T18:59:55.525185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:59:55.525185+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020778110000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":67,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.321676+0000","last_change":"2024-09-10T19:02:22.928743+0000","last_active":"2024-09-10T19:02:24.321676+0000","last_peered":"2024-09-10T19:02:24.321676+0000","last_clean":"2024-09-10T19:02:24.321676+0000","last_became_active":"2024-09-10T18:59:48.744923+0000","last_became_peered":"2024-09-10T18:59:48.744923+0000","last_unstale":"2024-09-10T19:02:24.321676+0000","last_undegraded":"2024-09-10T19:02:24.321676+0000","last_fullsized":"2024-09-10T19:02:24.321676+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:22.928652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:57.476724+0000","last_clean_scrub_stamp":"2024-09-10T19:02:22.928652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:49.708465+0000","scrub_duration":0.013720313,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.10","version":"70'81","reported_seq":247,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.324848+0000","last_change":"2024-09-10T19:00:04.612412+0000","last_active":"2024-09-10T19:02:24.324848+0000","last_peered":"2024-09-10T19:02:24.324848+0000","last_clean":"2024-09-10T19:02:24.324848+0000","last_became_active":"2024-09-10T18:57:02.443782+0000","last_became_peered":"2024-09-10T18:57:02.443782+0000","last_unstale":"2024-09-10T19:02:24.324848+0000","last_undegraded":"2024-09-10T19:02:24.324848+0000","last_fullsized":"2024-09-10T19:02:24.324848+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:00:04.612324+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:00:04.612324+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025961159999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"232'15","reported_seq":49,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.324235+0000","last_change":"2024-09-10T19:00:42.530883+0000","last_active":"2024-09-10T19:02:24.324235+0000","last_peered":"2024-09-10T19:02:24.324235+0000","last_clean":"2024-09-10T19:02:24.324235+0000","last_became_active":"2024-09-10T19:00:42.518672+0000","last_became_peered":"2024-09-10T19:00:42.518672+0000","last_unstale":"2024-09-10T19:02:24.324235+0000","last_undegraded":"2024-09-10T19:02:24.324235+0000","last_fullsized":"2024-09-10T19:02:24.324235+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T18:59:16.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:59:16.949954+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":356,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.324397+0000","last_change":"2024-09-10T19:02:03.587108+0000","last_active":"2024-09-10T19:02:24.324397+0000","last_peered":"2024-09-10T19:02:24.324397+0000","last_clean":"2024-09-10T19:02:24.324397+0000","last_became_active":"2024-09-10T18:55:40.390697+0000","last_became_peered":"2024-09-10T18:55:40.390697+0000","last_unstale":"2024-09-10T19:02:24.324397+0000","last_undegraded":"2024-09-10T19:02:24.324397+0000","last_fullsized":"2024-09-10T19:02:24.324397+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:03.586986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:03.586986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:23.618455+0000","scrub_duration":0.0079434420000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"273'11","reported_seq":607,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:26.277989+0000","last_change":"2024-09-10T18:59:53.158730+0000","last_active":"2024-09-10T19:02:26.277989+0000","last_peered":"2024-09-10T19:02:26.277989+0000","last_clean":"2024-09-10T19:02:26.277989+0000","last_became_active":"2024-09-10T18:57:02.445268+0000","last_became_peered":"2024-09-10T18:57:02.445268+0000","last_unstale":"2024-09-10T19:02:26.277989+0000","last_undegraded":"2024-09-10T19:02:26.277989+0000","last_fullsized":"2024-09-10T19:02:26.277989+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'11","last_scrub_stamp":"2024-09-10T18:59:53.158664+0000","last_deep_scrub":"81'9","last_deep_scrub_stamp":"2024-09-10T18:46:54.800790+0000","last_clean_scrub_stamp":"2024-09-10T18:59:53.158664+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017427930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":388,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.326065+0000","last_change":"2024-09-10T18:56:20.584899+0000","last_active":"2024-09-10T19:02:24.326065+0000","last_peered":"2024-09-10T19:02:24.326065+0000","last_clean":"2024-09-10T19:02:24.326065+0000","last_became_active":"2024-09-10T18:55:40.391662+0000","last_became_peered":"2024-09-10T18:55:40.391662+0000","last_unstale":"2024-09-10T19:02:24.326065+0000","last_undegraded":"2024-09-10T19:02:24.326065+0000","last_fullsized":"2024-09-10T19:02:24.326065+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:56:20.584808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_clean_scrub_stamp":"2024-09-10T18:56:20.584808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0035961249999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"232'15","reported_seq":441,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.327617+0000","last_change":"2024-09-10T18:59:53.320831+0000","last_active":"2024-09-10T19:02:24.327617+0000","last_peered":"2024-09-10T19:02:24.327617+0000","last_clean":"2024-09-10T19:02:24.327617+0000","last_became_active":"2024-09-10T18:55:40.400788+0000","last_became_peered":"2024-09-10T18:55:40.400788+0000","last_unstale":"2024-09-10T19:02:24.327617+0000","last_undegraded":"2024-09-10T19:02:24.327617+0000","last_fullsized":"2024-09-10T19:02:24.327617+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T18:59:53.320757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:59:53.320757+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001889578,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":420,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.328516+0000","last_change":"2024-09-10T19:00:42.518606+0000","last_active":"2024-09-10T19:02:24.328516+0000","last_peered":"2024-09-10T19:02:24.328516+0000","last_clean":"2024-09-10T19:02:24.328516+0000","last_became_active":"2024-09-10T19:00:42.518310+0000","last_became_peered":"2024-09-10T19:00:42.518310+0000","last_unstale":"2024-09-10T19:02:24.328516+0000","last_undegraded":"2024-09-10T19:02:24.328516+0000","last_fullsized":"2024-09-10T19:02:24.328516+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:48.622701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:59:48.622701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001531279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":381,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.328777+0000","last_change":"2024-09-10T19:02:22.258152+0000","last_active":"2024-09-10T19:02:24.328777+0000","last_peered":"2024-09-10T19:02:24.328777+0000","last_clean":"2024-09-10T19:02:24.328777+0000","last_became_active":"2024-09-10T18:55:40.398243+0000","last_became_peered":"2024-09-10T18:55:40.398243+0000","last_unstale":"2024-09-10T19:02:24.328777+0000","last_undegraded":"2024-09-10T19:02:24.328777+0000","last_fullsized":"2024-09-10T19:02:24.328777+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:22.258058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:03.571786+0000","last_clean_scrub_stamp":"2024-09-10T19:02:22.258058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:37.095041+0000","scrub_duration":0.0054525670000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":361,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.330517+0000","last_change":"2024-09-10T19:00:42.517459+0000","last_active":"2024-09-10T19:02:24.330517+0000","last_peered":"2024-09-10T19:02:24.330517+0000","last_clean":"2024-09-10T19:02:24.330517+0000","last_became_active":"2024-09-10T19:00:42.516833+0000","last_became_peered":"2024-09-10T19:00:42.516833+0000","last_unstale":"2024-09-10T19:02:24.330517+0000","last_undegraded":"2024-09-10T19:02:24.330517+0000","last_fullsized":"2024-09-10T19:02:24.330517+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:46.863758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:04.631558+0000","last_clean_scrub_stamp":"2024-09-10T18:59:46.863758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0031907379999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"232'24","reported_seq":401,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.325792+0000","last_change":"2024-09-10T18:58:58.698097+0000","last_active":"2024-09-10T19:02:24.325792+0000","last_peered":"2024-09-10T19:02:24.325792+0000","last_clean":"2024-09-10T19:02:24.325792+0000","last_became_active":"2024-09-10T18:45:20.056885+0000","last_became_peered":"2024-09-10T18:45:20.056885+0000","last_unstale":"2024-09-10T19:02:24.325792+0000","last_undegraded":"2024-09-10T19:02:24.325792+0000","last_fullsized":"2024-09-10T19:02:24.325792+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T18:58:58.698003+0000","last_deep_scrub":"223'16","last_deep_scrub_stamp":"2024-09-10T18:54:47.324044+0000","last_clean_scrub_stamp":"2024-09-10T18:58:58.698003+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.006955676,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":399,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.329238+0000","last_change":"2024-09-10T19:02:19.301370+0000","last_active":"2024-09-10T19:02:24.329238+0000","last_peered":"2024-09-10T19:02:24.329238+0000","last_clean":"2024-09-10T19:02:24.329238+0000","last_became_active":"2024-09-10T18:55:40.399198+0000","last_became_peered":"2024-09-10T18:55:40.399198+0000","last_unstale":"2024-09-10T19:02:24.329238+0000","last_undegraded":"2024-09-10T19:02:24.329238+0000","last_fullsized":"2024-09-10T19:02:24.329238+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:19.301278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:00.029389+0000","last_clean_scrub_stamp":"2024-09-10T19:02:19.301278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:30.235302+0000","scrub_duration":0.0054684379999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":381,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.327467+0000","last_change":"2024-09-10T19:02:15.321859+0000","last_active":"2024-09-10T19:02:24.327467+0000","last_peered":"2024-09-10T19:02:24.327467+0000","last_clean":"2024-09-10T19:02:24.327467+0000","last_became_active":"2024-09-10T18:45:22.070065+0000","last_became_peered":"2024-09-10T18:45:22.070065+0000","last_unstale":"2024-09-10T19:02:24.327467+0000","last_undegraded":"2024-09-10T19:02:24.327467+0000","last_fullsized":"2024-09-10T19:02:24.327467+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:15.321767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:02:15.321767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:41.137313+0000","scrub_duration":0.010698602999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":379,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.325559+0000","last_change":"2024-09-10T19:02:00.273929+0000","last_active":"2024-09-10T19:02:24.325559+0000","last_peered":"2024-09-10T19:02:24.325559+0000","last_clean":"2024-09-10T19:02:24.325559+0000","last_became_active":"2024-09-10T18:55:40.395595+0000","last_became_peered":"2024-09-10T18:55:40.395595+0000","last_unstale":"2024-09-10T19:02:24.325559+0000","last_undegraded":"2024-09-10T19:02:24.325559+0000","last_fullsized":"2024-09-10T19:02:24.325559+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:00.273863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:02:00.273863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:17.580651+0000","scrub_duration":0.002006552,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"232'8","reported_seq":378,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.324914+0000","last_change":"2024-09-10T18:56:30.802774+0000","last_active":"2024-09-10T19:02:24.324914+0000","last_peered":"2024-09-10T19:02:24.324914+0000","last_clean":"2024-09-10T19:02:24.324914+0000","last_became_active":"2024-09-10T18:45:20.059205+0000","last_became_peered":"2024-09-10T18:45:20.059205+0000","last_unstale":"2024-09-10T19:02:24.324914+0000","last_undegraded":"2024-09-10T19:02:24.324914+0000","last_fullsized":"2024-09-10T19:02:24.324914+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'8","last_scrub_stamp":"2024-09-10T18:56:30.802679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:56:30.802679+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034579609999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":395,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.324168+0000","last_change":"2024-09-10T19:02:16.704889+0000","last_active":"2024-09-10T19:02:24.324168+0000","last_peered":"2024-09-10T19:02:24.324168+0000","last_clean":"2024-09-10T19:02:24.324168+0000","last_became_active":"2024-09-10T18:47:36.415866+0000","last_became_peered":"2024-09-10T18:47:36.415866+0000","last_unstale":"2024-09-10T19:02:24.324168+0000","last_undegraded":"2024-09-10T19:02:24.324168+0000","last_fullsized":"2024-09-10T19:02:24.324168+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:16.704805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:16.704805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:44.249699+0000","scrub_duration":0.0083395729999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":406,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.327832+0000","last_change":"2024-09-10T19:02:08.092456+0000","last_active":"2024-09-10T19:02:24.327832+0000","last_peered":"2024-09-10T19:02:24.327832+0000","last_clean":"2024-09-10T19:02:24.327832+0000","last_became_active":"2024-09-10T18:57:02.453974+0000","last_became_peered":"2024-09-10T18:57:02.453974+0000","last_unstale":"2024-09-10T19:02:24.327832+0000","last_undegraded":"2024-09-10T19:02:24.327832+0000","last_fullsized":"2024-09-10T19:02:24.327832+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:08.092366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_clean_scrub_stamp":"2024-09-10T19:02:08.092366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:34.680292+0000","scrub_duration":0.0058035439999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":166,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.325453+0000","last_change":"2024-09-10T18:59:55.812370+0000","last_active":"2024-09-10T19:02:24.325453+0000","last_peered":"2024-09-10T19:02:24.325453+0000","last_clean":"2024-09-10T19:02:24.325453+0000","last_became_active":"2024-09-10T18:57:02.447566+0000","last_became_peered":"2024-09-10T18:57:02.447566+0000","last_unstale":"2024-09-10T19:02:24.325453+0000","last_undegraded":"2024-09-10T19:02:24.325453+0000","last_fullsized":"2024-09-10T19:02:24.325453+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:55.812312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:55.145638+0000","last_clean_scrub_stamp":"2024-09-10T18:59:55.812312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0055522669999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.14","version":"70'81","reported_seq":280,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.326212+0000","last_change":"2024-09-10T19:02:03.086382+0000","last_active":"2024-09-10T19:02:24.326212+0000","last_peered":"2024-09-10T19:02:24.326212+0000","last_clean":"2024-09-10T19:02:24.326212+0000","last_became_active":"2024-09-10T18:57:02.442457+0000","last_became_peered":"2024-09-10T18:57:02.442457+0000","last_unstale":"2024-09-10T19:02:24.326212+0000","last_undegraded":"2024-09-10T19:02:24.326212+0000","last_fullsized":"2024-09-10T19:02:24.326212+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:03.086329+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:57.184383+0000","last_clean_scrub_stamp":"2024-09-10T19:02:03.086329+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:19.694556+0000","scrub_duration":0.0026855270000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.11","version":"232'24","reported_seq":424,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.325316+0000","last_change":"2024-09-10T18:59:10.879686+0000","last_active":"2024-09-10T19:02:24.325316+0000","last_peered":"2024-09-10T19:02:24.325316+0000","last_clean":"2024-09-10T19:02:24.325316+0000","last_became_active":"2024-09-10T18:57:02.461909+0000","last_became_peered":"2024-09-10T18:57:02.461909+0000","last_unstale":"2024-09-10T19:02:24.325316+0000","last_undegraded":"2024-09-10T19:02:24.325316+0000","last_fullsized":"2024-09-10T19:02:24.325316+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T18:59:10.879597+0000","last_deep_scrub":"79'16","last_deep_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_clean_scrub_stamp":"2024-09-10T18:59:10.879597+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0090530790000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"56'1","reported_seq":411,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.329251+0000","last_change":"2024-09-10T19:01:58.121943+0000","last_active":"2024-09-10T19:02:24.329251+0000","last_peered":"2024-09-10T19:02:24.329251+0000","last_clean":"2024-09-10T19:02:24.329251+0000","last_became_active":"2024-09-10T18:47:36.421985+0000","last_became_peered":"2024-09-10T18:47:36.421985+0000","last_unstale":"2024-09-10T19:02:24.329251+0000","last_undegraded":"2024-09-10T19:02:24.329251+0000","last_fullsized":"2024-09-10T19:02:24.329251+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T19:01:58.121852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:01:58.121852+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:23.790654+0000","scrub_duration":0.002125984,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":123,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.328080+0000","last_change":"2024-09-10T19:02:14.088121+0000","last_active":"2024-09-10T19:02:24.328080+0000","last_peered":"2024-09-10T19:02:24.328080+0000","last_clean":"2024-09-10T19:02:24.328080+0000","last_became_active":"2024-09-10T18:57:02.435124+0000","last_became_peered":"2024-09-10T18:57:02.435124+0000","last_unstale":"2024-09-10T19:02:24.328080+0000","last_undegraded":"2024-09-10T19:02:24.328080+0000","last_fullsized":"2024-09-10T19:02:24.328080+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:14.088018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:02:14.088018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:15.045325+0000","scrub_duration":0.031802916000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":389,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.328503+0000","last_change":"2024-09-10T19:02:16.321814+0000","last_active":"2024-09-10T19:02:24.328503+0000","last_peered":"2024-09-10T19:02:24.328503+0000","last_clean":"2024-09-10T19:02:24.328503+0000","last_became_active":"2024-09-10T18:55:40.397523+0000","last_became_peered":"2024-09-10T18:55:40.397523+0000","last_unstale":"2024-09-10T19:02:24.328503+0000","last_undegraded":"2024-09-10T19:02:24.328503+0000","last_fullsized":"2024-09-10T19:02:24.328503+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:16.321715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:02:16.321715+0000","last_clean_scrub_stamp":"2024-09-10T19:02:16.321715+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:19.129657+0000","scrub_duration":0.0069778510000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"78'2","reported_seq":166,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.324254+0000","last_change":"2024-09-10T19:02:15.686223+0000","last_active":"2024-09-10T19:02:24.324254+0000","last_peered":"2024-09-10T19:02:24.324254+0000","last_clean":"2024-09-10T19:02:24.324254+0000","last_became_active":"2024-09-10T18:57:02.447198+0000","last_became_peered":"2024-09-10T18:57:02.447198+0000","last_unstale":"2024-09-10T19:02:24.324254+0000","last_undegraded":"2024-09-10T19:02:24.324254+0000","last_fullsized":"2024-09-10T19:02:24.324254+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'2","last_scrub_stamp":"2024-09-10T19:02:15.686138+0000","last_deep_scrub":"78'2","last_deep_scrub_stamp":"2024-09-10T19:02:15.686138+0000","last_clean_scrub_stamp":"2024-09-10T19:02:15.686138+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:16.977621+0000","scrub_duration":0.011160388,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":389,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.324988+0000","last_change":"2024-09-10T18:59:53.505658+0000","last_active":"2024-09-10T19:02:24.324988+0000","last_peered":"2024-09-10T19:02:24.324988+0000","last_clean":"2024-09-10T19:02:24.324988+0000","last_became_active":"2024-09-10T18:57:02.443353+0000","last_became_peered":"2024-09-10T18:57:02.443353+0000","last_unstale":"2024-09-10T19:02:24.324988+0000","last_undegraded":"2024-09-10T19:02:24.324988+0000","last_fullsized":"2024-09-10T19:02:24.324988+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:53.505561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:59:53.505561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001238605,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"232'20","reported_seq":405,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.321983+0000","last_change":"2024-09-10T19:02:04.609420+0000","last_active":"2024-09-10T19:02:24.321983+0000","last_peered":"2024-09-10T19:02:24.321983+0000","last_clean":"2024-09-10T19:02:24.321983+0000","last_became_active":"2024-09-10T18:57:02.452473+0000","last_became_peered":"2024-09-10T18:57:02.452473+0000","last_unstale":"2024-09-10T19:02:24.321983+0000","last_undegraded":"2024-09-10T19:02:24.321983+0000","last_fullsized":"2024-09-10T19:02:24.321983+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'20","last_scrub_stamp":"2024-09-10T19:02:04.609332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:04.609332+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:09.802099+0000","scrub_duration":0.0075275430000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":23,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"273'10","reported_seq":619,"reported_epoch":368,"state":"active+clean","last_fresh":"2024-09-10T19:02:27.328243+0000","last_change":"2024-09-10T18:59:22.516714+0000","last_active":"2024-09-10T19:02:27.328243+0000","last_peered":"2024-09-10T19:02:27.328243+0000","last_clean":"2024-09-10T19:02:27.328243+0000","last_became_active":"2024-09-10T18:57:02.441763+0000","last_became_peered":"2024-09-10T18:57:02.441763+0000","last_unstale":"2024-09-10T19:02:27.328243+0000","last_undegraded":"2024-09-10T19:02:27.328243+0000","last_fullsized":"2024-09-10T19:02:27.328243+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'10","last_scrub_stamp":"2024-09-10T18:59:22.516625+0000","last_deep_scrub":"190'9","last_deep_scrub_stamp":"2024-09-10T18:53:07.746309+0000","last_clean_scrub_stamp":"2024-09-10T18:59:22.516625+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021912820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":376,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.328783+0000","last_change":"2024-09-10T19:02:23.327859+0000","last_active":"2024-09-10T19:02:24.328783+0000","last_peered":"2024-09-10T19:02:24.328783+0000","last_clean":"2024-09-10T19:02:24.328783+0000","last_became_active":"2024-09-10T18:45:24.082318+0000","last_became_peered":"2024-09-10T18:45:24.082318+0000","last_unstale":"2024-09-10T19:02:24.328783+0000","last_undegraded":"2024-09-10T19:02:24.328783+0000","last_fullsized":"2024-09-10T19:02:24.328783+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:23.327767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:03.475299+0000","last_clean_scrub_stamp":"2024-09-10T19:02:23.327767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:31.863753+0000","scrub_duration":0.010585020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":385,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.324868+0000","last_change":"2024-09-10T19:00:10.182129+0000","last_active":"2024-09-10T19:02:24.324868+0000","last_peered":"2024-09-10T19:02:24.324868+0000","last_clean":"2024-09-10T19:02:24.324868+0000","last_became_active":"2024-09-10T18:55:40.392524+0000","last_became_peered":"2024-09-10T18:55:40.392524+0000","last_unstale":"2024-09-10T19:02:24.324868+0000","last_undegraded":"2024-09-10T19:02:24.324868+0000","last_fullsized":"2024-09-10T19:02:24.324868+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:10.181982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:00:10.181982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029510420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"232'19","reported_seq":422,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.329279+0000","last_change":"2024-09-10T19:02:12.120953+0000","last_active":"2024-09-10T19:02:24.329279+0000","last_peered":"2024-09-10T19:02:24.329279+0000","last_clean":"2024-09-10T19:02:24.329279+0000","last_became_active":"2024-09-10T18:45:20.058717+0000","last_became_peered":"2024-09-10T18:45:20.058717+0000","last_unstale":"2024-09-10T19:02:24.329279+0000","last_undegraded":"2024-09-10T19:02:24.329279+0000","last_fullsized":"2024-09-10T19:02:24.329279+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T19:02:12.120860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:12.120860+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:15.679651+0000","scrub_duration":0.0060875590000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":349,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.322177+0000","last_change":"2024-09-10T18:59:57.862722+0000","last_active":"2024-09-10T19:02:24.322177+0000","last_peered":"2024-09-10T19:02:24.322177+0000","last_clean":"2024-09-10T19:02:24.322177+0000","last_became_active":"2024-09-10T18:57:02.453464+0000","last_became_peered":"2024-09-10T18:57:02.453464+0000","last_unstale":"2024-09-10T19:02:24.322177+0000","last_undegraded":"2024-09-10T19:02:24.322177+0000","last_fullsized":"2024-09-10T19:02:24.322177+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:57.862666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:59:57.862666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001262242,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":361,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.326490+0000","last_change":"2024-09-10T18:58:55.726863+0000","last_active":"2024-09-10T19:02:24.326490+0000","last_peered":"2024-09-10T19:02:24.326490+0000","last_clean":"2024-09-10T19:02:24.326490+0000","last_became_active":"2024-09-10T18:45:24.267251+0000","last_became_peered":"2024-09-10T18:45:24.267251+0000","last_unstale":"2024-09-10T19:02:24.326490+0000","last_undegraded":"2024-09-10T19:02:24.326490+0000","last_fullsized":"2024-09-10T19:02:24.326490+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:55.726799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:58:55.726799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001154721,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"306'2","reported_seq":407,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.328458+0000","last_change":"2024-09-10T18:59:20.534927+0000","last_active":"2024-09-10T19:02:24.328458+0000","last_peered":"2024-09-10T19:02:24.328458+0000","last_clean":"2024-09-10T19:02:24.328458+0000","last_became_active":"2024-09-10T18:57:02.435758+0000","last_became_peered":"2024-09-10T18:57:02.435758+0000","last_unstale":"2024-09-10T19:02:24.328458+0000","last_undegraded":"2024-09-10T19:02:24.328458+0000","last_fullsized":"2024-09-10T19:02:24.328458+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"306'2","last_scrub_stamp":"2024-09-10T18:59:20.534869+0000","last_deep_scrub":"306'2","last_deep_scrub_stamp":"2024-09-10T18:59:20.534869+0000","last_clean_scrub_stamp":"2024-09-10T18:59:20.534869+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0072446200000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"258'34","reported_seq":495,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.329352+0000","last_change":"2024-09-10T19:02:02.373473+0000","last_active":"2024-09-10T19:02:24.329352+0000","last_peered":"2024-09-10T19:02:24.329352+0000","last_clean":"2024-09-10T19:02:24.329352+0000","last_became_active":"2024-09-10T18:59:48.744293+0000","last_became_peered":"2024-09-10T18:59:48.744293+0000","last_unstale":"2024-09-10T19:02:24.329352+0000","last_undegraded":"2024-09-10T19:02:24.329352+0000","last_fullsized":"2024-09-10T19:02:24.329352+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'34","last_scrub_stamp":"2024-09-10T19:02:02.373321+0000","last_deep_scrub":"258'34","last_deep_scrub_stamp":"2024-09-10T18:56:23.838951+0000","last_clean_scrub_stamp":"2024-09-10T19:02:02.373321+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:25.099818+0000","scrub_duration":0.010429884,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":385,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.328981+0000","last_change":"2024-09-10T19:02:03.343447+0000","last_active":"2024-09-10T19:02:24.328981+0000","last_peered":"2024-09-10T19:02:24.328981+0000","last_clean":"2024-09-10T19:02:24.328981+0000","last_became_active":"2024-09-10T18:55:40.397870+0000","last_became_peered":"2024-09-10T18:55:40.397870+0000","last_unstale":"2024-09-10T19:02:24.328981+0000","last_undegraded":"2024-09-10T19:02:24.328981+0000","last_fullsized":"2024-09-10T19:02:24.328981+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:03.343366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:02:03.343366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:17.971785+0000","scrub_duration":0.025125941999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":382,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.327590+0000","last_change":"2024-09-10T19:00:42.520936+0000","last_active":"2024-09-10T19:02:24.327590+0000","last_peered":"2024-09-10T19:02:24.327590+0000","last_clean":"2024-09-10T19:02:24.327590+0000","last_became_active":"2024-09-10T19:00:42.520239+0000","last_became_peered":"2024-09-10T19:00:42.520239+0000","last_unstale":"2024-09-10T19:02:24.327590+0000","last_undegraded":"2024-09-10T19:02:24.327590+0000","last_fullsized":"2024-09-10T19:02:24.327590+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:17.985545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:17.985545+0000","last_clean_scrub_stamp":"2024-09-10T18:59:17.985545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0086180830000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":388,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.329030+0000","last_change":"2024-09-10T18:58:57.919890+0000","last_active":"2024-09-10T19:02:24.329030+0000","last_peered":"2024-09-10T19:02:24.329030+0000","last_clean":"2024-09-10T19:02:24.329030+0000","last_became_active":"2024-09-10T18:57:02.460855+0000","last_became_peered":"2024-09-10T18:57:02.460855+0000","last_unstale":"2024-09-10T19:02:24.329030+0000","last_undegraded":"2024-09-10T19:02:24.329030+0000","last_fullsized":"2024-09-10T19:02:24.329030+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:57.919515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:58:57.919515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.01355234,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"232'18","reported_seq":398,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.328302+0000","last_change":"2024-09-10T18:58:52.890712+0000","last_active":"2024-09-10T19:02:24.328302+0000","last_peered":"2024-09-10T19:02:24.328302+0000","last_clean":"2024-09-10T19:02:24.328302+0000","last_became_active":"2024-09-10T18:45:20.058391+0000","last_became_peered":"2024-09-10T18:45:20.058391+0000","last_unstale":"2024-09-10T19:02:24.328302+0000","last_undegraded":"2024-09-10T19:02:24.328302+0000","last_fullsized":"2024-09-10T19:02:24.328302+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'18","last_scrub_stamp":"2024-09-10T18:58:52.890618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:58:52.890618+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016012082,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":393,"reported_epoch":367,"state":"active+clean","last_fresh":"2024-09-10T19:02:24.325935+0000","last_change":"2024-09-10T18:58:56.690660+0000","last_active":"2024-09-10T19:02:24.325935+0000","last_peered":"2024-09-10T19:02:24.325935+0000","last_clean":"2024-09-10T19:02:24.325935+0000","last_became_active":"2024-09-10T18:57:02.449135+0000","last_became_peered":"2024-09-10T18:57:02.449135+0000","last_unstale":"2024-09-10T19:02:24.325935+0000","last_undegraded":"2024-09-10T19:02:24.325935+0000","last_fullsized":"2024-09-10T19:02:24.325935+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:56.690606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:58:56.690606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060631599999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":1938817048,"num_objects":29585,"num_object_clones":0,"num_object_copies":88755,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29585,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":247727,"num_write_kb":8873921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21975,"num_bytes_recovered":1440088064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":7155757056,"data_stored":7155744840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57146,"ondisk_log_size":57146,"up":48,"acting":48,"num_store_stats":6},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":421,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1118,"num_read_kb":875,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":3372,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":499712,"data_stored":13268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":765,"ondisk_log_size":765,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1017,"num_read_kb":1012,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":262688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1701,"ondisk_log_size":1701,"up":63,"acting":63,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":0,"seq":223338299598,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":43,"seq":184683593944,"num_pgs":86,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1744760,"kb_used_data":846384,"kb_used_omap":1,"kb_used_meta":898302,"kb_avail":92000392,"statfs":{"total":95995035648,"available":94208401408,"internally_reserved":0,"allocated":866697216,"data_stored":866286727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":919861592},"hb_peers":[0,1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:01:57 2024","interfaces":[{"interface":"back","average":{"1min":3.2309999999999999,"5min":1.651,"15min":1.145},"min":{"1min":0.23799999999999999,"5min":0.23000000000000001,"15min":0.20899999999999999},"max":{"1min":49.353999999999999,"5min":49.353999999999999,"15min":49.353999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":3.4140000000000001,"5min":1.8089999999999999,"15min":1.2290000000000001},"min":{"1min":0.30299999999999999,"5min":0.219,"15min":0.219},"max":{"1min":49.341999999999999,"5min":49.341999999999999,"15min":49.341999999999999},"last":0.66200000000000003}]},{"osd":1,"last update":"Tue Sep 10 19:02:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.702,"15min":1.335},"min":{"1min":0.39500000000000002,"5min":0.22900000000000001,"15min":0.219},"max":{"1min":4.0759999999999996,"5min":45.311,"15min":45.311},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.94799999999999995,"5min":1.6299999999999999,"15min":1.2110000000000001},"min":{"1min":0.318,"5min":0.25900000000000001,"15min":0.214},"max":{"1min":2.532,"5min":45.027999999999999,"15min":45.027999999999999},"last":1.349}]},{"osd":2,"last update":"Tue Sep 10 19:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.71699999999999997,"15min":0.81399999999999995},"min":{"1min":0.28499999999999998,"5min":0.27200000000000002,"15min":0.27000000000000002},"max":{"1min":2.306,"5min":3.4009999999999998,"15min":4.4740000000000002},"last":0.70699999999999996},{"interface":"front","average":{"1min":1.0880000000000001,"5min":0.70999999999999996,"15min":0.75800000000000001},"min":{"1min":0.30399999999999999,"5min":0.249,"15min":0.249},"max":{"1min":4.0380000000000003,"5min":4.6580000000000004,"15min":4.8150000000000004},"last":0.60599999999999998}]},{"osd":3,"last update":"Tue Sep 10 19:02:07 2024","interfaces":[{"interface":"back","average":{"1min":3.0800000000000001,"5min":2.617,"15min":1.5960000000000001},"min":{"1min":0.34999999999999998,"5min":0.29599999999999999,"15min":0.25700000000000001},"max":{"1min":47.325000000000003,"5min":76.977999999999994,"15min":76.977999999999994},"last":1.2130000000000001},{"interface":"front","average":{"1min":3.1070000000000002,"5min":2.5870000000000002,"15min":1.694},"min":{"1min":0.31,"5min":0.27500000000000002,"15min":0.253},"max":{"1min":46.793999999999997,"5min":76.879999999999995,"15min":76.879999999999995},"last":1.403}]},{"osd":4,"last update":"Tue Sep 10 19:02:07 2024","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":0.79700000000000004,"15min":1.387},"min":{"1min":0.24299999999999999,"5min":0.23400000000000001,"15min":0.20999999999999999},"max":{"1min":4.1219999999999999,"5min":4.7220000000000004,"15min":165.03299999999999},"last":0.504},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.67300000000000004,"15min":1.2749999999999999},"min":{"1min":0.29299999999999998,"5min":0.26000000000000001,"15min":0.22500000000000001},"max":{"1min":3.827,"5min":3.827,"15min":163.959},"last":0.52100000000000002}]},{"osd":5,"last update":"Tue Sep 10 19:02:07 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":3.1219999999999999,"15min":1.8140000000000001},"min":{"1min":0.318,"5min":0.253,"15min":0.24099999999999999},"max":{"1min":5.0670000000000002,"5min":288.56999999999999,"15min":288.56999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":1.2569999999999999,"5min":3.0779999999999998,"15min":1.806},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":4.1449999999999996,"5min":291.05399999999997,"15min":291.05399999999997},"last":0.72799999999999998}]},{"osd":7,"last update":"Tue Sep 10 19:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.67800000000000005,"15min":0.83499999999999996},"min":{"1min":0.32200000000000001,"5min":0.27900000000000003,"15min":0.26200000000000001},"max":{"1min":2.2839999999999998,"5min":2.98,"15min":5.3680000000000003},"last":0.5},{"interface":"front","average":{"1min":1.1379999999999999,"5min":0.72499999999999998,"15min":0.88900000000000001},"min":{"1min":0.31,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":4.6820000000000004,"5min":4.6820000000000004,"15min":7.0350000000000001},"last":0.39500000000000002}]}]},{"osd":5,"up_from":37,"seq":158913790172,"num_pgs":84,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1849268,"kb_used_data":957292,"kb_used_omap":1,"kb_used_meta":891966,"kb_avail":91895884,"statfs":{"total":95995035648,"available":94101385216,"internally_reserved":0,"allocated":980267008,"data_stored":979859620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":913373528},"hb_peers":[0,1,2,3,4,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,3,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:01:50 2024","interfaces":[{"interface":"back","average":{"1min":2.0720000000000001,"5min":2.8580000000000001,"15min":1.9850000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":12.401,"5min":82.465000000000003,"15min":82.465000000000003},"last":0.23899999999999999},{"interface":"front","average":{"1min":2.3109999999999999,"5min":2.9049999999999998,"15min":2.149},"min":{"1min":0.26500000000000001,"5min":0.218,"15min":0.218},"max":{"1min":12.492000000000001,"5min":82.519000000000005,"15min":82.519000000000005},"last":0.307}]},{"osd":1,"last update":"Tue Sep 10 19:01:50 2024","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":2.7469999999999999,"15min":1.5600000000000001},"min":{"1min":0.29099999999999998,"5min":0.20999999999999999,"15min":0.19500000000000001},"max":{"1min":4.0179999999999998,"5min":67.582999999999998,"15min":67.582999999999998},"last":0.33700000000000002},{"interface":"front","average":{"1min":1.278,"5min":2.657,"15min":1.7190000000000001},"min":{"1min":0.29899999999999999,"5min":0.29699999999999999,"15min":0.20399999999999999},"max":{"1min":3.298,"5min":67.603999999999999,"15min":67.603999999999999},"last":0.43099999999999999}]},{"osd":2,"last update":"Tue Sep 10 19:01:50 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.33700000000000002,"5min":0.27200000000000002,"15min":0.249},"max":{"1min":2.1030000000000002,"5min":7.8879999999999999,"15min":7.8879999999999999},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.752,"15min":0.80200000000000005},"min":{"1min":0.28199999999999997,"5min":0.27100000000000002,"15min":0.24399999999999999},"max":{"1min":2.8479999999999999,"5min":6.7549999999999999,"15min":6.7549999999999999},"last":0.375}]},{"osd":3,"last update":"Tue Sep 10 19:01:50 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":4.8780000000000001,"15min":2.75},"min":{"1min":0.27700000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":5.2750000000000004,"5min":123.34399999999999,"15min":123.34399999999999},"last":0.32300000000000001},{"interface":"front","average":{"1min":1.524,"5min":3.3100000000000001,"15min":2.3479999999999999},"min":{"1min":0.312,"5min":0.27300000000000002,"15min":0.245},"max":{"1min":5.4580000000000002,"5min":82.415000000000006,"15min":82.415000000000006},"last":0.39800000000000002}]},{"osd":4,"last update":"Tue Sep 10 19:01:50 2024","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":0.70899999999999996,"15min":0.91400000000000003},"min":{"1min":0.36399999999999999,"5min":0.221,"15min":0.20899999999999999},"max":{"1min":3.9820000000000002,"5min":3.9820000000000002,"15min":9.0310000000000006},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.63700000000000001,"15min":0.81999999999999995},"min":{"1min":0.317,"5min":0.23899999999999999,"15min":0.23400000000000001},"max":{"1min":2.125,"5min":2.742,"15min":11.182},"last":0.98999999999999999}]},{"osd":6,"last update":"Tue Sep 10 19:02:10 2024","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":0.79000000000000004,"15min":1.0229999999999999},"min":{"1min":0.28799999999999998,"5min":0.23999999999999999,"15min":0.23400000000000001},"max":{"1min":5.4320000000000004,"5min":5.4320000000000004,"15min":11.17},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.123,"5min":0.72099999999999997,"15min":0.99199999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":5.8399999999999999,"5min":5.8399999999999999,"15min":11.18},"last":1.022}]},{"osd":7,"last update":"Tue Sep 10 19:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":0.69399999999999995,"15min":0.94399999999999995},"min":{"1min":0.308,"5min":0.28999999999999998,"15min":0.27600000000000002},"max":{"1min":2.6859999999999999,"5min":3.665,"15min":10.782999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.61299999999999999,"15min":0.83199999999999996},"min":{"1min":0.34699999999999998,"5min":0.26900000000000002,"15min":0.23599999999999999},"max":{"1min":2.6200000000000001,"5min":2.6200000000000001,"15min":11.846},"last":0.33800000000000002}]}]},{"osd":4,"up_from":30,"seq":128849019106,"num_pgs":85,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3016432,"kb_used_data":2130152,"kb_used_omap":1,"kb_used_meta":886142,"kb_avail":90728720,"statfs":{"total":95995035648,"available":92906209280,"internally_reserved":0,"allocated":2181275648,"data_stored":2180872373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":907409752},"hb_peers":[0,1,2,3,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:01:55 2024","interfaces":[{"interface":"back","average":{"1min":1.7569999999999999,"5min":1.5629999999999999,"15min":1.4790000000000001},"min":{"1min":0.37,"5min":0.223,"15min":0.223},"max":{"1min":10.601000000000001,"5min":46.234999999999999,"15min":46.234999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":1.806,"5min":1.4910000000000001,"15min":1.474},"min":{"1min":0.50700000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":10.634,"5min":46.264000000000003,"15min":46.264000000000003},"last":0.56799999999999995}]},{"osd":1,"last update":"Tue Sep 10 19:01:30 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":2.2829999999999999,"15min":1.5660000000000001},"min":{"1min":0.24199999999999999,"5min":0.223,"15min":0.223},"max":{"1min":2.254,"5min":125.15600000000001,"15min":125.15600000000001},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":2.4129999999999998,"15min":1.502},"min":{"1min":0.30199999999999999,"5min":0.23699999999999999,"15min":0.23100000000000001},"max":{"1min":1.48,"5min":125.14700000000001,"15min":125.14700000000001},"last":0.60199999999999998}]},{"osd":2,"last update":"Tue Sep 10 19:01:30 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67400000000000004,"15min":0.68200000000000005},"min":{"1min":0.28499999999999998,"5min":0.26200000000000001,"15min":0.25900000000000001},"max":{"1min":1.5900000000000001,"5min":2.8879999999999999,"15min":5.6189999999999998},"last":0.497},{"interface":"front","average":{"1min":1.2310000000000001,"5min":0.872,"15min":0.79100000000000004},"min":{"1min":0.34699999999999998,"5min":0.29999999999999999,"15min":0.27100000000000002},"max":{"1min":3.673,"5min":12.782999999999999,"15min":12.782999999999999},"last":1.365}]},{"osd":3,"last update":"Tue Sep 10 19:01:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":3.6030000000000002,"15min":2.1230000000000002},"min":{"1min":0.377,"5min":0.28299999999999997,"15min":0.23799999999999999},"max":{"1min":3.556,"5min":111.514,"15min":111.514},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.86399999999999999,"5min":2.6099999999999999,"15min":1.7130000000000001},"min":{"1min":0.27400000000000002,"5min":0.26900000000000002,"15min":0.25700000000000001},"max":{"1min":2.3730000000000002,"5min":110.914,"15min":110.914},"last":0.35899999999999999}]},{"osd":5,"last update":"Tue Sep 10 19:01:55 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.71299999999999997,"15min":0.89300000000000002},"min":{"1min":0.36299999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":2.6699999999999999,"5min":9.7309999999999999,"15min":10.912000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":1.024,"5min":0.75800000000000001,"15min":0.83599999999999997},"min":{"1min":0.33100000000000002,"5min":0.216,"15min":0.216},"max":{"1min":3.0019999999999998,"5min":10.147,"15min":10.147},"last":0.29599999999999999}]},{"osd":6,"last update":"Tue Sep 10 19:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.63100000000000001,"15min":0.77600000000000002},"min":{"1min":0.29699999999999999,"5min":0.221,"15min":0.221},"max":{"1min":1.982,"5min":2.5110000000000001,"15min":8.4030000000000005},"last":0.33900000000000002},{"interface":"front","average":{"1min":1.397,"5min":0.95399999999999996,"15min":0.97199999999999998},"min":{"1min":0.40400000000000003,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":4.8949999999999996,"5min":21.254000000000001,"15min":21.254000000000001},"last":0.46600000000000003}]},{"osd":7,"last update":"Tue Sep 10 19:01:46 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.60899999999999999,"15min":0.82999999999999996},"min":{"1min":0.40600000000000003,"5min":0.29099999999999998,"15min":0.27600000000000002},"max":{"1min":1.516,"5min":2.8879999999999999,"15min":6.4630000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":1.05,"5min":0.71099999999999997,"15min":0.83699999999999997},"min":{"1min":0.311,"5min":0.27500000000000002,"15min":0.251},"max":{"1min":3.1219999999999999,"5min":3.589,"15min":6.3949999999999996},"last":0.86399999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182631,"num_pgs":83,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1753140,"kb_used_data":859564,"kb_used_omap":1,"kb_used_meta":893566,"kb_avail":91992012,"statfs":{"total":95995035648,"available":94199820288,"internally_reserved":0,"allocated":880193536,"data_stored":879786039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":915011928},"hb_peers":[0,1,2,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:01:50 2024","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":4.7140000000000004,"15min":2.4180000000000001},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":5.6349999999999998,"5min":290.721,"15min":290.721},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.82099999999999995,"5min":4.8140000000000001,"15min":2.6150000000000002},"min":{"1min":0.188,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":2.335,"5min":293.59300000000002,"15min":293.59300000000002},"last":0.53000000000000003}]},{"osd":1,"last update":"Tue Sep 10 19:02:07 2024","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.804,"15min":1.623},"min":{"1min":0.246,"5min":0.20399999999999999,"15min":0.16200000000000001},"max":{"1min":3.242,"5min":10.645,"15min":58.215000000000003},"last":0.318},{"interface":"front","average":{"1min":0.86399999999999999,"5min":1.577,"15min":1.522},"min":{"1min":0.23599999999999999,"5min":0.20000000000000001,"15min":0.188},"max":{"1min":3.6819999999999999,"5min":13.263999999999999,"15min":57.896999999999998},"last":0.55600000000000005}]},{"osd":2,"last update":"Tue Sep 10 19:02:07 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":1.254,"15min":0.98599999999999999},"min":{"1min":0.25700000000000001,"5min":0.255,"15min":0.22800000000000001},"max":{"1min":3.7269999999999999,"5min":5.6680000000000001,"15min":5.8220000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.0940000000000001,"15min":0.89500000000000002},"min":{"1min":0.26200000000000001,"5min":0.22600000000000001,"15min":0.21199999999999999},"max":{"1min":5.46,"5min":5.46,"15min":7.0979999999999999},"last":0.42599999999999999}]},{"osd":4,"last update":"Tue Sep 10 19:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.423,"15min":1.2529999999999999},"min":{"1min":0.439,"5min":0.26400000000000001,"15min":0.247},"max":{"1min":3.4249999999999998,"5min":5.0490000000000004,"15min":29.013000000000002},"last":0.39200000000000002},{"interface":"front","average":{"1min":1.103,"5min":1.4430000000000001,"15min":1.248},"min":{"1min":0.39700000000000002,"5min":0.28000000000000003,"15min":0.26800000000000002},"max":{"1min":3.3879999999999999,"5min":8.8130000000000006,"15min":28.826000000000001},"last":2.6219999999999999}]},{"osd":5,"last update":"Tue Sep 10 19:01:54 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.3999999999999999,"15min":1.25},"min":{"1min":0.35399999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":2.6890000000000001,"5min":7.2300000000000004,"15min":7.2300000000000004},"last":1.2370000000000001},{"interface":"front","average":{"1min":1,"5min":1.357,"15min":1.2230000000000001},"min":{"1min":0.28299999999999997,"5min":0.27100000000000002,"15min":0.255},"max":{"1min":4.2789999999999999,"5min":5.4139999999999997,"15min":9.2509999999999994},"last":1.2210000000000001}]},{"osd":6,"last update":"Tue Sep 10 19:02:10 2024","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.5209999999999999,"15min":1.3169999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":3.9089999999999998,"5min":5.4480000000000004,"15min":9.3239999999999998},"last":1.492},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.351,"15min":1.2829999999999999},"min":{"1min":0.51600000000000001,"5min":0.308,"15min":0.29399999999999998},"max":{"1min":6.6390000000000002,"5min":6.6390000000000002,"15min":7.1559999999999997},"last":2.548}]},{"osd":7,"last update":"Tue Sep 10 19:01:40 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":1.5289999999999999,"15min":1.294},"min":{"1min":0.39700000000000002,"5min":0.35599999999999998,"15min":0.34899999999999998},"max":{"1min":2.7290000000000001,"5min":8.7989999999999995,"15min":8.7989999999999995},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":1.46,"15min":1.3140000000000001},"min":{"1min":0.42999999999999999,"5min":0.36899999999999999,"15min":0.33100000000000002},"max":{"1min":2.266,"5min":6.6029999999999998,"15min":9.2949999999999999},"last":0.71599999999999997}]}]},{"osd":2,"up_from":0,"seq":77309411563,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834575087,"num_pgs":87,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1488720,"kb_used_data":593736,"kb_used_omap":1,"kb_used_meta":894974,"kb_avail":92256432,"statfs":{"total":95995035648,"available":94470586368,"internally_reserved":0,"allocated":607985664,"data_stored":607550958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":916453720},"hb_peers":[0,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,5,0,2,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:01:51 2024","interfaces":[{"interface":"back","average":{"1min":2.7410000000000001,"5min":3.181,"15min":1.891},"min":{"1min":0.158,"5min":0.158,"15min":0.158},"max":{"1min":25.370999999999999,"5min":111.014,"15min":111.014},"last":0.247},{"interface":"front","average":{"1min":2.9199999999999999,"5min":3.3580000000000001,"15min":1.8660000000000001},"min":{"1min":0.217,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":27.809999999999999,"5min":111.072,"15min":111.072},"last":0.27100000000000002}]},{"osd":2,"last update":"Tue Sep 10 19:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.873,"15min":0.752},"min":{"1min":0.26500000000000001,"5min":0.23300000000000001,"15min":0.186},"max":{"1min":1.845,"5min":4.1719999999999997,"15min":4.1719999999999997},"last":0.30499999999999999},{"interface":"front","average":{"1min":1.073,"5min":0.95599999999999996,"15min":0.80500000000000005},"min":{"1min":0.25900000000000001,"5min":0.23899999999999999,"15min":0.215},"max":{"1min":4.2560000000000002,"5min":5.6929999999999996,"15min":7.093},"last":0.38300000000000001}]},{"osd":3,"last update":"Tue Sep 10 19:02:06 2024","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":3.089,"15min":2.097},"min":{"1min":0.248,"5min":0.22800000000000001,"15min":0.185},"max":{"1min":4.1349999999999998,"5min":92.069000000000003,"15min":92.069000000000003},"last":4.9050000000000002},{"interface":"front","average":{"1min":0.80900000000000005,"5min":2.496,"15min":1.9299999999999999},"min":{"1min":0.34899999999999998,"5min":0.218,"15min":0.19900000000000001},"max":{"1min":1.8839999999999999,"5min":88.658000000000001,"15min":88.658000000000001},"last":2.3799999999999999}]},{"osd":4,"last update":"Tue Sep 10 19:02:29 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.1299999999999999,"15min":1.1759999999999999},"min":{"1min":0.29699999999999999,"5min":0.27800000000000002,"15min":0.26200000000000001},"max":{"1min":2.359,"5min":9.125,"15min":15.965999999999999},"last":0.624},{"interface":"front","average":{"1min":1.363,"5min":1.139,"15min":1.0660000000000001},"min":{"1min":0.32900000000000001,"5min":0.252,"15min":0.246},"max":{"1min":4.4530000000000003,"5min":7.0940000000000003,"15min":7.0940000000000003},"last":0.32900000000000001}]},{"osd":5,"last update":"Tue Sep 10 19:01:51 2024","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":1.1899999999999999,"15min":1.21},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.28499999999999998},"max":{"1min":4.226,"5min":6.4690000000000003,"15min":10.334},"last":0.85899999999999999},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.075,"15min":1.2170000000000001},"min":{"1min":0.46800000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":3.5089999999999999,"5min":7.0590000000000002,"15min":16.039999999999999},"last":0.65900000000000003}]},{"osd":6,"last update":"Tue Sep 10 19:02:06 2024","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2729999999999999,"15min":1.5429999999999999},"min":{"1min":0.40200000000000002,"5min":0.33100000000000002,"15min":0.308},"max":{"1min":4.202,"5min":19.125,"15min":38.039999999999999},"last":1.2070000000000001},{"interface":"front","average":{"1min":0.83899999999999997,"5min":1.087,"15min":1.597},"min":{"1min":0.34799999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":2.2149999999999999,"5min":7.2380000000000004,"15min":38.100999999999999},"last":1.22}]},{"osd":7,"last update":"Tue Sep 10 19:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":1.2330000000000001,"15min":1.163},"min":{"1min":0.39800000000000002,"5min":0.33100000000000002,"15min":0.315},"max":{"1min":4.1710000000000003,"5min":19.169,"15min":19.169},"last":2.581},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.995,"15min":1.0589999999999999},"min":{"1min":0.39600000000000002,"5min":0.36499999999999999,"15min":0.32800000000000001},"max":{"1min":2.2410000000000001,"5min":4.46,"15min":4.9029999999999996},"last":2.4510000000000001}]}]},{"osd":0,"up_from":241,"seq":1035087118419,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2164020,"kb_used_data":1821864,"kb_used_omap":4,"kb_used_meta":342075,"kb_avail":91581132,"statfs":{"total":95995035648,"available":93779079168,"internally_reserved":0,"allocated":1865588736,"data_stored":1865184717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":350285318},"hb_peers":[1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 19:01:49 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.534,"15min":2.3639999999999999},"min":{"1min":0.22800000000000001,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":4.5810000000000004,"5min":13.617000000000001,"15min":34.692},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.875,"5min":1.419,"15min":2.3530000000000002},"min":{"1min":0.253,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":3.2040000000000002,"5min":9.7100000000000009,"15min":34.588999999999999},"last":0.30199999999999999}]},{"osd":2,"last update":"Tue Sep 10 19:01:49 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":1.0640000000000001,"15min":0.93999999999999995},"min":{"1min":0.23799999999999999,"5min":0.22,"15min":0.20699999999999999},"max":{"1min":4.6130000000000004,"5min":13.305,"15min":13.305},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.79000000000000004,"5min":1.0549999999999999,"15min":0.81399999999999995},"min":{"1min":0.27800000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":3.0960000000000001,"5min":13.693,"15min":13.693},"last":0.33700000000000002}]},{"osd":3,"last update":"Tue Sep 10 19:01:49 2024","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.4930000000000001,"15min":1.145},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":3.2589999999999999,"5min":21.108000000000001,"15min":21.108000000000001},"last":0.25900000000000001},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.9359999999999999,"15min":1.6839999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":6.2949999999999999,"5min":11.629,"15min":11.629},"last":0.316}]},{"osd":4,"last update":"Tue Sep 10 19:01:49 2024","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.212,"15min":0.88200000000000001},"min":{"1min":0.36699999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":4.2830000000000004,"5min":13.161,"15min":13.161},"last":1.427},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.2989999999999999,"15min":1.0349999999999999},"min":{"1min":0.34100000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":4.4020000000000001,"5min":5.0490000000000004,"15min":5.0490000000000004},"last":0.40699999999999997}]},{"osd":5,"last update":"Tue Sep 10 19:01:49 2024","interfaces":[{"interface":"back","average":{"1min":7.0060000000000002,"5min":2.4319999999999999,"15min":1.484},"min":{"1min":0.36399999999999999,"5min":0.31,"15min":0.31},"max":{"1min":97.296999999999997,"5min":97.296999999999997,"15min":97.296999999999997},"last":1.5720000000000001},{"interface":"front","average":{"1min":6.391,"5min":2.2639999999999998,"15min":1.274},"min":{"1min":0.48299999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":99.221999999999994,"5min":99.221999999999994,"15min":99.221999999999994},"last":3.7930000000000001}]},{"osd":6,"last update":"Tue Sep 10 19:01:49 2024","interfaces":[{"interface":"back","average":{"1min":1.524,"5min":1.3200000000000001,"15min":1.048},"min":{"1min":0.39000000000000001,"5min":0.312,"15min":0.312},"max":{"1min":7.351,"5min":7.351,"15min":7.351},"last":0.36699999999999999},{"interface":"front","average":{"1min":1.516,"5min":1.266,"15min":0.88600000000000001},"min":{"1min":0.40100000000000002,"5min":0.313,"15min":0.313},"max":{"1min":4.3849999999999998,"5min":13.279,"15min":13.279},"last":0.45900000000000002}]},{"osd":7,"last update":"Tue Sep 10 19:01:49 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.2509999999999999,"15min":0.96799999999999997},"min":{"1min":0.46899999999999997,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":4.3380000000000001,"5min":13.233000000000001,"15min":13.233000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":2.9430000000000001,"5min":1.494,"15min":1.262},"min":{"1min":0.40100000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":38.636000000000003,"5min":38.636000000000003,"15min":38.636000000000003},"last":0.76800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":2668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1811288064,"data_stored":1811283992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":587268096,"data_stored":587268096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":840368128,"data_stored":840368128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2152075264,"data_stored":2152071192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":927465472,"data_stored":927465472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":837292032,"data_stored":837287960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T19:02:35.204 INFO:tasks.cephadm.ceph_manager.ceph:would drop below min_pgs, proj 6, currently 16 2024-09-10T19:02:35.245 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:35.279 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:35.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:35.280 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:35.381 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:35.418 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:35.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:35.419 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:35.520 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:35.564 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:35.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:35.565 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:35.587 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:35 smithi103 ceph-mon[33163]: from='client.15729 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:02:35.588 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:35 smithi103 ceph-mon[33163]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:02:35.588 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:35 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T19:02:35.588 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:35 smithi103 ceph-mon[33163]: osdmap e369: 8 total, 7 up, 6 in 2024-09-10T19:02:35.665 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:35.701 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:35.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:35.702 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:35.802 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:35.831 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:35.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:35.832 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:35.933 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:35.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:35 smithi018 ceph-mon[40196]: from='client.15729 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:02:35.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:35 smithi018 ceph-mon[40196]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:02:35.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:35 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T19:02:35.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:35 smithi018 ceph-mon[40196]: osdmap e369: 8 total, 7 up, 6 in 2024-09-10T19:02:35.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:35 smithi018 ceph-mon[29336]: from='client.15729 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:02:35.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:35 smithi018 ceph-mon[29336]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:02:35.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:35 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T19:02:35.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:35 smithi018 ceph-mon[29336]: osdmap e369: 8 total, 7 up, 6 in 2024-09-10T19:02:35.967 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:35.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:35.969 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:36.034 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T19:02:36.034 INFO:teuthology.orchestra.run.smithi103.stdout: "size": 20, 2024-09-10T19:02:36.034 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 600, 2024-09-10T19:02:36.034 INFO:teuthology.orchestra.run.smithi103.stdout: "ops": [ 2024-09-10T19:02:36.034 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.034 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:194897 8.2 8:40155cf6:::benchmark_data_smithi018_106531_object48267:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e205)", 2024-09-10T19:02:36.035 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:54:00.261384+0000", 2024-09-10T19:02:36.035 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 515.76719581199995, 2024-09-10T19:02:36.035 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.068155805, 2024-09-10T19:02:36.035 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:36.035 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "reached pg", 2024-09-10T19:02:36.035 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:36.035 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T19:02:36.036 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T19:02:36.036 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 194897 2024-09-10T19:02:36.036 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.036 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:36.036 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.036 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:36.036 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.261384+0000", 2024-09-10T19:02:36.036 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.036 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.036 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.036 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:36.037 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.261384+0000", 2024-09-10T19:02:36.037 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.037 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.037 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.037 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:36.037 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.261382+0000", 2024-09-10T19:02:36.037 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999981 2024-09-10T19:02:36.037 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.037 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.037 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:36.037 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.261389+0000", 2024-09-10T19:02:36.037 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.4549999999999998e-06 2024-09-10T19:02:36.038 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.038 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.038 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:36.038 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.261402+0000", 2024-09-10T19:02:36.038 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.2532e-05 2024-09-10T19:02:36.038 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.038 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.038 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:36.038 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.261408+0000", 2024-09-10T19:02:36.038 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.263e-06 2024-09-10T19:02:36.038 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.038 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.038 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.039 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.261451+0000", 2024-09-10T19:02:36.039 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.2805000000000001e-05 2024-09-10T19:02:36.039 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.039 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.039 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:02:36.039 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.261552+0000", 2024-09-10T19:02:36.039 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00010141600000000001 2024-09-10T19:02:36.039 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.039 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.039 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,5", 2024-09-10T19:02:36.039 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.261654+0000", 2024-09-10T19:02:36.039 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00010126 2024-09-10T19:02:36.039 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.040 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.040 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:36.040 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.267121+0000", 2024-09-10T19:02:36.040 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0054670539999999998 2024-09-10T19:02:36.040 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.040 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.040 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:02:36.040 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.269163+0000", 2024-09-10T19:02:36.040 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0020424430000000001 2024-09-10T19:02:36.040 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.040 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.040 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.040 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.297708+0000", 2024-09-10T19:02:36.041 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.028544488999999999 2024-09-10T19:02:36.041 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.041 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.041 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:36.041 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.297715+0000", 2024-09-10T19:02:36.041 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.3130000000000001e-06 2024-09-10T19:02:36.041 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.042 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.042 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.042 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.329466+0000", 2024-09-10T19:02:36.042 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.031751203 2024-09-10T19:02:36.042 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.042 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.042 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:36.043 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.329540+0000", 2024-09-10T19:02:36.043 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.3632000000000002e-05 2024-09-10T19:02:36.043 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.043 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:36.043 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.043 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.043 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.044 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:194911 8.e 8:7e5954cf:::benchmark_data_smithi018_106531_object48281:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e206)", 2024-09-10T19:02:36.044 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:54:00.294645+0000", 2024-09-10T19:02:36.044 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 515.73393436399999, 2024-09-10T19:02:36.045 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.037229145, 2024-09-10T19:02:36.045 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:36.045 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "reached pg", 2024-09-10T19:02:36.045 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:36.045 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T19:02:36.045 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T19:02:36.046 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 194911 2024-09-10T19:02:36.046 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.046 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:36.046 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.046 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:36.046 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294645+0000", 2024-09-10T19:02:36.046 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.046 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.046 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.046 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:36.046 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294645+0000", 2024-09-10T19:02:36.046 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.046 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.047 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.047 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:36.047 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294644+0000", 2024-09-10T19:02:36.047 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999981 2024-09-10T19:02:36.047 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.047 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.047 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:36.047 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294651+0000", 2024-09-10T19:02:36.048 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.7729999999999999e-06 2024-09-10T19:02:36.048 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.048 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.048 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:36.048 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294664+0000", 2024-09-10T19:02:36.048 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.2401999999999999e-05 2024-09-10T19:02:36.048 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.048 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.048 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:36.048 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294672+0000", 2024-09-10T19:02:36.048 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.2870000000000004e-06 2024-09-10T19:02:36.048 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.049 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.049 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.049 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.295300+0000", 2024-09-10T19:02:36.049 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00062796 2024-09-10T19:02:36.049 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.049 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.049 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:36.049 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.295307+0000", 2024-09-10T19:02:36.049 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.2779999999999996e-06 2024-09-10T19:02:36.049 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.049 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.050 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.050 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.330427+0000", 2024-09-10T19:02:36.050 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.03511982 2024-09-10T19:02:36.050 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.050 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.050 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:36.050 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.331875+0000", 2024-09-10T19:02:36.050 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.001447427 2024-09-10T19:02:36.050 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.050 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:36.052 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.052 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.052 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.052 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:194919 8.2 8:4d34d6e8:::benchmark_data_smithi018_106531_object48289:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e206)", 2024-09-10T19:02:36.052 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:54:00.294686+0000", 2024-09-10T19:02:36.052 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 515.73389389399995, 2024-09-10T19:02:36.052 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0404138540000001, 2024-09-10T19:02:36.052 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:36.052 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:02:36.053 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:36.053 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T19:02:36.053 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T19:02:36.053 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 194919 2024-09-10T19:02:36.053 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.053 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:36.053 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.053 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:36.053 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294686+0000", 2024-09-10T19:02:36.053 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.053 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.053 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.054 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:36.054 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294686+0000", 2024-09-10T19:02:36.054 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.054 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.054 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.054 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:36.054 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294685+0000", 2024-09-10T19:02:36.054 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999995 2024-09-10T19:02:36.054 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.054 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.054 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:36.054 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294689+0000", 2024-09-10T19:02:36.055 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.737e-06 2024-09-10T19:02:36.055 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.056 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.056 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:36.056 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294695+0000", 2024-09-10T19:02:36.056 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.3400000000000003e-06 2024-09-10T19:02:36.056 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.056 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.056 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:36.056 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.294700+0000", 2024-09-10T19:02:36.056 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.1339999999999997e-06 2024-09-10T19:02:36.056 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.056 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.056 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.057 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.299920+0000", 2024-09-10T19:02:36.057 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.005220064 2024-09-10T19:02:36.057 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.057 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.057 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:36.057 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.299926+0000", 2024-09-10T19:02:36.057 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.9789999999999998e-06 2024-09-10T19:02:36.057 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.057 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.057 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.057 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.329569+0000", 2024-09-10T19:02:36.057 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.02964327 2024-09-10T19:02:36.057 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.058 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.058 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:02:36.058 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.329629+0000", 2024-09-10T19:02:36.058 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.0470000000000002e-05 2024-09-10T19:02:36.058 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.058 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.058 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 5", 2024-09-10T19:02:36.058 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.329727+0000", 2024-09-10T19:02:36.058 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.7558000000000003e-05 2024-09-10T19:02:36.058 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.058 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.058 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:36.058 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.334622+0000", 2024-09-10T19:02:36.058 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0048955750000000001 2024-09-10T19:02:36.058 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.059 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.059 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:02:36.059 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.334960+0000", 2024-09-10T19:02:36.059 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00033784 2024-09-10T19:02:36.059 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.059 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.059 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:02:36.059 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.335009+0000", 2024-09-10T19:02:36.059 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.8656000000000003e-05 2024-09-10T19:02:36.059 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.059 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.059 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:36.059 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.335100+0000", 2024-09-10T19:02:36.059 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.0801e-05 2024-09-10T19:02:36.060 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.060 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:36.060 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.060 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.060 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.060 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:194946 8.2 8:469a9e0a:::benchmark_data_smithi018_106531_object48316:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e206)", 2024-09-10T19:02:36.060 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:54:00.320261+0000", 2024-09-10T19:02:36.060 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 515.70831844600002, 2024-09-10T19:02:36.060 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0152080210000001, 2024-09-10T19:02:36.060 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:36.060 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:02:36.060 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:36.060 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T19:02:36.061 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T19:02:36.061 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 194946 2024-09-10T19:02:36.061 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.061 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:36.061 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.061 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:36.061 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320261+0000", 2024-09-10T19:02:36.061 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.061 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.061 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.061 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:36.061 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320261+0000", 2024-09-10T19:02:36.061 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.062 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.062 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.062 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:36.062 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320260+0000", 2024-09-10T19:02:36.062 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T19:02:36.062 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.062 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.062 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:36.062 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320266+0000", 2024-09-10T19:02:36.062 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.9780000000000002e-06 2024-09-10T19:02:36.062 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.062 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.062 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:36.062 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320275+0000", 2024-09-10T19:02:36.063 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.867e-06 2024-09-10T19:02:36.063 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.063 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.063 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:36.063 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320280+0000", 2024-09-10T19:02:36.063 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.4970000000000003e-06 2024-09-10T19:02:36.063 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.063 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.063 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.063 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320314+0000", 2024-09-10T19:02:36.063 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.4644999999999998e-05 2024-09-10T19:02:36.063 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.064 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.064 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:36.064 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320320+0000", 2024-09-10T19:02:36.064 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.2500000000000003e-06 2024-09-10T19:02:36.064 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.064 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.064 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.064 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.331780+0000", 2024-09-10T19:02:36.064 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.01145964 2024-09-10T19:02:36.064 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.064 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.064 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:02:36.064 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.331913+0000", 2024-09-10T19:02:36.065 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00013310700000000001 2024-09-10T19:02:36.065 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.065 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.065 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 5", 2024-09-10T19:02:36.065 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.332054+0000", 2024-09-10T19:02:36.065 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000141013 2024-09-10T19:02:36.065 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.065 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.065 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:02:36.065 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.335153+0000", 2024-09-10T19:02:36.065 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0030985829999999998 2024-09-10T19:02:36.065 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.065 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.066 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:36.066 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.335316+0000", 2024-09-10T19:02:36.066 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000162879 2024-09-10T19:02:36.066 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.066 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.066 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:02:36.066 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.335368+0000", 2024-09-10T19:02:36.066 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.2509999999999997e-05 2024-09-10T19:02:36.066 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.066 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.066 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:36.066 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.335469+0000", 2024-09-10T19:02:36.066 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00010118 2024-09-10T19:02:36.067 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.067 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:36.067 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.067 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.067 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.067 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.24947.0:194947 8.e 8:7cb25091:::benchmark_data_smithi018_106531_object48317:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e206)", 2024-09-10T19:02:36.067 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:54:00.320324+0000", 2024-09-10T19:02:36.067 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 515.708255667, 2024-09-10T19:02:36.067 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0290095960000001, 2024-09-10T19:02:36.067 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:36.067 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:02:36.067 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:36.067 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.24947", 2024-09-10T19:02:36.068 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/1821212698", 2024-09-10T19:02:36.068 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 194947 2024-09-10T19:02:36.068 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.068 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:36.068 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.068 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:36.068 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320324+0000", 2024-09-10T19:02:36.068 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.068 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.068 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.068 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:36.068 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320324+0000", 2024-09-10T19:02:36.068 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.069 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.069 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.069 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:36.069 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320323+0000", 2024-09-10T19:02:36.069 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T19:02:36.069 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.069 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.069 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:36.069 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320329+0000", 2024-09-10T19:02:36.069 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.9649999999999996e-06 2024-09-10T19:02:36.069 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.069 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.069 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:36.070 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320339+0000", 2024-09-10T19:02:36.070 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0263e-05 2024-09-10T19:02:36.070 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.070 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.070 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:36.070 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320345+0000", 2024-09-10T19:02:36.070 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.0419999999999999e-06 2024-09-10T19:02:36.070 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.070 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.070 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.070 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320372+0000", 2024-09-10T19:02:36.070 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.6452e-05 2024-09-10T19:02:36.070 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.071 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.071 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:36.071 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:00.320375+0000", 2024-09-10T19:02:36.071 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.5810000000000001e-06 2024-09-10T19:02:36.071 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.071 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.071 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.071 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.332371+0000", 2024-09-10T19:02:36.071 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.011995341 2024-09-10T19:02:36.071 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.071 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.071 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:02:36.071 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.332852+0000", 2024-09-10T19:02:36.072 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00048134099999999998 2024-09-10T19:02:36.072 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.072 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.072 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 7", 2024-09-10T19:02:36.072 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.333084+0000", 2024-09-10T19:02:36.072 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00023146899999999999 2024-09-10T19:02:36.072 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.072 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.072 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:02:36.072 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.346512+0000", 2024-09-10T19:02:36.072 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.013428697 2024-09-10T19:02:36.072 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.072 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.073 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:36.073 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.349159+0000", 2024-09-10T19:02:36.073 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0026464219999999998 2024-09-10T19:02:36.073 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.073 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.073 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:02:36.073 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.349241+0000", 2024-09-10T19:02:36.073 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.2155000000000002e-05 2024-09-10T19:02:36.073 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.073 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.073 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:36.073 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:54:01.349334+0000", 2024-09-10T19:02:36.074 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.2892000000000004e-05 2024-09-10T19:02:36.074 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.074 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:36.074 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.074 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.074 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.074 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15354.0:167599 9.e 9:71637c4b:::benchmark_data_smithi018_133594_object30847:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-10T19:02:36.074 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:57:50.774991+0000", 2024-09-10T19:02:36.074 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 285.25358880200002, 2024-09-10T19:02:36.074 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0426897470000001, 2024-09-10T19:02:36.074 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:36.074 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:02:36.075 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:36.075 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15354", 2024-09-10T19:02:36.075 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:02:36.075 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 167599 2024-09-10T19:02:36.075 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.075 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:36.075 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.075 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:36.075 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.774991+0000", 2024-09-10T19:02:36.075 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.075 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.075 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.075 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:36.076 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.774991+0000", 2024-09-10T19:02:36.076 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.076 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.076 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.076 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:36.076 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.774989+0000", 2024-09-10T19:02:36.076 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T19:02:36.076 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.076 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.076 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:36.076 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.774997+0000", 2024-09-10T19:02:36.076 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.3130000000000001e-06 2024-09-10T19:02:36.077 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.077 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.077 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:36.077 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.775010+0000", 2024-09-10T19:02:36.077 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.3139000000000001e-05 2024-09-10T19:02:36.077 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.077 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.077 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:36.077 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.775016+0000", 2024-09-10T19:02:36.077 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.5570000000000001e-06 2024-09-10T19:02:36.077 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.077 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.077 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.077 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.775067+0000", 2024-09-10T19:02:36.078 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.0877000000000001e-05 2024-09-10T19:02:36.078 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.078 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.078 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:36.078 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.775074+0000", 2024-09-10T19:02:36.078 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.7630000000000003e-06 2024-09-10T19:02:36.078 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.078 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.078 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.078 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.715900+0000", 2024-09-10T19:02:36.078 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.94082635299999995 2024-09-10T19:02:36.078 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.079 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.079 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:02:36.079 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.715981+0000", 2024-09-10T19:02:36.079 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.0375000000000002e-05 2024-09-10T19:02:36.079 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.079 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.079 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1", 2024-09-10T19:02:36.079 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.716057+0000", 2024-09-10T19:02:36.079 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.6440000000000007e-05 2024-09-10T19:02:36.079 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.079 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.079 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:36.079 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.720375+0000", 2024-09-10T19:02:36.080 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0043181039999999997 2024-09-10T19:02:36.080 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.080 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.080 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:02:36.080 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.720383+0000", 2024-09-10T19:02:36.080 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.3139999999999998e-06 2024-09-10T19:02:36.080 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.080 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.080 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:36.080 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.817622+0000", 2024-09-10T19:02:36.080 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.097239256999999996 2024-09-10T19:02:36.080 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.080 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.080 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:02:36.081 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.817656+0000", 2024-09-10T19:02:36.081 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.4356000000000001e-05 2024-09-10T19:02:36.081 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.081 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.081 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:36.081 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.817681+0000", 2024-09-10T19:02:36.081 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.4494999999999998e-05 2024-09-10T19:02:36.081 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.081 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:36.081 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.081 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.081 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.081 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15354.0:167621 9.e 9:7b014d0c:::benchmark_data_smithi018_133594_object30869:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-10T19:02:36.082 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:57:50.796574+0000", 2024-09-10T19:02:36.082 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 285.23200586299998, 2024-09-10T19:02:36.082 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0211979630000001, 2024-09-10T19:02:36.082 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:36.082 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:02:36.082 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:36.082 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15354", 2024-09-10T19:02:36.082 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:02:36.082 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 167621 2024-09-10T19:02:36.082 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.082 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:36.082 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.082 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:36.083 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.796574+0000", 2024-09-10T19:02:36.083 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.083 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.083 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.083 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:36.083 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.796574+0000", 2024-09-10T19:02:36.083 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.083 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.083 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.083 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:36.083 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.796572+0000", 2024-09-10T19:02:36.083 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T19:02:36.084 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.084 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.084 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:36.084 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.796580+0000", 2024-09-10T19:02:36.084 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.61e-06 2024-09-10T19:02:36.084 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.084 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.084 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:36.084 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.796598+0000", 2024-09-10T19:02:36.084 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.7652999999999999e-05 2024-09-10T19:02:36.084 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.084 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.084 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:36.084 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.796604+0000", 2024-09-10T19:02:36.085 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.1240000000000002e-06 2024-09-10T19:02:36.085 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.085 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.085 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.085 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.796645+0000", 2024-09-10T19:02:36.085 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.0661000000000003e-05 2024-09-10T19:02:36.085 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.085 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.085 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:36.085 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.796657+0000", 2024-09-10T19:02:36.085 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.2568000000000001e-05 2024-09-10T19:02:36.085 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.085 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.086 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.086 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.717750+0000", 2024-09-10T19:02:36.086 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.92109322400000004 2024-09-10T19:02:36.086 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.086 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.086 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:02:36.086 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.717874+0000", 2024-09-10T19:02:36.086 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00012409199999999999 2024-09-10T19:02:36.086 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.086 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.086 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1", 2024-09-10T19:02:36.086 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.717962+0000", 2024-09-10T19:02:36.086 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.7746999999999999e-05 2024-09-10T19:02:36.086 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.087 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.087 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:36.087 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.720488+0000", 2024-09-10T19:02:36.087 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.002526127 2024-09-10T19:02:36.087 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.087 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.087 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:02:36.087 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.721476+0000", 2024-09-10T19:02:36.087 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00098792900000000002 2024-09-10T19:02:36.087 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.087 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.087 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:36.088 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.817721+0000", 2024-09-10T19:02:36.088 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.096244747000000005 2024-09-10T19:02:36.088 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.088 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.088 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:02:36.088 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.817751+0000", 2024-09-10T19:02:36.088 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.9679000000000002e-05 2024-09-10T19:02:36.088 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.088 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.088 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:36.088 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.817772+0000", 2024-09-10T19:02:36.088 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.1280999999999998e-05 2024-09-10T19:02:36.088 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.089 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:36.089 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.089 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.089 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.089 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15354.0:167648 9.e 9:713cb2ff:::benchmark_data_smithi018_133594_object30896:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-10T19:02:36.089 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:57:50.822608+0000", 2024-09-10T19:02:36.089 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 285.20597168900002, 2024-09-10T19:02:36.089 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.99523505199999995, 2024-09-10T19:02:36.089 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:36.089 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:02:36.089 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:36.089 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15354", 2024-09-10T19:02:36.090 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:02:36.090 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 167648 2024-09-10T19:02:36.090 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.090 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:36.090 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.090 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:36.090 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.822608+0000", 2024-09-10T19:02:36.090 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.090 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.090 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.090 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:36.090 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.822608+0000", 2024-09-10T19:02:36.090 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.090 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.091 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.091 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:36.091 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.822607+0000", 2024-09-10T19:02:36.091 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T19:02:36.091 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.091 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.091 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:36.091 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.822612+0000", 2024-09-10T19:02:36.091 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.5840000000000002e-06 2024-09-10T19:02:36.091 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.091 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.091 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:36.091 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.822624+0000", 2024-09-10T19:02:36.092 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.1095e-05 2024-09-10T19:02:36.092 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.092 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.092 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:36.092 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.822628+0000", 2024-09-10T19:02:36.092 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.4889999999999997e-06 2024-09-10T19:02:36.092 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.092 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.092 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.092 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.822662+0000", 2024-09-10T19:02:36.092 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.4029999999999998e-05 2024-09-10T19:02:36.093 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.093 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.093 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:36.093 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.822668+0000", 2024-09-10T19:02:36.093 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.1469999999999998e-06 2024-09-10T19:02:36.093 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.093 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.093 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.093 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.719211+0000", 2024-09-10T19:02:36.093 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.89654309499999996 2024-09-10T19:02:36.093 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.093 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.093 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:02:36.093 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.719322+0000", 2024-09-10T19:02:36.094 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000110701 2024-09-10T19:02:36.094 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.094 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.094 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1", 2024-09-10T19:02:36.094 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.719440+0000", 2024-09-10T19:02:36.094 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000118015 2024-09-10T19:02:36.094 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.094 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.094 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:36.094 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.722546+0000", 2024-09-10T19:02:36.094 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0031055029999999999 2024-09-10T19:02:36.094 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.095 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.095 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:02:36.095 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.722553+0000", 2024-09-10T19:02:36.095 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.6040000000000004e-06 2024-09-10T19:02:36.095 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.095 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.095 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:36.095 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.817795+0000", 2024-09-10T19:02:36.095 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.095241328 2024-09-10T19:02:36.095 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.095 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.095 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:02:36.095 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.817816+0000", 2024-09-10T19:02:36.096 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.1263999999999998e-05 2024-09-10T19:02:36.096 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.096 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.096 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:36.096 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.817843+0000", 2024-09-10T19:02:36.096 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.7396e-05 2024-09-10T19:02:36.096 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.096 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:36.096 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.096 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.096 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.096 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15354.0:167692 9.e 9:7812ba06:::benchmark_data_smithi018_133594_object30940:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-10T19:02:36.097 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:57:50.858240+0000", 2024-09-10T19:02:36.097 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 285.17033941900002, 2024-09-10T19:02:36.097 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.95971880799999998, 2024-09-10T19:02:36.097 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:36.097 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:02:36.097 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:36.097 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15354", 2024-09-10T19:02:36.097 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:02:36.097 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 167692 2024-09-10T19:02:36.097 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.098 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:36.098 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.098 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:36.098 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.858240+0000", 2024-09-10T19:02:36.098 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.098 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.098 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.098 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:36.098 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.858240+0000", 2024-09-10T19:02:36.098 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.098 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.098 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.098 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:36.098 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.858239+0000", 2024-09-10T19:02:36.099 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T19:02:36.099 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.099 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.099 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:36.099 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.858246+0000", 2024-09-10T19:02:36.099 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.4139999999999999e-06 2024-09-10T19:02:36.099 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.099 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.099 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:36.099 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.858258+0000", 2024-09-10T19:02:36.099 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.2136999999999999e-05 2024-09-10T19:02:36.099 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.099 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.099 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:36.099 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.858264+0000", 2024-09-10T19:02:36.100 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:36.101 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.6220000000000003e-06 2024-09-10T19:02:36.101 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.101 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.101 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.101 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.858302+0000", 2024-09-10T19:02:36.101 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.7580000000000003e-05 2024-09-10T19:02:36.102 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.102 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.102 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:36.102 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.858309+0000", 2024-09-10T19:02:36.102 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.9439999999999999e-06 2024-09-10T19:02:36.102 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.102 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.102 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.102 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.720527+0000", 2024-09-10T19:02:36.102 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.86221863600000004 2024-09-10T19:02:36.102 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.102 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.102 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:02:36.102 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.720608+0000", 2024-09-10T19:02:36.103 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.0086000000000005e-05 2024-09-10T19:02:36.103 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.103 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.103 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1", 2024-09-10T19:02:36.103 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.720671+0000", 2024-09-10T19:02:36.103 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.3140999999999997e-05 2024-09-10T19:02:36.103 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.103 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.103 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:36.103 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.722601+0000", 2024-09-10T19:02:36.103 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0019302379999999999 2024-09-10T19:02:36.103 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.103 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.103 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:02:36.103 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.723572+0000", 2024-09-10T19:02:36.104 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00097062600000000004 2024-09-10T19:02:36.104 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.104 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.104 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:36.104 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.817884+0000", 2024-09-10T19:02:36.104 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.094312143000000001 2024-09-10T19:02:36.104 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.104 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.104 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:02:36.104 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.817928+0000", 2024-09-10T19:02:36.104 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.4007000000000001e-05 2024-09-10T19:02:36.104 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.104 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.105 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:36.105 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.817959+0000", 2024-09-10T19:02:36.105 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.1492999999999998e-05 2024-09-10T19:02:36.105 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.105 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:36.105 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.105 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.105 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.105 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15354.0:167728 9.e 9:72c3f2c6:::benchmark_data_smithi018_133594_object30976:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-10T19:02:36.105 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:57:50.891268+0000", 2024-09-10T19:02:36.105 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 285.137311367, 2024-09-10T19:02:36.105 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.930936718, 2024-09-10T19:02:36.105 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:36.106 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:02:36.106 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:36.106 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15354", 2024-09-10T19:02:36.106 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:02:36.106 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 167728 2024-09-10T19:02:36.106 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.106 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:36.106 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.106 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:36.106 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.891268+0000", 2024-09-10T19:02:36.106 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.106 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.106 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.106 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:36.107 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.891268+0000", 2024-09-10T19:02:36.107 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.107 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.107 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.107 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:36.107 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.891266+0000", 2024-09-10T19:02:36.107 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999981 2024-09-10T19:02:36.107 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.107 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.107 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:36.107 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.891274+0000", 2024-09-10T19:02:36.107 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.5689999999999999e-06 2024-09-10T19:02:36.107 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.107 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.107 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:36.108 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.891290+0000", 2024-09-10T19:02:36.108 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.5566e-05 2024-09-10T19:02:36.108 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.108 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.108 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:36.108 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.891296+0000", 2024-09-10T19:02:36.108 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.562e-06 2024-09-10T19:02:36.108 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.108 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.108 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.108 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.891342+0000", 2024-09-10T19:02:36.108 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.5819000000000003e-05 2024-09-10T19:02:36.108 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.108 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.109 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:36.109 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.891349+0000", 2024-09-10T19:02:36.109 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.5109999999999997e-06 2024-09-10T19:02:36.109 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.109 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.109 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.109 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.721510+0000", 2024-09-10T19:02:36.109 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.830160546 2024-09-10T19:02:36.109 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.109 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.109 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:02:36.109 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.721614+0000", 2024-09-10T19:02:36.110 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000104512 2024-09-10T19:02:36.110 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.110 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.110 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1", 2024-09-10T19:02:36.110 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.721731+0000", 2024-09-10T19:02:36.110 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000116742 2024-09-10T19:02:36.110 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.110 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.110 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:36.110 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.724674+0000", 2024-09-10T19:02:36.110 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0029431750000000001 2024-09-10T19:02:36.110 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.110 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.110 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:02:36.110 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.724682+0000", 2024-09-10T19:02:36.111 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.6109999999999996e-06 2024-09-10T19:02:36.111 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.111 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.111 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:36.111 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.822149+0000", 2024-09-10T19:02:36.111 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.097466786999999999 2024-09-10T19:02:36.111 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.111 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.111 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:02:36.111 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.822177+0000", 2024-09-10T19:02:36.111 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.8456999999999999e-05 2024-09-10T19:02:36.111 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.111 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.111 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:36.112 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.822205+0000", 2024-09-10T19:02:36.112 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.7889e-05 2024-09-10T19:02:36.112 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.112 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:36.112 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.112 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.112 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.112 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15354.0:167737 9.e 9:768ac333:::benchmark_data_smithi018_133594_object30985:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-10T19:02:36.112 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:57:50.896128+0000", 2024-09-10T19:02:36.112 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 285.132451484, 2024-09-10T19:02:36.112 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.92899408000000006, 2024-09-10T19:02:36.112 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:36.112 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:02:36.112 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:36.113 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15354", 2024-09-10T19:02:36.113 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:02:36.113 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 167737 2024-09-10T19:02:36.113 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.113 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:36.113 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.113 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:36.113 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.896128+0000", 2024-09-10T19:02:36.113 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.113 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.113 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.113 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:36.113 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.896128+0000", 2024-09-10T19:02:36.113 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.114 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.114 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.114 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:36.114 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.896126+0000", 2024-09-10T19:02:36.114 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999981 2024-09-10T19:02:36.114 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.114 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.114 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:36.114 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.896133+0000", 2024-09-10T19:02:36.114 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.0999999999999998e-06 2024-09-10T19:02:36.114 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.114 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.114 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:36.115 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.896150+0000", 2024-09-10T19:02:36.115 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.6208000000000001e-05 2024-09-10T19:02:36.115 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.115 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.115 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:36.115 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.896156+0000", 2024-09-10T19:02:36.115 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.9730000000000002e-06 2024-09-10T19:02:36.115 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.115 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.116 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.116 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.896198+0000", 2024-09-10T19:02:36.116 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.2080999999999999e-05 2024-09-10T19:02:36.116 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.116 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.116 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:36.116 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.896205+0000", 2024-09-10T19:02:36.116 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.878e-06 2024-09-10T19:02:36.116 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.116 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.116 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.116 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.722638+0000", 2024-09-10T19:02:36.116 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.82643313200000001 2024-09-10T19:02:36.117 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.117 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.117 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:02:36.117 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.722709+0000", 2024-09-10T19:02:36.117 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.0876000000000004e-05 2024-09-10T19:02:36.117 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.117 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.117 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1", 2024-09-10T19:02:36.117 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.722772+0000", 2024-09-10T19:02:36.117 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.3825000000000006e-05 2024-09-10T19:02:36.117 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.117 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.117 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:36.117 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.724720+0000", 2024-09-10T19:02:36.118 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0019477780000000001 2024-09-10T19:02:36.118 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.118 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.118 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:02:36.118 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.725699+0000", 2024-09-10T19:02:36.118 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00097864799999999993 2024-09-10T19:02:36.118 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.118 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.118 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:36.118 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.825043+0000", 2024-09-10T19:02:36.118 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.099344237000000002 2024-09-10T19:02:36.118 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.118 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.118 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:02:36.118 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.825088+0000", 2024-09-10T19:02:36.119 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.4781999999999999e-05 2024-09-10T19:02:36.119 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.119 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.119 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:36.119 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.825122+0000", 2024-09-10T19:02:36.119 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.4541e-05 2024-09-10T19:02:36.119 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.119 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:36.119 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.119 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.119 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.119 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15354.0:167745 9.e 9:7b15c270:::benchmark_data_smithi018_133594_object30993:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-10T19:02:36.120 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:57:50.907688+0000", 2024-09-10T19:02:36.120 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 285.12089212000001, 2024-09-10T19:02:36.120 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.93121002900000005, 2024-09-10T19:02:36.120 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:36.120 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:02:36.120 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:36.120 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15354", 2024-09-10T19:02:36.120 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:02:36.120 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 167745 2024-09-10T19:02:36.120 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.120 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:36.120 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.120 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:36.121 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.907688+0000", 2024-09-10T19:02:36.121 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.121 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.121 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.121 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:36.121 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.907688+0000", 2024-09-10T19:02:36.121 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.121 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.121 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.121 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:36.121 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.907686+0000", 2024-09-10T19:02:36.121 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999981 2024-09-10T19:02:36.121 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.122 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.122 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:36.122 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.907693+0000", 2024-09-10T19:02:36.122 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.2069999999999998e-06 2024-09-10T19:02:36.122 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.122 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.122 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:36.122 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.907706+0000", 2024-09-10T19:02:36.122 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.2911000000000001e-05 2024-09-10T19:02:36.122 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.122 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.122 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:36.122 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.907711+0000", 2024-09-10T19:02:36.123 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.5829999999999997e-06 2024-09-10T19:02:36.123 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.123 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.123 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.123 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.907759+0000", 2024-09-10T19:02:36.123 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.7346999999999999e-05 2024-09-10T19:02:36.123 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.123 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.123 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:36.123 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.907766+0000", 2024-09-10T19:02:36.123 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.2110000000000001e-06 2024-09-10T19:02:36.123 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.123 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.124 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.124 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.723607+0000", 2024-09-10T19:02:36.124 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.81584048899999995 2024-09-10T19:02:36.124 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.124 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.124 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:02:36.124 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.723704+0000", 2024-09-10T19:02:36.124 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.7391000000000003e-05 2024-09-10T19:02:36.124 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.124 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.124 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1", 2024-09-10T19:02:36.124 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.723818+0000", 2024-09-10T19:02:36.124 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00011437799999999999 2024-09-10T19:02:36.125 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.125 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.125 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:36.125 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.725749+0000", 2024-09-10T19:02:36.125 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0019303409999999999 2024-09-10T19:02:36.125 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.125 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.125 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:02:36.125 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.726768+0000", 2024-09-10T19:02:36.125 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0010194990000000001 2024-09-10T19:02:36.125 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.125 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.125 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:36.125 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.838790+0000", 2024-09-10T19:02:36.126 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.11202142599999999 2024-09-10T19:02:36.126 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.126 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.126 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:02:36.126 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.838848+0000", 2024-09-10T19:02:36.126 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.7976000000000002e-05 2024-09-10T19:02:36.126 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.126 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.126 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:36.126 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.838898+0000", 2024-09-10T19:02:36.126 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.0192000000000003e-05 2024-09-10T19:02:36.126 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.126 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:36.127 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.127 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.127 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.127 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15354.0:167756 9.e 9:7dcdb180:::benchmark_data_smithi018_133594_object31004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-10T19:02:36.127 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:57:50.918018+0000", 2024-09-10T19:02:36.127 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 285.11056162900002, 2024-09-10T19:02:36.127 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.92101938000000005, 2024-09-10T19:02:36.127 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:36.127 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:02:36.127 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:36.127 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15354", 2024-09-10T19:02:36.127 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:02:36.127 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 167756 2024-09-10T19:02:36.128 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.128 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:36.128 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.128 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:36.128 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.918018+0000", 2024-09-10T19:02:36.128 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.128 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.128 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.128 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:36.128 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.918018+0000", 2024-09-10T19:02:36.128 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.128 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.128 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.128 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:36.129 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.918017+0000", 2024-09-10T19:02:36.129 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T19:02:36.129 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.129 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.129 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:36.129 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.918024+0000", 2024-09-10T19:02:36.129 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.3810000000000001e-06 2024-09-10T19:02:36.129 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.129 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.129 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:36.129 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.918040+0000", 2024-09-10T19:02:36.129 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.6436e-05 2024-09-10T19:02:36.129 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.129 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.130 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:36.130 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.918047+0000", 2024-09-10T19:02:36.130 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.2530000000000001e-06 2024-09-10T19:02:36.130 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.130 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.130 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.130 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.918092+0000", 2024-09-10T19:02:36.130 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.5880000000000001e-05 2024-09-10T19:02:36.130 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.130 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.130 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:36.130 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.918100+0000", 2024-09-10T19:02:36.130 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.2089999999999999e-06 2024-09-10T19:02:36.131 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.131 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.131 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.131 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.724759+0000", 2024-09-10T19:02:36.131 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.80665893 2024-09-10T19:02:36.131 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.131 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.131 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:02:36.131 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.724836+0000", 2024-09-10T19:02:36.131 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.7366000000000004e-05 2024-09-10T19:02:36.131 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.131 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.131 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1", 2024-09-10T19:02:36.132 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.724897+0000", 2024-09-10T19:02:36.132 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.0674999999999998e-05 2024-09-10T19:02:36.132 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.132 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.132 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:36.132 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.727849+0000", 2024-09-10T19:02:36.132 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0029526740000000002 2024-09-10T19:02:36.132 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.132 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.133 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:02:36.133 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.727854+0000", 2024-09-10T19:02:36.133 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.4490000000000001e-06 2024-09-10T19:02:36.133 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.133 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.133 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:36.133 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.838949+0000", 2024-09-10T19:02:36.133 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.111095204 2024-09-10T19:02:36.133 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.134 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.134 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:02:36.134 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.839003+0000", 2024-09-10T19:02:36.134 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.427e-05 2024-09-10T19:02:36.134 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.134 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.134 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:36.134 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.839038+0000", 2024-09-10T19:02:36.134 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.4304000000000002e-05 2024-09-10T19:02:36.134 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.134 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:36.134 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.134 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.135 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.135 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15354.0:167762 9.e 9:7d919bc4:::benchmark_data_smithi018_133594_object31010:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-10T19:02:36.135 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:57:50.925728+0000", 2024-09-10T19:02:36.135 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 285.10285156600003, 2024-09-10T19:02:36.135 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.918852365, 2024-09-10T19:02:36.135 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:36.135 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:02:36.135 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:36.135 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15354", 2024-09-10T19:02:36.135 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:02:36.136 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 167762 2024-09-10T19:02:36.136 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.136 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:36.136 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.136 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:36.136 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.925728+0000", 2024-09-10T19:02:36.136 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.136 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.136 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.136 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:36.136 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.925728+0000", 2024-09-10T19:02:36.136 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.137 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.137 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.137 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:36.137 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.925726+0000", 2024-09-10T19:02:36.137 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999981 2024-09-10T19:02:36.137 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.137 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.137 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:36.137 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.925734+0000", 2024-09-10T19:02:36.137 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.588e-06 2024-09-10T19:02:36.137 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.138 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.138 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:36.138 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.925749+0000", 2024-09-10T19:02:36.138 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.4898e-05 2024-09-10T19:02:36.138 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.138 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.138 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:36.138 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.925755+0000", 2024-09-10T19:02:36.138 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.5810000000000002e-06 2024-09-10T19:02:36.138 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.138 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.138 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.139 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.925817+0000", 2024-09-10T19:02:36.139 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.1474000000000001e-05 2024-09-10T19:02:36.139 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.139 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.139 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:36.139 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.925841+0000", 2024-09-10T19:02:36.139 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.4457999999999999e-05 2024-09-10T19:02:36.139 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.139 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.139 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.139 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.725780+0000", 2024-09-10T19:02:36.139 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.79993912300000003 2024-09-10T19:02:36.140 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.140 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.140 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:02:36.140 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.725858+0000", 2024-09-10T19:02:36.140 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.7304000000000004e-05 2024-09-10T19:02:36.140 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.140 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.140 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1", 2024-09-10T19:02:36.140 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.725920+0000", 2024-09-10T19:02:36.140 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.2323999999999995e-05 2024-09-10T19:02:36.140 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.140 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.140 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:36.141 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.727885+0000", 2024-09-10T19:02:36.141 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0019648759999999999 2024-09-10T19:02:36.141 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.141 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.141 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:02:36.141 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.728835+0000", 2024-09-10T19:02:36.141 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00094996299999999996 2024-09-10T19:02:36.141 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.141 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.141 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:36.141 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.844492+0000", 2024-09-10T19:02:36.141 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.11565766399999999 2024-09-10T19:02:36.141 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.142 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.142 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:02:36.142 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.844542+0000", 2024-09-10T19:02:36.142 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.9907999999999997e-05 2024-09-10T19:02:36.142 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.142 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.142 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:36.142 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.844581+0000", 2024-09-10T19:02:36.142 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.8222000000000001e-05 2024-09-10T19:02:36.142 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.142 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:36.142 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.142 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.143 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.143 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15354.0:167809 9.e 9:71c76b43:::benchmark_data_smithi018_133594_object31057:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-10T19:02:36.143 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:57:50.978436+0000", 2024-09-10T19:02:36.143 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 285.05014425600001, 2024-09-10T19:02:36.143 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.86652940499999997, 2024-09-10T19:02:36.143 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:36.143 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:02:36.143 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:36.143 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15354", 2024-09-10T19:02:36.143 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:02:36.143 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 167809 2024-09-10T19:02:36.143 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.143 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:36.144 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.144 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:36.144 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.978436+0000", 2024-09-10T19:02:36.144 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.144 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.144 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.144 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:36.144 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.978436+0000", 2024-09-10T19:02:36.144 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.144 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.144 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.144 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:36.144 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.978434+0000", 2024-09-10T19:02:36.145 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T19:02:36.145 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.145 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.145 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:36.145 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.978441+0000", 2024-09-10T19:02:36.145 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.2649999999999999e-06 2024-09-10T19:02:36.145 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.145 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.145 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:36.145 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.978455+0000", 2024-09-10T19:02:36.145 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.3465000000000001e-05 2024-09-10T19:02:36.145 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.145 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.146 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:36.146 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.978460+0000", 2024-09-10T19:02:36.146 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.7470000000000002e-06 2024-09-10T19:02:36.146 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.146 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.146 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.146 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.978500+0000", 2024-09-10T19:02:36.146 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.9402e-05 2024-09-10T19:02:36.146 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.146 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.146 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:36.146 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.978508+0000", 2024-09-10T19:02:36.146 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.7950000000000008e-06 2024-09-10T19:02:36.146 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.147 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.147 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.147 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.726804+0000", 2024-09-10T19:02:36.147 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.74829651100000005 2024-09-10T19:02:36.147 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.147 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.147 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:02:36.147 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.726911+0000", 2024-09-10T19:02:36.147 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000107328 2024-09-10T19:02:36.147 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.147 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.147 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1", 2024-09-10T19:02:36.147 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.727026+0000", 2024-09-10T19:02:36.148 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000114868 2024-09-10T19:02:36.148 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.148 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.148 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:02:36.148 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.731181+0000", 2024-09-10T19:02:36.148 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0041549409999999997 2024-09-10T19:02:36.148 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.148 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.148 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:36.148 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.731226+0000", 2024-09-10T19:02:36.148 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.5114999999999997e-05 2024-09-10T19:02:36.148 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.149 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.149 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:36.149 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.844881+0000", 2024-09-10T19:02:36.149 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.113654422 2024-09-10T19:02:36.149 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.149 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.149 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:02:36.149 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.844927+0000", 2024-09-10T19:02:36.149 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.6462999999999999e-05 2024-09-10T19:02:36.149 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.149 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.149 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:36.149 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.844965+0000", 2024-09-10T19:02:36.150 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.769e-05 2024-09-10T19:02:36.150 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.150 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:36.150 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.150 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.150 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.150 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15354.0:167810 9.e 9:7d98078b:::benchmark_data_smithi018_133594_object31058:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-10T19:02:36.150 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:57:50.979544+0000", 2024-09-10T19:02:36.150 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 285.04903536900002, 2024-09-10T19:02:36.150 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.86553316000000002, 2024-09-10T19:02:36.150 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:36.150 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:02:36.150 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:36.150 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15354", 2024-09-10T19:02:36.151 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:02:36.151 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 167810 2024-09-10T19:02:36.151 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.151 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:36.151 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.151 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:36.151 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.979544+0000", 2024-09-10T19:02:36.151 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.151 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.151 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.151 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:36.152 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.979544+0000", 2024-09-10T19:02:36.152 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.152 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.152 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.152 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:36.152 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.979543+0000", 2024-09-10T19:02:36.152 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T19:02:36.152 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.152 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.152 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:36.152 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.979549+0000", 2024-09-10T19:02:36.153 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.5310000000000001e-06 2024-09-10T19:02:36.153 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.153 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.153 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:36.153 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.979563+0000", 2024-09-10T19:02:36.153 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.4177e-05 2024-09-10T19:02:36.153 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.153 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.153 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:36.153 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.979569+0000", 2024-09-10T19:02:36.153 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.4099999999999999e-06 2024-09-10T19:02:36.153 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.153 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.154 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.154 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.979616+0000", 2024-09-10T19:02:36.154 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.7318000000000003e-05 2024-09-10T19:02:36.154 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.154 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.154 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:36.154 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.979625+0000", 2024-09-10T19:02:36.154 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.003e-06 2024-09-10T19:02:36.154 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.154 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.154 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.154 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.727898+0000", 2024-09-10T19:02:36.154 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.74827312899999998 2024-09-10T19:02:36.155 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.155 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.155 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:02:36.155 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.727959+0000", 2024-09-10T19:02:36.155 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.122e-05 2024-09-10T19:02:36.155 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.155 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.155 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1", 2024-09-10T19:02:36.155 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.728021+0000", 2024-09-10T19:02:36.155 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.1752000000000001e-05 2024-09-10T19:02:36.155 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.155 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.155 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:36.156 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.731274+0000", 2024-09-10T19:02:36.156 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.003253443 2024-09-10T19:02:36.156 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.156 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.156 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:02:36.156 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.731281+0000", 2024-09-10T19:02:36.156 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.7959999999999998e-06 2024-09-10T19:02:36.156 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.156 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.156 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:36.156 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.845004+0000", 2024-09-10T19:02:36.156 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.113723184 2024-09-10T19:02:36.157 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.157 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.157 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:02:36.157 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.845051+0000", 2024-09-10T19:02:36.157 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.693e-05 2024-09-10T19:02:36.157 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.157 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.157 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:36.157 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.845078+0000", 2024-09-10T19:02:36.157 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.6211999999999999e-05 2024-09-10T19:02:36.157 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.157 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:36.157 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.158 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.158 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.158 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15354.0:167811 9.e 9:7a0a22e3:::benchmark_data_smithi018_133594_object31059:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-10T19:02:36.158 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:57:50.981876+0000", 2024-09-10T19:02:36.158 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 285.04670402800002, 2024-09-10T19:02:36.158 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.86330044299999997, 2024-09-10T19:02:36.158 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:36.158 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:02:36.158 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:36.158 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15354", 2024-09-10T19:02:36.158 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:02:36.158 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 167811 2024-09-10T19:02:36.158 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.158 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:36.158 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.159 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:36.159 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.981876+0000", 2024-09-10T19:02:36.159 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.159 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.159 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.159 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:36.159 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.981876+0000", 2024-09-10T19:02:36.159 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.159 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.159 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.159 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:36.159 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.981874+0000", 2024-09-10T19:02:36.159 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T19:02:36.159 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.160 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.160 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:36.160 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.981881+0000", 2024-09-10T19:02:36.160 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.3969999999999999e-06 2024-09-10T19:02:36.160 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.161 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.161 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:36.161 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.981894+0000", 2024-09-10T19:02:36.161 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.3688e-05 2024-09-10T19:02:36.161 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.161 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.161 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:36.161 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.981901+0000", 2024-09-10T19:02:36.161 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.3489999999999997e-06 2024-09-10T19:02:36.161 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.161 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.162 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.162 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.981950+0000", 2024-09-10T19:02:36.162 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.9154999999999999e-05 2024-09-10T19:02:36.162 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.162 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.162 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:36.162 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.981959+0000", 2024-09-10T19:02:36.162 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.6989999999999992e-06 2024-09-10T19:02:36.162 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.162 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.162 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.163 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.728874+0000", 2024-09-10T19:02:36.163 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.74691576100000001 2024-09-10T19:02:36.163 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.163 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.163 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:02:36.163 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.728973+0000", 2024-09-10T19:02:36.163 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.9053000000000002e-05 2024-09-10T19:02:36.163 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.163 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.163 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1", 2024-09-10T19:02:36.163 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.729085+0000", 2024-09-10T19:02:36.163 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000112018 2024-09-10T19:02:36.163 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.164 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.164 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:02:36.164 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.731288+0000", 2024-09-10T19:02:36.164 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.002202182 2024-09-10T19:02:36.164 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.164 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.164 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:36.164 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.731331+0000", 2024-09-10T19:02:36.164 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.3282999999999999e-05 2024-09-10T19:02:36.164 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.164 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.164 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:36.164 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.845117+0000", 2024-09-10T19:02:36.165 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.11378611 2024-09-10T19:02:36.165 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.165 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.165 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:02:36.165 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.845146+0000", 2024-09-10T19:02:36.165 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.8762999999999999e-05 2024-09-10T19:02:36.165 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.165 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.165 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:36.165 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.845176+0000", 2024-09-10T19:02:36.165 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.0389999999999999e-05 2024-09-10T19:02:36.165 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.166 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:36.166 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.166 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.166 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.166 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15354.0:167827 9.e 9:746d7e82:::benchmark_data_smithi018_133594_object31075:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-10T19:02:36.166 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:57:51.002754+0000", 2024-09-10T19:02:36.166 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 285.02582599599998, 2024-09-10T19:02:36.166 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.84277010500000005, 2024-09-10T19:02:36.166 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:36.166 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:02:36.166 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:36.166 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15354", 2024-09-10T19:02:36.166 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:02:36.167 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 167827 2024-09-10T19:02:36.167 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.167 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:36.167 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.167 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:36.167 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.002754+0000", 2024-09-10T19:02:36.167 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.167 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.167 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.167 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:36.167 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.002754+0000", 2024-09-10T19:02:36.167 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.168 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.168 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.168 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:36.168 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.002753+0000", 2024-09-10T19:02:36.168 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T19:02:36.168 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.168 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.168 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:36.168 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.002759+0000", 2024-09-10T19:02:36.168 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.4409999999999998e-06 2024-09-10T19:02:36.168 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.168 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.168 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:36.168 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.002772+0000", 2024-09-10T19:02:36.169 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.322e-05 2024-09-10T19:02:36.169 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.169 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.169 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:36.169 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.002779+0000", 2024-09-10T19:02:36.169 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.4280000000000001e-06 2024-09-10T19:02:36.169 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.169 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.169 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.169 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.002827+0000", 2024-09-10T19:02:36.169 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.8736999999999998e-05 2024-09-10T19:02:36.169 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.169 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.170 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:36.170 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.002835+0000", 2024-09-10T19:02:36.170 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.4089999999999997e-06 2024-09-10T19:02:36.170 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.170 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.170 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.170 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.729879+0000", 2024-09-10T19:02:36.170 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.72704418800000004 2024-09-10T19:02:36.170 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.170 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.170 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:02:36.170 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.729991+0000", 2024-09-10T19:02:36.170 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000111604 2024-09-10T19:02:36.171 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.171 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.171 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1", 2024-09-10T19:02:36.171 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.730113+0000", 2024-09-10T19:02:36.171 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00012227399999999999 2024-09-10T19:02:36.171 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.171 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.171 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:36.171 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.731379+0000", 2024-09-10T19:02:36.171 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0012659920000000001 2024-09-10T19:02:36.171 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.171 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.171 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:02:36.171 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.733829+0000", 2024-09-10T19:02:36.172 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.002449887 2024-09-10T19:02:36.172 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.172 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.172 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:36.172 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.845440+0000", 2024-09-10T19:02:36.172 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.111610945 2024-09-10T19:02:36.172 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.172 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.172 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:02:36.172 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.845484+0000", 2024-09-10T19:02:36.172 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.4742e-05 2024-09-10T19:02:36.172 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.172 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.173 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:36.173 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.845524+0000", 2024-09-10T19:02:36.173 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.9487999999999999e-05 2024-09-10T19:02:36.173 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.173 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:36.173 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.173 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.173 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.173 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15354.0:167861 9.e 9:7e1da624:::benchmark_data_smithi018_133594_object31109:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-10T19:02:36.173 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:57:51.222363+0000", 2024-09-10T19:02:36.173 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 284.80621714, 2024-09-10T19:02:36.173 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.62618236100000002, 2024-09-10T19:02:36.173 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:36.174 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:02:36.174 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:36.174 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15354", 2024-09-10T19:02:36.174 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:02:36.174 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 167861 2024-09-10T19:02:36.174 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.174 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:36.174 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.174 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:36.174 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.222363+0000", 2024-09-10T19:02:36.174 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.174 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.174 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.174 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:36.174 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.222363+0000", 2024-09-10T19:02:36.175 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.175 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.175 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.175 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:36.175 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.222361+0000", 2024-09-10T19:02:36.175 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T19:02:36.175 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.175 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.175 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:36.175 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.222368+0000", 2024-09-10T19:02:36.175 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.2099999999999998e-06 2024-09-10T19:02:36.175 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.175 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.175 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:36.176 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.222378+0000", 2024-09-10T19:02:36.176 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0134e-05 2024-09-10T19:02:36.176 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.176 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.176 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:36.176 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.222382+0000", 2024-09-10T19:02:36.176 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.4739999999999999e-06 2024-09-10T19:02:36.176 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.176 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.176 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.176 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.222421+0000", 2024-09-10T19:02:36.176 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.8720000000000002e-05 2024-09-10T19:02:36.176 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.176 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.177 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:36.177 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.222427+0000", 2024-09-10T19:02:36.177 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.8200000000000002e-06 2024-09-10T19:02:36.177 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.177 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.177 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.177 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.731414+0000", 2024-09-10T19:02:36.177 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.508986893 2024-09-10T19:02:36.177 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.177 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.177 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:02:36.177 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.731519+0000", 2024-09-10T19:02:36.177 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000105278 2024-09-10T19:02:36.177 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.177 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.178 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1", 2024-09-10T19:02:36.178 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.731634+0000", 2024-09-10T19:02:36.178 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000114666 2024-09-10T19:02:36.178 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.178 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.178 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:36.178 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.734867+0000", 2024-09-10T19:02:36.178 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0032337080000000001 2024-09-10T19:02:36.178 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.178 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.178 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:02:36.178 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.734875+0000", 2024-09-10T19:02:36.178 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.357e-06 2024-09-10T19:02:36.178 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.178 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.179 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:36.179 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.848474+0000", 2024-09-10T19:02:36.179 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.113599225 2024-09-10T19:02:36.179 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.179 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.179 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:02:36.179 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.848512+0000", 2024-09-10T19:02:36.179 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.8501000000000003e-05 2024-09-10T19:02:36.179 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.179 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.179 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:36.179 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.848545+0000", 2024-09-10T19:02:36.179 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.2580000000000003e-05 2024-09-10T19:02:36.179 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.180 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:36.180 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.180 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.180 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.180 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15354.0:167870 9.e 9:76bbcffa:::benchmark_data_smithi018_133594_object31118:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-10T19:02:36.180 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:57:51.243031+0000", 2024-09-10T19:02:36.180 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 284.785548758, 2024-09-10T19:02:36.180 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.605625631, 2024-09-10T19:02:36.180 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:36.180 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:02:36.180 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:36.180 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15354", 2024-09-10T19:02:36.180 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:02:36.180 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 167870 2024-09-10T19:02:36.181 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.181 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:36.181 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.181 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:36.181 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.243031+0000", 2024-09-10T19:02:36.181 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.181 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.181 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.181 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:36.181 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.243031+0000", 2024-09-10T19:02:36.181 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:36.181 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.181 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.181 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:36.182 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.243029+0000", 2024-09-10T19:02:36.182 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T19:02:36.182 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.182 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.182 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:36.182 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.243037+0000", 2024-09-10T19:02:36.182 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.1280000000000002e-06 2024-09-10T19:02:36.182 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.182 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.182 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:36.182 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.243048+0000", 2024-09-10T19:02:36.182 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.1419e-05 2024-09-10T19:02:36.182 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.183 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.183 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:36.183 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.243054+0000", 2024-09-10T19:02:36.183 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.0859999999999998e-06 2024-09-10T19:02:36.183 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.183 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.183 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.183 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.243109+0000", 2024-09-10T19:02:36.183 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.4691000000000002e-05 2024-09-10T19:02:36.183 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.183 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.183 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:36.183 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.243117+0000", 2024-09-10T19:02:36.184 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.0420000000000003e-06 2024-09-10T19:02:36.184 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.184 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.184 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:36.184 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.732475+0000", 2024-09-10T19:02:36.184 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.48935852800000001 2024-09-10T19:02:36.184 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.184 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.184 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:02:36.184 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.732582+0000", 2024-09-10T19:02:36.184 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000106531 2024-09-10T19:02:36.184 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.184 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.185 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1", 2024-09-10T19:02:36.185 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.732705+0000", 2024-09-10T19:02:36.185 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00012282800000000001 2024-09-10T19:02:36.185 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.185 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.185 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:36.185 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.734908+0000", 2024-09-10T19:02:36.185 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0022032779999999999 2024-09-10T19:02:36.185 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.185 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.185 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:02:36.185 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.734923+0000", 2024-09-10T19:02:36.185 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.4601e-05 2024-09-10T19:02:36.186 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.186 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.186 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:36.186 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.848583+0000", 2024-09-10T19:02:36.186 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.113659945 2024-09-10T19:02:36.186 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.186 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.186 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:02:36.186 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.848624+0000", 2024-09-10T19:02:36.186 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.1455000000000002e-05 2024-09-10T19:02:36.186 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:36.186 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:36.186 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:36.187 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.848657+0000", 2024-09-10T19:02:36.187 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.2688999999999998e-05 2024-09-10T19:02:36.187 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.187 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:36.187 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.187 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:36.187 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:36.187 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T19:02:36.192 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:36.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:36.193 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:36.294 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:36.331 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:36.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:36.333 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:36.434 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:36.465 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:36.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:36.467 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:36.567 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:36.601 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:36.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:36.601 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:36.702 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:36.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:36.740 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:36.741 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:36.741 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:36 smithi018 ceph-mon[29336]: pgmap v802: 168 pgs: 168 active+clean; 1.7 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 632 B/s rd, 0 B/s wr, 1.21k op/s 2024-09-10T19:02:36.742 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:36 smithi018 ceph-mon[40196]: pgmap v802: 168 pgs: 168 active+clean; 1.7 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 632 B/s rd, 0 B/s wr, 1.21k op/s 2024-09-10T19:02:36.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:36 smithi103 ceph-mon[33163]: pgmap v802: 168 pgs: 168 active+clean; 1.7 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 632 B/s rd, 0 B/s wr, 1.21k op/s 2024-09-10T19:02:36.841 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:36.870 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:36.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:36.870 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:36.971 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:37.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:37.007 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:37.008 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:37.108 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:37.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:37.158 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:37.159 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:37.260 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:37.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:37.298 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:37.298 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:37.340 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:02:37 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:02:37.380 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-10T19:02:37.399 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:37.452 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T19:02:37.452 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T19:02:37.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:37.467 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:37.467 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:37.481 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T19:02:37.482 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T19:02:37.513 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T19:02:37.514 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T19:02:37.542 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T19:02:37.543 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T19:02:37.567 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:37.580 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T19:02:37.580 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T19:02:37.580 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:37 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2287014393' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:02:37.583 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:37 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2287014393' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:02:37.607 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:37.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:37.608 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:37.613 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T19:02:37.614 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T19:02:37.648 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T19:02:37.648 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T19:02:37.649 INFO:teuthology.orchestra.run.smithi018.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-10T19:02:37.708 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:37.754 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:37.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:37.755 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:37.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:37 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2287014393' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:02:37.808 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:02:37.856 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:37.898 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:37.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:37.899 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:37.999 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:38.033 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:38.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:38.034 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:38.135 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:38.176 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:38.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:38.177 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:38.278 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:38.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:38.316 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:38.316 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:38.412 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-10T19:02:38.413 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-10T19:02:38.417 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:38.438 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:38.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:38.439 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:38.540 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:38.577 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:38.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:38.578 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:38.635 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-10T19:02:38.669 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:38 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:38.669 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:38 smithi018 ceph-mon[29336]: pgmap v803: 168 pgs: 168 active+clean; 1.3 GiB data, 9.9 GiB used, 526 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.64k op/s 2024-09-10T19:02:38.669 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:38 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:38.669 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:38 smithi018 ceph-mon[40196]: pgmap v803: 168 pgs: 168 active+clean; 1.3 GiB data, 9.9 GiB used, 526 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.64k op/s 2024-09-10T19:02:38.680 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:38.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:38.727 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:38.727 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:38.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:38 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:38.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:38 smithi103 ceph-mon[33163]: pgmap v803: 168 pgs: 168 active+clean; 1.3 GiB data, 9.9 GiB used, 526 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.64k op/s 2024-09-10T19:02:38.827 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:38.866 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:38.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:38.867 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:38.968 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:39.002 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:39.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:39.003 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:39.054 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:02:39.104 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:39.127 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:39.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:39.129 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:39.229 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:39.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:39.266 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:39.266 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:39.367 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:39.403 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:39.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:39.404 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:39.505 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:39.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:39.538 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:39.538 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:39.639 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:39.664 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:39.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:39.665 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:39.766 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:39.800 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:39.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:39.801 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:39.901 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:39.941 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:39.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:39.942 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:40.042 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:40.074 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:40.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:40.075 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:40.176 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:40.205 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6] out_osds: [2, 7] dead_osds: [7] live_osds: [1, 2, 3, 4, 5, 6, 0] 2024-09-10T19:02:40.205 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T19:02:40.206 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-10T19:02:40.206 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 0.010102 2024-09-10T19:02:40.206 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd primary-affinity 1 0.010102103415535302 2024-09-10T19:02:40.214 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:40.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:40.215 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:40.316 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:40.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:40.356 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:40.356 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:40.414 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T19:02:40.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:02:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T19:02:40.093+0000 7f66d46e1640 -1 mgr get_metadata_python Requested missing service osd.7 2024-09-10T19:02:40.457 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:40.491 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:40.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:40.492 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:40.495 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:02:40 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:02:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:02:40.593 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:40.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:40.629 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:40.630 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:40.730 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:40.757 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:40 smithi103 ceph-mon[33163]: pgmap v804: 168 pgs: 168 active+clean; 1.2 GiB data, 9.9 GiB used, 527 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.63k op/s 2024-09-10T19:02:40.757 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:40 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:02:40.771 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:40.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:40.772 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:40.817 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:40 smithi018 ceph-mon[29336]: pgmap v804: 168 pgs: 168 active+clean; 1.2 GiB data, 9.9 GiB used, 527 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.63k op/s 2024-09-10T19:02:40.817 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:40 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:02:40.817 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:40 smithi018 ceph-mon[40196]: pgmap v804: 168 pgs: 168 active+clean; 1.2 GiB data, 9.9 GiB used, 527 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.63k op/s 2024-09-10T19:02:40.817 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:40 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:02:40.873 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:40.916 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:40.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:40.917 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:41.018 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:41.054 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:41.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:41.055 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:41.070 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:02:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:02:40] "GET /metrics HTTP/1.1" 200 209529 "" "Prometheus/2.43.0" 2024-09-10T19:02:41.156 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:41.193 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:41.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:41.194 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:41.295 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:41.329 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:41.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:41.330 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:41.430 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:41.461 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:41.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:41.462 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:41.563 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:41.595 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:41.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:41.596 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:41.697 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:41.722 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T19:02:41.722 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T19:02:41.732 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:41.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:41.733 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:41.834 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:41.856 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:41.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:41.857 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:41.958 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:41.995 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:41.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:41.996 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:42.097 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:42.127 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:42.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:42.128 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:42.229 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:42.252 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:42.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:42.253 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:42.354 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:42.385 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:42.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:42.388 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:42.489 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:42.522 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:42.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:42.523 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:42.573 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-10T19:02:42.624 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:42.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:42.667 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:42.667 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:42.768 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:42.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:42 smithi103 ceph-mon[33163]: pgmap v805: 168 pgs: 168 active+clean; 1.2 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.63k op/s 2024-09-10T19:02:42.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:42.802 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:42.802 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:42.903 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:42.942 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:42.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:42.943 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:42.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:42 smithi018 ceph-mon[40196]: pgmap v805: 168 pgs: 168 active+clean; 1.2 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.63k op/s 2024-09-10T19:02:42.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:42 smithi018 ceph-mon[29336]: pgmap v805: 168 pgs: 168 active+clean; 1.2 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.63k op/s 2024-09-10T19:02:43.032 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:02:43.045 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:43.078 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:43.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:43.079 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:43.179 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:43.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:43.222 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:43.222 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:43.323 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:43.356 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:43.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:43.357 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:43.458 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:43.486 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:02:43.493 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:02:43.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:43.505 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:43.505 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:43.606 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:43.619 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:43 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:02:43.619 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:43 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:02:43.619 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:43 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:02:43.619 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:43 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:02:43.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:43.643 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:43.643 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:43.744 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:43.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:43 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:02:43.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:43 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:02:43.796 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:43.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:43.797 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:43.897 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:43.934 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:43.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:43.934 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:44.035 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:44.078 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:44.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:44.079 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:44.180 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:44.216 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:44.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:44.217 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:44.318 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:44.357 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:44.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:44.358 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:44.459 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:44.498 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:44.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:44.499 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:44.600 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:44.633 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:44.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:44.634 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:44.735 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:44.770 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:44.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:44.771 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:44.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:44 smithi103 ceph-mon[33163]: pgmap v806: 168 pgs: 168 active+clean; 839 MiB data, 8.7 GiB used, 528 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.62k op/s 2024-09-10T19:02:44.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:44 smithi103 ceph-mon[33163]: osdmap e370: 8 total, 7 up, 6 in 2024-09-10T19:02:44.872 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:44.906 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:44.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:44.907 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:44.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:44 smithi018 ceph-mon[29336]: pgmap v806: 168 pgs: 168 active+clean; 839 MiB data, 8.7 GiB used, 528 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.62k op/s 2024-09-10T19:02:44.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:44 smithi018 ceph-mon[29336]: osdmap e370: 8 total, 7 up, 6 in 2024-09-10T19:02:44.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:44 smithi018 ceph-mon[40196]: pgmap v806: 168 pgs: 168 active+clean; 839 MiB data, 8.7 GiB used, 528 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.62k op/s 2024-09-10T19:02:44.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:44 smithi018 ceph-mon[40196]: osdmap e370: 8 total, 7 up, 6 in 2024-09-10T19:02:45.008 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:45.053 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:45.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:45.054 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:45.155 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:45.185 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:45.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:45.186 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:45.287 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:45.327 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:45.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:45.328 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:45.429 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:45.452 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:45.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:45.453 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:45.538 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is unset 2024-09-10T19:02:45.554 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:45.596 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:45.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:45.597 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:45.698 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:45.723 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:45.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:45.724 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:45.824 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:45.861 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:45.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:45.863 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:45.939 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:45 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/393530363' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:02:45.939 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:45 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:02:45.940 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:45 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/393530363' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:02:45.940 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:45 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:02:45.963 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:45.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:45.995 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:45.995 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:46.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:45 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/393530363' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:02:46.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:45 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:02:46.096 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:46.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:46.128 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:46.129 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:46.229 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:46.267 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:46.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:46.268 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:46.369 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:46.404 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:46.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:46.405 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:46.506 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:46.544 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:46.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:46.545 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:46.646 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:46.675 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:46.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:46.676 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:46.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:46 smithi018 ceph-mon[40196]: pgmap v808: 168 pgs: 168 active+clean; 746 MiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.61k op/s 2024-09-10T19:02:46.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:46 smithi018 ceph-mon[40196]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:02:46.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:46 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T19:02:46.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:46 smithi018 ceph-mon[40196]: osdmap e371: 8 total, 7 up, 6 in 2024-09-10T19:02:46.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:46 smithi018 ceph-mon[40196]: 6.18 scrub starts 2024-09-10T19:02:46.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:46 smithi018 ceph-mon[40196]: 6.18 scrub ok 2024-09-10T19:02:46.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:46 smithi018 ceph-mon[40196]: 5.9 scrub starts 2024-09-10T19:02:46.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:46 smithi018 ceph-mon[40196]: 5.9 scrub ok 2024-09-10T19:02:46.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:46 smithi018 ceph-mon[29336]: pgmap v808: 168 pgs: 168 active+clean; 746 MiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.61k op/s 2024-09-10T19:02:46.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:46 smithi018 ceph-mon[29336]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:02:46.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:46 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T19:02:46.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:46 smithi018 ceph-mon[29336]: osdmap e371: 8 total, 7 up, 6 in 2024-09-10T19:02:46.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:46 smithi018 ceph-mon[29336]: 6.18 scrub starts 2024-09-10T19:02:46.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:46 smithi018 ceph-mon[29336]: 6.18 scrub ok 2024-09-10T19:02:46.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:46 smithi018 ceph-mon[29336]: 5.9 scrub starts 2024-09-10T19:02:46.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:46 smithi018 ceph-mon[29336]: 5.9 scrub ok 2024-09-10T19:02:46.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:46 smithi103 ceph-mon[33163]: pgmap v808: 168 pgs: 168 active+clean; 746 MiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.61k op/s 2024-09-10T19:02:46.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:46 smithi103 ceph-mon[33163]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:02:46.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:46 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T19:02:46.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:46 smithi103 ceph-mon[33163]: osdmap e371: 8 total, 7 up, 6 in 2024-09-10T19:02:46.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:46 smithi103 ceph-mon[33163]: 6.18 scrub starts 2024-09-10T19:02:46.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:46 smithi103 ceph-mon[33163]: 6.18 scrub ok 2024-09-10T19:02:46.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:46 smithi103 ceph-mon[33163]: 5.9 scrub starts 2024-09-10T19:02:46.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:46 smithi103 ceph-mon[33163]: 5.9 scrub ok 2024-09-10T19:02:46.777 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:46.817 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:46.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:46.818 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:46.918 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:46.960 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:46.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:46.962 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:47.062 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:47.090 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:47.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:47.091 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:47.192 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:47.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:47.233 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:47.234 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:47.334 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:47.368 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:47.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:47.369 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:47.470 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:47.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:47.507 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:47.507 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:47.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:02:47 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:02:47.608 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:47.657 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:47.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:47.658 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:47.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:47 smithi018 ceph-mon[40196]: 1.3 scrub starts 2024-09-10T19:02:47.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:47 smithi018 ceph-mon[40196]: 1.3 scrub ok 2024-09-10T19:02:47.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:47 smithi018 ceph-mon[29336]: 1.3 scrub starts 2024-09-10T19:02:47.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:47 smithi018 ceph-mon[29336]: 1.3 scrub ok 2024-09-10T19:02:47.760 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:47.801 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:47.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:47.802 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:47.903 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:47.919 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:47 smithi103 ceph-mon[33163]: 1.3 scrub starts 2024-09-10T19:02:47.920 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:47 smithi103 ceph-mon[33163]: 1.3 scrub ok 2024-09-10T19:02:47.937 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:47.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:47.939 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:48.039 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:48.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:48.078 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:48.078 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:48.179 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:48.220 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:48.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:48.221 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:48.238 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T19:02:48.238 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T19:02:48.267 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T19:02:48.268 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T19:02:48.295 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T19:02:48.295 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T19:02:48.321 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:48.329 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T19:02:48.329 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T19:02:48.359 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T19:02:48.360 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T19:02:48.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:48.361 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:48.361 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:48.385 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T19:02:48.385 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T19:02:48.422 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T19:02:48.423 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T19:02:48.424 INFO:teuthology.orchestra.run.smithi018.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-10T19:02:48.462 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:48.500 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:48.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:48.501 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:48.552 INFO:teuthology.orchestra.run.smithi018.stderr:set osd.1 primary-affinity to 0.0101021 (86622) 2024-09-10T19:02:48.603 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:48.632 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:48.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:48.633 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:48.645 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:48 smithi018 ceph-mon[40196]: 4.15 scrub starts 2024-09-10T19:02:48.645 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:48 smithi018 ceph-mon[40196]: 4.15 scrub ok 2024-09-10T19:02:48.646 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:48 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:48.646 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:48 smithi018 ceph-mon[40196]: pgmap v810: 168 pgs: 168 active+clean; 357 MiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.79k op/s 2024-09-10T19:02:48.646 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:48 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/4065512454' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 0.010102103415535302}]: dispatch 2024-09-10T19:02:48.646 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:48 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3177403486' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:02:48.646 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:48 smithi018 ceph-mon[29336]: 4.15 scrub starts 2024-09-10T19:02:48.646 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:48 smithi018 ceph-mon[29336]: 4.15 scrub ok 2024-09-10T19:02:48.646 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:48 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:48.646 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:48 smithi018 ceph-mon[29336]: pgmap v810: 168 pgs: 168 active+clean; 357 MiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.79k op/s 2024-09-10T19:02:48.646 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:48 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/4065512454' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 0.010102103415535302}]: dispatch 2024-09-10T19:02:48.646 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:48 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3177403486' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:02:48.683 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T19:02:48.684 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T19:02:48.734 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:48.758 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:48.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:48.759 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:48.860 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:48.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:48.884 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:48.885 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:48.985 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:49.008 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:49.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:49.009 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:49.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:48 smithi103 ceph-mon[33163]: 4.15 scrub starts 2024-09-10T19:02:49.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:48 smithi103 ceph-mon[33163]: 4.15 scrub ok 2024-09-10T19:02:49.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:48 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:49.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:48 smithi103 ceph-mon[33163]: pgmap v810: 168 pgs: 168 active+clean; 357 MiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.79k op/s 2024-09-10T19:02:49.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:48 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/4065512454' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 0.010102103415535302}]: dispatch 2024-09-10T19:02:49.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:48 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3177403486' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:02:49.110 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:49.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:49.133 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:49.134 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:49.235 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:49.256 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:49.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:49.257 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:49.358 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:49.359 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-10T19:02:49.360 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-10T19:02:49.380 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:49.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:49.381 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:49.404 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-10T19:02:49.483 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:49.504 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:49.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:49.505 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:49.606 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:49.644 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:49.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:49.645 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:49.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:49 smithi018 ceph-mon[29336]: 4.0 scrub starts 2024-09-10T19:02:49.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:49 smithi018 ceph-mon[29336]: 4.0 scrub ok 2024-09-10T19:02:49.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:49 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/4065512454' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 0.010102103415535302}]': finished 2024-09-10T19:02:49.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:49 smithi018 ceph-mon[29336]: osdmap e372: 8 total, 7 up, 6 in 2024-09-10T19:02:49.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:49 smithi018 ceph-mon[40196]: 4.0 scrub starts 2024-09-10T19:02:49.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:49 smithi018 ceph-mon[40196]: 4.0 scrub ok 2024-09-10T19:02:49.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:49 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/4065512454' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 0.010102103415535302}]': finished 2024-09-10T19:02:49.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:49 smithi018 ceph-mon[40196]: osdmap e372: 8 total, 7 up, 6 in 2024-09-10T19:02:49.746 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:49.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:49.782 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:49.783 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:49.883 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:49.916 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:49.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:49.917 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:49.947 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:02:50.018 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:50.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:49 smithi103 ceph-mon[33163]: 4.0 scrub starts 2024-09-10T19:02:50.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:49 smithi103 ceph-mon[33163]: 4.0 scrub ok 2024-09-10T19:02:50.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:49 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/4065512454' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 0.010102103415535302}]': finished 2024-09-10T19:02:50.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:49 smithi103 ceph-mon[33163]: osdmap e372: 8 total, 7 up, 6 in 2024-09-10T19:02:50.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:50.053 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:50.053 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:50.154 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:50.185 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:50.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:50.186 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:50.287 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:50.324 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:50.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:50.325 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:50.426 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:50.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:50.459 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:50.459 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:50.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:02:50 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:02:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:02:50.560 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:50.587 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:50.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:50.588 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:50.689 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:50.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:50 smithi018 ceph-mon[40196]: 6.d scrub starts 2024-09-10T19:02:50.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:50 smithi018 ceph-mon[40196]: 6.d scrub ok 2024-09-10T19:02:50.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:50 smithi018 ceph-mon[40196]: pgmap v812: 168 pgs: 5 peering, 163 active+clean; 262 MiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 682 B/s rd, 0 B/s wr, 1.54k op/s 2024-09-10T19:02:50.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:50 smithi018 ceph-mon[40196]: osdmap e373: 8 total, 7 up, 6 in 2024-09-10T19:02:50.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:50 smithi018 ceph-mon[40196]: Health check failed: Reduced data availability: 4 pgs inactive, 4 pgs peering (PG_AVAILABILITY) 2024-09-10T19:02:50.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:50 smithi018 ceph-mon[29336]: 6.d scrub starts 2024-09-10T19:02:50.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:50 smithi018 ceph-mon[29336]: 6.d scrub ok 2024-09-10T19:02:50.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:50 smithi018 ceph-mon[29336]: pgmap v812: 168 pgs: 5 peering, 163 active+clean; 262 MiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 682 B/s rd, 0 B/s wr, 1.54k op/s 2024-09-10T19:02:50.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:50 smithi018 ceph-mon[29336]: osdmap e373: 8 total, 7 up, 6 in 2024-09-10T19:02:50.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:50 smithi018 ceph-mon[29336]: Health check failed: Reduced data availability: 4 pgs inactive, 4 pgs peering (PG_AVAILABILITY) 2024-09-10T19:02:50.712 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:50.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:50.713 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:50.733 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Removed 138655 objects 2024-09-10T19:02:50.733 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Clean up completed and total clean up time :87.1117 2024-09-10T19:02:50.814 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:50.837 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:50.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:50.838 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:50.939 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:50.961 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:50.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:50.962 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:51.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:50 smithi103 ceph-mon[33163]: 6.d scrub starts 2024-09-10T19:02:51.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:50 smithi103 ceph-mon[33163]: 6.d scrub ok 2024-09-10T19:02:51.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:50 smithi103 ceph-mon[33163]: pgmap v812: 168 pgs: 5 peering, 163 active+clean; 262 MiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 682 B/s rd, 0 B/s wr, 1.54k op/s 2024-09-10T19:02:51.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:50 smithi103 ceph-mon[33163]: osdmap e373: 8 total, 7 up, 6 in 2024-09-10T19:02:51.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:50 smithi103 ceph-mon[33163]: Health check failed: Reduced data availability: 4 pgs inactive, 4 pgs peering (PG_AVAILABILITY) 2024-09-10T19:02:51.063 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:51.086 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:51.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:51.087 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:51.137 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:02:50 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:02:50] "GET /metrics HTTP/1.1" 200 209529 "" "Prometheus/2.43.0" 2024-09-10T19:02:51.188 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:51.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:51.213 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:51.213 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:51.314 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:51.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:51.338 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:51.338 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:51.361 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T19:02:51.440 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:51.462 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:51.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:51.463 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:51.564 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:51.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:51.591 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:51.591 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:51.692 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:51.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:51 smithi018 ceph-mon[29336]: 4.10 scrub starts 2024-09-10T19:02:51.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:51 smithi018 ceph-mon[29336]: 4.10 scrub ok 2024-09-10T19:02:51.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:51 smithi018 ceph-mon[29336]: 5.a scrub starts 2024-09-10T19:02:51.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:51 smithi018 ceph-mon[29336]: 5.a scrub ok 2024-09-10T19:02:51.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:51 smithi018 ceph-mon[40196]: 4.10 scrub starts 2024-09-10T19:02:51.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:51 smithi018 ceph-mon[40196]: 4.10 scrub ok 2024-09-10T19:02:51.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:51 smithi018 ceph-mon[40196]: 5.a scrub starts 2024-09-10T19:02:51.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:51 smithi018 ceph-mon[40196]: 5.a scrub ok 2024-09-10T19:02:51.715 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:51.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:51.716 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:51.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:51 smithi103 ceph-mon[33163]: 4.10 scrub starts 2024-09-10T19:02:51.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:51 smithi103 ceph-mon[33163]: 4.10 scrub ok 2024-09-10T19:02:51.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:51 smithi103 ceph-mon[33163]: 5.a scrub starts 2024-09-10T19:02:51.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:51 smithi103 ceph-mon[33163]: 5.a scrub ok 2024-09-10T19:02:51.817 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:51.842 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:51.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:51.843 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:51.848 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:02:51.944 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:51.971 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:51.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:51.972 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:52.073 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:52.096 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:52.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:52.097 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:52.197 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:52.224 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:52.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:52.225 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:52.273 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-10T19:02:52.326 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:52.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:52.349 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:52.349 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:52.450 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:52.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:52.473 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:52.473 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:52.524 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:02:52.559 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is unset 2024-09-10T19:02:52.574 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:52.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:52.597 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:52.597 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:52.698 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:52.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:52.722 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:52.722 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:52.823 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:52.844 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:52.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:52.849 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:52.856 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:52 smithi103 ceph-mon[33163]: pgmap v814: 168 pgs: 5 peering, 163 active+clean; 262 MiB data, 6.8 GiB used, 530 GiB / 536 GiB avail; 511 B/s rd, 0 B/s wr, 1.29k op/s 2024-09-10T19:02:52.857 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:52 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3874640476' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:02:52.857 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:52 smithi103 ceph-mon[33163]: 2.6 scrub starts 2024-09-10T19:02:52.857 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:52 smithi103 ceph-mon[33163]: 2.6 scrub ok 2024-09-10T19:02:52.900 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:52 smithi018 ceph-mon[29336]: pgmap v814: 168 pgs: 5 peering, 163 active+clean; 262 MiB data, 6.8 GiB used, 530 GiB / 536 GiB avail; 511 B/s rd, 0 B/s wr, 1.29k op/s 2024-09-10T19:02:52.900 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:52 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3874640476' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:02:52.900 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:52 smithi018 ceph-mon[29336]: 2.6 scrub starts 2024-09-10T19:02:52.900 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:52 smithi018 ceph-mon[29336]: 2.6 scrub ok 2024-09-10T19:02:52.900 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:52 smithi018 ceph-mon[40196]: pgmap v814: 168 pgs: 5 peering, 163 active+clean; 262 MiB data, 6.8 GiB used, 530 GiB / 536 GiB avail; 511 B/s rd, 0 B/s wr, 1.29k op/s 2024-09-10T19:02:52.900 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:52 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3874640476' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:02:52.901 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:52 smithi018 ceph-mon[40196]: 2.6 scrub starts 2024-09-10T19:02:52.901 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:52 smithi018 ceph-mon[40196]: 2.6 scrub ok 2024-09-10T19:02:52.950 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:52.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:52.972 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:52.972 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:53.073 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:53.094 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:53.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:53.095 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:53.196 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:53.221 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:53.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:53.222 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:53.323 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:53.344 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:53.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:53.346 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:53.446 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:53.470 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:53.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:53.471 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:53.572 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:53.592 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:53.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:53.593 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:53.694 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:53.716 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:53.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:53.717 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:53.817 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:53.823 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:53 smithi103 ceph-mon[33163]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T19:02:53.823 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:53 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3874640476' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T19:02:53.823 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:53 smithi103 ceph-mon[33163]: osdmap e374: 8 total, 7 up, 6 in 2024-09-10T19:02:53.823 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:53 smithi103 ceph-mon[33163]: 5.e scrub starts 2024-09-10T19:02:53.823 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:53 smithi103 ceph-mon[33163]: 5.e scrub ok 2024-09-10T19:02:53.839 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:53.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:53.840 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:53.921 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:53 smithi018 ceph-mon[29336]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T19:02:53.921 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:53 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3874640476' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T19:02:53.921 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:53 smithi018 ceph-mon[29336]: osdmap e374: 8 total, 7 up, 6 in 2024-09-10T19:02:53.921 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:53 smithi018 ceph-mon[29336]: 5.e scrub starts 2024-09-10T19:02:53.921 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:53 smithi018 ceph-mon[29336]: 5.e scrub ok 2024-09-10T19:02:53.921 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:53 smithi018 ceph-mon[40196]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T19:02:53.922 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:53 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3874640476' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T19:02:53.922 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:53 smithi018 ceph-mon[40196]: osdmap e374: 8 total, 7 up, 6 in 2024-09-10T19:02:53.922 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:53 smithi018 ceph-mon[40196]: 5.e scrub starts 2024-09-10T19:02:53.922 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:53 smithi018 ceph-mon[40196]: 5.e scrub ok 2024-09-10T19:02:53.940 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:53.962 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:53.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:53.963 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:54.028 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T19:02:54.028 INFO:teuthology.orchestra.run.smithi103.stdout: "size": 20, 2024-09-10T19:02:54.028 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 600, 2024-09-10T19:02:54.028 INFO:teuthology.orchestra.run.smithi103.stdout: "ops": [ 2024-09-10T19:02:54.028 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.028 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15354.0:178091 9.7 9:ea121bd7:::benchmark_data_smithi018_133594_object41339:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e285)", 2024-09-10T19:02:54.029 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:58:05.137375+0000", 2024-09-10T19:02:54.029 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 288.88831038500001, 2024-09-10T19:02:54.029 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.04106915, 2024-09-10T19:02:54.029 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:54.029 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "reached pg", 2024-09-10T19:02:54.029 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:54.029 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15354", 2024-09-10T19:02:54.029 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:02:54.029 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 178091 2024-09-10T19:02:54.029 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.029 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:54.029 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.030 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:54.030 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.137375+0000", 2024-09-10T19:02:54.030 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.030 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.030 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.030 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:54.030 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.137375+0000", 2024-09-10T19:02:54.030 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.030 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.030 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.030 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:54.030 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.137374+0000", 2024-09-10T19:02:54.030 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T19:02:54.031 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.031 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.031 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:54.031 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.137380+0000", 2024-09-10T19:02:54.031 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.4720000000000003e-06 2024-09-10T19:02:54.031 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.031 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.031 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:54.031 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.137389+0000", 2024-09-10T19:02:54.031 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.4790000000000002e-06 2024-09-10T19:02:54.031 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.031 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.031 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:54.032 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.137393+0000", 2024-09-10T19:02:54.032 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.9790000000000004e-06 2024-09-10T19:02:54.032 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.032 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.032 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.032 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.137424+0000", 2024-09-10T19:02:54.032 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.1068999999999997e-05 2024-09-10T19:02:54.032 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.032 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.032 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:02:54.032 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.137510+0000", 2024-09-10T19:02:54.032 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.5883000000000004e-05 2024-09-10T19:02:54.032 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.033 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.033 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 1,3,7", 2024-09-10T19:02:54.033 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.137592+0000", 2024-09-10T19:02:54.033 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.1625999999999994e-05 2024-09-10T19:02:54.033 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.033 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.033 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:02:54.033 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.140085+0000", 2024-09-10T19:02:54.033 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.002493168 2024-09-10T19:02:54.033 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.033 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.033 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:54.033 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.140339+0000", 2024-09-10T19:02:54.034 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000254035 2024-09-10T19:02:54.034 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.034 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.034 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:54.034 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.143554+0000", 2024-09-10T19:02:54.034 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.003215075 2024-09-10T19:02:54.034 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.034 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.034 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.034 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.148627+0000", 2024-09-10T19:02:54.034 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0050729 2024-09-10T19:02:54.034 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.035 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.035 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:54.035 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.148637+0000", 2024-09-10T19:02:54.035 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0052e-05 2024-09-10T19:02:54.035 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.035 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.035 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.035 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.178360+0000", 2024-09-10T19:02:54.035 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0297230610000001 2024-09-10T19:02:54.035 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.036 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.036 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:54.036 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.178444+0000", 2024-09-10T19:02:54.036 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.4282000000000004e-05 2024-09-10T19:02:54.036 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.036 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:54.036 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.036 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.036 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.036 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15354.0:178142 9.7 9:ecf42d8f:::benchmark_data_smithi018_133594_object41390:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e286)", 2024-09-10T19:02:54.036 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:58:05.245258+0000", 2024-09-10T19:02:54.036 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 288.780427691, 2024-09-10T19:02:54.037 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.95601217100000002, 2024-09-10T19:02:54.037 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:54.037 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:02:54.037 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:54.037 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15354", 2024-09-10T19:02:54.037 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:02:54.037 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 178142 2024-09-10T19:02:54.037 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.037 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:54.037 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.037 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:54.037 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.245258+0000", 2024-09-10T19:02:54.037 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.038 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.038 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.038 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:54.038 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.245258+0000", 2024-09-10T19:02:54.038 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.038 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.038 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.038 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:54.038 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.245257+0000", 2024-09-10T19:02:54.038 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T19:02:54.038 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.038 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.038 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:54.039 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.245262+0000", 2024-09-10T19:02:54.039 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.395e-06 2024-09-10T19:02:54.039 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.039 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.039 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:54.039 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.245276+0000", 2024-09-10T19:02:54.039 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.4122e-05 2024-09-10T19:02:54.039 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.039 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.039 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:54.039 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.245283+0000", 2024-09-10T19:02:54.039 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.3899999999999998e-06 2024-09-10T19:02:54.039 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.039 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.040 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.040 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.245359+0000", 2024-09-10T19:02:54.040 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.5957999999999993e-05 2024-09-10T19:02:54.040 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.040 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.040 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:54.040 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.245367+0000", 2024-09-10T19:02:54.040 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.8380000000000008e-06 2024-09-10T19:02:54.040 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.040 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.040 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.040 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.178478+0000", 2024-09-10T19:02:54.040 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.93311012599999998 2024-09-10T19:02:54.041 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.041 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.041 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:02:54.041 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.178559+0000", 2024-09-10T19:02:54.041 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.1964000000000003e-05 2024-09-10T19:02:54.041 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.041 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.041 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 1,3", 2024-09-10T19:02:54.041 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.178632+0000", 2024-09-10T19:02:54.041 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.2272000000000002e-05 2024-09-10T19:02:54.041 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.041 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.041 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:02:54.042 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.185456+0000", 2024-09-10T19:02:54.042 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0068239199999999998 2024-09-10T19:02:54.042 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.042 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.042 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:54.042 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.201040+0000", 2024-09-10T19:02:54.042 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.015584147 2024-09-10T19:02:54.042 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.042 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.042 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:54.042 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.201221+0000", 2024-09-10T19:02:54.042 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00018072800000000001 2024-09-10T19:02:54.043 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.043 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.043 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:02:54.043 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.201248+0000", 2024-09-10T19:02:54.043 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.7753999999999999e-05 2024-09-10T19:02:54.043 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.043 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.043 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:54.043 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.201270+0000", 2024-09-10T19:02:54.044 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.1594999999999999e-05 2024-09-10T19:02:54.044 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.044 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:54.044 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.044 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.045 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.045 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15354.0:178145 9.7 9:ed12a7f2:::benchmark_data_smithi018_133594_object41393:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e286)", 2024-09-10T19:02:54.045 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:58:05.248111+0000", 2024-09-10T19:02:54.045 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 288.77757443299998, 2024-09-10T19:02:54.045 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.95324090299999997, 2024-09-10T19:02:54.045 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:54.045 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:02:54.045 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:54.045 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15354", 2024-09-10T19:02:54.045 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:02:54.045 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 178145 2024-09-10T19:02:54.045 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.045 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:54.046 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.046 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:54.046 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.248111+0000", 2024-09-10T19:02:54.046 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.046 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.046 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.046 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:54.046 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.248111+0000", 2024-09-10T19:02:54.046 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.046 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.046 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.046 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:54.046 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.248110+0000", 2024-09-10T19:02:54.047 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T19:02:54.047 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.047 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.047 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:54.047 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.248115+0000", 2024-09-10T19:02:54.047 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.3229999999999997e-06 2024-09-10T19:02:54.047 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.047 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.047 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:54.047 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.248125+0000", 2024-09-10T19:02:54.047 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0122000000000001e-05 2024-09-10T19:02:54.047 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.047 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.048 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:54.048 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.248130+0000", 2024-09-10T19:02:54.048 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.3340000000000003e-06 2024-09-10T19:02:54.048 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.048 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.048 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.048 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.248171+0000", 2024-09-10T19:02:54.048 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.1183000000000002e-05 2024-09-10T19:02:54.048 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.048 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.048 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:54.048 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.248179+0000", 2024-09-10T19:02:54.048 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.3599999999999996e-06 2024-09-10T19:02:54.049 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.049 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.049 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.049 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.180559+0000", 2024-09-10T19:02:54.049 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.932380125 2024-09-10T19:02:54.049 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.049 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.049 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:02:54.049 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.180641+0000", 2024-09-10T19:02:54.049 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.1397999999999996e-05 2024-09-10T19:02:54.049 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.049 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.049 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 1,3", 2024-09-10T19:02:54.049 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.180733+0000", 2024-09-10T19:02:54.050 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.2374999999999996e-05 2024-09-10T19:02:54.050 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.050 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.050 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:02:54.050 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.188068+0000", 2024-09-10T19:02:54.050 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0073351120000000004 2024-09-10T19:02:54.050 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.050 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.050 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:54.050 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.201094+0000", 2024-09-10T19:02:54.050 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.013026072 2024-09-10T19:02:54.050 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.050 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.050 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:54.051 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.201298+0000", 2024-09-10T19:02:54.051 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00020397300000000001 2024-09-10T19:02:54.051 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.051 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.051 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:02:54.051 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.201325+0000", 2024-09-10T19:02:54.051 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.7114000000000002e-05 2024-09-10T19:02:54.051 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.051 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.051 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:54.051 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.201352+0000", 2024-09-10T19:02:54.051 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.6427e-05 2024-09-10T19:02:54.052 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.052 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:54.052 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.052 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.052 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.052 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15354.0:178206 9.7 9:eb990a99:::benchmark_data_smithi018_133594_object41454:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e286)", 2024-09-10T19:02:54.052 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:58:05.293610+0000", 2024-09-10T19:02:54.052 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 288.73207551000002, 2024-09-10T19:02:54.053 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.90805035700000003, 2024-09-10T19:02:54.053 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:54.053 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:02:54.053 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:54.053 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15354", 2024-09-10T19:02:54.053 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:02:54.053 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 178206 2024-09-10T19:02:54.053 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.053 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:54.053 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.053 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:54.053 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.293610+0000", 2024-09-10T19:02:54.053 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.054 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.054 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.054 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:54.054 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.293610+0000", 2024-09-10T19:02:54.054 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.054 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.054 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.054 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:54.054 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.293608+0000", 2024-09-10T19:02:54.054 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T19:02:54.054 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.054 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.054 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:54.055 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.293616+0000", 2024-09-10T19:02:54.055 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.3649999999999998e-06 2024-09-10T19:02:54.055 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.055 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.055 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:54.055 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.293635+0000", 2024-09-10T19:02:54.055 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.9449e-05 2024-09-10T19:02:54.055 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.055 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.055 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:54.055 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.293642+0000", 2024-09-10T19:02:54.055 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.7610000000000002e-06 2024-09-10T19:02:54.055 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.056 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.056 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.056 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.293697+0000", 2024-09-10T19:02:54.056 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.4722999999999998e-05 2024-09-10T19:02:54.056 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.056 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.056 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:54.056 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.293706+0000", 2024-09-10T19:02:54.056 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.5969999999999998e-06 2024-09-10T19:02:54.056 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.056 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.056 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.056 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.182434+0000", 2024-09-10T19:02:54.057 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.88872758399999996 2024-09-10T19:02:54.057 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.057 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.057 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:02:54.057 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.182501+0000", 2024-09-10T19:02:54.057 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.7255999999999994e-05 2024-09-10T19:02:54.057 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.057 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.057 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 1,3", 2024-09-10T19:02:54.057 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.182612+0000", 2024-09-10T19:02:54.057 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000110616 2024-09-10T19:02:54.057 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.057 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.058 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:02:54.058 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.188076+0000", 2024-09-10T19:02:54.058 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0054646870000000002 2024-09-10T19:02:54.058 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.058 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.058 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:54.058 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.201148+0000", 2024-09-10T19:02:54.058 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.013071891 2024-09-10T19:02:54.058 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.058 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.058 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:54.058 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.201578+0000", 2024-09-10T19:02:54.058 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00043022999999999998 2024-09-10T19:02:54.059 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.059 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.059 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:02:54.059 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.201622+0000", 2024-09-10T19:02:54.059 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.3603000000000002e-05 2024-09-10T19:02:54.059 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.059 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.059 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:54.059 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.201660+0000", 2024-09-10T19:02:54.059 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.8192999999999998e-05 2024-09-10T19:02:54.059 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.059 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:54.060 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.060 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.060 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.060 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15354.0:178238 9.7 9:e7168323:::benchmark_data_smithi018_133594_object41486:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e286)", 2024-09-10T19:02:54.060 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:58:05.323378+0000", 2024-09-10T19:02:54.060 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 288.70230765299999, 2024-09-10T19:02:54.060 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.88830853499999995, 2024-09-10T19:02:54.060 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:54.060 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:02:54.060 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:54.060 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15354", 2024-09-10T19:02:54.060 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:02:54.060 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 178238 2024-09-10T19:02:54.061 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.061 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:54.061 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.061 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:54.061 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.323378+0000", 2024-09-10T19:02:54.061 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.061 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.061 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.061 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:54.061 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.323378+0000", 2024-09-10T19:02:54.061 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.061 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.061 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.062 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:54.062 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.323377+0000", 2024-09-10T19:02:54.062 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T19:02:54.062 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.062 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.062 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:54.062 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.323381+0000", 2024-09-10T19:02:54.062 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.5179999999999998e-06 2024-09-10T19:02:54.062 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.062 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.062 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:54.062 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.323390+0000", 2024-09-10T19:02:54.063 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.149e-06 2024-09-10T19:02:54.063 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.063 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.063 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:54.063 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.323395+0000", 2024-09-10T19:02:54.063 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.5190000000000003e-06 2024-09-10T19:02:54.063 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.063 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.063 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.063 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.323437+0000", 2024-09-10T19:02:54.063 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.1876999999999999e-05 2024-09-10T19:02:54.063 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.064 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.064 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:54.064 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.323442+0000", 2024-09-10T19:02:54.064 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.6030000000000004e-06 2024-09-10T19:02:54.064 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.064 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.064 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.064 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.183807+0000", 2024-09-10T19:02:54.064 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.86036505399999996 2024-09-10T19:02:54.064 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.064 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.064 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:02:54.064 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.183886+0000", 2024-09-10T19:02:54.064 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.8207999999999994e-05 2024-09-10T19:02:54.065 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.065 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.065 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 1,3", 2024-09-10T19:02:54.065 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.183961+0000", 2024-09-10T19:02:54.065 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.5685999999999993e-05 2024-09-10T19:02:54.065 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.065 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.065 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:02:54.065 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.191042+0000", 2024-09-10T19:02:54.065 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0070804580000000004 2024-09-10T19:02:54.065 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.065 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.065 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:54.065 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.201182+0000", 2024-09-10T19:02:54.066 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.010140613 2024-09-10T19:02:54.066 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.066 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.066 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:54.066 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.211622+0000", 2024-09-10T19:02:54.066 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.010439201 2024-09-10T19:02:54.066 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.066 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.066 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:02:54.066 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.211654+0000", 2024-09-10T19:02:54.066 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.2650000000000001e-05 2024-09-10T19:02:54.066 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.066 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.067 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:54.067 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.211686+0000", 2024-09-10T19:02:54.067 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.2048000000000002e-05 2024-09-10T19:02:54.067 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.067 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:54.067 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.067 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.067 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.067 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15573.0:86363 10.9 10.14e8f969 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e338)", 2024-09-10T19:02:54.067 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:00:41.512921+0000", 2024-09-10T19:02:54.067 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 132.51276425200001, 2024-09-10T19:02:54.067 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.99603365899999996, 2024-09-10T19:02:54.067 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:54.067 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "reached pg", 2024-09-10T19:02:54.068 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:54.068 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15573", 2024-09-10T19:02:54.068 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:02:54.068 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 86363 2024-09-10T19:02:54.068 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.068 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:54.068 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.068 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:54.068 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.512921+0000", 2024-09-10T19:02:54.068 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.068 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.068 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.068 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:54.068 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.512921+0000", 2024-09-10T19:02:54.068 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.069 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.069 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.069 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:54.069 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.512919+0000", 2024-09-10T19:02:54.069 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999981 2024-09-10T19:02:54.069 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.069 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.069 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:54.069 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.512988+0000", 2024-09-10T19:02:54.069 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.8990000000000002e-05 2024-09-10T19:02:54.069 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.069 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.069 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:54.070 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.512999+0000", 2024-09-10T19:02:54.070 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.057e-05 2024-09-10T19:02:54.070 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.070 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.070 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:54.070 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.513005+0000", 2024-09-10T19:02:54.070 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.9410000000000004e-06 2024-09-10T19:02:54.070 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.070 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.070 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.070 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.519550+0000", 2024-09-10T19:02:54.070 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0065450090000000001 2024-09-10T19:02:54.070 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.070 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.071 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:54.071 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.519555+0000", 2024-09-10T19:02:54.071 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.1240000000000004e-06 2024-09-10T19:02:54.071 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.071 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.071 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.071 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:42.508945+0000", 2024-09-10T19:02:54.071 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.98938961700000005 2024-09-10T19:02:54.071 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.071 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.071 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:54.071 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:42.508955+0000", 2024-09-10T19:02:54.071 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0091e-05 2024-09-10T19:02:54.071 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.072 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:54.072 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.072 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.072 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.072 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15573.0:86373 10.9 10.aea01c89 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e338)", 2024-09-10T19:02:54.072 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:00:41.513019+0000", 2024-09-10T19:02:54.072 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 132.51266666500001, 2024-09-10T19:02:54.072 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.99596813100000003, 2024-09-10T19:02:54.072 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:54.072 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "reached pg", 2024-09-10T19:02:54.072 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:54.072 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15573", 2024-09-10T19:02:54.073 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:02:54.073 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 86373 2024-09-10T19:02:54.073 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.073 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:54.073 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.073 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:54.073 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.513019+0000", 2024-09-10T19:02:54.073 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.073 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.073 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.074 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:54.074 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.513019+0000", 2024-09-10T19:02:54.074 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.074 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.074 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.074 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:54.074 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.513018+0000", 2024-09-10T19:02:54.074 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999995 2024-09-10T19:02:54.074 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.074 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.074 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:54.074 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.513083+0000", 2024-09-10T19:02:54.074 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.4523999999999994e-05 2024-09-10T19:02:54.075 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.075 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.075 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:54.075 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.513091+0000", 2024-09-10T19:02:54.075 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.4530000000000002e-06 2024-09-10T19:02:54.075 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.075 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.075 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:54.075 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.513096+0000", 2024-09-10T19:02:54.075 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.215e-06 2024-09-10T19:02:54.075 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.075 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.076 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.076 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.519575+0000", 2024-09-10T19:02:54.076 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.006478518 2024-09-10T19:02:54.076 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.076 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.076 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:54.076 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.519579+0000", 2024-09-10T19:02:54.076 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.1819999999999999e-06 2024-09-10T19:02:54.076 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.076 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.076 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.076 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:42.508980+0000", 2024-09-10T19:02:54.076 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.98940066900000001 2024-09-10T19:02:54.077 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.077 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.077 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:54.077 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:42.508987+0000", 2024-09-10T19:02:54.077 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.0720000000000002e-06 2024-09-10T19:02:54.077 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.077 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:54.077 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.077 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.077 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.077 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15573.0:86376 10.9 10.5396bf79 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e338)", 2024-09-10T19:02:54.077 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:00:41.513102+0000", 2024-09-10T19:02:54.077 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 132.51258327599999, 2024-09-10T19:02:54.078 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.99591294200000002, 2024-09-10T19:02:54.078 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:54.078 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "reached pg", 2024-09-10T19:02:54.078 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:54.078 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15573", 2024-09-10T19:02:54.078 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:02:54.078 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 86376 2024-09-10T19:02:54.078 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.078 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:54.078 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.078 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:54.078 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.513102+0000", 2024-09-10T19:02:54.078 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.078 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.079 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.079 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:54.079 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.513102+0000", 2024-09-10T19:02:54.079 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.079 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.079 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.079 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:54.079 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.513102+0000", 2024-09-10T19:02:54.079 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999995 2024-09-10T19:02:54.079 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.079 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.079 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:54.079 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.513156+0000", 2024-09-10T19:02:54.080 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.4129999999999998e-05 2024-09-10T19:02:54.080 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.080 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.080 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:54.080 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.513163+0000", 2024-09-10T19:02:54.080 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.8450000000000003e-06 2024-09-10T19:02:54.080 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.080 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.081 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:54.081 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.513169+0000", 2024-09-10T19:02:54.081 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.7209999999999999e-06 2024-09-10T19:02:54.081 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.081 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.081 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.081 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.519598+0000", 2024-09-10T19:02:54.081 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0064288799999999997 2024-09-10T19:02:54.081 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.081 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.081 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:54.081 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.519602+0000", 2024-09-10T19:02:54.082 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.755e-06 2024-09-10T19:02:54.082 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.082 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.082 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.082 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:42.509005+0000", 2024-09-10T19:02:54.082 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.98940315400000001 2024-09-10T19:02:54.082 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.082 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.082 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:54.082 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:42.509015+0000", 2024-09-10T19:02:54.082 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0067e-05 2024-09-10T19:02:54.082 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.082 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:54.083 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.083 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.083 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.083 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15573.0:86398 10.9 10.40eac979 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-10T19:02:54.083 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:00:41.524392+0000", 2024-09-10T19:02:54.083 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 132.50129345400001, 2024-09-10T19:02:54.083 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.98464817400000004, 2024-09-10T19:02:54.083 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:54.083 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "reached pg", 2024-09-10T19:02:54.083 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:54.083 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15573", 2024-09-10T19:02:54.083 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:02:54.083 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 86398 2024-09-10T19:02:54.084 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.084 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:54.084 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.084 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:54.084 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.524392+0000", 2024-09-10T19:02:54.084 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.084 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.084 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.084 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:54.084 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.524392+0000", 2024-09-10T19:02:54.084 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.084 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.085 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.085 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:54.085 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.524391+0000", 2024-09-10T19:02:54.085 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T19:02:54.085 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.085 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.085 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:54.085 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.524444+0000", 2024-09-10T19:02:54.085 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.3287e-05 2024-09-10T19:02:54.086 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.086 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.086 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:54.086 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.524454+0000", 2024-09-10T19:02:54.086 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0283e-05 2024-09-10T19:02:54.086 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.086 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.086 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:54.086 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.524460+0000", 2024-09-10T19:02:54.086 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.4480000000000002e-06 2024-09-10T19:02:54.086 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.086 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.086 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.087 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.524500+0000", 2024-09-10T19:02:54.087 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.9839000000000003e-05 2024-09-10T19:02:54.087 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.087 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.087 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:54.087 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.524506+0000", 2024-09-10T19:02:54.087 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.4840000000000001e-06 2024-09-10T19:02:54.087 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.087 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.087 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.087 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:42.509032+0000", 2024-09-10T19:02:54.087 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.98452615799999998 2024-09-10T19:02:54.087 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.088 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.088 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:54.088 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:42.509040+0000", 2024-09-10T19:02:54.088 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.7309999999999993e-06 2024-09-10T19:02:54.088 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.088 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:54.088 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.088 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.088 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.088 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15573.0:86426 10.9 10.2941d579 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-10T19:02:54.088 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:00:41.551544+0000", 2024-09-10T19:02:54.088 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 132.47414091499999, 2024-09-10T19:02:54.088 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.95752248500000003, 2024-09-10T19:02:54.089 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:54.089 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "reached pg", 2024-09-10T19:02:54.089 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:54.089 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15573", 2024-09-10T19:02:54.089 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:02:54.089 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 86426 2024-09-10T19:02:54.089 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.089 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:54.089 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.089 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:54.089 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.551544+0000", 2024-09-10T19:02:54.089 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.090 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.090 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.090 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:54.090 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.551544+0000", 2024-09-10T19:02:54.090 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.090 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.090 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.090 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:54.090 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.551543+0000", 2024-09-10T19:02:54.090 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T19:02:54.090 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.090 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.090 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:54.091 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.551594+0000", 2024-09-10T19:02:54.091 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.02e-05 2024-09-10T19:02:54.091 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.091 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.091 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:54.091 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.551607+0000", 2024-09-10T19:02:54.091 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.3236e-05 2024-09-10T19:02:54.091 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.091 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.091 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:54.091 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.551612+0000", 2024-09-10T19:02:54.091 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.8470000000000003e-06 2024-09-10T19:02:54.091 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.092 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.092 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.092 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.551658+0000", 2024-09-10T19:02:54.092 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.6437999999999998e-05 2024-09-10T19:02:54.092 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.092 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.092 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:54.092 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.551679+0000", 2024-09-10T19:02:54.092 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.0531e-05 2024-09-10T19:02:54.092 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.092 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.092 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.092 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:42.509060+0000", 2024-09-10T19:02:54.093 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.95738168199999996 2024-09-10T19:02:54.093 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.093 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.093 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:54.093 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:42.509067+0000", 2024-09-10T19:02:54.093 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.7129999999999999e-06 2024-09-10T19:02:54.093 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.093 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:54.093 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.093 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.093 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.093 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15573.0:86433 10.9 10.60a2e869 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-10T19:02:54.094 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:00:41.555910+0000", 2024-09-10T19:02:54.094 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 132.46977559999999, 2024-09-10T19:02:54.094 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.95318143700000002, 2024-09-10T19:02:54.094 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:54.094 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "reached pg", 2024-09-10T19:02:54.094 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:54.094 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15573", 2024-09-10T19:02:54.094 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:02:54.094 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 86433 2024-09-10T19:02:54.094 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.094 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:54.094 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.094 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:54.094 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.555910+0000", 2024-09-10T19:02:54.095 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.095 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.095 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.095 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:54.095 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.555910+0000", 2024-09-10T19:02:54.095 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.095 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.095 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.095 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:54.095 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.555909+0000", 2024-09-10T19:02:54.095 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T19:02:54.095 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.095 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.095 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:54.096 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.555961+0000", 2024-09-10T19:02:54.096 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.1706000000000003e-05 2024-09-10T19:02:54.096 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.096 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.096 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:54.096 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.555969+0000", 2024-09-10T19:02:54.096 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.4619999999999996e-06 2024-09-10T19:02:54.096 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.096 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.096 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:54.096 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.555973+0000", 2024-09-10T19:02:54.096 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.07e-06 2024-09-10T19:02:54.096 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.097 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.097 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.097 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.556006+0000", 2024-09-10T19:02:54.097 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.2369000000000001e-05 2024-09-10T19:02:54.097 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.097 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.097 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:54.097 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.556010+0000", 2024-09-10T19:02:54.097 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.4290000000000003e-06 2024-09-10T19:02:54.097 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.097 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.097 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.097 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:42.509084+0000", 2024-09-10T19:02:54.098 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.95307423000000002 2024-09-10T19:02:54.098 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.098 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.098 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:54.098 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:42.509091+0000", 2024-09-10T19:02:54.098 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.9800000000000001e-06 2024-09-10T19:02:54.098 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.098 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:54.098 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.098 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.099 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.099 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15573.0:86445 10.9 10.1be393f9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-10T19:02:54.099 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:00:41.574133+0000", 2024-09-10T19:02:54.099 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 132.45155287700001, 2024-09-10T19:02:54.099 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.93648039400000005, 2024-09-10T19:02:54.099 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:54.099 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "reached pg", 2024-09-10T19:02:54.099 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:54.099 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15573", 2024-09-10T19:02:54.099 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:02:54.099 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 86445 2024-09-10T19:02:54.099 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.099 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:54.100 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.100 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:54.100 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.574133+0000", 2024-09-10T19:02:54.100 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.100 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.100 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.100 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:54.100 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.574133+0000", 2024-09-10T19:02:54.100 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.100 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.100 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.100 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:54.100 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.574132+0000", 2024-09-10T19:02:54.101 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T19:02:54.101 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.101 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.101 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:54.101 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.574189+0000", 2024-09-10T19:02:54.101 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.7184e-05 2024-09-10T19:02:54.101 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.101 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.101 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:54.101 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.574197+0000", 2024-09-10T19:02:54.101 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.8329999999999994e-06 2024-09-10T19:02:54.101 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.101 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.102 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:54.102 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.574200+0000", 2024-09-10T19:02:54.102 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.585e-06 2024-09-10T19:02:54.102 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.102 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.102 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.102 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.574230+0000", 2024-09-10T19:02:54.102 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.9411999999999999e-05 2024-09-10T19:02:54.102 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.102 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.102 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:54.102 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.574234+0000", 2024-09-10T19:02:54.102 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.2889999999999999e-06 2024-09-10T19:02:54.103 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.103 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.103 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.103 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:42.510605+0000", 2024-09-10T19:02:54.103 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.93637043099999995 2024-09-10T19:02:54.103 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.103 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.103 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:54.103 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:42.510613+0000", 2024-09-10T19:02:54.104 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.4039999999999995e-06 2024-09-10T19:02:54.104 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.104 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:54.104 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.104 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.104 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.104 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15573.0:139848 10.9 10:93a3c84c:::benchmark_data_smithi018_157943_object1190:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e355)", 2024-09-10T19:02:54.104 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:01:24.514699+0000", 2024-09-10T19:02:54.104 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 89.510986708999994, 2024-09-10T19:02:54.104 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0335216270000001, 2024-09-10T19:02:54.104 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:54.104 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "reached pg", 2024-09-10T19:02:54.104 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:54.105 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15573", 2024-09-10T19:02:54.105 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:02:54.105 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 139848 2024-09-10T19:02:54.105 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.105 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:54.105 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.105 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:54.105 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.514699+0000", 2024-09-10T19:02:54.105 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.105 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.105 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.105 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:54.105 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.514699+0000", 2024-09-10T19:02:54.106 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:54.107 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.107 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.107 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.107 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:54.107 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.514697+0000", 2024-09-10T19:02:54.107 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T19:02:54.107 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.107 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.107 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:54.107 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.514703+0000", 2024-09-10T19:02:54.108 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.5110000000000003e-06 2024-09-10T19:02:54.108 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.108 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.108 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:54.108 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.514713+0000", 2024-09-10T19:02:54.108 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.8889999999999997e-06 2024-09-10T19:02:54.108 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.108 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.108 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:54.108 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.514718+0000", 2024-09-10T19:02:54.108 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.3009999999999997e-06 2024-09-10T19:02:54.108 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.108 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.108 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.109 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.516858+0000", 2024-09-10T19:02:54.109 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0021399740000000002 2024-09-10T19:02:54.109 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.109 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.109 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op must wait for map", 2024-09-10T19:02:54.109 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.516864+0000", 2024-09-10T19:02:54.109 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.4880000000000004e-06 2024-09-10T19:02:54.109 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.109 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.109 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.109 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.519631+0000", 2024-09-10T19:02:54.109 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0027663010000000001 2024-09-10T19:02:54.109 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.109 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.110 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:54.110 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.519638+0000", 2024-09-10T19:02:54.110 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.4719999999999998e-06 2024-09-10T19:02:54.110 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.110 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.110 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.110 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.529452+0000", 2024-09-10T19:02:54.110 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0098136639999999 2024-09-10T19:02:54.110 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.110 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.110 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for missing object", 2024-09-10T19:02:54.110 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.529531+0000", 2024-09-10T19:02:54.110 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.8721999999999996e-05 2024-09-10T19:02:54.110 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.110 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.111 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.111 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.548158+0000", 2024-09-10T19:02:54.111 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.018627932999999999 2024-09-10T19:02:54.111 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.111 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.111 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:54.111 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.548220+0000", 2024-09-10T19:02:54.111 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.1864000000000003e-05 2024-09-10T19:02:54.111 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.111 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:54.111 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.111 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.111 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.111 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15573.0:139901 10.9 10:970b6964:::benchmark_data_smithi018_157943_object1243:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e355)", 2024-09-10T19:02:54.112 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:01:24.553536+0000", 2024-09-10T19:02:54.112 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 89.472149168000001, 2024-09-10T19:02:54.112 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.98862571499999996, 2024-09-10T19:02:54.112 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:54.112 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:02:54.112 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:54.112 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15573", 2024-09-10T19:02:54.112 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:02:54.112 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 139901 2024-09-10T19:02:54.112 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.112 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:54.112 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.112 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:54.113 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.553536+0000", 2024-09-10T19:02:54.113 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.113 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.113 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.113 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:54.113 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.553536+0000", 2024-09-10T19:02:54.113 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.113 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.113 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.113 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:54.113 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.553535+0000", 2024-09-10T19:02:54.113 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999981 2024-09-10T19:02:54.114 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.114 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.114 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:54.114 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.553541+0000", 2024-09-10T19:02:54.114 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.7549999999999997e-06 2024-09-10T19:02:54.114 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.114 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.114 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:54.114 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.553559+0000", 2024-09-10T19:02:54.114 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.8117999999999998e-05 2024-09-10T19:02:54.114 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.114 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.114 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:54.114 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.553566+0000", 2024-09-10T19:02:54.114 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.7059999999999998e-06 2024-09-10T19:02:54.115 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.115 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.115 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.115 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.554181+0000", 2024-09-10T19:02:54.115 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00061476499999999997 2024-09-10T19:02:54.115 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.115 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.115 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:54.115 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.554191+0000", 2024-09-10T19:02:54.115 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0567999999999999e-05 2024-09-10T19:02:54.115 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.115 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.115 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.116 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.529693+0000", 2024-09-10T19:02:54.116 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.97550131100000004 2024-09-10T19:02:54.116 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.116 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.116 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:02:54.116 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.529774+0000", 2024-09-10T19:02:54.116 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.0791999999999995e-05 2024-09-10T19:02:54.116 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.116 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.116 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,4", 2024-09-10T19:02:54.116 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.530016+0000", 2024-09-10T19:02:54.116 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00024236799999999999 2024-09-10T19:02:54.116 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.116 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.117 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:54.117 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.539388+0000", 2024-09-10T19:02:54.117 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.009372577 2024-09-10T19:02:54.117 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.117 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.117 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:54.117 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.539407+0000", 2024-09-10T19:02:54.117 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.8759000000000001e-05 2024-09-10T19:02:54.117 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.117 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.117 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:02:54.117 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.541995+0000", 2024-09-10T19:02:54.117 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0025879649999999998 2024-09-10T19:02:54.118 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.118 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.118 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:02:54.118 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.542044+0000", 2024-09-10T19:02:54.118 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.9277000000000003e-05 2024-09-10T19:02:54.118 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.118 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.118 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:54.118 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.542162+0000", 2024-09-10T19:02:54.118 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00011745599999999999 2024-09-10T19:02:54.118 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.118 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:54.118 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.118 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.119 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.119 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15573.0:139924 10.9 10:9ca72da2:::benchmark_data_smithi018_157943_object1266:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e355)", 2024-09-10T19:02:54.119 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:01:24.618093+0000", 2024-09-10T19:02:54.119 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 89.407592417999993, 2024-09-10T19:02:54.119 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.92420184500000002, 2024-09-10T19:02:54.119 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:54.119 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:02:54.119 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:54.119 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15573", 2024-09-10T19:02:54.119 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:02:54.119 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 139924 2024-09-10T19:02:54.119 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.119 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:54.119 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.120 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:54.120 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.618093+0000", 2024-09-10T19:02:54.120 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.120 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.120 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.120 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:54.120 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.618093+0000", 2024-09-10T19:02:54.120 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.120 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.120 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.120 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:54.120 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.618092+0000", 2024-09-10T19:02:54.120 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T19:02:54.121 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.121 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.121 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:54.121 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.618098+0000", 2024-09-10T19:02:54.121 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.4060000000000002e-06 2024-09-10T19:02:54.121 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.121 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.121 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:54.121 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.618111+0000", 2024-09-10T19:02:54.121 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.3071000000000001e-05 2024-09-10T19:02:54.121 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.121 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.121 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:54.122 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.618119+0000", 2024-09-10T19:02:54.122 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.8990000000000001e-06 2024-09-10T19:02:54.122 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.122 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.122 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.122 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.618161+0000", 2024-09-10T19:02:54.122 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.2463999999999999e-05 2024-09-10T19:02:54.122 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.122 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.122 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:54.122 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.618170+0000", 2024-09-10T19:02:54.122 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.7190000000000007e-06 2024-09-10T19:02:54.122 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.123 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.123 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.123 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.532104+0000", 2024-09-10T19:02:54.123 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.91393350200000001 2024-09-10T19:02:54.123 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.123 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.123 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:02:54.123 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.532229+0000", 2024-09-10T19:02:54.123 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00012533299999999999 2024-09-10T19:02:54.123 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.123 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.123 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,4", 2024-09-10T19:02:54.123 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.532362+0000", 2024-09-10T19:02:54.124 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00013297800000000001 2024-09-10T19:02:54.124 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.124 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.124 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:54.124 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.539423+0000", 2024-09-10T19:02:54.124 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0070610480000000003 2024-09-10T19:02:54.124 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.124 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.124 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:54.124 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.539442+0000", 2024-09-10T19:02:54.124 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.8675e-05 2024-09-10T19:02:54.124 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.124 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.125 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:02:54.125 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.542170+0000", 2024-09-10T19:02:54.125 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0027288590000000001 2024-09-10T19:02:54.125 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.125 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.125 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:02:54.125 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.542208+0000", 2024-09-10T19:02:54.125 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.7317999999999997e-05 2024-09-10T19:02:54.125 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.125 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.125 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:54.125 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.542295+0000", 2024-09-10T19:02:54.125 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.7045000000000004e-05 2024-09-10T19:02:54.125 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.126 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:54.126 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.126 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.126 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.126 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15573.0:139937 10.9 10:98533757:::benchmark_data_smithi018_157943_object1279:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e355)", 2024-09-10T19:02:54.126 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:01:24.624786+0000", 2024-09-10T19:02:54.126 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 89.400899654, 2024-09-10T19:02:54.126 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.92250637899999999, 2024-09-10T19:02:54.126 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:54.126 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:02:54.126 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:54.126 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15573", 2024-09-10T19:02:54.126 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:02:54.127 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 139937 2024-09-10T19:02:54.127 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.127 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:54.127 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.127 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:54.127 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.624786+0000", 2024-09-10T19:02:54.127 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.127 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.127 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.127 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:54.127 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.624786+0000", 2024-09-10T19:02:54.127 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.127 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.127 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.127 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:54.128 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.624785+0000", 2024-09-10T19:02:54.128 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T19:02:54.128 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.128 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.128 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:54.128 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.624791+0000", 2024-09-10T19:02:54.128 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.1299999999999998e-06 2024-09-10T19:02:54.128 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.128 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.128 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:54.128 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.624803+0000", 2024-09-10T19:02:54.128 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.1864e-05 2024-09-10T19:02:54.128 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.129 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.129 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:54.129 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.624809+0000", 2024-09-10T19:02:54.129 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.925e-06 2024-09-10T19:02:54.129 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.129 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.129 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.129 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.624860+0000", 2024-09-10T19:02:54.129 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.1304999999999998e-05 2024-09-10T19:02:54.129 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.129 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.129 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:54.129 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.624871+0000", 2024-09-10T19:02:54.129 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.148e-05 2024-09-10T19:02:54.130 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.130 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.130 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.130 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.539455+0000", 2024-09-10T19:02:54.130 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.91458371999999999 2024-09-10T19:02:54.130 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.130 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.130 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:02:54.130 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.539534+0000", 2024-09-10T19:02:54.130 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.8471000000000001e-05 2024-09-10T19:02:54.130 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.130 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.130 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,4", 2024-09-10T19:02:54.130 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.539614+0000", 2024-09-10T19:02:54.130 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.0758000000000002e-05 2024-09-10T19:02:54.131 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.131 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.131 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:54.131 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.541987+0000", 2024-09-10T19:02:54.131 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0023724190000000002 2024-09-10T19:02:54.131 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.131 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.131 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:02:54.131 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.542367+0000", 2024-09-10T19:02:54.131 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00038044400000000002 2024-09-10T19:02:54.131 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.131 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.131 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:54.132 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.547206+0000", 2024-09-10T19:02:54.132 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0048384919999999998 2024-09-10T19:02:54.132 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.132 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.132 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:02:54.132 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.547238+0000", 2024-09-10T19:02:54.132 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.2465000000000002e-05 2024-09-10T19:02:54.132 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.132 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.132 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:54.132 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.547292+0000", 2024-09-10T19:02:54.132 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.4030000000000003e-05 2024-09-10T19:02:54.132 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.133 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:54.133 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.133 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.133 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.133 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15573.0:139944 10.9 10:94c237c6:::benchmark_data_smithi018_157943_object1286:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e355)", 2024-09-10T19:02:54.133 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:01:24.627358+0000", 2024-09-10T19:02:54.133 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 89.398327546999994, 2024-09-10T19:02:54.133 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.92027273200000004, 2024-09-10T19:02:54.133 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:54.133 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:02:54.133 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:54.133 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15573", 2024-09-10T19:02:54.133 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:02:54.133 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 139944 2024-09-10T19:02:54.134 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.134 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:54.134 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.134 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:54.134 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.627358+0000", 2024-09-10T19:02:54.134 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.134 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.134 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.134 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:54.134 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.627358+0000", 2024-09-10T19:02:54.134 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.134 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.134 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.135 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:54.135 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.627357+0000", 2024-09-10T19:02:54.135 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T19:02:54.135 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.135 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.135 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:54.135 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.627363+0000", 2024-09-10T19:02:54.135 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.6890000000000001e-06 2024-09-10T19:02:54.135 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.135 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.135 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:54.135 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.627378+0000", 2024-09-10T19:02:54.135 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.5738000000000001e-05 2024-09-10T19:02:54.136 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.136 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.136 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:54.136 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.627385+0000", 2024-09-10T19:02:54.136 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.3890000000000002e-06 2024-09-10T19:02:54.136 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.136 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.136 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.136 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.627429+0000", 2024-09-10T19:02:54.137 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.3788999999999997e-05 2024-09-10T19:02:54.137 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.137 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.137 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:54.137 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.627436+0000", 2024-09-10T19:02:54.137 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.34e-06 2024-09-10T19:02:54.137 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.137 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.137 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.137 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.540704+0000", 2024-09-10T19:02:54.137 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.91326781000000001 2024-09-10T19:02:54.137 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.137 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.138 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:02:54.138 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.540800+0000", 2024-09-10T19:02:54.138 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.6714999999999998e-05 2024-09-10T19:02:54.138 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.138 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.138 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,4", 2024-09-10T19:02:54.138 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.540927+0000", 2024-09-10T19:02:54.138 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000126181 2024-09-10T19:02:54.138 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.138 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.138 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:54.138 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.542414+0000", 2024-09-10T19:02:54.138 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.001487036 2024-09-10T19:02:54.139 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.139 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.139 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:02:54.139 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.547305+0000", 2024-09-10T19:02:54.139 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0048917429999999996 2024-09-10T19:02:54.139 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.139 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.139 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:54.139 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.547547+0000", 2024-09-10T19:02:54.139 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00024154600000000001 2024-09-10T19:02:54.139 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.139 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.139 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:02:54.140 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.547578+0000", 2024-09-10T19:02:54.140 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.0768000000000001e-05 2024-09-10T19:02:54.140 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.140 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.140 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:54.140 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.547631+0000", 2024-09-10T19:02:54.140 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.2918999999999999e-05 2024-09-10T19:02:54.140 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.140 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:54.140 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.140 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.140 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.140 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15573.0:139984 10.9 10:9587b78e:::benchmark_data_smithi018_157943_object1326:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e355)", 2024-09-10T19:02:54.141 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:01:24.652294+0000", 2024-09-10T19:02:54.141 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 89.373391033999994, 2024-09-10T19:02:54.141 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.89563763500000004, 2024-09-10T19:02:54.141 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:54.141 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:02:54.141 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:54.141 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15573", 2024-09-10T19:02:54.141 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:02:54.141 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 139984 2024-09-10T19:02:54.141 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.141 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:54.141 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.141 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:54.141 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.652294+0000", 2024-09-10T19:02:54.142 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.142 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.142 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.142 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:54.142 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.652294+0000", 2024-09-10T19:02:54.142 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.142 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.142 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.142 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:54.142 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.652293+0000", 2024-09-10T19:02:54.142 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T19:02:54.142 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.142 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.143 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:54.143 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.652298+0000", 2024-09-10T19:02:54.143 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.8509999999999998e-06 2024-09-10T19:02:54.143 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.143 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.143 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:54.143 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.652313+0000", 2024-09-10T19:02:54.143 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.4810000000000001e-05 2024-09-10T19:02:54.143 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.143 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.143 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:54.143 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.652318+0000", 2024-09-10T19:02:54.143 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.8969999999999996e-06 2024-09-10T19:02:54.143 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.143 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.144 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.144 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.652371+0000", 2024-09-10T19:02:54.144 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.2219999999999998e-05 2024-09-10T19:02:54.144 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.144 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.144 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:54.144 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.652378+0000", 2024-09-10T19:02:54.144 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.1359999999999999e-06 2024-09-10T19:02:54.144 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.144 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.144 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.144 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.542442+0000", 2024-09-10T19:02:54.144 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.89006390999999996 2024-09-10T19:02:54.144 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.145 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.145 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:02:54.145 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.542532+0000", 2024-09-10T19:02:54.145 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.9917e-05 2024-09-10T19:02:54.145 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.145 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.145 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,4", 2024-09-10T19:02:54.145 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.542657+0000", 2024-09-10T19:02:54.145 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000125573 2024-09-10T19:02:54.145 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.145 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.145 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:54.145 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.547513+0000", 2024-09-10T19:02:54.145 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0048557169999999998 2024-09-10T19:02:54.146 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.146 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.146 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:02:54.146 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.547773+0000", 2024-09-10T19:02:54.146 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00025963800000000002 2024-09-10T19:02:54.146 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.146 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.146 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:54.146 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.547805+0000", 2024-09-10T19:02:54.146 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.2432999999999999e-05 2024-09-10T19:02:54.146 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.146 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.146 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:02:54.146 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.547853+0000", 2024-09-10T19:02:54.146 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.7969000000000001e-05 2024-09-10T19:02:54.147 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.147 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.147 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:54.147 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.547932+0000", 2024-09-10T19:02:54.147 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.9039e-05 2024-09-10T19:02:54.147 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.147 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:54.147 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.147 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.147 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.147 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15573.0:140018 10.9 10:9a882899:::benchmark_data_smithi018_157943_object1360:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e355)", 2024-09-10T19:02:54.147 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:01:24.671288+0000", 2024-09-10T19:02:54.147 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 89.354397595999998, 2024-09-10T19:02:54.147 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.88791288300000004, 2024-09-10T19:02:54.147 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:54.148 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:02:54.148 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:54.148 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15573", 2024-09-10T19:02:54.148 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:02:54.148 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 140018 2024-09-10T19:02:54.148 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.148 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:54.148 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.148 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:54.148 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.671288+0000", 2024-09-10T19:02:54.148 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.148 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.148 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.148 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:54.148 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.671288+0000", 2024-09-10T19:02:54.149 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.149 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.149 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.149 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:54.149 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.671287+0000", 2024-09-10T19:02:54.149 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999995 2024-09-10T19:02:54.149 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.149 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.149 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:54.149 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.671291+0000", 2024-09-10T19:02:54.149 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.602e-06 2024-09-10T19:02:54.149 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.149 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.149 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:54.150 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.671300+0000", 2024-09-10T19:02:54.150 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.4299999999999995e-06 2024-09-10T19:02:54.150 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.150 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.150 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:54.150 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.671306+0000", 2024-09-10T19:02:54.150 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.5260000000000001e-06 2024-09-10T19:02:54.150 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.150 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.150 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.150 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.671348+0000", 2024-09-10T19:02:54.150 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.2252999999999997e-05 2024-09-10T19:02:54.150 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.151 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.151 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:54.151 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.671355+0000", 2024-09-10T19:02:54.151 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.8000000000000001e-06 2024-09-10T19:02:54.151 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.151 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.151 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.151 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.548241+0000", 2024-09-10T19:02:54.151 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.87688596299999999 2024-09-10T19:02:54.151 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.151 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.151 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:02:54.151 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.548336+0000", 2024-09-10T19:02:54.152 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.4868e-05 2024-09-10T19:02:54.152 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.152 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.152 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,4", 2024-09-10T19:02:54.152 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.548952+0000", 2024-09-10T19:02:54.152 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000616013 2024-09-10T19:02:54.152 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.152 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.152 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:54.152 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.554652+0000", 2024-09-10T19:02:54.152 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0057007059999999998 2024-09-10T19:02:54.152 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.152 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.153 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:02:54.153 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.558950+0000", 2024-09-10T19:02:54.153 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0042978290000000004 2024-09-10T19:02:54.153 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.153 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.153 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:54.153 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.559060+0000", 2024-09-10T19:02:54.153 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00011017499999999999 2024-09-10T19:02:54.153 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.153 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.153 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:02:54.153 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.559108+0000", 2024-09-10T19:02:54.154 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.8220000000000002e-05 2024-09-10T19:02:54.154 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.154 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.154 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:54.154 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.559201+0000", 2024-09-10T19:02:54.154 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.2211000000000002e-05 2024-09-10T19:02:54.154 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.154 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:54.154 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.154 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.154 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.154 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15573.0:140019 10.9 10:9492a914:::benchmark_data_smithi018_157943_object1361:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e355)", 2024-09-10T19:02:54.154 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:01:24.671377+0000", 2024-09-10T19:02:54.154 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 89.354307942000005, 2024-09-10T19:02:54.155 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.88806792300000004, 2024-09-10T19:02:54.155 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:02:54.155 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:02:54.155 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:02:54.155 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15573", 2024-09-10T19:02:54.155 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:02:54.155 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 140019 2024-09-10T19:02:54.155 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.155 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:02:54.155 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.155 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:02:54.155 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.671377+0000", 2024-09-10T19:02:54.155 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.155 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.156 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.156 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:02:54.156 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.671377+0000", 2024-09-10T19:02:54.156 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:02:54.156 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.156 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.156 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:02:54.156 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.671377+0000", 2024-09-10T19:02:54.156 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T19:02:54.156 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.156 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.156 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:02:54.156 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.671381+0000", 2024-09-10T19:02:54.156 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.0589999999999996e-06 2024-09-10T19:02:54.157 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.157 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.157 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:02:54.157 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.671388+0000", 2024-09-10T19:02:54.157 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.5449999999999998e-06 2024-09-10T19:02:54.157 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.157 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.157 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:02:54.157 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.671393+0000", 2024-09-10T19:02:54.157 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.7999999999999998e-06 2024-09-10T19:02:54.157 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.157 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.157 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.158 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.671426+0000", 2024-09-10T19:02:54.158 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.2839000000000002e-05 2024-09-10T19:02:54.158 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.158 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.158 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:02:54.158 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.671432+0000", 2024-09-10T19:02:54.158 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.7819999999999999e-06 2024-09-10T19:02:54.158 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.158 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.158 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:02:54.158 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.550664+0000", 2024-09-10T19:02:54.158 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.87923176199999997 2024-09-10T19:02:54.158 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.158 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.159 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:02:54.159 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.550788+0000", 2024-09-10T19:02:54.159 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000124115 2024-09-10T19:02:54.159 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.159 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.159 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,4", 2024-09-10T19:02:54.159 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.550913+0000", 2024-09-10T19:02:54.159 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00012519899999999999 2024-09-10T19:02:54.159 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.159 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.159 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:54.159 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.559002+0000", 2024-09-10T19:02:54.159 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0080888290000000005 2024-09-10T19:02:54.159 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.159 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.160 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:02:54.160 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.559213+0000", 2024-09-10T19:02:54.160 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00021132200000000001 2024-09-10T19:02:54.160 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.160 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.160 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:02:54.160 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.559309+0000", 2024-09-10T19:02:54.160 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.5796000000000002e-05 2024-09-10T19:02:54.160 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.160 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.160 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:02:54.160 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.559355+0000", 2024-09-10T19:02:54.160 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.6709000000000003e-05 2024-09-10T19:02:54.160 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:02:54.161 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:02:54.161 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:02:54.161 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.559445+0000", 2024-09-10T19:02:54.161 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.9895999999999994e-05 2024-09-10T19:02:54.161 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.161 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:54.161 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.161 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:02:54.161 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:02:54.161 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T19:02:54.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:54.185 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:54.185 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:54.286 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:54.308 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:54.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:54.309 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:54.410 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:54.433 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:54.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:54.434 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:54.532 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T19:02:54.532 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T19:02:54.535 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:54.549 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T19:02:54.549 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T19:02:54.557 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:54.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:54.558 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:54.565 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T19:02:54.565 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T19:02:54.581 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T19:02:54.581 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T19:02:54.597 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T19:02:54.597 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T19:02:54.613 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T19:02:54.613 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T19:02:54.629 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T19:02:54.629 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T19:02:54.629 INFO:teuthology.orchestra.run.smithi018.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-10T19:02:54.658 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:54.680 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:54.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:54.681 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:54.782 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:54.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:54.805 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:54.805 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:54.905 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6] out_osds: [2, 7] dead_osds: [7] live_osds: [1, 2, 3, 4, 5, 6, 0] 2024-09-10T19:02:54.905 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T19:02:54.905 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-10T19:02:54.905 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-10T19:02:54.906 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-10T19:02:54.906 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.3 config set filestore_inject_stall 3 2024-09-10T19:02:54.906 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:54.928 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:54.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:54.929 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:54.929 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:54 smithi018 ceph-mon[29336]: pgmap v816: 168 pgs: 5 peering, 163 active+clean; 31 MiB data, 6.1 GiB used, 530 GiB / 536 GiB avail; 8.2 KiB/s wr, 870 op/s; 5.7 KiB/s, 0 objects/s recovering 2024-09-10T19:02:54.929 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:54 smithi018 ceph-mon[29336]: 4.3 scrub starts 2024-09-10T19:02:54.929 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:54 smithi018 ceph-mon[29336]: 4.3 scrub ok 2024-09-10T19:02:54.929 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:54 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3080848356' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:02:54.930 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:54 smithi018 ceph-mon[40196]: pgmap v816: 168 pgs: 5 peering, 163 active+clean; 31 MiB data, 6.1 GiB used, 530 GiB / 536 GiB avail; 8.2 KiB/s wr, 870 op/s; 5.7 KiB/s, 0 objects/s recovering 2024-09-10T19:02:54.930 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:54 smithi018 ceph-mon[40196]: 4.3 scrub starts 2024-09-10T19:02:54.930 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:54 smithi018 ceph-mon[40196]: 4.3 scrub ok 2024-09-10T19:02:54.930 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:54 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3080848356' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:02:55.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:54 smithi103 ceph-mon[33163]: pgmap v816: 168 pgs: 5 peering, 163 active+clean; 31 MiB data, 6.1 GiB used, 530 GiB / 536 GiB avail; 8.2 KiB/s wr, 870 op/s; 5.7 KiB/s, 0 objects/s recovering 2024-09-10T19:02:55.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:54 smithi103 ceph-mon[33163]: 4.3 scrub starts 2024-09-10T19:02:55.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:54 smithi103 ceph-mon[33163]: 4.3 scrub ok 2024-09-10T19:02:55.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:54 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3080848356' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:02:55.030 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:55.051 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:55.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:55.052 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:55.152 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-10T19:02:55.153 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:55.175 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:55.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:55.176 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:55.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:02:55 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T19:02:55.093+0000 7f66d46e1640 -1 mgr get_metadata_python Requested missing service osd.7 2024-09-10T19:02:55.278 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:55.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:55.301 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:55.301 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:55.318 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-10T19:02:55.319 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-10T19:02:55.392 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:02:55.398 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:02:55.402 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:55.424 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:55.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:55.425 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:55.526 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:55.549 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:55.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:55.550 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:55.630 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-10T19:02:55.651 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:55.695 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:55.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:55.696 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:55.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:55 smithi103 ceph-mon[33163]: 1.c scrub starts 2024-09-10T19:02:55.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:55 smithi103 ceph-mon[33163]: 1.c scrub ok 2024-09-10T19:02:55.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:55 smithi103 ceph-mon[33163]: 6.1 scrub starts 2024-09-10T19:02:55.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:55 smithi103 ceph-mon[33163]: 6.1 scrub ok 2024-09-10T19:02:55.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:55 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:02:55.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:02:55.797 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:55.820 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:55.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:55.821 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:55.832 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:55 smithi018 ceph-mon[40196]: 1.c scrub starts 2024-09-10T19:02:55.832 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:55 smithi018 ceph-mon[40196]: 1.c scrub ok 2024-09-10T19:02:55.833 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:55 smithi018 ceph-mon[40196]: 6.1 scrub starts 2024-09-10T19:02:55.833 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:55 smithi018 ceph-mon[40196]: 6.1 scrub ok 2024-09-10T19:02:55.833 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:55 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:02:55.833 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:02:55.833 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:55 smithi018 ceph-mon[29336]: 1.c scrub starts 2024-09-10T19:02:55.833 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:55 smithi018 ceph-mon[29336]: 1.c scrub ok 2024-09-10T19:02:55.833 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:55 smithi018 ceph-mon[29336]: 6.1 scrub starts 2024-09-10T19:02:55.833 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:55 smithi018 ceph-mon[29336]: 6.1 scrub ok 2024-09-10T19:02:55.833 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:55 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:02:55.833 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:02:55.922 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:55.945 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:55.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:55.946 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:56.046 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:56.070 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:56.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:56.071 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:56.172 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:56.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:56.195 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:56.195 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:56.296 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:56.318 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:56.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:56.319 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:56.420 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:56.442 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:56.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:56.443 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:56.490 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_3 2024-09-10T19:02:56.491 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd pool rm unique_pool_3 unique_pool_3 --yes-i-really-really-mean-it 2024-09-10T19:02:56.544 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:56.566 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:56.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:56.567 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:56.668 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:56.692 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:56.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:56.693 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:56.785 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:56 smithi018 ceph-mon[40196]: pgmap v817: 168 pgs: 168 active+clean; 600 KiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 7.1 KiB/s wr, 829 op/s; 5.0 KiB/s, 0 objects/s recovering 2024-09-10T19:02:56.785 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:56 smithi018 ceph-mon[40196]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs inactive, 4 pgs peering) 2024-09-10T19:02:56.785 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:56 smithi018 ceph-mon[40196]: Cluster is now healthy 2024-09-10T19:02:56.785 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:56 smithi018 ceph-mon[29336]: pgmap v817: 168 pgs: 168 active+clean; 600 KiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 7.1 KiB/s wr, 829 op/s; 5.0 KiB/s, 0 objects/s recovering 2024-09-10T19:02:56.785 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:56 smithi018 ceph-mon[29336]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs inactive, 4 pgs peering) 2024-09-10T19:02:56.785 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:56 smithi018 ceph-mon[29336]: Cluster is now healthy 2024-09-10T19:02:56.793 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:56.816 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:56.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:56.817 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:56.917 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:56.940 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:56.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:56.941 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:57.022 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:02:57.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:56 smithi103 ceph-mon[33163]: pgmap v817: 168 pgs: 168 active+clean; 600 KiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 7.1 KiB/s wr, 829 op/s; 5.0 KiB/s, 0 objects/s recovering 2024-09-10T19:02:57.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:56 smithi103 ceph-mon[33163]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs inactive, 4 pgs peering) 2024-09-10T19:02:57.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:56 smithi103 ceph-mon[33163]: Cluster is now healthy 2024-09-10T19:02:57.039 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T19:02:57.039 INFO:teuthology.orchestra.run.smithi018.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-10T19:02:57.039 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T19:02:57.042 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:57.067 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:57.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:57.068 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:57.169 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:57.192 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:57.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:57.193 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:57.290 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:02:57.294 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:57.317 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:57.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:57.318 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:57.319 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T19:02:57.352 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:02:57 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:02:57.419 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:57.441 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:57.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:57.442 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:57.543 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:57.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:57.569 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:57.569 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:57.646 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:57 smithi103 ceph-mon[33163]: 5.1b scrub starts 2024-09-10T19:02:57.646 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:57 smithi103 ceph-mon[33163]: 5.1b scrub ok 2024-09-10T19:02:57.646 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:57 smithi103 ceph-mon[33163]: 1.2 scrub starts 2024-09-10T19:02:57.646 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:57 smithi103 ceph-mon[33163]: 1.2 scrub ok 2024-09-10T19:02:57.670 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:57.686 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:57 smithi018 ceph-mon[40196]: 5.1b scrub starts 2024-09-10T19:02:57.686 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:57 smithi018 ceph-mon[40196]: 5.1b scrub ok 2024-09-10T19:02:57.686 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:57 smithi018 ceph-mon[40196]: 1.2 scrub starts 2024-09-10T19:02:57.687 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:57 smithi018 ceph-mon[40196]: 1.2 scrub ok 2024-09-10T19:02:57.687 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:57 smithi018 ceph-mon[29336]: 5.1b scrub starts 2024-09-10T19:02:57.687 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:57 smithi018 ceph-mon[29336]: 5.1b scrub ok 2024-09-10T19:02:57.687 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:57 smithi018 ceph-mon[29336]: 1.2 scrub starts 2024-09-10T19:02:57.687 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:57 smithi018 ceph-mon[29336]: 1.2 scrub ok 2024-09-10T19:02:57.694 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:57.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:57.695 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:57.762 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T19:02:57.763 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T19:02:57.796 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:57.817 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:57.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:57.819 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:57.919 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:57.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:57.941 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:57.942 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:58.042 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:58.064 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:58.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:58.065 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:58.166 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:58.191 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:58.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:58.192 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:58.292 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:58.317 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:58.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:58.318 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:58.418 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:58.442 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:58.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:58.443 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:58.452 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-10T19:02:58.544 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:58.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:58.567 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:58.567 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:58.668 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:02:58.690 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:58.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:58.691 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:02:58.743 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:02:58.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:58 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:58.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:58 smithi103 ceph-mon[33163]: pgmap v818: 168 pgs: 168 active+clean; 600 KiB data, 5.9 GiB used, 530 GiB / 536 GiB avail; 6.1 KiB/s wr, 522 op/s; 4.3 KiB/s, 0 objects/s recovering 2024-09-10T19:02:58.791 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:02:58.812 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:58.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:58.813 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:02:58.863 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:58 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:58.863 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:58 smithi018 ceph-mon[29336]: pgmap v818: 168 pgs: 168 active+clean; 600 KiB data, 5.9 GiB used, 530 GiB / 536 GiB avail; 6.1 KiB/s wr, 522 op/s; 4.3 KiB/s, 0 objects/s recovering 2024-09-10T19:02:58.863 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:58 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:58.863 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:58 smithi018 ceph-mon[40196]: pgmap v818: 168 pgs: 168 active+clean; 600 KiB data, 5.9 GiB used, 530 GiB / 536 GiB avail; 6.1 KiB/s wr, 522 op/s; 4.3 KiB/s, 0 objects/s recovering 2024-09-10T19:02:58.914 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:02:58.939 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:58.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:58.940 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:02:59.041 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:59.065 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:59.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:59.066 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:59.167 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:59.189 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:59.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:59.190 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:59.291 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:02:59.312 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:02:59.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:59.313 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:02:59.414 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:59.437 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:59.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:59.438 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:59.539 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:02:59.561 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:59.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:59.562 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:02:59.663 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:02:59.686 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:59.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:59.687 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:02:59.788 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:59.810 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:59.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:59.811 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:59.912 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:02:59.934 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:02:59.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:02:59.935 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:02:59.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:59 smithi018 ceph-mon[40196]: 6.8 scrub starts 2024-09-10T19:02:59.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:59 smithi018 ceph-mon[40196]: 6.8 scrub ok 2024-09-10T19:02:59.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:59 smithi018 ceph-mon[40196]: 6.f scrub starts 2024-09-10T19:02:59.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:59 smithi018 ceph-mon[40196]: 6.f scrub ok 2024-09-10T19:02:59.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:59 smithi018 ceph-mon[40196]: 1.5 scrub starts 2024-09-10T19:02:59.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:02:59 smithi018 ceph-mon[40196]: 1.5 scrub ok 2024-09-10T19:02:59.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:59 smithi018 ceph-mon[29336]: 6.8 scrub starts 2024-09-10T19:02:59.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:59 smithi018 ceph-mon[29336]: 6.8 scrub ok 2024-09-10T19:02:59.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:59 smithi018 ceph-mon[29336]: 6.f scrub starts 2024-09-10T19:02:59.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:59 smithi018 ceph-mon[29336]: 6.f scrub ok 2024-09-10T19:02:59.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:59 smithi018 ceph-mon[29336]: 1.5 scrub starts 2024-09-10T19:02:59.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:02:59 smithi018 ceph-mon[29336]: 1.5 scrub ok 2024-09-10T19:03:00.017 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:59 smithi103 ceph-mon[33163]: 6.8 scrub starts 2024-09-10T19:03:00.018 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:59 smithi103 ceph-mon[33163]: 6.8 scrub ok 2024-09-10T19:03:00.018 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:59 smithi103 ceph-mon[33163]: 6.f scrub starts 2024-09-10T19:03:00.018 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:59 smithi103 ceph-mon[33163]: 6.f scrub ok 2024-09-10T19:03:00.018 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:59 smithi103 ceph-mon[33163]: 1.5 scrub starts 2024-09-10T19:03:00.018 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:02:59 smithi103 ceph-mon[33163]: 1.5 scrub ok 2024-09-10T19:03:00.035 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:00.058 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:00.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:00.059 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:00.160 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:00.183 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:00.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:00.184 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:00.272 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:03:00 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:03:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:03:00.285 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:00.307 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:00.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:00.308 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:00.409 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:00.431 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:00.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:00.432 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:00.521 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T19:03:00.522 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T19:03:00.533 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:00.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:00.556 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:00.556 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:00.657 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:00.679 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:00.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:00.680 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:00.685 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:03:00.756 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is set 2024-09-10T19:03:00.781 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:00.804 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:00.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:00.805 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:00.835 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:00 smithi103 ceph-mon[33163]: pgmap v819: 168 pgs: 168 active+clean; 600 KiB data, 5.9 GiB used, 530 GiB / 536 GiB avail; 5.0 KiB/s wr, 423 op/s; 3.5 KiB/s, 0 objects/s recovering 2024-09-10T19:03:00.835 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:00 smithi103 ceph-mon[33163]: 1.0 scrub starts 2024-09-10T19:03:00.835 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:00 smithi103 ceph-mon[33163]: 1.0 scrub ok 2024-09-10T19:03:00.835 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:00 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/150558654' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T19:03:00.835 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:00 smithi103 ceph-mon[33163]: 2.1a scrub starts 2024-09-10T19:03:00.835 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:00 smithi103 ceph-mon[33163]: 2.1a scrub ok 2024-09-10T19:03:00.858 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:00 smithi018 ceph-mon[40196]: pgmap v819: 168 pgs: 168 active+clean; 600 KiB data, 5.9 GiB used, 530 GiB / 536 GiB avail; 5.0 KiB/s wr, 423 op/s; 3.5 KiB/s, 0 objects/s recovering 2024-09-10T19:03:00.858 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:00 smithi018 ceph-mon[40196]: 1.0 scrub starts 2024-09-10T19:03:00.858 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:00 smithi018 ceph-mon[40196]: 1.0 scrub ok 2024-09-10T19:03:00.858 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:00 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/150558654' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T19:03:00.858 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:00 smithi018 ceph-mon[40196]: 2.1a scrub starts 2024-09-10T19:03:00.858 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:00 smithi018 ceph-mon[40196]: 2.1a scrub ok 2024-09-10T19:03:00.858 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:00 smithi018 ceph-mon[29336]: pgmap v819: 168 pgs: 168 active+clean; 600 KiB data, 5.9 GiB used, 530 GiB / 536 GiB avail; 5.0 KiB/s wr, 423 op/s; 3.5 KiB/s, 0 objects/s recovering 2024-09-10T19:03:00.859 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:00 smithi018 ceph-mon[29336]: 1.0 scrub starts 2024-09-10T19:03:00.859 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:00 smithi018 ceph-mon[29336]: 1.0 scrub ok 2024-09-10T19:03:00.859 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:00 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/150558654' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T19:03:00.859 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:00 smithi018 ceph-mon[29336]: 2.1a scrub starts 2024-09-10T19:03:00.859 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:00 smithi018 ceph-mon[29336]: 2.1a scrub ok 2024-09-10T19:03:00.906 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:00.928 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:00.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:00.929 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:01.030 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:01.051 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:01.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:01.052 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:01.142 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-10T19:03:01.154 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:01.169 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:03:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:03:00] "GET /metrics HTTP/1.1" 200 209509 "" "Prometheus/2.43.0" 2024-09-10T19:03:01.206 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:01.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:01.207 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:01.308 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:01.330 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:01.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:01.331 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:01.386 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:03:01.432 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:01.454 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:01.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:01.455 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:01.555 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:01.577 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:01.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:01.578 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:01.678 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:01.706 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:01.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:01.707 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:01.755 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:01 smithi103 ceph-mon[33163]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:03:01.755 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:01 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/150558654' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T19:03:01.755 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:01 smithi103 ceph-mon[33163]: osdmap e375: 8 total, 7 up, 6 in 2024-09-10T19:03:01.808 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:01.830 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:01.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:01.831 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:01.932 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:01.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:01 smithi018 ceph-mon[29336]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:03:01.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:01 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/150558654' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T19:03:01.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:01 smithi018 ceph-mon[29336]: osdmap e375: 8 total, 7 up, 6 in 2024-09-10T19:03:01.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:01 smithi018 ceph-mon[40196]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:03:01.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:01 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/150558654' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T19:03:01.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:01 smithi018 ceph-mon[40196]: osdmap e375: 8 total, 7 up, 6 in 2024-09-10T19:03:01.953 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:01.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:01.954 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:02.054 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:02.075 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:02.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:02.076 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:02.177 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:02.204 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:02.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:02.205 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:02.306 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:02.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:02.333 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:02.333 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:02.434 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:02.456 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:02.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:02.457 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:02.558 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:02.579 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:02.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:02.580 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:02.681 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:02.705 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:02.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:02.706 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:02.733 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:02 smithi018 ceph-mon[29336]: pgmap v821: 168 pgs: 168 active+clean; 600 KiB data, 5.9 GiB used, 530 GiB / 536 GiB avail; 577 B/s rd, 0 B/s wr, 54 op/s 2024-09-10T19:03:02.733 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:02 smithi018 ceph-mon[40196]: pgmap v821: 168 pgs: 168 active+clean; 600 KiB data, 5.9 GiB used, 530 GiB / 536 GiB avail; 577 B/s rd, 0 B/s wr, 54 op/s 2024-09-10T19:03:02.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:02 smithi103 ceph-mon[33163]: pgmap v821: 168 pgs: 168 active+clean; 600 KiB data, 5.9 GiB used, 530 GiB / 536 GiB avail; 577 B/s rd, 0 B/s wr, 54 op/s 2024-09-10T19:03:02.806 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:02.830 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:02.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:02.831 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:02.931 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:02.954 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:02.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:02.955 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:03.039 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T19:03:03.039 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T19:03:03.056 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:03.077 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:03.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:03.078 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:03.123 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T19:03:03.123 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T19:03:03.135 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T19:03:03.136 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T19:03:03.150 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T19:03:03.151 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T19:03:03.166 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T19:03:03.166 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T19:03:03.179 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:03.182 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T19:03:03.182 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T19:03:03.197 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T19:03:03.197 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T19:03:03.203 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:03.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:03.203 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:03.214 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T19:03:03.214 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T19:03:03.214 INFO:teuthology.orchestra.run.smithi018.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-10T19:03:03.304 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:03.328 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:03.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:03.329 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:03.430 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:03.453 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:03.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:03.454 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:03.555 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:03.578 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:03.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:03.579 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:03.626 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-10T19:03:03.680 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:03.702 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:03.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:03.703 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:03.804 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:03.829 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:03.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:03.830 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:03.930 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:03.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:03 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:03:03.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:03 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:03:03.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:03 smithi018 ceph-mon[29336]: osdmap e376: 8 total, 7 up, 6 in 2024-09-10T19:03:03.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:03 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1438062221' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:03:03.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:03 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:03:03.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:03 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:03:03.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:03 smithi018 ceph-mon[40196]: osdmap e376: 8 total, 7 up, 6 in 2024-09-10T19:03:03.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:03 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1438062221' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:03:03.953 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:03.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:03.954 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:04.005 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-10T19:03:04.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:03 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:03:04.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:03 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:03:04.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:03 smithi103 ceph-mon[33163]: osdmap e376: 8 total, 7 up, 6 in 2024-09-10T19:03:04.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:03 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1438062221' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:03:04.055 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:04.077 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:04.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:04.078 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:04.149 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-10T19:03:04.149 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-10T19:03:04.179 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:04.203 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:04.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:04.204 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:04.305 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:04.328 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:04.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:04.329 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:04.430 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:04.442 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:03:04.453 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:04.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:04.454 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:04.476 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:03:04.555 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:04.578 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:04.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:04.579 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:04.680 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:04.703 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:04.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:04.704 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:04.805 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:04.827 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:04.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:04.828 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:04.925 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:04 smithi018 ceph-mon[40196]: pgmap v823: 168 pgs: 168 active+clean; 600 KiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:04.925 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:04 smithi018 ceph-mon[29336]: pgmap v823: 168 pgs: 168 active+clean; 600 KiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:04.929 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:04.951 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:04.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:04.952 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:05.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:04 smithi103 ceph-mon[33163]: pgmap v823: 168 pgs: 168 active+clean; 600 KiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:05.053 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:05.077 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:05.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:05.078 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:05.179 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:05.202 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:05.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:05.202 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:05.303 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:05.329 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:05.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:05.330 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:05.431 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:05.456 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:05.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:05.457 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:05.558 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:05.583 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:05.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:05.584 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:05.592 INFO:teuthology.orchestra.run.smithi018.stderr:pool 'unique_pool_3' removed 2024-09-10T19:03:05.647 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6] out_osds: [2, 7] dead_osds: [7] live_osds: [1, 2, 3, 4, 5, 6, 0] 2024-09-10T19:03:05.647 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T19:03:05.685 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:05.707 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:05.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:05.708 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:05.809 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:05.831 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:05.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:05.832 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:05.933 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:05.943 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:05 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2965942466' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T19:03:05.943 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:05 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T19:03:05.943 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:05 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2965942466' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T19:03:05.943 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:05 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T19:03:05.955 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:05.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:05.956 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:06.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:05 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2965942466' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T19:03:06.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:05 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T19:03:06.057 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:06.078 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:06.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:06.079 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:06.151 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T19:03:06.180 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:06.211 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:06.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:06.212 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:06.313 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:06.335 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:06.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:06.336 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:06.437 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:06.459 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:06.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:06.460 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:06.561 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:06.583 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:06.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:06.584 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:06.685 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:06.706 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:06.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:06.707 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:06.808 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:06.829 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:06.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:06.830 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:06.931 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:06.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:06 smithi018 ceph-mon[40196]: pgmap v824: 168 pgs: 168 active+clean; 600 KiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T19:03:06.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:06 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T19:03:06.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:06 smithi018 ceph-mon[40196]: osdmap e377: 8 total, 7 up, 6 in 2024-09-10T19:03:06.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:06 smithi018 ceph-mon[29336]: pgmap v824: 168 pgs: 168 active+clean; 600 KiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T19:03:06.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:06 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T19:03:06.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:06 smithi018 ceph-mon[29336]: osdmap e377: 8 total, 7 up, 6 in 2024-09-10T19:03:06.952 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:06.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:06.953 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:07.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:06 smithi103 ceph-mon[33163]: pgmap v824: 168 pgs: 168 active+clean; 600 KiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T19:03:07.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:06 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T19:03:07.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:06 smithi103 ceph-mon[33163]: osdmap e377: 8 total, 7 up, 6 in 2024-09-10T19:03:07.054 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:07.080 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:07.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:07.081 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:07.182 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:07.204 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:07.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:07.205 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:07.305 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:07.327 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:07.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:07.328 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:07.429 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:07.450 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:07.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:07.451 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:07.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:03:07 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:03:07.552 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:07.579 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:07.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:07.580 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:07.681 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:07.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:07.704 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:07.704 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:07.805 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:07.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:07.827 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:07.828 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:07.929 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:07.956 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:07.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:07.957 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:08.058 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:08.085 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:08.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:08.086 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:08.127 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-10T19:03:08.128 INFO:tasks.radosbench:Beginning radosbench... 2024-09-10T19:03:08.128 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_4 2024-09-10T19:03:08.128 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd pool create unique_pool_4 16 2024-09-10T19:03:08.188 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:08.234 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:08.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:08.235 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:08.336 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:08.359 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:08.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:08.360 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:08.461 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:08.483 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:08.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:08.484 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:08.585 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:08.608 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:08.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:08.609 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:08.710 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:08.733 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:08.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:08.735 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:08.835 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:08.858 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:08.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:08.859 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:08.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:08 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:08.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:08 smithi018 ceph-mon[40196]: pgmap v826: 152 pgs: 152 active+clean; 600 KiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T19:03:08.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:08 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:08.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:08 smithi018 ceph-mon[29336]: pgmap v826: 152 pgs: 152 active+clean; 600 KiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T19:03:08.960 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:08.985 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:08.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:08.988 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:09.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:08 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:09.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:08 smithi103 ceph-mon[33163]: pgmap v826: 152 pgs: 152 active+clean; 600 KiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T19:03:09.090 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:09.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:09.114 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:09.114 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:09.215 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:09.237 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:09.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:09.238 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:09.339 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:09.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:09.362 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:09.363 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:09.463 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:09.490 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:09.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:09.491 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:09.591 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:09.614 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:09.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:09.615 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:09.716 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:09.742 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:09.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:09.743 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:09.750 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:03:09.751 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:03:09.768 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T19:03:09.768 INFO:teuthology.orchestra.run.smithi018.stdout: "ops": [], 2024-09-10T19:03:09.768 INFO:teuthology.orchestra.run.smithi018.stdout: "num_ops": 0 2024-09-10T19:03:09.768 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T19:03:09.844 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:09.870 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:09.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:09.871 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:09.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:09 smithi018 ceph-mon[29336]: pgmap v827: 152 pgs: 152 active+clean; 600 KiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T19:03:09.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:09 smithi018 ceph-mon[40196]: pgmap v827: 152 pgs: 152 active+clean; 600 KiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T19:03:09.972 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:09.998 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:09.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:09.999 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:10.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:09 smithi103 ceph-mon[33163]: pgmap v827: 152 pgs: 152 active+clean; 600 KiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T19:03:10.100 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:10.124 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:10.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:10.125 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:10.226 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:10.248 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:10.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:10.250 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:10.348 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:03:10 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T19:03:10.094+0000 7f66d46e1640 -1 mgr get_metadata_python Requested missing service osd.7 2024-09-10T19:03:10.351 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:10.373 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:10.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:10.374 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:10.475 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:10.501 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:10.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:10.502 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:10.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:03:10 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:03:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:03:10.602 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:10.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:10.626 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:10.627 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:10.727 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:10.749 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:10.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:10.750 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:10.851 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:10.877 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:10.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:10.878 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:10.979 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:11.006 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:11.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:11.007 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:11.097 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is set 2024-09-10T19:03:11.107 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:11.131 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:11.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:11.132 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:11.233 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:11.255 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:11.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:11.257 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:11.357 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:11.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:11.385 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:11.385 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:11.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:11 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:03:11.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:11 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:03:11.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:11 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1060567394' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:03:11.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:11 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:03:11.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:11 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:03:11.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:11 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:03:11.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:11 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1060567394' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:03:11.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:11 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:03:11.445 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:03:10 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:03:10] "GET /metrics HTTP/1.1" 200 207069 "" "Prometheus/2.43.0" 2024-09-10T19:03:11.487 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:11.509 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:11.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:11.510 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:11.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:11 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:03:11.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:11 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:03:11.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:11 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1060567394' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:03:11.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:11 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:03:11.611 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:11.633 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:11.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:11.634 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:11.735 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:11.758 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:11.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:11.759 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:11.860 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:11.883 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:11.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:11.884 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:11.984 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:12.010 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:12.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:12.011 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:12.112 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:12.137 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:12.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:12.138 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:12.238 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:12.262 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:12.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:12.263 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:12.364 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:12.387 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:12.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:12.388 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:12.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:12 smithi018 ceph-mon[40196]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:03:12.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:12 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T19:03:12.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:12 smithi018 ceph-mon[40196]: osdmap e378: 8 total, 7 up, 6 in 2024-09-10T19:03:12.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:12 smithi018 ceph-mon[40196]: pgmap v829: 152 pgs: 152 active+clean; 600 KiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 511 B/s rd, 0 op/s 2024-09-10T19:03:12.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:12 smithi018 ceph-mon[29336]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:03:12.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:12 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T19:03:12.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:12 smithi018 ceph-mon[29336]: osdmap e378: 8 total, 7 up, 6 in 2024-09-10T19:03:12.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:12 smithi018 ceph-mon[29336]: pgmap v829: 152 pgs: 152 active+clean; 600 KiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 511 B/s rd, 0 op/s 2024-09-10T19:03:12.490 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:12.513 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:12.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:12.514 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:12.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:12 smithi103 ceph-mon[33163]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:03:12.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:12 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T19:03:12.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:12 smithi103 ceph-mon[33163]: osdmap e378: 8 total, 7 up, 6 in 2024-09-10T19:03:12.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:12 smithi103 ceph-mon[33163]: pgmap v829: 152 pgs: 152 active+clean; 600 KiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 511 B/s rd, 0 op/s 2024-09-10T19:03:12.615 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:12.639 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:12.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:12.640 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:12.741 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:12.767 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:12.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:12.768 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:12.869 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:12.894 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:12.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:12.895 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:12.996 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:13.022 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:13.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:13.023 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:13.123 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:13.146 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:13.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:13.147 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:13.248 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:13.275 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:13.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:13.276 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:13.376 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:13.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:13.400 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:13.401 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:13.501 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:13.528 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:13.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:13.529 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:13.629 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:13.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:13.654 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:13.654 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:13.661 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-10T19:03:13.755 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:13.779 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:13.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:13.780 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:13.881 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:13.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:13.906 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:13.906 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:14.007 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:14.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:14.031 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:14.031 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:14.132 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:14.157 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:14.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:14.158 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:14.169 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:03:14.259 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:14.271 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-10T19:03:14.291 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:14.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:14.292 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:14.393 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:14.416 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:14.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:14.417 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:14.518 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:14.544 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:14.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:14.545 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:14.645 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:14.670 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:14.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:14.671 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:14.771 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:14.796 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:14.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:14.797 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:14.816 INFO:teuthology.orchestra.run.smithi018.stderr:pool 'unique_pool_4' created 2024-09-10T19:03:14.875 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:14 smithi018 ceph-mon[40196]: pgmap v830: 152 pgs: 152 active+clean; 600 KiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T19:03:14.876 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:14 smithi018 ceph-mon[29336]: pgmap v830: 152 pgs: 152 active+clean; 600 KiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T19:03:14.897 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:14.923 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:14.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:14.924 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:15.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:14 smithi103 ceph-mon[33163]: pgmap v830: 152 pgs: 152 active+clean; 600 KiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T19:03:15.024 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:15.047 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:15.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:15.048 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:15.149 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:15.172 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:15.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:15.173 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:15.274 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:15.296 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:15.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:15.297 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:15.398 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:15.420 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:15.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:15.421 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:15.522 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:15.544 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:15.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:15.546 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:15.646 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:15.673 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:15.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:15.674 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:15.775 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:15.798 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:15.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:15.799 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:15.900 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:15.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:15.927 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:15.927 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:15.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:15 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1393815442' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]: dispatch 2024-09-10T19:03:15.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:15 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]: dispatch 2024-09-10T19:03:15.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:15 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]': finished 2024-09-10T19:03:15.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:15 smithi018 ceph-mon[40196]: osdmap e379: 8 total, 7 up, 6 in 2024-09-10T19:03:15.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:15 smithi018 ceph-mon[40196]: pgmap v832: 168 pgs: 16 unknown, 152 active+clean; 600 KiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T19:03:15.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:15 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1393815442' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]: dispatch 2024-09-10T19:03:15.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:15 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]: dispatch 2024-09-10T19:03:15.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:15 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]': finished 2024-09-10T19:03:15.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:15 smithi018 ceph-mon[29336]: osdmap e379: 8 total, 7 up, 6 in 2024-09-10T19:03:15.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:15 smithi018 ceph-mon[29336]: pgmap v832: 168 pgs: 16 unknown, 152 active+clean; 600 KiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T19:03:16.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:15 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1393815442' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]: dispatch 2024-09-10T19:03:16.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:15 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]: dispatch 2024-09-10T19:03:16.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:15 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]': finished 2024-09-10T19:03:16.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:15 smithi103 ceph-mon[33163]: osdmap e379: 8 total, 7 up, 6 in 2024-09-10T19:03:16.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:15 smithi103 ceph-mon[33163]: pgmap v832: 168 pgs: 16 unknown, 152 active+clean; 600 KiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T19:03:16.028 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:16.059 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd pool application enable unique_pool_4 rados --yes-i-really-mean-it || true 2024-09-10T19:03:16.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:16.060 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:16.060 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:16.161 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:16.185 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:16.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:16.186 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:16.287 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:16.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:16.316 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:16.316 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:16.417 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:16.443 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:16.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:16.444 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:16.545 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:16.563 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T19:03:16.564 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T19:03:16.570 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:16.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:16.571 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:16.579 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T19:03:16.579 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T19:03:16.594 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T19:03:16.594 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T19:03:16.609 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T19:03:16.610 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T19:03:16.625 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T19:03:16.626 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T19:03:16.641 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T19:03:16.642 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T19:03:16.657 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T19:03:16.657 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T19:03:16.658 INFO:teuthology.orchestra.run.smithi018.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-10T19:03:16.672 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:16.696 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:16.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:16.697 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:16.798 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:16.822 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:16.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:16.823 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:16.924 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:16.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:16 smithi018 ceph-mon[29336]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:03:16.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:16 smithi018 ceph-mon[29336]: osdmap e380: 8 total, 7 up, 6 in 2024-09-10T19:03:16.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:16 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1506809313' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:03:16.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:16 smithi018 ceph-mon[40196]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:03:16.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:16 smithi018 ceph-mon[40196]: osdmap e380: 8 total, 7 up, 6 in 2024-09-10T19:03:16.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:16 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1506809313' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:03:16.947 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:16.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:16.948 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:17.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:16 smithi103 ceph-mon[33163]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:03:17.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:16 smithi103 ceph-mon[33163]: osdmap e380: 8 total, 7 up, 6 in 2024-09-10T19:03:17.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:16 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1506809313' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:03:17.048 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:17.070 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:17.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:17.071 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:17.172 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:17.194 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:17.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:17.195 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:17.296 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:17.317 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:17.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:17.318 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:17.392 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-10T19:03:17.393 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-10T19:03:17.419 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:17.440 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:17.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:17.441 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:17.449 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:03:17 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:03:17.542 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:17.564 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:17.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:17.565 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:17.666 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:17.689 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:17.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:17.690 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:17.790 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:17.812 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:17.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:17.814 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:17.846 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:03:17.882 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:03:17.914 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:17.936 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:17.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:17.937 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:18.038 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:18.060 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:18.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:18.061 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:18.084 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:17 smithi018 ceph-mon[40196]: osdmap e381: 8 total, 7 up, 6 in 2024-09-10T19:03:18.085 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:17 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:18.085 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:17 smithi018 ceph-mon[40196]: pgmap v835: 168 pgs: 2 unknown, 166 active+clean; 600 KiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T19:03:18.085 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:17 smithi018 ceph-mon[29336]: osdmap e381: 8 total, 7 up, 6 in 2024-09-10T19:03:18.085 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:17 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:18.085 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:17 smithi018 ceph-mon[29336]: pgmap v835: 168 pgs: 2 unknown, 166 active+clean; 600 KiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T19:03:18.163 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:18.186 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:18.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:18.187 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:18.196 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:17 smithi103 ceph-mon[33163]: osdmap e381: 8 total, 7 up, 6 in 2024-09-10T19:03:18.197 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:17 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:18.197 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:17 smithi103 ceph-mon[33163]: pgmap v835: 168 pgs: 2 unknown, 166 active+clean; 600 KiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T19:03:18.287 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:18.310 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:18.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:18.311 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:18.412 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:18.436 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:18.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:18.437 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:18.538 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:18.561 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:18.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:18.562 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:18.662 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:18.685 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:18.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:18.686 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:18.786 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:18.807 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:18.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:18.808 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:18.909 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:18.929 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:18 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:03:18.929 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:18 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:03:18.929 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:18 smithi018 ceph-mon[40196]: osdmap e382: 8 total, 7 up, 6 in 2024-09-10T19:03:18.930 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:18 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:03:18.930 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:18 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:03:18.930 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:18 smithi018 ceph-mon[29336]: osdmap e382: 8 total, 7 up, 6 in 2024-09-10T19:03:18.930 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:18.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:18.931 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:19.032 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:19.056 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:19.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:19.057 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:19.157 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:19.181 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:19.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:19.182 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:19.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:18 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:03:19.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:18 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:03:19.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:18 smithi103 ceph-mon[33163]: osdmap e382: 8 total, 7 up, 6 in 2024-09-10T19:03:19.283 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:19.305 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:19.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:19.306 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:19.393 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T19:03:19.407 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:19.480 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:19.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:19.481 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:19.582 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:19.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:19.606 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:19.606 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:19.707 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:19.730 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:19.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:19.731 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:19.831 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:19.856 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:19.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:19.857 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:19.957 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:19.980 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:19.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:19.981 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:20.082 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:20.104 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:20.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:20.106 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:20.131 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:19 smithi018 ceph-mon[40196]: pgmap v837: 168 pgs: 168 active+clean; 600 KiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T19:03:20.132 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:19 smithi018 ceph-mon[29336]: pgmap v837: 168 pgs: 168 active+clean; 600 KiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T19:03:20.206 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:20.229 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:20.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:20.230 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:20.231 INFO:teuthology.orchestra.run.smithi018.stdout:ERROR: (22) Invalid argument 2024-09-10T19:03:20.231 INFO:teuthology.orchestra.run.smithi018.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T19:03:20.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:19 smithi103 ceph-mon[33163]: pgmap v837: 168 pgs: 168 active+clean; 600 KiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T19:03:20.330 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:20.357 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:20.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:20.358 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:20.458 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:20.483 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:20.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:20.484 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:20.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:03:20 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:03:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:03:20.584 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:20.610 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:20.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:20.610 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:20.711 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:20.737 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:20.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:20.738 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:20.838 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:20.862 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:20.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:20.863 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:20.962 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-10T19:03:20.964 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:20.986 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:20.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:20.987 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:21.088 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:21.111 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:21.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:21.112 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:21.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:03:20 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:03:20] "GET /metrics HTTP/1.1" 200 207069 "" "Prometheus/2.43.0" 2024-09-10T19:03:21.213 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:21.235 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:21.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:21.236 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:21.337 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:21.358 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:21.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:21.359 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:21.434 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is unset 2024-09-10T19:03:21.461 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:21.486 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:21.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:21.487 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:21.588 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:21.612 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:21.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:21.613 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:21.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:21 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/196247517' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:03:21.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:21 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:03:21.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:21 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/196247517' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:03:21.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:21 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:03:21.713 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:21.736 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:21.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:21.736 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:21.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:21 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/196247517' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:03:21.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:21 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:03:21.838 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:21.863 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:21.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:21.864 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:21.964 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:21.988 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:21.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:21.989 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:22.090 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:22.113 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:22.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:22.114 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:22.215 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:22.238 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:22.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:22.239 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:22.340 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:22.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:22.364 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:22.364 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:22.465 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:22.488 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:22.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:22.489 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:22.590 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:22.613 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:22.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:22.614 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:22.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:22 smithi018 ceph-mon[40196]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:03:22.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:22 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T19:03:22.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:22 smithi018 ceph-mon[40196]: osdmap e383: 8 total, 7 up, 6 in 2024-09-10T19:03:22.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:22 smithi018 ceph-mon[40196]: pgmap v839: 168 pgs: 168 active+clean; 600 KiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 911 B/s rd, 0 op/s 2024-09-10T19:03:22.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:22 smithi018 ceph-mon[40196]: 3.1 scrub starts 2024-09-10T19:03:22.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:22 smithi018 ceph-mon[40196]: 3.1 scrub ok 2024-09-10T19:03:22.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:22 smithi018 ceph-mon[29336]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:03:22.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:22 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T19:03:22.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:22 smithi018 ceph-mon[29336]: osdmap e383: 8 total, 7 up, 6 in 2024-09-10T19:03:22.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:22 smithi018 ceph-mon[29336]: pgmap v839: 168 pgs: 168 active+clean; 600 KiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 911 B/s rd, 0 op/s 2024-09-10T19:03:22.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:22 smithi018 ceph-mon[29336]: 3.1 scrub starts 2024-09-10T19:03:22.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:22 smithi018 ceph-mon[29336]: 3.1 scrub ok 2024-09-10T19:03:22.715 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:22.737 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:22.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:22.738 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:22.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:22 smithi103 ceph-mon[33163]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:03:22.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:22 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T19:03:22.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:22 smithi103 ceph-mon[33163]: osdmap e383: 8 total, 7 up, 6 in 2024-09-10T19:03:22.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:22 smithi103 ceph-mon[33163]: pgmap v839: 168 pgs: 168 active+clean; 600 KiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 911 B/s rd, 0 op/s 2024-09-10T19:03:22.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:22 smithi103 ceph-mon[33163]: 3.1 scrub starts 2024-09-10T19:03:22.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:22 smithi103 ceph-mon[33163]: 3.1 scrub ok 2024-09-10T19:03:22.839 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:22.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:22.863 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:22.863 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:22.964 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:22.986 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:22.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:22.987 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:23.029 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:03:23.030 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:03:23.087 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:23.109 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:23.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:23.110 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:23.211 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:23.232 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:23.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:23.233 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:23.334 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:23.356 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:23.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:23.357 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:23.446 INFO:teuthology.orchestra.run.smithi018.stderr:enabled application 'rados' on pool 'unique_pool_4' 2024-09-10T19:03:23.458 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:23.479 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:23.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:23.480 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:23.493 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:23 smithi018 ceph-mon[40196]: 1.8 scrub starts 2024-09-10T19:03:23.493 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:23 smithi018 ceph-mon[40196]: 1.8 scrub ok 2024-09-10T19:03:23.493 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:23 smithi018 ceph-mon[40196]: 3.2 scrub starts 2024-09-10T19:03:23.493 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:23 smithi018 ceph-mon[40196]: 3.2 scrub ok 2024-09-10T19:03:23.493 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:23 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2334577258' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:03:23.493 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:23 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:03:23.493 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:23 smithi018 ceph-mon[40196]: 6.10 scrub starts 2024-09-10T19:03:23.493 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:23 smithi018 ceph-mon[40196]: 6.10 scrub ok 2024-09-10T19:03:23.493 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:23 smithi018 ceph-mon[29336]: 1.8 scrub starts 2024-09-10T19:03:23.493 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:23 smithi018 ceph-mon[29336]: 1.8 scrub ok 2024-09-10T19:03:23.494 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:23 smithi018 ceph-mon[29336]: 3.2 scrub starts 2024-09-10T19:03:23.494 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:23 smithi018 ceph-mon[29336]: 3.2 scrub ok 2024-09-10T19:03:23.494 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:23 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2334577258' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:03:23.494 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:23 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:03:23.494 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:23 smithi018 ceph-mon[29336]: 6.10 scrub starts 2024-09-10T19:03:23.494 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:23 smithi018 ceph-mon[29336]: 6.10 scrub ok 2024-09-10T19:03:23.581 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:23.603 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:23.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:23.604 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:23.704 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:23.726 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:23.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:23.727 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:23.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:23 smithi103 ceph-mon[33163]: 1.8 scrub starts 2024-09-10T19:03:23.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:23 smithi103 ceph-mon[33163]: 1.8 scrub ok 2024-09-10T19:03:23.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:23 smithi103 ceph-mon[33163]: 3.2 scrub starts 2024-09-10T19:03:23.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:23 smithi103 ceph-mon[33163]: 3.2 scrub ok 2024-09-10T19:03:23.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:23 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2334577258' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:03:23.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:23 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:03:23.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:23 smithi103 ceph-mon[33163]: 6.10 scrub starts 2024-09-10T19:03:23.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:23 smithi103 ceph-mon[33163]: 6.10 scrub ok 2024-09-10T19:03:23.828 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:23.850 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:23.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:23.851 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:23.951 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:23.974 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:23.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:23.975 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:24.075 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:24.097 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:24.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:24.098 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:24.199 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:24.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:24.222 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:24.222 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:24.323 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:24.347 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:24.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:24.348 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:24.448 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:24.470 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:24.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:24.471 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:24.572 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:24.594 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:24.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:24.595 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:24.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:24 smithi018 ceph-mon[40196]: pgmap v840: 168 pgs: 168 active+clean; 600 KiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 774 B/s rd, 0 op/s 2024-09-10T19:03:24.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:24 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-10T19:03:24.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:24 smithi018 ceph-mon[40196]: osdmap e384: 8 total, 7 up, 6 in 2024-09-10T19:03:24.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:24 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:03:24.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:24 smithi018 ceph-mon[29336]: pgmap v840: 168 pgs: 168 active+clean; 600 KiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 774 B/s rd, 0 op/s 2024-09-10T19:03:24.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:24 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-10T19:03:24.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:24 smithi018 ceph-mon[29336]: osdmap e384: 8 total, 7 up, 6 in 2024-09-10T19:03:24.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:24 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:03:24.696 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:24.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:24.719 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:24.719 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:24.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:24 smithi103 ceph-mon[33163]: pgmap v840: 168 pgs: 168 active+clean; 600 KiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 774 B/s rd, 0 op/s 2024-09-10T19:03:24.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:24 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-10T19:03:24.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:24 smithi103 ceph-mon[33163]: osdmap e384: 8 total, 7 up, 6 in 2024-09-10T19:03:24.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:24 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:03:24.820 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:24.842 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:24.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:24.843 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:24.944 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:24.965 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:24.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:24.966 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:24.969 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-10T19:03:25.067 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:25.089 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:25.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:25.090 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:25.191 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:25.213 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:25.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:25.214 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:25.315 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:25.321 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:03:25 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T19:03:25.093+0000 7f66d46e1640 -1 mgr get_metadata_python Requested missing service osd.7 2024-09-10T19:03:25.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:25.338 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:25.338 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:25.439 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:25.460 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:25.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:25.461 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:25.562 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:25.583 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:25.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:25.585 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:25.685 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:25.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:25 smithi018 ceph-mon[29336]: 6.17 scrub starts 2024-09-10T19:03:25.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:25 smithi018 ceph-mon[29336]: 6.17 scrub ok 2024-09-10T19:03:25.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:25 smithi018 ceph-mon[29336]: 6.19 scrub starts 2024-09-10T19:03:25.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:25 smithi018 ceph-mon[29336]: 6.19 scrub ok 2024-09-10T19:03:25.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:25 smithi018 ceph-mon[29336]: 5.1f scrub starts 2024-09-10T19:03:25.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:25 smithi018 ceph-mon[29336]: 5.1f scrub ok 2024-09-10T19:03:25.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:25 smithi018 ceph-mon[29336]: 6.c scrub starts 2024-09-10T19:03:25.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:25 smithi018 ceph-mon[29336]: 6.c scrub ok 2024-09-10T19:03:25.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:25 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:03:25.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:25 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:03:25.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:25 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:03:25.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:25 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:03:25.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:25 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:03:25.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:25 smithi018 ceph-mon[40196]: 6.17 scrub starts 2024-09-10T19:03:25.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:25 smithi018 ceph-mon[40196]: 6.17 scrub ok 2024-09-10T19:03:25.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:25 smithi018 ceph-mon[40196]: 6.19 scrub starts 2024-09-10T19:03:25.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:25 smithi018 ceph-mon[40196]: 6.19 scrub ok 2024-09-10T19:03:25.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:25 smithi018 ceph-mon[40196]: 5.1f scrub starts 2024-09-10T19:03:25.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:25 smithi018 ceph-mon[40196]: 5.1f scrub ok 2024-09-10T19:03:25.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:25 smithi018 ceph-mon[40196]: 6.c scrub starts 2024-09-10T19:03:25.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:25 smithi018 ceph-mon[40196]: 6.c scrub ok 2024-09-10T19:03:25.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:25 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:03:25.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:25 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:03:25.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:25 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:03:25.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:25 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:03:25.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:25 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:03:25.710 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:25.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:25.710 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:25.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:25 smithi103 ceph-mon[33163]: 6.17 scrub starts 2024-09-10T19:03:25.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:25 smithi103 ceph-mon[33163]: 6.17 scrub ok 2024-09-10T19:03:25.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:25 smithi103 ceph-mon[33163]: 6.19 scrub starts 2024-09-10T19:03:25.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:25 smithi103 ceph-mon[33163]: 6.19 scrub ok 2024-09-10T19:03:25.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:25 smithi103 ceph-mon[33163]: 5.1f scrub starts 2024-09-10T19:03:25.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:25 smithi103 ceph-mon[33163]: 5.1f scrub ok 2024-09-10T19:03:25.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:25 smithi103 ceph-mon[33163]: 6.c scrub starts 2024-09-10T19:03:25.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:25 smithi103 ceph-mon[33163]: 6.c scrub ok 2024-09-10T19:03:25.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:25 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:03:25.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:25 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:03:25.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:25 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:03:25.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:25 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:03:25.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:25 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:03:25.812 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:25.834 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:25.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:25.835 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:25.935 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:25.957 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:25.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:25.958 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:26.059 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:26.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:26.083 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:26.083 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:26.184 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:26.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:26.207 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:26.207 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:26.308 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:26.344 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:26.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:26.345 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:26.445 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:26.467 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T19:03:26.468 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T19:03:26.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:26.468 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:26.469 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:26.476 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T19:03:26.476 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T19:03:26.491 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T19:03:26.491 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T19:03:26.506 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T19:03:26.506 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T19:03:26.506 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:26 smithi018 ceph-mon[40196]: pgmap v842: 168 pgs: 168 active+clean; 600 KiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 672 B/s rd, 0 op/s 2024-09-10T19:03:26.507 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:26 smithi018 ceph-mon[40196]: 5.1e scrub starts 2024-09-10T19:03:26.507 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:26 smithi018 ceph-mon[40196]: 5.1e scrub ok 2024-09-10T19:03:26.507 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:26 smithi018 ceph-mon[40196]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T19:03:26.507 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:26 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/521644435' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:03:26.507 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:26 smithi018 ceph-mon[29336]: pgmap v842: 168 pgs: 168 active+clean; 600 KiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 672 B/s rd, 0 op/s 2024-09-10T19:03:26.507 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:26 smithi018 ceph-mon[29336]: 5.1e scrub starts 2024-09-10T19:03:26.507 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:26 smithi018 ceph-mon[29336]: 5.1e scrub ok 2024-09-10T19:03:26.507 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:26 smithi018 ceph-mon[29336]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T19:03:26.507 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:26 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/521644435' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:03:26.522 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T19:03:26.522 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T19:03:26.538 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T19:03:26.538 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T19:03:26.553 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T19:03:26.554 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T19:03:26.554 INFO:teuthology.orchestra.run.smithi018.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-10T19:03:26.569 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:26.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:26.593 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:26.593 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:26.694 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:26.717 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:26.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:26.719 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:26.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:26 smithi103 ceph-mon[33163]: pgmap v842: 168 pgs: 168 active+clean; 600 KiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 672 B/s rd, 0 op/s 2024-09-10T19:03:26.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:26 smithi103 ceph-mon[33163]: 5.1e scrub starts 2024-09-10T19:03:26.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:26 smithi103 ceph-mon[33163]: 5.1e scrub ok 2024-09-10T19:03:26.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:26 smithi103 ceph-mon[33163]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T19:03:26.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:26 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/521644435' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:03:26.819 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:26.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:26.843 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:26.844 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:26.944 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:26.968 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:26.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:26.969 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:27.070 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:27.093 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:27.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:27.094 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:27.195 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:27.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:27.219 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:27.219 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:27.320 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:27.343 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:27.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:27.344 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:27.445 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:27.450 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:03:27 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:03:27.468 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:27.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:27.470 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:27.570 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:27.597 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:27.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:27.598 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:27.601 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:27 smithi018 ceph-mon[40196]: 5.8 scrub starts 2024-09-10T19:03:27.601 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:27 smithi018 ceph-mon[40196]: 5.8 scrub ok 2024-09-10T19:03:27.601 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:27 smithi018 ceph-mon[29336]: 5.8 scrub starts 2024-09-10T19:03:27.601 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:27 smithi018 ceph-mon[29336]: 5.8 scrub ok 2024-09-10T19:03:27.626 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-10T19:03:27.626 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-10T19:03:27.699 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:27.705 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:27 smithi103 ceph-mon[33163]: 5.8 scrub starts 2024-09-10T19:03:27.705 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:27 smithi103 ceph-mon[33163]: 5.8 scrub ok 2024-09-10T19:03:27.723 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:27.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:27.724 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:27.825 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:27.828 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T19:03:27.828 INFO:teuthology.orchestra.run.smithi018.stdout: "size": 20, 2024-09-10T19:03:27.828 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 600, 2024-09-10T19:03:27.828 INFO:teuthology.orchestra.run.smithi018.stdout: "ops": [ 2024-09-10T19:03:27.828 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.829 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15354.0:178424 9.5 9:afd06e58:::benchmark_data_smithi018_133594_object41672:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e287)", 2024-09-10T19:03:27.829 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:58:06.207518+0000", 2024-09-10T19:03:27.829 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 321.61647788300002, 2024-09-10T19:03:27.829 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.006247493, 2024-09-10T19:03:27.829 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:03:27.829 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:03:27.829 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:03:27.829 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15354", 2024-09-10T19:03:27.829 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:03:27.829 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 178424 2024-09-10T19:03:27.829 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.829 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:03:27.829 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.830 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:03:27.830 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:06.207518+0000", 2024-09-10T19:03:27.830 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.830 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.830 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.830 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:03:27.830 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:06.207518+0000", 2024-09-10T19:03:27.830 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.830 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.830 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.830 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:03:27.830 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:06.207516+0000", 2024-09-10T19:03:27.831 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999981 2024-09-10T19:03:27.831 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.831 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.831 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:03:27.831 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:06.207524+0000", 2024-09-10T19:03:27.831 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.7130000000000005e-06 2024-09-10T19:03:27.831 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.831 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.831 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:03:27.831 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:06.207540+0000", 2024-09-10T19:03:27.831 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.5605999999999999e-05 2024-09-10T19:03:27.831 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.831 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.832 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:03:27.832 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:06.207547+0000", 2024-09-10T19:03:27.832 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.8349999999999998e-06 2024-09-10T19:03:27.832 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.832 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.832 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.832 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:06.207640+0000", 2024-09-10T19:03:27.832 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.3074999999999999e-05 2024-09-10T19:03:27.832 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.832 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.832 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:03:27.832 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:06.207647+0000", 2024-09-10T19:03:27.832 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.6739999999999997e-06 2024-09-10T19:03:27.833 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.833 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.833 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.833 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:07.181908+0000", 2024-09-10T19:03:27.833 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.97426098699999997 2024-09-10T19:03:27.833 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.833 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.833 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:03:27.833 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:07.182044+0000", 2024-09-10T19:03:27.833 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00013536399999999999 2024-09-10T19:03:27.833 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.833 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.834 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,3", 2024-09-10T19:03:27.834 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:07.182176+0000", 2024-09-10T19:03:27.834 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00013270900000000001 2024-09-10T19:03:27.834 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.834 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.834 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:03:27.834 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:07.205499+0000", 2024-09-10T19:03:27.834 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.023322925000000001 2024-09-10T19:03:27.834 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.834 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.834 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.834 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:07.213654+0000", 2024-09-10T19:03:27.835 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0081547159999999994 2024-09-10T19:03:27.835 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.835 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.835 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.835 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:07.213685+0000", 2024-09-10T19:03:27.835 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.1078999999999998e-05 2024-09-10T19:03:27.835 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.835 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.835 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:03:27.835 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:07.213732+0000", 2024-09-10T19:03:27.835 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.6916000000000003e-05 2024-09-10T19:03:27.835 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.835 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.836 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:03:27.836 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:07.213766+0000", 2024-09-10T19:03:27.836 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.3973999999999997e-05 2024-09-10T19:03:27.836 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.836 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:03:27.836 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.836 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.836 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.836 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15354.0:178437 9.5 9:a4beb136:::benchmark_data_smithi018_133594_object41685:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e287)", 2024-09-10T19:03:27.836 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:58:06.219911+0000", 2024-09-10T19:03:27.836 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 321.60408532299999, 2024-09-10T19:03:27.836 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.99401177799999996, 2024-09-10T19:03:27.836 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:03:27.836 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:03:27.837 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:03:27.837 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15354", 2024-09-10T19:03:27.837 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:03:27.837 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 178437 2024-09-10T19:03:27.837 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.837 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:03:27.837 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.837 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:03:27.837 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:06.219911+0000", 2024-09-10T19:03:27.837 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.837 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.837 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.837 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:03:27.838 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:06.219911+0000", 2024-09-10T19:03:27.838 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.838 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.838 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.838 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:03:27.838 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:06.219910+0000", 2024-09-10T19:03:27.838 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T19:03:27.838 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.838 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.838 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:03:27.838 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:06.219915+0000", 2024-09-10T19:03:27.838 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.2460000000000003e-06 2024-09-10T19:03:27.838 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.838 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.838 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:03:27.839 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:06.219925+0000", 2024-09-10T19:03:27.839 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0552e-05 2024-09-10T19:03:27.839 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.839 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.839 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:03:27.839 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:06.219932+0000", 2024-09-10T19:03:27.839 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.8519999999999998e-06 2024-09-10T19:03:27.839 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.839 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.839 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.839 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:06.221812+0000", 2024-09-10T19:03:27.839 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.001880121 2024-09-10T19:03:27.839 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.839 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.839 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:03:27.840 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:06.221821+0000", 2024-09-10T19:03:27.840 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.5480000000000002e-06 2024-09-10T19:03:27.840 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.840 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.840 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.840 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:07.184011+0000", 2024-09-10T19:03:27.840 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.96219025999999996 2024-09-10T19:03:27.840 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.840 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.840 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:03:27.840 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:07.184131+0000", 2024-09-10T19:03:27.840 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000119535 2024-09-10T19:03:27.840 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.840 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.841 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,3", 2024-09-10T19:03:27.841 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:07.184256+0000", 2024-09-10T19:03:27.841 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00012562900000000001 2024-09-10T19:03:27.841 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.841 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.841 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:03:27.841 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:07.205503+0000", 2024-09-10T19:03:27.841 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.021246389000000001 2024-09-10T19:03:27.841 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.841 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.841 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.841 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:07.213798+0000", 2024-09-10T19:03:27.841 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0082950000000000003 2024-09-10T19:03:27.841 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.842 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.842 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.842 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:07.213849+0000", 2024-09-10T19:03:27.842 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.1263e-05 2024-09-10T19:03:27.842 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.842 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.842 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:03:27.842 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:07.213888+0000", 2024-09-10T19:03:27.842 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.9215000000000003e-05 2024-09-10T19:03:27.842 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.842 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.842 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:03:27.842 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:07.213923+0000", 2024-09-10T19:03:27.843 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.4563000000000001e-05 2024-09-10T19:03:27.843 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.843 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:03:27.843 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.843 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.843 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.843 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15354.0:178452 9.5 9:aad22c13:::benchmark_data_smithi018_133594_object41700:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e287)", 2024-09-10T19:03:27.843 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:58:06.232776+0000", 2024-09-10T19:03:27.843 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 321.591220185, 2024-09-10T19:03:27.843 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.98126763900000002, 2024-09-10T19:03:27.843 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:03:27.843 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:03:27.844 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:03:27.844 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15354", 2024-09-10T19:03:27.844 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:03:27.844 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 178452 2024-09-10T19:03:27.844 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.844 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:03:27.844 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.844 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:03:27.844 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:06.232776+0000", 2024-09-10T19:03:27.844 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.844 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.844 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.844 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:03:27.845 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:06.232776+0000", 2024-09-10T19:03:27.845 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.845 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.845 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.845 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:03:27.845 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:06.232776+0000", 2024-09-10T19:03:27.845 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T19:03:27.845 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.845 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.845 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:03:27.845 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:06.232778+0000", 2024-09-10T19:03:27.845 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.52e-06 2024-09-10T19:03:27.845 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.845 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.845 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:03:27.846 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:06.232784+0000", 2024-09-10T19:03:27.846 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.2650000000000004e-06 2024-09-10T19:03:27.846 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.846 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.846 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:03:27.846 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:06.232788+0000", 2024-09-10T19:03:27.846 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.9260000000000002e-06 2024-09-10T19:03:27.846 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.846 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.846 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.846 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:06.232850+0000", 2024-09-10T19:03:27.846 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.2144000000000007e-05 2024-09-10T19:03:27.846 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.846 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.846 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:03:27.847 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:06.232857+0000", 2024-09-10T19:03:27.847 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.6960000000000005e-06 2024-09-10T19:03:27.847 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.847 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.847 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.847 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:07.186046+0000", 2024-09-10T19:03:27.847 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.95318831599999998 2024-09-10T19:03:27.847 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.847 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.847 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:03:27.847 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:07.186160+0000", 2024-09-10T19:03:27.847 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00011414 2024-09-10T19:03:27.847 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.847 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.847 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,3", 2024-09-10T19:03:27.848 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:07.186282+0000", 2024-09-10T19:03:27.848 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00012242899999999999 2024-09-10T19:03:27.848 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.848 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.848 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:03:27.848 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:07.207297+0000", 2024-09-10T19:03:27.848 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.021014545999999999 2024-09-10T19:03:27.848 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.848 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.848 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.848 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:07.213822+0000", 2024-09-10T19:03:27.848 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0065253860000000002 2024-09-10T19:03:27.848 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.848 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.849 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.849 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:07.213957+0000", 2024-09-10T19:03:27.849 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000134664 2024-09-10T19:03:27.849 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.849 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.849 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:03:27.849 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:07.214010+0000", 2024-09-10T19:03:27.849 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.3371000000000001e-05 2024-09-10T19:03:27.849 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.849 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.849 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:03:27.849 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:07.214044+0000", 2024-09-10T19:03:27.849 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.3581999999999998e-05 2024-09-10T19:03:27.849 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.849 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:03:27.850 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.850 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.850 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.850 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15354.0:185477 9.9 9:949adb73:::benchmark_data_smithi018_133594_object48725:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e291)", 2024-09-10T19:03:27.850 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:58:14.725255+0000", 2024-09-10T19:03:27.850 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 313.098741141, 2024-09-10T19:03:27.850 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0443514009999999, 2024-09-10T19:03:27.850 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:03:27.850 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:03:27.850 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:03:27.850 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15354", 2024-09-10T19:03:27.850 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:03:27.850 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 185477 2024-09-10T19:03:27.850 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.851 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:03:27.851 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.851 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:03:27.851 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.725255+0000", 2024-09-10T19:03:27.851 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.851 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.851 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.851 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:03:27.851 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.725255+0000", 2024-09-10T19:03:27.851 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.851 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.851 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.852 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:03:27.852 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.725253+0000", 2024-09-10T19:03:27.852 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999981 2024-09-10T19:03:27.852 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.852 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.852 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:03:27.852 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.725258+0000", 2024-09-10T19:03:27.852 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.1789999999999999e-06 2024-09-10T19:03:27.852 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.852 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.852 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:03:27.852 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.725268+0000", 2024-09-10T19:03:27.852 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.6560000000000004e-06 2024-09-10T19:03:27.853 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.853 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.853 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:03:27.853 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.725274+0000", 2024-09-10T19:03:27.853 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.9819999999999996e-06 2024-09-10T19:03:27.853 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.853 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.853 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.853 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.725737+0000", 2024-09-10T19:03:27.853 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00046283300000000001 2024-09-10T19:03:27.853 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.853 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.853 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:03:27.854 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.725744+0000", 2024-09-10T19:03:27.854 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.832e-06 2024-09-10T19:03:27.854 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.854 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.854 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.854 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.736128+0000", 2024-09-10T19:03:27.854 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0103841280000001 2024-09-10T19:03:27.854 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.854 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.854 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:03:27.854 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.736268+0000", 2024-09-10T19:03:27.854 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00013978800000000001 2024-09-10T19:03:27.854 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.854 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.855 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,5", 2024-09-10T19:03:27.855 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.736395+0000", 2024-09-10T19:03:27.855 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000127675 2024-09-10T19:03:27.855 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.855 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.855 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:03:27.855 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.746414+0000", 2024-09-10T19:03:27.855 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.010018895 2024-09-10T19:03:27.855 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.855 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.855 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.855 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.769295+0000", 2024-09-10T19:03:27.855 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.022880701999999999 2024-09-10T19:03:27.856 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.856 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.856 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.856 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.769372+0000", 2024-09-10T19:03:27.856 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.7280999999999994e-05 2024-09-10T19:03:27.856 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.856 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.856 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:03:27.856 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.769435+0000", 2024-09-10T19:03:27.856 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.2662000000000004e-05 2024-09-10T19:03:27.856 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.857 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.857 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:03:27.857 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.769607+0000", 2024-09-10T19:03:27.857 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000171705 2024-09-10T19:03:27.857 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.857 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:03:27.857 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.857 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.857 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.858 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15354.0:185491 9.9 9:96f18129:::benchmark_data_smithi018_133594_object48739:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e291)", 2024-09-10T19:03:27.858 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:58:14.736150+0000", 2024-09-10T19:03:27.858 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 313.08784641099999, 2024-09-10T19:03:27.858 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0337192799999999, 2024-09-10T19:03:27.858 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:03:27.858 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:03:27.858 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:03:27.858 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15354", 2024-09-10T19:03:27.858 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:03:27.858 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 185491 2024-09-10T19:03:27.858 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.858 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:03:27.858 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.858 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:03:27.859 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.736150+0000", 2024-09-10T19:03:27.859 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.859 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.859 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.859 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:03:27.859 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.736150+0000", 2024-09-10T19:03:27.859 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.859 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.859 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.859 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:03:27.859 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.736148+0000", 2024-09-10T19:03:27.859 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T19:03:27.859 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.860 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.860 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:03:27.860 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.736155+0000", 2024-09-10T19:03:27.860 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.8730000000000001e-06 2024-09-10T19:03:27.860 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.860 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.860 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:03:27.860 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.736170+0000", 2024-09-10T19:03:27.860 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.4717e-05 2024-09-10T19:03:27.860 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.860 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.860 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:03:27.860 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.736177+0000", 2024-09-10T19:03:27.861 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.5679999999999996e-06 2024-09-10T19:03:27.861 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.861 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.861 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.861 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.736246+0000", 2024-09-10T19:03:27.861 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.9979000000000002e-05 2024-09-10T19:03:27.861 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.861 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.861 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:03:27.861 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.736255+0000", 2024-09-10T19:03:27.861 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.0949999999999996e-06 2024-09-10T19:03:27.861 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.861 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.862 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.862 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.738238+0000", 2024-09-10T19:03:27.862 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0019838059999999 2024-09-10T19:03:27.862 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.862 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.862 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:03:27.862 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.738357+0000", 2024-09-10T19:03:27.862 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000118725 2024-09-10T19:03:27.862 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.862 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.862 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,5", 2024-09-10T19:03:27.862 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.738494+0000", 2024-09-10T19:03:27.863 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000136754 2024-09-10T19:03:27.863 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.863 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.863 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:03:27.863 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.746420+0000", 2024-09-10T19:03:27.863 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0079265729999999993 2024-09-10T19:03:27.863 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.863 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.863 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.863 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.769339+0000", 2024-09-10T19:03:27.863 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.022918343000000001 2024-09-10T19:03:27.863 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.864 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.864 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.864 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.769743+0000", 2024-09-10T19:03:27.864 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00040463100000000002 2024-09-10T19:03:27.864 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.864 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.864 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:03:27.864 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.769829+0000", 2024-09-10T19:03:27.864 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.5123999999999993e-05 2024-09-10T19:03:27.864 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.864 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.864 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:03:27.865 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.769869+0000", 2024-09-10T19:03:27.865 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.0679000000000001e-05 2024-09-10T19:03:27.865 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.865 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:03:27.865 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.865 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.865 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.865 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15354.0:185496 9.9 9:9a2f2680:::benchmark_data_smithi018_133594_object48744:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e291)", 2024-09-10T19:03:27.865 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:58:14.740696+0000", 2024-09-10T19:03:27.865 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 313.08330077199997, 2024-09-10T19:03:27.865 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0295903689999999, 2024-09-10T19:03:27.865 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:03:27.865 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:03:27.865 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:03:27.866 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15354", 2024-09-10T19:03:27.866 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:03:27.866 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 185496 2024-09-10T19:03:27.866 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.866 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:03:27.866 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.866 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:03:27.866 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.740696+0000", 2024-09-10T19:03:27.866 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.866 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.866 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.867 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:03:27.867 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.740696+0000", 2024-09-10T19:03:27.867 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.867 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.867 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.867 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:03:27.867 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.740694+0000", 2024-09-10T19:03:27.867 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T19:03:27.867 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.867 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.867 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:03:27.867 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.740700+0000", 2024-09-10T19:03:27.867 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.4790000000000001e-06 2024-09-10T19:03:27.868 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.868 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.868 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:03:27.868 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.740715+0000", 2024-09-10T19:03:27.868 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.4791e-05 2024-09-10T19:03:27.868 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.868 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.868 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:03:27.868 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.740721+0000", 2024-09-10T19:03:27.868 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.7289999999999997e-06 2024-09-10T19:03:27.868 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.868 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.868 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.868 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.740792+0000", 2024-09-10T19:03:27.869 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.1577999999999998e-05 2024-09-10T19:03:27.869 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.869 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.869 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:03:27.869 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.740800+0000", 2024-09-10T19:03:27.869 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.4889999999999998e-06 2024-09-10T19:03:27.869 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.869 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.869 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.869 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.740403+0000", 2024-09-10T19:03:27.869 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.99960263199999999 2024-09-10T19:03:27.869 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.870 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.870 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:03:27.870 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.740541+0000", 2024-09-10T19:03:27.870 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00013869699999999999 2024-09-10T19:03:27.870 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.870 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.870 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,5", 2024-09-10T19:03:27.870 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.740671+0000", 2024-09-10T19:03:27.870 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00012951799999999999 2024-09-10T19:03:27.871 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.871 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.871 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:03:27.871 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.746428+0000", 2024-09-10T19:03:27.871 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0057567249999999999 2024-09-10T19:03:27.871 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.871 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.871 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.871 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.769697+0000", 2024-09-10T19:03:27.871 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.023269473999999998 2024-09-10T19:03:27.871 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.872 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.872 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.872 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.770209+0000", 2024-09-10T19:03:27.872 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00051148999999999997 2024-09-10T19:03:27.872 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.872 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.872 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:03:27.872 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.770252+0000", 2024-09-10T19:03:27.872 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.3004999999999999e-05 2024-09-10T19:03:27.872 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.872 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.872 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:03:27.872 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.770286+0000", 2024-09-10T19:03:27.873 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.4428999999999999e-05 2024-09-10T19:03:27.873 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.873 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:03:27.873 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.873 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.873 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.873 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15354.0:185512 9.9 9:93f0db15:::benchmark_data_smithi018_133594_object48760:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e291)", 2024-09-10T19:03:27.873 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:58:14.749826+0000", 2024-09-10T19:03:27.873 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 313.07417042499998, 2024-09-10T19:03:27.873 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.020604697, 2024-09-10T19:03:27.873 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:03:27.873 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:03:27.874 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:03:27.874 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15354", 2024-09-10T19:03:27.874 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:03:27.874 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 185512 2024-09-10T19:03:27.874 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.874 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:03:27.874 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.874 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:03:27.874 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.749826+0000", 2024-09-10T19:03:27.874 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.874 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.874 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.874 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:03:27.874 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.749826+0000", 2024-09-10T19:03:27.874 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.875 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.875 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.875 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:03:27.875 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.749825+0000", 2024-09-10T19:03:27.875 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T19:03:27.875 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.875 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.875 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:03:27.875 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.749830+0000", 2024-09-10T19:03:27.875 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.2050000000000001e-06 2024-09-10T19:03:27.875 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.875 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.875 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:03:27.875 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.749843+0000", 2024-09-10T19:03:27.875 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.3567000000000001e-05 2024-09-10T19:03:27.876 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.876 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.876 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:03:27.876 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.749850+0000", 2024-09-10T19:03:27.876 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.1539999999999999e-06 2024-09-10T19:03:27.876 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.876 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.876 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.876 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.750152+0000", 2024-09-10T19:03:27.876 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00030268299999999999 2024-09-10T19:03:27.876 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.876 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.877 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:03:27.877 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.750161+0000", 2024-09-10T19:03:27.877 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.1390000000000001e-06 2024-09-10T19:03:27.877 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.877 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.877 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.877 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.742288+0000", 2024-09-10T19:03:27.877 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.99212616899999995 2024-09-10T19:03:27.877 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.877 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.877 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:03:27.877 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.742469+0000", 2024-09-10T19:03:27.878 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000181609 2024-09-10T19:03:27.878 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.878 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.878 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,5", 2024-09-10T19:03:27.878 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.742614+0000", 2024-09-10T19:03:27.878 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000145024 2024-09-10T19:03:27.878 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.878 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.878 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:03:27.878 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.757457+0000", 2024-09-10T19:03:27.878 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.014842485000000001 2024-09-10T19:03:27.878 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.878 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.879 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.879 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.770119+0000", 2024-09-10T19:03:27.879 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.012662458 2024-09-10T19:03:27.879 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.879 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.879 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.879 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.770326+0000", 2024-09-10T19:03:27.879 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000206367 2024-09-10T19:03:27.879 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.879 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.879 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:03:27.879 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.770397+0000", 2024-09-10T19:03:27.879 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.1806999999999999e-05 2024-09-10T19:03:27.880 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.880 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.880 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:03:27.880 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.770431+0000", 2024-09-10T19:03:27.880 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.3271e-05 2024-09-10T19:03:27.880 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.880 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:03:27.880 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.880 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.880 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.880 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15354.0:185535 9.9 9:98cd59a5:::benchmark_data_smithi018_133594_object48783:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e291)", 2024-09-10T19:03:27.880 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:58:14.766373+0000", 2024-09-10T19:03:27.881 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 313.05762380599998, 2024-09-10T19:03:27.881 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0041714349999999, 2024-09-10T19:03:27.881 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:03:27.881 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:03:27.881 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:03:27.881 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15354", 2024-09-10T19:03:27.881 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:03:27.881 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 185535 2024-09-10T19:03:27.881 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.881 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:03:27.881 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.881 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:03:27.881 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.766373+0000", 2024-09-10T19:03:27.882 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.882 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.882 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.882 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:03:27.882 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.766373+0000", 2024-09-10T19:03:27.882 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.882 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.882 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.882 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:03:27.882 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.766371+0000", 2024-09-10T19:03:27.882 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T19:03:27.883 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.883 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.883 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:03:27.883 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.766378+0000", 2024-09-10T19:03:27.883 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.0639999999999996e-06 2024-09-10T19:03:27.883 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.883 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.883 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:03:27.883 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.766390+0000", 2024-09-10T19:03:27.883 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.2279999999999999e-05 2024-09-10T19:03:27.883 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.883 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.883 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:03:27.883 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.766397+0000", 2024-09-10T19:03:27.884 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.5019999999999998e-06 2024-09-10T19:03:27.884 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.884 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.884 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.884 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.766451+0000", 2024-09-10T19:03:27.884 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.4194000000000003e-05 2024-09-10T19:03:27.884 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.884 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.884 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:03:27.884 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.766460+0000", 2024-09-10T19:03:27.884 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.7110000000000001e-06 2024-09-10T19:03:27.884 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.885 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.885 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.885 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.744525+0000", 2024-09-10T19:03:27.885 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.97806574700000004 2024-09-10T19:03:27.885 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.885 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.885 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:03:27.885 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.744643+0000", 2024-09-10T19:03:27.885 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000117749 2024-09-10T19:03:27.885 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.885 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.885 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,5", 2024-09-10T19:03:27.885 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.744767+0000", 2024-09-10T19:03:27.886 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00012427900000000001 2024-09-10T19:03:27.886 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.886 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.886 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:03:27.886 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.759867+0000", 2024-09-10T19:03:27.886 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.015099571000000001 2024-09-10T19:03:27.886 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.886 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.886 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.886 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.770163+0000", 2024-09-10T19:03:27.886 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.010295512999999999 2024-09-10T19:03:27.886 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.886 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.887 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.887 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.770471+0000", 2024-09-10T19:03:27.887 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00030810000000000001 2024-09-10T19:03:27.887 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.887 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.887 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:03:27.888 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:03:27.888 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.770509+0000", 2024-09-10T19:03:27.888 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.8591999999999999e-05 2024-09-10T19:03:27.888 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.888 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.888 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:03:27.888 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.770544+0000", 2024-09-10T19:03:27.888 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.4705000000000001e-05 2024-09-10T19:03:27.888 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.888 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:03:27.888 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.889 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.889 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.889 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15354.0:185570 9.9 9:9377988d:::benchmark_data_smithi018_133594_object48818:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e291)", 2024-09-10T19:03:27.889 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:58:14.790105+0000", 2024-09-10T19:03:27.889 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 313.03389180800002, 2024-09-10T19:03:27.889 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.98104372799999995, 2024-09-10T19:03:27.889 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:03:27.889 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:03:27.889 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:03:27.889 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15354", 2024-09-10T19:03:27.889 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:03:27.889 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 185570 2024-09-10T19:03:27.889 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.889 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:03:27.890 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.890 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:03:27.890 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.790105+0000", 2024-09-10T19:03:27.890 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.890 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.890 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.890 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:03:27.890 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.790105+0000", 2024-09-10T19:03:27.890 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.890 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.890 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.890 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:03:27.890 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.790104+0000", 2024-09-10T19:03:27.890 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T19:03:27.890 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.891 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.891 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:03:27.891 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.790107+0000", 2024-09-10T19:03:27.891 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.4520000000000002e-06 2024-09-10T19:03:27.891 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.891 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.891 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:03:27.891 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.790115+0000", 2024-09-10T19:03:27.891 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.3880000000000002e-06 2024-09-10T19:03:27.891 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.891 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.891 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:03:27.892 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.790119+0000", 2024-09-10T19:03:27.892 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.5600000000000004e-06 2024-09-10T19:03:27.892 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.892 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.892 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.892 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.790164+0000", 2024-09-10T19:03:27.892 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.4443999999999997e-05 2024-09-10T19:03:27.892 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.893 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.893 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:03:27.893 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:14.790171+0000", 2024-09-10T19:03:27.893 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.8050000000000007e-06 2024-09-10T19:03:27.893 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.893 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.893 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.894 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.746482+0000", 2024-09-10T19:03:27.894 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.95631065599999998 2024-09-10T19:03:27.894 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.894 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.894 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:03:27.894 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.746588+0000", 2024-09-10T19:03:27.894 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000105682 2024-09-10T19:03:27.895 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.895 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.895 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,5", 2024-09-10T19:03:27.895 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.746892+0000", 2024-09-10T19:03:27.895 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00030408600000000002 2024-09-10T19:03:27.895 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.895 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.895 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:03:27.896 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.759873+0000", 2024-09-10T19:03:27.896 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.012981215000000001 2024-09-10T19:03:27.896 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.896 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.896 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.896 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.770579+0000", 2024-09-10T19:03:27.896 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.010705854000000001 2024-09-10T19:03:27.897 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.897 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.897 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.897 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.770674+0000", 2024-09-10T19:03:27.897 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.4878999999999997e-05 2024-09-10T19:03:27.897 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.897 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.897 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:03:27.898 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.770719+0000", 2024-09-10T19:03:27.898 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.5225999999999997e-05 2024-09-10T19:03:27.898 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.898 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.898 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:03:27.898 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:58:15.771148+0000", 2024-09-10T19:03:27.898 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00042917200000000002 2024-09-10T19:03:27.898 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.899 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:03:27.899 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.899 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.899 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.899 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:86386 10.5 10:a0c18088:::benchmark_data_smithi018_157943_object86385:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e337)", 2024-09-10T19:03:27.899 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:41.500350+0000", 2024-09-10T19:03:27.899 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 166.32364595499999, 2024-09-10T19:03:27.899 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.3425925100000002, 2024-09-10T19:03:27.900 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:03:27.900 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "reached pg", 2024-09-10T19:03:27.900 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:03:27.900 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:03:27.900 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:03:27.900 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 86386 2024-09-10T19:03:27.900 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.900 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:03:27.900 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.900 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:03:27.900 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.500350+0000", 2024-09-10T19:03:27.901 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.901 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.901 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.901 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:03:27.901 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.500350+0000", 2024-09-10T19:03:27.901 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.901 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.901 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.901 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:03:27.901 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.500350+0000", 2024-09-10T19:03:27.901 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T19:03:27.901 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.901 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.902 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:03:27.902 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.500375+0000", 2024-09-10T19:03:27.902 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.5171999999999999e-05 2024-09-10T19:03:27.902 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.902 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.902 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:03:27.902 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.500383+0000", 2024-09-10T19:03:27.902 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.5299999999999999e-06 2024-09-10T19:03:27.902 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.902 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.902 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:03:27.902 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.500387+0000", 2024-09-10T19:03:27.902 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.775e-06 2024-09-10T19:03:27.903 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.903 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.903 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.903 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.500593+0000", 2024-09-10T19:03:27.903 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00020583 2024-09-10T19:03:27.903 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.903 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.903 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:03:27.903 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.500709+0000", 2024-09-10T19:03:27.903 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000116257 2024-09-10T19:03:27.903 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.903 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.903 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,4", 2024-09-10T19:03:27.904 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.500998+0000", 2024-09-10T19:03:27.904 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00028884800000000002 2024-09-10T19:03:27.904 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.904 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.904 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.904 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.526528+0000", 2024-09-10T19:03:27.904 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.025529474 2024-09-10T19:03:27.904 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.904 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.904 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:03:27.904 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.526538+0000", 2024-09-10T19:03:27.904 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0084999999999999e-05 2024-09-10T19:03:27.904 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.904 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.905 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.905 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:42.508929+0000", 2024-09-10T19:03:27.905 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.98239073799999999 2024-09-10T19:03:27.905 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.905 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.905 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:03:27.905 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:42.508940+0000", 2024-09-10T19:03:27.905 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1806e-05 2024-09-10T19:03:27.905 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.905 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.905 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.905 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.842855+0000", 2024-09-10T19:03:27.905 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.3339148679999999 2024-09-10T19:03:27.906 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.906 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.906 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:03:27.906 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.842943+0000", 2024-09-10T19:03:27.906 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.7590000000000007e-05 2024-09-10T19:03:27.906 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.906 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:03:27.906 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.906 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.906 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.906 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.14415.0:2207 5.3 5:c609908c:::notify.5:head [watch reconnect cookie 94413763304848 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-10T19:03:27.906 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:41.506594+0000", 2024-09-10T19:03:27.906 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 166.31740189000001, 2024-09-10T19:03:27.907 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.022331375, 2024-09-10T19:03:27.907 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:03:27.907 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "started", 2024-09-10T19:03:27.907 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:03:27.907 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.14415", 2024-09-10T19:03:27.907 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/2965248470", 2024-09-10T19:03:27.907 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 2207 2024-09-10T19:03:27.907 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.907 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:03:27.907 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.907 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:03:27.907 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.506594+0000", 2024-09-10T19:03:27.907 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.907 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.908 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.908 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:03:27.908 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.506594+0000", 2024-09-10T19:03:27.908 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.908 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.908 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.908 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:03:27.908 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.506593+0000", 2024-09-10T19:03:27.908 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T19:03:27.908 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.908 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.908 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:03:27.908 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.506598+0000", 2024-09-10T19:03:27.908 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.797e-06 2024-09-10T19:03:27.909 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.909 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.909 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:03:27.909 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.506610+0000", 2024-09-10T19:03:27.909 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1895e-05 2024-09-10T19:03:27.909 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.909 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.909 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:03:27.909 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.506616+0000", 2024-09-10T19:03:27.910 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.7169999999999996e-06 2024-09-10T19:03:27.910 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.910 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.910 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.910 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.507492+0000", 2024-09-10T19:03:27.910 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00087686999999999997 2024-09-10T19:03:27.910 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.910 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.910 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:03:27.910 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.507500+0000", 2024-09-10T19:03:27.910 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.2599999999999999e-06 2024-09-10T19:03:27.910 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.910 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.910 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.911 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:42.520671+0000", 2024-09-10T19:03:27.911 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0131714679999999 2024-09-10T19:03:27.911 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.911 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.911 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for degraded object", 2024-09-10T19:03:27.911 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:42.520958+0000", 2024-09-10T19:03:27.911 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000287304 2024-09-10T19:03:27.911 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.911 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.911 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.911 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:42.527120+0000", 2024-09-10T19:03:27.911 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0061618580000000001 2024-09-10T19:03:27.911 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.912 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.912 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:03:27.912 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:42.527235+0000", 2024-09-10T19:03:27.912 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00011501399999999999 2024-09-10T19:03:27.912 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.912 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.912 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:03:27.912 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:42.528926+0000", 2024-09-10T19:03:27.912 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0016905030000000001 2024-09-10T19:03:27.912 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.912 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:03:27.912 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.912 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.912 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.913 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:86392 10.5 10:a9f1d4cb:::benchmark_data_smithi018_157943_object86391:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e337)", 2024-09-10T19:03:27.913 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:41.519716+0000", 2024-09-10T19:03:27.913 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 166.304280216, 2024-09-10T19:03:27.913 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.3352203760000001, 2024-09-10T19:03:27.913 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:03:27.913 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:03:27.913 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:03:27.913 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:03:27.913 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:03:27.913 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 86392 2024-09-10T19:03:27.913 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.913 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:03:27.914 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.914 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:03:27.914 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.519716+0000", 2024-09-10T19:03:27.914 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.914 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.914 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.914 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:03:27.914 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.519716+0000", 2024-09-10T19:03:27.914 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.914 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.914 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.914 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:03:27.914 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.519715+0000", 2024-09-10T19:03:27.914 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T19:03:27.915 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.915 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.915 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:03:27.915 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.519743+0000", 2024-09-10T19:03:27.915 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.8022e-05 2024-09-10T19:03:27.915 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.915 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.915 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:03:27.915 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.519794+0000", 2024-09-10T19:03:27.915 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.1428000000000003e-05 2024-09-10T19:03:27.915 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.915 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.915 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:03:27.915 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.519803+0000", 2024-09-10T19:03:27.916 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.7649999999999999e-06 2024-09-10T19:03:27.916 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.916 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.916 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.916 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.527200+0000", 2024-09-10T19:03:27.916 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00739699 2024-09-10T19:03:27.916 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.916 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.916 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:03:27.916 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.527208+0000", 2024-09-10T19:03:27.916 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.5390000000000002e-06 2024-09-10T19:03:27.916 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.916 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.916 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.916 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:42.508964+0000", 2024-09-10T19:03:27.917 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.98175653299999999 2024-09-10T19:03:27.917 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.917 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.917 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:03:27.917 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:42.508984+0000", 2024-09-10T19:03:27.917 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.9885999999999999e-05 2024-09-10T19:03:27.917 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.917 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.917 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.917 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.842983+0000", 2024-09-10T19:03:27.917 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.3339984069999999 2024-09-10T19:03:27.917 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.917 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.917 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:03:27.918 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.843100+0000", 2024-09-10T19:03:27.918 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00011788799999999999 2024-09-10T19:03:27.918 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.918 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.918 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,4,5", 2024-09-10T19:03:27.918 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.843301+0000", 2024-09-10T19:03:27.918 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00020033099999999999 2024-09-10T19:03:27.918 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.918 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.918 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:03:27.918 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.854414+0000", 2024-09-10T19:03:27.918 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.01111356 2024-09-10T19:03:27.919 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.919 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.919 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.919 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.854663+0000", 2024-09-10T19:03:27.919 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000248361 2024-09-10T19:03:27.919 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.919 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.919 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.919 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.854702+0000", 2024-09-10T19:03:27.919 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.9431999999999998e-05 2024-09-10T19:03:27.919 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.919 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.919 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.919 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.854775+0000", 2024-09-10T19:03:27.919 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.2988e-05 2024-09-10T19:03:27.920 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:27.921 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:27.921 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:27.922 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.922 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:03:27.922 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.854831+0000", 2024-09-10T19:03:27.922 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.5481e-05 2024-09-10T19:03:27.922 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.922 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.922 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:03:27.922 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.854937+0000", 2024-09-10T19:03:27.922 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000105876 2024-09-10T19:03:27.923 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.923 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:03:27.923 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.923 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.923 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.923 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:86402 10.5 10:af526166:::benchmark_data_smithi018_157943_object86401:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-10T19:03:27.923 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:41.534923+0000", 2024-09-10T19:03:27.923 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 166.28907354099999, 2024-09-10T19:03:27.923 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.320317556, 2024-09-10T19:03:27.923 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:03:27.923 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:03:27.923 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:03:27.923 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:03:27.924 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:03:27.924 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 86402 2024-09-10T19:03:27.924 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.924 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:03:27.924 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.924 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:03:27.924 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.534923+0000", 2024-09-10T19:03:27.924 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.924 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.924 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.924 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:03:27.924 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.534923+0000", 2024-09-10T19:03:27.925 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.925 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.925 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.925 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:03:27.925 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.534922+0000", 2024-09-10T19:03:27.925 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T19:03:27.925 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.925 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.925 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:03:27.925 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.534943+0000", 2024-09-10T19:03:27.925 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.1403000000000002e-05 2024-09-10T19:03:27.925 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.925 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.926 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:03:27.926 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.534955+0000", 2024-09-10T19:03:27.926 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1677e-05 2024-09-10T19:03:27.926 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.926 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.926 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:03:27.926 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.534960+0000", 2024-09-10T19:03:27.926 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.0370000000000002e-06 2024-09-10T19:03:27.926 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.926 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.926 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.926 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.535027+0000", 2024-09-10T19:03:27.926 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.6999e-05 2024-09-10T19:03:27.927 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.927 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.927 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:03:27.927 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.535036+0000", 2024-09-10T19:03:27.927 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.6149999999999997e-06 2024-09-10T19:03:27.927 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.927 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.927 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.927 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:42.509001+0000", 2024-09-10T19:03:27.927 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.97396503800000001 2024-09-10T19:03:27.927 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.927 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.927 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:03:27.927 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:42.509006+0000", 2024-09-10T19:03:27.927 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.1359999999999996e-06 2024-09-10T19:03:27.928 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.928 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.928 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.928 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.845128+0000", 2024-09-10T19:03:27.928 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.3361220789999999 2024-09-10T19:03:27.928 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.928 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.928 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:03:27.928 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.845262+0000", 2024-09-10T19:03:27.928 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000133585 2024-09-10T19:03:27.928 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.928 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.928 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,4,5", 2024-09-10T19:03:27.929 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.845442+0000", 2024-09-10T19:03:27.929 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00018037399999999999 2024-09-10T19:03:27.929 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.929 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.929 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:03:27.929 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.854421+0000", 2024-09-10T19:03:27.929 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0089787589999999994 2024-09-10T19:03:27.929 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.929 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.929 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.929 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.854738+0000", 2024-09-10T19:03:27.929 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000317134 2024-09-10T19:03:27.929 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.930 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.930 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.930 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.855015+0000", 2024-09-10T19:03:27.930 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00027738699999999998 2024-09-10T19:03:27.930 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.930 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.930 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.930 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.855079+0000", 2024-09-10T19:03:27.930 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.3745999999999996e-05 2024-09-10T19:03:27.930 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.930 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.930 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:03:27.930 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.855132+0000", 2024-09-10T19:03:27.930 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.3000000000000001e-05 2024-09-10T19:03:27.931 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.931 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.931 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:03:27.931 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.855240+0000", 2024-09-10T19:03:27.931 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00010843 2024-09-10T19:03:27.931 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.931 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:03:27.931 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.931 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.931 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.931 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:86414 10.5 10:a4781c1e:::benchmark_data_smithi018_157943_object86413:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-10T19:03:27.931 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:41.542162+0000", 2024-09-10T19:03:27.931 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 166.281834065, 2024-09-10T19:03:27.931 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.3132727270000002, 2024-09-10T19:03:27.932 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:03:27.932 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:03:27.932 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:03:27.932 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:03:27.932 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:03:27.932 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 86414 2024-09-10T19:03:27.932 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.932 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:03:27.932 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.932 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:03:27.932 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.542162+0000", 2024-09-10T19:03:27.933 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.933 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.933 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.933 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:03:27.933 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.542162+0000", 2024-09-10T19:03:27.933 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.933 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.933 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.933 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:03:27.933 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.542161+0000", 2024-09-10T19:03:27.934 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T19:03:27.934 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.934 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.934 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:03:27.934 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.542185+0000", 2024-09-10T19:03:27.934 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.4426e-05 2024-09-10T19:03:27.934 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.934 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.934 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:03:27.934 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.542197+0000", 2024-09-10T19:03:27.934 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1887e-05 2024-09-10T19:03:27.934 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.934 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.935 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:03:27.935 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.542202+0000", 2024-09-10T19:03:27.935 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.8509999999999998e-06 2024-09-10T19:03:27.935 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.935 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.935 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.935 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.542270+0000", 2024-09-10T19:03:27.935 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.7508000000000005e-05 2024-09-10T19:03:27.935 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.935 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.935 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:03:27.935 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.542278+0000", 2024-09-10T19:03:27.935 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.9209999999999998e-06 2024-09-10T19:03:27.936 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.936 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.936 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.936 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:42.509020+0000", 2024-09-10T19:03:27.936 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.96674117900000001 2024-09-10T19:03:27.936 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.936 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.936 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:03:27.936 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:42.509025+0000", 2024-09-10T19:03:27.936 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.8940000000000001e-06 2024-09-10T19:03:27.936 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.936 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.937 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.937 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.847305+0000", 2024-09-10T19:03:27.937 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.3382804909999999 2024-09-10T19:03:27.937 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.937 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.937 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:03:27.937 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.847413+0000", 2024-09-10T19:03:27.937 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000108015 2024-09-10T19:03:27.937 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.937 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.937 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,4,5", 2024-09-10T19:03:27.938 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.847573+0000", 2024-09-10T19:03:27.938 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000159918 2024-09-10T19:03:27.938 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.938 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.938 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:03:27.938 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.854427+0000", 2024-09-10T19:03:27.938 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0068536789999999997 2024-09-10T19:03:27.938 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.938 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.938 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.938 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.854983+0000", 2024-09-10T19:03:27.938 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00055630600000000001 2024-09-10T19:03:27.938 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.939 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.939 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.939 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.855049+0000", 2024-09-10T19:03:27.939 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.6000999999999994e-05 2024-09-10T19:03:27.939 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.939 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.939 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.939 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.855277+0000", 2024-09-10T19:03:27.939 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00022805699999999999 2024-09-10T19:03:27.939 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.939 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.939 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:03:27.940 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.855333+0000", 2024-09-10T19:03:27.940 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.5859000000000002e-05 2024-09-10T19:03:27.940 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.940 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.940 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:03:27.940 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.855435+0000", 2024-09-10T19:03:27.940 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000102157 2024-09-10T19:03:27.940 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.940 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:03:27.940 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.940 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.940 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.941 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:86421 10.5 10:a95eade5:::benchmark_data_smithi018_157943_object86420:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-10T19:03:27.941 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:41.547867+0000", 2024-09-10T19:03:27.941 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 166.276129395, 2024-09-10T19:03:27.941 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.3106902900000001, 2024-09-10T19:03:27.941 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:03:27.941 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:03:27.941 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:03:27.941 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:03:27.941 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:03:27.941 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 86421 2024-09-10T19:03:27.941 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.941 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:03:27.942 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.942 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:03:27.942 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.547867+0000", 2024-09-10T19:03:27.942 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.942 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.942 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.942 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:03:27.942 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.547867+0000", 2024-09-10T19:03:27.942 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.942 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.942 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.943 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:03:27.943 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.547865+0000", 2024-09-10T19:03:27.943 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999981 2024-09-10T19:03:27.943 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.943 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.943 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:03:27.943 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.547892+0000", 2024-09-10T19:03:27.943 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.6543e-05 2024-09-10T19:03:27.943 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.943 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.943 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:03:27.943 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.547908+0000", 2024-09-10T19:03:27.944 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.6166000000000001e-05 2024-09-10T19:03:27.944 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.944 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.944 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:03:27.944 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.547914+0000", 2024-09-10T19:03:27.944 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.0990000000000004e-06 2024-09-10T19:03:27.944 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.944 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.944 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.944 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.548082+0000", 2024-09-10T19:03:27.945 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00016819 2024-09-10T19:03:27.945 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.945 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.945 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:03:27.945 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.548093+0000", 2024-09-10T19:03:27.945 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1028e-05 2024-09-10T19:03:27.945 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.945 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.945 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.945 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:42.509038+0000", 2024-09-10T19:03:27.945 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.96094507500000004 2024-09-10T19:03:27.945 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.946 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.946 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:03:27.946 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:42.509043+0000", 2024-09-10T19:03:27.946 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.8590000000000004e-06 2024-09-10T19:03:27.946 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.946 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.946 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.946 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.849557+0000", 2024-09-10T19:03:27.946 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.3405136440000001 2024-09-10T19:03:27.946 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.946 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.946 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:03:27.946 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.849667+0000", 2024-09-10T19:03:27.946 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000110522 2024-09-10T19:03:27.946 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.947 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.947 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,4,5", 2024-09-10T19:03:27.947 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.849826+0000", 2024-09-10T19:03:27.947 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00015840299999999999 2024-09-10T19:03:27.947 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.947 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.947 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:03:27.947 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.854432+0000", 2024-09-10T19:03:27.947 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0046066450000000004 2024-09-10T19:03:27.947 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.947 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.947 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.948 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.855469+0000", 2024-09-10T19:03:27.948 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00103634 2024-09-10T19:03:27.948 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.948 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.948 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.948 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.855496+0000", 2024-09-10T19:03:27.948 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.72e-05 2024-09-10T19:03:27.948 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.948 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.948 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.948 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.858385+0000", 2024-09-10T19:03:27.949 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0028891730000000001 2024-09-10T19:03:27.949 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.949 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.949 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:03:27.949 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.858438+0000", 2024-09-10T19:03:27.949 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.3075000000000003e-05 2024-09-10T19:03:27.949 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.949 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.949 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:03:27.949 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.858557+0000", 2024-09-10T19:03:27.949 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000119182 2024-09-10T19:03:27.949 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.950 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:03:27.950 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.950 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.950 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.950 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:86428 10.5 10:a189f9a6:::benchmark_data_smithi018_157943_object86427:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-10T19:03:27.950 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:41.552063+0000", 2024-09-10T19:03:27.950 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 166.27193384, 2024-09-10T19:03:27.950 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.306941267, 2024-09-10T19:03:27.950 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:03:27.950 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:03:27.950 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:03:27.950 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:03:27.950 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:03:27.950 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 86428 2024-09-10T19:03:27.950 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.951 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:03:27.951 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.951 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:03:27.951 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.552063+0000", 2024-09-10T19:03:27.951 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.951 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.951 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.951 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:03:27.951 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.552063+0000", 2024-09-10T19:03:27.951 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.951 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.951 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.951 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:03:27.951 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.552061+0000", 2024-09-10T19:03:27.952 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999981 2024-09-10T19:03:27.952 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.952 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.952 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:03:27.952 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.552087+0000", 2024-09-10T19:03:27.952 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.6407999999999999e-05 2024-09-10T19:03:27.952 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.952 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.952 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:03:27.952 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.552105+0000", 2024-09-10T19:03:27.952 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.7873e-05 2024-09-10T19:03:27.952 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.952 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.953 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:03:27.953 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.552111+0000", 2024-09-10T19:03:27.953 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.6189999999999996e-06 2024-09-10T19:03:27.953 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.953 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.953 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.953 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.552170+0000", 2024-09-10T19:03:27.953 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.8625999999999997e-05 2024-09-10T19:03:27.953 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.953 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.953 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:03:27.953 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:41.552179+0000", 2024-09-10T19:03:27.953 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.3530000000000001e-06 2024-09-10T19:03:27.953 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.954 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.954 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.954 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:42.509056+0000", 2024-09-10T19:03:27.954 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.95687695500000003 2024-09-10T19:03:27.954 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.954 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.954 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:03:27.954 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:42.509061+0000", 2024-09-10T19:03:27.954 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.7550000000000002e-06 2024-09-10T19:03:27.954 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.954 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.954 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.954 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.851773+0000", 2024-09-10T19:03:27.955 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.342712106 2024-09-10T19:03:27.955 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.955 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.955 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:03:27.955 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.851898+0000", 2024-09-10T19:03:27.955 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000124964 2024-09-10T19:03:27.955 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.955 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.955 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,4,5", 2024-09-10T19:03:27.955 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.852061+0000", 2024-09-10T19:03:27.955 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00016302499999999999 2024-09-10T19:03:27.955 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.955 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.955 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.955 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.855523+0000", 2024-09-10T19:03:27.956 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0034616149999999999 2024-09-10T19:03:27.956 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.956 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.956 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.956 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.855818+0000", 2024-09-10T19:03:27.956 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00029469400000000002 2024-09-10T19:03:27.956 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.956 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.956 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.956 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.858604+0000", 2024-09-10T19:03:27.956 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0027862590000000001 2024-09-10T19:03:27.956 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.956 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.956 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:03:27.956 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.858781+0000", 2024-09-10T19:03:27.957 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00017759699999999999 2024-09-10T19:03:27.957 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.957 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.957 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:03:27.957 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.858827+0000", 2024-09-10T19:03:27.957 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.5909999999999999e-05 2024-09-10T19:03:27.957 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.957 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.957 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:03:27.957 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:43.859004+0000", 2024-09-10T19:03:27.957 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000176446 2024-09-10T19:03:27.957 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.957 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:03:27.958 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.958 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.958 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.958 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:275417 10.c 10:3e4f8079:::benchmark_data_smithi018_157943_object136759:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e372)", 2024-09-10T19:03:27.958 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:02:48.583605+0000", 2024-09-10T19:03:27.958 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 39.240390902999998, 2024-09-10T19:03:27.958 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0102979139999999, 2024-09-10T19:03:27.958 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:03:27.958 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:03:27.958 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:03:27.958 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:03:27.958 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:03:27.958 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 275417 2024-09-10T19:03:27.958 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.958 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:03:27.959 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.959 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:03:27.959 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:48.583605+0000", 2024-09-10T19:03:27.959 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.959 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.959 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.959 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:03:27.959 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:48.583605+0000", 2024-09-10T19:03:27.959 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.959 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.959 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.959 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:03:27.960 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:48.583603+0000", 2024-09-10T19:03:27.960 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999981 2024-09-10T19:03:27.960 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.960 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.960 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:03:27.960 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:48.583612+0000", 2024-09-10T19:03:27.960 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.6789999999999994e-06 2024-09-10T19:03:27.960 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.960 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.960 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:03:27.960 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:48.583629+0000", 2024-09-10T19:03:27.961 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.7166000000000001e-05 2024-09-10T19:03:27.961 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.961 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.961 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:03:27.961 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:48.583636+0000", 2024-09-10T19:03:27.961 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.4790000000000001e-06 2024-09-10T19:03:27.961 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.961 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.961 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.961 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:48.583703+0000", 2024-09-10T19:03:27.961 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.7170999999999998e-05 2024-09-10T19:03:27.961 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.962 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.962 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:03:27.962 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:48.583715+0000", 2024-09-10T19:03:27.962 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1739999999999999e-05 2024-09-10T19:03:27.962 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.962 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.962 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.962 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:49.582749+0000", 2024-09-10T19:03:27.962 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.99903390700000005 2024-09-10T19:03:27.962 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.962 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.962 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:03:27.963 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:49.582857+0000", 2024-09-10T19:03:27.963 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00010841900000000001 2024-09-10T19:03:27.963 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.963 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.963 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,3", 2024-09-10T19:03:27.963 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:49.582993+0000", 2024-09-10T19:03:27.963 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000136298 2024-09-10T19:03:27.963 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.963 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.963 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:03:27.963 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:49.589142+0000", 2024-09-10T19:03:27.963 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0061485649999999999 2024-09-10T19:03:27.963 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.964 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.964 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.964 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:49.593704+0000", 2024-09-10T19:03:27.964 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0045623599999999997 2024-09-10T19:03:27.964 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.964 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.964 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.964 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:49.593790+0000", 2024-09-10T19:03:27.964 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.5618000000000006e-05 2024-09-10T19:03:27.964 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.964 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.964 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:03:27.965 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:49.593865+0000", 2024-09-10T19:03:27.965 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.5082999999999999e-05 2024-09-10T19:03:27.965 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.965 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.965 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:03:27.965 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:49.593903+0000", 2024-09-10T19:03:27.965 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.8427999999999999e-05 2024-09-10T19:03:27.965 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.965 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:03:27.965 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.965 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.965 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.965 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:275440 10.c 10:351d1c0d:::benchmark_data_smithi018_157943_object136782:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e372)", 2024-09-10T19:03:27.966 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:02:48.596517+0000", 2024-09-10T19:03:27.966 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 39.227479215999999, 2024-09-10T19:03:27.966 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.99825936999999998, 2024-09-10T19:03:27.966 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:03:27.966 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:03:27.966 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:03:27.966 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:03:27.966 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:03:27.966 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 275440 2024-09-10T19:03:27.966 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.966 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:03:27.966 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.966 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:03:27.967 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:48.596517+0000", 2024-09-10T19:03:27.967 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.967 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.967 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.967 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:03:27.967 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:48.596517+0000", 2024-09-10T19:03:27.967 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.967 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.967 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.967 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:03:27.967 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:48.596516+0000", 2024-09-10T19:03:27.967 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T19:03:27.967 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.968 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.968 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:03:27.968 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:48.596522+0000", 2024-09-10T19:03:27.968 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.0900000000000001e-06 2024-09-10T19:03:27.968 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.968 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.968 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:03:27.968 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:48.596535+0000", 2024-09-10T19:03:27.968 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.2670999999999999e-05 2024-09-10T19:03:27.968 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.968 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.968 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:03:27.968 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:48.596541+0000", 2024-09-10T19:03:27.968 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.7880000000000003e-06 2024-09-10T19:03:27.969 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.969 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.969 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.969 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:48.596602+0000", 2024-09-10T19:03:27.969 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.1489999999999996e-05 2024-09-10T19:03:27.969 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.969 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.969 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:03:27.969 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:48.596611+0000", 2024-09-10T19:03:27.969 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.8310000000000007e-06 2024-09-10T19:03:27.969 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.969 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.969 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.970 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:49.584951+0000", 2024-09-10T19:03:27.970 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.98834033499999996 2024-09-10T19:03:27.970 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.970 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.970 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:03:27.970 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:49.585039+0000", 2024-09-10T19:03:27.970 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.8041999999999995e-05 2024-09-10T19:03:27.970 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.970 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.970 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,3", 2024-09-10T19:03:27.970 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:49.585106+0000", 2024-09-10T19:03:27.971 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.6828000000000005e-05 2024-09-10T19:03:27.971 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.971 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.971 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:03:27.971 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:49.591719+0000", 2024-09-10T19:03:27.971 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0066126600000000002 2024-09-10T19:03:27.971 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.971 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.971 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.971 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:49.593962+0000", 2024-09-10T19:03:27.971 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0022430480000000001 2024-09-10T19:03:27.971 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.972 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.972 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.972 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:49.594682+0000", 2024-09-10T19:03:27.972 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00071998099999999996 2024-09-10T19:03:27.972 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.972 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.972 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:03:27.972 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:49.594740+0000", 2024-09-10T19:03:27.972 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.8139e-05 2024-09-10T19:03:27.972 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.972 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.972 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:03:27.972 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:49.594776+0000", 2024-09-10T19:03:27.973 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.6523999999999998e-05 2024-09-10T19:03:27.973 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.973 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:03:27.973 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.973 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.973 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.973 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:275450 10.c 10:37d02928:::benchmark_data_smithi018_157943_object136792:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e372)", 2024-09-10T19:03:27.973 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:02:48.600537+0000", 2024-09-10T19:03:27.973 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 39.223459421000001, 2024-09-10T19:03:27.973 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.99434900199999998, 2024-09-10T19:03:27.973 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:03:27.973 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:03:27.974 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:03:27.974 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:03:27.974 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:03:27.974 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 275450 2024-09-10T19:03:27.974 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.974 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:03:27.974 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.974 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:03:27.974 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:48.600537+0000", 2024-09-10T19:03:27.974 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.974 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.974 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.974 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:03:27.975 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:48.600537+0000", 2024-09-10T19:03:27.975 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.975 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.975 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.975 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:03:27.975 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:48.600536+0000", 2024-09-10T19:03:27.975 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T19:03:27.975 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.975 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.975 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:03:27.975 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:48.600541+0000", 2024-09-10T19:03:27.975 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.1760000000000001e-06 2024-09-10T19:03:27.975 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.976 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.976 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:03:27.976 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:48.600552+0000", 2024-09-10T19:03:27.976 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.133e-05 2024-09-10T19:03:27.976 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.976 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.976 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:03:27.976 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:48.600558+0000", 2024-09-10T19:03:27.976 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.5350000000000004e-06 2024-09-10T19:03:27.976 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.976 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.976 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.976 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:48.600613+0000", 2024-09-10T19:03:27.977 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.5000999999999997e-05 2024-09-10T19:03:27.977 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.977 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.977 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:03:27.977 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:48.600624+0000", 2024-09-10T19:03:27.977 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0956e-05 2024-09-10T19:03:27.977 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.977 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.977 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.977 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:49.586289+0000", 2024-09-10T19:03:27.977 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.985664706 2024-09-10T19:03:27.977 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.978 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.978 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:03:27.978 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:49.586393+0000", 2024-09-10T19:03:27.978 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000104547 2024-09-10T19:03:27.978 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.978 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.978 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,3", 2024-09-10T19:03:27.978 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:49.586459+0000", 2024-09-10T19:03:27.978 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.6307000000000001e-05 2024-09-10T19:03:27.978 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.978 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.978 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:03:27.978 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:49.591722+0000", 2024-09-10T19:03:27.979 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0052625359999999999 2024-09-10T19:03:27.979 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.979 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.979 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.979 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:49.594610+0000", 2024-09-10T19:03:27.979 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.002888421 2024-09-10T19:03:27.979 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.979 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.979 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.979 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:49.594816+0000", 2024-09-10T19:03:27.979 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00020513199999999999 2024-09-10T19:03:27.979 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.979 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.980 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:03:27.980 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:49.594852+0000", 2024-09-10T19:03:27.980 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.6786999999999999e-05 2024-09-10T19:03:27.980 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.980 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.980 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:03:27.980 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:49.594886+0000", 2024-09-10T19:03:27.980 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.3559999999999997e-05 2024-09-10T19:03:27.980 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.980 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:03:27.980 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.980 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.980 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.981 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:275460 10.c 10:3b859502:::benchmark_data_smithi018_157943_object136802:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e372)", 2024-09-10T19:03:27.981 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:02:48.605270+0000", 2024-09-10T19:03:27.981 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 39.218726687999997, 2024-09-10T19:03:27.981 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.98978724200000001, 2024-09-10T19:03:27.981 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:03:27.981 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:03:27.981 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:03:27.981 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:03:27.981 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:03:27.981 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 275460 2024-09-10T19:03:27.981 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.981 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:03:27.981 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.982 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:03:27.982 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:48.605270+0000", 2024-09-10T19:03:27.982 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.982 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.982 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.982 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:03:27.982 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:48.605270+0000", 2024-09-10T19:03:27.982 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:27.982 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.982 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.982 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:03:27.982 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:48.605269+0000", 2024-09-10T19:03:27.983 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T19:03:27.983 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.983 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.983 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:03:27.983 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:48.605274+0000", 2024-09-10T19:03:27.983 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.9779999999999996e-06 2024-09-10T19:03:27.983 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.983 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.983 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:03:27.983 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:48.605289+0000", 2024-09-10T19:03:27.983 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.4844999999999999e-05 2024-09-10T19:03:27.983 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.983 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.983 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:03:27.984 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:48.605294+0000", 2024-09-10T19:03:27.984 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.8510000000000004e-06 2024-09-10T19:03:27.984 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.984 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.984 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.984 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:48.605348+0000", 2024-09-10T19:03:27.984 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.3287e-05 2024-09-10T19:03:27.984 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.984 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.984 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:03:27.984 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:48.605357+0000", 2024-09-10T19:03:27.984 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.7409999999999998e-06 2024-09-10T19:03:27.984 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.985 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.985 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:03:27.985 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:49.587772+0000", 2024-09-10T19:03:27.985 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.98241532899999995 2024-09-10T19:03:27.985 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.985 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.985 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:03:27.985 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:49.587885+0000", 2024-09-10T19:03:27.985 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000113572 2024-09-10T19:03:27.985 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.985 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.985 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 1,3", 2024-09-10T19:03:27.985 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:49.588012+0000", 2024-09-10T19:03:27.986 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00012656000000000001 2024-09-10T19:03:27.986 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.986 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.986 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:03:27.986 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:49.593383+0000", 2024-09-10T19:03:27.986 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0053706819999999999 2024-09-10T19:03:27.986 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.986 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.986 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.986 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:49.594653+0000", 2024-09-10T19:03:27.986 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00127013 2024-09-10T19:03:27.986 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.986 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.986 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:03:27.987 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:49.594968+0000", 2024-09-10T19:03:27.987 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00031545500000000002 2024-09-10T19:03:27.987 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.987 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.987 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:03:27.987 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:49.595029+0000", 2024-09-10T19:03:27.987 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.0402999999999997e-05 2024-09-10T19:03:27.987 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:27.987 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:27.987 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:03:27.987 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:02:49.595057+0000", 2024-09-10T19:03:27.987 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.8262000000000001e-05 2024-09-10T19:03:27.987 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.987 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:03:27.988 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.988 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:27.988 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:03:27.988 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T19:03:27.988 DEBUG:teuthology.orchestra.run.smithi018:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_4 bench 90 write' 2024-09-10T19:03:27.989 INFO:tasks.thrashosds.thrasher:fixing pg num pool .mgr 2024-09-10T19:03:27.989 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T19:03:27.991 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-10T19:03:27.991 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-10T19:03:28.022 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:28.054 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:28.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:28.055 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:28.065 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:hints = 1 2024-09-10T19:03:28.065 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-10T19:03:28.065 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Object prefix: benchmark_data_smithi018_181485 2024-09-10T19:03:28.066 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T19:03:28.066 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 0 0 0 0 0 0 - 0 2024-09-10T19:03:28.155 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:28.196 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:28.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:28.198 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:28.299 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:28.350 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:28.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:28.352 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:28.453 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:28.486 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:28.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:28.487 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:28.588 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:28.619 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:28.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:28.620 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:28.720 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:28.764 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:28.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:28.765 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:28.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:28 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:28.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:28 smithi103 ceph-mon[33163]: pgmap v843: 168 pgs: 168 active+clean; 600 KiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:28.866 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:28.904 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:28.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:28.905 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:28.936 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:28 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:28.936 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:28 smithi018 ceph-mon[40196]: pgmap v843: 168 pgs: 168 active+clean; 600 KiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:28.936 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:28 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:28.936 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:28 smithi018 ceph-mon[29336]: pgmap v843: 168 pgs: 168 active+clean; 600 KiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:29.006 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:29.045 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:29.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:29.047 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:29.065 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 1 16 3550 3534 220.863 220.875 0.00357023 0.00450936 2024-09-10T19:03:29.147 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:29.178 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:29.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:29.179 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:29.280 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:29.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:29.320 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:29.320 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:29.421 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:29.439 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-10T19:03:29.464 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:29.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:29.465 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:29.566 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:29.597 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:29.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:29.598 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:29.628 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T19:03:29.699 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:29.726 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:29.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:29.728 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:29.829 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:29.872 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:29.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:29.873 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:29.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:29 smithi018 ceph-mon[29336]: 2.5 scrub starts 2024-09-10T19:03:29.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:29 smithi018 ceph-mon[29336]: 2.5 scrub ok 2024-09-10T19:03:29.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:29 smithi018 ceph-mon[29336]: pgmap v844: 168 pgs: 168 active+clean; 20 MiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 1.2 KiB/s rd, 2.4 MiB/s wr, 39 op/s 2024-09-10T19:03:29.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:29 smithi018 ceph-mon[40196]: 2.5 scrub starts 2024-09-10T19:03:29.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:29 smithi018 ceph-mon[40196]: 2.5 scrub ok 2024-09-10T19:03:29.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:29 smithi018 ceph-mon[40196]: pgmap v844: 168 pgs: 168 active+clean; 20 MiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 1.2 KiB/s rd, 2.4 MiB/s wr, 39 op/s 2024-09-10T19:03:29.974 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:29.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:30.000 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:30.000 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:30.066 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 2 16 6755 6739 210.578 200.312 0.00379924 0.00474002 2024-09-10T19:03:30.101 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:30.132 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:30.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:30.133 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:30.233 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:30.240 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:29 smithi103 ceph-mon[33163]: 2.5 scrub starts 2024-09-10T19:03:30.240 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:29 smithi103 ceph-mon[33163]: 2.5 scrub ok 2024-09-10T19:03:30.240 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:29 smithi103 ceph-mon[33163]: pgmap v844: 168 pgs: 168 active+clean; 20 MiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 1.2 KiB/s rd, 2.4 MiB/s wr, 39 op/s 2024-09-10T19:03:30.266 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:30.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:30.267 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:30.368 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:30.399 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:30.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:30.400 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:30.457 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:03:30.500 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:30.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:03:30 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:03:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:03:30.533 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:03:30.569 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:30.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:30.570 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:30.671 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:30.714 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:30.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:30.715 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:30.816 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:30.857 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:30.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:30.858 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:30.958 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:30.991 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:30.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:30.992 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:31.040 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:30 smithi018 ceph-mon[29336]: 4.1 scrub starts 2024-09-10T19:03:31.040 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:30 smithi018 ceph-mon[29336]: 4.1 scrub ok 2024-09-10T19:03:31.040 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:30 smithi018 ceph-mon[29336]: 5.3 scrub starts 2024-09-10T19:03:31.040 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:30 smithi018 ceph-mon[29336]: 5.3 scrub ok 2024-09-10T19:03:31.041 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:30 smithi018 ceph-mon[29336]: 2.1d scrub starts 2024-09-10T19:03:31.041 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:30 smithi018 ceph-mon[29336]: 2.1d scrub ok 2024-09-10T19:03:31.041 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:30 smithi018 ceph-mon[40196]: 4.1 scrub starts 2024-09-10T19:03:31.041 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:30 smithi018 ceph-mon[40196]: 4.1 scrub ok 2024-09-10T19:03:31.041 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:30 smithi018 ceph-mon[40196]: 5.3 scrub starts 2024-09-10T19:03:31.041 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:30 smithi018 ceph-mon[40196]: 5.3 scrub ok 2024-09-10T19:03:31.041 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:30 smithi018 ceph-mon[40196]: 2.1d scrub starts 2024-09-10T19:03:31.041 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:30 smithi018 ceph-mon[40196]: 2.1d scrub ok 2024-09-10T19:03:31.042 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:03:30 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:03:30] "GET /metrics HTTP/1.1" 200 209396 "" "Prometheus/2.43.0" 2024-09-10T19:03:31.066 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 3 16 9929 9913 206.477 198.375 0.010141 0.00483414 2024-09-10T19:03:31.093 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:31.127 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:31.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:31.128 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:31.229 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:31.270 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:31.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:31.272 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:31.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:30 smithi103 ceph-mon[33163]: 4.1 scrub starts 2024-09-10T19:03:31.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:30 smithi103 ceph-mon[33163]: 4.1 scrub ok 2024-09-10T19:03:31.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:30 smithi103 ceph-mon[33163]: 5.3 scrub starts 2024-09-10T19:03:31.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:30 smithi103 ceph-mon[33163]: 5.3 scrub ok 2024-09-10T19:03:31.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:30 smithi103 ceph-mon[33163]: 2.1d scrub starts 2024-09-10T19:03:31.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:30 smithi103 ceph-mon[33163]: 2.1d scrub ok 2024-09-10T19:03:31.373 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:31.401 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:31.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:31.402 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:31.503 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:31.543 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:31.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:31.544 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:31.645 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:31.677 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:31.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:31.678 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:31.696 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:03:31.778 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:31.826 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:31.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:31.827 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:31.928 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:31.961 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:31.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:31.962 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:32.063 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:32.066 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 4 16 12961 12945 202.229 189.5 0.0074945 0.00493689 2024-09-10T19:03:32.094 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:32.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:32.095 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:32.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:31 smithi018 ceph-mon[40196]: 1.4 scrub starts 2024-09-10T19:03:32.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:31 smithi018 ceph-mon[40196]: 1.4 scrub ok 2024-09-10T19:03:32.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:31 smithi018 ceph-mon[40196]: pgmap v845: 168 pgs: 168 active+clean; 252 MiB data, 6.1 GiB used, 530 GiB / 536 GiB avail; 1023 B/s rd, 25 MiB/s wr, 403 op/s 2024-09-10T19:03:32.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:31 smithi018 ceph-mon[29336]: 1.4 scrub starts 2024-09-10T19:03:32.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:31 smithi018 ceph-mon[29336]: 1.4 scrub ok 2024-09-10T19:03:32.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:31 smithi018 ceph-mon[29336]: pgmap v845: 168 pgs: 168 active+clean; 252 MiB data, 6.1 GiB used, 530 GiB / 536 GiB avail; 1023 B/s rd, 25 MiB/s wr, 403 op/s 2024-09-10T19:03:32.196 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:32.235 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:32.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:32.237 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:32.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:31 smithi103 ceph-mon[33163]: 1.4 scrub starts 2024-09-10T19:03:32.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:31 smithi103 ceph-mon[33163]: 1.4 scrub ok 2024-09-10T19:03:32.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:31 smithi103 ceph-mon[33163]: pgmap v845: 168 pgs: 168 active+clean; 252 MiB data, 6.1 GiB used, 530 GiB / 536 GiB avail; 1023 B/s rd, 25 MiB/s wr, 403 op/s 2024-09-10T19:03:32.338 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:32.377 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:32.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:32.378 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:32.479 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:32.510 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:32.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:32.511 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:32.612 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:32.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:32.654 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:32.654 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:32.755 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:32.794 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:32.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:32.795 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:32.896 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:32.935 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:32.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:32.937 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:33.037 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:33.066 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 5 16 15682 15666 195.787 170.062 0.00981031 0.00509801 2024-09-10T19:03:33.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:33.082 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:33.082 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:33.154 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is unset 2024-09-10T19:03:33.183 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:33.215 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:33.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:33.216 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:33.317 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:33.359 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:33.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:33.360 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:33.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:33 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1382017630' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:03:33.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:33 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:03:33.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:33 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1382017630' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:03:33.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:33 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:03:33.461 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:33.493 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:33.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:33.494 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:33.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:33 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1382017630' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:03:33.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:33 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:03:33.594 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:33.633 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:33.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:33.634 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:33.735 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:33.767 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:33.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:33.768 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:33.869 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:33.899 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:33.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:33.901 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:34.001 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:34.042 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:34.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:34.044 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:34.066 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 6 16 18092 18076 188.259 150.625 0.00506797 0.00530424 2024-09-10T19:03:34.144 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:34.190 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:34.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:34.192 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:34.293 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:34.321 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:34.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:34.322 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:34.423 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:34.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:34 smithi018 ceph-mon[29336]: 6.a scrub starts 2024-09-10T19:03:34.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:34 smithi018 ceph-mon[29336]: 6.a scrub ok 2024-09-10T19:03:34.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:34 smithi018 ceph-mon[29336]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T19:03:34.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:34 smithi018 ceph-mon[29336]: Cluster is now healthy 2024-09-10T19:03:34.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:34 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T19:03:34.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:34 smithi018 ceph-mon[29336]: osdmap e385: 8 total, 7 up, 6 in 2024-09-10T19:03:34.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:34 smithi018 ceph-mon[29336]: pgmap v847: 168 pgs: 168 active+clean; 611 MiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 1 KiB/s rd, 61 MiB/s wr, 978 op/s 2024-09-10T19:03:34.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:34 smithi018 ceph-mon[29336]: osdmap e386: 8 total, 7 up, 6 in 2024-09-10T19:03:34.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:34 smithi018 ceph-mon[29336]: 1.7 scrub starts 2024-09-10T19:03:34.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:34 smithi018 ceph-mon[29336]: 1.7 scrub ok 2024-09-10T19:03:34.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:34 smithi018 ceph-mon[40196]: 6.a scrub starts 2024-09-10T19:03:34.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:34 smithi018 ceph-mon[40196]: 6.a scrub ok 2024-09-10T19:03:34.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:34 smithi018 ceph-mon[40196]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T19:03:34.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:34 smithi018 ceph-mon[40196]: Cluster is now healthy 2024-09-10T19:03:34.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:34 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T19:03:34.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:34 smithi018 ceph-mon[40196]: osdmap e385: 8 total, 7 up, 6 in 2024-09-10T19:03:34.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:34 smithi018 ceph-mon[40196]: pgmap v847: 168 pgs: 168 active+clean; 611 MiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 1 KiB/s rd, 61 MiB/s wr, 978 op/s 2024-09-10T19:03:34.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:34 smithi018 ceph-mon[40196]: osdmap e386: 8 total, 7 up, 6 in 2024-09-10T19:03:34.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:34 smithi018 ceph-mon[40196]: 1.7 scrub starts 2024-09-10T19:03:34.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:34 smithi018 ceph-mon[40196]: 1.7 scrub ok 2024-09-10T19:03:34.454 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:34.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:34.455 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:34.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:34 smithi103 ceph-mon[33163]: 6.a scrub starts 2024-09-10T19:03:34.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:34 smithi103 ceph-mon[33163]: 6.a scrub ok 2024-09-10T19:03:34.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:34 smithi103 ceph-mon[33163]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T19:03:34.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:34 smithi103 ceph-mon[33163]: Cluster is now healthy 2024-09-10T19:03:34.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:34 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T19:03:34.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:34 smithi103 ceph-mon[33163]: osdmap e385: 8 total, 7 up, 6 in 2024-09-10T19:03:34.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:34 smithi103 ceph-mon[33163]: pgmap v847: 168 pgs: 168 active+clean; 611 MiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 1 KiB/s rd, 61 MiB/s wr, 978 op/s 2024-09-10T19:03:34.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:34 smithi103 ceph-mon[33163]: osdmap e386: 8 total, 7 up, 6 in 2024-09-10T19:03:34.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:34 smithi103 ceph-mon[33163]: 1.7 scrub starts 2024-09-10T19:03:34.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:34 smithi103 ceph-mon[33163]: 1.7 scrub ok 2024-09-10T19:03:34.556 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:34.577 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:34.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:34.578 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:34.679 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:34.718 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:34.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:34.719 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:34.820 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:34.850 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:34.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:34.851 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:34.952 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:34.990 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:34.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:34.991 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:35.066 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 7 16 20489 20473 182.765 149.812 0.0041072 0.00546645 2024-09-10T19:03:35.092 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:35.121 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:35.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:35.122 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:35.223 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:35.263 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:35.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:35.265 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:35.366 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:35.409 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:35.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:35.410 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:35.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:35 smithi018 ceph-mon[29336]: 6.1f scrub starts 2024-09-10T19:03:35.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:35 smithi018 ceph-mon[29336]: 6.1f scrub ok 2024-09-10T19:03:35.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:35 smithi018 ceph-mon[29336]: 4.c scrub starts 2024-09-10T19:03:35.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:35 smithi018 ceph-mon[40196]: 6.1f scrub starts 2024-09-10T19:03:35.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:35 smithi018 ceph-mon[40196]: 6.1f scrub ok 2024-09-10T19:03:35.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:35 smithi018 ceph-mon[40196]: 4.c scrub starts 2024-09-10T19:03:35.511 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:35.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:35 smithi103 ceph-mon[33163]: 6.1f scrub starts 2024-09-10T19:03:35.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:35 smithi103 ceph-mon[33163]: 6.1f scrub ok 2024-09-10T19:03:35.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:35 smithi103 ceph-mon[33163]: 4.c scrub starts 2024-09-10T19:03:35.549 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:35.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:35.551 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:35.652 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:35.692 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:35.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:35.693 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:35.793 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:35.832 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:35.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:35.833 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:35.933 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:35.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:35.974 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:35.974 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:36.067 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 8 15 22787 22772 177.879 143.688 0.00445642 0.00561597 2024-09-10T19:03:36.075 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:36.116 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:36.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:36.117 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:36.218 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:36.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:36.261 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:36.261 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:36.300 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:36 smithi018 ceph-mon[29336]: 4.c scrub ok 2024-09-10T19:03:36.300 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:36 smithi018 ceph-mon[29336]: 5.15 scrub starts 2024-09-10T19:03:36.300 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:36 smithi018 ceph-mon[29336]: 5.15 scrub ok 2024-09-10T19:03:36.300 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:36 smithi018 ceph-mon[29336]: pgmap v849: 168 pgs: 168 active+clean; 730 MiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 639 B/s rd, 91 MiB/s wr, 1.46k op/s 2024-09-10T19:03:36.300 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:36 smithi018 ceph-mon[29336]: 2.10 scrub starts 2024-09-10T19:03:36.300 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:36 smithi018 ceph-mon[29336]: 2.10 scrub ok 2024-09-10T19:03:36.300 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:36 smithi018 ceph-mon[29336]: 2.14 scrub starts 2024-09-10T19:03:36.300 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:36 smithi018 ceph-mon[29336]: 2.14 scrub ok 2024-09-10T19:03:36.300 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:36 smithi018 ceph-mon[40196]: 4.c scrub ok 2024-09-10T19:03:36.301 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:36 smithi018 ceph-mon[40196]: 5.15 scrub starts 2024-09-10T19:03:36.301 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:36 smithi018 ceph-mon[40196]: 5.15 scrub ok 2024-09-10T19:03:36.301 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:36 smithi018 ceph-mon[40196]: pgmap v849: 168 pgs: 168 active+clean; 730 MiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 639 B/s rd, 91 MiB/s wr, 1.46k op/s 2024-09-10T19:03:36.301 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:36 smithi018 ceph-mon[40196]: 2.10 scrub starts 2024-09-10T19:03:36.301 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:36 smithi018 ceph-mon[40196]: 2.10 scrub ok 2024-09-10T19:03:36.301 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:36 smithi018 ceph-mon[40196]: 2.14 scrub starts 2024-09-10T19:03:36.301 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:36 smithi018 ceph-mon[40196]: 2.14 scrub ok 2024-09-10T19:03:36.362 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:36.400 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:36.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:36.401 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:36.502 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:36.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:36 smithi103 ceph-mon[33163]: 4.c scrub ok 2024-09-10T19:03:36.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:36 smithi103 ceph-mon[33163]: 5.15 scrub starts 2024-09-10T19:03:36.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:36 smithi103 ceph-mon[33163]: 5.15 scrub ok 2024-09-10T19:03:36.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:36 smithi103 ceph-mon[33163]: pgmap v849: 168 pgs: 168 active+clean; 730 MiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 639 B/s rd, 91 MiB/s wr, 1.46k op/s 2024-09-10T19:03:36.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:36 smithi103 ceph-mon[33163]: 2.10 scrub starts 2024-09-10T19:03:36.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:36 smithi103 ceph-mon[33163]: 2.10 scrub ok 2024-09-10T19:03:36.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:36 smithi103 ceph-mon[33163]: 2.14 scrub starts 2024-09-10T19:03:36.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:36 smithi103 ceph-mon[33163]: 2.14 scrub ok 2024-09-10T19:03:36.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:36.531 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:36.531 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:36.632 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:36.672 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:36.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:36.673 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:36.774 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:36.800 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:36.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:36.801 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:36.902 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:36.940 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:36.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:36.941 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:37.042 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:37.067 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 9 16 24918 24902 172.905 133.125 0.0104429 0.00577691 2024-09-10T19:03:37.083 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:37.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:37.085 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:37.185 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:37.215 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:37.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:37.217 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:37.317 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:37.365 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:37.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:37.366 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:37.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:37 smithi018 ceph-mon[40196]: 2.1b scrub starts 2024-09-10T19:03:37.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:37 smithi018 ceph-mon[40196]: 2.1b scrub ok 2024-09-10T19:03:37.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:37 smithi018 ceph-mon[40196]: 4.12 scrub starts 2024-09-10T19:03:37.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:37 smithi018 ceph-mon[29336]: 2.1b scrub starts 2024-09-10T19:03:37.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:37 smithi018 ceph-mon[29336]: 2.1b scrub ok 2024-09-10T19:03:37.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:37 smithi018 ceph-mon[29336]: 4.12 scrub starts 2024-09-10T19:03:37.468 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:37.518 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:37.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:37.520 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:37.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:37 smithi103 ceph-mon[33163]: 2.1b scrub starts 2024-09-10T19:03:37.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:37 smithi103 ceph-mon[33163]: 2.1b scrub ok 2024-09-10T19:03:37.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:37 smithi103 ceph-mon[33163]: 4.12 scrub starts 2024-09-10T19:03:37.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:03:37 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:03:37.621 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:37.661 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:37.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:37.665 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:37.766 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:37.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:37.808 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:37.808 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:37.812 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-10T19:03:37.909 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:37.950 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:37.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:37.951 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:38.052 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:38.068 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 10 16 27072 27056 169.073 134.625 0.01218 0.00590767 2024-09-10T19:03:38.079 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:38.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:38.081 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:38.173 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T19:03:38.173 INFO:teuthology.orchestra.run.smithi018.stdout: "ops": [ 2024-09-10T19:03:38.173 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:38.173 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.25706.0:27154 11.4 e386/379)", 2024-09-10T19:03:38.173 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:03:38.115524+0000", 2024-09-10T19:03:38.173 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 0.0039530850000000003, 2024-09-10T19:03:38.173 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0039777689999999999, 2024-09-10T19:03:38.173 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:03:38.173 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "queued for pg", 2024-09-10T19:03:38.174 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:03:38.174 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:38.174 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:03:38.174 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:38.115524+0000", 2024-09-10T19:03:38.174 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:38.174 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:38.174 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:38.174 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:03:38.174 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:38.115524+0000", 2024-09-10T19:03:38.174 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:38.174 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:38.174 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:38.175 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:03:38.175 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:38.115524+0000", 2024-09-10T19:03:38.175 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T19:03:38.175 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:38.175 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:38.175 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:03:38.175 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:38.115558+0000", 2024-09-10T19:03:38.175 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.3976999999999997e-05 2024-09-10T19:03:38.175 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:38.175 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:38.175 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:03:38.175 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:38.115567+0000", 2024-09-10T19:03:38.175 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.7189999999999996e-06 2024-09-10T19:03:38.176 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:38.176 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:38.176 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:03:38.176 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:38.115573+0000", 2024-09-10T19:03:38.176 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.2039999999999996e-06 2024-09-10T19:03:38.176 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:38.176 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:03:38.176 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:38.176 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:38.176 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:38.176 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.25706.0:27157 11.3 e386/379)", 2024-09-10T19:03:38.176 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:03:38.116550+0000", 2024-09-10T19:03:38.176 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 0.0029269600000000002, 2024-09-10T19:03:38.177 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0030030679999999998, 2024-09-10T19:03:38.177 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:03:38.177 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "queued for pg", 2024-09-10T19:03:38.177 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:03:38.177 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:38.177 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:03:38.177 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:38.116550+0000", 2024-09-10T19:03:38.177 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:38.177 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:38.177 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:38.177 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:03:38.177 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:38.116550+0000", 2024-09-10T19:03:38.177 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:38.177 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:38.178 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:38.178 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:03:38.178 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:38.116550+0000", 2024-09-10T19:03:38.178 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T19:03:38.178 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:38.178 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:38.178 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:03:38.178 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:38.116677+0000", 2024-09-10T19:03:38.178 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00012692000000000001 2024-09-10T19:03:38.178 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:38.178 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:38.178 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:03:38.178 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:38.116686+0000", 2024-09-10T19:03:38.178 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.4769999999999992e-06 2024-09-10T19:03:38.179 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:38.179 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:38.179 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:03:38.179 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:38.116691+0000", 2024-09-10T19:03:38.179 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.0710000000000002e-06 2024-09-10T19:03:38.179 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:38.179 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:03:38.179 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:38.179 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:38.179 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:38.179 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.25706.0:27152 11.0 e386/379)", 2024-09-10T19:03:38.179 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:03:38.118951+0000", 2024-09-10T19:03:38.179 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 0.00052626400000000003, 2024-09-10T19:03:38.180 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00064935999999999998, 2024-09-10T19:03:38.180 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:03:38.180 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "queued for pg", 2024-09-10T19:03:38.180 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:03:38.180 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:38.180 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:03:38.180 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:38.118951+0000", 2024-09-10T19:03:38.180 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:38.180 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:38.180 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:38.180 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:03:38.180 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:38.118951+0000", 2024-09-10T19:03:38.180 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:03:38.180 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:38.181 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:38.181 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:03:38.181 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:38.118950+0000", 2024-09-10T19:03:38.181 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T19:03:38.181 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:38.181 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:38.181 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:03:38.181 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:38.118979+0000", 2024-09-10T19:03:38.181 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.8742e-05 2024-09-10T19:03:38.181 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:38.181 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:38.181 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:03:38.181 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:38.118993+0000", 2024-09-10T19:03:38.182 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.3689000000000001e-05 2024-09-10T19:03:38.182 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:03:38.182 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:03:38.182 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:03:38.182 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:38.118999+0000", 2024-09-10T19:03:38.182 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.8850000000000004e-06 2024-09-10T19:03:38.182 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:38.182 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:03:38.182 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:38.182 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:03:38.182 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-09-10T19:03:38.182 INFO:teuthology.orchestra.run.smithi018.stdout: "num_ops": 3 2024-09-10T19:03:38.182 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T19:03:38.183 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:38.210 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:38.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:38.211 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:38.312 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:38.355 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:38.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:38.357 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:38.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:38 smithi018 ceph-mon[40196]: 6.2 scrub starts 2024-09-10T19:03:38.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:38 smithi018 ceph-mon[40196]: 6.2 scrub ok 2024-09-10T19:03:38.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:38 smithi018 ceph-mon[40196]: 4.12 scrub ok 2024-09-10T19:03:38.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:38 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:38.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:38 smithi018 ceph-mon[40196]: pgmap v850: 168 pgs: 168 active+clean; 1.4 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 1.2 KiB/s rd, 174 MiB/s wr, 2.79k op/s 2024-09-10T19:03:38.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:38 smithi018 ceph-mon[29336]: 6.2 scrub starts 2024-09-10T19:03:38.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:38 smithi018 ceph-mon[29336]: 6.2 scrub ok 2024-09-10T19:03:38.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:38 smithi018 ceph-mon[29336]: 4.12 scrub ok 2024-09-10T19:03:38.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:38 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:38.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:38 smithi018 ceph-mon[29336]: pgmap v850: 168 pgs: 168 active+clean; 1.4 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 1.2 KiB/s rd, 174 MiB/s wr, 2.79k op/s 2024-09-10T19:03:38.458 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:38.484 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:38.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:38.485 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:38.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:38 smithi103 ceph-mon[33163]: 6.2 scrub starts 2024-09-10T19:03:38.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:38 smithi103 ceph-mon[33163]: 6.2 scrub ok 2024-09-10T19:03:38.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:38 smithi103 ceph-mon[33163]: 4.12 scrub ok 2024-09-10T19:03:38.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:38 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:38.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:38 smithi103 ceph-mon[33163]: pgmap v850: 168 pgs: 168 active+clean; 1.4 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 1.2 KiB/s rd, 174 MiB/s wr, 2.79k op/s 2024-09-10T19:03:38.590 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:38.591 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T19:03:38.594 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-09-10T19:03:38.623 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:38.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:38.624 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:38.724 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:38.757 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:38.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:38.758 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:38.859 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:38.896 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:38.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:38.898 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:38.998 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:39.037 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:39.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:39.038 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:39.067 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 11 16 29153 29137 165.526 130.062 0.00805681 0.00603617 2024-09-10T19:03:39.140 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:39.181 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:39.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:39.182 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:39.283 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:39.313 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:39.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:39.315 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:39.416 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:39.443 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:39.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:39.444 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:39.493 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:39 smithi018 ceph-mon[29336]: 4.13 scrub starts 2024-09-10T19:03:39.493 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:39 smithi018 ceph-mon[29336]: 4.13 scrub ok 2024-09-10T19:03:39.493 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:39 smithi018 ceph-mon[29336]: from='client.25718 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:03:39.493 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:39 smithi018 ceph-mon[29336]: osdmap e387: 8 total, 7 up, 6 in 2024-09-10T19:03:39.493 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:39 smithi018 ceph-mon[29336]: 2.1f scrub starts 2024-09-10T19:03:39.493 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:39 smithi018 ceph-mon[29336]: 2.1f scrub ok 2024-09-10T19:03:39.495 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:39 smithi018 ceph-mon[40196]: 4.13 scrub starts 2024-09-10T19:03:39.495 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:39 smithi018 ceph-mon[40196]: 4.13 scrub ok 2024-09-10T19:03:39.495 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:39 smithi018 ceph-mon[40196]: from='client.25718 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:03:39.495 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:39 smithi018 ceph-mon[40196]: osdmap e387: 8 total, 7 up, 6 in 2024-09-10T19:03:39.495 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:39 smithi018 ceph-mon[40196]: 2.1f scrub starts 2024-09-10T19:03:39.495 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:39 smithi018 ceph-mon[40196]: 2.1f scrub ok 2024-09-10T19:03:39.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:39 smithi103 ceph-mon[33163]: 4.13 scrub starts 2024-09-10T19:03:39.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:39 smithi103 ceph-mon[33163]: 4.13 scrub ok 2024-09-10T19:03:39.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:39 smithi103 ceph-mon[33163]: from='client.25718 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:03:39.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:39 smithi103 ceph-mon[33163]: osdmap e387: 8 total, 7 up, 6 in 2024-09-10T19:03:39.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:39 smithi103 ceph-mon[33163]: 2.1f scrub starts 2024-09-10T19:03:39.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:39 smithi103 ceph-mon[33163]: 2.1f scrub ok 2024-09-10T19:03:39.545 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:39.568 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:39.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:39.569 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:39.669 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:39.701 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:39.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:39.702 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:39.803 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:39.838 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:39.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:39.839 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:39.940 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:39.972 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:39.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:39.974 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:40.067 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 12 16 31115 31099 161.95 122.625 0.00741024 0.00616901 2024-09-10T19:03:40.074 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:40.098 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:40.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:40.099 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:40.200 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:40.245 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:40.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:40.246 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:40.346 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:40.385 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:40.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:40.387 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:40.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:40 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:03:40.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:40 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:03:40.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:40 smithi018 ceph-mon[40196]: pgmap v852: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 1.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 814 B/s rd, 142 MiB/s wr, 2.27k op/s 2024-09-10T19:03:40.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:40 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:03:40.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:40 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:03:40.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:40 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:03:40.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:40 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:03:40.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:40 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:03:40.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:40 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:03:40.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:40 smithi018 ceph-mon[29336]: pgmap v852: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 1.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 814 B/s rd, 142 MiB/s wr, 2.27k op/s 2024-09-10T19:03:40.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:40 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:03:40.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:40 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:03:40.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:40 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:03:40.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:40 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:03:40.446 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:03:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T19:03:40.093+0000 7f66d46e1640 -1 mgr get_metadata_python Requested missing service osd.7 2024-09-10T19:03:40.487 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:40.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:03:40 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:03:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:03:40.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:40 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:03:40.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:40 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:03:40.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:40 smithi103 ceph-mon[33163]: pgmap v852: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 1.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 814 B/s rd, 142 MiB/s wr, 2.27k op/s 2024-09-10T19:03:40.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:40 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:03:40.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:40 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:03:40.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:40 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:03:40.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:40 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:03:40.527 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:40.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:40.528 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:40.629 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:40.651 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:40.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:40.652 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:40.753 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:40.782 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:40.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:40.783 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:40.884 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:40.919 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:40.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:40.920 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:41.021 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:41.061 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:41.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:41.062 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:41.067 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 13 16 32828 32812 157.727 107.062 0.0120175 0.00633381 2024-09-10T19:03:41.163 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:41.201 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:41.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:41.203 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:41.304 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:41.326 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:41.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:41.327 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:41.428 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:41.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:41 smithi018 ceph-mon[40196]: osdmap e388: 8 total, 7 up, 6 in 2024-09-10T19:03:41.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:41 smithi018 ceph-mon[40196]: 4.1d deep-scrub starts 2024-09-10T19:03:41.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:41 smithi018 ceph-mon[40196]: 4.6 scrub starts 2024-09-10T19:03:41.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:41 smithi018 ceph-mon[40196]: 4.6 scrub ok 2024-09-10T19:03:41.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:41 smithi018 ceph-mon[29336]: osdmap e388: 8 total, 7 up, 6 in 2024-09-10T19:03:41.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:41 smithi018 ceph-mon[29336]: 4.1d deep-scrub starts 2024-09-10T19:03:41.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:41 smithi018 ceph-mon[29336]: 4.6 scrub starts 2024-09-10T19:03:41.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:41 smithi018 ceph-mon[29336]: 4.6 scrub ok 2024-09-10T19:03:41.445 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:03:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:03:40] "GET /metrics HTTP/1.1" 200 209561 "" "Prometheus/2.43.0" 2024-09-10T19:03:41.459 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:41.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:41.460 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:41.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:41 smithi103 ceph-mon[33163]: osdmap e388: 8 total, 7 up, 6 in 2024-09-10T19:03:41.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:41 smithi103 ceph-mon[33163]: 4.1d deep-scrub starts 2024-09-10T19:03:41.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:41 smithi103 ceph-mon[33163]: 4.6 scrub starts 2024-09-10T19:03:41.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:41 smithi103 ceph-mon[33163]: 4.6 scrub ok 2024-09-10T19:03:41.561 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:41.597 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:41.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:41.598 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:41.699 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:41.739 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:41.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:41.740 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:41.841 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:41.873 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:41.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:41.874 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:41.975 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:41.997 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:41.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:41.998 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:42.067 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 14 16 34432 34416 153.622 100.25 0.00437214 0.00650219 2024-09-10T19:03:42.099 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:42.137 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:42.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:42.139 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:42.239 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:42.271 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:42.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:42.272 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:42.373 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:42.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:42.409 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:42.409 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:42.510 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:42.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:42 smithi103 ceph-mon[33163]: 4.1d deep-scrub ok 2024-09-10T19:03:42.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:42 smithi103 ceph-mon[33163]: pgmap v854: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 1.7 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 667 B/s rd, 147 MiB/s wr, 2.35k op/s 2024-09-10T19:03:42.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:42 smithi103 ceph-mon[33163]: 6.5 scrub starts 2024-09-10T19:03:42.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:42 smithi103 ceph-mon[33163]: 6.5 scrub ok 2024-09-10T19:03:42.546 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:42.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:42.547 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:42.648 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:42.687 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:42.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:42.689 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:42.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:42 smithi018 ceph-mon[40196]: 4.1d deep-scrub ok 2024-09-10T19:03:42.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:42 smithi018 ceph-mon[40196]: pgmap v854: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 1.7 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 667 B/s rd, 147 MiB/s wr, 2.35k op/s 2024-09-10T19:03:42.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:42 smithi018 ceph-mon[40196]: 6.5 scrub starts 2024-09-10T19:03:42.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:42 smithi018 ceph-mon[40196]: 6.5 scrub ok 2024-09-10T19:03:42.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:42 smithi018 ceph-mon[29336]: 4.1d deep-scrub ok 2024-09-10T19:03:42.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:42 smithi018 ceph-mon[29336]: pgmap v854: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 1.7 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 667 B/s rd, 147 MiB/s wr, 2.35k op/s 2024-09-10T19:03:42.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:42 smithi018 ceph-mon[29336]: 6.5 scrub starts 2024-09-10T19:03:42.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:42 smithi018 ceph-mon[29336]: 6.5 scrub ok 2024-09-10T19:03:42.790 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:42.821 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:42.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:42.822 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:42.923 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:42.962 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:42.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:42.963 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:43.064 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:43.067 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 15 16 36236 36220 150.896 112.75 0.00673425 0.00662223 2024-09-10T19:03:43.104 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:43.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:43.105 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:43.205 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:43.246 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:43.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:43.248 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:43.348 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:43.391 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:43.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:43.393 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:43.493 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:43.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:43 smithi103 ceph-mon[33163]: 6.1a scrub starts 2024-09-10T19:03:43.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:43 smithi103 ceph-mon[33163]: 6.1a scrub ok 2024-09-10T19:03:43.526 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:43.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:43.527 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:43.628 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:43.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:43.668 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:43.668 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:43.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:43 smithi018 ceph-mon[40196]: 6.1a scrub starts 2024-09-10T19:03:43.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:43 smithi018 ceph-mon[40196]: 6.1a scrub ok 2024-09-10T19:03:43.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:43 smithi018 ceph-mon[29336]: 6.1a scrub starts 2024-09-10T19:03:43.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:43 smithi018 ceph-mon[29336]: 6.1a scrub ok 2024-09-10T19:03:43.769 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:43.810 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:43.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:43.811 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:43.911 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:43.955 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:43.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:43.956 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:44.057 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:44.067 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 16 15 38083 38068 148.684 115.5 0.00800631 0.00672033 2024-09-10T19:03:44.086 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:44.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:44.087 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:44.188 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:44.214 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:44.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:44.215 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:44.316 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:44.360 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:44.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:44.361 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:44.462 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:44.501 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:44.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:44.503 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:44.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:44 smithi103 ceph-mon[33163]: 4.e scrub starts 2024-09-10T19:03:44.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:44 smithi103 ceph-mon[33163]: 4.e scrub ok 2024-09-10T19:03:44.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:44 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:03:44.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:44 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:03:44.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:44 smithi103 ceph-mon[33163]: 3.0 scrub starts 2024-09-10T19:03:44.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:44 smithi103 ceph-mon[33163]: 3.0 scrub ok 2024-09-10T19:03:44.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:44 smithi103 ceph-mon[33163]: osdmap e389: 8 total, 7 up, 6 in 2024-09-10T19:03:44.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:44 smithi103 ceph-mon[33163]: pgmap v856: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 2.0 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 852 B/s rd, 105 MiB/s wr, 1.68k op/s 2024-09-10T19:03:44.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:44 smithi103 ceph-mon[33163]: 4.11 scrub starts 2024-09-10T19:03:44.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:44 smithi103 ceph-mon[33163]: 4.11 scrub ok 2024-09-10T19:03:44.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:44 smithi103 ceph-mon[33163]: osdmap e390: 8 total, 7 up, 6 in 2024-09-10T19:03:44.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:44 smithi103 ceph-mon[33163]: 4.b scrub starts 2024-09-10T19:03:44.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:44 smithi103 ceph-mon[33163]: 4.b scrub ok 2024-09-10T19:03:44.604 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:44.642 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:44.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:44.643 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:44.677 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:44 smithi018 ceph-mon[40196]: 4.e scrub starts 2024-09-10T19:03:44.677 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:44 smithi018 ceph-mon[40196]: 4.e scrub ok 2024-09-10T19:03:44.677 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:44 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:03:44.677 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:44 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:03:44.677 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:44 smithi018 ceph-mon[40196]: 3.0 scrub starts 2024-09-10T19:03:44.677 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:44 smithi018 ceph-mon[40196]: 3.0 scrub ok 2024-09-10T19:03:44.677 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:44 smithi018 ceph-mon[40196]: osdmap e389: 8 total, 7 up, 6 in 2024-09-10T19:03:44.677 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:44 smithi018 ceph-mon[40196]: pgmap v856: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 2.0 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 852 B/s rd, 105 MiB/s wr, 1.68k op/s 2024-09-10T19:03:44.677 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:44 smithi018 ceph-mon[40196]: 4.11 scrub starts 2024-09-10T19:03:44.678 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:44 smithi018 ceph-mon[40196]: 4.11 scrub ok 2024-09-10T19:03:44.678 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:44 smithi018 ceph-mon[40196]: osdmap e390: 8 total, 7 up, 6 in 2024-09-10T19:03:44.678 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:44 smithi018 ceph-mon[40196]: 4.b scrub starts 2024-09-10T19:03:44.678 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:44 smithi018 ceph-mon[40196]: 4.b scrub ok 2024-09-10T19:03:44.678 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:44 smithi018 ceph-mon[29336]: 4.e scrub starts 2024-09-10T19:03:44.678 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:44 smithi018 ceph-mon[29336]: 4.e scrub ok 2024-09-10T19:03:44.678 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:44 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:03:44.678 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:44 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:03:44.678 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:44 smithi018 ceph-mon[29336]: 3.0 scrub starts 2024-09-10T19:03:44.678 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:44 smithi018 ceph-mon[29336]: 3.0 scrub ok 2024-09-10T19:03:44.679 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:44 smithi018 ceph-mon[29336]: osdmap e389: 8 total, 7 up, 6 in 2024-09-10T19:03:44.679 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:44 smithi018 ceph-mon[29336]: pgmap v856: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 2.0 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 852 B/s rd, 105 MiB/s wr, 1.68k op/s 2024-09-10T19:03:44.679 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:44 smithi018 ceph-mon[29336]: 4.11 scrub starts 2024-09-10T19:03:44.679 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:44 smithi018 ceph-mon[29336]: 4.11 scrub ok 2024-09-10T19:03:44.679 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:44 smithi018 ceph-mon[29336]: osdmap e390: 8 total, 7 up, 6 in 2024-09-10T19:03:44.679 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:44 smithi018 ceph-mon[29336]: 4.b scrub starts 2024-09-10T19:03:44.679 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:44 smithi018 ceph-mon[29336]: 4.b scrub ok 2024-09-10T19:03:44.744 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:44.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:44.774 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:44.774 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:44.875 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:44.919 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:44.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:44.920 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:45.021 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:45.033 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":850,"stamp":"2024-09-10T19:03:37.437940+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1484676798,"num_objects":22841,"num_object_clones":0,"num_object_copies":68523,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22841,"num_whiteouts":0,"num_read":1306,"num_read_kb":1061,"num_write":23364,"num_write_kb":1449372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":303369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25217,"ondisk_log_size":25217,"up":504,"acting":504,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":507,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":562470912,"kb_used":10096600,"kb_used_data":5005268,"kb_used_omap":12,"kb_used_meta":5091251,"kb_avail":552374312,"statfs":{"total":575970213888,"available":565631295488,"internally_reserved":0,"allocated":5125394432,"data_stored":5122682196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13122,"internal_metadata":5213441214},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,8,5,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1464074240,"num_objects":22340,"num_object_clones":0,"num_object_copies":67020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22340,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":22340,"num_write_kb":1429760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.004027"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":399,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.167301+0000","last_change":"2024-09-10T19:02:21.219606+0000","last_active":"2024-09-10T19:03:33.167301+0000","last_peered":"2024-09-10T19:03:33.167301+0000","last_clean":"2024-09-10T19:03:33.167301+0000","last_became_active":"2024-09-10T18:57:02.460451+0000","last_became_peered":"2024-09-10T18:57:02.460451+0000","last_unstale":"2024-09-10T19:03:33.167301+0000","last_undegraded":"2024-09-10T19:03:33.167301+0000","last_fullsized":"2024-09-10T19:03:33.167301+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:21.219520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:02:21.219520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056254590000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":386,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.165939+0000","last_change":"2024-09-10T18:59:59.848814+0000","last_active":"2024-09-10T19:03:33.165939+0000","last_peered":"2024-09-10T19:03:33.165939+0000","last_clean":"2024-09-10T19:03:33.165939+0000","last_became_active":"2024-09-10T18:57:02.440648+0000","last_became_peered":"2024-09-10T18:57:02.440648+0000","last_unstale":"2024-09-10T19:03:33.165939+0000","last_undegraded":"2024-09-10T19:03:33.165939+0000","last_fullsized":"2024-09-10T19:03:33.165939+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:59.848700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:59.848700+0000","last_clean_scrub_stamp":"2024-09-10T18:59:59.848700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047956400000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"232'27","reported_seq":484,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.165827+0000","last_change":"2024-09-10T19:02:20.251547+0000","last_active":"2024-09-10T19:03:33.165827+0000","last_peered":"2024-09-10T19:03:33.165827+0000","last_clean":"2024-09-10T19:03:33.165827+0000","last_became_active":"2024-09-10T18:55:40.394717+0000","last_became_peered":"2024-09-10T18:55:40.394717+0000","last_unstale":"2024-09-10T19:03:33.165827+0000","last_undegraded":"2024-09-10T19:03:33.165827+0000","last_fullsized":"2024-09-10T19:03:33.165827+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'27","last_scrub_stamp":"2024-09-10T19:02:20.251456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:20.251456+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011781700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":391,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.153061+0000","last_change":"2024-09-10T19:00:08.841172+0000","last_active":"2024-09-10T19:03:33.153061+0000","last_peered":"2024-09-10T19:03:33.153061+0000","last_clean":"2024-09-10T19:03:33.153061+0000","last_became_active":"2024-09-10T18:45:22.065619+0000","last_became_peered":"2024-09-10T18:45:22.065619+0000","last_unstale":"2024-09-10T19:03:33.153061+0000","last_undegraded":"2024-09-10T19:03:33.153061+0000","last_fullsized":"2024-09-10T19:03:33.153061+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:08.841125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.057932+0000","last_clean_scrub_stamp":"2024-09-10T19:00:08.841125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001059857,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":462,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.167618+0000","last_change":"2024-09-10T19:02:17.284387+0000","last_active":"2024-09-10T19:03:33.167618+0000","last_peered":"2024-09-10T19:03:33.167618+0000","last_clean":"2024-09-10T19:03:33.167618+0000","last_became_active":"2024-09-10T19:00:42.518824+0000","last_became_peered":"2024-09-10T19:00:42.518824+0000","last_unstale":"2024-09-10T19:03:33.167618+0000","last_undegraded":"2024-09-10T19:03:33.167618+0000","last_fullsized":"2024-09-10T19:03:33.167618+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:17.284305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:17.284305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018984430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"232'19","reported_seq":412,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.153015+0000","last_change":"2024-09-10T18:59:09.642743+0000","last_active":"2024-09-10T19:03:33.153015+0000","last_peered":"2024-09-10T19:03:33.153015+0000","last_clean":"2024-09-10T19:03:33.153015+0000","last_became_active":"2024-09-10T18:45:20.061266+0000","last_became_peered":"2024-09-10T18:45:20.061266+0000","last_unstale":"2024-09-10T19:03:33.153015+0000","last_undegraded":"2024-09-10T19:03:33.153015+0000","last_fullsized":"2024-09-10T19:03:33.153015+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T18:59:09.642642+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_clean_scrub_stamp":"2024-09-10T18:59:09.642642+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0095920940000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":27,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.155225+0000","last_change":"2024-09-10T19:02:49.572248+0000","last_active":"2024-09-10T19:03:33.155225+0000","last_peered":"2024-09-10T19:03:33.155225+0000","last_clean":"2024-09-10T19:03:33.155225+0000","last_became_active":"2024-09-10T19:02:49.571992+0000","last_became_peered":"2024-09-10T19:02:49.571992+0000","last_unstale":"2024-09-10T19:03:33.155225+0000","last_undegraded":"2024-09-10T19:03:33.155225+0000","last_fullsized":"2024-09-10T19:03:33.155225+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:01:54.384604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:01.135988+0000","last_clean_scrub_stamp":"2024-09-10T19:01:54.384604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002631478,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":379,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.153597+0000","last_change":"2024-09-10T18:58:51.682046+0000","last_active":"2024-09-10T19:03:33.153597+0000","last_peered":"2024-09-10T19:03:33.153597+0000","last_clean":"2024-09-10T19:03:33.153597+0000","last_became_active":"2024-09-10T18:45:24.081002+0000","last_became_peered":"2024-09-10T18:45:24.081002+0000","last_unstale":"2024-09-10T19:03:33.153597+0000","last_undegraded":"2024-09-10T19:03:33.153597+0000","last_fullsized":"2024-09-10T19:03:33.153597+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:51.681994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:58:51.681994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051776249999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":394,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.159212+0000","last_change":"2024-09-10T19:03:30.120786+0000","last_active":"2024-09-10T19:03:33.159212+0000","last_peered":"2024-09-10T19:03:33.159212+0000","last_clean":"2024-09-10T19:03:33.159212+0000","last_became_active":"2024-09-10T18:57:02.452259+0000","last_became_peered":"2024-09-10T18:57:02.452259+0000","last_unstale":"2024-09-10T19:03:33.159212+0000","last_undegraded":"2024-09-10T19:03:33.159212+0000","last_fullsized":"2024-09-10T19:03:33.159212+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:30.120695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:21.501984+0000","last_clean_scrub_stamp":"2024-09-10T19:03:30.120695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:46.598921+0000","scrub_duration":0.0034594489999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1b","version":"232'9","reported_seq":398,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.153660+0000","last_change":"2024-09-10T18:59:11.622597+0000","last_active":"2024-09-10T19:03:33.153660+0000","last_peered":"2024-09-10T19:03:33.153660+0000","last_clean":"2024-09-10T19:03:33.153660+0000","last_became_active":"2024-09-10T18:45:20.057567+0000","last_became_peered":"2024-09-10T18:45:20.057567+0000","last_unstale":"2024-09-10T19:03:33.153660+0000","last_undegraded":"2024-09-10T19:03:33.153660+0000","last_fullsized":"2024-09-10T19:03:33.153660+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'9","last_scrub_stamp":"2024-09-10T18:59:11.622509+0000","last_deep_scrub":"163'7","last_deep_scrub_stamp":"2024-09-10T18:51:54.261879+0000","last_clean_scrub_stamp":"2024-09-10T18:59:11.622509+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056120609999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":150,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.153520+0000","last_change":"2024-09-10T19:00:40.947163+0000","last_active":"2024-09-10T19:03:33.153520+0000","last_peered":"2024-09-10T19:03:33.153520+0000","last_clean":"2024-09-10T19:03:33.153520+0000","last_became_active":"2024-09-10T18:58:58.503924+0000","last_became_peered":"2024-09-10T18:58:58.503924+0000","last_unstale":"2024-09-10T19:03:33.153520+0000","last_undegraded":"2024-09-10T19:03:33.153520+0000","last_fullsized":"2024-09-10T19:03:33.153520+0000","mapping_epoch":303,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":304,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:40.947066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:36.509971+0000","last_clean_scrub_stamp":"2024-09-10T19:00:40.947066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012293715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":67,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.155294+0000","last_change":"2024-09-10T19:03:23.759714+0000","last_active":"2024-09-10T19:03:33.155294+0000","last_peered":"2024-09-10T19:03:33.155294+0000","last_clean":"2024-09-10T19:03:33.155294+0000","last_became_active":"2024-09-10T19:00:42.515840+0000","last_became_peered":"2024-09-10T19:00:42.515840+0000","last_unstale":"2024-09-10T19:03:33.155294+0000","last_undegraded":"2024-09-10T19:03:33.155294+0000","last_fullsized":"2024-09-10T19:03:33.155294+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:23.759624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:03:23.759624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:52.320552+0000","scrub_duration":0.0037605970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":200,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.156019+0000","last_change":"2024-09-10T19:02:12.656894+0000","last_active":"2024-09-10T19:03:33.156019+0000","last_peered":"2024-09-10T19:03:33.156019+0000","last_clean":"2024-09-10T19:03:33.156019+0000","last_became_active":"2024-09-10T18:57:02.440497+0000","last_became_peered":"2024-09-10T18:57:02.440497+0000","last_unstale":"2024-09-10T19:03:33.156019+0000","last_undegraded":"2024-09-10T19:03:33.156019+0000","last_fullsized":"2024-09-10T19:03:33.156019+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:12.656811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:12.656811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0082345850000000009,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"232'15","reported_seq":447,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.153895+0000","last_change":"2024-09-10T19:02:22.113293+0000","last_active":"2024-09-10T19:03:33.153895+0000","last_peered":"2024-09-10T19:03:33.153895+0000","last_clean":"2024-09-10T19:03:33.153895+0000","last_became_active":"2024-09-10T18:55:40.391571+0000","last_became_peered":"2024-09-10T18:55:40.391571+0000","last_unstale":"2024-09-10T19:03:33.153895+0000","last_undegraded":"2024-09-10T19:03:33.153895+0000","last_fullsized":"2024-09-10T19:03:33.153895+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T19:02:22.113140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:22.113140+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0061707400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":430,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.155676+0000","last_change":"2024-09-10T19:02:55.736359+0000","last_active":"2024-09-10T19:03:33.155676+0000","last_peered":"2024-09-10T19:03:33.155676+0000","last_clean":"2024-09-10T19:03:33.155676+0000","last_became_active":"2024-09-10T18:57:02.431585+0000","last_became_peered":"2024-09-10T18:57:02.431585+0000","last_unstale":"2024-09-10T19:03:33.155676+0000","last_undegraded":"2024-09-10T19:03:33.155676+0000","last_fullsized":"2024-09-10T19:03:33.155676+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:55.736308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:02:55.736308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:57.259852+0000","scrub_duration":0.0012293950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":85,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.161803+0000","last_change":"2024-09-10T19:02:45.952977+0000","last_active":"2024-09-10T19:03:33.161803+0000","last_peered":"2024-09-10T19:03:33.161803+0000","last_clean":"2024-09-10T19:03:33.161803+0000","last_became_active":"2024-09-10T18:59:48.737788+0000","last_became_peered":"2024-09-10T18:59:48.737788+0000","last_unstale":"2024-09-10T19:03:33.161803+0000","last_undegraded":"2024-09-10T19:03:33.161803+0000","last_fullsized":"2024-09-10T19:03:33.161803+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:45.952815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:59.425718+0000","last_clean_scrub_stamp":"2024-09-10T19:02:45.952815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:59.691910+0000","scrub_duration":0.043052992999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"386'1444","reported_seq":1458,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-09-10T19:03:36.066923+0000","last_change":"2024-09-10T19:03:16.041580+0000","last_active":"2024-09-10T19:03:36.066923+0000","last_peered":"2024-09-10T19:03:36.066923+0000","last_clean":"2024-09-10T19:03:36.066923+0000","last_became_active":"2024-09-10T19:03:16.041295+0000","last_became_peered":"2024-09-10T19:03:16.041295+0000","last_unstale":"2024-09-10T19:03:36.066923+0000","last_undegraded":"2024-09-10T19:03:36.066923+0000","last_fullsized":"2024-09-10T19:03:36.066923+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":1444,"log_dups_size":0,"ondisk_log_size":1444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:24.799591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":94633984,"num_objects":1444,"num_object_clones":0,"num_object_copies":4332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1444,"num_write_kb":92416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"70'81","reported_seq":71,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.155230+0000","last_change":"2024-09-10T19:03:21.755540+0000","last_active":"2024-09-10T19:03:33.155230+0000","last_peered":"2024-09-10T19:03:33.155230+0000","last_clean":"2024-09-10T19:03:33.155230+0000","last_became_active":"2024-09-10T19:00:42.517072+0000","last_became_peered":"2024-09-10T19:00:42.517072+0000","last_unstale":"2024-09-10T19:03:33.155230+0000","last_undegraded":"2024-09-10T19:03:33.155230+0000","last_fullsized":"2024-09-10T19:03:33.155230+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:03:21.755481+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:03:21.755481+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:26.030496+0000","scrub_duration":0.0016110499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":262688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"232'39","reported_seq":478,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.154848+0000","last_change":"2024-09-10T18:59:47.745859+0000","last_active":"2024-09-10T19:03:33.154848+0000","last_peered":"2024-09-10T19:03:33.154848+0000","last_clean":"2024-09-10T19:03:33.154848+0000","last_became_active":"2024-09-10T18:47:36.409300+0000","last_became_peered":"2024-09-10T18:47:36.409300+0000","last_unstale":"2024-09-10T19:03:33.154848+0000","last_undegraded":"2024-09-10T19:03:33.154848+0000","last_fullsized":"2024-09-10T19:03:33.154848+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'39","last_scrub_stamp":"2024-09-10T18:59:47.745766+0000","last_deep_scrub":"78'25","last_deep_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_clean_scrub_stamp":"2024-09-10T18:59:47.745766+0000","objects_scrubbed":8,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011279000000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":137,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.153858+0000","last_change":"2024-09-10T19:00:09.867278+0000","last_active":"2024-09-10T19:03:33.153858+0000","last_peered":"2024-09-10T19:03:33.153858+0000","last_clean":"2024-09-10T19:03:33.153858+0000","last_became_active":"2024-09-10T18:57:02.437260+0000","last_became_peered":"2024-09-10T18:57:02.437260+0000","last_unstale":"2024-09-10T19:03:33.153858+0000","last_undegraded":"2024-09-10T19:03:33.153858+0000","last_fullsized":"2024-09-10T19:03:33.153858+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:09.867183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:00:09.867183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026871460000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":254,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.153903+0000","last_change":"2024-09-10T19:02:49.594351+0000","last_active":"2024-09-10T19:03:33.153903+0000","last_peered":"2024-09-10T19:03:33.153903+0000","last_clean":"2024-09-10T19:03:33.153903+0000","last_became_active":"2024-09-10T19:02:49.594088+0000","last_became_peered":"2024-09-10T19:02:49.594088+0000","last_unstale":"2024-09-10T19:03:33.153903+0000","last_undegraded":"2024-09-10T19:03:33.153903+0000","last_fullsized":"2024-09-10T19:03:33.153903+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:09.470293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:04.512425+0000","last_clean_scrub_stamp":"2024-09-10T19:00:09.470293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016498730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":308,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.166090+0000","last_change":"2024-09-10T19:02:58.770406+0000","last_active":"2024-09-10T19:03:33.166090+0000","last_peered":"2024-09-10T19:03:33.166090+0000","last_clean":"2024-09-10T19:03:33.166090+0000","last_became_active":"2024-09-10T18:55:40.388154+0000","last_became_peered":"2024-09-10T18:55:40.388154+0000","last_unstale":"2024-09-10T19:03:33.166090+0000","last_undegraded":"2024-09-10T19:03:33.166090+0000","last_fullsized":"2024-09-10T19:03:33.166090+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:58.770344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:02:58.770344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:17.389406+0000","scrub_duration":0.0016377200000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3","version":"386'1465","reported_seq":1480,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-09-10T19:03:36.568147+0000","last_change":"2024-09-10T19:03:16.046339+0000","last_active":"2024-09-10T19:03:36.568147+0000","last_peered":"2024-09-10T19:03:36.568147+0000","last_clean":"2024-09-10T19:03:36.568147+0000","last_became_active":"2024-09-10T19:03:16.045964+0000","last_became_peered":"2024-09-10T19:03:16.045964+0000","last_unstale":"2024-09-10T19:03:36.568147+0000","last_undegraded":"2024-09-10T19:03:36.568147+0000","last_fullsized":"2024-09-10T19:03:36.568147+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":1465,"log_dups_size":0,"ondisk_log_size":1465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:30.109053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96010240,"num_objects":1465,"num_object_clones":0,"num_object_copies":4395,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1465,"num_write_kb":93760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.9","version":"70'81","reported_seq":293,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.166208+0000","last_change":"2024-09-10T18:59:56.164487+0000","last_active":"2024-09-10T19:03:33.166208+0000","last_peered":"2024-09-10T19:03:33.166208+0000","last_clean":"2024-09-10T19:03:33.166208+0000","last_became_active":"2024-09-10T18:57:02.463750+0000","last_became_peered":"2024-09-10T18:57:02.463750+0000","last_unstale":"2024-09-10T19:03:33.166208+0000","last_undegraded":"2024-09-10T19:03:33.166208+0000","last_fullsized":"2024-09-10T19:03:33.166208+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:56.164384+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:59:56.164384+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041056720000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"232'25","reported_seq":428,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-09-10T19:03:34.532113+0000","last_change":"2024-09-10T19:03:34.532084+0000","last_active":"2024-09-10T19:03:34.532113+0000","last_peered":"2024-09-10T19:03:34.532113+0000","last_clean":"2024-09-10T19:03:34.532113+0000","last_became_active":"2024-09-10T18:45:20.053428+0000","last_became_peered":"2024-09-10T18:45:20.053428+0000","last_unstale":"2024-09-10T19:03:34.532113+0000","last_undegraded":"2024-09-10T19:03:34.532113+0000","last_fullsized":"2024-09-10T19:03:34.532113+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'25","last_scrub_stamp":"2024-09-10T19:03:34.532019+0000","last_deep_scrub":"116'15","last_deep_scrub_stamp":"2024-09-10T18:49:12.374527+0000","last_clean_scrub_stamp":"2024-09-10T19:03:34.532019+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:38.997474+0000","scrub_duration":0.0020774220000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":299,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.167249+0000","last_change":"2024-09-10T19:02:49.572280+0000","last_active":"2024-09-10T19:03:33.167249+0000","last_peered":"2024-09-10T19:03:33.167249+0000","last_clean":"2024-09-10T19:03:33.167249+0000","last_became_active":"2024-09-10T19:02:49.571995+0000","last_became_peered":"2024-09-10T19:02:49.571995+0000","last_unstale":"2024-09-10T19:03:33.167249+0000","last_undegraded":"2024-09-10T19:03:33.167249+0000","last_fullsized":"2024-09-10T19:03:33.167249+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:01.343214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:01.343214+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044818080000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"273'13","reported_seq":477,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.780006+0000","last_change":"2024-09-10T19:01:59.557656+0000","last_active":"2024-09-10T19:03:33.780006+0000","last_peered":"2024-09-10T19:03:33.780006+0000","last_clean":"2024-09-10T19:03:33.780006+0000","last_became_active":"2024-09-10T18:57:02.448258+0000","last_became_peered":"2024-09-10T18:57:02.448258+0000","last_unstale":"2024-09-10T19:03:33.780006+0000","last_undegraded":"2024-09-10T19:03:33.780006+0000","last_fullsized":"2024-09-10T19:03:33.780006+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'13","last_scrub_stamp":"2024-09-10T19:01:59.557500+0000","last_deep_scrub":"202'10","last_deep_scrub_stamp":"2024-09-10T18:53:45.547686+0000","last_clean_scrub_stamp":"2024-09-10T19:01:59.557500+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027644549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":429,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.154403+0000","last_change":"2024-09-10T19:00:42.515842+0000","last_active":"2024-09-10T19:03:33.154403+0000","last_peered":"2024-09-10T19:03:33.154403+0000","last_clean":"2024-09-10T19:03:33.154403+0000","last_became_active":"2024-09-10T19:00:42.515523+0000","last_became_peered":"2024-09-10T19:00:42.515523+0000","last_unstale":"2024-09-10T19:03:33.154403+0000","last_undegraded":"2024-09-10T19:03:33.154403+0000","last_fullsized":"2024-09-10T19:03:33.154403+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:07.803916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:32.043666+0000","last_clean_scrub_stamp":"2024-09-10T19:00:07.803916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0011072429999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.0","version":"386'1469","reported_seq":1483,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-09-10T19:03:36.075284+0000","last_change":"2024-09-10T19:03:16.042622+0000","last_active":"2024-09-10T19:03:36.075284+0000","last_peered":"2024-09-10T19:03:36.075284+0000","last_clean":"2024-09-10T19:03:36.075284+0000","last_became_active":"2024-09-10T19:03:16.042319+0000","last_became_peered":"2024-09-10T19:03:16.042319+0000","last_unstale":"2024-09-10T19:03:36.075284+0000","last_undegraded":"2024-09-10T19:03:36.075284+0000","last_fullsized":"2024-09-10T19:03:36.075284+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":1469,"log_dups_size":0,"ondisk_log_size":1469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:27.169710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96272384,"num_objects":1469,"num_object_clones":0,"num_object_copies":4407,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1469,"num_write_kb":94016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.a","version":"70'81","reported_seq":318,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.155655+0000","last_change":"2024-09-10T19:02:23.723122+0000","last_active":"2024-09-10T19:03:33.155655+0000","last_peered":"2024-09-10T19:03:33.155655+0000","last_clean":"2024-09-10T19:03:33.155655+0000","last_became_active":"2024-09-10T18:50:08.398345+0000","last_became_peered":"2024-09-10T18:50:08.398345+0000","last_unstale":"2024-09-10T19:03:33.155655+0000","last_undegraded":"2024-09-10T19:03:33.155655+0000","last_fullsized":"2024-09-10T19:03:33.155655+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:23.723036+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:55:37.921519+0000","last_clean_scrub_stamp":"2024-09-10T19:02:23.723036+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0054938950000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"232'34","reported_seq":415,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.156312+0000","last_change":"2024-09-10T19:00:41.482710+0000","last_active":"2024-09-10T19:03:33.156312+0000","last_peered":"2024-09-10T19:03:33.156312+0000","last_clean":"2024-09-10T19:03:33.156312+0000","last_became_active":"2024-09-10T18:46:56.742946+0000","last_became_peered":"2024-09-10T18:46:56.742946+0000","last_unstale":"2024-09-10T19:03:33.156312+0000","last_undegraded":"2024-09-10T19:03:33.156312+0000","last_fullsized":"2024-09-10T19:03:33.156312+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'34","last_scrub_stamp":"2024-09-10T19:00:41.482619+0000","last_deep_scrub":"141'22","last_deep_scrub_stamp":"2024-09-10T18:50:57.119152+0000","last_clean_scrub_stamp":"2024-09-10T19:00:41.482619+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0069363309999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":302,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.166045+0000","last_change":"2024-09-10T19:02:49.563958+0000","last_active":"2024-09-10T19:03:33.166045+0000","last_peered":"2024-09-10T19:03:33.166045+0000","last_clean":"2024-09-10T19:03:33.166045+0000","last_became_active":"2024-09-10T19:02:49.563535+0000","last_became_peered":"2024-09-10T19:02:49.563535+0000","last_unstale":"2024-09-10T19:03:33.166045+0000","last_undegraded":"2024-09-10T19:03:33.166045+0000","last_fullsized":"2024-09-10T19:03:33.166045+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:51.286131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_clean_scrub_stamp":"2024-09-10T18:59:51.286131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058886299999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"242'10","reported_seq":638,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.769105+0000","last_change":"2024-09-10T19:02:53.030726+0000","last_active":"2024-09-10T19:03:33.769105+0000","last_peered":"2024-09-10T19:03:33.769105+0000","last_clean":"2024-09-10T19:03:33.769105+0000","last_became_active":"2024-09-10T18:55:40.397246+0000","last_became_peered":"2024-09-10T18:55:40.397246+0000","last_unstale":"2024-09-10T19:03:33.769105+0000","last_undegraded":"2024-09-10T19:03:33.769105+0000","last_fullsized":"2024-09-10T19:03:33.769105+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"242'10","last_scrub_stamp":"2024-09-10T19:02:53.030666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:02:53.030666+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:03.603089+0000","scrub_duration":0.0015331489999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":418,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.155976+0000","last_change":"2024-09-10T19:02:48.670915+0000","last_active":"2024-09-10T19:03:33.155976+0000","last_peered":"2024-09-10T19:03:33.155976+0000","last_clean":"2024-09-10T19:03:33.155976+0000","last_became_active":"2024-09-10T18:55:40.392003+0000","last_became_peered":"2024-09-10T18:55:40.392003+0000","last_unstale":"2024-09-10T19:03:33.155976+0000","last_undegraded":"2024-09-10T19:03:33.155976+0000","last_fullsized":"2024-09-10T19:03:33.155976+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:48.670862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:54.218602+0000","last_clean_scrub_stamp":"2024-09-10T19:02:48.670862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:54.240408+0000","scrub_duration":0.00108365,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1","version":"386'1431","reported_seq":1446,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-09-10T19:03:36.542564+0000","last_change":"2024-09-10T19:03:15.827713+0000","last_active":"2024-09-10T19:03:36.542564+0000","last_peered":"2024-09-10T19:03:36.542564+0000","last_clean":"2024-09-10T19:03:36.542564+0000","last_became_active":"2024-09-10T19:03:15.827177+0000","last_became_peered":"2024-09-10T19:03:15.827177+0000","last_unstale":"2024-09-10T19:03:36.542564+0000","last_undegraded":"2024-09-10T19:03:36.542564+0000","last_fullsized":"2024-09-10T19:03:36.542564+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":1431,"log_dups_size":0,"ondisk_log_size":1431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:32.752337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93782016,"num_objects":1431,"num_object_clones":0,"num_object_copies":4293,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1431,"num_write_kb":91584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.b","version":"70'81","reported_seq":96,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.162084+0000","last_change":"2024-09-10T19:02:49.572998+0000","last_active":"2024-09-10T19:03:33.162084+0000","last_peered":"2024-09-10T19:03:33.162084+0000","last_clean":"2024-09-10T19:03:33.162084+0000","last_became_active":"2024-09-10T19:02:49.572057+0000","last_became_peered":"2024-09-10T19:02:49.572057+0000","last_unstale":"2024-09-10T19:03:33.162084+0000","last_undegraded":"2024-09-10T19:03:33.162084+0000","last_fullsized":"2024-09-10T19:03:33.162084+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:17.370627+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T19:02:17.370627+0000","last_clean_scrub_stamp":"2024-09-10T19:02:17.370627+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007621203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"232'24","reported_seq":495,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.154543+0000","last_change":"2024-09-10T18:59:06.595605+0000","last_active":"2024-09-10T19:03:33.154543+0000","last_peered":"2024-09-10T19:03:33.154543+0000","last_clean":"2024-09-10T19:03:33.154543+0000","last_became_active":"2024-09-10T18:47:57.106935+0000","last_became_peered":"2024-09-10T18:47:57.106935+0000","last_unstale":"2024-09-10T19:03:33.154543+0000","last_undegraded":"2024-09-10T19:03:33.154543+0000","last_fullsized":"2024-09-10T19:03:33.154543+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T18:59:06.595554+0000","last_deep_scrub":"224'16","last_deep_scrub_stamp":"2024-09-10T18:54:49.296957+0000","last_clean_scrub_stamp":"2024-09-10T18:59:06.595554+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020642540000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":391,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.165097+0000","last_change":"2024-09-10T19:02:13.720500+0000","last_active":"2024-09-10T19:03:33.165097+0000","last_peered":"2024-09-10T19:03:33.165097+0000","last_clean":"2024-09-10T19:03:33.165097+0000","last_became_active":"2024-09-10T18:57:02.443816+0000","last_became_peered":"2024-09-10T18:57:02.443816+0000","last_unstale":"2024-09-10T19:03:33.165097+0000","last_undegraded":"2024-09-10T19:03:33.165097+0000","last_fullsized":"2024-09-10T19:03:33.165097+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:13.720416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:13.720416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:36.657529+0000","scrub_duration":0.014226275,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":406,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.156334+0000","last_change":"2024-09-10T19:02:07.623139+0000","last_active":"2024-09-10T19:03:33.156334+0000","last_peered":"2024-09-10T19:03:33.156334+0000","last_clean":"2024-09-10T19:03:33.156334+0000","last_became_active":"2024-09-10T18:45:22.068918+0000","last_became_peered":"2024-09-10T18:45:22.068918+0000","last_unstale":"2024-09-10T19:03:33.156334+0000","last_undegraded":"2024-09-10T19:03:33.156334+0000","last_fullsized":"2024-09-10T19:03:33.156334+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:07.623054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_clean_scrub_stamp":"2024-09-10T19:02:07.623054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003868888,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":389,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.166839+0000","last_change":"2024-09-10T19:03:24.593809+0000","last_active":"2024-09-10T19:03:33.166839+0000","last_peered":"2024-09-10T19:03:33.166839+0000","last_clean":"2024-09-10T19:03:33.166839+0000","last_became_active":"2024-09-10T18:45:24.264538+0000","last_became_peered":"2024-09-10T18:45:24.264538+0000","last_unstale":"2024-09-10T19:03:33.166839+0000","last_undegraded":"2024-09-10T19:03:33.166839+0000","last_fullsized":"2024-09-10T19:03:33.166839+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:24.593754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:51.398981+0000","last_clean_scrub_stamp":"2024-09-10T19:03:24.593754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:27.524766+0000","scrub_duration":0.001434999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.e","version":"386'1499","reported_seq":1514,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-09-10T19:03:36.440097+0000","last_change":"2024-09-10T19:03:16.044645+0000","last_active":"2024-09-10T19:03:36.440097+0000","last_peered":"2024-09-10T19:03:36.440097+0000","last_clean":"2024-09-10T19:03:36.440097+0000","last_became_active":"2024-09-10T19:03:16.044187+0000","last_became_peered":"2024-09-10T19:03:16.044187+0000","last_unstale":"2024-09-10T19:03:36.440097+0000","last_undegraded":"2024-09-10T19:03:36.440097+0000","last_fullsized":"2024-09-10T19:03:36.440097+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":1499,"log_dups_size":0,"ondisk_log_size":1499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:20.361561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":98238464,"num_objects":1499,"num_object_clones":0,"num_object_copies":4497,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1499,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1499,"num_write_kb":95936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"70'81","reported_seq":347,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.166983+0000","last_change":"2024-09-10T19:03:30.624986+0000","last_active":"2024-09-10T19:03:33.166983+0000","last_peered":"2024-09-10T19:03:33.166983+0000","last_clean":"2024-09-10T19:03:33.166983+0000","last_became_active":"2024-09-10T19:00:42.519742+0000","last_became_peered":"2024-09-10T19:00:42.519742+0000","last_unstale":"2024-09-10T19:03:33.166983+0000","last_undegraded":"2024-09-10T19:03:33.166983+0000","last_fullsized":"2024-09-10T19:03:33.166983+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:03:30.624868+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:03:30.624868+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:41.117180+0000","scrub_duration":0.0036172140000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"232'35","reported_seq":442,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.153377+0000","last_change":"2024-09-10T19:03:29.581426+0000","last_active":"2024-09-10T19:03:33.153377+0000","last_peered":"2024-09-10T19:03:33.153377+0000","last_clean":"2024-09-10T19:03:33.153377+0000","last_became_active":"2024-09-10T18:45:20.052243+0000","last_became_peered":"2024-09-10T18:45:20.052243+0000","last_unstale":"2024-09-10T19:03:33.153377+0000","last_undegraded":"2024-09-10T19:03:33.153377+0000","last_fullsized":"2024-09-10T19:03:33.153377+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'35","last_scrub_stamp":"2024-09-10T19:03:29.581339+0000","last_deep_scrub":"143'21","last_deep_scrub_stamp":"2024-09-10T18:51:02.540216+0000","last_clean_scrub_stamp":"2024-09-10T19:03:29.581339+0000","objects_scrubbed":7,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:36.823000+0000","scrub_duration":0.0042812099999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":428,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.158449+0000","last_change":"2024-09-10T19:01:55.150294+0000","last_active":"2024-09-10T19:03:33.158449+0000","last_peered":"2024-09-10T19:03:33.158449+0000","last_clean":"2024-09-10T19:03:33.158449+0000","last_became_active":"2024-09-10T18:57:02.453404+0000","last_became_peered":"2024-09-10T18:57:02.453404+0000","last_unstale":"2024-09-10T19:03:33.158449+0000","last_undegraded":"2024-09-10T19:03:33.158449+0000","last_fullsized":"2024-09-10T19:03:33.158449+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:01:55.150215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_clean_scrub_stamp":"2024-09-10T19:01:55.150215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060613990000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":618,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.166670+0000","last_change":"2024-09-10T19:01:56.290088+0000","last_active":"2024-09-10T19:03:33.166670+0000","last_peered":"2024-09-10T19:03:33.166670+0000","last_clean":"2024-09-10T19:03:33.166670+0000","last_became_active":"2024-09-10T18:45:22.074540+0000","last_became_peered":"2024-09-10T18:45:22.074540+0000","last_unstale":"2024-09-10T19:03:33.166670+0000","last_undegraded":"2024-09-10T19:03:33.166670+0000","last_fullsized":"2024-09-10T19:03:33.166670+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T19:01:56.289997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:01:56.289997+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040368549999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":195,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.158163+0000","last_change":"2024-09-10T19:02:10.106717+0000","last_active":"2024-09-10T19:03:33.158163+0000","last_peered":"2024-09-10T19:03:33.158163+0000","last_clean":"2024-09-10T19:03:33.158163+0000","last_became_active":"2024-09-10T18:57:02.437100+0000","last_became_peered":"2024-09-10T18:57:02.437100+0000","last_unstale":"2024-09-10T19:03:33.158163+0000","last_undegraded":"2024-09-10T19:03:33.158163+0000","last_fullsized":"2024-09-10T19:03:33.158163+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:10.106577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:02:10.106577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0035912439999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"386'1124","reported_seq":1139,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.813889+0000","last_change":"2024-09-10T19:03:16.046832+0000","last_active":"2024-09-10T19:03:33.813889+0000","last_peered":"2024-09-10T19:03:33.813889+0000","last_clean":"2024-09-10T19:03:33.813889+0000","last_became_active":"2024-09-10T19:03:16.046538+0000","last_became_peered":"2024-09-10T19:03:16.046538+0000","last_unstale":"2024-09-10T19:03:33.813889+0000","last_undegraded":"2024-09-10T19:03:33.813889+0000","last_fullsized":"2024-09-10T19:03:33.813889+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:43.692877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73662464,"num_objects":1124,"num_object_clones":0,"num_object_copies":3372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1124,"num_write_kb":71936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"70'81","reported_seq":313,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.154204+0000","last_change":"2024-09-10T19:02:58.888049+0000","last_active":"2024-09-10T19:03:33.154204+0000","last_peered":"2024-09-10T19:03:33.154204+0000","last_clean":"2024-09-10T19:03:33.154204+0000","last_became_active":"2024-09-10T18:52:14.584798+0000","last_became_peered":"2024-09-10T18:52:14.584798+0000","last_unstale":"2024-09-10T19:03:33.154204+0000","last_undegraded":"2024-09-10T19:03:33.154204+0000","last_fullsized":"2024-09-10T19:03:33.154204+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:58.887944+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:56:31.481303+0000","last_clean_scrub_stamp":"2024-09-10T19:02:58.887944+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:10.772150+0000","scrub_duration":0.0016190950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"232'30","reported_seq":559,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.167724+0000","last_change":"2024-09-10T19:02:48.361438+0000","last_active":"2024-09-10T19:03:33.167724+0000","last_peered":"2024-09-10T19:03:33.167724+0000","last_clean":"2024-09-10T19:03:33.167724+0000","last_became_active":"2024-09-10T18:59:48.742377+0000","last_became_peered":"2024-09-10T18:59:48.742377+0000","last_unstale":"2024-09-10T19:03:33.167724+0000","last_undegraded":"2024-09-10T19:03:33.167724+0000","last_fullsized":"2024-09-10T19:03:33.167724+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'30","last_scrub_stamp":"2024-09-10T19:02:48.361337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:48.361337+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:04.538552+0000","scrub_duration":0.011213163,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":36,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.158739+0000","last_change":"2024-09-10T19:02:52.036142+0000","last_active":"2024-09-10T19:03:33.158739+0000","last_peered":"2024-09-10T19:03:33.158739+0000","last_clean":"2024-09-10T19:03:33.158739+0000","last_became_active":"2024-09-10T19:02:49.578870+0000","last_became_peered":"2024-09-10T19:02:49.578870+0000","last_unstale":"2024-09-10T19:03:33.158739+0000","last_undegraded":"2024-09-10T19:03:33.158739+0000","last_fullsized":"2024-09-10T19:03:33.158739+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:52.036077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:46.142129+0000","last_clean_scrub_stamp":"2024-09-10T19:02:52.036077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:07.232263+0000","scrub_duration":0.0015638970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":401,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.153247+0000","last_change":"2024-09-10T18:59:57.903394+0000","last_active":"2024-09-10T19:03:33.153247+0000","last_peered":"2024-09-10T19:03:33.153247+0000","last_clean":"2024-09-10T19:03:33.153247+0000","last_became_active":"2024-09-10T18:57:02.448126+0000","last_became_peered":"2024-09-10T18:57:02.448126+0000","last_unstale":"2024-09-10T19:03:33.153247+0000","last_undegraded":"2024-09-10T19:03:33.153247+0000","last_fullsized":"2024-09-10T19:03:33.153247+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:57.903311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:57.903311+0000","last_clean_scrub_stamp":"2024-09-10T18:59:57.903311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011228738,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":389,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.153809+0000","last_change":"2024-09-10T18:58:48.633501+0000","last_active":"2024-09-10T19:03:33.153809+0000","last_peered":"2024-09-10T19:03:33.153809+0000","last_clean":"2024-09-10T19:03:33.153809+0000","last_became_active":"2024-09-10T18:47:36.410836+0000","last_became_peered":"2024-09-10T18:47:36.410836+0000","last_unstale":"2024-09-10T19:03:33.153809+0000","last_undegraded":"2024-09-10T19:03:33.153809+0000","last_fullsized":"2024-09-10T19:03:33.153809+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:48.633423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_clean_scrub_stamp":"2024-09-10T18:58:48.633423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0079262829999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"386'1434","reported_seq":1448,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-09-10T19:03:36.432006+0000","last_change":"2024-09-10T19:03:16.046878+0000","last_active":"2024-09-10T19:03:36.432006+0000","last_peered":"2024-09-10T19:03:36.432006+0000","last_clean":"2024-09-10T19:03:36.432006+0000","last_became_active":"2024-09-10T19:03:16.046484+0000","last_became_peered":"2024-09-10T19:03:16.046484+0000","last_unstale":"2024-09-10T19:03:36.432006+0000","last_undegraded":"2024-09-10T19:03:36.432006+0000","last_fullsized":"2024-09-10T19:03:36.432006+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":1434,"log_dups_size":0,"ondisk_log_size":1434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:43.162682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93978624,"num_objects":1434,"num_object_clones":0,"num_object_copies":4302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1434,"num_write_kb":91776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"70'81","reported_seq":411,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.158696+0000","last_change":"2024-09-10T19:02:21.983253+0000","last_active":"2024-09-10T19:03:33.158696+0000","last_peered":"2024-09-10T19:03:33.158696+0000","last_clean":"2024-09-10T19:03:33.158696+0000","last_became_active":"2024-09-10T18:55:40.396828+0000","last_became_peered":"2024-09-10T18:55:40.396828+0000","last_unstale":"2024-09-10T19:03:33.158696+0000","last_undegraded":"2024-09-10T19:03:33.158696+0000","last_fullsized":"2024-09-10T19:03:33.158696+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:21.983118+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:02:21.983118+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:37.401129+0000","scrub_duration":0.0056724799999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"232'29","reported_seq":453,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.166209+0000","last_change":"2024-09-10T19:02:53.679893+0000","last_active":"2024-09-10T19:03:33.166209+0000","last_peered":"2024-09-10T19:03:33.166209+0000","last_clean":"2024-09-10T19:03:33.166209+0000","last_became_active":"2024-09-10T18:57:02.442862+0000","last_became_peered":"2024-09-10T18:57:02.442862+0000","last_unstale":"2024-09-10T19:03:33.166209+0000","last_undegraded":"2024-09-10T19:03:33.166209+0000","last_fullsized":"2024-09-10T19:03:33.166209+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'29","last_scrub_stamp":"2024-09-10T19:02:53.679834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:53.679834+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:56.717750+0000","scrub_duration":0.0024571129999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":391,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.165255+0000","last_change":"2024-09-10T19:03:28.771956+0000","last_active":"2024-09-10T19:03:33.165255+0000","last_peered":"2024-09-10T19:03:33.165255+0000","last_clean":"2024-09-10T19:03:33.165255+0000","last_became_active":"2024-09-10T18:57:02.443522+0000","last_became_peered":"2024-09-10T18:57:02.443522+0000","last_unstale":"2024-09-10T19:03:33.165255+0000","last_undegraded":"2024-09-10T19:03:33.165255+0000","last_fullsized":"2024-09-10T19:03:33.165255+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:28.771864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:48.196065+0000","last_clean_scrub_stamp":"2024-09-10T19:03:28.771864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:33.555178+0000","scrub_duration":0.0070868659999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":421,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.158025+0000","last_change":"2024-09-10T19:02:17.064956+0000","last_active":"2024-09-10T19:03:33.158025+0000","last_peered":"2024-09-10T19:03:33.158025+0000","last_clean":"2024-09-10T19:03:33.158025+0000","last_became_active":"2024-09-10T18:55:40.387857+0000","last_became_peered":"2024-09-10T18:55:40.387857+0000","last_unstale":"2024-09-10T19:03:33.158025+0000","last_undegraded":"2024-09-10T19:03:33.158025+0000","last_fullsized":"2024-09-10T19:03:33.158025+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:17.064877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:02:17.064877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0089490170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":304,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.158110+0000","last_change":"2024-09-10T19:02:54.957936+0000","last_active":"2024-09-10T19:03:33.158110+0000","last_peered":"2024-09-10T19:03:33.158110+0000","last_clean":"2024-09-10T19:03:33.158110+0000","last_became_active":"2024-09-10T19:02:49.561359+0000","last_became_peered":"2024-09-10T19:02:49.561359+0000","last_unstale":"2024-09-10T19:03:33.158110+0000","last_undegraded":"2024-09-10T19:03:33.158110+0000","last_fullsized":"2024-09-10T19:03:33.158110+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:54.957846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:49:18.281230+0000","last_clean_scrub_stamp":"2024-09-10T19:02:54.957846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:10.717373+0000","scrub_duration":0.0015003740000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"386'1486","reported_seq":1501,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-09-10T19:03:36.436774+0000","last_change":"2024-09-10T19:03:16.044601+0000","last_active":"2024-09-10T19:03:36.436774+0000","last_peered":"2024-09-10T19:03:36.436774+0000","last_clean":"2024-09-10T19:03:36.436774+0000","last_became_active":"2024-09-10T19:03:16.044213+0000","last_became_peered":"2024-09-10T19:03:16.044213+0000","last_unstale":"2024-09-10T19:03:36.436774+0000","last_undegraded":"2024-09-10T19:03:36.436774+0000","last_fullsized":"2024-09-10T19:03:36.436774+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":1486,"log_dups_size":0,"ondisk_log_size":1486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:20.548149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":97386496,"num_objects":1486,"num_object_clones":0,"num_object_copies":4458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1486,"num_write_kb":95104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"70'81","reported_seq":414,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-09-10T19:03:34.083537+0000","last_change":"2024-09-10T19:03:34.083497+0000","last_active":"2024-09-10T19:03:34.083537+0000","last_peered":"2024-09-10T19:03:34.083537+0000","last_clean":"2024-09-10T19:03:34.083537+0000","last_became_active":"2024-09-10T18:57:02.452659+0000","last_became_peered":"2024-09-10T18:57:02.452659+0000","last_unstale":"2024-09-10T19:03:34.083537+0000","last_undegraded":"2024-09-10T19:03:34.083537+0000","last_fullsized":"2024-09-10T19:03:34.083537+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:03:34.083013+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:03:34.083013+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:35.239103+0000","scrub_duration":0.0023839249999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"232'25","reported_seq":462,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.155089+0000","last_change":"2024-09-10T19:02:49.580031+0000","last_active":"2024-09-10T19:03:33.155089+0000","last_peered":"2024-09-10T19:03:33.155089+0000","last_clean":"2024-09-10T19:03:33.155089+0000","last_became_active":"2024-09-10T19:02:49.579401+0000","last_became_peered":"2024-09-10T19:02:49.579401+0000","last_unstale":"2024-09-10T19:03:33.155089+0000","last_undegraded":"2024-09-10T19:03:33.155089+0000","last_fullsized":"2024-09-10T19:03:33.155089+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'25","last_scrub_stamp":"2024-09-10T19:00:07.390135+0000","last_deep_scrub":"83'15","last_deep_scrub_stamp":"2024-09-10T18:46:58.436473+0000","last_clean_scrub_stamp":"2024-09-10T19:00:07.390135+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002339874,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":317,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.165442+0000","last_change":"2024-09-10T19:02:49.561636+0000","last_active":"2024-09-10T19:03:33.165442+0000","last_peered":"2024-09-10T19:03:33.165442+0000","last_clean":"2024-09-10T19:03:33.165442+0000","last_became_active":"2024-09-10T19:02:49.561340+0000","last_became_peered":"2024-09-10T19:02:49.561340+0000","last_unstale":"2024-09-10T19:03:33.165442+0000","last_undegraded":"2024-09-10T19:03:33.165442+0000","last_fullsized":"2024-09-10T19:03:33.165442+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:21.266828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:05.140027+0000","last_clean_scrub_stamp":"2024-09-10T19:02:21.266828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:39.977022+0000","scrub_duration":0.0034952049999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"339'13","reported_seq":590,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.162280+0000","last_change":"2024-09-10T19:03:29.753145+0000","last_active":"2024-09-10T19:03:33.162280+0000","last_peered":"2024-09-10T19:03:33.162280+0000","last_clean":"2024-09-10T19:03:33.162280+0000","last_became_active":"2024-09-10T19:00:42.520360+0000","last_became_peered":"2024-09-10T19:00:42.520360+0000","last_unstale":"2024-09-10T19:03:33.162280+0000","last_undegraded":"2024-09-10T19:03:33.162280+0000","last_fullsized":"2024-09-10T19:03:33.162280+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"339'13","last_scrub_stamp":"2024-09-10T19:03:29.753059+0000","last_deep_scrub":"258'12","last_deep_scrub_stamp":"2024-09-10T18:56:21.098770+0000","last_clean_scrub_stamp":"2024-09-10T19:03:29.753059+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:48.781650+0000","scrub_duration":0.002665485,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":377,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.162185+0000","last_change":"2024-09-10T19:02:14.738026+0000","last_active":"2024-09-10T19:03:33.162185+0000","last_peered":"2024-09-10T19:03:33.162185+0000","last_clean":"2024-09-10T19:03:33.162185+0000","last_became_active":"2024-09-10T19:00:42.515144+0000","last_became_peered":"2024-09-10T19:00:42.515144+0000","last_unstale":"2024-09-10T19:03:33.162185+0000","last_undegraded":"2024-09-10T19:03:33.162185+0000","last_fullsized":"2024-09-10T19:03:33.162185+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:14.737936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:02:14.737936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003039609,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"386'1423","reported_seq":1436,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-09-10T19:03:36.087824+0000","last_change":"2024-09-10T19:03:16.048706+0000","last_active":"2024-09-10T19:03:36.087824+0000","last_peered":"2024-09-10T19:03:36.087824+0000","last_clean":"2024-09-10T19:03:36.087824+0000","last_became_active":"2024-09-10T19:03:16.048344+0000","last_became_peered":"2024-09-10T19:03:16.048344+0000","last_unstale":"2024-09-10T19:03:36.087824+0000","last_undegraded":"2024-09-10T19:03:36.087824+0000","last_fullsized":"2024-09-10T19:03:36.087824+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":1423,"log_dups_size":0,"ondisk_log_size":1423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:15.964765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93257728,"num_objects":1423,"num_object_clones":0,"num_object_copies":4269,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1423,"num_write_kb":91072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"70'81","reported_seq":335,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.155034+0000","last_change":"2024-09-10T19:00:40.472680+0000","last_active":"2024-09-10T19:03:33.155034+0000","last_peered":"2024-09-10T19:03:33.155034+0000","last_clean":"2024-09-10T19:03:33.155034+0000","last_became_active":"2024-09-10T18:55:40.392432+0000","last_became_peered":"2024-09-10T18:55:40.392432+0000","last_unstale":"2024-09-10T19:03:33.155034+0000","last_undegraded":"2024-09-10T19:03:33.155034+0000","last_fullsized":"2024-09-10T19:03:33.155034+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:00:40.472595+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:00:40.472595+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0072897099999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"373'42","reported_seq":473,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.166458+0000","last_change":"2024-09-10T19:02:49.563856+0000","last_active":"2024-09-10T19:03:33.166458+0000","last_peered":"2024-09-10T19:03:33.166458+0000","last_clean":"2024-09-10T19:03:33.166458+0000","last_became_active":"2024-09-10T19:02:49.563402+0000","last_became_peered":"2024-09-10T19:02:49.563402+0000","last_unstale":"2024-09-10T19:03:33.166458+0000","last_undegraded":"2024-09-10T19:03:33.166458+0000","last_fullsized":"2024-09-10T19:03:33.166458+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'41","last_scrub_stamp":"2024-09-10T19:02:12.270610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:12.270610+0000","objects_scrubbed":10,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012827955,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":410,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.155152+0000","last_change":"2024-09-10T19:02:13.666158+0000","last_active":"2024-09-10T19:03:33.155152+0000","last_peered":"2024-09-10T19:03:33.155152+0000","last_clean":"2024-09-10T19:03:33.155152+0000","last_became_active":"2024-09-10T18:45:17.964494+0000","last_became_peered":"2024-09-10T18:45:17.964494+0000","last_unstale":"2024-09-10T19:03:33.155152+0000","last_undegraded":"2024-09-10T19:03:33.155152+0000","last_fullsized":"2024-09-10T19:03:33.155152+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:13.665999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:12.982599+0000","last_clean_scrub_stamp":"2024-09-10T19:02:13.665999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007648719,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":352,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.158650+0000","last_change":"2024-09-10T19:02:01.110434+0000","last_active":"2024-09-10T19:03:33.158650+0000","last_peered":"2024-09-10T19:03:33.158650+0000","last_clean":"2024-09-10T19:03:33.158650+0000","last_became_active":"2024-09-10T18:47:57.107478+0000","last_became_peered":"2024-09-10T18:47:57.107478+0000","last_unstale":"2024-09-10T19:03:33.158650+0000","last_undegraded":"2024-09-10T19:03:33.158650+0000","last_fullsized":"2024-09-10T19:03:33.158650+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:01.110335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:02:01.110335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0062803349999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":389,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.167357+0000","last_change":"2024-09-10T19:02:09.267386+0000","last_active":"2024-09-10T19:03:33.167357+0000","last_peered":"2024-09-10T19:03:33.167357+0000","last_clean":"2024-09-10T19:03:33.167357+0000","last_became_active":"2024-09-10T18:57:02.460594+0000","last_became_peered":"2024-09-10T18:57:02.460594+0000","last_unstale":"2024-09-10T19:03:33.167357+0000","last_undegraded":"2024-09-10T19:03:33.167357+0000","last_fullsized":"2024-09-10T19:03:33.167357+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:09.267253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:22.891084+0000","last_clean_scrub_stamp":"2024-09-10T19:02:09.267253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0067979290000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"386'1392","reported_seq":1406,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-09-10T19:03:36.091670+0000","last_change":"2024-09-10T19:03:15.825120+0000","last_active":"2024-09-10T19:03:36.091670+0000","last_peered":"2024-09-10T19:03:36.091670+0000","last_clean":"2024-09-10T19:03:36.091670+0000","last_became_active":"2024-09-10T19:03:15.824855+0000","last_became_peered":"2024-09-10T19:03:15.824855+0000","last_unstale":"2024-09-10T19:03:36.091670+0000","last_undegraded":"2024-09-10T19:03:36.091670+0000","last_fullsized":"2024-09-10T19:03:36.091670+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":1392,"log_dups_size":0,"ondisk_log_size":1392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:30.722205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":91226112,"num_objects":1392,"num_object_clones":0,"num_object_copies":4176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1392,"num_write_kb":89088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"70'81","reported_seq":213,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.162184+0000","last_change":"2024-09-10T19:02:56.723503+0000","last_active":"2024-09-10T19:03:33.162184+0000","last_peered":"2024-09-10T19:03:33.162184+0000","last_clean":"2024-09-10T19:03:33.162184+0000","last_became_active":"2024-09-10T18:57:02.437944+0000","last_became_peered":"2024-09-10T18:57:02.437944+0000","last_unstale":"2024-09-10T19:03:33.162184+0000","last_undegraded":"2024-09-10T19:03:33.162184+0000","last_fullsized":"2024-09-10T19:03:33.162184+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:56.723447+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:02:56.723447+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:12.529013+0000","scrub_duration":0.001350382,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"232'22","reported_seq":281,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.156357+0000","last_change":"2024-09-10T19:02:49.573080+0000","last_active":"2024-09-10T19:03:33.156357+0000","last_peered":"2024-09-10T19:03:33.156357+0000","last_clean":"2024-09-10T19:03:33.156357+0000","last_became_active":"2024-09-10T19:02:49.572746+0000","last_became_peered":"2024-09-10T19:02:49.572746+0000","last_unstale":"2024-09-10T19:03:33.156357+0000","last_undegraded":"2024-09-10T19:03:33.156357+0000","last_fullsized":"2024-09-10T19:03:33.156357+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'22","last_scrub_stamp":"2024-09-10T19:00:08.468154+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-10T18:47:10.514951+0000","last_clean_scrub_stamp":"2024-09-10T19:00:08.468154+0000","objects_scrubbed":5,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020902970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"84'6","reported_seq":239,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-09-10T19:03:34.653937+0000","last_change":"2024-09-10T19:02:49.570833+0000","last_active":"2024-09-10T19:03:34.653937+0000","last_peered":"2024-09-10T19:03:34.653937+0000","last_clean":"2024-09-10T19:03:34.653937+0000","last_became_active":"2024-09-10T19:02:49.570177+0000","last_became_peered":"2024-09-10T19:02:49.570177+0000","last_unstale":"2024-09-10T19:03:34.653937+0000","last_undegraded":"2024-09-10T19:03:34.653937+0000","last_fullsized":"2024-09-10T19:03:34.653937+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'6","last_scrub_stamp":"2024-09-10T19:00:40.504174+0000","last_deep_scrub":"84'6","last_deep_scrub_stamp":"2024-09-10T18:47:22.545450+0000","last_clean_scrub_stamp":"2024-09-10T19:00:40.504174+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021933159999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":159,"num_read_kb":154,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":363,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.165746+0000","last_change":"2024-09-10T19:00:08.141664+0000","last_active":"2024-09-10T19:03:33.165746+0000","last_peered":"2024-09-10T19:03:33.165746+0000","last_clean":"2024-09-10T19:03:33.165746+0000","last_became_active":"2024-09-10T18:55:40.393389+0000","last_became_peered":"2024-09-10T18:55:40.393389+0000","last_unstale":"2024-09-10T19:03:33.165746+0000","last_undegraded":"2024-09-10T19:03:33.165746+0000","last_fullsized":"2024-09-10T19:03:33.165746+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:08.141603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:00:08.141603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001270729,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":381,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.156127+0000","last_change":"2024-09-10T19:02:09.600216+0000","last_active":"2024-09-10T19:03:33.156127+0000","last_peered":"2024-09-10T19:03:33.156127+0000","last_clean":"2024-09-10T19:03:33.156127+0000","last_became_active":"2024-09-10T18:57:02.438945+0000","last_became_peered":"2024-09-10T18:57:02.438945+0000","last_unstale":"2024-09-10T19:03:33.156127+0000","last_undegraded":"2024-09-10T19:03:33.156127+0000","last_fullsized":"2024-09-10T19:03:33.156127+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:09.600134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:02:09.600134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003812368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":138,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.158959+0000","last_change":"2024-09-10T19:00:41.794168+0000","last_active":"2024-09-10T19:03:33.158959+0000","last_peered":"2024-09-10T19:03:33.158959+0000","last_clean":"2024-09-10T19:03:33.158959+0000","last_became_active":"2024-09-10T18:57:02.453865+0000","last_became_peered":"2024-09-10T18:57:02.453865+0000","last_unstale":"2024-09-10T19:03:33.158959+0000","last_undegraded":"2024-09-10T19:03:33.158959+0000","last_fullsized":"2024-09-10T19:03:33.158959+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:41.794116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:07.150480+0000","last_clean_scrub_stamp":"2024-09-10T19:00:41.794116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001296148,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.9","version":"386'1396","reported_seq":1410,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-09-10T19:03:36.091417+0000","last_change":"2024-09-10T19:03:15.825703+0000","last_active":"2024-09-10T19:03:36.091417+0000","last_peered":"2024-09-10T19:03:36.091417+0000","last_clean":"2024-09-10T19:03:36.091417+0000","last_became_active":"2024-09-10T19:03:15.825424+0000","last_became_peered":"2024-09-10T19:03:15.825424+0000","last_unstale":"2024-09-10T19:03:36.091417+0000","last_undegraded":"2024-09-10T19:03:36.091417+0000","last_fullsized":"2024-09-10T19:03:36.091417+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":1396,"log_dups_size":0,"ondisk_log_size":1396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:35.248919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":91488256,"num_objects":1396,"num_object_clones":0,"num_object_copies":4188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1396,"num_write_kb":89344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"70'81","reported_seq":91,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.158189+0000","last_change":"2024-09-10T19:02:49.579385+0000","last_active":"2024-09-10T19:03:33.158189+0000","last_peered":"2024-09-10T19:03:33.158189+0000","last_clean":"2024-09-10T19:03:33.158189+0000","last_became_active":"2024-09-10T19:02:49.578413+0000","last_became_peered":"2024-09-10T19:02:49.578413+0000","last_unstale":"2024-09-10T19:03:33.158189+0000","last_undegraded":"2024-09-10T19:03:33.158189+0000","last_fullsized":"2024-09-10T19:03:33.158189+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:46.421787+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:02:46.421787+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019876809999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17890,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6,"num_write":3,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":35780,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"232'24","reported_seq":123,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.165242+0000","last_change":"2024-09-10T19:00:42.532827+0000","last_active":"2024-09-10T19:03:33.165242+0000","last_peered":"2024-09-10T19:03:33.165242+0000","last_clean":"2024-09-10T19:03:33.165242+0000","last_became_active":"2024-09-10T19:00:42.516521+0000","last_became_peered":"2024-09-10T19:00:42.516521+0000","last_unstale":"2024-09-10T19:03:33.165242+0000","last_undegraded":"2024-09-10T19:03:33.165242+0000","last_fullsized":"2024-09-10T19:03:33.165242+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T19:00:39.748509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:00:39.748509+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027783730000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"305'2","reported_seq":380,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.166302+0000","last_change":"2024-09-10T19:03:21.921314+0000","last_active":"2024-09-10T19:03:33.166302+0000","last_peered":"2024-09-10T19:03:33.166302+0000","last_clean":"2024-09-10T19:03:33.166302+0000","last_became_active":"2024-09-10T18:55:40.390022+0000","last_became_peered":"2024-09-10T18:55:40.390022+0000","last_unstale":"2024-09-10T19:03:33.166302+0000","last_undegraded":"2024-09-10T19:03:33.166302+0000","last_fullsized":"2024-09-10T19:03:33.166302+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"305'2","last_scrub_stamp":"2024-09-10T19:03:21.921254+0000","last_deep_scrub":"305'2","last_deep_scrub_stamp":"2024-09-10T18:59:18.897059+0000","last_clean_scrub_stamp":"2024-09-10T19:03:21.921254+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:40.677878+0000","scrub_duration":0.0021302700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":231,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.161901+0000","last_change":"2024-09-10T19:02:49.560608+0000","last_active":"2024-09-10T19:03:33.161901+0000","last_peered":"2024-09-10T19:03:33.161901+0000","last_clean":"2024-09-10T19:03:33.161901+0000","last_became_active":"2024-09-10T19:02:49.560325+0000","last_became_peered":"2024-09-10T19:02:49.560325+0000","last_unstale":"2024-09-10T19:03:33.161901+0000","last_undegraded":"2024-09-10T19:03:33.161901+0000","last_fullsized":"2024-09-10T19:03:33.161901+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:06.272690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:06.272690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001771613,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":57,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.156187+0000","last_change":"2024-09-10T19:00:42.518887+0000","last_active":"2024-09-10T19:03:33.156187+0000","last_peered":"2024-09-10T19:03:33.156187+0000","last_clean":"2024-09-10T19:03:33.156187+0000","last_became_active":"2024-09-10T19:00:42.518566+0000","last_became_peered":"2024-09-10T19:00:42.518566+0000","last_unstale":"2024-09-10T19:03:33.156187+0000","last_undegraded":"2024-09-10T19:03:33.156187+0000","last_fullsized":"2024-09-10T19:03:33.156187+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:39.752722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:00:39.752722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":26,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.155904+0000","last_change":"2024-09-10T19:02:49.573172+0000","last_active":"2024-09-10T19:03:33.155904+0000","last_peered":"2024-09-10T19:03:33.155904+0000","last_clean":"2024-09-10T19:03:33.155904+0000","last_became_active":"2024-09-10T19:02:49.572858+0000","last_became_peered":"2024-09-10T19:02:49.572858+0000","last_unstale":"2024-09-10T19:03:33.155904+0000","last_undegraded":"2024-09-10T19:03:33.155904+0000","last_fullsized":"2024-09-10T19:03:33.155904+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:20.272756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:02:20.272756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.a","version":"386'1450","reported_seq":1464,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-09-10T19:03:36.091860+0000","last_change":"2024-09-10T19:03:15.826075+0000","last_active":"2024-09-10T19:03:36.091860+0000","last_peered":"2024-09-10T19:03:36.091860+0000","last_clean":"2024-09-10T19:03:36.091860+0000","last_became_active":"2024-09-10T19:03:15.825784+0000","last_became_peered":"2024-09-10T19:03:15.825784+0000","last_unstale":"2024-09-10T19:03:36.091860+0000","last_undegraded":"2024-09-10T19:03:36.091860+0000","last_fullsized":"2024-09-10T19:03:36.091860+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":1450,"log_dups_size":0,"ondisk_log_size":1450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:39.268698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95027200,"num_objects":1450,"num_object_clones":0,"num_object_copies":4350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1450,"num_write_kb":92800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"232'33","reported_seq":489,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.159146+0000","last_change":"2024-09-10T19:02:16.059463+0000","last_active":"2024-09-10T19:03:33.159146+0000","last_peered":"2024-09-10T19:03:33.159146+0000","last_clean":"2024-09-10T19:03:33.159146+0000","last_became_active":"2024-09-10T18:57:02.435499+0000","last_became_peered":"2024-09-10T18:57:02.435499+0000","last_unstale":"2024-09-10T19:03:33.159146+0000","last_undegraded":"2024-09-10T19:03:33.159146+0000","last_fullsized":"2024-09-10T19:03:33.159146+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'33","last_scrub_stamp":"2024-09-10T19:02:16.059377+0000","last_deep_scrub":"232'33","last_deep_scrub_stamp":"2024-09-10T18:56:21.206960+0000","last_clean_scrub_stamp":"2024-09-10T19:02:16.059377+0000","objects_scrubbed":7,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0042245340000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"60'2","reported_seq":408,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.167405+0000","last_change":"2024-09-10T19:03:22.545399+0000","last_active":"2024-09-10T19:03:33.167405+0000","last_peered":"2024-09-10T19:03:33.167405+0000","last_clean":"2024-09-10T19:03:33.167405+0000","last_became_active":"2024-09-10T18:45:17.961660+0000","last_became_peered":"2024-09-10T18:45:17.961660+0000","last_unstale":"2024-09-10T19:03:33.167405+0000","last_undegraded":"2024-09-10T19:03:33.167405+0000","last_fullsized":"2024-09-10T19:03:33.167405+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-10T19:03:22.545342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:03:22.545342+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:51.573027+0000","scrub_duration":0.001448983,"objects_trimmed":0,"snaptrim_duration":0.0013942640000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":427,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.155837+0000","last_change":"2024-09-10T19:01:56.581104+0000","last_active":"2024-09-10T19:03:33.155837+0000","last_peered":"2024-09-10T19:03:33.155837+0000","last_clean":"2024-09-10T19:03:33.155837+0000","last_became_active":"2024-09-10T18:57:02.445712+0000","last_became_peered":"2024-09-10T18:57:02.445712+0000","last_unstale":"2024-09-10T19:03:33.155837+0000","last_undegraded":"2024-09-10T19:03:33.155837+0000","last_fullsized":"2024-09-10T19:03:33.155837+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:01:56.581018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:01:56.581018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048884530000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'81","reported_seq":213,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.161803+0000","last_change":"2024-09-10T19:02:59.753995+0000","last_active":"2024-09-10T19:03:33.161803+0000","last_peered":"2024-09-10T19:03:33.161803+0000","last_clean":"2024-09-10T19:03:33.161803+0000","last_became_active":"2024-09-10T18:57:02.440063+0000","last_became_peered":"2024-09-10T18:57:02.440063+0000","last_unstale":"2024-09-10T19:03:33.161803+0000","last_undegraded":"2024-09-10T19:03:33.161803+0000","last_fullsized":"2024-09-10T19:03:33.161803+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:59.753922+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:56.085580+0000","last_clean_scrub_stamp":"2024-09-10T19:02:59.753922+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:28.334036+0000","scrub_duration":0.001320882,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":396,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.156036+0000","last_change":"2024-09-10T19:01:55.620765+0000","last_active":"2024-09-10T19:03:33.156036+0000","last_peered":"2024-09-10T19:03:33.156036+0000","last_clean":"2024-09-10T19:03:33.156036+0000","last_became_active":"2024-09-10T18:57:02.431556+0000","last_became_peered":"2024-09-10T18:57:02.431556+0000","last_unstale":"2024-09-10T19:03:33.156036+0000","last_undegraded":"2024-09-10T19:03:33.156036+0000","last_fullsized":"2024-09-10T19:03:33.156036+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:01:55.620696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:01:55.620696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0068284349999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":401,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.156405+0000","last_change":"2024-09-10T19:00:07.624756+0000","last_active":"2024-09-10T19:03:33.156405+0000","last_peered":"2024-09-10T19:03:33.156405+0000","last_clean":"2024-09-10T19:03:33.156405+0000","last_became_active":"2024-09-10T18:45:24.073327+0000","last_became_peered":"2024-09-10T18:45:24.073327+0000","last_unstale":"2024-09-10T19:03:33.156405+0000","last_undegraded":"2024-09-10T19:03:33.156405+0000","last_fullsized":"2024-09-10T19:03:33.156405+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:07.624701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:00:07.624701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001137962,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.5","version":"386'1464","reported_seq":1479,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-09-10T19:03:36.634515+0000","last_change":"2024-09-10T19:03:15.824509+0000","last_active":"2024-09-10T19:03:36.634515+0000","last_peered":"2024-09-10T19:03:36.634515+0000","last_clean":"2024-09-10T19:03:36.634515+0000","last_became_active":"2024-09-10T19:03:15.823922+0000","last_became_peered":"2024-09-10T19:03:15.823922+0000","last_unstale":"2024-09-10T19:03:36.634515+0000","last_undegraded":"2024-09-10T19:03:36.634515+0000","last_fullsized":"2024-09-10T19:03:36.634515+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":1464,"log_dups_size":0,"ondisk_log_size":1464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:32.977483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95944704,"num_objects":1464,"num_object_clones":0,"num_object_copies":4392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1464,"num_write_kb":93696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.f","version":"70'81","reported_seq":406,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.158633+0000","last_change":"2024-09-10T19:02:13.085183+0000","last_active":"2024-09-10T19:03:33.158633+0000","last_peered":"2024-09-10T19:03:33.158633+0000","last_clean":"2024-09-10T19:03:33.158633+0000","last_became_active":"2024-09-10T18:52:16.598295+0000","last_became_peered":"2024-09-10T18:52:16.598295+0000","last_unstale":"2024-09-10T19:03:33.158633+0000","last_undegraded":"2024-09-10T19:03:33.158633+0000","last_fullsized":"2024-09-10T19:03:33.158633+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:13.085003+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:53.609885+0000","last_clean_scrub_stamp":"2024-09-10T19:02:13.085003+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:36.454967+0000","scrub_duration":0.00305172,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"232'35","reported_seq":494,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.158143+0000","last_change":"2024-09-10T19:00:43.819111+0000","last_active":"2024-09-10T19:03:33.158143+0000","last_peered":"2024-09-10T19:03:33.158143+0000","last_clean":"2024-09-10T19:03:33.158143+0000","last_became_active":"2024-09-10T18:57:02.436059+0000","last_became_peered":"2024-09-10T18:57:02.436059+0000","last_unstale":"2024-09-10T19:03:33.158143+0000","last_undegraded":"2024-09-10T19:03:33.158143+0000","last_fullsized":"2024-09-10T19:03:33.158143+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'35","last_scrub_stamp":"2024-09-10T19:00:43.819004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:00:43.819004+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0050234040000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":375,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.156094+0000","last_change":"2024-09-10T19:01:53.609239+0000","last_active":"2024-09-10T19:03:33.156094+0000","last_peered":"2024-09-10T19:03:33.156094+0000","last_clean":"2024-09-10T19:03:33.156094+0000","last_became_active":"2024-09-10T18:55:40.391849+0000","last_became_peered":"2024-09-10T18:55:40.391849+0000","last_unstale":"2024-09-10T19:03:33.156094+0000","last_undegraded":"2024-09-10T19:03:33.156094+0000","last_fullsized":"2024-09-10T19:03:33.156094+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:01:53.609184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:01:53.609184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.024484227000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":346,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.162018+0000","last_change":"2024-09-10T19:00:42.738386+0000","last_active":"2024-09-10T19:03:33.162018+0000","last_peered":"2024-09-10T19:03:33.162018+0000","last_clean":"2024-09-10T19:03:33.162018+0000","last_became_active":"2024-09-10T19:00:42.514709+0000","last_became_peered":"2024-09-10T19:00:42.514709+0000","last_unstale":"2024-09-10T19:03:33.162018+0000","last_undegraded":"2024-09-10T19:03:33.162018+0000","last_fullsized":"2024-09-10T19:03:33.162018+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:42.738331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:00:42.738331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001211314,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":192,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.166596+0000","last_change":"2024-09-10T19:02:58.491396+0000","last_active":"2024-09-10T19:03:33.166596+0000","last_peered":"2024-09-10T19:03:33.166596+0000","last_clean":"2024-09-10T19:03:33.166596+0000","last_became_active":"2024-09-10T18:57:02.468866+0000","last_became_peered":"2024-09-10T18:57:02.468866+0000","last_unstale":"2024-09-10T19:03:33.166596+0000","last_undegraded":"2024-09-10T19:03:33.166596+0000","last_fullsized":"2024-09-10T19:03:33.166596+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:58.491326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:02:58.491326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:23.761727+0000","scrub_duration":0.0017883440000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"386'1545","reported_seq":1560,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-09-10T19:03:36.560569+0000","last_change":"2024-09-10T19:03:16.045556+0000","last_active":"2024-09-10T19:03:36.560569+0000","last_peered":"2024-09-10T19:03:36.560569+0000","last_clean":"2024-09-10T19:03:36.560569+0000","last_became_active":"2024-09-10T19:03:16.045197+0000","last_became_peered":"2024-09-10T19:03:16.045197+0000","last_unstale":"2024-09-10T19:03:36.560569+0000","last_undegraded":"2024-09-10T19:03:36.560569+0000","last_fullsized":"2024-09-10T19:03:36.560569+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":1545,"log_dups_size":0,"ondisk_log_size":1545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:21.607179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":101253120,"num_objects":1545,"num_object_clones":0,"num_object_copies":4635,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1545,"num_write_kb":98880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,3],"acting":[6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.e","version":"70'81","reported_seq":264,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.154245+0000","last_change":"2024-09-10T18:59:23.592755+0000","last_active":"2024-09-10T19:03:33.154245+0000","last_peered":"2024-09-10T19:03:33.154245+0000","last_clean":"2024-09-10T19:03:33.154245+0000","last_became_active":"2024-09-10T18:57:02.449115+0000","last_became_peered":"2024-09-10T18:57:02.449115+0000","last_unstale":"2024-09-10T19:03:33.154245+0000","last_undegraded":"2024-09-10T19:03:33.154245+0000","last_fullsized":"2024-09-10T19:03:33.154245+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:23.592672+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:59:23.592672+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038976470000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"232'19","reported_seq":402,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.165133+0000","last_change":"2024-09-10T19:00:40.748808+0000","last_active":"2024-09-10T19:03:33.165133+0000","last_peered":"2024-09-10T19:03:33.165133+0000","last_clean":"2024-09-10T19:03:33.165133+0000","last_became_active":"2024-09-10T18:55:40.395257+0000","last_became_peered":"2024-09-10T18:55:40.395257+0000","last_unstale":"2024-09-10T19:03:33.165133+0000","last_undegraded":"2024-09-10T19:03:33.165133+0000","last_fullsized":"2024-09-10T19:03:33.165133+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T19:00:40.748713+0000","last_deep_scrub":"130'13","last_deep_scrub_stamp":"2024-09-10T18:51:58.427534+0000","last_clean_scrub_stamp":"2024-09-10T19:00:40.748713+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043426029999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":26,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.154972+0000","last_change":"2024-09-10T19:02:49.582119+0000","last_active":"2024-09-10T19:03:33.154972+0000","last_peered":"2024-09-10T19:03:33.154972+0000","last_clean":"2024-09-10T19:03:33.154972+0000","last_became_active":"2024-09-10T19:02:49.581709+0000","last_became_peered":"2024-09-10T19:02:49.581709+0000","last_unstale":"2024-09-10T19:03:33.154972+0000","last_undegraded":"2024-09-10T19:03:33.154972+0000","last_fullsized":"2024-09-10T19:03:33.154972+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:09.206317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:59:09.206317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":366,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.153676+0000","last_change":"2024-09-10T19:02:51.031764+0000","last_active":"2024-09-10T19:03:33.153676+0000","last_peered":"2024-09-10T19:03:33.153676+0000","last_clean":"2024-09-10T19:03:33.153676+0000","last_became_active":"2024-09-10T18:55:40.390771+0000","last_became_peered":"2024-09-10T18:55:40.390771+0000","last_unstale":"2024-09-10T19:03:33.153676+0000","last_undegraded":"2024-09-10T19:03:33.153676+0000","last_fullsized":"2024-09-10T19:03:33.153676+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:51.031711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_clean_scrub_stamp":"2024-09-10T19:02:51.031711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:19.854800+0000","scrub_duration":0.0015681510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":390,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.165862+0000","last_change":"2024-09-10T19:01:58.603370+0000","last_active":"2024-09-10T19:03:33.165862+0000","last_peered":"2024-09-10T19:03:33.165862+0000","last_clean":"2024-09-10T19:03:33.165862+0000","last_became_active":"2024-09-10T18:57:02.452387+0000","last_became_peered":"2024-09-10T18:57:02.452387+0000","last_unstale":"2024-09-10T19:03:33.165862+0000","last_undegraded":"2024-09-10T19:03:33.165862+0000","last_fullsized":"2024-09-10T19:03:33.165862+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:01:58.601096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:53.345184+0000","last_clean_scrub_stamp":"2024-09-10T19:01:58.601096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026721589999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"386'1546","reported_seq":1561,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-09-10T19:03:36.636171+0000","last_change":"2024-09-10T19:03:16.045204+0000","last_active":"2024-09-10T19:03:36.636171+0000","last_peered":"2024-09-10T19:03:36.636171+0000","last_clean":"2024-09-10T19:03:36.636171+0000","last_became_active":"2024-09-10T19:03:16.044291+0000","last_became_peered":"2024-09-10T19:03:16.044291+0000","last_unstale":"2024-09-10T19:03:36.636171+0000","last_undegraded":"2024-09-10T19:03:36.636171+0000","last_fullsized":"2024-09-10T19:03:36.636171+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":1546,"log_dups_size":0,"ondisk_log_size":1546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:34.501027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":101318656,"num_objects":1546,"num_object_clones":0,"num_object_copies":4638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1546,"num_write_kb":98944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.d","version":"70'81","reported_seq":205,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.154649+0000","last_change":"2024-09-10T19:02:01.044879+0000","last_active":"2024-09-10T19:03:33.154649+0000","last_peered":"2024-09-10T19:03:33.154649+0000","last_clean":"2024-09-10T19:03:33.154649+0000","last_became_active":"2024-09-10T18:57:02.439652+0000","last_became_peered":"2024-09-10T18:57:02.439652+0000","last_unstale":"2024-09-10T19:03:33.154649+0000","last_undegraded":"2024-09-10T19:03:33.154649+0000","last_fullsized":"2024-09-10T19:03:33.154649+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:01.044143+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:02:01.044143+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014088588000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"365'34","reported_seq":90,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.155317+0000","last_change":"2024-09-10T19:02:18.712390+0000","last_active":"2024-09-10T19:03:33.155317+0000","last_peered":"2024-09-10T19:03:33.155317+0000","last_clean":"2024-09-10T19:03:33.155317+0000","last_became_active":"2024-09-10T19:00:42.515903+0000","last_became_peered":"2024-09-10T19:00:42.515903+0000","last_unstale":"2024-09-10T19:03:33.155317+0000","last_undegraded":"2024-09-10T19:03:33.155317+0000","last_fullsized":"2024-09-10T19:03:33.155317+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"365'34","last_scrub_stamp":"2024-09-10T19:02:18.712306+0000","last_deep_scrub":"365'34","last_deep_scrub_stamp":"2024-09-10T19:02:18.712306+0000","last_clean_scrub_stamp":"2024-09-10T19:02:18.712306+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.017099106999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":33,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":356,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.167487+0000","last_change":"2024-09-10T19:02:08.306579+0000","last_active":"2024-09-10T19:03:33.167487+0000","last_peered":"2024-09-10T19:03:33.167487+0000","last_clean":"2024-09-10T19:03:33.167487+0000","last_became_active":"2024-09-10T18:57:02.461708+0000","last_became_peered":"2024-09-10T18:57:02.461708+0000","last_unstale":"2024-09-10T19:03:33.167487+0000","last_undegraded":"2024-09-10T19:03:33.167487+0000","last_fullsized":"2024-09-10T19:03:33.167487+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:08.306490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:05.603125+0000","last_clean_scrub_stamp":"2024-09-10T19:02:08.306490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043377579999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"273'10","reported_seq":299,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.776344+0000","last_change":"2024-09-10T19:02:46.026041+0000","last_active":"2024-09-10T19:03:33.776344+0000","last_peered":"2024-09-10T19:03:33.776344+0000","last_clean":"2024-09-10T19:03:33.776344+0000","last_became_active":"2024-09-10T18:57:02.441890+0000","last_became_peered":"2024-09-10T18:57:02.441890+0000","last_unstale":"2024-09-10T19:03:33.776344+0000","last_undegraded":"2024-09-10T19:03:33.776344+0000","last_fullsized":"2024-09-10T19:03:33.776344+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'10","last_scrub_stamp":"2024-09-10T19:02:46.025981+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-09-10T18:47:02.582229+0000","last_clean_scrub_stamp":"2024-09-10T19:02:46.025981+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:08.649061+0000","scrub_duration":0.003528864,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":418,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.155565+0000","last_change":"2024-09-10T19:03:32.808204+0000","last_active":"2024-09-10T19:03:33.155565+0000","last_peered":"2024-09-10T19:03:33.155565+0000","last_clean":"2024-09-10T19:03:33.155565+0000","last_became_active":"2024-09-10T18:55:40.389202+0000","last_became_peered":"2024-09-10T18:55:40.389202+0000","last_unstale":"2024-09-10T19:03:33.155565+0000","last_undegraded":"2024-09-10T19:03:33.155565+0000","last_fullsized":"2024-09-10T19:03:33.155565+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:32.808130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_clean_scrub_stamp":"2024-09-10T19:03:32.808130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:54.390143+0000","scrub_duration":0.003767603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"386'1077","reported_seq":1092,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.798512+0000","last_change":"2024-09-10T19:03:16.044527+0000","last_active":"2024-09-10T19:03:33.798512+0000","last_peered":"2024-09-10T19:03:33.798512+0000","last_clean":"2024-09-10T19:03:33.798512+0000","last_became_active":"2024-09-10T19:03:16.044184+0000","last_became_peered":"2024-09-10T19:03:16.044184+0000","last_unstale":"2024-09-10T19:03:33.798512+0000","last_undegraded":"2024-09-10T19:03:33.798512+0000","last_fullsized":"2024-09-10T19:03:33.798512+0000","mapping_epoch":379,"log_start":"0'0","ondisk_log_start":"0'0","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":1077,"log_dups_size":0,"ondisk_log_size":1077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:15.758376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70582272,"num_objects":1077,"num_object_clones":0,"num_object_copies":3231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1077,"num_write_kb":68928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.c","version":"70'81","reported_seq":349,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.166606+0000","last_change":"2024-09-10T19:02:54.540617+0000","last_active":"2024-09-10T19:03:33.166606+0000","last_peered":"2024-09-10T19:03:33.166606+0000","last_clean":"2024-09-10T19:03:33.166606+0000","last_became_active":"2024-09-10T18:55:40.390655+0000","last_became_peered":"2024-09-10T18:55:40.390655+0000","last_unstale":"2024-09-10T19:03:33.166606+0000","last_undegraded":"2024-09-10T19:03:33.166606+0000","last_fullsized":"2024-09-10T19:03:33.166606+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:54.540558+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:56:33.779154+0000","last_clean_scrub_stamp":"2024-09-10T19:02:54.540558+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:01.954015+0000","scrub_duration":0.0019874599999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"232'18","reported_seq":428,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.153350+0000","last_change":"2024-09-10T19:01:53.948548+0000","last_active":"2024-09-10T19:03:33.153350+0000","last_peered":"2024-09-10T19:03:33.153350+0000","last_clean":"2024-09-10T19:03:33.153350+0000","last_became_active":"2024-09-10T18:45:20.056869+0000","last_became_peered":"2024-09-10T18:45:20.056869+0000","last_unstale":"2024-09-10T19:03:33.153350+0000","last_undegraded":"2024-09-10T19:03:33.153350+0000","last_fullsized":"2024-09-10T19:03:33.153350+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'18","last_scrub_stamp":"2024-09-10T19:01:53.948309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:01:53.948309+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010806517,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"56'2","reported_seq":438,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.154828+0000","last_change":"2024-09-10T19:02:06.065853+0000","last_active":"2024-09-10T19:03:33.154828+0000","last_peered":"2024-09-10T19:03:33.154828+0000","last_clean":"2024-09-10T19:03:33.154828+0000","last_became_active":"2024-09-10T18:57:02.434475+0000","last_became_peered":"2024-09-10T18:57:02.434475+0000","last_unstale":"2024-09-10T19:03:33.154828+0000","last_undegraded":"2024-09-10T19:03:33.154828+0000","last_fullsized":"2024-09-10T19:03:33.154828+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-10T19:02:06.065768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:06.065768+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0050047989999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":356,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.162096+0000","last_change":"2024-09-10T19:03:26.818927+0000","last_active":"2024-09-10T19:03:33.162096+0000","last_peered":"2024-09-10T19:03:33.162096+0000","last_clean":"2024-09-10T19:03:33.162096+0000","last_became_active":"2024-09-10T19:00:42.513796+0000","last_became_peered":"2024-09-10T19:00:42.513796+0000","last_unstale":"2024-09-10T19:03:33.162096+0000","last_undegraded":"2024-09-10T19:03:33.162096+0000","last_fullsized":"2024-09-10T19:03:33.162096+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:26.818865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:02.711594+0000","last_clean_scrub_stamp":"2024-09-10T19:03:26.818865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:36.242234+0000","scrub_duration":0.001326844,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":389,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.167546+0000","last_change":"2024-09-10T18:59:11.827181+0000","last_active":"2024-09-10T19:03:33.167546+0000","last_peered":"2024-09-10T19:03:33.167546+0000","last_clean":"2024-09-10T19:03:33.167546+0000","last_became_active":"2024-09-10T18:57:02.459985+0000","last_became_peered":"2024-09-10T18:57:02.459985+0000","last_unstale":"2024-09-10T19:03:33.167546+0000","last_undegraded":"2024-09-10T19:03:33.167546+0000","last_fullsized":"2024-09-10T19:03:33.167546+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:11.827101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:53.352427+0000","last_clean_scrub_stamp":"2024-09-10T18:59:11.827101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034557899999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.13","version":"70'81","reported_seq":95,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.168087+0000","last_change":"2024-09-10T19:02:49.581193+0000","last_active":"2024-09-10T19:03:33.168087+0000","last_peered":"2024-09-10T19:03:33.168087+0000","last_clean":"2024-09-10T19:03:33.168087+0000","last_became_active":"2024-09-10T19:02:49.580656+0000","last_became_peered":"2024-09-10T19:02:49.580656+0000","last_unstale":"2024-09-10T19:03:33.168087+0000","last_undegraded":"2024-09-10T19:03:33.168087+0000","last_fullsized":"2024-09-10T19:03:33.168087+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:01:56.384747+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:01:56.384747+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007621203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"232'19","reported_seq":425,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.165761+0000","last_change":"2024-09-10T19:02:02.627107+0000","last_active":"2024-09-10T19:03:33.165761+0000","last_peered":"2024-09-10T19:03:33.165761+0000","last_clean":"2024-09-10T19:03:33.165761+0000","last_became_active":"2024-09-10T18:57:02.453001+0000","last_became_peered":"2024-09-10T18:57:02.453001+0000","last_unstale":"2024-09-10T19:03:33.165761+0000","last_undegraded":"2024-09-10T19:03:33.165761+0000","last_fullsized":"2024-09-10T19:03:33.165761+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T19:02:02.627031+0000","last_deep_scrub":"76'13","last_deep_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_clean_scrub_stamp":"2024-09-10T19:02:02.627031+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00634981,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":229,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.161901+0000","last_change":"2024-09-10T19:02:49.581182+0000","last_active":"2024-09-10T19:03:33.161901+0000","last_peered":"2024-09-10T19:03:33.161901+0000","last_clean":"2024-09-10T19:03:33.161901+0000","last_became_active":"2024-09-10T19:02:49.580701+0000","last_became_peered":"2024-09-10T19:02:49.580701+0000","last_unstale":"2024-09-10T19:03:33.161901+0000","last_undegraded":"2024-09-10T19:03:33.161901+0000","last_fullsized":"2024-09-10T19:03:33.161901+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:59.385349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:59:59.385349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041652920000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":401,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.167784+0000","last_change":"2024-09-10T19:02:23.261573+0000","last_active":"2024-09-10T19:03:33.167784+0000","last_peered":"2024-09-10T19:03:33.167784+0000","last_clean":"2024-09-10T19:03:33.167784+0000","last_became_active":"2024-09-10T18:57:02.455341+0000","last_became_peered":"2024-09-10T18:57:02.455341+0000","last_unstale":"2024-09-10T19:03:33.167784+0000","last_undegraded":"2024-09-10T19:03:33.167784+0000","last_fullsized":"2024-09-10T19:03:33.167784+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:23.261482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:00.754912+0000","last_clean_scrub_stamp":"2024-09-10T19:02:23.261482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0076325380000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"364'2","reported_seq":425,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.153898+0000","last_change":"2024-09-10T19:02:15.059462+0000","last_active":"2024-09-10T19:03:33.153898+0000","last_peered":"2024-09-10T19:03:33.153898+0000","last_clean":"2024-09-10T19:03:33.153898+0000","last_became_active":"2024-09-10T18:59:48.741275+0000","last_became_peered":"2024-09-10T18:59:48.741275+0000","last_unstale":"2024-09-10T19:03:33.153898+0000","last_undegraded":"2024-09-10T19:03:33.153898+0000","last_fullsized":"2024-09-10T19:03:33.153898+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"364'2","last_scrub_stamp":"2024-09-10T19:02:15.059320+0000","last_deep_scrub":"364'2","last_deep_scrub_stamp":"2024-09-10T19:02:15.059320+0000","last_clean_scrub_stamp":"2024-09-10T19:02:15.059320+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.020873968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":52,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.12","version":"70'81","reported_seq":265,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.155422+0000","last_change":"2024-09-10T19:02:05.565674+0000","last_active":"2024-09-10T19:03:33.155422+0000","last_peered":"2024-09-10T19:03:33.155422+0000","last_clean":"2024-09-10T19:03:33.155422+0000","last_became_active":"2024-09-10T18:52:16.598921+0000","last_became_peered":"2024-09-10T18:52:16.598921+0000","last_unstale":"2024-09-10T19:03:33.155422+0000","last_undegraded":"2024-09-10T19:03:33.155422+0000","last_fullsized":"2024-09-10T19:03:33.155422+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:05.565587+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:54.127476+0000","last_clean_scrub_stamp":"2024-09-10T19:02:05.565587+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051538479999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":431,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.158453+0000","last_change":"2024-09-10T19:02:18.973390+0000","last_active":"2024-09-10T19:03:33.158453+0000","last_peered":"2024-09-10T19:03:33.158453+0000","last_clean":"2024-09-10T19:03:33.158453+0000","last_became_active":"2024-09-10T18:57:02.442251+0000","last_became_peered":"2024-09-10T18:57:02.442251+0000","last_unstale":"2024-09-10T19:03:33.158453+0000","last_undegraded":"2024-09-10T19:03:33.158453+0000","last_fullsized":"2024-09-10T19:03:33.158453+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:18.973341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:18.973341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001038347,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"232'12","reported_seq":422,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.167989+0000","last_change":"2024-09-10T18:59:52.198399+0000","last_active":"2024-09-10T19:03:33.167989+0000","last_peered":"2024-09-10T19:03:33.167989+0000","last_clean":"2024-09-10T19:03:33.167989+0000","last_became_active":"2024-09-10T18:57:02.460430+0000","last_became_peered":"2024-09-10T18:57:02.460430+0000","last_unstale":"2024-09-10T19:03:33.167989+0000","last_undegraded":"2024-09-10T19:03:33.167989+0000","last_fullsized":"2024-09-10T19:03:33.167989+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'12","last_scrub_stamp":"2024-09-10T18:59:52.198340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:59:52.198340+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001790591,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":393,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.156246+0000","last_change":"2024-09-10T19:01:58.529083+0000","last_active":"2024-09-10T19:03:33.156246+0000","last_peered":"2024-09-10T19:03:33.156246+0000","last_clean":"2024-09-10T19:03:33.156246+0000","last_became_active":"2024-09-10T18:45:22.069653+0000","last_became_peered":"2024-09-10T18:45:22.069653+0000","last_unstale":"2024-09-10T19:03:33.156246+0000","last_undegraded":"2024-09-10T19:03:33.156246+0000","last_fullsized":"2024-09-10T19:03:33.156246+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:01:58.528538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:01:58.528538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0057370839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":158,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.158839+0000","last_change":"2024-09-10T19:02:14.037294+0000","last_active":"2024-09-10T19:03:33.158839+0000","last_peered":"2024-09-10T19:03:33.158839+0000","last_clean":"2024-09-10T19:03:33.158839+0000","last_became_active":"2024-09-10T19:00:42.519492+0000","last_became_peered":"2024-09-10T19:00:42.519492+0000","last_unstale":"2024-09-10T19:03:33.158839+0000","last_undegraded":"2024-09-10T19:03:33.158839+0000","last_fullsized":"2024-09-10T19:03:33.158839+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:14.037238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:02:14.037238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001210221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.11","version":"70'81","reported_seq":300,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.166071+0000","last_change":"2024-09-10T19:02:17.778329+0000","last_active":"2024-09-10T19:03:33.166071+0000","last_peered":"2024-09-10T19:03:33.166071+0000","last_clean":"2024-09-10T19:03:33.166071+0000","last_became_active":"2024-09-10T19:00:42.517311+0000","last_became_peered":"2024-09-10T19:00:42.517311+0000","last_unstale":"2024-09-10T19:03:33.166071+0000","last_undegraded":"2024-09-10T19:03:33.166071+0000","last_fullsized":"2024-09-10T19:03:33.166071+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:17.778227+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:02:17.778227+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0070248109999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"232'20","reported_seq":479,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.166223+0000","last_change":"2024-09-10T19:00:07.147063+0000","last_active":"2024-09-10T19:03:33.166223+0000","last_peered":"2024-09-10T19:03:33.166223+0000","last_clean":"2024-09-10T19:03:33.166223+0000","last_became_active":"2024-09-10T18:57:02.444437+0000","last_became_peered":"2024-09-10T18:57:02.444437+0000","last_unstale":"2024-09-10T19:03:33.166223+0000","last_undegraded":"2024-09-10T19:03:33.166223+0000","last_fullsized":"2024-09-10T19:03:33.166223+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'20","last_scrub_stamp":"2024-09-10T19:00:07.146928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:00:07.146928+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0071404040000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":417,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.155196+0000","last_change":"2024-09-10T19:02:20.755424+0000","last_active":"2024-09-10T19:03:33.155196+0000","last_peered":"2024-09-10T19:03:33.155196+0000","last_clean":"2024-09-10T19:03:33.155196+0000","last_became_active":"2024-09-10T18:57:02.445710+0000","last_became_peered":"2024-09-10T18:57:02.445710+0000","last_unstale":"2024-09-10T19:03:33.155196+0000","last_undegraded":"2024-09-10T19:03:33.155196+0000","last_fullsized":"2024-09-10T19:03:33.155196+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:20.755253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:20.755253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:37.147814+0000","scrub_duration":0.0023404490000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"304'11","reported_seq":665,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-09-10T19:03:34.811371+0000","last_change":"2024-09-10T19:03:34.811295+0000","last_active":"2024-09-10T19:03:34.811371+0000","last_peered":"2024-09-10T19:03:34.811371+0000","last_clean":"2024-09-10T19:03:34.811371+0000","last_became_active":"2024-09-10T18:58:58.489370+0000","last_became_peered":"2024-09-10T18:58:58.489370+0000","last_unstale":"2024-09-10T19:03:34.811371+0000","last_undegraded":"2024-09-10T19:03:34.811371+0000","last_fullsized":"2024-09-10T19:03:34.811371+0000","mapping_epoch":303,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":304,"parent":"0.0","parent_split_bits":0,"last_scrub":"304'11","last_scrub_stamp":"2024-09-10T19:03:34.811194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:03:34.811194+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:49.802623+0000","scrub_duration":0.0024856539999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":85,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.165502+0000","last_change":"2024-09-10T19:02:22.928743+0000","last_active":"2024-09-10T19:03:33.165502+0000","last_peered":"2024-09-10T19:03:33.165502+0000","last_clean":"2024-09-10T19:03:33.165502+0000","last_became_active":"2024-09-10T18:59:48.744923+0000","last_became_peered":"2024-09-10T18:59:48.744923+0000","last_unstale":"2024-09-10T19:03:33.165502+0000","last_undegraded":"2024-09-10T19:03:33.165502+0000","last_fullsized":"2024-09-10T19:03:33.165502+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:22.928652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:57.476724+0000","last_clean_scrub_stamp":"2024-09-10T19:02:22.928652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:49.708465+0000","scrub_duration":0.013720313,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.10","version":"70'81","reported_seq":265,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.155399+0000","last_change":"2024-09-10T19:00:04.612412+0000","last_active":"2024-09-10T19:03:33.155399+0000","last_peered":"2024-09-10T19:03:33.155399+0000","last_clean":"2024-09-10T19:03:33.155399+0000","last_became_active":"2024-09-10T18:57:02.443782+0000","last_became_peered":"2024-09-10T18:57:02.443782+0000","last_unstale":"2024-09-10T19:03:33.155399+0000","last_undegraded":"2024-09-10T19:03:33.155399+0000","last_fullsized":"2024-09-10T19:03:33.155399+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:00:04.612324+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:00:04.612324+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0025961159999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"232'15","reported_seq":77,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.156139+0000","last_change":"2024-09-10T19:02:46.674532+0000","last_active":"2024-09-10T19:03:33.156139+0000","last_peered":"2024-09-10T19:03:33.156139+0000","last_clean":"2024-09-10T19:03:33.156139+0000","last_became_active":"2024-09-10T19:00:42.518672+0000","last_became_peered":"2024-09-10T19:00:42.518672+0000","last_unstale":"2024-09-10T19:03:33.156139+0000","last_undegraded":"2024-09-10T19:03:33.156139+0000","last_fullsized":"2024-09-10T19:03:33.156139+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T19:02:46.674451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:46.674451+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:51.915396+0000","scrub_duration":0.0082292530000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":374,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.162098+0000","last_change":"2024-09-10T19:02:03.587108+0000","last_active":"2024-09-10T19:03:33.162098+0000","last_peered":"2024-09-10T19:03:33.162098+0000","last_clean":"2024-09-10T19:03:33.162098+0000","last_became_active":"2024-09-10T18:55:40.390697+0000","last_became_peered":"2024-09-10T18:55:40.390697+0000","last_unstale":"2024-09-10T19:03:33.162098+0000","last_undegraded":"2024-09-10T19:03:33.162098+0000","last_fullsized":"2024-09-10T19:03:33.162098+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:03.586986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:03.586986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0079434420000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"273'11","reported_seq":637,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.166689+0000","last_change":"2024-09-10T18:59:53.158730+0000","last_active":"2024-09-10T19:03:33.166689+0000","last_peered":"2024-09-10T19:03:33.166689+0000","last_clean":"2024-09-10T19:03:33.166689+0000","last_became_active":"2024-09-10T18:57:02.445268+0000","last_became_peered":"2024-09-10T18:57:02.445268+0000","last_unstale":"2024-09-10T19:03:33.166689+0000","last_undegraded":"2024-09-10T19:03:33.166689+0000","last_fullsized":"2024-09-10T19:03:33.166689+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'11","last_scrub_stamp":"2024-09-10T18:59:53.158664+0000","last_deep_scrub":"81'9","last_deep_scrub_stamp":"2024-09-10T18:46:54.800790+0000","last_clean_scrub_stamp":"2024-09-10T18:59:53.158664+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017427930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":416,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.154551+0000","last_change":"2024-09-10T19:03:23.737568+0000","last_active":"2024-09-10T19:03:33.154551+0000","last_peered":"2024-09-10T19:03:33.154551+0000","last_clean":"2024-09-10T19:03:33.154551+0000","last_became_active":"2024-09-10T18:55:40.391662+0000","last_became_peered":"2024-09-10T18:55:40.391662+0000","last_unstale":"2024-09-10T19:03:33.154551+0000","last_undegraded":"2024-09-10T19:03:33.154551+0000","last_fullsized":"2024-09-10T19:03:33.154551+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:23.737518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_clean_scrub_stamp":"2024-09-10T19:03:23.737518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:29.096316+0000","scrub_duration":0.0037070459999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"232'15","reported_seq":291,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.157983+0000","last_change":"2024-09-10T19:02:49.570493+0000","last_active":"2024-09-10T19:03:33.157983+0000","last_peered":"2024-09-10T19:03:33.157983+0000","last_clean":"2024-09-10T19:03:33.157983+0000","last_became_active":"2024-09-10T19:02:49.561403+0000","last_became_peered":"2024-09-10T19:02:49.561403+0000","last_unstale":"2024-09-10T19:03:33.157983+0000","last_undegraded":"2024-09-10T19:03:33.157983+0000","last_fullsized":"2024-09-10T19:03:33.157983+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T18:59:53.320757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:59:53.320757+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023792729999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":448,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-09-10T19:03:36.087645+0000","last_change":"2024-09-10T19:03:36.087607+0000","last_active":"2024-09-10T19:03:36.087645+0000","last_peered":"2024-09-10T19:03:36.087645+0000","last_clean":"2024-09-10T19:03:36.087645+0000","last_became_active":"2024-09-10T19:00:42.518310+0000","last_became_peered":"2024-09-10T19:00:42.518310+0000","last_unstale":"2024-09-10T19:03:36.087645+0000","last_undegraded":"2024-09-10T19:03:36.087645+0000","last_fullsized":"2024-09-10T19:03:36.087645+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:36.087523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:03:36.087523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:41.676390+0000","scrub_duration":0.0039520240000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":398,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.167654+0000","last_change":"2024-09-10T19:02:22.258152+0000","last_active":"2024-09-10T19:03:33.167654+0000","last_peered":"2024-09-10T19:03:33.167654+0000","last_clean":"2024-09-10T19:03:33.167654+0000","last_became_active":"2024-09-10T18:55:40.398243+0000","last_became_peered":"2024-09-10T18:55:40.398243+0000","last_unstale":"2024-09-10T19:03:33.167654+0000","last_undegraded":"2024-09-10T19:03:33.167654+0000","last_fullsized":"2024-09-10T19:03:33.167654+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:22.258058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:03.571786+0000","last_clean_scrub_stamp":"2024-09-10T19:02:22.258058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:37.095041+0000","scrub_duration":0.0054525670000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":389,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.162284+0000","last_change":"2024-09-10T19:03:22.930232+0000","last_active":"2024-09-10T19:03:33.162284+0000","last_peered":"2024-09-10T19:03:33.162284+0000","last_clean":"2024-09-10T19:03:33.162284+0000","last_became_active":"2024-09-10T19:00:42.516833+0000","last_became_peered":"2024-09-10T19:00:42.516833+0000","last_unstale":"2024-09-10T19:03:33.162284+0000","last_undegraded":"2024-09-10T19:03:33.162284+0000","last_fullsized":"2024-09-10T19:03:33.162284+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:22.930166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:04.631558+0000","last_clean_scrub_stamp":"2024-09-10T19:03:22.930166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:40.776332+0000","scrub_duration":0.001481163,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"232'24","reported_seq":419,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.153676+0000","last_change":"2024-09-10T18:58:58.698097+0000","last_active":"2024-09-10T19:03:33.153676+0000","last_peered":"2024-09-10T19:03:33.153676+0000","last_clean":"2024-09-10T19:03:33.153676+0000","last_became_active":"2024-09-10T18:45:20.056885+0000","last_became_peered":"2024-09-10T18:45:20.056885+0000","last_unstale":"2024-09-10T19:03:33.153676+0000","last_undegraded":"2024-09-10T19:03:33.153676+0000","last_fullsized":"2024-09-10T19:03:33.153676+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T18:58:58.698003+0000","last_deep_scrub":"223'16","last_deep_scrub_stamp":"2024-09-10T18:54:47.324044+0000","last_clean_scrub_stamp":"2024-09-10T18:58:58.698003+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.006955676,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":268,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.156236+0000","last_change":"2024-09-10T19:02:49.573080+0000","last_active":"2024-09-10T19:03:33.156236+0000","last_peered":"2024-09-10T19:03:33.156236+0000","last_clean":"2024-09-10T19:03:33.156236+0000","last_became_active":"2024-09-10T19:02:49.572748+0000","last_became_peered":"2024-09-10T19:02:49.572748+0000","last_unstale":"2024-09-10T19:03:33.156236+0000","last_undegraded":"2024-09-10T19:03:33.156236+0000","last_fullsized":"2024-09-10T19:03:33.156236+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:19.301278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:00.029389+0000","last_clean_scrub_stamp":"2024-09-10T19:02:19.301278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003302677,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":26,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.155559+0000","last_change":"2024-09-10T19:02:49.573280+0000","last_active":"2024-09-10T19:03:33.155559+0000","last_peered":"2024-09-10T19:03:33.155559+0000","last_clean":"2024-09-10T19:03:33.155559+0000","last_became_active":"2024-09-10T19:02:49.572998+0000","last_became_peered":"2024-09-10T19:02:49.572998+0000","last_unstale":"2024-09-10T19:03:33.155559+0000","last_undegraded":"2024-09-10T19:03:33.155559+0000","last_fullsized":"2024-09-10T19:03:33.155559+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:15.321767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:02:15.321767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:44.811026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":396,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.166891+0000","last_change":"2024-09-10T19:02:00.273929+0000","last_active":"2024-09-10T19:03:33.166891+0000","last_peered":"2024-09-10T19:03:33.166891+0000","last_clean":"2024-09-10T19:03:33.166891+0000","last_became_active":"2024-09-10T18:55:40.395595+0000","last_became_peered":"2024-09-10T18:55:40.395595+0000","last_unstale":"2024-09-10T19:03:33.166891+0000","last_undegraded":"2024-09-10T19:03:33.166891+0000","last_fullsized":"2024-09-10T19:03:33.166891+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:00.273863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:02:00.273863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002006552,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"232'8","reported_seq":405,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.166746+0000","last_change":"2024-09-10T19:02:50.391929+0000","last_active":"2024-09-10T19:03:33.166746+0000","last_peered":"2024-09-10T19:03:33.166746+0000","last_clean":"2024-09-10T19:03:33.166746+0000","last_became_active":"2024-09-10T18:45:20.059205+0000","last_became_peered":"2024-09-10T18:45:20.059205+0000","last_unstale":"2024-09-10T19:03:33.166746+0000","last_undegraded":"2024-09-10T19:03:33.166746+0000","last_fullsized":"2024-09-10T19:03:33.166746+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'8","last_scrub_stamp":"2024-09-10T19:02:50.391489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:50.391489+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:02.290160+0000","scrub_duration":0.0041757890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":413,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.156247+0000","last_change":"2024-09-10T19:02:16.704889+0000","last_active":"2024-09-10T19:03:33.156247+0000","last_peered":"2024-09-10T19:03:33.156247+0000","last_clean":"2024-09-10T19:03:33.156247+0000","last_became_active":"2024-09-10T18:47:36.415866+0000","last_became_peered":"2024-09-10T18:47:36.415866+0000","last_unstale":"2024-09-10T19:03:33.156247+0000","last_undegraded":"2024-09-10T19:03:33.156247+0000","last_fullsized":"2024-09-10T19:03:33.156247+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:16.704805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:16.704805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:44.249699+0000","scrub_duration":0.0083395729999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":424,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.158255+0000","last_change":"2024-09-10T19:02:08.092456+0000","last_active":"2024-09-10T19:03:33.158255+0000","last_peered":"2024-09-10T19:03:33.158255+0000","last_clean":"2024-09-10T19:03:33.158255+0000","last_became_active":"2024-09-10T18:57:02.453974+0000","last_became_peered":"2024-09-10T18:57:02.453974+0000","last_unstale":"2024-09-10T19:03:33.158255+0000","last_undegraded":"2024-09-10T19:03:33.158255+0000","last_fullsized":"2024-09-10T19:03:33.158255+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:08.092366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_clean_scrub_stamp":"2024-09-10T19:02:08.092366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:34.680292+0000","scrub_duration":0.0058035439999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":184,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.153521+0000","last_change":"2024-09-10T18:59:55.812370+0000","last_active":"2024-09-10T19:03:33.153521+0000","last_peered":"2024-09-10T19:03:33.153521+0000","last_clean":"2024-09-10T19:03:33.153521+0000","last_became_active":"2024-09-10T18:57:02.447566+0000","last_became_peered":"2024-09-10T18:57:02.447566+0000","last_unstale":"2024-09-10T19:03:33.153521+0000","last_undegraded":"2024-09-10T19:03:33.153521+0000","last_fullsized":"2024-09-10T19:03:33.153521+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:55.812312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:55.145638+0000","last_clean_scrub_stamp":"2024-09-10T18:59:55.812312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0055522669999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.14","version":"70'81","reported_seq":298,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.153995+0000","last_change":"2024-09-10T19:02:03.086382+0000","last_active":"2024-09-10T19:03:33.153995+0000","last_peered":"2024-09-10T19:03:33.153995+0000","last_clean":"2024-09-10T19:03:33.153995+0000","last_became_active":"2024-09-10T18:57:02.442457+0000","last_became_peered":"2024-09-10T18:57:02.442457+0000","last_unstale":"2024-09-10T19:03:33.153995+0000","last_undegraded":"2024-09-10T19:03:33.153995+0000","last_fullsized":"2024-09-10T19:03:33.153995+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:03.086329+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:57.184383+0000","last_clean_scrub_stamp":"2024-09-10T19:02:03.086329+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026855270000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.11","version":"232'24","reported_seq":441,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.166126+0000","last_change":"2024-09-10T18:59:10.879686+0000","last_active":"2024-09-10T19:03:33.166126+0000","last_peered":"2024-09-10T19:03:33.166126+0000","last_clean":"2024-09-10T19:03:33.166126+0000","last_became_active":"2024-09-10T18:57:02.461909+0000","last_became_peered":"2024-09-10T18:57:02.461909+0000","last_unstale":"2024-09-10T19:03:33.166126+0000","last_undegraded":"2024-09-10T19:03:33.166126+0000","last_fullsized":"2024-09-10T19:03:33.166126+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T18:59:10.879597+0000","last_deep_scrub":"79'16","last_deep_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_clean_scrub_stamp":"2024-09-10T18:59:10.879597+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0090530790000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"56'1","reported_seq":429,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.159170+0000","last_change":"2024-09-10T19:01:58.121943+0000","last_active":"2024-09-10T19:03:33.159170+0000","last_peered":"2024-09-10T19:03:33.159170+0000","last_clean":"2024-09-10T19:03:33.159170+0000","last_became_active":"2024-09-10T18:47:36.421985+0000","last_became_peered":"2024-09-10T18:47:36.421985+0000","last_unstale":"2024-09-10T19:03:33.159170+0000","last_undegraded":"2024-09-10T19:03:33.159170+0000","last_fullsized":"2024-09-10T19:03:33.159170+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T19:01:58.121852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:01:58.121852+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002125984,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":141,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.153097+0000","last_change":"2024-09-10T19:02:14.088121+0000","last_active":"2024-09-10T19:03:33.153097+0000","last_peered":"2024-09-10T19:03:33.153097+0000","last_clean":"2024-09-10T19:03:33.153097+0000","last_became_active":"2024-09-10T18:57:02.435124+0000","last_became_peered":"2024-09-10T18:57:02.435124+0000","last_unstale":"2024-09-10T19:03:33.153097+0000","last_undegraded":"2024-09-10T19:03:33.153097+0000","last_fullsized":"2024-09-10T19:03:33.153097+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:14.088018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:02:14.088018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.031802916000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":406,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.167382+0000","last_change":"2024-09-10T19:02:16.321814+0000","last_active":"2024-09-10T19:03:33.167382+0000","last_peered":"2024-09-10T19:03:33.167382+0000","last_clean":"2024-09-10T19:03:33.167382+0000","last_became_active":"2024-09-10T18:55:40.397523+0000","last_became_peered":"2024-09-10T18:55:40.397523+0000","last_unstale":"2024-09-10T19:03:33.167382+0000","last_undegraded":"2024-09-10T19:03:33.167382+0000","last_fullsized":"2024-09-10T19:03:33.167382+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:16.321715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:02:16.321715+0000","last_clean_scrub_stamp":"2024-09-10T19:02:16.321715+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0069778510000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"78'2","reported_seq":184,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.155928+0000","last_change":"2024-09-10T19:02:15.686223+0000","last_active":"2024-09-10T19:03:33.155928+0000","last_peered":"2024-09-10T19:03:33.155928+0000","last_clean":"2024-09-10T19:03:33.155928+0000","last_became_active":"2024-09-10T18:57:02.447198+0000","last_became_peered":"2024-09-10T18:57:02.447198+0000","last_unstale":"2024-09-10T19:03:33.155928+0000","last_undegraded":"2024-09-10T19:03:33.155928+0000","last_fullsized":"2024-09-10T19:03:33.155928+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'2","last_scrub_stamp":"2024-09-10T19:02:15.686138+0000","last_deep_scrub":"78'2","last_deep_scrub_stamp":"2024-09-10T19:02:15.686138+0000","last_clean_scrub_stamp":"2024-09-10T19:02:15.686138+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011160388,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":407,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.155521+0000","last_change":"2024-09-10T18:59:53.505658+0000","last_active":"2024-09-10T19:03:33.155521+0000","last_peered":"2024-09-10T19:03:33.155521+0000","last_clean":"2024-09-10T19:03:33.155521+0000","last_became_active":"2024-09-10T18:57:02.443353+0000","last_became_peered":"2024-09-10T18:57:02.443353+0000","last_unstale":"2024-09-10T19:03:33.155521+0000","last_undegraded":"2024-09-10T19:03:33.155521+0000","last_fullsized":"2024-09-10T19:03:33.155521+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:53.505561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:59:53.505561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001238605,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"232'20","reported_seq":423,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.165486+0000","last_change":"2024-09-10T19:02:04.609420+0000","last_active":"2024-09-10T19:03:33.165486+0000","last_peered":"2024-09-10T19:03:33.165486+0000","last_clean":"2024-09-10T19:03:33.165486+0000","last_became_active":"2024-09-10T18:57:02.452473+0000","last_became_peered":"2024-09-10T18:57:02.452473+0000","last_unstale":"2024-09-10T19:03:33.165486+0000","last_undegraded":"2024-09-10T19:03:33.165486+0000","last_fullsized":"2024-09-10T19:03:33.165486+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'20","last_scrub_stamp":"2024-09-10T19:02:04.609332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:04.609332+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0075275430000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":23,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"273'10","reported_seq":660,"reported_epoch":386,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.777155+0000","last_change":"2024-09-10T19:03:24.245622+0000","last_active":"2024-09-10T19:03:33.777155+0000","last_peered":"2024-09-10T19:03:33.777155+0000","last_clean":"2024-09-10T19:03:33.777155+0000","last_became_active":"2024-09-10T18:57:02.441763+0000","last_became_peered":"2024-09-10T18:57:02.441763+0000","last_unstale":"2024-09-10T19:03:33.777155+0000","last_undegraded":"2024-09-10T19:03:33.777155+0000","last_fullsized":"2024-09-10T19:03:33.777155+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'10","last_scrub_stamp":"2024-09-10T19:03:24.245532+0000","last_deep_scrub":"190'9","last_deep_scrub_stamp":"2024-09-10T18:53:07.746309+0000","last_clean_scrub_stamp":"2024-09-10T19:03:24.245532+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:42.657844+0000","scrub_duration":0.0019049480000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":26,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.155912+0000","last_change":"2024-09-10T19:02:49.582100+0000","last_active":"2024-09-10T19:03:33.155912+0000","last_peered":"2024-09-10T19:03:33.155912+0000","last_clean":"2024-09-10T19:03:33.155912+0000","last_became_active":"2024-09-10T19:02:49.581739+0000","last_became_peered":"2024-09-10T19:02:49.581739+0000","last_unstale":"2024-09-10T19:03:33.155912+0000","last_undegraded":"2024-09-10T19:03:33.155912+0000","last_fullsized":"2024-09-10T19:03:33.155912+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:23.327767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:03.475299+0000","last_clean_scrub_stamp":"2024-09-10T19:02:23.327767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:03:34.923224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":402,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.165673+0000","last_change":"2024-09-10T19:00:10.182129+0000","last_active":"2024-09-10T19:03:33.165673+0000","last_peered":"2024-09-10T19:03:33.165673+0000","last_clean":"2024-09-10T19:03:33.165673+0000","last_became_active":"2024-09-10T18:55:40.392524+0000","last_became_peered":"2024-09-10T18:55:40.392524+0000","last_unstale":"2024-09-10T19:03:33.165673+0000","last_undegraded":"2024-09-10T19:03:33.165673+0000","last_fullsized":"2024-09-10T19:03:33.165673+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:10.181982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:00:10.181982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029510420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"232'19","reported_seq":440,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.158549+0000","last_change":"2024-09-10T19:02:12.120953+0000","last_active":"2024-09-10T19:03:33.158549+0000","last_peered":"2024-09-10T19:03:33.158549+0000","last_clean":"2024-09-10T19:03:33.158549+0000","last_became_active":"2024-09-10T18:45:20.058717+0000","last_became_peered":"2024-09-10T18:45:20.058717+0000","last_unstale":"2024-09-10T19:03:33.158549+0000","last_undegraded":"2024-09-10T19:03:33.158549+0000","last_fullsized":"2024-09-10T19:03:33.158549+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T19:02:12.120860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:12.120860+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060875590000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":377,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.165689+0000","last_change":"2024-09-10T19:03:25.864320+0000","last_active":"2024-09-10T19:03:33.165689+0000","last_peered":"2024-09-10T19:03:33.165689+0000","last_clean":"2024-09-10T19:03:33.165689+0000","last_became_active":"2024-09-10T18:57:02.453464+0000","last_became_peered":"2024-09-10T18:57:02.453464+0000","last_unstale":"2024-09-10T19:03:33.165689+0000","last_undegraded":"2024-09-10T19:03:33.165689+0000","last_fullsized":"2024-09-10T19:03:33.165689+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:25.864264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:03:25.864264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:36.948487+0000","scrub_duration":0.0013441410000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":379,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.154455+0000","last_change":"2024-09-10T18:58:55.726863+0000","last_active":"2024-09-10T19:03:33.154455+0000","last_peered":"2024-09-10T19:03:33.154455+0000","last_clean":"2024-09-10T19:03:33.154455+0000","last_became_active":"2024-09-10T18:45:24.267251+0000","last_became_peered":"2024-09-10T18:45:24.267251+0000","last_unstale":"2024-09-10T19:03:33.154455+0000","last_undegraded":"2024-09-10T19:03:33.154455+0000","last_fullsized":"2024-09-10T19:03:33.154455+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:55.726799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T18:58:55.726799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001154721,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"306'2","reported_seq":435,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.158247+0000","last_change":"2024-09-10T19:03:00.120585+0000","last_active":"2024-09-10T19:03:33.158247+0000","last_peered":"2024-09-10T19:03:33.158247+0000","last_clean":"2024-09-10T19:03:33.158247+0000","last_became_active":"2024-09-10T18:57:02.435758+0000","last_became_peered":"2024-09-10T18:57:02.435758+0000","last_unstale":"2024-09-10T19:03:33.158247+0000","last_undegraded":"2024-09-10T19:03:33.158247+0000","last_fullsized":"2024-09-10T19:03:33.158247+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"306'2","last_scrub_stamp":"2024-09-10T19:03:00.120529+0000","last_deep_scrub":"306'2","last_deep_scrub_stamp":"2024-09-10T18:59:20.534869+0000","last_clean_scrub_stamp":"2024-09-10T19:03:00.120529+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:18.610353+0000","scrub_duration":0.0016153859999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"258'34","reported_seq":333,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.166376+0000","last_change":"2024-09-10T19:02:49.594744+0000","last_active":"2024-09-10T19:03:33.166376+0000","last_peered":"2024-09-10T19:03:33.166376+0000","last_clean":"2024-09-10T19:03:33.166376+0000","last_became_active":"2024-09-10T19:02:49.594207+0000","last_became_peered":"2024-09-10T19:02:49.594207+0000","last_unstale":"2024-09-10T19:03:33.166376+0000","last_undegraded":"2024-09-10T19:03:33.166376+0000","last_fullsized":"2024-09-10T19:03:33.166376+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'34","last_scrub_stamp":"2024-09-10T19:02:02.373321+0000","last_deep_scrub":"258'34","last_deep_scrub_stamp":"2024-09-10T18:56:23.838951+0000","last_clean_scrub_stamp":"2024-09-10T19:02:02.373321+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030853120000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":254,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.154446+0000","last_change":"2024-09-10T19:02:49.581903+0000","last_active":"2024-09-10T19:03:33.154446+0000","last_peered":"2024-09-10T19:03:33.154446+0000","last_clean":"2024-09-10T19:03:33.154446+0000","last_became_active":"2024-09-10T19:02:49.581575+0000","last_became_peered":"2024-09-10T19:02:49.581575+0000","last_unstale":"2024-09-10T19:03:33.154446+0000","last_undegraded":"2024-09-10T19:03:33.154446+0000","last_fullsized":"2024-09-10T19:03:33.154446+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:03.343366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:02:03.343366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0079641550000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":409,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.638188+0000","last_change":"2024-09-10T19:03:33.638108+0000","last_active":"2024-09-10T19:03:33.638188+0000","last_peered":"2024-09-10T19:03:33.638188+0000","last_clean":"2024-09-10T19:03:33.638188+0000","last_became_active":"2024-09-10T19:00:42.520239+0000","last_became_peered":"2024-09-10T19:00:42.520239+0000","last_unstale":"2024-09-10T19:03:33.638188+0000","last_undegraded":"2024-09-10T19:03:33.638188+0000","last_fullsized":"2024-09-10T19:03:33.638188+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:33.638010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:17.985545+0000","last_clean_scrub_stamp":"2024-09-10T19:03:33.638010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:48.051162+0000","scrub_duration":0.0035871750000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":405,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.167826+0000","last_change":"2024-09-10T18:58:57.919890+0000","last_active":"2024-09-10T19:03:33.167826+0000","last_peered":"2024-09-10T19:03:33.167826+0000","last_clean":"2024-09-10T19:03:33.167826+0000","last_became_active":"2024-09-10T18:57:02.460855+0000","last_became_peered":"2024-09-10T18:57:02.460855+0000","last_unstale":"2024-09-10T19:03:33.167826+0000","last_undegraded":"2024-09-10T19:03:33.167826+0000","last_fullsized":"2024-09-10T19:03:33.167826+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:57.919515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T18:58:57.919515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.01355234,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"232'18","reported_seq":415,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.166637+0000","last_change":"2024-09-10T18:58:52.890712+0000","last_active":"2024-09-10T19:03:33.166637+0000","last_peered":"2024-09-10T19:03:33.166637+0000","last_clean":"2024-09-10T19:03:33.166637+0000","last_became_active":"2024-09-10T18:45:20.058391+0000","last_became_peered":"2024-09-10T18:45:20.058391+0000","last_unstale":"2024-09-10T19:03:33.166637+0000","last_undegraded":"2024-09-10T19:03:33.166637+0000","last_fullsized":"2024-09-10T19:03:33.166637+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'18","last_scrub_stamp":"2024-09-10T18:58:52.890618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T18:58:52.890618+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016012082,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":411,"reported_epoch":385,"state":"active+clean","last_fresh":"2024-09-10T19:03:33.154022+0000","last_change":"2024-09-10T18:58:56.690660+0000","last_active":"2024-09-10T19:03:33.154022+0000","last_peered":"2024-09-10T19:03:33.154022+0000","last_clean":"2024-09-10T19:03:33.154022+0000","last_became_active":"2024-09-10T18:57:02.449135+0000","last_became_peered":"2024-09-10T18:57:02.449135+0000","last_unstale":"2024-09-10T19:03:33.154022+0000","last_undegraded":"2024-09-10T19:03:33.154022+0000","last_fullsized":"2024-09-10T19:03:33.154022+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:58:56.690606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T18:58:56.690606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060631599999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":16,"stat_sum":{"num_bytes":1484062720,"num_objects":22645,"num_object_clones":0,"num_object_copies":67935,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22645,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22645,"num_write_kb":1449280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4499636224,"data_stored":4499636224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22645,"ondisk_log_size":22645,"up":48,"acting":48,"num_store_stats":6},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":421,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1115,"num_read_kb":872,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":3152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":499712,"data_stored":13268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":766,"ondisk_log_size":766,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":159,"num_read_kb":154,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":608258,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":6,"num_write":3,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":298468,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1701,"ondisk_log_size":1701,"up":63,"acting":63,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":0,"seq":223338299598,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":43,"seq":184683593957,"num_pgs":88,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1710424,"kb_used_data":820432,"kb_used_omap":1,"kb_used_meta":889982,"kb_avail":92034728,"statfs":{"total":95995035648,"available":94243561472,"internally_reserved":0,"allocated":840122368,"data_stored":839671359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":911341912},"hb_peers":[0,1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:02:58 2024","interfaces":[{"interface":"back","average":{"1min":3.754,"5min":2.1970000000000001,"15min":1.3360000000000001},"min":{"1min":0.25700000000000001,"5min":0.23000000000000001,"15min":0.20899999999999999},"max":{"1min":34.018000000000001,"5min":49.353999999999999,"15min":49.353999999999999},"last":1.1910000000000001},{"interface":"front","average":{"1min":3.9220000000000002,"5min":2.234,"15min":1.4279999999999999},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":35.975000000000001,"5min":49.341999999999999,"15min":49.341999999999999},"last":1.3580000000000001}]},{"osd":1,"last update":"Tue Sep 10 19:03:13 2024","interfaces":[{"interface":"back","average":{"1min":2.234,"5min":1.96,"15min":1.4530000000000001},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":23.716000000000001,"5min":45.311,"15min":45.311},"last":1.4279999999999999},{"interface":"front","average":{"1min":2.21,"5min":1.8620000000000001,"15min":1.3280000000000001},"min":{"1min":0.33800000000000002,"5min":0.25900000000000001,"15min":0.214},"max":{"1min":23.594000000000001,"5min":45.027999999999999,"15min":45.027999999999999},"last":0.57699999999999996}]},{"osd":2,"last update":"Tue Sep 10 19:03:13 2024","interfaces":[{"interface":"back","average":{"1min":1.71,"5min":0.93200000000000005,"15min":0.89600000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":23.867000000000001,"5min":23.867000000000001,"15min":23.867000000000001},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.7050000000000001,"5min":0.94599999999999995,"15min":0.83999999999999997},"min":{"1min":0.29399999999999998,"5min":0.249,"15min":0.249},"max":{"1min":23.779,"5min":23.779,"15min":23.779},"last":1.611}]},{"osd":3,"last update":"Tue Sep 10 19:03:13 2024","interfaces":[{"interface":"back","average":{"1min":1.6100000000000001,"5min":2.6579999999999999,"15min":1.6699999999999999},"min":{"1min":0.33300000000000002,"5min":0.29599999999999999,"15min":0.25700000000000001},"max":{"1min":23.640999999999998,"5min":76.977999999999994,"15min":76.977999999999994},"last":0.70999999999999996},{"interface":"front","average":{"1min":1.724,"5min":2.6509999999999998,"15min":1.774},"min":{"1min":0.28499999999999998,"5min":0.27500000000000002,"15min":0.253},"max":{"1min":23.922000000000001,"5min":76.879999999999995,"15min":76.879999999999995},"last":1.831}]},{"osd":4,"last update":"Tue Sep 10 19:03:13 2024","interfaces":[{"interface":"back","average":{"1min":2.2829999999999999,"5min":1.1220000000000001,"15min":1.5109999999999999},"min":{"1min":0.27100000000000002,"5min":0.23400000000000001,"15min":0.20999999999999999},"max":{"1min":32.469000000000001,"5min":32.469000000000001,"15min":165.03299999999999},"last":1.153},{"interface":"front","average":{"1min":1.9299999999999999,"5min":0.94599999999999995,"15min":1.377},"min":{"1min":0.28999999999999998,"5min":0.26000000000000001,"15min":0.22500000000000001},"max":{"1min":32.5,"5min":32.5,"15min":163.959},"last":0.54500000000000004}]},{"osd":5,"last update":"Tue Sep 10 19:03:13 2024","interfaces":[{"interface":"back","average":{"1min":1.7729999999999999,"5min":3.3690000000000002,"15min":1.9039999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.24099999999999999},"max":{"1min":23.827000000000002,"5min":288.56999999999999,"15min":288.56999999999999},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.7010000000000001,"5min":3.27,"15min":1.891},"min":{"1min":0.28100000000000003,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":23.667000000000002,"5min":291.05399999999997,"15min":291.05399999999997},"last":1.2350000000000001}]},{"osd":7,"last update":"Tue Sep 10 19:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.67800000000000005,"15min":0.83499999999999996},"min":{"1min":0.32200000000000001,"5min":0.27900000000000003,"15min":0.26200000000000001},"max":{"1min":2.2839999999999998,"5min":2.98,"15min":5.3680000000000003},"last":0.5},{"interface":"front","average":{"1min":1.1379999999999999,"5min":0.72499999999999998,"15min":0.88900000000000001},"min":{"1min":0.31,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":4.6820000000000004,"5min":4.6820000000000004,"15min":7.0350000000000001},"last":0.39500000000000002}]}]},{"osd":5,"up_from":37,"seq":158913790185,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1425376,"kb_used_data":529232,"kb_used_omap":1,"kb_used_meta":896126,"kb_avail":92319776,"statfs":{"total":95995035648,"available":94535450624,"internally_reserved":0,"allocated":541933568,"data_stored":541481588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":917633368},"hb_peers":[0,1,2,3,4,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:02:51 2024","interfaces":[{"interface":"back","average":{"1min":2.1880000000000002,"5min":1.238,"15min":2.028},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":31.626999999999999,"5min":31.626999999999999,"15min":82.465000000000003},"last":1.5389999999999999},{"interface":"front","average":{"1min":2.5619999999999998,"5min":1.365,"15min":2.202},"min":{"1min":0.28799999999999998,"5min":0.218,"15min":0.218},"max":{"1min":31.734999999999999,"5min":31.734999999999999,"15min":82.519000000000005},"last":1.151}]},{"osd":1,"last update":"Tue Sep 10 19:02:51 2024","interfaces":[{"interface":"back","average":{"1min":2.4249999999999998,"5min":1.2190000000000001,"15min":1.6870000000000001},"min":{"1min":0.22500000000000001,"5min":0.20999999999999999,"15min":0.19500000000000001},"max":{"1min":31.657,"5min":31.657,"15min":67.582999999999998},"last":0.92600000000000005},{"interface":"front","average":{"1min":2.9430000000000001,"5min":1.3640000000000001,"15min":1.8819999999999999},"min":{"1min":0.34200000000000003,"5min":0.29699999999999999,"15min":0.20399999999999999},"max":{"1min":31.678999999999998,"5min":31.678999999999998,"15min":67.603999999999999},"last":0.93600000000000005}]},{"osd":2,"last update":"Tue Sep 10 19:02:51 2024","interfaces":[{"interface":"back","average":{"1min":2.4980000000000002,"5min":1.1319999999999999,"15min":0.92100000000000004},"min":{"1min":0.28699999999999998,"5min":0.27200000000000002,"15min":0.249},"max":{"1min":31.774000000000001,"5min":31.774000000000001,"15min":31.774000000000001},"last":0.312},{"interface":"front","average":{"1min":2.4049999999999998,"5min":1.123,"15min":0.93100000000000005},"min":{"1min":0.28199999999999997,"5min":0.27100000000000002,"15min":0.24399999999999999},"max":{"1min":31.722999999999999,"5min":31.722999999999999,"15min":31.722999999999999},"last":0.36499999999999999}]},{"osd":3,"last update":"Tue Sep 10 19:02:51 2024","interfaces":[{"interface":"back","average":{"1min":2.5489999999999999,"5min":1.2789999999999999,"15min":2.887},"min":{"1min":0.32300000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":31.753,"5min":31.753,"15min":123.34399999999999},"last":0.498},{"interface":"front","average":{"1min":2.5459999999999998,"5min":1.2490000000000001,"15min":2.4860000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.245},"max":{"1min":31.699999999999999,"5min":31.699999999999999,"15min":82.415000000000006},"last":0.57799999999999996}]},{"osd":4,"last update":"Tue Sep 10 19:02:51 2024","interfaces":[{"interface":"back","average":{"1min":3.9249999999999998,"5min":1.357,"15min":1.153},"min":{"1min":0.317,"5min":0.221,"15min":0.20899999999999999},"max":{"1min":51.070999999999998,"5min":51.070999999999998,"15min":51.070999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":3.7919999999999998,"5min":1.258,"15min":1.048},"min":{"1min":0.25,"5min":0.23899999999999999,"15min":0.23400000000000001},"max":{"1min":51.122999999999998,"5min":51.122999999999998,"15min":51.122999999999998},"last":0.252}]},{"osd":6,"last update":"Tue Sep 10 19:03:12 2024","interfaces":[{"interface":"back","average":{"1min":2.786,"5min":1.226,"15min":1.181},"min":{"1min":0.318,"5min":0.23999999999999999,"15min":0.23400000000000001},"max":{"1min":33.936999999999998,"5min":33.936999999999998,"15min":33.936999999999998},"last":0.48699999999999999},{"interface":"front","average":{"1min":2.8330000000000002,"5min":1.1850000000000001,"15min":1.1499999999999999},"min":{"1min":0.33100000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":33.951000000000001,"5min":33.951000000000001,"15min":33.951000000000001},"last":0.39100000000000001}]},{"osd":7,"last update":"Tue Sep 10 19:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":0.69399999999999995,"15min":0.94399999999999995},"min":{"1min":0.308,"5min":0.28999999999999998,"15min":0.27600000000000002},"max":{"1min":2.6859999999999999,"5min":3.665,"15min":10.782999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.61299999999999999,"15min":0.83199999999999996},"min":{"1min":0.34699999999999998,"5min":0.26900000000000002,"15min":0.23599999999999999},"max":{"1min":2.6200000000000001,"5min":2.6200000000000001,"15min":11.846},"last":0.33800000000000002}]}]},{"osd":4,"up_from":30,"seq":128849019119,"num_pgs":80,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1997464,"kb_used_data":1111304,"kb_used_omap":1,"kb_used_meta":886142,"kb_avail":91747688,"statfs":{"total":95995035648,"available":93949632512,"internally_reserved":0,"allocated":1137975296,"data_stored":1137531501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":907409752},"hb_peers":[0,1,2,3,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":1.603,"15min":1.446},"min":{"1min":0.23400000000000001,"5min":0.223,"15min":0.223},"max":{"1min":3.2749999999999999,"5min":46.234999999999999,"15min":46.234999999999999},"last":6.3819999999999997},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.5289999999999999,"15min":1.4430000000000001},"min":{"1min":0.33300000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":3.323,"5min":46.264000000000003,"15min":46.264000000000003},"last":6.6669999999999998}]},{"osd":1,"last update":"Tue Sep 10 19:03:35 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.82699999999999996,"15min":1.627},"min":{"1min":0.35799999999999998,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":3.5510000000000002,"5min":3.867,"15min":125.15600000000001},"last":1.6779999999999999},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72799999999999998,"15min":1.5329999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23100000000000001},"max":{"1min":3.2999999999999998,"5min":4.0069999999999997,"15min":125.14700000000001},"last":0.42699999999999999}]},{"osd":2,"last update":"Tue Sep 10 19:03:35 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.70999999999999996,"15min":0.71499999999999997},"min":{"1min":0.28499999999999998,"5min":0.26200000000000001,"15min":0.25900000000000001},"max":{"1min":3.3559999999999999,"5min":3.3999999999999999,"15min":5.6189999999999998},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.92400000000000004,"15min":0.82399999999999995},"min":{"1min":0.30199999999999999,"5min":0.28299999999999997,"15min":0.27100000000000002},"max":{"1min":2.427,"5min":12.782999999999999,"15min":12.782999999999999},"last":2.427}]},{"osd":3,"last update":"Tue Sep 10 19:03:35 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":1.5329999999999999,"15min":2.1699999999999999},"min":{"1min":0.317,"5min":0.28100000000000003,"15min":0.23799999999999999},"max":{"1min":2.431,"5min":34.676000000000002,"15min":111.514},"last":2.3889999999999998},{"interface":"front","average":{"1min":0.876,"5min":1.294,"15min":1.76},"min":{"1min":0.29199999999999998,"5min":0.26900000000000002,"15min":0.25700000000000001},"max":{"1min":3.3980000000000001,"5min":34.612000000000002,"15min":110.914},"last":1.786}]},{"osd":5,"last update":"Tue Sep 10 19:02:57 2024","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":0.91000000000000003,"15min":0.95899999999999996},"min":{"1min":0.316,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":6.1779999999999999,"5min":9.7309999999999999,"15min":10.912000000000001},"last":1.7090000000000001},{"interface":"front","average":{"1min":1.4890000000000001,"5min":0.91800000000000004,"15min":0.90900000000000003},"min":{"1min":0.29599999999999999,"5min":0.216,"15min":0.216},"max":{"1min":5.3849999999999998,"5min":10.147,"15min":10.147},"last":0.497}]},{"osd":6,"last update":"Tue Sep 10 19:03:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.71699999999999997,"15min":0.81399999999999995},"min":{"1min":0.254,"5min":0.221,"15min":0.221},"max":{"1min":5.351,"5min":5.351,"15min":8.4030000000000005},"last":0.73299999999999998},{"interface":"front","average":{"1min":1.151,"5min":1.081,"15min":1.0189999999999999},"min":{"1min":0.26700000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":4.3479999999999999,"5min":21.254000000000001,"15min":21.254000000000001},"last":1.8080000000000001}]},{"osd":7,"last update":"Tue Sep 10 19:01:46 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.60899999999999999,"15min":0.82999999999999996},"min":{"1min":0.40600000000000003,"5min":0.29099999999999998,"15min":0.27600000000000002},"max":{"1min":1.516,"5min":2.8879999999999999,"15min":6.4630000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":1.05,"5min":0.71099999999999997,"15min":0.83699999999999997},"min":{"1min":0.311,"5min":0.27500000000000002,"15min":0.251},"max":{"1min":3.1219999999999999,"5min":3.589,"15min":6.3949999999999996},"last":0.86399999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182644,"num_pgs":85,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1684508,"kb_used_data":786768,"kb_used_omap":1,"kb_used_meta":897726,"kb_avail":92060644,"statfs":{"total":95995035648,"available":94270099456,"internally_reserved":0,"allocated":805650432,"data_stored":805198343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":919271768},"hb_peers":[0,1,2,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:02:50 2024","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":4.5359999999999996,"15min":2.4159999999999999},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":3.2639999999999998,"5min":290.721,"15min":290.721},"last":1.9359999999999999},{"interface":"front","average":{"1min":1.2809999999999999,"5min":4.5880000000000001,"15min":2.5990000000000002},"min":{"1min":0.33000000000000002,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":4.774,"5min":293.59300000000002,"15min":293.59300000000002},"last":3.4620000000000002}]},{"osd":1,"last update":"Tue Sep 10 19:03:08 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":1.2849999999999999,"15min":1.6459999999999999},"min":{"1min":0.27200000000000002,"5min":0.20399999999999999,"15min":0.16200000000000001},"max":{"1min":2.7509999999999999,"5min":10.645,"15min":58.215000000000003},"last":1.9119999999999999},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.3500000000000001,"15min":1.5640000000000001},"min":{"1min":0.307,"5min":0.20000000000000001,"15min":0.188},"max":{"1min":4.8170000000000002,"5min":13.263999999999999,"15min":57.896999999999998},"last":1.208}]},{"osd":2,"last update":"Tue Sep 10 19:03:08 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":1.105,"15min":1.014},"min":{"1min":0.27800000000000002,"5min":0.255,"15min":0.22800000000000001},"max":{"1min":3.4430000000000001,"5min":5.6680000000000001,"15min":5.8220000000000001},"last":1.24},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.97299999999999998,"15min":0.92900000000000005},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":3.8410000000000002,"5min":5.46,"15min":7.0979999999999999},"last":0.48599999999999999}]},{"osd":4,"last update":"Tue Sep 10 19:02:35 2024","interfaces":[{"interface":"back","average":{"1min":1.587,"5min":1.3160000000000001,"15min":1.3220000000000001},"min":{"1min":0.39200000000000002,"5min":0.26400000000000001,"15min":0.247},"max":{"1min":6.5330000000000004,"5min":6.5330000000000004,"15min":29.013000000000002},"last":0.58499999999999996},{"interface":"front","average":{"1min":1.873,"5min":1.3420000000000001,"15min":1.3360000000000001},"min":{"1min":0.47099999999999997,"5min":0.28000000000000003,"15min":0.26800000000000002},"max":{"1min":6.6130000000000004,"5min":8.8130000000000006,"15min":28.826000000000001},"last":0.51600000000000001}]},{"osd":5,"last update":"Tue Sep 10 19:02:54 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.1419999999999999,"15min":1.278},"min":{"1min":0.27400000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":2.5379999999999998,"5min":7.2300000000000004,"15min":7.2300000000000004},"last":1.879},{"interface":"front","average":{"1min":2.1419999999999999,"5min":1.399,"15min":1.3260000000000001},"min":{"1min":0.38700000000000001,"5min":0.27100000000000002,"15min":0.255},"max":{"1min":17.899999999999999,"5min":17.899999999999999,"15min":17.899999999999999},"last":1.2769999999999999}]},{"osd":6,"last update":"Tue Sep 10 19:03:13 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":1.3340000000000001,"15min":1.347},"min":{"1min":0.39200000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":3.0049999999999999,"5min":5.4480000000000004,"15min":9.3239999999999998},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.22,"15min":1.3089999999999999},"min":{"1min":0.35799999999999998,"5min":0.34999999999999998,"15min":0.29399999999999998},"max":{"1min":3.3090000000000002,"5min":6.6390000000000002,"15min":7.1559999999999997},"last":0.74299999999999999}]},{"osd":7,"last update":"Tue Sep 10 19:01:40 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":1.5289999999999999,"15min":1.294},"min":{"1min":0.39700000000000002,"5min":0.35599999999999998,"15min":0.34899999999999998},"max":{"1min":2.7290000000000001,"5min":8.7989999999999995,"15min":8.7989999999999995},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":1.46,"15min":1.3140000000000001},"min":{"1min":0.42999999999999999,"5min":0.36899999999999999,"15min":0.33100000000000002},"max":{"1min":2.266,"5min":6.6029999999999998,"15min":9.2949999999999999},"last":0.71599999999999997}]}]},{"osd":2,"up_from":0,"seq":77309411576,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834575100,"num_pgs":93,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1922356,"kb_used_data":1031532,"kb_used_omap":1,"kb_used_meta":890814,"kb_avail":91822796,"statfs":{"total":95995035648,"available":94026543104,"internally_reserved":0,"allocated":1056288768,"data_stored":1055809470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":912193880},"hb_peers":[0,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:02:51 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":2.923,"15min":1.857},"min":{"1min":0.17999999999999999,"5min":0.158,"15min":0.158},"max":{"1min":2.5499999999999998,"5min":111.014,"15min":111.014},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":3.0920000000000001,"15min":1.845},"min":{"1min":0.27100000000000002,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":1.7609999999999999,"5min":111.072,"15min":111.072},"last":1.228}]},{"osd":2,"last update":"Tue Sep 10 19:02:39 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.79600000000000004,"15min":0.78400000000000003},"min":{"1min":0.30199999999999999,"5min":0.23300000000000001,"15min":0.186},"max":{"1min":2.3170000000000002,"5min":3.4900000000000002,"15min":4.1719999999999997},"last":1.2989999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.85099999999999998,"15min":0.82799999999999996},"min":{"1min":0.32500000000000001,"5min":0.23899999999999999,"15min":0.215},"max":{"1min":2.4489999999999998,"5min":4.2560000000000002,"15min":7.093},"last":0.46100000000000002}]},{"osd":3,"last update":"Tue Sep 10 19:03:06 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":2.278,"15min":2.149},"min":{"1min":0.29099999999999998,"5min":0.22800000000000001,"15min":0.185},"max":{"1min":4.9050000000000002,"5min":92.069000000000003,"15min":92.069000000000003},"last":1.2589999999999999},{"interface":"front","average":{"1min":0.81100000000000005,"5min":1.9830000000000001,"15min":1.9590000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.19900000000000001},"max":{"1min":3.5670000000000002,"5min":88.658000000000001,"15min":88.658000000000001},"last":1.6479999999999999}]},{"osd":4,"last update":"Tue Sep 10 19:03:30 2024","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.042,"15min":1.2190000000000001},"min":{"1min":0.33000000000000002,"5min":0.27800000000000002,"15min":0.26200000000000001},"max":{"1min":19.126999999999999,"5min":19.126999999999999,"15min":19.126999999999999},"last":0.41199999999999998},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.0740000000000001,"15min":1.133},"min":{"1min":0.37,"5min":0.28399999999999997,"15min":0.246},"max":{"1min":20.152000000000001,"5min":20.152000000000001,"15min":20.152000000000001},"last":1.268}]},{"osd":5,"last update":"Tue Sep 10 19:02:51 2024","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.1220000000000001,"15min":1.238},"min":{"1min":0.35299999999999998,"5min":0.29999999999999999,"15min":0.28499999999999998},"max":{"1min":2.1680000000000001,"5min":6.4690000000000003,"15min":10.334},"last":1.214},{"interface":"front","average":{"1min":1.04,"5min":1.0880000000000001,"15min":1.25},"min":{"1min":0.38,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":2.5030000000000001,"5min":7.0590000000000002,"15min":16.039999999999999},"last":0.433}]},{"osd":6,"last update":"Tue Sep 10 19:03:06 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.3029999999999999,"15min":1.587},"min":{"1min":0.36599999999999999,"5min":0.34100000000000003,"15min":0.308},"max":{"1min":4.5839999999999996,"5min":19.125,"15min":38.039999999999999},"last":1.1819999999999999},{"interface":"front","average":{"1min":2.077,"5min":1.3100000000000001,"15min":1.698},"min":{"1min":0.35699999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":17.352,"5min":17.352,"15min":38.100999999999999},"last":0.45300000000000001}]},{"osd":7,"last update":"Tue Sep 10 19:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":1.2330000000000001,"15min":1.163},"min":{"1min":0.39800000000000002,"5min":0.33100000000000002,"15min":0.315},"max":{"1min":4.1710000000000003,"5min":19.169,"15min":19.169},"last":2.581},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.995,"15min":1.0589999999999999},"min":{"1min":0.39600000000000002,"5min":0.36499999999999999,"15min":0.32800000000000001},"max":{"1min":2.2410000000000001,"5min":4.46,"15min":4.9029999999999996},"last":2.4510000000000001}]}]},{"osd":0,"up_from":241,"seq":1035087118432,"num_pgs":79,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1356472,"kb_used_data":726000,"kb_used_omap":4,"kb_used_meta":630459,"kb_avail":92388680,"statfs":{"total":95995035648,"available":94606008320,"internally_reserved":0,"allocated":743424000,"data_stored":742989935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":645590534},"hb_peers":[1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 19:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":1.038,"15min":2.2090000000000001},"min":{"1min":0.188,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":1.2569999999999999,"5min":6.9050000000000002,"15min":34.692},"last":1.478},{"interface":"front","average":{"1min":0.59199999999999997,"5min":1.0509999999999999,"15min":2.2040000000000002},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":1.45,"5min":7.5960000000000001,"15min":34.588999999999999},"last":2.0329999999999999}]},{"osd":2,"last update":"Tue Sep 10 19:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.77700000000000002,"15min":0.91100000000000003},"min":{"1min":0.23400000000000001,"5min":0.222,"15min":0.20699999999999999},"max":{"1min":0.85099999999999998,"5min":6.8639999999999999,"15min":13.305},"last":1.2809999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.81299999999999994,"15min":0.81399999999999995},"min":{"1min":0.253,"5min":0.23699999999999999,"15min":0.22800000000000001},"max":{"1min":3.0230000000000001,"5min":6.5899999999999999,"15min":13.693},"last":0.92200000000000004}]},{"osd":3,"last update":"Tue Sep 10 19:02:50 2024","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.2410000000000001,"15min":1.1519999999999999},"min":{"1min":0.25900000000000001,"5min":0.216,"15min":0.216},"max":{"1min":5.4829999999999997,"5min":7.2350000000000003,"15min":21.108000000000001},"last":2.153},{"interface":"front","average":{"1min":0.86699999999999999,"5min":1.3779999999999999,"15min":1.6379999999999999},"min":{"1min":0.28399999999999997,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":3.4689999999999999,"5min":7.5140000000000002,"15min":11.629},"last":2.4870000000000001}]},{"osd":4,"last update":"Tue Sep 10 19:02:50 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.083,"15min":0.89800000000000002},"min":{"1min":0.26400000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":2.5680000000000001,"5min":4.2830000000000004,"15min":13.161},"last":1.244},{"interface":"front","average":{"1min":0.88800000000000001,"5min":1.2070000000000001,"15min":1.034},"min":{"1min":0.318,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.9180000000000001,"5min":5.0490000000000004,"15min":5.0490000000000004},"last":0.42499999999999999}]},{"osd":5,"last update":"Tue Sep 10 19:02:50 2024","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":2.3500000000000001,"15min":1.4910000000000001},"min":{"1min":0.33200000000000002,"5min":0.31,"15min":0.31},"max":{"1min":2.8849999999999998,"5min":97.296999999999997,"15min":97.296999999999997},"last":0.47099999999999997},{"interface":"front","average":{"1min":1.395,"5min":2.1579999999999999,"15min":1.3149999999999999},"min":{"1min":0.28599999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":3.7930000000000001,"5min":99.221999999999994,"15min":99.221999999999994},"last":1.3660000000000001}]},{"osd":6,"last update":"Tue Sep 10 19:02:50 2024","interfaces":[{"interface":"back","average":{"1min":2.0750000000000002,"5min":1.454,"15min":1.1259999999999999},"min":{"1min":0.36699999999999999,"5min":0.312,"15min":0.312},"max":{"1min":14.537000000000001,"5min":14.537000000000001,"15min":14.537000000000001},"last":1.0329999999999999},{"interface":"front","average":{"1min":2.6000000000000001,"5min":1.472,"15min":1.0129999999999999},"min":{"1min":0.38500000000000001,"5min":0.313,"15min":0.313},"max":{"1min":18.931999999999999,"5min":18.931999999999999,"15min":18.931999999999999},"last":1.4159999999999999}]},{"osd":7,"last update":"Tue Sep 10 19:01:49 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.2509999999999999,"15min":0.96799999999999997},"min":{"1min":0.46899999999999997,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":4.3380000000000001,"5min":13.233000000000001,"15min":13.233000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":2.9430000000000001,"5min":1.494,"15min":1.262},"min":{"1min":0.40100000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":38.636000000000003,"5min":38.636000000000003,"15min":38.636000000000003},"last":0.76800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":2668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":500826112,"data_stored":500826112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1114112000,"data_stored":1114112000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":842072064,"data_stored":842072064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":582680576,"data_stored":582680576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":585367552,"data_stored":585367552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":874577920,"data_stored":874577920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T19:03:45.043 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd pool set .mgr pgp_num 21 2024-09-10T19:03:45.068 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 17 16 39930 39914 146.724 115.375 0.00677187 0.00681008 2024-09-10T19:03:45.072 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:45.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:45.073 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:45.174 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:45.221 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:45.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:45.223 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:45.324 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:45.361 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T19:03:45.361 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T19:03:45.374 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:45.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:45.375 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:45.402 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T19:03:45.403 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T19:03:45.439 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T19:03:45.439 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T19:03:45.472 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T19:03:45.472 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T19:03:45.477 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:45.505 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:45 smithi018 ceph-mon[29336]: 6.1e scrub starts 2024-09-10T19:03:45.505 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:45 smithi018 ceph-mon[29336]: 6.1e scrub ok 2024-09-10T19:03:45.505 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:45 smithi018 ceph-mon[29336]: 4.7 scrub starts 2024-09-10T19:03:45.506 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:45 smithi018 ceph-mon[29336]: 4.7 scrub ok 2024-09-10T19:03:45.506 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:45 smithi018 ceph-mon[40196]: 6.1e scrub starts 2024-09-10T19:03:45.506 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:45 smithi018 ceph-mon[40196]: 6.1e scrub ok 2024-09-10T19:03:45.506 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:45 smithi018 ceph-mon[40196]: 4.7 scrub starts 2024-09-10T19:03:45.506 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:45 smithi018 ceph-mon[40196]: 4.7 scrub ok 2024-09-10T19:03:45.506 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T19:03:45.507 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T19:03:45.515 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:45.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:45.516 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:45.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:45 smithi103 ceph-mon[33163]: 6.1e scrub starts 2024-09-10T19:03:45.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:45 smithi103 ceph-mon[33163]: 6.1e scrub ok 2024-09-10T19:03:45.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:45 smithi103 ceph-mon[33163]: 4.7 scrub starts 2024-09-10T19:03:45.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:45 smithi103 ceph-mon[33163]: 4.7 scrub ok 2024-09-10T19:03:45.538 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T19:03:45.538 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T19:03:45.574 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T19:03:45.574 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T19:03:45.575 INFO:teuthology.orchestra.run.smithi018.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-10T19:03:45.617 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:45.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:45.659 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:45.659 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:45.760 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:45.790 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:45.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:45.791 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:45.892 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:45.902 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-10T19:03:45.942 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:45.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:45.943 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:46.044 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:46.068 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 18 16 41576 41560 144.287 102.875 0.00431556 0.00692491 2024-09-10T19:03:46.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:46.085 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:46.085 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:46.186 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:46.241 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:46.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:46.242 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:46.343 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:46.392 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:46.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:46.393 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:46.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:46 smithi018 ceph-mon[40196]: 2.18 scrub starts 2024-09-10T19:03:46.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:46 smithi018 ceph-mon[40196]: 2.18 scrub ok 2024-09-10T19:03:46.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:46 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3791691431' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:03:46.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:46 smithi018 ceph-mon[40196]: pgmap v858: 168 pgs: 168 active+clean; 2.1 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 853 B/s rd, 103 MiB/s wr, 1.65k op/s 2024-09-10T19:03:46.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:46 smithi018 ceph-mon[29336]: 2.18 scrub starts 2024-09-10T19:03:46.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:46 smithi018 ceph-mon[29336]: 2.18 scrub ok 2024-09-10T19:03:46.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:46 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3791691431' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:03:46.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:46 smithi018 ceph-mon[29336]: pgmap v858: 168 pgs: 168 active+clean; 2.1 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 853 B/s rd, 103 MiB/s wr, 1.65k op/s 2024-09-10T19:03:46.494 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:46.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:46 smithi103 ceph-mon[33163]: 2.18 scrub starts 2024-09-10T19:03:46.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:46 smithi103 ceph-mon[33163]: 2.18 scrub ok 2024-09-10T19:03:46.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:46 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3791691431' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:03:46.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:46 smithi103 ceph-mon[33163]: pgmap v858: 168 pgs: 168 active+clean; 2.1 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 853 B/s rd, 103 MiB/s wr, 1.65k op/s 2024-09-10T19:03:46.536 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:46.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:46.538 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:46.638 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:46.640 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:03:46.643 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-10T19:03:46.643 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-10T19:03:46.696 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:03:46.696 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:46.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:46.698 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:46.798 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:46.847 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:46.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:46.848 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:46.949 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:47.000 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:47.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:47.001 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:47.068 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 19 16 43309 43293 142.393 108.312 0.00506912 0.00701795 2024-09-10T19:03:47.103 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:47.149 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:47.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:47.150 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:47.251 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:47.282 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:47.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:47.283 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:47.384 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:47.417 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:47.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:47.418 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:47.458 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:03:47.518 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:47.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:47 smithi103 ceph-mon[33163]: 4.a scrub starts 2024-09-10T19:03:47.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:47 smithi103 ceph-mon[33163]: 4.a scrub ok 2024-09-10T19:03:47.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:03:47 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:03:47.552 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:47.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:47.553 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:47.583 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:47 smithi018 ceph-mon[29336]: 4.a scrub starts 2024-09-10T19:03:47.583 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:47 smithi018 ceph-mon[29336]: 4.a scrub ok 2024-09-10T19:03:47.583 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:47 smithi018 ceph-mon[40196]: 4.a scrub starts 2024-09-10T19:03:47.583 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:47 smithi018 ceph-mon[40196]: 4.a scrub ok 2024-09-10T19:03:47.654 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:47.686 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:47.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:47.687 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:47.788 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:47.831 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:47.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:47.832 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:47.933 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:47.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:47.989 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:47.989 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:48.068 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:2024-09-10T19:03:48.067885+0000 min lat: 0.00116366 max lat: 0.111048 avg lat: 0.00708811 2024-09-10T19:03:48.069 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T19:03:48.069 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 20 16 45134 45118 140.976 114.062 0.00733248 0.00708811 2024-09-10T19:03:48.090 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:48.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:48.134 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:48.134 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:48.235 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:48.279 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:48.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:48.280 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:48.381 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:48.410 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:48.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:48.412 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:48.513 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:48.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:48 smithi103 ceph-mon[33163]: 5.1c scrub starts 2024-09-10T19:03:48.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:48 smithi103 ceph-mon[33163]: 5.1c scrub ok 2024-09-10T19:03:48.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:48 smithi103 ceph-mon[33163]: 6.b deep-scrub starts 2024-09-10T19:03:48.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:48 smithi103 ceph-mon[33163]: 6.b deep-scrub ok 2024-09-10T19:03:48.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:48 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:48.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:48 smithi103 ceph-mon[33163]: pgmap v859: 168 pgs: 168 active+clean; 2.5 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 1.4 KiB/s rd, 151 MiB/s wr, 2.42k op/s 2024-09-10T19:03:48.541 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:48.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:48.543 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:48.643 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:48.644 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T19:03:48.690 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:48.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:48.691 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:48.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:48 smithi018 ceph-mon[40196]: 5.1c scrub starts 2024-09-10T19:03:48.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:48 smithi018 ceph-mon[40196]: 5.1c scrub ok 2024-09-10T19:03:48.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:48 smithi018 ceph-mon[40196]: 6.b deep-scrub starts 2024-09-10T19:03:48.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:48 smithi018 ceph-mon[40196]: 6.b deep-scrub ok 2024-09-10T19:03:48.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:48 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:48.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:48 smithi018 ceph-mon[40196]: pgmap v859: 168 pgs: 168 active+clean; 2.5 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 1.4 KiB/s rd, 151 MiB/s wr, 2.42k op/s 2024-09-10T19:03:48.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:48 smithi018 ceph-mon[29336]: 5.1c scrub starts 2024-09-10T19:03:48.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:48 smithi018 ceph-mon[29336]: 5.1c scrub ok 2024-09-10T19:03:48.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:48 smithi018 ceph-mon[29336]: 6.b deep-scrub starts 2024-09-10T19:03:48.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:48 smithi018 ceph-mon[29336]: 6.b deep-scrub ok 2024-09-10T19:03:48.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:48 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:48.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:48 smithi018 ceph-mon[29336]: pgmap v859: 168 pgs: 168 active+clean; 2.5 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 1.4 KiB/s rd, 151 MiB/s wr, 2.42k op/s 2024-09-10T19:03:48.792 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:48.843 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:48.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:48.844 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:48.945 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:48.967 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:48.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:48.968 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:49.068 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 21 16 46696 46680 138.911 97.625 0.00509616 0.00719377 2024-09-10T19:03:49.068 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:49.114 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:49.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:49.115 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:49.215 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:49.247 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:49.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:49.248 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:49.349 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:49.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:49.389 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:49.389 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:49.490 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:49.511 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:49.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:49.512 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:49.612 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:49.641 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:49.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:49.642 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:49.743 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:49.776 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:49.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:49.777 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:49.877 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:49.910 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:49.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:49.911 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:50.012 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:50.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:50.042 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:50.042 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:50.068 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 22 16 48390 48374 137.409 105.875 0.00785315 0.00727237 2024-09-10T19:03:50.143 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:50.173 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:50.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:50.175 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:50.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:49 smithi018 ceph-mon[40196]: osdmap e391: 8 total, 7 up, 6 in 2024-09-10T19:03:50.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:49 smithi018 ceph-mon[40196]: 5.b scrub starts 2024-09-10T19:03:50.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:49 smithi018 ceph-mon[40196]: 5.b scrub ok 2024-09-10T19:03:50.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:49 smithi018 ceph-mon[40196]: pgmap v861: 168 pgs: 168 active+clean; 2.6 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 823 B/s rd, 99 MiB/s wr, 1.59k op/s 2024-09-10T19:03:50.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:49 smithi018 ceph-mon[29336]: osdmap e391: 8 total, 7 up, 6 in 2024-09-10T19:03:50.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:49 smithi018 ceph-mon[29336]: 5.b scrub starts 2024-09-10T19:03:50.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:49 smithi018 ceph-mon[29336]: 5.b scrub ok 2024-09-10T19:03:50.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:49 smithi018 ceph-mon[29336]: pgmap v861: 168 pgs: 168 active+clean; 2.6 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 823 B/s rd, 99 MiB/s wr, 1.59k op/s 2024-09-10T19:03:50.238 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:49 smithi103 ceph-mon[33163]: osdmap e391: 8 total, 7 up, 6 in 2024-09-10T19:03:50.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:49 smithi103 ceph-mon[33163]: 5.b scrub starts 2024-09-10T19:03:50.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:49 smithi103 ceph-mon[33163]: 5.b scrub ok 2024-09-10T19:03:50.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:49 smithi103 ceph-mon[33163]: pgmap v861: 168 pgs: 168 active+clean; 2.6 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 823 B/s rd, 99 MiB/s wr, 1.59k op/s 2024-09-10T19:03:50.276 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:50.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:50.312 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:50.312 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:50.413 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:50.436 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:50.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:50.437 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:50.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:03:50 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:03:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:03:50.538 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:50.567 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:50.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:50.568 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:50.668 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:50.707 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:50.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:50.708 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:50.809 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:50.837 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:50.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:50.838 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:50.939 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:50.987 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:50.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:50.988 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:51.068 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 23 16 49884 49868 135.495 93.375 0.00926412 0.00737463 2024-09-10T19:03:51.090 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:51.119 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:51.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:51.120 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:51.158 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:50 smithi018 ceph-mon[40196]: 5.14 deep-scrub starts 2024-09-10T19:03:51.158 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:50 smithi018 ceph-mon[40196]: 5.14 deep-scrub ok 2024-09-10T19:03:51.158 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:50 smithi018 ceph-mon[40196]: 1.10 scrub starts 2024-09-10T19:03:51.158 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:50 smithi018 ceph-mon[40196]: 1.10 scrub ok 2024-09-10T19:03:51.159 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:50 smithi018 ceph-mon[29336]: 5.14 deep-scrub starts 2024-09-10T19:03:51.159 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:50 smithi018 ceph-mon[29336]: 5.14 deep-scrub ok 2024-09-10T19:03:51.159 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:50 smithi018 ceph-mon[29336]: 1.10 scrub starts 2024-09-10T19:03:51.159 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:50 smithi018 ceph-mon[29336]: 1.10 scrub ok 2024-09-10T19:03:51.159 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:03:50 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:03:50] "GET /metrics HTTP/1.1" 200 209561 "" "Prometheus/2.43.0" 2024-09-10T19:03:51.221 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:51.258 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:51.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:51.259 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:51.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:50 smithi103 ceph-mon[33163]: 5.14 deep-scrub starts 2024-09-10T19:03:51.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:50 smithi103 ceph-mon[33163]: 5.14 deep-scrub ok 2024-09-10T19:03:51.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:50 smithi103 ceph-mon[33163]: 1.10 scrub starts 2024-09-10T19:03:51.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:50 smithi103 ceph-mon[33163]: 1.10 scrub ok 2024-09-10T19:03:51.360 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:51.397 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:51.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:51.398 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:51.499 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:51.536 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:51.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:51.537 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:51.638 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:51.666 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:51.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:51.667 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:51.768 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:51.794 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:51.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:51.795 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:51.896 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:51.918 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:51.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:51.919 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:52.019 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:52.056 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:52.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:52.057 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:52.068 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 24 16 51283 51267 133.492 87.4375 0.0135634 0.00748569 2024-09-10T19:03:52.158 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:52.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:51 smithi018 ceph-mon[29336]: 4.17 scrub starts 2024-09-10T19:03:52.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:51 smithi018 ceph-mon[29336]: 4.17 scrub ok 2024-09-10T19:03:52.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:51 smithi018 ceph-mon[29336]: 6.7 scrub starts 2024-09-10T19:03:52.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:51 smithi018 ceph-mon[29336]: 6.7 scrub ok 2024-09-10T19:03:52.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:51 smithi018 ceph-mon[29336]: pgmap v862: 168 pgs: 168 active+clean; 2.8 GiB data, 14 GiB used, 523 GiB / 536 GiB avail; 639 B/s rd, 101 MiB/s wr, 1.61k op/s 2024-09-10T19:03:52.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:51 smithi018 ceph-mon[40196]: 4.17 scrub starts 2024-09-10T19:03:52.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:51 smithi018 ceph-mon[40196]: 4.17 scrub ok 2024-09-10T19:03:52.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:51 smithi018 ceph-mon[40196]: 6.7 scrub starts 2024-09-10T19:03:52.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:51 smithi018 ceph-mon[40196]: 6.7 scrub ok 2024-09-10T19:03:52.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:51 smithi018 ceph-mon[40196]: pgmap v862: 168 pgs: 168 active+clean; 2.8 GiB data, 14 GiB used, 523 GiB / 536 GiB avail; 639 B/s rd, 101 MiB/s wr, 1.61k op/s 2024-09-10T19:03:52.195 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:52.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:52.197 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:52.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:51 smithi103 ceph-mon[33163]: 4.17 scrub starts 2024-09-10T19:03:52.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:51 smithi103 ceph-mon[33163]: 4.17 scrub ok 2024-09-10T19:03:52.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:51 smithi103 ceph-mon[33163]: 6.7 scrub starts 2024-09-10T19:03:52.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:51 smithi103 ceph-mon[33163]: 6.7 scrub ok 2024-09-10T19:03:52.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:51 smithi103 ceph-mon[33163]: pgmap v862: 168 pgs: 168 active+clean; 2.8 GiB data, 14 GiB used, 523 GiB / 536 GiB avail; 639 B/s rd, 101 MiB/s wr, 1.61k op/s 2024-09-10T19:03:52.297 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:52.334 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:52.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:52.335 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:52.436 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:52.476 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:52.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:52.477 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:52.578 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:52.609 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:52.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:52.610 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:52.710 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:52.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:52.753 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:52.753 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:52.809 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is set 2024-09-10T19:03:52.855 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:52.885 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:52.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:52.886 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:52.931 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:52 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:03:52.931 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:52 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:03:52.931 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:52 smithi018 ceph-mon[40196]: 2.d scrub starts 2024-09-10T19:03:52.932 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:52 smithi018 ceph-mon[40196]: 2.d scrub ok 2024-09-10T19:03:52.932 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:52 smithi018 ceph-mon[40196]: osdmap e392: 8 total, 7 up, 6 in 2024-09-10T19:03:52.932 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:52 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3605980943' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T19:03:52.932 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:52 smithi018 ceph-mon[40196]: 4.2 scrub starts 2024-09-10T19:03:52.932 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:52 smithi018 ceph-mon[40196]: 4.2 scrub ok 2024-09-10T19:03:52.932 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:52 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:03:52.932 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:52 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:03:52.932 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:52 smithi018 ceph-mon[29336]: 2.d scrub starts 2024-09-10T19:03:52.932 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:52 smithi018 ceph-mon[29336]: 2.d scrub ok 2024-09-10T19:03:52.932 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:52 smithi018 ceph-mon[29336]: osdmap e392: 8 total, 7 up, 6 in 2024-09-10T19:03:52.932 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:52 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3605980943' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T19:03:52.932 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:52 smithi018 ceph-mon[29336]: 4.2 scrub starts 2024-09-10T19:03:52.933 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:52 smithi018 ceph-mon[29336]: 4.2 scrub ok 2024-09-10T19:03:52.987 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:53.028 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:53.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:53.030 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:53.068 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 25 15 52743 52728 131.805 91.3125 0.0203172 0.0075818 2024-09-10T19:03:53.130 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:53.161 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:53.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:53.163 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:53.264 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:53.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:52 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:03:53.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:52 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:03:53.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:52 smithi103 ceph-mon[33163]: 2.d scrub starts 2024-09-10T19:03:53.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:52 smithi103 ceph-mon[33163]: 2.d scrub ok 2024-09-10T19:03:53.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:52 smithi103 ceph-mon[33163]: osdmap e392: 8 total, 7 up, 6 in 2024-09-10T19:03:53.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:52 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3605980943' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T19:03:53.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:52 smithi103 ceph-mon[33163]: 4.2 scrub starts 2024-09-10T19:03:53.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:52 smithi103 ceph-mon[33163]: 4.2 scrub ok 2024-09-10T19:03:53.306 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:53.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:53.308 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:53.408 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:53.436 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:53.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:53.437 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:53.538 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:53.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:53.579 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:53.579 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:53.680 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:53.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:53.724 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:53.724 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:53.825 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:53.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:53.856 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:53.856 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:53.957 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:53.988 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:53.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:53.990 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:54.001 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:53 smithi018 ceph-mon[29336]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:03:54.001 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:53 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3605980943' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T19:03:54.001 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:53 smithi018 ceph-mon[29336]: osdmap e393: 8 total, 7 up, 6 in 2024-09-10T19:03:54.001 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:53 smithi018 ceph-mon[29336]: 1.b scrub starts 2024-09-10T19:03:54.001 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:53 smithi018 ceph-mon[29336]: pgmap v865: 168 pgs: 168 active+clean; 3.0 GiB data, 15 GiB used, 522 GiB / 536 GiB avail; 852 B/s rd, 87 MiB/s wr, 1.39k op/s 2024-09-10T19:03:54.002 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:53 smithi018 ceph-mon[40196]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:03:54.002 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:53 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3605980943' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T19:03:54.002 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:53 smithi018 ceph-mon[40196]: osdmap e393: 8 total, 7 up, 6 in 2024-09-10T19:03:54.002 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:53 smithi018 ceph-mon[40196]: 1.b scrub starts 2024-09-10T19:03:54.002 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:53 smithi018 ceph-mon[40196]: pgmap v865: 168 pgs: 168 active+clean; 3.0 GiB data, 15 GiB used, 522 GiB / 536 GiB avail; 852 B/s rd, 87 MiB/s wr, 1.39k op/s 2024-09-10T19:03:54.068 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 26 16 54388 54372 130.687 102.75 0.0167311 0.00764613 2024-09-10T19:03:54.090 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:54.132 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:54.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:54.133 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:54.234 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:54.270 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:54.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:54.271 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:54.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:53 smithi103 ceph-mon[33163]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:03:54.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:53 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3605980943' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T19:03:54.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:53 smithi103 ceph-mon[33163]: osdmap e393: 8 total, 7 up, 6 in 2024-09-10T19:03:54.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:53 smithi103 ceph-mon[33163]: 1.b scrub starts 2024-09-10T19:03:54.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:53 smithi103 ceph-mon[33163]: pgmap v865: 168 pgs: 168 active+clean; 3.0 GiB data, 15 GiB used, 522 GiB / 536 GiB avail; 852 B/s rd, 87 MiB/s wr, 1.39k op/s 2024-09-10T19:03:54.372 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:54.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:54.403 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:54.403 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:54.504 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:54.547 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:54.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:54.548 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:54.649 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:54.679 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:54.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:54.680 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:54.781 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:54.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:54.823 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:54.823 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:54.924 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:54.967 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:03:55.022 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:55.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:55.023 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:55.069 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 27 16 55819 55803 129.156 89.4375 0.00952099 0.0077358 2024-09-10T19:03:55.124 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:55.196 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:55.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:55.198 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:55.299 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:55.329 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:55.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:55.330 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:55.361 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:03:55 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T19:03:55.106+0000 7f66d46e1640 -1 mgr get_metadata_python Requested missing service osd.7 2024-09-10T19:03:55.431 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:55.471 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:55.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:55.472 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:55.573 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:55.603 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:55.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:55.605 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:55.706 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:55.731 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:55.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:55.732 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:55.833 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:55.872 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:55.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:55.873 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:55.974 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:56.014 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:56.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:56.015 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:56.070 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 28 16 57353 57337 127.967 95.875 0.0112843 0.00780874 2024-09-10T19:03:56.116 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:56.123 INFO:teuthology.orchestra.run.smithi018.stderr:set pool 1 pgp_num to 21 2024-09-10T19:03:56.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:56.169 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:56.169 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:56.270 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:56.288 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:56 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:03:56.289 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:56 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:03:56.289 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:56 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1244724581' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "21"}]: dispatch 2024-09-10T19:03:56.289 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:56 smithi018 ceph-mon[40196]: pgmap v866: 168 pgs: 168 active+clean; 3.1 GiB data, 15 GiB used, 522 GiB / 536 GiB avail; 767 B/s rd, 88 MiB/s wr, 1.40k op/s 2024-09-10T19:03:56.289 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:56 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:03:56.289 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:56 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:03:56.289 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:56 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1244724581' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "21"}]: dispatch 2024-09-10T19:03:56.289 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:56 smithi018 ceph-mon[29336]: pgmap v866: 168 pgs: 168 active+clean; 3.1 GiB data, 15 GiB used, 522 GiB / 536 GiB avail; 767 B/s rd, 88 MiB/s wr, 1.40k op/s 2024-09-10T19:03:56.309 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:56.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:56.310 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:56.411 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:56.448 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:56.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:56.449 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:56.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:56 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:03:56.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:56 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:03:56.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:56 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1244724581' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "21"}]: dispatch 2024-09-10T19:03:56.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:56 smithi103 ceph-mon[33163]: pgmap v866: 168 pgs: 168 active+clean; 3.1 GiB data, 15 GiB used, 522 GiB / 536 GiB avail; 767 B/s rd, 88 MiB/s wr, 1.40k op/s 2024-09-10T19:03:56.550 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:56.582 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:56.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:56.583 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:56.684 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:56.691 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-10T19:03:56.713 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:56.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:56.714 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:56.815 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:56.867 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:56.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:56.868 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:56.969 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:57.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:57.022 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:57.022 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:57.069 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 29 16 58806 58790 126.685 90.8125 0.011341 0.00788897 2024-09-10T19:03:57.123 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:57.151 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:57.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:57.152 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:57.253 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:57.281 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:57.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:57.282 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:57.382 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:57.424 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:57.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:57.427 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:57.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:57 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1244724581' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "21"}]': finished 2024-09-10T19:03:57.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:57 smithi018 ceph-mon[40196]: osdmap e394: 8 total, 7 up, 6 in 2024-09-10T19:03:57.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:57 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1244724581' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "21"}]': finished 2024-09-10T19:03:57.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:57 smithi018 ceph-mon[29336]: osdmap e394: 8 total, 7 up, 6 in 2024-09-10T19:03:57.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:03:57 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:03:57.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:57 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1244724581' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "21"}]': finished 2024-09-10T19:03:57.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:57 smithi103 ceph-mon[33163]: osdmap e394: 8 total, 7 up, 6 in 2024-09-10T19:03:57.528 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:57.568 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:57.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:57.569 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:57.670 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:57.691 INFO:teuthology.orchestra.run.smithi018.stdout:ERROR: (22) Invalid argument 2024-09-10T19:03:57.691 INFO:teuthology.orchestra.run.smithi018.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T19:03:57.713 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:57.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:57.714 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:57.815 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:03:57.858 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:57.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:57.859 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:03:57.960 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:57.992 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:57.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:57.993 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:58.070 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 30 16 60213 60197 125.393 87.9375 0.00953182 0.00796955 2024-09-10T19:03:58.094 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:58.133 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:58.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:58.135 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:58.236 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:03:58.256 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:58.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:58.257 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:03:58.358 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:58.396 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:58.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:58.397 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:58.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:58 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:58.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:58 smithi018 ceph-mon[40196]: pgmap v868: 168 pgs: 168 active+clean; 3.5 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 1.7 KiB/s rd, 122 MiB/s wr, 1.95k op/s 2024-09-10T19:03:58.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:58 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:58.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:58 smithi018 ceph-mon[29336]: pgmap v868: 168 pgs: 168 active+clean; 3.5 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 1.7 KiB/s rd, 122 MiB/s wr, 1.95k op/s 2024-09-10T19:03:58.498 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:58.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:58 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:58.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:58 smithi103 ceph-mon[33163]: pgmap v868: 168 pgs: 168 active+clean; 3.5 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 1.7 KiB/s rd, 122 MiB/s wr, 1.95k op/s 2024-09-10T19:03:58.538 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:58.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:58.539 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:58.640 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:03:58.678 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:58.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:58.679 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:03:58.779 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:58.813 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:58.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:58.814 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:58.914 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:58.951 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:58.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:58.952 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:59.053 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:59.070 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 31 16 61576 61560 124.096 85.1875 0.00426653 0.0080532 2024-09-10T19:03:59.093 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:59.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:59.095 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:59.195 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:59.233 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:59.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:59.235 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:59.335 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:03:59.369 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:59.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:59.370 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:03:59.471 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:59.495 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:59.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:59.496 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:03:59.597 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:03:59.640 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:59.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:59.641 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:03:59.742 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:03:59.786 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:03:59.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:59.787 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:03:59.887 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:03:59.915 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:03:59.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:03:59.917 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:00.017 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:00.049 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:00.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:00.050 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:00.070 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 32 16 62867 62851 122.739 80.6875 0.00682288 0.00814208 2024-09-10T19:04:00.151 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:00.193 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:00.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:00.194 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:00.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:59 smithi018 ceph-mon[40196]: osdmap e395: 8 total, 7 up, 6 in 2024-09-10T19:04:00.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:03:59 smithi018 ceph-mon[40196]: pgmap v870: 168 pgs: 168 active+clean; 3.6 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 771 B/s rd, 79 MiB/s wr, 1.26k op/s 2024-09-10T19:04:00.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:59 smithi018 ceph-mon[29336]: osdmap e395: 8 total, 7 up, 6 in 2024-09-10T19:04:00.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:03:59 smithi018 ceph-mon[29336]: pgmap v870: 168 pgs: 168 active+clean; 3.6 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 771 B/s rd, 79 MiB/s wr, 1.26k op/s 2024-09-10T19:04:00.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:59 smithi103 ceph-mon[33163]: osdmap e395: 8 total, 7 up, 6 in 2024-09-10T19:04:00.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:03:59 smithi103 ceph-mon[33163]: pgmap v870: 168 pgs: 168 active+clean; 3.6 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 771 B/s rd, 79 MiB/s wr, 1.26k op/s 2024-09-10T19:04:00.295 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:00.331 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:00.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:00.332 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:00.433 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:00.462 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:00.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:00.463 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:00.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:04:00 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:04:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:04:00.564 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:00.601 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:00.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:00.602 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:00.703 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:00.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:00.745 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:00.745 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:00.846 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:00.889 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:00.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:00.890 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:00.990 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:01.031 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:01.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:01.032 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:01.070 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 33 16 64133 64117 121.417 79.125 0.0201211 0.00823009 2024-09-10T19:04:01.086 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T19:04:01.086 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T19:04:01.086 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:00 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:04:01.087 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:00 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:04:01.087 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:00 smithi018 ceph-mon[40196]: osdmap e396: 8 total, 7 up, 6 in 2024-09-10T19:04:01.087 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:00 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:04:01.087 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:00 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:04:01.087 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:00 smithi018 ceph-mon[29336]: osdmap e396: 8 total, 7 up, 6 in 2024-09-10T19:04:01.087 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:04:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:04:00] "GET /metrics HTTP/1.1" 200 209614 "" "Prometheus/2.43.0" 2024-09-10T19:04:01.117 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T19:04:01.117 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T19:04:01.133 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:01.152 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T19:04:01.152 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T19:04:01.159 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:01.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:01.160 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:01.182 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T19:04:01.183 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T19:04:01.213 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T19:04:01.213 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T19:04:01.245 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T19:04:01.245 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T19:04:01.261 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:01.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:00 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:04:01.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:00 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:04:01.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:00 smithi103 ceph-mon[33163]: osdmap e396: 8 total, 7 up, 6 in 2024-09-10T19:04:01.279 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T19:04:01.279 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T19:04:01.281 INFO:teuthology.orchestra.run.smithi018.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-10T19:04:01.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:01.304 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:01.304 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:01.405 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:01.445 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:01.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:01.446 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:01.547 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:01.579 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:01.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:01.580 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:01.681 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:01.722 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:01.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:01.723 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:01.824 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:01.866 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:01.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:01.867 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:01.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:01 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1088460310' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:04:01.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:01 smithi018 ceph-mon[40196]: pgmap v872: 168 pgs: 168 active+clean; 3.7 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 853 B/s rd, 103 MiB/s wr, 1.65k op/s 2024-09-10T19:04:01.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:01 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1088460310' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:04:01.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:01 smithi018 ceph-mon[29336]: pgmap v872: 168 pgs: 168 active+clean; 3.7 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 853 B/s rd, 103 MiB/s wr, 1.65k op/s 2024-09-10T19:04:01.968 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:01.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:01.999 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:01.999 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:02.070 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 34 16 65541 65525 120.434 88 0.00835413 0.00829761 2024-09-10T19:04:02.100 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:02.135 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:02.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:02.137 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:02.237 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:02.267 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:02.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:02.268 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:02.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:01 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1088460310' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:04:02.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:01 smithi103 ceph-mon[33163]: pgmap v872: 168 pgs: 168 active+clean; 3.7 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 853 B/s rd, 103 MiB/s wr, 1.65k op/s 2024-09-10T19:04:02.369 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:02.397 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:02.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:02.398 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:02.429 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-10T19:04:02.429 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-10T19:04:02.498 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:02.520 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:02.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:02.521 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:02.622 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:02.643 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:02.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:02.644 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:02.745 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:02.783 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:02.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:02.784 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:02.885 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:02.915 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:04:02.944 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:02.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:02.945 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:02.954 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-10T19:04:03.047 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:03.070 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 35 16 66647 66631 118.968 69.125 0.0132749 0.00840005 2024-09-10T19:04:03.075 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:03.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:03.076 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:03.177 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:03.246 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:03.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:03.247 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:03.348 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:03.376 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:03.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:03.377 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:03.478 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:03.522 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:03.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:03.523 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:03.624 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:03.665 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:03.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:03.666 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:03.767 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:03.788 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:03.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:03.789 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:03.889 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:03.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:03.942 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:03.942 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:04.043 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:04.065 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:04.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:04.066 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:04.070 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 36 16 67842 67826 117.738 74.6875 0.0317396 0.00848154 2024-09-10T19:04:04.167 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:04.209 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:04.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:04.210 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:04.311 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:04.339 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:04.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:04.341 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:04.430 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T19:04:04.442 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:04.495 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:04.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:04.496 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:04.597 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:04.629 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:04.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:04.630 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:04.731 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:04.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:04 smithi103 ceph-mon[33163]: pgmap v873: 168 pgs: 168 active+clean; 3.9 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 1.4 KiB/s rd, 113 MiB/s wr, 1.81k op/s 2024-09-10T19:04:04.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:04 smithi103 ceph-mon[33163]: osdmap e397: 8 total, 7 up, 6 in 2024-09-10T19:04:04.777 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:04.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:04.779 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:04.880 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:04.909 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:04.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:04.910 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:04.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:04 smithi018 ceph-mon[29336]: pgmap v873: 168 pgs: 168 active+clean; 3.9 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 1.4 KiB/s rd, 113 MiB/s wr, 1.81k op/s 2024-09-10T19:04:04.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:04 smithi018 ceph-mon[29336]: osdmap e397: 8 total, 7 up, 6 in 2024-09-10T19:04:04.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:04 smithi018 ceph-mon[40196]: pgmap v873: 168 pgs: 168 active+clean; 3.9 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 1.4 KiB/s rd, 113 MiB/s wr, 1.81k op/s 2024-09-10T19:04:04.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:04 smithi018 ceph-mon[40196]: osdmap e397: 8 total, 7 up, 6 in 2024-09-10T19:04:05.011 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:05.042 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:05.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:05.043 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:05.070 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 37 16 69102 69086 116.684 78.75 0.00439911 0.00855456 2024-09-10T19:04:05.143 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:05.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:05.188 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:05.188 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:05.289 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:05.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:05.334 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:05.334 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:05.435 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:05.473 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:05.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:05.477 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:05.577 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:05.617 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:05.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:05.618 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:05.719 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:05.755 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:05.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:05.757 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:05.857 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:05.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:05.896 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:05.896 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:05.997 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:06.035 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:06.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:06.037 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:06.071 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 38 16 70462 70446 115.85 85 0.00914685 0.0086263 2024-09-10T19:04:06.137 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:06.165 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:06.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:06.166 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:06.265 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:04:06.266 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:06.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:06.329 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:06.329 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:06.430 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:06.481 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:06.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:06.482 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:06.583 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:06.609 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:06.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:06.610 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:06.663 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6] out_osds: [2, 7] dead_osds: [7] live_osds: [1, 2, 3, 4, 5, 6, 0] 2024-09-10T19:04:06.663 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T19:04:06.663 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_4 2024-09-10T19:04:06.663 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T19:04:06.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:06 smithi018 ceph-mon[29336]: pgmap v875: 168 pgs: 168 active+clean; 4.0 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 767 B/s rd, 71 MiB/s wr, 1.13k op/s 2024-09-10T19:04:06.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:06 smithi018 ceph-mon[40196]: pgmap v875: 168 pgs: 168 active+clean; 4.0 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 767 B/s rd, 71 MiB/s wr, 1.13k op/s 2024-09-10T19:04:06.711 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:06.737 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:06.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:06.738 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:06.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:06 smithi103 ceph-mon[33163]: pgmap v875: 168 pgs: 168 active+clean; 4.0 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 767 B/s rd, 71 MiB/s wr, 1.13k op/s 2024-09-10T19:04:06.839 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:06.865 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:06.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:06.866 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:06.967 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:07.020 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:07.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:07.021 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:07.071 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 39 16 71717 71701 114.89 78.4375 0.00679933 0.00869877 2024-09-10T19:04:07.122 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:07.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:07.144 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:07.144 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:07.245 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:07.266 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:07.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:07.267 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:07.368 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:07.390 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:07.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:07.391 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:07.492 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:07.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:07.522 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:07.522 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:07.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:04:07 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:04:07.623 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:07.662 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:07.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:07.663 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:07.764 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:07.796 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:07.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:07.797 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:07.898 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:07.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:07.939 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:07.940 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:08.041 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:08.071 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:2024-09-10T19:04:08.070711+0000 min lat: 0.00116366 max lat: 0.294449 avg lat: 0.00879224 2024-09-10T19:04:08.071 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T19:04:08.071 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 40 16 72778 72762 113.676 66.3125 0.0109154 0.00879224 2024-09-10T19:04:08.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:08.072 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:08.072 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:08.173 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:08.218 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:08.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:08.219 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:08.320 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:08.364 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:08.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:08.365 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:08.466 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:08.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:08.528 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:08.528 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:08.629 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:08.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:08.686 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:08.686 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:08.693 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:08 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:08.693 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:08 smithi018 ceph-mon[29336]: pgmap v876: 168 pgs: 168 active+clean; 4.3 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 1.2 KiB/s rd, 96 MiB/s wr, 1.53k op/s 2024-09-10T19:04:08.693 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:08 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:08.693 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:08 smithi018 ceph-mon[40196]: pgmap v876: 168 pgs: 168 active+clean; 4.3 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 1.2 KiB/s rd, 96 MiB/s wr, 1.53k op/s 2024-09-10T19:04:08.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:08 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:08.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:08 smithi103 ceph-mon[33163]: pgmap v876: 168 pgs: 168 active+clean; 4.3 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 1.2 KiB/s rd, 96 MiB/s wr, 1.53k op/s 2024-09-10T19:04:08.788 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:08.832 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:08.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:08.833 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:08.934 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:08.961 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:08.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:08.963 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:09.037 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:04:09.064 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:09.071 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 41 16 74087 74071 112.898 81.8125 0.0132438 0.00885065 2024-09-10T19:04:09.106 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:09.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:09.107 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:09.208 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:09.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:09.249 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:09.249 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:09.350 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:09.352 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T19:04:09.352 INFO:teuthology.orchestra.run.smithi018.stdout: "size": 20, 2024-09-10T19:04:09.352 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 600, 2024-09-10T19:04:09.352 INFO:teuthology.orchestra.run.smithi018.stdout: "ops": [ 2024-09-10T19:04:09.352 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.352 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15354.0:121944 9.a 9:5da60af9:::benchmark_data_smithi018_133594_object121943:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e270)", 2024-09-10T19:04:09.352 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T18:57:01.468640+0000", 2024-09-10T19:04:09.352 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 427.877455725, 2024-09-10T19:04:09.352 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.0828585020000001, 2024-09-10T19:04:09.352 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:04:09.352 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:04:09.352 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:04:09.352 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15354", 2024-09-10T19:04:09.353 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:04:09.353 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 121944 2024-09-10T19:04:09.353 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.353 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:04:09.353 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.353 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:04:09.353 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:01.468640+0000", 2024-09-10T19:04:09.353 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.353 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.353 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.353 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:04:09.353 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:01.468640+0000", 2024-09-10T19:04:09.353 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.353 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.354 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.354 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:04:09.354 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:01.468638+0000", 2024-09-10T19:04:09.354 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999981 2024-09-10T19:04:09.354 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.354 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.354 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:04:09.354 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:01.468668+0000", 2024-09-10T19:04:09.354 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.9788e-05 2024-09-10T19:04:09.354 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.354 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.354 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:04:09.354 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:01.468682+0000", 2024-09-10T19:04:09.354 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.3730000000000001e-05 2024-09-10T19:04:09.355 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.355 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.355 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:04:09.355 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:01.468689+0000", 2024-09-10T19:04:09.355 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.8420000000000005e-06 2024-09-10T19:04:09.355 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.355 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.355 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:04:09.355 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:01.468980+0000", 2024-09-10T19:04:09.355 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00029055699999999999 2024-09-10T19:04:09.355 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.355 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.356 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:04:09.356 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:01.468990+0000", 2024-09-10T19:04:09.356 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0339999999999999e-05 2024-09-10T19:04:09.356 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.356 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.356 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:04:09.356 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:02.422407+0000", 2024-09-10T19:04:09.356 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.95341685300000001 2024-09-10T19:04:09.356 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.356 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.356 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:04:09.356 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:02.422415+0000", 2024-09-10T19:04:09.356 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.3669999999999999e-06 2024-09-10T19:04:09.357 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.357 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.357 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:04:09.357 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:03.525114+0000", 2024-09-10T19:04:09.357 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1026991399999999 2024-09-10T19:04:09.357 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.357 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.357 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:04:09.357 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:03.525255+0000", 2024-09-10T19:04:09.357 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00014135800000000001 2024-09-10T19:04:09.357 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.357 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.357 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 3,6,7", 2024-09-10T19:04:09.358 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:03.525693+0000", 2024-09-10T19:04:09.358 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00043795699999999999 2024-09-10T19:04:09.358 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.358 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.358 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:04:09.358 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:03.537237+0000", 2024-09-10T19:04:09.358 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0115438 2024-09-10T19:04:09.358 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.358 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.358 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:04:09.358 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:03.537465+0000", 2024-09-10T19:04:09.358 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00022770900000000001 2024-09-10T19:04:09.358 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.359 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.359 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:04:09.359 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:03.537519+0000", 2024-09-10T19:04:09.359 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.4184000000000002e-05 2024-09-10T19:04:09.359 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.359 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.359 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:04:09.359 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:03.551333+0000", 2024-09-10T19:04:09.359 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.013814180000000001 2024-09-10T19:04:09.359 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.359 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.359 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:04:09.359 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:03.551389+0000", 2024-09-10T19:04:09.359 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.6261999999999998e-05 2024-09-10T19:04:09.359 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.360 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.360 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:04:09.360 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T18:57:03.551498+0000", 2024-09-10T19:04:09.360 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00010918599999999999 2024-09-10T19:04:09.360 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.360 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:04:09.360 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.360 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.360 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.360 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:30066 10.c 10:3545e195:::benchmark_data_smithi018_157943_object30065:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e329)", 2024-09-10T19:04:09.360 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:07.349273+0000", 2024-09-10T19:04:09.360 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 241.99682254300001, 2024-09-10T19:04:09.360 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1219290630000001, 2024-09-10T19:04:09.361 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:04:09.361 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:04:09.361 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:04:09.361 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:04:09.361 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:04:09.361 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 30066 2024-09-10T19:04:09.361 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.361 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:04:09.361 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.361 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:04:09.361 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.349273+0000", 2024-09-10T19:04:09.361 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.361 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.362 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.362 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:04:09.362 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.349273+0000", 2024-09-10T19:04:09.362 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.362 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.364 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.364 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:04:09.364 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.349271+0000", 2024-09-10T19:04:09.364 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999981 2024-09-10T19:04:09.364 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.364 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.364 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:04:09.364 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.349303+0000", 2024-09-10T19:04:09.365 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.2063000000000001e-05 2024-09-10T19:04:09.365 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.365 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.365 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:04:09.365 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.349316+0000", 2024-09-10T19:04:09.365 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.2388000000000001e-05 2024-09-10T19:04:09.365 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.365 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.365 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:04:09.365 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.349322+0000", 2024-09-10T19:04:09.365 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.037e-06 2024-09-10T19:04:09.365 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.365 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.365 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:04:09.366 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.355006+0000", 2024-09-10T19:04:09.366 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0056842400000000001 2024-09-10T19:04:09.366 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.366 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.366 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:04:09.366 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.355015+0000", 2024-09-10T19:04:09.366 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.7569999999999993e-06 2024-09-10T19:04:09.366 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.366 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.366 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:04:09.366 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.467715+0000", 2024-09-10T19:04:09.366 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.112699814 2024-09-10T19:04:09.366 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.366 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.366 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:04:09.367 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.467852+0000", 2024-09-10T19:04:09.367 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000137835 2024-09-10T19:04:09.367 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.367 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.367 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 0,3", 2024-09-10T19:04:09.367 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.468052+0000", 2024-09-10T19:04:09.367 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00019955999999999999 2024-09-10T19:04:09.367 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.367 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.367 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:04:09.367 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.471004+0000", 2024-09-10T19:04:09.367 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0029518389999999999 2024-09-10T19:04:09.367 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.368 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.368 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:04:09.368 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.471065+0000", 2024-09-10T19:04:09.368 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.1542000000000001e-05 2024-09-10T19:04:09.368 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.368 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.368 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:04:09.368 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.471123+0000", 2024-09-10T19:04:09.368 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.7457999999999998e-05 2024-09-10T19:04:09.368 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.368 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.368 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:04:09.368 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.471173+0000", 2024-09-10T19:04:09.368 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.0084999999999999e-05 2024-09-10T19:04:09.369 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.369 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.369 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:04:09.369 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.471202+0000", 2024-09-10T19:04:09.369 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.9176999999999999e-05 2024-09-10T19:04:09.369 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.369 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:04:09.369 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.369 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.369 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.369 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:30069 10.c 10:30e91e09:::benchmark_data_smithi018_157943_object30068:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e329)", 2024-09-10T19:04:09.369 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:07.355152+0000", 2024-09-10T19:04:09.370 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 241.99094324800001, 2024-09-10T19:04:09.370 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1176073280000001, 2024-09-10T19:04:09.370 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:04:09.370 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:04:09.370 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:04:09.370 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:04:09.370 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:04:09.370 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 30069 2024-09-10T19:04:09.370 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.370 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:04:09.370 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.370 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:04:09.371 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.355152+0000", 2024-09-10T19:04:09.371 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.371 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.371 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.371 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:04:09.371 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.355152+0000", 2024-09-10T19:04:09.371 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.371 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.371 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.371 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:04:09.371 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.355151+0000", 2024-09-10T19:04:09.371 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999981 2024-09-10T19:04:09.371 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.372 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.372 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:04:09.372 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.355183+0000", 2024-09-10T19:04:09.372 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.2786000000000001e-05 2024-09-10T19:04:09.372 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.372 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.372 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:04:09.372 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.355196+0000", 2024-09-10T19:04:09.372 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.2388000000000001e-05 2024-09-10T19:04:09.372 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.372 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.372 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:04:09.372 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.355202+0000", 2024-09-10T19:04:09.372 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.4690000000000002e-06 2024-09-10T19:04:09.373 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.373 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.373 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:04:09.373 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.355380+0000", 2024-09-10T19:04:09.373 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00017756900000000001 2024-09-10T19:04:09.373 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.373 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.373 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:04:09.373 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.355388+0000", 2024-09-10T19:04:09.373 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.6799999999999999e-06 2024-09-10T19:04:09.373 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.373 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.373 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:04:09.373 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.469672+0000", 2024-09-10T19:04:09.374 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.114283312 2024-09-10T19:04:09.374 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.374 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.374 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:04:09.374 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.469808+0000", 2024-09-10T19:04:09.374 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00013658999999999999 2024-09-10T19:04:09.374 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.374 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.374 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 0,3", 2024-09-10T19:04:09.374 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.469964+0000", 2024-09-10T19:04:09.374 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000155835 2024-09-10T19:04:09.374 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.374 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.375 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:04:09.375 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.472575+0000", 2024-09-10T19:04:09.375 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0026112900000000001 2024-09-10T19:04:09.375 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.375 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.375 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:04:09.375 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.472591+0000", 2024-09-10T19:04:09.375 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.5607000000000002e-05 2024-09-10T19:04:09.375 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.375 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.375 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:04:09.375 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.472671+0000", 2024-09-10T19:04:09.375 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.0150999999999999e-05 2024-09-10T19:04:09.375 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.376 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.376 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:04:09.376 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.472719+0000", 2024-09-10T19:04:09.376 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.7922e-05 2024-09-10T19:04:09.376 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.376 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.376 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:04:09.376 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.472760+0000", 2024-09-10T19:04:09.376 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.0445000000000003e-05 2024-09-10T19:04:09.376 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.376 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:04:09.376 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.376 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.377 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.377 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:30086 10.c 10:3336df4e:::benchmark_data_smithi018_157943_object30085:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e329)", 2024-09-10T19:04:09.377 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:07.357870+0000", 2024-09-10T19:04:09.377 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 241.98822541000001, 2024-09-10T19:04:09.377 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1176519279999999, 2024-09-10T19:04:09.377 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:04:09.377 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:04:09.377 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:04:09.377 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:04:09.377 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:04:09.377 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 30086 2024-09-10T19:04:09.377 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.377 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:04:09.378 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.378 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:04:09.378 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.357870+0000", 2024-09-10T19:04:09.378 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.378 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.378 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.378 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:04:09.378 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.357870+0000", 2024-09-10T19:04:09.378 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.378 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.378 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.378 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:04:09.378 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.357869+0000", 2024-09-10T19:04:09.378 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T19:04:09.379 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.379 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.379 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:04:09.379 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.357892+0000", 2024-09-10T19:04:09.379 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.3281999999999998e-05 2024-09-10T19:04:09.379 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.379 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.379 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:04:09.379 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.357903+0000", 2024-09-10T19:04:09.379 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0423e-05 2024-09-10T19:04:09.380 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.380 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.380 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:04:09.380 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.357909+0000", 2024-09-10T19:04:09.380 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.0560000000000001e-06 2024-09-10T19:04:09.380 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.380 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.380 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:04:09.380 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.358223+0000", 2024-09-10T19:04:09.380 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00031434699999999998 2024-09-10T19:04:09.380 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.380 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.381 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:04:09.381 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.358232+0000", 2024-09-10T19:04:09.381 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.1370000000000008e-06 2024-09-10T19:04:09.381 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.381 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.381 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:04:09.381 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.471296+0000", 2024-09-10T19:04:09.381 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1130639929999999 2024-09-10T19:04:09.381 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.381 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.381 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:04:09.381 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.471427+0000", 2024-09-10T19:04:09.381 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000130412 2024-09-10T19:04:09.382 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.382 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.382 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 0,3", 2024-09-10T19:04:09.382 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.471573+0000", 2024-09-10T19:04:09.383 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00014568099999999999 2024-09-10T19:04:09.383 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.383 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.384 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:04:09.384 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.474058+0000", 2024-09-10T19:04:09.384 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0024856930000000002 2024-09-10T19:04:09.384 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.384 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.384 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:04:09.384 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.474129+0000", 2024-09-10T19:04:09.384 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.0660000000000004e-05 2024-09-10T19:04:09.384 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.384 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.384 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:04:09.384 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.475424+0000", 2024-09-10T19:04:09.385 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0012947290000000001 2024-09-10T19:04:09.385 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.385 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.385 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:04:09.385 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.475482+0000", 2024-09-10T19:04:09.385 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.8479e-05 2024-09-10T19:04:09.385 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.385 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.385 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:04:09.385 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.475522+0000", 2024-09-10T19:04:09.385 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.9991999999999999e-05 2024-09-10T19:04:09.385 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.385 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:04:09.385 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.386 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.386 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.386 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:30091 10.c 10:321e1c33:::benchmark_data_smithi018_157943_object30090:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e329)", 2024-09-10T19:04:09.386 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:07.363674+0000", 2024-09-10T19:04:09.386 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 241.982421746, 2024-09-10T19:04:09.386 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.113559628, 2024-09-10T19:04:09.386 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:04:09.386 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:04:09.386 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:04:09.386 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:04:09.386 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:04:09.386 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 30091 2024-09-10T19:04:09.386 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.387 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:04:09.387 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.387 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:04:09.387 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.363674+0000", 2024-09-10T19:04:09.387 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.387 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.387 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.387 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:04:09.387 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.363674+0000", 2024-09-10T19:04:09.387 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.387 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.387 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.387 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:04:09.387 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.363672+0000", 2024-09-10T19:04:09.388 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999986 2024-09-10T19:04:09.388 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.388 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.388 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:04:09.388 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.363703+0000", 2024-09-10T19:04:09.388 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.0219e-05 2024-09-10T19:04:09.388 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.388 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.388 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:04:09.388 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.363718+0000", 2024-09-10T19:04:09.388 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.5041999999999999e-05 2024-09-10T19:04:09.388 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.388 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.389 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:04:09.389 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.363723+0000", 2024-09-10T19:04:09.389 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.4199999999999998e-06 2024-09-10T19:04:09.389 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.389 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.389 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:04:09.389 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.364936+0000", 2024-09-10T19:04:09.389 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0012127939999999999 2024-09-10T19:04:09.389 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.389 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.389 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:04:09.389 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.364963+0000", 2024-09-10T19:04:09.389 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.7033e-05 2024-09-10T19:04:09.389 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.390 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.390 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:04:09.390 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.472794+0000", 2024-09-10T19:04:09.390 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1078312450000001 2024-09-10T19:04:09.390 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.390 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.390 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:04:09.390 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.472926+0000", 2024-09-10T19:04:09.390 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000131326 2024-09-10T19:04:09.390 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.390 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.390 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 0,3", 2024-09-10T19:04:09.390 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.473065+0000", 2024-09-10T19:04:09.391 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00013973300000000001 2024-09-10T19:04:09.391 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.391 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.391 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:04:09.391 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.477097+0000", 2024-09-10T19:04:09.391 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0040313700000000003 2024-09-10T19:04:09.391 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.391 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.391 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:04:09.391 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.477104+0000", 2024-09-10T19:04:09.391 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.0690000000000004e-06 2024-09-10T19:04:09.391 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.391 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.391 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:04:09.392 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.477147+0000", 2024-09-10T19:04:09.392 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.2840000000000003e-05 2024-09-10T19:04:09.392 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.392 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.392 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:04:09.392 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.477194+0000", 2024-09-10T19:04:09.392 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.7070999999999997e-05 2024-09-10T19:04:09.392 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.392 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.392 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:04:09.392 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.477233+0000", 2024-09-10T19:04:09.392 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.9762000000000003e-05 2024-09-10T19:04:09.392 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.393 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:04:09.393 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.393 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.393 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.393 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:30094 10.c 10:34d9c766:::benchmark_data_smithi018_157943_object30093:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e329)", 2024-09-10T19:04:09.393 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:07.363968+0000", 2024-09-10T19:04:09.393 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 241.982127776, 2024-09-10T19:04:09.393 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1134059510000001, 2024-09-10T19:04:09.393 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:04:09.394 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:04:09.394 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:04:09.394 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:04:09.394 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:04:09.394 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 30094 2024-09-10T19:04:09.394 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.394 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:04:09.394 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.394 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:04:09.394 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.363968+0000", 2024-09-10T19:04:09.394 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.394 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.394 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.395 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:04:09.395 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.363968+0000", 2024-09-10T19:04:09.395 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.395 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.395 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.395 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:04:09.395 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.363967+0000", 2024-09-10T19:04:09.395 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T19:04:09.395 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.395 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.395 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:04:09.395 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.364007+0000", 2024-09-10T19:04:09.395 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.9898000000000003e-05 2024-09-10T19:04:09.395 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.396 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.396 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:04:09.396 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.364021+0000", 2024-09-10T19:04:09.396 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.3842000000000001e-05 2024-09-10T19:04:09.396 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.396 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.396 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:04:09.396 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.364027+0000", 2024-09-10T19:04:09.396 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.0449999999999997e-06 2024-09-10T19:04:09.396 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.396 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.396 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:04:09.396 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.365009+0000", 2024-09-10T19:04:09.397 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00098269799999999995 2024-09-10T19:04:09.397 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.397 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.397 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:04:09.397 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.365017+0000", 2024-09-10T19:04:09.397 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.8809999999999996e-06 2024-09-10T19:04:09.397 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.397 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.397 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:04:09.397 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.474180+0000", 2024-09-10T19:04:09.397 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.109162371 2024-09-10T19:04:09.397 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.397 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.397 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:04:09.398 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.474309+0000", 2024-09-10T19:04:09.398 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000129286 2024-09-10T19:04:09.398 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.398 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.398 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 0,3", 2024-09-10T19:04:09.398 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.474489+0000", 2024-09-10T19:04:09.398 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00018039999999999999 2024-09-10T19:04:09.398 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.398 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.398 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:04:09.398 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.477245+0000", 2024-09-10T19:04:09.398 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.002755773 2024-09-10T19:04:09.399 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.399 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.399 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:04:09.399 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.477275+0000", 2024-09-10T19:04:09.399 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.9607e-05 2024-09-10T19:04:09.399 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.399 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.399 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:04:09.399 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.477303+0000", 2024-09-10T19:04:09.399 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.8107999999999999e-05 2024-09-10T19:04:09.399 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.399 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.399 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:04:09.399 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.477342+0000", 2024-09-10T19:04:09.399 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.9595000000000003e-05 2024-09-10T19:04:09.400 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.400 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.400 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:04:09.400 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.477374+0000", 2024-09-10T19:04:09.400 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.1374999999999997e-05 2024-09-10T19:04:09.400 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.400 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:04:09.400 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.400 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.400 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.400 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:30141 10.c 10:3ad97324:::benchmark_data_smithi018_157943_object30140:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e329)", 2024-09-10T19:04:09.400 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:07.385387+0000", 2024-09-10T19:04:09.400 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 241.96070804499999, 2024-09-10T19:04:09.400 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.093648599, 2024-09-10T19:04:09.401 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:04:09.401 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:04:09.401 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:04:09.401 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:04:09.401 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:04:09.401 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 30141 2024-09-10T19:04:09.401 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.401 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:04:09.401 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.401 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:04:09.401 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.385387+0000", 2024-09-10T19:04:09.401 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.401 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.402 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.402 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:04:09.402 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.385387+0000", 2024-09-10T19:04:09.402 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.402 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.402 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.402 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:04:09.402 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.385386+0000", 2024-09-10T19:04:09.402 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999981 2024-09-10T19:04:09.402 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.402 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.402 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:04:09.402 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.385411+0000", 2024-09-10T19:04:09.402 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.5398999999999999e-05 2024-09-10T19:04:09.403 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.403 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.403 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:04:09.403 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.385428+0000", 2024-09-10T19:04:09.403 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.7357e-05 2024-09-10T19:04:09.403 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.403 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.403 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:04:09.403 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.385435+0000", 2024-09-10T19:04:09.403 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.0190000000000001e-06 2024-09-10T19:04:09.403 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.403 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.403 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:04:09.404 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.385498+0000", 2024-09-10T19:04:09.404 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.2858e-05 2024-09-10T19:04:09.404 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.404 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.404 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:04:09.404 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.385508+0000", 2024-09-10T19:04:09.404 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.6129999999999993e-06 2024-09-10T19:04:09.404 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.404 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.404 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:04:09.404 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.475579+0000", 2024-09-10T19:04:09.404 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.090070761 2024-09-10T19:04:09.404 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.405 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.405 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:04:09.405 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.475723+0000", 2024-09-10T19:04:09.405 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00014458099999999999 2024-09-10T19:04:09.405 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.405 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.405 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 0,3", 2024-09-10T19:04:09.405 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.475849+0000", 2024-09-10T19:04:09.405 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00012578399999999999 2024-09-10T19:04:09.405 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.405 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.405 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:04:09.405 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.478826+0000", 2024-09-10T19:04:09.406 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.002977436 2024-09-10T19:04:09.406 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.406 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.406 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:04:09.406 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.478883+0000", 2024-09-10T19:04:09.406 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.6851000000000002e-05 2024-09-10T19:04:09.406 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.406 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.406 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:04:09.406 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.478928+0000", 2024-09-10T19:04:09.406 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.5136999999999998e-05 2024-09-10T19:04:09.406 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.406 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.406 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:04:09.406 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.478998+0000", 2024-09-10T19:04:09.407 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.9381999999999994e-05 2024-09-10T19:04:09.407 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.407 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.407 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:04:09.407 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.479036+0000", 2024-09-10T19:04:09.407 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.8368999999999998e-05 2024-09-10T19:04:09.407 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.407 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:04:09.407 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.407 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.407 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.407 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:30157 10.c 10:36dc0055:::benchmark_data_smithi018_157943_object30156:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e329)", 2024-09-10T19:04:09.407 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:07.392968+0000", 2024-09-10T19:04:09.407 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 241.953127908, 2024-09-10T19:04:09.408 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0948556270000001, 2024-09-10T19:04:09.408 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:04:09.408 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:04:09.408 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:04:09.408 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:04:09.408 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:04:09.408 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 30157 2024-09-10T19:04:09.408 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.408 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:04:09.408 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.408 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:04:09.408 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.392968+0000", 2024-09-10T19:04:09.408 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.409 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.409 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.409 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:04:09.409 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.392968+0000", 2024-09-10T19:04:09.409 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.409 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.409 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.409 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:04:09.409 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.392967+0000", 2024-09-10T19:04:09.409 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T19:04:09.409 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.409 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.409 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:04:09.410 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.393004+0000", 2024-09-10T19:04:09.410 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.7685999999999998e-05 2024-09-10T19:04:09.410 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.410 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.410 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:04:09.410 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.393018+0000", 2024-09-10T19:04:09.410 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.4029000000000001e-05 2024-09-10T19:04:09.410 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.410 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.410 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:04:09.410 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.393024+0000", 2024-09-10T19:04:09.410 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.6949999999999997e-06 2024-09-10T19:04:09.410 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.410 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.411 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:04:09.411 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.393087+0000", 2024-09-10T19:04:09.411 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.2749000000000005e-05 2024-09-10T19:04:09.411 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.411 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.411 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:04:09.411 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.393096+0000", 2024-09-10T19:04:09.411 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.6379999999999999e-06 2024-09-10T19:04:09.411 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.411 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.411 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:04:09.411 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.480350+0000", 2024-09-10T19:04:09.411 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0872540820000001 2024-09-10T19:04:09.411 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.411 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.412 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:04:09.412 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.480467+0000", 2024-09-10T19:04:09.412 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000116823 2024-09-10T19:04:09.412 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.412 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.412 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 0,3", 2024-09-10T19:04:09.412 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.480624+0000", 2024-09-10T19:04:09.412 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00015658499999999999 2024-09-10T19:04:09.412 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.412 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.412 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:04:09.412 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.482009+0000", 2024-09-10T19:04:09.413 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0013854640000000001 2024-09-10T19:04:09.413 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.413 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.413 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:04:09.413 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.487698+0000", 2024-09-10T19:04:09.413 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0056892269999999998 2024-09-10T19:04:09.413 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.413 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.413 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:04:09.413 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.487736+0000", 2024-09-10T19:04:09.413 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.7500999999999999e-05 2024-09-10T19:04:09.413 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.413 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.413 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:04:09.414 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.487788+0000", 2024-09-10T19:04:09.414 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.1637e-05 2024-09-10T19:04:09.414 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.414 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.414 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:04:09.414 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.487823+0000", 2024-09-10T19:04:09.414 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.5621000000000003e-05 2024-09-10T19:04:09.414 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.414 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:04:09.414 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.414 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.414 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.414 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:30176 10.c 10:3c44fac3:::benchmark_data_smithi018_157943_object30175:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e329)", 2024-09-10T19:04:09.415 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:07.401052+0000", 2024-09-10T19:04:09.415 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 241.945043352, 2024-09-10T19:04:09.415 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0946845709999999, 2024-09-10T19:04:09.415 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:04:09.415 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:04:09.415 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:04:09.415 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:04:09.415 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:04:09.415 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 30176 2024-09-10T19:04:09.415 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.415 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:04:09.415 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.415 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:04:09.416 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.401052+0000", 2024-09-10T19:04:09.416 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.416 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.416 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.416 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:04:09.416 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.401052+0000", 2024-09-10T19:04:09.416 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.416 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.416 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.416 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:04:09.416 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.401050+0000", 2024-09-10T19:04:09.416 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999981 2024-09-10T19:04:09.416 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.417 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.417 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:04:09.417 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.401083+0000", 2024-09-10T19:04:09.417 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.3191000000000001e-05 2024-09-10T19:04:09.417 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.417 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.417 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:04:09.417 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.401098+0000", 2024-09-10T19:04:09.417 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.5031000000000001e-05 2024-09-10T19:04:09.417 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.417 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.417 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:04:09.417 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.401106+0000", 2024-09-10T19:04:09.418 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.5399999999999998e-06 2024-09-10T19:04:09.418 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.418 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.418 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:04:09.418 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.401170+0000", 2024-09-10T19:04:09.418 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.3587e-05 2024-09-10T19:04:09.418 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.418 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.418 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:04:09.418 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.401181+0000", 2024-09-10T19:04:09.418 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1703e-05 2024-09-10T19:04:09.418 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.418 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.419 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:04:09.419 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.486342+0000", 2024-09-10T19:04:09.419 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0851604100000001 2024-09-10T19:04:09.419 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.419 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.419 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:04:09.419 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.486454+0000", 2024-09-10T19:04:09.419 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00011186 2024-09-10T19:04:09.419 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.419 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.419 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 0,3", 2024-09-10T19:04:09.419 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.486588+0000", 2024-09-10T19:04:09.419 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00013417600000000001 2024-09-10T19:04:09.420 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.420 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.420 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:04:09.420 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.492612+0000", 2024-09-10T19:04:09.420 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0060245369999999999 2024-09-10T19:04:09.420 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.420 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.420 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:04:09.420 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.492655+0000", 2024-09-10T19:04:09.420 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.2815000000000003e-05 2024-09-10T19:04:09.420 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.420 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.420 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:04:09.421 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.495655+0000", 2024-09-10T19:04:09.421 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0030003600000000001 2024-09-10T19:04:09.421 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.421 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.421 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:04:09.421 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.495699+0000", 2024-09-10T19:04:09.421 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.3615000000000002e-05 2024-09-10T19:04:09.421 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.421 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.421 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:04:09.421 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.495737+0000", 2024-09-10T19:04:09.421 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.7666000000000002e-05 2024-09-10T19:04:09.422 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.422 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:04:09.422 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.422 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.422 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.422 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:30184 10.c 10:3a927544:::benchmark_data_smithi018_157943_object30183:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e329)", 2024-09-10T19:04:09.422 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:07.405082+0000", 2024-09-10T19:04:09.422 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 241.94101375700001, 2024-09-10T19:04:09.422 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.093739512, 2024-09-10T19:04:09.422 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:04:09.422 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:04:09.422 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:04:09.422 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:04:09.423 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:04:09.423 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 30184 2024-09-10T19:04:09.423 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.423 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:04:09.423 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.423 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:04:09.423 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.405082+0000", 2024-09-10T19:04:09.423 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.423 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.423 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.423 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:04:09.423 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.405082+0000", 2024-09-10T19:04:09.423 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.423 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.424 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.424 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:04:09.424 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.405074+0000", 2024-09-10T19:04:09.424 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999924 2024-09-10T19:04:09.424 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.424 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.424 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:04:09.424 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.405106+0000", 2024-09-10T19:04:09.424 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.2462000000000002e-05 2024-09-10T19:04:09.424 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.424 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.424 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:04:09.424 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.405122+0000", 2024-09-10T19:04:09.424 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.5829e-05 2024-09-10T19:04:09.425 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.425 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.425 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:04:09.425 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.405130+0000", 2024-09-10T19:04:09.425 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.6550000000000004e-06 2024-09-10T19:04:09.425 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.425 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.425 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:04:09.425 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.405171+0000", 2024-09-10T19:04:09.425 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.1498999999999997e-05 2024-09-10T19:04:09.425 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.425 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.425 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:04:09.425 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.405178+0000", 2024-09-10T19:04:09.426 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.2840000000000003e-06 2024-09-10T19:04:09.426 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.426 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.426 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:04:09.426 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.487997+0000", 2024-09-10T19:04:09.426 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0828189509999999 2024-09-10T19:04:09.426 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.426 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.427 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:04:09.427 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.488121+0000", 2024-09-10T19:04:09.427 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000124354 2024-09-10T19:04:09.427 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.427 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.427 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 0,3", 2024-09-10T19:04:09.427 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.488427+0000", 2024-09-10T19:04:09.427 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00030601599999999999 2024-09-10T19:04:09.427 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.427 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.427 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:04:09.427 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.492619+0000", 2024-09-10T19:04:09.427 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0041922590000000003 2024-09-10T19:04:09.427 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.427 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.428 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:04:09.428 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.492709+0000", 2024-09-10T19:04:09.428 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 8.9599999999999996e-05 2024-09-10T19:04:09.428 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.428 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.428 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:04:09.428 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.498721+0000", 2024-09-10T19:04:09.428 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0060124610000000002 2024-09-10T19:04:09.428 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.428 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.428 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:04:09.428 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.498775+0000", 2024-09-10T19:04:09.428 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.3381000000000003e-05 2024-09-10T19:04:09.428 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.429 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.429 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:04:09.429 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.498821+0000", 2024-09-10T19:04:09.429 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.6520999999999997e-05 2024-09-10T19:04:09.429 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.429 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:04:09.429 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.429 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.429 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.429 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.15573.0:30187 10.c 10:3f41b8ee:::benchmark_data_smithi018_157943_object30186:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e329)", 2024-09-10T19:04:09.429 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:00:07.406336+0000", 2024-09-10T19:04:09.429 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 241.93975965000001, 2024-09-10T19:04:09.429 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.092607122, 2024-09-10T19:04:09.430 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:04:09.430 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:04:09.430 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:04:09.430 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.15573", 2024-09-10T19:04:09.430 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:04:09.430 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 30187 2024-09-10T19:04:09.430 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.430 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:04:09.430 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.430 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:04:09.430 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.406336+0000", 2024-09-10T19:04:09.430 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.430 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.430 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.430 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:04:09.431 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.406336+0000", 2024-09-10T19:04:09.431 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.431 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.431 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.431 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:04:09.431 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.406335+0000", 2024-09-10T19:04:09.431 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T19:04:09.431 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.431 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.431 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:04:09.431 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.406357+0000", 2024-09-10T19:04:09.431 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.2090999999999999e-05 2024-09-10T19:04:09.431 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.431 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.432 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:04:09.432 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.406368+0000", 2024-09-10T19:04:09.432 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1077000000000001e-05 2024-09-10T19:04:09.432 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.432 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.432 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:04:09.432 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.406373+0000", 2024-09-10T19:04:09.432 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.2259999999999996e-06 2024-09-10T19:04:09.432 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.432 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.432 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:04:09.432 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.406409+0000", 2024-09-10T19:04:09.432 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.5908999999999998e-05 2024-09-10T19:04:09.433 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.433 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.433 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for peered", 2024-09-10T19:04:09.433 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:07.406416+0000", 2024-09-10T19:04:09.433 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.8210000000000004e-06 2024-09-10T19:04:09.433 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.433 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.433 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:04:09.433 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.489738+0000", 2024-09-10T19:04:09.433 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0833220569999999 2024-09-10T19:04:09.433 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.433 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.434 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:04:09.434 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.489880+0000", 2024-09-10T19:04:09.434 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00014216899999999999 2024-09-10T19:04:09.434 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.434 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.434 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 0,3", 2024-09-10T19:04:09.434 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.490067+0000", 2024-09-10T19:04:09.434 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00018631999999999999 2024-09-10T19:04:09.434 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.434 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.434 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:04:09.434 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.492661+0000", 2024-09-10T19:04:09.434 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0025938049999999998 2024-09-10T19:04:09.434 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.435 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.435 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:04:09.435 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.495613+0000", 2024-09-10T19:04:09.435 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.002952391 2024-09-10T19:04:09.435 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.435 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.435 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:04:09.435 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.498866+0000", 2024-09-10T19:04:09.435 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0032528399999999999 2024-09-10T19:04:09.435 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.435 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.435 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:04:09.435 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.498911+0000", 2024-09-10T19:04:09.435 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.5664999999999997e-05 2024-09-10T19:04:09.435 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.436 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.436 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:04:09.436 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:00:08.498943+0000", 2024-09-10T19:04:09.436 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.1563000000000003e-05 2024-09-10T19:04:09.436 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.436 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:04:09.436 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.436 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.436 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.436 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.25706.0:33180 11.3 e388/379 11:cd54710c:::benchmark_data_smithi018_181485_object33179:head v 388'2050, mlcod=388'2048)", 2024-09-10T19:04:09.436 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:03:41.232057+0000", 2024-09-10T19:04:09.436 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 28.114038966999999, 2024-09-10T19:04:09.436 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.094837066999999997, 2024-09-10T19:04:09.436 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:04:09.437 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:04:09.437 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:04:09.437 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.437 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:04:09.437 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:41.232057+0000", 2024-09-10T19:04:09.437 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.437 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.437 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.437 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:04:09.437 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:41.232057+0000", 2024-09-10T19:04:09.437 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.437 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.437 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.437 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:04:09.438 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:41.232056+0000", 2024-09-10T19:04:09.438 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T19:04:09.438 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.438 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.438 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:04:09.438 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:41.232101+0000", 2024-09-10T19:04:09.438 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.5729999999999998e-05 2024-09-10T19:04:09.438 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.438 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.438 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:04:09.438 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:41.232112+0000", 2024-09-10T19:04:09.438 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0421999999999999e-05 2024-09-10T19:04:09.438 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.438 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.438 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:04:09.438 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:41.232118+0000", 2024-09-10T19:04:09.439 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.9880000000000001e-06 2024-09-10T19:04:09.439 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.439 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.439 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:04:09.439 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:41.232166+0000", 2024-09-10T19:04:09.439 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.7667999999999998e-05 2024-09-10T19:04:09.439 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.439 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.439 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:04:09.439 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:41.232183+0000", 2024-09-10T19:04:09.439 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.7535000000000001e-05 2024-09-10T19:04:09.440 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.440 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.440 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:04:09.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:09.441 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:09.441 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:09.441 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:41.234350+0000", 2024-09-10T19:04:09.442 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0021673389999999999 2024-09-10T19:04:09.442 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.442 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.442 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:04:09.442 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:41.326894+0000", 2024-09-10T19:04:09.442 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.092543234000000002 2024-09-10T19:04:09.442 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.442 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:04:09.442 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.442 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.442 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.442 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.25706.0:33189 11.c e388/379 11:35f10ea7:::benchmark_data_smithi018_181485_object33188:head v 388'2016, mlcod=388'2014)", 2024-09-10T19:04:09.442 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:03:41.234735+0000", 2024-09-10T19:04:09.442 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 28.111360098999999, 2024-09-10T19:04:09.443 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.094324564999999999, 2024-09-10T19:04:09.443 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:04:09.443 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:04:09.443 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:04:09.443 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.443 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:04:09.443 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:41.234735+0000", 2024-09-10T19:04:09.443 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.443 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.443 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.443 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:04:09.443 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:41.234735+0000", 2024-09-10T19:04:09.443 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.443 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.443 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.444 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:04:09.444 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:41.234735+0000", 2024-09-10T19:04:09.444 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T19:04:09.444 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.444 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.444 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:04:09.444 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:41.234797+0000", 2024-09-10T19:04:09.444 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.1741000000000004e-05 2024-09-10T19:04:09.444 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.444 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.444 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:04:09.444 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:41.326676+0000", 2024-09-10T19:04:09.444 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.091879384999999994 2024-09-10T19:04:09.444 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.444 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.445 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:04:09.445 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:41.326686+0000", 2024-09-10T19:04:09.445 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0214000000000001e-05 2024-09-10T19:04:09.445 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.445 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.445 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:04:09.445 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:41.326735+0000", 2024-09-10T19:04:09.445 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.8887999999999997e-05 2024-09-10T19:04:09.445 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.445 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.445 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:04:09.445 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:41.326756+0000", 2024-09-10T19:04:09.445 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.0525000000000001e-05 2024-09-10T19:04:09.445 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.446 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.446 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:04:09.446 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:41.328921+0000", 2024-09-10T19:04:09.446 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0021650739999999999 2024-09-10T19:04:09.446 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.446 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.446 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:04:09.446 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:03:41.329060+0000", 2024-09-10T19:04:09.446 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00013934400000000001 2024-09-10T19:04:09.446 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.446 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:04:09.446 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.446 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.446 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.446 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.25706.0:71852 11.b e397/379 11:df977e03:::benchmark_data_smithi018_181485_object71851:head v 397'4475, mlcod=397'4473)", 2024-09-10T19:04:09.447 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:04:07.176168+0000", 2024-09-10T19:04:09.447 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 2.1699277810000002, 2024-09-10T19:04:09.447 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.29094708000000002, 2024-09-10T19:04:09.447 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:04:09.447 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:04:09.447 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:04:09.447 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.447 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:04:09.447 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.176168+0000", 2024-09-10T19:04:09.447 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.447 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.447 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.447 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:04:09.447 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.176168+0000", 2024-09-10T19:04:09.447 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.448 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.448 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.448 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:04:09.448 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.176167+0000", 2024-09-10T19:04:09.448 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T19:04:09.448 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.448 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.448 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:04:09.448 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.176195+0000", 2024-09-10T19:04:09.448 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.8021000000000002e-05 2024-09-10T19:04:09.448 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.448 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.448 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:04:09.448 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.176209+0000", 2024-09-10T19:04:09.449 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.437e-05 2024-09-10T19:04:09.449 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.449 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.449 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:04:09.449 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.176215+0000", 2024-09-10T19:04:09.449 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.5729999999999998e-06 2024-09-10T19:04:09.449 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.449 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.449 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:04:09.449 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.177913+0000", 2024-09-10T19:04:09.449 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0016978830000000001 2024-09-10T19:04:09.449 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.449 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.450 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:04:09.450 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.177930+0000", 2024-09-10T19:04:09.450 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.7099999999999999e-05 2024-09-10T19:04:09.450 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.450 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.450 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:04:09.450 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.467067+0000", 2024-09-10T19:04:09.450 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.28913745800000001 2024-09-10T19:04:09.450 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.450 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.450 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:04:09.450 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.467115+0000", 2024-09-10T19:04:09.450 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.7398000000000003e-05 2024-09-10T19:04:09.450 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.451 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:04:09.451 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.451 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.451 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.451 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.25706.0:71853 11.7 e397/379 11:eb45c366:::benchmark_data_smithi018_181485_object71852:head v 397'4526, mlcod=397'4524)", 2024-09-10T19:04:09.451 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:04:07.176383+0000", 2024-09-10T19:04:09.451 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 2.1697127319999998, 2024-09-10T19:04:09.451 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.288568873, 2024-09-10T19:04:09.451 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:04:09.451 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:04:09.451 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:04:09.451 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.451 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:04:09.451 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.176383+0000", 2024-09-10T19:04:09.451 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.452 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.452 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.452 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:04:09.452 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.176383+0000", 2024-09-10T19:04:09.452 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.452 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.452 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.452 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:04:09.452 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.176382+0000", 2024-09-10T19:04:09.452 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T19:04:09.452 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.452 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.452 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:04:09.452 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.176456+0000", 2024-09-10T19:04:09.452 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.3458e-05 2024-09-10T19:04:09.453 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.453 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.453 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:04:09.453 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.176466+0000", 2024-09-10T19:04:09.453 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.978e-06 2024-09-10T19:04:09.453 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.453 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.453 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:04:09.453 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.176471+0000", 2024-09-10T19:04:09.453 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.3419999999999998e-06 2024-09-10T19:04:09.453 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.453 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.453 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:04:09.453 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.178009+0000", 2024-09-10T19:04:09.454 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0015384050000000001 2024-09-10T19:04:09.454 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.454 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.454 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:04:09.454 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.178028+0000", 2024-09-10T19:04:09.454 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.8712e-05 2024-09-10T19:04:09.454 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.454 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.454 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:04:09.454 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.464905+0000", 2024-09-10T19:04:09.454 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.28687750000000001 2024-09-10T19:04:09.454 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.454 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.454 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:04:09.454 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.464952+0000", 2024-09-10T19:04:09.455 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.6217999999999997e-05 2024-09-10T19:04:09.455 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.455 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:04:09.455 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.455 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.455 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.455 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.25706.0:71854 11.e e397/379 11:78265082:::benchmark_data_smithi018_181485_object71853:head v 397'4540, mlcod=397'4538)", 2024-09-10T19:04:09.455 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:04:07.176750+0000", 2024-09-10T19:04:09.455 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 2.169345694, 2024-09-10T19:04:09.455 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.28610401699999999, 2024-09-10T19:04:09.455 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:04:09.455 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:04:09.455 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:04:09.455 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.456 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:04:09.456 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.176750+0000", 2024-09-10T19:04:09.456 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.456 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.456 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.456 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:04:09.456 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.176750+0000", 2024-09-10T19:04:09.456 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.456 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.456 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.456 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:04:09.456 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.176749+0000", 2024-09-10T19:04:09.456 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T19:04:09.456 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.456 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.457 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:04:09.457 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.176793+0000", 2024-09-10T19:04:09.457 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.3924000000000002e-05 2024-09-10T19:04:09.457 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.457 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.457 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:04:09.457 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.176803+0000", 2024-09-10T19:04:09.457 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.5109999999999992e-06 2024-09-10T19:04:09.457 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.457 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.457 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:04:09.457 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.176808+0000", 2024-09-10T19:04:09.457 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.8779999999999997e-06 2024-09-10T19:04:09.457 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.458 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.458 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:04:09.458 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.176845+0000", 2024-09-10T19:04:09.458 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.7889999999999998e-05 2024-09-10T19:04:09.458 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.458 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.458 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:04:09.458 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.176863+0000", 2024-09-10T19:04:09.458 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.7212000000000001e-05 2024-09-10T19:04:09.458 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.458 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.458 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:04:09.458 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.179173+0000", 2024-09-10T19:04:09.458 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0023104900000000001 2024-09-10T19:04:09.458 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.459 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.459 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:04:09.459 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.462854+0000", 2024-09-10T19:04:09.459 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.28368074700000001 2024-09-10T19:04:09.459 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.459 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:04:09.459 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.459 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.459 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.459 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.25706.0:71848 11.c e397/379 11:30d382f5:::benchmark_data_smithi018_181485_object71847:head v 397'4520, mlcod=397'4517)", 2024-09-10T19:04:09.459 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:04:07.178423+0000", 2024-09-10T19:04:09.459 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 2.167672203, 2024-09-10T19:04:09.459 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.28445485500000001, 2024-09-10T19:04:09.459 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:04:09.459 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:04:09.460 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:04:09.460 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.460 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:04:09.460 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.178423+0000", 2024-09-10T19:04:09.460 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.460 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.460 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.460 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:04:09.460 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.178423+0000", 2024-09-10T19:04:09.460 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.460 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.460 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.460 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:04:09.460 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.178422+0000", 2024-09-10T19:04:09.461 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T19:04:09.461 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.461 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.461 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:04:09.461 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.178465+0000", 2024-09-10T19:04:09.461 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4.2236e-05 2024-09-10T19:04:09.461 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.461 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.461 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:04:09.461 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.178475+0000", 2024-09-10T19:04:09.461 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0284e-05 2024-09-10T19:04:09.461 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.461 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.462 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:04:09.462 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.178480+0000", 2024-09-10T19:04:09.462 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.5380000000000002e-06 2024-09-10T19:04:09.462 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.462 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.462 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:04:09.462 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.178785+0000", 2024-09-10T19:04:09.462 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.000304708 2024-09-10T19:04:09.462 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.462 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.462 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:04:09.462 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.178804+0000", 2024-09-10T19:04:09.462 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.8510999999999999e-05 2024-09-10T19:04:09.462 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.462 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.462 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:04:09.463 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.462850+0000", 2024-09-10T19:04:09.463 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.284045932 2024-09-10T19:04:09.463 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.463 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.463 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:04:09.463 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.462878+0000", 2024-09-10T19:04:09.463 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.8654999999999999e-05 2024-09-10T19:04:09.463 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.463 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:04:09.463 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.463 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.463 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.463 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.25706.0:71857 11.7 e397/379 11:edee4008:::benchmark_data_smithi018_181485_object71856:head v 397'4527, mlcod=397'4524)", 2024-09-10T19:04:09.463 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:04:07.179604+0000", 2024-09-10T19:04:09.463 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 2.1664918970000002, 2024-09-10T19:04:09.464 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.28958041299999998, 2024-09-10T19:04:09.464 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:04:09.464 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:04:09.464 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:04:09.464 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.464 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:04:09.464 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.179604+0000", 2024-09-10T19:04:09.464 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.464 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.464 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.464 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:04:09.464 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.179604+0000", 2024-09-10T19:04:09.464 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.464 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.465 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.465 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:04:09.465 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.179603+0000", 2024-09-10T19:04:09.465 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T19:04:09.465 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.465 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.465 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:04:09.465 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.179665+0000", 2024-09-10T19:04:09.465 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.2545000000000005e-05 2024-09-10T19:04:09.465 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.465 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.465 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:04:09.465 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.462828+0000", 2024-09-10T19:04:09.466 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.28316233600000001 2024-09-10T19:04:09.466 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.466 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.466 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:04:09.466 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.462841+0000", 2024-09-10T19:04:09.466 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.3173000000000001e-05 2024-09-10T19:04:09.466 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.466 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.466 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:04:09.466 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.462877+0000", 2024-09-10T19:04:09.466 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.6235000000000001e-05 2024-09-10T19:04:09.466 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.466 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.466 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:04:09.466 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.462900+0000", 2024-09-10T19:04:09.467 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.3215999999999999e-05 2024-09-10T19:04:09.467 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.467 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.467 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:04:09.467 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.469124+0000", 2024-09-10T19:04:09.467 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0062235240000000002 2024-09-10T19:04:09.467 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.467 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.467 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:04:09.467 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.469184+0000", 2024-09-10T19:04:09.467 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 6.0089e-05 2024-09-10T19:04:09.467 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.467 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:04:09.467 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.467 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.468 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.468 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.25706.0:71851 11.c e397/379 11:315a9b1d:::benchmark_data_smithi018_181485_object71850:head v 397'4521, mlcod=397'4517)", 2024-09-10T19:04:09.468 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:04:07.179663+0000", 2024-09-10T19:04:09.468 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 2.1664328369999999, 2024-09-10T19:04:09.468 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.285934297, 2024-09-10T19:04:09.468 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:04:09.468 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:04:09.468 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:04:09.468 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.468 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:04:09.468 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.179663+0000", 2024-09-10T19:04:09.468 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.468 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.468 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.468 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:04:09.469 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.179663+0000", 2024-09-10T19:04:09.469 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.469 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.469 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.469 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:04:09.469 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.179662+0000", 2024-09-10T19:04:09.469 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T19:04:09.469 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.469 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.469 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:04:09.469 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.179737+0000", 2024-09-10T19:04:09.469 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.5389000000000006e-05 2024-09-10T19:04:09.470 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.470 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.470 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:04:09.470 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.462838+0000", 2024-09-10T19:04:09.470 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.283100833 2024-09-10T19:04:09.470 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.470 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.470 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:04:09.470 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.462848+0000", 2024-09-10T19:04:09.470 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 9.2830000000000007e-06 2024-09-10T19:04:09.470 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.470 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.470 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:04:09.470 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.462902+0000", 2024-09-10T19:04:09.471 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.3959999999999998e-05 2024-09-10T19:04:09.471 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.471 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.471 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:04:09.471 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.462917+0000", 2024-09-10T19:04:09.471 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.5481999999999998e-05 2024-09-10T19:04:09.471 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.471 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.471 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:04:09.471 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.465414+0000", 2024-09-10T19:04:09.471 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0024969240000000002 2024-09-10T19:04:09.471 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.471 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.471 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:04:09.472 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.465597+0000", 2024-09-10T19:04:09.472 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00018303300000000001 2024-09-10T19:04:09.472 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.472 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:04:09.472 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.472 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.472 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.472 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_repop(client.25706.0:71855 11.0 e397/379 11:095aec38:::benchmark_data_smithi018_181485_object71854:head v 397'4424, mlcod=397'4422)", 2024-09-10T19:04:09.472 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:04:07.179802+0000", 2024-09-10T19:04:09.472 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 2.166293236, 2024-09-10T19:04:09.472 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.28579712699999998, 2024-09-10T19:04:09.472 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:04:09.472 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:04:09.472 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:04:09.473 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.473 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:04:09.473 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.179802+0000", 2024-09-10T19:04:09.473 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.473 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.473 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.473 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:04:09.473 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.179802+0000", 2024-09-10T19:04:09.473 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:04:09.473 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.473 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.473 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:04:09.473 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.179802+0000", 2024-09-10T19:04:09.473 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.9999995 2024-09-10T19:04:09.474 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.474 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.474 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:04:09.474 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.179823+0000", 2024-09-10T19:04:09.474 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.1801999999999999e-05 2024-09-10T19:04:09.474 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.474 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.474 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:04:09.474 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.462829+0000", 2024-09-10T19:04:09.474 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.28300551499999999 2024-09-10T19:04:09.474 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.474 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.474 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:04:09.474 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.462841+0000", 2024-09-10T19:04:09.474 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.1565e-05 2024-09-10T19:04:09.475 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.475 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.475 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:04:09.475 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.463188+0000", 2024-09-10T19:04:09.475 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00034790599999999999 2024-09-10T19:04:09.475 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.475 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.475 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:04:09.475 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.463209+0000", 2024-09-10T19:04:09.475 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 2.0581000000000002e-05 2024-09-10T19:04:09.475 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.475 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.475 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "commit_sent", 2024-09-10T19:04:09.475 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.465562+0000", 2024-09-10T19:04:09.475 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0023528469999999999 2024-09-10T19:04:09.476 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:04:09.476 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:04:09.476 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "done", 2024-09-10T19:04:09.476 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:04:07.465599+0000", 2024-09-10T19:04:09.476 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.7585e-05 2024-09-10T19:04:09.476 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.476 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:04:09.476 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.476 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:04:09.476 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:04:09.476 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T19:04:09.542 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:09.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:09.595 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:09.595 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:09.696 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:09.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:09.752 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:09.753 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:09.804 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is set 2024-09-10T19:04:09.853 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:09.881 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:09.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:09.882 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:09.983 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:10.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:10.014 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:10.014 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:10.071 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 42 16 75418 75402 112.191 83.1875 0.0113955 0.00890865 2024-09-10T19:04:10.102 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:09 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/496969632' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:04:10.102 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:09 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:04:10.102 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:09 smithi018 ceph-mon[40196]: pgmap v877: 168 pgs: 168 active+clean; 4.4 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 1.0 KiB/s rd, 85 MiB/s wr, 1.35k op/s 2024-09-10T19:04:10.102 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:09 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/496969632' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:04:10.102 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:09 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:04:10.102 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:09 smithi018 ceph-mon[29336]: pgmap v877: 168 pgs: 168 active+clean; 4.4 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 1.0 KiB/s rd, 85 MiB/s wr, 1.35k op/s 2024-09-10T19:04:10.115 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:10.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:10.157 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:10.157 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:10.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:09 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/496969632' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:04:10.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:09 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:04:10.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:09 smithi103 ceph-mon[33163]: pgmap v877: 168 pgs: 168 active+clean; 4.4 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 1.0 KiB/s rd, 85 MiB/s wr, 1.35k op/s 2024-09-10T19:04:10.257 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:10.289 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:10.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:10.290 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:10.391 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:10.421 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:10.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:10.422 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:10.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:04:10 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T19:04:10.095+0000 7f66d46e1640 -1 mgr get_metadata_python Requested missing service osd.7 2024-09-10T19:04:10.510 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:04:10.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:04:10 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:04:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:04:10.523 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:10.571 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:10.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:10.574 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:10.674 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:10.717 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:10.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:10.719 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:10.820 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:10.850 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:10.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:10.851 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:10.933 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:10 smithi018 ceph-mon[40196]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:04:10.934 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:10 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T19:04:10.934 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:10 smithi018 ceph-mon[40196]: osdmap e398: 8 total, 7 up, 6 in 2024-09-10T19:04:10.934 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:10 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:04:10.934 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:10 smithi018 ceph-mon[29336]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:04:10.934 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:10 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T19:04:10.934 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:10 smithi018 ceph-mon[29336]: osdmap e398: 8 total, 7 up, 6 in 2024-09-10T19:04:10.934 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:10 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:04:10.952 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:10.982 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:10.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:10.983 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:11.071 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 43 16 76967 76951 111.833 96.8125 0.00547882 0.00893748 2024-09-10T19:04:11.084 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:11.111 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:11.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:11.112 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:11.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:04:10 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:04:10] "GET /metrics HTTP/1.1" 200 209632 "" "Prometheus/2.43.0" 2024-09-10T19:04:11.213 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:11.245 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:11.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:11.246 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:11.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:10 smithi103 ceph-mon[33163]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:04:11.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:10 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T19:04:11.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:10 smithi103 ceph-mon[33163]: osdmap e398: 8 total, 7 up, 6 in 2024-09-10T19:04:11.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:10 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:04:11.347 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:11.391 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:11.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:11.392 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:11.414 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-10T19:04:11.493 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:11.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:11.556 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:11.556 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:11.657 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:11.682 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:11.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:11.683 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:11.784 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:11.834 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:11.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:11.835 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:11.936 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:11.965 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:11.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:11.967 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:12.067 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:12.071 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 44 16 78397 78381 111.322 89.375 0.00597067 0.00897782 2024-09-10T19:04:12.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:12.109 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:12.109 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:12.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:11 smithi018 ceph-mon[40196]: pgmap v879: 168 pgs: 168 active+clean; 4.5 GiB data, 19 GiB used, 517 GiB / 536 GiB avail; 639 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-10T19:04:12.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:11 smithi018 ceph-mon[29336]: pgmap v879: 168 pgs: 168 active+clean; 4.5 GiB data, 19 GiB used, 517 GiB / 536 GiB avail; 639 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-10T19:04:12.211 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:12.248 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:12.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:12.249 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:12.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:11 smithi103 ceph-mon[33163]: pgmap v879: 168 pgs: 168 active+clean; 4.5 GiB data, 19 GiB used, 517 GiB / 536 GiB avail; 639 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-10T19:04:12.350 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:12.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:12.390 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:12.391 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:12.492 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:12.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:12.539 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:12.539 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:12.640 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:12.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:12.677 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:12.677 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:12.778 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:12.807 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:12.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:12.808 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:12.909 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:12.949 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:12.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:12.950 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:13.052 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:13.072 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 45 16 79904 79888 110.938 94.1875 0.00835892 0.00900939 2024-09-10T19:04:13.095 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:13.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:13.096 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:13.197 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:13.226 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:13.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:13.227 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:13.328 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:13.359 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:13.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:13.360 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:13.461 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:13.491 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:13.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:13.492 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:13.593 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:13.642 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:13.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:13.643 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:13.744 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:13.765 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:13.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:13.766 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:13.868 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:13.891 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:13.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:13.892 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:13.993 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:14.025 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:14.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:14.026 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:14.073 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 46 16 81287 81271 110.406 86.4375 0.00959606 0.00905282 2024-09-10T19:04:14.101 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T19:04:14.101 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T19:04:14.101 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T19:04:14.101 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T19:04:14.101 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T19:04:14.101 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T19:04:14.101 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T19:04:14.101 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T19:04:14.102 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T19:04:14.102 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T19:04:14.102 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T19:04:14.102 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T19:04:14.102 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T19:04:14.102 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T19:04:14.102 INFO:teuthology.orchestra.run.smithi018.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-10T19:04:14.127 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:14.158 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:14.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:14.159 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:14.259 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:14.297 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:14.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:14.298 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:14.399 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:14.427 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:14.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:14.428 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:14.472 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-10T19:04:14.529 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:14.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:14.573 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:14.573 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:14.674 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:14.723 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:14 smithi018 ceph-mon[29336]: pgmap v880: 168 pgs: 168 active+clean; 4.7 GiB data, 20 GiB used, 517 GiB / 536 GiB avail; 1.0 KiB/s rd, 83 MiB/s wr, 1.33k op/s 2024-09-10T19:04:14.723 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:14 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1924216585' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:04:14.723 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:14 smithi018 ceph-mon[29336]: osdmap e399: 8 total, 7 up, 6 in 2024-09-10T19:04:14.723 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:14 smithi018 ceph-mon[40196]: pgmap v880: 168 pgs: 168 active+clean; 4.7 GiB data, 20 GiB used, 517 GiB / 536 GiB avail; 1.0 KiB/s rd, 83 MiB/s wr, 1.33k op/s 2024-09-10T19:04:14.723 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:14 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1924216585' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:04:14.723 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:14 smithi018 ceph-mon[40196]: osdmap e399: 8 total, 7 up, 6 in 2024-09-10T19:04:14.726 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:14.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:14.727 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:14.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:14 smithi103 ceph-mon[33163]: pgmap v880: 168 pgs: 168 active+clean; 4.7 GiB data, 20 GiB used, 517 GiB / 536 GiB avail; 1.0 KiB/s rd, 83 MiB/s wr, 1.33k op/s 2024-09-10T19:04:14.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:14 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1924216585' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:04:14.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:14 smithi103 ceph-mon[33163]: osdmap e399: 8 total, 7 up, 6 in 2024-09-10T19:04:14.828 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:14.850 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:14.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:14.851 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:14.952 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:15.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:15.007 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:15.007 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:15.073 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 47 16 82658 82642 109.88 85.6875 0.0210682 0.00909605 2024-09-10T19:04:15.107 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:15.135 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:15.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:15.136 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:15.237 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:15.267 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:15.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:15.268 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:15.369 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:15.412 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:15.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:15.413 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:15.514 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:15.562 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:15.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:15.563 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:15.664 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:15.690 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:15.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:15.722 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:15.823 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:15.858 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:15.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:15.859 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:15.905 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:04:15.913 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-10T19:04:15.914 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-10T19:04:15.960 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:16.011 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:16.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:16.012 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:16.073 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 48 16 84150 84134 109.533 93.25 0.0121903 0.00912491 2024-09-10T19:04:16.113 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:16.153 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:16.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:16.154 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:16.255 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:16.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:16.285 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:16.285 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:16.386 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:16.426 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:16.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:16.427 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:16.528 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:16.568 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:16.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:16.569 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:16.670 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:16.715 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:16.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:16.716 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:16.765 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:16 smithi018 ceph-mon[29336]: pgmap v882: 168 pgs: 168 active+clean; 4.8 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 639 B/s rd, 58 MiB/s wr, 927 op/s 2024-09-10T19:04:16.765 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:16 smithi018 ceph-mon[40196]: pgmap v882: 168 pgs: 168 active+clean; 4.8 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 639 B/s rd, 58 MiB/s wr, 927 op/s 2024-09-10T19:04:16.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:16 smithi103 ceph-mon[33163]: pgmap v882: 168 pgs: 168 active+clean; 4.8 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 639 B/s rd, 58 MiB/s wr, 927 op/s 2024-09-10T19:04:16.817 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:16.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:16.862 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:16.862 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:16.963 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:16.988 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:16.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:16.989 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:17.073 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 49 16 85479 85463 108.993 83.0625 0.0147059 0.00917033 2024-09-10T19:04:17.090 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:17.128 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:17.129 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T19:04:17.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:17.130 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:17.142 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-09-10T19:04:17.231 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:17.259 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:17.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:17.260 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:17.361 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:17.391 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:17.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:17.392 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:17.493 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:17.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:17.520 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:17.521 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:17.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:04:17 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:04:17.622 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:17.651 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:17.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:17.652 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:17.753 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:17.794 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:17.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:17.796 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:17.896 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:17.915 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T19:04:17.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:17.935 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:17.936 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:18.037 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:18.073 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 50 16 86796 86780 108.459 82.3125 0.0113863 0.00921491 2024-09-10T19:04:18.091 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:18.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:18.092 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:18.192 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:18.207 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":882,"stamp":"2024-09-10T19:04:15.454131+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5103902398,"num_objects":78066,"num_object_clones":0,"num_object_copies":234198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78066,"num_whiteouts":0,"num_read":1341,"num_read_kb":1096,"num_write":78589,"num_write_kb":4983772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":303369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":60947,"ondisk_log_size":60947,"up":504,"acting":504,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":507,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":562470912,"kb_used":20933708,"kb_used_data":15558156,"kb_used_omap":12,"kb_used_meta":5375475,"kb_avail":541537204,"statfs":{"total":575970213888,"available":554534096896,"internally_reserved":0,"allocated":15931551744,"data_stored":15928455690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13122,"internal_metadata":5504486590},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,8,7,8,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":486473728,"num_objects":7423,"num_object_clones":0,"num_object_copies":22269,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7423,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":7423,"num_write_kb":475072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.004907"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":411,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.803181+0000","last_change":"2024-09-10T19:02:21.219606+0000","last_active":"2024-09-10T19:04:03.803181+0000","last_peered":"2024-09-10T19:04:03.803181+0000","last_clean":"2024-09-10T19:04:03.803181+0000","last_became_active":"2024-09-10T18:57:02.460451+0000","last_became_peered":"2024-09-10T18:57:02.460451+0000","last_unstale":"2024-09-10T19:04:03.803181+0000","last_undegraded":"2024-09-10T19:04:03.803181+0000","last_fullsized":"2024-09-10T19:04:03.803181+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:21.219520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:02:21.219520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056254590000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":409,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:09.801742+0000","last_change":"2024-09-10T19:03:38.825020+0000","last_active":"2024-09-10T19:04:09.801742+0000","last_peered":"2024-09-10T19:04:09.801742+0000","last_clean":"2024-09-10T19:04:09.801742+0000","last_became_active":"2024-09-10T18:57:02.440648+0000","last_became_peered":"2024-09-10T18:57:02.440648+0000","last_unstale":"2024-09-10T19:04:09.801742+0000","last_undegraded":"2024-09-10T19:04:09.801742+0000","last_fullsized":"2024-09-10T19:04:09.801742+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:38.824816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:59.848700+0000","last_clean_scrub_stamp":"2024-09-10T19:03:38.824816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:59.151599+0000","scrub_duration":0.0071402059999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"232'27","reported_seq":496,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.803491+0000","last_change":"2024-09-10T19:02:20.251547+0000","last_active":"2024-09-10T19:04:03.803491+0000","last_peered":"2024-09-10T19:04:03.803491+0000","last_clean":"2024-09-10T19:04:03.803491+0000","last_became_active":"2024-09-10T18:55:40.394717+0000","last_became_peered":"2024-09-10T18:55:40.394717+0000","last_unstale":"2024-09-10T19:04:03.803491+0000","last_undegraded":"2024-09-10T19:04:03.803491+0000","last_fullsized":"2024-09-10T19:04:03.803491+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'27","last_scrub_stamp":"2024-09-10T19:02:20.251456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:20.251456+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011781700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":403,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.800504+0000","last_change":"2024-09-10T19:00:08.841172+0000","last_active":"2024-09-10T19:04:03.800504+0000","last_peered":"2024-09-10T19:04:03.800504+0000","last_clean":"2024-09-10T19:04:03.800504+0000","last_became_active":"2024-09-10T18:45:22.065619+0000","last_became_peered":"2024-09-10T18:45:22.065619+0000","last_unstale":"2024-09-10T19:04:03.800504+0000","last_undegraded":"2024-09-10T19:04:03.800504+0000","last_fullsized":"2024-09-10T19:04:03.800504+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:08.841125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.057932+0000","last_clean_scrub_stamp":"2024-09-10T19:00:08.841125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001059857,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":474,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.805153+0000","last_change":"2024-09-10T19:02:17.284387+0000","last_active":"2024-09-10T19:04:03.805153+0000","last_peered":"2024-09-10T19:04:03.805153+0000","last_clean":"2024-09-10T19:04:03.805153+0000","last_became_active":"2024-09-10T19:00:42.518824+0000","last_became_peered":"2024-09-10T19:00:42.518824+0000","last_unstale":"2024-09-10T19:04:03.805153+0000","last_undegraded":"2024-09-10T19:04:03.805153+0000","last_fullsized":"2024-09-10T19:04:03.805153+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:17.284305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:17.284305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018984430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"232'19","reported_seq":424,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.800433+0000","last_change":"2024-09-10T18:59:09.642743+0000","last_active":"2024-09-10T19:04:03.800433+0000","last_peered":"2024-09-10T19:04:03.800433+0000","last_clean":"2024-09-10T19:04:03.800433+0000","last_became_active":"2024-09-10T18:45:20.061266+0000","last_became_peered":"2024-09-10T18:45:20.061266+0000","last_unstale":"2024-09-10T19:04:03.800433+0000","last_undegraded":"2024-09-10T19:04:03.800433+0000","last_fullsized":"2024-09-10T19:04:03.800433+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T18:59:09.642642+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_clean_scrub_stamp":"2024-09-10T18:59:09.642642+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0095920940000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":39,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.806978+0000","last_change":"2024-09-10T19:02:49.572248+0000","last_active":"2024-09-10T19:04:03.806978+0000","last_peered":"2024-09-10T19:04:03.806978+0000","last_clean":"2024-09-10T19:04:03.806978+0000","last_became_active":"2024-09-10T19:02:49.571992+0000","last_became_peered":"2024-09-10T19:02:49.571992+0000","last_unstale":"2024-09-10T19:04:03.806978+0000","last_undegraded":"2024-09-10T19:04:03.806978+0000","last_fullsized":"2024-09-10T19:04:03.806978+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:01:54.384604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:01.135988+0000","last_clean_scrub_stamp":"2024-09-10T19:01:54.384604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002631478,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":401,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.800706+0000","last_change":"2024-09-10T19:03:41.617846+0000","last_active":"2024-09-10T19:04:03.800706+0000","last_peered":"2024-09-10T19:04:03.800706+0000","last_clean":"2024-09-10T19:04:03.800706+0000","last_became_active":"2024-09-10T18:45:24.081002+0000","last_became_peered":"2024-09-10T18:45:24.081002+0000","last_unstale":"2024-09-10T19:04:03.800706+0000","last_undegraded":"2024-09-10T19:04:03.800706+0000","last_fullsized":"2024-09-10T19:04:03.800706+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:41.617790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:03:41.617790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:58.995171+0000","scrub_duration":0.0014252819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":406,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.810365+0000","last_change":"2024-09-10T19:03:30.120786+0000","last_active":"2024-09-10T19:04:03.810365+0000","last_peered":"2024-09-10T19:04:03.810365+0000","last_clean":"2024-09-10T19:04:03.810365+0000","last_became_active":"2024-09-10T18:57:02.452259+0000","last_became_peered":"2024-09-10T18:57:02.452259+0000","last_unstale":"2024-09-10T19:04:03.810365+0000","last_undegraded":"2024-09-10T19:04:03.810365+0000","last_fullsized":"2024-09-10T19:04:03.810365+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:30.120695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:21.501984+0000","last_clean_scrub_stamp":"2024-09-10T19:03:30.120695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:46.598921+0000","scrub_duration":0.0034594489999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1b","version":"232'9","reported_seq":410,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.801666+0000","last_change":"2024-09-10T18:59:11.622597+0000","last_active":"2024-09-10T19:04:03.801666+0000","last_peered":"2024-09-10T19:04:03.801666+0000","last_clean":"2024-09-10T19:04:03.801666+0000","last_became_active":"2024-09-10T18:45:20.057567+0000","last_became_peered":"2024-09-10T18:45:20.057567+0000","last_unstale":"2024-09-10T19:04:03.801666+0000","last_undegraded":"2024-09-10T19:04:03.801666+0000","last_fullsized":"2024-09-10T19:04:03.801666+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'9","last_scrub_stamp":"2024-09-10T18:59:11.622509+0000","last_deep_scrub":"163'7","last_deep_scrub_stamp":"2024-09-10T18:51:54.261879+0000","last_clean_scrub_stamp":"2024-09-10T18:59:11.622509+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056120609999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":162,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.800635+0000","last_change":"2024-09-10T19:00:40.947163+0000","last_active":"2024-09-10T19:04:03.800635+0000","last_peered":"2024-09-10T19:04:03.800635+0000","last_clean":"2024-09-10T19:04:03.800635+0000","last_became_active":"2024-09-10T18:58:58.503924+0000","last_became_peered":"2024-09-10T18:58:58.503924+0000","last_unstale":"2024-09-10T19:04:03.800635+0000","last_undegraded":"2024-09-10T19:04:03.800635+0000","last_fullsized":"2024-09-10T19:04:03.800635+0000","mapping_epoch":303,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":304,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:40.947066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:36.509971+0000","last_clean_scrub_stamp":"2024-09-10T19:00:40.947066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012293715,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":79,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.807046+0000","last_change":"2024-09-10T19:03:23.759714+0000","last_active":"2024-09-10T19:04:03.807046+0000","last_peered":"2024-09-10T19:04:03.807046+0000","last_clean":"2024-09-10T19:04:03.807046+0000","last_became_active":"2024-09-10T19:00:42.515840+0000","last_became_peered":"2024-09-10T19:00:42.515840+0000","last_unstale":"2024-09-10T19:04:03.807046+0000","last_undegraded":"2024-09-10T19:04:03.807046+0000","last_fullsized":"2024-09-10T19:04:03.807046+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:23.759624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:03:23.759624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:52.320552+0000","scrub_duration":0.0037605970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":212,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.806632+0000","last_change":"2024-09-10T19:02:12.656894+0000","last_active":"2024-09-10T19:04:03.806632+0000","last_peered":"2024-09-10T19:04:03.806632+0000","last_clean":"2024-09-10T19:04:03.806632+0000","last_became_active":"2024-09-10T18:57:02.440497+0000","last_became_peered":"2024-09-10T18:57:02.440497+0000","last_unstale":"2024-09-10T19:04:03.806632+0000","last_undegraded":"2024-09-10T19:04:03.806632+0000","last_fullsized":"2024-09-10T19:04:03.806632+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:12.656811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:12.656811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0082345850000000009,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"232'15","reported_seq":459,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.801016+0000","last_change":"2024-09-10T19:02:22.113293+0000","last_active":"2024-09-10T19:04:03.801016+0000","last_peered":"2024-09-10T19:04:03.801016+0000","last_clean":"2024-09-10T19:04:03.801016+0000","last_became_active":"2024-09-10T18:55:40.391571+0000","last_became_peered":"2024-09-10T18:55:40.391571+0000","last_unstale":"2024-09-10T19:04:03.801016+0000","last_undegraded":"2024-09-10T19:04:03.801016+0000","last_fullsized":"2024-09-10T19:04:03.801016+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T19:02:22.113140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:22.113140+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0061707400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":442,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.805519+0000","last_change":"2024-09-10T19:02:55.736359+0000","last_active":"2024-09-10T19:04:03.805519+0000","last_peered":"2024-09-10T19:04:03.805519+0000","last_clean":"2024-09-10T19:04:03.805519+0000","last_became_active":"2024-09-10T18:57:02.431585+0000","last_became_peered":"2024-09-10T18:57:02.431585+0000","last_unstale":"2024-09-10T19:04:03.805519+0000","last_undegraded":"2024-09-10T19:04:03.805519+0000","last_fullsized":"2024-09-10T19:04:03.805519+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:55.736308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:02:55.736308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012293950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":98,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:09.796498+0000","last_change":"2024-09-10T19:02:45.952977+0000","last_active":"2024-09-10T19:04:09.796498+0000","last_peered":"2024-09-10T19:04:09.796498+0000","last_clean":"2024-09-10T19:04:09.796498+0000","last_became_active":"2024-09-10T18:59:48.737788+0000","last_became_peered":"2024-09-10T18:59:48.737788+0000","last_unstale":"2024-09-10T19:04:09.796498+0000","last_undegraded":"2024-09-10T19:04:09.796498+0000","last_fullsized":"2024-09-10T19:04:09.796498+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:45.952815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:59.425718+0000","last_clean_scrub_stamp":"2024-09-10T19:02:45.952815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.043052992999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"398'4952","reported_seq":4978,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:11.145321+0000","last_change":"2024-09-10T19:03:16.041580+0000","last_active":"2024-09-10T19:04:11.145321+0000","last_peered":"2024-09-10T19:04:11.145321+0000","last_clean":"2024-09-10T19:04:11.145321+0000","last_became_active":"2024-09-10T19:03:16.041295+0000","last_became_peered":"2024-09-10T19:03:16.041295+0000","last_unstale":"2024-09-10T19:04:11.145321+0000","last_undegraded":"2024-09-10T19:04:11.145321+0000","last_fullsized":"2024-09-10T19:04:11.145321+0000","mapping_epoch":379,"log_start":"386'1400","ondisk_log_start":"386'1400","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3552,"log_dups_size":0,"ondisk_log_size":3552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:24.799591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":324534272,"num_objects":4952,"num_object_clones":0,"num_object_copies":14856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4952,"num_write_kb":316928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"70'81","reported_seq":83,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.806287+0000","last_change":"2024-09-10T19:03:21.755540+0000","last_active":"2024-09-10T19:04:03.806287+0000","last_peered":"2024-09-10T19:04:03.806287+0000","last_clean":"2024-09-10T19:04:03.806287+0000","last_became_active":"2024-09-10T19:00:42.517072+0000","last_became_peered":"2024-09-10T19:00:42.517072+0000","last_unstale":"2024-09-10T19:04:03.806287+0000","last_undegraded":"2024-09-10T19:04:03.806287+0000","last_fullsized":"2024-09-10T19:04:03.806287+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:03:21.755481+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:03:21.755481+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:26.030496+0000","scrub_duration":0.0016110499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":262688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"232'39","reported_seq":490,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.806198+0000","last_change":"2024-09-10T18:59:47.745859+0000","last_active":"2024-09-10T19:04:03.806198+0000","last_peered":"2024-09-10T19:04:03.806198+0000","last_clean":"2024-09-10T19:04:03.806198+0000","last_became_active":"2024-09-10T18:47:36.409300+0000","last_became_peered":"2024-09-10T18:47:36.409300+0000","last_unstale":"2024-09-10T19:04:03.806198+0000","last_undegraded":"2024-09-10T19:04:03.806198+0000","last_fullsized":"2024-09-10T19:04:03.806198+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'39","last_scrub_stamp":"2024-09-10T18:59:47.745766+0000","last_deep_scrub":"78'25","last_deep_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_clean_scrub_stamp":"2024-09-10T18:59:47.745766+0000","objects_scrubbed":8,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011279000000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":149,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.801811+0000","last_change":"2024-09-10T19:00:09.867278+0000","last_active":"2024-09-10T19:04:03.801811+0000","last_peered":"2024-09-10T19:04:03.801811+0000","last_clean":"2024-09-10T19:04:03.801811+0000","last_became_active":"2024-09-10T18:57:02.437260+0000","last_became_peered":"2024-09-10T18:57:02.437260+0000","last_unstale":"2024-09-10T19:04:03.801811+0000","last_undegraded":"2024-09-10T19:04:03.801811+0000","last_fullsized":"2024-09-10T19:04:03.801811+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:09.867183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:00:09.867183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026871460000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":266,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.803516+0000","last_change":"2024-09-10T19:02:49.594351+0000","last_active":"2024-09-10T19:04:03.803516+0000","last_peered":"2024-09-10T19:04:03.803516+0000","last_clean":"2024-09-10T19:04:03.803516+0000","last_became_active":"2024-09-10T19:02:49.594088+0000","last_became_peered":"2024-09-10T19:02:49.594088+0000","last_unstale":"2024-09-10T19:04:03.803516+0000","last_undegraded":"2024-09-10T19:04:03.803516+0000","last_fullsized":"2024-09-10T19:04:03.803516+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:09.470293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:04.512425+0000","last_clean_scrub_stamp":"2024-09-10T19:00:09.470293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016498730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":321,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:09.801883+0000","last_change":"2024-09-10T19:02:58.770406+0000","last_active":"2024-09-10T19:04:09.801883+0000","last_peered":"2024-09-10T19:04:09.801883+0000","last_clean":"2024-09-10T19:04:09.801883+0000","last_became_active":"2024-09-10T18:55:40.388154+0000","last_became_peered":"2024-09-10T18:55:40.388154+0000","last_unstale":"2024-09-10T19:04:09.801883+0000","last_undegraded":"2024-09-10T19:04:09.801883+0000","last_fullsized":"2024-09-10T19:04:09.801883+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:58.770344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:02:58.770344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:17.389406+0000","scrub_duration":0.0016377200000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3","version":"398'4798","reported_seq":4825,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:11.563264+0000","last_change":"2024-09-10T19:03:16.046339+0000","last_active":"2024-09-10T19:04:11.563264+0000","last_peered":"2024-09-10T19:04:11.563264+0000","last_clean":"2024-09-10T19:04:11.563264+0000","last_became_active":"2024-09-10T19:03:16.045964+0000","last_became_peered":"2024-09-10T19:03:16.045964+0000","last_unstale":"2024-09-10T19:04:11.563264+0000","last_undegraded":"2024-09-10T19:04:11.563264+0000","last_fullsized":"2024-09-10T19:04:11.563264+0000","mapping_epoch":379,"log_start":"386'1300","ondisk_log_start":"386'1300","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3498,"log_dups_size":0,"ondisk_log_size":3498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:30.109053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":314441728,"num_objects":4798,"num_object_clones":0,"num_object_copies":14394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4798,"num_write_kb":307072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.9","version":"70'81","reported_seq":305,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.804007+0000","last_change":"2024-09-10T18:59:56.164487+0000","last_active":"2024-09-10T19:04:03.804007+0000","last_peered":"2024-09-10T19:04:03.804007+0000","last_clean":"2024-09-10T19:04:03.804007+0000","last_became_active":"2024-09-10T18:57:02.463750+0000","last_became_peered":"2024-09-10T18:57:02.463750+0000","last_unstale":"2024-09-10T19:04:03.804007+0000","last_undegraded":"2024-09-10T19:04:03.804007+0000","last_fullsized":"2024-09-10T19:04:03.804007+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:56.164384+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:59:56.164384+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041056720000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"232'25","reported_seq":440,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.803688+0000","last_change":"2024-09-10T19:03:34.532084+0000","last_active":"2024-09-10T19:04:03.803688+0000","last_peered":"2024-09-10T19:04:03.803688+0000","last_clean":"2024-09-10T19:04:03.803688+0000","last_became_active":"2024-09-10T18:45:20.053428+0000","last_became_peered":"2024-09-10T18:45:20.053428+0000","last_unstale":"2024-09-10T19:04:03.803688+0000","last_undegraded":"2024-09-10T19:04:03.803688+0000","last_fullsized":"2024-09-10T19:04:03.803688+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'25","last_scrub_stamp":"2024-09-10T19:03:34.532019+0000","last_deep_scrub":"116'15","last_deep_scrub_stamp":"2024-09-10T18:49:12.374527+0000","last_clean_scrub_stamp":"2024-09-10T19:03:34.532019+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:38.997474+0000","scrub_duration":0.0020774220000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":311,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.804175+0000","last_change":"2024-09-10T19:02:49.572280+0000","last_active":"2024-09-10T19:04:03.804175+0000","last_peered":"2024-09-10T19:04:03.804175+0000","last_clean":"2024-09-10T19:04:03.804175+0000","last_became_active":"2024-09-10T19:02:49.571995+0000","last_became_peered":"2024-09-10T19:02:49.571995+0000","last_unstale":"2024-09-10T19:04:03.804175+0000","last_undegraded":"2024-09-10T19:04:03.804175+0000","last_fullsized":"2024-09-10T19:04:03.804175+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:01.343214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:01.343214+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0044818080000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"273'13","reported_seq":496,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:09.802074+0000","last_change":"2024-09-10T19:01:59.557656+0000","last_active":"2024-09-10T19:04:09.802074+0000","last_peered":"2024-09-10T19:04:09.802074+0000","last_clean":"2024-09-10T19:04:09.802074+0000","last_became_active":"2024-09-10T18:57:02.448258+0000","last_became_peered":"2024-09-10T18:57:02.448258+0000","last_unstale":"2024-09-10T19:04:09.802074+0000","last_undegraded":"2024-09-10T19:04:09.802074+0000","last_fullsized":"2024-09-10T19:04:09.802074+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'13","last_scrub_stamp":"2024-09-10T19:01:59.557500+0000","last_deep_scrub":"202'10","last_deep_scrub_stamp":"2024-09-10T18:53:45.547686+0000","last_clean_scrub_stamp":"2024-09-10T19:01:59.557500+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0027644549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":441,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.805775+0000","last_change":"2024-09-10T19:00:42.515842+0000","last_active":"2024-09-10T19:04:03.805775+0000","last_peered":"2024-09-10T19:04:03.805775+0000","last_clean":"2024-09-10T19:04:03.805775+0000","last_became_active":"2024-09-10T19:00:42.515523+0000","last_became_peered":"2024-09-10T19:00:42.515523+0000","last_unstale":"2024-09-10T19:04:03.805775+0000","last_undegraded":"2024-09-10T19:04:03.805775+0000","last_fullsized":"2024-09-10T19:04:03.805775+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:07.803916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:32.043666+0000","last_clean_scrub_stamp":"2024-09-10T19:00:07.803916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0011072429999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.0","version":"398'4729","reported_seq":4755,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:11.083948+0000","last_change":"2024-09-10T19:03:16.042622+0000","last_active":"2024-09-10T19:04:11.083948+0000","last_peered":"2024-09-10T19:04:11.083948+0000","last_clean":"2024-09-10T19:04:11.083948+0000","last_became_active":"2024-09-10T19:03:16.042319+0000","last_became_peered":"2024-09-10T19:03:16.042319+0000","last_unstale":"2024-09-10T19:04:11.083948+0000","last_undegraded":"2024-09-10T19:04:11.083948+0000","last_fullsized":"2024-09-10T19:04:11.083948+0000","mapping_epoch":379,"log_start":"385'1100","ondisk_log_start":"385'1100","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3629,"log_dups_size":0,"ondisk_log_size":3629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:27.169710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":309919744,"num_objects":4729,"num_object_clones":0,"num_object_copies":14187,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4729,"num_write_kb":302656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.a","version":"70'81","reported_seq":330,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.806074+0000","last_change":"2024-09-10T19:02:23.723122+0000","last_active":"2024-09-10T19:04:03.806074+0000","last_peered":"2024-09-10T19:04:03.806074+0000","last_clean":"2024-09-10T19:04:03.806074+0000","last_became_active":"2024-09-10T18:50:08.398345+0000","last_became_peered":"2024-09-10T18:50:08.398345+0000","last_unstale":"2024-09-10T19:04:03.806074+0000","last_undegraded":"2024-09-10T19:04:03.806074+0000","last_fullsized":"2024-09-10T19:04:03.806074+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:23.723036+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:55:37.921519+0000","last_clean_scrub_stamp":"2024-09-10T19:02:23.723036+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0054938950000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"232'34","reported_seq":427,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.807644+0000","last_change":"2024-09-10T19:00:41.482710+0000","last_active":"2024-09-10T19:04:03.807644+0000","last_peered":"2024-09-10T19:04:03.807644+0000","last_clean":"2024-09-10T19:04:03.807644+0000","last_became_active":"2024-09-10T18:46:56.742946+0000","last_became_peered":"2024-09-10T18:46:56.742946+0000","last_unstale":"2024-09-10T19:04:03.807644+0000","last_undegraded":"2024-09-10T19:04:03.807644+0000","last_fullsized":"2024-09-10T19:04:03.807644+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'34","last_scrub_stamp":"2024-09-10T19:00:41.482619+0000","last_deep_scrub":"141'22","last_deep_scrub_stamp":"2024-09-10T18:50:57.119152+0000","last_clean_scrub_stamp":"2024-09-10T19:00:41.482619+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0069363309999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":314,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.803696+0000","last_change":"2024-09-10T19:02:49.563958+0000","last_active":"2024-09-10T19:04:03.803696+0000","last_peered":"2024-09-10T19:04:03.803696+0000","last_clean":"2024-09-10T19:04:03.803696+0000","last_became_active":"2024-09-10T19:02:49.563535+0000","last_became_peered":"2024-09-10T19:02:49.563535+0000","last_unstale":"2024-09-10T19:04:03.803696+0000","last_undegraded":"2024-09-10T19:04:03.803696+0000","last_fullsized":"2024-09-10T19:04:03.803696+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:51.286131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_clean_scrub_stamp":"2024-09-10T18:59:51.286131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058886299999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"242'10","reported_seq":657,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:09.797311+0000","last_change":"2024-09-10T19:02:53.030726+0000","last_active":"2024-09-10T19:04:09.797311+0000","last_peered":"2024-09-10T19:04:09.797311+0000","last_clean":"2024-09-10T19:04:09.797311+0000","last_became_active":"2024-09-10T18:55:40.397246+0000","last_became_peered":"2024-09-10T18:55:40.397246+0000","last_unstale":"2024-09-10T19:04:09.797311+0000","last_undegraded":"2024-09-10T19:04:09.797311+0000","last_fullsized":"2024-09-10T19:04:09.797311+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"242'10","last_scrub_stamp":"2024-09-10T19:02:53.030666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:02:53.030666+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015331489999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":430,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.808435+0000","last_change":"2024-09-10T19:02:48.670915+0000","last_active":"2024-09-10T19:04:03.808435+0000","last_peered":"2024-09-10T19:04:03.808435+0000","last_clean":"2024-09-10T19:04:03.808435+0000","last_became_active":"2024-09-10T18:55:40.392003+0000","last_became_peered":"2024-09-10T18:55:40.392003+0000","last_unstale":"2024-09-10T19:04:03.808435+0000","last_undegraded":"2024-09-10T19:04:03.808435+0000","last_fullsized":"2024-09-10T19:04:03.808435+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:48.670862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:54.218602+0000","last_clean_scrub_stamp":"2024-09-10T19:02:48.670862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00108365,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1","version":"398'4691","reported_seq":4718,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:11.548639+0000","last_change":"2024-09-10T19:03:15.827713+0000","last_active":"2024-09-10T19:04:11.548639+0000","last_peered":"2024-09-10T19:04:11.548639+0000","last_clean":"2024-09-10T19:04:11.548639+0000","last_became_active":"2024-09-10T19:03:15.827177+0000","last_became_peered":"2024-09-10T19:03:15.827177+0000","last_unstale":"2024-09-10T19:04:11.548639+0000","last_undegraded":"2024-09-10T19:04:11.548639+0000","last_fullsized":"2024-09-10T19:04:11.548639+0000","mapping_epoch":379,"log_start":"386'1200","ondisk_log_start":"386'1200","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3491,"log_dups_size":0,"ondisk_log_size":3491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:32.752337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":307429376,"num_objects":4691,"num_object_clones":0,"num_object_copies":14073,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4691,"num_write_kb":300224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.b","version":"70'81","reported_seq":115,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:09.798563+0000","last_change":"2024-09-10T19:03:52.822039+0000","last_active":"2024-09-10T19:04:09.798563+0000","last_peered":"2024-09-10T19:04:09.798563+0000","last_clean":"2024-09-10T19:04:09.798563+0000","last_became_active":"2024-09-10T19:02:49.572057+0000","last_became_peered":"2024-09-10T19:02:49.572057+0000","last_unstale":"2024-09-10T19:04:09.798563+0000","last_undegraded":"2024-09-10T19:04:09.798563+0000","last_fullsized":"2024-09-10T19:04:09.798563+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:17.370627+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T19:02:17.370627+0000","last_clean_scrub_stamp":"2024-09-10T19:02:17.370627+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007621203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"232'24","reported_seq":517,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.806013+0000","last_change":"2024-09-10T19:03:42.660938+0000","last_active":"2024-09-10T19:04:03.806013+0000","last_peered":"2024-09-10T19:04:03.806013+0000","last_clean":"2024-09-10T19:04:03.806013+0000","last_became_active":"2024-09-10T18:47:57.106935+0000","last_became_peered":"2024-09-10T18:47:57.106935+0000","last_unstale":"2024-09-10T19:04:03.806013+0000","last_undegraded":"2024-09-10T19:04:03.806013+0000","last_fullsized":"2024-09-10T19:04:03.806013+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T19:03:42.660792+0000","last_deep_scrub":"224'16","last_deep_scrub_stamp":"2024-09-10T18:54:49.296957+0000","last_clean_scrub_stamp":"2024-09-10T19:03:42.660792+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:05:06.026725+0000","scrub_duration":0.0053846639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":404,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:09.796636+0000","last_change":"2024-09-10T19:02:13.720500+0000","last_active":"2024-09-10T19:04:09.796636+0000","last_peered":"2024-09-10T19:04:09.796636+0000","last_clean":"2024-09-10T19:04:09.796636+0000","last_became_active":"2024-09-10T18:57:02.443816+0000","last_became_peered":"2024-09-10T18:57:02.443816+0000","last_unstale":"2024-09-10T19:04:09.796636+0000","last_undegraded":"2024-09-10T19:04:09.796636+0000","last_fullsized":"2024-09-10T19:04:09.796636+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:13.720416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:13.720416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014226275,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":418,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.807815+0000","last_change":"2024-09-10T19:02:07.623139+0000","last_active":"2024-09-10T19:04:03.807815+0000","last_peered":"2024-09-10T19:04:03.807815+0000","last_clean":"2024-09-10T19:04:03.807815+0000","last_became_active":"2024-09-10T18:45:22.068918+0000","last_became_peered":"2024-09-10T18:45:22.068918+0000","last_unstale":"2024-09-10T19:04:03.807815+0000","last_undegraded":"2024-09-10T19:04:03.807815+0000","last_fullsized":"2024-09-10T19:04:03.807815+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:07.623054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_clean_scrub_stamp":"2024-09-10T19:02:07.623054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003868888,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":401,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.804732+0000","last_change":"2024-09-10T19:03:24.593809+0000","last_active":"2024-09-10T19:04:03.804732+0000","last_peered":"2024-09-10T19:04:03.804732+0000","last_clean":"2024-09-10T19:04:03.804732+0000","last_became_active":"2024-09-10T18:45:24.264538+0000","last_became_peered":"2024-09-10T18:45:24.264538+0000","last_unstale":"2024-09-10T19:04:03.804732+0000","last_undegraded":"2024-09-10T19:04:03.804732+0000","last_fullsized":"2024-09-10T19:04:03.804732+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:24.593754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:51.398981+0000","last_clean_scrub_stamp":"2024-09-10T19:03:24.593754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:27.524766+0000","scrub_duration":0.001434999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.e","version":"398'4889","reported_seq":4916,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:11.445503+0000","last_change":"2024-09-10T19:03:16.044645+0000","last_active":"2024-09-10T19:04:11.445503+0000","last_peered":"2024-09-10T19:04:11.445503+0000","last_clean":"2024-09-10T19:04:11.445503+0000","last_became_active":"2024-09-10T19:03:16.044187+0000","last_became_peered":"2024-09-10T19:03:16.044187+0000","last_unstale":"2024-09-10T19:04:11.445503+0000","last_undegraded":"2024-09-10T19:04:11.445503+0000","last_fullsized":"2024-09-10T19:04:11.445503+0000","mapping_epoch":379,"log_start":"386'1100","ondisk_log_start":"386'1100","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3789,"log_dups_size":0,"ondisk_log_size":3789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:20.361561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":320405504,"num_objects":4889,"num_object_clones":0,"num_object_copies":14667,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4889,"num_write_kb":312896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"70'81","reported_seq":359,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.804867+0000","last_change":"2024-09-10T19:03:30.624986+0000","last_active":"2024-09-10T19:04:03.804867+0000","last_peered":"2024-09-10T19:04:03.804867+0000","last_clean":"2024-09-10T19:04:03.804867+0000","last_became_active":"2024-09-10T19:00:42.519742+0000","last_became_peered":"2024-09-10T19:00:42.519742+0000","last_unstale":"2024-09-10T19:04:03.804867+0000","last_undegraded":"2024-09-10T19:04:03.804867+0000","last_fullsized":"2024-09-10T19:04:03.804867+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:03:30.624868+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:03:30.624868+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:41.117180+0000","scrub_duration":0.0036172140000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"232'35","reported_seq":454,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.800892+0000","last_change":"2024-09-10T19:03:29.581426+0000","last_active":"2024-09-10T19:04:03.800892+0000","last_peered":"2024-09-10T19:04:03.800892+0000","last_clean":"2024-09-10T19:04:03.800892+0000","last_became_active":"2024-09-10T18:45:20.052243+0000","last_became_peered":"2024-09-10T18:45:20.052243+0000","last_unstale":"2024-09-10T19:04:03.800892+0000","last_undegraded":"2024-09-10T19:04:03.800892+0000","last_fullsized":"2024-09-10T19:04:03.800892+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'35","last_scrub_stamp":"2024-09-10T19:03:29.581339+0000","last_deep_scrub":"143'21","last_deep_scrub_stamp":"2024-09-10T18:51:02.540216+0000","last_clean_scrub_stamp":"2024-09-10T19:03:29.581339+0000","objects_scrubbed":7,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:36.823000+0000","scrub_duration":0.0042812099999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":440,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.806124+0000","last_change":"2024-09-10T19:01:55.150294+0000","last_active":"2024-09-10T19:04:03.806124+0000","last_peered":"2024-09-10T19:04:03.806124+0000","last_clean":"2024-09-10T19:04:03.806124+0000","last_became_active":"2024-09-10T18:57:02.453404+0000","last_became_peered":"2024-09-10T18:57:02.453404+0000","last_unstale":"2024-09-10T19:04:03.806124+0000","last_undegraded":"2024-09-10T19:04:03.806124+0000","last_fullsized":"2024-09-10T19:04:03.806124+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:01:55.150215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_clean_scrub_stamp":"2024-09-10T19:01:55.150215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060613990000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":637,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:06.297366+0000","last_change":"2024-09-10T19:01:56.290088+0000","last_active":"2024-09-10T19:04:06.297366+0000","last_peered":"2024-09-10T19:04:06.297366+0000","last_clean":"2024-09-10T19:04:06.297366+0000","last_became_active":"2024-09-10T18:45:22.074540+0000","last_became_peered":"2024-09-10T18:45:22.074540+0000","last_unstale":"2024-09-10T19:04:06.297366+0000","last_undegraded":"2024-09-10T19:04:06.297366+0000","last_fullsized":"2024-09-10T19:04:06.297366+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T19:01:56.289997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:01:56.289997+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040368549999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":207,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.808007+0000","last_change":"2024-09-10T19:02:10.106717+0000","last_active":"2024-09-10T19:04:03.808007+0000","last_peered":"2024-09-10T19:04:03.808007+0000","last_clean":"2024-09-10T19:04:03.808007+0000","last_became_active":"2024-09-10T18:57:02.437100+0000","last_became_peered":"2024-09-10T18:57:02.437100+0000","last_unstale":"2024-09-10T19:04:03.808007+0000","last_undegraded":"2024-09-10T19:04:03.808007+0000","last_fullsized":"2024-09-10T19:04:03.808007+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:10.106577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:02:10.106577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0035912439999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"399'5186","reported_seq":5214,"reported_epoch":399,"state":"active+clean","last_fresh":"2024-09-10T19:04:13.827221+0000","last_change":"2024-09-10T19:03:16.046832+0000","last_active":"2024-09-10T19:04:13.827221+0000","last_peered":"2024-09-10T19:04:13.827221+0000","last_clean":"2024-09-10T19:04:13.827221+0000","last_became_active":"2024-09-10T19:03:16.046538+0000","last_became_peered":"2024-09-10T19:03:16.046538+0000","last_unstale":"2024-09-10T19:04:13.827221+0000","last_undegraded":"2024-09-10T19:04:13.827221+0000","last_fullsized":"2024-09-10T19:04:13.827221+0000","mapping_epoch":379,"log_start":"386'1300","ondisk_log_start":"386'1300","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3886,"log_dups_size":0,"ondisk_log_size":3886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:43.692877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":339869696,"num_objects":5186,"num_object_clones":0,"num_object_copies":15558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5186,"num_write_kb":331904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"70'81","reported_seq":325,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.802071+0000","last_change":"2024-09-10T19:02:58.888049+0000","last_active":"2024-09-10T19:04:03.802071+0000","last_peered":"2024-09-10T19:04:03.802071+0000","last_clean":"2024-09-10T19:04:03.802071+0000","last_became_active":"2024-09-10T18:52:14.584798+0000","last_became_peered":"2024-09-10T18:52:14.584798+0000","last_unstale":"2024-09-10T19:04:03.802071+0000","last_undegraded":"2024-09-10T19:04:03.802071+0000","last_fullsized":"2024-09-10T19:04:03.802071+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:58.887944+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:56:31.481303+0000","last_clean_scrub_stamp":"2024-09-10T19:02:58.887944+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:10.772150+0000","scrub_duration":0.0016190950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"232'30","reported_seq":571,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.803446+0000","last_change":"2024-09-10T19:02:48.361438+0000","last_active":"2024-09-10T19:04:03.803446+0000","last_peered":"2024-09-10T19:04:03.803446+0000","last_clean":"2024-09-10T19:04:03.803446+0000","last_became_active":"2024-09-10T18:59:48.742377+0000","last_became_peered":"2024-09-10T18:59:48.742377+0000","last_unstale":"2024-09-10T19:04:03.803446+0000","last_undegraded":"2024-09-10T19:04:03.803446+0000","last_fullsized":"2024-09-10T19:04:03.803446+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'30","last_scrub_stamp":"2024-09-10T19:02:48.361337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:48.361337+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011213163,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":48,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.806038+0000","last_change":"2024-09-10T19:02:52.036142+0000","last_active":"2024-09-10T19:04:03.806038+0000","last_peered":"2024-09-10T19:04:03.806038+0000","last_clean":"2024-09-10T19:04:03.806038+0000","last_became_active":"2024-09-10T19:02:49.578870+0000","last_became_peered":"2024-09-10T19:02:49.578870+0000","last_unstale":"2024-09-10T19:04:03.806038+0000","last_undegraded":"2024-09-10T19:04:03.806038+0000","last_fullsized":"2024-09-10T19:04:03.806038+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:52.036077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:46.142129+0000","last_clean_scrub_stamp":"2024-09-10T19:02:52.036077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015638970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":413,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.800711+0000","last_change":"2024-09-10T18:59:57.903394+0000","last_active":"2024-09-10T19:04:03.800711+0000","last_peered":"2024-09-10T19:04:03.800711+0000","last_clean":"2024-09-10T19:04:03.800711+0000","last_became_active":"2024-09-10T18:57:02.448126+0000","last_became_peered":"2024-09-10T18:57:02.448126+0000","last_unstale":"2024-09-10T19:04:03.800711+0000","last_undegraded":"2024-09-10T19:04:03.800711+0000","last_fullsized":"2024-09-10T19:04:03.800711+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:57.903311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:57.903311+0000","last_clean_scrub_stamp":"2024-09-10T18:59:57.903311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011228738,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":411,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.800970+0000","last_change":"2024-09-10T19:03:36.541808+0000","last_active":"2024-09-10T19:04:03.800970+0000","last_peered":"2024-09-10T19:04:03.800970+0000","last_clean":"2024-09-10T19:04:03.800970+0000","last_became_active":"2024-09-10T18:47:36.410836+0000","last_became_peered":"2024-09-10T18:47:36.410836+0000","last_unstale":"2024-09-10T19:04:03.800970+0000","last_undegraded":"2024-09-10T19:04:03.800970+0000","last_fullsized":"2024-09-10T19:04:03.800970+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:36.541729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_clean_scrub_stamp":"2024-09-10T19:03:36.541729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:42.070356+0000","scrub_duration":0.001932762,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"398'4884","reported_seq":4911,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:11.437176+0000","last_change":"2024-09-10T19:03:16.046878+0000","last_active":"2024-09-10T19:04:11.437176+0000","last_peered":"2024-09-10T19:04:11.437176+0000","last_clean":"2024-09-10T19:04:11.437176+0000","last_became_active":"2024-09-10T19:03:16.046484+0000","last_became_peered":"2024-09-10T19:03:16.046484+0000","last_unstale":"2024-09-10T19:04:11.437176+0000","last_undegraded":"2024-09-10T19:04:11.437176+0000","last_fullsized":"2024-09-10T19:04:11.437176+0000","mapping_epoch":379,"log_start":"386'1100","ondisk_log_start":"386'1100","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3784,"log_dups_size":0,"ondisk_log_size":3784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:43.162682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":320077824,"num_objects":4884,"num_object_clones":0,"num_object_copies":14652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4884,"num_write_kb":312576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"70'81","reported_seq":423,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.805921+0000","last_change":"2024-09-10T19:02:21.983253+0000","last_active":"2024-09-10T19:04:03.805921+0000","last_peered":"2024-09-10T19:04:03.805921+0000","last_clean":"2024-09-10T19:04:03.805921+0000","last_became_active":"2024-09-10T18:55:40.396828+0000","last_became_peered":"2024-09-10T18:55:40.396828+0000","last_unstale":"2024-09-10T19:04:03.805921+0000","last_undegraded":"2024-09-10T19:04:03.805921+0000","last_fullsized":"2024-09-10T19:04:03.805921+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:21.983118+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:02:21.983118+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056724799999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"232'29","reported_seq":466,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:09.798824+0000","last_change":"2024-09-10T19:02:53.679893+0000","last_active":"2024-09-10T19:04:09.798824+0000","last_peered":"2024-09-10T19:04:09.798824+0000","last_clean":"2024-09-10T19:04:09.798824+0000","last_became_active":"2024-09-10T18:57:02.442862+0000","last_became_peered":"2024-09-10T18:57:02.442862+0000","last_unstale":"2024-09-10T19:04:09.798824+0000","last_undegraded":"2024-09-10T19:04:09.798824+0000","last_fullsized":"2024-09-10T19:04:09.798824+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'29","last_scrub_stamp":"2024-09-10T19:02:53.679834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:53.679834+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024571129999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":404,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:09.800252+0000","last_change":"2024-09-10T19:03:28.771956+0000","last_active":"2024-09-10T19:04:09.800252+0000","last_peered":"2024-09-10T19:04:09.800252+0000","last_clean":"2024-09-10T19:04:09.800252+0000","last_became_active":"2024-09-10T18:57:02.443522+0000","last_became_peered":"2024-09-10T18:57:02.443522+0000","last_unstale":"2024-09-10T19:04:09.800252+0000","last_undegraded":"2024-09-10T19:04:09.800252+0000","last_fullsized":"2024-09-10T19:04:09.800252+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:28.771864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:48.196065+0000","last_clean_scrub_stamp":"2024-09-10T19:03:28.771864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:33.555178+0000","scrub_duration":0.0070868659999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":433,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.807015+0000","last_change":"2024-09-10T19:02:17.064956+0000","last_active":"2024-09-10T19:04:03.807015+0000","last_peered":"2024-09-10T19:04:03.807015+0000","last_clean":"2024-09-10T19:04:03.807015+0000","last_became_active":"2024-09-10T18:55:40.387857+0000","last_became_peered":"2024-09-10T18:55:40.387857+0000","last_unstale":"2024-09-10T19:04:03.807015+0000","last_undegraded":"2024-09-10T19:04:03.807015+0000","last_fullsized":"2024-09-10T19:04:03.807015+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:17.064877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:02:17.064877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0089490170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":316,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.807921+0000","last_change":"2024-09-10T19:02:54.957936+0000","last_active":"2024-09-10T19:04:03.807921+0000","last_peered":"2024-09-10T19:04:03.807921+0000","last_clean":"2024-09-10T19:04:03.807921+0000","last_became_active":"2024-09-10T19:02:49.561359+0000","last_became_peered":"2024-09-10T19:02:49.561359+0000","last_unstale":"2024-09-10T19:04:03.807921+0000","last_undegraded":"2024-09-10T19:04:03.807921+0000","last_fullsized":"2024-09-10T19:04:03.807921+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:54.957846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:49:18.281230+0000","last_clean_scrub_stamp":"2024-09-10T19:02:54.957846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:10.717373+0000","scrub_duration":0.0015003740000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"398'4876","reported_seq":4903,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:11.448664+0000","last_change":"2024-09-10T19:03:16.044601+0000","last_active":"2024-09-10T19:04:11.448664+0000","last_peered":"2024-09-10T19:04:11.448664+0000","last_clean":"2024-09-10T19:04:11.448664+0000","last_became_active":"2024-09-10T19:03:16.044213+0000","last_became_peered":"2024-09-10T19:03:16.044213+0000","last_unstale":"2024-09-10T19:04:11.448664+0000","last_undegraded":"2024-09-10T19:04:11.448664+0000","last_fullsized":"2024-09-10T19:04:11.448664+0000","mapping_epoch":379,"log_start":"385'1100","ondisk_log_start":"385'1100","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3776,"log_dups_size":0,"ondisk_log_size":3776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:20.548149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319553536,"num_objects":4876,"num_object_clones":0,"num_object_copies":14628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4876,"num_write_kb":312064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"70'81","reported_seq":426,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.806586+0000","last_change":"2024-09-10T19:03:34.083497+0000","last_active":"2024-09-10T19:04:03.806586+0000","last_peered":"2024-09-10T19:04:03.806586+0000","last_clean":"2024-09-10T19:04:03.806586+0000","last_became_active":"2024-09-10T18:57:02.452659+0000","last_became_peered":"2024-09-10T18:57:02.452659+0000","last_unstale":"2024-09-10T19:04:03.806586+0000","last_undegraded":"2024-09-10T19:04:03.806586+0000","last_fullsized":"2024-09-10T19:04:03.806586+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:03:34.083013+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:03:34.083013+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:35.239103+0000","scrub_duration":0.0023839249999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"232'25","reported_seq":484,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.805332+0000","last_change":"2024-09-10T19:03:52.017990+0000","last_active":"2024-09-10T19:04:03.805332+0000","last_peered":"2024-09-10T19:04:03.805332+0000","last_clean":"2024-09-10T19:04:03.805332+0000","last_became_active":"2024-09-10T19:02:49.579401+0000","last_became_peered":"2024-09-10T19:02:49.579401+0000","last_unstale":"2024-09-10T19:04:03.805332+0000","last_undegraded":"2024-09-10T19:04:03.805332+0000","last_fullsized":"2024-09-10T19:04:03.805332+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'25","last_scrub_stamp":"2024-09-10T19:03:52.017902+0000","last_deep_scrub":"83'15","last_deep_scrub_stamp":"2024-09-10T18:46:58.436473+0000","last_clean_scrub_stamp":"2024-09-10T19:03:52.017902+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:05:21.018461+0000","scrub_duration":0.0024815190000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":330,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:09.800789+0000","last_change":"2024-09-10T19:02:49.561636+0000","last_active":"2024-09-10T19:04:09.800789+0000","last_peered":"2024-09-10T19:04:09.800789+0000","last_clean":"2024-09-10T19:04:09.800789+0000","last_became_active":"2024-09-10T19:02:49.561340+0000","last_became_peered":"2024-09-10T19:02:49.561340+0000","last_unstale":"2024-09-10T19:04:09.800789+0000","last_undegraded":"2024-09-10T19:04:09.800789+0000","last_fullsized":"2024-09-10T19:04:09.800789+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:21.266828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:05.140027+0000","last_clean_scrub_stamp":"2024-09-10T19:02:21.266828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034952049999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"339'13","reported_seq":611,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:11.298362+0000","last_change":"2024-09-10T19:03:29.753145+0000","last_active":"2024-09-10T19:04:11.298362+0000","last_peered":"2024-09-10T19:04:11.298362+0000","last_clean":"2024-09-10T19:04:11.298362+0000","last_became_active":"2024-09-10T19:00:42.520360+0000","last_became_peered":"2024-09-10T19:00:42.520360+0000","last_unstale":"2024-09-10T19:04:11.298362+0000","last_undegraded":"2024-09-10T19:04:11.298362+0000","last_fullsized":"2024-09-10T19:04:11.298362+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"339'13","last_scrub_stamp":"2024-09-10T19:03:29.753059+0000","last_deep_scrub":"258'12","last_deep_scrub_stamp":"2024-09-10T18:56:21.098770+0000","last_clean_scrub_stamp":"2024-09-10T19:03:29.753059+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:48.781650+0000","scrub_duration":0.002665485,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":390,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:09.796945+0000","last_change":"2024-09-10T19:02:14.738026+0000","last_active":"2024-09-10T19:04:09.796945+0000","last_peered":"2024-09-10T19:04:09.796945+0000","last_clean":"2024-09-10T19:04:09.796945+0000","last_became_active":"2024-09-10T19:00:42.515144+0000","last_became_peered":"2024-09-10T19:00:42.515144+0000","last_unstale":"2024-09-10T19:04:09.796945+0000","last_undegraded":"2024-09-10T19:04:09.796945+0000","last_fullsized":"2024-09-10T19:04:09.796945+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:14.737936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:02:14.737936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003039609,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"398'4805","reported_seq":4831,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:11.136260+0000","last_change":"2024-09-10T19:03:16.048706+0000","last_active":"2024-09-10T19:04:11.136260+0000","last_peered":"2024-09-10T19:04:11.136260+0000","last_clean":"2024-09-10T19:04:11.136260+0000","last_became_active":"2024-09-10T19:03:16.048344+0000","last_became_peered":"2024-09-10T19:03:16.048344+0000","last_unstale":"2024-09-10T19:04:11.136260+0000","last_undegraded":"2024-09-10T19:04:11.136260+0000","last_fullsized":"2024-09-10T19:04:11.136260+0000","mapping_epoch":379,"log_start":"386'1200","ondisk_log_start":"386'1200","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3605,"log_dups_size":0,"ondisk_log_size":3605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:15.964765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":314900480,"num_objects":4805,"num_object_clones":0,"num_object_copies":14415,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4805,"num_write_kb":307520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"70'81","reported_seq":347,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.806711+0000","last_change":"2024-09-10T19:00:40.472680+0000","last_active":"2024-09-10T19:04:03.806711+0000","last_peered":"2024-09-10T19:04:03.806711+0000","last_clean":"2024-09-10T19:04:03.806711+0000","last_became_active":"2024-09-10T18:55:40.392432+0000","last_became_peered":"2024-09-10T18:55:40.392432+0000","last_unstale":"2024-09-10T19:04:03.806711+0000","last_undegraded":"2024-09-10T19:04:03.806711+0000","last_fullsized":"2024-09-10T19:04:03.806711+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:00:40.472595+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:00:40.472595+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0072897099999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"373'42","reported_seq":492,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:06.297556+0000","last_change":"2024-09-10T19:02:49.563856+0000","last_active":"2024-09-10T19:04:06.297556+0000","last_peered":"2024-09-10T19:04:06.297556+0000","last_clean":"2024-09-10T19:04:06.297556+0000","last_became_active":"2024-09-10T19:02:49.563402+0000","last_became_peered":"2024-09-10T19:02:49.563402+0000","last_unstale":"2024-09-10T19:04:06.297556+0000","last_undegraded":"2024-09-10T19:04:06.297556+0000","last_fullsized":"2024-09-10T19:04:06.297556+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'41","last_scrub_stamp":"2024-09-10T19:02:12.270610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:12.270610+0000","objects_scrubbed":10,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012827955,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":422,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.805415+0000","last_change":"2024-09-10T19:02:13.666158+0000","last_active":"2024-09-10T19:04:03.805415+0000","last_peered":"2024-09-10T19:04:03.805415+0000","last_clean":"2024-09-10T19:04:03.805415+0000","last_became_active":"2024-09-10T18:45:17.964494+0000","last_became_peered":"2024-09-10T18:45:17.964494+0000","last_unstale":"2024-09-10T19:04:03.805415+0000","last_undegraded":"2024-09-10T19:04:03.805415+0000","last_fullsized":"2024-09-10T19:04:03.805415+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:13.665999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:12.982599+0000","last_clean_scrub_stamp":"2024-09-10T19:02:13.665999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007648719,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":364,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.808424+0000","last_change":"2024-09-10T19:02:01.110434+0000","last_active":"2024-09-10T19:04:03.808424+0000","last_peered":"2024-09-10T19:04:03.808424+0000","last_clean":"2024-09-10T19:04:03.808424+0000","last_became_active":"2024-09-10T18:47:57.107478+0000","last_became_peered":"2024-09-10T18:47:57.107478+0000","last_unstale":"2024-09-10T19:04:03.808424+0000","last_undegraded":"2024-09-10T19:04:03.808424+0000","last_fullsized":"2024-09-10T19:04:03.808424+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:01.110335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:02:01.110335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0062803349999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":401,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.804984+0000","last_change":"2024-09-10T19:02:09.267386+0000","last_active":"2024-09-10T19:04:03.804984+0000","last_peered":"2024-09-10T19:04:03.804984+0000","last_clean":"2024-09-10T19:04:03.804984+0000","last_became_active":"2024-09-10T18:57:02.460594+0000","last_became_peered":"2024-09-10T18:57:02.460594+0000","last_unstale":"2024-09-10T19:04:03.804984+0000","last_undegraded":"2024-09-10T19:04:03.804984+0000","last_fullsized":"2024-09-10T19:04:03.804984+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:09.267253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:22.891084+0000","last_clean_scrub_stamp":"2024-09-10T19:02:09.267253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0067979290000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"398'4802","reported_seq":4828,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:11.145561+0000","last_change":"2024-09-10T19:03:15.825120+0000","last_active":"2024-09-10T19:04:11.145561+0000","last_peered":"2024-09-10T19:04:11.145561+0000","last_clean":"2024-09-10T19:04:11.145561+0000","last_became_active":"2024-09-10T19:03:15.824855+0000","last_became_peered":"2024-09-10T19:03:15.824855+0000","last_unstale":"2024-09-10T19:04:11.145561+0000","last_undegraded":"2024-09-10T19:04:11.145561+0000","last_fullsized":"2024-09-10T19:04:11.145561+0000","mapping_epoch":379,"log_start":"386'1200","ondisk_log_start":"386'1200","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3602,"log_dups_size":0,"ondisk_log_size":3602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:30.722205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":314703872,"num_objects":4802,"num_object_clones":0,"num_object_copies":14406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4802,"num_write_kb":307328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"70'81","reported_seq":226,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:09.797213+0000","last_change":"2024-09-10T19:02:56.723503+0000","last_active":"2024-09-10T19:04:09.797213+0000","last_peered":"2024-09-10T19:04:09.797213+0000","last_clean":"2024-09-10T19:04:09.797213+0000","last_became_active":"2024-09-10T18:57:02.437944+0000","last_became_peered":"2024-09-10T18:57:02.437944+0000","last_unstale":"2024-09-10T19:04:09.797213+0000","last_undegraded":"2024-09-10T19:04:09.797213+0000","last_fullsized":"2024-09-10T19:04:09.797213+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:56.723447+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:02:56.723447+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001350382,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"232'22","reported_seq":303,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.807908+0000","last_change":"2024-09-10T19:03:43.899377+0000","last_active":"2024-09-10T19:04:03.807908+0000","last_peered":"2024-09-10T19:04:03.807908+0000","last_clean":"2024-09-10T19:04:03.807908+0000","last_became_active":"2024-09-10T19:02:49.572746+0000","last_became_peered":"2024-09-10T19:02:49.572746+0000","last_unstale":"2024-09-10T19:04:03.807908+0000","last_undegraded":"2024-09-10T19:04:03.807908+0000","last_fullsized":"2024-09-10T19:04:03.807908+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'22","last_scrub_stamp":"2024-09-10T19:03:43.899294+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-10T18:47:10.514951+0000","last_clean_scrub_stamp":"2024-09-10T19:03:43.899294+0000","objects_scrubbed":5,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:56.093268+0000","scrub_duration":0.0049335940000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"84'6","reported_seq":296,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:09.804492+0000","last_change":"2024-09-10T19:03:43.134996+0000","last_active":"2024-09-10T19:04:09.804492+0000","last_peered":"2024-09-10T19:04:09.804492+0000","last_clean":"2024-09-10T19:04:09.804492+0000","last_became_active":"2024-09-10T19:02:49.570177+0000","last_became_peered":"2024-09-10T19:02:49.570177+0000","last_unstale":"2024-09-10T19:04:09.804492+0000","last_undegraded":"2024-09-10T19:04:09.804492+0000","last_fullsized":"2024-09-10T19:04:09.804492+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'6","last_scrub_stamp":"2024-09-10T19:03:43.134912+0000","last_deep_scrub":"84'6","last_deep_scrub_stamp":"2024-09-10T18:47:22.545450+0000","last_clean_scrub_stamp":"2024-09-10T19:03:43.134912+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:05:05.604341+0000","scrub_duration":0.0024059210000000001,"objects_trimmed":0,"snaptrim_duration":0.0021933159999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":194,"num_read_kb":189,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":375,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.803339+0000","last_change":"2024-09-10T19:00:08.141664+0000","last_active":"2024-09-10T19:04:03.803339+0000","last_peered":"2024-09-10T19:04:03.803339+0000","last_clean":"2024-09-10T19:04:03.803339+0000","last_became_active":"2024-09-10T18:55:40.393389+0000","last_became_peered":"2024-09-10T18:55:40.393389+0000","last_unstale":"2024-09-10T19:04:03.803339+0000","last_undegraded":"2024-09-10T19:04:03.803339+0000","last_fullsized":"2024-09-10T19:04:03.803339+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:08.141603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:00:08.141603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001270729,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":393,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.805765+0000","last_change":"2024-09-10T19:02:09.600216+0000","last_active":"2024-09-10T19:04:03.805765+0000","last_peered":"2024-09-10T19:04:03.805765+0000","last_clean":"2024-09-10T19:04:03.805765+0000","last_became_active":"2024-09-10T18:57:02.438945+0000","last_became_peered":"2024-09-10T18:57:02.438945+0000","last_unstale":"2024-09-10T19:04:03.805765+0000","last_undegraded":"2024-09-10T19:04:03.805765+0000","last_fullsized":"2024-09-10T19:04:03.805765+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:09.600134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:02:09.600134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003812368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":160,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.810159+0000","last_change":"2024-09-10T19:03:42.107868+0000","last_active":"2024-09-10T19:04:03.810159+0000","last_peered":"2024-09-10T19:04:03.810159+0000","last_clean":"2024-09-10T19:04:03.810159+0000","last_became_active":"2024-09-10T18:57:02.453865+0000","last_became_peered":"2024-09-10T18:57:02.453865+0000","last_unstale":"2024-09-10T19:04:03.810159+0000","last_undegraded":"2024-09-10T19:04:03.810159+0000","last_fullsized":"2024-09-10T19:04:03.810159+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:42.107785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:07.150480+0000","last_clean_scrub_stamp":"2024-09-10T19:03:42.107785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:58.357519+0000","scrub_duration":0.0029595450000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.9","version":"398'4796","reported_seq":4821,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:11.128277+0000","last_change":"2024-09-10T19:03:15.825703+0000","last_active":"2024-09-10T19:04:11.128277+0000","last_peered":"2024-09-10T19:04:11.128277+0000","last_clean":"2024-09-10T19:04:11.128277+0000","last_became_active":"2024-09-10T19:03:15.825424+0000","last_became_peered":"2024-09-10T19:03:15.825424+0000","last_unstale":"2024-09-10T19:04:11.128277+0000","last_undegraded":"2024-09-10T19:04:11.128277+0000","last_fullsized":"2024-09-10T19:04:11.128277+0000","mapping_epoch":379,"log_start":"386'1200","ondisk_log_start":"386'1200","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3596,"log_dups_size":0,"ondisk_log_size":3596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:35.248919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":314310656,"num_objects":4796,"num_object_clones":0,"num_object_copies":14388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4796,"num_write_kb":306944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"70'81","reported_seq":103,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.807929+0000","last_change":"2024-09-10T19:02:49.579385+0000","last_active":"2024-09-10T19:04:03.807929+0000","last_peered":"2024-09-10T19:04:03.807929+0000","last_clean":"2024-09-10T19:04:03.807929+0000","last_became_active":"2024-09-10T19:02:49.578413+0000","last_became_peered":"2024-09-10T19:02:49.578413+0000","last_unstale":"2024-09-10T19:04:03.807929+0000","last_undegraded":"2024-09-10T19:04:03.807929+0000","last_fullsized":"2024-09-10T19:04:03.807929+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:46.421787+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:02:46.421787+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019876809999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17890,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6,"num_write":3,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":35780,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"232'24","reported_seq":146,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:09.801589+0000","last_change":"2024-09-10T19:03:40.879526+0000","last_active":"2024-09-10T19:04:09.801589+0000","last_peered":"2024-09-10T19:04:09.801589+0000","last_clean":"2024-09-10T19:04:09.801589+0000","last_became_active":"2024-09-10T19:00:42.516521+0000","last_became_peered":"2024-09-10T19:00:42.516521+0000","last_unstale":"2024-09-10T19:04:09.801589+0000","last_undegraded":"2024-09-10T19:04:09.801589+0000","last_fullsized":"2024-09-10T19:04:09.801589+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T19:03:40.879436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:03:40.879436+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:59.698095+0000","scrub_duration":0.0094657419999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"305'2","reported_seq":393,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:09.797955+0000","last_change":"2024-09-10T19:03:21.921314+0000","last_active":"2024-09-10T19:04:09.797955+0000","last_peered":"2024-09-10T19:04:09.797955+0000","last_clean":"2024-09-10T19:04:09.797955+0000","last_became_active":"2024-09-10T18:55:40.390022+0000","last_became_peered":"2024-09-10T18:55:40.390022+0000","last_unstale":"2024-09-10T19:04:09.797955+0000","last_undegraded":"2024-09-10T19:04:09.797955+0000","last_fullsized":"2024-09-10T19:04:09.797955+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"305'2","last_scrub_stamp":"2024-09-10T19:03:21.921254+0000","last_deep_scrub":"305'2","last_deep_scrub_stamp":"2024-09-10T18:59:18.897059+0000","last_clean_scrub_stamp":"2024-09-10T19:03:21.921254+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:40.677878+0000","scrub_duration":0.0021302700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":244,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:09.796604+0000","last_change":"2024-09-10T19:02:49.560608+0000","last_active":"2024-09-10T19:04:09.796604+0000","last_peered":"2024-09-10T19:04:09.796604+0000","last_clean":"2024-09-10T19:04:09.796604+0000","last_became_active":"2024-09-10T19:02:49.560325+0000","last_became_peered":"2024-09-10T19:02:49.560325+0000","last_unstale":"2024-09-10T19:04:09.796604+0000","last_undegraded":"2024-09-10T19:04:09.796604+0000","last_fullsized":"2024-09-10T19:04:09.796604+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:06.272690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:06.272690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001771613,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":69,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.807452+0000","last_change":"2024-09-10T19:00:42.518887+0000","last_active":"2024-09-10T19:04:03.807452+0000","last_peered":"2024-09-10T19:04:03.807452+0000","last_clean":"2024-09-10T19:04:03.807452+0000","last_became_active":"2024-09-10T19:00:42.518566+0000","last_became_peered":"2024-09-10T19:00:42.518566+0000","last_unstale":"2024-09-10T19:04:03.807452+0000","last_undegraded":"2024-09-10T19:04:03.807452+0000","last_fullsized":"2024-09-10T19:04:03.807452+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:39.752722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:00:39.752722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":38,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.806432+0000","last_change":"2024-09-10T19:02:49.573172+0000","last_active":"2024-09-10T19:04:03.806432+0000","last_peered":"2024-09-10T19:04:03.806432+0000","last_clean":"2024-09-10T19:04:03.806432+0000","last_became_active":"2024-09-10T19:02:49.572858+0000","last_became_peered":"2024-09-10T19:02:49.572858+0000","last_unstale":"2024-09-10T19:04:03.806432+0000","last_undegraded":"2024-09-10T19:04:03.806432+0000","last_fullsized":"2024-09-10T19:04:03.806432+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:20.272756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:02:20.272756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.a","version":"398'4742","reported_seq":4768,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:11.122706+0000","last_change":"2024-09-10T19:03:15.826075+0000","last_active":"2024-09-10T19:04:11.122706+0000","last_peered":"2024-09-10T19:04:11.122706+0000","last_clean":"2024-09-10T19:04:11.122706+0000","last_became_active":"2024-09-10T19:03:15.825784+0000","last_became_peered":"2024-09-10T19:03:15.825784+0000","last_unstale":"2024-09-10T19:04:11.122706+0000","last_undegraded":"2024-09-10T19:04:11.122706+0000","last_fullsized":"2024-09-10T19:04:11.122706+0000","mapping_epoch":379,"log_start":"386'1200","ondisk_log_start":"386'1200","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3542,"log_dups_size":0,"ondisk_log_size":3542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:39.268698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":310771712,"num_objects":4742,"num_object_clones":0,"num_object_copies":14226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4742,"num_write_kb":303488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"232'33","reported_seq":501,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.810294+0000","last_change":"2024-09-10T19:02:16.059463+0000","last_active":"2024-09-10T19:04:03.810294+0000","last_peered":"2024-09-10T19:04:03.810294+0000","last_clean":"2024-09-10T19:04:03.810294+0000","last_became_active":"2024-09-10T18:57:02.435499+0000","last_became_peered":"2024-09-10T18:57:02.435499+0000","last_unstale":"2024-09-10T19:04:03.810294+0000","last_undegraded":"2024-09-10T19:04:03.810294+0000","last_fullsized":"2024-09-10T19:04:03.810294+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'33","last_scrub_stamp":"2024-09-10T19:02:16.059377+0000","last_deep_scrub":"232'33","last_deep_scrub_stamp":"2024-09-10T18:56:21.206960+0000","last_clean_scrub_stamp":"2024-09-10T19:02:16.059377+0000","objects_scrubbed":7,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0042245340000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"60'2","reported_seq":420,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.804258+0000","last_change":"2024-09-10T19:03:22.545399+0000","last_active":"2024-09-10T19:04:03.804258+0000","last_peered":"2024-09-10T19:04:03.804258+0000","last_clean":"2024-09-10T19:04:03.804258+0000","last_became_active":"2024-09-10T18:45:17.961660+0000","last_became_peered":"2024-09-10T18:45:17.961660+0000","last_unstale":"2024-09-10T19:04:03.804258+0000","last_undegraded":"2024-09-10T19:04:03.804258+0000","last_fullsized":"2024-09-10T19:04:03.804258+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-10T19:03:22.545342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:03:22.545342+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:51.573027+0000","scrub_duration":0.001448983,"objects_trimmed":0,"snaptrim_duration":0.0013942640000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":439,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.805752+0000","last_change":"2024-09-10T19:01:56.581104+0000","last_active":"2024-09-10T19:04:03.805752+0000","last_peered":"2024-09-10T19:04:03.805752+0000","last_clean":"2024-09-10T19:04:03.805752+0000","last_became_active":"2024-09-10T18:57:02.445712+0000","last_became_peered":"2024-09-10T18:57:02.445712+0000","last_unstale":"2024-09-10T19:04:03.805752+0000","last_undegraded":"2024-09-10T19:04:03.805752+0000","last_fullsized":"2024-09-10T19:04:03.805752+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:01:56.581018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:01:56.581018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0048884530000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'81","reported_seq":226,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:09.796533+0000","last_change":"2024-09-10T19:02:59.753995+0000","last_active":"2024-09-10T19:04:09.796533+0000","last_peered":"2024-09-10T19:04:09.796533+0000","last_clean":"2024-09-10T19:04:09.796533+0000","last_became_active":"2024-09-10T18:57:02.440063+0000","last_became_peered":"2024-09-10T18:57:02.440063+0000","last_unstale":"2024-09-10T19:04:09.796533+0000","last_undegraded":"2024-09-10T19:04:09.796533+0000","last_fullsized":"2024-09-10T19:04:09.796533+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:59.753922+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:56.085580+0000","last_clean_scrub_stamp":"2024-09-10T19:02:59.753922+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:28.334036+0000","scrub_duration":0.001320882,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":408,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.806710+0000","last_change":"2024-09-10T19:01:55.620765+0000","last_active":"2024-09-10T19:04:03.806710+0000","last_peered":"2024-09-10T19:04:03.806710+0000","last_clean":"2024-09-10T19:04:03.806710+0000","last_became_active":"2024-09-10T18:57:02.431556+0000","last_became_peered":"2024-09-10T18:57:02.431556+0000","last_unstale":"2024-09-10T19:04:03.806710+0000","last_undegraded":"2024-09-10T19:04:03.806710+0000","last_fullsized":"2024-09-10T19:04:03.806710+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:01:55.620696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:01:55.620696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0068284349999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":423,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.808039+0000","last_change":"2024-09-10T19:03:51.037606+0000","last_active":"2024-09-10T19:04:03.808039+0000","last_peered":"2024-09-10T19:04:03.808039+0000","last_clean":"2024-09-10T19:04:03.808039+0000","last_became_active":"2024-09-10T18:45:24.073327+0000","last_became_peered":"2024-09-10T18:45:24.073327+0000","last_unstale":"2024-09-10T19:04:03.808039+0000","last_undegraded":"2024-09-10T19:04:03.808039+0000","last_fullsized":"2024-09-10T19:04:03.808039+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:51.037432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:03:51.037432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:05:07.894381+0000","scrub_duration":0.0085107059999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.5","version":"398'4868","reported_seq":4895,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:11.635831+0000","last_change":"2024-09-10T19:03:15.824509+0000","last_active":"2024-09-10T19:04:11.635831+0000","last_peered":"2024-09-10T19:04:11.635831+0000","last_clean":"2024-09-10T19:04:11.635831+0000","last_became_active":"2024-09-10T19:03:15.823922+0000","last_became_peered":"2024-09-10T19:03:15.823922+0000","last_unstale":"2024-09-10T19:04:11.635831+0000","last_undegraded":"2024-09-10T19:04:11.635831+0000","last_fullsized":"2024-09-10T19:04:11.635831+0000","mapping_epoch":379,"log_start":"386'1200","ondisk_log_start":"386'1200","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3668,"log_dups_size":0,"ondisk_log_size":3668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:32.977483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319029248,"num_objects":4868,"num_object_clones":0,"num_object_copies":14604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4868,"num_write_kb":311552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.f","version":"70'81","reported_seq":418,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.806311+0000","last_change":"2024-09-10T19:02:13.085183+0000","last_active":"2024-09-10T19:04:03.806311+0000","last_peered":"2024-09-10T19:04:03.806311+0000","last_clean":"2024-09-10T19:04:03.806311+0000","last_became_active":"2024-09-10T18:52:16.598295+0000","last_became_peered":"2024-09-10T18:52:16.598295+0000","last_unstale":"2024-09-10T19:04:03.806311+0000","last_undegraded":"2024-09-10T19:04:03.806311+0000","last_fullsized":"2024-09-10T19:04:03.806311+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:13.085003+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:53.609885+0000","last_clean_scrub_stamp":"2024-09-10T19:02:13.085003+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00305172,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"232'35","reported_seq":516,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.807306+0000","last_change":"2024-09-10T19:03:46.180416+0000","last_active":"2024-09-10T19:04:03.807306+0000","last_peered":"2024-09-10T19:04:03.807306+0000","last_clean":"2024-09-10T19:04:03.807306+0000","last_became_active":"2024-09-10T18:57:02.436059+0000","last_became_peered":"2024-09-10T18:57:02.436059+0000","last_unstale":"2024-09-10T19:04:03.807306+0000","last_undegraded":"2024-09-10T19:04:03.807306+0000","last_fullsized":"2024-09-10T19:04:03.807306+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'35","last_scrub_stamp":"2024-09-10T19:03:46.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:03:46.179242+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:05:09.346612+0000","scrub_duration":0.0052365550000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":387,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.806782+0000","last_change":"2024-09-10T19:01:53.609239+0000","last_active":"2024-09-10T19:04:03.806782+0000","last_peered":"2024-09-10T19:04:03.806782+0000","last_clean":"2024-09-10T19:04:03.806782+0000","last_became_active":"2024-09-10T18:55:40.391849+0000","last_became_peered":"2024-09-10T18:55:40.391849+0000","last_unstale":"2024-09-10T19:04:03.806782+0000","last_undegraded":"2024-09-10T19:04:03.806782+0000","last_fullsized":"2024-09-10T19:04:03.806782+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:01:53.609184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:01:53.609184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.024484227000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":369,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:09.798463+0000","last_change":"2024-09-10T19:03:48.853831+0000","last_active":"2024-09-10T19:04:09.798463+0000","last_peered":"2024-09-10T19:04:09.798463+0000","last_clean":"2024-09-10T19:04:09.798463+0000","last_became_active":"2024-09-10T19:00:42.514709+0000","last_became_peered":"2024-09-10T19:00:42.514709+0000","last_unstale":"2024-09-10T19:04:09.798463+0000","last_undegraded":"2024-09-10T19:04:09.798463+0000","last_fullsized":"2024-09-10T19:04:09.798463+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:48.853752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:03:48.853752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:05:01.788787+0000","scrub_duration":0.0059332969999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":204,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.803579+0000","last_change":"2024-09-10T19:02:58.491396+0000","last_active":"2024-09-10T19:04:03.803579+0000","last_peered":"2024-09-10T19:04:03.803579+0000","last_clean":"2024-09-10T19:04:03.803579+0000","last_became_active":"2024-09-10T18:57:02.468866+0000","last_became_peered":"2024-09-10T18:57:02.468866+0000","last_unstale":"2024-09-10T19:04:03.803579+0000","last_undegraded":"2024-09-10T19:04:03.803579+0000","last_fullsized":"2024-09-10T19:04:03.803579+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:58.491326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:02:58.491326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:23.761727+0000","scrub_duration":0.0017883440000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"398'4912","reported_seq":4939,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:11.535752+0000","last_change":"2024-09-10T19:03:16.045556+0000","last_active":"2024-09-10T19:04:11.535752+0000","last_peered":"2024-09-10T19:04:11.535752+0000","last_clean":"2024-09-10T19:04:11.535752+0000","last_became_active":"2024-09-10T19:03:16.045197+0000","last_became_peered":"2024-09-10T19:03:16.045197+0000","last_unstale":"2024-09-10T19:04:11.535752+0000","last_undegraded":"2024-09-10T19:04:11.535752+0000","last_fullsized":"2024-09-10T19:04:11.535752+0000","mapping_epoch":379,"log_start":"386'1500","ondisk_log_start":"386'1500","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3412,"log_dups_size":0,"ondisk_log_size":3412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:21.607179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":321912832,"num_objects":4912,"num_object_clones":0,"num_object_copies":14736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4912,"num_write_kb":314368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,3],"acting":[6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.e","version":"70'81","reported_seq":276,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.805479+0000","last_change":"2024-09-10T18:59:23.592755+0000","last_active":"2024-09-10T19:04:03.805479+0000","last_peered":"2024-09-10T19:04:03.805479+0000","last_clean":"2024-09-10T19:04:03.805479+0000","last_became_active":"2024-09-10T18:57:02.449115+0000","last_became_peered":"2024-09-10T18:57:02.449115+0000","last_unstale":"2024-09-10T19:04:03.805479+0000","last_undegraded":"2024-09-10T19:04:03.805479+0000","last_fullsized":"2024-09-10T19:04:03.805479+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T18:59:23.592672+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T18:59:23.592672+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0038976470000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"232'19","reported_seq":425,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:09.798993+0000","last_change":"2024-09-10T19:03:43.850751+0000","last_active":"2024-09-10T19:04:09.798993+0000","last_peered":"2024-09-10T19:04:09.798993+0000","last_clean":"2024-09-10T19:04:09.798993+0000","last_became_active":"2024-09-10T18:55:40.395257+0000","last_became_peered":"2024-09-10T18:55:40.395257+0000","last_unstale":"2024-09-10T19:04:09.798993+0000","last_undegraded":"2024-09-10T19:04:09.798993+0000","last_fullsized":"2024-09-10T19:04:09.798993+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T19:03:43.850670+0000","last_deep_scrub":"130'13","last_deep_scrub_stamp":"2024-09-10T18:51:58.427534+0000","last_clean_scrub_stamp":"2024-09-10T19:03:43.850670+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:05:06.652694+0000","scrub_duration":0.0026924499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":48,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.806320+0000","last_change":"2024-09-10T19:03:51.598347+0000","last_active":"2024-09-10T19:04:03.806320+0000","last_peered":"2024-09-10T19:04:03.806320+0000","last_clean":"2024-09-10T19:04:03.806320+0000","last_became_active":"2024-09-10T19:02:49.581709+0000","last_became_peered":"2024-09-10T19:02:49.581709+0000","last_unstale":"2024-09-10T19:04:03.806320+0000","last_undegraded":"2024-09-10T19:04:03.806320+0000","last_fullsized":"2024-09-10T19:04:03.806320+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:51.598266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:03:51.598266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:05:19.902053+0000","scrub_duration":0.0047300789999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":378,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.800777+0000","last_change":"2024-09-10T19:02:51.031764+0000","last_active":"2024-09-10T19:04:03.800777+0000","last_peered":"2024-09-10T19:04:03.800777+0000","last_clean":"2024-09-10T19:04:03.800777+0000","last_became_active":"2024-09-10T18:55:40.390771+0000","last_became_peered":"2024-09-10T18:55:40.390771+0000","last_unstale":"2024-09-10T19:04:03.800777+0000","last_undegraded":"2024-09-10T19:04:03.800777+0000","last_fullsized":"2024-09-10T19:04:03.800777+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:51.031711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_clean_scrub_stamp":"2024-09-10T19:02:51.031711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:19.854800+0000","scrub_duration":0.0015681510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":403,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:09.797538+0000","last_change":"2024-09-10T19:01:58.603370+0000","last_active":"2024-09-10T19:04:09.797538+0000","last_peered":"2024-09-10T19:04:09.797538+0000","last_clean":"2024-09-10T19:04:09.797538+0000","last_became_active":"2024-09-10T18:57:02.452387+0000","last_became_peered":"2024-09-10T18:57:02.452387+0000","last_unstale":"2024-09-10T19:04:09.797538+0000","last_undegraded":"2024-09-10T19:04:09.797538+0000","last_fullsized":"2024-09-10T19:04:09.797538+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:01:58.601096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:53.345184+0000","last_clean_scrub_stamp":"2024-09-10T19:01:58.601096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026721589999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"398'4878","reported_seq":4905,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:11.599696+0000","last_change":"2024-09-10T19:03:16.045204+0000","last_active":"2024-09-10T19:04:11.599696+0000","last_peered":"2024-09-10T19:04:11.599696+0000","last_clean":"2024-09-10T19:04:11.599696+0000","last_became_active":"2024-09-10T19:03:16.044291+0000","last_became_peered":"2024-09-10T19:03:16.044291+0000","last_unstale":"2024-09-10T19:04:11.599696+0000","last_undegraded":"2024-09-10T19:04:11.599696+0000","last_fullsized":"2024-09-10T19:04:11.599696+0000","mapping_epoch":379,"log_start":"386'1200","ondisk_log_start":"386'1200","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3678,"log_dups_size":0,"ondisk_log_size":3678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:34.501027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319684608,"num_objects":4878,"num_object_clones":0,"num_object_copies":14634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4878,"num_write_kb":312192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.d","version":"70'81","reported_seq":217,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.806055+0000","last_change":"2024-09-10T19:02:01.044879+0000","last_active":"2024-09-10T19:04:03.806055+0000","last_peered":"2024-09-10T19:04:03.806055+0000","last_clean":"2024-09-10T19:04:03.806055+0000","last_became_active":"2024-09-10T18:57:02.439652+0000","last_became_peered":"2024-09-10T18:57:02.439652+0000","last_unstale":"2024-09-10T19:04:03.806055+0000","last_undegraded":"2024-09-10T19:04:03.806055+0000","last_fullsized":"2024-09-10T19:04:03.806055+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:01.044143+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:02:01.044143+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014088588000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"365'34","reported_seq":102,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.806361+0000","last_change":"2024-09-10T19:02:18.712390+0000","last_active":"2024-09-10T19:04:03.806361+0000","last_peered":"2024-09-10T19:04:03.806361+0000","last_clean":"2024-09-10T19:04:03.806361+0000","last_became_active":"2024-09-10T19:00:42.515903+0000","last_became_peered":"2024-09-10T19:00:42.515903+0000","last_unstale":"2024-09-10T19:04:03.806361+0000","last_undegraded":"2024-09-10T19:04:03.806361+0000","last_fullsized":"2024-09-10T19:04:03.806361+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"365'34","last_scrub_stamp":"2024-09-10T19:02:18.712306+0000","last_deep_scrub":"365'34","last_deep_scrub_stamp":"2024-09-10T19:02:18.712306+0000","last_clean_scrub_stamp":"2024-09-10T19:02:18.712306+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.017099106999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":33,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":368,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.805187+0000","last_change":"2024-09-10T19:02:08.306579+0000","last_active":"2024-09-10T19:04:03.805187+0000","last_peered":"2024-09-10T19:04:03.805187+0000","last_clean":"2024-09-10T19:04:03.805187+0000","last_became_active":"2024-09-10T18:57:02.461708+0000","last_became_peered":"2024-09-10T18:57:02.461708+0000","last_unstale":"2024-09-10T19:04:03.805187+0000","last_undegraded":"2024-09-10T19:04:03.805187+0000","last_fullsized":"2024-09-10T19:04:03.805187+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:08.306490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:05.603125+0000","last_clean_scrub_stamp":"2024-09-10T19:02:08.306490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0043377579999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"273'10","reported_seq":318,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:09.802915+0000","last_change":"2024-09-10T19:02:46.026041+0000","last_active":"2024-09-10T19:04:09.802915+0000","last_peered":"2024-09-10T19:04:09.802915+0000","last_clean":"2024-09-10T19:04:09.802915+0000","last_became_active":"2024-09-10T18:57:02.441890+0000","last_became_peered":"2024-09-10T18:57:02.441890+0000","last_unstale":"2024-09-10T19:04:09.802915+0000","last_undegraded":"2024-09-10T19:04:09.802915+0000","last_fullsized":"2024-09-10T19:04:09.802915+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'10","last_scrub_stamp":"2024-09-10T19:02:46.025981+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-09-10T18:47:02.582229+0000","last_clean_scrub_stamp":"2024-09-10T19:02:46.025981+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003528864,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":430,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.805960+0000","last_change":"2024-09-10T19:03:32.808204+0000","last_active":"2024-09-10T19:04:03.805960+0000","last_peered":"2024-09-10T19:04:03.805960+0000","last_clean":"2024-09-10T19:04:03.805960+0000","last_became_active":"2024-09-10T18:55:40.389202+0000","last_became_peered":"2024-09-10T18:55:40.389202+0000","last_unstale":"2024-09-10T19:04:03.805960+0000","last_undegraded":"2024-09-10T19:04:03.805960+0000","last_fullsized":"2024-09-10T19:04:03.805960+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:32.808130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_clean_scrub_stamp":"2024-09-10T19:03:32.808130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:54.390143+0000","scrub_duration":0.003767603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"399'5062","reported_seq":5090,"reported_epoch":399,"state":"active+clean","last_fresh":"2024-09-10T19:04:13.828171+0000","last_change":"2024-09-10T19:03:16.044527+0000","last_active":"2024-09-10T19:04:13.828171+0000","last_peered":"2024-09-10T19:04:13.828171+0000","last_clean":"2024-09-10T19:04:13.828171+0000","last_became_active":"2024-09-10T19:03:16.044184+0000","last_became_peered":"2024-09-10T19:03:16.044184+0000","last_unstale":"2024-09-10T19:04:13.828171+0000","last_undegraded":"2024-09-10T19:04:13.828171+0000","last_fullsized":"2024-09-10T19:04:13.828171+0000","mapping_epoch":379,"log_start":"386'1200","ondisk_log_start":"386'1200","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3862,"log_dups_size":0,"ondisk_log_size":3862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:15.758376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":331743232,"num_objects":5062,"num_object_clones":0,"num_object_copies":15186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5062,"num_write_kb":323968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.c","version":"70'81","reported_seq":361,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.804487+0000","last_change":"2024-09-10T19:02:54.540617+0000","last_active":"2024-09-10T19:04:03.804487+0000","last_peered":"2024-09-10T19:04:03.804487+0000","last_clean":"2024-09-10T19:04:03.804487+0000","last_became_active":"2024-09-10T18:55:40.390655+0000","last_became_peered":"2024-09-10T18:55:40.390655+0000","last_unstale":"2024-09-10T19:04:03.804487+0000","last_undegraded":"2024-09-10T19:04:03.804487+0000","last_fullsized":"2024-09-10T19:04:03.804487+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:54.540558+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:56:33.779154+0000","last_clean_scrub_stamp":"2024-09-10T19:02:54.540558+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019874599999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"232'18","reported_seq":440,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.800852+0000","last_change":"2024-09-10T19:01:53.948548+0000","last_active":"2024-09-10T19:04:03.800852+0000","last_peered":"2024-09-10T19:04:03.800852+0000","last_clean":"2024-09-10T19:04:03.800852+0000","last_became_active":"2024-09-10T18:45:20.056869+0000","last_became_peered":"2024-09-10T18:45:20.056869+0000","last_unstale":"2024-09-10T19:04:03.800852+0000","last_undegraded":"2024-09-10T19:04:03.800852+0000","last_fullsized":"2024-09-10T19:04:03.800852+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'18","last_scrub_stamp":"2024-09-10T19:01:53.948309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:01:53.948309+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.010806517,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"56'2","reported_seq":450,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.804882+0000","last_change":"2024-09-10T19:02:06.065853+0000","last_active":"2024-09-10T19:04:03.804882+0000","last_peered":"2024-09-10T19:04:03.804882+0000","last_clean":"2024-09-10T19:04:03.804882+0000","last_became_active":"2024-09-10T18:57:02.434475+0000","last_became_peered":"2024-09-10T18:57:02.434475+0000","last_unstale":"2024-09-10T19:04:03.804882+0000","last_undegraded":"2024-09-10T19:04:03.804882+0000","last_fullsized":"2024-09-10T19:04:03.804882+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-10T19:02:06.065768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:06.065768+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0050047989999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":369,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:09.796869+0000","last_change":"2024-09-10T19:03:26.818927+0000","last_active":"2024-09-10T19:04:09.796869+0000","last_peered":"2024-09-10T19:04:09.796869+0000","last_clean":"2024-09-10T19:04:09.796869+0000","last_became_active":"2024-09-10T19:00:42.513796+0000","last_became_peered":"2024-09-10T19:00:42.513796+0000","last_unstale":"2024-09-10T19:04:09.796869+0000","last_undegraded":"2024-09-10T19:04:09.796869+0000","last_fullsized":"2024-09-10T19:04:09.796869+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:26.818865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:02.711594+0000","last_clean_scrub_stamp":"2024-09-10T19:03:26.818865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:36.242234+0000","scrub_duration":0.001326844,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":411,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.803317+0000","last_change":"2024-09-10T19:03:46.711248+0000","last_active":"2024-09-10T19:04:03.803317+0000","last_peered":"2024-09-10T19:04:03.803317+0000","last_clean":"2024-09-10T19:04:03.803317+0000","last_became_active":"2024-09-10T18:57:02.459985+0000","last_became_peered":"2024-09-10T18:57:02.459985+0000","last_unstale":"2024-09-10T19:04:03.803317+0000","last_undegraded":"2024-09-10T19:04:03.803317+0000","last_fullsized":"2024-09-10T19:04:03.803317+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:46.710933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:46.710933+0000","last_clean_scrub_stamp":"2024-09-10T19:03:46.710933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:05:13.869222+0000","scrub_duration":0.0061568430000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.13","version":"70'81","reported_seq":107,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.803506+0000","last_change":"2024-09-10T19:02:49.581193+0000","last_active":"2024-09-10T19:04:03.803506+0000","last_peered":"2024-09-10T19:04:03.803506+0000","last_clean":"2024-09-10T19:04:03.803506+0000","last_became_active":"2024-09-10T19:02:49.580656+0000","last_became_peered":"2024-09-10T19:02:49.580656+0000","last_unstale":"2024-09-10T19:04:03.803506+0000","last_undegraded":"2024-09-10T19:04:03.803506+0000","last_fullsized":"2024-09-10T19:04:03.803506+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:01:56.384747+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:01:56.384747+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.007621203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"232'19","reported_seq":438,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:09.802026+0000","last_change":"2024-09-10T19:02:02.627107+0000","last_active":"2024-09-10T19:04:09.802026+0000","last_peered":"2024-09-10T19:04:09.802026+0000","last_clean":"2024-09-10T19:04:09.802026+0000","last_became_active":"2024-09-10T18:57:02.453001+0000","last_became_peered":"2024-09-10T18:57:02.453001+0000","last_unstale":"2024-09-10T19:04:09.802026+0000","last_undegraded":"2024-09-10T19:04:09.802026+0000","last_fullsized":"2024-09-10T19:04:09.802026+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T19:02:02.627031+0000","last_deep_scrub":"76'13","last_deep_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_clean_scrub_stamp":"2024-09-10T19:02:02.627031+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00634981,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":252,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:09.796569+0000","last_change":"2024-09-10T19:03:35.858172+0000","last_active":"2024-09-10T19:04:09.796569+0000","last_peered":"2024-09-10T19:04:09.796569+0000","last_clean":"2024-09-10T19:04:09.796569+0000","last_became_active":"2024-09-10T19:02:49.580701+0000","last_became_peered":"2024-09-10T19:02:49.580701+0000","last_unstale":"2024-09-10T19:04:09.796569+0000","last_undegraded":"2024-09-10T19:04:09.796569+0000","last_fullsized":"2024-09-10T19:04:09.796569+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:35.858086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:03:35.858086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:05:02.478958+0000","scrub_duration":0.0061699850000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":413,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.804711+0000","last_change":"2024-09-10T19:02:23.261573+0000","last_active":"2024-09-10T19:04:03.804711+0000","last_peered":"2024-09-10T19:04:03.804711+0000","last_clean":"2024-09-10T19:04:03.804711+0000","last_became_active":"2024-09-10T18:57:02.455341+0000","last_became_peered":"2024-09-10T18:57:02.455341+0000","last_unstale":"2024-09-10T19:04:03.804711+0000","last_undegraded":"2024-09-10T19:04:03.804711+0000","last_fullsized":"2024-09-10T19:04:03.804711+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:23.261482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:00.754912+0000","last_clean_scrub_stamp":"2024-09-10T19:02:23.261482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0076325380000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"364'2","reported_seq":437,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.803523+0000","last_change":"2024-09-10T19:02:15.059462+0000","last_active":"2024-09-10T19:04:03.803523+0000","last_peered":"2024-09-10T19:04:03.803523+0000","last_clean":"2024-09-10T19:04:03.803523+0000","last_became_active":"2024-09-10T18:59:48.741275+0000","last_became_peered":"2024-09-10T18:59:48.741275+0000","last_unstale":"2024-09-10T19:04:03.803523+0000","last_undegraded":"2024-09-10T19:04:03.803523+0000","last_fullsized":"2024-09-10T19:04:03.803523+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"364'2","last_scrub_stamp":"2024-09-10T19:02:15.059320+0000","last_deep_scrub":"364'2","last_deep_scrub_stamp":"2024-09-10T19:02:15.059320+0000","last_clean_scrub_stamp":"2024-09-10T19:02:15.059320+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.020873968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":52,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.12","version":"70'81","reported_seq":277,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.805778+0000","last_change":"2024-09-10T19:02:05.565674+0000","last_active":"2024-09-10T19:04:03.805778+0000","last_peered":"2024-09-10T19:04:03.805778+0000","last_clean":"2024-09-10T19:04:03.805778+0000","last_became_active":"2024-09-10T18:52:16.598921+0000","last_became_peered":"2024-09-10T18:52:16.598921+0000","last_unstale":"2024-09-10T19:04:03.805778+0000","last_undegraded":"2024-09-10T19:04:03.805778+0000","last_fullsized":"2024-09-10T19:04:03.805778+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:05.565587+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:54.127476+0000","last_clean_scrub_stamp":"2024-09-10T19:02:05.565587+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051538479999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":443,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.808199+0000","last_change":"2024-09-10T19:02:18.973390+0000","last_active":"2024-09-10T19:04:03.808199+0000","last_peered":"2024-09-10T19:04:03.808199+0000","last_clean":"2024-09-10T19:04:03.808199+0000","last_became_active":"2024-09-10T18:57:02.442251+0000","last_became_peered":"2024-09-10T18:57:02.442251+0000","last_unstale":"2024-09-10T19:04:03.808199+0000","last_undegraded":"2024-09-10T19:04:03.808199+0000","last_fullsized":"2024-09-10T19:04:03.808199+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:18.973341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:18.973341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001038347,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"232'12","reported_seq":444,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.804913+0000","last_change":"2024-09-10T19:03:50.702400+0000","last_active":"2024-09-10T19:04:03.804913+0000","last_peered":"2024-09-10T19:04:03.804913+0000","last_clean":"2024-09-10T19:04:03.804913+0000","last_became_active":"2024-09-10T18:57:02.460430+0000","last_became_peered":"2024-09-10T18:57:02.460430+0000","last_unstale":"2024-09-10T19:04:03.804913+0000","last_undegraded":"2024-09-10T19:04:03.804913+0000","last_fullsized":"2024-09-10T19:04:03.804913+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'12","last_scrub_stamp":"2024-09-10T19:03:50.702317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:03:50.702317+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:05:06.743100+0000","scrub_duration":0.0055835499999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":405,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.805934+0000","last_change":"2024-09-10T19:01:58.529083+0000","last_active":"2024-09-10T19:04:03.805934+0000","last_peered":"2024-09-10T19:04:03.805934+0000","last_clean":"2024-09-10T19:04:03.805934+0000","last_became_active":"2024-09-10T18:45:22.069653+0000","last_became_peered":"2024-09-10T18:45:22.069653+0000","last_unstale":"2024-09-10T19:04:03.805934+0000","last_undegraded":"2024-09-10T19:04:03.805934+0000","last_fullsized":"2024-09-10T19:04:03.805934+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:01:58.528538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:01:58.528538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0057370839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":170,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.810022+0000","last_change":"2024-09-10T19:02:14.037294+0000","last_active":"2024-09-10T19:04:03.810022+0000","last_peered":"2024-09-10T19:04:03.810022+0000","last_clean":"2024-09-10T19:04:03.810022+0000","last_became_active":"2024-09-10T19:00:42.519492+0000","last_became_peered":"2024-09-10T19:00:42.519492+0000","last_unstale":"2024-09-10T19:04:03.810022+0000","last_undegraded":"2024-09-10T19:04:03.810022+0000","last_fullsized":"2024-09-10T19:04:03.810022+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:14.037238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:02:14.037238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001210221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.11","version":"70'81","reported_seq":313,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:09.797696+0000","last_change":"2024-09-10T19:02:17.778329+0000","last_active":"2024-09-10T19:04:09.797696+0000","last_peered":"2024-09-10T19:04:09.797696+0000","last_clean":"2024-09-10T19:04:09.797696+0000","last_became_active":"2024-09-10T19:00:42.517311+0000","last_became_peered":"2024-09-10T19:00:42.517311+0000","last_unstale":"2024-09-10T19:04:09.797696+0000","last_undegraded":"2024-09-10T19:04:09.797696+0000","last_fullsized":"2024-09-10T19:04:09.797696+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:17.778227+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:02:17.778227+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0070248109999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"232'20","reported_seq":491,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.804059+0000","last_change":"2024-09-10T19:00:07.147063+0000","last_active":"2024-09-10T19:04:03.804059+0000","last_peered":"2024-09-10T19:04:03.804059+0000","last_clean":"2024-09-10T19:04:03.804059+0000","last_became_active":"2024-09-10T18:57:02.444437+0000","last_became_peered":"2024-09-10T18:57:02.444437+0000","last_unstale":"2024-09-10T19:04:03.804059+0000","last_undegraded":"2024-09-10T19:04:03.804059+0000","last_fullsized":"2024-09-10T19:04:03.804059+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'20","last_scrub_stamp":"2024-09-10T19:00:07.146928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:00:07.146928+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0071404040000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":429,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.805485+0000","last_change":"2024-09-10T19:02:20.755424+0000","last_active":"2024-09-10T19:04:03.805485+0000","last_peered":"2024-09-10T19:04:03.805485+0000","last_clean":"2024-09-10T19:04:03.805485+0000","last_became_active":"2024-09-10T18:57:02.445710+0000","last_became_peered":"2024-09-10T18:57:02.445710+0000","last_unstale":"2024-09-10T19:04:03.805485+0000","last_undegraded":"2024-09-10T19:04:03.805485+0000","last_fullsized":"2024-09-10T19:04:03.805485+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:20.755253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:20.755253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023404490000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"304'11","reported_seq":684,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:09.802332+0000","last_change":"2024-09-10T19:03:34.811295+0000","last_active":"2024-09-10T19:04:09.802332+0000","last_peered":"2024-09-10T19:04:09.802332+0000","last_clean":"2024-09-10T19:04:09.802332+0000","last_became_active":"2024-09-10T18:58:58.489370+0000","last_became_peered":"2024-09-10T18:58:58.489370+0000","last_unstale":"2024-09-10T19:04:09.802332+0000","last_undegraded":"2024-09-10T19:04:09.802332+0000","last_fullsized":"2024-09-10T19:04:09.802332+0000","mapping_epoch":303,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":304,"parent":"0.0","parent_split_bits":0,"last_scrub":"304'11","last_scrub_stamp":"2024-09-10T19:03:34.811194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:03:34.811194+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:49.802623+0000","scrub_duration":0.0024856539999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":98,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:09.799986+0000","last_change":"2024-09-10T19:02:22.928743+0000","last_active":"2024-09-10T19:04:09.799986+0000","last_peered":"2024-09-10T19:04:09.799986+0000","last_clean":"2024-09-10T19:04:09.799986+0000","last_became_active":"2024-09-10T18:59:48.744923+0000","last_became_peered":"2024-09-10T18:59:48.744923+0000","last_unstale":"2024-09-10T19:04:09.799986+0000","last_undegraded":"2024-09-10T19:04:09.799986+0000","last_fullsized":"2024-09-10T19:04:09.799986+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:22.928652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:57.476724+0000","last_clean_scrub_stamp":"2024-09-10T19:02:22.928652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013720313,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.10","version":"70'81","reported_seq":287,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.806435+0000","last_change":"2024-09-10T19:03:50.010677+0000","last_active":"2024-09-10T19:04:03.806435+0000","last_peered":"2024-09-10T19:04:03.806435+0000","last_clean":"2024-09-10T19:04:03.806435+0000","last_became_active":"2024-09-10T18:57:02.443782+0000","last_became_peered":"2024-09-10T18:57:02.443782+0000","last_unstale":"2024-09-10T19:04:03.806435+0000","last_undegraded":"2024-09-10T19:04:03.806435+0000","last_fullsized":"2024-09-10T19:04:03.806435+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:03:50.010596+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:03:50.010596+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:05:00.834175+0000","scrub_duration":0.0030874829999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"232'15","reported_seq":89,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.808718+0000","last_change":"2024-09-10T19:02:46.674532+0000","last_active":"2024-09-10T19:04:03.808718+0000","last_peered":"2024-09-10T19:04:03.808718+0000","last_clean":"2024-09-10T19:04:03.808718+0000","last_became_active":"2024-09-10T19:00:42.518672+0000","last_became_peered":"2024-09-10T19:00:42.518672+0000","last_unstale":"2024-09-10T19:04:03.808718+0000","last_undegraded":"2024-09-10T19:04:03.808718+0000","last_fullsized":"2024-09-10T19:04:03.808718+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T19:02:46.674451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:46.674451+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0082292530000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":387,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:09.798640+0000","last_change":"2024-09-10T19:02:03.587108+0000","last_active":"2024-09-10T19:04:09.798640+0000","last_peered":"2024-09-10T19:04:09.798640+0000","last_clean":"2024-09-10T19:04:09.798640+0000","last_became_active":"2024-09-10T18:55:40.390697+0000","last_became_peered":"2024-09-10T18:55:40.390697+0000","last_unstale":"2024-09-10T19:04:09.798640+0000","last_undegraded":"2024-09-10T19:04:09.798640+0000","last_fullsized":"2024-09-10T19:04:09.798640+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:03.586986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:03.586986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0079434420000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"273'11","reported_seq":666,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:06.297345+0000","last_change":"2024-09-10T19:03:49.744278+0000","last_active":"2024-09-10T19:04:06.297345+0000","last_peered":"2024-09-10T19:04:06.297345+0000","last_clean":"2024-09-10T19:04:06.297345+0000","last_became_active":"2024-09-10T18:57:02.445268+0000","last_became_peered":"2024-09-10T18:57:02.445268+0000","last_unstale":"2024-09-10T19:04:06.297345+0000","last_undegraded":"2024-09-10T19:04:06.297345+0000","last_fullsized":"2024-09-10T19:04:06.297345+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'11","last_scrub_stamp":"2024-09-10T19:03:49.744177+0000","last_deep_scrub":"273'11","last_deep_scrub_stamp":"2024-09-10T19:03:49.744177+0000","last_clean_scrub_stamp":"2024-09-10T19:03:49.744177+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:05:06.141804+0000","scrub_duration":0.0069874999999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":428,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.804551+0000","last_change":"2024-09-10T19:03:23.737568+0000","last_active":"2024-09-10T19:04:03.804551+0000","last_peered":"2024-09-10T19:04:03.804551+0000","last_clean":"2024-09-10T19:04:03.804551+0000","last_became_active":"2024-09-10T18:55:40.391662+0000","last_became_peered":"2024-09-10T18:55:40.391662+0000","last_unstale":"2024-09-10T19:04:03.804551+0000","last_undegraded":"2024-09-10T19:04:03.804551+0000","last_fullsized":"2024-09-10T19:04:03.804551+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:23.737518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_clean_scrub_stamp":"2024-09-10T19:03:23.737518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:29.096316+0000","scrub_duration":0.0037070459999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"232'15","reported_seq":313,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.807082+0000","last_change":"2024-09-10T19:03:37.134082+0000","last_active":"2024-09-10T19:04:03.807082+0000","last_peered":"2024-09-10T19:04:03.807082+0000","last_clean":"2024-09-10T19:04:03.807082+0000","last_became_active":"2024-09-10T19:02:49.561403+0000","last_became_peered":"2024-09-10T19:02:49.561403+0000","last_unstale":"2024-09-10T19:04:03.807082+0000","last_undegraded":"2024-09-10T19:04:03.807082+0000","last_fullsized":"2024-09-10T19:04:03.807082+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T19:03:37.134028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:03:37.134028+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:50.121559+0000","scrub_duration":0.0058255470000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":460,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.812143+0000","last_change":"2024-09-10T19:03:36.087607+0000","last_active":"2024-09-10T19:04:03.812143+0000","last_peered":"2024-09-10T19:04:03.812143+0000","last_clean":"2024-09-10T19:04:03.812143+0000","last_became_active":"2024-09-10T19:00:42.518310+0000","last_became_peered":"2024-09-10T19:00:42.518310+0000","last_unstale":"2024-09-10T19:04:03.812143+0000","last_undegraded":"2024-09-10T19:04:03.812143+0000","last_fullsized":"2024-09-10T19:04:03.812143+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:36.087523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:03:36.087523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:41.676390+0000","scrub_duration":0.0039520240000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":410,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.803407+0000","last_change":"2024-09-10T19:02:22.258152+0000","last_active":"2024-09-10T19:04:03.803407+0000","last_peered":"2024-09-10T19:04:03.803407+0000","last_clean":"2024-09-10T19:04:03.803407+0000","last_became_active":"2024-09-10T18:55:40.398243+0000","last_became_peered":"2024-09-10T18:55:40.398243+0000","last_unstale":"2024-09-10T19:04:03.803407+0000","last_undegraded":"2024-09-10T19:04:03.803407+0000","last_fullsized":"2024-09-10T19:04:03.803407+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:22.258058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:03.571786+0000","last_clean_scrub_stamp":"2024-09-10T19:02:22.258058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0054525670000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":402,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:09.797052+0000","last_change":"2024-09-10T19:03:22.930232+0000","last_active":"2024-09-10T19:04:09.797052+0000","last_peered":"2024-09-10T19:04:09.797052+0000","last_clean":"2024-09-10T19:04:09.797052+0000","last_became_active":"2024-09-10T19:00:42.516833+0000","last_became_peered":"2024-09-10T19:00:42.516833+0000","last_unstale":"2024-09-10T19:04:09.797052+0000","last_undegraded":"2024-09-10T19:04:09.797052+0000","last_fullsized":"2024-09-10T19:04:09.797052+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:22.930166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:04.631558+0000","last_clean_scrub_stamp":"2024-09-10T19:03:22.930166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:40.776332+0000","scrub_duration":0.001481163,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"232'24","reported_seq":441,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.801734+0000","last_change":"2024-09-10T19:03:37.562262+0000","last_active":"2024-09-10T19:04:03.801734+0000","last_peered":"2024-09-10T19:04:03.801734+0000","last_clean":"2024-09-10T19:04:03.801734+0000","last_became_active":"2024-09-10T18:45:20.056885+0000","last_became_peered":"2024-09-10T18:45:20.056885+0000","last_unstale":"2024-09-10T19:04:03.801734+0000","last_undegraded":"2024-09-10T19:04:03.801734+0000","last_fullsized":"2024-09-10T19:04:03.801734+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T19:03:37.562188+0000","last_deep_scrub":"223'16","last_deep_scrub_stamp":"2024-09-10T18:54:47.324044+0000","last_clean_scrub_stamp":"2024-09-10T19:03:37.562188+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:48.728898+0000","scrub_duration":0.0041736869999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":280,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.808845+0000","last_change":"2024-09-10T19:02:49.573080+0000","last_active":"2024-09-10T19:04:03.808845+0000","last_peered":"2024-09-10T19:04:03.808845+0000","last_clean":"2024-09-10T19:04:03.808845+0000","last_became_active":"2024-09-10T19:02:49.572748+0000","last_became_peered":"2024-09-10T19:02:49.572748+0000","last_unstale":"2024-09-10T19:04:03.808845+0000","last_undegraded":"2024-09-10T19:04:03.808845+0000","last_fullsized":"2024-09-10T19:04:03.808845+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:19.301278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:00.029389+0000","last_clean_scrub_stamp":"2024-09-10T19:02:19.301278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003302677,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":38,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.805850+0000","last_change":"2024-09-10T19:02:49.573280+0000","last_active":"2024-09-10T19:04:03.805850+0000","last_peered":"2024-09-10T19:04:03.805850+0000","last_clean":"2024-09-10T19:04:03.805850+0000","last_became_active":"2024-09-10T19:02:49.572998+0000","last_became_peered":"2024-09-10T19:02:49.572998+0000","last_unstale":"2024-09-10T19:04:03.805850+0000","last_undegraded":"2024-09-10T19:04:03.805850+0000","last_fullsized":"2024-09-10T19:04:03.805850+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:15.321767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:02:15.321767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":408,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.804120+0000","last_change":"2024-09-10T19:02:00.273929+0000","last_active":"2024-09-10T19:04:03.804120+0000","last_peered":"2024-09-10T19:04:03.804120+0000","last_clean":"2024-09-10T19:04:03.804120+0000","last_became_active":"2024-09-10T18:55:40.395595+0000","last_became_peered":"2024-09-10T18:55:40.395595+0000","last_unstale":"2024-09-10T19:04:03.804120+0000","last_undegraded":"2024-09-10T19:04:03.804120+0000","last_fullsized":"2024-09-10T19:04:03.804120+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:00.273863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:02:00.273863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002006552,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"232'8","reported_seq":417,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.803355+0000","last_change":"2024-09-10T19:02:50.391929+0000","last_active":"2024-09-10T19:04:03.803355+0000","last_peered":"2024-09-10T19:04:03.803355+0000","last_clean":"2024-09-10T19:04:03.803355+0000","last_became_active":"2024-09-10T18:45:20.059205+0000","last_became_peered":"2024-09-10T18:45:20.059205+0000","last_unstale":"2024-09-10T19:04:03.803355+0000","last_undegraded":"2024-09-10T19:04:03.803355+0000","last_fullsized":"2024-09-10T19:04:03.803355+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'8","last_scrub_stamp":"2024-09-10T19:02:50.391489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:50.391489+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041757890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":425,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.805996+0000","last_change":"2024-09-10T19:02:16.704889+0000","last_active":"2024-09-10T19:04:03.805996+0000","last_peered":"2024-09-10T19:04:03.805996+0000","last_clean":"2024-09-10T19:04:03.805996+0000","last_became_active":"2024-09-10T18:47:36.415866+0000","last_became_peered":"2024-09-10T18:47:36.415866+0000","last_unstale":"2024-09-10T19:04:03.805996+0000","last_undegraded":"2024-09-10T19:04:03.805996+0000","last_fullsized":"2024-09-10T19:04:03.805996+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:16.704805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:16.704805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0083395729999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":436,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.808853+0000","last_change":"2024-09-10T19:02:08.092456+0000","last_active":"2024-09-10T19:04:03.808853+0000","last_peered":"2024-09-10T19:04:03.808853+0000","last_clean":"2024-09-10T19:04:03.808853+0000","last_became_active":"2024-09-10T18:57:02.453974+0000","last_became_peered":"2024-09-10T18:57:02.453974+0000","last_unstale":"2024-09-10T19:04:03.808853+0000","last_undegraded":"2024-09-10T19:04:03.808853+0000","last_fullsized":"2024-09-10T19:04:03.808853+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:08.092366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_clean_scrub_stamp":"2024-09-10T19:02:08.092366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058035439999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":196,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.800559+0000","last_change":"2024-09-10T18:59:55.812370+0000","last_active":"2024-09-10T19:04:03.800559+0000","last_peered":"2024-09-10T19:04:03.800559+0000","last_clean":"2024-09-10T19:04:03.800559+0000","last_became_active":"2024-09-10T18:57:02.447566+0000","last_became_peered":"2024-09-10T18:57:02.447566+0000","last_unstale":"2024-09-10T19:04:03.800559+0000","last_undegraded":"2024-09-10T19:04:03.800559+0000","last_fullsized":"2024-09-10T19:04:03.800559+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:59:55.812312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:55.145638+0000","last_clean_scrub_stamp":"2024-09-10T18:59:55.812312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0055522669999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.14","version":"70'81","reported_seq":310,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.803611+0000","last_change":"2024-09-10T19:02:03.086382+0000","last_active":"2024-09-10T19:04:03.803611+0000","last_peered":"2024-09-10T19:04:03.803611+0000","last_clean":"2024-09-10T19:04:03.803611+0000","last_became_active":"2024-09-10T18:57:02.442457+0000","last_became_peered":"2024-09-10T18:57:02.442457+0000","last_unstale":"2024-09-10T19:04:03.803611+0000","last_undegraded":"2024-09-10T19:04:03.803611+0000","last_fullsized":"2024-09-10T19:04:03.803611+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:03.086329+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:57.184383+0000","last_clean_scrub_stamp":"2024-09-10T19:02:03.086329+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026855270000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.11","version":"232'24","reported_seq":463,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.803710+0000","last_change":"2024-09-10T19:03:43.653800+0000","last_active":"2024-09-10T19:04:03.803710+0000","last_peered":"2024-09-10T19:04:03.803710+0000","last_clean":"2024-09-10T19:04:03.803710+0000","last_became_active":"2024-09-10T18:57:02.461909+0000","last_became_peered":"2024-09-10T18:57:02.461909+0000","last_unstale":"2024-09-10T19:04:03.803710+0000","last_undegraded":"2024-09-10T19:04:03.803710+0000","last_fullsized":"2024-09-10T19:04:03.803710+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T19:03:43.653711+0000","last_deep_scrub":"79'16","last_deep_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_clean_scrub_stamp":"2024-09-10T19:03:43.653711+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:05:09.089603+0000","scrub_duration":0.0037824379999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"56'1","reported_seq":441,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.806198+0000","last_change":"2024-09-10T19:01:58.121943+0000","last_active":"2024-09-10T19:04:03.806198+0000","last_peered":"2024-09-10T19:04:03.806198+0000","last_clean":"2024-09-10T19:04:03.806198+0000","last_became_active":"2024-09-10T18:47:36.421985+0000","last_became_peered":"2024-09-10T18:47:36.421985+0000","last_unstale":"2024-09-10T19:04:03.806198+0000","last_undegraded":"2024-09-10T19:04:03.806198+0000","last_fullsized":"2024-09-10T19:04:03.806198+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T19:01:58.121852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:01:58.121852+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002125984,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":153,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.800568+0000","last_change":"2024-09-10T19:02:14.088121+0000","last_active":"2024-09-10T19:04:03.800568+0000","last_peered":"2024-09-10T19:04:03.800568+0000","last_clean":"2024-09-10T19:04:03.800568+0000","last_became_active":"2024-09-10T18:57:02.435124+0000","last_became_peered":"2024-09-10T18:57:02.435124+0000","last_unstale":"2024-09-10T19:04:03.800568+0000","last_undegraded":"2024-09-10T19:04:03.800568+0000","last_fullsized":"2024-09-10T19:04:03.800568+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:14.088018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:02:14.088018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.031802916000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":418,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.803220+0000","last_change":"2024-09-10T19:02:16.321814+0000","last_active":"2024-09-10T19:04:03.803220+0000","last_peered":"2024-09-10T19:04:03.803220+0000","last_clean":"2024-09-10T19:04:03.803220+0000","last_became_active":"2024-09-10T18:55:40.397523+0000","last_became_peered":"2024-09-10T18:55:40.397523+0000","last_unstale":"2024-09-10T19:04:03.803220+0000","last_undegraded":"2024-09-10T19:04:03.803220+0000","last_fullsized":"2024-09-10T19:04:03.803220+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:16.321715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:02:16.321715+0000","last_clean_scrub_stamp":"2024-09-10T19:02:16.321715+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0069778510000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"78'2","reported_seq":196,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.806511+0000","last_change":"2024-09-10T19:02:15.686223+0000","last_active":"2024-09-10T19:04:03.806511+0000","last_peered":"2024-09-10T19:04:03.806511+0000","last_clean":"2024-09-10T19:04:03.806511+0000","last_became_active":"2024-09-10T18:57:02.447198+0000","last_became_peered":"2024-09-10T18:57:02.447198+0000","last_unstale":"2024-09-10T19:04:03.806511+0000","last_undegraded":"2024-09-10T19:04:03.806511+0000","last_fullsized":"2024-09-10T19:04:03.806511+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'2","last_scrub_stamp":"2024-09-10T19:02:15.686138+0000","last_deep_scrub":"78'2","last_deep_scrub_stamp":"2024-09-10T19:02:15.686138+0000","last_clean_scrub_stamp":"2024-09-10T19:02:15.686138+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011160388,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":429,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.806550+0000","last_change":"2024-09-10T19:03:44.943009+0000","last_active":"2024-09-10T19:04:03.806550+0000","last_peered":"2024-09-10T19:04:03.806550+0000","last_clean":"2024-09-10T19:04:03.806550+0000","last_became_active":"2024-09-10T18:57:02.443353+0000","last_became_peered":"2024-09-10T18:57:02.443353+0000","last_unstale":"2024-09-10T19:04:03.806550+0000","last_undegraded":"2024-09-10T19:04:03.806550+0000","last_fullsized":"2024-09-10T19:04:03.806550+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:44.942954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:03:44.942954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:50.259911+0000","scrub_duration":0.0016874889999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"232'20","reported_seq":436,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:09.801830+0000","last_change":"2024-09-10T19:02:04.609420+0000","last_active":"2024-09-10T19:04:09.801830+0000","last_peered":"2024-09-10T19:04:09.801830+0000","last_clean":"2024-09-10T19:04:09.801830+0000","last_became_active":"2024-09-10T18:57:02.452473+0000","last_became_peered":"2024-09-10T18:57:02.452473+0000","last_unstale":"2024-09-10T19:04:09.801830+0000","last_undegraded":"2024-09-10T19:04:09.801830+0000","last_fullsized":"2024-09-10T19:04:09.801830+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'20","last_scrub_stamp":"2024-09-10T19:02:04.609332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:04.609332+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0075275430000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":23,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"273'10","reported_seq":679,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:09.804086+0000","last_change":"2024-09-10T19:03:24.245622+0000","last_active":"2024-09-10T19:04:09.804086+0000","last_peered":"2024-09-10T19:04:09.804086+0000","last_clean":"2024-09-10T19:04:09.804086+0000","last_became_active":"2024-09-10T18:57:02.441763+0000","last_became_peered":"2024-09-10T18:57:02.441763+0000","last_unstale":"2024-09-10T19:04:09.804086+0000","last_undegraded":"2024-09-10T19:04:09.804086+0000","last_fullsized":"2024-09-10T19:04:09.804086+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'10","last_scrub_stamp":"2024-09-10T19:03:24.245532+0000","last_deep_scrub":"190'9","last_deep_scrub_stamp":"2024-09-10T18:53:07.746309+0000","last_clean_scrub_stamp":"2024-09-10T19:03:24.245532+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:42.657844+0000","scrub_duration":0.0019049480000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":38,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.808368+0000","last_change":"2024-09-10T19:02:49.582100+0000","last_active":"2024-09-10T19:04:03.808368+0000","last_peered":"2024-09-10T19:04:03.808368+0000","last_clean":"2024-09-10T19:04:03.808368+0000","last_became_active":"2024-09-10T19:02:49.581739+0000","last_became_peered":"2024-09-10T19:02:49.581739+0000","last_unstale":"2024-09-10T19:04:03.808368+0000","last_undegraded":"2024-09-10T19:04:03.808368+0000","last_fullsized":"2024-09-10T19:04:03.808368+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:23.327767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:03.475299+0000","last_clean_scrub_stamp":"2024-09-10T19:02:23.327767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":414,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.803262+0000","last_change":"2024-09-10T19:00:10.182129+0000","last_active":"2024-09-10T19:04:03.803262+0000","last_peered":"2024-09-10T19:04:03.803262+0000","last_clean":"2024-09-10T19:04:03.803262+0000","last_became_active":"2024-09-10T18:55:40.392524+0000","last_became_peered":"2024-09-10T18:55:40.392524+0000","last_unstale":"2024-09-10T19:04:03.803262+0000","last_undegraded":"2024-09-10T19:04:03.803262+0000","last_fullsized":"2024-09-10T19:04:03.803262+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:00:10.181982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:00:10.181982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029510420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"232'19","reported_seq":452,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.806791+0000","last_change":"2024-09-10T19:02:12.120953+0000","last_active":"2024-09-10T19:04:03.806791+0000","last_peered":"2024-09-10T19:04:03.806791+0000","last_clean":"2024-09-10T19:04:03.806791+0000","last_became_active":"2024-09-10T18:45:20.058717+0000","last_became_peered":"2024-09-10T18:45:20.058717+0000","last_unstale":"2024-09-10T19:04:03.806791+0000","last_undegraded":"2024-09-10T19:04:03.806791+0000","last_fullsized":"2024-09-10T19:04:03.806791+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T19:02:12.120860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:12.120860+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060875590000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":390,"reported_epoch":398,"state":"active+clean","last_fresh":"2024-09-10T19:04:09.802106+0000","last_change":"2024-09-10T19:03:25.864320+0000","last_active":"2024-09-10T19:04:09.802106+0000","last_peered":"2024-09-10T19:04:09.802106+0000","last_clean":"2024-09-10T19:04:09.802106+0000","last_became_active":"2024-09-10T18:57:02.453464+0000","last_became_peered":"2024-09-10T18:57:02.453464+0000","last_unstale":"2024-09-10T19:04:09.802106+0000","last_undegraded":"2024-09-10T19:04:09.802106+0000","last_fullsized":"2024-09-10T19:04:09.802106+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:25.864264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:03:25.864264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:36.948487+0000","scrub_duration":0.0013441410000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":401,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.805856+0000","last_change":"2024-09-10T19:03:43.645318+0000","last_active":"2024-09-10T19:04:03.805856+0000","last_peered":"2024-09-10T19:04:03.805856+0000","last_clean":"2024-09-10T19:04:03.805856+0000","last_became_active":"2024-09-10T18:45:24.267251+0000","last_became_peered":"2024-09-10T18:45:24.267251+0000","last_unstale":"2024-09-10T19:04:03.805856+0000","last_undegraded":"2024-09-10T19:04:03.805856+0000","last_fullsized":"2024-09-10T19:04:03.805856+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:43.645260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:03:43.645260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:48.541822+0000","scrub_duration":0.003834732,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"306'2","reported_seq":447,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.807486+0000","last_change":"2024-09-10T19:03:00.120585+0000","last_active":"2024-09-10T19:04:03.807486+0000","last_peered":"2024-09-10T19:04:03.807486+0000","last_clean":"2024-09-10T19:04:03.807486+0000","last_became_active":"2024-09-10T18:57:02.435758+0000","last_became_peered":"2024-09-10T18:57:02.435758+0000","last_unstale":"2024-09-10T19:04:03.807486+0000","last_undegraded":"2024-09-10T19:04:03.807486+0000","last_fullsized":"2024-09-10T19:04:03.807486+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"306'2","last_scrub_stamp":"2024-09-10T19:03:00.120529+0000","last_deep_scrub":"306'2","last_deep_scrub_stamp":"2024-09-10T18:59:20.534869+0000","last_clean_scrub_stamp":"2024-09-10T19:03:00.120529+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:18.610353+0000","scrub_duration":0.0016153859999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"258'34","reported_seq":345,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.804148+0000","last_change":"2024-09-10T19:02:49.594744+0000","last_active":"2024-09-10T19:04:03.804148+0000","last_peered":"2024-09-10T19:04:03.804148+0000","last_clean":"2024-09-10T19:04:03.804148+0000","last_became_active":"2024-09-10T19:02:49.594207+0000","last_became_peered":"2024-09-10T19:02:49.594207+0000","last_unstale":"2024-09-10T19:04:03.804148+0000","last_undegraded":"2024-09-10T19:04:03.804148+0000","last_fullsized":"2024-09-10T19:04:03.804148+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'34","last_scrub_stamp":"2024-09-10T19:02:02.373321+0000","last_deep_scrub":"258'34","last_deep_scrub_stamp":"2024-09-10T18:56:23.838951+0000","last_clean_scrub_stamp":"2024-09-10T19:02:02.373321+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030853120000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":266,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.805803+0000","last_change":"2024-09-10T19:02:49.581903+0000","last_active":"2024-09-10T19:04:03.805803+0000","last_peered":"2024-09-10T19:04:03.805803+0000","last_clean":"2024-09-10T19:04:03.805803+0000","last_became_active":"2024-09-10T19:02:49.581575+0000","last_became_peered":"2024-09-10T19:02:49.581575+0000","last_unstale":"2024-09-10T19:04:03.805803+0000","last_undegraded":"2024-09-10T19:04:03.805803+0000","last_fullsized":"2024-09-10T19:04:03.805803+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:03.343366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:02:03.343366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0079641550000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":421,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.805081+0000","last_change":"2024-09-10T19:03:33.638108+0000","last_active":"2024-09-10T19:04:03.805081+0000","last_peered":"2024-09-10T19:04:03.805081+0000","last_clean":"2024-09-10T19:04:03.805081+0000","last_became_active":"2024-09-10T19:00:42.520239+0000","last_became_peered":"2024-09-10T19:00:42.520239+0000","last_unstale":"2024-09-10T19:04:03.805081+0000","last_undegraded":"2024-09-10T19:04:03.805081+0000","last_fullsized":"2024-09-10T19:04:03.805081+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:33.638010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:17.985545+0000","last_clean_scrub_stamp":"2024-09-10T19:03:33.638010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:04:48.051162+0000","scrub_duration":0.0035871750000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":427,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.804395+0000","last_change":"2024-09-10T19:03:36.615878+0000","last_active":"2024-09-10T19:04:03.804395+0000","last_peered":"2024-09-10T19:04:03.804395+0000","last_clean":"2024-09-10T19:04:03.804395+0000","last_became_active":"2024-09-10T18:57:02.460855+0000","last_became_peered":"2024-09-10T18:57:02.460855+0000","last_unstale":"2024-09-10T19:04:03.804395+0000","last_undegraded":"2024-09-10T19:04:03.804395+0000","last_fullsized":"2024-09-10T19:04:03.804395+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:36.615788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:03:36.615788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:05:00.940861+0000","scrub_duration":0.004374223,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"388'23","reported_seq":442,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.804626+0000","last_change":"2024-09-10T19:03:40.698360+0000","last_active":"2024-09-10T19:04:03.804626+0000","last_peered":"2024-09-10T19:04:03.804626+0000","last_clean":"2024-09-10T19:04:03.804626+0000","last_became_active":"2024-09-10T18:45:20.058391+0000","last_became_peered":"2024-09-10T18:45:20.058391+0000","last_unstale":"2024-09-10T19:04:03.804626+0000","last_undegraded":"2024-09-10T19:04:03.804626+0000","last_fullsized":"2024-09-10T19:04:03.804626+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"388'23","last_scrub_stamp":"2024-09-10T19:03:40.697702+0000","last_deep_scrub":"388'23","last_deep_scrub_stamp":"2024-09-10T19:03:40.697702+0000","last_clean_scrub_stamp":"2024-09-10T19:03:40.697702+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:05:03.137428+0000","scrub_duration":0.076817413000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":433,"reported_epoch":397,"state":"active+clean","last_fresh":"2024-09-10T19:04:03.803599+0000","last_change":"2024-09-10T19:03:46.623850+0000","last_active":"2024-09-10T19:04:03.803599+0000","last_peered":"2024-09-10T19:04:03.803599+0000","last_clean":"2024-09-10T19:04:03.803599+0000","last_became_active":"2024-09-10T18:57:02.449135+0000","last_became_peered":"2024-09-10T18:57:02.449135+0000","last_unstale":"2024-09-10T19:04:03.803599+0000","last_undegraded":"2024-09-10T19:04:03.803599+0000","last_fullsized":"2024-09-10T19:04:03.803599+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:46.623782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:03:46.623782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:05:14.666450+0000","scrub_duration":0.0075382380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":16,"stat_sum":{"num_bytes":5103288320,"num_objects":77870,"num_object_clones":0,"num_object_copies":233610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":77870,"num_write_kb":4983680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15366356992,"data_stored":15366356992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58370,"ondisk_log_size":58370,"up":48,"acting":48,"num_store_stats":6},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":421,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1115,"num_read_kb":872,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":3152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":499712,"data_stored":13268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":771,"ondisk_log_size":771,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":194,"num_read_kb":189,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":608258,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":6,"num_write":3,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":298468,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1701,"ondisk_log_size":1701,"up":63,"acting":63,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":0,"seq":223338299598,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":43,"seq":184683593964,"num_pgs":88,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3648044,"kb_used_data":2758048,"kb_used_omap":1,"kb_used_meta":889982,"kb_avail":90097108,"statfs":{"total":95995035648,"available":92259438592,"internally_reserved":0,"allocated":2824241152,"data_stored":2823727919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":911341912},"hb_peers":[0,1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:03:59 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":1.968,"15min":1.3240000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.20899999999999999},"max":{"1min":3.2109999999999999,"5min":49.353999999999999,"15min":49.353999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.77900000000000003,"5min":2.0139999999999998,"15min":1.4179999999999999},"min":{"1min":0.28699999999999998,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":3.2010000000000001,"5min":49.341999999999999,"15min":49.341999999999999},"last":0.624}]},{"osd":1,"last update":"Tue Sep 10 19:03:13 2024","interfaces":[{"interface":"back","average":{"1min":2.234,"5min":1.96,"15min":1.4530000000000001},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":23.716000000000001,"5min":45.311,"15min":45.311},"last":2.7519999999999998},{"interface":"front","average":{"1min":2.21,"5min":1.8620000000000001,"15min":1.3280000000000001},"min":{"1min":0.33800000000000002,"5min":0.25900000000000001,"15min":0.214},"max":{"1min":23.594000000000001,"5min":45.027999999999999,"15min":45.027999999999999},"last":0.98699999999999999}]},{"osd":2,"last update":"Tue Sep 10 19:03:13 2024","interfaces":[{"interface":"back","average":{"1min":1.71,"5min":0.93200000000000005,"15min":0.89600000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":23.867000000000001,"5min":23.867000000000001,"15min":23.867000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":1.7050000000000001,"5min":0.94599999999999995,"15min":0.83999999999999997},"min":{"1min":0.29399999999999998,"5min":0.249,"15min":0.249},"max":{"1min":23.779,"5min":23.779,"15min":23.779},"last":0.371}]},{"osd":3,"last update":"Tue Sep 10 19:03:13 2024","interfaces":[{"interface":"back","average":{"1min":1.6100000000000001,"5min":2.6579999999999999,"15min":1.6699999999999999},"min":{"1min":0.33300000000000002,"5min":0.29599999999999999,"15min":0.25700000000000001},"max":{"1min":23.640999999999998,"5min":76.977999999999994,"15min":76.977999999999994},"last":0.58099999999999996},{"interface":"front","average":{"1min":1.724,"5min":2.6509999999999998,"15min":1.774},"min":{"1min":0.28499999999999998,"5min":0.27500000000000002,"15min":0.253},"max":{"1min":23.922000000000001,"5min":76.879999999999995,"15min":76.879999999999995},"last":0.80800000000000005}]},{"osd":4,"last update":"Tue Sep 10 19:03:13 2024","interfaces":[{"interface":"back","average":{"1min":2.2829999999999999,"5min":1.1220000000000001,"15min":1.5109999999999999},"min":{"1min":0.27100000000000002,"5min":0.23400000000000001,"15min":0.20999999999999999},"max":{"1min":32.469000000000001,"5min":32.469000000000001,"15min":165.03299999999999},"last":0.318},{"interface":"front","average":{"1min":1.9299999999999999,"5min":0.94599999999999995,"15min":1.377},"min":{"1min":0.28999999999999998,"5min":0.26000000000000001,"15min":0.22500000000000001},"max":{"1min":32.5,"5min":32.5,"15min":163.959},"last":0.38500000000000001}]},{"osd":5,"last update":"Tue Sep 10 19:03:13 2024","interfaces":[{"interface":"back","average":{"1min":1.7729999999999999,"5min":3.3690000000000002,"15min":1.9039999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.24099999999999999},"max":{"1min":23.827000000000002,"5min":288.56999999999999,"15min":288.56999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":1.7010000000000001,"5min":3.27,"15min":1.891},"min":{"1min":0.28100000000000003,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":23.667000000000002,"5min":291.05399999999997,"15min":291.05399999999997},"last":0.40799999999999997}]},{"osd":7,"last update":"Tue Sep 10 19:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.67800000000000005,"15min":0.83499999999999996},"min":{"1min":0.32200000000000001,"5min":0.27900000000000003,"15min":0.26200000000000001},"max":{"1min":2.2839999999999998,"5min":2.98,"15min":5.3680000000000003},"last":0.5},{"interface":"front","average":{"1min":1.1379999999999999,"5min":0.72499999999999998,"15min":0.88900000000000001},"min":{"1min":0.31,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":4.6820000000000004,"5min":4.6820000000000004,"15min":7.0350000000000001},"last":0.39500000000000002}]}]},{"osd":5,"up_from":37,"seq":158913790192,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2734892,"kb_used_data":1842912,"kb_used_omap":1,"kb_used_meta":891966,"kb_avail":91010260,"statfs":{"total":95995035648,"available":93194506240,"internally_reserved":0,"allocated":1887141888,"data_stored":1886627684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":913373528},"hb_peers":[0,1,2,3,4,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:03:52 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":1.2450000000000001,"15min":1.9690000000000001},"min":{"1min":0.30399999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.5389999999999999,"5min":31.626999999999999,"15min":82.465000000000003},"last":6.2709999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":1.3580000000000001,"15min":2.1269999999999998},"min":{"1min":0.314,"5min":0.218,"15min":0.218},"max":{"1min":1.3280000000000001,"5min":31.734999999999999,"15min":82.519000000000005},"last":6.2590000000000003}]},{"osd":1,"last update":"Tue Sep 10 19:03:52 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":1.167,"15min":1.6739999999999999},"min":{"1min":0.29499999999999998,"5min":0.20999999999999999,"15min":0.19500000000000001},"max":{"1min":0.92600000000000005,"5min":31.657,"15min":67.582999999999998},"last":5.835},{"interface":"front","average":{"1min":0.51300000000000001,"5min":1.296,"15min":1.835},"min":{"1min":0.309,"5min":0.29699999999999999,"15min":0.20399999999999999},"max":{"1min":0.93600000000000005,"5min":31.678999999999998,"15min":67.603999999999999},"last":5.8230000000000004}]},{"osd":2,"last update":"Tue Sep 10 19:03:52 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":1.097,"15min":0.92000000000000004},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.71699999999999997,"5min":31.774000000000001,"15min":31.774000000000001},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.42199999999999999,"5min":1.046,"15min":0.86799999999999999},"min":{"1min":0.27200000000000002,"5min":0.27100000000000002,"15min":0.24399999999999999},"max":{"1min":0.58499999999999996,"5min":31.722999999999999,"15min":31.722999999999999},"last":0.32200000000000001}]},{"osd":3,"last update":"Tue Sep 10 19:03:52 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":1.234,"15min":2.8559999999999999},"min":{"1min":0.26300000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":1.2250000000000001,"5min":31.753,"15min":123.34399999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.745,"5min":1.2569999999999999,"15min":2.4420000000000002},"min":{"1min":0.313,"5min":0.27100000000000002,"15min":0.245},"max":{"1min":4.5049999999999999,"5min":31.699999999999999,"15min":82.415000000000006},"last":0.26200000000000001}]},{"osd":4,"last update":"Tue Sep 10 19:03:52 2024","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":1.345,"15min":1.1439999999999999},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.60299999999999998,"5min":51.070999999999998,"15min":51.070999999999998},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.33700000000000002,"5min":1.2529999999999999,"15min":1.0469999999999999},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.58999999999999997,"5min":51.122999999999998,"15min":51.122999999999998},"last":0.23300000000000001}]},{"osd":6,"last update":"Tue Sep 10 19:03:12 2024","interfaces":[{"interface":"back","average":{"1min":2.786,"5min":1.226,"15min":1.181},"min":{"1min":0.318,"5min":0.23999999999999999,"15min":0.23400000000000001},"max":{"1min":33.936999999999998,"5min":33.936999999999998,"15min":33.936999999999998},"last":0.29499999999999998},{"interface":"front","average":{"1min":2.8330000000000002,"5min":1.1850000000000001,"15min":1.1499999999999999},"min":{"1min":0.33100000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":33.951000000000001,"5min":33.951000000000001,"15min":33.951000000000001},"last":0.307}]},{"osd":7,"last update":"Tue Sep 10 19:01:31 2024","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":0.69399999999999995,"15min":0.94399999999999995},"min":{"1min":0.308,"5min":0.28999999999999998,"15min":0.27600000000000002},"max":{"1min":2.6859999999999999,"5min":3.665,"15min":10.782999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.61299999999999999,"15min":0.83199999999999996},"min":{"1min":0.34699999999999998,"5min":0.26900000000000002,"15min":0.23599999999999999},"max":{"1min":2.6200000000000001,"5min":2.6200000000000001,"15min":11.846},"last":0.33800000000000002}]}]},{"osd":4,"up_from":30,"seq":128849019126,"num_pgs":80,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3183012,"kb_used_data":2296856,"kb_used_omap":1,"kb_used_meta":886142,"kb_avail":90562140,"statfs":{"total":95995035648,"available":92735631360,"internally_reserved":0,"allocated":2351980544,"data_stored":2351474525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":907409752},"hb_peers":[0,1,2,3,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":1.238,"15min":1.4019999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":6.3819999999999997,"5min":13.119999999999999,"15min":46.234999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.83499999999999996,"5min":1.1619999999999999,"15min":1.3999999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":6.6669999999999998,"5min":10.634,"15min":46.264000000000003},"last":0.46800000000000003}]},{"osd":1,"last update":"Tue Sep 10 19:03:35 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.82699999999999996,"15min":1.627},"min":{"1min":0.35799999999999998,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":3.5510000000000002,"5min":3.867,"15min":125.15600000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72799999999999998,"15min":1.5329999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23100000000000001},"max":{"1min":3.2999999999999998,"5min":4.0069999999999997,"15min":125.14700000000001},"last":0.53000000000000003}]},{"osd":2,"last update":"Tue Sep 10 19:03:35 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.70999999999999996,"15min":0.71499999999999997},"min":{"1min":0.28499999999999998,"5min":0.26200000000000001,"15min":0.25900000000000001},"max":{"1min":3.3559999999999999,"5min":3.3999999999999999,"15min":5.6189999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.92400000000000004,"15min":0.82399999999999995},"min":{"1min":0.30199999999999999,"5min":0.28299999999999997,"15min":0.27100000000000002},"max":{"1min":2.427,"5min":12.782999999999999,"15min":12.782999999999999},"last":0.41499999999999998}]},{"osd":3,"last update":"Tue Sep 10 19:03:35 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":1.5329999999999999,"15min":2.1699999999999999},"min":{"1min":0.317,"5min":0.28100000000000003,"15min":0.23799999999999999},"max":{"1min":2.431,"5min":34.676000000000002,"15min":111.514},"last":2.0019999999999998},{"interface":"front","average":{"1min":0.876,"5min":1.294,"15min":1.76},"min":{"1min":0.29199999999999998,"5min":0.26900000000000002,"15min":0.25700000000000001},"max":{"1min":3.3980000000000001,"5min":34.612000000000002,"15min":110.914},"last":1.6060000000000001}]},{"osd":5,"last update":"Tue Sep 10 19:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.79300000000000004,"15min":0.95399999999999996},"min":{"1min":0.24099999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.7090000000000001,"5min":6.1779999999999999,"15min":10.912000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.78400000000000003,"15min":0.90300000000000002},"min":{"1min":0.24099999999999999,"5min":0.216,"15min":0.216},"max":{"1min":0.72799999999999998,"5min":5.3849999999999998,"15min":10.147},"last":0.42499999999999999}]},{"osd":6,"last update":"Tue Sep 10 19:03:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.71699999999999997,"15min":0.81399999999999995},"min":{"1min":0.254,"5min":0.221,"15min":0.221},"max":{"1min":5.351,"5min":5.351,"15min":8.4030000000000005},"last":0.502},{"interface":"front","average":{"1min":1.151,"5min":1.081,"15min":1.0189999999999999},"min":{"1min":0.26700000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":4.3479999999999999,"5min":21.254000000000001,"15min":21.254000000000001},"last":0.39400000000000002}]},{"osd":7,"last update":"Tue Sep 10 19:01:46 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.60899999999999999,"15min":0.82999999999999996},"min":{"1min":0.40600000000000003,"5min":0.29099999999999998,"15min":0.27600000000000002},"max":{"1min":1.516,"5min":2.8879999999999999,"15min":6.4630000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":1.05,"5min":0.71099999999999997,"15min":0.83699999999999997},"min":{"1min":0.311,"5min":0.27500000000000002,"15min":0.251},"max":{"1min":3.1219999999999999,"5min":3.589,"15min":6.3949999999999996},"last":0.86399999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182651,"num_pgs":85,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3919724,"kb_used_data":2733600,"kb_used_omap":1,"kb_used_meta":1186110,"kb_avail":89825428,"statfs":{"total":95995035648,"available":91981238272,"internally_reserved":0,"allocated":2799206400,"data_stored":2798692087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":1214576984},"hb_peers":[0,1,2,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,5],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:03:53 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":1.073,"15min":2.3860000000000001},"min":{"1min":0.22500000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":3.9660000000000002,"5min":5.6349999999999998,"15min":290.721},"last":5.2240000000000002},{"interface":"front","average":{"1min":1.167,"5min":1.218,"15min":2.5760000000000001},"min":{"1min":0.18099999999999999,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":5.242,"5min":6.609,"15min":293.59300000000002},"last":1.9079999999999999}]},{"osd":1,"last update":"Tue Sep 10 19:04:10 2024","interfaces":[{"interface":"back","average":{"1min":1.649,"5min":1.1259999999999999,"15min":1.6359999999999999},"min":{"1min":0.21099999999999999,"5min":0.20399999999999999,"15min":0.16200000000000001},"max":{"1min":5.8159999999999998,"5min":10.645,"15min":58.215000000000003},"last":5.3499999999999996},{"interface":"front","average":{"1min":1.744,"5min":1.2050000000000001,"15min":1.577},"min":{"1min":0.29999999999999999,"5min":0.22700000000000001,"15min":0.188},"max":{"1min":5.8390000000000004,"5min":13.263999999999999,"15min":57.896999999999998},"last":2.1429999999999998}]},{"osd":2,"last update":"Tue Sep 10 19:04:10 2024","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":0.90900000000000003,"15min":1.006},"min":{"1min":0.26600000000000001,"5min":0.255,"15min":0.22800000000000001},"max":{"1min":4.2409999999999997,"5min":4.5670000000000002,"15min":5.8220000000000001},"last":2.202},{"interface":"front","average":{"1min":1.1819999999999999,"5min":0.88,"15min":0.96299999999999997},"min":{"1min":0.34499999999999997,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":5.0250000000000004,"5min":5.46,"15min":7.0979999999999999},"last":3.1819999999999999}]},{"osd":4,"last update":"Tue Sep 10 19:03:36 2024","interfaces":[{"interface":"back","average":{"1min":1.659,"5min":1.4259999999999999,"15min":1.3819999999999999},"min":{"1min":0.34000000000000002,"5min":0.33400000000000002,"15min":0.247},"max":{"1min":16.356000000000002,"5min":16.356000000000002,"15min":29.013000000000002},"last":3.0670000000000002},{"interface":"front","average":{"1min":1.7010000000000001,"5min":1.4530000000000001,"15min":1.397},"min":{"1min":0.374,"5min":0.31900000000000001,"15min":0.26800000000000002},"max":{"1min":17.289999999999999,"5min":17.289999999999999,"15min":28.826000000000001},"last":3.1549999999999998}]},{"osd":5,"last update":"Tue Sep 10 19:03:54 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1,"15min":1.276},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":4.2089999999999996,"5min":5.7839999999999998,"15min":7.2300000000000004},"last":5.1920000000000002},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.3859999999999999,"15min":1.3560000000000001},"min":{"1min":0.34000000000000002,"5min":0.28299999999999997,"15min":0.255},"max":{"1min":5.4119999999999999,"5min":17.899999999999999,"15min":17.899999999999999},"last":0.308}]},{"osd":6,"last update":"Tue Sep 10 19:03:13 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":1.3340000000000001,"15min":1.347},"min":{"1min":0.39200000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":3.0049999999999999,"5min":5.4480000000000004,"15min":9.3239999999999998},"last":0.33700000000000002},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.22,"15min":1.3089999999999999},"min":{"1min":0.35799999999999998,"5min":0.34999999999999998,"15min":0.29399999999999998},"max":{"1min":3.3090000000000002,"5min":6.6390000000000002,"15min":7.1559999999999997},"last":3.1240000000000001}]},{"osd":7,"last update":"Tue Sep 10 19:01:40 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":1.5289999999999999,"15min":1.294},"min":{"1min":0.39700000000000002,"5min":0.35599999999999998,"15min":0.34899999999999998},"max":{"1min":2.7290000000000001,"5min":8.7989999999999995,"15min":8.7989999999999995},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":1.46,"15min":1.3140000000000001},"min":{"1min":0.42999999999999999,"5min":0.36899999999999999,"15min":0.33100000000000002},"max":{"1min":2.266,"5min":6.6029999999999998,"15min":9.2949999999999999},"last":0.71599999999999997}]}]},{"osd":2,"up_from":0,"seq":77309411583,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834575107,"num_pgs":93,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4319236,"kb_used_data":3428412,"kb_used_omap":1,"kb_used_meta":890814,"kb_avail":89425916,"statfs":{"total":95995035648,"available":91572137984,"internally_reserved":0,"allocated":3510693888,"data_stored":3510152366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":912193880},"hb_peers":[0,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:03:55 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":1.53,"15min":1.827},"min":{"1min":0.214,"5min":0.158,"15min":0.158},"max":{"1min":4.351,"5min":45.786000000000001,"15min":111.014},"last":1.151},{"interface":"front","average":{"1min":0.86099999999999999,"5min":1.613,"15min":1.8280000000000001},"min":{"1min":0.29599999999999999,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":2.6640000000000001,"5min":46.311999999999998,"15min":111.072},"last":1.1200000000000001}]},{"osd":2,"last update":"Tue Sep 10 19:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.69999999999999996,"15min":0.77300000000000002},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.186},"max":{"1min":2.3490000000000002,"5min":3.2970000000000002,"15min":4.1719999999999997},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.79100000000000004,"15min":0.82499999999999996},"min":{"1min":0.27300000000000002,"5min":0.23899999999999999,"15min":0.215},"max":{"1min":1.446,"5min":4.2560000000000002,"15min":7.093},"last":0.57299999999999995}]},{"osd":3,"last update":"Tue Sep 10 19:04:08 2024","interfaces":[{"interface":"back","average":{"1min":3.5030000000000001,"5min":1.671,"15min":2.2810000000000001},"min":{"1min":0.27500000000000002,"5min":0.22800000000000001,"15min":0.185},"max":{"1min":48.582999999999998,"5min":48.582999999999998,"15min":92.069000000000003},"last":1.1739999999999999},{"interface":"front","average":{"1min":3.5070000000000001,"5min":1.3140000000000001,"15min":2.044},"min":{"1min":0.254,"5min":0.23100000000000001,"15min":0.19900000000000001},"max":{"1min":51.847000000000001,"5min":51.847000000000001,"15min":88.658000000000001},"last":0.86299999999999999}]},{"osd":4,"last update":"Tue Sep 10 19:03:30 2024","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.042,"15min":1.2190000000000001},"min":{"1min":0.33000000000000002,"5min":0.27800000000000002,"15min":0.26200000000000001},"max":{"1min":19.126999999999999,"5min":19.126999999999999,"15min":19.126999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.0740000000000001,"15min":1.133},"min":{"1min":0.37,"5min":0.28399999999999997,"15min":0.246},"max":{"1min":20.152000000000001,"5min":20.152000000000001,"15min":20.152000000000001},"last":0.51100000000000001}]},{"osd":5,"last update":"Tue Sep 10 19:03:55 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":1.0389999999999999,"15min":1.234},"min":{"1min":0.317,"5min":0.29999999999999999,"15min":0.28499999999999998},"max":{"1min":1.538,"5min":4.226,"15min":10.334},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.93799999999999994,"15min":1.2230000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.28799999999999998},"max":{"1min":1.1659999999999999,"5min":7.0590000000000002,"15min":16.039999999999999},"last":0.52600000000000002}]},{"osd":6,"last update":"Tue Sep 10 19:04:08 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":1.2549999999999999,"15min":1.5720000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.308},"max":{"1min":1.49,"5min":19.125,"15min":38.039999999999999},"last":1.482},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.2150000000000001,"15min":1.679},"min":{"1min":0.35699999999999998,"5min":0.32600000000000001,"15min":0.26600000000000001},"max":{"1min":4.3360000000000003,"5min":17.352,"15min":38.100999999999999},"last":1.444}]},{"osd":7,"last update":"Tue Sep 10 19:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":1.2330000000000001,"15min":1.163},"min":{"1min":0.39800000000000002,"5min":0.33100000000000002,"15min":0.315},"max":{"1min":4.1710000000000003,"5min":19.169,"15min":19.169},"last":2.581},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.995,"15min":1.0589999999999999},"min":{"1min":0.39600000000000002,"5min":0.36499999999999999,"15min":0.32800000000000001},"max":{"1min":2.2410000000000001,"5min":4.46,"15min":4.9029999999999996},"last":2.4510000000000001}]}]},{"osd":0,"up_from":241,"seq":1035087118440,"num_pgs":79,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3128800,"kb_used_data":2498328,"kb_used_omap":4,"kb_used_meta":630459,"kb_avail":90616352,"statfs":{"total":95995035648,"available":92791144448,"internally_reserved":0,"allocated":2558287872,"data_stored":2557781109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":645590534},"hb_peers":[1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,2,1,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 19:03:52 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.90900000000000003,"15min":2.0640000000000001},"min":{"1min":0.184,"5min":0.184,"15min":0.17100000000000001},"max":{"1min":1.5029999999999999,"5min":6.9050000000000002,"15min":34.692},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.90000000000000002,"15min":2.0609999999999999},"min":{"1min":0.20100000000000001,"5min":0.193,"15min":0.193},"max":{"1min":2.4769999999999999,"5min":7.5960000000000001,"15min":34.588999999999999},"last":0.29299999999999998}]},{"osd":2,"last update":"Tue Sep 10 19:03:52 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.73699999999999999,"15min":0.89100000000000001},"min":{"1min":0.24299999999999999,"5min":0.23400000000000001,"15min":0.20699999999999999},"max":{"1min":1.3819999999999999,"5min":6.8639999999999999,"15min":13.305},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.435,"5min":0.61199999999999999,"15min":0.79700000000000004},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22800000000000001},"max":{"1min":1.1200000000000001,"5min":3.0960000000000001,"15min":13.693},"last":0.47499999999999998}]},{"osd":3,"last update":"Tue Sep 10 19:03:52 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":1.026,"15min":1.137},"min":{"1min":0.32600000000000001,"5min":0.216,"15min":0.216},"max":{"1min":2.153,"5min":6.2560000000000002,"15min":21.108000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.76200000000000001,"5min":1.155,"15min":1.585},"min":{"1min":0.26600000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":2.4870000000000001,"5min":7.5140000000000002,"15min":11.629},"last":0.45900000000000002}]},{"osd":4,"last update":"Tue Sep 10 19:03:52 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":1.0609999999999999,"15min":0.89500000000000002},"min":{"1min":0.35399999999999998,"5min":0.26400000000000001,"15min":0.26000000000000001},"max":{"1min":1.415,"5min":4.2830000000000004,"15min":13.161},"last":2.0379999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":1.1220000000000001,"15min":1.0089999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.27600000000000002},"max":{"1min":0.75900000000000001,"5min":5.0490000000000004,"15min":5.0490000000000004},"last":0.40500000000000003}]},{"osd":5,"last update":"Tue Sep 10 19:03:52 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":2.246,"15min":1.456},"min":{"1min":0.35699999999999998,"5min":0.33200000000000002,"15min":0.31},"max":{"1min":0.65300000000000002,"5min":97.296999999999997,"15min":97.296999999999997},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":2.1360000000000001,"15min":1.3109999999999999},"min":{"1min":0.39400000000000002,"5min":0.28599999999999998,"15min":0.28399999999999997},"max":{"1min":1.4430000000000001,"5min":99.221999999999994,"15min":99.221999999999994},"last":1.802}]},{"osd":6,"last update":"Tue Sep 10 19:03:52 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":1.383,"15min":1.111},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.312},"max":{"1min":2.2989999999999999,"5min":14.537000000000001,"15min":14.537000000000001},"last":0.435},{"interface":"front","average":{"1min":0.81200000000000006,"5min":1.46,"15min":1.0209999999999999},"min":{"1min":0.442,"5min":0.375,"15min":0.313},"max":{"1min":1.518,"5min":18.931999999999999,"15min":18.931999999999999},"last":2.0070000000000001}]},{"osd":7,"last update":"Tue Sep 10 19:01:49 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.2509999999999999,"15min":0.96799999999999997},"min":{"1min":0.46899999999999997,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":4.3380000000000001,"5min":13.233000000000001,"15min":13.233000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":2.9430000000000001,"5min":1.494,"15min":1.262},"min":{"1min":0.40100000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":38.636000000000003,"5min":38.636000000000003,"15min":38.636000000000003},"last":0.76800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":2668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2326134784,"data_stored":2326134784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3539009536,"data_stored":3539009536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2829713408,"data_stored":2829713408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1921974272,"data_stored":1921974272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1906835456,"data_stored":1906835456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2842689536,"data_stored":2842689536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T19:04:18.215 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2024-09-10T19:04:18.215 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd pool set unique_pool_4 pg_num 26 2024-09-10T19:04:18.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:18.257 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:18.257 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:18.358 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:18.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:18.409 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:18.409 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:18.510 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:18.553 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:18.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:18.555 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:18.655 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:18.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:18 smithi018 ceph-mon[29336]: from='client.15870 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:04:18.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:18 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:18.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:18 smithi018 ceph-mon[29336]: pgmap v883: 168 pgs: 168 active+clean; 5.1 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 1.2 KiB/s rd, 101 MiB/s wr, 1.61k op/s 2024-09-10T19:04:18.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:18 smithi018 ceph-mon[40196]: from='client.15870 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:04:18.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:18 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:18.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:18 smithi018 ceph-mon[40196]: pgmap v883: 168 pgs: 168 active+clean; 5.1 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 1.2 KiB/s rd, 101 MiB/s wr, 1.61k op/s 2024-09-10T19:04:18.701 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:18.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:18.702 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:18.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:18 smithi103 ceph-mon[33163]: from='client.15870 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:04:18.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:18 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:18.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:18 smithi103 ceph-mon[33163]: pgmap v883: 168 pgs: 168 active+clean; 5.1 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 1.2 KiB/s rd, 101 MiB/s wr, 1.61k op/s 2024-09-10T19:04:18.803 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:18.828 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:18.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:18.829 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:18.868 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:04:18.931 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:18.960 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:18.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:18.961 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:19.062 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:19.074 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 51 16 88104 88088 107.933 81.75 0.00705603 0.00925986 2024-09-10T19:04:19.090 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:19.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:19.091 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:19.192 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:19.218 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:19.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:19.219 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:19.320 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:19.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:19.365 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:19.366 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:19.467 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:19.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:19.500 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:19.500 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:19.601 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:19.629 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:19.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:19.630 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:19.731 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:19.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:19.770 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:19.770 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:19.871 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:19.895 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:19.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:19.896 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:19.997 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:20.039 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:20.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:20.040 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:20.074 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 52 16 89604 89588 107.66 93.75 0.00678053 0.00928384 2024-09-10T19:04:20.141 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:20.181 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:20.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:20.182 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:20.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:19 smithi018 ceph-mon[29336]: pgmap v884: 168 pgs: 168 active+clean; 5.2 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 1.0 KiB/s rd, 89 MiB/s wr, 1.43k op/s 2024-09-10T19:04:20.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:19 smithi018 ceph-mon[40196]: pgmap v884: 168 pgs: 168 active+clean; 5.2 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 1.0 KiB/s rd, 89 MiB/s wr, 1.43k op/s 2024-09-10T19:04:20.238 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:19 smithi103 ceph-mon[33163]: pgmap v884: 168 pgs: 168 active+clean; 5.2 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 1.0 KiB/s rd, 89 MiB/s wr, 1.43k op/s 2024-09-10T19:04:20.283 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:20.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:20.320 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:20.321 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:20.421 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:20.459 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:20.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:20.460 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:20.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:04:20 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:04:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:04:20.561 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:20.601 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:20.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:20.602 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:20.703 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:20.709 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:04:20.717 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:04:20.731 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:20.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:20.732 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:20.833 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:20.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:20.864 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:20.864 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:20.965 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:20.991 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:20.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:20.993 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:21.043 INFO:teuthology.orchestra.run.smithi018.stdout:ERROR: (22) Invalid argument 2024-09-10T19:04:21.043 INFO:teuthology.orchestra.run.smithi018.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T19:04:21.074 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 53 16 91062 91046 107.348 91.125 0.0142999 0.00931099 2024-09-10T19:04:21.094 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:21.120 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:21.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:21.121 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:21.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:20 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:04:21.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:20 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:04:21.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:20 smithi018 ceph-mon[29336]: osdmap e400: 8 total, 7 up, 6 in 2024-09-10T19:04:21.195 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:04:20 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:04:20] "GET /metrics HTTP/1.1" 200 209632 "" "Prometheus/2.43.0" 2024-09-10T19:04:21.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:20 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:04:21.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:20 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:04:21.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:20 smithi018 ceph-mon[40196]: osdmap e400: 8 total, 7 up, 6 in 2024-09-10T19:04:21.223 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:21.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:20 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:04:21.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:20 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:04:21.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:20 smithi103 ceph-mon[33163]: osdmap e400: 8 total, 7 up, 6 in 2024-09-10T19:04:21.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:21.273 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:21.273 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:21.374 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:21.414 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:21.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:21.416 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:21.516 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:21.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:21.559 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:21.559 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:21.660 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:21.704 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:21.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:21.705 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:21.806 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:21.835 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:21.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:21.836 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:21.854 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:21 smithi018 ceph-mon[29336]: pgmap v886: 168 pgs: 168 active+clean; 5.3 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 639 B/s rd, 83 MiB/s wr, 1.33k op/s 2024-09-10T19:04:21.854 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:21 smithi018 ceph-mon[40196]: pgmap v886: 168 pgs: 168 active+clean; 5.3 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 639 B/s rd, 83 MiB/s wr, 1.33k op/s 2024-09-10T19:04:21.937 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:21.963 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:21.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:21.965 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:22.065 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:22.074 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 54 16 92566 92550 107.1 94 0.0164282 0.00933199 2024-09-10T19:04:22.095 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:22.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:22.096 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:22.197 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:22.227 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:22.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:22.228 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:22.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:21 smithi103 ceph-mon[33163]: pgmap v886: 168 pgs: 168 active+clean; 5.3 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 639 B/s rd, 83 MiB/s wr, 1.33k op/s 2024-09-10T19:04:22.328 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:22.361 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-10T19:04:22.378 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:22.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:22.381 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:22.482 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:22.547 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:22.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:22.549 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:22.649 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:22.677 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:22.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:22.678 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:22.779 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:22.835 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:22.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:22.836 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:22.925 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:22 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:04:22.925 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:22 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:04:22.925 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:22 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:04:22.926 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:22 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:04:22.937 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:22.988 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:22.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:22.990 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:23.075 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 55 15 94038 94023 106.827 92.0625 0.00526673 0.00935654 2024-09-10T19:04:23.091 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:23.132 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:23.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:23.133 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:23.234 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:23.266 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:23.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:23.267 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:23.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:22 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:04:23.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:22 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:04:23.369 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:23.401 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:23.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:23.402 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:23.503 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:23.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:23.547 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:23.547 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:23.648 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:23.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:23.701 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:23.702 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:23.802 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:23.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:23.842 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:23.842 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:23.925 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:23 smithi018 ceph-mon[29336]: osdmap e401: 8 total, 7 up, 6 in 2024-09-10T19:04:23.925 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:23 smithi018 ceph-mon[29336]: pgmap v888: 168 pgs: 168 active+clean; 5.6 GiB data, 22 GiB used, 514 GiB / 536 GiB avail; 1.2 KiB/s rd, 103 MiB/s wr, 1.65k op/s 2024-09-10T19:04:23.925 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:23 smithi018 ceph-mon[40196]: osdmap e401: 8 total, 7 up, 6 in 2024-09-10T19:04:23.925 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:23 smithi018 ceph-mon[40196]: pgmap v888: 168 pgs: 168 active+clean; 5.6 GiB data, 22 GiB used, 514 GiB / 536 GiB avail; 1.2 KiB/s rd, 103 MiB/s wr, 1.65k op/s 2024-09-10T19:04:23.944 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:23.971 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:23.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:23.972 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:24.072 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:24.076 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 56 16 95525 95509 106.578 92.875 0.0081121 0.00937818 2024-09-10T19:04:24.117 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:24.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:24.118 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:24.219 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:24.244 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:24.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:24.245 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:24.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:23 smithi103 ceph-mon[33163]: osdmap e401: 8 total, 7 up, 6 in 2024-09-10T19:04:24.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:23 smithi103 ceph-mon[33163]: pgmap v888: 168 pgs: 168 active+clean; 5.6 GiB data, 22 GiB used, 514 GiB / 536 GiB avail; 1.2 KiB/s rd, 103 MiB/s wr, 1.65k op/s 2024-09-10T19:04:24.346 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:24.372 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:24.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:24.373 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:24.430 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T19:04:24.430 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T19:04:24.437 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T19:04:24.438 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T19:04:24.471 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T19:04:24.472 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T19:04:24.474 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:24.510 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T19:04:24.510 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T19:04:24.517 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:24.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:24.518 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:24.551 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T19:04:24.552 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T19:04:24.591 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T19:04:24.592 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T19:04:24.619 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:24.630 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T19:04:24.631 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T19:04:24.631 INFO:teuthology.orchestra.run.smithi018.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-10T19:04:24.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:24.666 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:24.666 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:24.767 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:24.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:24.815 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:24.815 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:24.873 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is unset 2024-09-10T19:04:24.916 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:24.946 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:24.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:24.947 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:25.048 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:25.072 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:24 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1532598052' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:04:25.072 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:24 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/365381352' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:04:25.072 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:24 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:04:25.073 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:24 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1532598052' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:04:25.073 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:24 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/365381352' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:04:25.073 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:24 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:04:25.075 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 57 16 96867 96851 106.179 83.875 0.00754816 0.00941346 2024-09-10T19:04:25.090 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:25.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:25.092 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:25.192 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:25.223 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:25.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:25.224 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:25.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:24 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1532598052' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:04:25.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:24 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/365381352' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:04:25.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:24 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:04:25.325 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:25.364 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:25.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:25.366 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:25.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:04:25 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T19:04:25.098+0000 7f66d46e1640 -1 mgr get_metadata_python Requested missing service osd.7 2024-09-10T19:04:25.467 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:25.509 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:25.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:25.510 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:25.611 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:25.645 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:25.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:25.646 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:25.748 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:25.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:25.789 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:25.790 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:25.891 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:25.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:25.927 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:25.927 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:26.028 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:26.057 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:26.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:26.059 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:26.075 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 58 16 98232 98216 105.819 85.3125 0.00834008 0.0094458 2024-09-10T19:04:26.160 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:26.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:25 smithi018 ceph-mon[40196]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:04:26.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:25 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T19:04:26.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:25 smithi018 ceph-mon[40196]: osdmap e402: 8 total, 7 up, 6 in 2024-09-10T19:04:26.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:25 smithi018 ceph-mon[40196]: 1.1 scrub starts 2024-09-10T19:04:26.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:25 smithi018 ceph-mon[40196]: 1.1 scrub ok 2024-09-10T19:04:26.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:25 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:04:26.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:25 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:04:26.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:25 smithi018 ceph-mon[40196]: pgmap v890: 168 pgs: 168 active+clean; 5.6 GiB data, 23 GiB used, 514 GiB / 536 GiB avail; 853 B/s rd, 73 MiB/s wr, 1.17k op/s 2024-09-10T19:04:26.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:25 smithi018 ceph-mon[29336]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:04:26.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:25 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T19:04:26.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:25 smithi018 ceph-mon[29336]: osdmap e402: 8 total, 7 up, 6 in 2024-09-10T19:04:26.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:25 smithi018 ceph-mon[29336]: 1.1 scrub starts 2024-09-10T19:04:26.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:25 smithi018 ceph-mon[29336]: 1.1 scrub ok 2024-09-10T19:04:26.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:25 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:04:26.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:25 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:04:26.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:25 smithi018 ceph-mon[29336]: pgmap v890: 168 pgs: 168 active+clean; 5.6 GiB data, 23 GiB used, 514 GiB / 536 GiB avail; 853 B/s rd, 73 MiB/s wr, 1.17k op/s 2024-09-10T19:04:26.201 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:26.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:26.203 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:26.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:25 smithi103 ceph-mon[33163]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:04:26.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:25 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T19:04:26.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:25 smithi103 ceph-mon[33163]: osdmap e402: 8 total, 7 up, 6 in 2024-09-10T19:04:26.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:25 smithi103 ceph-mon[33163]: 1.1 scrub starts 2024-09-10T19:04:26.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:25 smithi103 ceph-mon[33163]: 1.1 scrub ok 2024-09-10T19:04:26.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:25 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:04:26.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:25 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:04:26.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:25 smithi103 ceph-mon[33163]: pgmap v890: 168 pgs: 168 active+clean; 5.6 GiB data, 23 GiB used, 514 GiB / 536 GiB avail; 853 B/s rd, 73 MiB/s wr, 1.17k op/s 2024-09-10T19:04:26.303 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:26.336 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:26.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:26.337 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:26.438 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:26.467 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:26.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:26.468 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:26.569 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:26.615 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:26.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:26.616 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:26.717 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:26.751 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:26.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:26.752 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:26.853 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:26.894 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:26.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:26.895 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:26.996 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:27.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:27.024 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:27.025 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:27.075 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 59 16 99667 99651 105.546 89.6875 0.00794451 0.00946992 2024-09-10T19:04:27.125 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:27.132 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:26 smithi018 ceph-mon[29336]: 4.1b scrub starts 2024-09-10T19:04:27.132 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:26 smithi018 ceph-mon[29336]: 4.1b scrub ok 2024-09-10T19:04:27.133 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:26 smithi018 ceph-mon[40196]: 4.1b scrub starts 2024-09-10T19:04:27.133 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:26 smithi018 ceph-mon[40196]: 4.1b scrub ok 2024-09-10T19:04:27.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:27.168 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:27.168 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:27.190 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:26 smithi103 ceph-mon[33163]: 4.1b scrub starts 2024-09-10T19:04:27.190 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:26 smithi103 ceph-mon[33163]: 4.1b scrub ok 2024-09-10T19:04:27.269 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:27.290 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:27.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:27.291 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:27.392 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:27.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:27.422 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:27.422 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:27.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:04:27 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:04:27.524 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:27.551 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:27.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:27.553 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:27.654 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:27.694 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:27.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:27.695 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:27.796 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:27.827 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:27.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:27.828 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:27.928 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:27.973 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:27.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:27.975 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:28.075 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:2024-09-10T19:04:28.074906+0000 min lat: 0.00116366 max lat: 0.294449 avg lat: 0.00950416 2024-09-10T19:04:28.075 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T19:04:28.075 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 60 16 100992 100976 105.167 82.8125 0.0166183 0.00950416 2024-09-10T19:04:28.076 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:28.100 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:28.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:28.101 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:28.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:27 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:28.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:27 smithi018 ceph-mon[29336]: pgmap v891: 168 pgs: 168 active+clean; 6.0 GiB data, 24 GiB used, 513 GiB / 536 GiB avail; 1.3 KiB/s rd, 108 MiB/s wr, 1.72k op/s 2024-09-10T19:04:28.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:27 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:28.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:27 smithi018 ceph-mon[40196]: pgmap v891: 168 pgs: 168 active+clean; 6.0 GiB data, 24 GiB used, 513 GiB / 536 GiB avail; 1.3 KiB/s rd, 108 MiB/s wr, 1.72k op/s 2024-09-10T19:04:28.204 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:28.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:28.243 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:28.243 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:28.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:27 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:28.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:27 smithi103 ceph-mon[33163]: pgmap v891: 168 pgs: 168 active+clean; 6.0 GiB data, 24 GiB used, 513 GiB / 536 GiB avail; 1.3 KiB/s rd, 108 MiB/s wr, 1.72k op/s 2024-09-10T19:04:28.344 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:28.374 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:28.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:28.375 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:28.476 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:28.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:28.518 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:28.518 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:28.619 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:28.657 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:28.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:28.658 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:28.759 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:28.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:28.801 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:28.801 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:28.902 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:28.940 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:28.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:28.942 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:29.042 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:29.075 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 61 16 102313 102297 104.796 82.5625 0.00881603 0.0095378 2024-09-10T19:04:29.082 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:29.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:29.083 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:29.184 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:29.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:28 smithi018 ceph-mon[29336]: 4.18 scrub starts 2024-09-10T19:04:29.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:28 smithi018 ceph-mon[29336]: 4.18 scrub ok 2024-09-10T19:04:29.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:28 smithi018 ceph-mon[40196]: 4.18 scrub starts 2024-09-10T19:04:29.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:28 smithi018 ceph-mon[40196]: 4.18 scrub ok 2024-09-10T19:04:29.215 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:29.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:29.216 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:29.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:28 smithi103 ceph-mon[33163]: 4.18 scrub starts 2024-09-10T19:04:29.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:28 smithi103 ceph-mon[33163]: 4.18 scrub ok 2024-09-10T19:04:29.317 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:29.338 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:29.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:29.339 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:29.440 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:29.481 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:29.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:29.482 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:29.583 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:29.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:29.622 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:29.622 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:29.723 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:29.762 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:29.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:29.763 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:29.864 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:29.894 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:29.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:29.895 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:29.996 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:30.022 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:30.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:30.023 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:30.075 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 62 16 103493 103477 104.295 73.75 0.010198 0.00958386 2024-09-10T19:04:30.124 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:30.161 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:30.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:30.162 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:30.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:29 smithi018 ceph-mon[29336]: 1.3 scrub starts 2024-09-10T19:04:30.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:29 smithi018 ceph-mon[29336]: 1.3 scrub ok 2024-09-10T19:04:30.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:29 smithi018 ceph-mon[29336]: 6.9 scrub starts 2024-09-10T19:04:30.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:29 smithi018 ceph-mon[29336]: 6.9 scrub ok 2024-09-10T19:04:30.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:29 smithi018 ceph-mon[29336]: 2.1 scrub starts 2024-09-10T19:04:30.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:29 smithi018 ceph-mon[29336]: pgmap v892: 168 pgs: 168 active+clean; 6.0 GiB data, 24 GiB used, 513 GiB / 536 GiB avail; 1.2 KiB/s rd, 89 MiB/s wr, 1.43k op/s 2024-09-10T19:04:30.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:29 smithi018 ceph-mon[29336]: 2.1 scrub ok 2024-09-10T19:04:30.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:29 smithi018 ceph-mon[40196]: 1.3 scrub starts 2024-09-10T19:04:30.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:29 smithi018 ceph-mon[40196]: 1.3 scrub ok 2024-09-10T19:04:30.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:29 smithi018 ceph-mon[40196]: 6.9 scrub starts 2024-09-10T19:04:30.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:29 smithi018 ceph-mon[40196]: 6.9 scrub ok 2024-09-10T19:04:30.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:29 smithi018 ceph-mon[40196]: 2.1 scrub starts 2024-09-10T19:04:30.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:29 smithi018 ceph-mon[40196]: pgmap v892: 168 pgs: 168 active+clean; 6.0 GiB data, 24 GiB used, 513 GiB / 536 GiB avail; 1.2 KiB/s rd, 89 MiB/s wr, 1.43k op/s 2024-09-10T19:04:30.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:29 smithi018 ceph-mon[40196]: 2.1 scrub ok 2024-09-10T19:04:30.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:29 smithi103 ceph-mon[33163]: 1.3 scrub starts 2024-09-10T19:04:30.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:29 smithi103 ceph-mon[33163]: 1.3 scrub ok 2024-09-10T19:04:30.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:29 smithi103 ceph-mon[33163]: 6.9 scrub starts 2024-09-10T19:04:30.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:29 smithi103 ceph-mon[33163]: 6.9 scrub ok 2024-09-10T19:04:30.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:29 smithi103 ceph-mon[33163]: 2.1 scrub starts 2024-09-10T19:04:30.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:29 smithi103 ceph-mon[33163]: pgmap v892: 168 pgs: 168 active+clean; 6.0 GiB data, 24 GiB used, 513 GiB / 536 GiB avail; 1.2 KiB/s rd, 89 MiB/s wr, 1.43k op/s 2024-09-10T19:04:30.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:29 smithi103 ceph-mon[33163]: 2.1 scrub ok 2024-09-10T19:04:30.263 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:30.292 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:30.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:30.293 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:30.394 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:30.426 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:30.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:30.427 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:30.523 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:04:30 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:04:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:04:30.528 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:30.562 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:30.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:30.563 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:30.664 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:30.702 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:30.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:30.703 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:30.804 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:30.835 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:30.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:30.836 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:30.937 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:30.979 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-10T19:04:30.980 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-10T19:04:30.990 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:30.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:30.991 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:31.075 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 63 16 104923 104907 104.058 89.375 0.00436686 0.00960549 2024-09-10T19:04:31.093 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:31.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:31.133 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:31.133 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:31.158 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:30 smithi018 ceph-mon[29336]: 2.7 scrub starts 2024-09-10T19:04:31.159 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:30 smithi018 ceph-mon[29336]: 2.7 scrub ok 2024-09-10T19:04:31.159 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:30 smithi018 ceph-mon[29336]: 5.7 scrub starts 2024-09-10T19:04:31.159 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:30 smithi018 ceph-mon[29336]: 5.7 scrub ok 2024-09-10T19:04:31.159 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:30 smithi018 ceph-mon[40196]: 2.7 scrub starts 2024-09-10T19:04:31.159 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:30 smithi018 ceph-mon[40196]: 2.7 scrub ok 2024-09-10T19:04:31.159 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:30 smithi018 ceph-mon[40196]: 5.7 scrub starts 2024-09-10T19:04:31.159 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:30 smithi018 ceph-mon[40196]: 5.7 scrub ok 2024-09-10T19:04:31.159 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:04:30 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:04:30] "GET /metrics HTTP/1.1" 200 209629 "" "Prometheus/2.43.0" 2024-09-10T19:04:31.234 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:31.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:30 smithi103 ceph-mon[33163]: 2.7 scrub starts 2024-09-10T19:04:31.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:30 smithi103 ceph-mon[33163]: 2.7 scrub ok 2024-09-10T19:04:31.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:30 smithi103 ceph-mon[33163]: 5.7 scrub starts 2024-09-10T19:04:31.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:30 smithi103 ceph-mon[33163]: 5.7 scrub ok 2024-09-10T19:04:31.273 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:31.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:31.274 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:31.375 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:31.415 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:31.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:31.416 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:31.517 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:31.555 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:31.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:31.556 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:31.657 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:31.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:31.695 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:31.695 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:31.796 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:31.831 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:31.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:31.832 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:31.933 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:31.975 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:31.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:31.977 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:32.076 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 64 16 106427 106411 103.9 94 0.00921756 0.00962007 2024-09-10T19:04:32.078 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:32.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:32.125 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:32.125 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:32.177 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:31 smithi018 ceph-mon[40196]: 1.e scrub starts 2024-09-10T19:04:32.177 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:31 smithi018 ceph-mon[40196]: 1.e scrub ok 2024-09-10T19:04:32.177 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:31 smithi018 ceph-mon[40196]: pgmap v893: 168 pgs: 168 active+clean; 6.2 GiB data, 24 GiB used, 512 GiB / 536 GiB avail; 593 B/s rd, 76 MiB/s wr, 1.22k op/s 2024-09-10T19:04:32.177 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:31 smithi018 ceph-mon[40196]: 1.9 scrub starts 2024-09-10T19:04:32.177 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:31 smithi018 ceph-mon[40196]: 1.9 scrub ok 2024-09-10T19:04:32.178 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:31 smithi018 ceph-mon[29336]: 1.e scrub starts 2024-09-10T19:04:32.178 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:31 smithi018 ceph-mon[29336]: 1.e scrub ok 2024-09-10T19:04:32.178 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:31 smithi018 ceph-mon[29336]: pgmap v893: 168 pgs: 168 active+clean; 6.2 GiB data, 24 GiB used, 512 GiB / 536 GiB avail; 593 B/s rd, 76 MiB/s wr, 1.22k op/s 2024-09-10T19:04:32.178 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:31 smithi018 ceph-mon[29336]: 1.9 scrub starts 2024-09-10T19:04:32.178 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:31 smithi018 ceph-mon[29336]: 1.9 scrub ok 2024-09-10T19:04:32.226 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:32.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:31 smithi103 ceph-mon[33163]: 1.e scrub starts 2024-09-10T19:04:32.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:31 smithi103 ceph-mon[33163]: 1.e scrub ok 2024-09-10T19:04:32.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:31 smithi103 ceph-mon[33163]: pgmap v893: 168 pgs: 168 active+clean; 6.2 GiB data, 24 GiB used, 512 GiB / 536 GiB avail; 593 B/s rd, 76 MiB/s wr, 1.22k op/s 2024-09-10T19:04:32.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:31 smithi103 ceph-mon[33163]: 1.9 scrub starts 2024-09-10T19:04:32.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:31 smithi103 ceph-mon[33163]: 1.9 scrub ok 2024-09-10T19:04:32.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:32.274 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:32.275 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:32.376 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:32.406 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:32.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:32.407 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:32.508 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:32.535 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:32.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:32.536 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:32.637 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:32.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:32.702 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:32.702 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:32.715 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:04:32.803 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:32.854 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:32.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:32.855 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:32.956 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:32.981 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T19:04:32.985 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:32.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:32.986 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:33.077 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 65 16 107832 107816 103.651 87.8125 0.0108244 0.00964259 2024-09-10T19:04:33.087 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:33.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:33.143 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:33.143 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:33.147 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:32 smithi018 ceph-mon[40196]: 4.1e scrub starts 2024-09-10T19:04:33.147 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:32 smithi018 ceph-mon[40196]: 4.1e scrub ok 2024-09-10T19:04:33.147 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:32 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:04:33.147 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:32 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:04:33.148 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:32 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:04:33.148 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:32 smithi018 ceph-mon[40196]: 6.12 scrub starts 2024-09-10T19:04:33.148 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:32 smithi018 ceph-mon[40196]: 6.12 scrub ok 2024-09-10T19:04:33.148 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:32 smithi018 ceph-mon[40196]: 2.9 scrub starts 2024-09-10T19:04:33.148 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:32 smithi018 ceph-mon[40196]: 2.9 scrub ok 2024-09-10T19:04:33.148 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:32 smithi018 ceph-mon[29336]: 4.1e scrub starts 2024-09-10T19:04:33.148 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:32 smithi018 ceph-mon[29336]: 4.1e scrub ok 2024-09-10T19:04:33.148 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:32 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:04:33.148 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:32 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:04:33.148 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:32 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:04:33.149 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:32 smithi018 ceph-mon[29336]: 6.12 scrub starts 2024-09-10T19:04:33.149 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:32 smithi018 ceph-mon[29336]: 6.12 scrub ok 2024-09-10T19:04:33.149 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:32 smithi018 ceph-mon[29336]: 2.9 scrub starts 2024-09-10T19:04:33.149 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:32 smithi018 ceph-mon[29336]: 2.9 scrub ok 2024-09-10T19:04:33.245 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:33.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:32 smithi103 ceph-mon[33163]: 4.1e scrub starts 2024-09-10T19:04:33.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:32 smithi103 ceph-mon[33163]: 4.1e scrub ok 2024-09-10T19:04:33.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:32 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:04:33.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:32 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:04:33.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:32 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:04:33.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:32 smithi103 ceph-mon[33163]: 6.12 scrub starts 2024-09-10T19:04:33.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:32 smithi103 ceph-mon[33163]: 6.12 scrub ok 2024-09-10T19:04:33.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:32 smithi103 ceph-mon[33163]: 2.9 scrub starts 2024-09-10T19:04:33.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:32 smithi103 ceph-mon[33163]: 2.9 scrub ok 2024-09-10T19:04:33.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:33.297 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:33.297 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:33.398 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:33.428 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:33.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:33.429 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:33.530 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:33.557 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:33.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:33.558 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:33.659 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:33.703 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:33.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:33.705 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:33.805 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:33.848 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:33.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:33.850 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:33.950 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:34.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:34.005 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:34.005 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:34.017 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-10T19:04:34.077 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 66 16 109110 109094 103.29 79.875 0.0147232 0.00967675 2024-09-10T19:04:34.107 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:34.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:34.188 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:34.189 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:34.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:33 smithi018 ceph-mon[29336]: 5.5 scrub starts 2024-09-10T19:04:34.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:33 smithi018 ceph-mon[29336]: 5.5 scrub ok 2024-09-10T19:04:34.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:33 smithi018 ceph-mon[29336]: 4.f scrub starts 2024-09-10T19:04:34.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:33 smithi018 ceph-mon[29336]: 4.f scrub ok 2024-09-10T19:04:34.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:33 smithi018 ceph-mon[29336]: 4.d scrub starts 2024-09-10T19:04:34.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:33 smithi018 ceph-mon[29336]: 4.d scrub ok 2024-09-10T19:04:34.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:33 smithi018 ceph-mon[29336]: 4.14 scrub starts 2024-09-10T19:04:34.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:33 smithi018 ceph-mon[29336]: pgmap v894: 168 pgs: 168 active+clean; 6.4 GiB data, 25 GiB used, 512 GiB / 536 GiB avail; 409 B/s rd, 87 MiB/s wr, 1.39k op/s 2024-09-10T19:04:34.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:33 smithi018 ceph-mon[29336]: 4.14 scrub ok 2024-09-10T19:04:34.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:33 smithi018 ceph-mon[40196]: 5.5 scrub starts 2024-09-10T19:04:34.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:33 smithi018 ceph-mon[40196]: 5.5 scrub ok 2024-09-10T19:04:34.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:33 smithi018 ceph-mon[40196]: 4.f scrub starts 2024-09-10T19:04:34.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:33 smithi018 ceph-mon[40196]: 4.f scrub ok 2024-09-10T19:04:34.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:33 smithi018 ceph-mon[40196]: 4.d scrub starts 2024-09-10T19:04:34.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:33 smithi018 ceph-mon[40196]: 4.d scrub ok 2024-09-10T19:04:34.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:33 smithi018 ceph-mon[40196]: 4.14 scrub starts 2024-09-10T19:04:34.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:33 smithi018 ceph-mon[40196]: pgmap v894: 168 pgs: 168 active+clean; 6.4 GiB data, 25 GiB used, 512 GiB / 536 GiB avail; 409 B/s rd, 87 MiB/s wr, 1.39k op/s 2024-09-10T19:04:34.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:33 smithi018 ceph-mon[40196]: 4.14 scrub ok 2024-09-10T19:04:34.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:33 smithi103 ceph-mon[33163]: 5.5 scrub starts 2024-09-10T19:04:34.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:33 smithi103 ceph-mon[33163]: 5.5 scrub ok 2024-09-10T19:04:34.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:33 smithi103 ceph-mon[33163]: 4.f scrub starts 2024-09-10T19:04:34.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:33 smithi103 ceph-mon[33163]: 4.f scrub ok 2024-09-10T19:04:34.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:33 smithi103 ceph-mon[33163]: 4.d scrub starts 2024-09-10T19:04:34.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:33 smithi103 ceph-mon[33163]: 4.d scrub ok 2024-09-10T19:04:34.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:33 smithi103 ceph-mon[33163]: 4.14 scrub starts 2024-09-10T19:04:34.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:33 smithi103 ceph-mon[33163]: pgmap v894: 168 pgs: 168 active+clean; 6.4 GiB data, 25 GiB used, 512 GiB / 536 GiB avail; 409 B/s rd, 87 MiB/s wr, 1.39k op/s 2024-09-10T19:04:34.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:33 smithi103 ceph-mon[33163]: 4.14 scrub ok 2024-09-10T19:04:34.289 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:34.314 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:34.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:34.315 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:34.416 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:34.444 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:34.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:34.445 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:34.545 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:34.585 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:34.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:34.587 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:34.687 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:34.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:34.728 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:34.728 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:34.829 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:34.862 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:04:34.895 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:34.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:34.896 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:34.928 INFO:teuthology.orchestra.run.smithi018.stderr:set pool 11 pg_num to 26 2024-09-10T19:04:34.997 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:35.018 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:35.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:35.019 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:35.077 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 67 16 110226 110210 102.789 69.75 0.0165042 0.00972414 2024-09-10T19:04:35.120 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:35.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:35.167 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:35.167 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:35.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:34 smithi018 ceph-mon[40196]: 4.1f scrub starts 2024-09-10T19:04:35.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:34 smithi018 ceph-mon[40196]: 4.1f scrub ok 2024-09-10T19:04:35.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:34 smithi018 ceph-mon[40196]: 1.11 scrub starts 2024-09-10T19:04:35.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:34 smithi018 ceph-mon[40196]: 1.11 scrub ok 2024-09-10T19:04:35.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:34 smithi018 ceph-mon[40196]: 5.4 scrub starts 2024-09-10T19:04:35.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:34 smithi018 ceph-mon[40196]: 5.4 scrub ok 2024-09-10T19:04:35.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:34 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3161184281' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num", "val": "26"}]: dispatch 2024-09-10T19:04:35.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:34 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num", "val": "26"}]: dispatch 2024-09-10T19:04:35.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:34 smithi018 ceph-mon[40196]: 2.19 scrub starts 2024-09-10T19:04:35.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:34 smithi018 ceph-mon[40196]: 2.19 scrub ok 2024-09-10T19:04:35.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:34 smithi018 ceph-mon[29336]: 4.1f scrub starts 2024-09-10T19:04:35.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:34 smithi018 ceph-mon[29336]: 4.1f scrub ok 2024-09-10T19:04:35.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:34 smithi018 ceph-mon[29336]: 1.11 scrub starts 2024-09-10T19:04:35.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:34 smithi018 ceph-mon[29336]: 1.11 scrub ok 2024-09-10T19:04:35.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:34 smithi018 ceph-mon[29336]: 5.4 scrub starts 2024-09-10T19:04:35.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:34 smithi018 ceph-mon[29336]: 5.4 scrub ok 2024-09-10T19:04:35.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:34 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3161184281' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num", "val": "26"}]: dispatch 2024-09-10T19:04:35.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:34 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num", "val": "26"}]: dispatch 2024-09-10T19:04:35.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:34 smithi018 ceph-mon[29336]: 2.19 scrub starts 2024-09-10T19:04:35.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:34 smithi018 ceph-mon[29336]: 2.19 scrub ok 2024-09-10T19:04:35.267 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:35.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:34 smithi103 ceph-mon[33163]: 4.1f scrub starts 2024-09-10T19:04:35.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:34 smithi103 ceph-mon[33163]: 4.1f scrub ok 2024-09-10T19:04:35.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:34 smithi103 ceph-mon[33163]: 1.11 scrub starts 2024-09-10T19:04:35.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:34 smithi103 ceph-mon[33163]: 1.11 scrub ok 2024-09-10T19:04:35.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:34 smithi103 ceph-mon[33163]: 5.4 scrub starts 2024-09-10T19:04:35.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:34 smithi103 ceph-mon[33163]: 5.4 scrub ok 2024-09-10T19:04:35.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:34 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3161184281' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num", "val": "26"}]: dispatch 2024-09-10T19:04:35.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:34 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num", "val": "26"}]: dispatch 2024-09-10T19:04:35.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:34 smithi103 ceph-mon[33163]: 2.19 scrub starts 2024-09-10T19:04:35.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:34 smithi103 ceph-mon[33163]: 2.19 scrub ok 2024-09-10T19:04:35.295 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:35.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:35.296 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:35.397 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:35.426 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:35.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:35.427 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:35.527 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:35.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:35.573 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:35.573 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:35.674 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:35.696 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:35.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:35.697 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:35.797 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:35.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:35.820 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:35.820 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:35.921 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:35.950 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:35.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:35.951 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:36.052 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:36.077 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 68 16 111179 111163 102.154 59.5625 0.0245315 0.0097652 2024-09-10T19:04:36.096 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:36.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:36.097 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:36.198 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:36.219 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:36.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:36.220 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:36.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:35 smithi103 ceph-mon[33163]: 2.13 scrub starts 2024-09-10T19:04:36.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:35 smithi103 ceph-mon[33163]: 2.13 scrub ok 2024-09-10T19:04:36.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:35 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num", "val": "26"}]': finished 2024-09-10T19:04:36.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:35 smithi103 ceph-mon[33163]: osdmap e403: 8 total, 7 up, 6 in 2024-09-10T19:04:36.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:35 smithi103 ceph-mon[33163]: pgmap v896: 168 pgs: 168 active+clean; 6.5 GiB data, 25 GiB used, 511 GiB / 536 GiB avail; 409 B/s rd, 86 MiB/s wr, 1.38k op/s 2024-09-10T19:04:36.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:35 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-09-10T19:04:36.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:35 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-09-10T19:04:36.321 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:36.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:36.345 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:36.345 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:36.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:35 smithi018 ceph-mon[29336]: 2.13 scrub starts 2024-09-10T19:04:36.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:35 smithi018 ceph-mon[29336]: 2.13 scrub ok 2024-09-10T19:04:36.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:35 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num", "val": "26"}]': finished 2024-09-10T19:04:36.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:35 smithi018 ceph-mon[29336]: osdmap e403: 8 total, 7 up, 6 in 2024-09-10T19:04:36.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:35 smithi018 ceph-mon[29336]: pgmap v896: 168 pgs: 168 active+clean; 6.5 GiB data, 25 GiB used, 511 GiB / 536 GiB avail; 409 B/s rd, 86 MiB/s wr, 1.38k op/s 2024-09-10T19:04:36.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:35 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-09-10T19:04:36.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:35 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-09-10T19:04:36.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:35 smithi018 ceph-mon[40196]: 2.13 scrub starts 2024-09-10T19:04:36.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:35 smithi018 ceph-mon[40196]: 2.13 scrub ok 2024-09-10T19:04:36.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:35 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num", "val": "26"}]': finished 2024-09-10T19:04:36.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:35 smithi018 ceph-mon[40196]: osdmap e403: 8 total, 7 up, 6 in 2024-09-10T19:04:36.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:35 smithi018 ceph-mon[40196]: pgmap v896: 168 pgs: 168 active+clean; 6.5 GiB data, 25 GiB used, 511 GiB / 536 GiB avail; 409 B/s rd, 86 MiB/s wr, 1.38k op/s 2024-09-10T19:04:36.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:35 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-09-10T19:04:36.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:35 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-09-10T19:04:36.447 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:36.469 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:36.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:36.470 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:36.563 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:04:36.571 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:36.594 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:36.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:36.595 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:36.696 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:36.718 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:36.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:36.719 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:36.820 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:36.845 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:36.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:36.846 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:36.947 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:36.983 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:36.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:36.984 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:37.078 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 69 16 111358 111342 100.835 11.1875 0.0053893 0.00991279 2024-09-10T19:04:37.084 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:37.117 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:37.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:37.118 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:37.200 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:36 smithi103 ceph-mon[33163]: 4.16 scrub starts 2024-09-10T19:04:37.200 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:36 smithi103 ceph-mon[33163]: 4.16 scrub ok 2024-09-10T19:04:37.200 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:36 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num_actual", "val": "26"}]': finished 2024-09-10T19:04:37.200 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:36 smithi103 ceph-mon[33163]: osdmap e404: 8 total, 7 up, 6 in 2024-09-10T19:04:37.201 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:36 smithi103 ceph-mon[33163]: 2.3 scrub starts 2024-09-10T19:04:37.201 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:36 smithi103 ceph-mon[33163]: 2.3 scrub ok 2024-09-10T19:04:37.201 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:36 smithi103 ceph-mon[33163]: 5.1 scrub starts 2024-09-10T19:04:37.201 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:36 smithi103 ceph-mon[33163]: 5.1 scrub ok 2024-09-10T19:04:37.201 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:36 smithi103 ceph-mon[33163]: 1.13 scrub starts 2024-09-10T19:04:37.201 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:36 smithi103 ceph-mon[33163]: 1.13 scrub ok 2024-09-10T19:04:37.219 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:37.259 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:36 smithi018 ceph-mon[40196]: 4.16 scrub starts 2024-09-10T19:04:37.260 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:36 smithi018 ceph-mon[40196]: 4.16 scrub ok 2024-09-10T19:04:37.260 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:36 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num_actual", "val": "26"}]': finished 2024-09-10T19:04:37.260 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:36 smithi018 ceph-mon[40196]: osdmap e404: 8 total, 7 up, 6 in 2024-09-10T19:04:37.260 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:36 smithi018 ceph-mon[40196]: 2.3 scrub starts 2024-09-10T19:04:37.260 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:36 smithi018 ceph-mon[40196]: 2.3 scrub ok 2024-09-10T19:04:37.260 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:36 smithi018 ceph-mon[40196]: 5.1 scrub starts 2024-09-10T19:04:37.260 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:36 smithi018 ceph-mon[40196]: 5.1 scrub ok 2024-09-10T19:04:37.260 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:36 smithi018 ceph-mon[40196]: 1.13 scrub starts 2024-09-10T19:04:37.260 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:36 smithi018 ceph-mon[40196]: 1.13 scrub ok 2024-09-10T19:04:37.260 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:36 smithi018 ceph-mon[29336]: 4.16 scrub starts 2024-09-10T19:04:37.261 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:36 smithi018 ceph-mon[29336]: 4.16 scrub ok 2024-09-10T19:04:37.261 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:36 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pg_num_actual", "val": "26"}]': finished 2024-09-10T19:04:37.261 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:36 smithi018 ceph-mon[29336]: osdmap e404: 8 total, 7 up, 6 in 2024-09-10T19:04:37.261 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:36 smithi018 ceph-mon[29336]: 2.3 scrub starts 2024-09-10T19:04:37.261 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:36 smithi018 ceph-mon[29336]: 2.3 scrub ok 2024-09-10T19:04:37.261 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:36 smithi018 ceph-mon[29336]: 5.1 scrub starts 2024-09-10T19:04:37.261 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:36 smithi018 ceph-mon[29336]: 5.1 scrub ok 2024-09-10T19:04:37.261 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:36 smithi018 ceph-mon[29336]: 1.13 scrub starts 2024-09-10T19:04:37.261 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:36 smithi018 ceph-mon[29336]: 1.13 scrub ok 2024-09-10T19:04:37.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:37.265 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:37.265 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:37.367 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:37.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:37.415 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:37.415 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:37.516 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:37.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:04:37 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:04:37.557 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:37.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:37.558 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:37.659 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:37.698 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:37.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:37.699 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:37.800 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:37.844 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:37.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:37.845 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:37.946 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:37.976 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:37.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:37.977 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:38.078 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:38.079 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 70 16 113110 113094 100.959 109.5 0.00190983 0.00988813 2024-09-10T19:04:38.099 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:38.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:38.100 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:38.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:37 smithi018 ceph-mon[29336]: osdmap e405: 8 total, 7 up, 6 in 2024-09-10T19:04:38.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:37 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:38.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:37 smithi018 ceph-mon[29336]: pgmap v899: 178 pgs: 9 peering, 10 unknown, 159 active+clean; 4.9 GiB data, 26 GiB used, 510 GiB / 536 GiB avail; 682 B/s rd, 1 op/s 2024-09-10T19:04:38.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:37 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num_actual", "val": "26"}]: dispatch 2024-09-10T19:04:38.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:37 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num_actual", "val": "26"}]: dispatch 2024-09-10T19:04:38.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:37 smithi018 ceph-mon[40196]: osdmap e405: 8 total, 7 up, 6 in 2024-09-10T19:04:38.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:37 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:38.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:37 smithi018 ceph-mon[40196]: pgmap v899: 178 pgs: 9 peering, 10 unknown, 159 active+clean; 4.9 GiB data, 26 GiB used, 510 GiB / 536 GiB avail; 682 B/s rd, 1 op/s 2024-09-10T19:04:38.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:37 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num_actual", "val": "26"}]: dispatch 2024-09-10T19:04:38.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:37 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num_actual", "val": "26"}]: dispatch 2024-09-10T19:04:38.200 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:38.222 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:38.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:38.224 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:38.260 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T19:04:38.260 INFO:teuthology.orchestra.run.smithi018.stdout: "ops": [], 2024-09-10T19:04:38.260 INFO:teuthology.orchestra.run.smithi018.stdout: "complaint_time": 30, 2024-09-10T19:04:38.260 INFO:teuthology.orchestra.run.smithi018.stdout: "num_blocked_ops": 0 2024-09-10T19:04:38.261 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T19:04:38.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:37 smithi103 ceph-mon[33163]: osdmap e405: 8 total, 7 up, 6 in 2024-09-10T19:04:38.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:37 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:38.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:37 smithi103 ceph-mon[33163]: pgmap v899: 178 pgs: 9 peering, 10 unknown, 159 active+clean; 4.9 GiB data, 26 GiB used, 510 GiB / 536 GiB avail; 682 B/s rd, 1 op/s 2024-09-10T19:04:38.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:37 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num_actual", "val": "26"}]: dispatch 2024-09-10T19:04:38.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:37 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num_actual", "val": "26"}]: dispatch 2024-09-10T19:04:38.325 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:38.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:38.348 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:38.348 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:38.449 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:38.473 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:38.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:38.474 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:38.575 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:38.598 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:38.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:38.600 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:38.700 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:38.722 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:38.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:38.723 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:38.824 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:38.846 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:38.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:38.847 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:38.948 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:38.971 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:38.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:38.972 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:38.998 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-10T19:04:39.073 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:39.078 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 71 16 113110 113094 99.5372 0 - 0.00988813 2024-09-10T19:04:39.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:39.102 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:39.102 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:39.203 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:39.230 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:39.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:39.231 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:39.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:38 smithi103 ceph-mon[33163]: 6.3 scrub starts 2024-09-10T19:04:39.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:38 smithi103 ceph-mon[33163]: 6.3 scrub ok 2024-09-10T19:04:39.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:38 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num_actual", "val": "26"}]': finished 2024-09-10T19:04:39.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:38 smithi103 ceph-mon[33163]: osdmap e406: 8 total, 7 up, 6 in 2024-09-10T19:04:39.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:38 smithi103 ceph-mon[33163]: 2.b scrub starts 2024-09-10T19:04:39.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:38 smithi103 ceph-mon[33163]: 2.b scrub ok 2024-09-10T19:04:39.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:38 smithi103 ceph-mon[33163]: 4.1c scrub starts 2024-09-10T19:04:39.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:38 smithi103 ceph-mon[33163]: 4.1c scrub ok 2024-09-10T19:04:39.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:38 smithi103 ceph-mon[33163]: osdmap e407: 8 total, 7 up, 6 in 2024-09-10T19:04:39.300 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:38 smithi018 ceph-mon[40196]: 6.3 scrub starts 2024-09-10T19:04:39.300 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:38 smithi018 ceph-mon[40196]: 6.3 scrub ok 2024-09-10T19:04:39.300 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:38 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num_actual", "val": "26"}]': finished 2024-09-10T19:04:39.301 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:38 smithi018 ceph-mon[40196]: osdmap e406: 8 total, 7 up, 6 in 2024-09-10T19:04:39.301 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:38 smithi018 ceph-mon[40196]: 2.b scrub starts 2024-09-10T19:04:39.301 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:38 smithi018 ceph-mon[40196]: 2.b scrub ok 2024-09-10T19:04:39.301 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:38 smithi018 ceph-mon[40196]: 4.1c scrub starts 2024-09-10T19:04:39.301 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:38 smithi018 ceph-mon[40196]: 4.1c scrub ok 2024-09-10T19:04:39.301 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:38 smithi018 ceph-mon[40196]: osdmap e407: 8 total, 7 up, 6 in 2024-09-10T19:04:39.301 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:38 smithi018 ceph-mon[29336]: 6.3 scrub starts 2024-09-10T19:04:39.301 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:38 smithi018 ceph-mon[29336]: 6.3 scrub ok 2024-09-10T19:04:39.301 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:38 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num_actual", "val": "26"}]': finished 2024-09-10T19:04:39.301 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:38 smithi018 ceph-mon[29336]: osdmap e406: 8 total, 7 up, 6 in 2024-09-10T19:04:39.301 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:38 smithi018 ceph-mon[29336]: 2.b scrub starts 2024-09-10T19:04:39.302 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:38 smithi018 ceph-mon[29336]: 2.b scrub ok 2024-09-10T19:04:39.302 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:38 smithi018 ceph-mon[29336]: 4.1c scrub starts 2024-09-10T19:04:39.302 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:38 smithi018 ceph-mon[29336]: 4.1c scrub ok 2024-09-10T19:04:39.302 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:38 smithi018 ceph-mon[29336]: osdmap e407: 8 total, 7 up, 6 in 2024-09-10T19:04:39.331 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:39.353 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:39.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:39.354 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:39.455 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:39.477 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:39.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:39.478 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:39.579 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:39.605 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:39.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:39.606 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:39.707 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:39.729 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:39.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:39.731 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:39.831 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:39.861 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:39.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:39.862 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:39.963 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:40.003 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:40.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:40.004 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:40.078 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 72 15 113451 113436 98.4517 10.6875 0.0110495 0.0101525 2024-09-10T19:04:40.106 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:40.137 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:40.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:40.138 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:40.238 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:40.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:39 smithi103 ceph-mon[33163]: 5.19 scrub starts 2024-09-10T19:04:40.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:39 smithi103 ceph-mon[33163]: 5.19 scrub ok 2024-09-10T19:04:40.240 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:39 smithi103 ceph-mon[33163]: pgmap v902: 178 pgs: 9 peering, 3 remapped+peering, 7 unknown, 159 active+clean; 5.3 GiB data, 26 GiB used, 510 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:40.240 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:39 smithi103 ceph-mon[33163]: 5.18 scrub starts 2024-09-10T19:04:40.240 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:39 smithi103 ceph-mon[33163]: 5.18 scrub ok 2024-09-10T19:04:40.240 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:39 smithi103 ceph-mon[33163]: osdmap e408: 8 total, 7 up, 6 in 2024-09-10T19:04:40.283 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:40.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:40.284 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:40.385 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:40.417 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:40.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:40.418 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:40.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:39 smithi018 ceph-mon[40196]: 5.19 scrub starts 2024-09-10T19:04:40.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:39 smithi018 ceph-mon[40196]: 5.19 scrub ok 2024-09-10T19:04:40.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:39 smithi018 ceph-mon[40196]: pgmap v902: 178 pgs: 9 peering, 3 remapped+peering, 7 unknown, 159 active+clean; 5.3 GiB data, 26 GiB used, 510 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:40.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:39 smithi018 ceph-mon[40196]: 5.18 scrub starts 2024-09-10T19:04:40.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:39 smithi018 ceph-mon[40196]: 5.18 scrub ok 2024-09-10T19:04:40.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:39 smithi018 ceph-mon[40196]: osdmap e408: 8 total, 7 up, 6 in 2024-09-10T19:04:40.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:39 smithi018 ceph-mon[29336]: 5.19 scrub starts 2024-09-10T19:04:40.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:39 smithi018 ceph-mon[29336]: 5.19 scrub ok 2024-09-10T19:04:40.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:39 smithi018 ceph-mon[29336]: pgmap v902: 178 pgs: 9 peering, 3 remapped+peering, 7 unknown, 159 active+clean; 5.3 GiB data, 26 GiB used, 510 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:40.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:39 smithi018 ceph-mon[29336]: 5.18 scrub starts 2024-09-10T19:04:40.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:39 smithi018 ceph-mon[29336]: 5.18 scrub ok 2024-09-10T19:04:40.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:39 smithi018 ceph-mon[29336]: osdmap e408: 8 total, 7 up, 6 in 2024-09-10T19:04:40.446 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:04:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T19:04:40.112+0000 7f66d46e1640 -1 mgr get_metadata_python Requested missing service osd.7 2024-09-10T19:04:40.519 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:40.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:04:40 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:04:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:04:40.553 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:40.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:40.554 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:40.655 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:40.717 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:40.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:40.718 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:40.818 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:40.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:40.863 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:40.863 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:40.964 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:41.005 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:41.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:41.006 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:41.078 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 73 16 115015 114999 98.4411 97.6875 0.00591429 0.0101541 2024-09-10T19:04:41.107 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:41.155 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:41.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:41.156 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:41.257 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:41.269 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:40 smithi018 ceph-mon[29336]: 6.0 scrub starts 2024-09-10T19:04:41.270 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:40 smithi018 ceph-mon[29336]: 6.0 scrub ok 2024-09-10T19:04:41.270 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:40 smithi018 ceph-mon[29336]: 11.19 starting backfill to osd.1 from (0'0,0'0] MAX to 405'6872 2024-09-10T19:04:41.270 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:40 smithi018 ceph-mon[29336]: 11.16 starting backfill to osd.3 from (0'0,0'0] MAX to 405'7011 2024-09-10T19:04:41.270 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:40 smithi018 ceph-mon[29336]: 11.12 starting backfill to osd.5 from (0'0,0'0] MAX to 405'7160 2024-09-10T19:04:41.270 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:40 smithi018 ceph-mon[29336]: 11.18 starting backfill to osd.4 from (0'0,0'0] MAX to 405'7071 2024-09-10T19:04:41.270 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:40 smithi018 ceph-mon[29336]: 11.15 starting backfill to osd.1 from (0'0,0'0] MAX to 405'6975 2024-09-10T19:04:41.270 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:40 smithi018 ceph-mon[29336]: 11.10 starting backfill to osd.0 from (0'0,0'0] MAX to 405'6870 2024-09-10T19:04:41.270 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:40 smithi018 ceph-mon[29336]: 11.17 starting backfill to osd.6 from (0'0,0'0] MAX to 405'6982 2024-09-10T19:04:41.270 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:40 smithi018 ceph-mon[29336]: 11.14 starting backfill to osd.4 from (0'0,0'0] MAX to 405'7137 2024-09-10T19:04:41.270 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:40 smithi018 ceph-mon[29336]: 11.13 starting backfill to osd.0 from (0'0,0'0] MAX to 405'6934 2024-09-10T19:04:41.270 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:40 smithi018 ceph-mon[29336]: 11.11 starting backfill to osd.5 from (0'0,0'0] MAX to 405'6870 2024-09-10T19:04:41.270 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:40 smithi018 ceph-mon[29336]: 11.10 starting backfill to osd.5 from (0'0,0'0] MAX to 405'6870 2024-09-10T19:04:41.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:40 smithi018 ceph-mon[29336]: 11.13 starting backfill to osd.4 from (0'0,0'0] MAX to 405'6934 2024-09-10T19:04:41.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:40 smithi018 ceph-mon[29336]: 11.16 starting backfill to osd.5 from (0'0,0'0] MAX to 405'7011 2024-09-10T19:04:41.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:40 smithi018 ceph-mon[29336]: 11.15 starting backfill to osd.4 from (0'0,0'0] MAX to 405'6975 2024-09-10T19:04:41.271 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:40 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:04:41.271 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:40 smithi018 ceph-mon[40196]: 6.0 scrub starts 2024-09-10T19:04:41.271 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:40 smithi018 ceph-mon[40196]: 6.0 scrub ok 2024-09-10T19:04:41.271 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:40 smithi018 ceph-mon[40196]: 11.19 starting backfill to osd.1 from (0'0,0'0] MAX to 405'6872 2024-09-10T19:04:41.271 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:40 smithi018 ceph-mon[40196]: 11.16 starting backfill to osd.3 from (0'0,0'0] MAX to 405'7011 2024-09-10T19:04:41.272 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:40 smithi018 ceph-mon[40196]: 11.12 starting backfill to osd.5 from (0'0,0'0] MAX to 405'7160 2024-09-10T19:04:41.272 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:40 smithi018 ceph-mon[40196]: 11.18 starting backfill to osd.4 from (0'0,0'0] MAX to 405'7071 2024-09-10T19:04:41.272 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:40 smithi018 ceph-mon[40196]: 11.15 starting backfill to osd.1 from (0'0,0'0] MAX to 405'6975 2024-09-10T19:04:41.272 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:40 smithi018 ceph-mon[40196]: 11.10 starting backfill to osd.0 from (0'0,0'0] MAX to 405'6870 2024-09-10T19:04:41.272 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:40 smithi018 ceph-mon[40196]: 11.17 starting backfill to osd.6 from (0'0,0'0] MAX to 405'6982 2024-09-10T19:04:41.272 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:40 smithi018 ceph-mon[40196]: 11.14 starting backfill to osd.4 from (0'0,0'0] MAX to 405'7137 2024-09-10T19:04:41.272 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:40 smithi018 ceph-mon[40196]: 11.13 starting backfill to osd.0 from (0'0,0'0] MAX to 405'6934 2024-09-10T19:04:41.272 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:40 smithi018 ceph-mon[40196]: 11.11 starting backfill to osd.5 from (0'0,0'0] MAX to 405'6870 2024-09-10T19:04:41.272 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:40 smithi018 ceph-mon[40196]: 11.10 starting backfill to osd.5 from (0'0,0'0] MAX to 405'6870 2024-09-10T19:04:41.272 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:40 smithi018 ceph-mon[40196]: 11.13 starting backfill to osd.4 from (0'0,0'0] MAX to 405'6934 2024-09-10T19:04:41.272 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:40 smithi018 ceph-mon[40196]: 11.16 starting backfill to osd.5 from (0'0,0'0] MAX to 405'7011 2024-09-10T19:04:41.272 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:40 smithi018 ceph-mon[40196]: 11.15 starting backfill to osd.4 from (0'0,0'0] MAX to 405'6975 2024-09-10T19:04:41.272 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:40 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:04:41.273 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:04:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:04:40] "GET /metrics HTTP/1.1" 200 209609 "" "Prometheus/2.43.0" 2024-09-10T19:04:41.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:40 smithi103 ceph-mon[33163]: 6.0 scrub starts 2024-09-10T19:04:41.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:40 smithi103 ceph-mon[33163]: 6.0 scrub ok 2024-09-10T19:04:41.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:40 smithi103 ceph-mon[33163]: 11.19 starting backfill to osd.1 from (0'0,0'0] MAX to 405'6872 2024-09-10T19:04:41.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:40 smithi103 ceph-mon[33163]: 11.16 starting backfill to osd.3 from (0'0,0'0] MAX to 405'7011 2024-09-10T19:04:41.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:40 smithi103 ceph-mon[33163]: 11.12 starting backfill to osd.5 from (0'0,0'0] MAX to 405'7160 2024-09-10T19:04:41.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:40 smithi103 ceph-mon[33163]: 11.18 starting backfill to osd.4 from (0'0,0'0] MAX to 405'7071 2024-09-10T19:04:41.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:40 smithi103 ceph-mon[33163]: 11.15 starting backfill to osd.1 from (0'0,0'0] MAX to 405'6975 2024-09-10T19:04:41.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:40 smithi103 ceph-mon[33163]: 11.10 starting backfill to osd.0 from (0'0,0'0] MAX to 405'6870 2024-09-10T19:04:41.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:40 smithi103 ceph-mon[33163]: 11.17 starting backfill to osd.6 from (0'0,0'0] MAX to 405'6982 2024-09-10T19:04:41.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:40 smithi103 ceph-mon[33163]: 11.14 starting backfill to osd.4 from (0'0,0'0] MAX to 405'7137 2024-09-10T19:04:41.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:40 smithi103 ceph-mon[33163]: 11.13 starting backfill to osd.0 from (0'0,0'0] MAX to 405'6934 2024-09-10T19:04:41.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:40 smithi103 ceph-mon[33163]: 11.11 starting backfill to osd.5 from (0'0,0'0] MAX to 405'6870 2024-09-10T19:04:41.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:40 smithi103 ceph-mon[33163]: 11.10 starting backfill to osd.5 from (0'0,0'0] MAX to 405'6870 2024-09-10T19:04:41.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:40 smithi103 ceph-mon[33163]: 11.13 starting backfill to osd.4 from (0'0,0'0] MAX to 405'6934 2024-09-10T19:04:41.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:40 smithi103 ceph-mon[33163]: 11.16 starting backfill to osd.5 from (0'0,0'0] MAX to 405'7011 2024-09-10T19:04:41.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:40 smithi103 ceph-mon[33163]: 11.15 starting backfill to osd.4 from (0'0,0'0] MAX to 405'6975 2024-09-10T19:04:41.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:40 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:04:41.291 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:41.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:41.293 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:41.394 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:41.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:41.437 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:41.438 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:41.538 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:41.576 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:41.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:41.577 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:41.678 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:41.721 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:41.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:41.722 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:41.823 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:41.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:41.868 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:41.868 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:41.969 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:42.001 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:42.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:42.002 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:42.078 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 74 16 116525 116509 98.3861 94.375 0.0134853 0.0101593 2024-09-10T19:04:42.103 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:42.147 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:42.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:42.148 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:42.240 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:42 smithi018 ceph-mon[40196]: pgmap v904: 178 pgs: 1 active+remapped+backfilling, 5 peering, 3 remapped+peering, 4 unknown, 2 active+remapped+backfill_toofull, 163 active+clean; 6.0 GiB data, 26 GiB used, 510 GiB / 536 GiB avail; 268 MiB/s wr, 4.29k op/s; 14092/297720 objects misplaced (4.733%); 3.2 MiB/s, 51 objects/s recovering 2024-09-10T19:04:42.240 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:41 smithi018 ceph-mon[29336]: pgmap v904: 178 pgs: 1 active+remapped+backfilling, 5 peering, 3 remapped+peering, 4 unknown, 2 active+remapped+backfill_toofull, 163 active+clean; 6.0 GiB data, 26 GiB used, 510 GiB / 536 GiB avail; 268 MiB/s wr, 4.29k op/s; 14092/297720 objects misplaced (4.733%); 3.2 MiB/s, 51 objects/s recovering 2024-09-10T19:04:42.250 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:42.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:41 smithi103 ceph-mon[33163]: pgmap v904: 178 pgs: 1 active+remapped+backfilling, 5 peering, 3 remapped+peering, 4 unknown, 2 active+remapped+backfill_toofull, 163 active+clean; 6.0 GiB data, 26 GiB used, 510 GiB / 536 GiB avail; 268 MiB/s wr, 4.29k op/s; 14092/297720 objects misplaced (4.733%); 3.2 MiB/s, 51 objects/s recovering 2024-09-10T19:04:42.293 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:42.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:42.294 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:42.380 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:04:42.395 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:42.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:42.451 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:42.451 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:42.456 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T19:04:42.456 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T19:04:42.494 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T19:04:42.495 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T19:04:42.525 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T19:04:42.525 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T19:04:42.553 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:42.562 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T19:04:42.562 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T19:04:42.585 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:42.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:42.587 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:42.597 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T19:04:42.597 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T19:04:42.629 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T19:04:42.630 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T19:04:42.663 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T19:04:42.664 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T19:04:42.664 INFO:teuthology.orchestra.run.smithi018.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-10T19:04:42.688 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:42.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:42.719 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:42.719 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:42.820 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:42.855 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:42.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:42.856 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:42.957 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:42.977 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6] out_osds: [2, 7] dead_osds: [7] live_osds: [1, 2, 3, 4, 5, 6, 0] 2024-09-10T19:04:42.977 INFO:tasks.thrashosds.thrasher:Reviving osd 7 2024-09-10T19:04:42.977 INFO:tasks.cephadm.osd.7:Restarting osd.7 (starting--it wasn't running)... 2024-09-10T19:04:42.977 DEBUG:teuthology.orchestra.run.smithi103:osd.7> sudo journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@osd.7.service 2024-09-10T19:04:42.980 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl start ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@osd.7 2024-09-10T19:04:42.998 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:42.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:42.999 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:43.005 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is unset 2024-09-10T19:04:43.078 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 75 16 117923 117907 98.2392 87.375 0.00593224 0.0101746 2024-09-10T19:04:43.100 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:43.137 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:43.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:43.138 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:43.239 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:43.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:43 smithi103 ceph-mon[33163]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-10T19:04:43.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:43 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1879237413' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:04:43.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:43 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1233305837' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:04:43.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:43 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:04:43.274 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:04:43 smithi103 systemd[1]: Starting Ceph osd.7 for ba327f6c-6fa3-11ef-bcea-c7b262605968... 2024-09-10T19:04:43.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:43.277 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:43.277 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:43.378 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:43.417 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:43.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:43.418 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:43.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:43 smithi018 ceph-mon[29336]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-10T19:04:43.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:43 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1879237413' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:04:43.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:43 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1233305837' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:04:43.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:43 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:04:43.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:43 smithi018 ceph-mon[40196]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-10T19:04:43.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:43 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1879237413' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:04:43.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:43 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1233305837' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:04:43.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:43 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:04:43.519 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:43.560 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:43.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:43.561 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:43.662 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:43.686 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:04:43 smithi103 podman[140525]: 2024-09-10 19:04:43.293694093 +0000 UTC m=+0.030662178 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:04:43.686 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:04:43 smithi103 podman[140525]: 2024-09-10 19:04:43.423993893 +0000 UTC m=+0.160961969 container create eab625b561036adf763e52c8610d2ca8081db7d35e64682738867da949bb7f3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7-activate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-10T19:04:43.707 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:43.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:43.708 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:43.809 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:43.849 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:43.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:43.851 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:43.937 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-10T19:04:43.937 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-10T19:04:43.951 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:43.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:43.992 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:43.992 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:44.017 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:04:43 smithi103 podman[140525]: 2024-09-10 19:04:43.685523714 +0000 UTC m=+0.422491804 container init eab625b561036adf763e52c8610d2ca8081db7d35e64682738867da949bb7f3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7-activate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-10T19:04:44.017 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:04:43 smithi103 podman[140525]: 2024-09-10 19:04:43.694593609 +0000 UTC m=+0.431561680 container start eab625b561036adf763e52c8610d2ca8081db7d35e64682738867da949bb7f3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7-activate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-10T19:04:44.017 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:04:43 smithi103 podman[140525]: 2024-09-10 19:04:43.811037802 +0000 UTC m=+0.548005914 container attach eab625b561036adf763e52c8610d2ca8081db7d35e64682738867da949bb7f3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7-activate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True) 2024-09-10T19:04:44.078 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 76 16 119420 119404 98.1775 93.5625 0.0121638 0.0101815 2024-09-10T19:04:44.093 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:44.133 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:44.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:44.134 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:44.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:44 smithi018 ceph-mon[29336]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T19:04:44.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:44 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T19:04:44.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:44 smithi018 ceph-mon[29336]: osdmap e409: 8 total, 7 up, 6 in 2024-09-10T19:04:44.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:44 smithi018 ceph-mon[29336]: pgmap v906: 178 pgs: 3 active+remapped+backfilling, 3 remapped+peering, 4 active+remapped+backfill_toofull, 168 active+clean; 7.0 GiB data, 27 GiB used, 509 GiB / 536 GiB avail; 929 B/s rd, 401 MiB/s wr, 6.42k op/s; 34664/345810 objects misplaced (10.024%); 11 MiB/s, 179 objects/s recovering 2024-09-10T19:04:44.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:44 smithi018 ceph-mon[40196]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T19:04:44.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:44 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T19:04:44.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:44 smithi018 ceph-mon[40196]: osdmap e409: 8 total, 7 up, 6 in 2024-09-10T19:04:44.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:44 smithi018 ceph-mon[40196]: pgmap v906: 178 pgs: 3 active+remapped+backfilling, 3 remapped+peering, 4 active+remapped+backfill_toofull, 168 active+clean; 7.0 GiB data, 27 GiB used, 509 GiB / 536 GiB avail; 929 B/s rd, 401 MiB/s wr, 6.42k op/s; 34664/345810 objects misplaced (10.024%); 11 MiB/s, 179 objects/s recovering 2024-09-10T19:04:44.235 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:44.271 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:44.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:44.273 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:44.273 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:04:44 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7-activate[140544]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-10T19:04:44.273 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:04:44 smithi103 bash[140525]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-10T19:04:44.273 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:04:44 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7-activate[140544]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-7 --no-mon-config --dev /dev/mapper/vg_nvme-lv_1 2024-09-10T19:04:44.273 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:04:44 smithi103 bash[140525]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-7 --no-mon-config --dev /dev/mapper/vg_nvme-lv_1 2024-09-10T19:04:44.273 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:04:44 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7-activate[140544]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_1 2024-09-10T19:04:44.273 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:04:44 smithi103 bash[140525]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_1 2024-09-10T19:04:44.273 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:04:44 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7-activate[140544]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-0 2024-09-10T19:04:44.273 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:04:44 smithi103 bash[140525]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-0 2024-09-10T19:04:44.273 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:04:44 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7-activate[140544]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-7/block 2024-09-10T19:04:44.273 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:04:44 smithi103 bash[140525]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-7/block 2024-09-10T19:04:44.274 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:04:44 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7-activate[140544]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-10T19:04:44.274 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:04:44 smithi103 bash[140525]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-10T19:04:44.274 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:04:44 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7-activate[140544]: --> ceph-volume raw activate successful for osd ID: 7 2024-09-10T19:04:44.274 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:04:44 smithi103 bash[140525]: --> ceph-volume raw activate successful for osd ID: 7 2024-09-10T19:04:44.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:44 smithi103 ceph-mon[33163]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T19:04:44.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:44 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T19:04:44.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:44 smithi103 ceph-mon[33163]: osdmap e409: 8 total, 7 up, 6 in 2024-09-10T19:04:44.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:44 smithi103 ceph-mon[33163]: pgmap v906: 178 pgs: 3 active+remapped+backfilling, 3 remapped+peering, 4 active+remapped+backfill_toofull, 168 active+clean; 7.0 GiB data, 27 GiB used, 509 GiB / 536 GiB avail; 929 B/s rd, 401 MiB/s wr, 6.42k op/s; 34664/345810 objects misplaced (10.024%); 11 MiB/s, 179 objects/s recovering 2024-09-10T19:04:44.374 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:44.407 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:44.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:44.409 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:44.509 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:44.548 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:44.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:44.549 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:44.650 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:44.681 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:44.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:44.683 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:44.686 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:04:44 smithi103 podman[140733]: 2024-09-10 19:04:44.329770298 +0000 UTC m=+0.033199736 container died eab625b561036adf763e52c8610d2ca8081db7d35e64682738867da949bb7f3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7-activate, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-10T19:04:44.783 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:44.825 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:44.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:44.826 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:44.927 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:44.979 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:44.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:44.981 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:45.078 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 77 16 120976 120960 98.1654 97.25 0.00878836 0.010183 2024-09-10T19:04:45.081 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:45.127 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:45.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:45.129 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:45.229 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:45.268 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:45.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:45.269 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:45.370 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:45.412 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:45.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:45.413 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:45.514 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:45.523 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:04:45 smithi103 podman[140733]: 2024-09-10 19:04:45.209585877 +0000 UTC m=+0.913015294 container remove eab625b561036adf763e52c8610d2ca8081db7d35e64682738867da949bb7f3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7-activate, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True) 2024-09-10T19:04:45.547 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:45.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:45.548 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:45.649 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:45.687 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:45.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:45.688 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:45.789 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:45.830 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:45.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:45.831 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:45.859 INFO:teuthology.orchestra.run.smithi018.stdout:ERROR: (22) Invalid argument 2024-09-10T19:04:45.859 INFO:teuthology.orchestra.run.smithi018.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T19:04:45.933 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:45.939 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T19:04:45.965 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:45.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:45.967 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:46.022 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:04:45 smithi103 podman[140862]: 2024-09-10 19:04:45.424332784 +0000 UTC m=+0.028541749 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:04:46.067 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:46.078 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 78 16 122384 122368 98.035 88 0.00483466 0.0101956 2024-09-10T19:04:46.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:46.125 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:46.125 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:46.226 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:46.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:46.277 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:46.277 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:46.378 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:46.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:46.439 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:46.439 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:46.449 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:04:46 smithi103 podman[140862]: 2024-09-10 19:04:46.178333392 +0000 UTC m=+0.782542351 container create 8df6b53b45aed00fe72a0e2a908a48b0b129e9cf93086b19132228096a417bc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-10T19:04:46.541 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:46.583 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:46.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:46.585 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:46.591 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-10T19:04:46.683 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:46 smithi018 ceph-mon[29336]: pgmap v907: 178 pgs: 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 168 active+clean; 7.1 GiB data, 27 GiB used, 509 GiB / 536 GiB avail; 767 B/s rd, 341 MiB/s wr, 5.46k op/s; 48855/349083 objects misplaced (13.995%); 15 MiB/s, 247 objects/s recovering 2024-09-10T19:04:46.683 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:46 smithi018 ceph-mon[40196]: pgmap v907: 178 pgs: 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 168 active+clean; 7.1 GiB data, 27 GiB used, 509 GiB / 536 GiB avail; 767 B/s rd, 341 MiB/s wr, 5.46k op/s; 48855/349083 objects misplaced (13.995%); 15 MiB/s, 247 objects/s recovering 2024-09-10T19:04:46.685 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:46.734 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-10T19:04:46.739 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:46.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:46.741 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:46.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:46 smithi103 ceph-mon[33163]: pgmap v907: 178 pgs: 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 168 active+clean; 7.1 GiB data, 27 GiB used, 509 GiB / 536 GiB avail; 767 B/s rd, 341 MiB/s wr, 5.46k op/s; 48855/349083 objects misplaced (13.995%); 15 MiB/s, 247 objects/s recovering 2024-09-10T19:04:46.775 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:04:46 smithi103 podman[140862]: 2024-09-10 19:04:46.448543056 +0000 UTC m=+1.052752027 container init 8df6b53b45aed00fe72a0e2a908a48b0b129e9cf93086b19132228096a417bc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-10T19:04:46.775 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:04:46 smithi103 podman[140862]: 2024-09-10 19:04:46.455934779 +0000 UTC m=+1.060143740 container start 8df6b53b45aed00fe72a0e2a908a48b0b129e9cf93086b19132228096a417bc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-10T19:04:46.775 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:04:46 smithi103 bash[140862]: 8df6b53b45aed00fe72a0e2a908a48b0b129e9cf93086b19132228096a417bc2 2024-09-10T19:04:46.775 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:04:46 smithi103 systemd[1]: Started Ceph osd.7 for ba327f6c-6fa3-11ef-bcea-c7b262605968. 2024-09-10T19:04:46.841 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:46.860 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:04:46.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:46.891 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:46.891 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:46.969 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-10T19:04:46.992 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:47.061 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:04:47.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:47.065 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:47.066 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:47.080 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 79 16 123671 123655 97.8109 80.4375 0.00762712 0.0102192 2024-09-10T19:04:47.166 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:47.199 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:04:46 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:04:46.770+0000 7fadc9915440 -1 Falling back to public interface 2024-09-10T19:04:47.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:47.222 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:47.222 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:47.323 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:47.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:47.368 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:47.368 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:47.469 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:47.496 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:04:47 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:04:47.517 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:47.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:47.519 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:47.619 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:47.661 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:47.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:47.662 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:47.763 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:47.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:47.809 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:47.809 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:47.910 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:47.942 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:47.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:47.943 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:48.044 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:48.080 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:2024-09-10T19:04:48.079578+0000 min lat: 0.00116366 max lat: 1.9215 avg lat: 0.010233 2024-09-10T19:04:48.080 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T19:04:48.080 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 80 16 125067 125051 97.6788 87.25 0.0111685 0.010233 2024-09-10T19:04:48.091 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:48.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:48.092 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:48.192 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:48.235 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:48.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:48.237 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:48.337 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:48.349 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:04:48.367 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:48.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:48.368 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:48.386 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:04:48.469 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:48.516 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:48.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:48.517 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:48.618 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:48.654 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:48.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:48.655 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:48.755 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:48.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:48 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:48.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:48 smithi103 ceph-mon[33163]: pgmap v908: 178 pgs: 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 168 active+clean; 7.5 GiB data, 28 GiB used, 508 GiB / 536 GiB avail; 1.2 KiB/s rd, 271 MiB/s wr, 4.34k op/s; 47575/366855 objects misplaced (12.968%); 34 MiB/s, 549 objects/s recovering 2024-09-10T19:04:48.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:48.800 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:48.801 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:48.807 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:48 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:48.807 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:48 smithi018 ceph-mon[29336]: pgmap v908: 178 pgs: 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 168 active+clean; 7.5 GiB data, 28 GiB used, 508 GiB / 536 GiB avail; 1.2 KiB/s rd, 271 MiB/s wr, 4.34k op/s; 47575/366855 objects misplaced (12.968%); 34 MiB/s, 549 objects/s recovering 2024-09-10T19:04:48.807 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:48 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:48.807 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:48 smithi018 ceph-mon[40196]: pgmap v908: 178 pgs: 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 168 active+clean; 7.5 GiB data, 28 GiB used, 508 GiB / 536 GiB avail; 1.2 KiB/s rd, 271 MiB/s wr, 4.34k op/s; 47575/366855 objects misplaced (12.968%); 34 MiB/s, 549 objects/s recovering 2024-09-10T19:04:48.902 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:48.942 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:48.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:48.943 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:49.044 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:49.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:49.081 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:49.081 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:49.082 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 81 16 126360 126344 97.4705 80.8125 0.00810259 0.0102548 2024-09-10T19:04:49.114 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-10T19:04:49.115 INFO:teuthology.orchestra.run.smithi103.stderr:0 2024-09-10T19:04:49.115 INFO:teuthology.orchestra.run.smithi103.stderr:1 2024-09-10T19:04:49.115 INFO:teuthology.orchestra.run.smithi103.stderr:2 2024-09-10T19:04:49.115 INFO:teuthology.orchestra.run.smithi103.stderr:abort 2024-09-10T19:04:49.115 INFO:teuthology.orchestra.run.smithi103.stderr:assert 2024-09-10T19:04:49.115 INFO:teuthology.orchestra.run.smithi103.stderr:bluestore allocator dump block 2024-09-10T19:04:49.115 INFO:teuthology.orchestra.run.smithi103.stderr:bluestore allocator fragmentation block 2024-09-10T19:04:49.115 INFO:teuthology.orchestra.run.smithi103.stderr:bluestore allocator score block 2024-09-10T19:04:49.115 INFO:teuthology.orchestra.run.smithi103.stderr:config diff 2024-09-10T19:04:49.115 INFO:teuthology.orchestra.run.smithi103.stderr:config diff get 2024-09-10T19:04:49.116 INFO:teuthology.orchestra.run.smithi103.stderr:admin_socket: invalid command 2024-09-10T19:04:49.155 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T19:04:49.155 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T19:04:49.183 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:49.194 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T19:04:49.195 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T19:04:49.218 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:49.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:49.219 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:49.229 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T19:04:49.229 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T19:04:49.262 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T19:04:49.262 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T19:04:49.297 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T19:04:49.298 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T19:04:49.320 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:49.336 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T19:04:49.337 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T19:04:49.375 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:49.376 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T19:04:49.377 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T19:04:49.377 INFO:teuthology.orchestra.run.smithi018.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-10T19:04:49.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:49.378 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:49.479 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:49.505 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:04:49 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:04:49.242+0000 7fadc9915440 -1 osd.7 361 log_to_monitors true 2024-09-10T19:04:49.526 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:49.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:49.527 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:49.615 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:49 smithi018 ceph-mon[29336]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 6 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-10T19:04:49.615 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:49 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1359476326' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:04:49.615 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:49 smithi018 ceph-mon[29336]: from='osd.7 v2:172.21.15.103:6812/3644107622' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T19:04:49.615 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:49 smithi018 ceph-mon[29336]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T19:04:49.616 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:49 smithi018 ceph-mon[40196]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 6 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-10T19:04:49.616 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:49 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1359476326' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:04:49.616 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:49 smithi018 ceph-mon[40196]: from='osd.7 v2:172.21.15.103:6812/3644107622' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T19:04:49.616 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:49 smithi018 ceph-mon[40196]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T19:04:49.628 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:49.663 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:49.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:49.664 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:49.765 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:49.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:49 smithi103 ceph-mon[33163]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 6 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-10T19:04:49.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:49 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1359476326' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:04:49.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:49 smithi103 ceph-mon[33163]: from='osd.7 v2:172.21.15.103:6812/3644107622' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T19:04:49.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:49 smithi103 ceph-mon[33163]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T19:04:49.799 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:49.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:49.801 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:49.901 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:49.940 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:49.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:49.942 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:49.966 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-10T19:04:49.966 INFO:tasks.cephadm.ceph_manager.ceph:waiting on admin_socket for osd-7, ['dump_ops_in_flight'] 2024-09-10T19:04:50.043 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:50.080 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 82 16 127712 127696 97.3122 84.5 0.00986263 0.0102718 2024-09-10T19:04:50.091 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:50.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:50.091 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:50.192 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:50.230 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:50.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:50.231 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:50.238 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:04:49 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:04:49.850+0000 7fadbfebe640 -1 osd.7 410 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T19:04:50.332 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:50.363 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:50.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:50.364 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:50.465 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:50.507 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:50.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:50.508 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:50.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:04:50 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:04:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:04:50.609 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:50.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:50.651 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:50.651 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:50.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:50 smithi018 ceph-mon[29336]: pgmap v909: 178 pgs: 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 168 active+clean; 7.5 GiB data, 29 GiB used, 508 GiB / 536 GiB avail; 1.0 KiB/s rd, 233 MiB/s wr, 3.73k op/s; 47538/370725 objects misplaced (12.823%); 34 MiB/s, 539 objects/s recovering 2024-09-10T19:04:50.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:50 smithi018 ceph-mon[29336]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T19:04:50.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:50 smithi018 ceph-mon[29336]: osdmap e410: 8 total, 7 up, 6 in 2024-09-10T19:04:50.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:50 smithi018 ceph-mon[29336]: from='osd.7 v2:172.21.15.103:6812/3644107622' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T19:04:50.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:50 smithi018 ceph-mon[29336]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T19:04:50.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:50 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:04:50.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:50 smithi018 ceph-mon[40196]: pgmap v909: 178 pgs: 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 168 active+clean; 7.5 GiB data, 29 GiB used, 508 GiB / 536 GiB avail; 1.0 KiB/s rd, 233 MiB/s wr, 3.73k op/s; 47538/370725 objects misplaced (12.823%); 34 MiB/s, 539 objects/s recovering 2024-09-10T19:04:50.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:50 smithi018 ceph-mon[40196]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T19:04:50.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:50 smithi018 ceph-mon[40196]: osdmap e410: 8 total, 7 up, 6 in 2024-09-10T19:04:50.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:50 smithi018 ceph-mon[40196]: from='osd.7 v2:172.21.15.103:6812/3644107622' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T19:04:50.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:50 smithi018 ceph-mon[40196]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T19:04:50.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:50 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:04:50.752 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:50.801 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:50.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:50.802 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:50.903 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:50.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:50.957 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:50.957 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:51.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:50 smithi103 ceph-mon[33163]: pgmap v909: 178 pgs: 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 168 active+clean; 7.5 GiB data, 29 GiB used, 508 GiB / 536 GiB avail; 1.0 KiB/s rd, 233 MiB/s wr, 3.73k op/s; 47538/370725 objects misplaced (12.823%); 34 MiB/s, 539 objects/s recovering 2024-09-10T19:04:51.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:50 smithi103 ceph-mon[33163]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T19:04:51.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:50 smithi103 ceph-mon[33163]: osdmap e410: 8 total, 7 up, 6 in 2024-09-10T19:04:51.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:50 smithi103 ceph-mon[33163]: from='osd.7 v2:172.21.15.103:6812/3644107622' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T19:04:51.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:50 smithi103 ceph-mon[33163]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T19:04:51.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:50 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:04:51.057 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:51.080 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 83 16 129015 128999 97.1208 81.4375 0.0113587 0.0102921 2024-09-10T19:04:51.084 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:04:50 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:04:50] "GET /metrics HTTP/1.1" 200 209609 "" "Prometheus/2.43.0" 2024-09-10T19:04:51.098 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:51.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:51.100 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:51.200 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:51.236 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:51.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:51.237 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:51.338 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:51.385 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:51.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:51.386 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:51.487 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:51.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:51.527 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:51.528 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:51.628 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:51.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:51.667 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:51.668 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:51.768 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:51.796 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:51.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:51.797 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:51.898 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:51.923 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:51.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:51.924 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:52.025 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:52.069 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:52.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:52.070 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:52.080 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 84 16 130196 130180 96.8433 73.8125 0.0102737 0.0103216 2024-09-10T19:04:52.171 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:52.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:51 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:04:52.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:51 smithi018 ceph-mon[40196]: osd.7 v2:172.21.15.103:6812/3644107622 boot 2024-09-10T19:04:52.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:51 smithi018 ceph-mon[40196]: osdmap e411: 8 total, 8 up, 6 in 2024-09-10T19:04:52.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:51 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:04:52.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:51 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:04:52.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:51 smithi018 ceph-mon[40196]: pgmap v912: 178 pgs: 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 168 active+clean; 7.7 GiB data, 29 GiB used, 507 GiB / 536 GiB avail; 639 B/s rd, 84 MiB/s wr, 1.34k op/s; 47527/378012 objects misplaced (12.573%); 39 MiB/s, 629 objects/s recovering 2024-09-10T19:04:52.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:51 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:04:52.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:51 smithi018 ceph-mon[29336]: osd.7 v2:172.21.15.103:6812/3644107622 boot 2024-09-10T19:04:52.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:51 smithi018 ceph-mon[29336]: osdmap e411: 8 total, 8 up, 6 in 2024-09-10T19:04:52.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:51 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:04:52.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:51 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:04:52.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:51 smithi018 ceph-mon[29336]: pgmap v912: 178 pgs: 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 168 active+clean; 7.7 GiB data, 29 GiB used, 507 GiB / 536 GiB avail; 639 B/s rd, 84 MiB/s wr, 1.34k op/s; 47527/378012 objects misplaced (12.573%); 39 MiB/s, 629 objects/s recovering 2024-09-10T19:04:52.212 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:52.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:52.213 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:52.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:51 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:04:52.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:51 smithi103 ceph-mon[33163]: osd.7 v2:172.21.15.103:6812/3644107622 boot 2024-09-10T19:04:52.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:51 smithi103 ceph-mon[33163]: osdmap e411: 8 total, 8 up, 6 in 2024-09-10T19:04:52.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:51 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:04:52.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:51 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:04:52.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:51 smithi103 ceph-mon[33163]: pgmap v912: 178 pgs: 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 168 active+clean; 7.7 GiB data, 29 GiB used, 507 GiB / 536 GiB avail; 639 B/s rd, 84 MiB/s wr, 1.34k op/s; 47527/378012 objects misplaced (12.573%); 39 MiB/s, 629 objects/s recovering 2024-09-10T19:04:52.314 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:52.350 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:52.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:52.351 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:52.452 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:52.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:52.493 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:52.493 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:52.594 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:52.636 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:52.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:52.637 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:52.738 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:52.772 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:52.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:52.773 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:52.874 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:52.917 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:52.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:52.918 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:53.019 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:53.061 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:53.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:53.062 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:53.080 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 85 16 131637 131621 96.7634 90.0625 0.00633953 0.0103298 2024-09-10T19:04:53.163 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:53.200 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:52 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:04:53.201 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:52 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:04:53.201 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:52 smithi018 ceph-mon[29336]: osdmap e412: 8 total, 8 up, 6 in 2024-09-10T19:04:53.201 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:52 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:04:53.201 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:52 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:04:53.201 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:52 smithi018 ceph-mon[40196]: osdmap e412: 8 total, 8 up, 6 in 2024-09-10T19:04:53.204 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:53.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:53.205 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:53.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:52 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:04:53.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:52 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:04:53.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:52 smithi103 ceph-mon[33163]: osdmap e412: 8 total, 8 up, 6 in 2024-09-10T19:04:53.306 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:53.343 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:53.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:53.345 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:53.446 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:53.486 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:53.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:53.487 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:53.588 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:53.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:53.622 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:53.622 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:53.723 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:53.763 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:53.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:53.764 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:53.865 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:53.901 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:53.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:53.903 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:54.003 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:54.049 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:54.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:54.050 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:54.080 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 86 16 133086 133070 96.6912 90.5625 0.00849706 0.010338 2024-09-10T19:04:54.151 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:54.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:54.178 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:54.178 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:54.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:53 smithi018 ceph-mon[29336]: pgmap v914: 178 pgs: 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 168 active+clean; 7.9 GiB data, 30 GiB used, 506 GiB / 536 GiB avail; 852 B/s rd, 71 MiB/s wr, 1.13k op/s; 45818/387174 objects misplaced (11.834%); 39 MiB/s, 618 objects/s recovering 2024-09-10T19:04:54.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:53 smithi018 ceph-mon[40196]: pgmap v914: 178 pgs: 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 168 active+clean; 7.9 GiB data, 30 GiB used, 506 GiB / 536 GiB avail; 852 B/s rd, 71 MiB/s wr, 1.13k op/s; 45818/387174 objects misplaced (11.834%); 39 MiB/s, 618 objects/s recovering 2024-09-10T19:04:54.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:53 smithi103 ceph-mon[33163]: pgmap v914: 178 pgs: 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 168 active+clean; 7.9 GiB data, 30 GiB used, 506 GiB / 536 GiB avail; 852 B/s rd, 71 MiB/s wr, 1.13k op/s; 45818/387174 objects misplaced (11.834%); 39 MiB/s, 618 objects/s recovering 2024-09-10T19:04:54.279 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:54.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:54.314 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:54.314 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:54.415 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:54.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:54.454 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:54.454 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:54.555 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:54.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:54.582 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:54.583 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:54.683 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:54.715 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:54.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:54.716 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:54.820 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:54.855 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:54.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:54.856 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:54.957 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:54.967 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-10T19:04:54.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:54.987 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:54.987 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:55.080 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 87 16 134212 134196 96.3887 70.375 0.00486501 0.0103579 2024-09-10T19:04:55.088 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:55.109 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:55.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:55.110 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:55.210 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:55.249 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:55.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:55.250 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:55.319 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:04:55.351 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:55.388 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:55.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:55.389 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:55.489 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:55.526 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:55.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:55.527 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:55.628 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:55.659 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:55.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:55.661 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:55.762 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:55.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:55.790 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:55.790 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:55.891 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:55.930 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:55.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:55.932 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:56.032 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:56.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:56.055 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:56.056 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:56.081 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 88 16 134891 134875 95.7756 42.4375 0.108747 0.0104286 2024-09-10T19:04:56.156 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:56.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:56.183 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:56.183 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:56.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:55 smithi018 ceph-mon[29336]: osdmap e413: 8 total, 8 up, 6 in 2024-09-10T19:04:56.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:55 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:04:56.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:04:56.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:55 smithi018 ceph-mon[29336]: pgmap v916: 178 pgs: 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 168 active+clean; 7.9 GiB data, 30 GiB used, 506 GiB / 536 GiB avail; 905 B/s rd, 74 MiB/s wr, 1.19k op/s; 45711/390876 objects misplaced (11.695%); 41 MiB/s, 656 objects/s recovering 2024-09-10T19:04:56.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:55 smithi018 ceph-mon[40196]: osdmap e413: 8 total, 8 up, 6 in 2024-09-10T19:04:56.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:55 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:04:56.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:04:56.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:55 smithi018 ceph-mon[40196]: pgmap v916: 178 pgs: 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 168 active+clean; 7.9 GiB data, 30 GiB used, 506 GiB / 536 GiB avail; 905 B/s rd, 74 MiB/s wr, 1.19k op/s; 45711/390876 objects misplaced (11.695%); 41 MiB/s, 656 objects/s recovering 2024-09-10T19:04:56.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:55 smithi103 ceph-mon[33163]: osdmap e413: 8 total, 8 up, 6 in 2024-09-10T19:04:56.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:55 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:04:56.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:04:56.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:55 smithi103 ceph-mon[33163]: pgmap v916: 178 pgs: 4 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 168 active+clean; 7.9 GiB data, 30 GiB used, 506 GiB / 536 GiB avail; 905 B/s rd, 74 MiB/s wr, 1.19k op/s; 45711/390876 objects misplaced (11.695%); 41 MiB/s, 656 objects/s recovering 2024-09-10T19:04:56.284 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:56.316 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:56.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:56.317 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:56.418 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:56.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:56.446 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:56.446 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:56.547 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:56.586 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:56.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:56.587 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:56.688 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:56.722 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:56.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:56.723 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:56.823 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:56.846 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:56 smithi018 ceph-mon[29336]: osdmap e414: 8 total, 8 up, 6 in 2024-09-10T19:04:56.850 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:56.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:56.851 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:56.952 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:56.984 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:56.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:56.985 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:57.081 INFO:tasks.radosbench.radosbench.0.smithi018.stdout: 89 16 136101 136085 95.5491 75.625 0.018888 0.0104614 2024-09-10T19:04:57.086 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:57.116 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:57.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:57.117 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:57.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:56 smithi018 ceph-mon[40196]: osdmap e414: 8 total, 8 up, 6 in 2024-09-10T19:04:57.218 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:57.221 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:56 smithi103 ceph-mon[33163]: osdmap e414: 8 total, 8 up, 6 in 2024-09-10T19:04:57.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:57.249 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:57.249 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:57.350 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:57.378 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:57.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:57.379 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:57.480 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:57.501 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:57.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:57.503 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:57.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:04:57 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:04:57.604 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:57.642 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:57.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:57.644 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:57.745 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:57.785 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:57.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:57.786 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:57.886 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:57.917 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:57.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:57.919 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:58.019 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:58.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:58.061 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:58.061 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:58.081 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Total time run: 90.0051 2024-09-10T19:04:58.081 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Total writes made: 137089 2024-09-10T19:04:58.081 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Write size: 65536 2024-09-10T19:04:58.081 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Object size: 65536 2024-09-10T19:04:58.081 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Bandwidth (MB/sec): 95.1953 2024-09-10T19:04:58.081 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Stddev Bandwidth: 34.3458 2024-09-10T19:04:58.081 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Max bandwidth (MB/sec): 220.875 2024-09-10T19:04:58.081 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Min bandwidth (MB/sec): 0 2024-09-10T19:04:58.081 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Average IOPS: 1523 2024-09-10T19:04:58.082 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Stddev IOPS: 549.533 2024-09-10T19:04:58.082 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Max IOPS: 3534 2024-09-10T19:04:58.082 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Min IOPS: 0 2024-09-10T19:04:58.082 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Average Latency(s): 0.0105008 2024-09-10T19:04:58.082 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Stddev Latency(s): 0.0254314 2024-09-10T19:04:58.082 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Max latency(s): 1.9215 2024-09-10T19:04:58.082 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Min latency(s): 0.00116366 2024-09-10T19:04:58.084 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Cleaning up (deleting benchmark objects) 2024-09-10T19:04:58.162 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:58.183 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:58.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:58.184 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:58.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:57 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:58.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:57 smithi018 ceph-mon[29336]: pgmap v918: 178 pgs: 3 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 169 active+clean; 8.2 GiB data, 32 GiB used, 505 GiB / 536 GiB avail; 1.7 KiB/s rd, 94 MiB/s wr, 1.50k op/s; 43952/404976 objects misplaced (10.853%); 55 MiB/s, 885 objects/s recovering 2024-09-10T19:04:58.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:57 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:58.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:57 smithi018 ceph-mon[40196]: pgmap v918: 178 pgs: 3 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 169 active+clean; 8.2 GiB data, 32 GiB used, 505 GiB / 536 GiB avail; 1.7 KiB/s rd, 94 MiB/s wr, 1.50k op/s; 43952/404976 objects misplaced (10.853%); 55 MiB/s, 885 objects/s recovering 2024-09-10T19:04:58.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:57 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:58.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:57 smithi103 ceph-mon[33163]: pgmap v918: 178 pgs: 3 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 169 active+clean; 8.2 GiB data, 32 GiB used, 505 GiB / 536 GiB avail; 1.7 KiB/s rd, 94 MiB/s wr, 1.50k op/s; 43952/404976 objects misplaced (10.853%); 55 MiB/s, 885 objects/s recovering 2024-09-10T19:04:58.285 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:58.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:58.323 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:58.324 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:04:58.424 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:58.456 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:58.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:58.457 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:58.558 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:58.580 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:58.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:58.581 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:58.682 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:58.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:58.705 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:58.706 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:58.806 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:04:58.839 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:58.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:58.840 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:04:58.941 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:58.962 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:58.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:58.963 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:59.064 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:59.103 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:59.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:59.105 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:59.205 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:04:59.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:59.230 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:59.230 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:04:59.331 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:04:59.365 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:59.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:59.366 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:04:59.467 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:04:59.506 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:59.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:59.508 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:04:59.609 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:04:59.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:59.640 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:04:59.641 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:04:59.741 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:04:59.773 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:59.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:59.774 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:04:59.875 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:04:59.916 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:04:59.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:59.917 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:00.018 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:00.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:59 smithi103 ceph-mon[33163]: 4.5 scrub starts 2024-09-10T19:05:00.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:04:59 smithi103 ceph-mon[33163]: pgmap v919: 178 pgs: 3 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 169 active+clean; 8.3 GiB data, 32 GiB used, 505 GiB / 536 GiB avail; 669 B/s rd, 59 MiB/s wr, 954 op/s; 43314/407007 objects misplaced (10.642%); 34 MiB/s, 551 objects/s recovering 2024-09-10T19:05:00.047 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:00.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:00.048 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:00.149 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:00.176 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:00.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:00.177 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:00.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:59 smithi018 ceph-mon[40196]: 4.5 scrub starts 2024-09-10T19:05:00.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:04:59 smithi018 ceph-mon[40196]: pgmap v919: 178 pgs: 3 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 169 active+clean; 8.3 GiB data, 32 GiB used, 505 GiB / 536 GiB avail; 669 B/s rd, 59 MiB/s wr, 954 op/s; 43314/407007 objects misplaced (10.642%); 34 MiB/s, 551 objects/s recovering 2024-09-10T19:05:00.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:59 smithi018 ceph-mon[29336]: 4.5 scrub starts 2024-09-10T19:05:00.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:04:59 smithi018 ceph-mon[29336]: pgmap v919: 178 pgs: 3 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 169 active+clean; 8.3 GiB data, 32 GiB used, 505 GiB / 536 GiB avail; 669 B/s rd, 59 MiB/s wr, 954 op/s; 43314/407007 objects misplaced (10.642%); 34 MiB/s, 551 objects/s recovering 2024-09-10T19:05:00.278 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:00.304 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:00.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:00.305 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:00.406 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:00.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:00.431 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:00.432 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:00.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:05:00 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:05:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:05:00.533 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:00.574 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:00.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:00.575 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:00.676 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:00.697 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:00.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:00.698 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:00.799 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:00.831 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:00.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:00.832 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:00.933 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:00.972 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:00.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:00.973 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:01.074 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:01.095 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:01.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:01.096 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:01.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:00 smithi018 ceph-mon[29336]: 2.17 scrub starts 2024-09-10T19:05:01.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:00 smithi018 ceph-mon[40196]: 2.17 scrub starts 2024-09-10T19:05:01.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:05:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:05:00] "GET /metrics HTTP/1.1" 200 224427 "" "Prometheus/2.43.0" 2024-09-10T19:05:01.197 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:01.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:01.250 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:01.251 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:01.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:00 smithi103 ceph-mon[33163]: 2.17 scrub starts 2024-09-10T19:05:01.351 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:01.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:01.383 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:01.384 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:01.484 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:01.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:01.527 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:01.527 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:01.628 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:01.649 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:01.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:01.650 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:01.751 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:01.792 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:01.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:01.805 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:01.905 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:01.943 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:01.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:01.944 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:02.045 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:02.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:02.068 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:02.069 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:02.151 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:01 smithi103 ceph-mon[33163]: 5.2 scrub starts 2024-09-10T19:05:02.152 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:01 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:05:02.152 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:01 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:05:02.152 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:01 smithi103 ceph-mon[33163]: osdmap e415: 8 total, 8 up, 6 in 2024-09-10T19:05:02.152 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:01 smithi103 ceph-mon[33163]: pgmap v921: 178 pgs: 3 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 169 active+clean; 8.2 GiB data, 32 GiB used, 505 GiB / 536 GiB avail; 771 B/s rd, 59 MiB/s wr, 1.12k op/s; 42403/406038 objects misplaced (10.443%); 40 MiB/s, 633 objects/s recovering 2024-09-10T19:05:02.169 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:02.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:01 smithi018 ceph-mon[40196]: 5.2 scrub starts 2024-09-10T19:05:02.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:01 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:05:02.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:01 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:05:02.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:01 smithi018 ceph-mon[40196]: osdmap e415: 8 total, 8 up, 6 in 2024-09-10T19:05:02.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:01 smithi018 ceph-mon[40196]: pgmap v921: 178 pgs: 3 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 169 active+clean; 8.2 GiB data, 32 GiB used, 505 GiB / 536 GiB avail; 771 B/s rd, 59 MiB/s wr, 1.12k op/s; 42403/406038 objects misplaced (10.443%); 40 MiB/s, 633 objects/s recovering 2024-09-10T19:05:02.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:01 smithi018 ceph-mon[29336]: 5.2 scrub starts 2024-09-10T19:05:02.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:01 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:05:02.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:01 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:05:02.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:01 smithi018 ceph-mon[29336]: osdmap e415: 8 total, 8 up, 6 in 2024-09-10T19:05:02.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:01 smithi018 ceph-mon[29336]: pgmap v921: 178 pgs: 3 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 169 active+clean; 8.2 GiB data, 32 GiB used, 505 GiB / 536 GiB avail; 771 B/s rd, 59 MiB/s wr, 1.12k op/s; 42403/406038 objects misplaced (10.443%); 40 MiB/s, 633 objects/s recovering 2024-09-10T19:05:02.207 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:02.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:02.208 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:02.310 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:02.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:02.349 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:02.349 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:02.450 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:02.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:02.481 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:02.481 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:02.582 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:02.617 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:02.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:02.618 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:02.719 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:02.747 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:02.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:02.748 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:02.849 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:02.878 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:02.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:02.879 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:02.961 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T19:05:02.961 INFO:teuthology.orchestra.run.smithi103.stdout: "ops": [], 2024-09-10T19:05:02.962 INFO:teuthology.orchestra.run.smithi103.stdout: "num_ops": 0 2024-09-10T19:05:02.962 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T19:05:02.980 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:03.021 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:03.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:03.022 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:03.123 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:03.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:03.161 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:03.162 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:03.262 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:03.284 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:03.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:03.285 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:03.385 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:03.417 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:03.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:03.421 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:03.521 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:03.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:03.543 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:03.543 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:03.644 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:03.671 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:03.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:03.672 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:03.773 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:03.795 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:03.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:03.796 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:03.801 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd reweight 0 1 2024-09-10T19:05:03.897 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:03.927 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:03.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:03.928 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:04.029 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:04.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:04.063 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:04.063 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:04.078 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:04 smithi018 ceph-mon[29336]: osdmap e416: 8 total, 8 up, 6 in 2024-09-10T19:05:04.164 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:04.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:04.188 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:04.189 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:04.289 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:04.312 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:04.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:04.313 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:04.414 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:04.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:04.442 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:04.442 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:04.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:04 smithi018 ceph-mon[29336]: pgmap v923: 178 pgs: 3 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 169 active+clean; 8.2 GiB data, 32 GiB used, 505 GiB / 536 GiB avail; 1.4 KiB/s rd, 57 MiB/s wr, 1.30k op/s; 40858/403023 objects misplaced (10.138%); 39 MiB/s, 626 objects/s recovering 2024-09-10T19:05:04.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:04 smithi018 ceph-mon[29336]: osdmap e417: 8 total, 8 up, 6 in 2024-09-10T19:05:04.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:04 smithi018 ceph-mon[40196]: osdmap e416: 8 total, 8 up, 6 in 2024-09-10T19:05:04.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:04 smithi018 ceph-mon[40196]: pgmap v923: 178 pgs: 3 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 169 active+clean; 8.2 GiB data, 32 GiB used, 505 GiB / 536 GiB avail; 1.4 KiB/s rd, 57 MiB/s wr, 1.30k op/s; 40858/403023 objects misplaced (10.138%); 39 MiB/s, 626 objects/s recovering 2024-09-10T19:05:04.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:04 smithi018 ceph-mon[40196]: osdmap e417: 8 total, 8 up, 6 in 2024-09-10T19:05:04.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:04 smithi103 ceph-mon[33163]: osdmap e416: 8 total, 8 up, 6 in 2024-09-10T19:05:04.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:04 smithi103 ceph-mon[33163]: pgmap v923: 178 pgs: 3 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 169 active+clean; 8.2 GiB data, 32 GiB used, 505 GiB / 536 GiB avail; 1.4 KiB/s rd, 57 MiB/s wr, 1.30k op/s; 40858/403023 objects misplaced (10.138%); 39 MiB/s, 626 objects/s recovering 2024-09-10T19:05:04.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:04 smithi103 ceph-mon[33163]: osdmap e417: 8 total, 8 up, 6 in 2024-09-10T19:05:04.543 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:04.566 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:04.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:04.567 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:04.668 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:04.691 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:04.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:04.693 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:04.715 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T19:05:04.715 INFO:teuthology.orchestra.run.smithi018.stdout: "ops": [ 2024-09-10T19:05:04.715 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:05:04.715 INFO:teuthology.orchestra.run.smithi018.stdout: "description": "osd_op(client.25706.0:142582 11.2 11:425c2ed4:::benchmark_data_smithi018_181485_object5490:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e416)", 2024-09-10T19:05:04.716 INFO:teuthology.orchestra.run.smithi018.stdout: "initiated_at": "2024-09-10T19:05:03.779208+0000", 2024-09-10T19:05:04.716 INFO:teuthology.orchestra.run.smithi018.stdout: "age": 0.028852525, 2024-09-10T19:05:04.716 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.028875006000000002, 2024-09-10T19:05:04.716 INFO:teuthology.orchestra.run.smithi018.stdout: "type_data": { 2024-09-10T19:05:04.716 INFO:teuthology.orchestra.run.smithi018.stdout: "flag_point": "waiting for sub ops", 2024-09-10T19:05:04.716 INFO:teuthology.orchestra.run.smithi018.stdout: "client_info": { 2024-09-10T19:05:04.716 INFO:teuthology.orchestra.run.smithi018.stdout: "client": "client.25706", 2024-09-10T19:05:04.716 INFO:teuthology.orchestra.run.smithi018.stdout: "client_addr": "172.21.15.18:0/42081399", 2024-09-10T19:05:04.716 INFO:teuthology.orchestra.run.smithi018.stdout: "tid": 142582 2024-09-10T19:05:04.716 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:05:04.717 INFO:teuthology.orchestra.run.smithi018.stdout: "events": [ 2024-09-10T19:05:04.717 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:05:04.717 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "initiated", 2024-09-10T19:05:04.717 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:05:03.779208+0000", 2024-09-10T19:05:04.717 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:05:04.717 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:05:04.717 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:05:04.717 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "throttled", 2024-09-10T19:05:04.717 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:05:03.779208+0000", 2024-09-10T19:05:04.717 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0 2024-09-10T19:05:04.717 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:05:04.717 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:05:04.717 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "header_read", 2024-09-10T19:05:04.718 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:05:03.779207+0000", 2024-09-10T19:05:04.718 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 4294967295.999999 2024-09-10T19:05:04.718 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:05:04.718 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:05:04.718 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "all_read", 2024-09-10T19:05:04.718 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:05:03.779212+0000", 2024-09-10T19:05:04.718 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.1889999999999998e-06 2024-09-10T19:05:04.718 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:05:04.718 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:05:04.718 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "dispatched", 2024-09-10T19:05:04.719 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:05:03.779222+0000", 2024-09-10T19:05:04.719 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 1.0502999999999999e-05 2024-09-10T19:05:04.719 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:05:04.719 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:05:04.719 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "queued_for_pg", 2024-09-10T19:05:04.719 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:05:03.779228+0000", 2024-09-10T19:05:04.719 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 5.7939999999999999e-06 2024-09-10T19:05:04.719 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:05:04.719 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:05:04.719 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "reached_pg", 2024-09-10T19:05:04.719 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:05:03.779262+0000", 2024-09-10T19:05:04.719 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 3.3331000000000003e-05 2024-09-10T19:05:04.719 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:05:04.720 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:05:04.720 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "started", 2024-09-10T19:05:04.720 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:05:03.779333+0000", 2024-09-10T19:05:04.720 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 7.1873999999999997e-05 2024-09-10T19:05:04.720 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:05:04.720 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:05:04.720 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "waiting for subops from 0,1", 2024-09-10T19:05:04.720 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:05:03.779579+0000", 2024-09-10T19:05:04.720 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00024553500000000001 2024-09-10T19:05:04.720 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:05:04.720 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:05:04.720 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "op_commit", 2024-09-10T19:05:04.721 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:05:03.780718+0000", 2024-09-10T19:05:04.721 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.0011394389999999999 2024-09-10T19:05:04.721 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:05:04.721 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:05:04.721 INFO:teuthology.orchestra.run.smithi018.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:04.721 INFO:teuthology.orchestra.run.smithi018.stdout: "time": "2024-09-10T19:05:03.781025+0000", 2024-09-10T19:05:04.721 INFO:teuthology.orchestra.run.smithi018.stdout: "duration": 0.00030695700000000001 2024-09-10T19:05:04.721 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:05:04.721 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:05:04.721 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:05:04.721 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:05:04.721 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-09-10T19:05:04.721 INFO:teuthology.orchestra.run.smithi018.stdout: "num_ops": 1 2024-09-10T19:05:04.721 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T19:05:04.793 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:04.819 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:04.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:04.820 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:04.920 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:04.952 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:04.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:04.953 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:05.054 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:05.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:05.088 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:05.088 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:05.189 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:05.223 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:05.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:05.224 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:05.324 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:05.364 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:05.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:05.365 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:05.465 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:05.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:05.507 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:05.507 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:05.608 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:05.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:05.650 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:05.650 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:05.725 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-10T19:05:05.726 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-10T19:05:05.751 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:05.785 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:05.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:05.786 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:05.887 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:05.928 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:05.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:05.929 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:06.030 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:06.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:06.074 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:06.074 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:06.175 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:06.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:05 smithi018 ceph-mon[40196]: osdmap e418: 8 total, 8 up, 6 in 2024-09-10T19:05:06.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:05 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:05:06.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:05 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:05:06.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:05 smithi018 ceph-mon[40196]: pgmap v926: 178 pgs: 2 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 170 active+clean; 8.1 GiB data, 32 GiB used, 505 GiB / 536 GiB avail; 1.3 KiB/s rd, 20 MiB/s wr, 1.10k op/s; 39641/400482 objects misplaced (9.898%); 29 MiB/s, 466 objects/s recovering 2024-09-10T19:05:06.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:05 smithi018 ceph-mon[29336]: osdmap e418: 8 total, 8 up, 6 in 2024-09-10T19:05:06.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:05 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:05:06.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:05 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:05:06.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:05 smithi018 ceph-mon[29336]: pgmap v926: 178 pgs: 2 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 170 active+clean; 8.1 GiB data, 32 GiB used, 505 GiB / 536 GiB avail; 1.3 KiB/s rd, 20 MiB/s wr, 1.10k op/s; 39641/400482 objects misplaced (9.898%); 29 MiB/s, 466 objects/s recovering 2024-09-10T19:05:06.209 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:06.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:06.211 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:06.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:05 smithi103 ceph-mon[33163]: osdmap e418: 8 total, 8 up, 6 in 2024-09-10T19:05:06.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:05 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:05:06.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:05 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:05:06.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:05 smithi103 ceph-mon[33163]: pgmap v926: 178 pgs: 2 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 170 active+clean; 8.1 GiB data, 32 GiB used, 505 GiB / 536 GiB avail; 1.3 KiB/s rd, 20 MiB/s wr, 1.10k op/s; 39641/400482 objects misplaced (9.898%); 29 MiB/s, 466 objects/s recovering 2024-09-10T19:05:06.295 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-10T19:05:06.312 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:06.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:06.393 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:06.393 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:06.494 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:06.529 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:06.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:06.531 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:06.631 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:06.682 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:06.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:06.683 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:06.784 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:06.818 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:06.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:06.819 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:06.839 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:06 smithi018 ceph-mon[40196]: 2.11 scrub starts 2024-09-10T19:05:06.839 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:06 smithi018 ceph-mon[40196]: osdmap e419: 8 total, 8 up, 6 in 2024-09-10T19:05:06.840 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:06 smithi018 ceph-mon[29336]: 2.11 scrub starts 2024-09-10T19:05:06.840 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:06 smithi018 ceph-mon[29336]: osdmap e419: 8 total, 8 up, 6 in 2024-09-10T19:05:06.851 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:05:06.919 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:06.977 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:06.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:06.978 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:07.079 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:07.128 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:07.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:07.129 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:07.230 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:07.233 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:06 smithi103 ceph-mon[33163]: 2.11 scrub starts 2024-09-10T19:05:07.233 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:06 smithi103 ceph-mon[33163]: osdmap e419: 8 total, 8 up, 6 in 2024-09-10T19:05:07.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:07.264 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:07.265 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:07.366 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:07.394 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:07.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:07.395 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:07.496 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:07.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:05:07 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:05:07.525 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:07.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:07.526 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:07.627 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:07.668 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:07.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:07.669 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:07.713 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:05:07.727 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T19:05:07.771 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:07.828 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:07.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:07.830 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:07.838 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is set 2024-09-10T19:05:07.930 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:07.983 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:07.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:07.984 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:08.085 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:08.115 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:08.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:08.116 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:08.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:07 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:08.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:07 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2918579776' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T19:05:08.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:07 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T19:05:08.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:07 smithi018 ceph-mon[40196]: pgmap v928: 178 pgs: 1 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 171 active+clean; 7.9 GiB data, 31 GiB used, 506 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.15k op/s; 36197/387804 objects misplaced (9.334%); 32 MiB/s, 510 objects/s recovering 2024-09-10T19:05:08.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:07 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:08.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:07 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2918579776' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T19:05:08.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:07 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T19:05:08.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:07 smithi018 ceph-mon[29336]: pgmap v928: 178 pgs: 1 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 171 active+clean; 7.9 GiB data, 31 GiB used, 506 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.15k op/s; 36197/387804 objects misplaced (9.334%); 32 MiB/s, 510 objects/s recovering 2024-09-10T19:05:08.217 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:08.238 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:08.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:08.239 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:08.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:07 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:08.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:07 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2918579776' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T19:05:08.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:07 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T19:05:08.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:07 smithi103 ceph-mon[33163]: pgmap v928: 178 pgs: 1 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 171 active+clean; 7.9 GiB data, 31 GiB used, 506 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.15k op/s; 36197/387804 objects misplaced (9.334%); 32 MiB/s, 510 objects/s recovering 2024-09-10T19:05:08.339 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:08.362 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:08.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:08.364 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:08.464 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:08.486 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:08.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:08.487 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:08.588 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:08.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:08.611 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:08.611 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:08.712 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:08.734 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:08.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:08.735 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:08.836 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:08.873 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:08.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:08.874 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:08.974 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:09.004 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:09.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:09.005 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:09.105 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:09.149 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:09.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:09.150 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:09.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:08 smithi018 ceph-mon[40196]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:05:09.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:08 smithi018 ceph-mon[40196]: 5.c scrub starts 2024-09-10T19:05:09.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:08 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T19:05:09.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:08 smithi018 ceph-mon[40196]: osdmap e420: 8 total, 8 up, 6 in 2024-09-10T19:05:09.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:08 smithi018 ceph-mon[40196]: osdmap e421: 8 total, 8 up, 6 in 2024-09-10T19:05:09.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:08 smithi018 ceph-mon[29336]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:05:09.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:08 smithi018 ceph-mon[29336]: 5.c scrub starts 2024-09-10T19:05:09.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:08 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T19:05:09.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:08 smithi018 ceph-mon[29336]: osdmap e420: 8 total, 8 up, 6 in 2024-09-10T19:05:09.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:08 smithi018 ceph-mon[29336]: osdmap e421: 8 total, 8 up, 6 in 2024-09-10T19:05:09.250 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:09.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:08 smithi103 ceph-mon[33163]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:05:09.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:08 smithi103 ceph-mon[33163]: 5.c scrub starts 2024-09-10T19:05:09.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:08 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T19:05:09.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:08 smithi103 ceph-mon[33163]: osdmap e420: 8 total, 8 up, 6 in 2024-09-10T19:05:09.274 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:08 smithi103 ceph-mon[33163]: osdmap e421: 8 total, 8 up, 6 in 2024-09-10T19:05:09.283 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:09.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:09.284 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:09.385 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:09.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:09.432 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:09.433 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:09.534 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:09.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:09.573 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:09.573 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:09.674 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:09.710 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:09.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:09.711 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:09.811 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:09.852 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:09.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:09.853 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:09.953 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:09.984 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:09.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:09.986 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:10.086 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:10.114 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:10.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:10.115 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:10.147 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:09 smithi018 ceph-mon[29336]: pgmap v931: 178 pgs: 1 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 171 active+clean; 7.8 GiB data, 31 GiB used, 506 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.14k op/s; 35640/384552 objects misplaced (9.268%); 19 MiB/s, 311 objects/s recovering 2024-09-10T19:05:10.147 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:09 smithi018 ceph-mon[40196]: pgmap v931: 178 pgs: 1 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 171 active+clean; 7.8 GiB data, 31 GiB used, 506 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.14k op/s; 35640/384552 objects misplaced (9.268%); 19 MiB/s, 311 objects/s recovering 2024-09-10T19:05:10.216 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:10.238 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:09 smithi103 ceph-mon[33163]: pgmap v931: 178 pgs: 1 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 171 active+clean; 7.8 GiB data, 31 GiB used, 506 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.14k op/s; 35640/384552 objects misplaced (9.268%); 19 MiB/s, 311 objects/s recovering 2024-09-10T19:05:10.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:10.271 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:10.271 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:10.372 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:10.402 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:10.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:10.403 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:10.504 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:10.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:05:10 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:05:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:05:10.556 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:10.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:10.557 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:10.658 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:10.690 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:10.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:10.691 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:10.791 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:10.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:10.821 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:10.821 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:10.922 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:10.949 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:10.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:10.950 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:11.051 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:11.077 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:05:10 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:05:10] "GET /metrics HTTP/1.1" 200 224481 "" "Prometheus/2.43.0" 2024-09-10T19:05:11.104 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:11.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:11.105 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:11.206 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:11.239 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:11.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:11.240 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:11.341 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:11.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:11.380 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:11.380 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:11.383 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:11 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:05:11.383 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:11 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:05:11.383 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:11 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:05:11.383 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:11 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:05:11.391 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:05:11.482 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:11.507 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:11.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:11.508 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:11.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:11 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:05:11.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:11 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:05:11.609 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:11.633 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:11.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:11.634 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:11.735 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:11.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:11.780 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:11.780 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:11.881 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:11.912 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:11.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:11.913 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:12.014 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:12.049 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:12.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:12.050 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:12.120 INFO:teuthology.orchestra.run.smithi018.stderr:reweighted osd.0 to 1 (10000) 2024-09-10T19:05:12.150 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:12.180 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:12.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:12.181 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:12.281 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:12.310 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:12.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:12.311 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:12.405 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:12 smithi018 ceph-mon[40196]: pgmap v932: 178 pgs: 1 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 171 active+clean; 7.7 GiB data, 31 GiB used, 506 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 1.24k op/s; 35022/378246 objects misplaced (9.259%); 15 MiB/s, 242 objects/s recovering 2024-09-10T19:05:12.405 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:12 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2555298494' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-10T19:05:12.405 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:12 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-10T19:05:12.406 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:12 smithi018 ceph-mon[29336]: pgmap v932: 178 pgs: 1 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 171 active+clean; 7.7 GiB data, 31 GiB used, 506 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 1.24k op/s; 35022/378246 objects misplaced (9.259%); 15 MiB/s, 242 objects/s recovering 2024-09-10T19:05:12.406 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:12 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2555298494' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-10T19:05:12.406 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:12 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-10T19:05:12.412 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:12.456 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:12.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:12.457 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:12.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:12 smithi103 ceph-mon[33163]: pgmap v932: 178 pgs: 1 active+remapped+backfilling, 6 active+remapped+backfill_toofull, 171 active+clean; 7.7 GiB data, 31 GiB used, 506 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 1.24k op/s; 35022/378246 objects misplaced (9.259%); 15 MiB/s, 242 objects/s recovering 2024-09-10T19:05:12.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:12 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2555298494' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-10T19:05:12.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:12 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-10T19:05:12.558 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:12.585 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:12.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:12.586 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:12.687 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:12.705 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T19:05:12.705 INFO:teuthology.orchestra.run.smithi018.stdout: "ops": [], 2024-09-10T19:05:12.705 INFO:teuthology.orchestra.run.smithi018.stdout: "complaint_time": 30, 2024-09-10T19:05:12.705 INFO:teuthology.orchestra.run.smithi018.stdout: "num_blocked_ops": 0 2024-09-10T19:05:12.705 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T19:05:12.715 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:12.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:12.716 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:12.817 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:12.847 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:12.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:12.849 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:12.861 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-10T19:05:12.949 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:12.979 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:12.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:12.980 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:13.080 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:13.126 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:13.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:13.127 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:13.228 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:13.262 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:13.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:13.263 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:13.364 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:13.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:13.394 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:13.395 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:13.404 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:13 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-10T19:05:13.404 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:13 smithi018 ceph-mon[40196]: osdmap e422: 8 total, 8 up, 6 in 2024-09-10T19:05:13.404 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:13 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-10T19:05:13.404 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:13 smithi018 ceph-mon[29336]: osdmap e422: 8 total, 8 up, 6 in 2024-09-10T19:05:13.496 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:13.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:13 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-10T19:05:13.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:13 smithi103 ceph-mon[33163]: osdmap e422: 8 total, 8 up, 6 in 2024-09-10T19:05:13.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:13.537 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:13.537 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:13.638 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:13.660 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:13.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:13.661 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:13.762 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:13.789 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:13.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:13.790 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:13.891 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:13.936 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:13.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:13.938 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:14.039 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:14.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:14.063 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:14.063 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:14.164 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:14.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:14.195 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:14.195 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:14.296 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:14.333 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:14.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:14.334 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:14.435 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:14.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:14 smithi018 ceph-mon[29336]: pgmap v934: 178 pgs: 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 172 active+clean; 7.5 GiB data, 30 GiB used, 507 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 1.09k op/s; 33423/368253 objects misplaced (9.076%); 6.8 MiB/s, 108 objects/s recovering 2024-09-10T19:05:14.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:14 smithi018 ceph-mon[40196]: pgmap v934: 178 pgs: 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 172 active+clean; 7.5 GiB data, 30 GiB used, 507 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 1.09k op/s; 33423/368253 objects misplaced (9.076%); 6.8 MiB/s, 108 objects/s recovering 2024-09-10T19:05:14.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:14.458 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:14.458 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:14.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:14 smithi103 ceph-mon[33163]: pgmap v934: 178 pgs: 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 172 active+clean; 7.5 GiB data, 30 GiB used, 507 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 1.09k op/s; 33423/368253 objects misplaced (9.076%); 6.8 MiB/s, 108 objects/s recovering 2024-09-10T19:05:14.559 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:14.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:14.616 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:14.616 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:14.717 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:14.740 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:14.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:14.741 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:14.842 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:14.873 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:14.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:14.874 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:14.975 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:15.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:15.012 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:15.012 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:15.113 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:15.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:15.150 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:15.150 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:15.251 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:15.283 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:15.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:15.284 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:15.385 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:15.412 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:15.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:15.413 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:15.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:15 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:05:15.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:15 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:05:15.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:15 smithi018 ceph-mon[29336]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 4 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-10T19:05:15.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:15 smithi018 ceph-mon[29336]: osdmap e423: 8 total, 8 up, 6 in 2024-09-10T19:05:15.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:15 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:05:15.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:15 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:05:15.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:15 smithi018 ceph-mon[40196]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 4 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-10T19:05:15.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:15 smithi018 ceph-mon[40196]: osdmap e423: 8 total, 8 up, 6 in 2024-09-10T19:05:15.514 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:15.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:15 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:05:15.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:15 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:05:15.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:15 smithi103 ceph-mon[33163]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 4 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-10T19:05:15.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:15 smithi103 ceph-mon[33163]: osdmap e423: 8 total, 8 up, 6 in 2024-09-10T19:05:15.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:15.544 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:15.544 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:15.645 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:15.680 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:15.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:15.681 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:15.782 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:15.811 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:15.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:15.813 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:15.914 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:15.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:15.937 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:15.937 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:16.038 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:16.069 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:16.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:16.071 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:16.171 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:16.199 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:16.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:16.200 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:16.301 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:16.347 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:16.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:16.348 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:16.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:16 smithi103 ceph-mon[33163]: pgmap v936: 178 pgs: 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 172 active+clean; 7.4 GiB data, 30 GiB used, 507 GiB / 536 GiB avail; 766 B/s rd, 0 B/s wr, 1.18k op/s; 31450/364155 objects misplaced (8.636%); 15 MiB/s, 234 objects/s recovering 2024-09-10T19:05:16.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:16 smithi018 ceph-mon[29336]: pgmap v936: 178 pgs: 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 172 active+clean; 7.4 GiB data, 30 GiB used, 507 GiB / 536 GiB avail; 766 B/s rd, 0 B/s wr, 1.18k op/s; 31450/364155 objects misplaced (8.636%); 15 MiB/s, 234 objects/s recovering 2024-09-10T19:05:16.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:16 smithi018 ceph-mon[40196]: pgmap v936: 178 pgs: 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 172 active+clean; 7.4 GiB data, 30 GiB used, 507 GiB / 536 GiB avail; 766 B/s rd, 0 B/s wr, 1.18k op/s; 31450/364155 objects misplaced (8.636%); 15 MiB/s, 234 objects/s recovering 2024-09-10T19:05:16.449 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:16.470 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:16.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:16.472 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:16.572 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:16.605 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:16.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:16.606 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:16.706 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:16.747 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:16.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:16.748 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:16.849 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:16.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:16.876 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:16.876 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:16.977 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:17.017 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:17.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:17.019 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:17.120 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:17.166 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:17.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:17.168 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:17.268 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:17.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:17.293 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:17.293 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:17.394 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:17.434 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:17.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:17.435 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:17.448 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:05:17 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:05:17.536 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:17.574 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:17.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:17.575 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:17.676 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:17.713 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:17.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:17.714 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:17.815 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:17.847 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:17.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:17.848 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:17.949 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:17.989 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:17.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:17.991 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:18.091 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:18.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:18.121 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:18.121 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:18.222 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:18.265 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:18.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:18.266 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:18.367 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:18.395 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:18.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:18.396 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:18.497 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:18.523 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:18.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:18.525 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:18.626 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:18.665 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:18.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:18.666 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:18.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:18 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:18.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:18 smithi018 ceph-mon[29336]: pgmap v937: 178 pgs: 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 172 active+clean; 7.1 GiB data, 29 GiB used, 508 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.46k op/s; 28766/349623 objects misplaced (8.228%); 21 MiB/s, 342 objects/s recovering 2024-09-10T19:05:18.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:18 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:18.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:18 smithi018 ceph-mon[40196]: pgmap v937: 178 pgs: 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 172 active+clean; 7.1 GiB data, 29 GiB used, 508 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.46k op/s; 28766/349623 objects misplaced (8.228%); 21 MiB/s, 342 objects/s recovering 2024-09-10T19:05:18.767 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:18.789 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:18.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:18.790 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:18.891 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:18.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:18.935 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:18.936 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:19.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:18 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:19.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:18 smithi103 ceph-mon[33163]: pgmap v937: 178 pgs: 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 172 active+clean; 7.1 GiB data, 29 GiB used, 508 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.46k op/s; 28766/349623 objects misplaced (8.228%); 21 MiB/s, 342 objects/s recovering 2024-09-10T19:05:19.037 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:19.059 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:19.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:19.060 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:19.161 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:19.200 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:19.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:19.201 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:19.302 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:19.333 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:19.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:19.334 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:19.435 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:19.472 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:19.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:19.474 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:19.575 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:19.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:19.618 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:19.618 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:19.719 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:19.742 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:19.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:19.743 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:19.844 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:19.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:19.871 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:19.872 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:19.973 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:19.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:19.995 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:19.995 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:20.096 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:20.126 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:20.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:20.127 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:20.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:19 smithi018 ceph-mon[29336]: osdmap e424: 8 total, 8 up, 6 in 2024-09-10T19:05:20.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:19 smithi018 ceph-mon[29336]: pgmap v939: 178 pgs: 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 172 active+clean; 7.0 GiB data, 29 GiB used, 508 GiB / 536 GiB avail; 694 B/s rd, 0 B/s wr, 989 op/s; 26904/346389 objects misplaced (7.767%); 26 MiB/s, 418 objects/s recovering 2024-09-10T19:05:20.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:19 smithi018 ceph-mon[40196]: osdmap e424: 8 total, 8 up, 6 in 2024-09-10T19:05:20.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:19 smithi018 ceph-mon[40196]: pgmap v939: 178 pgs: 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 172 active+clean; 7.0 GiB data, 29 GiB used, 508 GiB / 536 GiB avail; 694 B/s rd, 0 B/s wr, 989 op/s; 26904/346389 objects misplaced (7.767%); 26 MiB/s, 418 objects/s recovering 2024-09-10T19:05:20.228 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:20.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:19 smithi103 ceph-mon[33163]: osdmap e424: 8 total, 8 up, 6 in 2024-09-10T19:05:20.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:19 smithi103 ceph-mon[33163]: pgmap v939: 178 pgs: 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 172 active+clean; 7.0 GiB data, 29 GiB used, 508 GiB / 536 GiB avail; 694 B/s rd, 0 B/s wr, 989 op/s; 26904/346389 objects misplaced (7.767%); 26 MiB/s, 418 objects/s recovering 2024-09-10T19:05:20.257 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:20.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:20.258 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:20.359 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:20.391 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:20.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:20.392 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:20.493 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:20.522 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:20.523 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:05:20 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:05:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:05:20.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:20.525 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:20.625 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:20.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:20.667 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:20.668 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:20.768 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:20.791 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:20.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:20.792 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:20.893 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:20.937 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:20.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:20.938 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:21.039 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:21.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:21.069 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:21.069 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:21.171 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:21.192 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:21.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:21.193 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:21.294 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:21.340 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:21.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:21.341 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:21.441 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:21.445 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:05:20 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:05:20] "GET /metrics HTTP/1.1" 200 224481 "" "Prometheus/2.43.0" 2024-09-10T19:05:21.483 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:21.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:21.484 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:21.536 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd reweight 1 1 2024-09-10T19:05:21.585 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:21.612 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:21.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:21.613 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:21.714 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:21.741 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:21.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:21.742 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:21.843 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:21.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:21.907 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:21.907 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:22.008 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:22.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:22.040 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:22.040 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:22.141 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:22.172 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:22.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:22.174 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:22.274 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:22.328 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:22.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:22.329 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:22.430 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:22.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:22.472 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:22.472 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:22.562 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-10T19:05:22.573 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:22.589 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:05:22.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:22.634 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:22.635 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:22.735 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:22.763 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:22.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:22.764 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:22.865 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:22.869 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:22 smithi018 ceph-mon[29336]: pgmap v940: 178 pgs: 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 172 active+clean; 6.9 GiB data, 28 GiB used, 508 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 1.12k op/s; 26378/341250 objects misplaced (7.730%); 24 MiB/s, 385 objects/s recovering 2024-09-10T19:05:22.869 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:22 smithi018 ceph-mon[40196]: pgmap v940: 178 pgs: 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 172 active+clean; 6.9 GiB data, 28 GiB used, 508 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 1.12k op/s; 26378/341250 objects misplaced (7.730%); 24 MiB/s, 385 objects/s recovering 2024-09-10T19:05:22.890 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:22.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:22.891 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:22.991 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:23.018 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:23.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:23.019 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:23.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:22 smithi103 ceph-mon[33163]: pgmap v940: 178 pgs: 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 172 active+clean; 6.9 GiB data, 28 GiB used, 508 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 1.12k op/s; 26378/341250 objects misplaced (7.730%); 24 MiB/s, 385 objects/s recovering 2024-09-10T19:05:23.119 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:23.149 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:23.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:23.150 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:23.251 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:23.273 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:23.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:23.274 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:23.375 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:23.405 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:23.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:23.406 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:23.507 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:23.539 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:23.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:23.540 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:23.641 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:23.668 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:23.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:23.669 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:23.763 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:23 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:05:23.763 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:23 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:05:23.764 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:23 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:05:23.764 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:23 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:05:23.770 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:23.798 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:23.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:23.799 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:23.900 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:23.943 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:23.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:23.944 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:24.007 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T19:05:24.007 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T19:05:24.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:23 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:05:24.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:23 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:05:24.040 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T19:05:24.041 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T19:05:24.045 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:24.056 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T19:05:24.057 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T19:05:24.068 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:24.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:24.069 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:24.073 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T19:05:24.073 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T19:05:24.089 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T19:05:24.089 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T19:05:24.116 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T19:05:24.116 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T19:05:24.149 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T19:05:24.149 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T19:05:24.170 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:24.185 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T19:05:24.186 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T19:05:24.210 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:24.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:24.212 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:24.312 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:24.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:24.342 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:24.343 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:24.443 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:24.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:24.481 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:24.481 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:24.582 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:24.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:24.624 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:24.624 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:24.725 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:24.755 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:24.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:24.757 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:24.857 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:24.899 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:24.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:24.900 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:24.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:24 smithi018 ceph-mon[29336]: pgmap v941: 178 pgs: 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 172 active+clean; 6.8 GiB data, 28 GiB used, 509 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.11k op/s; 24177/332817 objects misplaced (7.264%); 23 MiB/s, 368 objects/s recovering 2024-09-10T19:05:24.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:24 smithi018 ceph-mon[29336]: osdmap e425: 8 total, 8 up, 6 in 2024-09-10T19:05:24.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:24 smithi018 ceph-mon[29336]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-10T19:05:24.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:24 smithi018 ceph-mon[29336]: osdmap e426: 8 total, 8 up, 6 in 2024-09-10T19:05:24.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:24 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1271212181' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:05:24.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:24 smithi018 ceph-mon[40196]: pgmap v941: 178 pgs: 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 172 active+clean; 6.8 GiB data, 28 GiB used, 509 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.11k op/s; 24177/332817 objects misplaced (7.264%); 23 MiB/s, 368 objects/s recovering 2024-09-10T19:05:24.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:24 smithi018 ceph-mon[40196]: osdmap e425: 8 total, 8 up, 6 in 2024-09-10T19:05:24.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:24 smithi018 ceph-mon[40196]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-10T19:05:24.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:24 smithi018 ceph-mon[40196]: osdmap e426: 8 total, 8 up, 6 in 2024-09-10T19:05:24.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:24 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1271212181' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:05:25.001 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:25.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:24 smithi103 ceph-mon[33163]: pgmap v941: 178 pgs: 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 172 active+clean; 6.8 GiB data, 28 GiB used, 509 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.11k op/s; 24177/332817 objects misplaced (7.264%); 23 MiB/s, 368 objects/s recovering 2024-09-10T19:05:25.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:24 smithi103 ceph-mon[33163]: osdmap e425: 8 total, 8 up, 6 in 2024-09-10T19:05:25.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:24 smithi103 ceph-mon[33163]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-10T19:05:25.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:24 smithi103 ceph-mon[33163]: osdmap e426: 8 total, 8 up, 6 in 2024-09-10T19:05:25.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:24 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1271212181' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:05:25.041 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:25.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:25.042 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:25.143 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:25.170 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:25.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:25.171 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:25.272 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:25.294 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:25.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:25.295 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:25.396 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:25.441 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:25.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:25.442 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:25.543 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:25.572 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:25.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:25.573 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:25.674 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:25.695 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:25.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:25.696 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:25.797 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:25.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:25.839 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:25.839 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:25.940 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:25.976 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:25.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:25.977 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:26.078 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:26.110 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:26.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:26.111 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:26.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:25 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:05:26.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:25 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:05:26.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:25 smithi018 ceph-mon[29336]: osdmap e427: 8 total, 8 up, 6 in 2024-09-10T19:05:26.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:25 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:05:26.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:25 smithi018 ceph-mon[29336]: pgmap v945: 178 pgs: 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 172 active+clean; 6.7 GiB data, 28 GiB used, 509 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 934 op/s; 22106/329574 objects misplaced (6.707%); 26 MiB/s, 415 objects/s recovering 2024-09-10T19:05:26.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:25 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:05:26.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:25 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:05:26.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:25 smithi018 ceph-mon[40196]: osdmap e427: 8 total, 8 up, 6 in 2024-09-10T19:05:26.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:25 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:05:26.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:25 smithi018 ceph-mon[40196]: pgmap v945: 178 pgs: 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 172 active+clean; 6.7 GiB data, 28 GiB used, 509 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 934 op/s; 22106/329574 objects misplaced (6.707%); 26 MiB/s, 415 objects/s recovering 2024-09-10T19:05:26.211 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:26.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:26.242 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:26.242 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:26.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:25 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:05:26.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:25 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:05:26.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:25 smithi103 ceph-mon[33163]: osdmap e427: 8 total, 8 up, 6 in 2024-09-10T19:05:26.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:25 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:05:26.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:25 smithi103 ceph-mon[33163]: pgmap v945: 178 pgs: 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 172 active+clean; 6.7 GiB data, 28 GiB used, 509 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 934 op/s; 22106/329574 objects misplaced (6.707%); 26 MiB/s, 415 objects/s recovering 2024-09-10T19:05:26.342 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:26.369 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:26.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:26.370 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:26.471 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:26.496 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:26.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:26.497 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:26.600 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:26.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:26.646 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:26.646 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:26.746 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:26.776 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:26.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:26.777 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:26.878 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:26.909 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:26.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:26.910 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:27.011 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:27.041 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:27.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:27.042 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:27.142 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:27.178 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:27.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:27.179 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:27.280 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:27.313 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:27.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:27.314 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:27.415 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:27.459 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:27.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:27.460 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:27.523 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:05:27 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:05:27.561 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:27.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:27.606 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:27.606 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:27.707 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:27.732 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:27.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:27.733 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:27.834 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:27.859 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:27.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:27.860 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:27.961 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:28.008 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:28.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:28.009 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:28.110 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:28.151 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:28.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:28.152 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:28.253 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:28.285 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:28.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:28.286 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:28.387 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:28.409 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:28.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:28.410 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:28.511 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:28.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:28.562 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:28.562 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:28.663 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:28.700 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:28.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:28.701 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:28.802 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:28.831 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:28.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:28.832 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:28.932 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:28.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:28 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:28.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:28 smithi018 ceph-mon[40196]: pgmap v946: 178 pgs: 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 172 active+clean; 6.4 GiB data, 27 GiB used, 509 GiB / 536 GiB avail; 28 KiB/s rd, 0 B/s wr, 1.53k op/s; 19352/314586 objects misplaced (6.152%); 40 MiB/s, 641 objects/s recovering 2024-09-10T19:05:28.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:28 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:28.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:28 smithi018 ceph-mon[29336]: pgmap v946: 178 pgs: 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 172 active+clean; 6.4 GiB data, 27 GiB used, 509 GiB / 536 GiB avail; 28 KiB/s rd, 0 B/s wr, 1.53k op/s; 19352/314586 objects misplaced (6.152%); 40 MiB/s, 641 objects/s recovering 2024-09-10T19:05:28.963 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:28.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:28.964 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:29.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:28 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:29.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:28 smithi103 ceph-mon[33163]: pgmap v946: 178 pgs: 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 172 active+clean; 6.4 GiB data, 27 GiB used, 509 GiB / 536 GiB avail; 28 KiB/s rd, 0 B/s wr, 1.53k op/s; 19352/314586 objects misplaced (6.152%); 40 MiB/s, 641 objects/s recovering 2024-09-10T19:05:29.065 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:29.100 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:29.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:29.101 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:29.202 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:29.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:29.238 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:29.238 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:29.339 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:29.383 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:29.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:29.384 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:29.484 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:29.523 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:29.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:29.524 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:29.625 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:29.670 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:29.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:29.672 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:29.773 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:29.793 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:29.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:29.794 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:29.895 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:29.921 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:29.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:29.922 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:30.023 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:30.054 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:30.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:30.055 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:30.156 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:30.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:29 smithi018 ceph-mon[40196]: osdmap e428: 8 total, 8 up, 6 in 2024-09-10T19:05:30.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:29 smithi018 ceph-mon[40196]: pgmap v948: 178 pgs: 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 172 active+clean; 6.3 GiB data, 27 GiB used, 509 GiB / 536 GiB avail; 39 KiB/s rd, 0 B/s wr, 1.27k op/s; 17691/311358 objects misplaced (5.682%); 37 MiB/s, 593 objects/s recovering 2024-09-10T19:05:30.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:29 smithi018 ceph-mon[29336]: osdmap e428: 8 total, 8 up, 6 in 2024-09-10T19:05:30.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:29 smithi018 ceph-mon[29336]: pgmap v948: 178 pgs: 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 172 active+clean; 6.3 GiB data, 27 GiB used, 509 GiB / 536 GiB avail; 39 KiB/s rd, 0 B/s wr, 1.27k op/s; 17691/311358 objects misplaced (5.682%); 37 MiB/s, 593 objects/s recovering 2024-09-10T19:05:30.196 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:30.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:30.197 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:30.238 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:29 smithi103 ceph-mon[33163]: osdmap e428: 8 total, 8 up, 6 in 2024-09-10T19:05:30.238 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:29 smithi103 ceph-mon[33163]: pgmap v948: 178 pgs: 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 172 active+clean; 6.3 GiB data, 27 GiB used, 509 GiB / 536 GiB avail; 39 KiB/s rd, 0 B/s wr, 1.27k op/s; 17691/311358 objects misplaced (5.682%); 37 MiB/s, 593 objects/s recovering 2024-09-10T19:05:30.298 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:30.338 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:30.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:30.339 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:30.440 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:30.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:30.466 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:30.466 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:30.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:05:30 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:05:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:05:30.567 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:30.597 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:30.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:30.598 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:30.699 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:30.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:30.722 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:30.722 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:30.823 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:30.869 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:30.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:30.870 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:30.971 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:31.013 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:31.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:31.015 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:31.115 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:31.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:31.147 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:31.147 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:31.244 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:05:30 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:05:30] "GET /metrics HTTP/1.1" 200 224476 "" "Prometheus/2.43.0" 2024-09-10T19:05:31.248 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:31.293 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:31.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:31.294 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:31.345 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:05:31.395 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:31.467 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:31.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:31.469 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:31.569 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:31.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:31.599 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:31.600 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:31.700 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:31.747 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:31.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:31.748 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:31.848 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:31.879 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:31.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:31.880 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:31.980 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:32.010 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:32.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:32.011 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:32.111 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:32.123 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:05:32.140 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:32.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:32.141 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:32.241 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:32.263 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:32.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:32.264 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:32.364 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:32.392 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:32.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:32.393 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:32.494 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:32.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:32.542 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:32.542 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:32.643 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:32.691 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:32.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:32.692 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:32.793 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:32.820 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:32.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:32.821 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:32.922 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:32.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:32 smithi018 ceph-mon[29336]: pgmap v949: 178 pgs: 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 172 active+clean; 6.1 GiB data, 27 GiB used, 510 GiB / 536 GiB avail; 47 KiB/s rd, 0 B/s wr, 1.39k op/s; 17197/302508 objects misplaced (5.685%); 29 MiB/s, 458 objects/s recovering 2024-09-10T19:05:32.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:32 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:05:32.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:32 smithi018 ceph-mon[40196]: pgmap v949: 178 pgs: 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 172 active+clean; 6.1 GiB data, 27 GiB used, 510 GiB / 536 GiB avail; 47 KiB/s rd, 0 B/s wr, 1.39k op/s; 17197/302508 objects misplaced (5.685%); 29 MiB/s, 458 objects/s recovering 2024-09-10T19:05:32.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:32 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:05:32.969 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:32.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:32.970 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:33.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:32 smithi103 ceph-mon[33163]: pgmap v949: 178 pgs: 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 172 active+clean; 6.1 GiB data, 27 GiB used, 510 GiB / 536 GiB avail; 47 KiB/s rd, 0 B/s wr, 1.39k op/s; 17197/302508 objects misplaced (5.685%); 29 MiB/s, 458 objects/s recovering 2024-09-10T19:05:33.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:32 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:05:33.071 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:33.111 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:33.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:33.112 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:33.212 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:33.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:33.242 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:33.242 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:33.245 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T19:05:33.343 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:33.366 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:33.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:33.369 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:33.470 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:33.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:33.518 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:33.518 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:33.619 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:33.644 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:33.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:33.645 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:33.746 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:33.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:33.772 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:33.772 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:33.873 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:33.911 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:33.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:33.912 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:33.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:33 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:05:33.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:33 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:05:33.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:33 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:05:33.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:33 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:05:33.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:33 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:05:33.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:33 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:05:34.013 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:34.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:33 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:05:34.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:33 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:05:34.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:33 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:05:34.041 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:34.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:34.042 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:34.143 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:34.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:34.166 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:34.167 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:34.267 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:34.310 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:34.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:34.311 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:34.411 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:34.439 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:34.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:34.441 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:34.541 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:34.570 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:34.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:34.571 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:34.672 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:34.701 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:34.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:34.703 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:34.803 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:34.833 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:34.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:34.834 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:34.884 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:34 smithi018 ceph-mon[40196]: pgmap v950: 178 pgs: 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 172 active+clean; 6.0 GiB data, 26 GiB used, 510 GiB / 536 GiB avail; 51 KiB/s rd, 0 B/s wr, 1.31k op/s; 15305/297600 objects misplaced (5.143%); 25 MiB/s, 405 objects/s recovering 2024-09-10T19:05:34.884 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:34 smithi018 ceph-mon[40196]: osdmap e429: 8 total, 8 up, 6 in 2024-09-10T19:05:34.884 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:34 smithi018 ceph-mon[40196]: osdmap e430: 8 total, 8 up, 6 in 2024-09-10T19:05:34.884 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:34 smithi018 ceph-mon[29336]: pgmap v950: 178 pgs: 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 172 active+clean; 6.0 GiB data, 26 GiB used, 510 GiB / 536 GiB avail; 51 KiB/s rd, 0 B/s wr, 1.31k op/s; 15305/297600 objects misplaced (5.143%); 25 MiB/s, 405 objects/s recovering 2024-09-10T19:05:34.884 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:34 smithi018 ceph-mon[29336]: osdmap e429: 8 total, 8 up, 6 in 2024-09-10T19:05:34.884 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:34 smithi018 ceph-mon[29336]: osdmap e430: 8 total, 8 up, 6 in 2024-09-10T19:05:34.935 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:34.965 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:34.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:34.966 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:35.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:34 smithi103 ceph-mon[33163]: pgmap v950: 178 pgs: 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 172 active+clean; 6.0 GiB data, 26 GiB used, 510 GiB / 536 GiB avail; 51 KiB/s rd, 0 B/s wr, 1.31k op/s; 15305/297600 objects misplaced (5.143%); 25 MiB/s, 405 objects/s recovering 2024-09-10T19:05:35.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:34 smithi103 ceph-mon[33163]: osdmap e429: 8 total, 8 up, 6 in 2024-09-10T19:05:35.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:34 smithi103 ceph-mon[33163]: osdmap e430: 8 total, 8 up, 6 in 2024-09-10T19:05:35.067 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:35.098 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:35.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:35.099 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:35.200 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:35.223 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:35.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:35.224 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:35.325 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:35.365 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:35.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:35.366 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:35.466 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:35.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:35.496 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:35.496 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:35.570 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is set 2024-09-10T19:05:35.596 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:35.636 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:35.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:35.637 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:35.738 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:35.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:35.786 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:35.787 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:35.888 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:35.919 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:35.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:35.920 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:35.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:35 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1441739377' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:05:35.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:35 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1441739377' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:05:36.021 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:36.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:35 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1441739377' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:05:36.055 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:36.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:36.056 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:36.157 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:36.187 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:36.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:36.189 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:36.290 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:36.321 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:36.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:36.322 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:36.422 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:36.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:36.451 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:36.451 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:36.552 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:36.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:36.601 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:36.601 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:36.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:36 smithi018 ceph-mon[29336]: pgmap v953: 178 pgs: 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 172 active+clean; 6.0 GiB data, 26 GiB used, 510 GiB / 536 GiB avail; 41 KiB/s rd, 0 B/s wr, 1.09k op/s; 13507/294090 objects misplaced (4.593%); 30 MiB/s, 486 objects/s recovering 2024-09-10T19:05:36.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:36 smithi018 ceph-mon[29336]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:05:36.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:36 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1441739377' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T19:05:36.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:36 smithi018 ceph-mon[29336]: osdmap e431: 8 total, 8 up, 6 in 2024-09-10T19:05:36.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:36 smithi018 ceph-mon[40196]: pgmap v953: 178 pgs: 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 172 active+clean; 6.0 GiB data, 26 GiB used, 510 GiB / 536 GiB avail; 41 KiB/s rd, 0 B/s wr, 1.09k op/s; 13507/294090 objects misplaced (4.593%); 30 MiB/s, 486 objects/s recovering 2024-09-10T19:05:36.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:36 smithi018 ceph-mon[40196]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:05:36.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:36 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1441739377' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T19:05:36.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:36 smithi018 ceph-mon[40196]: osdmap e431: 8 total, 8 up, 6 in 2024-09-10T19:05:36.701 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:36.732 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:36.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:36.734 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:36.834 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:36.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:36.867 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:36.867 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:36.968 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:36.990 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:36.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:36.992 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:37.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:36 smithi103 ceph-mon[33163]: pgmap v953: 178 pgs: 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 172 active+clean; 6.0 GiB data, 26 GiB used, 510 GiB / 536 GiB avail; 41 KiB/s rd, 0 B/s wr, 1.09k op/s; 13507/294090 objects misplaced (4.593%); 30 MiB/s, 486 objects/s recovering 2024-09-10T19:05:37.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:36 smithi103 ceph-mon[33163]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:05:37.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:36 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1441739377' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T19:05:37.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:36 smithi103 ceph-mon[33163]: osdmap e431: 8 total, 8 up, 6 in 2024-09-10T19:05:37.092 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:37.138 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:37.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:37.139 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:37.240 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:37.262 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:37.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:37.263 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:37.364 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:37.403 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:37.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:37.404 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:37.504 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:37.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:05:37 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:05:37.537 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:37.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:37.539 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:37.639 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:37.674 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:37.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:37.675 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:37.776 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:37.817 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:37 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:05:37.817 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:37 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:05:37.817 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:37 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:05:37.817 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:37 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:05:37.818 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:37.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:37.820 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:37.921 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:37.955 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:37.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:37.956 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:38.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:37 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:05:38.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:37 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:05:38.057 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:38.082 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:38.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:38.084 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:38.173 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:05:38.184 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:38.207 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:38.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:38.208 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:38.309 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:38.331 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:38.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:38.332 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:38.433 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:38.455 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:38.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:38.456 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:38.556 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:38.581 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:38.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:38.582 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:38.683 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:38.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:38 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:38.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:38 smithi018 ceph-mon[29336]: pgmap v955: 178 pgs: 3 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 173 active+clean; 5.7 GiB data, 25 GiB used, 511 GiB / 536 GiB avail; 14 KiB/s rd, 0 B/s wr, 1.34k op/s; 11625/278781 objects misplaced (4.170%); 31 MiB/s, 488 objects/s recovering 2024-09-10T19:05:38.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:38 smithi018 ceph-mon[29336]: osdmap e432: 8 total, 8 up, 6 in 2024-09-10T19:05:38.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:38 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:38.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:38 smithi018 ceph-mon[40196]: pgmap v955: 178 pgs: 3 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 173 active+clean; 5.7 GiB data, 25 GiB used, 511 GiB / 536 GiB avail; 14 KiB/s rd, 0 B/s wr, 1.34k op/s; 11625/278781 objects misplaced (4.170%); 31 MiB/s, 488 objects/s recovering 2024-09-10T19:05:38.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:38 smithi018 ceph-mon[40196]: osdmap e432: 8 total, 8 up, 6 in 2024-09-10T19:05:38.713 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:38.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:38.714 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:38.814 INFO:teuthology.orchestra.run.smithi018.stderr:reweighted osd.1 to 1 (10000) 2024-09-10T19:05:38.815 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:38.840 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:38.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:38.841 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:38.942 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:38.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:38.981 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:38.981 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:39.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:38 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:39.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:38 smithi103 ceph-mon[33163]: pgmap v955: 178 pgs: 3 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 173 active+clean; 5.7 GiB data, 25 GiB used, 511 GiB / 536 GiB avail; 14 KiB/s rd, 0 B/s wr, 1.34k op/s; 11625/278781 objects misplaced (4.170%); 31 MiB/s, 488 objects/s recovering 2024-09-10T19:05:39.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:38 smithi103 ceph-mon[33163]: osdmap e432: 8 total, 8 up, 6 in 2024-09-10T19:05:39.082 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:39.121 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:39.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:39.122 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:39.223 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:39.266 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:39.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:39.267 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:39.368 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:39.405 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:39.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:39.407 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:39.508 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:39.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:39.548 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:39.548 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:39.649 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:39.687 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:39.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:39.688 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:39.789 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:39.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:39.817 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:39.817 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:39.918 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:39.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:39 smithi018 ceph-mon[40196]: osdmap e433: 8 total, 8 up, 6 in 2024-09-10T19:05:39.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:39 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/4158123106' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-10T19:05:39.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:39 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-10T19:05:39.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:39 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-10T19:05:39.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:39 smithi018 ceph-mon[40196]: osdmap e434: 8 total, 8 up, 6 in 2024-09-10T19:05:39.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:39 smithi018 ceph-mon[29336]: osdmap e433: 8 total, 8 up, 6 in 2024-09-10T19:05:39.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:39 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/4158123106' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-10T19:05:39.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:39 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-10T19:05:39.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:39 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-10T19:05:39.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:39 smithi018 ceph-mon[29336]: osdmap e434: 8 total, 8 up, 6 in 2024-09-10T19:05:39.951 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:39.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:39.953 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:40.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:39 smithi103 ceph-mon[33163]: osdmap e433: 8 total, 8 up, 6 in 2024-09-10T19:05:40.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:39 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/4158123106' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-10T19:05:40.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:39 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-10T19:05:40.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:39 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-10T19:05:40.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:39 smithi103 ceph-mon[33163]: osdmap e434: 8 total, 8 up, 6 in 2024-09-10T19:05:40.054 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:40.088 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:40.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:40.090 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:40.191 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:40.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:40.235 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:40.236 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:40.337 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:40.371 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:40.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:40.372 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:40.473 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:40.503 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:40.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:40.505 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:40.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:05:40 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:05:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:05:40.606 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:40.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:40.649 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:40.649 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:40.750 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:40.772 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:40.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:40.773 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:40.874 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:40.902 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:40.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:40.903 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:40.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:40 smithi018 ceph-mon[40196]: pgmap v959: 178 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 175 active+clean; 5.6 GiB data, 25 GiB used, 511 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.56k op/s; 8751/275307 objects misplaced (3.179%); 35 MiB/s, 555 objects/s recovering 2024-09-10T19:05:40.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:40 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:05:40.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:40 smithi018 ceph-mon[29336]: pgmap v959: 178 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 175 active+clean; 5.6 GiB data, 25 GiB used, 511 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.56k op/s; 8751/275307 objects misplaced (3.179%); 35 MiB/s, 555 objects/s recovering 2024-09-10T19:05:40.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:40 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:05:41.004 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:41.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:40 smithi103 ceph-mon[33163]: pgmap v959: 178 pgs: 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 175 active+clean; 5.6 GiB data, 25 GiB used, 511 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.56k op/s; 8751/275307 objects misplaced (3.179%); 35 MiB/s, 555 objects/s recovering 2024-09-10T19:05:41.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:40 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:05:41.037 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:41.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:41.038 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:41.139 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:41.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:41.169 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:41.169 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:41.271 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:41.295 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:41.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:41.296 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:41.396 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:41.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:41.419 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:41.419 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:41.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:05:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:05:40] "GET /metrics HTTP/1.1" 200 224518 "" "Prometheus/2.43.0" 2024-09-10T19:05:41.520 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:41.546 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:41.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:41.547 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:41.647 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:41.680 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:41.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:41.681 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:41.782 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:41.809 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:41.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:41.810 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:41.911 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:41.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:41 smithi018 ceph-mon[29336]: osdmap e435: 8 total, 8 up, 6 in 2024-09-10T19:05:41.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:41 smithi018 ceph-mon[40196]: osdmap e435: 8 total, 8 up, 6 in 2024-09-10T19:05:41.952 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:41.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:41.954 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:42.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:41 smithi103 ceph-mon[33163]: osdmap e435: 8 total, 8 up, 6 in 2024-09-10T19:05:42.056 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:42.093 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:42.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:42.095 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:42.196 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:42.228 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:42.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:42.229 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:42.330 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:42.369 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:42.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:42.370 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:42.470 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:42.512 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:42.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:42.513 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:42.614 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:42.652 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:42.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:42.653 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:42.754 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:42.781 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:42.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:42.782 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:42.805 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd reweight 3 1 2024-09-10T19:05:42.847 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:42 smithi018 ceph-mon[40196]: pgmap v961: 178 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 175 active+clean; 5.4 GiB data, 25 GiB used, 512 GiB / 536 GiB avail; 0 B/s wr, 1.08k op/s; 7662/265851 objects misplaced (2.882%); 29 MiB/s, 460 objects/s recovering 2024-09-10T19:05:42.847 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:42 smithi018 ceph-mon[40196]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-10T19:05:42.847 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:42 smithi018 ceph-mon[40196]: osdmap e436: 8 total, 8 up, 6 in 2024-09-10T19:05:42.847 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:42 smithi018 ceph-mon[29336]: pgmap v961: 178 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 175 active+clean; 5.4 GiB data, 25 GiB used, 512 GiB / 536 GiB avail; 0 B/s wr, 1.08k op/s; 7662/265851 objects misplaced (2.882%); 29 MiB/s, 460 objects/s recovering 2024-09-10T19:05:42.847 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:42 smithi018 ceph-mon[29336]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-10T19:05:42.847 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:42 smithi018 ceph-mon[29336]: osdmap e436: 8 total, 8 up, 6 in 2024-09-10T19:05:42.883 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:42.908 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:42.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:42.909 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:43.010 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:43.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:42 smithi103 ceph-mon[33163]: pgmap v961: 178 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 175 active+clean; 5.4 GiB data, 25 GiB used, 512 GiB / 536 GiB avail; 0 B/s wr, 1.08k op/s; 7662/265851 objects misplaced (2.882%); 29 MiB/s, 460 objects/s recovering 2024-09-10T19:05:43.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:42 smithi103 ceph-mon[33163]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-10T19:05:43.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:42 smithi103 ceph-mon[33163]: osdmap e436: 8 total, 8 up, 6 in 2024-09-10T19:05:43.052 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:43.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:43.053 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:43.154 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:43.183 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:43.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:43.184 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:43.285 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:43.316 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:43.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:43.317 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:43.417 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:43.461 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:43.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:43.462 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:43.562 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:43.603 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:43.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:43.604 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:43.705 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:43.732 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:43.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:43.733 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:43.834 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:43.867 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:43.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:43.868 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:43.969 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:43.995 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:43.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:43.996 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:44.097 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:44.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:44.145 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:44.146 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:44.246 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:44.280 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:44.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:44.281 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:44.382 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:44.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:44.420 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:44.420 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:44.521 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:44.554 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:44.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:44.555 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:44.656 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:44.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:44 smithi018 ceph-mon[29336]: pgmap v963: 178 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 176 active+clean; 5.3 GiB data, 24 GiB used, 512 GiB / 536 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.18k op/s; 7532/261327 objects misplaced (2.882%); 25 MiB/s, 392 objects/s recovering 2024-09-10T19:05:44.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:44 smithi018 ceph-mon[40196]: pgmap v963: 178 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 176 active+clean; 5.3 GiB data, 24 GiB used, 512 GiB / 536 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.18k op/s; 7532/261327 objects misplaced (2.882%); 25 MiB/s, 392 objects/s recovering 2024-09-10T19:05:44.702 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:44.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:44.703 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:44.804 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:44.841 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:44.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:44.843 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:44.944 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:44.974 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:44.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:44.975 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:45.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:44 smithi103 ceph-mon[33163]: pgmap v963: 178 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 176 active+clean; 5.3 GiB data, 24 GiB used, 512 GiB / 536 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.18k op/s; 7532/261327 objects misplaced (2.882%); 25 MiB/s, 392 objects/s recovering 2024-09-10T19:05:45.076 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:45.124 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:45.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:45.125 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:45.226 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:45.255 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:45.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:45.257 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:45.272 INFO:teuthology.orchestra.run.smithi018.stdout:ERROR: (22) Invalid argument 2024-09-10T19:05:45.272 INFO:teuthology.orchestra.run.smithi018.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T19:05:45.358 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:45.397 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:45.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:45.398 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:45.435 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-10T19:05:45.499 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:45.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:45.559 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:45.559 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:45.660 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:45.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:45.689 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:45.689 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:45.789 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:45.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:45.831 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:45.831 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:45.932 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:45.970 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:45.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:45.971 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:46.072 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:46.098 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:46.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:46.100 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:46.201 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:46.239 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:46.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:46.240 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:46.340 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:46.363 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:46.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:46.364 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:46.465 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:46.500 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:46.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:46.502 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:46.603 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:46.630 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:46.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:46.631 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:46.732 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:46.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:46.765 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:46.765 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:46.866 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:46.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:46.905 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:46.905 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:46.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:46 smithi018 ceph-mon[40196]: pgmap v964: 178 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 176 active+clean; 5.2 GiB data, 24 GiB used, 512 GiB / 536 GiB avail; 765 B/s rd, 0 B/s wr, 1.08k op/s; 7532/257145 objects misplaced (2.929%); 18 MiB/s, 288 objects/s recovering 2024-09-10T19:05:46.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:46 smithi018 ceph-mon[29336]: pgmap v964: 178 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 176 active+clean; 5.2 GiB data, 24 GiB used, 512 GiB / 536 GiB avail; 765 B/s rd, 0 B/s wr, 1.08k op/s; 7532/257145 objects misplaced (2.929%); 18 MiB/s, 288 objects/s recovering 2024-09-10T19:05:47.006 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:47.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:46 smithi103 ceph-mon[33163]: pgmap v964: 178 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 176 active+clean; 5.2 GiB data, 24 GiB used, 512 GiB / 536 GiB avail; 765 B/s rd, 0 B/s wr, 1.08k op/s; 7532/257145 objects misplaced (2.929%); 18 MiB/s, 288 objects/s recovering 2024-09-10T19:05:47.033 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:47.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:47.034 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:47.135 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:47.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:47.167 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:47.168 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:47.268 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:47.301 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:47.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:47.302 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:47.403 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:47.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:47.436 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:47.437 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:47.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:05:47 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:05:47.538 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:47.571 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:47.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:47.572 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:47.673 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:47.705 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:47.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:47.706 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:47.807 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:47.850 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:47.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:47.851 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:47.951 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:47.985 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:47.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:47.986 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:48.087 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:48.122 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:48.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:48.123 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:48.223 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:48.252 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:48.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:48.253 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:48.354 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:48.375 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:48.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:48.376 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:48.477 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:48.511 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:48.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:48.513 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:48.613 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:48.659 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:48.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:48.660 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:48.668 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-10T19:05:48.761 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:48.797 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:48.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:48.798 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:48.891 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:48 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:48.891 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:48 smithi018 ceph-mon[29336]: pgmap v965: 178 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 176 active+clean; 4.8 GiB data, 23 GiB used, 513 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.54k op/s; 5484/238323 objects misplaced (2.301%); 8.2 MiB/s, 131 objects/s recovering 2024-09-10T19:05:48.891 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:48 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:48.891 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:48 smithi018 ceph-mon[40196]: pgmap v965: 178 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 176 active+clean; 4.8 GiB data, 23 GiB used, 513 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.54k op/s; 5484/238323 objects misplaced (2.301%); 8.2 MiB/s, 131 objects/s recovering 2024-09-10T19:05:48.898 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:48.944 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:48.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:48.945 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:49.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:48 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:49.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:48 smithi103 ceph-mon[33163]: pgmap v965: 178 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 176 active+clean; 4.8 GiB data, 23 GiB used, 513 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.54k op/s; 5484/238323 objects misplaced (2.301%); 8.2 MiB/s, 131 objects/s recovering 2024-09-10T19:05:49.028 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:05:49.046 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:49.047 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:05:49.091 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:05:49.098 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T19:05:49.099 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T19:05:49.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:49.102 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:49.102 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:49.132 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T19:05:49.133 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T19:05:49.168 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T19:05:49.169 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T19:05:49.203 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:49.204 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T19:05:49.204 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T19:05:49.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:49.237 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:49.237 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:49.241 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T19:05:49.241 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T19:05:49.276 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T19:05:49.276 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T19:05:49.307 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T19:05:49.307 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T19:05:49.337 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:49.338 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T19:05:49.339 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T19:05:49.369 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:49.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:49.371 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:49.471 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:49.512 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:49.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:49.513 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:49.614 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:49.659 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:49.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:49.660 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:49.761 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:49.797 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:49.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:49.798 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:49.899 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:49.936 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:49.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:49.937 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:49.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:49 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/305892255' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:05:49.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:49 smithi018 ceph-mon[29336]: pgmap v966: 178 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 176 active+clean; 4.8 GiB data, 23 GiB used, 514 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.54k op/s; 5484/234165 objects misplaced (2.342%); 7.4 MiB/s, 118 objects/s recovering 2024-09-10T19:05:49.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:49 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/305892255' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:05:49.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:49 smithi018 ceph-mon[40196]: pgmap v966: 178 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 176 active+clean; 4.8 GiB data, 23 GiB used, 514 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.54k op/s; 5484/234165 objects misplaced (2.342%); 7.4 MiB/s, 118 objects/s recovering 2024-09-10T19:05:50.038 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:50.078 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:50.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:50.079 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:50.180 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:50.219 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:50.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:50.220 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:50.235 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:49 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/305892255' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:05:50.235 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:49 smithi103 ceph-mon[33163]: pgmap v966: 178 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 176 active+clean; 4.8 GiB data, 23 GiB used, 514 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.54k op/s; 5484/234165 objects misplaced (2.342%); 7.4 MiB/s, 118 objects/s recovering 2024-09-10T19:05:50.321 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:50.365 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:50.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:50.366 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:50.466 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:50.497 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:50.497 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:05:50 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:05:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:05:50.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:50.498 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:50.600 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:50.639 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:50.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:50.640 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:50.741 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:50.770 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:50.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:50.771 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:50.872 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:50.911 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:50.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:50.912 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:50.982 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T19:05:50.982 INFO:teuthology.orchestra.run.smithi103.stdout: "ops": [ 2024-09-10T19:05:50.982 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:50.982 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.25706.0:202668 11.e 11:7dc5b174:::benchmark_data_smithi018_181485_object65576:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e436)", 2024-09-10T19:05:50.982 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:05:50.966935+0000", 2024-09-10T19:05:50.982 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.012845766, 2024-09-10T19:05:50.982 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.012875361, 2024-09-10T19:05:50.983 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:05:50.983 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "waiting for sub ops", 2024-09-10T19:05:50.983 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:05:50.983 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.25706", 2024-09-10T19:05:50.983 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/42081399", 2024-09-10T19:05:50.983 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 202668 2024-09-10T19:05:50.983 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:50.983 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:05:50.983 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:50.983 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:05:50.983 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.966935+0000", 2024-09-10T19:05:50.983 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:50.984 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:50.984 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:50.984 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:05:50.984 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.966935+0000", 2024-09-10T19:05:50.984 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:50.984 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:50.984 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:50.984 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:05:50.984 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.966934+0000", 2024-09-10T19:05:50.984 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T19:05:50.984 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:50.984 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:50.985 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:05:50.985 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.966939+0000", 2024-09-10T19:05:50.985 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.7979999999999996e-06 2024-09-10T19:05:50.985 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:50.985 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:50.985 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:05:50.985 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.966948+0000", 2024-09-10T19:05:50.985 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.8599999999999999e-06 2024-09-10T19:05:50.985 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:50.985 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:50.985 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:05:50.986 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.966953+0000", 2024-09-10T19:05:50.986 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.0390000000000004e-06 2024-09-10T19:05:50.986 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:50.986 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:50.986 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:50.986 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.973951+0000", 2024-09-10T19:05:50.986 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0069980219999999996 2024-09-10T19:05:50.986 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:50.986 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:50.986 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:05:50.986 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.974047+0000", 2024-09-10T19:05:50.986 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.6367999999999996e-05 2024-09-10T19:05:50.987 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:50.987 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:50.987 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1", 2024-09-10T19:05:50.987 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.974162+0000", 2024-09-10T19:05:50.987 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000114475 2024-09-10T19:05:50.987 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:50.987 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:50.987 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:50.987 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.976502+0000", 2024-09-10T19:05:50.987 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0023405409999999998 2024-09-10T19:05:50.987 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:50.987 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:50.988 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:05:50.988 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.977092+0000", 2024-09-10T19:05:50.988 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000590074 2024-09-10T19:05:50.988 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:50.988 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:05:50.988 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:50.988 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:50.988 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:50.988 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.25706.0:202675 11.c 11:3d7002d7:::benchmark_data_smithi018_181485_object65583:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e436)", 2024-09-10T19:05:50.988 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:05:50.974939+0000", 2024-09-10T19:05:50.988 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.004842318, 2024-09-10T19:05:50.988 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0049697329999999996, 2024-09-10T19:05:50.989 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:05:50.989 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "waiting for sub ops", 2024-09-10T19:05:50.989 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:05:50.989 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.25706", 2024-09-10T19:05:50.989 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/42081399", 2024-09-10T19:05:50.989 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 202675 2024-09-10T19:05:50.989 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:50.989 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:05:50.989 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:50.989 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:05:50.989 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.974939+0000", 2024-09-10T19:05:50.989 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:50.990 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:50.990 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:50.990 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:05:50.990 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.974939+0000", 2024-09-10T19:05:50.990 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:50.990 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:50.990 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:50.990 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:05:50.990 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.974936+0000", 2024-09-10T19:05:50.990 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999976 2024-09-10T19:05:50.990 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:50.990 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:50.990 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:05:50.991 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.974944+0000", 2024-09-10T19:05:50.991 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.7910000000000005e-06 2024-09-10T19:05:50.991 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:50.991 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:50.991 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:05:50.991 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.974959+0000", 2024-09-10T19:05:50.991 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.4538e-05 2024-09-10T19:05:50.991 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:50.991 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:50.991 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:05:50.991 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.974964+0000", 2024-09-10T19:05:50.991 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.7610000000000004e-06 2024-09-10T19:05:50.991 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:50.992 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:50.992 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:50.992 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.977280+0000", 2024-09-10T19:05:50.992 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0023155850000000002 2024-09-10T19:05:50.992 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:50.992 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:50.992 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:05:50.992 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.977373+0000", 2024-09-10T19:05:50.992 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.2513e-05 2024-09-10T19:05:50.992 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:50.992 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:50.992 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 1,5", 2024-09-10T19:05:50.993 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.977469+0000", 2024-09-10T19:05:50.993 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.6841000000000003e-05 2024-09-10T19:05:50.993 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:50.993 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:05:50.993 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:50.993 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:50.993 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:50.993 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.25706.0:202677 11.18 11:19f83e2c:::benchmark_data_smithi018_181485_object65585:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e436)", 2024-09-10T19:05:50.993 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:05:50.974974+0000", 2024-09-10T19:05:50.993 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.004806767, 2024-09-10T19:05:50.993 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0050262639999999999, 2024-09-10T19:05:50.993 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:05:50.993 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "waiting for sub ops", 2024-09-10T19:05:50.994 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:05:50.994 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.25706", 2024-09-10T19:05:50.994 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/42081399", 2024-09-10T19:05:50.994 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 202677 2024-09-10T19:05:50.994 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:50.994 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:05:50.994 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:50.994 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:05:50.994 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.974974+0000", 2024-09-10T19:05:50.994 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:50.994 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:50.994 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:50.994 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:05:50.995 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.974974+0000", 2024-09-10T19:05:50.995 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:50.995 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:50.995 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:50.995 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:05:50.995 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.974974+0000", 2024-09-10T19:05:50.995 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999995 2024-09-10T19:05:50.995 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:50.995 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:50.995 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:05:50.995 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.974977+0000", 2024-09-10T19:05:50.995 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.05e-06 2024-09-10T19:05:50.995 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:50.996 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:50.996 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:05:50.996 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.974981+0000", 2024-09-10T19:05:50.996 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.6800000000000001e-06 2024-09-10T19:05:50.996 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:50.996 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:50.996 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:05:50.996 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.974985+0000", 2024-09-10T19:05:50.996 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.5420000000000001e-06 2024-09-10T19:05:50.996 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:50.996 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:50.996 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:50.996 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.975045+0000", 2024-09-10T19:05:50.997 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.0220000000000003e-05 2024-09-10T19:05:50.997 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:50.997 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:50.997 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:05:50.997 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.975147+0000", 2024-09-10T19:05:50.997 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000101486 2024-09-10T19:05:50.997 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:50.997 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:50.997 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 5,6", 2024-09-10T19:05:50.997 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.975477+0000", 2024-09-10T19:05:50.998 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00033034900000000002 2024-09-10T19:05:50.998 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:50.998 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:50.998 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:05:50.998 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.977580+0000", 2024-09-10T19:05:50.998 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0021025570000000001 2024-09-10T19:05:50.998 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:50.998 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:05:50.998 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:50.999 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:50.999 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:50.999 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.25706.0:202678 11.d 11:b525d806:::benchmark_data_smithi018_181485_object65586:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e436)", 2024-09-10T19:05:50.999 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:05:50.976858+0000", 2024-09-10T19:05:50.999 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0029227889999999999, 2024-09-10T19:05:50.999 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0032182809999999999, 2024-09-10T19:05:50.999 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:05:50.999 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "waiting for sub ops", 2024-09-10T19:05:50.999 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:05:50.999 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.25706", 2024-09-10T19:05:50.999 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/42081399", 2024-09-10T19:05:50.999 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 202678 2024-09-10T19:05:50.999 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:51.000 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:05:51.000 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:51.000 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:05:51.000 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.976858+0000", 2024-09-10T19:05:51.000 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:51.000 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:51.000 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:51.000 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:05:51.000 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.976858+0000", 2024-09-10T19:05:51.000 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:51.000 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:51.000 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:51.000 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:05:51.001 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.976857+0000", 2024-09-10T19:05:51.001 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T19:05:51.001 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:51.001 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:51.001 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:05:51.001 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.976863+0000", 2024-09-10T19:05:51.001 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.8669999999999999e-06 2024-09-10T19:05:51.001 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:51.001 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:51.001 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:05:51.001 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.976879+0000", 2024-09-10T19:05:51.001 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.649e-05 2024-09-10T19:05:51.001 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:51.002 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:51.002 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:05:51.002 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.976893+0000", 2024-09-10T19:05:51.002 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.3886999999999999e-05 2024-09-10T19:05:51.002 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:51.002 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:51.002 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:51.002 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.978137+0000", 2024-09-10T19:05:51.002 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0012437329999999999 2024-09-10T19:05:51.002 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:51.002 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:51.002 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:05:51.003 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.978242+0000", 2024-09-10T19:05:51.003 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00010446300000000001 2024-09-10T19:05:51.003 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:51.003 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:51.003 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 1,6", 2024-09-10T19:05:51.003 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.978324+0000", 2024-09-10T19:05:51.003 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.2168000000000003e-05 2024-09-10T19:05:51.003 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:51.003 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:05:51.003 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:51.003 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:51.003 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:51.004 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.25706.0:202680 11.c 11.f0b2d0dc (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e436)", 2024-09-10T19:05:51.004 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:05:50.977955+0000", 2024-09-10T19:05:51.004 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.001826502, 2024-09-10T19:05:51.004 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.002186038, 2024-09-10T19:05:51.004 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:05:51.004 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "queued for pg", 2024-09-10T19:05:51.004 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:05:51.004 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.25706", 2024-09-10T19:05:51.004 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/42081399", 2024-09-10T19:05:51.004 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 202680 2024-09-10T19:05:51.004 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:51.005 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:05:51.005 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:51.005 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:05:51.005 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.977955+0000", 2024-09-10T19:05:51.005 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:51.005 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:51.005 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:51.007 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:05:51.008 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.977955+0000", 2024-09-10T19:05:51.008 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:51.008 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:51.008 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:51.008 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:05:51.008 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.977954+0000", 2024-09-10T19:05:51.008 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999995 2024-09-10T19:05:51.008 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:51.008 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:51.008 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:05:51.008 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.977959+0000", 2024-09-10T19:05:51.008 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.8069999999999999e-06 2024-09-10T19:05:51.009 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:51.009 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:51.009 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:05:51.009 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.977972+0000", 2024-09-10T19:05:51.009 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.3423e-05 2024-09-10T19:05:51.009 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:51.009 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:51.009 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:05:51.009 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.977978+0000", 2024-09-10T19:05:51.010 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.0279999999999997e-06 2024-09-10T19:05:51.010 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:51.010 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:05:51.010 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:51.010 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:51.010 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:51.010 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.25706.0:202681 11.18 11:1fd7bdd5:::benchmark_data_smithi018_181485_object65589:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e436)", 2024-09-10T19:05:51.010 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:05:50.977988+0000", 2024-09-10T19:05:51.010 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.001792887, 2024-09-10T19:05:51.010 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0022043119999999999, 2024-09-10T19:05:51.010 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:05:51.010 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "waiting for sub ops", 2024-09-10T19:05:51.011 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:05:51.011 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.25706", 2024-09-10T19:05:51.011 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/42081399", 2024-09-10T19:05:51.011 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 202681 2024-09-10T19:05:51.011 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:51.011 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:05:51.011 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:51.011 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:05:51.011 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.977988+0000", 2024-09-10T19:05:51.011 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:51.012 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:51.012 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:51.012 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:05:51.012 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.977988+0000", 2024-09-10T19:05:51.012 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:51.012 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:51.012 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:51.012 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:05:51.012 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.977988+0000", 2024-09-10T19:05:51.012 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999995 2024-09-10T19:05:51.012 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:51.013 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:51.013 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:05:51.013 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.977991+0000", 2024-09-10T19:05:51.013 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.033e-06 2024-09-10T19:05:51.013 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:51.013 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:51.013 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:05:51.013 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.977997+0000", 2024-09-10T19:05:51.013 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.9340000000000003e-06 2024-09-10T19:05:51.013 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:51.013 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:51.013 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:05:51.014 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.978001+0000", 2024-09-10T19:05:51.014 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.9750000000000001e-06 2024-09-10T19:05:51.014 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:51.014 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:51.014 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:51.014 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.978032+0000", 2024-09-10T19:05:51.014 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.0922e-05 2024-09-10T19:05:51.014 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:51.014 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:51.014 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:05:51.015 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.978136+0000", 2024-09-10T19:05:51.015 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00010433900000000001 2024-09-10T19:05:51.015 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:51.015 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:51.015 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 5,6", 2024-09-10T19:05:51.015 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.978448+0000", 2024-09-10T19:05:51.015 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00031176400000000002 2024-09-10T19:05:51.015 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:51.015 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:05:51.015 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:51.015 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:51.015 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:51.016 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_repop(client.25706.0:202679 11.14 e436/413 11:286d8cf2:::benchmark_data_smithi018_181485_object65587:head v 436'9990, mlcod=436'9988)", 2024-09-10T19:05:51.016 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:05:50.978061+0000", 2024-09-10T19:05:51.016 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0017199209999999999, 2024-09-10T19:05:51.016 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0022031609999999999, 2024-09-10T19:05:51.016 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:05:51.016 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "started", 2024-09-10T19:05:51.016 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:05:51.016 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:51.016 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:05:51.016 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.978061+0000", 2024-09-10T19:05:51.016 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:51.017 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:51.017 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:51.017 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:05:51.017 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.978061+0000", 2024-09-10T19:05:51.017 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:51.017 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:51.017 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:51.017 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:05:51.017 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.978061+0000", 2024-09-10T19:05:51.018 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999995 2024-09-10T19:05:51.018 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:51.018 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:51.018 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:05:51.018 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.978066+0000", 2024-09-10T19:05:51.018 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.3770000000000004e-06 2024-09-10T19:05:51.018 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:51.018 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:51.018 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:05:51.018 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.978072+0000", 2024-09-10T19:05:51.018 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.392e-06 2024-09-10T19:05:51.018 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:51.018 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:51.019 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:05:51.019 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.978077+0000", 2024-09-10T19:05:51.019 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.4009999999999999e-06 2024-09-10T19:05:51.019 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:51.019 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:51.019 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:51.019 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.979396+0000", 2024-09-10T19:05:51.019 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0013195640000000001 2024-09-10T19:05:51.019 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:51.019 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:51.019 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:05:51.019 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:50.979413+0000", 2024-09-10T19:05:51.019 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.6545000000000002e-05 2024-09-10T19:05:51.020 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:51.020 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:05:51.020 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:51.020 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:51.020 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-10T19:05:51.020 INFO:teuthology.orchestra.run.smithi103.stdout: "num_ops": 7 2024-09-10T19:05:51.020 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T19:05:51.020 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:51.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:51.067 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:51.067 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:51.156 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:05:50 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:05:50] "GET /metrics HTTP/1.1" 200 224518 "" "Prometheus/2.43.0" 2024-09-10T19:05:51.168 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:51.201 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:51.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:51.202 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:51.303 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:51.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:51.342 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:51.342 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:51.443 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:51.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:51.464 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:51.465 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:51.565 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:51.605 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:51.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:51.606 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:51.707 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:51.746 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:51.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:51.747 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:51.848 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:51.885 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:51.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:51.886 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:51.905 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-10T19:05:51.987 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:52.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:52.044 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:52.044 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:52.145 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:52.186 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:52.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:52.187 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:52.288 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:52.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:52.338 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:52.338 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:52.397 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:05:52.439 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:52.475 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:52.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:52.476 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:52.576 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:52.599 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:52.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:52.600 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:52.701 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:52.726 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:52.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:52.727 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:52.728 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:52 smithi103 ceph-mon[33163]: pgmap v967: 178 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 176 active+clean; 4.5 GiB data, 22 GiB used, 514 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.48k op/s; 3916/221412 objects misplaced (1.769%); 8.4 MiB/s, 134 objects/s recovering 2024-09-10T19:05:52.826 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:52 smithi018 ceph-mon[40196]: pgmap v967: 178 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 176 active+clean; 4.5 GiB data, 22 GiB used, 514 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.48k op/s; 3916/221412 objects misplaced (1.769%); 8.4 MiB/s, 134 objects/s recovering 2024-09-10T19:05:52.827 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:52 smithi018 ceph-mon[29336]: pgmap v967: 178 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 176 active+clean; 4.5 GiB data, 22 GiB used, 514 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.48k op/s; 3916/221412 objects misplaced (1.769%); 8.4 MiB/s, 134 objects/s recovering 2024-09-10T19:05:52.828 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:52.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:52.851 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:52.851 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:52.952 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:52.988 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:53.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:53.053 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:53.154 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:53.191 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:53.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:53.192 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:53.293 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:53.335 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:53.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:53.336 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:53.437 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:53.476 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:53.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:53.477 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:53.578 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:53.624 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:53.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:53.625 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:53.725 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:53.761 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:53.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:53.762 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:53.863 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:53.895 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:53.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:53.896 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:53.997 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:54.041 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:54.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:54.042 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:54.143 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:54.172 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:54.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:54.174 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:54.275 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:54.309 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:54.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:54.310 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:54.411 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:54.449 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:54.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:54.450 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:54.551 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:54.583 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:54.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:54.584 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:54.685 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:54.719 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:54.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:54.720 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:54.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:54 smithi103 ceph-mon[33163]: pgmap v968: 178 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 176 active+clean; 4.4 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.42k op/s; 3916/215040 objects misplaced (1.821%); 7.1 MiB/s, 113 objects/s recovering 2024-09-10T19:05:54.820 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:54.827 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T19:05:54.854 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:54.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:54.855 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:54.870 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:54 smithi018 ceph-mon[29336]: pgmap v968: 178 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 176 active+clean; 4.4 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.42k op/s; 3916/215040 objects misplaced (1.821%); 7.1 MiB/s, 113 objects/s recovering 2024-09-10T19:05:54.871 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:54 smithi018 ceph-mon[40196]: pgmap v968: 178 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 176 active+clean; 4.4 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.42k op/s; 3916/215040 objects misplaced (1.821%); 7.1 MiB/s, 113 objects/s recovering 2024-09-10T19:05:54.955 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:54.987 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T19:05:54.987 INFO:teuthology.orchestra.run.smithi103.stdout: "ops": [], 2024-09-10T19:05:54.987 INFO:teuthology.orchestra.run.smithi103.stdout: "complaint_time": 30, 2024-09-10T19:05:54.988 INFO:teuthology.orchestra.run.smithi103.stdout: "num_blocked_ops": 0 2024-09-10T19:05:54.988 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T19:05:55.022 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:55.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:55.023 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:55.124 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:55.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:55.190 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:55.190 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:55.291 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:05:55.322 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:55.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:55.324 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:05:55.425 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:55.448 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:55.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:55.449 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:55.550 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:55.594 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:55.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:55.595 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:55.696 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:55.701 INFO:teuthology.orchestra.run.smithi018.stderr:reweighted osd.3 to 1 (10000) 2024-09-10T19:05:55.708 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:05:55.708 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:55 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2474276129' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-10T19:05:55.735 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:55.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:55.736 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:55.754 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-10T19:05:55.837 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:55.879 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:55.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:55.880 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:55.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:05:55.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:55 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2474276129' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-10T19:05:55.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:05:55.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:55 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2474276129' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-10T19:05:55.981 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:56.053 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:56.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:56.054 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:56.138 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:05:56.155 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:56.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:56.193 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:56.193 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:56.294 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:56.319 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:56.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:56.320 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:56.421 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:56.456 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:56.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:56.457 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:56.558 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:56.588 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:56.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:56.589 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:56.690 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:56.719 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:56.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:56.720 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:56.821 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:56.843 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:56.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:56.845 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:56.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:56 smithi018 ceph-mon[29336]: pgmap v969: 178 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 176 active+clean; 4.3 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 1.40k op/s; 3916/210981 objects misplaced (1.856%); 6.6 MiB/s, 105 objects/s recovering 2024-09-10T19:05:56.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:56 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2474276129' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-10T19:05:56.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:56 smithi018 ceph-mon[29336]: osdmap e437: 8 total, 8 up, 6 in 2024-09-10T19:05:56.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:56 smithi018 ceph-mon[40196]: pgmap v969: 178 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 176 active+clean; 4.3 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 1.40k op/s; 3916/210981 objects misplaced (1.856%); 6.6 MiB/s, 105 objects/s recovering 2024-09-10T19:05:56.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:56 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2474276129' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-10T19:05:56.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:56 smithi018 ceph-mon[40196]: osdmap e437: 8 total, 8 up, 6 in 2024-09-10T19:05:56.945 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:56.978 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:56.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:56.980 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:57.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:56 smithi103 ceph-mon[33163]: pgmap v969: 178 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 176 active+clean; 4.3 GiB data, 21 GiB used, 515 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 1.40k op/s; 3916/210981 objects misplaced (1.856%); 6.6 MiB/s, 105 objects/s recovering 2024-09-10T19:05:57.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:56 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2474276129' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-10T19:05:57.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:56 smithi103 ceph-mon[33163]: osdmap e437: 8 total, 8 up, 6 in 2024-09-10T19:05:57.080 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:57.112 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:57.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:57.113 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:57.214 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:57.239 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:57.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:57.241 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:57.341 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:57.379 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:57.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:57.380 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:57.481 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:57.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:57.522 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:57.523 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:57.624 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:05:57.633 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:05:57 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:05:57.648 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:57.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:57.650 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:05:57.750 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:57.772 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:57.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:57.773 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:57.874 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:57.914 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:57.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:57.915 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:58.016 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:58.046 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:58.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:58.048 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:58.149 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:58.186 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:58.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:58.187 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:58.288 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:58.328 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:58.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:58.330 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:58.431 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:05:58.469 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:58.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:58.471 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:05:58.483 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd reweight 4 1 2024-09-10T19:05:58.551 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:05:58.572 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:58.603 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:58.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:58.604 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:58.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:58 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:58.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:05:58 smithi018 ceph-mon[40196]: pgmap v971: 178 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 176 active+clean; 3.9 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.46k op/s; 2540/194676 objects misplaced (1.305%); 6.7 MiB/s, 107 objects/s recovering 2024-09-10T19:05:58.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:58 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:58.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:05:58 smithi018 ceph-mon[29336]: pgmap v971: 178 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 176 active+clean; 3.9 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.46k op/s; 2540/194676 objects misplaced (1.305%); 6.7 MiB/s, 107 objects/s recovering 2024-09-10T19:05:58.705 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:58.743 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:58.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:58.745 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:58.845 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:58.870 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:58.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:58.871 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:05:58.972 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:05:58.997 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:58.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:58.998 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:05:59.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:58 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:59.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:05:58 smithi103 ceph-mon[33163]: pgmap v971: 178 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 176 active+clean; 3.9 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.46k op/s; 2540/194676 objects misplaced (1.305%); 6.7 MiB/s, 107 objects/s recovering 2024-09-10T19:05:59.099 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:59.147 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:59.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:59.148 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:59.249 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:05:59.288 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:59.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:59.289 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:05:59.390 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:59.429 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:59.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:59.430 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:59.531 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:05:59.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:59.572 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:59.572 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:05:59.619 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T19:05:59.619 INFO:teuthology.orchestra.run.smithi103.stdout: "size": 20, 2024-09-10T19:05:59.619 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 600, 2024-09-10T19:05:59.619 INFO:teuthology.orchestra.run.smithi103.stdout: "ops": [ 2024-09-10T19:05:59.619 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.619 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15354.0:167599 9.e 9:71637c4b:::benchmark_data_smithi018_133594_object30847:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-10T19:05:59.619 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:57:50.774991+0000", 2024-09-10T19:05:59.620 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 487.89891937300001, 2024-09-10T19:05:59.620 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0426897470000001, 2024-09-10T19:05:59.620 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:05:59.620 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:05:59.620 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:05:59.620 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15354", 2024-09-10T19:05:59.620 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:05:59.620 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 167599 2024-09-10T19:05:59.620 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.620 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:05:59.620 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.620 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:05:59.621 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.774991+0000", 2024-09-10T19:05:59.621 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.621 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.621 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.621 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:05:59.621 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.774991+0000", 2024-09-10T19:05:59.621 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.621 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.621 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.621 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:05:59.621 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.774989+0000", 2024-09-10T19:05:59.621 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T19:05:59.622 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.622 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.622 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:05:59.622 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.774997+0000", 2024-09-10T19:05:59.622 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.3130000000000001e-06 2024-09-10T19:05:59.622 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.622 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.622 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:05:59.622 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.775010+0000", 2024-09-10T19:05:59.622 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.3139000000000001e-05 2024-09-10T19:05:59.622 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.622 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.623 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:05:59.623 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.775016+0000", 2024-09-10T19:05:59.623 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.5570000000000001e-06 2024-09-10T19:05:59.623 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.623 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.623 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.623 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.775067+0000", 2024-09-10T19:05:59.623 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.0877000000000001e-05 2024-09-10T19:05:59.623 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.623 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.623 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:05:59.623 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.775074+0000", 2024-09-10T19:05:59.624 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.7630000000000003e-06 2024-09-10T19:05:59.624 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.624 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.624 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.624 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.715900+0000", 2024-09-10T19:05:59.624 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.94082635299999995 2024-09-10T19:05:59.624 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.624 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.624 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:05:59.624 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.715981+0000", 2024-09-10T19:05:59.624 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.0375000000000002e-05 2024-09-10T19:05:59.624 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.624 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.624 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1", 2024-09-10T19:05:59.625 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.716057+0000", 2024-09-10T19:05:59.625 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.6440000000000007e-05 2024-09-10T19:05:59.625 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.625 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.625 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.625 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.720375+0000", 2024-09-10T19:05:59.625 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0043181039999999997 2024-09-10T19:05:59.625 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.625 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.625 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:05:59.626 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.720383+0000", 2024-09-10T19:05:59.626 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.3139999999999998e-06 2024-09-10T19:05:59.626 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.626 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.626 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.626 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.817622+0000", 2024-09-10T19:05:59.626 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.097239256999999996 2024-09-10T19:05:59.626 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.626 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.626 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:05:59.626 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.817656+0000", 2024-09-10T19:05:59.626 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.4356000000000001e-05 2024-09-10T19:05:59.626 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.626 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.627 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:05:59.627 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.817681+0000", 2024-09-10T19:05:59.627 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.4494999999999998e-05 2024-09-10T19:05:59.627 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.627 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:05:59.627 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.627 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.627 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.627 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15354.0:167621 9.e 9:7b014d0c:::benchmark_data_smithi018_133594_object30869:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-10T19:05:59.627 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:57:50.796574+0000", 2024-09-10T19:05:59.627 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 487.87733643399997, 2024-09-10T19:05:59.627 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0211979630000001, 2024-09-10T19:05:59.628 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:05:59.628 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:05:59.628 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:05:59.628 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15354", 2024-09-10T19:05:59.628 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:05:59.628 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 167621 2024-09-10T19:05:59.628 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.628 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:05:59.628 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.628 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:05:59.628 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.796574+0000", 2024-09-10T19:05:59.628 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.628 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.628 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.629 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:05:59.629 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.796574+0000", 2024-09-10T19:05:59.629 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.629 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.629 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.629 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:05:59.629 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.796572+0000", 2024-09-10T19:05:59.629 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T19:05:59.629 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.629 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.629 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:05:59.629 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.796580+0000", 2024-09-10T19:05:59.629 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.61e-06 2024-09-10T19:05:59.629 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.630 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.630 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:05:59.630 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.796598+0000", 2024-09-10T19:05:59.630 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.7652999999999999e-05 2024-09-10T19:05:59.630 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.630 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.630 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:05:59.630 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.796604+0000", 2024-09-10T19:05:59.630 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.1240000000000002e-06 2024-09-10T19:05:59.630 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.630 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.630 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.630 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.796645+0000", 2024-09-10T19:05:59.631 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.0661000000000003e-05 2024-09-10T19:05:59.631 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.631 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.631 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:05:59.631 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.796657+0000", 2024-09-10T19:05:59.631 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.2568000000000001e-05 2024-09-10T19:05:59.631 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.631 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.631 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.631 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.717750+0000", 2024-09-10T19:05:59.631 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.92109322400000004 2024-09-10T19:05:59.631 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.631 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.632 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:05:59.632 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.717874+0000", 2024-09-10T19:05:59.632 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00012409199999999999 2024-09-10T19:05:59.632 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.632 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.632 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1", 2024-09-10T19:05:59.632 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.717962+0000", 2024-09-10T19:05:59.632 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.7746999999999999e-05 2024-09-10T19:05:59.632 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.632 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.632 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.632 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.720488+0000", 2024-09-10T19:05:59.632 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.002526127 2024-09-10T19:05:59.633 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.633 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.633 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:05:59.633 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.721476+0000", 2024-09-10T19:05:59.633 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00098792900000000002 2024-09-10T19:05:59.633 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.634 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.634 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.634 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.817721+0000", 2024-09-10T19:05:59.635 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.096244747000000005 2024-09-10T19:05:59.635 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.635 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.635 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:05:59.635 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.817751+0000", 2024-09-10T19:05:59.635 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.9679000000000002e-05 2024-09-10T19:05:59.635 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.635 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.635 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:05:59.635 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.817772+0000", 2024-09-10T19:05:59.635 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.1280999999999998e-05 2024-09-10T19:05:59.635 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.636 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:05:59.636 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.636 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.636 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.636 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15354.0:167648 9.e 9:713cb2ff:::benchmark_data_smithi018_133594_object30896:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-10T19:05:59.636 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:57:50.822608+0000", 2024-09-10T19:05:59.636 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 487.85130226000001, 2024-09-10T19:05:59.636 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.99523505199999995, 2024-09-10T19:05:59.636 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:05:59.636 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:05:59.636 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:05:59.636 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15354", 2024-09-10T19:05:59.637 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:05:59.637 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 167648 2024-09-10T19:05:59.637 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.637 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:05:59.637 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.637 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:05:59.637 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.822608+0000", 2024-09-10T19:05:59.637 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.637 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.637 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.637 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:05:59.638 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.822608+0000", 2024-09-10T19:05:59.638 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.638 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.638 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.638 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:05:59.638 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.822607+0000", 2024-09-10T19:05:59.638 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T19:05:59.638 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.638 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.638 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:05:59.638 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.822612+0000", 2024-09-10T19:05:59.638 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.5840000000000002e-06 2024-09-10T19:05:59.639 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.639 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.639 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:05:59.639 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.822624+0000", 2024-09-10T19:05:59.639 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.1095e-05 2024-09-10T19:05:59.639 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.639 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.639 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:05:59.640 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.822628+0000", 2024-09-10T19:05:59.640 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.4889999999999997e-06 2024-09-10T19:05:59.640 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.640 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.640 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.640 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.822662+0000", 2024-09-10T19:05:59.640 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.4029999999999998e-05 2024-09-10T19:05:59.640 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.640 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.640 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:05:59.640 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.822668+0000", 2024-09-10T19:05:59.640 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.1469999999999998e-06 2024-09-10T19:05:59.640 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.641 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.641 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.641 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.719211+0000", 2024-09-10T19:05:59.641 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.89654309499999996 2024-09-10T19:05:59.641 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.641 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.641 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:05:59.641 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.719322+0000", 2024-09-10T19:05:59.641 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000110701 2024-09-10T19:05:59.641 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.641 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.641 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1", 2024-09-10T19:05:59.641 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.719440+0000", 2024-09-10T19:05:59.642 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000118015 2024-09-10T19:05:59.642 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.642 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.642 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.642 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.722546+0000", 2024-09-10T19:05:59.642 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0031055029999999999 2024-09-10T19:05:59.642 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.642 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.642 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:05:59.642 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.722553+0000", 2024-09-10T19:05:59.642 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.6040000000000004e-06 2024-09-10T19:05:59.642 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.642 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.643 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.643 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.817795+0000", 2024-09-10T19:05:59.643 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.095241328 2024-09-10T19:05:59.643 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.643 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.643 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:05:59.643 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.817816+0000", 2024-09-10T19:05:59.643 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.1263999999999998e-05 2024-09-10T19:05:59.643 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.643 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.643 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:05:59.643 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.817843+0000", 2024-09-10T19:05:59.644 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.7396e-05 2024-09-10T19:05:59.644 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.644 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:05:59.644 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.644 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.644 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.644 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15354.0:167692 9.e 9:7812ba06:::benchmark_data_smithi018_133594_object30940:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-10T19:05:59.644 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:57:50.858240+0000", 2024-09-10T19:05:59.644 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 487.81566999, 2024-09-10T19:05:59.644 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.95971880799999998, 2024-09-10T19:05:59.644 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:05:59.644 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:05:59.645 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:05:59.645 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15354", 2024-09-10T19:05:59.645 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:05:59.645 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 167692 2024-09-10T19:05:59.645 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.645 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:05:59.645 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.645 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:05:59.645 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.858240+0000", 2024-09-10T19:05:59.645 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.645 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.645 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.645 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:05:59.645 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.858240+0000", 2024-09-10T19:05:59.646 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.646 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.646 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.646 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:05:59.646 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.858239+0000", 2024-09-10T19:05:59.646 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T19:05:59.646 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.646 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.646 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:05:59.646 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.858246+0000", 2024-09-10T19:05:59.646 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.4139999999999999e-06 2024-09-10T19:05:59.646 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.646 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.647 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:05:59.647 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.858258+0000", 2024-09-10T19:05:59.647 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.2136999999999999e-05 2024-09-10T19:05:59.647 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.647 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.647 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:05:59.647 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.858264+0000", 2024-09-10T19:05:59.647 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.6220000000000003e-06 2024-09-10T19:05:59.647 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.647 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.647 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.647 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.858302+0000", 2024-09-10T19:05:59.647 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.7580000000000003e-05 2024-09-10T19:05:59.648 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.648 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.648 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:05:59.648 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.858309+0000", 2024-09-10T19:05:59.648 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.9439999999999999e-06 2024-09-10T19:05:59.648 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.648 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.648 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.648 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.720527+0000", 2024-09-10T19:05:59.648 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.86221863600000004 2024-09-10T19:05:59.648 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.648 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.648 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:05:59.649 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.720608+0000", 2024-09-10T19:05:59.649 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.0086000000000005e-05 2024-09-10T19:05:59.649 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.649 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.649 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1", 2024-09-10T19:05:59.649 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.720671+0000", 2024-09-10T19:05:59.649 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.3140999999999997e-05 2024-09-10T19:05:59.649 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.649 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.649 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.649 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.722601+0000", 2024-09-10T19:05:59.649 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0019302379999999999 2024-09-10T19:05:59.650 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.650 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.650 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:05:59.650 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.723572+0000", 2024-09-10T19:05:59.650 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00097062600000000004 2024-09-10T19:05:59.650 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.650 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.650 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.650 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.817884+0000", 2024-09-10T19:05:59.650 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.094312143000000001 2024-09-10T19:05:59.650 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.651 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.651 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:05:59.651 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.817928+0000", 2024-09-10T19:05:59.651 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.4007000000000001e-05 2024-09-10T19:05:59.651 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.651 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.651 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:05:59.651 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.817959+0000", 2024-09-10T19:05:59.651 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.1492999999999998e-05 2024-09-10T19:05:59.651 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.651 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:05:59.651 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.652 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.652 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.652 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15354.0:167728 9.e 9:72c3f2c6:::benchmark_data_smithi018_133594_object30976:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-10T19:05:59.652 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:57:50.891268+0000", 2024-09-10T19:05:59.652 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 487.78264193799998, 2024-09-10T19:05:59.652 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.930936718, 2024-09-10T19:05:59.652 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:05:59.652 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:05:59.652 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:05:59.652 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15354", 2024-09-10T19:05:59.652 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:05:59.652 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 167728 2024-09-10T19:05:59.652 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.653 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:05:59.653 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.653 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:05:59.653 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.891268+0000", 2024-09-10T19:05:59.653 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.653 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.653 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.653 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:05:59.653 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.891268+0000", 2024-09-10T19:05:59.653 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.653 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.653 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.654 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:05:59.654 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.891266+0000", 2024-09-10T19:05:59.654 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999981 2024-09-10T19:05:59.654 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.654 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.654 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:05:59.654 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.891274+0000", 2024-09-10T19:05:59.654 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.5689999999999999e-06 2024-09-10T19:05:59.654 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.654 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.654 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:05:59.654 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.891290+0000", 2024-09-10T19:05:59.655 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.5566e-05 2024-09-10T19:05:59.655 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.655 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.655 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:05:59.655 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.891296+0000", 2024-09-10T19:05:59.655 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.562e-06 2024-09-10T19:05:59.655 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.655 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.655 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.655 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.891342+0000", 2024-09-10T19:05:59.655 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.5819000000000003e-05 2024-09-10T19:05:59.655 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.656 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.656 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:05:59.656 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.891349+0000", 2024-09-10T19:05:59.656 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.5109999999999997e-06 2024-09-10T19:05:59.656 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.656 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.656 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.656 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.721510+0000", 2024-09-10T19:05:59.656 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.830160546 2024-09-10T19:05:59.656 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.656 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.656 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:05:59.656 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.721614+0000", 2024-09-10T19:05:59.657 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000104512 2024-09-10T19:05:59.657 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.657 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.657 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1", 2024-09-10T19:05:59.657 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.721731+0000", 2024-09-10T19:05:59.657 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000116742 2024-09-10T19:05:59.657 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.657 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.657 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.657 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.724674+0000", 2024-09-10T19:05:59.657 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0029431750000000001 2024-09-10T19:05:59.658 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.658 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.658 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:05:59.658 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.724682+0000", 2024-09-10T19:05:59.658 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.6109999999999996e-06 2024-09-10T19:05:59.658 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.658 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.658 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.658 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.822149+0000", 2024-09-10T19:05:59.658 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.097466786999999999 2024-09-10T19:05:59.658 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.658 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.658 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:05:59.659 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.822177+0000", 2024-09-10T19:05:59.659 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.8456999999999999e-05 2024-09-10T19:05:59.659 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.659 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.659 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:05:59.659 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.822205+0000", 2024-09-10T19:05:59.659 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.7889e-05 2024-09-10T19:05:59.659 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.659 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:05:59.659 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.659 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.659 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.659 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15354.0:167745 9.e 9:7b15c270:::benchmark_data_smithi018_133594_object30993:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e279)", 2024-09-10T19:05:59.659 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:57:50.907688+0000", 2024-09-10T19:05:59.659 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 487.766222691, 2024-09-10T19:05:59.660 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.93121002900000005, 2024-09-10T19:05:59.660 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:05:59.660 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:05:59.660 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:05:59.660 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15354", 2024-09-10T19:05:59.660 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:05:59.660 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 167745 2024-09-10T19:05:59.660 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.660 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:05:59.660 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.660 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:05:59.660 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.907688+0000", 2024-09-10T19:05:59.661 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.661 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.661 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.661 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:05:59.661 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.907688+0000", 2024-09-10T19:05:59.661 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.661 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.661 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.661 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:05:59.661 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.907686+0000", 2024-09-10T19:05:59.661 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999981 2024-09-10T19:05:59.661 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.662 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.662 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:05:59.662 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.907693+0000", 2024-09-10T19:05:59.662 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.2069999999999998e-06 2024-09-10T19:05:59.662 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.662 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.662 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:05:59.662 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.907706+0000", 2024-09-10T19:05:59.662 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.2911000000000001e-05 2024-09-10T19:05:59.662 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.663 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.663 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:05:59.663 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.907711+0000", 2024-09-10T19:05:59.663 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.5829999999999997e-06 2024-09-10T19:05:59.663 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.663 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.663 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.663 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.907759+0000", 2024-09-10T19:05:59.663 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.7346999999999999e-05 2024-09-10T19:05:59.663 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.663 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.663 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:05:59.664 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:50.907766+0000", 2024-09-10T19:05:59.664 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.2110000000000001e-06 2024-09-10T19:05:59.664 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.664 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.664 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.664 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.723607+0000", 2024-09-10T19:05:59.664 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.81584048899999995 2024-09-10T19:05:59.664 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.664 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.664 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:05:59.664 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.723704+0000", 2024-09-10T19:05:59.664 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.7391000000000003e-05 2024-09-10T19:05:59.665 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.665 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.665 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1", 2024-09-10T19:05:59.665 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.723818+0000", 2024-09-10T19:05:59.665 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00011437799999999999 2024-09-10T19:05:59.665 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.665 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.665 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.665 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.725749+0000", 2024-09-10T19:05:59.665 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0019303409999999999 2024-09-10T19:05:59.665 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.665 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.665 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:05:59.666 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.726768+0000", 2024-09-10T19:05:59.666 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0010194990000000001 2024-09-10T19:05:59.666 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.666 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.666 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.666 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.838790+0000", 2024-09-10T19:05:59.666 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.11202142599999999 2024-09-10T19:05:59.666 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.666 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.666 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:05:59.666 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.838848+0000", 2024-09-10T19:05:59.666 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.7976000000000002e-05 2024-09-10T19:05:59.666 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.667 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.667 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:05:59.667 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:57:51.838898+0000", 2024-09-10T19:05:59.667 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.0192000000000003e-05 2024-09-10T19:05:59.667 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.667 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:05:59.667 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.667 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.667 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.667 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15573.0:275413 10.2 10:4c27ae99:::benchmark_data_smithi018_157943_object136755:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e372)", 2024-09-10T19:05:59.667 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:02:48.576250+0000", 2024-09-10T19:05:59.667 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 190.09765989600001, 2024-09-10T19:05:59.667 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0385874479999999, 2024-09-10T19:05:59.668 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:05:59.668 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:05:59.668 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:05:59.668 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15573", 2024-09-10T19:05:59.668 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:05:59.668 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 275413 2024-09-10T19:05:59.668 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.668 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:05:59.668 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.668 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:05:59.668 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.576250+0000", 2024-09-10T19:05:59.668 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.668 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.668 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.669 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:05:59.669 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.576250+0000", 2024-09-10T19:05:59.669 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.669 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.669 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.669 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:05:59.669 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.576249+0000", 2024-09-10T19:05:59.669 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T19:05:59.669 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.669 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.669 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:05:59.669 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.576256+0000", 2024-09-10T19:05:59.670 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.8619999999999997e-06 2024-09-10T19:05:59.670 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.670 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.670 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:05:59.670 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.576268+0000", 2024-09-10T19:05:59.670 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.2026e-05 2024-09-10T19:05:59.670 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.670 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.670 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:05:59.670 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.576277+0000", 2024-09-10T19:05:59.670 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.6430000000000001e-06 2024-09-10T19:05:59.670 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.670 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.670 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.670 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.581257+0000", 2024-09-10T19:05:59.671 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0049805509999999997 2024-09-10T19:05:59.671 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.671 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.671 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:05:59.671 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.581265+0000", 2024-09-10T19:05:59.671 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.2400000000000007e-06 2024-09-10T19:05:59.671 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.671 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.671 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.671 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.596056+0000", 2024-09-10T19:05:59.671 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0147910200000001 2024-09-10T19:05:59.671 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.671 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.671 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:05:59.672 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.597712+0000", 2024-09-10T19:05:59.672 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0016556450000000001 2024-09-10T19:05:59.672 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.672 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.672 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1", 2024-09-10T19:05:59.672 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.597810+0000", 2024-09-10T19:05:59.672 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.7751000000000007e-05 2024-09-10T19:05:59.672 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.672 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.672 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:05:59.672 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.607572+0000", 2024-09-10T19:05:59.672 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.009762316 2024-09-10T19:05:59.673 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.673 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.673 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.673 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.614703+0000", 2024-09-10T19:05:59.673 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0071303620000000003 2024-09-10T19:05:59.673 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.673 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.673 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.673 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.614764+0000", 2024-09-10T19:05:59.673 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.1784000000000003e-05 2024-09-10T19:05:59.673 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.673 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.673 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:05:59.673 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.614795+0000", 2024-09-10T19:05:59.674 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.0877000000000003e-05 2024-09-10T19:05:59.674 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.674 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.674 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:05:59.674 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.614838+0000", 2024-09-10T19:05:59.674 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.2738999999999999e-05 2024-09-10T19:05:59.674 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.674 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:05:59.674 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.674 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.674 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.674 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15573.0:275436 10.2 10:4a4300a7:::benchmark_data_smithi018_157943_object136778:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e372)", 2024-09-10T19:05:59.674 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:02:48.595429+0000", 2024-09-10T19:05:59.675 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 190.078481858, 2024-09-10T19:05:59.675 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0195747390000001, 2024-09-10T19:05:59.675 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:05:59.675 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:05:59.675 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:05:59.675 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15573", 2024-09-10T19:05:59.675 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:05:59.675 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 275436 2024-09-10T19:05:59.675 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.675 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:05:59.675 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.676 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:05:59.676 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.595429+0000", 2024-09-10T19:05:59.676 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.676 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.676 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.676 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:05:59.676 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.595429+0000", 2024-09-10T19:05:59.676 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.676 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.676 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.676 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:05:59.676 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.595427+0000", 2024-09-10T19:05:59.677 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T19:05:59.677 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.677 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.677 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:05:59.677 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.595433+0000", 2024-09-10T19:05:59.677 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.5740000000000003e-06 2024-09-10T19:05:59.677 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.677 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.677 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:05:59.677 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.595442+0000", 2024-09-10T19:05:59.677 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.0080000000000008e-06 2024-09-10T19:05:59.677 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.677 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.678 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:05:59.678 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.595447+0000", 2024-09-10T19:05:59.678 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.848e-06 2024-09-10T19:05:59.678 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.678 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.678 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.678 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.596591+0000", 2024-09-10T19:05:59.678 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0011442410000000001 2024-09-10T19:05:59.678 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.678 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.678 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:05:59.678 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.596600+0000", 2024-09-10T19:05:59.678 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.4409999999999999e-06 2024-09-10T19:05:59.679 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.679 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.679 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.679 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.599387+0000", 2024-09-10T19:05:59.679 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0027862080000001 2024-09-10T19:05:59.679 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.679 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.679 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:05:59.679 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.599461+0000", 2024-09-10T19:05:59.679 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.4110999999999996e-05 2024-09-10T19:05:59.679 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.679 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.679 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1", 2024-09-10T19:05:59.680 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.599526+0000", 2024-09-10T19:05:59.680 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.5376999999999994e-05 2024-09-10T19:05:59.680 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.680 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.680 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:05:59.680 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.610110+0000", 2024-09-10T19:05:59.680 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.010583894999999999 2024-09-10T19:05:59.680 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.680 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.680 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.680 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.614739+0000", 2024-09-10T19:05:59.680 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0046292709999999999 2024-09-10T19:05:59.680 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.681 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.681 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.681 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.614921+0000", 2024-09-10T19:05:59.681 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00018209899999999999 2024-09-10T19:05:59.681 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.681 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.681 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:05:59.681 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.614967+0000", 2024-09-10T19:05:59.681 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.5895e-05 2024-09-10T19:05:59.681 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.681 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.682 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:05:59.682 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.615003+0000", 2024-09-10T19:05:59.682 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.6084000000000002e-05 2024-09-10T19:05:59.682 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.682 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:05:59.682 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.682 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.682 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.682 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15573.0:275439 10.2 10:4e7426c6:::benchmark_data_smithi018_157943_object136781:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e372)", 2024-09-10T19:05:59.683 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:02:48.597419+0000", 2024-09-10T19:05:59.683 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 190.07649187199999, 2024-09-10T19:05:59.683 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.020229308, 2024-09-10T19:05:59.683 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:05:59.683 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:05:59.683 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:05:59.683 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15573", 2024-09-10T19:05:59.683 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:05:59.683 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 275439 2024-09-10T19:05:59.684 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.684 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:05:59.684 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.684 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:05:59.684 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.597419+0000", 2024-09-10T19:05:59.684 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.684 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.684 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.684 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:05:59.685 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.597419+0000", 2024-09-10T19:05:59.685 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.685 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.685 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.685 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:05:59.685 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.597417+0000", 2024-09-10T19:05:59.685 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T19:05:59.685 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.685 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.685 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:05:59.685 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.597424+0000", 2024-09-10T19:05:59.685 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.8220000000000001e-06 2024-09-10T19:05:59.685 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.686 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.686 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:05:59.686 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.597445+0000", 2024-09-10T19:05:59.686 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.1515999999999999e-05 2024-09-10T19:05:59.686 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.686 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.686 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:05:59.686 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.597451+0000", 2024-09-10T19:05:59.686 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.9689999999999999e-06 2024-09-10T19:05:59.686 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.686 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.686 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.686 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.597498+0000", 2024-09-10T19:05:59.687 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.6244e-05 2024-09-10T19:05:59.687 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.687 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.687 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:05:59.687 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.597506+0000", 2024-09-10T19:05:59.687 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.1839999999999999e-06 2024-09-10T19:05:59.687 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.687 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.687 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.687 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.600704+0000", 2024-09-10T19:05:59.687 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.003198026 2024-09-10T19:05:59.687 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.688 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.688 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:05:59.688 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.600769+0000", 2024-09-10T19:05:59.688 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.5240999999999994e-05 2024-09-10T19:05:59.688 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.688 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.688 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1", 2024-09-10T19:05:59.688 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.600858+0000", 2024-09-10T19:05:59.688 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.9139000000000002e-05 2024-09-10T19:05:59.688 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.689 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.689 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:05:59.689 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.610116+0000", 2024-09-10T19:05:59.689 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0092573949999999999 2024-09-10T19:05:59.689 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.689 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.689 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.689 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.615044+0000", 2024-09-10T19:05:59.689 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0049287430000000002 2024-09-10T19:05:59.689 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.689 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.689 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.689 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.617567+0000", 2024-09-10T19:05:59.690 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0025225870000000002 2024-09-10T19:05:59.690 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.690 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.690 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:05:59.690 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.617613+0000", 2024-09-10T19:05:59.690 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.6091000000000003e-05 2024-09-10T19:05:59.690 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.690 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.690 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:05:59.690 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.617648+0000", 2024-09-10T19:05:59.690 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.4832000000000001e-05 2024-09-10T19:05:59.690 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.690 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:05:59.691 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.691 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.691 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.691 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15573.0:275469 10.2 10:41979c9b:::benchmark_data_smithi018_157943_object136811:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e372)", 2024-09-10T19:05:59.691 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:02:48.614537+0000", 2024-09-10T19:05:59.691 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 190.05937292499999, 2024-09-10T19:05:59.691 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0032040900000001, 2024-09-10T19:05:59.691 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:05:59.691 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:05:59.691 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:05:59.691 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15573", 2024-09-10T19:05:59.691 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:05:59.691 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 275469 2024-09-10T19:05:59.692 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.692 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:05:59.692 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.692 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:05:59.692 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.614537+0000", 2024-09-10T19:05:59.692 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.692 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.692 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.692 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:05:59.692 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.614537+0000", 2024-09-10T19:05:59.693 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.693 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.693 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.693 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:05:59.693 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.614536+0000", 2024-09-10T19:05:59.693 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T19:05:59.693 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.693 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.693 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:05:59.694 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.614542+0000", 2024-09-10T19:05:59.694 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.5529999999999998e-06 2024-09-10T19:05:59.694 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.694 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.694 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:05:59.694 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.614557+0000", 2024-09-10T19:05:59.694 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.419e-05 2024-09-10T19:05:59.694 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.694 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.694 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:05:59.694 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.614563+0000", 2024-09-10T19:05:59.694 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.6540000000000002e-06 2024-09-10T19:05:59.695 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.695 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.695 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.695 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.614615+0000", 2024-09-10T19:05:59.695 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.2135000000000001e-05 2024-09-10T19:05:59.695 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.695 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.695 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:05:59.695 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.614624+0000", 2024-09-10T19:05:59.695 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.0569999999999999e-06 2024-09-10T19:05:59.696 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.696 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.696 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.696 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.601853+0000", 2024-09-10T19:05:59.696 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.98722824799999997 2024-09-10T19:05:59.696 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.696 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.696 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:05:59.696 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.601941+0000", 2024-09-10T19:05:59.696 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.8197999999999998e-05 2024-09-10T19:05:59.696 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.697 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.697 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1", 2024-09-10T19:05:59.697 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.602035+0000", 2024-09-10T19:05:59.697 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.4421999999999998e-05 2024-09-10T19:05:59.697 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.697 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.697 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:05:59.697 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.610122+0000", 2024-09-10T19:05:59.697 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0080862820000000002 2024-09-10T19:05:59.697 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.697 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.697 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.698 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.615085+0000", 2024-09-10T19:05:59.698 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0049637989999999996 2024-09-10T19:05:59.698 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.698 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.698 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.698 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.617678+0000", 2024-09-10T19:05:59.698 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0025924139999999999 2024-09-10T19:05:59.698 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.698 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.698 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:05:59.698 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.617713+0000", 2024-09-10T19:05:59.699 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.4759999999999999e-05 2024-09-10T19:05:59.699 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.699 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.699 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:05:59.699 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.617742+0000", 2024-09-10T19:05:59.699 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.8949999999999999e-05 2024-09-10T19:05:59.699 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.699 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:05:59.699 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.699 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.699 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.699 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15573.0:275485 10.2 10:4d10b6ee:::benchmark_data_smithi018_157943_object136827:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e372)", 2024-09-10T19:05:59.699 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:02:48.624060+0000", 2024-09-10T19:05:59.700 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 190.04985041200001, 2024-09-10T19:05:59.700 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.993778991, 2024-09-10T19:05:59.700 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:05:59.700 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:05:59.700 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:05:59.700 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15573", 2024-09-10T19:05:59.700 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:05:59.700 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 275485 2024-09-10T19:05:59.700 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.700 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:05:59.700 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.701 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:05:59.701 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.624060+0000", 2024-09-10T19:05:59.701 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.701 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.701 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.701 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:05:59.701 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.624060+0000", 2024-09-10T19:05:59.701 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.701 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.701 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.701 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:05:59.701 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.624059+0000", 2024-09-10T19:05:59.701 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T19:05:59.702 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.702 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.702 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:05:59.702 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.624064+0000", 2024-09-10T19:05:59.702 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.1819999999999997e-06 2024-09-10T19:05:59.702 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.702 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.702 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:05:59.702 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.624075+0000", 2024-09-10T19:05:59.702 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.1243999999999999e-05 2024-09-10T19:05:59.702 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.703 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.703 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:05:59.703 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.624080+0000", 2024-09-10T19:05:59.703 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.2429999999999996e-06 2024-09-10T19:05:59.703 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.703 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.703 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.703 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.624137+0000", 2024-09-10T19:05:59.703 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.7037999999999998e-05 2024-09-10T19:05:59.703 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.703 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.703 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:05:59.703 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.624145+0000", 2024-09-10T19:05:59.704 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.0390000000000005e-06 2024-09-10T19:05:59.704 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.704 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.704 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.704 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.603306+0000", 2024-09-10T19:05:59.704 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.97916007900000002 2024-09-10T19:05:59.704 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.704 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.704 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:05:59.704 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.603410+0000", 2024-09-10T19:05:59.704 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000104337 2024-09-10T19:05:59.704 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.705 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.705 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1", 2024-09-10T19:05:59.705 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.603511+0000", 2024-09-10T19:05:59.705 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000101137 2024-09-10T19:05:59.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.705 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.705 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:05:59.705 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.610127+0000", 2024-09-10T19:05:59.705 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0066155850000000002 2024-09-10T19:05:59.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.705 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.705 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.706 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.615105+0000", 2024-09-10T19:05:59.706 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0049781390000000003 2024-09-10T19:05:59.706 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.706 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.706 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.706 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.617773+0000", 2024-09-10T19:05:59.706 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0026678040000000002 2024-09-10T19:05:59.706 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.706 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.706 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:05:59.706 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.617805+0000", 2024-09-10T19:05:59.706 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.2270000000000001e-05 2024-09-10T19:05:59.707 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.707 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.707 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:05:59.707 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.617839+0000", 2024-09-10T19:05:59.707 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.4150000000000003e-05 2024-09-10T19:05:59.707 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.707 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:05:59.707 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.707 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.707 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.707 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15573.0:275487 10.2 10:49c1a52e:::benchmark_data_smithi018_157943_object136829:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e372)", 2024-09-10T19:05:59.707 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:02:48.626078+0000", 2024-09-10T19:05:59.707 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 190.047832591, 2024-09-10T19:05:59.708 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.99187293799999998, 2024-09-10T19:05:59.708 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:05:59.708 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:05:59.708 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:05:59.708 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15573", 2024-09-10T19:05:59.708 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:05:59.708 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 275487 2024-09-10T19:05:59.708 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.708 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:05:59.708 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.708 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:05:59.708 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.626078+0000", 2024-09-10T19:05:59.708 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.709 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.709 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.709 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:05:59.709 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.626078+0000", 2024-09-10T19:05:59.709 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.709 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.709 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.709 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:05:59.709 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.626077+0000", 2024-09-10T19:05:59.709 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T19:05:59.709 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.709 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.709 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:05:59.709 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.626082+0000", 2024-09-10T19:05:59.710 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.6840000000000004e-06 2024-09-10T19:05:59.710 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.710 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.710 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:05:59.710 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.626091+0000", 2024-09-10T19:05:59.710 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.764e-06 2024-09-10T19:05:59.710 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.710 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.710 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:05:59.710 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.626096+0000", 2024-09-10T19:05:59.710 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.0790000000000002e-06 2024-09-10T19:05:59.710 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.710 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.711 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.711 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.626129+0000", 2024-09-10T19:05:59.711 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.3806999999999997e-05 2024-09-10T19:05:59.711 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.711 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.711 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:05:59.711 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.626134+0000", 2024-09-10T19:05:59.711 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.0189999999999997e-06 2024-09-10T19:05:59.711 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.711 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.711 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.711 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.604675+0000", 2024-09-10T19:05:59.712 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.97854074099999999 2024-09-10T19:05:59.712 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.712 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.712 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:05:59.712 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.604776+0000", 2024-09-10T19:05:59.712 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000101065 2024-09-10T19:05:59.712 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.712 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.712 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1", 2024-09-10T19:05:59.712 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.604894+0000", 2024-09-10T19:05:59.712 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00011775800000000001 2024-09-10T19:05:59.712 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.712 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.712 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:05:59.713 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.610131+0000", 2024-09-10T19:05:59.713 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0052374220000000003 2024-09-10T19:05:59.713 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.713 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.713 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.713 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.615149+0000", 2024-09-10T19:05:59.713 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0050177900000000003 2024-09-10T19:05:59.713 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.713 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.713 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.713 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.617880+0000", 2024-09-10T19:05:59.713 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0027311039999999998 2024-09-10T19:05:59.713 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.714 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.714 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:05:59.714 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.617915+0000", 2024-09-10T19:05:59.714 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.5015000000000003e-05 2024-09-10T19:05:59.714 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.714 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.714 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:05:59.714 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.617951+0000", 2024-09-10T19:05:59.714 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.5460000000000003e-05 2024-09-10T19:05:59.714 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.714 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:05:59.714 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.714 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.715 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.715 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15573.0:275493 10.2 10:478b41e5:::benchmark_data_smithi018_157943_object136835:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e372)", 2024-09-10T19:05:59.715 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:02:48.630532+0000", 2024-09-10T19:05:59.715 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 190.04337858599999, 2024-09-10T19:05:59.715 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.98758235900000002, 2024-09-10T19:05:59.715 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:05:59.715 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:05:59.715 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:05:59.715 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15573", 2024-09-10T19:05:59.715 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:05:59.715 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 275493 2024-09-10T19:05:59.715 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.715 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:05:59.716 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.716 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:05:59.716 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.630532+0000", 2024-09-10T19:05:59.716 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.716 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.716 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.716 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:05:59.716 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.630532+0000", 2024-09-10T19:05:59.716 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.716 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.716 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.716 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:05:59.716 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.630531+0000", 2024-09-10T19:05:59.717 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T19:05:59.717 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.717 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.717 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:05:59.717 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.630535+0000", 2024-09-10T19:05:59.717 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.3939999999999998e-06 2024-09-10T19:05:59.717 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.717 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.717 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:05:59.717 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.630545+0000", 2024-09-10T19:05:59.717 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.3430000000000002e-06 2024-09-10T19:05:59.718 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.718 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.718 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:05:59.718 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.630549+0000", 2024-09-10T19:05:59.718 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.1269999999999996e-06 2024-09-10T19:05:59.718 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.718 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.718 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.718 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.630600+0000", 2024-09-10T19:05:59.718 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.113e-05 2024-09-10T19:05:59.718 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.718 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.718 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:05:59.719 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.630610+0000", 2024-09-10T19:05:59.719 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.7040000000000006e-06 2024-09-10T19:05:59.719 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.719 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.719 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.719 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.606166+0000", 2024-09-10T19:05:59.719 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.97555623700000005 2024-09-10T19:05:59.719 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.719 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.719 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:05:59.719 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.606286+0000", 2024-09-10T19:05:59.719 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000120363 2024-09-10T19:05:59.719 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.720 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.720 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1", 2024-09-10T19:05:59.720 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.606405+0000", 2024-09-10T19:05:59.720 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00011878400000000001 2024-09-10T19:05:59.720 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.720 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.720 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:05:59.720 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.613951+0000", 2024-09-10T19:05:59.720 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0075458019999999999 2024-09-10T19:05:59.720 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.720 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.720 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.721 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.617979+0000", 2024-09-10T19:05:59.721 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0040284930000000002 2024-09-10T19:05:59.721 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.721 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.721 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.721 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.618052+0000", 2024-09-10T19:05:59.721 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.2640999999999998e-05 2024-09-10T19:05:59.721 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.721 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.721 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:05:59.721 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.618085+0000", 2024-09-10T19:05:59.721 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.3484e-05 2024-09-10T19:05:59.721 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.722 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.722 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:05:59.722 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.618114+0000", 2024-09-10T19:05:59.722 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.8819999999999999e-05 2024-09-10T19:05:59.722 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.722 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:05:59.722 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.722 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.722 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.722 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15573.0:275498 10.2 10:4f3fa5f0:::benchmark_data_smithi018_157943_object136840:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e372)", 2024-09-10T19:05:59.722 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:02:48.634854+0000", 2024-09-10T19:05:59.722 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 190.03905630099999, 2024-09-10T19:05:59.722 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.98370333499999996, 2024-09-10T19:05:59.723 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:05:59.723 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:05:59.723 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:05:59.723 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15573", 2024-09-10T19:05:59.723 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:05:59.723 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 275498 2024-09-10T19:05:59.723 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.723 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:05:59.723 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.723 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:05:59.723 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.634854+0000", 2024-09-10T19:05:59.723 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.723 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.724 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.724 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:05:59.724 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.634854+0000", 2024-09-10T19:05:59.724 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.724 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.724 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.724 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:05:59.724 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.634853+0000", 2024-09-10T19:05:59.724 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T19:05:59.724 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.724 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.724 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:05:59.725 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.634859+0000", 2024-09-10T19:05:59.725 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.9780000000000002e-06 2024-09-10T19:05:59.725 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.725 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.725 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:05:59.725 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.634870+0000", 2024-09-10T19:05:59.725 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.1596999999999999e-05 2024-09-10T19:05:59.725 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.726 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.726 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:05:59.726 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.634876+0000", 2024-09-10T19:05:59.726 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.993e-06 2024-09-10T19:05:59.726 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.726 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.726 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.726 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.634941+0000", 2024-09-10T19:05:59.726 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.5166999999999995e-05 2024-09-10T19:05:59.727 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.727 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.727 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:05:59.727 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.634951+0000", 2024-09-10T19:05:59.727 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.4509999999999998e-06 2024-09-10T19:05:59.728 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.728 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.728 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.728 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.607596+0000", 2024-09-10T19:05:59.728 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.97264472000000002 2024-09-10T19:05:59.728 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.728 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.728 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:05:59.728 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.607684+0000", 2024-09-10T19:05:59.728 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.8750000000000002e-05 2024-09-10T19:05:59.728 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.729 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.729 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1", 2024-09-10T19:05:59.729 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.607778+0000", 2024-09-10T19:05:59.729 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.3453000000000001e-05 2024-09-10T19:05:59.729 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.729 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.729 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:05:59.729 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.614073+0000", 2024-09-10T19:05:59.729 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0062952809999999998 2024-09-10T19:05:59.729 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.729 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.729 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.730 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.618004+0000", 2024-09-10T19:05:59.730 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.003930551 2024-09-10T19:05:59.730 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.730 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.730 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.730 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.618474+0000", 2024-09-10T19:05:59.730 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00047017800000000002 2024-09-10T19:05:59.730 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.730 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.730 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:05:59.730 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.618518+0000", 2024-09-10T19:05:59.730 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.4159000000000002e-05 2024-09-10T19:05:59.730 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.731 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.731 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:05:59.731 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.618557+0000", 2024-09-10T19:05:59.731 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.9453000000000003e-05 2024-09-10T19:05:59.731 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.731 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:05:59.731 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.731 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.731 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.731 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15573.0:275513 10.2 10:45047990:::benchmark_data_smithi018_157943_object136855:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e372)", 2024-09-10T19:05:59.731 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:02:48.652805+0000", 2024-09-10T19:05:59.731 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 190.02110572500001, 2024-09-10T19:05:59.731 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.97232516999999996, 2024-09-10T19:05:59.732 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:05:59.732 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:05:59.732 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:05:59.732 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15573", 2024-09-10T19:05:59.732 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:05:59.732 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 275513 2024-09-10T19:05:59.732 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.732 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:05:59.732 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.732 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:05:59.732 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.652805+0000", 2024-09-10T19:05:59.732 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.732 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.733 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.733 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:05:59.733 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.652805+0000", 2024-09-10T19:05:59.733 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.733 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.733 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.733 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:05:59.733 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.652804+0000", 2024-09-10T19:05:59.733 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T19:05:59.733 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.733 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.733 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:05:59.733 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.652808+0000", 2024-09-10T19:05:59.734 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.6870000000000002e-06 2024-09-10T19:05:59.734 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.734 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.734 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:05:59.734 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.652818+0000", 2024-09-10T19:05:59.734 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.4460000000000007e-06 2024-09-10T19:05:59.734 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.734 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.734 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:05:59.734 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.652833+0000", 2024-09-10T19:05:59.734 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.5014000000000001e-05 2024-09-10T19:05:59.734 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.735 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.735 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.735 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.652890+0000", 2024-09-10T19:05:59.735 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.7534999999999997e-05 2024-09-10T19:05:59.735 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.735 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.735 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:05:59.735 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:48.652900+0000", 2024-09-10T19:05:59.735 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.8069999999999994e-06 2024-09-10T19:05:59.735 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.735 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.735 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.736 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.610867+0000", 2024-09-10T19:05:59.736 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.957967126 2024-09-10T19:05:59.736 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.736 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.736 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:05:59.736 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.610975+0000", 2024-09-10T19:05:59.736 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000107175 2024-09-10T19:05:59.736 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.736 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.736 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1", 2024-09-10T19:05:59.736 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.611092+0000", 2024-09-10T19:05:59.736 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00011780100000000001 2024-09-10T19:05:59.736 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.737 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.737 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:05:59.737 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.614202+0000", 2024-09-10T19:05:59.737 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0031098039999999999 2024-09-10T19:05:59.737 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.737 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.737 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.737 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.618028+0000", 2024-09-10T19:05:59.737 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0038261219999999999 2024-09-10T19:05:59.737 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.737 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.737 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.737 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.625012+0000", 2024-09-10T19:05:59.738 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0069837529999999997 2024-09-10T19:05:59.738 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.738 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.738 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:05:59.738 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.625093+0000", 2024-09-10T19:05:59.738 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.1155999999999993e-05 2024-09-10T19:05:59.738 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.738 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.738 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:05:59.738 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:02:49.625130+0000", 2024-09-10T19:05:59.738 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.6671000000000002e-05 2024-09-10T19:05:59.738 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.738 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:05:59.739 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.739 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.739 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.739 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.25706.0:142592 11.18 11:1edcbf0a:::benchmark_data_smithi018_181485_object5500:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e417)", 2024-09-10T19:05:59.739 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:05:03.856364+0000", 2024-09-10T19:05:59.739 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 54.817545983999999, 2024-09-10T19:05:59.739 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.97133153800000005, 2024-09-10T19:05:59.739 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:05:59.739 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:05:59.739 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:05:59.739 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.25706", 2024-09-10T19:05:59.739 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/42081399", 2024-09-10T19:05:59.739 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 142592 2024-09-10T19:05:59.739 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.740 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:05:59.740 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.740 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:05:59.740 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.856364+0000", 2024-09-10T19:05:59.740 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.740 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.740 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.740 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:05:59.740 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.856364+0000", 2024-09-10T19:05:59.741 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.741 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.741 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.741 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:05:59.741 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.856363+0000", 2024-09-10T19:05:59.741 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T19:05:59.741 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.741 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.741 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:05:59.741 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.856369+0000", 2024-09-10T19:05:59.741 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.3219999999999998e-06 2024-09-10T19:05:59.741 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.741 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.742 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:05:59.742 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.856382+0000", 2024-09-10T19:05:59.742 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.2795e-05 2024-09-10T19:05:59.742 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.742 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.742 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:05:59.742 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.856388+0000", 2024-09-10T19:05:59.742 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.7409999999999998e-06 2024-09-10T19:05:59.742 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.742 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.742 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.742 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.856430+0000", 2024-09-10T19:05:59.742 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.2231000000000003e-05 2024-09-10T19:05:59.743 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.743 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.743 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:05:59.743 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.856436+0000", 2024-09-10T19:05:59.743 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.8470000000000001e-06 2024-09-10T19:05:59.743 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.743 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.743 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.743 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.808544+0000", 2024-09-10T19:05:59.743 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.95210839199999997 2024-09-10T19:05:59.743 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.743 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.743 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:05:59.744 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.808656+0000", 2024-09-10T19:05:59.744 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000111863 2024-09-10T19:05:59.744 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.744 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.744 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 5,6", 2024-09-10T19:05:59.744 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.808846+0000", 2024-09-10T19:05:59.744 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00019014600000000001 2024-09-10T19:05:59.744 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.744 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.744 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:05:59.744 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.814362+0000", 2024-09-10T19:05:59.744 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0055163149999999999 2024-09-10T19:05:59.745 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.745 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.745 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.745 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.827383+0000", 2024-09-10T19:05:59.745 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.013020356 2024-09-10T19:05:59.745 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.745 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.745 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.745 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.827573+0000", 2024-09-10T19:05:59.745 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000190378 2024-09-10T19:05:59.745 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.745 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.746 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:05:59.746 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.827625+0000", 2024-09-10T19:05:59.746 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.2120000000000002e-05 2024-09-10T19:05:59.746 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.746 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.746 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:05:59.746 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.827696+0000", 2024-09-10T19:05:59.746 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.0580000000000005e-05 2024-09-10T19:05:59.746 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.746 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:05:59.746 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.746 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.747 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.747 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.25706.0:142600 11.18 11:1a3b37bf:::benchmark_data_smithi018_181485_object5508:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e417)", 2024-09-10T19:05:59.747 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:05:03.858390+0000", 2024-09-10T19:05:59.747 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 54.815520491000001, 2024-09-10T19:05:59.747 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.96940188299999996, 2024-09-10T19:05:59.747 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:05:59.747 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:05:59.747 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:05:59.747 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.25706", 2024-09-10T19:05:59.747 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/42081399", 2024-09-10T19:05:59.747 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 142600 2024-09-10T19:05:59.747 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.747 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:05:59.747 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.748 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:05:59.748 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.858390+0000", 2024-09-10T19:05:59.748 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.748 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.748 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.748 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:05:59.748 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.858390+0000", 2024-09-10T19:05:59.748 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.748 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.748 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.748 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:05:59.748 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.858389+0000", 2024-09-10T19:05:59.749 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T19:05:59.749 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.749 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.749 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:05:59.749 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.858394+0000", 2024-09-10T19:05:59.749 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.1320000000000002e-06 2024-09-10T19:05:59.749 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.749 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.749 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:05:59.749 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.858406+0000", 2024-09-10T19:05:59.749 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.1524e-05 2024-09-10T19:05:59.749 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.749 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.750 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:05:59.750 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.858410+0000", 2024-09-10T19:05:59.750 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.2780000000000004e-06 2024-09-10T19:05:59.750 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.750 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.750 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.750 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.858448+0000", 2024-09-10T19:05:59.750 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.8445000000000002e-05 2024-09-10T19:05:59.750 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.750 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.750 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:05:59.750 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.858454+0000", 2024-09-10T19:05:59.750 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.6810000000000003e-06 2024-09-10T19:05:59.750 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.751 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.751 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.751 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.810073+0000", 2024-09-10T19:05:59.751 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.95161914599999997 2024-09-10T19:05:59.751 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.751 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.751 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:05:59.751 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.810148+0000", 2024-09-10T19:05:59.751 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.4948000000000001e-05 2024-09-10T19:05:59.751 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.751 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.751 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 5,6", 2024-09-10T19:05:59.752 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.810293+0000", 2024-09-10T19:05:59.752 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00014450500000000001 2024-09-10T19:05:59.752 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.752 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.752 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:05:59.752 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.816468+0000", 2024-09-10T19:05:59.752 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0061751829999999999 2024-09-10T19:05:59.752 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.752 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.752 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.752 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.827464+0000", 2024-09-10T19:05:59.752 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.010996196999999999 2024-09-10T19:05:59.752 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.753 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.753 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.753 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.827741+0000", 2024-09-10T19:05:59.753 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000276758 2024-09-10T19:05:59.753 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.753 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.753 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:05:59.753 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.827771+0000", 2024-09-10T19:05:59.753 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.9963000000000001e-05 2024-09-10T19:05:59.753 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.753 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.753 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:05:59.753 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.827792+0000", 2024-09-10T19:05:59.754 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.1092999999999999e-05 2024-09-10T19:05:59.754 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.754 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:05:59.754 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.754 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.754 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.754 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.25706.0:142610 11.18 11:1b19b3fc:::benchmark_data_smithi018_181485_object5518:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e417)", 2024-09-10T19:05:59.754 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:05:03.864570+0000", 2024-09-10T19:05:59.754 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 54.809340495000001, 2024-09-10T19:05:59.754 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.96335404099999999, 2024-09-10T19:05:59.754 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:05:59.754 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:05:59.754 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:05:59.755 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.25706", 2024-09-10T19:05:59.755 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/42081399", 2024-09-10T19:05:59.755 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 142610 2024-09-10T19:05:59.755 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.755 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:05:59.755 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.755 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:05:59.755 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.864570+0000", 2024-09-10T19:05:59.755 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.755 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.755 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.755 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:05:59.755 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.864570+0000", 2024-09-10T19:05:59.755 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.756 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.756 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.756 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:05:59.756 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.864569+0000", 2024-09-10T19:05:59.756 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T19:05:59.756 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.756 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.756 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:05:59.756 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.864575+0000", 2024-09-10T19:05:59.756 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.3779999999999998e-06 2024-09-10T19:05:59.756 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.756 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.756 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:05:59.757 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.864588+0000", 2024-09-10T19:05:59.757 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.3158e-05 2024-09-10T19:05:59.757 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.757 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.757 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:05:59.757 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.864595+0000", 2024-09-10T19:05:59.757 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.6839999999999999e-06 2024-09-10T19:05:59.757 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.757 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.757 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.757 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.864645+0000", 2024-09-10T19:05:59.757 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.9985999999999999e-05 2024-09-10T19:05:59.757 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.758 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.758 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:05:59.758 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.864652+0000", 2024-09-10T19:05:59.758 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.883e-06 2024-09-10T19:05:59.758 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.758 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.758 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.758 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.811087+0000", 2024-09-10T19:05:59.758 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.94643493599999995 2024-09-10T19:05:59.758 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.758 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.758 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:05:59.758 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.811147+0000", 2024-09-10T19:05:59.758 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.0424000000000003e-05 2024-09-10T19:05:59.759 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.759 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.759 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 5,6", 2024-09-10T19:05:59.759 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.811288+0000", 2024-09-10T19:05:59.759 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00014125699999999999 2024-09-10T19:05:59.759 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.759 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.759 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:05:59.759 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.816475+0000", 2024-09-10T19:05:59.759 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.005187026 2024-09-10T19:05:59.759 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.759 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.759 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.760 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.827516+0000", 2024-09-10T19:05:59.760 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.011040559 2024-09-10T19:05:59.760 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.760 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.760 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.760 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.827839+0000", 2024-09-10T19:05:59.760 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00032333399999999999 2024-09-10T19:05:59.760 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.760 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.760 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:05:59.760 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.827885+0000", 2024-09-10T19:05:59.760 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.5309000000000003e-05 2024-09-10T19:05:59.760 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.761 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.761 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:05:59.761 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.827924+0000", 2024-09-10T19:05:59.761 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.9304999999999997e-05 2024-09-10T19:05:59.761 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.761 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:05:59.761 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.761 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.761 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.761 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.25706.0:142617 11.18 11:19b748cb:::benchmark_data_smithi018_181485_object5525:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e417)", 2024-09-10T19:05:59.761 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:05:03.868602+0000", 2024-09-10T19:05:59.761 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 54.805308615000001, 2024-09-10T19:05:59.761 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.95956408800000004, 2024-09-10T19:05:59.762 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:05:59.762 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:05:59.762 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:05:59.762 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.25706", 2024-09-10T19:05:59.762 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/42081399", 2024-09-10T19:05:59.762 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 142617 2024-09-10T19:05:59.762 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.762 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:05:59.762 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.762 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:05:59.762 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.868602+0000", 2024-09-10T19:05:59.762 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.762 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.763 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.763 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:05:59.763 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.868602+0000", 2024-09-10T19:05:59.763 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.763 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.763 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.763 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:05:59.763 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.868601+0000", 2024-09-10T19:05:59.763 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T19:05:59.763 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.763 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.763 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:05:59.763 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.868606+0000", 2024-09-10T19:05:59.764 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.7989999999999999e-06 2024-09-10T19:05:59.764 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.764 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.764 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:05:59.764 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.868616+0000", 2024-09-10T19:05:59.764 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.1870000000000003e-06 2024-09-10T19:05:59.764 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.764 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.764 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:05:59.764 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.868621+0000", 2024-09-10T19:05:59.764 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.2959999999999998e-06 2024-09-10T19:05:59.764 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.764 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.765 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.765 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.868671+0000", 2024-09-10T19:05:59.765 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.9870000000000002e-05 2024-09-10T19:05:59.765 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.765 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.765 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:05:59.765 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.868678+0000", 2024-09-10T19:05:59.765 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.3919999999999997e-06 2024-09-10T19:05:59.765 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.765 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.765 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.765 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.812090+0000", 2024-09-10T19:05:59.765 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.94341146399999998 2024-09-10T19:05:59.766 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.766 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.766 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:05:59.766 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.812195+0000", 2024-09-10T19:05:59.766 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00010494599999999999 2024-09-10T19:05:59.766 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.766 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.766 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 5,6", 2024-09-10T19:05:59.766 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.812384+0000", 2024-09-10T19:05:59.766 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00018928900000000001 2024-09-10T19:05:59.766 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.766 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.767 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:05:59.767 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.816481+0000", 2024-09-10T19:05:59.767 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0040969459999999997 2024-09-10T19:05:59.767 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.767 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.767 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.767 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.827979+0000", 2024-09-10T19:05:59.767 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.011497729999999999 2024-09-10T19:05:59.767 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.767 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.767 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.767 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.828102+0000", 2024-09-10T19:05:59.767 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000123378 2024-09-10T19:05:59.767 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.768 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.768 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:05:59.768 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.828145+0000", 2024-09-10T19:05:59.768 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.3365999999999998e-05 2024-09-10T19:05:59.768 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.768 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.768 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:05:59.768 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.828166+0000", 2024-09-10T19:05:59.768 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.05e-05 2024-09-10T19:05:59.768 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.768 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:05:59.768 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.768 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.769 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.769 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.25706.0:142631 11.18 11:1878a6a7:::benchmark_data_smithi018_181485_object5539:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e417)", 2024-09-10T19:05:59.769 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:05:03.875270+0000", 2024-09-10T19:05:59.769 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 54.798640472000002, 2024-09-10T19:05:59.769 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.95311856399999995, 2024-09-10T19:05:59.769 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:05:59.769 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:05:59.769 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:05:59.769 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.25706", 2024-09-10T19:05:59.770 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/42081399", 2024-09-10T19:05:59.770 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 142631 2024-09-10T19:05:59.770 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.770 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:05:59.770 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.770 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:05:59.770 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.875270+0000", 2024-09-10T19:05:59.770 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.770 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.770 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.770 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:05:59.770 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.875270+0000", 2024-09-10T19:05:59.770 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:05:59.771 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.771 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.771 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:05:59.771 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.875269+0000", 2024-09-10T19:05:59.771 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T19:05:59.771 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.771 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.771 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:05:59.771 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.875274+0000", 2024-09-10T19:05:59.771 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.8629999999999999e-06 2024-09-10T19:05:59.771 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.771 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.771 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:05:59.771 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.875283+0000", 2024-09-10T19:05:59.772 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.2269999999999999e-06 2024-09-10T19:05:59.772 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.772 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.772 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:05:59.772 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.875289+0000", 2024-09-10T19:05:59.772 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.212e-06 2024-09-10T19:05:59.772 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.772 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.772 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.772 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.875334+0000", 2024-09-10T19:05:59.772 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.4657999999999998e-05 2024-09-10T19:05:59.772 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.773 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.773 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:05:59.773 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:03.875340+0000", 2024-09-10T19:05:59.773 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.4420000000000003e-06 2024-09-10T19:05:59.773 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.773 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.773 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:05:59.773 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.813288+0000", 2024-09-10T19:05:59.773 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.93794714099999998 2024-09-10T19:05:59.773 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.773 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.774 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:05:59.774 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.813403+0000", 2024-09-10T19:05:59.774 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00011592 2024-09-10T19:05:59.774 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.774 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.774 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 5,6", 2024-09-10T19:05:59.774 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.813590+0000", 2024-09-10T19:05:59.774 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000186226 2024-09-10T19:05:59.774 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.774 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.774 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:05:59.774 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.818724+0000", 2024-09-10T19:05:59.774 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0051341310000000001 2024-09-10T19:05:59.774 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.775 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.775 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.775 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.828213+0000", 2024-09-10T19:05:59.775 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00948945 2024-09-10T19:05:59.775 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.775 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.775 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:05:59.775 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.828320+0000", 2024-09-10T19:05:59.775 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000107163 2024-09-10T19:05:59.775 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.775 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.775 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:05:59.775 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.828353+0000", 2024-09-10T19:05:59.775 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.2654999999999998e-05 2024-09-10T19:05:59.776 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:05:59.776 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:05:59.776 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:05:59.776 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:04.828388+0000", 2024-09-10T19:05:59.776 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.5361999999999998e-05 2024-09-10T19:05:59.776 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.776 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:05:59.776 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.776 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:05:59.776 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:05:59.776 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T19:05:59.777 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:05:59.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:59.817 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:05:59.817 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:05:59.918 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:05:59.944 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:05:59.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:05:59.945 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:00.046 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:00.071 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:00.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:00.072 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:00.173 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:00.202 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:00.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:00.203 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:00.304 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:00.339 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:00.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:00.340 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:00.444 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:00.477 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:00.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:00.478 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:00.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:06:00 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:06:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:06:00.580 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:00.586 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:06:00.601 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is unset 2024-09-10T19:06:00.650 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:00.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:00.653 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:00.754 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:00.788 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:00.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:00.789 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:00.890 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:00.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:00.924 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:00.924 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:00.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:00 smithi018 ceph-mon[40196]: pgmap v972: 178 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 176 active+clean; 3.9 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.45k op/s; 2540/190716 objects misplaced (1.332%); 6.7 MiB/s, 107 objects/s recovering 2024-09-10T19:06:00.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:00 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2069617708' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:06:00.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:00 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:06:00.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:00 smithi018 ceph-mon[29336]: pgmap v972: 178 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 176 active+clean; 3.9 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.45k op/s; 2540/190716 objects misplaced (1.332%); 6.7 MiB/s, 107 objects/s recovering 2024-09-10T19:06:00.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:00 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2069617708' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:06:00.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:00 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:06:00.994 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:00 smithi103 ceph-mon[33163]: pgmap v972: 178 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 176 active+clean; 3.9 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 1.45k op/s; 2540/190716 objects misplaced (1.332%); 6.7 MiB/s, 107 objects/s recovering 2024-09-10T19:06:00.994 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:00 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2069617708' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:06:00.994 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:00 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:06:01.025 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:01.048 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:01.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:01.049 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:01.149 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:01.170 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:01.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:01.171 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:01.272 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:01.296 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:01.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:01.297 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:01.315 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:06:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:06:00] "GET /metrics HTTP/1.1" 200 224490 "" "Prometheus/2.43.0" 2024-09-10T19:06:01.397 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:01.420 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:01.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:01.421 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:01.423 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T19:06:01.424 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T19:06:01.429 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T19:06:01.429 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T19:06:01.444 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T19:06:01.444 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T19:06:01.459 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T19:06:01.459 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T19:06:01.475 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T19:06:01.475 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T19:06:01.491 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T19:06:01.491 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T19:06:01.503 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-10T19:06:01.506 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T19:06:01.506 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T19:06:01.522 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:01.522 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T19:06:01.523 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T19:06:01.567 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:01.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:01.568 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:01.669 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:01.707 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:01.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:01.708 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:01.775 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:01 smithi103 ceph-mon[33163]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:06:01.776 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:01 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T19:06:01.776 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:01 smithi103 ceph-mon[33163]: osdmap e438: 8 total, 8 up, 6 in 2024-09-10T19:06:01.776 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:01 smithi103 ceph-mon[33163]: 2.0 scrub starts 2024-09-10T19:06:01.776 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:01 smithi103 ceph-mon[33163]: 2.0 scrub ok 2024-09-10T19:06:01.776 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:01 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1065566701' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:06:01.776 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:01 smithi103 ceph-mon[33163]: osdmap e439: 8 total, 8 up, 6 in 2024-09-10T19:06:01.810 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:01.847 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:01.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:01.848 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:01.856 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:06:01.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:01 smithi018 ceph-mon[40196]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:06:01.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:01 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T19:06:01.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:01 smithi018 ceph-mon[40196]: osdmap e438: 8 total, 8 up, 6 in 2024-09-10T19:06:01.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:01 smithi018 ceph-mon[40196]: 2.0 scrub starts 2024-09-10T19:06:01.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:01 smithi018 ceph-mon[40196]: 2.0 scrub ok 2024-09-10T19:06:01.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:01 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1065566701' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:06:01.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:01 smithi018 ceph-mon[40196]: osdmap e439: 8 total, 8 up, 6 in 2024-09-10T19:06:01.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:01 smithi018 ceph-mon[29336]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:06:01.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:01 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T19:06:01.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:01 smithi018 ceph-mon[29336]: osdmap e438: 8 total, 8 up, 6 in 2024-09-10T19:06:01.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:01 smithi018 ceph-mon[29336]: 2.0 scrub starts 2024-09-10T19:06:01.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:01 smithi018 ceph-mon[29336]: 2.0 scrub ok 2024-09-10T19:06:01.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:01 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1065566701' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:06:01.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:01 smithi018 ceph-mon[29336]: osdmap e439: 8 total, 8 up, 6 in 2024-09-10T19:06:01.949 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:01.986 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:01.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:01.987 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:02.088 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:02.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:02.128 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:02.128 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:02.229 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:02.263 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:02.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:02.264 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:02.365 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:02.405 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:02.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:02.406 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:02.507 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:02.539 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:02.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:02.541 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:02.642 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:02.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:02.684 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:02.684 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:02.759 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:02 smithi018 ceph-mon[40196]: pgmap v974: 178 pgs: 1 peering, 1 active+remapped+backfill_toofull, 176 active+clean; 3.7 GiB data, 20 GiB used, 517 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 1.45k op/s; 1837/180318 objects misplaced (1.019%); 6.7 MiB/s, 107 objects/s recovering 2024-09-10T19:06:02.760 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:02 smithi018 ceph-mon[40196]: 4.5 scrub starts 2024-09-10T19:06:02.760 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:02 smithi018 ceph-mon[40196]: 4.5 scrub ok 2024-09-10T19:06:02.760 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:02 smithi018 ceph-mon[40196]: 2.8 scrub starts 2024-09-10T19:06:02.760 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:02 smithi018 ceph-mon[40196]: 2.8 scrub ok 2024-09-10T19:06:02.760 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:02 smithi018 ceph-mon[29336]: pgmap v974: 178 pgs: 1 peering, 1 active+remapped+backfill_toofull, 176 active+clean; 3.7 GiB data, 20 GiB used, 517 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 1.45k op/s; 1837/180318 objects misplaced (1.019%); 6.7 MiB/s, 107 objects/s recovering 2024-09-10T19:06:02.760 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:02 smithi018 ceph-mon[29336]: 4.5 scrub starts 2024-09-10T19:06:02.760 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:02 smithi018 ceph-mon[29336]: 4.5 scrub ok 2024-09-10T19:06:02.760 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:02 smithi018 ceph-mon[29336]: 2.8 scrub starts 2024-09-10T19:06:02.760 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:02 smithi018 ceph-mon[29336]: 2.8 scrub ok 2024-09-10T19:06:02.789 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:02.828 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:02.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:02.829 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:02.930 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:02.964 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:02.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:02.965 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:03.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:02 smithi103 ceph-mon[33163]: pgmap v974: 178 pgs: 1 peering, 1 active+remapped+backfill_toofull, 176 active+clean; 3.7 GiB data, 20 GiB used, 517 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 1.45k op/s; 1837/180318 objects misplaced (1.019%); 6.7 MiB/s, 107 objects/s recovering 2024-09-10T19:06:03.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:02 smithi103 ceph-mon[33163]: 4.5 scrub starts 2024-09-10T19:06:03.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:02 smithi103 ceph-mon[33163]: 4.5 scrub ok 2024-09-10T19:06:03.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:02 smithi103 ceph-mon[33163]: 2.8 scrub starts 2024-09-10T19:06:03.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:02 smithi103 ceph-mon[33163]: 2.8 scrub ok 2024-09-10T19:06:03.065 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:03.103 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:03.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:03.105 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:03.205 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:03.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:03.239 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:03.239 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:03.340 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:03.384 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:03.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:03.385 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:03.486 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:03.526 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:03.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:03.528 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:03.628 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:03.664 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:03.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:03.665 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:03.751 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T19:06:03.751 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T19:06:03.766 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:03.808 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:03.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:03.809 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:03.903 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:03 smithi018 ceph-mon[40196]: 2.c scrub starts 2024-09-10T19:06:03.903 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:03 smithi018 ceph-mon[40196]: 2.c scrub ok 2024-09-10T19:06:03.903 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:03 smithi018 ceph-mon[40196]: 2.4 scrub starts 2024-09-10T19:06:03.903 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:03 smithi018 ceph-mon[40196]: 2.4 scrub ok 2024-09-10T19:06:03.904 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:03 smithi018 ceph-mon[29336]: 2.c scrub starts 2024-09-10T19:06:03.904 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:03 smithi018 ceph-mon[29336]: 2.c scrub ok 2024-09-10T19:06:03.904 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:03 smithi018 ceph-mon[29336]: 2.4 scrub starts 2024-09-10T19:06:03.904 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:03 smithi018 ceph-mon[29336]: 2.4 scrub ok 2024-09-10T19:06:03.910 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:03.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:03.956 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:03.956 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:04.026 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:03 smithi103 ceph-mon[33163]: 2.c scrub starts 2024-09-10T19:06:04.026 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:03 smithi103 ceph-mon[33163]: 2.c scrub ok 2024-09-10T19:06:04.026 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:03 smithi103 ceph-mon[33163]: 2.4 scrub starts 2024-09-10T19:06:04.026 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:03 smithi103 ceph-mon[33163]: 2.4 scrub ok 2024-09-10T19:06:04.057 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:04.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:04.104 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:04.105 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:04.206 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:04.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:04.243 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:04.243 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:04.344 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:04.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:04.376 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:04.376 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:04.477 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:04.508 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-10T19:06:04.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:04.510 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:04.510 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:04.510 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T19:06:04.611 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:04.644 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:04.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:04.645 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:04.746 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:04.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:04 smithi103 ceph-mon[33163]: 6.e scrub starts 2024-09-10T19:06:04.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:04 smithi103 ceph-mon[33163]: 6.e scrub ok 2024-09-10T19:06:04.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:04 smithi103 ceph-mon[33163]: 5.f scrub starts 2024-09-10T19:06:04.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:04 smithi103 ceph-mon[33163]: 5.f scrub ok 2024-09-10T19:06:04.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:04 smithi103 ceph-mon[33163]: pgmap v976: 178 pgs: 1 peering, 1 active+remapped+backfill_toofull, 176 active+clean; 3.5 GiB data, 19 GiB used, 517 GiB / 536 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.52k op/s; 1837/174867 objects misplaced (1.051%); 6.7 MiB/s, 107 objects/s recovering 2024-09-10T19:06:04.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:04.797 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:04.797 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:04.898 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:04.921 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:06:04.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:04 smithi018 ceph-mon[40196]: 6.e scrub starts 2024-09-10T19:06:04.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:04 smithi018 ceph-mon[40196]: 6.e scrub ok 2024-09-10T19:06:04.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:04 smithi018 ceph-mon[40196]: 5.f scrub starts 2024-09-10T19:06:04.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:04 smithi018 ceph-mon[40196]: 5.f scrub ok 2024-09-10T19:06:04.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:04 smithi018 ceph-mon[40196]: pgmap v976: 178 pgs: 1 peering, 1 active+remapped+backfill_toofull, 176 active+clean; 3.5 GiB data, 19 GiB used, 517 GiB / 536 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.52k op/s; 1837/174867 objects misplaced (1.051%); 6.7 MiB/s, 107 objects/s recovering 2024-09-10T19:06:04.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:04 smithi018 ceph-mon[29336]: 6.e scrub starts 2024-09-10T19:06:04.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:04 smithi018 ceph-mon[29336]: 6.e scrub ok 2024-09-10T19:06:04.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:04 smithi018 ceph-mon[29336]: 5.f scrub starts 2024-09-10T19:06:04.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:04 smithi018 ceph-mon[29336]: 5.f scrub ok 2024-09-10T19:06:04.947 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:04 smithi018 ceph-mon[29336]: pgmap v976: 178 pgs: 1 peering, 1 active+remapped+backfill_toofull, 176 active+clean; 3.5 GiB data, 19 GiB used, 517 GiB / 536 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.52k op/s; 1837/174867 objects misplaced (1.051%); 6.7 MiB/s, 107 objects/s recovering 2024-09-10T19:06:04.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:04.969 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:04.970 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:05.000 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:06:05.070 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:05.110 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:05.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:05.111 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:05.212 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:05.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:05.246 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:05.246 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:05.347 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:05.384 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:05.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:05.385 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:05.486 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:05.492 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-10T19:06:05.516 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:05.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:05.517 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:05.618 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:05.638 INFO:teuthology.orchestra.run.smithi018.stderr:reweighted osd.4 to 1 (10000) 2024-09-10T19:06:05.645 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:05.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:05.646 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:05.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:05 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3513428384' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-10T19:06:05.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:05 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-10T19:06:05.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:05 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3513428384' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-10T19:06:05.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:05 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-10T19:06:05.747 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:05.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:05.789 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:05.789 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:05.890 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:05.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:05.924 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:05.924 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:06.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:05 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3513428384' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-10T19:06:06.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:05 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-10T19:06:06.025 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:06.055 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:06.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:06.057 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:06.158 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:06.192 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:06.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:06.193 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:06.294 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:06.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:06.325 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:06.325 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:06.426 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:06.454 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:06.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:06.455 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:06.556 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:06.594 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:06.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:06.595 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:06.696 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:06.717 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:06.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:06.718 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:06.819 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:06.855 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:06.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:06.856 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:06.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:06 smithi018 ceph-mon[40196]: pgmap v977: 178 pgs: 1 peering, 1 active+remapped+backfill_toofull, 176 active+clean; 3.5 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 996 op/s; 1837/170772 objects misplaced (1.076%); 2.5 MiB/s, 40 objects/s recovering 2024-09-10T19:06:06.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:06 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-10T19:06:06.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:06 smithi018 ceph-mon[40196]: osdmap e440: 8 total, 8 up, 6 in 2024-09-10T19:06:06.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:06 smithi018 ceph-mon[29336]: pgmap v977: 178 pgs: 1 peering, 1 active+remapped+backfill_toofull, 176 active+clean; 3.5 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 996 op/s; 1837/170772 objects misplaced (1.076%); 2.5 MiB/s, 40 objects/s recovering 2024-09-10T19:06:06.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:06 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-10T19:06:06.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:06 smithi018 ceph-mon[29336]: osdmap e440: 8 total, 8 up, 6 in 2024-09-10T19:06:06.957 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:06.998 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:06.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:06.999 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:07.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:06 smithi103 ceph-mon[33163]: pgmap v977: 178 pgs: 1 peering, 1 active+remapped+backfill_toofull, 176 active+clean; 3.5 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 996 op/s; 1837/170772 objects misplaced (1.076%); 2.5 MiB/s, 40 objects/s recovering 2024-09-10T19:06:07.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:06 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-10T19:06:07.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:06 smithi103 ceph-mon[33163]: osdmap e440: 8 total, 8 up, 6 in 2024-09-10T19:06:07.100 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:07.139 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:07.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:07.141 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:07.242 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:07.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:07.274 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:07.274 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:07.375 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:07.412 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:07.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:07.413 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:07.514 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:07.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:06:07 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:06:07.553 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:07.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:07.554 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:07.654 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:07.659 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd reweight 5 1 2024-09-10T19:06:07.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:07.690 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:07.690 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:07.694 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:06:07.792 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:07.796 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:07 smithi018 ceph-mon[29336]: 2.17 scrub starts 2024-09-10T19:06:07.796 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:07 smithi018 ceph-mon[29336]: 2.17 scrub ok 2024-09-10T19:06:07.796 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:07 smithi018 ceph-mon[29336]: 6.16 scrub starts 2024-09-10T19:06:07.796 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:07 smithi018 ceph-mon[29336]: 6.16 scrub ok 2024-09-10T19:06:07.799 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:07 smithi018 ceph-mon[40196]: 2.17 scrub starts 2024-09-10T19:06:07.799 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:07 smithi018 ceph-mon[40196]: 2.17 scrub ok 2024-09-10T19:06:07.799 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:07 smithi018 ceph-mon[40196]: 6.16 scrub starts 2024-09-10T19:06:07.799 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:07 smithi018 ceph-mon[40196]: 6.16 scrub ok 2024-09-10T19:06:07.837 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:07.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:07.838 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:07.939 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:07.985 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:07.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:07.986 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:08.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:07 smithi103 ceph-mon[33163]: 2.17 scrub starts 2024-09-10T19:06:08.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:07 smithi103 ceph-mon[33163]: 2.17 scrub ok 2024-09-10T19:06:08.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:07 smithi103 ceph-mon[33163]: 6.16 scrub starts 2024-09-10T19:06:08.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:07 smithi103 ceph-mon[33163]: 6.16 scrub ok 2024-09-10T19:06:08.087 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:08.129 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:08.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:08.130 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:08.231 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:08.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:08.261 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:08.261 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:08.332 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T19:06:08.333 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T19:06:08.362 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:08.365 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T19:06:08.365 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T19:06:08.391 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:08.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:08.392 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:08.402 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T19:06:08.403 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T19:06:08.438 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T19:06:08.439 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T19:06:08.472 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T19:06:08.473 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T19:06:08.493 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:08.502 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T19:06:08.502 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T19:06:08.522 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:08.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:08.523 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:08.540 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T19:06:08.541 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T19:06:08.574 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T19:06:08.574 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T19:06:08.624 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:08.657 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:08.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:08.659 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:08.759 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:08.771 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:08 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:08.771 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:08 smithi103 ceph-mon[33163]: 5.2 scrub starts 2024-09-10T19:06:08.771 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:08 smithi103 ceph-mon[33163]: pgmap v979: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 3.1 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.92k op/s; 1594/150900 objects misplaced (1.056%); 2.9 MiB/s, 47 objects/s recovering 2024-09-10T19:06:08.771 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:08 smithi103 ceph-mon[33163]: 5.2 scrub ok 2024-09-10T19:06:08.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:08 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1326804271' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:06:08.791 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:08.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:08.792 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:08.893 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:08.927 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:08.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:08.928 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:08.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:08 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:08.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:08 smithi018 ceph-mon[40196]: 5.2 scrub starts 2024-09-10T19:06:08.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:08 smithi018 ceph-mon[40196]: pgmap v979: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 3.1 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.92k op/s; 1594/150900 objects misplaced (1.056%); 2.9 MiB/s, 47 objects/s recovering 2024-09-10T19:06:08.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:08 smithi018 ceph-mon[40196]: 5.2 scrub ok 2024-09-10T19:06:08.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:08 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1326804271' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:06:08.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:08 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:08.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:08 smithi018 ceph-mon[29336]: 5.2 scrub starts 2024-09-10T19:06:08.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:08 smithi018 ceph-mon[29336]: pgmap v979: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 3.1 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.92k op/s; 1594/150900 objects misplaced (1.056%); 2.9 MiB/s, 47 objects/s recovering 2024-09-10T19:06:08.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:08 smithi018 ceph-mon[29336]: 5.2 scrub ok 2024-09-10T19:06:08.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:08 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1326804271' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:06:09.029 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:09.068 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:09.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:09.069 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:09.170 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:09.206 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:09.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:09.208 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:09.308 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:09.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:09.344 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:09.344 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:09.445 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:09.485 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:09.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:09.486 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:09.561 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T19:06:09.561 INFO:teuthology.orchestra.run.smithi103.stdout: "ops": [], 2024-09-10T19:06:09.561 INFO:teuthology.orchestra.run.smithi103.stdout: "complaint_time": 30, 2024-09-10T19:06:09.561 INFO:teuthology.orchestra.run.smithi103.stdout: "num_blocked_ops": 0 2024-09-10T19:06:09.562 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T19:06:09.587 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:09.624 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:09.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:09.625 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:09.726 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:09.730 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:06:09.765 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:09.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:09.766 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:09.867 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:09.872 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:09 smithi018 ceph-mon[40196]: 5.0 scrub starts 2024-09-10T19:06:09.872 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:09 smithi018 ceph-mon[40196]: 5.0 scrub ok 2024-09-10T19:06:09.872 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:09 smithi018 ceph-mon[40196]: 1.12 scrub starts 2024-09-10T19:06:09.872 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:09 smithi018 ceph-mon[40196]: 1.12 scrub ok 2024-09-10T19:06:09.872 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:09 smithi018 ceph-mon[40196]: 2.11 scrub starts 2024-09-10T19:06:09.873 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:09 smithi018 ceph-mon[40196]: 2.11 scrub ok 2024-09-10T19:06:09.873 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:09 smithi018 ceph-mon[40196]: pgmap v980: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 3.0 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.41k op/s; 1594/146556 objects misplaced (1.088%); 0 B/s, 0 objects/s recovering 2024-09-10T19:06:09.873 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:09 smithi018 ceph-mon[29336]: 5.0 scrub starts 2024-09-10T19:06:09.873 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:09 smithi018 ceph-mon[29336]: 5.0 scrub ok 2024-09-10T19:06:09.873 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:09 smithi018 ceph-mon[29336]: 1.12 scrub starts 2024-09-10T19:06:09.873 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:09 smithi018 ceph-mon[29336]: 1.12 scrub ok 2024-09-10T19:06:09.873 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:09 smithi018 ceph-mon[29336]: 2.11 scrub starts 2024-09-10T19:06:09.873 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:09 smithi018 ceph-mon[29336]: 2.11 scrub ok 2024-09-10T19:06:09.873 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:09 smithi018 ceph-mon[29336]: pgmap v980: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 3.0 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.41k op/s; 1594/146556 objects misplaced (1.088%); 0 B/s, 0 objects/s recovering 2024-09-10T19:06:09.899 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:09.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:09.900 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:10.001 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:10.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:09 smithi103 ceph-mon[33163]: 5.0 scrub starts 2024-09-10T19:06:10.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:09 smithi103 ceph-mon[33163]: 5.0 scrub ok 2024-09-10T19:06:10.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:09 smithi103 ceph-mon[33163]: 1.12 scrub starts 2024-09-10T19:06:10.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:09 smithi103 ceph-mon[33163]: 1.12 scrub ok 2024-09-10T19:06:10.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:09 smithi103 ceph-mon[33163]: 2.11 scrub starts 2024-09-10T19:06:10.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:09 smithi103 ceph-mon[33163]: 2.11 scrub ok 2024-09-10T19:06:10.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:09 smithi103 ceph-mon[33163]: pgmap v980: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 3.0 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.41k op/s; 1594/146556 objects misplaced (1.088%); 0 B/s, 0 objects/s recovering 2024-09-10T19:06:10.039 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:10.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:10.040 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:10.141 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:10.180 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:10.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:10.182 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:10.282 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:10.284 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-10T19:06:10.330 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:10.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:10.332 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:10.432 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:10.473 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:10.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:10.474 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:10.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:06:10 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:06:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:06:10.575 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:10.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:10.615 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:10.615 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:10.690 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:06:10.716 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:10.761 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:10.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:10.762 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:10.863 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:10.902 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:10.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:10.903 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:11.004 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:11.039 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:11.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:11.041 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:11.131 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:10 smithi103 ceph-mon[33163]: 6.11 scrub starts 2024-09-10T19:06:11.131 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:10 smithi103 ceph-mon[33163]: 6.11 scrub ok 2024-09-10T19:06:11.131 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:10 smithi103 ceph-mon[33163]: 5.c scrub starts 2024-09-10T19:06:11.131 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:10 smithi103 ceph-mon[33163]: 5.c scrub ok 2024-09-10T19:06:11.132 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:10 smithi103 ceph-mon[33163]: 5.6 scrub starts 2024-09-10T19:06:11.132 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:10 smithi103 ceph-mon[33163]: 5.6 scrub ok 2024-09-10T19:06:11.132 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:10 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:06:11.132 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:10 smithi018 ceph-mon[40196]: 6.11 scrub starts 2024-09-10T19:06:11.132 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:10 smithi018 ceph-mon[40196]: 6.11 scrub ok 2024-09-10T19:06:11.133 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:10 smithi018 ceph-mon[40196]: 5.c scrub starts 2024-09-10T19:06:11.133 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:10 smithi018 ceph-mon[40196]: 5.c scrub ok 2024-09-10T19:06:11.133 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:10 smithi018 ceph-mon[40196]: 5.6 scrub starts 2024-09-10T19:06:11.133 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:10 smithi018 ceph-mon[40196]: 5.6 scrub ok 2024-09-10T19:06:11.133 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:10 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:06:11.133 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:10 smithi018 ceph-mon[29336]: 6.11 scrub starts 2024-09-10T19:06:11.133 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:10 smithi018 ceph-mon[29336]: 6.11 scrub ok 2024-09-10T19:06:11.134 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:10 smithi018 ceph-mon[29336]: 5.c scrub starts 2024-09-10T19:06:11.134 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:10 smithi018 ceph-mon[29336]: 5.c scrub ok 2024-09-10T19:06:11.134 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:10 smithi018 ceph-mon[29336]: 5.6 scrub starts 2024-09-10T19:06:11.134 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:10 smithi018 ceph-mon[29336]: 5.6 scrub ok 2024-09-10T19:06:11.134 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:10 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:06:11.134 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:06:10 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:06:10] "GET /metrics HTTP/1.1" 200 224502 "" "Prometheus/2.43.0" 2024-09-10T19:06:11.141 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:11.177 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:11.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:11.178 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:11.279 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:11.313 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:11.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:11.314 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:11.415 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:11.454 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:11.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:11.456 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:11.557 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:11.585 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:11.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:11.586 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:11.687 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:11.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:11.728 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:11.729 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:11.731 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T19:06:11.830 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:11.863 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:11.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:11.864 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:11.965 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:12.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:12.012 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:12.012 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:12.113 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:12.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:12.146 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:12.146 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:12.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:11 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:06:12.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:11 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:06:12.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:11 smithi018 ceph-mon[40196]: osdmap e441: 8 total, 8 up, 6 in 2024-09-10T19:06:12.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:11 smithi018 ceph-mon[40196]: 2.1c scrub starts 2024-09-10T19:06:12.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:11 smithi018 ceph-mon[40196]: 2.1c scrub ok 2024-09-10T19:06:12.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:11 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:06:12.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:11 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:06:12.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:11 smithi018 ceph-mon[40196]: pgmap v982: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 2.7 GiB data, 17 GiB used, 520 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 1.72k op/s; 1329/133668 objects misplaced (0.994%); 0 B/s, 0 objects/s recovering 2024-09-10T19:06:12.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:11 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:06:12.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:11 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:06:12.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:11 smithi018 ceph-mon[29336]: osdmap e441: 8 total, 8 up, 6 in 2024-09-10T19:06:12.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:11 smithi018 ceph-mon[29336]: 2.1c scrub starts 2024-09-10T19:06:12.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:11 smithi018 ceph-mon[29336]: 2.1c scrub ok 2024-09-10T19:06:12.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:11 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:06:12.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:11 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:06:12.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:11 smithi018 ceph-mon[29336]: pgmap v982: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 2.7 GiB data, 17 GiB used, 520 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 1.72k op/s; 1329/133668 objects misplaced (0.994%); 0 B/s, 0 objects/s recovering 2024-09-10T19:06:12.247 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:12.250 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:11 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:06:12.250 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:11 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:06:12.250 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:11 smithi103 ceph-mon[33163]: osdmap e441: 8 total, 8 up, 6 in 2024-09-10T19:06:12.250 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:11 smithi103 ceph-mon[33163]: 2.1c scrub starts 2024-09-10T19:06:12.250 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:11 smithi103 ceph-mon[33163]: 2.1c scrub ok 2024-09-10T19:06:12.251 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:11 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:06:12.251 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:11 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:06:12.251 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:11 smithi103 ceph-mon[33163]: pgmap v982: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 2.7 GiB data, 17 GiB used, 520 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 1.72k op/s; 1329/133668 objects misplaced (0.994%); 0 B/s, 0 objects/s recovering 2024-09-10T19:06:12.288 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:12.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:12.289 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:12.390 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:12.427 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:12.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:12.428 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:12.529 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:12.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:12.565 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:12.565 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:12.666 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:12.703 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:12.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:12.705 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:12.718 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T19:06:12.718 INFO:teuthology.orchestra.run.smithi103.stdout: "size": 20, 2024-09-10T19:06:12.719 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 600, 2024-09-10T19:06:12.719 INFO:teuthology.orchestra.run.smithi103.stdout: "ops": [ 2024-09-10T19:06:12.719 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.719 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15354.0:178091 9.7 9:ea121bd7:::benchmark_data_smithi018_133594_object41339:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e285)", 2024-09-10T19:06:12.719 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:58:05.137375+0000", 2024-09-10T19:06:12.719 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 487.57722606900001, 2024-09-10T19:06:12.719 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.04106915, 2024-09-10T19:06:12.719 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:06:12.719 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "reached pg", 2024-09-10T19:06:12.719 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:06:12.719 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15354", 2024-09-10T19:06:12.719 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:06:12.720 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 178091 2024-09-10T19:06:12.720 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.720 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:06:12.720 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.720 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:06:12.720 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.137375+0000", 2024-09-10T19:06:12.720 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.720 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.720 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.720 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:06:12.720 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.137375+0000", 2024-09-10T19:06:12.721 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.721 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.721 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.721 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:06:12.721 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.137374+0000", 2024-09-10T19:06:12.721 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T19:06:12.721 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.721 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.721 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:06:12.721 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.137380+0000", 2024-09-10T19:06:12.721 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.4720000000000003e-06 2024-09-10T19:06:12.721 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.721 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.722 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:06:12.722 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.137389+0000", 2024-09-10T19:06:12.722 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.4790000000000002e-06 2024-09-10T19:06:12.722 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.722 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.722 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:06:12.722 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.137393+0000", 2024-09-10T19:06:12.722 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.9790000000000004e-06 2024-09-10T19:06:12.722 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.722 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.722 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.722 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.137424+0000", 2024-09-10T19:06:12.723 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.1068999999999997e-05 2024-09-10T19:06:12.723 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.723 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.723 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:06:12.723 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.137510+0000", 2024-09-10T19:06:12.723 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.5883000000000004e-05 2024-09-10T19:06:12.723 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.723 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.723 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 1,3,7", 2024-09-10T19:06:12.723 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.137592+0000", 2024-09-10T19:06:12.723 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.1625999999999994e-05 2024-09-10T19:06:12.723 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.724 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.724 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:06:12.724 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.140085+0000", 2024-09-10T19:06:12.724 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.002493168 2024-09-10T19:06:12.724 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.724 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.724 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:06:12.724 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.140339+0000", 2024-09-10T19:06:12.724 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000254035 2024-09-10T19:06:12.724 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.724 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.725 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:06:12.725 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.143554+0000", 2024-09-10T19:06:12.725 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.003215075 2024-09-10T19:06:12.725 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.725 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.725 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.725 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.148627+0000", 2024-09-10T19:06:12.725 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0050729 2024-09-10T19:06:12.725 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.725 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.725 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:06:12.726 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.148637+0000", 2024-09-10T19:06:12.726 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0052e-05 2024-09-10T19:06:12.726 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.726 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.726 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.726 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.178360+0000", 2024-09-10T19:06:12.726 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0297230610000001 2024-09-10T19:06:12.726 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.726 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.726 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:06:12.726 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.178444+0000", 2024-09-10T19:06:12.726 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.4282000000000004e-05 2024-09-10T19:06:12.726 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.727 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:06:12.727 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.727 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.727 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.727 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15354.0:178142 9.7 9:ecf42d8f:::benchmark_data_smithi018_133594_object41390:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e286)", 2024-09-10T19:06:12.727 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:58:05.245258+0000", 2024-09-10T19:06:12.727 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 487.46934337499999, 2024-09-10T19:06:12.727 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.95601217100000002, 2024-09-10T19:06:12.727 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:06:12.727 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:06:12.727 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:06:12.727 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15354", 2024-09-10T19:06:12.727 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:06:12.728 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 178142 2024-09-10T19:06:12.728 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.728 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:06:12.728 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.728 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:06:12.728 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.245258+0000", 2024-09-10T19:06:12.728 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.728 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.728 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.728 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:06:12.728 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.245258+0000", 2024-09-10T19:06:12.728 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.728 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.729 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.729 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:06:12.729 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.245257+0000", 2024-09-10T19:06:12.729 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T19:06:12.729 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.729 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.729 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:06:12.729 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.245262+0000", 2024-09-10T19:06:12.729 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.395e-06 2024-09-10T19:06:12.729 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.729 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.729 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:06:12.729 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.245276+0000", 2024-09-10T19:06:12.730 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.4122e-05 2024-09-10T19:06:12.730 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.730 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.730 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:06:12.730 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.245283+0000", 2024-09-10T19:06:12.730 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.3899999999999998e-06 2024-09-10T19:06:12.730 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.730 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.730 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.730 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.245359+0000", 2024-09-10T19:06:12.730 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.5957999999999993e-05 2024-09-10T19:06:12.730 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.730 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.731 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:06:12.731 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.245367+0000", 2024-09-10T19:06:12.731 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.8380000000000008e-06 2024-09-10T19:06:12.731 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.731 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.731 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.731 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.178478+0000", 2024-09-10T19:06:12.731 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.93311012599999998 2024-09-10T19:06:12.731 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.731 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.731 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:06:12.731 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.178559+0000", 2024-09-10T19:06:12.731 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.1964000000000003e-05 2024-09-10T19:06:12.732 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.732 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.732 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 1,3", 2024-09-10T19:06:12.732 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.178632+0000", 2024-09-10T19:06:12.732 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.2272000000000002e-05 2024-09-10T19:06:12.732 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.732 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.732 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:06:12.732 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.185456+0000", 2024-09-10T19:06:12.732 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0068239199999999998 2024-09-10T19:06:12.732 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.732 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.733 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:06:12.733 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.201040+0000", 2024-09-10T19:06:12.733 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.015584147 2024-09-10T19:06:12.733 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.733 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.733 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:06:12.733 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.201221+0000", 2024-09-10T19:06:12.733 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00018072800000000001 2024-09-10T19:06:12.733 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.733 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.733 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:06:12.733 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.201248+0000", 2024-09-10T19:06:12.733 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.7753999999999999e-05 2024-09-10T19:06:12.734 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.734 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.734 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:06:12.734 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.201270+0000", 2024-09-10T19:06:12.734 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.1594999999999999e-05 2024-09-10T19:06:12.735 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.735 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:06:12.735 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.735 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.735 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.735 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15354.0:178145 9.7 9:ed12a7f2:::benchmark_data_smithi018_133594_object41393:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e286)", 2024-09-10T19:06:12.735 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T18:58:05.248111+0000", 2024-09-10T19:06:12.735 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 487.46649011699998, 2024-09-10T19:06:12.735 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.95324090299999997, 2024-09-10T19:06:12.735 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:06:12.735 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:06:12.735 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:06:12.736 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15354", 2024-09-10T19:06:12.736 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/572380528", 2024-09-10T19:06:12.736 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 178145 2024-09-10T19:06:12.736 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.736 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:06:12.736 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.736 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:06:12.736 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.248111+0000", 2024-09-10T19:06:12.736 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.736 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.736 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.736 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:06:12.737 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.248111+0000", 2024-09-10T19:06:12.737 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.737 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.737 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.737 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:06:12.737 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.248110+0000", 2024-09-10T19:06:12.737 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T19:06:12.737 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.737 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.737 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:06:12.737 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.248115+0000", 2024-09-10T19:06:12.737 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.3229999999999997e-06 2024-09-10T19:06:12.737 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.738 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.738 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:06:12.738 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.248125+0000", 2024-09-10T19:06:12.738 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0122000000000001e-05 2024-09-10T19:06:12.738 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.738 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.738 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:06:12.738 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.248130+0000", 2024-09-10T19:06:12.738 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.3340000000000003e-06 2024-09-10T19:06:12.738 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.738 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.738 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.738 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.248171+0000", 2024-09-10T19:06:12.739 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.1183000000000002e-05 2024-09-10T19:06:12.739 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.739 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.739 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:06:12.739 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:05.248179+0000", 2024-09-10T19:06:12.739 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.3599999999999996e-06 2024-09-10T19:06:12.739 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.739 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.739 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.739 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.180559+0000", 2024-09-10T19:06:12.739 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.932380125 2024-09-10T19:06:12.739 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.740 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.740 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:06:12.740 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.180641+0000", 2024-09-10T19:06:12.740 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.1397999999999996e-05 2024-09-10T19:06:12.740 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.740 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.740 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 1,3", 2024-09-10T19:06:12.740 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.180733+0000", 2024-09-10T19:06:12.740 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.2374999999999996e-05 2024-09-10T19:06:12.740 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.740 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.740 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:06:12.740 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.188068+0000", 2024-09-10T19:06:12.741 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0073351120000000004 2024-09-10T19:06:12.741 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.741 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.741 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:06:12.741 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.201094+0000", 2024-09-10T19:06:12.741 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.013026072 2024-09-10T19:06:12.741 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.741 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.741 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:06:12.741 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.201298+0000", 2024-09-10T19:06:12.741 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00020397300000000001 2024-09-10T19:06:12.741 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.742 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.742 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:06:12.742 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.201325+0000", 2024-09-10T19:06:12.742 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.7114000000000002e-05 2024-09-10T19:06:12.742 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.742 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.742 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:06:12.742 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T18:58:06.201352+0000", 2024-09-10T19:06:12.742 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.6427e-05 2024-09-10T19:06:12.742 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.742 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:06:12.742 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.742 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.743 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.743 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15573.0:86363 10.9 10.14e8f969 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e338)", 2024-09-10T19:06:12.743 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:00:41.512921+0000", 2024-09-10T19:06:12.743 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 331.20167993600001, 2024-09-10T19:06:12.743 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.99603365899999996, 2024-09-10T19:06:12.743 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:06:12.743 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "reached pg", 2024-09-10T19:06:12.743 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:06:12.743 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15573", 2024-09-10T19:06:12.743 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:06:12.743 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 86363 2024-09-10T19:06:12.743 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.743 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:06:12.744 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.744 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:06:12.744 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.512921+0000", 2024-09-10T19:06:12.744 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.744 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.744 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.744 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:06:12.744 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.512921+0000", 2024-09-10T19:06:12.744 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.744 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.744 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.744 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:06:12.744 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.512919+0000", 2024-09-10T19:06:12.745 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999981 2024-09-10T19:06:12.745 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.745 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.745 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:06:12.745 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.512988+0000", 2024-09-10T19:06:12.745 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.8990000000000002e-05 2024-09-10T19:06:12.745 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.745 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.745 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:06:12.745 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.512999+0000", 2024-09-10T19:06:12.745 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.057e-05 2024-09-10T19:06:12.745 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.745 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.746 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:06:12.746 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.513005+0000", 2024-09-10T19:06:12.746 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.9410000000000004e-06 2024-09-10T19:06:12.746 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.746 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.746 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.746 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.519550+0000", 2024-09-10T19:06:12.746 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0065450090000000001 2024-09-10T19:06:12.746 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.746 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.746 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:06:12.746 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.519555+0000", 2024-09-10T19:06:12.747 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.1240000000000004e-06 2024-09-10T19:06:12.747 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.747 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.747 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.747 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:42.508945+0000", 2024-09-10T19:06:12.747 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.98938961700000005 2024-09-10T19:06:12.747 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.747 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.747 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:06:12.747 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:42.508955+0000", 2024-09-10T19:06:12.747 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0091e-05 2024-09-10T19:06:12.747 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.747 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:06:12.747 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.748 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.748 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.748 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15573.0:86373 10.9 10.aea01c89 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e338)", 2024-09-10T19:06:12.748 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:00:41.513019+0000", 2024-09-10T19:06:12.748 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 331.20158234899998, 2024-09-10T19:06:12.748 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.99596813100000003, 2024-09-10T19:06:12.748 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:06:12.748 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "reached pg", 2024-09-10T19:06:12.748 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:06:12.748 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15573", 2024-09-10T19:06:12.748 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:06:12.748 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 86373 2024-09-10T19:06:12.749 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.749 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:06:12.749 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.749 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:06:12.749 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.513019+0000", 2024-09-10T19:06:12.749 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.749 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.749 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.749 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:06:12.749 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.513019+0000", 2024-09-10T19:06:12.749 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.749 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.749 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.750 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:06:12.750 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.513018+0000", 2024-09-10T19:06:12.750 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999995 2024-09-10T19:06:12.750 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.750 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.750 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:06:12.750 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.513083+0000", 2024-09-10T19:06:12.750 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.4523999999999994e-05 2024-09-10T19:06:12.750 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.750 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.750 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:06:12.750 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.513091+0000", 2024-09-10T19:06:12.751 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.4530000000000002e-06 2024-09-10T19:06:12.751 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.751 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.751 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:06:12.751 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.513096+0000", 2024-09-10T19:06:12.751 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.215e-06 2024-09-10T19:06:12.751 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.751 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.751 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.751 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.519575+0000", 2024-09-10T19:06:12.751 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.006478518 2024-09-10T19:06:12.751 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.752 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.752 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:06:12.752 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.519579+0000", 2024-09-10T19:06:12.752 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.1819999999999999e-06 2024-09-10T19:06:12.752 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.752 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.752 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.752 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:42.508980+0000", 2024-09-10T19:06:12.752 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.98940066900000001 2024-09-10T19:06:12.752 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.752 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.752 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:06:12.752 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:42.508987+0000", 2024-09-10T19:06:12.753 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.0720000000000002e-06 2024-09-10T19:06:12.753 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.753 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:06:12.753 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.753 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.753 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.753 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15573.0:86376 10.9 10.5396bf79 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e338)", 2024-09-10T19:06:12.753 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:00:41.513102+0000", 2024-09-10T19:06:12.753 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 331.20149895999998, 2024-09-10T19:06:12.753 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.99591294200000002, 2024-09-10T19:06:12.753 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:06:12.753 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "reached pg", 2024-09-10T19:06:12.753 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:06:12.754 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15573", 2024-09-10T19:06:12.754 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:06:12.754 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 86376 2024-09-10T19:06:12.754 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.754 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:06:12.754 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.754 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:06:12.754 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.513102+0000", 2024-09-10T19:06:12.754 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.754 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.754 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.754 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:06:12.754 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.513102+0000", 2024-09-10T19:06:12.755 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.755 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.755 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.755 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:06:12.755 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.513102+0000", 2024-09-10T19:06:12.755 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999995 2024-09-10T19:06:12.755 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.755 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.755 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:06:12.755 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.513156+0000", 2024-09-10T19:06:12.755 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.4129999999999998e-05 2024-09-10T19:06:12.755 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.755 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.756 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:06:12.756 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.513163+0000", 2024-09-10T19:06:12.756 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.8450000000000003e-06 2024-09-10T19:06:12.756 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.756 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.756 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:06:12.756 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.513169+0000", 2024-09-10T19:06:12.756 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.7209999999999999e-06 2024-09-10T19:06:12.756 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.756 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.756 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.756 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.519598+0000", 2024-09-10T19:06:12.756 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0064288799999999997 2024-09-10T19:06:12.757 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.757 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.757 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:06:12.757 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.519602+0000", 2024-09-10T19:06:12.757 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.755e-06 2024-09-10T19:06:12.757 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.757 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.757 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.757 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:42.509005+0000", 2024-09-10T19:06:12.757 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.98940315400000001 2024-09-10T19:06:12.757 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.757 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.757 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:06:12.758 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:42.509015+0000", 2024-09-10T19:06:12.758 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0067e-05 2024-09-10T19:06:12.758 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.758 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:06:12.758 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.758 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.758 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.758 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15573.0:86398 10.9 10.40eac979 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-10T19:06:12.758 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:00:41.524392+0000", 2024-09-10T19:06:12.758 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 331.190209138, 2024-09-10T19:06:12.758 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.98464817400000004, 2024-09-10T19:06:12.758 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:06:12.758 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "reached pg", 2024-09-10T19:06:12.758 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:06:12.759 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15573", 2024-09-10T19:06:12.759 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:06:12.759 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 86398 2024-09-10T19:06:12.759 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.759 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:06:12.759 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.759 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:06:12.759 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.524392+0000", 2024-09-10T19:06:12.759 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.759 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.759 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.759 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:06:12.759 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.524392+0000", 2024-09-10T19:06:12.760 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.760 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.760 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.760 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:06:12.760 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.524391+0000", 2024-09-10T19:06:12.760 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T19:06:12.760 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.760 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.760 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:06:12.760 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.524444+0000", 2024-09-10T19:06:12.760 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.3287e-05 2024-09-10T19:06:12.760 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.761 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.761 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:06:12.761 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.524454+0000", 2024-09-10T19:06:12.761 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0283e-05 2024-09-10T19:06:12.761 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.761 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.761 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:06:12.761 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.524460+0000", 2024-09-10T19:06:12.761 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.4480000000000002e-06 2024-09-10T19:06:12.761 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.761 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.761 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.762 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.524500+0000", 2024-09-10T19:06:12.762 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.9839000000000003e-05 2024-09-10T19:06:12.762 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.762 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.762 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:06:12.762 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.524506+0000", 2024-09-10T19:06:12.762 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.4840000000000001e-06 2024-09-10T19:06:12.762 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.762 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.762 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.762 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:42.509032+0000", 2024-09-10T19:06:12.762 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.98452615799999998 2024-09-10T19:06:12.762 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.762 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.763 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:06:12.763 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:42.509040+0000", 2024-09-10T19:06:12.763 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.7309999999999993e-06 2024-09-10T19:06:12.763 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.763 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:06:12.763 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.763 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.763 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.763 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15573.0:86426 10.9 10.2941d579 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-10T19:06:12.763 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:00:41.551544+0000", 2024-09-10T19:06:12.763 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 331.16305659900002, 2024-09-10T19:06:12.763 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.95752248500000003, 2024-09-10T19:06:12.763 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:06:12.764 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "reached pg", 2024-09-10T19:06:12.764 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:06:12.764 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15573", 2024-09-10T19:06:12.764 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:06:12.764 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 86426 2024-09-10T19:06:12.764 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.764 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:06:12.764 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.764 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:06:12.765 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.551544+0000", 2024-09-10T19:06:12.765 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.765 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.765 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.765 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:06:12.765 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.551544+0000", 2024-09-10T19:06:12.765 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.765 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.765 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.765 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:06:12.765 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.551543+0000", 2024-09-10T19:06:12.766 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T19:06:12.766 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.766 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.766 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:06:12.766 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.551594+0000", 2024-09-10T19:06:12.767 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.02e-05 2024-09-10T19:06:12.767 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.767 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.767 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:06:12.767 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.551607+0000", 2024-09-10T19:06:12.767 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.3236e-05 2024-09-10T19:06:12.767 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.767 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.767 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:06:12.767 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.551612+0000", 2024-09-10T19:06:12.767 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.8470000000000003e-06 2024-09-10T19:06:12.767 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.768 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.768 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.768 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.551658+0000", 2024-09-10T19:06:12.768 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.6437999999999998e-05 2024-09-10T19:06:12.768 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.768 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.768 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:06:12.768 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.551679+0000", 2024-09-10T19:06:12.768 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.0531e-05 2024-09-10T19:06:12.768 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.768 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.769 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.769 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:42.509060+0000", 2024-09-10T19:06:12.769 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.95738168199999996 2024-09-10T19:06:12.769 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.769 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.769 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:06:12.769 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:42.509067+0000", 2024-09-10T19:06:12.769 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.7129999999999999e-06 2024-09-10T19:06:12.769 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.769 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:06:12.769 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.769 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.770 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.770 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15573.0:86433 10.9 10.60a2e869 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-10T19:06:12.770 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:00:41.555910+0000", 2024-09-10T19:06:12.770 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 331.15869128399999, 2024-09-10T19:06:12.770 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.95318143700000002, 2024-09-10T19:06:12.770 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:06:12.770 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "reached pg", 2024-09-10T19:06:12.770 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:06:12.770 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15573", 2024-09-10T19:06:12.770 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:06:12.770 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 86433 2024-09-10T19:06:12.770 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.771 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:06:12.771 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.771 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:06:12.771 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.555910+0000", 2024-09-10T19:06:12.771 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.771 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.771 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.771 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:06:12.771 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.555910+0000", 2024-09-10T19:06:12.771 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.771 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.771 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.771 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:06:12.772 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.555909+0000", 2024-09-10T19:06:12.772 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T19:06:12.772 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.772 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.772 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:06:12.772 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.555961+0000", 2024-09-10T19:06:12.772 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.1706000000000003e-05 2024-09-10T19:06:12.772 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.772 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.772 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:06:12.772 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.555969+0000", 2024-09-10T19:06:12.772 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.4619999999999996e-06 2024-09-10T19:06:12.773 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.773 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.773 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:06:12.773 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.555973+0000", 2024-09-10T19:06:12.773 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.07e-06 2024-09-10T19:06:12.773 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.773 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.773 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.773 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.556006+0000", 2024-09-10T19:06:12.773 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.2369000000000001e-05 2024-09-10T19:06:12.773 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.773 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.774 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:06:12.774 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.556010+0000", 2024-09-10T19:06:12.774 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.4290000000000003e-06 2024-09-10T19:06:12.774 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.774 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.774 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.774 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:42.509084+0000", 2024-09-10T19:06:12.774 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.95307423000000002 2024-09-10T19:06:12.774 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.774 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.774 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:06:12.774 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:42.509091+0000", 2024-09-10T19:06:12.775 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.9800000000000001e-06 2024-09-10T19:06:12.775 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.775 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:06:12.775 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.775 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.775 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.775 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15573.0:86445 10.9 10.1be393f9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-10T19:06:12.775 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:00:41.574133+0000", 2024-09-10T19:06:12.775 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 331.14046856099998, 2024-09-10T19:06:12.775 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.93648039400000005, 2024-09-10T19:06:12.775 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:06:12.775 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "reached pg", 2024-09-10T19:06:12.776 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:06:12.776 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15573", 2024-09-10T19:06:12.776 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:06:12.776 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 86445 2024-09-10T19:06:12.776 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.776 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:06:12.776 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.776 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:06:12.776 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.574133+0000", 2024-09-10T19:06:12.776 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.776 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.776 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.777 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:06:12.777 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.574133+0000", 2024-09-10T19:06:12.777 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.777 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.777 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.777 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:06:12.777 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.574132+0000", 2024-09-10T19:06:12.777 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T19:06:12.777 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.777 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.777 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:06:12.777 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.574189+0000", 2024-09-10T19:06:12.777 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.7184e-05 2024-09-10T19:06:12.777 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.778 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.778 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:06:12.778 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.574197+0000", 2024-09-10T19:06:12.778 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.8329999999999994e-06 2024-09-10T19:06:12.778 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.778 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.778 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:06:12.778 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.574200+0000", 2024-09-10T19:06:12.778 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.585e-06 2024-09-10T19:06:12.778 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.778 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.778 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.778 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.574230+0000", 2024-09-10T19:06:12.779 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 2.9411999999999999e-05 2024-09-10T19:06:12.779 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.779 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.779 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:06:12.779 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:41.574234+0000", 2024-09-10T19:06:12.779 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.2889999999999999e-06 2024-09-10T19:06:12.779 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.779 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.779 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.779 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:42.510605+0000", 2024-09-10T19:06:12.779 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.93637043099999995 2024-09-10T19:06:12.779 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.779 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.780 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:06:12.780 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:00:42.510613+0000", 2024-09-10T19:06:12.780 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.4039999999999995e-06 2024-09-10T19:06:12.780 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.780 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:06:12.780 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.780 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.780 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.780 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15573.0:139848 10.9 10:93a3c84c:::benchmark_data_smithi018_157943_object1190:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e355)", 2024-09-10T19:06:12.780 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:01:24.514699+0000", 2024-09-10T19:06:12.780 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 288.199902393, 2024-09-10T19:06:12.780 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0335216270000001, 2024-09-10T19:06:12.781 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:06:12.781 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "reached pg", 2024-09-10T19:06:12.781 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:06:12.781 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15573", 2024-09-10T19:06:12.781 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:06:12.781 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 139848 2024-09-10T19:06:12.781 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.781 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:06:12.781 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.781 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:06:12.781 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.514699+0000", 2024-09-10T19:06:12.781 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.781 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.782 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.782 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:06:12.782 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.514699+0000", 2024-09-10T19:06:12.782 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.782 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.782 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.782 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:06:12.782 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.514697+0000", 2024-09-10T19:06:12.782 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T19:06:12.782 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.782 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.782 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:06:12.782 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.514703+0000", 2024-09-10T19:06:12.783 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.5110000000000003e-06 2024-09-10T19:06:12.783 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.783 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.783 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:06:12.783 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.514713+0000", 2024-09-10T19:06:12.783 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.8889999999999997e-06 2024-09-10T19:06:12.783 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.783 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.783 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:06:12.783 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.514718+0000", 2024-09-10T19:06:12.783 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.3009999999999997e-06 2024-09-10T19:06:12.783 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.784 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.784 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.784 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.516858+0000", 2024-09-10T19:06:12.784 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0021399740000000002 2024-09-10T19:06:12.784 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.784 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.784 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op must wait for map", 2024-09-10T19:06:12.784 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.516864+0000", 2024-09-10T19:06:12.784 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.4880000000000004e-06 2024-09-10T19:06:12.784 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.784 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.784 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.785 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.519631+0000", 2024-09-10T19:06:12.785 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0027663010000000001 2024-09-10T19:06:12.785 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.785 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.785 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:06:12.785 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.519638+0000", 2024-09-10T19:06:12.785 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.4719999999999998e-06 2024-09-10T19:06:12.785 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.785 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.785 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.785 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.529452+0000", 2024-09-10T19:06:12.785 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0098136639999999 2024-09-10T19:06:12.785 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.786 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.786 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for missing object", 2024-09-10T19:06:12.786 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.529531+0000", 2024-09-10T19:06:12.786 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.8721999999999996e-05 2024-09-10T19:06:12.786 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.786 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.786 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.786 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.548158+0000", 2024-09-10T19:06:12.786 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.018627932999999999 2024-09-10T19:06:12.786 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.786 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.786 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:06:12.786 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.548220+0000", 2024-09-10T19:06:12.787 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.1864000000000003e-05 2024-09-10T19:06:12.787 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.787 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:06:12.787 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.787 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.787 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.787 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15573.0:139901 10.9 10:970b6964:::benchmark_data_smithi018_157943_object1243:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e355)", 2024-09-10T19:06:12.787 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:01:24.553536+0000", 2024-09-10T19:06:12.787 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 288.16106485199998, 2024-09-10T19:06:12.787 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.98862571499999996, 2024-09-10T19:06:12.787 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:06:12.787 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:06:12.787 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:06:12.787 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15573", 2024-09-10T19:06:12.788 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:06:12.788 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 139901 2024-09-10T19:06:12.788 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.788 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:06:12.788 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.788 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:06:12.788 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.553536+0000", 2024-09-10T19:06:12.788 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.788 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.788 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.788 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:06:12.788 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.553536+0000", 2024-09-10T19:06:12.788 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.789 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.789 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.789 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:06:12.789 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.553535+0000", 2024-09-10T19:06:12.789 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999981 2024-09-10T19:06:12.789 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.789 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.789 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:06:12.789 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.553541+0000", 2024-09-10T19:06:12.789 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.7549999999999997e-06 2024-09-10T19:06:12.789 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.789 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.790 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:06:12.790 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.553559+0000", 2024-09-10T19:06:12.790 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.8117999999999998e-05 2024-09-10T19:06:12.790 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.790 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.790 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:06:12.790 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.553566+0000", 2024-09-10T19:06:12.790 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.7059999999999998e-06 2024-09-10T19:06:12.790 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.790 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.790 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.790 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.554181+0000", 2024-09-10T19:06:12.791 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00061476499999999997 2024-09-10T19:06:12.791 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.791 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.791 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:06:12.791 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.554191+0000", 2024-09-10T19:06:12.791 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0567999999999999e-05 2024-09-10T19:06:12.791 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.791 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.791 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.791 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.529693+0000", 2024-09-10T19:06:12.791 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.97550131100000004 2024-09-10T19:06:12.792 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.792 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.792 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:06:12.792 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.529774+0000", 2024-09-10T19:06:12.792 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.0791999999999995e-05 2024-09-10T19:06:12.792 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.792 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.792 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,4", 2024-09-10T19:06:12.792 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.530016+0000", 2024-09-10T19:06:12.792 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00024236799999999999 2024-09-10T19:06:12.793 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.793 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.793 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:06:12.793 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.539388+0000", 2024-09-10T19:06:12.793 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.009372577 2024-09-10T19:06:12.793 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.793 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.793 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:06:12.793 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.539407+0000", 2024-09-10T19:06:12.793 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.8759000000000001e-05 2024-09-10T19:06:12.793 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.793 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.794 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:06:12.794 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.541995+0000", 2024-09-10T19:06:12.794 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0025879649999999998 2024-09-10T19:06:12.794 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.794 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.794 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:06:12.794 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.542044+0000", 2024-09-10T19:06:12.794 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.9277000000000003e-05 2024-09-10T19:06:12.794 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.794 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.794 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:06:12.794 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.542162+0000", 2024-09-10T19:06:12.794 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00011745599999999999 2024-09-10T19:06:12.795 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.795 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:06:12.795 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.795 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.795 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.795 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15573.0:139924 10.9 10:9ca72da2:::benchmark_data_smithi018_157943_object1266:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e355)", 2024-09-10T19:06:12.795 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:01:24.618093+0000", 2024-09-10T19:06:12.795 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 288.09650810199997, 2024-09-10T19:06:12.795 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.92420184500000002, 2024-09-10T19:06:12.795 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:06:12.795 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:06:12.795 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:06:12.796 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15573", 2024-09-10T19:06:12.796 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:06:12.796 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 139924 2024-09-10T19:06:12.796 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.796 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:06:12.796 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.796 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:06:12.796 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.618093+0000", 2024-09-10T19:06:12.796 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.796 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.796 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.797 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:06:12.797 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.618093+0000", 2024-09-10T19:06:12.797 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.797 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.797 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.797 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:06:12.797 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.618092+0000", 2024-09-10T19:06:12.797 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T19:06:12.797 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.797 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.797 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:06:12.798 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.618098+0000", 2024-09-10T19:06:12.798 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.4060000000000002e-06 2024-09-10T19:06:12.798 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.798 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.798 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:06:12.798 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.618111+0000", 2024-09-10T19:06:12.798 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.3071000000000001e-05 2024-09-10T19:06:12.798 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.798 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.798 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:06:12.798 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.618119+0000", 2024-09-10T19:06:12.798 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.8990000000000001e-06 2024-09-10T19:06:12.798 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.799 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.799 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.799 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.618161+0000", 2024-09-10T19:06:12.799 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.2463999999999999e-05 2024-09-10T19:06:12.799 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.799 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.799 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:06:12.799 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.618170+0000", 2024-09-10T19:06:12.799 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.7190000000000007e-06 2024-09-10T19:06:12.799 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.799 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.799 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.799 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.532104+0000", 2024-09-10T19:06:12.800 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.91393350200000001 2024-09-10T19:06:12.800 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.800 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.800 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:06:12.800 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.532229+0000", 2024-09-10T19:06:12.800 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00012533299999999999 2024-09-10T19:06:12.800 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.800 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.800 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,4", 2024-09-10T19:06:12.800 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.532362+0000", 2024-09-10T19:06:12.800 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00013297800000000001 2024-09-10T19:06:12.800 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.801 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.801 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:06:12.801 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.539423+0000", 2024-09-10T19:06:12.801 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0070610480000000003 2024-09-10T19:06:12.801 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.801 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.801 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:06:12.801 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.539442+0000", 2024-09-10T19:06:12.801 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.8675e-05 2024-09-10T19:06:12.801 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.802 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.802 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:06:12.802 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.542170+0000", 2024-09-10T19:06:12.802 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0027288590000000001 2024-09-10T19:06:12.802 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.802 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.802 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:06:12.802 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.542208+0000", 2024-09-10T19:06:12.802 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.7317999999999997e-05 2024-09-10T19:06:12.802 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.802 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.802 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:06:12.802 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.542295+0000", 2024-09-10T19:06:12.803 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.7045000000000004e-05 2024-09-10T19:06:12.803 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.803 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:06:12.803 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.803 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.803 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.803 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15573.0:139937 10.9 10:98533757:::benchmark_data_smithi018_157943_object1279:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e355)", 2024-09-10T19:06:12.803 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:01:24.624786+0000", 2024-09-10T19:06:12.803 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 288.08981533799999, 2024-09-10T19:06:12.803 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.92250637899999999, 2024-09-10T19:06:12.803 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:06:12.803 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:06:12.804 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:06:12.804 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15573", 2024-09-10T19:06:12.804 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:06:12.804 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 139937 2024-09-10T19:06:12.804 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.804 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:06:12.804 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.804 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:06:12.804 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.624786+0000", 2024-09-10T19:06:12.804 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.804 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.804 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.804 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:06:12.805 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.624786+0000", 2024-09-10T19:06:12.805 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.805 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.805 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.805 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:06:12.805 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.624785+0000", 2024-09-10T19:06:12.805 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T19:06:12.805 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.805 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.805 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:06:12.805 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.624791+0000", 2024-09-10T19:06:12.805 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.1299999999999998e-06 2024-09-10T19:06:12.805 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.806 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.806 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:06:12.806 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.624803+0000", 2024-09-10T19:06:12.806 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.1864e-05 2024-09-10T19:06:12.806 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.806 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.806 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:06:12.806 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.624809+0000", 2024-09-10T19:06:12.806 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.925e-06 2024-09-10T19:06:12.806 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.806 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.806 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.806 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.624860+0000", 2024-09-10T19:06:12.806 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.1304999999999998e-05 2024-09-10T19:06:12.807 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.807 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.807 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:06:12.807 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.624871+0000", 2024-09-10T19:06:12.807 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.148e-05 2024-09-10T19:06:12.807 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.807 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.807 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.807 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.539455+0000", 2024-09-10T19:06:12.807 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.91458371999999999 2024-09-10T19:06:12.807 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.807 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.808 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:06:12.808 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.539534+0000", 2024-09-10T19:06:12.808 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.8471000000000001e-05 2024-09-10T19:06:12.808 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.808 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.808 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,4", 2024-09-10T19:06:12.808 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.539614+0000", 2024-09-10T19:06:12.808 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.0758000000000002e-05 2024-09-10T19:06:12.808 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.808 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.808 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:06:12.808 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.541987+0000", 2024-09-10T19:06:12.808 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0023724190000000002 2024-09-10T19:06:12.809 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.809 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.809 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:06:12.809 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.542367+0000", 2024-09-10T19:06:12.809 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00038044400000000002 2024-09-10T19:06:12.809 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.809 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.809 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:06:12.809 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.547206+0000", 2024-09-10T19:06:12.809 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0048384919999999998 2024-09-10T19:06:12.809 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.809 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.809 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:06:12.810 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.547238+0000", 2024-09-10T19:06:12.810 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.2465000000000002e-05 2024-09-10T19:06:12.810 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.810 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.810 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:06:12.810 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.547292+0000", 2024-09-10T19:06:12.810 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.4030000000000003e-05 2024-09-10T19:06:12.810 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.810 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:06:12.810 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.810 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.810 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.810 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.15573.0:139944 10.9 10:94c237c6:::benchmark_data_smithi018_157943_object1286:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e355)", 2024-09-10T19:06:12.811 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:01:24.627358+0000", 2024-09-10T19:06:12.811 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 288.087243231, 2024-09-10T19:06:12.811 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.92027273200000004, 2024-09-10T19:06:12.811 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:06:12.811 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:06:12.811 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:06:12.811 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.15573", 2024-09-10T19:06:12.811 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/3323078471", 2024-09-10T19:06:12.811 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 139944 2024-09-10T19:06:12.811 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.811 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:06:12.811 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.811 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:06:12.812 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.627358+0000", 2024-09-10T19:06:12.812 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.812 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.812 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.812 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:06:12.812 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.627358+0000", 2024-09-10T19:06:12.812 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.812 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.812 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.812 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:06:12.812 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.627357+0000", 2024-09-10T19:06:12.812 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T19:06:12.813 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:12.814 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.814 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.814 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:06:12.814 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.627363+0000", 2024-09-10T19:06:12.814 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.6890000000000001e-06 2024-09-10T19:06:12.814 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.814 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.815 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:06:12.815 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.627378+0000", 2024-09-10T19:06:12.815 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.5738000000000001e-05 2024-09-10T19:06:12.815 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.815 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.815 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:06:12.815 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.627385+0000", 2024-09-10T19:06:12.815 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.3890000000000002e-06 2024-09-10T19:06:12.815 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.815 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.816 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.816 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.627429+0000", 2024-09-10T19:06:12.816 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.3788999999999997e-05 2024-09-10T19:06:12.816 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.816 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.816 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:06:12.816 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:24.627436+0000", 2024-09-10T19:06:12.816 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.34e-06 2024-09-10T19:06:12.816 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.816 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.816 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.817 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.540704+0000", 2024-09-10T19:06:12.817 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.91326781000000001 2024-09-10T19:06:12.817 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.817 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.817 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:06:12.817 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.540800+0000", 2024-09-10T19:06:12.817 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.6714999999999998e-05 2024-09-10T19:06:12.817 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.817 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.817 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 3,4", 2024-09-10T19:06:12.817 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.540927+0000", 2024-09-10T19:06:12.817 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000126181 2024-09-10T19:06:12.817 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.818 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.818 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:06:12.818 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.542414+0000", 2024-09-10T19:06:12.818 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.001487036 2024-09-10T19:06:12.818 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.818 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.818 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:06:12.818 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.547305+0000", 2024-09-10T19:06:12.818 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0048917429999999996 2024-09-10T19:06:12.818 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.818 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.818 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:06:12.819 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.547547+0000", 2024-09-10T19:06:12.819 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00024154600000000001 2024-09-10T19:06:12.819 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.819 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.819 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:06:12.819 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.547578+0000", 2024-09-10T19:06:12.819 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.0768000000000001e-05 2024-09-10T19:06:12.819 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.819 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.819 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:06:12.819 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:01:25.547631+0000", 2024-09-10T19:06:12.820 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.2918999999999999e-05 2024-09-10T19:06:12.820 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.820 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:06:12.820 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.820 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.820 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.820 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.25706.0:113068 11.13 11:cc61eb63:::benchmark_data_smithi018_181485_object113067:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e406)", 2024-09-10T19:06:12.820 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:04:37.981415+0000", 2024-09-10T19:06:12.820 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 94.733186184999994, 2024-09-10T19:06:12.820 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.8789438650000001, 2024-09-10T19:06:12.820 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:06:12.820 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:06:12.820 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:06:12.821 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.25706", 2024-09-10T19:06:12.821 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/42081399", 2024-09-10T19:06:12.821 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 113068 2024-09-10T19:06:12.821 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.821 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:06:12.821 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.821 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:06:12.821 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:37.981415+0000", 2024-09-10T19:06:12.821 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.821 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.821 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.821 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:06:12.822 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:37.981415+0000", 2024-09-10T19:06:12.822 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.822 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.822 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.822 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:06:12.822 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:37.981414+0000", 2024-09-10T19:06:12.822 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T19:06:12.822 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.822 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.822 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:06:12.822 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:37.981475+0000", 2024-09-10T19:06:12.822 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.1373999999999999e-05 2024-09-10T19:06:12.823 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.823 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.823 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:06:12.823 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:37.981486+0000", 2024-09-10T19:06:12.823 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.1060000000000001e-05 2024-09-10T19:06:12.823 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.823 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.823 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:06:12.823 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:37.981492+0000", 2024-09-10T19:06:12.823 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.0759999999999999e-06 2024-09-10T19:06:12.823 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.823 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.823 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.824 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:37.981527+0000", 2024-09-10T19:06:12.824 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.4594000000000001e-05 2024-09-10T19:06:12.824 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.824 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.824 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:06:12.824 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:37.981533+0000", 2024-09-10T19:06:12.824 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.8690000000000001e-06 2024-09-10T19:06:12.824 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.824 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.824 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.824 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:38.798745+0000", 2024-09-10T19:06:12.824 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.81721254200000004 2024-09-10T19:06:12.825 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.825 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.825 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:06:12.825 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:38.798755+0000", 2024-09-10T19:06:12.825 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.5540000000000003e-06 2024-09-10T19:06:12.825 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.825 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.825 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.825 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:39.851146+0000", 2024-09-10T19:06:12.825 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.052391584 2024-09-10T19:06:12.825 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.825 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.825 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:06:12.826 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:39.851281+0000", 2024-09-10T19:06:12.826 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000135148 2024-09-10T19:06:12.826 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.826 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.826 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1,4,6", 2024-09-10T19:06:12.826 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:39.851597+0000", 2024-09-10T19:06:12.826 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00031530099999999998 2024-09-10T19:06:12.826 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.826 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.826 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:06:12.826 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:39.858313+0000", 2024-09-10T19:06:12.826 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0067159539999999997 2024-09-10T19:06:12.826 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.827 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.827 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:06:12.827 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:39.858373+0000", 2024-09-10T19:06:12.827 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.9781000000000002e-05 2024-09-10T19:06:12.827 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.827 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.827 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:06:12.827 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:39.858410+0000", 2024-09-10T19:06:12.827 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.7268999999999998e-05 2024-09-10T19:06:12.827 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.827 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.827 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:06:12.827 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:39.860076+0000", 2024-09-10T19:06:12.828 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00166596 2024-09-10T19:06:12.828 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.828 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.828 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:06:12.828 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:39.860205+0000", 2024-09-10T19:06:12.828 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00012959199999999999 2024-09-10T19:06:12.828 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.828 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.828 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:06:12.828 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:39.860249+0000", 2024-09-10T19:06:12.828 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.3192999999999998e-05 2024-09-10T19:06:12.828 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.828 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.828 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:06:12.829 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:39.860359+0000", 2024-09-10T19:06:12.829 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00011023600000000001 2024-09-10T19:06:12.829 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.829 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:06:12.829 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.829 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.829 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.829 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.25706.0:113089 11.13 11:c9f09bd9:::benchmark_data_smithi018_181485_object113088:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e406)", 2024-09-10T19:06:12.829 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:04:37.991854+0000", 2024-09-10T19:06:12.829 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 94.722747361000003, 2024-09-10T19:06:12.829 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.877370717, 2024-09-10T19:06:12.829 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:06:12.829 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:06:12.830 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:06:12.830 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.25706", 2024-09-10T19:06:12.830 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/42081399", 2024-09-10T19:06:12.830 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 113089 2024-09-10T19:06:12.830 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.830 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:06:12.830 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.830 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:06:12.830 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:37.991854+0000", 2024-09-10T19:06:12.830 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.830 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.830 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.830 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:06:12.831 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:37.991854+0000", 2024-09-10T19:06:12.831 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.831 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.831 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.831 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:06:12.831 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:37.991851+0000", 2024-09-10T19:06:12.831 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999976 2024-09-10T19:06:12.831 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.831 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.831 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:06:12.831 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:37.991909+0000", 2024-09-10T19:06:12.831 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.7917999999999997e-05 2024-09-10T19:06:12.831 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.832 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.832 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:06:12.832 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:37.991922+0000", 2024-09-10T19:06:12.832 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.2378000000000001e-05 2024-09-10T19:06:12.832 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.832 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.832 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:06:12.832 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:37.991933+0000", 2024-09-10T19:06:12.832 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.1094000000000001e-05 2024-09-10T19:06:12.832 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.832 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.832 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.833 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:37.991986+0000", 2024-09-10T19:06:12.833 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.2689999999999999e-05 2024-09-10T19:06:12.833 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.833 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.833 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:06:12.833 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:37.991994+0000", 2024-09-10T19:06:12.833 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.8000000000000004e-06 2024-09-10T19:06:12.833 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.833 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.833 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.833 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:38.798775+0000", 2024-09-10T19:06:12.833 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.80678116099999997 2024-09-10T19:06:12.834 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.834 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.834 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:06:12.834 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:38.798782+0000", 2024-09-10T19:06:12.834 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.246e-06 2024-09-10T19:06:12.834 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.834 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.834 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.834 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:39.856066+0000", 2024-09-10T19:06:12.834 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0572845399999999 2024-09-10T19:06:12.834 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.834 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.834 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:06:12.834 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:39.856222+0000", 2024-09-10T19:06:12.835 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00015536599999999999 2024-09-10T19:06:12.835 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.835 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.835 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1,4,6", 2024-09-10T19:06:12.835 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:39.856516+0000", 2024-09-10T19:06:12.835 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00029441599999999998 2024-09-10T19:06:12.835 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.835 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.835 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:06:12.835 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:39.858444+0000", 2024-09-10T19:06:12.835 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0019277400000000001 2024-09-10T19:06:12.835 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.835 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.836 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:06:12.836 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:39.858484+0000", 2024-09-10T19:06:12.836 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.0365999999999999e-05 2024-09-10T19:06:12.836 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.836 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.836 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:06:12.836 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:39.860082+0000", 2024-09-10T19:06:12.836 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0015982030000000001 2024-09-10T19:06:12.836 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.836 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.836 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:06:12.836 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:39.860391+0000", 2024-09-10T19:06:12.836 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00030872499999999998 2024-09-10T19:06:12.837 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.837 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.837 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:06:12.837 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:39.869060+0000", 2024-09-10T19:06:12.837 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0086691870000000001 2024-09-10T19:06:12.837 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.837 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.837 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:06:12.837 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:39.869109+0000", 2024-09-10T19:06:12.837 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.8578000000000001e-05 2024-09-10T19:06:12.837 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.837 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.837 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:06:12.838 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:39.869224+0000", 2024-09-10T19:06:12.838 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000115609 2024-09-10T19:06:12.838 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.838 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:06:12.838 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.838 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.838 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.838 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.25706.0:113098 11.13 11:c89e15f9:::benchmark_data_smithi018_181485_object113097:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e406)", 2024-09-10T19:06:12.838 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:04:37.997573+0000", 2024-09-10T19:06:12.838 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 94.717028571, 2024-09-10T19:06:12.838 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.8720230839999998, 2024-09-10T19:06:12.838 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:06:12.838 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:06:12.838 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:06:12.839 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.25706", 2024-09-10T19:06:12.839 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/42081399", 2024-09-10T19:06:12.839 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 113098 2024-09-10T19:06:12.839 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.839 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:06:12.839 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.839 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:06:12.839 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:37.997573+0000", 2024-09-10T19:06:12.839 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.839 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.839 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.839 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:06:12.839 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:37.997573+0000", 2024-09-10T19:06:12.839 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.840 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.840 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.840 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:06:12.840 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:37.997572+0000", 2024-09-10T19:06:12.840 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T19:06:12.840 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.840 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.840 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:06:12.840 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:37.997626+0000", 2024-09-10T19:06:12.840 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.4267000000000001e-05 2024-09-10T19:06:12.840 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.840 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.840 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:06:12.841 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:37.997638+0000", 2024-09-10T19:06:12.841 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.2401e-05 2024-09-10T19:06:12.841 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.841 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.841 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:06:12.841 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:37.997644+0000", 2024-09-10T19:06:12.841 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.9710000000000001e-06 2024-09-10T19:06:12.841 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.841 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.841 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.841 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:37.997686+0000", 2024-09-10T19:06:12.841 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.1315e-05 2024-09-10T19:06:12.841 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.841 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.842 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:06:12.842 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:37.997691+0000", 2024-09-10T19:06:12.842 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.7649999999999999e-06 2024-09-10T19:06:12.842 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.842 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.842 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.842 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:38.798802+0000", 2024-09-10T19:06:12.842 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.80111075099999995 2024-09-10T19:06:12.842 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.842 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.842 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:06:12.842 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:38.798809+0000", 2024-09-10T19:06:12.842 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.9199999999999998e-06 2024-09-10T19:06:12.842 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.843 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.843 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.843 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:39.858513+0000", 2024-09-10T19:06:12.843 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0597044980000001 2024-09-10T19:06:12.843 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.843 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.843 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:06:12.843 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:39.858664+0000", 2024-09-10T19:06:12.843 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000150066 2024-09-10T19:06:12.843 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.843 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.843 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,1,4,6", 2024-09-10T19:06:12.844 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:39.859030+0000", 2024-09-10T19:06:12.844 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00036652400000000002 2024-09-10T19:06:12.844 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.844 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.844 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:06:12.844 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:39.861289+0000", 2024-09-10T19:06:12.844 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0022584969999999999 2024-09-10T19:06:12.844 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.844 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.844 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:06:12.844 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:39.869027+0000", 2024-09-10T19:06:12.844 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0077388810000000004 2024-09-10T19:06:12.844 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.845 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.845 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:06:12.845 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:39.869344+0000", 2024-09-10T19:06:12.845 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00031669700000000001 2024-09-10T19:06:12.845 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.845 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.845 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:06:12.845 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:39.869356+0000", 2024-09-10T19:06:12.845 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.1891e-05 2024-09-10T19:06:12.845 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.845 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.845 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:06:12.845 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:39.869414+0000", 2024-09-10T19:06:12.845 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.8356000000000002e-05 2024-09-10T19:06:12.846 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.846 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.846 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:06:12.846 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:39.869491+0000", 2024-09-10T19:06:12.846 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.6934999999999994e-05 2024-09-10T19:06:12.846 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.846 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.846 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:06:12.846 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:04:39.869596+0000", 2024-09-10T19:06:12.846 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00010427799999999999 2024-09-10T19:06:12.846 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.846 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:06:12.846 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.847 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.847 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.847 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.25706.0:147847 11.13 11:cd8d281e:::benchmark_data_smithi018_181485_object10755:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e420)", 2024-09-10T19:06:12.847 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:05:07.847365+0000", 2024-09-10T19:06:12.847 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 64.867235622999999, 2024-09-10T19:06:12.847 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.96695033600000002, 2024-09-10T19:06:12.847 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:06:12.847 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:06:12.847 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:06:12.847 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.25706", 2024-09-10T19:06:12.847 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/42081399", 2024-09-10T19:06:12.847 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 147847 2024-09-10T19:06:12.847 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.847 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:06:12.848 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.848 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:06:12.848 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:07.847365+0000", 2024-09-10T19:06:12.848 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.848 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.848 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.848 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:06:12.848 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:07.847365+0000", 2024-09-10T19:06:12.848 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.848 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.848 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.849 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:06:12.849 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:07.847364+0000", 2024-09-10T19:06:12.849 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T19:06:12.849 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.849 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.849 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:06:12.849 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:07.847369+0000", 2024-09-10T19:06:12.849 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.6949999999999999e-06 2024-09-10T19:06:12.849 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.849 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.849 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:06:12.849 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:07.847380+0000", 2024-09-10T19:06:12.849 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 1.0651e-05 2024-09-10T19:06:12.849 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.850 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.850 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:06:12.850 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:07.847384+0000", 2024-09-10T19:06:12.850 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.651e-06 2024-09-10T19:06:12.850 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.850 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.850 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.850 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:07.847417+0000", 2024-09-10T19:06:12.850 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.2808000000000002e-05 2024-09-10T19:06:12.850 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.850 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.850 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:06:12.850 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:07.847423+0000", 2024-09-10T19:06:12.851 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.0820000000000004e-06 2024-09-10T19:06:12.851 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.851 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.851 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.851 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:08.809768+0000", 2024-09-10T19:06:12.851 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.96234451099999996 2024-09-10T19:06:12.851 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.851 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.851 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:06:12.851 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:08.809876+0000", 2024-09-10T19:06:12.851 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00010847 2024-09-10T19:06:12.851 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.851 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.852 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,4", 2024-09-10T19:06:12.852 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:08.810133+0000", 2024-09-10T19:06:12.852 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00025653299999999998 2024-09-10T19:06:12.852 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.852 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.852 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:06:12.852 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:08.814164+0000", 2024-09-10T19:06:12.852 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0040310260000000001 2024-09-10T19:06:12.852 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.852 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.852 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:06:12.852 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:08.814217+0000", 2024-09-10T19:06:12.852 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.3177000000000003e-05 2024-09-10T19:06:12.852 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.853 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.853 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:06:12.853 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:08.814226+0000", 2024-09-10T19:06:12.853 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.6209999999999993e-06 2024-09-10T19:06:12.853 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.853 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.853 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:06:12.853 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:08.814273+0000", 2024-09-10T19:06:12.853 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.7407000000000002e-05 2024-09-10T19:06:12.853 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.853 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.853 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:06:12.853 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:08.814316+0000", 2024-09-10T19:06:12.854 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.2738999999999999e-05 2024-09-10T19:06:12.854 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.854 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:06:12.854 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.854 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.854 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.854 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.25706.0:147902 11.13 11:c9ac9de9:::benchmark_data_smithi018_181485_object10810:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e420)", 2024-09-10T19:06:12.854 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:05:07.893093+0000", 2024-09-10T19:06:12.854 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 64.821507789999998, 2024-09-10T19:06:12.854 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.92282431799999998, 2024-09-10T19:06:12.854 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:06:12.854 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-10T19:06:12.854 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:06:12.855 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.25706", 2024-09-10T19:06:12.855 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/42081399", 2024-09-10T19:06:12.855 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 147902 2024-09-10T19:06:12.855 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.855 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:06:12.855 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.855 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:06:12.855 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:07.893093+0000", 2024-09-10T19:06:12.855 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.855 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.855 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.855 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:06:12.855 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:07.893093+0000", 2024-09-10T19:06:12.855 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:12.856 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.856 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.856 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:06:12.856 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:07.893092+0000", 2024-09-10T19:06:12.856 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T19:06:12.856 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.856 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.856 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:06:12.856 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:07.893097+0000", 2024-09-10T19:06:12.857 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.2789999999999998e-06 2024-09-10T19:06:12.857 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.857 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.857 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:06:12.857 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:07.893107+0000", 2024-09-10T19:06:12.857 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.9459999999999993e-06 2024-09-10T19:06:12.857 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.857 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.857 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:06:12.857 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:07.893113+0000", 2024-09-10T19:06:12.857 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.1919999999999996e-06 2024-09-10T19:06:12.857 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.857 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.857 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.858 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:07.893154+0000", 2024-09-10T19:06:12.858 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.1341999999999998e-05 2024-09-10T19:06:12.858 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.858 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.858 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for peered", 2024-09-10T19:06:12.858 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:07.893162+0000", 2024-09-10T19:06:12.858 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.9480000000000008e-06 2024-09-10T19:06:12.858 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.858 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.858 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:12.858 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:08.811555+0000", 2024-09-10T19:06:12.858 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.91839319200000002 2024-09-10T19:06:12.858 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.859 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.859 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:06:12.859 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:08.811688+0000", 2024-09-10T19:06:12.859 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00013336000000000001 2024-09-10T19:06:12.859 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.859 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.859 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 0,4", 2024-09-10T19:06:12.859 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:08.811900+0000", 2024-09-10T19:06:12.859 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00021196799999999999 2024-09-10T19:06:12.859 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.859 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.859 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:06:12.859 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:08.815692+0000", 2024-09-10T19:06:12.859 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0037920929999999999 2024-09-10T19:06:12.860 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.860 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.860 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:06:12.860 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:08.815743+0000", 2024-09-10T19:06:12.860 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.0986000000000003e-05 2024-09-10T19:06:12.860 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.860 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.860 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:06:12.860 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:08.815836+0000", 2024-09-10T19:06:12.860 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 9.2103999999999998e-05 2024-09-10T19:06:12.860 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.860 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.860 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "commit_sent", 2024-09-10T19:06:12.861 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:08.815878+0000", 2024-09-10T19:06:12.861 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.2679999999999998e-05 2024-09-10T19:06:12.861 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:12.861 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:12.861 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "done", 2024-09-10T19:06:12.861 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:05:08.815918+0000", 2024-09-10T19:06:12.861 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.9399e-05 2024-09-10T19:06:12.861 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.861 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:06:12.861 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.864 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:12.864 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:06:12.864 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T19:06:12.900 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:12.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:12.901 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:13.002 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:13.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:13.035 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:13.035 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:13.136 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:13.171 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:13.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:13.172 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:13.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:12 smithi018 ceph-mon[40196]: 1.b scrub starts 2024-09-10T19:06:13.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:12 smithi018 ceph-mon[40196]: 1.b scrub ok 2024-09-10T19:06:13.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:12 smithi018 ceph-mon[40196]: 5.1a scrub starts 2024-09-10T19:06:13.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:12 smithi018 ceph-mon[40196]: osdmap e442: 8 total, 8 up, 6 in 2024-09-10T19:06:13.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:12 smithi018 ceph-mon[40196]: 5.1a scrub ok 2024-09-10T19:06:13.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:12 smithi018 ceph-mon[29336]: 1.b scrub starts 2024-09-10T19:06:13.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:12 smithi018 ceph-mon[29336]: 1.b scrub ok 2024-09-10T19:06:13.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:12 smithi018 ceph-mon[29336]: 5.1a scrub starts 2024-09-10T19:06:13.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:12 smithi018 ceph-mon[29336]: osdmap e442: 8 total, 8 up, 6 in 2024-09-10T19:06:13.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:12 smithi018 ceph-mon[29336]: 5.1a scrub ok 2024-09-10T19:06:13.255 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:12 smithi103 ceph-mon[33163]: 1.b scrub starts 2024-09-10T19:06:13.255 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:12 smithi103 ceph-mon[33163]: 1.b scrub ok 2024-09-10T19:06:13.255 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:12 smithi103 ceph-mon[33163]: 5.1a scrub starts 2024-09-10T19:06:13.255 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:12 smithi103 ceph-mon[33163]: osdmap e442: 8 total, 8 up, 6 in 2024-09-10T19:06:13.255 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:12 smithi103 ceph-mon[33163]: 5.1a scrub ok 2024-09-10T19:06:13.273 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:13.301 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:13.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:13.302 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:13.403 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:13.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:13.433 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:13.433 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:13.534 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:13.564 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:13.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:13.566 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:13.667 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:13.705 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:13.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:13.706 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:13.807 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:13.846 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:13.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:13.847 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:13.864 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-10T19:06:13.948 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:13.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:13.985 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:13.985 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:14.086 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:14.111 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:14.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:14.112 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:14.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:13 smithi018 ceph-mon[29336]: 4.3 scrub starts 2024-09-10T19:06:14.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:13 smithi018 ceph-mon[29336]: 4.3 scrub ok 2024-09-10T19:06:14.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:13 smithi018 ceph-mon[29336]: 4.9 scrub starts 2024-09-10T19:06:14.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:13 smithi018 ceph-mon[29336]: 4.9 scrub ok 2024-09-10T19:06:14.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:13 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:06:14.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:13 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:06:14.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:13 smithi018 ceph-mon[29336]: pgmap v984: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 2.6 GiB data, 16 GiB used, 521 GiB / 536 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.86k op/s; 1329/126912 objects misplaced (1.047%); 0 B/s, 0 objects/s recovering 2024-09-10T19:06:14.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:13 smithi018 ceph-mon[40196]: 4.3 scrub starts 2024-09-10T19:06:14.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:13 smithi018 ceph-mon[40196]: 4.3 scrub ok 2024-09-10T19:06:14.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:13 smithi018 ceph-mon[40196]: 4.9 scrub starts 2024-09-10T19:06:14.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:13 smithi018 ceph-mon[40196]: 4.9 scrub ok 2024-09-10T19:06:14.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:13 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:06:14.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:13 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:06:14.196 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:13 smithi018 ceph-mon[40196]: pgmap v984: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 2.6 GiB data, 16 GiB used, 521 GiB / 536 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.86k op/s; 1329/126912 objects misplaced (1.047%); 0 B/s, 0 objects/s recovering 2024-09-10T19:06:14.213 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:14.253 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:06:14.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:14.265 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:14.266 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:14.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:13 smithi103 ceph-mon[33163]: 4.3 scrub starts 2024-09-10T19:06:14.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:13 smithi103 ceph-mon[33163]: 4.3 scrub ok 2024-09-10T19:06:14.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:13 smithi103 ceph-mon[33163]: 4.9 scrub starts 2024-09-10T19:06:14.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:13 smithi103 ceph-mon[33163]: 4.9 scrub ok 2024-09-10T19:06:14.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:13 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:06:14.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:13 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:06:14.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:13 smithi103 ceph-mon[33163]: pgmap v984: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 2.6 GiB data, 16 GiB used, 521 GiB / 536 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.86k op/s; 1329/126912 objects misplaced (1.047%); 0 B/s, 0 objects/s recovering 2024-09-10T19:06:14.366 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:14.408 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:14.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:14.409 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:14.510 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:14.546 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:14.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:14.547 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:14.648 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:14.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:14.675 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:14.675 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:14.776 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:14.818 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:14.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:14.819 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:14.920 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:14.962 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:14.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:14.963 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:15.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:14 smithi103 ceph-mon[33163]: osdmap e443: 8 total, 8 up, 6 in 2024-09-10T19:06:15.064 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:15.102 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:15.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:15.103 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:15.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:14 smithi018 ceph-mon[40196]: osdmap e443: 8 total, 8 up, 6 in 2024-09-10T19:06:15.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:14 smithi018 ceph-mon[29336]: osdmap e443: 8 total, 8 up, 6 in 2024-09-10T19:06:15.204 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:15.237 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:15.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:15.238 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:15.339 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:15.363 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:15.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:15.364 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:15.465 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:15.498 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:15.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:15.499 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:15.600 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:15.635 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:15.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:15.637 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:15.737 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:15.781 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:15.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:15.782 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:15.883 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:15.913 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:15.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:15.914 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:15.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:15 smithi018 ceph-mon[40196]: 2.f scrub starts 2024-09-10T19:06:15.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:15 smithi018 ceph-mon[40196]: 2.f scrub ok 2024-09-10T19:06:15.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:15 smithi018 ceph-mon[40196]: pgmap v986: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 2.5 GiB data, 16 GiB used, 521 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 1.36k op/s; 1329/122007 objects misplaced (1.089%) 2024-09-10T19:06:15.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:15 smithi018 ceph-mon[29336]: 2.f scrub starts 2024-09-10T19:06:15.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:15 smithi018 ceph-mon[29336]: 2.f scrub ok 2024-09-10T19:06:15.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:15 smithi018 ceph-mon[29336]: pgmap v986: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 2.5 GiB data, 16 GiB used, 521 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 1.36k op/s; 1329/122007 objects misplaced (1.089%) 2024-09-10T19:06:16.015 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:16.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:15 smithi103 ceph-mon[33163]: 2.f scrub starts 2024-09-10T19:06:16.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:15 smithi103 ceph-mon[33163]: 2.f scrub ok 2024-09-10T19:06:16.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:15 smithi103 ceph-mon[33163]: pgmap v986: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 2.5 GiB data, 16 GiB used, 521 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 1.36k op/s; 1329/122007 objects misplaced (1.089%) 2024-09-10T19:06:16.037 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:16.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:16.038 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:16.139 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:16.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:16.190 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:16.190 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:16.291 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:16.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:16.328 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:16.328 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:16.429 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:16.465 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:16.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:16.466 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:16.567 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:16.607 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:16.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:16.608 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:16.709 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:16.742 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:16.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:16.743 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:16.844 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:16.870 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:16.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:16.871 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:16.880 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T19:06:16.880 INFO:teuthology.orchestra.run.smithi103.stdout: "ops": [ 2024-09-10T19:06:16.880 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:16.880 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.25706.0:241720 11.4 11:2209f042:::benchmark_data_smithi018_181485_object104628:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e443)", 2024-09-10T19:06:16.880 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:06:16.856608+0000", 2024-09-10T19:06:16.881 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.022292844999999999, 2024-09-10T19:06:16.881 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.022316285000000002, 2024-09-10T19:06:16.881 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:06:16.881 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "waiting for sub ops", 2024-09-10T19:06:16.881 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:06:16.881 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.25706", 2024-09-10T19:06:16.881 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/42081399", 2024-09-10T19:06:16.881 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 241720 2024-09-10T19:06:16.881 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:16.881 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:06:16.881 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:16.881 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:06:16.881 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:06:16.856608+0000", 2024-09-10T19:06:16.882 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:16.882 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:16.882 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:16.882 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:06:16.882 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:06:16.856608+0000", 2024-09-10T19:06:16.882 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:16.882 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:16.882 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:16.882 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:06:16.882 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:06:16.856607+0000", 2024-09-10T19:06:16.882 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.999999 2024-09-10T19:06:16.882 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:16.883 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:16.883 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:06:16.883 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:06:16.856613+0000", 2024-09-10T19:06:16.883 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 6.0109999999999997e-06 2024-09-10T19:06:16.883 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:16.883 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:16.883 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:06:16.883 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:06:16.856621+0000", 2024-09-10T19:06:16.883 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 8.0090000000000007e-06 2024-09-10T19:06:16.883 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:16.883 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:16.883 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:06:16.883 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:06:16.856627+0000", 2024-09-10T19:06:16.884 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.5960000000000003e-06 2024-09-10T19:06:16.884 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:16.884 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:16.884 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:16.884 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:06:16.856659+0000", 2024-09-10T19:06:16.884 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 3.2067000000000003e-05 2024-09-10T19:06:16.884 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:16.884 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:16.884 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:06:16.884 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:06:16.856785+0000", 2024-09-10T19:06:16.884 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000125726 2024-09-10T19:06:16.884 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:16.884 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:16.885 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 1,3", 2024-09-10T19:06:16.885 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:06:16.856915+0000", 2024-09-10T19:06:16.885 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00013010100000000001 2024-09-10T19:06:16.885 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:16.885 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:16.885 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:06:16.885 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:06:16.858902+0000", 2024-09-10T19:06:16.885 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0019874609999999998 2024-09-10T19:06:16.885 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:16.886 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:16.886 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:06:16.886 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:06:16.859096+0000", 2024-09-10T19:06:16.886 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00019399799999999999 2024-09-10T19:06:16.886 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:16.886 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:06:16.886 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:16.886 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:16.886 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:16.886 INFO:teuthology.orchestra.run.smithi103.stdout: "description": "osd_op(client.25706.0:241714 11.4 11:23051c80:::benchmark_data_smithi018_181485_object104622:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e443)", 2024-09-10T19:06:16.886 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-10T19:06:16.853208+0000", 2024-09-10T19:06:16.886 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.025693075999999999, 2024-09-10T19:06:16.886 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.025770732000000001, 2024-09-10T19:06:16.886 INFO:teuthology.orchestra.run.smithi103.stdout: "type_data": { 2024-09-10T19:06:16.887 INFO:teuthology.orchestra.run.smithi103.stdout: "flag_point": "waiting for sub ops", 2024-09-10T19:06:16.887 INFO:teuthology.orchestra.run.smithi103.stdout: "client_info": { 2024-09-10T19:06:16.887 INFO:teuthology.orchestra.run.smithi103.stdout: "client": "client.25706", 2024-09-10T19:06:16.887 INFO:teuthology.orchestra.run.smithi103.stdout: "client_addr": "172.21.15.18:0/42081399", 2024-09-10T19:06:16.887 INFO:teuthology.orchestra.run.smithi103.stdout: "tid": 241714 2024-09-10T19:06:16.887 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:16.887 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-10T19:06:16.887 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:16.887 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "initiated", 2024-09-10T19:06:16.887 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:06:16.853208+0000", 2024-09-10T19:06:16.887 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:16.887 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:16.887 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:16.888 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "throttled", 2024-09-10T19:06:16.888 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:06:16.853208+0000", 2024-09-10T19:06:16.888 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0 2024-09-10T19:06:16.888 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:16.888 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:16.888 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "header_read", 2024-09-10T19:06:16.888 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:06:16.853207+0000", 2024-09-10T19:06:16.888 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4294967295.9999986 2024-09-10T19:06:16.888 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:16.888 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:16.888 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "all_read", 2024-09-10T19:06:16.888 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:06:16.853211+0000", 2024-09-10T19:06:16.888 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.5859999999999998e-06 2024-09-10T19:06:16.889 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:16.889 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:16.889 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "dispatched", 2024-09-10T19:06:16.889 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:06:16.853219+0000", 2024-09-10T19:06:16.889 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 7.7740000000000004e-06 2024-09-10T19:06:16.889 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:16.889 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:16.889 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "queued_for_pg", 2024-09-10T19:06:16.889 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:06:16.853223+0000", 2024-09-10T19:06:16.889 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 4.2899999999999996e-06 2024-09-10T19:06:16.889 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:16.889 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:16.889 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "reached_pg", 2024-09-10T19:06:16.889 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:06:16.853280+0000", 2024-09-10T19:06:16.890 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 5.6604000000000002e-05 2024-09-10T19:06:16.890 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:16.890 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:16.890 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "started", 2024-09-10T19:06:16.890 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:06:16.853445+0000", 2024-09-10T19:06:16.890 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.000165568 2024-09-10T19:06:16.890 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:16.890 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:16.890 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "waiting for subops from 1,3", 2024-09-10T19:06:16.890 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:06:16.853872+0000", 2024-09-10T19:06:16.890 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.00042658000000000003 2024-09-10T19:06:16.890 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:16.890 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:16.891 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "op_commit", 2024-09-10T19:06:16.891 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:06:16.856576+0000", 2024-09-10T19:06:16.891 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.0027038359999999998 2024-09-10T19:06:16.891 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T19:06:16.891 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-10T19:06:16.891 INFO:teuthology.orchestra.run.smithi103.stdout: "event": "sub_op_commit_rec", 2024-09-10T19:06:16.891 INFO:teuthology.orchestra.run.smithi103.stdout: "time": "2024-09-10T19:06:16.858745+0000", 2024-09-10T19:06:16.891 INFO:teuthology.orchestra.run.smithi103.stdout: "duration": 0.002168672 2024-09-10T19:06:16.891 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:16.891 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T19:06:16.891 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:16.891 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T19:06:16.891 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-10T19:06:16.891 INFO:teuthology.orchestra.run.smithi103.stdout: "num_ops": 2 2024-09-10T19:06:16.892 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T19:06:16.892 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is unset 2024-09-10T19:06:16.972 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:17.009 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:17.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:17.010 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:17.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:16 smithi103 ceph-mon[33163]: 6.18 scrub starts 2024-09-10T19:06:17.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:16 smithi103 ceph-mon[33163]: 6.18 scrub ok 2024-09-10T19:06:17.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:16 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3268171807' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:06:17.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:16 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:06:17.111 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:17.149 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:17.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:17.150 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:17.158 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:16 smithi018 ceph-mon[29336]: 6.18 scrub starts 2024-09-10T19:06:17.158 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:16 smithi018 ceph-mon[29336]: 6.18 scrub ok 2024-09-10T19:06:17.158 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:16 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3268171807' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:06:17.158 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:16 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:06:17.158 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:16 smithi018 ceph-mon[40196]: 6.18 scrub starts 2024-09-10T19:06:17.158 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:16 smithi018 ceph-mon[40196]: 6.18 scrub ok 2024-09-10T19:06:17.159 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:16 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3268171807' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:06:17.159 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:16 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:06:17.250 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:17.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:17.292 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:17.292 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:17.393 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:17.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:17.432 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:17.432 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:17.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:06:17 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:06:17.533 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:17.565 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:17.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:17.566 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:17.667 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:17.707 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:17.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:17.708 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:17.809 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:17.849 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:17.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:17.850 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:17.951 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:17.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:17.974 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:17.974 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:18.075 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:18.131 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:18.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:18.132 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:18.137 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:06:18.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:17 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:06:18.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:17 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:06:18.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:17 smithi018 ceph-mon[40196]: 1.2 scrub starts 2024-09-10T19:06:18.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:17 smithi018 ceph-mon[40196]: 1.2 scrub ok 2024-09-10T19:06:18.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:17 smithi018 ceph-mon[40196]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T19:06:18.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:17 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T19:06:18.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:17 smithi018 ceph-mon[40196]: osdmap e444: 8 total, 8 up, 6 in 2024-09-10T19:06:18.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:17 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:18.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:17 smithi018 ceph-mon[40196]: pgmap v988: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 2.1 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.76k op/s; 1054/101970 objects misplaced (1.034%) 2024-09-10T19:06:18.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:17 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:06:18.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:17 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:06:18.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:17 smithi018 ceph-mon[29336]: 1.2 scrub starts 2024-09-10T19:06:18.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:17 smithi018 ceph-mon[29336]: 1.2 scrub ok 2024-09-10T19:06:18.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:17 smithi018 ceph-mon[29336]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T19:06:18.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:17 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T19:06:18.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:17 smithi018 ceph-mon[29336]: osdmap e444: 8 total, 8 up, 6 in 2024-09-10T19:06:18.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:17 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:18.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:17 smithi018 ceph-mon[29336]: pgmap v988: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 2.1 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.76k op/s; 1054/101970 objects misplaced (1.034%) 2024-09-10T19:06:18.233 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:18.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:17 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:06:18.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:17 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:06:18.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:17 smithi103 ceph-mon[33163]: 1.2 scrub starts 2024-09-10T19:06:18.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:17 smithi103 ceph-mon[33163]: 1.2 scrub ok 2024-09-10T19:06:18.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:17 smithi103 ceph-mon[33163]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T19:06:18.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:17 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T19:06:18.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:17 smithi103 ceph-mon[33163]: osdmap e444: 8 total, 8 up, 6 in 2024-09-10T19:06:18.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:17 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:18.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:17 smithi103 ceph-mon[33163]: pgmap v988: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 2.1 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.76k op/s; 1054/101970 objects misplaced (1.034%) 2024-09-10T19:06:18.275 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:18.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:18.277 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:18.377 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:18.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:18.408 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:18.408 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:18.509 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:18.542 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:18.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:18.543 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:18.644 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:18.683 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:18.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:18.684 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:18.785 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:18.815 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:18.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:18.816 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:18.917 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:18.960 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:18.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:18.962 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:19.062 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:19.091 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:19.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:19.093 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:19.115 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:18 smithi018 ceph-mon[40196]: 2.a scrub starts 2024-09-10T19:06:19.116 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:18 smithi018 ceph-mon[40196]: 2.a scrub ok 2024-09-10T19:06:19.116 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:18 smithi018 ceph-mon[40196]: 6.15 scrub starts 2024-09-10T19:06:19.116 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:18 smithi018 ceph-mon[40196]: 6.15 scrub ok 2024-09-10T19:06:19.116 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:18 smithi018 ceph-mon[40196]: 2.2 scrub starts 2024-09-10T19:06:19.116 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:18 smithi018 ceph-mon[40196]: 2.2 scrub ok 2024-09-10T19:06:19.116 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:18 smithi018 ceph-mon[29336]: 2.a scrub starts 2024-09-10T19:06:19.117 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:18 smithi018 ceph-mon[29336]: 2.a scrub ok 2024-09-10T19:06:19.117 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:18 smithi018 ceph-mon[29336]: 6.15 scrub starts 2024-09-10T19:06:19.117 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:18 smithi018 ceph-mon[29336]: 6.15 scrub ok 2024-09-10T19:06:19.117 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:18 smithi018 ceph-mon[29336]: 2.2 scrub starts 2024-09-10T19:06:19.117 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:18 smithi018 ceph-mon[29336]: 2.2 scrub ok 2024-09-10T19:06:19.193 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:19.222 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:19.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:19.223 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:19.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:18 smithi103 ceph-mon[33163]: 2.a scrub starts 2024-09-10T19:06:19.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:18 smithi103 ceph-mon[33163]: 2.a scrub ok 2024-09-10T19:06:19.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:18 smithi103 ceph-mon[33163]: 6.15 scrub starts 2024-09-10T19:06:19.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:18 smithi103 ceph-mon[33163]: 6.15 scrub ok 2024-09-10T19:06:19.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:18 smithi103 ceph-mon[33163]: 2.2 scrub starts 2024-09-10T19:06:19.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:18 smithi103 ceph-mon[33163]: 2.2 scrub ok 2024-09-10T19:06:19.324 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:19.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:19.352 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:19.352 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:19.453 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:19.493 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:19.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:19.494 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:19.566 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-10T19:06:19.595 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:19.629 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:19.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:19.630 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:19.732 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:19.776 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:19.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:19.777 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:19.878 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:19.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:19.911 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:19.911 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:19.938 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:19 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/300746170' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-10T19:06:19.938 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:19 smithi103 ceph-mon[33163]: pgmap v989: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 2.0 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 668 B/s rd, 0 B/s wr, 1.27k op/s; 1054/97728 objects misplaced (1.079%) 2024-09-10T19:06:19.943 INFO:teuthology.orchestra.run.smithi018.stderr:reweighted osd.5 to 1 (10000) 2024-09-10T19:06:19.968 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:06:20.012 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:20.056 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:20.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:20.058 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:20.108 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set noscrub 2024-09-10T19:06:20.159 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:20.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:19 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/300746170' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-10T19:06:20.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:19 smithi018 ceph-mon[40196]: pgmap v989: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 2.0 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 668 B/s rd, 0 B/s wr, 1.27k op/s; 1054/97728 objects misplaced (1.079%) 2024-09-10T19:06:20.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:19 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/300746170' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-10T19:06:20.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:19 smithi018 ceph-mon[29336]: pgmap v989: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 2.0 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 668 B/s rd, 0 B/s wr, 1.27k op/s; 1054/97728 objects misplaced (1.079%) 2024-09-10T19:06:20.204 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:20.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:20.205 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:20.306 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:20.356 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:20.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:20.357 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:20.458 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:20.504 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:20.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:20.505 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:20.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:06:20 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:06:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:06:20.606 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:20.648 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:20.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:20.649 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:20.710 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd reweight 6 1 2024-09-10T19:06:20.750 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:20.779 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:20.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:20.781 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:20.881 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:20.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:20.922 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:20.922 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:21.023 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:21.067 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:21.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:21.068 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:21.169 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:21.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:20 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/300746170' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-10T19:06:21.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:20 smithi018 ceph-mon[29336]: osdmap e445: 8 total, 8 up, 6 in 2024-09-10T19:06:21.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:20 smithi018 ceph-mon[29336]: 6.13 deep-scrub starts 2024-09-10T19:06:21.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:20 smithi018 ceph-mon[29336]: 6.13 deep-scrub ok 2024-09-10T19:06:21.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:20 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/300746170' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-10T19:06:21.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:20 smithi018 ceph-mon[40196]: osdmap e445: 8 total, 8 up, 6 in 2024-09-10T19:06:21.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:20 smithi018 ceph-mon[40196]: 6.13 deep-scrub starts 2024-09-10T19:06:21.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:20 smithi018 ceph-mon[40196]: 6.13 deep-scrub ok 2024-09-10T19:06:21.195 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:06:20 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:06:20] "GET /metrics HTTP/1.1" 200 224502 "" "Prometheus/2.43.0" 2024-09-10T19:06:21.205 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:21.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:21.206 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:21.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:20 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/300746170' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-10T19:06:21.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:20 smithi103 ceph-mon[33163]: osdmap e445: 8 total, 8 up, 6 in 2024-09-10T19:06:21.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:20 smithi103 ceph-mon[33163]: 6.13 deep-scrub starts 2024-09-10T19:06:21.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:20 smithi103 ceph-mon[33163]: 6.13 deep-scrub ok 2024-09-10T19:06:21.307 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:21.329 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:21.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:21.331 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:21.431 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:21.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:21.471 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:21.471 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:21.572 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:21.616 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:21.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:21.617 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:21.718 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:21.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:21.755 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:21.756 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:21.856 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:21.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:21.892 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:21.892 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:21.976 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T19:06:21.976 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T19:06:21.976 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T19:06:21.976 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T19:06:21.976 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T19:06:21.976 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T19:06:21.976 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T19:06:21.976 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T19:06:21.976 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T19:06:21.977 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T19:06:21.977 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T19:06:21.977 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T19:06:21.977 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T19:06:21.977 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T19:06:21.977 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T19:06:21.977 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T19:06:21.992 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:22.023 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:22.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:22.024 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:22.124 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:22.158 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:22.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:22.161 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:22.206 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:21 smithi103 ceph-mon[33163]: 5.11 scrub starts 2024-09-10T19:06:22.206 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:21 smithi103 ceph-mon[33163]: 5.11 scrub ok 2024-09-10T19:06:22.206 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:21 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2110548293' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:06:22.206 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:21 smithi103 ceph-mon[33163]: pgmap v991: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 1.7 GiB data, 14 GiB used, 523 GiB / 536 GiB avail; 669 B/s rd, 0 B/s wr, 1.82k op/s; 784/85146 objects misplaced (0.921%) 2024-09-10T19:06:22.262 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:22.300 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:22.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:22.302 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:22.403 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:22.439 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:22.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:22.440 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:22.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:21 smithi018 ceph-mon[29336]: 5.11 scrub starts 2024-09-10T19:06:22.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:21 smithi018 ceph-mon[29336]: 5.11 scrub ok 2024-09-10T19:06:22.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:21 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2110548293' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:06:22.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:21 smithi018 ceph-mon[29336]: pgmap v991: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 1.7 GiB data, 14 GiB used, 523 GiB / 536 GiB avail; 669 B/s rd, 0 B/s wr, 1.82k op/s; 784/85146 objects misplaced (0.921%) 2024-09-10T19:06:22.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:21 smithi018 ceph-mon[40196]: 5.11 scrub starts 2024-09-10T19:06:22.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:21 smithi018 ceph-mon[40196]: 5.11 scrub ok 2024-09-10T19:06:22.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:21 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2110548293' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:06:22.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:21 smithi018 ceph-mon[40196]: pgmap v991: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 1.7 GiB data, 14 GiB used, 523 GiB / 536 GiB avail; 669 B/s rd, 0 B/s wr, 1.82k op/s; 784/85146 objects misplaced (0.921%) 2024-09-10T19:06:22.541 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:22.576 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:22.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:22.577 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:22.678 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:22.715 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:22.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:22.716 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:22.817 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:22.853 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:22.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:22.854 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:22.955 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:22.991 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:22.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:22.992 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:23.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:22 smithi103 ceph-mon[33163]: 1.f deep-scrub starts 2024-09-10T19:06:23.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:22 smithi103 ceph-mon[33163]: 1.f deep-scrub ok 2024-09-10T19:06:23.054 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:06:23.074 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:06:23.094 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:23.166 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:23.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:23.167 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:23.208 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:22 smithi018 ceph-mon[40196]: 1.f deep-scrub starts 2024-09-10T19:06:23.208 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:22 smithi018 ceph-mon[40196]: 1.f deep-scrub ok 2024-09-10T19:06:23.208 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:22 smithi018 ceph-mon[29336]: 1.f deep-scrub starts 2024-09-10T19:06:23.208 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:22 smithi018 ceph-mon[29336]: 1.f deep-scrub ok 2024-09-10T19:06:23.268 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:23.297 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:23.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:23.298 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:23.398 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:23.433 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:23.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:23.434 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:23.535 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:23.559 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:23.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:23.560 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:23.645 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T19:06:23.646 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T19:06:23.661 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:23.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:23.700 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:23.701 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:23.801 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:23.830 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:23.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:23.831 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:23.932 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:23.960 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:23.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:23.961 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:24.062 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:24.099 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:24.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:24.100 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:24.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:23 smithi018 ceph-mon[40196]: 6.f scrub starts 2024-09-10T19:06:24.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:23 smithi018 ceph-mon[40196]: 6.f scrub ok 2024-09-10T19:06:24.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:23 smithi018 ceph-mon[40196]: 5.d deep-scrub starts 2024-09-10T19:06:24.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:23 smithi018 ceph-mon[40196]: 5.d deep-scrub ok 2024-09-10T19:06:24.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:23 smithi018 ceph-mon[40196]: 2.e scrub starts 2024-09-10T19:06:24.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:23 smithi018 ceph-mon[40196]: 2.e scrub ok 2024-09-10T19:06:24.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:23 smithi018 ceph-mon[40196]: pgmap v992: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 1.6 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.80k op/s; 784/78747 objects misplaced (0.996%) 2024-09-10T19:06:24.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:23 smithi018 ceph-mon[29336]: 6.f scrub starts 2024-09-10T19:06:24.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:23 smithi018 ceph-mon[29336]: 6.f scrub ok 2024-09-10T19:06:24.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:23 smithi018 ceph-mon[29336]: 5.d deep-scrub starts 2024-09-10T19:06:24.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:23 smithi018 ceph-mon[29336]: 5.d deep-scrub ok 2024-09-10T19:06:24.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:23 smithi018 ceph-mon[29336]: 2.e scrub starts 2024-09-10T19:06:24.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:23 smithi018 ceph-mon[29336]: 2.e scrub ok 2024-09-10T19:06:24.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:23 smithi018 ceph-mon[29336]: pgmap v992: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 1.6 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.80k op/s; 784/78747 objects misplaced (0.996%) 2024-09-10T19:06:24.202 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:24.238 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:24.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:24.239 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:24.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:23 smithi103 ceph-mon[33163]: 6.f scrub starts 2024-09-10T19:06:24.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:23 smithi103 ceph-mon[33163]: 6.f scrub ok 2024-09-10T19:06:24.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:23 smithi103 ceph-mon[33163]: 5.d deep-scrub starts 2024-09-10T19:06:24.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:23 smithi103 ceph-mon[33163]: 5.d deep-scrub ok 2024-09-10T19:06:24.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:23 smithi103 ceph-mon[33163]: 2.e scrub starts 2024-09-10T19:06:24.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:23 smithi103 ceph-mon[33163]: 2.e scrub ok 2024-09-10T19:06:24.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:23 smithi103 ceph-mon[33163]: pgmap v992: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 1.6 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.80k op/s; 784/78747 objects misplaced (0.996%) 2024-09-10T19:06:24.341 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:24.378 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:24.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:24.379 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:24.480 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:24.513 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:24.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:24.514 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:24.615 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:24.637 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:24.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:24.638 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:24.739 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:24.767 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:24.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:24.768 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:24.869 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:24.901 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:24.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:24.903 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:25.004 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:25.024 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-10T19:06:25.041 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:25.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:25.042 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:25.143 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:25.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:25.172 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:25.173 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:25.274 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:25.309 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-10T19:06:25.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:25.310 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:25.311 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:25.412 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:25.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:25.449 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:25.449 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:25.550 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:25.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:25.592 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:25.593 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:25.693 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:25.730 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:25.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:25.733 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:25.756 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:06:25.834 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:25.880 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:25.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:25.881 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:25.982 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:26.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:26.017 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:26.018 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:26.116 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:26 smithi018 ceph-mon[29336]: 1.0 scrub starts 2024-09-10T19:06:26.117 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:26 smithi018 ceph-mon[29336]: 1.0 scrub ok 2024-09-10T19:06:26.117 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:26 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:06:26.117 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:26 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:06:26.117 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:26 smithi018 ceph-mon[29336]: pgmap v993: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 1.5 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 593 B/s rd, 0 B/s wr, 1.07k op/s; 784/74316 objects misplaced (1.055%) 2024-09-10T19:06:26.118 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:26.149 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:26.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:26.152 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:26.164 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:26 smithi103 ceph-mon[33163]: 1.0 scrub starts 2024-09-10T19:06:26.164 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:26 smithi103 ceph-mon[33163]: 1.0 scrub ok 2024-09-10T19:06:26.165 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:26 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:06:26.165 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:26 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:06:26.165 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:26 smithi103 ceph-mon[33163]: pgmap v993: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 1.5 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 593 B/s rd, 0 B/s wr, 1.07k op/s; 784/74316 objects misplaced (1.055%) 2024-09-10T19:06:26.252 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:26.296 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:26.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:26.297 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:26.398 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:26.404 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:26 smithi018 ceph-mon[40196]: 1.0 scrub starts 2024-09-10T19:06:26.404 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:26 smithi018 ceph-mon[40196]: 1.0 scrub ok 2024-09-10T19:06:26.404 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:26 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:06:26.404 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:26 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:06:26.404 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:26 smithi018 ceph-mon[40196]: pgmap v993: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 1.5 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 593 B/s rd, 0 B/s wr, 1.07k op/s; 784/74316 objects misplaced (1.055%) 2024-09-10T19:06:26.427 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:26.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:26.428 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:26.529 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:26.563 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:26.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:26.565 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:26.665 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:26.707 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:26.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:26.708 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:26.809 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:26.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:26.854 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:26.854 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:26.955 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:26.988 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:26.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:26.989 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:27.090 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:27.124 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:27.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:27.129 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:27.175 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:27 smithi103 ceph-mon[33163]: 1.6 deep-scrub starts 2024-09-10T19:06:27.176 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:27 smithi103 ceph-mon[33163]: 1.6 deep-scrub ok 2024-09-10T19:06:27.176 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:27 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3555988224' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-10T19:06:27.176 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:27 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-10T19:06:27.228 INFO:teuthology.orchestra.run.smithi018.stderr:reweighted osd.6 to 1 (10000) 2024-09-10T19:06:27.231 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:27.268 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:27.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:27.269 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:27.370 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:27.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:27.413 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:27.414 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:27.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:27 smithi018 ceph-mon[40196]: 1.6 deep-scrub starts 2024-09-10T19:06:27.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:27 smithi018 ceph-mon[40196]: 1.6 deep-scrub ok 2024-09-10T19:06:27.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:27 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3555988224' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-10T19:06:27.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:27 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-10T19:06:27.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:27 smithi018 ceph-mon[29336]: 1.6 deep-scrub starts 2024-09-10T19:06:27.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:27 smithi018 ceph-mon[29336]: 1.6 deep-scrub ok 2024-09-10T19:06:27.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:27 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3555988224' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-10T19:06:27.446 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:27 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-10T19:06:27.514 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:27.518 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:06:27 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:06:27.551 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:27.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:27.552 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:27.653 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:27.666 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T19:06:27.666 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T19:06:27.692 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:27.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:27.693 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:27.794 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:27.815 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:27.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:27.816 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:27.917 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:27.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:27.954 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:27.955 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:28.055 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:28.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:28.095 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:28.095 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:28.196 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:28.224 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:28.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:28.225 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:28.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:28 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-10T19:06:28.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:28 smithi103 ceph-mon[33163]: osdmap e446: 8 total, 8 up, 6 in 2024-09-10T19:06:28.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:28 smithi103 ceph-mon[33163]: 6.1b scrub starts 2024-09-10T19:06:28.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:28 smithi103 ceph-mon[33163]: 6.1b scrub ok 2024-09-10T19:06:28.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:28 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:28.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:28 smithi103 ceph-mon[33163]: pgmap v995: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 1.1 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.80k op/s; 547/54432 objects misplaced (1.005%) 2024-09-10T19:06:28.326 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:28.366 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-10T19:06:28.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:28.367 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:28.368 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:28.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:28 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-10T19:06:28.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:28 smithi018 ceph-mon[40196]: osdmap e446: 8 total, 8 up, 6 in 2024-09-10T19:06:28.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:28 smithi018 ceph-mon[40196]: 6.1b scrub starts 2024-09-10T19:06:28.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:28 smithi018 ceph-mon[40196]: 6.1b scrub ok 2024-09-10T19:06:28.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:28 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:28.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:28 smithi018 ceph-mon[40196]: pgmap v995: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 1.1 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.80k op/s; 547/54432 objects misplaced (1.005%) 2024-09-10T19:06:28.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:28 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-10T19:06:28.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:28 smithi018 ceph-mon[29336]: osdmap e446: 8 total, 8 up, 6 in 2024-09-10T19:06:28.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:28 smithi018 ceph-mon[29336]: 6.1b scrub starts 2024-09-10T19:06:28.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:28 smithi018 ceph-mon[29336]: 6.1b scrub ok 2024-09-10T19:06:28.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:28 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:28.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:28 smithi018 ceph-mon[29336]: pgmap v995: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 1.1 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.80k op/s; 547/54432 objects misplaced (1.005%) 2024-09-10T19:06:28.469 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:28.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:28.497 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:28.498 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:28.598 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:28.626 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:28.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:28.627 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:28.728 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:28.750 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:28.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:28.752 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:28.853 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:28.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:28.896 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:28.896 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:28.997 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:29.020 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:29.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:29.021 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:29.121 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:29.152 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:29.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:29.153 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:29.254 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:29.281 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:29.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:29.282 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:29.383 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:29.405 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:29.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:29.406 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:29.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:29 smithi018 ceph-mon[40196]: 2.6 scrub starts 2024-09-10T19:06:29.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:29 smithi018 ceph-mon[40196]: 2.6 scrub ok 2024-09-10T19:06:29.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:29 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:06:29.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:29 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:06:29.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:29 smithi018 ceph-mon[40196]: 6.1c scrub starts 2024-09-10T19:06:29.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:29 smithi018 ceph-mon[40196]: 6.1c scrub ok 2024-09-10T19:06:29.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:29 smithi018 ceph-mon[40196]: osdmap e447: 8 total, 8 up, 6 in 2024-09-10T19:06:29.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:29 smithi018 ceph-mon[29336]: 2.6 scrub starts 2024-09-10T19:06:29.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:29 smithi018 ceph-mon[29336]: 2.6 scrub ok 2024-09-10T19:06:29.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:29 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:06:29.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:29 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:06:29.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:29 smithi018 ceph-mon[29336]: 6.1c scrub starts 2024-09-10T19:06:29.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:29 smithi018 ceph-mon[29336]: 6.1c scrub ok 2024-09-10T19:06:29.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:29 smithi018 ceph-mon[29336]: osdmap e447: 8 total, 8 up, 6 in 2024-09-10T19:06:29.507 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:29.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:29 smithi103 ceph-mon[33163]: 2.6 scrub starts 2024-09-10T19:06:29.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:29 smithi103 ceph-mon[33163]: 2.6 scrub ok 2024-09-10T19:06:29.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:29 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:06:29.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:29 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:06:29.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:29 smithi103 ceph-mon[33163]: 6.1c scrub starts 2024-09-10T19:06:29.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:29 smithi103 ceph-mon[33163]: 6.1c scrub ok 2024-09-10T19:06:29.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:29 smithi103 ceph-mon[33163]: osdmap e447: 8 total, 8 up, 6 in 2024-09-10T19:06:29.542 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:29.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:29.543 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:29.644 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:29.674 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:29.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:29.676 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:29.776 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:29.821 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:29.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:29.822 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:29.922 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:29.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:29.963 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:29.964 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:30.064 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:30.088 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:30.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:30.089 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:30.190 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:30.230 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:30.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:30.231 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:30.332 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:30.360 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:30.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:30.361 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:30.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:30 smithi018 ceph-mon[40196]: 5.9 scrub starts 2024-09-10T19:06:30.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:30 smithi018 ceph-mon[40196]: 5.9 scrub ok 2024-09-10T19:06:30.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:30 smithi018 ceph-mon[40196]: 5.10 scrub starts 2024-09-10T19:06:30.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:30 smithi018 ceph-mon[40196]: 5.10 scrub ok 2024-09-10T19:06:30.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:30 smithi018 ceph-mon[40196]: pgmap v997: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 1.0 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.46k op/s; 547/50049 objects misplaced (1.093%) 2024-09-10T19:06:30.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:30 smithi018 ceph-mon[29336]: 5.9 scrub starts 2024-09-10T19:06:30.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:30 smithi018 ceph-mon[29336]: 5.9 scrub ok 2024-09-10T19:06:30.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:30 smithi018 ceph-mon[29336]: 5.10 scrub starts 2024-09-10T19:06:30.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:30 smithi018 ceph-mon[29336]: 5.10 scrub ok 2024-09-10T19:06:30.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:30 smithi018 ceph-mon[29336]: pgmap v997: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 1.0 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.46k op/s; 547/50049 objects misplaced (1.093%) 2024-09-10T19:06:30.462 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:30.503 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:30.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:30.504 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:30.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:06:30 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:06:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:06:30.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:30 smithi103 ceph-mon[33163]: 5.9 scrub starts 2024-09-10T19:06:30.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:30 smithi103 ceph-mon[33163]: 5.9 scrub ok 2024-09-10T19:06:30.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:30 smithi103 ceph-mon[33163]: 5.10 scrub starts 2024-09-10T19:06:30.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:30 smithi103 ceph-mon[33163]: 5.10 scrub ok 2024-09-10T19:06:30.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:30 smithi103 ceph-mon[33163]: pgmap v997: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 1.0 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.46k op/s; 547/50049 objects misplaced (1.093%) 2024-09-10T19:06:30.604 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:30.630 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:30.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:30.632 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:30.732 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:30.754 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:30.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:30.755 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:30.856 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:30.885 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:30.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:30.886 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:30.987 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:31.017 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:31.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:31.019 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:31.119 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:31.157 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:31.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:31.158 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:31.259 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:31.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:31.289 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:31.289 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:31.390 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:31.422 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:31.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:31.423 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:31.442 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:31 smithi018 ceph-mon[40196]: 6.1 scrub starts 2024-09-10T19:06:31.442 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:31 smithi018 ceph-mon[40196]: 6.1 scrub ok 2024-09-10T19:06:31.442 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:31 smithi018 ceph-mon[40196]: 2.5 deep-scrub starts 2024-09-10T19:06:31.442 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:31 smithi018 ceph-mon[40196]: 2.5 deep-scrub ok 2024-09-10T19:06:31.442 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:31 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:06:31.442 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:31 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:06:31.442 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:31 smithi018 ceph-mon[40196]: 5.16 scrub starts 2024-09-10T19:06:31.443 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:31 smithi018 ceph-mon[40196]: 5.16 scrub ok 2024-09-10T19:06:31.443 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:31 smithi018 ceph-mon[40196]: 5.17 scrub starts 2024-09-10T19:06:31.443 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:31 smithi018 ceph-mon[40196]: 5.17 scrub ok 2024-09-10T19:06:31.443 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:31 smithi018 ceph-mon[40196]: osdmap e448: 8 total, 8 up, 6 in 2024-09-10T19:06:31.443 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:31 smithi018 ceph-mon[29336]: 6.1 scrub starts 2024-09-10T19:06:31.443 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:31 smithi018 ceph-mon[29336]: 6.1 scrub ok 2024-09-10T19:06:31.443 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:31 smithi018 ceph-mon[29336]: 2.5 deep-scrub starts 2024-09-10T19:06:31.443 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:31 smithi018 ceph-mon[29336]: 2.5 deep-scrub ok 2024-09-10T19:06:31.443 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:31 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:06:31.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:31 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:06:31.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:31 smithi018 ceph-mon[29336]: 5.16 scrub starts 2024-09-10T19:06:31.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:31 smithi018 ceph-mon[29336]: 5.16 scrub ok 2024-09-10T19:06:31.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:31 smithi018 ceph-mon[29336]: 5.17 scrub starts 2024-09-10T19:06:31.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:31 smithi018 ceph-mon[29336]: 5.17 scrub ok 2024-09-10T19:06:31.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:31 smithi018 ceph-mon[29336]: osdmap e448: 8 total, 8 up, 6 in 2024-09-10T19:06:31.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:06:30 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:06:30] "GET /metrics HTTP/1.1" 200 224508 "" "Prometheus/2.43.0" 2024-09-10T19:06:31.484 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-10T19:06:31.484 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T19:06:31.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:31 smithi103 ceph-mon[33163]: 6.1 scrub starts 2024-09-10T19:06:31.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:31 smithi103 ceph-mon[33163]: 6.1 scrub ok 2024-09-10T19:06:31.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:31 smithi103 ceph-mon[33163]: 2.5 deep-scrub starts 2024-09-10T19:06:31.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:31 smithi103 ceph-mon[33163]: 2.5 deep-scrub ok 2024-09-10T19:06:31.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:31 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:06:31.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:31 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:06:31.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:31 smithi103 ceph-mon[33163]: 5.16 scrub starts 2024-09-10T19:06:31.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:31 smithi103 ceph-mon[33163]: 5.16 scrub ok 2024-09-10T19:06:31.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:31 smithi103 ceph-mon[33163]: 5.17 scrub starts 2024-09-10T19:06:31.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:31 smithi103 ceph-mon[33163]: 5.17 scrub ok 2024-09-10T19:06:31.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:31 smithi103 ceph-mon[33163]: osdmap e448: 8 total, 8 up, 6 in 2024-09-10T19:06:31.524 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:31.578 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:31.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:31.579 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:31.680 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:31.727 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:31.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:31.728 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:31.829 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:31.856 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:31.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:31.857 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:31.883 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:06:31.958 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:31.966 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:06:31.982 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:31.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:31.983 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:31.984 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:06:32.084 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:32.111 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:32.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:32.112 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:32.187 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is set 2024-09-10T19:06:32.213 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:32.240 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:32.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:32.241 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:32.305 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:32 smithi018 ceph-mon[29336]: 11.6 deep-scrub starts 2024-09-10T19:06:32.306 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:32 smithi018 ceph-mon[29336]: pgmap v999: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 803 MiB data, 11 GiB used, 526 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 1.96k op/s; 343/39117 objects misplaced (0.877%) 2024-09-10T19:06:32.306 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:32 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3162410471' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T19:06:32.306 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:32 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T19:06:32.306 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:32 smithi018 ceph-mon[40196]: 11.6 deep-scrub starts 2024-09-10T19:06:32.306 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:32 smithi018 ceph-mon[40196]: pgmap v999: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 803 MiB data, 11 GiB used, 526 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 1.96k op/s; 343/39117 objects misplaced (0.877%) 2024-09-10T19:06:32.306 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:32 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3162410471' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T19:06:32.306 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:32 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T19:06:32.342 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:32.382 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:32.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:32.383 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:32.484 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:32.512 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:32.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:32.513 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:32.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:32 smithi103 ceph-mon[33163]: 11.6 deep-scrub starts 2024-09-10T19:06:32.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:32 smithi103 ceph-mon[33163]: pgmap v999: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 803 MiB data, 11 GiB used, 526 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 1.96k op/s; 343/39117 objects misplaced (0.877%) 2024-09-10T19:06:32.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:32 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3162410471' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T19:06:32.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:32 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-10T19:06:32.614 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:32.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:32.650 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:32.650 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:32.750 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:32.780 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:32.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:32.781 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:32.882 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:32.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:32.912 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:32.912 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:33.013 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:33.048 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:33.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:33.049 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:33.150 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:33.189 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:33.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:33.190 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:33.198 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:33 smithi018 ceph-mon[29336]: 5.8 scrub starts 2024-09-10T19:06:33.199 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:33 smithi018 ceph-mon[29336]: 5.8 scrub ok 2024-09-10T19:06:33.199 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:33 smithi018 ceph-mon[29336]: 1.14 scrub starts 2024-09-10T19:06:33.199 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:33 smithi018 ceph-mon[29336]: 1.14 scrub ok 2024-09-10T19:06:33.199 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:33 smithi018 ceph-mon[29336]: 2.15 scrub starts 2024-09-10T19:06:33.199 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:33 smithi018 ceph-mon[29336]: 2.15 scrub ok 2024-09-10T19:06:33.199 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:33 smithi018 ceph-mon[29336]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:06:33.199 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:33 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T19:06:33.199 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:33 smithi018 ceph-mon[29336]: osdmap e449: 8 total, 8 up, 6 in 2024-09-10T19:06:33.199 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:33 smithi018 ceph-mon[40196]: 5.8 scrub starts 2024-09-10T19:06:33.199 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:33 smithi018 ceph-mon[40196]: 5.8 scrub ok 2024-09-10T19:06:33.199 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:33 smithi018 ceph-mon[40196]: 1.14 scrub starts 2024-09-10T19:06:33.199 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:33 smithi018 ceph-mon[40196]: 1.14 scrub ok 2024-09-10T19:06:33.200 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:33 smithi018 ceph-mon[40196]: 2.15 scrub starts 2024-09-10T19:06:33.200 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:33 smithi018 ceph-mon[40196]: 2.15 scrub ok 2024-09-10T19:06:33.200 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:33 smithi018 ceph-mon[40196]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:06:33.200 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:33 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T19:06:33.200 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:33 smithi018 ceph-mon[40196]: osdmap e449: 8 total, 8 up, 6 in 2024-09-10T19:06:33.290 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:33.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:33.328 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:33.328 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:33.429 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:33.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:33.463 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:33.463 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:33.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:33 smithi103 ceph-mon[33163]: 5.8 scrub starts 2024-09-10T19:06:33.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:33 smithi103 ceph-mon[33163]: 5.8 scrub ok 2024-09-10T19:06:33.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:33 smithi103 ceph-mon[33163]: 1.14 scrub starts 2024-09-10T19:06:33.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:33 smithi103 ceph-mon[33163]: 1.14 scrub ok 2024-09-10T19:06:33.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:33 smithi103 ceph-mon[33163]: 2.15 scrub starts 2024-09-10T19:06:33.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:33 smithi103 ceph-mon[33163]: 2.15 scrub ok 2024-09-10T19:06:33.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:33 smithi103 ceph-mon[33163]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:06:33.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:33 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-10T19:06:33.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:33 smithi103 ceph-mon[33163]: osdmap e449: 8 total, 8 up, 6 in 2024-09-10T19:06:33.564 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:33.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:33.611 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:33.611 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:33.712 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:33.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:33.775 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:33.775 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:33.876 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:33.922 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:33.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:33.923 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:34.024 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:34.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:34.062 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:34.062 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:34.163 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:34.201 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:34.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:34.202 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:34.303 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:34.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:34.354 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:34.354 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:34.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:34 smithi018 ceph-mon[29336]: 11.6 deep-scrub ok 2024-09-10T19:06:34.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:34 smithi018 ceph-mon[29336]: pgmap v1001: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 685 MiB data, 9.9 GiB used, 527 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 1.17k op/s; 343/33456 objects misplaced (1.025%) 2024-09-10T19:06:34.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:34 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:06:34.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:34 smithi018 ceph-mon[40196]: 11.6 deep-scrub ok 2024-09-10T19:06:34.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:34 smithi018 ceph-mon[40196]: pgmap v1001: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 685 MiB data, 9.9 GiB used, 527 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 1.17k op/s; 343/33456 objects misplaced (1.025%) 2024-09-10T19:06:34.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:34 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:06:34.454 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:34.483 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:34.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:34.484 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:34.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:34 smithi103 ceph-mon[33163]: 11.6 deep-scrub ok 2024-09-10T19:06:34.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:34 smithi103 ceph-mon[33163]: pgmap v1001: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 685 MiB data, 9.9 GiB used, 527 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 1.17k op/s; 343/33456 objects misplaced (1.025%) 2024-09-10T19:06:34.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:34 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:06:34.585 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:34.615 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:34.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:34.616 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:34.717 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:34.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:34.751 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:34.751 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:34.852 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:34.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:34.898 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:34.898 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:34.999 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:35.029 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:35.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:35.030 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:35.130 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:35.168 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:35.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:35.169 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:35.270 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:35.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:35.307 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:35.307 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:35.408 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:35.437 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:35.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:35.438 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:35.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:35 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:06:35.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:35 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:06:35.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:35 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:06:35.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:35 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:06:35.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:35 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:06:35.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:35 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:06:35.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:35 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:06:35.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:35 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:06:35.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:35 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:06:35.539 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:35.571 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:35.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:35.572 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:35.673 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:35.709 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:35.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:35.710 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:35.811 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:35.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:35.849 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:35.849 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:35.950 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:35.987 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:35.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:35.988 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:36.089 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:36.121 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:36.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:36.122 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:36.223 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:36.253 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:36.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:36.254 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:36.355 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:36.361 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-10T19:06:36.398 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:36.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:36.399 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:36.500 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:36.510 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:36 smithi018 ceph-mon[40196]: pgmap v1002: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 605 MiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 694 B/s rd, 0 B/s wr, 5 op/s; 343/29586 objects misplaced (1.159%) 2024-09-10T19:06:36.510 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:36 smithi018 ceph-mon[29336]: pgmap v1002: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 605 MiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 694 B/s rd, 0 B/s wr, 5 op/s; 343/29586 objects misplaced (1.159%) 2024-09-10T19:06:36.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:36 smithi103 ceph-mon[33163]: pgmap v1002: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 605 MiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 694 B/s rd, 0 B/s wr, 5 op/s; 343/29586 objects misplaced (1.159%) 2024-09-10T19:06:36.569 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:36.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:36.571 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:36.671 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:36.701 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:36.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:36.702 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:36.802 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:36.833 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:36.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:36.834 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:36.935 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:36.965 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:36.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:36.966 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:37.067 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:37.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:37.105 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:37.105 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:37.206 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:37.245 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:37.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:37.246 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:37.347 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:37.391 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:37.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:37.393 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:37.493 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:37.521 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:37.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:37.523 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:37.623 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:37.670 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:37.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:37.671 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:37.772 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:37.773 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:06:37 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:06:37.822 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:37.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:37.823 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:37.890 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Removed 137089 objects 2024-09-10T19:06:37.890 INFO:tasks.radosbench.radosbench.0.smithi018.stdout:Clean up completed and total clean up time :99.8051 2024-09-10T19:06:37.924 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:37.947 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:37.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:37.948 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:38.049 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:38.072 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:38.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:38.073 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:38.174 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:38.196 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:38.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:38.197 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:38.298 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:38.321 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:38.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:38.322 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:38.354 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T19:06:38.354 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T19:06:38.372 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T19:06:38.372 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T19:06:38.389 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T19:06:38.389 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T19:06:38.404 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T19:06:38.404 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T19:06:38.421 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T19:06:38.421 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T19:06:38.423 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:38.436 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T19:06:38.436 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T19:06:38.446 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:38.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:38.447 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:38.452 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T19:06:38.453 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T19:06:38.469 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T19:06:38.469 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T19:06:38.548 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:38.571 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:38.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:38.572 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:38.673 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:38.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:38 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:38.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:38 smithi018 ceph-mon[40196]: pgmap v1003: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 196 MiB data, 8.3 GiB used, 528 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 641 op/s; 54/9960 objects misplaced (0.542%) 2024-09-10T19:06:38.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:38 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1848852002' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:06:38.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:38 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:38.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:38 smithi018 ceph-mon[29336]: pgmap v1003: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 196 MiB data, 8.3 GiB used, 528 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 641 op/s; 54/9960 objects misplaced (0.542%) 2024-09-10T19:06:38.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:38 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1848852002' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:06:38.695 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:38.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:38.696 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:38.797 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:38.822 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:38.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:38.823 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:38.924 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:38.949 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:38.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:38.950 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:39.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:38 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:39.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:38 smithi103 ceph-mon[33163]: pgmap v1003: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 196 MiB data, 8.3 GiB used, 528 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 641 op/s; 54/9960 objects misplaced (0.542%) 2024-09-10T19:06:39.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:38 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1848852002' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:06:39.051 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:39.060 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T19:06:39.063 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-09-10T19:06:39.075 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:39.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:39.076 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:39.176 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:39.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:39.202 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:39.202 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:39.303 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:39.326 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:39.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:39.327 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:39.428 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:39.451 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:39.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:39.452 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:39.553 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:39.580 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:39.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:39.581 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:39.681 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:39.705 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:39.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:39.706 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:39.807 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:39.832 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:39.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:39.834 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:39.934 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:39.957 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:39.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:39.959 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:40.060 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:40.083 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:40.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:40.084 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:40.184 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:40.211 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:40.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:40.212 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:40.285 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":1003,"stamp":"2024-09-10T19:06:37.516737+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":205297908,"num_objects":3320,"num_object_clones":0,"num_object_copies":9960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":54,"num_objects_unfound":0,"num_objects_dirty":3320,"num_whiteouts":0,"num_read":1913,"num_read_kb":1665,"num_write":263820,"num_write_kb":8499500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31730,"num_bytes_recovered":2063734085,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91361,"ondisk_log_size":91361,"up":534,"acting":534,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":548,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":562470912,"kb_used":8747192,"kb_used_data":1583432,"kb_used_omap":12,"kb_used_meta":7163507,"kb_avail":553723720,"statfs":{"total":575970213888,"available":567013089280,"internally_reserved":0,"allocated":1621434368,"data_stored":1619178158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13122,"internal_metadata":7335431358},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-875724800,"num_objects":-13363,"num_object_clones":0,"num_object_copies":-40089,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-493,"num_objects_unfound":0,"num_objects_dirty":-13363,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":5125,"num_write_kb":-274240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002399"},"pg_stats":[{"pgid":"11.16","version":"449'11921","reported_seq":21710,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:36.659479+0000","last_change":"2024-09-10T19:05:41.625531+0000","last_active":"2024-09-10T19:06:36.659479+0000","last_peered":"2024-09-10T19:06:36.659479+0000","last_clean":"2024-09-10T19:06:36.659479+0000","last_became_active":"2024-09-10T19:05:41.596690+0000","last_became_peered":"2024-09-10T19:05:41.596690+0000","last_unstale":"2024-09-10T19:06:36.659479+0000","last_undegraded":"2024-09-10T19:06:36.659479+0000","last_fullsized":"2024-09-10T19:06:36.659479+0000","mapping_epoch":435,"log_start":"424'8585","ondisk_log_start":"424'8585","created":404,"last_epoch_clean":436,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3336,"log_dups_size":0,"ondisk_log_size":3336,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3080192,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8447,"num_write_kb":271808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3225,"num_bytes_recovered":211353600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":472,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.171716+0000","last_change":"2024-09-10T19:06:27.199518+0000","last_active":"2024-09-10T19:06:30.171716+0000","last_peered":"2024-09-10T19:06:30.171716+0000","last_clean":"2024-09-10T19:06:30.171716+0000","last_became_active":"2024-09-10T18:57:02.460451+0000","last_became_peered":"2024-09-10T18:57:02.460451+0000","last_unstale":"2024-09-10T19:06:30.171716+0000","last_undegraded":"2024-09-10T19:06:30.171716+0000","last_fullsized":"2024-09-10T19:06:30.171716+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:27.199436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:06:27.199436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:29.399699+0000","scrub_duration":0.0067911150000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":459,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.168481+0000","last_change":"2024-09-10T19:03:38.825020+0000","last_active":"2024-09-10T19:06:30.168481+0000","last_peered":"2024-09-10T19:06:30.168481+0000","last_clean":"2024-09-10T19:06:30.168481+0000","last_became_active":"2024-09-10T18:57:02.440648+0000","last_became_peered":"2024-09-10T18:57:02.440648+0000","last_unstale":"2024-09-10T19:06:30.168481+0000","last_undegraded":"2024-09-10T19:06:30.168481+0000","last_fullsized":"2024-09-10T19:06:30.168481+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:38.824816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:59.848700+0000","last_clean_scrub_stamp":"2024-09-10T19:03:38.824816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0071402059999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"427'39","reported_seq":565,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.172715+0000","last_change":"2024-09-10T19:02:20.251547+0000","last_active":"2024-09-10T19:06:30.172715+0000","last_peered":"2024-09-10T19:06:30.172715+0000","last_clean":"2024-09-10T19:06:30.172715+0000","last_became_active":"2024-09-10T18:55:40.394717+0000","last_became_peered":"2024-09-10T18:55:40.394717+0000","last_unstale":"2024-09-10T19:06:30.172715+0000","last_undegraded":"2024-09-10T19:06:30.172715+0000","last_fullsized":"2024-09-10T19:06:30.172715+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'27","last_scrub_stamp":"2024-09-10T19:02:20.251456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:20.251456+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011781700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":464,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169133+0000","last_change":"2024-09-10T19:04:39.483398+0000","last_active":"2024-09-10T19:06:30.169133+0000","last_peered":"2024-09-10T19:06:30.169133+0000","last_clean":"2024-09-10T19:06:30.169133+0000","last_became_active":"2024-09-10T18:45:22.065619+0000","last_became_peered":"2024-09-10T18:45:22.065619+0000","last_unstale":"2024-09-10T19:06:30.169133+0000","last_undegraded":"2024-09-10T19:06:30.169133+0000","last_fullsized":"2024-09-10T19:06:30.169133+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:39.483330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.057932+0000","last_clean_scrub_stamp":"2024-09-10T19:04:39.483330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016448439999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"449'11862","reported_seq":18713,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:36.654530+0000","last_change":"2024-09-10T19:05:33.839174+0000","last_active":"2024-09-10T19:06:36.654530+0000","last_peered":"2024-09-10T19:06:36.654530+0000","last_clean":"2024-09-10T19:06:36.654530+0000","last_became_active":"2024-09-10T19:05:33.821585+0000","last_became_peered":"2024-09-10T19:05:33.821585+0000","last_unstale":"2024-09-10T19:06:36.654530+0000","last_undegraded":"2024-09-10T19:06:36.654530+0000","last_fullsized":"2024-09-10T19:06:36.654530+0000","mapping_epoch":429,"log_start":"426'8646","ondisk_log_start":"426'8646","created":404,"last_epoch_clean":430,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3216,"log_dups_size":0,"ondisk_log_size":3216,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4030464,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8400,"num_write_kb":270752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3369,"num_bytes_recovered":220790784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":525,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.171118+0000","last_change":"2024-09-10T19:02:17.284387+0000","last_active":"2024-09-10T19:06:30.171118+0000","last_peered":"2024-09-10T19:06:30.171118+0000","last_clean":"2024-09-10T19:06:30.171118+0000","last_became_active":"2024-09-10T19:00:42.518824+0000","last_became_peered":"2024-09-10T19:00:42.518824+0000","last_unstale":"2024-09-10T19:06:30.171118+0000","last_undegraded":"2024-09-10T19:06:30.171118+0000","last_fullsized":"2024-09-10T19:06:30.171118+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:17.284305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:17.284305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018984430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"427'25","reported_seq":494,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169058+0000","last_change":"2024-09-10T19:04:28.445705+0000","last_active":"2024-09-10T19:06:30.169058+0000","last_peered":"2024-09-10T19:06:30.169058+0000","last_clean":"2024-09-10T19:06:30.169058+0000","last_became_active":"2024-09-10T18:45:20.061266+0000","last_became_peered":"2024-09-10T18:45:20.061266+0000","last_unstale":"2024-09-10T19:06:30.169058+0000","last_undegraded":"2024-09-10T19:06:30.169058+0000","last_fullsized":"2024-09-10T19:06:30.169058+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T19:04:28.445625+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_clean_scrub_stamp":"2024-09-10T19:04:28.445625+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047732149999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":100,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.166050+0000","last_change":"2024-09-10T19:04:39.088299+0000","last_active":"2024-09-10T19:06:30.166050+0000","last_peered":"2024-09-10T19:06:30.166050+0000","last_clean":"2024-09-10T19:06:30.166050+0000","last_became_active":"2024-09-10T19:02:49.571992+0000","last_became_peered":"2024-09-10T19:02:49.571992+0000","last_unstale":"2024-09-10T19:06:30.166050+0000","last_undegraded":"2024-09-10T19:06:30.166050+0000","last_fullsized":"2024-09-10T19:06:30.166050+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:39.088231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:01.135988+0000","last_clean_scrub_stamp":"2024-09-10T19:04:39.088231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014640530000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":452,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169704+0000","last_change":"2024-09-10T19:03:41.617846+0000","last_active":"2024-09-10T19:06:30.169704+0000","last_peered":"2024-09-10T19:06:30.169704+0000","last_clean":"2024-09-10T19:06:30.169704+0000","last_became_active":"2024-09-10T18:45:24.081002+0000","last_became_peered":"2024-09-10T18:45:24.081002+0000","last_unstale":"2024-09-10T19:06:30.169704+0000","last_undegraded":"2024-09-10T19:06:30.169704+0000","last_fullsized":"2024-09-10T19:06:30.169704+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:41.617790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:03:41.617790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014252819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.14","version":"449'12154","reported_seq":20145,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:36.641963+0000","last_change":"2024-09-10T19:04:55.871256+0000","last_active":"2024-09-10T19:06:36.641963+0000","last_peered":"2024-09-10T19:06:36.641963+0000","last_clean":"2024-09-10T19:06:36.641963+0000","last_became_active":"2024-09-10T19:04:55.852590+0000","last_became_peered":"2024-09-10T19:04:55.852590+0000","last_unstale":"2024-09-10T19:06:36.641963+0000","last_undegraded":"2024-09-10T19:06:36.641963+0000","last_fullsized":"2024-09-10T19:06:36.641963+0000","mapping_epoch":413,"log_start":"427'8992","ondisk_log_start":"427'8992","created":404,"last_epoch_clean":414,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3162,"log_dups_size":0,"ondisk_log_size":3162,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3407872,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8616,"num_write_kb":277376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3939,"num_bytes_recovered":258146304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":457,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.164460+0000","last_change":"2024-09-10T19:03:30.120786+0000","last_active":"2024-09-10T19:06:30.164460+0000","last_peered":"2024-09-10T19:06:30.164460+0000","last_clean":"2024-09-10T19:06:30.164460+0000","last_became_active":"2024-09-10T18:57:02.452259+0000","last_became_peered":"2024-09-10T18:57:02.452259+0000","last_unstale":"2024-09-10T19:06:30.164460+0000","last_undegraded":"2024-09-10T19:06:30.164460+0000","last_fullsized":"2024-09-10T19:06:30.164460+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:30.120695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:21.501984+0000","last_clean_scrub_stamp":"2024-09-10T19:03:30.120695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034594489999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1b","version":"427'11","reported_seq":474,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169230+0000","last_change":"2024-09-10T19:04:26.520312+0000","last_active":"2024-09-10T19:06:30.169230+0000","last_peered":"2024-09-10T19:06:30.169230+0000","last_clean":"2024-09-10T19:06:30.169230+0000","last_became_active":"2024-09-10T18:45:20.057567+0000","last_became_peered":"2024-09-10T18:45:20.057567+0000","last_unstale":"2024-09-10T19:06:30.169230+0000","last_undegraded":"2024-09-10T19:06:30.169230+0000","last_fullsized":"2024-09-10T19:06:30.169230+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'9","last_scrub_stamp":"2024-09-10T19:04:26.520225+0000","last_deep_scrub":"163'7","last_deep_scrub_stamp":"2024-09-10T18:51:54.261879+0000","last_clean_scrub_stamp":"2024-09-10T19:04:26.520225+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005488153,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":13,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":223,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169636+0000","last_change":"2024-09-10T19:06:11.858817+0000","last_active":"2024-09-10T19:06:30.169636+0000","last_peered":"2024-09-10T19:06:30.169636+0000","last_clean":"2024-09-10T19:06:30.169636+0000","last_became_active":"2024-09-10T18:58:58.503924+0000","last_became_peered":"2024-09-10T18:58:58.503924+0000","last_unstale":"2024-09-10T19:06:30.169636+0000","last_undegraded":"2024-09-10T19:06:30.169636+0000","last_fullsized":"2024-09-10T19:06:30.169636+0000","mapping_epoch":303,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":304,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:11.858728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:36.509971+0000","last_clean_scrub_stamp":"2024-09-10T19:06:11.858728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:19.964860+0000","scrub_duration":0.0083694589999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":130,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.166199+0000","last_change":"2024-09-10T19:03:23.759714+0000","last_active":"2024-09-10T19:06:30.166199+0000","last_peered":"2024-09-10T19:06:30.166199+0000","last_clean":"2024-09-10T19:06:30.166199+0000","last_became_active":"2024-09-10T19:00:42.515840+0000","last_became_peered":"2024-09-10T19:00:42.515840+0000","last_unstale":"2024-09-10T19:06:30.166199+0000","last_undegraded":"2024-09-10T19:06:30.166199+0000","last_fullsized":"2024-09-10T19:06:30.166199+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:23.759624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:03:23.759624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037605970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.15","version":"449'12050","reported_seq":19287,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:36.433727+0000","last_change":"2024-09-10T19:06:01.672213+0000","last_active":"2024-09-10T19:06:36.433727+0000","last_peered":"2024-09-10T19:06:36.433727+0000","last_clean":"2024-09-10T19:06:36.433727+0000","last_became_active":"2024-09-10T19:06:01.618213+0000","last_became_peered":"2024-09-10T19:06:01.618213+0000","last_unstale":"2024-09-10T19:06:36.433727+0000","last_undegraded":"2024-09-10T19:06:36.433727+0000","last_fullsized":"2024-09-10T19:06:36.433727+0000","mapping_epoch":438,"log_start":"423'8460","ondisk_log_start":"423'8460","created":404,"last_epoch_clean":439,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3590,"log_dups_size":0,"ondisk_log_size":3590,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8589,"num_write_kb":274464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2372,"num_bytes_recovered":155385856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":273,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.167374+0000","last_change":"2024-09-10T19:06:11.016240+0000","last_active":"2024-09-10T19:06:30.167374+0000","last_peered":"2024-09-10T19:06:30.167374+0000","last_clean":"2024-09-10T19:06:30.167374+0000","last_became_active":"2024-09-10T18:57:02.440497+0000","last_became_peered":"2024-09-10T18:57:02.440497+0000","last_unstale":"2024-09-10T19:06:30.167374+0000","last_undegraded":"2024-09-10T19:06:30.167374+0000","last_fullsized":"2024-09-10T19:06:30.167374+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:11.016144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:06:11.016144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:29.801511+0000","scrub_duration":0.015619935999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"427'21","reported_seq":519,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169972+0000","last_change":"2024-09-10T19:02:22.113293+0000","last_active":"2024-09-10T19:06:30.169972+0000","last_peered":"2024-09-10T19:06:30.169972+0000","last_clean":"2024-09-10T19:06:30.169972+0000","last_became_active":"2024-09-10T18:55:40.391571+0000","last_became_peered":"2024-09-10T18:55:40.391571+0000","last_unstale":"2024-09-10T19:06:30.169972+0000","last_undegraded":"2024-09-10T19:06:30.169972+0000","last_fullsized":"2024-09-10T19:06:30.169972+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T19:02:22.113140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:22.113140+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0061707400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":493,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.167013+0000","last_change":"2024-09-10T19:02:55.736359+0000","last_active":"2024-09-10T19:06:30.167013+0000","last_peered":"2024-09-10T19:06:30.167013+0000","last_clean":"2024-09-10T19:06:30.167013+0000","last_became_active":"2024-09-10T18:57:02.431585+0000","last_became_peered":"2024-09-10T18:57:02.431585+0000","last_unstale":"2024-09-10T19:06:30.167013+0000","last_undegraded":"2024-09-10T19:06:30.167013+0000","last_fullsized":"2024-09-10T19:06:30.167013+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:55.736308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:02:55.736308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012293950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":158,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.165362+0000","last_change":"2024-09-10T19:06:15.668283+0000","last_active":"2024-09-10T19:06:30.165362+0000","last_peered":"2024-09-10T19:06:30.165362+0000","last_clean":"2024-09-10T19:06:30.165362+0000","last_became_active":"2024-09-10T18:59:48.737788+0000","last_became_peered":"2024-09-10T18:59:48.737788+0000","last_unstale":"2024-09-10T19:06:30.165362+0000","last_undegraded":"2024-09-10T19:06:30.165362+0000","last_fullsized":"2024-09-10T19:06:30.165362+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:15.668048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:59.425718+0000","last_clean_scrub_stamp":"2024-09-10T19:06:15.668048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:33.048610+0000","scrub_duration":0.0064859770000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"449'12203","reported_seq":12291,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:36.397262+0000","last_change":"2024-09-10T19:04:36.967513+0000","last_active":"2024-09-10T19:06:36.397262+0000","last_peered":"2024-09-10T19:06:36.397262+0000","last_clean":"2024-09-10T19:06:36.397262+0000","last_became_active":"2024-09-10T19:04:36.964769+0000","last_became_peered":"2024-09-10T19:04:36.964769+0000","last_unstale":"2024-09-10T19:06:36.397262+0000","last_undegraded":"2024-09-10T19:06:36.397262+0000","last_fullsized":"2024-09-10T19:06:36.397262+0000","mapping_epoch":379,"log_start":"427'8921","ondisk_log_start":"427'8921","created":379,"last_epoch_clean":405,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3282,"log_dups_size":0,"ondisk_log_size":3282,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5144576,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8654,"num_write_kb":279456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"70'81","reported_seq":134,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.165726+0000","last_change":"2024-09-10T19:03:21.755540+0000","last_active":"2024-09-10T19:06:30.165726+0000","last_peered":"2024-09-10T19:06:30.165726+0000","last_clean":"2024-09-10T19:06:30.165726+0000","last_became_active":"2024-09-10T19:00:42.517072+0000","last_became_peered":"2024-09-10T19:00:42.517072+0000","last_unstale":"2024-09-10T19:06:30.165726+0000","last_undegraded":"2024-09-10T19:06:30.165726+0000","last_fullsized":"2024-09-10T19:06:30.165726+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:03:21.755481+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:03:21.755481+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016110499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":262688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"427'53","reported_seq":572,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169673+0000","last_change":"2024-09-10T19:04:33.387338+0000","last_active":"2024-09-10T19:06:30.169673+0000","last_peered":"2024-09-10T19:06:30.169673+0000","last_clean":"2024-09-10T19:06:30.169673+0000","last_became_active":"2024-09-10T18:47:36.409300+0000","last_became_peered":"2024-09-10T18:47:36.409300+0000","last_unstale":"2024-09-10T19:06:30.169673+0000","last_undegraded":"2024-09-10T19:06:30.169673+0000","last_fullsized":"2024-09-10T19:06:30.169673+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'39","last_scrub_stamp":"2024-09-10T19:04:33.387246+0000","last_deep_scrub":"78'25","last_deep_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_clean_scrub_stamp":"2024-09-10T19:04:33.387246+0000","objects_scrubbed":8,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056627190000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":71,"num_read_kb":61,"num_write":46,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":210,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169382+0000","last_change":"2024-09-10T19:04:38.456757+0000","last_active":"2024-09-10T19:06:30.169382+0000","last_peered":"2024-09-10T19:06:30.169382+0000","last_clean":"2024-09-10T19:06:30.169382+0000","last_became_active":"2024-09-10T18:57:02.437260+0000","last_became_peered":"2024-09-10T18:57:02.437260+0000","last_unstale":"2024-09-10T19:06:30.169382+0000","last_undegraded":"2024-09-10T19:06:30.169382+0000","last_fullsized":"2024-09-10T19:06:30.169382+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:38.456644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:04:38.456644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012890429999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":333,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169552+0000","last_change":"2024-09-10T19:06:09.867580+0000","last_active":"2024-09-10T19:06:30.169552+0000","last_peered":"2024-09-10T19:06:30.169552+0000","last_clean":"2024-09-10T19:06:30.169552+0000","last_became_active":"2024-09-10T19:02:49.594088+0000","last_became_peered":"2024-09-10T19:02:49.594088+0000","last_unstale":"2024-09-10T19:06:30.169552+0000","last_undegraded":"2024-09-10T19:06:30.169552+0000","last_fullsized":"2024-09-10T19:06:30.169552+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:09.867514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:04.512425+0000","last_clean_scrub_stamp":"2024-09-10T19:06:09.867514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:31.816149+0000","scrub_duration":0.0030367480000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":381,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.168618+0000","last_change":"2024-09-10T19:06:22.676283+0000","last_active":"2024-09-10T19:06:30.168618+0000","last_peered":"2024-09-10T19:06:30.168618+0000","last_clean":"2024-09-10T19:06:30.168618+0000","last_became_active":"2024-09-10T18:55:40.388154+0000","last_became_peered":"2024-09-10T18:55:40.388154+0000","last_unstale":"2024-09-10T19:06:30.168618+0000","last_undegraded":"2024-09-10T19:06:30.168618+0000","last_fullsized":"2024-09-10T19:06:30.168618+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:22.676194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:06:22.676194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:43.573594+0000","scrub_duration":0.0084613770000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3","version":"449'11827","reported_seq":11915,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:36.609239+0000","last_change":"2024-09-10T19:04:36.969685+0000","last_active":"2024-09-10T19:06:36.609239+0000","last_peered":"2024-09-10T19:06:36.609239+0000","last_clean":"2024-09-10T19:06:36.609239+0000","last_became_active":"2024-09-10T19:04:36.967115+0000","last_became_peered":"2024-09-10T19:04:36.967115+0000","last_unstale":"2024-09-10T19:06:36.609239+0000","last_undegraded":"2024-09-10T19:06:36.609239+0000","last_fullsized":"2024-09-10T19:06:36.609239+0000","mapping_epoch":379,"log_start":"427'8633","ondisk_log_start":"427'8633","created":379,"last_epoch_clean":405,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3194,"log_dups_size":0,"ondisk_log_size":3194,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3309568,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8390,"num_write_kb":270112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.9","version":"70'81","reported_seq":366,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.173048+0000","last_change":"2024-09-10T19:04:31.516816+0000","last_active":"2024-09-10T19:06:30.173048+0000","last_peered":"2024-09-10T19:06:30.173048+0000","last_clean":"2024-09-10T19:06:30.173048+0000","last_became_active":"2024-09-10T18:57:02.463750+0000","last_became_peered":"2024-09-10T18:57:02.463750+0000","last_unstale":"2024-09-10T19:06:30.173048+0000","last_undegraded":"2024-09-10T19:06:30.173048+0000","last_fullsized":"2024-09-10T19:06:30.173048+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:04:31.516600+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:04:31.516600+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011586271,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"427'35","reported_seq":506,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169661+0000","last_change":"2024-09-10T19:03:34.532084+0000","last_active":"2024-09-10T19:06:30.169661+0000","last_peered":"2024-09-10T19:06:30.169661+0000","last_clean":"2024-09-10T19:06:30.169661+0000","last_became_active":"2024-09-10T18:45:20.053428+0000","last_became_peered":"2024-09-10T18:45:20.053428+0000","last_unstale":"2024-09-10T19:06:30.169661+0000","last_undegraded":"2024-09-10T19:06:30.169661+0000","last_fullsized":"2024-09-10T19:06:30.169661+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'25","last_scrub_stamp":"2024-09-10T19:03:34.532019+0000","last_deep_scrub":"116'15","last_deep_scrub_stamp":"2024-09-10T18:49:12.374527+0000","last_clean_scrub_stamp":"2024-09-10T19:03:34.532019+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020774220000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":372,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.170762+0000","last_change":"2024-09-10T19:06:17.129138+0000","last_active":"2024-09-10T19:06:30.170762+0000","last_peered":"2024-09-10T19:06:30.170762+0000","last_clean":"2024-09-10T19:06:30.170762+0000","last_became_active":"2024-09-10T19:02:49.571995+0000","last_became_peered":"2024-09-10T19:02:49.571995+0000","last_unstale":"2024-09-10T19:06:30.170762+0000","last_undegraded":"2024-09-10T19:06:30.170762+0000","last_fullsized":"2024-09-10T19:06:30.170762+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:17.129057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:06:17.129057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:42.800560+0000","scrub_duration":0.0074624499999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"273'13","reported_seq":586,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:32.181300+0000","last_change":"2024-09-10T19:06:23.008122+0000","last_active":"2024-09-10T19:06:32.181300+0000","last_peered":"2024-09-10T19:06:32.181300+0000","last_clean":"2024-09-10T19:06:32.181300+0000","last_became_active":"2024-09-10T18:57:02.448258+0000","last_became_peered":"2024-09-10T18:57:02.448258+0000","last_unstale":"2024-09-10T19:06:32.181300+0000","last_undegraded":"2024-09-10T19:06:32.181300+0000","last_fullsized":"2024-09-10T19:06:32.181300+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'13","last_scrub_stamp":"2024-09-10T19:06:23.008037+0000","last_deep_scrub":"273'13","last_deep_scrub_stamp":"2024-09-10T19:06:23.008037+0000","last_clean_scrub_stamp":"2024-09-10T19:06:23.008037+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:42.109075+0000","scrub_duration":0.013286312,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":502,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169054+0000","last_change":"2024-09-10T19:06:02.941293+0000","last_active":"2024-09-10T19:06:30.169054+0000","last_peered":"2024-09-10T19:06:30.169054+0000","last_clean":"2024-09-10T19:06:30.169054+0000","last_became_active":"2024-09-10T19:00:42.515523+0000","last_became_peered":"2024-09-10T19:00:42.515523+0000","last_unstale":"2024-09-10T19:06:30.169054+0000","last_undegraded":"2024-09-10T19:06:30.169054+0000","last_fullsized":"2024-09-10T19:06:30.169054+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:02.941202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:32.043666+0000","last_clean_scrub_stamp":"2024-09-10T19:06:02.941202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:07.258125+0000","scrub_duration":0.0062417180000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.0","version":"449'11838","reported_seq":11926,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:36.385586+0000","last_change":"2024-09-10T19:04:36.990388+0000","last_active":"2024-09-10T19:06:36.385586+0000","last_peered":"2024-09-10T19:06:36.385586+0000","last_clean":"2024-09-10T19:06:36.385586+0000","last_became_active":"2024-09-10T19:04:36.984829+0000","last_became_peered":"2024-09-10T19:04:36.984829+0000","last_unstale":"2024-09-10T19:06:36.385586+0000","last_undegraded":"2024-09-10T19:06:36.385586+0000","last_fullsized":"2024-09-10T19:06:36.385586+0000","mapping_epoch":379,"log_start":"424'8586","ondisk_log_start":"424'8586","created":379,"last_epoch_clean":405,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3252,"log_dups_size":0,"ondisk_log_size":3252,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4489216,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8430,"num_write_kb":271968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.a","version":"70'81","reported_seq":381,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.167084+0000","last_change":"2024-09-10T19:02:23.723122+0000","last_active":"2024-09-10T19:06:30.167084+0000","last_peered":"2024-09-10T19:06:30.167084+0000","last_clean":"2024-09-10T19:06:30.167084+0000","last_became_active":"2024-09-10T18:50:08.398345+0000","last_became_peered":"2024-09-10T18:50:08.398345+0000","last_unstale":"2024-09-10T19:06:30.167084+0000","last_undegraded":"2024-09-10T19:06:30.167084+0000","last_fullsized":"2024-09-10T19:06:30.167084+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:23.723036+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:55:37.921519+0000","last_clean_scrub_stamp":"2024-09-10T19:02:23.723036+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0054938950000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"427'46","reported_seq":506,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169717+0000","last_change":"2024-09-10T19:04:33.010380+0000","last_active":"2024-09-10T19:06:30.169717+0000","last_peered":"2024-09-10T19:06:30.169717+0000","last_clean":"2024-09-10T19:06:30.169717+0000","last_became_active":"2024-09-10T18:46:56.742946+0000","last_became_peered":"2024-09-10T18:46:56.742946+0000","last_unstale":"2024-09-10T19:06:30.169717+0000","last_undegraded":"2024-09-10T19:06:30.169717+0000","last_fullsized":"2024-09-10T19:06:30.169717+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'34","last_scrub_stamp":"2024-09-10T19:04:33.010321+0000","last_deep_scrub":"141'22","last_deep_scrub_stamp":"2024-09-10T18:50:57.119152+0000","last_clean_scrub_stamp":"2024-09-10T19:04:33.010321+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0031820009999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":375,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.172888+0000","last_change":"2024-09-10T19:04:32.480047+0000","last_active":"2024-09-10T19:06:30.172888+0000","last_peered":"2024-09-10T19:06:30.172888+0000","last_clean":"2024-09-10T19:06:30.172888+0000","last_became_active":"2024-09-10T19:02:49.563535+0000","last_became_peered":"2024-09-10T19:02:49.563535+0000","last_unstale":"2024-09-10T19:06:30.172888+0000","last_undegraded":"2024-09-10T19:06:30.172888+0000","last_fullsized":"2024-09-10T19:06:30.172888+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:32.479945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_clean_scrub_stamp":"2024-09-10T19:04:32.479945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060807650000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"242'10","reported_seq":737,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:32.171463+0000","last_change":"2024-09-10T19:02:53.030726+0000","last_active":"2024-09-10T19:06:32.171463+0000","last_peered":"2024-09-10T19:06:32.171463+0000","last_clean":"2024-09-10T19:06:32.171463+0000","last_became_active":"2024-09-10T18:55:40.397246+0000","last_became_peered":"2024-09-10T18:55:40.397246+0000","last_unstale":"2024-09-10T19:06:32.171463+0000","last_undegraded":"2024-09-10T19:06:32.171463+0000","last_fullsized":"2024-09-10T19:06:32.171463+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"242'10","last_scrub_stamp":"2024-09-10T19:02:53.030666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:02:53.030666+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015331489999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":481,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.167876+0000","last_change":"2024-09-10T19:02:48.670915+0000","last_active":"2024-09-10T19:06:30.167876+0000","last_peered":"2024-09-10T19:06:30.167876+0000","last_clean":"2024-09-10T19:06:30.167876+0000","last_became_active":"2024-09-10T18:55:40.392003+0000","last_became_peered":"2024-09-10T18:55:40.392003+0000","last_unstale":"2024-09-10T19:06:30.167876+0000","last_undegraded":"2024-09-10T19:06:30.167876+0000","last_fullsized":"2024-09-10T19:06:30.167876+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:48.670862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:54.218602+0000","last_clean_scrub_stamp":"2024-09-10T19:02:48.670862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00108365,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1","version":"449'11804","reported_seq":11893,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:36.621875+0000","last_change":"2024-09-10T19:04:36.972439+0000","last_active":"2024-09-10T19:06:36.621875+0000","last_peered":"2024-09-10T19:06:36.621875+0000","last_clean":"2024-09-10T19:06:36.621875+0000","last_became_active":"2024-09-10T19:04:36.969739+0000","last_became_peered":"2024-09-10T19:04:36.969739+0000","last_unstale":"2024-09-10T19:06:36.621875+0000","last_undegraded":"2024-09-10T19:06:36.621875+0000","last_fullsized":"2024-09-10T19:06:36.621875+0000","mapping_epoch":379,"log_start":"427'8674","ondisk_log_start":"427'8674","created":379,"last_epoch_clean":405,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3130,"log_dups_size":0,"ondisk_log_size":3130,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6651904,"num_objects":101,"num_object_clones":0,"num_object_copies":303,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8395,"num_write_kb":271904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.b","version":"70'81","reported_seq":175,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.167513+0000","last_change":"2024-09-10T19:06:11.695308+0000","last_active":"2024-09-10T19:06:30.167513+0000","last_peered":"2024-09-10T19:06:30.167513+0000","last_clean":"2024-09-10T19:06:30.167513+0000","last_became_active":"2024-09-10T19:02:49.572057+0000","last_became_peered":"2024-09-10T19:02:49.572057+0000","last_unstale":"2024-09-10T19:06:30.167513+0000","last_undegraded":"2024-09-10T19:06:30.167513+0000","last_fullsized":"2024-09-10T19:06:30.167513+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:06:11.695225+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T19:02:17.370627+0000","last_clean_scrub_stamp":"2024-09-10T19:06:11.695225+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:24.298666+0000","scrub_duration":0.003975236,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"427'32","reported_seq":580,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.171053+0000","last_change":"2024-09-10T19:03:42.660938+0000","last_active":"2024-09-10T19:06:30.171053+0000","last_peered":"2024-09-10T19:06:30.171053+0000","last_clean":"2024-09-10T19:06:30.171053+0000","last_became_active":"2024-09-10T18:47:57.106935+0000","last_became_peered":"2024-09-10T18:47:57.106935+0000","last_unstale":"2024-09-10T19:06:30.171053+0000","last_undegraded":"2024-09-10T19:06:30.171053+0000","last_fullsized":"2024-09-10T19:06:30.171053+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T19:03:42.660792+0000","last_deep_scrub":"224'16","last_deep_scrub_stamp":"2024-09-10T18:54:49.296957+0000","last_clean_scrub_stamp":"2024-09-10T19:03:42.660792+0000","objects_scrubbed":5,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053846639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":464,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.167290+0000","last_change":"2024-09-10T19:06:01.706867+0000","last_active":"2024-09-10T19:06:30.167290+0000","last_peered":"2024-09-10T19:06:30.167290+0000","last_clean":"2024-09-10T19:06:30.167290+0000","last_became_active":"2024-09-10T18:57:02.443816+0000","last_became_peered":"2024-09-10T18:57:02.443816+0000","last_unstale":"2024-09-10T19:06:30.167290+0000","last_undegraded":"2024-09-10T19:06:30.167290+0000","last_fullsized":"2024-09-10T19:06:30.167290+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:01.706783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:06:01.706783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:28.681911+0000","scrub_duration":0.0097313599999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":479,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169594+0000","last_change":"2024-09-10T19:06:02.997863+0000","last_active":"2024-09-10T19:06:30.169594+0000","last_peered":"2024-09-10T19:06:30.169594+0000","last_clean":"2024-09-10T19:06:30.169594+0000","last_became_active":"2024-09-10T18:45:22.068918+0000","last_became_peered":"2024-09-10T18:45:22.068918+0000","last_unstale":"2024-09-10T19:06:30.169594+0000","last_undegraded":"2024-09-10T19:06:30.169594+0000","last_fullsized":"2024-09-10T19:06:30.169594+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:02.997745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_clean_scrub_stamp":"2024-09-10T19:06:02.997745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:10.273645+0000","scrub_duration":0.0015124940000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":452,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.170033+0000","last_change":"2024-09-10T19:03:24.593809+0000","last_active":"2024-09-10T19:06:30.170033+0000","last_peered":"2024-09-10T19:06:30.170033+0000","last_clean":"2024-09-10T19:06:30.170033+0000","last_became_active":"2024-09-10T18:45:24.264538+0000","last_became_peered":"2024-09-10T18:45:24.264538+0000","last_unstale":"2024-09-10T19:06:30.170033+0000","last_undegraded":"2024-09-10T19:06:30.170033+0000","last_fullsized":"2024-09-10T19:06:30.170033+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:24.593754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:51.398981+0000","last_clean_scrub_stamp":"2024-09-10T19:03:24.593754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001434999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.e","version":"449'16947","reported_seq":17025,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:36.492768+0000","last_change":"2024-09-10T19:03:16.044645+0000","last_active":"2024-09-10T19:06:36.492768+0000","last_peered":"2024-09-10T19:06:36.492768+0000","last_clean":"2024-09-10T19:06:36.492768+0000","last_became_active":"2024-09-10T19:03:16.044187+0000","last_became_peered":"2024-09-10T19:03:16.044187+0000","last_unstale":"2024-09-10T19:06:36.492768+0000","last_undegraded":"2024-09-10T19:06:36.492768+0000","last_fullsized":"2024-09-10T19:06:36.492768+0000","mapping_epoch":379,"log_start":"437'13283","ondisk_log_start":"437'13283","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3664,"log_dups_size":0,"ondisk_log_size":3664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10158080,"num_objects":155,"num_object_clones":0,"num_object_copies":465,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":547264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"70'81","reported_seq":410,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.170242+0000","last_change":"2024-09-10T19:03:30.624986+0000","last_active":"2024-09-10T19:06:30.170242+0000","last_peered":"2024-09-10T19:06:30.170242+0000","last_clean":"2024-09-10T19:06:30.170242+0000","last_became_active":"2024-09-10T19:00:42.519742+0000","last_became_peered":"2024-09-10T19:00:42.519742+0000","last_unstale":"2024-09-10T19:06:30.170242+0000","last_undegraded":"2024-09-10T19:06:30.170242+0000","last_fullsized":"2024-09-10T19:06:30.170242+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:03:30.624868+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:03:30.624868+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0036172140000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"427'49","reported_seq":526,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.168654+0000","last_change":"2024-09-10T19:03:29.581426+0000","last_active":"2024-09-10T19:06:30.168654+0000","last_peered":"2024-09-10T19:06:30.168654+0000","last_clean":"2024-09-10T19:06:30.168654+0000","last_became_active":"2024-09-10T18:45:20.052243+0000","last_became_peered":"2024-09-10T18:45:20.052243+0000","last_unstale":"2024-09-10T19:06:30.168654+0000","last_undegraded":"2024-09-10T19:06:30.168654+0000","last_fullsized":"2024-09-10T19:06:30.168654+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'35","last_scrub_stamp":"2024-09-10T19:03:29.581339+0000","last_deep_scrub":"143'21","last_deep_scrub_stamp":"2024-09-10T18:51:02.540216+0000","last_clean_scrub_stamp":"2024-09-10T19:03:29.581339+0000","objects_scrubbed":7,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0042812099999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":63,"num_read_kb":56,"num_write":42,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":501,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.162308+0000","last_change":"2024-09-10T19:04:29.252531+0000","last_active":"2024-09-10T19:06:30.162308+0000","last_peered":"2024-09-10T19:06:30.162308+0000","last_clean":"2024-09-10T19:06:30.162308+0000","last_became_active":"2024-09-10T18:57:02.453404+0000","last_became_peered":"2024-09-10T18:57:02.453404+0000","last_unstale":"2024-09-10T19:06:30.162308+0000","last_undegraded":"2024-09-10T19:06:30.162308+0000","last_fullsized":"2024-09-10T19:06:30.162308+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:29.252456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_clean_scrub_stamp":"2024-09-10T19:04:29.252456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0036547900000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":728,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:32.178560+0000","last_change":"2024-09-10T19:06:07.959700+0000","last_active":"2024-09-10T19:06:32.178560+0000","last_peered":"2024-09-10T19:06:32.178560+0000","last_clean":"2024-09-10T19:06:32.178560+0000","last_became_active":"2024-09-10T18:45:22.074540+0000","last_became_peered":"2024-09-10T18:45:22.074540+0000","last_unstale":"2024-09-10T19:06:32.178560+0000","last_undegraded":"2024-09-10T19:06:32.178560+0000","last_fullsized":"2024-09-10T19:06:32.178560+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T19:06:07.959488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:06:07.959488+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:21.540597+0000","scrub_duration":0.01085156,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":268,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.162095+0000","last_change":"2024-09-10T19:04:37.416533+0000","last_active":"2024-09-10T19:06:30.162095+0000","last_peered":"2024-09-10T19:06:30.162095+0000","last_clean":"2024-09-10T19:06:30.162095+0000","last_became_active":"2024-09-10T18:57:02.437100+0000","last_became_peered":"2024-09-10T18:57:02.437100+0000","last_unstale":"2024-09-10T19:06:30.162095+0000","last_undegraded":"2024-09-10T19:06:30.162095+0000","last_fullsized":"2024-09-10T19:06:30.162095+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:37.416450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:04:37.416450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0094483369999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"449'16876","reported_seq":16954,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:33.981506+0000","last_change":"2024-09-10T19:03:16.046832+0000","last_active":"2024-09-10T19:06:33.981506+0000","last_peered":"2024-09-10T19:06:33.981506+0000","last_clean":"2024-09-10T19:06:33.981506+0000","last_became_active":"2024-09-10T19:03:16.046538+0000","last_became_peered":"2024-09-10T19:03:16.046538+0000","last_unstale":"2024-09-10T19:06:33.981506+0000","last_undegraded":"2024-09-10T19:06:33.981506+0000","last_fullsized":"2024-09-10T19:06:33.981506+0000","mapping_epoch":379,"log_start":"437'13266","ondisk_log_start":"437'13266","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3610,"log_dups_size":0,"ondisk_log_size":3610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27787264,"num_objects":424,"num_object_clones":0,"num_object_copies":1272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16876,"num_write_kb":553600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"70'81","reported_seq":376,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.170061+0000","last_change":"2024-09-10T19:02:58.888049+0000","last_active":"2024-09-10T19:06:30.170061+0000","last_peered":"2024-09-10T19:06:30.170061+0000","last_clean":"2024-09-10T19:06:30.170061+0000","last_became_active":"2024-09-10T18:52:14.584798+0000","last_became_peered":"2024-09-10T18:52:14.584798+0000","last_unstale":"2024-09-10T19:06:30.170061+0000","last_undegraded":"2024-09-10T19:06:30.170061+0000","last_fullsized":"2024-09-10T19:06:30.170061+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:58.887944+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:56:31.481303+0000","last_clean_scrub_stamp":"2024-09-10T19:02:58.887944+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016190950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"427'42","reported_seq":640,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.168489+0000","last_change":"2024-09-10T19:02:48.361438+0000","last_active":"2024-09-10T19:06:30.168489+0000","last_peered":"2024-09-10T19:06:30.168489+0000","last_clean":"2024-09-10T19:06:30.168489+0000","last_became_active":"2024-09-10T18:59:48.742377+0000","last_became_peered":"2024-09-10T18:59:48.742377+0000","last_unstale":"2024-09-10T19:06:30.168489+0000","last_undegraded":"2024-09-10T19:06:30.168489+0000","last_fullsized":"2024-09-10T19:06:30.168489+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'30","last_scrub_stamp":"2024-09-10T19:02:48.361337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:48.361337+0000","objects_scrubbed":8,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011213163,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":70,"num_read_kb":58,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":109,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.162517+0000","last_change":"2024-09-10T19:06:27.610141+0000","last_active":"2024-09-10T19:06:30.162517+0000","last_peered":"2024-09-10T19:06:30.162517+0000","last_clean":"2024-09-10T19:06:30.162517+0000","last_became_active":"2024-09-10T19:02:49.578870+0000","last_became_peered":"2024-09-10T19:02:49.578870+0000","last_unstale":"2024-09-10T19:06:30.162517+0000","last_undegraded":"2024-09-10T19:06:30.162517+0000","last_fullsized":"2024-09-10T19:06:30.162517+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:27.610058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:46.142129+0000","last_clean_scrub_stamp":"2024-09-10T19:06:27.610058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:36.151562+0000","scrub_duration":0.0042716869999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":474,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.168429+0000","last_change":"2024-09-10T19:04:36.430586+0000","last_active":"2024-09-10T19:06:30.168429+0000","last_peered":"2024-09-10T19:06:30.168429+0000","last_clean":"2024-09-10T19:06:30.168429+0000","last_became_active":"2024-09-10T18:57:02.448126+0000","last_became_peered":"2024-09-10T18:57:02.448126+0000","last_unstale":"2024-09-10T19:06:30.168429+0000","last_undegraded":"2024-09-10T19:06:30.168429+0000","last_fullsized":"2024-09-10T19:06:30.168429+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:36.430526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:57.903311+0000","last_clean_scrub_stamp":"2024-09-10T19:04:36.430526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00133559,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":462,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169899+0000","last_change":"2024-09-10T19:03:36.541808+0000","last_active":"2024-09-10T19:06:30.169899+0000","last_peered":"2024-09-10T19:06:30.169899+0000","last_clean":"2024-09-10T19:06:30.169899+0000","last_became_active":"2024-09-10T18:47:36.410836+0000","last_became_peered":"2024-09-10T18:47:36.410836+0000","last_unstale":"2024-09-10T19:06:30.169899+0000","last_undegraded":"2024-09-10T19:06:30.169899+0000","last_fullsized":"2024-09-10T19:06:30.169899+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:36.541729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_clean_scrub_stamp":"2024-09-10T19:03:36.541729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001932762,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"449'17189","reported_seq":17267,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:36.509378+0000","last_change":"2024-09-10T19:03:16.046878+0000","last_active":"2024-09-10T19:06:36.509378+0000","last_peered":"2024-09-10T19:06:36.509378+0000","last_clean":"2024-09-10T19:06:36.509378+0000","last_became_active":"2024-09-10T19:03:16.046484+0000","last_became_peered":"2024-09-10T19:03:16.046484+0000","last_unstale":"2024-09-10T19:06:36.509378+0000","last_undegraded":"2024-09-10T19:06:36.509378+0000","last_fullsized":"2024-09-10T19:06:36.509378+0000","mapping_epoch":379,"log_start":"437'13580","ondisk_log_start":"437'13580","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3609,"log_dups_size":0,"ondisk_log_size":3609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8978432,"num_objects":137,"num_object_clones":0,"num_object_copies":411,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17189,"num_write_kb":554432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"70'81","reported_seq":484,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.162391+0000","last_change":"2024-09-10T19:06:25.532401+0000","last_active":"2024-09-10T19:06:30.162391+0000","last_peered":"2024-09-10T19:06:30.162391+0000","last_clean":"2024-09-10T19:06:30.162391+0000","last_became_active":"2024-09-10T18:55:40.396828+0000","last_became_peered":"2024-09-10T18:55:40.396828+0000","last_unstale":"2024-09-10T19:06:30.162391+0000","last_undegraded":"2024-09-10T19:06:30.162391+0000","last_fullsized":"2024-09-10T19:06:30.162391+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:06:25.532348+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T19:06:25.532348+0000","last_clean_scrub_stamp":"2024-09-10T19:06:25.532348+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:45.622134+0000","scrub_duration":0.0053591250000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"427'39","reported_seq":541,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.168212+0000","last_change":"2024-09-10T19:06:12.729237+0000","last_active":"2024-09-10T19:06:30.168212+0000","last_peered":"2024-09-10T19:06:30.168212+0000","last_clean":"2024-09-10T19:06:30.168212+0000","last_became_active":"2024-09-10T18:57:02.442862+0000","last_became_peered":"2024-09-10T18:57:02.442862+0000","last_unstale":"2024-09-10T19:06:30.168212+0000","last_undegraded":"2024-09-10T19:06:30.168212+0000","last_fullsized":"2024-09-10T19:06:30.168212+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'39","last_scrub_stamp":"2024-09-10T19:06:12.729127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:06:12.729127+0000","objects_scrubbed":8,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:35.794829+0000","scrub_duration":0.005856895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":68,"num_read_kb":54,"num_write":42,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":464,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.166413+0000","last_change":"2024-09-10T19:06:29.659585+0000","last_active":"2024-09-10T19:06:30.166413+0000","last_peered":"2024-09-10T19:06:30.166413+0000","last_clean":"2024-09-10T19:06:30.166413+0000","last_became_active":"2024-09-10T18:57:02.443522+0000","last_became_peered":"2024-09-10T18:57:02.443522+0000","last_unstale":"2024-09-10T19:06:30.166413+0000","last_undegraded":"2024-09-10T19:06:30.166413+0000","last_fullsized":"2024-09-10T19:06:30.166413+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:29.659530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:06:29.659530+0000","last_clean_scrub_stamp":"2024-09-10T19:06:29.659530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:49.158272+0000","scrub_duration":0.0029405880000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":497,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.163627+0000","last_change":"2024-09-10T19:06:07.512604+0000","last_active":"2024-09-10T19:06:30.163627+0000","last_peered":"2024-09-10T19:06:30.163627+0000","last_clean":"2024-09-10T19:06:30.163627+0000","last_became_active":"2024-09-10T18:55:40.387857+0000","last_became_peered":"2024-09-10T18:55:40.387857+0000","last_unstale":"2024-09-10T19:06:30.163627+0000","last_undegraded":"2024-09-10T19:06:30.163627+0000","last_fullsized":"2024-09-10T19:06:30.163627+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:07.512453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:06:07.512453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:09.727452+0000","scrub_duration":0.0057807240000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":377,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.164728+0000","last_change":"2024-09-10T19:06:29.568836+0000","last_active":"2024-09-10T19:06:30.164728+0000","last_peered":"2024-09-10T19:06:30.164728+0000","last_clean":"2024-09-10T19:06:30.164728+0000","last_became_active":"2024-09-10T19:02:49.561359+0000","last_became_peered":"2024-09-10T19:02:49.561359+0000","last_unstale":"2024-09-10T19:06:30.164728+0000","last_undegraded":"2024-09-10T19:06:30.164728+0000","last_fullsized":"2024-09-10T19:06:30.164728+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:29.568754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:49:18.281230+0000","last_clean_scrub_stamp":"2024-09-10T19:06:29.568754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:38.060685+0000","scrub_duration":0.002692238,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"449'17136","reported_seq":17215,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:36.450858+0000","last_change":"2024-09-10T19:03:16.044601+0000","last_active":"2024-09-10T19:06:36.450858+0000","last_peered":"2024-09-10T19:06:36.450858+0000","last_clean":"2024-09-10T19:06:36.450858+0000","last_became_active":"2024-09-10T19:03:16.044213+0000","last_became_peered":"2024-09-10T19:03:16.044213+0000","last_unstale":"2024-09-10T19:06:36.450858+0000","last_undegraded":"2024-09-10T19:06:36.450858+0000","last_fullsized":"2024-09-10T19:06:36.450858+0000","mapping_epoch":379,"log_start":"437'13477","ondisk_log_start":"437'13477","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3659,"log_dups_size":0,"ondisk_log_size":3659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11206680,"num_objects":172,"num_object_clones":0,"num_object_copies":516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17136,"num_write_kb":553793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"70'81","reported_seq":477,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.163315+0000","last_change":"2024-09-10T19:03:34.083497+0000","last_active":"2024-09-10T19:06:30.163315+0000","last_peered":"2024-09-10T19:06:30.163315+0000","last_clean":"2024-09-10T19:06:30.163315+0000","last_became_active":"2024-09-10T18:57:02.452659+0000","last_became_peered":"2024-09-10T18:57:02.452659+0000","last_unstale":"2024-09-10T19:06:30.163315+0000","last_undegraded":"2024-09-10T19:06:30.163315+0000","last_fullsized":"2024-09-10T19:06:30.163315+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:03:34.083013+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:03:34.083013+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023839249999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"427'35","reported_seq":550,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.166351+0000","last_change":"2024-09-10T19:03:52.017990+0000","last_active":"2024-09-10T19:06:30.166351+0000","last_peered":"2024-09-10T19:06:30.166351+0000","last_clean":"2024-09-10T19:06:30.166351+0000","last_became_active":"2024-09-10T19:02:49.579401+0000","last_became_peered":"2024-09-10T19:02:49.579401+0000","last_unstale":"2024-09-10T19:06:30.166351+0000","last_undegraded":"2024-09-10T19:06:30.166351+0000","last_fullsized":"2024-09-10T19:06:30.166351+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'25","last_scrub_stamp":"2024-09-10T19:03:52.017902+0000","last_deep_scrub":"83'15","last_deep_scrub_stamp":"2024-09-10T18:46:58.436473+0000","last_clean_scrub_stamp":"2024-09-10T19:03:52.017902+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024815190000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":390,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.165316+0000","last_change":"2024-09-10T19:06:02.753654+0000","last_active":"2024-09-10T19:06:30.165316+0000","last_peered":"2024-09-10T19:06:30.165316+0000","last_clean":"2024-09-10T19:06:30.165316+0000","last_became_active":"2024-09-10T19:02:49.561340+0000","last_became_peered":"2024-09-10T19:02:49.561340+0000","last_unstale":"2024-09-10T19:06:30.165316+0000","last_undegraded":"2024-09-10T19:06:30.165316+0000","last_fullsized":"2024-09-10T19:06:30.165316+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:02.753555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:05.140027+0000","last_clean_scrub_stamp":"2024-09-10T19:06:02.753555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:30.070995+0000","scrub_duration":0.0082683540000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"339'13","reported_seq":689,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:31.329500+0000","last_change":"2024-09-10T19:03:29.753145+0000","last_active":"2024-09-10T19:06:31.329500+0000","last_peered":"2024-09-10T19:06:31.329500+0000","last_clean":"2024-09-10T19:06:31.329500+0000","last_became_active":"2024-09-10T19:00:42.520360+0000","last_became_peered":"2024-09-10T19:00:42.520360+0000","last_unstale":"2024-09-10T19:06:31.329500+0000","last_undegraded":"2024-09-10T19:06:31.329500+0000","last_fullsized":"2024-09-10T19:06:31.329500+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"339'13","last_scrub_stamp":"2024-09-10T19:03:29.753059+0000","last_deep_scrub":"258'12","last_deep_scrub_stamp":"2024-09-10T18:56:21.098770+0000","last_clean_scrub_stamp":"2024-09-10T19:03:29.753059+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002665485,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":450,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.167506+0000","last_change":"2024-09-10T19:04:39.703670+0000","last_active":"2024-09-10T19:06:30.167506+0000","last_peered":"2024-09-10T19:06:30.167506+0000","last_clean":"2024-09-10T19:06:30.167506+0000","last_became_active":"2024-09-10T19:00:42.515144+0000","last_became_peered":"2024-09-10T19:00:42.515144+0000","last_unstale":"2024-09-10T19:06:30.167506+0000","last_undegraded":"2024-09-10T19:06:30.167506+0000","last_fullsized":"2024-09-10T19:06:30.167506+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:39.703610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:04:39.703610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022926890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"449'16995","reported_seq":17072,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:36.412789+0000","last_change":"2024-09-10T19:03:16.048706+0000","last_active":"2024-09-10T19:06:36.412789+0000","last_peered":"2024-09-10T19:06:36.412789+0000","last_clean":"2024-09-10T19:06:36.412789+0000","last_became_active":"2024-09-10T19:03:16.048344+0000","last_became_peered":"2024-09-10T19:03:16.048344+0000","last_unstale":"2024-09-10T19:06:36.412789+0000","last_undegraded":"2024-09-10T19:06:36.412789+0000","last_fullsized":"2024-09-10T19:06:36.412789+0000","mapping_epoch":379,"log_start":"439'13760","ondisk_log_start":"439'13760","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3235,"log_dups_size":0,"ondisk_log_size":3235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9502720,"num_objects":145,"num_object_clones":0,"num_object_copies":435,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16995,"num_write_kb":548480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"70'81","reported_seq":408,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.165816+0000","last_change":"2024-09-10T19:04:24.971869+0000","last_active":"2024-09-10T19:06:30.165816+0000","last_peered":"2024-09-10T19:06:30.165816+0000","last_clean":"2024-09-10T19:06:30.165816+0000","last_became_active":"2024-09-10T18:55:40.392432+0000","last_became_peered":"2024-09-10T18:55:40.392432+0000","last_unstale":"2024-09-10T19:06:30.165816+0000","last_undegraded":"2024-09-10T19:06:30.165816+0000","last_fullsized":"2024-09-10T19:06:30.165816+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:04:24.971752+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:04:24.971752+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034804469999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"427'54","reported_seq":591,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:32.185354+0000","last_change":"2024-09-10T19:02:49.563856+0000","last_active":"2024-09-10T19:06:32.185354+0000","last_peered":"2024-09-10T19:06:32.185354+0000","last_clean":"2024-09-10T19:06:32.185354+0000","last_became_active":"2024-09-10T19:02:49.563402+0000","last_became_peered":"2024-09-10T19:02:49.563402+0000","last_unstale":"2024-09-10T19:06:32.185354+0000","last_undegraded":"2024-09-10T19:06:32.185354+0000","last_fullsized":"2024-09-10T19:06:32.185354+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'41","last_scrub_stamp":"2024-09-10T19:02:12.270610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:12.270610+0000","objects_scrubbed":10,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012827955,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":83,"num_read_kb":68,"num_write":50,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":483,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.166429+0000","last_change":"2024-09-10T19:06:18.033122+0000","last_active":"2024-09-10T19:06:30.166429+0000","last_peered":"2024-09-10T19:06:30.166429+0000","last_clean":"2024-09-10T19:06:30.166429+0000","last_became_active":"2024-09-10T18:45:17.964494+0000","last_became_peered":"2024-09-10T18:45:17.964494+0000","last_unstale":"2024-09-10T19:06:30.166429+0000","last_undegraded":"2024-09-10T19:06:30.166429+0000","last_fullsized":"2024-09-10T19:06:30.166429+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:18.033032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:12.982599+0000","last_clean_scrub_stamp":"2024-09-10T19:06:18.033032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:38.662714+0000","scrub_duration":0.0025216739999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":425,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.162654+0000","last_change":"2024-09-10T19:04:32.317459+0000","last_active":"2024-09-10T19:06:30.162654+0000","last_peered":"2024-09-10T19:06:30.162654+0000","last_clean":"2024-09-10T19:06:30.162654+0000","last_became_active":"2024-09-10T18:47:57.107478+0000","last_became_peered":"2024-09-10T18:47:57.107478+0000","last_unstale":"2024-09-10T19:06:30.162654+0000","last_undegraded":"2024-09-10T19:06:30.162654+0000","last_fullsized":"2024-09-10T19:06:30.162654+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:32.317339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:04:32.317339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0077591379999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":452,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.170859+0000","last_change":"2024-09-10T19:02:09.267386+0000","last_active":"2024-09-10T19:06:30.170859+0000","last_peered":"2024-09-10T19:06:30.170859+0000","last_clean":"2024-09-10T19:06:30.170859+0000","last_became_active":"2024-09-10T18:57:02.460594+0000","last_became_peered":"2024-09-10T18:57:02.460594+0000","last_unstale":"2024-09-10T19:06:30.170859+0000","last_undegraded":"2024-09-10T19:06:30.170859+0000","last_fullsized":"2024-09-10T19:06:30.170859+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:09.267253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:22.891084+0000","last_clean_scrub_stamp":"2024-09-10T19:02:09.267253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0067979290000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"449'12023","reported_seq":12110,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:36.396377+0000","last_change":"2024-09-10T19:04:36.990471+0000","last_active":"2024-09-10T19:06:36.396377+0000","last_peered":"2024-09-10T19:06:36.396377+0000","last_clean":"2024-09-10T19:06:36.396377+0000","last_became_active":"2024-09-10T19:04:36.984736+0000","last_became_peered":"2024-09-10T19:04:36.984736+0000","last_unstale":"2024-09-10T19:06:36.396377+0000","last_undegraded":"2024-09-10T19:06:36.396377+0000","last_fullsized":"2024-09-10T19:06:36.396377+0000","mapping_epoch":379,"log_start":"424'8741","ondisk_log_start":"424'8741","created":379,"last_epoch_clean":405,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3282,"log_dups_size":0,"ondisk_log_size":3282,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8290304,"num_objects":126,"num_object_clones":0,"num_object_copies":378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8514,"num_write_kb":276512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"70'81","reported_seq":286,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.166776+0000","last_change":"2024-09-10T19:06:16.693268+0000","last_active":"2024-09-10T19:06:30.166776+0000","last_peered":"2024-09-10T19:06:30.166776+0000","last_clean":"2024-09-10T19:06:30.166776+0000","last_became_active":"2024-09-10T18:57:02.437944+0000","last_became_peered":"2024-09-10T18:57:02.437944+0000","last_unstale":"2024-09-10T19:06:30.166776+0000","last_undegraded":"2024-09-10T19:06:30.166776+0000","last_fullsized":"2024-09-10T19:06:30.166776+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:06:16.692486+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:06:16.692486+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:31.831517+0000","scrub_duration":0.0077222740000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"427'26","reported_seq":360,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169523+0000","last_change":"2024-09-10T19:03:43.899377+0000","last_active":"2024-09-10T19:06:30.169523+0000","last_peered":"2024-09-10T19:06:30.169523+0000","last_clean":"2024-09-10T19:06:30.169523+0000","last_became_active":"2024-09-10T19:02:49.572746+0000","last_became_peered":"2024-09-10T19:02:49.572746+0000","last_unstale":"2024-09-10T19:06:30.169523+0000","last_undegraded":"2024-09-10T19:06:30.169523+0000","last_fullsized":"2024-09-10T19:06:30.169523+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'22","last_scrub_stamp":"2024-09-10T19:03:43.899294+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-10T18:47:10.514951+0000","last_clean_scrub_stamp":"2024-09-10T19:03:43.899294+0000","objects_scrubbed":5,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0049335940000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":41,"num_read_kb":30,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"84'6","reported_seq":492,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:34.928132+0000","last_change":"2024-09-10T19:03:43.134996+0000","last_active":"2024-09-10T19:06:34.928132+0000","last_peered":"2024-09-10T19:06:34.928132+0000","last_clean":"2024-09-10T19:06:34.928132+0000","last_became_active":"2024-09-10T19:02:49.570177+0000","last_became_peered":"2024-09-10T19:02:49.570177+0000","last_unstale":"2024-09-10T19:06:34.928132+0000","last_undegraded":"2024-09-10T19:06:34.928132+0000","last_fullsized":"2024-09-10T19:06:34.928132+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'6","last_scrub_stamp":"2024-09-10T19:03:43.134912+0000","last_deep_scrub":"84'6","last_deep_scrub_stamp":"2024-09-10T18:47:22.545450+0000","last_clean_scrub_stamp":"2024-09-10T19:03:43.134912+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024059210000000001,"objects_trimmed":0,"snaptrim_duration":0.0021933159999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":339,"num_read_kb":334,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":436,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.172637+0000","last_change":"2024-09-10T19:04:29.468182+0000","last_active":"2024-09-10T19:06:30.172637+0000","last_peered":"2024-09-10T19:06:30.172637+0000","last_clean":"2024-09-10T19:06:30.172637+0000","last_became_active":"2024-09-10T18:55:40.393389+0000","last_became_peered":"2024-09-10T18:55:40.393389+0000","last_unstale":"2024-09-10T19:06:30.172637+0000","last_undegraded":"2024-09-10T19:06:30.172637+0000","last_fullsized":"2024-09-10T19:06:30.172637+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:29.468090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:04:29.468090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0099267039999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":454,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.167837+0000","last_change":"2024-09-10T19:06:09.998326+0000","last_active":"2024-09-10T19:06:30.167837+0000","last_peered":"2024-09-10T19:06:30.167837+0000","last_clean":"2024-09-10T19:06:30.167837+0000","last_became_active":"2024-09-10T18:57:02.438945+0000","last_became_peered":"2024-09-10T18:57:02.438945+0000","last_unstale":"2024-09-10T19:06:30.167837+0000","last_undegraded":"2024-09-10T19:06:30.167837+0000","last_fullsized":"2024-09-10T19:06:30.167837+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:09.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:06:09.998251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:23.808265+0000","scrub_duration":0.0039357020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":211,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.164262+0000","last_change":"2024-09-10T19:03:42.107868+0000","last_active":"2024-09-10T19:06:30.164262+0000","last_peered":"2024-09-10T19:06:30.164262+0000","last_clean":"2024-09-10T19:06:30.164262+0000","last_became_active":"2024-09-10T18:57:02.453865+0000","last_became_peered":"2024-09-10T18:57:02.453865+0000","last_unstale":"2024-09-10T19:06:30.164262+0000","last_undegraded":"2024-09-10T19:06:30.164262+0000","last_fullsized":"2024-09-10T19:06:30.164262+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:42.107785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:07.150480+0000","last_clean_scrub_stamp":"2024-09-10T19:03:42.107785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029595450000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.9","version":"449'11713","reported_seq":11799,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:36.376933+0000","last_change":"2024-09-10T19:04:36.986402+0000","last_active":"2024-09-10T19:06:36.376933+0000","last_peered":"2024-09-10T19:06:36.376933+0000","last_clean":"2024-09-10T19:06:36.376933+0000","last_became_active":"2024-09-10T19:04:36.984334+0000","last_became_peered":"2024-09-10T19:04:36.984334+0000","last_unstale":"2024-09-10T19:06:36.376933+0000","last_undegraded":"2024-09-10T19:06:36.376933+0000","last_fullsized":"2024-09-10T19:06:36.376933+0000","mapping_epoch":379,"log_start":"424'8459","ondisk_log_start":"424'8459","created":379,"last_epoch_clean":405,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3254,"log_dups_size":0,"ondisk_log_size":3254,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4980736,"num_objects":76,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8302,"num_write_kb":268096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"70'81","reported_seq":164,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.162016+0000","last_change":"2024-09-10T19:04:28.278727+0000","last_active":"2024-09-10T19:06:30.162016+0000","last_peered":"2024-09-10T19:06:30.162016+0000","last_clean":"2024-09-10T19:06:30.162016+0000","last_became_active":"2024-09-10T19:02:49.578413+0000","last_became_peered":"2024-09-10T19:02:49.578413+0000","last_unstale":"2024-09-10T19:06:30.162016+0000","last_undegraded":"2024-09-10T19:06:30.162016+0000","last_fullsized":"2024-09-10T19:06:30.162016+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:04:28.278564+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:04:28.278564+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0066667519999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"427'36","reported_seq":214,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.165474+0000","last_change":"2024-09-10T19:03:40.879526+0000","last_active":"2024-09-10T19:06:30.165474+0000","last_peered":"2024-09-10T19:06:30.165474+0000","last_clean":"2024-09-10T19:06:30.165474+0000","last_became_active":"2024-09-10T19:00:42.516521+0000","last_became_peered":"2024-09-10T19:00:42.516521+0000","last_unstale":"2024-09-10T19:06:30.165474+0000","last_undegraded":"2024-09-10T19:06:30.165474+0000","last_fullsized":"2024-09-10T19:06:30.165474+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T19:03:40.879436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:03:40.879436+0000","objects_scrubbed":6,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0094657419999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":47,"num_write":36,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"305'2","reported_seq":443,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.173919+0000","last_change":"2024-09-10T19:03:21.921314+0000","last_active":"2024-09-10T19:06:30.173919+0000","last_peered":"2024-09-10T19:06:30.173919+0000","last_clean":"2024-09-10T19:06:30.173919+0000","last_became_active":"2024-09-10T18:55:40.390022+0000","last_became_peered":"2024-09-10T18:55:40.390022+0000","last_unstale":"2024-09-10T19:06:30.173919+0000","last_undegraded":"2024-09-10T19:06:30.173919+0000","last_fullsized":"2024-09-10T19:06:30.173919+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"305'2","last_scrub_stamp":"2024-09-10T19:03:21.921254+0000","last_deep_scrub":"305'2","last_deep_scrub_stamp":"2024-09-10T18:59:18.897059+0000","last_clean_scrub_stamp":"2024-09-10T19:03:21.921254+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021302700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":304,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.167217+0000","last_change":"2024-09-10T19:06:00.735401+0000","last_active":"2024-09-10T19:06:30.167217+0000","last_peered":"2024-09-10T19:06:30.167217+0000","last_clean":"2024-09-10T19:06:30.167217+0000","last_became_active":"2024-09-10T19:02:49.560325+0000","last_became_peered":"2024-09-10T19:02:49.560325+0000","last_unstale":"2024-09-10T19:06:30.167217+0000","last_undegraded":"2024-09-10T19:06:30.167217+0000","last_fullsized":"2024-09-10T19:06:30.167217+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:00.735322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:06:00.735322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:08.617305+0000","scrub_duration":0.0022871300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":130,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169893+0000","last_change":"2024-09-10T19:04:30.022221+0000","last_active":"2024-09-10T19:06:30.169893+0000","last_peered":"2024-09-10T19:06:30.169893+0000","last_clean":"2024-09-10T19:06:30.169893+0000","last_became_active":"2024-09-10T19:00:42.518566+0000","last_became_peered":"2024-09-10T19:00:42.518566+0000","last_unstale":"2024-09-10T19:06:30.169893+0000","last_undegraded":"2024-09-10T19:06:30.169893+0000","last_fullsized":"2024-09-10T19:06:30.169893+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:30.022167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:04:30.022167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001515976,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":89,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.167175+0000","last_change":"2024-09-10T19:02:49.573172+0000","last_active":"2024-09-10T19:06:30.167175+0000","last_peered":"2024-09-10T19:06:30.167175+0000","last_clean":"2024-09-10T19:06:30.167175+0000","last_became_active":"2024-09-10T19:02:49.572858+0000","last_became_peered":"2024-09-10T19:02:49.572858+0000","last_unstale":"2024-09-10T19:06:30.167175+0000","last_undegraded":"2024-09-10T19:06:30.167175+0000","last_fullsized":"2024-09-10T19:06:30.167175+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:20.272756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:02:20.272756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.a","version":"449'16874","reported_seq":16951,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:36.391285+0000","last_change":"2024-09-10T19:03:15.826075+0000","last_active":"2024-09-10T19:06:36.391285+0000","last_peered":"2024-09-10T19:06:36.391285+0000","last_clean":"2024-09-10T19:06:36.391285+0000","last_became_active":"2024-09-10T19:03:15.825784+0000","last_became_peered":"2024-09-10T19:03:15.825784+0000","last_unstale":"2024-09-10T19:06:36.391285+0000","last_undegraded":"2024-09-10T19:06:36.391285+0000","last_fullsized":"2024-09-10T19:06:36.391285+0000","mapping_epoch":379,"log_start":"439'13563","ondisk_log_start":"439'13563","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3311,"log_dups_size":0,"ondisk_log_size":3311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10878976,"num_objects":166,"num_object_clones":0,"num_object_copies":498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16874,"num_write_kb":545280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"427'45","reported_seq":583,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.164389+0000","last_change":"2024-09-10T19:06:01.618690+0000","last_active":"2024-09-10T19:06:30.164389+0000","last_peered":"2024-09-10T19:06:30.164389+0000","last_clean":"2024-09-10T19:06:30.164389+0000","last_became_active":"2024-09-10T18:57:02.435499+0000","last_became_peered":"2024-09-10T18:57:02.435499+0000","last_unstale":"2024-09-10T19:06:30.164389+0000","last_undegraded":"2024-09-10T19:06:30.164389+0000","last_fullsized":"2024-09-10T19:06:30.164389+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'45","last_scrub_stamp":"2024-09-10T19:06:01.618608+0000","last_deep_scrub":"232'33","last_deep_scrub_stamp":"2024-09-10T18:56:21.206960+0000","last_clean_scrub_stamp":"2024-09-10T19:06:01.618608+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:06.102994+0000","scrub_duration":0.008985346,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":61,"num_read_kb":51,"num_write":37,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"60'2","reported_seq":471,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.170995+0000","last_change":"2024-09-10T19:03:22.545399+0000","last_active":"2024-09-10T19:06:30.170995+0000","last_peered":"2024-09-10T19:06:30.170995+0000","last_clean":"2024-09-10T19:06:30.170995+0000","last_became_active":"2024-09-10T18:45:17.961660+0000","last_became_peered":"2024-09-10T18:45:17.961660+0000","last_unstale":"2024-09-10T19:06:30.170995+0000","last_undegraded":"2024-09-10T19:06:30.170995+0000","last_fullsized":"2024-09-10T19:06:30.170995+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-10T19:03:22.545342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:03:22.545342+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001448983,"objects_trimmed":0,"snaptrim_duration":0.0013942640000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":500,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.167234+0000","last_change":"2024-09-10T19:04:36.102267+0000","last_active":"2024-09-10T19:06:30.167234+0000","last_peered":"2024-09-10T19:06:30.167234+0000","last_clean":"2024-09-10T19:06:30.167234+0000","last_became_active":"2024-09-10T18:57:02.445712+0000","last_became_peered":"2024-09-10T18:57:02.445712+0000","last_unstale":"2024-09-10T19:06:30.167234+0000","last_undegraded":"2024-09-10T19:06:30.167234+0000","last_fullsized":"2024-09-10T19:06:30.167234+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:36.102203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:04:36.102203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.095986274999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'81","reported_seq":286,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.165443+0000","last_change":"2024-09-10T19:06:24.713689+0000","last_active":"2024-09-10T19:06:30.165443+0000","last_peered":"2024-09-10T19:06:30.165443+0000","last_clean":"2024-09-10T19:06:30.165443+0000","last_became_active":"2024-09-10T18:57:02.440063+0000","last_became_peered":"2024-09-10T18:57:02.440063+0000","last_unstale":"2024-09-10T19:06:30.165443+0000","last_undegraded":"2024-09-10T19:06:30.165443+0000","last_fullsized":"2024-09-10T19:06:30.165443+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:06:24.713626+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:56.085580+0000","last_clean_scrub_stamp":"2024-09-10T19:06:24.713626+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:40.045571+0000","scrub_duration":0.0013738730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":469,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.168850+0000","last_change":"2024-09-10T19:04:33.972472+0000","last_active":"2024-09-10T19:06:30.168850+0000","last_peered":"2024-09-10T19:06:30.168850+0000","last_clean":"2024-09-10T19:06:30.168850+0000","last_became_active":"2024-09-10T18:57:02.431556+0000","last_became_peered":"2024-09-10T18:57:02.431556+0000","last_unstale":"2024-09-10T19:06:30.168850+0000","last_undegraded":"2024-09-10T19:06:30.168850+0000","last_fullsized":"2024-09-10T19:06:30.168850+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:33.972423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:04:33.972423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0074396560000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":474,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169370+0000","last_change":"2024-09-10T19:03:51.037606+0000","last_active":"2024-09-10T19:06:30.169370+0000","last_peered":"2024-09-10T19:06:30.169370+0000","last_clean":"2024-09-10T19:06:30.169370+0000","last_became_active":"2024-09-10T18:45:24.073327+0000","last_became_peered":"2024-09-10T18:45:24.073327+0000","last_unstale":"2024-09-10T19:06:30.169370+0000","last_undegraded":"2024-09-10T19:06:30.169370+0000","last_fullsized":"2024-09-10T19:06:30.169370+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:51.037432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:03:51.037432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0085107059999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.5","version":"449'11846","reported_seq":11935,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:36.641841+0000","last_change":"2024-09-10T19:04:36.975636+0000","last_active":"2024-09-10T19:06:36.641841+0000","last_peered":"2024-09-10T19:06:36.641841+0000","last_clean":"2024-09-10T19:06:36.641841+0000","last_became_active":"2024-09-10T19:04:36.973383+0000","last_became_peered":"2024-09-10T19:04:36.973383+0000","last_unstale":"2024-09-10T19:06:36.641841+0000","last_undegraded":"2024-09-10T19:06:36.641841+0000","last_fullsized":"2024-09-10T19:06:36.641841+0000","mapping_epoch":379,"log_start":"424'8507","ondisk_log_start":"424'8507","created":379,"last_epoch_clean":405,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3339,"log_dups_size":0,"ondisk_log_size":3339,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8945664,"num_objects":136,"num_object_clones":0,"num_object_copies":408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8384,"num_write_kb":272672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.f","version":"70'81","reported_seq":479,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.162535+0000","last_change":"2024-09-10T19:06:21.520670+0000","last_active":"2024-09-10T19:06:30.162535+0000","last_peered":"2024-09-10T19:06:30.162535+0000","last_clean":"2024-09-10T19:06:30.162535+0000","last_became_active":"2024-09-10T18:52:16.598295+0000","last_became_peered":"2024-09-10T18:52:16.598295+0000","last_unstale":"2024-09-10T19:06:30.162535+0000","last_undegraded":"2024-09-10T19:06:30.162535+0000","last_fullsized":"2024-09-10T19:06:30.162535+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:06:21.520615+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T19:06:21.520615+0000","last_clean_scrub_stamp":"2024-09-10T19:06:21.520615+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:33.819484+0000","scrub_duration":0.0011289640000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"427'51","reported_seq":591,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.163916+0000","last_change":"2024-09-10T19:03:46.180416+0000","last_active":"2024-09-10T19:06:30.163916+0000","last_peered":"2024-09-10T19:06:30.163916+0000","last_clean":"2024-09-10T19:06:30.163916+0000","last_became_active":"2024-09-10T18:57:02.436059+0000","last_became_peered":"2024-09-10T18:57:02.436059+0000","last_unstale":"2024-09-10T19:06:30.163916+0000","last_undegraded":"2024-09-10T19:06:30.163916+0000","last_fullsized":"2024-09-10T19:06:30.163916+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'35","last_scrub_stamp":"2024-09-10T19:03:46.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:03:46.179242+0000","objects_scrubbed":9,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0052365550000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":80,"num_read_kb":69,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":448,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.168930+0000","last_change":"2024-09-10T19:06:02.034786+0000","last_active":"2024-09-10T19:06:30.168930+0000","last_peered":"2024-09-10T19:06:30.168930+0000","last_clean":"2024-09-10T19:06:30.168930+0000","last_became_active":"2024-09-10T18:55:40.391849+0000","last_became_peered":"2024-09-10T18:55:40.391849+0000","last_unstale":"2024-09-10T19:06:30.168930+0000","last_undegraded":"2024-09-10T19:06:30.168930+0000","last_fullsized":"2024-09-10T19:06:30.168930+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:02.034673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:06:02.034673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:21.764260+0000","scrub_duration":0.0021020129999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":419,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.167437+0000","last_change":"2024-09-10T19:03:48.853831+0000","last_active":"2024-09-10T19:06:30.167437+0000","last_peered":"2024-09-10T19:06:30.167437+0000","last_clean":"2024-09-10T19:06:30.167437+0000","last_became_active":"2024-09-10T19:00:42.514709+0000","last_became_peered":"2024-09-10T19:00:42.514709+0000","last_unstale":"2024-09-10T19:06:30.167437+0000","last_undegraded":"2024-09-10T19:06:30.167437+0000","last_fullsized":"2024-09-10T19:06:30.167437+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:48.853752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:03:48.853752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0059332969999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":255,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.168243+0000","last_change":"2024-09-10T19:02:58.491396+0000","last_active":"2024-09-10T19:06:30.168243+0000","last_peered":"2024-09-10T19:06:30.168243+0000","last_clean":"2024-09-10T19:06:30.168243+0000","last_became_active":"2024-09-10T18:57:02.468866+0000","last_became_peered":"2024-09-10T18:57:02.468866+0000","last_unstale":"2024-09-10T19:06:30.168243+0000","last_undegraded":"2024-09-10T19:06:30.168243+0000","last_fullsized":"2024-09-10T19:06:30.168243+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:58.491326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:02:58.491326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017883440000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"449'12182","reported_seq":12270,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:36.633081+0000","last_change":"2024-09-10T19:04:36.977141+0000","last_active":"2024-09-10T19:06:36.633081+0000","last_peered":"2024-09-10T19:06:36.633081+0000","last_clean":"2024-09-10T19:06:36.633081+0000","last_became_active":"2024-09-10T19:04:36.975442+0000","last_became_peered":"2024-09-10T19:04:36.975442+0000","last_unstale":"2024-09-10T19:06:36.633081+0000","last_undegraded":"2024-09-10T19:06:36.633081+0000","last_fullsized":"2024-09-10T19:06:36.633081+0000","mapping_epoch":379,"log_start":"427'8989","ondisk_log_start":"427'8989","created":379,"last_epoch_clean":405,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3193,"log_dups_size":0,"ondisk_log_size":3193,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8644,"num_write_kb":279168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,3],"acting":[6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.e","version":"70'81","reported_seq":337,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.168427+0000","last_change":"2024-09-10T19:04:31.431314+0000","last_active":"2024-09-10T19:06:30.168427+0000","last_peered":"2024-09-10T19:06:30.168427+0000","last_clean":"2024-09-10T19:06:30.168427+0000","last_became_active":"2024-09-10T18:57:02.449115+0000","last_became_peered":"2024-09-10T18:57:02.449115+0000","last_unstale":"2024-09-10T19:06:30.168427+0000","last_undegraded":"2024-09-10T19:06:30.168427+0000","last_fullsized":"2024-09-10T19:06:30.168427+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:04:31.431216+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:04:31.431216+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011858260000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"427'25","reported_seq":484,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.168060+0000","last_change":"2024-09-10T19:03:43.850751+0000","last_active":"2024-09-10T19:06:30.168060+0000","last_peered":"2024-09-10T19:06:30.168060+0000","last_clean":"2024-09-10T19:06:30.168060+0000","last_became_active":"2024-09-10T18:55:40.395257+0000","last_became_peered":"2024-09-10T18:55:40.395257+0000","last_unstale":"2024-09-10T19:06:30.168060+0000","last_undegraded":"2024-09-10T19:06:30.168060+0000","last_fullsized":"2024-09-10T19:06:30.168060+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T19:03:43.850670+0000","last_deep_scrub":"130'13","last_deep_scrub_stamp":"2024-09-10T18:51:58.427534+0000","last_clean_scrub_stamp":"2024-09-10T19:03:43.850670+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026924499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":99,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169651+0000","last_change":"2024-09-10T19:03:51.598347+0000","last_active":"2024-09-10T19:06:30.169651+0000","last_peered":"2024-09-10T19:06:30.169651+0000","last_clean":"2024-09-10T19:06:30.169651+0000","last_became_active":"2024-09-10T19:02:49.581709+0000","last_became_peered":"2024-09-10T19:02:49.581709+0000","last_unstale":"2024-09-10T19:06:30.169651+0000","last_undegraded":"2024-09-10T19:06:30.169651+0000","last_fullsized":"2024-09-10T19:06:30.169651+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:51.598266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:03:51.598266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047300789999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":429,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169771+0000","last_change":"2024-09-10T19:02:51.031764+0000","last_active":"2024-09-10T19:06:30.169771+0000","last_peered":"2024-09-10T19:06:30.169771+0000","last_clean":"2024-09-10T19:06:30.169771+0000","last_became_active":"2024-09-10T18:55:40.390771+0000","last_became_peered":"2024-09-10T18:55:40.390771+0000","last_unstale":"2024-09-10T19:06:30.169771+0000","last_undegraded":"2024-09-10T19:06:30.169771+0000","last_fullsized":"2024-09-10T19:06:30.169771+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:51.031711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_clean_scrub_stamp":"2024-09-10T19:02:51.031711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015681510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":463,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.173527+0000","last_change":"2024-09-10T19:04:28.767375+0000","last_active":"2024-09-10T19:06:30.173527+0000","last_peered":"2024-09-10T19:06:30.173527+0000","last_clean":"2024-09-10T19:06:30.173527+0000","last_became_active":"2024-09-10T18:57:02.452387+0000","last_became_peered":"2024-09-10T18:57:02.452387+0000","last_unstale":"2024-09-10T19:06:30.173527+0000","last_undegraded":"2024-09-10T19:06:30.173527+0000","last_fullsized":"2024-09-10T19:06:30.173527+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:28.766947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:53.345184+0000","last_clean_scrub_stamp":"2024-09-10T19:04:28.766947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0079684749999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"449'11962","reported_seq":12051,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:36.676230+0000","last_change":"2024-09-10T19:04:36.980107+0000","last_active":"2024-09-10T19:06:36.676230+0000","last_peered":"2024-09-10T19:06:36.676230+0000","last_clean":"2024-09-10T19:06:36.676230+0000","last_became_active":"2024-09-10T19:04:36.977753+0000","last_became_peered":"2024-09-10T19:04:36.977753+0000","last_unstale":"2024-09-10T19:06:36.676230+0000","last_undegraded":"2024-09-10T19:06:36.676230+0000","last_fullsized":"2024-09-10T19:06:36.676230+0000","mapping_epoch":379,"log_start":"424'8580","ondisk_log_start":"424'8580","created":379,"last_epoch_clean":405,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3382,"log_dups_size":0,"ondisk_log_size":3382,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4685824,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8499,"num_write_kb":274272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.d","version":"70'81","reported_seq":268,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169759+0000","last_change":"2024-09-10T19:02:01.044879+0000","last_active":"2024-09-10T19:06:30.169759+0000","last_peered":"2024-09-10T19:06:30.169759+0000","last_clean":"2024-09-10T19:06:30.169759+0000","last_became_active":"2024-09-10T18:57:02.439652+0000","last_became_peered":"2024-09-10T18:57:02.439652+0000","last_unstale":"2024-09-10T19:06:30.169759+0000","last_undegraded":"2024-09-10T19:06:30.169759+0000","last_fullsized":"2024-09-10T19:06:30.169759+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:01.044143+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:02:01.044143+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014088588000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"427'46","reported_seq":171,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.165750+0000","last_change":"2024-09-10T19:02:18.712390+0000","last_active":"2024-09-10T19:06:30.165750+0000","last_peered":"2024-09-10T19:06:30.165750+0000","last_clean":"2024-09-10T19:06:30.165750+0000","last_became_active":"2024-09-10T19:00:42.515903+0000","last_became_peered":"2024-09-10T19:00:42.515903+0000","last_unstale":"2024-09-10T19:06:30.165750+0000","last_undegraded":"2024-09-10T19:06:30.165750+0000","last_fullsized":"2024-09-10T19:06:30.165750+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"365'34","last_scrub_stamp":"2024-09-10T19:02:18.712306+0000","last_deep_scrub":"365'34","last_deep_scrub_stamp":"2024-09-10T19:02:18.712306+0000","last_clean_scrub_stamp":"2024-09-10T19:02:18.712306+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.017099106999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":60,"num_read_kb":51,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":429,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.170992+0000","last_change":"2024-09-10T19:06:23.202241+0000","last_active":"2024-09-10T19:06:30.170992+0000","last_peered":"2024-09-10T19:06:30.170992+0000","last_clean":"2024-09-10T19:06:30.170992+0000","last_became_active":"2024-09-10T18:57:02.461708+0000","last_became_peered":"2024-09-10T18:57:02.461708+0000","last_unstale":"2024-09-10T19:06:30.170992+0000","last_undegraded":"2024-09-10T19:06:30.170992+0000","last_fullsized":"2024-09-10T19:06:30.170992+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:23.202164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:05.603125+0000","last_clean_scrub_stamp":"2024-09-10T19:06:23.202164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:30.512580+0000","scrub_duration":0.012238550000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"273'10","reported_seq":408,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:32.178585+0000","last_change":"2024-09-10T19:06:28.588050+0000","last_active":"2024-09-10T19:06:32.178585+0000","last_peered":"2024-09-10T19:06:32.178585+0000","last_clean":"2024-09-10T19:06:32.178585+0000","last_became_active":"2024-09-10T18:57:02.441890+0000","last_became_peered":"2024-09-10T18:57:02.441890+0000","last_unstale":"2024-09-10T19:06:32.178585+0000","last_undegraded":"2024-09-10T19:06:32.178585+0000","last_fullsized":"2024-09-10T19:06:32.178585+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'10","last_scrub_stamp":"2024-09-10T19:06:28.587892+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-09-10T18:47:02.582229+0000","last_clean_scrub_stamp":"2024-09-10T19:06:28.587892+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:35.607577+0000","scrub_duration":0.0043958399999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":481,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.166966+0000","last_change":"2024-09-10T19:03:32.808204+0000","last_active":"2024-09-10T19:06:30.166966+0000","last_peered":"2024-09-10T19:06:30.166966+0000","last_clean":"2024-09-10T19:06:30.166966+0000","last_became_active":"2024-09-10T18:55:40.389202+0000","last_became_peered":"2024-09-10T18:55:40.389202+0000","last_unstale":"2024-09-10T19:06:30.166966+0000","last_undegraded":"2024-09-10T19:06:30.166966+0000","last_fullsized":"2024-09-10T19:06:30.166966+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:32.808130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_clean_scrub_stamp":"2024-09-10T19:03:32.808130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003767603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"449'12131","reported_seq":12245,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:33.970064+0000","last_change":"2024-09-10T19:06:32.570307+0000","last_active":"2024-09-10T19:06:33.970064+0000","last_peered":"2024-09-10T19:06:33.970064+0000","last_clean":"2024-09-10T19:06:33.970064+0000","last_became_active":"2024-09-10T19:04:36.963484+0000","last_became_peered":"2024-09-10T19:04:36.963484+0000","last_unstale":"2024-09-10T19:06:33.970064+0000","last_undegraded":"2024-09-10T19:06:33.970064+0000","last_fullsized":"2024-09-10T19:06:33.970064+0000","mapping_epoch":379,"log_start":"423'8490","ondisk_log_start":"423'8490","created":379,"last_epoch_clean":405,"parent":"0.0","parent_split_bits":0,"last_scrub":"449'12055","last_scrub_stamp":"2024-09-10T19:06:32.570104+0000","last_deep_scrub":"449'12055","last_deep_scrub_stamp":"2024-09-10T19:06:32.570104+0000","last_clean_scrub_stamp":"2024-09-10T19:06:32.570104+0000","objects_scrubbed":321,"log_size":3641,"log_dups_size":0,"ondisk_log_size":3641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:40.160089+0000","scrub_duration":1.91816849,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13762560,"num_objects":210,"num_object_clones":0,"num_object_copies":630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.c","version":"70'81","reported_seq":412,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169812+0000","last_change":"2024-09-10T19:02:54.540617+0000","last_active":"2024-09-10T19:06:30.169812+0000","last_peered":"2024-09-10T19:06:30.169812+0000","last_clean":"2024-09-10T19:06:30.169812+0000","last_became_active":"2024-09-10T18:55:40.390655+0000","last_became_peered":"2024-09-10T18:55:40.390655+0000","last_unstale":"2024-09-10T19:06:30.169812+0000","last_undegraded":"2024-09-10T19:06:30.169812+0000","last_fullsized":"2024-09-10T19:06:30.169812+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:54.540558+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:56:33.779154+0000","last_clean_scrub_stamp":"2024-09-10T19:02:54.540558+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019874599999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"427'24","reported_seq":510,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.168570+0000","last_change":"2024-09-10T19:06:12.856434+0000","last_active":"2024-09-10T19:06:30.168570+0000","last_peered":"2024-09-10T19:06:30.168570+0000","last_clean":"2024-09-10T19:06:30.168570+0000","last_became_active":"2024-09-10T18:45:20.056869+0000","last_became_peered":"2024-09-10T18:45:20.056869+0000","last_unstale":"2024-09-10T19:06:30.168570+0000","last_undegraded":"2024-09-10T19:06:30.168570+0000","last_fullsized":"2024-09-10T19:06:30.168570+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'24","last_scrub_stamp":"2024-09-10T19:06:12.856360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:06:12.856360+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:23.852248+0000","scrub_duration":0.0042329550000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"56'2","reported_seq":511,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.171473+0000","last_change":"2024-09-10T19:06:14.852777+0000","last_active":"2024-09-10T19:06:30.171473+0000","last_peered":"2024-09-10T19:06:30.171473+0000","last_clean":"2024-09-10T19:06:30.171473+0000","last_became_active":"2024-09-10T18:57:02.434475+0000","last_became_peered":"2024-09-10T18:57:02.434475+0000","last_unstale":"2024-09-10T19:06:30.171473+0000","last_undegraded":"2024-09-10T19:06:30.171473+0000","last_fullsized":"2024-09-10T19:06:30.171473+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-10T19:06:14.852698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:06:14.852698+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:43.077218+0000","scrub_duration":0.006938662,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":429,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:31.634809+0000","last_change":"2024-09-10T19:06:31.634746+0000","last_active":"2024-09-10T19:06:31.634809+0000","last_peered":"2024-09-10T19:06:31.634809+0000","last_clean":"2024-09-10T19:06:31.634809+0000","last_became_active":"2024-09-10T19:00:42.513796+0000","last_became_peered":"2024-09-10T19:00:42.513796+0000","last_unstale":"2024-09-10T19:06:31.634809+0000","last_undegraded":"2024-09-10T19:06:31.634809+0000","last_fullsized":"2024-09-10T19:06:31.634809+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:31.634653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:02.711594+0000","last_clean_scrub_stamp":"2024-09-10T19:06:31.634653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:33.241478+0000","scrub_duration":0.0052320470000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":462,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.174042+0000","last_change":"2024-09-10T19:03:46.711248+0000","last_active":"2024-09-10T19:06:30.174042+0000","last_peered":"2024-09-10T19:06:30.174042+0000","last_clean":"2024-09-10T19:06:30.174042+0000","last_became_active":"2024-09-10T18:57:02.459985+0000","last_became_peered":"2024-09-10T18:57:02.459985+0000","last_unstale":"2024-09-10T19:06:30.174042+0000","last_undegraded":"2024-09-10T19:06:30.174042+0000","last_fullsized":"2024-09-10T19:06:30.174042+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:46.710933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:46.710933+0000","last_clean_scrub_stamp":"2024-09-10T19:03:46.710933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0061568430000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.19","version":"449'11590","reported_seq":18223,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:33.969638+0000","last_change":"2024-09-10T19:05:38.613447+0000","last_active":"2024-09-10T19:06:33.969638+0000","last_peered":"2024-09-10T19:06:33.969638+0000","last_clean":"2024-09-10T19:06:33.969638+0000","last_became_active":"2024-09-10T19:05:38.577625+0000","last_became_peered":"2024-09-10T19:05:38.577625+0000","last_unstale":"2024-09-10T19:06:33.969638+0000","last_undegraded":"2024-09-10T19:06:33.969638+0000","last_fullsized":"2024-09-10T19:06:33.969638+0000","mapping_epoch":432,"log_start":"421'7995","ondisk_log_start":"421'7995","created":404,"last_epoch_clean":433,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3595,"log_dups_size":0,"ondisk_log_size":3595,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12910592,"num_objects":197,"num_object_clones":0,"num_object_copies":591,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8179,"num_write_kb":268032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3260,"num_bytes_recovered":213647360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.13","version":"70'81","reported_seq":168,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.174243+0000","last_change":"2024-09-10T19:04:36.470253+0000","last_active":"2024-09-10T19:06:30.174243+0000","last_peered":"2024-09-10T19:06:30.174243+0000","last_clean":"2024-09-10T19:06:30.174243+0000","last_became_active":"2024-09-10T19:02:49.580656+0000","last_became_peered":"2024-09-10T19:02:49.580656+0000","last_unstale":"2024-09-10T19:06:30.174243+0000","last_undegraded":"2024-09-10T19:06:30.174243+0000","last_fullsized":"2024-09-10T19:06:30.174243+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:04:36.470196+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:04:36.470196+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001517404,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"427'25","reported_seq":507,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.171678+0000","last_change":"2024-09-10T19:04:35.830252+0000","last_active":"2024-09-10T19:06:30.171678+0000","last_peered":"2024-09-10T19:06:30.171678+0000","last_clean":"2024-09-10T19:06:30.171678+0000","last_became_active":"2024-09-10T18:57:02.453001+0000","last_became_peered":"2024-09-10T18:57:02.453001+0000","last_unstale":"2024-09-10T19:06:30.171678+0000","last_undegraded":"2024-09-10T19:06:30.171678+0000","last_fullsized":"2024-09-10T19:06:30.171678+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T19:04:35.830196+0000","last_deep_scrub":"76'13","last_deep_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_clean_scrub_stamp":"2024-09-10T19:04:35.830196+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001824537,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":302,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.165524+0000","last_change":"2024-09-10T19:03:35.858172+0000","last_active":"2024-09-10T19:06:30.165524+0000","last_peered":"2024-09-10T19:06:30.165524+0000","last_clean":"2024-09-10T19:06:30.165524+0000","last_became_active":"2024-09-10T19:02:49.580701+0000","last_became_peered":"2024-09-10T19:02:49.580701+0000","last_unstale":"2024-09-10T19:06:30.165524+0000","last_undegraded":"2024-09-10T19:06:30.165524+0000","last_fullsized":"2024-09-10T19:06:30.165524+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:35.858086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:03:35.858086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0061699850000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":474,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169146+0000","last_change":"2024-09-10T19:06:30.141526+0000","last_active":"2024-09-10T19:06:30.169146+0000","last_peered":"2024-09-10T19:06:30.169146+0000","last_clean":"2024-09-10T19:06:30.169146+0000","last_became_active":"2024-09-10T18:57:02.455341+0000","last_became_peered":"2024-09-10T18:57:02.455341+0000","last_unstale":"2024-09-10T19:06:30.169146+0000","last_undegraded":"2024-09-10T19:06:30.169146+0000","last_fullsized":"2024-09-10T19:06:30.169146+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:30.141437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:00.754912+0000","last_clean_scrub_stamp":"2024-09-10T19:06:30.141437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:33.141155+0000","scrub_duration":0.0090402390000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"364'2","reported_seq":488,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169498+0000","last_change":"2024-09-10T19:02:15.059462+0000","last_active":"2024-09-10T19:06:30.169498+0000","last_peered":"2024-09-10T19:06:30.169498+0000","last_clean":"2024-09-10T19:06:30.169498+0000","last_became_active":"2024-09-10T18:59:48.741275+0000","last_became_peered":"2024-09-10T18:59:48.741275+0000","last_unstale":"2024-09-10T19:06:30.169498+0000","last_undegraded":"2024-09-10T19:06:30.169498+0000","last_fullsized":"2024-09-10T19:06:30.169498+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"364'2","last_scrub_stamp":"2024-09-10T19:02:15.059320+0000","last_deep_scrub":"364'2","last_deep_scrub_stamp":"2024-09-10T19:02:15.059320+0000","last_clean_scrub_stamp":"2024-09-10T19:02:15.059320+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.020873968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":52,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.18","version":"449'12074","reported_seq":20302,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:36.504642+0000","last_change":"2024-09-10T19:05:04.827272+0000","last_active":"2024-09-10T19:06:36.504642+0000","last_peered":"2024-09-10T19:06:36.504642+0000","last_clean":"2024-09-10T19:06:36.504642+0000","last_became_active":"2024-09-10T19:05:04.805312+0000","last_became_peered":"2024-09-10T19:05:04.805312+0000","last_unstale":"2024-09-10T19:06:36.504642+0000","last_undegraded":"2024-09-10T19:06:36.504642+0000","last_fullsized":"2024-09-10T19:06:36.504642+0000","mapping_epoch":417,"log_start":"423'8437","ondisk_log_start":"423'8437","created":404,"last_epoch_clean":418,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3637,"log_dups_size":0,"ondisk_log_size":3637,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1146880,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8566,"num_write_kb":274656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4058,"num_bytes_recovered":265945088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.12","version":"70'81","reported_seq":338,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.166774+0000","last_change":"2024-09-10T19:06:08.005069+0000","last_active":"2024-09-10T19:06:30.166774+0000","last_peered":"2024-09-10T19:06:30.166774+0000","last_clean":"2024-09-10T19:06:30.166774+0000","last_became_active":"2024-09-10T18:52:16.598921+0000","last_became_peered":"2024-09-10T18:52:16.598921+0000","last_unstale":"2024-09-10T19:06:30.166774+0000","last_undegraded":"2024-09-10T19:06:30.166774+0000","last_fullsized":"2024-09-10T19:06:30.166774+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:06:08.005010+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:54.127476+0000","last_clean_scrub_stamp":"2024-09-10T19:06:08.005010+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:33.753507+0000","scrub_duration":0.0036749040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":507,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.164109+0000","last_change":"2024-09-10T19:06:08.468915+0000","last_active":"2024-09-10T19:06:30.164109+0000","last_peered":"2024-09-10T19:06:30.164109+0000","last_clean":"2024-09-10T19:06:30.164109+0000","last_became_active":"2024-09-10T18:57:02.442251+0000","last_became_peered":"2024-09-10T18:57:02.442251+0000","last_unstale":"2024-09-10T19:06:30.164109+0000","last_undegraded":"2024-09-10T19:06:30.164109+0000","last_fullsized":"2024-09-10T19:06:30.164109+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:08.468838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:06:08.468838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:18.444598+0000","scrub_duration":0.0086820209999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"427'18","reported_seq":504,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.170652+0000","last_change":"2024-09-10T19:03:50.702400+0000","last_active":"2024-09-10T19:06:30.170652+0000","last_peered":"2024-09-10T19:06:30.170652+0000","last_clean":"2024-09-10T19:06:30.170652+0000","last_became_active":"2024-09-10T18:57:02.460430+0000","last_became_peered":"2024-09-10T18:57:02.460430+0000","last_unstale":"2024-09-10T19:06:30.170652+0000","last_undegraded":"2024-09-10T19:06:30.170652+0000","last_fullsized":"2024-09-10T19:06:30.170652+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'12","last_scrub_stamp":"2024-09-10T19:03:50.702317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:03:50.702317+0000","objects_scrubbed":3,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0055835499999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":24,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":466,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.168962+0000","last_change":"2024-09-10T19:06:30.117244+0000","last_active":"2024-09-10T19:06:30.168962+0000","last_peered":"2024-09-10T19:06:30.168962+0000","last_clean":"2024-09-10T19:06:30.168962+0000","last_became_active":"2024-09-10T18:45:22.069653+0000","last_became_peered":"2024-09-10T18:45:22.069653+0000","last_unstale":"2024-09-10T19:06:30.168962+0000","last_undegraded":"2024-09-10T19:06:30.168962+0000","last_fullsized":"2024-09-10T19:06:30.168962+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:30.117162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:06:30.117162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:36.601291+0000","scrub_duration":0.022352659,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":231,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.164513+0000","last_change":"2024-09-10T19:06:17.445445+0000","last_active":"2024-09-10T19:06:30.164513+0000","last_peered":"2024-09-10T19:06:30.164513+0000","last_clean":"2024-09-10T19:06:30.164513+0000","last_became_active":"2024-09-10T19:00:42.519492+0000","last_became_peered":"2024-09-10T19:00:42.519492+0000","last_unstale":"2024-09-10T19:06:30.164513+0000","last_undegraded":"2024-09-10T19:06:30.164513+0000","last_fullsized":"2024-09-10T19:06:30.164513+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:17.445379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:06:17.445379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:24.930408+0000","scrub_duration":0.002512116,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.11","version":"70'81","reported_seq":373,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.173649+0000","last_change":"2024-09-10T19:04:33.761605+0000","last_active":"2024-09-10T19:06:30.173649+0000","last_peered":"2024-09-10T19:06:30.173649+0000","last_clean":"2024-09-10T19:06:30.173649+0000","last_became_active":"2024-09-10T19:00:42.517311+0000","last_became_peered":"2024-09-10T19:00:42.517311+0000","last_unstale":"2024-09-10T19:06:30.173649+0000","last_undegraded":"2024-09-10T19:06:30.173649+0000","last_fullsized":"2024-09-10T19:06:30.173649+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:04:33.761515+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:04:33.761515+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0042906769999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"427'30","reported_seq":567,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169431+0000","last_change":"2024-09-10T19:04:33.464299+0000","last_active":"2024-09-10T19:06:30.169431+0000","last_peered":"2024-09-10T19:06:30.169431+0000","last_clean":"2024-09-10T19:06:30.169431+0000","last_became_active":"2024-09-10T18:57:02.444437+0000","last_became_peered":"2024-09-10T18:57:02.444437+0000","last_unstale":"2024-09-10T19:06:30.169431+0000","last_undegraded":"2024-09-10T19:06:30.169431+0000","last_fullsized":"2024-09-10T19:06:30.169431+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'20","last_scrub_stamp":"2024-09-10T19:04:33.464197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:04:33.464197+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040076549999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":480,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.166495+0000","last_change":"2024-09-10T19:02:20.755424+0000","last_active":"2024-09-10T19:06:30.166495+0000","last_peered":"2024-09-10T19:06:30.166495+0000","last_clean":"2024-09-10T19:06:30.166495+0000","last_became_active":"2024-09-10T18:57:02.445710+0000","last_became_peered":"2024-09-10T18:57:02.445710+0000","last_unstale":"2024-09-10T19:06:30.166495+0000","last_undegraded":"2024-09-10T19:06:30.166495+0000","last_fullsized":"2024-09-10T19:06:30.166495+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:20.755253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:20.755253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023404490000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"304'11","reported_seq":764,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:32.181433+0000","last_change":"2024-09-10T19:03:34.811295+0000","last_active":"2024-09-10T19:06:32.181433+0000","last_peered":"2024-09-10T19:06:32.181433+0000","last_clean":"2024-09-10T19:06:32.181433+0000","last_became_active":"2024-09-10T18:58:58.489370+0000","last_became_peered":"2024-09-10T18:58:58.489370+0000","last_unstale":"2024-09-10T19:06:32.181433+0000","last_undegraded":"2024-09-10T19:06:32.181433+0000","last_fullsized":"2024-09-10T19:06:32.181433+0000","mapping_epoch":303,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":304,"parent":"0.0","parent_split_bits":0,"last_scrub":"304'11","last_scrub_stamp":"2024-09-10T19:03:34.811194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:03:34.811194+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024856539999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":158,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.165260+0000","last_change":"2024-09-10T19:06:06.714131+0000","last_active":"2024-09-10T19:06:30.165260+0000","last_peered":"2024-09-10T19:06:30.165260+0000","last_clean":"2024-09-10T19:06:30.165260+0000","last_became_active":"2024-09-10T18:59:48.744923+0000","last_became_peered":"2024-09-10T18:59:48.744923+0000","last_unstale":"2024-09-10T19:06:30.165260+0000","last_undegraded":"2024-09-10T19:06:30.165260+0000","last_fullsized":"2024-09-10T19:06:30.165260+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:06.714078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:57.476724+0000","last_clean_scrub_stamp":"2024-09-10T19:06:06.714078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:33.025365+0000","scrub_duration":0.001109358,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.10","version":"70'81","reported_seq":338,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.165816+0000","last_change":"2024-09-10T19:03:50.010677+0000","last_active":"2024-09-10T19:06:30.165816+0000","last_peered":"2024-09-10T19:06:30.165816+0000","last_clean":"2024-09-10T19:06:30.165816+0000","last_became_active":"2024-09-10T18:57:02.443782+0000","last_became_peered":"2024-09-10T18:57:02.443782+0000","last_unstale":"2024-09-10T19:06:30.165816+0000","last_undegraded":"2024-09-10T19:06:30.165816+0000","last_fullsized":"2024-09-10T19:06:30.165816+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:03:50.010596+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:03:50.010596+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030874829999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"427'21","reported_seq":149,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.168086+0000","last_change":"2024-09-10T19:02:46.674532+0000","last_active":"2024-09-10T19:06:30.168086+0000","last_peered":"2024-09-10T19:06:30.168086+0000","last_clean":"2024-09-10T19:06:30.168086+0000","last_became_active":"2024-09-10T19:00:42.518672+0000","last_became_peered":"2024-09-10T19:00:42.518672+0000","last_unstale":"2024-09-10T19:06:30.168086+0000","last_undegraded":"2024-09-10T19:06:30.168086+0000","last_fullsized":"2024-09-10T19:06:30.168086+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T19:02:46.674451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:46.674451+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0082292530000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":33,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":447,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.167895+0000","last_change":"2024-09-10T19:04:34.812187+0000","last_active":"2024-09-10T19:06:30.167895+0000","last_peered":"2024-09-10T19:06:30.167895+0000","last_clean":"2024-09-10T19:06:30.167895+0000","last_became_active":"2024-09-10T18:55:40.390697+0000","last_became_peered":"2024-09-10T18:55:40.390697+0000","last_unstale":"2024-09-10T19:06:30.167895+0000","last_undegraded":"2024-09-10T19:06:30.167895+0000","last_fullsized":"2024-09-10T19:06:30.167895+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:34.812044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:04:34.812044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013174425,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"273'11","reported_seq":747,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:32.185599+0000","last_change":"2024-09-10T19:03:49.744278+0000","last_active":"2024-09-10T19:06:32.185599+0000","last_peered":"2024-09-10T19:06:32.185599+0000","last_clean":"2024-09-10T19:06:32.185599+0000","last_became_active":"2024-09-10T18:57:02.445268+0000","last_became_peered":"2024-09-10T18:57:02.445268+0000","last_unstale":"2024-09-10T19:06:32.185599+0000","last_undegraded":"2024-09-10T19:06:32.185599+0000","last_fullsized":"2024-09-10T19:06:32.185599+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'11","last_scrub_stamp":"2024-09-10T19:03:49.744177+0000","last_deep_scrub":"273'11","last_deep_scrub_stamp":"2024-09-10T19:03:49.744177+0000","last_clean_scrub_stamp":"2024-09-10T19:03:49.744177+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0069874999999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":479,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.171284+0000","last_change":"2024-09-10T19:03:23.737568+0000","last_active":"2024-09-10T19:06:30.171284+0000","last_peered":"2024-09-10T19:06:30.171284+0000","last_clean":"2024-09-10T19:06:30.171284+0000","last_became_active":"2024-09-10T18:55:40.391662+0000","last_became_peered":"2024-09-10T18:55:40.391662+0000","last_unstale":"2024-09-10T19:06:30.171284+0000","last_undegraded":"2024-09-10T19:06:30.171284+0000","last_fullsized":"2024-09-10T19:06:30.171284+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:23.737518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_clean_scrub_stamp":"2024-09-10T19:03:23.737518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037070459999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"427'21","reported_seq":373,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.163728+0000","last_change":"2024-09-10T19:03:37.134082+0000","last_active":"2024-09-10T19:06:30.163728+0000","last_peered":"2024-09-10T19:06:30.163728+0000","last_clean":"2024-09-10T19:06:30.163728+0000","last_became_active":"2024-09-10T19:02:49.561403+0000","last_became_peered":"2024-09-10T19:02:49.561403+0000","last_unstale":"2024-09-10T19:06:30.163728+0000","last_undegraded":"2024-09-10T19:06:30.163728+0000","last_fullsized":"2024-09-10T19:06:30.163728+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T19:03:37.134028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:03:37.134028+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058255470000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":511,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.162832+0000","last_change":"2024-09-10T19:03:36.087607+0000","last_active":"2024-09-10T19:06:30.162832+0000","last_peered":"2024-09-10T19:06:30.162832+0000","last_clean":"2024-09-10T19:06:30.162832+0000","last_became_active":"2024-09-10T19:00:42.518310+0000","last_became_peered":"2024-09-10T19:00:42.518310+0000","last_unstale":"2024-09-10T19:06:30.162832+0000","last_undegraded":"2024-09-10T19:06:30.162832+0000","last_fullsized":"2024-09-10T19:06:30.162832+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:36.087523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:03:36.087523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039520240000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":461,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.174172+0000","last_change":"2024-09-10T19:02:22.258152+0000","last_active":"2024-09-10T19:06:30.174172+0000","last_peered":"2024-09-10T19:06:30.174172+0000","last_clean":"2024-09-10T19:06:30.174172+0000","last_became_active":"2024-09-10T18:55:40.398243+0000","last_became_peered":"2024-09-10T18:55:40.398243+0000","last_unstale":"2024-09-10T19:06:30.174172+0000","last_undegraded":"2024-09-10T19:06:30.174172+0000","last_fullsized":"2024-09-10T19:06:30.174172+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:22.258058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:03.571786+0000","last_clean_scrub_stamp":"2024-09-10T19:02:22.258058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0054525670000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":452,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.167632+0000","last_change":"2024-09-10T19:03:22.930232+0000","last_active":"2024-09-10T19:06:30.167632+0000","last_peered":"2024-09-10T19:06:30.167632+0000","last_clean":"2024-09-10T19:06:30.167632+0000","last_became_active":"2024-09-10T19:00:42.516833+0000","last_became_peered":"2024-09-10T19:00:42.516833+0000","last_unstale":"2024-09-10T19:06:30.167632+0000","last_undegraded":"2024-09-10T19:06:30.167632+0000","last_fullsized":"2024-09-10T19:06:30.167632+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:22.930166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:04.631558+0000","last_clean_scrub_stamp":"2024-09-10T19:03:22.930166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001481163,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"427'32","reported_seq":504,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169309+0000","last_change":"2024-09-10T19:03:37.562262+0000","last_active":"2024-09-10T19:06:30.169309+0000","last_peered":"2024-09-10T19:06:30.169309+0000","last_clean":"2024-09-10T19:06:30.169309+0000","last_became_active":"2024-09-10T18:45:20.056885+0000","last_became_peered":"2024-09-10T18:45:20.056885+0000","last_unstale":"2024-09-10T19:06:30.169309+0000","last_undegraded":"2024-09-10T19:06:30.169309+0000","last_fullsized":"2024-09-10T19:06:30.169309+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T19:03:37.562188+0000","last_deep_scrub":"223'16","last_deep_scrub_stamp":"2024-09-10T18:54:47.324044+0000","last_clean_scrub_stamp":"2024-09-10T19:03:37.562188+0000","objects_scrubbed":5,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041736869999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":341,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:32.121093+0000","last_change":"2024-09-10T19:06:32.121019+0000","last_active":"2024-09-10T19:06:32.121093+0000","last_peered":"2024-09-10T19:06:32.121093+0000","last_clean":"2024-09-10T19:06:32.121093+0000","last_became_active":"2024-09-10T19:02:49.572748+0000","last_became_peered":"2024-09-10T19:02:49.572748+0000","last_unstale":"2024-09-10T19:06:32.121093+0000","last_undegraded":"2024-09-10T19:06:32.121093+0000","last_fullsized":"2024-09-10T19:06:32.121093+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:32.120944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:00.029389+0000","last_clean_scrub_stamp":"2024-09-10T19:06:32.120944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:35.654481+0000","scrub_duration":0.0048228450000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":89,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.166850+0000","last_change":"2024-09-10T19:02:49.573280+0000","last_active":"2024-09-10T19:06:30.166850+0000","last_peered":"2024-09-10T19:06:30.166850+0000","last_clean":"2024-09-10T19:06:30.166850+0000","last_became_active":"2024-09-10T19:02:49.572998+0000","last_became_peered":"2024-09-10T19:02:49.572998+0000","last_unstale":"2024-09-10T19:06:30.166850+0000","last_undegraded":"2024-09-10T19:06:30.166850+0000","last_fullsized":"2024-09-10T19:06:30.166850+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:15.321767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:02:15.321767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":469,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.173208+0000","last_change":"2024-09-10T19:06:08.922418+0000","last_active":"2024-09-10T19:06:30.173208+0000","last_peered":"2024-09-10T19:06:30.173208+0000","last_clean":"2024-09-10T19:06:30.173208+0000","last_became_active":"2024-09-10T18:55:40.395595+0000","last_became_peered":"2024-09-10T18:55:40.395595+0000","last_unstale":"2024-09-10T19:06:30.173208+0000","last_undegraded":"2024-09-10T19:06:30.173208+0000","last_fullsized":"2024-09-10T19:06:30.173208+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:08.922339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:06:08.922339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:33.632921+0000","scrub_duration":0.0055359140000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"427'12","reported_seq":474,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.168403+0000","last_change":"2024-09-10T19:02:50.391929+0000","last_active":"2024-09-10T19:06:30.168403+0000","last_peered":"2024-09-10T19:06:30.168403+0000","last_clean":"2024-09-10T19:06:30.168403+0000","last_became_active":"2024-09-10T18:45:20.059205+0000","last_became_peered":"2024-09-10T18:45:20.059205+0000","last_unstale":"2024-09-10T19:06:30.168403+0000","last_undegraded":"2024-09-10T19:06:30.168403+0000","last_fullsized":"2024-09-10T19:06:30.168403+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'8","last_scrub_stamp":"2024-09-10T19:02:50.391489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:50.391489+0000","objects_scrubbed":2,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041757890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":16,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":476,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169085+0000","last_change":"2024-09-10T19:02:16.704889+0000","last_active":"2024-09-10T19:06:30.169085+0000","last_peered":"2024-09-10T19:06:30.169085+0000","last_clean":"2024-09-10T19:06:30.169085+0000","last_became_active":"2024-09-10T18:47:36.415866+0000","last_became_peered":"2024-09-10T18:47:36.415866+0000","last_unstale":"2024-09-10T19:06:30.169085+0000","last_undegraded":"2024-09-10T19:06:30.169085+0000","last_fullsized":"2024-09-10T19:06:30.169085+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:16.704805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:16.704805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0083395729999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":497,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.164657+0000","last_change":"2024-09-10T19:06:20.478219+0000","last_active":"2024-09-10T19:06:30.164657+0000","last_peered":"2024-09-10T19:06:30.164657+0000","last_clean":"2024-09-10T19:06:30.164657+0000","last_became_active":"2024-09-10T18:57:02.453974+0000","last_became_peered":"2024-09-10T18:57:02.453974+0000","last_unstale":"2024-09-10T19:06:30.164657+0000","last_undegraded":"2024-09-10T19:06:30.164657+0000","last_fullsized":"2024-09-10T19:06:30.164657+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:20.478081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_clean_scrub_stamp":"2024-09-10T19:06:20.478081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:35.087111+0000","scrub_duration":0.0034106200000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":257,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169565+0000","last_change":"2024-09-10T19:04:32.388680+0000","last_active":"2024-09-10T19:06:30.169565+0000","last_peered":"2024-09-10T19:06:30.169565+0000","last_clean":"2024-09-10T19:06:30.169565+0000","last_became_active":"2024-09-10T18:57:02.447566+0000","last_became_peered":"2024-09-10T18:57:02.447566+0000","last_unstale":"2024-09-10T19:06:30.169565+0000","last_undegraded":"2024-09-10T19:06:30.169565+0000","last_fullsized":"2024-09-10T19:06:30.169565+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:32.388609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:55.145638+0000","last_clean_scrub_stamp":"2024-09-10T19:04:32.388609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0098792380000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.14","version":"70'81","reported_seq":371,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:31.848167+0000","last_change":"2024-09-10T19:06:31.848113+0000","last_active":"2024-09-10T19:06:31.848167+0000","last_peered":"2024-09-10T19:06:31.848167+0000","last_clean":"2024-09-10T19:06:31.848167+0000","last_became_active":"2024-09-10T18:57:02.442457+0000","last_became_peered":"2024-09-10T18:57:02.442457+0000","last_unstale":"2024-09-10T19:06:31.848167+0000","last_undegraded":"2024-09-10T19:06:31.848167+0000","last_fullsized":"2024-09-10T19:06:31.848167+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:06:31.848054+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:57.184383+0000","last_clean_scrub_stamp":"2024-09-10T19:06:31.848054+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:58.376194+0000","scrub_duration":0.0083693320000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.11","version":"427'32","reported_seq":526,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.172960+0000","last_change":"2024-09-10T19:03:43.653800+0000","last_active":"2024-09-10T19:06:30.172960+0000","last_peered":"2024-09-10T19:06:30.172960+0000","last_clean":"2024-09-10T19:06:30.172960+0000","last_became_active":"2024-09-10T18:57:02.461909+0000","last_became_peered":"2024-09-10T18:57:02.461909+0000","last_unstale":"2024-09-10T19:06:30.172960+0000","last_undegraded":"2024-09-10T19:06:30.172960+0000","last_fullsized":"2024-09-10T19:06:30.172960+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T19:03:43.653711+0000","last_deep_scrub":"79'16","last_deep_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_clean_scrub_stamp":"2024-09-10T19:03:43.653711+0000","objects_scrubbed":5,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037824379999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"56'1","reported_seq":505,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.162380+0000","last_change":"2024-09-10T19:06:06.554681+0000","last_active":"2024-09-10T19:06:30.162380+0000","last_peered":"2024-09-10T19:06:30.162380+0000","last_clean":"2024-09-10T19:06:30.162380+0000","last_became_active":"2024-09-10T18:47:36.421985+0000","last_became_peered":"2024-09-10T18:47:36.421985+0000","last_unstale":"2024-09-10T19:06:30.162380+0000","last_undegraded":"2024-09-10T19:06:30.162380+0000","last_fullsized":"2024-09-10T19:06:30.162380+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T19:06:06.554594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:06:06.554594+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:23.803720+0000","scrub_duration":0.0030413559999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":214,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169251+0000","last_change":"2024-09-10T19:06:28.848162+0000","last_active":"2024-09-10T19:06:30.169251+0000","last_peered":"2024-09-10T19:06:30.169251+0000","last_clean":"2024-09-10T19:06:30.169251+0000","last_became_active":"2024-09-10T18:57:02.435124+0000","last_became_peered":"2024-09-10T18:57:02.435124+0000","last_unstale":"2024-09-10T19:06:30.169251+0000","last_undegraded":"2024-09-10T19:06:30.169251+0000","last_fullsized":"2024-09-10T19:06:30.169251+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:28.848105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:06:28.848105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:51.629925+0000","scrub_duration":0.0051945860000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":479,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.171795+0000","last_change":"2024-09-10T19:06:20.199145+0000","last_active":"2024-09-10T19:06:30.171795+0000","last_peered":"2024-09-10T19:06:30.171795+0000","last_clean":"2024-09-10T19:06:30.171795+0000","last_became_active":"2024-09-10T18:55:40.397523+0000","last_became_peered":"2024-09-10T18:55:40.397523+0000","last_unstale":"2024-09-10T19:06:30.171795+0000","last_undegraded":"2024-09-10T19:06:30.171795+0000","last_fullsized":"2024-09-10T19:06:30.171795+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:20.199071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:06:20.199071+0000","last_clean_scrub_stamp":"2024-09-10T19:06:20.199071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:32.556730+0000","scrub_duration":0.0051592299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.11","version":"449'11818","reported_seq":11937,"reported_epoch":449,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-10T19:06:36.400446+0000","last_change":"2024-09-10T19:06:09.896045+0000","last_active":"2024-09-10T19:06:36.400446+0000","last_peered":"2024-09-10T19:06:36.400446+0000","last_clean":"2024-09-10T19:04:37.944310+0000","last_became_active":"2024-09-10T19:04:39.870192+0000","last_became_peered":"2024-09-10T19:04:39.870192+0000","last_unstale":"2024-09-10T19:06:36.400446+0000","last_undegraded":"2024-09-10T19:06:36.400446+0000","last_fullsized":"2024-09-10T19:06:36.400446+0000","mapping_epoch":407,"log_start":"426'8521","ondisk_log_start":"426'8521","created":404,"last_epoch_clean":405,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3297,"log_dups_size":0,"ondisk_log_size":3297,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3506176,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":54,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8410,"num_write_kb":270816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,0],"avail_no_missing":["3","0","6"],"object_location_counts":[{"shards":"0,3,6","objects":54}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"78'2","reported_seq":257,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.167260+0000","last_change":"2024-09-10T19:06:28.113529+0000","last_active":"2024-09-10T19:06:30.167260+0000","last_peered":"2024-09-10T19:06:30.167260+0000","last_clean":"2024-09-10T19:06:30.167260+0000","last_became_active":"2024-09-10T18:57:02.447198+0000","last_became_peered":"2024-09-10T18:57:02.447198+0000","last_unstale":"2024-09-10T19:06:30.167260+0000","last_undegraded":"2024-09-10T19:06:30.167260+0000","last_fullsized":"2024-09-10T19:06:30.167260+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'2","last_scrub_stamp":"2024-09-10T19:06:28.113436+0000","last_deep_scrub":"78'2","last_deep_scrub_stamp":"2024-09-10T19:02:15.686138+0000","last_clean_scrub_stamp":"2024-09-10T19:06:28.113436+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:35.604709+0000","scrub_duration":0.0025017680000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":480,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.165878+0000","last_change":"2024-09-10T19:03:44.943009+0000","last_active":"2024-09-10T19:06:30.165878+0000","last_peered":"2024-09-10T19:06:30.165878+0000","last_clean":"2024-09-10T19:06:30.165878+0000","last_became_active":"2024-09-10T18:57:02.443353+0000","last_became_peered":"2024-09-10T18:57:02.443353+0000","last_unstale":"2024-09-10T19:06:30.165878+0000","last_undegraded":"2024-09-10T19:06:30.165878+0000","last_fullsized":"2024-09-10T19:06:30.165878+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:44.942954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:03:44.942954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016874889999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"427'30","reported_seq":511,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.171556+0000","last_change":"2024-09-10T19:04:31.733896+0000","last_active":"2024-09-10T19:06:30.171556+0000","last_peered":"2024-09-10T19:06:30.171556+0000","last_clean":"2024-09-10T19:06:30.171556+0000","last_became_active":"2024-09-10T18:57:02.452473+0000","last_became_peered":"2024-09-10T18:57:02.452473+0000","last_unstale":"2024-09-10T19:06:30.171556+0000","last_undegraded":"2024-09-10T19:06:30.171556+0000","last_fullsized":"2024-09-10T19:06:30.171556+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'20","last_scrub_stamp":"2024-09-10T19:04:31.733806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:04:31.733806+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051209239999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":38,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"273'10","reported_seq":759,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:32.179233+0000","last_change":"2024-09-10T19:03:24.245622+0000","last_active":"2024-09-10T19:06:32.179233+0000","last_peered":"2024-09-10T19:06:32.179233+0000","last_clean":"2024-09-10T19:06:32.179233+0000","last_became_active":"2024-09-10T18:57:02.441763+0000","last_became_peered":"2024-09-10T18:57:02.441763+0000","last_unstale":"2024-09-10T19:06:32.179233+0000","last_undegraded":"2024-09-10T19:06:32.179233+0000","last_fullsized":"2024-09-10T19:06:32.179233+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'10","last_scrub_stamp":"2024-09-10T19:03:24.245532+0000","last_deep_scrub":"190'9","last_deep_scrub_stamp":"2024-09-10T18:53:07.746309+0000","last_clean_scrub_stamp":"2024-09-10T19:03:24.245532+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019049480000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"449'11654","reported_seq":21370,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:33.965848+0000","last_change":"2024-09-10T19:05:38.890706+0000","last_active":"2024-09-10T19:06:33.965848+0000","last_peered":"2024-09-10T19:06:33.965848+0000","last_clean":"2024-09-10T19:06:33.965848+0000","last_became_active":"2024-09-10T19:05:38.851253+0000","last_became_peered":"2024-09-10T19:05:38.851253+0000","last_unstale":"2024-09-10T19:06:33.965848+0000","last_undegraded":"2024-09-10T19:06:33.965848+0000","last_fullsized":"2024-09-10T19:06:33.965848+0000","mapping_epoch":433,"log_start":"421'8041","ondisk_log_start":"421'8041","created":404,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3613,"log_dups_size":0,"ondisk_log_size":3613,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13664256,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8247,"num_write_kb":270560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3202,"num_bytes_recovered":209846272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":89,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.167383+0000","last_change":"2024-09-10T19:02:49.582100+0000","last_active":"2024-09-10T19:06:30.167383+0000","last_peered":"2024-09-10T19:06:30.167383+0000","last_clean":"2024-09-10T19:06:30.167383+0000","last_became_active":"2024-09-10T19:02:49.581739+0000","last_became_peered":"2024-09-10T19:02:49.581739+0000","last_unstale":"2024-09-10T19:06:30.167383+0000","last_undegraded":"2024-09-10T19:06:30.167383+0000","last_fullsized":"2024-09-10T19:06:30.167383+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:23.327767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:03.475299+0000","last_clean_scrub_stamp":"2024-09-10T19:02:23.327767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":475,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.172563+0000","last_change":"2024-09-10T19:04:34.504898+0000","last_active":"2024-09-10T19:06:30.172563+0000","last_peered":"2024-09-10T19:06:30.172563+0000","last_clean":"2024-09-10T19:06:30.172563+0000","last_became_active":"2024-09-10T18:55:40.392524+0000","last_became_peered":"2024-09-10T18:55:40.392524+0000","last_unstale":"2024-09-10T19:06:30.172563+0000","last_undegraded":"2024-09-10T19:06:30.172563+0000","last_fullsized":"2024-09-10T19:06:30.172563+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:34.504828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:04:34.504828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016884452000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"427'27","reported_seq":525,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.162453+0000","last_change":"2024-09-10T19:04:33.326902+0000","last_active":"2024-09-10T19:06:30.162453+0000","last_peered":"2024-09-10T19:06:30.162453+0000","last_clean":"2024-09-10T19:06:30.162453+0000","last_became_active":"2024-09-10T18:45:20.058717+0000","last_became_peered":"2024-09-10T18:45:20.058717+0000","last_unstale":"2024-09-10T19:06:30.162453+0000","last_undegraded":"2024-09-10T19:06:30.162453+0000","last_fullsized":"2024-09-10T19:06:30.162453+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T19:04:33.326849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:04:33.326849+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0036326280000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":440,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.171694+0000","last_change":"2024-09-10T19:03:25.864320+0000","last_active":"2024-09-10T19:06:30.171694+0000","last_peered":"2024-09-10T19:06:30.171694+0000","last_clean":"2024-09-10T19:06:30.171694+0000","last_became_active":"2024-09-10T18:57:02.453464+0000","last_became_peered":"2024-09-10T18:57:02.453464+0000","last_unstale":"2024-09-10T19:06:30.171694+0000","last_undegraded":"2024-09-10T19:06:30.171694+0000","last_fullsized":"2024-09-10T19:06:30.171694+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:25.864264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:03:25.864264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013441410000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.13","version":"449'11852","reported_seq":23878,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:36.642271+0000","last_change":"2024-09-10T19:05:08.811448+0000","last_active":"2024-09-10T19:06:36.642271+0000","last_peered":"2024-09-10T19:06:36.642271+0000","last_clean":"2024-09-10T19:06:36.642271+0000","last_became_active":"2024-09-10T19:05:08.804974+0000","last_became_peered":"2024-09-10T19:05:08.804974+0000","last_unstale":"2024-09-10T19:06:36.642271+0000","last_undegraded":"2024-09-10T19:06:36.642271+0000","last_fullsized":"2024-09-10T19:06:36.642271+0000","mapping_epoch":420,"log_start":"424'8459","ondisk_log_start":"424'8459","created":404,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3393,"log_dups_size":0,"ondisk_log_size":3393,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4030464,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8416,"num_write_kb":271264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3971,"num_bytes_recovered":260243456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":452,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.170962+0000","last_change":"2024-09-10T19:03:43.645318+0000","last_active":"2024-09-10T19:06:30.170962+0000","last_peered":"2024-09-10T19:06:30.170962+0000","last_clean":"2024-09-10T19:06:30.170962+0000","last_became_active":"2024-09-10T18:45:24.267251+0000","last_became_peered":"2024-09-10T18:45:24.267251+0000","last_unstale":"2024-09-10T19:06:30.170962+0000","last_undegraded":"2024-09-10T19:06:30.170962+0000","last_fullsized":"2024-09-10T19:06:30.170962+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:43.645260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:03:43.645260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003834732,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"306'2","reported_seq":498,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.164072+0000","last_change":"2024-09-10T19:03:00.120585+0000","last_active":"2024-09-10T19:06:30.164072+0000","last_peered":"2024-09-10T19:06:30.164072+0000","last_clean":"2024-09-10T19:06:30.164072+0000","last_became_active":"2024-09-10T18:57:02.435758+0000","last_became_peered":"2024-09-10T18:57:02.435758+0000","last_unstale":"2024-09-10T19:06:30.164072+0000","last_undegraded":"2024-09-10T19:06:30.164072+0000","last_fullsized":"2024-09-10T19:06:30.164072+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"306'2","last_scrub_stamp":"2024-09-10T19:03:00.120529+0000","last_deep_scrub":"306'2","last_deep_scrub_stamp":"2024-09-10T18:59:20.534869+0000","last_clean_scrub_stamp":"2024-09-10T19:03:00.120529+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016153859999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"427'46","reported_seq":424,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169583+0000","last_change":"2024-09-10T19:04:38.485166+0000","last_active":"2024-09-10T19:06:30.169583+0000","last_peered":"2024-09-10T19:06:30.169583+0000","last_clean":"2024-09-10T19:06:30.169583+0000","last_became_active":"2024-09-10T19:02:49.594207+0000","last_became_peered":"2024-09-10T19:02:49.594207+0000","last_unstale":"2024-09-10T19:06:30.169583+0000","last_undegraded":"2024-09-10T19:06:30.169583+0000","last_fullsized":"2024-09-10T19:06:30.169583+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'34","last_scrub_stamp":"2024-09-10T19:04:38.485104+0000","last_deep_scrub":"258'34","last_deep_scrub_stamp":"2024-09-10T18:56:23.838951+0000","last_clean_scrub_stamp":"2024-09-10T19:04:38.485104+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001834518,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":317,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169906+0000","last_change":"2024-09-10T19:02:49.581903+0000","last_active":"2024-09-10T19:06:30.169906+0000","last_peered":"2024-09-10T19:06:30.169906+0000","last_clean":"2024-09-10T19:06:30.169906+0000","last_became_active":"2024-09-10T19:02:49.581575+0000","last_became_peered":"2024-09-10T19:02:49.581575+0000","last_unstale":"2024-09-10T19:06:30.169906+0000","last_undegraded":"2024-09-10T19:06:30.169906+0000","last_fullsized":"2024-09-10T19:06:30.169906+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:03.343366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:02:03.343366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0079641550000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.12","version":"449'12156","reported_seq":20446,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:33.979346+0000","last_change":"2024-09-10T19:05:03.855780+0000","last_active":"2024-09-10T19:06:33.979346+0000","last_peered":"2024-09-10T19:06:33.979346+0000","last_clean":"2024-09-10T19:06:33.979346+0000","last_became_active":"2024-09-10T19:05:03.829776+0000","last_became_peered":"2024-09-10T19:05:03.829776+0000","last_unstale":"2024-09-10T19:06:33.979346+0000","last_undegraded":"2024-09-10T19:06:33.979346+0000","last_fullsized":"2024-09-10T19:06:33.979346+0000","mapping_epoch":416,"log_start":"422'8534","ondisk_log_start":"422'8534","created":404,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3622,"log_dups_size":0,"ondisk_log_size":3622,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14909440,"num_objects":228,"num_object_clones":0,"num_object_copies":684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8608,"num_write_kb":282720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4090,"num_bytes_recovered":268042240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":472,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169011+0000","last_change":"2024-09-10T19:03:33.638108+0000","last_active":"2024-09-10T19:06:30.169011+0000","last_peered":"2024-09-10T19:06:30.169011+0000","last_clean":"2024-09-10T19:06:30.169011+0000","last_became_active":"2024-09-10T19:00:42.520239+0000","last_became_peered":"2024-09-10T19:00:42.520239+0000","last_unstale":"2024-09-10T19:06:30.169011+0000","last_undegraded":"2024-09-10T19:06:30.169011+0000","last_fullsized":"2024-09-10T19:06:30.169011+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:33.638010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:17.985545+0000","last_clean_scrub_stamp":"2024-09-10T19:03:33.638010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0035871750000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":478,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.174433+0000","last_change":"2024-09-10T19:03:36.615878+0000","last_active":"2024-09-10T19:06:30.174433+0000","last_peered":"2024-09-10T19:06:30.174433+0000","last_clean":"2024-09-10T19:06:30.174433+0000","last_became_active":"2024-09-10T18:57:02.460855+0000","last_became_peered":"2024-09-10T18:57:02.460855+0000","last_unstale":"2024-09-10T19:06:30.174433+0000","last_undegraded":"2024-09-10T19:06:30.174433+0000","last_fullsized":"2024-09-10T19:06:30.174433+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:36.615788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:03:36.615788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004374223,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"427'29","reported_seq":502,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169933+0000","last_change":"2024-09-10T19:03:40.698360+0000","last_active":"2024-09-10T19:06:30.169933+0000","last_peered":"2024-09-10T19:06:30.169933+0000","last_clean":"2024-09-10T19:06:30.169933+0000","last_became_active":"2024-09-10T18:45:20.058391+0000","last_became_peered":"2024-09-10T18:45:20.058391+0000","last_unstale":"2024-09-10T19:06:30.169933+0000","last_undegraded":"2024-09-10T19:06:30.169933+0000","last_fullsized":"2024-09-10T19:06:30.169933+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"388'23","last_scrub_stamp":"2024-09-10T19:03:40.697702+0000","last_deep_scrub":"388'23","last_deep_scrub_stamp":"2024-09-10T19:03:40.697702+0000","last_clean_scrub_stamp":"2024-09-10T19:03:40.697702+0000","objects_scrubbed":5,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.076817413000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":484,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169577+0000","last_change":"2024-09-10T19:03:46.623850+0000","last_active":"2024-09-10T19:06:30.169577+0000","last_peered":"2024-09-10T19:06:30.169577+0000","last_clean":"2024-09-10T19:06:30.169577+0000","last_became_active":"2024-09-10T18:57:02.449135+0000","last_became_peered":"2024-09-10T18:57:02.449135+0000","last_unstale":"2024-09-10T19:06:30.169577+0000","last_undegraded":"2024-09-10T19:06:30.169577+0000","last_fullsized":"2024-09-10T19:06:30.169577+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:46.623782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:03:46.623782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0075382380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":26,"stat_sum":{"num_bytes":204701720,"num_objects":3124,"num_object_clones":0,"num_object_copies":9372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":54,"num_objects_unfound":0,"num_objects_dirty":3124,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":262818,"num_write_kb":8499457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31487,"num_bytes_recovered":2063466496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1492135936,"data_stored":1492123720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88498,"ondisk_log_size":88498,"up":78,"acting":78,"num_store_stats":6},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":421,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1544,"num_read_kb":1301,"num_write":989,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":3152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":499712,"data_stored":13268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1057,"ondisk_log_size":1057,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":339,"num_read_kb":334,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":262688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1701,"ondisk_log_size":1701,"up":63,"acting":63,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":0,"seq":1765231558678,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":43,"seq":184683593993,"num_pgs":96,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1054068,"kb_used_data":164076,"kb_used_omap":1,"kb_used_meta":889982,"kb_avail":92691084,"statfs":{"total":95995035648,"available":94915670016,"internally_reserved":0,"allocated":168013824,"data_stored":167638529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":911341912},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:06:07 2024","interfaces":[{"interface":"back","average":{"1min":6.4509999999999996,"5min":3.0209999999999999,"15min":1.698},"min":{"1min":0.217,"5min":0.217,"15min":0.20899999999999999},"max":{"1min":44.320999999999998,"5min":49.353999999999999,"15min":49.353999999999999},"last":3.2349999999999999},{"interface":"front","average":{"1min":6.7450000000000001,"5min":3.202,"15min":1.819},"min":{"1min":0.23799999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":44.311999999999998,"5min":49.341999999999999,"15min":49.341999999999999},"last":3.5699999999999998}]},{"osd":1,"last update":"Tue Sep 10 19:06:18 2024","interfaces":[{"interface":"back","average":{"1min":4.9950000000000001,"5min":2.194,"15min":1.728},"min":{"1min":0.23200000000000001,"5min":0.217,"15min":0.217},"max":{"1min":85.775000000000006,"5min":85.775000000000006,"15min":85.775000000000006},"last":0.247},{"interface":"front","average":{"1min":4.9580000000000002,"5min":2.0550000000000002,"15min":1.621},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.23300000000000001},"max":{"1min":85.670000000000002,"5min":85.670000000000002,"15min":85.670000000000002},"last":0.48799999999999999}]},{"osd":2,"last update":"Tue Sep 10 19:06:18 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":0.999,"15min":0.86299999999999999},"min":{"1min":0.27000000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":4.0469999999999997,"5min":23.867000000000001,"15min":23.867000000000001},"last":2.306},{"interface":"front","average":{"1min":0.61099999999999999,"5min":1.149,"15min":0.90100000000000002},"min":{"1min":0.29399999999999998,"5min":0.28699999999999998,"15min":0.249},"max":{"1min":2.1360000000000001,"5min":23.779,"15min":23.779},"last":0.31}]},{"osd":3,"last update":"Tue Sep 10 19:06:18 2024","interfaces":[{"interface":"back","average":{"1min":1.8720000000000001,"5min":1.758,"15min":1.742},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":20.994,"5min":47.325000000000003,"15min":76.977999999999994},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.9870000000000001,"5min":1.7689999999999999,"15min":1.776},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.253},"max":{"1min":20.177,"5min":46.793999999999997,"15min":76.879999999999995},"last":2.3250000000000002}]},{"osd":4,"last update":"Tue Sep 10 19:06:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":1.1060000000000001,"15min":1.5109999999999999},"min":{"1min":0.26100000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.9039999999999999,"5min":32.469000000000001,"15min":165.03299999999999},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.877,"15min":1.365},"min":{"1min":0.30399999999999999,"5min":0.252,"15min":0.22500000000000001},"max":{"1min":3.0819999999999999,"5min":32.5,"15min":163.959},"last":0.34999999999999998}]},{"osd":5,"last update":"Tue Sep 10 19:06:18 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.077,"15min":1.9259999999999999},"min":{"1min":0.29499999999999998,"5min":0.246,"15min":0.246},"max":{"1min":2.968,"5min":23.827000000000002,"15min":288.56999999999999},"last":2.2519999999999998},{"interface":"front","average":{"1min":0.626,"5min":1.0089999999999999,"15min":1.897},"min":{"1min":0.32900000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":2.3809999999999998,"5min":23.667000000000002,"15min":291.05399999999997},"last":1.498}]},{"osd":7,"last update":"Tue Sep 10 19:05:52 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":2.794,"5min":2.794,"15min":2.794},"last":2.282},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.33500000000000002}]}]},{"osd":5,"up_from":37,"seq":158913790221,"num_pgs":90,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1310896,"kb_used_data":134696,"kb_used_omap":1,"kb_used_meta":1176190,"kb_avail":92434256,"statfs":{"total":95995035648,"available":94652678144,"internally_reserved":0,"allocated":137928704,"data_stored":137556510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":1204418904},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:05:58 2024","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.5149999999999999,"15min":1.9390000000000001},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":14.31,"5min":31.626999999999999,"15min":82.465000000000003},"last":2.9100000000000001},{"interface":"front","average":{"1min":1.9730000000000001,"5min":1.758,"15min":2.109},"min":{"1min":0.28000000000000003,"5min":0.26500000000000001,"15min":0.218},"max":{"1min":14.323,"5min":31.734999999999999,"15min":82.519000000000005},"last":2.9209999999999998}]},{"osd":1,"last update":"Tue Sep 10 19:05:58 2024","interfaces":[{"interface":"back","average":{"1min":9.7650000000000006,"5min":2.9460000000000002,"15min":2.282},"min":{"1min":0.23300000000000001,"5min":0.22500000000000001,"15min":0.19500000000000001},"max":{"1min":153.17099999999999,"5min":153.17099999999999,"15min":153.17099999999999},"last":2.2229999999999999},{"interface":"front","average":{"1min":9.7599999999999998,"5min":3.1000000000000001,"15min":2.4369999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.20399999999999999},"max":{"1min":153.196,"5min":153.196,"15min":153.196},"last":2.5819999999999999}]},{"osd":2,"last update":"Tue Sep 10 19:05:58 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":1.0309999999999999,"15min":0.89200000000000002},"min":{"1min":0.29299999999999998,"5min":0.249,"15min":0.249},"max":{"1min":2.5619999999999998,"5min":31.774000000000001,"15min":31.774000000000001},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.97699999999999998,"15min":0.85599999999999998},"min":{"1min":0.28000000000000003,"5min":0.27200000000000002,"15min":0.24399999999999999},"max":{"1min":2.8799999999999999,"5min":31.722999999999999,"15min":31.722999999999999},"last":2.6389999999999998}]},{"osd":3,"last update":"Tue Sep 10 19:05:58 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":1.2250000000000001,"15min":2.5870000000000002},"min":{"1min":0.33100000000000002,"5min":0.26300000000000001,"15min":0.24299999999999999},"max":{"1min":2.5800000000000001,"5min":31.753,"15min":123.34399999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.159,"5min":1.347,"15min":2.1989999999999998},"min":{"1min":0.29899999999999999,"5min":0.26200000000000001,"15min":0.245},"max":{"1min":5.117,"5min":31.699999999999999,"15min":82.415000000000006},"last":2.2599999999999998}]},{"osd":4,"last update":"Tue Sep 10 19:05:58 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":1.381,"15min":1.1479999999999999},"min":{"1min":0.20999999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":1.7749999999999999,"5min":51.070999999999998,"15min":51.070999999999998},"last":2.6749999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":1.2629999999999999,"15min":1.0509999999999999},"min":{"1min":0.25800000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":3.524,"5min":51.122999999999998,"15min":51.122999999999998},"last":1.0109999999999999}]},{"osd":6,"last update":"Tue Sep 10 19:06:20 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":1.2430000000000001,"15min":1.127},"min":{"1min":0.24199999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":4.0490000000000004,"5min":33.936999999999998,"15min":33.936999999999998},"last":2.7120000000000002},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.2230000000000001,"15min":1.0920000000000001},"min":{"1min":0.307,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":5.1710000000000003,"5min":33.951000000000001,"15min":33.951000000000001},"last":0.98999999999999999}]},{"osd":7,"last update":"Tue Sep 10 19:05:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":2.657,"5min":2.657,"15min":2.657},"last":2.2890000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":2.6230000000000002}]}]},{"osd":4,"up_from":30,"seq":128849019155,"num_pgs":84,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1407788,"kb_used_data":233188,"kb_used_omap":1,"kb_used_meta":1174526,"kb_avail":92337364,"statfs":{"total":95995035648,"available":94553460736,"internally_reserved":0,"allocated":238784512,"data_stored":238416431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":1202714968},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:05:59 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":1.514,"15min":1.4830000000000001},"min":{"1min":0.251,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":2.819,"5min":40.481000000000002,"15min":46.234999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.93200000000000005,"5min":1.5649999999999999,"15min":1.4850000000000001},"min":{"1min":0.35499999999999998,"5min":0.253,"15min":0.253},"max":{"1min":2.8380000000000001,"5min":40.512,"15min":46.264000000000003},"last":0.61599999999999999}]},{"osd":1,"last update":"Tue Sep 10 19:05:37 2024","interfaces":[{"interface":"back","average":{"1min":1.9259999999999999,"5min":1.0900000000000001,"15min":1.6499999999999999},"min":{"1min":0.34200000000000003,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":6.04,"5min":6.04,"15min":125.15600000000001},"last":0.31},{"interface":"front","average":{"1min":1.2290000000000001,"5min":0.85499999999999998,"15min":1.5649999999999999},"min":{"1min":0.25600000000000001,"5min":0.23300000000000001,"15min":0.23100000000000001},"max":{"1min":6.4219999999999997,"5min":6.4219999999999997,"15min":125.14700000000001},"last":0.45000000000000001}]},{"osd":2,"last update":"Tue Sep 10 19:05:37 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.70199999999999996,"15min":0.72799999999999998},"min":{"1min":0.29199999999999998,"5min":0.28499999999999998,"15min":0.25900000000000001},"max":{"1min":2.54,"5min":3.3999999999999999,"15min":5.6189999999999998},"last":0.55000000000000004},{"interface":"front","average":{"1min":11.096,"5min":2.8730000000000002,"15min":1.5149999999999999},"min":{"1min":0.33100000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":190.38200000000001,"5min":190.38200000000001,"15min":190.38200000000001},"last":0.47999999999999998}]},{"osd":3,"last update":"Tue Sep 10 19:05:37 2024","interfaces":[{"interface":"back","average":{"1min":11.266,"5min":3.8479999999999999,"15min":2.9900000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.23799999999999999},"max":{"1min":190.41300000000001,"5min":190.41300000000001,"15min":190.41300000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.609,"15min":1.887},"min":{"1min":0.315,"5min":0.26000000000000001,"15min":0.25700000000000001},"max":{"1min":6.0880000000000001,"5min":39.323,"15min":110.914},"last":0.52600000000000002}]},{"osd":5,"last update":"Tue Sep 10 19:05:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.83099999999999996,"15min":0.96499999999999997},"min":{"1min":0.24299999999999999,"5min":0.221,"15min":0.20499999999999999},"max":{"1min":8.1430000000000007,"5min":8.1430000000000007,"15min":10.912000000000001},"last":0.34599999999999997},{"interface":"front","average":{"1min":1.05,"5min":0.89700000000000002,"15min":0.94099999999999995},"min":{"1min":0.251,"5min":0.23200000000000001,"15min":0.216},"max":{"1min":8.3170000000000002,"5min":8.3170000000000002,"15min":10.147},"last":0.41699999999999998}]},{"osd":6,"last update":"Tue Sep 10 19:06:23 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.74199999999999999,"15min":0.81399999999999995},"min":{"1min":0.33900000000000002,"5min":0.24099999999999999,"15min":0.221},"max":{"1min":2.3740000000000001,"5min":5.351,"15min":8.4030000000000005},"last":0.57099999999999995},{"interface":"front","average":{"1min":1.0680000000000001,"5min":0.95899999999999996,"15min":1.026},"min":{"1min":0.29499999999999998,"5min":0.26300000000000001,"15min":0.23899999999999999},"max":{"1min":4.0099999999999998,"5min":4.8949999999999996,"15min":21.254000000000001},"last":0.60399999999999998}]},{"osd":7,"last update":"Tue Sep 10 19:05:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":2.347,"5min":2.347,"15min":2.347},"last":0.48699999999999999},{"interface":"front","average":{"1min":9.8569999999999993,"5min":9.8569999999999993,"15min":9.8569999999999993},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":190.33799999999999,"5min":190.33799999999999,"15min":190.33799999999999},"last":0.83199999999999996}]}]},{"osd":3,"up_from":25,"seq":107374182680,"num_pgs":93,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1940784,"kb_used_data":447848,"kb_used_omap":1,"kb_used_meta":1492926,"kb_avail":91804368,"statfs":{"total":95995035648,"available":94007672832,"internally_reserved":0,"allocated":458596352,"data_stored":458224049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":1528756568},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:05:59 2024","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.2210000000000001,"15min":2.4039999999999999},"min":{"1min":0.30599999999999999,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":4.7770000000000001,"5min":5.6349999999999998,"15min":290.721},"last":0.57699999999999996},{"interface":"front","average":{"1min":1.421,"5min":1.3480000000000001,"15min":2.6059999999999999},"min":{"1min":0.33100000000000002,"5min":0.18099999999999999,"15min":0.17499999999999999},"max":{"1min":3.5649999999999999,"5min":12.103,"15min":293.59300000000002},"last":0.27800000000000002}]},{"osd":1,"last update":"Tue Sep 10 19:06:12 2024","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.419,"15min":1.619},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.16200000000000001},"max":{"1min":6.0330000000000004,"5min":16.081,"15min":58.215000000000003},"last":0.63700000000000001},{"interface":"front","average":{"1min":1.915,"5min":1.6100000000000001,"15min":1.581},"min":{"1min":0.374,"5min":0.23599999999999999,"15min":0.188},"max":{"1min":7.165,"5min":16.329000000000001,"15min":57.896999999999998},"last":0.25600000000000001}]},{"osd":2,"last update":"Tue Sep 10 19:06:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":0.98099999999999998,"15min":0.96599999999999997},"min":{"1min":0.27900000000000003,"5min":0.25700000000000001,"15min":0.22800000000000001},"max":{"1min":3.1629999999999998,"5min":4.2409999999999997,"15min":5.6680000000000001},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":1.0189999999999999,"15min":0.88900000000000001},"min":{"1min":0.35199999999999998,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":3.0910000000000002,"5min":5.46,"15min":5.46},"last":0.5}]},{"osd":4,"last update":"Tue Sep 10 19:05:41 2024","interfaces":[{"interface":"back","average":{"1min":1.673,"5min":1.4950000000000001,"15min":1.417},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.247},"max":{"1min":3.9510000000000001,"5min":16.356000000000002,"15min":29.013000000000002},"last":1.643},{"interface":"front","average":{"1min":1.472,"5min":1.4710000000000001,"15min":1.4239999999999999},"min":{"1min":0.35099999999999998,"5min":0.33000000000000002,"15min":0.26800000000000002},"max":{"1min":3.9780000000000002,"5min":17.289999999999999,"15min":28.826000000000001},"last":1.022}]},{"osd":5,"last update":"Tue Sep 10 19:05:59 2024","interfaces":[{"interface":"back","average":{"1min":1.645,"5min":1.222,"15min":1.288},"min":{"1min":0.51100000000000001,"5min":0.248,"15min":0.248},"max":{"1min":4.8730000000000002,"5min":5.1920000000000002,"15min":7.2300000000000004},"last":1.0780000000000001},{"interface":"front","average":{"1min":2.1749999999999998,"5min":1.597,"15min":1.3839999999999999},"min":{"1min":0.59099999999999997,"5min":0.28299999999999997,"15min":0.255},"max":{"1min":13.117000000000001,"5min":17.899999999999999,"15min":17.899999999999999},"last":1.056}]},{"osd":6,"last update":"Tue Sep 10 19:06:21 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.296,"15min":1.298},"min":{"1min":0.34200000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.9530000000000001,"5min":5.8339999999999996,"15min":5.8339999999999996},"last":0.47299999999999998},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.236,"15min":1.2450000000000001},"min":{"1min":0.38400000000000001,"5min":0.33000000000000002,"15min":0.29399999999999998},"max":{"1min":4.0019999999999998,"5min":6.6390000000000002,"15min":6.6390000000000002},"last":1.8340000000000001}]},{"osd":7,"last update":"Tue Sep 10 19:05:53 2024","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":4.8499999999999996,"5min":4.8499999999999996,"15min":4.8499999999999996},"last":0.78200000000000003},{"interface":"front","average":{"1min":2.2370000000000001,"5min":2.2370000000000001,"15min":2.2370000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":13.045999999999999,"5min":13.045999999999999,"15min":13.045999999999999},"last":0.55700000000000005}]}]},{"osd":2,"up_from":0,"seq":77309411612,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834575136,"num_pgs":101,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1807568,"kb_used_data":307400,"kb_used_omap":1,"kb_used_meta":1500158,"kb_avail":91937584,"statfs":{"total":95995035648,"available":94144086016,"internally_reserved":0,"allocated":314777600,"data_stored":314374016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":1536162136},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:06:01 2024","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.3480000000000001,"15min":1.8260000000000001},"min":{"1min":0.26800000000000002,"5min":0.158,"15min":0.158},"max":{"1min":4.9829999999999997,"5min":25.370999999999999,"15min":111.014},"last":2.3109999999999999},{"interface":"front","average":{"1min":1.784,"5min":1.5209999999999999,"15min":1.8799999999999999},"min":{"1min":0.30599999999999999,"5min":0.217,"15min":0.19600000000000001},"max":{"1min":5.1029999999999998,"5min":27.809999999999999,"15min":111.072},"last":0.755}]},{"osd":2,"last update":"Tue Sep 10 19:05:45 2024","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":0.77900000000000003,"15min":0.79600000000000004},"min":{"1min":0.222,"5min":0.222,"15min":0.186},"max":{"1min":4.835,"5min":4.835,"15min":4.835},"last":0.71599999999999997},{"interface":"front","average":{"1min":1.419,"5min":0.92900000000000005,"15min":0.82499999999999996},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.215},"max":{"1min":9.484,"5min":9.484,"15min":9.484},"last":0.61899999999999999}]},{"osd":3,"last update":"Tue Sep 10 19:06:13 2024","interfaces":[{"interface":"back","average":{"1min":2.1760000000000002,"5min":2.1859999999999999,"15min":2.3100000000000001},"min":{"1min":0.29399999999999998,"5min":0.218,"15min":0.185},"max":{"1min":6.5149999999999997,"5min":48.582999999999998,"15min":92.069000000000003},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.72,"5min":1.6359999999999999,"15min":1.917},"min":{"1min":0.29199999999999998,"5min":0.23100000000000001,"15min":0.19900000000000001},"max":{"1min":9.8409999999999993,"5min":51.847000000000001,"15min":88.658000000000001},"last":2.286}]},{"osd":4,"last update":"Tue Sep 10 19:06:34 2024","interfaces":[{"interface":"back","average":{"1min":1.649,"5min":1.2450000000000001,"15min":1.2050000000000001},"min":{"1min":0.41299999999999998,"5min":0.28100000000000003,"15min":0.27800000000000002},"max":{"1min":7.2699999999999996,"5min":19.126999999999999,"15min":19.126999999999999},"last":2.2080000000000002},{"interface":"front","average":{"1min":1.871,"5min":1.431,"15min":1.202},"min":{"1min":0.40699999999999997,"5min":0.26300000000000001,"15min":0.252},"max":{"1min":5.9340000000000002,"5min":20.152000000000001,"15min":20.152000000000001},"last":0.63}]},{"osd":5,"last update":"Tue Sep 10 19:06:01 2024","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.2549999999999999,"15min":1.274},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":3.677,"5min":9.9369999999999994,"15min":10.334},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.97899999999999998,"15min":1.1499999999999999},"min":{"1min":0.34799999999999998,"5min":0.29199999999999998,"15min":0.28799999999999998},"max":{"1min":2.5950000000000002,"5min":4.3600000000000003,"15min":16.039999999999999},"last":2.262}]},{"osd":6,"last update":"Tue Sep 10 19:06:13 2024","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.25,"15min":1.458},"min":{"1min":0.435,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":3.7210000000000001,"5min":9.9760000000000009,"15min":33.290999999999997},"last":0.85899999999999999},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.286,"15min":1.4910000000000001},"min":{"1min":0.44500000000000001,"5min":0.32700000000000001,"15min":0.26600000000000001},"max":{"1min":7.7400000000000002,"5min":17.352,"15min":33.310000000000002},"last":3.23}]},{"osd":7,"last update":"Tue Sep 10 19:05:51 2024","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":3.6800000000000002,"5min":3.6800000000000002,"15min":3.6800000000000002},"last":0.68400000000000005},{"interface":"front","average":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":10.010999999999999,"5min":10.010999999999999,"15min":10.010999999999999},"last":0.88500000000000001}]}]},{"osd":0,"up_from":241,"seq":1035087118468,"num_pgs":84,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1226088,"kb_used_data":296224,"kb_used_omap":4,"kb_used_meta":929723,"kb_avail":92519064,"statfs":{"total":95995035648,"available":94739521536,"internally_reserved":0,"allocated":303333376,"data_stored":302968623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":952036870},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 19:05:59 2024","interfaces":[{"interface":"back","average":{"1min":4.3029999999999999,"5min":1.6779999999999999,"15min":2.1160000000000001},"min":{"1min":0.187,"5min":0.184,"15min":0.17100000000000001},"max":{"1min":47.237000000000002,"5min":47.237000000000002,"15min":47.237000000000002},"last":1.746},{"interface":"front","average":{"1min":4.117,"5min":1.5669999999999999,"15min":2.0649999999999999},"min":{"1min":0.224,"5min":0.193,"15min":0.193},"max":{"1min":47.198999999999998,"5min":47.198999999999998,"15min":47.198999999999998},"last":2.1640000000000001}]},{"osd":2,"last update":"Tue Sep 10 19:05:59 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.82999999999999996,"15min":0.92200000000000004},"min":{"1min":0.223,"5min":0.223,"15min":0.20699999999999999},"max":{"1min":6.0170000000000003,"5min":6.0170000000000003,"15min":13.305},"last":1.107},{"interface":"front","average":{"1min":1.0980000000000001,"5min":0.85799999999999998,"15min":0.86199999999999999},"min":{"1min":0.23100000000000001,"5min":0.22900000000000001,"15min":0.22800000000000001},"max":{"1min":6.1909999999999998,"5min":9.5099999999999998,"15min":13.693},"last":0.91100000000000003}]},{"osd":3,"last update":"Tue Sep 10 19:05:59 2024","interfaces":[{"interface":"back","average":{"1min":1.5780000000000001,"5min":1.25,"15min":1.2210000000000001},"min":{"1min":0.34999999999999998,"5min":0.216,"15min":0.216},"max":{"1min":6.8380000000000001,"5min":6.8380000000000001,"15min":21.108000000000001},"last":2.1869999999999998},{"interface":"front","average":{"1min":1.9330000000000001,"5min":1.3400000000000001,"15min":1.627},"min":{"1min":0.26100000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":8.0549999999999997,"5min":8.0549999999999997,"15min":11.629},"last":0.72799999999999998}]},{"osd":4,"last update":"Tue Sep 10 19:05:59 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.111,"15min":0.96799999999999997},"min":{"1min":0.30499999999999999,"5min":0.26400000000000001,"15min":0.26000000000000001},"max":{"1min":5.9560000000000004,"5min":5.9560000000000004,"15min":13.161},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.603,"5min":1.1100000000000001,"15min":1.0660000000000001},"min":{"1min":0.312,"5min":0.28100000000000003,"15min":0.27600000000000002},"max":{"1min":6.3719999999999999,"5min":6.3719999999999999,"15min":6.3719999999999999},"last":0.752}]},{"osd":5,"last update":"Tue Sep 10 19:05:59 2024","interfaces":[{"interface":"back","average":{"1min":1.583,"5min":2.2610000000000001,"15min":1.5009999999999999},"min":{"1min":0.34000000000000002,"5min":0.311,"15min":0.31},"max":{"1min":6.5679999999999996,"5min":97.296999999999997,"15min":97.296999999999997},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.1439999999999999,"5min":2.218,"15min":1.379},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":5.7939999999999996,"5min":99.221999999999994,"15min":99.221999999999994},"last":0.60399999999999998}]},{"osd":6,"last update":"Tue Sep 10 19:05:59 2024","interfaces":[{"interface":"back","average":{"1min":9.8659999999999997,"5min":3.0609999999999999,"15min":1.724},"min":{"1min":0.35299999999999998,"5min":0.33600000000000002,"15min":0.312},"max":{"1min":216.21100000000001,"5min":216.21100000000001,"15min":216.21100000000001},"last":0.85399999999999998},{"interface":"front","average":{"1min":9.5069999999999997,"5min":3.1960000000000002,"15min":1.665},"min":{"1min":0.379,"5min":0.379,"15min":0.313},"max":{"1min":216.054,"5min":216.054,"15min":216.054},"last":0.63400000000000001}]},{"osd":7,"last update":"Tue Sep 10 19:05:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":6.3979999999999997,"5min":6.3979999999999997,"15min":6.3979999999999997},"last":2.1309999999999998},{"interface":"front","average":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":6.8700000000000001,"5min":6.8700000000000001,"15min":6.8700000000000001},"last":0.77900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":2668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":283049984,"data_stored":283049984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":303763456,"data_stored":303759384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":430440448,"data_stored":430440448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":206639104,"data_stored":206635032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":108462080,"data_stored":108462080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":159780864,"data_stored":159776792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T19:06:40.294 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T19:06:40.312 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:40.337 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:40.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:40.338 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:40.439 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:40.462 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:40.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:40.463 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:40.508 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:06:40.514 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T19:06:40.514 INFO:teuthology.orchestra.run.smithi018.stdout: "ops": [], 2024-09-10T19:06:40.514 INFO:teuthology.orchestra.run.smithi018.stdout: "num_ops": 0 2024-09-10T19:06:40.514 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T19:06:40.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:06:40 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:06:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:06:40.564 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:40.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:40.588 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:40.588 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:40.689 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:40.715 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:40.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:40.716 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:40.817 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:40.843 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:40.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:40.844 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:40.862 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:40 smithi018 ceph-mon[40196]: from='client.15981 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:06:40.863 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:40 smithi018 ceph-mon[40196]: pgmap v1004: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 117 MiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 683 op/s; 54/6183 objects misplaced (0.873%) 2024-09-10T19:06:40.863 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:40 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:06:40.863 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:40 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:06:40.863 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:40 smithi018 ceph-mon[29336]: from='client.15981 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:06:40.863 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:40 smithi018 ceph-mon[29336]: pgmap v1004: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 117 MiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 683 op/s; 54/6183 objects misplaced (0.873%) 2024-09-10T19:06:40.863 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:40 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:06:40.863 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:40 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:06:40.945 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:40.967 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:40.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:40.968 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:41.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:40 smithi103 ceph-mon[33163]: from='client.15981 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:06:41.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:40 smithi103 ceph-mon[33163]: pgmap v1004: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 117 MiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 683 op/s; 54/6183 objects misplaced (0.873%) 2024-09-10T19:06:41.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:40 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:06:41.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:40 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:06:41.069 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:41.091 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:41.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:41.092 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:41.193 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:41.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:06:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:06:40] "GET /metrics HTTP/1.1" 200 224459 "" "Prometheus/2.43.0" 2024-09-10T19:06:41.216 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:41.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:41.217 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:41.318 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:41.343 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:41.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:41.344 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:41.424 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-10T19:06:41.445 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:41.471 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:41.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:41.472 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:41.573 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:41.597 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:41.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:41.597 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:41.698 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:41.725 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:41.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:41.726 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:41.801 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:06:41.825 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-10T19:06:41.826 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:41.851 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:41.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:41.852 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:41.953 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:41.975 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:41.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:41.976 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:42.077 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:42.100 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:42.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:42.101 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:42.201 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:42.228 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:42.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:42.229 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:42.330 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:42.355 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:42.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:42.357 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:42.457 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:42.481 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:42.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:42.482 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:42.583 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:42.605 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:42.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:42.606 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:42.707 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:42.729 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:42.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:42.730 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:42.831 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:42.854 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:42.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:42.855 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:42.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:42 smithi018 ceph-mon[29336]: pgmap v1005: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 45 MiB data, 7.9 GiB used, 529 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 405 op/s 2024-09-10T19:06:42.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:42 smithi018 ceph-mon[40196]: pgmap v1005: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 45 MiB data, 7.9 GiB used, 529 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 405 op/s 2024-09-10T19:06:42.956 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:42.982 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:42.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:42.983 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:43.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:42 smithi103 ceph-mon[33163]: pgmap v1005: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 45 MiB data, 7.9 GiB used, 529 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 405 op/s 2024-09-10T19:06:43.085 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:43.111 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:43.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:43.112 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:43.212 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:43.239 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:43.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:43.240 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:43.271 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_4 2024-09-10T19:06:43.271 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd pool rm unique_pool_4 unique_pool_4 --yes-i-really-really-mean-it 2024-09-10T19:06:43.342 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:43.368 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:43.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:43.369 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:43.469 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:43.495 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:43.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:43.496 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:43.597 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:43.620 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:43.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:43.621 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:43.721 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:43.748 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:43.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:43.749 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:43.850 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:43.875 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:43.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:43.876 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:43.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:43 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:06:43.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:43 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:06:43.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:43 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:06:43.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:43 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:06:43.977 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:44.003 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:44.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:44.004 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:44.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:43 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:06:44.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:43 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:06:44.105 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:44.129 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:44.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:44.130 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:44.231 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:44.257 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:44.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:44.258 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:44.359 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:44.384 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:44.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:44.385 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:44.439 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:06:44.439 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:06:44.486 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:44.509 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:44.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:44.511 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:44.611 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:44.638 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:44.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:44.639 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:44.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:44 smithi018 ceph-mon[29336]: pgmap v1006: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 12 MiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 902 B/s rd, 0 B/s wr, 241 op/s 2024-09-10T19:06:44.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:44 smithi018 ceph-mon[29336]: osdmap e450: 8 total, 8 up, 6 in 2024-09-10T19:06:44.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:44 smithi018 ceph-mon[40196]: pgmap v1006: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 12 MiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 902 B/s rd, 0 B/s wr, 241 op/s 2024-09-10T19:06:44.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:44 smithi018 ceph-mon[40196]: osdmap e450: 8 total, 8 up, 6 in 2024-09-10T19:06:44.740 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:44.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:44.764 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:44.764 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:44.865 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:44.890 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:44.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:44.892 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:44.992 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:45.015 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:45.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:45.016 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:45.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:44 smithi103 ceph-mon[33163]: pgmap v1006: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 12 MiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 902 B/s rd, 0 B/s wr, 241 op/s 2024-09-10T19:06:45.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:44 smithi103 ceph-mon[33163]: osdmap e450: 8 total, 8 up, 6 in 2024-09-10T19:06:45.119 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:45.145 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:45.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:45.146 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:45.247 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:45.272 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:45.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:45.273 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:45.374 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:45.397 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:45.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:45.398 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:45.498 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:45.521 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:45.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:45.522 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:45.622 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:45.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:45.649 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:45.650 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:45.750 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:45.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:45.774 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:45.774 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:45.874 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:45.901 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:45.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:45.902 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:46.003 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:46.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:46.030 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:46.030 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:46.131 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:46.154 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:46.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:46.155 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:46.256 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:46.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:46.280 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:46.280 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:46.381 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:46.407 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:46.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:46.408 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:46.484 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:06:46.509 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:46.535 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:46.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:46.536 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:46.637 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:46.662 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:46.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:46.663 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:46.664 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T19:06:46.664 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:46 smithi018 ceph-mon[29336]: pgmap v1008: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 12 MiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 969 op/s 2024-09-10T19:06:46.665 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:46 smithi018 ceph-mon[40196]: pgmap v1008: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 12 MiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 969 op/s 2024-09-10T19:06:46.671 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-09-10T19:06:46.763 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:46.788 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:46.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:46.789 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:46.840 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:46 smithi103 ceph-mon[33163]: pgmap v1008: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 12 MiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 969 op/s 2024-09-10T19:06:46.889 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:46.912 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:46.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:46.914 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:47.014 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:47.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:47.037 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:47.038 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:47.140 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:47.163 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:47.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:47.164 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:47.265 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:47.287 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:47.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:47.289 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:47.389 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:47.412 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:47.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:47.413 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:47.514 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:47.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:06:47 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:06:47.536 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:47.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:47.537 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:47.593 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is set 2024-09-10T19:06:47.638 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:47.660 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:47.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:47.661 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:47.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:47 smithi018 ceph-mon[29336]: from='client.25883 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:06:47.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:47 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/384100582' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:06:47.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:47 smithi018 ceph-mon[40196]: from='client.25883 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:06:47.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:47 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/384100582' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:06:47.762 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:47.785 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:47.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:47.786 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:47.887 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:47.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:47.911 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:47.911 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:48.012 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:48.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:47 smithi103 ceph-mon[33163]: from='client.25883 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:06:48.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:47 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/384100582' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:06:48.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:48.034 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:48.034 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:48.135 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:48.158 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:48.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:48.159 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:48.260 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:48.282 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:48.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:48.283 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:48.384 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:48.406 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:48.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:48.407 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:48.508 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:48.530 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:48.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:48.531 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:48.632 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:48.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:48.655 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:48.655 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:48.756 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:48.779 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:48.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:48.780 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:48.881 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:48.895 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:48 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:48.896 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:48 smithi018 ceph-mon[40196]: pgmap v1009: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 12 MiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 314 op/s 2024-09-10T19:06:48.896 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:48 smithi018 ceph-mon[40196]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:06:48.896 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:48 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/384100582' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T19:06:48.896 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:48 smithi018 ceph-mon[40196]: osdmap e451: 8 total, 8 up, 6 in 2024-09-10T19:06:48.896 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:48 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:48.896 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:48 smithi018 ceph-mon[29336]: pgmap v1009: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 12 MiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 314 op/s 2024-09-10T19:06:48.896 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:48 smithi018 ceph-mon[29336]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:06:48.897 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:48 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/384100582' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T19:06:48.897 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:48 smithi018 ceph-mon[29336]: osdmap e451: 8 total, 8 up, 6 in 2024-09-10T19:06:48.903 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:48.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:48.904 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:49.005 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:49.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:48 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:49.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:48 smithi103 ceph-mon[33163]: pgmap v1009: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 12 MiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 314 op/s 2024-09-10T19:06:49.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:48 smithi103 ceph-mon[33163]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:06:49.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:48 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/384100582' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-10T19:06:49.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:48 smithi103 ceph-mon[33163]: osdmap e451: 8 total, 8 up, 6 in 2024-09-10T19:06:49.027 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:49.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:49.028 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:49.130 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:49.153 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:49.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:49.154 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:49.255 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:49.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:49.279 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:49.279 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:49.380 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:49.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:49.404 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:49.404 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:49.505 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:49.527 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:49.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:49.528 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:49.629 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:49.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:49.662 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:49.662 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:49.763 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:49.786 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:49.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:49.787 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:49.889 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:49.912 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:49.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:49.913 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:50.014 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:50.036 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:50.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:50.037 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:50.138 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:50.145 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:49 smithi018 ceph-mon[40196]: pgmap v1011: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 12 MiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 73 op/s 2024-09-10T19:06:50.146 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:49 smithi018 ceph-mon[29336]: pgmap v1011: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 12 MiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 73 op/s 2024-09-10T19:06:50.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:50.162 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:50.162 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:50.170 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":1008,"stamp":"2024-09-10T19:06:45.519459+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":12097756,"num_objects":369,"num_object_clones":0,"num_object_copies":1107,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":369,"num_whiteouts":0,"num_read":1918,"num_read_kb":1670,"num_write":266957,"num_write_kb":8499500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31730,"num_bytes_recovered":2063734085,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91498,"ondisk_log_size":91498,"up":534,"acting":534,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":548,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":562470912,"kb_used":7994172,"kb_used_data":809800,"kb_used_omap":12,"kb_used_meta":7184307,"kb_avail":554476740,"statfs":{"total":575970213888,"available":567784181760,"internally_reserved":0,"allocated":829235200,"data_stored":826986052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13122,"internal_metadata":7356730558},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-621903896,"num_objects":-9493,"num_object_clones":0,"num_object_copies":-28479,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-343,"num_objects_unfound":0,"num_objects_dirty":-9493,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":9690,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003299"},"pg_stats":[{"pgid":"11.16","version":"449'11992","reported_seq":21781,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:37.861211+0000","last_change":"2024-09-10T19:05:41.625531+0000","last_active":"2024-09-10T19:06:37.861211+0000","last_peered":"2024-09-10T19:06:37.861211+0000","last_clean":"2024-09-10T19:06:37.861211+0000","last_became_active":"2024-09-10T19:05:41.596690+0000","last_became_peered":"2024-09-10T19:05:41.596690+0000","last_unstale":"2024-09-10T19:06:37.861211+0000","last_undegraded":"2024-09-10T19:06:37.861211+0000","last_fullsized":"2024-09-10T19:06:37.861211+0000","mapping_epoch":435,"log_start":"424'8585","ondisk_log_start":"424'8585","created":404,"last_epoch_clean":436,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3407,"log_dups_size":0,"ondisk_log_size":3407,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8518,"num_write_kb":271808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3225,"num_bytes_recovered":211353600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":472,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.171716+0000","last_change":"2024-09-10T19:06:27.199518+0000","last_active":"2024-09-10T19:06:30.171716+0000","last_peered":"2024-09-10T19:06:30.171716+0000","last_clean":"2024-09-10T19:06:30.171716+0000","last_became_active":"2024-09-10T18:57:02.460451+0000","last_became_peered":"2024-09-10T18:57:02.460451+0000","last_unstale":"2024-09-10T19:06:30.171716+0000","last_undegraded":"2024-09-10T19:06:30.171716+0000","last_fullsized":"2024-09-10T19:06:30.171716+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:27.199436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:06:27.199436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:29.399699+0000","scrub_duration":0.0067911150000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":460,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:32.188503+0000","last_change":"2024-09-10T19:03:38.825020+0000","last_active":"2024-09-10T19:06:32.188503+0000","last_peered":"2024-09-10T19:06:32.188503+0000","last_clean":"2024-09-10T19:06:32.188503+0000","last_became_active":"2024-09-10T18:57:02.440648+0000","last_became_peered":"2024-09-10T18:57:02.440648+0000","last_unstale":"2024-09-10T19:06:32.188503+0000","last_undegraded":"2024-09-10T19:06:32.188503+0000","last_fullsized":"2024-09-10T19:06:32.188503+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:38.824816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:59.848700+0000","last_clean_scrub_stamp":"2024-09-10T19:03:38.824816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0071402059999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"427'39","reported_seq":565,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.172715+0000","last_change":"2024-09-10T19:02:20.251547+0000","last_active":"2024-09-10T19:06:30.172715+0000","last_peered":"2024-09-10T19:06:30.172715+0000","last_clean":"2024-09-10T19:06:30.172715+0000","last_became_active":"2024-09-10T18:55:40.394717+0000","last_became_peered":"2024-09-10T18:55:40.394717+0000","last_unstale":"2024-09-10T19:06:30.172715+0000","last_undegraded":"2024-09-10T19:06:30.172715+0000","last_fullsized":"2024-09-10T19:06:30.172715+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'27","last_scrub_stamp":"2024-09-10T19:02:20.251456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:20.251456+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011781700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":464,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169133+0000","last_change":"2024-09-10T19:04:39.483398+0000","last_active":"2024-09-10T19:06:30.169133+0000","last_peered":"2024-09-10T19:06:30.169133+0000","last_clean":"2024-09-10T19:06:30.169133+0000","last_became_active":"2024-09-10T18:45:22.065619+0000","last_became_peered":"2024-09-10T18:45:22.065619+0000","last_unstale":"2024-09-10T19:06:30.169133+0000","last_undegraded":"2024-09-10T19:06:30.169133+0000","last_fullsized":"2024-09-10T19:06:30.169133+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:39.483330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.057932+0000","last_clean_scrub_stamp":"2024-09-10T19:04:39.483330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016448439999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"449'11925","reported_seq":18776,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:37.880970+0000","last_change":"2024-09-10T19:05:33.839174+0000","last_active":"2024-09-10T19:06:37.880970+0000","last_peered":"2024-09-10T19:06:37.880970+0000","last_clean":"2024-09-10T19:06:37.880970+0000","last_became_active":"2024-09-10T19:05:33.821585+0000","last_became_peered":"2024-09-10T19:05:33.821585+0000","last_unstale":"2024-09-10T19:06:37.880970+0000","last_undegraded":"2024-09-10T19:06:37.880970+0000","last_fullsized":"2024-09-10T19:06:37.880970+0000","mapping_epoch":429,"log_start":"427'8746","ondisk_log_start":"427'8746","created":404,"last_epoch_clean":430,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3179,"log_dups_size":0,"ondisk_log_size":3179,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8463,"num_write_kb":270752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3369,"num_bytes_recovered":220790784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":525,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.171118+0000","last_change":"2024-09-10T19:02:17.284387+0000","last_active":"2024-09-10T19:06:30.171118+0000","last_peered":"2024-09-10T19:06:30.171118+0000","last_clean":"2024-09-10T19:06:30.171118+0000","last_became_active":"2024-09-10T19:00:42.518824+0000","last_became_peered":"2024-09-10T19:00:42.518824+0000","last_unstale":"2024-09-10T19:06:30.171118+0000","last_undegraded":"2024-09-10T19:06:30.171118+0000","last_fullsized":"2024-09-10T19:06:30.171118+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:17.284305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:17.284305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018984430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"427'25","reported_seq":494,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169058+0000","last_change":"2024-09-10T19:04:28.445705+0000","last_active":"2024-09-10T19:06:30.169058+0000","last_peered":"2024-09-10T19:06:30.169058+0000","last_clean":"2024-09-10T19:06:30.169058+0000","last_became_active":"2024-09-10T18:45:20.061266+0000","last_became_peered":"2024-09-10T18:45:20.061266+0000","last_unstale":"2024-09-10T19:06:30.169058+0000","last_undegraded":"2024-09-10T19:06:30.169058+0000","last_fullsized":"2024-09-10T19:06:30.169058+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T19:04:28.445625+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_clean_scrub_stamp":"2024-09-10T19:04:28.445625+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047732149999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":100,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.166050+0000","last_change":"2024-09-10T19:04:39.088299+0000","last_active":"2024-09-10T19:06:30.166050+0000","last_peered":"2024-09-10T19:06:30.166050+0000","last_clean":"2024-09-10T19:06:30.166050+0000","last_became_active":"2024-09-10T19:02:49.571992+0000","last_became_peered":"2024-09-10T19:02:49.571992+0000","last_unstale":"2024-09-10T19:06:30.166050+0000","last_undegraded":"2024-09-10T19:06:30.166050+0000","last_fullsized":"2024-09-10T19:06:30.166050+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:39.088231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:01.135988+0000","last_clean_scrub_stamp":"2024-09-10T19:04:39.088231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014640530000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":452,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169704+0000","last_change":"2024-09-10T19:03:41.617846+0000","last_active":"2024-09-10T19:06:30.169704+0000","last_peered":"2024-09-10T19:06:30.169704+0000","last_clean":"2024-09-10T19:06:30.169704+0000","last_became_active":"2024-09-10T18:45:24.081002+0000","last_became_peered":"2024-09-10T18:45:24.081002+0000","last_unstale":"2024-09-10T19:06:30.169704+0000","last_undegraded":"2024-09-10T19:06:30.169704+0000","last_fullsized":"2024-09-10T19:06:30.169704+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:41.617790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:03:41.617790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014252819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.14","version":"449'12209","reported_seq":20200,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:37.874919+0000","last_change":"2024-09-10T19:04:55.871256+0000","last_active":"2024-09-10T19:06:37.874919+0000","last_peered":"2024-09-10T19:06:37.874919+0000","last_clean":"2024-09-10T19:06:37.874919+0000","last_became_active":"2024-09-10T19:04:55.852590+0000","last_became_peered":"2024-09-10T19:04:55.852590+0000","last_unstale":"2024-09-10T19:06:37.874919+0000","last_undegraded":"2024-09-10T19:06:37.874919+0000","last_fullsized":"2024-09-10T19:06:37.874919+0000","mapping_epoch":413,"log_start":"427'8992","ondisk_log_start":"427'8992","created":404,"last_epoch_clean":414,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3217,"log_dups_size":0,"ondisk_log_size":3217,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8671,"num_write_kb":277376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3939,"num_bytes_recovered":258146304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":457,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.164460+0000","last_change":"2024-09-10T19:03:30.120786+0000","last_active":"2024-09-10T19:06:30.164460+0000","last_peered":"2024-09-10T19:06:30.164460+0000","last_clean":"2024-09-10T19:06:30.164460+0000","last_became_active":"2024-09-10T18:57:02.452259+0000","last_became_peered":"2024-09-10T18:57:02.452259+0000","last_unstale":"2024-09-10T19:06:30.164460+0000","last_undegraded":"2024-09-10T19:06:30.164460+0000","last_fullsized":"2024-09-10T19:06:30.164460+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:30.120695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:21.501984+0000","last_clean_scrub_stamp":"2024-09-10T19:03:30.120695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034594489999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1b","version":"427'11","reported_seq":474,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169230+0000","last_change":"2024-09-10T19:04:26.520312+0000","last_active":"2024-09-10T19:06:30.169230+0000","last_peered":"2024-09-10T19:06:30.169230+0000","last_clean":"2024-09-10T19:06:30.169230+0000","last_became_active":"2024-09-10T18:45:20.057567+0000","last_became_peered":"2024-09-10T18:45:20.057567+0000","last_unstale":"2024-09-10T19:06:30.169230+0000","last_undegraded":"2024-09-10T19:06:30.169230+0000","last_fullsized":"2024-09-10T19:06:30.169230+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'9","last_scrub_stamp":"2024-09-10T19:04:26.520225+0000","last_deep_scrub":"163'7","last_deep_scrub_stamp":"2024-09-10T18:51:54.261879+0000","last_clean_scrub_stamp":"2024-09-10T19:04:26.520225+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005488153,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":13,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":223,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169636+0000","last_change":"2024-09-10T19:06:11.858817+0000","last_active":"2024-09-10T19:06:30.169636+0000","last_peered":"2024-09-10T19:06:30.169636+0000","last_clean":"2024-09-10T19:06:30.169636+0000","last_became_active":"2024-09-10T18:58:58.503924+0000","last_became_peered":"2024-09-10T18:58:58.503924+0000","last_unstale":"2024-09-10T19:06:30.169636+0000","last_undegraded":"2024-09-10T19:06:30.169636+0000","last_fullsized":"2024-09-10T19:06:30.169636+0000","mapping_epoch":303,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":304,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:11.858728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:36.509971+0000","last_clean_scrub_stamp":"2024-09-10T19:06:11.858728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:19.964860+0000","scrub_duration":0.0083694589999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":130,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.166199+0000","last_change":"2024-09-10T19:03:23.759714+0000","last_active":"2024-09-10T19:06:30.166199+0000","last_peered":"2024-09-10T19:06:30.166199+0000","last_clean":"2024-09-10T19:06:30.166199+0000","last_became_active":"2024-09-10T19:00:42.515840+0000","last_became_peered":"2024-09-10T19:00:42.515840+0000","last_unstale":"2024-09-10T19:06:30.166199+0000","last_undegraded":"2024-09-10T19:06:30.166199+0000","last_fullsized":"2024-09-10T19:06:30.166199+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:23.759624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:03:23.759624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037605970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.15","version":"449'12121","reported_seq":19358,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:37.739897+0000","last_change":"2024-09-10T19:06:01.672213+0000","last_active":"2024-09-10T19:06:37.739897+0000","last_peered":"2024-09-10T19:06:37.739897+0000","last_clean":"2024-09-10T19:06:37.739897+0000","last_became_active":"2024-09-10T19:06:01.618213+0000","last_became_peered":"2024-09-10T19:06:01.618213+0000","last_unstale":"2024-09-10T19:06:37.739897+0000","last_undegraded":"2024-09-10T19:06:37.739897+0000","last_fullsized":"2024-09-10T19:06:37.739897+0000","mapping_epoch":438,"log_start":"423'8460","ondisk_log_start":"423'8460","created":404,"last_epoch_clean":439,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3661,"log_dups_size":0,"ondisk_log_size":3661,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8660,"num_write_kb":274464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2372,"num_bytes_recovered":155385856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":273,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.167374+0000","last_change":"2024-09-10T19:06:11.016240+0000","last_active":"2024-09-10T19:06:30.167374+0000","last_peered":"2024-09-10T19:06:30.167374+0000","last_clean":"2024-09-10T19:06:30.167374+0000","last_became_active":"2024-09-10T18:57:02.440497+0000","last_became_peered":"2024-09-10T18:57:02.440497+0000","last_unstale":"2024-09-10T19:06:30.167374+0000","last_undegraded":"2024-09-10T19:06:30.167374+0000","last_fullsized":"2024-09-10T19:06:30.167374+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:11.016144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:06:11.016144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:29.801511+0000","scrub_duration":0.015619935999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"427'21","reported_seq":519,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169972+0000","last_change":"2024-09-10T19:02:22.113293+0000","last_active":"2024-09-10T19:06:30.169972+0000","last_peered":"2024-09-10T19:06:30.169972+0000","last_clean":"2024-09-10T19:06:30.169972+0000","last_became_active":"2024-09-10T18:55:40.391571+0000","last_became_peered":"2024-09-10T18:55:40.391571+0000","last_unstale":"2024-09-10T19:06:30.169972+0000","last_undegraded":"2024-09-10T19:06:30.169972+0000","last_fullsized":"2024-09-10T19:06:30.169972+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T19:02:22.113140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:22.113140+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0061707400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":493,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.167013+0000","last_change":"2024-09-10T19:02:55.736359+0000","last_active":"2024-09-10T19:06:30.167013+0000","last_peered":"2024-09-10T19:06:30.167013+0000","last_clean":"2024-09-10T19:06:30.167013+0000","last_became_active":"2024-09-10T18:57:02.431585+0000","last_became_peered":"2024-09-10T18:57:02.431585+0000","last_unstale":"2024-09-10T19:06:30.167013+0000","last_undegraded":"2024-09-10T19:06:30.167013+0000","last_fullsized":"2024-09-10T19:06:30.167013+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:55.736308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:02:55.736308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012293950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":159,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:32.188439+0000","last_change":"2024-09-10T19:06:15.668283+0000","last_active":"2024-09-10T19:06:32.188439+0000","last_peered":"2024-09-10T19:06:32.188439+0000","last_clean":"2024-09-10T19:06:32.188439+0000","last_became_active":"2024-09-10T18:59:48.737788+0000","last_became_peered":"2024-09-10T18:59:48.737788+0000","last_unstale":"2024-09-10T19:06:32.188439+0000","last_undegraded":"2024-09-10T19:06:32.188439+0000","last_fullsized":"2024-09-10T19:06:32.188439+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:15.668048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:59.425718+0000","last_clean_scrub_stamp":"2024-09-10T19:06:15.668048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:33.048610+0000","scrub_duration":0.0064859770000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"449'12281","reported_seq":12369,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:37.876937+0000","last_change":"2024-09-10T19:04:36.967513+0000","last_active":"2024-09-10T19:06:37.876937+0000","last_peered":"2024-09-10T19:06:37.876937+0000","last_clean":"2024-09-10T19:06:37.876937+0000","last_became_active":"2024-09-10T19:04:36.964769+0000","last_became_peered":"2024-09-10T19:04:36.964769+0000","last_unstale":"2024-09-10T19:06:37.876937+0000","last_undegraded":"2024-09-10T19:06:37.876937+0000","last_fullsized":"2024-09-10T19:06:37.876937+0000","mapping_epoch":379,"log_start":"427'9021","ondisk_log_start":"427'9021","created":379,"last_epoch_clean":405,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3260,"log_dups_size":0,"ondisk_log_size":3260,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32768,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8732,"num_write_kb":279456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"70'81","reported_seq":134,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.165726+0000","last_change":"2024-09-10T19:03:21.755540+0000","last_active":"2024-09-10T19:06:30.165726+0000","last_peered":"2024-09-10T19:06:30.165726+0000","last_clean":"2024-09-10T19:06:30.165726+0000","last_became_active":"2024-09-10T19:00:42.517072+0000","last_became_peered":"2024-09-10T19:00:42.517072+0000","last_unstale":"2024-09-10T19:06:30.165726+0000","last_undegraded":"2024-09-10T19:06:30.165726+0000","last_fullsized":"2024-09-10T19:06:30.165726+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:03:21.755481+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:03:21.755481+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016110499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":262688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"427'53","reported_seq":572,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169673+0000","last_change":"2024-09-10T19:04:33.387338+0000","last_active":"2024-09-10T19:06:30.169673+0000","last_peered":"2024-09-10T19:06:30.169673+0000","last_clean":"2024-09-10T19:06:30.169673+0000","last_became_active":"2024-09-10T18:47:36.409300+0000","last_became_peered":"2024-09-10T18:47:36.409300+0000","last_unstale":"2024-09-10T19:06:30.169673+0000","last_undegraded":"2024-09-10T19:06:30.169673+0000","last_fullsized":"2024-09-10T19:06:30.169673+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'39","last_scrub_stamp":"2024-09-10T19:04:33.387246+0000","last_deep_scrub":"78'25","last_deep_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_clean_scrub_stamp":"2024-09-10T19:04:33.387246+0000","objects_scrubbed":8,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056627190000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":71,"num_read_kb":61,"num_write":46,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":210,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169382+0000","last_change":"2024-09-10T19:04:38.456757+0000","last_active":"2024-09-10T19:06:30.169382+0000","last_peered":"2024-09-10T19:06:30.169382+0000","last_clean":"2024-09-10T19:06:30.169382+0000","last_became_active":"2024-09-10T18:57:02.437260+0000","last_became_peered":"2024-09-10T18:57:02.437260+0000","last_unstale":"2024-09-10T19:06:30.169382+0000","last_undegraded":"2024-09-10T19:06:30.169382+0000","last_fullsized":"2024-09-10T19:06:30.169382+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:38.456644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:04:38.456644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012890429999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":333,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169552+0000","last_change":"2024-09-10T19:06:09.867580+0000","last_active":"2024-09-10T19:06:30.169552+0000","last_peered":"2024-09-10T19:06:30.169552+0000","last_clean":"2024-09-10T19:06:30.169552+0000","last_became_active":"2024-09-10T19:02:49.594088+0000","last_became_peered":"2024-09-10T19:02:49.594088+0000","last_unstale":"2024-09-10T19:06:30.169552+0000","last_undegraded":"2024-09-10T19:06:30.169552+0000","last_fullsized":"2024-09-10T19:06:30.169552+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:09.867514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:04.512425+0000","last_clean_scrub_stamp":"2024-09-10T19:06:09.867514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:31.816149+0000","scrub_duration":0.0030367480000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":382,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:32.188651+0000","last_change":"2024-09-10T19:06:22.676283+0000","last_active":"2024-09-10T19:06:32.188651+0000","last_peered":"2024-09-10T19:06:32.188651+0000","last_clean":"2024-09-10T19:06:32.188651+0000","last_became_active":"2024-09-10T18:55:40.388154+0000","last_became_peered":"2024-09-10T18:55:40.388154+0000","last_unstale":"2024-09-10T19:06:32.188651+0000","last_undegraded":"2024-09-10T19:06:32.188651+0000","last_fullsized":"2024-09-10T19:06:32.188651+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:22.676194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:06:22.676194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:43.573594+0000","scrub_duration":0.0084613770000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3","version":"449'11882","reported_seq":11970,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:37.847283+0000","last_change":"2024-09-10T19:04:36.969685+0000","last_active":"2024-09-10T19:06:37.847283+0000","last_peered":"2024-09-10T19:06:37.847283+0000","last_clean":"2024-09-10T19:06:37.847283+0000","last_became_active":"2024-09-10T19:04:36.967115+0000","last_became_peered":"2024-09-10T19:04:36.967115+0000","last_unstale":"2024-09-10T19:06:37.847283+0000","last_undegraded":"2024-09-10T19:06:37.847283+0000","last_fullsized":"2024-09-10T19:06:37.847283+0000","mapping_epoch":379,"log_start":"427'8733","ondisk_log_start":"427'8733","created":379,"last_epoch_clean":405,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3149,"log_dups_size":0,"ondisk_log_size":3149,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8445,"num_write_kb":270112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.9","version":"70'81","reported_seq":366,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.173048+0000","last_change":"2024-09-10T19:04:31.516816+0000","last_active":"2024-09-10T19:06:30.173048+0000","last_peered":"2024-09-10T19:06:30.173048+0000","last_clean":"2024-09-10T19:06:30.173048+0000","last_became_active":"2024-09-10T18:57:02.463750+0000","last_became_peered":"2024-09-10T18:57:02.463750+0000","last_unstale":"2024-09-10T19:06:30.173048+0000","last_undegraded":"2024-09-10T19:06:30.173048+0000","last_fullsized":"2024-09-10T19:06:30.173048+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:04:31.516600+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:04:31.516600+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011586271,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"427'35","reported_seq":506,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169661+0000","last_change":"2024-09-10T19:03:34.532084+0000","last_active":"2024-09-10T19:06:30.169661+0000","last_peered":"2024-09-10T19:06:30.169661+0000","last_clean":"2024-09-10T19:06:30.169661+0000","last_became_active":"2024-09-10T18:45:20.053428+0000","last_became_peered":"2024-09-10T18:45:20.053428+0000","last_unstale":"2024-09-10T19:06:30.169661+0000","last_undegraded":"2024-09-10T19:06:30.169661+0000","last_fullsized":"2024-09-10T19:06:30.169661+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'25","last_scrub_stamp":"2024-09-10T19:03:34.532019+0000","last_deep_scrub":"116'15","last_deep_scrub_stamp":"2024-09-10T18:49:12.374527+0000","last_clean_scrub_stamp":"2024-09-10T19:03:34.532019+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020774220000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":372,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.170762+0000","last_change":"2024-09-10T19:06:17.129138+0000","last_active":"2024-09-10T19:06:30.170762+0000","last_peered":"2024-09-10T19:06:30.170762+0000","last_clean":"2024-09-10T19:06:30.170762+0000","last_became_active":"2024-09-10T19:02:49.571995+0000","last_became_peered":"2024-09-10T19:02:49.571995+0000","last_unstale":"2024-09-10T19:06:30.170762+0000","last_undegraded":"2024-09-10T19:06:30.170762+0000","last_fullsized":"2024-09-10T19:06:30.170762+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:17.129057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:06:17.129057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:42.800560+0000","scrub_duration":0.0074624499999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"273'13","reported_seq":587,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:36.328755+0000","last_change":"2024-09-10T19:06:23.008122+0000","last_active":"2024-09-10T19:06:36.328755+0000","last_peered":"2024-09-10T19:06:36.328755+0000","last_clean":"2024-09-10T19:06:36.328755+0000","last_became_active":"2024-09-10T18:57:02.448258+0000","last_became_peered":"2024-09-10T18:57:02.448258+0000","last_unstale":"2024-09-10T19:06:36.328755+0000","last_undegraded":"2024-09-10T19:06:36.328755+0000","last_fullsized":"2024-09-10T19:06:36.328755+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'13","last_scrub_stamp":"2024-09-10T19:06:23.008037+0000","last_deep_scrub":"273'13","last_deep_scrub_stamp":"2024-09-10T19:06:23.008037+0000","last_clean_scrub_stamp":"2024-09-10T19:06:23.008037+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:42.109075+0000","scrub_duration":0.013286312,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":502,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169054+0000","last_change":"2024-09-10T19:06:02.941293+0000","last_active":"2024-09-10T19:06:30.169054+0000","last_peered":"2024-09-10T19:06:30.169054+0000","last_clean":"2024-09-10T19:06:30.169054+0000","last_became_active":"2024-09-10T19:00:42.515523+0000","last_became_peered":"2024-09-10T19:00:42.515523+0000","last_unstale":"2024-09-10T19:06:30.169054+0000","last_undegraded":"2024-09-10T19:06:30.169054+0000","last_fullsized":"2024-09-10T19:06:30.169054+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:02.941202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:32.043666+0000","last_clean_scrub_stamp":"2024-09-10T19:06:02.941202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:07.258125+0000","scrub_duration":0.0062417180000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.0","version":"449'11906","reported_seq":11994,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:37.868795+0000","last_change":"2024-09-10T19:04:36.990388+0000","last_active":"2024-09-10T19:06:37.868795+0000","last_peered":"2024-09-10T19:06:37.868795+0000","last_clean":"2024-09-10T19:06:37.868795+0000","last_became_active":"2024-09-10T19:04:36.984829+0000","last_became_peered":"2024-09-10T19:04:36.984829+0000","last_unstale":"2024-09-10T19:06:37.868795+0000","last_undegraded":"2024-09-10T19:06:37.868795+0000","last_fullsized":"2024-09-10T19:06:37.868795+0000","mapping_epoch":379,"log_start":"424'8586","ondisk_log_start":"424'8586","created":379,"last_epoch_clean":405,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3320,"log_dups_size":0,"ondisk_log_size":3320,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32768,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8498,"num_write_kb":271968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.a","version":"70'81","reported_seq":381,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.167084+0000","last_change":"2024-09-10T19:02:23.723122+0000","last_active":"2024-09-10T19:06:30.167084+0000","last_peered":"2024-09-10T19:06:30.167084+0000","last_clean":"2024-09-10T19:06:30.167084+0000","last_became_active":"2024-09-10T18:50:08.398345+0000","last_became_peered":"2024-09-10T18:50:08.398345+0000","last_unstale":"2024-09-10T19:06:30.167084+0000","last_undegraded":"2024-09-10T19:06:30.167084+0000","last_fullsized":"2024-09-10T19:06:30.167084+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:23.723036+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:55:37.921519+0000","last_clean_scrub_stamp":"2024-09-10T19:02:23.723036+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0054938950000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"427'46","reported_seq":506,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169717+0000","last_change":"2024-09-10T19:04:33.010380+0000","last_active":"2024-09-10T19:06:30.169717+0000","last_peered":"2024-09-10T19:06:30.169717+0000","last_clean":"2024-09-10T19:06:30.169717+0000","last_became_active":"2024-09-10T18:46:56.742946+0000","last_became_peered":"2024-09-10T18:46:56.742946+0000","last_unstale":"2024-09-10T19:06:30.169717+0000","last_undegraded":"2024-09-10T19:06:30.169717+0000","last_fullsized":"2024-09-10T19:06:30.169717+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'34","last_scrub_stamp":"2024-09-10T19:04:33.010321+0000","last_deep_scrub":"141'22","last_deep_scrub_stamp":"2024-09-10T18:50:57.119152+0000","last_clean_scrub_stamp":"2024-09-10T19:04:33.010321+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0031820009999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":375,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.172888+0000","last_change":"2024-09-10T19:04:32.480047+0000","last_active":"2024-09-10T19:06:30.172888+0000","last_peered":"2024-09-10T19:06:30.172888+0000","last_clean":"2024-09-10T19:06:30.172888+0000","last_became_active":"2024-09-10T19:02:49.563535+0000","last_became_peered":"2024-09-10T19:02:49.563535+0000","last_unstale":"2024-09-10T19:06:30.172888+0000","last_undegraded":"2024-09-10T19:06:30.172888+0000","last_fullsized":"2024-09-10T19:06:30.172888+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:32.479945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_clean_scrub_stamp":"2024-09-10T19:04:32.479945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060807650000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"242'10","reported_seq":738,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:36.330881+0000","last_change":"2024-09-10T19:02:53.030726+0000","last_active":"2024-09-10T19:06:36.330881+0000","last_peered":"2024-09-10T19:06:36.330881+0000","last_clean":"2024-09-10T19:06:36.330881+0000","last_became_active":"2024-09-10T18:55:40.397246+0000","last_became_peered":"2024-09-10T18:55:40.397246+0000","last_unstale":"2024-09-10T19:06:36.330881+0000","last_undegraded":"2024-09-10T19:06:36.330881+0000","last_fullsized":"2024-09-10T19:06:36.330881+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"242'10","last_scrub_stamp":"2024-09-10T19:02:53.030666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:02:53.030666+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015331489999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":481,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.167876+0000","last_change":"2024-09-10T19:02:48.670915+0000","last_active":"2024-09-10T19:06:30.167876+0000","last_peered":"2024-09-10T19:06:30.167876+0000","last_clean":"2024-09-10T19:06:30.167876+0000","last_became_active":"2024-09-10T18:55:40.392003+0000","last_became_peered":"2024-09-10T18:55:40.392003+0000","last_unstale":"2024-09-10T19:06:30.167876+0000","last_undegraded":"2024-09-10T19:06:30.167876+0000","last_fullsized":"2024-09-10T19:06:30.167876+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:48.670862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:54.218602+0000","last_clean_scrub_stamp":"2024-09-10T19:02:48.670862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00108365,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1","version":"449'11878","reported_seq":11967,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:37.860463+0000","last_change":"2024-09-10T19:04:36.972439+0000","last_active":"2024-09-10T19:06:37.860463+0000","last_peered":"2024-09-10T19:06:37.860463+0000","last_clean":"2024-09-10T19:06:37.860463+0000","last_became_active":"2024-09-10T19:04:36.969739+0000","last_became_peered":"2024-09-10T19:04:36.969739+0000","last_unstale":"2024-09-10T19:06:37.860463+0000","last_undegraded":"2024-09-10T19:06:37.860463+0000","last_fullsized":"2024-09-10T19:06:37.860463+0000","mapping_epoch":379,"log_start":"427'8674","ondisk_log_start":"427'8674","created":379,"last_epoch_clean":405,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3204,"log_dups_size":0,"ondisk_log_size":3204,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1802240,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8469,"num_write_kb":271904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.b","version":"70'81","reported_seq":176,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:32.189053+0000","last_change":"2024-09-10T19:06:11.695308+0000","last_active":"2024-09-10T19:06:32.189053+0000","last_peered":"2024-09-10T19:06:32.189053+0000","last_clean":"2024-09-10T19:06:32.189053+0000","last_became_active":"2024-09-10T19:02:49.572057+0000","last_became_peered":"2024-09-10T19:02:49.572057+0000","last_unstale":"2024-09-10T19:06:32.189053+0000","last_undegraded":"2024-09-10T19:06:32.189053+0000","last_fullsized":"2024-09-10T19:06:32.189053+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:06:11.695225+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T19:02:17.370627+0000","last_clean_scrub_stamp":"2024-09-10T19:06:11.695225+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:24.298666+0000","scrub_duration":0.003975236,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"427'32","reported_seq":580,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.171053+0000","last_change":"2024-09-10T19:03:42.660938+0000","last_active":"2024-09-10T19:06:30.171053+0000","last_peered":"2024-09-10T19:06:30.171053+0000","last_clean":"2024-09-10T19:06:30.171053+0000","last_became_active":"2024-09-10T18:47:57.106935+0000","last_became_peered":"2024-09-10T18:47:57.106935+0000","last_unstale":"2024-09-10T19:06:30.171053+0000","last_undegraded":"2024-09-10T19:06:30.171053+0000","last_fullsized":"2024-09-10T19:06:30.171053+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T19:03:42.660792+0000","last_deep_scrub":"224'16","last_deep_scrub_stamp":"2024-09-10T18:54:49.296957+0000","last_clean_scrub_stamp":"2024-09-10T19:03:42.660792+0000","objects_scrubbed":5,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053846639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":465,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:32.188751+0000","last_change":"2024-09-10T19:06:01.706867+0000","last_active":"2024-09-10T19:06:32.188751+0000","last_peered":"2024-09-10T19:06:32.188751+0000","last_clean":"2024-09-10T19:06:32.188751+0000","last_became_active":"2024-09-10T18:57:02.443816+0000","last_became_peered":"2024-09-10T18:57:02.443816+0000","last_unstale":"2024-09-10T19:06:32.188751+0000","last_undegraded":"2024-09-10T19:06:32.188751+0000","last_fullsized":"2024-09-10T19:06:32.188751+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:01.706783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:06:01.706783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:28.681911+0000","scrub_duration":0.0097313599999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":479,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169594+0000","last_change":"2024-09-10T19:06:02.997863+0000","last_active":"2024-09-10T19:06:30.169594+0000","last_peered":"2024-09-10T19:06:30.169594+0000","last_clean":"2024-09-10T19:06:30.169594+0000","last_became_active":"2024-09-10T18:45:22.068918+0000","last_became_peered":"2024-09-10T18:45:22.068918+0000","last_unstale":"2024-09-10T19:06:30.169594+0000","last_undegraded":"2024-09-10T19:06:30.169594+0000","last_fullsized":"2024-09-10T19:06:30.169594+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:02.997745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_clean_scrub_stamp":"2024-09-10T19:06:02.997745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:10.273645+0000","scrub_duration":0.0015124940000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":452,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.170033+0000","last_change":"2024-09-10T19:03:24.593809+0000","last_active":"2024-09-10T19:06:30.170033+0000","last_peered":"2024-09-10T19:06:30.170033+0000","last_clean":"2024-09-10T19:06:30.170033+0000","last_became_active":"2024-09-10T18:45:24.264538+0000","last_became_peered":"2024-09-10T18:45:24.264538+0000","last_unstale":"2024-09-10T19:06:30.170033+0000","last_undegraded":"2024-09-10T19:06:30.170033+0000","last_fullsized":"2024-09-10T19:06:30.170033+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:24.593754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:51.398981+0000","last_clean_scrub_stamp":"2024-09-10T19:03:24.593754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001434999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.e","version":"449'17102","reported_seq":17180,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:37.889246+0000","last_change":"2024-09-10T19:03:16.044645+0000","last_active":"2024-09-10T19:06:37.889246+0000","last_peered":"2024-09-10T19:06:37.889246+0000","last_clean":"2024-09-10T19:06:37.889246+0000","last_became_active":"2024-09-10T19:03:16.044187+0000","last_became_peered":"2024-09-10T19:03:16.044187+0000","last_unstale":"2024-09-10T19:06:37.889246+0000","last_undegraded":"2024-09-10T19:06:37.889246+0000","last_fullsized":"2024-09-10T19:06:37.889246+0000","mapping_epoch":379,"log_start":"437'13483","ondisk_log_start":"437'13483","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3619,"log_dups_size":0,"ondisk_log_size":3619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17102,"num_write_kb":547264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"70'81","reported_seq":410,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.170242+0000","last_change":"2024-09-10T19:03:30.624986+0000","last_active":"2024-09-10T19:06:30.170242+0000","last_peered":"2024-09-10T19:06:30.170242+0000","last_clean":"2024-09-10T19:06:30.170242+0000","last_became_active":"2024-09-10T19:00:42.519742+0000","last_became_peered":"2024-09-10T19:00:42.519742+0000","last_unstale":"2024-09-10T19:06:30.170242+0000","last_undegraded":"2024-09-10T19:06:30.170242+0000","last_fullsized":"2024-09-10T19:06:30.170242+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:03:30.624868+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:03:30.624868+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0036172140000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"427'49","reported_seq":526,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.168654+0000","last_change":"2024-09-10T19:03:29.581426+0000","last_active":"2024-09-10T19:06:30.168654+0000","last_peered":"2024-09-10T19:06:30.168654+0000","last_clean":"2024-09-10T19:06:30.168654+0000","last_became_active":"2024-09-10T18:45:20.052243+0000","last_became_peered":"2024-09-10T18:45:20.052243+0000","last_unstale":"2024-09-10T19:06:30.168654+0000","last_undegraded":"2024-09-10T19:06:30.168654+0000","last_fullsized":"2024-09-10T19:06:30.168654+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'35","last_scrub_stamp":"2024-09-10T19:03:29.581339+0000","last_deep_scrub":"143'21","last_deep_scrub_stamp":"2024-09-10T18:51:02.540216+0000","last_clean_scrub_stamp":"2024-09-10T19:03:29.581339+0000","objects_scrubbed":7,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0042812099999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":63,"num_read_kb":56,"num_write":42,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":501,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.162308+0000","last_change":"2024-09-10T19:04:29.252531+0000","last_active":"2024-09-10T19:06:30.162308+0000","last_peered":"2024-09-10T19:06:30.162308+0000","last_clean":"2024-09-10T19:06:30.162308+0000","last_became_active":"2024-09-10T18:57:02.453404+0000","last_became_peered":"2024-09-10T18:57:02.453404+0000","last_unstale":"2024-09-10T19:06:30.162308+0000","last_undegraded":"2024-09-10T19:06:30.162308+0000","last_fullsized":"2024-09-10T19:06:30.162308+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:29.252456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_clean_scrub_stamp":"2024-09-10T19:04:29.252456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0036547900000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":729,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:36.328754+0000","last_change":"2024-09-10T19:06:07.959700+0000","last_active":"2024-09-10T19:06:36.328754+0000","last_peered":"2024-09-10T19:06:36.328754+0000","last_clean":"2024-09-10T19:06:36.328754+0000","last_became_active":"2024-09-10T18:45:22.074540+0000","last_became_peered":"2024-09-10T18:45:22.074540+0000","last_unstale":"2024-09-10T19:06:36.328754+0000","last_undegraded":"2024-09-10T19:06:36.328754+0000","last_fullsized":"2024-09-10T19:06:36.328754+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T19:06:07.959488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:06:07.959488+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:21.540597+0000","scrub_duration":0.01085156,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":268,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.162095+0000","last_change":"2024-09-10T19:04:37.416533+0000","last_active":"2024-09-10T19:06:30.162095+0000","last_peered":"2024-09-10T19:06:30.162095+0000","last_clean":"2024-09-10T19:06:30.162095+0000","last_became_active":"2024-09-10T18:57:02.437100+0000","last_became_peered":"2024-09-10T18:57:02.437100+0000","last_unstale":"2024-09-10T19:06:30.162095+0000","last_undegraded":"2024-09-10T19:06:30.162095+0000","last_fullsized":"2024-09-10T19:06:30.162095+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:37.416450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:04:37.416450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0094483369999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"449'17300","reported_seq":17379,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:37.878943+0000","last_change":"2024-09-10T19:03:16.046832+0000","last_active":"2024-09-10T19:06:37.878943+0000","last_peered":"2024-09-10T19:06:37.878943+0000","last_clean":"2024-09-10T19:06:37.878943+0000","last_became_active":"2024-09-10T19:03:16.046538+0000","last_became_peered":"2024-09-10T19:03:16.046538+0000","last_unstale":"2024-09-10T19:06:37.878943+0000","last_undegraded":"2024-09-10T19:06:37.878943+0000","last_fullsized":"2024-09-10T19:06:37.878943+0000","mapping_epoch":379,"log_start":"437'13666","ondisk_log_start":"437'13666","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3634,"log_dups_size":0,"ondisk_log_size":3634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17300,"num_write_kb":553600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"70'81","reported_seq":376,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.170061+0000","last_change":"2024-09-10T19:02:58.888049+0000","last_active":"2024-09-10T19:06:30.170061+0000","last_peered":"2024-09-10T19:06:30.170061+0000","last_clean":"2024-09-10T19:06:30.170061+0000","last_became_active":"2024-09-10T18:52:14.584798+0000","last_became_peered":"2024-09-10T18:52:14.584798+0000","last_unstale":"2024-09-10T19:06:30.170061+0000","last_undegraded":"2024-09-10T19:06:30.170061+0000","last_fullsized":"2024-09-10T19:06:30.170061+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:58.887944+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:56:31.481303+0000","last_clean_scrub_stamp":"2024-09-10T19:02:58.887944+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016190950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"427'42","reported_seq":640,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.168489+0000","last_change":"2024-09-10T19:02:48.361438+0000","last_active":"2024-09-10T19:06:30.168489+0000","last_peered":"2024-09-10T19:06:30.168489+0000","last_clean":"2024-09-10T19:06:30.168489+0000","last_became_active":"2024-09-10T18:59:48.742377+0000","last_became_peered":"2024-09-10T18:59:48.742377+0000","last_unstale":"2024-09-10T19:06:30.168489+0000","last_undegraded":"2024-09-10T19:06:30.168489+0000","last_fullsized":"2024-09-10T19:06:30.168489+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'30","last_scrub_stamp":"2024-09-10T19:02:48.361337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:48.361337+0000","objects_scrubbed":8,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011213163,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":70,"num_read_kb":58,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":109,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.162517+0000","last_change":"2024-09-10T19:06:27.610141+0000","last_active":"2024-09-10T19:06:30.162517+0000","last_peered":"2024-09-10T19:06:30.162517+0000","last_clean":"2024-09-10T19:06:30.162517+0000","last_became_active":"2024-09-10T19:02:49.578870+0000","last_became_peered":"2024-09-10T19:02:49.578870+0000","last_unstale":"2024-09-10T19:06:30.162517+0000","last_undegraded":"2024-09-10T19:06:30.162517+0000","last_fullsized":"2024-09-10T19:06:30.162517+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:27.610058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:46.142129+0000","last_clean_scrub_stamp":"2024-09-10T19:06:27.610058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:36.151562+0000","scrub_duration":0.0042716869999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":474,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.168429+0000","last_change":"2024-09-10T19:04:36.430586+0000","last_active":"2024-09-10T19:06:30.168429+0000","last_peered":"2024-09-10T19:06:30.168429+0000","last_clean":"2024-09-10T19:06:30.168429+0000","last_became_active":"2024-09-10T18:57:02.448126+0000","last_became_peered":"2024-09-10T18:57:02.448126+0000","last_unstale":"2024-09-10T19:06:30.168429+0000","last_undegraded":"2024-09-10T19:06:30.168429+0000","last_fullsized":"2024-09-10T19:06:30.168429+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:36.430526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:57.903311+0000","last_clean_scrub_stamp":"2024-09-10T19:04:36.430526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00133559,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":462,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169899+0000","last_change":"2024-09-10T19:03:36.541808+0000","last_active":"2024-09-10T19:06:30.169899+0000","last_peered":"2024-09-10T19:06:30.169899+0000","last_clean":"2024-09-10T19:06:30.169899+0000","last_became_active":"2024-09-10T18:47:36.410836+0000","last_became_peered":"2024-09-10T18:47:36.410836+0000","last_unstale":"2024-09-10T19:06:30.169899+0000","last_undegraded":"2024-09-10T19:06:30.169899+0000","last_fullsized":"2024-09-10T19:06:30.169899+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:36.541729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_clean_scrub_stamp":"2024-09-10T19:03:36.541729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001932762,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"449'17326","reported_seq":17404,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:37.880561+0000","last_change":"2024-09-10T19:03:16.046878+0000","last_active":"2024-09-10T19:06:37.880561+0000","last_peered":"2024-09-10T19:06:37.880561+0000","last_clean":"2024-09-10T19:06:37.880561+0000","last_became_active":"2024-09-10T19:03:16.046484+0000","last_became_peered":"2024-09-10T19:03:16.046484+0000","last_unstale":"2024-09-10T19:06:37.880561+0000","last_undegraded":"2024-09-10T19:06:37.880561+0000","last_fullsized":"2024-09-10T19:06:37.880561+0000","mapping_epoch":379,"log_start":"439'13680","ondisk_log_start":"439'13680","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3646,"log_dups_size":0,"ondisk_log_size":3646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17326,"num_write_kb":554432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"70'81","reported_seq":484,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.162391+0000","last_change":"2024-09-10T19:06:25.532401+0000","last_active":"2024-09-10T19:06:30.162391+0000","last_peered":"2024-09-10T19:06:30.162391+0000","last_clean":"2024-09-10T19:06:30.162391+0000","last_became_active":"2024-09-10T18:55:40.396828+0000","last_became_peered":"2024-09-10T18:55:40.396828+0000","last_unstale":"2024-09-10T19:06:30.162391+0000","last_undegraded":"2024-09-10T19:06:30.162391+0000","last_fullsized":"2024-09-10T19:06:30.162391+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:06:25.532348+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T19:06:25.532348+0000","last_clean_scrub_stamp":"2024-09-10T19:06:25.532348+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:45.622134+0000","scrub_duration":0.0053591250000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"427'39","reported_seq":542,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:32.189324+0000","last_change":"2024-09-10T19:06:12.729237+0000","last_active":"2024-09-10T19:06:32.189324+0000","last_peered":"2024-09-10T19:06:32.189324+0000","last_clean":"2024-09-10T19:06:32.189324+0000","last_became_active":"2024-09-10T18:57:02.442862+0000","last_became_peered":"2024-09-10T18:57:02.442862+0000","last_unstale":"2024-09-10T19:06:32.189324+0000","last_undegraded":"2024-09-10T19:06:32.189324+0000","last_fullsized":"2024-09-10T19:06:32.189324+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'39","last_scrub_stamp":"2024-09-10T19:06:12.729127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:06:12.729127+0000","objects_scrubbed":8,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:35.794829+0000","scrub_duration":0.005856895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":68,"num_read_kb":54,"num_write":42,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":465,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:32.188347+0000","last_change":"2024-09-10T19:06:29.659585+0000","last_active":"2024-09-10T19:06:32.188347+0000","last_peered":"2024-09-10T19:06:32.188347+0000","last_clean":"2024-09-10T19:06:32.188347+0000","last_became_active":"2024-09-10T18:57:02.443522+0000","last_became_peered":"2024-09-10T18:57:02.443522+0000","last_unstale":"2024-09-10T19:06:32.188347+0000","last_undegraded":"2024-09-10T19:06:32.188347+0000","last_fullsized":"2024-09-10T19:06:32.188347+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:29.659530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:06:29.659530+0000","last_clean_scrub_stamp":"2024-09-10T19:06:29.659530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:49.158272+0000","scrub_duration":0.0029405880000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":497,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.163627+0000","last_change":"2024-09-10T19:06:07.512604+0000","last_active":"2024-09-10T19:06:30.163627+0000","last_peered":"2024-09-10T19:06:30.163627+0000","last_clean":"2024-09-10T19:06:30.163627+0000","last_became_active":"2024-09-10T18:55:40.387857+0000","last_became_peered":"2024-09-10T18:55:40.387857+0000","last_unstale":"2024-09-10T19:06:30.163627+0000","last_undegraded":"2024-09-10T19:06:30.163627+0000","last_fullsized":"2024-09-10T19:06:30.163627+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:07.512453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:06:07.512453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:09.727452+0000","scrub_duration":0.0057807240000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":377,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.164728+0000","last_change":"2024-09-10T19:06:29.568836+0000","last_active":"2024-09-10T19:06:30.164728+0000","last_peered":"2024-09-10T19:06:30.164728+0000","last_clean":"2024-09-10T19:06:30.164728+0000","last_became_active":"2024-09-10T19:02:49.561359+0000","last_became_peered":"2024-09-10T19:02:49.561359+0000","last_unstale":"2024-09-10T19:06:30.164728+0000","last_undegraded":"2024-09-10T19:06:30.164728+0000","last_fullsized":"2024-09-10T19:06:30.164728+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:29.568754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:49:18.281230+0000","last_clean_scrub_stamp":"2024-09-10T19:06:29.568754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:38.060685+0000","scrub_duration":0.002692238,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"449'17308","reported_seq":17387,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:37.887883+0000","last_change":"2024-09-10T19:03:16.044601+0000","last_active":"2024-09-10T19:06:37.887883+0000","last_peered":"2024-09-10T19:06:37.887883+0000","last_clean":"2024-09-10T19:06:37.887883+0000","last_became_active":"2024-09-10T19:03:16.044213+0000","last_became_peered":"2024-09-10T19:03:16.044213+0000","last_unstale":"2024-09-10T19:06:37.887883+0000","last_undegraded":"2024-09-10T19:06:37.887883+0000","last_fullsized":"2024-09-10T19:06:37.887883+0000","mapping_epoch":379,"log_start":"439'13677","ondisk_log_start":"439'13677","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3631,"log_dups_size":0,"ondisk_log_size":3631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17308,"num_write_kb":553793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"70'81","reported_seq":477,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.163315+0000","last_change":"2024-09-10T19:03:34.083497+0000","last_active":"2024-09-10T19:06:30.163315+0000","last_peered":"2024-09-10T19:06:30.163315+0000","last_clean":"2024-09-10T19:06:30.163315+0000","last_became_active":"2024-09-10T18:57:02.452659+0000","last_became_peered":"2024-09-10T18:57:02.452659+0000","last_unstale":"2024-09-10T19:06:30.163315+0000","last_undegraded":"2024-09-10T19:06:30.163315+0000","last_fullsized":"2024-09-10T19:06:30.163315+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:03:34.083013+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:03:34.083013+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023839249999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"427'35","reported_seq":550,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.166351+0000","last_change":"2024-09-10T19:03:52.017990+0000","last_active":"2024-09-10T19:06:30.166351+0000","last_peered":"2024-09-10T19:06:30.166351+0000","last_clean":"2024-09-10T19:06:30.166351+0000","last_became_active":"2024-09-10T19:02:49.579401+0000","last_became_peered":"2024-09-10T19:02:49.579401+0000","last_unstale":"2024-09-10T19:06:30.166351+0000","last_undegraded":"2024-09-10T19:06:30.166351+0000","last_fullsized":"2024-09-10T19:06:30.166351+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'25","last_scrub_stamp":"2024-09-10T19:03:52.017902+0000","last_deep_scrub":"83'15","last_deep_scrub_stamp":"2024-09-10T18:46:58.436473+0000","last_clean_scrub_stamp":"2024-09-10T19:03:52.017902+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024815190000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":391,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:32.192342+0000","last_change":"2024-09-10T19:06:02.753654+0000","last_active":"2024-09-10T19:06:32.192342+0000","last_peered":"2024-09-10T19:06:32.192342+0000","last_clean":"2024-09-10T19:06:32.192342+0000","last_became_active":"2024-09-10T19:02:49.561340+0000","last_became_peered":"2024-09-10T19:02:49.561340+0000","last_unstale":"2024-09-10T19:06:32.192342+0000","last_undegraded":"2024-09-10T19:06:32.192342+0000","last_fullsized":"2024-09-10T19:06:32.192342+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:02.753555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:05.140027+0000","last_clean_scrub_stamp":"2024-09-10T19:06:02.753555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:30.070995+0000","scrub_duration":0.0082683540000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"339'13","reported_seq":692,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:41.328698+0000","last_change":"2024-09-10T19:03:29.753145+0000","last_active":"2024-09-10T19:06:41.328698+0000","last_peered":"2024-09-10T19:06:41.328698+0000","last_clean":"2024-09-10T19:06:41.328698+0000","last_became_active":"2024-09-10T19:00:42.520360+0000","last_became_peered":"2024-09-10T19:00:42.520360+0000","last_unstale":"2024-09-10T19:06:41.328698+0000","last_undegraded":"2024-09-10T19:06:41.328698+0000","last_fullsized":"2024-09-10T19:06:41.328698+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"339'13","last_scrub_stamp":"2024-09-10T19:03:29.753059+0000","last_deep_scrub":"258'12","last_deep_scrub_stamp":"2024-09-10T18:56:21.098770+0000","last_clean_scrub_stamp":"2024-09-10T19:03:29.753059+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002665485,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":451,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:32.189025+0000","last_change":"2024-09-10T19:04:39.703670+0000","last_active":"2024-09-10T19:06:32.189025+0000","last_peered":"2024-09-10T19:06:32.189025+0000","last_clean":"2024-09-10T19:06:32.189025+0000","last_became_active":"2024-09-10T19:00:42.515144+0000","last_became_peered":"2024-09-10T19:00:42.515144+0000","last_unstale":"2024-09-10T19:06:32.189025+0000","last_undegraded":"2024-09-10T19:06:32.189025+0000","last_fullsized":"2024-09-10T19:06:32.189025+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:39.703610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:04:39.703610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022926890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"449'17140","reported_seq":17217,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:37.877081+0000","last_change":"2024-09-10T19:03:16.048706+0000","last_active":"2024-09-10T19:06:37.877081+0000","last_peered":"2024-09-10T19:06:37.877081+0000","last_clean":"2024-09-10T19:06:37.877081+0000","last_became_active":"2024-09-10T19:03:16.048344+0000","last_became_peered":"2024-09-10T19:03:16.048344+0000","last_unstale":"2024-09-10T19:06:37.877081+0000","last_undegraded":"2024-09-10T19:06:37.877081+0000","last_fullsized":"2024-09-10T19:06:37.877081+0000","mapping_epoch":379,"log_start":"439'13860","ondisk_log_start":"439'13860","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3280,"log_dups_size":0,"ondisk_log_size":3280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17140,"num_write_kb":548480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"70'81","reported_seq":408,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.165816+0000","last_change":"2024-09-10T19:04:24.971869+0000","last_active":"2024-09-10T19:06:30.165816+0000","last_peered":"2024-09-10T19:06:30.165816+0000","last_clean":"2024-09-10T19:06:30.165816+0000","last_became_active":"2024-09-10T18:55:40.392432+0000","last_became_peered":"2024-09-10T18:55:40.392432+0000","last_unstale":"2024-09-10T19:06:30.165816+0000","last_undegraded":"2024-09-10T19:06:30.165816+0000","last_fullsized":"2024-09-10T19:06:30.165816+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:04:24.971752+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:04:24.971752+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034804469999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"427'54","reported_seq":592,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:36.329163+0000","last_change":"2024-09-10T19:02:49.563856+0000","last_active":"2024-09-10T19:06:36.329163+0000","last_peered":"2024-09-10T19:06:36.329163+0000","last_clean":"2024-09-10T19:06:36.329163+0000","last_became_active":"2024-09-10T19:02:49.563402+0000","last_became_peered":"2024-09-10T19:02:49.563402+0000","last_unstale":"2024-09-10T19:06:36.329163+0000","last_undegraded":"2024-09-10T19:06:36.329163+0000","last_fullsized":"2024-09-10T19:06:36.329163+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'41","last_scrub_stamp":"2024-09-10T19:02:12.270610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:12.270610+0000","objects_scrubbed":10,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012827955,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":83,"num_read_kb":68,"num_write":50,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":483,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.166429+0000","last_change":"2024-09-10T19:06:18.033122+0000","last_active":"2024-09-10T19:06:30.166429+0000","last_peered":"2024-09-10T19:06:30.166429+0000","last_clean":"2024-09-10T19:06:30.166429+0000","last_became_active":"2024-09-10T18:45:17.964494+0000","last_became_peered":"2024-09-10T18:45:17.964494+0000","last_unstale":"2024-09-10T19:06:30.166429+0000","last_undegraded":"2024-09-10T19:06:30.166429+0000","last_fullsized":"2024-09-10T19:06:30.166429+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:18.033032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:12.982599+0000","last_clean_scrub_stamp":"2024-09-10T19:06:18.033032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:38.662714+0000","scrub_duration":0.0025216739999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":425,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.162654+0000","last_change":"2024-09-10T19:04:32.317459+0000","last_active":"2024-09-10T19:06:30.162654+0000","last_peered":"2024-09-10T19:06:30.162654+0000","last_clean":"2024-09-10T19:06:30.162654+0000","last_became_active":"2024-09-10T18:47:57.107478+0000","last_became_peered":"2024-09-10T18:47:57.107478+0000","last_unstale":"2024-09-10T19:06:30.162654+0000","last_undegraded":"2024-09-10T19:06:30.162654+0000","last_fullsized":"2024-09-10T19:06:30.162654+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:32.317339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:04:32.317339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0077591379999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":452,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.170859+0000","last_change":"2024-09-10T19:02:09.267386+0000","last_active":"2024-09-10T19:06:30.170859+0000","last_peered":"2024-09-10T19:06:30.170859+0000","last_clean":"2024-09-10T19:06:30.170859+0000","last_became_active":"2024-09-10T18:57:02.460594+0000","last_became_peered":"2024-09-10T18:57:02.460594+0000","last_unstale":"2024-09-10T19:06:30.170859+0000","last_undegraded":"2024-09-10T19:06:30.170859+0000","last_fullsized":"2024-09-10T19:06:30.170859+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:09.267253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:22.891084+0000","last_clean_scrub_stamp":"2024-09-10T19:02:09.267253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0067979290000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"449'12103","reported_seq":12190,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:37.859544+0000","last_change":"2024-09-10T19:04:36.990471+0000","last_active":"2024-09-10T19:06:37.859544+0000","last_peered":"2024-09-10T19:06:37.859544+0000","last_clean":"2024-09-10T19:06:37.859544+0000","last_became_active":"2024-09-10T19:04:36.984736+0000","last_became_peered":"2024-09-10T19:04:36.984736+0000","last_unstale":"2024-09-10T19:06:37.859544+0000","last_undegraded":"2024-09-10T19:06:37.859544+0000","last_fullsized":"2024-09-10T19:06:37.859544+0000","mapping_epoch":379,"log_start":"427'8841","ondisk_log_start":"427'8841","created":379,"last_epoch_clean":405,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3262,"log_dups_size":0,"ondisk_log_size":3262,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3047424,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8594,"num_write_kb":276512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"70'81","reported_seq":287,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:32.188864+0000","last_change":"2024-09-10T19:06:16.693268+0000","last_active":"2024-09-10T19:06:32.188864+0000","last_peered":"2024-09-10T19:06:32.188864+0000","last_clean":"2024-09-10T19:06:32.188864+0000","last_became_active":"2024-09-10T18:57:02.437944+0000","last_became_peered":"2024-09-10T18:57:02.437944+0000","last_unstale":"2024-09-10T19:06:32.188864+0000","last_undegraded":"2024-09-10T19:06:32.188864+0000","last_fullsized":"2024-09-10T19:06:32.188864+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:06:16.692486+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:06:16.692486+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:31.831517+0000","scrub_duration":0.0077222740000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"427'26","reported_seq":360,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169523+0000","last_change":"2024-09-10T19:03:43.899377+0000","last_active":"2024-09-10T19:06:30.169523+0000","last_peered":"2024-09-10T19:06:30.169523+0000","last_clean":"2024-09-10T19:06:30.169523+0000","last_became_active":"2024-09-10T19:02:49.572746+0000","last_became_peered":"2024-09-10T19:02:49.572746+0000","last_unstale":"2024-09-10T19:06:30.169523+0000","last_undegraded":"2024-09-10T19:06:30.169523+0000","last_fullsized":"2024-09-10T19:06:30.169523+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'22","last_scrub_stamp":"2024-09-10T19:03:43.899294+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-10T18:47:10.514951+0000","last_clean_scrub_stamp":"2024-09-10T19:03:43.899294+0000","objects_scrubbed":5,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0049335940000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":41,"num_read_kb":30,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"84'6","reported_seq":497,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:39.936074+0000","last_change":"2024-09-10T19:03:43.134996+0000","last_active":"2024-09-10T19:06:39.936074+0000","last_peered":"2024-09-10T19:06:39.936074+0000","last_clean":"2024-09-10T19:06:39.936074+0000","last_became_active":"2024-09-10T19:02:49.570177+0000","last_became_peered":"2024-09-10T19:02:49.570177+0000","last_unstale":"2024-09-10T19:06:39.936074+0000","last_undegraded":"2024-09-10T19:06:39.936074+0000","last_fullsized":"2024-09-10T19:06:39.936074+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'6","last_scrub_stamp":"2024-09-10T19:03:43.134912+0000","last_deep_scrub":"84'6","last_deep_scrub_stamp":"2024-09-10T18:47:22.545450+0000","last_clean_scrub_stamp":"2024-09-10T19:03:43.134912+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024059210000000001,"objects_trimmed":0,"snaptrim_duration":0.0021933159999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":344,"num_read_kb":339,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":436,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.172637+0000","last_change":"2024-09-10T19:04:29.468182+0000","last_active":"2024-09-10T19:06:30.172637+0000","last_peered":"2024-09-10T19:06:30.172637+0000","last_clean":"2024-09-10T19:06:30.172637+0000","last_became_active":"2024-09-10T18:55:40.393389+0000","last_became_peered":"2024-09-10T18:55:40.393389+0000","last_unstale":"2024-09-10T19:06:30.172637+0000","last_undegraded":"2024-09-10T19:06:30.172637+0000","last_fullsized":"2024-09-10T19:06:30.172637+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:29.468090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:04:29.468090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0099267039999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":454,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.167837+0000","last_change":"2024-09-10T19:06:09.998326+0000","last_active":"2024-09-10T19:06:30.167837+0000","last_peered":"2024-09-10T19:06:30.167837+0000","last_clean":"2024-09-10T19:06:30.167837+0000","last_became_active":"2024-09-10T18:57:02.438945+0000","last_became_peered":"2024-09-10T18:57:02.438945+0000","last_unstale":"2024-09-10T19:06:30.167837+0000","last_undegraded":"2024-09-10T19:06:30.167837+0000","last_fullsized":"2024-09-10T19:06:30.167837+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:09.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:06:09.998251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:23.808265+0000","scrub_duration":0.0039357020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":211,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.164262+0000","last_change":"2024-09-10T19:03:42.107868+0000","last_active":"2024-09-10T19:06:30.164262+0000","last_peered":"2024-09-10T19:06:30.164262+0000","last_clean":"2024-09-10T19:06:30.164262+0000","last_became_active":"2024-09-10T18:57:02.453865+0000","last_became_peered":"2024-09-10T18:57:02.453865+0000","last_unstale":"2024-09-10T19:06:30.164262+0000","last_undegraded":"2024-09-10T19:06:30.164262+0000","last_fullsized":"2024-09-10T19:06:30.164262+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:42.107785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:07.150480+0000","last_clean_scrub_stamp":"2024-09-10T19:03:42.107785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029595450000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.9","version":"449'11798","reported_seq":11884,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:37.828905+0000","last_change":"2024-09-10T19:04:36.986402+0000","last_active":"2024-09-10T19:06:37.828905+0000","last_peered":"2024-09-10T19:06:37.828905+0000","last_clean":"2024-09-10T19:06:37.828905+0000","last_became_active":"2024-09-10T19:04:36.984334+0000","last_became_peered":"2024-09-10T19:04:36.984334+0000","last_unstale":"2024-09-10T19:06:37.828905+0000","last_undegraded":"2024-09-10T19:06:37.828905+0000","last_fullsized":"2024-09-10T19:06:37.828905+0000","mapping_epoch":379,"log_start":"427'8559","ondisk_log_start":"427'8559","created":379,"last_epoch_clean":405,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3239,"log_dups_size":0,"ondisk_log_size":3239,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8387,"num_write_kb":268096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"70'81","reported_seq":164,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.162016+0000","last_change":"2024-09-10T19:04:28.278727+0000","last_active":"2024-09-10T19:06:30.162016+0000","last_peered":"2024-09-10T19:06:30.162016+0000","last_clean":"2024-09-10T19:06:30.162016+0000","last_became_active":"2024-09-10T19:02:49.578413+0000","last_became_peered":"2024-09-10T19:02:49.578413+0000","last_unstale":"2024-09-10T19:06:30.162016+0000","last_undegraded":"2024-09-10T19:06:30.162016+0000","last_fullsized":"2024-09-10T19:06:30.162016+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:04:28.278564+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:04:28.278564+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0066667519999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"427'36","reported_seq":215,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:32.212714+0000","last_change":"2024-09-10T19:03:40.879526+0000","last_active":"2024-09-10T19:06:32.212714+0000","last_peered":"2024-09-10T19:06:32.212714+0000","last_clean":"2024-09-10T19:06:32.212714+0000","last_became_active":"2024-09-10T19:00:42.516521+0000","last_became_peered":"2024-09-10T19:00:42.516521+0000","last_unstale":"2024-09-10T19:06:32.212714+0000","last_undegraded":"2024-09-10T19:06:32.212714+0000","last_fullsized":"2024-09-10T19:06:32.212714+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T19:03:40.879436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:03:40.879436+0000","objects_scrubbed":6,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0094657419999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":47,"num_write":36,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"305'2","reported_seq":444,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:32.190515+0000","last_change":"2024-09-10T19:03:21.921314+0000","last_active":"2024-09-10T19:06:32.190515+0000","last_peered":"2024-09-10T19:06:32.190515+0000","last_clean":"2024-09-10T19:06:32.190515+0000","last_became_active":"2024-09-10T18:55:40.390022+0000","last_became_peered":"2024-09-10T18:55:40.390022+0000","last_unstale":"2024-09-10T19:06:32.190515+0000","last_undegraded":"2024-09-10T19:06:32.190515+0000","last_fullsized":"2024-09-10T19:06:32.190515+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"305'2","last_scrub_stamp":"2024-09-10T19:03:21.921254+0000","last_deep_scrub":"305'2","last_deep_scrub_stamp":"2024-09-10T18:59:18.897059+0000","last_clean_scrub_stamp":"2024-09-10T19:03:21.921254+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021302700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":305,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:32.188675+0000","last_change":"2024-09-10T19:06:00.735401+0000","last_active":"2024-09-10T19:06:32.188675+0000","last_peered":"2024-09-10T19:06:32.188675+0000","last_clean":"2024-09-10T19:06:32.188675+0000","last_became_active":"2024-09-10T19:02:49.560325+0000","last_became_peered":"2024-09-10T19:02:49.560325+0000","last_unstale":"2024-09-10T19:06:32.188675+0000","last_undegraded":"2024-09-10T19:06:32.188675+0000","last_fullsized":"2024-09-10T19:06:32.188675+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:00.735322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:06:00.735322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:08.617305+0000","scrub_duration":0.0022871300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":130,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169893+0000","last_change":"2024-09-10T19:04:30.022221+0000","last_active":"2024-09-10T19:06:30.169893+0000","last_peered":"2024-09-10T19:06:30.169893+0000","last_clean":"2024-09-10T19:06:30.169893+0000","last_became_active":"2024-09-10T19:00:42.518566+0000","last_became_peered":"2024-09-10T19:00:42.518566+0000","last_unstale":"2024-09-10T19:06:30.169893+0000","last_undegraded":"2024-09-10T19:06:30.169893+0000","last_fullsized":"2024-09-10T19:06:30.169893+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:30.022167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:04:30.022167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001515976,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":89,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.167175+0000","last_change":"2024-09-10T19:02:49.573172+0000","last_active":"2024-09-10T19:06:30.167175+0000","last_peered":"2024-09-10T19:06:30.167175+0000","last_clean":"2024-09-10T19:06:30.167175+0000","last_became_active":"2024-09-10T19:02:49.572858+0000","last_became_peered":"2024-09-10T19:02:49.572858+0000","last_unstale":"2024-09-10T19:06:30.167175+0000","last_undegraded":"2024-09-10T19:06:30.167175+0000","last_fullsized":"2024-09-10T19:06:30.167175+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:20.272756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:02:20.272756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.a","version":"449'17040","reported_seq":17117,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:37.871817+0000","last_change":"2024-09-10T19:03:15.826075+0000","last_active":"2024-09-10T19:06:37.871817+0000","last_peered":"2024-09-10T19:06:37.871817+0000","last_clean":"2024-09-10T19:06:37.871817+0000","last_became_active":"2024-09-10T19:03:15.825784+0000","last_became_peered":"2024-09-10T19:03:15.825784+0000","last_unstale":"2024-09-10T19:06:37.871817+0000","last_undegraded":"2024-09-10T19:06:37.871817+0000","last_fullsized":"2024-09-10T19:06:37.871817+0000","mapping_epoch":379,"log_start":"439'13763","ondisk_log_start":"439'13763","created":379,"last_epoch_clean":380,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3277,"log_dups_size":0,"ondisk_log_size":3277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17040,"num_write_kb":545280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"427'45","reported_seq":583,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.164389+0000","last_change":"2024-09-10T19:06:01.618690+0000","last_active":"2024-09-10T19:06:30.164389+0000","last_peered":"2024-09-10T19:06:30.164389+0000","last_clean":"2024-09-10T19:06:30.164389+0000","last_became_active":"2024-09-10T18:57:02.435499+0000","last_became_peered":"2024-09-10T18:57:02.435499+0000","last_unstale":"2024-09-10T19:06:30.164389+0000","last_undegraded":"2024-09-10T19:06:30.164389+0000","last_fullsized":"2024-09-10T19:06:30.164389+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'45","last_scrub_stamp":"2024-09-10T19:06:01.618608+0000","last_deep_scrub":"232'33","last_deep_scrub_stamp":"2024-09-10T18:56:21.206960+0000","last_clean_scrub_stamp":"2024-09-10T19:06:01.618608+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:06.102994+0000","scrub_duration":0.008985346,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":61,"num_read_kb":51,"num_write":37,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"60'2","reported_seq":471,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.170995+0000","last_change":"2024-09-10T19:03:22.545399+0000","last_active":"2024-09-10T19:06:30.170995+0000","last_peered":"2024-09-10T19:06:30.170995+0000","last_clean":"2024-09-10T19:06:30.170995+0000","last_became_active":"2024-09-10T18:45:17.961660+0000","last_became_peered":"2024-09-10T18:45:17.961660+0000","last_unstale":"2024-09-10T19:06:30.170995+0000","last_undegraded":"2024-09-10T19:06:30.170995+0000","last_fullsized":"2024-09-10T19:06:30.170995+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-10T19:03:22.545342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:03:22.545342+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001448983,"objects_trimmed":0,"snaptrim_duration":0.0013942640000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":500,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.167234+0000","last_change":"2024-09-10T19:04:36.102267+0000","last_active":"2024-09-10T19:06:30.167234+0000","last_peered":"2024-09-10T19:06:30.167234+0000","last_clean":"2024-09-10T19:06:30.167234+0000","last_became_active":"2024-09-10T18:57:02.445712+0000","last_became_peered":"2024-09-10T18:57:02.445712+0000","last_unstale":"2024-09-10T19:06:30.167234+0000","last_undegraded":"2024-09-10T19:06:30.167234+0000","last_fullsized":"2024-09-10T19:06:30.167234+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:36.102203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:04:36.102203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.095986274999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'81","reported_seq":287,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:32.188521+0000","last_change":"2024-09-10T19:06:24.713689+0000","last_active":"2024-09-10T19:06:32.188521+0000","last_peered":"2024-09-10T19:06:32.188521+0000","last_clean":"2024-09-10T19:06:32.188521+0000","last_became_active":"2024-09-10T18:57:02.440063+0000","last_became_peered":"2024-09-10T18:57:02.440063+0000","last_unstale":"2024-09-10T19:06:32.188521+0000","last_undegraded":"2024-09-10T19:06:32.188521+0000","last_fullsized":"2024-09-10T19:06:32.188521+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:06:24.713626+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:56.085580+0000","last_clean_scrub_stamp":"2024-09-10T19:06:24.713626+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:40.045571+0000","scrub_duration":0.0013738730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":469,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.168850+0000","last_change":"2024-09-10T19:04:33.972472+0000","last_active":"2024-09-10T19:06:30.168850+0000","last_peered":"2024-09-10T19:06:30.168850+0000","last_clean":"2024-09-10T19:06:30.168850+0000","last_became_active":"2024-09-10T18:57:02.431556+0000","last_became_peered":"2024-09-10T18:57:02.431556+0000","last_unstale":"2024-09-10T19:06:30.168850+0000","last_undegraded":"2024-09-10T19:06:30.168850+0000","last_fullsized":"2024-09-10T19:06:30.168850+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:33.972423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:04:33.972423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0074396560000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":474,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169370+0000","last_change":"2024-09-10T19:03:51.037606+0000","last_active":"2024-09-10T19:06:30.169370+0000","last_peered":"2024-09-10T19:06:30.169370+0000","last_clean":"2024-09-10T19:06:30.169370+0000","last_became_active":"2024-09-10T18:45:24.073327+0000","last_became_peered":"2024-09-10T18:45:24.073327+0000","last_unstale":"2024-09-10T19:06:30.169370+0000","last_undegraded":"2024-09-10T19:06:30.169370+0000","last_fullsized":"2024-09-10T19:06:30.169370+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:51.037432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:03:51.037432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0085107059999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.5","version":"449'11900","reported_seq":11989,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:37.887524+0000","last_change":"2024-09-10T19:04:36.975636+0000","last_active":"2024-09-10T19:06:37.887524+0000","last_peered":"2024-09-10T19:06:37.887524+0000","last_clean":"2024-09-10T19:06:37.887524+0000","last_became_active":"2024-09-10T19:04:36.973383+0000","last_became_peered":"2024-09-10T19:04:36.973383+0000","last_unstale":"2024-09-10T19:06:37.887524+0000","last_undegraded":"2024-09-10T19:06:37.887524+0000","last_fullsized":"2024-09-10T19:06:37.887524+0000","mapping_epoch":379,"log_start":"424'8507","ondisk_log_start":"424'8507","created":379,"last_epoch_clean":405,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3393,"log_dups_size":0,"ondisk_log_size":3393,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5406720,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8438,"num_write_kb":272672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.f","version":"70'81","reported_seq":479,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.162535+0000","last_change":"2024-09-10T19:06:21.520670+0000","last_active":"2024-09-10T19:06:30.162535+0000","last_peered":"2024-09-10T19:06:30.162535+0000","last_clean":"2024-09-10T19:06:30.162535+0000","last_became_active":"2024-09-10T18:52:16.598295+0000","last_became_peered":"2024-09-10T18:52:16.598295+0000","last_unstale":"2024-09-10T19:06:30.162535+0000","last_undegraded":"2024-09-10T19:06:30.162535+0000","last_fullsized":"2024-09-10T19:06:30.162535+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:06:21.520615+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T19:06:21.520615+0000","last_clean_scrub_stamp":"2024-09-10T19:06:21.520615+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:33.819484+0000","scrub_duration":0.0011289640000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"427'51","reported_seq":591,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.163916+0000","last_change":"2024-09-10T19:03:46.180416+0000","last_active":"2024-09-10T19:06:30.163916+0000","last_peered":"2024-09-10T19:06:30.163916+0000","last_clean":"2024-09-10T19:06:30.163916+0000","last_became_active":"2024-09-10T18:57:02.436059+0000","last_became_peered":"2024-09-10T18:57:02.436059+0000","last_unstale":"2024-09-10T19:06:30.163916+0000","last_undegraded":"2024-09-10T19:06:30.163916+0000","last_fullsized":"2024-09-10T19:06:30.163916+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'35","last_scrub_stamp":"2024-09-10T19:03:46.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:03:46.179242+0000","objects_scrubbed":9,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0052365550000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":80,"num_read_kb":69,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":448,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.168930+0000","last_change":"2024-09-10T19:06:02.034786+0000","last_active":"2024-09-10T19:06:30.168930+0000","last_peered":"2024-09-10T19:06:30.168930+0000","last_clean":"2024-09-10T19:06:30.168930+0000","last_became_active":"2024-09-10T18:55:40.391849+0000","last_became_peered":"2024-09-10T18:55:40.391849+0000","last_unstale":"2024-09-10T19:06:30.168930+0000","last_undegraded":"2024-09-10T19:06:30.168930+0000","last_fullsized":"2024-09-10T19:06:30.168930+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:02.034673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:06:02.034673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:21.764260+0000","scrub_duration":0.0021020129999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":420,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:32.188961+0000","last_change":"2024-09-10T19:03:48.853831+0000","last_active":"2024-09-10T19:06:32.188961+0000","last_peered":"2024-09-10T19:06:32.188961+0000","last_clean":"2024-09-10T19:06:32.188961+0000","last_became_active":"2024-09-10T19:00:42.514709+0000","last_became_peered":"2024-09-10T19:00:42.514709+0000","last_unstale":"2024-09-10T19:06:32.188961+0000","last_undegraded":"2024-09-10T19:06:32.188961+0000","last_fullsized":"2024-09-10T19:06:32.188961+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:48.853752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:03:48.853752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0059332969999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":255,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.168243+0000","last_change":"2024-09-10T19:02:58.491396+0000","last_active":"2024-09-10T19:06:30.168243+0000","last_peered":"2024-09-10T19:06:30.168243+0000","last_clean":"2024-09-10T19:06:30.168243+0000","last_became_active":"2024-09-10T18:57:02.468866+0000","last_became_peered":"2024-09-10T18:57:02.468866+0000","last_unstale":"2024-09-10T19:06:30.168243+0000","last_undegraded":"2024-09-10T19:06:30.168243+0000","last_fullsized":"2024-09-10T19:06:30.168243+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:58.491326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:02:58.491326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017883440000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"449'12259","reported_seq":12347,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:37.880691+0000","last_change":"2024-09-10T19:04:36.977141+0000","last_active":"2024-09-10T19:06:37.880691+0000","last_peered":"2024-09-10T19:06:37.880691+0000","last_clean":"2024-09-10T19:06:37.880691+0000","last_became_active":"2024-09-10T19:04:36.975442+0000","last_became_peered":"2024-09-10T19:04:36.975442+0000","last_unstale":"2024-09-10T19:06:37.880691+0000","last_undegraded":"2024-09-10T19:06:37.880691+0000","last_fullsized":"2024-09-10T19:06:37.880691+0000","mapping_epoch":379,"log_start":"428'9089","ondisk_log_start":"428'9089","created":379,"last_epoch_clean":405,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3170,"log_dups_size":0,"ondisk_log_size":3170,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8721,"num_write_kb":279168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,3],"acting":[6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.e","version":"70'81","reported_seq":337,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.168427+0000","last_change":"2024-09-10T19:04:31.431314+0000","last_active":"2024-09-10T19:06:30.168427+0000","last_peered":"2024-09-10T19:06:30.168427+0000","last_clean":"2024-09-10T19:06:30.168427+0000","last_became_active":"2024-09-10T18:57:02.449115+0000","last_became_peered":"2024-09-10T18:57:02.449115+0000","last_unstale":"2024-09-10T19:06:30.168427+0000","last_undegraded":"2024-09-10T19:06:30.168427+0000","last_fullsized":"2024-09-10T19:06:30.168427+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:04:31.431216+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:04:31.431216+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011858260000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"427'25","reported_seq":485,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:32.189490+0000","last_change":"2024-09-10T19:03:43.850751+0000","last_active":"2024-09-10T19:06:32.189490+0000","last_peered":"2024-09-10T19:06:32.189490+0000","last_clean":"2024-09-10T19:06:32.189490+0000","last_became_active":"2024-09-10T18:55:40.395257+0000","last_became_peered":"2024-09-10T18:55:40.395257+0000","last_unstale":"2024-09-10T19:06:32.189490+0000","last_undegraded":"2024-09-10T19:06:32.189490+0000","last_fullsized":"2024-09-10T19:06:32.189490+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T19:03:43.850670+0000","last_deep_scrub":"130'13","last_deep_scrub_stamp":"2024-09-10T18:51:58.427534+0000","last_clean_scrub_stamp":"2024-09-10T19:03:43.850670+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026924499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":99,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169651+0000","last_change":"2024-09-10T19:03:51.598347+0000","last_active":"2024-09-10T19:06:30.169651+0000","last_peered":"2024-09-10T19:06:30.169651+0000","last_clean":"2024-09-10T19:06:30.169651+0000","last_became_active":"2024-09-10T19:02:49.581709+0000","last_became_peered":"2024-09-10T19:02:49.581709+0000","last_unstale":"2024-09-10T19:06:30.169651+0000","last_undegraded":"2024-09-10T19:06:30.169651+0000","last_fullsized":"2024-09-10T19:06:30.169651+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:51.598266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:03:51.598266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047300789999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":429,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169771+0000","last_change":"2024-09-10T19:02:51.031764+0000","last_active":"2024-09-10T19:06:30.169771+0000","last_peered":"2024-09-10T19:06:30.169771+0000","last_clean":"2024-09-10T19:06:30.169771+0000","last_became_active":"2024-09-10T18:55:40.390771+0000","last_became_peered":"2024-09-10T18:55:40.390771+0000","last_unstale":"2024-09-10T19:06:30.169771+0000","last_undegraded":"2024-09-10T19:06:30.169771+0000","last_fullsized":"2024-09-10T19:06:30.169771+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:51.031711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_clean_scrub_stamp":"2024-09-10T19:02:51.031711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015681510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":464,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:32.190158+0000","last_change":"2024-09-10T19:04:28.767375+0000","last_active":"2024-09-10T19:06:32.190158+0000","last_peered":"2024-09-10T19:06:32.190158+0000","last_clean":"2024-09-10T19:06:32.190158+0000","last_became_active":"2024-09-10T18:57:02.452387+0000","last_became_peered":"2024-09-10T18:57:02.452387+0000","last_unstale":"2024-09-10T19:06:32.190158+0000","last_undegraded":"2024-09-10T19:06:32.190158+0000","last_fullsized":"2024-09-10T19:06:32.190158+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:28.766947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:53.345184+0000","last_clean_scrub_stamp":"2024-09-10T19:04:28.766947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0079684749999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"449'12032","reported_seq":12121,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:37.830205+0000","last_change":"2024-09-10T19:04:36.980107+0000","last_active":"2024-09-10T19:06:37.830205+0000","last_peered":"2024-09-10T19:06:37.830205+0000","last_clean":"2024-09-10T19:06:37.830205+0000","last_became_active":"2024-09-10T19:04:36.977753+0000","last_became_peered":"2024-09-10T19:04:36.977753+0000","last_unstale":"2024-09-10T19:06:37.830205+0000","last_undegraded":"2024-09-10T19:06:37.830205+0000","last_fullsized":"2024-09-10T19:06:37.830205+0000","mapping_epoch":379,"log_start":"424'8680","ondisk_log_start":"424'8680","created":379,"last_epoch_clean":405,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3352,"log_dups_size":0,"ondisk_log_size":3352,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":98304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8569,"num_write_kb":274272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.d","version":"70'81","reported_seq":268,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169759+0000","last_change":"2024-09-10T19:02:01.044879+0000","last_active":"2024-09-10T19:06:30.169759+0000","last_peered":"2024-09-10T19:06:30.169759+0000","last_clean":"2024-09-10T19:06:30.169759+0000","last_became_active":"2024-09-10T18:57:02.439652+0000","last_became_peered":"2024-09-10T18:57:02.439652+0000","last_unstale":"2024-09-10T19:06:30.169759+0000","last_undegraded":"2024-09-10T19:06:30.169759+0000","last_fullsized":"2024-09-10T19:06:30.169759+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:01.044143+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:02:01.044143+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014088588000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"427'46","reported_seq":171,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.165750+0000","last_change":"2024-09-10T19:02:18.712390+0000","last_active":"2024-09-10T19:06:30.165750+0000","last_peered":"2024-09-10T19:06:30.165750+0000","last_clean":"2024-09-10T19:06:30.165750+0000","last_became_active":"2024-09-10T19:00:42.515903+0000","last_became_peered":"2024-09-10T19:00:42.515903+0000","last_unstale":"2024-09-10T19:06:30.165750+0000","last_undegraded":"2024-09-10T19:06:30.165750+0000","last_fullsized":"2024-09-10T19:06:30.165750+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"365'34","last_scrub_stamp":"2024-09-10T19:02:18.712306+0000","last_deep_scrub":"365'34","last_deep_scrub_stamp":"2024-09-10T19:02:18.712306+0000","last_clean_scrub_stamp":"2024-09-10T19:02:18.712306+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.017099106999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":60,"num_read_kb":51,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":429,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.170992+0000","last_change":"2024-09-10T19:06:23.202241+0000","last_active":"2024-09-10T19:06:30.170992+0000","last_peered":"2024-09-10T19:06:30.170992+0000","last_clean":"2024-09-10T19:06:30.170992+0000","last_became_active":"2024-09-10T18:57:02.461708+0000","last_became_peered":"2024-09-10T18:57:02.461708+0000","last_unstale":"2024-09-10T19:06:30.170992+0000","last_undegraded":"2024-09-10T19:06:30.170992+0000","last_fullsized":"2024-09-10T19:06:30.170992+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:23.202164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:05.603125+0000","last_clean_scrub_stamp":"2024-09-10T19:06:23.202164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:30.512580+0000","scrub_duration":0.012238550000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"273'10","reported_seq":409,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:36.331479+0000","last_change":"2024-09-10T19:06:28.588050+0000","last_active":"2024-09-10T19:06:36.331479+0000","last_peered":"2024-09-10T19:06:36.331479+0000","last_clean":"2024-09-10T19:06:36.331479+0000","last_became_active":"2024-09-10T18:57:02.441890+0000","last_became_peered":"2024-09-10T18:57:02.441890+0000","last_unstale":"2024-09-10T19:06:36.331479+0000","last_undegraded":"2024-09-10T19:06:36.331479+0000","last_fullsized":"2024-09-10T19:06:36.331479+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'10","last_scrub_stamp":"2024-09-10T19:06:28.587892+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-09-10T18:47:02.582229+0000","last_clean_scrub_stamp":"2024-09-10T19:06:28.587892+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:35.607577+0000","scrub_duration":0.0043958399999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":481,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.166966+0000","last_change":"2024-09-10T19:03:32.808204+0000","last_active":"2024-09-10T19:06:30.166966+0000","last_peered":"2024-09-10T19:06:30.166966+0000","last_clean":"2024-09-10T19:06:30.166966+0000","last_became_active":"2024-09-10T18:55:40.389202+0000","last_became_peered":"2024-09-10T18:55:40.389202+0000","last_unstale":"2024-09-10T19:06:30.166966+0000","last_undegraded":"2024-09-10T19:06:30.166966+0000","last_fullsized":"2024-09-10T19:06:30.166966+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:32.808130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_clean_scrub_stamp":"2024-09-10T19:03:32.808130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003767603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"449'12341","reported_seq":12456,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:37.869588+0000","last_change":"2024-09-10T19:06:32.570307+0000","last_active":"2024-09-10T19:06:37.869588+0000","last_peered":"2024-09-10T19:06:37.869588+0000","last_clean":"2024-09-10T19:06:37.869588+0000","last_became_active":"2024-09-10T19:04:36.963484+0000","last_became_peered":"2024-09-10T19:04:36.963484+0000","last_unstale":"2024-09-10T19:06:37.869588+0000","last_undegraded":"2024-09-10T19:06:37.869588+0000","last_fullsized":"2024-09-10T19:06:37.869588+0000","mapping_epoch":379,"log_start":"425'8690","ondisk_log_start":"425'8690","created":379,"last_epoch_clean":405,"parent":"0.0","parent_split_bits":0,"last_scrub":"449'12055","last_scrub_stamp":"2024-09-10T19:06:32.570104+0000","last_deep_scrub":"449'12055","last_deep_scrub_stamp":"2024-09-10T19:06:32.570104+0000","last_clean_scrub_stamp":"2024-09-10T19:06:32.570104+0000","objects_scrubbed":321,"log_size":3651,"log_dups_size":0,"ondisk_log_size":3651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:40.160089+0000","scrub_duration":1.91816849,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":321,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.c","version":"70'81","reported_seq":412,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169812+0000","last_change":"2024-09-10T19:02:54.540617+0000","last_active":"2024-09-10T19:06:30.169812+0000","last_peered":"2024-09-10T19:06:30.169812+0000","last_clean":"2024-09-10T19:06:30.169812+0000","last_became_active":"2024-09-10T18:55:40.390655+0000","last_became_peered":"2024-09-10T18:55:40.390655+0000","last_unstale":"2024-09-10T19:06:30.169812+0000","last_undegraded":"2024-09-10T19:06:30.169812+0000","last_fullsized":"2024-09-10T19:06:30.169812+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:54.540558+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:56:33.779154+0000","last_clean_scrub_stamp":"2024-09-10T19:02:54.540558+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019874599999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"427'24","reported_seq":510,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.168570+0000","last_change":"2024-09-10T19:06:12.856434+0000","last_active":"2024-09-10T19:06:30.168570+0000","last_peered":"2024-09-10T19:06:30.168570+0000","last_clean":"2024-09-10T19:06:30.168570+0000","last_became_active":"2024-09-10T18:45:20.056869+0000","last_became_peered":"2024-09-10T18:45:20.056869+0000","last_unstale":"2024-09-10T19:06:30.168570+0000","last_undegraded":"2024-09-10T19:06:30.168570+0000","last_fullsized":"2024-09-10T19:06:30.168570+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'24","last_scrub_stamp":"2024-09-10T19:06:12.856360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:06:12.856360+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:23.852248+0000","scrub_duration":0.0042329550000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"56'2","reported_seq":511,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.171473+0000","last_change":"2024-09-10T19:06:14.852777+0000","last_active":"2024-09-10T19:06:30.171473+0000","last_peered":"2024-09-10T19:06:30.171473+0000","last_clean":"2024-09-10T19:06:30.171473+0000","last_became_active":"2024-09-10T18:57:02.434475+0000","last_became_peered":"2024-09-10T18:57:02.434475+0000","last_unstale":"2024-09-10T19:06:30.171473+0000","last_undegraded":"2024-09-10T19:06:30.171473+0000","last_fullsized":"2024-09-10T19:06:30.171473+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-10T19:06:14.852698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:06:14.852698+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:43.077218+0000","scrub_duration":0.006938662,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":430,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:32.188941+0000","last_change":"2024-09-10T19:06:31.634746+0000","last_active":"2024-09-10T19:06:32.188941+0000","last_peered":"2024-09-10T19:06:32.188941+0000","last_clean":"2024-09-10T19:06:32.188941+0000","last_became_active":"2024-09-10T19:00:42.513796+0000","last_became_peered":"2024-09-10T19:00:42.513796+0000","last_unstale":"2024-09-10T19:06:32.188941+0000","last_undegraded":"2024-09-10T19:06:32.188941+0000","last_fullsized":"2024-09-10T19:06:32.188941+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:31.634653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:02.711594+0000","last_clean_scrub_stamp":"2024-09-10T19:06:31.634653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:33.241478+0000","scrub_duration":0.0052320470000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":462,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.174042+0000","last_change":"2024-09-10T19:03:46.711248+0000","last_active":"2024-09-10T19:06:30.174042+0000","last_peered":"2024-09-10T19:06:30.174042+0000","last_clean":"2024-09-10T19:06:30.174042+0000","last_became_active":"2024-09-10T18:57:02.459985+0000","last_became_peered":"2024-09-10T18:57:02.459985+0000","last_unstale":"2024-09-10T19:06:30.174042+0000","last_undegraded":"2024-09-10T19:06:30.174042+0000","last_fullsized":"2024-09-10T19:06:30.174042+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:46.710933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:46.710933+0000","last_clean_scrub_stamp":"2024-09-10T19:03:46.710933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0061568430000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.19","version":"449'11778","reported_seq":18412,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:37.887223+0000","last_change":"2024-09-10T19:05:38.613447+0000","last_active":"2024-09-10T19:06:37.887223+0000","last_peered":"2024-09-10T19:06:37.887223+0000","last_clean":"2024-09-10T19:06:37.887223+0000","last_became_active":"2024-09-10T19:05:38.577625+0000","last_became_peered":"2024-09-10T19:05:38.577625+0000","last_unstale":"2024-09-10T19:06:37.887223+0000","last_undegraded":"2024-09-10T19:06:37.887223+0000","last_fullsized":"2024-09-10T19:06:37.887223+0000","mapping_epoch":432,"log_start":"423'8195","ondisk_log_start":"423'8195","created":404,"last_epoch_clean":433,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3583,"log_dups_size":0,"ondisk_log_size":3583,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":589824,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8367,"num_write_kb":268032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3260,"num_bytes_recovered":213647360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.13","version":"70'81","reported_seq":168,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.174243+0000","last_change":"2024-09-10T19:04:36.470253+0000","last_active":"2024-09-10T19:06:30.174243+0000","last_peered":"2024-09-10T19:06:30.174243+0000","last_clean":"2024-09-10T19:06:30.174243+0000","last_became_active":"2024-09-10T19:02:49.580656+0000","last_became_peered":"2024-09-10T19:02:49.580656+0000","last_unstale":"2024-09-10T19:06:30.174243+0000","last_undegraded":"2024-09-10T19:06:30.174243+0000","last_fullsized":"2024-09-10T19:06:30.174243+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:04:36.470196+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:04:36.470196+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001517404,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"427'25","reported_seq":508,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:32.214644+0000","last_change":"2024-09-10T19:04:35.830252+0000","last_active":"2024-09-10T19:06:32.214644+0000","last_peered":"2024-09-10T19:06:32.214644+0000","last_clean":"2024-09-10T19:06:32.214644+0000","last_became_active":"2024-09-10T18:57:02.453001+0000","last_became_peered":"2024-09-10T18:57:02.453001+0000","last_unstale":"2024-09-10T19:06:32.214644+0000","last_undegraded":"2024-09-10T19:06:32.214644+0000","last_fullsized":"2024-09-10T19:06:32.214644+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T19:04:35.830196+0000","last_deep_scrub":"76'13","last_deep_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_clean_scrub_stamp":"2024-09-10T19:04:35.830196+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001824537,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":303,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:32.188603+0000","last_change":"2024-09-10T19:03:35.858172+0000","last_active":"2024-09-10T19:06:32.188603+0000","last_peered":"2024-09-10T19:06:32.188603+0000","last_clean":"2024-09-10T19:06:32.188603+0000","last_became_active":"2024-09-10T19:02:49.580701+0000","last_became_peered":"2024-09-10T19:02:49.580701+0000","last_unstale":"2024-09-10T19:06:32.188603+0000","last_undegraded":"2024-09-10T19:06:32.188603+0000","last_fullsized":"2024-09-10T19:06:32.188603+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:35.858086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:03:35.858086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0061699850000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":474,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169146+0000","last_change":"2024-09-10T19:06:30.141526+0000","last_active":"2024-09-10T19:06:30.169146+0000","last_peered":"2024-09-10T19:06:30.169146+0000","last_clean":"2024-09-10T19:06:30.169146+0000","last_became_active":"2024-09-10T18:57:02.455341+0000","last_became_peered":"2024-09-10T18:57:02.455341+0000","last_unstale":"2024-09-10T19:06:30.169146+0000","last_undegraded":"2024-09-10T19:06:30.169146+0000","last_fullsized":"2024-09-10T19:06:30.169146+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:30.141437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:00.754912+0000","last_clean_scrub_stamp":"2024-09-10T19:06:30.141437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:33.141155+0000","scrub_duration":0.0090402390000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"364'2","reported_seq":488,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169498+0000","last_change":"2024-09-10T19:02:15.059462+0000","last_active":"2024-09-10T19:06:30.169498+0000","last_peered":"2024-09-10T19:06:30.169498+0000","last_clean":"2024-09-10T19:06:30.169498+0000","last_became_active":"2024-09-10T18:59:48.741275+0000","last_became_peered":"2024-09-10T18:59:48.741275+0000","last_unstale":"2024-09-10T19:06:30.169498+0000","last_undegraded":"2024-09-10T19:06:30.169498+0000","last_fullsized":"2024-09-10T19:06:30.169498+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"364'2","last_scrub_stamp":"2024-09-10T19:02:15.059320+0000","last_deep_scrub":"364'2","last_deep_scrub_stamp":"2024-09-10T19:02:15.059320+0000","last_clean_scrub_stamp":"2024-09-10T19:02:15.059320+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.020873968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":52,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.18","version":"449'12138","reported_seq":20366,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:37.876501+0000","last_change":"2024-09-10T19:05:04.827272+0000","last_active":"2024-09-10T19:06:37.876501+0000","last_peered":"2024-09-10T19:06:37.876501+0000","last_clean":"2024-09-10T19:06:37.876501+0000","last_became_active":"2024-09-10T19:05:04.805312+0000","last_became_peered":"2024-09-10T19:05:04.805312+0000","last_unstale":"2024-09-10T19:06:37.876501+0000","last_undegraded":"2024-09-10T19:06:37.876501+0000","last_fullsized":"2024-09-10T19:06:37.876501+0000","mapping_epoch":417,"log_start":"424'8537","ondisk_log_start":"424'8537","created":404,"last_epoch_clean":418,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3601,"log_dups_size":0,"ondisk_log_size":3601,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8630,"num_write_kb":274656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4058,"num_bytes_recovered":265945088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.12","version":"70'81","reported_seq":338,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.166774+0000","last_change":"2024-09-10T19:06:08.005069+0000","last_active":"2024-09-10T19:06:30.166774+0000","last_peered":"2024-09-10T19:06:30.166774+0000","last_clean":"2024-09-10T19:06:30.166774+0000","last_became_active":"2024-09-10T18:52:16.598921+0000","last_became_peered":"2024-09-10T18:52:16.598921+0000","last_unstale":"2024-09-10T19:06:30.166774+0000","last_undegraded":"2024-09-10T19:06:30.166774+0000","last_fullsized":"2024-09-10T19:06:30.166774+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:06:08.005010+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:54.127476+0000","last_clean_scrub_stamp":"2024-09-10T19:06:08.005010+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:33.753507+0000","scrub_duration":0.0036749040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":507,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.164109+0000","last_change":"2024-09-10T19:06:08.468915+0000","last_active":"2024-09-10T19:06:30.164109+0000","last_peered":"2024-09-10T19:06:30.164109+0000","last_clean":"2024-09-10T19:06:30.164109+0000","last_became_active":"2024-09-10T18:57:02.442251+0000","last_became_peered":"2024-09-10T18:57:02.442251+0000","last_unstale":"2024-09-10T19:06:30.164109+0000","last_undegraded":"2024-09-10T19:06:30.164109+0000","last_fullsized":"2024-09-10T19:06:30.164109+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:08.468838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:06:08.468838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:18.444598+0000","scrub_duration":0.0086820209999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"427'18","reported_seq":504,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.170652+0000","last_change":"2024-09-10T19:03:50.702400+0000","last_active":"2024-09-10T19:06:30.170652+0000","last_peered":"2024-09-10T19:06:30.170652+0000","last_clean":"2024-09-10T19:06:30.170652+0000","last_became_active":"2024-09-10T18:57:02.460430+0000","last_became_peered":"2024-09-10T18:57:02.460430+0000","last_unstale":"2024-09-10T19:06:30.170652+0000","last_undegraded":"2024-09-10T19:06:30.170652+0000","last_fullsized":"2024-09-10T19:06:30.170652+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'12","last_scrub_stamp":"2024-09-10T19:03:50.702317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:03:50.702317+0000","objects_scrubbed":3,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0055835499999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":24,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":466,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.168962+0000","last_change":"2024-09-10T19:06:30.117244+0000","last_active":"2024-09-10T19:06:30.168962+0000","last_peered":"2024-09-10T19:06:30.168962+0000","last_clean":"2024-09-10T19:06:30.168962+0000","last_became_active":"2024-09-10T18:45:22.069653+0000","last_became_peered":"2024-09-10T18:45:22.069653+0000","last_unstale":"2024-09-10T19:06:30.168962+0000","last_undegraded":"2024-09-10T19:06:30.168962+0000","last_fullsized":"2024-09-10T19:06:30.168962+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:30.117162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:06:30.117162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:36.601291+0000","scrub_duration":0.022352659,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":231,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.164513+0000","last_change":"2024-09-10T19:06:17.445445+0000","last_active":"2024-09-10T19:06:30.164513+0000","last_peered":"2024-09-10T19:06:30.164513+0000","last_clean":"2024-09-10T19:06:30.164513+0000","last_became_active":"2024-09-10T19:00:42.519492+0000","last_became_peered":"2024-09-10T19:00:42.519492+0000","last_unstale":"2024-09-10T19:06:30.164513+0000","last_undegraded":"2024-09-10T19:06:30.164513+0000","last_fullsized":"2024-09-10T19:06:30.164513+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:17.445379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:06:17.445379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:24.930408+0000","scrub_duration":0.002512116,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.11","version":"70'81","reported_seq":374,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:32.190285+0000","last_change":"2024-09-10T19:04:33.761605+0000","last_active":"2024-09-10T19:06:32.190285+0000","last_peered":"2024-09-10T19:06:32.190285+0000","last_clean":"2024-09-10T19:06:32.190285+0000","last_became_active":"2024-09-10T19:00:42.517311+0000","last_became_peered":"2024-09-10T19:00:42.517311+0000","last_unstale":"2024-09-10T19:06:32.190285+0000","last_undegraded":"2024-09-10T19:06:32.190285+0000","last_fullsized":"2024-09-10T19:06:32.190285+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:04:33.761515+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:04:33.761515+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0042906769999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"427'30","reported_seq":567,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169431+0000","last_change":"2024-09-10T19:04:33.464299+0000","last_active":"2024-09-10T19:06:30.169431+0000","last_peered":"2024-09-10T19:06:30.169431+0000","last_clean":"2024-09-10T19:06:30.169431+0000","last_became_active":"2024-09-10T18:57:02.444437+0000","last_became_peered":"2024-09-10T18:57:02.444437+0000","last_unstale":"2024-09-10T19:06:30.169431+0000","last_undegraded":"2024-09-10T19:06:30.169431+0000","last_fullsized":"2024-09-10T19:06:30.169431+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'20","last_scrub_stamp":"2024-09-10T19:04:33.464197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:04:33.464197+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040076549999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":480,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.166495+0000","last_change":"2024-09-10T19:02:20.755424+0000","last_active":"2024-09-10T19:06:30.166495+0000","last_peered":"2024-09-10T19:06:30.166495+0000","last_clean":"2024-09-10T19:06:30.166495+0000","last_became_active":"2024-09-10T18:57:02.445710+0000","last_became_peered":"2024-09-10T18:57:02.445710+0000","last_unstale":"2024-09-10T19:06:30.166495+0000","last_undegraded":"2024-09-10T19:06:30.166495+0000","last_fullsized":"2024-09-10T19:06:30.166495+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:20.755253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:20.755253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023404490000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"304'11","reported_seq":765,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:36.328565+0000","last_change":"2024-09-10T19:03:34.811295+0000","last_active":"2024-09-10T19:06:36.328565+0000","last_peered":"2024-09-10T19:06:36.328565+0000","last_clean":"2024-09-10T19:06:36.328565+0000","last_became_active":"2024-09-10T18:58:58.489370+0000","last_became_peered":"2024-09-10T18:58:58.489370+0000","last_unstale":"2024-09-10T19:06:36.328565+0000","last_undegraded":"2024-09-10T19:06:36.328565+0000","last_fullsized":"2024-09-10T19:06:36.328565+0000","mapping_epoch":303,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":304,"parent":"0.0","parent_split_bits":0,"last_scrub":"304'11","last_scrub_stamp":"2024-09-10T19:03:34.811194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:03:34.811194+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024856539999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":159,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:32.212581+0000","last_change":"2024-09-10T19:06:06.714131+0000","last_active":"2024-09-10T19:06:32.212581+0000","last_peered":"2024-09-10T19:06:32.212581+0000","last_clean":"2024-09-10T19:06:32.212581+0000","last_became_active":"2024-09-10T18:59:48.744923+0000","last_became_peered":"2024-09-10T18:59:48.744923+0000","last_unstale":"2024-09-10T19:06:32.212581+0000","last_undegraded":"2024-09-10T19:06:32.212581+0000","last_fullsized":"2024-09-10T19:06:32.212581+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:06.714078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:57.476724+0000","last_clean_scrub_stamp":"2024-09-10T19:06:06.714078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:33.025365+0000","scrub_duration":0.001109358,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.10","version":"70'81","reported_seq":338,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.165816+0000","last_change":"2024-09-10T19:03:50.010677+0000","last_active":"2024-09-10T19:06:30.165816+0000","last_peered":"2024-09-10T19:06:30.165816+0000","last_clean":"2024-09-10T19:06:30.165816+0000","last_became_active":"2024-09-10T18:57:02.443782+0000","last_became_peered":"2024-09-10T18:57:02.443782+0000","last_unstale":"2024-09-10T19:06:30.165816+0000","last_undegraded":"2024-09-10T19:06:30.165816+0000","last_fullsized":"2024-09-10T19:06:30.165816+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:03:50.010596+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:03:50.010596+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030874829999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"427'21","reported_seq":149,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.168086+0000","last_change":"2024-09-10T19:02:46.674532+0000","last_active":"2024-09-10T19:06:30.168086+0000","last_peered":"2024-09-10T19:06:30.168086+0000","last_clean":"2024-09-10T19:06:30.168086+0000","last_became_active":"2024-09-10T19:00:42.518672+0000","last_became_peered":"2024-09-10T19:00:42.518672+0000","last_unstale":"2024-09-10T19:06:30.168086+0000","last_undegraded":"2024-09-10T19:06:30.168086+0000","last_fullsized":"2024-09-10T19:06:30.168086+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T19:02:46.674451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:46.674451+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0082292530000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":33,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":448,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:32.189131+0000","last_change":"2024-09-10T19:04:34.812187+0000","last_active":"2024-09-10T19:06:32.189131+0000","last_peered":"2024-09-10T19:06:32.189131+0000","last_clean":"2024-09-10T19:06:32.189131+0000","last_became_active":"2024-09-10T18:55:40.390697+0000","last_became_peered":"2024-09-10T18:55:40.390697+0000","last_unstale":"2024-09-10T19:06:32.189131+0000","last_undegraded":"2024-09-10T19:06:32.189131+0000","last_fullsized":"2024-09-10T19:06:32.189131+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:34.812044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:04:34.812044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013174425,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"273'11","reported_seq":748,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:36.328769+0000","last_change":"2024-09-10T19:03:49.744278+0000","last_active":"2024-09-10T19:06:36.328769+0000","last_peered":"2024-09-10T19:06:36.328769+0000","last_clean":"2024-09-10T19:06:36.328769+0000","last_became_active":"2024-09-10T18:57:02.445268+0000","last_became_peered":"2024-09-10T18:57:02.445268+0000","last_unstale":"2024-09-10T19:06:36.328769+0000","last_undegraded":"2024-09-10T19:06:36.328769+0000","last_fullsized":"2024-09-10T19:06:36.328769+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'11","last_scrub_stamp":"2024-09-10T19:03:49.744177+0000","last_deep_scrub":"273'11","last_deep_scrub_stamp":"2024-09-10T19:03:49.744177+0000","last_clean_scrub_stamp":"2024-09-10T19:03:49.744177+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0069874999999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":479,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.171284+0000","last_change":"2024-09-10T19:03:23.737568+0000","last_active":"2024-09-10T19:06:30.171284+0000","last_peered":"2024-09-10T19:06:30.171284+0000","last_clean":"2024-09-10T19:06:30.171284+0000","last_became_active":"2024-09-10T18:55:40.391662+0000","last_became_peered":"2024-09-10T18:55:40.391662+0000","last_unstale":"2024-09-10T19:06:30.171284+0000","last_undegraded":"2024-09-10T19:06:30.171284+0000","last_fullsized":"2024-09-10T19:06:30.171284+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:23.737518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_clean_scrub_stamp":"2024-09-10T19:03:23.737518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037070459999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"427'21","reported_seq":373,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.163728+0000","last_change":"2024-09-10T19:03:37.134082+0000","last_active":"2024-09-10T19:06:30.163728+0000","last_peered":"2024-09-10T19:06:30.163728+0000","last_clean":"2024-09-10T19:06:30.163728+0000","last_became_active":"2024-09-10T19:02:49.561403+0000","last_became_peered":"2024-09-10T19:02:49.561403+0000","last_unstale":"2024-09-10T19:06:30.163728+0000","last_undegraded":"2024-09-10T19:06:30.163728+0000","last_fullsized":"2024-09-10T19:06:30.163728+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T19:03:37.134028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:03:37.134028+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058255470000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":511,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.162832+0000","last_change":"2024-09-10T19:03:36.087607+0000","last_active":"2024-09-10T19:06:30.162832+0000","last_peered":"2024-09-10T19:06:30.162832+0000","last_clean":"2024-09-10T19:06:30.162832+0000","last_became_active":"2024-09-10T19:00:42.518310+0000","last_became_peered":"2024-09-10T19:00:42.518310+0000","last_unstale":"2024-09-10T19:06:30.162832+0000","last_undegraded":"2024-09-10T19:06:30.162832+0000","last_fullsized":"2024-09-10T19:06:30.162832+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:36.087523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:03:36.087523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039520240000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":461,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.174172+0000","last_change":"2024-09-10T19:02:22.258152+0000","last_active":"2024-09-10T19:06:30.174172+0000","last_peered":"2024-09-10T19:06:30.174172+0000","last_clean":"2024-09-10T19:06:30.174172+0000","last_became_active":"2024-09-10T18:55:40.398243+0000","last_became_peered":"2024-09-10T18:55:40.398243+0000","last_unstale":"2024-09-10T19:06:30.174172+0000","last_undegraded":"2024-09-10T19:06:30.174172+0000","last_fullsized":"2024-09-10T19:06:30.174172+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:22.258058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:03.571786+0000","last_clean_scrub_stamp":"2024-09-10T19:02:22.258058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0054525670000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":453,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:32.189097+0000","last_change":"2024-09-10T19:03:22.930232+0000","last_active":"2024-09-10T19:06:32.189097+0000","last_peered":"2024-09-10T19:06:32.189097+0000","last_clean":"2024-09-10T19:06:32.189097+0000","last_became_active":"2024-09-10T19:00:42.516833+0000","last_became_peered":"2024-09-10T19:00:42.516833+0000","last_unstale":"2024-09-10T19:06:32.189097+0000","last_undegraded":"2024-09-10T19:06:32.189097+0000","last_fullsized":"2024-09-10T19:06:32.189097+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:22.930166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:04.631558+0000","last_clean_scrub_stamp":"2024-09-10T19:03:22.930166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001481163,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"427'32","reported_seq":504,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169309+0000","last_change":"2024-09-10T19:03:37.562262+0000","last_active":"2024-09-10T19:06:30.169309+0000","last_peered":"2024-09-10T19:06:30.169309+0000","last_clean":"2024-09-10T19:06:30.169309+0000","last_became_active":"2024-09-10T18:45:20.056885+0000","last_became_peered":"2024-09-10T18:45:20.056885+0000","last_unstale":"2024-09-10T19:06:30.169309+0000","last_undegraded":"2024-09-10T19:06:30.169309+0000","last_fullsized":"2024-09-10T19:06:30.169309+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T19:03:37.562188+0000","last_deep_scrub":"223'16","last_deep_scrub_stamp":"2024-09-10T18:54:47.324044+0000","last_clean_scrub_stamp":"2024-09-10T19:03:37.562188+0000","objects_scrubbed":5,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041736869999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":341,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:32.121093+0000","last_change":"2024-09-10T19:06:32.121019+0000","last_active":"2024-09-10T19:06:32.121093+0000","last_peered":"2024-09-10T19:06:32.121093+0000","last_clean":"2024-09-10T19:06:32.121093+0000","last_became_active":"2024-09-10T19:02:49.572748+0000","last_became_peered":"2024-09-10T19:02:49.572748+0000","last_unstale":"2024-09-10T19:06:32.121093+0000","last_undegraded":"2024-09-10T19:06:32.121093+0000","last_fullsized":"2024-09-10T19:06:32.121093+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:32.120944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:00.029389+0000","last_clean_scrub_stamp":"2024-09-10T19:06:32.120944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:35.654481+0000","scrub_duration":0.0048228450000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":89,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.166850+0000","last_change":"2024-09-10T19:02:49.573280+0000","last_active":"2024-09-10T19:06:30.166850+0000","last_peered":"2024-09-10T19:06:30.166850+0000","last_clean":"2024-09-10T19:06:30.166850+0000","last_became_active":"2024-09-10T19:02:49.572998+0000","last_became_peered":"2024-09-10T19:02:49.572998+0000","last_unstale":"2024-09-10T19:06:30.166850+0000","last_undegraded":"2024-09-10T19:06:30.166850+0000","last_fullsized":"2024-09-10T19:06:30.166850+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:15.321767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:02:15.321767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":469,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.173208+0000","last_change":"2024-09-10T19:06:08.922418+0000","last_active":"2024-09-10T19:06:30.173208+0000","last_peered":"2024-09-10T19:06:30.173208+0000","last_clean":"2024-09-10T19:06:30.173208+0000","last_became_active":"2024-09-10T18:55:40.395595+0000","last_became_peered":"2024-09-10T18:55:40.395595+0000","last_unstale":"2024-09-10T19:06:30.173208+0000","last_undegraded":"2024-09-10T19:06:30.173208+0000","last_fullsized":"2024-09-10T19:06:30.173208+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:08.922339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:06:08.922339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:33.632921+0000","scrub_duration":0.0055359140000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"427'12","reported_seq":474,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.168403+0000","last_change":"2024-09-10T19:02:50.391929+0000","last_active":"2024-09-10T19:06:30.168403+0000","last_peered":"2024-09-10T19:06:30.168403+0000","last_clean":"2024-09-10T19:06:30.168403+0000","last_became_active":"2024-09-10T18:45:20.059205+0000","last_became_peered":"2024-09-10T18:45:20.059205+0000","last_unstale":"2024-09-10T19:06:30.168403+0000","last_undegraded":"2024-09-10T19:06:30.168403+0000","last_fullsized":"2024-09-10T19:06:30.168403+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'8","last_scrub_stamp":"2024-09-10T19:02:50.391489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:50.391489+0000","objects_scrubbed":2,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041757890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":16,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":476,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169085+0000","last_change":"2024-09-10T19:02:16.704889+0000","last_active":"2024-09-10T19:06:30.169085+0000","last_peered":"2024-09-10T19:06:30.169085+0000","last_clean":"2024-09-10T19:06:30.169085+0000","last_became_active":"2024-09-10T18:47:36.415866+0000","last_became_peered":"2024-09-10T18:47:36.415866+0000","last_unstale":"2024-09-10T19:06:30.169085+0000","last_undegraded":"2024-09-10T19:06:30.169085+0000","last_fullsized":"2024-09-10T19:06:30.169085+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:16.704805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:16.704805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0083395729999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":497,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.164657+0000","last_change":"2024-09-10T19:06:20.478219+0000","last_active":"2024-09-10T19:06:30.164657+0000","last_peered":"2024-09-10T19:06:30.164657+0000","last_clean":"2024-09-10T19:06:30.164657+0000","last_became_active":"2024-09-10T18:57:02.453974+0000","last_became_peered":"2024-09-10T18:57:02.453974+0000","last_unstale":"2024-09-10T19:06:30.164657+0000","last_undegraded":"2024-09-10T19:06:30.164657+0000","last_fullsized":"2024-09-10T19:06:30.164657+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:20.478081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_clean_scrub_stamp":"2024-09-10T19:06:20.478081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:35.087111+0000","scrub_duration":0.0034106200000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":257,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169565+0000","last_change":"2024-09-10T19:04:32.388680+0000","last_active":"2024-09-10T19:06:30.169565+0000","last_peered":"2024-09-10T19:06:30.169565+0000","last_clean":"2024-09-10T19:06:30.169565+0000","last_became_active":"2024-09-10T18:57:02.447566+0000","last_became_peered":"2024-09-10T18:57:02.447566+0000","last_unstale":"2024-09-10T19:06:30.169565+0000","last_undegraded":"2024-09-10T19:06:30.169565+0000","last_fullsized":"2024-09-10T19:06:30.169565+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:32.388609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:55.145638+0000","last_clean_scrub_stamp":"2024-09-10T19:04:32.388609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0098792380000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.14","version":"70'81","reported_seq":371,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:31.848167+0000","last_change":"2024-09-10T19:06:31.848113+0000","last_active":"2024-09-10T19:06:31.848167+0000","last_peered":"2024-09-10T19:06:31.848167+0000","last_clean":"2024-09-10T19:06:31.848167+0000","last_became_active":"2024-09-10T18:57:02.442457+0000","last_became_peered":"2024-09-10T18:57:02.442457+0000","last_unstale":"2024-09-10T19:06:31.848167+0000","last_undegraded":"2024-09-10T19:06:31.848167+0000","last_fullsized":"2024-09-10T19:06:31.848167+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:06:31.848054+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:57.184383+0000","last_clean_scrub_stamp":"2024-09-10T19:06:31.848054+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:58.376194+0000","scrub_duration":0.0083693320000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.11","version":"427'32","reported_seq":526,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.172960+0000","last_change":"2024-09-10T19:03:43.653800+0000","last_active":"2024-09-10T19:06:30.172960+0000","last_peered":"2024-09-10T19:06:30.172960+0000","last_clean":"2024-09-10T19:06:30.172960+0000","last_became_active":"2024-09-10T18:57:02.461909+0000","last_became_peered":"2024-09-10T18:57:02.461909+0000","last_unstale":"2024-09-10T19:06:30.172960+0000","last_undegraded":"2024-09-10T19:06:30.172960+0000","last_fullsized":"2024-09-10T19:06:30.172960+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T19:03:43.653711+0000","last_deep_scrub":"79'16","last_deep_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_clean_scrub_stamp":"2024-09-10T19:03:43.653711+0000","objects_scrubbed":5,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037824379999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"56'1","reported_seq":505,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.162380+0000","last_change":"2024-09-10T19:06:06.554681+0000","last_active":"2024-09-10T19:06:30.162380+0000","last_peered":"2024-09-10T19:06:30.162380+0000","last_clean":"2024-09-10T19:06:30.162380+0000","last_became_active":"2024-09-10T18:47:36.421985+0000","last_became_peered":"2024-09-10T18:47:36.421985+0000","last_unstale":"2024-09-10T19:06:30.162380+0000","last_undegraded":"2024-09-10T19:06:30.162380+0000","last_fullsized":"2024-09-10T19:06:30.162380+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T19:06:06.554594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:06:06.554594+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:23.803720+0000","scrub_duration":0.0030413559999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":214,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169251+0000","last_change":"2024-09-10T19:06:28.848162+0000","last_active":"2024-09-10T19:06:30.169251+0000","last_peered":"2024-09-10T19:06:30.169251+0000","last_clean":"2024-09-10T19:06:30.169251+0000","last_became_active":"2024-09-10T18:57:02.435124+0000","last_became_peered":"2024-09-10T18:57:02.435124+0000","last_unstale":"2024-09-10T19:06:30.169251+0000","last_undegraded":"2024-09-10T19:06:30.169251+0000","last_fullsized":"2024-09-10T19:06:30.169251+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:28.848105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:06:28.848105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:51.629925+0000","scrub_duration":0.0051945860000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":479,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.171795+0000","last_change":"2024-09-10T19:06:20.199145+0000","last_active":"2024-09-10T19:06:30.171795+0000","last_peered":"2024-09-10T19:06:30.171795+0000","last_clean":"2024-09-10T19:06:30.171795+0000","last_became_active":"2024-09-10T18:55:40.397523+0000","last_became_peered":"2024-09-10T18:55:40.397523+0000","last_unstale":"2024-09-10T19:06:30.171795+0000","last_undegraded":"2024-09-10T19:06:30.171795+0000","last_fullsized":"2024-09-10T19:06:30.171795+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:20.199071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:06:20.199071+0000","last_clean_scrub_stamp":"2024-09-10T19:06:20.199071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:32.556730+0000","scrub_duration":0.0051592299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.11","version":"449'11899","reported_seq":12022,"reported_epoch":449,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-10T19:06:39.897022+0000","last_change":"2024-09-10T19:06:39.897022+0000","last_active":"2024-09-10T19:06:39.897022+0000","last_peered":"2024-09-10T19:06:39.897022+0000","last_clean":"2024-09-10T19:04:37.944310+0000","last_became_active":"2024-09-10T19:04:39.870192+0000","last_became_peered":"2024-09-10T19:04:39.870192+0000","last_unstale":"2024-09-10T19:06:39.897022+0000","last_undegraded":"2024-09-10T19:06:39.897022+0000","last_fullsized":"2024-09-10T19:06:39.897022+0000","mapping_epoch":407,"log_start":"427'8621","ondisk_log_start":"427'8621","created":404,"last_epoch_clean":405,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3278,"log_dups_size":0,"ondisk_log_size":3278,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8491,"num_write_kb":270816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,0],"avail_no_missing":["3","0","5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"78'2","reported_seq":257,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.167260+0000","last_change":"2024-09-10T19:06:28.113529+0000","last_active":"2024-09-10T19:06:30.167260+0000","last_peered":"2024-09-10T19:06:30.167260+0000","last_clean":"2024-09-10T19:06:30.167260+0000","last_became_active":"2024-09-10T18:57:02.447198+0000","last_became_peered":"2024-09-10T18:57:02.447198+0000","last_unstale":"2024-09-10T19:06:30.167260+0000","last_undegraded":"2024-09-10T19:06:30.167260+0000","last_fullsized":"2024-09-10T19:06:30.167260+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'2","last_scrub_stamp":"2024-09-10T19:06:28.113436+0000","last_deep_scrub":"78'2","last_deep_scrub_stamp":"2024-09-10T19:02:15.686138+0000","last_clean_scrub_stamp":"2024-09-10T19:06:28.113436+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:35.604709+0000","scrub_duration":0.0025017680000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":480,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.165878+0000","last_change":"2024-09-10T19:03:44.943009+0000","last_active":"2024-09-10T19:06:30.165878+0000","last_peered":"2024-09-10T19:06:30.165878+0000","last_clean":"2024-09-10T19:06:30.165878+0000","last_became_active":"2024-09-10T18:57:02.443353+0000","last_became_peered":"2024-09-10T18:57:02.443353+0000","last_unstale":"2024-09-10T19:06:30.165878+0000","last_undegraded":"2024-09-10T19:06:30.165878+0000","last_fullsized":"2024-09-10T19:06:30.165878+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:44.942954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:03:44.942954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016874889999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"427'30","reported_seq":512,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:32.212897+0000","last_change":"2024-09-10T19:04:31.733896+0000","last_active":"2024-09-10T19:06:32.212897+0000","last_peered":"2024-09-10T19:06:32.212897+0000","last_clean":"2024-09-10T19:06:32.212897+0000","last_became_active":"2024-09-10T18:57:02.452473+0000","last_became_peered":"2024-09-10T18:57:02.452473+0000","last_unstale":"2024-09-10T19:06:32.212897+0000","last_undegraded":"2024-09-10T19:06:32.212897+0000","last_fullsized":"2024-09-10T19:06:32.212897+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'20","last_scrub_stamp":"2024-09-10T19:04:31.733806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:04:31.733806+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051209239999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":38,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"273'10","reported_seq":760,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:36.328432+0000","last_change":"2024-09-10T19:03:24.245622+0000","last_active":"2024-09-10T19:06:36.328432+0000","last_peered":"2024-09-10T19:06:36.328432+0000","last_clean":"2024-09-10T19:06:36.328432+0000","last_became_active":"2024-09-10T18:57:02.441763+0000","last_became_peered":"2024-09-10T18:57:02.441763+0000","last_unstale":"2024-09-10T19:06:36.328432+0000","last_undegraded":"2024-09-10T19:06:36.328432+0000","last_fullsized":"2024-09-10T19:06:36.328432+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'10","last_scrub_stamp":"2024-09-10T19:03:24.245532+0000","last_deep_scrub":"190'9","last_deep_scrub_stamp":"2024-09-10T18:53:07.746309+0000","last_clean_scrub_stamp":"2024-09-10T19:03:24.245532+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019049480000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"449'11863","reported_seq":21580,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:37.833444+0000","last_change":"2024-09-10T19:05:38.890706+0000","last_active":"2024-09-10T19:06:37.833444+0000","last_peered":"2024-09-10T19:06:37.833444+0000","last_clean":"2024-09-10T19:06:37.833444+0000","last_became_active":"2024-09-10T19:05:38.851253+0000","last_became_peered":"2024-09-10T19:05:38.851253+0000","last_unstale":"2024-09-10T19:06:37.833444+0000","last_undegraded":"2024-09-10T19:06:37.833444+0000","last_fullsized":"2024-09-10T19:06:37.833444+0000","mapping_epoch":433,"log_start":"423'8241","ondisk_log_start":"423'8241","created":404,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3622,"log_dups_size":0,"ondisk_log_size":3622,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8456,"num_write_kb":270560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3202,"num_bytes_recovered":209846272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":89,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.167383+0000","last_change":"2024-09-10T19:02:49.582100+0000","last_active":"2024-09-10T19:06:30.167383+0000","last_peered":"2024-09-10T19:06:30.167383+0000","last_clean":"2024-09-10T19:06:30.167383+0000","last_became_active":"2024-09-10T19:02:49.581739+0000","last_became_peered":"2024-09-10T19:02:49.581739+0000","last_unstale":"2024-09-10T19:06:30.167383+0000","last_undegraded":"2024-09-10T19:06:30.167383+0000","last_fullsized":"2024-09-10T19:06:30.167383+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:23.327767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:03.475299+0000","last_clean_scrub_stamp":"2024-09-10T19:02:23.327767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":475,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.172563+0000","last_change":"2024-09-10T19:04:34.504898+0000","last_active":"2024-09-10T19:06:30.172563+0000","last_peered":"2024-09-10T19:06:30.172563+0000","last_clean":"2024-09-10T19:06:30.172563+0000","last_became_active":"2024-09-10T18:55:40.392524+0000","last_became_peered":"2024-09-10T18:55:40.392524+0000","last_unstale":"2024-09-10T19:06:30.172563+0000","last_undegraded":"2024-09-10T19:06:30.172563+0000","last_fullsized":"2024-09-10T19:06:30.172563+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:34.504828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:04:34.504828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016884452000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"427'27","reported_seq":525,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.162453+0000","last_change":"2024-09-10T19:04:33.326902+0000","last_active":"2024-09-10T19:06:30.162453+0000","last_peered":"2024-09-10T19:06:30.162453+0000","last_clean":"2024-09-10T19:06:30.162453+0000","last_became_active":"2024-09-10T18:45:20.058717+0000","last_became_peered":"2024-09-10T18:45:20.058717+0000","last_unstale":"2024-09-10T19:06:30.162453+0000","last_undegraded":"2024-09-10T19:06:30.162453+0000","last_fullsized":"2024-09-10T19:06:30.162453+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T19:04:33.326849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:04:33.326849+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0036326280000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":441,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:32.214811+0000","last_change":"2024-09-10T19:03:25.864320+0000","last_active":"2024-09-10T19:06:32.214811+0000","last_peered":"2024-09-10T19:06:32.214811+0000","last_clean":"2024-09-10T19:06:32.214811+0000","last_became_active":"2024-09-10T18:57:02.453464+0000","last_became_peered":"2024-09-10T18:57:02.453464+0000","last_unstale":"2024-09-10T19:06:32.214811+0000","last_undegraded":"2024-09-10T19:06:32.214811+0000","last_fullsized":"2024-09-10T19:06:32.214811+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:25.864264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:03:25.864264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013441410000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.13","version":"449'11909","reported_seq":23935,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:37.877106+0000","last_change":"2024-09-10T19:05:08.811448+0000","last_active":"2024-09-10T19:06:37.877106+0000","last_peered":"2024-09-10T19:06:37.877106+0000","last_clean":"2024-09-10T19:06:37.877106+0000","last_became_active":"2024-09-10T19:05:08.804974+0000","last_became_peered":"2024-09-10T19:05:08.804974+0000","last_unstale":"2024-09-10T19:06:37.877106+0000","last_undegraded":"2024-09-10T19:06:37.877106+0000","last_fullsized":"2024-09-10T19:06:37.877106+0000","mapping_epoch":420,"log_start":"424'8559","ondisk_log_start":"424'8559","created":404,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3350,"log_dups_size":0,"ondisk_log_size":3350,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":294912,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8473,"num_write_kb":271264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3971,"num_bytes_recovered":260243456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":452,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.170962+0000","last_change":"2024-09-10T19:03:43.645318+0000","last_active":"2024-09-10T19:06:30.170962+0000","last_peered":"2024-09-10T19:06:30.170962+0000","last_clean":"2024-09-10T19:06:30.170962+0000","last_became_active":"2024-09-10T18:45:24.267251+0000","last_became_peered":"2024-09-10T18:45:24.267251+0000","last_unstale":"2024-09-10T19:06:30.170962+0000","last_undegraded":"2024-09-10T19:06:30.170962+0000","last_fullsized":"2024-09-10T19:06:30.170962+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:43.645260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:03:43.645260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003834732,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"306'2","reported_seq":498,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.164072+0000","last_change":"2024-09-10T19:03:00.120585+0000","last_active":"2024-09-10T19:06:30.164072+0000","last_peered":"2024-09-10T19:06:30.164072+0000","last_clean":"2024-09-10T19:06:30.164072+0000","last_became_active":"2024-09-10T18:57:02.435758+0000","last_became_peered":"2024-09-10T18:57:02.435758+0000","last_unstale":"2024-09-10T19:06:30.164072+0000","last_undegraded":"2024-09-10T19:06:30.164072+0000","last_fullsized":"2024-09-10T19:06:30.164072+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"306'2","last_scrub_stamp":"2024-09-10T19:03:00.120529+0000","last_deep_scrub":"306'2","last_deep_scrub_stamp":"2024-09-10T18:59:20.534869+0000","last_clean_scrub_stamp":"2024-09-10T19:03:00.120529+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016153859999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"427'46","reported_seq":424,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169583+0000","last_change":"2024-09-10T19:04:38.485166+0000","last_active":"2024-09-10T19:06:30.169583+0000","last_peered":"2024-09-10T19:06:30.169583+0000","last_clean":"2024-09-10T19:06:30.169583+0000","last_became_active":"2024-09-10T19:02:49.594207+0000","last_became_peered":"2024-09-10T19:02:49.594207+0000","last_unstale":"2024-09-10T19:06:30.169583+0000","last_undegraded":"2024-09-10T19:06:30.169583+0000","last_fullsized":"2024-09-10T19:06:30.169583+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'34","last_scrub_stamp":"2024-09-10T19:04:38.485104+0000","last_deep_scrub":"258'34","last_deep_scrub_stamp":"2024-09-10T18:56:23.838951+0000","last_clean_scrub_stamp":"2024-09-10T19:04:38.485104+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001834518,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":317,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169906+0000","last_change":"2024-09-10T19:02:49.581903+0000","last_active":"2024-09-10T19:06:30.169906+0000","last_peered":"2024-09-10T19:06:30.169906+0000","last_clean":"2024-09-10T19:06:30.169906+0000","last_became_active":"2024-09-10T19:02:49.581575+0000","last_became_peered":"2024-09-10T19:02:49.581575+0000","last_unstale":"2024-09-10T19:06:30.169906+0000","last_undegraded":"2024-09-10T19:06:30.169906+0000","last_fullsized":"2024-09-10T19:06:30.169906+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:03.343366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:02:03.343366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0079641550000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.12","version":"449'12384","reported_seq":20676,"reported_epoch":449,"state":"active+clean","last_fresh":"2024-09-10T19:06:37.886492+0000","last_change":"2024-09-10T19:05:03.855780+0000","last_active":"2024-09-10T19:06:37.886492+0000","last_peered":"2024-09-10T19:06:37.886492+0000","last_clean":"2024-09-10T19:06:37.886492+0000","last_became_active":"2024-09-10T19:05:03.829776+0000","last_became_peered":"2024-09-10T19:05:03.829776+0000","last_unstale":"2024-09-10T19:06:37.886492+0000","last_undegraded":"2024-09-10T19:06:37.886492+0000","last_fullsized":"2024-09-10T19:06:37.886492+0000","mapping_epoch":416,"log_start":"424'8734","ondisk_log_start":"424'8734","created":404,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:14.808148+0000","last_clean_scrub_stamp":"2024-09-10T19:03:14.808148+0000","objects_scrubbed":0,"log_size":3650,"log_dups_size":0,"ondisk_log_size":3650,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8836,"num_write_kb":282720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4090,"num_bytes_recovered":268042240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":472,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169011+0000","last_change":"2024-09-10T19:03:33.638108+0000","last_active":"2024-09-10T19:06:30.169011+0000","last_peered":"2024-09-10T19:06:30.169011+0000","last_clean":"2024-09-10T19:06:30.169011+0000","last_became_active":"2024-09-10T19:00:42.520239+0000","last_became_peered":"2024-09-10T19:00:42.520239+0000","last_unstale":"2024-09-10T19:06:30.169011+0000","last_undegraded":"2024-09-10T19:06:30.169011+0000","last_fullsized":"2024-09-10T19:06:30.169011+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:33.638010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:17.985545+0000","last_clean_scrub_stamp":"2024-09-10T19:03:33.638010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0035871750000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":478,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.174433+0000","last_change":"2024-09-10T19:03:36.615878+0000","last_active":"2024-09-10T19:06:30.174433+0000","last_peered":"2024-09-10T19:06:30.174433+0000","last_clean":"2024-09-10T19:06:30.174433+0000","last_became_active":"2024-09-10T18:57:02.460855+0000","last_became_peered":"2024-09-10T18:57:02.460855+0000","last_unstale":"2024-09-10T19:06:30.174433+0000","last_undegraded":"2024-09-10T19:06:30.174433+0000","last_fullsized":"2024-09-10T19:06:30.174433+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:36.615788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:03:36.615788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004374223,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"427'29","reported_seq":502,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169933+0000","last_change":"2024-09-10T19:03:40.698360+0000","last_active":"2024-09-10T19:06:30.169933+0000","last_peered":"2024-09-10T19:06:30.169933+0000","last_clean":"2024-09-10T19:06:30.169933+0000","last_became_active":"2024-09-10T18:45:20.058391+0000","last_became_peered":"2024-09-10T18:45:20.058391+0000","last_unstale":"2024-09-10T19:06:30.169933+0000","last_undegraded":"2024-09-10T19:06:30.169933+0000","last_fullsized":"2024-09-10T19:06:30.169933+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"388'23","last_scrub_stamp":"2024-09-10T19:03:40.697702+0000","last_deep_scrub":"388'23","last_deep_scrub_stamp":"2024-09-10T19:03:40.697702+0000","last_clean_scrub_stamp":"2024-09-10T19:03:40.697702+0000","objects_scrubbed":5,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.076817413000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":484,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-10T19:06:30.169577+0000","last_change":"2024-09-10T19:03:46.623850+0000","last_active":"2024-09-10T19:06:30.169577+0000","last_peered":"2024-09-10T19:06:30.169577+0000","last_clean":"2024-09-10T19:06:30.169577+0000","last_became_active":"2024-09-10T18:57:02.449135+0000","last_became_peered":"2024-09-10T18:57:02.449135+0000","last_unstale":"2024-09-10T19:06:30.169577+0000","last_undegraded":"2024-09-10T19:06:30.169577+0000","last_fullsized":"2024-09-10T19:06:30.169577+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:46.623782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:03:46.623782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0075382380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":26,"stat_sum":{"num_bytes":11501568,"num_objects":173,"num_object_clones":0,"num_object_copies":519,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":265955,"num_write_kb":8499457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31487,"num_bytes_recovered":2063466496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":815005696,"data_stored":815005696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88635,"ondisk_log_size":88635,"up":78,"acting":78,"num_store_stats":6},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":421,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1544,"num_read_kb":1301,"num_write":989,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":3152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":499712,"data_stored":13268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1057,"ondisk_log_size":1057,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":344,"num_read_kb":339,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":262688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1701,"ondisk_log_size":1701,"up":63,"acting":63,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":0,"seq":1765231558679,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":43,"seq":184683593994,"num_pgs":96,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":965936,"kb_used_data":71784,"kb_used_omap":1,"kb_used_meta":894142,"kb_avail":92779216,"statfs":{"total":95995035648,"available":95005917184,"internally_reserved":0,"allocated":73506816,"data_stored":73135593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":915601752},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:06:07 2024","interfaces":[{"interface":"back","average":{"1min":6.4509999999999996,"5min":3.0209999999999999,"15min":1.698},"min":{"1min":0.217,"5min":0.217,"15min":0.20899999999999999},"max":{"1min":44.320999999999998,"5min":49.353999999999999,"15min":49.353999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":6.7450000000000001,"5min":3.202,"15min":1.819},"min":{"1min":0.23799999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":44.311999999999998,"5min":49.341999999999999,"15min":49.341999999999999},"last":0.63900000000000001}]},{"osd":1,"last update":"Tue Sep 10 19:06:18 2024","interfaces":[{"interface":"back","average":{"1min":4.9950000000000001,"5min":2.194,"15min":1.728},"min":{"1min":0.23200000000000001,"5min":0.217,"15min":0.217},"max":{"1min":85.775000000000006,"5min":85.775000000000006,"15min":85.775000000000006},"last":0.47099999999999997},{"interface":"front","average":{"1min":4.9580000000000002,"5min":2.0550000000000002,"15min":1.621},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.23300000000000001},"max":{"1min":85.670000000000002,"5min":85.670000000000002,"15min":85.670000000000002},"last":0.625}]},{"osd":2,"last update":"Tue Sep 10 19:06:18 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":0.999,"15min":0.86299999999999999},"min":{"1min":0.27000000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":4.0469999999999997,"5min":23.867000000000001,"15min":23.867000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":1.149,"15min":0.90100000000000002},"min":{"1min":0.29399999999999998,"5min":0.28699999999999998,"15min":0.249},"max":{"1min":2.1360000000000001,"5min":23.779,"15min":23.779},"last":0.51100000000000001}]},{"osd":3,"last update":"Tue Sep 10 19:06:18 2024","interfaces":[{"interface":"back","average":{"1min":1.8720000000000001,"5min":1.758,"15min":1.742},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":20.994,"5min":47.325000000000003,"15min":76.977999999999994},"last":0.53200000000000003},{"interface":"front","average":{"1min":1.9870000000000001,"5min":1.7689999999999999,"15min":1.776},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.253},"max":{"1min":20.177,"5min":46.793999999999997,"15min":76.879999999999995},"last":0.54300000000000004}]},{"osd":4,"last update":"Tue Sep 10 19:06:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":1.1060000000000001,"15min":1.5109999999999999},"min":{"1min":0.26100000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.9039999999999999,"5min":32.469000000000001,"15min":165.03299999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.628,"5min":0.877,"15min":1.365},"min":{"1min":0.30399999999999999,"5min":0.252,"15min":0.22500000000000001},"max":{"1min":3.0819999999999999,"5min":32.5,"15min":163.959},"last":0.40899999999999997}]},{"osd":5,"last update":"Tue Sep 10 19:06:18 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.077,"15min":1.9259999999999999},"min":{"1min":0.29499999999999998,"5min":0.246,"15min":0.246},"max":{"1min":2.968,"5min":23.827000000000002,"15min":288.56999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.626,"5min":1.0089999999999999,"15min":1.897},"min":{"1min":0.32900000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":2.3809999999999998,"5min":23.667000000000002,"15min":291.05399999999997},"last":0.55500000000000005}]},{"osd":7,"last update":"Tue Sep 10 19:05:52 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":2.794,"5min":2.794,"15min":2.794},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.61199999999999999}]}]},{"osd":5,"up_from":37,"seq":158913790222,"num_pgs":90,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1222704,"kb_used_data":42344,"kb_used_omap":1,"kb_used_meta":1180350,"kb_avail":92522448,"statfs":{"total":95995035648,"available":94742986752,"internally_reserved":0,"allocated":43360256,"data_stored":42988062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":1208678744},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:05:58 2024","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.5149999999999999,"15min":1.9390000000000001},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":14.31,"5min":31.626999999999999,"15min":82.465000000000003},"last":0.33500000000000002},{"interface":"front","average":{"1min":1.9730000000000001,"5min":1.758,"15min":2.109},"min":{"1min":0.28000000000000003,"5min":0.26500000000000001,"15min":0.218},"max":{"1min":14.323,"5min":31.734999999999999,"15min":82.519000000000005},"last":0.35299999999999998}]},{"osd":1,"last update":"Tue Sep 10 19:05:58 2024","interfaces":[{"interface":"back","average":{"1min":9.7650000000000006,"5min":2.9460000000000002,"15min":2.282},"min":{"1min":0.23300000000000001,"5min":0.22500000000000001,"15min":0.19500000000000001},"max":{"1min":153.17099999999999,"5min":153.17099999999999,"15min":153.17099999999999},"last":3.0859999999999999},{"interface":"front","average":{"1min":9.7599999999999998,"5min":3.1000000000000001,"15min":2.4369999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.20399999999999999},"max":{"1min":153.196,"5min":153.196,"15min":153.196},"last":2.923}]},{"osd":2,"last update":"Tue Sep 10 19:05:58 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":1.0309999999999999,"15min":0.89200000000000002},"min":{"1min":0.29299999999999998,"5min":0.249,"15min":0.249},"max":{"1min":2.5619999999999998,"5min":31.774000000000001,"15min":31.774000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.97699999999999998,"15min":0.85599999999999998},"min":{"1min":0.28000000000000003,"5min":0.27200000000000002,"15min":0.24399999999999999},"max":{"1min":2.8799999999999999,"5min":31.722999999999999,"15min":31.722999999999999},"last":0.436}]},{"osd":3,"last update":"Tue Sep 10 19:05:58 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":1.2250000000000001,"15min":2.5870000000000002},"min":{"1min":0.33100000000000002,"5min":0.26300000000000001,"15min":0.24299999999999999},"max":{"1min":2.5800000000000001,"5min":31.753,"15min":123.34399999999999},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.159,"5min":1.347,"15min":2.1989999999999998},"min":{"1min":0.29899999999999999,"5min":0.26200000000000001,"15min":0.245},"max":{"1min":5.117,"5min":31.699999999999999,"15min":82.415000000000006},"last":0.40500000000000003}]},{"osd":4,"last update":"Tue Sep 10 19:05:58 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":1.381,"15min":1.1479999999999999},"min":{"1min":0.20999999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":1.7749999999999999,"5min":51.070999999999998,"15min":51.070999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.75700000000000001,"5min":1.2629999999999999,"15min":1.0509999999999999},"min":{"1min":0.25800000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":3.524,"5min":51.122999999999998,"15min":51.122999999999998},"last":0.42599999999999999}]},{"osd":6,"last update":"Tue Sep 10 19:06:20 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":1.2430000000000001,"15min":1.127},"min":{"1min":0.24199999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":4.0490000000000004,"5min":33.936999999999998,"15min":33.936999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.2230000000000001,"15min":1.0920000000000001},"min":{"1min":0.307,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":5.1710000000000003,"5min":33.951000000000001,"15min":33.951000000000001},"last":0.45600000000000002}]},{"osd":7,"last update":"Tue Sep 10 19:05:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":2.657,"5min":2.657,"15min":2.657},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.58599999999999997}]}]},{"osd":4,"up_from":30,"seq":128849019156,"num_pgs":84,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1302376,"kb_used_data":119520,"kb_used_omap":1,"kb_used_meta":1182846,"kb_avail":92442776,"statfs":{"total":95995035648,"available":94661402624,"internally_reserved":0,"allocated":122388480,"data_stored":122024471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":1211234648},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:05:59 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":1.514,"15min":1.4830000000000001},"min":{"1min":0.251,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":2.819,"5min":40.481000000000002,"15min":46.234999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.93200000000000005,"5min":1.5649999999999999,"15min":1.4850000000000001},"min":{"1min":0.35499999999999998,"5min":0.253,"15min":0.253},"max":{"1min":2.8380000000000001,"5min":40.512,"15min":46.264000000000003},"last":0.65700000000000003}]},{"osd":1,"last update":"Tue Sep 10 19:06:38 2024","interfaces":[{"interface":"back","average":{"1min":2.1640000000000001,"5min":1.3720000000000001,"15min":1.7350000000000001},"min":{"1min":0.28799999999999998,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":16.535,"5min":16.535,"15min":125.15600000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":1.8280000000000001,"5min":1.081,"15min":1.619},"min":{"1min":0.28199999999999997,"5min":0.23300000000000001,"15min":0.23100000000000001},"max":{"1min":16.504000000000001,"5min":16.504000000000001,"15min":125.14700000000001},"last":0.59399999999999997}]},{"osd":2,"last update":"Tue Sep 10 19:06:38 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.68500000000000005,"15min":0.72599999999999998},"min":{"1min":0.28999999999999998,"5min":0.28499999999999998,"15min":0.25900000000000001},"max":{"1min":2.0350000000000001,"5min":3.3999999999999999,"15min":5.6189999999999998},"last":0.63},{"interface":"front","average":{"1min":0.67000000000000004,"5min":2.7599999999999998,"15min":1.5229999999999999},"min":{"1min":0.35599999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":2.3519999999999999,"5min":190.38200000000001,"15min":190.38200000000001},"last":0.48899999999999999}]},{"osd":3,"last update":"Tue Sep 10 19:06:38 2024","interfaces":[{"interface":"back","average":{"1min":2.2549999999999999,"5min":4.0279999999999996,"15min":3.0649999999999999},"min":{"1min":0.38300000000000001,"5min":0.27200000000000002,"15min":0.23799999999999999},"max":{"1min":17.068999999999999,"5min":190.41300000000001,"15min":190.41300000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":2.1320000000000001,"5min":1.863,"15min":1.9690000000000001},"min":{"1min":0.30199999999999999,"5min":0.26000000000000001,"15min":0.25700000000000001},"max":{"1min":17.120999999999999,"5min":39.323,"15min":110.914},"last":0.60699999999999998}]},{"osd":5,"last update":"Tue Sep 10 19:05:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.83099999999999996,"15min":0.96499999999999997},"min":{"1min":0.24299999999999999,"5min":0.221,"15min":0.20499999999999999},"max":{"1min":8.1430000000000007,"5min":8.1430000000000007,"15min":10.912000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":1.05,"5min":0.89700000000000002,"15min":0.94099999999999995},"min":{"1min":0.251,"5min":0.23200000000000001,"15min":0.216},"max":{"1min":8.3170000000000002,"5min":8.3170000000000002,"15min":10.147},"last":0.42099999999999999}]},{"osd":6,"last update":"Tue Sep 10 19:06:23 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.74199999999999999,"15min":0.81399999999999995},"min":{"1min":0.33900000000000002,"5min":0.24099999999999999,"15min":0.221},"max":{"1min":2.3740000000000001,"5min":5.351,"15min":8.4030000000000005},"last":0.50800000000000001},{"interface":"front","average":{"1min":1.0680000000000001,"5min":0.95899999999999996,"15min":1.026},"min":{"1min":0.29499999999999998,"5min":0.26300000000000001,"15min":0.23899999999999999},"max":{"1min":4.0099999999999998,"5min":4.8949999999999996,"15min":21.254000000000001},"last":0.57799999999999996}]},{"osd":7,"last update":"Tue Sep 10 19:05:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":2.347,"5min":2.347,"15min":2.347},"last":0.54300000000000004},{"interface":"front","average":{"1min":9.8569999999999993,"5min":9.8569999999999993,"15min":9.8569999999999993},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":190.33799999999999,"5min":190.33799999999999,"15min":190.33799999999999},"last":0.55300000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182681,"num_pgs":93,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1806320,"kb_used_data":309224,"kb_used_omap":1,"kb_used_meta":1497086,"kb_avail":91938832,"statfs":{"total":95995035648,"available":94145363968,"internally_reserved":0,"allocated":316645376,"data_stored":316273073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":1533016408},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:05:59 2024","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.2210000000000001,"15min":2.4039999999999999},"min":{"1min":0.30599999999999999,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":4.7770000000000001,"5min":5.6349999999999998,"15min":290.721},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.421,"5min":1.3480000000000001,"15min":2.6059999999999999},"min":{"1min":0.33100000000000002,"5min":0.18099999999999999,"15min":0.17499999999999999},"max":{"1min":3.5649999999999999,"5min":12.103,"15min":293.59300000000002},"last":0.59699999999999998}]},{"osd":1,"last update":"Tue Sep 10 19:06:12 2024","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.419,"15min":1.619},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.16200000000000001},"max":{"1min":6.0330000000000004,"5min":16.081,"15min":58.215000000000003},"last":0.505},{"interface":"front","average":{"1min":1.915,"5min":1.6100000000000001,"15min":1.581},"min":{"1min":0.374,"5min":0.23599999999999999,"15min":0.188},"max":{"1min":7.165,"5min":16.329000000000001,"15min":57.896999999999998},"last":0.63500000000000001}]},{"osd":2,"last update":"Tue Sep 10 19:06:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":0.98099999999999998,"15min":0.96599999999999997},"min":{"1min":0.27900000000000003,"5min":0.25700000000000001,"15min":0.22800000000000001},"max":{"1min":3.1629999999999998,"5min":4.2409999999999997,"15min":5.6680000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.72999999999999998,"5min":1.0189999999999999,"15min":0.88900000000000001},"min":{"1min":0.35199999999999998,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":3.0910000000000002,"5min":5.46,"15min":5.46},"last":0.55200000000000005}]},{"osd":4,"last update":"Tue Sep 10 19:05:41 2024","interfaces":[{"interface":"back","average":{"1min":1.673,"5min":1.4950000000000001,"15min":1.417},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.247},"max":{"1min":3.9510000000000001,"5min":16.356000000000002,"15min":29.013000000000002},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.472,"5min":1.4710000000000001,"15min":1.4239999999999999},"min":{"1min":0.35099999999999998,"5min":0.33000000000000002,"15min":0.26800000000000002},"max":{"1min":3.9780000000000002,"5min":17.289999999999999,"15min":28.826000000000001},"last":0.77000000000000002}]},{"osd":5,"last update":"Tue Sep 10 19:05:59 2024","interfaces":[{"interface":"back","average":{"1min":1.645,"5min":1.222,"15min":1.288},"min":{"1min":0.51100000000000001,"5min":0.248,"15min":0.248},"max":{"1min":4.8730000000000002,"5min":5.1920000000000002,"15min":7.2300000000000004},"last":0.82199999999999995},{"interface":"front","average":{"1min":2.1749999999999998,"5min":1.597,"15min":1.3839999999999999},"min":{"1min":0.59099999999999997,"5min":0.28299999999999997,"15min":0.255},"max":{"1min":13.117000000000001,"5min":17.899999999999999,"15min":17.899999999999999},"last":0.79200000000000004}]},{"osd":6,"last update":"Tue Sep 10 19:06:21 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.296,"15min":1.298},"min":{"1min":0.34200000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.9530000000000001,"5min":5.8339999999999996,"15min":5.8339999999999996},"last":0.753},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.236,"15min":1.2450000000000001},"min":{"1min":0.38400000000000001,"5min":0.33000000000000002,"15min":0.29399999999999998},"max":{"1min":4.0019999999999998,"5min":6.6390000000000002,"15min":6.6390000000000002},"last":0.71999999999999997}]},{"osd":7,"last update":"Tue Sep 10 19:05:53 2024","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":4.8499999999999996,"5min":4.8499999999999996,"15min":4.8499999999999996},"last":0.79900000000000004},{"interface":"front","average":{"1min":2.2370000000000001,"5min":2.2370000000000001,"15min":2.2370000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":13.045999999999999,"5min":13.045999999999999,"15min":13.045999999999999},"last":0.81499999999999995}]}]},{"osd":2,"up_from":0,"seq":77309411613,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834575137,"num_pgs":101,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1715276,"kb_used_data":210948,"kb_used_omap":1,"kb_used_meta":1504318,"kb_avail":92029876,"statfs":{"total":95995035648,"available":94238593024,"internally_reserved":0,"allocated":216010752,"data_stored":215611240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":1540421976},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:06:01 2024","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.3480000000000001,"15min":1.8260000000000001},"min":{"1min":0.26800000000000002,"5min":0.158,"15min":0.158},"max":{"1min":4.9829999999999997,"5min":25.370999999999999,"15min":111.014},"last":0.29199999999999998},{"interface":"front","average":{"1min":1.784,"5min":1.5209999999999999,"15min":1.8799999999999999},"min":{"1min":0.30599999999999999,"5min":0.217,"15min":0.19600000000000001},"max":{"1min":5.1029999999999998,"5min":27.809999999999999,"15min":111.072},"last":0.26900000000000002}]},{"osd":2,"last update":"Tue Sep 10 19:05:45 2024","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":0.77900000000000003,"15min":0.79600000000000004},"min":{"1min":0.222,"5min":0.222,"15min":0.186},"max":{"1min":4.835,"5min":4.835,"15min":4.835},"last":0.32000000000000001},{"interface":"front","average":{"1min":1.419,"5min":0.92900000000000005,"15min":0.82499999999999996},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.215},"max":{"1min":9.484,"5min":9.484,"15min":9.484},"last":0.307}]},{"osd":3,"last update":"Tue Sep 10 19:06:13 2024","interfaces":[{"interface":"back","average":{"1min":2.1760000000000002,"5min":2.1859999999999999,"15min":2.3100000000000001},"min":{"1min":0.29399999999999998,"5min":0.218,"15min":0.185},"max":{"1min":6.5149999999999997,"5min":48.582999999999998,"15min":92.069000000000003},"last":0.33300000000000002},{"interface":"front","average":{"1min":1.72,"5min":1.6359999999999999,"15min":1.917},"min":{"1min":0.29199999999999998,"5min":0.23100000000000001,"15min":0.19900000000000001},"max":{"1min":9.8409999999999993,"5min":51.847000000000001,"15min":88.658000000000001},"last":0.249}]},{"osd":4,"last update":"Tue Sep 10 19:06:34 2024","interfaces":[{"interface":"back","average":{"1min":1.649,"5min":1.2450000000000001,"15min":1.2050000000000001},"min":{"1min":0.41299999999999998,"5min":0.28100000000000003,"15min":0.27800000000000002},"max":{"1min":7.2699999999999996,"5min":19.126999999999999,"15min":19.126999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":1.871,"5min":1.431,"15min":1.202},"min":{"1min":0.40699999999999997,"5min":0.26300000000000001,"15min":0.252},"max":{"1min":5.9340000000000002,"5min":20.152000000000001,"15min":20.152000000000001},"last":0.5}]},{"osd":5,"last update":"Tue Sep 10 19:06:01 2024","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.2549999999999999,"15min":1.274},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":3.677,"5min":9.9369999999999994,"15min":10.334},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.97899999999999998,"15min":1.1499999999999999},"min":{"1min":0.34799999999999998,"5min":0.29199999999999998,"15min":0.28799999999999998},"max":{"1min":2.5950000000000002,"5min":4.3600000000000003,"15min":16.039999999999999},"last":0.53400000000000003}]},{"osd":6,"last update":"Tue Sep 10 19:06:13 2024","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.25,"15min":1.458},"min":{"1min":0.435,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":3.7210000000000001,"5min":9.9760000000000009,"15min":33.290999999999997},"last":0.52100000000000002},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.286,"15min":1.4910000000000001},"min":{"1min":0.44500000000000001,"5min":0.32700000000000001,"15min":0.26600000000000001},"max":{"1min":7.7400000000000002,"5min":17.352,"15min":33.310000000000002},"last":0.51000000000000001}]},{"osd":7,"last update":"Tue Sep 10 19:05:51 2024","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":3.6800000000000002,"5min":3.6800000000000002,"15min":3.6800000000000002},"last":0.434},{"interface":"front","average":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":10.010999999999999,"5min":10.010999999999999,"15min":10.010999999999999},"last":0.54700000000000004}]}]},{"osd":0,"up_from":241,"seq":1035087118470,"num_pgs":84,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":981560,"kb_used_data":55980,"kb_used_omap":4,"kb_used_meta":925563,"kb_avail":92763592,"statfs":{"total":95995035648,"available":94989918208,"internally_reserved":0,"allocated":57323520,"data_stored":56953613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":947777030},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 19:05:59 2024","interfaces":[{"interface":"back","average":{"1min":4.3029999999999999,"5min":1.6779999999999999,"15min":2.1160000000000001},"min":{"1min":0.187,"5min":0.184,"15min":0.17100000000000001},"max":{"1min":47.237000000000002,"5min":47.237000000000002,"15min":47.237000000000002},"last":0.32200000000000001},{"interface":"front","average":{"1min":4.117,"5min":1.5669999999999999,"15min":2.0649999999999999},"min":{"1min":0.224,"5min":0.193,"15min":0.193},"max":{"1min":47.198999999999998,"5min":47.198999999999998,"15min":47.198999999999998},"last":0.48899999999999999}]},{"osd":2,"last update":"Tue Sep 10 19:05:59 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.82999999999999996,"15min":0.92200000000000004},"min":{"1min":0.223,"5min":0.223,"15min":0.20699999999999999},"max":{"1min":6.0170000000000003,"5min":6.0170000000000003,"15min":13.305},"last":0.41999999999999998},{"interface":"front","average":{"1min":1.0980000000000001,"5min":0.85799999999999998,"15min":0.86199999999999999},"min":{"1min":0.23100000000000001,"5min":0.22900000000000001,"15min":0.22800000000000001},"max":{"1min":6.1909999999999998,"5min":9.5099999999999998,"15min":13.693},"last":0.373}]},{"osd":3,"last update":"Tue Sep 10 19:05:59 2024","interfaces":[{"interface":"back","average":{"1min":1.5780000000000001,"5min":1.25,"15min":1.2210000000000001},"min":{"1min":0.34999999999999998,"5min":0.216,"15min":0.216},"max":{"1min":6.8380000000000001,"5min":6.8380000000000001,"15min":21.108000000000001},"last":0.436},{"interface":"front","average":{"1min":1.9330000000000001,"5min":1.3400000000000001,"15min":1.627},"min":{"1min":0.26100000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":8.0549999999999997,"5min":8.0549999999999997,"15min":11.629},"last":0.38900000000000001}]},{"osd":4,"last update":"Tue Sep 10 19:05:59 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.111,"15min":0.96799999999999997},"min":{"1min":0.30499999999999999,"5min":0.26400000000000001,"15min":0.26000000000000001},"max":{"1min":5.9560000000000004,"5min":5.9560000000000004,"15min":13.161},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.603,"5min":1.1100000000000001,"15min":1.0660000000000001},"min":{"1min":0.312,"5min":0.28100000000000003,"15min":0.27600000000000002},"max":{"1min":6.3719999999999999,"5min":6.3719999999999999,"15min":6.3719999999999999},"last":0.91000000000000003}]},{"osd":5,"last update":"Tue Sep 10 19:05:59 2024","interfaces":[{"interface":"back","average":{"1min":1.583,"5min":2.2610000000000001,"15min":1.5009999999999999},"min":{"1min":0.34000000000000002,"5min":0.311,"15min":0.31},"max":{"1min":6.5679999999999996,"5min":97.296999999999997,"15min":97.296999999999997},"last":0.73599999999999999},{"interface":"front","average":{"1min":1.1439999999999999,"5min":2.218,"15min":1.379},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":5.7939999999999996,"5min":99.221999999999994,"15min":99.221999999999994},"last":0.82699999999999996}]},{"osd":6,"last update":"Tue Sep 10 19:05:59 2024","interfaces":[{"interface":"back","average":{"1min":9.8659999999999997,"5min":3.0609999999999999,"15min":1.724},"min":{"1min":0.35299999999999998,"5min":0.33600000000000002,"15min":0.312},"max":{"1min":216.21100000000001,"5min":216.21100000000001,"15min":216.21100000000001},"last":0.627},{"interface":"front","average":{"1min":9.5069999999999997,"5min":3.1960000000000002,"15min":1.665},"min":{"1min":0.379,"5min":0.379,"15min":0.313},"max":{"1min":216.054,"5min":216.054,"15min":216.054},"last":0.67400000000000004}]},{"osd":7,"last update":"Tue Sep 10 19:05:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":6.3979999999999997,"5min":6.3979999999999997,"15min":6.3979999999999997},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":6.8700000000000001,"5min":6.8700000000000001,"15min":6.8700000000000001},"last":0.71199999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":2668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":56229888,"data_stored":56229888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":215220224,"data_stored":215220224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":309395456,"data_stored":309395456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":119668736,"data_stored":119668736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":42336256,"data_stored":42336256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":72155136,"data_stored":72155136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T19:06:50.178 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-10T19:06:50.178 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-10T19:06:50.178 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-10T19:06:50.179 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd dump -f json-pretty 2024-09-10T19:06:50.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:49 smithi103 ceph-mon[33163]: pgmap v1011: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 12 MiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 73 op/s 2024-09-10T19:06:50.262 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T19:06:50.263 INFO:teuthology.orchestra.run.smithi018.stdout: "ops": [], 2024-09-10T19:06:50.263 INFO:teuthology.orchestra.run.smithi018.stdout: "complaint_time": 30, 2024-09-10T19:06:50.263 INFO:teuthology.orchestra.run.smithi018.stdout: "num_blocked_ops": 0 2024-09-10T19:06:50.263 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T19:06:50.263 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:50.286 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:50.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:50.287 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:50.367 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-10T19:06:50.387 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:50.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:50.434 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:50.434 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:50.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:06:50 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:06:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:06:50.535 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:50.561 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:50.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:50.562 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:50.663 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:50.688 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:50.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:50.689 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:50.790 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:50.816 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:50.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:50.817 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:50.918 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:50.925 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-10T19:06:50.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:50.945 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:50.945 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:51.046 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:51.071 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:51.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:51.072 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:51.172 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:51.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:06:50 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:06:50] "GET /metrics HTTP/1.1" 200 224459 "" "Prometheus/2.43.0" 2024-09-10T19:06:51.196 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:51.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:51.197 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:51.298 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:51.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:51.325 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:51.325 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:51.426 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:51.450 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:51.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:51.451 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:51.551 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:51.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:51.576 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:51.576 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:51.677 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:51.700 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:51.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:51.701 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:51.802 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:51.805 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:06:51.826 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:51.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:51.827 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:51.847 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:06:51.928 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:51.951 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:51.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:51.952 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:52.053 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:52.058 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:51 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:06:52.058 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:51 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:06:52.058 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:51 smithi018 ceph-mon[29336]: osdmap e452: 8 total, 8 up, 6 in 2024-09-10T19:06:52.058 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:51 smithi018 ceph-mon[29336]: pgmap v1013: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 12 MiB data, 7.4 GiB used, 529 GiB / 536 GiB avail; 644 B/s rd, 0 op/s 2024-09-10T19:06:52.058 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:51 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:06:52.058 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:51 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:06:52.058 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:51 smithi018 ceph-mon[40196]: osdmap e452: 8 total, 8 up, 6 in 2024-09-10T19:06:52.059 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:51 smithi018 ceph-mon[40196]: pgmap v1013: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 12 MiB data, 7.4 GiB used, 529 GiB / 536 GiB avail; 644 B/s rd, 0 op/s 2024-09-10T19:06:52.076 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:52.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:52.077 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:52.178 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:52.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:52.203 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:52.203 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:52.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:51 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:06:52.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:51 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:06:52.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:51 smithi103 ceph-mon[33163]: osdmap e452: 8 total, 8 up, 6 in 2024-09-10T19:06:52.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:51 smithi103 ceph-mon[33163]: pgmap v1013: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 12 MiB data, 7.4 GiB used, 529 GiB / 536 GiB avail; 644 B/s rd, 0 op/s 2024-09-10T19:06:52.304 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:52.328 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:52.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:52.329 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:52.430 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:52.453 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:52.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:52.454 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:52.556 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:52.580 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:52.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:52.581 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:52.682 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:52.706 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:52.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:52.707 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:52.808 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:52.831 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:52.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:52.832 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:52.933 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:52.959 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:52.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:52.960 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:53.061 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:53.085 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:53.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:53.086 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:53.187 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:53.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:53.213 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:53.214 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:53.314 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:53.339 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:53.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:53.341 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:53.441 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:53.467 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:53.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:53.468 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:53.569 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:53.592 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:53.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:53.593 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:53.694 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:53.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:53.721 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:53.722 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:53.822 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:53.845 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:53.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:53.847 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:53.947 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:53.970 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:53.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:53.971 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:54.071 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:54.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:54.094 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:54.094 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:54.195 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:54.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:54.218 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:54.219 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:54.319 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:54.342 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:54.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:54.343 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:54.444 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:54.466 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:54.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:54.467 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:54.568 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:54.595 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:54.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:54.596 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:54.697 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:54.723 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:54.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:54.724 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:54.804 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:54 smithi018 ceph-mon[29336]: pgmap v1014: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 12 MiB data, 7.4 GiB used, 529 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:54.805 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:54 smithi018 ceph-mon[40196]: pgmap v1014: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 12 MiB data, 7.4 GiB used, 529 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:54.824 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:54.847 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:54.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:54.848 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:54.949 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:54.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:54.972 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:54.972 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:55.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:54 smithi103 ceph-mon[33163]: pgmap v1014: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 12 MiB data, 7.4 GiB used, 529 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:55.073 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:55.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:55.097 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:55.097 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:55.163 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:06:55.200 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:55.226 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:55.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:55.227 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:55.328 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:55.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:55.355 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:55.355 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:55.456 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:55.478 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:55.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:55.479 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:55.580 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:55.595 INFO:teuthology.orchestra.run.smithi018.stderr:pool 'unique_pool_4' removed 2024-09-10T19:06:55.619 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:55.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:55.620 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:55.655 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:06:55.655 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:55 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2267023463' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T19:06:55.655 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:55 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T19:06:55.656 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:06:55.656 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:55 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2267023463' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T19:06:55.656 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:55 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T19:06:55.721 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:55.745 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:55.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:55.746 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:55.847 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:55.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:55.870 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:55.870 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:55.875 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T19:06:55.875 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-10T19:06:55.875 INFO:teuthology.orchestra.run.smithi018.stdout: "epoch": 453, 2024-09-10T19:06:55.875 INFO:teuthology.orchestra.run.smithi018.stdout: "fsid": "ba327f6c-6fa3-11ef-bcea-c7b262605968", 2024-09-10T19:06:55.876 INFO:teuthology.orchestra.run.smithi018.stdout: "created": "2024-09-10T18:40:07.669687+0000", 2024-09-10T19:06:55.876 INFO:teuthology.orchestra.run.smithi018.stdout: "modified": "2024-09-10T19:06:55.576315+0000", 2024-09-10T19:06:55.876 INFO:teuthology.orchestra.run.smithi018.stdout: "last_up_change": "2024-09-10T19:04:50.812385+0000", 2024-09-10T19:06:55.876 INFO:teuthology.orchestra.run.smithi018.stdout: "last_in_change": "2024-09-10T18:57:01.396827+0000", 2024-09-10T19:06:55.876 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-10T19:06:55.876 INFO:teuthology.orchestra.run.smithi018.stdout: "flags_num": 5806080, 2024-09-10T19:06:55.876 INFO:teuthology.orchestra.run.smithi018.stdout: "flags_set": [ 2024-09-10T19:06:55.876 INFO:teuthology.orchestra.run.smithi018.stdout: "nodeep-scrub", 2024-09-10T19:06:55.876 INFO:teuthology.orchestra.run.smithi018.stdout: "noscrub", 2024-09-10T19:06:55.876 INFO:teuthology.orchestra.run.smithi018.stdout: "pglog_hardlimit", 2024-09-10T19:06:55.876 INFO:teuthology.orchestra.run.smithi018.stdout: "purged_snapdirs", 2024-09-10T19:06:55.877 INFO:teuthology.orchestra.run.smithi018.stdout: "recovery_deletes", 2024-09-10T19:06:55.877 INFO:teuthology.orchestra.run.smithi018.stdout: "sortbitwise" 2024-09-10T19:06:55.877 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-09-10T19:06:55.877 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_version": 20, 2024-09-10T19:06:55.877 INFO:teuthology.orchestra.run.smithi018.stdout: "full_ratio": 0.94999998807907104, 2024-09-10T19:06:55.877 INFO:teuthology.orchestra.run.smithi018.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-10T19:06:55.877 INFO:teuthology.orchestra.run.smithi018.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-10T19:06:55.877 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_snapshot": "", 2024-09-10T19:06:55.877 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_max": 11, 2024-09-10T19:06:55.877 INFO:teuthology.orchestra.run.smithi018.stdout: "max_osd": 8, 2024-09-10T19:06:55.877 INFO:teuthology.orchestra.run.smithi018.stdout: "require_min_compat_client": "luminous", 2024-09-10T19:06:55.877 INFO:teuthology.orchestra.run.smithi018.stdout: "min_compat_client": "luminous", 2024-09-10T19:06:55.878 INFO:teuthology.orchestra.run.smithi018.stdout: "require_osd_release": "quincy", 2024-09-10T19:06:55.878 INFO:teuthology.orchestra.run.smithi018.stdout: "pools": [ 2024-09-10T19:06:55.878 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.878 INFO:teuthology.orchestra.run.smithi018.stdout: "pool": 1, 2024-09-10T19:06:55.878 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_name": ".mgr", 2024-09-10T19:06:55.878 INFO:teuthology.orchestra.run.smithi018.stdout: "create_time": "2024-09-10T18:43:08.162659+0000", 2024-09-10T19:06:55.878 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-09-10T19:06:55.878 INFO:teuthology.orchestra.run.smithi018.stdout: "flags_names": "hashpspool", 2024-09-10T19:06:55.878 INFO:teuthology.orchestra.run.smithi018.stdout: "type": 1, 2024-09-10T19:06:55.878 INFO:teuthology.orchestra.run.smithi018.stdout: "size": 3, 2024-09-10T19:06:55.878 INFO:teuthology.orchestra.run.smithi018.stdout: "min_size": 2, 2024-09-10T19:06:55.878 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_rule": 0, 2024-09-10T19:06:55.878 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_count": 0, 2024-09-10T19:06:55.879 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_target": 0, 2024-09-10T19:06:55.879 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T19:06:55.879 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T19:06:55.879 INFO:teuthology.orchestra.run.smithi018.stdout: "object_hash": 2, 2024-09-10T19:06:55.879 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscale_mode": "off", 2024-09-10T19:06:55.879 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num": 21, 2024-09-10T19:06:55.879 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num": 21, 2024-09-10T19:06:55.879 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num_target": 21, 2024-09-10T19:06:55.879 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_target": 21, 2024-09-10T19:06:55.879 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_pending": 21, 2024-09-10T19:06:55.879 INFO:teuthology.orchestra.run.smithi018.stdout: "last_pg_merge_meta": { 2024-09-10T19:06:55.879 INFO:teuthology.orchestra.run.smithi018.stdout: "source_pgid": "0.0", 2024-09-10T19:06:55.880 INFO:teuthology.orchestra.run.smithi018.stdout: "ready_epoch": 0, 2024-09-10T19:06:55.880 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_started": 0, 2024-09-10T19:06:55.880 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_clean": 0, 2024-09-10T19:06:55.880 INFO:teuthology.orchestra.run.smithi018.stdout: "source_version": "0'0", 2024-09-10T19:06:55.880 INFO:teuthology.orchestra.run.smithi018.stdout: "target_version": "0'0" 2024-09-10T19:06:55.880 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.880 INFO:teuthology.orchestra.run.smithi018.stdout: "last_change": "394", 2024-09-10T19:06:55.880 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend": "0", 2024-09-10T19:06:55.880 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T19:06:55.880 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_preluminous": "167", 2024-09-10T19:06:55.880 INFO:teuthology.orchestra.run.smithi018.stdout: "auid": 0, 2024-09-10T19:06:55.880 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_mode": "selfmanaged", 2024-09-10T19:06:55.881 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_seq": 0, 2024-09-10T19:06:55.881 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_epoch": 0, 2024-09-10T19:06:55.881 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_snaps": [], 2024-09-10T19:06:55.881 INFO:teuthology.orchestra.run.smithi018.stdout: "removed_snaps": "[]", 2024-09-10T19:06:55.881 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_bytes": 0, 2024-09-10T19:06:55.881 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_objects": 0, 2024-09-10T19:06:55.881 INFO:teuthology.orchestra.run.smithi018.stdout: "tiers": [], 2024-09-10T19:06:55.881 INFO:teuthology.orchestra.run.smithi018.stdout: "tier_of": -1, 2024-09-10T19:06:55.881 INFO:teuthology.orchestra.run.smithi018.stdout: "read_tier": -1, 2024-09-10T19:06:55.881 INFO:teuthology.orchestra.run.smithi018.stdout: "write_tier": -1, 2024-09-10T19:06:55.881 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_mode": "none", 2024-09-10T19:06:55.881 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_bytes": 0, 2024-09-10T19:06:55.881 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_objects": 0, 2024-09-10T19:06:55.882 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T19:06:55.882 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T19:06:55.882 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T19:06:55.882 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_flush_age": 0, 2024-09-10T19:06:55.882 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_evict_age": 0, 2024-09-10T19:06:55.882 INFO:teuthology.orchestra.run.smithi018.stdout: "erasure_code_profile": "", 2024-09-10T19:06:55.882 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_params": { 2024-09-10T19:06:55.882 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "none" 2024-09-10T19:06:55.882 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.882 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_period": 0, 2024-09-10T19:06:55.882 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_count": 0, 2024-09-10T19:06:55.882 INFO:teuthology.orchestra.run.smithi018.stdout: "use_gmt_hitset": true, 2024-09-10T19:06:55.883 INFO:teuthology.orchestra.run.smithi018.stdout: "min_read_recency_for_promote": 0, 2024-09-10T19:06:55.883 INFO:teuthology.orchestra.run.smithi018.stdout: "min_write_recency_for_promote": 0, 2024-09-10T19:06:55.883 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T19:06:55.883 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_search_last_n": 0, 2024-09-10T19:06:55.883 INFO:teuthology.orchestra.run.smithi018.stdout: "grade_table": [], 2024-09-10T19:06:55.883 INFO:teuthology.orchestra.run.smithi018.stdout: "stripe_width": 0, 2024-09-10T19:06:55.883 INFO:teuthology.orchestra.run.smithi018.stdout: "expected_num_objects": 0, 2024-09-10T19:06:55.883 INFO:teuthology.orchestra.run.smithi018.stdout: "fast_read": false, 2024-09-10T19:06:55.883 INFO:teuthology.orchestra.run.smithi018.stdout: "options": { 2024-09-10T19:06:55.883 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_max": 32, 2024-09-10T19:06:55.883 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_min": 1 2024-09-10T19:06:55.884 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.884 INFO:teuthology.orchestra.run.smithi018.stdout: "application_metadata": { 2024-09-10T19:06:55.884 INFO:teuthology.orchestra.run.smithi018.stdout: "mgr": {} 2024-09-10T19:06:55.884 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.884 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.884 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.884 INFO:teuthology.orchestra.run.smithi018.stdout: "pool": 2, 2024-09-10T19:06:55.884 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_name": ".rgw.root", 2024-09-10T19:06:55.884 INFO:teuthology.orchestra.run.smithi018.stdout: "create_time": "2024-09-10T18:45:16.512219+0000", 2024-09-10T19:06:55.884 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-09-10T19:06:55.884 INFO:teuthology.orchestra.run.smithi018.stdout: "flags_names": "hashpspool", 2024-09-10T19:06:55.884 INFO:teuthology.orchestra.run.smithi018.stdout: "type": 1, 2024-09-10T19:06:55.885 INFO:teuthology.orchestra.run.smithi018.stdout: "size": 3, 2024-09-10T19:06:55.885 INFO:teuthology.orchestra.run.smithi018.stdout: "min_size": 2, 2024-09-10T19:06:55.885 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_rule": 0, 2024-09-10T19:06:55.885 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_count": 0, 2024-09-10T19:06:55.885 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_target": 0, 2024-09-10T19:06:55.885 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T19:06:55.885 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T19:06:55.885 INFO:teuthology.orchestra.run.smithi018.stdout: "object_hash": 2, 2024-09-10T19:06:55.885 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscale_mode": "off", 2024-09-10T19:06:55.885 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num": 32, 2024-09-10T19:06:55.885 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num": 32, 2024-09-10T19:06:55.885 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num_target": 32, 2024-09-10T19:06:55.886 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_target": 32, 2024-09-10T19:06:55.886 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_pending": 32, 2024-09-10T19:06:55.886 INFO:teuthology.orchestra.run.smithi018.stdout: "last_pg_merge_meta": { 2024-09-10T19:06:55.886 INFO:teuthology.orchestra.run.smithi018.stdout: "source_pgid": "0.0", 2024-09-10T19:06:55.886 INFO:teuthology.orchestra.run.smithi018.stdout: "ready_epoch": 0, 2024-09-10T19:06:55.886 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_started": 0, 2024-09-10T19:06:55.886 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_clean": 0, 2024-09-10T19:06:55.886 INFO:teuthology.orchestra.run.smithi018.stdout: "source_version": "0'0", 2024-09-10T19:06:55.886 INFO:teuthology.orchestra.run.smithi018.stdout: "target_version": "0'0" 2024-09-10T19:06:55.886 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.886 INFO:teuthology.orchestra.run.smithi018.stdout: "last_change": "57", 2024-09-10T19:06:55.886 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend": "0", 2024-09-10T19:06:55.886 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T19:06:55.886 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T19:06:55.887 INFO:teuthology.orchestra.run.smithi018.stdout: "auid": 0, 2024-09-10T19:06:55.887 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_mode": "selfmanaged", 2024-09-10T19:06:55.887 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_seq": 0, 2024-09-10T19:06:55.887 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_epoch": 0, 2024-09-10T19:06:55.887 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_snaps": [], 2024-09-10T19:06:55.887 INFO:teuthology.orchestra.run.smithi018.stdout: "removed_snaps": "[]", 2024-09-10T19:06:55.887 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_bytes": 0, 2024-09-10T19:06:55.887 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_objects": 0, 2024-09-10T19:06:55.887 INFO:teuthology.orchestra.run.smithi018.stdout: "tiers": [], 2024-09-10T19:06:55.887 INFO:teuthology.orchestra.run.smithi018.stdout: "tier_of": -1, 2024-09-10T19:06:55.887 INFO:teuthology.orchestra.run.smithi018.stdout: "read_tier": -1, 2024-09-10T19:06:55.887 INFO:teuthology.orchestra.run.smithi018.stdout: "write_tier": -1, 2024-09-10T19:06:55.887 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_mode": "none", 2024-09-10T19:06:55.887 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_bytes": 0, 2024-09-10T19:06:55.888 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_objects": 0, 2024-09-10T19:06:55.888 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T19:06:55.888 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T19:06:55.888 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T19:06:55.888 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_flush_age": 0, 2024-09-10T19:06:55.888 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_evict_age": 0, 2024-09-10T19:06:55.888 INFO:teuthology.orchestra.run.smithi018.stdout: "erasure_code_profile": "", 2024-09-10T19:06:55.888 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_params": { 2024-09-10T19:06:55.888 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "none" 2024-09-10T19:06:55.888 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.888 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_period": 0, 2024-09-10T19:06:55.888 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_count": 0, 2024-09-10T19:06:55.888 INFO:teuthology.orchestra.run.smithi018.stdout: "use_gmt_hitset": true, 2024-09-10T19:06:55.889 INFO:teuthology.orchestra.run.smithi018.stdout: "min_read_recency_for_promote": 0, 2024-09-10T19:06:55.889 INFO:teuthology.orchestra.run.smithi018.stdout: "min_write_recency_for_promote": 0, 2024-09-10T19:06:55.889 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T19:06:55.889 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_search_last_n": 0, 2024-09-10T19:06:55.889 INFO:teuthology.orchestra.run.smithi018.stdout: "grade_table": [], 2024-09-10T19:06:55.889 INFO:teuthology.orchestra.run.smithi018.stdout: "stripe_width": 0, 2024-09-10T19:06:55.889 INFO:teuthology.orchestra.run.smithi018.stdout: "expected_num_objects": 0, 2024-09-10T19:06:55.889 INFO:teuthology.orchestra.run.smithi018.stdout: "fast_read": false, 2024-09-10T19:06:55.889 INFO:teuthology.orchestra.run.smithi018.stdout: "options": {}, 2024-09-10T19:06:55.889 INFO:teuthology.orchestra.run.smithi018.stdout: "application_metadata": { 2024-09-10T19:06:55.889 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw": {} 2024-09-10T19:06:55.889 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.889 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.889 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.890 INFO:teuthology.orchestra.run.smithi018.stdout: "pool": 3, 2024-09-10T19:06:55.890 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_name": "datapool", 2024-09-10T19:06:55.890 INFO:teuthology.orchestra.run.smithi018.stdout: "create_time": "2024-09-10T18:45:16.943626+0000", 2024-09-10T19:06:55.890 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 8193, 2024-09-10T19:06:55.890 INFO:teuthology.orchestra.run.smithi018.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-10T19:06:55.890 INFO:teuthology.orchestra.run.smithi018.stdout: "type": 1, 2024-09-10T19:06:55.890 INFO:teuthology.orchestra.run.smithi018.stdout: "size": 3, 2024-09-10T19:06:55.890 INFO:teuthology.orchestra.run.smithi018.stdout: "min_size": 2, 2024-09-10T19:06:55.890 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_rule": 0, 2024-09-10T19:06:55.890 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_count": 0, 2024-09-10T19:06:55.890 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_target": 0, 2024-09-10T19:06:55.890 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T19:06:55.890 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T19:06:55.891 INFO:teuthology.orchestra.run.smithi018.stdout: "object_hash": 2, 2024-09-10T19:06:55.891 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscale_mode": "off", 2024-09-10T19:06:55.891 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num": 3, 2024-09-10T19:06:55.891 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num": 3, 2024-09-10T19:06:55.891 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num_target": 3, 2024-09-10T19:06:55.891 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_target": 3, 2024-09-10T19:06:55.891 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_pending": 3, 2024-09-10T19:06:55.891 INFO:teuthology.orchestra.run.smithi018.stdout: "last_pg_merge_meta": { 2024-09-10T19:06:55.891 INFO:teuthology.orchestra.run.smithi018.stdout: "source_pgid": "0.0", 2024-09-10T19:06:55.891 INFO:teuthology.orchestra.run.smithi018.stdout: "ready_epoch": 0, 2024-09-10T19:06:55.891 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_started": 0, 2024-09-10T19:06:55.891 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_clean": 0, 2024-09-10T19:06:55.891 INFO:teuthology.orchestra.run.smithi018.stdout: "source_version": "0'0", 2024-09-10T19:06:55.892 INFO:teuthology.orchestra.run.smithi018.stdout: "target_version": "0'0" 2024-09-10T19:06:55.892 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.892 INFO:teuthology.orchestra.run.smithi018.stdout: "last_change": "60", 2024-09-10T19:06:55.892 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend": "0", 2024-09-10T19:06:55.892 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T19:06:55.892 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T19:06:55.892 INFO:teuthology.orchestra.run.smithi018.stdout: "auid": 0, 2024-09-10T19:06:55.892 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_mode": "selfmanaged", 2024-09-10T19:06:55.892 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_seq": 3, 2024-09-10T19:06:55.892 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_epoch": 60, 2024-09-10T19:06:55.892 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_snaps": [], 2024-09-10T19:06:55.892 INFO:teuthology.orchestra.run.smithi018.stdout: "removed_snaps": "[]", 2024-09-10T19:06:55.892 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_bytes": 0, 2024-09-10T19:06:55.892 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_objects": 0, 2024-09-10T19:06:55.893 INFO:teuthology.orchestra.run.smithi018.stdout: "tiers": [], 2024-09-10T19:06:55.893 INFO:teuthology.orchestra.run.smithi018.stdout: "tier_of": -1, 2024-09-10T19:06:55.893 INFO:teuthology.orchestra.run.smithi018.stdout: "read_tier": -1, 2024-09-10T19:06:55.893 INFO:teuthology.orchestra.run.smithi018.stdout: "write_tier": -1, 2024-09-10T19:06:55.893 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_mode": "none", 2024-09-10T19:06:55.893 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_bytes": 0, 2024-09-10T19:06:55.893 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_objects": 0, 2024-09-10T19:06:55.893 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T19:06:55.893 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T19:06:55.893 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T19:06:55.893 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_flush_age": 0, 2024-09-10T19:06:55.893 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_evict_age": 0, 2024-09-10T19:06:55.893 INFO:teuthology.orchestra.run.smithi018.stdout: "erasure_code_profile": "", 2024-09-10T19:06:55.894 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_params": { 2024-09-10T19:06:55.894 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "none" 2024-09-10T19:06:55.894 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.894 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_period": 0, 2024-09-10T19:06:55.895 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_count": 0, 2024-09-10T19:06:55.895 INFO:teuthology.orchestra.run.smithi018.stdout: "use_gmt_hitset": true, 2024-09-10T19:06:55.895 INFO:teuthology.orchestra.run.smithi018.stdout: "min_read_recency_for_promote": 0, 2024-09-10T19:06:55.895 INFO:teuthology.orchestra.run.smithi018.stdout: "min_write_recency_for_promote": 0, 2024-09-10T19:06:55.895 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T19:06:55.895 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_search_last_n": 0, 2024-09-10T19:06:55.895 INFO:teuthology.orchestra.run.smithi018.stdout: "grade_table": [], 2024-09-10T19:06:55.895 INFO:teuthology.orchestra.run.smithi018.stdout: "stripe_width": 0, 2024-09-10T19:06:55.895 INFO:teuthology.orchestra.run.smithi018.stdout: "expected_num_objects": 0, 2024-09-10T19:06:55.895 INFO:teuthology.orchestra.run.smithi018.stdout: "fast_read": false, 2024-09-10T19:06:55.895 INFO:teuthology.orchestra.run.smithi018.stdout: "options": {}, 2024-09-10T19:06:55.895 INFO:teuthology.orchestra.run.smithi018.stdout: "application_metadata": { 2024-09-10T19:06:55.896 INFO:teuthology.orchestra.run.smithi018.stdout: "rbd": {} 2024-09-10T19:06:55.896 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.896 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.896 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.896 INFO:teuthology.orchestra.run.smithi018.stdout: "pool": 4, 2024-09-10T19:06:55.896 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_name": "default.rgw.log", 2024-09-10T19:06:55.896 INFO:teuthology.orchestra.run.smithi018.stdout: "create_time": "2024-09-10T18:45:18.450026+0000", 2024-09-10T19:06:55.896 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-09-10T19:06:55.896 INFO:teuthology.orchestra.run.smithi018.stdout: "flags_names": "hashpspool", 2024-09-10T19:06:55.896 INFO:teuthology.orchestra.run.smithi018.stdout: "type": 1, 2024-09-10T19:06:55.896 INFO:teuthology.orchestra.run.smithi018.stdout: "size": 3, 2024-09-10T19:06:55.896 INFO:teuthology.orchestra.run.smithi018.stdout: "min_size": 2, 2024-09-10T19:06:55.896 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_rule": 0, 2024-09-10T19:06:55.897 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_count": 0, 2024-09-10T19:06:55.897 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_target": 0, 2024-09-10T19:06:55.897 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T19:06:55.897 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T19:06:55.897 INFO:teuthology.orchestra.run.smithi018.stdout: "object_hash": 2, 2024-09-10T19:06:55.897 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscale_mode": "off", 2024-09-10T19:06:55.897 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num": 32, 2024-09-10T19:06:55.897 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num": 32, 2024-09-10T19:06:55.897 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num_target": 32, 2024-09-10T19:06:55.897 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_target": 32, 2024-09-10T19:06:55.897 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_pending": 32, 2024-09-10T19:06:55.897 INFO:teuthology.orchestra.run.smithi018.stdout: "last_pg_merge_meta": { 2024-09-10T19:06:55.897 INFO:teuthology.orchestra.run.smithi018.stdout: "source_pgid": "0.0", 2024-09-10T19:06:55.898 INFO:teuthology.orchestra.run.smithi018.stdout: "ready_epoch": 0, 2024-09-10T19:06:55.898 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_started": 0, 2024-09-10T19:06:55.898 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_clean": 0, 2024-09-10T19:06:55.898 INFO:teuthology.orchestra.run.smithi018.stdout: "source_version": "0'0", 2024-09-10T19:06:55.898 INFO:teuthology.orchestra.run.smithi018.stdout: "target_version": "0'0" 2024-09-10T19:06:55.898 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.898 INFO:teuthology.orchestra.run.smithi018.stdout: "last_change": "59", 2024-09-10T19:06:55.898 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend": "0", 2024-09-10T19:06:55.898 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T19:06:55.898 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T19:06:55.898 INFO:teuthology.orchestra.run.smithi018.stdout: "auid": 0, 2024-09-10T19:06:55.898 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_mode": "selfmanaged", 2024-09-10T19:06:55.898 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_seq": 0, 2024-09-10T19:06:55.898 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_epoch": 0, 2024-09-10T19:06:55.899 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_snaps": [], 2024-09-10T19:06:55.899 INFO:teuthology.orchestra.run.smithi018.stdout: "removed_snaps": "[]", 2024-09-10T19:06:55.899 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_bytes": 0, 2024-09-10T19:06:55.899 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_objects": 0, 2024-09-10T19:06:55.899 INFO:teuthology.orchestra.run.smithi018.stdout: "tiers": [], 2024-09-10T19:06:55.899 INFO:teuthology.orchestra.run.smithi018.stdout: "tier_of": -1, 2024-09-10T19:06:55.899 INFO:teuthology.orchestra.run.smithi018.stdout: "read_tier": -1, 2024-09-10T19:06:55.899 INFO:teuthology.orchestra.run.smithi018.stdout: "write_tier": -1, 2024-09-10T19:06:55.899 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_mode": "none", 2024-09-10T19:06:55.899 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_bytes": 0, 2024-09-10T19:06:55.899 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_objects": 0, 2024-09-10T19:06:55.899 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T19:06:55.899 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T19:06:55.900 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T19:06:55.900 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_flush_age": 0, 2024-09-10T19:06:55.900 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_evict_age": 0, 2024-09-10T19:06:55.900 INFO:teuthology.orchestra.run.smithi018.stdout: "erasure_code_profile": "", 2024-09-10T19:06:55.900 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_params": { 2024-09-10T19:06:55.900 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "none" 2024-09-10T19:06:55.900 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.900 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_period": 0, 2024-09-10T19:06:55.900 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_count": 0, 2024-09-10T19:06:55.900 INFO:teuthology.orchestra.run.smithi018.stdout: "use_gmt_hitset": true, 2024-09-10T19:06:55.900 INFO:teuthology.orchestra.run.smithi018.stdout: "min_read_recency_for_promote": 0, 2024-09-10T19:06:55.900 INFO:teuthology.orchestra.run.smithi018.stdout: "min_write_recency_for_promote": 0, 2024-09-10T19:06:55.900 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T19:06:55.901 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_search_last_n": 0, 2024-09-10T19:06:55.901 INFO:teuthology.orchestra.run.smithi018.stdout: "grade_table": [], 2024-09-10T19:06:55.901 INFO:teuthology.orchestra.run.smithi018.stdout: "stripe_width": 0, 2024-09-10T19:06:55.901 INFO:teuthology.orchestra.run.smithi018.stdout: "expected_num_objects": 0, 2024-09-10T19:06:55.901 INFO:teuthology.orchestra.run.smithi018.stdout: "fast_read": false, 2024-09-10T19:06:55.901 INFO:teuthology.orchestra.run.smithi018.stdout: "options": {}, 2024-09-10T19:06:55.901 INFO:teuthology.orchestra.run.smithi018.stdout: "application_metadata": { 2024-09-10T19:06:55.901 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw": {} 2024-09-10T19:06:55.901 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.901 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.901 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.901 INFO:teuthology.orchestra.run.smithi018.stdout: "pool": 5, 2024-09-10T19:06:55.901 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_name": "default.rgw.control", 2024-09-10T19:06:55.902 INFO:teuthology.orchestra.run.smithi018.stdout: "create_time": "2024-09-10T18:45:20.494618+0000", 2024-09-10T19:06:55.902 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-09-10T19:06:55.902 INFO:teuthology.orchestra.run.smithi018.stdout: "flags_names": "hashpspool", 2024-09-10T19:06:55.902 INFO:teuthology.orchestra.run.smithi018.stdout: "type": 1, 2024-09-10T19:06:55.902 INFO:teuthology.orchestra.run.smithi018.stdout: "size": 3, 2024-09-10T19:06:55.902 INFO:teuthology.orchestra.run.smithi018.stdout: "min_size": 2, 2024-09-10T19:06:55.902 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_rule": 0, 2024-09-10T19:06:55.902 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_count": 0, 2024-09-10T19:06:55.902 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_target": 0, 2024-09-10T19:06:55.902 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T19:06:55.902 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T19:06:55.902 INFO:teuthology.orchestra.run.smithi018.stdout: "object_hash": 2, 2024-09-10T19:06:55.902 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscale_mode": "off", 2024-09-10T19:06:55.903 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num": 32, 2024-09-10T19:06:55.903 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num": 32, 2024-09-10T19:06:55.903 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num_target": 32, 2024-09-10T19:06:55.903 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_target": 32, 2024-09-10T19:06:55.903 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_pending": 32, 2024-09-10T19:06:55.903 INFO:teuthology.orchestra.run.smithi018.stdout: "last_pg_merge_meta": { 2024-09-10T19:06:55.903 INFO:teuthology.orchestra.run.smithi018.stdout: "source_pgid": "0.0", 2024-09-10T19:06:55.903 INFO:teuthology.orchestra.run.smithi018.stdout: "ready_epoch": 0, 2024-09-10T19:06:55.903 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_started": 0, 2024-09-10T19:06:55.903 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_clean": 0, 2024-09-10T19:06:55.903 INFO:teuthology.orchestra.run.smithi018.stdout: "source_version": "0'0", 2024-09-10T19:06:55.903 INFO:teuthology.orchestra.run.smithi018.stdout: "target_version": "0'0" 2024-09-10T19:06:55.904 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.904 INFO:teuthology.orchestra.run.smithi018.stdout: "last_change": "61", 2024-09-10T19:06:55.904 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend": "0", 2024-09-10T19:06:55.904 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T19:06:55.904 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T19:06:55.904 INFO:teuthology.orchestra.run.smithi018.stdout: "auid": 0, 2024-09-10T19:06:55.904 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_mode": "selfmanaged", 2024-09-10T19:06:55.904 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_seq": 0, 2024-09-10T19:06:55.904 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_epoch": 0, 2024-09-10T19:06:55.904 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_snaps": [], 2024-09-10T19:06:55.904 INFO:teuthology.orchestra.run.smithi018.stdout: "removed_snaps": "[]", 2024-09-10T19:06:55.904 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_bytes": 0, 2024-09-10T19:06:55.905 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_objects": 0, 2024-09-10T19:06:55.905 INFO:teuthology.orchestra.run.smithi018.stdout: "tiers": [], 2024-09-10T19:06:55.905 INFO:teuthology.orchestra.run.smithi018.stdout: "tier_of": -1, 2024-09-10T19:06:55.905 INFO:teuthology.orchestra.run.smithi018.stdout: "read_tier": -1, 2024-09-10T19:06:55.905 INFO:teuthology.orchestra.run.smithi018.stdout: "write_tier": -1, 2024-09-10T19:06:55.905 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_mode": "none", 2024-09-10T19:06:55.905 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_bytes": 0, 2024-09-10T19:06:55.905 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_objects": 0, 2024-09-10T19:06:55.905 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T19:06:55.905 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T19:06:55.905 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T19:06:55.905 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_flush_age": 0, 2024-09-10T19:06:55.905 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_evict_age": 0, 2024-09-10T19:06:55.906 INFO:teuthology.orchestra.run.smithi018.stdout: "erasure_code_profile": "", 2024-09-10T19:06:55.906 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_params": { 2024-09-10T19:06:55.906 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "none" 2024-09-10T19:06:55.906 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.906 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_period": 0, 2024-09-10T19:06:55.906 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_count": 0, 2024-09-10T19:06:55.906 INFO:teuthology.orchestra.run.smithi018.stdout: "use_gmt_hitset": true, 2024-09-10T19:06:55.906 INFO:teuthology.orchestra.run.smithi018.stdout: "min_read_recency_for_promote": 0, 2024-09-10T19:06:55.906 INFO:teuthology.orchestra.run.smithi018.stdout: "min_write_recency_for_promote": 0, 2024-09-10T19:06:55.906 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T19:06:55.906 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_search_last_n": 0, 2024-09-10T19:06:55.906 INFO:teuthology.orchestra.run.smithi018.stdout: "grade_table": [], 2024-09-10T19:06:55.907 INFO:teuthology.orchestra.run.smithi018.stdout: "stripe_width": 0, 2024-09-10T19:06:55.907 INFO:teuthology.orchestra.run.smithi018.stdout: "expected_num_objects": 0, 2024-09-10T19:06:55.907 INFO:teuthology.orchestra.run.smithi018.stdout: "fast_read": false, 2024-09-10T19:06:55.907 INFO:teuthology.orchestra.run.smithi018.stdout: "options": {}, 2024-09-10T19:06:55.907 INFO:teuthology.orchestra.run.smithi018.stdout: "application_metadata": { 2024-09-10T19:06:55.907 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw": {} 2024-09-10T19:06:55.907 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.907 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.907 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.907 INFO:teuthology.orchestra.run.smithi018.stdout: "pool": 6, 2024-09-10T19:06:55.907 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_name": "default.rgw.meta", 2024-09-10T19:06:55.907 INFO:teuthology.orchestra.run.smithi018.stdout: "create_time": "2024-09-10T18:45:22.566992+0000", 2024-09-10T19:06:55.908 INFO:teuthology.orchestra.run.smithi018.stdout: "flags": 1, 2024-09-10T19:06:55.908 INFO:teuthology.orchestra.run.smithi018.stdout: "flags_names": "hashpspool", 2024-09-10T19:06:55.908 INFO:teuthology.orchestra.run.smithi018.stdout: "type": 1, 2024-09-10T19:06:55.908 INFO:teuthology.orchestra.run.smithi018.stdout: "size": 3, 2024-09-10T19:06:55.908 INFO:teuthology.orchestra.run.smithi018.stdout: "min_size": 2, 2024-09-10T19:06:55.908 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_rule": 0, 2024-09-10T19:06:55.908 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_count": 0, 2024-09-10T19:06:55.908 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_target": 0, 2024-09-10T19:06:55.908 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T19:06:55.908 INFO:teuthology.orchestra.run.smithi018.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T19:06:55.908 INFO:teuthology.orchestra.run.smithi018.stdout: "object_hash": 2, 2024-09-10T19:06:55.908 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscale_mode": "off", 2024-09-10T19:06:55.909 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num": 32, 2024-09-10T19:06:55.909 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num": 32, 2024-09-10T19:06:55.909 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_placement_num_target": 32, 2024-09-10T19:06:55.909 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_target": 32, 2024-09-10T19:06:55.909 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_num_pending": 32, 2024-09-10T19:06:55.909 INFO:teuthology.orchestra.run.smithi018.stdout: "last_pg_merge_meta": { 2024-09-10T19:06:55.909 INFO:teuthology.orchestra.run.smithi018.stdout: "source_pgid": "0.0", 2024-09-10T19:06:55.909 INFO:teuthology.orchestra.run.smithi018.stdout: "ready_epoch": 0, 2024-09-10T19:06:55.909 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_started": 0, 2024-09-10T19:06:55.909 INFO:teuthology.orchestra.run.smithi018.stdout: "last_epoch_clean": 0, 2024-09-10T19:06:55.909 INFO:teuthology.orchestra.run.smithi018.stdout: "source_version": "0'0", 2024-09-10T19:06:55.909 INFO:teuthology.orchestra.run.smithi018.stdout: "target_version": "0'0" 2024-09-10T19:06:55.909 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.910 INFO:teuthology.orchestra.run.smithi018.stdout: "last_change": "63", 2024-09-10T19:06:55.910 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend": "0", 2024-09-10T19:06:55.910 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T19:06:55.910 INFO:teuthology.orchestra.run.smithi018.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T19:06:55.910 INFO:teuthology.orchestra.run.smithi018.stdout: "auid": 0, 2024-09-10T19:06:55.910 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_mode": "selfmanaged", 2024-09-10T19:06:55.910 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_seq": 0, 2024-09-10T19:06:55.910 INFO:teuthology.orchestra.run.smithi018.stdout: "snap_epoch": 0, 2024-09-10T19:06:55.910 INFO:teuthology.orchestra.run.smithi018.stdout: "pool_snaps": [], 2024-09-10T19:06:55.910 INFO:teuthology.orchestra.run.smithi018.stdout: "removed_snaps": "[]", 2024-09-10T19:06:55.910 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_bytes": 0, 2024-09-10T19:06:55.911 INFO:teuthology.orchestra.run.smithi018.stdout: "quota_max_objects": 0, 2024-09-10T19:06:55.911 INFO:teuthology.orchestra.run.smithi018.stdout: "tiers": [], 2024-09-10T19:06:55.911 INFO:teuthology.orchestra.run.smithi018.stdout: "tier_of": -1, 2024-09-10T19:06:55.911 INFO:teuthology.orchestra.run.smithi018.stdout: "read_tier": -1, 2024-09-10T19:06:55.911 INFO:teuthology.orchestra.run.smithi018.stdout: "write_tier": -1, 2024-09-10T19:06:55.911 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_mode": "none", 2024-09-10T19:06:55.911 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_bytes": 0, 2024-09-10T19:06:55.911 INFO:teuthology.orchestra.run.smithi018.stdout: "target_max_objects": 0, 2024-09-10T19:06:55.911 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T19:06:55.911 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T19:06:55.911 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T19:06:55.911 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_flush_age": 0, 2024-09-10T19:06:55.911 INFO:teuthology.orchestra.run.smithi018.stdout: "cache_min_evict_age": 0, 2024-09-10T19:06:55.912 INFO:teuthology.orchestra.run.smithi018.stdout: "erasure_code_profile": "", 2024-09-10T19:06:55.912 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_params": { 2024-09-10T19:06:55.912 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "none" 2024-09-10T19:06:55.912 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.912 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_period": 0, 2024-09-10T19:06:55.912 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_count": 0, 2024-09-10T19:06:55.912 INFO:teuthology.orchestra.run.smithi018.stdout: "use_gmt_hitset": true, 2024-09-10T19:06:55.912 INFO:teuthology.orchestra.run.smithi018.stdout: "min_read_recency_for_promote": 0, 2024-09-10T19:06:55.912 INFO:teuthology.orchestra.run.smithi018.stdout: "min_write_recency_for_promote": 0, 2024-09-10T19:06:55.912 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T19:06:55.913 INFO:teuthology.orchestra.run.smithi018.stdout: "hit_set_search_last_n": 0, 2024-09-10T19:06:55.913 INFO:teuthology.orchestra.run.smithi018.stdout: "grade_table": [], 2024-09-10T19:06:55.913 INFO:teuthology.orchestra.run.smithi018.stdout: "stripe_width": 0, 2024-09-10T19:06:55.913 INFO:teuthology.orchestra.run.smithi018.stdout: "expected_num_objects": 0, 2024-09-10T19:06:55.913 INFO:teuthology.orchestra.run.smithi018.stdout: "fast_read": false, 2024-09-10T19:06:55.913 INFO:teuthology.orchestra.run.smithi018.stdout: "options": { 2024-09-10T19:06:55.913 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_autoscale_bias": 4 2024-09-10T19:06:55.913 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.913 INFO:teuthology.orchestra.run.smithi018.stdout: "application_metadata": { 2024-09-10T19:06:55.913 INFO:teuthology.orchestra.run.smithi018.stdout: "rgw": {} 2024-09-10T19:06:55.913 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.913 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.913 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-09-10T19:06:55.914 INFO:teuthology.orchestra.run.smithi018.stdout: "osds": [ 2024-09-10T19:06:55.914 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.914 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 0, 2024-09-10T19:06:55.914 INFO:teuthology.orchestra.run.smithi018.stdout: "uuid": "eb4c7e67-300c-4a1a-b9f5-bb028eeeabc2", 2024-09-10T19:06:55.914 INFO:teuthology.orchestra.run.smithi018.stdout: "up": 1, 2024-09-10T19:06:55.914 INFO:teuthology.orchestra.run.smithi018.stdout: "in": 1, 2024-09-10T19:06:55.914 INFO:teuthology.orchestra.run.smithi018.stdout: "weight": 1, 2024-09-10T19:06:55.914 INFO:teuthology.orchestra.run.smithi018.stdout: "primary_affinity": 1, 2024-09-10T19:06:55.914 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_begin": 8, 2024-09-10T19:06:55.914 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_end": 205, 2024-09-10T19:06:55.914 INFO:teuthology.orchestra.run.smithi018.stdout: "up_from": 241, 2024-09-10T19:06:55.914 INFO:teuthology.orchestra.run.smithi018.stdout: "up_thru": 434, 2024-09-10T19:06:55.914 INFO:teuthology.orchestra.run.smithi018.stdout: "down_at": 206, 2024-09-10T19:06:55.914 INFO:teuthology.orchestra.run.smithi018.stdout: "lost_at": 0, 2024-09-10T19:06:55.915 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addrs": { 2024-09-10T19:06:55.915 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T19:06:55.915 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.915 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T19:06:55.915 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6801", 2024-09-10T19:06:55.915 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 4100931169 2024-09-10T19:06:55.915 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.915 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.915 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.915 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addrs": { 2024-09-10T19:06:55.915 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T19:06:55.915 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.915 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T19:06:55.916 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6802", 2024-09-10T19:06:55.916 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 4100931169 2024-09-10T19:06:55.916 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.916 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.916 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.916 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addrs": { 2024-09-10T19:06:55.916 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T19:06:55.916 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.916 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T19:06:55.917 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6804", 2024-09-10T19:06:55.917 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 4100931169 2024-09-10T19:06:55.917 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.917 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.917 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.917 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addrs": { 2024-09-10T19:06:55.917 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T19:06:55.917 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.917 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T19:06:55.917 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6803", 2024-09-10T19:06:55.917 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 4100931169 2024-09-10T19:06:55.917 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.918 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.918 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.918 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addr": "172.21.15.18:6801/4100931169", 2024-09-10T19:06:55.918 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addr": "172.21.15.18:6802/4100931169", 2024-09-10T19:06:55.918 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addr": "172.21.15.18:6804/4100931169", 2024-09-10T19:06:55.918 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addr": "172.21.15.18:6803/4100931169", 2024-09-10T19:06:55.918 INFO:teuthology.orchestra.run.smithi018.stdout: "state": [ 2024-09-10T19:06:55.918 INFO:teuthology.orchestra.run.smithi018.stdout: "exists", 2024-09-10T19:06:55.918 INFO:teuthology.orchestra.run.smithi018.stdout: "up" 2024-09-10T19:06:55.918 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.918 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.918 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.918 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 1, 2024-09-10T19:06:55.919 INFO:teuthology.orchestra.run.smithi018.stdout: "uuid": "63bf6a98-e349-4cc0-9502-a31693070c5b", 2024-09-10T19:06:55.919 INFO:teuthology.orchestra.run.smithi018.stdout: "up": 1, 2024-09-10T19:06:55.919 INFO:teuthology.orchestra.run.smithi018.stdout: "in": 1, 2024-09-10T19:06:55.919 INFO:teuthology.orchestra.run.smithi018.stdout: "weight": 1, 2024-09-10T19:06:55.919 INFO:teuthology.orchestra.run.smithi018.stdout: "primary_affinity": 0.010101318359375, 2024-09-10T19:06:55.919 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_begin": 0, 2024-09-10T19:06:55.919 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_end": 0, 2024-09-10T19:06:55.919 INFO:teuthology.orchestra.run.smithi018.stdout: "up_from": 13, 2024-09-10T19:06:55.919 INFO:teuthology.orchestra.run.smithi018.stdout: "up_thru": 432, 2024-09-10T19:06:55.919 INFO:teuthology.orchestra.run.smithi018.stdout: "down_at": 0, 2024-09-10T19:06:55.919 INFO:teuthology.orchestra.run.smithi018.stdout: "lost_at": 0, 2024-09-10T19:06:55.919 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addrs": { 2024-09-10T19:06:55.919 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T19:06:55.919 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.920 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T19:06:55.920 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6805", 2024-09-10T19:06:55.920 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3855640936 2024-09-10T19:06:55.920 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.920 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.920 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.920 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addrs": { 2024-09-10T19:06:55.920 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T19:06:55.920 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.920 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T19:06:55.920 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6806", 2024-09-10T19:06:55.920 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3855640936 2024-09-10T19:06:55.921 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.921 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.921 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.921 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addrs": { 2024-09-10T19:06:55.921 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T19:06:55.921 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.921 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T19:06:55.921 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6808", 2024-09-10T19:06:55.921 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3855640936 2024-09-10T19:06:55.921 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.921 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.921 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.921 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addrs": { 2024-09-10T19:06:55.922 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T19:06:55.922 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.922 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T19:06:55.922 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6807", 2024-09-10T19:06:55.922 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3855640936 2024-09-10T19:06:55.922 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.922 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.922 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.922 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addr": "172.21.15.18:6805/3855640936", 2024-09-10T19:06:55.922 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addr": "172.21.15.18:6806/3855640936", 2024-09-10T19:06:55.922 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addr": "172.21.15.18:6808/3855640936", 2024-09-10T19:06:55.922 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addr": "172.21.15.18:6807/3855640936", 2024-09-10T19:06:55.922 INFO:teuthology.orchestra.run.smithi018.stdout: "state": [ 2024-09-10T19:06:55.923 INFO:teuthology.orchestra.run.smithi018.stdout: "exists", 2024-09-10T19:06:55.923 INFO:teuthology.orchestra.run.smithi018.stdout: "up" 2024-09-10T19:06:55.923 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.923 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.923 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.923 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 2, 2024-09-10T19:06:55.923 INFO:teuthology.orchestra.run.smithi018.stdout: "uuid": "79a01c29-4f4d-43d1-b1ea-a3fcf014f84b", 2024-09-10T19:06:55.923 INFO:teuthology.orchestra.run.smithi018.stdout: "up": 1, 2024-09-10T19:06:55.923 INFO:teuthology.orchestra.run.smithi018.stdout: "in": 0, 2024-09-10T19:06:55.923 INFO:teuthology.orchestra.run.smithi018.stdout: "weight": 0, 2024-09-10T19:06:55.923 INFO:teuthology.orchestra.run.smithi018.stdout: "primary_affinity": 1, 2024-09-10T19:06:55.923 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_begin": 0, 2024-09-10T19:06:55.923 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_end": 0, 2024-09-10T19:06:55.923 INFO:teuthology.orchestra.run.smithi018.stdout: "up_from": 18, 2024-09-10T19:06:55.924 INFO:teuthology.orchestra.run.smithi018.stdout: "up_thru": 86, 2024-09-10T19:06:55.924 INFO:teuthology.orchestra.run.smithi018.stdout: "down_at": 0, 2024-09-10T19:06:55.924 INFO:teuthology.orchestra.run.smithi018.stdout: "lost_at": 0, 2024-09-10T19:06:55.924 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addrs": { 2024-09-10T19:06:55.924 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T19:06:55.924 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.924 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T19:06:55.924 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6809", 2024-09-10T19:06:55.924 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 4107623855 2024-09-10T19:06:55.924 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.924 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.924 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.924 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addrs": { 2024-09-10T19:06:55.925 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T19:06:55.925 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.925 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T19:06:55.925 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6810", 2024-09-10T19:06:55.925 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 4107623855 2024-09-10T19:06:55.925 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.925 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.925 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.925 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addrs": { 2024-09-10T19:06:55.925 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T19:06:55.925 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.925 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T19:06:55.925 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6812", 2024-09-10T19:06:55.926 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 4107623855 2024-09-10T19:06:55.926 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.926 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.926 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.926 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addrs": { 2024-09-10T19:06:55.926 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T19:06:55.926 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.926 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T19:06:55.926 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6811", 2024-09-10T19:06:55.926 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 4107623855 2024-09-10T19:06:55.926 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.926 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.926 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.926 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addr": "172.21.15.18:6809/4107623855", 2024-09-10T19:06:55.927 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addr": "172.21.15.18:6810/4107623855", 2024-09-10T19:06:55.927 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addr": "172.21.15.18:6812/4107623855", 2024-09-10T19:06:55.927 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addr": "172.21.15.18:6811/4107623855", 2024-09-10T19:06:55.927 INFO:teuthology.orchestra.run.smithi018.stdout: "state": [ 2024-09-10T19:06:55.927 INFO:teuthology.orchestra.run.smithi018.stdout: "exists", 2024-09-10T19:06:55.927 INFO:teuthology.orchestra.run.smithi018.stdout: "up" 2024-09-10T19:06:55.927 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.927 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.927 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.927 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 3, 2024-09-10T19:06:55.927 INFO:teuthology.orchestra.run.smithi018.stdout: "uuid": "1809e677-b98c-4924-982e-9297e37563af", 2024-09-10T19:06:55.927 INFO:teuthology.orchestra.run.smithi018.stdout: "up": 1, 2024-09-10T19:06:55.927 INFO:teuthology.orchestra.run.smithi018.stdout: "in": 1, 2024-09-10T19:06:55.928 INFO:teuthology.orchestra.run.smithi018.stdout: "weight": 1, 2024-09-10T19:06:55.928 INFO:teuthology.orchestra.run.smithi018.stdout: "primary_affinity": 1, 2024-09-10T19:06:55.928 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_begin": 0, 2024-09-10T19:06:55.928 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_end": 0, 2024-09-10T19:06:55.928 INFO:teuthology.orchestra.run.smithi018.stdout: "up_from": 25, 2024-09-10T19:06:55.928 INFO:teuthology.orchestra.run.smithi018.stdout: "up_thru": 437, 2024-09-10T19:06:55.928 INFO:teuthology.orchestra.run.smithi018.stdout: "down_at": 0, 2024-09-10T19:06:55.928 INFO:teuthology.orchestra.run.smithi018.stdout: "lost_at": 0, 2024-09-10T19:06:55.928 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addrs": { 2024-09-10T19:06:55.929 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T19:06:55.929 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.929 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T19:06:55.929 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6813", 2024-09-10T19:06:55.929 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 2732204253 2024-09-10T19:06:55.929 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.929 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.929 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.929 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addrs": { 2024-09-10T19:06:55.929 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T19:06:55.929 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.929 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T19:06:55.929 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6814", 2024-09-10T19:06:55.930 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 2732204253 2024-09-10T19:06:55.930 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.930 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.930 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.930 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addrs": { 2024-09-10T19:06:55.930 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T19:06:55.930 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.930 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T19:06:55.930 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6816", 2024-09-10T19:06:55.930 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 2732204253 2024-09-10T19:06:55.930 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.930 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.930 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.931 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addrs": { 2024-09-10T19:06:55.931 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T19:06:55.931 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.931 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T19:06:55.931 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.18:6815", 2024-09-10T19:06:55.931 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 2732204253 2024-09-10T19:06:55.931 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.931 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.931 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.931 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addr": "172.21.15.18:6813/2732204253", 2024-09-10T19:06:55.931 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addr": "172.21.15.18:6814/2732204253", 2024-09-10T19:06:55.931 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addr": "172.21.15.18:6816/2732204253", 2024-09-10T19:06:55.931 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addr": "172.21.15.18:6815/2732204253", 2024-09-10T19:06:55.931 INFO:teuthology.orchestra.run.smithi018.stdout: "state": [ 2024-09-10T19:06:55.932 INFO:teuthology.orchestra.run.smithi018.stdout: "exists", 2024-09-10T19:06:55.932 INFO:teuthology.orchestra.run.smithi018.stdout: "up" 2024-09-10T19:06:55.932 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.932 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.932 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.932 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 4, 2024-09-10T19:06:55.932 INFO:teuthology.orchestra.run.smithi018.stdout: "uuid": "b682a300-bb53-4f36-8a9f-0753164ec4b7", 2024-09-10T19:06:55.932 INFO:teuthology.orchestra.run.smithi018.stdout: "up": 1, 2024-09-10T19:06:55.932 INFO:teuthology.orchestra.run.smithi018.stdout: "in": 1, 2024-09-10T19:06:55.932 INFO:teuthology.orchestra.run.smithi018.stdout: "weight": 1, 2024-09-10T19:06:55.932 INFO:teuthology.orchestra.run.smithi018.stdout: "primary_affinity": 1, 2024-09-10T19:06:55.932 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_begin": 0, 2024-09-10T19:06:55.932 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_end": 0, 2024-09-10T19:06:55.933 INFO:teuthology.orchestra.run.smithi018.stdout: "up_from": 30, 2024-09-10T19:06:55.933 INFO:teuthology.orchestra.run.smithi018.stdout: "up_thru": 438, 2024-09-10T19:06:55.933 INFO:teuthology.orchestra.run.smithi018.stdout: "down_at": 0, 2024-09-10T19:06:55.933 INFO:teuthology.orchestra.run.smithi018.stdout: "lost_at": 0, 2024-09-10T19:06:55.933 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addrs": { 2024-09-10T19:06:55.933 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T19:06:55.933 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.933 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T19:06:55.933 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6800", 2024-09-10T19:06:55.933 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 465860081 2024-09-10T19:06:55.933 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.933 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.934 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.934 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addrs": { 2024-09-10T19:06:55.934 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T19:06:55.934 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.934 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T19:06:55.934 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6801", 2024-09-10T19:06:55.934 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 465860081 2024-09-10T19:06:55.934 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.934 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.934 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.934 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addrs": { 2024-09-10T19:06:55.934 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T19:06:55.934 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.935 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T19:06:55.935 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6803", 2024-09-10T19:06:55.935 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 465860081 2024-09-10T19:06:55.935 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.935 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.935 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.935 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addrs": { 2024-09-10T19:06:55.935 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T19:06:55.935 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.935 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T19:06:55.935 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6802", 2024-09-10T19:06:55.935 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 465860081 2024-09-10T19:06:55.935 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.935 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.936 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.936 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addr": "172.21.15.103:6800/465860081", 2024-09-10T19:06:55.936 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addr": "172.21.15.103:6801/465860081", 2024-09-10T19:06:55.936 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addr": "172.21.15.103:6803/465860081", 2024-09-10T19:06:55.936 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addr": "172.21.15.103:6802/465860081", 2024-09-10T19:06:55.936 INFO:teuthology.orchestra.run.smithi018.stdout: "state": [ 2024-09-10T19:06:55.936 INFO:teuthology.orchestra.run.smithi018.stdout: "exists", 2024-09-10T19:06:55.936 INFO:teuthology.orchestra.run.smithi018.stdout: "up" 2024-09-10T19:06:55.936 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.936 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.936 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.936 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 5, 2024-09-10T19:06:55.936 INFO:teuthology.orchestra.run.smithi018.stdout: "uuid": "2bbfdb6c-0709-4fe4-8a4a-006f576f18e6", 2024-09-10T19:06:55.936 INFO:teuthology.orchestra.run.smithi018.stdout: "up": 1, 2024-09-10T19:06:55.937 INFO:teuthology.orchestra.run.smithi018.stdout: "in": 1, 2024-09-10T19:06:55.937 INFO:teuthology.orchestra.run.smithi018.stdout: "weight": 1, 2024-09-10T19:06:55.937 INFO:teuthology.orchestra.run.smithi018.stdout: "primary_affinity": 1, 2024-09-10T19:06:55.937 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_begin": 0, 2024-09-10T19:06:55.937 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_end": 0, 2024-09-10T19:06:55.937 INFO:teuthology.orchestra.run.smithi018.stdout: "up_from": 37, 2024-09-10T19:06:55.937 INFO:teuthology.orchestra.run.smithi018.stdout: "up_thru": 435, 2024-09-10T19:06:55.937 INFO:teuthology.orchestra.run.smithi018.stdout: "down_at": 0, 2024-09-10T19:06:55.937 INFO:teuthology.orchestra.run.smithi018.stdout: "lost_at": 0, 2024-09-10T19:06:55.937 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addrs": { 2024-09-10T19:06:55.937 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T19:06:55.937 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.937 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T19:06:55.937 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6804", 2024-09-10T19:06:55.938 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3498719258 2024-09-10T19:06:55.938 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.938 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.938 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.938 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addrs": { 2024-09-10T19:06:55.938 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T19:06:55.938 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.938 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T19:06:55.938 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6805", 2024-09-10T19:06:55.938 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3498719258 2024-09-10T19:06:55.938 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.938 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.938 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.939 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addrs": { 2024-09-10T19:06:55.939 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T19:06:55.939 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.939 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T19:06:55.939 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6807", 2024-09-10T19:06:55.939 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3498719258 2024-09-10T19:06:55.939 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.939 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.939 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.939 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addrs": { 2024-09-10T19:06:55.939 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T19:06:55.939 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.939 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T19:06:55.939 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6806", 2024-09-10T19:06:55.940 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3498719258 2024-09-10T19:06:55.940 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.940 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.940 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.940 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addr": "172.21.15.103:6804/3498719258", 2024-09-10T19:06:55.940 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addr": "172.21.15.103:6805/3498719258", 2024-09-10T19:06:55.940 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addr": "172.21.15.103:6807/3498719258", 2024-09-10T19:06:55.940 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addr": "172.21.15.103:6806/3498719258", 2024-09-10T19:06:55.940 INFO:teuthology.orchestra.run.smithi018.stdout: "state": [ 2024-09-10T19:06:55.940 INFO:teuthology.orchestra.run.smithi018.stdout: "exists", 2024-09-10T19:06:55.940 INFO:teuthology.orchestra.run.smithi018.stdout: "up" 2024-09-10T19:06:55.940 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.940 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.940 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.941 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 6, 2024-09-10T19:06:55.941 INFO:teuthology.orchestra.run.smithi018.stdout: "uuid": "31f9711e-0ba8-4aeb-b852-7d93e8fcce33", 2024-09-10T19:06:55.941 INFO:teuthology.orchestra.run.smithi018.stdout: "up": 1, 2024-09-10T19:06:55.941 INFO:teuthology.orchestra.run.smithi018.stdout: "in": 1, 2024-09-10T19:06:55.941 INFO:teuthology.orchestra.run.smithi018.stdout: "weight": 1, 2024-09-10T19:06:55.941 INFO:teuthology.orchestra.run.smithi018.stdout: "primary_affinity": 1, 2024-09-10T19:06:55.941 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_begin": 0, 2024-09-10T19:06:55.941 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_end": 0, 2024-09-10T19:06:55.941 INFO:teuthology.orchestra.run.smithi018.stdout: "up_from": 43, 2024-09-10T19:06:55.941 INFO:teuthology.orchestra.run.smithi018.stdout: "up_thru": 429, 2024-09-10T19:06:55.941 INFO:teuthology.orchestra.run.smithi018.stdout: "down_at": 0, 2024-09-10T19:06:55.941 INFO:teuthology.orchestra.run.smithi018.stdout: "lost_at": 0, 2024-09-10T19:06:55.941 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addrs": { 2024-09-10T19:06:55.941 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T19:06:55.941 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.942 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T19:06:55.942 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6808", 2024-09-10T19:06:55.942 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 184353788 2024-09-10T19:06:55.942 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.942 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.942 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.942 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addrs": { 2024-09-10T19:06:55.942 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T19:06:55.942 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.942 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T19:06:55.943 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6809", 2024-09-10T19:06:55.943 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 184353788 2024-09-10T19:06:55.943 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.943 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.943 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.943 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addrs": { 2024-09-10T19:06:55.943 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T19:06:55.943 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.943 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T19:06:55.943 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6811", 2024-09-10T19:06:55.943 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 184353788 2024-09-10T19:06:55.943 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.943 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.944 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.944 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addrs": { 2024-09-10T19:06:55.944 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T19:06:55.944 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.944 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T19:06:55.944 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6810", 2024-09-10T19:06:55.944 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 184353788 2024-09-10T19:06:55.944 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.944 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.944 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.944 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addr": "172.21.15.103:6808/184353788", 2024-09-10T19:06:55.944 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addr": "172.21.15.103:6809/184353788", 2024-09-10T19:06:55.944 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addr": "172.21.15.103:6811/184353788", 2024-09-10T19:06:55.944 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addr": "172.21.15.103:6810/184353788", 2024-09-10T19:06:55.944 INFO:teuthology.orchestra.run.smithi018.stdout: "state": [ 2024-09-10T19:06:55.945 INFO:teuthology.orchestra.run.smithi018.stdout: "exists", 2024-09-10T19:06:55.945 INFO:teuthology.orchestra.run.smithi018.stdout: "up" 2024-09-10T19:06:55.945 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.945 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.945 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.945 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 7, 2024-09-10T19:06:55.945 INFO:teuthology.orchestra.run.smithi018.stdout: "uuid": "f12282f8-4747-45ff-8336-95440560a88f", 2024-09-10T19:06:55.945 INFO:teuthology.orchestra.run.smithi018.stdout: "up": 1, 2024-09-10T19:06:55.945 INFO:teuthology.orchestra.run.smithi018.stdout: "in": 0, 2024-09-10T19:06:55.945 INFO:teuthology.orchestra.run.smithi018.stdout: "weight": 0, 2024-09-10T19:06:55.945 INFO:teuthology.orchestra.run.smithi018.stdout: "primary_affinity": 1, 2024-09-10T19:06:55.945 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_begin": 52, 2024-09-10T19:06:55.945 INFO:teuthology.orchestra.run.smithi018.stdout: "last_clean_end": 361, 2024-09-10T19:06:55.945 INFO:teuthology.orchestra.run.smithi018.stdout: "up_from": 411, 2024-09-10T19:06:55.946 INFO:teuthology.orchestra.run.smithi018.stdout: "up_thru": 241, 2024-09-10T19:06:55.946 INFO:teuthology.orchestra.run.smithi018.stdout: "down_at": 362, 2024-09-10T19:06:55.946 INFO:teuthology.orchestra.run.smithi018.stdout: "lost_at": 0, 2024-09-10T19:06:55.946 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addrs": { 2024-09-10T19:06:55.946 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T19:06:55.946 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.946 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T19:06:55.946 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6812", 2024-09-10T19:06:55.946 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3644107622 2024-09-10T19:06:55.946 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.946 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.946 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.946 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addrs": { 2024-09-10T19:06:55.946 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T19:06:55.946 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.946 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T19:06:55.947 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6813", 2024-09-10T19:06:55.947 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3644107622 2024-09-10T19:06:55.947 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.947 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.947 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.947 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addrs": { 2024-09-10T19:06:55.947 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T19:06:55.947 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.947 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T19:06:55.947 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6815", 2024-09-10T19:06:55.947 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3644107622 2024-09-10T19:06:55.947 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.947 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.947 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.948 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addrs": { 2024-09-10T19:06:55.948 INFO:teuthology.orchestra.run.smithi018.stdout: "addrvec": [ 2024-09-10T19:06:55.948 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.948 INFO:teuthology.orchestra.run.smithi018.stdout: "type": "v2", 2024-09-10T19:06:55.948 INFO:teuthology.orchestra.run.smithi018.stdout: "addr": "172.21.15.103:6814", 2024-09-10T19:06:55.948 INFO:teuthology.orchestra.run.smithi018.stdout: "nonce": 3644107622 2024-09-10T19:06:55.948 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.948 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.948 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.948 INFO:teuthology.orchestra.run.smithi018.stdout: "public_addr": "172.21.15.103:6812/3644107622", 2024-09-10T19:06:55.948 INFO:teuthology.orchestra.run.smithi018.stdout: "cluster_addr": "172.21.15.103:6813/3644107622", 2024-09-10T19:06:55.948 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_back_addr": "172.21.15.103:6815/3644107622", 2024-09-10T19:06:55.948 INFO:teuthology.orchestra.run.smithi018.stdout: "heartbeat_front_addr": "172.21.15.103:6814/3644107622", 2024-09-10T19:06:55.948 INFO:teuthology.orchestra.run.smithi018.stdout: "state": [ 2024-09-10T19:06:55.948 INFO:teuthology.orchestra.run.smithi018.stdout: "exists", 2024-09-10T19:06:55.949 INFO:teuthology.orchestra.run.smithi018.stdout: "up" 2024-09-10T19:06:55.949 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.949 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.949 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-09-10T19:06:55.949 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_xinfo": [ 2024-09-10T19:06:55.949 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.949 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 0, 2024-09-10T19:06:55.949 INFO:teuthology.orchestra.run.smithi018.stdout: "down_stamp": "0.000000", 2024-09-10T19:06:55.949 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_probability": 0, 2024-09-10T19:06:55.949 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_interval": 0, 2024-09-10T19:06:55.949 INFO:teuthology.orchestra.run.smithi018.stdout: "features": 4540138320759226367, 2024-09-10T19:06:55.949 INFO:teuthology.orchestra.run.smithi018.stdout: "old_weight": 0, 2024-09-10T19:06:55.949 INFO:teuthology.orchestra.run.smithi018.stdout: "last_purged_snaps_scrub": "2024-09-10T19:06:42.790903+0000", 2024-09-10T19:06:55.949 INFO:teuthology.orchestra.run.smithi018.stdout: "dead_epoch": 205 2024-09-10T19:06:55.950 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.950 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.950 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 1, 2024-09-10T19:06:55.950 INFO:teuthology.orchestra.run.smithi018.stdout: "down_stamp": "0.000000", 2024-09-10T19:06:55.950 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_probability": 0, 2024-09-10T19:06:55.950 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_interval": 0, 2024-09-10T19:06:55.950 INFO:teuthology.orchestra.run.smithi018.stdout: "features": 4540138320759226367, 2024-09-10T19:06:55.950 INFO:teuthology.orchestra.run.smithi018.stdout: "old_weight": 0, 2024-09-10T19:06:55.950 INFO:teuthology.orchestra.run.smithi018.stdout: "last_purged_snaps_scrub": "2024-09-10T19:06:50.800706+0000", 2024-09-10T19:06:55.950 INFO:teuthology.orchestra.run.smithi018.stdout: "dead_epoch": 0 2024-09-10T19:06:55.950 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.950 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.950 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 2, 2024-09-10T19:06:55.951 INFO:teuthology.orchestra.run.smithi018.stdout: "down_stamp": "0.000000", 2024-09-10T19:06:55.951 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_probability": 0, 2024-09-10T19:06:55.951 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_interval": 0, 2024-09-10T19:06:55.951 INFO:teuthology.orchestra.run.smithi018.stdout: "features": 4540138320759226367, 2024-09-10T19:06:55.951 INFO:teuthology.orchestra.run.smithi018.stdout: "old_weight": 65536, 2024-09-10T19:06:55.951 INFO:teuthology.orchestra.run.smithi018.stdout: "last_purged_snaps_scrub": "2024-09-10T19:06:27.893606+0000", 2024-09-10T19:06:55.951 INFO:teuthology.orchestra.run.smithi018.stdout: "dead_epoch": 0 2024-09-10T19:06:55.951 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.951 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.951 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 3, 2024-09-10T19:06:55.951 INFO:teuthology.orchestra.run.smithi018.stdout: "down_stamp": "0.000000", 2024-09-10T19:06:55.951 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_probability": 0, 2024-09-10T19:06:55.951 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_interval": 0, 2024-09-10T19:06:55.951 INFO:teuthology.orchestra.run.smithi018.stdout: "features": 4540138320759226367, 2024-09-10T19:06:55.952 INFO:teuthology.orchestra.run.smithi018.stdout: "old_weight": 0, 2024-09-10T19:06:55.952 INFO:teuthology.orchestra.run.smithi018.stdout: "last_purged_snaps_scrub": "2024-09-10T19:06:16.654188+0000", 2024-09-10T19:06:55.952 INFO:teuthology.orchestra.run.smithi018.stdout: "dead_epoch": 0 2024-09-10T19:06:55.952 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.952 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.952 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 4, 2024-09-10T19:06:55.952 INFO:teuthology.orchestra.run.smithi018.stdout: "down_stamp": "0.000000", 2024-09-10T19:06:55.952 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_probability": 0, 2024-09-10T19:06:55.952 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_interval": 0, 2024-09-10T19:06:55.952 INFO:teuthology.orchestra.run.smithi018.stdout: "features": 4540138320759226367, 2024-09-10T19:06:55.952 INFO:teuthology.orchestra.run.smithi018.stdout: "old_weight": 0, 2024-09-10T19:06:55.952 INFO:teuthology.orchestra.run.smithi018.stdout: "last_purged_snaps_scrub": "2024-09-10T19:06:11.379179+0000", 2024-09-10T19:06:55.953 INFO:teuthology.orchestra.run.smithi018.stdout: "dead_epoch": 0 2024-09-10T19:06:55.953 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.953 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.953 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 5, 2024-09-10T19:06:55.953 INFO:teuthology.orchestra.run.smithi018.stdout: "down_stamp": "0.000000", 2024-09-10T19:06:55.953 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_probability": 0, 2024-09-10T19:06:55.954 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_interval": 0, 2024-09-10T19:06:55.954 INFO:teuthology.orchestra.run.smithi018.stdout: "features": 4540138320759226367, 2024-09-10T19:06:55.954 INFO:teuthology.orchestra.run.smithi018.stdout: "old_weight": 0, 2024-09-10T19:06:55.954 INFO:teuthology.orchestra.run.smithi018.stdout: "last_purged_snaps_scrub": "2024-09-10T19:06:12.925067+0000", 2024-09-10T19:06:55.954 INFO:teuthology.orchestra.run.smithi018.stdout: "dead_epoch": 0 2024-09-10T19:06:55.954 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.954 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.954 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 6, 2024-09-10T19:06:55.954 INFO:teuthology.orchestra.run.smithi018.stdout: "down_stamp": "0.000000", 2024-09-10T19:06:55.954 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_probability": 0, 2024-09-10T19:06:55.954 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_interval": 0, 2024-09-10T19:06:55.954 INFO:teuthology.orchestra.run.smithi018.stdout: "features": 4540138320759226367, 2024-09-10T19:06:55.954 INFO:teuthology.orchestra.run.smithi018.stdout: "old_weight": 0, 2024-09-10T19:06:55.954 INFO:teuthology.orchestra.run.smithi018.stdout: "last_purged_snaps_scrub": "2024-09-10T19:06:30.048912+0000", 2024-09-10T19:06:55.955 INFO:teuthology.orchestra.run.smithi018.stdout: "dead_epoch": 0 2024-09-10T19:06:55.955 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.955 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.955 INFO:teuthology.orchestra.run.smithi018.stdout: "osd": 7, 2024-09-10T19:06:55.955 INFO:teuthology.orchestra.run.smithi018.stdout: "down_stamp": "0.000000", 2024-09-10T19:06:55.955 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_probability": 0, 2024-09-10T19:06:55.955 INFO:teuthology.orchestra.run.smithi018.stdout: "laggy_interval": 0, 2024-09-10T19:06:55.956 INFO:teuthology.orchestra.run.smithi018.stdout: "features": 4540138320759226367, 2024-09-10T19:06:55.956 INFO:teuthology.orchestra.run.smithi018.stdout: "old_weight": 65536, 2024-09-10T19:06:55.956 INFO:teuthology.orchestra.run.smithi018.stdout: "last_purged_snaps_scrub": "2024-09-10T19:06:10.600332+0000", 2024-09-10T19:06:55.956 INFO:teuthology.orchestra.run.smithi018.stdout: "dead_epoch": 361 2024-09-10T19:06:55.956 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.956 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-09-10T19:06:55.956 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_upmap": [ 2024-09-10T19:06:55.956 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.957 INFO:teuthology.orchestra.run.smithi018.stdout: "pgid": "2.11", 2024-09-10T19:06:55.957 INFO:teuthology.orchestra.run.smithi018.stdout: "osds": [ 2024-09-10T19:06:55.957 INFO:teuthology.orchestra.run.smithi018.stdout: 6, 2024-09-10T19:06:55.957 INFO:teuthology.orchestra.run.smithi018.stdout: 7, 2024-09-10T19:06:55.957 INFO:teuthology.orchestra.run.smithi018.stdout: 5 2024-09-10T19:06:55.957 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.957 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.957 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-09-10T19:06:55.958 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_upmap_items": [ 2024-09-10T19:06:55.958 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.958 INFO:teuthology.orchestra.run.smithi018.stdout: "pgid": "4.e", 2024-09-10T19:06:55.958 INFO:teuthology.orchestra.run.smithi018.stdout: "mappings": [ 2024-09-10T19:06:55.958 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.958 INFO:teuthology.orchestra.run.smithi018.stdout: "from": 1, 2024-09-10T19:06:55.958 INFO:teuthology.orchestra.run.smithi018.stdout: "to": 5 2024-09-10T19:06:55.958 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.958 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.958 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.958 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.958 INFO:teuthology.orchestra.run.smithi018.stdout: "pgid": "4.f", 2024-09-10T19:06:55.958 INFO:teuthology.orchestra.run.smithi018.stdout: "mappings": [ 2024-09-10T19:06:55.959 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.959 INFO:teuthology.orchestra.run.smithi018.stdout: "from": 1, 2024-09-10T19:06:55.959 INFO:teuthology.orchestra.run.smithi018.stdout: "to": 5 2024-09-10T19:06:55.959 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.959 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.959 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.959 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.959 INFO:teuthology.orchestra.run.smithi018.stdout: "pgid": "5.5", 2024-09-10T19:06:55.959 INFO:teuthology.orchestra.run.smithi018.stdout: "mappings": [ 2024-09-10T19:06:55.959 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.959 INFO:teuthology.orchestra.run.smithi018.stdout: "from": 0, 2024-09-10T19:06:55.959 INFO:teuthology.orchestra.run.smithi018.stdout: "to": 6 2024-09-10T19:06:55.959 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.960 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.960 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.960 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.960 INFO:teuthology.orchestra.run.smithi018.stdout: "pgid": "5.7", 2024-09-10T19:06:55.960 INFO:teuthology.orchestra.run.smithi018.stdout: "mappings": [ 2024-09-10T19:06:55.960 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.960 INFO:teuthology.orchestra.run.smithi018.stdout: "from": 1, 2024-09-10T19:06:55.960 INFO:teuthology.orchestra.run.smithi018.stdout: "to": 6 2024-09-10T19:06:55.960 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.960 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.960 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.960 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.961 INFO:teuthology.orchestra.run.smithi018.stdout: "pgid": "5.11", 2024-09-10T19:06:55.961 INFO:teuthology.orchestra.run.smithi018.stdout: "mappings": [ 2024-09-10T19:06:55.961 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.961 INFO:teuthology.orchestra.run.smithi018.stdout: "from": 4, 2024-09-10T19:06:55.961 INFO:teuthology.orchestra.run.smithi018.stdout: "to": 3 2024-09-10T19:06:55.961 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.961 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.961 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.961 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.961 INFO:teuthology.orchestra.run.smithi018.stdout: "pgid": "5.15", 2024-09-10T19:06:55.961 INFO:teuthology.orchestra.run.smithi018.stdout: "mappings": [ 2024-09-10T19:06:55.961 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.961 INFO:teuthology.orchestra.run.smithi018.stdout: "from": 1, 2024-09-10T19:06:55.962 INFO:teuthology.orchestra.run.smithi018.stdout: "to": 6 2024-09-10T19:06:55.962 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.962 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.962 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.962 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.962 INFO:teuthology.orchestra.run.smithi018.stdout: "pgid": "5.19", 2024-09-10T19:06:55.962 INFO:teuthology.orchestra.run.smithi018.stdout: "mappings": [ 2024-09-10T19:06:55.962 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.962 INFO:teuthology.orchestra.run.smithi018.stdout: "from": 1, 2024-09-10T19:06:55.962 INFO:teuthology.orchestra.run.smithi018.stdout: "to": 6 2024-09-10T19:06:55.962 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.962 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.963 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.963 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.963 INFO:teuthology.orchestra.run.smithi018.stdout: "pgid": "5.1a", 2024-09-10T19:06:55.963 INFO:teuthology.orchestra.run.smithi018.stdout: "mappings": [ 2024-09-10T19:06:55.963 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.963 INFO:teuthology.orchestra.run.smithi018.stdout: "from": 1, 2024-09-10T19:06:55.963 INFO:teuthology.orchestra.run.smithi018.stdout: "to": 3 2024-09-10T19:06:55.963 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.963 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.963 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.963 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.963 INFO:teuthology.orchestra.run.smithi018.stdout: "pgid": "6.f", 2024-09-10T19:06:55.963 INFO:teuthology.orchestra.run.smithi018.stdout: "mappings": [ 2024-09-10T19:06:55.964 INFO:teuthology.orchestra.run.smithi018.stdout: { 2024-09-10T19:06:55.964 INFO:teuthology.orchestra.run.smithi018.stdout: "from": 3, 2024-09-10T19:06:55.964 INFO:teuthology.orchestra.run.smithi018.stdout: "to": 0 2024-09-10T19:06:55.964 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.964 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-10T19:06:55.964 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.964 INFO:teuthology.orchestra.run.smithi018.stdout: ], 2024-09-10T19:06:55.964 INFO:teuthology.orchestra.run.smithi018.stdout: "pg_temp": [], 2024-09-10T19:06:55.964 INFO:teuthology.orchestra.run.smithi018.stdout: "primary_temp": [], 2024-09-10T19:06:55.964 INFO:teuthology.orchestra.run.smithi018.stdout: "blocklist": { 2024-09-10T19:06:55.964 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/3021537013": "2024-09-11T18:45:54.998616+0000", 2024-09-10T19:06:55.965 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/458572898": "2024-09-11T18:40:28.880042+0000", 2024-09-10T19:06:55.965 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/1859541419": "2024-09-11T18:40:28.880042+0000", 2024-09-10T19:06:55.965 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/3090642970": "2024-09-11T18:40:59.659411+0000", 2024-09-10T19:06:55.965 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/2875894312": "2024-09-11T18:40:59.659411+0000", 2024-09-10T19:06:55.965 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/178638059": "2024-09-11T18:40:59.659411+0000", 2024-09-10T19:06:55.965 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/692931402": "2024-09-11T18:45:54.998616+0000", 2024-09-10T19:06:55.965 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/2408360238": "2024-09-11T18:45:54.998616+0000", 2024-09-10T19:06:55.965 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/57192865": "2024-09-11T18:40:28.880042+0000", 2024-09-10T19:06:55.965 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:6800/2748756682": "2024-09-11T18:40:59.659411+0000", 2024-09-10T19:06:55.965 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:6800/732375116": "2024-09-11T18:45:54.998616+0000", 2024-09-10T19:06:55.965 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:6800/3869801150": "2024-09-11T18:40:28.880042+0000", 2024-09-10T19:06:55.966 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/2982361259": "2024-09-11T18:45:54.998616+0000", 2024-09-10T19:06:55.966 INFO:teuthology.orchestra.run.smithi018.stdout: "172.21.15.18:0/185700038": "2024-09-11T18:45:54.998616+0000" 2024-09-10T19:06:55.966 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.966 INFO:teuthology.orchestra.run.smithi018.stdout: "range_blocklist": {}, 2024-09-10T19:06:55.966 INFO:teuthology.orchestra.run.smithi018.stdout: "erasure_code_profiles": { 2024-09-10T19:06:55.966 INFO:teuthology.orchestra.run.smithi018.stdout: "default": { 2024-09-10T19:06:55.966 INFO:teuthology.orchestra.run.smithi018.stdout: "crush-failure-domain": "osd", 2024-09-10T19:06:55.966 INFO:teuthology.orchestra.run.smithi018.stdout: "k": "2", 2024-09-10T19:06:55.966 INFO:teuthology.orchestra.run.smithi018.stdout: "m": "1", 2024-09-10T19:06:55.966 INFO:teuthology.orchestra.run.smithi018.stdout: "plugin": "jerasure", 2024-09-10T19:06:55.966 INFO:teuthology.orchestra.run.smithi018.stdout: "technique": "reed_sol_van" 2024-09-10T19:06:55.966 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.967 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-10T19:06:55.967 INFO:teuthology.orchestra.run.smithi018.stdout: "removed_snaps_queue": [], 2024-09-10T19:06:55.967 INFO:teuthology.orchestra.run.smithi018.stdout: "new_removed_snaps": [], 2024-09-10T19:06:55.967 INFO:teuthology.orchestra.run.smithi018.stdout: "new_purged_snaps": [], 2024-09-10T19:06:55.967 INFO:teuthology.orchestra.run.smithi018.stdout: "crush_node_flags": {}, 2024-09-10T19:06:55.967 INFO:teuthology.orchestra.run.smithi018.stdout: "device_class_flags": {}, 2024-09-10T19:06:55.967 INFO:teuthology.orchestra.run.smithi018.stdout: "stretch_mode": { 2024-09-10T19:06:55.967 INFO:teuthology.orchestra.run.smithi018.stdout: "stretch_mode_enabled": false, 2024-09-10T19:06:55.967 INFO:teuthology.orchestra.run.smithi018.stdout: "stretch_bucket_count": 0, 2024-09-10T19:06:55.967 INFO:teuthology.orchestra.run.smithi018.stdout: "degraded_stretch_mode": 0, 2024-09-10T19:06:55.967 INFO:teuthology.orchestra.run.smithi018.stdout: "recovering_stretch_mode": 0, 2024-09-10T19:06:55.967 INFO:teuthology.orchestra.run.smithi018.stdout: "stretch_mode_bucket": 0 2024-09-10T19:06:55.968 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-10T19:06:55.968 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-10T19:06:55.971 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:55.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:55.994 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:55.995 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:56.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:06:56.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:55 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2267023463' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T19:06:56.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:55 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T19:06:56.095 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:56.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:56.118 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:56.118 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:56.219 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:56.250 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:56.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:56.252 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:56.353 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:56.377 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:56.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:56.378 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:56.479 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:56.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:56.502 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:56.502 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:56.603 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2 2024-09-10T19:06:56.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:56.626 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:56.626 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.2' 2024-09-10T19:06:56.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:56 smithi018 ceph-mon[29336]: pgmap v1015: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 12 MiB data, 7.4 GiB used, 529 GiB / 536 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T19:06:56.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:56 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T19:06:56.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:56 smithi018 ceph-mon[29336]: osdmap e453: 8 total, 8 up, 6 in 2024-09-10T19:06:56.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:56 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1583546274' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-10T19:06:56.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:56 smithi018 ceph-mon[40196]: pgmap v1015: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 12 MiB data, 7.4 GiB used, 529 GiB / 536 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T19:06:56.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:56 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T19:06:56.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:56 smithi018 ceph-mon[40196]: osdmap e453: 8 total, 8 up, 6 in 2024-09-10T19:06:56.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:56 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1583546274' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-10T19:06:56.727 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:56.749 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:56.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:56.750 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:56.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:56 smithi103 ceph-mon[33163]: pgmap v1015: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 12 MiB data, 7.4 GiB used, 529 GiB / 536 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T19:06:56.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:56 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T19:06:56.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:56 smithi103 ceph-mon[33163]: osdmap e453: 8 total, 8 up, 6 in 2024-09-10T19:06:56.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:56 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1583546274' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-10T19:06:56.773 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 19:06:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a[63552]: logger=cleanup t=2024-09-10T19:06:56.43892614Z level=info msg="Completed cleanup jobs" duration=12.981933ms 2024-09-10T19:06:56.851 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:56.874 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:56.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:56.875 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:56.976 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:56.999 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:56.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:57.000 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:57.101 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:57.123 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:57.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:57.124 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:57.225 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1 2024-09-10T19:06:57.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:57.248 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:57.249 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.1' 2024-09-10T19:06:57.350 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:57.371 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:57.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:57.372 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:57.473 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:57.495 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:57.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:57.496 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:57.597 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3 2024-09-10T19:06:57.621 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:57.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:57.622 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.3' 2024-09-10T19:06:57.636 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:06:57 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:06:57.723 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:57.745 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:57.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:57.746 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:57.847 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:57.869 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:57.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:57.871 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:57.891 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:57 smithi018 ceph-mon[29336]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-10T19:06:57.891 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:57 smithi018 ceph-mon[40196]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-10T19:06:57.909 INFO:teuthology.orchestra.run.smithi018.stdout:osd.0: {} 2024-09-10T19:06:57.909 INFO:teuthology.orchestra.run.smithi018.stdout:osd.1: {} 2024-09-10T19:06:57.910 INFO:teuthology.orchestra.run.smithi018.stdout:osd.2: {} 2024-09-10T19:06:57.910 INFO:teuthology.orchestra.run.smithi018.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-10T19:06:57.910 INFO:teuthology.orchestra.run.smithi018.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-10T19:06:57.910 INFO:teuthology.orchestra.run.smithi018.stdout:osd.3: {} 2024-09-10T19:06:57.910 INFO:teuthology.orchestra.run.smithi018.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-10T19:06:57.911 INFO:teuthology.orchestra.run.smithi018.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-10T19:06:57.922 INFO:teuthology.orchestra.run.smithi018.stdout:osd.4: {} 2024-09-10T19:06:57.922 INFO:teuthology.orchestra.run.smithi018.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-10T19:06:57.937 INFO:teuthology.orchestra.run.smithi018.stdout:osd.5: {} 2024-09-10T19:06:57.938 INFO:teuthology.orchestra.run.smithi018.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-10T19:06:57.953 INFO:teuthology.orchestra.run.smithi018.stdout:osd.6: {} 2024-09-10T19:06:57.953 INFO:teuthology.orchestra.run.smithi018.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-10T19:06:57.972 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:57.973 INFO:teuthology.orchestra.run.smithi018.stdout:osd.7: {} 2024-09-10T19:06:57.973 INFO:teuthology.orchestra.run.smithi018.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-10T19:06:57.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:57.994 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:57.994 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:58.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:57 smithi103 ceph-mon[33163]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-10T19:06:58.095 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:58.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:58.117 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:58.118 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:58.218 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6 2024-09-10T19:06:58.241 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:58.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:58.242 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.6' 2024-09-10T19:06:58.343 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:58.367 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:58.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:58.368 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:58.468 DEBUG:teuthology.orchestra.run.smithi018:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0 2024-09-10T19:06:58.493 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: docker: command not found 2024-09-10T19:06:58.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:58.494 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi018 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.0' 2024-09-10T19:06:58.518 INFO:tasks.thrashosds.thrasher:j is {'epoch': 453, 'fsid': 'ba327f6c-6fa3-11ef-bcea-c7b262605968', 'created': '2024-09-10T18:40:07.669687+0000', 'modified': '2024-09-10T19:06:55.576315+0000', 'last_up_change': '2024-09-10T19:04:50.812385+0000', 'last_in_change': '2024-09-10T18:57:01.396827+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 20, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 11, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T18:43:08.162659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 21, 'pg_placement_num': 21, 'pg_placement_num_target': 21, 'pg_num_target': 21, 'pg_num_pending': 21, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '394', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '167', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': '.rgw.root', 'create_time': '2024-09-10T18:45:16.512219+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 3, 'pool_name': 'datapool', 'create_time': '2024-09-10T18:45:16.943626+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 60, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-10T18:45:18.450026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-10T18:45:20.494618+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-10T18:45:22.566992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}}], 'osds': [{'osd': 0, 'uuid': 'eb4c7e67-300c-4a1a-b9f5-bb028eeeabc2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 8, 'last_clean_end': 205, 'up_from': 241, 'up_thru': 434, 'down_at': 206, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6801', 'nonce': 4100931169}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6802', 'nonce': 4100931169}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6804', 'nonce': 4100931169}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6803', 'nonce': 4100931169}]}, 'public_addr': '172.21.15.18:6801/4100931169', 'cluster_addr': '172.21.15.18:6802/4100931169', 'heartbeat_back_addr': '172.21.15.18:6804/4100931169', 'heartbeat_front_addr': '172.21.15.18:6803/4100931169', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '63bf6a98-e349-4cc0-9502-a31693070c5b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.010101318359375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 432, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6805', 'nonce': 3855640936}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6806', 'nonce': 3855640936}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6808', 'nonce': 3855640936}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6807', 'nonce': 3855640936}]}, 'public_addr': '172.21.15.18:6805/3855640936', 'cluster_addr': '172.21.15.18:6806/3855640936', 'heartbeat_back_addr': '172.21.15.18:6808/3855640936', 'heartbeat_front_addr': '172.21.15.18:6807/3855640936', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '79a01c29-4f4d-43d1-b1ea-a3fcf014f84b', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 86, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6809', 'nonce': 4107623855}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6810', 'nonce': 4107623855}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6812', 'nonce': 4107623855}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6811', 'nonce': 4107623855}]}, 'public_addr': '172.21.15.18:6809/4107623855', 'cluster_addr': '172.21.15.18:6810/4107623855', 'heartbeat_back_addr': '172.21.15.18:6812/4107623855', 'heartbeat_front_addr': '172.21.15.18:6811/4107623855', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '1809e677-b98c-4924-982e-9297e37563af', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 25, 'up_thru': 437, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6813', 'nonce': 2732204253}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6814', 'nonce': 2732204253}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6816', 'nonce': 2732204253}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6815', 'nonce': 2732204253}]}, 'public_addr': '172.21.15.18:6813/2732204253', 'cluster_addr': '172.21.15.18:6814/2732204253', 'heartbeat_back_addr': '172.21.15.18:6816/2732204253', 'heartbeat_front_addr': '172.21.15.18:6815/2732204253', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'b682a300-bb53-4f36-8a9f-0753164ec4b7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 30, 'up_thru': 438, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6800', 'nonce': 465860081}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6801', 'nonce': 465860081}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6803', 'nonce': 465860081}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6802', 'nonce': 465860081}]}, 'public_addr': '172.21.15.103:6800/465860081', 'cluster_addr': '172.21.15.103:6801/465860081', 'heartbeat_back_addr': '172.21.15.103:6803/465860081', 'heartbeat_front_addr': '172.21.15.103:6802/465860081', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '2bbfdb6c-0709-4fe4-8a4a-006f576f18e6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 37, 'up_thru': 435, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6804', 'nonce': 3498719258}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6805', 'nonce': 3498719258}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6807', 'nonce': 3498719258}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6806', 'nonce': 3498719258}]}, 'public_addr': '172.21.15.103:6804/3498719258', 'cluster_addr': '172.21.15.103:6805/3498719258', 'heartbeat_back_addr': '172.21.15.103:6807/3498719258', 'heartbeat_front_addr': '172.21.15.103:6806/3498719258', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '31f9711e-0ba8-4aeb-b852-7d93e8fcce33', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 43, 'up_thru': 429, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6808', 'nonce': 184353788}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6809', 'nonce': 184353788}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6811', 'nonce': 184353788}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6810', 'nonce': 184353788}]}, 'public_addr': '172.21.15.103:6808/184353788', 'cluster_addr': '172.21.15.103:6809/184353788', 'heartbeat_back_addr': '172.21.15.103:6811/184353788', 'heartbeat_front_addr': '172.21.15.103:6810/184353788', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'f12282f8-4747-45ff-8336-95440560a88f', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 52, 'last_clean_end': 361, 'up_from': 411, 'up_thru': 241, 'down_at': 362, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6812', 'nonce': 3644107622}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6813', 'nonce': 3644107622}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6815', 'nonce': 3644107622}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6814', 'nonce': 3644107622}]}, 'public_addr': '172.21.15.103:6812/3644107622', 'cluster_addr': '172.21.15.103:6813/3644107622', 'heartbeat_back_addr': '172.21.15.103:6815/3644107622', 'heartbeat_front_addr': '172.21.15.103:6814/3644107622', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T19:06:42.790903+0000', 'dead_epoch': 205}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T19:06:50.800706+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-10T19:06:27.893606+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T19:06:16.654188+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T19:06:11.379179+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T19:06:12.925067+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-10T19:06:30.048912+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-10T19:06:10.600332+0000', 'dead_epoch': 361}], 'pg_upmap': [{'pgid': '2.11', 'osds': [6, 7, 5]}], 'pg_upmap_items': [{'pgid': '4.e', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.f', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '5.5', 'mappings': [{'from': 0, 'to': 6}]}, {'pgid': '5.7', 'mappings': [{'from': 1, 'to': 6}]}, {'pgid': '5.11', 'mappings': [{'from': 4, 'to': 3}]}, {'pgid': '5.15', 'mappings': [{'from': 1, 'to': 6}]}, {'pgid': '5.19', 'mappings': [{'from': 1, 'to': 6}]}, {'pgid': '5.1a', 'mappings': [{'from': 1, 'to': 3}]}, {'pgid': '6.f', 'mappings': [{'from': 3, 'to': 0}]}], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.18:0/3021537013': '2024-09-11T18:45:54.998616+0000', '172.21.15.18:0/458572898': '2024-09-11T18:40:28.880042+0000', '172.21.15.18:0/1859541419': '2024-09-11T18:40:28.880042+0000', '172.21.15.18:0/3090642970': '2024-09-11T18:40:59.659411+0000', '172.21.15.18:0/2875894312': '2024-09-11T18:40:59.659411+0000', '172.21.15.18:0/178638059': '2024-09-11T18:40:59.659411+0000', '172.21.15.18:0/692931402': '2024-09-11T18:45:54.998616+0000', '172.21.15.18:0/2408360238': '2024-09-11T18:45:54.998616+0000', '172.21.15.18:0/57192865': '2024-09-11T18:40:28.880042+0000', '172.21.15.18:6800/2748756682': '2024-09-11T18:40:59.659411+0000', '172.21.15.18:6800/732375116': '2024-09-11T18:45:54.998616+0000', '172.21.15.18:6800/3869801150': '2024-09-11T18:40:28.880042+0000', '172.21.15.18:0/2982361259': '2024-09-11T18:45:54.998616+0000', '172.21.15.18:0/185700038': '2024-09-11T18:45:54.998616+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-10T19:06:58.518 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T19:06:58.595 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5 2024-09-10T19:06:58.617 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:58.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:58.618 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.5' 2024-09-10T19:06:58.719 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4 2024-09-10T19:06:58.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:58.741 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:58.742 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.4' 2024-09-10T19:06:58.842 DEBUG:teuthology.orchestra.run.smithi103:> sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7 2024-09-10T19:06:58.864 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: docker: command not found 2024-09-10T19:06:58.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:06:58.865 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi103 with status 1: 'sudo docker kill -s 1 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd.7' 2024-09-10T19:06:58.876 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:58 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:58.877 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:58 smithi018 ceph-mon[29336]: pgmap v1017: 152 pgs: 152 active+clean; 582 KiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T19:06:58.877 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:06:58 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3857421807' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:06:58.877 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:58 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:58.877 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:58 smithi018 ceph-mon[40196]: pgmap v1017: 152 pgs: 152 active+clean; 582 KiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T19:06:58.877 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:06:58 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3857421807' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:06:58.907 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-10T19:06:58.920 INFO:tasks.thrashosds:joining thrashosds 2024-09-10T19:06:59.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:58 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:59.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:58 smithi103 ceph-mon[33163]: pgmap v1017: 152 pgs: 152 active+clean; 582 KiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T19:06:59.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:06:58 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3857421807' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-10T19:07:00.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:07:00 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:07:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:07:00.642 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is unset 2024-09-10T19:07:00.706 INFO:teuthology.orchestra.run.smithi018.stdout:ERROR: (22) Invalid argument 2024-09-10T19:07:00.706 INFO:teuthology.orchestra.run.smithi018.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T19:07:00.714 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:00 smithi018 ceph-mon[40196]: pgmap v1018: 152 pgs: 152 active+clean; 582 KiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T19:07:00.714 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:00 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1147592044' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:07:00.715 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:00 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:07:00.715 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:00 smithi018 ceph-mon[29336]: pgmap v1018: 152 pgs: 152 active+clean; 582 KiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T19:07:00.715 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:00 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1147592044' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:07:00.715 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:00 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:07:01.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:00 smithi103 ceph-mon[33163]: pgmap v1018: 152 pgs: 152 active+clean; 582 KiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T19:07:01.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:00 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1147592044' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:07:01.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:00 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:07:01.327 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:07:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:07:00] "GET /metrics HTTP/1.1" 200 224422 "" "Prometheus/2.43.0" 2024-09-10T19:07:01.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:01 smithi018 ceph-mon[29336]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:07:01.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:01 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T19:07:01.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:01 smithi018 ceph-mon[29336]: osdmap e454: 8 total, 8 up, 6 in 2024-09-10T19:07:01.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:01 smithi018 ceph-mon[40196]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:07:01.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:01 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T19:07:01.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:01 smithi018 ceph-mon[40196]: osdmap e454: 8 total, 8 up, 6 in 2024-09-10T19:07:01.979 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:07:02.002 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-10T19:07:02.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:01 smithi103 ceph-mon[33163]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-10T19:07:02.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:01 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T19:07:02.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:01 smithi103 ceph-mon[33163]: osdmap e454: 8 total, 8 up, 6 in 2024-09-10T19:07:02.263 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:07:02.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:02 smithi018 ceph-mon[29336]: 4.1a scrub starts 2024-09-10T19:07:02.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:02 smithi018 ceph-mon[29336]: 4.1a scrub ok 2024-09-10T19:07:02.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:02 smithi018 ceph-mon[29336]: pgmap v1020: 152 pgs: 152 active+clean; 582 KiB data, 7.2 GiB used, 529 GiB / 536 GiB avail; 511 B/s rd, 0 op/s 2024-09-10T19:07:02.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:02 smithi018 ceph-mon[29336]: 2.1a scrub starts 2024-09-10T19:07:02.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:02 smithi018 ceph-mon[29336]: 2.1a scrub ok 2024-09-10T19:07:02.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:02 smithi018 ceph-mon[40196]: 4.1a scrub starts 2024-09-10T19:07:02.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:02 smithi018 ceph-mon[40196]: 4.1a scrub ok 2024-09-10T19:07:02.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:02 smithi018 ceph-mon[40196]: pgmap v1020: 152 pgs: 152 active+clean; 582 KiB data, 7.2 GiB used, 529 GiB / 536 GiB avail; 511 B/s rd, 0 op/s 2024-09-10T19:07:02.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:02 smithi018 ceph-mon[40196]: 2.1a scrub starts 2024-09-10T19:07:02.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:02 smithi018 ceph-mon[40196]: 2.1a scrub ok 2024-09-10T19:07:03.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:02 smithi103 ceph-mon[33163]: 4.1a scrub starts 2024-09-10T19:07:03.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:02 smithi103 ceph-mon[33163]: 4.1a scrub ok 2024-09-10T19:07:03.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:02 smithi103 ceph-mon[33163]: pgmap v1020: 152 pgs: 152 active+clean; 582 KiB data, 7.2 GiB used, 529 GiB / 536 GiB avail; 511 B/s rd, 0 op/s 2024-09-10T19:07:03.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:02 smithi103 ceph-mon[33163]: 2.1a scrub starts 2024-09-10T19:07:03.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:02 smithi103 ceph-mon[33163]: 2.1a scrub ok 2024-09-10T19:07:03.354 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset noscrub 2024-09-10T19:07:03.699 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:03 smithi103 ceph-mon[33163]: 6.14 scrub starts 2024-09-10T19:07:03.699 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:03 smithi103 ceph-mon[33163]: 6.14 scrub ok 2024-09-10T19:07:03.700 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:03 smithi103 ceph-mon[33163]: 1.7 scrub starts 2024-09-10T19:07:03.700 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:03 smithi103 ceph-mon[33163]: 1.7 scrub ok 2024-09-10T19:07:03.703 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:07:03.806 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T19:07:03.809 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-09-10T19:07:03.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:03 smithi018 ceph-mon[29336]: 6.14 scrub starts 2024-09-10T19:07:03.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:03 smithi018 ceph-mon[29336]: 6.14 scrub ok 2024-09-10T19:07:03.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:03 smithi018 ceph-mon[29336]: 1.7 scrub starts 2024-09-10T19:07:03.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:03 smithi018 ceph-mon[29336]: 1.7 scrub ok 2024-09-10T19:07:03.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:03 smithi018 ceph-mon[40196]: 6.14 scrub starts 2024-09-10T19:07:03.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:03 smithi018 ceph-mon[40196]: 6.14 scrub ok 2024-09-10T19:07:03.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:03 smithi018 ceph-mon[40196]: 1.7 scrub starts 2024-09-10T19:07:03.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:03 smithi018 ceph-mon[40196]: 1.7 scrub ok 2024-09-10T19:07:04.127 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T19:07:04.127 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T19:07:04.791 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-10T19:07:04.793 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":1021,"stamp":"2024-09-10T19:07:03.525456+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":1937,"num_read_kb":1689,"num_write":1002,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":267589,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2863,"ondisk_log_size":2863,"up":456,"acting":456,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":479,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":562470912,"kb_used":7507684,"kb_used_data":323300,"kb_used_omap":12,"kb_used_meta":7184307,"kb_avail":554963228,"statfs":{"total":575970213888,"available":568282345472,"internally_reserved":0,"allocated":331059200,"data_stored":328643902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13122,"internal_metadata":7356730558},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11501568,"num_objects":-173,"num_object_clones":0,"num_object_copies":-519,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-173,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":-265955,"num_write_kb":-8499457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-31487,"num_bytes_recovered":-2063466496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002514"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":477,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.316362+0000","last_change":"2024-09-10T19:06:27.199518+0000","last_active":"2024-09-10T19:06:56.316362+0000","last_peered":"2024-09-10T19:06:56.316362+0000","last_clean":"2024-09-10T19:06:56.316362+0000","last_became_active":"2024-09-10T18:57:02.460451+0000","last_became_peered":"2024-09-10T18:57:02.460451+0000","last_unstale":"2024-09-10T19:06:56.316362+0000","last_undegraded":"2024-09-10T19:06:56.316362+0000","last_fullsized":"2024-09-10T19:06:56.316362+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:27.199436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:06:27.199436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:29.399699+0000","scrub_duration":0.0067911150000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":463,"reported_epoch":452,"state":"active+clean","last_fresh":"2024-09-10T19:06:50.839149+0000","last_change":"2024-09-10T19:03:38.825020+0000","last_active":"2024-09-10T19:06:50.839149+0000","last_peered":"2024-09-10T19:06:50.839149+0000","last_clean":"2024-09-10T19:06:50.839149+0000","last_became_active":"2024-09-10T18:57:02.440648+0000","last_became_peered":"2024-09-10T18:57:02.440648+0000","last_unstale":"2024-09-10T19:06:50.839149+0000","last_undegraded":"2024-09-10T19:06:50.839149+0000","last_fullsized":"2024-09-10T19:06:50.839149+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:38.824816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:59.848700+0000","last_clean_scrub_stamp":"2024-09-10T19:03:38.824816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0071402059999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"427'39","reported_seq":570,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.320383+0000","last_change":"2024-09-10T19:02:20.251547+0000","last_active":"2024-09-10T19:06:56.320383+0000","last_peered":"2024-09-10T19:06:56.320383+0000","last_clean":"2024-09-10T19:06:56.320383+0000","last_became_active":"2024-09-10T18:55:40.394717+0000","last_became_peered":"2024-09-10T18:55:40.394717+0000","last_unstale":"2024-09-10T19:06:56.320383+0000","last_undegraded":"2024-09-10T19:06:56.320383+0000","last_fullsized":"2024-09-10T19:06:56.320383+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'27","last_scrub_stamp":"2024-09-10T19:02:20.251456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:20.251456+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011781700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":469,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.062703+0000","last_change":"2024-09-10T19:04:39.483398+0000","last_active":"2024-09-10T19:06:56.062703+0000","last_peered":"2024-09-10T19:06:56.062703+0000","last_clean":"2024-09-10T19:06:56.062703+0000","last_became_active":"2024-09-10T18:45:22.065619+0000","last_became_peered":"2024-09-10T18:45:22.065619+0000","last_unstale":"2024-09-10T19:06:56.062703+0000","last_undegraded":"2024-09-10T19:06:56.062703+0000","last_fullsized":"2024-09-10T19:06:56.062703+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:39.483330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.057932+0000","last_clean_scrub_stamp":"2024-09-10T19:04:39.483330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016448439999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":530,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.316140+0000","last_change":"2024-09-10T19:02:17.284387+0000","last_active":"2024-09-10T19:06:56.316140+0000","last_peered":"2024-09-10T19:06:56.316140+0000","last_clean":"2024-09-10T19:06:56.316140+0000","last_became_active":"2024-09-10T19:00:42.518824+0000","last_became_peered":"2024-09-10T19:00:42.518824+0000","last_unstale":"2024-09-10T19:06:56.316140+0000","last_undegraded":"2024-09-10T19:06:56.316140+0000","last_fullsized":"2024-09-10T19:06:56.316140+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:17.284305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:17.284305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0018984430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"427'25","reported_seq":499,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.062618+0000","last_change":"2024-09-10T19:04:28.445705+0000","last_active":"2024-09-10T19:06:56.062618+0000","last_peered":"2024-09-10T19:06:56.062618+0000","last_clean":"2024-09-10T19:06:56.062618+0000","last_became_active":"2024-09-10T18:45:20.061266+0000","last_became_peered":"2024-09-10T18:45:20.061266+0000","last_unstale":"2024-09-10T19:06:56.062618+0000","last_undegraded":"2024-09-10T19:06:56.062618+0000","last_fullsized":"2024-09-10T19:06:56.062618+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T19:04:28.445625+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_clean_scrub_stamp":"2024-09-10T19:04:28.445625+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047732149999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":105,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.213534+0000","last_change":"2024-09-10T19:04:39.088299+0000","last_active":"2024-09-10T19:06:56.213534+0000","last_peered":"2024-09-10T19:06:56.213534+0000","last_clean":"2024-09-10T19:06:56.213534+0000","last_became_active":"2024-09-10T19:02:49.571992+0000","last_became_peered":"2024-09-10T19:02:49.571992+0000","last_unstale":"2024-09-10T19:06:56.213534+0000","last_undegraded":"2024-09-10T19:06:56.213534+0000","last_fullsized":"2024-09-10T19:06:56.213534+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:39.088231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:01.135988+0000","last_clean_scrub_stamp":"2024-09-10T19:04:39.088231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014640530000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":457,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.062887+0000","last_change":"2024-09-10T19:03:41.617846+0000","last_active":"2024-09-10T19:06:56.062887+0000","last_peered":"2024-09-10T19:06:56.062887+0000","last_clean":"2024-09-10T19:06:56.062887+0000","last_became_active":"2024-09-10T18:45:24.081002+0000","last_became_peered":"2024-09-10T18:45:24.081002+0000","last_unstale":"2024-09-10T19:06:56.062887+0000","last_undegraded":"2024-09-10T19:06:56.062887+0000","last_fullsized":"2024-09-10T19:06:56.062887+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:41.617790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:03:41.617790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0014252819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":462,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.384536+0000","last_change":"2024-09-10T19:03:30.120786+0000","last_active":"2024-09-10T19:06:56.384536+0000","last_peered":"2024-09-10T19:06:56.384536+0000","last_clean":"2024-09-10T19:06:56.384536+0000","last_became_active":"2024-09-10T18:57:02.452259+0000","last_became_peered":"2024-09-10T18:57:02.452259+0000","last_unstale":"2024-09-10T19:06:56.384536+0000","last_undegraded":"2024-09-10T19:06:56.384536+0000","last_fullsized":"2024-09-10T19:06:56.384536+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:30.120695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:21.501984+0000","last_clean_scrub_stamp":"2024-09-10T19:03:30.120695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034594489999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1b","version":"427'11","reported_seq":479,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.063384+0000","last_change":"2024-09-10T19:04:26.520312+0000","last_active":"2024-09-10T19:06:56.063384+0000","last_peered":"2024-09-10T19:06:56.063384+0000","last_clean":"2024-09-10T19:06:56.063384+0000","last_became_active":"2024-09-10T18:45:20.057567+0000","last_became_peered":"2024-09-10T18:45:20.057567+0000","last_unstale":"2024-09-10T19:06:56.063384+0000","last_undegraded":"2024-09-10T19:06:56.063384+0000","last_fullsized":"2024-09-10T19:06:56.063384+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'9","last_scrub_stamp":"2024-09-10T19:04:26.520225+0000","last_deep_scrub":"163'7","last_deep_scrub_stamp":"2024-09-10T18:51:54.261879+0000","last_clean_scrub_stamp":"2024-09-10T19:04:26.520225+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.005488153,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":13,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":228,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.062877+0000","last_change":"2024-09-10T19:06:11.858817+0000","last_active":"2024-09-10T19:06:56.062877+0000","last_peered":"2024-09-10T19:06:56.062877+0000","last_clean":"2024-09-10T19:06:56.062877+0000","last_became_active":"2024-09-10T18:58:58.503924+0000","last_became_peered":"2024-09-10T18:58:58.503924+0000","last_unstale":"2024-09-10T19:06:56.062877+0000","last_undegraded":"2024-09-10T19:06:56.062877+0000","last_fullsized":"2024-09-10T19:06:56.062877+0000","mapping_epoch":303,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":304,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:11.858728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:36.509971+0000","last_clean_scrub_stamp":"2024-09-10T19:06:11.858728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:19.964860+0000","scrub_duration":0.0083694589999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":135,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.213641+0000","last_change":"2024-09-10T19:03:23.759714+0000","last_active":"2024-09-10T19:06:56.213641+0000","last_peered":"2024-09-10T19:06:56.213641+0000","last_clean":"2024-09-10T19:06:56.213641+0000","last_became_active":"2024-09-10T19:00:42.515840+0000","last_became_peered":"2024-09-10T19:00:42.515840+0000","last_unstale":"2024-09-10T19:06:56.213641+0000","last_undegraded":"2024-09-10T19:06:56.213641+0000","last_fullsized":"2024-09-10T19:06:56.213641+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:23.759624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:03:23.759624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037605970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":278,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.215252+0000","last_change":"2024-09-10T19:06:11.016240+0000","last_active":"2024-09-10T19:06:56.215252+0000","last_peered":"2024-09-10T19:06:56.215252+0000","last_clean":"2024-09-10T19:06:56.215252+0000","last_became_active":"2024-09-10T18:57:02.440497+0000","last_became_peered":"2024-09-10T18:57:02.440497+0000","last_unstale":"2024-09-10T19:06:56.215252+0000","last_undegraded":"2024-09-10T19:06:56.215252+0000","last_fullsized":"2024-09-10T19:06:56.215252+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:11.016144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:06:11.016144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:29.801511+0000","scrub_duration":0.015619935999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"427'21","reported_seq":534,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-10T19:07:01.088980+0000","last_change":"2024-09-10T19:07:01.088944+0000","last_active":"2024-09-10T19:07:01.088980+0000","last_peered":"2024-09-10T19:07:01.088980+0000","last_clean":"2024-09-10T19:07:01.088980+0000","last_became_active":"2024-09-10T18:55:40.391571+0000","last_became_peered":"2024-09-10T18:55:40.391571+0000","last_unstale":"2024-09-10T19:07:01.088980+0000","last_undegraded":"2024-09-10T19:07:01.088980+0000","last_fullsized":"2024-09-10T19:07:01.088980+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'21","last_scrub_stamp":"2024-09-10T19:07:01.088872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:07:01.088872+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:08:22.821400+0000","scrub_duration":0.0018969589999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":498,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.214751+0000","last_change":"2024-09-10T19:02:55.736359+0000","last_active":"2024-09-10T19:06:56.214751+0000","last_peered":"2024-09-10T19:06:56.214751+0000","last_clean":"2024-09-10T19:06:56.214751+0000","last_became_active":"2024-09-10T18:57:02.431585+0000","last_became_peered":"2024-09-10T18:57:02.431585+0000","last_unstale":"2024-09-10T19:06:56.214751+0000","last_undegraded":"2024-09-10T19:06:56.214751+0000","last_fullsized":"2024-09-10T19:06:56.214751+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:55.736308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:02:55.736308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012293950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":162,"reported_epoch":452,"state":"active+clean","last_fresh":"2024-09-10T19:06:50.837870+0000","last_change":"2024-09-10T19:06:15.668283+0000","last_active":"2024-09-10T19:06:50.837870+0000","last_peered":"2024-09-10T19:06:50.837870+0000","last_clean":"2024-09-10T19:06:50.837870+0000","last_became_active":"2024-09-10T18:59:48.737788+0000","last_became_peered":"2024-09-10T18:59:48.737788+0000","last_unstale":"2024-09-10T19:06:50.837870+0000","last_undegraded":"2024-09-10T19:06:50.837870+0000","last_fullsized":"2024-09-10T19:06:50.837870+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:15.668048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:59.425718+0000","last_clean_scrub_stamp":"2024-09-10T19:06:15.668048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:33.048610+0000","scrub_duration":0.0064859770000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.8","version":"70'81","reported_seq":139,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.213791+0000","last_change":"2024-09-10T19:03:21.755540+0000","last_active":"2024-09-10T19:06:56.213791+0000","last_peered":"2024-09-10T19:06:56.213791+0000","last_clean":"2024-09-10T19:06:56.213791+0000","last_became_active":"2024-09-10T19:00:42.517072+0000","last_became_peered":"2024-09-10T19:00:42.517072+0000","last_unstale":"2024-09-10T19:06:56.213791+0000","last_undegraded":"2024-09-10T19:06:56.213791+0000","last_fullsized":"2024-09-10T19:06:56.213791+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:03:21.755481+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:03:21.755481+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016110499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":262688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"427'53","reported_seq":577,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.063580+0000","last_change":"2024-09-10T19:04:33.387338+0000","last_active":"2024-09-10T19:06:56.063580+0000","last_peered":"2024-09-10T19:06:56.063580+0000","last_clean":"2024-09-10T19:06:56.063580+0000","last_became_active":"2024-09-10T18:47:36.409300+0000","last_became_peered":"2024-09-10T18:47:36.409300+0000","last_unstale":"2024-09-10T19:06:56.063580+0000","last_undegraded":"2024-09-10T19:06:56.063580+0000","last_fullsized":"2024-09-10T19:06:56.063580+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'39","last_scrub_stamp":"2024-09-10T19:04:33.387246+0000","last_deep_scrub":"78'25","last_deep_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_clean_scrub_stamp":"2024-09-10T19:04:33.387246+0000","objects_scrubbed":8,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0056627190000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":71,"num_read_kb":61,"num_write":46,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":215,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.063591+0000","last_change":"2024-09-10T19:04:38.456757+0000","last_active":"2024-09-10T19:06:56.063591+0000","last_peered":"2024-09-10T19:06:56.063591+0000","last_clean":"2024-09-10T19:06:56.063591+0000","last_became_active":"2024-09-10T18:57:02.437260+0000","last_became_peered":"2024-09-10T18:57:02.437260+0000","last_unstale":"2024-09-10T19:06:56.063591+0000","last_undegraded":"2024-09-10T19:06:56.063591+0000","last_fullsized":"2024-09-10T19:06:56.063591+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:38.456644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:04:38.456644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0012890429999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":338,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.066653+0000","last_change":"2024-09-10T19:06:09.867580+0000","last_active":"2024-09-10T19:06:56.066653+0000","last_peered":"2024-09-10T19:06:56.066653+0000","last_clean":"2024-09-10T19:06:56.066653+0000","last_became_active":"2024-09-10T19:02:49.594088+0000","last_became_peered":"2024-09-10T19:02:49.594088+0000","last_unstale":"2024-09-10T19:06:56.066653+0000","last_undegraded":"2024-09-10T19:06:56.066653+0000","last_fullsized":"2024-09-10T19:06:56.066653+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:09.867514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:04.512425+0000","last_clean_scrub_stamp":"2024-09-10T19:06:09.867514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:31.816149+0000","scrub_duration":0.0030367480000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":385,"reported_epoch":452,"state":"active+clean","last_fresh":"2024-09-10T19:06:50.839230+0000","last_change":"2024-09-10T19:06:22.676283+0000","last_active":"2024-09-10T19:06:50.839230+0000","last_peered":"2024-09-10T19:06:50.839230+0000","last_clean":"2024-09-10T19:06:50.839230+0000","last_became_active":"2024-09-10T18:55:40.388154+0000","last_became_peered":"2024-09-10T18:55:40.388154+0000","last_unstale":"2024-09-10T19:06:50.839230+0000","last_undegraded":"2024-09-10T19:06:50.839230+0000","last_fullsized":"2024-09-10T19:06:50.839230+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:22.676194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:06:22.676194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:43.573594+0000","scrub_duration":0.0084613770000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.9","version":"70'81","reported_seq":371,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.320676+0000","last_change":"2024-09-10T19:04:31.516816+0000","last_active":"2024-09-10T19:06:56.320676+0000","last_peered":"2024-09-10T19:06:56.320676+0000","last_clean":"2024-09-10T19:06:56.320676+0000","last_became_active":"2024-09-10T18:57:02.463750+0000","last_became_peered":"2024-09-10T18:57:02.463750+0000","last_unstale":"2024-09-10T19:06:56.320676+0000","last_undegraded":"2024-09-10T19:06:56.320676+0000","last_fullsized":"2024-09-10T19:06:56.320676+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:04:31.516600+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:04:31.516600+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011586271,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"427'35","reported_seq":511,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.066941+0000","last_change":"2024-09-10T19:03:34.532084+0000","last_active":"2024-09-10T19:06:56.066941+0000","last_peered":"2024-09-10T19:06:56.066941+0000","last_clean":"2024-09-10T19:06:56.066941+0000","last_became_active":"2024-09-10T18:45:20.053428+0000","last_became_peered":"2024-09-10T18:45:20.053428+0000","last_unstale":"2024-09-10T19:06:56.066941+0000","last_undegraded":"2024-09-10T19:06:56.066941+0000","last_fullsized":"2024-09-10T19:06:56.066941+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'25","last_scrub_stamp":"2024-09-10T19:03:34.532019+0000","last_deep_scrub":"116'15","last_deep_scrub_stamp":"2024-09-10T18:49:12.374527+0000","last_clean_scrub_stamp":"2024-09-10T19:03:34.532019+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0020774220000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":377,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.317177+0000","last_change":"2024-09-10T19:06:17.129138+0000","last_active":"2024-09-10T19:06:56.317177+0000","last_peered":"2024-09-10T19:06:56.317177+0000","last_clean":"2024-09-10T19:06:56.317177+0000","last_became_active":"2024-09-10T19:02:49.571995+0000","last_became_peered":"2024-09-10T19:02:49.571995+0000","last_unstale":"2024-09-10T19:06:56.317177+0000","last_undegraded":"2024-09-10T19:06:56.317177+0000","last_fullsized":"2024-09-10T19:06:56.317177+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:17.129057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:06:17.129057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:42.800560+0000","scrub_duration":0.0074624499999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"273'13","reported_seq":596,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-10T19:07:01.273666+0000","last_change":"2024-09-10T19:06:23.008122+0000","last_active":"2024-09-10T19:07:01.273666+0000","last_peered":"2024-09-10T19:07:01.273666+0000","last_clean":"2024-09-10T19:07:01.273666+0000","last_became_active":"2024-09-10T18:57:02.448258+0000","last_became_peered":"2024-09-10T18:57:02.448258+0000","last_unstale":"2024-09-10T19:07:01.273666+0000","last_undegraded":"2024-09-10T19:07:01.273666+0000","last_fullsized":"2024-09-10T19:07:01.273666+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'13","last_scrub_stamp":"2024-09-10T19:06:23.008037+0000","last_deep_scrub":"273'13","last_deep_scrub_stamp":"2024-09-10T19:06:23.008037+0000","last_clean_scrub_stamp":"2024-09-10T19:06:23.008037+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:42.109075+0000","scrub_duration":0.013286312,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":507,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.066356+0000","last_change":"2024-09-10T19:06:02.941293+0000","last_active":"2024-09-10T19:06:56.066356+0000","last_peered":"2024-09-10T19:06:56.066356+0000","last_clean":"2024-09-10T19:06:56.066356+0000","last_became_active":"2024-09-10T19:00:42.515523+0000","last_became_peered":"2024-09-10T19:00:42.515523+0000","last_unstale":"2024-09-10T19:06:56.066356+0000","last_undegraded":"2024-09-10T19:06:56.066356+0000","last_fullsized":"2024-09-10T19:06:56.066356+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:02.941202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:32.043666+0000","last_clean_scrub_stamp":"2024-09-10T19:06:02.941202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:07.258125+0000","scrub_duration":0.0062417180000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"70'81","reported_seq":386,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.216095+0000","last_change":"2024-09-10T19:02:23.723122+0000","last_active":"2024-09-10T19:06:56.216095+0000","last_peered":"2024-09-10T19:06:56.216095+0000","last_clean":"2024-09-10T19:06:56.216095+0000","last_became_active":"2024-09-10T18:50:08.398345+0000","last_became_peered":"2024-09-10T18:50:08.398345+0000","last_unstale":"2024-09-10T19:06:56.216095+0000","last_undegraded":"2024-09-10T19:06:56.216095+0000","last_fullsized":"2024-09-10T19:06:56.216095+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:23.723036+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:55:37.921519+0000","last_clean_scrub_stamp":"2024-09-10T19:02:23.723036+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0054938950000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"427'46","reported_seq":511,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.215943+0000","last_change":"2024-09-10T19:04:33.010380+0000","last_active":"2024-09-10T19:06:56.215943+0000","last_peered":"2024-09-10T19:06:56.215943+0000","last_clean":"2024-09-10T19:06:56.215943+0000","last_became_active":"2024-09-10T18:46:56.742946+0000","last_became_peered":"2024-09-10T18:46:56.742946+0000","last_unstale":"2024-09-10T19:06:56.215943+0000","last_undegraded":"2024-09-10T19:06:56.215943+0000","last_fullsized":"2024-09-10T19:06:56.215943+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'34","last_scrub_stamp":"2024-09-10T19:04:33.010321+0000","last_deep_scrub":"141'22","last_deep_scrub_stamp":"2024-09-10T18:50:57.119152+0000","last_clean_scrub_stamp":"2024-09-10T19:04:33.010321+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0031820009999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":380,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.320588+0000","last_change":"2024-09-10T19:04:32.480047+0000","last_active":"2024-09-10T19:06:56.320588+0000","last_peered":"2024-09-10T19:06:56.320588+0000","last_clean":"2024-09-10T19:06:56.320588+0000","last_became_active":"2024-09-10T19:02:49.563535+0000","last_became_peered":"2024-09-10T19:02:49.563535+0000","last_unstale":"2024-09-10T19:06:56.320588+0000","last_undegraded":"2024-09-10T19:06:56.320588+0000","last_fullsized":"2024-09-10T19:06:56.320588+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:32.479945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_clean_scrub_stamp":"2024-09-10T19:04:32.479945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0060807650000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"242'10","reported_seq":747,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-10T19:07:00.640505+0000","last_change":"2024-09-10T19:02:53.030726+0000","last_active":"2024-09-10T19:07:00.640505+0000","last_peered":"2024-09-10T19:07:00.640505+0000","last_clean":"2024-09-10T19:07:00.640505+0000","last_became_active":"2024-09-10T18:55:40.397246+0000","last_became_peered":"2024-09-10T18:55:40.397246+0000","last_unstale":"2024-09-10T19:07:00.640505+0000","last_undegraded":"2024-09-10T19:07:00.640505+0000","last_fullsized":"2024-09-10T19:07:00.640505+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"242'10","last_scrub_stamp":"2024-09-10T19:02:53.030666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:02:53.030666+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015331489999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":486,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.216837+0000","last_change":"2024-09-10T19:02:48.670915+0000","last_active":"2024-09-10T19:06:56.216837+0000","last_peered":"2024-09-10T19:06:56.216837+0000","last_clean":"2024-09-10T19:06:56.216837+0000","last_became_active":"2024-09-10T18:55:40.392003+0000","last_became_peered":"2024-09-10T18:55:40.392003+0000","last_unstale":"2024-09-10T19:06:56.216837+0000","last_undegraded":"2024-09-10T19:06:56.216837+0000","last_fullsized":"2024-09-10T19:06:56.216837+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:48.670862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:54.218602+0000","last_clean_scrub_stamp":"2024-09-10T19:02:48.670862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00108365,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"70'81","reported_seq":179,"reported_epoch":452,"state":"active+clean","last_fresh":"2024-09-10T19:06:50.838254+0000","last_change":"2024-09-10T19:06:11.695308+0000","last_active":"2024-09-10T19:06:50.838254+0000","last_peered":"2024-09-10T19:06:50.838254+0000","last_clean":"2024-09-10T19:06:50.838254+0000","last_became_active":"2024-09-10T19:02:49.572057+0000","last_became_peered":"2024-09-10T19:02:49.572057+0000","last_unstale":"2024-09-10T19:06:50.838254+0000","last_undegraded":"2024-09-10T19:06:50.838254+0000","last_fullsized":"2024-09-10T19:06:50.838254+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:06:11.695225+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T19:02:17.370627+0000","last_clean_scrub_stamp":"2024-09-10T19:06:11.695225+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:24.298666+0000","scrub_duration":0.003975236,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"427'32","reported_seq":585,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.066891+0000","last_change":"2024-09-10T19:03:42.660938+0000","last_active":"2024-09-10T19:06:56.066891+0000","last_peered":"2024-09-10T19:06:56.066891+0000","last_clean":"2024-09-10T19:06:56.066891+0000","last_became_active":"2024-09-10T18:47:57.106935+0000","last_became_peered":"2024-09-10T18:47:57.106935+0000","last_unstale":"2024-09-10T19:06:56.066891+0000","last_undegraded":"2024-09-10T19:06:56.066891+0000","last_fullsized":"2024-09-10T19:06:56.066891+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T19:03:42.660792+0000","last_deep_scrub":"224'16","last_deep_scrub_stamp":"2024-09-10T18:54:49.296957+0000","last_clean_scrub_stamp":"2024-09-10T19:03:42.660792+0000","objects_scrubbed":5,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0053846639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":468,"reported_epoch":452,"state":"active+clean","last_fresh":"2024-09-10T19:06:50.838102+0000","last_change":"2024-09-10T19:06:01.706867+0000","last_active":"2024-09-10T19:06:50.838102+0000","last_peered":"2024-09-10T19:06:50.838102+0000","last_clean":"2024-09-10T19:06:50.838102+0000","last_became_active":"2024-09-10T18:57:02.443816+0000","last_became_peered":"2024-09-10T18:57:02.443816+0000","last_unstale":"2024-09-10T19:06:50.838102+0000","last_undegraded":"2024-09-10T19:06:50.838102+0000","last_fullsized":"2024-09-10T19:06:50.838102+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:01.706783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:06:01.706783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:28.681911+0000","scrub_duration":0.0097313599999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":484,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.215731+0000","last_change":"2024-09-10T19:06:02.997863+0000","last_active":"2024-09-10T19:06:56.215731+0000","last_peered":"2024-09-10T19:06:56.215731+0000","last_clean":"2024-09-10T19:06:56.215731+0000","last_became_active":"2024-09-10T18:45:22.068918+0000","last_became_peered":"2024-09-10T18:45:22.068918+0000","last_unstale":"2024-09-10T19:06:56.215731+0000","last_undegraded":"2024-09-10T19:06:56.215731+0000","last_fullsized":"2024-09-10T19:06:56.215731+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:02.997745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_clean_scrub_stamp":"2024-09-10T19:06:02.997745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:10.273645+0000","scrub_duration":0.0015124940000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":457,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.316070+0000","last_change":"2024-09-10T19:03:24.593809+0000","last_active":"2024-09-10T19:06:56.316070+0000","last_peered":"2024-09-10T19:06:56.316070+0000","last_clean":"2024-09-10T19:06:56.316070+0000","last_became_active":"2024-09-10T18:45:24.264538+0000","last_became_peered":"2024-09-10T18:45:24.264538+0000","last_unstale":"2024-09-10T19:06:56.316070+0000","last_undegraded":"2024-09-10T19:06:56.316070+0000","last_fullsized":"2024-09-10T19:06:56.316070+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:24.593754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:51.398981+0000","last_clean_scrub_stamp":"2024-09-10T19:03:24.593754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001434999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"70'81","reported_seq":415,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.316224+0000","last_change":"2024-09-10T19:03:30.624986+0000","last_active":"2024-09-10T19:06:56.316224+0000","last_peered":"2024-09-10T19:06:56.316224+0000","last_clean":"2024-09-10T19:06:56.316224+0000","last_became_active":"2024-09-10T19:00:42.519742+0000","last_became_peered":"2024-09-10T19:00:42.519742+0000","last_unstale":"2024-09-10T19:06:56.316224+0000","last_undegraded":"2024-09-10T19:06:56.316224+0000","last_fullsized":"2024-09-10T19:06:56.316224+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:03:30.624868+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:03:30.624868+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0036172140000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"427'49","reported_seq":531,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.062834+0000","last_change":"2024-09-10T19:03:29.581426+0000","last_active":"2024-09-10T19:06:56.062834+0000","last_peered":"2024-09-10T19:06:56.062834+0000","last_clean":"2024-09-10T19:06:56.062834+0000","last_became_active":"2024-09-10T18:45:20.052243+0000","last_became_peered":"2024-09-10T18:45:20.052243+0000","last_unstale":"2024-09-10T19:06:56.062834+0000","last_undegraded":"2024-09-10T19:06:56.062834+0000","last_fullsized":"2024-09-10T19:06:56.062834+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'35","last_scrub_stamp":"2024-09-10T19:03:29.581339+0000","last_deep_scrub":"143'21","last_deep_scrub_stamp":"2024-09-10T18:51:02.540216+0000","last_clean_scrub_stamp":"2024-09-10T19:03:29.581339+0000","objects_scrubbed":7,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0042812099999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":63,"num_read_kb":56,"num_write":42,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":506,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.382963+0000","last_change":"2024-09-10T19:04:29.252531+0000","last_active":"2024-09-10T19:06:56.382963+0000","last_peered":"2024-09-10T19:06:56.382963+0000","last_clean":"2024-09-10T19:06:56.382963+0000","last_became_active":"2024-09-10T18:57:02.453404+0000","last_became_peered":"2024-09-10T18:57:02.453404+0000","last_unstale":"2024-09-10T19:06:56.382963+0000","last_undegraded":"2024-09-10T19:06:56.382963+0000","last_fullsized":"2024-09-10T19:06:56.382963+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:29.252456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:25.206731+0000","last_clean_scrub_stamp":"2024-09-10T19:04:29.252456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0036547900000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":738,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-10T19:07:01.035457+0000","last_change":"2024-09-10T19:06:07.959700+0000","last_active":"2024-09-10T19:07:01.035457+0000","last_peered":"2024-09-10T19:07:01.035457+0000","last_clean":"2024-09-10T19:07:01.035457+0000","last_became_active":"2024-09-10T18:45:22.074540+0000","last_became_peered":"2024-09-10T18:45:22.074540+0000","last_unstale":"2024-09-10T19:07:01.035457+0000","last_undegraded":"2024-09-10T19:07:01.035457+0000","last_fullsized":"2024-09-10T19:07:01.035457+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T19:06:07.959488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:06:07.959488+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:21.540597+0000","scrub_duration":0.01085156,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":273,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.383188+0000","last_change":"2024-09-10T19:04:37.416533+0000","last_active":"2024-09-10T19:06:56.383188+0000","last_peered":"2024-09-10T19:06:56.383188+0000","last_clean":"2024-09-10T19:06:56.383188+0000","last_became_active":"2024-09-10T18:57:02.437100+0000","last_became_peered":"2024-09-10T18:57:02.437100+0000","last_unstale":"2024-09-10T19:06:56.383188+0000","last_undegraded":"2024-09-10T19:06:56.383188+0000","last_fullsized":"2024-09-10T19:06:56.383188+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:37.416450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:04:37.416450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0094483369999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"70'81","reported_seq":381,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.067155+0000","last_change":"2024-09-10T19:02:58.888049+0000","last_active":"2024-09-10T19:06:56.067155+0000","last_peered":"2024-09-10T19:06:56.067155+0000","last_clean":"2024-09-10T19:06:56.067155+0000","last_became_active":"2024-09-10T18:52:14.584798+0000","last_became_peered":"2024-09-10T18:52:14.584798+0000","last_unstale":"2024-09-10T19:06:56.067155+0000","last_undegraded":"2024-09-10T19:06:56.067155+0000","last_fullsized":"2024-09-10T19:06:56.067155+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:58.887944+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:56:31.481303+0000","last_clean_scrub_stamp":"2024-09-10T19:02:58.887944+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016190950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"427'42","reported_seq":645,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.320776+0000","last_change":"2024-09-10T19:02:48.361438+0000","last_active":"2024-09-10T19:06:56.320776+0000","last_peered":"2024-09-10T19:06:56.320776+0000","last_clean":"2024-09-10T19:06:56.320776+0000","last_became_active":"2024-09-10T18:59:48.742377+0000","last_became_peered":"2024-09-10T18:59:48.742377+0000","last_unstale":"2024-09-10T19:06:56.320776+0000","last_undegraded":"2024-09-10T19:06:56.320776+0000","last_fullsized":"2024-09-10T19:06:56.320776+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'30","last_scrub_stamp":"2024-09-10T19:02:48.361337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:48.361337+0000","objects_scrubbed":8,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011213163,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":70,"num_read_kb":58,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":114,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.383022+0000","last_change":"2024-09-10T19:06:27.610141+0000","last_active":"2024-09-10T19:06:56.383022+0000","last_peered":"2024-09-10T19:06:56.383022+0000","last_clean":"2024-09-10T19:06:56.383022+0000","last_became_active":"2024-09-10T19:02:49.578870+0000","last_became_peered":"2024-09-10T19:02:49.578870+0000","last_unstale":"2024-09-10T19:06:56.383022+0000","last_undegraded":"2024-09-10T19:06:56.383022+0000","last_fullsized":"2024-09-10T19:06:56.383022+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:27.610058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:46.142129+0000","last_clean_scrub_stamp":"2024-09-10T19:06:27.610058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:36.151562+0000","scrub_duration":0.0042716869999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":479,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.062706+0000","last_change":"2024-09-10T19:04:36.430586+0000","last_active":"2024-09-10T19:06:56.062706+0000","last_peered":"2024-09-10T19:06:56.062706+0000","last_clean":"2024-09-10T19:06:56.062706+0000","last_became_active":"2024-09-10T18:57:02.448126+0000","last_became_peered":"2024-09-10T18:57:02.448126+0000","last_unstale":"2024-09-10T19:06:56.062706+0000","last_undegraded":"2024-09-10T19:06:56.062706+0000","last_fullsized":"2024-09-10T19:06:56.062706+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:36.430526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:57.903311+0000","last_clean_scrub_stamp":"2024-09-10T19:04:36.430526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.00133559,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":467,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.063324+0000","last_change":"2024-09-10T19:03:36.541808+0000","last_active":"2024-09-10T19:06:56.063324+0000","last_peered":"2024-09-10T19:06:56.063324+0000","last_clean":"2024-09-10T19:06:56.063324+0000","last_became_active":"2024-09-10T18:47:36.410836+0000","last_became_peered":"2024-09-10T18:47:36.410836+0000","last_unstale":"2024-09-10T19:06:56.063324+0000","last_undegraded":"2024-09-10T19:06:56.063324+0000","last_fullsized":"2024-09-10T19:06:56.063324+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:36.541729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_clean_scrub_stamp":"2024-09-10T19:03:36.541729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001932762,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"70'81","reported_seq":489,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.382904+0000","last_change":"2024-09-10T19:06:25.532401+0000","last_active":"2024-09-10T19:06:56.382904+0000","last_peered":"2024-09-10T19:06:56.382904+0000","last_clean":"2024-09-10T19:06:56.382904+0000","last_became_active":"2024-09-10T18:55:40.396828+0000","last_became_peered":"2024-09-10T18:55:40.396828+0000","last_unstale":"2024-09-10T19:06:56.382904+0000","last_undegraded":"2024-09-10T19:06:56.382904+0000","last_fullsized":"2024-09-10T19:06:56.382904+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:06:25.532348+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T19:06:25.532348+0000","last_clean_scrub_stamp":"2024-09-10T19:06:25.532348+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:45.622134+0000","scrub_duration":0.0053591250000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"427'39","reported_seq":545,"reported_epoch":452,"state":"active+clean","last_fresh":"2024-09-10T19:06:50.838560+0000","last_change":"2024-09-10T19:06:12.729237+0000","last_active":"2024-09-10T19:06:50.838560+0000","last_peered":"2024-09-10T19:06:50.838560+0000","last_clean":"2024-09-10T19:06:50.838560+0000","last_became_active":"2024-09-10T18:57:02.442862+0000","last_became_peered":"2024-09-10T18:57:02.442862+0000","last_unstale":"2024-09-10T19:06:50.838560+0000","last_undegraded":"2024-09-10T19:06:50.838560+0000","last_fullsized":"2024-09-10T19:06:50.838560+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'39","last_scrub_stamp":"2024-09-10T19:06:12.729127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:06:12.729127+0000","objects_scrubbed":8,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:35.794829+0000","scrub_duration":0.005856895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":68,"num_read_kb":54,"num_write":42,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":468,"reported_epoch":452,"state":"active+clean","last_fresh":"2024-09-10T19:06:50.838501+0000","last_change":"2024-09-10T19:06:29.659585+0000","last_active":"2024-09-10T19:06:50.838501+0000","last_peered":"2024-09-10T19:06:50.838501+0000","last_clean":"2024-09-10T19:06:50.838501+0000","last_became_active":"2024-09-10T18:57:02.443522+0000","last_became_peered":"2024-09-10T18:57:02.443522+0000","last_unstale":"2024-09-10T19:06:50.838501+0000","last_undegraded":"2024-09-10T19:06:50.838501+0000","last_fullsized":"2024-09-10T19:06:50.838501+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:29.659530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:06:29.659530+0000","last_clean_scrub_stamp":"2024-09-10T19:06:29.659530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:49.158272+0000","scrub_duration":0.0029405880000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":502,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.382372+0000","last_change":"2024-09-10T19:06:07.512604+0000","last_active":"2024-09-10T19:06:56.382372+0000","last_peered":"2024-09-10T19:06:56.382372+0000","last_clean":"2024-09-10T19:06:56.382372+0000","last_became_active":"2024-09-10T18:55:40.387857+0000","last_became_peered":"2024-09-10T18:55:40.387857+0000","last_unstale":"2024-09-10T19:06:56.382372+0000","last_undegraded":"2024-09-10T19:06:56.382372+0000","last_fullsized":"2024-09-10T19:06:56.382372+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:07.512453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:06:07.512453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:09.727452+0000","scrub_duration":0.0057807240000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":382,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.384398+0000","last_change":"2024-09-10T19:06:29.568836+0000","last_active":"2024-09-10T19:06:56.384398+0000","last_peered":"2024-09-10T19:06:56.384398+0000","last_clean":"2024-09-10T19:06:56.384398+0000","last_became_active":"2024-09-10T19:02:49.561359+0000","last_became_peered":"2024-09-10T19:02:49.561359+0000","last_unstale":"2024-09-10T19:06:56.384398+0000","last_undegraded":"2024-09-10T19:06:56.384398+0000","last_fullsized":"2024-09-10T19:06:56.384398+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:29.568754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:49:18.281230+0000","last_clean_scrub_stamp":"2024-09-10T19:06:29.568754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:38.060685+0000","scrub_duration":0.002692238,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"70'81","reported_seq":482,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.383424+0000","last_change":"2024-09-10T19:03:34.083497+0000","last_active":"2024-09-10T19:06:56.383424+0000","last_peered":"2024-09-10T19:06:56.383424+0000","last_clean":"2024-09-10T19:06:56.383424+0000","last_became_active":"2024-09-10T18:57:02.452659+0000","last_became_peered":"2024-09-10T18:57:02.452659+0000","last_unstale":"2024-09-10T19:06:56.383424+0000","last_undegraded":"2024-09-10T19:06:56.383424+0000","last_fullsized":"2024-09-10T19:06:56.383424+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:03:34.083013+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:03:34.083013+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023839249999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"427'35","reported_seq":555,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.213212+0000","last_change":"2024-09-10T19:03:52.017990+0000","last_active":"2024-09-10T19:06:56.213212+0000","last_peered":"2024-09-10T19:06:56.213212+0000","last_clean":"2024-09-10T19:06:56.213212+0000","last_became_active":"2024-09-10T19:02:49.579401+0000","last_became_peered":"2024-09-10T19:02:49.579401+0000","last_unstale":"2024-09-10T19:06:56.213212+0000","last_undegraded":"2024-09-10T19:06:56.213212+0000","last_fullsized":"2024-09-10T19:06:56.213212+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'25","last_scrub_stamp":"2024-09-10T19:03:52.017902+0000","last_deep_scrub":"83'15","last_deep_scrub_stamp":"2024-09-10T18:46:58.436473+0000","last_clean_scrub_stamp":"2024-09-10T19:03:52.017902+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024815190000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":394,"reported_epoch":452,"state":"active+clean","last_fresh":"2024-09-10T19:06:50.838480+0000","last_change":"2024-09-10T19:06:02.753654+0000","last_active":"2024-09-10T19:06:50.838480+0000","last_peered":"2024-09-10T19:06:50.838480+0000","last_clean":"2024-09-10T19:06:50.838480+0000","last_became_active":"2024-09-10T19:02:49.561340+0000","last_became_peered":"2024-09-10T19:02:49.561340+0000","last_unstale":"2024-09-10T19:06:50.838480+0000","last_undegraded":"2024-09-10T19:06:50.838480+0000","last_fullsized":"2024-09-10T19:06:50.838480+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:02.753555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:05.140027+0000","last_clean_scrub_stamp":"2024-09-10T19:06:02.753555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:30.070995+0000","scrub_duration":0.0082683540000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"339'13","reported_seq":698,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:55.586629+0000","last_change":"2024-09-10T19:03:29.753145+0000","last_active":"2024-09-10T19:06:55.586629+0000","last_peered":"2024-09-10T19:06:55.586629+0000","last_clean":"2024-09-10T19:06:55.586629+0000","last_became_active":"2024-09-10T19:00:42.520360+0000","last_became_peered":"2024-09-10T19:00:42.520360+0000","last_unstale":"2024-09-10T19:06:55.586629+0000","last_undegraded":"2024-09-10T19:06:55.586629+0000","last_fullsized":"2024-09-10T19:06:55.586629+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"339'13","last_scrub_stamp":"2024-09-10T19:03:29.753059+0000","last_deep_scrub":"258'12","last_deep_scrub_stamp":"2024-09-10T18:56:21.098770+0000","last_clean_scrub_stamp":"2024-09-10T19:03:29.753059+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.002665485,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":454,"reported_epoch":452,"state":"active+clean","last_fresh":"2024-09-10T19:06:50.838283+0000","last_change":"2024-09-10T19:04:39.703670+0000","last_active":"2024-09-10T19:06:50.838283+0000","last_peered":"2024-09-10T19:06:50.838283+0000","last_clean":"2024-09-10T19:06:50.838283+0000","last_became_active":"2024-09-10T19:00:42.515144+0000","last_became_peered":"2024-09-10T19:00:42.515144+0000","last_unstale":"2024-09-10T19:06:50.838283+0000","last_undegraded":"2024-09-10T19:06:50.838283+0000","last_fullsized":"2024-09-10T19:06:50.838283+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:39.703610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:04:39.703610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0022926890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"70'81","reported_seq":413,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.213243+0000","last_change":"2024-09-10T19:04:24.971869+0000","last_active":"2024-09-10T19:06:56.213243+0000","last_peered":"2024-09-10T19:06:56.213243+0000","last_clean":"2024-09-10T19:06:56.213243+0000","last_became_active":"2024-09-10T18:55:40.392432+0000","last_became_peered":"2024-09-10T18:55:40.392432+0000","last_unstale":"2024-09-10T19:06:56.213243+0000","last_undegraded":"2024-09-10T19:06:56.213243+0000","last_fullsized":"2024-09-10T19:06:56.213243+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:04:24.971752+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:04:24.971752+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0034804469999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"427'54","reported_seq":601,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-10T19:07:01.036342+0000","last_change":"2024-09-10T19:02:49.563856+0000","last_active":"2024-09-10T19:07:01.036342+0000","last_peered":"2024-09-10T19:07:01.036342+0000","last_clean":"2024-09-10T19:07:01.036342+0000","last_became_active":"2024-09-10T19:02:49.563402+0000","last_became_peered":"2024-09-10T19:02:49.563402+0000","last_unstale":"2024-09-10T19:07:01.036342+0000","last_undegraded":"2024-09-10T19:07:01.036342+0000","last_fullsized":"2024-09-10T19:07:01.036342+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'41","last_scrub_stamp":"2024-09-10T19:02:12.270610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:12.270610+0000","objects_scrubbed":10,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.012827955,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":83,"num_read_kb":68,"num_write":50,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":488,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.213277+0000","last_change":"2024-09-10T19:06:18.033122+0000","last_active":"2024-09-10T19:06:56.213277+0000","last_peered":"2024-09-10T19:06:56.213277+0000","last_clean":"2024-09-10T19:06:56.213277+0000","last_became_active":"2024-09-10T18:45:17.964494+0000","last_became_peered":"2024-09-10T18:45:17.964494+0000","last_unstale":"2024-09-10T19:06:56.213277+0000","last_undegraded":"2024-09-10T19:06:56.213277+0000","last_fullsized":"2024-09-10T19:06:56.213277+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:18.033032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:12.982599+0000","last_clean_scrub_stamp":"2024-09-10T19:06:18.033032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:38.662714+0000","scrub_duration":0.0025216739999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":430,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.383740+0000","last_change":"2024-09-10T19:04:32.317459+0000","last_active":"2024-09-10T19:06:56.383740+0000","last_peered":"2024-09-10T19:06:56.383740+0000","last_clean":"2024-09-10T19:06:56.383740+0000","last_became_active":"2024-09-10T18:47:57.107478+0000","last_became_peered":"2024-09-10T18:47:57.107478+0000","last_unstale":"2024-09-10T19:06:56.383740+0000","last_undegraded":"2024-09-10T19:06:56.383740+0000","last_fullsized":"2024-09-10T19:06:56.383740+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:32.317339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:04:32.317339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0077591379999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":457,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.315631+0000","last_change":"2024-09-10T19:02:09.267386+0000","last_active":"2024-09-10T19:06:56.315631+0000","last_peered":"2024-09-10T19:06:56.315631+0000","last_clean":"2024-09-10T19:06:56.315631+0000","last_became_active":"2024-09-10T18:57:02.460594+0000","last_became_peered":"2024-09-10T18:57:02.460594+0000","last_unstale":"2024-09-10T19:06:56.315631+0000","last_undegraded":"2024-09-10T19:06:56.315631+0000","last_fullsized":"2024-09-10T19:06:56.315631+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:09.267253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:22.891084+0000","last_clean_scrub_stamp":"2024-09-10T19:02:09.267253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0067979290000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"70'81","reported_seq":290,"reported_epoch":452,"state":"active+clean","last_fresh":"2024-09-10T19:06:50.838759+0000","last_change":"2024-09-10T19:06:16.693268+0000","last_active":"2024-09-10T19:06:50.838759+0000","last_peered":"2024-09-10T19:06:50.838759+0000","last_clean":"2024-09-10T19:06:50.838759+0000","last_became_active":"2024-09-10T18:57:02.437944+0000","last_became_peered":"2024-09-10T18:57:02.437944+0000","last_unstale":"2024-09-10T19:06:50.838759+0000","last_undegraded":"2024-09-10T19:06:50.838759+0000","last_fullsized":"2024-09-10T19:06:50.838759+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:06:16.692486+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:06:16.692486+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:31.831517+0000","scrub_duration":0.0077222740000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"427'26","reported_seq":365,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.215600+0000","last_change":"2024-09-10T19:03:43.899377+0000","last_active":"2024-09-10T19:06:56.215600+0000","last_peered":"2024-09-10T19:06:56.215600+0000","last_clean":"2024-09-10T19:06:56.215600+0000","last_became_active":"2024-09-10T19:02:49.572746+0000","last_became_peered":"2024-09-10T19:02:49.572746+0000","last_unstale":"2024-09-10T19:06:56.215600+0000","last_undegraded":"2024-09-10T19:06:56.215600+0000","last_fullsized":"2024-09-10T19:06:56.215600+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'22","last_scrub_stamp":"2024-09-10T19:03:43.899294+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-10T18:47:10.514951+0000","last_clean_scrub_stamp":"2024-09-10T19:03:43.899294+0000","objects_scrubbed":5,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0049335940000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":41,"num_read_kb":30,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"84'6","reported_seq":522,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:07:00.970208+0000","last_change":"2024-09-10T19:03:43.134996+0000","last_active":"2024-09-10T19:07:00.970208+0000","last_peered":"2024-09-10T19:07:00.970208+0000","last_clean":"2024-09-10T19:07:00.970208+0000","last_became_active":"2024-09-10T19:02:49.570177+0000","last_became_peered":"2024-09-10T19:02:49.570177+0000","last_unstale":"2024-09-10T19:07:00.970208+0000","last_undegraded":"2024-09-10T19:07:00.970208+0000","last_fullsized":"2024-09-10T19:07:00.970208+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'6","last_scrub_stamp":"2024-09-10T19:03:43.134912+0000","last_deep_scrub":"84'6","last_deep_scrub_stamp":"2024-09-10T18:47:22.545450+0000","last_clean_scrub_stamp":"2024-09-10T19:03:43.134912+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024059210000000001,"objects_trimmed":0,"snaptrim_duration":0.0021933159999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":364,"num_read_kb":359,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":441,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.320300+0000","last_change":"2024-09-10T19:04:29.468182+0000","last_active":"2024-09-10T19:06:56.320300+0000","last_peered":"2024-09-10T19:06:56.320300+0000","last_clean":"2024-09-10T19:06:56.320300+0000","last_became_active":"2024-09-10T18:55:40.393389+0000","last_became_peered":"2024-09-10T18:55:40.393389+0000","last_unstale":"2024-09-10T19:06:56.320300+0000","last_undegraded":"2024-09-10T19:06:56.320300+0000","last_fullsized":"2024-09-10T19:06:56.320300+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:29.468090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:04:29.468090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0099267039999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":459,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.214294+0000","last_change":"2024-09-10T19:06:09.998326+0000","last_active":"2024-09-10T19:06:56.214294+0000","last_peered":"2024-09-10T19:06:56.214294+0000","last_clean":"2024-09-10T19:06:56.214294+0000","last_became_active":"2024-09-10T18:57:02.438945+0000","last_became_peered":"2024-09-10T18:57:02.438945+0000","last_unstale":"2024-09-10T19:06:56.214294+0000","last_undegraded":"2024-09-10T19:06:56.214294+0000","last_fullsized":"2024-09-10T19:06:56.214294+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:09.998251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:06:09.998251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:23.808265+0000","scrub_duration":0.0039357020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":216,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.384192+0000","last_change":"2024-09-10T19:03:42.107868+0000","last_active":"2024-09-10T19:06:56.384192+0000","last_peered":"2024-09-10T19:06:56.384192+0000","last_clean":"2024-09-10T19:06:56.384192+0000","last_became_active":"2024-09-10T18:57:02.453865+0000","last_became_peered":"2024-09-10T18:57:02.453865+0000","last_unstale":"2024-09-10T19:06:56.384192+0000","last_undegraded":"2024-09-10T19:06:56.384192+0000","last_fullsized":"2024-09-10T19:06:56.384192+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:42.107785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:07.150480+0000","last_clean_scrub_stamp":"2024-09-10T19:03:42.107785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0029595450000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"70'81","reported_seq":169,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.383070+0000","last_change":"2024-09-10T19:04:28.278727+0000","last_active":"2024-09-10T19:06:56.383070+0000","last_peered":"2024-09-10T19:06:56.383070+0000","last_clean":"2024-09-10T19:06:56.383070+0000","last_became_active":"2024-09-10T19:02:49.578413+0000","last_became_peered":"2024-09-10T19:02:49.578413+0000","last_unstale":"2024-09-10T19:06:56.383070+0000","last_undegraded":"2024-09-10T19:06:56.383070+0000","last_fullsized":"2024-09-10T19:06:56.383070+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:04:28.278564+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:04:28.278564+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0066667519999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"427'36","reported_seq":218,"reported_epoch":452,"state":"active+clean","last_fresh":"2024-09-10T19:06:50.839297+0000","last_change":"2024-09-10T19:03:40.879526+0000","last_active":"2024-09-10T19:06:50.839297+0000","last_peered":"2024-09-10T19:06:50.839297+0000","last_clean":"2024-09-10T19:06:50.839297+0000","last_became_active":"2024-09-10T19:00:42.516521+0000","last_became_peered":"2024-09-10T19:00:42.516521+0000","last_unstale":"2024-09-10T19:06:50.839297+0000","last_undegraded":"2024-09-10T19:06:50.839297+0000","last_fullsized":"2024-09-10T19:06:50.839297+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T19:03:40.879436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:03:40.879436+0000","objects_scrubbed":6,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0094657419999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":47,"num_write":36,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"305'2","reported_seq":447,"reported_epoch":452,"state":"active+clean","last_fresh":"2024-09-10T19:06:50.838726+0000","last_change":"2024-09-10T19:03:21.921314+0000","last_active":"2024-09-10T19:06:50.838726+0000","last_peered":"2024-09-10T19:06:50.838726+0000","last_clean":"2024-09-10T19:06:50.838726+0000","last_became_active":"2024-09-10T18:55:40.390022+0000","last_became_peered":"2024-09-10T18:55:40.390022+0000","last_unstale":"2024-09-10T19:06:50.838726+0000","last_undegraded":"2024-09-10T19:06:50.838726+0000","last_fullsized":"2024-09-10T19:06:50.838726+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"305'2","last_scrub_stamp":"2024-09-10T19:03:21.921254+0000","last_deep_scrub":"305'2","last_deep_scrub_stamp":"2024-09-10T18:59:18.897059+0000","last_clean_scrub_stamp":"2024-09-10T19:03:21.921254+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0021302700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":308,"reported_epoch":452,"state":"active+clean","last_fresh":"2024-09-10T19:06:50.837993+0000","last_change":"2024-09-10T19:06:00.735401+0000","last_active":"2024-09-10T19:06:50.837993+0000","last_peered":"2024-09-10T19:06:50.837993+0000","last_clean":"2024-09-10T19:06:50.837993+0000","last_became_active":"2024-09-10T19:02:49.560325+0000","last_became_peered":"2024-09-10T19:02:49.560325+0000","last_unstale":"2024-09-10T19:06:50.837993+0000","last_undegraded":"2024-09-10T19:06:50.837993+0000","last_fullsized":"2024-09-10T19:06:50.837993+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:00.735322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:06:00.735322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:08.617305+0000","scrub_duration":0.0022871300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":135,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.216109+0000","last_change":"2024-09-10T19:04:30.022221+0000","last_active":"2024-09-10T19:06:56.216109+0000","last_peered":"2024-09-10T19:06:56.216109+0000","last_clean":"2024-09-10T19:06:56.216109+0000","last_became_active":"2024-09-10T19:00:42.518566+0000","last_became_peered":"2024-09-10T19:00:42.518566+0000","last_unstale":"2024-09-10T19:06:56.216109+0000","last_undegraded":"2024-09-10T19:06:56.216109+0000","last_fullsized":"2024-09-10T19:06:56.216109+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:30.022167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:04:30.022167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001515976,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":94,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.215058+0000","last_change":"2024-09-10T19:02:49.573172+0000","last_active":"2024-09-10T19:06:56.215058+0000","last_peered":"2024-09-10T19:06:56.215058+0000","last_clean":"2024-09-10T19:06:56.215058+0000","last_became_active":"2024-09-10T19:02:49.572858+0000","last_became_peered":"2024-09-10T19:02:49.572858+0000","last_unstale":"2024-09-10T19:06:56.215058+0000","last_undegraded":"2024-09-10T19:06:56.215058+0000","last_fullsized":"2024-09-10T19:06:56.215058+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:20.272756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:02:20.272756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"427'45","reported_seq":588,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.384476+0000","last_change":"2024-09-10T19:06:01.618690+0000","last_active":"2024-09-10T19:06:56.384476+0000","last_peered":"2024-09-10T19:06:56.384476+0000","last_clean":"2024-09-10T19:06:56.384476+0000","last_became_active":"2024-09-10T18:57:02.435499+0000","last_became_peered":"2024-09-10T18:57:02.435499+0000","last_unstale":"2024-09-10T19:06:56.384476+0000","last_undegraded":"2024-09-10T19:06:56.384476+0000","last_fullsized":"2024-09-10T19:06:56.384476+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'45","last_scrub_stamp":"2024-09-10T19:06:01.618608+0000","last_deep_scrub":"232'33","last_deep_scrub_stamp":"2024-09-10T18:56:21.206960+0000","last_clean_scrub_stamp":"2024-09-10T19:06:01.618608+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:06.102994+0000","scrub_duration":0.008985346,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":61,"num_read_kb":51,"num_write":37,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"60'2","reported_seq":476,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.319938+0000","last_change":"2024-09-10T19:03:22.545399+0000","last_active":"2024-09-10T19:06:56.319938+0000","last_peered":"2024-09-10T19:06:56.319938+0000","last_clean":"2024-09-10T19:06:56.319938+0000","last_became_active":"2024-09-10T18:45:17.961660+0000","last_became_peered":"2024-09-10T18:45:17.961660+0000","last_unstale":"2024-09-10T19:06:56.319938+0000","last_undegraded":"2024-09-10T19:06:56.319938+0000","last_fullsized":"2024-09-10T19:06:56.319938+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-10T19:03:22.545342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:03:22.545342+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001448983,"objects_trimmed":0,"snaptrim_duration":0.0013942640000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":505,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.214982+0000","last_change":"2024-09-10T19:04:36.102267+0000","last_active":"2024-09-10T19:06:56.214982+0000","last_peered":"2024-09-10T19:06:56.214982+0000","last_clean":"2024-09-10T19:06:56.214982+0000","last_became_active":"2024-09-10T18:57:02.445712+0000","last_became_peered":"2024-09-10T18:57:02.445712+0000","last_unstale":"2024-09-10T19:06:56.214982+0000","last_undegraded":"2024-09-10T19:06:56.214982+0000","last_fullsized":"2024-09-10T19:06:56.214982+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:36.102203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:04:36.102203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.095986274999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'81","reported_seq":290,"reported_epoch":452,"state":"active+clean","last_fresh":"2024-09-10T19:06:50.837884+0000","last_change":"2024-09-10T19:06:24.713689+0000","last_active":"2024-09-10T19:06:50.837884+0000","last_peered":"2024-09-10T19:06:50.837884+0000","last_clean":"2024-09-10T19:06:50.837884+0000","last_became_active":"2024-09-10T18:57:02.440063+0000","last_became_peered":"2024-09-10T18:57:02.440063+0000","last_unstale":"2024-09-10T19:06:50.837884+0000","last_undegraded":"2024-09-10T19:06:50.837884+0000","last_fullsized":"2024-09-10T19:06:50.837884+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:06:24.713626+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:56.085580+0000","last_clean_scrub_stamp":"2024-09-10T19:06:24.713626+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:40.045571+0000","scrub_duration":0.0013738730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":474,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.215329+0000","last_change":"2024-09-10T19:04:33.972472+0000","last_active":"2024-09-10T19:06:56.215329+0000","last_peered":"2024-09-10T19:06:56.215329+0000","last_clean":"2024-09-10T19:06:56.215329+0000","last_became_active":"2024-09-10T18:57:02.431556+0000","last_became_peered":"2024-09-10T18:57:02.431556+0000","last_unstale":"2024-09-10T19:06:56.215329+0000","last_undegraded":"2024-09-10T19:06:56.215329+0000","last_fullsized":"2024-09-10T19:06:56.215329+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:33.972423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:04:33.972423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0074396560000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":479,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.215358+0000","last_change":"2024-09-10T19:03:51.037606+0000","last_active":"2024-09-10T19:06:56.215358+0000","last_peered":"2024-09-10T19:06:56.215358+0000","last_clean":"2024-09-10T19:06:56.215358+0000","last_became_active":"2024-09-10T18:45:24.073327+0000","last_became_peered":"2024-09-10T18:45:24.073327+0000","last_unstale":"2024-09-10T19:06:56.215358+0000","last_undegraded":"2024-09-10T19:06:56.215358+0000","last_fullsized":"2024-09-10T19:06:56.215358+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:51.037432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:03:51.037432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0085107059999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.f","version":"70'81","reported_seq":484,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.383113+0000","last_change":"2024-09-10T19:06:21.520670+0000","last_active":"2024-09-10T19:06:56.383113+0000","last_peered":"2024-09-10T19:06:56.383113+0000","last_clean":"2024-09-10T19:06:56.383113+0000","last_became_active":"2024-09-10T18:52:16.598295+0000","last_became_peered":"2024-09-10T18:52:16.598295+0000","last_unstale":"2024-09-10T19:06:56.383113+0000","last_undegraded":"2024-09-10T19:06:56.383113+0000","last_fullsized":"2024-09-10T19:06:56.383113+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:06:21.520615+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T19:06:21.520615+0000","last_clean_scrub_stamp":"2024-09-10T19:06:21.520615+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:33.819484+0000","scrub_duration":0.0011289640000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"427'51","reported_seq":596,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.382867+0000","last_change":"2024-09-10T19:03:46.180416+0000","last_active":"2024-09-10T19:06:56.382867+0000","last_peered":"2024-09-10T19:06:56.382867+0000","last_clean":"2024-09-10T19:06:56.382867+0000","last_became_active":"2024-09-10T18:57:02.436059+0000","last_became_peered":"2024-09-10T18:57:02.436059+0000","last_unstale":"2024-09-10T19:06:56.382867+0000","last_undegraded":"2024-09-10T19:06:56.382867+0000","last_fullsized":"2024-09-10T19:06:56.382867+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'35","last_scrub_stamp":"2024-09-10T19:03:46.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:03:46.179242+0000","objects_scrubbed":9,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0052365550000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":80,"num_read_kb":69,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":453,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.215402+0000","last_change":"2024-09-10T19:06:02.034786+0000","last_active":"2024-09-10T19:06:56.215402+0000","last_peered":"2024-09-10T19:06:56.215402+0000","last_clean":"2024-09-10T19:06:56.215402+0000","last_became_active":"2024-09-10T18:55:40.391849+0000","last_became_peered":"2024-09-10T18:55:40.391849+0000","last_unstale":"2024-09-10T19:06:56.215402+0000","last_undegraded":"2024-09-10T19:06:56.215402+0000","last_fullsized":"2024-09-10T19:06:56.215402+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:02.034673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:06:02.034673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:21.764260+0000","scrub_duration":0.0021020129999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":423,"reported_epoch":452,"state":"active+clean","last_fresh":"2024-09-10T19:06:50.838164+0000","last_change":"2024-09-10T19:03:48.853831+0000","last_active":"2024-09-10T19:06:50.838164+0000","last_peered":"2024-09-10T19:06:50.838164+0000","last_clean":"2024-09-10T19:06:50.838164+0000","last_became_active":"2024-09-10T19:00:42.514709+0000","last_became_peered":"2024-09-10T19:00:42.514709+0000","last_unstale":"2024-09-10T19:06:50.838164+0000","last_undegraded":"2024-09-10T19:06:50.838164+0000","last_fullsized":"2024-09-10T19:06:50.838164+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:48.853752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:03:48.853752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0059332969999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":260,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.320532+0000","last_change":"2024-09-10T19:02:58.491396+0000","last_active":"2024-09-10T19:06:56.320532+0000","last_peered":"2024-09-10T19:06:56.320532+0000","last_clean":"2024-09-10T19:06:56.320532+0000","last_became_active":"2024-09-10T18:57:02.468866+0000","last_became_peered":"2024-09-10T18:57:02.468866+0000","last_unstale":"2024-09-10T19:06:56.320532+0000","last_undegraded":"2024-09-10T19:06:56.320532+0000","last_fullsized":"2024-09-10T19:06:56.320532+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:58.491326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:02:58.491326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0017883440000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.e","version":"70'81","reported_seq":342,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.063663+0000","last_change":"2024-09-10T19:04:31.431314+0000","last_active":"2024-09-10T19:06:56.063663+0000","last_peered":"2024-09-10T19:06:56.063663+0000","last_clean":"2024-09-10T19:06:56.063663+0000","last_became_active":"2024-09-10T18:57:02.449115+0000","last_became_peered":"2024-09-10T18:57:02.449115+0000","last_unstale":"2024-09-10T19:06:56.063663+0000","last_undegraded":"2024-09-10T19:06:56.063663+0000","last_fullsized":"2024-09-10T19:06:56.063663+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:04:31.431216+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:04:31.431216+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.011858260000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"427'25","reported_seq":488,"reported_epoch":452,"state":"active+clean","last_fresh":"2024-09-10T19:06:50.838735+0000","last_change":"2024-09-10T19:03:43.850751+0000","last_active":"2024-09-10T19:06:50.838735+0000","last_peered":"2024-09-10T19:06:50.838735+0000","last_clean":"2024-09-10T19:06:50.838735+0000","last_became_active":"2024-09-10T18:55:40.395257+0000","last_became_peered":"2024-09-10T18:55:40.395257+0000","last_unstale":"2024-09-10T19:06:50.838735+0000","last_undegraded":"2024-09-10T19:06:50.838735+0000","last_fullsized":"2024-09-10T19:06:50.838735+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T19:03:43.850670+0000","last_deep_scrub":"130'13","last_deep_scrub_stamp":"2024-09-10T18:51:58.427534+0000","last_clean_scrub_stamp":"2024-09-10T19:03:43.850670+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0026924499999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":104,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.063412+0000","last_change":"2024-09-10T19:03:51.598347+0000","last_active":"2024-09-10T19:06:56.063412+0000","last_peered":"2024-09-10T19:06:56.063412+0000","last_clean":"2024-09-10T19:06:56.063412+0000","last_became_active":"2024-09-10T19:02:49.581709+0000","last_became_peered":"2024-09-10T19:02:49.581709+0000","last_unstale":"2024-09-10T19:06:56.063412+0000","last_undegraded":"2024-09-10T19:06:56.063412+0000","last_fullsized":"2024-09-10T19:06:56.063412+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:51.598266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:03:51.598266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0047300789999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":434,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.063048+0000","last_change":"2024-09-10T19:02:51.031764+0000","last_active":"2024-09-10T19:06:56.063048+0000","last_peered":"2024-09-10T19:06:56.063048+0000","last_clean":"2024-09-10T19:06:56.063048+0000","last_became_active":"2024-09-10T18:55:40.390771+0000","last_became_peered":"2024-09-10T18:55:40.390771+0000","last_unstale":"2024-09-10T19:06:56.063048+0000","last_undegraded":"2024-09-10T19:06:56.063048+0000","last_fullsized":"2024-09-10T19:06:56.063048+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:51.031711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_clean_scrub_stamp":"2024-09-10T19:02:51.031711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0015681510000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":467,"reported_epoch":452,"state":"active+clean","last_fresh":"2024-09-10T19:06:50.838215+0000","last_change":"2024-09-10T19:04:28.767375+0000","last_active":"2024-09-10T19:06:50.838215+0000","last_peered":"2024-09-10T19:06:50.838215+0000","last_clean":"2024-09-10T19:06:50.838215+0000","last_became_active":"2024-09-10T18:57:02.452387+0000","last_became_peered":"2024-09-10T18:57:02.452387+0000","last_unstale":"2024-09-10T19:06:50.838215+0000","last_undegraded":"2024-09-10T19:06:50.838215+0000","last_fullsized":"2024-09-10T19:06:50.838215+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:28.766947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:53.345184+0000","last_clean_scrub_stamp":"2024-09-10T19:04:28.766947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0079684749999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.d","version":"70'81","reported_seq":273,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.063595+0000","last_change":"2024-09-10T19:02:01.044879+0000","last_active":"2024-09-10T19:06:56.063595+0000","last_peered":"2024-09-10T19:06:56.063595+0000","last_clean":"2024-09-10T19:06:56.063595+0000","last_became_active":"2024-09-10T18:57:02.439652+0000","last_became_peered":"2024-09-10T18:57:02.439652+0000","last_unstale":"2024-09-10T19:06:56.063595+0000","last_undegraded":"2024-09-10T19:06:56.063595+0000","last_fullsized":"2024-09-10T19:06:56.063595+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:01.044143+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:02:01.044143+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.014088588000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"427'46","reported_seq":176,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.213932+0000","last_change":"2024-09-10T19:02:18.712390+0000","last_active":"2024-09-10T19:06:56.213932+0000","last_peered":"2024-09-10T19:06:56.213932+0000","last_clean":"2024-09-10T19:06:56.213932+0000","last_became_active":"2024-09-10T19:00:42.515903+0000","last_became_peered":"2024-09-10T19:00:42.515903+0000","last_unstale":"2024-09-10T19:06:56.213932+0000","last_undegraded":"2024-09-10T19:06:56.213932+0000","last_fullsized":"2024-09-10T19:06:56.213932+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"365'34","last_scrub_stamp":"2024-09-10T19:02:18.712306+0000","last_deep_scrub":"365'34","last_deep_scrub_stamp":"2024-09-10T19:02:18.712306+0000","last_clean_scrub_stamp":"2024-09-10T19:02:18.712306+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.017099106999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":60,"num_read_kb":51,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":434,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.315751+0000","last_change":"2024-09-10T19:06:23.202241+0000","last_active":"2024-09-10T19:06:56.315751+0000","last_peered":"2024-09-10T19:06:56.315751+0000","last_clean":"2024-09-10T19:06:56.315751+0000","last_became_active":"2024-09-10T18:57:02.461708+0000","last_became_peered":"2024-09-10T18:57:02.461708+0000","last_unstale":"2024-09-10T19:06:56.315751+0000","last_undegraded":"2024-09-10T19:06:56.315751+0000","last_fullsized":"2024-09-10T19:06:56.315751+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:23.202164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:05.603125+0000","last_clean_scrub_stamp":"2024-09-10T19:06:23.202164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:30.512580+0000","scrub_duration":0.012238550000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"273'10","reported_seq":418,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-10T19:07:01.297299+0000","last_change":"2024-09-10T19:06:28.588050+0000","last_active":"2024-09-10T19:07:01.297299+0000","last_peered":"2024-09-10T19:07:01.297299+0000","last_clean":"2024-09-10T19:07:01.297299+0000","last_became_active":"2024-09-10T18:57:02.441890+0000","last_became_peered":"2024-09-10T18:57:02.441890+0000","last_unstale":"2024-09-10T19:07:01.297299+0000","last_undegraded":"2024-09-10T19:07:01.297299+0000","last_fullsized":"2024-09-10T19:07:01.297299+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'10","last_scrub_stamp":"2024-09-10T19:06:28.587892+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-09-10T18:47:02.582229+0000","last_clean_scrub_stamp":"2024-09-10T19:06:28.587892+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:35.607577+0000","scrub_duration":0.0043958399999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":486,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.215982+0000","last_change":"2024-09-10T19:03:32.808204+0000","last_active":"2024-09-10T19:06:56.215982+0000","last_peered":"2024-09-10T19:06:56.215982+0000","last_clean":"2024-09-10T19:06:56.215982+0000","last_became_active":"2024-09-10T18:55:40.389202+0000","last_became_peered":"2024-09-10T18:55:40.389202+0000","last_unstale":"2024-09-10T19:06:56.215982+0000","last_undegraded":"2024-09-10T19:06:56.215982+0000","last_fullsized":"2024-09-10T19:06:56.215982+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:32.808130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_clean_scrub_stamp":"2024-09-10T19:03:32.808130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003767603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.c","version":"70'81","reported_seq":417,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.315940+0000","last_change":"2024-09-10T19:02:54.540617+0000","last_active":"2024-09-10T19:06:56.315940+0000","last_peered":"2024-09-10T19:06:56.315940+0000","last_clean":"2024-09-10T19:06:56.315940+0000","last_became_active":"2024-09-10T18:55:40.390655+0000","last_became_peered":"2024-09-10T18:55:40.390655+0000","last_unstale":"2024-09-10T19:06:56.315940+0000","last_undegraded":"2024-09-10T19:06:56.315940+0000","last_fullsized":"2024-09-10T19:06:56.315940+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:02:54.540558+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:56:33.779154+0000","last_clean_scrub_stamp":"2024-09-10T19:02:54.540558+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019874599999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"427'24","reported_seq":515,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.062843+0000","last_change":"2024-09-10T19:06:12.856434+0000","last_active":"2024-09-10T19:06:56.062843+0000","last_peered":"2024-09-10T19:06:56.062843+0000","last_clean":"2024-09-10T19:06:56.062843+0000","last_became_active":"2024-09-10T18:45:20.056869+0000","last_became_peered":"2024-09-10T18:45:20.056869+0000","last_unstale":"2024-09-10T19:06:56.062843+0000","last_undegraded":"2024-09-10T19:06:56.062843+0000","last_fullsized":"2024-09-10T19:06:56.062843+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'24","last_scrub_stamp":"2024-09-10T19:06:12.856360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:06:12.856360+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:23.852248+0000","scrub_duration":0.0042329550000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"56'2","reported_seq":516,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.066817+0000","last_change":"2024-09-10T19:06:14.852777+0000","last_active":"2024-09-10T19:06:56.066817+0000","last_peered":"2024-09-10T19:06:56.066817+0000","last_clean":"2024-09-10T19:06:56.066817+0000","last_became_active":"2024-09-10T18:57:02.434475+0000","last_became_peered":"2024-09-10T18:57:02.434475+0000","last_unstale":"2024-09-10T19:06:56.066817+0000","last_undegraded":"2024-09-10T19:06:56.066817+0000","last_fullsized":"2024-09-10T19:06:56.066817+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-10T19:06:14.852698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:06:14.852698+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:43.077218+0000","scrub_duration":0.006938662,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":433,"reported_epoch":452,"state":"active+clean","last_fresh":"2024-09-10T19:06:50.838190+0000","last_change":"2024-09-10T19:06:31.634746+0000","last_active":"2024-09-10T19:06:50.838190+0000","last_peered":"2024-09-10T19:06:50.838190+0000","last_clean":"2024-09-10T19:06:50.838190+0000","last_became_active":"2024-09-10T19:00:42.513796+0000","last_became_peered":"2024-09-10T19:00:42.513796+0000","last_unstale":"2024-09-10T19:06:50.838190+0000","last_undegraded":"2024-09-10T19:06:50.838190+0000","last_fullsized":"2024-09-10T19:06:50.838190+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:31.634653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:02.711594+0000","last_clean_scrub_stamp":"2024-09-10T19:06:31.634653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:33.241478+0000","scrub_duration":0.0052320470000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":467,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.316457+0000","last_change":"2024-09-10T19:03:46.711248+0000","last_active":"2024-09-10T19:06:56.316457+0000","last_peered":"2024-09-10T19:06:56.316457+0000","last_clean":"2024-09-10T19:06:56.316457+0000","last_became_active":"2024-09-10T18:57:02.459985+0000","last_became_peered":"2024-09-10T18:57:02.459985+0000","last_unstale":"2024-09-10T19:06:56.316457+0000","last_undegraded":"2024-09-10T19:06:56.316457+0000","last_fullsized":"2024-09-10T19:06:56.316457+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:46.710933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:46.710933+0000","last_clean_scrub_stamp":"2024-09-10T19:03:46.710933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0061568430000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.13","version":"70'81","reported_seq":173,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.316661+0000","last_change":"2024-09-10T19:04:36.470253+0000","last_active":"2024-09-10T19:06:56.316661+0000","last_peered":"2024-09-10T19:06:56.316661+0000","last_clean":"2024-09-10T19:06:56.316661+0000","last_became_active":"2024-09-10T19:02:49.580656+0000","last_became_peered":"2024-09-10T19:02:49.580656+0000","last_unstale":"2024-09-10T19:06:56.316661+0000","last_undegraded":"2024-09-10T19:06:56.316661+0000","last_fullsized":"2024-09-10T19:06:56.316661+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:04:36.470196+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:04:36.470196+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001517404,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"427'25","reported_seq":511,"reported_epoch":452,"state":"active+clean","last_fresh":"2024-09-10T19:06:50.839461+0000","last_change":"2024-09-10T19:04:35.830252+0000","last_active":"2024-09-10T19:06:50.839461+0000","last_peered":"2024-09-10T19:06:50.839461+0000","last_clean":"2024-09-10T19:06:50.839461+0000","last_became_active":"2024-09-10T18:57:02.453001+0000","last_became_peered":"2024-09-10T18:57:02.453001+0000","last_unstale":"2024-09-10T19:06:50.839461+0000","last_undegraded":"2024-09-10T19:06:50.839461+0000","last_fullsized":"2024-09-10T19:06:50.839461+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T19:04:35.830196+0000","last_deep_scrub":"76'13","last_deep_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_clean_scrub_stamp":"2024-09-10T19:04:35.830196+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001824537,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":306,"reported_epoch":452,"state":"active+clean","last_fresh":"2024-09-10T19:06:50.838010+0000","last_change":"2024-09-10T19:03:35.858172+0000","last_active":"2024-09-10T19:06:50.838010+0000","last_peered":"2024-09-10T19:06:50.838010+0000","last_clean":"2024-09-10T19:06:50.838010+0000","last_became_active":"2024-09-10T19:02:49.580701+0000","last_became_peered":"2024-09-10T19:02:49.580701+0000","last_unstale":"2024-09-10T19:06:50.838010+0000","last_undegraded":"2024-09-10T19:06:50.838010+0000","last_fullsized":"2024-09-10T19:06:50.838010+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:35.858086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:03:35.858086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0061699850000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":479,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.315997+0000","last_change":"2024-09-10T19:06:30.141526+0000","last_active":"2024-09-10T19:06:56.315997+0000","last_peered":"2024-09-10T19:06:56.315997+0000","last_clean":"2024-09-10T19:06:56.315997+0000","last_became_active":"2024-09-10T18:57:02.455341+0000","last_became_peered":"2024-09-10T18:57:02.455341+0000","last_unstale":"2024-09-10T19:06:56.315997+0000","last_undegraded":"2024-09-10T19:06:56.315997+0000","last_fullsized":"2024-09-10T19:06:56.315997+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:30.141437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:00.754912+0000","last_clean_scrub_stamp":"2024-09-10T19:06:30.141437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:33.141155+0000","scrub_duration":0.0090402390000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"364'2","reported_seq":493,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.066536+0000","last_change":"2024-09-10T19:02:15.059462+0000","last_active":"2024-09-10T19:06:56.066536+0000","last_peered":"2024-09-10T19:06:56.066536+0000","last_clean":"2024-09-10T19:06:56.066536+0000","last_became_active":"2024-09-10T18:59:48.741275+0000","last_became_peered":"2024-09-10T18:59:48.741275+0000","last_unstale":"2024-09-10T19:06:56.066536+0000","last_undegraded":"2024-09-10T19:06:56.066536+0000","last_fullsized":"2024-09-10T19:06:56.066536+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"364'2","last_scrub_stamp":"2024-09-10T19:02:15.059320+0000","last_deep_scrub":"364'2","last_deep_scrub_stamp":"2024-09-10T19:02:15.059320+0000","last_clean_scrub_stamp":"2024-09-10T19:02:15.059320+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.020873968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":52,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.12","version":"70'81","reported_seq":343,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.215798+0000","last_change":"2024-09-10T19:06:08.005069+0000","last_active":"2024-09-10T19:06:56.215798+0000","last_peered":"2024-09-10T19:06:56.215798+0000","last_clean":"2024-09-10T19:06:56.215798+0000","last_became_active":"2024-09-10T18:52:16.598921+0000","last_became_peered":"2024-09-10T18:52:16.598921+0000","last_unstale":"2024-09-10T19:06:56.215798+0000","last_undegraded":"2024-09-10T19:06:56.215798+0000","last_fullsized":"2024-09-10T19:06:56.215798+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:06:08.005010+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:54.127476+0000","last_clean_scrub_stamp":"2024-09-10T19:06:08.005010+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:33.753507+0000","scrub_duration":0.0036749040000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":512,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.384037+0000","last_change":"2024-09-10T19:06:08.468915+0000","last_active":"2024-09-10T19:06:56.384037+0000","last_peered":"2024-09-10T19:06:56.384037+0000","last_clean":"2024-09-10T19:06:56.384037+0000","last_became_active":"2024-09-10T18:57:02.442251+0000","last_became_peered":"2024-09-10T18:57:02.442251+0000","last_unstale":"2024-09-10T19:06:56.384037+0000","last_undegraded":"2024-09-10T19:06:56.384037+0000","last_fullsized":"2024-09-10T19:06:56.384037+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:08.468838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:06:08.468838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:18.444598+0000","scrub_duration":0.0086820209999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"427'18","reported_seq":509,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.316175+0000","last_change":"2024-09-10T19:03:50.702400+0000","last_active":"2024-09-10T19:06:56.316175+0000","last_peered":"2024-09-10T19:06:56.316175+0000","last_clean":"2024-09-10T19:06:56.316175+0000","last_became_active":"2024-09-10T18:57:02.460430+0000","last_became_peered":"2024-09-10T18:57:02.460430+0000","last_unstale":"2024-09-10T19:06:56.316175+0000","last_undegraded":"2024-09-10T19:06:56.316175+0000","last_fullsized":"2024-09-10T19:06:56.316175+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'12","last_scrub_stamp":"2024-09-10T19:03:50.702317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:03:50.702317+0000","objects_scrubbed":3,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0055835499999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":24,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":471,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.214446+0000","last_change":"2024-09-10T19:06:30.117244+0000","last_active":"2024-09-10T19:06:56.214446+0000","last_peered":"2024-09-10T19:06:56.214446+0000","last_clean":"2024-09-10T19:06:56.214446+0000","last_became_active":"2024-09-10T18:45:22.069653+0000","last_became_peered":"2024-09-10T18:45:22.069653+0000","last_unstale":"2024-09-10T19:06:56.214446+0000","last_undegraded":"2024-09-10T19:06:56.214446+0000","last_fullsized":"2024-09-10T19:06:56.214446+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:30.117162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:06:30.117162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:36.601291+0000","scrub_duration":0.022352659,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":236,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.384071+0000","last_change":"2024-09-10T19:06:17.445445+0000","last_active":"2024-09-10T19:06:56.384071+0000","last_peered":"2024-09-10T19:06:56.384071+0000","last_clean":"2024-09-10T19:06:56.384071+0000","last_became_active":"2024-09-10T19:00:42.519492+0000","last_became_peered":"2024-09-10T19:00:42.519492+0000","last_unstale":"2024-09-10T19:06:56.384071+0000","last_undegraded":"2024-09-10T19:06:56.384071+0000","last_fullsized":"2024-09-10T19:06:56.384071+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:17.445379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:06:17.445379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:24.930408+0000","scrub_duration":0.002512116,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.11","version":"70'81","reported_seq":377,"reported_epoch":452,"state":"active+clean","last_fresh":"2024-09-10T19:06:50.838367+0000","last_change":"2024-09-10T19:04:33.761605+0000","last_active":"2024-09-10T19:06:50.838367+0000","last_peered":"2024-09-10T19:06:50.838367+0000","last_clean":"2024-09-10T19:06:50.838367+0000","last_became_active":"2024-09-10T19:00:42.517311+0000","last_became_peered":"2024-09-10T19:00:42.517311+0000","last_unstale":"2024-09-10T19:06:50.838367+0000","last_undegraded":"2024-09-10T19:06:50.838367+0000","last_fullsized":"2024-09-10T19:06:50.838367+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:04:33.761515+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:04:33.761515+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0042906769999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"427'30","reported_seq":572,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.315598+0000","last_change":"2024-09-10T19:04:33.464299+0000","last_active":"2024-09-10T19:06:56.315598+0000","last_peered":"2024-09-10T19:06:56.315598+0000","last_clean":"2024-09-10T19:06:56.315598+0000","last_became_active":"2024-09-10T18:57:02.444437+0000","last_became_peered":"2024-09-10T18:57:02.444437+0000","last_unstale":"2024-09-10T19:06:56.315598+0000","last_undegraded":"2024-09-10T19:06:56.315598+0000","last_fullsized":"2024-09-10T19:06:56.315598+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'20","last_scrub_stamp":"2024-09-10T19:04:33.464197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:04:33.464197+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0040076549999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":485,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.213325+0000","last_change":"2024-09-10T19:02:20.755424+0000","last_active":"2024-09-10T19:06:56.213325+0000","last_peered":"2024-09-10T19:06:56.213325+0000","last_clean":"2024-09-10T19:06:56.213325+0000","last_became_active":"2024-09-10T18:57:02.445710+0000","last_became_peered":"2024-09-10T18:57:02.445710+0000","last_unstale":"2024-09-10T19:06:56.213325+0000","last_undegraded":"2024-09-10T19:06:56.213325+0000","last_fullsized":"2024-09-10T19:06:56.213325+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:20.755253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:20.755253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0023404490000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"304'11","reported_seq":774,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-10T19:07:01.273773+0000","last_change":"2024-09-10T19:03:34.811295+0000","last_active":"2024-09-10T19:07:01.273773+0000","last_peered":"2024-09-10T19:07:01.273773+0000","last_clean":"2024-09-10T19:07:01.273773+0000","last_became_active":"2024-09-10T18:58:58.489370+0000","last_became_peered":"2024-09-10T18:58:58.489370+0000","last_unstale":"2024-09-10T19:07:01.273773+0000","last_undegraded":"2024-09-10T19:07:01.273773+0000","last_fullsized":"2024-09-10T19:07:01.273773+0000","mapping_epoch":303,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":304,"parent":"0.0","parent_split_bits":0,"last_scrub":"304'11","last_scrub_stamp":"2024-09-10T19:03:34.811194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:03:34.811194+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0024856539999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":162,"reported_epoch":452,"state":"active+clean","last_fresh":"2024-09-10T19:06:50.839216+0000","last_change":"2024-09-10T19:06:06.714131+0000","last_active":"2024-09-10T19:06:50.839216+0000","last_peered":"2024-09-10T19:06:50.839216+0000","last_clean":"2024-09-10T19:06:50.839216+0000","last_became_active":"2024-09-10T18:59:48.744923+0000","last_became_peered":"2024-09-10T18:59:48.744923+0000","last_unstale":"2024-09-10T19:06:50.839216+0000","last_undegraded":"2024-09-10T19:06:50.839216+0000","last_fullsized":"2024-09-10T19:06:50.839216+0000","mapping_epoch":320,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":321,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:06.714078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:57.476724+0000","last_clean_scrub_stamp":"2024-09-10T19:06:06.714078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:33.025365+0000","scrub_duration":0.001109358,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.10","version":"70'81","reported_seq":343,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.214016+0000","last_change":"2024-09-10T19:03:50.010677+0000","last_active":"2024-09-10T19:06:56.214016+0000","last_peered":"2024-09-10T19:06:56.214016+0000","last_clean":"2024-09-10T19:06:56.214016+0000","last_became_active":"2024-09-10T18:57:02.443782+0000","last_became_peered":"2024-09-10T18:57:02.443782+0000","last_unstale":"2024-09-10T19:06:56.214016+0000","last_undegraded":"2024-09-10T19:06:56.214016+0000","last_fullsized":"2024-09-10T19:06:56.214016+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:03:50.010596+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:03:50.010596+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0030874829999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"427'21","reported_seq":154,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.217355+0000","last_change":"2024-09-10T19:02:46.674532+0000","last_active":"2024-09-10T19:06:56.217355+0000","last_peered":"2024-09-10T19:06:56.217355+0000","last_clean":"2024-09-10T19:06:56.217355+0000","last_became_active":"2024-09-10T19:00:42.518672+0000","last_became_peered":"2024-09-10T19:00:42.518672+0000","last_unstale":"2024-09-10T19:06:56.217355+0000","last_undegraded":"2024-09-10T19:06:56.217355+0000","last_fullsized":"2024-09-10T19:06:56.217355+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T19:02:46.674451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:46.674451+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0082292530000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":33,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":451,"reported_epoch":452,"state":"active+clean","last_fresh":"2024-09-10T19:06:50.838338+0000","last_change":"2024-09-10T19:04:34.812187+0000","last_active":"2024-09-10T19:06:50.838338+0000","last_peered":"2024-09-10T19:06:50.838338+0000","last_clean":"2024-09-10T19:06:50.838338+0000","last_became_active":"2024-09-10T18:55:40.390697+0000","last_became_peered":"2024-09-10T18:55:40.390697+0000","last_unstale":"2024-09-10T19:06:50.838338+0000","last_undegraded":"2024-09-10T19:06:50.838338+0000","last_fullsized":"2024-09-10T19:06:50.838338+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:34.812044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:04:34.812044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.013174425,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"273'11","reported_seq":757,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-10T19:07:01.036527+0000","last_change":"2024-09-10T19:03:49.744278+0000","last_active":"2024-09-10T19:07:01.036527+0000","last_peered":"2024-09-10T19:07:01.036527+0000","last_clean":"2024-09-10T19:07:01.036527+0000","last_became_active":"2024-09-10T18:57:02.445268+0000","last_became_peered":"2024-09-10T18:57:02.445268+0000","last_unstale":"2024-09-10T19:07:01.036527+0000","last_undegraded":"2024-09-10T19:07:01.036527+0000","last_fullsized":"2024-09-10T19:07:01.036527+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'11","last_scrub_stamp":"2024-09-10T19:03:49.744177+0000","last_deep_scrub":"273'11","last_deep_scrub_stamp":"2024-09-10T19:03:49.744177+0000","last_clean_scrub_stamp":"2024-09-10T19:03:49.744177+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0069874999999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":484,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.066490+0000","last_change":"2024-09-10T19:03:23.737568+0000","last_active":"2024-09-10T19:06:56.066490+0000","last_peered":"2024-09-10T19:06:56.066490+0000","last_clean":"2024-09-10T19:06:56.066490+0000","last_became_active":"2024-09-10T18:55:40.391662+0000","last_became_peered":"2024-09-10T18:55:40.391662+0000","last_unstale":"2024-09-10T19:06:56.066490+0000","last_undegraded":"2024-09-10T19:06:56.066490+0000","last_fullsized":"2024-09-10T19:06:56.066490+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:23.737518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_clean_scrub_stamp":"2024-09-10T19:03:23.737518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037070459999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"427'21","reported_seq":378,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.382426+0000","last_change":"2024-09-10T19:03:37.134082+0000","last_active":"2024-09-10T19:06:56.382426+0000","last_peered":"2024-09-10T19:06:56.382426+0000","last_clean":"2024-09-10T19:06:56.382426+0000","last_became_active":"2024-09-10T19:02:49.561403+0000","last_became_peered":"2024-09-10T19:02:49.561403+0000","last_unstale":"2024-09-10T19:06:56.382426+0000","last_undegraded":"2024-09-10T19:06:56.382426+0000","last_fullsized":"2024-09-10T19:06:56.382426+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'15","last_scrub_stamp":"2024-09-10T19:03:37.134028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:03:37.134028+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0058255470000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":516,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.386440+0000","last_change":"2024-09-10T19:03:36.087607+0000","last_active":"2024-09-10T19:06:56.386440+0000","last_peered":"2024-09-10T19:06:56.386440+0000","last_clean":"2024-09-10T19:06:56.386440+0000","last_became_active":"2024-09-10T19:00:42.518310+0000","last_became_peered":"2024-09-10T19:00:42.518310+0000","last_unstale":"2024-09-10T19:06:56.386440+0000","last_undegraded":"2024-09-10T19:06:56.386440+0000","last_fullsized":"2024-09-10T19:06:56.386440+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:36.087523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:03:36.087523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0039520240000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":466,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.316593+0000","last_change":"2024-09-10T19:02:22.258152+0000","last_active":"2024-09-10T19:06:56.316593+0000","last_peered":"2024-09-10T19:06:56.316593+0000","last_clean":"2024-09-10T19:06:56.316593+0000","last_became_active":"2024-09-10T18:55:40.398243+0000","last_became_peered":"2024-09-10T18:55:40.398243+0000","last_unstale":"2024-09-10T19:06:56.316593+0000","last_undegraded":"2024-09-10T19:06:56.316593+0000","last_fullsized":"2024-09-10T19:06:56.316593+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:22.258058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:03.571786+0000","last_clean_scrub_stamp":"2024-09-10T19:02:22.258058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0054525670000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":456,"reported_epoch":452,"state":"active+clean","last_fresh":"2024-09-10T19:06:50.838366+0000","last_change":"2024-09-10T19:03:22.930232+0000","last_active":"2024-09-10T19:06:50.838366+0000","last_peered":"2024-09-10T19:06:50.838366+0000","last_clean":"2024-09-10T19:06:50.838366+0000","last_became_active":"2024-09-10T19:00:42.516833+0000","last_became_peered":"2024-09-10T19:00:42.516833+0000","last_unstale":"2024-09-10T19:06:50.838366+0000","last_undegraded":"2024-09-10T19:06:50.838366+0000","last_fullsized":"2024-09-10T19:06:50.838366+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:22.930166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:04.631558+0000","last_clean_scrub_stamp":"2024-09-10T19:03:22.930166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001481163,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"427'32","reported_seq":509,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.063496+0000","last_change":"2024-09-10T19:03:37.562262+0000","last_active":"2024-09-10T19:06:56.063496+0000","last_peered":"2024-09-10T19:06:56.063496+0000","last_clean":"2024-09-10T19:06:56.063496+0000","last_became_active":"2024-09-10T18:45:20.056885+0000","last_became_peered":"2024-09-10T18:45:20.056885+0000","last_unstale":"2024-09-10T19:06:56.063496+0000","last_undegraded":"2024-09-10T19:06:56.063496+0000","last_fullsized":"2024-09-10T19:06:56.063496+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T19:03:37.562188+0000","last_deep_scrub":"223'16","last_deep_scrub_stamp":"2024-09-10T18:54:47.324044+0000","last_clean_scrub_stamp":"2024-09-10T19:03:37.562188+0000","objects_scrubbed":5,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041736869999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":346,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.217479+0000","last_change":"2024-09-10T19:06:32.121019+0000","last_active":"2024-09-10T19:06:56.217479+0000","last_peered":"2024-09-10T19:06:56.217479+0000","last_clean":"2024-09-10T19:06:56.217479+0000","last_became_active":"2024-09-10T19:02:49.572748+0000","last_became_peered":"2024-09-10T19:02:49.572748+0000","last_unstale":"2024-09-10T19:06:56.217479+0000","last_undegraded":"2024-09-10T19:06:56.217479+0000","last_fullsized":"2024-09-10T19:06:56.217479+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:32.120944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:00.029389+0000","last_clean_scrub_stamp":"2024-09-10T19:06:32.120944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:35.654481+0000","scrub_duration":0.0048228450000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":94,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.215871+0000","last_change":"2024-09-10T19:02:49.573280+0000","last_active":"2024-09-10T19:06:56.215871+0000","last_peered":"2024-09-10T19:06:56.215871+0000","last_clean":"2024-09-10T19:06:56.215871+0000","last_became_active":"2024-09-10T19:02:49.572998+0000","last_became_peered":"2024-09-10T19:02:49.572998+0000","last_unstale":"2024-09-10T19:06:56.215871+0000","last_undegraded":"2024-09-10T19:06:56.215871+0000","last_fullsized":"2024-09-10T19:06:56.215871+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:15.321767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:02:15.321767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":474,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.320847+0000","last_change":"2024-09-10T19:06:08.922418+0000","last_active":"2024-09-10T19:06:56.320847+0000","last_peered":"2024-09-10T19:06:56.320847+0000","last_clean":"2024-09-10T19:06:56.320847+0000","last_became_active":"2024-09-10T18:55:40.395595+0000","last_became_peered":"2024-09-10T18:55:40.395595+0000","last_unstale":"2024-09-10T19:06:56.320847+0000","last_undegraded":"2024-09-10T19:06:56.320847+0000","last_fullsized":"2024-09-10T19:06:56.320847+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:08.922339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:06:08.922339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:33.632921+0000","scrub_duration":0.0055359140000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"427'12","reported_seq":479,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.320694+0000","last_change":"2024-09-10T19:02:50.391929+0000","last_active":"2024-09-10T19:06:56.320694+0000","last_peered":"2024-09-10T19:06:56.320694+0000","last_clean":"2024-09-10T19:06:56.320694+0000","last_became_active":"2024-09-10T18:45:20.059205+0000","last_became_peered":"2024-09-10T18:45:20.059205+0000","last_unstale":"2024-09-10T19:06:56.320694+0000","last_undegraded":"2024-09-10T19:06:56.320694+0000","last_fullsized":"2024-09-10T19:06:56.320694+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'8","last_scrub_stamp":"2024-09-10T19:02:50.391489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:02:50.391489+0000","objects_scrubbed":2,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0041757890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":16,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":481,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.214501+0000","last_change":"2024-09-10T19:02:16.704889+0000","last_active":"2024-09-10T19:06:56.214501+0000","last_peered":"2024-09-10T19:06:56.214501+0000","last_clean":"2024-09-10T19:06:56.214501+0000","last_became_active":"2024-09-10T18:47:36.415866+0000","last_became_peered":"2024-09-10T18:47:36.415866+0000","last_unstale":"2024-09-10T19:06:56.214501+0000","last_undegraded":"2024-09-10T19:06:56.214501+0000","last_fullsized":"2024-09-10T19:06:56.214501+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:16.704805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:02:16.704805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0083395729999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":502,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.384322+0000","last_change":"2024-09-10T19:06:20.478219+0000","last_active":"2024-09-10T19:06:56.384322+0000","last_peered":"2024-09-10T19:06:56.384322+0000","last_clean":"2024-09-10T19:06:56.384322+0000","last_became_active":"2024-09-10T18:57:02.453974+0000","last_became_peered":"2024-09-10T18:57:02.453974+0000","last_unstale":"2024-09-10T19:06:56.384322+0000","last_undegraded":"2024-09-10T19:06:56.384322+0000","last_fullsized":"2024-09-10T19:06:56.384322+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:20.478081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:35.172861+0000","last_clean_scrub_stamp":"2024-09-10T19:06:20.478081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:35.087111+0000","scrub_duration":0.0034106200000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":262,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.062774+0000","last_change":"2024-09-10T19:04:32.388680+0000","last_active":"2024-09-10T19:06:56.062774+0000","last_peered":"2024-09-10T19:06:56.062774+0000","last_clean":"2024-09-10T19:06:56.062774+0000","last_became_active":"2024-09-10T18:57:02.447566+0000","last_became_peered":"2024-09-10T18:57:02.447566+0000","last_unstale":"2024-09-10T19:06:56.062774+0000","last_undegraded":"2024-09-10T19:06:56.062774+0000","last_fullsized":"2024-09-10T19:06:56.062774+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:32.388609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:55.145638+0000","last_clean_scrub_stamp":"2024-09-10T19:04:32.388609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0098792380000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.14","version":"70'81","reported_seq":376,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.066861+0000","last_change":"2024-09-10T19:06:31.848113+0000","last_active":"2024-09-10T19:06:56.066861+0000","last_peered":"2024-09-10T19:06:56.066861+0000","last_clean":"2024-09-10T19:06:56.066861+0000","last_became_active":"2024-09-10T18:57:02.442457+0000","last_became_peered":"2024-09-10T18:57:02.442457+0000","last_unstale":"2024-09-10T19:06:56.066861+0000","last_undegraded":"2024-09-10T19:06:56.066861+0000","last_fullsized":"2024-09-10T19:06:56.066861+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:06:31.848054+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:57.184383+0000","last_clean_scrub_stamp":"2024-09-10T19:06:31.848054+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:58.376194+0000","scrub_duration":0.0083693320000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.11","version":"427'32","reported_seq":531,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.320668+0000","last_change":"2024-09-10T19:03:43.653800+0000","last_active":"2024-09-10T19:06:56.320668+0000","last_peered":"2024-09-10T19:06:56.320668+0000","last_clean":"2024-09-10T19:06:56.320668+0000","last_became_active":"2024-09-10T18:57:02.461909+0000","last_became_peered":"2024-09-10T18:57:02.461909+0000","last_unstale":"2024-09-10T19:06:56.320668+0000","last_undegraded":"2024-09-10T19:06:56.320668+0000","last_fullsized":"2024-09-10T19:06:56.320668+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'24","last_scrub_stamp":"2024-09-10T19:03:43.653711+0000","last_deep_scrub":"79'16","last_deep_scrub_stamp":"2024-09-10T18:46:48.744287+0000","last_clean_scrub_stamp":"2024-09-10T19:03:43.653711+0000","objects_scrubbed":5,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0037824379999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"56'1","reported_seq":510,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.383034+0000","last_change":"2024-09-10T19:06:06.554681+0000","last_active":"2024-09-10T19:06:56.383034+0000","last_peered":"2024-09-10T19:06:56.383034+0000","last_clean":"2024-09-10T19:06:56.383034+0000","last_became_active":"2024-09-10T18:47:36.421985+0000","last_became_peered":"2024-09-10T18:47:36.421985+0000","last_unstale":"2024-09-10T19:06:56.383034+0000","last_undegraded":"2024-09-10T19:06:56.383034+0000","last_fullsized":"2024-09-10T19:06:56.383034+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T19:06:06.554594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:06:06.554594+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:23.803720+0000","scrub_duration":0.0030413559999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":219,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.062843+0000","last_change":"2024-09-10T19:06:28.848162+0000","last_active":"2024-09-10T19:06:56.062843+0000","last_peered":"2024-09-10T19:06:56.062843+0000","last_clean":"2024-09-10T19:06:56.062843+0000","last_became_active":"2024-09-10T18:57:02.435124+0000","last_became_peered":"2024-09-10T18:57:02.435124+0000","last_unstale":"2024-09-10T19:06:56.062843+0000","last_undegraded":"2024-09-10T19:06:56.062843+0000","last_fullsized":"2024-09-10T19:06:56.062843+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:28.848105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:06:28.848105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:51.629925+0000","scrub_duration":0.0051945860000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":484,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.315227+0000","last_change":"2024-09-10T19:06:20.199145+0000","last_active":"2024-09-10T19:06:56.315227+0000","last_peered":"2024-09-10T19:06:56.315227+0000","last_clean":"2024-09-10T19:06:56.315227+0000","last_became_active":"2024-09-10T18:55:40.397523+0000","last_became_peered":"2024-09-10T18:55:40.397523+0000","last_unstale":"2024-09-10T19:06:56.315227+0000","last_undegraded":"2024-09-10T19:06:56.315227+0000","last_fullsized":"2024-09-10T19:06:56.315227+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:06:20.199071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:06:20.199071+0000","last_clean_scrub_stamp":"2024-09-10T19:06:20.199071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:32.556730+0000","scrub_duration":0.0051592299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"78'2","reported_seq":262,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.215136+0000","last_change":"2024-09-10T19:06:28.113529+0000","last_active":"2024-09-10T19:06:56.215136+0000","last_peered":"2024-09-10T19:06:56.215136+0000","last_clean":"2024-09-10T19:06:56.215136+0000","last_became_active":"2024-09-10T18:57:02.447198+0000","last_became_peered":"2024-09-10T18:57:02.447198+0000","last_unstale":"2024-09-10T19:06:56.215136+0000","last_undegraded":"2024-09-10T19:06:56.215136+0000","last_fullsized":"2024-09-10T19:06:56.215136+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'2","last_scrub_stamp":"2024-09-10T19:06:28.113436+0000","last_deep_scrub":"78'2","last_deep_scrub_stamp":"2024-09-10T19:02:15.686138+0000","last_clean_scrub_stamp":"2024-09-10T19:06:28.113436+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:07:35.604709+0000","scrub_duration":0.0025017680000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":485,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.215042+0000","last_change":"2024-09-10T19:03:44.943009+0000","last_active":"2024-09-10T19:06:56.215042+0000","last_peered":"2024-09-10T19:06:56.215042+0000","last_clean":"2024-09-10T19:06:56.215042+0000","last_became_active":"2024-09-10T18:57:02.443353+0000","last_became_peered":"2024-09-10T18:57:02.443353+0000","last_unstale":"2024-09-10T19:06:56.215042+0000","last_undegraded":"2024-09-10T19:06:56.215042+0000","last_fullsized":"2024-09-10T19:06:56.215042+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:44.942954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:03:44.942954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016874889999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"427'30","reported_seq":515,"reported_epoch":452,"state":"active+clean","last_fresh":"2024-09-10T19:06:50.839376+0000","last_change":"2024-09-10T19:04:31.733896+0000","last_active":"2024-09-10T19:06:50.839376+0000","last_peered":"2024-09-10T19:06:50.839376+0000","last_clean":"2024-09-10T19:06:50.839376+0000","last_became_active":"2024-09-10T18:57:02.452473+0000","last_became_peered":"2024-09-10T18:57:02.452473+0000","last_unstale":"2024-09-10T19:06:50.839376+0000","last_undegraded":"2024-09-10T19:06:50.839376+0000","last_fullsized":"2024-09-10T19:06:50.839376+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'20","last_scrub_stamp":"2024-09-10T19:04:31.733806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:04:31.733806+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0051209239999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":38,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"273'10","reported_seq":769,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-10T19:07:01.297969+0000","last_change":"2024-09-10T19:03:24.245622+0000","last_active":"2024-09-10T19:07:01.297969+0000","last_peered":"2024-09-10T19:07:01.297969+0000","last_clean":"2024-09-10T19:07:01.297969+0000","last_became_active":"2024-09-10T18:57:02.441763+0000","last_became_peered":"2024-09-10T18:57:02.441763+0000","last_unstale":"2024-09-10T19:07:01.297969+0000","last_undegraded":"2024-09-10T19:07:01.297969+0000","last_fullsized":"2024-09-10T19:07:01.297969+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'10","last_scrub_stamp":"2024-09-10T19:03:24.245532+0000","last_deep_scrub":"190'9","last_deep_scrub_stamp":"2024-09-10T18:53:07.746309+0000","last_clean_scrub_stamp":"2024-09-10T19:03:24.245532+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0019049480000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":94,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.214140+0000","last_change":"2024-09-10T19:02:49.582100+0000","last_active":"2024-09-10T19:06:56.214140+0000","last_peered":"2024-09-10T19:06:56.214140+0000","last_clean":"2024-09-10T19:06:56.214140+0000","last_became_active":"2024-09-10T19:02:49.581739+0000","last_became_peered":"2024-09-10T19:02:49.581739+0000","last_unstale":"2024-09-10T19:06:56.214140+0000","last_undegraded":"2024-09-10T19:06:56.214140+0000","last_fullsized":"2024-09-10T19:06:56.214140+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:23.327767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:03.475299+0000","last_clean_scrub_stamp":"2024-09-10T19:02:23.327767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":480,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.320223+0000","last_change":"2024-09-10T19:04:34.504898+0000","last_active":"2024-09-10T19:06:56.320223+0000","last_peered":"2024-09-10T19:06:56.320223+0000","last_clean":"2024-09-10T19:06:56.320223+0000","last_became_active":"2024-09-10T18:55:40.392524+0000","last_became_peered":"2024-09-10T18:55:40.392524+0000","last_unstale":"2024-09-10T19:06:56.320223+0000","last_undegraded":"2024-09-10T19:06:56.320223+0000","last_fullsized":"2024-09-10T19:06:56.320223+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:04:34.504828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:04:34.504828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.016884452000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"427'27","reported_seq":530,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.381960+0000","last_change":"2024-09-10T19:04:33.326902+0000","last_active":"2024-09-10T19:06:56.381960+0000","last_peered":"2024-09-10T19:06:56.381960+0000","last_clean":"2024-09-10T19:06:56.381960+0000","last_became_active":"2024-09-10T18:45:20.058717+0000","last_became_peered":"2024-09-10T18:45:20.058717+0000","last_unstale":"2024-09-10T19:06:56.381960+0000","last_undegraded":"2024-09-10T19:06:56.381960+0000","last_fullsized":"2024-09-10T19:06:56.381960+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'19","last_scrub_stamp":"2024-09-10T19:04:33.326849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:04:33.326849+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0036326280000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":444,"reported_epoch":452,"state":"active+clean","last_fresh":"2024-09-10T19:06:50.839441+0000","last_change":"2024-09-10T19:03:25.864320+0000","last_active":"2024-09-10T19:06:50.839441+0000","last_peered":"2024-09-10T19:06:50.839441+0000","last_clean":"2024-09-10T19:06:50.839441+0000","last_became_active":"2024-09-10T18:57:02.453464+0000","last_became_peered":"2024-09-10T18:57:02.453464+0000","last_unstale":"2024-09-10T19:06:50.839441+0000","last_undegraded":"2024-09-10T19:06:50.839441+0000","last_fullsized":"2024-09-10T19:06:50.839441+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:25.864264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:03:25.864264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0013441410000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":457,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.066658+0000","last_change":"2024-09-10T19:03:43.645318+0000","last_active":"2024-09-10T19:06:56.066658+0000","last_peered":"2024-09-10T19:06:56.066658+0000","last_clean":"2024-09-10T19:06:56.066658+0000","last_became_active":"2024-09-10T18:45:24.267251+0000","last_became_peered":"2024-09-10T18:45:24.267251+0000","last_unstale":"2024-09-10T19:06:56.066658+0000","last_undegraded":"2024-09-10T19:06:56.066658+0000","last_fullsized":"2024-09-10T19:06:56.066658+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:43.645260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:03:43.645260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.003834732,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"306'2","reported_seq":503,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.383769+0000","last_change":"2024-09-10T19:03:00.120585+0000","last_active":"2024-09-10T19:06:56.383769+0000","last_peered":"2024-09-10T19:06:56.383769+0000","last_clean":"2024-09-10T19:06:56.383769+0000","last_became_active":"2024-09-10T18:57:02.435758+0000","last_became_peered":"2024-09-10T18:57:02.435758+0000","last_unstale":"2024-09-10T19:06:56.383769+0000","last_undegraded":"2024-09-10T19:06:56.383769+0000","last_fullsized":"2024-09-10T19:06:56.383769+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"306'2","last_scrub_stamp":"2024-09-10T19:03:00.120529+0000","last_deep_scrub":"306'2","last_deep_scrub_stamp":"2024-09-10T18:59:20.534869+0000","last_clean_scrub_stamp":"2024-09-10T19:03:00.120529+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0016153859999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":874,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"427'46","reported_seq":429,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.315708+0000","last_change":"2024-09-10T19:04:38.485166+0000","last_active":"2024-09-10T19:06:56.315708+0000","last_peered":"2024-09-10T19:06:56.315708+0000","last_clean":"2024-09-10T19:06:56.315708+0000","last_became_active":"2024-09-10T19:02:49.594207+0000","last_became_peered":"2024-09-10T19:02:49.594207+0000","last_unstale":"2024-09-10T19:06:56.315708+0000","last_undegraded":"2024-09-10T19:06:56.315708+0000","last_fullsized":"2024-09-10T19:06:56.315708+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'34","last_scrub_stamp":"2024-09-10T19:04:38.485104+0000","last_deep_scrub":"258'34","last_deep_scrub_stamp":"2024-09-10T18:56:23.838951+0000","last_clean_scrub_stamp":"2024-09-10T19:04:38.485104+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.001834518,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":322,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.063954+0000","last_change":"2024-09-10T19:02:49.581903+0000","last_active":"2024-09-10T19:06:56.063954+0000","last_peered":"2024-09-10T19:06:56.063954+0000","last_clean":"2024-09-10T19:06:56.063954+0000","last_became_active":"2024-09-10T19:02:49.581575+0000","last_became_peered":"2024-09-10T19:02:49.581575+0000","last_unstale":"2024-09-10T19:06:56.063954+0000","last_undegraded":"2024-09-10T19:06:56.063954+0000","last_fullsized":"2024-09-10T19:06:56.063954+0000","mapping_epoch":372,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:02:03.343366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:02:03.343366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0079641550000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":477,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.315879+0000","last_change":"2024-09-10T19:03:33.638108+0000","last_active":"2024-09-10T19:06:56.315879+0000","last_peered":"2024-09-10T19:06:56.315879+0000","last_clean":"2024-09-10T19:06:56.315879+0000","last_became_active":"2024-09-10T19:00:42.520239+0000","last_became_peered":"2024-09-10T19:00:42.520239+0000","last_unstale":"2024-09-10T19:06:56.315879+0000","last_undegraded":"2024-09-10T19:06:56.315879+0000","last_fullsized":"2024-09-10T19:06:56.315879+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:33.638010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:17.985545+0000","last_clean_scrub_stamp":"2024-09-10T19:03:33.638010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0035871750000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":483,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.316782+0000","last_change":"2024-09-10T19:03:36.615878+0000","last_active":"2024-09-10T19:06:56.316782+0000","last_peered":"2024-09-10T19:06:56.316782+0000","last_clean":"2024-09-10T19:06:56.316782+0000","last_became_active":"2024-09-10T18:57:02.460855+0000","last_became_peered":"2024-09-10T18:57:02.460855+0000","last_unstale":"2024-09-10T19:06:56.316782+0000","last_undegraded":"2024-09-10T19:06:56.316782+0000","last_fullsized":"2024-09-10T19:06:56.316782+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:36.615788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:03:36.615788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.004374223,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"427'29","reported_seq":507,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.317414+0000","last_change":"2024-09-10T19:03:40.698360+0000","last_active":"2024-09-10T19:06:56.317414+0000","last_peered":"2024-09-10T19:06:56.317414+0000","last_clean":"2024-09-10T19:06:56.317414+0000","last_became_active":"2024-09-10T18:45:20.058391+0000","last_became_peered":"2024-09-10T18:45:20.058391+0000","last_unstale":"2024-09-10T19:06:56.317414+0000","last_undegraded":"2024-09-10T19:06:56.317414+0000","last_fullsized":"2024-09-10T19:06:56.317414+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"388'23","last_scrub_stamp":"2024-09-10T19:03:40.697702+0000","last_deep_scrub":"388'23","last_deep_scrub_stamp":"2024-09-10T19:03:40.697702+0000","last_clean_scrub_stamp":"2024-09-10T19:03:40.697702+0000","objects_scrubbed":5,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.076817413000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":489,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-10T19:06:56.066749+0000","last_change":"2024-09-10T19:03:46.623850+0000","last_active":"2024-09-10T19:06:56.066749+0000","last_peered":"2024-09-10T19:06:56.066749+0000","last_clean":"2024-09-10T19:06:56.066749+0000","last_became_active":"2024-09-10T18:57:02.449135+0000","last_became_peered":"2024-09-10T18:57:02.449135+0000","last_unstale":"2024-09-10T19:06:56.066749+0000","last_undegraded":"2024-09-10T19:06:56.066749+0000","last_fullsized":"2024-09-10T19:06:56.066749+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:03:46.623782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:03:46.623782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":0.0075382380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":421,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1544,"num_read_kb":1301,"num_write":989,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":3152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":499712,"data_stored":13268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1057,"ondisk_log_size":1057,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":364,"num_read_kb":359,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":262688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1701,"ondisk_log_size":1701,"up":63,"acting":63,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":0,"seq":1765231558683,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":43,"seq":184683593998,"num_pgs":80,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":929588,"kb_used_data":35432,"kb_used_omap":1,"kb_used_meta":894142,"kb_avail":92815564,"statfs":{"total":95995035648,"available":95043137536,"internally_reserved":0,"allocated":36282368,"data_stored":35880727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":915601752},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:06:07 2024","interfaces":[{"interface":"back","average":{"1min":6.4509999999999996,"5min":3.0209999999999999,"15min":1.698},"min":{"1min":0.217,"5min":0.217,"15min":0.20899999999999999},"max":{"1min":44.320999999999998,"5min":49.353999999999999,"15min":49.353999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":6.7450000000000001,"5min":3.202,"15min":1.819},"min":{"1min":0.23799999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":44.311999999999998,"5min":49.341999999999999,"15min":49.341999999999999},"last":0.64600000000000002}]},{"osd":1,"last update":"Tue Sep 10 19:06:18 2024","interfaces":[{"interface":"back","average":{"1min":4.9950000000000001,"5min":2.194,"15min":1.728},"min":{"1min":0.23200000000000001,"5min":0.217,"15min":0.217},"max":{"1min":85.775000000000006,"5min":85.775000000000006,"15min":85.775000000000006},"last":0.53000000000000003},{"interface":"front","average":{"1min":4.9580000000000002,"5min":2.0550000000000002,"15min":1.621},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.23300000000000001},"max":{"1min":85.670000000000002,"5min":85.670000000000002,"15min":85.670000000000002},"last":0.48699999999999999}]},{"osd":2,"last update":"Tue Sep 10 19:06:18 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":0.999,"15min":0.86299999999999999},"min":{"1min":0.27000000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":4.0469999999999997,"5min":23.867000000000001,"15min":23.867000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":1.149,"15min":0.90100000000000002},"min":{"1min":0.29399999999999998,"5min":0.28699999999999998,"15min":0.249},"max":{"1min":2.1360000000000001,"5min":23.779,"15min":23.779},"last":0.54800000000000004}]},{"osd":3,"last update":"Tue Sep 10 19:06:18 2024","interfaces":[{"interface":"back","average":{"1min":1.8720000000000001,"5min":1.758,"15min":1.742},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":20.994,"5min":47.325000000000003,"15min":76.977999999999994},"last":0.46700000000000003},{"interface":"front","average":{"1min":1.9870000000000001,"5min":1.7689999999999999,"15min":1.776},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.253},"max":{"1min":20.177,"5min":46.793999999999997,"15min":76.879999999999995},"last":0.53700000000000003}]},{"osd":4,"last update":"Tue Sep 10 19:06:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":1.1060000000000001,"15min":1.5109999999999999},"min":{"1min":0.26100000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.9039999999999999,"5min":32.469000000000001,"15min":165.03299999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.877,"15min":1.365},"min":{"1min":0.30399999999999999,"5min":0.252,"15min":0.22500000000000001},"max":{"1min":3.0819999999999999,"5min":32.5,"15min":163.959},"last":0.32800000000000001}]},{"osd":5,"last update":"Tue Sep 10 19:06:18 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.077,"15min":1.9259999999999999},"min":{"1min":0.29499999999999998,"5min":0.246,"15min":0.246},"max":{"1min":2.968,"5min":23.827000000000002,"15min":288.56999999999999},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.626,"5min":1.0089999999999999,"15min":1.897},"min":{"1min":0.32900000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":2.3809999999999998,"5min":23.667000000000002,"15min":291.05399999999997},"last":0.374}]},{"osd":7,"last update":"Tue Sep 10 19:06:53 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":0.78100000000000003,"15min":0.73099999999999998},"min":{"1min":0.34799999999999998,"5min":0.307,"15min":0.307},"max":{"1min":2.9140000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":0.317},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.33500000000000002,"5min":0.317,"15min":0.317},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":0.39400000000000002}]}]},{"osd":5,"up_from":37,"seq":158913790226,"num_pgs":77,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1184420,"kb_used_data":4060,"kb_used_omap":1,"kb_used_meta":1180350,"kb_avail":92560732,"statfs":{"total":95995035648,"available":94782189568,"internally_reserved":0,"allocated":4157440,"data_stored":3760430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":1208678744},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:07:00 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.306,"15min":1.9039999999999999},"min":{"1min":0.22900000000000001,"5min":0.224,"15min":0.224},"max":{"1min":5.048,"5min":31.626999999999999,"15min":82.465000000000003},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.92800000000000005,"5min":1.482,"15min":2.052},"min":{"1min":0.33200000000000002,"5min":0.27900000000000003,"15min":0.218},"max":{"1min":2.9209999999999998,"5min":31.734999999999999,"15min":82.519000000000005},"last":0.73899999999999999}]},{"osd":1,"last update":"Tue Sep 10 19:07:00 2024","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":3.0059999999999998,"15min":2.3359999999999999},"min":{"1min":0.25900000000000001,"5min":0.22500000000000001,"15min":0.19500000000000001},"max":{"1min":5.452,"5min":153.17099999999999,"15min":153.17099999999999},"last":0.36499999999999999},{"interface":"front","average":{"1min":1.4279999999999999,"5min":3.1299999999999999,"15min":2.4950000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.20399999999999999},"max":{"1min":5.4420000000000002,"5min":153.196,"15min":153.196},"last":0.56599999999999995}]},{"osd":2,"last update":"Tue Sep 10 19:07:00 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":1.01,"15min":0.90600000000000003},"min":{"1min":0.32000000000000001,"5min":0.249,"15min":0.249},"max":{"1min":3.0990000000000002,"5min":31.774000000000001,"15min":31.774000000000001},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.92900000000000005,"15min":0.871},"min":{"1min":0.318,"5min":0.27200000000000002,"15min":0.24399999999999999},"max":{"1min":2.6389999999999998,"5min":31.722999999999999,"15min":31.722999999999999},"last":0.41399999999999998}]},{"osd":3,"last update":"Tue Sep 10 19:07:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.194,"15min":2.597},"min":{"1min":0.40200000000000002,"5min":0.26300000000000001,"15min":0.24299999999999999},"max":{"1min":3.0760000000000001,"5min":31.753,"15min":123.34399999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":1.1879999999999999,"15min":2.1600000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":2.609,"5min":31.699999999999999,"15min":82.415000000000006},"last":0.46899999999999997}]},{"osd":4,"last update":"Tue Sep 10 19:07:00 2024","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.331,"15min":1.198},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":5.0099999999999998,"5min":51.070999999999998,"15min":51.070999999999998},"last":0.624},{"interface":"front","average":{"1min":0.87,"5min":1.2470000000000001,"15min":1.069},"min":{"1min":0.27100000000000002,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":3.0419999999999998,"5min":51.122999999999998,"15min":51.122999999999998},"last":0.44700000000000001}]},{"osd":6,"last update":"Tue Sep 10 19:06:20 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":1.2430000000000001,"15min":1.127},"min":{"1min":0.24199999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":4.0490000000000004,"5min":33.936999999999998,"15min":33.936999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.2230000000000001,"15min":1.0920000000000001},"min":{"1min":0.307,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":5.1710000000000003,"5min":33.951000000000001,"15min":33.951000000000001},"last":0.53700000000000003}]},{"osd":7,"last update":"Tue Sep 10 19:06:53 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.67800000000000005,"15min":0.69199999999999995},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.2890000000000001,"5min":2.657,"15min":2.657},"last":0.498},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.74399999999999999,"15min":0.72599999999999998},"min":{"1min":0.33300000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":2.6230000000000002,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":0.76600000000000001}]}]},{"osd":4,"up_from":30,"seq":128849019160,"num_pgs":84,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1225772,"kb_used_data":42912,"kb_used_omap":1,"kb_used_meta":1182846,"kb_avail":92519380,"statfs":{"total":95995035648,"available":94739845120,"internally_reserved":0,"allocated":43941888,"data_stored":43547461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":1211234648},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:05:59 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":1.514,"15min":1.4830000000000001},"min":{"1min":0.251,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":2.819,"5min":40.481000000000002,"15min":46.234999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.93200000000000005,"5min":1.5649999999999999,"15min":1.4850000000000001},"min":{"1min":0.35499999999999998,"5min":0.253,"15min":0.253},"max":{"1min":2.8380000000000001,"5min":40.512,"15min":46.264000000000003},"last":0.746}]},{"osd":1,"last update":"Tue Sep 10 19:06:38 2024","interfaces":[{"interface":"back","average":{"1min":2.1640000000000001,"5min":1.3720000000000001,"15min":1.7350000000000001},"min":{"1min":0.28799999999999998,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":16.535,"5min":16.535,"15min":125.15600000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":1.8280000000000001,"5min":1.081,"15min":1.619},"min":{"1min":0.28199999999999997,"5min":0.23300000000000001,"15min":0.23100000000000001},"max":{"1min":16.504000000000001,"5min":16.504000000000001,"15min":125.14700000000001},"last":0.65700000000000003}]},{"osd":2,"last update":"Tue Sep 10 19:06:38 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.68500000000000005,"15min":0.72599999999999998},"min":{"1min":0.28999999999999998,"5min":0.28499999999999998,"15min":0.25900000000000001},"max":{"1min":2.0350000000000001,"5min":3.3999999999999999,"15min":5.6189999999999998},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":2.7599999999999998,"15min":1.5229999999999999},"min":{"1min":0.35599999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":2.3519999999999999,"5min":190.38200000000001,"15min":190.38200000000001},"last":0.62}]},{"osd":3,"last update":"Tue Sep 10 19:06:38 2024","interfaces":[{"interface":"back","average":{"1min":2.2549999999999999,"5min":4.0279999999999996,"15min":3.0649999999999999},"min":{"1min":0.38300000000000001,"5min":0.27200000000000002,"15min":0.23799999999999999},"max":{"1min":17.068999999999999,"5min":190.41300000000001,"15min":190.41300000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":2.1320000000000001,"5min":1.863,"15min":1.9690000000000001},"min":{"1min":0.30199999999999999,"5min":0.26000000000000001,"15min":0.25700000000000001},"max":{"1min":17.120999999999999,"5min":39.323,"15min":110.914},"last":0.63800000000000001}]},{"osd":5,"last update":"Tue Sep 10 19:05:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.83099999999999996,"15min":0.96499999999999997},"min":{"1min":0.24299999999999999,"5min":0.221,"15min":0.20499999999999999},"max":{"1min":8.1430000000000007,"5min":8.1430000000000007,"15min":10.912000000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":1.05,"5min":0.89700000000000002,"15min":0.94099999999999995},"min":{"1min":0.251,"5min":0.23200000000000001,"15min":0.216},"max":{"1min":8.3170000000000002,"5min":8.3170000000000002,"15min":10.147},"last":0.38700000000000001}]},{"osd":6,"last update":"Tue Sep 10 19:06:23 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.74199999999999999,"15min":0.81399999999999995},"min":{"1min":0.33900000000000002,"5min":0.24099999999999999,"15min":0.221},"max":{"1min":2.3740000000000001,"5min":5.351,"15min":8.4030000000000005},"last":0.34499999999999997},{"interface":"front","average":{"1min":1.0680000000000001,"5min":0.95899999999999996,"15min":1.026},"min":{"1min":0.29499999999999998,"5min":0.26300000000000001,"15min":0.23899999999999999},"max":{"1min":4.0099999999999998,"5min":4.8949999999999996,"15min":21.254000000000001},"last":0.36099999999999999}]},{"osd":7,"last update":"Tue Sep 10 19:06:56 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.36799999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":3.472,"5min":3.472,"15min":3.472},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.89600000000000002,"5min":8.0640000000000001,"15min":9.2590000000000003},"min":{"1min":0.34899999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":4.1479999999999997,"5min":190.33799999999999,"15min":190.33799999999999},"last":0.49399999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182685,"num_pgs":81,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1612788,"kb_used_data":115688,"kb_used_omap":1,"kb_used_meta":1497086,"kb_avail":92132364,"statfs":{"total":95995035648,"available":94343540736,"internally_reserved":0,"allocated":118464512,"data_stored":118061791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":1533016408},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:07:01 2024","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.2230000000000001,"15min":2.4020000000000001},"min":{"1min":0.182,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":6.3099999999999996,"5min":6.3099999999999996,"15min":290.721},"last":0.311},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.393,"15min":2.5739999999999998},"min":{"1min":0.27800000000000002,"5min":0.18099999999999999,"15min":0.17499999999999999},"max":{"1min":3.9889999999999999,"5min":12.103,"15min":293.59300000000002},"last":0.42199999999999999}]},{"osd":1,"last update":"Tue Sep 10 19:06:12 2024","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.419,"15min":1.619},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.16200000000000001},"max":{"1min":6.0330000000000004,"5min":16.081,"15min":58.215000000000003},"last":0.44600000000000001},{"interface":"front","average":{"1min":1.915,"5min":1.6100000000000001,"15min":1.581},"min":{"1min":0.374,"5min":0.23599999999999999,"15min":0.188},"max":{"1min":7.165,"5min":16.329000000000001,"15min":57.896999999999998},"last":0.33700000000000002}]},{"osd":2,"last update":"Tue Sep 10 19:06:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":0.98099999999999998,"15min":0.96599999999999997},"min":{"1min":0.27900000000000003,"5min":0.25700000000000001,"15min":0.22800000000000001},"max":{"1min":3.1629999999999998,"5min":4.2409999999999997,"15min":5.6680000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":1.0189999999999999,"15min":0.88900000000000001},"min":{"1min":0.35199999999999998,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":3.0910000000000002,"5min":5.46,"15min":5.46},"last":0.52000000000000002}]},{"osd":4,"last update":"Tue Sep 10 19:06:44 2024","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.4570000000000001,"15min":1.3839999999999999},"min":{"1min":0.41299999999999998,"5min":0.29699999999999999,"15min":0.247},"max":{"1min":3.7360000000000002,"5min":16.356000000000002,"15min":29.013000000000002},"last":0.59199999999999997},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.466,"15min":1.3759999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.26800000000000002},"max":{"1min":3.7599999999999998,"5min":17.289999999999999,"15min":28.826000000000001},"last":0.67300000000000004}]},{"osd":5,"last update":"Tue Sep 10 19:07:01 2024","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.2450000000000001,"15min":1.3109999999999999},"min":{"1min":0.32500000000000001,"5min":0.248,"15min":0.248},"max":{"1min":3.9620000000000002,"5min":5.1920000000000002,"15min":7.2300000000000004},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.83299999999999996,"5min":1.5629999999999999,"15min":1.3680000000000001},"min":{"1min":0.38900000000000001,"5min":0.308,"15min":0.255},"max":{"1min":2.1949999999999998,"5min":17.899999999999999,"15min":17.899999999999999},"last":0.60399999999999998}]},{"osd":6,"last update":"Tue Sep 10 19:06:21 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.296,"15min":1.298},"min":{"1min":0.34200000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.9530000000000001,"5min":5.8339999999999996,"15min":5.8339999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.236,"15min":1.2450000000000001},"min":{"1min":0.38400000000000001,"5min":0.33000000000000002,"15min":0.29399999999999998},"max":{"1min":4.0019999999999998,"5min":6.6390000000000002,"15min":6.6390000000000002},"last":0.62}]},{"osd":7,"last update":"Tue Sep 10 19:06:55 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.3300000000000001,"15min":1.3839999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":3.9950000000000001,"5min":4.8499999999999996,"15min":4.8499999999999996},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.878,"5min":1.9650000000000001,"15min":2.1459999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.988,"5min":13.045999999999999,"15min":13.045999999999999},"last":0.50700000000000001}]}]},{"osd":2,"up_from":0,"seq":77309411617,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834575141,"num_pgs":85,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1600332,"kb_used_data":96004,"kb_used_omap":1,"kb_used_meta":1504318,"kb_avail":92144820,"statfs":{"total":95995035648,"available":94356295680,"internally_reserved":0,"allocated":98308096,"data_stored":97878166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":1540421976},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:06:01 2024","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.3480000000000001,"15min":1.8260000000000001},"min":{"1min":0.26800000000000002,"5min":0.158,"15min":0.158},"max":{"1min":4.9829999999999997,"5min":25.370999999999999,"15min":111.014},"last":0.26500000000000001},{"interface":"front","average":{"1min":1.784,"5min":1.5209999999999999,"15min":1.8799999999999999},"min":{"1min":0.30599999999999999,"5min":0.217,"15min":0.19600000000000001},"max":{"1min":5.1029999999999998,"5min":27.809999999999999,"15min":111.072},"last":0.33600000000000002}]},{"osd":2,"last update":"Tue Sep 10 19:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.83099999999999996,"15min":0.80400000000000005},"min":{"1min":0.25600000000000001,"5min":0.222,"15min":0.186},"max":{"1min":3.089,"5min":4.835,"15min":4.835},"last":0.375},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.90000000000000002,"15min":0.82199999999999995},"min":{"1min":0.307,"5min":0.23999999999999999,"15min":0.215},"max":{"1min":3.4990000000000001,"5min":9.484,"15min":9.484},"last":0.40799999999999997}]},{"osd":3,"last update":"Tue Sep 10 19:06:13 2024","interfaces":[{"interface":"back","average":{"1min":2.1760000000000002,"5min":2.1859999999999999,"15min":2.3100000000000001},"min":{"1min":0.29399999999999998,"5min":0.218,"15min":0.185},"max":{"1min":6.5149999999999997,"5min":48.582999999999998,"15min":92.069000000000003},"last":0.442},{"interface":"front","average":{"1min":1.72,"5min":1.6359999999999999,"15min":1.917},"min":{"1min":0.29199999999999998,"5min":0.23100000000000001,"15min":0.19900000000000001},"max":{"1min":9.8409999999999993,"5min":51.847000000000001,"15min":88.658000000000001},"last":0.35799999999999998}]},{"osd":4,"last update":"Tue Sep 10 19:06:34 2024","interfaces":[{"interface":"back","average":{"1min":1.649,"5min":1.2450000000000001,"15min":1.2050000000000001},"min":{"1min":0.41299999999999998,"5min":0.28100000000000003,"15min":0.27800000000000002},"max":{"1min":7.2699999999999996,"5min":19.126999999999999,"15min":19.126999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":1.871,"5min":1.431,"15min":1.202},"min":{"1min":0.40699999999999997,"5min":0.26300000000000001,"15min":0.252},"max":{"1min":5.9340000000000002,"5min":20.152000000000001,"15min":20.152000000000001},"last":0.76400000000000001}]},{"osd":5,"last update":"Tue Sep 10 19:06:01 2024","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.2549999999999999,"15min":1.274},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":3.677,"5min":9.9369999999999994,"15min":10.334},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.97899999999999998,"15min":1.1499999999999999},"min":{"1min":0.34799999999999998,"5min":0.29199999999999998,"15min":0.28799999999999998},"max":{"1min":2.5950000000000002,"5min":4.3600000000000003,"15min":16.039999999999999},"last":0.60999999999999999}]},{"osd":6,"last update":"Tue Sep 10 19:06:13 2024","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.25,"15min":1.458},"min":{"1min":0.435,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":3.7210000000000001,"5min":9.9760000000000009,"15min":33.290999999999997},"last":0.871},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.286,"15min":1.4910000000000001},"min":{"1min":0.44500000000000001,"5min":0.32700000000000001,"15min":0.26600000000000001},"max":{"1min":7.7400000000000002,"5min":17.352,"15min":33.310000000000002},"last":0.79100000000000004}]},{"osd":7,"last update":"Tue Sep 10 19:06:53 2024","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.4199999999999999,"15min":1.4219999999999999},"min":{"1min":0.42899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":5.9889999999999999,"5min":5.9889999999999999,"15min":5.9889999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.7190000000000001,"15min":1.829},"min":{"1min":0.41599999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.0609999999999999,"5min":10.010999999999999,"15min":10.010999999999999},"last":0.80000000000000004}]}]},{"osd":0,"up_from":241,"seq":1035087118473,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":954784,"kb_used_data":29204,"kb_used_omap":4,"kb_used_meta":925563,"kb_avail":92790368,"statfs":{"total":95995035648,"available":95017336832,"internally_reserved":0,"allocated":29904896,"data_stored":29515327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":947777030},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 19:05:59 2024","interfaces":[{"interface":"back","average":{"1min":4.3029999999999999,"5min":1.6779999999999999,"15min":2.1160000000000001},"min":{"1min":0.187,"5min":0.184,"15min":0.17100000000000001},"max":{"1min":47.237000000000002,"5min":47.237000000000002,"15min":47.237000000000002},"last":0.35299999999999998},{"interface":"front","average":{"1min":4.117,"5min":1.5669999999999999,"15min":2.0649999999999999},"min":{"1min":0.224,"5min":0.193,"15min":0.193},"max":{"1min":47.198999999999998,"5min":47.198999999999998,"15min":47.198999999999998},"last":0.42799999999999999}]},{"osd":2,"last update":"Tue Sep 10 19:05:59 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.82999999999999996,"15min":0.92200000000000004},"min":{"1min":0.223,"5min":0.223,"15min":0.20699999999999999},"max":{"1min":6.0170000000000003,"5min":6.0170000000000003,"15min":13.305},"last":0.39500000000000002},{"interface":"front","average":{"1min":1.0980000000000001,"5min":0.85799999999999998,"15min":0.86199999999999999},"min":{"1min":0.23100000000000001,"5min":0.22900000000000001,"15min":0.22800000000000001},"max":{"1min":6.1909999999999998,"5min":9.5099999999999998,"15min":13.693},"last":0.33500000000000002}]},{"osd":3,"last update":"Tue Sep 10 19:05:59 2024","interfaces":[{"interface":"back","average":{"1min":1.5780000000000001,"5min":1.25,"15min":1.2210000000000001},"min":{"1min":0.34999999999999998,"5min":0.216,"15min":0.216},"max":{"1min":6.8380000000000001,"5min":6.8380000000000001,"15min":21.108000000000001},"last":0.378},{"interface":"front","average":{"1min":1.9330000000000001,"5min":1.3400000000000001,"15min":1.627},"min":{"1min":0.26100000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":8.0549999999999997,"5min":8.0549999999999997,"15min":11.629},"last":0.317}]},{"osd":4,"last update":"Tue Sep 10 19:05:59 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.111,"15min":0.96799999999999997},"min":{"1min":0.30499999999999999,"5min":0.26400000000000001,"15min":0.26000000000000001},"max":{"1min":5.9560000000000004,"5min":5.9560000000000004,"15min":13.161},"last":0.45300000000000001},{"interface":"front","average":{"1min":1.603,"5min":1.1100000000000001,"15min":1.0660000000000001},"min":{"1min":0.312,"5min":0.28100000000000003,"15min":0.27600000000000002},"max":{"1min":6.3719999999999999,"5min":6.3719999999999999,"15min":6.3719999999999999},"last":0.54200000000000004}]},{"osd":5,"last update":"Tue Sep 10 19:05:59 2024","interfaces":[{"interface":"back","average":{"1min":1.583,"5min":2.2610000000000001,"15min":1.5009999999999999},"min":{"1min":0.34000000000000002,"5min":0.311,"15min":0.31},"max":{"1min":6.5679999999999996,"5min":97.296999999999997,"15min":97.296999999999997},"last":0.48999999999999999},{"interface":"front","average":{"1min":1.1439999999999999,"5min":2.218,"15min":1.379},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":5.7939999999999996,"5min":99.221999999999994,"15min":99.221999999999994},"last":0.501}]},{"osd":6,"last update":"Tue Sep 10 19:05:59 2024","interfaces":[{"interface":"back","average":{"1min":9.8659999999999997,"5min":3.0609999999999999,"15min":1.724},"min":{"1min":0.35299999999999998,"5min":0.33600000000000002,"15min":0.312},"max":{"1min":216.21100000000001,"5min":216.21100000000001,"15min":216.21100000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":9.5069999999999997,"5min":3.1960000000000002,"15min":1.665},"min":{"1min":0.379,"5min":0.379,"15min":0.313},"max":{"1min":216.054,"5min":216.054,"15min":216.054},"last":0.52700000000000002}]},{"osd":7,"last update":"Tue Sep 10 19:06:53 2024","interfaces":[{"interface":"back","average":{"1min":2.0600000000000001,"5min":1.244,"15min":1.1080000000000001},"min":{"1min":0.378,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":20.417999999999999,"5min":20.417999999999999,"15min":20.417999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":1.7689999999999999,"5min":1.597,"15min":1.569},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":18.059000000000001,"5min":18.059000000000001,"15min":18.059000000000001},"last":0.51500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":2668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T19:07:04.800 INFO:tasks.thrashosds.thrasher:Setting 6.9 to [4, 1, 6] 2024-09-10T19:07:04.801 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '6.9', '4', '1', '6'] 2024-09-10T19:07:04.801 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd pg-upmap 6.9 4 1 6 2024-09-10T19:07:04.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:04 smithi018 ceph-mon[40196]: 5.1e scrub starts 2024-09-10T19:07:04.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:04 smithi018 ceph-mon[40196]: 5.1e scrub ok 2024-09-10T19:07:04.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:04 smithi018 ceph-mon[40196]: pgmap v1021: 152 pgs: 152 active+clean; 582 KiB data, 7.2 GiB used, 529 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T19:07:04.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:04 smithi018 ceph-mon[40196]: 2.14 scrub starts 2024-09-10T19:07:04.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:04 smithi018 ceph-mon[40196]: 2.14 scrub ok 2024-09-10T19:07:04.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:04 smithi018 ceph-mon[29336]: 5.1e scrub starts 2024-09-10T19:07:04.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:04 smithi018 ceph-mon[29336]: 5.1e scrub ok 2024-09-10T19:07:04.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:04 smithi018 ceph-mon[29336]: pgmap v1021: 152 pgs: 152 active+clean; 582 KiB data, 7.2 GiB used, 529 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T19:07:04.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:04 smithi018 ceph-mon[29336]: 2.14 scrub starts 2024-09-10T19:07:04.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:04 smithi018 ceph-mon[29336]: 2.14 scrub ok 2024-09-10T19:07:05.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:04 smithi103 ceph-mon[33163]: 5.1e scrub starts 2024-09-10T19:07:05.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:04 smithi103 ceph-mon[33163]: 5.1e scrub ok 2024-09-10T19:07:05.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:04 smithi103 ceph-mon[33163]: pgmap v1021: 152 pgs: 152 active+clean; 582 KiB data, 7.2 GiB used, 529 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T19:07:05.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:04 smithi103 ceph-mon[33163]: 2.14 scrub starts 2024-09-10T19:07:05.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:04 smithi103 ceph-mon[33163]: 2.14 scrub ok 2024-09-10T19:07:05.091 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:07:05.246 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:07:05.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:05 smithi103 ceph-mon[33163]: from='client.16023 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:07:05.887 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:05 smithi018 ceph-mon[29336]: from='client.16023 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:07:05.887 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:05 smithi018 ceph-mon[40196]: from='client.16023 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:07:06.759 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:06 smithi103 ceph-mon[33163]: pgmap v1022: 152 pgs: 152 active+clean; 582 KiB data, 7.2 GiB used, 529 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T19:07:06.762 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T19:07:06.762 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T19:07:06.908 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:06 smithi018 ceph-mon[40196]: pgmap v1022: 152 pgs: 152 active+clean; 582 KiB data, 7.2 GiB used, 529 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T19:07:06.909 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:06 smithi018 ceph-mon[29336]: pgmap v1022: 152 pgs: 152 active+clean; 582 KiB data, 7.2 GiB used, 529 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T19:07:07.392 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-10T19:07:07.642 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:07:07.665 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:07:07 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:07:07.666 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:07 smithi018 ceph-mon[29336]: 1.d scrub starts 2024-09-10T19:07:07.666 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:07 smithi018 ceph-mon[29336]: 1.d scrub ok 2024-09-10T19:07:07.666 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:07 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2499645982' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:07:07.666 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:07 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:07:07.669 INFO:teuthology.orchestra.run.smithi018.stderr:noscrub is unset 2024-09-10T19:07:07.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:07 smithi018 ceph-mon[40196]: 1.d scrub starts 2024-09-10T19:07:07.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:07 smithi018 ceph-mon[40196]: 1.d scrub ok 2024-09-10T19:07:07.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:07 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2499645982' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:07:07.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:07 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:07:08.000 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:07 smithi103 ceph-mon[33163]: 1.d scrub starts 2024-09-10T19:07:08.000 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:07 smithi103 ceph-mon[33163]: 1.d scrub ok 2024-09-10T19:07:08.000 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:07 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2499645982' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:07:08.000 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:07 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-10T19:07:08.675 INFO:teuthology.orchestra.run.smithi018.stderr:set 6.9 pg_upmap mapping to [4,1,6] 2024-09-10T19:07:08.694 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-10T19:07:08.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:08 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:08.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:08 smithi018 ceph-mon[40196]: 2.1e scrub starts 2024-09-10T19:07:08.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:08 smithi018 ceph-mon[40196]: 2.1e scrub ok 2024-09-10T19:07:08.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:08 smithi018 ceph-mon[40196]: pgmap v1023: 152 pgs: 152 active+clean; 582 KiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:07:08.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:08 smithi018 ceph-mon[40196]: 3.1 scrub starts 2024-09-10T19:07:08.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:08 smithi018 ceph-mon[40196]: 3.1 scrub ok 2024-09-10T19:07:08.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:08 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T19:07:08.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:08 smithi018 ceph-mon[40196]: osdmap e455: 8 total, 8 up, 6 in 2024-09-10T19:07:08.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:08 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1770726542' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "6.9", "id": [4, 1, 6]}]: dispatch 2024-09-10T19:07:08.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:08 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "6.9", "id": [4, 1, 6]}]: dispatch 2024-09-10T19:07:08.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:08 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:08.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:08 smithi018 ceph-mon[29336]: 2.1e scrub starts 2024-09-10T19:07:08.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:08 smithi018 ceph-mon[29336]: 2.1e scrub ok 2024-09-10T19:07:08.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:08 smithi018 ceph-mon[29336]: pgmap v1023: 152 pgs: 152 active+clean; 582 KiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:07:08.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:08 smithi018 ceph-mon[29336]: 3.1 scrub starts 2024-09-10T19:07:08.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:08 smithi018 ceph-mon[29336]: 3.1 scrub ok 2024-09-10T19:07:08.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:08 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T19:07:08.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:08 smithi018 ceph-mon[29336]: osdmap e455: 8 total, 8 up, 6 in 2024-09-10T19:07:08.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:08 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1770726542' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "6.9", "id": [4, 1, 6]}]: dispatch 2024-09-10T19:07:08.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:08 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "6.9", "id": [4, 1, 6]}]: dispatch 2024-09-10T19:07:08.964 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:08 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:08.964 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:08 smithi103 ceph-mon[33163]: 2.1e scrub starts 2024-09-10T19:07:08.964 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:08 smithi103 ceph-mon[33163]: 2.1e scrub ok 2024-09-10T19:07:08.964 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:08 smithi103 ceph-mon[33163]: pgmap v1023: 152 pgs: 152 active+clean; 582 KiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:07:08.964 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:08 smithi103 ceph-mon[33163]: 3.1 scrub starts 2024-09-10T19:07:08.964 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:08 smithi103 ceph-mon[33163]: 3.1 scrub ok 2024-09-10T19:07:08.964 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:08 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-10T19:07:08.964 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:08 smithi103 ceph-mon[33163]: osdmap e455: 8 total, 8 up, 6 in 2024-09-10T19:07:08.964 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:08 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1770726542' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "6.9", "id": [4, 1, 6]}]: dispatch 2024-09-10T19:07:08.965 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:08 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "6.9", "id": [4, 1, 6]}]: dispatch 2024-09-10T19:07:09.405 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:07:09.492 INFO:teuthology.orchestra.run.smithi103.stdout:ERROR: (22) Invalid argument 2024-09-10T19:07:09.492 INFO:teuthology.orchestra.run.smithi103.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-10T19:07:09.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:09 smithi103 ceph-mon[33163]: 5.1d scrub starts 2024-09-10T19:07:09.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:09 smithi103 ceph-mon[33163]: 5.1d scrub ok 2024-09-10T19:07:09.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:09 smithi103 ceph-mon[33163]: 4.8 scrub starts 2024-09-10T19:07:09.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:09 smithi103 ceph-mon[33163]: 4.8 scrub ok 2024-09-10T19:07:09.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:09 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "6.9", "id": [4, 1, 6]}]': finished 2024-09-10T19:07:09.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:09 smithi103 ceph-mon[33163]: osdmap e456: 8 total, 8 up, 6 in 2024-09-10T19:07:09.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:09 smithi018 ceph-mon[40196]: 5.1d scrub starts 2024-09-10T19:07:09.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:09 smithi018 ceph-mon[40196]: 5.1d scrub ok 2024-09-10T19:07:09.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:09 smithi018 ceph-mon[40196]: 4.8 scrub starts 2024-09-10T19:07:09.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:09 smithi018 ceph-mon[40196]: 4.8 scrub ok 2024-09-10T19:07:09.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:09 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "6.9", "id": [4, 1, 6]}]': finished 2024-09-10T19:07:09.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:09 smithi018 ceph-mon[40196]: osdmap e456: 8 total, 8 up, 6 in 2024-09-10T19:07:09.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:09 smithi018 ceph-mon[29336]: 5.1d scrub starts 2024-09-10T19:07:09.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:09 smithi018 ceph-mon[29336]: 5.1d scrub ok 2024-09-10T19:07:09.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:09 smithi018 ceph-mon[29336]: 4.8 scrub starts 2024-09-10T19:07:09.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:09 smithi018 ceph-mon[29336]: 4.8 scrub ok 2024-09-10T19:07:09.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:09 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "6.9", "id": [4, 1, 6]}]': finished 2024-09-10T19:07:09.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:09 smithi018 ceph-mon[29336]: osdmap e456: 8 total, 8 up, 6 in 2024-09-10T19:07:10.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:07:10 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:07:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:07:10.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:10 smithi018 ceph-mon[40196]: 4.19 scrub starts 2024-09-10T19:07:10.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:10 smithi018 ceph-mon[40196]: 4.19 scrub ok 2024-09-10T19:07:10.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:10 smithi018 ceph-mon[40196]: pgmap v1026: 152 pgs: 152 active+clean; 582 KiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:10.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:10 smithi018 ceph-mon[40196]: osdmap e457: 8 total, 8 up, 6 in 2024-09-10T19:07:10.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:10 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:07:10.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:10 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:07:10.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:10 smithi018 ceph-mon[29336]: 4.19 scrub starts 2024-09-10T19:07:10.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:10 smithi018 ceph-mon[29336]: 4.19 scrub ok 2024-09-10T19:07:10.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:10 smithi018 ceph-mon[29336]: pgmap v1026: 152 pgs: 152 active+clean; 582 KiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:10.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:10 smithi018 ceph-mon[29336]: osdmap e457: 8 total, 8 up, 6 in 2024-09-10T19:07:10.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:10 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:07:10.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:10 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:07:11.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:10 smithi103 ceph-mon[33163]: 4.19 scrub starts 2024-09-10T19:07:11.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:10 smithi103 ceph-mon[33163]: 4.19 scrub ok 2024-09-10T19:07:11.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:10 smithi103 ceph-mon[33163]: pgmap v1026: 152 pgs: 152 active+clean; 582 KiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:11.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:10 smithi103 ceph-mon[33163]: osdmap e457: 8 total, 8 up, 6 in 2024-09-10T19:07:11.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:10 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:07:11.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:10 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:07:11.445 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:07:10 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:07:10] "GET /metrics HTTP/1.1" 200 221991 "" "Prometheus/2.43.0" 2024-09-10T19:07:11.687 INFO:teuthology.orchestra.run.smithi018.stderr:nodeep-scrub is unset 2024-09-10T19:07:11.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:11 smithi018 ceph-mon[29336]: 6.10 scrub starts 2024-09-10T19:07:11.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:11 smithi018 ceph-mon[29336]: 6.10 scrub ok 2024-09-10T19:07:11.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:11 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2979129097' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:07:11.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:11 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:07:11.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:11 smithi018 ceph-mon[29336]: 5.1f scrub starts 2024-09-10T19:07:11.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:11 smithi018 ceph-mon[29336]: 5.1f scrub ok 2024-09-10T19:07:11.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:11 smithi018 ceph-mon[29336]: pgmap v1028: 152 pgs: 152 active+clean; 582 KiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:11.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:11 smithi018 ceph-mon[40196]: 6.10 scrub starts 2024-09-10T19:07:11.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:11 smithi018 ceph-mon[40196]: 6.10 scrub ok 2024-09-10T19:07:11.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:11 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2979129097' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:07:11.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:11 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:07:11.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:11 smithi018 ceph-mon[40196]: 5.1f scrub starts 2024-09-10T19:07:11.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:11 smithi018 ceph-mon[40196]: 5.1f scrub ok 2024-09-10T19:07:11.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:11 smithi018 ceph-mon[40196]: pgmap v1028: 152 pgs: 152 active+clean; 582 KiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:12.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:11 smithi103 ceph-mon[33163]: 6.10 scrub starts 2024-09-10T19:07:12.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:11 smithi103 ceph-mon[33163]: 6.10 scrub ok 2024-09-10T19:07:12.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:11 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2979129097' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:07:12.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:11 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-10T19:07:12.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:11 smithi103 ceph-mon[33163]: 5.1f scrub starts 2024-09-10T19:07:12.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:11 smithi103 ceph-mon[33163]: 5.1f scrub ok 2024-09-10T19:07:12.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:11 smithi103 ceph-mon[33163]: pgmap v1028: 152 pgs: 152 active+clean; 582 KiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:12.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:12 smithi018 ceph-mon[29336]: 1.a scrub starts 2024-09-10T19:07:12.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:12 smithi018 ceph-mon[29336]: 1.a scrub ok 2024-09-10T19:07:12.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:12 smithi018 ceph-mon[29336]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T19:07:12.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:12 smithi018 ceph-mon[29336]: Cluster is now healthy 2024-09-10T19:07:12.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:12 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T19:07:12.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:12 smithi018 ceph-mon[29336]: osdmap e458: 8 total, 8 up, 6 in 2024-09-10T19:07:12.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:12 smithi018 ceph-mon[29336]: 2.1d scrub starts 2024-09-10T19:07:12.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:12 smithi018 ceph-mon[29336]: 2.1d scrub ok 2024-09-10T19:07:12.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:12 smithi018 ceph-mon[40196]: 1.a scrub starts 2024-09-10T19:07:12.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:12 smithi018 ceph-mon[40196]: 1.a scrub ok 2024-09-10T19:07:12.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:12 smithi018 ceph-mon[40196]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T19:07:12.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:12 smithi018 ceph-mon[40196]: Cluster is now healthy 2024-09-10T19:07:12.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:12 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T19:07:12.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:12 smithi018 ceph-mon[40196]: osdmap e458: 8 total, 8 up, 6 in 2024-09-10T19:07:12.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:12 smithi018 ceph-mon[40196]: 2.1d scrub starts 2024-09-10T19:07:12.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:12 smithi018 ceph-mon[40196]: 2.1d scrub ok 2024-09-10T19:07:13.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:12 smithi103 ceph-mon[33163]: 1.a scrub starts 2024-09-10T19:07:13.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:12 smithi103 ceph-mon[33163]: 1.a scrub ok 2024-09-10T19:07:13.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:12 smithi103 ceph-mon[33163]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-10T19:07:13.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:12 smithi103 ceph-mon[33163]: Cluster is now healthy 2024-09-10T19:07:13.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:12 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-10T19:07:13.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:12 smithi103 ceph-mon[33163]: osdmap e458: 8 total, 8 up, 6 in 2024-09-10T19:07:13.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:12 smithi103 ceph-mon[33163]: 2.1d scrub starts 2024-09-10T19:07:13.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:12 smithi103 ceph-mon[33163]: 2.1d scrub ok 2024-09-10T19:07:13.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:13 smithi018 ceph-mon[29336]: 5.e scrub starts 2024-09-10T19:07:13.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:13 smithi018 ceph-mon[29336]: 5.e scrub ok 2024-09-10T19:07:13.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:13 smithi018 ceph-mon[29336]: 6.6 deep-scrub starts 2024-09-10T19:07:13.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:13 smithi018 ceph-mon[29336]: 6.6 deep-scrub ok 2024-09-10T19:07:13.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:13 smithi018 ceph-mon[29336]: pgmap v1030: 152 pgs: 152 active+clean; 582 KiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 873 B/s rd, 0 op/s 2024-09-10T19:07:13.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:13 smithi018 ceph-mon[40196]: 5.e scrub starts 2024-09-10T19:07:13.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:13 smithi018 ceph-mon[40196]: 5.e scrub ok 2024-09-10T19:07:13.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:13 smithi018 ceph-mon[40196]: 6.6 deep-scrub starts 2024-09-10T19:07:13.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:13 smithi018 ceph-mon[40196]: 6.6 deep-scrub ok 2024-09-10T19:07:13.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:13 smithi018 ceph-mon[40196]: pgmap v1030: 152 pgs: 152 active+clean; 582 KiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 873 B/s rd, 0 op/s 2024-09-10T19:07:14.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:13 smithi103 ceph-mon[33163]: 5.e scrub starts 2024-09-10T19:07:14.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:13 smithi103 ceph-mon[33163]: 5.e scrub ok 2024-09-10T19:07:14.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:13 smithi103 ceph-mon[33163]: 6.6 deep-scrub starts 2024-09-10T19:07:14.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:13 smithi103 ceph-mon[33163]: 6.6 deep-scrub ok 2024-09-10T19:07:14.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:13 smithi103 ceph-mon[33163]: pgmap v1030: 152 pgs: 152 active+clean; 582 KiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 873 B/s rd, 0 op/s 2024-09-10T19:07:14.395 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-10T19:07:14.396 INFO:tasks.thrashosds.thrasher:Adding osd 7 2024-09-10T19:07:14.396 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd in 7 2024-09-10T19:07:14.669 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:07:14.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:14 smithi018 ceph-mon[29336]: 6.4 scrub starts 2024-09-10T19:07:14.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:14 smithi018 ceph-mon[29336]: 6.4 scrub ok 2024-09-10T19:07:14.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:14 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:07:14.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:14 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:07:14.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:14 smithi018 ceph-mon[29336]: osdmap e459: 8 total, 8 up, 6 in 2024-09-10T19:07:14.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:14 smithi018 ceph-mon[40196]: 6.4 scrub starts 2024-09-10T19:07:14.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:14 smithi018 ceph-mon[40196]: 6.4 scrub ok 2024-09-10T19:07:14.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:14 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:07:14.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:14 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:07:14.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:14 smithi018 ceph-mon[40196]: osdmap e459: 8 total, 8 up, 6 in 2024-09-10T19:07:15.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:14 smithi103 ceph-mon[33163]: 6.4 scrub starts 2024-09-10T19:07:15.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:14 smithi103 ceph-mon[33163]: 6.4 scrub ok 2024-09-10T19:07:15.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:14 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:07:15.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:14 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:07:15.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:14 smithi103 ceph-mon[33163]: osdmap e459: 8 total, 8 up, 6 in 2024-09-10T19:07:15.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:15 smithi018 ceph-mon[29336]: 1.5 scrub starts 2024-09-10T19:07:15.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:15 smithi018 ceph-mon[29336]: 1.5 scrub ok 2024-09-10T19:07:15.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:15 smithi018 ceph-mon[29336]: pgmap v1032: 152 pgs: 152 active+clean; 582 KiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:15.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:15 smithi018 ceph-mon[40196]: 1.5 scrub starts 2024-09-10T19:07:15.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:15 smithi018 ceph-mon[40196]: 1.5 scrub ok 2024-09-10T19:07:15.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:15 smithi018 ceph-mon[40196]: pgmap v1032: 152 pgs: 152 active+clean; 582 KiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:16.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:15 smithi103 ceph-mon[33163]: 1.5 scrub starts 2024-09-10T19:07:16.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:15 smithi103 ceph-mon[33163]: 1.5 scrub ok 2024-09-10T19:07:16.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:15 smithi103 ceph-mon[33163]: pgmap v1032: 152 pgs: 152 active+clean; 582 KiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:16.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:16 smithi018 ceph-mon[40196]: 6.1d scrub starts 2024-09-10T19:07:16.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:16 smithi018 ceph-mon[40196]: 6.1d scrub ok 2024-09-10T19:07:16.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:16 smithi018 ceph-mon[40196]: 4.4 scrub starts 2024-09-10T19:07:16.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:16 smithi018 ceph-mon[40196]: 4.4 scrub ok 2024-09-10T19:07:16.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:16 smithi018 ceph-mon[29336]: 6.1d scrub starts 2024-09-10T19:07:16.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:16 smithi018 ceph-mon[29336]: 6.1d scrub ok 2024-09-10T19:07:16.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:16 smithi018 ceph-mon[29336]: 4.4 scrub starts 2024-09-10T19:07:16.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:16 smithi018 ceph-mon[29336]: 4.4 scrub ok 2024-09-10T19:07:16.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:16 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mon-a[29332]: 2024-09-10T19:07:16.727+0000 7f4eb0f2a640 -1 mon.a@0(leader).osd e460 definitely_dead 0 2024-09-10T19:07:17.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:16 smithi103 ceph-mon[33163]: 6.1d scrub starts 2024-09-10T19:07:17.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:16 smithi103 ceph-mon[33163]: 6.1d scrub ok 2024-09-10T19:07:17.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:16 smithi103 ceph-mon[33163]: 4.4 scrub starts 2024-09-10T19:07:17.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:16 smithi103 ceph-mon[33163]: 4.4 scrub ok 2024-09-10T19:07:17.698 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:07:17 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:07:17.716 INFO:teuthology.orchestra.run.smithi018.stderr:marked in osd.7. 2024-09-10T19:07:18.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:17 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:07:18.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:17 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:07:18.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:17 smithi103 ceph-mon[33163]: 5.a scrub starts 2024-09-10T19:07:18.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:17 smithi103 ceph-mon[33163]: 5.a scrub ok 2024-09-10T19:07:18.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:17 smithi103 ceph-mon[33163]: osdmap e460: 8 total, 8 up, 6 in 2024-09-10T19:07:18.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:17 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/4061214991' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["7"]}]: dispatch 2024-09-10T19:07:18.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:17 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["7"]}]: dispatch 2024-09-10T19:07:18.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:17 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:18.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:17 smithi103 ceph-mon[33163]: pgmap v1034: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T19:07:18.088 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:17 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:07:18.088 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:17 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:07:18.088 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:17 smithi018 ceph-mon[40196]: 5.a scrub starts 2024-09-10T19:07:18.088 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:17 smithi018 ceph-mon[40196]: 5.a scrub ok 2024-09-10T19:07:18.088 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:17 smithi018 ceph-mon[40196]: osdmap e460: 8 total, 8 up, 6 in 2024-09-10T19:07:18.088 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:17 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/4061214991' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["7"]}]: dispatch 2024-09-10T19:07:18.088 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:17 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["7"]}]: dispatch 2024-09-10T19:07:18.088 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:17 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:18.088 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:17 smithi018 ceph-mon[40196]: pgmap v1034: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T19:07:18.089 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:17 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:07:18.089 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:17 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:07:18.089 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:17 smithi018 ceph-mon[29336]: 5.a scrub starts 2024-09-10T19:07:18.089 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:17 smithi018 ceph-mon[29336]: 5.a scrub ok 2024-09-10T19:07:18.089 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:17 smithi018 ceph-mon[29336]: osdmap e460: 8 total, 8 up, 6 in 2024-09-10T19:07:18.089 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:17 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/4061214991' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["7"]}]: dispatch 2024-09-10T19:07:18.089 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:17 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["7"]}]: dispatch 2024-09-10T19:07:18.089 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:17 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:18.089 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:17 smithi018 ceph-mon[29336]: pgmap v1034: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T19:07:18.503 INFO:tasks.thrashosds.thrasher:Added osd 7 2024-09-10T19:07:18.504 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-10T19:07:18.504 INFO:tasks.thrashosds.thrasher:Adding osd 2 2024-09-10T19:07:18.504 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd in 2 2024-09-10T19:07:18.790 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:07:18.911 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:18 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["7"]}]': finished 2024-09-10T19:07:18.912 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:18 smithi018 ceph-mon[29336]: osdmap e461: 8 total, 8 up, 7 in 2024-09-10T19:07:18.912 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:18 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["7"]}]': finished 2024-09-10T19:07:18.912 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:18 smithi018 ceph-mon[40196]: osdmap e461: 8 total, 8 up, 7 in 2024-09-10T19:07:19.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:18 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["7"]}]': finished 2024-09-10T19:07:19.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:18 smithi103 ceph-mon[33163]: osdmap e461: 8 total, 8 up, 7 in 2024-09-10T19:07:19.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:19 smithi018 ceph-mon[29336]: 5.13 scrub starts 2024-09-10T19:07:19.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:19 smithi018 ceph-mon[29336]: 5.13 scrub ok 2024-09-10T19:07:19.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:19 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:07:19.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:19 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:07:19.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:19 smithi018 ceph-mon[29336]: osdmap e462: 8 total, 8 up, 7 in 2024-09-10T19:07:19.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:19 smithi018 ceph-mon[29336]: osdmap e463: 8 total, 8 up, 7 in 2024-09-10T19:07:19.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:19 smithi018 ceph-mon[29336]: pgmap v1038: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 109 B/s, 5 objects/s recovering 2024-09-10T19:07:19.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:19 smithi018 ceph-mon[40196]: 5.13 scrub starts 2024-09-10T19:07:19.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:19 smithi018 ceph-mon[40196]: 5.13 scrub ok 2024-09-10T19:07:19.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:19 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:07:19.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:19 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:07:19.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:19 smithi018 ceph-mon[40196]: osdmap e462: 8 total, 8 up, 7 in 2024-09-10T19:07:19.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:19 smithi018 ceph-mon[40196]: osdmap e463: 8 total, 8 up, 7 in 2024-09-10T19:07:19.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:19 smithi018 ceph-mon[40196]: pgmap v1038: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 109 B/s, 5 objects/s recovering 2024-09-10T19:07:20.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:19 smithi103 ceph-mon[33163]: 5.13 scrub starts 2024-09-10T19:07:20.072 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:19 smithi103 ceph-mon[33163]: 5.13 scrub ok 2024-09-10T19:07:20.073 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:19 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:07:20.073 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:19 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:07:20.073 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:19 smithi103 ceph-mon[33163]: osdmap e462: 8 total, 8 up, 7 in 2024-09-10T19:07:20.073 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:19 smithi103 ceph-mon[33163]: osdmap e463: 8 total, 8 up, 7 in 2024-09-10T19:07:20.073 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:19 smithi103 ceph-mon[33163]: pgmap v1038: 152 pgs: 152 active+clean; 582 KiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 109 B/s, 5 objects/s recovering 2024-09-10T19:07:20.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:07:20 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:07:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:07:20.843 INFO:teuthology.orchestra.run.smithi018.stderr:marked in osd.2. 2024-09-10T19:07:20.844 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:20 smithi018 ceph-mon[40196]: 6.17 scrub starts 2024-09-10T19:07:20.844 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:20 smithi018 ceph-mon[40196]: 6.17 scrub ok 2024-09-10T19:07:20.844 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:20 smithi018 ceph-mon[40196]: 6.8 scrub starts 2024-09-10T19:07:20.844 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:20 smithi018 ceph-mon[40196]: 6.8 scrub ok 2024-09-10T19:07:20.844 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:20 smithi018 ceph-mon[40196]: 2.1f deep-scrub starts 2024-09-10T19:07:20.844 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:20 smithi018 ceph-mon[40196]: 2.1f deep-scrub ok 2024-09-10T19:07:20.844 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:20 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1453392948' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["2"]}]: dispatch 2024-09-10T19:07:20.844 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:20 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mon-a[29332]: 2024-09-10T19:07:20.702+0000 7f4eb0f2a640 -1 mon.a@0(leader).osd e463 definitely_dead 0 2024-09-10T19:07:20.844 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:20 smithi018 ceph-mon[29336]: 6.17 scrub starts 2024-09-10T19:07:20.844 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:20 smithi018 ceph-mon[29336]: 6.17 scrub ok 2024-09-10T19:07:20.845 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:20 smithi018 ceph-mon[29336]: 6.8 scrub starts 2024-09-10T19:07:20.845 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:20 smithi018 ceph-mon[29336]: 6.8 scrub ok 2024-09-10T19:07:20.845 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:20 smithi018 ceph-mon[29336]: 2.1f deep-scrub starts 2024-09-10T19:07:20.845 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:20 smithi018 ceph-mon[29336]: 2.1f deep-scrub ok 2024-09-10T19:07:20.845 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:20 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1453392948' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["2"]}]: dispatch 2024-09-10T19:07:21.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:20 smithi103 ceph-mon[33163]: 6.17 scrub starts 2024-09-10T19:07:21.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:20 smithi103 ceph-mon[33163]: 6.17 scrub ok 2024-09-10T19:07:21.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:20 smithi103 ceph-mon[33163]: 6.8 scrub starts 2024-09-10T19:07:21.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:20 smithi103 ceph-mon[33163]: 6.8 scrub ok 2024-09-10T19:07:21.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:20 smithi103 ceph-mon[33163]: 2.1f deep-scrub starts 2024-09-10T19:07:21.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:20 smithi103 ceph-mon[33163]: 2.1f deep-scrub ok 2024-09-10T19:07:21.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:20 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1453392948' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["2"]}]: dispatch 2024-09-10T19:07:21.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:07:20 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:07:20] "GET /metrics HTTP/1.1" 200 221991 "" "Prometheus/2.43.0" 2024-09-10T19:07:21.574 INFO:tasks.thrashosds.thrasher:Added osd 2 2024-09-10T19:07:21.574 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-10T19:07:21.833 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:07:21.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:21 smithi018 ceph-mon[29336]: 4.1 scrub starts 2024-09-10T19:07:21.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:21 smithi018 ceph-mon[29336]: 4.1 scrub ok 2024-09-10T19:07:21.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:21 smithi018 ceph-mon[29336]: 6.5 scrub starts 2024-09-10T19:07:21.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:21 smithi018 ceph-mon[29336]: 6.5 scrub ok 2024-09-10T19:07:21.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:21 smithi018 ceph-mon[29336]: 1.c scrub starts 2024-09-10T19:07:21.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:21 smithi018 ceph-mon[29336]: 1.c scrub ok 2024-09-10T19:07:21.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:21 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1453392948' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["2"]}]': finished 2024-09-10T19:07:21.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:21 smithi018 ceph-mon[29336]: osdmap e464: 8 total, 8 up, 8 in 2024-09-10T19:07:21.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:21 smithi018 ceph-mon[29336]: pgmap v1040: 152 pgs: 30 peering, 122 active+clean; 581 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 85 B/s, 4 objects/s recovering 2024-09-10T19:07:21.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:21 smithi018 ceph-mon[40196]: 4.1 scrub starts 2024-09-10T19:07:21.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:21 smithi018 ceph-mon[40196]: 4.1 scrub ok 2024-09-10T19:07:21.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:21 smithi018 ceph-mon[40196]: 6.5 scrub starts 2024-09-10T19:07:21.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:21 smithi018 ceph-mon[40196]: 6.5 scrub ok 2024-09-10T19:07:21.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:21 smithi018 ceph-mon[40196]: 1.c scrub starts 2024-09-10T19:07:21.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:21 smithi018 ceph-mon[40196]: 1.c scrub ok 2024-09-10T19:07:21.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:21 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1453392948' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["2"]}]': finished 2024-09-10T19:07:21.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:21 smithi018 ceph-mon[40196]: osdmap e464: 8 total, 8 up, 8 in 2024-09-10T19:07:21.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:21 smithi018 ceph-mon[40196]: pgmap v1040: 152 pgs: 30 peering, 122 active+clean; 581 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 85 B/s, 4 objects/s recovering 2024-09-10T19:07:22.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:21 smithi103 ceph-mon[33163]: 4.1 scrub starts 2024-09-10T19:07:22.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:21 smithi103 ceph-mon[33163]: 4.1 scrub ok 2024-09-10T19:07:22.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:21 smithi103 ceph-mon[33163]: 6.5 scrub starts 2024-09-10T19:07:22.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:21 smithi103 ceph-mon[33163]: 6.5 scrub ok 2024-09-10T19:07:22.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:21 smithi103 ceph-mon[33163]: 1.c scrub starts 2024-09-10T19:07:22.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:21 smithi103 ceph-mon[33163]: 1.c scrub ok 2024-09-10T19:07:22.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:21 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1453392948' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["2"]}]': finished 2024-09-10T19:07:22.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:21 smithi103 ceph-mon[33163]: osdmap e464: 8 total, 8 up, 8 in 2024-09-10T19:07:22.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:21 smithi103 ceph-mon[33163]: pgmap v1040: 152 pgs: 30 peering, 122 active+clean; 581 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 85 B/s, 4 objects/s recovering 2024-09-10T19:07:22.997 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:22 smithi018 ceph-mon[40196]: 4.c scrub starts 2024-09-10T19:07:22.997 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:22 smithi018 ceph-mon[40196]: 4.c scrub ok 2024-09-10T19:07:22.997 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:22 smithi018 ceph-mon[40196]: 5.12 scrub starts 2024-09-10T19:07:22.997 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:22 smithi018 ceph-mon[40196]: 5.12 scrub ok 2024-09-10T19:07:22.997 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:22 smithi018 ceph-mon[40196]: 4.a scrub starts 2024-09-10T19:07:22.998 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:22 smithi018 ceph-mon[40196]: 4.a scrub ok 2024-09-10T19:07:22.998 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:22 smithi018 ceph-mon[40196]: osdmap e465: 8 total, 8 up, 8 in 2024-09-10T19:07:22.998 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:22 smithi018 ceph-mon[40196]: 2.10 deep-scrub starts 2024-09-10T19:07:22.998 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:22 smithi018 ceph-mon[40196]: 2.10 deep-scrub ok 2024-09-10T19:07:22.998 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:22 smithi018 ceph-mon[29336]: 4.c scrub starts 2024-09-10T19:07:22.998 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:22 smithi018 ceph-mon[29336]: 4.c scrub ok 2024-09-10T19:07:22.998 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:22 smithi018 ceph-mon[29336]: 5.12 scrub starts 2024-09-10T19:07:22.998 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:22 smithi018 ceph-mon[29336]: 5.12 scrub ok 2024-09-10T19:07:22.998 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:22 smithi018 ceph-mon[29336]: 4.a scrub starts 2024-09-10T19:07:22.998 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:22 smithi018 ceph-mon[29336]: 4.a scrub ok 2024-09-10T19:07:22.998 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:22 smithi018 ceph-mon[29336]: osdmap e465: 8 total, 8 up, 8 in 2024-09-10T19:07:22.999 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:22 smithi018 ceph-mon[29336]: 2.10 deep-scrub starts 2024-09-10T19:07:22.999 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:22 smithi018 ceph-mon[29336]: 2.10 deep-scrub ok 2024-09-10T19:07:23.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:22 smithi103 ceph-mon[33163]: 4.c scrub starts 2024-09-10T19:07:23.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:22 smithi103 ceph-mon[33163]: 4.c scrub ok 2024-09-10T19:07:23.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:22 smithi103 ceph-mon[33163]: 5.12 scrub starts 2024-09-10T19:07:23.024 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:22 smithi103 ceph-mon[33163]: 5.12 scrub ok 2024-09-10T19:07:23.024 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:22 smithi103 ceph-mon[33163]: 4.a scrub starts 2024-09-10T19:07:23.024 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:22 smithi103 ceph-mon[33163]: 4.a scrub ok 2024-09-10T19:07:23.024 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:22 smithi103 ceph-mon[33163]: osdmap e465: 8 total, 8 up, 8 in 2024-09-10T19:07:23.024 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:22 smithi103 ceph-mon[33163]: 2.10 deep-scrub starts 2024-09-10T19:07:23.024 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:22 smithi103 ceph-mon[33163]: 2.10 deep-scrub ok 2024-09-10T19:07:23.568 INFO:teuthology.orchestra.run.smithi018.stdout:mon.a: {} 2024-09-10T19:07:23.568 INFO:teuthology.orchestra.run.smithi018.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-10T19:07:23.576 INFO:teuthology.orchestra.run.smithi018.stdout:mon.b: {} 2024-09-10T19:07:23.577 INFO:teuthology.orchestra.run.smithi018.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-10T19:07:23.585 INFO:teuthology.orchestra.run.smithi018.stdout:mon.c: {} 2024-09-10T19:07:23.585 INFO:teuthology.orchestra.run.smithi018.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-10T19:07:23.930 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:23 smithi018 ceph-mon[29336]: 4.12 scrub starts 2024-09-10T19:07:23.930 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:23 smithi018 ceph-mon[29336]: 4.12 scrub ok 2024-09-10T19:07:23.930 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:23 smithi018 ceph-mon[29336]: pgmap v1042: 152 pgs: 39 peering, 113 active+clean; 581 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 352 B/s, 9 objects/s recovering 2024-09-10T19:07:23.930 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:23 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3670295765' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T19:07:23.930 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:23 smithi018 ceph-mon[40196]: 4.12 scrub starts 2024-09-10T19:07:23.930 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:23 smithi018 ceph-mon[40196]: 4.12 scrub ok 2024-09-10T19:07:23.930 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:23 smithi018 ceph-mon[40196]: pgmap v1042: 152 pgs: 39 peering, 113 active+clean; 581 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 352 B/s, 9 objects/s recovering 2024-09-10T19:07:23.930 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:23 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3670295765' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T19:07:24.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:23 smithi103 ceph-mon[33163]: 4.12 scrub starts 2024-09-10T19:07:24.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:23 smithi103 ceph-mon[33163]: 4.12 scrub ok 2024-09-10T19:07:24.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:23 smithi103 ceph-mon[33163]: pgmap v1042: 152 pgs: 39 peering, 113 active+clean; 581 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 352 B/s, 9 objects/s recovering 2024-09-10T19:07:24.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:23 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3670295765' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T19:07:24.335 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd reweight 1 1 2024-09-10T19:07:24.599 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:07:25.017 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:24 smithi018 ceph-mon[40196]: osdmap e466: 8 total, 8 up, 8 in 2024-09-10T19:07:25.017 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:24 smithi018 ceph-mon[40196]: 6.1e deep-scrub starts 2024-09-10T19:07:25.017 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:24 smithi018 ceph-mon[40196]: 6.1e deep-scrub ok 2024-09-10T19:07:25.017 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:24 smithi018 ceph-mon[29336]: osdmap e466: 8 total, 8 up, 8 in 2024-09-10T19:07:25.017 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:24 smithi018 ceph-mon[29336]: 6.1e deep-scrub starts 2024-09-10T19:07:25.018 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:24 smithi018 ceph-mon[29336]: 6.1e deep-scrub ok 2024-09-10T19:07:25.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:24 smithi103 ceph-mon[33163]: osdmap e466: 8 total, 8 up, 8 in 2024-09-10T19:07:25.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:24 smithi103 ceph-mon[33163]: 6.1e deep-scrub starts 2024-09-10T19:07:25.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:24 smithi103 ceph-mon[33163]: 6.1e deep-scrub ok 2024-09-10T19:07:26.076 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:25 smithi018 ceph-mon[29336]: 4.10 scrub starts 2024-09-10T19:07:26.076 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:25 smithi018 ceph-mon[29336]: 4.10 scrub ok 2024-09-10T19:07:26.077 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:25 smithi018 ceph-mon[29336]: 4.6 scrub starts 2024-09-10T19:07:26.077 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:25 smithi018 ceph-mon[29336]: 4.6 scrub ok 2024-09-10T19:07:26.077 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:25 smithi018 ceph-mon[29336]: 2.7 deep-scrub starts 2024-09-10T19:07:26.077 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:25 smithi018 ceph-mon[29336]: 2.7 deep-scrub ok 2024-09-10T19:07:26.077 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:25 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:07:26.077 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:25 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:07:26.077 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:25 smithi018 ceph-mon[29336]: 5.b scrub starts 2024-09-10T19:07:26.077 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:25 smithi018 ceph-mon[29336]: 5.b scrub ok 2024-09-10T19:07:26.077 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:25 smithi018 ceph-mon[29336]: 2.16 scrub starts 2024-09-10T19:07:26.077 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:25 smithi018 ceph-mon[29336]: 2.16 scrub ok 2024-09-10T19:07:26.077 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:25 smithi018 ceph-mon[29336]: pgmap v1044: 152 pgs: 39 peering, 113 active+clean; 581 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 209 B/s, 4 objects/s recovering 2024-09-10T19:07:26.077 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:25 smithi018 ceph-mon[40196]: 4.10 scrub starts 2024-09-10T19:07:26.078 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:25 smithi018 ceph-mon[40196]: 4.10 scrub ok 2024-09-10T19:07:26.078 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:25 smithi018 ceph-mon[40196]: 4.6 scrub starts 2024-09-10T19:07:26.078 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:25 smithi018 ceph-mon[40196]: 4.6 scrub ok 2024-09-10T19:07:26.078 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:25 smithi018 ceph-mon[40196]: 2.7 deep-scrub starts 2024-09-10T19:07:26.078 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:25 smithi018 ceph-mon[40196]: 2.7 deep-scrub ok 2024-09-10T19:07:26.078 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:25 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:07:26.078 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:25 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:07:26.078 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:25 smithi018 ceph-mon[40196]: 5.b scrub starts 2024-09-10T19:07:26.078 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:25 smithi018 ceph-mon[40196]: 5.b scrub ok 2024-09-10T19:07:26.078 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:25 smithi018 ceph-mon[40196]: 2.16 scrub starts 2024-09-10T19:07:26.078 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:25 smithi018 ceph-mon[40196]: 2.16 scrub ok 2024-09-10T19:07:26.078 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:25 smithi018 ceph-mon[40196]: pgmap v1044: 152 pgs: 39 peering, 113 active+clean; 581 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 209 B/s, 4 objects/s recovering 2024-09-10T19:07:26.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:25 smithi103 ceph-mon[33163]: 4.10 scrub starts 2024-09-10T19:07:26.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:25 smithi103 ceph-mon[33163]: 4.10 scrub ok 2024-09-10T19:07:26.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:25 smithi103 ceph-mon[33163]: 4.6 scrub starts 2024-09-10T19:07:26.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:25 smithi103 ceph-mon[33163]: 4.6 scrub ok 2024-09-10T19:07:26.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:25 smithi103 ceph-mon[33163]: 2.7 deep-scrub starts 2024-09-10T19:07:26.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:25 smithi103 ceph-mon[33163]: 2.7 deep-scrub ok 2024-09-10T19:07:26.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:25 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:07:26.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:25 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:07:26.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:25 smithi103 ceph-mon[33163]: 5.b scrub starts 2024-09-10T19:07:26.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:25 smithi103 ceph-mon[33163]: 5.b scrub ok 2024-09-10T19:07:26.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:25 smithi103 ceph-mon[33163]: 2.16 scrub starts 2024-09-10T19:07:26.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:25 smithi103 ceph-mon[33163]: 2.16 scrub ok 2024-09-10T19:07:26.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:25 smithi103 ceph-mon[33163]: pgmap v1044: 152 pgs: 39 peering, 113 active+clean; 581 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 209 B/s, 4 objects/s recovering 2024-09-10T19:07:27.138 INFO:teuthology.orchestra.run.smithi018.stderr:reweighted osd.1 to 1 (10000) 2024-09-10T19:07:27.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:27 smithi018 ceph-mon[29336]: 2.9 scrub starts 2024-09-10T19:07:27.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:27 smithi018 ceph-mon[29336]: 2.9 scrub ok 2024-09-10T19:07:27.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:27 smithi018 ceph-mon[29336]: 5.5 scrub starts 2024-09-10T19:07:27.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:27 smithi018 ceph-mon[29336]: 5.5 scrub ok 2024-09-10T19:07:27.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:27 smithi018 ceph-mon[29336]: 2.12 scrub starts 2024-09-10T19:07:27.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:27 smithi018 ceph-mon[29336]: 2.12 scrub ok 2024-09-10T19:07:27.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:27 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3565333169' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-10T19:07:27.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:27 smithi018 ceph-mon[40196]: 2.9 scrub starts 2024-09-10T19:07:27.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:27 smithi018 ceph-mon[40196]: 2.9 scrub ok 2024-09-10T19:07:27.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:27 smithi018 ceph-mon[40196]: 5.5 scrub starts 2024-09-10T19:07:27.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:27 smithi018 ceph-mon[40196]: 5.5 scrub ok 2024-09-10T19:07:27.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:27 smithi018 ceph-mon[40196]: 2.12 scrub starts 2024-09-10T19:07:27.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:27 smithi018 ceph-mon[40196]: 2.12 scrub ok 2024-09-10T19:07:27.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:27 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3565333169' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-10T19:07:27.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:07:27 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:07:27.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:27 smithi103 ceph-mon[33163]: 2.9 scrub starts 2024-09-10T19:07:27.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:27 smithi103 ceph-mon[33163]: 2.9 scrub ok 2024-09-10T19:07:27.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:27 smithi103 ceph-mon[33163]: 5.5 scrub starts 2024-09-10T19:07:27.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:27 smithi103 ceph-mon[33163]: 5.5 scrub ok 2024-09-10T19:07:27.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:27 smithi103 ceph-mon[33163]: 2.12 scrub starts 2024-09-10T19:07:27.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:27 smithi103 ceph-mon[33163]: 2.12 scrub ok 2024-09-10T19:07:27.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:27 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3565333169' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-10T19:07:27.862 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd primary-affinity 1 1 2024-09-10T19:07:28.128 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:07:28.398 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:28 smithi018 ceph-mon[40196]: 6.3 scrub starts 2024-09-10T19:07:28.398 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:28 smithi018 ceph-mon[40196]: 6.3 scrub ok 2024-09-10T19:07:28.398 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:28 smithi018 ceph-mon[40196]: 4.13 scrub starts 2024-09-10T19:07:28.398 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:28 smithi018 ceph-mon[40196]: 4.13 scrub ok 2024-09-10T19:07:28.398 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:28 smithi018 ceph-mon[40196]: 3.0 scrub starts 2024-09-10T19:07:28.398 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:28 smithi018 ceph-mon[40196]: 3.0 scrub ok 2024-09-10T19:07:28.398 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:28 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3565333169' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-10T19:07:28.398 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:28 smithi018 ceph-mon[40196]: osdmap e467: 8 total, 8 up, 8 in 2024-09-10T19:07:28.398 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:28 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:28.399 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:28 smithi018 ceph-mon[40196]: pgmap v1046: 152 pgs: 152 active+clean; 600 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 42 KiB/s rd, 9.3 KiB/s wr, 62 op/s; 49 KiB/s, 17 objects/s recovering 2024-09-10T19:07:28.399 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:28 smithi018 ceph-mon[29336]: 6.3 scrub starts 2024-09-10T19:07:28.399 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:28 smithi018 ceph-mon[29336]: 6.3 scrub ok 2024-09-10T19:07:28.399 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:28 smithi018 ceph-mon[29336]: 4.13 scrub starts 2024-09-10T19:07:28.399 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:28 smithi018 ceph-mon[29336]: 4.13 scrub ok 2024-09-10T19:07:28.399 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:28 smithi018 ceph-mon[29336]: 3.0 scrub starts 2024-09-10T19:07:28.399 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:28 smithi018 ceph-mon[29336]: 3.0 scrub ok 2024-09-10T19:07:28.399 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:28 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3565333169' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-10T19:07:28.399 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:28 smithi018 ceph-mon[29336]: osdmap e467: 8 total, 8 up, 8 in 2024-09-10T19:07:28.399 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:28 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:28.399 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:28 smithi018 ceph-mon[29336]: pgmap v1046: 152 pgs: 152 active+clean; 600 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 42 KiB/s rd, 9.3 KiB/s wr, 62 op/s; 49 KiB/s, 17 objects/s recovering 2024-09-10T19:07:28.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:28 smithi103 ceph-mon[33163]: 6.3 scrub starts 2024-09-10T19:07:28.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:28 smithi103 ceph-mon[33163]: 6.3 scrub ok 2024-09-10T19:07:28.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:28 smithi103 ceph-mon[33163]: 4.13 scrub starts 2024-09-10T19:07:28.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:28 smithi103 ceph-mon[33163]: 4.13 scrub ok 2024-09-10T19:07:28.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:28 smithi103 ceph-mon[33163]: 3.0 scrub starts 2024-09-10T19:07:28.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:28 smithi103 ceph-mon[33163]: 3.0 scrub ok 2024-09-10T19:07:28.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:28 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3565333169' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-10T19:07:28.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:28 smithi103 ceph-mon[33163]: osdmap e467: 8 total, 8 up, 8 in 2024-09-10T19:07:28.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:28 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:28.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:28 smithi103 ceph-mon[33163]: pgmap v1046: 152 pgs: 152 active+clean; 600 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 42 KiB/s rd, 9.3 KiB/s wr, 62 op/s; 49 KiB/s, 17 objects/s recovering 2024-09-10T19:07:29.396 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:29 smithi018 ceph-mon[29336]: 5.3 scrub starts 2024-09-10T19:07:29.396 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:29 smithi018 ceph-mon[29336]: 5.3 scrub ok 2024-09-10T19:07:29.396 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:29 smithi018 ceph-mon[29336]: 6.2 scrub starts 2024-09-10T19:07:29.396 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:29 smithi018 ceph-mon[29336]: 6.2 scrub ok 2024-09-10T19:07:29.396 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:29 smithi018 ceph-mon[29336]: 4.1c scrub starts 2024-09-10T19:07:29.396 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:29 smithi018 ceph-mon[29336]: 4.1c scrub ok 2024-09-10T19:07:29.396 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:29 smithi018 ceph-mon[29336]: 6.d scrub starts 2024-09-10T19:07:29.396 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:29 smithi018 ceph-mon[29336]: 6.d scrub ok 2024-09-10T19:07:29.396 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:29 smithi018 ceph-mon[29336]: 4.0 scrub starts 2024-09-10T19:07:29.396 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:29 smithi018 ceph-mon[29336]: 4.0 scrub ok 2024-09-10T19:07:29.397 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:29 smithi018 ceph-mon[29336]: osdmap e468: 8 total, 8 up, 8 in 2024-09-10T19:07:29.397 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:29 smithi018 ceph-mon[40196]: 5.3 scrub starts 2024-09-10T19:07:29.397 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:29 smithi018 ceph-mon[40196]: 5.3 scrub ok 2024-09-10T19:07:29.397 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:29 smithi018 ceph-mon[40196]: 6.2 scrub starts 2024-09-10T19:07:29.397 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:29 smithi018 ceph-mon[40196]: 6.2 scrub ok 2024-09-10T19:07:29.397 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:29 smithi018 ceph-mon[40196]: 4.1c scrub starts 2024-09-10T19:07:29.397 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:29 smithi018 ceph-mon[40196]: 4.1c scrub ok 2024-09-10T19:07:29.397 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:29 smithi018 ceph-mon[40196]: 6.d scrub starts 2024-09-10T19:07:29.397 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:29 smithi018 ceph-mon[40196]: 6.d scrub ok 2024-09-10T19:07:29.397 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:29 smithi018 ceph-mon[40196]: 4.0 scrub starts 2024-09-10T19:07:29.397 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:29 smithi018 ceph-mon[40196]: 4.0 scrub ok 2024-09-10T19:07:29.398 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:29 smithi018 ceph-mon[40196]: osdmap e468: 8 total, 8 up, 8 in 2024-09-10T19:07:29.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:29 smithi103 ceph-mon[33163]: 5.3 scrub starts 2024-09-10T19:07:29.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:29 smithi103 ceph-mon[33163]: 5.3 scrub ok 2024-09-10T19:07:29.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:29 smithi103 ceph-mon[33163]: 6.2 scrub starts 2024-09-10T19:07:29.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:29 smithi103 ceph-mon[33163]: 6.2 scrub ok 2024-09-10T19:07:29.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:29 smithi103 ceph-mon[33163]: 4.1c scrub starts 2024-09-10T19:07:29.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:29 smithi103 ceph-mon[33163]: 4.1c scrub ok 2024-09-10T19:07:29.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:29 smithi103 ceph-mon[33163]: 6.d scrub starts 2024-09-10T19:07:29.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:29 smithi103 ceph-mon[33163]: 6.d scrub ok 2024-09-10T19:07:29.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:29 smithi103 ceph-mon[33163]: 4.0 scrub starts 2024-09-10T19:07:29.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:29 smithi103 ceph-mon[33163]: 4.0 scrub ok 2024-09-10T19:07:29.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:29 smithi103 ceph-mon[33163]: osdmap e468: 8 total, 8 up, 8 in 2024-09-10T19:07:30.150 INFO:teuthology.orchestra.run.smithi018.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-10T19:07:30.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:30 smithi018 ceph-mon[29336]: 4.1f deep-scrub starts 2024-09-10T19:07:30.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:30 smithi018 ceph-mon[29336]: 4.1f deep-scrub ok 2024-09-10T19:07:30.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:30 smithi018 ceph-mon[29336]: 6.1a deep-scrub starts 2024-09-10T19:07:30.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:30 smithi018 ceph-mon[29336]: 6.1a deep-scrub ok 2024-09-10T19:07:30.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:30 smithi018 ceph-mon[29336]: 2.1 scrub starts 2024-09-10T19:07:30.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:30 smithi018 ceph-mon[29336]: 2.1 scrub ok 2024-09-10T19:07:30.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:30 smithi018 ceph-mon[29336]: pgmap v1048: 152 pgs: 152 active+clean; 600 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 42 KiB/s rd, 9.3 KiB/s wr, 62 op/s; 49 KiB/s, 12 objects/s recovering 2024-09-10T19:07:30.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:30 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/4240159914' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-09-10T19:07:30.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:30 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-09-10T19:07:30.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:30 smithi018 ceph-mon[40196]: 4.1f deep-scrub starts 2024-09-10T19:07:30.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:30 smithi018 ceph-mon[40196]: 4.1f deep-scrub ok 2024-09-10T19:07:30.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:30 smithi018 ceph-mon[40196]: 6.1a deep-scrub starts 2024-09-10T19:07:30.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:30 smithi018 ceph-mon[40196]: 6.1a deep-scrub ok 2024-09-10T19:07:30.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:30 smithi018 ceph-mon[40196]: 2.1 scrub starts 2024-09-10T19:07:30.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:30 smithi018 ceph-mon[40196]: 2.1 scrub ok 2024-09-10T19:07:30.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:30 smithi018 ceph-mon[40196]: pgmap v1048: 152 pgs: 152 active+clean; 600 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 42 KiB/s rd, 9.3 KiB/s wr, 62 op/s; 49 KiB/s, 12 objects/s recovering 2024-09-10T19:07:30.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:30 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/4240159914' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-09-10T19:07:30.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:30 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-09-10T19:07:30.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:07:30 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:07:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:07:30.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:30 smithi103 ceph-mon[33163]: 4.1f deep-scrub starts 2024-09-10T19:07:30.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:30 smithi103 ceph-mon[33163]: 4.1f deep-scrub ok 2024-09-10T19:07:30.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:30 smithi103 ceph-mon[33163]: 6.1a deep-scrub starts 2024-09-10T19:07:30.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:30 smithi103 ceph-mon[33163]: 6.1a deep-scrub ok 2024-09-10T19:07:30.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:30 smithi103 ceph-mon[33163]: 2.1 scrub starts 2024-09-10T19:07:30.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:30 smithi103 ceph-mon[33163]: 2.1 scrub ok 2024-09-10T19:07:30.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:30 smithi103 ceph-mon[33163]: pgmap v1048: 152 pgs: 152 active+clean; 600 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 42 KiB/s rd, 9.3 KiB/s wr, 62 op/s; 49 KiB/s, 12 objects/s recovering 2024-09-10T19:07:30.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:30 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/4240159914' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-09-10T19:07:30.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:30 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-09-10T19:07:30.949 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd reweight 2 1 2024-09-10T19:07:31.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:07:30 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:07:30] "GET /metrics HTTP/1.1" 200 222082 "" "Prometheus/2.43.0" 2024-09-10T19:07:31.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:31 smithi018 ceph-mon[40196]: 1.3 scrub starts 2024-09-10T19:07:31.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:31 smithi018 ceph-mon[40196]: 1.3 scrub ok 2024-09-10T19:07:31.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:31 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-09-10T19:07:31.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:31 smithi018 ceph-mon[40196]: osdmap e469: 8 total, 8 up, 8 in 2024-09-10T19:07:31.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:31 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:07:31.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:31 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:07:31.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:31 smithi018 ceph-mon[40196]: 6.c scrub starts 2024-09-10T19:07:31.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:31 smithi018 ceph-mon[40196]: 6.c scrub ok 2024-09-10T19:07:31.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:31 smithi018 ceph-mon[29336]: 1.3 scrub starts 2024-09-10T19:07:31.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:31 smithi018 ceph-mon[29336]: 1.3 scrub ok 2024-09-10T19:07:31.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:31 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-09-10T19:07:31.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:31 smithi018 ceph-mon[29336]: osdmap e469: 8 total, 8 up, 8 in 2024-09-10T19:07:31.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:31 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:07:31.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:31 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:07:31.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:31 smithi018 ceph-mon[29336]: 6.c scrub starts 2024-09-10T19:07:31.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:31 smithi018 ceph-mon[29336]: 6.c scrub ok 2024-09-10T19:07:31.224 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:07:31.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:31 smithi103 ceph-mon[33163]: 1.3 scrub starts 2024-09-10T19:07:31.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:31 smithi103 ceph-mon[33163]: 1.3 scrub ok 2024-09-10T19:07:31.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:31 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-09-10T19:07:31.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:31 smithi103 ceph-mon[33163]: osdmap e469: 8 total, 8 up, 8 in 2024-09-10T19:07:31.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:31 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:07:31.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:31 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:07:31.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:31 smithi103 ceph-mon[33163]: 6.c scrub starts 2024-09-10T19:07:31.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:31 smithi103 ceph-mon[33163]: 6.c scrub ok 2024-09-10T19:07:32.364 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:32 smithi018 ceph-mon[29336]: 1.9 scrub starts 2024-09-10T19:07:32.364 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:32 smithi018 ceph-mon[29336]: 1.9 scrub ok 2024-09-10T19:07:32.364 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:32 smithi018 ceph-mon[29336]: 4.e scrub starts 2024-09-10T19:07:32.364 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:32 smithi018 ceph-mon[29336]: 4.e scrub ok 2024-09-10T19:07:32.365 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:32 smithi018 ceph-mon[29336]: osdmap e470: 8 total, 8 up, 8 in 2024-09-10T19:07:32.365 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:32 smithi018 ceph-mon[29336]: 4.15 scrub starts 2024-09-10T19:07:32.365 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:32 smithi018 ceph-mon[29336]: 4.15 scrub ok 2024-09-10T19:07:32.365 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:32 smithi018 ceph-mon[29336]: pgmap v1051: 152 pgs: 152 active+clean; 600 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail 2024-09-10T19:07:32.365 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:32 smithi018 ceph-mon[40196]: 1.9 scrub starts 2024-09-10T19:07:32.365 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:32 smithi018 ceph-mon[40196]: 1.9 scrub ok 2024-09-10T19:07:32.365 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:32 smithi018 ceph-mon[40196]: 4.e scrub starts 2024-09-10T19:07:32.365 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:32 smithi018 ceph-mon[40196]: 4.e scrub ok 2024-09-10T19:07:32.365 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:32 smithi018 ceph-mon[40196]: osdmap e470: 8 total, 8 up, 8 in 2024-09-10T19:07:32.365 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:32 smithi018 ceph-mon[40196]: 4.15 scrub starts 2024-09-10T19:07:32.366 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:32 smithi018 ceph-mon[40196]: 4.15 scrub ok 2024-09-10T19:07:32.366 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:32 smithi018 ceph-mon[40196]: pgmap v1051: 152 pgs: 152 active+clean; 600 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail 2024-09-10T19:07:32.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:32 smithi103 ceph-mon[33163]: 1.9 scrub starts 2024-09-10T19:07:32.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:32 smithi103 ceph-mon[33163]: 1.9 scrub ok 2024-09-10T19:07:32.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:32 smithi103 ceph-mon[33163]: 4.e scrub starts 2024-09-10T19:07:32.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:32 smithi103 ceph-mon[33163]: 4.e scrub ok 2024-09-10T19:07:32.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:32 smithi103 ceph-mon[33163]: osdmap e470: 8 total, 8 up, 8 in 2024-09-10T19:07:32.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:32 smithi103 ceph-mon[33163]: 4.15 scrub starts 2024-09-10T19:07:32.525 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:32 smithi103 ceph-mon[33163]: 4.15 scrub ok 2024-09-10T19:07:32.525 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:32 smithi103 ceph-mon[33163]: pgmap v1051: 152 pgs: 152 active+clean; 600 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail 2024-09-10T19:07:33.167 INFO:teuthology.orchestra.run.smithi018.stderr:reweighted osd.2 to 1 (10000) 2024-09-10T19:07:33.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:33 smithi018 ceph-mon[40196]: 5.19 scrub starts 2024-09-10T19:07:33.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:33 smithi018 ceph-mon[40196]: 5.19 scrub ok 2024-09-10T19:07:33.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:33 smithi018 ceph-mon[40196]: 5.1c scrub starts 2024-09-10T19:07:33.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:33 smithi018 ceph-mon[40196]: 5.1c scrub ok 2024-09-10T19:07:33.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:33 smithi018 ceph-mon[40196]: 1.4 scrub starts 2024-09-10T19:07:33.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:33 smithi018 ceph-mon[40196]: 1.4 scrub ok 2024-09-10T19:07:33.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:33 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1845697670' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-10T19:07:33.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:33 smithi018 ceph-mon[29336]: 5.19 scrub starts 2024-09-10T19:07:33.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:33 smithi018 ceph-mon[29336]: 5.19 scrub ok 2024-09-10T19:07:33.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:33 smithi018 ceph-mon[29336]: 5.1c scrub starts 2024-09-10T19:07:33.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:33 smithi018 ceph-mon[29336]: 5.1c scrub ok 2024-09-10T19:07:33.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:33 smithi018 ceph-mon[29336]: 1.4 scrub starts 2024-09-10T19:07:33.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:33 smithi018 ceph-mon[29336]: 1.4 scrub ok 2024-09-10T19:07:33.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:33 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1845697670' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-10T19:07:33.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:33 smithi103 ceph-mon[33163]: 5.19 scrub starts 2024-09-10T19:07:33.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:33 smithi103 ceph-mon[33163]: 5.19 scrub ok 2024-09-10T19:07:33.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:33 smithi103 ceph-mon[33163]: 5.1c scrub starts 2024-09-10T19:07:33.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:33 smithi103 ceph-mon[33163]: 5.1c scrub ok 2024-09-10T19:07:33.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:33 smithi103 ceph-mon[33163]: 1.4 scrub starts 2024-09-10T19:07:33.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:33 smithi103 ceph-mon[33163]: 1.4 scrub ok 2024-09-10T19:07:33.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:33 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1845697670' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-10T19:07:33.994 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd primary-affinity 2 1 2024-09-10T19:07:34.256 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:07:34.378 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:34 smithi018 ceph-mon[29336]: 5.6 deep-scrub starts 2024-09-10T19:07:34.378 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:34 smithi018 ceph-mon[29336]: 5.6 deep-scrub ok 2024-09-10T19:07:34.378 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:34 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1845697670' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-10T19:07:34.378 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:34 smithi018 ceph-mon[29336]: osdmap e471: 8 total, 8 up, 8 in 2024-09-10T19:07:34.378 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:34 smithi018 ceph-mon[29336]: 4.7 scrub starts 2024-09-10T19:07:34.378 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:34 smithi018 ceph-mon[29336]: 4.7 scrub ok 2024-09-10T19:07:34.378 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:34 smithi018 ceph-mon[29336]: pgmap v1053: 152 pgs: 152 active+clean; 600 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 190 KiB/s rd, 190 op/s 2024-09-10T19:07:34.378 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:34 smithi018 ceph-mon[29336]: 2.1b scrub starts 2024-09-10T19:07:34.378 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:34 smithi018 ceph-mon[29336]: 2.1b scrub ok 2024-09-10T19:07:34.378 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:34 smithi018 ceph-mon[29336]: osdmap e472: 8 total, 8 up, 8 in 2024-09-10T19:07:34.379 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:34 smithi018 ceph-mon[40196]: 5.6 deep-scrub starts 2024-09-10T19:07:34.379 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:34 smithi018 ceph-mon[40196]: 5.6 deep-scrub ok 2024-09-10T19:07:34.379 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:34 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1845697670' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-10T19:07:34.379 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:34 smithi018 ceph-mon[40196]: osdmap e471: 8 total, 8 up, 8 in 2024-09-10T19:07:34.379 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:34 smithi018 ceph-mon[40196]: 4.7 scrub starts 2024-09-10T19:07:34.379 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:34 smithi018 ceph-mon[40196]: 4.7 scrub ok 2024-09-10T19:07:34.379 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:34 smithi018 ceph-mon[40196]: pgmap v1053: 152 pgs: 152 active+clean; 600 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 190 KiB/s rd, 190 op/s 2024-09-10T19:07:34.379 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:34 smithi018 ceph-mon[40196]: 2.1b scrub starts 2024-09-10T19:07:34.379 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:34 smithi018 ceph-mon[40196]: 2.1b scrub ok 2024-09-10T19:07:34.379 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:34 smithi018 ceph-mon[40196]: osdmap e472: 8 total, 8 up, 8 in 2024-09-10T19:07:34.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:34 smithi103 ceph-mon[33163]: 5.6 deep-scrub starts 2024-09-10T19:07:34.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:34 smithi103 ceph-mon[33163]: 5.6 deep-scrub ok 2024-09-10T19:07:34.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:34 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1845697670' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-10T19:07:34.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:34 smithi103 ceph-mon[33163]: osdmap e471: 8 total, 8 up, 8 in 2024-09-10T19:07:34.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:34 smithi103 ceph-mon[33163]: 4.7 scrub starts 2024-09-10T19:07:34.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:34 smithi103 ceph-mon[33163]: 4.7 scrub ok 2024-09-10T19:07:34.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:34 smithi103 ceph-mon[33163]: pgmap v1053: 152 pgs: 152 active+clean; 600 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 190 KiB/s rd, 190 op/s 2024-09-10T19:07:34.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:34 smithi103 ceph-mon[33163]: 2.1b scrub starts 2024-09-10T19:07:34.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:34 smithi103 ceph-mon[33163]: 2.1b scrub ok 2024-09-10T19:07:34.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:34 smithi103 ceph-mon[33163]: osdmap e472: 8 total, 8 up, 8 in 2024-09-10T19:07:35.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:35 smithi018 ceph-mon[40196]: 4.b scrub starts 2024-09-10T19:07:35.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:35 smithi018 ceph-mon[40196]: 4.b scrub ok 2024-09-10T19:07:35.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:35 smithi018 ceph-mon[40196]: 1.14 deep-scrub starts 2024-09-10T19:07:35.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:35 smithi018 ceph-mon[40196]: 1.14 deep-scrub ok 2024-09-10T19:07:35.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:35 smithi018 ceph-mon[40196]: 4.11 deep-scrub starts 2024-09-10T19:07:35.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:35 smithi018 ceph-mon[40196]: 4.11 deep-scrub ok 2024-09-10T19:07:35.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:35 smithi018 ceph-mon[29336]: 4.b scrub starts 2024-09-10T19:07:35.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:35 smithi018 ceph-mon[29336]: 4.b scrub ok 2024-09-10T19:07:35.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:35 smithi018 ceph-mon[29336]: 1.14 deep-scrub starts 2024-09-10T19:07:35.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:35 smithi018 ceph-mon[29336]: 1.14 deep-scrub ok 2024-09-10T19:07:35.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:35 smithi018 ceph-mon[29336]: 4.11 deep-scrub starts 2024-09-10T19:07:35.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:35 smithi018 ceph-mon[29336]: 4.11 deep-scrub ok 2024-09-10T19:07:35.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:35 smithi103 ceph-mon[33163]: 4.b scrub starts 2024-09-10T19:07:35.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:35 smithi103 ceph-mon[33163]: 4.b scrub ok 2024-09-10T19:07:35.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:35 smithi103 ceph-mon[33163]: 1.14 deep-scrub starts 2024-09-10T19:07:35.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:35 smithi103 ceph-mon[33163]: 1.14 deep-scrub ok 2024-09-10T19:07:35.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:35 smithi103 ceph-mon[33163]: 4.11 deep-scrub starts 2024-09-10T19:07:35.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:35 smithi103 ceph-mon[33163]: 4.11 deep-scrub ok 2024-09-10T19:07:36.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:36 smithi018 ceph-mon[40196]: 4.1b scrub starts 2024-09-10T19:07:36.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:36 smithi018 ceph-mon[40196]: 4.1b scrub ok 2024-09-10T19:07:36.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:36 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:07:36.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:36 smithi018 ceph-mon[40196]: pgmap v1055: 152 pgs: 152 active+clean; 600 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 166 KiB/s rd, 166 op/s 2024-09-10T19:07:36.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:36 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:07:36.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:36 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:07:36.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:36 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:07:36.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:36 smithi018 ceph-mon[29336]: 4.1b scrub starts 2024-09-10T19:07:36.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:36 smithi018 ceph-mon[29336]: 4.1b scrub ok 2024-09-10T19:07:36.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:36 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:07:36.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:36 smithi018 ceph-mon[29336]: pgmap v1055: 152 pgs: 152 active+clean; 600 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 166 KiB/s rd, 166 op/s 2024-09-10T19:07:36.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:36 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:07:36.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:36 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:07:36.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:36 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:07:36.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:36 smithi103 ceph-mon[33163]: 4.1b scrub starts 2024-09-10T19:07:36.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:36 smithi103 ceph-mon[33163]: 4.1b scrub ok 2024-09-10T19:07:36.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:36 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:07:36.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:36 smithi103 ceph-mon[33163]: pgmap v1055: 152 pgs: 152 active+clean; 600 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 166 KiB/s rd, 166 op/s 2024-09-10T19:07:36.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:36 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:07:36.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:36 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:07:36.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:36 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:07:37.187 INFO:teuthology.orchestra.run.smithi018.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-10T19:07:37.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:37 smithi018 ceph-mon[29336]: 2.13 scrub starts 2024-09-10T19:07:37.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:37 smithi018 ceph-mon[29336]: 2.13 scrub ok 2024-09-10T19:07:37.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:37 smithi018 ceph-mon[29336]: 2.c scrub starts 2024-09-10T19:07:37.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:37 smithi018 ceph-mon[29336]: 2.c scrub ok 2024-09-10T19:07:37.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:37 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2477481929' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]: dispatch 2024-09-10T19:07:37.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:37 smithi018 ceph-mon[29336]: 6.1f scrub starts 2024-09-10T19:07:37.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:37 smithi018 ceph-mon[29336]: 6.1f scrub ok 2024-09-10T19:07:37.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:37 smithi018 ceph-mon[40196]: 2.13 scrub starts 2024-09-10T19:07:37.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:37 smithi018 ceph-mon[40196]: 2.13 scrub ok 2024-09-10T19:07:37.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:37 smithi018 ceph-mon[40196]: 2.c scrub starts 2024-09-10T19:07:37.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:37 smithi018 ceph-mon[40196]: 2.c scrub ok 2024-09-10T19:07:37.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:37 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2477481929' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]: dispatch 2024-09-10T19:07:37.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:37 smithi018 ceph-mon[40196]: 6.1f scrub starts 2024-09-10T19:07:37.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:37 smithi018 ceph-mon[40196]: 6.1f scrub ok 2024-09-10T19:07:37.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:07:37 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:07:37.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:37 smithi103 ceph-mon[33163]: 2.13 scrub starts 2024-09-10T19:07:37.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:37 smithi103 ceph-mon[33163]: 2.13 scrub ok 2024-09-10T19:07:37.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:37 smithi103 ceph-mon[33163]: 2.c scrub starts 2024-09-10T19:07:37.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:37 smithi103 ceph-mon[33163]: 2.c scrub ok 2024-09-10T19:07:37.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:37 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2477481929' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]: dispatch 2024-09-10T19:07:37.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:37 smithi103 ceph-mon[33163]: 6.1f scrub starts 2024-09-10T19:07:37.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:37 smithi103 ceph-mon[33163]: 6.1f scrub ok 2024-09-10T19:07:37.998 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd reweight 3 1 2024-09-10T19:07:38.264 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:07:38.385 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:38 smithi018 ceph-mon[29336]: 5.4 scrub starts 2024-09-10T19:07:38.386 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:38 smithi018 ceph-mon[29336]: 5.4 scrub ok 2024-09-10T19:07:38.386 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:38 smithi018 ceph-mon[29336]: 1.e scrub starts 2024-09-10T19:07:38.386 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:38 smithi018 ceph-mon[29336]: 1.e scrub ok 2024-09-10T19:07:38.386 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:38 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2477481929' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]': finished 2024-09-10T19:07:38.386 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:38 smithi018 ceph-mon[29336]: osdmap e473: 8 total, 8 up, 8 in 2024-09-10T19:07:38.386 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:38 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:38.386 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:38 smithi018 ceph-mon[29336]: pgmap v1057: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 149 KiB/s rd, 149 op/s 2024-09-10T19:07:38.386 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:38 smithi018 ceph-mon[40196]: 5.4 scrub starts 2024-09-10T19:07:38.386 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:38 smithi018 ceph-mon[40196]: 5.4 scrub ok 2024-09-10T19:07:38.387 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:38 smithi018 ceph-mon[40196]: 1.e scrub starts 2024-09-10T19:07:38.387 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:38 smithi018 ceph-mon[40196]: 1.e scrub ok 2024-09-10T19:07:38.387 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:38 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2477481929' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]': finished 2024-09-10T19:07:38.387 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:38 smithi018 ceph-mon[40196]: osdmap e473: 8 total, 8 up, 8 in 2024-09-10T19:07:38.387 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:38 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:38.387 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:38 smithi018 ceph-mon[40196]: pgmap v1057: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 149 KiB/s rd, 149 op/s 2024-09-10T19:07:38.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:38 smithi103 ceph-mon[33163]: 5.4 scrub starts 2024-09-10T19:07:38.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:38 smithi103 ceph-mon[33163]: 5.4 scrub ok 2024-09-10T19:07:38.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:38 smithi103 ceph-mon[33163]: 1.e scrub starts 2024-09-10T19:07:38.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:38 smithi103 ceph-mon[33163]: 1.e scrub ok 2024-09-10T19:07:38.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:38 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2477481929' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]': finished 2024-09-10T19:07:38.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:38 smithi103 ceph-mon[33163]: osdmap e473: 8 total, 8 up, 8 in 2024-09-10T19:07:38.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:38 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:38.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:38 smithi103 ceph-mon[33163]: pgmap v1057: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 149 KiB/s rd, 149 op/s 2024-09-10T19:07:39.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:39 smithi103 ceph-mon[33163]: 6.12 scrub starts 2024-09-10T19:07:39.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:39 smithi103 ceph-mon[33163]: 6.12 scrub ok 2024-09-10T19:07:39.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:39 smithi103 ceph-mon[33163]: 5.1b scrub starts 2024-09-10T19:07:39.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:39 smithi103 ceph-mon[33163]: 5.1b scrub ok 2024-09-10T19:07:39.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:39 smithi103 ceph-mon[33163]: 2.d scrub starts 2024-09-10T19:07:39.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:39 smithi103 ceph-mon[33163]: 2.d scrub ok 2024-09-10T19:07:39.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:39 smithi103 ceph-mon[33163]: 3.2 scrub starts 2024-09-10T19:07:39.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:39 smithi103 ceph-mon[33163]: 3.2 scrub ok 2024-09-10T19:07:39.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:39 smithi103 ceph-mon[33163]: osdmap e474: 8 total, 8 up, 8 in 2024-09-10T19:07:39.537 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:39 smithi018 ceph-mon[40196]: 6.12 scrub starts 2024-09-10T19:07:39.537 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:39 smithi018 ceph-mon[40196]: 6.12 scrub ok 2024-09-10T19:07:39.537 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:39 smithi018 ceph-mon[40196]: 5.1b scrub starts 2024-09-10T19:07:39.537 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:39 smithi018 ceph-mon[40196]: 5.1b scrub ok 2024-09-10T19:07:39.537 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:39 smithi018 ceph-mon[40196]: 2.d scrub starts 2024-09-10T19:07:39.537 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:39 smithi018 ceph-mon[40196]: 2.d scrub ok 2024-09-10T19:07:39.537 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:39 smithi018 ceph-mon[40196]: 3.2 scrub starts 2024-09-10T19:07:39.538 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:39 smithi018 ceph-mon[40196]: 3.2 scrub ok 2024-09-10T19:07:39.538 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:39 smithi018 ceph-mon[40196]: osdmap e474: 8 total, 8 up, 8 in 2024-09-10T19:07:39.538 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:39 smithi018 ceph-mon[29336]: 6.12 scrub starts 2024-09-10T19:07:39.538 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:39 smithi018 ceph-mon[29336]: 6.12 scrub ok 2024-09-10T19:07:39.538 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:39 smithi018 ceph-mon[29336]: 5.1b scrub starts 2024-09-10T19:07:39.538 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:39 smithi018 ceph-mon[29336]: 5.1b scrub ok 2024-09-10T19:07:39.538 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:39 smithi018 ceph-mon[29336]: 2.d scrub starts 2024-09-10T19:07:39.538 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:39 smithi018 ceph-mon[29336]: 2.d scrub ok 2024-09-10T19:07:39.538 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:39 smithi018 ceph-mon[29336]: 3.2 scrub starts 2024-09-10T19:07:39.538 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:39 smithi018 ceph-mon[29336]: 3.2 scrub ok 2024-09-10T19:07:39.538 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:39 smithi018 ceph-mon[29336]: osdmap e474: 8 total, 8 up, 8 in 2024-09-10T19:07:40.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:07:40 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:07:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:07:40.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:40 smithi103 ceph-mon[33163]: 5.2 scrub starts 2024-09-10T19:07:40.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:40 smithi103 ceph-mon[33163]: 5.2 scrub ok 2024-09-10T19:07:40.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:40 smithi103 ceph-mon[33163]: 4.2 scrub starts 2024-09-10T19:07:40.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:40 smithi103 ceph-mon[33163]: 4.2 scrub ok 2024-09-10T19:07:40.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:40 smithi103 ceph-mon[33163]: pgmap v1059: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-10T19:07:40.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:40 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:07:40.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:40 smithi018 ceph-mon[29336]: 5.2 scrub starts 2024-09-10T19:07:40.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:40 smithi018 ceph-mon[29336]: 5.2 scrub ok 2024-09-10T19:07:40.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:40 smithi018 ceph-mon[29336]: 4.2 scrub starts 2024-09-10T19:07:40.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:40 smithi018 ceph-mon[29336]: 4.2 scrub ok 2024-09-10T19:07:40.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:40 smithi018 ceph-mon[29336]: pgmap v1059: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-10T19:07:40.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:40 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:07:40.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:40 smithi018 ceph-mon[40196]: 5.2 scrub starts 2024-09-10T19:07:40.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:40 smithi018 ceph-mon[40196]: 5.2 scrub ok 2024-09-10T19:07:40.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:40 smithi018 ceph-mon[40196]: 4.2 scrub starts 2024-09-10T19:07:40.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:40 smithi018 ceph-mon[40196]: 4.2 scrub ok 2024-09-10T19:07:40.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:40 smithi018 ceph-mon[40196]: pgmap v1059: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-10T19:07:40.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:40 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:07:41.206 INFO:teuthology.orchestra.run.smithi018.stderr:reweighted osd.3 to 1 (10000) 2024-09-10T19:07:41.222 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:07:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:07:40] "GET /metrics HTTP/1.1" 200 222137 "" "Prometheus/2.43.0" 2024-09-10T19:07:41.222 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:41 smithi018 ceph-mon[29336]: 6.0 scrub starts 2024-09-10T19:07:41.223 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:41 smithi018 ceph-mon[29336]: 6.0 scrub ok 2024-09-10T19:07:41.223 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:41 smithi018 ceph-mon[29336]: 6.f scrub starts 2024-09-10T19:07:41.223 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:41 smithi018 ceph-mon[29336]: 6.f scrub ok 2024-09-10T19:07:41.223 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:41 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2296556379' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-10T19:07:41.223 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:41 smithi018 ceph-mon[40196]: 6.0 scrub starts 2024-09-10T19:07:41.223 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:41 smithi018 ceph-mon[40196]: 6.0 scrub ok 2024-09-10T19:07:41.223 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:41 smithi018 ceph-mon[40196]: 6.f scrub starts 2024-09-10T19:07:41.223 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:41 smithi018 ceph-mon[40196]: 6.f scrub ok 2024-09-10T19:07:41.223 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:41 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2296556379' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-10T19:07:41.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:41 smithi103 ceph-mon[33163]: 6.0 scrub starts 2024-09-10T19:07:41.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:41 smithi103 ceph-mon[33163]: 6.0 scrub ok 2024-09-10T19:07:41.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:41 smithi103 ceph-mon[33163]: 6.f scrub starts 2024-09-10T19:07:41.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:41 smithi103 ceph-mon[33163]: 6.f scrub ok 2024-09-10T19:07:41.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:41 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2296556379' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-10T19:07:41.966 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd primary-affinity 3 1 2024-09-10T19:07:42.234 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:07:42.492 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:42 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2296556379' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-10T19:07:42.492 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:42 smithi018 ceph-mon[40196]: osdmap e475: 8 total, 8 up, 8 in 2024-09-10T19:07:42.492 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:42 smithi018 ceph-mon[40196]: pgmap v1061: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-10T19:07:42.492 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:42 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2296556379' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-10T19:07:42.492 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:42 smithi018 ceph-mon[29336]: osdmap e475: 8 total, 8 up, 8 in 2024-09-10T19:07:42.492 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:42 smithi018 ceph-mon[29336]: pgmap v1061: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-10T19:07:42.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:42 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2296556379' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-10T19:07:42.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:42 smithi103 ceph-mon[33163]: osdmap e475: 8 total, 8 up, 8 in 2024-09-10T19:07:42.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:42 smithi103 ceph-mon[33163]: pgmap v1061: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-10T19:07:43.341 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:43 smithi018 ceph-mon[29336]: 2.b scrub starts 2024-09-10T19:07:43.341 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:43 smithi018 ceph-mon[29336]: 2.b scrub ok 2024-09-10T19:07:43.341 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:43 smithi018 ceph-mon[29336]: 4.16 scrub starts 2024-09-10T19:07:43.341 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:43 smithi018 ceph-mon[29336]: 4.16 scrub ok 2024-09-10T19:07:43.341 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:43 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:07:43.341 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:43 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:07:43.341 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:43 smithi018 ceph-mon[29336]: osdmap e476: 8 total, 8 up, 8 in 2024-09-10T19:07:43.342 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:43 smithi018 ceph-mon[29336]: 1.13 scrub starts 2024-09-10T19:07:43.342 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:43 smithi018 ceph-mon[29336]: 1.13 scrub ok 2024-09-10T19:07:43.342 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:43 smithi018 ceph-mon[29336]: 6.b scrub starts 2024-09-10T19:07:43.342 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:43 smithi018 ceph-mon[29336]: 6.b scrub ok 2024-09-10T19:07:43.342 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:43 smithi018 ceph-mon[40196]: 2.b scrub starts 2024-09-10T19:07:43.342 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:43 smithi018 ceph-mon[40196]: 2.b scrub ok 2024-09-10T19:07:43.342 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:43 smithi018 ceph-mon[40196]: 4.16 scrub starts 2024-09-10T19:07:43.342 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:43 smithi018 ceph-mon[40196]: 4.16 scrub ok 2024-09-10T19:07:43.342 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:43 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:07:43.342 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:43 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:07:43.343 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:43 smithi018 ceph-mon[40196]: osdmap e476: 8 total, 8 up, 8 in 2024-09-10T19:07:43.343 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:43 smithi018 ceph-mon[40196]: 1.13 scrub starts 2024-09-10T19:07:43.343 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:43 smithi018 ceph-mon[40196]: 1.13 scrub ok 2024-09-10T19:07:43.343 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:43 smithi018 ceph-mon[40196]: 6.b scrub starts 2024-09-10T19:07:43.343 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:43 smithi018 ceph-mon[40196]: 6.b scrub ok 2024-09-10T19:07:43.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:43 smithi103 ceph-mon[33163]: 2.b scrub starts 2024-09-10T19:07:43.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:43 smithi103 ceph-mon[33163]: 2.b scrub ok 2024-09-10T19:07:43.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:43 smithi103 ceph-mon[33163]: 4.16 scrub starts 2024-09-10T19:07:43.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:43 smithi103 ceph-mon[33163]: 4.16 scrub ok 2024-09-10T19:07:43.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:43 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:07:43.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:43 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:07:43.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:43 smithi103 ceph-mon[33163]: osdmap e476: 8 total, 8 up, 8 in 2024-09-10T19:07:43.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:43 smithi103 ceph-mon[33163]: 1.13 scrub starts 2024-09-10T19:07:43.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:43 smithi103 ceph-mon[33163]: 1.13 scrub ok 2024-09-10T19:07:43.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:43 smithi103 ceph-mon[33163]: 6.b scrub starts 2024-09-10T19:07:43.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:43 smithi103 ceph-mon[33163]: 6.b scrub ok 2024-09-10T19:07:44.220 INFO:teuthology.orchestra.run.smithi018.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-10T19:07:44.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:44 smithi103 ceph-mon[33163]: 2.11 deep-scrub starts 2024-09-10T19:07:44.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:44 smithi103 ceph-mon[33163]: 2.11 deep-scrub ok 2024-09-10T19:07:44.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:44 smithi103 ceph-mon[33163]: 4.d scrub starts 2024-09-10T19:07:44.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:44 smithi103 ceph-mon[33163]: 4.d scrub ok 2024-09-10T19:07:44.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:44 smithi103 ceph-mon[33163]: pgmap v1063: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:44.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:44 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1816184112' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-09-10T19:07:44.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:44 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-09-10T19:07:44.590 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:44 smithi018 ceph-mon[29336]: 2.11 deep-scrub starts 2024-09-10T19:07:44.591 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:44 smithi018 ceph-mon[29336]: 2.11 deep-scrub ok 2024-09-10T19:07:44.591 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:44 smithi018 ceph-mon[29336]: 4.d scrub starts 2024-09-10T19:07:44.591 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:44 smithi018 ceph-mon[29336]: 4.d scrub ok 2024-09-10T19:07:44.591 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:44 smithi018 ceph-mon[29336]: pgmap v1063: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:44.591 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:44 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1816184112' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-09-10T19:07:44.591 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:44 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-09-10T19:07:44.591 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:44 smithi018 ceph-mon[40196]: 2.11 deep-scrub starts 2024-09-10T19:07:44.591 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:44 smithi018 ceph-mon[40196]: 2.11 deep-scrub ok 2024-09-10T19:07:44.591 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:44 smithi018 ceph-mon[40196]: 4.d scrub starts 2024-09-10T19:07:44.592 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:44 smithi018 ceph-mon[40196]: 4.d scrub ok 2024-09-10T19:07:44.592 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:44 smithi018 ceph-mon[40196]: pgmap v1063: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:44.592 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:44 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1816184112' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-09-10T19:07:44.592 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:44 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-09-10T19:07:44.990 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd reweight 4 1 2024-09-10T19:07:45.254 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:07:45.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:45 smithi103 ceph-mon[33163]: 1.8 scrub starts 2024-09-10T19:07:45.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:45 smithi103 ceph-mon[33163]: 1.8 scrub ok 2024-09-10T19:07:45.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:45 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-09-10T19:07:45.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:45 smithi103 ceph-mon[33163]: osdmap e477: 8 total, 8 up, 8 in 2024-09-10T19:07:45.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:45 smithi103 ceph-mon[33163]: 2.4 scrub starts 2024-09-10T19:07:45.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:45 smithi103 ceph-mon[33163]: 2.4 scrub ok 2024-09-10T19:07:45.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:45 smithi103 ceph-mon[33163]: 5.14 scrub starts 2024-09-10T19:07:45.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:45 smithi103 ceph-mon[33163]: 5.14 scrub ok 2024-09-10T19:07:45.602 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:45 smithi018 ceph-mon[29336]: 1.8 scrub starts 2024-09-10T19:07:45.602 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:45 smithi018 ceph-mon[29336]: 1.8 scrub ok 2024-09-10T19:07:45.602 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:45 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-09-10T19:07:45.602 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:45 smithi018 ceph-mon[29336]: osdmap e477: 8 total, 8 up, 8 in 2024-09-10T19:07:45.602 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:45 smithi018 ceph-mon[29336]: 2.4 scrub starts 2024-09-10T19:07:45.602 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:45 smithi018 ceph-mon[29336]: 2.4 scrub ok 2024-09-10T19:07:45.602 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:45 smithi018 ceph-mon[29336]: 5.14 scrub starts 2024-09-10T19:07:45.602 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:45 smithi018 ceph-mon[29336]: 5.14 scrub ok 2024-09-10T19:07:45.603 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:45 smithi018 ceph-mon[40196]: 1.8 scrub starts 2024-09-10T19:07:45.603 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:45 smithi018 ceph-mon[40196]: 1.8 scrub ok 2024-09-10T19:07:45.603 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:45 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-09-10T19:07:45.603 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:45 smithi018 ceph-mon[40196]: osdmap e477: 8 total, 8 up, 8 in 2024-09-10T19:07:45.603 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:45 smithi018 ceph-mon[40196]: 2.4 scrub starts 2024-09-10T19:07:45.603 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:45 smithi018 ceph-mon[40196]: 2.4 scrub ok 2024-09-10T19:07:45.603 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:45 smithi018 ceph-mon[40196]: 5.14 scrub starts 2024-09-10T19:07:45.603 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:45 smithi018 ceph-mon[40196]: 5.14 scrub ok 2024-09-10T19:07:46.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:46 smithi103 ceph-mon[33163]: 5.c scrub starts 2024-09-10T19:07:46.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:46 smithi103 ceph-mon[33163]: 5.c scrub ok 2024-09-10T19:07:46.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:46 smithi103 ceph-mon[33163]: pgmap v1065: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:46.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:46 smithi103 ceph-mon[33163]: 4.1d scrub starts 2024-09-10T19:07:46.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:46 smithi103 ceph-mon[33163]: 4.1d scrub ok 2024-09-10T19:07:46.566 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:46 smithi018 ceph-mon[40196]: 5.c scrub starts 2024-09-10T19:07:46.567 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:46 smithi018 ceph-mon[40196]: 5.c scrub ok 2024-09-10T19:07:46.567 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:46 smithi018 ceph-mon[40196]: pgmap v1065: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:46.567 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:46 smithi018 ceph-mon[40196]: 4.1d scrub starts 2024-09-10T19:07:46.567 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:46 smithi018 ceph-mon[40196]: 4.1d scrub ok 2024-09-10T19:07:46.567 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:46 smithi018 ceph-mon[29336]: 5.c scrub starts 2024-09-10T19:07:46.567 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:46 smithi018 ceph-mon[29336]: 5.c scrub ok 2024-09-10T19:07:46.567 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:46 smithi018 ceph-mon[29336]: pgmap v1065: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:46.567 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:46 smithi018 ceph-mon[29336]: 4.1d scrub starts 2024-09-10T19:07:46.567 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:46 smithi018 ceph-mon[29336]: 4.1d scrub ok 2024-09-10T19:07:47.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:47 smithi018 ceph-mon[40196]: 2.17 scrub starts 2024-09-10T19:07:47.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:47 smithi018 ceph-mon[40196]: 2.17 scrub ok 2024-09-10T19:07:47.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:47 smithi018 ceph-mon[40196]: 6.1 scrub starts 2024-09-10T19:07:47.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:47 smithi018 ceph-mon[40196]: 6.1 scrub ok 2024-09-10T19:07:47.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:47 smithi018 ceph-mon[29336]: 2.17 scrub starts 2024-09-10T19:07:47.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:47 smithi018 ceph-mon[29336]: 2.17 scrub ok 2024-09-10T19:07:47.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:47 smithi018 ceph-mon[29336]: 6.1 scrub starts 2024-09-10T19:07:47.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:47 smithi018 ceph-mon[29336]: 6.1 scrub ok 2024-09-10T19:07:47.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:47 smithi103 ceph-mon[33163]: 2.17 scrub starts 2024-09-10T19:07:47.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:47 smithi103 ceph-mon[33163]: 2.17 scrub ok 2024-09-10T19:07:47.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:47 smithi103 ceph-mon[33163]: 6.1 scrub starts 2024-09-10T19:07:47.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:47 smithi103 ceph-mon[33163]: 6.1 scrub ok 2024-09-10T19:07:47.523 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:07:47 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:07:48.236 INFO:teuthology.orchestra.run.smithi018.stderr:reweighted osd.4 to 1 (10000) 2024-09-10T19:07:48.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:48 smithi103 ceph-mon[33163]: 2.8 scrub starts 2024-09-10T19:07:48.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:48 smithi103 ceph-mon[33163]: 2.8 scrub ok 2024-09-10T19:07:48.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:48 smithi103 ceph-mon[33163]: 4.18 scrub starts 2024-09-10T19:07:48.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:48 smithi103 ceph-mon[33163]: 4.18 scrub ok 2024-09-10T19:07:48.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:48 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2212566529' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-10T19:07:48.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:48 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:48.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:48 smithi103 ceph-mon[33163]: pgmap v1066: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-09-10T19:07:48.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:48 smithi103 ceph-mon[33163]: 4.17 deep-scrub starts 2024-09-10T19:07:48.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:48 smithi103 ceph-mon[33163]: 4.17 deep-scrub ok 2024-09-10T19:07:48.625 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:48 smithi018 ceph-mon[29336]: 2.8 scrub starts 2024-09-10T19:07:48.625 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:48 smithi018 ceph-mon[29336]: 2.8 scrub ok 2024-09-10T19:07:48.625 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:48 smithi018 ceph-mon[29336]: 4.18 scrub starts 2024-09-10T19:07:48.625 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:48 smithi018 ceph-mon[29336]: 4.18 scrub ok 2024-09-10T19:07:48.625 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:48 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2212566529' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-10T19:07:48.625 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:48 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:48.625 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:48 smithi018 ceph-mon[29336]: pgmap v1066: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-09-10T19:07:48.625 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:48 smithi018 ceph-mon[29336]: 4.17 deep-scrub starts 2024-09-10T19:07:48.625 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:48 smithi018 ceph-mon[29336]: 4.17 deep-scrub ok 2024-09-10T19:07:48.626 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:48 smithi018 ceph-mon[40196]: 2.8 scrub starts 2024-09-10T19:07:48.626 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:48 smithi018 ceph-mon[40196]: 2.8 scrub ok 2024-09-10T19:07:48.626 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:48 smithi018 ceph-mon[40196]: 4.18 scrub starts 2024-09-10T19:07:48.626 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:48 smithi018 ceph-mon[40196]: 4.18 scrub ok 2024-09-10T19:07:48.626 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:48 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2212566529' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-10T19:07:48.626 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:48 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:48.626 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:48 smithi018 ceph-mon[40196]: pgmap v1066: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-09-10T19:07:48.626 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:48 smithi018 ceph-mon[40196]: 4.17 deep-scrub starts 2024-09-10T19:07:48.626 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:48 smithi018 ceph-mon[40196]: 4.17 deep-scrub ok 2024-09-10T19:07:49.029 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd primary-affinity 4 1 2024-09-10T19:07:49.289 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:07:49.411 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:49 smithi018 ceph-mon[29336]: 4.1e scrub starts 2024-09-10T19:07:49.411 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:49 smithi018 ceph-mon[29336]: 4.1e scrub ok 2024-09-10T19:07:49.411 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:49 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2212566529' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-10T19:07:49.411 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:49 smithi018 ceph-mon[29336]: osdmap e478: 8 total, 8 up, 8 in 2024-09-10T19:07:49.411 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:49 smithi018 ceph-mon[29336]: 5.15 scrub starts 2024-09-10T19:07:49.411 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:49 smithi018 ceph-mon[29336]: 5.15 scrub ok 2024-09-10T19:07:49.411 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:49 smithi018 ceph-mon[29336]: 4.14 scrub starts 2024-09-10T19:07:49.411 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:49 smithi018 ceph-mon[29336]: 4.14 scrub ok 2024-09-10T19:07:49.412 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:49 smithi018 ceph-mon[40196]: 4.1e scrub starts 2024-09-10T19:07:49.412 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:49 smithi018 ceph-mon[40196]: 4.1e scrub ok 2024-09-10T19:07:49.412 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:49 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2212566529' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-10T19:07:49.412 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:49 smithi018 ceph-mon[40196]: osdmap e478: 8 total, 8 up, 8 in 2024-09-10T19:07:49.412 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:49 smithi018 ceph-mon[40196]: 5.15 scrub starts 2024-09-10T19:07:49.412 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:49 smithi018 ceph-mon[40196]: 5.15 scrub ok 2024-09-10T19:07:49.412 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:49 smithi018 ceph-mon[40196]: 4.14 scrub starts 2024-09-10T19:07:49.412 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:49 smithi018 ceph-mon[40196]: 4.14 scrub ok 2024-09-10T19:07:49.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:49 smithi103 ceph-mon[33163]: 4.1e scrub starts 2024-09-10T19:07:49.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:49 smithi103 ceph-mon[33163]: 4.1e scrub ok 2024-09-10T19:07:49.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:49 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2212566529' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-10T19:07:49.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:49 smithi103 ceph-mon[33163]: osdmap e478: 8 total, 8 up, 8 in 2024-09-10T19:07:49.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:49 smithi103 ceph-mon[33163]: 5.15 scrub starts 2024-09-10T19:07:49.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:49 smithi103 ceph-mon[33163]: 5.15 scrub ok 2024-09-10T19:07:49.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:49 smithi103 ceph-mon[33163]: 4.14 scrub starts 2024-09-10T19:07:49.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:49 smithi103 ceph-mon[33163]: 4.14 scrub ok 2024-09-10T19:07:50.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:07:50 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:07:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:07:50.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:50 smithi103 ceph-mon[33163]: 1.11 scrub starts 2024-09-10T19:07:50.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:50 smithi103 ceph-mon[33163]: 1.11 scrub ok 2024-09-10T19:07:50.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:50 smithi103 ceph-mon[33163]: 6.19 scrub starts 2024-09-10T19:07:50.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:50 smithi103 ceph-mon[33163]: 6.19 scrub ok 2024-09-10T19:07:50.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:50 smithi103 ceph-mon[33163]: 1.b scrub starts 2024-09-10T19:07:50.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:50 smithi103 ceph-mon[33163]: 1.b scrub ok 2024-09-10T19:07:50.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:50 smithi103 ceph-mon[33163]: pgmap v1068: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 699 B/s rd, 0 op/s 2024-09-10T19:07:50.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:50 smithi103 ceph-mon[33163]: 2.19 scrub starts 2024-09-10T19:07:50.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:50 smithi103 ceph-mon[33163]: 2.19 scrub ok 2024-09-10T19:07:50.524 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:50 smithi018 ceph-mon[40196]: 1.11 scrub starts 2024-09-10T19:07:50.524 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:50 smithi018 ceph-mon[40196]: 1.11 scrub ok 2024-09-10T19:07:50.524 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:50 smithi018 ceph-mon[40196]: 6.19 scrub starts 2024-09-10T19:07:50.524 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:50 smithi018 ceph-mon[40196]: 6.19 scrub ok 2024-09-10T19:07:50.524 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:50 smithi018 ceph-mon[40196]: 1.b scrub starts 2024-09-10T19:07:50.524 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:50 smithi018 ceph-mon[40196]: 1.b scrub ok 2024-09-10T19:07:50.524 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:50 smithi018 ceph-mon[40196]: pgmap v1068: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 699 B/s rd, 0 op/s 2024-09-10T19:07:50.524 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:50 smithi018 ceph-mon[40196]: 2.19 scrub starts 2024-09-10T19:07:50.524 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:50 smithi018 ceph-mon[40196]: 2.19 scrub ok 2024-09-10T19:07:50.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:50 smithi018 ceph-mon[29336]: 1.11 scrub starts 2024-09-10T19:07:50.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:50 smithi018 ceph-mon[29336]: 1.11 scrub ok 2024-09-10T19:07:50.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:50 smithi018 ceph-mon[29336]: 6.19 scrub starts 2024-09-10T19:07:50.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:50 smithi018 ceph-mon[29336]: 6.19 scrub ok 2024-09-10T19:07:50.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:50 smithi018 ceph-mon[29336]: 1.b scrub starts 2024-09-10T19:07:50.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:50 smithi018 ceph-mon[29336]: 1.b scrub ok 2024-09-10T19:07:50.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:50 smithi018 ceph-mon[29336]: pgmap v1068: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 699 B/s rd, 0 op/s 2024-09-10T19:07:50.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:50 smithi018 ceph-mon[29336]: 2.19 scrub starts 2024-09-10T19:07:50.525 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:50 smithi018 ceph-mon[29336]: 2.19 scrub ok 2024-09-10T19:07:51.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:07:50 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:07:50] "GET /metrics HTTP/1.1" 200 222137 "" "Prometheus/2.43.0" 2024-09-10T19:07:51.248 INFO:teuthology.orchestra.run.smithi018.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-10T19:07:51.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:51 smithi103 ceph-mon[33163]: 5.18 scrub starts 2024-09-10T19:07:51.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:51 smithi103 ceph-mon[33163]: 5.18 scrub ok 2024-09-10T19:07:51.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:51 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3709336334' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-09-10T19:07:51.548 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:51 smithi018 ceph-mon[40196]: 5.18 scrub starts 2024-09-10T19:07:51.549 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:51 smithi018 ceph-mon[40196]: 5.18 scrub ok 2024-09-10T19:07:51.549 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:51 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3709336334' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-09-10T19:07:51.549 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:51 smithi018 ceph-mon[29336]: 5.18 scrub starts 2024-09-10T19:07:51.549 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:51 smithi018 ceph-mon[29336]: 5.18 scrub ok 2024-09-10T19:07:51.549 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:51 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3709336334' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-09-10T19:07:51.868 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd reweight 5 1 2024-09-10T19:07:52.125 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:07:52.499 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:52 smithi018 ceph-mon[40196]: 5.d scrub starts 2024-09-10T19:07:52.499 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:52 smithi018 ceph-mon[40196]: 5.d scrub ok 2024-09-10T19:07:52.500 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:52 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3709336334' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]': finished 2024-09-10T19:07:52.500 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:52 smithi018 ceph-mon[40196]: osdmap e479: 8 total, 8 up, 8 in 2024-09-10T19:07:52.500 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:52 smithi018 ceph-mon[40196]: 2.15 scrub starts 2024-09-10T19:07:52.500 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:52 smithi018 ceph-mon[40196]: 2.15 scrub ok 2024-09-10T19:07:52.500 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:52 smithi018 ceph-mon[40196]: pgmap v1070: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 699 B/s rd, 0 op/s 2024-09-10T19:07:52.500 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:52 smithi018 ceph-mon[29336]: 5.d scrub starts 2024-09-10T19:07:52.500 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:52 smithi018 ceph-mon[29336]: 5.d scrub ok 2024-09-10T19:07:52.500 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:52 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3709336334' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]': finished 2024-09-10T19:07:52.501 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:52 smithi018 ceph-mon[29336]: osdmap e479: 8 total, 8 up, 8 in 2024-09-10T19:07:52.501 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:52 smithi018 ceph-mon[29336]: 2.15 scrub starts 2024-09-10T19:07:52.501 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:52 smithi018 ceph-mon[29336]: 2.15 scrub ok 2024-09-10T19:07:52.501 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:52 smithi018 ceph-mon[29336]: pgmap v1070: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 699 B/s rd, 0 op/s 2024-09-10T19:07:52.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:52 smithi103 ceph-mon[33163]: 5.d scrub starts 2024-09-10T19:07:52.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:52 smithi103 ceph-mon[33163]: 5.d scrub ok 2024-09-10T19:07:52.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:52 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3709336334' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]': finished 2024-09-10T19:07:52.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:52 smithi103 ceph-mon[33163]: osdmap e479: 8 total, 8 up, 8 in 2024-09-10T19:07:52.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:52 smithi103 ceph-mon[33163]: 2.15 scrub starts 2024-09-10T19:07:52.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:52 smithi103 ceph-mon[33163]: 2.15 scrub ok 2024-09-10T19:07:52.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:52 smithi103 ceph-mon[33163]: pgmap v1070: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 699 B/s rd, 0 op/s 2024-09-10T19:07:53.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:53 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:07:53.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:53 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:07:53.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:53 smithi103 ceph-mon[33163]: osdmap e480: 8 total, 8 up, 8 in 2024-09-10T19:07:53.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:53 smithi103 ceph-mon[33163]: 6.a scrub starts 2024-09-10T19:07:53.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:53 smithi103 ceph-mon[33163]: 6.a scrub ok 2024-09-10T19:07:53.578 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:53 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:07:53.579 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:53 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:07:53.579 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:53 smithi018 ceph-mon[29336]: osdmap e480: 8 total, 8 up, 8 in 2024-09-10T19:07:53.579 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:53 smithi018 ceph-mon[29336]: 6.a scrub starts 2024-09-10T19:07:53.579 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:53 smithi018 ceph-mon[29336]: 6.a scrub ok 2024-09-10T19:07:53.579 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:53 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:07:53.579 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:53 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:07:53.579 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:53 smithi018 ceph-mon[40196]: osdmap e480: 8 total, 8 up, 8 in 2024-09-10T19:07:53.579 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:53 smithi018 ceph-mon[40196]: 6.a scrub starts 2024-09-10T19:07:53.579 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:53 smithi018 ceph-mon[40196]: 6.a scrub ok 2024-09-10T19:07:54.261 INFO:teuthology.orchestra.run.smithi018.stderr:reweighted osd.5 to 1 (10000) 2024-09-10T19:07:54.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:54 smithi103 ceph-mon[33163]: 5.8 scrub starts 2024-09-10T19:07:54.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:54 smithi103 ceph-mon[33163]: 5.8 scrub ok 2024-09-10T19:07:54.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:54 smithi103 ceph-mon[33163]: 2.6 scrub starts 2024-09-10T19:07:54.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:54 smithi103 ceph-mon[33163]: 2.6 scrub ok 2024-09-10T19:07:54.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:54 smithi103 ceph-mon[33163]: pgmap v1072: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:54.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:54 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2506556636' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-10T19:07:54.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:54 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-10T19:07:54.558 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:54 smithi018 ceph-mon[40196]: 5.8 scrub starts 2024-09-10T19:07:54.558 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:54 smithi018 ceph-mon[40196]: 5.8 scrub ok 2024-09-10T19:07:54.558 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:54 smithi018 ceph-mon[40196]: 2.6 scrub starts 2024-09-10T19:07:54.558 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:54 smithi018 ceph-mon[40196]: 2.6 scrub ok 2024-09-10T19:07:54.558 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:54 smithi018 ceph-mon[40196]: pgmap v1072: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:54.558 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:54 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2506556636' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-10T19:07:54.558 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:54 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-10T19:07:54.558 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:54 smithi018 ceph-mon[29336]: 5.8 scrub starts 2024-09-10T19:07:54.558 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:54 smithi018 ceph-mon[29336]: 5.8 scrub ok 2024-09-10T19:07:54.558 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:54 smithi018 ceph-mon[29336]: 2.6 scrub starts 2024-09-10T19:07:54.558 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:54 smithi018 ceph-mon[29336]: 2.6 scrub ok 2024-09-10T19:07:54.559 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:54 smithi018 ceph-mon[29336]: pgmap v1072: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:54.559 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:54 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2506556636' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-10T19:07:54.559 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:54 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-10T19:07:54.993 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd primary-affinity 5 1 2024-09-10T19:07:55.261 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:07:55.263 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:55 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-10T19:07:55.263 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:55 smithi018 ceph-mon[29336]: osdmap e481: 8 total, 8 up, 8 in 2024-09-10T19:07:55.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:55 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-10T19:07:55.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:55 smithi103 ceph-mon[33163]: osdmap e481: 8 total, 8 up, 8 in 2024-09-10T19:07:55.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:55 smithi103 ceph-mon[33163]: 2.a scrub starts 2024-09-10T19:07:55.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:55 smithi103 ceph-mon[33163]: 2.a scrub ok 2024-09-10T19:07:55.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:07:55.538 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:55 smithi018 ceph-mon[29336]: 2.a scrub starts 2024-09-10T19:07:55.538 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:55 smithi018 ceph-mon[29336]: 2.a scrub ok 2024-09-10T19:07:55.538 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:07:55.538 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:55 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-10T19:07:55.538 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:55 smithi018 ceph-mon[40196]: osdmap e481: 8 total, 8 up, 8 in 2024-09-10T19:07:55.538 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:55 smithi018 ceph-mon[40196]: 2.a scrub starts 2024-09-10T19:07:55.538 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:55 smithi018 ceph-mon[40196]: 2.a scrub ok 2024-09-10T19:07:55.539 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:07:56.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:56 smithi103 ceph-mon[33163]: 4.5 scrub starts 2024-09-10T19:07:56.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:56 smithi103 ceph-mon[33163]: 4.5 scrub ok 2024-09-10T19:07:56.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:56 smithi103 ceph-mon[33163]: 2.e scrub starts 2024-09-10T19:07:56.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:56 smithi103 ceph-mon[33163]: 2.e scrub ok 2024-09-10T19:07:56.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:56 smithi103 ceph-mon[33163]: 6.9 scrub starts 2024-09-10T19:07:56.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:56 smithi103 ceph-mon[33163]: 6.9 scrub ok 2024-09-10T19:07:56.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:56 smithi103 ceph-mon[33163]: pgmap v1074: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:56.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:56 smithi103 ceph-mon[33163]: 5.0 scrub starts 2024-09-10T19:07:56.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:56 smithi103 ceph-mon[33163]: 5.0 scrub ok 2024-09-10T19:07:56.680 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:56 smithi018 ceph-mon[40196]: 4.5 scrub starts 2024-09-10T19:07:56.680 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:56 smithi018 ceph-mon[40196]: 4.5 scrub ok 2024-09-10T19:07:56.680 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:56 smithi018 ceph-mon[40196]: 2.e scrub starts 2024-09-10T19:07:56.680 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:56 smithi018 ceph-mon[40196]: 2.e scrub ok 2024-09-10T19:07:56.680 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:56 smithi018 ceph-mon[40196]: 6.9 scrub starts 2024-09-10T19:07:56.680 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:56 smithi018 ceph-mon[40196]: 6.9 scrub ok 2024-09-10T19:07:56.680 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:56 smithi018 ceph-mon[40196]: pgmap v1074: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:56.680 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:56 smithi018 ceph-mon[40196]: 5.0 scrub starts 2024-09-10T19:07:56.681 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:56 smithi018 ceph-mon[40196]: 5.0 scrub ok 2024-09-10T19:07:56.681 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:56 smithi018 ceph-mon[29336]: 4.5 scrub starts 2024-09-10T19:07:56.681 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:56 smithi018 ceph-mon[29336]: 4.5 scrub ok 2024-09-10T19:07:56.681 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:56 smithi018 ceph-mon[29336]: 2.e scrub starts 2024-09-10T19:07:56.681 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:56 smithi018 ceph-mon[29336]: 2.e scrub ok 2024-09-10T19:07:56.681 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:56 smithi018 ceph-mon[29336]: 6.9 scrub starts 2024-09-10T19:07:56.681 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:56 smithi018 ceph-mon[29336]: 6.9 scrub ok 2024-09-10T19:07:56.681 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:56 smithi018 ceph-mon[29336]: pgmap v1074: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:07:56.681 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:56 smithi018 ceph-mon[29336]: 5.0 scrub starts 2024-09-10T19:07:56.681 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:56 smithi018 ceph-mon[29336]: 5.0 scrub ok 2024-09-10T19:07:57.277 INFO:teuthology.orchestra.run.smithi018.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-10T19:07:57.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:57 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1265586151' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-09-10T19:07:57.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:57 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-09-10T19:07:57.523 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:07:57 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:07:57.574 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:57 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1265586151' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-09-10T19:07:57.574 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:57 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-09-10T19:07:57.574 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:57 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1265586151' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-09-10T19:07:57.575 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:57 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-09-10T19:07:58.037 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd reweight 6 1 2024-09-10T19:07:58.306 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:07:58.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:58 smithi103 ceph-mon[33163]: 1.d scrub starts 2024-09-10T19:07:58.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:58 smithi103 ceph-mon[33163]: 1.d scrub ok 2024-09-10T19:07:58.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:58 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:07:58.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:58 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:07:58.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:58 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]': finished 2024-09-10T19:07:58.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:58 smithi103 ceph-mon[33163]: osdmap e482: 8 total, 8 up, 8 in 2024-09-10T19:07:58.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:58 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:58.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:58 smithi103 ceph-mon[33163]: pgmap v1076: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T19:07:58.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:58 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2024-09-10T19:07:58.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:58 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 4]}]: dispatch 2024-09-10T19:07:58.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:58 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2024-09-10T19:07:58.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:58 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-09-10T19:07:58.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:58 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 4]}]: dispatch 2024-09-10T19:07:58.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:58 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-09-10T19:07:58.569 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:58 smithi018 ceph-mon[40196]: 1.d scrub starts 2024-09-10T19:07:58.569 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:58 smithi018 ceph-mon[40196]: 1.d scrub ok 2024-09-10T19:07:58.569 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:58 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:07:58.569 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:58 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:07:58.569 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:58 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]': finished 2024-09-10T19:07:58.570 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:58 smithi018 ceph-mon[40196]: osdmap e482: 8 total, 8 up, 8 in 2024-09-10T19:07:58.570 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:58 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:58.570 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:58 smithi018 ceph-mon[40196]: pgmap v1076: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T19:07:58.570 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:58 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2024-09-10T19:07:58.570 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:58 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 4]}]: dispatch 2024-09-10T19:07:58.570 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:58 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2024-09-10T19:07:58.570 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:58 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-09-10T19:07:58.570 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:58 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 4]}]: dispatch 2024-09-10T19:07:58.570 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:58 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-09-10T19:07:58.570 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:58 smithi018 ceph-mon[29336]: 1.d scrub starts 2024-09-10T19:07:58.570 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:58 smithi018 ceph-mon[29336]: 1.d scrub ok 2024-09-10T19:07:58.571 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:58 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:07:58.571 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:58 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:07:58.571 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:58 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]': finished 2024-09-10T19:07:58.571 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:58 smithi018 ceph-mon[29336]: osdmap e482: 8 total, 8 up, 8 in 2024-09-10T19:07:58.571 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:58 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:58.571 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:58 smithi018 ceph-mon[29336]: pgmap v1076: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T19:07:58.571 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:58 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2024-09-10T19:07:58.571 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:58 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 4]}]: dispatch 2024-09-10T19:07:58.571 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:58 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2024-09-10T19:07:58.571 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:58 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-09-10T19:07:58.571 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:58 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 4]}]: dispatch 2024-09-10T19:07:58.571 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:58 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-09-10T19:07:59.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:59 smithi018 ceph-mon[29336]: 5.1 scrub starts 2024-09-10T19:07:59.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:59 smithi018 ceph-mon[29336]: 5.1 scrub ok 2024-09-10T19:07:59.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:59 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]': finished 2024-09-10T19:07:59.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:59 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 4]}]': finished 2024-09-10T19:07:59.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:59 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]': finished 2024-09-10T19:07:59.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:59 smithi018 ceph-mon[29336]: osdmap e483: 8 total, 8 up, 8 in 2024-09-10T19:07:59.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:59 smithi018 ceph-mon[29336]: 2.18 scrub starts 2024-09-10T19:07:59.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:59 smithi018 ceph-mon[29336]: 2.18 scrub ok 2024-09-10T19:07:59.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:59 smithi018 ceph-mon[29336]: osdmap e484: 8 total, 8 up, 8 in 2024-09-10T19:07:59.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:59 smithi018 ceph-mon[29336]: 6.13 scrub starts 2024-09-10T19:07:59.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:07:59 smithi018 ceph-mon[29336]: 6.13 scrub ok 2024-09-10T19:07:59.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:59 smithi018 ceph-mon[40196]: 5.1 scrub starts 2024-09-10T19:07:59.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:59 smithi018 ceph-mon[40196]: 5.1 scrub ok 2024-09-10T19:07:59.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:59 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]': finished 2024-09-10T19:07:59.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:59 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 4]}]': finished 2024-09-10T19:07:59.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:59 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]': finished 2024-09-10T19:07:59.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:59 smithi018 ceph-mon[40196]: osdmap e483: 8 total, 8 up, 8 in 2024-09-10T19:07:59.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:59 smithi018 ceph-mon[40196]: 2.18 scrub starts 2024-09-10T19:07:59.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:59 smithi018 ceph-mon[40196]: 2.18 scrub ok 2024-09-10T19:07:59.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:59 smithi018 ceph-mon[40196]: osdmap e484: 8 total, 8 up, 8 in 2024-09-10T19:07:59.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:59 smithi018 ceph-mon[40196]: 6.13 scrub starts 2024-09-10T19:07:59.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:07:59 smithi018 ceph-mon[40196]: 6.13 scrub ok 2024-09-10T19:07:59.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:59 smithi103 ceph-mon[33163]: 5.1 scrub starts 2024-09-10T19:07:59.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:59 smithi103 ceph-mon[33163]: 5.1 scrub ok 2024-09-10T19:07:59.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:59 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]': finished 2024-09-10T19:07:59.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:59 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [7, 4]}]': finished 2024-09-10T19:07:59.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:59 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]': finished 2024-09-10T19:07:59.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:59 smithi103 ceph-mon[33163]: osdmap e483: 8 total, 8 up, 8 in 2024-09-10T19:07:59.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:59 smithi103 ceph-mon[33163]: 2.18 scrub starts 2024-09-10T19:07:59.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:59 smithi103 ceph-mon[33163]: 2.18 scrub ok 2024-09-10T19:07:59.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:59 smithi103 ceph-mon[33163]: osdmap e484: 8 total, 8 up, 8 in 2024-09-10T19:07:59.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:59 smithi103 ceph-mon[33163]: 6.13 scrub starts 2024-09-10T19:07:59.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:07:59 smithi103 ceph-mon[33163]: 6.13 scrub ok 2024-09-10T19:08:00.285 INFO:teuthology.orchestra.run.smithi018.stderr:reweighted osd.6 to 1 (10000) 2024-09-10T19:08:00.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:08:00 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:08:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:08:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:00 smithi103 ceph-mon[33163]: 6.16 scrub starts 2024-09-10T19:08:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:00 smithi103 ceph-mon[33163]: 6.16 scrub ok 2024-09-10T19:08:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:00 smithi103 ceph-mon[33163]: 1.f deep-scrub starts 2024-09-10T19:08:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:00 smithi103 ceph-mon[33163]: 1.f deep-scrub ok 2024-09-10T19:08:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:00 smithi103 ceph-mon[33163]: 2.f scrub starts 2024-09-10T19:08:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:00 smithi103 ceph-mon[33163]: 2.f scrub ok 2024-09-10T19:08:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:00 smithi103 ceph-mon[33163]: pgmap v1079: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 969 B/s rd, 0 op/s 2024-09-10T19:08:00.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:00 smithi103 ceph-mon[33163]: 5.17 scrub starts 2024-09-10T19:08:00.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:00 smithi103 ceph-mon[33163]: 5.17 scrub ok 2024-09-10T19:08:00.524 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:00 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3672652309' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-10T19:08:00.579 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:00 smithi018 ceph-mon[29336]: 6.16 scrub starts 2024-09-10T19:08:00.579 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:00 smithi018 ceph-mon[29336]: 6.16 scrub ok 2024-09-10T19:08:00.580 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:00 smithi018 ceph-mon[29336]: 1.f deep-scrub starts 2024-09-10T19:08:00.580 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:00 smithi018 ceph-mon[29336]: 1.f deep-scrub ok 2024-09-10T19:08:00.580 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:00 smithi018 ceph-mon[29336]: 2.f scrub starts 2024-09-10T19:08:00.580 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:00 smithi018 ceph-mon[29336]: 2.f scrub ok 2024-09-10T19:08:00.580 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:00 smithi018 ceph-mon[29336]: pgmap v1079: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 969 B/s rd, 0 op/s 2024-09-10T19:08:00.580 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:00 smithi018 ceph-mon[29336]: 5.17 scrub starts 2024-09-10T19:08:00.580 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:00 smithi018 ceph-mon[29336]: 5.17 scrub ok 2024-09-10T19:08:00.580 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:00 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3672652309' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-10T19:08:00.580 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:00 smithi018 ceph-mon[40196]: 6.16 scrub starts 2024-09-10T19:08:00.581 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:00 smithi018 ceph-mon[40196]: 6.16 scrub ok 2024-09-10T19:08:00.581 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:00 smithi018 ceph-mon[40196]: 1.f deep-scrub starts 2024-09-10T19:08:00.581 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:00 smithi018 ceph-mon[40196]: 1.f deep-scrub ok 2024-09-10T19:08:00.581 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:00 smithi018 ceph-mon[40196]: 2.f scrub starts 2024-09-10T19:08:00.581 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:00 smithi018 ceph-mon[40196]: 2.f scrub ok 2024-09-10T19:08:00.581 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:00 smithi018 ceph-mon[40196]: pgmap v1079: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 969 B/s rd, 0 op/s 2024-09-10T19:08:00.581 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:00 smithi018 ceph-mon[40196]: 5.17 scrub starts 2024-09-10T19:08:00.581 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:00 smithi018 ceph-mon[40196]: 5.17 scrub ok 2024-09-10T19:08:00.581 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:00 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3672652309' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-10T19:08:00.992 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd primary-affinity 6 1 2024-09-10T19:08:01.261 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:08:01.287 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:08:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:08:00] "GET /metrics HTTP/1.1" 200 222124 "" "Prometheus/2.43.0" 2024-09-10T19:08:01.546 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:01 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3672652309' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-10T19:08:01.546 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:01 smithi018 ceph-mon[40196]: osdmap e485: 8 total, 8 up, 8 in 2024-09-10T19:08:01.546 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:01 smithi018 ceph-mon[40196]: 6.15 deep-scrub starts 2024-09-10T19:08:01.546 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:01 smithi018 ceph-mon[40196]: 6.15 deep-scrub ok 2024-09-10T19:08:01.546 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:01 smithi018 ceph-mon[40196]: 6.11 scrub starts 2024-09-10T19:08:01.546 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:01 smithi018 ceph-mon[40196]: 6.11 scrub ok 2024-09-10T19:08:01.546 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:01 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3672652309' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-10T19:08:01.547 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:01 smithi018 ceph-mon[29336]: osdmap e485: 8 total, 8 up, 8 in 2024-09-10T19:08:01.547 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:01 smithi018 ceph-mon[29336]: 6.15 deep-scrub starts 2024-09-10T19:08:01.547 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:01 smithi018 ceph-mon[29336]: 6.15 deep-scrub ok 2024-09-10T19:08:01.547 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:01 smithi018 ceph-mon[29336]: 6.11 scrub starts 2024-09-10T19:08:01.547 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:01 smithi018 ceph-mon[29336]: 6.11 scrub ok 2024-09-10T19:08:01.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:01 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3672652309' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-10T19:08:01.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:01 smithi103 ceph-mon[33163]: osdmap e485: 8 total, 8 up, 8 in 2024-09-10T19:08:01.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:01 smithi103 ceph-mon[33163]: 6.15 deep-scrub starts 2024-09-10T19:08:01.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:01 smithi103 ceph-mon[33163]: 6.15 deep-scrub ok 2024-09-10T19:08:01.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:01 smithi103 ceph-mon[33163]: 6.11 scrub starts 2024-09-10T19:08:01.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:01 smithi103 ceph-mon[33163]: 6.11 scrub ok 2024-09-10T19:08:02.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:02 smithi018 ceph-mon[29336]: 5.11 deep-scrub starts 2024-09-10T19:08:02.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:02 smithi018 ceph-mon[29336]: 5.11 deep-scrub ok 2024-09-10T19:08:02.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:02 smithi018 ceph-mon[29336]: 2.5 scrub starts 2024-09-10T19:08:02.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:02 smithi018 ceph-mon[29336]: 2.5 scrub ok 2024-09-10T19:08:02.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:02 smithi018 ceph-mon[29336]: pgmap v1081: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 0 B/s, 1 objects/s recovering 2024-09-10T19:08:02.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:02 smithi018 ceph-mon[40196]: 5.11 deep-scrub starts 2024-09-10T19:08:02.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:02 smithi018 ceph-mon[40196]: 5.11 deep-scrub ok 2024-09-10T19:08:02.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:02 smithi018 ceph-mon[40196]: 2.5 scrub starts 2024-09-10T19:08:02.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:02 smithi018 ceph-mon[40196]: 2.5 scrub ok 2024-09-10T19:08:02.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:02 smithi018 ceph-mon[40196]: pgmap v1081: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 0 B/s, 1 objects/s recovering 2024-09-10T19:08:02.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:02 smithi103 ceph-mon[33163]: 5.11 deep-scrub starts 2024-09-10T19:08:02.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:02 smithi103 ceph-mon[33163]: 5.11 deep-scrub ok 2024-09-10T19:08:02.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:02 smithi103 ceph-mon[33163]: 2.5 scrub starts 2024-09-10T19:08:02.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:02 smithi103 ceph-mon[33163]: 2.5 scrub ok 2024-09-10T19:08:02.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:02 smithi103 ceph-mon[33163]: pgmap v1081: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 0 B/s, 1 objects/s recovering 2024-09-10T19:08:03.297 INFO:teuthology.orchestra.run.smithi018.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-10T19:08:03.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:03 smithi018 ceph-mon[29336]: 4.3 scrub starts 2024-09-10T19:08:03.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:03 smithi018 ceph-mon[29336]: 4.3 scrub ok 2024-09-10T19:08:03.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:03 smithi018 ceph-mon[29336]: 4.9 scrub starts 2024-09-10T19:08:03.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:03 smithi018 ceph-mon[29336]: 4.9 scrub ok 2024-09-10T19:08:03.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:03 smithi018 ceph-mon[29336]: 1.10 scrub starts 2024-09-10T19:08:03.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:03 smithi018 ceph-mon[29336]: 1.10 scrub ok 2024-09-10T19:08:03.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:03 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/444309928' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]: dispatch 2024-09-10T19:08:03.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:03 smithi018 ceph-mon[40196]: 4.3 scrub starts 2024-09-10T19:08:03.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:03 smithi018 ceph-mon[40196]: 4.3 scrub ok 2024-09-10T19:08:03.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:03 smithi018 ceph-mon[40196]: 4.9 scrub starts 2024-09-10T19:08:03.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:03 smithi018 ceph-mon[40196]: 4.9 scrub ok 2024-09-10T19:08:03.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:03 smithi018 ceph-mon[40196]: 1.10 scrub starts 2024-09-10T19:08:03.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:03 smithi018 ceph-mon[40196]: 1.10 scrub ok 2024-09-10T19:08:03.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:03 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/444309928' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]: dispatch 2024-09-10T19:08:03.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:03 smithi103 ceph-mon[33163]: 4.3 scrub starts 2024-09-10T19:08:03.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:03 smithi103 ceph-mon[33163]: 4.3 scrub ok 2024-09-10T19:08:03.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:03 smithi103 ceph-mon[33163]: 4.9 scrub starts 2024-09-10T19:08:03.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:03 smithi103 ceph-mon[33163]: 4.9 scrub ok 2024-09-10T19:08:03.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:03 smithi103 ceph-mon[33163]: 1.10 scrub starts 2024-09-10T19:08:03.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:03 smithi103 ceph-mon[33163]: 1.10 scrub ok 2024-09-10T19:08:03.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:03 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/444309928' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]: dispatch 2024-09-10T19:08:04.038 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd reweight 0 1 2024-09-10T19:08:04.304 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:08:04.564 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:04 smithi018 ceph-mon[40196]: 1.6 scrub starts 2024-09-10T19:08:04.564 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:04 smithi018 ceph-mon[40196]: 1.6 scrub ok 2024-09-10T19:08:04.564 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:04 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/444309928' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-09-10T19:08:04.564 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:04 smithi018 ceph-mon[40196]: osdmap e486: 8 total, 8 up, 8 in 2024-09-10T19:08:04.564 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:04 smithi018 ceph-mon[40196]: 2.0 scrub starts 2024-09-10T19:08:04.565 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:04 smithi018 ceph-mon[40196]: 2.0 scrub ok 2024-09-10T19:08:04.565 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:04 smithi018 ceph-mon[40196]: pgmap v1083: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 972 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T19:08:04.565 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:04 smithi018 ceph-mon[40196]: 6.1b scrub starts 2024-09-10T19:08:04.565 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:04 smithi018 ceph-mon[40196]: 6.1b scrub ok 2024-09-10T19:08:04.565 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:04 smithi018 ceph-mon[29336]: 1.6 scrub starts 2024-09-10T19:08:04.565 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:04 smithi018 ceph-mon[29336]: 1.6 scrub ok 2024-09-10T19:08:04.565 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:04 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/444309928' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-09-10T19:08:04.565 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:04 smithi018 ceph-mon[29336]: osdmap e486: 8 total, 8 up, 8 in 2024-09-10T19:08:04.565 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:04 smithi018 ceph-mon[29336]: 2.0 scrub starts 2024-09-10T19:08:04.565 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:04 smithi018 ceph-mon[29336]: 2.0 scrub ok 2024-09-10T19:08:04.565 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:04 smithi018 ceph-mon[29336]: pgmap v1083: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 972 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T19:08:04.566 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:04 smithi018 ceph-mon[29336]: 6.1b scrub starts 2024-09-10T19:08:04.566 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:04 smithi018 ceph-mon[29336]: 6.1b scrub ok 2024-09-10T19:08:04.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:04 smithi103 ceph-mon[33163]: 1.6 scrub starts 2024-09-10T19:08:04.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:04 smithi103 ceph-mon[33163]: 1.6 scrub ok 2024-09-10T19:08:04.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:04 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/444309928' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-09-10T19:08:04.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:04 smithi103 ceph-mon[33163]: osdmap e486: 8 total, 8 up, 8 in 2024-09-10T19:08:04.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:04 smithi103 ceph-mon[33163]: 2.0 scrub starts 2024-09-10T19:08:04.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:04 smithi103 ceph-mon[33163]: 2.0 scrub ok 2024-09-10T19:08:04.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:04 smithi103 ceph-mon[33163]: pgmap v1083: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 972 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T19:08:04.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:04 smithi103 ceph-mon[33163]: 6.1b scrub starts 2024-09-10T19:08:04.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:04 smithi103 ceph-mon[33163]: 6.1b scrub ok 2024-09-10T19:08:05.568 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:05 smithi018 ceph-mon[40196]: 6.18 scrub starts 2024-09-10T19:08:05.568 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:05 smithi018 ceph-mon[40196]: 6.18 scrub ok 2024-09-10T19:08:05.568 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:05 smithi018 ceph-mon[40196]: 6.e scrub starts 2024-09-10T19:08:05.568 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:05 smithi018 ceph-mon[40196]: 6.e scrub ok 2024-09-10T19:08:05.568 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:05 smithi018 ceph-mon[40196]: 5.1a scrub starts 2024-09-10T19:08:05.569 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:05 smithi018 ceph-mon[40196]: 5.1a scrub ok 2024-09-10T19:08:05.569 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:05 smithi018 ceph-mon[29336]: 6.18 scrub starts 2024-09-10T19:08:05.569 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:05 smithi018 ceph-mon[29336]: 6.18 scrub ok 2024-09-10T19:08:05.569 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:05 smithi018 ceph-mon[29336]: 6.e scrub starts 2024-09-10T19:08:05.569 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:05 smithi018 ceph-mon[29336]: 6.e scrub ok 2024-09-10T19:08:05.569 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:05 smithi018 ceph-mon[29336]: 5.1a scrub starts 2024-09-10T19:08:05.569 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:05 smithi018 ceph-mon[29336]: 5.1a scrub ok 2024-09-10T19:08:05.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:05 smithi103 ceph-mon[33163]: 6.18 scrub starts 2024-09-10T19:08:05.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:05 smithi103 ceph-mon[33163]: 6.18 scrub ok 2024-09-10T19:08:05.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:05 smithi103 ceph-mon[33163]: 6.e scrub starts 2024-09-10T19:08:05.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:05 smithi103 ceph-mon[33163]: 6.e scrub ok 2024-09-10T19:08:05.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:05 smithi103 ceph-mon[33163]: 5.1a scrub starts 2024-09-10T19:08:05.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:05 smithi103 ceph-mon[33163]: 5.1a scrub ok 2024-09-10T19:08:06.309 INFO:teuthology.orchestra.run.smithi018.stderr:reweighted osd.0 to 1 (10000) 2024-09-10T19:08:06.682 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:06 smithi018 ceph-mon[40196]: 6.7 scrub starts 2024-09-10T19:08:06.682 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:06 smithi018 ceph-mon[40196]: 6.7 scrub ok 2024-09-10T19:08:06.682 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:06 smithi018 ceph-mon[40196]: pgmap v1084: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 764 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T19:08:06.682 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:06 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3890886086' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-10T19:08:06.682 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:06 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-10T19:08:06.682 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:06 smithi018 ceph-mon[29336]: 6.7 scrub starts 2024-09-10T19:08:06.683 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:06 smithi018 ceph-mon[29336]: 6.7 scrub ok 2024-09-10T19:08:06.683 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:06 smithi018 ceph-mon[29336]: pgmap v1084: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 764 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T19:08:06.683 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:06 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3890886086' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-10T19:08:06.683 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:06 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-10T19:08:06.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:06 smithi103 ceph-mon[33163]: 6.7 scrub starts 2024-09-10T19:08:06.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:06 smithi103 ceph-mon[33163]: 6.7 scrub ok 2024-09-10T19:08:06.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:06 smithi103 ceph-mon[33163]: pgmap v1084: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 764 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T19:08:06.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:06 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3890886086' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-10T19:08:06.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:06 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-10T19:08:07.123 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd primary-affinity 0 1 2024-09-10T19:08:07.388 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:08:07.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:07 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-10T19:08:07.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:07 smithi018 ceph-mon[29336]: osdmap e487: 8 total, 8 up, 8 in 2024-09-10T19:08:07.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:07 smithi018 ceph-mon[29336]: 1.2 scrub starts 2024-09-10T19:08:07.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:07 smithi018 ceph-mon[29336]: 1.2 scrub ok 2024-09-10T19:08:07.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:07 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-10T19:08:07.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:07 smithi018 ceph-mon[40196]: osdmap e487: 8 total, 8 up, 8 in 2024-09-10T19:08:07.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:07 smithi018 ceph-mon[40196]: 1.2 scrub starts 2024-09-10T19:08:07.446 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:07 smithi018 ceph-mon[40196]: 1.2 scrub ok 2024-09-10T19:08:07.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:07 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-10T19:08:07.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:07 smithi103 ceph-mon[33163]: osdmap e487: 8 total, 8 up, 8 in 2024-09-10T19:08:07.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:07 smithi103 ceph-mon[33163]: 1.2 scrub starts 2024-09-10T19:08:07.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:07 smithi103 ceph-mon[33163]: 1.2 scrub ok 2024-09-10T19:08:07.773 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:08:07 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:08:08.507 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:08 smithi018 ceph-mon[40196]: 2.1c scrub starts 2024-09-10T19:08:08.508 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:08 smithi018 ceph-mon[40196]: 2.1c scrub ok 2024-09-10T19:08:08.508 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:08 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:08.508 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:08 smithi018 ceph-mon[40196]: pgmap v1086: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T19:08:08.508 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:08 smithi018 ceph-mon[29336]: 2.1c scrub starts 2024-09-10T19:08:08.509 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:08 smithi018 ceph-mon[29336]: 2.1c scrub ok 2024-09-10T19:08:08.509 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:08 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:08.509 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:08 smithi018 ceph-mon[29336]: pgmap v1086: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T19:08:08.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:08 smithi103 ceph-mon[33163]: 2.1c scrub starts 2024-09-10T19:08:08.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:08 smithi103 ceph-mon[33163]: 2.1c scrub ok 2024-09-10T19:08:08.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:08 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:08.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:08 smithi103 ceph-mon[33163]: pgmap v1086: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T19:08:09.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:09 smithi018 ceph-mon[29336]: 1.1 scrub starts 2024-09-10T19:08:09.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:09 smithi018 ceph-mon[29336]: 1.1 scrub ok 2024-09-10T19:08:09.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:09 smithi018 ceph-mon[29336]: osdmap e488: 8 total, 8 up, 8 in 2024-09-10T19:08:09.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:09 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2628727197' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-09-10T19:08:09.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:09 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-09-10T19:08:09.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:09 smithi018 ceph-mon[40196]: 1.1 scrub starts 2024-09-10T19:08:09.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:09 smithi018 ceph-mon[40196]: 1.1 scrub ok 2024-09-10T19:08:09.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:09 smithi018 ceph-mon[40196]: osdmap e488: 8 total, 8 up, 8 in 2024-09-10T19:08:09.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:09 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2628727197' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-09-10T19:08:09.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:09 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-09-10T19:08:09.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:09 smithi103 ceph-mon[33163]: 1.1 scrub starts 2024-09-10T19:08:09.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:09 smithi103 ceph-mon[33163]: 1.1 scrub ok 2024-09-10T19:08:09.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:09 smithi103 ceph-mon[33163]: osdmap e488: 8 total, 8 up, 8 in 2024-09-10T19:08:09.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:09 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2628727197' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-09-10T19:08:09.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:09 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-09-10T19:08:09.867 INFO:teuthology.orchestra.run.smithi018.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-10T19:08:10.522 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:10 smithi103 ceph-mon[33163]: pgmap v1088: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 819 B/s rd, 0 op/s 2024-09-10T19:08:10.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:10 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-09-10T19:08:10.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:10 smithi103 ceph-mon[33163]: osdmap e489: 8 total, 8 up, 8 in 2024-09-10T19:08:10.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:10 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:10.523 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:08:10 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:08:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:08:10.649 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:10 smithi018 ceph-mon[29336]: pgmap v1088: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 819 B/s rd, 0 op/s 2024-09-10T19:08:10.649 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:10 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-09-10T19:08:10.649 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:10 smithi018 ceph-mon[29336]: osdmap e489: 8 total, 8 up, 8 in 2024-09-10T19:08:10.649 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:10 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:10.650 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:10 smithi018 ceph-mon[40196]: pgmap v1088: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 819 B/s rd, 0 op/s 2024-09-10T19:08:10.650 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:10 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-09-10T19:08:10.650 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:10 smithi018 ceph-mon[40196]: osdmap e489: 8 total, 8 up, 8 in 2024-09-10T19:08:10.650 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:10 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:10.681 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd reweight 7 1 2024-09-10T19:08:10.944 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:08:11.223 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:08:10 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:08:10] "GET /metrics HTTP/1.1" 200 222111 "" "Prometheus/2.43.0" 2024-09-10T19:08:11.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:11 smithi018 ceph-mon[29336]: 5.10 deep-scrub starts 2024-09-10T19:08:11.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:11 smithi018 ceph-mon[29336]: 5.10 deep-scrub ok 2024-09-10T19:08:11.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:11 smithi018 ceph-mon[40196]: 5.10 deep-scrub starts 2024-09-10T19:08:11.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:11 smithi018 ceph-mon[40196]: 5.10 deep-scrub ok 2024-09-10T19:08:11.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:11 smithi103 ceph-mon[33163]: 5.10 deep-scrub starts 2024-09-10T19:08:11.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:11 smithi103 ceph-mon[33163]: 5.10 deep-scrub ok 2024-09-10T19:08:12.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:12 smithi018 ceph-mon[29336]: 5.7 scrub starts 2024-09-10T19:08:12.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:12 smithi018 ceph-mon[29336]: 5.7 scrub ok 2024-09-10T19:08:12.445 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:12 smithi018 ceph-mon[29336]: pgmap v1090: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:12.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:12 smithi018 ceph-mon[40196]: 5.7 scrub starts 2024-09-10T19:08:12.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:12 smithi018 ceph-mon[40196]: 5.7 scrub ok 2024-09-10T19:08:12.445 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:12 smithi018 ceph-mon[40196]: pgmap v1090: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:12.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:12 smithi103 ceph-mon[33163]: 5.7 scrub starts 2024-09-10T19:08:12.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:12 smithi103 ceph-mon[33163]: 5.7 scrub ok 2024-09-10T19:08:12.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:12 smithi103 ceph-mon[33163]: pgmap v1090: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:13.334 INFO:teuthology.orchestra.run.smithi018.stderr:reweighted osd.7 to 1 (10000) 2024-09-10T19:08:13.665 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:13 smithi018 ceph-mon[29336]: 2.3 scrub starts 2024-09-10T19:08:13.665 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:13 smithi018 ceph-mon[29336]: 2.3 scrub ok 2024-09-10T19:08:13.665 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:13 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/4246616775' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-10T19:08:13.665 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:13 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-10T19:08:13.666 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:13 smithi018 ceph-mon[40196]: 2.3 scrub starts 2024-09-10T19:08:13.666 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:13 smithi018 ceph-mon[40196]: 2.3 scrub ok 2024-09-10T19:08:13.666 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:13 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/4246616775' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-10T19:08:13.666 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:13 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-10T19:08:13.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:13 smithi103 ceph-mon[33163]: 2.3 scrub starts 2024-09-10T19:08:13.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:13 smithi103 ceph-mon[33163]: 2.3 scrub ok 2024-09-10T19:08:13.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:13 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/4246616775' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-10T19:08:13.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:13 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-10T19:08:14.080 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd primary-affinity 7 1 2024-09-10T19:08:14.347 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:08:14.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:14 smithi018 ceph-mon[29336]: 5.1e scrub starts 2024-09-10T19:08:14.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:14 smithi018 ceph-mon[29336]: 5.1e scrub ok 2024-09-10T19:08:14.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:14 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-10T19:08:14.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:14 smithi018 ceph-mon[29336]: osdmap e490: 8 total, 8 up, 8 in 2024-09-10T19:08:14.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:14 smithi018 ceph-mon[29336]: pgmap v1092: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:14.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:14 smithi018 ceph-mon[40196]: 5.1e scrub starts 2024-09-10T19:08:14.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:14 smithi018 ceph-mon[40196]: 5.1e scrub ok 2024-09-10T19:08:14.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:14 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-10T19:08:14.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:14 smithi018 ceph-mon[40196]: osdmap e490: 8 total, 8 up, 8 in 2024-09-10T19:08:14.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:14 smithi018 ceph-mon[40196]: pgmap v1092: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:14.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:14 smithi103 ceph-mon[33163]: 5.1e scrub starts 2024-09-10T19:08:14.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:14 smithi103 ceph-mon[33163]: 5.1e scrub ok 2024-09-10T19:08:14.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:14 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-10T19:08:14.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:14 smithi103 ceph-mon[33163]: osdmap e490: 8 total, 8 up, 8 in 2024-09-10T19:08:14.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:14 smithi103 ceph-mon[33163]: pgmap v1092: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:15.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:15 smithi018 ceph-mon[29336]: 6.10 scrub starts 2024-09-10T19:08:15.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:15 smithi018 ceph-mon[29336]: 6.10 scrub ok 2024-09-10T19:08:15.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:15 smithi018 ceph-mon[40196]: 6.10 scrub starts 2024-09-10T19:08:15.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:15 smithi018 ceph-mon[40196]: 6.10 scrub ok 2024-09-10T19:08:15.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:15 smithi103 ceph-mon[33163]: 6.10 scrub starts 2024-09-10T19:08:15.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:15 smithi103 ceph-mon[33163]: 6.10 scrub ok 2024-09-10T19:08:16.518 INFO:teuthology.orchestra.run.smithi018.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-10T19:08:16.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:16 smithi103 ceph-mon[33163]: 4.f scrub starts 2024-09-10T19:08:16.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:16 smithi103 ceph-mon[33163]: 4.f scrub ok 2024-09-10T19:08:16.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:16 smithi103 ceph-mon[33163]: pgmap v1093: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 765 B/s rd, 0 op/s 2024-09-10T19:08:16.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:16 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1557304565' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-09-10T19:08:16.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:16 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-09-10T19:08:16.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:16 smithi018 ceph-mon[40196]: 4.f scrub starts 2024-09-10T19:08:16.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:16 smithi018 ceph-mon[40196]: 4.f scrub ok 2024-09-10T19:08:16.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:16 smithi018 ceph-mon[40196]: pgmap v1093: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 765 B/s rd, 0 op/s 2024-09-10T19:08:16.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:16 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1557304565' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-09-10T19:08:16.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:16 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-09-10T19:08:16.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:16 smithi018 ceph-mon[29336]: 4.f scrub starts 2024-09-10T19:08:16.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:16 smithi018 ceph-mon[29336]: 4.f scrub ok 2024-09-10T19:08:16.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:16 smithi018 ceph-mon[29336]: pgmap v1093: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 765 B/s rd, 0 op/s 2024-09-10T19:08:16.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:16 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1557304565' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-09-10T19:08:16.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:16 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-09-10T19:08:17.414 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-10T19:08:17.414 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-10T19:08:17.414 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-10T19:08:17.414 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-10T19:08:17.414 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-10T19:08:17.414 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T19:08:17.681 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:08:17.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:17 smithi018 ceph-mon[29336]: 2.1e scrub starts 2024-09-10T19:08:17.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:17 smithi018 ceph-mon[29336]: 2.1e scrub ok 2024-09-10T19:08:17.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:17 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:08:17.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:17 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:08:17.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:17 smithi018 ceph-mon[29336]: 1.0 scrub starts 2024-09-10T19:08:17.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:17 smithi018 ceph-mon[29336]: 1.0 scrub ok 2024-09-10T19:08:17.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:17 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-09-10T19:08:17.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:17 smithi018 ceph-mon[29336]: osdmap e491: 8 total, 8 up, 8 in 2024-09-10T19:08:17.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:17 smithi018 ceph-mon[40196]: 2.1e scrub starts 2024-09-10T19:08:17.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:17 smithi018 ceph-mon[40196]: 2.1e scrub ok 2024-09-10T19:08:17.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:17 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:08:17.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:17 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:08:17.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:17 smithi018 ceph-mon[40196]: 1.0 scrub starts 2024-09-10T19:08:17.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:17 smithi018 ceph-mon[40196]: 1.0 scrub ok 2024-09-10T19:08:17.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:17 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-09-10T19:08:17.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:17 smithi018 ceph-mon[40196]: osdmap e491: 8 total, 8 up, 8 in 2024-09-10T19:08:17.772 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:08:17 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:08:17.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:17 smithi103 ceph-mon[33163]: 2.1e scrub starts 2024-09-10T19:08:17.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:17 smithi103 ceph-mon[33163]: 2.1e scrub ok 2024-09-10T19:08:17.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:17 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:08:17.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:17 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:08:17.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:17 smithi103 ceph-mon[33163]: 1.0 scrub starts 2024-09-10T19:08:17.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:17 smithi103 ceph-mon[33163]: 1.0 scrub ok 2024-09-10T19:08:17.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:17 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-09-10T19:08:17.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:17 smithi103 ceph-mon[33163]: osdmap e491: 8 total, 8 up, 8 in 2024-09-10T19:08:18.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:18 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:18.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:18 smithi103 ceph-mon[33163]: 6.1c scrub starts 2024-09-10T19:08:18.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:18 smithi103 ceph-mon[33163]: 6.1c scrub ok 2024-09-10T19:08:18.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:18 smithi103 ceph-mon[33163]: pgmap v1095: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T19:08:18.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:18 smithi103 ceph-mon[33163]: 3.1 scrub starts 2024-09-10T19:08:18.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:18 smithi103 ceph-mon[33163]: 3.1 scrub ok 2024-09-10T19:08:18.857 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:18 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:18.857 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:18 smithi018 ceph-mon[29336]: 6.1c scrub starts 2024-09-10T19:08:18.857 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:18 smithi018 ceph-mon[29336]: 6.1c scrub ok 2024-09-10T19:08:18.857 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:18 smithi018 ceph-mon[29336]: pgmap v1095: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T19:08:18.857 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:18 smithi018 ceph-mon[29336]: 3.1 scrub starts 2024-09-10T19:08:18.857 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:18 smithi018 ceph-mon[29336]: 3.1 scrub ok 2024-09-10T19:08:18.857 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:18 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:18.857 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:18 smithi018 ceph-mon[40196]: 6.1c scrub starts 2024-09-10T19:08:18.858 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:18 smithi018 ceph-mon[40196]: 6.1c scrub ok 2024-09-10T19:08:18.858 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:18 smithi018 ceph-mon[40196]: pgmap v1095: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T19:08:18.858 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:18 smithi018 ceph-mon[40196]: 3.1 scrub starts 2024-09-10T19:08:18.858 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:18 smithi018 ceph-mon[40196]: 3.1 scrub ok 2024-09-10T19:08:19.562 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T19:08:19.562 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":492,"fsid":"ba327f6c-6fa3-11ef-bcea-c7b262605968","created":"2024-09-10T18:40:07.669687+0000","modified":"2024-09-10T19:08:18.854874+0000","last_up_change":"2024-09-10T19:04:50.812385+0000","last_in_change":"2024-09-10T19:07:20.833261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T18:43:08.162659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":21,"pg_placement_num":21,"pg_placement_num_target":21,"pg_num_target":21,"pg_num_pending":21,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"394","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"167","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-10T18:45:16.512219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-10T18:45:16.943626+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":60,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-10T18:45:18.450026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-10T18:45:20.494618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-10T18:45:22.566992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"eb4c7e67-300c-4a1a-b9f5-bb028eeeabc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":205,"up_from":241,"up_thru":464,"down_at":206,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6801","nonce":4100931169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":4100931169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":4100931169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6803","nonce":4100931169}]},"public_addr":"172.21.15.18:6801/4100931169","cluster_addr":"172.21.15.18:6802/4100931169","heartbeat_back_addr":"172.21.15.18:6804/4100931169","heartbeat_front_addr":"172.21.15.18:6803/4100931169","state":["exists","up"]},{"osd":1,"uuid":"63bf6a98-e349-4cc0-9502-a31693070c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":469,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6805","nonce":3855640936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":3855640936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":3855640936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6807","nonce":3855640936}]},"public_addr":"172.21.15.18:6805/3855640936","cluster_addr":"172.21.15.18:6806/3855640936","heartbeat_back_addr":"172.21.15.18:6808/3855640936","heartbeat_front_addr":"172.21.15.18:6807/3855640936","state":["exists","up"]},{"osd":2,"uuid":"79a01c29-4f4d-43d1-b1ea-a3fcf014f84b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":483,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6809","nonce":4107623855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":4107623855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":4107623855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6811","nonce":4107623855}]},"public_addr":"172.21.15.18:6809/4107623855","cluster_addr":"172.21.15.18:6810/4107623855","heartbeat_back_addr":"172.21.15.18:6812/4107623855","heartbeat_front_addr":"172.21.15.18:6811/4107623855","state":["exists","up"]},{"osd":3,"uuid":"1809e677-b98c-4924-982e-9297e37563af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":483,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6813","nonce":2732204253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2732204253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":2732204253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6815","nonce":2732204253}]},"public_addr":"172.21.15.18:6813/2732204253","cluster_addr":"172.21.15.18:6814/2732204253","heartbeat_back_addr":"172.21.15.18:6816/2732204253","heartbeat_front_addr":"172.21.15.18:6815/2732204253","state":["exists","up"]},{"osd":4,"uuid":"b682a300-bb53-4f36-8a9f-0753164ec4b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":464,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":465860081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6801","nonce":465860081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6803","nonce":465860081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":465860081}]},"public_addr":"172.21.15.103:6800/465860081","cluster_addr":"172.21.15.103:6801/465860081","heartbeat_back_addr":"172.21.15.103:6803/465860081","heartbeat_front_addr":"172.21.15.103:6802/465860081","state":["exists","up"]},{"osd":5,"uuid":"2bbfdb6c-0709-4fe4-8a4a-006f576f18e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":464,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3498719258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6805","nonce":3498719258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6807","nonce":3498719258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3498719258}]},"public_addr":"172.21.15.103:6804/3498719258","cluster_addr":"172.21.15.103:6805/3498719258","heartbeat_back_addr":"172.21.15.103:6807/3498719258","heartbeat_front_addr":"172.21.15.103:6806/3498719258","state":["exists","up"]},{"osd":6,"uuid":"31f9711e-0ba8-4aeb-b852-7d93e8fcce33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":464,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":184353788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6809","nonce":184353788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6811","nonce":184353788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":184353788}]},"public_addr":"172.21.15.103:6808/184353788","cluster_addr":"172.21.15.103:6809/184353788","heartbeat_back_addr":"172.21.15.103:6811/184353788","heartbeat_front_addr":"172.21.15.103:6810/184353788","state":["exists","up"]},{"osd":7,"uuid":"f12282f8-4747-45ff-8336-95440560a88f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":52,"last_clean_end":361,"up_from":411,"up_thru":464,"down_at":362,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":3644107622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6813","nonce":3644107622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6815","nonce":3644107622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":3644107622}]},"public_addr":"172.21.15.103:6812/3644107622","cluster_addr":"172.21.15.103:6813/3644107622","heartbeat_back_addr":"172.21.15.103:6815/3644107622","heartbeat_front_addr":"172.21.15.103:6814/3644107622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:07:57.115250+0000","dead_epoch":205},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:08:16.297166+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:07:51.831041+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:07:30.206792+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:07:18.457545+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:07:15.801590+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:07:42.170312+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:07:13.548046+0000","dead_epoch":361}],"pg_upmap":[{"pgid":"2.11","osds":[6,7,5]},{"pgid":"6.9","osds":[4,1,6]}],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":7,"to":2}]},{"pgid":"2.1b","mappings":[{"from":7,"to":4}]},{"pgid":"4.e","mappings":[{"from":1,"to":5}]},{"pgid":"4.f","mappings":[{"from":1,"to":5}]},{"pgid":"4.14","mappings":[{"from":1,"to":2}]},{"pgid":"5.5","mappings":[{"from":0,"to":6}]},{"pgid":"5.7","mappings":[{"from":1,"to":6}]},{"pgid":"5.11","mappings":[{"from":4,"to":3}]},{"pgid":"5.15","mappings":[{"from":1,"to":6}]},{"pgid":"5.19","mappings":[{"from":1,"to":6}]},{"pgid":"5.1a","mappings":[{"from":1,"to":3}]},{"pgid":"6.f","mappings":[{"from":3,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/3021537013":"2024-09-11T18:45:54.998616+0000","172.21.15.18:0/458572898":"2024-09-11T18:40:28.880042+0000","172.21.15.18:0/1859541419":"2024-09-11T18:40:28.880042+0000","172.21.15.18:0/3090642970":"2024-09-11T18:40:59.659411+0000","172.21.15.18:0/2875894312":"2024-09-11T18:40:59.659411+0000","172.21.15.18:0/178638059":"2024-09-11T18:40:59.659411+0000","172.21.15.18:0/692931402":"2024-09-11T18:45:54.998616+0000","172.21.15.18:0/2408360238":"2024-09-11T18:45:54.998616+0000","172.21.15.18:0/57192865":"2024-09-11T18:40:28.880042+0000","172.21.15.18:6800/2748756682":"2024-09-11T18:40:59.659411+0000","172.21.15.18:6800/732375116":"2024-09-11T18:45:54.998616+0000","172.21.15.18:6800/3869801150":"2024-09-11T18:40:28.880042+0000","172.21.15.18:0/2982361259":"2024-09-11T18:45:54.998616+0000","172.21.15.18:0/185700038":"2024-09-11T18:45:54.998616+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T19:08:19.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:19 smithi103 ceph-mon[33163]: 5.e scrub starts 2024-09-10T19:08:19.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:19 smithi103 ceph-mon[33163]: 5.e scrub ok 2024-09-10T19:08:19.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:19 smithi103 ceph-mon[33163]: 5.f scrub starts 2024-09-10T19:08:19.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:19 smithi103 ceph-mon[33163]: 5.f scrub ok 2024-09-10T19:08:19.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:19 smithi103 ceph-mon[33163]: 1.7 scrub starts 2024-09-10T19:08:19.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:19 smithi103 ceph-mon[33163]: 1.7 scrub ok 2024-09-10T19:08:19.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:19 smithi103 ceph-mon[33163]: osdmap e492: 8 total, 8 up, 8 in 2024-09-10T19:08:19.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:19 smithi018 ceph-mon[40196]: 5.e scrub starts 2024-09-10T19:08:19.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:19 smithi018 ceph-mon[40196]: 5.e scrub ok 2024-09-10T19:08:19.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:19 smithi018 ceph-mon[40196]: 5.f scrub starts 2024-09-10T19:08:19.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:19 smithi018 ceph-mon[40196]: 5.f scrub ok 2024-09-10T19:08:19.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:19 smithi018 ceph-mon[40196]: 1.7 scrub starts 2024-09-10T19:08:19.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:19 smithi018 ceph-mon[40196]: 1.7 scrub ok 2024-09-10T19:08:19.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:19 smithi018 ceph-mon[40196]: osdmap e492: 8 total, 8 up, 8 in 2024-09-10T19:08:19.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:19 smithi018 ceph-mon[29336]: 5.e scrub starts 2024-09-10T19:08:19.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:19 smithi018 ceph-mon[29336]: 5.e scrub ok 2024-09-10T19:08:19.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:19 smithi018 ceph-mon[29336]: 5.f scrub starts 2024-09-10T19:08:19.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:19 smithi018 ceph-mon[29336]: 5.f scrub ok 2024-09-10T19:08:19.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:19 smithi018 ceph-mon[29336]: 1.7 scrub starts 2024-09-10T19:08:19.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:19 smithi018 ceph-mon[29336]: 1.7 scrub ok 2024-09-10T19:08:19.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:19 smithi018 ceph-mon[29336]: osdmap e492: 8 total, 8 up, 8 in 2024-09-10T19:08:20.498 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-10T19:08:20.498 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T19:08:20.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:08:20 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:08:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:08:20.765 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:08:20.887 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:20 smithi018 ceph-mon[29336]: pgmap v1097: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 823 B/s rd, 0 op/s 2024-09-10T19:08:20.887 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:20 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/4091734044' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T19:08:20.887 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:20 smithi018 ceph-mon[29336]: 2.14 scrub starts 2024-09-10T19:08:20.887 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:20 smithi018 ceph-mon[29336]: 2.14 scrub ok 2024-09-10T19:08:20.887 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:20 smithi018 ceph-mon[40196]: pgmap v1097: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 823 B/s rd, 0 op/s 2024-09-10T19:08:20.887 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:20 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/4091734044' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T19:08:20.888 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:20 smithi018 ceph-mon[40196]: 2.14 scrub starts 2024-09-10T19:08:20.888 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:20 smithi018 ceph-mon[40196]: 2.14 scrub ok 2024-09-10T19:08:21.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:20 smithi103 ceph-mon[33163]: pgmap v1097: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 823 B/s rd, 0 op/s 2024-09-10T19:08:21.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:20 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/4091734044' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T19:08:21.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:20 smithi103 ceph-mon[33163]: 2.14 scrub starts 2024-09-10T19:08:21.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:20 smithi103 ceph-mon[33163]: 2.14 scrub ok 2024-09-10T19:08:21.143 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:08:20 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:08:20] "GET /metrics HTTP/1.1" 200 222111 "" "Prometheus/2.43.0" 2024-09-10T19:08:21.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:21 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:08:21.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:21 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:08:21.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:21 smithi018 ceph-mon[29336]: osdmap e493: 8 total, 8 up, 8 in 2024-09-10T19:08:21.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:21 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:08:21.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:21 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:08:21.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:21 smithi018 ceph-mon[40196]: osdmap e493: 8 total, 8 up, 8 in 2024-09-10T19:08:22.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:21 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:08:22.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:21 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:08:22.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:21 smithi103 ceph-mon[33163]: osdmap e493: 8 total, 8 up, 8 in 2024-09-10T19:08:22.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:22 smithi018 ceph-mon[29336]: 6.1d scrub starts 2024-09-10T19:08:22.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:22 smithi018 ceph-mon[29336]: 6.1d scrub ok 2024-09-10T19:08:22.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:22 smithi018 ceph-mon[29336]: pgmap v1099: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:22.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:22 smithi018 ceph-mon[29336]: 1.12 scrub starts 2024-09-10T19:08:22.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:22 smithi018 ceph-mon[29336]: 1.12 scrub ok 2024-09-10T19:08:22.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:22 smithi018 ceph-mon[40196]: 6.1d scrub starts 2024-09-10T19:08:22.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:22 smithi018 ceph-mon[40196]: 6.1d scrub ok 2024-09-10T19:08:22.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:22 smithi018 ceph-mon[40196]: pgmap v1099: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:22.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:22 smithi018 ceph-mon[40196]: 1.12 scrub starts 2024-09-10T19:08:22.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:22 smithi018 ceph-mon[40196]: 1.12 scrub ok 2024-09-10T19:08:22.701 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T19:08:22.701 INFO:teuthology.orchestra.run.smithi018.stdout:{"epoch":493,"fsid":"ba327f6c-6fa3-11ef-bcea-c7b262605968","created":"2024-09-10T18:40:07.669687+0000","modified":"2024-09-10T19:08:20.520943+0000","last_up_change":"2024-09-10T19:04:50.812385+0000","last_in_change":"2024-09-10T19:07:20.833261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T18:43:08.162659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":21,"pg_placement_num":21,"pg_placement_num_target":21,"pg_num_target":21,"pg_num_pending":21,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"394","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"167","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-10T18:45:16.512219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-10T18:45:16.943626+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":60,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-10T18:45:18.450026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-10T18:45:20.494618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-10T18:45:22.566992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"eb4c7e67-300c-4a1a-b9f5-bb028eeeabc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":205,"up_from":241,"up_thru":464,"down_at":206,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6801","nonce":4100931169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":4100931169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":4100931169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6803","nonce":4100931169}]},"public_addr":"172.21.15.18:6801/4100931169","cluster_addr":"172.21.15.18:6802/4100931169","heartbeat_back_addr":"172.21.15.18:6804/4100931169","heartbeat_front_addr":"172.21.15.18:6803/4100931169","state":["exists","up"]},{"osd":1,"uuid":"63bf6a98-e349-4cc0-9502-a31693070c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":469,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6805","nonce":3855640936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":3855640936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":3855640936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6807","nonce":3855640936}]},"public_addr":"172.21.15.18:6805/3855640936","cluster_addr":"172.21.15.18:6806/3855640936","heartbeat_back_addr":"172.21.15.18:6808/3855640936","heartbeat_front_addr":"172.21.15.18:6807/3855640936","state":["exists","up"]},{"osd":2,"uuid":"79a01c29-4f4d-43d1-b1ea-a3fcf014f84b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":483,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6809","nonce":4107623855}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":4107623855}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":4107623855}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6811","nonce":4107623855}]},"public_addr":"172.21.15.18:6809/4107623855","cluster_addr":"172.21.15.18:6810/4107623855","heartbeat_back_addr":"172.21.15.18:6812/4107623855","heartbeat_front_addr":"172.21.15.18:6811/4107623855","state":["exists","up"]},{"osd":3,"uuid":"1809e677-b98c-4924-982e-9297e37563af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":483,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6813","nonce":2732204253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2732204253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":2732204253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6815","nonce":2732204253}]},"public_addr":"172.21.15.18:6813/2732204253","cluster_addr":"172.21.15.18:6814/2732204253","heartbeat_back_addr":"172.21.15.18:6816/2732204253","heartbeat_front_addr":"172.21.15.18:6815/2732204253","state":["exists","up"]},{"osd":4,"uuid":"b682a300-bb53-4f36-8a9f-0753164ec4b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":464,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":465860081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6801","nonce":465860081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6803","nonce":465860081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":465860081}]},"public_addr":"172.21.15.103:6800/465860081","cluster_addr":"172.21.15.103:6801/465860081","heartbeat_back_addr":"172.21.15.103:6803/465860081","heartbeat_front_addr":"172.21.15.103:6802/465860081","state":["exists","up"]},{"osd":5,"uuid":"2bbfdb6c-0709-4fe4-8a4a-006f576f18e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":464,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3498719258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6805","nonce":3498719258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6807","nonce":3498719258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3498719258}]},"public_addr":"172.21.15.103:6804/3498719258","cluster_addr":"172.21.15.103:6805/3498719258","heartbeat_back_addr":"172.21.15.103:6807/3498719258","heartbeat_front_addr":"172.21.15.103:6806/3498719258","state":["exists","up"]},{"osd":6,"uuid":"31f9711e-0ba8-4aeb-b852-7d93e8fcce33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":464,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":184353788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6809","nonce":184353788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6811","nonce":184353788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":184353788}]},"public_addr":"172.21.15.103:6808/184353788","cluster_addr":"172.21.15.103:6809/184353788","heartbeat_back_addr":"172.21.15.103:6811/184353788","heartbeat_front_addr":"172.21.15.103:6810/184353788","state":["exists","up"]},{"osd":7,"uuid":"f12282f8-4747-45ff-8336-95440560a88f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":52,"last_clean_end":361,"up_from":411,"up_thru":464,"down_at":362,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":3644107622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6813","nonce":3644107622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6815","nonce":3644107622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":3644107622}]},"public_addr":"172.21.15.103:6812/3644107622","cluster_addr":"172.21.15.103:6813/3644107622","heartbeat_back_addr":"172.21.15.103:6815/3644107622","heartbeat_front_addr":"172.21.15.103:6814/3644107622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:07:57.115250+0000","dead_epoch":205},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:08:16.297166+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:07:51.831041+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:07:30.206792+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:07:18.457545+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:07:15.801590+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:07:42.170312+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:08:20.464333+0000","dead_epoch":361}],"pg_upmap":[{"pgid":"2.11","osds":[6,7,5]},{"pgid":"6.9","osds":[4,1,6]}],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":7,"to":2}]},{"pgid":"2.1b","mappings":[{"from":7,"to":4}]},{"pgid":"4.e","mappings":[{"from":1,"to":5}]},{"pgid":"4.f","mappings":[{"from":1,"to":5}]},{"pgid":"4.14","mappings":[{"from":1,"to":2}]},{"pgid":"5.5","mappings":[{"from":0,"to":6}]},{"pgid":"5.7","mappings":[{"from":1,"to":6}]},{"pgid":"5.11","mappings":[{"from":4,"to":3}]},{"pgid":"5.15","mappings":[{"from":1,"to":6}]},{"pgid":"5.19","mappings":[{"from":1,"to":6}]},{"pgid":"5.1a","mappings":[{"from":1,"to":3}]},{"pgid":"6.f","mappings":[{"from":3,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.18:0/3021537013":"2024-09-11T18:45:54.998616+0000","172.21.15.18:0/458572898":"2024-09-11T18:40:28.880042+0000","172.21.15.18:0/1859541419":"2024-09-11T18:40:28.880042+0000","172.21.15.18:0/3090642970":"2024-09-11T18:40:59.659411+0000","172.21.15.18:0/2875894312":"2024-09-11T18:40:59.659411+0000","172.21.15.18:0/178638059":"2024-09-11T18:40:59.659411+0000","172.21.15.18:0/692931402":"2024-09-11T18:45:54.998616+0000","172.21.15.18:0/2408360238":"2024-09-11T18:45:54.998616+0000","172.21.15.18:0/57192865":"2024-09-11T18:40:28.880042+0000","172.21.15.18:6800/2748756682":"2024-09-11T18:40:59.659411+0000","172.21.15.18:6800/732375116":"2024-09-11T18:45:54.998616+0000","172.21.15.18:6800/3869801150":"2024-09-11T18:40:28.880042+0000","172.21.15.18:0/2982361259":"2024-09-11T18:45:54.998616+0000","172.21.15.18:0/185700038":"2024-09-11T18:45:54.998616+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T19:08:23.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:22 smithi103 ceph-mon[33163]: 6.1d scrub starts 2024-09-10T19:08:23.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:22 smithi103 ceph-mon[33163]: 6.1d scrub ok 2024-09-10T19:08:23.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:22 smithi103 ceph-mon[33163]: pgmap v1099: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:23.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:22 smithi103 ceph-mon[33163]: 1.12 scrub starts 2024-09-10T19:08:23.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:22 smithi103 ceph-mon[33163]: 1.12 scrub ok 2024-09-10T19:08:23.487 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-10T19:08:23.487 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-10T19:08:23.488 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-10T19:08:23.488 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-10T19:08:23.489 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-10T19:08:23.489 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-10T19:08:23.490 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-10T19:08:23.491 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-10T19:08:23.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:23 smithi018 ceph-mon[29336]: 5.9 scrub starts 2024-09-10T19:08:23.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:23 smithi018 ceph-mon[29336]: 5.9 scrub ok 2024-09-10T19:08:23.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:23 smithi018 ceph-mon[29336]: 5.16 scrub starts 2024-09-10T19:08:23.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:23 smithi018 ceph-mon[29336]: 5.16 scrub ok 2024-09-10T19:08:23.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:23 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:08:23.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:23 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:08:23.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:23 smithi018 ceph-mon[29336]: 5.1f deep-scrub starts 2024-09-10T19:08:23.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:23 smithi018 ceph-mon[29336]: 5.1f deep-scrub ok 2024-09-10T19:08:23.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:23 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2733139764' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T19:08:23.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:23 smithi018 ceph-mon[29336]: 2.1d deep-scrub starts 2024-09-10T19:08:23.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:23 smithi018 ceph-mon[29336]: 2.1d deep-scrub ok 2024-09-10T19:08:23.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:23 smithi018 ceph-mon[40196]: 5.9 scrub starts 2024-09-10T19:08:23.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:23 smithi018 ceph-mon[40196]: 5.9 scrub ok 2024-09-10T19:08:23.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:23 smithi018 ceph-mon[40196]: 5.16 scrub starts 2024-09-10T19:08:23.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:23 smithi018 ceph-mon[40196]: 5.16 scrub ok 2024-09-10T19:08:23.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:23 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:08:23.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:23 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:08:23.947 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:23 smithi018 ceph-mon[40196]: 5.1f deep-scrub starts 2024-09-10T19:08:23.947 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:23 smithi018 ceph-mon[40196]: 5.1f deep-scrub ok 2024-09-10T19:08:23.947 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:23 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2733139764' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T19:08:23.947 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:23 smithi018 ceph-mon[40196]: 2.1d deep-scrub starts 2024-09-10T19:08:23.947 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:23 smithi018 ceph-mon[40196]: 2.1d deep-scrub ok 2024-09-10T19:08:24.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:23 smithi103 ceph-mon[33163]: 5.9 scrub starts 2024-09-10T19:08:24.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:23 smithi103 ceph-mon[33163]: 5.9 scrub ok 2024-09-10T19:08:24.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:23 smithi103 ceph-mon[33163]: 5.16 scrub starts 2024-09-10T19:08:24.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:23 smithi103 ceph-mon[33163]: 5.16 scrub ok 2024-09-10T19:08:24.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:23 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:08:24.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:23 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:08:24.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:23 smithi103 ceph-mon[33163]: 5.1f deep-scrub starts 2024-09-10T19:08:24.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:23 smithi103 ceph-mon[33163]: 5.1f deep-scrub ok 2024-09-10T19:08:24.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:23 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2733139764' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T19:08:24.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:23 smithi103 ceph-mon[33163]: 2.1d deep-scrub starts 2024-09-10T19:08:24.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:23 smithi103 ceph-mon[33163]: 2.1d deep-scrub ok 2024-09-10T19:08:24.037 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:08:24.043 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:08:24.043 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:08:24.054 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:08:24.055 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:08:24.055 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:08:24.056 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:08:24.058 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:08:24.890 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:24 smithi018 ceph-mon[29336]: 6.14 scrub starts 2024-09-10T19:08:24.890 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:24 smithi018 ceph-mon[29336]: 6.14 scrub ok 2024-09-10T19:08:24.890 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:24 smithi018 ceph-mon[29336]: pgmap v1101: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:24.890 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:24 smithi018 ceph-mon[29336]: osdmap e494: 8 total, 8 up, 8 in 2024-09-10T19:08:24.890 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:24 smithi018 ceph-mon[40196]: 6.14 scrub starts 2024-09-10T19:08:24.890 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:24 smithi018 ceph-mon[40196]: 6.14 scrub ok 2024-09-10T19:08:24.890 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:24 smithi018 ceph-mon[40196]: pgmap v1101: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:24.890 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:24 smithi018 ceph-mon[40196]: osdmap e494: 8 total, 8 up, 8 in 2024-09-10T19:08:25.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:24 smithi103 ceph-mon[33163]: 6.14 scrub starts 2024-09-10T19:08:25.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:24 smithi103 ceph-mon[33163]: 6.14 scrub ok 2024-09-10T19:08:25.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:24 smithi103 ceph-mon[33163]: pgmap v1101: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:25.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:24 smithi103 ceph-mon[33163]: osdmap e494: 8 total, 8 up, 8 in 2024-09-10T19:08:25.806 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:25 smithi018 ceph-mon[40196]: 2.2 scrub starts 2024-09-10T19:08:25.806 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:25 smithi018 ceph-mon[40196]: 2.2 scrub ok 2024-09-10T19:08:25.806 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:25 smithi018 ceph-mon[40196]: 4.19 scrub starts 2024-09-10T19:08:25.806 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:25 smithi018 ceph-mon[40196]: 4.19 scrub ok 2024-09-10T19:08:25.806 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:25 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:25.806 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:25 smithi018 ceph-mon[29336]: 2.2 scrub starts 2024-09-10T19:08:25.806 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:25 smithi018 ceph-mon[29336]: 2.2 scrub ok 2024-09-10T19:08:25.806 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:25 smithi018 ceph-mon[29336]: 4.19 scrub starts 2024-09-10T19:08:25.807 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:25 smithi018 ceph-mon[29336]: 4.19 scrub ok 2024-09-10T19:08:25.807 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:25 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:26.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:25 smithi103 ceph-mon[33163]: 2.2 scrub starts 2024-09-10T19:08:26.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:25 smithi103 ceph-mon[33163]: 2.2 scrub ok 2024-09-10T19:08:26.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:25 smithi103 ceph-mon[33163]: 4.19 scrub starts 2024-09-10T19:08:26.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:25 smithi103 ceph-mon[33163]: 4.19 scrub ok 2024-09-10T19:08:26.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:25 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:26.939 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:26 smithi018 ceph-mon[29336]: pgmap v1102: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 765 B/s rd, 0 op/s 2024-09-10T19:08:26.939 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:26 smithi018 ceph-mon[29336]: 6.5 scrub starts 2024-09-10T19:08:26.939 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:26 smithi018 ceph-mon[29336]: 6.5 scrub ok 2024-09-10T19:08:26.940 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:26 smithi018 ceph-mon[40196]: pgmap v1102: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 765 B/s rd, 0 op/s 2024-09-10T19:08:26.940 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:26 smithi018 ceph-mon[40196]: 6.5 scrub starts 2024-09-10T19:08:26.940 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:26 smithi018 ceph-mon[40196]: 6.5 scrub ok 2024-09-10T19:08:27.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:26 smithi103 ceph-mon[33163]: pgmap v1102: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 765 B/s rd, 0 op/s 2024-09-10T19:08:27.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:26 smithi103 ceph-mon[33163]: 6.5 scrub starts 2024-09-10T19:08:27.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:26 smithi103 ceph-mon[33163]: 6.5 scrub ok 2024-09-10T19:08:27.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:27 smithi103 ceph-mon[33163]: 2.7 scrub starts 2024-09-10T19:08:27.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:27 smithi103 ceph-mon[33163]: 2.7 scrub ok 2024-09-10T19:08:27.773 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:08:27 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:08:27.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:27 smithi018 ceph-mon[40196]: 2.7 scrub starts 2024-09-10T19:08:27.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:27 smithi018 ceph-mon[40196]: 2.7 scrub ok 2024-09-10T19:08:27.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:27 smithi018 ceph-mon[29336]: 2.7 scrub starts 2024-09-10T19:08:27.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:27 smithi018 ceph-mon[29336]: 2.7 scrub ok 2024-09-10T19:08:28.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:28 smithi018 ceph-mon[29336]: 4.1a scrub starts 2024-09-10T19:08:28.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:28 smithi018 ceph-mon[29336]: 4.1a scrub ok 2024-09-10T19:08:28.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:28 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:28.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:28 smithi018 ceph-mon[29336]: 4.12 scrub starts 2024-09-10T19:08:28.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:28 smithi018 ceph-mon[29336]: 4.12 scrub ok 2024-09-10T19:08:28.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:28 smithi018 ceph-mon[29336]: pgmap v1103: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:28.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:28 smithi018 ceph-mon[40196]: 4.1a scrub starts 2024-09-10T19:08:28.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:28 smithi018 ceph-mon[40196]: 4.1a scrub ok 2024-09-10T19:08:28.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:28 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:28.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:28 smithi018 ceph-mon[40196]: 4.12 scrub starts 2024-09-10T19:08:28.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:28 smithi018 ceph-mon[40196]: 4.12 scrub ok 2024-09-10T19:08:28.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:28 smithi018 ceph-mon[40196]: pgmap v1103: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:29.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:28 smithi103 ceph-mon[33163]: 4.1a scrub starts 2024-09-10T19:08:29.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:28 smithi103 ceph-mon[33163]: 4.1a scrub ok 2024-09-10T19:08:29.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:28 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:29.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:28 smithi103 ceph-mon[33163]: 4.12 scrub starts 2024-09-10T19:08:29.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:28 smithi103 ceph-mon[33163]: 4.12 scrub ok 2024-09-10T19:08:29.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:28 smithi103 ceph-mon[33163]: pgmap v1103: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:29.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:29 smithi018 ceph-mon[29336]: 3.0 scrub starts 2024-09-10T19:08:29.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:29 smithi018 ceph-mon[29336]: 3.0 scrub ok 2024-09-10T19:08:29.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:29 smithi018 ceph-mon[29336]: 4.8 scrub starts 2024-09-10T19:08:29.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:29 smithi018 ceph-mon[29336]: 4.8 scrub ok 2024-09-10T19:08:29.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:29 smithi018 ceph-mon[29336]: 2.1a scrub starts 2024-09-10T19:08:29.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:29 smithi018 ceph-mon[29336]: 2.1a scrub ok 2024-09-10T19:08:29.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:29 smithi018 ceph-mon[40196]: 3.0 scrub starts 2024-09-10T19:08:29.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:29 smithi018 ceph-mon[40196]: 3.0 scrub ok 2024-09-10T19:08:29.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:29 smithi018 ceph-mon[40196]: 4.8 scrub starts 2024-09-10T19:08:29.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:29 smithi018 ceph-mon[40196]: 4.8 scrub ok 2024-09-10T19:08:29.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:29 smithi018 ceph-mon[40196]: 2.1a scrub starts 2024-09-10T19:08:29.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:29 smithi018 ceph-mon[40196]: 2.1a scrub ok 2024-09-10T19:08:30.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:29 smithi103 ceph-mon[33163]: 3.0 scrub starts 2024-09-10T19:08:30.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:29 smithi103 ceph-mon[33163]: 3.0 scrub ok 2024-09-10T19:08:30.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:29 smithi103 ceph-mon[33163]: 4.8 scrub starts 2024-09-10T19:08:30.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:29 smithi103 ceph-mon[33163]: 4.8 scrub ok 2024-09-10T19:08:30.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:29 smithi103 ceph-mon[33163]: 2.1a scrub starts 2024-09-10T19:08:30.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:29 smithi103 ceph-mon[33163]: 2.1a scrub ok 2024-09-10T19:08:30.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:08:30 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:08:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:08:30.847 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:30 smithi018 ceph-mon[40196]: 6.6 deep-scrub starts 2024-09-10T19:08:30.847 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:30 smithi018 ceph-mon[40196]: 6.6 deep-scrub ok 2024-09-10T19:08:30.847 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:30 smithi018 ceph-mon[40196]: pgmap v1104: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T19:08:30.847 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:30 smithi018 ceph-mon[29336]: 6.6 deep-scrub starts 2024-09-10T19:08:30.848 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:30 smithi018 ceph-mon[29336]: 6.6 deep-scrub ok 2024-09-10T19:08:30.848 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:30 smithi018 ceph-mon[29336]: pgmap v1104: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T19:08:31.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:30 smithi103 ceph-mon[33163]: 6.6 deep-scrub starts 2024-09-10T19:08:31.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:30 smithi103 ceph-mon[33163]: 6.6 deep-scrub ok 2024-09-10T19:08:31.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:30 smithi103 ceph-mon[33163]: pgmap v1104: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T19:08:31.194 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:08:30 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:08:30] "GET /metrics HTTP/1.1" 200 222106 "" "Prometheus/2.43.0" 2024-09-10T19:08:31.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:31 smithi018 ceph-mon[29336]: 1.a scrub starts 2024-09-10T19:08:31.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:31 smithi018 ceph-mon[29336]: 1.a scrub ok 2024-09-10T19:08:31.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:31 smithi018 ceph-mon[40196]: 1.a scrub starts 2024-09-10T19:08:31.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:31 smithi018 ceph-mon[40196]: 1.a scrub ok 2024-09-10T19:08:32.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:31 smithi103 ceph-mon[33163]: 1.a scrub starts 2024-09-10T19:08:32.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:31 smithi103 ceph-mon[33163]: 1.a scrub ok 2024-09-10T19:08:32.692 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:32 smithi018 ceph-mon[29336]: pgmap v1105: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:08:32.693 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:32 smithi018 ceph-mon[29336]: 2.16 scrub starts 2024-09-10T19:08:32.693 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:32 smithi018 ceph-mon[29336]: 2.16 scrub ok 2024-09-10T19:08:32.693 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:32 smithi018 ceph-mon[40196]: pgmap v1105: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:08:32.693 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:32 smithi018 ceph-mon[40196]: 2.16 scrub starts 2024-09-10T19:08:32.693 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:32 smithi018 ceph-mon[40196]: 2.16 scrub ok 2024-09-10T19:08:33.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:32 smithi103 ceph-mon[33163]: pgmap v1105: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:08:33.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:32 smithi103 ceph-mon[33163]: 2.16 scrub starts 2024-09-10T19:08:33.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:32 smithi103 ceph-mon[33163]: 2.16 scrub ok 2024-09-10T19:08:33.894 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:33 smithi018 ceph-mon[29336]: 6.17 scrub starts 2024-09-10T19:08:33.894 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:33 smithi018 ceph-mon[29336]: 6.17 scrub ok 2024-09-10T19:08:33.894 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:33 smithi018 ceph-mon[29336]: 4.1f scrub starts 2024-09-10T19:08:33.894 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:33 smithi018 ceph-mon[29336]: 4.1f scrub ok 2024-09-10T19:08:33.895 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:33 smithi018 ceph-mon[40196]: 6.17 scrub starts 2024-09-10T19:08:33.895 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:33 smithi018 ceph-mon[40196]: 6.17 scrub ok 2024-09-10T19:08:33.895 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:33 smithi018 ceph-mon[40196]: 4.1f scrub starts 2024-09-10T19:08:33.895 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:33 smithi018 ceph-mon[40196]: 4.1f scrub ok 2024-09-10T19:08:34.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:33 smithi103 ceph-mon[33163]: 6.17 scrub starts 2024-09-10T19:08:34.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:33 smithi103 ceph-mon[33163]: 6.17 scrub ok 2024-09-10T19:08:34.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:33 smithi103 ceph-mon[33163]: 4.1f scrub starts 2024-09-10T19:08:34.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:33 smithi103 ceph-mon[33163]: 4.1f scrub ok 2024-09-10T19:08:34.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:34 smithi018 ceph-mon[29336]: pgmap v1106: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:08:34.694 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:34 smithi018 ceph-mon[40196]: pgmap v1106: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:08:35.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:34 smithi103 ceph-mon[33163]: pgmap v1106: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:08:35.448 INFO:teuthology.orchestra.run.smithi018.stdout:1035087118493 2024-09-10T19:08:35.448 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-10T19:08:36.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:36 smithi018 ceph-mon[40196]: 5.a scrub starts 2024-09-10T19:08:36.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:36 smithi018 ceph-mon[40196]: 5.a scrub ok 2024-09-10T19:08:36.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:36 smithi018 ceph-mon[40196]: pgmap v1107: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:08:36.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:36 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:08:36.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:36 smithi018 ceph-mon[29336]: 5.a scrub starts 2024-09-10T19:08:36.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:36 smithi018 ceph-mon[29336]: 5.a scrub ok 2024-09-10T19:08:36.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:36 smithi018 ceph-mon[29336]: pgmap v1107: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:08:36.946 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:36 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:08:37.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:36 smithi103 ceph-mon[33163]: 5.a scrub starts 2024-09-10T19:08:37.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:36 smithi103 ceph-mon[33163]: 5.a scrub ok 2024-09-10T19:08:37.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:36 smithi103 ceph-mon[33163]: pgmap v1107: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:08:37.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:36 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:08:37.582 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:37 smithi018 ceph-mon[29336]: 4.0 deep-scrub starts 2024-09-10T19:08:37.583 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:37 smithi018 ceph-mon[29336]: 6.1e scrub starts 2024-09-10T19:08:37.583 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:37 smithi018 ceph-mon[29336]: 4.0 deep-scrub ok 2024-09-10T19:08:37.583 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:37 smithi018 ceph-mon[29336]: 6.1e scrub ok 2024-09-10T19:08:37.583 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:37 smithi018 ceph-mon[29336]: 5.5 deep-scrub starts 2024-09-10T19:08:37.583 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:37 smithi018 ceph-mon[29336]: 5.5 deep-scrub ok 2024-09-10T19:08:37.583 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:37 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:08:37.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:37 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:08:37.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:37 smithi018 ceph-mon[29336]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:08:37.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:37 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:08:37.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:37 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:08:37.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:37 smithi018 ceph-mon[29336]: 2.1f scrub starts 2024-09-10T19:08:37.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:37 smithi018 ceph-mon[29336]: 2.1f scrub ok 2024-09-10T19:08:37.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:37 smithi018 ceph-mon[29336]: 1.4 deep-scrub starts 2024-09-10T19:08:37.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:37 smithi018 ceph-mon[29336]: 1.4 deep-scrub ok 2024-09-10T19:08:37.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:37 smithi018 ceph-mon[40196]: 4.0 deep-scrub starts 2024-09-10T19:08:37.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:37 smithi018 ceph-mon[40196]: 6.1e scrub starts 2024-09-10T19:08:37.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:37 smithi018 ceph-mon[40196]: 4.0 deep-scrub ok 2024-09-10T19:08:37.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:37 smithi018 ceph-mon[40196]: 6.1e scrub ok 2024-09-10T19:08:37.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:37 smithi018 ceph-mon[40196]: 5.5 deep-scrub starts 2024-09-10T19:08:37.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:37 smithi018 ceph-mon[40196]: 5.5 deep-scrub ok 2024-09-10T19:08:37.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:37 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:08:37.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:37 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:08:37.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:37 smithi018 ceph-mon[40196]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:08:37.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:37 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:08:37.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:37 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:08:37.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:37 smithi018 ceph-mon[40196]: 2.1f scrub starts 2024-09-10T19:08:37.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:37 smithi018 ceph-mon[40196]: 2.1f scrub ok 2024-09-10T19:08:37.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:37 smithi018 ceph-mon[40196]: 1.4 deep-scrub starts 2024-09-10T19:08:37.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:37 smithi018 ceph-mon[40196]: 1.4 deep-scrub ok 2024-09-10T19:08:37.772 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:08:37 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:08:37.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:37 smithi103 ceph-mon[33163]: 4.0 deep-scrub starts 2024-09-10T19:08:37.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:37 smithi103 ceph-mon[33163]: 6.1e scrub starts 2024-09-10T19:08:37.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:37 smithi103 ceph-mon[33163]: 4.0 deep-scrub ok 2024-09-10T19:08:37.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:37 smithi103 ceph-mon[33163]: 6.1e scrub ok 2024-09-10T19:08:37.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:37 smithi103 ceph-mon[33163]: 5.5 deep-scrub starts 2024-09-10T19:08:37.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:37 smithi103 ceph-mon[33163]: 5.5 deep-scrub ok 2024-09-10T19:08:37.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:37 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:08:37.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:37 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:08:37.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:37 smithi103 ceph-mon[33163]: from='mgr.14538 ' entity='mgr.y' 2024-09-10T19:08:37.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:37 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:08:37.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:37 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:08:37.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:37 smithi103 ceph-mon[33163]: 2.1f scrub starts 2024-09-10T19:08:37.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:37 smithi103 ceph-mon[33163]: 2.1f scrub ok 2024-09-10T19:08:37.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:37 smithi103 ceph-mon[33163]: 1.4 deep-scrub starts 2024-09-10T19:08:37.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:37 smithi103 ceph-mon[33163]: 1.4 deep-scrub ok 2024-09-10T19:08:37.875 INFO:teuthology.orchestra.run.smithi018.stdout:55834575160 2024-09-10T19:08:37.875 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-10T19:08:38.431 INFO:teuthology.orchestra.run.smithi018.stdout:77309411637 2024-09-10T19:08:38.432 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-10T19:08:38.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:38 smithi018 ceph-mon[29336]: 4.1b scrub starts 2024-09-10T19:08:38.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:38 smithi018 ceph-mon[29336]: 4.1b scrub ok 2024-09-10T19:08:38.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:38 smithi018 ceph-mon[29336]: 2.1 scrub starts 2024-09-10T19:08:38.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:38 smithi018 ceph-mon[29336]: 2.1 scrub ok 2024-09-10T19:08:38.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:38 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:38.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:38 smithi018 ceph-mon[29336]: pgmap v1108: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:38.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:38 smithi018 ceph-mon[29336]: 2.12 scrub starts 2024-09-10T19:08:38.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:38 smithi018 ceph-mon[29336]: 2.12 scrub ok 2024-09-10T19:08:38.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:38 smithi018 ceph-mon[29336]: osdmap e495: 8 total, 8 up, 8 in 2024-09-10T19:08:38.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:38 smithi018 ceph-mon[40196]: 4.1b scrub starts 2024-09-10T19:08:38.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:38 smithi018 ceph-mon[40196]: 4.1b scrub ok 2024-09-10T19:08:38.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:38 smithi018 ceph-mon[40196]: 2.1 scrub starts 2024-09-10T19:08:38.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:38 smithi018 ceph-mon[40196]: 2.1 scrub ok 2024-09-10T19:08:38.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:38 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:38.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:38 smithi018 ceph-mon[40196]: pgmap v1108: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:38.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:38 smithi018 ceph-mon[40196]: 2.12 scrub starts 2024-09-10T19:08:38.698 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:38 smithi018 ceph-mon[40196]: 2.12 scrub ok 2024-09-10T19:08:38.698 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:38 smithi018 ceph-mon[40196]: osdmap e495: 8 total, 8 up, 8 in 2024-09-10T19:08:39.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:38 smithi103 ceph-mon[33163]: 4.1b scrub starts 2024-09-10T19:08:39.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:38 smithi103 ceph-mon[33163]: 4.1b scrub ok 2024-09-10T19:08:39.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:38 smithi103 ceph-mon[33163]: 2.1 scrub starts 2024-09-10T19:08:39.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:38 smithi103 ceph-mon[33163]: 2.1 scrub ok 2024-09-10T19:08:39.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:38 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:39.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:38 smithi103 ceph-mon[33163]: pgmap v1108: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:39.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:38 smithi103 ceph-mon[33163]: 2.12 scrub starts 2024-09-10T19:08:39.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:38 smithi103 ceph-mon[33163]: 2.12 scrub ok 2024-09-10T19:08:39.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:38 smithi103 ceph-mon[33163]: osdmap e495: 8 total, 8 up, 8 in 2024-09-10T19:08:39.668 INFO:teuthology.orchestra.run.smithi018.stdout:128849019179 2024-09-10T19:08:39.668 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-10T19:08:39.895 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:39 smithi018 ceph-mon[40196]: 4.1 scrub starts 2024-09-10T19:08:39.895 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:39 smithi018 ceph-mon[40196]: 4.1 scrub ok 2024-09-10T19:08:39.895 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:39 smithi018 ceph-mon[29336]: 4.1 scrub starts 2024-09-10T19:08:39.895 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:39 smithi018 ceph-mon[29336]: 4.1 scrub ok 2024-09-10T19:08:40.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:39 smithi103 ceph-mon[33163]: 4.1 scrub starts 2024-09-10T19:08:40.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:39 smithi103 ceph-mon[33163]: 4.1 scrub ok 2024-09-10T19:08:40.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:08:40 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:08:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:08:40.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:40 smithi018 ceph-mon[29336]: pgmap v1110: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:08:40.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:40 smithi018 ceph-mon[29336]: 4.7 scrub starts 2024-09-10T19:08:40.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:40 smithi018 ceph-mon[29336]: 4.7 scrub ok 2024-09-10T19:08:40.694 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:40 smithi018 ceph-mon[29336]: 6.d scrub starts 2024-09-10T19:08:40.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:40 smithi018 ceph-mon[29336]: 6.d scrub ok 2024-09-10T19:08:40.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:40 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:40.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:40 smithi018 ceph-mon[29336]: 5.6 scrub starts 2024-09-10T19:08:40.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:40 smithi018 ceph-mon[29336]: 5.6 scrub ok 2024-09-10T19:08:40.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:40 smithi018 ceph-mon[40196]: pgmap v1110: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:08:40.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:40 smithi018 ceph-mon[40196]: 4.7 scrub starts 2024-09-10T19:08:40.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:40 smithi018 ceph-mon[40196]: 4.7 scrub ok 2024-09-10T19:08:40.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:40 smithi018 ceph-mon[40196]: 6.d scrub starts 2024-09-10T19:08:40.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:40 smithi018 ceph-mon[40196]: 6.d scrub ok 2024-09-10T19:08:40.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:40 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:40.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:40 smithi018 ceph-mon[40196]: 5.6 scrub starts 2024-09-10T19:08:40.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:40 smithi018 ceph-mon[40196]: 5.6 scrub ok 2024-09-10T19:08:41.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:40 smithi103 ceph-mon[33163]: pgmap v1110: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:08:41.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:40 smithi103 ceph-mon[33163]: 4.7 scrub starts 2024-09-10T19:08:41.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:40 smithi103 ceph-mon[33163]: 4.7 scrub ok 2024-09-10T19:08:41.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:40 smithi103 ceph-mon[33163]: 6.d scrub starts 2024-09-10T19:08:41.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:40 smithi103 ceph-mon[33163]: 6.d scrub ok 2024-09-10T19:08:41.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:40 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:41.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:40 smithi103 ceph-mon[33163]: 5.6 scrub starts 2024-09-10T19:08:41.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:40 smithi103 ceph-mon[33163]: 5.6 scrub ok 2024-09-10T19:08:41.376 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:08:40 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:08:40] "GET /metrics HTTP/1.1" 200 222084 "" "Prometheus/2.43.0" 2024-09-10T19:08:41.695 INFO:teuthology.orchestra.run.smithi018.stdout:158913790246 2024-09-10T19:08:41.696 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-10T19:08:41.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:41 smithi018 ceph-mon[29336]: 4.6 scrub starts 2024-09-10T19:08:41.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:41 smithi018 ceph-mon[29336]: 4.6 scrub ok 2024-09-10T19:08:41.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:41 smithi018 ceph-mon[40196]: 4.6 scrub starts 2024-09-10T19:08:41.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:41 smithi018 ceph-mon[40196]: 4.6 scrub ok 2024-09-10T19:08:42.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:41 smithi103 ceph-mon[33163]: 4.6 scrub starts 2024-09-10T19:08:42.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:41 smithi103 ceph-mon[33163]: 4.6 scrub ok 2024-09-10T19:08:42.177 INFO:teuthology.orchestra.run.smithi018.stdout:1765231558703 2024-09-10T19:08:42.177 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-10T19:08:42.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:42 smithi018 ceph-mon[29336]: 5.1d scrub starts 2024-09-10T19:08:42.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:42 smithi018 ceph-mon[29336]: pgmap v1111: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:08:42.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:42 smithi018 ceph-mon[29336]: 5.1d scrub ok 2024-09-10T19:08:42.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:42 smithi018 ceph-mon[29336]: 4.1c scrub starts 2024-09-10T19:08:42.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:42 smithi018 ceph-mon[29336]: 4.1c scrub ok 2024-09-10T19:08:42.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:42 smithi018 ceph-mon[40196]: 5.1d scrub starts 2024-09-10T19:08:42.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:42 smithi018 ceph-mon[40196]: pgmap v1111: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:08:42.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:42 smithi018 ceph-mon[40196]: 5.1d scrub ok 2024-09-10T19:08:42.695 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:42 smithi018 ceph-mon[40196]: 4.1c scrub starts 2024-09-10T19:08:42.696 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:42 smithi018 ceph-mon[40196]: 4.1c scrub ok 2024-09-10T19:08:43.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:42 smithi103 ceph-mon[33163]: 5.1d scrub starts 2024-09-10T19:08:43.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:42 smithi103 ceph-mon[33163]: pgmap v1111: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:08:43.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:42 smithi103 ceph-mon[33163]: 5.1d scrub ok 2024-09-10T19:08:43.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:42 smithi103 ceph-mon[33163]: 4.1c scrub starts 2024-09-10T19:08:43.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:42 smithi103 ceph-mon[33163]: 4.1c scrub ok 2024-09-10T19:08:43.246 INFO:teuthology.orchestra.run.smithi018.stdout:107374182706 2024-09-10T19:08:43.247 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-10T19:08:43.735 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:08:43.774 INFO:teuthology.orchestra.run.smithi018.stdout:184683594018 2024-09-10T19:08:43.775 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-10T19:08:43.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:43 smithi018 ceph-mon[29336]: 4.4 scrub starts 2024-09-10T19:08:43.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:43 smithi018 ceph-mon[29336]: 4.4 scrub ok 2024-09-10T19:08:43.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:43 smithi018 ceph-mon[29336]: 6.3 scrub starts 2024-09-10T19:08:43.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:43 smithi018 ceph-mon[29336]: 6.3 scrub ok 2024-09-10T19:08:43.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:43 smithi018 ceph-mon[40196]: 4.4 scrub starts 2024-09-10T19:08:43.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:43 smithi018 ceph-mon[40196]: 4.4 scrub ok 2024-09-10T19:08:43.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:43 smithi018 ceph-mon[40196]: 6.3 scrub starts 2024-09-10T19:08:43.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:43 smithi018 ceph-mon[40196]: 6.3 scrub ok 2024-09-10T19:08:44.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:43 smithi103 ceph-mon[33163]: 4.4 scrub starts 2024-09-10T19:08:44.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:43 smithi103 ceph-mon[33163]: 4.4 scrub ok 2024-09-10T19:08:44.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:43 smithi103 ceph-mon[33163]: 6.3 scrub starts 2024-09-10T19:08:44.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:43 smithi103 ceph-mon[33163]: 6.3 scrub ok 2024-09-10T19:08:44.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:44 smithi018 ceph-mon[29336]: 4.e scrub starts 2024-09-10T19:08:44.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:44 smithi018 ceph-mon[29336]: 4.e scrub ok 2024-09-10T19:08:44.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:44 smithi018 ceph-mon[29336]: pgmap v1112: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:08:44.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:44 smithi018 ceph-mon[29336]: 6.4 deep-scrub starts 2024-09-10T19:08:44.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:44 smithi018 ceph-mon[29336]: 6.4 deep-scrub ok 2024-09-10T19:08:44.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:44 smithi018 ceph-mon[29336]: 5.3 scrub starts 2024-09-10T19:08:44.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:44 smithi018 ceph-mon[29336]: 5.3 scrub ok 2024-09-10T19:08:44.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:44 smithi018 ceph-mon[40196]: 4.e scrub starts 2024-09-10T19:08:44.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:44 smithi018 ceph-mon[40196]: 4.e scrub ok 2024-09-10T19:08:44.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:44 smithi018 ceph-mon[40196]: pgmap v1112: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:08:44.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:44 smithi018 ceph-mon[40196]: 6.4 deep-scrub starts 2024-09-10T19:08:44.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:44 smithi018 ceph-mon[40196]: 6.4 deep-scrub ok 2024-09-10T19:08:44.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:44 smithi018 ceph-mon[40196]: 5.3 scrub starts 2024-09-10T19:08:44.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:44 smithi018 ceph-mon[40196]: 5.3 scrub ok 2024-09-10T19:08:45.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:44 smithi103 ceph-mon[33163]: 4.e scrub starts 2024-09-10T19:08:45.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:44 smithi103 ceph-mon[33163]: 4.e scrub ok 2024-09-10T19:08:45.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:44 smithi103 ceph-mon[33163]: pgmap v1112: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:08:45.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:44 smithi103 ceph-mon[33163]: 6.4 deep-scrub starts 2024-09-10T19:08:45.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:44 smithi103 ceph-mon[33163]: 6.4 deep-scrub ok 2024-09-10T19:08:45.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:44 smithi103 ceph-mon[33163]: 5.3 scrub starts 2024-09-10T19:08:45.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:44 smithi103 ceph-mon[33163]: 5.3 scrub ok 2024-09-10T19:08:45.275 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:08:45.276 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:08:45.278 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:08:45.279 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:08:45.280 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:08:45.457 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:08:45.655 INFO:teuthology.orchestra.run.smithi018.stdout:77309411638 2024-09-10T19:08:46.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:46 smithi018 ceph-mon[29336]: pgmap v1113: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 818 B/s rd, 0 op/s 2024-09-10T19:08:46.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:46 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3949312060' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T19:08:46.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:46 smithi018 ceph-mon[40196]: pgmap v1113: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 818 B/s rd, 0 op/s 2024-09-10T19:08:46.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:46 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3949312060' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T19:08:47.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:46 smithi103 ceph-mon[33163]: pgmap v1113: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 818 B/s rd, 0 op/s 2024-09-10T19:08:47.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:46 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3949312060' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T19:08:47.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:47 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:08:47.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:47 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:08:47.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:47 smithi018 ceph-mon[29336]: 6.8 scrub starts 2024-09-10T19:08:47.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:47 smithi018 ceph-mon[29336]: 6.8 scrub ok 2024-09-10T19:08:47.695 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:47 smithi018 ceph-mon[29336]: 5.19 scrub starts 2024-09-10T19:08:47.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:47 smithi018 ceph-mon[29336]: 5.19 scrub ok 2024-09-10T19:08:47.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:47 smithi018 ceph-mon[29336]: osdmap e496: 8 total, 8 up, 8 in 2024-09-10T19:08:47.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:47 smithi018 ceph-mon[29336]: 1.c scrub starts 2024-09-10T19:08:47.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:47 smithi018 ceph-mon[29336]: 1.c scrub ok 2024-09-10T19:08:47.696 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:47 smithi018 ceph-mon[29336]: 6.f scrub starts 2024-09-10T19:08:47.697 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:47 smithi018 ceph-mon[29336]: 6.f scrub ok 2024-09-10T19:08:47.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:47 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:08:47.697 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:47 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:08:47.698 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:47 smithi018 ceph-mon[40196]: 6.8 scrub starts 2024-09-10T19:08:47.698 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:47 smithi018 ceph-mon[40196]: 6.8 scrub ok 2024-09-10T19:08:47.698 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:47 smithi018 ceph-mon[40196]: 5.19 scrub starts 2024-09-10T19:08:47.698 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:47 smithi018 ceph-mon[40196]: 5.19 scrub ok 2024-09-10T19:08:47.698 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:47 smithi018 ceph-mon[40196]: osdmap e496: 8 total, 8 up, 8 in 2024-09-10T19:08:47.698 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:47 smithi018 ceph-mon[40196]: 1.c scrub starts 2024-09-10T19:08:47.698 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:47 smithi018 ceph-mon[40196]: 1.c scrub ok 2024-09-10T19:08:47.699 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:47 smithi018 ceph-mon[40196]: 6.f scrub starts 2024-09-10T19:08:47.699 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:47 smithi018 ceph-mon[40196]: 6.f scrub ok 2024-09-10T19:08:47.773 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:08:47 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:08:47.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:47 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:08:47.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:47 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:08:47.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:47 smithi103 ceph-mon[33163]: 6.8 scrub starts 2024-09-10T19:08:47.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:47 smithi103 ceph-mon[33163]: 6.8 scrub ok 2024-09-10T19:08:47.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:47 smithi103 ceph-mon[33163]: 5.19 scrub starts 2024-09-10T19:08:47.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:47 smithi103 ceph-mon[33163]: 5.19 scrub ok 2024-09-10T19:08:47.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:47 smithi103 ceph-mon[33163]: osdmap e496: 8 total, 8 up, 8 in 2024-09-10T19:08:47.773 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:47 smithi103 ceph-mon[33163]: 1.c scrub starts 2024-09-10T19:08:47.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:47 smithi103 ceph-mon[33163]: 1.c scrub ok 2024-09-10T19:08:47.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:47 smithi103 ceph-mon[33163]: 6.f scrub starts 2024-09-10T19:08:47.774 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:47 smithi103 ceph-mon[33163]: 6.f scrub ok 2024-09-10T19:08:48.259 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:08:48.654 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:48 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:48.655 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:48 smithi018 ceph-mon[29336]: pgmap v1115: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T19:08:48.655 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:48 smithi018 ceph-mon[29336]: 4.a scrub starts 2024-09-10T19:08:48.655 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:48 smithi018 ceph-mon[29336]: 4.a scrub ok 2024-09-10T19:08:48.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:48 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:48.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:48 smithi018 ceph-mon[40196]: pgmap v1115: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T19:08:48.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:48 smithi018 ceph-mon[40196]: 4.a scrub starts 2024-09-10T19:08:48.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:48 smithi018 ceph-mon[40196]: 4.a scrub ok 2024-09-10T19:08:49.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:48 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:49.022 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:48 smithi103 ceph-mon[33163]: pgmap v1115: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T19:08:49.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:48 smithi103 ceph-mon[33163]: 4.a scrub starts 2024-09-10T19:08:49.023 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:48 smithi103 ceph-mon[33163]: 4.a scrub ok 2024-09-10T19:08:49.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:49 smithi018 ceph-mon[29336]: 6.12 scrub starts 2024-09-10T19:08:49.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:49 smithi018 ceph-mon[29336]: 6.12 scrub ok 2024-09-10T19:08:49.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:49 smithi018 ceph-mon[29336]: 2.d scrub starts 2024-09-10T19:08:49.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:49 smithi018 ceph-mon[29336]: 2.d scrub ok 2024-09-10T19:08:49.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:49 smithi018 ceph-mon[29336]: 1.5 scrub starts 2024-09-10T19:08:49.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:49 smithi018 ceph-mon[29336]: 1.5 scrub ok 2024-09-10T19:08:49.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:49 smithi018 ceph-mon[29336]: pgmap v1116: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:08:49.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:49 smithi018 ceph-mon[40196]: 6.12 scrub starts 2024-09-10T19:08:49.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:49 smithi018 ceph-mon[40196]: 6.12 scrub ok 2024-09-10T19:08:49.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:49 smithi018 ceph-mon[40196]: 2.d scrub starts 2024-09-10T19:08:49.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:49 smithi018 ceph-mon[40196]: 2.d scrub ok 2024-09-10T19:08:49.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:49 smithi018 ceph-mon[40196]: 1.5 scrub starts 2024-09-10T19:08:49.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:49 smithi018 ceph-mon[40196]: 1.5 scrub ok 2024-09-10T19:08:49.946 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:49 smithi018 ceph-mon[40196]: pgmap v1116: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:08:50.041 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411637 got 77309411638 for osd.2 2024-09-10T19:08:50.041 DEBUG:teuthology.parallel:result is None 2024-09-10T19:08:50.238 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:49 smithi103 ceph-mon[33163]: 6.12 scrub starts 2024-09-10T19:08:50.238 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:49 smithi103 ceph-mon[33163]: 6.12 scrub ok 2024-09-10T19:08:50.238 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:49 smithi103 ceph-mon[33163]: 2.d scrub starts 2024-09-10T19:08:50.238 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:49 smithi103 ceph-mon[33163]: 2.d scrub ok 2024-09-10T19:08:50.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:49 smithi103 ceph-mon[33163]: 1.5 scrub starts 2024-09-10T19:08:50.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:49 smithi103 ceph-mon[33163]: 1.5 scrub ok 2024-09-10T19:08:50.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:49 smithi103 ceph-mon[33163]: pgmap v1116: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:08:50.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:08:50 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:08:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:08:50.971 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:50 smithi018 ceph-mon[29336]: 4.10 deep-scrub starts 2024-09-10T19:08:50.971 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:50 smithi018 ceph-mon[29336]: 4.10 deep-scrub ok 2024-09-10T19:08:50.971 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:50 smithi018 ceph-mon[29336]: 5.b scrub starts 2024-09-10T19:08:50.971 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:50 smithi018 ceph-mon[29336]: 5.b scrub ok 2024-09-10T19:08:50.972 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:50 smithi018 ceph-mon[40196]: 4.10 deep-scrub starts 2024-09-10T19:08:50.972 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:50 smithi018 ceph-mon[40196]: 4.10 deep-scrub ok 2024-09-10T19:08:50.972 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:50 smithi018 ceph-mon[40196]: 5.b scrub starts 2024-09-10T19:08:50.972 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:50 smithi018 ceph-mon[40196]: 5.b scrub ok 2024-09-10T19:08:51.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:50 smithi103 ceph-mon[33163]: 4.10 deep-scrub starts 2024-09-10T19:08:51.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:50 smithi103 ceph-mon[33163]: 4.10 deep-scrub ok 2024-09-10T19:08:51.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:50 smithi103 ceph-mon[33163]: 5.b scrub starts 2024-09-10T19:08:51.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:50 smithi103 ceph-mon[33163]: 5.b scrub ok 2024-09-10T19:08:51.392 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:08:50 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:08:50] "GET /metrics HTTP/1.1" 200 222084 "" "Prometheus/2.43.0" 2024-09-10T19:08:52.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:51 smithi018 ceph-mon[29336]: pgmap v1117: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:08:52.205 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:51 smithi018 ceph-mon[40196]: pgmap v1117: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:08:52.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:51 smithi103 ceph-mon[33163]: pgmap v1117: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:08:53.020 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:52 smithi018 ceph-mon[29336]: 6.c scrub starts 2024-09-10T19:08:53.020 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:52 smithi018 ceph-mon[29336]: 6.c scrub ok 2024-09-10T19:08:53.020 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:52 smithi018 ceph-mon[29336]: 6.1a scrub starts 2024-09-10T19:08:53.021 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:52 smithi018 ceph-mon[29336]: 6.1a scrub ok 2024-09-10T19:08:53.021 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:52 smithi018 ceph-mon[40196]: 6.c scrub starts 2024-09-10T19:08:53.021 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:52 smithi018 ceph-mon[40196]: 6.c scrub ok 2024-09-10T19:08:53.021 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:52 smithi018 ceph-mon[40196]: 6.1a scrub starts 2024-09-10T19:08:53.021 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:52 smithi018 ceph-mon[40196]: 6.1a scrub ok 2024-09-10T19:08:53.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:52 smithi103 ceph-mon[33163]: 6.c scrub starts 2024-09-10T19:08:53.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:52 smithi103 ceph-mon[33163]: 6.c scrub ok 2024-09-10T19:08:53.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:52 smithi103 ceph-mon[33163]: 6.1a scrub starts 2024-09-10T19:08:53.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:52 smithi103 ceph-mon[33163]: 6.1a scrub ok 2024-09-10T19:08:53.649 INFO:teuthology.orchestra.run.smithi018.stdout:1765231558706 2024-09-10T19:08:54.187 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:53 smithi018 ceph-mon[40196]: 4.13 scrub starts 2024-09-10T19:08:54.188 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:53 smithi018 ceph-mon[40196]: 4.13 scrub ok 2024-09-10T19:08:54.188 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:53 smithi018 ceph-mon[40196]: pgmap v1118: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:08:54.188 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:53 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3815133082' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T19:08:54.189 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:53 smithi018 ceph-mon[29336]: 4.13 scrub starts 2024-09-10T19:08:54.189 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:53 smithi018 ceph-mon[29336]: 4.13 scrub ok 2024-09-10T19:08:54.189 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:53 smithi018 ceph-mon[29336]: pgmap v1118: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:08:54.189 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:53 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3815133082' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T19:08:54.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:53 smithi103 ceph-mon[33163]: 4.13 scrub starts 2024-09-10T19:08:54.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:53 smithi103 ceph-mon[33163]: 4.13 scrub ok 2024-09-10T19:08:54.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:53 smithi103 ceph-mon[33163]: pgmap v1118: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:08:54.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:53 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3815133082' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T19:08:54.360 INFO:teuthology.orchestra.run.smithi018.stdout:128849019183 2024-09-10T19:08:54.801 INFO:tasks.cephadm.ceph_manager.ceph:need seq 1765231558703 got 1765231558706 for osd.7 2024-09-10T19:08:54.801 DEBUG:teuthology.parallel:result is None 2024-09-10T19:08:55.098 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:54 smithi018 ceph-mon[29336]: 5.13 scrub starts 2024-09-10T19:08:55.098 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:54 smithi018 ceph-mon[29336]: 5.13 scrub ok 2024-09-10T19:08:55.098 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:54 smithi018 ceph-mon[29336]: 1.14 scrub starts 2024-09-10T19:08:55.098 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:54 smithi018 ceph-mon[29336]: 1.14 scrub ok 2024-09-10T19:08:55.099 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:54 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1464184970' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T19:08:55.099 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:54 smithi018 ceph-mon[40196]: 5.13 scrub starts 2024-09-10T19:08:55.099 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:54 smithi018 ceph-mon[40196]: 5.13 scrub ok 2024-09-10T19:08:55.099 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:54 smithi018 ceph-mon[40196]: 1.14 scrub starts 2024-09-10T19:08:55.099 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:54 smithi018 ceph-mon[40196]: 1.14 scrub ok 2024-09-10T19:08:55.100 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:54 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1464184970' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T19:08:55.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:54 smithi103 ceph-mon[33163]: 5.13 scrub starts 2024-09-10T19:08:55.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:54 smithi103 ceph-mon[33163]: 5.13 scrub ok 2024-09-10T19:08:55.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:54 smithi103 ceph-mon[33163]: 1.14 scrub starts 2024-09-10T19:08:55.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:54 smithi103 ceph-mon[33163]: 1.14 scrub ok 2024-09-10T19:08:55.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:54 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1464184970' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T19:08:56.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:55 smithi018 ceph-mon[40196]: 2.4 deep-scrub starts 2024-09-10T19:08:56.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:55 smithi018 ceph-mon[40196]: 2.4 deep-scrub ok 2024-09-10T19:08:56.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:55 smithi018 ceph-mon[40196]: 3.2 scrub starts 2024-09-10T19:08:56.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:55 smithi018 ceph-mon[40196]: 3.2 scrub ok 2024-09-10T19:08:56.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:55 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:56.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:55 smithi018 ceph-mon[40196]: 2.10 scrub starts 2024-09-10T19:08:56.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:55 smithi018 ceph-mon[40196]: 2.10 scrub ok 2024-09-10T19:08:56.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:55 smithi018 ceph-mon[40196]: pgmap v1119: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:08:56.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:55 smithi018 ceph-mon[29336]: 2.4 deep-scrub starts 2024-09-10T19:08:56.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:55 smithi018 ceph-mon[29336]: 2.4 deep-scrub ok 2024-09-10T19:08:56.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:55 smithi018 ceph-mon[29336]: 3.2 scrub starts 2024-09-10T19:08:56.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:55 smithi018 ceph-mon[29336]: 3.2 scrub ok 2024-09-10T19:08:56.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:55 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:56.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:55 smithi018 ceph-mon[29336]: 2.10 scrub starts 2024-09-10T19:08:56.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:55 smithi018 ceph-mon[29336]: 2.10 scrub ok 2024-09-10T19:08:56.195 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:55 smithi018 ceph-mon[29336]: pgmap v1119: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:08:56.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:55 smithi103 ceph-mon[33163]: 2.4 deep-scrub starts 2024-09-10T19:08:56.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:55 smithi103 ceph-mon[33163]: 2.4 deep-scrub ok 2024-09-10T19:08:56.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:55 smithi103 ceph-mon[33163]: 3.2 scrub starts 2024-09-10T19:08:56.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:55 smithi103 ceph-mon[33163]: 3.2 scrub ok 2024-09-10T19:08:56.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:55 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:56.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:55 smithi103 ceph-mon[33163]: 2.10 scrub starts 2024-09-10T19:08:56.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:55 smithi103 ceph-mon[33163]: 2.10 scrub ok 2024-09-10T19:08:56.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:55 smithi103 ceph-mon[33163]: pgmap v1119: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:08:56.896 INFO:teuthology.orchestra.run.smithi018.stdout:1035087118497 2024-09-10T19:08:57.123 INFO:teuthology.orchestra.run.smithi018.stdout:107374182708 2024-09-10T19:08:57.124 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:56 smithi018 ceph-mon[40196]: 5.2 scrub starts 2024-09-10T19:08:57.124 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:56 smithi018 ceph-mon[40196]: 5.2 scrub ok 2024-09-10T19:08:57.124 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:56 smithi018 ceph-mon[40196]: 4.c deep-scrub starts 2024-09-10T19:08:57.124 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:56 smithi018 ceph-mon[40196]: 4.c deep-scrub ok 2024-09-10T19:08:57.124 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:56 smithi018 ceph-mon[40196]: 4.15 scrub starts 2024-09-10T19:08:57.124 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:56 smithi018 ceph-mon[40196]: 4.15 scrub ok 2024-09-10T19:08:57.124 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:56 smithi018 ceph-mon[29336]: 5.2 scrub starts 2024-09-10T19:08:57.125 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:56 smithi018 ceph-mon[29336]: 5.2 scrub ok 2024-09-10T19:08:57.125 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:56 smithi018 ceph-mon[29336]: 4.c deep-scrub starts 2024-09-10T19:08:57.125 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:56 smithi018 ceph-mon[29336]: 4.c deep-scrub ok 2024-09-10T19:08:57.125 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:56 smithi018 ceph-mon[29336]: 4.15 scrub starts 2024-09-10T19:08:57.125 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:56 smithi018 ceph-mon[29336]: 4.15 scrub ok 2024-09-10T19:08:57.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:56 smithi103 ceph-mon[33163]: 5.2 scrub starts 2024-09-10T19:08:57.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:56 smithi103 ceph-mon[33163]: 5.2 scrub ok 2024-09-10T19:08:57.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:56 smithi103 ceph-mon[33163]: 4.c deep-scrub starts 2024-09-10T19:08:57.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:56 smithi103 ceph-mon[33163]: 4.c deep-scrub ok 2024-09-10T19:08:57.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:56 smithi103 ceph-mon[33163]: 4.15 scrub starts 2024-09-10T19:08:57.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:56 smithi103 ceph-mon[33163]: 4.15 scrub ok 2024-09-10T19:08:57.403 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849019179 got 128849019183 for osd.4 2024-09-10T19:08:57.403 DEBUG:teuthology.parallel:result is None 2024-09-10T19:08:57.772 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:08:57 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:08:57.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:57 smithi018 ceph-mon[29336]: 5.12 deep-scrub starts 2024-09-10T19:08:57.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:57 smithi018 ceph-mon[29336]: 5.12 deep-scrub ok 2024-09-10T19:08:57.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:57 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3873248576' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T19:08:57.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:57 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3535911320' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T19:08:57.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:57 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:57.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:57 smithi018 ceph-mon[29336]: pgmap v1120: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-10T19:08:57.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:57 smithi018 ceph-mon[40196]: 5.12 deep-scrub starts 2024-09-10T19:08:57.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:57 smithi018 ceph-mon[40196]: 5.12 deep-scrub ok 2024-09-10T19:08:57.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:57 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3873248576' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T19:08:57.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:57 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3535911320' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T19:08:57.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:57 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:57.945 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:57 smithi018 ceph-mon[40196]: pgmap v1120: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-10T19:08:58.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:57 smithi103 ceph-mon[33163]: 5.12 deep-scrub starts 2024-09-10T19:08:58.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:57 smithi103 ceph-mon[33163]: 5.12 deep-scrub ok 2024-09-10T19:08:58.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:57 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3873248576' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T19:08:58.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:57 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3535911320' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T19:08:58.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:57 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:58.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:57 smithi103 ceph-mon[33163]: pgmap v1120: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-10T19:08:58.335 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182706 got 107374182708 for osd.3 2024-09-10T19:08:58.336 DEBUG:teuthology.parallel:result is None 2024-09-10T19:08:59.015 INFO:tasks.cephadm.ceph_manager.ceph:need seq 1035087118493 got 1035087118497 for osd.0 2024-09-10T19:08:59.015 DEBUG:teuthology.parallel:result is None 2024-09-10T19:08:59.154 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:58 smithi018 ceph-mon[29336]: 5.1c deep-scrub starts 2024-09-10T19:08:59.155 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:58 smithi018 ceph-mon[29336]: 5.1c deep-scrub ok 2024-09-10T19:08:59.155 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:58 smithi018 ceph-mon[40196]: 5.1c deep-scrub starts 2024-09-10T19:08:59.155 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:58 smithi018 ceph-mon[40196]: 5.1c deep-scrub ok 2024-09-10T19:08:59.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:58 smithi103 ceph-mon[33163]: 5.1c deep-scrub starts 2024-09-10T19:08:59.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:58 smithi103 ceph-mon[33163]: 5.1c deep-scrub ok 2024-09-10T19:08:59.414 INFO:teuthology.orchestra.run.smithi018.stdout:158913790250 2024-09-10T19:09:00.120 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:59 smithi018 ceph-mon[40196]: 2.17 scrub starts 2024-09-10T19:09:00.120 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:59 smithi018 ceph-mon[40196]: 2.17 scrub ok 2024-09-10T19:09:00.120 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:59 smithi018 ceph-mon[40196]: 4.14 deep-scrub starts 2024-09-10T19:09:00.120 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:59 smithi018 ceph-mon[40196]: 4.b scrub starts 2024-09-10T19:09:00.120 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:59 smithi018 ceph-mon[40196]: 4.b scrub ok 2024-09-10T19:09:00.120 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:59 smithi018 ceph-mon[40196]: 4.14 deep-scrub ok 2024-09-10T19:09:00.121 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:59 smithi018 ceph-mon[40196]: 4.18 scrub starts 2024-09-10T19:09:00.121 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:59 smithi018 ceph-mon[40196]: 4.18 scrub ok 2024-09-10T19:09:00.121 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:59 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/3707139937' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T19:09:00.121 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:08:59 smithi018 ceph-mon[40196]: pgmap v1121: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:09:00.121 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:59 smithi018 ceph-mon[29336]: 2.17 scrub starts 2024-09-10T19:09:00.121 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:59 smithi018 ceph-mon[29336]: 2.17 scrub ok 2024-09-10T19:09:00.121 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:59 smithi018 ceph-mon[29336]: 4.14 deep-scrub starts 2024-09-10T19:09:00.121 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:59 smithi018 ceph-mon[29336]: 4.b scrub starts 2024-09-10T19:09:00.121 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:59 smithi018 ceph-mon[29336]: 4.b scrub ok 2024-09-10T19:09:00.121 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:59 smithi018 ceph-mon[29336]: 4.14 deep-scrub ok 2024-09-10T19:09:00.122 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:59 smithi018 ceph-mon[29336]: 4.18 scrub starts 2024-09-10T19:09:00.122 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:59 smithi018 ceph-mon[29336]: 4.18 scrub ok 2024-09-10T19:09:00.122 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:59 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/3707139937' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T19:09:00.122 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:08:59 smithi018 ceph-mon[29336]: pgmap v1121: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:09:00.146 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790246 got 158913790250 for osd.5 2024-09-10T19:09:00.146 DEBUG:teuthology.parallel:result is None 2024-09-10T19:09:00.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:59 smithi103 ceph-mon[33163]: 2.17 scrub starts 2024-09-10T19:09:00.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:59 smithi103 ceph-mon[33163]: 2.17 scrub ok 2024-09-10T19:09:00.240 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:59 smithi103 ceph-mon[33163]: 4.14 deep-scrub starts 2024-09-10T19:09:00.240 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:59 smithi103 ceph-mon[33163]: 4.b scrub starts 2024-09-10T19:09:00.240 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:59 smithi103 ceph-mon[33163]: 4.b scrub ok 2024-09-10T19:09:00.240 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:59 smithi103 ceph-mon[33163]: 4.14 deep-scrub ok 2024-09-10T19:09:00.240 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:59 smithi103 ceph-mon[33163]: 4.18 scrub starts 2024-09-10T19:09:00.240 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:59 smithi103 ceph-mon[33163]: 4.18 scrub ok 2024-09-10T19:09:00.240 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:59 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/3707139937' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T19:09:00.240 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:08:59 smithi103 ceph-mon[33163]: pgmap v1121: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:09:00.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:09:00 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:09:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:09:00.822 INFO:teuthology.orchestra.run.smithi018.stdout:184683594022 2024-09-10T19:09:00.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:00 smithi018 ceph-mon[40196]: 4.d scrub starts 2024-09-10T19:09:00.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:00 smithi018 ceph-mon[40196]: 4.d scrub ok 2024-09-10T19:09:00.944 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:00 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/1244390106' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T19:09:00.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:00 smithi018 ceph-mon[29336]: 4.d scrub starts 2024-09-10T19:09:00.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:00 smithi018 ceph-mon[29336]: 4.d scrub ok 2024-09-10T19:09:00.945 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:00 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/1244390106' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T19:09:01.034 INFO:teuthology.orchestra.run.smithi018.stdout:55834575165 2024-09-10T19:09:01.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:00 smithi103 ceph-mon[33163]: 4.d scrub starts 2024-09-10T19:09:01.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:00 smithi103 ceph-mon[33163]: 4.d scrub ok 2024-09-10T19:09:01.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:00 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/1244390106' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T19:09:01.437 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:09:00 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:09:00] "GET /metrics HTTP/1.1" 200 222103 "" "Prometheus/2.43.0" 2024-09-10T19:09:02.124 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683594018 got 184683594022 for osd.6 2024-09-10T19:09:02.124 DEBUG:teuthology.parallel:result is None 2024-09-10T19:09:02.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:01 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/76514166' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T19:09:02.196 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:01 smithi018 ceph-mon[29336]: pgmap v1122: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:09:02.197 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:01 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/76514166' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T19:09:02.197 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:01 smithi018 ceph-mon[40196]: pgmap v1122: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:09:02.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:01 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/76514166' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T19:09:02.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:01 smithi103 ceph-mon[33163]: pgmap v1122: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:09:02.576 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834575160 got 55834575165 for osd.1 2024-09-10T19:09:02.576 DEBUG:teuthology.parallel:result is None 2024-09-10T19:09:02.576 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-10T19:09:02.577 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T19:09:02.845 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:09:03.173 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:02 smithi018 ceph-mon[40196]: 5.4 scrub starts 2024-09-10T19:09:03.173 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:02 smithi018 ceph-mon[40196]: 5.4 scrub ok 2024-09-10T19:09:03.173 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:02 smithi018 ceph-mon[40196]: 2.9 scrub starts 2024-09-10T19:09:03.173 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:02 smithi018 ceph-mon[40196]: 2.9 scrub ok 2024-09-10T19:09:03.173 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:02 smithi018 ceph-mon[40196]: 1.8 scrub starts 2024-09-10T19:09:03.173 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:02 smithi018 ceph-mon[40196]: 1.8 scrub ok 2024-09-10T19:09:03.173 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:02 smithi018 ceph-mon[40196]: 4.16 scrub starts 2024-09-10T19:09:03.173 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:02 smithi018 ceph-mon[40196]: 4.16 scrub ok 2024-09-10T19:09:03.174 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:02 smithi018 ceph-mon[29336]: 5.4 scrub starts 2024-09-10T19:09:03.174 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:02 smithi018 ceph-mon[29336]: 5.4 scrub ok 2024-09-10T19:09:03.174 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:02 smithi018 ceph-mon[29336]: 2.9 scrub starts 2024-09-10T19:09:03.174 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:02 smithi018 ceph-mon[29336]: 2.9 scrub ok 2024-09-10T19:09:03.174 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:02 smithi018 ceph-mon[29336]: 1.8 scrub starts 2024-09-10T19:09:03.174 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:02 smithi018 ceph-mon[29336]: 1.8 scrub ok 2024-09-10T19:09:03.174 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:02 smithi018 ceph-mon[29336]: 4.16 scrub starts 2024-09-10T19:09:03.174 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:02 smithi018 ceph-mon[29336]: 4.16 scrub ok 2024-09-10T19:09:03.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:02 smithi103 ceph-mon[33163]: 5.4 scrub starts 2024-09-10T19:09:03.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:02 smithi103 ceph-mon[33163]: 5.4 scrub ok 2024-09-10T19:09:03.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:02 smithi103 ceph-mon[33163]: 2.9 scrub starts 2024-09-10T19:09:03.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:02 smithi103 ceph-mon[33163]: 2.9 scrub ok 2024-09-10T19:09:03.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:02 smithi103 ceph-mon[33163]: 1.8 scrub starts 2024-09-10T19:09:03.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:02 smithi103 ceph-mon[33163]: 1.8 scrub ok 2024-09-10T19:09:03.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:02 smithi103 ceph-mon[33163]: 4.16 scrub starts 2024-09-10T19:09:03.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:02 smithi103 ceph-mon[33163]: 4.16 scrub ok 2024-09-10T19:09:04.184 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:03 smithi018 ceph-mon[29336]: 1.9 scrub starts 2024-09-10T19:09:04.185 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:03 smithi018 ceph-mon[29336]: 1.9 scrub ok 2024-09-10T19:09:04.185 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:03 smithi018 ceph-mon[29336]: 2.1b deep-scrub starts 2024-09-10T19:09:04.185 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:03 smithi018 ceph-mon[29336]: 2.1b deep-scrub ok 2024-09-10T19:09:04.185 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:03 smithi018 ceph-mon[29336]: 2.13 scrub starts 2024-09-10T19:09:04.185 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:03 smithi018 ceph-mon[29336]: 2.13 scrub ok 2024-09-10T19:09:04.185 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:03 smithi018 ceph-mon[29336]: 6.2 scrub starts 2024-09-10T19:09:04.185 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:03 smithi018 ceph-mon[29336]: 6.2 scrub ok 2024-09-10T19:09:04.185 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:03 smithi018 ceph-mon[29336]: pgmap v1123: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:09:04.185 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:03 smithi018 ceph-mon[40196]: 1.9 scrub starts 2024-09-10T19:09:04.186 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:03 smithi018 ceph-mon[40196]: 1.9 scrub ok 2024-09-10T19:09:04.186 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:03 smithi018 ceph-mon[40196]: 2.1b deep-scrub starts 2024-09-10T19:09:04.186 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:03 smithi018 ceph-mon[40196]: 2.1b deep-scrub ok 2024-09-10T19:09:04.186 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:03 smithi018 ceph-mon[40196]: 2.13 scrub starts 2024-09-10T19:09:04.186 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:03 smithi018 ceph-mon[40196]: 2.13 scrub ok 2024-09-10T19:09:04.186 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:03 smithi018 ceph-mon[40196]: 6.2 scrub starts 2024-09-10T19:09:04.186 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:03 smithi018 ceph-mon[40196]: 6.2 scrub ok 2024-09-10T19:09:04.186 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:03 smithi018 ceph-mon[40196]: pgmap v1123: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:09:04.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:03 smithi103 ceph-mon[33163]: 1.9 scrub starts 2024-09-10T19:09:04.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:03 smithi103 ceph-mon[33163]: 1.9 scrub ok 2024-09-10T19:09:04.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:03 smithi103 ceph-mon[33163]: 2.1b deep-scrub starts 2024-09-10T19:09:04.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:03 smithi103 ceph-mon[33163]: 2.1b deep-scrub ok 2024-09-10T19:09:04.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:03 smithi103 ceph-mon[33163]: 2.13 scrub starts 2024-09-10T19:09:04.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:03 smithi103 ceph-mon[33163]: 2.13 scrub ok 2024-09-10T19:09:04.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:03 smithi103 ceph-mon[33163]: 6.2 scrub starts 2024-09-10T19:09:04.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:03 smithi103 ceph-mon[33163]: 6.2 scrub ok 2024-09-10T19:09:04.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:03 smithi103 ceph-mon[33163]: pgmap v1123: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:09:04.755 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T19:09:04.758 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-09-10T19:09:05.110 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:04 smithi018 ceph-mon[40196]: 2.c scrub starts 2024-09-10T19:09:05.111 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:04 smithi018 ceph-mon[40196]: 2.c scrub ok 2024-09-10T19:09:05.111 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:04 smithi018 ceph-mon[40196]: 4.17 scrub starts 2024-09-10T19:09:05.111 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:04 smithi018 ceph-mon[40196]: 4.17 scrub ok 2024-09-10T19:09:05.111 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:04 smithi018 ceph-mon[40196]: 1.e deep-scrub starts 2024-09-10T19:09:05.111 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:04 smithi018 ceph-mon[40196]: 1.e deep-scrub ok 2024-09-10T19:09:05.111 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:04 smithi018 ceph-mon[29336]: 2.c scrub starts 2024-09-10T19:09:05.111 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:04 smithi018 ceph-mon[29336]: 2.c scrub ok 2024-09-10T19:09:05.111 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:04 smithi018 ceph-mon[29336]: 4.17 scrub starts 2024-09-10T19:09:05.111 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:04 smithi018 ceph-mon[29336]: 4.17 scrub ok 2024-09-10T19:09:05.111 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:04 smithi018 ceph-mon[29336]: 1.e deep-scrub starts 2024-09-10T19:09:05.111 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:04 smithi018 ceph-mon[29336]: 1.e deep-scrub ok 2024-09-10T19:09:05.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:04 smithi103 ceph-mon[33163]: 2.c scrub starts 2024-09-10T19:09:05.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:04 smithi103 ceph-mon[33163]: 2.c scrub ok 2024-09-10T19:09:05.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:04 smithi103 ceph-mon[33163]: 4.17 scrub starts 2024-09-10T19:09:05.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:04 smithi103 ceph-mon[33163]: 4.17 scrub ok 2024-09-10T19:09:05.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:04 smithi103 ceph-mon[33163]: 1.e deep-scrub starts 2024-09-10T19:09:05.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:04 smithi103 ceph-mon[33163]: 1.e deep-scrub ok 2024-09-10T19:09:05.423 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":1123,"stamp":"2024-09-10T19:09:03.557887+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":2689,"num_read_kb":2441,"num_write":1002,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":358,"num_bytes_recovered":533960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2894,"ondisk_log_size":2894,"up":456,"acting":456,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":456,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":7825668,"kb_used_data":7556,"kb_used_omap":18,"kb_used_meta":7818029,"kb_avail":742135548,"statfs":{"total":767960285184,"available":759946801152,"internally_reserved":0,"allocated":7737344,"data_stored":4722692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19428,"internal_metadata":8005661724},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003510"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":538,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090040+0000","last_change":"2024-09-10T19:08:03.909476+0000","last_active":"2024-09-10T19:08:38.090040+0000","last_peered":"2024-09-10T19:08:38.090040+0000","last_clean":"2024-09-10T19:08:38.090040+0000","last_became_active":"2024-09-10T19:07:18.744367+0000","last_became_peered":"2024-09-10T19:07:18.744367+0000","last_unstale":"2024-09-10T19:08:38.090040+0000","last_undegraded":"2024-09-10T19:08:38.090040+0000","last_fullsized":"2024-09-10T19:08:38.090040+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:03.909412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:08:03.909412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:17.315981+0000","scrub_duration":0.0038138130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":536,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.623171+0000","last_change":"2024-09-10T19:08:36.825855+0000","last_active":"2024-09-10T19:08:37.623171+0000","last_peered":"2024-09-10T19:08:37.623171+0000","last_clean":"2024-09-10T19:08:37.623171+0000","last_became_active":"2024-09-10T19:07:18.728054+0000","last_became_peered":"2024-09-10T19:07:18.728054+0000","last_unstale":"2024-09-10T19:08:37.623171+0000","last_undegraded":"2024-09-10T19:08:37.623171+0000","last_fullsized":"2024-09-10T19:08:37.623171+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:36.825802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:07:19.562907+0000","last_clean_scrub_stamp":"2024-09-10T19:08:36.825802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:42.666484+0000","scrub_duration":0.0015749329999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"427'39","reported_seq":657,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090101+0000","last_change":"2024-09-10T19:08:24.946416+0000","last_active":"2024-09-10T19:08:38.090101+0000","last_peered":"2024-09-10T19:08:38.090101+0000","last_clean":"2024-09-10T19:08:38.090101+0000","last_became_active":"2024-09-10T19:07:21.856836+0000","last_became_peered":"2024-09-10T19:07:21.856836+0000","last_unstale":"2024-09-10T19:08:38.090101+0000","last_undegraded":"2024-09-10T19:08:38.090101+0000","last_fullsized":"2024-09-10T19:08:38.090101+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'39","last_scrub_stamp":"2024-09-10T19:08:24.946351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:08:24.946351+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:36.731315+0000","scrub_duration":0.0020189969999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":521,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.624648+0000","last_change":"2024-09-10T19:07:49.987352+0000","last_active":"2024-09-10T19:08:37.624648+0000","last_peered":"2024-09-10T19:08:37.624648+0000","last_clean":"2024-09-10T19:08:37.624648+0000","last_became_active":"2024-09-10T18:45:22.065619+0000","last_became_peered":"2024-09-10T18:45:22.065619+0000","last_unstale":"2024-09-10T19:08:37.624648+0000","last_undegraded":"2024-09-10T19:08:37.624648+0000","last_fullsized":"2024-09-10T19:08:37.624648+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:49.987284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.057932+0000","last_clean_scrub_stamp":"2024-09-10T19:07:49.987284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:01.961061+0000","scrub_duration":0.0015713820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":591,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.091257+0000","last_change":"2024-09-10T19:08:16.047139+0000","last_active":"2024-09-10T19:08:38.091257+0000","last_peered":"2024-09-10T19:08:38.091257+0000","last_clean":"2024-09-10T19:08:38.091257+0000","last_became_active":"2024-09-10T19:00:42.518824+0000","last_became_peered":"2024-09-10T19:00:42.518824+0000","last_unstale":"2024-09-10T19:08:38.091257+0000","last_undegraded":"2024-09-10T19:08:38.091257+0000","last_fullsized":"2024-09-10T19:08:38.091257+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:16.047073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:08:16.047073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:18.361252+0000","scrub_duration":0.0018722070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"427'25","reported_seq":561,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:58.981990+0000","last_change":"2024-09-10T19:08:58.981933+0000","last_active":"2024-09-10T19:08:58.981990+0000","last_peered":"2024-09-10T19:08:58.981990+0000","last_clean":"2024-09-10T19:08:58.981990+0000","last_became_active":"2024-09-10T18:45:20.061266+0000","last_became_peered":"2024-09-10T18:45:20.061266+0000","last_unstale":"2024-09-10T19:08:58.981990+0000","last_undegraded":"2024-09-10T19:08:58.981990+0000","last_fullsized":"2024-09-10T19:08:58.981990+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'25","last_scrub_stamp":"2024-09-10T19:08:58.981855+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_clean_scrub_stamp":"2024-09-10T19:08:58.981855+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:26.746618+0000","scrub_duration":0.0020552090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":177,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:46.605222+0000","last_change":"2024-09-10T19:08:46.605177+0000","last_active":"2024-09-10T19:08:46.605222+0000","last_peered":"2024-09-10T19:08:46.605222+0000","last_clean":"2024-09-10T19:08:46.605222+0000","last_became_active":"2024-09-10T19:07:18.746014+0000","last_became_peered":"2024-09-10T19:07:18.746014+0000","last_unstale":"2024-09-10T19:08:46.605222+0000","last_undegraded":"2024-09-10T19:08:46.605222+0000","last_fullsized":"2024-09-10T19:08:46.605222+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:46.605114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:01.135988+0000","last_clean_scrub_stamp":"2024-09-10T19:08:46.605114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:47.784144+0000","scrub_duration":0.0016228950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,7],"acting":[6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":519,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:51.979591+0000","last_change":"2024-09-10T19:08:51.979558+0000","last_active":"2024-09-10T19:08:51.979591+0000","last_peered":"2024-09-10T19:08:51.979591+0000","last_clean":"2024-09-10T19:08:51.979591+0000","last_became_active":"2024-09-10T18:45:24.081002+0000","last_became_peered":"2024-09-10T18:45:24.081002+0000","last_unstale":"2024-09-10T19:08:51.979591+0000","last_undegraded":"2024-09-10T19:08:51.979591+0000","last_fullsized":"2024-09-10T19:08:51.979591+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:51.979490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:07:29.035184+0000","last_clean_scrub_stamp":"2024-09-10T19:08:51.979490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:57.861037+0000","scrub_duration":0.001735161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":53,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.091015+0000","last_change":"2024-09-10T19:08:23.256410+0000","last_active":"2024-09-10T19:08:38.091015+0000","last_peered":"2024-09-10T19:08:38.091015+0000","last_clean":"2024-09-10T19:08:38.091015+0000","last_became_active":"2024-09-10T19:07:18.744458+0000","last_became_peered":"2024-09-10T19:07:18.744458+0000","last_unstale":"2024-09-10T19:08:38.091015+0000","last_undegraded":"2024-09-10T19:08:38.091015+0000","last_fullsized":"2024-09-10T19:08:38.091015+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:23.256344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:08:23.256344+0000","last_clean_scrub_stamp":"2024-09-10T19:08:23.256344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:34.853431+0000","scrub_duration":0.001568037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"427'11","reported_seq":541,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.625121+0000","last_change":"2024-09-10T19:08:36.962624+0000","last_active":"2024-09-10T19:08:37.625121+0000","last_peered":"2024-09-10T19:08:37.625121+0000","last_clean":"2024-09-10T19:08:37.625121+0000","last_became_active":"2024-09-10T18:45:20.057567+0000","last_became_peered":"2024-09-10T18:45:20.057567+0000","last_unstale":"2024-09-10T19:08:37.625121+0000","last_undegraded":"2024-09-10T19:08:37.625121+0000","last_fullsized":"2024-09-10T19:08:37.625121+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'11","last_scrub_stamp":"2024-09-10T19:08:36.962569+0000","last_deep_scrub":"163'7","last_deep_scrub_stamp":"2024-09-10T18:51:54.261879+0000","last_clean_scrub_stamp":"2024-09-10T19:08:36.962569+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:53.987704+0000","scrub_duration":0.0014505620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":13,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":53,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090500+0000","last_change":"2024-09-10T19:08:04.344276+0000","last_active":"2024-09-10T19:08:38.090500+0000","last_peered":"2024-09-10T19:08:38.090500+0000","last_clean":"2024-09-10T19:08:38.090500+0000","last_became_active":"2024-09-10T19:07:18.740022+0000","last_became_peered":"2024-09-10T19:07:18.740022+0000","last_unstale":"2024-09-10T19:08:38.090500+0000","last_undegraded":"2024-09-10T19:08:38.090500+0000","last_fullsized":"2024-09-10T19:08:38.090500+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:04.344202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:36.509971+0000","last_clean_scrub_stamp":"2024-09-10T19:08:04.344202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:28.179056+0000","scrub_duration":0.001698984,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":186,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.999696+0000","last_change":"2024-09-10T19:07:49.336488+0000","last_active":"2024-09-10T19:08:37.999696+0000","last_peered":"2024-09-10T19:08:37.999696+0000","last_clean":"2024-09-10T19:08:37.999696+0000","last_became_active":"2024-09-10T19:00:42.515840+0000","last_became_peered":"2024-09-10T19:00:42.515840+0000","last_unstale":"2024-09-10T19:08:37.999696+0000","last_undegraded":"2024-09-10T19:08:37.999696+0000","last_fullsized":"2024-09-10T19:08:37.999696+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:49.336430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:07:49.336430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:08:55.574589+0000","scrub_duration":0.0013951339999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":63,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090839+0000","last_change":"2024-09-10T19:08:07.350292+0000","last_active":"2024-09-10T19:08:38.090839+0000","last_peered":"2024-09-10T19:08:38.090839+0000","last_clean":"2024-09-10T19:08:38.090839+0000","last_became_active":"2024-09-10T19:07:21.861354+0000","last_became_peered":"2024-09-10T19:07:21.861354+0000","last_unstale":"2024-09-10T19:08:38.090839+0000","last_undegraded":"2024-09-10T19:08:38.090839+0000","last_fullsized":"2024-09-10T19:08:38.090839+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:07.350220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:08:07.350220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:20.162947+0000","scrub_duration":0.0015941410000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"427'21","reported_seq":586,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.625067+0000","last_change":"2024-09-10T19:08:26.918304+0000","last_active":"2024-09-10T19:08:37.625067+0000","last_peered":"2024-09-10T19:08:37.625067+0000","last_clean":"2024-09-10T19:08:37.625067+0000","last_became_active":"2024-09-10T18:55:40.391571+0000","last_became_peered":"2024-09-10T18:55:40.391571+0000","last_unstale":"2024-09-10T19:08:37.625067+0000","last_undegraded":"2024-09-10T19:08:37.625067+0000","last_fullsized":"2024-09-10T19:08:37.625067+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'21","last_scrub_stamp":"2024-09-10T19:08:26.918240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:08:26.918240+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:43.250535+0000","scrub_duration":0.0021185179999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":559,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.999847+0000","last_change":"2024-09-10T19:07:38.321617+0000","last_active":"2024-09-10T19:08:37.999847+0000","last_peered":"2024-09-10T19:08:37.999847+0000","last_clean":"2024-09-10T19:08:37.999847+0000","last_became_active":"2024-09-10T19:07:18.723010+0000","last_became_peered":"2024-09-10T19:07:18.723010+0000","last_unstale":"2024-09-10T19:08:37.999847+0000","last_undegraded":"2024-09-10T19:08:37.999847+0000","last_fullsized":"2024-09-10T19:08:37.999847+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:38.321553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:07:38.321553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:08:56.406814+0000","scrub_duration":0.0015874929999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":225,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.621742+0000","last_change":"2024-09-10T19:08:03.730646+0000","last_active":"2024-09-10T19:08:37.621742+0000","last_peered":"2024-09-10T19:08:37.621742+0000","last_clean":"2024-09-10T19:08:37.621742+0000","last_became_active":"2024-09-10T19:07:18.727841+0000","last_became_peered":"2024-09-10T19:07:18.727841+0000","last_unstale":"2024-09-10T19:08:37.621742+0000","last_undegraded":"2024-09-10T19:08:37.621742+0000","last_fullsized":"2024-09-10T19:08:37.621742+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:03.730586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:59.425718+0000","last_clean_scrub_stamp":"2024-09-10T19:08:03.730586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:13.323613+0000","scrub_duration":0.001653133,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.8","version":"70'81","reported_seq":204,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.999342+0000","last_change":"2024-09-10T19:07:44.202146+0000","last_active":"2024-09-10T19:08:37.999342+0000","last_peered":"2024-09-10T19:08:37.999342+0000","last_clean":"2024-09-10T19:08:37.999342+0000","last_became_active":"2024-09-10T19:07:21.862158+0000","last_became_peered":"2024-09-10T19:07:21.862158+0000","last_unstale":"2024-09-10T19:08:37.999342+0000","last_undegraded":"2024-09-10T19:08:37.999342+0000","last_fullsized":"2024-09-10T19:08:37.999342+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:07:44.202071+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:07:44.202071+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:08:53.927364+0000","scrub_duration":0.001732999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":525376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"427'53","reported_seq":667,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:09:00.029067+0000","last_change":"2024-09-10T19:09:00.029031+0000","last_active":"2024-09-10T19:09:00.029067+0000","last_peered":"2024-09-10T19:09:00.029067+0000","last_clean":"2024-09-10T19:09:00.029067+0000","last_became_active":"2024-09-10T19:07:21.862457+0000","last_became_peered":"2024-09-10T19:07:21.862457+0000","last_unstale":"2024-09-10T19:09:00.029067+0000","last_undegraded":"2024-09-10T19:09:00.029067+0000","last_fullsized":"2024-09-10T19:09:00.029067+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'53","last_scrub_stamp":"2024-09-10T19:09:00.028958+0000","last_deep_scrub":"78'25","last_deep_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_clean_scrub_stamp":"2024-09-10T19:09:00.028958+0000","objects_scrubbed":8,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:09.264364+0000","scrub_duration":0.0026767560000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":71,"num_read_kb":61,"num_write":46,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":53,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090378+0000","last_change":"2024-09-10T19:07:41.402347+0000","last_active":"2024-09-10T19:08:38.090378+0000","last_peered":"2024-09-10T19:08:38.090378+0000","last_clean":"2024-09-10T19:08:38.090378+0000","last_became_active":"2024-09-10T19:07:18.747252+0000","last_became_peered":"2024-09-10T19:07:18.747252+0000","last_unstale":"2024-09-10T19:08:38.090378+0000","last_undegraded":"2024-09-10T19:08:38.090378+0000","last_fullsized":"2024-09-10T19:08:38.090378+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:41.402224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:07:41.402224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:07.678852+0000","scrub_duration":0.0022755280000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":433,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.622962+0000","last_change":"2024-09-10T19:07:45.367587+0000","last_active":"2024-09-10T19:08:37.622962+0000","last_peered":"2024-09-10T19:08:37.622962+0000","last_clean":"2024-09-10T19:08:37.622962+0000","last_became_active":"2024-09-10T19:07:31.168843+0000","last_became_peered":"2024-09-10T19:07:31.168843+0000","last_unstale":"2024-09-10T19:08:37.622962+0000","last_undegraded":"2024-09-10T19:08:37.622962+0000","last_fullsized":"2024-09-10T19:08:37.622962+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:45.367535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:04.512425+0000","last_clean_scrub_stamp":"2024-09-10T19:07:45.367535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:05.379732+0000","scrub_duration":0.001540963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":59,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:47.245955+0000","last_change":"2024-09-10T19:08:47.245914+0000","last_active":"2024-09-10T19:08:47.245955+0000","last_peered":"2024-09-10T19:08:47.245955+0000","last_clean":"2024-09-10T19:08:47.245955+0000","last_became_active":"2024-09-10T19:07:21.856673+0000","last_became_peered":"2024-09-10T19:07:21.856673+0000","last_unstale":"2024-09-10T19:08:47.245955+0000","last_undegraded":"2024-09-10T19:08:47.245955+0000","last_fullsized":"2024-09-10T19:08:47.245955+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:47.245843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:08:47.245843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:05.486776+0000","scrub_duration":0.002629616,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,4],"acting":[2,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"70'81","reported_seq":63,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090420+0000","last_change":"2024-09-10T19:07:30.307185+0000","last_active":"2024-09-10T19:08:38.090420+0000","last_peered":"2024-09-10T19:08:38.090420+0000","last_clean":"2024-09-10T19:08:38.090420+0000","last_became_active":"2024-09-10T19:07:21.856173+0000","last_became_peered":"2024-09-10T19:07:21.856173+0000","last_unstale":"2024-09-10T19:08:38.090420+0000","last_undegraded":"2024-09-10T19:08:38.090420+0000","last_fullsized":"2024-09-10T19:08:38.090420+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:07:30.307121+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:07:30.307121+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:00.306654+0000","scrub_duration":0.0015656310000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"427'35","reported_seq":573,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:56.002380+0000","last_change":"2024-09-10T19:08:56.002347+0000","last_active":"2024-09-10T19:08:56.002380+0000","last_peered":"2024-09-10T19:08:56.002380+0000","last_clean":"2024-09-10T19:08:56.002380+0000","last_became_active":"2024-09-10T18:45:20.053428+0000","last_became_peered":"2024-09-10T18:45:20.053428+0000","last_unstale":"2024-09-10T19:08:56.002380+0000","last_undegraded":"2024-09-10T19:08:56.002380+0000","last_fullsized":"2024-09-10T19:08:56.002380+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'35","last_scrub_stamp":"2024-09-10T19:08:56.002267+0000","last_deep_scrub":"427'35","last_deep_scrub_stamp":"2024-09-10T19:08:56.002267+0000","last_clean_scrub_stamp":"2024-09-10T19:08:56.002267+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:21.723214+0000","scrub_duration":0.0021565209999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":443,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.623268+0000","last_change":"2024-09-10T19:07:54.316841+0000","last_active":"2024-09-10T19:08:37.623268+0000","last_peered":"2024-09-10T19:08:37.623268+0000","last_clean":"2024-09-10T19:08:37.623268+0000","last_became_active":"2024-09-10T19:07:31.278407+0000","last_became_peered":"2024-09-10T19:07:31.278407+0000","last_unstale":"2024-09-10T19:08:37.623268+0000","last_undegraded":"2024-09-10T19:08:37.623268+0000","last_fullsized":"2024-09-10T19:08:37.623268+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:54.316788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:07:54.316788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:19.073310+0000","scrub_duration":0.002303673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"465'15","reported_seq":74,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:51.353488+0000","last_change":"2024-09-10T19:07:51.118530+0000","last_active":"2024-09-10T19:08:51.353488+0000","last_peered":"2024-09-10T19:08:51.353488+0000","last_clean":"2024-09-10T19:08:51.353488+0000","last_became_active":"2024-09-10T19:07:21.861317+0000","last_became_peered":"2024-09-10T19:07:21.861317+0000","last_unstale":"2024-09-10T19:08:51.353488+0000","last_undegraded":"2024-09-10T19:08:51.353488+0000","last_fullsized":"2024-09-10T19:08:51.353488+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"465'15","last_scrub_stamp":"2024-09-10T19:07:51.118468+0000","last_deep_scrub":"273'13","last_deep_scrub_stamp":"2024-09-10T19:06:23.008037+0000","last_clean_scrub_stamp":"2024-09-10T19:07:51.118468+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:08.037522+0000","scrub_duration":0.001824387,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":569,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.625535+0000","last_change":"2024-09-10T19:08:04.107896+0000","last_active":"2024-09-10T19:08:37.625535+0000","last_peered":"2024-09-10T19:08:37.625535+0000","last_clean":"2024-09-10T19:08:37.625535+0000","last_became_active":"2024-09-10T19:07:21.861634+0000","last_became_peered":"2024-09-10T19:07:21.861634+0000","last_unstale":"2024-09-10T19:08:37.625535+0000","last_undegraded":"2024-09-10T19:08:37.625535+0000","last_fullsized":"2024-09-10T19:08:37.625535+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:04.107827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:32.043666+0000","last_clean_scrub_stamp":"2024-09-10T19:08:04.107827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:16.388292+0000","scrub_duration":0.0035664350000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"70'81","reported_seq":457,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.000788+0000","last_change":"2024-09-10T19:08:30.577616+0000","last_active":"2024-09-10T19:08:38.000788+0000","last_peered":"2024-09-10T19:08:38.000788+0000","last_clean":"2024-09-10T19:08:38.000788+0000","last_became_active":"2024-09-10T19:07:21.862300+0000","last_became_peered":"2024-09-10T19:07:21.862300+0000","last_unstale":"2024-09-10T19:08:38.000788+0000","last_undegraded":"2024-09-10T19:08:38.000788+0000","last_fullsized":"2024-09-10T19:08:38.000788+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:08:30.577541+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:55:37.921519+0000","last_clean_scrub_stamp":"2024-09-10T19:08:30.577541+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:37.579895+0000","scrub_duration":0.0020382659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"427'46","reported_seq":562,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.000773+0000","last_change":"2024-09-10T19:08:15.511332+0000","last_active":"2024-09-10T19:08:38.000773+0000","last_peered":"2024-09-10T19:08:38.000773+0000","last_clean":"2024-09-10T19:08:38.000773+0000","last_became_active":"2024-09-10T18:46:56.742946+0000","last_became_peered":"2024-09-10T18:46:56.742946+0000","last_unstale":"2024-09-10T19:08:38.000773+0000","last_undegraded":"2024-09-10T19:08:38.000773+0000","last_fullsized":"2024-09-10T19:08:38.000773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'46","last_scrub_stamp":"2024-09-10T19:08:15.511257+0000","last_deep_scrub":"141'22","last_deep_scrub_stamp":"2024-09-10T18:50:57.119152+0000","last_clean_scrub_stamp":"2024-09-10T19:08:15.511257+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:25.528407+0000","scrub_duration":0.0023435169999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":443,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:09:01.689776+0000","last_change":"2024-09-10T19:09:01.689746+0000","last_active":"2024-09-10T19:09:01.689776+0000","last_peered":"2024-09-10T19:09:01.689776+0000","last_clean":"2024-09-10T19:09:01.689776+0000","last_became_active":"2024-09-10T19:07:31.278450+0000","last_became_peered":"2024-09-10T19:07:31.278450+0000","last_unstale":"2024-09-10T19:09:01.689776+0000","last_undegraded":"2024-09-10T19:09:01.689776+0000","last_fullsized":"2024-09-10T19:09:01.689776+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:09:01.689684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_clean_scrub_stamp":"2024-09-10T19:09:01.689684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:24.062807+0000","scrub_duration":0.0016748430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"242'10","reported_seq":833,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:56.354835+0000","last_change":"2024-09-10T19:08:17.922625+0000","last_active":"2024-09-10T19:08:56.354835+0000","last_peered":"2024-09-10T19:08:56.354835+0000","last_clean":"2024-09-10T19:08:56.354835+0000","last_became_active":"2024-09-10T18:55:40.397246+0000","last_became_peered":"2024-09-10T18:55:40.397246+0000","last_unstale":"2024-09-10T19:08:56.354835+0000","last_undegraded":"2024-09-10T19:08:56.354835+0000","last_fullsized":"2024-09-10T19:08:56.354835+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"242'10","last_scrub_stamp":"2024-09-10T19:08:17.922559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:08:17.922559+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:22.000888+0000","scrub_duration":0.001537615,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":547,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:39.657991+0000","last_change":"2024-09-10T19:08:39.657943+0000","last_active":"2024-09-10T19:08:39.657991+0000","last_peered":"2024-09-10T19:08:39.657991+0000","last_clean":"2024-09-10T19:08:39.657991+0000","last_became_active":"2024-09-10T18:55:40.392003+0000","last_became_peered":"2024-09-10T18:55:40.392003+0000","last_unstale":"2024-09-10T19:08:39.657991+0000","last_undegraded":"2024-09-10T19:08:39.657991+0000","last_fullsized":"2024-09-10T19:08:39.657991+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:39.657872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:54.218602+0000","last_clean_scrub_stamp":"2024-09-10T19:08:39.657872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:43.299476+0000","scrub_duration":0.001575821,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"70'81","reported_seq":396,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.623253+0000","last_change":"2024-09-10T19:07:49.366327+0000","last_active":"2024-09-10T19:08:37.623253+0000","last_peered":"2024-09-10T19:08:37.623253+0000","last_clean":"2024-09-10T19:08:37.623253+0000","last_became_active":"2024-09-10T19:07:31.169897+0000","last_became_peered":"2024-09-10T19:07:31.169897+0000","last_unstale":"2024-09-10T19:08:37.623253+0000","last_undegraded":"2024-09-10T19:08:37.623253+0000","last_fullsized":"2024-09-10T19:08:37.623253+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:07:49.366271+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T19:02:17.370627+0000","last_clean_scrub_stamp":"2024-09-10T19:07:49.366271+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:17.353130+0000","scrub_duration":0.001464066,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"427'32","reported_seq":647,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:43.007918+0000","last_change":"2024-09-10T19:08:43.007856+0000","last_active":"2024-09-10T19:08:43.007918+0000","last_peered":"2024-09-10T19:08:43.007918+0000","last_clean":"2024-09-10T19:08:43.007918+0000","last_became_active":"2024-09-10T18:47:57.106935+0000","last_became_peered":"2024-09-10T18:47:57.106935+0000","last_unstale":"2024-09-10T19:08:43.007918+0000","last_undegraded":"2024-09-10T19:08:43.007918+0000","last_fullsized":"2024-09-10T19:08:43.007918+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'32","last_scrub_stamp":"2024-09-10T19:08:43.007778+0000","last_deep_scrub":"224'16","last_deep_scrub_stamp":"2024-09-10T18:54:49.296957+0000","last_clean_scrub_stamp":"2024-09-10T19:08:43.007778+0000","objects_scrubbed":5,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:47.840641+0000","scrub_duration":0.0020834780000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":53,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090039+0000","last_change":"2024-09-10T19:07:46.355729+0000","last_active":"2024-09-10T19:08:38.090039+0000","last_peered":"2024-09-10T19:08:38.090039+0000","last_clean":"2024-09-10T19:08:38.090039+0000","last_became_active":"2024-09-10T19:07:18.743428+0000","last_became_peered":"2024-09-10T19:07:18.743428+0000","last_unstale":"2024-09-10T19:08:38.090039+0000","last_undegraded":"2024-09-10T19:08:38.090039+0000","last_fullsized":"2024-09-10T19:08:38.090039+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:46.355646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:07:46.355646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:12.574680+0000","scrub_duration":0.0018423389999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":535,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.000659+0000","last_change":"2024-09-10T19:08:18.509197+0000","last_active":"2024-09-10T19:08:38.000659+0000","last_peered":"2024-09-10T19:08:38.000659+0000","last_clean":"2024-09-10T19:08:38.000659+0000","last_became_active":"2024-09-10T18:45:22.068918+0000","last_became_peered":"2024-09-10T18:45:22.068918+0000","last_unstale":"2024-09-10T19:08:38.000659+0000","last_undegraded":"2024-09-10T19:08:38.000659+0000","last_fullsized":"2024-09-10T19:08:38.000659+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:18.509132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_clean_scrub_stamp":"2024-09-10T19:08:18.509132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:28.163911+0000","scrub_duration":0.0014485699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":518,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:51.810435+0000","last_change":"2024-09-10T19:08:51.810402+0000","last_active":"2024-09-10T19:08:51.810435+0000","last_peered":"2024-09-10T19:08:51.810435+0000","last_clean":"2024-09-10T19:08:51.810435+0000","last_became_active":"2024-09-10T18:45:24.264538+0000","last_became_peered":"2024-09-10T18:45:24.264538+0000","last_unstale":"2024-09-10T19:08:51.810435+0000","last_undegraded":"2024-09-10T19:08:51.810435+0000","last_fullsized":"2024-09-10T19:08:51.810435+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:51.810332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:51.398981+0000","last_clean_scrub_stamp":"2024-09-10T19:08:51.810332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:13.049208+0000","scrub_duration":0.001888303,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"70'81","reported_seq":486,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090574+0000","last_change":"2024-09-10T19:08:36.957950+0000","last_active":"2024-09-10T19:08:38.090574+0000","last_peered":"2024-09-10T19:08:38.090574+0000","last_clean":"2024-09-10T19:08:38.090574+0000","last_became_active":"2024-09-10T19:07:18.749200+0000","last_became_peered":"2024-09-10T19:07:18.749200+0000","last_unstale":"2024-09-10T19:08:38.090574+0000","last_undegraded":"2024-09-10T19:08:38.090574+0000","last_fullsized":"2024-09-10T19:08:38.090574+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:08:36.957895+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T19:08:36.957895+0000","last_clean_scrub_stamp":"2024-09-10T19:08:36.957895+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:00.195143+0000","scrub_duration":0.0014594219999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"427'49","reported_seq":593,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.002851+0000","last_change":"2024-09-10T19:08:38.002762+0000","last_active":"2024-09-10T19:08:38.002851+0000","last_peered":"2024-09-10T19:08:38.002851+0000","last_clean":"2024-09-10T19:08:38.002851+0000","last_became_active":"2024-09-10T18:45:20.052243+0000","last_became_peered":"2024-09-10T18:45:20.052243+0000","last_unstale":"2024-09-10T19:08:38.002851+0000","last_undegraded":"2024-09-10T19:08:38.002851+0000","last_fullsized":"2024-09-10T19:08:38.002851+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'49","last_scrub_stamp":"2024-09-10T19:08:38.002692+0000","last_deep_scrub":"143'21","last_deep_scrub_stamp":"2024-09-10T18:51:02.540216+0000","last_clean_scrub_stamp":"2024-09-10T19:08:38.002692+0000","objects_scrubbed":7,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:50.793533+0000","scrub_duration":0.0052600499999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":63,"num_read_kb":56,"num_write":42,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":587,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.000905+0000","last_change":"2024-09-10T19:08:26.620048+0000","last_active":"2024-09-10T19:08:38.000905+0000","last_peered":"2024-09-10T19:08:38.000905+0000","last_clean":"2024-09-10T19:08:38.000905+0000","last_became_active":"2024-09-10T19:07:21.862634+0000","last_became_peered":"2024-09-10T19:07:21.862634+0000","last_unstale":"2024-09-10T19:08:38.000905+0000","last_undegraded":"2024-09-10T19:08:38.000905+0000","last_fullsized":"2024-09-10T19:08:38.000905+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:26.619972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:07:24.531890+0000","last_clean_scrub_stamp":"2024-09-10T19:08:26.619972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:44.577127+0000","scrub_duration":0.002152316,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":813,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:56.354522+0000","last_change":"2024-09-10T19:07:55.791569+0000","last_active":"2024-09-10T19:08:56.354522+0000","last_peered":"2024-09-10T19:08:56.354522+0000","last_clean":"2024-09-10T19:08:56.354522+0000","last_became_active":"2024-09-10T18:45:22.074540+0000","last_became_peered":"2024-09-10T18:45:22.074540+0000","last_unstale":"2024-09-10T19:08:56.354522+0000","last_undegraded":"2024-09-10T19:08:56.354522+0000","last_fullsized":"2024-09-10T19:08:56.354522+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T19:07:55.791512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:07:55.791512+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:12.140532+0000","scrub_duration":0.0020915980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":73,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:43.272153+0000","last_change":"2024-09-10T19:08:43.272062+0000","last_active":"2024-09-10T19:08:43.272153+0000","last_peered":"2024-09-10T19:08:43.272153+0000","last_clean":"2024-09-10T19:08:43.272153+0000","last_became_active":"2024-09-10T19:07:21.856089+0000","last_became_peered":"2024-09-10T19:07:21.856089+0000","last_unstale":"2024-09-10T19:08:43.272153+0000","last_undegraded":"2024-09-10T19:08:43.272153+0000","last_fullsized":"2024-09-10T19:08:43.272153+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:43.271994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:08:43.271994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:08.049841+0000","scrub_duration":0.0014747440000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"70'81","reported_seq":453,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:48.943532+0000","last_change":"2024-09-10T19:08:48.943483+0000","last_active":"2024-09-10T19:08:48.943532+0000","last_peered":"2024-09-10T19:08:48.943532+0000","last_clean":"2024-09-10T19:08:48.943532+0000","last_became_active":"2024-09-10T19:07:21.862339+0000","last_became_peered":"2024-09-10T19:07:21.862339+0000","last_unstale":"2024-09-10T19:08:48.943532+0000","last_undegraded":"2024-09-10T19:08:48.943532+0000","last_fullsized":"2024-09-10T19:08:48.943532+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:08:48.943416+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:56:31.481303+0000","last_clean_scrub_stamp":"2024-09-10T19:08:48.943416+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:16.574945+0000","scrub_duration":0.0018045769999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"494'50","reported_seq":742,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090005+0000","last_change":"2024-09-10T19:08:35.991723+0000","last_active":"2024-09-10T19:08:38.090005+0000","last_peered":"2024-09-10T19:08:38.090005+0000","last_clean":"2024-09-10T19:08:38.090005+0000","last_became_active":"2024-09-10T19:07:18.741434+0000","last_became_peered":"2024-09-10T19:07:18.741434+0000","last_unstale":"2024-09-10T19:08:38.090005+0000","last_undegraded":"2024-09-10T19:08:38.090005+0000","last_fullsized":"2024-09-10T19:08:38.090005+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"494'50","last_scrub_stamp":"2024-09-10T19:08:35.991665+0000","last_deep_scrub":"494'50","last_deep_scrub_stamp":"2024-09-10T19:08:35.991665+0000","last_clean_scrub_stamp":"2024-09-10T19:08:35.991665+0000","objects_scrubbed":8,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:01.728712+0000","scrub_duration":0.0056939010000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":70,"num_read_kb":58,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":423,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.623246+0000","last_change":"2024-09-10T19:07:53.350128+0000","last_active":"2024-09-10T19:08:37.623246+0000","last_peered":"2024-09-10T19:08:37.623246+0000","last_clean":"2024-09-10T19:08:37.623246+0000","last_became_active":"2024-09-10T19:07:31.172524+0000","last_became_peered":"2024-09-10T19:07:31.172524+0000","last_unstale":"2024-09-10T19:08:37.623246+0000","last_undegraded":"2024-09-10T19:08:37.623246+0000","last_fullsized":"2024-09-10T19:08:37.623246+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:53.350060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:46.142129+0000","last_clean_scrub_stamp":"2024-09-10T19:07:53.350060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:19.894132+0000","scrub_duration":0.001627835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":541,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.624654+0000","last_change":"2024-09-10T19:07:58.078461+0000","last_active":"2024-09-10T19:08:37.624654+0000","last_peered":"2024-09-10T19:08:37.624654+0000","last_clean":"2024-09-10T19:08:37.624654+0000","last_became_active":"2024-09-10T19:07:18.740494+0000","last_became_peered":"2024-09-10T19:07:18.740494+0000","last_unstale":"2024-09-10T19:08:37.624654+0000","last_undegraded":"2024-09-10T19:08:37.624654+0000","last_fullsized":"2024-09-10T19:08:37.624654+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:58.078390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:57.903311+0000","last_clean_scrub_stamp":"2024-09-10T19:07:58.078390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:20.683491+0000","scrub_duration":0.0036123769999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":529,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.625038+0000","last_change":"2024-09-10T19:07:28.016189+0000","last_active":"2024-09-10T19:08:37.625038+0000","last_peered":"2024-09-10T19:08:37.625038+0000","last_clean":"2024-09-10T19:08:37.625038+0000","last_became_active":"2024-09-10T19:07:21.861522+0000","last_became_peered":"2024-09-10T19:07:21.861522+0000","last_unstale":"2024-09-10T19:08:37.625038+0000","last_undegraded":"2024-09-10T19:08:37.625038+0000","last_fullsized":"2024-09-10T19:08:37.625038+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:28.016123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_clean_scrub_stamp":"2024-09-10T19:07:28.016123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:08:49.931914+0000","scrub_duration":0.0030237900000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"70'81","reported_seq":540,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.000842+0000","last_change":"2024-09-10T19:08:02.673654+0000","last_active":"2024-09-10T19:08:38.000842+0000","last_peered":"2024-09-10T19:08:38.000842+0000","last_clean":"2024-09-10T19:08:38.000842+0000","last_became_active":"2024-09-10T18:55:40.396828+0000","last_became_peered":"2024-09-10T18:55:40.396828+0000","last_unstale":"2024-09-10T19:08:38.000842+0000","last_undegraded":"2024-09-10T19:08:38.000842+0000","last_fullsized":"2024-09-10T19:08:38.000842+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:08:02.673587+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T19:06:25.532348+0000","last_clean_scrub_stamp":"2024-09-10T19:08:02.673587+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:25.686387+0000","scrub_duration":0.001604711,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"427'39","reported_seq":626,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.622528+0000","last_change":"2024-09-10T19:08:01.796313+0000","last_active":"2024-09-10T19:08:37.622528+0000","last_peered":"2024-09-10T19:08:37.622528+0000","last_clean":"2024-09-10T19:08:37.622528+0000","last_became_active":"2024-09-10T19:07:18.728478+0000","last_became_peered":"2024-09-10T19:07:18.728478+0000","last_unstale":"2024-09-10T19:08:37.622528+0000","last_undegraded":"2024-09-10T19:08:37.622528+0000","last_fullsized":"2024-09-10T19:08:37.622528+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'39","last_scrub_stamp":"2024-09-10T19:08:01.796254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:08:01.796254+0000","objects_scrubbed":8,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:27.185638+0000","scrub_duration":0.0020776369999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":68,"num_read_kb":54,"num_write":42,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":32,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.882263+0000","last_change":"2024-09-10T19:08:00.795185+0000","last_active":"2024-09-10T19:08:37.882263+0000","last_peered":"2024-09-10T19:08:37.882263+0000","last_clean":"2024-09-10T19:08:37.882263+0000","last_became_active":"2024-09-10T19:07:58.849381+0000","last_became_peered":"2024-09-10T19:07:58.849381+0000","last_unstale":"2024-09-10T19:08:37.882263+0000","last_undegraded":"2024-09-10T19:08:37.882263+0000","last_fullsized":"2024-09-10T19:08:37.882263+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:00.795107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:06:29.659530+0000","last_clean_scrub_stamp":"2024-09-10T19:08:00.795107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:23.747687+0000","scrub_duration":0.0016343250000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,4],"acting":[2,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":563,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:55.514538+0000","last_change":"2024-09-10T19:08:55.514506+0000","last_active":"2024-09-10T19:08:55.514538+0000","last_peered":"2024-09-10T19:08:55.514538+0000","last_clean":"2024-09-10T19:08:55.514538+0000","last_became_active":"2024-09-10T18:55:40.387857+0000","last_became_peered":"2024-09-10T18:55:40.387857+0000","last_unstale":"2024-09-10T19:08:55.514538+0000","last_undegraded":"2024-09-10T19:08:55.514538+0000","last_fullsized":"2024-09-10T19:08:55.514538+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:55.514438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:08:55.514438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:03.224089+0000","scrub_duration":0.0015132629999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":453,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.622438+0000","last_change":"2024-09-10T19:07:46.331848+0000","last_active":"2024-09-10T19:08:37.622438+0000","last_peered":"2024-09-10T19:08:37.622438+0000","last_clean":"2024-09-10T19:08:37.622438+0000","last_became_active":"2024-09-10T19:07:31.171664+0000","last_became_peered":"2024-09-10T19:07:31.171664+0000","last_unstale":"2024-09-10T19:08:37.622438+0000","last_undegraded":"2024-09-10T19:08:37.622438+0000","last_fullsized":"2024-09-10T19:08:37.622438+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:46.331784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:49:18.281230+0000","last_clean_scrub_stamp":"2024-09-10T19:07:46.331784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:08:57.504226+0000","scrub_duration":0.001909979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"70'81","reported_seq":553,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.001257+0000","last_change":"2024-09-10T19:08:18.704584+0000","last_active":"2024-09-10T19:08:38.001257+0000","last_peered":"2024-09-10T19:08:38.001257+0000","last_clean":"2024-09-10T19:08:38.001257+0000","last_became_active":"2024-09-10T19:07:18.748748+0000","last_became_peered":"2024-09-10T19:07:18.748748+0000","last_unstale":"2024-09-10T19:08:38.001257+0000","last_undegraded":"2024-09-10T19:08:38.001257+0000","last_fullsized":"2024-09-10T19:08:38.001257+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:08:18.704518+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:08:18.704518+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:33.053071+0000","scrub_duration":0.00168654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"427'35","reported_seq":575,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.623131+0000","last_change":"2024-09-10T19:07:39.339297+0000","last_active":"2024-09-10T19:08:37.623131+0000","last_peered":"2024-09-10T19:08:37.623131+0000","last_clean":"2024-09-10T19:08:37.623131+0000","last_became_active":"2024-09-10T19:07:31.168654+0000","last_became_peered":"2024-09-10T19:07:31.168654+0000","last_unstale":"2024-09-10T19:08:37.623131+0000","last_undegraded":"2024-09-10T19:08:37.623131+0000","last_fullsized":"2024-09-10T19:08:37.623131+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'35","last_scrub_stamp":"2024-09-10T19:07:39.339228+0000","last_deep_scrub":"83'15","last_deep_scrub_stamp":"2024-09-10T18:46:58.436473+0000","last_clean_scrub_stamp":"2024-09-10T19:07:39.339228+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:08:52.095148+0000","scrub_duration":0.002305788,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":39,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":480,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:54.664120+0000","last_change":"2024-09-10T19:08:54.664091+0000","last_active":"2024-09-10T19:08:54.664120+0000","last_peered":"2024-09-10T19:08:54.664120+0000","last_clean":"2024-09-10T19:08:54.664120+0000","last_became_active":"2024-09-10T19:07:31.279211+0000","last_became_peered":"2024-09-10T19:07:31.279211+0000","last_unstale":"2024-09-10T19:08:54.664120+0000","last_undegraded":"2024-09-10T19:08:54.664120+0000","last_fullsized":"2024-09-10T19:08:54.664120+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:54.664029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:08:54.664029+0000","last_clean_scrub_stamp":"2024-09-10T19:08:54.664029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:12.841783+0000","scrub_duration":0.0015902010000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"339'13","reported_seq":785,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:51.353559+0000","last_change":"2024-09-10T19:08:43.700196+0000","last_active":"2024-09-10T19:08:51.353559+0000","last_peered":"2024-09-10T19:08:51.353559+0000","last_clean":"2024-09-10T19:08:51.353559+0000","last_became_active":"2024-09-10T19:00:42.520360+0000","last_became_peered":"2024-09-10T19:00:42.520360+0000","last_unstale":"2024-09-10T19:08:51.353559+0000","last_undegraded":"2024-09-10T19:08:51.353559+0000","last_fullsized":"2024-09-10T19:08:51.353559+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"339'13","last_scrub_stamp":"2024-09-10T19:08:43.700133+0000","last_deep_scrub":"258'12","last_deep_scrub_stamp":"2024-09-10T18:56:21.098770+0000","last_clean_scrub_stamp":"2024-09-10T19:08:43.700133+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:05.632734+0000","scrub_duration":0.0020569899999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":517,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.621959+0000","last_change":"2024-09-10T19:07:39.706206+0000","last_active":"2024-09-10T19:08:37.621959+0000","last_peered":"2024-09-10T19:08:37.621959+0000","last_clean":"2024-09-10T19:08:37.621959+0000","last_became_active":"2024-09-10T19:07:21.853717+0000","last_became_peered":"2024-09-10T19:07:21.853717+0000","last_unstale":"2024-09-10T19:08:37.621959+0000","last_undegraded":"2024-09-10T19:08:37.621959+0000","last_fullsized":"2024-09-10T19:08:37.621959+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:39.706147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:07:39.706147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:09.495498+0000","scrub_duration":0.0013607339999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"70'81","reported_seq":474,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.999461+0000","last_change":"2024-09-10T19:08:08.484018+0000","last_active":"2024-09-10T19:08:37.999461+0000","last_peered":"2024-09-10T19:08:37.999461+0000","last_clean":"2024-09-10T19:08:37.999461+0000","last_became_active":"2024-09-10T19:07:21.864419+0000","last_became_peered":"2024-09-10T19:07:21.864419+0000","last_unstale":"2024-09-10T19:08:37.999461+0000","last_undegraded":"2024-09-10T19:08:37.999461+0000","last_fullsized":"2024-09-10T19:08:37.999461+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:08:08.483943+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:08:08.483943+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:38.260370+0000","scrub_duration":0.002236937,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"470'56","reported_seq":621,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:56.354504+0000","last_change":"2024-09-10T19:08:42.551398+0000","last_active":"2024-09-10T19:08:56.354504+0000","last_peered":"2024-09-10T19:08:56.354504+0000","last_clean":"2024-09-10T19:08:56.354504+0000","last_became_active":"2024-09-10T19:07:31.174284+0000","last_became_peered":"2024-09-10T19:07:31.174284+0000","last_unstale":"2024-09-10T19:08:56.354504+0000","last_undegraded":"2024-09-10T19:08:56.354504+0000","last_fullsized":"2024-09-10T19:08:56.354504+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"470'56","last_scrub_stamp":"2024-09-10T19:08:42.551336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:08:42.551336+0000","objects_scrubbed":10,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:54.478510+0000","scrub_duration":0.0021253499999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":82,"num_read_kb":67,"num_write":50,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":539,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.999633+0000","last_change":"2024-09-10T19:08:24.542798+0000","last_active":"2024-09-10T19:08:37.999633+0000","last_peered":"2024-09-10T19:08:37.999633+0000","last_clean":"2024-09-10T19:08:37.999633+0000","last_became_active":"2024-09-10T18:45:17.964494+0000","last_became_peered":"2024-09-10T18:45:17.964494+0000","last_unstale":"2024-09-10T19:08:37.999633+0000","last_undegraded":"2024-09-10T19:08:37.999633+0000","last_fullsized":"2024-09-10T19:08:37.999633+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:24.542735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:12.982599+0000","last_clean_scrub_stamp":"2024-09-10T19:08:24.542735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:35.432204+0000","scrub_duration":0.0016080980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":491,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.000297+0000","last_change":"2024-09-10T19:08:36.566225+0000","last_active":"2024-09-10T19:08:38.000297+0000","last_peered":"2024-09-10T19:08:38.000297+0000","last_clean":"2024-09-10T19:08:38.000297+0000","last_became_active":"2024-09-10T18:47:57.107478+0000","last_became_peered":"2024-09-10T18:47:57.107478+0000","last_unstale":"2024-09-10T19:08:38.000297+0000","last_undegraded":"2024-09-10T19:08:38.000297+0000","last_fullsized":"2024-09-10T19:08:38.000297+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:36.566169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:08:36.566169+0000","last_clean_scrub_stamp":"2024-09-10T19:08:36.566169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:41.902876+0000","scrub_duration":0.001212032,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":528,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.091196+0000","last_change":"2024-09-10T19:08:29.013154+0000","last_active":"2024-09-10T19:08:38.091196+0000","last_peered":"2024-09-10T19:08:38.091196+0000","last_clean":"2024-09-10T19:08:38.091196+0000","last_became_active":"2024-09-10T19:07:18.742594+0000","last_became_peered":"2024-09-10T19:07:18.742594+0000","last_unstale":"2024-09-10T19:08:38.091196+0000","last_undegraded":"2024-09-10T19:08:38.091196+0000","last_fullsized":"2024-09-10T19:08:38.091196+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:29.013095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:08:29.013095+0000","last_clean_scrub_stamp":"2024-09-10T19:08:29.013095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:49.531497+0000","scrub_duration":0.001858181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"70'81","reported_seq":53,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090465+0000","last_change":"2024-09-10T19:08:06.395361+0000","last_active":"2024-09-10T19:08:38.090465+0000","last_peered":"2024-09-10T19:08:38.090465+0000","last_clean":"2024-09-10T19:08:38.090465+0000","last_became_active":"2024-09-10T19:07:18.741282+0000","last_became_peered":"2024-09-10T19:07:18.741282+0000","last_unstale":"2024-09-10T19:08:38.090465+0000","last_undegraded":"2024-09-10T19:08:38.090465+0000","last_fullsized":"2024-09-10T19:08:38.090465+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:08:06.395226+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:08:06.395226+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:35.100800+0000","scrub_duration":0.0022028500000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"427'26","reported_seq":395,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:39.587966+0000","last_change":"2024-09-10T19:08:39.587912+0000","last_active":"2024-09-10T19:08:39.587966+0000","last_peered":"2024-09-10T19:08:39.587966+0000","last_clean":"2024-09-10T19:08:39.587966+0000","last_became_active":"2024-09-10T19:07:31.168702+0000","last_became_peered":"2024-09-10T19:07:31.168702+0000","last_unstale":"2024-09-10T19:08:39.587966+0000","last_undegraded":"2024-09-10T19:08:39.587966+0000","last_fullsized":"2024-09-10T19:08:39.587966+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'26","last_scrub_stamp":"2024-09-10T19:08:39.587845+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-10T18:47:10.514951+0000","last_clean_scrub_stamp":"2024-09-10T19:08:39.587845+0000","objects_scrubbed":5,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:59.290992+0000","scrub_duration":0.0024897130000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":40,"num_read_kb":29,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"84'6","reported_seq":1588,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:09:00.327295+0000","last_change":"2024-09-10T19:08:28.432639+0000","last_active":"2024-09-10T19:09:00.327295+0000","last_peered":"2024-09-10T19:09:00.327295+0000","last_clean":"2024-09-10T19:09:00.327295+0000","last_became_active":"2024-09-10T19:07:31.170254+0000","last_became_peered":"2024-09-10T19:07:31.170254+0000","last_unstale":"2024-09-10T19:09:00.327295+0000","last_undegraded":"2024-09-10T19:09:00.327295+0000","last_fullsized":"2024-09-10T19:09:00.327295+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'6","last_scrub_stamp":"2024-09-10T19:08:28.432573+0000","last_deep_scrub":"84'6","last_deep_scrub_stamp":"2024-09-10T18:47:22.545450+0000","last_clean_scrub_stamp":"2024-09-10T19:08:28.432573+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:35.555013+0000","scrub_duration":0.00157843,"objects_trimmed":0,"snaptrim_duration":0.0021933159999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1121,"num_read_kb":1116,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":59,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.881996+0000","last_change":"2024-09-10T19:08:37.373940+0000","last_active":"2024-09-10T19:08:37.881996+0000","last_peered":"2024-09-10T19:08:37.881996+0000","last_clean":"2024-09-10T19:08:37.881996+0000","last_became_active":"2024-09-10T19:07:21.859912+0000","last_became_peered":"2024-09-10T19:07:21.859912+0000","last_unstale":"2024-09-10T19:08:37.881996+0000","last_undegraded":"2024-09-10T19:08:37.881996+0000","last_fullsized":"2024-09-10T19:08:37.881996+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:37.373879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:08:37.373879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:07.265854+0000","scrub_duration":0.0014641960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":59,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:40.292695+0000","last_change":"2024-09-10T19:08:40.292640+0000","last_active":"2024-09-10T19:08:40.292695+0000","last_peered":"2024-09-10T19:08:40.292695+0000","last_clean":"2024-09-10T19:08:40.292695+0000","last_became_active":"2024-09-10T19:07:21.861256+0000","last_became_peered":"2024-09-10T19:07:21.861256+0000","last_unstale":"2024-09-10T19:08:40.292695+0000","last_undegraded":"2024-09-10T19:08:40.292695+0000","last_fullsized":"2024-09-10T19:08:40.292695+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:40.292584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:07:33.132422+0000","last_clean_scrub_stamp":"2024-09-10T19:08:40.292584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:08.890319+0000","scrub_duration":0.001399511,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":63,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090906+0000","last_change":"2024-09-10T19:08:26.248390+0000","last_active":"2024-09-10T19:08:38.090906+0000","last_peered":"2024-09-10T19:08:38.090906+0000","last_clean":"2024-09-10T19:08:38.090906+0000","last_became_active":"2024-09-10T19:07:18.749761+0000","last_became_peered":"2024-09-10T19:07:18.749761+0000","last_unstale":"2024-09-10T19:08:38.090906+0000","last_undegraded":"2024-09-10T19:08:38.090906+0000","last_fullsized":"2024-09-10T19:08:38.090906+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:26.248323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:07.150480+0000","last_clean_scrub_stamp":"2024-09-10T19:08:26.248323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:45.832107+0000","scrub_duration":0.0016957179999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"70'81","reported_seq":337,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.623555+0000","last_change":"2024-09-10T19:07:31.170822+0000","last_active":"2024-09-10T19:08:37.623555+0000","last_peered":"2024-09-10T19:08:37.623555+0000","last_clean":"2024-09-10T19:08:37.623555+0000","last_became_active":"2024-09-10T19:07:31.170346+0000","last_became_peered":"2024-09-10T19:07:31.170346+0000","last_unstale":"2024-09-10T19:08:37.623555+0000","last_undegraded":"2024-09-10T19:08:37.623555+0000","last_fullsized":"2024-09-10T19:08:37.623555+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:07:29.520653+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:07:29.520653+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:08:56.625275+0000","scrub_duration":0.0023558350000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"427'36","reported_seq":329,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:40.796464+0000","last_change":"2024-09-10T19:08:40.796386+0000","last_active":"2024-09-10T19:08:40.796464+0000","last_peered":"2024-09-10T19:08:40.796464+0000","last_clean":"2024-09-10T19:08:40.796464+0000","last_became_active":"2024-09-10T19:07:21.854534+0000","last_became_peered":"2024-09-10T19:07:21.854534+0000","last_unstale":"2024-09-10T19:08:40.796464+0000","last_undegraded":"2024-09-10T19:08:40.796464+0000","last_fullsized":"2024-09-10T19:08:40.796464+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'36","last_scrub_stamp":"2024-09-10T19:08:40.796321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:08:40.796321+0000","objects_scrubbed":6,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:05.606848+0000","scrub_duration":0.0019102520000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":47,"num_write":36,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"305'2","reported_seq":510,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.622125+0000","last_change":"2024-09-10T19:08:17.796015+0000","last_active":"2024-09-10T19:08:37.622125+0000","last_peered":"2024-09-10T19:08:37.622125+0000","last_clean":"2024-09-10T19:08:37.622125+0000","last_became_active":"2024-09-10T18:55:40.390022+0000","last_became_peered":"2024-09-10T18:55:40.390022+0000","last_unstale":"2024-09-10T19:08:37.622125+0000","last_undegraded":"2024-09-10T19:08:37.622125+0000","last_fullsized":"2024-09-10T19:08:37.622125+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"305'2","last_scrub_stamp":"2024-09-10T19:08:17.795948+0000","last_deep_scrub":"305'2","last_deep_scrub_stamp":"2024-09-10T18:59:18.897059+0000","last_clean_scrub_stamp":"2024-09-10T19:08:17.795948+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:42.727821+0000","scrub_duration":0.0016584340000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":53,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090195+0000","last_change":"2024-09-10T19:08:03.363244+0000","last_active":"2024-09-10T19:08:38.090195+0000","last_peered":"2024-09-10T19:08:38.090195+0000","last_clean":"2024-09-10T19:08:38.090195+0000","last_became_active":"2024-09-10T19:07:18.740617+0000","last_became_peered":"2024-09-10T19:07:18.740617+0000","last_unstale":"2024-09-10T19:08:38.090195+0000","last_undegraded":"2024-09-10T19:08:38.090195+0000","last_fullsized":"2024-09-10T19:08:38.090195+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:03.363145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:08:03.363145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:06.600546+0000","scrub_duration":0.002509477,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":186,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.000866+0000","last_change":"2024-09-10T19:08:11.517617+0000","last_active":"2024-09-10T19:08:38.000866+0000","last_peered":"2024-09-10T19:08:38.000866+0000","last_clean":"2024-09-10T19:08:38.000866+0000","last_became_active":"2024-09-10T19:00:42.518566+0000","last_became_peered":"2024-09-10T19:00:42.518566+0000","last_unstale":"2024-09-10T19:08:38.000866+0000","last_undegraded":"2024-09-10T19:08:38.000866+0000","last_fullsized":"2024-09-10T19:08:38.000866+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:11.517549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:08:11.517549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:34.632717+0000","scrub_duration":0.0017203699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":430,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:43.583648+0000","last_change":"2024-09-10T19:08:43.583611+0000","last_active":"2024-09-10T19:08:43.583648+0000","last_peered":"2024-09-10T19:08:43.583648+0000","last_clean":"2024-09-10T19:08:43.583648+0000","last_became_active":"2024-09-10T19:07:31.173496+0000","last_became_peered":"2024-09-10T19:07:31.173496+0000","last_unstale":"2024-09-10T19:08:43.583648+0000","last_undegraded":"2024-09-10T19:08:43.583648+0000","last_fullsized":"2024-09-10T19:08:43.583648+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:43.583556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:08:43.583556+0000","last_clean_scrub_stamp":"2024-09-10T19:08:43.583556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:10.598278+0000","scrub_duration":0.00127001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"427'45","reported_seq":665,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.000779+0000","last_change":"2024-09-10T19:07:54.737766+0000","last_active":"2024-09-10T19:08:38.000779+0000","last_peered":"2024-09-10T19:08:38.000779+0000","last_clean":"2024-09-10T19:08:38.000779+0000","last_became_active":"2024-09-10T19:07:18.747716+0000","last_became_peered":"2024-09-10T19:07:18.747716+0000","last_unstale":"2024-09-10T19:08:38.000779+0000","last_undegraded":"2024-09-10T19:08:38.000779+0000","last_fullsized":"2024-09-10T19:08:38.000779+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'45","last_scrub_stamp":"2024-09-10T19:07:54.737699+0000","last_deep_scrub":"232'33","last_deep_scrub_stamp":"2024-09-10T18:56:21.206960+0000","last_clean_scrub_stamp":"2024-09-10T19:07:54.737699+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:08:57.477407+0000","scrub_duration":0.0031575549999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":61,"num_read_kb":51,"num_write":37,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":308,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"60'2","reported_seq":537,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:54.845767+0000","last_change":"2024-09-10T19:08:54.845742+0000","last_active":"2024-09-10T19:08:54.845767+0000","last_peered":"2024-09-10T19:08:54.845767+0000","last_clean":"2024-09-10T19:08:54.845767+0000","last_became_active":"2024-09-10T18:45:17.961660+0000","last_became_peered":"2024-09-10T18:45:17.961660+0000","last_unstale":"2024-09-10T19:08:54.845767+0000","last_undegraded":"2024-09-10T19:08:54.845767+0000","last_fullsized":"2024-09-10T19:08:54.845767+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-10T19:08:54.845684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:08:54.845684+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:16.073437+0000","scrub_duration":0.0019642700000000002,"objects_trimmed":0,"snaptrim_duration":0.0013942640000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":576,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.000284+0000","last_change":"2024-09-10T19:08:12.478323+0000","last_active":"2024-09-10T19:08:38.000284+0000","last_peered":"2024-09-10T19:08:38.000284+0000","last_clean":"2024-09-10T19:08:38.000284+0000","last_became_active":"2024-09-10T19:07:21.863455+0000","last_became_peered":"2024-09-10T19:07:21.863455+0000","last_unstale":"2024-09-10T19:08:38.000284+0000","last_undegraded":"2024-09-10T19:08:38.000284+0000","last_fullsized":"2024-09-10T19:08:38.000284+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:12.478251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:08:12.478251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:29.916361+0000","scrub_duration":0.001413135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'81","reported_seq":53,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090334+0000","last_change":"2024-09-10T19:08:16.350878+0000","last_active":"2024-09-10T19:08:38.090334+0000","last_peered":"2024-09-10T19:08:38.090334+0000","last_clean":"2024-09-10T19:08:38.090334+0000","last_became_active":"2024-09-10T19:07:18.746996+0000","last_became_peered":"2024-09-10T19:07:18.746996+0000","last_unstale":"2024-09-10T19:08:38.090334+0000","last_undegraded":"2024-09-10T19:08:38.090334+0000","last_fullsized":"2024-09-10T19:08:38.090334+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:08:16.350759+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:56.085580+0000","last_clean_scrub_stamp":"2024-09-10T19:08:16.350759+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:17.938788+0000","scrub_duration":0.001500136,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":63,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.091014+0000","last_change":"2024-09-10T19:07:36.341204+0000","last_active":"2024-09-10T19:08:38.091014+0000","last_peered":"2024-09-10T19:08:38.091014+0000","last_clean":"2024-09-10T19:08:38.091014+0000","last_became_active":"2024-09-10T19:07:21.860275+0000","last_became_peered":"2024-09-10T19:07:21.860275+0000","last_unstale":"2024-09-10T19:08:38.091014+0000","last_undegraded":"2024-09-10T19:08:38.091014+0000","last_fullsized":"2024-09-10T19:08:38.091014+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:36.341134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:07:36.341134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:08:55.991437+0000","scrub_duration":0.001199571,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":530,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.000483+0000","last_change":"2024-09-10T19:08:05.504409+0000","last_active":"2024-09-10T19:08:38.000483+0000","last_peered":"2024-09-10T19:08:38.000483+0000","last_clean":"2024-09-10T19:08:38.000483+0000","last_became_active":"2024-09-10T18:45:24.073327+0000","last_became_peered":"2024-09-10T18:45:24.073327+0000","last_unstale":"2024-09-10T19:08:38.000483+0000","last_undegraded":"2024-09-10T19:08:38.000483+0000","last_fullsized":"2024-09-10T19:08:38.000483+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:05.504340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:08:05.504340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:08.259505+0000","scrub_duration":0.001813633,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.f","version":"70'81","reported_seq":535,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.001055+0000","last_change":"2024-09-10T19:07:58.802241+0000","last_active":"2024-09-10T19:08:38.001055+0000","last_peered":"2024-09-10T19:08:38.001055+0000","last_clean":"2024-09-10T19:08:38.001055+0000","last_became_active":"2024-09-10T18:52:16.598295+0000","last_became_peered":"2024-09-10T18:52:16.598295+0000","last_unstale":"2024-09-10T19:08:38.001055+0000","last_undegraded":"2024-09-10T19:08:38.001055+0000","last_fullsized":"2024-09-10T19:08:38.001055+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:07:58.802189+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T19:07:58.802189+0000","last_clean_scrub_stamp":"2024-09-10T19:07:58.802189+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:14.862555+0000","scrub_duration":0.001211047,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"427'51","reported_seq":687,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:47.604189+0000","last_change":"2024-09-10T19:08:47.604156+0000","last_active":"2024-09-10T19:08:47.604189+0000","last_peered":"2024-09-10T19:08:47.604189+0000","last_clean":"2024-09-10T19:08:47.604189+0000","last_became_active":"2024-09-10T19:07:18.746562+0000","last_became_peered":"2024-09-10T19:07:18.746562+0000","last_unstale":"2024-09-10T19:08:47.604189+0000","last_undegraded":"2024-09-10T19:08:47.604189+0000","last_fullsized":"2024-09-10T19:08:47.604189+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'51","last_scrub_stamp":"2024-09-10T19:08:47.604085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:08:47.604085+0000","objects_scrubbed":9,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:02.828141+0000","scrub_duration":0.002354989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":80,"num_read_kb":69,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":49,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.882284+0000","last_change":"2024-09-10T19:07:36.140535+0000","last_active":"2024-09-10T19:08:37.882284+0000","last_peered":"2024-09-10T19:08:37.882284+0000","last_clean":"2024-09-10T19:08:37.882284+0000","last_became_active":"2024-09-10T19:07:21.869555+0000","last_became_peered":"2024-09-10T19:07:21.869555+0000","last_unstale":"2024-09-10T19:08:37.882284+0000","last_undegraded":"2024-09-10T19:08:37.882284+0000","last_fullsized":"2024-09-10T19:08:37.882284+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:36.140480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:07:36.140480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:01.966689+0000","scrub_duration":0.001305016,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":59,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:50.268609+0000","last_change":"2024-09-10T19:08:50.268564+0000","last_active":"2024-09-10T19:08:50.268609+0000","last_peered":"2024-09-10T19:08:50.268609+0000","last_clean":"2024-09-10T19:08:50.268609+0000","last_became_active":"2024-09-10T19:07:21.868368+0000","last_became_peered":"2024-09-10T19:07:21.868368+0000","last_unstale":"2024-09-10T19:08:50.268609+0000","last_undegraded":"2024-09-10T19:08:50.268609+0000","last_fullsized":"2024-09-10T19:08:50.268609+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:50.268480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:08:50.268480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:56.971191+0000","scrub_duration":0.001670185,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":73,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:46.385437+0000","last_change":"2024-09-10T19:08:46.385385+0000","last_active":"2024-09-10T19:08:46.385437+0000","last_peered":"2024-09-10T19:08:46.385437+0000","last_clean":"2024-09-10T19:08:46.385437+0000","last_became_active":"2024-09-10T19:07:21.857233+0000","last_became_peered":"2024-09-10T19:07:21.857233+0000","last_unstale":"2024-09-10T19:08:46.385437+0000","last_undegraded":"2024-09-10T19:08:46.385437+0000","last_fullsized":"2024-09-10T19:08:46.385437+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:46.385312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:08:46.385312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:14.903106+0000","scrub_duration":0.00178462,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.e","version":"70'81","reported_seq":404,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.625422+0000","last_change":"2024-09-10T19:07:37.093614+0000","last_active":"2024-09-10T19:08:37.625422+0000","last_peered":"2024-09-10T19:08:37.625422+0000","last_clean":"2024-09-10T19:08:37.625422+0000","last_became_active":"2024-09-10T19:07:18.733276+0000","last_became_peered":"2024-09-10T19:07:18.733276+0000","last_unstale":"2024-09-10T19:08:37.625422+0000","last_undegraded":"2024-09-10T19:08:37.625422+0000","last_fullsized":"2024-09-10T19:08:37.625422+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:07:37.093545+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:07:37.093545+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:08:52.743131+0000","scrub_duration":0.0014376840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"427'25","reported_seq":551,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:58.759028+0000","last_change":"2024-09-10T19:08:58.759000+0000","last_active":"2024-09-10T19:08:58.759028+0000","last_peered":"2024-09-10T19:08:58.759028+0000","last_clean":"2024-09-10T19:08:58.759028+0000","last_became_active":"2024-09-10T18:55:40.395257+0000","last_became_peered":"2024-09-10T18:55:40.395257+0000","last_unstale":"2024-09-10T19:08:58.759028+0000","last_undegraded":"2024-09-10T19:08:58.759028+0000","last_fullsized":"2024-09-10T19:08:58.759028+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'25","last_scrub_stamp":"2024-09-10T19:08:58.758929+0000","last_deep_scrub":"130'13","last_deep_scrub_stamp":"2024-09-10T18:51:58.427534+0000","last_clean_scrub_stamp":"2024-09-10T19:08:58.758929+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:15.839540+0000","scrub_duration":0.002359355,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":430,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:48.488720+0000","last_change":"2024-09-10T19:08:48.488661+0000","last_active":"2024-09-10T19:08:48.488720+0000","last_peered":"2024-09-10T19:08:48.488720+0000","last_clean":"2024-09-10T19:08:48.488720+0000","last_became_active":"2024-09-10T19:07:31.174814+0000","last_became_peered":"2024-09-10T19:07:31.174814+0000","last_unstale":"2024-09-10T19:08:48.488720+0000","last_undegraded":"2024-09-10T19:08:48.488720+0000","last_fullsized":"2024-09-10T19:08:48.488720+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:48.488581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:08:48.488581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:10.547013+0000","scrub_duration":0.001868705,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":49,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.882055+0000","last_change":"2024-09-10T19:08:35.383303+0000","last_active":"2024-09-10T19:08:37.882055+0000","last_peered":"2024-09-10T19:08:37.882055+0000","last_clean":"2024-09-10T19:08:37.882055+0000","last_became_active":"2024-09-10T19:07:21.859818+0000","last_became_peered":"2024-09-10T19:07:21.859818+0000","last_unstale":"2024-09-10T19:08:37.882055+0000","last_undegraded":"2024-09-10T19:08:37.882055+0000","last_fullsized":"2024-09-10T19:08:37.882055+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:35.383240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_clean_scrub_stamp":"2024-09-10T19:08:35.383240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:00.735949+0000","scrub_duration":0.0018617659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":59,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.624939+0000","last_change":"2024-09-10T19:07:55.020542+0000","last_active":"2024-09-10T19:08:37.624939+0000","last_peered":"2024-09-10T19:08:37.624939+0000","last_clean":"2024-09-10T19:08:37.624939+0000","last_became_active":"2024-09-10T19:07:10.335110+0000","last_became_peered":"2024-09-10T19:07:10.335110+0000","last_unstale":"2024-09-10T19:08:37.624939+0000","last_undegraded":"2024-09-10T19:08:37.624939+0000","last_fullsized":"2024-09-10T19:08:37.624939+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:55.020478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:53.345184+0000","last_clean_scrub_stamp":"2024-09-10T19:07:55.020478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:09.050199+0000","scrub_duration":0.001414558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.d","version":"70'81","reported_seq":53,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090759+0000","last_change":"2024-09-10T19:07:56.415636+0000","last_active":"2024-09-10T19:08:38.090759+0000","last_peered":"2024-09-10T19:08:38.090759+0000","last_clean":"2024-09-10T19:08:38.090759+0000","last_became_active":"2024-09-10T19:07:18.738294+0000","last_became_peered":"2024-09-10T19:07:18.738294+0000","last_unstale":"2024-09-10T19:08:38.090759+0000","last_undegraded":"2024-09-10T19:08:38.090759+0000","last_fullsized":"2024-09-10T19:08:38.090759+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:07:56.415568+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:07:56.415568+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:11.205660+0000","scrub_duration":0.0018649300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"427'46","reported_seq":263,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.999505+0000","last_change":"2024-09-10T19:08:28.594621+0000","last_active":"2024-09-10T19:08:37.999505+0000","last_peered":"2024-09-10T19:08:37.999505+0000","last_clean":"2024-09-10T19:08:37.999505+0000","last_became_active":"2024-09-10T19:07:18.736452+0000","last_became_peered":"2024-09-10T19:07:18.736452+0000","last_unstale":"2024-09-10T19:08:37.999505+0000","last_undegraded":"2024-09-10T19:08:37.999505+0000","last_fullsized":"2024-09-10T19:08:37.999505+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'46","last_scrub_stamp":"2024-09-10T19:08:28.594544+0000","last_deep_scrub":"365'34","last_deep_scrub_stamp":"2024-09-10T19:02:18.712306+0000","last_clean_scrub_stamp":"2024-09-10T19:08:28.594544+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:40.319493+0000","scrub_duration":0.0022179560000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":60,"num_read_kb":51,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":49,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.882606+0000","last_change":"2024-09-10T19:07:54.991286+0000","last_active":"2024-09-10T19:08:37.882606+0000","last_peered":"2024-09-10T19:08:37.882606+0000","last_clean":"2024-09-10T19:08:37.882606+0000","last_became_active":"2024-09-10T19:07:21.860619+0000","last_became_peered":"2024-09-10T19:07:21.860619+0000","last_unstale":"2024-09-10T19:08:37.882606+0000","last_undegraded":"2024-09-10T19:08:37.882606+0000","last_fullsized":"2024-09-10T19:08:37.882606+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:54.991205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:05.603125+0000","last_clean_scrub_stamp":"2024-09-10T19:07:54.991205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:22.141928+0000","scrub_duration":0.001854613,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"464'11","reported_seq":77,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:51.353728+0000","last_change":"2024-09-10T19:08:22.289110+0000","last_active":"2024-09-10T19:08:51.353728+0000","last_peered":"2024-09-10T19:08:51.353728+0000","last_clean":"2024-09-10T19:08:51.353728+0000","last_became_active":"2024-09-10T19:07:18.744158+0000","last_became_peered":"2024-09-10T19:07:18.744158+0000","last_unstale":"2024-09-10T19:08:51.353728+0000","last_undegraded":"2024-09-10T19:08:51.353728+0000","last_fullsized":"2024-09-10T19:08:51.353728+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"464'11","last_scrub_stamp":"2024-09-10T19:08:22.289040+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-09-10T18:47:02.582229+0000","last_clean_scrub_stamp":"2024-09-10T19:08:22.289040+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:34.235717+0000","scrub_duration":0.0017950539999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":537,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.000869+0000","last_change":"2024-09-10T19:07:52.307859+0000","last_active":"2024-09-10T19:08:38.000869+0000","last_peered":"2024-09-10T19:08:38.000869+0000","last_clean":"2024-09-10T19:08:38.000869+0000","last_became_active":"2024-09-10T18:55:40.389202+0000","last_became_peered":"2024-09-10T18:55:40.389202+0000","last_unstale":"2024-09-10T19:08:38.000869+0000","last_undegraded":"2024-09-10T19:08:38.000869+0000","last_fullsized":"2024-09-10T19:08:38.000869+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:52.307747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_clean_scrub_stamp":"2024-09-10T19:07:52.307747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:19.489744+0000","scrub_duration":0.001619391,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.c","version":"70'81","reported_seq":478,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:46.912249+0000","last_change":"2024-09-10T19:08:46.912219+0000","last_active":"2024-09-10T19:08:46.912249+0000","last_peered":"2024-09-10T19:08:46.912249+0000","last_clean":"2024-09-10T19:08:46.912249+0000","last_became_active":"2024-09-10T18:55:40.390655+0000","last_became_peered":"2024-09-10T18:55:40.390655+0000","last_unstale":"2024-09-10T19:08:46.912249+0000","last_undegraded":"2024-09-10T19:08:46.912249+0000","last_fullsized":"2024-09-10T19:08:46.912249+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:08:46.912148+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:56:33.779154+0000","last_clean_scrub_stamp":"2024-09-10T19:08:46.912148+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:09.086147+0000","scrub_duration":0.00455558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"427'24","reported_seq":567,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.624769+0000","last_change":"2024-09-10T19:08:02.119915+0000","last_active":"2024-09-10T19:08:37.624769+0000","last_peered":"2024-09-10T19:08:37.624769+0000","last_clean":"2024-09-10T19:08:37.624769+0000","last_became_active":"2024-09-10T18:45:20.056869+0000","last_became_peered":"2024-09-10T18:45:20.056869+0000","last_unstale":"2024-09-10T19:08:37.624769+0000","last_undegraded":"2024-09-10T19:08:37.624769+0000","last_fullsized":"2024-09-10T19:08:37.624769+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'24","last_scrub_stamp":"2024-09-10T19:08:02.119848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:08:02.119848+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:06.509071+0000","scrub_duration":0.0024358679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"56'2","reported_seq":584,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.625848+0000","last_change":"2024-09-10T19:07:59.100435+0000","last_active":"2024-09-10T19:08:37.625848+0000","last_peered":"2024-09-10T19:08:37.625848+0000","last_clean":"2024-09-10T19:08:37.625848+0000","last_became_active":"2024-09-10T19:07:18.733655+0000","last_became_peered":"2024-09-10T19:07:18.733655+0000","last_unstale":"2024-09-10T19:08:37.625848+0000","last_undegraded":"2024-09-10T19:08:37.625848+0000","last_fullsized":"2024-09-10T19:08:37.625848+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-10T19:07:59.100378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:07:59.100378+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:05.990069+0000","scrub_duration":0.0018153150000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":49,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.881835+0000","last_change":"2024-09-10T19:07:53.056021+0000","last_active":"2024-09-10T19:08:37.881835+0000","last_peered":"2024-09-10T19:08:37.881835+0000","last_clean":"2024-09-10T19:08:37.881835+0000","last_became_active":"2024-09-10T19:07:21.860783+0000","last_became_peered":"2024-09-10T19:07:21.860783+0000","last_unstale":"2024-09-10T19:08:37.881835+0000","last_undegraded":"2024-09-10T19:08:37.881835+0000","last_fullsized":"2024-09-10T19:08:37.881835+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:53.055935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:02.711594+0000","last_clean_scrub_stamp":"2024-09-10T19:07:53.055935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:10.070848+0000","scrub_duration":0.001537409,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":528,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090156+0000","last_change":"2024-09-10T19:07:42.668256+0000","last_active":"2024-09-10T19:08:38.090156+0000","last_peered":"2024-09-10T19:08:38.090156+0000","last_clean":"2024-09-10T19:08:38.090156+0000","last_became_active":"2024-09-10T19:07:18.744805+0000","last_became_peered":"2024-09-10T19:07:18.744805+0000","last_unstale":"2024-09-10T19:08:38.090156+0000","last_undegraded":"2024-09-10T19:08:38.090156+0000","last_fullsized":"2024-09-10T19:08:38.090156+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:42.668198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:46.710933+0000","last_clean_scrub_stamp":"2024-09-10T19:07:42.668198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:07.198960+0000","scrub_duration":0.0014811570000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.13","version":"70'81","reported_seq":359,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.623349+0000","last_change":"2024-09-10T19:07:42.333202+0000","last_active":"2024-09-10T19:08:37.623349+0000","last_peered":"2024-09-10T19:08:37.623349+0000","last_clean":"2024-09-10T19:08:37.623349+0000","last_became_active":"2024-09-10T19:07:31.174330+0000","last_became_peered":"2024-09-10T19:07:31.174330+0000","last_unstale":"2024-09-10T19:08:37.623349+0000","last_undegraded":"2024-09-10T19:08:37.623349+0000","last_fullsized":"2024-09-10T19:08:37.623349+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:07:42.333149+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:07:42.333149+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:11.231065+0000","scrub_duration":0.0022502709999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"427'25","reported_seq":584,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.623576+0000","last_change":"2024-09-10T19:07:41.717926+0000","last_active":"2024-09-10T19:08:37.623576+0000","last_peered":"2024-09-10T19:08:37.623576+0000","last_clean":"2024-09-10T19:08:37.623576+0000","last_became_active":"2024-09-10T19:07:18.741530+0000","last_became_peered":"2024-09-10T19:07:18.741530+0000","last_unstale":"2024-09-10T19:08:37.623576+0000","last_undegraded":"2024-09-10T19:08:37.623576+0000","last_fullsized":"2024-09-10T19:08:37.623576+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'25","last_scrub_stamp":"2024-09-10T19:07:41.717856+0000","last_deep_scrub":"76'13","last_deep_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_clean_scrub_stamp":"2024-09-10T19:07:41.717856+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:00.977373+0000","scrub_duration":0.0022350550000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":59,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:55.281463+0000","last_change":"2024-09-10T19:08:55.281438+0000","last_active":"2024-09-10T19:08:55.281463+0000","last_peered":"2024-09-10T19:08:55.281463+0000","last_clean":"2024-09-10T19:08:55.281463+0000","last_became_active":"2024-09-10T19:07:21.871872+0000","last_became_peered":"2024-09-10T19:07:21.871872+0000","last_unstale":"2024-09-10T19:08:55.281463+0000","last_undegraded":"2024-09-10T19:08:55.281463+0000","last_fullsized":"2024-09-10T19:08:55.281463+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:55.281383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:07:22.144520+0000","last_clean_scrub_stamp":"2024-09-10T19:08:55.281383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:01.608331+0000","scrub_duration":0.0012327600000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":540,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.091153+0000","last_change":"2024-09-10T19:07:59.953800+0000","last_active":"2024-09-10T19:08:38.091153+0000","last_peered":"2024-09-10T19:08:38.091153+0000","last_clean":"2024-09-10T19:08:38.091153+0000","last_became_active":"2024-09-10T19:07:18.740503+0000","last_became_peered":"2024-09-10T19:07:18.740503+0000","last_unstale":"2024-09-10T19:08:38.091153+0000","last_undegraded":"2024-09-10T19:08:38.091153+0000","last_fullsized":"2024-09-10T19:08:38.091153+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:59.953745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:00.754912+0000","last_clean_scrub_stamp":"2024-09-10T19:07:59.953745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:29.248893+0000","scrub_duration":0.001465907,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"364'2","reported_seq":53,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.882590+0000","last_change":"2024-09-10T19:08:23.438820+0000","last_active":"2024-09-10T19:08:37.882590+0000","last_peered":"2024-09-10T19:08:37.882590+0000","last_clean":"2024-09-10T19:08:37.882590+0000","last_became_active":"2024-09-10T19:07:21.869572+0000","last_became_peered":"2024-09-10T19:07:21.869572+0000","last_unstale":"2024-09-10T19:08:37.882590+0000","last_undegraded":"2024-09-10T19:08:37.882590+0000","last_fullsized":"2024-09-10T19:08:37.882590+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"364'2","last_scrub_stamp":"2024-09-10T19:08:23.438750+0000","last_deep_scrub":"364'2","last_deep_scrub_stamp":"2024-09-10T19:02:15.059320+0000","last_clean_scrub_stamp":"2024-09-10T19:08:23.438750+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:39.690587+0000","scrub_duration":0.0021737689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":65,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.12","version":"70'81","reported_seq":394,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.000050+0000","last_change":"2024-09-10T19:08:21.550650+0000","last_active":"2024-09-10T19:08:38.000050+0000","last_peered":"2024-09-10T19:08:38.000050+0000","last_clean":"2024-09-10T19:08:38.000050+0000","last_became_active":"2024-09-10T18:52:16.598921+0000","last_became_peered":"2024-09-10T18:52:16.598921+0000","last_unstale":"2024-09-10T19:08:38.000050+0000","last_undegraded":"2024-09-10T19:08:38.000050+0000","last_fullsized":"2024-09-10T19:08:38.000050+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:08:21.550578+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:54.127476+0000","last_clean_scrub_stamp":"2024-09-10T19:08:21.550578+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:41.526899+0000","scrub_duration":0.001579219,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":573,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.999457+0000","last_change":"2024-09-10T19:07:42.639042+0000","last_active":"2024-09-10T19:08:37.999457+0000","last_peered":"2024-09-10T19:08:37.999457+0000","last_clean":"2024-09-10T19:08:37.999457+0000","last_became_active":"2024-09-10T19:07:18.745376+0000","last_became_peered":"2024-09-10T19:07:18.745376+0000","last_unstale":"2024-09-10T19:08:37.999457+0000","last_undegraded":"2024-09-10T19:08:37.999457+0000","last_fullsized":"2024-09-10T19:08:37.999457+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:42.638976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:07:42.638976+0000","last_clean_scrub_stamp":"2024-09-10T19:07:42.638976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:08.399814+0000","scrub_duration":0.0027179790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,5],"acting":[6,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"477'21","reported_seq":581,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.091238+0000","last_change":"2024-09-10T19:07:47.764411+0000","last_active":"2024-09-10T19:08:38.091238+0000","last_peered":"2024-09-10T19:08:38.091238+0000","last_clean":"2024-09-10T19:08:38.091238+0000","last_became_active":"2024-09-10T19:07:18.744389+0000","last_became_peered":"2024-09-10T19:07:18.744389+0000","last_unstale":"2024-09-10T19:08:38.091238+0000","last_undegraded":"2024-09-10T19:08:38.091238+0000","last_fullsized":"2024-09-10T19:08:38.091238+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"477'21","last_scrub_stamp":"2024-09-10T19:07:47.764352+0000","last_deep_scrub":"477'21","last_deep_scrub_stamp":"2024-09-10T19:07:47.764352+0000","last_clean_scrub_stamp":"2024-09-10T19:07:47.764352+0000","objects_scrubbed":3,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:08:54.145582+0000","scrub_duration":0.0029292720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":24,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":522,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.000429+0000","last_change":"2024-09-10T19:08:22.507010+0000","last_active":"2024-09-10T19:08:38.000429+0000","last_peered":"2024-09-10T19:08:38.000429+0000","last_clean":"2024-09-10T19:08:38.000429+0000","last_became_active":"2024-09-10T18:45:22.069653+0000","last_became_peered":"2024-09-10T18:45:22.069653+0000","last_unstale":"2024-09-10T19:08:38.000429+0000","last_undegraded":"2024-09-10T19:08:38.000429+0000","last_fullsized":"2024-09-10T19:08:38.000429+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:22.506947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:08:22.506947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:23.445528+0000","scrub_duration":0.001519482,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":53,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090762+0000","last_change":"2024-09-10T19:08:00.345603+0000","last_active":"2024-09-10T19:08:38.090762+0000","last_peered":"2024-09-10T19:08:38.090762+0000","last_clean":"2024-09-10T19:08:38.090762+0000","last_became_active":"2024-09-10T19:07:18.747593+0000","last_became_peered":"2024-09-10T19:07:18.747593+0000","last_unstale":"2024-09-10T19:08:38.090762+0000","last_undegraded":"2024-09-10T19:08:38.090762+0000","last_fullsized":"2024-09-10T19:08:38.090762+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:00.345535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:08:00.345535+0000","last_clean_scrub_stamp":"2024-09-10T19:08:00.345535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:15.358692+0000","scrub_duration":0.0022865070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.11","version":"70'81","reported_seq":430,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.621883+0000","last_change":"2024-09-10T19:07:48.748563+0000","last_active":"2024-09-10T19:08:37.621883+0000","last_peered":"2024-09-10T19:08:37.621883+0000","last_clean":"2024-09-10T19:08:37.621883+0000","last_became_active":"2024-09-10T19:00:42.517311+0000","last_became_peered":"2024-09-10T19:00:42.517311+0000","last_unstale":"2024-09-10T19:08:37.621883+0000","last_undegraded":"2024-09-10T19:08:37.621883+0000","last_fullsized":"2024-09-10T19:08:37.621883+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:07:48.748483+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:07:48.748483+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:16.397728+0000","scrub_duration":0.001963743,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"496'35","reported_seq":682,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:58.759252+0000","last_change":"2024-09-10T19:08:58.759200+0000","last_active":"2024-09-10T19:08:58.759252+0000","last_peered":"2024-09-10T19:08:58.759252+0000","last_clean":"2024-09-10T19:08:58.759252+0000","last_became_active":"2024-09-10T19:07:58.853107+0000","last_became_peered":"2024-09-10T19:07:58.853107+0000","last_unstale":"2024-09-10T19:08:58.759252+0000","last_undegraded":"2024-09-10T19:08:58.759252+0000","last_fullsized":"2024-09-10T19:08:58.759252+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":0,"last_scrub":"496'35","last_scrub_stamp":"2024-09-10T19:08:58.759128+0000","last_deep_scrub":"496'35","last_deep_scrub_stamp":"2024-09-10T19:08:58.759128+0000","last_clean_scrub_stamp":"2024-09-10T19:08:58.759128+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:08.120710+0000","scrub_duration":0.00451542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":556,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.999711+0000","last_change":"2024-09-10T19:08:37.591591+0000","last_active":"2024-09-10T19:08:37.999711+0000","last_peered":"2024-09-10T19:08:37.999711+0000","last_clean":"2024-09-10T19:08:37.999711+0000","last_became_active":"2024-09-10T19:07:18.728727+0000","last_became_peered":"2024-09-10T19:07:18.728727+0000","last_unstale":"2024-09-10T19:08:37.999711+0000","last_undegraded":"2024-09-10T19:08:37.999711+0000","last_fullsized":"2024-09-10T19:08:37.999711+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:37.591523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:08:37.591523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:43.402962+0000","scrub_duration":0.0013764280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"304'11","reported_seq":849,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:56.354838+0000","last_change":"2024-09-10T19:07:48.293221+0000","last_active":"2024-09-10T19:08:56.354838+0000","last_peered":"2024-09-10T19:08:56.354838+0000","last_clean":"2024-09-10T19:08:56.354838+0000","last_became_active":"2024-09-10T18:58:58.489370+0000","last_became_peered":"2024-09-10T18:58:58.489370+0000","last_unstale":"2024-09-10T19:08:56.354838+0000","last_undegraded":"2024-09-10T19:08:56.354838+0000","last_fullsized":"2024-09-10T19:08:56.354838+0000","mapping_epoch":303,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":304,"parent":"0.0","parent_split_bits":0,"last_scrub":"304'11","last_scrub_stamp":"2024-09-10T19:07:48.293171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:07:48.293171+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:06.354160+0000","scrub_duration":0.0022523560000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":225,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.623354+0000","last_change":"2024-09-10T19:07:58.762471+0000","last_active":"2024-09-10T19:08:37.623354+0000","last_peered":"2024-09-10T19:08:37.623354+0000","last_clean":"2024-09-10T19:08:37.623354+0000","last_became_active":"2024-09-10T19:07:18.728626+0000","last_became_peered":"2024-09-10T19:07:18.728626+0000","last_unstale":"2024-09-10T19:08:37.623354+0000","last_undegraded":"2024-09-10T19:08:37.623354+0000","last_fullsized":"2024-09-10T19:08:37.623354+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:58.762413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:57.476724+0000","last_clean_scrub_stamp":"2024-09-10T19:07:58.762413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:25.545987+0000","scrub_duration":0.0013127989999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.10","version":"70'81","reported_seq":404,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.999459+0000","last_change":"2024-09-10T19:08:02.499016+0000","last_active":"2024-09-10T19:08:37.999459+0000","last_peered":"2024-09-10T19:08:37.999459+0000","last_clean":"2024-09-10T19:08:37.999459+0000","last_became_active":"2024-09-10T19:07:18.736928+0000","last_became_peered":"2024-09-10T19:07:18.736928+0000","last_unstale":"2024-09-10T19:08:37.999459+0000","last_undegraded":"2024-09-10T19:08:37.999459+0000","last_fullsized":"2024-09-10T19:08:37.999459+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:08:02.498946+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:08:02.498946+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:29.176994+0000","scrub_duration":0.001578643,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"427'21","reported_seq":235,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:56.713248+0000","last_change":"2024-09-10T19:08:56.713210+0000","last_active":"2024-09-10T19:08:56.713248+0000","last_peered":"2024-09-10T19:08:56.713248+0000","last_clean":"2024-09-10T19:08:56.713248+0000","last_became_active":"2024-09-10T19:07:18.735867+0000","last_became_peered":"2024-09-10T19:07:18.735867+0000","last_unstale":"2024-09-10T19:08:56.713248+0000","last_undegraded":"2024-09-10T19:08:56.713248+0000","last_fullsized":"2024-09-10T19:08:56.713248+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'21","last_scrub_stamp":"2024-09-10T19:08:56.713143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:08:56.713143+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:05.888161+0000","scrub_duration":0.0017699300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":33,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":504,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.622334+0000","last_change":"2024-09-10T19:07:35.695340+0000","last_active":"2024-09-10T19:08:37.622334+0000","last_peered":"2024-09-10T19:08:37.622334+0000","last_clean":"2024-09-10T19:08:37.622334+0000","last_became_active":"2024-09-10T18:55:40.390697+0000","last_became_peered":"2024-09-10T18:55:40.390697+0000","last_unstale":"2024-09-10T19:08:37.622334+0000","last_undegraded":"2024-09-10T19:08:37.622334+0000","last_fullsized":"2024-09-10T19:08:37.622334+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:35.695282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:07:35.695282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:01.337711+0000","scrub_duration":0.0015554239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"465'13","reported_seq":861,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:56.354542+0000","last_change":"2024-09-10T19:07:44.727669+0000","last_active":"2024-09-10T19:08:56.354542+0000","last_peered":"2024-09-10T19:08:56.354542+0000","last_clean":"2024-09-10T19:08:56.354542+0000","last_became_active":"2024-09-10T19:07:21.862583+0000","last_became_peered":"2024-09-10T19:07:21.862583+0000","last_unstale":"2024-09-10T19:08:56.354542+0000","last_undegraded":"2024-09-10T19:08:56.354542+0000","last_fullsized":"2024-09-10T19:08:56.354542+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"465'13","last_scrub_stamp":"2024-09-10T19:07:44.727587+0000","last_deep_scrub":"273'11","last_deep_scrub_stamp":"2024-09-10T19:03:49.744177+0000","last_clean_scrub_stamp":"2024-09-10T19:07:44.727587+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:08.978598+0000","scrub_duration":0.0024831419999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":556,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.625600+0000","last_change":"2024-09-10T19:08:31.955198+0000","last_active":"2024-09-10T19:08:37.625600+0000","last_peered":"2024-09-10T19:08:37.625600+0000","last_clean":"2024-09-10T19:08:37.625600+0000","last_became_active":"2024-09-10T19:07:21.864485+0000","last_became_peered":"2024-09-10T19:07:21.864485+0000","last_unstale":"2024-09-10T19:08:37.625600+0000","last_undegraded":"2024-09-10T19:08:37.625600+0000","last_fullsized":"2024-09-10T19:08:37.625600+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:31.955134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_clean_scrub_stamp":"2024-09-10T19:08:31.955134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:59.445634+0000","scrub_duration":0.001675099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"427'21","reported_seq":398,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.622961+0000","last_change":"2024-09-10T19:08:27.455864+0000","last_active":"2024-09-10T19:08:37.622961+0000","last_peered":"2024-09-10T19:08:37.622961+0000","last_clean":"2024-09-10T19:08:37.622961+0000","last_became_active":"2024-09-10T19:07:31.172489+0000","last_became_peered":"2024-09-10T19:07:31.172489+0000","last_unstale":"2024-09-10T19:08:37.622961+0000","last_undegraded":"2024-09-10T19:08:37.622961+0000","last_fullsized":"2024-09-10T19:08:37.622961+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'21","last_scrub_stamp":"2024-09-10T19:08:27.455794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:08:27.455794+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:55.978396+0000","scrub_duration":0.0021581159999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":33,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":577,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.000044+0000","last_change":"2024-09-10T19:08:19.699615+0000","last_active":"2024-09-10T19:08:38.000044+0000","last_peered":"2024-09-10T19:08:38.000044+0000","last_clean":"2024-09-10T19:08:38.000044+0000","last_became_active":"2024-09-10T19:00:42.518310+0000","last_became_peered":"2024-09-10T19:00:42.518310+0000","last_unstale":"2024-09-10T19:08:38.000044+0000","last_undegraded":"2024-09-10T19:08:38.000044+0000","last_fullsized":"2024-09-10T19:08:38.000044+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:19.699546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:08:19.699546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:46.708862+0000","scrub_duration":0.001746081,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":527,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:53.809578+0000","last_change":"2024-09-10T19:08:53.809553+0000","last_active":"2024-09-10T19:08:53.809578+0000","last_peered":"2024-09-10T19:08:53.809578+0000","last_clean":"2024-09-10T19:08:53.809578+0000","last_became_active":"2024-09-10T18:55:40.398243+0000","last_became_peered":"2024-09-10T18:55:40.398243+0000","last_unstale":"2024-09-10T19:08:53.809578+0000","last_undegraded":"2024-09-10T19:08:53.809578+0000","last_fullsized":"2024-09-10T19:08:53.809578+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:53.809498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:03.571786+0000","last_clean_scrub_stamp":"2024-09-10T19:08:53.809498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:13.709684+0000","scrub_duration":0.001244666,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":519,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.621992+0000","last_change":"2024-09-10T19:08:14.749177+0000","last_active":"2024-09-10T19:08:37.621992+0000","last_peered":"2024-09-10T19:08:37.621992+0000","last_clean":"2024-09-10T19:08:37.621992+0000","last_became_active":"2024-09-10T19:00:42.516833+0000","last_became_peered":"2024-09-10T19:00:42.516833+0000","last_unstale":"2024-09-10T19:08:37.621992+0000","last_undegraded":"2024-09-10T19:08:37.621992+0000","last_fullsized":"2024-09-10T19:08:37.621992+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:14.749121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:04.631558+0000","last_clean_scrub_stamp":"2024-09-10T19:08:14.749121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:33.382416+0000","scrub_duration":0.001392431,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"427'32","reported_seq":571,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:52.995569+0000","last_change":"2024-09-10T19:08:52.995534+0000","last_active":"2024-09-10T19:08:52.995569+0000","last_peered":"2024-09-10T19:08:52.995569+0000","last_clean":"2024-09-10T19:08:52.995569+0000","last_became_active":"2024-09-10T18:45:20.056885+0000","last_became_peered":"2024-09-10T18:45:20.056885+0000","last_unstale":"2024-09-10T19:08:52.995569+0000","last_undegraded":"2024-09-10T19:08:52.995569+0000","last_fullsized":"2024-09-10T19:08:52.995569+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'32","last_scrub_stamp":"2024-09-10T19:08:52.995463+0000","last_deep_scrub":"223'16","last_deep_scrub_stamp":"2024-09-10T18:54:47.324044+0000","last_clean_scrub_stamp":"2024-09-10T19:08:52.995463+0000","objects_scrubbed":5,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:04.448078+0000","scrub_duration":0.002102151,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":450,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.623833+0000","last_change":"2024-09-10T19:07:51.360152+0000","last_active":"2024-09-10T19:08:37.623833+0000","last_peered":"2024-09-10T19:08:37.623833+0000","last_clean":"2024-09-10T19:08:37.623833+0000","last_became_active":"2024-09-10T19:07:31.169826+0000","last_became_peered":"2024-09-10T19:07:31.169826+0000","last_unstale":"2024-09-10T19:08:37.623833+0000","last_undegraded":"2024-09-10T19:08:37.623833+0000","last_fullsized":"2024-09-10T19:08:37.623833+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:51.360097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:00.029389+0000","last_clean_scrub_stamp":"2024-09-10T19:07:51.360097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:16.451579+0000","scrub_duration":0.0036568999999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":432,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:56.707495+0000","last_change":"2024-09-10T19:08:56.707466+0000","last_active":"2024-09-10T19:08:56.707495+0000","last_peered":"2024-09-10T19:08:56.707495+0000","last_clean":"2024-09-10T19:08:56.707495+0000","last_became_active":"2024-09-10T19:07:31.174416+0000","last_became_peered":"2024-09-10T19:07:31.174416+0000","last_unstale":"2024-09-10T19:08:56.707495+0000","last_undegraded":"2024-09-10T19:08:56.707495+0000","last_fullsized":"2024-09-10T19:08:56.707495+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:56.707409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:08:56.707409+0000","last_clean_scrub_stamp":"2024-09-10T19:08:56.707409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:07.472583+0000","scrub_duration":0.001457666,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":525,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090386+0000","last_change":"2024-09-10T19:08:00.930572+0000","last_active":"2024-09-10T19:08:38.090386+0000","last_peered":"2024-09-10T19:08:38.090386+0000","last_clean":"2024-09-10T19:08:38.090386+0000","last_became_active":"2024-09-10T18:55:40.395595+0000","last_became_peered":"2024-09-10T18:55:40.395595+0000","last_unstale":"2024-09-10T19:08:38.090386+0000","last_undegraded":"2024-09-10T19:08:38.090386+0000","last_fullsized":"2024-09-10T19:08:38.090386+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:00.930516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:08:00.930516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:28.285123+0000","scrub_duration":0.003078005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"496'14","reported_seq":542,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:49.878817+0000","last_change":"2024-09-10T19:08:49.878776+0000","last_active":"2024-09-10T19:08:49.878817+0000","last_peered":"2024-09-10T19:08:49.878817+0000","last_clean":"2024-09-10T19:08:49.878817+0000","last_became_active":"2024-09-10T18:45:20.059205+0000","last_became_peered":"2024-09-10T18:45:20.059205+0000","last_unstale":"2024-09-10T19:08:49.878817+0000","last_undegraded":"2024-09-10T19:08:49.878817+0000","last_fullsized":"2024-09-10T19:08:49.878817+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"496'14","last_scrub_stamp":"2024-09-10T19:08:49.878709+0000","last_deep_scrub":"496'14","last_deep_scrub_stamp":"2024-09-10T19:08:49.878709+0000","last_clean_scrub_stamp":"2024-09-10T19:08:49.878709+0000","objects_scrubbed":2,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:14.266577+0000","scrub_duration":0.0033456639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":16,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":552,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.000503+0000","last_change":"2024-09-10T19:08:31.587485+0000","last_active":"2024-09-10T19:08:38.000503+0000","last_peered":"2024-09-10T19:08:38.000503+0000","last_clean":"2024-09-10T19:08:38.000503+0000","last_became_active":"2024-09-10T19:07:21.861473+0000","last_became_peered":"2024-09-10T19:07:21.861473+0000","last_unstale":"2024-09-10T19:08:38.000503+0000","last_undegraded":"2024-09-10T19:08:38.000503+0000","last_fullsized":"2024-09-10T19:08:38.000503+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:31.587412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:08:31.587412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:47.798853+0000","scrub_duration":0.001726331,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":563,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.999865+0000","last_change":"2024-09-10T19:08:00.729598+0000","last_active":"2024-09-10T19:08:37.999865+0000","last_peered":"2024-09-10T19:08:37.999865+0000","last_clean":"2024-09-10T19:08:37.999865+0000","last_became_active":"2024-09-10T19:07:18.735293+0000","last_became_peered":"2024-09-10T19:07:18.735293+0000","last_unstale":"2024-09-10T19:08:37.999865+0000","last_undegraded":"2024-09-10T19:08:37.999865+0000","last_fullsized":"2024-09-10T19:08:37.999865+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:00.729528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:08:00.729528+0000","last_clean_scrub_stamp":"2024-09-10T19:08:00.729528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:21.429536+0000","scrub_duration":0.0014191550000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":73,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:48.327963+0000","last_change":"2024-09-10T19:08:48.327919+0000","last_active":"2024-09-10T19:08:48.327963+0000","last_peered":"2024-09-10T19:08:48.327963+0000","last_clean":"2024-09-10T19:08:48.327963+0000","last_became_active":"2024-09-10T19:07:21.857464+0000","last_became_peered":"2024-09-10T19:07:21.857464+0000","last_unstale":"2024-09-10T19:08:48.327963+0000","last_undegraded":"2024-09-10T19:08:48.327963+0000","last_fullsized":"2024-09-10T19:08:48.327963+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:48.327842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:55.145638+0000","last_clean_scrub_stamp":"2024-09-10T19:08:48.327842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:01.011992+0000","scrub_duration":0.0018529830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.14","version":"70'81","reported_seq":59,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:54.261915+0000","last_change":"2024-09-10T19:08:54.261870+0000","last_active":"2024-09-10T19:08:54.261915+0000","last_peered":"2024-09-10T19:08:54.261915+0000","last_clean":"2024-09-10T19:08:54.261915+0000","last_became_active":"2024-09-10T19:07:21.869220+0000","last_became_peered":"2024-09-10T19:07:21.869220+0000","last_unstale":"2024-09-10T19:08:54.261915+0000","last_undegraded":"2024-09-10T19:08:54.261915+0000","last_fullsized":"2024-09-10T19:08:54.261915+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:08:54.261813+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T19:07:34.099033+0000","last_clean_scrub_stamp":"2024-09-10T19:08:54.261813+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:00.497853+0000","scrub_duration":0.001847813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"427'32","reported_seq":604,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090136+0000","last_change":"2024-09-10T19:07:34.601600+0000","last_active":"2024-09-10T19:08:38.090136+0000","last_peered":"2024-09-10T19:08:38.090136+0000","last_clean":"2024-09-10T19:08:38.090136+0000","last_became_active":"2024-09-10T19:07:18.748467+0000","last_became_peered":"2024-09-10T19:07:18.748467+0000","last_unstale":"2024-09-10T19:08:38.090136+0000","last_undegraded":"2024-09-10T19:08:38.090136+0000","last_fullsized":"2024-09-10T19:08:38.090136+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'32","last_scrub_stamp":"2024-09-10T19:07:34.601536+0000","last_deep_scrub":"427'32","last_deep_scrub_stamp":"2024-09-10T19:07:34.601536+0000","last_clean_scrub_stamp":"2024-09-10T19:07:34.601536+0000","objects_scrubbed":5,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:08:57.681075+0000","scrub_duration":0.0042432520000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"56'1","reported_seq":585,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:58.554838+0000","last_change":"2024-09-10T19:08:58.554789+0000","last_active":"2024-09-10T19:08:58.554838+0000","last_peered":"2024-09-10T19:08:58.554838+0000","last_clean":"2024-09-10T19:08:58.554838+0000","last_became_active":"2024-09-10T19:07:21.863281+0000","last_became_peered":"2024-09-10T19:07:21.863281+0000","last_unstale":"2024-09-10T19:08:58.554838+0000","last_undegraded":"2024-09-10T19:08:58.554838+0000","last_fullsized":"2024-09-10T19:08:58.554838+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T19:08:58.554722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:08:58.554722+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:07.497294+0000","scrub_duration":0.0016735700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":724,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":53,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090281+0000","last_change":"2024-09-10T19:08:10.318322+0000","last_active":"2024-09-10T19:08:38.090281+0000","last_peered":"2024-09-10T19:08:38.090281+0000","last_clean":"2024-09-10T19:08:38.090281+0000","last_became_active":"2024-09-10T19:07:18.736879+0000","last_became_peered":"2024-09-10T19:07:18.736879+0000","last_unstale":"2024-09-10T19:08:38.090281+0000","last_undegraded":"2024-09-10T19:08:38.090281+0000","last_fullsized":"2024-09-10T19:08:38.090281+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:10.318253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:08:10.318253+0000","last_clean_scrub_stamp":"2024-09-10T19:08:10.318253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:25.532507+0000","scrub_duration":0.00153034,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":535,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090165+0000","last_change":"2024-09-10T19:07:58.918510+0000","last_active":"2024-09-10T19:08:38.090165+0000","last_peered":"2024-09-10T19:08:38.090165+0000","last_clean":"2024-09-10T19:08:38.090165+0000","last_became_active":"2024-09-10T18:55:40.397523+0000","last_became_peered":"2024-09-10T18:55:40.397523+0000","last_unstale":"2024-09-10T19:08:38.090165+0000","last_undegraded":"2024-09-10T19:08:38.090165+0000","last_fullsized":"2024-09-10T19:08:38.090165+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:58.918461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:06:20.199071+0000","last_clean_scrub_stamp":"2024-09-10T19:07:58.918461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:08:59.082533+0000","scrub_duration":0.0029530379999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"78'2","reported_seq":71,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090931+0000","last_change":"2024-09-10T19:08:17.397159+0000","last_active":"2024-09-10T19:08:38.090931+0000","last_peered":"2024-09-10T19:08:38.090931+0000","last_clean":"2024-09-10T19:08:38.090931+0000","last_became_active":"2024-09-10T19:07:21.860858+0000","last_became_peered":"2024-09-10T19:07:21.860858+0000","last_unstale":"2024-09-10T19:08:38.090931+0000","last_undegraded":"2024-09-10T19:08:38.090931+0000","last_fullsized":"2024-09-10T19:08:38.090931+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'2","last_scrub_stamp":"2024-09-10T19:08:17.397089+0000","last_deep_scrub":"78'2","last_deep_scrub_stamp":"2024-09-10T19:02:15.686138+0000","last_clean_scrub_stamp":"2024-09-10T19:08:17.397089+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:45.895036+0000","scrub_duration":0.001821238,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":546,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.999553+0000","last_change":"2024-09-10T19:07:58.383056+0000","last_active":"2024-09-10T19:08:37.999553+0000","last_peered":"2024-09-10T19:08:37.999553+0000","last_clean":"2024-09-10T19:08:37.999553+0000","last_became_active":"2024-09-10T19:07:18.745185+0000","last_became_peered":"2024-09-10T19:07:18.745185+0000","last_unstale":"2024-09-10T19:08:37.999553+0000","last_undegraded":"2024-09-10T19:08:37.999553+0000","last_fullsized":"2024-09-10T19:08:37.999553+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:58.382907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:07:58.382907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:21.843482+0000","scrub_duration":0.0021153690000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"427'30","reported_seq":590,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.623501+0000","last_change":"2024-09-10T19:07:47.791846+0000","last_active":"2024-09-10T19:08:37.623501+0000","last_peered":"2024-09-10T19:08:37.623501+0000","last_clean":"2024-09-10T19:08:37.623501+0000","last_became_active":"2024-09-10T19:07:18.740753+0000","last_became_peered":"2024-09-10T19:07:18.740753+0000","last_unstale":"2024-09-10T19:08:37.623501+0000","last_undegraded":"2024-09-10T19:08:37.623501+0000","last_fullsized":"2024-09-10T19:08:37.623501+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'30","last_scrub_stamp":"2024-09-10T19:07:47.791781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:07:47.791781+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:12.941691+0000","scrub_duration":0.001737281,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":38,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"464'11","reported_seq":868,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:56.354887+0000","last_change":"2024-09-10T19:08:22.668669+0000","last_active":"2024-09-10T19:08:56.354887+0000","last_peered":"2024-09-10T19:08:56.354887+0000","last_clean":"2024-09-10T19:08:56.354887+0000","last_became_active":"2024-09-10T19:07:18.746575+0000","last_became_peered":"2024-09-10T19:07:18.746575+0000","last_unstale":"2024-09-10T19:08:56.354887+0000","last_undegraded":"2024-09-10T19:08:56.354887+0000","last_fullsized":"2024-09-10T19:08:56.354887+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"464'11","last_scrub_stamp":"2024-09-10T19:08:22.668594+0000","last_deep_scrub":"464'11","last_deep_scrub_stamp":"2024-09-10T19:08:22.668594+0000","last_clean_scrub_stamp":"2024-09-10T19:08:22.668594+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:40.683399+0000","scrub_duration":0.001681697,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":427,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.623509+0000","last_change":"2024-09-10T19:08:21.482218+0000","last_active":"2024-09-10T19:08:37.623509+0000","last_peered":"2024-09-10T19:08:37.623509+0000","last_clean":"2024-09-10T19:08:37.623509+0000","last_became_active":"2024-09-10T19:07:31.169258+0000","last_became_peered":"2024-09-10T19:07:31.169258+0000","last_unstale":"2024-09-10T19:08:37.623509+0000","last_undegraded":"2024-09-10T19:08:37.623509+0000","last_fullsized":"2024-09-10T19:08:37.623509+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:21.482163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:03.475299+0000","last_clean_scrub_stamp":"2024-09-10T19:08:21.482163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:39.197709+0000","scrub_duration":0.001695834,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":531,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.089873+0000","last_change":"2024-09-10T19:07:49.807776+0000","last_active":"2024-09-10T19:08:38.089873+0000","last_peered":"2024-09-10T19:08:38.089873+0000","last_clean":"2024-09-10T19:08:38.089873+0000","last_became_active":"2024-09-10T18:55:40.392524+0000","last_became_peered":"2024-09-10T18:55:40.392524+0000","last_unstale":"2024-09-10T19:08:38.089873+0000","last_undegraded":"2024-09-10T19:08:38.089873+0000","last_fullsized":"2024-09-10T19:08:38.089873+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:49.807719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:07:49.807719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:07.698247+0000","scrub_duration":0.0014287670000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"467'32","reported_seq":596,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.001075+0000","last_change":"2024-09-10T19:08:32.507503+0000","last_active":"2024-09-10T19:08:38.001075+0000","last_peered":"2024-09-10T19:08:38.001075+0000","last_clean":"2024-09-10T19:08:38.001075+0000","last_became_active":"2024-09-10T18:45:20.058717+0000","last_became_peered":"2024-09-10T18:45:20.058717+0000","last_unstale":"2024-09-10T19:08:38.001075+0000","last_undegraded":"2024-09-10T19:08:38.001075+0000","last_fullsized":"2024-09-10T19:08:38.001075+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"467'32","last_scrub_stamp":"2024-09-10T19:08:32.507433+0000","last_deep_scrub":"467'32","last_deep_scrub_stamp":"2024-09-10T19:07:28.508578+0000","last_clean_scrub_stamp":"2024-09-10T19:08:32.507433+0000","objects_scrubbed":5,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:41.206760+0000","scrub_duration":0.0020411029999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":527,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.623617+0000","last_change":"2024-09-10T19:08:12.749647+0000","last_active":"2024-09-10T19:08:37.623617+0000","last_peered":"2024-09-10T19:08:37.623617+0000","last_clean":"2024-09-10T19:08:37.623617+0000","last_became_active":"2024-09-10T19:07:21.854982+0000","last_became_peered":"2024-09-10T19:07:21.854982+0000","last_unstale":"2024-09-10T19:08:37.623617+0000","last_undegraded":"2024-09-10T19:08:37.623617+0000","last_fullsized":"2024-09-10T19:08:37.623617+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:12.749588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:08:12.749588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:35.434829+0000","scrub_duration":0.001727239,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":519,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.625594+0000","last_change":"2024-09-10T19:08:35.992279+0000","last_active":"2024-09-10T19:08:37.625594+0000","last_peered":"2024-09-10T19:08:37.625594+0000","last_clean":"2024-09-10T19:08:37.625594+0000","last_became_active":"2024-09-10T18:45:24.267251+0000","last_became_peered":"2024-09-10T18:45:24.267251+0000","last_unstale":"2024-09-10T19:08:37.625594+0000","last_undegraded":"2024-09-10T19:08:37.625594+0000","last_fullsized":"2024-09-10T19:08:37.625594+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:35.992224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:07:24.058804+0000","last_clean_scrub_stamp":"2024-09-10T19:08:35.992224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:44.279878+0000","scrub_duration":0.0013396269999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"306'2","reported_seq":578,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.000452+0000","last_change":"2024-09-10T19:08:28.601563+0000","last_active":"2024-09-10T19:08:38.000452+0000","last_peered":"2024-09-10T19:08:38.000452+0000","last_clean":"2024-09-10T19:08:38.000452+0000","last_became_active":"2024-09-10T19:07:18.745262+0000","last_became_peered":"2024-09-10T19:07:18.745262+0000","last_unstale":"2024-09-10T19:08:38.000452+0000","last_undegraded":"2024-09-10T19:08:38.000452+0000","last_fullsized":"2024-09-10T19:08:38.000452+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"306'2","last_scrub_stamp":"2024-09-10T19:08:28.601496+0000","last_deep_scrub":"306'2","last_deep_scrub_stamp":"2024-09-10T18:59:20.534869+0000","last_clean_scrub_stamp":"2024-09-10T19:08:28.601496+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:29.281677+0000","scrub_duration":0.0015540160000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1748,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"427'46","reported_seq":75,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:42.258164+0000","last_change":"2024-09-10T19:08:42.258110+0000","last_active":"2024-09-10T19:08:42.258164+0000","last_peered":"2024-09-10T19:08:42.258164+0000","last_clean":"2024-09-10T19:08:42.258164+0000","last_became_active":"2024-09-10T19:07:21.860334+0000","last_became_peered":"2024-09-10T19:07:21.860334+0000","last_unstale":"2024-09-10T19:08:42.258164+0000","last_undegraded":"2024-09-10T19:08:42.258164+0000","last_fullsized":"2024-09-10T19:08:42.258164+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'46","last_scrub_stamp":"2024-09-10T19:08:42.258053+0000","last_deep_scrub":"258'34","last_deep_scrub_stamp":"2024-09-10T18:56:23.838951+0000","last_clean_scrub_stamp":"2024-09-10T19:08:42.258053+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:49.938751+0000","scrub_duration":0.001904918,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":60,"num_read_kb":51,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":436,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:41.552765+0000","last_change":"2024-09-10T19:08:41.552705+0000","last_active":"2024-09-10T19:08:41.552765+0000","last_peered":"2024-09-10T19:08:41.552765+0000","last_clean":"2024-09-10T19:08:41.552765+0000","last_became_active":"2024-09-10T19:07:31.169897+0000","last_became_peered":"2024-09-10T19:07:31.169897+0000","last_unstale":"2024-09-10T19:08:41.552765+0000","last_undegraded":"2024-09-10T19:08:41.552765+0000","last_fullsized":"2024-09-10T19:08:41.552765+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:41.552642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:08:41.552642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:51.019574+0000","scrub_duration":0.0017133509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":528,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.091130+0000","last_change":"2024-09-10T19:07:36.612678+0000","last_active":"2024-09-10T19:08:38.091130+0000","last_peered":"2024-09-10T19:08:38.091130+0000","last_clean":"2024-09-10T19:08:38.091130+0000","last_became_active":"2024-09-10T19:00:42.520239+0000","last_became_peered":"2024-09-10T19:00:42.520239+0000","last_unstale":"2024-09-10T19:08:38.091130+0000","last_undegraded":"2024-09-10T19:08:38.091130+0000","last_fullsized":"2024-09-10T19:08:38.091130+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:36.612617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:17.985545+0000","last_clean_scrub_stamp":"2024-09-10T19:07:36.612617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:05.452711+0000","scrub_duration":0.0016732839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":554,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090458+0000","last_change":"2024-09-10T19:07:58.917961+0000","last_active":"2024-09-10T19:08:38.090458+0000","last_peered":"2024-09-10T19:08:38.090458+0000","last_clean":"2024-09-10T19:08:38.090458+0000","last_became_active":"2024-09-10T19:07:58.917683+0000","last_became_peered":"2024-09-10T19:07:58.917683+0000","last_unstale":"2024-09-10T19:08:38.090458+0000","last_undegraded":"2024-09-10T19:08:38.090458+0000","last_fullsized":"2024-09-10T19:08:38.090458+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:33.551339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:07:33.551339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:08:53.686595+0000","scrub_duration":0.0018062569999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"427'29","reported_seq":558,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.091086+0000","last_change":"2024-09-10T19:07:45.772541+0000","last_active":"2024-09-10T19:08:38.091086+0000","last_peered":"2024-09-10T19:08:38.091086+0000","last_clean":"2024-09-10T19:08:38.091086+0000","last_became_active":"2024-09-10T18:45:20.058391+0000","last_became_peered":"2024-09-10T18:45:20.058391+0000","last_unstale":"2024-09-10T19:08:38.091086+0000","last_undegraded":"2024-09-10T19:08:38.091086+0000","last_fullsized":"2024-09-10T19:08:38.091086+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'29","last_scrub_stamp":"2024-09-10T19:07:45.772483+0000","last_deep_scrub":"388'23","last_deep_scrub_stamp":"2024-09-10T19:03:40.697702+0000","last_clean_scrub_stamp":"2024-09-10T19:07:45.772483+0000","objects_scrubbed":5,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:04.451769+0000","scrub_duration":0.0019090470000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":571,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:57.955353+0000","last_change":"2024-09-10T19:08:57.955316+0000","last_active":"2024-09-10T19:08:57.955353+0000","last_peered":"2024-09-10T19:08:57.955353+0000","last_clean":"2024-09-10T19:08:57.955353+0000","last_became_active":"2024-09-10T19:07:21.863857+0000","last_became_peered":"2024-09-10T19:07:21.863857+0000","last_unstale":"2024-09-10T19:08:57.955353+0000","last_undegraded":"2024-09-10T19:08:57.955353+0000","last_fullsized":"2024-09-10T19:08:57.955353+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:57.955246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:08:57.955246+0000","last_clean_scrub_stamp":"2024-09-10T19:08:57.955246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:09.298496+0000","scrub_duration":0.0019214060000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":803,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1539,"num_read_kb":1296,"num_write":989,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":329,"num_bytes_recovered":4736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1082,"ondisk_log_size":1082,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1121,"num_read_kb":1116,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":525376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1701,"ondisk_log_size":1701,"up":63,"acting":63,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":411,"seq":1765231558708,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":320480,"kb_used_data":728,"kb_used_omap":4,"kb_used_meta":319739,"kb_avail":93424672,"statfs":{"total":95995035648,"available":95666864128,"internally_reserved":0,"allocated":745472,"data_stored":369368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":327413254},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":6.1959999999999997,"15min":8.641},"min":{"1min":0.34000000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.77500000000000002,"5min":100.34,"15min":100.34},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.94699999999999995,"5min":6.4859999999999998,"15min":8.9469999999999992},"min":{"1min":0.378,"5min":0.245,"15min":0.245},"max":{"1min":8.3339999999999996,"5min":101.623,"15min":101.623},"last":0.434}]},{"osd":1,"last update":"Tue Sep 10 19:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":5.9930000000000003,"15min":8.3829999999999991},"min":{"1min":0.32300000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.73299999999999998,"5min":95.744,"15min":95.744},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.50600000000000001,"5min":6.0190000000000001,"15min":8.4309999999999992},"min":{"1min":0.371,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.69599999999999995,"5min":95.709999999999994,"15min":95.709999999999994},"last":0.377}]},{"osd":2,"last update":"Tue Sep 10 19:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.63800000000000001,"15min":0.56200000000000006},"min":{"1min":0.35499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":8.3840000000000003,"5min":8.3840000000000003,"15min":8.3840000000000003},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.56499999999999995,"15min":0.48199999999999998},"min":{"1min":0.38700000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72099999999999997,"5min":7.625,"15min":7.625},"last":0.497}]},{"osd":3,"last update":"Tue Sep 10 19:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":6.5529999999999999,"15min":9.1579999999999995},"min":{"1min":0.45000000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.751,"5min":96.021000000000001,"15min":96.021000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":6.5620000000000003,"15min":9.1539999999999999},"min":{"1min":0.35999999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77600000000000002,"5min":96,"15min":96},"last":0.55200000000000005}]},{"osd":4,"last update":"Tue Sep 10 19:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":2.2789999999999999,"15min":3.0470000000000002},"min":{"1min":0.28999999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.67200000000000004,"5min":66.769999999999996,"15min":66.769999999999996},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.48899999999999999,"5min":2.1930000000000001,"15min":2.9420000000000002},"min":{"1min":0.308,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.754,"5min":67.028999999999996,"15min":67.028999999999996},"last":0.58299999999999996}]},{"osd":5,"last update":"Tue Sep 10 19:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":2.1829999999999998,"15min":2.9180000000000001},"min":{"1min":0.32800000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.73399999999999999,"5min":67.450999999999993,"15min":67.450999999999993},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":2.2450000000000001,"15min":2.9740000000000002},"min":{"1min":0.34100000000000003,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72199999999999998,"5min":66.850999999999999,"15min":66.850999999999999},"last":0.56699999999999995}]},{"osd":6,"last update":"Tue Sep 10 19:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.63700000000000001,"15min":0.65200000000000002},"min":{"1min":0.34599999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.67800000000000005,"5min":2.5579999999999998,"15min":2.5579999999999998},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.69599999999999995,"15min":0.70199999999999996},"min":{"1min":0.40000000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.80000000000000004,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.63300000000000001}]}]},{"osd":6,"up_from":43,"seq":184683594023,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":895188,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":894142,"kb_avail":92849964,"statfs":{"total":95995035648,"available":95078363136,"internally_reserved":0,"allocated":1060864,"data_stored":697124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":915601752},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:08:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":1.9259999999999999,"15min":1.6799999999999999},"min":{"1min":0.34999999999999998,"5min":0.217,"15min":0.20899999999999999},"max":{"1min":0.70399999999999996,"5min":44.320999999999998,"15min":49.353999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.54800000000000004,"5min":2.0790000000000002,"15min":1.8080000000000001},"min":{"1min":0.33900000000000002,"5min":0.23799999999999999,"15min":0.20699999999999999},"max":{"1min":0.71599999999999997,"5min":44.311999999999998,"15min":49.341999999999999},"last":0.51900000000000002}]},{"osd":1,"last update":"Tue Sep 10 19:08:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":1.74,"15min":1.655},"min":{"1min":0.30499999999999999,"5min":0.23200000000000001,"15min":0.217},"max":{"1min":0.66300000000000003,"5min":85.775000000000006,"15min":85.775000000000006},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":1.639,"15min":1.552},"min":{"1min":0.34100000000000003,"5min":0.27100000000000002,"15min":0.23300000000000001},"max":{"1min":0.71399999999999997,"5min":85.670000000000002,"15min":85.670000000000002},"last":0.48899999999999999}]},{"osd":2,"last update":"Tue Sep 10 19:08:21 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.71699999999999997,"15min":0.81799999999999995},"min":{"1min":0.32000000000000001,"5min":0.27000000000000002,"15min":0.26000000000000001},"max":{"1min":0.63700000000000001,"5min":4.0469999999999997,"15min":23.867000000000001},"last":12.194000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.81299999999999994,"15min":0.86899999999999999},"min":{"1min":0.42799999999999999,"5min":0.28699999999999998,"15min":0.249},"max":{"1min":0.70199999999999996,"5min":7.8200000000000003,"15min":23.779},"last":0.42799999999999999}]},{"osd":3,"last update":"Tue Sep 10 19:08:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":1.042,"15min":1.6899999999999999},"min":{"1min":0.375,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.73399999999999999,"5min":20.994,"15min":76.977999999999994},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":1.0660000000000001,"15min":1.702},"min":{"1min":0.39100000000000001,"5min":0.25600000000000001,"15min":0.253},"max":{"1min":0.72699999999999998,"5min":20.177,"15min":76.879999999999995},"last":0.56399999999999995}]},{"osd":4,"last update":"Tue Sep 10 19:08:21 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.57099999999999995,"15min":1.478},"min":{"1min":0.32300000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.58999999999999997,"5min":2.746,"15min":165.03299999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.48899999999999999,"15min":1.337},"min":{"1min":0.32900000000000001,"5min":0.252,"15min":0.22500000000000001},"max":{"1min":0.67900000000000005,"5min":3.0819999999999999,"15min":163.959},"last":0.55200000000000005}]},{"osd":5,"last update":"Tue Sep 10 19:08:21 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.68300000000000005,"15min":1.847},"min":{"1min":0.32800000000000001,"5min":0.26300000000000001,"15min":0.246},"max":{"1min":0.66100000000000003,"5min":2.968,"15min":288.56999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.64200000000000002,"15min":1.8520000000000001},"min":{"1min":0.36899999999999999,"5min":0.25800000000000001,"15min":0.22600000000000001},"max":{"1min":0.67800000000000005,"5min":2.7269999999999999,"15min":291.05399999999997},"last":0.47099999999999997}]},{"osd":7,"last update":"Tue Sep 10 19:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.69599999999999995,"15min":0.70299999999999996},"min":{"1min":0.36199999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.75600000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57699999999999996,"15min":0.59399999999999997},"min":{"1min":0.41599999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.70799999999999996,"5min":1.702,"15min":1.702},"last":0.623}]}]},{"osd":5,"up_from":37,"seq":158913790251,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1185500,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":1184510,"kb_avail":92559652,"statfs":{"total":95995035648,"available":94781083648,"internally_reserved":0,"allocated":1003520,"data_stored":631414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":1212938584},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:08:05 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.97899999999999998,"15min":1.8380000000000001},"min":{"1min":0.28599999999999998,"5min":0.224,"15min":0.224},"max":{"1min":0.91700000000000004,"5min":14.31,"15min":82.465000000000003},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.55900000000000005,"5min":1.081,"15min":1.972},"min":{"1min":0.42899999999999999,"5min":0.27900000000000003,"15min":0.218},"max":{"1min":0.76900000000000002,"5min":14.323,"15min":82.519000000000005},"last":0.64000000000000001}]},{"osd":1,"last update":"Tue Sep 10 19:08:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":2.629,"15min":2.323},"min":{"1min":0.30199999999999999,"5min":0.23300000000000001,"15min":0.19500000000000001},"max":{"1min":0.85499999999999998,"5min":153.17099999999999,"15min":153.17099999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":2.6539999999999999,"15min":2.4900000000000002},"min":{"1min":0.437,"5min":0.29199999999999998,"15min":0.20399999999999999},"max":{"1min":0.72899999999999998,"5min":153.196,"15min":153.196},"last":0.53600000000000003}]},{"osd":2,"last update":"Tue Sep 10 19:08:05 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.61399999999999999,"15min":0.89800000000000002},"min":{"1min":0.32900000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.75800000000000001,"5min":3.0990000000000002,"15min":31.774000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.54500000000000004,"15min":0.85799999999999998},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.24399999999999999},"max":{"1min":0.73599999999999999,"5min":2.8799999999999999,"15min":31.722999999999999},"last":0.50600000000000001}]},{"osd":3,"last update":"Tue Sep 10 19:08:05 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.78400000000000003,"15min":2.492},"min":{"1min":0.36499999999999999,"5min":0.26300000000000001,"15min":0.24299999999999999},"max":{"1min":0.83299999999999996,"5min":3.0760000000000001,"15min":123.34399999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.79200000000000004,"15min":2.0339999999999998},"min":{"1min":0.40600000000000003,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.95499999999999996,"5min":5.117,"15min":82.415000000000006},"last":0.61299999999999999}]},{"osd":4,"last update":"Tue Sep 10 19:08:05 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.63800000000000001,"15min":1.1719999999999999},"min":{"1min":0.309,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.72399999999999998,"5min":5.0099999999999998,"15min":51.070999999999998},"last":0.434},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.57299999999999995,"15min":1.0549999999999999},"min":{"1min":0.27000000000000002,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.55600000000000005,"5min":3.524,"15min":51.122999999999998},"last":0.59699999999999998}]},{"osd":6,"last update":"Tue Sep 10 19:08:27 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.67400000000000004,"15min":1.0960000000000001},"min":{"1min":0.31900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.83899999999999997,"5min":4.0490000000000004,"15min":33.936999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.66000000000000003,"15min":1.0449999999999999},"min":{"1min":0.379,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.081,"5min":5.1710000000000003,"15min":33.951000000000001},"last":0.628}]},{"osd":7,"last update":"Tue Sep 10 19:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.59499999999999997,"15min":0.66400000000000003},"min":{"1min":0.34399999999999997,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.63600000000000001,"5min":2.657,"15min":2.657},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.66600000000000004,"15min":0.69999999999999996},"min":{"1min":0.36299999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67100000000000004,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":0.68300000000000005}]}]},{"osd":4,"up_from":30,"seq":128849019185,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1183588,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":1182846,"kb_avail":92561564,"statfs":{"total":95995035648,"available":94783041536,"internally_reserved":0,"allocated":749568,"data_stored":369219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":1211234648},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:08:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":1.2529999999999999,"15min":1.3839999999999999},"min":{"1min":0.33200000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.81899999999999995,"5min":40.481000000000002,"15min":46.234999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":1.296,"15min":1.3859999999999999},"min":{"1min":0.40799999999999997,"5min":0.253,"15min":0.253},"max":{"1min":0.91600000000000004,"5min":40.512,"15min":46.264000000000003},"last":0.439}]},{"osd":1,"last update":"Tue Sep 10 19:08:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":1.198,"15min":1.677},"min":{"1min":0.29599999999999999,"5min":0.23999999999999999,"15min":0.20899999999999999},"max":{"1min":0.69999999999999996,"5min":16.535,"15min":125.15600000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.998,"15min":1.5720000000000001},"min":{"1min":0.38100000000000001,"5min":0.24099999999999999,"15min":0.23100000000000001},"max":{"1min":0.73899999999999999,"5min":16.504000000000001,"15min":125.14700000000001},"last":7.6929999999999996}]},{"osd":2,"last update":"Tue Sep 10 19:08:42 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":0.74099999999999999,"15min":0.73199999999999998},"min":{"1min":0.38300000000000001,"5min":0.28699999999999998,"15min":0.25900000000000001},"max":{"1min":11.589,"5min":11.589,"15min":11.589},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":2.6560000000000001,"15min":1.4570000000000001},"min":{"1min":0.317,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.65900000000000003,"5min":190.38200000000001,"15min":190.38200000000001},"last":0.41699999999999998}]},{"osd":3,"last update":"Tue Sep 10 19:08:42 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":3.8799999999999999,"15min":2.98},"min":{"1min":0.35499999999999998,"5min":0.27200000000000002,"15min":0.249},"max":{"1min":0.67900000000000005,"5min":190.41300000000001,"15min":190.41300000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":1.7150000000000001,"15min":1.8999999999999999},"min":{"1min":0.34499999999999997,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.68200000000000005,"5min":39.323,"15min":110.914},"last":0.51400000000000001}]},{"osd":5,"last update":"Tue Sep 10 19:08:04 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.629,"15min":0.93600000000000005},"min":{"1min":0.26300000000000001,"5min":0.221,"15min":0.20499999999999999},"max":{"1min":0.623,"5min":8.1430000000000007,"15min":10.912000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.63500000000000001,"15min":0.91500000000000004},"min":{"1min":0.315,"5min":0.23200000000000001,"15min":0.216},"max":{"1min":0.96399999999999997,"5min":8.3170000000000002,"15min":10.147},"last":0.34599999999999997}]},{"osd":6,"last update":"Tue Sep 10 19:08:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58399999999999996,"15min":0.79400000000000004},"min":{"1min":0.33400000000000002,"5min":0.24099999999999999,"15min":0.221},"max":{"1min":0.86499999999999999,"5min":2.4100000000000001,"15min":8.4030000000000005},"last":0.503},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.66600000000000004,"15min":0.95299999999999996},"min":{"1min":0.38500000000000001,"5min":0.26300000000000001,"15min":0.23899999999999999},"max":{"1min":0.85099999999999998,"5min":4.0099999999999998,"15min":21.254000000000001},"last":0.55200000000000005}]},{"osd":7,"last update":"Tue Sep 10 19:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57899999999999996,"15min":0.56499999999999995},"min":{"1min":0.35999999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.872,"5min":3.472,"15min":3.472},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.48899999999999999,"5min":6.1909999999999998,"15min":8.6349999999999998},"min":{"1min":0.30199999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.74199999999999999,"5min":190.33799999999999,"15min":190.33799999999999},"last":0.48599999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182710,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1502576,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":1501246,"kb_avail":92242576,"statfs":{"total":95995035648,"available":94456397824,"internally_reserved":0,"allocated":1351680,"data_stored":959051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":1537276248},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:08:03 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":1.0509999999999999,"15min":2.343},"min":{"1min":0.221,"5min":0.182,"15min":0.17000000000000001},"max":{"1min":0.64600000000000002,"5min":6.3099999999999996,"15min":290.721},"last":0.438},{"interface":"front","average":{"1min":0.376,"5min":1.212,"15min":2.4980000000000002},"min":{"1min":0.23799999999999999,"5min":0.18099999999999999,"15min":0.17499999999999999},"max":{"1min":0.63200000000000001,"5min":12.103,"15min":293.59300000000002},"last":0.53100000000000003}]},{"osd":1,"last update":"Tue Sep 10 19:08:15 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":1.373,"15min":1.4299999999999999},"min":{"1min":0.246,"5min":0.18099999999999999,"15min":0.16200000000000001},"max":{"1min":0.82499999999999996,"5min":16.081,"15min":16.081},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.44400000000000001,"5min":1.514,"15min":1.361},"min":{"1min":0.28699999999999998,"5min":0.25600000000000001,"15min":0.188},"max":{"1min":0.86699999999999999,"5min":16.329000000000001,"15min":16.329000000000001},"last":0.48699999999999999}]},{"osd":2,"last update":"Tue Sep 10 19:08:15 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.83699999999999997,"15min":0.93300000000000005},"min":{"1min":0.29999999999999999,"5min":0.26200000000000001,"15min":0.22800000000000001},"max":{"1min":0.64200000000000002,"5min":4.2409999999999997,"15min":5.6680000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.82599999999999996,"15min":0.90100000000000002},"min":{"1min":0.251,"5min":0.24099999999999999,"15min":0.20599999999999999},"max":{"1min":0.57199999999999995,"5min":5.0250000000000004,"15min":5.46},"last":0.503}]},{"osd":4,"last update":"Tue Sep 10 19:08:48 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":1.044,"15min":1.3460000000000001},"min":{"1min":0.437,"5min":0.29699999999999999,"15min":0.247},"max":{"1min":0.90200000000000002,"5min":4.4580000000000002,"15min":29.013000000000002},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.65100000000000002,"5min":1.0029999999999999,"15min":1.369},"min":{"1min":0.44,"5min":0.30499999999999999,"15min":0.26800000000000002},"max":{"1min":0.83999999999999997,"5min":4.3760000000000003,"15min":28.826000000000001},"last":0.63600000000000001}]},{"osd":5,"last update":"Tue Sep 10 19:08:03 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":1.167,"15min":1.286},"min":{"1min":0.308,"5min":0.248,"15min":0.248},"max":{"1min":0.84199999999999997,"5min":5.1920000000000002,"15min":7.2300000000000004},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.628,"5min":1.2609999999999999,"15min":1.3320000000000001},"min":{"1min":0.42299999999999999,"5min":0.308,"15min":0.255},"max":{"1min":0.85099999999999998,"5min":13.117000000000001,"15min":17.899999999999999},"last":0.69099999999999995}]},{"osd":6,"last update":"Tue Sep 10 19:08:23 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":1.0840000000000001,"15min":1.2509999999999999},"min":{"1min":0.42499999999999999,"5min":0.307,"15min":0.30399999999999999},"max":{"1min":0.84899999999999998,"5min":5.8339999999999996,"15min":5.8339999999999996},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":1.0549999999999999,"15min":1.222},"min":{"1min":0.46100000000000002,"5min":0.32200000000000001,"15min":0.29399999999999998},"max":{"1min":0.82799999999999996,"5min":5.1340000000000003,"15min":6.6390000000000002},"last":0.71699999999999997}]},{"osd":7,"last update":"Tue Sep 10 19:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":1.012,"15min":1.278},"min":{"1min":0.44400000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.96599999999999997,"5min":4.8499999999999996,"15min":4.8499999999999996},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":1.333,"15min":1.9350000000000001},"min":{"1min":0.46000000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.1910000000000001,"5min":13.045999999999999,"15min":13.045999999999999},"last":0.73199999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411642,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":302484,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93442668,"statfs":{"total":95995035648,"available":95685292032,"internally_reserved":0,"allocated":995328,"data_stored":631385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:08:00 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":2.0569999999999999,"15min":1.3069999999999999},"min":{"1min":0.19900000000000001,"5min":0.189,"15min":0.17699999999999999},"max":{"1min":0.54900000000000004,"5min":64.134,"15min":64.134},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":2.1850000000000001,"15min":1.4710000000000001},"min":{"1min":0.245,"5min":0.192,"15min":0.192},"max":{"1min":0.59399999999999997,"5min":64.165999999999997,"15min":64.165999999999997},"last":0.438}]},{"osd":1,"last update":"Tue Sep 10 19:08:00 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":1.839,"15min":1.5920000000000001},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.186},"max":{"1min":0.51500000000000001,"5min":65.606999999999999,"15min":65.606999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":2.048,"15min":1.645},"min":{"1min":0.25900000000000001,"5min":0.21099999999999999,"15min":0.20000000000000001},"max":{"1min":0.67300000000000004,"5min":68.674000000000007,"15min":68.674000000000007},"last":0.378}]},{"osd":3,"last update":"Tue Sep 10 19:08:27 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":2.54,"15min":2.165},"min":{"1min":0.249,"5min":0.249,"15min":0.188},"max":{"1min":0.88600000000000001,"5min":65.207999999999998,"15min":164.47},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.42099999999999999,"5min":2.544,"15min":2.145},"min":{"1min":0.28999999999999998,"5min":0.251,"15min":0.21099999999999999},"max":{"1min":0.64200000000000002,"5min":70.965000000000003,"15min":164.321},"last":0.47199999999999998}]},{"osd":4,"last update":"Tue Sep 10 19:08:48 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.73099999999999998,"15min":1.7909999999999999},"min":{"1min":0.44700000000000001,"5min":0.29099999999999998,"15min":0.23499999999999999},"max":{"1min":0.77500000000000002,"5min":4.4279999999999999,"15min":167.86199999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.58599999999999997,"15min":1.675},"min":{"1min":0.40999999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.86199999999999999,"5min":3.3809999999999998,"15min":166.92699999999999},"last":0.58399999999999996}]},{"osd":5,"last update":"Tue Sep 10 19:08:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":3.0499999999999998,"15min":1.6319999999999999},"min":{"1min":0.372,"5min":0.27700000000000002,"15min":0.26100000000000001},"max":{"1min":0.73799999999999999,"5min":228.25399999999999,"15min":228.25399999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.54800000000000004,"5min":3.0070000000000001,"15min":1.6479999999999999},"min":{"1min":0.35999999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.65800000000000003,"5min":227.78100000000001,"15min":227.78100000000001},"last":0.63600000000000001}]},{"osd":6,"last update":"Tue Sep 10 19:08:48 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60999999999999999,"15min":0.997},"min":{"1min":0.436,"5min":0.32000000000000001,"15min":0.29899999999999999},"max":{"1min":0.85099999999999998,"5min":3.423,"15min":73.640000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.67100000000000004,"15min":0.99299999999999999},"min":{"1min":0.39500000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.88,"5min":3.2839999999999998,"15min":73.269999999999996},"last":0.62}]},{"osd":7,"last update":"Tue Sep 10 19:08:56 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.67500000000000004,"15min":0.69899999999999995},"min":{"1min":0.38700000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.86299999999999999,"5min":3.5179999999999998,"15min":3.5179999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65800000000000003,"15min":0.68000000000000005},"min":{"1min":0.47199999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89300000000000002,"5min":3.4609999999999999,"15min":3.4609999999999999},"last":0.69799999999999995}]}]},{"osd":1,"up_from":13,"seq":55834575166,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1505076,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":1504318,"kb_avail":92240076,"statfs":{"total":95995035648,"available":94453837824,"internally_reserved":0,"allocated":765952,"data_stored":369273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":1540421976},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:08:06 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.94299999999999995,"15min":1.756},"min":{"1min":0.215,"5min":0.191,"15min":0.158},"max":{"1min":0.55900000000000005,"5min":7.3840000000000003,"15min":111.014},"last":0.307},{"interface":"front","average":{"1min":0.373,"5min":1.1020000000000001,"15min":1.839},"min":{"1min":0.28399999999999997,"5min":0.23799999999999999,"15min":0.19600000000000001},"max":{"1min":0.51200000000000001,"5min":9.7360000000000007,"15min":111.072},"last":0.33100000000000002}]},{"osd":2,"last update":"Tue Sep 10 19:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.71899999999999997,"15min":0.751},"min":{"1min":0.26600000000000001,"5min":0.214,"15min":0.186},"max":{"1min":0.71199999999999997,"5min":4.835,"15min":4.835},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.82199999999999995,"15min":0.79800000000000004},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.215},"max":{"1min":0.73199999999999998,"5min":9.484,"15min":9.484},"last":0.34799999999999998}]},{"osd":3,"last update":"Tue Sep 10 19:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":1.9299999999999999,"15min":2.2490000000000001},"min":{"1min":0.249,"5min":0.218,"15min":0.185},"max":{"1min":0.59399999999999997,"5min":48.582999999999998,"15min":92.069000000000003},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.33300000000000002,"5min":1.5720000000000001,"15min":1.8919999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.19900000000000001},"max":{"1min":0.47099999999999997,"5min":51.847000000000001,"15min":88.658000000000001},"last":0.25600000000000001}]},{"osd":4,"last update":"Tue Sep 10 19:08:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.98099999999999998,"15min":1.0820000000000001},"min":{"1min":0.35099999999999998,"5min":0.28100000000000003,"15min":0.27800000000000002},"max":{"1min":0.84899999999999998,"5min":7.2699999999999996,"15min":19.126999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":1.071,"15min":1.1399999999999999},"min":{"1min":0.38700000000000001,"5min":0.26300000000000001,"15min":0.252},"max":{"1min":1.2430000000000001,"5min":5.9340000000000002,"15min":20.152000000000001},"last":0.39300000000000002}]},{"osd":5,"last update":"Tue Sep 10 19:08:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":1.1000000000000001,"15min":1.2250000000000001},"min":{"1min":0.44500000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72599999999999998,"5min":9.9369999999999994,"15min":10.334},"last":0.439},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.88200000000000001,"15min":1.0669999999999999},"min":{"1min":0.371,"5min":0.29199999999999998,"15min":0.28799999999999998},"max":{"1min":0.79500000000000004,"5min":7.351,"15min":7.9969999999999999},"last":0.48799999999999999}]},{"osd":6,"last update":"Tue Sep 10 19:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":1.097,"15min":1.417},"min":{"1min":0.44400000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80200000000000005,"5min":9.9760000000000009,"15min":33.290999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.624,"5min":1.008,"15min":1.3759999999999999},"min":{"1min":0.40300000000000002,"5min":0.32700000000000001,"15min":0.26600000000000001},"max":{"1min":1.2529999999999999,"5min":7.7400000000000002,"15min":33.310000000000002},"last":0.40600000000000003}]},{"osd":7,"last update":"Tue Sep 10 19:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":1.0820000000000001,"15min":1.3100000000000001},"min":{"1min":0.51600000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80100000000000005,"5min":5.9889999999999999,"15min":5.9889999999999999},"last":0.496},{"interface":"front","average":{"1min":0.63400000000000001,"5min":1.2110000000000001,"15min":1.659},"min":{"1min":0.441,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82599999999999996,"5min":10.010999999999999,"15min":10.010999999999999},"last":0.57899999999999996}]}]},{"osd":0,"up_from":241,"seq":1035087118498,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":930776,"kb_used_data":1040,"kb_used_omap":4,"kb_used_meta":929723,"kb_avail":92814376,"statfs":{"total":95995035648,"available":95041921024,"internally_reserved":0,"allocated":1064960,"data_stored":695858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":952036870},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 19:08:05 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":1.585,"15min":1.8109999999999999},"min":{"1min":0.217,"5min":0.18099999999999999,"15min":0.17100000000000001},"max":{"1min":0.434,"5min":47.237000000000002,"15min":47.237000000000002},"last":0.31},{"interface":"front","average":{"1min":0.376,"5min":1.4810000000000001,"15min":1.758},"min":{"1min":0.26900000000000002,"5min":0.20100000000000001,"15min":0.193},"max":{"1min":0.57499999999999996,"5min":47.198999999999998,"15min":47.198999999999998},"last":0.34100000000000003}]},{"osd":2,"last update":"Tue Sep 10 19:08:05 2024","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.92600000000000005,"15min":0.92700000000000005},"min":{"1min":0.247,"5min":0.223,"15min":0.20699999999999999},"max":{"1min":0.52400000000000002,"5min":17.715,"15min":17.715},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.91500000000000004,"15min":0.88800000000000001},"min":{"1min":0.23300000000000001,"5min":0.22900000000000001,"15min":0.22800000000000001},"max":{"1min":0.48399999999999999,"5min":17.809000000000001,"15min":17.809000000000001},"last":0.39700000000000002}]},{"osd":3,"last update":"Tue Sep 10 19:08:05 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":1.208,"15min":1.2310000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.216},"max":{"1min":0.56299999999999994,"5min":20.454999999999998,"15min":21.108000000000001},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.33800000000000002,"5min":1.3,"15min":1.552},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.497,"5min":18.728999999999999,"15min":18.728999999999999},"last":0.32500000000000001}]},{"osd":4,"last update":"Tue Sep 10 19:08:05 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":1.091,"15min":1.0229999999999999},"min":{"1min":0.38600000000000001,"5min":0.30499999999999999,"15min":0.26000000000000001},"max":{"1min":0.78400000000000003,"5min":17.992000000000001,"15min":17.992000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":1.0620000000000001,"15min":1.087},"min":{"1min":0.41899999999999998,"5min":0.28100000000000003,"15min":0.27600000000000002},"max":{"1min":0.747,"5min":18.172999999999998,"15min":18.172999999999998},"last":7.7869999999999999}]},{"osd":5,"last update":"Tue Sep 10 19:08:05 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":1.0629999999999999,"15min":1.5089999999999999},"min":{"1min":0.40699999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.79200000000000004,"5min":18.151,"15min":97.296999999999997},"last":7.8120000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":1.119,"15min":1.4279999999999999},"min":{"1min":0.377,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.76100000000000001,"5min":18.016999999999999,"15min":99.221999999999994},"last":0.67900000000000005}]},{"osd":6,"last update":"Tue Sep 10 19:08:05 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":2.7879999999999998,"15min":1.7509999999999999},"min":{"1min":0.38900000000000001,"5min":0.33600000000000002,"15min":0.312},"max":{"1min":0.86199999999999999,"5min":216.21100000000001,"15min":216.21100000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.57199999999999995,"5min":2.831,"15min":1.7250000000000001},"min":{"1min":0.39600000000000002,"5min":0.379,"15min":0.313},"max":{"1min":0.75600000000000001,"5min":216.054,"15min":216.054},"last":0.71199999999999997}]},{"osd":7,"last update":"Tue Sep 10 19:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":1.149,"15min":1.077},"min":{"1min":0.40400000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.74199999999999999,"5min":20.417999999999999,"15min":20.417999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":1.3979999999999999,"15min":1.502},"min":{"1min":0.44500000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.73999999999999999,"5min":18.059000000000001,"15min":18.059000000000001},"last":0.73999999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T19:09:05.432 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T19:09:05.693 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:09:06.068 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:05 smithi018 ceph-mon[40196]: from='client.26168 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:09:06.069 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:05 smithi018 ceph-mon[40196]: 4.11 deep-scrub starts 2024-09-10T19:09:06.069 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:05 smithi018 ceph-mon[40196]: 4.11 deep-scrub ok 2024-09-10T19:09:06.069 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:05 smithi018 ceph-mon[40196]: 6.19 scrub starts 2024-09-10T19:09:06.069 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:05 smithi018 ceph-mon[40196]: 6.19 scrub ok 2024-09-10T19:09:06.069 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:05 smithi018 ceph-mon[40196]: pgmap v1124: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:09:06.069 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:05 smithi018 ceph-mon[29336]: from='client.26168 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:09:06.069 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:05 smithi018 ceph-mon[29336]: 4.11 deep-scrub starts 2024-09-10T19:09:06.069 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:05 smithi018 ceph-mon[29336]: 4.11 deep-scrub ok 2024-09-10T19:09:06.069 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:05 smithi018 ceph-mon[29336]: 6.19 scrub starts 2024-09-10T19:09:06.069 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:05 smithi018 ceph-mon[29336]: 6.19 scrub ok 2024-09-10T19:09:06.070 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:05 smithi018 ceph-mon[29336]: pgmap v1124: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:09:06.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:05 smithi103 ceph-mon[33163]: from='client.26168 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:09:06.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:05 smithi103 ceph-mon[33163]: 4.11 deep-scrub starts 2024-09-10T19:09:06.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:05 smithi103 ceph-mon[33163]: 4.11 deep-scrub ok 2024-09-10T19:09:06.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:05 smithi103 ceph-mon[33163]: 6.19 scrub starts 2024-09-10T19:09:06.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:05 smithi103 ceph-mon[33163]: 6.19 scrub ok 2024-09-10T19:09:06.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:05 smithi103 ceph-mon[33163]: pgmap v1124: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:09:07.093 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:06 smithi018 ceph-mon[29336]: 4.2 scrub starts 2024-09-10T19:09:07.093 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:06 smithi018 ceph-mon[29336]: 4.2 scrub ok 2024-09-10T19:09:07.093 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:06 smithi018 ceph-mon[29336]: 6.13 scrub starts 2024-09-10T19:09:07.094 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:06 smithi018 ceph-mon[29336]: 6.13 scrub ok 2024-09-10T19:09:07.094 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:06 smithi018 ceph-mon[40196]: 4.2 scrub starts 2024-09-10T19:09:07.094 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:06 smithi018 ceph-mon[40196]: 4.2 scrub ok 2024-09-10T19:09:07.094 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:06 smithi018 ceph-mon[40196]: 6.13 scrub starts 2024-09-10T19:09:07.094 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:06 smithi018 ceph-mon[40196]: 6.13 scrub ok 2024-09-10T19:09:07.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:06 smithi103 ceph-mon[33163]: 4.2 scrub starts 2024-09-10T19:09:07.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:06 smithi103 ceph-mon[33163]: 4.2 scrub ok 2024-09-10T19:09:07.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:06 smithi103 ceph-mon[33163]: 6.13 scrub starts 2024-09-10T19:09:07.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:06 smithi103 ceph-mon[33163]: 6.13 scrub ok 2024-09-10T19:09:07.487 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-10T19:09:07.489 INFO:teuthology.orchestra.run.smithi018.stderr:dumped all 2024-09-10T19:09:07.772 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:09:07 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug there is no tcmu-runner data available 2024-09-10T19:09:08.186 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:07 smithi018 ceph-mon[29336]: 5.18 scrub starts 2024-09-10T19:09:08.186 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:07 smithi018 ceph-mon[29336]: 5.18 scrub ok 2024-09-10T19:09:08.186 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:07 smithi018 ceph-mon[29336]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:09:08.186 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:07 smithi018 ceph-mon[29336]: from='client.16281 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:09:08.186 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:07 smithi018 ceph-mon[29336]: pgmap v1125: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:09:08.187 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:07 smithi018 ceph-mon[40196]: 5.18 scrub starts 2024-09-10T19:09:08.187 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:07 smithi018 ceph-mon[40196]: 5.18 scrub ok 2024-09-10T19:09:08.187 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:07 smithi018 ceph-mon[40196]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:09:08.187 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:07 smithi018 ceph-mon[40196]: from='client.16281 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:09:08.187 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:07 smithi018 ceph-mon[40196]: pgmap v1125: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:09:08.211 INFO:teuthology.orchestra.run.smithi018.stdout:{"pg_ready":true,"pg_map":{"version":1124,"stamp":"2024-09-10T19:09:05.558388+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":2689,"num_read_kb":2441,"num_write":1002,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":358,"num_bytes_recovered":533960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2894,"ondisk_log_size":2894,"up":456,"acting":456,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":456,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":7825668,"kb_used_data":7556,"kb_used_omap":18,"kb_used_meta":7818029,"kb_avail":742135548,"statfs":{"total":767960285184,"available":759946801152,"internally_reserved":0,"allocated":7737344,"data_stored":4722692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19428,"internal_metadata":8005661724},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003360"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":538,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090040+0000","last_change":"2024-09-10T19:08:03.909476+0000","last_active":"2024-09-10T19:08:38.090040+0000","last_peered":"2024-09-10T19:08:38.090040+0000","last_clean":"2024-09-10T19:08:38.090040+0000","last_became_active":"2024-09-10T19:07:18.744367+0000","last_became_peered":"2024-09-10T19:07:18.744367+0000","last_unstale":"2024-09-10T19:08:38.090040+0000","last_undegraded":"2024-09-10T19:08:38.090040+0000","last_fullsized":"2024-09-10T19:08:38.090040+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:03.909412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:08:03.909412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:17.315981+0000","scrub_duration":0.0038138130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":536,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.623171+0000","last_change":"2024-09-10T19:08:36.825855+0000","last_active":"2024-09-10T19:08:37.623171+0000","last_peered":"2024-09-10T19:08:37.623171+0000","last_clean":"2024-09-10T19:08:37.623171+0000","last_became_active":"2024-09-10T19:07:18.728054+0000","last_became_peered":"2024-09-10T19:07:18.728054+0000","last_unstale":"2024-09-10T19:08:37.623171+0000","last_undegraded":"2024-09-10T19:08:37.623171+0000","last_fullsized":"2024-09-10T19:08:37.623171+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:36.825802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:07:19.562907+0000","last_clean_scrub_stamp":"2024-09-10T19:08:36.825802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:42.666484+0000","scrub_duration":0.0015749329999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"427'39","reported_seq":657,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090101+0000","last_change":"2024-09-10T19:08:24.946416+0000","last_active":"2024-09-10T19:08:38.090101+0000","last_peered":"2024-09-10T19:08:38.090101+0000","last_clean":"2024-09-10T19:08:38.090101+0000","last_became_active":"2024-09-10T19:07:21.856836+0000","last_became_peered":"2024-09-10T19:07:21.856836+0000","last_unstale":"2024-09-10T19:08:38.090101+0000","last_undegraded":"2024-09-10T19:08:38.090101+0000","last_fullsized":"2024-09-10T19:08:38.090101+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'39","last_scrub_stamp":"2024-09-10T19:08:24.946351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:08:24.946351+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:36.731315+0000","scrub_duration":0.0020189969999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":521,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.624648+0000","last_change":"2024-09-10T19:07:49.987352+0000","last_active":"2024-09-10T19:08:37.624648+0000","last_peered":"2024-09-10T19:08:37.624648+0000","last_clean":"2024-09-10T19:08:37.624648+0000","last_became_active":"2024-09-10T18:45:22.065619+0000","last_became_peered":"2024-09-10T18:45:22.065619+0000","last_unstale":"2024-09-10T19:08:37.624648+0000","last_undegraded":"2024-09-10T19:08:37.624648+0000","last_fullsized":"2024-09-10T19:08:37.624648+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:49.987284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:29.057932+0000","last_clean_scrub_stamp":"2024-09-10T19:07:49.987284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:01.961061+0000","scrub_duration":0.0015713820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":591,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.091257+0000","last_change":"2024-09-10T19:08:16.047139+0000","last_active":"2024-09-10T19:08:38.091257+0000","last_peered":"2024-09-10T19:08:38.091257+0000","last_clean":"2024-09-10T19:08:38.091257+0000","last_became_active":"2024-09-10T19:00:42.518824+0000","last_became_peered":"2024-09-10T19:00:42.518824+0000","last_unstale":"2024-09-10T19:08:38.091257+0000","last_undegraded":"2024-09-10T19:08:38.091257+0000","last_fullsized":"2024-09-10T19:08:38.091257+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:16.047073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:08:16.047073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:18.361252+0000","scrub_duration":0.0018722070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"427'25","reported_seq":561,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:58.981990+0000","last_change":"2024-09-10T19:08:58.981933+0000","last_active":"2024-09-10T19:08:58.981990+0000","last_peered":"2024-09-10T19:08:58.981990+0000","last_clean":"2024-09-10T19:08:58.981990+0000","last_became_active":"2024-09-10T18:45:20.061266+0000","last_became_peered":"2024-09-10T18:45:20.061266+0000","last_unstale":"2024-09-10T19:08:58.981990+0000","last_undegraded":"2024-09-10T19:08:58.981990+0000","last_fullsized":"2024-09-10T19:08:58.981990+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'25","last_scrub_stamp":"2024-09-10T19:08:58.981855+0000","last_deep_scrub":"83'13","last_deep_scrub_stamp":"2024-09-10T18:46:59.906634+0000","last_clean_scrub_stamp":"2024-09-10T19:08:58.981855+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:26.746618+0000","scrub_duration":0.0020552090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":177,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:46.605222+0000","last_change":"2024-09-10T19:08:46.605177+0000","last_active":"2024-09-10T19:08:46.605222+0000","last_peered":"2024-09-10T19:08:46.605222+0000","last_clean":"2024-09-10T19:08:46.605222+0000","last_became_active":"2024-09-10T19:07:18.746014+0000","last_became_peered":"2024-09-10T19:07:18.746014+0000","last_unstale":"2024-09-10T19:08:46.605222+0000","last_undegraded":"2024-09-10T19:08:46.605222+0000","last_fullsized":"2024-09-10T19:08:46.605222+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:46.605114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:01.135988+0000","last_clean_scrub_stamp":"2024-09-10T19:08:46.605114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:47.784144+0000","scrub_duration":0.0016228950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,7],"acting":[6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":519,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:51.979591+0000","last_change":"2024-09-10T19:08:51.979558+0000","last_active":"2024-09-10T19:08:51.979591+0000","last_peered":"2024-09-10T19:08:51.979591+0000","last_clean":"2024-09-10T19:08:51.979591+0000","last_became_active":"2024-09-10T18:45:24.081002+0000","last_became_peered":"2024-09-10T18:45:24.081002+0000","last_unstale":"2024-09-10T19:08:51.979591+0000","last_undegraded":"2024-09-10T19:08:51.979591+0000","last_fullsized":"2024-09-10T19:08:51.979591+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:51.979490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:07:29.035184+0000","last_clean_scrub_stamp":"2024-09-10T19:08:51.979490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:57.861037+0000","scrub_duration":0.001735161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":53,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.091015+0000","last_change":"2024-09-10T19:08:23.256410+0000","last_active":"2024-09-10T19:08:38.091015+0000","last_peered":"2024-09-10T19:08:38.091015+0000","last_clean":"2024-09-10T19:08:38.091015+0000","last_became_active":"2024-09-10T19:07:18.744458+0000","last_became_peered":"2024-09-10T19:07:18.744458+0000","last_unstale":"2024-09-10T19:08:38.091015+0000","last_undegraded":"2024-09-10T19:08:38.091015+0000","last_fullsized":"2024-09-10T19:08:38.091015+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:23.256344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:08:23.256344+0000","last_clean_scrub_stamp":"2024-09-10T19:08:23.256344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:34.853431+0000","scrub_duration":0.001568037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"427'11","reported_seq":541,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.625121+0000","last_change":"2024-09-10T19:08:36.962624+0000","last_active":"2024-09-10T19:08:37.625121+0000","last_peered":"2024-09-10T19:08:37.625121+0000","last_clean":"2024-09-10T19:08:37.625121+0000","last_became_active":"2024-09-10T18:45:20.057567+0000","last_became_peered":"2024-09-10T18:45:20.057567+0000","last_unstale":"2024-09-10T19:08:37.625121+0000","last_undegraded":"2024-09-10T19:08:37.625121+0000","last_fullsized":"2024-09-10T19:08:37.625121+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'11","last_scrub_stamp":"2024-09-10T19:08:36.962569+0000","last_deep_scrub":"163'7","last_deep_scrub_stamp":"2024-09-10T18:51:54.261879+0000","last_clean_scrub_stamp":"2024-09-10T19:08:36.962569+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:53.987704+0000","scrub_duration":0.0014505620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":13,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":53,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090500+0000","last_change":"2024-09-10T19:08:04.344276+0000","last_active":"2024-09-10T19:08:38.090500+0000","last_peered":"2024-09-10T19:08:38.090500+0000","last_clean":"2024-09-10T19:08:38.090500+0000","last_became_active":"2024-09-10T19:07:18.740022+0000","last_became_peered":"2024-09-10T19:07:18.740022+0000","last_unstale":"2024-09-10T19:08:38.090500+0000","last_undegraded":"2024-09-10T19:08:38.090500+0000","last_fullsized":"2024-09-10T19:08:38.090500+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:04.344202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:36.509971+0000","last_clean_scrub_stamp":"2024-09-10T19:08:04.344202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:28.179056+0000","scrub_duration":0.001698984,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":186,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.999696+0000","last_change":"2024-09-10T19:07:49.336488+0000","last_active":"2024-09-10T19:08:37.999696+0000","last_peered":"2024-09-10T19:08:37.999696+0000","last_clean":"2024-09-10T19:08:37.999696+0000","last_became_active":"2024-09-10T19:00:42.515840+0000","last_became_peered":"2024-09-10T19:00:42.515840+0000","last_unstale":"2024-09-10T19:08:37.999696+0000","last_undegraded":"2024-09-10T19:08:37.999696+0000","last_fullsized":"2024-09-10T19:08:37.999696+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:49.336430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:07:49.336430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:08:55.574589+0000","scrub_duration":0.0013951339999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":63,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090839+0000","last_change":"2024-09-10T19:08:07.350292+0000","last_active":"2024-09-10T19:08:38.090839+0000","last_peered":"2024-09-10T19:08:38.090839+0000","last_clean":"2024-09-10T19:08:38.090839+0000","last_became_active":"2024-09-10T19:07:21.861354+0000","last_became_peered":"2024-09-10T19:07:21.861354+0000","last_unstale":"2024-09-10T19:08:38.090839+0000","last_undegraded":"2024-09-10T19:08:38.090839+0000","last_fullsized":"2024-09-10T19:08:38.090839+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:07.350220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:08:07.350220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:20.162947+0000","scrub_duration":0.0015941410000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"427'21","reported_seq":586,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.625067+0000","last_change":"2024-09-10T19:08:26.918304+0000","last_active":"2024-09-10T19:08:37.625067+0000","last_peered":"2024-09-10T19:08:37.625067+0000","last_clean":"2024-09-10T19:08:37.625067+0000","last_became_active":"2024-09-10T18:55:40.391571+0000","last_became_peered":"2024-09-10T18:55:40.391571+0000","last_unstale":"2024-09-10T19:08:37.625067+0000","last_undegraded":"2024-09-10T19:08:37.625067+0000","last_fullsized":"2024-09-10T19:08:37.625067+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'21","last_scrub_stamp":"2024-09-10T19:08:26.918240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:08:26.918240+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:43.250535+0000","scrub_duration":0.0021185179999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":559,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.999847+0000","last_change":"2024-09-10T19:07:38.321617+0000","last_active":"2024-09-10T19:08:37.999847+0000","last_peered":"2024-09-10T19:08:37.999847+0000","last_clean":"2024-09-10T19:08:37.999847+0000","last_became_active":"2024-09-10T19:07:18.723010+0000","last_became_peered":"2024-09-10T19:07:18.723010+0000","last_unstale":"2024-09-10T19:08:37.999847+0000","last_undegraded":"2024-09-10T19:08:37.999847+0000","last_fullsized":"2024-09-10T19:08:37.999847+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:38.321553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:07:38.321553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:08:56.406814+0000","scrub_duration":0.0015874929999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":225,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.621742+0000","last_change":"2024-09-10T19:08:03.730646+0000","last_active":"2024-09-10T19:08:37.621742+0000","last_peered":"2024-09-10T19:08:37.621742+0000","last_clean":"2024-09-10T19:08:37.621742+0000","last_became_active":"2024-09-10T19:07:18.727841+0000","last_became_peered":"2024-09-10T19:07:18.727841+0000","last_unstale":"2024-09-10T19:08:37.621742+0000","last_undegraded":"2024-09-10T19:08:37.621742+0000","last_fullsized":"2024-09-10T19:08:37.621742+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:03.730586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:59.425718+0000","last_clean_scrub_stamp":"2024-09-10T19:08:03.730586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:13.323613+0000","scrub_duration":0.001653133,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.8","version":"70'81","reported_seq":204,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.999342+0000","last_change":"2024-09-10T19:07:44.202146+0000","last_active":"2024-09-10T19:08:37.999342+0000","last_peered":"2024-09-10T19:08:37.999342+0000","last_clean":"2024-09-10T19:08:37.999342+0000","last_became_active":"2024-09-10T19:07:21.862158+0000","last_became_peered":"2024-09-10T19:07:21.862158+0000","last_unstale":"2024-09-10T19:08:37.999342+0000","last_undegraded":"2024-09-10T19:08:37.999342+0000","last_fullsized":"2024-09-10T19:08:37.999342+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:07:44.202071+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:07:44.202071+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:08:53.927364+0000","scrub_duration":0.001732999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":525376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"427'53","reported_seq":667,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:09:00.029067+0000","last_change":"2024-09-10T19:09:00.029031+0000","last_active":"2024-09-10T19:09:00.029067+0000","last_peered":"2024-09-10T19:09:00.029067+0000","last_clean":"2024-09-10T19:09:00.029067+0000","last_became_active":"2024-09-10T19:07:21.862457+0000","last_became_peered":"2024-09-10T19:07:21.862457+0000","last_unstale":"2024-09-10T19:09:00.029067+0000","last_undegraded":"2024-09-10T19:09:00.029067+0000","last_fullsized":"2024-09-10T19:09:00.029067+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'53","last_scrub_stamp":"2024-09-10T19:09:00.028958+0000","last_deep_scrub":"78'25","last_deep_scrub_stamp":"2024-09-10T18:46:44.316146+0000","last_clean_scrub_stamp":"2024-09-10T19:09:00.028958+0000","objects_scrubbed":8,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:09.264364+0000","scrub_duration":0.0026767560000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":71,"num_read_kb":61,"num_write":46,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":53,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090378+0000","last_change":"2024-09-10T19:07:41.402347+0000","last_active":"2024-09-10T19:08:38.090378+0000","last_peered":"2024-09-10T19:08:38.090378+0000","last_clean":"2024-09-10T19:08:38.090378+0000","last_became_active":"2024-09-10T19:07:18.747252+0000","last_became_peered":"2024-09-10T19:07:18.747252+0000","last_unstale":"2024-09-10T19:08:38.090378+0000","last_undegraded":"2024-09-10T19:08:38.090378+0000","last_fullsized":"2024-09-10T19:08:38.090378+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:41.402224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:07:41.402224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:07.678852+0000","scrub_duration":0.0022755280000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":433,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.622962+0000","last_change":"2024-09-10T19:07:45.367587+0000","last_active":"2024-09-10T19:08:37.622962+0000","last_peered":"2024-09-10T19:08:37.622962+0000","last_clean":"2024-09-10T19:08:37.622962+0000","last_became_active":"2024-09-10T19:07:31.168843+0000","last_became_peered":"2024-09-10T19:07:31.168843+0000","last_unstale":"2024-09-10T19:08:37.622962+0000","last_undegraded":"2024-09-10T19:08:37.622962+0000","last_fullsized":"2024-09-10T19:08:37.622962+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:45.367535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:04.512425+0000","last_clean_scrub_stamp":"2024-09-10T19:07:45.367535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:05.379732+0000","scrub_duration":0.001540963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":59,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:47.245955+0000","last_change":"2024-09-10T19:08:47.245914+0000","last_active":"2024-09-10T19:08:47.245955+0000","last_peered":"2024-09-10T19:08:47.245955+0000","last_clean":"2024-09-10T19:08:47.245955+0000","last_became_active":"2024-09-10T19:07:21.856673+0000","last_became_peered":"2024-09-10T19:07:21.856673+0000","last_unstale":"2024-09-10T19:08:47.245955+0000","last_undegraded":"2024-09-10T19:08:47.245955+0000","last_fullsized":"2024-09-10T19:08:47.245955+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:47.245843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:08:47.245843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:05.486776+0000","scrub_duration":0.002629616,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,4],"acting":[2,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"70'81","reported_seq":63,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090420+0000","last_change":"2024-09-10T19:07:30.307185+0000","last_active":"2024-09-10T19:08:38.090420+0000","last_peered":"2024-09-10T19:08:38.090420+0000","last_clean":"2024-09-10T19:08:38.090420+0000","last_became_active":"2024-09-10T19:07:21.856173+0000","last_became_peered":"2024-09-10T19:07:21.856173+0000","last_unstale":"2024-09-10T19:08:38.090420+0000","last_undegraded":"2024-09-10T19:08:38.090420+0000","last_fullsized":"2024-09-10T19:08:38.090420+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:07:30.307121+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:07:30.307121+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:00.306654+0000","scrub_duration":0.0015656310000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"427'35","reported_seq":573,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:56.002380+0000","last_change":"2024-09-10T19:08:56.002347+0000","last_active":"2024-09-10T19:08:56.002380+0000","last_peered":"2024-09-10T19:08:56.002380+0000","last_clean":"2024-09-10T19:08:56.002380+0000","last_became_active":"2024-09-10T18:45:20.053428+0000","last_became_peered":"2024-09-10T18:45:20.053428+0000","last_unstale":"2024-09-10T19:08:56.002380+0000","last_undegraded":"2024-09-10T19:08:56.002380+0000","last_fullsized":"2024-09-10T19:08:56.002380+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'35","last_scrub_stamp":"2024-09-10T19:08:56.002267+0000","last_deep_scrub":"427'35","last_deep_scrub_stamp":"2024-09-10T19:08:56.002267+0000","last_clean_scrub_stamp":"2024-09-10T19:08:56.002267+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:21.723214+0000","scrub_duration":0.0021565209999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":443,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.623268+0000","last_change":"2024-09-10T19:07:54.316841+0000","last_active":"2024-09-10T19:08:37.623268+0000","last_peered":"2024-09-10T19:08:37.623268+0000","last_clean":"2024-09-10T19:08:37.623268+0000","last_became_active":"2024-09-10T19:07:31.278407+0000","last_became_peered":"2024-09-10T19:07:31.278407+0000","last_unstale":"2024-09-10T19:08:37.623268+0000","last_undegraded":"2024-09-10T19:08:37.623268+0000","last_fullsized":"2024-09-10T19:08:37.623268+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:54.316788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:07:54.316788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:19.073310+0000","scrub_duration":0.002303673,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"465'15","reported_seq":74,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:51.353488+0000","last_change":"2024-09-10T19:07:51.118530+0000","last_active":"2024-09-10T19:08:51.353488+0000","last_peered":"2024-09-10T19:08:51.353488+0000","last_clean":"2024-09-10T19:08:51.353488+0000","last_became_active":"2024-09-10T19:07:21.861317+0000","last_became_peered":"2024-09-10T19:07:21.861317+0000","last_unstale":"2024-09-10T19:08:51.353488+0000","last_undegraded":"2024-09-10T19:08:51.353488+0000","last_fullsized":"2024-09-10T19:08:51.353488+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"465'15","last_scrub_stamp":"2024-09-10T19:07:51.118468+0000","last_deep_scrub":"273'13","last_deep_scrub_stamp":"2024-09-10T19:06:23.008037+0000","last_clean_scrub_stamp":"2024-09-10T19:07:51.118468+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:08.037522+0000","scrub_duration":0.001824387,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":569,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.625535+0000","last_change":"2024-09-10T19:08:04.107896+0000","last_active":"2024-09-10T19:08:37.625535+0000","last_peered":"2024-09-10T19:08:37.625535+0000","last_clean":"2024-09-10T19:08:37.625535+0000","last_became_active":"2024-09-10T19:07:21.861634+0000","last_became_peered":"2024-09-10T19:07:21.861634+0000","last_unstale":"2024-09-10T19:08:37.625535+0000","last_undegraded":"2024-09-10T19:08:37.625535+0000","last_fullsized":"2024-09-10T19:08:37.625535+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:04.107827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:48:32.043666+0000","last_clean_scrub_stamp":"2024-09-10T19:08:04.107827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:16.388292+0000","scrub_duration":0.0035664350000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"70'81","reported_seq":457,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.000788+0000","last_change":"2024-09-10T19:08:30.577616+0000","last_active":"2024-09-10T19:08:38.000788+0000","last_peered":"2024-09-10T19:08:38.000788+0000","last_clean":"2024-09-10T19:08:38.000788+0000","last_became_active":"2024-09-10T19:07:21.862300+0000","last_became_peered":"2024-09-10T19:07:21.862300+0000","last_unstale":"2024-09-10T19:08:38.000788+0000","last_undegraded":"2024-09-10T19:08:38.000788+0000","last_fullsized":"2024-09-10T19:08:38.000788+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:08:30.577541+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:55:37.921519+0000","last_clean_scrub_stamp":"2024-09-10T19:08:30.577541+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:37.579895+0000","scrub_duration":0.0020382659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"427'46","reported_seq":562,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.000773+0000","last_change":"2024-09-10T19:08:15.511332+0000","last_active":"2024-09-10T19:08:38.000773+0000","last_peered":"2024-09-10T19:08:38.000773+0000","last_clean":"2024-09-10T19:08:38.000773+0000","last_became_active":"2024-09-10T18:46:56.742946+0000","last_became_peered":"2024-09-10T18:46:56.742946+0000","last_unstale":"2024-09-10T19:08:38.000773+0000","last_undegraded":"2024-09-10T19:08:38.000773+0000","last_fullsized":"2024-09-10T19:08:38.000773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'46","last_scrub_stamp":"2024-09-10T19:08:15.511257+0000","last_deep_scrub":"141'22","last_deep_scrub_stamp":"2024-09-10T18:50:57.119152+0000","last_clean_scrub_stamp":"2024-09-10T19:08:15.511257+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:25.528407+0000","scrub_duration":0.0023435169999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":443,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:09:01.689776+0000","last_change":"2024-09-10T19:09:01.689746+0000","last_active":"2024-09-10T19:09:01.689776+0000","last_peered":"2024-09-10T19:09:01.689776+0000","last_clean":"2024-09-10T19:09:01.689776+0000","last_became_active":"2024-09-10T19:07:31.278450+0000","last_became_peered":"2024-09-10T19:07:31.278450+0000","last_unstale":"2024-09-10T19:09:01.689776+0000","last_undegraded":"2024-09-10T19:09:01.689776+0000","last_fullsized":"2024-09-10T19:09:01.689776+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:09:01.689684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:28.320704+0000","last_clean_scrub_stamp":"2024-09-10T19:09:01.689684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:24.062807+0000","scrub_duration":0.0016748430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"242'10","reported_seq":833,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:56.354835+0000","last_change":"2024-09-10T19:08:17.922625+0000","last_active":"2024-09-10T19:08:56.354835+0000","last_peered":"2024-09-10T19:08:56.354835+0000","last_clean":"2024-09-10T19:08:56.354835+0000","last_became_active":"2024-09-10T18:55:40.397246+0000","last_became_peered":"2024-09-10T18:55:40.397246+0000","last_unstale":"2024-09-10T19:08:56.354835+0000","last_undegraded":"2024-09-10T19:08:56.354835+0000","last_fullsized":"2024-09-10T19:08:56.354835+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"242'10","last_scrub_stamp":"2024-09-10T19:08:17.922559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:08:17.922559+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:22.000888+0000","scrub_duration":0.001537615,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":547,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:39.657991+0000","last_change":"2024-09-10T19:08:39.657943+0000","last_active":"2024-09-10T19:08:39.657991+0000","last_peered":"2024-09-10T19:08:39.657991+0000","last_clean":"2024-09-10T19:08:39.657991+0000","last_became_active":"2024-09-10T18:55:40.392003+0000","last_became_peered":"2024-09-10T18:55:40.392003+0000","last_unstale":"2024-09-10T19:08:39.657991+0000","last_undegraded":"2024-09-10T19:08:39.657991+0000","last_fullsized":"2024-09-10T19:08:39.657991+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:39.657872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:54.218602+0000","last_clean_scrub_stamp":"2024-09-10T19:08:39.657872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:43.299476+0000","scrub_duration":0.001575821,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"70'81","reported_seq":396,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.623253+0000","last_change":"2024-09-10T19:07:49.366327+0000","last_active":"2024-09-10T19:08:37.623253+0000","last_peered":"2024-09-10T19:08:37.623253+0000","last_clean":"2024-09-10T19:08:37.623253+0000","last_became_active":"2024-09-10T19:07:31.169897+0000","last_became_peered":"2024-09-10T19:07:31.169897+0000","last_unstale":"2024-09-10T19:08:37.623253+0000","last_undegraded":"2024-09-10T19:08:37.623253+0000","last_fullsized":"2024-09-10T19:08:37.623253+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:07:49.366271+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T19:02:17.370627+0000","last_clean_scrub_stamp":"2024-09-10T19:07:49.366271+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:17.353130+0000","scrub_duration":0.001464066,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"427'32","reported_seq":647,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:43.007918+0000","last_change":"2024-09-10T19:08:43.007856+0000","last_active":"2024-09-10T19:08:43.007918+0000","last_peered":"2024-09-10T19:08:43.007918+0000","last_clean":"2024-09-10T19:08:43.007918+0000","last_became_active":"2024-09-10T18:47:57.106935+0000","last_became_peered":"2024-09-10T18:47:57.106935+0000","last_unstale":"2024-09-10T19:08:43.007918+0000","last_undegraded":"2024-09-10T19:08:43.007918+0000","last_fullsized":"2024-09-10T19:08:43.007918+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'32","last_scrub_stamp":"2024-09-10T19:08:43.007778+0000","last_deep_scrub":"224'16","last_deep_scrub_stamp":"2024-09-10T18:54:49.296957+0000","last_clean_scrub_stamp":"2024-09-10T19:08:43.007778+0000","objects_scrubbed":5,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:47.840641+0000","scrub_duration":0.0020834780000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":53,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090039+0000","last_change":"2024-09-10T19:07:46.355729+0000","last_active":"2024-09-10T19:08:38.090039+0000","last_peered":"2024-09-10T19:08:38.090039+0000","last_clean":"2024-09-10T19:08:38.090039+0000","last_became_active":"2024-09-10T19:07:18.743428+0000","last_became_peered":"2024-09-10T19:07:18.743428+0000","last_unstale":"2024-09-10T19:08:38.090039+0000","last_undegraded":"2024-09-10T19:08:38.090039+0000","last_fullsized":"2024-09-10T19:08:38.090039+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:46.355646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:07:46.355646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:12.574680+0000","scrub_duration":0.0018423389999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":535,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.000659+0000","last_change":"2024-09-10T19:08:18.509197+0000","last_active":"2024-09-10T19:08:38.000659+0000","last_peered":"2024-09-10T19:08:38.000659+0000","last_clean":"2024-09-10T19:08:38.000659+0000","last_became_active":"2024-09-10T18:45:22.068918+0000","last_became_peered":"2024-09-10T18:45:22.068918+0000","last_unstale":"2024-09-10T19:08:38.000659+0000","last_undegraded":"2024-09-10T19:08:38.000659+0000","last_fullsized":"2024-09-10T19:08:38.000659+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:18.509132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:48.116358+0000","last_clean_scrub_stamp":"2024-09-10T19:08:18.509132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:28.163911+0000","scrub_duration":0.0014485699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":518,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:51.810435+0000","last_change":"2024-09-10T19:08:51.810402+0000","last_active":"2024-09-10T19:08:51.810435+0000","last_peered":"2024-09-10T19:08:51.810435+0000","last_clean":"2024-09-10T19:08:51.810435+0000","last_became_active":"2024-09-10T18:45:24.264538+0000","last_became_peered":"2024-09-10T18:45:24.264538+0000","last_unstale":"2024-09-10T19:08:51.810435+0000","last_undegraded":"2024-09-10T19:08:51.810435+0000","last_fullsized":"2024-09-10T19:08:51.810435+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:51.810332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:51.398981+0000","last_clean_scrub_stamp":"2024-09-10T19:08:51.810332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:13.049208+0000","scrub_duration":0.001888303,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"70'81","reported_seq":486,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090574+0000","last_change":"2024-09-10T19:08:36.957950+0000","last_active":"2024-09-10T19:08:38.090574+0000","last_peered":"2024-09-10T19:08:38.090574+0000","last_clean":"2024-09-10T19:08:38.090574+0000","last_became_active":"2024-09-10T19:07:18.749200+0000","last_became_peered":"2024-09-10T19:07:18.749200+0000","last_unstale":"2024-09-10T19:08:38.090574+0000","last_undegraded":"2024-09-10T19:08:38.090574+0000","last_fullsized":"2024-09-10T19:08:38.090574+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:08:36.957895+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T19:08:36.957895+0000","last_clean_scrub_stamp":"2024-09-10T19:08:36.957895+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:00.195143+0000","scrub_duration":0.0014594219999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"427'49","reported_seq":593,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.002851+0000","last_change":"2024-09-10T19:08:38.002762+0000","last_active":"2024-09-10T19:08:38.002851+0000","last_peered":"2024-09-10T19:08:38.002851+0000","last_clean":"2024-09-10T19:08:38.002851+0000","last_became_active":"2024-09-10T18:45:20.052243+0000","last_became_peered":"2024-09-10T18:45:20.052243+0000","last_unstale":"2024-09-10T19:08:38.002851+0000","last_undegraded":"2024-09-10T19:08:38.002851+0000","last_fullsized":"2024-09-10T19:08:38.002851+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'49","last_scrub_stamp":"2024-09-10T19:08:38.002692+0000","last_deep_scrub":"143'21","last_deep_scrub_stamp":"2024-09-10T18:51:02.540216+0000","last_clean_scrub_stamp":"2024-09-10T19:08:38.002692+0000","objects_scrubbed":7,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:50.793533+0000","scrub_duration":0.0052600499999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":63,"num_read_kb":56,"num_write":42,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":587,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.000905+0000","last_change":"2024-09-10T19:08:26.620048+0000","last_active":"2024-09-10T19:08:38.000905+0000","last_peered":"2024-09-10T19:08:38.000905+0000","last_clean":"2024-09-10T19:08:38.000905+0000","last_became_active":"2024-09-10T19:07:21.862634+0000","last_became_peered":"2024-09-10T19:07:21.862634+0000","last_unstale":"2024-09-10T19:08:38.000905+0000","last_undegraded":"2024-09-10T19:08:38.000905+0000","last_fullsized":"2024-09-10T19:08:38.000905+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:26.619972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:07:24.531890+0000","last_clean_scrub_stamp":"2024-09-10T19:08:26.619972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:44.577127+0000","scrub_duration":0.002152316,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"63'8","reported_seq":813,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:56.354522+0000","last_change":"2024-09-10T19:07:55.791569+0000","last_active":"2024-09-10T19:08:56.354522+0000","last_peered":"2024-09-10T19:08:56.354522+0000","last_clean":"2024-09-10T19:08:56.354522+0000","last_became_active":"2024-09-10T18:45:22.074540+0000","last_became_peered":"2024-09-10T18:45:22.074540+0000","last_unstale":"2024-09-10T19:08:56.354522+0000","last_undegraded":"2024-09-10T19:08:56.354522+0000","last_fullsized":"2024-09-10T19:08:56.354522+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'8","last_scrub_stamp":"2024-09-10T19:07:55.791512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:07:55.791512+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:12.140532+0000","scrub_duration":0.0020915980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":73,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:43.272153+0000","last_change":"2024-09-10T19:08:43.272062+0000","last_active":"2024-09-10T19:08:43.272153+0000","last_peered":"2024-09-10T19:08:43.272153+0000","last_clean":"2024-09-10T19:08:43.272153+0000","last_became_active":"2024-09-10T19:07:21.856089+0000","last_became_peered":"2024-09-10T19:07:21.856089+0000","last_unstale":"2024-09-10T19:08:43.272153+0000","last_undegraded":"2024-09-10T19:08:43.272153+0000","last_fullsized":"2024-09-10T19:08:43.272153+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:43.271994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:08:43.271994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:08.049841+0000","scrub_duration":0.0014747440000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"70'81","reported_seq":453,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:48.943532+0000","last_change":"2024-09-10T19:08:48.943483+0000","last_active":"2024-09-10T19:08:48.943532+0000","last_peered":"2024-09-10T19:08:48.943532+0000","last_clean":"2024-09-10T19:08:48.943532+0000","last_became_active":"2024-09-10T19:07:21.862339+0000","last_became_peered":"2024-09-10T19:07:21.862339+0000","last_unstale":"2024-09-10T19:08:48.943532+0000","last_undegraded":"2024-09-10T19:08:48.943532+0000","last_fullsized":"2024-09-10T19:08:48.943532+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:08:48.943416+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:56:31.481303+0000","last_clean_scrub_stamp":"2024-09-10T19:08:48.943416+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:16.574945+0000","scrub_duration":0.0018045769999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"494'50","reported_seq":742,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090005+0000","last_change":"2024-09-10T19:08:35.991723+0000","last_active":"2024-09-10T19:08:38.090005+0000","last_peered":"2024-09-10T19:08:38.090005+0000","last_clean":"2024-09-10T19:08:38.090005+0000","last_became_active":"2024-09-10T19:07:18.741434+0000","last_became_peered":"2024-09-10T19:07:18.741434+0000","last_unstale":"2024-09-10T19:08:38.090005+0000","last_undegraded":"2024-09-10T19:08:38.090005+0000","last_fullsized":"2024-09-10T19:08:38.090005+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"494'50","last_scrub_stamp":"2024-09-10T19:08:35.991665+0000","last_deep_scrub":"494'50","last_deep_scrub_stamp":"2024-09-10T19:08:35.991665+0000","last_clean_scrub_stamp":"2024-09-10T19:08:35.991665+0000","objects_scrubbed":8,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:01.728712+0000","scrub_duration":0.0056939010000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":70,"num_read_kb":58,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":423,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.623246+0000","last_change":"2024-09-10T19:07:53.350128+0000","last_active":"2024-09-10T19:08:37.623246+0000","last_peered":"2024-09-10T19:08:37.623246+0000","last_clean":"2024-09-10T19:08:37.623246+0000","last_became_active":"2024-09-10T19:07:31.172524+0000","last_became_peered":"2024-09-10T19:07:31.172524+0000","last_unstale":"2024-09-10T19:08:37.623246+0000","last_undegraded":"2024-09-10T19:08:37.623246+0000","last_fullsized":"2024-09-10T19:08:37.623246+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:53.350060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:54:46.142129+0000","last_clean_scrub_stamp":"2024-09-10T19:07:53.350060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:19.894132+0000","scrub_duration":0.001627835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":541,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.624654+0000","last_change":"2024-09-10T19:07:58.078461+0000","last_active":"2024-09-10T19:08:37.624654+0000","last_peered":"2024-09-10T19:08:37.624654+0000","last_clean":"2024-09-10T19:08:37.624654+0000","last_became_active":"2024-09-10T19:07:18.740494+0000","last_became_peered":"2024-09-10T19:07:18.740494+0000","last_unstale":"2024-09-10T19:08:37.624654+0000","last_undegraded":"2024-09-10T19:08:37.624654+0000","last_fullsized":"2024-09-10T19:08:37.624654+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:58.078390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:57.903311+0000","last_clean_scrub_stamp":"2024-09-10T19:07:58.078390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:20.683491+0000","scrub_duration":0.0036123769999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":529,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.625038+0000","last_change":"2024-09-10T19:07:28.016189+0000","last_active":"2024-09-10T19:08:37.625038+0000","last_peered":"2024-09-10T19:08:37.625038+0000","last_clean":"2024-09-10T19:08:37.625038+0000","last_became_active":"2024-09-10T19:07:21.861522+0000","last_became_peered":"2024-09-10T19:07:21.861522+0000","last_unstale":"2024-09-10T19:08:37.625038+0000","last_undegraded":"2024-09-10T19:08:37.625038+0000","last_fullsized":"2024-09-10T19:08:37.625038+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:28.016123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:53.087539+0000","last_clean_scrub_stamp":"2024-09-10T19:07:28.016123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:08:49.931914+0000","scrub_duration":0.0030237900000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"70'81","reported_seq":540,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.000842+0000","last_change":"2024-09-10T19:08:02.673654+0000","last_active":"2024-09-10T19:08:38.000842+0000","last_peered":"2024-09-10T19:08:38.000842+0000","last_clean":"2024-09-10T19:08:38.000842+0000","last_became_active":"2024-09-10T18:55:40.396828+0000","last_became_peered":"2024-09-10T18:55:40.396828+0000","last_unstale":"2024-09-10T19:08:38.000842+0000","last_undegraded":"2024-09-10T19:08:38.000842+0000","last_fullsized":"2024-09-10T19:08:38.000842+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:08:02.673587+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T19:06:25.532348+0000","last_clean_scrub_stamp":"2024-09-10T19:08:02.673587+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:25.686387+0000","scrub_duration":0.001604711,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"427'39","reported_seq":626,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.622528+0000","last_change":"2024-09-10T19:08:01.796313+0000","last_active":"2024-09-10T19:08:37.622528+0000","last_peered":"2024-09-10T19:08:37.622528+0000","last_clean":"2024-09-10T19:08:37.622528+0000","last_became_active":"2024-09-10T19:07:18.728478+0000","last_became_peered":"2024-09-10T19:07:18.728478+0000","last_unstale":"2024-09-10T19:08:37.622528+0000","last_undegraded":"2024-09-10T19:08:37.622528+0000","last_fullsized":"2024-09-10T19:08:37.622528+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'39","last_scrub_stamp":"2024-09-10T19:08:01.796254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:08:01.796254+0000","objects_scrubbed":8,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:27.185638+0000","scrub_duration":0.0020776369999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":68,"num_read_kb":54,"num_write":42,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":32,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.882263+0000","last_change":"2024-09-10T19:08:00.795185+0000","last_active":"2024-09-10T19:08:37.882263+0000","last_peered":"2024-09-10T19:08:37.882263+0000","last_clean":"2024-09-10T19:08:37.882263+0000","last_became_active":"2024-09-10T19:07:58.849381+0000","last_became_peered":"2024-09-10T19:07:58.849381+0000","last_unstale":"2024-09-10T19:08:37.882263+0000","last_undegraded":"2024-09-10T19:08:37.882263+0000","last_fullsized":"2024-09-10T19:08:37.882263+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:00.795107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:06:29.659530+0000","last_clean_scrub_stamp":"2024-09-10T19:08:00.795107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:23.747687+0000","scrub_duration":0.0016343250000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,4],"acting":[2,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":563,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:55.514538+0000","last_change":"2024-09-10T19:08:55.514506+0000","last_active":"2024-09-10T19:08:55.514538+0000","last_peered":"2024-09-10T19:08:55.514538+0000","last_clean":"2024-09-10T19:08:55.514538+0000","last_became_active":"2024-09-10T18:55:40.387857+0000","last_became_peered":"2024-09-10T18:55:40.387857+0000","last_unstale":"2024-09-10T19:08:55.514538+0000","last_undegraded":"2024-09-10T19:08:55.514538+0000","last_fullsized":"2024-09-10T19:08:55.514538+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:55.514438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:08:55.514438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:03.224089+0000","scrub_duration":0.0015132629999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":453,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.622438+0000","last_change":"2024-09-10T19:07:46.331848+0000","last_active":"2024-09-10T19:08:37.622438+0000","last_peered":"2024-09-10T19:08:37.622438+0000","last_clean":"2024-09-10T19:08:37.622438+0000","last_became_active":"2024-09-10T19:07:31.171664+0000","last_became_peered":"2024-09-10T19:07:31.171664+0000","last_unstale":"2024-09-10T19:08:37.622438+0000","last_undegraded":"2024-09-10T19:08:37.622438+0000","last_fullsized":"2024-09-10T19:08:37.622438+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:46.331784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:49:18.281230+0000","last_clean_scrub_stamp":"2024-09-10T19:07:46.331784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:08:57.504226+0000","scrub_duration":0.001909979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"70'81","reported_seq":553,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.001257+0000","last_change":"2024-09-10T19:08:18.704584+0000","last_active":"2024-09-10T19:08:38.001257+0000","last_peered":"2024-09-10T19:08:38.001257+0000","last_clean":"2024-09-10T19:08:38.001257+0000","last_became_active":"2024-09-10T19:07:18.748748+0000","last_became_peered":"2024-09-10T19:07:18.748748+0000","last_unstale":"2024-09-10T19:08:38.001257+0000","last_undegraded":"2024-09-10T19:08:38.001257+0000","last_fullsized":"2024-09-10T19:08:38.001257+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:08:18.704518+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:08:18.704518+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:33.053071+0000","scrub_duration":0.00168654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"427'35","reported_seq":575,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.623131+0000","last_change":"2024-09-10T19:07:39.339297+0000","last_active":"2024-09-10T19:08:37.623131+0000","last_peered":"2024-09-10T19:08:37.623131+0000","last_clean":"2024-09-10T19:08:37.623131+0000","last_became_active":"2024-09-10T19:07:31.168654+0000","last_became_peered":"2024-09-10T19:07:31.168654+0000","last_unstale":"2024-09-10T19:08:37.623131+0000","last_undegraded":"2024-09-10T19:08:37.623131+0000","last_fullsized":"2024-09-10T19:08:37.623131+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'35","last_scrub_stamp":"2024-09-10T19:07:39.339228+0000","last_deep_scrub":"83'15","last_deep_scrub_stamp":"2024-09-10T18:46:58.436473+0000","last_clean_scrub_stamp":"2024-09-10T19:07:39.339228+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:08:52.095148+0000","scrub_duration":0.002305788,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":39,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":480,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:54.664120+0000","last_change":"2024-09-10T19:08:54.664091+0000","last_active":"2024-09-10T19:08:54.664120+0000","last_peered":"2024-09-10T19:08:54.664120+0000","last_clean":"2024-09-10T19:08:54.664120+0000","last_became_active":"2024-09-10T19:07:31.279211+0000","last_became_peered":"2024-09-10T19:07:31.279211+0000","last_unstale":"2024-09-10T19:08:54.664120+0000","last_undegraded":"2024-09-10T19:08:54.664120+0000","last_fullsized":"2024-09-10T19:08:54.664120+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:54.664029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:08:54.664029+0000","last_clean_scrub_stamp":"2024-09-10T19:08:54.664029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:12.841783+0000","scrub_duration":0.0015902010000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"339'13","reported_seq":786,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:56.354459+0000","last_change":"2024-09-10T19:08:43.700196+0000","last_active":"2024-09-10T19:08:56.354459+0000","last_peered":"2024-09-10T19:08:56.354459+0000","last_clean":"2024-09-10T19:08:56.354459+0000","last_became_active":"2024-09-10T19:00:42.520360+0000","last_became_peered":"2024-09-10T19:00:42.520360+0000","last_unstale":"2024-09-10T19:08:56.354459+0000","last_undegraded":"2024-09-10T19:08:56.354459+0000","last_fullsized":"2024-09-10T19:08:56.354459+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"339'13","last_scrub_stamp":"2024-09-10T19:08:43.700133+0000","last_deep_scrub":"258'12","last_deep_scrub_stamp":"2024-09-10T18:56:21.098770+0000","last_clean_scrub_stamp":"2024-09-10T19:08:43.700133+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:05.632734+0000","scrub_duration":0.0020569899999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":517,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.621959+0000","last_change":"2024-09-10T19:07:39.706206+0000","last_active":"2024-09-10T19:08:37.621959+0000","last_peered":"2024-09-10T19:08:37.621959+0000","last_clean":"2024-09-10T19:08:37.621959+0000","last_became_active":"2024-09-10T19:07:21.853717+0000","last_became_peered":"2024-09-10T19:07:21.853717+0000","last_unstale":"2024-09-10T19:08:37.621959+0000","last_undegraded":"2024-09-10T19:08:37.621959+0000","last_fullsized":"2024-09-10T19:08:37.621959+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:39.706147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:07:39.706147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:09.495498+0000","scrub_duration":0.0013607339999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"70'81","reported_seq":474,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.999461+0000","last_change":"2024-09-10T19:08:08.484018+0000","last_active":"2024-09-10T19:08:37.999461+0000","last_peered":"2024-09-10T19:08:37.999461+0000","last_clean":"2024-09-10T19:08:37.999461+0000","last_became_active":"2024-09-10T19:07:21.864419+0000","last_became_peered":"2024-09-10T19:07:21.864419+0000","last_unstale":"2024-09-10T19:08:37.999461+0000","last_undegraded":"2024-09-10T19:08:37.999461+0000","last_fullsized":"2024-09-10T19:08:37.999461+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:08:08.483943+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:08:08.483943+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:38.260370+0000","scrub_duration":0.002236937,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"470'56","reported_seq":621,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:56.354504+0000","last_change":"2024-09-10T19:08:42.551398+0000","last_active":"2024-09-10T19:08:56.354504+0000","last_peered":"2024-09-10T19:08:56.354504+0000","last_clean":"2024-09-10T19:08:56.354504+0000","last_became_active":"2024-09-10T19:07:31.174284+0000","last_became_peered":"2024-09-10T19:07:31.174284+0000","last_unstale":"2024-09-10T19:08:56.354504+0000","last_undegraded":"2024-09-10T19:08:56.354504+0000","last_fullsized":"2024-09-10T19:08:56.354504+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"470'56","last_scrub_stamp":"2024-09-10T19:08:42.551336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:08:42.551336+0000","objects_scrubbed":10,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:54.478510+0000","scrub_duration":0.0021253499999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":82,"num_read_kb":67,"num_write":50,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":539,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.999633+0000","last_change":"2024-09-10T19:08:24.542798+0000","last_active":"2024-09-10T19:08:37.999633+0000","last_peered":"2024-09-10T19:08:37.999633+0000","last_clean":"2024-09-10T19:08:37.999633+0000","last_became_active":"2024-09-10T18:45:17.964494+0000","last_became_peered":"2024-09-10T18:45:17.964494+0000","last_unstale":"2024-09-10T19:08:37.999633+0000","last_undegraded":"2024-09-10T19:08:37.999633+0000","last_fullsized":"2024-09-10T19:08:37.999633+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:24.542735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:12.982599+0000","last_clean_scrub_stamp":"2024-09-10T19:08:24.542735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:35.432204+0000","scrub_duration":0.0016080980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":491,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.000297+0000","last_change":"2024-09-10T19:08:36.566225+0000","last_active":"2024-09-10T19:08:38.000297+0000","last_peered":"2024-09-10T19:08:38.000297+0000","last_clean":"2024-09-10T19:08:38.000297+0000","last_became_active":"2024-09-10T18:47:57.107478+0000","last_became_peered":"2024-09-10T18:47:57.107478+0000","last_unstale":"2024-09-10T19:08:38.000297+0000","last_undegraded":"2024-09-10T19:08:38.000297+0000","last_fullsized":"2024-09-10T19:08:38.000297+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:36.566169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:08:36.566169+0000","last_clean_scrub_stamp":"2024-09-10T19:08:36.566169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:41.902876+0000","scrub_duration":0.001212032,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":528,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.091196+0000","last_change":"2024-09-10T19:08:29.013154+0000","last_active":"2024-09-10T19:08:38.091196+0000","last_peered":"2024-09-10T19:08:38.091196+0000","last_clean":"2024-09-10T19:08:38.091196+0000","last_became_active":"2024-09-10T19:07:18.742594+0000","last_became_peered":"2024-09-10T19:07:18.742594+0000","last_unstale":"2024-09-10T19:08:38.091196+0000","last_undegraded":"2024-09-10T19:08:38.091196+0000","last_fullsized":"2024-09-10T19:08:38.091196+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:29.013095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:08:29.013095+0000","last_clean_scrub_stamp":"2024-09-10T19:08:29.013095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:49.531497+0000","scrub_duration":0.001858181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"70'81","reported_seq":53,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090465+0000","last_change":"2024-09-10T19:08:06.395361+0000","last_active":"2024-09-10T19:08:38.090465+0000","last_peered":"2024-09-10T19:08:38.090465+0000","last_clean":"2024-09-10T19:08:38.090465+0000","last_became_active":"2024-09-10T19:07:18.741282+0000","last_became_peered":"2024-09-10T19:07:18.741282+0000","last_unstale":"2024-09-10T19:08:38.090465+0000","last_undegraded":"2024-09-10T19:08:38.090465+0000","last_fullsized":"2024-09-10T19:08:38.090465+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:08:06.395226+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:08:06.395226+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:35.100800+0000","scrub_duration":0.0022028500000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"427'26","reported_seq":395,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:39.587966+0000","last_change":"2024-09-10T19:08:39.587912+0000","last_active":"2024-09-10T19:08:39.587966+0000","last_peered":"2024-09-10T19:08:39.587966+0000","last_clean":"2024-09-10T19:08:39.587966+0000","last_became_active":"2024-09-10T19:07:31.168702+0000","last_became_peered":"2024-09-10T19:07:31.168702+0000","last_unstale":"2024-09-10T19:08:39.587966+0000","last_undegraded":"2024-09-10T19:08:39.587966+0000","last_fullsized":"2024-09-10T19:08:39.587966+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'26","last_scrub_stamp":"2024-09-10T19:08:39.587845+0000","last_deep_scrub":"84'18","last_deep_scrub_stamp":"2024-09-10T18:47:10.514951+0000","last_clean_scrub_stamp":"2024-09-10T19:08:39.587845+0000","objects_scrubbed":5,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:59.290992+0000","scrub_duration":0.0024897130000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":40,"num_read_kb":29,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"84'6","reported_seq":1588,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:09:00.327295+0000","last_change":"2024-09-10T19:08:28.432639+0000","last_active":"2024-09-10T19:09:00.327295+0000","last_peered":"2024-09-10T19:09:00.327295+0000","last_clean":"2024-09-10T19:09:00.327295+0000","last_became_active":"2024-09-10T19:07:31.170254+0000","last_became_peered":"2024-09-10T19:07:31.170254+0000","last_unstale":"2024-09-10T19:09:00.327295+0000","last_undegraded":"2024-09-10T19:09:00.327295+0000","last_fullsized":"2024-09-10T19:09:00.327295+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'6","last_scrub_stamp":"2024-09-10T19:08:28.432573+0000","last_deep_scrub":"84'6","last_deep_scrub_stamp":"2024-09-10T18:47:22.545450+0000","last_clean_scrub_stamp":"2024-09-10T19:08:28.432573+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:35.555013+0000","scrub_duration":0.00157843,"objects_trimmed":0,"snaptrim_duration":0.0021933159999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1121,"num_read_kb":1116,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":59,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.881996+0000","last_change":"2024-09-10T19:08:37.373940+0000","last_active":"2024-09-10T19:08:37.881996+0000","last_peered":"2024-09-10T19:08:37.881996+0000","last_clean":"2024-09-10T19:08:37.881996+0000","last_became_active":"2024-09-10T19:07:21.859912+0000","last_became_peered":"2024-09-10T19:07:21.859912+0000","last_unstale":"2024-09-10T19:08:37.881996+0000","last_undegraded":"2024-09-10T19:08:37.881996+0000","last_fullsized":"2024-09-10T19:08:37.881996+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:37.373879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:08:37.373879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:07.265854+0000","scrub_duration":0.0014641960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":59,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:40.292695+0000","last_change":"2024-09-10T19:08:40.292640+0000","last_active":"2024-09-10T19:08:40.292695+0000","last_peered":"2024-09-10T19:08:40.292695+0000","last_clean":"2024-09-10T19:08:40.292695+0000","last_became_active":"2024-09-10T19:07:21.861256+0000","last_became_peered":"2024-09-10T19:07:21.861256+0000","last_unstale":"2024-09-10T19:08:40.292695+0000","last_undegraded":"2024-09-10T19:08:40.292695+0000","last_fullsized":"2024-09-10T19:08:40.292695+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:40.292584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:07:33.132422+0000","last_clean_scrub_stamp":"2024-09-10T19:08:40.292584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:08.890319+0000","scrub_duration":0.001399511,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":63,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090906+0000","last_change":"2024-09-10T19:08:26.248390+0000","last_active":"2024-09-10T19:08:38.090906+0000","last_peered":"2024-09-10T19:08:38.090906+0000","last_clean":"2024-09-10T19:08:38.090906+0000","last_became_active":"2024-09-10T19:07:18.749761+0000","last_became_peered":"2024-09-10T19:07:18.749761+0000","last_unstale":"2024-09-10T19:08:38.090906+0000","last_undegraded":"2024-09-10T19:08:38.090906+0000","last_fullsized":"2024-09-10T19:08:38.090906+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:26.248323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:07.150480+0000","last_clean_scrub_stamp":"2024-09-10T19:08:26.248323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:45.832107+0000","scrub_duration":0.0016957179999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"70'81","reported_seq":337,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.623555+0000","last_change":"2024-09-10T19:07:31.170822+0000","last_active":"2024-09-10T19:08:37.623555+0000","last_peered":"2024-09-10T19:08:37.623555+0000","last_clean":"2024-09-10T19:08:37.623555+0000","last_became_active":"2024-09-10T19:07:31.170346+0000","last_became_peered":"2024-09-10T19:07:31.170346+0000","last_unstale":"2024-09-10T19:08:37.623555+0000","last_undegraded":"2024-09-10T19:08:37.623555+0000","last_fullsized":"2024-09-10T19:08:37.623555+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":3,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:07:29.520653+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:07:29.520653+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:08:56.625275+0000","scrub_duration":0.0023558350000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"427'36","reported_seq":329,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:40.796464+0000","last_change":"2024-09-10T19:08:40.796386+0000","last_active":"2024-09-10T19:08:40.796464+0000","last_peered":"2024-09-10T19:08:40.796464+0000","last_clean":"2024-09-10T19:08:40.796464+0000","last_became_active":"2024-09-10T19:07:21.854534+0000","last_became_peered":"2024-09-10T19:07:21.854534+0000","last_unstale":"2024-09-10T19:08:40.796464+0000","last_undegraded":"2024-09-10T19:08:40.796464+0000","last_fullsized":"2024-09-10T19:08:40.796464+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'36","last_scrub_stamp":"2024-09-10T19:08:40.796321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:08:40.796321+0000","objects_scrubbed":6,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:05.606848+0000","scrub_duration":0.0019102520000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":47,"num_write":36,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"305'2","reported_seq":510,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.622125+0000","last_change":"2024-09-10T19:08:17.796015+0000","last_active":"2024-09-10T19:08:37.622125+0000","last_peered":"2024-09-10T19:08:37.622125+0000","last_clean":"2024-09-10T19:08:37.622125+0000","last_became_active":"2024-09-10T18:55:40.390022+0000","last_became_peered":"2024-09-10T18:55:40.390022+0000","last_unstale":"2024-09-10T19:08:37.622125+0000","last_undegraded":"2024-09-10T19:08:37.622125+0000","last_fullsized":"2024-09-10T19:08:37.622125+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"305'2","last_scrub_stamp":"2024-09-10T19:08:17.795948+0000","last_deep_scrub":"305'2","last_deep_scrub_stamp":"2024-09-10T18:59:18.897059+0000","last_clean_scrub_stamp":"2024-09-10T19:08:17.795948+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:42.727821+0000","scrub_duration":0.0016584340000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":53,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090195+0000","last_change":"2024-09-10T19:08:03.363244+0000","last_active":"2024-09-10T19:08:38.090195+0000","last_peered":"2024-09-10T19:08:38.090195+0000","last_clean":"2024-09-10T19:08:38.090195+0000","last_became_active":"2024-09-10T19:07:18.740617+0000","last_became_peered":"2024-09-10T19:07:18.740617+0000","last_unstale":"2024-09-10T19:08:38.090195+0000","last_undegraded":"2024-09-10T19:08:38.090195+0000","last_fullsized":"2024-09-10T19:08:38.090195+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:03.363145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:08:03.363145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:06.600546+0000","scrub_duration":0.002509477,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":186,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.000866+0000","last_change":"2024-09-10T19:08:11.517617+0000","last_active":"2024-09-10T19:08:38.000866+0000","last_peered":"2024-09-10T19:08:38.000866+0000","last_clean":"2024-09-10T19:08:38.000866+0000","last_became_active":"2024-09-10T19:00:42.518566+0000","last_became_peered":"2024-09-10T19:00:42.518566+0000","last_unstale":"2024-09-10T19:08:38.000866+0000","last_undegraded":"2024-09-10T19:08:38.000866+0000","last_fullsized":"2024-09-10T19:08:38.000866+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:11.517549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:08:11.517549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:34.632717+0000","scrub_duration":0.0017203699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":430,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:43.583648+0000","last_change":"2024-09-10T19:08:43.583611+0000","last_active":"2024-09-10T19:08:43.583648+0000","last_peered":"2024-09-10T19:08:43.583648+0000","last_clean":"2024-09-10T19:08:43.583648+0000","last_became_active":"2024-09-10T19:07:31.173496+0000","last_became_peered":"2024-09-10T19:07:31.173496+0000","last_unstale":"2024-09-10T19:08:43.583648+0000","last_undegraded":"2024-09-10T19:08:43.583648+0000","last_fullsized":"2024-09-10T19:08:43.583648+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:43.583556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:08:43.583556+0000","last_clean_scrub_stamp":"2024-09-10T19:08:43.583556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:10.598278+0000","scrub_duration":0.00127001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"427'45","reported_seq":665,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.000779+0000","last_change":"2024-09-10T19:07:54.737766+0000","last_active":"2024-09-10T19:08:38.000779+0000","last_peered":"2024-09-10T19:08:38.000779+0000","last_clean":"2024-09-10T19:08:38.000779+0000","last_became_active":"2024-09-10T19:07:18.747716+0000","last_became_peered":"2024-09-10T19:07:18.747716+0000","last_unstale":"2024-09-10T19:08:38.000779+0000","last_undegraded":"2024-09-10T19:08:38.000779+0000","last_fullsized":"2024-09-10T19:08:38.000779+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'45","last_scrub_stamp":"2024-09-10T19:07:54.737699+0000","last_deep_scrub":"232'33","last_deep_scrub_stamp":"2024-09-10T18:56:21.206960+0000","last_clean_scrub_stamp":"2024-09-10T19:07:54.737699+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:08:57.477407+0000","scrub_duration":0.0031575549999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":61,"num_read_kb":51,"num_write":37,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":308,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"60'2","reported_seq":537,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:54.845767+0000","last_change":"2024-09-10T19:08:54.845742+0000","last_active":"2024-09-10T19:08:54.845767+0000","last_peered":"2024-09-10T19:08:54.845767+0000","last_clean":"2024-09-10T19:08:54.845767+0000","last_became_active":"2024-09-10T18:45:17.961660+0000","last_became_peered":"2024-09-10T18:45:17.961660+0000","last_unstale":"2024-09-10T19:08:54.845767+0000","last_undegraded":"2024-09-10T19:08:54.845767+0000","last_fullsized":"2024-09-10T19:08:54.845767+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-10T19:08:54.845684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:08:54.845684+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:16.073437+0000","scrub_duration":0.0019642700000000002,"objects_trimmed":0,"snaptrim_duration":0.0013942640000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":576,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.000284+0000","last_change":"2024-09-10T19:08:12.478323+0000","last_active":"2024-09-10T19:08:38.000284+0000","last_peered":"2024-09-10T19:08:38.000284+0000","last_clean":"2024-09-10T19:08:38.000284+0000","last_became_active":"2024-09-10T19:07:21.863455+0000","last_became_peered":"2024-09-10T19:07:21.863455+0000","last_unstale":"2024-09-10T19:08:38.000284+0000","last_undegraded":"2024-09-10T19:08:38.000284+0000","last_fullsized":"2024-09-10T19:08:38.000284+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:12.478251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:08:12.478251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:29.916361+0000","scrub_duration":0.001413135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'81","reported_seq":53,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090334+0000","last_change":"2024-09-10T19:08:16.350878+0000","last_active":"2024-09-10T19:08:38.090334+0000","last_peered":"2024-09-10T19:08:38.090334+0000","last_clean":"2024-09-10T19:08:38.090334+0000","last_became_active":"2024-09-10T19:07:18.746996+0000","last_became_peered":"2024-09-10T19:07:18.746996+0000","last_unstale":"2024-09-10T19:08:38.090334+0000","last_undegraded":"2024-09-10T19:08:38.090334+0000","last_fullsized":"2024-09-10T19:08:38.090334+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:08:16.350759+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:56.085580+0000","last_clean_scrub_stamp":"2024-09-10T19:08:16.350759+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:17.938788+0000","scrub_duration":0.001500136,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":63,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.091014+0000","last_change":"2024-09-10T19:07:36.341204+0000","last_active":"2024-09-10T19:08:38.091014+0000","last_peered":"2024-09-10T19:08:38.091014+0000","last_clean":"2024-09-10T19:08:38.091014+0000","last_became_active":"2024-09-10T19:07:21.860275+0000","last_became_peered":"2024-09-10T19:07:21.860275+0000","last_unstale":"2024-09-10T19:08:38.091014+0000","last_undegraded":"2024-09-10T19:08:38.091014+0000","last_fullsized":"2024-09-10T19:08:38.091014+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:36.341134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:07:36.341134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:08:55.991437+0000","scrub_duration":0.001199571,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":530,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.000483+0000","last_change":"2024-09-10T19:08:05.504409+0000","last_active":"2024-09-10T19:08:38.000483+0000","last_peered":"2024-09-10T19:08:38.000483+0000","last_clean":"2024-09-10T19:08:38.000483+0000","last_became_active":"2024-09-10T18:45:24.073327+0000","last_became_peered":"2024-09-10T18:45:24.073327+0000","last_unstale":"2024-09-10T19:08:38.000483+0000","last_undegraded":"2024-09-10T19:08:38.000483+0000","last_fullsized":"2024-09-10T19:08:38.000483+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:05.504340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:08:05.504340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:08.259505+0000","scrub_duration":0.001813633,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.f","version":"70'81","reported_seq":535,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.001055+0000","last_change":"2024-09-10T19:07:58.802241+0000","last_active":"2024-09-10T19:08:38.001055+0000","last_peered":"2024-09-10T19:08:38.001055+0000","last_clean":"2024-09-10T19:08:38.001055+0000","last_became_active":"2024-09-10T18:52:16.598295+0000","last_became_peered":"2024-09-10T18:52:16.598295+0000","last_unstale":"2024-09-10T19:08:38.001055+0000","last_undegraded":"2024-09-10T19:08:38.001055+0000","last_fullsized":"2024-09-10T19:08:38.001055+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:07:58.802189+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T19:07:58.802189+0000","last_clean_scrub_stamp":"2024-09-10T19:07:58.802189+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:14.862555+0000","scrub_duration":0.001211047,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"427'51","reported_seq":687,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:47.604189+0000","last_change":"2024-09-10T19:08:47.604156+0000","last_active":"2024-09-10T19:08:47.604189+0000","last_peered":"2024-09-10T19:08:47.604189+0000","last_clean":"2024-09-10T19:08:47.604189+0000","last_became_active":"2024-09-10T19:07:18.746562+0000","last_became_peered":"2024-09-10T19:07:18.746562+0000","last_unstale":"2024-09-10T19:08:47.604189+0000","last_undegraded":"2024-09-10T19:08:47.604189+0000","last_fullsized":"2024-09-10T19:08:47.604189+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'51","last_scrub_stamp":"2024-09-10T19:08:47.604085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:08:47.604085+0000","objects_scrubbed":9,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:02.828141+0000","scrub_duration":0.002354989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":80,"num_read_kb":69,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":49,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.882284+0000","last_change":"2024-09-10T19:07:36.140535+0000","last_active":"2024-09-10T19:08:37.882284+0000","last_peered":"2024-09-10T19:08:37.882284+0000","last_clean":"2024-09-10T19:08:37.882284+0000","last_became_active":"2024-09-10T19:07:21.869555+0000","last_became_peered":"2024-09-10T19:07:21.869555+0000","last_unstale":"2024-09-10T19:08:37.882284+0000","last_undegraded":"2024-09-10T19:08:37.882284+0000","last_fullsized":"2024-09-10T19:08:37.882284+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:36.140480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:07:36.140480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:01.966689+0000","scrub_duration":0.001305016,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":59,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:50.268609+0000","last_change":"2024-09-10T19:08:50.268564+0000","last_active":"2024-09-10T19:08:50.268609+0000","last_peered":"2024-09-10T19:08:50.268609+0000","last_clean":"2024-09-10T19:08:50.268609+0000","last_became_active":"2024-09-10T19:07:21.868368+0000","last_became_peered":"2024-09-10T19:07:21.868368+0000","last_unstale":"2024-09-10T19:08:50.268609+0000","last_undegraded":"2024-09-10T19:08:50.268609+0000","last_fullsized":"2024-09-10T19:08:50.268609+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:50.268480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:08:50.268480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:56.971191+0000","scrub_duration":0.001670185,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":73,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:46.385437+0000","last_change":"2024-09-10T19:08:46.385385+0000","last_active":"2024-09-10T19:08:46.385437+0000","last_peered":"2024-09-10T19:08:46.385437+0000","last_clean":"2024-09-10T19:08:46.385437+0000","last_became_active":"2024-09-10T19:07:21.857233+0000","last_became_peered":"2024-09-10T19:07:21.857233+0000","last_unstale":"2024-09-10T19:08:46.385437+0000","last_undegraded":"2024-09-10T19:08:46.385437+0000","last_fullsized":"2024-09-10T19:08:46.385437+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:46.385312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:08:46.385312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:14.903106+0000","scrub_duration":0.00178462,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.e","version":"70'81","reported_seq":404,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.625422+0000","last_change":"2024-09-10T19:07:37.093614+0000","last_active":"2024-09-10T19:08:37.625422+0000","last_peered":"2024-09-10T19:08:37.625422+0000","last_clean":"2024-09-10T19:08:37.625422+0000","last_became_active":"2024-09-10T19:07:18.733276+0000","last_became_peered":"2024-09-10T19:07:18.733276+0000","last_unstale":"2024-09-10T19:08:37.625422+0000","last_undegraded":"2024-09-10T19:08:37.625422+0000","last_fullsized":"2024-09-10T19:08:37.625422+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:07:37.093545+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:07:37.093545+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:08:52.743131+0000","scrub_duration":0.0014376840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"427'25","reported_seq":551,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:58.759028+0000","last_change":"2024-09-10T19:08:58.759000+0000","last_active":"2024-09-10T19:08:58.759028+0000","last_peered":"2024-09-10T19:08:58.759028+0000","last_clean":"2024-09-10T19:08:58.759028+0000","last_became_active":"2024-09-10T18:55:40.395257+0000","last_became_peered":"2024-09-10T18:55:40.395257+0000","last_unstale":"2024-09-10T19:08:58.759028+0000","last_undegraded":"2024-09-10T19:08:58.759028+0000","last_fullsized":"2024-09-10T19:08:58.759028+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'25","last_scrub_stamp":"2024-09-10T19:08:58.758929+0000","last_deep_scrub":"130'13","last_deep_scrub_stamp":"2024-09-10T18:51:58.427534+0000","last_clean_scrub_stamp":"2024-09-10T19:08:58.758929+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:15.839540+0000","scrub_duration":0.002359355,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":430,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:48.488720+0000","last_change":"2024-09-10T19:08:48.488661+0000","last_active":"2024-09-10T19:08:48.488720+0000","last_peered":"2024-09-10T19:08:48.488720+0000","last_clean":"2024-09-10T19:08:48.488720+0000","last_became_active":"2024-09-10T19:07:31.174814+0000","last_became_peered":"2024-09-10T19:07:31.174814+0000","last_unstale":"2024-09-10T19:08:48.488720+0000","last_undegraded":"2024-09-10T19:08:48.488720+0000","last_fullsized":"2024-09-10T19:08:48.488720+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:48.488581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:08:48.488581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:10.547013+0000","scrub_duration":0.001868705,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":49,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.882055+0000","last_change":"2024-09-10T19:08:35.383303+0000","last_active":"2024-09-10T19:08:37.882055+0000","last_peered":"2024-09-10T19:08:37.882055+0000","last_clean":"2024-09-10T19:08:37.882055+0000","last_became_active":"2024-09-10T19:07:21.859818+0000","last_became_peered":"2024-09-10T19:07:21.859818+0000","last_unstale":"2024-09-10T19:08:37.882055+0000","last_undegraded":"2024-09-10T19:08:37.882055+0000","last_fullsized":"2024-09-10T19:08:37.882055+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:35.383240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:55.684250+0000","last_clean_scrub_stamp":"2024-09-10T19:08:35.383240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:00.735949+0000","scrub_duration":0.0018617659999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":59,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.624939+0000","last_change":"2024-09-10T19:07:55.020542+0000","last_active":"2024-09-10T19:08:37.624939+0000","last_peered":"2024-09-10T19:08:37.624939+0000","last_clean":"2024-09-10T19:08:37.624939+0000","last_became_active":"2024-09-10T19:07:10.335110+0000","last_became_peered":"2024-09-10T19:07:10.335110+0000","last_unstale":"2024-09-10T19:08:37.624939+0000","last_undegraded":"2024-09-10T19:08:37.624939+0000","last_fullsized":"2024-09-10T19:08:37.624939+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:55.020478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:53.345184+0000","last_clean_scrub_stamp":"2024-09-10T19:07:55.020478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:09.050199+0000","scrub_duration":0.001414558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.d","version":"70'81","reported_seq":53,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090759+0000","last_change":"2024-09-10T19:07:56.415636+0000","last_active":"2024-09-10T19:08:38.090759+0000","last_peered":"2024-09-10T19:08:38.090759+0000","last_clean":"2024-09-10T19:08:38.090759+0000","last_became_active":"2024-09-10T19:07:18.738294+0000","last_became_peered":"2024-09-10T19:07:18.738294+0000","last_unstale":"2024-09-10T19:08:38.090759+0000","last_undegraded":"2024-09-10T19:08:38.090759+0000","last_fullsized":"2024-09-10T19:08:38.090759+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:07:56.415568+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:07:56.415568+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:11.205660+0000","scrub_duration":0.0018649300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"427'46","reported_seq":263,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.999505+0000","last_change":"2024-09-10T19:08:28.594621+0000","last_active":"2024-09-10T19:08:37.999505+0000","last_peered":"2024-09-10T19:08:37.999505+0000","last_clean":"2024-09-10T19:08:37.999505+0000","last_became_active":"2024-09-10T19:07:18.736452+0000","last_became_peered":"2024-09-10T19:07:18.736452+0000","last_unstale":"2024-09-10T19:08:37.999505+0000","last_undegraded":"2024-09-10T19:08:37.999505+0000","last_fullsized":"2024-09-10T19:08:37.999505+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'46","last_scrub_stamp":"2024-09-10T19:08:28.594544+0000","last_deep_scrub":"365'34","last_deep_scrub_stamp":"2024-09-10T19:02:18.712306+0000","last_clean_scrub_stamp":"2024-09-10T19:08:28.594544+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:40.319493+0000","scrub_duration":0.0022179560000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":60,"num_read_kb":51,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":49,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.882606+0000","last_change":"2024-09-10T19:07:54.991286+0000","last_active":"2024-09-10T19:08:37.882606+0000","last_peered":"2024-09-10T19:08:37.882606+0000","last_clean":"2024-09-10T19:08:37.882606+0000","last_became_active":"2024-09-10T19:07:21.860619+0000","last_became_peered":"2024-09-10T19:07:21.860619+0000","last_unstale":"2024-09-10T19:08:37.882606+0000","last_undegraded":"2024-09-10T19:08:37.882606+0000","last_fullsized":"2024-09-10T19:08:37.882606+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:54.991205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:05.603125+0000","last_clean_scrub_stamp":"2024-09-10T19:07:54.991205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:22.141928+0000","scrub_duration":0.001854613,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"464'11","reported_seq":77,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:51.353728+0000","last_change":"2024-09-10T19:08:22.289110+0000","last_active":"2024-09-10T19:08:51.353728+0000","last_peered":"2024-09-10T19:08:51.353728+0000","last_clean":"2024-09-10T19:08:51.353728+0000","last_became_active":"2024-09-10T19:07:18.744158+0000","last_became_peered":"2024-09-10T19:07:18.744158+0000","last_unstale":"2024-09-10T19:08:51.353728+0000","last_undegraded":"2024-09-10T19:08:51.353728+0000","last_fullsized":"2024-09-10T19:08:51.353728+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"464'11","last_scrub_stamp":"2024-09-10T19:08:22.289040+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-09-10T18:47:02.582229+0000","last_clean_scrub_stamp":"2024-09-10T19:08:22.289040+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:34.235717+0000","scrub_duration":0.0017950539999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":537,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.000869+0000","last_change":"2024-09-10T19:07:52.307859+0000","last_active":"2024-09-10T19:08:38.000869+0000","last_peered":"2024-09-10T19:08:38.000869+0000","last_clean":"2024-09-10T19:08:38.000869+0000","last_became_active":"2024-09-10T18:55:40.389202+0000","last_became_peered":"2024-09-10T18:55:40.389202+0000","last_unstale":"2024-09-10T19:08:38.000869+0000","last_undegraded":"2024-09-10T19:08:38.000869+0000","last_fullsized":"2024-09-10T19:08:38.000869+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:52.307747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:45.083425+0000","last_clean_scrub_stamp":"2024-09-10T19:07:52.307747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:19.489744+0000","scrub_duration":0.001619391,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.c","version":"70'81","reported_seq":478,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:46.912249+0000","last_change":"2024-09-10T19:08:46.912219+0000","last_active":"2024-09-10T19:08:46.912249+0000","last_peered":"2024-09-10T19:08:46.912249+0000","last_clean":"2024-09-10T19:08:46.912249+0000","last_became_active":"2024-09-10T18:55:40.390655+0000","last_became_peered":"2024-09-10T18:55:40.390655+0000","last_unstale":"2024-09-10T19:08:46.912249+0000","last_undegraded":"2024-09-10T19:08:46.912249+0000","last_fullsized":"2024-09-10T19:08:46.912249+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":4,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:08:46.912148+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:56:33.779154+0000","last_clean_scrub_stamp":"2024-09-10T19:08:46.912148+0000","objects_scrubbed":1,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:09.086147+0000","scrub_duration":0.00455558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"427'24","reported_seq":567,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.624769+0000","last_change":"2024-09-10T19:08:02.119915+0000","last_active":"2024-09-10T19:08:37.624769+0000","last_peered":"2024-09-10T19:08:37.624769+0000","last_clean":"2024-09-10T19:08:37.624769+0000","last_became_active":"2024-09-10T18:45:20.056869+0000","last_became_peered":"2024-09-10T18:45:20.056869+0000","last_unstale":"2024-09-10T19:08:37.624769+0000","last_undegraded":"2024-09-10T19:08:37.624769+0000","last_fullsized":"2024-09-10T19:08:37.624769+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'24","last_scrub_stamp":"2024-09-10T19:08:02.119848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:08:02.119848+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:06.509071+0000","scrub_duration":0.0024358679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"56'2","reported_seq":584,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.625848+0000","last_change":"2024-09-10T19:07:59.100435+0000","last_active":"2024-09-10T19:08:37.625848+0000","last_peered":"2024-09-10T19:08:37.625848+0000","last_clean":"2024-09-10T19:08:37.625848+0000","last_became_active":"2024-09-10T19:07:18.733655+0000","last_became_peered":"2024-09-10T19:07:18.733655+0000","last_unstale":"2024-09-10T19:08:37.625848+0000","last_undegraded":"2024-09-10T19:08:37.625848+0000","last_fullsized":"2024-09-10T19:08:37.625848+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'2","last_scrub_stamp":"2024-09-10T19:07:59.100378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:07:59.100378+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:05.990069+0000","scrub_duration":0.0018153150000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":49,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.881835+0000","last_change":"2024-09-10T19:07:53.056021+0000","last_active":"2024-09-10T19:08:37.881835+0000","last_peered":"2024-09-10T19:08:37.881835+0000","last_clean":"2024-09-10T19:08:37.881835+0000","last_became_active":"2024-09-10T19:07:21.860783+0000","last_became_peered":"2024-09-10T19:07:21.860783+0000","last_unstale":"2024-09-10T19:08:37.881835+0000","last_undegraded":"2024-09-10T19:08:37.881835+0000","last_fullsized":"2024-09-10T19:08:37.881835+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:53.055935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:02.711594+0000","last_clean_scrub_stamp":"2024-09-10T19:07:53.055935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:10.070848+0000","scrub_duration":0.001537409,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":528,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090156+0000","last_change":"2024-09-10T19:07:42.668256+0000","last_active":"2024-09-10T19:08:38.090156+0000","last_peered":"2024-09-10T19:08:38.090156+0000","last_clean":"2024-09-10T19:08:38.090156+0000","last_became_active":"2024-09-10T19:07:18.744805+0000","last_became_peered":"2024-09-10T19:07:18.744805+0000","last_unstale":"2024-09-10T19:08:38.090156+0000","last_undegraded":"2024-09-10T19:08:38.090156+0000","last_fullsized":"2024-09-10T19:08:38.090156+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:42.668198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:03:46.710933+0000","last_clean_scrub_stamp":"2024-09-10T19:07:42.668198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:07.198960+0000","scrub_duration":0.0014811570000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.13","version":"70'81","reported_seq":359,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.623349+0000","last_change":"2024-09-10T19:07:42.333202+0000","last_active":"2024-09-10T19:08:37.623349+0000","last_peered":"2024-09-10T19:08:37.623349+0000","last_clean":"2024-09-10T19:08:37.623349+0000","last_became_active":"2024-09-10T19:07:31.174330+0000","last_became_peered":"2024-09-10T19:07:31.174330+0000","last_unstale":"2024-09-10T19:08:37.623349+0000","last_undegraded":"2024-09-10T19:08:37.623349+0000","last_fullsized":"2024-09-10T19:08:37.623349+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:07:42.333149+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:07:42.333149+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:11.231065+0000","scrub_duration":0.0022502709999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"427'25","reported_seq":594,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:09:01.816674+0000","last_change":"2024-09-10T19:09:01.816639+0000","last_active":"2024-09-10T19:09:01.816674+0000","last_peered":"2024-09-10T19:09:01.816674+0000","last_clean":"2024-09-10T19:09:01.816674+0000","last_became_active":"2024-09-10T19:07:18.741530+0000","last_became_peered":"2024-09-10T19:07:18.741530+0000","last_unstale":"2024-09-10T19:09:01.816674+0000","last_undegraded":"2024-09-10T19:09:01.816674+0000","last_fullsized":"2024-09-10T19:09:01.816674+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'25","last_scrub_stamp":"2024-09-10T19:09:01.816578+0000","last_deep_scrub":"76'13","last_deep_scrub_stamp":"2024-09-10T18:46:24.626859+0000","last_clean_scrub_stamp":"2024-09-10T19:09:01.816578+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:06.193188+0000","scrub_duration":0.0020950389999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":59,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:55.281463+0000","last_change":"2024-09-10T19:08:55.281438+0000","last_active":"2024-09-10T19:08:55.281463+0000","last_peered":"2024-09-10T19:08:55.281463+0000","last_clean":"2024-09-10T19:08:55.281463+0000","last_became_active":"2024-09-10T19:07:21.871872+0000","last_became_peered":"2024-09-10T19:07:21.871872+0000","last_unstale":"2024-09-10T19:08:55.281463+0000","last_undegraded":"2024-09-10T19:08:55.281463+0000","last_fullsized":"2024-09-10T19:08:55.281463+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:55.281383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:07:22.144520+0000","last_clean_scrub_stamp":"2024-09-10T19:08:55.281383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:01.608331+0000","scrub_duration":0.0012327600000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":540,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.091153+0000","last_change":"2024-09-10T19:07:59.953800+0000","last_active":"2024-09-10T19:08:38.091153+0000","last_peered":"2024-09-10T19:08:38.091153+0000","last_clean":"2024-09-10T19:08:38.091153+0000","last_became_active":"2024-09-10T19:07:18.740503+0000","last_became_peered":"2024-09-10T19:07:18.740503+0000","last_unstale":"2024-09-10T19:08:38.091153+0000","last_undegraded":"2024-09-10T19:08:38.091153+0000","last_fullsized":"2024-09-10T19:08:38.091153+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:59.953745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:00.754912+0000","last_clean_scrub_stamp":"2024-09-10T19:07:59.953745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:29.248893+0000","scrub_duration":0.001465907,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"364'2","reported_seq":53,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.882590+0000","last_change":"2024-09-10T19:08:23.438820+0000","last_active":"2024-09-10T19:08:37.882590+0000","last_peered":"2024-09-10T19:08:37.882590+0000","last_clean":"2024-09-10T19:08:37.882590+0000","last_became_active":"2024-09-10T19:07:21.869572+0000","last_became_peered":"2024-09-10T19:07:21.869572+0000","last_unstale":"2024-09-10T19:08:37.882590+0000","last_undegraded":"2024-09-10T19:08:37.882590+0000","last_fullsized":"2024-09-10T19:08:37.882590+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"364'2","last_scrub_stamp":"2024-09-10T19:08:23.438750+0000","last_deep_scrub":"364'2","last_deep_scrub_stamp":"2024-09-10T19:02:15.059320+0000","last_clean_scrub_stamp":"2024-09-10T19:08:23.438750+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:39.690587+0000","scrub_duration":0.0021737689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":65,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.12","version":"70'81","reported_seq":394,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.000050+0000","last_change":"2024-09-10T19:08:21.550650+0000","last_active":"2024-09-10T19:08:38.000050+0000","last_peered":"2024-09-10T19:08:38.000050+0000","last_clean":"2024-09-10T19:08:38.000050+0000","last_became_active":"2024-09-10T18:52:16.598921+0000","last_became_peered":"2024-09-10T18:52:16.598921+0000","last_unstale":"2024-09-10T19:08:38.000050+0000","last_undegraded":"2024-09-10T19:08:38.000050+0000","last_fullsized":"2024-09-10T19:08:38.000050+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:08:21.550578+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T18:52:54.127476+0000","last_clean_scrub_stamp":"2024-09-10T19:08:21.550578+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:41.526899+0000","scrub_duration":0.001579219,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":573,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.999457+0000","last_change":"2024-09-10T19:07:42.639042+0000","last_active":"2024-09-10T19:08:37.999457+0000","last_peered":"2024-09-10T19:08:37.999457+0000","last_clean":"2024-09-10T19:08:37.999457+0000","last_became_active":"2024-09-10T19:07:18.745376+0000","last_became_peered":"2024-09-10T19:07:18.745376+0000","last_unstale":"2024-09-10T19:08:37.999457+0000","last_undegraded":"2024-09-10T19:08:37.999457+0000","last_fullsized":"2024-09-10T19:08:37.999457+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:42.638976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:07:42.638976+0000","last_clean_scrub_stamp":"2024-09-10T19:07:42.638976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:08.399814+0000","scrub_duration":0.0027179790000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,5],"acting":[6,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"477'21","reported_seq":581,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.091238+0000","last_change":"2024-09-10T19:07:47.764411+0000","last_active":"2024-09-10T19:08:38.091238+0000","last_peered":"2024-09-10T19:08:38.091238+0000","last_clean":"2024-09-10T19:08:38.091238+0000","last_became_active":"2024-09-10T19:07:18.744389+0000","last_became_peered":"2024-09-10T19:07:18.744389+0000","last_unstale":"2024-09-10T19:08:38.091238+0000","last_undegraded":"2024-09-10T19:08:38.091238+0000","last_fullsized":"2024-09-10T19:08:38.091238+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"477'21","last_scrub_stamp":"2024-09-10T19:07:47.764352+0000","last_deep_scrub":"477'21","last_deep_scrub_stamp":"2024-09-10T19:07:47.764352+0000","last_clean_scrub_stamp":"2024-09-10T19:07:47.764352+0000","objects_scrubbed":3,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:08:54.145582+0000","scrub_duration":0.0029292720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":24,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":522,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.000429+0000","last_change":"2024-09-10T19:08:22.507010+0000","last_active":"2024-09-10T19:08:38.000429+0000","last_peered":"2024-09-10T19:08:38.000429+0000","last_clean":"2024-09-10T19:08:38.000429+0000","last_became_active":"2024-09-10T18:45:22.069653+0000","last_became_peered":"2024-09-10T18:45:22.069653+0000","last_unstale":"2024-09-10T19:08:38.000429+0000","last_undegraded":"2024-09-10T19:08:38.000429+0000","last_fullsized":"2024-09-10T19:08:38.000429+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:22.506947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:08:22.506947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:23.445528+0000","scrub_duration":0.001519482,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":53,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090762+0000","last_change":"2024-09-10T19:08:00.345603+0000","last_active":"2024-09-10T19:08:38.090762+0000","last_peered":"2024-09-10T19:08:38.090762+0000","last_clean":"2024-09-10T19:08:38.090762+0000","last_became_active":"2024-09-10T19:07:18.747593+0000","last_became_peered":"2024-09-10T19:07:18.747593+0000","last_unstale":"2024-09-10T19:08:38.090762+0000","last_undegraded":"2024-09-10T19:08:38.090762+0000","last_fullsized":"2024-09-10T19:08:38.090762+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:00.345535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:08:00.345535+0000","last_clean_scrub_stamp":"2024-09-10T19:08:00.345535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:15.358692+0000","scrub_duration":0.0022865070000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.11","version":"70'81","reported_seq":430,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.621883+0000","last_change":"2024-09-10T19:07:48.748563+0000","last_active":"2024-09-10T19:08:37.621883+0000","last_peered":"2024-09-10T19:08:37.621883+0000","last_clean":"2024-09-10T19:08:37.621883+0000","last_became_active":"2024-09-10T19:00:42.517311+0000","last_became_peered":"2024-09-10T19:00:42.517311+0000","last_unstale":"2024-09-10T19:08:37.621883+0000","last_undegraded":"2024-09-10T19:08:37.621883+0000","last_fullsized":"2024-09-10T19:08:37.621883+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:07:48.748483+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:07:48.748483+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:16.397728+0000","scrub_duration":0.001963743,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"496'35","reported_seq":682,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:58.759252+0000","last_change":"2024-09-10T19:08:58.759200+0000","last_active":"2024-09-10T19:08:58.759252+0000","last_peered":"2024-09-10T19:08:58.759252+0000","last_clean":"2024-09-10T19:08:58.759252+0000","last_became_active":"2024-09-10T19:07:58.853107+0000","last_became_peered":"2024-09-10T19:07:58.853107+0000","last_unstale":"2024-09-10T19:08:58.759252+0000","last_undegraded":"2024-09-10T19:08:58.759252+0000","last_fullsized":"2024-09-10T19:08:58.759252+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":0,"last_scrub":"496'35","last_scrub_stamp":"2024-09-10T19:08:58.759128+0000","last_deep_scrub":"496'35","last_deep_scrub_stamp":"2024-09-10T19:08:58.759128+0000","last_clean_scrub_stamp":"2024-09-10T19:08:58.759128+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:08.120710+0000","scrub_duration":0.00451542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":556,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.999711+0000","last_change":"2024-09-10T19:08:37.591591+0000","last_active":"2024-09-10T19:08:37.999711+0000","last_peered":"2024-09-10T19:08:37.999711+0000","last_clean":"2024-09-10T19:08:37.999711+0000","last_became_active":"2024-09-10T19:07:18.728727+0000","last_became_peered":"2024-09-10T19:07:18.728727+0000","last_unstale":"2024-09-10T19:08:37.999711+0000","last_undegraded":"2024-09-10T19:08:37.999711+0000","last_fullsized":"2024-09-10T19:08:37.999711+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:37.591523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:08:37.591523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:43.402962+0000","scrub_duration":0.0013764280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"304'11","reported_seq":849,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:56.354838+0000","last_change":"2024-09-10T19:07:48.293221+0000","last_active":"2024-09-10T19:08:56.354838+0000","last_peered":"2024-09-10T19:08:56.354838+0000","last_clean":"2024-09-10T19:08:56.354838+0000","last_became_active":"2024-09-10T18:58:58.489370+0000","last_became_peered":"2024-09-10T18:58:58.489370+0000","last_unstale":"2024-09-10T19:08:56.354838+0000","last_undegraded":"2024-09-10T19:08:56.354838+0000","last_fullsized":"2024-09-10T19:08:56.354838+0000","mapping_epoch":303,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":304,"parent":"0.0","parent_split_bits":0,"last_scrub":"304'11","last_scrub_stamp":"2024-09-10T19:07:48.293171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:07:48.293171+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:06.354160+0000","scrub_duration":0.0022523560000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":225,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.623354+0000","last_change":"2024-09-10T19:07:58.762471+0000","last_active":"2024-09-10T19:08:37.623354+0000","last_peered":"2024-09-10T19:08:37.623354+0000","last_clean":"2024-09-10T19:08:37.623354+0000","last_became_active":"2024-09-10T19:07:18.728626+0000","last_became_peered":"2024-09-10T19:07:18.728626+0000","last_unstale":"2024-09-10T19:08:37.623354+0000","last_undegraded":"2024-09-10T19:08:37.623354+0000","last_fullsized":"2024-09-10T19:08:37.623354+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:58.762413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:52:57.476724+0000","last_clean_scrub_stamp":"2024-09-10T19:07:58.762413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:25.545987+0000","scrub_duration":0.0013127989999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.10","version":"70'81","reported_seq":404,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.999459+0000","last_change":"2024-09-10T19:08:02.499016+0000","last_active":"2024-09-10T19:08:37.999459+0000","last_peered":"2024-09-10T19:08:37.999459+0000","last_clean":"2024-09-10T19:08:37.999459+0000","last_became_active":"2024-09-10T19:07:18.736928+0000","last_became_peered":"2024-09-10T19:07:18.736928+0000","last_unstale":"2024-09-10T19:08:37.999459+0000","last_undegraded":"2024-09-10T19:08:37.999459+0000","last_fullsized":"2024-09-10T19:08:37.999459+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:08:02.498946+0000","last_deep_scrub":"39'74","last_deep_scrub_stamp":"2024-09-10T18:44:18.839142+0000","last_clean_scrub_stamp":"2024-09-10T19:08:02.498946+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:29.176994+0000","scrub_duration":0.001578643,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"427'21","reported_seq":235,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:56.713248+0000","last_change":"2024-09-10T19:08:56.713210+0000","last_active":"2024-09-10T19:08:56.713248+0000","last_peered":"2024-09-10T19:08:56.713248+0000","last_clean":"2024-09-10T19:08:56.713248+0000","last_became_active":"2024-09-10T19:07:18.735867+0000","last_became_peered":"2024-09-10T19:07:18.735867+0000","last_unstale":"2024-09-10T19:08:56.713248+0000","last_undegraded":"2024-09-10T19:08:56.713248+0000","last_fullsized":"2024-09-10T19:08:56.713248+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'21","last_scrub_stamp":"2024-09-10T19:08:56.713143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:08:56.713143+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:05.888161+0000","scrub_duration":0.0017699300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":33,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":514,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:09:02.814647+0000","last_change":"2024-09-10T19:09:02.814621+0000","last_active":"2024-09-10T19:09:02.814647+0000","last_peered":"2024-09-10T19:09:02.814647+0000","last_clean":"2024-09-10T19:09:02.814647+0000","last_became_active":"2024-09-10T18:55:40.390697+0000","last_became_peered":"2024-09-10T18:55:40.390697+0000","last_unstale":"2024-09-10T19:09:02.814647+0000","last_undegraded":"2024-09-10T19:09:02.814647+0000","last_fullsized":"2024-09-10T19:09:02.814647+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:09:02.814568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:09:02.814568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:16.155858+0000","scrub_duration":0.0015870960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"465'13","reported_seq":861,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:56.354542+0000","last_change":"2024-09-10T19:07:44.727669+0000","last_active":"2024-09-10T19:08:56.354542+0000","last_peered":"2024-09-10T19:08:56.354542+0000","last_clean":"2024-09-10T19:08:56.354542+0000","last_became_active":"2024-09-10T19:07:21.862583+0000","last_became_peered":"2024-09-10T19:07:21.862583+0000","last_unstale":"2024-09-10T19:08:56.354542+0000","last_undegraded":"2024-09-10T19:08:56.354542+0000","last_fullsized":"2024-09-10T19:08:56.354542+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"465'13","last_scrub_stamp":"2024-09-10T19:07:44.727587+0000","last_deep_scrub":"273'11","last_deep_scrub_stamp":"2024-09-10T19:03:49.744177+0000","last_clean_scrub_stamp":"2024-09-10T19:07:44.727587+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:08.978598+0000","scrub_duration":0.0024831419999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":556,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.625600+0000","last_change":"2024-09-10T19:08:31.955198+0000","last_active":"2024-09-10T19:08:37.625600+0000","last_peered":"2024-09-10T19:08:37.625600+0000","last_clean":"2024-09-10T19:08:37.625600+0000","last_became_active":"2024-09-10T19:07:21.864485+0000","last_became_peered":"2024-09-10T19:07:21.864485+0000","last_unstale":"2024-09-10T19:08:37.625600+0000","last_undegraded":"2024-09-10T19:08:37.625600+0000","last_fullsized":"2024-09-10T19:08:37.625600+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:31.955134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:46:36.305199+0000","last_clean_scrub_stamp":"2024-09-10T19:08:31.955134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:59.445634+0000","scrub_duration":0.001675099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"427'21","reported_seq":398,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.622961+0000","last_change":"2024-09-10T19:08:27.455864+0000","last_active":"2024-09-10T19:08:37.622961+0000","last_peered":"2024-09-10T19:08:37.622961+0000","last_clean":"2024-09-10T19:08:37.622961+0000","last_became_active":"2024-09-10T19:07:31.172489+0000","last_became_peered":"2024-09-10T19:07:31.172489+0000","last_unstale":"2024-09-10T19:08:37.622961+0000","last_undegraded":"2024-09-10T19:08:37.622961+0000","last_fullsized":"2024-09-10T19:08:37.622961+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'21","last_scrub_stamp":"2024-09-10T19:08:27.455794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:08:27.455794+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:55.978396+0000","scrub_duration":0.0021581159999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":33,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":577,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.000044+0000","last_change":"2024-09-10T19:08:19.699615+0000","last_active":"2024-09-10T19:08:38.000044+0000","last_peered":"2024-09-10T19:08:38.000044+0000","last_clean":"2024-09-10T19:08:38.000044+0000","last_became_active":"2024-09-10T19:00:42.518310+0000","last_became_peered":"2024-09-10T19:00:42.518310+0000","last_unstale":"2024-09-10T19:08:38.000044+0000","last_undegraded":"2024-09-10T19:08:38.000044+0000","last_fullsized":"2024-09-10T19:08:38.000044+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:19.699546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:08:19.699546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:46.708862+0000","scrub_duration":0.001746081,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":527,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:53.809578+0000","last_change":"2024-09-10T19:08:53.809553+0000","last_active":"2024-09-10T19:08:53.809578+0000","last_peered":"2024-09-10T19:08:53.809578+0000","last_clean":"2024-09-10T19:08:53.809578+0000","last_became_active":"2024-09-10T18:55:40.398243+0000","last_became_peered":"2024-09-10T18:55:40.398243+0000","last_unstale":"2024-09-10T19:08:53.809578+0000","last_undegraded":"2024-09-10T19:08:53.809578+0000","last_fullsized":"2024-09-10T19:08:53.809578+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:53.809498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:03.571786+0000","last_clean_scrub_stamp":"2024-09-10T19:08:53.809498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:13.709684+0000","scrub_duration":0.001244666,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":519,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.621992+0000","last_change":"2024-09-10T19:08:14.749177+0000","last_active":"2024-09-10T19:08:37.621992+0000","last_peered":"2024-09-10T19:08:37.621992+0000","last_clean":"2024-09-10T19:08:37.621992+0000","last_became_active":"2024-09-10T19:00:42.516833+0000","last_became_peered":"2024-09-10T19:00:42.516833+0000","last_unstale":"2024-09-10T19:08:37.621992+0000","last_undegraded":"2024-09-10T19:08:37.621992+0000","last_fullsized":"2024-09-10T19:08:37.621992+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:14.749121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:50:04.631558+0000","last_clean_scrub_stamp":"2024-09-10T19:08:14.749121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:33.382416+0000","scrub_duration":0.001392431,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"427'32","reported_seq":571,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:52.995569+0000","last_change":"2024-09-10T19:08:52.995534+0000","last_active":"2024-09-10T19:08:52.995569+0000","last_peered":"2024-09-10T19:08:52.995569+0000","last_clean":"2024-09-10T19:08:52.995569+0000","last_became_active":"2024-09-10T18:45:20.056885+0000","last_became_peered":"2024-09-10T18:45:20.056885+0000","last_unstale":"2024-09-10T19:08:52.995569+0000","last_undegraded":"2024-09-10T19:08:52.995569+0000","last_fullsized":"2024-09-10T19:08:52.995569+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'32","last_scrub_stamp":"2024-09-10T19:08:52.995463+0000","last_deep_scrub":"223'16","last_deep_scrub_stamp":"2024-09-10T18:54:47.324044+0000","last_clean_scrub_stamp":"2024-09-10T19:08:52.995463+0000","objects_scrubbed":5,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:04.448078+0000","scrub_duration":0.002102151,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":450,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.623833+0000","last_change":"2024-09-10T19:07:51.360152+0000","last_active":"2024-09-10T19:08:37.623833+0000","last_peered":"2024-09-10T19:08:37.623833+0000","last_clean":"2024-09-10T19:08:37.623833+0000","last_became_active":"2024-09-10T19:07:31.169826+0000","last_became_peered":"2024-09-10T19:07:31.169826+0000","last_unstale":"2024-09-10T19:08:37.623833+0000","last_undegraded":"2024-09-10T19:08:37.623833+0000","last_fullsized":"2024-09-10T19:08:37.623833+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:51.360097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:00.029389+0000","last_clean_scrub_stamp":"2024-09-10T19:07:51.360097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:16.451579+0000","scrub_duration":0.0036568999999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":432,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:56.707495+0000","last_change":"2024-09-10T19:08:56.707466+0000","last_active":"2024-09-10T19:08:56.707495+0000","last_peered":"2024-09-10T19:08:56.707495+0000","last_clean":"2024-09-10T19:08:56.707495+0000","last_became_active":"2024-09-10T19:07:31.174416+0000","last_became_peered":"2024-09-10T19:07:31.174416+0000","last_unstale":"2024-09-10T19:08:56.707495+0000","last_undegraded":"2024-09-10T19:08:56.707495+0000","last_fullsized":"2024-09-10T19:08:56.707495+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:56.707409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:08:56.707409+0000","last_clean_scrub_stamp":"2024-09-10T19:08:56.707409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:07.472583+0000","scrub_duration":0.001457666,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":525,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090386+0000","last_change":"2024-09-10T19:08:00.930572+0000","last_active":"2024-09-10T19:08:38.090386+0000","last_peered":"2024-09-10T19:08:38.090386+0000","last_clean":"2024-09-10T19:08:38.090386+0000","last_became_active":"2024-09-10T18:55:40.395595+0000","last_became_peered":"2024-09-10T18:55:40.395595+0000","last_unstale":"2024-09-10T19:08:38.090386+0000","last_undegraded":"2024-09-10T19:08:38.090386+0000","last_fullsized":"2024-09-10T19:08:38.090386+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:00.930516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:23.053377+0000","last_clean_scrub_stamp":"2024-09-10T19:08:00.930516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:28.285123+0000","scrub_duration":0.003078005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"496'14","reported_seq":542,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:49.878817+0000","last_change":"2024-09-10T19:08:49.878776+0000","last_active":"2024-09-10T19:08:49.878817+0000","last_peered":"2024-09-10T19:08:49.878817+0000","last_clean":"2024-09-10T19:08:49.878817+0000","last_became_active":"2024-09-10T18:45:20.059205+0000","last_became_peered":"2024-09-10T18:45:20.059205+0000","last_unstale":"2024-09-10T19:08:49.878817+0000","last_undegraded":"2024-09-10T19:08:49.878817+0000","last_fullsized":"2024-09-10T19:08:49.878817+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"496'14","last_scrub_stamp":"2024-09-10T19:08:49.878709+0000","last_deep_scrub":"496'14","last_deep_scrub_stamp":"2024-09-10T19:08:49.878709+0000","last_clean_scrub_stamp":"2024-09-10T19:08:49.878709+0000","objects_scrubbed":2,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:14.266577+0000","scrub_duration":0.0033456639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":16,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":552,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.000503+0000","last_change":"2024-09-10T19:08:31.587485+0000","last_active":"2024-09-10T19:08:38.000503+0000","last_peered":"2024-09-10T19:08:38.000503+0000","last_clean":"2024-09-10T19:08:38.000503+0000","last_became_active":"2024-09-10T19:07:21.861473+0000","last_became_peered":"2024-09-10T19:07:21.861473+0000","last_unstale":"2024-09-10T19:08:38.000503+0000","last_undegraded":"2024-09-10T19:08:38.000503+0000","last_fullsized":"2024-09-10T19:08:38.000503+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:31.587412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:08:31.587412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:47.798853+0000","scrub_duration":0.001726331,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":563,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.999865+0000","last_change":"2024-09-10T19:08:00.729598+0000","last_active":"2024-09-10T19:08:37.999865+0000","last_peered":"2024-09-10T19:08:37.999865+0000","last_clean":"2024-09-10T19:08:37.999865+0000","last_became_active":"2024-09-10T19:07:18.735293+0000","last_became_peered":"2024-09-10T19:07:18.735293+0000","last_unstale":"2024-09-10T19:08:37.999865+0000","last_undegraded":"2024-09-10T19:08:37.999865+0000","last_fullsized":"2024-09-10T19:08:37.999865+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:00.729528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:08:00.729528+0000","last_clean_scrub_stamp":"2024-09-10T19:08:00.729528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:21.429536+0000","scrub_duration":0.0014191550000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":73,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:48.327963+0000","last_change":"2024-09-10T19:08:48.327919+0000","last_active":"2024-09-10T19:08:48.327963+0000","last_peered":"2024-09-10T19:08:48.327963+0000","last_clean":"2024-09-10T19:08:48.327963+0000","last_became_active":"2024-09-10T19:07:21.857464+0000","last_became_peered":"2024-09-10T19:07:21.857464+0000","last_unstale":"2024-09-10T19:08:48.327963+0000","last_undegraded":"2024-09-10T19:08:48.327963+0000","last_fullsized":"2024-09-10T19:08:48.327963+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:48.327842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:55.145638+0000","last_clean_scrub_stamp":"2024-09-10T19:08:48.327842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:01.011992+0000","scrub_duration":0.0018529830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.14","version":"70'81","reported_seq":59,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:54.261915+0000","last_change":"2024-09-10T19:08:54.261870+0000","last_active":"2024-09-10T19:08:54.261915+0000","last_peered":"2024-09-10T19:08:54.261915+0000","last_clean":"2024-09-10T19:08:54.261915+0000","last_became_active":"2024-09-10T19:07:21.869220+0000","last_became_peered":"2024-09-10T19:07:21.869220+0000","last_unstale":"2024-09-10T19:08:54.261915+0000","last_undegraded":"2024-09-10T19:08:54.261915+0000","last_fullsized":"2024-09-10T19:08:54.261915+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":5,"last_scrub":"70'81","last_scrub_stamp":"2024-09-10T19:08:54.261813+0000","last_deep_scrub":"70'81","last_deep_scrub_stamp":"2024-09-10T19:07:34.099033+0000","last_clean_scrub_stamp":"2024-09-10T19:08:54.261813+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:00.497853+0000","scrub_duration":0.001847813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"427'32","reported_seq":604,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090136+0000","last_change":"2024-09-10T19:07:34.601600+0000","last_active":"2024-09-10T19:08:38.090136+0000","last_peered":"2024-09-10T19:08:38.090136+0000","last_clean":"2024-09-10T19:08:38.090136+0000","last_became_active":"2024-09-10T19:07:18.748467+0000","last_became_peered":"2024-09-10T19:07:18.748467+0000","last_unstale":"2024-09-10T19:08:38.090136+0000","last_undegraded":"2024-09-10T19:08:38.090136+0000","last_fullsized":"2024-09-10T19:08:38.090136+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'32","last_scrub_stamp":"2024-09-10T19:07:34.601536+0000","last_deep_scrub":"427'32","last_deep_scrub_stamp":"2024-09-10T19:07:34.601536+0000","last_clean_scrub_stamp":"2024-09-10T19:07:34.601536+0000","objects_scrubbed":5,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:08:57.681075+0000","scrub_duration":0.0042432520000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"56'1","reported_seq":585,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:58.554838+0000","last_change":"2024-09-10T19:08:58.554789+0000","last_active":"2024-09-10T19:08:58.554838+0000","last_peered":"2024-09-10T19:08:58.554838+0000","last_clean":"2024-09-10T19:08:58.554838+0000","last_became_active":"2024-09-10T19:07:21.863281+0000","last_became_peered":"2024-09-10T19:07:21.863281+0000","last_unstale":"2024-09-10T19:08:58.554838+0000","last_undegraded":"2024-09-10T19:08:58.554838+0000","last_fullsized":"2024-09-10T19:08:58.554838+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'1","last_scrub_stamp":"2024-09-10T19:08:58.554722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:08:58.554722+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:07.497294+0000","scrub_duration":0.0016735700000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":724,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":53,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090281+0000","last_change":"2024-09-10T19:08:10.318322+0000","last_active":"2024-09-10T19:08:38.090281+0000","last_peered":"2024-09-10T19:08:38.090281+0000","last_clean":"2024-09-10T19:08:38.090281+0000","last_became_active":"2024-09-10T19:07:18.736879+0000","last_became_peered":"2024-09-10T19:07:18.736879+0000","last_unstale":"2024-09-10T19:08:38.090281+0000","last_undegraded":"2024-09-10T19:08:38.090281+0000","last_fullsized":"2024-09-10T19:08:38.090281+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:10.318253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:08:10.318253+0000","last_clean_scrub_stamp":"2024-09-10T19:08:10.318253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:25.532507+0000","scrub_duration":0.00153034,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":535,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090165+0000","last_change":"2024-09-10T19:07:58.918510+0000","last_active":"2024-09-10T19:08:38.090165+0000","last_peered":"2024-09-10T19:08:38.090165+0000","last_clean":"2024-09-10T19:08:38.090165+0000","last_became_active":"2024-09-10T18:55:40.397523+0000","last_became_peered":"2024-09-10T18:55:40.397523+0000","last_unstale":"2024-09-10T19:08:38.090165+0000","last_undegraded":"2024-09-10T19:08:38.090165+0000","last_fullsized":"2024-09-10T19:08:38.090165+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:58.918461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:06:20.199071+0000","last_clean_scrub_stamp":"2024-09-10T19:07:58.918461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:08:59.082533+0000","scrub_duration":0.0029530379999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"78'2","reported_seq":71,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090931+0000","last_change":"2024-09-10T19:08:17.397159+0000","last_active":"2024-09-10T19:08:38.090931+0000","last_peered":"2024-09-10T19:08:38.090931+0000","last_clean":"2024-09-10T19:08:38.090931+0000","last_became_active":"2024-09-10T19:07:21.860858+0000","last_became_peered":"2024-09-10T19:07:21.860858+0000","last_unstale":"2024-09-10T19:08:38.090931+0000","last_undegraded":"2024-09-10T19:08:38.090931+0000","last_fullsized":"2024-09-10T19:08:38.090931+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'2","last_scrub_stamp":"2024-09-10T19:08:17.397089+0000","last_deep_scrub":"78'2","last_deep_scrub_stamp":"2024-09-10T19:02:15.686138+0000","last_clean_scrub_stamp":"2024-09-10T19:08:17.397089+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:45.895036+0000","scrub_duration":0.001821238,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":738,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":546,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.999553+0000","last_change":"2024-09-10T19:07:58.383056+0000","last_active":"2024-09-10T19:08:37.999553+0000","last_peered":"2024-09-10T19:08:37.999553+0000","last_clean":"2024-09-10T19:08:37.999553+0000","last_became_active":"2024-09-10T19:07:18.745185+0000","last_became_peered":"2024-09-10T19:07:18.745185+0000","last_unstale":"2024-09-10T19:08:37.999553+0000","last_undegraded":"2024-09-10T19:08:37.999553+0000","last_fullsized":"2024-09-10T19:08:37.999553+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:58.382907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:07:58.382907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:21.843482+0000","scrub_duration":0.0021153690000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"427'30","reported_seq":590,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.623501+0000","last_change":"2024-09-10T19:07:47.791846+0000","last_active":"2024-09-10T19:08:37.623501+0000","last_peered":"2024-09-10T19:08:37.623501+0000","last_clean":"2024-09-10T19:08:37.623501+0000","last_became_active":"2024-09-10T19:07:18.740753+0000","last_became_peered":"2024-09-10T19:07:18.740753+0000","last_unstale":"2024-09-10T19:08:37.623501+0000","last_undegraded":"2024-09-10T19:08:37.623501+0000","last_fullsized":"2024-09-10T19:08:37.623501+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'30","last_scrub_stamp":"2024-09-10T19:07:47.791781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:19.033249+0000","last_clean_scrub_stamp":"2024-09-10T19:07:47.791781+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:12.941691+0000","scrub_duration":0.001737281,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":38,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"464'11","reported_seq":868,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:56.354887+0000","last_change":"2024-09-10T19:08:22.668669+0000","last_active":"2024-09-10T19:08:56.354887+0000","last_peered":"2024-09-10T19:08:56.354887+0000","last_clean":"2024-09-10T19:08:56.354887+0000","last_became_active":"2024-09-10T19:07:18.746575+0000","last_became_peered":"2024-09-10T19:07:18.746575+0000","last_unstale":"2024-09-10T19:08:56.354887+0000","last_undegraded":"2024-09-10T19:08:56.354887+0000","last_fullsized":"2024-09-10T19:08:56.354887+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"464'11","last_scrub_stamp":"2024-09-10T19:08:22.668594+0000","last_deep_scrub":"464'11","last_deep_scrub_stamp":"2024-09-10T19:08:22.668594+0000","last_clean_scrub_stamp":"2024-09-10T19:08:22.668594+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:40.683399+0000","scrub_duration":0.001681697,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":427,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.623509+0000","last_change":"2024-09-10T19:08:21.482218+0000","last_active":"2024-09-10T19:08:37.623509+0000","last_peered":"2024-09-10T19:08:37.623509+0000","last_clean":"2024-09-10T19:08:37.623509+0000","last_became_active":"2024-09-10T19:07:31.169258+0000","last_became_peered":"2024-09-10T19:07:31.169258+0000","last_unstale":"2024-09-10T19:08:37.623509+0000","last_undegraded":"2024-09-10T19:08:37.623509+0000","last_fullsized":"2024-09-10T19:08:37.623509+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:21.482163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:47:03.475299+0000","last_clean_scrub_stamp":"2024-09-10T19:08:21.482163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:39.197709+0000","scrub_duration":0.001695834,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":531,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.089873+0000","last_change":"2024-09-10T19:07:49.807776+0000","last_active":"2024-09-10T19:08:38.089873+0000","last_peered":"2024-09-10T19:08:38.089873+0000","last_clean":"2024-09-10T19:08:38.089873+0000","last_became_active":"2024-09-10T18:55:40.392524+0000","last_became_peered":"2024-09-10T18:55:40.392524+0000","last_unstale":"2024-09-10T19:08:38.089873+0000","last_undegraded":"2024-09-10T19:08:38.089873+0000","last_fullsized":"2024-09-10T19:08:38.089873+0000","mapping_epoch":241,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":242,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:49.807719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:07:49.807719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:07.698247+0000","scrub_duration":0.0014287670000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"467'32","reported_seq":596,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.001075+0000","last_change":"2024-09-10T19:08:32.507503+0000","last_active":"2024-09-10T19:08:38.001075+0000","last_peered":"2024-09-10T19:08:38.001075+0000","last_clean":"2024-09-10T19:08:38.001075+0000","last_became_active":"2024-09-10T18:45:20.058717+0000","last_became_peered":"2024-09-10T18:45:20.058717+0000","last_unstale":"2024-09-10T19:08:38.001075+0000","last_undegraded":"2024-09-10T19:08:38.001075+0000","last_fullsized":"2024-09-10T19:08:38.001075+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"467'32","last_scrub_stamp":"2024-09-10T19:08:32.507433+0000","last_deep_scrub":"467'32","last_deep_scrub_stamp":"2024-09-10T19:07:28.508578+0000","last_clean_scrub_stamp":"2024-09-10T19:08:32.507433+0000","objects_scrubbed":5,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:41.206760+0000","scrub_duration":0.0020411029999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":527,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.623617+0000","last_change":"2024-09-10T19:08:12.749647+0000","last_active":"2024-09-10T19:08:37.623617+0000","last_peered":"2024-09-10T19:08:37.623617+0000","last_clean":"2024-09-10T19:08:37.623617+0000","last_became_active":"2024-09-10T19:07:21.854982+0000","last_became_peered":"2024-09-10T19:07:21.854982+0000","last_unstale":"2024-09-10T19:08:37.623617+0000","last_undegraded":"2024-09-10T19:08:37.623617+0000","last_fullsized":"2024-09-10T19:08:37.623617+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:12.749588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:08:12.749588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:35.434829+0000","scrub_duration":0.001727239,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":519,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:37.625594+0000","last_change":"2024-09-10T19:08:35.992279+0000","last_active":"2024-09-10T19:08:37.625594+0000","last_peered":"2024-09-10T19:08:37.625594+0000","last_clean":"2024-09-10T19:08:37.625594+0000","last_became_active":"2024-09-10T18:45:24.267251+0000","last_became_peered":"2024-09-10T18:45:24.267251+0000","last_unstale":"2024-09-10T19:08:37.625594+0000","last_undegraded":"2024-09-10T19:08:37.625594+0000","last_fullsized":"2024-09-10T19:08:37.625594+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:35.992224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:07:24.058804+0000","last_clean_scrub_stamp":"2024-09-10T19:08:35.992224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:44.279878+0000","scrub_duration":0.0013396269999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"306'2","reported_seq":578,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.000452+0000","last_change":"2024-09-10T19:08:28.601563+0000","last_active":"2024-09-10T19:08:38.000452+0000","last_peered":"2024-09-10T19:08:38.000452+0000","last_clean":"2024-09-10T19:08:38.000452+0000","last_became_active":"2024-09-10T19:07:18.745262+0000","last_became_peered":"2024-09-10T19:07:18.745262+0000","last_unstale":"2024-09-10T19:08:38.000452+0000","last_undegraded":"2024-09-10T19:08:38.000452+0000","last_fullsized":"2024-09-10T19:08:38.000452+0000","mapping_epoch":461,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":462,"parent":"0.0","parent_split_bits":0,"last_scrub":"306'2","last_scrub_stamp":"2024-09-10T19:08:28.601496+0000","last_deep_scrub":"306'2","last_deep_scrub_stamp":"2024-09-10T18:59:20.534869+0000","last_clean_scrub_stamp":"2024-09-10T19:08:28.601496+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:29.281677+0000","scrub_duration":0.0015540160000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1748,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"427'46","reported_seq":75,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:42.258164+0000","last_change":"2024-09-10T19:08:42.258110+0000","last_active":"2024-09-10T19:08:42.258164+0000","last_peered":"2024-09-10T19:08:42.258164+0000","last_clean":"2024-09-10T19:08:42.258164+0000","last_became_active":"2024-09-10T19:07:21.860334+0000","last_became_peered":"2024-09-10T19:07:21.860334+0000","last_unstale":"2024-09-10T19:08:42.258164+0000","last_undegraded":"2024-09-10T19:08:42.258164+0000","last_fullsized":"2024-09-10T19:08:42.258164+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'46","last_scrub_stamp":"2024-09-10T19:08:42.258053+0000","last_deep_scrub":"258'34","last_deep_scrub_stamp":"2024-09-10T18:56:23.838951+0000","last_clean_scrub_stamp":"2024-09-10T19:08:42.258053+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:49.938751+0000","scrub_duration":0.001904918,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":60,"num_read_kb":51,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":436,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:41.552765+0000","last_change":"2024-09-10T19:08:41.552705+0000","last_active":"2024-09-10T19:08:41.552765+0000","last_peered":"2024-09-10T19:08:41.552765+0000","last_clean":"2024-09-10T19:08:41.552765+0000","last_became_active":"2024-09-10T19:07:31.169897+0000","last_became_peered":"2024-09-10T19:07:31.169897+0000","last_unstale":"2024-09-10T19:08:41.552765+0000","last_undegraded":"2024-09-10T19:08:41.552765+0000","last_fullsized":"2024-09-10T19:08:41.552765+0000","mapping_epoch":469,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":470,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:41.552642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:21.043052+0000","last_clean_scrub_stamp":"2024-09-10T19:08:41.552642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:51.019574+0000","scrub_duration":0.0017133509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":528,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.091130+0000","last_change":"2024-09-10T19:07:36.612678+0000","last_active":"2024-09-10T19:08:38.091130+0000","last_peered":"2024-09-10T19:08:38.091130+0000","last_clean":"2024-09-10T19:08:38.091130+0000","last_became_active":"2024-09-10T19:00:42.520239+0000","last_became_peered":"2024-09-10T19:00:42.520239+0000","last_unstale":"2024-09-10T19:08:38.091130+0000","last_undegraded":"2024-09-10T19:08:38.091130+0000","last_fullsized":"2024-09-10T19:08:38.091130+0000","mapping_epoch":338,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:36.612617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:59:17.985545+0000","last_clean_scrub_stamp":"2024-09-10T19:07:36.612617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:05.452711+0000","scrub_duration":0.0016732839999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":554,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.090458+0000","last_change":"2024-09-10T19:07:58.917961+0000","last_active":"2024-09-10T19:08:38.090458+0000","last_peered":"2024-09-10T19:08:38.090458+0000","last_clean":"2024-09-10T19:08:38.090458+0000","last_became_active":"2024-09-10T19:07:58.917683+0000","last_became_peered":"2024-09-10T19:07:58.917683+0000","last_unstale":"2024-09-10T19:08:38.090458+0000","last_undegraded":"2024-09-10T19:08:38.090458+0000","last_fullsized":"2024-09-10T19:08:38.090458+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:07:33.551339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:45:16.944036+0000","last_clean_scrub_stamp":"2024-09-10T19:07:33.551339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:08:53.686595+0000","scrub_duration":0.0018062569999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"427'29","reported_seq":558,"reported_epoch":495,"state":"active+clean","last_fresh":"2024-09-10T19:08:38.091086+0000","last_change":"2024-09-10T19:07:45.772541+0000","last_active":"2024-09-10T19:08:38.091086+0000","last_peered":"2024-09-10T19:08:38.091086+0000","last_clean":"2024-09-10T19:08:38.091086+0000","last_became_active":"2024-09-10T18:45:20.058391+0000","last_became_peered":"2024-09-10T18:45:20.058391+0000","last_unstale":"2024-09-10T19:08:38.091086+0000","last_undegraded":"2024-09-10T19:08:38.091086+0000","last_fullsized":"2024-09-10T19:08:38.091086+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"427'29","last_scrub_stamp":"2024-09-10T19:07:45.772483+0000","last_deep_scrub":"388'23","last_deep_scrub_stamp":"2024-09-10T19:03:40.697702+0000","last_clean_scrub_stamp":"2024-09-10T19:07:45.772483+0000","objects_scrubbed":5,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:09:04.451769+0000","scrub_duration":0.0019090470000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":571,"reported_epoch":496,"state":"active+clean","last_fresh":"2024-09-10T19:08:57.955353+0000","last_change":"2024-09-10T19:08:57.955316+0000","last_active":"2024-09-10T19:08:57.955353+0000","last_peered":"2024-09-10T19:08:57.955353+0000","last_clean":"2024-09-10T19:08:57.955353+0000","last_became_active":"2024-09-10T19:07:21.863857+0000","last_became_peered":"2024-09-10T19:07:21.863857+0000","last_unstale":"2024-09-10T19:08:57.955353+0000","last_undegraded":"2024-09-10T19:08:57.955353+0000","last_fullsized":"2024-09-10T19:08:57.955353+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:08:57.955246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:08:57.955246+0000","last_clean_scrub_stamp":"2024-09-10T19:08:57.955246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-10T19:10:09.298496+0000","scrub_duration":0.0019214060000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":803,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1539,"num_read_kb":1296,"num_write":989,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":329,"num_bytes_recovered":4736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1082,"ondisk_log_size":1082,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1121,"num_read_kb":1116,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":525376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1701,"ondisk_log_size":1701,"up":63,"acting":63,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":411,"seq":1765231558708,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":320480,"kb_used_data":728,"kb_used_omap":4,"kb_used_meta":319739,"kb_avail":93424672,"statfs":{"total":95995035648,"available":95666864128,"internally_reserved":0,"allocated":745472,"data_stored":369368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":327413254},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":6.1959999999999997,"15min":8.641},"min":{"1min":0.34000000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.77500000000000002,"5min":100.34,"15min":100.34},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.94699999999999995,"5min":6.4859999999999998,"15min":8.9469999999999992},"min":{"1min":0.378,"5min":0.245,"15min":0.245},"max":{"1min":8.3339999999999996,"5min":101.623,"15min":101.623},"last":0.434}]},{"osd":1,"last update":"Tue Sep 10 19:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":5.9930000000000003,"15min":8.3829999999999991},"min":{"1min":0.32300000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.73299999999999998,"5min":95.744,"15min":95.744},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.50600000000000001,"5min":6.0190000000000001,"15min":8.4309999999999992},"min":{"1min":0.371,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.69599999999999995,"5min":95.709999999999994,"15min":95.709999999999994},"last":0.377}]},{"osd":2,"last update":"Tue Sep 10 19:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.63800000000000001,"15min":0.56200000000000006},"min":{"1min":0.35499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":8.3840000000000003,"5min":8.3840000000000003,"15min":8.3840000000000003},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.56499999999999995,"15min":0.48199999999999998},"min":{"1min":0.38700000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72099999999999997,"5min":7.625,"15min":7.625},"last":0.497}]},{"osd":3,"last update":"Tue Sep 10 19:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":6.5529999999999999,"15min":9.1579999999999995},"min":{"1min":0.45000000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.751,"5min":96.021000000000001,"15min":96.021000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":6.5620000000000003,"15min":9.1539999999999999},"min":{"1min":0.35999999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77600000000000002,"5min":96,"15min":96},"last":0.55200000000000005}]},{"osd":4,"last update":"Tue Sep 10 19:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":2.2789999999999999,"15min":3.0470000000000002},"min":{"1min":0.28999999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.67200000000000004,"5min":66.769999999999996,"15min":66.769999999999996},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.48899999999999999,"5min":2.1930000000000001,"15min":2.9420000000000002},"min":{"1min":0.308,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.754,"5min":67.028999999999996,"15min":67.028999999999996},"last":0.58299999999999996}]},{"osd":5,"last update":"Tue Sep 10 19:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":2.1829999999999998,"15min":2.9180000000000001},"min":{"1min":0.32800000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.73399999999999999,"5min":67.450999999999993,"15min":67.450999999999993},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":2.2450000000000001,"15min":2.9740000000000002},"min":{"1min":0.34100000000000003,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72199999999999998,"5min":66.850999999999999,"15min":66.850999999999999},"last":0.56699999999999995}]},{"osd":6,"last update":"Tue Sep 10 19:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.63700000000000001,"15min":0.65200000000000002},"min":{"1min":0.34599999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.67800000000000005,"5min":2.5579999999999998,"15min":2.5579999999999998},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.69599999999999995,"15min":0.70199999999999996},"min":{"1min":0.40000000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.80000000000000004,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.63300000000000001}]}]},{"osd":6,"up_from":43,"seq":184683594023,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":895188,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":894142,"kb_avail":92849964,"statfs":{"total":95995035648,"available":95078363136,"internally_reserved":0,"allocated":1060864,"data_stored":697124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":915601752},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:08:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":1.9259999999999999,"15min":1.6799999999999999},"min":{"1min":0.34999999999999998,"5min":0.217,"15min":0.20899999999999999},"max":{"1min":0.70399999999999996,"5min":44.320999999999998,"15min":49.353999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.54800000000000004,"5min":2.0790000000000002,"15min":1.8080000000000001},"min":{"1min":0.33900000000000002,"5min":0.23799999999999999,"15min":0.20699999999999999},"max":{"1min":0.71599999999999997,"5min":44.311999999999998,"15min":49.341999999999999},"last":0.51900000000000002}]},{"osd":1,"last update":"Tue Sep 10 19:08:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":1.74,"15min":1.655},"min":{"1min":0.30499999999999999,"5min":0.23200000000000001,"15min":0.217},"max":{"1min":0.66300000000000003,"5min":85.775000000000006,"15min":85.775000000000006},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":1.639,"15min":1.552},"min":{"1min":0.34100000000000003,"5min":0.27100000000000002,"15min":0.23300000000000001},"max":{"1min":0.71399999999999997,"5min":85.670000000000002,"15min":85.670000000000002},"last":0.48899999999999999}]},{"osd":2,"last update":"Tue Sep 10 19:08:21 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.71699999999999997,"15min":0.81799999999999995},"min":{"1min":0.32000000000000001,"5min":0.27000000000000002,"15min":0.26000000000000001},"max":{"1min":0.63700000000000001,"5min":4.0469999999999997,"15min":23.867000000000001},"last":12.194000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.81299999999999994,"15min":0.86899999999999999},"min":{"1min":0.42799999999999999,"5min":0.28699999999999998,"15min":0.249},"max":{"1min":0.70199999999999996,"5min":7.8200000000000003,"15min":23.779},"last":0.42799999999999999}]},{"osd":3,"last update":"Tue Sep 10 19:08:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":1.042,"15min":1.6899999999999999},"min":{"1min":0.375,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.73399999999999999,"5min":20.994,"15min":76.977999999999994},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":1.0660000000000001,"15min":1.702},"min":{"1min":0.39100000000000001,"5min":0.25600000000000001,"15min":0.253},"max":{"1min":0.72699999999999998,"5min":20.177,"15min":76.879999999999995},"last":0.56399999999999995}]},{"osd":4,"last update":"Tue Sep 10 19:08:21 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.57099999999999995,"15min":1.478},"min":{"1min":0.32300000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.58999999999999997,"5min":2.746,"15min":165.03299999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.48899999999999999,"15min":1.337},"min":{"1min":0.32900000000000001,"5min":0.252,"15min":0.22500000000000001},"max":{"1min":0.67900000000000005,"5min":3.0819999999999999,"15min":163.959},"last":0.55200000000000005}]},{"osd":5,"last update":"Tue Sep 10 19:08:21 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.68300000000000005,"15min":1.847},"min":{"1min":0.32800000000000001,"5min":0.26300000000000001,"15min":0.246},"max":{"1min":0.66100000000000003,"5min":2.968,"15min":288.56999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.64200000000000002,"15min":1.8520000000000001},"min":{"1min":0.36899999999999999,"5min":0.25800000000000001,"15min":0.22600000000000001},"max":{"1min":0.67800000000000005,"5min":2.7269999999999999,"15min":291.05399999999997},"last":0.47099999999999997}]},{"osd":7,"last update":"Tue Sep 10 19:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.69599999999999995,"15min":0.70299999999999996},"min":{"1min":0.36199999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.75600000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57699999999999996,"15min":0.59399999999999997},"min":{"1min":0.41599999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.70799999999999996,"5min":1.702,"15min":1.702},"last":0.623}]}]},{"osd":5,"up_from":37,"seq":158913790251,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1185500,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":1184510,"kb_avail":92559652,"statfs":{"total":95995035648,"available":94781083648,"internally_reserved":0,"allocated":1003520,"data_stored":631414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":1212938584},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:08:05 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.97899999999999998,"15min":1.8380000000000001},"min":{"1min":0.28599999999999998,"5min":0.224,"15min":0.224},"max":{"1min":0.91700000000000004,"5min":14.31,"15min":82.465000000000003},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.55900000000000005,"5min":1.081,"15min":1.972},"min":{"1min":0.42899999999999999,"5min":0.27900000000000003,"15min":0.218},"max":{"1min":0.76900000000000002,"5min":14.323,"15min":82.519000000000005},"last":0.64000000000000001}]},{"osd":1,"last update":"Tue Sep 10 19:08:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":2.629,"15min":2.323},"min":{"1min":0.30199999999999999,"5min":0.23300000000000001,"15min":0.19500000000000001},"max":{"1min":0.85499999999999998,"5min":153.17099999999999,"15min":153.17099999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":2.6539999999999999,"15min":2.4900000000000002},"min":{"1min":0.437,"5min":0.29199999999999998,"15min":0.20399999999999999},"max":{"1min":0.72899999999999998,"5min":153.196,"15min":153.196},"last":0.53600000000000003}]},{"osd":2,"last update":"Tue Sep 10 19:08:05 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.61399999999999999,"15min":0.89800000000000002},"min":{"1min":0.32900000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.75800000000000001,"5min":3.0990000000000002,"15min":31.774000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.54500000000000004,"15min":0.85799999999999998},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.24399999999999999},"max":{"1min":0.73599999999999999,"5min":2.8799999999999999,"15min":31.722999999999999},"last":0.50600000000000001}]},{"osd":3,"last update":"Tue Sep 10 19:08:05 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.78400000000000003,"15min":2.492},"min":{"1min":0.36499999999999999,"5min":0.26300000000000001,"15min":0.24299999999999999},"max":{"1min":0.83299999999999996,"5min":3.0760000000000001,"15min":123.34399999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.79200000000000004,"15min":2.0339999999999998},"min":{"1min":0.40600000000000003,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.95499999999999996,"5min":5.117,"15min":82.415000000000006},"last":0.61299999999999999}]},{"osd":4,"last update":"Tue Sep 10 19:08:05 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.63800000000000001,"15min":1.1719999999999999},"min":{"1min":0.309,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.72399999999999998,"5min":5.0099999999999998,"15min":51.070999999999998},"last":0.434},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.57299999999999995,"15min":1.0549999999999999},"min":{"1min":0.27000000000000002,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.55600000000000005,"5min":3.524,"15min":51.122999999999998},"last":0.59699999999999998}]},{"osd":6,"last update":"Tue Sep 10 19:08:27 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.67400000000000004,"15min":1.0960000000000001},"min":{"1min":0.31900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.83899999999999997,"5min":4.0490000000000004,"15min":33.936999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.66000000000000003,"15min":1.0449999999999999},"min":{"1min":0.379,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.081,"5min":5.1710000000000003,"15min":33.951000000000001},"last":0.628}]},{"osd":7,"last update":"Tue Sep 10 19:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.59499999999999997,"15min":0.66400000000000003},"min":{"1min":0.34399999999999997,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.63600000000000001,"5min":2.657,"15min":2.657},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.66600000000000004,"15min":0.69999999999999996},"min":{"1min":0.36299999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67100000000000004,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":0.68300000000000005}]}]},{"osd":4,"up_from":30,"seq":128849019185,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1183588,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":1182846,"kb_avail":92561564,"statfs":{"total":95995035648,"available":94783041536,"internally_reserved":0,"allocated":749568,"data_stored":369219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":1211234648},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:08:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":1.2529999999999999,"15min":1.3839999999999999},"min":{"1min":0.33200000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.81899999999999995,"5min":40.481000000000002,"15min":46.234999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":1.296,"15min":1.3859999999999999},"min":{"1min":0.40799999999999997,"5min":0.253,"15min":0.253},"max":{"1min":0.91600000000000004,"5min":40.512,"15min":46.264000000000003},"last":0.439}]},{"osd":1,"last update":"Tue Sep 10 19:08:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":1.198,"15min":1.677},"min":{"1min":0.29599999999999999,"5min":0.23999999999999999,"15min":0.20899999999999999},"max":{"1min":0.69999999999999996,"5min":16.535,"15min":125.15600000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.998,"15min":1.5720000000000001},"min":{"1min":0.38100000000000001,"5min":0.24099999999999999,"15min":0.23100000000000001},"max":{"1min":0.73899999999999999,"5min":16.504000000000001,"15min":125.14700000000001},"last":7.6929999999999996}]},{"osd":2,"last update":"Tue Sep 10 19:08:42 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":0.74099999999999999,"15min":0.73199999999999998},"min":{"1min":0.38300000000000001,"5min":0.28699999999999998,"15min":0.25900000000000001},"max":{"1min":11.589,"5min":11.589,"15min":11.589},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":2.6560000000000001,"15min":1.4570000000000001},"min":{"1min":0.317,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.65900000000000003,"5min":190.38200000000001,"15min":190.38200000000001},"last":0.41699999999999998}]},{"osd":3,"last update":"Tue Sep 10 19:08:42 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":3.8799999999999999,"15min":2.98},"min":{"1min":0.35499999999999998,"5min":0.27200000000000002,"15min":0.249},"max":{"1min":0.67900000000000005,"5min":190.41300000000001,"15min":190.41300000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":1.7150000000000001,"15min":1.8999999999999999},"min":{"1min":0.34499999999999997,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.68200000000000005,"5min":39.323,"15min":110.914},"last":0.51400000000000001}]},{"osd":5,"last update":"Tue Sep 10 19:08:04 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.629,"15min":0.93600000000000005},"min":{"1min":0.26300000000000001,"5min":0.221,"15min":0.20499999999999999},"max":{"1min":0.623,"5min":8.1430000000000007,"15min":10.912000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.63500000000000001,"15min":0.91500000000000004},"min":{"1min":0.315,"5min":0.23200000000000001,"15min":0.216},"max":{"1min":0.96399999999999997,"5min":8.3170000000000002,"15min":10.147},"last":0.34599999999999997}]},{"osd":6,"last update":"Tue Sep 10 19:08:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58399999999999996,"15min":0.79400000000000004},"min":{"1min":0.33400000000000002,"5min":0.24099999999999999,"15min":0.221},"max":{"1min":0.86499999999999999,"5min":2.4100000000000001,"15min":8.4030000000000005},"last":0.503},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.66600000000000004,"15min":0.95299999999999996},"min":{"1min":0.38500000000000001,"5min":0.26300000000000001,"15min":0.23899999999999999},"max":{"1min":0.85099999999999998,"5min":4.0099999999999998,"15min":21.254000000000001},"last":0.55200000000000005}]},{"osd":7,"last update":"Tue Sep 10 19:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57899999999999996,"15min":0.56499999999999995},"min":{"1min":0.35999999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.872,"5min":3.472,"15min":3.472},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.48899999999999999,"5min":6.1909999999999998,"15min":8.6349999999999998},"min":{"1min":0.30199999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.74199999999999999,"5min":190.33799999999999,"15min":190.33799999999999},"last":0.48599999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182710,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1502576,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":1501246,"kb_avail":92242576,"statfs":{"total":95995035648,"available":94456397824,"internally_reserved":0,"allocated":1351680,"data_stored":959051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":1537276248},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:08:03 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":1.0509999999999999,"15min":2.343},"min":{"1min":0.221,"5min":0.182,"15min":0.17000000000000001},"max":{"1min":0.64600000000000002,"5min":6.3099999999999996,"15min":290.721},"last":0.438},{"interface":"front","average":{"1min":0.376,"5min":1.212,"15min":2.4980000000000002},"min":{"1min":0.23799999999999999,"5min":0.18099999999999999,"15min":0.17499999999999999},"max":{"1min":0.63200000000000001,"5min":12.103,"15min":293.59300000000002},"last":0.53100000000000003}]},{"osd":1,"last update":"Tue Sep 10 19:08:15 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":1.373,"15min":1.4299999999999999},"min":{"1min":0.246,"5min":0.18099999999999999,"15min":0.16200000000000001},"max":{"1min":0.82499999999999996,"5min":16.081,"15min":16.081},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.44400000000000001,"5min":1.514,"15min":1.361},"min":{"1min":0.28699999999999998,"5min":0.25600000000000001,"15min":0.188},"max":{"1min":0.86699999999999999,"5min":16.329000000000001,"15min":16.329000000000001},"last":0.48699999999999999}]},{"osd":2,"last update":"Tue Sep 10 19:08:15 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.83699999999999997,"15min":0.93300000000000005},"min":{"1min":0.29999999999999999,"5min":0.26200000000000001,"15min":0.22800000000000001},"max":{"1min":0.64200000000000002,"5min":4.2409999999999997,"15min":5.6680000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.82599999999999996,"15min":0.90100000000000002},"min":{"1min":0.251,"5min":0.24099999999999999,"15min":0.20599999999999999},"max":{"1min":0.57199999999999995,"5min":5.0250000000000004,"15min":5.46},"last":0.503}]},{"osd":4,"last update":"Tue Sep 10 19:08:48 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":1.044,"15min":1.3460000000000001},"min":{"1min":0.437,"5min":0.29699999999999999,"15min":0.247},"max":{"1min":0.90200000000000002,"5min":4.4580000000000002,"15min":29.013000000000002},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.65100000000000002,"5min":1.0029999999999999,"15min":1.369},"min":{"1min":0.44,"5min":0.30499999999999999,"15min":0.26800000000000002},"max":{"1min":0.83999999999999997,"5min":4.3760000000000003,"15min":28.826000000000001},"last":0.63600000000000001}]},{"osd":5,"last update":"Tue Sep 10 19:08:03 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":1.167,"15min":1.286},"min":{"1min":0.308,"5min":0.248,"15min":0.248},"max":{"1min":0.84199999999999997,"5min":5.1920000000000002,"15min":7.2300000000000004},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.628,"5min":1.2609999999999999,"15min":1.3320000000000001},"min":{"1min":0.42299999999999999,"5min":0.308,"15min":0.255},"max":{"1min":0.85099999999999998,"5min":13.117000000000001,"15min":17.899999999999999},"last":0.69099999999999995}]},{"osd":6,"last update":"Tue Sep 10 19:08:23 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":1.0840000000000001,"15min":1.2509999999999999},"min":{"1min":0.42499999999999999,"5min":0.307,"15min":0.30399999999999999},"max":{"1min":0.84899999999999998,"5min":5.8339999999999996,"15min":5.8339999999999996},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":1.0549999999999999,"15min":1.222},"min":{"1min":0.46100000000000002,"5min":0.32200000000000001,"15min":0.29399999999999998},"max":{"1min":0.82799999999999996,"5min":5.1340000000000003,"15min":6.6390000000000002},"last":0.71699999999999997}]},{"osd":7,"last update":"Tue Sep 10 19:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":1.012,"15min":1.278},"min":{"1min":0.44400000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.96599999999999997,"5min":4.8499999999999996,"15min":4.8499999999999996},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":1.333,"15min":1.9350000000000001},"min":{"1min":0.46000000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.1910000000000001,"5min":13.045999999999999,"15min":13.045999999999999},"last":0.73199999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411642,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":302484,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93442668,"statfs":{"total":95995035648,"available":95685292032,"internally_reserved":0,"allocated":995328,"data_stored":631385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:08:00 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":2.0569999999999999,"15min":1.3069999999999999},"min":{"1min":0.19900000000000001,"5min":0.189,"15min":0.17699999999999999},"max":{"1min":0.54900000000000004,"5min":64.134,"15min":64.134},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":2.1850000000000001,"15min":1.4710000000000001},"min":{"1min":0.245,"5min":0.192,"15min":0.192},"max":{"1min":0.59399999999999997,"5min":64.165999999999997,"15min":64.165999999999997},"last":0.438}]},{"osd":1,"last update":"Tue Sep 10 19:08:00 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":1.839,"15min":1.5920000000000001},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.186},"max":{"1min":0.51500000000000001,"5min":65.606999999999999,"15min":65.606999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":2.048,"15min":1.645},"min":{"1min":0.25900000000000001,"5min":0.21099999999999999,"15min":0.20000000000000001},"max":{"1min":0.67300000000000004,"5min":68.674000000000007,"15min":68.674000000000007},"last":0.378}]},{"osd":3,"last update":"Tue Sep 10 19:08:27 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":2.54,"15min":2.165},"min":{"1min":0.249,"5min":0.249,"15min":0.188},"max":{"1min":0.88600000000000001,"5min":65.207999999999998,"15min":164.47},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.42099999999999999,"5min":2.544,"15min":2.145},"min":{"1min":0.28999999999999998,"5min":0.251,"15min":0.21099999999999999},"max":{"1min":0.64200000000000002,"5min":70.965000000000003,"15min":164.321},"last":0.47199999999999998}]},{"osd":4,"last update":"Tue Sep 10 19:08:48 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.73099999999999998,"15min":1.7909999999999999},"min":{"1min":0.44700000000000001,"5min":0.29099999999999998,"15min":0.23499999999999999},"max":{"1min":0.77500000000000002,"5min":4.4279999999999999,"15min":167.86199999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.58599999999999997,"15min":1.675},"min":{"1min":0.40999999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.86199999999999999,"5min":3.3809999999999998,"15min":166.92699999999999},"last":0.58399999999999996}]},{"osd":5,"last update":"Tue Sep 10 19:08:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":3.0499999999999998,"15min":1.6319999999999999},"min":{"1min":0.372,"5min":0.27700000000000002,"15min":0.26100000000000001},"max":{"1min":0.73799999999999999,"5min":228.25399999999999,"15min":228.25399999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.54800000000000004,"5min":3.0070000000000001,"15min":1.6479999999999999},"min":{"1min":0.35999999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.65800000000000003,"5min":227.78100000000001,"15min":227.78100000000001},"last":0.63600000000000001}]},{"osd":6,"last update":"Tue Sep 10 19:08:48 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60999999999999999,"15min":0.997},"min":{"1min":0.436,"5min":0.32000000000000001,"15min":0.29899999999999999},"max":{"1min":0.85099999999999998,"5min":3.423,"15min":73.640000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.67100000000000004,"15min":0.99299999999999999},"min":{"1min":0.39500000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.88,"5min":3.2839999999999998,"15min":73.269999999999996},"last":0.62}]},{"osd":7,"last update":"Tue Sep 10 19:08:56 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.67500000000000004,"15min":0.69899999999999995},"min":{"1min":0.38700000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.86299999999999999,"5min":3.5179999999999998,"15min":3.5179999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65800000000000003,"15min":0.68000000000000005},"min":{"1min":0.47199999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89300000000000002,"5min":3.4609999999999999,"15min":3.4609999999999999},"last":0.69799999999999995}]}]},{"osd":1,"up_from":13,"seq":55834575166,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1505076,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":1504318,"kb_avail":92240076,"statfs":{"total":95995035648,"available":94453837824,"internally_reserved":0,"allocated":765952,"data_stored":369273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":1540421976},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:08:06 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.94299999999999995,"15min":1.756},"min":{"1min":0.215,"5min":0.191,"15min":0.158},"max":{"1min":0.55900000000000005,"5min":7.3840000000000003,"15min":111.014},"last":0.307},{"interface":"front","average":{"1min":0.373,"5min":1.1020000000000001,"15min":1.839},"min":{"1min":0.28399999999999997,"5min":0.23799999999999999,"15min":0.19600000000000001},"max":{"1min":0.51200000000000001,"5min":9.7360000000000007,"15min":111.072},"last":0.33100000000000002}]},{"osd":2,"last update":"Tue Sep 10 19:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.71899999999999997,"15min":0.751},"min":{"1min":0.26600000000000001,"5min":0.214,"15min":0.186},"max":{"1min":0.71199999999999997,"5min":4.835,"15min":4.835},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.82199999999999995,"15min":0.79800000000000004},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.215},"max":{"1min":0.73199999999999998,"5min":9.484,"15min":9.484},"last":0.34799999999999998}]},{"osd":3,"last update":"Tue Sep 10 19:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":1.9299999999999999,"15min":2.2490000000000001},"min":{"1min":0.249,"5min":0.218,"15min":0.185},"max":{"1min":0.59399999999999997,"5min":48.582999999999998,"15min":92.069000000000003},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.33300000000000002,"5min":1.5720000000000001,"15min":1.8919999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.19900000000000001},"max":{"1min":0.47099999999999997,"5min":51.847000000000001,"15min":88.658000000000001},"last":0.25600000000000001}]},{"osd":4,"last update":"Tue Sep 10 19:08:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.98099999999999998,"15min":1.0820000000000001},"min":{"1min":0.35099999999999998,"5min":0.28100000000000003,"15min":0.27800000000000002},"max":{"1min":0.84899999999999998,"5min":7.2699999999999996,"15min":19.126999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":1.071,"15min":1.1399999999999999},"min":{"1min":0.38700000000000001,"5min":0.26300000000000001,"15min":0.252},"max":{"1min":1.2430000000000001,"5min":5.9340000000000002,"15min":20.152000000000001},"last":0.39300000000000002}]},{"osd":5,"last update":"Tue Sep 10 19:08:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":1.1000000000000001,"15min":1.2250000000000001},"min":{"1min":0.44500000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72599999999999998,"5min":9.9369999999999994,"15min":10.334},"last":0.439},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.88200000000000001,"15min":1.0669999999999999},"min":{"1min":0.371,"5min":0.29199999999999998,"15min":0.28799999999999998},"max":{"1min":0.79500000000000004,"5min":7.351,"15min":7.9969999999999999},"last":0.48799999999999999}]},{"osd":6,"last update":"Tue Sep 10 19:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":1.097,"15min":1.417},"min":{"1min":0.44400000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80200000000000005,"5min":9.9760000000000009,"15min":33.290999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.624,"5min":1.008,"15min":1.3759999999999999},"min":{"1min":0.40300000000000002,"5min":0.32700000000000001,"15min":0.26600000000000001},"max":{"1min":1.2529999999999999,"5min":7.7400000000000002,"15min":33.310000000000002},"last":0.40600000000000003}]},{"osd":7,"last update":"Tue Sep 10 19:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":1.0820000000000001,"15min":1.3100000000000001},"min":{"1min":0.51600000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80100000000000005,"5min":5.9889999999999999,"15min":5.9889999999999999},"last":0.496},{"interface":"front","average":{"1min":0.63400000000000001,"5min":1.2110000000000001,"15min":1.659},"min":{"1min":0.441,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82599999999999996,"5min":10.010999999999999,"15min":10.010999999999999},"last":0.57899999999999996}]}]},{"osd":0,"up_from":241,"seq":1035087118499,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":930776,"kb_used_data":1040,"kb_used_omap":4,"kb_used_meta":929723,"kb_avail":92814376,"statfs":{"total":95995035648,"available":95041921024,"internally_reserved":0,"allocated":1064960,"data_stored":695858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4602,"internal_metadata":952036870},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 19:08:05 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":1.585,"15min":1.8109999999999999},"min":{"1min":0.217,"5min":0.18099999999999999,"15min":0.17100000000000001},"max":{"1min":0.434,"5min":47.237000000000002,"15min":47.237000000000002},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.376,"5min":1.4810000000000001,"15min":1.758},"min":{"1min":0.26900000000000002,"5min":0.20100000000000001,"15min":0.193},"max":{"1min":0.57499999999999996,"5min":47.198999999999998,"15min":47.198999999999998},"last":0.38500000000000001}]},{"osd":2,"last update":"Tue Sep 10 19:08:05 2024","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.92600000000000005,"15min":0.92700000000000005},"min":{"1min":0.247,"5min":0.223,"15min":0.20699999999999999},"max":{"1min":0.52400000000000002,"5min":17.715,"15min":17.715},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.91500000000000004,"15min":0.88800000000000001},"min":{"1min":0.23300000000000001,"5min":0.22900000000000001,"15min":0.22800000000000001},"max":{"1min":0.48399999999999999,"5min":17.809000000000001,"15min":17.809000000000001},"last":0.35899999999999999}]},{"osd":3,"last update":"Tue Sep 10 19:08:05 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":1.208,"15min":1.2310000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.216},"max":{"1min":0.56299999999999994,"5min":20.454999999999998,"15min":21.108000000000001},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.33800000000000002,"5min":1.3,"15min":1.552},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.497,"5min":18.728999999999999,"15min":18.728999999999999},"last":0.34100000000000003}]},{"osd":4,"last update":"Tue Sep 10 19:08:05 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":1.091,"15min":1.0229999999999999},"min":{"1min":0.38600000000000001,"5min":0.30499999999999999,"15min":0.26000000000000001},"max":{"1min":0.78400000000000003,"5min":17.992000000000001,"15min":17.992000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.56399999999999995,"5min":1.0620000000000001,"15min":1.087},"min":{"1min":0.41899999999999998,"5min":0.28100000000000003,"15min":0.27600000000000002},"max":{"1min":0.747,"5min":18.172999999999998,"15min":18.172999999999998},"last":0.64600000000000002}]},{"osd":5,"last update":"Tue Sep 10 19:08:05 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":1.0629999999999999,"15min":1.5089999999999999},"min":{"1min":0.40699999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.79200000000000004,"5min":18.151,"15min":97.296999999999997},"last":0.753},{"interface":"front","average":{"1min":0.58099999999999996,"5min":1.119,"15min":1.4279999999999999},"min":{"1min":0.377,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.76100000000000001,"5min":18.016999999999999,"15min":99.221999999999994},"last":0.54700000000000004}]},{"osd":6,"last update":"Tue Sep 10 19:08:05 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":2.7879999999999998,"15min":1.7509999999999999},"min":{"1min":0.38900000000000001,"5min":0.33600000000000002,"15min":0.312},"max":{"1min":0.86199999999999999,"5min":216.21100000000001,"15min":216.21100000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57199999999999995,"5min":2.831,"15min":1.7250000000000001},"min":{"1min":0.39600000000000002,"5min":0.379,"15min":0.313},"max":{"1min":0.75600000000000001,"5min":216.054,"15min":216.054},"last":0.71199999999999997}]},{"osd":7,"last update":"Tue Sep 10 19:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":1.0640000000000001,"15min":1.048},"min":{"1min":0.443,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.127,"5min":20.417999999999999,"15min":20.417999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.623,"5min":1.212,"15min":1.4399999999999999},"min":{"1min":0.42999999999999999,"5min":0.379,"15min":0.379},"max":{"1min":1.0049999999999999,"5min":18.059000000000001,"15min":18.059000000000001},"last":0.72199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T19:09:08.219 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-10T19:09:08.219 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 -- ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-10T19:09:08.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:07 smithi103 ceph-mon[33163]: 5.18 scrub starts 2024-09-10T19:09:08.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:07 smithi103 ceph-mon[33163]: 5.18 scrub ok 2024-09-10T19:09:08.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:07 smithi103 ceph-mon[33163]: from='client.24409 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:09:08.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:07 smithi103 ceph-mon[33163]: from='client.16281 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:09:08.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:07 smithi103 ceph-mon[33163]: pgmap v1125: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:09:08.475 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:09:09.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:08 smithi018 ceph-mon[29336]: 2.0 scrub starts 2024-09-10T19:09:09.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:08 smithi018 ceph-mon[29336]: 2.0 scrub ok 2024-09-10T19:09:09.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:08 smithi018 ceph-mon[40196]: 2.0 scrub starts 2024-09-10T19:09:09.194 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:08 smithi018 ceph-mon[40196]: 2.0 scrub ok 2024-09-10T19:09:09.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:08 smithi103 ceph-mon[33163]: 2.0 scrub starts 2024-09-10T19:09:09.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:08 smithi103 ceph-mon[33163]: 2.0 scrub ok 2024-09-10T19:09:10.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:09 smithi018 ceph-mon[29336]: 5.d scrub starts 2024-09-10T19:09:10.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:09 smithi018 ceph-mon[29336]: 5.d scrub ok 2024-09-10T19:09:10.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:09 smithi018 ceph-mon[29336]: 4.1d scrub starts 2024-09-10T19:09:10.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:09 smithi018 ceph-mon[29336]: 4.1d scrub ok 2024-09-10T19:09:10.194 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:09 smithi018 ceph-mon[29336]: pgmap v1126: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:09:10.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:09 smithi018 ceph-mon[40196]: 5.d scrub starts 2024-09-10T19:09:10.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:09 smithi018 ceph-mon[40196]: 5.d scrub ok 2024-09-10T19:09:10.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:09 smithi018 ceph-mon[40196]: 4.1d scrub starts 2024-09-10T19:09:10.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:09 smithi018 ceph-mon[40196]: 4.1d scrub ok 2024-09-10T19:09:10.195 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:09 smithi018 ceph-mon[40196]: pgmap v1126: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:09:10.238 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:09 smithi103 ceph-mon[33163]: 5.d scrub starts 2024-09-10T19:09:10.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:09 smithi103 ceph-mon[33163]: 5.d scrub ok 2024-09-10T19:09:10.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:09 smithi103 ceph-mon[33163]: 4.1d scrub starts 2024-09-10T19:09:10.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:09 smithi103 ceph-mon[33163]: 4.1d scrub ok 2024-09-10T19:09:10.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:09 smithi103 ceph-mon[33163]: pgmap v1126: 152 pgs: 152 active+clean; 582 KiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:09:10.522 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:09:10 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x[34997]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:09:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:09:10.582 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:09:10 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:19:09:10] ENGINE Bus STOPPING 2024-09-10T19:09:10.582 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:09:10 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:19:09:10] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T19:09:10.582 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:09:10 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:19:09:10] ENGINE Bus STOPPED 2024-09-10T19:09:10.583 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:09:10 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:19:09:10] ENGINE Bus STARTING 2024-09-10T19:09:10.583 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:09:10 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:19:09:10] ENGINE Serving on http://:::9283 2024-09-10T19:09:10.583 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:09:10 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:19:09:10] ENGINE Bus STARTED 2024-09-10T19:09:11.136 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:10 smithi018 ceph-mon[29336]: 1.3 scrub starts 2024-09-10T19:09:11.136 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:10 smithi018 ceph-mon[29336]: 1.3 scrub ok 2024-09-10T19:09:11.137 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:10 smithi018 ceph-mon[29336]: 4.5 deep-scrub starts 2024-09-10T19:09:11.137 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:10 smithi018 ceph-mon[29336]: 4.5 deep-scrub ok 2024-09-10T19:09:11.137 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:10 smithi018 ceph-mon[29336]: 6.1f scrub starts 2024-09-10T19:09:11.137 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:10 smithi018 ceph-mon[29336]: 6.1f scrub ok 2024-09-10T19:09:11.137 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:10 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:09:11.137 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:10 smithi018 ceph-mon[29336]: purged_snaps scrub starts 2024-09-10T19:09:11.137 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:10 smithi018 ceph-mon[29336]: purged_snaps scrub ok 2024-09-10T19:09:11.137 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:10 smithi018 ceph-mon[29336]: from='client.? 172.21.15.18:0/2887294267' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2024-09-10T19:09:11.137 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:10 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2024-09-10T19:09:11.137 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:10 smithi018 ceph-mon[29336]: from='client.? ' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2024-09-10T19:09:11.137 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:10 smithi018 ceph-mon[29336]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:09:11.138 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:10 smithi018 ceph-mon[40196]: 1.3 scrub starts 2024-09-10T19:09:11.138 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:10 smithi018 ceph-mon[40196]: 1.3 scrub ok 2024-09-10T19:09:11.138 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:10 smithi018 ceph-mon[40196]: 4.5 deep-scrub starts 2024-09-10T19:09:11.138 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:10 smithi018 ceph-mon[40196]: 4.5 deep-scrub ok 2024-09-10T19:09:11.138 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:10 smithi018 ceph-mon[40196]: 6.1f scrub starts 2024-09-10T19:09:11.138 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:10 smithi018 ceph-mon[40196]: 6.1f scrub ok 2024-09-10T19:09:11.138 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:10 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:09:11.138 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:10 smithi018 ceph-mon[40196]: purged_snaps scrub starts 2024-09-10T19:09:11.138 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:10 smithi018 ceph-mon[40196]: purged_snaps scrub ok 2024-09-10T19:09:11.138 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:10 smithi018 ceph-mon[40196]: from='client.? 172.21.15.18:0/2887294267' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2024-09-10T19:09:11.139 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:10 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2024-09-10T19:09:11.139 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:10 smithi018 ceph-mon[40196]: from='client.? ' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2024-09-10T19:09:11.139 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:10 smithi018 ceph-mon[40196]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:09:11.139 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:09:10 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: ::ffff:172.21.15.103 - - [10/Sep/2024:19:09:10] "GET /metrics HTTP/1.1" 200 222089 "" "Prometheus/2.43.0" 2024-09-10T19:09:11.163 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-10T19:09:11.172 INFO:tasks.cephadm:Teardown begin 2024-09-10T19:09:11.173 DEBUG:teuthology.orchestra.run.smithi018:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T19:09:11.214 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T19:09:11.238 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:10 smithi103 ceph-mon[33163]: 1.3 scrub starts 2024-09-10T19:09:11.238 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:10 smithi103 ceph-mon[33163]: 1.3 scrub ok 2024-09-10T19:09:11.238 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:10 smithi103 ceph-mon[33163]: 4.5 deep-scrub starts 2024-09-10T19:09:11.238 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:10 smithi103 ceph-mon[33163]: 4.5 deep-scrub ok 2024-09-10T19:09:11.238 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:10 smithi103 ceph-mon[33163]: 6.1f scrub starts 2024-09-10T19:09:11.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:10 smithi103 ceph-mon[33163]: 6.1f scrub ok 2024-09-10T19:09:11.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:10 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:09:11.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:10 smithi103 ceph-mon[33163]: purged_snaps scrub starts 2024-09-10T19:09:11.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:10 smithi103 ceph-mon[33163]: purged_snaps scrub ok 2024-09-10T19:09:11.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:10 smithi103 ceph-mon[33163]: from='client.? 172.21.15.18:0/2887294267' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2024-09-10T19:09:11.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:10 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2024-09-10T19:09:11.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:10 smithi103 ceph-mon[33163]: from='client.? ' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2024-09-10T19:09:11.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:10 smithi103 ceph-mon[33163]: from='mgr.14538 172.21.15.18:0/3661627509' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:09:11.258 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-10T19:09:11.259 DEBUG:teuthology.orchestra.run.smithi018:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T19:09:11.275 DEBUG:teuthology.orchestra.run.smithi103:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T19:09:11.313 INFO:tasks.cephadm:Stopping all daemons... 2024-09-10T19:09:11.313 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-10T19:09:11.313 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@mon.a 2024-09-10T19:09:11.444 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:11 smithi018 systemd[1]: Stopping Ceph mon.a for ba327f6c-6fa3-11ef-bcea-c7b262605968... 2024-09-10T19:09:11.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:11 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mon-a[29332]: 2024-09-10T19:09:11.646+0000 7f4eb6735640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:09:11.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:11 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mon-a[29332]: 2024-09-10T19:09:11.646+0000 7f4eb6735640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-10T19:09:11.944 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:11 smithi018 podman[210551]: 2024-09-10 19:09:11.739214681 +0000 UTC m=+0.218007372 container died c1f97cd118ccbc2e0de9e6e30b48098f1b3fe3cefb330b4408ff67b690c64cb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-10T19:09:12.407 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:12 smithi018 podman[210551]: 2024-09-10 19:09:12.172207175 +0000 UTC m=+0.650999864 container cleanup c1f97cd118ccbc2e0de9e6e30b48098f1b3fe3cefb330b4408ff67b690c64cb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-10T19:09:12.408 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:12 smithi018 bash[210551]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mon-a 2024-09-10T19:09:12.654 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@mon.a.service' 2024-09-10T19:09:12.843 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:12 smithi018 podman[210563]: 2024-09-10 19:09:12.406592878 +0000 UTC m=+0.658497212 container remove c1f97cd118ccbc2e0de9e6e30b48098f1b3fe3cefb330b4408ff67b690c64cb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T19:09:12.843 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:12 smithi018 systemd[1]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-10T19:09:12.843 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:12 smithi018 systemd[1]: Stopped Ceph mon.a for ba327f6c-6fa3-11ef-bcea-c7b262605968. 2024-09-10T19:09:12.843 INFO:journalctl@ceph.mon.a.smithi018.stdout:Sep 10 19:09:12 smithi018 systemd[1]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@mon.a.service: Consumed 37.778s CPU time. 2024-09-10T19:09:12.893 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:09:12.894 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-10T19:09:12.894 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-09-10T19:09:12.894 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@mon.c 2024-09-10T19:09:13.186 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:12 smithi018 systemd[1]: Stopping Ceph mon.c for ba327f6c-6fa3-11ef-bcea-c7b262605968... 2024-09-10T19:09:13.444 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:09:13 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:19:09:13] ENGINE Bus STOPPING 2024-09-10T19:09:13.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:13 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mon-c[40192]: 2024-09-10T19:09:13.184+0000 7f1b4649c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:09:13.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:13 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mon-c[40192]: 2024-09-10T19:09:13.184+0000 7f1b4649c640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-09-10T19:09:13.444 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:13 smithi018 podman[210672]: 2024-09-10 19:09:13.359350773 +0000 UTC m=+0.272526342 container died 2d199c7c18be616fc886c24a687b07c35f5e304347e2d50517fbceddef6dac02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mon-c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-10T19:09:13.929 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:13 smithi018 podman[210672]: 2024-09-10 19:09:13.701763332 +0000 UTC m=+0.614938898 container cleanup 2d199c7c18be616fc886c24a687b07c35f5e304347e2d50517fbceddef6dac02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mon-c, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-09-10T19:09:13.929 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:13 smithi018 bash[210672]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mon-c 2024-09-10T19:09:13.929 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:09:13 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:19:09:13] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T19:09:13.929 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:09:13 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:19:09:13] ENGINE Bus STOPPED 2024-09-10T19:09:13.930 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:09:13 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:19:09:13] ENGINE Bus STARTING 2024-09-10T19:09:13.930 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:09:13 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:19:09:13] ENGINE Serving on http://:::9283 2024-09-10T19:09:13.930 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:09:13 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: [10/Sep/2024:19:09:13] ENGINE Bus STARTED 2024-09-10T19:09:14.188 INFO:journalctl@ceph.mon.c.smithi018.stdout:Sep 10 19:09:13 smithi018 podman[210684]: 2024-09-10 19:09:13.928009214 +0000 UTC m=+0.566525420 container remove 2d199c7c18be616fc886c24a687b07c35f5e304347e2d50517fbceddef6dac02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mon-c, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T19:09:14.192 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@mon.c.service' 2024-09-10T19:09:14.229 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:09:14.229 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-09-10T19:09:14.230 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-09-10T19:09:14.230 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@mon.b 2024-09-10T19:09:14.523 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:14 smithi103 systemd[1]: Stopping Ceph mon.b for ba327f6c-6fa3-11ef-bcea-c7b262605968... 2024-09-10T19:09:15.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:14 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mon-b[33159]: 2024-09-10T19:09:14.974+0000 7fbe92b5d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:09:15.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:14 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mon-b[33159]: 2024-09-10T19:09:14.974+0000 7fbe92b5d640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-09-10T19:09:15.273 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:15 smithi103 podman[152534]: 2024-09-10 19:09:15.029992947 +0000 UTC m=+0.166487807 container died f40a52d205a9fc4a55dfa78ebc60f5e3173b3d4b5ed9105e06d82f397dcde16a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mon-b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T19:09:15.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:15 smithi103 podman[152534]: 2024-09-10 19:09:15.474877413 +0000 UTC m=+0.611372305 container cleanup f40a52d205a9fc4a55dfa78ebc60f5e3173b3d4b5ed9105e06d82f397dcde16a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-10T19:09:15.772 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 19:09:15 smithi103 bash[152534]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mon-b 2024-09-10T19:09:15.937 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@mon.b.service' 2024-09-10T19:09:15.976 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:09:15.977 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-09-10T19:09:15.977 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-10T19:09:15.977 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@mgr.y 2024-09-10T19:09:16.290 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:09:16 smithi018 systemd[1]: Stopping Ceph mgr.y for ba327f6c-6fa3-11ef-bcea-c7b262605968... 2024-09-10T19:09:16.617 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:09:16 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T19:09:16.288+0000 7f670c280640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:09:16.618 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:09:16 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y[29564]: 2024-09-10T19:09:16.288+0000 7f670c280640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-10T19:09:16.618 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:09:16 smithi018 podman[210807]: 2024-09-10 19:09:16.320415824 +0000 UTC m=+0.146282013 container died 91e59d2b2d2d2712e6adec4fbc7c43897964e4e5cf20eb2b0706ecee8af784aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-10T19:09:16.944 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:09:16 smithi018 podman[210807]: 2024-09-10 19:09:16.824034119 +0000 UTC m=+0.649900298 container cleanup 91e59d2b2d2d2712e6adec4fbc7c43897964e4e5cf20eb2b0706ecee8af784aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-10T19:09:16.944 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 10 19:09:16 smithi018 bash[210807]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-y 2024-09-10T19:09:17.325 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-10T19:09:17.361 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:09:17.361 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-10T19:09:17.362 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-10T19:09:17.362 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@mgr.x 2024-09-10T19:09:17.607 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:09:17 smithi103 systemd[1]: Stopping Ceph mgr.x for ba327f6c-6fa3-11ef-bcea-c7b262605968... 2024-09-10T19:09:17.993 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:09:17 smithi103 podman[152654]: 2024-09-10 19:09:17.718530459 +0000 UTC m=+0.154685283 container died 108a62debc0ba072c28a06c4bb3ab9201aac96b5b3a341dec8113bccf93055ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0) 2024-09-10T19:09:18.272 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:09:18 smithi103 podman[152654]: 2024-09-10 19:09:18.080618245 +0000 UTC m=+0.516773066 container cleanup 108a62debc0ba072c28a06c4bb3ab9201aac96b5b3a341dec8113bccf93055ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-10T19:09:18.272 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 10 19:09:18 smithi103 bash[152654]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-mgr-x 2024-09-10T19:09:18.539 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-10T19:09:18.578 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:09:18.578 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-10T19:09:18.578 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-10T19:09:18.578 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@osd.0 2024-09-10T19:09:18.872 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 19:09:18 smithi018 systemd[1]: Stopping Ceph osd.0 for ba327f6c-6fa3-11ef-bcea-c7b262605968... 2024-09-10T19:09:19.194 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 19:09:18 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0[130951]: 2024-09-10T19:09:18.870+0000 7fbd2eb33640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:09:19.194 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 19:09:18 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0[130951]: 2024-09-10T19:09:18.870+0000 7fbd2eb33640 -1 osd.0 497 *** Got signal Terminated *** 2024-09-10T19:09:19.194 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 19:09:18 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0[130951]: 2024-09-10T19:09:18.870+0000 7fbd2eb33640 -1 osd.0 497 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T19:09:24.622 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 19:09:24 smithi018 podman[210928]: 2024-09-10 19:09:24.31475595 +0000 UTC m=+5.543474509 container died e4449adc28b27c6b3584f3c0692e354e893cf1b14bf0ccb8ab08036ad1180169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-10T19:09:24.944 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 19:09:24 smithi018 podman[210928]: 2024-09-10 19:09:24.820364162 +0000 UTC m=+6.049082763 container cleanup e4449adc28b27c6b3584f3c0692e354e893cf1b14bf0ccb8ab08036ad1180169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-10T19:09:24.944 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 19:09:24 smithi018 bash[210928]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0 2024-09-10T19:09:25.443 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 19:09:25 smithi018 podman[210945]: 2024-09-10 19:09:25.072267773 +0000 UTC m=+0.759686943 container remove e4449adc28b27c6b3584f3c0692e354e893cf1b14bf0ccb8ab08036ad1180169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=) 2024-09-10T19:09:25.694 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 19:09:25 smithi018 podman[211034]: 2024-09-10 19:09:25.34534655 +0000 UTC m=+0.016601672 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:09:25.694 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 19:09:25 smithi018 podman[211034]: 2024-09-10 19:09:25.473365579 +0000 UTC m=+0.144620729 container create f5ea504f86be5c3331237ed08c19d8183aa7804c95eae286dfdf9fc147ff4940 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0-deactivate, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T19:09:26.005 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 19:09:25 smithi018 podman[211034]: 2024-09-10 19:09:25.742519187 +0000 UTC m=+0.413774297 container init f5ea504f86be5c3331237ed08c19d8183aa7804c95eae286dfdf9fc147ff4940 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD) 2024-09-10T19:09:26.005 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 19:09:25 smithi018 podman[211034]: 2024-09-10 19:09:25.74627888 +0000 UTC m=+0.417533999 container start f5ea504f86be5c3331237ed08c19d8183aa7804c95eae286dfdf9fc147ff4940 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-10T19:09:26.005 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 19:09:25 smithi018 podman[211034]: 2024-09-10 19:09:25.817402831 +0000 UTC m=+0.488657939 container attach f5ea504f86be5c3331237ed08c19d8183aa7804c95eae286dfdf9fc147ff4940 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0-deactivate, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-10T19:09:26.305 INFO:journalctl@ceph.osd.0.smithi018.stdout:Sep 10 19:09:26 smithi018 podman[211034]: 2024-09-10 19:09:26.004443671 +0000 UTC m=+0.675698791 container died f5ea504f86be5c3331237ed08c19d8183aa7804c95eae286dfdf9fc147ff4940 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-0-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-10T19:09:26.725 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@osd.0.service' 2024-09-10T19:09:26.766 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:09:26.767 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-10T19:09:26.767 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-10T19:09:26.767 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@osd.1 2024-09-10T19:09:27.127 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 10 19:09:26 smithi018 systemd[1]: Stopping Ceph osd.1 for ba327f6c-6fa3-11ef-bcea-c7b262605968... 2024-09-10T19:09:27.444 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 10 19:09:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-1[49824]: 2024-09-10T19:09:27.124+0000 7f97a7d68640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:09:27.444 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 10 19:09:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-1[49824]: 2024-09-10T19:09:27.124+0000 7f97a7d68640 -1 osd.1 497 *** Got signal Terminated *** 2024-09-10T19:09:27.444 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 10 19:09:27 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-1[49824]: 2024-09-10T19:09:27.124+0000 7f97a7d68640 -1 osd.1 497 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T19:09:32.944 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 10 19:09:32 smithi018 podman[211137]: 2024-09-10 19:09:32.642965543 +0000 UTC m=+5.637854641 container died 317613b07e77bb037fd5ef795eaade467fedbce490e78c945abc3ff7b72a4d65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-10T19:09:33.354 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 10 19:09:33 smithi018 podman[211137]: 2024-09-10 19:09:33.149966275 +0000 UTC m=+6.144855359 container cleanup 317613b07e77bb037fd5ef795eaade467fedbce490e78c945abc3ff7b72a4d65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-10T19:09:33.355 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 10 19:09:33 smithi018 bash[211137]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-1 2024-09-10T19:09:33.694 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 10 19:09:33 smithi018 podman[211157]: 2024-09-10 19:09:33.409712459 +0000 UTC m=+0.754754307 container remove 317613b07e77bb037fd5ef795eaade467fedbce490e78c945abc3ff7b72a4d65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-10T19:09:34.036 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 10 19:09:33 smithi018 podman[211255]: 2024-09-10 19:09:33.665590776 +0000 UTC m=+0.016542718 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:09:34.036 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 10 19:09:33 smithi018 podman[211255]: 2024-09-10 19:09:33.802499248 +0000 UTC m=+0.153451193 container create f30f888161a06957fd50dcbff3632a84109298afe4e331d897e25375f973eb0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-10T19:09:34.036 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 10 19:09:34 smithi018 podman[211255]: 2024-09-10 19:09:34.034624752 +0000 UTC m=+0.385576695 container init f30f888161a06957fd50dcbff3632a84109298afe4e331d897e25375f973eb0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-10T19:09:34.445 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 10 19:09:34 smithi018 podman[211255]: 2024-09-10 19:09:34.038808925 +0000 UTC m=+0.389760869 container start f30f888161a06957fd50dcbff3632a84109298afe4e331d897e25375f973eb0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T19:09:34.445 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 10 19:09:34 smithi018 podman[211255]: 2024-09-10 19:09:34.113623449 +0000 UTC m=+0.464575406 container attach f30f888161a06957fd50dcbff3632a84109298afe4e331d897e25375f973eb0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD) 2024-09-10T19:09:34.445 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 10 19:09:34 smithi018 podman[211255]: 2024-09-10 19:09:34.222835488 +0000 UTC m=+0.573787431 container died f30f888161a06957fd50dcbff3632a84109298afe4e331d897e25375f973eb0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-10T19:09:34.970 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@osd.1.service' 2024-09-10T19:09:35.014 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:09:35.014 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-10T19:09:35.014 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-10T19:09:35.014 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@osd.2 2024-09-10T19:09:35.194 INFO:journalctl@ceph.osd.2.smithi018.stdout:Sep 10 19:09:35 smithi018 systemd[1]: Stopping Ceph osd.2 for ba327f6c-6fa3-11ef-bcea-c7b262605968... 2024-09-10T19:09:35.694 INFO:journalctl@ceph.osd.2.smithi018.stdout:Sep 10 19:09:35 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-2[55543]: 2024-09-10T19:09:35.353+0000 7f2172f1c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:09:35.694 INFO:journalctl@ceph.osd.2.smithi018.stdout:Sep 10 19:09:35 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-2[55543]: 2024-09-10T19:09:35.353+0000 7f2172f1c640 -1 osd.2 497 *** Got signal Terminated *** 2024-09-10T19:09:35.694 INFO:journalctl@ceph.osd.2.smithi018.stdout:Sep 10 19:09:35 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-2[55543]: 2024-09-10T19:09:35.353+0000 7f2172f1c640 -1 osd.2 497 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T19:09:41.168 INFO:journalctl@ceph.osd.2.smithi018.stdout:Sep 10 19:09:40 smithi018 podman[211359]: 2024-09-10 19:09:40.867667054 +0000 UTC m=+5.629020764 container died ec0aa641356d860f5ee94ae135e5131a2398fcd394832fbce2e311aa939e1aa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-10T19:09:41.444 INFO:journalctl@ceph.osd.2.smithi018.stdout:Sep 10 19:09:41 smithi018 podman[211359]: 2024-09-10 19:09:41.379123647 +0000 UTC m=+6.140477399 container cleanup ec0aa641356d860f5ee94ae135e5131a2398fcd394832fbce2e311aa939e1aa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T19:09:41.445 INFO:journalctl@ceph.osd.2.smithi018.stdout:Sep 10 19:09:41 smithi018 bash[211359]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-2 2024-09-10T19:09:41.944 INFO:journalctl@ceph.osd.2.smithi018.stdout:Sep 10 19:09:41 smithi018 podman[211375]: 2024-09-10 19:09:41.655423756 +0000 UTC m=+0.778772790 container remove ec0aa641356d860f5ee94ae135e5131a2398fcd394832fbce2e311aa939e1aa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-10T19:09:42.326 INFO:journalctl@ceph.osd.2.smithi018.stdout:Sep 10 19:09:41 smithi018 podman[211465]: 2024-09-10 19:09:41.872657 +0000 UTC m=+0.016324045 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:09:42.326 INFO:journalctl@ceph.osd.2.smithi018.stdout:Sep 10 19:09:41 smithi018 podman[211465]: 2024-09-10 19:09:41.998044308 +0000 UTC m=+0.141711387 container create b756c71762e6333a45d04796896ff294cb580d81cfc59751baee2690fce3e997 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-10T19:09:42.326 INFO:journalctl@ceph.osd.2.smithi018.stdout:Sep 10 19:09:42 smithi018 podman[211465]: 2024-09-10 19:09:42.20876707 +0000 UTC m=+0.352434111 container init b756c71762e6333a45d04796896ff294cb580d81cfc59751baee2690fce3e997 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-2-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-10T19:09:42.326 INFO:journalctl@ceph.osd.2.smithi018.stdout:Sep 10 19:09:42 smithi018 podman[211465]: 2024-09-10 19:09:42.212245816 +0000 UTC m=+0.355912852 container start b756c71762e6333a45d04796896ff294cb580d81cfc59751baee2690fce3e997 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-10T19:09:42.327 INFO:journalctl@ceph.osd.2.smithi018.stdout:Sep 10 19:09:42 smithi018 podman[211465]: 2024-09-10 19:09:42.325285554 +0000 UTC m=+0.468952603 container attach b756c71762e6333a45d04796896ff294cb580d81cfc59751baee2690fce3e997 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-10T19:09:42.694 INFO:journalctl@ceph.osd.2.smithi018.stdout:Sep 10 19:09:42 smithi018 podman[211465]: 2024-09-10 19:09:42.396932008 +0000 UTC m=+0.540599045 container died b756c71762e6333a45d04796896ff294cb580d81cfc59751baee2690fce3e997 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, GIT_BRANCH=HEAD) 2024-09-10T19:09:43.172 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@osd.2.service' 2024-09-10T19:09:43.213 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:09:43.213 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-10T19:09:43.213 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-10T19:09:43.214 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@osd.3 2024-09-10T19:09:43.444 INFO:journalctl@ceph.osd.3.smithi018.stdout:Sep 10 19:09:43 smithi018 systemd[1]: Stopping Ceph osd.3 for ba327f6c-6fa3-11ef-bcea-c7b262605968... 2024-09-10T19:09:43.944 INFO:journalctl@ceph.osd.3.smithi018.stdout:Sep 10 19:09:43 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-3[61244]: 2024-09-10T19:09:43.556+0000 7fc4c40a6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:09:43.944 INFO:journalctl@ceph.osd.3.smithi018.stdout:Sep 10 19:09:43 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-3[61244]: 2024-09-10T19:09:43.556+0000 7fc4c40a6640 -1 osd.3 497 *** Got signal Terminated *** 2024-09-10T19:09:43.944 INFO:journalctl@ceph.osd.3.smithi018.stdout:Sep 10 19:09:43 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-3[61244]: 2024-09-10T19:09:43.556+0000 7fc4c40a6640 -1 osd.3 497 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T19:09:46.023 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 19:09:45 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-4[38039]: 2024-09-10T19:09:45.529+0000 7fbb124e7640 -1 osd.4 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.194289+0000 front 2024-09-10T19:09:21.194335+0000 (oldest deadline 2024-09-10T19:09:45.294059+0000) 2024-09-10T19:09:46.023 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:09:45 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:09:45.608+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:20.280088+0000 front 2024-09-10T19:09:20.280106+0000 (oldest deadline 2024-09-10T19:09:44.979964+0000) 2024-09-10T19:09:46.444 INFO:journalctl@ceph.osd.3.smithi018.stdout:Sep 10 19:09:46 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-3[61244]: 2024-09-10T19:09:46.141+0000 7fc4c16c0640 -1 osd.3 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:19.954951+0000 front 2024-09-10T19:09:19.954913+0000 (oldest deadline 2024-09-10T19:09:45.255009+0000) 2024-09-10T19:09:47.022 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:09:46 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:09:46.612+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:20.280088+0000 front 2024-09-10T19:09:20.280106+0000 (oldest deadline 2024-09-10T19:09:44.979964+0000) 2024-09-10T19:09:47.023 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 19:09:46 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-4[38039]: 2024-09-10T19:09:46.533+0000 7fbb124e7640 -1 osd.4 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.194289+0000 front 2024-09-10T19:09:21.194335+0000 (oldest deadline 2024-09-10T19:09:45.294059+0000) 2024-09-10T19:09:47.444 INFO:journalctl@ceph.osd.3.smithi018.stdout:Sep 10 19:09:47 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-3[61244]: 2024-09-10T19:09:47.184+0000 7fc4c16c0640 -1 osd.3 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:19.954951+0000 front 2024-09-10T19:09:19.954913+0000 (oldest deadline 2024-09-10T19:09:45.255009+0000) 2024-09-10T19:09:47.609 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 19:09:47 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-4[38039]: 2024-09-10T19:09:47.498+0000 7fbb124e7640 -1 osd.4 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.194289+0000 front 2024-09-10T19:09:21.194335+0000 (oldest deadline 2024-09-10T19:09:45.294059+0000) 2024-09-10T19:09:48.005 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:09:47 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:09:47.606+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:20.280088+0000 front 2024-09-10T19:09:20.280106+0000 (oldest deadline 2024-09-10T19:09:44.979964+0000) 2024-09-10T19:09:48.268 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:09:48 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5[43919]: 2024-09-10T19:09:48.002+0000 7f5ff7f84640 -1 osd.5 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:23.013808+0000 front 2024-09-10T19:09:23.013766+0000 (oldest deadline 2024-09-10T19:09:47.113528+0000) 2024-09-10T19:09:48.444 INFO:journalctl@ceph.osd.3.smithi018.stdout:Sep 10 19:09:48 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-3[61244]: 2024-09-10T19:09:48.190+0000 7fc4c16c0640 -1 osd.3 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:19.954951+0000 front 2024-09-10T19:09:19.954913+0000 (oldest deadline 2024-09-10T19:09:45.255009+0000) 2024-09-10T19:09:48.522 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 19:09:48 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-4[38039]: 2024-09-10T19:09:48.484+0000 7fbb124e7640 -1 osd.4 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.194289+0000 front 2024-09-10T19:09:21.194335+0000 (oldest deadline 2024-09-10T19:09:45.294059+0000) 2024-09-10T19:09:48.523 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:09:48 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:09:48.388+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.285935+0000 front 2024-09-10T19:09:21.285773+0000 (oldest deadline 2024-09-10T19:09:48.286196+0000) 2024-09-10T19:09:48.974 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:09:48 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:09:48.641+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:20.280088+0000 front 2024-09-10T19:09:20.280106+0000 (oldest deadline 2024-09-10T19:09:44.979964+0000) 2024-09-10T19:09:49.272 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:09:48 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5[43919]: 2024-09-10T19:09:48.972+0000 7f5ff7f84640 -1 osd.5 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:23.013808+0000 front 2024-09-10T19:09:23.013766+0000 (oldest deadline 2024-09-10T19:09:47.113528+0000) 2024-09-10T19:09:49.375 INFO:journalctl@ceph.osd.3.smithi018.stdout:Sep 10 19:09:49 smithi018 podman[211566]: 2024-09-10 19:09:49.102351949 +0000 UTC m=+5.666330122 container died 8c5103fcdae4098dbd6b3229c71fd72d1050cf6022d5b16e747dfbce157e1b8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-10T19:09:49.694 INFO:journalctl@ceph.osd.3.smithi018.stdout:Sep 10 19:09:49 smithi018 podman[211566]: 2024-09-10 19:09:49.540252755 +0000 UTC m=+6.104230929 container cleanup 8c5103fcdae4098dbd6b3229c71fd72d1050cf6022d5b16e747dfbce157e1b8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-10T19:09:49.694 INFO:journalctl@ceph.osd.3.smithi018.stdout:Sep 10 19:09:49 smithi018 bash[211566]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-3 2024-09-10T19:09:49.772 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 19:09:49 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-4[38039]: 2024-09-10T19:09:49.495+0000 7fbb124e7640 -1 osd.4 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.194289+0000 front 2024-09-10T19:09:21.194335+0000 (oldest deadline 2024-09-10T19:09:45.294059+0000) 2024-09-10T19:09:49.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:09:49 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:09:49.641+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:20.280088+0000 front 2024-09-10T19:09:20.280106+0000 (oldest deadline 2024-09-10T19:09:44.979964+0000) 2024-09-10T19:09:49.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:09:49 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:09:49.402+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.285935+0000 front 2024-09-10T19:09:21.285773+0000 (oldest deadline 2024-09-10T19:09:48.286196+0000) 2024-09-10T19:09:50.194 INFO:journalctl@ceph.osd.3.smithi018.stdout:Sep 10 19:09:49 smithi018 podman[211583]: 2024-09-10 19:09:49.758645386 +0000 UTC m=+0.649666218 container remove 8c5103fcdae4098dbd6b3229c71fd72d1050cf6022d5b16e747dfbce157e1b8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T19:09:50.272 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:09:49 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5[43919]: 2024-09-10T19:09:49.975+0000 7f5ff7f84640 -1 osd.5 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:23.013808+0000 front 2024-09-10T19:09:23.013766+0000 (oldest deadline 2024-09-10T19:09:47.113528+0000) 2024-09-10T19:09:50.488 INFO:journalctl@ceph.osd.3.smithi018.stdout:Sep 10 19:09:50 smithi018 podman[211670]: 2024-09-10 19:09:50.106524921 +0000 UTC m=+0.016106927 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:09:50.488 INFO:journalctl@ceph.osd.3.smithi018.stdout:Sep 10 19:09:50 smithi018 podman[211670]: 2024-09-10 19:09:50.209225703 +0000 UTC m=+0.118807711 container create 2debe47e4a35e8ffd0182fadbea12109a081fbd47814f08d4c208622d40efee2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-10T19:09:50.488 INFO:journalctl@ceph.osd.3.smithi018.stdout:Sep 10 19:09:50 smithi018 podman[211670]: 2024-09-10 19:09:50.411978901 +0000 UTC m=+0.321560907 container init 2debe47e4a35e8ffd0182fadbea12109a081fbd47814f08d4c208622d40efee2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-10T19:09:50.488 INFO:journalctl@ceph.osd.3.smithi018.stdout:Sep 10 19:09:50 smithi018 podman[211670]: 2024-09-10 19:09:50.415357401 +0000 UTC m=+0.324939415 container start 2debe47e4a35e8ffd0182fadbea12109a081fbd47814f08d4c208622d40efee2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-10T19:09:50.488 INFO:journalctl@ceph.osd.3.smithi018.stdout:Sep 10 19:09:50 smithi018 podman[211670]: 2024-09-10 19:09:50.486864016 +0000 UTC m=+0.396446032 container attach 2debe47e4a35e8ffd0182fadbea12109a081fbd47814f08d4c208622d40efee2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-10T19:09:50.772 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:09:50 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:09:50.671+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:20.280088+0000 front 2024-09-10T19:09:20.280106+0000 (oldest deadline 2024-09-10T19:09:44.979964+0000) 2024-09-10T19:09:50.773 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 19:09:50 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-4[38039]: 2024-09-10T19:09:50.511+0000 7fbb124e7640 -1 osd.4 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.194289+0000 front 2024-09-10T19:09:21.194335+0000 (oldest deadline 2024-09-10T19:09:45.294059+0000) 2024-09-10T19:09:50.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:09:50 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:09:50.423+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.285935+0000 front 2024-09-10T19:09:21.285773+0000 (oldest deadline 2024-09-10T19:09:48.286196+0000) 2024-09-10T19:09:50.891 INFO:journalctl@ceph.osd.3.smithi018.stdout:Sep 10 19:09:50 smithi018 podman[211670]: 2024-09-10 19:09:50.597290989 +0000 UTC m=+0.506872995 container died 2debe47e4a35e8ffd0182fadbea12109a081fbd47814f08d4c208622d40efee2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T19:09:51.268 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@osd.3.service' 2024-09-10T19:09:51.272 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:09:51 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5[43919]: 2024-09-10T19:09:51.003+0000 7f5ff7f84640 -1 osd.5 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:23.013808+0000 front 2024-09-10T19:09:23.013766+0000 (oldest deadline 2024-09-10T19:09:47.113528+0000) 2024-09-10T19:09:51.312 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:09:51.312 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-10T19:09:51.312 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-10T19:09:51.312 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@osd.4 2024-09-10T19:09:51.618 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:09:51 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:09:51.442+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.285935+0000 front 2024-09-10T19:09:21.285773+0000 (oldest deadline 2024-09-10T19:09:48.286196+0000) 2024-09-10T19:09:51.619 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 19:09:51 smithi103 systemd[1]: Stopping Ceph osd.4 for ba327f6c-6fa3-11ef-bcea-c7b262605968... 2024-09-10T19:09:51.619 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 19:09:51 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-4[38039]: 2024-09-10T19:09:51.517+0000 7fbb124e7640 -1 osd.4 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.194289+0000 front 2024-09-10T19:09:21.194335+0000 (oldest deadline 2024-09-10T19:09:45.294059+0000) 2024-09-10T19:09:52.022 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:09:51 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:09:51.640+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:20.280088+0000 front 2024-09-10T19:09:20.280106+0000 (oldest deadline 2024-09-10T19:09:44.979964+0000) 2024-09-10T19:09:52.022 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 19:09:51 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-4[38039]: 2024-09-10T19:09:51.617+0000 7fbb16ed1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:09:52.022 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 19:09:51 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-4[38039]: 2024-09-10T19:09:51.617+0000 7fbb16ed1640 -1 osd.4 497 *** Got signal Terminated *** 2024-09-10T19:09:52.023 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 19:09:51 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-4[38039]: 2024-09-10T19:09:51.617+0000 7fbb16ed1640 -1 osd.4 497 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T19:09:52.464 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:09:52 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5[43919]: 2024-09-10T19:09:52.049+0000 7f5ff7f84640 -1 osd.5 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:23.013808+0000 front 2024-09-10T19:09:23.013766+0000 (oldest deadline 2024-09-10T19:09:47.113528+0000) 2024-09-10T19:09:52.772 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 19:09:52 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-4[38039]: 2024-09-10T19:09:52.468+0000 7fbb124e7640 -1 osd.4 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.194289+0000 front 2024-09-10T19:09:21.194335+0000 (oldest deadline 2024-09-10T19:09:45.294059+0000) 2024-09-10T19:09:52.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:09:52 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:09:52.668+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:20.280088+0000 front 2024-09-10T19:09:20.280106+0000 (oldest deadline 2024-09-10T19:09:44.979964+0000) 2024-09-10T19:09:52.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:09:52 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:09:52.462+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.285935+0000 front 2024-09-10T19:09:21.285773+0000 (oldest deadline 2024-09-10T19:09:48.286196+0000) 2024-09-10T19:09:53.449 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:09:53 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5[43919]: 2024-09-10T19:09:53.072+0000 7f5ff7f84640 -1 osd.5 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:23.013808+0000 front 2024-09-10T19:09:23.013766+0000 (oldest deadline 2024-09-10T19:09:47.113528+0000) 2024-09-10T19:09:53.449 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:09:53 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5[43919]: 2024-09-10T19:09:53.072+0000 7f5ff7f84640 -1 osd.5 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.614577+0000 front 2024-09-10T19:09:30.614568+0000 (oldest deadline 2024-09-10T19:09:52.914555+0000) 2024-09-10T19:09:53.772 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 19:09:53 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-4[38039]: 2024-09-10T19:09:53.447+0000 7fbb124e7640 -1 osd.4 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.194289+0000 front 2024-09-10T19:09:21.194335+0000 (oldest deadline 2024-09-10T19:09:45.294059+0000) 2024-09-10T19:09:53.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:09:53 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:09:53.634+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:20.280088+0000 front 2024-09-10T19:09:20.280106+0000 (oldest deadline 2024-09-10T19:09:44.979964+0000) 2024-09-10T19:09:53.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:09:53 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:09:53.504+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.285935+0000 front 2024-09-10T19:09:21.285773+0000 (oldest deadline 2024-09-10T19:09:48.286196+0000) 2024-09-10T19:09:53.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:09:53 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:09:53.504+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.587236+0000 front 2024-09-10T19:09:30.587183+0000 (oldest deadline 2024-09-10T19:09:53.487178+0000) 2024-09-10T19:09:54.413 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:09:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5[43919]: 2024-09-10T19:09:54.025+0000 7f5ff7f84640 -1 osd.5 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:23.013808+0000 front 2024-09-10T19:09:23.013766+0000 (oldest deadline 2024-09-10T19:09:47.113528+0000) 2024-09-10T19:09:54.413 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:09:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5[43919]: 2024-09-10T19:09:54.025+0000 7f5ff7f84640 -1 osd.5 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.614577+0000 front 2024-09-10T19:09:30.614568+0000 (oldest deadline 2024-09-10T19:09:52.914555+0000) 2024-09-10T19:09:54.772 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 19:09:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-4[38039]: 2024-09-10T19:09:54.411+0000 7fbb124e7640 -1 osd.4 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.194289+0000 front 2024-09-10T19:09:21.194335+0000 (oldest deadline 2024-09-10T19:09:45.294059+0000) 2024-09-10T19:09:54.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:09:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:09:54.631+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:20.280088+0000 front 2024-09-10T19:09:20.280106+0000 (oldest deadline 2024-09-10T19:09:44.979964+0000) 2024-09-10T19:09:54.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:09:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:09:54.631+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:31.381355+0000 front 2024-09-10T19:09:31.381441+0000 (oldest deadline 2024-09-10T19:09:54.281390+0000) 2024-09-10T19:09:54.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:09:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:09:54.455+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.285935+0000 front 2024-09-10T19:09:21.285773+0000 (oldest deadline 2024-09-10T19:09:48.286196+0000) 2024-09-10T19:09:54.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:09:54 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:09:54.455+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.587236+0000 front 2024-09-10T19:09:30.587183+0000 (oldest deadline 2024-09-10T19:09:53.487178+0000) 2024-09-10T19:09:55.383 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:09:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5[43919]: 2024-09-10T19:09:55.045+0000 7f5ff7f84640 -1 osd.5 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:23.013808+0000 front 2024-09-10T19:09:23.013766+0000 (oldest deadline 2024-09-10T19:09:47.113528+0000) 2024-09-10T19:09:55.383 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:09:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5[43919]: 2024-09-10T19:09:55.045+0000 7f5ff7f84640 -1 osd.5 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.614577+0000 front 2024-09-10T19:09:30.614568+0000 (oldest deadline 2024-09-10T19:09:52.914555+0000) 2024-09-10T19:09:55.772 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:09:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:09:55.593+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:20.280088+0000 front 2024-09-10T19:09:20.280106+0000 (oldest deadline 2024-09-10T19:09:44.979964+0000) 2024-09-10T19:09:55.772 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:09:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:09:55.593+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:31.381355+0000 front 2024-09-10T19:09:31.381441+0000 (oldest deadline 2024-09-10T19:09:54.281390+0000) 2024-09-10T19:09:55.773 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 19:09:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-4[38039]: 2024-09-10T19:09:55.381+0000 7fbb124e7640 -1 osd.4 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.194289+0000 front 2024-09-10T19:09:21.194335+0000 (oldest deadline 2024-09-10T19:09:45.294059+0000) 2024-09-10T19:09:55.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:09:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:09:55.412+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.285935+0000 front 2024-09-10T19:09:21.285773+0000 (oldest deadline 2024-09-10T19:09:48.286196+0000) 2024-09-10T19:09:55.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:09:55 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:09:55.413+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.587236+0000 front 2024-09-10T19:09:30.587183+0000 (oldest deadline 2024-09-10T19:09:53.487178+0000) 2024-09-10T19:09:56.399 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:09:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5[43919]: 2024-09-10T19:09:56.043+0000 7f5ff7f84640 -1 osd.5 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:23.013808+0000 front 2024-09-10T19:09:23.013766+0000 (oldest deadline 2024-09-10T19:09:47.113528+0000) 2024-09-10T19:09:56.399 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:09:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5[43919]: 2024-09-10T19:09:56.043+0000 7f5ff7f84640 -1 osd.5 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.614577+0000 front 2024-09-10T19:09:30.614568+0000 (oldest deadline 2024-09-10T19:09:52.914555+0000) 2024-09-10T19:09:56.772 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:09:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:09:56.605+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:20.280088+0000 front 2024-09-10T19:09:20.280106+0000 (oldest deadline 2024-09-10T19:09:44.979964+0000) 2024-09-10T19:09:56.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:09:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:09:56.605+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:31.381355+0000 front 2024-09-10T19:09:31.381441+0000 (oldest deadline 2024-09-10T19:09:54.281390+0000) 2024-09-10T19:09:56.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:09:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:09:56.443+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.285935+0000 front 2024-09-10T19:09:21.285773+0000 (oldest deadline 2024-09-10T19:09:48.286196+0000) 2024-09-10T19:09:56.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:09:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:09:56.443+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.587236+0000 front 2024-09-10T19:09:30.587183+0000 (oldest deadline 2024-09-10T19:09:53.487178+0000) 2024-09-10T19:09:56.773 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 19:09:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-4[38039]: 2024-09-10T19:09:56.397+0000 7fbb124e7640 -1 osd.4 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.194289+0000 front 2024-09-10T19:09:21.194335+0000 (oldest deadline 2024-09-10T19:09:45.294059+0000) 2024-09-10T19:09:56.773 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 19:09:56 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-4[38039]: 2024-09-10T19:09:56.397+0000 7fbb124e7640 -1 osd.4 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.595070+0000 front 2024-09-10T19:09:30.595093+0000 (oldest deadline 2024-09-10T19:09:55.895027+0000) 2024-09-10T19:09:57.421 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 19:09:57 smithi103 podman[152803]: 2024-09-10 19:09:57.235554224 +0000 UTC m=+5.729929686 container died 29eae64e4b256b3c09a21034ace5cfb93169500dba22df39ebcd3da773ada1eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-10T19:09:57.422 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:09:57 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5[43919]: 2024-09-10T19:09:57.067+0000 7f5ff7f84640 -1 osd.5 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:23.013808+0000 front 2024-09-10T19:09:23.013766+0000 (oldest deadline 2024-09-10T19:09:47.113528+0000) 2024-09-10T19:09:57.422 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:09:57 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5[43919]: 2024-09-10T19:09:57.067+0000 7f5ff7f84640 -1 osd.5 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.614577+0000 front 2024-09-10T19:09:30.614568+0000 (oldest deadline 2024-09-10T19:09:52.914555+0000) 2024-09-10T19:09:57.743 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:09:57 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:09:57.624+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:20.280088+0000 front 2024-09-10T19:09:20.280106+0000 (oldest deadline 2024-09-10T19:09:44.979964+0000) 2024-09-10T19:09:57.743 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:09:57 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:09:57.624+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:31.381355+0000 front 2024-09-10T19:09:31.381441+0000 (oldest deadline 2024-09-10T19:09:54.281390+0000) 2024-09-10T19:09:57.744 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:09:57 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:09:57.419+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.285935+0000 front 2024-09-10T19:09:21.285773+0000 (oldest deadline 2024-09-10T19:09:48.286196+0000) 2024-09-10T19:09:57.744 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:09:57 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:09:57.419+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.587236+0000 front 2024-09-10T19:09:30.587183+0000 (oldest deadline 2024-09-10T19:09:53.487178+0000) 2024-09-10T19:09:58.012 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 19:09:57 smithi103 podman[152803]: 2024-09-10 19:09:57.742627663 +0000 UTC m=+6.237003126 container cleanup 29eae64e4b256b3c09a21034ace5cfb93169500dba22df39ebcd3da773ada1eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T19:09:58.012 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 19:09:57 smithi103 bash[152803]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-4 2024-09-10T19:09:58.272 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 19:09:58 smithi103 podman[152817]: 2024-09-10 19:09:58.010935152 +0000 UTC m=+0.766611839 container remove 29eae64e4b256b3c09a21034ace5cfb93169500dba22df39ebcd3da773ada1eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-4, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-10T19:09:58.273 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:09:58 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5[43919]: 2024-09-10T19:09:58.113+0000 7f5ff7f84640 -1 osd.5 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:23.013808+0000 front 2024-09-10T19:09:23.013766+0000 (oldest deadline 2024-09-10T19:09:47.113528+0000) 2024-09-10T19:09:58.273 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:09:58 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5[43919]: 2024-09-10T19:09:58.113+0000 7f5ff7f84640 -1 osd.5 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.614577+0000 front 2024-09-10T19:09:30.614568+0000 (oldest deadline 2024-09-10T19:09:52.914555+0000) 2024-09-10T19:09:58.590 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:09:58 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:09:58.386+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.285935+0000 front 2024-09-10T19:09:21.285773+0000 (oldest deadline 2024-09-10T19:09:48.286196+0000) 2024-09-10T19:09:58.591 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:09:58 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:09:58.386+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.587236+0000 front 2024-09-10T19:09:30.587183+0000 (oldest deadline 2024-09-10T19:09:53.487178+0000) 2024-09-10T19:09:58.591 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 19:09:58 smithi103 podman[152909]: 2024-09-10 19:09:58.223458196 +0000 UTC m=+0.015844453 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:09:58.591 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 19:09:58 smithi103 podman[152909]: 2024-09-10 19:09:58.345260284 +0000 UTC m=+0.137646530 container create 1d6bde1d0eab9241cae97b74b8464a4acef4a17466930ae39c42743dcb4a360f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-10T19:09:58.591 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 19:09:58 smithi103 podman[152909]: 2024-09-10 19:09:58.589469966 +0000 UTC m=+0.381856211 container init 1d6bde1d0eab9241cae97b74b8464a4acef4a17466930ae39c42743dcb4a360f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-10T19:09:59.022 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:09:58 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:09:58.590+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:20.280088+0000 front 2024-09-10T19:09:20.280106+0000 (oldest deadline 2024-09-10T19:09:44.979964+0000) 2024-09-10T19:09:59.023 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:09:58 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:09:58.590+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:31.381355+0000 front 2024-09-10T19:09:31.381441+0000 (oldest deadline 2024-09-10T19:09:54.281390+0000) 2024-09-10T19:09:59.023 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 19:09:58 smithi103 podman[152909]: 2024-09-10 19:09:58.593216666 +0000 UTC m=+0.385602911 container start 1d6bde1d0eab9241cae97b74b8464a4acef4a17466930ae39c42743dcb4a360f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-10T19:09:59.023 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 19:09:58 smithi103 podman[152909]: 2024-09-10 19:09:58.697942366 +0000 UTC m=+0.490328637 container attach 1d6bde1d0eab9241cae97b74b8464a4acef4a17466930ae39c42743dcb4a360f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-10T19:09:59.023 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 19:09:58 smithi103 podman[152909]: 2024-09-10 19:09:58.801816753 +0000 UTC m=+0.594203008 container died 1d6bde1d0eab9241cae97b74b8464a4acef4a17466930ae39c42743dcb4a360f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-4-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-10T19:09:59.403 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:09:59 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5[43919]: 2024-09-10T19:09:59.112+0000 7f5ff7f84640 -1 osd.5 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:23.013808+0000 front 2024-09-10T19:09:23.013766+0000 (oldest deadline 2024-09-10T19:09:47.113528+0000) 2024-09-10T19:09:59.403 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:09:59 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5[43919]: 2024-09-10T19:09:59.113+0000 7f5ff7f84640 -1 osd.5 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.614577+0000 front 2024-09-10T19:09:30.614568+0000 (oldest deadline 2024-09-10T19:09:52.914555+0000) 2024-09-10T19:09:59.427 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@osd.4.service' 2024-09-10T19:09:59.469 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:09:59.469 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-10T19:09:59.469 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-10T19:09:59.470 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@osd.5 2024-09-10T19:09:59.689 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:09:59 smithi103 systemd[1]: Stopping Ceph osd.5 for ba327f6c-6fa3-11ef-bcea-c7b262605968... 2024-09-10T19:09:59.690 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:09:59 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:09:59.545+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:20.280088+0000 front 2024-09-10T19:09:20.280106+0000 (oldest deadline 2024-09-10T19:09:44.979964+0000) 2024-09-10T19:09:59.690 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:09:59 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:09:59.545+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:31.381355+0000 front 2024-09-10T19:09:31.381441+0000 (oldest deadline 2024-09-10T19:09:54.281390+0000) 2024-09-10T19:09:59.690 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:09:59 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:09:59.420+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.285935+0000 front 2024-09-10T19:09:21.285773+0000 (oldest deadline 2024-09-10T19:09:48.286196+0000) 2024-09-10T19:09:59.690 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:09:59 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:09:59.420+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.587236+0000 front 2024-09-10T19:09:30.587183+0000 (oldest deadline 2024-09-10T19:09:53.487178+0000) 2024-09-10T19:10:00.022 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:09:59 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5[43919]: 2024-09-10T19:09:59.796+0000 7f5ffc16d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:10:00.022 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:09:59 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5[43919]: 2024-09-10T19:09:59.796+0000 7f5ffc16d640 -1 osd.5 497 *** Got signal Terminated *** 2024-09-10T19:10:00.023 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:09:59 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5[43919]: 2024-09-10T19:09:59.796+0000 7f5ffc16d640 -1 osd.5 497 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T19:10:00.432 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:10:00 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5[43919]: 2024-09-10T19:10:00.096+0000 7f5ff7f84640 -1 osd.5 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:23.013808+0000 front 2024-09-10T19:09:23.013766+0000 (oldest deadline 2024-09-10T19:09:47.113528+0000) 2024-09-10T19:10:00.432 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:10:00 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5[43919]: 2024-09-10T19:10:00.096+0000 7f5ff7f84640 -1 osd.5 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.614577+0000 front 2024-09-10T19:09:30.614568+0000 (oldest deadline 2024-09-10T19:09:52.914555+0000) 2024-09-10T19:10:00.772 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:00 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:00.430+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.285935+0000 front 2024-09-10T19:09:21.285773+0000 (oldest deadline 2024-09-10T19:09:48.286196+0000) 2024-09-10T19:10:00.772 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:00 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:00.430+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.587236+0000 front 2024-09-10T19:09:30.587183+0000 (oldest deadline 2024-09-10T19:09:53.487178+0000) 2024-09-10T19:10:00.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:00 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:00.572+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:20.280088+0000 front 2024-09-10T19:09:20.280106+0000 (oldest deadline 2024-09-10T19:09:44.979964+0000) 2024-09-10T19:10:00.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:00 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:00.572+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:31.381355+0000 front 2024-09-10T19:09:31.381441+0000 (oldest deadline 2024-09-10T19:09:54.281390+0000) 2024-09-10T19:10:01.396 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:10:01 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5[43919]: 2024-09-10T19:10:01.060+0000 7f5ff7f84640 -1 osd.5 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:23.013808+0000 front 2024-09-10T19:09:23.013766+0000 (oldest deadline 2024-09-10T19:09:47.113528+0000) 2024-09-10T19:10:01.396 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:10:01 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5[43919]: 2024-09-10T19:10:01.060+0000 7f5ff7f84640 -1 osd.5 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.614577+0000 front 2024-09-10T19:09:30.614568+0000 (oldest deadline 2024-09-10T19:09:52.914555+0000) 2024-09-10T19:10:01.772 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:01 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:01.395+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.285935+0000 front 2024-09-10T19:09:21.285773+0000 (oldest deadline 2024-09-10T19:09:48.286196+0000) 2024-09-10T19:10:01.772 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:01 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:01.395+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.587236+0000 front 2024-09-10T19:09:30.587183+0000 (oldest deadline 2024-09-10T19:09:53.487178+0000) 2024-09-10T19:10:01.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:01 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:01.547+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:20.280088+0000 front 2024-09-10T19:09:20.280106+0000 (oldest deadline 2024-09-10T19:09:44.979964+0000) 2024-09-10T19:10:01.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:01 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:01.547+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:31.381355+0000 front 2024-09-10T19:09:31.381441+0000 (oldest deadline 2024-09-10T19:09:54.281390+0000) 2024-09-10T19:10:01.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:01 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:01.547+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6811 osd.2 since back 2024-09-10T19:09:38.282762+0000 front 2024-09-10T19:09:38.282862+0000 (oldest deadline 2024-09-10T19:10:01.182759+0000) 2024-09-10T19:10:02.358 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:10:02 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5[43919]: 2024-09-10T19:10:02.060+0000 7f5ff7f84640 -1 osd.5 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:23.013808+0000 front 2024-09-10T19:09:23.013766+0000 (oldest deadline 2024-09-10T19:09:47.113528+0000) 2024-09-10T19:10:02.358 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:10:02 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5[43919]: 2024-09-10T19:10:02.060+0000 7f5ff7f84640 -1 osd.5 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.614577+0000 front 2024-09-10T19:09:30.614568+0000 (oldest deadline 2024-09-10T19:09:52.914555+0000) 2024-09-10T19:10:02.358 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:10:02 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5[43919]: 2024-09-10T19:10:02.060+0000 7f5ff7f84640 -1 osd.5 497 heartbeat_check: no reply from 172.21.15.18:6811 osd.2 since back 2024-09-10T19:09:39.216639+0000 front 2024-09-10T19:09:39.216626+0000 (oldest deadline 2024-09-10T19:10:01.516395+0000) 2024-09-10T19:10:02.772 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:02 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:02.515+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:20.280088+0000 front 2024-09-10T19:09:20.280106+0000 (oldest deadline 2024-09-10T19:09:44.979964+0000) 2024-09-10T19:10:02.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:02 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:02.515+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:31.381355+0000 front 2024-09-10T19:09:31.381441+0000 (oldest deadline 2024-09-10T19:09:54.281390+0000) 2024-09-10T19:10:02.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:02 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:02.515+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6811 osd.2 since back 2024-09-10T19:09:38.282762+0000 front 2024-09-10T19:09:38.282862+0000 (oldest deadline 2024-09-10T19:10:01.182759+0000) 2024-09-10T19:10:02.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:02 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:02.356+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.285935+0000 front 2024-09-10T19:09:21.285773+0000 (oldest deadline 2024-09-10T19:09:48.286196+0000) 2024-09-10T19:10:02.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:02 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:02.356+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.587236+0000 front 2024-09-10T19:09:30.587183+0000 (oldest deadline 2024-09-10T19:09:53.487178+0000) 2024-09-10T19:10:02.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:02 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:02.356+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6811 osd.2 since back 2024-09-10T19:09:37.588068+0000 front 2024-09-10T19:09:37.588110+0000 (oldest deadline 2024-09-10T19:10:01.688018+0000) 2024-09-10T19:10:03.272 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:10:03 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5[43919]: 2024-09-10T19:10:03.013+0000 7f5ff7f84640 -1 osd.5 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:23.013808+0000 front 2024-09-10T19:09:23.013766+0000 (oldest deadline 2024-09-10T19:09:47.113528+0000) 2024-09-10T19:10:03.273 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:10:03 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5[43919]: 2024-09-10T19:10:03.013+0000 7f5ff7f84640 -1 osd.5 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.614577+0000 front 2024-09-10T19:09:30.614568+0000 (oldest deadline 2024-09-10T19:09:52.914555+0000) 2024-09-10T19:10:03.273 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:10:03 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5[43919]: 2024-09-10T19:10:03.013+0000 7f5ff7f84640 -1 osd.5 497 heartbeat_check: no reply from 172.21.15.18:6811 osd.2 since back 2024-09-10T19:09:39.216639+0000 front 2024-09-10T19:09:39.216626+0000 (oldest deadline 2024-09-10T19:10:01.516395+0000) 2024-09-10T19:10:03.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:03 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:03.555+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:20.280088+0000 front 2024-09-10T19:09:20.280106+0000 (oldest deadline 2024-09-10T19:09:44.979964+0000) 2024-09-10T19:10:03.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:03 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:03.555+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:31.381355+0000 front 2024-09-10T19:09:31.381441+0000 (oldest deadline 2024-09-10T19:09:54.281390+0000) 2024-09-10T19:10:03.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:03 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:03.555+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6811 osd.2 since back 2024-09-10T19:09:38.282762+0000 front 2024-09-10T19:09:38.282862+0000 (oldest deadline 2024-09-10T19:10:01.182759+0000) 2024-09-10T19:10:03.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:03 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:03.365+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.285935+0000 front 2024-09-10T19:09:21.285773+0000 (oldest deadline 2024-09-10T19:09:48.286196+0000) 2024-09-10T19:10:03.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:03 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:03.365+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.587236+0000 front 2024-09-10T19:09:30.587183+0000 (oldest deadline 2024-09-10T19:09:53.487178+0000) 2024-09-10T19:10:03.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:03 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:03.365+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6811 osd.2 since back 2024-09-10T19:09:37.588068+0000 front 2024-09-10T19:09:37.588110+0000 (oldest deadline 2024-09-10T19:10:01.688018+0000) 2024-09-10T19:10:04.272 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:10:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5[43919]: 2024-09-10T19:10:04.006+0000 7f5ff7f84640 -1 osd.5 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:23.013808+0000 front 2024-09-10T19:09:23.013766+0000 (oldest deadline 2024-09-10T19:09:47.113528+0000) 2024-09-10T19:10:04.273 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:10:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5[43919]: 2024-09-10T19:10:04.006+0000 7f5ff7f84640 -1 osd.5 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.614577+0000 front 2024-09-10T19:09:30.614568+0000 (oldest deadline 2024-09-10T19:09:52.914555+0000) 2024-09-10T19:10:04.273 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:10:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5[43919]: 2024-09-10T19:10:04.006+0000 7f5ff7f84640 -1 osd.5 497 heartbeat_check: no reply from 172.21.15.18:6811 osd.2 since back 2024-09-10T19:09:39.216639+0000 front 2024-09-10T19:09:39.216626+0000 (oldest deadline 2024-09-10T19:10:01.516395+0000) 2024-09-10T19:10:04.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:04.569+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:20.280088+0000 front 2024-09-10T19:09:20.280106+0000 (oldest deadline 2024-09-10T19:09:44.979964+0000) 2024-09-10T19:10:04.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:04.569+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:31.381355+0000 front 2024-09-10T19:09:31.381441+0000 (oldest deadline 2024-09-10T19:09:54.281390+0000) 2024-09-10T19:10:04.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:04.569+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6811 osd.2 since back 2024-09-10T19:09:38.282762+0000 front 2024-09-10T19:09:38.282862+0000 (oldest deadline 2024-09-10T19:10:01.182759+0000) 2024-09-10T19:10:04.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:04.415+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.285935+0000 front 2024-09-10T19:09:21.285773+0000 (oldest deadline 2024-09-10T19:09:48.286196+0000) 2024-09-10T19:10:04.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:04.415+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.587236+0000 front 2024-09-10T19:09:30.587183+0000 (oldest deadline 2024-09-10T19:09:53.487178+0000) 2024-09-10T19:10:04.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:04 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:04.415+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6811 osd.2 since back 2024-09-10T19:09:37.588068+0000 front 2024-09-10T19:09:37.588110+0000 (oldest deadline 2024-09-10T19:10:01.688018+0000) 2024-09-10T19:10:05.612 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:10:05 smithi103 podman[153011]: 2024-09-10 19:10:05.360553013 +0000 UTC m=+5.662687607 container died 34e412ebb048aae2746addd183636745a9ee651e4261e1d9ca8a7cb6405819f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-10T19:10:05.613 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:05 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:05.458+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.285935+0000 front 2024-09-10T19:09:21.285773+0000 (oldest deadline 2024-09-10T19:09:48.286196+0000) 2024-09-10T19:10:05.613 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:05 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:05.458+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.587236+0000 front 2024-09-10T19:09:30.587183+0000 (oldest deadline 2024-09-10T19:09:53.487178+0000) 2024-09-10T19:10:05.613 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:05 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:05.458+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6811 osd.2 since back 2024-09-10T19:09:37.588068+0000 front 2024-09-10T19:09:37.588110+0000 (oldest deadline 2024-09-10T19:10:01.688018+0000) 2024-09-10T19:10:05.865 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:05 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:05.611+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:20.280088+0000 front 2024-09-10T19:09:20.280106+0000 (oldest deadline 2024-09-10T19:09:44.979964+0000) 2024-09-10T19:10:05.866 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:05 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:05.611+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:31.381355+0000 front 2024-09-10T19:09:31.381441+0000 (oldest deadline 2024-09-10T19:09:54.281390+0000) 2024-09-10T19:10:05.866 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:05 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:05.611+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6811 osd.2 since back 2024-09-10T19:09:38.282762+0000 front 2024-09-10T19:09:38.282862+0000 (oldest deadline 2024-09-10T19:10:01.182759+0000) 2024-09-10T19:10:05.866 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:10:05 smithi103 podman[153011]: 2024-09-10 19:10:05.846649929 +0000 UTC m=+6.148784524 container cleanup 34e412ebb048aae2746addd183636745a9ee651e4261e1d9ca8a7cb6405819f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-10T19:10:05.866 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:10:05 smithi103 bash[153011]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5 2024-09-10T19:10:06.461 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:10:06 smithi103 podman[153040]: 2024-09-10 19:10:06.173048197 +0000 UTC m=+0.804430390 container remove 34e412ebb048aae2746addd183636745a9ee651e4261e1d9ca8a7cb6405819f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2) 2024-09-10T19:10:06.772 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:06 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:06.459+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.285935+0000 front 2024-09-10T19:09:21.285773+0000 (oldest deadline 2024-09-10T19:09:48.286196+0000) 2024-09-10T19:10:06.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:06 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:06.459+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.587236+0000 front 2024-09-10T19:09:30.587183+0000 (oldest deadline 2024-09-10T19:09:53.487178+0000) 2024-09-10T19:10:06.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:06 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:06.459+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6811 osd.2 since back 2024-09-10T19:09:37.588068+0000 front 2024-09-10T19:09:37.588110+0000 (oldest deadline 2024-09-10T19:10:01.688018+0000) 2024-09-10T19:10:06.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:06 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:06.622+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:20.280088+0000 front 2024-09-10T19:09:20.280106+0000 (oldest deadline 2024-09-10T19:09:44.979964+0000) 2024-09-10T19:10:06.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:06 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:06.622+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:31.381355+0000 front 2024-09-10T19:09:31.381441+0000 (oldest deadline 2024-09-10T19:09:54.281390+0000) 2024-09-10T19:10:06.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:06 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:06.622+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6811 osd.2 since back 2024-09-10T19:09:38.282762+0000 front 2024-09-10T19:09:38.282862+0000 (oldest deadline 2024-09-10T19:10:01.182759+0000) 2024-09-10T19:10:06.773 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:10:06 smithi103 podman[153129]: 2024-09-10 19:10:06.473943192 +0000 UTC m=+0.104215624 container create 94bc8356aac80e1d9b3b140d51b38afd4780a83f4db09004ed9cac233ff06c4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-10T19:10:06.774 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:10:06 smithi103 podman[153129]: 2024-09-10 19:10:06.385872339 +0000 UTC m=+0.016144773 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:10:06.774 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:10:06 smithi103 podman[153129]: 2024-09-10 19:10:06.685496631 +0000 UTC m=+0.315769064 container init 94bc8356aac80e1d9b3b140d51b38afd4780a83f4db09004ed9cac233ff06c4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-10T19:10:06.774 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:10:06 smithi103 podman[153129]: 2024-09-10 19:10:06.688852914 +0000 UTC m=+0.319125354 container start 94bc8356aac80e1d9b3b140d51b38afd4780a83f4db09004ed9cac233ff06c4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5-deactivate, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-10T19:10:07.206 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:10:06 smithi103 podman[153129]: 2024-09-10 19:10:06.803266541 +0000 UTC m=+0.433539008 container attach 94bc8356aac80e1d9b3b140d51b38afd4780a83f4db09004ed9cac233ff06c4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-10T19:10:07.207 INFO:journalctl@ceph.osd.5.smithi103.stdout:Sep 10 19:10:06 smithi103 podman[153129]: 2024-09-10 19:10:06.862500433 +0000 UTC m=+0.492772874 container died 94bc8356aac80e1d9b3b140d51b38afd4780a83f4db09004ed9cac233ff06c4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T19:10:07.506 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@osd.5.service' 2024-09-10T19:10:07.549 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:10:07.550 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-10T19:10:07.550 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-10T19:10:07.550 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@osd.6 2024-09-10T19:10:07.772 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:07 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:07.465+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.285935+0000 front 2024-09-10T19:09:21.285773+0000 (oldest deadline 2024-09-10T19:09:48.286196+0000) 2024-09-10T19:10:07.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:07 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:07.465+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.587236+0000 front 2024-09-10T19:09:30.587183+0000 (oldest deadline 2024-09-10T19:09:53.487178+0000) 2024-09-10T19:10:07.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:07 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:07.465+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6811 osd.2 since back 2024-09-10T19:09:37.588068+0000 front 2024-09-10T19:09:37.588110+0000 (oldest deadline 2024-09-10T19:10:01.688018+0000) 2024-09-10T19:10:07.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:07 smithi103 systemd[1]: Stopping Ceph osd.6 for ba327f6c-6fa3-11ef-bcea-c7b262605968... 2024-09-10T19:10:07.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:07 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:07.647+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:20.280088+0000 front 2024-09-10T19:09:20.280106+0000 (oldest deadline 2024-09-10T19:09:44.979964+0000) 2024-09-10T19:10:07.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:07 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:07.647+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:31.381355+0000 front 2024-09-10T19:09:31.381441+0000 (oldest deadline 2024-09-10T19:09:54.281390+0000) 2024-09-10T19:10:07.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:07 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:07.647+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6811 osd.2 since back 2024-09-10T19:09:38.282762+0000 front 2024-09-10T19:09:38.282862+0000 (oldest deadline 2024-09-10T19:10:01.182759+0000) 2024-09-10T19:10:08.272 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:07 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:07.883+0000 7f636da7d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:10:08.273 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:07 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:07.883+0000 7f636da7d640 -1 osd.6 497 *** Got signal Terminated *** 2024-09-10T19:10:08.273 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:07 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:07.883+0000 7f636da7d640 -1 osd.6 497 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T19:10:08.772 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:08 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:08.673+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:20.280088+0000 front 2024-09-10T19:09:20.280106+0000 (oldest deadline 2024-09-10T19:09:44.979964+0000) 2024-09-10T19:10:08.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:08 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:08.673+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:31.381355+0000 front 2024-09-10T19:09:31.381441+0000 (oldest deadline 2024-09-10T19:09:54.281390+0000) 2024-09-10T19:10:08.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:08 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:08.673+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6811 osd.2 since back 2024-09-10T19:09:38.282762+0000 front 2024-09-10T19:09:38.282862+0000 (oldest deadline 2024-09-10T19:10:01.182759+0000) 2024-09-10T19:10:08.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:08 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:08.435+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.285935+0000 front 2024-09-10T19:09:21.285773+0000 (oldest deadline 2024-09-10T19:09:48.286196+0000) 2024-09-10T19:10:08.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:08 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:08.435+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.587236+0000 front 2024-09-10T19:09:30.587183+0000 (oldest deadline 2024-09-10T19:09:53.487178+0000) 2024-09-10T19:10:08.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:08 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:08.435+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6811 osd.2 since back 2024-09-10T19:09:37.588068+0000 front 2024-09-10T19:09:37.588110+0000 (oldest deadline 2024-09-10T19:10:01.688018+0000) 2024-09-10T19:10:09.772 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:09 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:09.668+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:20.280088+0000 front 2024-09-10T19:09:20.280106+0000 (oldest deadline 2024-09-10T19:09:44.979964+0000) 2024-09-10T19:10:09.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:09 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:09.668+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:31.381355+0000 front 2024-09-10T19:09:31.381441+0000 (oldest deadline 2024-09-10T19:09:54.281390+0000) 2024-09-10T19:10:09.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:09 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:09.668+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6811 osd.2 since back 2024-09-10T19:09:38.282762+0000 front 2024-09-10T19:09:38.282862+0000 (oldest deadline 2024-09-10T19:10:01.182759+0000) 2024-09-10T19:10:09.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:09 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:09.434+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.285935+0000 front 2024-09-10T19:09:21.285773+0000 (oldest deadline 2024-09-10T19:09:48.286196+0000) 2024-09-10T19:10:09.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:09 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:09.434+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.587236+0000 front 2024-09-10T19:09:30.587183+0000 (oldest deadline 2024-09-10T19:09:53.487178+0000) 2024-09-10T19:10:09.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:09 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:09.434+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6811 osd.2 since back 2024-09-10T19:09:37.588068+0000 front 2024-09-10T19:09:37.588110+0000 (oldest deadline 2024-09-10T19:10:01.688018+0000) 2024-09-10T19:10:10.727 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:10 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:10.621+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:20.280088+0000 front 2024-09-10T19:09:20.280106+0000 (oldest deadline 2024-09-10T19:09:44.979964+0000) 2024-09-10T19:10:10.727 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:10 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:10.621+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:31.381355+0000 front 2024-09-10T19:09:31.381441+0000 (oldest deadline 2024-09-10T19:09:54.281390+0000) 2024-09-10T19:10:10.727 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:10 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:10.621+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6811 osd.2 since back 2024-09-10T19:09:38.282762+0000 front 2024-09-10T19:09:38.282862+0000 (oldest deadline 2024-09-10T19:10:01.182759+0000) 2024-09-10T19:10:10.728 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:10 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:10.452+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.285935+0000 front 2024-09-10T19:09:21.285773+0000 (oldest deadline 2024-09-10T19:09:48.286196+0000) 2024-09-10T19:10:10.728 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:10 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:10.452+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.587236+0000 front 2024-09-10T19:09:30.587183+0000 (oldest deadline 2024-09-10T19:09:53.487178+0000) 2024-09-10T19:10:10.728 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:10 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:10.452+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6811 osd.2 since back 2024-09-10T19:09:37.588068+0000 front 2024-09-10T19:09:37.588110+0000 (oldest deadline 2024-09-10T19:10:01.688018+0000) 2024-09-10T19:10:11.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:11 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:11.575+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:20.280088+0000 front 2024-09-10T19:09:20.280106+0000 (oldest deadline 2024-09-10T19:09:44.979964+0000) 2024-09-10T19:10:11.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:11 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:11.575+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:31.381355+0000 front 2024-09-10T19:09:31.381441+0000 (oldest deadline 2024-09-10T19:09:54.281390+0000) 2024-09-10T19:10:11.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:11 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:11.575+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6811 osd.2 since back 2024-09-10T19:09:38.282762+0000 front 2024-09-10T19:09:38.282862+0000 (oldest deadline 2024-09-10T19:10:01.182759+0000) 2024-09-10T19:10:11.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:11 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:11.457+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.285935+0000 front 2024-09-10T19:09:21.285773+0000 (oldest deadline 2024-09-10T19:09:48.286196+0000) 2024-09-10T19:10:11.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:11 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:11.457+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.587236+0000 front 2024-09-10T19:09:30.587183+0000 (oldest deadline 2024-09-10T19:09:53.487178+0000) 2024-09-10T19:10:11.774 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:11 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:11.457+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6811 osd.2 since back 2024-09-10T19:09:37.588068+0000 front 2024-09-10T19:09:37.588110+0000 (oldest deadline 2024-09-10T19:10:01.688018+0000) 2024-09-10T19:10:12.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:12 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:12.604+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:20.280088+0000 front 2024-09-10T19:09:20.280106+0000 (oldest deadline 2024-09-10T19:09:44.979964+0000) 2024-09-10T19:10:12.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:12 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:12.604+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:31.381355+0000 front 2024-09-10T19:09:31.381441+0000 (oldest deadline 2024-09-10T19:09:54.281390+0000) 2024-09-10T19:10:12.773 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:12 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6[49849]: 2024-09-10T19:10:12.604+0000 7f6368091640 -1 osd.6 497 heartbeat_check: no reply from 172.21.15.18:6811 osd.2 since back 2024-09-10T19:09:38.282762+0000 front 2024-09-10T19:09:38.282862+0000 (oldest deadline 2024-09-10T19:10:01.182759+0000) 2024-09-10T19:10:12.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:12 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:12.469+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.285935+0000 front 2024-09-10T19:09:21.285773+0000 (oldest deadline 2024-09-10T19:09:48.286196+0000) 2024-09-10T19:10:12.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:12 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:12.469+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.587236+0000 front 2024-09-10T19:09:30.587183+0000 (oldest deadline 2024-09-10T19:09:53.487178+0000) 2024-09-10T19:10:12.774 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:12 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:12.469+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6811 osd.2 since back 2024-09-10T19:09:37.588068+0000 front 2024-09-10T19:09:37.588110+0000 (oldest deadline 2024-09-10T19:10:01.688018+0000) 2024-09-10T19:10:13.772 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:13 smithi103 podman[153233]: 2024-09-10 19:10:13.523420014 +0000 UTC m=+5.739978497 container died ae16f8f84ae58dd1e5a6a1dd5e3bc0be034e36b0bf1ab177a422f12936ea6df7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T19:10:13.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:13 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:13.460+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.285935+0000 front 2024-09-10T19:09:21.285773+0000 (oldest deadline 2024-09-10T19:09:48.286196+0000) 2024-09-10T19:10:13.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:13 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:13.460+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.587236+0000 front 2024-09-10T19:09:30.587183+0000 (oldest deadline 2024-09-10T19:09:53.487178+0000) 2024-09-10T19:10:13.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:13 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:13.460+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6811 osd.2 since back 2024-09-10T19:09:37.588068+0000 front 2024-09-10T19:09:37.588110+0000 (oldest deadline 2024-09-10T19:10:01.688018+0000) 2024-09-10T19:10:14.219 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:13 smithi103 podman[153233]: 2024-09-10 19:10:13.90039288 +0000 UTC m=+6.116951321 container cleanup ae16f8f84ae58dd1e5a6a1dd5e3bc0be034e36b0bf1ab177a422f12936ea6df7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-10T19:10:14.219 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:13 smithi103 bash[153233]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6 2024-09-10T19:10:14.522 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:14 smithi103 podman[153248]: 2024-09-10 19:10:14.218546866 +0000 UTC m=+0.682083719 container remove ae16f8f84ae58dd1e5a6a1dd5e3bc0be034e36b0bf1ab177a422f12936ea6df7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-10T19:10:14.523 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:14 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:14.450+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.285935+0000 front 2024-09-10T19:09:21.285773+0000 (oldest deadline 2024-09-10T19:09:48.286196+0000) 2024-09-10T19:10:14.523 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:14 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:14.450+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.587236+0000 front 2024-09-10T19:09:30.587183+0000 (oldest deadline 2024-09-10T19:09:53.487178+0000) 2024-09-10T19:10:14.523 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:14 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:14.450+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6811 osd.2 since back 2024-09-10T19:09:37.588068+0000 front 2024-09-10T19:09:37.588110+0000 (oldest deadline 2024-09-10T19:10:01.688018+0000) 2024-09-10T19:10:14.523 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:14 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:14.450+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6815 osd.3 since back 2024-09-10T19:09:47.890005+0000 front 2024-09-10T19:09:47.890023+0000 (oldest deadline 2024-09-10T19:10:13.790044+0000) 2024-09-10T19:10:14.848 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:14 smithi103 podman[153336]: 2024-09-10 19:10:14.427543698 +0000 UTC m=+0.015578379 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:10:14.848 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:14 smithi103 podman[153336]: 2024-09-10 19:10:14.536222779 +0000 UTC m=+0.124257500 container create daa9cbaf7328267ffe670cc1dc759243e116ce80caf2ec86c7d6a963cdb4e362 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-10T19:10:14.848 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:14 smithi103 podman[153336]: 2024-09-10 19:10:14.746885665 +0000 UTC m=+0.334920341 container init daa9cbaf7328267ffe670cc1dc759243e116ce80caf2ec86c7d6a963cdb4e362 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T19:10:14.848 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:14 smithi103 podman[153336]: 2024-09-10 19:10:14.750094963 +0000 UTC m=+0.338129640 container start daa9cbaf7328267ffe670cc1dc759243e116ce80caf2ec86c7d6a963cdb4e362 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T19:10:14.848 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:14 smithi103 podman[153336]: 2024-09-10 19:10:14.846965072 +0000 UTC m=+0.434999766 container attach daa9cbaf7328267ffe670cc1dc759243e116ce80caf2ec86c7d6a963cdb4e362 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-10T19:10:15.234 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 10 19:10:14 smithi103 podman[153336]: 2024-09-10 19:10:14.924793741 +0000 UTC m=+0.512828420 container died daa9cbaf7328267ffe670cc1dc759243e116ce80caf2ec86c7d6a963cdb4e362 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T19:10:15.495 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:15 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:15.468+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.285935+0000 front 2024-09-10T19:09:21.285773+0000 (oldest deadline 2024-09-10T19:09:48.286196+0000) 2024-09-10T19:10:15.495 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:15 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:15.468+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.587236+0000 front 2024-09-10T19:09:30.587183+0000 (oldest deadline 2024-09-10T19:09:53.487178+0000) 2024-09-10T19:10:15.496 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:15 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:15.468+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6811 osd.2 since back 2024-09-10T19:09:37.588068+0000 front 2024-09-10T19:09:37.588110+0000 (oldest deadline 2024-09-10T19:10:01.688018+0000) 2024-09-10T19:10:15.496 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:15 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:15.468+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6815 osd.3 since back 2024-09-10T19:09:47.890005+0000 front 2024-09-10T19:09:47.890023+0000 (oldest deadline 2024-09-10T19:10:13.790044+0000) 2024-09-10T19:10:15.517 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@osd.6.service' 2024-09-10T19:10:15.555 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:10:15.556 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-10T19:10:15.556 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-10T19:10:15.556 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@osd.7 2024-09-10T19:10:15.772 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:15 smithi103 systemd[1]: Stopping Ceph osd.7 for ba327f6c-6fa3-11ef-bcea-c7b262605968... 2024-09-10T19:10:16.272 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:15 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:15.880+0000 7fadc60ab640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:10:16.272 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:15 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:15.880+0000 7fadc60ab640 -1 osd.7 497 *** Got signal Terminated *** 2024-09-10T19:10:16.273 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:15 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:15.880+0000 7fadc60ab640 -1 osd.7 497 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T19:10:16.533 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:16 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:16.478+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.285935+0000 front 2024-09-10T19:09:21.285773+0000 (oldest deadline 2024-09-10T19:09:48.286196+0000) 2024-09-10T19:10:16.533 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:16 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:16.478+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.587236+0000 front 2024-09-10T19:09:30.587183+0000 (oldest deadline 2024-09-10T19:09:53.487178+0000) 2024-09-10T19:10:16.533 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:16 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:16.478+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6811 osd.2 since back 2024-09-10T19:09:37.588068+0000 front 2024-09-10T19:09:37.588110+0000 (oldest deadline 2024-09-10T19:10:01.688018+0000) 2024-09-10T19:10:16.534 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:16 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:16.478+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6815 osd.3 since back 2024-09-10T19:09:47.890005+0000 front 2024-09-10T19:09:47.890023+0000 (oldest deadline 2024-09-10T19:10:13.790044+0000) 2024-09-10T19:10:17.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:17 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:17.451+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.285935+0000 front 2024-09-10T19:09:21.285773+0000 (oldest deadline 2024-09-10T19:09:48.286196+0000) 2024-09-10T19:10:17.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:17 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:17.451+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.587236+0000 front 2024-09-10T19:09:30.587183+0000 (oldest deadline 2024-09-10T19:09:53.487178+0000) 2024-09-10T19:10:17.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:17 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:17.451+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6811 osd.2 since back 2024-09-10T19:09:37.588068+0000 front 2024-09-10T19:09:37.588110+0000 (oldest deadline 2024-09-10T19:10:01.688018+0000) 2024-09-10T19:10:17.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:17 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:17.451+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6815 osd.3 since back 2024-09-10T19:09:47.890005+0000 front 2024-09-10T19:09:47.890023+0000 (oldest deadline 2024-09-10T19:10:13.790044+0000) 2024-09-10T19:10:18.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:18 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:18.414+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.285935+0000 front 2024-09-10T19:09:21.285773+0000 (oldest deadline 2024-09-10T19:09:48.286196+0000) 2024-09-10T19:10:18.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:18 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:18.414+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.587236+0000 front 2024-09-10T19:09:30.587183+0000 (oldest deadline 2024-09-10T19:09:53.487178+0000) 2024-09-10T19:10:18.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:18 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:18.414+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6811 osd.2 since back 2024-09-10T19:09:37.588068+0000 front 2024-09-10T19:09:37.588110+0000 (oldest deadline 2024-09-10T19:10:01.688018+0000) 2024-09-10T19:10:18.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:18 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:18.414+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6815 osd.3 since back 2024-09-10T19:09:47.890005+0000 front 2024-09-10T19:09:47.890023+0000 (oldest deadline 2024-09-10T19:10:13.790044+0000) 2024-09-10T19:10:19.772 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:19 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:19.416+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.285935+0000 front 2024-09-10T19:09:21.285773+0000 (oldest deadline 2024-09-10T19:09:48.286196+0000) 2024-09-10T19:10:19.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:19 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:19.416+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.587236+0000 front 2024-09-10T19:09:30.587183+0000 (oldest deadline 2024-09-10T19:09:53.487178+0000) 2024-09-10T19:10:19.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:19 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:19.416+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6811 osd.2 since back 2024-09-10T19:09:37.588068+0000 front 2024-09-10T19:09:37.588110+0000 (oldest deadline 2024-09-10T19:10:01.688018+0000) 2024-09-10T19:10:19.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:19 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:19.416+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6815 osd.3 since back 2024-09-10T19:09:47.890005+0000 front 2024-09-10T19:09:47.890023+0000 (oldest deadline 2024-09-10T19:10:13.790044+0000) 2024-09-10T19:10:20.772 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:20 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:20.402+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6803 osd.0 since back 2024-09-10T19:09:21.285935+0000 front 2024-09-10T19:09:21.285773+0000 (oldest deadline 2024-09-10T19:09:48.286196+0000) 2024-09-10T19:10:20.772 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:20 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:20.402+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6807 osd.1 since back 2024-09-10T19:09:30.587236+0000 front 2024-09-10T19:09:30.587183+0000 (oldest deadline 2024-09-10T19:09:53.487178+0000) 2024-09-10T19:10:20.772 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:20 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:20.402+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6811 osd.2 since back 2024-09-10T19:09:37.588068+0000 front 2024-09-10T19:09:37.588110+0000 (oldest deadline 2024-09-10T19:10:01.688018+0000) 2024-09-10T19:10:20.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:20 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7[140911]: 2024-09-10T19:10:20.402+0000 7fadc16c1640 -1 osd.7 497 heartbeat_check: no reply from 172.21.15.18:6815 osd.3 since back 2024-09-10T19:09:47.890005+0000 front 2024-09-10T19:09:47.890023+0000 (oldest deadline 2024-09-10T19:10:13.790044+0000) 2024-09-10T19:10:21.515 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:21 smithi103 podman[153439]: 2024-09-10 19:10:21.243421866 +0000 UTC m=+5.459262591 container died 8df6b53b45aed00fe72a0e2a908a48b0b129e9cf93086b19132228096a417bc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-10T19:10:21.772 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:21 smithi103 podman[153439]: 2024-09-10 19:10:21.61401162 +0000 UTC m=+5.829852333 container cleanup 8df6b53b45aed00fe72a0e2a908a48b0b129e9cf93086b19132228096a417bc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T19:10:21.772 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:21 smithi103 bash[153439]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7 2024-09-10T19:10:22.179 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:21 smithi103 podman[153465]: 2024-09-10 19:10:21.864986793 +0000 UTC m=+0.616248523 container remove 8df6b53b45aed00fe72a0e2a908a48b0b129e9cf93086b19132228096a417bc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T19:10:22.494 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:22 smithi103 podman[153556]: 2024-09-10 19:10:22.081815504 +0000 UTC m=+0.015901547 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:10:22.495 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:22 smithi103 podman[153556]: 2024-09-10 19:10:22.182638474 +0000 UTC m=+0.116724560 container create e3508b5e4279d493308baf98fe1220323ebeee2edda8593bdaade8fe381e36f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T19:10:22.495 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:22 smithi103 podman[153556]: 2024-09-10 19:10:22.418604455 +0000 UTC m=+0.352690499 container init e3508b5e4279d493308baf98fe1220323ebeee2edda8593bdaade8fe381e36f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-10T19:10:22.495 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:22 smithi103 podman[153556]: 2024-09-10 19:10:22.421868969 +0000 UTC m=+0.355955010 container start e3508b5e4279d493308baf98fe1220323ebeee2edda8593bdaade8fe381e36f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-10T19:10:22.495 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:22 smithi103 podman[153556]: 2024-09-10 19:10:22.493589298 +0000 UTC m=+0.427675364 container attach e3508b5e4279d493308baf98fe1220323ebeee2edda8593bdaade8fe381e36f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS) 2024-09-10T19:10:22.772 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:22 smithi103 podman[153556]: 2024-09-10 19:10:22.598931341 +0000 UTC m=+0.533017392 container died e3508b5e4279d493308baf98fe1220323ebeee2edda8593bdaade8fe381e36f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-10T19:10:23.173 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 19:10:23 smithi103 podman[153579]: 2024-09-10 19:10:23.155302594 +0000 UTC m=+0.545838636 container remove e3508b5e4279d493308baf98fe1220323ebeee2edda8593bdaade8fe381e36f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-10T19:10:23.179 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@osd.7.service' 2024-09-10T19:10:23.222 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:10:23.222 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-10T19:10:23.222 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-09-10T19:10:23.223 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@rgw.foo.a 2024-09-10T19:10:23.533 INFO:journalctl@ceph.rgw.foo.a.smithi018.stdout:Sep 10 19:10:23 smithi018 systemd[1]: Stopping Ceph rgw.foo.a for ba327f6c-6fa3-11ef-bcea-c7b262605968... 2024-09-10T19:10:23.944 INFO:journalctl@ceph.rgw.foo.a.smithi018.stdout:Sep 10 19:10:23 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-rgw-foo-a[66157]: 2024-09-10T19:10:23.530+0000 7f8a79b04640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:10:23.944 INFO:journalctl@ceph.rgw.foo.a.smithi018.stdout:Sep 10 19:10:23 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-rgw-foo-a[66157]: 2024-09-10T19:10:23.530+0000 7f8a7e00c7c0 -1 shutting down 2024-09-10T19:10:31.944 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 19:10:31 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a[75395]: ts=2024-09-10T19:10:31.629Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi018.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.18:8443: connect: connection refused" 2024-09-10T19:10:31.945 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 19:10:31 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a[75395]: ts=2024-09-10T19:10:31.629Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi103.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.103:8443: connect: connection refused" 2024-09-10T19:10:33.883 INFO:journalctl@ceph.rgw.foo.a.smithi018.stdout:Sep 10 19:10:33 smithi018 bash[211788]: time="2024-09-10T19:10:33Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-09-10T19:10:33.883 INFO:journalctl@ceph.rgw.foo.a.smithi018.stdout:Sep 10 19:10:33 smithi018 podman[211788]: 2024-09-10 19:10:33.549403408 +0000 UTC m=+10.124753029 container died 6c15e50f4ae2df669c3ea3ae8422e469ad872cf663c36a3d79c7dff2c949bdf3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-rgw-foo-a, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True) 2024-09-10T19:10:34.194 INFO:journalctl@ceph.rgw.foo.a.smithi018.stdout:Sep 10 19:10:34 smithi018 podman[211788]: 2024-09-10 19:10:34.06318565 +0000 UTC m=+10.638535315 container cleanup 6c15e50f4ae2df669c3ea3ae8422e469ad872cf663c36a3d79c7dff2c949bdf3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-rgw-foo-a, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True) 2024-09-10T19:10:34.194 INFO:journalctl@ceph.rgw.foo.a.smithi018.stdout:Sep 10 19:10:34 smithi018 bash[211788]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-rgw-foo-a 2024-09-10T19:10:34.450 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@rgw.foo.a.service' 2024-09-10T19:10:34.486 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:10:34.487 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-09-10T19:10:34.487 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-09-10T19:10:34.487 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@prometheus.a 2024-09-10T19:10:34.767 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 19:10:34 smithi103 systemd[1]: Stopping Ceph prometheus.a for ba327f6c-6fa3-11ef-bcea-c7b262605968... 2024-09-10T19:10:35.023 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 19:10:34 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[64913]: ts=2024-09-10T19:10:34.765Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-10T19:10:35.023 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 19:10:34 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[64913]: ts=2024-09-10T19:10:34.766Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-10T19:10:35.023 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 19:10:34 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[64913]: ts=2024-09-10T19:10:34.766Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-10T19:10:35.023 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 19:10:34 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[64913]: ts=2024-09-10T19:10:34.766Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-10T19:10:35.023 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 19:10:34 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[64913]: ts=2024-09-10T19:10:34.766Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-10T19:10:35.023 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 19:10:34 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[64913]: ts=2024-09-10T19:10:34.766Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-10T19:10:35.023 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 19:10:34 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[64913]: ts=2024-09-10T19:10:34.766Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-10T19:10:35.023 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 19:10:34 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[64913]: ts=2024-09-10T19:10:34.766Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-10T19:10:35.023 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 19:10:34 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[64913]: ts=2024-09-10T19:10:34.770Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-10T19:10:35.024 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 19:10:34 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[64913]: ts=2024-09-10T19:10:34.770Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-10T19:10:35.024 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 19:10:34 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[64913]: ts=2024-09-10T19:10:34.770Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-10T19:10:35.024 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 19:10:34 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a[64913]: ts=2024-09-10T19:10:34.770Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-10T19:10:35.024 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 19:10:34 smithi103 podman[153661]: 2024-09-10 19:10:34.776377309 +0000 UTC m=+0.102503624 container died 1b039b6cb12d53ab37cd6276fa8b4f30b4f7d39ce7fda8ee25daccf4dcfc096b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T19:10:35.393 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 19:10:35 smithi103 podman[153661]: 2024-09-10 19:10:35.132611306 +0000 UTC m=+0.458737613 container cleanup 1b039b6cb12d53ab37cd6276fa8b4f30b4f7d39ce7fda8ee25daccf4dcfc096b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T19:10:35.393 INFO:journalctl@ceph.prometheus.a.smithi103.stdout:Sep 10 19:10:35 smithi103 bash[153661]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-prometheus-a 2024-09-10T19:10:35.549 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@prometheus.a.service' 2024-09-10T19:10:35.583 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:10:35.583 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-09-10T19:10:35.583 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T19:10:37.611 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 19:10:37 smithi018 systemd[1]: Stopping Ceph alertmanager.a for ba327f6c-6fa3-11ef-bcea-c7b262605968... 2024-09-10T19:10:37.611 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 19:10:37 smithi018 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a[75395]: ts=2024-09-10T19:10:37.569Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-10T19:10:37.888 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 19:10:37 smithi018 podman[212234]: 2024-09-10 19:10:37.610152176 +0000 UTC m=+0.131044585 container died bfb4ed37f3aa8bae5954b592c776c8b562c237c1b6d4a76128308ca545da84ba (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T19:10:38.194 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 19:10:38 smithi018 podman[212234]: 2024-09-10 19:10:38.069009233 +0000 UTC m=+0.589901650 container cleanup bfb4ed37f3aa8bae5954b592c776c8b562c237c1b6d4a76128308ca545da84ba (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T19:10:38.194 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 19:10:38 smithi018 bash[212234]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a 2024-09-10T19:10:38.694 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 19:10:38 smithi018 podman[212246]: 2024-09-10 19:10:38.370648032 +0000 UTC m=+0.751837366 container remove bfb4ed37f3aa8bae5954b592c776c8b562c237c1b6d4a76128308ca545da84ba (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T19:10:38.694 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 19:10:38 smithi018 podman[212246]: 2024-09-10 19:10:38.447109034 +0000 UTC m=+0.828298367 volume remove 7e3928d98152420fa3c16aa48f414214724372270d58229199a72f9b36de8b75 2024-09-10T19:10:38.694 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 19:10:38 smithi018 systemd[1]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-10T19:10:38.694 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 19:10:38 smithi018 systemd[1]: Stopped Ceph alertmanager.a for ba327f6c-6fa3-11ef-bcea-c7b262605968. 2024-09-10T19:10:38.694 INFO:journalctl@ceph.alertmanager.a.smithi018.stdout:Sep 10 19:10:38 smithi018 systemd[1]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@alertmanager.a.service: Consumed 1.861s CPU time. 2024-09-10T19:10:39.058 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 19:10:38 smithi018 systemd[1]: Stopping Ceph node-exporter.a for ba327f6c-6fa3-11ef-bcea-c7b262605968... 2024-09-10T19:10:39.310 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 19:10:39 smithi018 podman[212355]: 2024-09-10 19:10:39.05752891 +0000 UTC m=+0.136505672 container died 9ca2531486a95de2ebb44bdbbbcd0d1f8ffcde49b99d3c88aa11791d66e5cdef (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T19:10:39.576 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 19:10:39 smithi018 podman[212355]: 2024-09-10 19:10:39.399124505 +0000 UTC m=+0.478101277 container cleanup 9ca2531486a95de2ebb44bdbbbcd0d1f8ffcde49b99d3c88aa11791d66e5cdef (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T19:10:39.577 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 19:10:39 smithi018 bash[212355]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a 2024-09-10T19:10:39.833 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 19:10:39 smithi018 podman[212368]: 2024-09-10 19:10:39.575268677 +0000 UTC m=+0.516028300 container remove 9ca2531486a95de2ebb44bdbbbcd0d1f8ffcde49b99d3c88aa11791d66e5cdef (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T19:10:39.834 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 19:10:39 smithi018 systemd[1]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-09-10T19:10:40.194 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 19:10:39 smithi018 systemd[1]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-09-10T19:10:40.194 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 19:10:39 smithi018 systemd[1]: Stopped Ceph node-exporter.a for ba327f6c-6fa3-11ef-bcea-c7b262605968. 2024-09-10T19:10:40.194 INFO:journalctl@ceph.node-exporter.a.smithi018.stdout:Sep 10 19:10:39 smithi018 systemd[1]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@node-exporter.a.service: Consumed 4.979s CPU time. 2024-09-10T19:10:40.584 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T19:10:42.273 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:10:41 smithi103 systemd[1]: Stopping Ceph iscsi.iscsi.a for ba327f6c-6fa3-11ef-bcea-c7b262605968... 2024-09-10T19:10:42.273 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:10:42 smithi103 ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[60173]: debug Shutdown received 2024-09-10T19:10:52.489 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:10:52 smithi103 bash[154036]: time="2024-09-10T19:10:52Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-09-10T19:10:52.489 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:10:52 smithi103 podman[154036]: 2024-09-10 19:10:52.207207628 +0000 UTC m=+10.123641375 container died 8f9cfa438ded371803362d153d614ecc06f7debbe50a6ea4594fb04f7d50d30e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T19:10:52.772 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:10:52 smithi103 podman[154036]: 2024-09-10 19:10:52.694889693 +0000 UTC m=+10.611323470 container cleanup 8f9cfa438ded371803362d153d614ecc06f7debbe50a6ea4594fb04f7d50d30e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-10T19:10:52.772 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:10:52 smithi103 bash[154036]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a 2024-09-10T19:10:53.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:10:53 smithi103 podman[154049]: 2024-09-10 19:10:53.030463905 +0000 UTC m=+0.823021156 container remove 8f9cfa438ded371803362d153d614ecc06f7debbe50a6ea4594fb04f7d50d30e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-10T19:10:53.522 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:10:53 smithi103 systemd[1]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-09-10T19:10:53.525 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:10:53 smithi103 podman[154093]: 2024-09-10 19:10:53.260414898 +0000 UTC m=+0.116149123 container died f16fb2418ffaf6745141ce4dca650da41eaad8bba3a1cf0477ec083a7ebc6146 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-10T19:10:53.772 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:10:53 smithi103 podman[154093]: 2024-09-10 19:10:53.616329457 +0000 UTC m=+0.472063682 container cleanup f16fb2418ffaf6745141ce4dca650da41eaad8bba3a1cf0477ec083a7ebc6146 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T19:10:53.772 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:10:53 smithi103 bash[154093]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu 2024-09-10T19:10:53.773 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:10:53 smithi103 systemd[1]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-09-10T19:10:53.773 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:10:53 smithi103 systemd[1]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Unit process 60000 (conmon) remains running after unit stopped. 2024-09-10T19:10:53.773 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:10:53 smithi103 systemd[1]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Unit process 154104 (podman) remains running after unit stopped. 2024-09-10T19:10:53.773 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:10:53 smithi103 systemd[1]: Stopped Ceph iscsi.iscsi.a for ba327f6c-6fa3-11ef-bcea-c7b262605968. 2024-09-10T19:10:53.773 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:10:53 smithi103 systemd[1]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Consumed 3.102s CPU time. 2024-09-10T19:10:54.266 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 19:10:54 smithi103 systemd[1]: Stopping Ceph grafana.a for ba327f6c-6fa3-11ef-bcea-c7b262605968... 2024-09-10T19:10:54.267 INFO:journalctl@ceph.iscsi.iscsi.a.smithi103.stdout:Sep 10 19:10:53 smithi103 podman[154104]: 2024-09-10 19:10:53.959316559 +0000 UTC m=+0.702305521 container remove f16fb2418ffaf6745141ce4dca650da41eaad8bba3a1cf0477ec083a7ebc6146 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T19:10:54.522 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 19:10:54 smithi103 podman[154225]: 2024-09-10 19:10:54.264972813 +0000 UTC m=+0.102568065 container died a78c34d334cf9f4f32500e0d0912ccdf5d88a382d6af1a7b99057f30151e56d1 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.component=ubi8-container, build-date=2022-04-29T11:48:11.055897, name=ubi8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, description=Ceph Grafana Container, io.openshift.expose-services=, vcs-type=git, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vendor=Red Hat, Inc., version=8.5, architecture=x86_64, distribution-scope=public, io.buildah.version=1.32.0, io.openshift.tags=base rhel8, maintainer=Paul Cuzner , release=239.1651231664) 2024-09-10T19:10:54.906 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 19:10:54 smithi103 podman[154225]: 2024-09-10 19:10:54.612185363 +0000 UTC m=+0.449780605 container cleanup a78c34d334cf9f4f32500e0d0912ccdf5d88a382d6af1a7b99057f30151e56d1 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a, name=ubi8, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vcs-type=git, vendor=Red Hat, Inc., io.buildah.version=1.32.0, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.component=ubi8-container, architecture=x86_64, release=239.1651231664, summary=Grafana Container configured for Ceph mgr/dashboard integration, distribution-scope=public, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, description=Ceph Grafana Container, io.openshift.expose-services=, version=8.5, io.openshift.tags=base rhel8, maintainer=Paul Cuzner , build-date=2022-04-29T11:48:11.055897, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664) 2024-09-10T19:10:54.907 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 19:10:54 smithi103 bash[154225]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a 2024-09-10T19:10:55.226 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 19:10:54 smithi103 podman[154238]: 2024-09-10 19:10:54.905476112 +0000 UTC m=+0.642393484 container remove a78c34d334cf9f4f32500e0d0912ccdf5d88a382d6af1a7b99057f30151e56d1 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-grafana-a, io.openshift.tags=base rhel8, vendor=Red Hat, Inc., build-date=2022-04-29T11:48:11.055897, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, description=Ceph Grafana Container, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.component=ubi8-container, maintainer=Paul Cuzner , version=8.5, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-type=git, io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.expose-services=, release=239.1651231664, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, distribution-scope=public, architecture=x86_64, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.buildah.version=1.32.0, name=ubi8, summary=Grafana Container configured for Ceph mgr/dashboard integration) 2024-09-10T19:10:55.227 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 19:10:54 smithi103 systemd[1]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-09-10T19:10:55.227 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 19:10:55 smithi103 systemd[1]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@grafana.a.service: Failed with result 'exit-code'. 2024-09-10T19:10:55.227 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 19:10:55 smithi103 systemd[1]: Stopped Ceph grafana.a for ba327f6c-6fa3-11ef-bcea-c7b262605968. 2024-09-10T19:10:55.227 INFO:journalctl@ceph.grafana.a.smithi103.stdout:Sep 10 19:10:55 smithi103 systemd[1]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@grafana.a.service: Consumed 13.246s CPU time. 2024-09-10T19:10:55.503 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 19:10:55 smithi103 systemd[1]: Stopping Ceph node-exporter.b for ba327f6c-6fa3-11ef-bcea-c7b262605968... 2024-09-10T19:10:55.772 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 19:10:55 smithi103 podman[154353]: 2024-09-10 19:10:55.502251406 +0000 UTC m=+0.115329003 container died 5606b97e04a08016a1f4cebb321b8923acfb5638165a7e4402ba137f47d54c17 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T19:10:56.272 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 19:10:55 smithi103 podman[154353]: 2024-09-10 19:10:55.950308854 +0000 UTC m=+0.563386450 container cleanup 5606b97e04a08016a1f4cebb321b8923acfb5638165a7e4402ba137f47d54c17 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T19:10:56.272 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 19:10:55 smithi103 bash[154353]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b 2024-09-10T19:10:56.693 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 19:10:56 smithi103 podman[154365]: 2024-09-10 19:10:56.318873034 +0000 UTC m=+0.820506311 container remove 5606b97e04a08016a1f4cebb321b8923acfb5638165a7e4402ba137f47d54c17 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T19:10:56.694 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 19:10:56 smithi103 systemd[1]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-09-10T19:10:56.694 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 19:10:56 smithi103 systemd[1]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-09-10T19:10:56.694 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 19:10:56 smithi103 systemd[1]: Stopped Ceph node-exporter.b for ba327f6c-6fa3-11ef-bcea-c7b262605968. 2024-09-10T19:10:56.694 INFO:journalctl@ceph.node-exporter.b.smithi103.stdout:Sep 10 19:10:56 smithi103 systemd[1]: ceph-ba327f6c-6fa3-11ef-bcea-c7b262605968@node-exporter.b.service: Consumed 4.390s CPU time. 2024-09-10T19:10:57.187 DEBUG:teuthology.orchestra.run.smithi018:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T19:10:57.223 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T19:10:57.251 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-10T19:10:57.252 DEBUG:teuthology.misc:Transferring archived files from smithi018:/var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900244/remote/smithi018/crash 2024-09-10T19:10:57.253 DEBUG:teuthology.orchestra.run.smithi018:> sudo tar c -f - -C /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/crash -- . 2024-09-10T19:10:57.305 INFO:teuthology.orchestra.run.smithi018.stderr:tar: /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T19:10:57.306 INFO:teuthology.orchestra.run.smithi018.stderr:tar: Error is not recoverable: exiting now 2024-09-10T19:10:57.307 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900244/remote/smithi103/crash 2024-09-10T19:10:57.308 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/crash -- . 2024-09-10T19:10:57.348 INFO:teuthology.orchestra.run.smithi103.stderr:tar: /var/lib/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T19:10:57.349 INFO:teuthology.orchestra.run.smithi103.stderr:tar: Error is not recoverable: exiting now 2024-09-10T19:10:57.350 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-10T19:10:57.350 DEBUG:teuthology.orchestra.run.smithi018:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | head -n 1 2024-09-10T19:10:57.379 INFO:teuthology.orchestra.run.smithi018.stderr:grep: /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph.log: No such file or directory 2024-09-10T19:10:57.381 INFO:tasks.cephadm:Compressing logs... 2024-09-10T19:10:57.381 DEBUG:teuthology.orchestra.run.smithi018:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T19:10:57.423 DEBUG:teuthology.orchestra.run.smithi103:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T19:10:57.458 INFO:teuthology.orchestra.run.smithi018.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-10T19:10:57.458 INFO:teuthology.orchestra.run.smithi018.stderr: 2024-09-10T19:10:57.458 INFO:teuthology.orchestra.run.smithi018.stderr:gzip -5 --verbose -- /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-10T19:10:57.458 INFO:teuthology.orchestra.run.smithi018.stderr:gzip -5 --verbose -- /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/objectstore_tool.2.log 2024-09-10T19:10:57.458 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-osd.0.log: /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T19:10:57.458 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/objectstore_tool.2.log: gzip -5 --verbose -- /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-10T19:10:57.458 INFO:teuthology.orchestra.run.smithi018.stderr:gzip -5 --verbose -- /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-10T19:10:57.461 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-10T19:10:57.461 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-mon.c.log: /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-10T19:10:57.461 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log 2024-09-10T19:10:57.461 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-10T19:10:57.461 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T19:10:57.462 INFO:teuthology.orchestra.run.smithi018.stderr: 58.7% -- replaced with /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log.gz 2024-09-10T19:10:57.463 INFO:teuthology.orchestra.run.smithi103.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/tcmu-runner.log 2024-09-10T19:10:57.463 INFO:teuthology.orchestra.run.smithi103.stderr: 2024-09-10T19:10:57.463 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/objectstore_tool.2.log 2024-09-10T19:10:57.463 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/tcmu-runner.log: gzip -5 --verbose -- /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-10T19:10:57.463 INFO:teuthology.orchestra.run.smithi103.stderr: 62.9% -- replaced with /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/tcmu-runner.log.gz 2024-09-10T19:10:57.463 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/objectstore_tool.2.log: gzip -5 --verbose -- /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T19:10:57.463 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-10T19:10:57.463 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-10T19:10:57.463 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-10T19:10:57.463 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-10T19:10:57.464 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-10T19:10:57.464 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T19:10:57.546 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 91.0% -- replaced with /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T19:10:57.579 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-mon.b.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-10T19:10:57.592 INFO:teuthology.orchestra.run.smithi103.stderr: 94.7% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T19:10:57.602 INFO:teuthology.orchestra.run.smithi018.stderr: 95.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T19:10:57.612 INFO:teuthology.orchestra.run.smithi103.stderr: 91.1% -- replaced with /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T19:10:57.631 INFO:teuthology.orchestra.run.smithi018.stderr: 94.8% -- replaced with /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/objectstore_tool.2.log.gz 2024-09-10T19:10:57.960 INFO:teuthology.orchestra.run.smithi103.stderr: 94.4% -- replaced with /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/objectstore_tool.2.log.gz 2024-09-10T19:10:58.701 INFO:teuthology.orchestra.run.smithi103.stderr: 91.2% -- replaced with /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-10T19:11:00.555 INFO:teuthology.orchestra.run.smithi018.stderr: 92.8% -- replaced with /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-10T19:11:02.074 INFO:teuthology.orchestra.run.smithi018.stderr: 89.4% -- replaced with /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-10T19:11:02.937 INFO:teuthology.orchestra.run.smithi018.stderr: 91.1% -- replaced with /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-10T19:11:04.490 INFO:teuthology.orchestra.run.smithi018.stderr: 90.5% -- replaced with /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-10T19:13:29.841 INFO:teuthology.orchestra.run.smithi018.stderr: 92.9% -- replaced with /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-10T19:17:22.162 INFO:teuthology.orchestra.run.smithi103.stderr: 93.8% -- replaced with /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-10T19:18:41.683 INFO:teuthology.orchestra.run.smithi018.stderr: 92.7% -- replaced with /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-10T19:19:16.983 INFO:teuthology.orchestra.run.smithi018.stderr: 92.5% -- replaced with /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-10T19:19:16.985 INFO:teuthology.orchestra.run.smithi018.stderr: 2024-09-10T19:19:16.985 INFO:teuthology.orchestra.run.smithi018.stderr:real 8m19.548s 2024-09-10T19:19:16.985 INFO:teuthology.orchestra.run.smithi018.stderr:user 6m33.655s 2024-09-10T19:19:16.985 INFO:teuthology.orchestra.run.smithi018.stderr:sys 0m15.765s 2024-09-10T19:19:22.448 INFO:teuthology.orchestra.run.smithi103.stderr: 93.3% -- replaced with /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-10T19:20:19.791 INFO:teuthology.orchestra.run.smithi103.stderr: 93.2% -- replaced with /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-10T19:20:27.964 INFO:teuthology.orchestra.run.smithi103.stderr: 93.2% -- replaced with /var/log/ceph/ba327f6c-6fa3-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-10T19:20:27.966 INFO:teuthology.orchestra.run.smithi103.stderr: 2024-09-10T19:20:27.966 INFO:teuthology.orchestra.run.smithi103.stderr:real 9m30.529s 2024-09-10T19:20:27.967 INFO:teuthology.orchestra.run.smithi103.stderr:user 5m7.363s 2024-09-10T19:20:27.967 INFO:teuthology.orchestra.run.smithi103.stderr:sys 0m15.139s 2024-09-10T19:20:27.967 INFO:tasks.cephadm:Archiving logs... 2024-09-10T19:20:27.968 DEBUG:teuthology.misc:Transferring archived files from smithi018:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900244/remote/smithi018/log 2024-09-10T19:20:27.969 DEBUG:teuthology.orchestra.run.smithi018:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T19:21:26.857 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900244/remote/smithi103/log 2024-09-10T19:21:26.858 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T19:22:14.317 INFO:tasks.cephadm:Removing cluster... 2024-09-10T19:22:14.318 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 --force 2024-09-10T19:22:15.299 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ba327f6c-6fa3-11ef-bcea-c7b262605968 --force 2024-09-10T19:22:16.155 INFO:tasks.cephadm:Removing cephadm ... 2024-09-10T19:22:16.156 DEBUG:teuthology.orchestra.run.smithi018:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T19:22:16.176 DEBUG:teuthology.orchestra.run.smithi103:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T19:22:16.191 INFO:tasks.cephadm:Teardown complete 2024-09-10T19:22:16.192 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T19:22:16.201 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T19:22:16.201 DEBUG:teuthology.orchestra.run.smithi018:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T19:22:16.219 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T19:22:16.322 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T19:22:16.331 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T19:22:16.331 DEBUG:teuthology.orchestra.run.smithi018:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T19:22:16.334 DEBUG:teuthology.orchestra.run.smithi103:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T19:22:16.348 INFO:teuthology.orchestra.run.smithi103.stderr:bash: line 1: ntpq: command not found 2024-09-10T19:22:16.349 INFO:teuthology.orchestra.run.smithi018.stderr:bash: line 1: ntpq: command not found 2024-09-10T19:22:16.368 INFO:teuthology.orchestra.run.smithi103.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T19:22:16.368 INFO:teuthology.orchestra.run.smithi103.stdout:=============================================================================== 2024-09-10T19:22:16.368 INFO:teuthology.orchestra.run.smithi103.stdout:^* hv01.front.sepia.ceph.com 3 6 377 40 -305us[ -302us] +/- 37ms 2024-09-10T19:22:16.368 INFO:teuthology.orchestra.run.smithi103.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 37 +1435us[+1435us] +/- 56ms 2024-09-10T19:22:16.368 INFO:teuthology.orchestra.run.smithi103.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 33 -1322us[-1322us] +/- 82ms 2024-09-10T19:22:16.368 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T19:22:16.369 INFO:teuthology.orchestra.run.smithi018.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T19:22:16.369 INFO:teuthology.orchestra.run.smithi018.stdout:=============================================================================== 2024-09-10T19:22:16.369 INFO:teuthology.orchestra.run.smithi018.stdout:^* hv01.front.sepia.ceph.com 3 6 377 34 -292us[ -291us] +/- 37ms 2024-09-10T19:22:16.370 INFO:teuthology.orchestra.run.smithi018.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 34 +1469us[+1469us] +/- 56ms 2024-09-10T19:22:16.370 INFO:teuthology.orchestra.run.smithi018.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 33 -1284us[-1284us] +/- 82ms 2024-09-10T19:22:16.370 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T19:22:16.370 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T19:22:16.525 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T19:22:16.526 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T19:22:16.685 DEBUG:teuthology.orchestra.run.smithi018:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T19:22:16.798 DEBUG:teuthology.orchestra.run.smithi103:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T19:22:16.939 DEBUG:teuthology.orchestra.run.smithi018:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T19:22:16.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:22:16.974 DEBUG:teuthology.orchestra.run.smithi103:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T19:22:17.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:22:17.000 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T19:22:17.065 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T19:22:17.106 INFO:teuthology.task.internal:Duration was 3047.287795 seconds 2024-09-10T19:22:17.106 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T19:22:17.369 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T19:22:17.369 DEBUG:teuthology.orchestra.run.smithi018:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T19:22:17.372 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T19:22:17.443 INFO:teuthology.orchestra.run.smithi103.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T19:22:17.452 INFO:teuthology.orchestra.run.smithi018.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T19:22:17.838 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T19:22:17.838 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi018.front.sepia.ceph.com 2024-09-10T19:22:17.838 DEBUG:teuthology.orchestra.run.smithi018:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T19:22:17.875 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi103.front.sepia.ceph.com 2024-09-10T19:22:17.876 DEBUG:teuthology.orchestra.run.smithi103:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T19:22:17.905 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T19:22:17.905 DEBUG:teuthology.orchestra.run.smithi018:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T19:22:17.918 DEBUG:teuthology.orchestra.run.smithi103:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T19:22:17.977 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T19:22:17.977 DEBUG:teuthology.orchestra.run.smithi018:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T19:22:17.990 DEBUG:teuthology.orchestra.run.smithi103:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T19:22:19.401 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T19:22:19.412 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T19:22:19.412 DEBUG:teuthology.orchestra.run.smithi018:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T19:22:19.450 DEBUG:teuthology.orchestra.run.smithi103:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T19:22:19.480 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T19:22:19.490 DEBUG:teuthology.orchestra.run.smithi018:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T19:22:19.493 DEBUG:teuthology.orchestra.run.smithi103:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T19:22:19.517 INFO:teuthology.orchestra.run.smithi018.stdout:kernel.core_pattern = core 2024-09-10T19:22:19.554 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern = core 2024-09-10T19:22:19.574 DEBUG:teuthology.orchestra.run.smithi018:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T19:22:19.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:22:19.600 DEBUG:teuthology.orchestra.run.smithi103:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T19:22:19.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:22:19.630 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T19:22:19.640 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T19:22:19.641 DEBUG:teuthology.misc:Transferring archived files from smithi018:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900244/remote/smithi018 2024-09-10T19:22:19.641 DEBUG:teuthology.orchestra.run.smithi018:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T19:22:19.714 DEBUG:teuthology.misc:Transferring archived files from smithi103:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900244/remote/smithi103 2024-09-10T19:22:19.714 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T19:22:19.753 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T19:22:19.753 DEBUG:teuthology.orchestra.run.smithi018:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T19:22:19.756 DEBUG:teuthology.orchestra.run.smithi103:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T19:22:19.803 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T19:22:19.812 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T19:22:19.813 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T19:22:19.917 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T19:22:19.917 DEBUG:teuthology.orchestra.run.smithi018:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T19:22:19.920 DEBUG:teuthology.orchestra.run.smithi103:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T19:22:19.933 INFO:teuthology.orchestra.run.smithi018.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 19:22 /home/ubuntu/cephtest 2024-09-10T19:22:19.936 INFO:teuthology.orchestra.run.smithi103.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 19:22 /home/ubuntu/cephtest 2024-09-10T19:22:19.937 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T19:22:19.965 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T19:22:20.100 INFO:teuthology.run:Summary data: description: orch:cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} duration: 3047.287794828415 flavor: default owner: scheduled_gabrioux@teuthology success: true 2024-09-10T19:22:20.101 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T19:22:20.184 INFO:teuthology.run:pass